2024-09-21T05:36:00.162 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T05:36:00.191 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T05:36:00.275 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913507 branch: wip-pdonnell-testing-20240920.212106-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/iogen}} email: pdonnell@redhat.com first_in_suite: false job_id: '7913507' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240920.212106-debug ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mgr/snap_schedule/allow_m_granularity: true mgr/snap_schedule/dump_on_update: true mgr/snap_schedule/log_level: debug mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mds: mds_bal_replicate_threshold: 1 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request mgr-modules: - snap_schedule sha1: 1843221895adccfb3b692060251dd6a0183bc410 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: 1843221895adccfb3b692060251dd6a0183bc410 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240920.212106-debug sha1: 1843221895adccfb3b692060251dd6a0183bc410 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7864 sha1: 1843221895adccfb3b692060251dd6a0183bc410 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240920.212106-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1843221895adccfb3b692060251dd6a0183bc410 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi117.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOtuRHdCd9xAwY10GgYQcfriXlST/PN43puDt0BFDb0IrMBj6Bufkg6qyD6mtOjIKercQiX6JKqxdYrb734/Yps= smithi138.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAUulry9PBdtc30VphLaguFA/noykFYFkS3ZX6uEJrEs4JgACEwfbkG4tRWCPn6QBUnwN0Qw6ooShndWfG628uE= smithi192.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLZeMpEHr60AH6sIu4tqstEaK3ccuHbY1RJRts1G4TWinOW/oR8cvEbHgAhYSwEr80N9CZWSVr9gFoZxKcKts2c= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/iogen.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-20_23:48:22 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T05:36:00.275 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa; will attempt to use it 2024-09-21T05:36:00.276 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks 2024-09-21T05:36:00.276 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T05:36:00.277 INFO:teuthology.task.internal:Checking packages... 2024-09-21T05:36:00.300 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1843221895adccfb3b692060251dd6a0183bc410' 2024-09-21T05:36:00.301 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T05:36:00.301 INFO:teuthology.packaging:ref: None 2024-09-21T05:36:00.301 INFO:teuthology.packaging:tag: None 2024-09-21T05:36:00.301 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240920.212106-debug 2024-09-21T05:36:00.301 INFO:teuthology.packaging:sha1: 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T05:36:00.301 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240920.212106-debug 2024-09-21T05:36:00.442 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5120.g18432218 2024-09-21T05:36:00.444 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T05:36:00.451 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T05:36:00.451 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T05:36:00.464 INFO:teuthology.task.internal:Saving configuration 2024-09-21T05:36:00.497 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T05:36:00.504 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T05:36:00.529 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi117.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913507', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 05:31:00.464606', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOtuRHdCd9xAwY10GgYQcfriXlST/PN43puDt0BFDb0IrMBj6Bufkg6qyD6mtOjIKercQiX6JKqxdYrb734/Yps='} 2024-09-21T05:36:00.553 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913507', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 05:31:00.465895', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAUulry9PBdtc30VphLaguFA/noykFYFkS3ZX6uEJrEs4JgACEwfbkG4tRWCPn6QBUnwN0Qw6ooShndWfG628uE='} 2024-09-21T05:36:00.572 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913507', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 05:31:00.463077', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLZeMpEHr60AH6sIu4tqstEaK3ccuHbY1RJRts1G4TWinOW/oR8cvEbHgAhYSwEr80N9CZWSVr9gFoZxKcKts2c='} 2024-09-21T05:36:00.572 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T05:36:00.579 INFO:teuthology.task.internal:roles: ubuntu@smithi117.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-21T05:36:00.579 INFO:teuthology.task.internal:roles: ubuntu@smithi138.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-21T05:36:00.579 INFO:teuthology.task.internal:roles: ubuntu@smithi192.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-21T05:36:00.579 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T05:36:00.666 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7f7d054160>, signals=[15]) 2024-09-21T05:36:00.667 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T05:36:00.683 INFO:teuthology.task.internal:Opening connections... 2024-09-21T05:36:00.683 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi117.front.sepia.ceph.com 2024-09-21T05:36:00.684 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:36:00.759 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi138.front.sepia.ceph.com 2024-09-21T05:36:00.760 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:36:00.841 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi192.front.sepia.ceph.com 2024-09-21T05:36:00.842 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:36:00.915 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T05:36:00.923 DEBUG:teuthology.orchestra.run.smithi117:> uname -m 2024-09-21T05:36:00.941 INFO:teuthology.orchestra.run.smithi117.stdout:x86_64 2024-09-21T05:36:00.941 DEBUG:teuthology.orchestra.run.smithi117:> cat /etc/os-release 2024-09-21T05:36:01.000 INFO:teuthology.orchestra.run.smithi117.stdout:NAME="CentOS Stream" 2024-09-21T05:36:01.000 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION="9" 2024-09-21T05:36:01.000 INFO:teuthology.orchestra.run.smithi117.stdout:ID="centos" 2024-09-21T05:36:01.000 INFO:teuthology.orchestra.run.smithi117.stdout:ID_LIKE="rhel fedora" 2024-09-21T05:36:01.000 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION_ID="9" 2024-09-21T05:36:01.001 INFO:teuthology.orchestra.run.smithi117.stdout:PLATFORM_ID="platform:el9" 2024-09-21T05:36:01.001 INFO:teuthology.orchestra.run.smithi117.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T05:36:01.001 INFO:teuthology.orchestra.run.smithi117.stdout:ANSI_COLOR="0;31" 2024-09-21T05:36:01.001 INFO:teuthology.orchestra.run.smithi117.stdout:LOGO="fedora-logo-icon" 2024-09-21T05:36:01.001 INFO:teuthology.orchestra.run.smithi117.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T05:36:01.001 INFO:teuthology.orchestra.run.smithi117.stdout:HOME_URL="https://centos.org/" 2024-09-21T05:36:01.001 INFO:teuthology.orchestra.run.smithi117.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T05:36:01.001 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T05:36:01.001 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T05:36:01.002 INFO:teuthology.lock.ops:Updating smithi117.front.sepia.ceph.com on lock server 2024-09-21T05:36:01.035 DEBUG:teuthology.orchestra.run.smithi138:> uname -m 2024-09-21T05:36:01.053 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2024-09-21T05:36:01.054 DEBUG:teuthology.orchestra.run.smithi138:> cat /etc/os-release 2024-09-21T05:36:01.112 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="CentOS Stream" 2024-09-21T05:36:01.112 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="9" 2024-09-21T05:36:01.112 INFO:teuthology.orchestra.run.smithi138.stdout:ID="centos" 2024-09-21T05:36:01.112 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="rhel fedora" 2024-09-21T05:36:01.112 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="9" 2024-09-21T05:36:01.112 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el9" 2024-09-21T05:36:01.112 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T05:36:01.112 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2024-09-21T05:36:01.112 INFO:teuthology.orchestra.run.smithi138.stdout:LOGO="fedora-logo-icon" 2024-09-21T05:36:01.113 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T05:36:01.113 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://centos.org/" 2024-09-21T05:36:01.113 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T05:36:01.113 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T05:36:01.113 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T05:36:01.113 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2024-09-21T05:36:01.143 DEBUG:teuthology.orchestra.run.smithi192:> uname -m 2024-09-21T05:36:01.161 INFO:teuthology.orchestra.run.smithi192.stdout:x86_64 2024-09-21T05:36:01.162 DEBUG:teuthology.orchestra.run.smithi192:> cat /etc/os-release 2024-09-21T05:36:01.220 INFO:teuthology.orchestra.run.smithi192.stdout:NAME="CentOS Stream" 2024-09-21T05:36:01.220 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION="9" 2024-09-21T05:36:01.220 INFO:teuthology.orchestra.run.smithi192.stdout:ID="centos" 2024-09-21T05:36:01.220 INFO:teuthology.orchestra.run.smithi192.stdout:ID_LIKE="rhel fedora" 2024-09-21T05:36:01.220 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_ID="9" 2024-09-21T05:36:01.220 INFO:teuthology.orchestra.run.smithi192.stdout:PLATFORM_ID="platform:el9" 2024-09-21T05:36:01.220 INFO:teuthology.orchestra.run.smithi192.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T05:36:01.220 INFO:teuthology.orchestra.run.smithi192.stdout:ANSI_COLOR="0;31" 2024-09-21T05:36:01.220 INFO:teuthology.orchestra.run.smithi192.stdout:LOGO="fedora-logo-icon" 2024-09-21T05:36:01.220 INFO:teuthology.orchestra.run.smithi192.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T05:36:01.220 INFO:teuthology.orchestra.run.smithi192.stdout:HOME_URL="https://centos.org/" 2024-09-21T05:36:01.220 INFO:teuthology.orchestra.run.smithi192.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T05:36:01.220 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T05:36:01.221 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T05:36:01.221 INFO:teuthology.lock.ops:Updating smithi192.front.sepia.ceph.com on lock server 2024-09-21T05:36:01.251 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T05:36:01.262 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T05:36:01.268 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T05:36:01.268 DEBUG:teuthology.orchestra.run.smithi117:> test '!' -e /home/ubuntu/cephtest 2024-09-21T05:36:01.271 DEBUG:teuthology.orchestra.run.smithi138:> test '!' -e /home/ubuntu/cephtest 2024-09-21T05:36:01.273 DEBUG:teuthology.orchestra.run.smithi192:> test '!' -e /home/ubuntu/cephtest 2024-09-21T05:36:01.290 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T05:36:01.297 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T05:36:01.297 DEBUG:teuthology.orchestra.run.smithi117:> test -z $(ls -A /var/lib/ceph) 2024-09-21T05:36:01.328 DEBUG:teuthology.orchestra.run.smithi138:> test -z $(ls -A /var/lib/ceph) 2024-09-21T05:36:01.331 DEBUG:teuthology.orchestra.run.smithi192:> test -z $(ls -A /var/lib/ceph) 2024-09-21T05:36:01.429 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T05:36:01.503 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T05:36:01.519 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-21T05:36:01.519 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-21T05:36:01.519 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-21T05:36:01.519 DEBUG:teuthology.orchestra.run.smithi117:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T05:36:01.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:36:01.536 DEBUG:teuthology.orchestra.run.smithi117:> uname -r 2024-09-21T05:36:01.594 INFO:teuthology.orchestra.run.smithi117.stdout:5.14.0-503.el9.x86_64 2024-09-21T05:36:01.594 INFO:teuthology.task.kernel:Running kernel on smithi117: 5.14.0-503.el9.x86_64 2024-09-21T05:36:01.594 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum install -y kernel 2024-09-21T05:36:05.329 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - BaseOS 4.1 MB/s | 8.3 MB 00:01 2024-09-21T05:36:11.312 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - AppStream 4.1 MB/s | 20 MB 00:04 2024-09-21T05:36:16.904 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.5 MB 00:01 2024-09-21T05:36:18.380 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - Extras packages 79 kB/s | 19 kB 00:00 2024-09-21T05:36:19.250 INFO:teuthology.orchestra.run.smithi117.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-21T05:36:24.839 INFO:teuthology.orchestra.run.smithi117.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-21T05:36:26.754 INFO:teuthology.orchestra.run.smithi117.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T05:36:26.754 INFO:teuthology.orchestra.run.smithi117.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T05:36:26.846 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-09-21T05:36:26.857 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-21T05:36:26.858 INFO:teuthology.orchestra.run.smithi117.stdout: Package Architecture Version Repository Size 2024-09-21T05:36:26.858 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-21T05:36:26.858 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-09-21T05:36:26.858 INFO:teuthology.orchestra.run.smithi117.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T05:36:26.858 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-09-21T05:36:26.858 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T05:36:26.858 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T05:36:26.858 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T05:36:26.858 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:36:26.858 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-09-21T05:36:26.858 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-21T05:36:26.858 INFO:teuthology.orchestra.run.smithi117.stdout:Install 4 Packages 2024-09-21T05:36:26.859 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:36:26.859 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 79 M 2024-09-21T05:36:26.859 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 126 M 2024-09-21T05:36:26.859 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-09-21T05:36:26.990 INFO:teuthology.orchestra.run.smithi117.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 340 kB/s | 36 kB 00:00 2024-09-21T05:36:27.608 INFO:teuthology.orchestra.run.smithi117.stdout:(2/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 48 MB/s | 35 MB 00:00 2024-09-21T05:36:27.866 INFO:teuthology.orchestra.run.smithi117.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 33 MB/s | 29 MB 00:00 2024-09-21T05:36:28.241 INFO:teuthology.orchestra.run.smithi117.stdout:(4/4): kernel-core-5.14.0-508.el9.x86_64.rpm 12 MB/s | 16 MB 00:01 2024-09-21T05:36:28.242 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-09-21T05:36:28.242 INFO:teuthology.orchestra.run.smithi117.stdout:Total 57 MB/s | 79 MB 00:01 2024-09-21T05:36:28.921 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-09-21T05:36:29.279 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-09-21T05:36:29.279 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-09-21T05:36:30.231 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-09-21T05:36:30.232 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-09-21T05:36:32.660 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-09-21T05:36:33.292 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T05:36:33.401 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T05:36:36.147 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T05:36:36.277 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T05:36:41.121 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T05:36:41.219 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T05:36:46.061 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T05:37:24.264 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T05:37:24.304 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T05:37:24.666 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T05:37:24.666 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T05:37:24.666 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T05:37:24.667 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T05:37:25.507 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T05:37:25.507 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:37:25.508 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-09-21T05:37:25.508 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T05:37:25.508 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T05:37:25.508 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T05:37:25.508 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T05:37:25.508 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:37:25.508 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-09-21T05:37:25.854 DEBUG:teuthology.orchestra.run.smithi117:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T05:37:25.943 INFO:teuthology.orchestra.run.smithi117.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T05:37:25.944 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T05:37:25.944 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T05:37:25.944 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T05:37:25.944 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T05:37:25.944 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-21T05:37:25.944 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi117.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T05:37:25.944 DEBUG:teuthology.orchestra.run.smithi117:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T05:37:26.003 INFO:teuthology.orchestra.run.smithi117.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T05:37:26.003 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T05:37:26.003 DEBUG:teuthology.orchestra.run.smithi117:> sudo rpm -qi grub2-tools 2024-09-21T05:37:26.035 INFO:teuthology.orchestra.run.smithi117.stdout:Name : grub2-tools 2024-09-21T05:37:26.035 INFO:teuthology.orchestra.run.smithi117.stdout:Epoch : 1 2024-09-21T05:37:26.035 INFO:teuthology.orchestra.run.smithi117.stdout:Version : 2.06 2024-09-21T05:37:26.035 INFO:teuthology.orchestra.run.smithi117.stdout:Release : 82.el9 2024-09-21T05:37:26.035 INFO:teuthology.orchestra.run.smithi117.stdout:Architecture: x86_64 2024-09-21T05:37:26.035 INFO:teuthology.orchestra.run.smithi117.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T05:37:26.035 INFO:teuthology.orchestra.run.smithi117.stdout:Group : Unspecified 2024-09-21T05:37:26.035 INFO:teuthology.orchestra.run.smithi117.stdout:Size : 8274670 2024-09-21T05:37:26.035 INFO:teuthology.orchestra.run.smithi117.stdout:License : GPLv3+ 2024-09-21T05:37:26.035 INFO:teuthology.orchestra.run.smithi117.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T05:37:26.035 INFO:teuthology.orchestra.run.smithi117.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T05:37:26.035 INFO:teuthology.orchestra.run.smithi117.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T05:37:26.036 INFO:teuthology.orchestra.run.smithi117.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T05:37:26.036 INFO:teuthology.orchestra.run.smithi117.stdout:Packager : builder@centos.org 2024-09-21T05:37:26.036 INFO:teuthology.orchestra.run.smithi117.stdout:Vendor : CentOS 2024-09-21T05:37:26.036 INFO:teuthology.orchestra.run.smithi117.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T05:37:26.036 INFO:teuthology.orchestra.run.smithi117.stdout:Summary : Support tools for GRUB. 2024-09-21T05:37:26.036 INFO:teuthology.orchestra.run.smithi117.stdout:Description : 2024-09-21T05:37:26.036 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:37:26.036 INFO:teuthology.orchestra.run.smithi117.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T05:37:26.036 INFO:teuthology.orchestra.run.smithi117.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T05:37:26.036 INFO:teuthology.orchestra.run.smithi117.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T05:37:26.036 INFO:teuthology.orchestra.run.smithi117.stdout:hardware devices. 2024-09-21T05:37:26.036 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:37:26.036 INFO:teuthology.orchestra.run.smithi117.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T05:37:26.037 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T05:37:26.037 INFO:teuthology.task.kernel:Updating grub on smithi117 to boot 5.14.0-508.el9.x86_64 2024-09-21T05:37:26.037 DEBUG:teuthology.orchestra.run.smithi117:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T05:37:26.724 INFO:teuthology.orchestra.run.smithi117.stderr:Generating grub configuration file ... 2024-09-21T05:37:27.982 INFO:teuthology.orchestra.run.smithi117.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T05:37:28.022 INFO:teuthology.orchestra.run.smithi117.stderr:done 2024-09-21T05:37:28.024 DEBUG:teuthology.orchestra.run.smithi117:> mktemp 2024-09-21T05:37:28.039 INFO:teuthology.orchestra.run.smithi117.stdout:/tmp/tmp.X9CQtYqvGW 2024-09-21T05:37:28.040 DEBUG:teuthology.orchestra.run.smithi117:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.X9CQtYqvGW 2024-09-21T05:37:28.107 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0666 /tmp/tmp.X9CQtYqvGW 2024-09-21T05:37:28.251 DEBUG:teuthology.orchestra.remote:smithi117:/tmp/tmp.X9CQtYqvGW is 6KB 2024-09-21T05:37:28.300 DEBUG:teuthology.orchestra.run.smithi117:> rm -fr /tmp/tmp.X9CQtYqvGW 2024-09-21T05:37:28.315 DEBUG:teuthology.orchestra.run.smithi117:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T05:37:28.381 INFO:teuthology.orchestra.run.smithi117.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T05:37:28.381 INFO:teuthology.orchestra.run.smithi117.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T05:37:28.381 INFO:teuthology.orchestra.run.smithi117.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T05:37:28.381 INFO:teuthology.orchestra.run.smithi117.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T05:37:28.382 DEBUG:teuthology.orchestra.run.smithi117:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T05:37:28.541 DEBUG:teuthology.orchestra.run.smithi117:> sudo shutdown -r now 2024-09-21T05:37:58.543 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-21T05:37:58.543 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-09-21T05:37:58.544 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:38:16.943 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.117 2024-09-21T05:38:25.943 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-09-21T05:38:25.944 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:38:29.007 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.117 2024-09-21T05:38:41.008 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-09-21T05:38:41.008 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:38:59.438 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.117 2024-09-21T05:39:14.439 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-09-21T05:39:14.440 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:39:14.802 DEBUG:teuthology.orchestra.run.smithi117:> true 2024-09-21T05:39:15.271 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-09-21T05:39:15.271 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-508.el9.x86_64"... 2024-09-21T05:39:15.271 DEBUG:teuthology.orchestra.run.smithi117:> uname -r 2024-09-21T05:39:15.285 INFO:teuthology.orchestra.run.smithi117.stdout:5.14.0-508.el9.x86_64 2024-09-21T05:39:15.285 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T05:39:15.286 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T05:39:15.286 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T05:39:16.286 DEBUG:teuthology.parallel:result is None 2024-09-21T05:39:16.287 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T05:39:16.300 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T05:39:16.300 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T05:39:16.303 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T05:39:16.305 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T05:39:16.324 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T05:39:16.330 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T05:39:16.338 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T05:39:16.339 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T05:39:16.362 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T05:39:16.365 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T05:39:16.445 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T05:39:16.452 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T05:39:16.452 DEBUG:teuthology.orchestra.run.smithi138:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T05:39:16.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:39:16.468 DEBUG:teuthology.orchestra.run.smithi192:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T05:39:16.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:39:16.485 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T05:39:16.492 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T05:39:16.511 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T05:39:16.555 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T05:39:16.564 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T05:39:16.573 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T05:39:16.585 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T05:39:16.609 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T05:39:16.762 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T05:39:16.764 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T05:39:16.773 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T05:39:16.773 DEBUG:teuthology.orchestra.run.smithi117:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T05:39:16.779 DEBUG:teuthology.orchestra.run.smithi138:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T05:39:16.781 DEBUG:teuthology.orchestra.run.smithi192:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T05:39:16.859 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T05:39:16.867 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T05:39:16.867 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T05:39:16.871 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T05:39:16.873 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T05:39:16.906 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T05:39:17.021 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T05:39:17.086 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T05:39:17.162 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T05:39:17.224 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-21T05:39:17.224 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T05:39:17.295 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T05:39:17.340 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T05:39:17.394 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T05:39:17.475 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T05:39:17.522 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-21T05:39:17.522 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T05:39:17.590 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T05:39:17.632 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T05:39:17.688 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T05:39:17.773 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T05:39:17.816 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-21T05:39:17.816 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T05:39:17.883 DEBUG:teuthology.orchestra.run.smithi117:> sudo service rsyslog restart 2024-09-21T05:39:17.885 DEBUG:teuthology.orchestra.run.smithi138:> sudo service rsyslog restart 2024-09-21T05:39:17.888 DEBUG:teuthology.orchestra.run.smithi192:> sudo service rsyslog restart 2024-09-21T05:39:17.928 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T05:39:17.961 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T05:39:18.047 INFO:teuthology.orchestra.run.smithi117.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T05:39:18.491 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T05:39:18.499 INFO:teuthology.task.internal:Starting timer... 2024-09-21T05:39:18.499 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T05:39:18.523 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T05:39:18.537 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-21T05:39:18.537 DEBUG:teuthology.orchestra.run.smithi117:> sudo service auditd rotate 2024-09-21T05:39:18.576 INFO:teuthology.orchestra.run.smithi117.stdout:Rotating logs: 2024-09-21T05:39:18.578 DEBUG:teuthology.orchestra.run.smithi138:> sudo service auditd rotate 2024-09-21T05:39:18.646 INFO:teuthology.orchestra.run.smithi138.stdout:Rotating logs: 2024-09-21T05:39:18.648 DEBUG:teuthology.orchestra.run.smithi192:> sudo service auditd rotate 2024-09-21T05:39:18.704 INFO:teuthology.orchestra.run.smithi192.stdout:Rotating logs: 2024-09-21T05:39:18.706 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T05:39:18.706 DEBUG:teuthology.orchestra.run.smithi117:> /usr/sbin/getenforce 2024-09-21T05:39:18.724 INFO:teuthology.orchestra.run.smithi117.stdout:Permissive 2024-09-21T05:39:18.724 DEBUG:teuthology.orchestra.run.smithi138:> /usr/sbin/getenforce 2024-09-21T05:39:18.743 INFO:teuthology.orchestra.run.smithi138.stdout:Permissive 2024-09-21T05:39:18.743 DEBUG:teuthology.orchestra.run.smithi192:> /usr/sbin/getenforce 2024-09-21T05:39:18.765 INFO:teuthology.orchestra.run.smithi192.stdout:Permissive 2024-09-21T05:39:18.765 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi117.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi138.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi192.front.sepia.ceph.com': 'permissive'} 2024-09-21T05:39:18.765 DEBUG:teuthology.orchestra.run.smithi117:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T05:39:18.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:39:18.790 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T05:39:18.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:39:18.820 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T05:39:18.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:39:18.850 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T05:39:18.850 DEBUG:teuthology.orchestra.run.smithi117:> sudo /usr/sbin/setenforce permissive 2024-09-21T05:39:18.889 DEBUG:teuthology.orchestra.run.smithi138:> sudo /usr/sbin/setenforce permissive 2024-09-21T05:39:18.918 DEBUG:teuthology.orchestra.run.smithi192:> sudo /usr/sbin/setenforce permissive 2024-09-21T05:39:18.954 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T05:39:18.963 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-21T05:39:18.965 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-21T05:39:19.069 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T05:39:19.079 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T05:39:19.080 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi117.front.sepia.ceph.com,smithi138.front.sepia.ceph.com,smithi192.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T05:43:22.026 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi117.front.sepia.ceph.com'), Remote(name='ubuntu@smithi138.front.sepia.ceph.com'), Remote(name='ubuntu@smithi192.front.sepia.ceph.com')] 2024-09-21T05:43:22.027 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-09-21T05:43:22.028 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:43:22.109 DEBUG:teuthology.orchestra.run.smithi117:> true 2024-09-21T05:43:22.197 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-09-21T05:43:22.197 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-09-21T05:43:22.198 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:43:22.288 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-09-21T05:43:22.368 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-09-21T05:43:22.368 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-09-21T05:43:22.369 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:43:22.447 DEBUG:teuthology.orchestra.run.smithi192:> true 2024-09-21T05:43:22.528 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-09-21T05:43:22.529 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T05:43:22.538 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T05:43:22.538 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T05:43:22.539 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T05:43:22.541 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T05:43:22.541 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T05:43:22.543 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T05:43:22.543 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T05:43:22.574 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T05:43:22.575 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T05:43:22.590 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T05:43:22.591 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T05:43:22.600 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T05:43:22.615 INFO:teuthology.orchestra.run.smithi117.stderr:sudo: ntpd: command not found 2024-09-21T05:43:22.616 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T05:43:22.617 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: ntpd: command not found 2024-09-21T05:43:22.627 INFO:teuthology.orchestra.run.smithi117.stdout:506 Cannot talk to daemon 2024-09-21T05:43:22.629 INFO:teuthology.orchestra.run.smithi138.stdout:506 Cannot talk to daemon 2024-09-21T05:43:22.641 INFO:teuthology.orchestra.run.smithi192.stderr:sudo: ntpd: command not found 2024-09-21T05:43:22.642 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T05:43:22.644 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T05:43:22.653 INFO:teuthology.orchestra.run.smithi192.stdout:506 Cannot talk to daemon 2024-09-21T05:43:22.657 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T05:43:22.658 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T05:43:22.668 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T05:43:22.683 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T05:43:22.715 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-09-21T05:43:22.717 INFO:teuthology.orchestra.run.smithi117.stderr:bash: line 1: ntpq: command not found 2024-09-21T05:43:22.718 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T05:43:22.718 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-09-21T05:43:22.718 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:43:22.718 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:43:22.719 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:43:22.719 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:43:22.721 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T05:43:22.721 INFO:teuthology.orchestra.run.smithi117.stdout:=============================================================================== 2024-09-21T05:43:22.721 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:43:22.722 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:43:22.722 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:43:22.722 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:43:22.745 INFO:teuthology.orchestra.run.smithi192.stderr:bash: line 1: ntpq: command not found 2024-09-21T05:43:22.748 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T05:43:22.748 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2024-09-21T05:43:22.748 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:43:22.748 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:43:22.748 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:43:22.748 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:43:22.749 INFO:teuthology.run_tasks:Running task install... 2024-09-21T05:43:22.757 DEBUG:teuthology.task.install:project ceph 2024-09-21T05:43:22.757 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}} 2024-09-21T05:43:22.757 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'} 2024-09-21T05:43:22.757 INFO:teuthology.task.install:Using flavor: default 2024-09-21T05:43:22.762 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T05:43:22.762 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-21T05:43:22.763 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T05:43:22.763 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T05:43:22.765 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T05:43:22.765 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T05:43:22.766 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T05:43:22.766 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T05:43:22.920 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T05:43:22.920 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T05:43:22.927 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T05:43:22.927 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T05:43:22.931 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T05:43:22.931 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T05:43:23.029 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T05:43:23.029 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-21T05:43:23.029 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T05:43:23.039 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T05:43:23.039 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-21T05:43:23.039 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T05:43:23.048 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T05:43:23.048 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-21T05:43:23.048 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T05:43:23.054 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T05:43:23.055 DEBUG:teuthology.orchestra.run.smithi192:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T05:43:23.064 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T05:43:23.065 DEBUG:teuthology.orchestra.run.smithi117:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T05:43:23.074 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T05:43:23.075 DEBUG:teuthology.orchestra.run.smithi138:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T05:43:23.120 DEBUG:teuthology.orchestra.run.smithi192:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T05:43:23.130 DEBUG:teuthology.orchestra.run.smithi117:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T05:43:23.140 DEBUG:teuthology.orchestra.run.smithi138:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T05:43:23.196 DEBUG:teuthology.orchestra.run.smithi192:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T05:43:23.205 DEBUG:teuthology.orchestra.run.smithi117:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T05:43:23.214 DEBUG:teuthology.orchestra.run.smithi138:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T05:43:23.260 INFO:teuthology.orchestra.run.smithi192.stdout:check_obsoletes = 1 2024-09-21T05:43:23.262 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum clean all 2024-09-21T05:43:23.270 INFO:teuthology.orchestra.run.smithi117.stdout:check_obsoletes = 1 2024-09-21T05:43:23.272 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum clean all 2024-09-21T05:43:23.278 INFO:teuthology.orchestra.run.smithi138.stdout:check_obsoletes = 1 2024-09-21T05:43:23.280 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum clean all 2024-09-21T05:43:23.489 INFO:teuthology.orchestra.run.smithi192.stdout:45 files removed 2024-09-21T05:43:23.505 INFO:teuthology.orchestra.run.smithi138.stdout:45 files removed 2024-09-21T05:43:23.508 INFO:teuthology.orchestra.run.smithi117.stdout:45 files removed 2024-09-21T05:43:23.515 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T05:43:23.532 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T05:43:23.535 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T05:43:24.066 INFO:teuthology.orchestra.run.smithi192.stdout:ceph packages for x86_64 307 kB/s | 84 kB 00:00 2024-09-21T05:43:24.089 INFO:teuthology.orchestra.run.smithi117.stdout:ceph packages for x86_64 312 kB/s | 84 kB 00:00 2024-09-21T05:43:24.092 INFO:teuthology.orchestra.run.smithi138.stdout:ceph packages for x86_64 299 kB/s | 84 kB 00:00 2024-09-21T05:43:24.367 INFO:teuthology.orchestra.run.smithi192.stdout:ceph noarch packages 94 kB/s | 24 kB 00:00 2024-09-21T05:43:24.384 INFO:teuthology.orchestra.run.smithi138.stdout:ceph noarch packages 96 kB/s | 24 kB 00:00 2024-09-21T05:43:24.389 INFO:teuthology.orchestra.run.smithi117.stdout:ceph noarch packages 94 kB/s | 24 kB 00:00 2024-09-21T05:43:24.642 INFO:teuthology.orchestra.run.smithi192.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-21T05:43:24.669 INFO:teuthology.orchestra.run.smithi138.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-21T05:43:24.690 INFO:teuthology.orchestra.run.smithi117.stdout:ceph source packages 7.2 kB/s | 1.9 kB 00:00 2024-09-21T05:43:25.286 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.3 MB 00:00 2024-09-21T05:43:25.567 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - BaseOS 9.3 MB/s | 8.3 MB 00:00 2024-09-21T05:43:26.406 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.3 MB 00:01 2024-09-21T05:43:27.593 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - AppStream 21 MB/s | 20 MB 00:00 2024-09-21T05:43:28.232 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - AppStream 27 MB/s | 20 MB 00:00 2024-09-21T05:43:29.904 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - AppStream 5.7 MB/s | 20 MB 00:03 2024-09-21T05:43:32.053 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-21T05:43:34.230 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-21T05:43:34.303 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 19 kB 00:01 2024-09-21T05:43:35.241 INFO:teuthology.orchestra.run.smithi192.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-21T05:43:35.367 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - CRB 2.0 MB/s | 6.5 MB 00:03 2024-09-21T05:43:35.723 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - Extras packages 66 kB/s | 19 kB 00:00 2024-09-21T05:43:36.627 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-21T05:43:37.008 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 19 kB 00:00 2024-09-21T05:43:37.888 INFO:teuthology.orchestra.run.smithi117.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-21T05:43:40.613 INFO:teuthology.orchestra.run.smithi192.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-21T05:43:42.025 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-09-21T05:43:42.757 INFO:teuthology.orchestra.run.smithi192.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T05:43:42.758 INFO:teuthology.orchestra.run.smithi192.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T05:43:42.769 INFO:teuthology.orchestra.run.smithi192.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T05:43:42.769 INFO:teuthology.orchestra.run.smithi192.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T05:43:42.770 INFO:teuthology.orchestra.run.smithi192.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T05:43:42.771 INFO:teuthology.orchestra.run.smithi192.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T05:43:42.773 INFO:teuthology.orchestra.run.smithi192.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T05:43:42.775 INFO:teuthology.orchestra.run.smithi192.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T05:43:42.776 INFO:teuthology.orchestra.run.smithi192.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T05:43:42.838 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-21T05:43:42.854 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-09-21T05:43:42.854 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-09-21T05:43:42.854 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-09-21T05:43:42.854 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-21T05:43:42.854 INFO:teuthology.orchestra.run.smithi192.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T05:43:42.854 INFO:teuthology.orchestra.run.smithi192.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T05:43:42.854 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T05:43:42.855 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T05:43:42.855 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T05:43:42.855 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T05:43:42.855 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T05:43:42.855 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T05:43:42.855 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T05:43:42.855 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T05:43:42.855 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T05:43:42.855 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T05:43:42.855 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T05:43:42.855 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T05:43:42.855 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T05:43:42.855 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T05:43:42.855 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T05:43:42.856 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T05:43:42.856 INFO:teuthology.orchestra.run.smithi192.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T05:43:42.856 INFO:teuthology.orchestra.run.smithi192.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T05:43:42.856 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T05:43:42.856 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T05:43:42.856 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T05:43:42.856 INFO:teuthology.orchestra.run.smithi192.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T05:43:42.856 INFO:teuthology.orchestra.run.smithi192.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T05:43:42.856 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T05:43:42.856 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T05:43:42.856 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T05:43:42.856 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T05:43:42.856 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T05:43:42.857 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T05:43:42.857 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T05:43:42.857 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T05:43:42.857 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T05:43:42.857 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T05:43:42.857 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T05:43:42.857 INFO:teuthology.orchestra.run.smithi192.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T05:43:42.857 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrading: 2024-09-21T05:43:42.857 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T05:43:42.857 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T05:43:42.857 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T05:43:42.857 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T05:43:42.857 INFO:teuthology.orchestra.run.smithi192.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T05:43:42.857 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T05:43:42.858 INFO:teuthology.orchestra.run.smithi192.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T05:43:42.858 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T05:43:42.858 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-09-21T05:43:42.858 INFO:teuthology.orchestra.run.smithi192.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T05:43:42.858 INFO:teuthology.orchestra.run.smithi192.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T05:43:42.858 INFO:teuthology.orchestra.run.smithi192.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T05:43:42.858 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T05:43:42.858 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T05:43:42.858 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T05:43:42.858 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T05:43:42.858 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T05:43:42.858 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T05:43:42.858 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T05:43:42.858 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T05:43:42.859 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T05:43:42.859 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T05:43:42.859 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T05:43:42.859 INFO:teuthology.orchestra.run.smithi192.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T05:43:42.859 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T05:43:42.859 INFO:teuthology.orchestra.run.smithi192.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T05:43:42.859 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T05:43:42.859 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T05:43:42.859 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T05:43:42.859 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T05:43:42.859 INFO:teuthology.orchestra.run.smithi192.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T05:43:42.859 INFO:teuthology.orchestra.run.smithi192.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T05:43:42.859 INFO:teuthology.orchestra.run.smithi192.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T05:43:42.859 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T05:43:42.859 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T05:43:42.860 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T05:43:42.860 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T05:43:42.860 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T05:43:42.860 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T05:43:42.860 INFO:teuthology.orchestra.run.smithi192.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T05:43:42.860 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T05:43:42.860 INFO:teuthology.orchestra.run.smithi192.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T05:43:42.860 INFO:teuthology.orchestra.run.smithi192.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T05:43:42.860 INFO:teuthology.orchestra.run.smithi192.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T05:43:42.860 INFO:teuthology.orchestra.run.smithi192.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T05:43:42.860 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T05:43:42.860 INFO:teuthology.orchestra.run.smithi192.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T05:43:42.860 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T05:43:42.860 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T05:43:42.861 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T05:43:42.861 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T05:43:42.861 INFO:teuthology.orchestra.run.smithi192.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T05:43:42.861 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T05:43:42.861 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T05:43:42.861 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T05:43:42.861 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T05:43:42.861 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T05:43:42.861 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T05:43:42.861 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T05:43:42.861 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T05:43:42.861 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T05:43:42.861 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T05:43:42.862 INFO:teuthology.orchestra.run.smithi192.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T05:43:42.862 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T05:43:42.862 INFO:teuthology.orchestra.run.smithi192.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T05:43:42.862 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T05:43:42.862 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T05:43:42.862 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T05:43:42.862 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T05:43:42.862 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T05:43:42.862 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T05:43:42.862 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T05:43:42.862 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T05:43:42.862 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T05:43:42.862 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T05:43:42.862 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T05:43:42.863 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T05:43:42.863 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T05:43:42.863 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T05:43:42.863 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T05:43:42.863 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T05:43:42.863 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T05:43:42.863 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T05:43:42.863 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T05:43:42.863 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T05:43:42.863 INFO:teuthology.orchestra.run.smithi192.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T05:43:42.863 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T05:43:42.863 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T05:43:42.863 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T05:43:42.863 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T05:43:42.864 INFO:teuthology.orchestra.run.smithi192.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T05:43:42.864 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T05:43:42.864 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T05:43:42.864 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T05:43:42.864 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T05:43:42.864 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T05:43:42.864 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T05:43:42.864 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T05:43:42.864 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T05:43:42.864 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T05:43:42.864 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T05:43:42.864 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T05:43:42.864 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T05:43:42.864 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T05:43:42.864 INFO:teuthology.orchestra.run.smithi192.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T05:43:42.865 INFO:teuthology.orchestra.run.smithi192.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T05:43:42.865 INFO:teuthology.orchestra.run.smithi192.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T05:43:42.865 INFO:teuthology.orchestra.run.smithi192.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T05:43:42.865 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T05:43:42.865 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T05:43:42.865 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T05:43:42.865 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T05:43:42.865 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T05:43:42.865 INFO:teuthology.orchestra.run.smithi192.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T05:43:42.865 INFO:teuthology.orchestra.run.smithi192.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T05:43:42.865 INFO:teuthology.orchestra.run.smithi192.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T05:43:42.865 INFO:teuthology.orchestra.run.smithi192.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T05:43:42.865 INFO:teuthology.orchestra.run.smithi192.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T05:43:42.865 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T05:43:42.865 INFO:teuthology.orchestra.run.smithi192.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T05:43:42.866 INFO:teuthology.orchestra.run.smithi192.stdout:Installing weak dependencies: 2024-09-21T05:43:42.866 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T05:43:42.866 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-21T05:43:42.866 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-21T05:43:42.866 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-09-21T05:43:42.866 INFO:teuthology.orchestra.run.smithi192.stdout:Install 149 Packages 2024-09-21T05:43:42.866 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrade 8 Packages 2024-09-21T05:43:42.866 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-21T05:43:42.867 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 228 M 2024-09-21T05:43:42.867 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-21T05:43:43.293 INFO:teuthology.orchestra.run.smithi117.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-21T05:43:43.323 INFO:teuthology.orchestra.run.smithi192.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 49 kB/s | 6.6 kB 00:00 2024-09-21T05:43:43.590 INFO:teuthology.orchestra.run.smithi192.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 3.2 MB/s | 873 kB 00:00 2024-09-21T05:43:43.674 INFO:teuthology.orchestra.run.smithi192.stdout:(3/157): ceph-base-19.3.0-5120.g18432218.el9.x8 11 MB/s | 5.5 MB 00:00 2024-09-21T05:43:43.691 INFO:teuthology.orchestra.run.smithi192.stdout:(4/157): ceph-immutable-object-cache-19.3.0-512 1.4 MB/s | 148 kB 00:00 2024-09-21T05:43:43.825 INFO:teuthology.orchestra.run.smithi192.stdout:(5/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 15 MB/s | 2.3 MB 00:00 2024-09-21T05:43:44.000 INFO:teuthology.orchestra.run.smithi192.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 4.9 MB/s | 1.5 MB 00:00 2024-09-21T05:43:44.117 INFO:teuthology.orchestra.run.smithi192.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 17 MB/s | 5.0 MB 00:00 2024-09-21T05:43:44.146 INFO:teuthology.orchestra.run.smithi138.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T05:43:44.147 INFO:teuthology.orchestra.run.smithi138.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T05:43:44.160 INFO:teuthology.orchestra.run.smithi138.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T05:43:44.161 INFO:teuthology.orchestra.run.smithi138.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T05:43:44.162 INFO:teuthology.orchestra.run.smithi138.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T05:43:44.163 INFO:teuthology.orchestra.run.smithi138.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T05:43:44.165 INFO:teuthology.orchestra.run.smithi138.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T05:43:44.168 INFO:teuthology.orchestra.run.smithi138.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T05:43:44.169 INFO:teuthology.orchestra.run.smithi138.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T05:43:44.232 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-21T05:43:44.253 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-09-21T05:43:44.253 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-09-21T05:43:44.253 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-09-21T05:43:44.253 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-09-21T05:43:44.253 INFO:teuthology.orchestra.run.smithi138.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T05:43:44.253 INFO:teuthology.orchestra.run.smithi138.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T05:43:44.253 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T05:43:44.253 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T05:43:44.254 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T05:43:44.254 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T05:43:44.254 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T05:43:44.254 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T05:43:44.254 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T05:43:44.254 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T05:43:44.254 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T05:43:44.254 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T05:43:44.254 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T05:43:44.254 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T05:43:44.254 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T05:43:44.254 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T05:43:44.254 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T05:43:44.255 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T05:43:44.255 INFO:teuthology.orchestra.run.smithi138.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T05:43:44.255 INFO:teuthology.orchestra.run.smithi138.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T05:43:44.255 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T05:43:44.255 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T05:43:44.255 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T05:43:44.255 INFO:teuthology.orchestra.run.smithi138.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T05:43:44.255 INFO:teuthology.orchestra.run.smithi138.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T05:43:44.255 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T05:43:44.255 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T05:43:44.255 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T05:43:44.255 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T05:43:44.255 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T05:43:44.255 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T05:43:44.256 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T05:43:44.256 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T05:43:44.256 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T05:43:44.256 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T05:43:44.256 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T05:43:44.256 INFO:teuthology.orchestra.run.smithi138.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T05:43:44.256 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrading: 2024-09-21T05:43:44.256 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T05:43:44.256 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T05:43:44.256 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T05:43:44.256 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T05:43:44.256 INFO:teuthology.orchestra.run.smithi138.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T05:43:44.256 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T05:43:44.256 INFO:teuthology.orchestra.run.smithi138.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T05:43:44.257 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T05:43:44.257 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-09-21T05:43:44.257 INFO:teuthology.orchestra.run.smithi138.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T05:43:44.257 INFO:teuthology.orchestra.run.smithi138.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T05:43:44.257 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T05:43:44.257 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T05:43:44.257 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T05:43:44.257 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T05:43:44.257 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T05:43:44.257 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T05:43:44.257 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T05:43:44.257 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T05:43:44.257 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T05:43:44.257 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T05:43:44.257 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T05:43:44.258 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T05:43:44.258 INFO:teuthology.orchestra.run.smithi138.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T05:43:44.258 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T05:43:44.258 INFO:teuthology.orchestra.run.smithi138.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T05:43:44.258 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T05:43:44.258 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T05:43:44.258 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T05:43:44.258 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T05:43:44.258 INFO:teuthology.orchestra.run.smithi138.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T05:43:44.258 INFO:teuthology.orchestra.run.smithi138.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T05:43:44.258 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T05:43:44.258 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T05:43:44.258 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T05:43:44.259 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T05:43:44.259 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T05:43:44.259 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T05:43:44.259 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T05:43:44.259 INFO:teuthology.orchestra.run.smithi138.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T05:43:44.259 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T05:43:44.259 INFO:teuthology.orchestra.run.smithi138.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T05:43:44.259 INFO:teuthology.orchestra.run.smithi138.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T05:43:44.259 INFO:teuthology.orchestra.run.smithi138.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T05:43:44.259 INFO:teuthology.orchestra.run.smithi138.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T05:43:44.259 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T05:43:44.259 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T05:43:44.259 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T05:43:44.259 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T05:43:44.260 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T05:43:44.260 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T05:43:44.260 INFO:teuthology.orchestra.run.smithi138.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T05:43:44.260 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T05:43:44.260 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T05:43:44.260 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T05:43:44.260 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T05:43:44.260 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T05:43:44.260 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T05:43:44.260 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T05:43:44.260 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T05:43:44.261 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T05:43:44.261 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T05:43:44.261 INFO:teuthology.orchestra.run.smithi138.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T05:43:44.261 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T05:43:44.261 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T05:43:44.261 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T05:43:44.261 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T05:43:44.261 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T05:43:44.261 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T05:43:44.261 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T05:43:44.261 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T05:43:44.261 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T05:43:44.261 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T05:43:44.261 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T05:43:44.262 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T05:43:44.262 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T05:43:44.262 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T05:43:44.262 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T05:43:44.262 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T05:43:44.262 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T05:43:44.262 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T05:43:44.262 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T05:43:44.262 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T05:43:44.262 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T05:43:44.263 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T05:43:44.263 INFO:teuthology.orchestra.run.smithi138.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T05:43:44.263 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T05:43:44.263 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T05:43:44.263 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T05:43:44.263 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T05:43:44.263 INFO:teuthology.orchestra.run.smithi138.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T05:43:44.263 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T05:43:44.263 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T05:43:44.263 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T05:43:44.263 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T05:43:44.263 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T05:43:44.264 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T05:43:44.264 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T05:43:44.264 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T05:43:44.264 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T05:43:44.264 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T05:43:44.264 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T05:43:44.264 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T05:43:44.264 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T05:43:44.264 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T05:43:44.264 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T05:43:44.264 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T05:43:44.264 INFO:teuthology.orchestra.run.smithi138.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T05:43:44.265 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T05:43:44.265 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T05:43:44.265 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T05:43:44.265 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T05:43:44.265 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T05:43:44.265 INFO:teuthology.orchestra.run.smithi138.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T05:43:44.265 INFO:teuthology.orchestra.run.smithi138.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T05:43:44.265 INFO:teuthology.orchestra.run.smithi138.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T05:43:44.265 INFO:teuthology.orchestra.run.smithi138.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T05:43:44.265 INFO:teuthology.orchestra.run.smithi138.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T05:43:44.265 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T05:43:44.265 INFO:teuthology.orchestra.run.smithi138.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T05:43:44.266 INFO:teuthology.orchestra.run.smithi138.stdout:Installing weak dependencies: 2024-09-21T05:43:44.266 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T05:43:44.266 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-21T05:43:44.266 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-09-21T05:43:44.266 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-09-21T05:43:44.266 INFO:teuthology.orchestra.run.smithi138.stdout:Install 149 Packages 2024-09-21T05:43:44.266 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrade 8 Packages 2024-09-21T05:43:44.266 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-21T05:43:44.266 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 228 M 2024-09-21T05:43:44.267 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-09-21T05:43:44.308 INFO:teuthology.orchestra.run.smithi192.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 18 MB/s | 20 MB 00:01 2024-09-21T05:43:44.350 INFO:teuthology.orchestra.run.smithi192.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 607 kB/s | 25 kB 00:00 2024-09-21T05:43:44.626 INFO:teuthology.orchestra.run.smithi192.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 20 MB/s | 10 MB 00:00 2024-09-21T05:43:44.676 INFO:teuthology.orchestra.run.smithi192.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 4.4 MB/s | 226 kB 00:00 2024-09-21T05:43:44.751 INFO:teuthology.orchestra.run.smithi192.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 360 kB/s | 27 kB 00:00 2024-09-21T05:43:44.826 INFO:teuthology.orchestra.run.smithi192.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 424 kB/s | 32 kB 00:00 2024-09-21T05:43:44.927 INFO:teuthology.orchestra.run.smithi192.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 7.5 MB/s | 766 kB 00:00 2024-09-21T05:43:45.010 INFO:teuthology.orchestra.run.smithi192.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 1.9 MB/s | 158 kB 00:00 2024-09-21T05:43:45.102 INFO:teuthology.orchestra.run.smithi192.stdout:(16/157): librados-devel-19.3.0-5120.g18432218. 1.4 MB/s | 127 kB 00:00 2024-09-21T05:43:45.211 INFO:teuthology.orchestra.run.smithi192.stdout:(17/157): libradosstriper1-19.3.0-5120.g1843221 4.6 MB/s | 507 kB 00:00 2024-09-21T05:43:45.520 INFO:teuthology.orchestra.run.smithi192.stdout:(18/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 12 MB/s | 18 MB 00:01 2024-09-21T05:43:45.545 INFO:teuthology.orchestra.run.smithi117.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T05:43:45.546 INFO:teuthology.orchestra.run.smithi117.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T05:43:45.557 INFO:teuthology.orchestra.run.smithi117.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T05:43:45.558 INFO:teuthology.orchestra.run.smithi117.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T05:43:45.559 INFO:teuthology.orchestra.run.smithi117.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T05:43:45.560 INFO:teuthology.orchestra.run.smithi117.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T05:43:45.561 INFO:teuthology.orchestra.run.smithi192.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 1.1 MB/s | 45 kB 00:00 2024-09-21T05:43:45.562 INFO:teuthology.orchestra.run.smithi117.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T05:43:45.564 INFO:teuthology.orchestra.run.smithi117.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T05:43:45.565 INFO:teuthology.orchestra.run.smithi117.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T05:43:45.627 INFO:teuthology.orchestra.run.smithi138.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 13 kB/s | 6.6 kB 00:00 2024-09-21T05:43:45.628 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-09-21T05:43:45.636 INFO:teuthology.orchestra.run.smithi192.stdout:(20/157): librgw2-19.3.0-5120.g18432218.el9.x86 12 MB/s | 5.3 MB 00:00 2024-09-21T05:43:45.645 INFO:teuthology.orchestra.run.smithi117.stdout:======================================================================================= 2024-09-21T05:43:45.645 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-09-21T05:43:45.645 INFO:teuthology.orchestra.run.smithi117.stdout:======================================================================================= 2024-09-21T05:43:45.646 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-09-21T05:43:45.646 INFO:teuthology.orchestra.run.smithi117.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T05:43:45.646 INFO:teuthology.orchestra.run.smithi117.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T05:43:45.646 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T05:43:45.646 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T05:43:45.646 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T05:43:45.646 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T05:43:45.646 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T05:43:45.646 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T05:43:45.646 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T05:43:45.646 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T05:43:45.646 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T05:43:45.647 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T05:43:45.647 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T05:43:45.647 INFO:teuthology.orchestra.run.smithi117.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T05:43:45.647 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T05:43:45.647 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T05:43:45.647 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T05:43:45.647 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T05:43:45.647 INFO:teuthology.orchestra.run.smithi117.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T05:43:45.647 INFO:teuthology.orchestra.run.smithi117.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T05:43:45.647 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T05:43:45.647 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T05:43:45.647 INFO:teuthology.orchestra.run.smithi117.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T05:43:45.647 INFO:teuthology.orchestra.run.smithi117.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T05:43:45.647 INFO:teuthology.orchestra.run.smithi117.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T05:43:45.648 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T05:43:45.648 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T05:43:45.648 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T05:43:45.648 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T05:43:45.648 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T05:43:45.648 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T05:43:45.648 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T05:43:45.648 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T05:43:45.648 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T05:43:45.648 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T05:43:45.648 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T05:43:45.648 INFO:teuthology.orchestra.run.smithi117.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T05:43:45.648 INFO:teuthology.orchestra.run.smithi117.stdout:Upgrading: 2024-09-21T05:43:45.648 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T05:43:45.649 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T05:43:45.649 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T05:43:45.649 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T05:43:45.649 INFO:teuthology.orchestra.run.smithi117.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T05:43:45.649 INFO:teuthology.orchestra.run.smithi117.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T05:43:45.649 INFO:teuthology.orchestra.run.smithi117.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T05:43:45.649 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T05:43:45.649 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-09-21T05:43:45.649 INFO:teuthology.orchestra.run.smithi117.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T05:43:45.649 INFO:teuthology.orchestra.run.smithi117.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T05:43:45.649 INFO:teuthology.orchestra.run.smithi117.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T05:43:45.649 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T05:43:45.649 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T05:43:45.650 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T05:43:45.650 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T05:43:45.650 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T05:43:45.650 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T05:43:45.650 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T05:43:45.650 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T05:43:45.650 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T05:43:45.650 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T05:43:45.650 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T05:43:45.650 INFO:teuthology.orchestra.run.smithi117.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T05:43:45.650 INFO:teuthology.orchestra.run.smithi117.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T05:43:45.650 INFO:teuthology.orchestra.run.smithi117.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T05:43:45.650 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T05:43:45.650 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T05:43:45.651 INFO:teuthology.orchestra.run.smithi117.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T05:43:45.651 INFO:teuthology.orchestra.run.smithi117.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T05:43:45.651 INFO:teuthology.orchestra.run.smithi117.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T05:43:45.651 INFO:teuthology.orchestra.run.smithi117.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T05:43:45.651 INFO:teuthology.orchestra.run.smithi117.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T05:43:45.651 INFO:teuthology.orchestra.run.smithi117.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T05:43:45.651 INFO:teuthology.orchestra.run.smithi117.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T05:43:45.651 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T05:43:45.651 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T05:43:45.651 INFO:teuthology.orchestra.run.smithi117.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T05:43:45.651 INFO:teuthology.orchestra.run.smithi117.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T05:43:45.651 INFO:teuthology.orchestra.run.smithi117.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T05:43:45.651 INFO:teuthology.orchestra.run.smithi117.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T05:43:45.652 INFO:teuthology.orchestra.run.smithi117.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T05:43:45.652 INFO:teuthology.orchestra.run.smithi117.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T05:43:45.652 INFO:teuthology.orchestra.run.smithi117.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T05:43:45.652 INFO:teuthology.orchestra.run.smithi117.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T05:43:45.652 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T05:43:45.652 INFO:teuthology.orchestra.run.smithi117.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T05:43:45.652 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T05:43:45.652 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T05:43:45.652 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T05:43:45.652 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T05:43:45.652 INFO:teuthology.orchestra.run.smithi117.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T05:43:45.652 INFO:teuthology.orchestra.run.smithi117.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T05:43:45.652 INFO:teuthology.orchestra.run.smithi117.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T05:43:45.652 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T05:43:45.653 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T05:43:45.653 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T05:43:45.653 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T05:43:45.653 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T05:43:45.653 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T05:43:45.653 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T05:43:45.653 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T05:43:45.653 INFO:teuthology.orchestra.run.smithi117.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T05:43:45.653 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T05:43:45.653 INFO:teuthology.orchestra.run.smithi117.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T05:43:45.653 INFO:teuthology.orchestra.run.smithi117.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T05:43:45.653 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T05:43:45.653 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T05:43:45.653 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T05:43:45.654 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T05:43:45.654 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T05:43:45.654 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T05:43:45.654 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T05:43:45.654 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T05:43:45.654 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T05:43:45.654 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T05:43:45.654 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T05:43:45.654 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T05:43:45.654 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T05:43:45.654 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T05:43:45.654 INFO:teuthology.orchestra.run.smithi117.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T05:43:45.654 INFO:teuthology.orchestra.run.smithi117.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T05:43:45.655 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T05:43:45.655 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T05:43:45.655 INFO:teuthology.orchestra.run.smithi117.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T05:43:45.655 INFO:teuthology.orchestra.run.smithi117.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T05:43:45.655 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T05:43:45.655 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T05:43:45.655 INFO:teuthology.orchestra.run.smithi117.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T05:43:45.655 INFO:teuthology.orchestra.run.smithi117.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T05:43:45.655 INFO:teuthology.orchestra.run.smithi117.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T05:43:45.655 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T05:43:45.655 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T05:43:45.655 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T05:43:45.655 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T05:43:45.655 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T05:43:45.656 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T05:43:45.656 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T05:43:45.656 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T05:43:45.656 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T05:43:45.656 INFO:teuthology.orchestra.run.smithi117.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T05:43:45.656 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T05:43:45.656 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T05:43:45.656 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T05:43:45.656 INFO:teuthology.orchestra.run.smithi117.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T05:43:45.656 INFO:teuthology.orchestra.run.smithi117.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T05:43:45.656 INFO:teuthology.orchestra.run.smithi117.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T05:43:45.657 INFO:teuthology.orchestra.run.smithi117.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T05:43:45.657 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T05:43:45.657 INFO:teuthology.orchestra.run.smithi117.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T05:43:45.657 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T05:43:45.657 INFO:teuthology.orchestra.run.smithi117.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T05:43:45.657 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T05:43:45.657 INFO:teuthology.orchestra.run.smithi117.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T05:43:45.657 INFO:teuthology.orchestra.run.smithi117.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T05:43:45.657 INFO:teuthology.orchestra.run.smithi117.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T05:43:45.657 INFO:teuthology.orchestra.run.smithi117.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T05:43:45.657 INFO:teuthology.orchestra.run.smithi117.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T05:43:45.657 INFO:teuthology.orchestra.run.smithi117.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T05:43:45.657 INFO:teuthology.orchestra.run.smithi117.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T05:43:45.658 INFO:teuthology.orchestra.run.smithi117.stdout:Installing weak dependencies: 2024-09-21T05:43:45.658 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T05:43:45.658 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:43:45.658 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-09-21T05:43:45.658 INFO:teuthology.orchestra.run.smithi117.stdout:======================================================================================= 2024-09-21T05:43:45.658 INFO:teuthology.orchestra.run.smithi117.stdout:Install 149 Packages 2024-09-21T05:43:45.658 INFO:teuthology.orchestra.run.smithi117.stdout:Upgrade 8 Packages 2024-09-21T05:43:45.658 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:43:45.658 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 228 M 2024-09-21T05:43:45.659 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-09-21T05:43:45.662 INFO:teuthology.orchestra.run.smithi192.stdout:(21/157): python3-ceph-common-19.3.0-5120.g1843 1.4 MB/s | 148 kB 00:00 2024-09-21T05:43:45.687 INFO:teuthology.orchestra.run.smithi192.stdout:(22/157): python3-cephfs-19.3.0-5120.g18432218. 3.1 MB/s | 161 kB 00:00 2024-09-21T05:43:45.713 INFO:teuthology.orchestra.run.smithi192.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 6.2 MB/s | 321 kB 00:00 2024-09-21T05:43:45.738 INFO:teuthology.orchestra.run.smithi192.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 6.0 MB/s | 307 kB 00:00 2024-09-21T05:43:45.755 INFO:teuthology.orchestra.run.smithi192.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 2.3 MB/s | 99 kB 00:00 2024-09-21T05:43:45.797 INFO:teuthology.orchestra.run.smithi192.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.4 MB/s | 86 kB 00:00 2024-09-21T05:43:45.919 INFO:teuthology.orchestra.run.smithi138.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 2.9 MB/s | 873 kB 00:00 2024-09-21T05:43:45.994 INFO:teuthology.orchestra.run.smithi138.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 1.9 MB/s | 148 kB 00:00 2024-09-21T05:43:46.078 INFO:teuthology.orchestra.run.smithi138.stdout:(4/157): ceph-base-19.3.0-5120.g18432218.el9.x8 5.8 MB/s | 5.5 MB 00:00 2024-09-21T05:43:46.156 INFO:teuthology.orchestra.run.smithi192.stdout:(27/157): ceph-test-19.3.0-5120.g18432218.el9.x 22 MB/s | 40 MB 00:01 2024-09-21T05:43:46.181 INFO:teuthology.orchestra.run.smithi192.stdout:(28/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 443 kB/s | 170 kB 00:00 2024-09-21T05:43:46.220 INFO:teuthology.orchestra.run.smithi138.stdout:(5/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 11 MB/s | 1.5 MB 00:00 2024-09-21T05:43:46.236 INFO:teuthology.orchestra.run.smithi117.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 37 kB/s | 6.6 kB 00:00 2024-09-21T05:43:46.265 INFO:teuthology.orchestra.run.smithi192.stdout:(29/157): rbd-mirror-19.3.0-5120.g18432218.el9. 6.1 MB/s | 3.1 MB 00:00 2024-09-21T05:43:46.270 INFO:teuthology.orchestra.run.smithi138.stdout:(6/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 8.3 MB/s | 2.3 MB 00:00 2024-09-21T05:43:46.290 INFO:teuthology.orchestra.run.smithi192.stdout:(30/157): ceph-grafana-dashboards-19.3.0-5120.g 256 kB/s | 34 kB 00:00 2024-09-21T05:43:46.315 INFO:teuthology.orchestra.run.smithi192.stdout:(31/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 1.1 MB/s | 154 kB 00:00 2024-09-21T05:43:46.695 INFO:teuthology.orchestra.run.smithi138.stdout:(7/157): ceph-common-19.3.0-5120.g18432218.el9. 13 MB/s | 20 MB 00:01 2024-09-21T05:43:46.771 INFO:teuthology.orchestra.run.smithi138.stdout:(8/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 9.0 MB/s | 5.0 MB 00:00 2024-09-21T05:43:46.790 INFO:teuthology.orchestra.run.smithi192.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 615 kB/s | 292 kB 00:00 2024-09-21T05:43:46.794 INFO:teuthology.orchestra.run.smithi117.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 1.5 MB/s | 873 kB 00:00 2024-09-21T05:43:46.854 INFO:teuthology.orchestra.run.smithi138.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 302 kB/s | 25 kB 00:00 2024-09-21T05:43:46.883 INFO:teuthology.orchestra.run.smithi192.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 539 kB/s | 49 kB 00:00 2024-09-21T05:43:46.945 INFO:teuthology.orchestra.run.smithi117.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 989 kB/s | 148 kB 00:00 2024-09-21T05:43:46.974 INFO:teuthology.orchestra.run.smithi192.stdout:(34/157): ceph-prometheus-alerts-19.3.0-5120.g1 180 kB/s | 16 kB 00:00 2024-09-21T05:43:47.066 INFO:teuthology.orchestra.run.smithi192.stdout:(35/157): ceph-mgr-diskprediction-local-19.3.0- 9.5 MB/s | 7.4 MB 00:00 2024-09-21T05:43:47.092 INFO:teuthology.orchestra.run.smithi192.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 2.4 MB/s | 285 kB 00:00 2024-09-21T05:43:47.175 INFO:teuthology.orchestra.run.smithi192.stdout:(37/157): cephadm-19.3.0-5120.g18432218.el9.noa 7.1 MB/s | 784 kB 00:00 2024-09-21T05:43:47.200 INFO:teuthology.orchestra.run.smithi192.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 169 kB/s | 18 kB 00:00 2024-09-21T05:43:47.342 INFO:teuthology.orchestra.run.smithi192.stdout:(39/157): ceph-mgr-dashboard-19.3.0-5120.g18432 13 MB/s | 14 MB 00:01 2024-09-21T05:43:47.355 INFO:teuthology.orchestra.run.smithi138.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 16 MB/s | 10 MB 00:00 2024-09-21T05:43:47.405 INFO:teuthology.orchestra.run.smithi138.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 4.4 MB/s | 226 kB 00:00 2024-09-21T05:43:47.463 INFO:teuthology.orchestra.run.smithi138.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 463 kB/s | 27 kB 00:00 2024-09-21T05:43:47.503 INFO:teuthology.orchestra.run.smithi117.stdout:(4/157): ceph-base-19.3.0-5120.g18432218.el9.x8 3.8 MB/s | 5.5 MB 00:01 2024-09-21T05:43:47.513 INFO:teuthology.orchestra.run.smithi138.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 636 kB/s | 32 kB 00:00 2024-09-21T05:43:47.581 INFO:teuthology.orchestra.run.smithi138.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 11 MB/s | 766 kB 00:00 2024-09-21T05:43:47.626 INFO:teuthology.orchestra.run.smithi192.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 177 kB/s | 80 kB 00:00 2024-09-21T05:43:47.639 INFO:teuthology.orchestra.run.smithi138.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 2.7 MB/s | 158 kB 00:00 2024-09-21T05:43:47.653 INFO:teuthology.orchestra.run.smithi117.stdout:(5/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 10 MB/s | 1.5 MB 00:00 2024-09-21T05:43:47.698 INFO:teuthology.orchestra.run.smithi138.stdout:(16/157): librados-devel-19.3.0-5120.g18432218. 2.1 MB/s | 127 kB 00:00 2024-09-21T05:43:47.765 INFO:teuthology.orchestra.run.smithi138.stdout:(17/157): libradosstriper1-19.3.0-5120.g1843221 7.4 MB/s | 507 kB 00:00 2024-09-21T05:43:47.893 INFO:teuthology.orchestra.run.smithi192.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 348 kB/s | 191 kB 00:00 2024-09-21T05:43:47.912 INFO:teuthology.orchestra.run.smithi117.stdout:(6/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 2.4 MB/s | 2.3 MB 00:00 2024-09-21T05:43:47.918 INFO:teuthology.orchestra.run.smithi192.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 868 kB/s | 253 kB 00:00 2024-09-21T05:43:47.985 INFO:teuthology.orchestra.run.smithi192.stdout:(43/157): libgfortran-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 801 kB 00:00 2024-09-21T05:43:48.027 INFO:teuthology.orchestra.run.smithi192.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 981 kB/s | 106 kB 00:00 2024-09-21T05:43:48.066 INFO:teuthology.orchestra.run.smithi138.stdout:(18/157): librgw2-19.3.0-5120.g18432218.el9.x86 18 MB/s | 5.3 MB 00:00 2024-09-21T05:43:48.102 INFO:teuthology.orchestra.run.smithi192.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-09-21T05:43:48.116 INFO:teuthology.orchestra.run.smithi138.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 906 kB/s | 45 kB 00:00 2024-09-21T05:43:48.120 INFO:teuthology.orchestra.run.smithi117.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 11 MB/s | 5.0 MB 00:00 2024-09-21T05:43:48.144 INFO:teuthology.orchestra.run.smithi192.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-09-21T05:43:48.195 INFO:teuthology.orchestra.run.smithi192.stdout:(47/157): python3-cryptography-36.0.1-4.el9.x86 4.1 MB/s | 1.2 MB 00:00 2024-09-21T05:43:48.220 INFO:teuthology.orchestra.run.smithi192.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-09-21T05:43:48.317 INFO:teuthology.orchestra.run.smithi138.stdout:(20/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 8.7 MB/s | 18 MB 00:02 2024-09-21T05:43:48.341 INFO:teuthology.orchestra.run.smithi138.stdout:(21/157): python3-ceph-common-19.3.0-5120.g1843 659 kB/s | 148 kB 00:00 2024-09-21T05:43:48.345 INFO:teuthology.orchestra.run.smithi192.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 857 kB/s | 107 kB 00:00 2024-09-21T05:43:48.370 INFO:teuthology.orchestra.run.smithi192.stdout:(50/157): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 689 kB 00:00 2024-09-21T05:43:48.375 INFO:teuthology.orchestra.run.smithi138.stdout:(22/157): python3-cephfs-19.3.0-5120.g18432218. 2.7 MB/s | 161 kB 00:00 2024-09-21T05:43:48.395 INFO:teuthology.orchestra.run.smithi192.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 432 kB/s | 22 kB 00:00 2024-09-21T05:43:48.409 INFO:teuthology.orchestra.run.smithi138.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 4.7 MB/s | 321 kB 00:00 2024-09-21T05:43:48.421 INFO:teuthology.orchestra.run.smithi192.stdout:(52/157): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-09-21T05:43:48.443 INFO:teuthology.orchestra.run.smithi138.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 4.5 MB/s | 307 kB 00:00 2024-09-21T05:43:48.446 INFO:teuthology.orchestra.run.smithi192.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 654 kB/s | 33 kB 00:00 2024-09-21T05:43:48.476 INFO:teuthology.orchestra.run.smithi138.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.5 MB/s | 99 kB 00:00 2024-09-21T05:43:48.479 INFO:teuthology.orchestra.run.smithi192.stdout:(54/157): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 943 kB 00:00 2024-09-21T05:43:48.501 INFO:teuthology.orchestra.run.smithi138.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.4 MB/s | 86 kB 00:00 2024-09-21T05:43:48.506 INFO:teuthology.orchestra.run.smithi192.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 310 kB/s | 18 kB 00:00 2024-09-21T05:43:48.547 INFO:teuthology.orchestra.run.smithi192.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 166 kB/s | 11 kB 00:00 2024-09-21T05:43:48.569 INFO:teuthology.orchestra.run.smithi138.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 2.5 MB/s | 170 kB 00:00 2024-09-21T05:43:48.572 INFO:teuthology.orchestra.run.smithi192.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.5 MB/s | 171 kB 00:00 2024-09-21T05:43:48.588 INFO:teuthology.orchestra.run.smithi117.stdout:(8/157): ceph-radosgw-19.3.0-5120.g18432218.el9 22 MB/s | 10 MB 00:00 2024-09-21T05:43:48.614 INFO:teuthology.orchestra.run.smithi192.stdout:(58/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-21T05:43:48.636 INFO:teuthology.orchestra.run.smithi138.stdout:(28/157): rbd-mirror-19.3.0-5120.g18432218.el9. 20 MB/s | 3.1 MB 00:00 2024-09-21T05:43:48.638 INFO:teuthology.orchestra.run.smithi117.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 506 kB/s | 25 kB 00:00 2024-09-21T05:43:48.639 INFO:teuthology.orchestra.run.smithi192.stdout:(59/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-09-21T05:43:48.661 INFO:teuthology.orchestra.run.smithi138.stdout:(29/157): ceph-grafana-dashboards-19.3.0-5120.g 372 kB/s | 34 kB 00:00 2024-09-21T05:43:48.686 INFO:teuthology.orchestra.run.smithi138.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 3.0 MB/s | 154 kB 00:00 2024-09-21T05:43:48.689 INFO:teuthology.orchestra.run.smithi192.stdout:(60/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-09-21T05:43:48.723 INFO:teuthology.orchestra.run.smithi192.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 5.9 MB/s | 662 kB 00:00 2024-09-21T05:43:48.757 INFO:teuthology.orchestra.run.smithi192.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 365 kB/s | 24 kB 00:00 2024-09-21T05:43:48.782 INFO:teuthology.orchestra.run.smithi192.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 808 kB/s | 47 kB 00:00 2024-09-21T05:43:48.808 INFO:teuthology.orchestra.run.smithi192.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-21T05:43:48.833 INFO:teuthology.orchestra.run.smithi192.stdout:(65/157): libtool-2.4.6-46.el9.x86_64.rpm 2.9 MB/s | 581 kB 00:00 2024-09-21T05:43:48.858 INFO:teuthology.orchestra.run.smithi192.stdout:(66/157): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-09-21T05:43:48.883 INFO:teuthology.orchestra.run.smithi192.stdout:(67/157): openblas-0.3.26-2.el9.x86_64.rpm 510 kB/s | 38 kB 00:00 2024-09-21T05:43:48.976 INFO:teuthology.orchestra.run.smithi192.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 35 MB/s | 4.9 MB 00:00 2024-09-21T05:43:49.009 INFO:teuthology.orchestra.run.smithi192.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-21T05:43:49.042 INFO:teuthology.orchestra.run.smithi192.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 13 MB/s | 910 kB 00:00 2024-09-21T05:43:49.068 INFO:teuthology.orchestra.run.smithi192.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-21T05:43:49.126 INFO:teuthology.orchestra.run.smithi192.stdout:(72/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-09-21T05:43:49.160 INFO:teuthology.orchestra.run.smithi192.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-09-21T05:43:49.185 INFO:teuthology.orchestra.run.smithi192.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-21T05:43:49.203 INFO:teuthology.orchestra.run.smithi138.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 14 MB/s | 7.4 MB 00:00 2024-09-21T05:43:49.269 INFO:teuthology.orchestra.run.smithi192.stdout:(75/157): postgresql-server-13.16-1.el9.x86_64. 26 MB/s | 5.8 MB 00:00 2024-09-21T05:43:49.287 INFO:teuthology.orchestra.run.smithi138.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 3.4 MB/s | 292 kB 00:00 2024-09-21T05:43:49.294 INFO:teuthology.orchestra.run.smithi192.stdout:(76/157): python3-colorama-0.4.6-3.el9.noarch.r 563 kB/s | 61 kB 00:00 2024-09-21T05:43:49.319 INFO:teuthology.orchestra.run.smithi192.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-09-21T05:43:49.345 INFO:teuthology.orchestra.run.smithi192.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-21T05:43:49.354 INFO:teuthology.orchestra.run.smithi138.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 740 kB/s | 49 kB 00:00 2024-09-21T05:43:49.370 INFO:teuthology.orchestra.run.smithi192.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-09-21T05:43:49.395 INFO:teuthology.orchestra.run.smithi192.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-09-21T05:43:49.420 INFO:teuthology.orchestra.run.smithi192.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-21T05:43:49.421 INFO:teuthology.orchestra.run.smithi138.stdout:(34/157): ceph-prometheus-alerts-19.3.0-5120.g1 246 kB/s | 16 kB 00:00 2024-09-21T05:43:49.446 INFO:teuthology.orchestra.run.smithi192.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 691 kB/s | 35 kB 00:00 2024-09-21T05:43:49.505 INFO:teuthology.orchestra.run.smithi138.stdout:(35/157): ceph-volume-19.3.0-5120.g18432218.el9 3.3 MB/s | 285 kB 00:00 2024-09-21T05:43:49.521 INFO:teuthology.orchestra.run.smithi192.stdout:(83/157): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-09-21T05:43:49.546 INFO:teuthology.orchestra.run.smithi192.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-21T05:43:49.572 INFO:teuthology.orchestra.run.smithi192.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-21T05:43:49.589 INFO:teuthology.orchestra.run.smithi138.stdout:(36/157): cephadm-19.3.0-5120.g18432218.el9.noa 9.2 MB/s | 784 kB 00:00 2024-09-21T05:43:49.597 INFO:teuthology.orchestra.run.smithi192.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-21T05:43:49.630 INFO:teuthology.orchestra.run.smithi192.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 723 kB/s | 42 kB 00:00 2024-09-21T05:43:49.656 INFO:teuthology.orchestra.run.smithi192.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-21T05:43:49.656 INFO:teuthology.orchestra.run.smithi138.stdout:(37/157): cephfs-top-19.3.0-5120.g18432218.el9. 277 kB/s | 18 kB 00:00 2024-09-21T05:43:49.681 INFO:teuthology.orchestra.run.smithi192.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-21T05:43:49.706 INFO:teuthology.orchestra.run.smithi192.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-21T05:43:49.731 INFO:teuthology.orchestra.run.smithi192.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-21T05:43:49.757 INFO:teuthology.orchestra.run.smithi192.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-21T05:43:49.841 INFO:teuthology.orchestra.run.smithi192.stdout:(93/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-21T05:43:49.866 INFO:teuthology.orchestra.run.smithi192.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 382 kB/s | 42 kB 00:00 2024-09-21T05:43:49.891 INFO:teuthology.orchestra.run.smithi192.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 887 kB/s | 44 kB 00:00 2024-09-21T05:43:49.916 INFO:teuthology.orchestra.run.smithi192.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-21T05:43:49.943 INFO:teuthology.orchestra.run.smithi192.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 714 kB/s | 36 kB 00:00 2024-09-21T05:43:49.967 INFO:teuthology.orchestra.run.smithi192.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-09-21T05:43:49.992 INFO:teuthology.orchestra.run.smithi192.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-21T05:43:50.017 INFO:teuthology.orchestra.run.smithi192.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-21T05:43:50.043 INFO:teuthology.orchestra.run.smithi192.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-21T05:43:50.068 INFO:teuthology.orchestra.run.smithi192.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-21T05:43:50.093 INFO:teuthology.orchestra.run.smithi192.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-09-21T05:43:50.119 INFO:teuthology.orchestra.run.smithi192.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 442 kB/s | 22 kB 00:00 2024-09-21T05:43:50.131 INFO:teuthology.orchestra.run.smithi138.stdout:(38/157): ceph-test-19.3.0-5120.g18432218.el9.x 12 MB/s | 40 MB 00:03 2024-09-21T05:43:50.156 INFO:teuthology.orchestra.run.smithi138.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 159 kB/s | 80 kB 00:00 2024-09-21T05:43:50.160 INFO:teuthology.orchestra.run.smithi192.stdout:(105/157): protobuf-compiler-3.14.0-13.el9.x86_ 13 MB/s | 863 kB 00:00 2024-09-21T05:43:50.203 INFO:teuthology.orchestra.run.smithi192.stdout:(106/157): abseil-cpp-20211102.0-3.el9.x86_64.r 6.4 MB/s | 548 kB 00:00 2024-09-21T05:43:50.228 INFO:teuthology.orchestra.run.smithi192.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 846 kB/s | 19 kB 00:00 2024-09-21T05:43:50.261 INFO:teuthology.orchestra.run.smithi192.stdout:(108/157): gperftools-libs-2.9.1-3.el9.x86_64.r 3.1 MB/s | 308 kB 00:00 2024-09-21T05:43:50.287 INFO:teuthology.orchestra.run.smithi192.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-21T05:43:50.312 INFO:teuthology.orchestra.run.smithi192.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-21T05:43:50.337 INFO:teuthology.orchestra.run.smithi192.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-21T05:43:50.357 INFO:teuthology.orchestra.run.smithi138.stdout:(40/157): ceph-mgr-dashboard-19.3.0-5120.g18432 8.4 MB/s | 14 MB 00:01 2024-09-21T05:43:50.362 INFO:teuthology.orchestra.run.smithi192.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 6.0 MB/s | 151 kB 00:00 2024-09-21T05:43:50.407 INFO:teuthology.orchestra.run.smithi138.stdout:(41/157): libgfortran-11.5.0-2.el9.x86_64.rpm 2.8 MB/s | 801 kB 00:00 2024-09-21T05:43:50.413 INFO:teuthology.orchestra.run.smithi192.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 16 MB/s | 839 kB 00:00 2024-09-21T05:43:50.455 INFO:teuthology.orchestra.run.smithi192.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-21T05:43:50.457 INFO:teuthology.orchestra.run.smithi138.stdout:(42/157): libquadmath-11.5.0-2.el9.x86_64.rpm 635 kB/s | 191 kB 00:00 2024-09-21T05:43:50.463 INFO:teuthology.orchestra.run.smithi117.stdout:(10/157): ceph-test-19.3.0-5120.g18432218.el9.x 22 MB/s | 40 MB 00:01 2024-09-21T05:43:50.480 INFO:teuthology.orchestra.run.smithi192.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T05:43:50.491 INFO:teuthology.orchestra.run.smithi138.stdout:(43/157): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-09-21T05:43:50.505 INFO:teuthology.orchestra.run.smithi117.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 5.4 MB/s | 226 kB 00:00 2024-09-21T05:43:50.505 INFO:teuthology.orchestra.run.smithi192.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T05:43:50.531 INFO:teuthology.orchestra.run.smithi192.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 564 kB/s | 14 kB 00:00 2024-09-21T05:43:50.538 INFO:teuthology.orchestra.run.smithi117.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 815 kB/s | 27 kB 00:00 2024-09-21T05:43:50.541 INFO:teuthology.orchestra.run.smithi138.stdout:(44/157): python3-pycparser-2.20-6.el9.noarch.r 2.7 MB/s | 135 kB 00:00 2024-09-21T05:43:50.556 INFO:teuthology.orchestra.run.smithi192.stdout:(118/157): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-21T05:43:50.567 INFO:teuthology.orchestra.run.smithi138.stdout:(45/157): python3-ply-3.11-14.el9.noarch.rpm 978 kB/s | 106 kB 00:00 2024-09-21T05:43:50.572 INFO:teuthology.orchestra.run.smithi117.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 953 kB/s | 32 kB 00:00 2024-09-21T05:43:50.589 INFO:teuthology.orchestra.run.smithi192.stdout:(119/157): python3-cherrypy-18.6.1-2.el9.noarch 11 MB/s | 358 kB 00:00 2024-09-21T05:43:50.592 INFO:teuthology.orchestra.run.smithi138.stdout:(46/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-09-21T05:43:50.617 INFO:teuthology.orchestra.run.smithi138.stdout:(47/157): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-09-21T05:43:50.623 INFO:teuthology.orchestra.run.smithi192.stdout:(120/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-09-21T05:43:50.647 INFO:teuthology.orchestra.run.smithi117.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 10 MB/s | 766 kB 00:00 2024-09-21T05:43:50.649 INFO:teuthology.orchestra.run.smithi192.stdout:(121/157): python3-google-auth-2.34.0-1.el9.noa 8.6 MB/s | 220 kB 00:00 2024-09-21T05:43:50.693 INFO:teuthology.orchestra.run.smithi138.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 1.7 MB/s | 215 kB 00:00 2024-09-21T05:43:50.697 INFO:teuthology.orchestra.run.smithi117.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 3.1 MB/s | 158 kB 00:00 2024-09-21T05:43:50.716 INFO:teuthology.orchestra.run.smithi192.stdout:(122/157): libarrow-9.0.0-11.el9.x86_64.rpm 9.1 MB/s | 4.4 MB 00:00 2024-09-21T05:43:50.741 INFO:teuthology.orchestra.run.smithi192.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-21T05:43:50.748 INFO:teuthology.orchestra.run.smithi117.stdout:(16/157): librados-devel-19.3.0-5120.g18432218. 2.5 MB/s | 127 kB 00:00 2024-09-21T05:43:50.766 INFO:teuthology.orchestra.run.smithi192.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 431 kB/s | 11 kB 00:00 2024-09-21T05:43:50.791 INFO:teuthology.orchestra.run.smithi192.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 714 kB/s | 18 kB 00:00 2024-09-21T05:43:50.798 INFO:teuthology.orchestra.run.smithi117.stdout:(17/157): libradosstriper1-19.3.0-5120.g1843221 10 MB/s | 507 kB 00:00 2024-09-21T05:43:50.801 INFO:teuthology.orchestra.run.smithi138.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 990 kB/s | 107 kB 00:00 2024-09-21T05:43:50.817 INFO:teuthology.orchestra.run.smithi192.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 926 kB/s | 23 kB 00:00 2024-09-21T05:43:50.839 INFO:teuthology.orchestra.run.smithi138.stdout:(50/157): automake-1.16.2-8.el9.noarch.rpm 2.7 MB/s | 689 kB 00:00 2024-09-21T05:43:50.856 INFO:teuthology.orchestra.run.smithi138.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 399 kB/s | 22 kB 00:00 2024-09-21T05:43:50.890 INFO:teuthology.orchestra.run.smithi138.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 3.4 MB/s | 943 kB 00:00 2024-09-21T05:43:50.907 INFO:teuthology.orchestra.run.smithi138.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 652 kB/s | 33 kB 00:00 2024-09-21T05:43:50.932 INFO:teuthology.orchestra.run.smithi138.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 313 kB 00:00 2024-09-21T05:43:50.949 INFO:teuthology.orchestra.run.smithi138.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 432 kB/s | 18 kB 00:00 2024-09-21T05:43:50.966 INFO:teuthology.orchestra.run.smithi138.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 333 kB/s | 11 kB 00:00 2024-09-21T05:43:50.984 INFO:teuthology.orchestra.run.smithi192.stdout:(127/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 13 MB/s | 16 MB 00:01 2024-09-21T05:43:50.998 INFO:teuthology.orchestra.run.smithi117.stdout:(18/157): librgw2-19.3.0-5120.g18432218.el9.x86 26 MB/s | 5.3 MB 00:00 2024-09-21T05:43:51.009 INFO:teuthology.orchestra.run.smithi192.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 101 kB/s | 19 kB 00:00 2024-09-21T05:43:51.025 INFO:teuthology.orchestra.run.smithi138.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.2 MB/s | 171 kB 00:00 2024-09-21T05:43:51.048 INFO:teuthology.orchestra.run.smithi117.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 910 kB/s | 45 kB 00:00 2024-09-21T05:43:51.050 INFO:teuthology.orchestra.run.smithi138.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-09-21T05:43:51.067 INFO:teuthology.orchestra.run.smithi138.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-21T05:43:51.084 INFO:teuthology.orchestra.run.smithi192.stdout:(129/157): python3-grpcio-1.46.7-10.el9.x86_64. 4.7 MB/s | 2.0 MB 00:00 2024-09-21T05:43:51.099 INFO:teuthology.orchestra.run.smithi117.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 2.9 MB/s | 148 kB 00:00 2024-09-21T05:43:51.110 INFO:teuthology.orchestra.run.smithi192.stdout:(130/157): python3-jaraco-text-3.2.0-6.el9.noar 157 kB/s | 20 kB 00:00 2024-09-21T05:43:51.126 INFO:teuthology.orchestra.run.smithi138.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 662 kB 00:00 2024-09-21T05:43:51.135 INFO:teuthology.orchestra.run.smithi192.stdout:(131/157): python3-logutils-0.3.5-21.el9.noarch 924 kB/s | 46 kB 00:00 2024-09-21T05:43:51.149 INFO:teuthology.orchestra.run.smithi117.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 3.2 MB/s | 161 kB 00:00 2024-09-21T05:43:51.160 INFO:teuthology.orchestra.run.smithi138.stdout:(61/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 737 kB/s | 24 kB 00:00 2024-09-21T05:43:51.160 INFO:teuthology.orchestra.run.smithi192.stdout:(132/157): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-09-21T05:43:51.193 INFO:teuthology.orchestra.run.smithi138.stdout:(62/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-09-21T05:43:51.194 INFO:teuthology.orchestra.run.smithi192.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2024-09-21T05:43:51.199 INFO:teuthology.orchestra.run.smithi117.stdout:(22/157): python3-rados-19.3.0-5120.g18432218.e 6.3 MB/s | 321 kB 00:00 2024-09-21T05:43:51.219 INFO:teuthology.orchestra.run.smithi192.stdout:(134/157): python3-natsort-7.1.1-5.el9.noarch.r 690 kB/s | 58 kB 00:00 2024-09-21T05:43:51.235 INFO:teuthology.orchestra.run.smithi138.stdout:(63/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.9 MB/s | 292 kB 00:00 2024-09-21T05:43:51.244 INFO:teuthology.orchestra.run.smithi192.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-09-21T05:43:51.249 INFO:teuthology.orchestra.run.smithi117.stdout:(23/157): python3-rbd-19.3.0-5120.g18432218.el9 6.0 MB/s | 307 kB 00:00 2024-09-21T05:43:51.270 INFO:teuthology.orchestra.run.smithi192.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-21T05:43:51.286 INFO:teuthology.orchestra.run.smithi138.stdout:(64/157): libtool-2.4.6-46.el9.x86_64.rpm 2.6 MB/s | 581 kB 00:00 2024-09-21T05:43:51.295 INFO:teuthology.orchestra.run.smithi192.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-21T05:43:51.300 INFO:teuthology.orchestra.run.smithi117.stdout:(24/157): python3-rgw-19.3.0-5120.g18432218.el9 1.9 MB/s | 99 kB 00:00 2024-09-21T05:43:51.311 INFO:teuthology.orchestra.run.smithi138.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-09-21T05:43:51.320 INFO:teuthology.orchestra.run.smithi192.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-21T05:43:51.328 INFO:teuthology.orchestra.run.smithi138.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 918 kB/s | 38 kB 00:00 2024-09-21T05:43:51.345 INFO:teuthology.orchestra.run.smithi192.stdout:(139/157): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-09-21T05:43:51.350 INFO:teuthology.orchestra.run.smithi117.stdout:(25/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-21T05:43:51.371 INFO:teuthology.orchestra.run.smithi192.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-21T05:43:51.396 INFO:teuthology.orchestra.run.smithi192.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-21T05:43:51.430 INFO:teuthology.orchestra.run.smithi192.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-09-21T05:43:51.455 INFO:teuthology.orchestra.run.smithi192.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 888 kB/s | 74 kB 00:00 2024-09-21T05:43:51.480 INFO:teuthology.orchestra.run.smithi192.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-09-21T05:43:51.492 INFO:teuthology.orchestra.run.smithi117.stdout:(26/157): rbd-mirror-19.3.0-5120.g18432218.el9. 22 MB/s | 3.1 MB 00:00 2024-09-21T05:43:51.506 INFO:teuthology.orchestra.run.smithi192.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 799 kB/s | 20 kB 00:00 2024-09-21T05:43:51.531 INFO:teuthology.orchestra.run.smithi192.stdout:(146/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-09-21T05:43:51.542 INFO:teuthology.orchestra.run.smithi117.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 3.3 MB/s | 170 kB 00:00 2024-09-21T05:43:51.556 INFO:teuthology.orchestra.run.smithi192.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-21T05:43:51.589 INFO:teuthology.orchestra.run.smithi192.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-21T05:43:51.592 INFO:teuthology.orchestra.run.smithi117.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 688 kB/s | 34 kB 00:00 2024-09-21T05:43:51.640 INFO:teuthology.orchestra.run.smithi192.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-21T05:43:51.643 INFO:teuthology.orchestra.run.smithi117.stdout:(29/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 3.0 MB/s | 154 kB 00:00 2024-09-21T05:43:51.653 INFO:teuthology.orchestra.run.smithi138.stdout:(67/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.9 MB/s | 3.0 MB 00:00 2024-09-21T05:43:51.907 INFO:teuthology.orchestra.run.smithi192.stdout:(150/157): librados2-19.3.0-5120.g18432218.el9. 9.7 MB/s | 3.4 MB 00:00 2024-09-21T05:43:52.087 INFO:teuthology.orchestra.run.smithi138.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 3.7 MB/s | 1.6 MB 00:00 2024-09-21T05:43:52.090 INFO:teuthology.orchestra.run.smithi192.stdout:(151/157): librbd1-19.3.0-5120.g18432218.el9.x8 6.3 MB/s | 3.2 MB 00:00 2024-09-21T05:43:52.201 INFO:teuthology.orchestra.run.smithi117.stdout:(30/157): ceph-mgr-dashboard-19.3.0-5120.g18432 25 MB/s | 14 MB 00:00 2024-09-21T05:43:52.362 INFO:teuthology.orchestra.run.smithi138.stdout:(69/157): postgresql-contrib-13.16-1.el9.x86_64 3.2 MB/s | 910 kB 00:00 2024-09-21T05:43:52.463 INFO:teuthology.orchestra.run.smithi117.stdout:(31/157): ceph-common-19.3.0-5120.g18432218.el9 3.1 MB/s | 20 MB 00:06 2024-09-21T05:43:52.463 INFO:teuthology.orchestra.run.smithi138.stdout:(70/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.9 MB/s | 4.4 MB 00:01 2024-09-21T05:43:52.463 INFO:teuthology.orchestra.run.smithi192.stdout:(152/157): NetworkManager-1.51.0-1.el9.x86_64.r 3.1 MB/s | 2.3 MB 00:00 2024-09-21T05:43:52.464 INFO:teuthology.orchestra.run.smithi192.stdout:(153/157): NetworkManager-team-1.51.0-1.el9.x86 129 kB/s | 41 kB 00:00 2024-09-21T05:43:52.488 INFO:teuthology.orchestra.run.smithi138.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-21T05:43:52.508 INFO:teuthology.orchestra.run.smithi192.stdout:(154/157): NetworkManager-tui-1.51.0-1.el9.x86_ 1.9 MB/s | 249 kB 00:00 2024-09-21T05:43:52.535 INFO:teuthology.orchestra.run.smithi117.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 2.3 MB/s | 292 kB 00:00 2024-09-21T05:43:52.627 INFO:teuthology.orchestra.run.smithi117.stdout:(33/157): ceph-mgr-diskprediction-local-19.3.0- 17 MB/s | 7.4 MB 00:00 2024-09-21T05:43:52.652 INFO:teuthology.orchestra.run.smithi117.stdout:(34/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 423 kB/s | 49 kB 00:00 2024-09-21T05:43:52.658 INFO:teuthology.orchestra.run.smithi192.stdout:(155/157): NetworkManager-libnm-1.51.0-1.el9.x8 2.4 MB/s | 1.8 MB 00:00 2024-09-21T05:43:52.678 INFO:teuthology.orchestra.run.smithi117.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 329 kB/s | 16 kB 00:00 2024-09-21T05:43:52.688 INFO:teuthology.orchestra.run.smithi138.stdout:(72/157): openblas-openmp-0.3.26-2.el9.x86_64.r 3.6 MB/s | 4.9 MB 00:01 2024-09-21T05:43:52.725 INFO:teuthology.orchestra.run.smithi192.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-09-21T05:43:52.728 INFO:teuthology.orchestra.run.smithi117.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 3.7 MB/s | 285 kB 00:00 2024-09-21T05:43:52.747 INFO:teuthology.orchestra.run.smithi138.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-21T05:43:52.761 INFO:teuthology.orchestra.run.smithi117.stdout:(37/157): cephadm-19.3.0-5120.g18432218.el9.noa 9.2 MB/s | 784 kB 00:00 2024-09-21T05:43:52.787 INFO:teuthology.orchestra.run.smithi117.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 313 kB/s | 18 kB 00:00 2024-09-21T05:43:52.856 INFO:teuthology.orchestra.run.smithi138.stdout:(74/157): protobuf-3.14.0-13.el9.x86_64.rpm 2.7 MB/s | 1.0 MB 00:00 2024-09-21T05:43:52.892 INFO:teuthology.orchestra.run.smithi192.stdout:(157/157): openssl-3.2.2-6.el9.x86_64.rpm 2.8 MB/s | 1.4 MB 00:00 2024-09-21T05:43:52.895 INFO:teuthology.orchestra.run.smithi117.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 598 kB/s | 80 kB 00:00 2024-09-21T05:43:52.897 INFO:teuthology.orchestra.run.smithi138.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-21T05:43:52.899 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-21T05:43:52.899 INFO:teuthology.orchestra.run.smithi192.stdout:Total 23 MB/s | 228 MB 00:10 2024-09-21T05:43:52.962 INFO:teuthology.orchestra.run.smithi117.stdout:(40/157): libquadmath-11.5.0-2.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-21T05:43:52.981 INFO:teuthology.orchestra.run.smithi138.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-09-21T05:43:52.987 INFO:teuthology.orchestra.run.smithi117.stdout:(41/157): libgfortran-11.5.0-2.el9.x86_64.rpm 3.9 MB/s | 801 kB 00:00 2024-09-21T05:43:53.013 INFO:teuthology.orchestra.run.smithi117.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-09-21T05:43:53.055 INFO:teuthology.orchestra.run.smithi117.stdout:(43/157): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-09-21T05:43:53.073 INFO:teuthology.orchestra.run.smithi138.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-09-21T05:43:53.080 INFO:teuthology.orchestra.run.smithi117.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-09-21T05:43:53.105 INFO:teuthology.orchestra.run.smithi117.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-21T05:43:53.115 INFO:teuthology.orchestra.run.smithi138.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 616 kB/s | 26 kB 00:00 2024-09-21T05:43:53.131 INFO:teuthology.orchestra.run.smithi117.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-21T05:43:53.157 INFO:teuthology.orchestra.run.smithi138.stdout:(79/157): python3-jsonpointer-2.0-4.el9.noarch. 460 kB/s | 19 kB 00:00 2024-09-21T05:43:53.233 INFO:teuthology.orchestra.run.smithi138.stdout:(80/157): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-09-21T05:43:53.275 INFO:teuthology.orchestra.run.smithi138.stdout:(81/157): python3-markupsafe-1.1.1-12.el9.x86_6 836 kB/s | 35 kB 00:00 2024-09-21T05:43:53.298 INFO:teuthology.orchestra.run.smithi117.stdout:(47/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 3.3 MB/s | 18 MB 00:05 2024-09-21T05:43:53.323 INFO:teuthology.orchestra.run.smithi117.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 989 kB/s | 215 kB 00:00 2024-09-21T05:43:53.990 INFO:teuthology.orchestra.run.smithi117.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 160 kB/s | 107 kB 00:00 2024-09-21T05:43:54.150 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-21T05:43:54.175 INFO:teuthology.orchestra.run.smithi138.stdout:(82/157): postgresql-server-13.16-1.el9.x86_64. 3.4 MB/s | 5.8 MB 00:01 2024-09-21T05:43:54.182 INFO:teuthology.orchestra.run.smithi117.stdout:(50/157): automake-1.16.2-8.el9.noarch.rpm 656 kB/s | 689 kB 00:01 2024-09-21T05:43:54.284 INFO:teuthology.orchestra.run.smithi138.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-09-21T05:43:54.304 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-21T05:43:54.305 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-21T05:43:54.307 INFO:teuthology.orchestra.run.smithi117.stdout:(51/157): bison-3.7.4-5.el9.x86_64.rpm 935 kB/s | 943 kB 00:01 2024-09-21T05:43:54.367 INFO:teuthology.orchestra.run.smithi138.stdout:(84/157): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-09-21T05:43:54.426 INFO:teuthology.orchestra.run.smithi138.stdout:(85/157): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-21T05:43:54.468 INFO:teuthology.orchestra.run.smithi138.stdout:(86/157): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-21T05:43:54.560 INFO:teuthology.orchestra.run.smithi138.stdout:(87/157): python3-babel-2.9.1-2.el9.noarch.rpm 3.3 MB/s | 6.0 MB 00:01 2024-09-21T05:43:54.574 INFO:teuthology.orchestra.run.smithi117.stdout:(52/157): elfutils-libelf-devel-0.191-4.el9.x86 37 kB/s | 22 kB 00:00 2024-09-21T05:43:54.619 INFO:teuthology.orchestra.run.smithi138.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 1.8 MB/s | 269 kB 00:00 2024-09-21T05:43:54.661 INFO:teuthology.orchestra.run.smithi138.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 159 kB 00:00 2024-09-21T05:43:54.703 INFO:teuthology.orchestra.run.smithi138.stdout:(90/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-21T05:43:54.745 INFO:teuthology.orchestra.run.smithi138.stdout:(91/157): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 54 kB 00:00 2024-09-21T05:43:54.770 INFO:teuthology.orchestra.run.smithi138.stdout:(92/157): python3-pyasn1-modules-0.4.8-6.el9.no 1.8 MB/s | 279 kB 00:00 2024-09-21T05:43:54.812 INFO:teuthology.orchestra.run.smithi138.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-21T05:43:54.854 INFO:teuthology.orchestra.run.smithi138.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-09-21T05:43:54.874 INFO:teuthology.orchestra.run.smithi117.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 58 kB/s | 33 kB 00:00 2024-09-21T05:43:54.930 INFO:teuthology.orchestra.run.smithi138.stdout:(95/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.1 MB/s | 5.1 MB 00:01 2024-09-21T05:43:54.958 INFO:teuthology.orchestra.run.smithi117.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 404 kB/s | 313 kB 00:00 2024-09-21T05:43:54.972 INFO:teuthology.orchestra.run.smithi138.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 861 kB/s | 36 kB 00:00 2024-09-21T05:43:54.997 INFO:teuthology.orchestra.run.smithi138.stdout:(97/157): qatlib-24.02.0-1.el9.x86_64.rpm 1.5 MB/s | 221 kB 00:00 2024-09-21T05:43:55.089 INFO:teuthology.orchestra.run.smithi138.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 406 kB/s | 47 kB 00:00 2024-09-21T05:43:55.140 INFO:teuthology.orchestra.run.smithi138.stdout:(99/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-21T05:43:55.182 INFO:teuthology.orchestra.run.smithi138.stdout:(100/157): socat-1.7.4.1-6.el9.x86_64.rpm 1.6 MB/s | 304 kB 00:00 2024-09-21T05:43:55.223 INFO:teuthology.orchestra.run.smithi138.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.1 MB/s | 96 kB 00:00 2024-09-21T05:43:55.266 INFO:teuthology.orchestra.run.smithi138.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 762 kB/s | 64 kB 00:00 2024-09-21T05:43:55.291 INFO:teuthology.orchestra.run.smithi138.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 666 kB/s | 45 kB 00:00 2024-09-21T05:43:55.375 INFO:teuthology.orchestra.run.smithi117.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 36 kB/s | 18 kB 00:00 2024-09-21T05:43:55.425 INFO:teuthology.orchestra.run.smithi117.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 24 kB/s | 11 kB 00:00 2024-09-21T05:43:55.667 INFO:teuthology.orchestra.run.smithi117.stdout:(57/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.7 MB/s | 3.0 MB 00:01 2024-09-21T05:43:55.858 INFO:teuthology.orchestra.run.smithi138.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 38 kB/s | 22 kB 00:00 2024-09-21T05:43:55.926 INFO:teuthology.orchestra.run.smithi138.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 8.0 MB/s | 548 kB 00:00 2024-09-21T05:43:55.959 INFO:teuthology.orchestra.run.smithi138.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 9.1 MB/s | 308 kB 00:00 2024-09-21T05:43:55.993 INFO:teuthology.orchestra.run.smithi138.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 584 kB/s | 19 kB 00:00 2024-09-21T05:43:56.092 INFO:teuthology.orchestra.run.smithi117.stdout:(58/157): libnbd-1.20.2-2.el9.x86_64.rpm 238 kB/s | 171 kB 00:00 2024-09-21T05:43:56.118 INFO:teuthology.orchestra.run.smithi138.stdout:(108/157): libarrow-9.0.0-11.el9.x86_64.rpm 35 MB/s | 4.4 MB 00:00 2024-09-21T05:43:56.120 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-21T05:43:56.120 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-21T05:43:56.143 INFO:teuthology.orchestra.run.smithi138.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-21T05:43:56.159 INFO:teuthology.orchestra.run.smithi117.stdout:(59/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 218 kB/s | 160 kB 00:00 2024-09-21T05:43:56.169 INFO:teuthology.orchestra.run.smithi138.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-21T05:43:56.211 INFO:teuthology.orchestra.run.smithi138.stdout:(111/157): protobuf-compiler-3.14.0-13.el9.x86_ 939 kB/s | 863 kB 00:00 2024-09-21T05:43:56.236 INFO:teuthology.orchestra.run.smithi138.stdout:(112/157): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-09-21T05:43:56.242 INFO:teuthology.orchestra.run.smithi117.stdout:(60/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 80 kB/s | 45 kB 00:00 2024-09-21T05:43:56.287 INFO:teuthology.orchestra.run.smithi138.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 19 MB/s | 839 kB 00:00 2024-09-21T05:43:56.312 INFO:teuthology.orchestra.run.smithi138.stdout:(114/157): luarocks-3.9.2-5.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-09-21T05:43:56.337 INFO:teuthology.orchestra.run.smithi138.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T05:43:56.363 INFO:teuthology.orchestra.run.smithi138.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T05:43:56.388 INFO:teuthology.orchestra.run.smithi138.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 561 kB/s | 14 kB 00:00 2024-09-21T05:43:56.422 INFO:teuthology.orchestra.run.smithi138.stdout:(118/157): python3-asyncssh-2.13.2-5.el9.noarch 4.0 MB/s | 548 kB 00:00 2024-09-21T05:43:56.447 INFO:teuthology.orchestra.run.smithi138.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-21T05:43:56.472 INFO:teuthology.orchestra.run.smithi138.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-09-21T05:43:56.506 INFO:teuthology.orchestra.run.smithi138.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-21T05:43:56.531 INFO:teuthology.orchestra.run.smithi138.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-21T05:43:56.557 INFO:teuthology.orchestra.run.smithi138.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-21T05:43:56.582 INFO:teuthology.orchestra.run.smithi138.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 427 kB/s | 11 kB 00:00 2024-09-21T05:43:56.608 INFO:teuthology.orchestra.run.smithi138.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 707 kB/s | 18 kB 00:00 2024-09-21T05:43:56.633 INFO:teuthology.orchestra.run.smithi138.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 927 kB/s | 23 kB 00:00 2024-09-21T05:43:56.667 INFO:teuthology.orchestra.run.smithi138.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 578 kB/s | 19 kB 00:00 2024-09-21T05:43:56.692 INFO:teuthology.orchestra.run.smithi138.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 786 kB/s | 20 kB 00:00 2024-09-21T05:43:56.743 INFO:teuthology.orchestra.run.smithi138.stdout:(129/157): python3-grpcio-1.46.7-10.el9.x86_64. 8.6 MB/s | 2.0 MB 00:00 2024-09-21T05:43:56.768 INFO:teuthology.orchestra.run.smithi138.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T05:43:56.793 INFO:teuthology.orchestra.run.smithi117.stdout:(61/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 44 kB/s | 24 kB 00:00 2024-09-21T05:43:56.793 INFO:teuthology.orchestra.run.smithi138.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T05:43:56.819 INFO:teuthology.orchestra.run.smithi138.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T05:43:56.853 INFO:teuthology.orchestra.run.smithi138.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-09-21T05:43:56.886 INFO:teuthology.orchestra.run.smithi138.stdout:(134/157): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-09-21T05:43:56.912 INFO:teuthology.orchestra.run.smithi138.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-09-21T05:43:56.937 INFO:teuthology.orchestra.run.smithi138.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-21T05:43:56.962 INFO:teuthology.orchestra.run.smithi138.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-21T05:43:56.968 INFO:teuthology.orchestra.run.smithi117.stdout:(62/157): librdkafka-1.6.1-102.el9.x86_64.rpm 757 kB/s | 662 kB 00:00 2024-09-21T05:43:56.988 INFO:teuthology.orchestra.run.smithi138.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 610 kB/s | 31 kB 00:00 2024-09-21T05:43:57.010 INFO:teuthology.orchestra.run.smithi117.stdout:(63/157): libtool-2.4.6-46.el9.x86_64.rpm 683 kB/s | 581 kB 00:00 2024-09-21T05:43:57.013 INFO:teuthology.orchestra.run.smithi138.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-21T05:43:57.038 INFO:teuthology.orchestra.run.smithi138.stdout:(140/157): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-21T05:43:57.064 INFO:teuthology.orchestra.run.smithi138.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 708 kB/s | 36 kB 00:00 2024-09-21T05:43:57.081 INFO:teuthology.orchestra.run.smithi138.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.7 MB/s | 74 kB 00:00 2024-09-21T05:43:57.106 INFO:teuthology.orchestra.run.smithi138.stdout:(143/157): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-21T05:43:57.131 INFO:teuthology.orchestra.run.smithi138.stdout:(144/157): python3-webob-1.8.8-2.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-09-21T05:43:57.157 INFO:teuthology.orchestra.run.smithi138.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 799 kB/s | 20 kB 00:00 2024-09-21T05:43:57.182 INFO:teuthology.orchestra.run.smithi138.stdout:(146/157): re2-20211101-20.el9.x86_64.rpm 7.4 MB/s | 191 kB 00:00 2024-09-21T05:43:57.208 INFO:teuthology.orchestra.run.smithi138.stdout:(147/157): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-09-21T05:43:57.258 INFO:teuthology.orchestra.run.smithi138.stdout:(148/157): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-21T05:43:57.402 INFO:teuthology.orchestra.run.smithi117.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 78 kB/s | 47 kB 00:00 2024-09-21T05:43:57.492 INFO:teuthology.orchestra.run.smithi138.stdout:(149/157): librados2-19.3.0-5120.g18432218.el9. 12 MB/s | 3.4 MB 00:00 2024-09-21T05:43:57.559 INFO:teuthology.orchestra.run.smithi138.stdout:(150/157): librbd1-19.3.0-5120.g18432218.el9.x8 11 MB/s | 3.2 MB 00:00 2024-09-21T05:43:57.727 INFO:teuthology.orchestra.run.smithi117.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 263 kB/s | 188 kB 00:00 2024-09-21T05:43:57.801 INFO:teuthology.orchestra.run.smithi138.stdout:(151/157): NetworkManager-1.51.0-1.el9.x86_64.r 7.5 MB/s | 2.3 MB 00:00 2024-09-21T05:43:57.810 INFO:teuthology.orchestra.run.smithi117.stdout:(66/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 347 kB/s | 292 kB 00:00 2024-09-21T05:43:57.843 INFO:teuthology.orchestra.run.smithi138.stdout:(152/157): NetworkManager-libnm-1.51.0-1.el9.x8 6.4 MB/s | 1.8 MB 00:00 2024-09-21T05:43:57.860 INFO:teuthology.orchestra.run.smithi138.stdout:(153/157): NetworkManager-team-1.51.0-1.el9.x86 699 kB/s | 41 kB 00:00 2024-09-21T05:43:57.894 INFO:teuthology.orchestra.run.smithi138.stdout:(154/157): NetworkManager-tui-1.51.0-1.el9.x86_ 4.8 MB/s | 249 kB 00:00 2024-09-21T05:43:57.936 INFO:teuthology.orchestra.run.smithi138.stdout:(155/157): openssl-3.2.2-6.el9.x86_64.rpm 18 MB/s | 1.4 MB 00:00 2024-09-21T05:43:57.961 INFO:teuthology.orchestra.run.smithi117.stdout:(67/157): openblas-0.3.26-2.el9.x86_64.rpm 69 kB/s | 38 kB 00:00 2024-09-21T05:43:58.003 INFO:teuthology.orchestra.run.smithi138.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-09-21T05:43:58.861 INFO:teuthology.orchestra.run.smithi117.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 4.4 MB/s | 4.9 MB 00:01 2024-09-21T05:43:58.998 INFO:teuthology.orchestra.run.smithi117.stdout:(69/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.7 MB/s | 4.4 MB 00:01 2024-09-21T05:43:59.053 INFO:teuthology.orchestra.run.smithi117.stdout:(70/157): postgresql-13.16-1.el9.x86_64.rpm 1.5 MB/s | 1.6 MB 00:01 2024-09-21T05:43:59.695 INFO:teuthology.orchestra.run.smithi117.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 197 kB/s | 137 kB 00:00 2024-09-21T05:43:59.820 INFO:teuthology.orchestra.run.smithi117.stdout:(72/157): postgresql-contrib-13.16-1.el9.x86_64 949 kB/s | 910 kB 00:00 2024-09-21T05:44:00.045 INFO:teuthology.orchestra.run.smithi138.stdout:(157/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.1 MB/s | 16 MB 00:05 2024-09-21T05:44:00.051 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-09-21T05:44:00.051 INFO:teuthology.orchestra.run.smithi138.stdout:Total 14 MB/s | 228 MB 00:15 2024-09-21T05:44:00.187 INFO:teuthology.orchestra.run.smithi117.stdout:(73/157): postgresql-server-13.16-1.el9.x86_64. 5.1 MB/s | 5.8 MB 00:01 2024-09-21T05:44:00.546 INFO:teuthology.orchestra.run.smithi117.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 121 kB/s | 87 kB 00:00 2024-09-21T05:44:00.713 INFO:teuthology.orchestra.run.smithi117.stdout:(75/157): protobuf-3.14.0-13.el9.x86_64.rpm 1.0 MB/s | 1.0 MB 00:01 2024-09-21T05:44:01.230 INFO:teuthology.orchestra.run.smithi117.stdout:(76/157): python3-colorama-0.4.6-3.el9.noarch.r 90 kB/s | 61 kB 00:00 2024-09-21T05:44:01.298 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-09-21T05:44:01.338 INFO:teuthology.orchestra.run.smithi117.stdout:(77/157): python3-babel-2.9.1-2.el9.noarch.rpm 5.2 MB/s | 6.0 MB 00:01 2024-09-21T05:44:01.450 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-09-21T05:44:01.450 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-09-21T05:44:01.472 INFO:teuthology.orchestra.run.smithi117.stdout:(78/157): python3-devel-3.9.19-8.el9.x86_64.rpm 323 kB/s | 245 kB 00:00 2024-09-21T05:44:01.880 INFO:teuthology.orchestra.run.smithi117.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 47 kB/s | 26 kB 00:00 2024-09-21T05:44:01.980 INFO:teuthology.orchestra.run.smithi117.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 38 kB/s | 19 kB 00:00 2024-09-21T05:44:02.022 INFO:teuthology.orchestra.run.smithi117.stdout:(81/157): python3-jinja2-2.11.3-6.el9.noarch.rp 313 kB/s | 248 kB 00:00 2024-09-21T05:44:02.383 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-21T05:44:02.479 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T05:44:02.518 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T05:44:02.578 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T05:44:02.597 INFO:teuthology.orchestra.run.smithi117.stdout:(82/157): python3-mako-1.1.4-6.el9.noarch.rpm 240 kB/s | 172 kB 00:00 2024-09-21T05:44:02.612 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T05:44:02.614 INFO:teuthology.orchestra.run.smithi117.stdout:(83/157): python3-markupsafe-1.1.1-12.el9.x86_6 55 kB/s | 35 kB 00:00 2024-09-21T05:44:02.638 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T05:44:02.681 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T05:44:03.089 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T05:44:03.140 INFO:teuthology.orchestra.run.smithi117.stdout:(84/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.6 MB/s | 5.1 MB 00:01 2024-09-21T05:44:03.206 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T05:44:03.266 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-09-21T05:44:03.266 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-09-21T05:44:03.333 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T05:44:03.365 INFO:teuthology.orchestra.run.smithi117.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 296 kB/s | 222 kB 00:00 2024-09-21T05:44:03.365 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T05:44:03.390 INFO:teuthology.orchestra.run.smithi117.stdout:(86/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 316 kB/s | 250 kB 00:00 2024-09-21T05:44:03.458 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T05:44:03.486 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T05:44:03.515 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T05:44:03.669 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T05:44:03.717 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T05:44:03.745 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T05:44:03.849 INFO:teuthology.orchestra.run.smithi117.stdout:(87/157): python3-packaging-20.9-5.el9.noarch.r 109 kB/s | 77 kB 00:00 2024-09-21T05:44:03.852 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T05:44:03.879 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T05:44:03.932 INFO:teuthology.orchestra.run.smithi117.stdout:(88/157): python3-prettytable-0.7.2-27.el9.noar 75 kB/s | 42 kB 00:00 2024-09-21T05:44:03.942 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T05:44:03.964 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T05:44:03.990 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T05:44:04.182 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T05:44:04.207 INFO:teuthology.orchestra.run.smithi117.stdout:(89/157): python3-protobuf-3.14.0-13.el9.noarch 333 kB/s | 269 kB 00:00 2024-09-21T05:44:04.308 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T05:44:04.428 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T05:44:04.531 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T05:44:04.613 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T05:44:04.624 INFO:teuthology.orchestra.run.smithi117.stdout:(90/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 205 kB/s | 159 kB 00:00 2024-09-21T05:44:04.766 INFO:teuthology.orchestra.run.smithi117.stdout:(91/157): python3-pyasn1-modules-0.4.8-6.el9.no 334 kB/s | 279 kB 00:00 2024-09-21T05:44:04.816 INFO:teuthology.orchestra.run.smithi117.stdout:(92/157): python3-pytz-2021.1-5.el9.noarch.rpm 84 kB/s | 51 kB 00:00 2024-09-21T05:44:04.849 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T05:44:04.964 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T05:44:04.993 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T05:44:05.031 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T05:44:05.105 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T05:44:05.157 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T05:44:05.197 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T05:44:05.225 INFO:teuthology.orchestra.run.smithi117.stdout:(93/157): python3-requests-oauthlib-1.3.0-12.el 89 kB/s | 54 kB 00:00 2024-09-21T05:44:05.240 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T05:44:05.273 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T05:44:05.301 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T05:44:05.370 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T05:44:05.375 INFO:teuthology.orchestra.run.smithi117.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 74 kB/s | 42 kB 00:00 2024-09-21T05:44:05.423 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T05:44:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T05:44:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T05:44:05.571 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T05:44:05.608 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T05:44:05.643 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T05:44:05.672 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T05:44:05.742 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T05:44:05.786 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T05:44:05.800 INFO:teuthology.orchestra.run.smithi117.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 77 kB/s | 44 kB 00:00 2024-09-21T05:44:05.861 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T05:44:05.995 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T05:44:06.047 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T05:44:06.226 INFO:teuthology.orchestra.run.smithi117.stdout:(96/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-21T05:44:06.251 INFO:teuthology.orchestra.run.smithi117.stdout:(97/157): qatlib-24.02.0-1.el9.x86_64.rpm 252 kB/s | 221 kB 00:00 2024-09-21T05:44:06.493 INFO:teuthology.orchestra.run.smithi117.stdout:(98/157): qatlib-service-24.02.0-1.el9.x86_64.r 52 kB/s | 36 kB 00:00 2024-09-21T05:44:06.568 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T05:44:06.650 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T05:44:06.680 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T05:44:06.722 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T05:44:06.760 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T05:44:06.801 INFO:teuthology.orchestra.run.smithi117.stdout:(99/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 82 kB/s | 47 kB 00:00 2024-09-21T05:44:06.802 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T05:44:06.964 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T05:44:07.024 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T05:44:07.027 INFO:teuthology.orchestra.run.smithi117.stdout:(100/157): socat-1.7.4.1-6.el9.x86_64.rpm 392 kB/s | 304 kB 00:00 2024-09-21T05:44:07.115 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T05:44:07.152 INFO:teuthology.orchestra.run.smithi117.stdout:(101/157): uuid-1.6.2-55.el9.x86_64.rpm 88 kB/s | 58 kB 00:00 2024-09-21T05:44:07.171 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T05:44:07.245 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T05:44:07.284 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T05:44:07.320 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T05:44:07.367 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T05:44:07.402 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T05:44:07.446 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T05:44:07.469 INFO:teuthology.orchestra.run.smithi117.stdout:(102/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 144 kB/s | 96 kB 00:00 2024-09-21T05:44:07.488 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T05:44:07.577 INFO:teuthology.orchestra.run.smithi117.stdout:(103/157): lua-devel-5.4.4-4.el9.x86_64.rpm 206 kB/s | 22 kB 00:00 2024-09-21T05:44:07.625 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T05:44:07.652 INFO:teuthology.orchestra.run.smithi117.stdout:(104/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 103 kB/s | 64 kB 00:00 2024-09-21T05:44:07.672 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T05:44:07.703 INFO:teuthology.orchestra.run.smithi117.stdout:(105/157): protobuf-compiler-3.14.0-13.el9.x86_ 6.7 MB/s | 863 kB 00:00 2024-09-21T05:44:07.714 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T05:44:07.728 INFO:teuthology.orchestra.run.smithi117.stdout:(106/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 78 kB/s | 45 kB 00:00 2024-09-21T05:44:07.770 INFO:teuthology.orchestra.run.smithi117.stdout:(107/157): gperftools-libs-2.9.1-3.el9.x86_64.r 4.5 MB/s | 308 kB 00:00 2024-09-21T05:44:07.804 INFO:teuthology.orchestra.run.smithi117.stdout:(108/157): abseil-cpp-20211102.0-3.el9.x86_64.r 3.6 MB/s | 548 kB 00:00 2024-09-21T05:44:07.816 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T05:44:07.829 INFO:teuthology.orchestra.run.smithi117.stdout:(109/157): grpc-data-1.46.7-10.el9.noarch.rpm 197 kB/s | 19 kB 00:00 2024-09-21T05:44:07.847 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T05:44:07.854 INFO:teuthology.orchestra.run.smithi117.stdout:(110/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 521 kB/s | 26 kB 00:00 2024-09-21T05:44:07.879 INFO:teuthology.orchestra.run.smithi117.stdout:(111/157): liboath-2.6.7-2.el9.x86_64.rpm 969 kB/s | 49 kB 00:00 2024-09-21T05:44:07.905 INFO:teuthology.orchestra.run.smithi117.stdout:(112/157): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-21T05:44:07.938 INFO:teuthology.orchestra.run.smithi117.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-09-21T05:44:07.989 INFO:teuthology.orchestra.run.smithi117.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 839 kB 00:00 2024-09-21T05:44:07.996 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T05:44:08.014 INFO:teuthology.orchestra.run.smithi117.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T05:44:08.021 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T05:44:08.039 INFO:teuthology.orchestra.run.smithi117.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T05:44:08.046 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T05:44:08.064 INFO:teuthology.orchestra.run.smithi117.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 566 kB/s | 14 kB 00:00 2024-09-21T05:44:08.090 INFO:teuthology.orchestra.run.smithi117.stdout:(118/157): python3-asyncssh-2.13.2-5.el9.noarch 3.5 MB/s | 548 kB 00:00 2024-09-21T05:44:08.115 INFO:teuthology.orchestra.run.smithi117.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-21T05:44:08.121 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T05:44:08.148 INFO:teuthology.orchestra.run.smithi117.stdout:(120/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-09-21T05:44:08.174 INFO:teuthology.orchestra.run.smithi117.stdout:(121/157): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-09-21T05:44:08.199 INFO:teuthology.orchestra.run.smithi117.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-21T05:44:08.224 INFO:teuthology.orchestra.run.smithi117.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-21T05:44:08.250 INFO:teuthology.orchestra.run.smithi117.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-21T05:44:08.254 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T05:44:08.283 INFO:teuthology.orchestra.run.smithi117.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 534 kB/s | 18 kB 00:00 2024-09-21T05:44:08.295 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T05:44:08.309 INFO:teuthology.orchestra.run.smithi117.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 924 kB/s | 23 kB 00:00 2024-09-21T05:44:08.322 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T05:44:08.356 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T05:44:08.375 INFO:teuthology.orchestra.run.smithi117.stdout:(127/157): libarrow-9.0.0-11.el9.x86_64.rpm 7.3 MB/s | 4.4 MB 00:00 2024-09-21T05:44:08.395 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T05:44:08.401 INFO:teuthology.orchestra.run.smithi117.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-09-21T05:44:08.426 INFO:teuthology.orchestra.run.smithi117.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-09-21T05:44:08.427 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T05:44:08.451 INFO:teuthology.orchestra.run.smithi117.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T05:44:08.457 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T05:44:08.477 INFO:teuthology.orchestra.run.smithi117.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T05:44:08.492 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T05:44:08.502 INFO:teuthology.orchestra.run.smithi117.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T05:44:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T05:44:08.544 INFO:teuthology.orchestra.run.smithi117.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-09-21T05:44:08.566 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T05:44:08.569 INFO:teuthology.orchestra.run.smithi117.stdout:(134/157): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-09-21T05:44:08.594 INFO:teuthology.orchestra.run.smithi117.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 661 kB/s | 16 kB 00:00 2024-09-21T05:44:08.601 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T05:44:08.611 INFO:teuthology.orchestra.run.smithi117.stdout:(136/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.1 MB/s | 90 kB 00:00 2024-09-21T05:44:08.637 INFO:teuthology.orchestra.run.smithi117.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 673 kB/s | 28 kB 00:00 2024-09-21T05:44:08.662 INFO:teuthology.orchestra.run.smithi117.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-21T05:44:08.687 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T05:44:08.721 INFO:teuthology.orchestra.run.smithi117.stdout:(139/157): python3-grpcio-1.46.7-10.el9.x86_64. 3.7 MB/s | 2.0 MB 00:00 2024-09-21T05:44:08.731 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T05:44:08.746 INFO:teuthology.orchestra.run.smithi117.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 708 kB/s | 59 kB 00:00 2024-09-21T05:44:08.762 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T05:44:08.771 INFO:teuthology.orchestra.run.smithi117.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-21T05:44:08.796 INFO:teuthology.orchestra.run.smithi117.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-21T05:44:08.822 INFO:teuthology.orchestra.run.smithi117.stdout:(143/157): python3-routes-2.5.1-5.el9.noarch.rp 1.0 MB/s | 188 kB 00:00 2024-09-21T05:44:08.839 INFO:teuthology.orchestra.run.smithi117.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-09-21T05:44:08.855 INFO:teuthology.orchestra.run.smithi117.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-09-21T05:44:08.865 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T05:44:08.881 INFO:teuthology.orchestra.run.smithi117.stdout:(146/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-21T05:44:08.906 INFO:teuthology.orchestra.run.smithi117.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-21T05:44:08.921 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T05:44:08.940 INFO:teuthology.orchestra.run.smithi117.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.5 MB/s | 427 kB 00:00 2024-09-21T05:44:08.982 INFO:teuthology.orchestra.run.smithi117.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-21T05:44:09.165 INFO:teuthology.orchestra.run.smithi117.stdout:(150/157): NetworkManager-1.51.0-1.el9.x86_64.r 13 MB/s | 2.3 MB 00:00 2024-09-21T05:44:09.316 INFO:teuthology.orchestra.run.smithi117.stdout:(151/157): librbd1-19.3.0-5120.g18432218.el9.x8 8.4 MB/s | 3.2 MB 00:00 2024-09-21T05:44:09.358 INFO:teuthology.orchestra.run.smithi117.stdout:(152/157): NetworkManager-libnm-1.51.0-1.el9.x8 9.4 MB/s | 1.8 MB 00:00 2024-09-21T05:44:09.391 INFO:teuthology.orchestra.run.smithi117.stdout:(153/157): NetworkManager-team-1.51.0-1.el9.x86 544 kB/s | 41 kB 00:00 2024-09-21T05:44:09.416 INFO:teuthology.orchestra.run.smithi117.stdout:(154/157): NetworkManager-tui-1.51.0-1.el9.x86_ 4.2 MB/s | 249 kB 00:00 2024-09-21T05:44:09.492 INFO:teuthology.orchestra.run.smithi117.stdout:(155/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 32 MB/s | 2.4 MB 00:00 2024-09-21T05:44:09.558 INFO:teuthology.orchestra.run.smithi117.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 8.1 MB/s | 1.4 MB 00:00 2024-09-21T05:44:09.634 INFO:teuthology.orchestra.run.smithi117.stdout:(157/157): librados2-19.3.0-5120.g18432218.el9. 4.7 MB/s | 3.4 MB 00:00 2024-09-21T05:44:09.641 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-09-21T05:44:09.641 INFO:teuthology.orchestra.run.smithi117.stdout:Total 9.5 MB/s | 228 MB 00:23 2024-09-21T05:44:09.644 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T05:44:09.685 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T05:44:09.719 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T05:44:09.745 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-09-21T05:44:09.818 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T05:44:09.848 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T05:44:09.865 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T05:44:09.886 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T05:44:09.910 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T05:44:09.948 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T05:44:09.982 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T05:44:09.997 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T05:44:10.009 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T05:44:10.029 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T05:44:10.051 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T05:44:10.213 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T05:44:10.243 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T05:44:10.321 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T05:44:10.442 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T05:44:10.502 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T05:44:10.628 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T05:44:10.653 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T05:44:10.737 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T05:44:10.766 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T05:44:10.796 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T05:44:10.836 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-09-21T05:44:10.953 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T05:44:10.955 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T05:44:10.990 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-09-21T05:44:10.990 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-09-21T05:44:11.007 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T05:44:11.035 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T05:44:11.058 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T05:44:11.132 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T05:44:11.161 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T05:44:11.224 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T05:44:11.246 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T05:44:11.272 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T05:44:11.448 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T05:44:11.582 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T05:44:11.754 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T05:44:11.849 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T05:44:11.915 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T05:44:12.150 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T05:44:12.259 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T05:44:12.286 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T05:44:12.289 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T05:44:12.322 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T05:44:12.389 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T05:44:12.404 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T05:44:12.425 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T05:44:12.457 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T05:44:12.458 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T05:44:12.496 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T05:44:12.540 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T05:44:12.581 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T05:44:12.609 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T05:44:12.679 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T05:44:12.728 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T05:44:12.758 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T05:44:12.782 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T05:44:12.794 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T05:44:12.821 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T05:44:12.876 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T05:44:12.913 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T05:44:12.913 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T05:44:12.942 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T05:44:12.957 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T05:44:12.978 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T05:44:12.985 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T05:44:13.055 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T05:44:13.100 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T05:44:13.174 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T05:44:13.308 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T05:44:13.317 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-09-21T05:44:13.317 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-09-21T05:44:13.370 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T05:44:13.692 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T05:44:13.724 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T05:44:13.795 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T05:44:13.825 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T05:44:13.922 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T05:44:14.008 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T05:44:14.046 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T05:44:14.088 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T05:44:14.127 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T05:44:14.168 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T05:44:14.325 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T05:44:14.389 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T05:44:14.480 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T05:44:14.554 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T05:44:14.627 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T05:44:14.658 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T05:44:14.692 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T05:44:14.739 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T05:44:14.775 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T05:44:14.818 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T05:44:14.860 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T05:44:14.998 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T05:44:15.038 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T05:44:15.080 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T05:44:15.182 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T05:44:15.214 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T05:44:15.254 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T05:44:15.280 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T05:44:15.304 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T05:44:15.378 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T05:44:15.507 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T05:44:15.746 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T05:44:15.817 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T05:44:15.852 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T05:44:15.892 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T05:44:15.931 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T05:44:15.962 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T05:44:15.987 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T05:44:16.021 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T05:44:16.063 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T05:44:16.097 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T05:44:16.120 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T05:44:16.148 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T05:44:16.187 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T05:44:16.236 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T05:44:16.267 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T05:44:16.371 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T05:44:16.426 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T05:44:16.715 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T05:44:16.752 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T05:44:16.857 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T05:44:16.857 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T05:44:16.857 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T05:44:16.857 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-21T05:44:16.895 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T05:44:17.150 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T05:44:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T05:44:17.225 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T05:44:17.323 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T05:44:17.373 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T05:44:17.426 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T05:44:17.514 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T05:44:17.562 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T05:44:17.745 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T05:44:17.777 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T05:44:17.853 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T05:44:18.501 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T05:44:18.619 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T05:44:19.849 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T05:44:19.884 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-09-21T05:44:19.936 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T05:44:19.980 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T05:44:20.004 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T05:44:20.013 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T05:44:20.043 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T05:44:20.096 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T05:44:20.138 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T05:44:20.163 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T05:44:20.218 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T05:44:20.338 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T05:44:20.385 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T05:44:20.476 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T05:44:20.499 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T05:44:20.534 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T05:44:20.555 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T05:44:20.672 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T05:44:20.782 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T05:44:20.808 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T05:44:20.891 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T05:44:20.921 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T05:44:20.951 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T05:44:21.031 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T05:44:21.064 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T05:44:21.128 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T05:44:21.135 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T05:44:21.166 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T05:44:21.185 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T05:44:21.213 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T05:44:21.302 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T05:44:21.330 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T05:44:21.394 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T05:44:21.423 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T05:44:21.442 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T05:44:21.625 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T05:44:21.806 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T05:44:21.930 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T05:44:22.040 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T05:44:22.107 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T05:44:22.334 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T05:44:22.436 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T05:44:22.473 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T05:44:22.512 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T05:44:22.585 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T05:44:22.638 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T05:44:22.671 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T05:44:22.713 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T05:44:22.747 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T05:44:22.774 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T05:44:22.843 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T05:44:22.897 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T05:44:22.927 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T05:44:22.971 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T05:44:23.059 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T05:44:23.262 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T05:44:23.262 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T05:44:23.262 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T05:44:23.262 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T05:44:23.293 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T05:44:23.368 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T05:44:23.445 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T05:44:23.480 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T05:44:23.521 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T05:44:23.571 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T05:44:24.045 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T05:44:24.093 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T05:44:24.106 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T05:44:24.167 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T05:44:24.167 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T05:44:24.167 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T05:44:24.167 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-21T05:44:24.186 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T05:44:24.191 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T05:44:24.220 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T05:44:24.263 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T05:44:24.318 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T05:44:24.361 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T05:44:24.534 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T05:44:24.612 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T05:44:24.711 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T05:44:24.784 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T05:44:24.858 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T05:44:24.889 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T05:44:24.925 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T05:44:24.972 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T05:44:25.017 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T05:44:25.076 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T05:44:25.128 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T05:44:25.263 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T05:44:25.303 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T05:44:25.346 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T05:44:25.448 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T05:44:25.479 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T05:44:25.519 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T05:44:25.552 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T05:44:25.577 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T05:44:25.665 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T05:44:25.787 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T05:44:25.826 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T05:44:25.859 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T05:44:25.897 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T05:44:25.936 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T05:44:25.967 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T05:44:25.997 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T05:44:26.032 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T05:44:26.065 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T05:44:26.107 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T05:44:26.144 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T05:44:26.230 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T05:44:26.274 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T05:44:26.314 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T05:44:26.425 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T05:44:26.490 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T05:44:27.205 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T05:44:27.245 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T05:44:27.279 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T05:44:27.378 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T05:44:27.434 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T05:44:27.479 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T05:44:27.567 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T05:44:27.607 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T05:44:27.791 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T05:44:27.821 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T05:44:27.899 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T05:44:28.546 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T05:44:28.654 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T05:44:28.708 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T05:44:28.709 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /sys 2024-09-21T05:44:28.709 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /proc 2024-09-21T05:44:28.709 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /mnt 2024-09-21T05:44:28.709 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /var/tmp 2024-09-21T05:44:28.709 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /home 2024-09-21T05:44:28.709 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /root 2024-09-21T05:44:28.709 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /tmp 2024-09-21T05:44:28.709 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-21T05:44:28.888 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T05:44:28.904 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T05:44:28.904 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:44:28.905 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T05:44:28.905 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-21T05:44:29.910 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T05:44:30.010 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T05:44:30.038 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T05:44:30.071 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T05:44:30.404 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T05:44:30.451 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T05:44:30.492 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T05:44:30.519 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T05:44:30.519 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:44:30.519 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T05:44:30.519 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T05:44:30.519 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T05:44:30.519 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-21T05:44:30.542 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T05:44:30.572 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T05:44:30.610 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T05:44:30.742 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T05:44:30.766 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T05:44:30.766 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:44:30.766 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T05:44:30.766 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T05:44:30.766 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T05:44:30.766 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-21T05:44:31.106 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T05:44:31.139 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T05:44:31.209 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T05:44:31.227 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T05:44:31.240 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T05:44:31.250 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T05:44:31.251 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:44:31.251 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T05:44:31.251 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T05:44:31.251 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T05:44:31.251 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-21T05:44:31.357 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T05:44:31.414 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T05:44:31.469 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T05:44:31.655 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T05:44:31.702 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T05:44:31.781 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T05:44:31.886 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T05:44:32.025 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T05:44:32.069 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T05:44:32.101 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T05:44:32.129 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T05:44:32.549 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T05:44:32.577 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T05:44:32.609 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T05:44:32.675 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T05:44:32.700 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T05:44:33.511 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T05:44:33.544 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T05:44:34.123 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T05:44:34.164 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T05:44:34.254 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T05:44:34.254 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T05:44:34.254 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T05:44:34.254 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:44:34.307 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T05:44:35.920 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T05:44:35.981 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T05:44:36.252 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T05:44:36.252 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /sys 2024-09-21T05:44:36.253 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /proc 2024-09-21T05:44:36.253 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /mnt 2024-09-21T05:44:36.253 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /var/tmp 2024-09-21T05:44:36.253 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /home 2024-09-21T05:44:36.253 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /root 2024-09-21T05:44:36.253 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /tmp 2024-09-21T05:44:36.253 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-21T05:44:36.425 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T05:44:36.441 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T05:44:36.442 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:44:36.442 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T05:44:36.442 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-21T05:44:36.943 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T05:44:36.965 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T05:44:37.081 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T05:44:37.240 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T05:44:37.275 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T05:44:37.300 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T05:44:37.300 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:44:37.300 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T05:44:37.300 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T05:44:37.300 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T05:44:37.300 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-21T05:44:37.350 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T05:44:37.363 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T05:44:37.462 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T05:44:37.516 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T05:44:38.028 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T05:44:38.055 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T05:44:38.055 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:44:38.055 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T05:44:38.056 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T05:44:38.056 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T05:44:38.056 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-21T05:44:38.279 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T05:44:38.303 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T05:44:38.303 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:44:38.303 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T05:44:38.303 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T05:44:38.303 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T05:44:38.303 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-21T05:44:38.476 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T05:44:38.503 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T05:44:38.527 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T05:44:38.527 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:44:38.527 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T05:44:38.527 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T05:44:38.527 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T05:44:38.527 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-21T05:44:38.570 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T05:44:38.593 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T05:44:38.593 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:44:38.593 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T05:44:38.593 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-21T05:44:38.646 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T05:44:38.670 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T05:44:38.670 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:44:38.670 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T05:44:38.670 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T05:44:38.670 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T05:44:38.670 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-21T05:44:38.772 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T05:44:38.796 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T05:44:38.796 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:44:38.796 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T05:44:38.796 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T05:44:38.796 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T05:44:38.796 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-21T05:44:38.894 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T05:44:38.951 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T05:44:38.997 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T05:44:39.006 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T05:44:39.021 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T05:44:39.021 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:44:39.021 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T05:44:39.021 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T05:44:39.021 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T05:44:39.021 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-21T05:44:39.191 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T05:44:39.240 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T05:44:39.311 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T05:44:39.399 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T05:44:39.547 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T05:44:39.583 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T05:44:39.615 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T05:44:39.643 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T05:44:40.046 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T05:44:40.076 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T05:44:40.108 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T05:44:40.175 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T05:44:40.199 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T05:44:42.993 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T05:44:43.034 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T05:44:43.075 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T05:44:43.107 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T05:44:43.133 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T05:44:43.323 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T05:44:43.364 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T05:44:43.367 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T05:44:43.394 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T05:44:43.423 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T05:44:43.432 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T05:44:43.470 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T05:44:44.042 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T05:44:44.084 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T05:44:44.105 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T05:44:44.388 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T05:44:44.416 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T05:44:44.530 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T05:44:44.699 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T05:44:44.726 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T05:44:44.751 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T05:44:44.751 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:44:44.751 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T05:44:44.751 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T05:44:44.751 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T05:44:44.752 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-21T05:44:44.802 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T05:44:44.814 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T05:44:44.913 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T05:44:44.944 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T05:44:44.958 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T05:44:45.022 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T05:44:45.063 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T05:44:45.145 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T05:44:45.175 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T05:44:45.200 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T05:44:45.234 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T05:44:45.234 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T05:44:45.310 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T05:44:45.310 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T05:44:45.395 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T05:44:45.419 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T05:44:45.426 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T05:44:45.426 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T05:44:45.604 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T05:44:45.604 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T05:44:45.689 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T05:44:45.760 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T05:44:45.845 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T05:44:45.845 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /sys 2024-09-21T05:44:45.845 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /proc 2024-09-21T05:44:45.845 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /mnt 2024-09-21T05:44:45.845 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /var/tmp 2024-09-21T05:44:45.845 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /home 2024-09-21T05:44:45.845 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /root 2024-09-21T05:44:45.845 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /tmp 2024-09-21T05:44:45.845 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:44:45.919 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T05:44:45.956 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T05:44:45.980 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T05:44:45.980 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:44:45.980 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T05:44:45.980 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T05:44:45.980 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T05:44:45.980 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-21T05:44:46.019 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T05:44:46.023 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T05:44:46.035 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T05:44:46.035 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:44:46.035 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T05:44:46.035 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:44:46.047 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T05:44:46.047 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:44:46.047 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T05:44:46.047 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-21T05:44:46.099 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T05:44:46.122 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T05:44:46.122 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:44:46.123 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T05:44:46.123 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T05:44:46.123 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T05:44:46.123 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-21T05:44:46.441 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T05:44:46.465 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T05:44:46.465 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:44:46.465 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T05:44:46.465 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T05:44:46.466 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T05:44:46.466 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-21T05:44:47.638 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T05:44:47.664 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T05:44:47.664 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:44:47.665 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T05:44:47.665 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T05:44:47.665 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T05:44:47.665 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:44:47.889 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T05:44:47.912 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T05:44:47.912 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:44:47.912 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T05:44:47.912 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T05:44:47.912 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T05:44:47.913 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:44:48.381 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T05:44:48.406 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T05:44:48.406 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:44:48.406 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T05:44:48.406 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T05:44:48.406 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T05:44:48.407 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:44:48.512 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T05:44:48.569 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T05:44:48.624 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T05:44:48.810 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T05:44:48.865 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T05:44:48.944 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T05:44:49.041 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T05:44:49.192 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T05:44:49.232 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T05:44:49.264 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T05:44:49.292 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T05:44:49.768 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T05:44:49.799 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T05:44:49.831 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T05:44:49.898 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T05:44:49.922 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T05:44:50.430 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T05:44:50.471 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T05:44:50.512 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T05:44:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T05:44:50.572 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T05:44:50.753 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T05:44:50.804 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T05:44:50.834 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T05:44:50.864 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T05:44:50.910 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T05:44:51.531 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T05:44:51.568 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T05:44:51.589 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T05:44:52.330 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T05:44:52.331 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T05:44:52.331 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T05:44:52.331 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T05:44:52.331 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T05:44:52.331 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T05:44:52.331 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T05:44:52.331 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T05:44:52.331 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T05:44:52.331 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T05:44:52.331 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T05:44:52.331 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T05:44:52.331 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T05:44:52.331 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T05:44:52.332 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T05:44:52.332 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T05:44:52.332 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T05:44:52.332 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T05:44:52.332 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T05:44:52.332 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T05:44:52.332 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T05:44:52.332 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T05:44:52.332 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T05:44:52.332 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T05:44:52.332 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T05:44:52.332 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T05:44:52.333 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T05:44:52.333 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T05:44:52.345 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T05:44:52.345 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T05:44:52.345 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T05:44:52.345 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T05:44:52.346 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T05:44:52.346 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T05:44:52.346 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T05:44:52.346 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T05:44:52.346 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T05:44:52.346 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T05:44:52.346 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T05:44:52.346 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T05:44:52.346 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T05:44:52.346 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T05:44:52.346 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T05:44:52.346 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T05:44:52.346 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T05:44:52.347 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T05:44:52.347 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T05:44:52.347 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T05:44:52.347 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T05:44:52.347 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T05:44:52.347 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T05:44:52.347 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T05:44:52.347 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T05:44:52.347 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T05:44:52.347 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T05:44:52.347 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T05:44:52.347 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T05:44:52.347 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T05:44:52.347 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T05:44:52.348 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T05:44:52.348 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T05:44:52.348 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T05:44:52.348 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T05:44:52.348 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T05:44:52.348 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T05:44:52.348 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T05:44:52.348 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T05:44:52.348 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T05:44:52.348 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T05:44:52.348 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T05:44:52.348 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T05:44:52.348 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T05:44:52.348 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T05:44:52.349 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T05:44:52.349 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T05:44:52.349 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T05:44:52.349 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T05:44:52.349 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T05:44:52.349 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T05:44:52.349 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T05:44:52.349 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T05:44:52.349 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T05:44:52.349 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T05:44:52.349 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T05:44:52.349 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T05:44:52.349 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T05:44:52.350 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T05:44:52.350 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T05:44:52.350 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T05:44:52.350 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T05:44:52.350 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T05:44:52.350 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T05:44:52.350 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T05:44:52.350 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T05:44:52.350 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T05:44:52.350 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T05:44:52.350 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T05:44:52.350 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T05:44:52.350 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T05:44:52.350 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T05:44:52.351 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T05:44:52.351 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T05:44:52.351 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T05:44:52.351 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T05:44:52.351 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T05:44:52.351 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T05:44:52.351 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T05:44:52.351 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T05:44:52.351 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T05:44:52.351 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T05:44:52.351 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T05:44:52.352 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T05:44:52.352 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T05:44:52.352 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T05:44:52.352 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T05:44:52.352 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T05:44:52.352 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T05:44:52.352 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T05:44:52.352 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T05:44:52.352 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T05:44:52.352 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T05:44:52.352 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T05:44:52.352 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T05:44:52.352 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T05:44:52.352 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T05:44:52.353 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T05:44:52.353 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T05:44:52.353 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T05:44:52.353 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T05:44:52.353 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T05:44:52.353 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T05:44:52.353 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T05:44:52.353 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T05:44:52.353 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T05:44:52.353 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T05:44:52.353 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T05:44:52.353 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T05:44:52.353 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T05:44:52.353 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T05:44:52.354 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T05:44:52.354 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T05:44:52.354 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T05:44:52.354 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T05:44:52.354 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T05:44:52.354 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T05:44:52.354 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T05:44:52.354 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T05:44:52.354 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T05:44:52.354 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T05:44:52.354 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T05:44:52.354 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T05:44:52.354 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T05:44:52.354 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T05:44:52.355 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T05:44:52.355 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T05:44:52.355 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T05:44:52.355 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T05:44:52.355 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T05:44:52.355 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T05:44:52.355 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T05:44:52.355 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T05:44:52.355 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T05:44:52.355 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T05:44:52.355 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T05:44:52.355 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T05:44:52.420 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T05:44:52.501 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T05:44:52.542 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T05:44:52.656 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T05:44:52.705 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T05:44:52.739 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T05:44:52.765 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T05:44:52.765 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T05:44:52.849 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T05:44:52.849 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T05:44:52.933 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T05:44:52.958 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T05:44:52.964 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T05:44:52.964 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T05:44:53.141 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T05:44:53.143 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T05:44:53.144 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T05:44:53.203 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T05:44:53.228 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T05:44:53.300 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T05:44:54.165 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T05:44:54.186 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T05:44:54.299 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T05:44:54.470 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T05:44:54.505 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T05:44:54.529 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T05:44:54.529 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:44:54.529 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T05:44:54.529 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T05:44:54.529 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T05:44:54.529 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:44:54.573 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T05:44:54.584 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T05:44:54.683 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T05:44:54.741 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T05:44:54.821 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T05:44:54.821 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-21T05:44:54.821 INFO:teuthology.orchestra.run.smithi192.stdout:Upgraded: 2024-09-21T05:44:54.821 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T05:44:54.821 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T05:44:54.821 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T05:44:54.821 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T05:44:54.821 INFO:teuthology.orchestra.run.smithi192.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:44:54.821 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:44:54.821 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T05:44:54.822 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T05:44:54.822 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-21T05:44:54.822 INFO:teuthology.orchestra.run.smithi192.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T05:44:54.822 INFO:teuthology.orchestra.run.smithi192.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T05:44:54.822 INFO:teuthology.orchestra.run.smithi192.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T05:44:54.822 INFO:teuthology.orchestra.run.smithi192.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T05:44:54.822 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:44:54.822 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:44:54.822 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:44:54.822 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:44:54.822 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:44:54.822 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:44:54.823 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:44:54.823 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:44:54.823 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:44:54.823 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:44:54.823 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:44:54.823 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:44:54.823 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:44:54.823 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:44:54.823 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:44:54.823 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:44:54.823 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:44:54.823 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:44:54.824 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:44:54.824 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:44:54.824 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:44:54.824 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:44:54.824 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:44:54.824 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:44:54.824 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T05:44:54.824 INFO:teuthology.orchestra.run.smithi192.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T05:44:54.824 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T05:44:54.824 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T05:44:54.824 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T05:44:54.824 INFO:teuthology.orchestra.run.smithi192.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T05:44:54.824 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T05:44:54.825 INFO:teuthology.orchestra.run.smithi192.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T05:44:54.825 INFO:teuthology.orchestra.run.smithi192.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T05:44:54.825 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T05:44:54.825 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T05:44:54.825 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:44:54.825 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:44:54.825 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:44:54.825 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T05:44:54.825 INFO:teuthology.orchestra.run.smithi192.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T05:44:54.825 INFO:teuthology.orchestra.run.smithi192.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T05:44:54.825 INFO:teuthology.orchestra.run.smithi192.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T05:44:54.825 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T05:44:54.825 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T05:44:54.826 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:44:54.826 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:44:54.826 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T05:44:54.826 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:44:54.826 INFO:teuthology.orchestra.run.smithi192.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T05:44:54.826 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T05:44:54.826 INFO:teuthology.orchestra.run.smithi192.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T05:44:54.826 INFO:teuthology.orchestra.run.smithi192.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T05:44:54.826 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T05:44:54.826 INFO:teuthology.orchestra.run.smithi192.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T05:44:54.826 INFO:teuthology.orchestra.run.smithi192.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T05:44:54.826 INFO:teuthology.orchestra.run.smithi192.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T05:44:54.827 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T05:44:54.827 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T05:44:54.827 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T05:44:54.827 INFO:teuthology.orchestra.run.smithi192.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T05:44:54.827 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T05:44:54.827 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T05:44:54.827 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T05:44:54.827 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T05:44:54.827 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T05:44:54.827 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T05:44:54.827 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T05:44:54.827 INFO:teuthology.orchestra.run.smithi192.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T05:44:54.827 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T05:44:54.827 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T05:44:54.828 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T05:44:54.828 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:44:54.828 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:44:54.828 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:44:54.828 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T05:44:54.828 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T05:44:54.828 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T05:44:54.828 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T05:44:54.828 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T05:44:54.828 INFO:teuthology.orchestra.run.smithi192.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T05:44:54.828 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T05:44:54.828 INFO:teuthology.orchestra.run.smithi192.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T05:44:54.828 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T05:44:54.829 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T05:44:54.829 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T05:44:54.829 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T05:44:54.829 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T05:44:54.829 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T05:44:54.829 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T05:44:54.829 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T05:44:54.829 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T05:44:54.829 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T05:44:54.829 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T05:44:54.829 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T05:44:54.829 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T05:44:54.830 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T05:44:54.830 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T05:44:54.830 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T05:44:54.830 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T05:44:54.830 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T05:44:54.830 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T05:44:54.830 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T05:44:54.830 INFO:teuthology.orchestra.run.smithi192.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T05:44:54.830 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T05:44:54.830 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T05:44:54.830 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T05:44:54.830 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T05:44:54.830 INFO:teuthology.orchestra.run.smithi192.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T05:44:54.830 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T05:44:54.831 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T05:44:54.831 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T05:44:54.831 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T05:44:54.831 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T05:44:54.831 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T05:44:54.831 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:44:54.831 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:44:54.831 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T05:44:54.831 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T05:44:54.831 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T05:44:54.831 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:44:54.831 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T05:44:54.831 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T05:44:54.831 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T05:44:54.832 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T05:44:54.832 INFO:teuthology.orchestra.run.smithi192.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T05:44:54.832 INFO:teuthology.orchestra.run.smithi192.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T05:44:54.832 INFO:teuthology.orchestra.run.smithi192.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T05:44:54.832 INFO:teuthology.orchestra.run.smithi192.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T05:44:54.832 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T05:44:54.832 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T05:44:54.832 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T05:44:54.832 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T05:44:54.832 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T05:44:54.832 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T05:44:54.832 INFO:teuthology.orchestra.run.smithi192.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T05:44:54.833 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:44:54.833 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:44:54.833 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:44:54.833 INFO:teuthology.orchestra.run.smithi192.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T05:44:54.833 INFO:teuthology.orchestra.run.smithi192.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T05:44:54.833 INFO:teuthology.orchestra.run.smithi192.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T05:44:54.833 INFO:teuthology.orchestra.run.smithi192.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T05:44:54.833 INFO:teuthology.orchestra.run.smithi192.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T05:44:54.833 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T05:44:54.833 INFO:teuthology.orchestra.run.smithi192.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T05:44:54.834 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-21T05:44:54.834 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-21T05:44:55.256 DEBUG:teuthology.parallel:result is None 2024-09-21T05:44:55.698 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T05:44:55.725 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T05:44:55.748 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T05:44:55.748 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:44:55.749 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T05:44:55.749 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T05:44:55.749 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T05:44:55.749 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:44:55.793 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T05:44:55.815 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T05:44:55.816 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:44:55.816 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T05:44:55.816 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:44:55.869 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T05:44:55.891 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T05:44:55.891 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:44:55.891 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T05:44:55.891 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T05:44:55.891 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T05:44:55.891 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:44:56.219 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T05:44:56.242 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T05:44:56.242 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:44:56.242 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T05:44:56.242 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T05:44:56.243 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T05:44:56.243 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:44:59.830 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T05:44:59.831 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T05:44:59.831 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T05:44:59.831 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T05:44:59.831 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T05:44:59.831 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T05:44:59.831 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T05:44:59.831 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T05:44:59.831 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T05:44:59.831 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T05:44:59.831 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T05:44:59.831 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T05:44:59.831 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T05:44:59.832 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T05:44:59.832 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T05:44:59.832 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T05:44:59.832 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T05:44:59.832 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T05:44:59.832 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T05:44:59.832 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T05:44:59.832 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T05:44:59.832 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T05:44:59.832 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T05:44:59.832 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T05:44:59.832 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T05:44:59.832 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T05:44:59.832 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T05:44:59.832 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T05:44:59.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T05:44:59.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T05:44:59.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T05:44:59.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T05:44:59.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T05:44:59.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T05:44:59.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T05:44:59.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T05:44:59.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T05:44:59.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T05:44:59.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T05:44:59.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T05:44:59.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T05:44:59.833 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T05:44:59.834 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T05:44:59.834 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T05:44:59.834 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T05:44:59.834 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T05:44:59.834 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T05:44:59.834 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T05:44:59.834 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T05:44:59.834 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T05:44:59.834 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T05:44:59.834 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T05:44:59.834 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T05:44:59.834 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T05:44:59.834 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T05:44:59.835 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T05:44:59.835 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T05:44:59.835 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T05:44:59.835 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T05:44:59.835 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T05:44:59.835 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T05:44:59.835 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T05:44:59.835 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T05:44:59.835 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T05:44:59.835 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T05:44:59.835 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T05:44:59.835 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T05:44:59.835 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T05:44:59.835 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T05:44:59.835 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T05:44:59.836 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T05:44:59.836 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T05:44:59.836 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T05:44:59.836 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T05:44:59.836 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T05:44:59.836 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T05:44:59.836 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T05:44:59.836 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T05:44:59.836 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T05:44:59.836 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T05:44:59.836 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T05:44:59.836 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T05:44:59.836 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T05:44:59.836 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T05:44:59.836 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T05:44:59.837 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T05:44:59.837 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T05:44:59.837 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T05:44:59.837 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T05:44:59.837 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T05:44:59.837 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T05:44:59.837 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T05:44:59.837 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T05:44:59.837 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T05:44:59.837 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T05:44:59.837 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T05:44:59.837 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T05:44:59.837 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T05:44:59.837 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T05:44:59.837 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T05:44:59.838 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T05:44:59.838 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T05:44:59.838 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T05:44:59.838 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T05:44:59.838 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T05:44:59.838 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T05:44:59.838 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T05:44:59.838 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T05:44:59.838 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T05:44:59.838 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T05:44:59.838 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T05:44:59.838 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T05:44:59.838 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T05:44:59.838 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T05:44:59.839 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T05:44:59.839 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T05:44:59.839 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T05:44:59.839 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T05:44:59.839 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T05:44:59.839 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T05:44:59.839 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T05:44:59.839 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T05:44:59.839 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T05:44:59.839 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T05:44:59.839 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T05:44:59.839 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T05:44:59.839 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T05:44:59.839 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T05:44:59.839 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T05:44:59.840 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T05:44:59.840 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T05:44:59.840 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T05:44:59.840 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T05:44:59.840 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T05:44:59.840 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T05:44:59.840 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T05:44:59.840 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T05:44:59.840 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T05:44:59.840 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T05:44:59.840 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T05:44:59.840 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T05:44:59.840 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T05:44:59.840 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T05:44:59.840 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T05:44:59.841 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T05:44:59.841 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T05:44:59.841 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T05:44:59.841 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T05:44:59.841 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T05:44:59.841 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T05:44:59.841 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T05:44:59.841 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T05:44:59.841 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T05:44:59.841 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T05:44:59.841 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T05:44:59.841 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T05:44:59.841 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T05:44:59.841 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T05:44:59.841 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T05:44:59.842 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T05:44:59.842 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T05:44:59.842 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T05:44:59.842 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T05:44:59.842 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T05:44:59.842 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T05:45:00.264 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T05:45:00.306 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T05:45:00.347 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T05:45:00.378 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T05:45:00.414 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T05:45:00.619 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T05:45:00.669 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T05:45:00.699 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T05:45:00.729 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T05:45:00.767 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T05:45:01.305 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T05:45:01.348 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T05:45:01.369 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T05:45:02.225 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T05:45:02.311 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T05:45:02.359 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T05:45:02.450 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T05:45:02.489 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T05:45:02.523 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T05:45:02.542 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T05:45:02.542 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-21T05:45:02.542 INFO:teuthology.orchestra.run.smithi138.stdout:Upgraded: 2024-09-21T05:45:02.542 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T05:45:02.543 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T05:45:02.543 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T05:45:02.543 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T05:45:02.543 INFO:teuthology.orchestra.run.smithi138.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:02.543 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:02.543 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T05:45:02.543 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T05:45:02.543 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-09-21T05:45:02.543 INFO:teuthology.orchestra.run.smithi138.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T05:45:02.543 INFO:teuthology.orchestra.run.smithi138.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T05:45:02.543 INFO:teuthology.orchestra.run.smithi138.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T05:45:02.543 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T05:45:02.543 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:02.544 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:02.544 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:02.544 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:02.544 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:45:02.544 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:02.544 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:02.544 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:02.544 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:45:02.544 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:45:02.544 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:45:02.544 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:45:02.544 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:45:02.544 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:02.545 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:02.545 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:45:02.545 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:02.545 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:02.545 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:02.545 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:45:02.545 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:45:02.545 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:02.545 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:02.545 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:45:02.545 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T05:45:02.545 INFO:teuthology.orchestra.run.smithi138.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T05:45:02.545 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T05:45:02.545 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T05:45:02.546 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T05:45:02.546 INFO:teuthology.orchestra.run.smithi138.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T05:45:02.546 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T05:45:02.546 INFO:teuthology.orchestra.run.smithi138.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T05:45:02.546 INFO:teuthology.orchestra.run.smithi138.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T05:45:02.546 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T05:45:02.546 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T05:45:02.546 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:02.546 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:02.546 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:02.546 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T05:45:02.546 INFO:teuthology.orchestra.run.smithi138.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T05:45:02.546 INFO:teuthology.orchestra.run.smithi138.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T05:45:02.547 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T05:45:02.547 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T05:45:02.547 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T05:45:02.547 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:02.547 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:02.547 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T05:45:02.547 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:02.547 INFO:teuthology.orchestra.run.smithi138.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T05:45:02.547 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T05:45:02.547 INFO:teuthology.orchestra.run.smithi138.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T05:45:02.547 INFO:teuthology.orchestra.run.smithi138.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T05:45:02.547 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T05:45:02.547 INFO:teuthology.orchestra.run.smithi138.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T05:45:02.547 INFO:teuthology.orchestra.run.smithi138.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T05:45:02.547 INFO:teuthology.orchestra.run.smithi138.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T05:45:02.548 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T05:45:02.548 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T05:45:02.548 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T05:45:02.548 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T05:45:02.548 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T05:45:02.548 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T05:45:02.548 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T05:45:02.548 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T05:45:02.548 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T05:45:02.548 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T05:45:02.548 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T05:45:02.548 INFO:teuthology.orchestra.run.smithi138.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T05:45:02.548 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T05:45:02.548 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T05:45:02.549 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T05:45:02.549 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:02.549 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:02.549 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:02.549 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T05:45:02.549 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T05:45:02.549 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T05:45:02.549 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T05:45:02.549 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T05:45:02.549 INFO:teuthology.orchestra.run.smithi138.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T05:45:02.549 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T05:45:02.549 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T05:45:02.550 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T05:45:02.550 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T05:45:02.550 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T05:45:02.550 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T05:45:02.550 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T05:45:02.550 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T05:45:02.550 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T05:45:02.550 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T05:45:02.550 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T05:45:02.550 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T05:45:02.551 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T05:45:02.551 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T05:45:02.551 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T05:45:02.551 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T05:45:02.551 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T05:45:02.551 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T05:45:02.551 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T05:45:02.551 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T05:45:02.551 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T05:45:02.551 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T05:45:02.551 INFO:teuthology.orchestra.run.smithi138.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T05:45:02.551 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T05:45:02.551 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T05:45:02.551 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T05:45:02.552 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T05:45:02.552 INFO:teuthology.orchestra.run.smithi138.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T05:45:02.552 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T05:45:02.552 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T05:45:02.552 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T05:45:02.552 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T05:45:02.552 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T05:45:02.552 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T05:45:02.552 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:02.552 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:02.552 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T05:45:02.552 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T05:45:02.552 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T05:45:02.552 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:02.553 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T05:45:02.553 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T05:45:02.553 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T05:45:02.553 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T05:45:02.553 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T05:45:02.553 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T05:45:02.553 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T05:45:02.553 INFO:teuthology.orchestra.run.smithi138.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T05:45:02.553 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T05:45:02.553 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T05:45:02.553 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T05:45:02.553 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T05:45:02.554 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T05:45:02.554 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T05:45:02.554 INFO:teuthology.orchestra.run.smithi138.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T05:45:02.554 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:02.554 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:02.554 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:02.554 INFO:teuthology.orchestra.run.smithi138.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T05:45:02.554 INFO:teuthology.orchestra.run.smithi138.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T05:45:02.554 INFO:teuthology.orchestra.run.smithi138.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T05:45:02.554 INFO:teuthology.orchestra.run.smithi138.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T05:45:02.554 INFO:teuthology.orchestra.run.smithi138.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T05:45:02.554 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T05:45:02.555 INFO:teuthology.orchestra.run.smithi138.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T05:45:02.555 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-21T05:45:02.555 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-21T05:45:02.555 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T05:45:02.555 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T05:45:02.633 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T05:45:02.633 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T05:45:02.717 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T05:45:02.742 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T05:45:02.747 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T05:45:02.747 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T05:45:02.927 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T05:45:02.927 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T05:45:02.975 DEBUG:teuthology.parallel:result is None 2024-09-21T05:45:03.011 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T05:45:03.086 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T05:45:09.585 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T05:45:09.586 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T05:45:09.586 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T05:45:09.586 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T05:45:09.586 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T05:45:09.586 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T05:45:09.586 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T05:45:09.586 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T05:45:09.586 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T05:45:09.586 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T05:45:09.586 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T05:45:09.586 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T05:45:09.586 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T05:45:09.586 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T05:45:09.587 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T05:45:09.587 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T05:45:09.587 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T05:45:09.587 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T05:45:09.587 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T05:45:09.587 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T05:45:09.587 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T05:45:09.587 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T05:45:09.587 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T05:45:09.587 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T05:45:09.587 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T05:45:09.587 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T05:45:09.587 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T05:45:09.587 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T05:45:09.588 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T05:45:09.588 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T05:45:09.588 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T05:45:09.588 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T05:45:09.588 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T05:45:09.588 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T05:45:09.588 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T05:45:09.588 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T05:45:09.588 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T05:45:09.588 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T05:45:09.588 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T05:45:09.588 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T05:45:09.588 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T05:45:09.588 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T05:45:09.588 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T05:45:09.589 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T05:45:09.589 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T05:45:09.589 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T05:45:09.589 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T05:45:09.589 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T05:45:09.589 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T05:45:09.589 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T05:45:09.589 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T05:45:09.589 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T05:45:09.589 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T05:45:09.589 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T05:45:09.589 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T05:45:09.589 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T05:45:09.589 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T05:45:09.589 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T05:45:09.590 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T05:45:09.590 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T05:45:09.590 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T05:45:09.590 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T05:45:09.590 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T05:45:09.590 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T05:45:09.590 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T05:45:09.590 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T05:45:09.590 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T05:45:09.590 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T05:45:09.590 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T05:45:09.590 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T05:45:09.590 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T05:45:09.590 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T05:45:09.590 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T05:45:09.591 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T05:45:09.591 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T05:45:09.591 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T05:45:09.591 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T05:45:09.591 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T05:45:09.591 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T05:45:09.591 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T05:45:09.591 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T05:45:09.591 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T05:45:09.591 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T05:45:09.591 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T05:45:09.591 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T05:45:09.591 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T05:45:09.591 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T05:45:09.591 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T05:45:09.592 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T05:45:09.592 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T05:45:09.592 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T05:45:09.592 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T05:45:09.592 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T05:45:09.592 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T05:45:09.592 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T05:45:09.592 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T05:45:09.592 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T05:45:09.592 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T05:45:09.592 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T05:45:09.592 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T05:45:09.592 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T05:45:09.592 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T05:45:09.592 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T05:45:09.592 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T05:45:09.593 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T05:45:09.593 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T05:45:09.593 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T05:45:09.593 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T05:45:09.593 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T05:45:09.593 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T05:45:09.593 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T05:45:09.593 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T05:45:09.593 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T05:45:09.593 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T05:45:09.593 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T05:45:09.593 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T05:45:09.593 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T05:45:09.593 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T05:45:09.594 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T05:45:09.594 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T05:45:09.594 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T05:45:09.594 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T05:45:09.594 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T05:45:09.594 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T05:45:09.594 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T05:45:09.594 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T05:45:09.594 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T05:45:09.594 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T05:45:09.594 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T05:45:09.594 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T05:45:09.594 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T05:45:09.594 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T05:45:09.595 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T05:45:09.595 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T05:45:09.595 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T05:45:09.595 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T05:45:09.595 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T05:45:09.595 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T05:45:09.595 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T05:45:09.595 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T05:45:09.595 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T05:45:09.595 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T05:45:09.595 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T05:45:09.595 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T05:45:09.595 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T05:45:09.595 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T05:45:09.596 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T05:45:09.596 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T05:45:09.596 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T05:45:09.596 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T05:45:09.596 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T05:45:09.596 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T05:45:09.596 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T05:45:09.596 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T05:45:09.596 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T05:45:09.596 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T05:45:09.596 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T05:45:09.596 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T05:45:09.596 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T05:45:09.596 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T05:45:09.596 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T05:45:09.597 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T05:45:09.597 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T05:45:09.597 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T05:45:09.597 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T05:45:12.269 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T05:45:12.269 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:45:12.269 INFO:teuthology.orchestra.run.smithi117.stdout:Upgraded: 2024-09-21T05:45:12.269 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T05:45:12.269 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T05:45:12.269 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T05:45:12.269 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T05:45:12.269 INFO:teuthology.orchestra.run.smithi117.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:12.269 INFO:teuthology.orchestra.run.smithi117.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:12.269 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T05:45:12.269 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T05:45:12.270 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-09-21T05:45:12.270 INFO:teuthology.orchestra.run.smithi117.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T05:45:12.270 INFO:teuthology.orchestra.run.smithi117.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T05:45:12.270 INFO:teuthology.orchestra.run.smithi117.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T05:45:12.270 INFO:teuthology.orchestra.run.smithi117.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T05:45:12.270 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:12.270 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:12.270 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:12.270 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:12.270 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:45:12.270 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:12.270 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:12.270 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:12.271 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:45:12.271 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:45:12.271 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:45:12.271 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:45:12.271 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:45:12.271 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:12.271 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:12.271 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:45:12.271 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:12.271 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:12.271 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:12.271 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:45:12.271 INFO:teuthology.orchestra.run.smithi117.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:45:12.272 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:12.272 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:12.272 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:45:12.272 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T05:45:12.272 INFO:teuthology.orchestra.run.smithi117.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T05:45:12.272 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T05:45:12.272 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T05:45:12.272 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T05:45:12.272 INFO:teuthology.orchestra.run.smithi117.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T05:45:12.272 INFO:teuthology.orchestra.run.smithi117.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T05:45:12.272 INFO:teuthology.orchestra.run.smithi117.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T05:45:12.272 INFO:teuthology.orchestra.run.smithi117.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T05:45:12.272 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T05:45:12.272 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T05:45:12.273 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:12.273 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:12.273 INFO:teuthology.orchestra.run.smithi117.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:12.273 INFO:teuthology.orchestra.run.smithi117.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T05:45:12.273 INFO:teuthology.orchestra.run.smithi117.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T05:45:12.273 INFO:teuthology.orchestra.run.smithi117.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T05:45:12.273 INFO:teuthology.orchestra.run.smithi117.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T05:45:12.273 INFO:teuthology.orchestra.run.smithi117.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T05:45:12.273 INFO:teuthology.orchestra.run.smithi117.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T05:45:12.273 INFO:teuthology.orchestra.run.smithi117.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:12.273 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:12.273 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T05:45:12.273 INFO:teuthology.orchestra.run.smithi117.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:12.274 INFO:teuthology.orchestra.run.smithi117.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T05:45:12.274 INFO:teuthology.orchestra.run.smithi117.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T05:45:12.274 INFO:teuthology.orchestra.run.smithi117.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T05:45:12.274 INFO:teuthology.orchestra.run.smithi117.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T05:45:12.274 INFO:teuthology.orchestra.run.smithi117.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T05:45:12.274 INFO:teuthology.orchestra.run.smithi117.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T05:45:12.274 INFO:teuthology.orchestra.run.smithi117.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T05:45:12.274 INFO:teuthology.orchestra.run.smithi117.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T05:45:12.274 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T05:45:12.274 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T05:45:12.274 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T05:45:12.274 INFO:teuthology.orchestra.run.smithi117.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T05:45:12.274 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T05:45:12.275 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T05:45:12.275 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T05:45:12.275 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T05:45:12.275 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T05:45:12.275 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T05:45:12.275 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T05:45:12.275 INFO:teuthology.orchestra.run.smithi117.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T05:45:12.275 INFO:teuthology.orchestra.run.smithi117.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T05:45:12.275 INFO:teuthology.orchestra.run.smithi117.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T05:45:12.275 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T05:45:12.275 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:12.275 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:12.275 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:12.275 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T05:45:12.276 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T05:45:12.276 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T05:45:12.276 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T05:45:12.276 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T05:45:12.276 INFO:teuthology.orchestra.run.smithi117.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T05:45:12.276 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T05:45:12.276 INFO:teuthology.orchestra.run.smithi117.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T05:45:12.276 INFO:teuthology.orchestra.run.smithi117.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T05:45:12.276 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T05:45:12.276 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T05:45:12.276 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T05:45:12.276 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T05:45:12.276 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T05:45:12.277 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T05:45:12.277 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T05:45:12.277 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T05:45:12.277 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T05:45:12.277 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T05:45:12.277 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T05:45:12.277 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T05:45:12.277 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T05:45:12.277 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T05:45:12.277 INFO:teuthology.orchestra.run.smithi117.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T05:45:12.277 INFO:teuthology.orchestra.run.smithi117.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T05:45:12.277 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T05:45:12.277 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T05:45:12.277 INFO:teuthology.orchestra.run.smithi117.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T05:45:12.278 INFO:teuthology.orchestra.run.smithi117.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T05:45:12.278 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T05:45:12.278 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T05:45:12.278 INFO:teuthology.orchestra.run.smithi117.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T05:45:12.278 INFO:teuthology.orchestra.run.smithi117.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T05:45:12.278 INFO:teuthology.orchestra.run.smithi117.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T05:45:12.278 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T05:45:12.278 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T05:45:12.278 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T05:45:12.278 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T05:45:12.278 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T05:45:12.278 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T05:45:12.278 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:12.278 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:12.278 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T05:45:12.279 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T05:45:12.279 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T05:45:12.279 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:12.279 INFO:teuthology.orchestra.run.smithi117.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T05:45:12.279 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T05:45:12.279 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T05:45:12.279 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T05:45:12.279 INFO:teuthology.orchestra.run.smithi117.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T05:45:12.279 INFO:teuthology.orchestra.run.smithi117.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T05:45:12.279 INFO:teuthology.orchestra.run.smithi117.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T05:45:12.279 INFO:teuthology.orchestra.run.smithi117.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T05:45:12.279 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T05:45:12.279 INFO:teuthology.orchestra.run.smithi117.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T05:45:12.280 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T05:45:12.280 INFO:teuthology.orchestra.run.smithi117.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T05:45:12.280 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T05:45:12.280 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T05:45:12.280 INFO:teuthology.orchestra.run.smithi117.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T05:45:12.280 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:12.280 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:12.280 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:45:12.280 INFO:teuthology.orchestra.run.smithi117.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T05:45:12.280 INFO:teuthology.orchestra.run.smithi117.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T05:45:12.280 INFO:teuthology.orchestra.run.smithi117.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T05:45:12.280 INFO:teuthology.orchestra.run.smithi117.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T05:45:12.280 INFO:teuthology.orchestra.run.smithi117.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T05:45:12.280 INFO:teuthology.orchestra.run.smithi117.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T05:45:12.281 INFO:teuthology.orchestra.run.smithi117.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T05:45:12.281 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:45:12.281 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-09-21T05:45:12.700 DEBUG:teuthology.parallel:result is None 2024-09-21T05:45:12.700 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T05:45:12.852 DEBUG:teuthology.orchestra.run.smithi117:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T05:45:12.874 INFO:teuthology.orchestra.run.smithi117.stdout:19.3.0-5120.g18432218.el9 2024-09-21T05:45:12.875 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T05:45:12.875 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T05:45:12.876 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T05:45:13.022 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T05:45:13.049 INFO:teuthology.orchestra.run.smithi138.stdout:19.3.0-5120.g18432218.el9 2024-09-21T05:45:13.049 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T05:45:13.049 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T05:45:13.051 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T05:45:13.199 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T05:45:13.225 INFO:teuthology.orchestra.run.smithi192.stdout:19.3.0-5120.g18432218.el9 2024-09-21T05:45:13.226 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T05:45:13.226 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T05:45:13.227 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T05:45:13.227 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-21T05:45:13.228 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T05:45:13.256 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-21T05:45:13.256 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T05:45:13.288 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-21T05:45:13.288 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T05:45:13.321 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T05:45:13.321 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-21T05:45:13.321 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T05:45:13.346 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T05:45:13.412 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-21T05:45:13.412 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T05:45:13.442 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T05:45:13.511 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-21T05:45:13.511 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T05:45:13.541 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T05:45:13.609 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T05:45:13.609 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-21T05:45:13.610 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T05:45:13.635 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T05:45:13.699 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-21T05:45:13.699 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T05:45:13.728 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T05:45:13.795 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-21T05:45:13.796 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T05:45:13.821 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T05:45:13.887 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T05:45:13.887 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-21T05:45:13.887 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T05:45:13.911 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T05:45:13.976 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-21T05:45:13.976 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T05:45:14.004 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T05:45:14.073 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-21T05:45:14.073 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T05:45:14.098 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T05:45:14.162 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-21T05:45:14.270 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mgr/snap_schedule/allow_m_granularity': True, 'mgr/snap_schedule/dump_on_update': True, 'mgr/snap_schedule/log_level': 'debug', 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mds': {'mds_bal_replicate_threshold': 1}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'mgr-modules': ['snap_schedule'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-21T05:45:14.271 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T05:45:14.271 INFO:tasks.cephadm:Cluster fsid is ac8581a2-77dc-11ef-baf5-efdc52797490 2024-09-21T05:45:14.271 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-21T05:45:14.271 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.117', 'mon.b': '172.21.15.138', 'mon.c': '172.21.15.192'} 2024-09-21T05:45:14.272 INFO:tasks.cephadm:First mon is mon.a on smithi117 2024-09-21T05:45:14.272 INFO:tasks.cephadm:First mgr is x 2024-09-21T05:45:14.272 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-21T05:45:14.272 DEBUG:teuthology.orchestra.run.smithi117:> sudo hostname $(hostname -s) 2024-09-21T05:45:14.297 DEBUG:teuthology.orchestra.run.smithi138:> sudo hostname $(hostname -s) 2024-09-21T05:45:14.327 DEBUG:teuthology.orchestra.run.smithi192:> sudo hostname $(hostname -s) 2024-09-21T05:45:14.353 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-21T05:45:14.353 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T05:45:14.499 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.4.75+confusa13', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-21 00:04:17.484514', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['x86_64', 'source', 'arm64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.2.22+braggi22', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-20 22:54:16.499090', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-21T05:45:14.647 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-pdonnell-testing-20240920.212106-debug, sha1 1843221895adccfb3b692060251dd6a0183bc410 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T05:45:14.649 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T05:45:14.649 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T05:45:14.649 DEBUG:teuthology.orchestra.run.smithi117:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T05:45:15.196 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 05:45 /home/ubuntu/cephtest/cephadm 2024-09-21T05:45:15.196 DEBUG:teuthology.orchestra.run.smithi138:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T05:45:15.732 INFO:teuthology.orchestra.run.smithi138.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 05:45 /home/ubuntu/cephtest/cephadm 2024-09-21T05:45:15.732 DEBUG:teuthology.orchestra.run.smithi192:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T05:45:16.255 INFO:teuthology.orchestra.run.smithi192.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 05:45 /home/ubuntu/cephtest/cephadm 2024-09-21T05:45:16.256 DEBUG:teuthology.orchestra.run.smithi117:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T05:45:16.275 DEBUG:teuthology.orchestra.run.smithi138:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T05:45:16.294 DEBUG:teuthology.orchestra.run.smithi192:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T05:45:16.328 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 on all hosts... 2024-09-21T05:45:16.328 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T05:45:16.331 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T05:45:16.338 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T05:45:17.932 INFO:teuthology.orchestra.run.smithi117.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T05:45:17.965 INFO:teuthology.orchestra.run.smithi138.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T05:45:18.002 INFO:teuthology.orchestra.run.smithi192.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T05:46:02.484 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T05:46:02.484 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T05:46:02.484 INFO:teuthology.orchestra.run.smithi117.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T05:46:02.484 INFO:teuthology.orchestra.run.smithi117.stdout: "repo_digests": [ 2024-09-21T05:46:02.485 INFO:teuthology.orchestra.run.smithi117.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T05:46:02.485 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-21T05:46:02.485 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T05:46:36.855 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-21T05:46:36.855 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T05:46:36.855 INFO:teuthology.orchestra.run.smithi138.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T05:46:36.855 INFO:teuthology.orchestra.run.smithi138.stdout: "repo_digests": [ 2024-09-21T05:46:36.855 INFO:teuthology.orchestra.run.smithi138.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T05:46:36.855 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-21T05:46:36.855 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-21T05:46:38.257 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-21T05:46:38.258 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T05:46:38.258 INFO:teuthology.orchestra.run.smithi192.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T05:46:38.258 INFO:teuthology.orchestra.run.smithi192.stdout: "repo_digests": [ 2024-09-21T05:46:38.258 INFO:teuthology.orchestra.run.smithi192.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T05:46:38.258 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-21T05:46:38.258 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-21T05:46:38.283 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /etc/ceph 2024-09-21T05:46:38.317 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /etc/ceph 2024-09-21T05:46:38.350 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /etc/ceph 2024-09-21T05:46:38.378 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 777 /etc/ceph 2024-09-21T05:46:38.407 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 777 /etc/ceph 2024-09-21T05:46:38.435 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 777 /etc/ceph 2024-09-21T05:46:38.459 INFO:tasks.cephadm:Writing seed config... 2024-09-21T05:46:38.460 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-21T05:46:38.460 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-21T05:46:38.460 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-21T05:46:38.461 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-21T05:46:38.461 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-21T05:46:38.461 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-21T05:46:38.461 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-21T05:46:38.461 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-21T05:46:38.461 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-21T05:46:38.461 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-21T05:46:38.461 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-21T05:46:38.461 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-21T05:46:38.461 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-21T05:46:38.461 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-21T05:46:38.461 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-21T05:46:38.461 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-21T05:46:38.461 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-21T05:46:38.461 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-21T05:46:38.462 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-21T05:46:38.462 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-21T05:46:38.462 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-21T05:46:38.462 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-21T05:46:38.462 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-21T05:46:38.462 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-21T05:46:38.462 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-21T05:46:38.462 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-21T05:46:38.480 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = ac8581a2-77dc-11ef-baf5-efdc52797490 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [mds] mds_bal_replicate_threshold = 1 2024-09-21T05:46:38.480 DEBUG:teuthology.orchestra.run.smithi117:mon.a> sudo journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mon.a.service 2024-09-21T05:46:38.523 DEBUG:teuthology.orchestra.run.smithi117:mgr.x> sudo journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mgr.x.service 2024-09-21T05:46:38.565 INFO:tasks.cephadm:Bootstrapping... 2024-09-21T05:46:38.566 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 -v bootstrap --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.117 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-21T05:46:38.776 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-09-21T05:46:38.776 INFO:teuthology.orchestra.run.smithi117.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', '-v', 'bootstrap', '--fsid', 'ac8581a2-77dc-11ef-baf5-efdc52797490', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.117', '--skip-admin-label'] 2024-09-21T05:46:38.777 INFO:teuthology.orchestra.run.smithi117.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-21T05:46:38.777 INFO:teuthology.orchestra.run.smithi117.stdout:Verifying podman|docker is present... 2024-09-21T05:46:38.802 INFO:teuthology.orchestra.run.smithi117.stdout:/bin/podman: stdout 5.2.2 2024-09-21T05:46:38.802 INFO:teuthology.orchestra.run.smithi117.stdout:Verifying lvm2 is present... 2024-09-21T05:46:38.802 INFO:teuthology.orchestra.run.smithi117.stdout:Verifying time synchronization is in place... 2024-09-21T05:46:38.810 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T05:46:38.810 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T05:46:38.817 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T05:46:38.817 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout inactive 2024-09-21T05:46:38.824 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout enabled 2024-09-21T05:46:38.831 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout active 2024-09-21T05:46:38.831 INFO:teuthology.orchestra.run.smithi117.stdout:Unit chronyd.service is enabled and running 2024-09-21T05:46:38.831 INFO:teuthology.orchestra.run.smithi117.stdout:Repeating the final host check... 2024-09-21T05:46:38.854 INFO:teuthology.orchestra.run.smithi117.stdout:/bin/podman: stdout 5.2.2 2024-09-21T05:46:38.854 INFO:teuthology.orchestra.run.smithi117.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-21T05:46:38.854 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl is present 2024-09-21T05:46:38.854 INFO:teuthology.orchestra.run.smithi117.stdout:lvcreate is present 2024-09-21T05:46:38.861 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T05:46:38.861 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T05:46:38.868 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T05:46:38.868 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout inactive 2024-09-21T05:46:38.875 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout enabled 2024-09-21T05:46:38.883 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout active 2024-09-21T05:46:38.883 INFO:teuthology.orchestra.run.smithi117.stdout:Unit chronyd.service is enabled and running 2024-09-21T05:46:38.883 INFO:teuthology.orchestra.run.smithi117.stdout:Host looks OK 2024-09-21T05:46:38.883 INFO:teuthology.orchestra.run.smithi117.stdout:Cluster fsid: ac8581a2-77dc-11ef-baf5-efdc52797490 2024-09-21T05:46:38.884 INFO:teuthology.orchestra.run.smithi117.stdout:Acquiring lock 139887788104048 on /run/cephadm/ac8581a2-77dc-11ef-baf5-efdc52797490.lock 2024-09-21T05:46:38.884 INFO:teuthology.orchestra.run.smithi117.stdout:Lock 139887788104048 acquired on /run/cephadm/ac8581a2-77dc-11ef-baf5-efdc52797490.lock 2024-09-21T05:46:38.884 INFO:teuthology.orchestra.run.smithi117.stdout:Verifying IP 172.21.15.117 port 3300 ... 2024-09-21T05:46:38.884 INFO:teuthology.orchestra.run.smithi117.stdout:Verifying IP 172.21.15.117 port 6789 ... 2024-09-21T05:46:38.884 INFO:teuthology.orchestra.run.smithi117.stdout:Base mon IP(s) is [172.21.15.117:3300, 172.21.15.117:6789], mon addrv is [v2:172.21.15.117:3300,v1:172.21.15.117:6789] 2024-09-21T05:46:38.886 INFO:teuthology.orchestra.run.smithi117.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.117 metric 100 2024-09-21T05:46:38.887 INFO:teuthology.orchestra.run.smithi117.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.117 metric 100 2024-09-21T05:46:38.889 INFO:teuthology.orchestra.run.smithi117.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-21T05:46:38.889 INFO:teuthology.orchestra.run.smithi117.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-21T05:46:38.889 INFO:teuthology.orchestra.run.smithi117.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-21T05:46:38.892 INFO:teuthology.orchestra.run.smithi117.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-21T05:46:38.892 INFO:teuthology.orchestra.run.smithi117.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-21T05:46:38.893 INFO:teuthology.orchestra.run.smithi117.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T05:46:38.893 INFO:teuthology.orchestra.run.smithi117.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-21T05:46:38.893 INFO:teuthology.orchestra.run.smithi117.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:4463/64 scope link noprefixroute 2024-09-21T05:46:38.893 INFO:teuthology.orchestra.run.smithi117.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T05:46:38.893 INFO:teuthology.orchestra.run.smithi117.stdout:Mon IP `172.21.15.117` is in CIDR network `172.21.0.0/20` 2024-09-21T05:46:38.893 INFO:teuthology.orchestra.run.smithi117.stdout:Mon IP `172.21.15.117` is in CIDR network `172.21.0.0/20` 2024-09-21T05:46:38.893 INFO:teuthology.orchestra.run.smithi117.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-21T05:46:38.894 INFO:teuthology.orchestra.run.smithi117.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-21T05:46:38.894 INFO:teuthology.orchestra.run.smithi117.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T05:46:39.347 INFO:teuthology.orchestra.run.smithi117.stdout:/bin/podman: stdout ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T05:46:39.347 INFO:teuthology.orchestra.run.smithi117.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T05:46:39.347 INFO:teuthology.orchestra.run.smithi117.stdout:/bin/podman: stderr Getting image source signatures 2024-09-21T05:46:39.347 INFO:teuthology.orchestra.run.smithi117.stdout:/bin/podman: stderr Copying blob sha256:8425bc7408d91b180e7fdc62f23e7ae6603757f4f36e9969a4eaf9fd9416b764 2024-09-21T05:46:39.347 INFO:teuthology.orchestra.run.smithi117.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-21T05:46:39.347 INFO:teuthology.orchestra.run.smithi117.stdout:/bin/podman: stderr Copying config sha256:ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T05:46:39.347 INFO:teuthology.orchestra.run.smithi117.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-21T05:46:40.259 INFO:teuthology.orchestra.run.smithi117.stdout:ceph: stdout ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T05:46:40.260 INFO:teuthology.orchestra.run.smithi117.stdout:Ceph version: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T05:46:40.260 INFO:teuthology.orchestra.run.smithi117.stdout:Extracting ceph user uid/gid from container image... 2024-09-21T05:46:41.132 INFO:teuthology.orchestra.run.smithi117.stdout:stat: stdout 167 167 2024-09-21T05:46:41.132 INFO:teuthology.orchestra.run.smithi117.stdout:Creating initial keys... 2024-09-21T05:46:42.029 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-authtool: stdout AQDBXe5mNvFfGxAAo1QNF54QZNdxDJLD/Fc5Tw== 2024-09-21T05:46:42.992 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-authtool: stdout AQDCXe5mV2CXFRAAihxNYYiFtZ3jji88uqbykg== 2024-09-21T05:46:43.825 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-authtool: stdout AQDDXe5mJCyNEhAAwSiEYO2e7B/YX0Pe7Rk7Ug== 2024-09-21T05:46:43.826 INFO:teuthology.orchestra.run.smithi117.stdout:Creating initial monmap... 2024-09-21T05:46:44.720 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T05:46:44.720 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-21T05:46:44.720 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to ac8581a2-77dc-11ef-baf5-efdc52797490 2024-09-21T05:46:44.720 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T05:46:44.721 INFO:teuthology.orchestra.run.smithi117.stdout:monmaptool for a [v2:172.21.15.117:3300,v1:172.21.15.117:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T05:46:44.721 INFO:teuthology.orchestra.run.smithi117.stdout:setting min_mon_release = quincy 2024-09-21T05:46:44.721 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/monmaptool: set fsid to ac8581a2-77dc-11ef-baf5-efdc52797490 2024-09-21T05:46:44.721 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T05:46:44.721 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:46:44.721 INFO:teuthology.orchestra.run.smithi117.stdout:Creating mon... 2024-09-21T05:46:45.618 INFO:teuthology.orchestra.run.smithi117.stdout:create mon.a on 2024-09-21T05:46:45.821 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-21T05:46:45.964 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-21T05:46:46.132 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-ac8581a2-77dc-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-ac8581a2-77dc-11ef-baf5-efdc52797490.target. 2024-09-21T05:46:46.133 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-ac8581a2-77dc-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-ac8581a2-77dc-11ef-baf5-efdc52797490.target. 2024-09-21T05:46:46.359 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mon.a 2024-09-21T05:46:46.359 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Failed to reset failed state of unit ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mon.a.service: Unit ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mon.a.service not loaded. 2024-09-21T05:46:46.523 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-ac8581a2-77dc-11ef-baf5-efdc52797490.target.wants/ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mon.a.service → /etc/systemd/system/ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@.service. 2024-09-21T05:46:47.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:46 smithi117 podman[30561]: 2024-09-21 05:46:46.972265904 +0000 UTC m=+0.331499526 container start 8d2937e2c07526457d10a490bdc4676f57c11ecd4a36d2623c5e37b32cb4736d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-21T05:46:47.067 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T05:46:47.067 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout disabled 2024-09-21T05:46:47.074 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T05:46:47.074 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout inactive 2024-09-21T05:46:47.074 INFO:teuthology.orchestra.run.smithi117.stdout:firewalld.service is not enabled 2024-09-21T05:46:47.074 INFO:teuthology.orchestra.run.smithi117.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T05:46:47.075 INFO:teuthology.orchestra.run.smithi117.stdout:Waiting for mon to start... 2024-09-21T05:46:47.075 INFO:teuthology.orchestra.run.smithi117.stdout:Waiting for mon... 2024-09-21T05:46:47.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:47 smithi117 ceph-mon[30580]: mkfs ac8581a2-77dc-11ef-baf5-efdc52797490 2024-09-21T05:46:47.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:47 smithi117 ceph-mon[30580]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T05:46:47.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:47 smithi117 bash[30561]: 8d2937e2c07526457d10a490bdc4676f57c11ecd4a36d2623c5e37b32cb4736d 2024-09-21T05:46:47.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:47 smithi117 systemd[1]: Started Ceph mon.a for ac8581a2-77dc-11ef-baf5-efdc52797490. 2024-09-21T05:46:48.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:48 smithi117 ceph-mon[30580]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T05:46:48.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:48 smithi117 ceph-mon[30580]: monmap epoch 1 2024-09-21T05:46:48.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:48 smithi117 ceph-mon[30580]: fsid ac8581a2-77dc-11ef-baf5-efdc52797490 2024-09-21T05:46:48.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:48 smithi117 ceph-mon[30580]: last_changed 2024-09-21T05:46:44.194497+0000 2024-09-21T05:46:48.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:48 smithi117 ceph-mon[30580]: created 2024-09-21T05:46:44.194497+0000 2024-09-21T05:46:48.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:48 smithi117 ceph-mon[30580]: min_mon_release 19 (squid) 2024-09-21T05:46:48.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:48 smithi117 ceph-mon[30580]: election_strategy: 1 2024-09-21T05:46:48.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:48 smithi117 ceph-mon[30580]: 0: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.a 2024-09-21T05:46:48.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:48 smithi117 ceph-mon[30580]: fsmap 2024-09-21T05:46:48.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:48 smithi117 ceph-mon[30580]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T05:46:48.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:48 smithi117 ceph-mon[30580]: mgrmap e1: no daemons active 2024-09-21T05:46:48.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:48 smithi117 ceph-mon[30580]: from='client.? 172.21.15.117:0/3167567094' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-09-21T05:46:48.379 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout cluster: 2024-09-21T05:46:48.381 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout id: ac8581a2-77dc-11ef-baf5-efdc52797490 2024-09-21T05:46:48.381 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-21T05:46:48.381 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-09-21T05:46:48.381 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout services: 2024-09-21T05:46:48.381 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.53196s) 2024-09-21T05:46:48.381 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-21T05:46:48.381 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-21T05:46:48.382 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-09-21T05:46:48.382 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout data: 2024-09-21T05:46:48.382 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-21T05:46:48.382 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-21T05:46:48.382 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-21T05:46:48.382 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout pgs: 2024-09-21T05:46:48.382 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-09-21T05:46:48.382 INFO:teuthology.orchestra.run.smithi117.stdout:mon is available 2024-09-21T05:46:48.382 INFO:teuthology.orchestra.run.smithi117.stdout:Assimilating anything we can from ceph.conf... 2024-09-21T05:46:49.383 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:49 smithi117 ceph-mon[30580]: from='client.? 172.21.15.117:0/2264745087' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T05:46:49.383 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:49 smithi117 ceph-mon[30580]: from='client.? 172.21.15.117:0/2264745087' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-21T05:46:49.388 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-09-21T05:46:49.388 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T05:46:49.388 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout fsid = ac8581a2-77dc-11ef-baf5-efdc52797490 2024-09-21T05:46:49.388 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T05:46:49.388 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.117:3300,v1:172.21.15.117:6789] 2024-09-21T05:46:49.388 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T05:46:49.388 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T05:46:49.388 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T05:46:49.388 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T05:46:49.388 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-09-21T05:46:49.388 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T05:46:49.388 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T05:46:49.388 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-09-21T05:46:49.389 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T05:46:49.389 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T05:46:49.389 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T05:46:49.389 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T05:46:49.389 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T05:46:49.389 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T05:46:49.389 INFO:teuthology.orchestra.run.smithi117.stdout:Generating new minimal ceph.conf... 2024-09-21T05:46:50.414 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:50 smithi117 ceph-mon[30580]: from='client.? 172.21.15.117:0/1676581246' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:46:50.417 INFO:teuthology.orchestra.run.smithi117.stdout:Restarting the monitor... 2024-09-21T05:46:50.684 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:50 smithi117 systemd[1]: Stopping Ceph mon.a for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T05:46:51.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:50 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mon-a[30576]: 2024-09-21T05:46:50.681+0000 7fa55f73f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T05:46:51.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:50 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mon-a[30576]: 2024-09-21T05:46:50.681+0000 7fa55f73f640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-21T05:46:51.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:50 smithi117 podman[30821]: 2024-09-21 05:46:50.815354072 +0000 UTC m=+0.220606750 container died 8d2937e2c07526457d10a490bdc4676f57c11ecd4a36d2623c5e37b32cb4736d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-21T05:46:51.350 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:51 smithi117 podman[30821]: 2024-09-21 05:46:51.148376463 +0000 UTC m=+0.553629133 container cleanup 8d2937e2c07526457d10a490bdc4676f57c11ecd4a36d2623c5e37b32cb4736d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mon-a, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-21T05:46:51.351 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:51 smithi117 bash[30821]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mon-a 2024-09-21T05:46:51.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:51 smithi117 podman[30834]: 2024-09-21 05:46:51.349210388 +0000 UTC m=+0.528284817 container remove 8d2937e2c07526457d10a490bdc4676f57c11ecd4a36d2623c5e37b32cb4736d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mon-a, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux ) 2024-09-21T05:46:51.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:51 smithi117 systemd[1]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mon.a.service: Deactivated successfully. 2024-09-21T05:46:51.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:51 smithi117 systemd[1]: Stopped Ceph mon.a for ac8581a2-77dc-11ef-baf5-efdc52797490. 2024-09-21T05:46:51.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:51 smithi117 systemd[1]: Starting Ceph mon.a for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T05:46:52.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:51 smithi117 podman[30928]: 2024-09-21 05:46:51.77490459 +0000 UTC m=+0.111163597 container create 9aed4354fc168f03a339a36fb1a7f3641121e277dc139edf5959fedb82e767ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mon-a, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T05:46:52.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:51 smithi117 podman[30928]: 2024-09-21 05:46:51.679477467 +0000 UTC m=+0.015736470 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T05:46:52.124 INFO:teuthology.orchestra.run.smithi117.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-21T05:46:52.351 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 podman[30928]: 2024-09-21 05:46:52.027093469 +0000 UTC m=+0.363352472 container init 9aed4354fc168f03a339a36fb1a7f3641121e277dc139edf5959fedb82e767ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mon-a, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T05:46:52.351 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 podman[30928]: 2024-09-21 05:46:52.030357241 +0000 UTC m=+0.366616252 container start 9aed4354fc168f03a339a36fb1a7f3641121e277dc139edf5959fedb82e767ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD) 2024-09-21T05:46:52.351 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T05:46:52.352 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T05:46:52.352 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: pidfile_write: ignore empty --pid-file 2024-09-21T05:46:52.352 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: load: jerasure load: lrc 2024-09-21T05:46:52.352 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: RocksDB version: 7.9.2 2024-09-21T05:46:52.352 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Git sha 0 2024-09-21T05:46:52.352 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T05:46:52.352 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: DB SUMMARY 2024-09-21T05:46:52.352 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: DB Session ID: ZXLCPD944ELJW4G20MX0 2024-09-21T05:46:52.352 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: CURRENT file: CURRENT 2024-09-21T05:46:52.352 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T05:46:52.352 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-21T05:46:52.352 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-21T05:46:52.352 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 94838 ; 2024-09-21T05:46:52.353 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.error_if_exists: 0 2024-09-21T05:46:52.353 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.create_if_missing: 0 2024-09-21T05:46:52.353 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.paranoid_checks: 1 2024-09-21T05:46:52.353 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T05:46:52.353 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T05:46:52.353 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T05:46:52.353 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.env: 0x559b5a8ecd60 2024-09-21T05:46:52.353 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T05:46:52.353 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.info_log: 0x559b5fa81360 2024-09-21T05:46:52.353 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T05:46:52.354 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.statistics: (nil) 2024-09-21T05:46:52.355 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.use_fsync: 0 2024-09-21T05:46:52.355 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.max_log_file_size: 0 2024-09-21T05:46:52.355 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T05:46:52.355 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T05:46:52.355 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T05:46:52.355 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T05:46:52.355 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.allow_fallocate: 1 2024-09-21T05:46:52.355 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T05:46:52.355 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T05:46:52.355 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.use_direct_reads: 0 2024-09-21T05:46:52.355 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T05:46:52.355 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T05:46:52.355 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.db_log_dir: 2024-09-21T05:46:52.355 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.wal_dir: 2024-09-21T05:46:52.356 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T05:46:52.356 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T05:46:52.356 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T05:46:52.356 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T05:46:52.356 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T05:46:52.356 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T05:46:52.356 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T05:46:52.356 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T05:46:52.356 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.write_buffer_manager: 0x559b5fa85d60 2024-09-21T05:46:52.356 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T05:46:52.356 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T05:46:52.356 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T05:46:52.356 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T05:46:52.356 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T05:46:52.356 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T05:46:52.357 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T05:46:52.357 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T05:46:52.357 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.unordered_write: 0 2024-09-21T05:46:52.357 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T05:46:52.357 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T05:46:52.357 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T05:46:52.357 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T05:46:52.357 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.row_cache: None 2024-09-21T05:46:52.357 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.wal_filter: None 2024-09-21T05:46:52.357 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T05:46:52.357 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T05:46:52.357 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.two_write_queues: 0 2024-09-21T05:46:52.357 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T05:46:52.357 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.wal_compression: 0 2024-09-21T05:46:52.357 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.atomic_flush: 0 2024-09-21T05:46:52.358 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T05:46:52.358 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T05:46:52.358 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T05:46:52.358 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.log_readahead_size: 0 2024-09-21T05:46:52.358 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T05:46:52.358 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T05:46:52.358 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T05:46:52.358 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T05:46:52.358 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T05:46:52.358 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T05:46:52.358 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T05:46:52.358 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.max_background_jobs: 2 2024-09-21T05:46:52.358 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.max_background_compactions: -1 2024-09-21T05:46:52.358 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.max_subcompactions: 1 2024-09-21T05:46:52.358 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T05:46:52.359 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T05:46:52.359 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T05:46:52.359 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T05:46:52.359 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T05:46:52.359 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T05:46:52.359 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T05:46:52.359 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T05:46:52.359 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.max_open_files: -1 2024-09-21T05:46:52.359 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T05:46:52.359 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T05:46:52.359 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T05:46:52.359 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T05:46:52.359 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.max_background_flushes: -1 2024-09-21T05:46:52.359 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Compression algorithms supported: 2024-09-21T05:46:52.359 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: kZSTD supported: 0 2024-09-21T05:46:52.359 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: kXpressCompression supported: 0 2024-09-21T05:46:52.360 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: kBZip2Compression supported: 0 2024-09-21T05:46:52.360 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T05:46:52.360 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: kLZ4Compression supported: 1 2024-09-21T05:46:52.360 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: kZlibCompression supported: 1 2024-09-21T05:46:52.360 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T05:46:52.360 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: kSnappyCompression supported: 1 2024-09-21T05:46:52.360 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T05:46:52.360 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T05:46:52.360 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-21T05:46:52.360 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T05:46:52.360 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T05:46:52.360 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.merge_operator: 2024-09-21T05:46:52.360 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.compaction_filter: None 2024-09-21T05:46:52.360 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T05:46:52.360 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T05:46:52.360 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T05:46:52.361 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T05:46:52.361 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559b5fa80fa0) 2024-09-21T05:46:52.361 INFO:journalctl@ceph.mon.a.smithi117.stdout: cache_index_and_filter_blocks: 1 2024-09-21T05:46:52.361 INFO:journalctl@ceph.mon.a.smithi117.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T05:46:52.361 INFO:journalctl@ceph.mon.a.smithi117.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T05:46:52.361 INFO:journalctl@ceph.mon.a.smithi117.stdout: pin_top_level_index_and_filter: 1 2024-09-21T05:46:52.361 INFO:journalctl@ceph.mon.a.smithi117.stdout: index_type: 0 2024-09-21T05:46:52.361 INFO:journalctl@ceph.mon.a.smithi117.stdout: data_block_index_type: 0 2024-09-21T05:46:52.361 INFO:journalctl@ceph.mon.a.smithi117.stdout: index_shortening: 1 2024-09-21T05:46:52.361 INFO:journalctl@ceph.mon.a.smithi117.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T05:46:52.361 INFO:journalctl@ceph.mon.a.smithi117.stdout: checksum: 4 2024-09-21T05:46:52.361 INFO:journalctl@ceph.mon.a.smithi117.stdout: no_block_cache: 0 2024-09-21T05:46:52.361 INFO:journalctl@ceph.mon.a.smithi117.stdout: block_cache: 0x559b5ba45610 2024-09-21T05:46:52.361 INFO:journalctl@ceph.mon.a.smithi117.stdout: block_cache_name: BinnedLRUCache 2024-09-21T05:46:52.361 INFO:journalctl@ceph.mon.a.smithi117.stdout: block_cache_options: 2024-09-21T05:46:52.362 INFO:journalctl@ceph.mon.a.smithi117.stdout: capacity : 536870912 2024-09-21T05:46:52.362 INFO:journalctl@ceph.mon.a.smithi117.stdout: num_shard_bits : 4 2024-09-21T05:46:52.362 INFO:journalctl@ceph.mon.a.smithi117.stdout: strict_capacity_limit : 0 2024-09-21T05:46:52.362 INFO:journalctl@ceph.mon.a.smithi117.stdout: high_pri_pool_ratio: 0.000 2024-09-21T05:46:52.362 INFO:journalctl@ceph.mon.a.smithi117.stdout: block_cache_compressed: (nil) 2024-09-21T05:46:52.362 INFO:journalctl@ceph.mon.a.smithi117.stdout: persistent_cache: (nil) 2024-09-21T05:46:52.362 INFO:journalctl@ceph.mon.a.smithi117.stdout: block_size: 4096 2024-09-21T05:46:52.362 INFO:journalctl@ceph.mon.a.smithi117.stdout: block_size_deviation: 10 2024-09-21T05:46:52.362 INFO:journalctl@ceph.mon.a.smithi117.stdout: block_restart_interval: 16 2024-09-21T05:46:52.362 INFO:journalctl@ceph.mon.a.smithi117.stdout: index_block_restart_interval: 1 2024-09-21T05:46:52.362 INFO:journalctl@ceph.mon.a.smithi117.stdout: metadata_block_size: 4096 2024-09-21T05:46:52.362 INFO:journalctl@ceph.mon.a.smithi117.stdout: partition_filters: 0 2024-09-21T05:46:52.362 INFO:journalctl@ceph.mon.a.smithi117.stdout: use_delta_encoding: 1 2024-09-21T05:46:52.362 INFO:journalctl@ceph.mon.a.smithi117.stdout: filter_policy: bloomfilter 2024-09-21T05:46:52.363 INFO:journalctl@ceph.mon.a.smithi117.stdout: whole_key_filtering: 1 2024-09-21T05:46:52.363 INFO:journalctl@ceph.mon.a.smithi117.stdout: verify_compression: 0 2024-09-21T05:46:52.363 INFO:journalctl@ceph.mon.a.smithi117.stdout: read_amp_bytes_per_bit: 0 2024-09-21T05:46:52.363 INFO:journalctl@ceph.mon.a.smithi117.stdout: format_version: 5 2024-09-21T05:46:52.363 INFO:journalctl@ceph.mon.a.smithi117.stdout: enable_index_compression: 1 2024-09-21T05:46:52.363 INFO:journalctl@ceph.mon.a.smithi117.stdout: block_align: 0 2024-09-21T05:46:52.363 INFO:journalctl@ceph.mon.a.smithi117.stdout: max_auto_readahead_size: 262144 2024-09-21T05:46:52.363 INFO:journalctl@ceph.mon.a.smithi117.stdout: prepopulate_block_cache: 0 2024-09-21T05:46:52.363 INFO:journalctl@ceph.mon.a.smithi117.stdout: initial_auto_readahead_size: 8192 2024-09-21T05:46:52.363 INFO:journalctl@ceph.mon.a.smithi117.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T05:46:52.363 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T05:46:52.363 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T05:46:52.363 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.compression: NoCompression 2024-09-21T05:46:52.363 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T05:46:52.364 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T05:46:52.364 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T05:46:52.364 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.num_levels: 7 2024-09-21T05:46:52.364 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T05:46:52.364 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T05:46:52.364 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T05:46:52.364 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T05:46:52.364 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T05:46:52.364 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T05:46:52.364 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T05:46:52.364 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T05:46:52.364 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T05:46:52.364 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T05:46:52.364 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T05:46:52.364 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T05:46:52.364 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T05:46:52.364 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T05:46:52.365 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T05:46:52.365 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T05:46:52.365 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T05:46:52.365 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T05:46:52.365 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T05:46:52.365 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T05:46:52.365 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T05:46:52.365 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T05:46:52.365 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T05:46:52.365 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T05:46:52.365 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T05:46:52.365 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T05:46:52.365 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T05:46:52.365 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T05:46:52.365 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T05:46:52.365 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T05:46:52.366 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T05:46:52.366 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T05:46:52.366 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T05:46:52.366 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T05:46:52.366 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T05:46:52.366 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T05:46:52.366 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T05:46:52.366 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T05:46:52.366 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T05:46:52.366 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T05:46:52.366 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T05:46:52.366 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T05:46:52.366 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T05:46:52.366 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T05:46:52.367 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T05:46:52.367 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T05:46:52.367 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T05:46:52.367 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T05:46:52.367 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T05:46:52.367 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T05:46:52.367 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T05:46:52.367 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T05:46:52.367 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T05:46:52.367 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T05:46:52.367 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.inplace_update_support: 0 2024-09-21T05:46:52.367 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T05:46:52.367 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T05:46:52.367 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T05:46:52.367 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T05:46:52.368 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.bloom_locality: 0 2024-09-21T05:46:52.368 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.max_successive_merges: 0 2024-09-21T05:46:52.368 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T05:46:52.368 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T05:46:52.368 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T05:46:52.368 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T05:46:52.368 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.ttl: 2592000 2024-09-21T05:46:52.368 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T05:46:52.368 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T05:46:52.368 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T05:46:52.368 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.enable_blob_files: false 2024-09-21T05:46:52.368 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.min_blob_size: 0 2024-09-21T05:46:52.368 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T05:46:52.368 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T05:46:52.368 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T05:46:52.369 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T05:46:52.369 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T05:46:52.369 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T05:46:52.369 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T05:46:52.369 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T05:46:52.369 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-21T05:46:52.369 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-21T05:46:52.369 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 94e8168d-7b2a-4516-a77a-4deb2a874c5d 2024-09-21T05:46:52.369 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726897612066499, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-21T05:46:52.369 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-21T05:46:52.369 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726897612067664, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 91089, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 283, "table_properties": {"data_size": 89130, "index_size": 279, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11597, "raw_average_key_size": 47, "raw_value_size": 82538, "raw_average_value_size": 341, "num_data_blocks": 12, "num_entries": 242, "num_filter_entries": 242, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726897612, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "94e8168d-7b2a-4516-a77a-4deb2a874c5d", "db_session_id": "ZXLCPD944ELJW4G20MX0", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-21T05:46:52.369 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726897612067774, "job": 1, "event": "recovery_finished"} 2024-09-21T05:46:52.369 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-21T05:46:52.369 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T05:46:52.370 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559b5faa4a80 2024-09-21T05:46:52.370 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: DB pointer 0x559b6084a000 2024-09-21T05:46:52.370 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T05:46:52.370 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T05:46:52.370 INFO:journalctl@ceph.mon.a.smithi117.stdout: ** DB Stats ** 2024-09-21T05:46:52.370 INFO:journalctl@ceph.mon.a.smithi117.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T05:46:52.370 INFO:journalctl@ceph.mon.a.smithi117.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T05:46:52.370 INFO:journalctl@ceph.mon.a.smithi117.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T05:46:52.370 INFO:journalctl@ceph.mon.a.smithi117.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T05:46:52.370 INFO:journalctl@ceph.mon.a.smithi117.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T05:46:52.370 INFO:journalctl@ceph.mon.a.smithi117.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T05:46:52.370 INFO:journalctl@ceph.mon.a.smithi117.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T05:46:52.370 INFO:journalctl@ceph.mon.a.smithi117.stdout: 2024-09-21T05:46:52.370 INFO:journalctl@ceph.mon.a.smithi117.stdout: ** Compaction Stats [default] ** 2024-09-21T05:46:52.371 INFO:journalctl@ceph.mon.a.smithi117.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T05:46:52.371 INFO:journalctl@ceph.mon.a.smithi117.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T05:46:52.371 INFO:journalctl@ceph.mon.a.smithi117.stdout: L0 2/0 90.81 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 118.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T05:46:52.371 INFO:journalctl@ceph.mon.a.smithi117.stdout: Sum 2/0 90.81 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 118.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T05:46:52.371 INFO:journalctl@ceph.mon.a.smithi117.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 118.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T05:46:52.371 INFO:journalctl@ceph.mon.a.smithi117.stdout: 2024-09-21T05:46:52.371 INFO:journalctl@ceph.mon.a.smithi117.stdout: ** Compaction Stats [default] ** 2024-09-21T05:46:52.371 INFO:journalctl@ceph.mon.a.smithi117.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T05:46:52.371 INFO:journalctl@ceph.mon.a.smithi117.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T05:46:52.371 INFO:journalctl@ceph.mon.a.smithi117.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 118.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T05:46:52.371 INFO:journalctl@ceph.mon.a.smithi117.stdout: 2024-09-21T05:46:52.371 INFO:journalctl@ceph.mon.a.smithi117.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T05:46:52.371 INFO:journalctl@ceph.mon.a.smithi117.stdout: 2024-09-21T05:46:52.371 INFO:journalctl@ceph.mon.a.smithi117.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T05:46:52.372 INFO:journalctl@ceph.mon.a.smithi117.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T05:46:52.372 INFO:journalctl@ceph.mon.a.smithi117.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T05:46:52.372 INFO:journalctl@ceph.mon.a.smithi117.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T05:46:52.372 INFO:journalctl@ceph.mon.a.smithi117.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T05:46:52.372 INFO:journalctl@ceph.mon.a.smithi117.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T05:46:52.372 INFO:journalctl@ceph.mon.a.smithi117.stdout: Cumulative compaction: 0.00 GB write, 26.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T05:46:52.372 INFO:journalctl@ceph.mon.a.smithi117.stdout: Interval compaction: 0.00 GB write, 26.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T05:46:52.372 INFO:journalctl@ceph.mon.a.smithi117.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T05:46:52.372 INFO:journalctl@ceph.mon.a.smithi117.stdout: Block cache BinnedLRUCache@0x559b5ba45610#2 capacity: 512.00 MB usage: 1.31 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-21T05:46:52.372 INFO:journalctl@ceph.mon.a.smithi117.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.84 KB,0.000160933%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-09-21T05:46:52.372 INFO:journalctl@ceph.mon.a.smithi117.stdout: 2024-09-21T05:46:52.372 INFO:journalctl@ceph.mon.a.smithi117.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T05:46:52.372 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: starting mon.a rank 0 at public addrs [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] at bind addrs [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid ac8581a2-77dc-11ef-baf5-efdc52797490 2024-09-21T05:46:52.372 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: mon.a@-1(???) e1 preinit fsid ac8581a2-77dc-11ef-baf5-efdc52797490 2024-09-21T05:46:52.373 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: mon.a@-1(???).mds e1 new map 2024-09-21T05:46:52.373 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: mon.a@-1(???).mds e1 print_map 2024-09-21T05:46:52.373 INFO:journalctl@ceph.mon.a.smithi117.stdout: e1 2024-09-21T05:46:52.373 INFO:journalctl@ceph.mon.a.smithi117.stdout: btime 2024-09-21T05:46:47:021637+0000 2024-09-21T05:46:52.373 INFO:journalctl@ceph.mon.a.smithi117.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T05:46:52.373 INFO:journalctl@ceph.mon.a.smithi117.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:46:52.373 INFO:journalctl@ceph.mon.a.smithi117.stdout: legacy client fscid: -1 2024-09-21T05:46:52.373 INFO:journalctl@ceph.mon.a.smithi117.stdout: 2024-09-21T05:46:52.373 INFO:journalctl@ceph.mon.a.smithi117.stdout: No filesystems configured 2024-09-21T05:46:52.373 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T05:46:52.373 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T05:46:52.373 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T05:46:52.373 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T05:46:52.373 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-21T05:46:52.373 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 bash[30928]: 9aed4354fc168f03a339a36fb1a7f3641121e277dc139edf5959fedb82e767ee 2024-09-21T05:46:52.374 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 systemd[1]: Started Ceph mon.a for ac8581a2-77dc-11ef-baf5-efdc52797490. 2024-09-21T05:46:52.374 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T05:46:52.374 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: monmap epoch 1 2024-09-21T05:46:52.374 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: fsid ac8581a2-77dc-11ef-baf5-efdc52797490 2024-09-21T05:46:52.374 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: last_changed 2024-09-21T05:46:44.194497+0000 2024-09-21T05:46:52.374 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: created 2024-09-21T05:46:44.194497+0000 2024-09-21T05:46:52.374 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: min_mon_release 19 (squid) 2024-09-21T05:46:52.374 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: election_strategy: 1 2024-09-21T05:46:52.374 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: 0: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.a 2024-09-21T05:46:52.374 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: fsmap 2024-09-21T05:46:52.374 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T05:46:52.374 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:52 smithi117 ceph-mon[30947]: mgrmap e1: no daemons active 2024-09-21T05:46:53.162 INFO:teuthology.orchestra.run.smithi117.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-21T05:46:53.184 INFO:teuthology.orchestra.run.smithi117.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-21T05:46:53.184 INFO:teuthology.orchestra.run.smithi117.stdout:Creating mgr... 2024-09-21T05:46:53.184 INFO:teuthology.orchestra.run.smithi117.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-21T05:46:53.185 INFO:teuthology.orchestra.run.smithi117.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-21T05:46:53.433 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mgr.x 2024-09-21T05:46:53.433 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Failed to reset failed state of unit ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mgr.x.service: Unit ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mgr.x.service not loaded. 2024-09-21T05:46:53.588 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-ac8581a2-77dc-11ef-baf5-efdc52797490.target.wants/ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mgr.x.service → /etc/systemd/system/ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@.service. 2024-09-21T05:46:53.937 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:53 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/277917824' entity='client.admin' 2024-09-21T05:46:54.154 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T05:46:54.154 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout disabled 2024-09-21T05:46:54.161 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T05:46:54.161 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout inactive 2024-09-21T05:46:54.161 INFO:teuthology.orchestra.run.smithi117.stdout:firewalld.service is not enabled 2024-09-21T05:46:54.161 INFO:teuthology.orchestra.run.smithi117.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T05:46:54.169 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T05:46:54.169 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout disabled 2024-09-21T05:46:54.177 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T05:46:54.177 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout inactive 2024-09-21T05:46:54.177 INFO:teuthology.orchestra.run.smithi117.stdout:firewalld.service is not enabled 2024-09-21T05:46:54.177 INFO:teuthology.orchestra.run.smithi117.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-21T05:46:54.177 INFO:teuthology.orchestra.run.smithi117.stdout:Waiting for mgr to start... 2024-09-21T05:46:54.177 INFO:teuthology.orchestra.run.smithi117.stdout:Waiting for mgr... 2024-09-21T05:46:54.277 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:46:54 smithi117 systemd[1]: Started Ceph mgr.x for ac8581a2-77dc-11ef-baf5-efdc52797490. 2024-09-21T05:46:55.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:54 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/1764225024' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T05:46:55.267 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-09-21T05:46:55.267 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout { 2024-09-21T05:46:55.267 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "fsid": "ac8581a2-77dc-11ef-baf5-efdc52797490", 2024-09-21T05:46:55.267 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T05:46:55.267 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T05:46:55.267 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T05:46:55.267 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T05:46:55.267 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-09-21T05:46:55.267 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T05:46:55.267 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T05:46:55.267 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 0 2024-09-21T05:46:55.267 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-09-21T05:46:55.268 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T05:46:55.268 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T05:46:55.268 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-09-21T05:46:55.268 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-21T05:46:55.268 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T05:46:55.268 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T05:46:55.268 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T05:46:55.268 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T05:46:55.268 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-09-21T05:46:55.268 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T05:46:55.268 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T05:46:55.268 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T05:46:55.268 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T05:46:55.268 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T05:46:55.271 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T05:46:55.272 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T05:46:55.272 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T05:46:55.272 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-09-21T05:46:55.272 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T05:46:55.272 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T05:46:55.272 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T05:46:55.272 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T05:46:55.272 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T05:46:55.272 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T05:46:55.272 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T05:46:55.272 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T05:46:55.272 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T05:46:55.273 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-09-21T05:46:55.273 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T05:46:55.273 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T05:46:55.273 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T05:46:47:021637+0000", 2024-09-21T05:46:55.273 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T05:46:55.273 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T05:46:55.273 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-09-21T05:46:55.273 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T05:46:55.273 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T05:46:55.273 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T05:46:55.273 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T05:46:55.273 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T05:46:55.273 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T05:46:55.273 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T05:46:55.273 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-09-21T05:46:55.274 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T05:46:55.274 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-09-21T05:46:55.274 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T05:46:55.274 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T05:46:55.274 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T05:46:47.022435+0000", 2024-09-21T05:46:55.274 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T05:46:55.274 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-09-21T05:46:55.274 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T05:46:55.274 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout } 2024-09-21T05:46:55.274 INFO:teuthology.orchestra.run.smithi117.stdout:mgr not available, waiting (1/15)... 2024-09-21T05:46:58.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:57 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3791437899' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T05:46:58.381 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-09-21T05:46:58.381 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout { 2024-09-21T05:46:58.381 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "fsid": "ac8581a2-77dc-11ef-baf5-efdc52797490", 2024-09-21T05:46:58.381 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T05:46:58.381 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T05:46:58.381 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T05:46:58.381 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T05:46:58.381 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-09-21T05:46:58.381 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T05:46:58.381 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T05:46:58.381 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 0 2024-09-21T05:46:58.381 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-09-21T05:46:58.381 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T05:46:58.382 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T05:46:58.382 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-09-21T05:46:58.382 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-21T05:46:58.382 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T05:46:58.382 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T05:46:58.382 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T05:46:58.382 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T05:46:58.382 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-09-21T05:46:58.382 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T05:46:58.382 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T05:46:58.382 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T05:46:58.382 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T05:46:58.382 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T05:46:58.382 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T05:46:58.383 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T05:46:58.383 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T05:46:58.385 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-09-21T05:46:58.385 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T05:46:58.386 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T05:46:58.386 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T05:46:58.386 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T05:46:58.386 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T05:46:58.386 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T05:46:58.386 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T05:46:58.386 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T05:46:58.386 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T05:46:58.386 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-09-21T05:46:58.386 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T05:46:58.386 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T05:46:58.386 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T05:46:47:021637+0000", 2024-09-21T05:46:58.386 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T05:46:58.386 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T05:46:58.386 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-09-21T05:46:58.387 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T05:46:58.387 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T05:46:58.387 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T05:46:58.387 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T05:46:58.387 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T05:46:58.387 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T05:46:58.387 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T05:46:58.387 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-09-21T05:46:58.387 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T05:46:58.387 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-09-21T05:46:58.387 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T05:46:58.387 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T05:46:58.387 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T05:46:47.022435+0000", 2024-09-21T05:46:58.387 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T05:46:58.387 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-09-21T05:46:58.387 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T05:46:58.387 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout } 2024-09-21T05:46:58.388 INFO:teuthology.orchestra.run.smithi117.stdout:mgr not available, waiting (2/15)... 2024-09-21T05:46:59.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:59 smithi117 ceph-mon[30947]: Activating manager daemon x 2024-09-21T05:46:59.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:59 smithi117 ceph-mon[30947]: mgrmap e2: x(active, starting, since 0.00389558s) 2024-09-21T05:46:59.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:59 smithi117 ceph-mon[30947]: from='mgr.14100 172.21.15.117:0/1867632054' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T05:46:59.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:59 smithi117 ceph-mon[30947]: from='mgr.14100 172.21.15.117:0/1867632054' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T05:46:59.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:59 smithi117 ceph-mon[30947]: from='mgr.14100 172.21.15.117:0/1867632054' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T05:46:59.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:59 smithi117 ceph-mon[30947]: from='mgr.14100 172.21.15.117:0/1867632054' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T05:46:59.528 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:59 smithi117 ceph-mon[30947]: from='mgr.14100 172.21.15.117:0/1867632054' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T05:46:59.528 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:59 smithi117 ceph-mon[30947]: Manager daemon x is now available 2024-09-21T05:46:59.528 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:59 smithi117 ceph-mon[30947]: from='mgr.14100 172.21.15.117:0/1867632054' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T05:46:59.528 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:59 smithi117 ceph-mon[30947]: from='mgr.14100 172.21.15.117:0/1867632054' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T05:46:59.528 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:59 smithi117 ceph-mon[30947]: from='mgr.14100 172.21.15.117:0/1867632054' entity='mgr.x' 2024-09-21T05:46:59.528 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:59 smithi117 ceph-mon[30947]: from='mgr.14100 172.21.15.117:0/1867632054' entity='mgr.x' 2024-09-21T05:46:59.528 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:46:59 smithi117 ceph-mon[30947]: from='mgr.14100 172.21.15.117:0/1867632054' entity='mgr.x' 2024-09-21T05:47:01.512 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:01 smithi117 ceph-mon[30947]: mgrmap e3: x(active, since 1.00783s) 2024-09-21T05:47:01.512 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:01 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2753632378' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T05:47:01.677 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-09-21T05:47:01.677 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout { 2024-09-21T05:47:01.678 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "fsid": "ac8581a2-77dc-11ef-baf5-efdc52797490", 2024-09-21T05:47:01.678 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T05:47:01.678 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T05:47:01.678 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T05:47:01.678 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T05:47:01.678 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-09-21T05:47:01.678 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T05:47:01.678 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T05:47:01.678 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 0 2024-09-21T05:47:01.678 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-09-21T05:47:01.678 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T05:47:01.678 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T05:47:01.678 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-09-21T05:47:01.679 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-09-21T05:47:01.679 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T05:47:01.679 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T05:47:01.679 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T05:47:01.679 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T05:47:01.679 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-09-21T05:47:01.679 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T05:47:01.679 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T05:47:01.679 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T05:47:01.679 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T05:47:01.679 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T05:47:01.682 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T05:47:01.683 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T05:47:01.683 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T05:47:01.683 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-09-21T05:47:01.683 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T05:47:01.683 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T05:47:01.683 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T05:47:01.683 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T05:47:01.683 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T05:47:01.683 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T05:47:01.683 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T05:47:01.683 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T05:47:01.683 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T05:47:01.683 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-09-21T05:47:01.683 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T05:47:01.684 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T05:47:01.684 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T05:46:47:021637+0000", 2024-09-21T05:47:01.684 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T05:47:01.684 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T05:47:01.684 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-09-21T05:47:01.684 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T05:47:01.684 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T05:47:01.684 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T05:47:01.684 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T05:47:01.684 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T05:47:01.684 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T05:47:01.684 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T05:47:01.684 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-09-21T05:47:01.684 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T05:47:01.684 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-09-21T05:47:01.685 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T05:47:01.685 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T05:47:01.685 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T05:46:47.022435+0000", 2024-09-21T05:47:01.685 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T05:47:01.685 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-09-21T05:47:01.685 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T05:47:01.685 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout } 2024-09-21T05:47:01.685 INFO:teuthology.orchestra.run.smithi117.stdout:mgr is available 2024-09-21T05:47:02.526 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:02 smithi117 ceph-mon[30947]: mgrmap e4: x(active, since 2s) 2024-09-21T05:47:02.808 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-09-21T05:47:02.808 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T05:47:02.808 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout fsid = ac8581a2-77dc-11ef-baf5-efdc52797490 2024-09-21T05:47:02.808 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T05:47:02.809 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.117:3300,v1:172.21.15.117:6789] 2024-09-21T05:47:02.809 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T05:47:02.809 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T05:47:02.809 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T05:47:02.809 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T05:47:02.809 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-09-21T05:47:02.809 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T05:47:02.809 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T05:47:02.809 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-09-21T05:47:02.809 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T05:47:02.809 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T05:47:02.809 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T05:47:02.809 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T05:47:02.809 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T05:47:02.809 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T05:47:02.810 INFO:teuthology.orchestra.run.smithi117.stdout:Enabling cephadm module... 2024-09-21T05:47:03.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:03 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3267695419' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T05:47:04.426 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:04 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2164947281' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-21T05:47:04.427 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:04 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: ignoring --setuser ceph since I am not root 2024-09-21T05:47:04.427 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:04 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: ignoring --setgroup ceph since I am not root 2024-09-21T05:47:05.393 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:05 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2164947281' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-21T05:47:05.393 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:05 smithi117 ceph-mon[30947]: mgrmap e5: x(active, since 5s) 2024-09-21T05:47:06.044 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout { 2024-09-21T05:47:06.045 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-21T05:47:06.045 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T05:47:06.045 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T05:47:06.045 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T05:47:06.045 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout } 2024-09-21T05:47:06.045 INFO:teuthology.orchestra.run.smithi117.stdout:Waiting for the mgr to restart... 2024-09-21T05:47:06.045 INFO:teuthology.orchestra.run.smithi117.stdout:Waiting for mgr epoch 5... 2024-09-21T05:47:06.325 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:06 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2685731439' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T05:47:09.480 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:09 smithi117 ceph-mon[30947]: Active manager daemon x restarted 2024-09-21T05:47:09.480 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:09 smithi117 ceph-mon[30947]: Activating manager daemon x 2024-09-21T05:47:09.480 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:09 smithi117 ceph-mon[30947]: osdmap e2: 0 total, 0 up, 0 in 2024-09-21T05:47:09.480 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:09 smithi117 ceph-mon[30947]: mgrmap e6: x(active, starting, since 0.0036732s) 2024-09-21T05:47:09.480 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:09 smithi117 ceph-mon[30947]: from='mgr.14118 172.21.15.117:0/2629615445' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T05:47:09.480 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:09 smithi117 ceph-mon[30947]: from='mgr.14118 172.21.15.117:0/2629615445' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T05:47:09.481 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:09 smithi117 ceph-mon[30947]: from='mgr.14118 172.21.15.117:0/2629615445' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T05:47:09.481 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:09 smithi117 ceph-mon[30947]: from='mgr.14118 172.21.15.117:0/2629615445' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T05:47:09.481 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:09 smithi117 ceph-mon[30947]: from='mgr.14118 172.21.15.117:0/2629615445' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T05:47:09.481 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:09 smithi117 ceph-mon[30947]: Manager daemon x is now available 2024-09-21T05:47:10.641 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout { 2024-09-21T05:47:10.641 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-21T05:47:10.641 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T05:47:10.641 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout } 2024-09-21T05:47:10.642 INFO:teuthology.orchestra.run.smithi117.stdout:mgr epoch 5 is available 2024-09-21T05:47:10.642 INFO:teuthology.orchestra.run.smithi117.stdout:Verifying orchestrator module is enabled... 2024-09-21T05:47:11.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:10 smithi117 ceph-mon[30947]: from='mgr.14118 172.21.15.117:0/2629615445' entity='mgr.x' 2024-09-21T05:47:11.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:10 smithi117 ceph-mon[30947]: from='mgr.14118 172.21.15.117:0/2629615445' entity='mgr.x' 2024-09-21T05:47:11.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:10 smithi117 ceph-mon[30947]: Found migration_current of "None". Setting to last migration. 2024-09-21T05:47:11.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:10 smithi117 ceph-mon[30947]: from='mgr.14118 172.21.15.117:0/2629615445' entity='mgr.x' 2024-09-21T05:47:11.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:10 smithi117 ceph-mon[30947]: from='mgr.14118 172.21.15.117:0/2629615445' entity='mgr.x' 2024-09-21T05:47:11.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:10 smithi117 ceph-mon[30947]: from='mgr.14118 172.21.15.117:0/2629615445' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:47:11.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:10 smithi117 ceph-mon[30947]: from='mgr.14118 172.21.15.117:0/2629615445' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:47:11.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:10 smithi117 ceph-mon[30947]: from='mgr.14118 172.21.15.117:0/2629615445' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T05:47:11.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:10 smithi117 ceph-mon[30947]: from='mgr.14118 172.21.15.117:0/2629615445' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T05:47:11.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:10 smithi117 ceph-mon[30947]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-21T05:47:11.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:10 smithi117 ceph-mon[30947]: mgrmap e7: x(active, since 1.00804s) 2024-09-21T05:47:11.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:10 smithi117 ceph-mon[30947]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-21T05:47:11.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:10 smithi117 ceph-mon[30947]: from='mgr.14118 172.21.15.117:0/2629615445' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:47:11.279 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:10 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: 2024-09-21T05:47:10.899+0000 7f0a324b6640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:05:47:10] ENGINE Error in HTTPServer.serve 2024-09-21T05:47:11.279 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:10 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: Traceback (most recent call last): 2024-09-21T05:47:11.279 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:10 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T05:47:11.279 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:10 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: self._connections.run(self.expiration_interval) 2024-09-21T05:47:11.279 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:10 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T05:47:11.279 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:10 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: self._run(expiration_interval) 2024-09-21T05:47:11.279 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:10 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T05:47:11.279 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:10 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T05:47:11.279 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:10 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T05:47:11.279 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:10 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T05:47:11.279 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:10 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T05:47:11.279 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:10 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: s = self.context.wrap_socket( 2024-09-21T05:47:11.279 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:10 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T05:47:11.279 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:10 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: return self.sslsocket_class._create( 2024-09-21T05:47:11.280 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:10 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T05:47:11.280 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:10 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: self.do_handshake() 2024-09-21T05:47:11.280 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:10 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T05:47:11.280 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:10 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: self._sslobj.do_handshake() 2024-09-21T05:47:11.280 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:10 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T05:47:11.280 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:10 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: 2024-09-21T05:47:12.211 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:11 smithi117 ceph-mon[30947]: [21/Sep/2024:05:47:10] ENGINE Bus STARTING 2024-09-21T05:47:12.211 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:11 smithi117 ceph-mon[30947]: [21/Sep/2024:05:47:10] ENGINE Serving on http://172.21.15.117:8765 2024-09-21T05:47:12.211 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:11 smithi117 ceph-mon[30947]: [21/Sep/2024:05:47:10] ENGINE Serving on https://172.21.15.117:7150 2024-09-21T05:47:12.211 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:11 smithi117 ceph-mon[30947]: [21/Sep/2024:05:47:10] ENGINE Bus STARTED 2024-09-21T05:47:12.211 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:11 smithi117 ceph-mon[30947]: [21/Sep/2024:05:47:10] ENGINE Error in HTTPServer.serve 2024-09-21T05:47:12.211 INFO:journalctl@ceph.mon.a.smithi117.stdout: Traceback (most recent call last): 2024-09-21T05:47:12.211 INFO:journalctl@ceph.mon.a.smithi117.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T05:47:12.211 INFO:journalctl@ceph.mon.a.smithi117.stdout: self._connections.run(self.expiration_interval) 2024-09-21T05:47:12.211 INFO:journalctl@ceph.mon.a.smithi117.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T05:47:12.212 INFO:journalctl@ceph.mon.a.smithi117.stdout: self._run(expiration_interval) 2024-09-21T05:47:12.212 INFO:journalctl@ceph.mon.a.smithi117.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T05:47:12.212 INFO:journalctl@ceph.mon.a.smithi117.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T05:47:12.212 INFO:journalctl@ceph.mon.a.smithi117.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T05:47:12.212 INFO:journalctl@ceph.mon.a.smithi117.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T05:47:12.212 INFO:journalctl@ceph.mon.a.smithi117.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T05:47:12.212 INFO:journalctl@ceph.mon.a.smithi117.stdout: s = self.context.wrap_socket( 2024-09-21T05:47:12.212 INFO:journalctl@ceph.mon.a.smithi117.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T05:47:12.212 INFO:journalctl@ceph.mon.a.smithi117.stdout: return self.sslsocket_class._create( 2024-09-21T05:47:12.212 INFO:journalctl@ceph.mon.a.smithi117.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T05:47:12.212 INFO:journalctl@ceph.mon.a.smithi117.stdout: self.do_handshake() 2024-09-21T05:47:12.212 INFO:journalctl@ceph.mon.a.smithi117.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T05:47:12.212 INFO:journalctl@ceph.mon.a.smithi117.stdout: self._sslobj.do_handshake() 2024-09-21T05:47:12.212 INFO:journalctl@ceph.mon.a.smithi117.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T05:47:12.213 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:11 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/4258425967' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "orchestrator"} : dispatch 2024-09-21T05:47:12.447 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr module 'orchestrator' is already enabled (always-on) 2024-09-21T05:47:12.447 INFO:teuthology.orchestra.run.smithi117.stdout:Setting orchestrator backend to cephadm... 2024-09-21T05:47:13.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:12 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/4258425967' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "orchestrator"}]': finished 2024-09-21T05:47:13.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:12 smithi117 ceph-mon[30947]: mgrmap e8: x(active, since 2s) 2024-09-21T05:47:14.230 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:14 smithi117 ceph-mon[30947]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:47:14.230 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:14 smithi117 ceph-mon[30947]: from='mgr.14118 172.21.15.117:0/2629615445' entity='mgr.x' 2024-09-21T05:47:14.230 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:14 smithi117 ceph-mon[30947]: from='mgr.14118 172.21.15.117:0/2629615445' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:47:14.766 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-21T05:47:14.766 INFO:teuthology.orchestra.run.smithi117.stdout:Generating ssh key... 2024-09-21T05:47:15.526 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:15 smithi117 ceph-mon[30947]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:47:15.861 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:15 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: Generating public/private rsa key pair. 2024-09-21T05:47:15.861 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:15 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: Your identification has been saved in /tmp/tmp6y0axu5a/key 2024-09-21T05:47:15.861 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:15 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: Your public key has been saved in /tmp/tmp6y0axu5a/key.pub 2024-09-21T05:47:15.861 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:15 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: The key fingerprint is: 2024-09-21T05:47:15.861 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:15 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: SHA256:FlMZWNK6VlrP24JU3QNOIi3Wcf9gAmuVvLaUV6XR/ZE ceph-ac8581a2-77dc-11ef-baf5-efdc52797490 2024-09-21T05:47:15.861 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:15 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: The key's randomart image is: 2024-09-21T05:47:15.861 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:15 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: +---[RSA 3072]----+ 2024-09-21T05:47:15.861 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:15 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: | .+B=oo..=| 2024-09-21T05:47:15.861 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:15 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: | .*oB+o.E+| 2024-09-21T05:47:15.861 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:15 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: | +.= =+=+o| 2024-09-21T05:47:15.862 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:15 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: | .+o *+oo+| 2024-09-21T05:47:15.862 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:15 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: | S= * o o| 2024-09-21T05:47:15.862 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:15 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: | .+ . + | 2024-09-21T05:47:15.862 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:15 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: | . . . o | 2024-09-21T05:47:15.862 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:15 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: | . o . | 2024-09-21T05:47:15.862 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:15 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: | . | 2024-09-21T05:47:15.862 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:15 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: +----[SHA256]-----+ 2024-09-21T05:47:16.741 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:16 smithi117 ceph-mon[30947]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:47:16.741 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:16 smithi117 ceph-mon[30947]: Generating ssh key... 2024-09-21T05:47:16.741 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:16 smithi117 ceph-mon[30947]: from='mgr.14118 172.21.15.117:0/2629615445' entity='mgr.x' 2024-09-21T05:47:16.741 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:16 smithi117 ceph-mon[30947]: from='mgr.14118 172.21.15.117:0/2629615445' entity='mgr.x' 2024-09-21T05:47:17.278 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC61nzOkWGnYTUC4sg4lgP+1jwRARIGIFVCFsv2kMULDHnBdpTDaBcwliYZClA9QLka/ypDWa9TM/PJSnx1HDVBUesyvVd+IetrL28CDl0SUPQaFJ8jaFw2c4cxAbMJPOeKu0u+EMtO0hVpVc7+Dtl9+YVfVjtgYyqByFJaAFkYl6osm4kceA99QI6azOB9T2G/mosm32evt2qjwU6BNbYHu+rrSOvRDPrZx2KKq2jNajwrvG2y0rKaLQPhu96TVYXaDyy4dbgXxitk1XY7QvSTuX8qw+i7KvFIipVYl+X8QkHQ+mTs3xbJ5wDIW2kbZHHgARUlb7ur3eZiAn7cbcV8H9T1J/fEyJnAP6qPhWUGvCTJS442iowkgEUYRCQeKZ4mz4GYXHjoAR6I16tLVHOzB32UbrQGBnpDxoFQcPYjdNiw6aJKmArzzOnntfRCDcbDCK1lmrt1lEzPJ53+zoWf/QUb7IqSxhw2vi6U+X2yVuW3+Q5KNyg0P+tBpgAsrb8= ceph-ac8581a2-77dc-11ef-baf5-efdc52797490 2024-09-21T05:47:17.278 INFO:teuthology.orchestra.run.smithi117.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-21T05:47:17.278 INFO:teuthology.orchestra.run.smithi117.stdout:Adding key to root@localhost authorized_keys... 2024-09-21T05:47:17.283 INFO:teuthology.orchestra.run.smithi117.stdout:Adding host smithi117... 2024-09-21T05:47:18.011 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:17 smithi117 ceph-mon[30947]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:47:18.661 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:18 smithi117 ceph-mon[30947]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi117", "addr": "172.21.15.117", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:47:19.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:19 smithi117 ceph-mon[30947]: Deploying cephadm binary to smithi117 2024-09-21T05:47:21.578 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout Added host 'smithi117' with addr '172.21.15.117' 2024-09-21T05:47:21.579 INFO:teuthology.orchestra.run.smithi117.stdout:Deploying unmanaged mon service... 2024-09-21T05:47:22.131 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:22 smithi117 ceph-mon[30947]: from='mgr.14118 172.21.15.117:0/2629615445' entity='mgr.x' 2024-09-21T05:47:22.131 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:22 smithi117 ceph-mon[30947]: Added host smithi117 2024-09-21T05:47:22.131 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:22 smithi117 ceph-mon[30947]: from='mgr.14118 172.21.15.117:0/2629615445' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:47:22.829 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-21T05:47:22.829 INFO:teuthology.orchestra.run.smithi117.stdout:Deploying unmanaged mgr service... 2024-09-21T05:47:23.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:23 smithi117 ceph-mon[30947]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:47:23.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:23 smithi117 ceph-mon[30947]: Saving service mon spec with placement count:5 2024-09-21T05:47:23.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:23 smithi117 ceph-mon[30947]: from='mgr.14118 172.21.15.117:0/2629615445' entity='mgr.x' 2024-09-21T05:47:24.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:24 smithi117 ceph-mon[30947]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:47:24.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:24 smithi117 ceph-mon[30947]: Saving service mgr spec with placement count:2 2024-09-21T05:47:24.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:24 smithi117 ceph-mon[30947]: from='mgr.14118 172.21.15.117:0/2629615445' entity='mgr.x' 2024-09-21T05:47:24.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:24 smithi117 ceph-mon[30947]: from='mgr.14118 172.21.15.117:0/2629615445' entity='mgr.x' 2024-09-21T05:47:24.749 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-21T05:47:26.095 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:25 smithi117 ceph-mon[30947]: from='mgr.14118 172.21.15.117:0/2629615445' entity='mgr.x' 2024-09-21T05:47:26.096 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:25 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3884309784' entity='client.admin' 2024-09-21T05:47:27.647 INFO:teuthology.orchestra.run.smithi117.stdout:Enabling the dashboard module... 2024-09-21T05:47:27.915 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:27 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/4080924334' entity='client.admin' 2024-09-21T05:47:27.915 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:27 smithi117 ceph-mon[30947]: from='mgr.14118 172.21.15.117:0/2629615445' entity='mgr.x' 2024-09-21T05:47:29.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:28 smithi117 ceph-mon[30947]: from='mgr.14118 172.21.15.117:0/2629615445' entity='mgr.x' 2024-09-21T05:47:29.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:28 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3178988461' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-21T05:47:29.027 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:28 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: ignoring --setuser ceph since I am not root 2024-09-21T05:47:29.027 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:28 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: ignoring --setgroup ceph since I am not root 2024-09-21T05:47:29.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:29 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3178988461' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-21T05:47:29.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:29 smithi117 ceph-mon[30947]: mgrmap e9: x(active, since 19s) 2024-09-21T05:47:30.746 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:30 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2997007553' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T05:47:31.049 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout { 2024-09-21T05:47:31.050 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-21T05:47:31.050 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T05:47:31.050 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T05:47:31.050 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T05:47:31.050 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout } 2024-09-21T05:47:31.050 INFO:teuthology.orchestra.run.smithi117.stdout:Waiting for the mgr to restart... 2024-09-21T05:47:31.050 INFO:teuthology.orchestra.run.smithi117.stdout:Waiting for mgr epoch 9... 2024-09-21T05:47:34.171 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:33 smithi117 ceph-mon[30947]: Active manager daemon x restarted 2024-09-21T05:47:34.171 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:33 smithi117 ceph-mon[30947]: Activating manager daemon x 2024-09-21T05:47:34.171 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:33 smithi117 ceph-mon[30947]: osdmap e3: 0 total, 0 up, 0 in 2024-09-21T05:47:34.171 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:33 smithi117 ceph-mon[30947]: mgrmap e10: x(active, starting, since 0.00410164s) 2024-09-21T05:47:34.171 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:33 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T05:47:34.171 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:33 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T05:47:34.171 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:33 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T05:47:34.171 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:33 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T05:47:34.171 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:33 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T05:47:34.172 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:33 smithi117 ceph-mon[30947]: Manager daemon x is now available 2024-09-21T05:47:35.097 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:34 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:47:35.097 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:34 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T05:47:35.097 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:34 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T05:47:35.097 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:34 smithi117 ceph-mon[30947]: mgrmap e11: x(active, since 1.00776s) 2024-09-21T05:47:35.778 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:35 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: 2024-09-21T05:47:35.418+0000 7f27774b5640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:05:47:35] ENGINE Error in HTTPServer.serve 2024-09-21T05:47:35.778 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:35 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: Traceback (most recent call last): 2024-09-21T05:47:35.778 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:35 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T05:47:35.778 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:35 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: self._connections.run(self.expiration_interval) 2024-09-21T05:47:35.778 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:35 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T05:47:35.778 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:35 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: self._run(expiration_interval) 2024-09-21T05:47:35.778 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:35 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T05:47:35.778 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:35 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T05:47:35.778 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:35 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T05:47:35.778 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:35 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T05:47:35.778 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:35 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T05:47:35.778 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:35 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: s = self.context.wrap_socket( 2024-09-21T05:47:35.779 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:35 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T05:47:35.779 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:35 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: return self.sslsocket_class._create( 2024-09-21T05:47:35.779 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:35 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T05:47:35.779 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:35 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: self.do_handshake() 2024-09-21T05:47:35.779 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:35 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T05:47:35.779 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:35 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: self._sslobj.do_handshake() 2024-09-21T05:47:35.779 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:35 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T05:47:35.779 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:47:35 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: 2024-09-21T05:47:35.817 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout { 2024-09-21T05:47:35.817 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-21T05:47:35.817 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T05:47:35.817 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout } 2024-09-21T05:47:35.817 INFO:teuthology.orchestra.run.smithi117.stdout:mgr epoch 9 is available 2024-09-21T05:47:35.817 INFO:teuthology.orchestra.run.smithi117.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-21T05:47:36.723 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:36 smithi117 ceph-mon[30947]: [21/Sep/2024:05:47:35] ENGINE Bus STARTING 2024-09-21T05:47:36.723 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:36 smithi117 ceph-mon[30947]: [21/Sep/2024:05:47:35] ENGINE Serving on http://172.21.15.117:8765 2024-09-21T05:47:36.723 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:36 smithi117 ceph-mon[30947]: [21/Sep/2024:05:47:35] ENGINE Serving on https://172.21.15.117:7150 2024-09-21T05:47:36.723 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:36 smithi117 ceph-mon[30947]: [21/Sep/2024:05:47:35] ENGINE Bus STARTED 2024-09-21T05:47:36.723 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:36 smithi117 ceph-mon[30947]: [21/Sep/2024:05:47:35] ENGINE Error in HTTPServer.serve 2024-09-21T05:47:36.723 INFO:journalctl@ceph.mon.a.smithi117.stdout: Traceback (most recent call last): 2024-09-21T05:47:36.723 INFO:journalctl@ceph.mon.a.smithi117.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T05:47:36.723 INFO:journalctl@ceph.mon.a.smithi117.stdout: self._connections.run(self.expiration_interval) 2024-09-21T05:47:36.724 INFO:journalctl@ceph.mon.a.smithi117.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T05:47:36.724 INFO:journalctl@ceph.mon.a.smithi117.stdout: self._run(expiration_interval) 2024-09-21T05:47:36.724 INFO:journalctl@ceph.mon.a.smithi117.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T05:47:36.724 INFO:journalctl@ceph.mon.a.smithi117.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T05:47:36.724 INFO:journalctl@ceph.mon.a.smithi117.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T05:47:36.724 INFO:journalctl@ceph.mon.a.smithi117.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T05:47:36.724 INFO:journalctl@ceph.mon.a.smithi117.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T05:47:36.724 INFO:journalctl@ceph.mon.a.smithi117.stdout: s = self.context.wrap_socket( 2024-09-21T05:47:36.724 INFO:journalctl@ceph.mon.a.smithi117.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T05:47:36.724 INFO:journalctl@ceph.mon.a.smithi117.stdout: return self.sslsocket_class._create( 2024-09-21T05:47:36.724 INFO:journalctl@ceph.mon.a.smithi117.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T05:47:36.724 INFO:journalctl@ceph.mon.a.smithi117.stdout: self.do_handshake() 2024-09-21T05:47:36.724 INFO:journalctl@ceph.mon.a.smithi117.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T05:47:36.724 INFO:journalctl@ceph.mon.a.smithi117.stdout: self._sslobj.do_handshake() 2024-09-21T05:47:36.724 INFO:journalctl@ceph.mon.a.smithi117.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T05:47:36.724 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:36 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:47:36.725 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:36 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:47:37.670 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIFETCCAvmgAwIBAgIUZKYSkIXxUiYGfCD/He2U1B6QpJcwDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkyMTA1NDczNloXDTM0\nMDkyMjA1NDczNlowGDEWMBQGA1UEAwwNMTcyLjIxLjE1LjExNzCCAiIwDQYJKoZI\nhvcNAQEBBQADggIPADCCAgoCggIBAMjLiAOLfRsrdyv2vrA6iy5YOedAWajjylgB\n6xKCVXpVQSf2T03gncK21hgl77vpnIJhBh4NHYl7JVhWtxgHs2thfYVCltkQhG5Y\n67B1JmHw9/Xwz3QXtt2+fgeolqh01Kqi72j9Kadn2klnbH3z10a2vzB0kTj9BPtQ\npHpuAj/4k1MIHXMTgQDfC9F4y3zp+1JEqN5N7dFCQqjdOABG3PNVOCZh/EzQ8/Ts\nzNtHEYwd8d9BNBJX0vG3Lu7/oed4cs/ZObhdZ28IJ4LqioD06cHGFjRmYcei0yWJ\npr6A4sQ8/hR7fiW4Us8UqWFNUN6TjitNvXJaBSqHqDUdSMMx7eHd1h9VZQmmsr4P\nLIB+Cc7Ycb/qlAyzKMaUCJqVm2hgbfVNoWUz1QjiircyyRRXc/aLCMFNJih6Y30X\nzMB5tK7NdfMqwROm1OpNyod6WTajy0hJXCT7U1bain2aAZA6hVs7+XHZp9RTXhCQ\nwFQ5pTnQ2hSDDVRyTohRSu8qHAJc8519cJTNwLtutfFxEeP1+CPxvfhdB9sXtjTq\nxGqC6w+KyrFfvHRA6ntaW6XdzhqWPy+puclkjspAcLwrJyRTPhbLzOu/BXAAqc19\nPU8ZEfpyUk6TRjSBOKCZ2XQ0MJ4O+vd9xPY4HjDPWZmYKD5nWseuTTPp/OnFL7mx\ndV0NYrSNAgMBAAGjVDBSMEIGA1UdEQQ7MDmCHnNtaXRoaTExNy5mcm9udC5zZXBp\nYS5jZXBoLmNvbYIRZGFzaGJvYXJkX3NlcnZlcnOHBKwVD3UwDAYDVR0TAQH/BAIw\nADANBgkqhkiG9w0BAQsFAAOCAgEAjx8CeRIaLqSUDuNo8VvpptI67AtdpjNOqfTU\nzFzngbnulfytbiRZl+5EqwIdjZWrvBEXUhxwN6UwypAR4TuCEyl3nQ9TAWU6n1ww\nOG7JVVKHTF5jYH55lXOretDZeUPHFh7GYUcTPgd/SSzmLjvROn0Ib8Occik2te3y\nWil7Xi+mI6ARjG36gAh4sVF5n4wHLA9UOq5Kr9xosUYuHolU4OR3qXZVNsGLqH/M\nm0QovP80/bTUh9aHuHy7/DloQLo1FJh0ukLDm1kb0utUpVhe0F/+FgXm0+rh2Nq/\nE7pk8DDAWlWDmJTGLVZ4kc4pdp5B9ZUWRn8jrzG5WVs09Vx4g8whPb4ZA/UF1l6d\nCS7pwPlgH5Xx/zD/LkcszcTh/KthhOTz50nKR2d4wwnpN3QPKQB6k9uWbvlByfTJ\n+Y1kt0en7mfWvRRgrTnm+5dh11/iq6jOG7aWc/PgB6C8bBSJ9+h9PR7bAGdXN8/N\nmHSaA9Yy9C63yBHDEOc4U3DAxSdrrL32tcumBvcf9Xk2KsyEKpGXu1qgH28o54eY\n+G+Apdq71oHoiiN8YdJYPyeYTi0flZDr0QTjuHi3RSTRMSc71mnWeyaOONk5OD+u\nvedH1d1HQgpMrtZkRsb2qV1693yXPfjdUEf1EiFcz/subnKOnVnpTKGOaHBnP52n\nw9IuTRM=\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKAIBAAKCAgEAyMuIA4t9Gyt3K/a+sDqLLlg550BZqOPKWAHrEoJVelVBJ/ZP\nTeCdwrbWGCXvu+mcgmEGHg0diXslWFa3GAeza2F9hUKW2RCEbljrsHUmYfD39fDP\ndBe23b5+B6iWqHTUqqLvaP0pp2faSWdsffPXRra/MHSROP0E+1Ckem4CP/iTUwgd\ncxOBAN8L0XjLfOn7UkSo3k3t0UJCqN04AEbc81U4JmH8TNDz9OzM20cRjB3x30E0\nElfS8bcu7v+h53hyz9k5uF1nbwgnguqKgPTpwcYWNGZhx6LTJYmmvoDixDz+FHt+\nJbhSzxSpYU1Q3pOOK029cloFKoeoNR1IwzHt4d3WH1VlCaayvg8sgH4Jzthxv+qU\nDLMoxpQImpWbaGBt9U2hZTPVCOKKtzLJFFdz9osIwU0mKHpjfRfMwHm0rs118yrB\nE6bU6k3Kh3pZNqPLSElcJPtTVtqKfZoBkDqFWzv5cdmn1FNeEJDAVDmlOdDaFIMN\nVHJOiFFK7yocAlzznX1wlM3Au2618XER4/X4I/G9+F0H2xe2NOrEaoLrD4rKsV+8\ndEDqe1pbpd3OGpY/L6m5yWSOykBwvCsnJFM+FsvM678FcACpzX09TxkR+nJSTpNG\nNIE4oJnZdDQwng76933E9jgeMM9ZmZgoPmdax65NM+n86cUvubF1XQ1itI0CAwEA\nAQKCAgAGmRWHlEpQnb2FPs7mSI4xQCnq7OTlVVLdUHbSfWDkq69/6iqR8wsMDmj7\nkyeNA7CmYhzK2QB694ZzICpVCk+9fA8dVWJSkKXhYF+iRT81Ia9sL/UJKgxSvgMz\nKjmFII2qa781MnhvbUyxc8oPW3db9f5TT8lD16CVx4EYGsxNINRqeo5JLT9kcW/Y\nesEpAiFFg0cPQv1JDoXVOauw6qMxBcBaFoctWQJjLnLO08eCVKuRe/mDjso6MhxD\nMzg1PHJ9XPR2F++hqOFvflh9w055FRK+twyIfFO23I6TnPWWoDBzKNms0KE798BX\nF6zMG4ok5yGOA+I+kwWWdTdiXp4j6l58Q0A9zEPyXkzJEa7OYq97+2P4ynb0uupd\nLbk2ZB4c0wlsUSqTXewNxYaiKJWK4ibP7B+uSUeeso2GgNzNj94E3Lbx738upHmu\n/gzwGjfkB1j8edHffNDOyZczkSrGYLoz0fW8BEJk+EQAOOKZPFbqSRxbjMogOK5c\nPaX44YSH5oqCfZAn4N6wAbH3vFakJQSuLdmzPUIT8OpxsHAVDAHiIbzvqK4a2WM3\nPxwMUToAY2oo3Njwki1Ngepw2nbnVx6MAy0l8Txn80ox2LFFl8+kbNG/EJ+IIPjf\n1s1NxtT4hYSVwE1gzLcOnH+AWRaw5D+8k3fNcoqHz2p2fn8CgQKCAQEA/zho7XEQ\nVQRdo9QUGrIOxsYilWscHkhmHCDQBVOqerS/B3jcvukqT5wNHiDvp+ZtchuN+POl\n3j/+exFmUC1JBIs2scrjuNr2B08Vg9yBHq77XHDSnW17kY1R2FZ4K8OpCRqv+gEI\n+BpJ9zV5x5SV0RfrahsEdIDQ9971ZZNSRNlxCjKPJGlz0TorpEiApGFMoNgTzvFo\nLmR31Nl8WmxWuPDZTlJPsPqeU20pTTO14ZuhK9ARomxRufuLhqL4X2GclBRfj9Wc\neswGnof/7w+WXGNlKoI3IQygT3Tqvu8YfWnSh4/GwuqbQw+R5+76ggkDrnpkrvaK\nWf6PeS00pmlOTQKCAQEAyWiPJhhYHz+gpIGwY5hpK+XnSd9GPUt2D0h8ZF5QjSkF\nosr+qyRFswnN+MqnqVjvyIr/+2JtktFpydGigwGcgdG+tBDqobQJ1SeFulgTtfgx\nMu+A3tZOg2GrJP2igTGc3DMnX+mbCLc8n/cXcVBI1A2xB4OenE3dDld6kbBjmTZb\nZdHmgdtMfG2U5XTgCv3KOH1gig9srOR3uNnrJBB3TurEcGN/fR+HfxuVjLRgZLdV\n+eUq458ugsmhMUWcXIBP5R2Ebkp/93h43SVGQwTLQqmE9p+/CmQj5MDOrrczK9bD\ni0QShl0eirONp8UkNF4SWIyiRWGuA7aAlv5n4xSfQQKCAQAAoYEQ5TCnJSWeHA4Y\nqswCjcbUG7msEX2/IA2jlbtcJko6mH/jMOxb2gTmSCqPfbBTkeHd9aw5Sms0nqxk\nEliV3Eo8UkYR1Oi6x+VDuC3Rh5kzd0vt+7FMgM7s+/m8EfW/AgPhWFNPWbFJHSa5\n6ZIdPDPmxT5Tk2Xw1upAYAbaCLeQ0OMtsd0JbV1iQJOg1GS+GQqjflFFnBX3vrnh\n91aXinDfErEDvPrbNe/eMvyGHD5fv1nrzty82vRTVt/KTJF4hDxgPKTwYMS7a5ak\nXqN+gvhjDINaX4+d4qoZ4rlqxfIxR5e0L6RGnCk+K4cO/zq1GbRaAl4BLvHn2MbC\niowZAoIBAD/8tQXgVBbRayL4hOaPZFJq/xksEy+kCsT+PW7Ru/us3bqDjXcn1fD0\nm3iAQdX+iGeJKZxe//rWByr0LBI879EapuCkQzbOcRVEMisqklCMomzRRHsHUPhS\njkH5RqiO74vRwJ+rIvMY6+zSXqNEY27jTqvZeAmmfyVwS2N/y5TAfTcTAY2d82XQ\nlq5zfD5htN5Pjt0t21PuXc7fhOH3TUBIO9KM+dLxsmWCiPej0sAMswbqdL7OMsOg\ndVKnD+8fYMKkpw83IgKoRpXTosfl/QGXw8NqqxINEI47Zy6Ay4QQ/hz/qnb/nJVq\nSAXWDxOIE4IIZAFw8A0D7USLXMv7IEECggEBAOMzzi6hDgrx5JmnRuBpGPu5ns86\ngYEkxxdNpb6tZ0+K/TpuHNSMm5Rz9JQWJDK9XRXKtX6HGX29t3YlZ/7BGarINyIP\nTtKT72xXTo8X8y8Zf+35RBVeehyib/giK5m5BCVcC9bRD8+0oVN1j1W3/U4Oc7b2\npKKcNegnUJiYYT+EL2Qcv/DQTkziKngoX9PcQgEtJi7xcTyL6/fKU6DLvEG3GD7X\n1uY4WQ4BI1xENA/HUhWj12NUxSwpEzr6DbuQjV1jAy3tGtFzdl+qDv6Ex6sUnNRY\nzlDcXB/XQ86xIZGTuqCtUvt6LV5ePZTwXL1fDFgk5/hcUKhR6jUbq4YFupE=\n-----END RSA PRIVATE KEY-----\n"} 2024-09-21T05:47:37.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:37 smithi117 ceph-mon[30947]: mgrmap e12: x(active, since 2s) 2024-09-21T05:47:37.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:37 smithi117 ceph-mon[30947]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:47:39.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-21T05:47:39.898 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:39 smithi117 ceph-mon[30947]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:47:39.898 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:39 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:47:40.457 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-21T05:47:40.457 INFO:teuthology.orchestra.run.smithi117.stdout:Creating initial admin user... 2024-09-21T05:47:41.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:40 smithi117 ceph-mon[30947]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:47:41.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:40 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:47:41.847 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$60ZubQF1oTeKvvWONQKxm.hu4ZPfsSOaSQDAZdY5oPLoV.9rkhAP2", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726897661, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-21T05:47:41.847 INFO:teuthology.orchestra.run.smithi117.stdout:Fetching dashboard port number... 2024-09-21T05:47:42.475 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:42 smithi117 ceph-mon[30947]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:47:42.475 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:42 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:47:43.061 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 8443 2024-09-21T05:47:43.072 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T05:47:43.072 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout disabled 2024-09-21T05:47:43.080 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T05:47:43.080 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout inactive 2024-09-21T05:47:43.080 INFO:teuthology.orchestra.run.smithi117.stdout:firewalld.service is not enabled 2024-09-21T05:47:43.080 INFO:teuthology.orchestra.run.smithi117.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-21T05:47:43.082 INFO:teuthology.orchestra.run.smithi117.stdout:Ceph Dashboard is now available at: 2024-09-21T05:47:43.082 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:47:43.082 INFO:teuthology.orchestra.run.smithi117.stdout: URL: https://smithi117.front.sepia.ceph.com:8443/ 2024-09-21T05:47:43.082 INFO:teuthology.orchestra.run.smithi117.stdout: User: admin 2024-09-21T05:47:43.082 INFO:teuthology.orchestra.run.smithi117.stdout: Password: ikyex1hfwb 2024-09-21T05:47:43.082 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:47:43.083 INFO:teuthology.orchestra.run.smithi117.stdout:Saving cluster configuration to /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config directory 2024-09-21T05:47:43.511 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:43 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2492934051' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-21T05:47:44.368 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-21T05:47:44.368 INFO:teuthology.orchestra.run.smithi117.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-21T05:47:44.368 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:47:44.368 INFO:teuthology.orchestra.run.smithi117.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-21T05:47:44.368 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:47:44.368 INFO:teuthology.orchestra.run.smithi117.stdout:Or, if you are only running a single cluster on this host: 2024-09-21T05:47:44.368 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:47:44.368 INFO:teuthology.orchestra.run.smithi117.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-21T05:47:44.368 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:47:44.368 INFO:teuthology.orchestra.run.smithi117.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-21T05:47:44.368 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:47:44.369 INFO:teuthology.orchestra.run.smithi117.stdout: ceph telemetry on 2024-09-21T05:47:44.369 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:47:44.369 INFO:teuthology.orchestra.run.smithi117.stdout:For more information see: 2024-09-21T05:47:44.369 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:47:44.369 INFO:teuthology.orchestra.run.smithi117.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-21T05:47:44.369 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:47:44.369 INFO:teuthology.orchestra.run.smithi117.stdout:Bootstrap complete. 2024-09-21T05:47:44.408 INFO:tasks.cephadm:Fetching config... 2024-09-21T05:47:44.408 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-21T05:47:44.408 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-21T05:47:44.424 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-21T05:47:44.424 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-21T05:47:44.424 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-21T05:47:44.480 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-21T05:47:44.480 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-21T05:47:44.480 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/keyring of=/dev/stdout 2024-09-21T05:47:44.545 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-21T05:47:44.545 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-21T05:47:44.545 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-21T05:47:44.599 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-21T05:47:44.600 DEBUG:teuthology.orchestra.run.smithi117:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC61nzOkWGnYTUC4sg4lgP+1jwRARIGIFVCFsv2kMULDHnBdpTDaBcwliYZClA9QLka/ypDWa9TM/PJSnx1HDVBUesyvVd+IetrL28CDl0SUPQaFJ8jaFw2c4cxAbMJPOeKu0u+EMtO0hVpVc7+Dtl9+YVfVjtgYyqByFJaAFkYl6osm4kceA99QI6azOB9T2G/mosm32evt2qjwU6BNbYHu+rrSOvRDPrZx2KKq2jNajwrvG2y0rKaLQPhu96TVYXaDyy4dbgXxitk1XY7QvSTuX8qw+i7KvFIipVYl+X8QkHQ+mTs3xbJ5wDIW2kbZHHgARUlb7ur3eZiAn7cbcV8H9T1J/fEyJnAP6qPhWUGvCTJS442iowkgEUYRCQeKZ4mz4GYXHjoAR6I16tLVHOzB32UbrQGBnpDxoFQcPYjdNiw6aJKmArzzOnntfRCDcbDCK1lmrt1lEzPJ53+zoWf/QUb7IqSxhw2vi6U+X2yVuW3+Q5KNyg0P+tBpgAsrb8= ceph-ac8581a2-77dc-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T05:47:44.673 INFO:teuthology.orchestra.run.smithi117.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC61nzOkWGnYTUC4sg4lgP+1jwRARIGIFVCFsv2kMULDHnBdpTDaBcwliYZClA9QLka/ypDWa9TM/PJSnx1HDVBUesyvVd+IetrL28CDl0SUPQaFJ8jaFw2c4cxAbMJPOeKu0u+EMtO0hVpVc7+Dtl9+YVfVjtgYyqByFJaAFkYl6osm4kceA99QI6azOB9T2G/mosm32evt2qjwU6BNbYHu+rrSOvRDPrZx2KKq2jNajwrvG2y0rKaLQPhu96TVYXaDyy4dbgXxitk1XY7QvSTuX8qw+i7KvFIipVYl+X8QkHQ+mTs3xbJ5wDIW2kbZHHgARUlb7ur3eZiAn7cbcV8H9T1J/fEyJnAP6qPhWUGvCTJS442iowkgEUYRCQeKZ4mz4GYXHjoAR6I16tLVHOzB32UbrQGBnpDxoFQcPYjdNiw6aJKmArzzOnntfRCDcbDCK1lmrt1lEzPJ53+zoWf/QUb7IqSxhw2vi6U+X2yVuW3+Q5KNyg0P+tBpgAsrb8= ceph-ac8581a2-77dc-11ef-baf5-efdc52797490 2024-09-21T05:47:44.684 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC61nzOkWGnYTUC4sg4lgP+1jwRARIGIFVCFsv2kMULDHnBdpTDaBcwliYZClA9QLka/ypDWa9TM/PJSnx1HDVBUesyvVd+IetrL28CDl0SUPQaFJ8jaFw2c4cxAbMJPOeKu0u+EMtO0hVpVc7+Dtl9+YVfVjtgYyqByFJaAFkYl6osm4kceA99QI6azOB9T2G/mosm32evt2qjwU6BNbYHu+rrSOvRDPrZx2KKq2jNajwrvG2y0rKaLQPhu96TVYXaDyy4dbgXxitk1XY7QvSTuX8qw+i7KvFIipVYl+X8QkHQ+mTs3xbJ5wDIW2kbZHHgARUlb7ur3eZiAn7cbcV8H9T1J/fEyJnAP6qPhWUGvCTJS442iowkgEUYRCQeKZ4mz4GYXHjoAR6I16tLVHOzB32UbrQGBnpDxoFQcPYjdNiw6aJKmArzzOnntfRCDcbDCK1lmrt1lEzPJ53+zoWf/QUb7IqSxhw2vi6U+X2yVuW3+Q5KNyg0P+tBpgAsrb8= ceph-ac8581a2-77dc-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T05:47:44.726 INFO:teuthology.orchestra.run.smithi138.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC61nzOkWGnYTUC4sg4lgP+1jwRARIGIFVCFsv2kMULDHnBdpTDaBcwliYZClA9QLka/ypDWa9TM/PJSnx1HDVBUesyvVd+IetrL28CDl0SUPQaFJ8jaFw2c4cxAbMJPOeKu0u+EMtO0hVpVc7+Dtl9+YVfVjtgYyqByFJaAFkYl6osm4kceA99QI6azOB9T2G/mosm32evt2qjwU6BNbYHu+rrSOvRDPrZx2KKq2jNajwrvG2y0rKaLQPhu96TVYXaDyy4dbgXxitk1XY7QvSTuX8qw+i7KvFIipVYl+X8QkHQ+mTs3xbJ5wDIW2kbZHHgARUlb7ur3eZiAn7cbcV8H9T1J/fEyJnAP6qPhWUGvCTJS442iowkgEUYRCQeKZ4mz4GYXHjoAR6I16tLVHOzB32UbrQGBnpDxoFQcPYjdNiw6aJKmArzzOnntfRCDcbDCK1lmrt1lEzPJ53+zoWf/QUb7IqSxhw2vi6U+X2yVuW3+Q5KNyg0P+tBpgAsrb8= ceph-ac8581a2-77dc-11ef-baf5-efdc52797490 2024-09-21T05:47:44.739 DEBUG:teuthology.orchestra.run.smithi192:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC61nzOkWGnYTUC4sg4lgP+1jwRARIGIFVCFsv2kMULDHnBdpTDaBcwliYZClA9QLka/ypDWa9TM/PJSnx1HDVBUesyvVd+IetrL28CDl0SUPQaFJ8jaFw2c4cxAbMJPOeKu0u+EMtO0hVpVc7+Dtl9+YVfVjtgYyqByFJaAFkYl6osm4kceA99QI6azOB9T2G/mosm32evt2qjwU6BNbYHu+rrSOvRDPrZx2KKq2jNajwrvG2y0rKaLQPhu96TVYXaDyy4dbgXxitk1XY7QvSTuX8qw+i7KvFIipVYl+X8QkHQ+mTs3xbJ5wDIW2kbZHHgARUlb7ur3eZiAn7cbcV8H9T1J/fEyJnAP6qPhWUGvCTJS442iowkgEUYRCQeKZ4mz4GYXHjoAR6I16tLVHOzB32UbrQGBnpDxoFQcPYjdNiw6aJKmArzzOnntfRCDcbDCK1lmrt1lEzPJ53+zoWf/QUb7IqSxhw2vi6U+X2yVuW3+Q5KNyg0P+tBpgAsrb8= ceph-ac8581a2-77dc-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T05:47:44.780 INFO:teuthology.orchestra.run.smithi192.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC61nzOkWGnYTUC4sg4lgP+1jwRARIGIFVCFsv2kMULDHnBdpTDaBcwliYZClA9QLka/ypDWa9TM/PJSnx1HDVBUesyvVd+IetrL28CDl0SUPQaFJ8jaFw2c4cxAbMJPOeKu0u+EMtO0hVpVc7+Dtl9+YVfVjtgYyqByFJaAFkYl6osm4kceA99QI6azOB9T2G/mosm32evt2qjwU6BNbYHu+rrSOvRDPrZx2KKq2jNajwrvG2y0rKaLQPhu96TVYXaDyy4dbgXxitk1XY7QvSTuX8qw+i7KvFIipVYl+X8QkHQ+mTs3xbJ5wDIW2kbZHHgARUlb7ur3eZiAn7cbcV8H9T1J/fEyJnAP6qPhWUGvCTJS442iowkgEUYRCQeKZ4mz4GYXHjoAR6I16tLVHOzB32UbrQGBnpDxoFQcPYjdNiw6aJKmArzzOnntfRCDcbDCK1lmrt1lEzPJ53+zoWf/QUb7IqSxhw2vi6U+X2yVuW3+Q5KNyg0P+tBpgAsrb8= ceph-ac8581a2-77dc-11ef-baf5-efdc52797490 2024-09-21T05:47:44.793 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-21T05:47:44.918 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:44 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2275659773' entity='client.admin' 2024-09-21T05:47:45.039 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:47:47.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:46 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/1168749098' entity='client.admin' 2024-09-21T05:47:47.185 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-21T05:47:47.185 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-21T05:47:47.919 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:47 smithi117 ceph-mon[30947]: mgrmap e13: x(active, since 12s) 2024-09-21T05:47:47.967 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:47:49.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:48 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:47:49.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:48 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:47:49.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:48 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi117", "name": "osd_memory_target"} : dispatch 2024-09-21T05:47:49.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:48 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:47:49.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:48 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:47:49.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:48 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:47:49.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:48 smithi117 ceph-mon[30947]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:47:49.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:48 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:47:49.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:48 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:47:49.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:48 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:47:49.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:48 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:47:49.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:48 smithi117 ceph-mon[30947]: Updating smithi117:/etc/ceph/ceph.conf 2024-09-21T05:47:49.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:48 smithi117 ceph-mon[30947]: Updating smithi117:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T05:47:49.404 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi138 2024-09-21T05:47:49.405 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-21T05:47:49.405 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/etc/ceph/ceph.conf 2024-09-21T05:47:49.425 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-21T05:47:49.425 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T05:47:49.482 INFO:tasks.cephadm:Adding host smithi138 to orchestrator... 2024-09-21T05:47:49.483 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph orch host add smithi138 2024-09-21T05:47:49.723 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:47:50.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:50 smithi117 ceph-mon[30947]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-09-21T05:47:50.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:50 smithi117 ceph-mon[30947]: Updating smithi117:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T05:47:50.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:50 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:47:50.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:50 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:47:50.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:50 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:47:51.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:51 smithi117 ceph-mon[30947]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi138", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:47:52.526 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:52 smithi117 ceph-mon[30947]: Deploying cephadm binary to smithi138 2024-09-21T05:47:53.404 INFO:teuthology.orchestra.run.smithi117.stdout:Added host 'smithi138' with addr '172.21.15.138' 2024-09-21T05:47:54.019 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T05:47:54.255 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:47:54.704 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:54 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:47:54.704 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:54 smithi117 ceph-mon[30947]: Added host smithi138 2024-09-21T05:47:54.704 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:54 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:47:54.704 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:54 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:47:54.704 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:54 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:47:54.902 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:47:54.902 INFO:teuthology.orchestra.run.smithi117.stdout:[{"addr": "172.21.15.117", "hostname": "smithi117", "labels": [], "status": ""}, {"addr": "172.21.15.138", "hostname": "smithi138", "labels": [], "status": ""}] 2024-09-21T05:47:55.558 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi192 2024-09-21T05:47:55.558 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-21T05:47:55.558 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/etc/ceph/ceph.conf 2024-09-21T05:47:55.579 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-21T05:47:55.579 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T05:47:55.638 INFO:tasks.cephadm:Adding host smithi192 to orchestrator... 2024-09-21T05:47:55.638 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph orch host add smithi192 2024-09-21T05:47:55.880 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:47:56.000 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:55 smithi117 ceph-mon[30947]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:47:56.000 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:55 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:47:57.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:56 smithi117 ceph-mon[30947]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T05:47:58.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:57 smithi117 ceph-mon[30947]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:47:58.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:57 smithi117 ceph-mon[30947]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi192", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:47:58.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:57 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:47:58.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:57 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:47:58.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:57 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:47:58.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:57 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:47:58.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:57 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi138", "name": "osd_memory_target"} : dispatch 2024-09-21T05:47:58.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:57 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:47:58.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:57 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:47:59.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:58 smithi117 ceph-mon[30947]: Deploying cephadm binary to smithi192 2024-09-21T05:47:59.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:58 smithi117 ceph-mon[30947]: Updating smithi138:/etc/ceph/ceph.conf 2024-09-21T05:47:59.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:58 smithi117 ceph-mon[30947]: Updating smithi138:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T05:47:59.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:58 smithi117 ceph-mon[30947]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-09-21T05:47:59.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:58 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:47:59.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:58 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:47:59.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:58 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:47:59.555 INFO:teuthology.orchestra.run.smithi117.stdout:Added host 'smithi192' with addr '172.21.15.192' 2024-09-21T05:47:59.879 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:59 smithi117 ceph-mon[30947]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:47:59.879 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:59 smithi117 ceph-mon[30947]: Updating smithi138:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T05:47:59.879 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:59 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:47:59.879 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:47:59 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:48:00.198 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T05:48:00.439 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:48:00.857 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:00 smithi117 ceph-mon[30947]: Added host smithi192 2024-09-21T05:48:00.857 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:00 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:00.858 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:00 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:01.060 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:48:01.061 INFO:teuthology.orchestra.run.smithi117.stdout:[{"addr": "172.21.15.117", "hostname": "smithi117", "labels": [], "status": ""}, {"addr": "172.21.15.138", "hostname": "smithi138", "labels": [], "status": ""}, {"addr": "172.21.15.192", "hostname": "smithi192", "labels": [], "status": ""}] 2024-09-21T05:48:01.673 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-21T05:48:01.673 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph osd crush tunables default 2024-09-21T05:48:01.915 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:48:02.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:01 smithi117 ceph-mon[30947]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:02.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:01 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:02.765 INFO:teuthology.orchestra.run.smithi117.stderr:adjusted tunables profile to default 2024-09-21T05:48:03.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:02 smithi117 ceph-mon[30947]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T05:48:03.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:02 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2034460815' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T05:48:03.377 INFO:tasks.cephadm:Adding mon.a on smithi117 2024-09-21T05:48:03.378 INFO:tasks.cephadm:Adding mon.b on smithi138 2024-09-21T05:48:03.378 INFO:tasks.cephadm:Adding mon.c on smithi192 2024-09-21T05:48:03.378 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph orch apply mon '3;smithi117:172.21.15.117=a;smithi138:172.21.15.138=b;smithi192:172.21.15.192=c' 2024-09-21T05:48:03.694 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T05:48:03.739 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T05:48:04.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:03 smithi117 ceph-mon[30947]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:04.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:03 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2034460815' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T05:48:04.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:03 smithi117 ceph-mon[30947]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T05:48:04.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:03 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:04.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:03 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:04.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:03 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:04.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:03 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:04.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:03 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi192", "name": "osd_memory_target"} : dispatch 2024-09-21T05:48:04.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:03 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:04.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:03 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:48:04.415 INFO:teuthology.orchestra.run.smithi192.stdout:Scheduled mon update... 2024-09-21T05:48:05.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:04 smithi117 ceph-mon[30947]: Updating smithi192:/etc/ceph/ceph.conf 2024-09-21T05:48:05.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:04 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:05.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:04 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:05.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:04 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:05.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:04 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:05.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:04 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:48:05.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:04 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:05.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:04 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:48:05.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:04 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:05.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:04 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T05:48:05.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:04 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:05.037 DEBUG:teuthology.orchestra.run.smithi138:mon.b> sudo journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mon.b.service 2024-09-21T05:48:05.040 DEBUG:teuthology.orchestra.run.smithi192:mon.c> sudo journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mon.c.service 2024-09-21T05:48:05.042 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T05:48:05.043 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T05:48:06.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:05 smithi117 ceph-mon[30947]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:06.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:05 smithi117 ceph-mon[30947]: Updating smithi192:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T05:48:06.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:05 smithi117 ceph-mon[30947]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-09-21T05:48:06.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:05 smithi117 ceph-mon[30947]: Updating smithi192:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T05:48:06.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:05 smithi117 ceph-mon[30947]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi117:172.21.15.117=a;smithi138:172.21.15.138=b;smithi192:172.21.15.192=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:48:06.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:05 smithi117 ceph-mon[30947]: Saving service mon spec with placement smithi117:172.21.15.117=a;smithi138:172.21.15.138=b;smithi192:172.21.15.192=c;count:3 2024-09-21T05:48:06.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:05 smithi117 ceph-mon[30947]: Deploying daemon mon.c on smithi192 2024-09-21T05:48:06.864 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T05:48:07.769 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-21T05:48:07.769 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"ac8581a2-77dc-11ef-baf5-efdc52797490","modified":"2024-09-21T05:46:44.194497Z","created":"2024-09-21T05:46:44.194497Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T05:48:07.769 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-09-21T05:48:08.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:07 smithi117 ceph-mon[30947]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:08.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:07 smithi117 ceph-mon[30947]: from='client.? 172.21.15.192:0/2785297717' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T05:48:08.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:08 smithi192 systemd[1]: Started Ceph mon.c for ac8581a2-77dc-11ef-baf5-efdc52797490. 2024-09-21T05:48:09.368 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T05:48:09.369 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T05:48:09.629 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T05:48:14.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:13 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T05:48:14.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:13 smithi192 ceph-mon[34700]: mon.a calling monitor election 2024-09-21T05:48:14.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:13 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:48:14.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:13 smithi192 ceph-mon[34700]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:14.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:13 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:48:14.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:13 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:48:14.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:13 smithi192 ceph-mon[34700]: mon.c calling monitor election 2024-09-21T05:48:14.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:13 smithi192 ceph-mon[34700]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:14.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:13 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:48:14.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:13 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:48:14.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:13 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:48:14.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:13 smithi192 ceph-mon[34700]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T05:48:14.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:13 smithi192 ceph-mon[34700]: monmap epoch 2 2024-09-21T05:48:14.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:13 smithi192 ceph-mon[34700]: fsid ac8581a2-77dc-11ef-baf5-efdc52797490 2024-09-21T05:48:14.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:13 smithi192 ceph-mon[34700]: last_changed 2024-09-21T05:48:08.846539+0000 2024-09-21T05:48:14.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:13 smithi192 ceph-mon[34700]: created 2024-09-21T05:46:44.194497+0000 2024-09-21T05:48:14.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:13 smithi192 ceph-mon[34700]: min_mon_release 19 (squid) 2024-09-21T05:48:14.232 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:13 smithi192 ceph-mon[34700]: election_strategy: 1 2024-09-21T05:48:14.232 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:13 smithi192 ceph-mon[34700]: 0: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.a 2024-09-21T05:48:14.232 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:13 smithi192 ceph-mon[34700]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.c 2024-09-21T05:48:14.232 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:13 smithi192 ceph-mon[34700]: fsmap 2024-09-21T05:48:14.232 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:13 smithi192 ceph-mon[34700]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T05:48:14.232 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:13 smithi192 ceph-mon[34700]: mgrmap e13: x(active, since 40s) 2024-09-21T05:48:14.232 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:13 smithi192 ceph-mon[34700]: overall HEALTH_OK 2024-09-21T05:48:14.232 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:13 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:14.232 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:13 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:14.232 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:13 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:14.232 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:13 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T05:48:14.232 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:13 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:14.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:13 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T05:48:14.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:13 smithi117 ceph-mon[30947]: mon.a calling monitor election 2024-09-21T05:48:14.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:13 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:48:14.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:13 smithi117 ceph-mon[30947]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:14.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:13 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:48:14.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:13 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:48:14.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:13 smithi117 ceph-mon[30947]: mon.c calling monitor election 2024-09-21T05:48:14.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:13 smithi117 ceph-mon[30947]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:14.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:13 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:48:14.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:13 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:48:14.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:13 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:48:14.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:13 smithi117 ceph-mon[30947]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T05:48:14.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:13 smithi117 ceph-mon[30947]: monmap epoch 2 2024-09-21T05:48:14.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:13 smithi117 ceph-mon[30947]: fsid ac8581a2-77dc-11ef-baf5-efdc52797490 2024-09-21T05:48:14.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:13 smithi117 ceph-mon[30947]: last_changed 2024-09-21T05:48:08.846539+0000 2024-09-21T05:48:14.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:13 smithi117 ceph-mon[30947]: created 2024-09-21T05:46:44.194497+0000 2024-09-21T05:48:14.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:13 smithi117 ceph-mon[30947]: min_mon_release 19 (squid) 2024-09-21T05:48:14.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:13 smithi117 ceph-mon[30947]: election_strategy: 1 2024-09-21T05:48:14.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:13 smithi117 ceph-mon[30947]: 0: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.a 2024-09-21T05:48:14.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:13 smithi117 ceph-mon[30947]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.c 2024-09-21T05:48:14.279 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:13 smithi117 ceph-mon[30947]: fsmap 2024-09-21T05:48:14.279 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:13 smithi117 ceph-mon[30947]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T05:48:14.279 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:13 smithi117 ceph-mon[30947]: mgrmap e13: x(active, since 40s) 2024-09-21T05:48:14.279 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:13 smithi117 ceph-mon[30947]: overall HEALTH_OK 2024-09-21T05:48:14.279 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:13 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:14.279 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:13 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:14.279 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:13 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:14.279 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:13 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T05:48:14.279 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:13 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:14.709 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-21T05:48:14.710 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":2,"fsid":"ac8581a2-77dc-11ef-baf5-efdc52797490","modified":"2024-09-21T05:48:08.846539Z","created":"2024-09-21T05:46:44.194497Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T05:48:14.710 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 2 2024-09-21T05:48:14.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:14 smithi192 ceph-mon[34700]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:14.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:14 smithi192 ceph-mon[34700]: Deploying daemon mon.b on smithi138 2024-09-21T05:48:14.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:14 smithi192 ceph-mon[34700]: from='client.? 172.21.15.192:0/1506302290' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T05:48:14.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:14 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:48:15.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:14 smithi117 ceph-mon[30947]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:15.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:14 smithi117 ceph-mon[30947]: Deploying daemon mon.b on smithi138 2024-09-21T05:48:15.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:14 smithi117 ceph-mon[30947]: from='client.? 172.21.15.192:0/1506302290' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T05:48:15.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:14 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:48:16.388 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T05:48:16.388 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T05:48:16.657 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T05:48:17.028 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:16 smithi192 ceph-mon[34700]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:17.261 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 systemd[1]: Starting Ceph mon.b for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T05:48:17.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:16 smithi117 ceph-mon[30947]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:17.299 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-21T05:48:17.299 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":2,"fsid":"ac8581a2-77dc-11ef-baf5-efdc52797490","modified":"2024-09-21T05:48:08.846539Z","created":"2024-09-21T05:46:44.194497Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T05:48:17.299 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 2 2024-09-21T05:48:17.541 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 podman[34373]: 2024-09-21 05:48:17.442837547 +0000 UTC m=+0.112384996 container create 01a69c7125fc345f42649f069275f216a94e0c351ca6f1d88d49a21cb64e5568 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mon-b, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T05:48:17.541 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 podman[34373]: 2024-09-21 05:48:17.347990638 +0000 UTC m=+0.017538107 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T05:48:18.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 podman[34373]: 2024-09-21 05:48:17.636656797 +0000 UTC m=+0.306204250 container init 01a69c7125fc345f42649f069275f216a94e0c351ca6f1d88d49a21cb64e5568 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mon-b, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731) 2024-09-21T05:48:18.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 podman[34373]: 2024-09-21 05:48:17.639894068 +0000 UTC m=+0.309441518 container start 01a69c7125fc345f42649f069275f216a94e0c351ca6f1d88d49a21cb64e5568 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mon-b, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T05:48:18.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T05:48:18.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T05:48:18.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: pidfile_write: ignore empty --pid-file 2024-09-21T05:48:18.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: load: jerasure load: lrc 2024-09-21T05:48:18.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: RocksDB version: 7.9.2 2024-09-21T05:48:18.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Git sha 0 2024-09-21T05:48:18.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T05:48:18.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: DB SUMMARY 2024-09-21T05:48:18.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: DB Session ID: A6AOEU0WW5NPQMG1O3QR 2024-09-21T05:48:18.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: CURRENT file: CURRENT 2024-09-21T05:48:18.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T05:48:18.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-21T05:48:18.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-21T05:48:18.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-09-21T05:48:18.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.error_if_exists: 0 2024-09-21T05:48:18.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.create_if_missing: 0 2024-09-21T05:48:18.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.paranoid_checks: 1 2024-09-21T05:48:18.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T05:48:18.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T05:48:18.118 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T05:48:18.118 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.env: 0x55aec67f4d60 2024-09-21T05:48:18.118 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T05:48:18.118 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.info_log: 0x55aecc033360 2024-09-21T05:48:18.118 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T05:48:18.119 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.statistics: (nil) 2024-09-21T05:48:18.119 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.use_fsync: 0 2024-09-21T05:48:18.119 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.max_log_file_size: 0 2024-09-21T05:48:18.119 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T05:48:18.120 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T05:48:18.120 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T05:48:18.120 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T05:48:18.120 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.allow_fallocate: 1 2024-09-21T05:48:18.120 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T05:48:18.120 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T05:48:18.120 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.use_direct_reads: 0 2024-09-21T05:48:18.120 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T05:48:18.120 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T05:48:18.120 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.db_log_dir: 2024-09-21T05:48:18.120 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.wal_dir: 2024-09-21T05:48:18.120 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T05:48:18.120 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T05:48:18.121 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T05:48:18.121 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T05:48:18.121 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T05:48:18.121 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T05:48:18.121 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T05:48:18.121 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T05:48:18.121 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.write_buffer_manager: 0x55aecc037d60 2024-09-21T05:48:18.121 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T05:48:18.121 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T05:48:18.121 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T05:48:18.121 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T05:48:18.121 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T05:48:18.121 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T05:48:18.121 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T05:48:18.122 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T05:48:18.122 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.unordered_write: 0 2024-09-21T05:48:18.122 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T05:48:18.122 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T05:48:18.122 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T05:48:18.122 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T05:48:18.122 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.row_cache: None 2024-09-21T05:48:18.122 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.wal_filter: None 2024-09-21T05:48:18.122 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T05:48:18.122 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T05:48:18.122 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.two_write_queues: 0 2024-09-21T05:48:18.122 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T05:48:18.122 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.wal_compression: 0 2024-09-21T05:48:18.122 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.atomic_flush: 0 2024-09-21T05:48:18.123 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T05:48:18.123 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T05:48:18.123 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T05:48:18.123 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.log_readahead_size: 0 2024-09-21T05:48:18.123 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T05:48:18.123 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T05:48:18.123 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T05:48:18.123 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T05:48:18.123 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T05:48:18.123 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T05:48:18.123 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T05:48:18.123 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.max_background_jobs: 2 2024-09-21T05:48:18.123 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.max_background_compactions: -1 2024-09-21T05:48:18.123 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.max_subcompactions: 1 2024-09-21T05:48:18.124 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T05:48:18.124 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T05:48:18.124 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T05:48:18.124 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T05:48:18.124 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T05:48:18.124 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T05:48:18.124 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T05:48:18.124 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T05:48:18.124 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.max_open_files: -1 2024-09-21T05:48:18.124 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T05:48:18.124 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T05:48:18.124 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T05:48:18.124 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T05:48:18.124 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.max_background_flushes: -1 2024-09-21T05:48:18.124 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Compression algorithms supported: 2024-09-21T05:48:18.125 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: kZSTD supported: 0 2024-09-21T05:48:18.125 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: kXpressCompression supported: 0 2024-09-21T05:48:18.125 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: kBZip2Compression supported: 0 2024-09-21T05:48:18.125 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T05:48:18.125 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: kLZ4Compression supported: 1 2024-09-21T05:48:18.125 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: kZlibCompression supported: 1 2024-09-21T05:48:18.125 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T05:48:18.125 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: kSnappyCompression supported: 1 2024-09-21T05:48:18.125 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T05:48:18.125 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T05:48:18.125 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-21T05:48:18.125 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T05:48:18.125 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T05:48:18.125 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.merge_operator: 2024-09-21T05:48:18.126 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.compaction_filter: None 2024-09-21T05:48:18.126 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T05:48:18.126 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T05:48:18.126 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T05:48:18.126 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T05:48:18.126 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55aecc032fa0) 2024-09-21T05:48:18.126 INFO:journalctl@ceph.mon.b.smithi138.stdout: cache_index_and_filter_blocks: 1 2024-09-21T05:48:18.126 INFO:journalctl@ceph.mon.b.smithi138.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T05:48:18.126 INFO:journalctl@ceph.mon.b.smithi138.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T05:48:18.126 INFO:journalctl@ceph.mon.b.smithi138.stdout: pin_top_level_index_and_filter: 1 2024-09-21T05:48:18.126 INFO:journalctl@ceph.mon.b.smithi138.stdout: index_type: 0 2024-09-21T05:48:18.126 INFO:journalctl@ceph.mon.b.smithi138.stdout: data_block_index_type: 0 2024-09-21T05:48:18.126 INFO:journalctl@ceph.mon.b.smithi138.stdout: index_shortening: 1 2024-09-21T05:48:18.126 INFO:journalctl@ceph.mon.b.smithi138.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T05:48:18.127 INFO:journalctl@ceph.mon.b.smithi138.stdout: checksum: 4 2024-09-21T05:48:18.127 INFO:journalctl@ceph.mon.b.smithi138.stdout: no_block_cache: 0 2024-09-21T05:48:18.127 INFO:journalctl@ceph.mon.b.smithi138.stdout: block_cache: 0x55aec7ff7610 2024-09-21T05:48:18.127 INFO:journalctl@ceph.mon.b.smithi138.stdout: block_cache_name: BinnedLRUCache 2024-09-21T05:48:18.127 INFO:journalctl@ceph.mon.b.smithi138.stdout: block_cache_options: 2024-09-21T05:48:18.127 INFO:journalctl@ceph.mon.b.smithi138.stdout: capacity : 536870912 2024-09-21T05:48:18.127 INFO:journalctl@ceph.mon.b.smithi138.stdout: num_shard_bits : 4 2024-09-21T05:48:18.127 INFO:journalctl@ceph.mon.b.smithi138.stdout: strict_capacity_limit : 0 2024-09-21T05:48:18.127 INFO:journalctl@ceph.mon.b.smithi138.stdout: high_pri_pool_ratio: 0.000 2024-09-21T05:48:18.127 INFO:journalctl@ceph.mon.b.smithi138.stdout: block_cache_compressed: (nil) 2024-09-21T05:48:18.127 INFO:journalctl@ceph.mon.b.smithi138.stdout: persistent_cache: (nil) 2024-09-21T05:48:18.127 INFO:journalctl@ceph.mon.b.smithi138.stdout: block_size: 4096 2024-09-21T05:48:18.127 INFO:journalctl@ceph.mon.b.smithi138.stdout: block_size_deviation: 10 2024-09-21T05:48:18.127 INFO:journalctl@ceph.mon.b.smithi138.stdout: block_restart_interval: 16 2024-09-21T05:48:18.128 INFO:journalctl@ceph.mon.b.smithi138.stdout: index_block_restart_interval: 1 2024-09-21T05:48:18.128 INFO:journalctl@ceph.mon.b.smithi138.stdout: metadata_block_size: 4096 2024-09-21T05:48:18.128 INFO:journalctl@ceph.mon.b.smithi138.stdout: partition_filters: 0 2024-09-21T05:48:18.128 INFO:journalctl@ceph.mon.b.smithi138.stdout: use_delta_encoding: 1 2024-09-21T05:48:18.128 INFO:journalctl@ceph.mon.b.smithi138.stdout: filter_policy: bloomfilter 2024-09-21T05:48:18.128 INFO:journalctl@ceph.mon.b.smithi138.stdout: whole_key_filtering: 1 2024-09-21T05:48:18.128 INFO:journalctl@ceph.mon.b.smithi138.stdout: verify_compression: 0 2024-09-21T05:48:18.128 INFO:journalctl@ceph.mon.b.smithi138.stdout: read_amp_bytes_per_bit: 0 2024-09-21T05:48:18.128 INFO:journalctl@ceph.mon.b.smithi138.stdout: format_version: 5 2024-09-21T05:48:18.128 INFO:journalctl@ceph.mon.b.smithi138.stdout: enable_index_compression: 1 2024-09-21T05:48:18.128 INFO:journalctl@ceph.mon.b.smithi138.stdout: block_align: 0 2024-09-21T05:48:18.128 INFO:journalctl@ceph.mon.b.smithi138.stdout: max_auto_readahead_size: 262144 2024-09-21T05:48:18.128 INFO:journalctl@ceph.mon.b.smithi138.stdout: prepopulate_block_cache: 0 2024-09-21T05:48:18.129 INFO:journalctl@ceph.mon.b.smithi138.stdout: initial_auto_readahead_size: 8192 2024-09-21T05:48:18.129 INFO:journalctl@ceph.mon.b.smithi138.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T05:48:18.129 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T05:48:18.129 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T05:48:18.129 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.compression: NoCompression 2024-09-21T05:48:18.129 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T05:48:18.129 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T05:48:18.129 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T05:48:18.129 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.num_levels: 7 2024-09-21T05:48:18.129 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T05:48:18.129 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T05:48:18.129 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T05:48:18.130 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T05:48:18.130 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T05:48:18.130 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T05:48:18.130 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T05:48:18.130 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T05:48:18.130 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T05:48:18.130 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T05:48:18.130 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T05:48:18.130 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T05:48:18.130 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T05:48:18.130 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T05:48:18.130 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T05:48:18.130 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T05:48:18.130 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T05:48:18.131 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T05:48:18.131 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T05:48:18.131 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T05:48:18.131 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T05:48:18.131 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T05:48:18.131 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T05:48:18.131 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T05:48:18.131 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T05:48:18.131 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T05:48:18.131 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T05:48:18.131 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T05:48:18.131 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T05:48:18.131 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T05:48:18.132 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T05:48:18.132 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T05:48:18.132 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T05:48:18.132 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T05:48:18.132 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T05:48:18.132 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T05:48:18.132 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T05:48:18.132 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T05:48:18.132 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T05:48:18.132 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T05:48:18.132 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T05:48:18.132 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T05:48:18.132 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T05:48:18.132 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T05:48:18.133 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T05:48:18.133 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T05:48:18.133 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T05:48:18.133 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T05:48:18.133 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T05:48:18.133 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T05:48:18.133 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T05:48:18.133 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T05:48:18.133 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T05:48:18.133 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T05:48:18.133 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.inplace_update_support: 0 2024-09-21T05:48:18.133 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T05:48:18.133 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T05:48:18.133 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T05:48:18.133 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T05:48:18.134 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.bloom_locality: 0 2024-09-21T05:48:18.134 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.max_successive_merges: 0 2024-09-21T05:48:18.134 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T05:48:18.134 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T05:48:18.134 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T05:48:18.134 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T05:48:18.134 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.ttl: 2592000 2024-09-21T05:48:18.134 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T05:48:18.134 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T05:48:18.134 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T05:48:18.134 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.enable_blob_files: false 2024-09-21T05:48:18.134 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.min_blob_size: 0 2024-09-21T05:48:18.134 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T05:48:18.134 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T05:48:18.135 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T05:48:18.135 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T05:48:18.135 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T05:48:18.135 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T05:48:18.135 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T05:48:18.135 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T05:48:18.135 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-21T05:48:18.135 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-21T05:48:18.135 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 28c6473b-5735-4439-9e5c-406cf31387e0 2024-09-21T05:48:18.135 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726897697677260, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-21T05:48:18.135 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-21T05:48:18.135 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726897697677863, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726897697, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "28c6473b-5735-4439-9e5c-406cf31387e0", "db_session_id": "A6AOEU0WW5NPQMG1O3QR", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-21T05:48:18.135 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726897697677990, "job": 1, "event": "recovery_finished"} 2024-09-21T05:48:18.135 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-21T05:48:18.136 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T05:48:18.136 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55aecc056a80 2024-09-21T05:48:18.136 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: DB pointer 0x55aeccdfc000 2024-09-21T05:48:18.136 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T05:48:18.136 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T05:48:18.136 INFO:journalctl@ceph.mon.b.smithi138.stdout: ** DB Stats ** 2024-09-21T05:48:18.136 INFO:journalctl@ceph.mon.b.smithi138.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T05:48:18.136 INFO:journalctl@ceph.mon.b.smithi138.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T05:48:18.136 INFO:journalctl@ceph.mon.b.smithi138.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T05:48:18.136 INFO:journalctl@ceph.mon.b.smithi138.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T05:48:18.136 INFO:journalctl@ceph.mon.b.smithi138.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T05:48:18.136 INFO:journalctl@ceph.mon.b.smithi138.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T05:48:18.136 INFO:journalctl@ceph.mon.b.smithi138.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T05:48:18.136 INFO:journalctl@ceph.mon.b.smithi138.stdout: 2024-09-21T05:48:18.136 INFO:journalctl@ceph.mon.b.smithi138.stdout: ** Compaction Stats [default] ** 2024-09-21T05:48:18.137 INFO:journalctl@ceph.mon.b.smithi138.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T05:48:18.137 INFO:journalctl@ceph.mon.b.smithi138.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T05:48:18.137 INFO:journalctl@ceph.mon.b.smithi138.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T05:48:18.137 INFO:journalctl@ceph.mon.b.smithi138.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T05:48:18.137 INFO:journalctl@ceph.mon.b.smithi138.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T05:48:18.137 INFO:journalctl@ceph.mon.b.smithi138.stdout: 2024-09-21T05:48:18.137 INFO:journalctl@ceph.mon.b.smithi138.stdout: ** Compaction Stats [default] ** 2024-09-21T05:48:18.137 INFO:journalctl@ceph.mon.b.smithi138.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T05:48:18.137 INFO:journalctl@ceph.mon.b.smithi138.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T05:48:18.137 INFO:journalctl@ceph.mon.b.smithi138.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T05:48:18.137 INFO:journalctl@ceph.mon.b.smithi138.stdout: 2024-09-21T05:48:18.138 INFO:journalctl@ceph.mon.b.smithi138.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T05:48:18.138 INFO:journalctl@ceph.mon.b.smithi138.stdout: 2024-09-21T05:48:18.138 INFO:journalctl@ceph.mon.b.smithi138.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T05:48:18.138 INFO:journalctl@ceph.mon.b.smithi138.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T05:48:18.138 INFO:journalctl@ceph.mon.b.smithi138.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T05:48:18.138 INFO:journalctl@ceph.mon.b.smithi138.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T05:48:18.138 INFO:journalctl@ceph.mon.b.smithi138.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T05:48:18.138 INFO:journalctl@ceph.mon.b.smithi138.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T05:48:18.138 INFO:journalctl@ceph.mon.b.smithi138.stdout: Cumulative compaction: 0.00 GB write, 0.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T05:48:18.138 INFO:journalctl@ceph.mon.b.smithi138.stdout: Interval compaction: 0.00 GB write, 0.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T05:48:18.138 INFO:journalctl@ceph.mon.b.smithi138.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T05:48:18.138 INFO:journalctl@ceph.mon.b.smithi138.stdout: Block cache BinnedLRUCache@0x55aec7ff7610#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-09-21T05:48:18.138 INFO:journalctl@ceph.mon.b.smithi138.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-21T05:48:18.139 INFO:journalctl@ceph.mon.b.smithi138.stdout: 2024-09-21T05:48:18.139 INFO:journalctl@ceph.mon.b.smithi138.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T05:48:18.139 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-21T05:48:18.139 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: using public_addr v2:172.21.15.138:0/0 -> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] 2024-09-21T05:48:18.139 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: starting mon.b rank -1 at public addrs [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] at bind addrs [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid ac8581a2-77dc-11ef-baf5-efdc52797490 2024-09-21T05:48:18.139 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: mon.b@-1(???) e0 preinit fsid ac8581a2-77dc-11ef-baf5-efdc52797490 2024-09-21T05:48:18.139 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: mon.b@-1(synchronizing).mds e1 new map 2024-09-21T05:48:18.140 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-21T05:48:18.140 INFO:journalctl@ceph.mon.b.smithi138.stdout: e1 2024-09-21T05:48:18.140 INFO:journalctl@ceph.mon.b.smithi138.stdout: btime 2024-09-21T05:46:47:021637+0000 2024-09-21T05:48:18.140 INFO:journalctl@ceph.mon.b.smithi138.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T05:48:18.140 INFO:journalctl@ceph.mon.b.smithi138.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:48:18.140 INFO:journalctl@ceph.mon.b.smithi138.stdout: legacy client fscid: -1 2024-09-21T05:48:18.140 INFO:journalctl@ceph.mon.b.smithi138.stdout: 2024-09-21T05:48:18.140 INFO:journalctl@ceph.mon.b.smithi138.stdout: No filesystems configured 2024-09-21T05:48:18.140 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-21T05:48:18.140 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-21T05:48:18.140 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-21T05:48:18.140 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-21T05:48:18.141 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-21T05:48:18.141 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-21T05:48:18.141 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T05:48:18.141 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T05:48:18.141 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T05:48:18.141 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T05:48:18.141 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T05:48:18.142 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:18.142 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi192", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:48:18.142 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:18.142 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:18.142 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:18.142 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:18.142 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi138", "name": "osd_memory_target"} : dispatch 2024-09-21T05:48:18.143 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:18.143 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:48:18.143 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: Deploying cephadm binary to smithi192 2024-09-21T05:48:18.143 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: Updating smithi138:/etc/ceph/ceph.conf 2024-09-21T05:48:18.143 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: Updating smithi138:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T05:48:18.143 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-09-21T05:48:18.143 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:18.143 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:18.143 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:18.143 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:18.144 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: Updating smithi138:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T05:48:18.144 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:18.144 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:48:18.144 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: Added host smithi192 2024-09-21T05:48:18.144 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:18.144 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:18.144 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:18.144 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:18.145 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T05:48:18.145 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2034460815' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T05:48:18.145 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:18.145 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2034460815' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T05:48:18.145 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T05:48:18.145 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:18.145 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:18.145 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:18.145 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:18.145 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi192", "name": "osd_memory_target"} : dispatch 2024-09-21T05:48:18.145 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:18.145 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:48:18.145 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: Updating smithi192:/etc/ceph/ceph.conf 2024-09-21T05:48:18.146 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:18.146 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:18.146 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:18.146 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:18.146 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:48:18.146 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:18.146 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:48:18.146 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:18.146 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T05:48:18.146 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:18.146 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:18.146 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: Updating smithi192:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T05:48:18.146 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-09-21T05:48:18.147 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: Updating smithi192:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T05:48:18.147 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi117:172.21.15.117=a;smithi138:172.21.15.138=b;smithi192:172.21.15.192=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:48:18.147 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: Saving service mon spec with placement smithi117:172.21.15.117=a;smithi138:172.21.15.138=b;smithi192:172.21.15.192=c;count:3 2024-09-21T05:48:18.147 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: Deploying daemon mon.c on smithi192 2024-09-21T05:48:18.147 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:18.147 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='client.? 172.21.15.192:0/2785297717' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T05:48:18.147 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T05:48:18.147 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: mon.a calling monitor election 2024-09-21T05:48:18.147 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:48:18.147 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:18.147 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:48:18.148 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:48:18.148 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: mon.c calling monitor election 2024-09-21T05:48:18.148 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:18.148 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:48:18.148 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:48:18.148 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:48:18.148 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T05:48:18.148 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: monmap epoch 2 2024-09-21T05:48:18.148 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: fsid ac8581a2-77dc-11ef-baf5-efdc52797490 2024-09-21T05:48:18.148 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: last_changed 2024-09-21T05:48:08.846539+0000 2024-09-21T05:48:18.148 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: created 2024-09-21T05:46:44.194497+0000 2024-09-21T05:48:18.148 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: min_mon_release 19 (squid) 2024-09-21T05:48:18.148 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: election_strategy: 1 2024-09-21T05:48:18.148 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: 0: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.a 2024-09-21T05:48:18.148 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.c 2024-09-21T05:48:18.149 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: fsmap 2024-09-21T05:48:18.149 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T05:48:18.149 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: mgrmap e13: x(active, since 40s) 2024-09-21T05:48:18.149 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: overall HEALTH_OK 2024-09-21T05:48:18.149 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:18.149 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:18.149 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:18.149 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T05:48:18.149 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:18.149 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:18.149 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: Deploying daemon mon.b on smithi138 2024-09-21T05:48:18.149 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='client.? 172.21.15.192:0/1506302290' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T05:48:18.149 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:48:18.149 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:18.149 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 ceph-mon[34391]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-21T05:48:18.150 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 bash[34373]: 01a69c7125fc345f42649f069275f216a94e0c351ca6f1d88d49a21cb64e5568 2024-09-21T05:48:18.150 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:17 smithi138 systemd[1]: Started Ceph mon.b for ac8581a2-77dc-11ef-baf5-efdc52797490. 2024-09-21T05:48:18.897 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T05:48:18.897 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T05:48:19.150 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T05:48:23.106 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-21T05:48:23.107 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":3,"fsid":"ac8581a2-77dc-11ef-baf5-efdc52797490","modified":"2024-09-21T05:48:17.708244Z","created":"2024-09-21T05:46:44.194497Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:3300","nonce":0},{"type":"v1","addr":"172.21.15.138:6789","nonce":0}]},"addr":"172.21.15.138:6789/0","public_addr":"172.21.15.138:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-21T05:48:23.107 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 3 2024-09-21T05:48:23.107 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:22 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T05:48:23.107 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:22 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T05:48:23.107 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:22 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:48:23.107 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:22 smithi192 ceph-mon[34700]: mon.a calling monitor election 2024-09-21T05:48:23.107 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:22 smithi192 ceph-mon[34700]: mon.c calling monitor election 2024-09-21T05:48:23.107 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:22 smithi192 ceph-mon[34700]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:23.108 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:22 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T05:48:23.108 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:22 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T05:48:23.108 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:22 smithi192 ceph-mon[34700]: mon.b calling monitor election 2024-09-21T05:48:23.108 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:22 smithi192 ceph-mon[34700]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:23.108 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:22 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T05:48:23.108 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:22 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T05:48:23.108 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:22 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T05:48:23.108 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:22 smithi192 ceph-mon[34700]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T05:48:23.108 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:22 smithi192 ceph-mon[34700]: monmap epoch 3 2024-09-21T05:48:23.108 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:22 smithi192 ceph-mon[34700]: fsid ac8581a2-77dc-11ef-baf5-efdc52797490 2024-09-21T05:48:23.108 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:22 smithi192 ceph-mon[34700]: last_changed 2024-09-21T05:48:17.708244+0000 2024-09-21T05:48:23.108 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:22 smithi192 ceph-mon[34700]: created 2024-09-21T05:46:44.194497+0000 2024-09-21T05:48:23.108 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:22 smithi192 ceph-mon[34700]: min_mon_release 19 (squid) 2024-09-21T05:48:23.108 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:22 smithi192 ceph-mon[34700]: election_strategy: 1 2024-09-21T05:48:23.108 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:22 smithi192 ceph-mon[34700]: 0: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.a 2024-09-21T05:48:23.108 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:22 smithi192 ceph-mon[34700]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.c 2024-09-21T05:48:23.109 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:22 smithi192 ceph-mon[34700]: 2: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.b 2024-09-21T05:48:23.109 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:22 smithi192 ceph-mon[34700]: fsmap 2024-09-21T05:48:23.109 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:22 smithi192 ceph-mon[34700]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T05:48:23.109 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:22 smithi192 ceph-mon[34700]: mgrmap e13: x(active, since 48s) 2024-09-21T05:48:23.109 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:22 smithi192 ceph-mon[34700]: overall HEALTH_OK 2024-09-21T05:48:23.109 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:22 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:23.109 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:22 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:23.109 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:22 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:23.109 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:22 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:23.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:22 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T05:48:23.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:22 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T05:48:23.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:22 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:48:23.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:22 smithi138 ceph-mon[34391]: mon.a calling monitor election 2024-09-21T05:48:23.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:22 smithi138 ceph-mon[34391]: mon.c calling monitor election 2024-09-21T05:48:23.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:22 smithi138 ceph-mon[34391]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:23.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:22 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T05:48:23.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:22 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T05:48:23.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:22 smithi138 ceph-mon[34391]: mon.b calling monitor election 2024-09-21T05:48:23.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:22 smithi138 ceph-mon[34391]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:23.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:22 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T05:48:23.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:22 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T05:48:23.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:22 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T05:48:23.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:22 smithi138 ceph-mon[34391]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T05:48:23.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:22 smithi138 ceph-mon[34391]: monmap epoch 3 2024-09-21T05:48:23.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:22 smithi138 ceph-mon[34391]: fsid ac8581a2-77dc-11ef-baf5-efdc52797490 2024-09-21T05:48:23.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:22 smithi138 ceph-mon[34391]: last_changed 2024-09-21T05:48:17.708244+0000 2024-09-21T05:48:23.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:22 smithi138 ceph-mon[34391]: created 2024-09-21T05:46:44.194497+0000 2024-09-21T05:48:23.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:22 smithi138 ceph-mon[34391]: min_mon_release 19 (squid) 2024-09-21T05:48:23.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:22 smithi138 ceph-mon[34391]: election_strategy: 1 2024-09-21T05:48:23.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:22 smithi138 ceph-mon[34391]: 0: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.a 2024-09-21T05:48:23.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:22 smithi138 ceph-mon[34391]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.c 2024-09-21T05:48:23.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:22 smithi138 ceph-mon[34391]: 2: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.b 2024-09-21T05:48:23.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:22 smithi138 ceph-mon[34391]: fsmap 2024-09-21T05:48:23.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:22 smithi138 ceph-mon[34391]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T05:48:23.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:22 smithi138 ceph-mon[34391]: mgrmap e13: x(active, since 48s) 2024-09-21T05:48:23.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:22 smithi138 ceph-mon[34391]: overall HEALTH_OK 2024-09-21T05:48:23.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:22 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:23.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:22 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:23.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:22 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:23.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:22 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:23.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:22 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T05:48:23.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:22 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T05:48:23.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:22 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:48:23.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:22 smithi117 ceph-mon[30947]: mon.a calling monitor election 2024-09-21T05:48:23.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:22 smithi117 ceph-mon[30947]: mon.c calling monitor election 2024-09-21T05:48:23.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:22 smithi117 ceph-mon[30947]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:23.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:22 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T05:48:23.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:22 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T05:48:23.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:22 smithi117 ceph-mon[30947]: mon.b calling monitor election 2024-09-21T05:48:23.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:22 smithi117 ceph-mon[30947]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:23.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:22 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T05:48:23.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:22 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T05:48:23.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:22 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T05:48:23.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:22 smithi117 ceph-mon[30947]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T05:48:23.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:22 smithi117 ceph-mon[30947]: monmap epoch 3 2024-09-21T05:48:23.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:22 smithi117 ceph-mon[30947]: fsid ac8581a2-77dc-11ef-baf5-efdc52797490 2024-09-21T05:48:23.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:22 smithi117 ceph-mon[30947]: last_changed 2024-09-21T05:48:17.708244+0000 2024-09-21T05:48:23.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:22 smithi117 ceph-mon[30947]: created 2024-09-21T05:46:44.194497+0000 2024-09-21T05:48:23.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:22 smithi117 ceph-mon[30947]: min_mon_release 19 (squid) 2024-09-21T05:48:23.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:22 smithi117 ceph-mon[30947]: election_strategy: 1 2024-09-21T05:48:23.279 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:22 smithi117 ceph-mon[30947]: 0: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.a 2024-09-21T05:48:23.279 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:22 smithi117 ceph-mon[30947]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.c 2024-09-21T05:48:23.279 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:22 smithi117 ceph-mon[30947]: 2: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.b 2024-09-21T05:48:23.279 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:22 smithi117 ceph-mon[30947]: fsmap 2024-09-21T05:48:23.279 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:22 smithi117 ceph-mon[30947]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T05:48:23.279 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:22 smithi117 ceph-mon[30947]: mgrmap e13: x(active, since 48s) 2024-09-21T05:48:23.279 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:22 smithi117 ceph-mon[30947]: overall HEALTH_OK 2024-09-21T05:48:23.279 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:22 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:23.279 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:22 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:23.279 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:22 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:23.279 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:22 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:23.679 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-21T05:48:23.679 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph config generate-minimal-conf 2024-09-21T05:48:23.939 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:48:23.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:23 smithi192 ceph-mon[34700]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:23.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:23 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:48:23.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:23 smithi192 ceph-mon[34700]: from='client.? 172.21.15.192:0/1141029341' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T05:48:23.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:23 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T05:48:24.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:23 smithi117 ceph-mon[30947]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:24.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:23 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:48:24.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:23 smithi117 ceph-mon[30947]: from='client.? 172.21.15.192:0/1141029341' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T05:48:24.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:23 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T05:48:24.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:23 smithi138 ceph-mon[34391]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:24.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:23 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:48:24.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:23 smithi138 ceph-mon[34391]: from='client.? 172.21.15.192:0/1141029341' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T05:48:24.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:23 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T05:48:24.619 INFO:teuthology.orchestra.run.smithi117.stdout:# minimal ceph.conf for ac8581a2-77dc-11ef-baf5-efdc52797490 2024-09-21T05:48:24.619 INFO:teuthology.orchestra.run.smithi117.stdout:[global] 2024-09-21T05:48:24.619 INFO:teuthology.orchestra.run.smithi117.stdout: fsid = ac8581a2-77dc-11ef-baf5-efdc52797490 2024-09-21T05:48:24.619 INFO:teuthology.orchestra.run.smithi117.stdout: mon_host = [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] 2024-09-21T05:48:25.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:24 smithi117 ceph-mon[30947]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:25.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:24 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:24 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:24 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:24 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:24 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:24 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:25.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:24 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:48:25.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:24 smithi117 ceph-mon[30947]: Updating smithi117:/etc/ceph/ceph.conf 2024-09-21T05:48:25.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:24 smithi117 ceph-mon[30947]: Updating smithi138:/etc/ceph/ceph.conf 2024-09-21T05:48:25.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:24 smithi117 ceph-mon[30947]: Updating smithi192:/etc/ceph/ceph.conf 2024-09-21T05:48:25.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:24 smithi117 ceph-mon[30947]: Updating smithi192:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T05:48:25.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:24 smithi117 ceph-mon[30947]: Updating smithi117:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T05:48:25.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:24 smithi117 ceph-mon[30947]: Updating smithi138:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T05:48:25.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:24 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/4101494848' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:25.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:24 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:24 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:24 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:24 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.029 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:24 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.029 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:24 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.029 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:24 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.029 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:24 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.029 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:24 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.029 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:24 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.029 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:24 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.029 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:24 smithi117 ceph-mon[30947]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T05:48:25.029 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:24 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T05:48:25.029 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:24 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T05:48:25.029 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:24 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:25.029 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:24 smithi117 ceph-mon[30947]: Reconfiguring daemon mon.a on smithi117 2024-09-21T05:48:25.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:24 smithi138 ceph-mon[34391]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:25.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:24 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:24 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:24 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:24 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:24 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:24 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:25.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:24 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:48:25.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:24 smithi138 ceph-mon[34391]: Updating smithi117:/etc/ceph/ceph.conf 2024-09-21T05:48:25.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:24 smithi138 ceph-mon[34391]: Updating smithi138:/etc/ceph/ceph.conf 2024-09-21T05:48:25.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:24 smithi138 ceph-mon[34391]: Updating smithi192:/etc/ceph/ceph.conf 2024-09-21T05:48:25.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:24 smithi138 ceph-mon[34391]: Updating smithi192:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T05:48:25.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:24 smithi138 ceph-mon[34391]: Updating smithi117:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T05:48:25.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:24 smithi138 ceph-mon[34391]: Updating smithi138:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T05:48:25.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:24 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/4101494848' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:25.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:24 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:24 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:24 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:24 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:24 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:24 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:24 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:24 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:24 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:24 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:24 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:24 smithi138 ceph-mon[34391]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T05:48:25.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:24 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T05:48:25.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:24 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T05:48:25.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:24 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:25.118 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:24 smithi138 ceph-mon[34391]: Reconfiguring daemon mon.a on smithi117 2024-09-21T05:48:25.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:24 smithi192 ceph-mon[34700]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:25.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:24 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:24 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:24 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:24 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:24 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:24 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:25.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:24 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:48:25.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:24 smithi192 ceph-mon[34700]: Updating smithi117:/etc/ceph/ceph.conf 2024-09-21T05:48:25.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:24 smithi192 ceph-mon[34700]: Updating smithi138:/etc/ceph/ceph.conf 2024-09-21T05:48:25.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:24 smithi192 ceph-mon[34700]: Updating smithi192:/etc/ceph/ceph.conf 2024-09-21T05:48:25.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:24 smithi192 ceph-mon[34700]: Updating smithi192:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T05:48:25.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:24 smithi192 ceph-mon[34700]: Updating smithi117:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T05:48:25.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:24 smithi192 ceph-mon[34700]: Updating smithi138:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T05:48:25.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:24 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/4101494848' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:25.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:24 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:24 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:24 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:24 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:24 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:24 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:24 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.232 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:24 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.232 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:24 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.232 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:24 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.232 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:24 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:25.232 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:24 smithi192 ceph-mon[34700]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T05:48:25.232 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:24 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T05:48:25.232 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:24 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T05:48:25.232 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:24 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:25.232 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:24 smithi192 ceph-mon[34700]: Reconfiguring daemon mon.a on smithi117 2024-09-21T05:48:25.367 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-21T05:48:25.367 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-21T05:48:25.368 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T05:48:25.394 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-21T05:48:25.394 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T05:48:25.458 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-21T05:48:25.458 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T05:48:25.489 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-21T05:48:25.489 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T05:48:25.557 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-21T05:48:25.557 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T05:48:25.586 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-21T05:48:25.586 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T05:48:25.653 INFO:tasks.cephadm:enabling module snap_schedule 2024-09-21T05:48:25.653 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- sudo ceph --cluster ceph mgr module enable snap_schedule 2024-09-21T05:48:26.002 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:48:27.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:27 smithi192 ceph-mon[34700]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:27.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:27 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:27.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:27 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:27.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:27 smithi192 ceph-mon[34700]: Reconfiguring mon.b (monmap changed)... 2024-09-21T05:48:27.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:27 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T05:48:27.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:27 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T05:48:27.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:27 smithi192 ceph-mon[34700]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:27.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:27 smithi192 ceph-mon[34700]: Reconfiguring daemon mon.b on smithi138 2024-09-21T05:48:27.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:27 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2100023723' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T05:48:27.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:27 smithi117 ceph-mon[30947]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:27.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:27 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:27.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:27 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:27.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:27 smithi117 ceph-mon[30947]: Reconfiguring mon.b (monmap changed)... 2024-09-21T05:48:27.778 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:27 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T05:48:27.778 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:27 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T05:48:27.778 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:27 smithi117 ceph-mon[30947]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:27.778 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:27 smithi117 ceph-mon[30947]: Reconfiguring daemon mon.b on smithi138 2024-09-21T05:48:27.778 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:27 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2100023723' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T05:48:27.778 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:48:27 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: ignoring --setuser ceph since I am not root 2024-09-21T05:48:27.778 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:48:27 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: ignoring --setgroup ceph since I am not root 2024-09-21T05:48:27.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:27 smithi138 ceph-mon[34391]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:27.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:27 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:27.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:27 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' 2024-09-21T05:48:27.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:27 smithi138 ceph-mon[34391]: Reconfiguring mon.b (monmap changed)... 2024-09-21T05:48:27.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:27 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T05:48:27.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:27 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T05:48:27.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:27 smithi138 ceph-mon[34391]: from='mgr.14152 172.21.15.117:0/2162191632' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:27.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:27 smithi138 ceph-mon[34391]: Reconfiguring daemon mon.b on smithi138 2024-09-21T05:48:27.852 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:27 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2100023723' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T05:48:28.138 INFO:tasks.cephadm:Adding mgr.x on smithi117 2024-09-21T05:48:28.138 INFO:tasks.cephadm:Adding mgr.y on smithi138 2024-09-21T05:48:28.138 INFO:tasks.cephadm:Adding mgr.z on smithi192 2024-09-21T05:48:28.138 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph orch apply mgr '3;smithi117=x;smithi138=y;smithi192=z' 2024-09-21T05:48:28.375 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T05:48:28.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:28 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2100023723' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T05:48:28.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:28 smithi117 ceph-mon[30947]: mgrmap e14: x(active, since 53s) 2024-09-21T05:48:28.797 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:28 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2100023723' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T05:48:28.800 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:28 smithi192 ceph-mon[34700]: mgrmap e14: x(active, since 53s) 2024-09-21T05:48:28.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:28 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2100023723' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T05:48:28.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:28 smithi138 ceph-mon[34391]: mgrmap e14: x(active, since 53s) 2024-09-21T05:48:32.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:32 smithi192 ceph-mon[34700]: Active manager daemon x restarted 2024-09-21T05:48:32.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:32 smithi192 ceph-mon[34700]: Activating manager daemon x 2024-09-21T05:48:32.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:32 smithi192 ceph-mon[34700]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T05:48:32.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:32 smithi192 ceph-mon[34700]: mgrmap e15: x(active, starting, since 0.00792367s) 2024-09-21T05:48:32.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:32 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T05:48:32.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:32 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T05:48:32.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:32 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:48:32.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:32 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T05:48:32.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:32 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T05:48:32.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:32 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T05:48:32.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:32 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T05:48:32.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:32 smithi192 ceph-mon[34700]: Manager daemon x is now available 2024-09-21T05:48:32.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:32 smithi117 ceph-mon[30947]: Active manager daemon x restarted 2024-09-21T05:48:32.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:32 smithi117 ceph-mon[30947]: Activating manager daemon x 2024-09-21T05:48:32.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:32 smithi117 ceph-mon[30947]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T05:48:32.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:32 smithi117 ceph-mon[30947]: mgrmap e15: x(active, starting, since 0.00792367s) 2024-09-21T05:48:32.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:32 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T05:48:32.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:32 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T05:48:32.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:32 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:48:32.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:32 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T05:48:32.778 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:32 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T05:48:32.778 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:32 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T05:48:32.778 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:32 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T05:48:32.778 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:32 smithi117 ceph-mon[30947]: Manager daemon x is now available 2024-09-21T05:48:32.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:32 smithi138 ceph-mon[34391]: Active manager daemon x restarted 2024-09-21T05:48:32.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:32 smithi138 ceph-mon[34391]: Activating manager daemon x 2024-09-21T05:48:32.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:32 smithi138 ceph-mon[34391]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T05:48:32.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:32 smithi138 ceph-mon[34391]: mgrmap e15: x(active, starting, since 0.00792367s) 2024-09-21T05:48:32.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:32 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T05:48:32.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:32 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T05:48:32.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:32 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:48:32.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:32 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T05:48:32.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:32 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T05:48:32.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:32 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T05:48:32.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:32 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T05:48:32.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:32 smithi138 ceph-mon[34391]: Manager daemon x is now available 2024-09-21T05:48:33.425 INFO:teuthology.orchestra.run.smithi192.stdout:Scheduled mgr update... 2024-09-21T05:48:33.717 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:33 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:48:33.717 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:33 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T05:48:33.717 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:33 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T05:48:33.717 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:33 smithi192 ceph-mon[34700]: mgrmap e16: x(active, since 1.01146s) 2024-09-21T05:48:33.717 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:33 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:33.722 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:33 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:48:33.722 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:33 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T05:48:33.722 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:33 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T05:48:33.722 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:33 smithi117 ceph-mon[30947]: mgrmap e16: x(active, since 1.01146s) 2024-09-21T05:48:33.722 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:33 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:33.801 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:33 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:48:33.801 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:33 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T05:48:33.801 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:33 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T05:48:33.801 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:33 smithi138 ceph-mon[34391]: mgrmap e16: x(active, since 1.01146s) 2024-09-21T05:48:33.801 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:33 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:34.027 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:48:33 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: 2024-09-21T05:48:33.718+0000 7f95a1b87640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:05:48:33] ENGINE Error in HTTPServer.serve 2024-09-21T05:48:34.027 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:48:33 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: Traceback (most recent call last): 2024-09-21T05:48:34.027 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:48:33 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T05:48:34.027 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:48:33 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: self._connections.run(self.expiration_interval) 2024-09-21T05:48:34.027 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:48:33 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T05:48:34.027 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:48:33 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: self._run(expiration_interval) 2024-09-21T05:48:34.028 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:48:33 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T05:48:34.028 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:48:33 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T05:48:34.028 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:48:33 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T05:48:34.028 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:48:33 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T05:48:34.028 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:48:33 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T05:48:34.028 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:48:33 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: s = self.context.wrap_socket( 2024-09-21T05:48:34.028 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:48:33 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T05:48:34.028 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:48:33 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: return self.sslsocket_class._create( 2024-09-21T05:48:34.028 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:48:33 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T05:48:34.028 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:48:33 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: self.do_handshake() 2024-09-21T05:48:34.028 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:48:33 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T05:48:34.028 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:48:33 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: self._sslobj.do_handshake() 2024-09-21T05:48:34.028 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:48:33 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T05:48:34.028 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:48:33 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: 2024-09-21T05:48:34.047 DEBUG:teuthology.orchestra.run.smithi138:mgr.y> sudo journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mgr.y.service 2024-09-21T05:48:34.049 DEBUG:teuthology.orchestra.run.smithi192:mgr.z> sudo journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mgr.z.service 2024-09-21T05:48:34.051 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-09-21T05:48:34.051 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph config set client client_mount_timeout 600 2024-09-21T05:48:34.053 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-09-21T05:48:34.054 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph config set client debug_client 20 2024-09-21T05:48:34.096 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-09-21T05:48:34.097 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph config set client debug_ms 1 2024-09-21T05:48:34.141 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-09-21T05:48:34.141 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph config set client rados_mon_op_timeout 900 2024-09-21T05:48:34.185 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-09-21T05:48:34.185 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph config set client rados_osd_op_timeout 900 2024-09-21T05:48:34.229 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-09-21T05:48:34.229 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds 20 2024-09-21T05:48:34.273 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-09-21T05:48:34.273 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds_balancer 20 2024-09-21T05:48:34.318 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-09-21T05:48:34.318 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph config set mds debug_ms 2 2024-09-21T05:48:34.366 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-09-21T05:48:34.366 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-09-21T05:48:34.403 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:48:34.409 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-09-21T05:48:34.409 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_merge_size 5 2024-09-21T05:48:34.460 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-09-21T05:48:34.460 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_bits 3 2024-09-21T05:48:34.515 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-09-21T05:48:34.516 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_size 100 2024-09-21T05:48:34.562 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-09-21T05:48:34.562 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_frag True 2024-09-21T05:48:34.609 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-09-21T05:48:34.609 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_scatterstat True 2024-09-21T05:48:34.665 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-09-21T05:48:34.665 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph config set mds mds_op_complaint_time 180 2024-09-21T05:48:34.669 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:34 smithi117 ceph-mon[30947]: Saving service mgr spec with placement smithi117=x;smithi138=y;smithi192=z;count:3 2024-09-21T05:48:34.669 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:34 smithi117 ceph-mon[30947]: [21/Sep/2024:05:48:33] ENGINE Bus STARTING 2024-09-21T05:48:34.669 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:34 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:34.669 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:34 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:34.669 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:34 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:34.669 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:34 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:34.693 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-09-21T05:48:34.693 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph config set mds mds_verify_scatter True 2024-09-21T05:48:34.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:34 smithi192 ceph-mon[34700]: Saving service mgr spec with placement smithi117=x;smithi138=y;smithi192=z;count:3 2024-09-21T05:48:34.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:34 smithi192 ceph-mon[34700]: [21/Sep/2024:05:48:33] ENGINE Bus STARTING 2024-09-21T05:48:34.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:34 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:34.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:34 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:34.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:34 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:34.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:34 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:34.750 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-09-21T05:48:34.750 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph config set mds osd_op_complaint_time 180 2024-09-21T05:48:34.810 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-09-21T05:48:34.810 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph config set mds rados_mon_op_timeout 900 2024-09-21T05:48:34.864 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-09-21T05:48:34.864 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph config set mds rados_osd_op_timeout 900 2024-09-21T05:48:34.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:34 smithi138 ceph-mon[34391]: Saving service mgr spec with placement smithi117=x;smithi138=y;smithi192=z;count:3 2024-09-21T05:48:34.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:34 smithi138 ceph-mon[34391]: [21/Sep/2024:05:48:33] ENGINE Bus STARTING 2024-09-21T05:48:34.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:34 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:34.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:34 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:34.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:34 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:34.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:34 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:34.915 INFO:tasks.cephadm:setting config [mgr] client_mount_timeout = 30 2024-09-21T05:48:34.915 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph config set mgr client_mount_timeout 30 2024-09-21T05:48:34.965 INFO:tasks.cephadm:setting config [mgr] debug_cephsqlite = 20 2024-09-21T05:48:34.965 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph config set mgr debug_cephsqlite 20 2024-09-21T05:48:35.031 INFO:tasks.cephadm:setting config [mgr] debug_client = 20 2024-09-21T05:48:35.031 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph config set mgr debug_client 20 2024-09-21T05:48:35.110 INFO:tasks.cephadm:setting config [mgr] debug_mgr = 20 2024-09-21T05:48:35.111 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph config set mgr debug_mgr 20 2024-09-21T05:48:35.156 INFO:tasks.cephadm:setting config [mgr] debug_ms = 1 2024-09-21T05:48:35.157 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph config set mgr debug_ms 1 2024-09-21T05:48:35.207 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/allow_m_granularity = True 2024-09-21T05:48:35.208 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-09-21T05:48:35.232 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:48:35.251 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/dump_on_update = True 2024-09-21T05:48:35.251 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-09-21T05:48:35.321 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/log_level = debug 2024-09-21T05:48:35.321 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/log_level debug 2024-09-21T05:48:35.386 INFO:tasks.cephadm:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-21T05:48:35.386 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph config set mgr mon_warn_on_pool_no_app False 2024-09-21T05:48:35.430 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-09-21T05:48:35.430 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph config set mon mon_op_complaint_time 120 2024-09-21T05:48:35.477 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-09-21T05:48:35.477 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph config set osd osd_op_complaint_time 180 2024-09-21T05:48:35.528 DEBUG:tasks.cephadm:set 30 configs 2024-09-21T05:48:35.528 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ac8581a2-77dc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi117') 2024-09-21T05:48:35.737 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:35 smithi117 ceph-mon[30947]: [21/Sep/2024:05:48:33] ENGINE Serving on http://172.21.15.117:8765 2024-09-21T05:48:35.737 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:35 smithi117 ceph-mon[30947]: [21/Sep/2024:05:48:33] ENGINE Serving on https://172.21.15.117:7150 2024-09-21T05:48:35.738 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:35 smithi117 ceph-mon[30947]: [21/Sep/2024:05:48:33] ENGINE Bus STARTED 2024-09-21T05:48:35.738 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:35 smithi117 ceph-mon[30947]: [21/Sep/2024:05:48:33] ENGINE Error in HTTPServer.serve 2024-09-21T05:48:35.738 INFO:journalctl@ceph.mon.a.smithi117.stdout: Traceback (most recent call last): 2024-09-21T05:48:35.738 INFO:journalctl@ceph.mon.a.smithi117.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T05:48:35.738 INFO:journalctl@ceph.mon.a.smithi117.stdout: self._connections.run(self.expiration_interval) 2024-09-21T05:48:35.738 INFO:journalctl@ceph.mon.a.smithi117.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T05:48:35.738 INFO:journalctl@ceph.mon.a.smithi117.stdout: self._run(expiration_interval) 2024-09-21T05:48:35.738 INFO:journalctl@ceph.mon.a.smithi117.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T05:48:35.738 INFO:journalctl@ceph.mon.a.smithi117.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T05:48:35.738 INFO:journalctl@ceph.mon.a.smithi117.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T05:48:35.738 INFO:journalctl@ceph.mon.a.smithi117.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T05:48:35.738 INFO:journalctl@ceph.mon.a.smithi117.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T05:48:35.739 INFO:journalctl@ceph.mon.a.smithi117.stdout: s = self.context.wrap_socket( 2024-09-21T05:48:35.739 INFO:journalctl@ceph.mon.a.smithi117.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T05:48:35.739 INFO:journalctl@ceph.mon.a.smithi117.stdout: return self.sslsocket_class._create( 2024-09-21T05:48:35.739 INFO:journalctl@ceph.mon.a.smithi117.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T05:48:35.739 INFO:journalctl@ceph.mon.a.smithi117.stdout: self.do_handshake() 2024-09-21T05:48:35.739 INFO:journalctl@ceph.mon.a.smithi117.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T05:48:35.739 INFO:journalctl@ceph.mon.a.smithi117.stdout: self._sslobj.do_handshake() 2024-09-21T05:48:35.739 INFO:journalctl@ceph.mon.a.smithi117.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T05:48:35.739 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:35.739 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:35.739 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:35.739 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:35.739 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi138", "name": "osd_memory_target"} : dispatch 2024-09-21T05:48:35.739 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:35 smithi117 ceph-mon[30947]: mgrmap e17: x(active, since 2s) 2024-09-21T05:48:35.739 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:35.740 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:35.740 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi192", "name": "osd_memory_target"} : dispatch 2024-09-21T05:48:35.740 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:35.740 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:35.740 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi117", "name": "osd_memory_target"} : dispatch 2024-09-21T05:48:35.740 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:35.740 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:48:35.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:35 smithi138 ceph-mon[34391]: [21/Sep/2024:05:48:33] ENGINE Serving on http://172.21.15.117:8765 2024-09-21T05:48:35.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:35 smithi138 ceph-mon[34391]: [21/Sep/2024:05:48:33] ENGINE Serving on https://172.21.15.117:7150 2024-09-21T05:48:35.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:35 smithi138 ceph-mon[34391]: [21/Sep/2024:05:48:33] ENGINE Bus STARTED 2024-09-21T05:48:35.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:35 smithi138 ceph-mon[34391]: [21/Sep/2024:05:48:33] ENGINE Error in HTTPServer.serve 2024-09-21T05:48:35.866 INFO:journalctl@ceph.mon.b.smithi138.stdout: Traceback (most recent call last): 2024-09-21T05:48:35.866 INFO:journalctl@ceph.mon.b.smithi138.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T05:48:35.866 INFO:journalctl@ceph.mon.b.smithi138.stdout: self._connections.run(self.expiration_interval) 2024-09-21T05:48:35.866 INFO:journalctl@ceph.mon.b.smithi138.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T05:48:35.866 INFO:journalctl@ceph.mon.b.smithi138.stdout: self._run(expiration_interval) 2024-09-21T05:48:35.866 INFO:journalctl@ceph.mon.b.smithi138.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T05:48:35.866 INFO:journalctl@ceph.mon.b.smithi138.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T05:48:35.866 INFO:journalctl@ceph.mon.b.smithi138.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T05:48:35.867 INFO:journalctl@ceph.mon.b.smithi138.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T05:48:35.867 INFO:journalctl@ceph.mon.b.smithi138.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T05:48:35.867 INFO:journalctl@ceph.mon.b.smithi138.stdout: s = self.context.wrap_socket( 2024-09-21T05:48:35.867 INFO:journalctl@ceph.mon.b.smithi138.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T05:48:35.867 INFO:journalctl@ceph.mon.b.smithi138.stdout: return self.sslsocket_class._create( 2024-09-21T05:48:35.867 INFO:journalctl@ceph.mon.b.smithi138.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T05:48:35.867 INFO:journalctl@ceph.mon.b.smithi138.stdout: self.do_handshake() 2024-09-21T05:48:35.867 INFO:journalctl@ceph.mon.b.smithi138.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T05:48:35.867 INFO:journalctl@ceph.mon.b.smithi138.stdout: self._sslobj.do_handshake() 2024-09-21T05:48:35.867 INFO:journalctl@ceph.mon.b.smithi138.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T05:48:35.867 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:35.867 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:35.867 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:35.867 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:35.867 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi138", "name": "osd_memory_target"} : dispatch 2024-09-21T05:48:35.868 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:35 smithi138 ceph-mon[34391]: mgrmap e17: x(active, since 2s) 2024-09-21T05:48:35.868 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:35.868 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:35.868 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi192", "name": "osd_memory_target"} : dispatch 2024-09-21T05:48:35.868 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:35.868 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:35.868 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi117", "name": "osd_memory_target"} : dispatch 2024-09-21T05:48:35.868 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:35.868 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:48:35.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:35 smithi192 ceph-mon[34700]: [21/Sep/2024:05:48:33] ENGINE Serving on http://172.21.15.117:8765 2024-09-21T05:48:35.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:35 smithi192 ceph-mon[34700]: [21/Sep/2024:05:48:33] ENGINE Serving on https://172.21.15.117:7150 2024-09-21T05:48:35.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:35 smithi192 ceph-mon[34700]: [21/Sep/2024:05:48:33] ENGINE Bus STARTED 2024-09-21T05:48:35.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:35 smithi192 ceph-mon[34700]: [21/Sep/2024:05:48:33] ENGINE Error in HTTPServer.serve 2024-09-21T05:48:35.980 INFO:journalctl@ceph.mon.c.smithi192.stdout: Traceback (most recent call last): 2024-09-21T05:48:35.980 INFO:journalctl@ceph.mon.c.smithi192.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T05:48:35.981 INFO:journalctl@ceph.mon.c.smithi192.stdout: self._connections.run(self.expiration_interval) 2024-09-21T05:48:35.981 INFO:journalctl@ceph.mon.c.smithi192.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T05:48:35.981 INFO:journalctl@ceph.mon.c.smithi192.stdout: self._run(expiration_interval) 2024-09-21T05:48:35.981 INFO:journalctl@ceph.mon.c.smithi192.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T05:48:35.981 INFO:journalctl@ceph.mon.c.smithi192.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T05:48:35.981 INFO:journalctl@ceph.mon.c.smithi192.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T05:48:35.981 INFO:journalctl@ceph.mon.c.smithi192.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T05:48:35.981 INFO:journalctl@ceph.mon.c.smithi192.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T05:48:35.981 INFO:journalctl@ceph.mon.c.smithi192.stdout: s = self.context.wrap_socket( 2024-09-21T05:48:35.981 INFO:journalctl@ceph.mon.c.smithi192.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T05:48:35.981 INFO:journalctl@ceph.mon.c.smithi192.stdout: return self.sslsocket_class._create( 2024-09-21T05:48:35.981 INFO:journalctl@ceph.mon.c.smithi192.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T05:48:35.981 INFO:journalctl@ceph.mon.c.smithi192.stdout: self.do_handshake() 2024-09-21T05:48:35.981 INFO:journalctl@ceph.mon.c.smithi192.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T05:48:35.982 INFO:journalctl@ceph.mon.c.smithi192.stdout: self._sslobj.do_handshake() 2024-09-21T05:48:35.982 INFO:journalctl@ceph.mon.c.smithi192.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T05:48:35.982 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:35.982 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:35.982 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:35.982 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:35.982 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi138", "name": "osd_memory_target"} : dispatch 2024-09-21T05:48:35.982 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:35 smithi192 ceph-mon[34700]: mgrmap e17: x(active, since 2s) 2024-09-21T05:48:35.982 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:35.982 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:35.982 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi192", "name": "osd_memory_target"} : dispatch 2024-09-21T05:48:35.982 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:35.982 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:35.982 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi117", "name": "osd_memory_target"} : dispatch 2024-09-21T05:48:35.983 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:35.983 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:48:36.186 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:48:36.195 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:48:36.243 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:48:36.249 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:48:36.250 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:48:36.257 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:48:36.670 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:36 smithi117 ceph-mon[30947]: Updating smithi117:/etc/ceph/ceph.conf 2024-09-21T05:48:36.670 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:36 smithi117 ceph-mon[30947]: Updating smithi138:/etc/ceph/ceph.conf 2024-09-21T05:48:36.670 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:36 smithi117 ceph-mon[30947]: Updating smithi192:/etc/ceph/ceph.conf 2024-09-21T05:48:36.670 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:36 smithi117 ceph-mon[30947]: Updating smithi138:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T05:48:36.670 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:36 smithi117 ceph-mon[30947]: Updating smithi192:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T05:48:36.670 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:36 smithi117 ceph-mon[30947]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-09-21T05:48:36.670 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:36 smithi117 ceph-mon[30947]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-09-21T05:48:36.670 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:36 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2421601426' entity='client.admin' 2024-09-21T05:48:36.670 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:36 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:36.670 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:36 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:36.670 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:36 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:36.671 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:36 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:36.671 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:36 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/1362459975' entity='client.admin' 2024-09-21T05:48:36.671 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:36 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:36.671 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:36 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:36.671 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:36 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:36.671 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:36 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T05:48:36.671 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:36 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T05:48:36.671 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:36 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T05:48:36.671 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:36 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:36.855 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:36 smithi192 ceph-mon[34700]: Updating smithi117:/etc/ceph/ceph.conf 2024-09-21T05:48:36.856 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:36 smithi192 ceph-mon[34700]: Updating smithi138:/etc/ceph/ceph.conf 2024-09-21T05:48:36.856 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:36 smithi192 ceph-mon[34700]: Updating smithi192:/etc/ceph/ceph.conf 2024-09-21T05:48:36.856 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:36 smithi192 ceph-mon[34700]: Updating smithi138:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T05:48:36.856 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:36 smithi192 ceph-mon[34700]: Updating smithi192:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T05:48:36.856 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:36 smithi192 ceph-mon[34700]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-09-21T05:48:36.856 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:36 smithi192 ceph-mon[34700]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-09-21T05:48:36.856 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:36 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2421601426' entity='client.admin' 2024-09-21T05:48:36.856 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:36 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:36.856 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:36 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:36.856 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:36 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:36.856 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:36 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:36.856 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:36 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/1362459975' entity='client.admin' 2024-09-21T05:48:36.856 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:36 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:36.856 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:36 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:36.857 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:36 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:36.857 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:36 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T05:48:36.857 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:36 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T05:48:36.857 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:36 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T05:48:36.857 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:36 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:36.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:36 smithi138 ceph-mon[34391]: Updating smithi117:/etc/ceph/ceph.conf 2024-09-21T05:48:36.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:36 smithi138 ceph-mon[34391]: Updating smithi138:/etc/ceph/ceph.conf 2024-09-21T05:48:36.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:36 smithi138 ceph-mon[34391]: Updating smithi192:/etc/ceph/ceph.conf 2024-09-21T05:48:36.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:36 smithi138 ceph-mon[34391]: Updating smithi138:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T05:48:36.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:36 smithi138 ceph-mon[34391]: Updating smithi192:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T05:48:36.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:36 smithi138 ceph-mon[34391]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-09-21T05:48:36.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:36 smithi138 ceph-mon[34391]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-09-21T05:48:36.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:36 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2421601426' entity='client.admin' 2024-09-21T05:48:36.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:36 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:36.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:36 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:36.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:36 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:36.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:36 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:36.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:36 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/1362459975' entity='client.admin' 2024-09-21T05:48:36.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:36 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:36.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:36 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:36.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:36 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:36.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:36 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T05:48:36.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:36 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T05:48:36.867 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:36 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T05:48:36.867 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:36 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:37.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:37 smithi117 ceph-mon[30947]: Updating smithi138:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T05:48:37.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:37 smithi117 ceph-mon[30947]: Updating smithi192:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T05:48:37.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:37 smithi117 ceph-mon[30947]: Updating smithi117:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T05:48:37.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:37 smithi117 ceph-mon[30947]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-09-21T05:48:37.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:37 smithi117 ceph-mon[30947]: Updating smithi117:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T05:48:37.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:37 smithi117 ceph-mon[30947]: Deploying daemon mgr.z on smithi192 2024-09-21T05:48:37.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:37 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/1402459027' entity='client.admin' 2024-09-21T05:48:37.809 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:37 smithi192 ceph-mon[34700]: Updating smithi138:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T05:48:37.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:37 smithi192 ceph-mon[34700]: Updating smithi192:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T05:48:37.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:37 smithi192 ceph-mon[34700]: Updating smithi117:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T05:48:37.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:37 smithi192 ceph-mon[34700]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-09-21T05:48:37.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:37 smithi192 ceph-mon[34700]: Updating smithi117:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T05:48:37.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:37 smithi192 ceph-mon[34700]: Deploying daemon mgr.z on smithi192 2024-09-21T05:48:37.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:37 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/1402459027' entity='client.admin' 2024-09-21T05:48:37.839 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:48:37.849 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:48:37.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:37 smithi138 ceph-mon[34391]: Updating smithi138:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T05:48:37.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:37 smithi138 ceph-mon[34391]: Updating smithi192:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T05:48:37.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:37 smithi138 ceph-mon[34391]: Updating smithi117:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T05:48:37.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:37 smithi138 ceph-mon[34391]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-09-21T05:48:37.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:37 smithi138 ceph-mon[34391]: Updating smithi117:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T05:48:37.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:37 smithi138 ceph-mon[34391]: Deploying daemon mgr.z on smithi192 2024-09-21T05:48:37.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:37 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/1402459027' entity='client.admin' 2024-09-21T05:48:38.200 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:48:38.200 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:48:38.200 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:48:38.203 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:48:38.203 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:48:38.211 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:48:38.247 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:48:38.255 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:48:38.730 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 21 05:48:38 smithi192 systemd[1]: Started Ceph mgr.z for ac8581a2-77dc-11ef-baf5-efdc52797490. 2024-09-21T05:48:39.478 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:39 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2637966866' entity='client.admin' 2024-09-21T05:48:39.479 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:39 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:39.479 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:39 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:39.479 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:39 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:39.479 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:39 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T05:48:39.479 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:39 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T05:48:39.479 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:39 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T05:48:39.479 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:39 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:39.512 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:48:39.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:39 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2637966866' entity='client.admin' 2024-09-21T05:48:39.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:39 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:39.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:39 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:39.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:39 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:39.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:39 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T05:48:39.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:39 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T05:48:39.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:39 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T05:48:39.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:39 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:39.741 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:39 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2637966866' entity='client.admin' 2024-09-21T05:48:39.741 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:39 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:39.741 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:39 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:39.741 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:39 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:39.742 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:39 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T05:48:39.742 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:39 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T05:48:39.742 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:39 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T05:48:39.742 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:39 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:40.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:40 smithi138 ceph-mon[34391]: Deploying daemon mgr.y on smithi138 2024-09-21T05:48:40.691 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:40 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3518263332' entity='client.admin' 2024-09-21T05:48:40.729 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:40 smithi192 ceph-mon[34700]: Deploying daemon mgr.y on smithi138 2024-09-21T05:48:40.729 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:40 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3518263332' entity='client.admin' 2024-09-21T05:48:40.753 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:40 smithi117 ceph-mon[30947]: Deploying daemon mgr.y on smithi138 2024-09-21T05:48:40.754 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:40 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3518263332' entity='client.admin' 2024-09-21T05:48:41.115 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 21 05:48:40 smithi138 systemd[1]: Started Ceph mgr.y for ac8581a2-77dc-11ef-baf5-efdc52797490. 2024-09-21T05:48:41.228 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:48:41.228 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:48:41.229 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:48:42.125 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:41 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:42.125 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:41 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:42.125 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:41 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3580271369' entity='client.admin' 2024-09-21T05:48:42.125 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:41 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:42.125 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:41 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:42.125 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:41 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:48:42.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:41 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:42.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:41 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:42.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:41 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3580271369' entity='client.admin' 2024-09-21T05:48:42.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:41 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:42.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:41 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:42.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:41 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:48:42.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:41 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:42.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:41 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:42.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:41 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3580271369' entity='client.admin' 2024-09-21T05:48:42.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:41 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:42.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:41 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:42.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:41 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:48:43.065 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:42 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:43.065 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:42 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:43.065 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:42 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:43.065 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:42 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:43.065 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:42 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:43.065 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:42 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:48:43.065 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:42 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:43.065 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:42 smithi117 ceph-mon[30947]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T05:48:43.065 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:42 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T05:48:43.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:42 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T05:48:43.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:42 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:43.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:42 smithi117 ceph-mon[30947]: Reconfiguring daemon mgr.x on smithi117 2024-09-21T05:48:43.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:42 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:43.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:42 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:43.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:42 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:43.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:42 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:43.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:42 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:43.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:42 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:43.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:42 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:48:43.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:42 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:43.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:42 smithi138 ceph-mon[34391]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T05:48:43.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:42 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T05:48:43.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:42 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T05:48:43.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:42 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:43.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:42 smithi138 ceph-mon[34391]: Reconfiguring daemon mgr.x on smithi117 2024-09-21T05:48:43.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:42 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:43.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:42 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:43.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:42 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:43.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:42 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:43.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:42 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:43.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:42 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:43.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:42 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:48:43.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:42 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:43.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:42 smithi192 ceph-mon[34700]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T05:48:43.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:42 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T05:48:43.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:42 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T05:48:43.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:42 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:43.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:42 smithi192 ceph-mon[34700]: Reconfiguring daemon mgr.x on smithi117 2024-09-21T05:48:43.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:42 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:44.217 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:44 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/156403284' entity='client.admin' 2024-09-21T05:48:44.217 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:44 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/1669027481' entity='client.admin' 2024-09-21T05:48:44.217 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:44 smithi117 ceph-mon[30947]: Standby manager daemon z started 2024-09-21T05:48:44.217 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:44 smithi117 ceph-mon[30947]: from='mgr.? 172.21.15.192:0/3258041413' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T05:48:44.217 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:44 smithi117 ceph-mon[30947]: from='mgr.? 172.21.15.192:0/3258041413' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T05:48:44.217 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:44 smithi117 ceph-mon[30947]: from='mgr.? 172.21.15.192:0/3258041413' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T05:48:44.217 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:44 smithi117 ceph-mon[30947]: from='mgr.? 172.21.15.192:0/3258041413' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T05:48:44.217 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:44 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/961066380' entity='client.admin' 2024-09-21T05:48:44.217 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:44 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/427505022' entity='client.admin' 2024-09-21T05:48:44.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:44 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/156403284' entity='client.admin' 2024-09-21T05:48:44.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:44 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/1669027481' entity='client.admin' 2024-09-21T05:48:44.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:44 smithi138 ceph-mon[34391]: Standby manager daemon z started 2024-09-21T05:48:44.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:44 smithi138 ceph-mon[34391]: from='mgr.? 172.21.15.192:0/3258041413' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T05:48:44.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:44 smithi138 ceph-mon[34391]: from='mgr.? 172.21.15.192:0/3258041413' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T05:48:44.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:44 smithi138 ceph-mon[34391]: from='mgr.? 172.21.15.192:0/3258041413' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T05:48:44.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:44 smithi138 ceph-mon[34391]: from='mgr.? 172.21.15.192:0/3258041413' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T05:48:44.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:44 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/961066380' entity='client.admin' 2024-09-21T05:48:44.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:44 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/427505022' entity='client.admin' 2024-09-21T05:48:44.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:44 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/156403284' entity='client.admin' 2024-09-21T05:48:44.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:44 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/1669027481' entity='client.admin' 2024-09-21T05:48:44.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:44 smithi192 ceph-mon[34700]: Standby manager daemon z started 2024-09-21T05:48:44.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:44 smithi192 ceph-mon[34700]: from='mgr.? 172.21.15.192:0/3258041413' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T05:48:44.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:44 smithi192 ceph-mon[34700]: from='mgr.? 172.21.15.192:0/3258041413' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T05:48:44.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:44 smithi192 ceph-mon[34700]: from='mgr.? 172.21.15.192:0/3258041413' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T05:48:44.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:44 smithi192 ceph-mon[34700]: from='mgr.? 172.21.15.192:0/3258041413' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T05:48:44.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:44 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/961066380' entity='client.admin' 2024-09-21T05:48:44.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:44 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/427505022' entity='client.admin' 2024-09-21T05:48:44.984 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ac8581a2-77dc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi117') 2024-09-21T05:48:44.984 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ac8581a2-77dc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi117') 2024-09-21T05:48:45.601 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:45 smithi117 ceph-mon[30947]: mgrmap e18: x(active, since 12s), standbys: z 2024-09-21T05:48:45.601 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:45 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T05:48:45.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:45 smithi138 ceph-mon[34391]: mgrmap e18: x(active, since 12s), standbys: z 2024-09-21T05:48:45.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:45 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T05:48:45.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:45 smithi192 ceph-mon[34700]: mgrmap e18: x(active, since 12s), standbys: z 2024-09-21T05:48:45.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:45 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T05:48:46.766 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:46 smithi117 ceph-mon[30947]: Standby manager daemon y started 2024-09-21T05:48:46.766 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:46 smithi117 ceph-mon[30947]: from='mgr.? 172.21.15.138:0/3954876164' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T05:48:46.766 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:46 smithi117 ceph-mon[30947]: from='mgr.? 172.21.15.138:0/3954876164' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T05:48:46.766 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:46 smithi117 ceph-mon[30947]: from='mgr.? 172.21.15.138:0/3954876164' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T05:48:46.766 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:46 smithi117 ceph-mon[30947]: from='mgr.? 172.21.15.138:0/3954876164' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T05:48:46.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:46 smithi138 ceph-mon[34391]: Standby manager daemon y started 2024-09-21T05:48:46.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:46 smithi138 ceph-mon[34391]: from='mgr.? 172.21.15.138:0/3954876164' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T05:48:46.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:46 smithi138 ceph-mon[34391]: from='mgr.? 172.21.15.138:0/3954876164' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T05:48:46.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:46 smithi138 ceph-mon[34391]: from='mgr.? 172.21.15.138:0/3954876164' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T05:48:46.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:46 smithi138 ceph-mon[34391]: from='mgr.? 172.21.15.138:0/3954876164' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T05:48:46.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:46 smithi192 ceph-mon[34700]: Standby manager daemon y started 2024-09-21T05:48:46.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:46 smithi192 ceph-mon[34700]: from='mgr.? 172.21.15.138:0/3954876164' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T05:48:46.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:46 smithi192 ceph-mon[34700]: from='mgr.? 172.21.15.138:0/3954876164' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T05:48:46.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:46 smithi192 ceph-mon[34700]: from='mgr.? 172.21.15.138:0/3954876164' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T05:48:46.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:46 smithi192 ceph-mon[34700]: from='mgr.? 172.21.15.138:0/3954876164' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T05:48:47.655 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:47 smithi117 ceph-mon[30947]: mgrmap e19: x(active, since 14s), standbys: z, y 2024-09-21T05:48:47.655 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:47 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T05:48:47.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:47 smithi138 ceph-mon[34391]: mgrmap e19: x(active, since 14s), standbys: z, y 2024-09-21T05:48:47.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:47 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T05:48:47.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:47 smithi192 ceph-mon[34700]: mgrmap e19: x(active, since 14s), standbys: z, y 2024-09-21T05:48:47.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:47 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T05:48:48.898 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:48 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2777023850' entity='client.admin' 2024-09-21T05:48:49.093 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:48:49.094 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:48:49.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:48 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2777023850' entity='client.admin' 2024-09-21T05:48:49.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:48 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2777023850' entity='client.admin' 2024-09-21T05:48:49.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:49.589+0000 7fdd6c942640 1 -- 172.21.15.117:0/2562941309 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fdd500b64d0 msgr2=0x7fdd500b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:49.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:49.589+0000 7fdd6c942640 1 --2- 172.21.15.117:0/2562941309 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fdd500b64d0 0x7fdd500b89e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800ca70 tx=0x7fdd58007040 comp rx=0 tx=0).stop 2024-09-21T05:48:49.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:49.589+0000 7fdd6c942640 1 -- 172.21.15.117:0/2562941309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd6810de90 msgr2=0x7fdd6819fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:49.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:49.589+0000 7fdd6c942640 1 --2- 172.21.15.117:0/2562941309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd6810de90 0x7fdd6819fc70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdd6000eaa0 tx=0x7fdd6000ef60 comp rx=0 tx=0).stop 2024-09-21T05:48:49.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:49.589+0000 7fdd6c942640 1 -- 172.21.15.117:0/2562941309 shutdown_connections 2024-09-21T05:48:49.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:49.589+0000 7fdd6c942640 1 --2- 172.21.15.117:0/2562941309 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fdd500b64d0 0x7fdd500b89e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:49.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:49.589+0000 7fdd6c942640 1 --2- 172.21.15.117:0/2562941309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd6810de90 0x7fdd6819fc70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:49.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:49.589+0000 7fdd6c942640 1 --2- 172.21.15.117:0/2562941309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd68105380 0x7fdd6819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:49.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:49.589+0000 7fdd6c942640 1 --2- 172.21.15.117:0/2562941309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd68104a30 0x7fdd6819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:49.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:49.589+0000 7fdd6c942640 1 -- 172.21.15.117:0/2562941309 >> 172.21.15.117:0/2562941309 conn(0x7fdd68079770 msgr2=0x7fdd68100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:49.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:49.589+0000 7fdd6c942640 1 -- 172.21.15.117:0/2562941309 shutdown_connections 2024-09-21T05:48:49.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:49.589+0000 7fdd6c942640 1 -- 172.21.15.117:0/2562941309 wait complete. 2024-09-21T05:48:50.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.041+0000 7fcb9ca0a640 1 -- 172.21.15.117:0/1081343728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb981094e0 msgr2=0x7fcb981098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:50.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.041+0000 7fcb9ca0a640 1 --2- 172.21.15.117:0/1081343728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb981094e0 0x7fcb981098f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009900 tx=0x7fcb8002a410 comp rx=0 tx=0).stop 2024-09-21T05:48:50.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.041+0000 7fcb9ca0a640 1 -- 172.21.15.117:0/1081343728 shutdown_connections 2024-09-21T05:48:50.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.041+0000 7fcb9ca0a640 1 --2- 172.21.15.117:0/1081343728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb981094e0 0x7fcb981098f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:50.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.041+0000 7fcb9ca0a640 1 --2- 172.21.15.117:0/1081343728 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb98109e30 0x7fcb98110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:50.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.041+0000 7fcb9ca0a640 1 --2- 172.21.15.117:0/1081343728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb98103ba0 0x7fcb98103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:50.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.041+0000 7fcb9ca0a640 1 -- 172.21.15.117:0/1081343728 >> 172.21.15.117:0/1081343728 conn(0x7fcb980ff630 msgr2=0x7fcb98101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:50.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.041+0000 7fcb9ca0a640 1 -- 172.21.15.117:0/1081343728 shutdown_connections 2024-09-21T05:48:50.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.041+0000 7fcb9ca0a640 1 -- 172.21.15.117:0/1081343728 wait complete. 2024-09-21T05:48:50.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.042+0000 7fcb9ca0a640 1 Processor -- start 2024-09-21T05:48:50.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.042+0000 7fcb9ca0a640 1 -- start start 2024-09-21T05:48:50.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.042+0000 7fcb9ca0a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb98103ba0 0x7fcb9819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:50.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.042+0000 7fcb9ca0a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb981094e0 0x7fcb9819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:50.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.042+0000 7fcb9ca0a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb98109e30 0x7fcb9819fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:50.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.042+0000 7fcb9ca0a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcb98113b10 con 0x7fcb98109e30 2024-09-21T05:48:50.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.042+0000 7fcb9ca0a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcb98113990 con 0x7fcb98103ba0 2024-09-21T05:48:50.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.042+0000 7fcb9ca0a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcb98113c90 con 0x7fcb981094e0 2024-09-21T05:48:50.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.042+0000 7fcb96d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb98109e30 0x7fcb9819fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:50.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.042+0000 7fcb96d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb98109e30 0x7fcb9819fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49548/0 (socket says 172.21.15.117:49548) 2024-09-21T05:48:50.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.042+0000 7fcb96d9d640 1 -- 172.21.15.117:0/3195042779 learned_addr learned my addr 172.21.15.117:0/3195042779 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:48:50.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.042+0000 7fcb9659c640 1 --2- 172.21.15.117:0/3195042779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb98103ba0 0x7fcb9819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:50.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.042+0000 7fcb95d9b640 1 --2- 172.21.15.117:0/3195042779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb981094e0 0x7fcb9819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:50.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.043+0000 7fcb96d9d640 1 -- 172.21.15.117:0/3195042779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb981094e0 msgr2=0x7fcb9819aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:50.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.043+0000 7fcb96d9d640 1 --2- 172.21.15.117:0/3195042779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb981094e0 0x7fcb9819aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:50.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.043+0000 7fcb96d9d640 1 -- 172.21.15.117:0/3195042779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb98103ba0 msgr2=0x7fcb9819a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:50.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.043+0000 7fcb96d9d640 1 --2- 172.21.15.117:0/3195042779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb98103ba0 0x7fcb9819a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:50.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.043+0000 7fcb96d9d640 1 -- 172.21.15.117:0/3195042779 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb981a04b0 con 0x7fcb98109e30 2024-09-21T05:48:50.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.043+0000 7fcb96d9d640 1 --2- 172.21.15.117:0/3195042779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb98109e30 0x7fcb9819fd80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcb90009fd0 tx=0x7fcb9000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:50.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.043+0000 7fcb8f7fe640 1 -- 172.21.15.117:0/3195042779 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb90011830 con 0x7fcb98109e30 2024-09-21T05:48:50.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.043+0000 7fcb8f7fe640 1 -- 172.21.15.117:0/3195042779 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb90016070 con 0x7fcb98109e30 2024-09-21T05:48:50.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.043+0000 7fcb8f7fe640 1 -- 172.21.15.117:0/3195042779 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb90010de0 con 0x7fcb98109e30 2024-09-21T05:48:50.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.043+0000 7fcb9ca0a640 1 -- 172.21.15.117:0/3195042779 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb981a07a0 con 0x7fcb98109e30 2024-09-21T05:48:50.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.043+0000 7fcb9ca0a640 1 -- 172.21.15.117:0/3195042779 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb981a6f00 con 0x7fcb98109e30 2024-09-21T05:48:50.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.044+0000 7fcb9ca0a640 1 -- 172.21.15.117:0/3195042779 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb98104e20 con 0x7fcb98109e30 2024-09-21T05:48:50.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.047+0000 7fcb8f7fe640 1 -- 172.21.15.117:0/3195042779 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7fcb900119d0 con 0x7fcb98109e30 2024-09-21T05:48:50.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.049+0000 7fcb8f7fe640 1 --2- 172.21.15.117:0/3195042779 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fcb840b6550 0x7fcb840b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:50.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.049+0000 7fcb8f7fe640 1 -- 172.21.15.117:0/3195042779 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fcb900e5230 con 0x7fcb98109e30 2024-09-21T05:48:50.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.049+0000 7fcb9659c640 1 --2- 172.21.15.117:0/3195042779 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fcb840b6550 0x7fcb840b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:50.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.050+0000 7fcb9659c640 1 --2- 172.21.15.117:0/3195042779 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fcb840b6550 0x7fcb840b8a60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcb88005fd0 tx=0x7fcb88009680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:50.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.052+0000 7fcb8f7fe640 1 -- 172.21.15.117:0/3195042779 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcb900ad6c0 con 0x7fcb98109e30 2024-09-21T05:48:50.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.239+0000 7fcb9ca0a640 1 -- 172.21.15.117:0/3195042779 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7fcb98103fb0 con 0x7fcb98109e30 2024-09-21T05:48:50.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.245+0000 7fcb8f7fe640 1 -- 172.21.15.117:0/3195042779 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v23)=0 v23) ==== 119+0+0 (secure 0 0 0) 0x7fcb98103fb0 con 0x7fcb98109e30 2024-09-21T05:48:50.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.247+0000 7fcb9ca0a640 1 -- 172.21.15.117:0/3195042779 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fcb840b6550 msgr2=0x7fcb840b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:50.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.247+0000 7fcb9ca0a640 1 --2- 172.21.15.117:0/3195042779 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fcb840b6550 0x7fcb840b8a60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcb88005fd0 tx=0x7fcb88009680 comp rx=0 tx=0).stop 2024-09-21T05:48:50.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.247+0000 7fcb9ca0a640 1 -- 172.21.15.117:0/3195042779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb98109e30 msgr2=0x7fcb9819fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:50.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.247+0000 7fcb9ca0a640 1 --2- 172.21.15.117:0/3195042779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb98109e30 0x7fcb9819fd80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcb90009fd0 tx=0x7fcb9000b8e0 comp rx=0 tx=0).stop 2024-09-21T05:48:50.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.247+0000 7fcb9ca0a640 1 -- 172.21.15.117:0/3195042779 shutdown_connections 2024-09-21T05:48:50.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.247+0000 7fcb9ca0a640 1 --2- 172.21.15.117:0/3195042779 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fcb840b6550 0x7fcb840b8a60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:50.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.247+0000 7fcb9ca0a640 1 --2- 172.21.15.117:0/3195042779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb98109e30 0x7fcb9819fd80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:50.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.247+0000 7fcb9ca0a640 1 --2- 172.21.15.117:0/3195042779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb981094e0 0x7fcb9819aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:50.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.247+0000 7fcb9ca0a640 1 --2- 172.21.15.117:0/3195042779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb98103ba0 0x7fcb9819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:50.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.247+0000 7fcb9ca0a640 1 -- 172.21.15.117:0/3195042779 >> 172.21.15.117:0/3195042779 conn(0x7fcb980ff630 msgr2=0x7fcb98101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:50.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.248+0000 7fcb9ca0a640 1 -- 172.21.15.117:0/3195042779 shutdown_connections 2024-09-21T05:48:50.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.248+0000 7fcb9ca0a640 1 -- 172.21.15.117:0/3195042779 wait complete. 2024-09-21T05:48:50.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:50 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2735707420' entity='client.admin' 2024-09-21T05:48:50.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:50 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2562941309' entity='client.admin' 2024-09-21T05:48:50.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:50 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2735707420' entity='client.admin' 2024-09-21T05:48:50.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:50 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2562941309' entity='client.admin' 2024-09-21T05:48:50.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:50 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2735707420' entity='client.admin' 2024-09-21T05:48:50.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:50 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2562941309' entity='client.admin' 2024-09-21T05:48:50.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.763+0000 7fcf58617640 1 -- 172.21.15.117:0/3407085583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf501083a0 msgr2=0x7fcf501087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:50.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.763+0000 7fcf58617640 1 --2- 172.21.15.117:0/3407085583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf501083a0 0x7fcf501087b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009900 tx=0x7fcf4002a410 comp rx=0 tx=0).stop 2024-09-21T05:48:50.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.764+0000 7fcf58617640 1 -- 172.21.15.117:0/3407085583 shutdown_connections 2024-09-21T05:48:50.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.764+0000 7fcf58617640 1 --2- 172.21.15.117:0/3407085583 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf501083a0 0x7fcf501087b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:50.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.764+0000 7fcf58617640 1 --2- 172.21.15.117:0/3407085583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf50108cf0 0x7fcf5010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:50.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.764+0000 7fcf58617640 1 --2- 172.21.15.117:0/3407085583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf50102a60 0x7fcf50102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:50.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.764+0000 7fcf58617640 1 -- 172.21.15.117:0/3407085583 >> 172.21.15.117:0/3407085583 conn(0x7fcf50079730 msgr2=0x7fcf50079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:50.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.764+0000 7fcf58617640 1 -- 172.21.15.117:0/3407085583 shutdown_connections 2024-09-21T05:48:50.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.764+0000 7fcf58617640 1 -- 172.21.15.117:0/3407085583 wait complete. 2024-09-21T05:48:50.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.764+0000 7fcf58617640 1 Processor -- start 2024-09-21T05:48:50.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.764+0000 7fcf58617640 1 -- start start 2024-09-21T05:48:50.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.765+0000 7fcf58617640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf50102a60 0x7fcf500ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:50.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.765+0000 7fcf58617640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf501083a0 0x7fcf500ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:50.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.765+0000 7fcf58617640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf50108cf0 0x7fcf501000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:50.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.765+0000 7fcf58617640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcf50112960 con 0x7fcf50102a60 2024-09-21T05:48:50.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.765+0000 7fcf58617640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcf501127e0 con 0x7fcf50108cf0 2024-09-21T05:48:50.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.765+0000 7fcf58617640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcf50112ae0 con 0x7fcf501083a0 2024-09-21T05:48:50.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.765+0000 7fcf563b3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf50102a60 0x7fcf500ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:50.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.765+0000 7fcf563b3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf50102a60 0x7fcf500ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49596/0 (socket says 172.21.15.117:49596) 2024-09-21T05:48:50.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.765+0000 7fcf563b3640 1 -- 172.21.15.117:0/3053617935 learned_addr learned my addr 172.21.15.117:0/3053617935 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:48:50.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.765+0000 7fcf55bb2640 1 --2- 172.21.15.117:0/3053617935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf501083a0 0x7fcf500ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:50.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.765+0000 7fcf56bb4640 1 --2- 172.21.15.117:0/3053617935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf50108cf0 0x7fcf501000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:50.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.765+0000 7fcf563b3640 1 -- 172.21.15.117:0/3053617935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf501083a0 msgr2=0x7fcf500ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:50.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.765+0000 7fcf563b3640 1 --2- 172.21.15.117:0/3053617935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf501083a0 0x7fcf500ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:50.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.765+0000 7fcf563b3640 1 -- 172.21.15.117:0/3053617935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf50108cf0 msgr2=0x7fcf501000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:50.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.765+0000 7fcf563b3640 1 --2- 172.21.15.117:0/3053617935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf50108cf0 0x7fcf501000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:50.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.765+0000 7fcf563b3640 1 -- 172.21.15.117:0/3053617935 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf501a8760 con 0x7fcf50102a60 2024-09-21T05:48:50.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.765+0000 7fcf563b3640 1 --2- 172.21.15.117:0/3053617935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf50102a60 0x7fcf500ff670 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00d770 tx=0x7fcf4c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:50.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.765+0000 7fcf477fe640 1 -- 172.21.15.117:0/3053617935 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c017070 con 0x7fcf50102a60 2024-09-21T05:48:50.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.766+0000 7fcf477fe640 1 -- 172.21.15.117:0/3053617935 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf4c0054d0 con 0x7fcf50102a60 2024-09-21T05:48:50.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.766+0000 7fcf58617640 1 -- 172.21.15.117:0/3053617935 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf501a8a50 con 0x7fcf50102a60 2024-09-21T05:48:50.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.766+0000 7fcf477fe640 1 -- 172.21.15.117:0/3053617935 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c004c30 con 0x7fcf50102a60 2024-09-21T05:48:50.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.766+0000 7fcf58617640 1 -- 172.21.15.117:0/3053617935 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf501a8f10 con 0x7fcf50102a60 2024-09-21T05:48:50.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.767+0000 7fcf58617640 1 -- 172.21.15.117:0/3053617935 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf50103ce0 con 0x7fcf50102a60 2024-09-21T05:48:50.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.768+0000 7fcf477fe640 1 -- 172.21.15.117:0/3053617935 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7fcf4c004e10 con 0x7fcf50102a60 2024-09-21T05:48:50.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.769+0000 7fcf477fe640 1 --2- 172.21.15.117:0/3053617935 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fcf3c0b64d0 0x7fcf3c0b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:50.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.769+0000 7fcf55bb2640 1 --2- 172.21.15.117:0/3053617935 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fcf3c0b64d0 0x7fcf3c0b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:50.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.769+0000 7fcf477fe640 1 -- 172.21.15.117:0/3053617935 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fcf4c0e5240 con 0x7fcf50102a60 2024-09-21T05:48:50.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.769+0000 7fcf55bb2640 1 --2- 172.21.15.117:0/3053617935 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fcf3c0b64d0 0x7fcf3c0b89e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcf40000c00 tx=0x7fcf40031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:50.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.773+0000 7fcf477fe640 1 -- 172.21.15.117:0/3053617935 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcf4c0ad6d0 con 0x7fcf50102a60 2024-09-21T05:48:50.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.965+0000 7fcf58617640 1 -- 172.21.15.117:0/3053617935 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7fcf5002e640 con 0x7fcf50102a60 2024-09-21T05:48:50.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.971+0000 7fcf477fe640 1 -- 172.21.15.117:0/3053617935 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v25)=0 v25) ==== 129+0+0 (secure 0 0 0) 0x7fcf4c0b1ab0 con 0x7fcf50102a60 2024-09-21T05:48:50.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.973+0000 7fcf58617640 1 -- 172.21.15.117:0/3053617935 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fcf3c0b64d0 msgr2=0x7fcf3c0b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:50.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.973+0000 7fcf58617640 1 --2- 172.21.15.117:0/3053617935 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fcf3c0b64d0 0x7fcf3c0b89e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcf40000c00 tx=0x7fcf40031400 comp rx=0 tx=0).stop 2024-09-21T05:48:50.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.973+0000 7fcf58617640 1 -- 172.21.15.117:0/3053617935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf50102a60 msgr2=0x7fcf500ff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:50.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.973+0000 7fcf58617640 1 --2- 172.21.15.117:0/3053617935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf50102a60 0x7fcf500ff670 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00d770 tx=0x7fcf4c00dc30 comp rx=0 tx=0).stop 2024-09-21T05:48:50.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.974+0000 7fcf58617640 1 -- 172.21.15.117:0/3053617935 shutdown_connections 2024-09-21T05:48:50.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.974+0000 7fcf58617640 1 --2- 172.21.15.117:0/3053617935 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fcf3c0b64d0 0x7fcf3c0b89e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:50.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.974+0000 7fcf58617640 1 --2- 172.21.15.117:0/3053617935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf50108cf0 0x7fcf501000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:50.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.974+0000 7fcf58617640 1 --2- 172.21.15.117:0/3053617935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf501083a0 0x7fcf500ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:50.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.974+0000 7fcf58617640 1 --2- 172.21.15.117:0/3053617935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf50102a60 0x7fcf500ff670 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:50.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.974+0000 7fcf58617640 1 -- 172.21.15.117:0/3053617935 >> 172.21.15.117:0/3053617935 conn(0x7fcf50079730 msgr2=0x7fcf5006b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:50.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.974+0000 7fcf58617640 1 -- 172.21.15.117:0/3053617935 shutdown_connections 2024-09-21T05:48:50.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.974+0000 7fcf58617640 1 -- 172.21.15.117:0/3053617935 wait complete. 2024-09-21T05:48:51.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:51 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3195042779' entity='client.admin' 2024-09-21T05:48:51.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:51 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2596730750' entity='client.admin' 2024-09-21T05:48:51.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:51 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3053617935' entity='client.admin' 2024-09-21T05:48:51.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:51 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3195042779' entity='client.admin' 2024-09-21T05:48:51.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:51 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2596730750' entity='client.admin' 2024-09-21T05:48:51.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:51 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3053617935' entity='client.admin' 2024-09-21T05:48:51.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:51 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3195042779' entity='client.admin' 2024-09-21T05:48:51.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:51 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2596730750' entity='client.admin' 2024-09-21T05:48:51.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:51 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3053617935' entity='client.admin' 2024-09-21T05:48:51.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.752+0000 7f4b7de73640 1 -- 172.21.15.117:0/3019029732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b78102a90 msgr2=0x7f4b78102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:51.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.752+0000 7f4b7de73640 1 --2- 172.21.15.117:0/3019029732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b78102a90 0x7f4b78102ea0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4b68009ca0 tx=0x7f4b6802a8c0 comp rx=0 tx=0).stop 2024-09-21T05:48:51.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.752+0000 7f4b7de73640 1 -- 172.21.15.117:0/3019029732 shutdown_connections 2024-09-21T05:48:51.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.752+0000 7f4b7de73640 1 --2- 172.21.15.117:0/3019029732 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b78108d20 0x7f4b7810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:51.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.752+0000 7f4b7de73640 1 --2- 172.21.15.117:0/3019029732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b781083d0 0x7f4b781087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:51.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.752+0000 7f4b7de73640 1 --2- 172.21.15.117:0/3019029732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b78102a90 0x7f4b78102ea0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:51.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.752+0000 7f4b7de73640 1 -- 172.21.15.117:0/3019029732 >> 172.21.15.117:0/3019029732 conn(0x7f4b78079750 msgr2=0x7f4b78079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:51.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.752+0000 7f4b7de73640 1 -- 172.21.15.117:0/3019029732 shutdown_connections 2024-09-21T05:48:51.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.752+0000 7f4b7de73640 1 -- 172.21.15.117:0/3019029732 wait complete. 2024-09-21T05:48:51.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.753+0000 7f4b7de73640 1 Processor -- start 2024-09-21T05:48:51.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.753+0000 7f4b7de73640 1 -- start start 2024-09-21T05:48:51.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.754+0000 7f4b7de73640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b78102a90 0x7f4b78199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:51.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.754+0000 7f4b7de73640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b781083d0 0x7f4b78199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:51.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.754+0000 7f4b7de73640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b78108d20 0x7f4b7819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:51.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.754+0000 7f4b7de73640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4b781129e0 con 0x7f4b78108d20 2024-09-21T05:48:51.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.754+0000 7f4b7de73640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4b78112860 con 0x7f4b781083d0 2024-09-21T05:48:51.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.754+0000 7f4b7de73640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4b78112b60 con 0x7f4b78102a90 2024-09-21T05:48:51.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.754+0000 7f4b77fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b78108d20 0x7f4b7819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:51.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.754+0000 7f4b77fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b78108d20 0x7f4b7819ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49620/0 (socket says 172.21.15.117:49620) 2024-09-21T05:48:51.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.754+0000 7f4b77fff640 1 -- 172.21.15.117:0/2356710509 learned_addr learned my addr 172.21.15.117:0/2356710509 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:48:51.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.754+0000 7f4b6ffff640 1 --2- 172.21.15.117:0/2356710509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b781083d0 0x7f4b78199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:51.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.754+0000 7f4b777fe640 1 --2- 172.21.15.117:0/2356710509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b78102a90 0x7f4b78199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:51.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.754+0000 7f4b77fff640 1 -- 172.21.15.117:0/2356710509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b78102a90 msgr2=0x7f4b78199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:51.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.754+0000 7f4b77fff640 1 --2- 172.21.15.117:0/2356710509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b78102a90 0x7f4b78199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:51.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.754+0000 7f4b77fff640 1 -- 172.21.15.117:0/2356710509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b781083d0 msgr2=0x7f4b78199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:51.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.754+0000 7f4b77fff640 1 --2- 172.21.15.117:0/2356710509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b781083d0 0x7f4b78199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:51.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.754+0000 7f4b77fff640 1 -- 172.21.15.117:0/2356710509 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b7819f220 con 0x7f4b78108d20 2024-09-21T05:48:51.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.754+0000 7f4b77fff640 1 --2- 172.21.15.117:0/2356710509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b78108d20 0x7f4b7819ebb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4b70009860 tx=0x7f4b70009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:51.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.754+0000 7f4b757fa640 1 -- 172.21.15.117:0/2356710509 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b70005020 con 0x7f4b78108d20 2024-09-21T05:48:51.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.755+0000 7f4b757fa640 1 -- 172.21.15.117:0/2356710509 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b700047a0 con 0x7f4b78108d20 2024-09-21T05:48:51.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.755+0000 7f4b757fa640 1 -- 172.21.15.117:0/2356710509 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b70007a10 con 0x7f4b78108d20 2024-09-21T05:48:51.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.755+0000 7f4b7de73640 1 -- 172.21.15.117:0/2356710509 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b7819f510 con 0x7f4b78108d20 2024-09-21T05:48:51.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.755+0000 7f4b7de73640 1 -- 172.21.15.117:0/2356710509 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b7806bb80 con 0x7f4b78108d20 2024-09-21T05:48:51.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.756+0000 7f4b7de73640 1 -- 172.21.15.117:0/2356710509 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b78103d10 con 0x7f4b78108d20 2024-09-21T05:48:51.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.757+0000 7f4b757fa640 1 -- 172.21.15.117:0/2356710509 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7f4b70004950 con 0x7f4b78108d20 2024-09-21T05:48:51.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.758+0000 7f4b757fa640 1 --2- 172.21.15.117:0/2356710509 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4b640b6580 0x7f4b640b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:51.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.758+0000 7f4b777fe640 1 --2- 172.21.15.117:0/2356710509 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4b640b6580 0x7f4b640b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:51.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.758+0000 7f4b757fa640 1 -- 172.21.15.117:0/2356710509 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f4b700e7180 con 0x7f4b78108d20 2024-09-21T05:48:51.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.758+0000 7f4b777fe640 1 --2- 172.21.15.117:0/2356710509 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4b640b6580 0x7f4b640b8a90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4b68006010 tx=0x7f4b68009820 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:51.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.761+0000 7f4b757fa640 1 -- 172.21.15.117:0/2356710509 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4b700af610 con 0x7f4b78108d20 2024-09-21T05:48:51.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.950+0000 7f4b7de73640 1 -- 172.21.15.117:0/2356710509 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f4b78102ea0 con 0x7f4b78108d20 2024-09-21T05:48:51.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.956+0000 7f4b757fa640 1 -- 172.21.15.117:0/2356710509 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v26)=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7f4b700b39f0 con 0x7f4b78108d20 2024-09-21T05:48:51.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.959+0000 7f4b7de73640 1 -- 172.21.15.117:0/2356710509 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4b640b6580 msgr2=0x7f4b640b8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:51.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.959+0000 7f4b7de73640 1 --2- 172.21.15.117:0/2356710509 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4b640b6580 0x7f4b640b8a90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4b68006010 tx=0x7f4b68009820 comp rx=0 tx=0).stop 2024-09-21T05:48:51.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.959+0000 7f4b7de73640 1 -- 172.21.15.117:0/2356710509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b78108d20 msgr2=0x7f4b7819ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:51.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.959+0000 7f4b7de73640 1 --2- 172.21.15.117:0/2356710509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b78108d20 0x7f4b7819ebb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4b70009860 tx=0x7f4b70009c60 comp rx=0 tx=0).stop 2024-09-21T05:48:51.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.959+0000 7f4b7de73640 1 -- 172.21.15.117:0/2356710509 shutdown_connections 2024-09-21T05:48:51.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.959+0000 7f4b7de73640 1 --2- 172.21.15.117:0/2356710509 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4b640b6580 0x7f4b640b8a90 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:51.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.959+0000 7f4b7de73640 1 --2- 172.21.15.117:0/2356710509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b78108d20 0x7f4b7819ebb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:51.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.959+0000 7f4b7de73640 1 --2- 172.21.15.117:0/2356710509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b781083d0 0x7f4b78199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:51.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.959+0000 7f4b7de73640 1 --2- 172.21.15.117:0/2356710509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b78102a90 0x7f4b78199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:51.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.959+0000 7f4b7de73640 1 -- 172.21.15.117:0/2356710509 >> 172.21.15.117:0/2356710509 conn(0x7f4b78079750 msgr2=0x7f4b78101310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:51.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.959+0000 7f4b7de73640 1 -- 172.21.15.117:0/2356710509 shutdown_connections 2024-09-21T05:48:51.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:51.959+0000 7f4b7de73640 1 -- 172.21.15.117:0/2356710509 wait complete. 2024-09-21T05:48:52.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.427+0000 7f3209c98640 1 -- 172.21.15.117:0/1158883599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3204108420 msgr2=0x7f3204108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:52.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.427+0000 7f3209c98640 1 --2- 172.21.15.117:0/1158883599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3204108420 0x7f3204108830 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f31f8009900 tx=0x7f31f802a410 comp rx=0 tx=0).stop 2024-09-21T05:48:52.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.427+0000 7f3209c98640 1 -- 172.21.15.117:0/1158883599 shutdown_connections 2024-09-21T05:48:52.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.427+0000 7f3209c98640 1 --2- 172.21.15.117:0/1158883599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3204108420 0x7f3204108830 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:52.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.427+0000 7f3209c98640 1 --2- 172.21.15.117:0/1158883599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3204108d70 0x7f320410f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:52.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.427+0000 7f3209c98640 1 --2- 172.21.15.117:0/1158883599 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3204102ae0 0x7f3204102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:52.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.427+0000 7f3209c98640 1 -- 172.21.15.117:0/1158883599 >> 172.21.15.117:0/1158883599 conn(0x7f3204079770 msgr2=0x7f3204079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:52.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.427+0000 7f3209c98640 1 -- 172.21.15.117:0/1158883599 shutdown_connections 2024-09-21T05:48:52.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.427+0000 7f3209c98640 1 -- 172.21.15.117:0/1158883599 wait complete. 2024-09-21T05:48:52.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.428+0000 7f3209c98640 1 Processor -- start 2024-09-21T05:48:52.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.428+0000 7f3209c98640 1 -- start start 2024-09-21T05:48:52.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.428+0000 7f3209c98640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3204102ae0 0x7f320419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:52.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.428+0000 7f3209c98640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3204108420 0x7f320419b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:52.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.428+0000 7f3209c98640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3204108d70 0x7f320419be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:52.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.428+0000 7f3209c98640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3204112a50 con 0x7f3204108d70 2024-09-21T05:48:52.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.428+0000 7f3209c98640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f32041128d0 con 0x7f3204108420 2024-09-21T05:48:52.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.428+0000 7f3209c98640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3204112bd0 con 0x7f3204102ae0 2024-09-21T05:48:52.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.428+0000 7f3203fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3204108d70 0x7f320419be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:52.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.429+0000 7f3203fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3204108d70 0x7f320419be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49632/0 (socket says 172.21.15.117:49632) 2024-09-21T05:48:52.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.429+0000 7f3203fff640 1 -- 172.21.15.117:0/1975035208 learned_addr learned my addr 172.21.15.117:0/1975035208 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:48:52.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.429+0000 7f32037fe640 1 --2- 172.21.15.117:0/1975035208 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3204102ae0 0x7f320419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:52.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.429+0000 7f3202ffd640 1 --2- 172.21.15.117:0/1975035208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3204108420 0x7f320419b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:52.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.429+0000 7f3203fff640 1 -- 172.21.15.117:0/1975035208 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3204102ae0 msgr2=0x7f320419b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:52.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.429+0000 7f3203fff640 1 --2- 172.21.15.117:0/1975035208 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3204102ae0 0x7f320419b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:52.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.429+0000 7f3203fff640 1 -- 172.21.15.117:0/1975035208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3204108420 msgr2=0x7f320419b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:52.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.429+0000 7f3203fff640 1 --2- 172.21.15.117:0/1975035208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3204108420 0x7f320419b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:52.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.429+0000 7f3203fff640 1 -- 172.21.15.117:0/1975035208 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32041a6630 con 0x7f3204108d70 2024-09-21T05:48:52.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.429+0000 7f3202ffd640 1 --2- 172.21.15.117:0/1975035208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3204108420 0x7f320419b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:48:52.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.429+0000 7f3203fff640 1 --2- 172.21.15.117:0/1975035208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3204108d70 0x7f320419be20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f31fc00b970 tx=0x7f31fc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:52.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.429+0000 7f3200ff9640 1 -- 172.21.15.117:0/1975035208 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc019070 con 0x7f3204108d70 2024-09-21T05:48:52.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.429+0000 7f3200ff9640 1 -- 172.21.15.117:0/1975035208 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31fc014440 con 0x7f3204108d70 2024-09-21T05:48:52.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.429+0000 7f3209c98640 1 -- 172.21.15.117:0/1975035208 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32041a6920 con 0x7f3204108d70 2024-09-21T05:48:52.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.429+0000 7f3200ff9640 1 -- 172.21.15.117:0/1975035208 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc0075d0 con 0x7f3204108d70 2024-09-21T05:48:52.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.429+0000 7f3209c98640 1 -- 172.21.15.117:0/1975035208 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32041a6de0 con 0x7f3204108d70 2024-09-21T05:48:52.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.430+0000 7f3209c98640 1 -- 172.21.15.117:0/1975035208 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3204103d60 con 0x7f3204108d70 2024-09-21T05:48:52.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.432+0000 7f3200ff9640 1 -- 172.21.15.117:0/1975035208 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7f31fc01c070 con 0x7f3204108d70 2024-09-21T05:48:52.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.433+0000 7f3200ff9640 1 --2- 172.21.15.117:0/1975035208 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f31ec0b6500 0x7f31ec0b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:52.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.433+0000 7f32037fe640 1 --2- 172.21.15.117:0/1975035208 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f31ec0b6500 0x7f31ec0b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:52.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.433+0000 7f3200ff9640 1 -- 172.21.15.117:0/1975035208 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f31fc0e5050 con 0x7f3204108d70 2024-09-21T05:48:52.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.433+0000 7f32037fe640 1 --2- 172.21.15.117:0/1975035208 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f31ec0b6500 0x7f31ec0b8a10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f31f40058c0 tx=0x7f31f400a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:52.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.436+0000 7f3200ff9640 1 -- 172.21.15.117:0/1975035208 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f31fc0b18c0 con 0x7f3204108d70 2024-09-21T05:48:52.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.619+0000 7f3209c98640 1 -- 172.21.15.117:0/1975035208 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f3204102ef0 con 0x7f3204108d70 2024-09-21T05:48:52.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.623+0000 7f3200ff9640 1 -- 172.21.15.117:0/1975035208 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v27)=0 v27) ==== 121+0+0 (secure 0 0 0) 0x7f3204102ef0 con 0x7f3204108d70 2024-09-21T05:48:52.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.626+0000 7f3209c98640 1 -- 172.21.15.117:0/1975035208 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f31ec0b6500 msgr2=0x7f31ec0b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:52.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.626+0000 7f3209c98640 1 --2- 172.21.15.117:0/1975035208 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f31ec0b6500 0x7f31ec0b8a10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f31f40058c0 tx=0x7f31f400a2d0 comp rx=0 tx=0).stop 2024-09-21T05:48:52.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.626+0000 7f3209c98640 1 -- 172.21.15.117:0/1975035208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3204108d70 msgr2=0x7f320419be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:52.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.626+0000 7f3209c98640 1 --2- 172.21.15.117:0/1975035208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3204108d70 0x7f320419be20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f31fc00b970 tx=0x7f31fc00be30 comp rx=0 tx=0).stop 2024-09-21T05:48:52.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.626+0000 7f3209c98640 1 -- 172.21.15.117:0/1975035208 shutdown_connections 2024-09-21T05:48:52.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.626+0000 7f3209c98640 1 --2- 172.21.15.117:0/1975035208 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f31ec0b6500 0x7f31ec0b8a10 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:52.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.626+0000 7f3209c98640 1 --2- 172.21.15.117:0/1975035208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3204108d70 0x7f320419be20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:52.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.626+0000 7f3209c98640 1 --2- 172.21.15.117:0/1975035208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3204108420 0x7f320419b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:52.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.626+0000 7f3209c98640 1 --2- 172.21.15.117:0/1975035208 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3204102ae0 0x7f320419b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:52.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.626+0000 7f3209c98640 1 -- 172.21.15.117:0/1975035208 >> 172.21.15.117:0/1975035208 conn(0x7f3204079770 msgr2=0x7f32041064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:52.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.626+0000 7f3209c98640 1 -- 172.21.15.117:0/1975035208 shutdown_connections 2024-09-21T05:48:52.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:52.626+0000 7f3209c98640 1 -- 172.21.15.117:0/1975035208 wait complete. 2024-09-21T05:48:52.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.654+0000 7fefe06f3640 1 -- 172.21.15.117:0/4283739528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefd8103b60 msgr2=0x7fefd8103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:52.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.654+0000 7fefe06f3640 1 --2- 172.21.15.117:0/4283739528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefd8103b60 0x7fefd8103f70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fefd4009900 tx=0x7fefd402a410 comp rx=0 tx=0).stop 2024-09-21T05:48:52.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.655+0000 7fefe06f3640 1 -- 172.21.15.117:0/4283739528 shutdown_connections 2024-09-21T05:48:52.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.655+0000 7fefe06f3640 1 --2- 172.21.15.117:0/4283739528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefd8109df0 0x7fefd8110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:52.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.655+0000 7fefe06f3640 1 --2- 172.21.15.117:0/4283739528 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefd81094a0 0x7fefd81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:52.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.655+0000 7fefe06f3640 1 --2- 172.21.15.117:0/4283739528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefd8103b60 0x7fefd8103f70 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:52.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.655+0000 7fefe06f3640 1 -- 172.21.15.117:0/4283739528 >> 172.21.15.117:0/4283739528 conn(0x7fefd807b850 msgr2=0x7fefd807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:52.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.655+0000 7fefe06f3640 1 -- 172.21.15.117:0/4283739528 shutdown_connections 2024-09-21T05:48:52.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.655+0000 7fefe06f3640 1 -- 172.21.15.117:0/4283739528 wait complete. 2024-09-21T05:48:52.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.655+0000 7fefe06f3640 1 Processor -- start 2024-09-21T05:48:52.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.655+0000 7fefe06f3640 1 -- start start 2024-09-21T05:48:52.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.656+0000 7fefe06f3640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefd8103b60 0x7fefd819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:52.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.656+0000 7fefe06f3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefd81094a0 0x7fefd819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:52.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.656+0000 7fefe06f3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefd8109df0 0x7fefd819fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:52.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.656+0000 7fefe06f3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fefd8113ab0 con 0x7fefd81094a0 2024-09-21T05:48:52.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.656+0000 7fefe06f3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fefd8113930 con 0x7fefd8109df0 2024-09-21T05:48:52.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.656+0000 7fefe06f3640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fefd8113c30 con 0x7fefd8103b60 2024-09-21T05:48:52.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.656+0000 7fefddc8e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefd81094a0 0x7fefd819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:52.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.656+0000 7fefddc8e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefd81094a0 0x7fefd819aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49574/0 (socket says 172.21.15.117:49574) 2024-09-21T05:48:52.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.656+0000 7fefddc8e640 1 -- 172.21.15.117:0/2596730750 learned_addr learned my addr 172.21.15.117:0/2596730750 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:48:52.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.656+0000 7fefde48f640 1 --2- 172.21.15.117:0/2596730750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefd8103b60 0x7fefd819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:52.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.656+0000 7fefdec90640 1 --2- 172.21.15.117:0/2596730750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefd8109df0 0x7fefd819fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:52.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.656+0000 7fefddc8e640 1 -- 172.21.15.117:0/2596730750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefd8103b60 msgr2=0x7fefd819a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:52.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.656+0000 7fefddc8e640 1 --2- 172.21.15.117:0/2596730750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefd8103b60 0x7fefd819a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:52.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.656+0000 7fefddc8e640 1 -- 172.21.15.117:0/2596730750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefd8109df0 msgr2=0x7fefd819fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:52.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.656+0000 7fefddc8e640 1 --2- 172.21.15.117:0/2596730750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefd8109df0 0x7fefd819fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:52.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.656+0000 7fefddc8e640 1 -- 172.21.15.117:0/2596730750 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd81a0490 con 0x7fefd81094a0 2024-09-21T05:48:52.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.656+0000 7fefddc8e640 1 --2- 172.21.15.117:0/2596730750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefd81094a0 0x7fefd819aac0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fefc800d750 tx=0x7fefc800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:52.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.656+0000 7fefcf7fe640 1 -- 172.21.15.117:0/2596730750 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc8017070 con 0x7fefd81094a0 2024-09-21T05:48:52.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.656+0000 7fefcf7fe640 1 -- 172.21.15.117:0/2596730750 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefc80054d0 con 0x7fefd81094a0 2024-09-21T05:48:52.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.656+0000 7fefcf7fe640 1 -- 172.21.15.117:0/2596730750 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc8004c30 con 0x7fefd81094a0 2024-09-21T05:48:52.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.656+0000 7fefe06f3640 1 -- 172.21.15.117:0/2596730750 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefd81a0780 con 0x7fefd81094a0 2024-09-21T05:48:52.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.656+0000 7fefe06f3640 1 -- 172.21.15.117:0/2596730750 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefd81a6ee0 con 0x7fefd81094a0 2024-09-21T05:48:52.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.657+0000 7fefe06f3640 1 -- 172.21.15.117:0/2596730750 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefd8103f70 con 0x7fefd81094a0 2024-09-21T05:48:52.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.660+0000 7fefcf7fe640 1 -- 172.21.15.117:0/2596730750 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7fefc8004dd0 con 0x7fefd81094a0 2024-09-21T05:48:52.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.661+0000 7fefcf7fe640 1 --2- 172.21.15.117:0/2596730750 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fefc40b6550 0x7fefc40b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:52.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.661+0000 7fefde48f640 1 --2- 172.21.15.117:0/2596730750 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fefc40b6550 0x7fefc40b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:52.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.661+0000 7fefcf7fe640 1 -- 172.21.15.117:0/2596730750 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fefc80e5480 con 0x7fefd81094a0 2024-09-21T05:48:52.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.661+0000 7fefde48f640 1 --2- 172.21.15.117:0/2596730750 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fefc40b6550 0x7fefc40b8a60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fefd4031440 tx=0x7fefd4009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:52.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.664+0000 7fefcf7fe640 1 -- 172.21.15.117:0/2596730750 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fefc80ad990 con 0x7fefd81094a0 2024-09-21T05:48:52.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.855+0000 7fefe06f3640 1 -- 172.21.15.117:0/2596730750 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7fefd819b650 con 0x7fefd81094a0 2024-09-21T05:48:52.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.860+0000 7fefcf7fe640 1 -- 172.21.15.117:0/2596730750 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v24)=0 v24) ==== 116+0+0 (secure 0 0 0) 0x7fefd819b650 con 0x7fefd81094a0 2024-09-21T05:48:52.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.863+0000 7fefe06f3640 1 -- 172.21.15.117:0/2596730750 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fefc40b6550 msgr2=0x7fefc40b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:52.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.863+0000 7fefe06f3640 1 --2- 172.21.15.117:0/2596730750 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fefc40b6550 0x7fefc40b8a60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fefd4031440 tx=0x7fefd4009560 comp rx=0 tx=0).stop 2024-09-21T05:48:52.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.863+0000 7fefe06f3640 1 -- 172.21.15.117:0/2596730750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefd81094a0 msgr2=0x7fefd819aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:52.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.863+0000 7fefe06f3640 1 --2- 172.21.15.117:0/2596730750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefd81094a0 0x7fefd819aac0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fefc800d750 tx=0x7fefc800dc10 comp rx=0 tx=0).stop 2024-09-21T05:48:52.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.863+0000 7fefe06f3640 1 -- 172.21.15.117:0/2596730750 shutdown_connections 2024-09-21T05:48:52.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.863+0000 7fefe06f3640 1 --2- 172.21.15.117:0/2596730750 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fefc40b6550 0x7fefc40b8a60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:52.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.863+0000 7fefe06f3640 1 --2- 172.21.15.117:0/2596730750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefd81094a0 0x7fefd819aac0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:52.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.863+0000 7fefe06f3640 1 --2- 172.21.15.117:0/2596730750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefd8109df0 0x7fefd819fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:52.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.863+0000 7fefe06f3640 1 --2- 172.21.15.117:0/2596730750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefd8103b60 0x7fefd819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:52.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.863+0000 7fefe06f3640 1 -- 172.21.15.117:0/2596730750 >> 172.21.15.117:0/2596730750 conn(0x7fefd807b850 msgr2=0x7fefd81075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:52.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.863+0000 7fefe06f3640 1 -- 172.21.15.117:0/2596730750 shutdown_connections 2024-09-21T05:48:52.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:50.863+0000 7fefe06f3640 1 -- 172.21.15.117:0/2596730750 wait complete. 2024-09-21T05:48:53.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:52 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2356710509' entity='client.admin' 2024-09-21T05:48:53.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:52 smithi192 ceph-mon[34700]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:53.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:52 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/1975035208' entity='client.admin' 2024-09-21T05:48:53.249 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:52 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2356710509' entity='client.admin' 2024-09-21T05:48:53.249 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:52 smithi117 ceph-mon[30947]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:53.249 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:52 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/1975035208' entity='client.admin' 2024-09-21T05:48:53.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:52 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2356710509' entity='client.admin' 2024-09-21T05:48:53.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:52 smithi138 ceph-mon[34391]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:53.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:52 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/1975035208' entity='client.admin' 2024-09-21T05:48:54.412 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ac8581a2-77dc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi117') 2024-09-21T05:48:54.412 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ac8581a2-77dc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi117') 2024-09-21T05:48:54.412 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ac8581a2-77dc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi117') 2024-09-21T05:48:54.412 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ac8581a2-77dc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi117') 2024-09-21T05:48:54.422 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:48:54.422 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:48:55.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:55 smithi192 ceph-mon[34700]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:55.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:55 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2223068127' entity='client.admin' 2024-09-21T05:48:55.762 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:55 smithi117 ceph-mon[30947]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:55.762 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:55 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2223068127' entity='client.admin' 2024-09-21T05:48:55.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:55 smithi138 ceph-mon[34391]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:55.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:55 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2223068127' entity='client.admin' 2024-09-21T05:48:56.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.892+0000 7f6cc1666640 1 -- 172.21.15.117:0/2223026512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cbc105380 msgr2=0x7f6cbc10d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:56.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.892+0000 7f6cc1666640 1 --2- 172.21.15.117:0/2223026512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cbc105380 0x7f6cbc10d8d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0009900 tx=0x7f6cb002a410 comp rx=0 tx=0).stop 2024-09-21T05:48:56.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.892+0000 7f6cc1666640 1 -- 172.21.15.117:0/2223026512 shutdown_connections 2024-09-21T05:48:56.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.892+0000 7f6cc1666640 1 --2- 172.21.15.117:0/2223026512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cbc105380 0x7f6cbc10d8d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:56.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.892+0000 7f6cc1666640 1 --2- 172.21.15.117:0/2223026512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cbc10de90 0x7f6cbc110380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:56.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.892+0000 7f6cc1666640 1 --2- 172.21.15.117:0/2223026512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cbc104a30 0x7f6cbc104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:56.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.892+0000 7f6cc1666640 1 -- 172.21.15.117:0/2223026512 >> 172.21.15.117:0/2223026512 conn(0x7f6cbc079770 msgr2=0x7f6cbc079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:56.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.893+0000 7f6cc1666640 1 -- 172.21.15.117:0/2223026512 shutdown_connections 2024-09-21T05:48:56.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.893+0000 7f6cc1666640 1 -- 172.21.15.117:0/2223026512 wait complete. 2024-09-21T05:48:56.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.893+0000 7f6cc1666640 1 Processor -- start 2024-09-21T05:48:56.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.894+0000 7f6cc1666640 1 -- start start 2024-09-21T05:48:56.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.894+0000 7f6cc1666640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cbc104a30 0x7f6cbc19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:56.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.894+0000 7f6cc1666640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cbc105380 0x7f6cbc19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:56.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.894+0000 7f6cc1666640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cbc10de90 0x7f6cbc19fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:56.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.894+0000 7f6cc1666640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6cbc113b40 con 0x7f6cbc10de90 2024-09-21T05:48:56.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.894+0000 7f6cc1666640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6cbc1139c0 con 0x7f6cbc104a30 2024-09-21T05:48:56.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.894+0000 7f6cc1666640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6cbc113cc0 con 0x7f6cbc105380 2024-09-21T05:48:56.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.894+0000 7f6cbb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cbc10de90 0x7f6cbc19fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:56.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.894+0000 7f6cbaffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cbc104a30 0x7f6cbc19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:56.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.894+0000 7f6cbaffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cbc104a30 0x7f6cbc19a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:41584/0 (socket says 172.21.15.117:41584) 2024-09-21T05:48:56.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.894+0000 7f6cba7fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cbc105380 0x7f6cbc19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:56.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.894+0000 7f6cbaffd640 1 -- 172.21.15.117:0/2822285218 learned_addr learned my addr 172.21.15.117:0/2822285218 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:48:56.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.894+0000 7f6cbb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cbc10de90 0x7f6cbc19fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40648/0 (socket says 172.21.15.117:40648) 2024-09-21T05:48:56.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.894+0000 7f6cbb7fe640 1 -- 172.21.15.117:0/2822285218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cbc105380 msgr2=0x7f6cbc19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:56.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.894+0000 7f6cbb7fe640 1 --2- 172.21.15.117:0/2822285218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cbc105380 0x7f6cbc19ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:56.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.894+0000 7f6cbb7fe640 1 -- 172.21.15.117:0/2822285218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cbc104a30 msgr2=0x7f6cbc19a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:56.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.894+0000 7f6cbb7fe640 1 --2- 172.21.15.117:0/2822285218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cbc104a30 0x7f6cbc19a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:56.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.894+0000 7f6cbb7fe640 1 -- 172.21.15.117:0/2822285218 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cbc1a0540 con 0x7f6cbc10de90 2024-09-21T05:48:56.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.894+0000 7f6cbaffd640 1 --2- 172.21.15.117:0/2822285218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cbc104a30 0x7f6cbc19a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:48:56.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.895+0000 7f6cbb7fe640 1 --2- 172.21.15.117:0/2822285218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cbc10de90 0x7f6cbc19fe10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400d680 tx=0x7f6cb400db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:56.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.895+0000 7f6ca3fff640 1 -- 172.21.15.117:0/2822285218 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4017070 con 0x7f6cbc10de90 2024-09-21T05:48:56.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.895+0000 7f6cc1666640 1 -- 172.21.15.117:0/2822285218 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cbc1a0830 con 0x7f6cbc10de90 2024-09-21T05:48:56.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.895+0000 7f6ca3fff640 1 -- 172.21.15.117:0/2822285218 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cb4009820 con 0x7f6cbc10de90 2024-09-21T05:48:56.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.895+0000 7f6ca3fff640 1 -- 172.21.15.117:0/2822285218 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4004ce0 con 0x7f6cbc10de90 2024-09-21T05:48:56.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.895+0000 7f6cc1666640 1 -- 172.21.15.117:0/2822285218 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cbc1a6f90 con 0x7f6cbc10de90 2024-09-21T05:48:56.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.896+0000 7f6cc1666640 1 -- 172.21.15.117:0/2822285218 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cbc06c260 con 0x7f6cbc10de90 2024-09-21T05:48:56.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.897+0000 7f6ca3fff640 1 -- 172.21.15.117:0/2822285218 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7f6cb4005220 con 0x7f6cbc10de90 2024-09-21T05:48:56.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.898+0000 7f6ca3fff640 1 --2- 172.21.15.117:0/2822285218 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6ca40b64d0 0x7f6ca40b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:56.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.898+0000 7f6cbaffd640 1 --2- 172.21.15.117:0/2822285218 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6ca40b64d0 0x7f6ca40b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:56.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.898+0000 7f6ca3fff640 1 -- 172.21.15.117:0/2822285218 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f6cb40e4ed0 con 0x7f6cbc10de90 2024-09-21T05:48:56.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.899+0000 7f6cbaffd640 1 --2- 172.21.15.117:0/2822285218 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6ca40b64d0 0x7f6ca40b89e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6cac009fd0 tx=0x7f6cac009700 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:56.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:56.902+0000 7f6ca3fff640 1 -- 172.21.15.117:0/2822285218 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6cb40b1740 con 0x7f6cbc10de90 2024-09-21T05:48:57.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.088+0000 7f6cc1666640 1 -- 172.21.15.117:0/2822285218 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7f6cbc1a7160 con 0x7f6cbc10de90 2024-09-21T05:48:57.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.092+0000 7f6ca3fff640 1 -- 172.21.15.117:0/2822285218 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v29)=0 v29) ==== 116+0+0 (secure 0 0 0) 0x7f6cb40b6680 con 0x7f6cbc10de90 2024-09-21T05:48:57.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.095+0000 7f6cc1666640 1 -- 172.21.15.117:0/2822285218 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6ca40b64d0 msgr2=0x7f6ca40b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:57.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.095+0000 7f6cc1666640 1 --2- 172.21.15.117:0/2822285218 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6ca40b64d0 0x7f6ca40b89e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6cac009fd0 tx=0x7f6cac009700 comp rx=0 tx=0).stop 2024-09-21T05:48:57.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.095+0000 7f6cc1666640 1 -- 172.21.15.117:0/2822285218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cbc10de90 msgr2=0x7f6cbc19fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:57.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.095+0000 7f6cc1666640 1 --2- 172.21.15.117:0/2822285218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cbc10de90 0x7f6cbc19fe10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400d680 tx=0x7f6cb400db40 comp rx=0 tx=0).stop 2024-09-21T05:48:57.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.095+0000 7f6cc1666640 1 -- 172.21.15.117:0/2822285218 shutdown_connections 2024-09-21T05:48:57.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.095+0000 7f6cc1666640 1 --2- 172.21.15.117:0/2822285218 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6ca40b64d0 0x7f6ca40b89e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:57.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.095+0000 7f6cc1666640 1 --2- 172.21.15.117:0/2822285218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6cbc10de90 0x7f6cbc19fe10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:57.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.095+0000 7f6cc1666640 1 --2- 172.21.15.117:0/2822285218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cbc105380 0x7f6cbc19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:57.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.095+0000 7f6cc1666640 1 --2- 172.21.15.117:0/2822285218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cbc104a30 0x7f6cbc19a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:57.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.095+0000 7f6cc1666640 1 -- 172.21.15.117:0/2822285218 >> 172.21.15.117:0/2822285218 conn(0x7f6cbc079770 msgr2=0x7f6cbc107990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:57.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.095+0000 7f6cc1666640 1 -- 172.21.15.117:0/2822285218 shutdown_connections 2024-09-21T05:48:57.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.095+0000 7f6cc1666640 1 -- 172.21.15.117:0/2822285218 wait complete. 2024-09-21T05:48:57.512 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:57 smithi117 ceph-mon[30947]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:57.512 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:57 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:57.512 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:57 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:57.512 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:57 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T05:48:57.512 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:57 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T05:48:57.512 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:57 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:57.512 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:57 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2822285218' entity='client.admin' 2024-09-21T05:48:57.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.513+0000 7fb43787e640 1 -- 172.21.15.117:0/1570690590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4301083c0 msgr2=0x7fb4301087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:57.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.513+0000 7fb43787e640 1 --2- 172.21.15.117:0/1570690590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4301083c0 0x7fb4301087d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb420009900 tx=0x7fb42002a410 comp rx=0 tx=0).stop 2024-09-21T05:48:57.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.513+0000 7fb43787e640 1 -- 172.21.15.117:0/1570690590 shutdown_connections 2024-09-21T05:48:57.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.513+0000 7fb43787e640 1 --2- 172.21.15.117:0/1570690590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4301083c0 0x7fb4301087d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:57.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.513+0000 7fb43787e640 1 --2- 172.21.15.117:0/1570690590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb430108d10 0x7fb43010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:57.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.513+0000 7fb43787e640 1 --2- 172.21.15.117:0/1570690590 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb430102a80 0x7fb430102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:57.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.513+0000 7fb43787e640 1 -- 172.21.15.117:0/1570690590 >> 172.21.15.117:0/1570690590 conn(0x7fb430079740 msgr2=0x7fb430079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:57.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.513+0000 7fb43787e640 1 -- 172.21.15.117:0/1570690590 shutdown_connections 2024-09-21T05:48:57.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.513+0000 7fb43787e640 1 -- 172.21.15.117:0/1570690590 wait complete. 2024-09-21T05:48:57.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.514+0000 7fb43787e640 1 Processor -- start 2024-09-21T05:48:57.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.514+0000 7fb43787e640 1 -- start start 2024-09-21T05:48:57.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.514+0000 7fb43787e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb430102a80 0x7fb43019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:57.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.514+0000 7fb43787e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4301083c0 0x7fb43019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:57.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.514+0000 7fb43787e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb430108d10 0x7fb43019fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:57.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.514+0000 7fb43787e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb430113aa0 con 0x7fb430108d10 2024-09-21T05:48:57.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.514+0000 7fb43787e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb430113920 con 0x7fb4301083c0 2024-09-21T05:48:57.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.514+0000 7fb43787e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb430113c20 con 0x7fb430102a80 2024-09-21T05:48:57.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.514+0000 7fb435e1b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb430108d10 0x7fb43019fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:57.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.515+0000 7fb435e1b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb430108d10 0x7fb43019fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40656/0 (socket says 172.21.15.117:40656) 2024-09-21T05:48:57.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.515+0000 7fb435e1b640 1 -- 172.21.15.117:0/2129395410 learned_addr learned my addr 172.21.15.117:0/2129395410 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:48:57.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.515+0000 7fb43561a640 1 --2- 172.21.15.117:0/2129395410 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb430102a80 0x7fb43019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:57.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.515+0000 7fb434e19640 1 --2- 172.21.15.117:0/2129395410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4301083c0 0x7fb43019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:57.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.515+0000 7fb435e1b640 1 -- 172.21.15.117:0/2129395410 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb430102a80 msgr2=0x7fb43019a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:57.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.515+0000 7fb435e1b640 1 --2- 172.21.15.117:0/2129395410 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb430102a80 0x7fb43019a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:57.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.515+0000 7fb435e1b640 1 -- 172.21.15.117:0/2129395410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4301083c0 msgr2=0x7fb43019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:57.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.515+0000 7fb435e1b640 1 --2- 172.21.15.117:0/2129395410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4301083c0 0x7fb43019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:57.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.515+0000 7fb435e1b640 1 -- 172.21.15.117:0/2129395410 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4301a0300 con 0x7fb430108d10 2024-09-21T05:48:57.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.515+0000 7fb434e19640 1 --2- 172.21.15.117:0/2129395410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4301083c0 0x7fb43019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:48:57.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.515+0000 7fb435e1b640 1 --2- 172.21.15.117:0/2129395410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb430108d10 0x7fb43019fc90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb42800eaa0 tx=0x7fb42800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:57.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.515+0000 7fb4267fc640 1 -- 172.21.15.117:0/2129395410 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb428004d20 con 0x7fb430108d10 2024-09-21T05:48:57.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.515+0000 7fb4267fc640 1 -- 172.21.15.117:0/2129395410 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb428005020 con 0x7fb430108d10 2024-09-21T05:48:57.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.515+0000 7fb43787e640 1 -- 172.21.15.117:0/2129395410 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4301a05f0 con 0x7fb430108d10 2024-09-21T05:48:57.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.515+0000 7fb4267fc640 1 -- 172.21.15.117:0/2129395410 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4280116b0 con 0x7fb430108d10 2024-09-21T05:48:57.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.515+0000 7fb43787e640 1 -- 172.21.15.117:0/2129395410 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4301a0ab0 con 0x7fb430108d10 2024-09-21T05:48:57.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.516+0000 7fb43787e640 1 -- 172.21.15.117:0/2129395410 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb430102e90 con 0x7fb430108d10 2024-09-21T05:48:57.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.518+0000 7fb4267fc640 1 -- 172.21.15.117:0/2129395410 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7fb4280051d0 con 0x7fb430108d10 2024-09-21T05:48:57.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.519+0000 7fb4267fc640 1 --2- 172.21.15.117:0/2129395410 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb41c0b6500 0x7fb41c0b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:57.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.519+0000 7fb43561a640 1 --2- 172.21.15.117:0/2129395410 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb41c0b6500 0x7fb41c0b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:57.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.519+0000 7fb4267fc640 1 -- 172.21.15.117:0/2129395410 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fb4280e5d60 con 0x7fb430108d10 2024-09-21T05:48:57.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.520+0000 7fb43561a640 1 --2- 172.21.15.117:0/2129395410 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb41c0b6500 0x7fb41c0b8a10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb42c00abc0 tx=0x7fb42c00a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:57.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.523+0000 7fb4267fc640 1 -- 172.21.15.117:0/2129395410 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb428016030 con 0x7fb430108d10 2024-09-21T05:48:57.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.706+0000 7fb43787e640 1 -- 172.21.15.117:0/2129395410 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7fb43002e640 con 0x7fb430108d10 2024-09-21T05:48:57.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.710+0000 7fb4267fc640 1 -- 172.21.15.117:0/2129395410 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v30)=0 v30) ==== 111+0+0 (secure 0 0 0) 0x7fb4280ae1d0 con 0x7fb430108d10 2024-09-21T05:48:57.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.713+0000 7fb43787e640 1 -- 172.21.15.117:0/2129395410 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb41c0b6500 msgr2=0x7fb41c0b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:57.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.713+0000 7fb43787e640 1 --2- 172.21.15.117:0/2129395410 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb41c0b6500 0x7fb41c0b8a10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb42c00abc0 tx=0x7fb42c00a2d0 comp rx=0 tx=0).stop 2024-09-21T05:48:57.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.713+0000 7fb43787e640 1 -- 172.21.15.117:0/2129395410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb430108d10 msgr2=0x7fb43019fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:57.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.713+0000 7fb43787e640 1 --2- 172.21.15.117:0/2129395410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb430108d10 0x7fb43019fc90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb42800eaa0 tx=0x7fb42800eea0 comp rx=0 tx=0).stop 2024-09-21T05:48:57.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.713+0000 7fb43787e640 1 -- 172.21.15.117:0/2129395410 shutdown_connections 2024-09-21T05:48:57.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.713+0000 7fb43787e640 1 --2- 172.21.15.117:0/2129395410 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb41c0b6500 0x7fb41c0b8a10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:57.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.713+0000 7fb43787e640 1 --2- 172.21.15.117:0/2129395410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb430108d10 0x7fb43019fc90 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:57.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.713+0000 7fb43787e640 1 --2- 172.21.15.117:0/2129395410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4301083c0 0x7fb43019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:57.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.713+0000 7fb43787e640 1 --2- 172.21.15.117:0/2129395410 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb430102a80 0x7fb43019a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:57.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.713+0000 7fb43787e640 1 -- 172.21.15.117:0/2129395410 >> 172.21.15.117:0/2129395410 conn(0x7fb430079740 msgr2=0x7fb4301052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:57.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.713+0000 7fb43787e640 1 -- 172.21.15.117:0/2129395410 shutdown_connections 2024-09-21T05:48:57.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.713+0000 7fb43787e640 1 -- 172.21.15.117:0/2129395410 wait complete. 2024-09-21T05:48:57.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:57 smithi138 ceph-mon[34391]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:57.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:57 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:57.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:57 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:57.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:57 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T05:48:57.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:57 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T05:48:57.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:57 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:57.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:57 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2822285218' entity='client.admin' 2024-09-21T05:48:57.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.913+0000 7f521f8d9640 1 -- 172.21.15.117:0/3458458827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5218109df0 msgr2=0x7f5218110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:57.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.913+0000 7f521f8d9640 1 --2- 172.21.15.117:0/3458458827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5218109df0 0x7f5218110620 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5210009ca0 tx=0x7f521002a8c0 comp rx=0 tx=0).stop 2024-09-21T05:48:57.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.913+0000 7f521f8d9640 1 -- 172.21.15.117:0/3458458827 shutdown_connections 2024-09-21T05:48:57.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.913+0000 7f521f8d9640 1 --2- 172.21.15.117:0/3458458827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5218109df0 0x7f5218110620 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:57.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.913+0000 7f521f8d9640 1 --2- 172.21.15.117:0/3458458827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52181094a0 0x7f52181098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:57.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.913+0000 7f521f8d9640 1 --2- 172.21.15.117:0/3458458827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5218103b60 0x7f5218103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:57.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.913+0000 7f521f8d9640 1 -- 172.21.15.117:0/3458458827 >> 172.21.15.117:0/3458458827 conn(0x7f521807b850 msgr2=0x7f521807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:57.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.913+0000 7f521f8d9640 1 -- 172.21.15.117:0/3458458827 shutdown_connections 2024-09-21T05:48:57.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.913+0000 7f521f8d9640 1 -- 172.21.15.117:0/3458458827 wait complete. 2024-09-21T05:48:57.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.914+0000 7f521f8d9640 1 Processor -- start 2024-09-21T05:48:57.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.914+0000 7f521f8d9640 1 -- start start 2024-09-21T05:48:57.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.914+0000 7f521f8d9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5218103b60 0x7f521819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:57.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.914+0000 7f521f8d9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52181094a0 0x7f521819a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:57.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.914+0000 7f521f8d9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5218109df0 0x7f521819fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:57.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.914+0000 7f521f8d9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5218113ab0 con 0x7f52181094a0 2024-09-21T05:48:57.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.914+0000 7f521f8d9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5218113930 con 0x7f5218103b60 2024-09-21T05:48:57.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.914+0000 7f521f8d9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5218113c30 con 0x7f5218109df0 2024-09-21T05:48:57.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.915+0000 7f521ce74640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52181094a0 0x7f521819a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:57.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.915+0000 7f521ce74640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52181094a0 0x7f521819a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40676/0 (socket says 172.21.15.117:40676) 2024-09-21T05:48:57.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.915+0000 7f521ce74640 1 -- 172.21.15.117:0/343448911 learned_addr learned my addr 172.21.15.117:0/343448911 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:48:57.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.915+0000 7f521d675640 1 --2- 172.21.15.117:0/343448911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5218103b60 0x7f521819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:57.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.915+0000 7f521de76640 1 --2- 172.21.15.117:0/343448911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5218109df0 0x7f521819fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:57.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.915+0000 7f521ce74640 1 -- 172.21.15.117:0/343448911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5218109df0 msgr2=0x7f521819fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:57.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.915+0000 7f521ce74640 1 --2- 172.21.15.117:0/343448911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5218109df0 0x7f521819fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:57.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.915+0000 7f521ce74640 1 -- 172.21.15.117:0/343448911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5218103b60 msgr2=0x7f521819a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:57.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.915+0000 7f521ce74640 1 --2- 172.21.15.117:0/343448911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5218103b60 0x7f521819a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:57.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.915+0000 7f521ce74640 1 -- 172.21.15.117:0/343448911 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52181a0370 con 0x7f52181094a0 2024-09-21T05:48:57.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.915+0000 7f521ce74640 1 --2- 172.21.15.117:0/343448911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52181094a0 0x7f521819a9a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f520800c970 tx=0x7f520800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:57.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.915+0000 7f520e7fc640 1 -- 172.21.15.117:0/343448911 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5208007bb0 con 0x7f52181094a0 2024-09-21T05:48:57.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.915+0000 7f520e7fc640 1 -- 172.21.15.117:0/343448911 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5208007d50 con 0x7f52181094a0 2024-09-21T05:48:57.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.915+0000 7f520e7fc640 1 -- 172.21.15.117:0/343448911 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520800f450 con 0x7f52181094a0 2024-09-21T05:48:57.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.915+0000 7f521f8d9640 1 -- 172.21.15.117:0/343448911 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52181a0660 con 0x7f52181094a0 2024-09-21T05:48:57.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.915+0000 7f521f8d9640 1 -- 172.21.15.117:0/343448911 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52181a6dc0 con 0x7f52181094a0 2024-09-21T05:48:57.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.916+0000 7f521f8d9640 1 -- 172.21.15.117:0/343448911 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5218103f70 con 0x7f52181094a0 2024-09-21T05:48:57.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.919+0000 7f520e7fc640 1 -- 172.21.15.117:0/343448911 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7f5208004eb0 con 0x7f52181094a0 2024-09-21T05:48:57.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.920+0000 7f520e7fc640 1 --2- 172.21.15.117:0/343448911 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f52040b6550 0x7f52040b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:57.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.920+0000 7f521d675640 1 --2- 172.21.15.117:0/343448911 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f52040b6550 0x7f52040b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:57.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.920+0000 7f520e7fc640 1 -- 172.21.15.117:0/343448911 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f52080e57d0 con 0x7f52181094a0 2024-09-21T05:48:57.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.920+0000 7f521d675640 1 --2- 172.21.15.117:0/343448911 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f52040b6550 0x7f52040b8a60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5214009bd0 tx=0x7f5214005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:57.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:57.923+0000 7f520e7fc640 1 -- 172.21.15.117:0/343448911 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5208014030 con 0x7f52181094a0 2024-09-21T05:48:57.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:57 smithi192 ceph-mon[34700]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:57.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:57 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:57.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:57 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:57.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:57 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T05:48:57.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:57 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T05:48:57.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:57 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:57.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:57 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2822285218' entity='client.admin' 2024-09-21T05:48:58.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:58.109+0000 7f521f8d9640 1 -- 172.21.15.117:0/343448911 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f521802e640 con 0x7f52181094a0 2024-09-21T05:48:58.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:58.113+0000 7f520e7fc640 1 -- 172.21.15.117:0/343448911 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v31)=0 v31) ==== 121+0+0 (secure 0 0 0) 0x7f52080adce0 con 0x7f52181094a0 2024-09-21T05:48:58.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:58.116+0000 7f521f8d9640 1 -- 172.21.15.117:0/343448911 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f52040b6550 msgr2=0x7f52040b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:58.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:58.116+0000 7f521f8d9640 1 --2- 172.21.15.117:0/343448911 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f52040b6550 0x7f52040b8a60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5214009bd0 tx=0x7f5214005f90 comp rx=0 tx=0).stop 2024-09-21T05:48:58.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:58.116+0000 7f521f8d9640 1 -- 172.21.15.117:0/343448911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52181094a0 msgr2=0x7f521819a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:58.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:58.116+0000 7f521f8d9640 1 --2- 172.21.15.117:0/343448911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52181094a0 0x7f521819a9a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f520800c970 tx=0x7f520800ce30 comp rx=0 tx=0).stop 2024-09-21T05:48:58.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:58.116+0000 7f521f8d9640 1 -- 172.21.15.117:0/343448911 shutdown_connections 2024-09-21T05:48:58.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:58.116+0000 7f521f8d9640 1 --2- 172.21.15.117:0/343448911 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f52040b6550 0x7f52040b8a60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:58.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:58.116+0000 7f521f8d9640 1 --2- 172.21.15.117:0/343448911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52181094a0 0x7f521819a9a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:58.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:58.116+0000 7f521f8d9640 1 --2- 172.21.15.117:0/343448911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5218109df0 0x7f521819fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:58.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:58.116+0000 7f521f8d9640 1 --2- 172.21.15.117:0/343448911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5218103b60 0x7f521819a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:58.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:58.116+0000 7f521f8d9640 1 -- 172.21.15.117:0/343448911 >> 172.21.15.117:0/343448911 conn(0x7f521807b850 msgr2=0x7f5218107490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:58.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:58.116+0000 7f521f8d9640 1 -- 172.21.15.117:0/343448911 shutdown_connections 2024-09-21T05:48:58.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:58.116+0000 7f521f8d9640 1 -- 172.21.15.117:0/343448911 wait complete. 2024-09-21T05:48:58.668 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:58 smithi117 ceph-mon[30947]: Reconfiguring mon.b (monmap changed)... 2024-09-21T05:48:58.668 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:58 smithi117 ceph-mon[30947]: Reconfiguring daemon mon.b on smithi138 2024-09-21T05:48:58.669 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:58 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2129395410' entity='client.admin' 2024-09-21T05:48:58.669 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:58 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:58.669 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:58 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:58.669 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:58 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T05:48:58.669 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:58 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T05:48:58.669 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:58 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:58.669 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:58 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/343448911' entity='client.admin' 2024-09-21T05:48:58.703 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:58 smithi192 ceph-mon[34700]: Reconfiguring mon.b (monmap changed)... 2024-09-21T05:48:58.703 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:58 smithi192 ceph-mon[34700]: Reconfiguring daemon mon.b on smithi138 2024-09-21T05:48:58.703 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:58 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2129395410' entity='client.admin' 2024-09-21T05:48:58.704 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:58 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:58.704 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:58 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:58.704 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:58 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T05:48:58.704 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:58 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T05:48:58.704 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:58 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:58.704 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:58 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/343448911' entity='client.admin' 2024-09-21T05:48:58.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:58 smithi138 ceph-mon[34391]: Reconfiguring mon.b (monmap changed)... 2024-09-21T05:48:58.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:58 smithi138 ceph-mon[34391]: Reconfiguring daemon mon.b on smithi138 2024-09-21T05:48:58.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:58 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2129395410' entity='client.admin' 2024-09-21T05:48:58.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:58 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:58.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:58 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:48:58.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:58 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T05:48:58.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:58 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T05:48:58.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:58 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:48:58.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:58 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/343448911' entity='client.admin' 2024-09-21T05:48:59.518 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:48:59.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:59 smithi192 ceph-mon[34700]: Reconfiguring mon.c (monmap changed)... 2024-09-21T05:48:59.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:59 smithi192 ceph-mon[34700]: Reconfiguring daemon mon.c on smithi192 2024-09-21T05:48:59.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:48:59 smithi192 ceph-mon[34700]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:59.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:59 smithi117 ceph-mon[30947]: Reconfiguring mon.c (monmap changed)... 2024-09-21T05:48:59.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:59 smithi117 ceph-mon[30947]: Reconfiguring daemon mon.c on smithi192 2024-09-21T05:48:59.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:48:59 smithi117 ceph-mon[30947]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:48:59.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:59 smithi138 ceph-mon[34391]: Reconfiguring mon.c (monmap changed)... 2024-09-21T05:48:59.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:59 smithi138 ceph-mon[34391]: Reconfiguring daemon mon.c on smithi192 2024-09-21T05:48:59.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:48:59 smithi138 ceph-mon[34391]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:00.296 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ac8581a2-77dc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi117') 2024-09-21T05:49:00.296 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ac8581a2-77dc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi117') 2024-09-21T05:49:00.296 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ac8581a2-77dc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi117') 2024-09-21T05:49:00.296 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ac8581a2-77dc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi117') 2024-09-21T05:49:00.296 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ac8581a2-77dc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi117') 2024-09-21T05:49:00.296 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ac8581a2-77dc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi117') 2024-09-21T05:49:00.296 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ac8581a2-77dc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi117') 2024-09-21T05:49:00.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.623+0000 7fa80c59d640 1 -- 172.21.15.117:0/3137875171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa804103bb0 msgr2=0x7fa804103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:00.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.623+0000 7fa80c59d640 1 --2- 172.21.15.117:0/3137875171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa804103bb0 0x7fa804103fc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa800009900 tx=0x7fa80002a410 comp rx=0 tx=0).stop 2024-09-21T05:49:00.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.623+0000 7fa80c59d640 1 -- 172.21.15.117:0/3137875171 shutdown_connections 2024-09-21T05:49:00.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.623+0000 7fa80c59d640 1 --2- 172.21.15.117:0/3137875171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa804109e40 0x7fa804110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:00.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.623+0000 7fa80c59d640 1 --2- 172.21.15.117:0/3137875171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8041094f0 0x7fa804109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:00.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.623+0000 7fa80c59d640 1 --2- 172.21.15.117:0/3137875171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa804103bb0 0x7fa804103fc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:00.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.623+0000 7fa80c59d640 1 -- 172.21.15.117:0/3137875171 >> 172.21.15.117:0/3137875171 conn(0x7fa8040ff600 msgr2=0x7fa804101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:49:00.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.623+0000 7fa80c59d640 1 -- 172.21.15.117:0/3137875171 shutdown_connections 2024-09-21T05:49:00.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.623+0000 7fa80c59d640 1 -- 172.21.15.117:0/3137875171 wait complete. 2024-09-21T05:49:00.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.624+0000 7fa80c59d640 1 Processor -- start 2024-09-21T05:49:00.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.624+0000 7fa80c59d640 1 -- start start 2024-09-21T05:49:00.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.624+0000 7fa80c59d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa804103bb0 0x7fa80419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:00.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.624+0000 7fa80c59d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8041094f0 0x7fa80419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:00.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.624+0000 7fa80c59d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa804109e40 0x7fa80419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:00.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.625+0000 7fa80c59d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa804113b00 con 0x7fa804109e40 2024-09-21T05:49:00.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.625+0000 7fa80c59d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa804113980 con 0x7fa8041094f0 2024-09-21T05:49:00.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.625+0000 7fa80c59d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa804113c80 con 0x7fa804103bb0 2024-09-21T05:49:00.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.625+0000 7fa80ab3a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa804109e40 0x7fa80419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:00.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.625+0000 7fa80ab3a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa804109e40 0x7fa80419fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40628/0 (socket says 172.21.15.117:40628) 2024-09-21T05:49:00.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.625+0000 7fa80ab3a640 1 -- 172.21.15.117:0/2223068127 learned_addr learned my addr 172.21.15.117:0/2223068127 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:49:00.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.625+0000 7fa809b38640 1 --2- 172.21.15.117:0/2223068127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8041094f0 0x7fa80419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:00.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.625+0000 7fa80a339640 1 --2- 172.21.15.117:0/2223068127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa804103bb0 0x7fa80419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:00.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.625+0000 7fa80ab3a640 1 -- 172.21.15.117:0/2223068127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa804103bb0 msgr2=0x7fa80419a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:00.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.625+0000 7fa80ab3a640 1 --2- 172.21.15.117:0/2223068127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa804103bb0 0x7fa80419a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:00.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.625+0000 7fa80ab3a640 1 -- 172.21.15.117:0/2223068127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8041094f0 msgr2=0x7fa80419aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:00.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.625+0000 7fa80ab3a640 1 --2- 172.21.15.117:0/2223068127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8041094f0 0x7fa80419aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:00.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.625+0000 7fa80ab3a640 1 -- 172.21.15.117:0/2223068127 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8041a03a0 con 0x7fa804109e40 2024-09-21T05:49:00.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.625+0000 7fa80ab3a640 1 --2- 172.21.15.117:0/2223068127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa804109e40 0x7fa80419fd30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc00cce0 tx=0x7fa7fc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:49:00.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.625+0000 7fa7fb7fe640 1 -- 172.21.15.117:0/2223068127 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc00f450 con 0x7fa804109e40 2024-09-21T05:49:00.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.625+0000 7fa7fb7fe640 1 -- 172.21.15.117:0/2223068127 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7fc00faa0 con 0x7fa804109e40 2024-09-21T05:49:00.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.625+0000 7fa80c59d640 1 -- 172.21.15.117:0/2223068127 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8041a0690 con 0x7fa804109e40 2024-09-21T05:49:00.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.625+0000 7fa7fb7fe640 1 -- 172.21.15.117:0/2223068127 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc018b70 con 0x7fa804109e40 2024-09-21T05:49:00.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.625+0000 7fa80c59d640 1 -- 172.21.15.117:0/2223068127 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8041a6eb0 con 0x7fa804109e40 2024-09-21T05:49:00.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.626+0000 7fa80c59d640 1 -- 172.21.15.117:0/2223068127 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa804103fc0 con 0x7fa804109e40 2024-09-21T05:49:00.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.629+0000 7fa7fb7fe640 1 -- 172.21.15.117:0/2223068127 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7fa7fc018d10 con 0x7fa804109e40 2024-09-21T05:49:00.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.630+0000 7fa7fb7fe640 1 --2- 172.21.15.117:0/2223068127 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa7f00b6580 0x7fa7f00b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:00.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.630+0000 7fa80a339640 1 --2- 172.21.15.117:0/2223068127 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa7f00b6580 0x7fa7f00b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:00.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.630+0000 7fa7fb7fe640 1 -- 172.21.15.117:0/2223068127 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fa7fc0e5b80 con 0x7fa804109e40 2024-09-21T05:49:00.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.630+0000 7fa80a339640 1 --2- 172.21.15.117:0/2223068127 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa7f00b6580 0x7fa7f00b8a90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa800031440 tx=0x7fa800005d50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:49:00.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.633+0000 7fa7fb7fe640 1 -- 172.21.15.117:0/2223068127 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa7fc014030 con 0x7fa804109e40 2024-09-21T05:49:00.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.816+0000 7fa80c59d640 1 -- 172.21.15.117:0/2223068127 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fa80419b620 con 0x7fa804109e40 2024-09-21T05:49:00.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.822+0000 7fa7fb7fe640 1 -- 172.21.15.117:0/2223068127 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v28)=0 v28) ==== 121+0+0 (secure 0 0 0) 0x7fa80419b620 con 0x7fa804109e40 2024-09-21T05:49:00.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.824+0000 7fa80c59d640 1 -- 172.21.15.117:0/2223068127 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa7f00b6580 msgr2=0x7fa7f00b8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:00.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.824+0000 7fa80c59d640 1 --2- 172.21.15.117:0/2223068127 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa7f00b6580 0x7fa7f00b8a90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa800031440 tx=0x7fa800005d50 comp rx=0 tx=0).stop 2024-09-21T05:49:00.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.824+0000 7fa80c59d640 1 -- 172.21.15.117:0/2223068127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa804109e40 msgr2=0x7fa80419fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:00.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.824+0000 7fa80c59d640 1 --2- 172.21.15.117:0/2223068127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa804109e40 0x7fa80419fd30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc00cce0 tx=0x7fa7fc00a620 comp rx=0 tx=0).stop 2024-09-21T05:49:00.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.824+0000 7fa80c59d640 1 -- 172.21.15.117:0/2223068127 shutdown_connections 2024-09-21T05:49:00.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.824+0000 7fa80c59d640 1 --2- 172.21.15.117:0/2223068127 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa7f00b6580 0x7fa7f00b8a90 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:00.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.824+0000 7fa80c59d640 1 --2- 172.21.15.117:0/2223068127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa804109e40 0x7fa80419fd30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:00.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.824+0000 7fa80c59d640 1 --2- 172.21.15.117:0/2223068127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8041094f0 0x7fa80419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:00.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.824+0000 7fa80c59d640 1 --2- 172.21.15.117:0/2223068127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa804103bb0 0x7fa80419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:00.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.824+0000 7fa80c59d640 1 -- 172.21.15.117:0/2223068127 >> 172.21.15.117:0/2223068127 conn(0x7fa8040ff600 msgr2=0x7fa804101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:49:00.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.824+0000 7fa80c59d640 1 -- 172.21.15.117:0/2223068127 shutdown_connections 2024-09-21T05:49:00.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:48:54.824+0000 7fa80c59d640 1 -- 172.21.15.117:0/2223068127 wait complete. 2024-09-21T05:49:00.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:00 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:00.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:00 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:00.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:00 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:49:00.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:00 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:49:00.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:00 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:49:00.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:00 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:00.558 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:49:00.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:00 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:00.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:00 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:00.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:00 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:49:00.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:00 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:49:00.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:00 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:49:00.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:00 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:00.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:00 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:00.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:00 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:00.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:00 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:49:00.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:00 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:49:00.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:00 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:49:00.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:00 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:01.644 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:01 smithi117 ceph-mon[30947]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:01.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.854+0000 7f8969c2f640 1 -- 172.21.15.117:0/3413936259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8964103c30 msgr2=0x7f8964104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:01.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.854+0000 7f8969c2f640 1 --2- 172.21.15.117:0/3413936259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8964103c30 0x7f8964104040 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8954009900 tx=0x7f895402a410 comp rx=0 tx=0).stop 2024-09-21T05:49:01.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.854+0000 7f8969c2f640 1 -- 172.21.15.117:0/3413936259 shutdown_connections 2024-09-21T05:49:01.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.854+0000 7f8969c2f640 1 --2- 172.21.15.117:0/3413936259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8964109ec0 0x7f89641106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:01.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.854+0000 7f8969c2f640 1 --2- 172.21.15.117:0/3413936259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8964109570 0x7f8964109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:01.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.854+0000 7f8969c2f640 1 --2- 172.21.15.117:0/3413936259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8964103c30 0x7f8964104040 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:01.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.854+0000 7f8969c2f640 1 -- 172.21.15.117:0/3413936259 >> 172.21.15.117:0/3413936259 conn(0x7f896407b8d0 msgr2=0x7f896407bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:49:01.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.854+0000 7f8969c2f640 1 -- 172.21.15.117:0/3413936259 shutdown_connections 2024-09-21T05:49:01.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.854+0000 7f8969c2f640 1 -- 172.21.15.117:0/3413936259 wait complete. 2024-09-21T05:49:01.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.855+0000 7f8969c2f640 1 Processor -- start 2024-09-21T05:49:01.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.855+0000 7f8969c2f640 1 -- start start 2024-09-21T05:49:01.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.855+0000 7f8969c2f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8964103c30 0x7f896419c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:01.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.855+0000 7f8969c2f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8964109570 0x7f896419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:01.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.855+0000 7f8969c2f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8964109ec0 0x7f89641a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:01.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.855+0000 7f8969c2f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8964113ae0 con 0x7f8964109ec0 2024-09-21T05:49:01.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.855+0000 7f8969c2f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8964113960 con 0x7f8964103c30 2024-09-21T05:49:01.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.855+0000 7f8969c2f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8964113c60 con 0x7f8964109570 2024-09-21T05:49:01.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.855+0000 7f8963fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8964109ec0 0x7f89641a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:01.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.855+0000 7f8963fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8964109ec0 0x7f89641a1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40706/0 (socket says 172.21.15.117:40706) 2024-09-21T05:49:01.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.855+0000 7f8963fff640 1 -- 172.21.15.117:0/3534251602 learned_addr learned my addr 172.21.15.117:0/3534251602 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:49:01.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.855+0000 7f895bfff640 1 --2- 172.21.15.117:0/3534251602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8964109570 0x7f896419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:01.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.855+0000 7f89637fe640 1 --2- 172.21.15.117:0/3534251602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8964103c30 0x7f896419c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:01.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.855+0000 7f8963fff640 1 -- 172.21.15.117:0/3534251602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8964109570 msgr2=0x7f896419cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:01.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.855+0000 7f8963fff640 1 --2- 172.21.15.117:0/3534251602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8964109570 0x7f896419cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:01.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.855+0000 7f8963fff640 1 -- 172.21.15.117:0/3534251602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8964103c30 msgr2=0x7f896419c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:01.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.855+0000 7f8963fff640 1 --2- 172.21.15.117:0/3534251602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8964103c30 0x7f896419c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:01.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.855+0000 7f8963fff640 1 -- 172.21.15.117:0/3534251602 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89641a2620 con 0x7f8964109ec0 2024-09-21T05:49:01.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.855+0000 7f895bfff640 1 --2- 172.21.15.117:0/3534251602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8964109570 0x7f896419cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:49:01.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.855+0000 7f89637fe640 1 --2- 172.21.15.117:0/3534251602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8964103c30 0x7f896419c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:49:01.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.856+0000 7f8963fff640 1 --2- 172.21.15.117:0/3534251602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8964109ec0 0x7f89641a1ef0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f895c00cad0 tx=0x7f895c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:49:01.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.856+0000 7f89617fa640 1 -- 172.21.15.117:0/3534251602 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c00ae20 con 0x7f8964109ec0 2024-09-21T05:49:01.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.856+0000 7f8969c2f640 1 -- 172.21.15.117:0/3534251602 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89641a2910 con 0x7f8964109ec0 2024-09-21T05:49:01.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.856+0000 7f89617fa640 1 -- 172.21.15.117:0/3534251602 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f895c005020 con 0x7f8964109ec0 2024-09-21T05:49:01.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.856+0000 7f89617fa640 1 -- 172.21.15.117:0/3534251602 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c00f6b0 con 0x7f8964109ec0 2024-09-21T05:49:01.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.856+0000 7f8969c2f640 1 -- 172.21.15.117:0/3534251602 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89641a9070 con 0x7f8964109ec0 2024-09-21T05:49:01.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.857+0000 7f8969c2f640 1 -- 172.21.15.117:0/3534251602 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8964104040 con 0x7f8964109ec0 2024-09-21T05:49:01.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.859+0000 7f89617fa640 1 -- 172.21.15.117:0/3534251602 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7f895c0051d0 con 0x7f8964109ec0 2024-09-21T05:49:01.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.859+0000 7f89617fa640 1 --2- 172.21.15.117:0/3534251602 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f89500b6550 0x7f89500b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:01.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.859+0000 7f89637fe640 1 --2- 172.21.15.117:0/3534251602 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f89500b6550 0x7f89500b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:01.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.860+0000 7f89617fa640 1 -- 172.21.15.117:0/3534251602 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f895c0e5aa0 con 0x7f8964109ec0 2024-09-21T05:49:01.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.860+0000 7f89637fe640 1 --2- 172.21.15.117:0/3534251602 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f89500b6550 0x7f89500b8a60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8954031440 tx=0x7f8954009680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:49:01.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:01 smithi138 ceph-mon[34391]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:01.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.867+0000 7f89617fa640 1 -- 172.21.15.117:0/3534251602 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f895c014030 con 0x7f8964109ec0 2024-09-21T05:49:01.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:01 smithi192 ceph-mon[34700]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:02.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:02.062+0000 7f8969c2f640 1 -- 172.21.15.117:0/3534251602 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f896419d7e0 con 0x7f8964109ec0 2024-09-21T05:49:02.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:02.068+0000 7f89617fa640 1 -- 172.21.15.117:0/3534251602 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v33)=0 v33) ==== 125+0+0 (secure 0 0 0) 0x7f896419d7e0 con 0x7f8964109ec0 2024-09-21T05:49:02.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:02.071+0000 7f8969c2f640 1 -- 172.21.15.117:0/3534251602 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f89500b6550 msgr2=0x7f89500b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:02.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:02.071+0000 7f8969c2f640 1 --2- 172.21.15.117:0/3534251602 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f89500b6550 0x7f89500b8a60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8954031440 tx=0x7f8954009680 comp rx=0 tx=0).stop 2024-09-21T05:49:02.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:02.071+0000 7f8969c2f640 1 -- 172.21.15.117:0/3534251602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8964109ec0 msgr2=0x7f89641a1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:02.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:02.071+0000 7f8969c2f640 1 --2- 172.21.15.117:0/3534251602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8964109ec0 0x7f89641a1ef0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f895c00cad0 tx=0x7f895c00cf90 comp rx=0 tx=0).stop 2024-09-21T05:49:02.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:02.071+0000 7f8969c2f640 1 -- 172.21.15.117:0/3534251602 shutdown_connections 2024-09-21T05:49:02.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:02.071+0000 7f8969c2f640 1 --2- 172.21.15.117:0/3534251602 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f89500b6550 0x7f89500b8a60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:02.071+0000 7f8969c2f640 1 --2- 172.21.15.117:0/3534251602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8964109ec0 0x7f89641a1ef0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:02.071+0000 7f8969c2f640 1 --2- 172.21.15.117:0/3534251602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8964109570 0x7f896419cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:02.071+0000 7f8969c2f640 1 --2- 172.21.15.117:0/3534251602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8964103c30 0x7f896419c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:02.071+0000 7f8969c2f640 1 -- 172.21.15.117:0/3534251602 >> 172.21.15.117:0/3534251602 conn(0x7f896407b8d0 msgr2=0x7f89641073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:49:02.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:02.071+0000 7f8969c2f640 1 -- 172.21.15.117:0/3534251602 shutdown_connections 2024-09-21T05:49:02.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:02.071+0000 7f8969c2f640 1 -- 172.21.15.117:0/3534251602 wait complete. 2024-09-21T05:49:02.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.737+0000 7ff0680ff640 1 -- 172.21.15.117:0/1673734076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff060105380 msgr2=0x7ff06010d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:02.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.737+0000 7ff0680ff640 1 --2- 172.21.15.117:0/1673734076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff060105380 0x7ff06010d8d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff050009900 tx=0x7ff05002a410 comp rx=0 tx=0).stop 2024-09-21T05:49:02.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.737+0000 7ff0680ff640 1 -- 172.21.15.117:0/1673734076 shutdown_connections 2024-09-21T05:49:02.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.737+0000 7ff0680ff640 1 --2- 172.21.15.117:0/1673734076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff060105380 0x7ff06010d8d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.737+0000 7ff0680ff640 1 --2- 172.21.15.117:0/1673734076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff06010de90 0x7ff060110380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.737+0000 7ff0680ff640 1 --2- 172.21.15.117:0/1673734076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff060104a30 0x7ff060104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.737+0000 7ff0680ff640 1 -- 172.21.15.117:0/1673734076 >> 172.21.15.117:0/1673734076 conn(0x7ff060079770 msgr2=0x7ff060079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:49:02.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.737+0000 7ff0680ff640 1 -- 172.21.15.117:0/1673734076 shutdown_connections 2024-09-21T05:49:02.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.737+0000 7ff0680ff640 1 -- 172.21.15.117:0/1673734076 wait complete. 2024-09-21T05:49:02.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.738+0000 7ff0680ff640 1 Processor -- start 2024-09-21T05:49:02.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.738+0000 7ff0680ff640 1 -- start start 2024-09-21T05:49:02.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.738+0000 7ff0680ff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff060104a30 0x7ff06019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:02.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.738+0000 7ff0680ff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff060105380 0x7ff06019aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:02.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.738+0000 7ff0680ff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff06010de90 0x7ff06019fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:02.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.738+0000 7ff0680ff640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff060113af0 con 0x7ff060105380 2024-09-21T05:49:02.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.738+0000 7ff0680ff640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff060113970 con 0x7ff060104a30 2024-09-21T05:49:02.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.738+0000 7ff0680ff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff060113c70 con 0x7ff06010de90 2024-09-21T05:49:02.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.738+0000 7ff06569a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff060105380 0x7ff06019aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:02.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.738+0000 7ff06569a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff060105380 0x7ff06019aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40694/0 (socket says 172.21.15.117:40694) 2024-09-21T05:49:02.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.738+0000 7ff06569a640 1 -- 172.21.15.117:0/2595734376 learned_addr learned my addr 172.21.15.117:0/2595734376 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:49:02.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.738+0000 7ff06569a640 1 -- 172.21.15.117:0/2595734376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff06010de90 msgr2=0x7ff06019fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:02.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.738+0000 7ff06569a640 1 --2- 172.21.15.117:0/2595734376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff06010de90 0x7ff06019fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.738+0000 7ff06569a640 1 -- 172.21.15.117:0/2595734376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff060104a30 msgr2=0x7ff06019a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:02.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.738+0000 7ff06569a640 1 --2- 172.21.15.117:0/2595734376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff060104a30 0x7ff06019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.738+0000 7ff06569a640 1 -- 172.21.15.117:0/2595734376 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0601a04b0 con 0x7ff060105380 2024-09-21T05:49:02.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.739+0000 7ff06569a640 1 --2- 172.21.15.117:0/2595734376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff060105380 0x7ff06019aae0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff05002a930 tx=0x7ff050004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:49:02.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.739+0000 7ff056ffd640 1 -- 172.21.15.117:0/2595734376 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05003f030 con 0x7ff060105380 2024-09-21T05:49:02.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.739+0000 7ff056ffd640 1 -- 172.21.15.117:0/2595734376 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff050003cb0 con 0x7ff060105380 2024-09-21T05:49:02.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.739+0000 7ff0680ff640 1 -- 172.21.15.117:0/2595734376 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0601a0740 con 0x7ff060105380 2024-09-21T05:49:02.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.739+0000 7ff056ffd640 1 -- 172.21.15.117:0/2595734376 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff050035070 con 0x7ff060105380 2024-09-21T05:49:02.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.739+0000 7ff0680ff640 1 -- 172.21.15.117:0/2595734376 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0601a6f00 con 0x7ff060105380 2024-09-21T05:49:02.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.740+0000 7ff0680ff640 1 -- 172.21.15.117:0/2595734376 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff06006c260 con 0x7ff060105380 2024-09-21T05:49:02.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.742+0000 7ff056ffd640 1 -- 172.21.15.117:0/2595734376 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7ff050042030 con 0x7ff060105380 2024-09-21T05:49:02.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.743+0000 7ff056ffd640 1 --2- 172.21.15.117:0/2595734376 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff04c0b64d0 0x7ff04c0b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:02.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.743+0000 7ff065e9b640 1 --2- 172.21.15.117:0/2595734376 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff04c0b64d0 0x7ff04c0b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:02.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.743+0000 7ff056ffd640 1 -- 172.21.15.117:0/2595734376 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7ff050106230 con 0x7ff060105380 2024-09-21T05:49:02.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.744+0000 7ff065e9b640 1 --2- 172.21.15.117:0/2595734376 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff04c0b64d0 0x7ff04c0b89e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff05c00a8f0 tx=0x7ff05c005bf0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:49:02.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.746+0000 7ff056ffd640 1 -- 172.21.15.117:0/2595734376 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff0500ce6c0 con 0x7ff060105380 2024-09-21T05:49:02.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.940+0000 7ff0680ff640 1 -- 172.21.15.117:0/2595734376 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7ff060104e40 con 0x7ff060105380 2024-09-21T05:49:02.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.946+0000 7ff056ffd640 1 -- 172.21.15.117:0/2595734376 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v32)=0 v32) ==== 119+0+0 (secure 0 0 0) 0x7ff0500d5e20 con 0x7ff060105380 2024-09-21T05:49:02.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.949+0000 7ff0680ff640 1 -- 172.21.15.117:0/2595734376 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff04c0b64d0 msgr2=0x7ff04c0b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:02.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.949+0000 7ff0680ff640 1 --2- 172.21.15.117:0/2595734376 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff04c0b64d0 0x7ff04c0b89e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff05c00a8f0 tx=0x7ff05c005bf0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.949+0000 7ff0680ff640 1 -- 172.21.15.117:0/2595734376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff060105380 msgr2=0x7ff06019aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:02.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.949+0000 7ff0680ff640 1 --2- 172.21.15.117:0/2595734376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff060105380 0x7ff06019aae0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff05002a930 tx=0x7ff050004270 comp rx=0 tx=0).stop 2024-09-21T05:49:02.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.949+0000 7ff0680ff640 1 -- 172.21.15.117:0/2595734376 shutdown_connections 2024-09-21T05:49:02.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.949+0000 7ff0680ff640 1 --2- 172.21.15.117:0/2595734376 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff04c0b64d0 0x7ff04c0b89e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.949+0000 7ff0680ff640 1 --2- 172.21.15.117:0/2595734376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff060105380 0x7ff06019aae0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.949+0000 7ff0680ff640 1 --2- 172.21.15.117:0/2595734376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff06010de90 0x7ff06019fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.949+0000 7ff0680ff640 1 --2- 172.21.15.117:0/2595734376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff060104a30 0x7ff06019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.949+0000 7ff0680ff640 1 -- 172.21.15.117:0/2595734376 >> 172.21.15.117:0/2595734376 conn(0x7ff060079770 msgr2=0x7ff060100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:49:02.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.949+0000 7ff0680ff640 1 -- 172.21.15.117:0/2595734376 shutdown_connections 2024-09-21T05:49:02.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:01.949+0000 7ff0680ff640 1 -- 172.21.15.117:0/2595734376 wait complete. 2024-09-21T05:49:03.085 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:02 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2595734376' entity='client.admin' 2024-09-21T05:49:03.085 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:02 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:49:03.085 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:02 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:49:03.085 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:02 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:49:03.085 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:02 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:03.085 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:02 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3534251602' entity='client.admin' 2024-09-21T05:49:03.086 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:02 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:49:03.086 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:02 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:49:03.086 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:02 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:49:03.086 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:02 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:03.086 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:02 smithi117 ceph-mon[30947]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:03.115 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ac8581a2-77dc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi117') 2024-09-21T05:49:03.115 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ac8581a2-77dc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi117') 2024-09-21T05:49:03.115 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ac8581a2-77dc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi117') 2024-09-21T05:49:03.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:02 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2595734376' entity='client.admin' 2024-09-21T05:49:03.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:02 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:49:03.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:02 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:49:03.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:02 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:49:03.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:02 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:03.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:02 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3534251602' entity='client.admin' 2024-09-21T05:49:03.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:02 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:49:03.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:02 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:49:03.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:02 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:49:03.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:02 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:03.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:02 smithi192 ceph-mon[34700]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:03.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:02 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2595734376' entity='client.admin' 2024-09-21T05:49:03.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:02 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:49:03.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:02 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:49:03.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:02 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:49:03.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:02 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:03.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:02 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3534251602' entity='client.admin' 2024-09-21T05:49:03.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:02 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:49:03.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:02 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:49:03.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:02 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:49:03.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:02 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:03.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:02 smithi138 ceph-mon[34391]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:04.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.256+0000 7f872b39e640 1 -- 172.21.15.117:0/1841865101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8724102a80 msgr2=0x7f8724102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:04.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.256+0000 7f872b39e640 1 --2- 172.21.15.117:0/1841865101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8724102a80 0x7f8724102e90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8720009900 tx=0x7f872002a410 comp rx=0 tx=0).stop 2024-09-21T05:49:04.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.256+0000 7f872b39e640 1 -- 172.21.15.117:0/1841865101 shutdown_connections 2024-09-21T05:49:04.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.256+0000 7f872b39e640 1 --2- 172.21.15.117:0/1841865101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8724108d10 0x7f872410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:04.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.256+0000 7f872b39e640 1 --2- 172.21.15.117:0/1841865101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87241083c0 0x7f87241087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:04.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.256+0000 7f872b39e640 1 --2- 172.21.15.117:0/1841865101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8724102a80 0x7f8724102e90 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:04.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.256+0000 7f872b39e640 1 -- 172.21.15.117:0/1841865101 >> 172.21.15.117:0/1841865101 conn(0x7f8724079740 msgr2=0x7f8724079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:49:04.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.256+0000 7f872b39e640 1 -- 172.21.15.117:0/1841865101 shutdown_connections 2024-09-21T05:49:04.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.256+0000 7f872b39e640 1 -- 172.21.15.117:0/1841865101 wait complete. 2024-09-21T05:49:04.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.257+0000 7f872b39e640 1 Processor -- start 2024-09-21T05:49:04.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.257+0000 7f872b39e640 1 -- start start 2024-09-21T05:49:04.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.257+0000 7f872b39e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8724102a80 0x7f872419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:04.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.257+0000 7f872b39e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87241083c0 0x7f872419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:04.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.257+0000 7f872b39e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8724108d10 0x7f872419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:04.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.257+0000 7f872b39e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8724113aa0 con 0x7f8724108d10 2024-09-21T05:49:04.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.257+0000 7f872b39e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8724113920 con 0x7f8724102a80 2024-09-21T05:49:04.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.257+0000 7f872b39e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8724113c20 con 0x7f87241083c0 2024-09-21T05:49:04.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.258+0000 7f872993b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8724108d10 0x7f872419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:04.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.258+0000 7f872993b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8724108d10 0x7f872419fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40726/0 (socket says 172.21.15.117:40726) 2024-09-21T05:49:04.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.258+0000 7f872993b640 1 -- 172.21.15.117:0/3478240521 learned_addr learned my addr 172.21.15.117:0/3478240521 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:49:04.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.258+0000 7f872913a640 1 --2- 172.21.15.117:0/3478240521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8724102a80 0x7f872419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:04.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.258+0000 7f8728939640 1 --2- 172.21.15.117:0/3478240521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87241083c0 0x7f872419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:04.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.258+0000 7f872993b640 1 -- 172.21.15.117:0/3478240521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87241083c0 msgr2=0x7f872419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:04.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.258+0000 7f872993b640 1 --2- 172.21.15.117:0/3478240521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87241083c0 0x7f872419aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:04.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.258+0000 7f872993b640 1 -- 172.21.15.117:0/3478240521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8724102a80 msgr2=0x7f872419a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:04.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.258+0000 7f872993b640 1 --2- 172.21.15.117:0/3478240521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8724102a80 0x7f872419a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:04.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.258+0000 7f872993b640 1 -- 172.21.15.117:0/3478240521 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87241a0440 con 0x7f8724108d10 2024-09-21T05:49:04.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.258+0000 7f8728939640 1 --2- 172.21.15.117:0/3478240521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87241083c0 0x7f872419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:49:04.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.258+0000 7f872993b640 1 --2- 172.21.15.117:0/3478240521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8724108d10 0x7f872419fd10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f871c00cad0 tx=0x7f871c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:49:04.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.258+0000 7f871a7fc640 1 -- 172.21.15.117:0/3478240521 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c00ae20 con 0x7f8724108d10 2024-09-21T05:49:04.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.258+0000 7f872b39e640 1 -- 172.21.15.117:0/3478240521 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87241a0730 con 0x7f8724108d10 2024-09-21T05:49:04.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.258+0000 7f871a7fc640 1 -- 172.21.15.117:0/3478240521 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f871c005020 con 0x7f8724108d10 2024-09-21T05:49:04.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.258+0000 7f871a7fc640 1 -- 172.21.15.117:0/3478240521 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c00f6b0 con 0x7f8724108d10 2024-09-21T05:49:04.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.258+0000 7f872b39e640 1 -- 172.21.15.117:0/3478240521 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87241a6e90 con 0x7f8724108d10 2024-09-21T05:49:04.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.259+0000 7f872b39e640 1 -- 172.21.15.117:0/3478240521 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8724103d00 con 0x7f8724108d10 2024-09-21T05:49:04.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.261+0000 7f871a7fc640 1 -- 172.21.15.117:0/3478240521 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7f871c0051d0 con 0x7f8724108d10 2024-09-21T05:49:04.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.261+0000 7f871a7fc640 1 --2- 172.21.15.117:0/3478240521 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f87100b64d0 0x7f87100b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:04.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.262+0000 7f872913a640 1 --2- 172.21.15.117:0/3478240521 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f87100b64d0 0x7f87100b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:04.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.262+0000 7f871a7fc640 1 -- 172.21.15.117:0/3478240521 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f871c0e57f0 con 0x7f8724108d10 2024-09-21T05:49:04.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.262+0000 7f872913a640 1 --2- 172.21.15.117:0/3478240521 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f87100b64d0 0x7f87100b89e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8720031440 tx=0x7f8720005d50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:49:04.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.266+0000 7f871a7fc640 1 -- 172.21.15.117:0/3478240521 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f871c014030 con 0x7f8724108d10 2024-09-21T05:49:04.450 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.448+0000 7f872b39e640 1 -- 172.21.15.117:0/3478240521 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f872402e640 con 0x7f8724108d10 2024-09-21T05:49:04.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.452+0000 7f871a7fc640 1 -- 172.21.15.117:0/3478240521 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v34)=0 v34) ==== 120+0+0 (secure 0 0 0) 0x7f871c0adc80 con 0x7f8724108d10 2024-09-21T05:49:04.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.455+0000 7f872b39e640 1 -- 172.21.15.117:0/3478240521 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f87100b64d0 msgr2=0x7f87100b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:04.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.455+0000 7f872b39e640 1 --2- 172.21.15.117:0/3478240521 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f87100b64d0 0x7f87100b89e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8720031440 tx=0x7f8720005d50 comp rx=0 tx=0).stop 2024-09-21T05:49:04.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.455+0000 7f872b39e640 1 -- 172.21.15.117:0/3478240521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8724108d10 msgr2=0x7f872419fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:04.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.455+0000 7f872b39e640 1 --2- 172.21.15.117:0/3478240521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8724108d10 0x7f872419fd10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f871c00cad0 tx=0x7f871c00cf90 comp rx=0 tx=0).stop 2024-09-21T05:49:04.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.455+0000 7f872b39e640 1 -- 172.21.15.117:0/3478240521 shutdown_connections 2024-09-21T05:49:04.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.455+0000 7f872b39e640 1 --2- 172.21.15.117:0/3478240521 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f87100b64d0 0x7f87100b89e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:04.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.455+0000 7f872b39e640 1 --2- 172.21.15.117:0/3478240521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8724108d10 0x7f872419fd10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:04.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.455+0000 7f872b39e640 1 --2- 172.21.15.117:0/3478240521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87241083c0 0x7f872419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:04.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.455+0000 7f872b39e640 1 --2- 172.21.15.117:0/3478240521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8724102a80 0x7f872419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:04.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.455+0000 7f872b39e640 1 -- 172.21.15.117:0/3478240521 >> 172.21.15.117:0/3478240521 conn(0x7f8724079740 msgr2=0x7f8724101470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:49:04.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.455+0000 7f872b39e640 1 -- 172.21.15.117:0/3478240521 shutdown_connections 2024-09-21T05:49:04.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:04.455+0000 7f872b39e640 1 -- 172.21.15.117:0/3478240521 wait complete. 2024-09-21T05:49:05.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.011+0000 7f12547e3640 1 -- 172.21.15.117:0/178035836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f124c10de10 msgr2=0x7f124c110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:05.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.011+0000 7f12547e3640 1 --2- 172.21.15.117:0/178035836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f124c10de10 0x7f124c110300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1244009900 tx=0x7f124402a410 comp rx=0 tx=0).stop 2024-09-21T05:49:05.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.011+0000 7f12547e3640 1 -- 172.21.15.117:0/178035836 shutdown_connections 2024-09-21T05:49:05.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.011+0000 7f12547e3640 1 --2- 172.21.15.117:0/178035836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f124c10de10 0x7f124c110300 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:05.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.011+0000 7f12547e3640 1 --2- 172.21.15.117:0/178035836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f124c105300 0x7f124c10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:05.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.011+0000 7f12547e3640 1 --2- 172.21.15.117:0/178035836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f124c1049b0 0x7f124c104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:05.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.011+0000 7f12547e3640 1 -- 172.21.15.117:0/178035836 >> 172.21.15.117:0/178035836 conn(0x7f124c079730 msgr2=0x7f124c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:49:05.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.011+0000 7f12547e3640 1 -- 172.21.15.117:0/178035836 shutdown_connections 2024-09-21T05:49:05.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.011+0000 7f12547e3640 1 -- 172.21.15.117:0/178035836 wait complete. 2024-09-21T05:49:05.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.012+0000 7f12547e3640 1 Processor -- start 2024-09-21T05:49:05.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.012+0000 7f12547e3640 1 -- start start 2024-09-21T05:49:05.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.012+0000 7f12547e3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f124c1049b0 0x7f124c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:05.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.012+0000 7f12547e3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f124c105300 0x7f124c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:05.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.012+0000 7f12547e3640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f124c10de10 0x7f124c19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:05.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.012+0000 7f12547e3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f124c113a90 con 0x7f124c1049b0 2024-09-21T05:49:05.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.012+0000 7f12547e3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f124c113910 con 0x7f124c105300 2024-09-21T05:49:05.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.012+0000 7f12547e3640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f124c113c10 con 0x7f124c10de10 2024-09-21T05:49:05.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.012+0000 7f125257f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f124c1049b0 0x7f124c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:05.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.012+0000 7f125257f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f124c1049b0 0x7f124c19a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39804/0 (socket says 172.21.15.117:39804) 2024-09-21T05:49:05.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.012+0000 7f125257f640 1 -- 172.21.15.117:0/3006848427 learned_addr learned my addr 172.21.15.117:0/3006848427 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:49:05.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.012+0000 7f1252d80640 1 --2- 172.21.15.117:0/3006848427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f124c10de10 0x7f124c19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:05.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.012+0000 7f1251d7e640 1 --2- 172.21.15.117:0/3006848427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f124c105300 0x7f124c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:05.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.012+0000 7f125257f640 1 -- 172.21.15.117:0/3006848427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f124c10de10 msgr2=0x7f124c19fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:05.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.012+0000 7f125257f640 1 --2- 172.21.15.117:0/3006848427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f124c10de10 0x7f124c19fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:05.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.012+0000 7f125257f640 1 -- 172.21.15.117:0/3006848427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f124c105300 msgr2=0x7f124c19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:05.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.012+0000 7f125257f640 1 --2- 172.21.15.117:0/3006848427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f124c105300 0x7f124c19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:05.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.012+0000 7f125257f640 1 -- 172.21.15.117:0/3006848427 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f124c1a03d0 con 0x7f124c1049b0 2024-09-21T05:49:05.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.013+0000 7f125257f640 1 --2- 172.21.15.117:0/3006848427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f124c1049b0 0x7f124c19a460 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f123c00eaa0 tx=0x7f123c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:49:05.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.013+0000 7f124b7fe640 1 -- 172.21.15.117:0/3006848427 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c016070 con 0x7f124c1049b0 2024-09-21T05:49:05.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.013+0000 7f124b7fe640 1 -- 172.21.15.117:0/3006848427 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f123c003e20 con 0x7f124c1049b0 2024-09-21T05:49:05.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.013+0000 7f124b7fe640 1 -- 172.21.15.117:0/3006848427 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c0116d0 con 0x7f124c1049b0 2024-09-21T05:49:05.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.013+0000 7f12547e3640 1 -- 172.21.15.117:0/3006848427 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f124c1a06c0 con 0x7f124c1049b0 2024-09-21T05:49:05.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.013+0000 7f12547e3640 1 -- 172.21.15.117:0/3006848427 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f124c1a6e20 con 0x7f124c1049b0 2024-09-21T05:49:05.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.014+0000 7f12547e3640 1 -- 172.21.15.117:0/3006848427 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f124c104dc0 con 0x7f124c1049b0 2024-09-21T05:49:05.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.015+0000 7f124b7fe640 1 -- 172.21.15.117:0/3006848427 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7f123c019070 con 0x7f124c1049b0 2024-09-21T05:49:05.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.016+0000 7f124b7fe640 1 --2- 172.21.15.117:0/3006848427 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f12340b6550 0x7f12340b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:05.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.017+0000 7f1251d7e640 1 --2- 172.21.15.117:0/3006848427 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f12340b6550 0x7f12340b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:05.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.017+0000 7f124b7fe640 1 -- 172.21.15.117:0/3006848427 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f123c0e5a90 con 0x7f124c1049b0 2024-09-21T05:49:05.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.017+0000 7f1251d7e640 1 --2- 172.21.15.117:0/3006848427 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f12340b6550 0x7f12340b8a60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f124c10fa50 tx=0x7f124000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:49:05.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.020+0000 7f124b7fe640 1 -- 172.21.15.117:0/3006848427 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f123c0b2380 con 0x7f124c1049b0 2024-09-21T05:49:05.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.201+0000 7f12547e3640 1 -- 172.21.15.117:0/3006848427 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f124c1a7150 con 0x7f124c1049b0 2024-09-21T05:49:05.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.205+0000 7f124b7fe640 1 -- 172.21.15.117:0/3006848427 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v35)=0 v35) ==== 121+0+0 (secure 0 0 0) 0x7f123c0b72c0 con 0x7f124c1049b0 2024-09-21T05:49:05.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.208+0000 7f12547e3640 1 -- 172.21.15.117:0/3006848427 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f12340b6550 msgr2=0x7f12340b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:05.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.208+0000 7f12547e3640 1 --2- 172.21.15.117:0/3006848427 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f12340b6550 0x7f12340b8a60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f124c10fa50 tx=0x7f124000b390 comp rx=0 tx=0).stop 2024-09-21T05:49:05.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.208+0000 7f12547e3640 1 -- 172.21.15.117:0/3006848427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f124c1049b0 msgr2=0x7f124c19a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:05.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.208+0000 7f12547e3640 1 --2- 172.21.15.117:0/3006848427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f124c1049b0 0x7f124c19a460 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f123c00eaa0 tx=0x7f123c00ef60 comp rx=0 tx=0).stop 2024-09-21T05:49:05.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.208+0000 7f12547e3640 1 -- 172.21.15.117:0/3006848427 shutdown_connections 2024-09-21T05:49:05.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.208+0000 7f12547e3640 1 --2- 172.21.15.117:0/3006848427 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f12340b6550 0x7f12340b8a60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:05.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.208+0000 7f12547e3640 1 --2- 172.21.15.117:0/3006848427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f124c10de10 0x7f124c19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:05.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.208+0000 7f12547e3640 1 --2- 172.21.15.117:0/3006848427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f124c105300 0x7f124c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:05.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.208+0000 7f12547e3640 1 --2- 172.21.15.117:0/3006848427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f124c1049b0 0x7f124c19a460 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:05.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.208+0000 7f12547e3640 1 -- 172.21.15.117:0/3006848427 >> 172.21.15.117:0/3006848427 conn(0x7f124c079730 msgr2=0x7f124c0ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:49:05.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.208+0000 7f12547e3640 1 -- 172.21.15.117:0/3006848427 shutdown_connections 2024-09-21T05:49:05.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:05.208+0000 7f12547e3640 1 -- 172.21.15.117:0/3006848427 wait complete. 2024-09-21T05:49:05.311 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:49:05.311 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:49:05.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:05 smithi192 ceph-mon[34700]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:05.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:05 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3478240521' entity='client.admin' 2024-09-21T05:49:05.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:05 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3006848427' entity='client.admin' 2024-09-21T05:49:05.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:05 smithi117 ceph-mon[30947]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:05.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:05 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3478240521' entity='client.admin' 2024-09-21T05:49:05.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:05 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3006848427' entity='client.admin' 2024-09-21T05:49:05.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:05 smithi138 ceph-mon[34391]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:05.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:05 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3478240521' entity='client.admin' 2024-09-21T05:49:05.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:05 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3006848427' entity='client.admin' 2024-09-21T05:49:06.403 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ac8581a2-77dc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi117') 2024-09-21T05:49:06.403 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ac8581a2-77dc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi117') 2024-09-21T05:49:07.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.032+0000 7f10a4ce3640 1 -- 172.21.15.117:0/1806232674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a0109df0 msgr2=0x7f10a0110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:07.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.032+0000 7f10a4ce3640 1 --2- 172.21.15.117:0/1806232674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a0109df0 0x7f10a0110620 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1098009ca0 tx=0x7f109802a8c0 comp rx=0 tx=0).stop 2024-09-21T05:49:07.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.032+0000 7f10a4ce3640 1 -- 172.21.15.117:0/1806232674 shutdown_connections 2024-09-21T05:49:07.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.032+0000 7f10a4ce3640 1 --2- 172.21.15.117:0/1806232674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a0109df0 0x7f10a0110620 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:07.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.032+0000 7f10a4ce3640 1 --2- 172.21.15.117:0/1806232674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10a01094a0 0x7f10a01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:07.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.032+0000 7f10a4ce3640 1 --2- 172.21.15.117:0/1806232674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10a0103b60 0x7f10a0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:07.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.032+0000 7f10a4ce3640 1 -- 172.21.15.117:0/1806232674 >> 172.21.15.117:0/1806232674 conn(0x7f10a007b850 msgr2=0x7f10a007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:49:07.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.032+0000 7f10a4ce3640 1 -- 172.21.15.117:0/1806232674 shutdown_connections 2024-09-21T05:49:07.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.032+0000 7f10a4ce3640 1 -- 172.21.15.117:0/1806232674 wait complete. 2024-09-21T05:49:07.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.033+0000 7f10a4ce3640 1 Processor -- start 2024-09-21T05:49:07.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.033+0000 7f10a4ce3640 1 -- start start 2024-09-21T05:49:07.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.033+0000 7f10a4ce3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a0103b60 0x7f10a019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:07.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.033+0000 7f10a4ce3640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10a01094a0 0x7f10a019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:07.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.033+0000 7f10a4ce3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10a0109df0 0x7f10a019fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:07.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.033+0000 7f10a4ce3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f10a0113b10 con 0x7f10a0103b60 2024-09-21T05:49:07.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.033+0000 7f10a4ce3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f10a0113990 con 0x7f10a0109df0 2024-09-21T05:49:07.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.033+0000 7f10a4ce3640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f10a0113c90 con 0x7f10a01094a0 2024-09-21T05:49:07.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.034+0000 7f109e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a0103b60 0x7f10a019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:07.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.034+0000 7f109e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a0103b60 0x7f10a019a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39816/0 (socket says 172.21.15.117:39816) 2024-09-21T05:49:07.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.034+0000 7f109e59c640 1 -- 172.21.15.117:0/1404393810 learned_addr learned my addr 172.21.15.117:0/1404393810 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:49:07.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.034+0000 7f109ed9d640 1 --2- 172.21.15.117:0/1404393810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10a0109df0 0x7f10a019fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:07.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.034+0000 7f1097fff640 1 --2- 172.21.15.117:0/1404393810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10a01094a0 0x7f10a019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:07.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.034+0000 7f109e59c640 1 -- 172.21.15.117:0/1404393810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10a01094a0 msgr2=0x7f10a019aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:07.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.034+0000 7f109e59c640 1 --2- 172.21.15.117:0/1404393810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10a01094a0 0x7f10a019aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:07.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.034+0000 7f109e59c640 1 -- 172.21.15.117:0/1404393810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10a0109df0 msgr2=0x7f10a019fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:07.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.034+0000 7f109e59c640 1 --2- 172.21.15.117:0/1404393810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10a0109df0 0x7f10a019fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:07.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.034+0000 7f109e59c640 1 -- 172.21.15.117:0/1404393810 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a01a0390 con 0x7f10a0103b60 2024-09-21T05:49:07.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.034+0000 7f109e59c640 1 --2- 172.21.15.117:0/1404393810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a0103b60 0x7f10a019a4e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f109000b4a0 tx=0x7f109000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:49:07.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.034+0000 7f10977fe640 1 -- 172.21.15.117:0/1404393810 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1090014070 con 0x7f10a0103b60 2024-09-21T05:49:07.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.034+0000 7f10977fe640 1 -- 172.21.15.117:0/1404393810 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1090010ac0 con 0x7f10a0103b60 2024-09-21T05:49:07.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.034+0000 7f10a4ce3640 1 -- 172.21.15.117:0/1404393810 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10a01a0680 con 0x7f10a0103b60 2024-09-21T05:49:07.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.034+0000 7f10977fe640 1 -- 172.21.15.117:0/1404393810 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1090017c30 con 0x7f10a0103b60 2024-09-21T05:49:07.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.034+0000 7f10a4ce3640 1 -- 172.21.15.117:0/1404393810 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10a01a6ea0 con 0x7f10a0103b60 2024-09-21T05:49:07.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.036+0000 7f10a4ce3640 1 -- 172.21.15.117:0/1404393810 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10a0103f70 con 0x7f10a0103b60 2024-09-21T05:49:07.040 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.037+0000 7f10977fe640 1 -- 172.21.15.117:0/1404393810 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7f1090017070 con 0x7f10a0103b60 2024-09-21T05:49:07.040 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.038+0000 7f10977fe640 1 --2- 172.21.15.117:0/1404393810 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f108c0b6580 0x7f108c0b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:07.040 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.038+0000 7f1097fff640 1 --2- 172.21.15.117:0/1404393810 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f108c0b6580 0x7f108c0b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:07.040 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.038+0000 7f10977fe640 1 -- 172.21.15.117:0/1404393810 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f10900e4cf0 con 0x7f10a0103b60 2024-09-21T05:49:07.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.038+0000 7f1097fff640 1 --2- 172.21.15.117:0/1404393810 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f108c0b6580 0x7f108c0b8a90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1088005e10 tx=0x7f1088005da0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:49:07.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.041+0000 7f10977fe640 1 -- 172.21.15.117:0/1404393810 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f10900ad180 con 0x7f10a0103b60 2024-09-21T05:49:07.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.225+0000 7f10a4ce3640 1 -- 172.21.15.117:0/1404393810 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f10a019b610 con 0x7f10a0103b60 2024-09-21T05:49:07.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.225+0000 7ff252f46640 1 -- 172.21.15.117:0/4016496268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff24c102a60 msgr2=0x7ff24c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:07.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.225+0000 7ff252f46640 1 --2- 172.21.15.117:0/4016496268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff24c102a60 0x7ff24c102e70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff240009900 tx=0x7ff24002a410 comp rx=0 tx=0).stop 2024-09-21T05:49:07.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.226+0000 7ff252f46640 1 -- 172.21.15.117:0/4016496268 shutdown_connections 2024-09-21T05:49:07.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.226+0000 7ff252f46640 1 --2- 172.21.15.117:0/4016496268 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff24c108cf0 0x7ff24c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:07.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.226+0000 7ff252f46640 1 --2- 172.21.15.117:0/4016496268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff24c1083a0 0x7ff24c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:07.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.226+0000 7ff252f46640 1 --2- 172.21.15.117:0/4016496268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff24c102a60 0x7ff24c102e70 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:07.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.226+0000 7ff252f46640 1 -- 172.21.15.117:0/4016496268 >> 172.21.15.117:0/4016496268 conn(0x7ff24c079730 msgr2=0x7ff24c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:49:07.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.226+0000 7ff252f46640 1 -- 172.21.15.117:0/4016496268 shutdown_connections 2024-09-21T05:49:07.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.226+0000 7ff252f46640 1 -- 172.21.15.117:0/4016496268 wait complete. 2024-09-21T05:49:07.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.227+0000 7ff252f46640 1 Processor -- start 2024-09-21T05:49:07.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.227+0000 7ff252f46640 1 -- start start 2024-09-21T05:49:07.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.227+0000 7ff252f46640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff24c102a60 0x7ff24c199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:07.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.227+0000 7ff252f46640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff24c1083a0 0x7ff24c1998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:07.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.227+0000 7ff252f46640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff24c108cf0 0x7ff24c19eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:07.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.227+0000 7ff252f46640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c112960 con 0x7ff24c102a60 2024-09-21T05:49:07.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.227+0000 7ff252f46640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c1127e0 con 0x7ff24c108cf0 2024-09-21T05:49:07.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.227+0000 7ff252f46640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c112ae0 con 0x7ff24c1083a0 2024-09-21T05:49:07.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.227+0000 7ff250ce2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff24c102a60 0x7ff24c199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:07.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.227+0000 7ff250ce2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff24c102a60 0x7ff24c199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39838/0 (socket says 172.21.15.117:39838) 2024-09-21T05:49:07.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.227+0000 7ff250ce2640 1 -- 172.21.15.117:0/737035063 learned_addr learned my addr 172.21.15.117:0/737035063 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:49:07.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.227+0000 7ff24bfff640 1 --2- 172.21.15.117:0/737035063 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff24c1083a0 0x7ff24c1998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:07.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.228+0000 7ff250ce2640 1 -- 172.21.15.117:0/737035063 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff24c1083a0 msgr2=0x7ff24c1998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:07.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.228+0000 7ff250ce2640 1 --2- 172.21.15.117:0/737035063 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff24c1083a0 0x7ff24c1998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:07.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.228+0000 7ff250ce2640 1 -- 172.21.15.117:0/737035063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff24c108cf0 msgr2=0x7ff24c19eb90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T05:49:07.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.228+0000 7ff250ce2640 1 --2- 172.21.15.117:0/737035063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff24c108cf0 0x7ff24c19eb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:07.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.228+0000 7ff250ce2640 1 -- 172.21.15.117:0/737035063 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff24c19f2c0 con 0x7ff24c102a60 2024-09-21T05:49:07.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.228+0000 7ff250ce2640 1 --2- 172.21.15.117:0/737035063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff24c102a60 0x7ff24c199350 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff240005940 tx=0x7ff2400040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:49:07.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.228+0000 7ff249ffb640 1 -- 172.21.15.117:0/737035063 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24003f030 con 0x7ff24c102a60 2024-09-21T05:49:07.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.228+0000 7ff249ffb640 1 -- 172.21.15.117:0/737035063 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2400044c0 con 0x7ff24c102a60 2024-09-21T05:49:07.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.228+0000 7ff252f46640 1 -- 172.21.15.117:0/737035063 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff24c19f550 con 0x7ff24c102a60 2024-09-21T05:49:07.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.228+0000 7ff249ffb640 1 -- 172.21.15.117:0/737035063 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff240035070 con 0x7ff24c102a60 2024-09-21T05:49:07.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.228+0000 7ff252f46640 1 -- 172.21.15.117:0/737035063 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff24c1a5d10 con 0x7ff24c102a60 2024-09-21T05:49:07.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.229+0000 7ff252f46640 1 -- 172.21.15.117:0/737035063 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff24c102e70 con 0x7ff24c102a60 2024-09-21T05:49:07.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.231+0000 7f10977fe640 1 -- 172.21.15.117:0/1404393810 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v36)=0 v36) ==== 120+0+0 (secure 0 0 0) 0x7f10a019b610 con 0x7f10a0103b60 2024-09-21T05:49:07.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.234+0000 7f10a4ce3640 1 -- 172.21.15.117:0/1404393810 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f108c0b6580 msgr2=0x7f108c0b8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:07.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.234+0000 7f10a4ce3640 1 --2- 172.21.15.117:0/1404393810 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f108c0b6580 0x7f108c0b8a90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1088005e10 tx=0x7f1088005da0 comp rx=0 tx=0).stop 2024-09-21T05:49:07.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.234+0000 7f10a4ce3640 1 -- 172.21.15.117:0/1404393810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a0103b60 msgr2=0x7f10a019a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:07.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.234+0000 7f10a4ce3640 1 --2- 172.21.15.117:0/1404393810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a0103b60 0x7f10a019a4e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f109000b4a0 tx=0x7f109000b960 comp rx=0 tx=0).stop 2024-09-21T05:49:07.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.234+0000 7f10a4ce3640 1 -- 172.21.15.117:0/1404393810 shutdown_connections 2024-09-21T05:49:07.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.234+0000 7f10a4ce3640 1 --2- 172.21.15.117:0/1404393810 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f108c0b6580 0x7f108c0b8a90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:07.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.234+0000 7f10a4ce3640 1 --2- 172.21.15.117:0/1404393810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f10a0109df0 0x7f10a019fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:07.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.234+0000 7f10a4ce3640 1 --2- 172.21.15.117:0/1404393810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10a01094a0 0x7f10a019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:07.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.234+0000 7f10a4ce3640 1 --2- 172.21.15.117:0/1404393810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a0103b60 0x7f10a019a4e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:07.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.234+0000 7f10a4ce3640 1 -- 172.21.15.117:0/1404393810 >> 172.21.15.117:0/1404393810 conn(0x7f10a007b850 msgr2=0x7f10a01074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:49:07.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.234+0000 7f10a4ce3640 1 -- 172.21.15.117:0/1404393810 shutdown_connections 2024-09-21T05:49:07.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.234+0000 7f10a4ce3640 1 -- 172.21.15.117:0/1404393810 wait complete. 2024-09-21T05:49:07.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.236+0000 7ff249ffb640 1 -- 172.21.15.117:0/737035063 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7ff240042030 con 0x7ff24c102a60 2024-09-21T05:49:07.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.236+0000 7ff249ffb640 1 --2- 172.21.15.117:0/737035063 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff23c0b6550 0x7ff23c0b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:07.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.236+0000 7ff24bfff640 1 --2- 172.21.15.117:0/737035063 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff23c0b6550 0x7ff23c0b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:07.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.236+0000 7ff249ffb640 1 -- 172.21.15.117:0/737035063 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7ff240105dd0 con 0x7ff24c102a60 2024-09-21T05:49:07.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.237+0000 7ff24bfff640 1 --2- 172.21.15.117:0/737035063 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff23c0b6550 0x7ff23c0b8a60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff24c100c50 tx=0x7ff238005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:49:07.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.239+0000 7ff249ffb640 1 -- 172.21.15.117:0/737035063 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff2400ce260 con 0x7ff24c102a60 2024-09-21T05:49:07.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.428+0000 7ff252f46640 1 -- 172.21.15.117:0/737035063 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7ff24c19a480 con 0x7ff24c102a60 2024-09-21T05:49:07.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.434+0000 7ff249ffb640 1 -- 172.21.15.117:0/737035063 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v37)=0 v37) ==== 133+0+0 (secure 0 0 0) 0x7ff2400d2640 con 0x7ff24c102a60 2024-09-21T05:49:07.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.436+0000 7ff252f46640 1 -- 172.21.15.117:0/737035063 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff23c0b6550 msgr2=0x7ff23c0b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:07.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.436+0000 7ff252f46640 1 --2- 172.21.15.117:0/737035063 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff23c0b6550 0x7ff23c0b8a60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff24c100c50 tx=0x7ff238005f90 comp rx=0 tx=0).stop 2024-09-21T05:49:07.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.436+0000 7ff252f46640 1 -- 172.21.15.117:0/737035063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff24c102a60 msgr2=0x7ff24c199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:07.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.436+0000 7ff252f46640 1 --2- 172.21.15.117:0/737035063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff24c102a60 0x7ff24c199350 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff240005940 tx=0x7ff2400040c0 comp rx=0 tx=0).stop 2024-09-21T05:49:07.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.437+0000 7ff252f46640 1 -- 172.21.15.117:0/737035063 shutdown_connections 2024-09-21T05:49:07.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.437+0000 7ff252f46640 1 --2- 172.21.15.117:0/737035063 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff23c0b6550 0x7ff23c0b8a60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:07.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.437+0000 7ff252f46640 1 --2- 172.21.15.117:0/737035063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff24c108cf0 0x7ff24c19eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:07.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.437+0000 7ff252f46640 1 --2- 172.21.15.117:0/737035063 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff24c1083a0 0x7ff24c1998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:07.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.437+0000 7ff252f46640 1 --2- 172.21.15.117:0/737035063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff24c102a60 0x7ff24c199350 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:07.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.437+0000 7ff252f46640 1 -- 172.21.15.117:0/737035063 >> 172.21.15.117:0/737035063 conn(0x7ff24c079730 msgr2=0x7ff24c1013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:49:07.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.437+0000 7ff252f46640 1 -- 172.21.15.117:0/737035063 shutdown_connections 2024-09-21T05:49:07.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:07.437+0000 7ff252f46640 1 -- 172.21.15.117:0/737035063 wait complete. 2024-09-21T05:49:07.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:07 smithi117 ceph-mon[30947]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:07.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:07 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/1404393810' entity='client.admin' 2024-09-21T05:49:07.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:07 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/737035063' entity='client.admin' 2024-09-21T05:49:07.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:07 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:49:07.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:07 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:49:07.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:07 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:49:07.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:07 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:07.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:07 smithi192 ceph-mon[34700]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:07.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:07 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/1404393810' entity='client.admin' 2024-09-21T05:49:07.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:07 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/737035063' entity='client.admin' 2024-09-21T05:49:07.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:07 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:49:07.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:07 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:49:07.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:07 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:49:07.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:07 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:07.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:07 smithi138 ceph-mon[34391]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:07.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:07 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/1404393810' entity='client.admin' 2024-09-21T05:49:07.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:07 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/737035063' entity='client.admin' 2024-09-21T05:49:07.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:07 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:49:07.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:07 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:49:07.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:07 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:49:07.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:07 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:08.560 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ac8581a2-77dc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi117') 2024-09-21T05:49:08.560 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ac8581a2-77dc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_cephsqlite', '20'], check_status=True, hostname='smithi117') 2024-09-21T05:49:08.560 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ac8581a2-77dc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi117') 2024-09-21T05:49:08.560 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ac8581a2-77dc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi117') 2024-09-21T05:49:08.560 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ac8581a2-77dc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi117') 2024-09-21T05:49:08.560 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ac8581a2-77dc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/allow_m_granularity', 'True'], check_status=True, hostname='smithi117') 2024-09-21T05:49:08.560 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ac8581a2-77dc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/dump_on_update', 'True'], check_status=True, hostname='smithi117') 2024-09-21T05:49:08.561 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ac8581a2-77dc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/log_level', 'debug'], check_status=True, hostname='smithi117') 2024-09-21T05:49:08.561 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ac8581a2-77dc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi117') 2024-09-21T05:49:08.561 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ac8581a2-77dc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi117') 2024-09-21T05:49:08.561 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ac8581a2-77dc-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi117') 2024-09-21T05:49:08.561 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph config dump 2024-09-21T05:49:08.802 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:49:09.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.249+0000 7f40fb5c3640 1 -- 172.21.15.117:0/3360946648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40f41083d0 msgr2=0x7f40f41087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:09.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.249+0000 7f40fb5c3640 1 --2- 172.21.15.117:0/3360946648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40f41083d0 0x7f40f41087e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f40e4009900 tx=0x7f40e402a410 comp rx=0 tx=0).stop 2024-09-21T05:49:09.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.250+0000 7f40fb5c3640 1 -- 172.21.15.117:0/3360946648 shutdown_connections 2024-09-21T05:49:09.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.250+0000 7f40fb5c3640 1 --2- 172.21.15.117:0/3360946648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40f41083d0 0x7f40f41087e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:09.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.250+0000 7f40fb5c3640 1 --2- 172.21.15.117:0/3360946648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40f4108d20 0x7f40f410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:09.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.250+0000 7f40fb5c3640 1 --2- 172.21.15.117:0/3360946648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40f4102ae0 0x7f40f4102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:09.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.250+0000 7f40fb5c3640 1 -- 172.21.15.117:0/3360946648 >> 172.21.15.117:0/3360946648 conn(0x7f40f4079760 msgr2=0x7f40f4079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:49:09.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.250+0000 7f40fb5c3640 1 -- 172.21.15.117:0/3360946648 shutdown_connections 2024-09-21T05:49:09.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.250+0000 7f40fb5c3640 1 -- 172.21.15.117:0/3360946648 wait complete. 2024-09-21T05:49:09.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.250+0000 7f40fb5c3640 1 Processor -- start 2024-09-21T05:49:09.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.250+0000 7f40fb5c3640 1 -- start start 2024-09-21T05:49:09.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.251+0000 7f40fb5c3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40f4102ae0 0x7f40f41990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:09.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.251+0000 7f40fb5c3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40f41083d0 0x7f40f4199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:09.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.251+0000 7f40fb5c3640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40f4108d20 0x7f40f4199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:09.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.251+0000 7f40fb5c3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f40f4112990 con 0x7f40f41083d0 2024-09-21T05:49:09.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.251+0000 7f40fb5c3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f40f4112810 con 0x7f40f4102ae0 2024-09-21T05:49:09.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.251+0000 7f40fb5c3640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f40f4112b10 con 0x7f40f4108d20 2024-09-21T05:49:09.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.251+0000 7f40f8b5e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40f41083d0 0x7f40f4199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:09.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.251+0000 7f40f8b5e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40f41083d0 0x7f40f4199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39844/0 (socket says 172.21.15.117:39844) 2024-09-21T05:49:09.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.251+0000 7f40f8b5e640 1 -- 172.21.15.117:0/3780941486 learned_addr learned my addr 172.21.15.117:0/3780941486 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:49:09.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.251+0000 7f40f9b60640 1 --2- 172.21.15.117:0/3780941486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40f4108d20 0x7f40f4199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:09.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.251+0000 7f40f935f640 1 --2- 172.21.15.117:0/3780941486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40f4102ae0 0x7f40f41990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:09.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.251+0000 7f40f8b5e640 1 -- 172.21.15.117:0/3780941486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40f4108d20 msgr2=0x7f40f4199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:09.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.251+0000 7f40f8b5e640 1 --2- 172.21.15.117:0/3780941486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40f4108d20 0x7f40f4199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:09.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.251+0000 7f40f8b5e640 1 -- 172.21.15.117:0/3780941486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40f4102ae0 msgr2=0x7f40f41990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:09.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.251+0000 7f40f8b5e640 1 --2- 172.21.15.117:0/3780941486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40f4102ae0 0x7f40f41990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:09.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.251+0000 7f40f8b5e640 1 -- 172.21.15.117:0/3780941486 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f41a8780 con 0x7f40f41083d0 2024-09-21T05:49:09.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.251+0000 7f40f9b60640 1 --2- 172.21.15.117:0/3780941486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40f4108d20 0x7f40f4199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:49:09.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.251+0000 7f40f8b5e640 1 --2- 172.21.15.117:0/3780941486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40f41083d0 0x7f40f4199610 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f40e4032fd0 tx=0x7f40e4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:49:09.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.251+0000 7f40ea7fc640 1 -- 172.21.15.117:0/3780941486 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e403f030 con 0x7f40f41083d0 2024-09-21T05:49:09.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.251+0000 7f40fb5c3640 1 -- 172.21.15.117:0/3780941486 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f41a8a10 con 0x7f40f41083d0 2024-09-21T05:49:09.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.251+0000 7f40ea7fc640 1 -- 172.21.15.117:0/3780941486 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40e4004490 con 0x7f40f41083d0 2024-09-21T05:49:09.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.251+0000 7f40ea7fc640 1 -- 172.21.15.117:0/3780941486 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e4035070 con 0x7f40f41083d0 2024-09-21T05:49:09.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.252+0000 7f40fb5c3640 1 -- 172.21.15.117:0/3780941486 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f41a8ef0 con 0x7f40f41083d0 2024-09-21T05:49:09.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.253+0000 7f40fb5c3640 1 -- 172.21.15.117:0/3780941486 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40f4103d10 con 0x7f40f41083d0 2024-09-21T05:49:09.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.254+0000 7f40ea7fc640 1 -- 172.21.15.117:0/3780941486 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7f40e4042030 con 0x7f40f41083d0 2024-09-21T05:49:09.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.255+0000 7f40ea7fc640 1 --2- 172.21.15.117:0/3780941486 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f40e00b64d0 0x7f40e00b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:09.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.255+0000 7f40f935f640 1 --2- 172.21.15.117:0/3780941486 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f40e00b64d0 0x7f40e00b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:09.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.255+0000 7f40ea7fc640 1 -- 172.21.15.117:0/3780941486 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f40e4105990 con 0x7f40f41083d0 2024-09-21T05:49:09.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.255+0000 7f40f935f640 1 --2- 172.21.15.117:0/3780941486 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f40e00b64d0 0x7f40e00b89e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f40f0005c40 tx=0x7f40f000a3b0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:49:09.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.259+0000 7f40ea7fc640 1 -- 172.21.15.117:0/3780941486 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f40e40cde20 con 0x7f40f41083d0 2024-09-21T05:49:09.441 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.439+0000 7f40fb5c3640 1 -- 172.21.15.117:0/3780941486 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f40f406bf70 con 0x7f40f41083d0 2024-09-21T05:49:09.441 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.439+0000 7f40ea7fc640 1 -- 172.21.15.117:0/3780941486 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v37) ==== 59+0+22127 (secure 0 0 0) 0x7f40f406bf70 con 0x7f40f41083d0 2024-09-21T05:49:09.442 INFO:teuthology.orchestra.run.smithi117.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T05:49:09.442 INFO:teuthology.orchestra.run.smithi117.stdout:global dev auth_debug true 2024-09-21T05:49:09.442 INFO:teuthology.orchestra.run.smithi117.stdout:global basic container_image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 * 2024-09-21T05:49:09.442 INFO:teuthology.orchestra.run.smithi117.stdout:global dev debug_asserts_on_shutdown true 2024-09-21T05:49:09.442 INFO:teuthology.orchestra.run.smithi117.stdout:global basic log_to_file true 2024-09-21T05:49:09.442 INFO:teuthology.orchestra.run.smithi117.stdout:global basic log_to_journald false 2024-09-21T05:49:09.442 INFO:teuthology.orchestra.run.smithi117.stdout:global basic log_to_stderr false 2024-09-21T05:49:09.442 INFO:teuthology.orchestra.run.smithi117.stdout:global advanced mon_allow_pool_delete true 2024-09-21T05:49:09.442 INFO:teuthology.orchestra.run.smithi117.stdout:global advanced mon_clock_drift_allowed 1.000000 2024-09-21T05:49:09.443 INFO:teuthology.orchestra.run.smithi117.stdout:global advanced mon_cluster_log_to_file true 2024-09-21T05:49:09.443 INFO:teuthology.orchestra.run.smithi117.stdout:global advanced mon_max_pg_per_osd 10000 2024-09-21T05:49:09.443 INFO:teuthology.orchestra.run.smithi117.stdout:global advanced mon_pg_warn_max_object_skew 0.000000 2024-09-21T05:49:09.443 INFO:teuthology.orchestra.run.smithi117.stdout:global advanced mon_warn_on_crush_straw_calc_version_zero false 2024-09-21T05:49:09.443 INFO:teuthology.orchestra.run.smithi117.stdout:global advanced mon_warn_on_legacy_crush_tunables false 2024-09-21T05:49:09.443 INFO:teuthology.orchestra.run.smithi117.stdout:global advanced mon_warn_on_osd_down_out_interval_zero false 2024-09-21T05:49:09.443 INFO:teuthology.orchestra.run.smithi117.stdout:global dev mon_warn_on_pool_pg_num_not_power_of_two false 2024-09-21T05:49:09.443 INFO:teuthology.orchestra.run.smithi117.stdout:global advanced mon_warn_on_too_few_osds false 2024-09-21T05:49:09.443 INFO:teuthology.orchestra.run.smithi117.stdout:global dev ms_die_on_bug true 2024-09-21T05:49:09.443 INFO:teuthology.orchestra.run.smithi117.stdout:global dev ms_die_on_old_message true 2024-09-21T05:49:09.443 INFO:teuthology.orchestra.run.smithi117.stdout:global dev ms_die_on_skipped_message false 2024-09-21T05:49:09.443 INFO:teuthology.orchestra.run.smithi117.stdout:global advanced osd_pool_default_erasure_code_profile plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T05:49:09.443 INFO:teuthology.orchestra.run.smithi117.stdout:global advanced osd_pool_default_pg_autoscale_mode off 2024-09-21T05:49:09.443 INFO:teuthology.orchestra.run.smithi117.stdout:mon advanced auth_allow_insecure_global_id_reclaim false 2024-09-21T05:49:09.443 INFO:teuthology.orchestra.run.smithi117.stdout:mon advanced auth_mon_ticket_ttl 660.000000 2024-09-21T05:49:09.444 INFO:teuthology.orchestra.run.smithi117.stdout:mon advanced auth_service_ticket_ttl 240.000000 2024-09-21T05:49:09.444 INFO:teuthology.orchestra.run.smithi117.stdout:mon advanced debug_mon 20/20 2024-09-21T05:49:09.444 INFO:teuthology.orchestra.run.smithi117.stdout:mon advanced debug_ms 1/1 2024-09-21T05:49:09.444 INFO:teuthology.orchestra.run.smithi117.stdout:mon advanced debug_paxos 20/20 2024-09-21T05:49:09.444 INFO:teuthology.orchestra.run.smithi117.stdout:mon advanced mon_data_avail_warn 5 2024-09-21T05:49:09.444 INFO:teuthology.orchestra.run.smithi117.stdout:mon advanced mon_down_mkfs_grace 300 2024-09-21T05:49:09.444 INFO:teuthology.orchestra.run.smithi117.stdout:mon advanced mon_mgr_mkfs_grace 240 2024-09-21T05:49:09.444 INFO:teuthology.orchestra.run.smithi117.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T05:49:09.444 INFO:teuthology.orchestra.run.smithi117.stdout:mon dev mon_osd_prime_pg_temp true 2024-09-21T05:49:09.444 INFO:teuthology.orchestra.run.smithi117.stdout:mon advanced mon_osd_reporter_subtree_level osd 2024-09-21T05:49:09.444 INFO:teuthology.orchestra.run.smithi117.stdout:mon advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T05:49:09.444 INFO:teuthology.orchestra.run.smithi117.stdout:mon advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T05:49:09.444 INFO:teuthology.orchestra.run.smithi117.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim false 2024-09-21T05:49:09.444 INFO:teuthology.orchestra.run.smithi117.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim_allowed false 2024-09-21T05:49:09.444 INFO:teuthology.orchestra.run.smithi117.stdout:mon advanced public_network 172.21.0.0/20 * 2024-09-21T05:49:09.444 INFO:teuthology.orchestra.run.smithi117.stdout:mgr advanced cephsqlite_lock_renewal_timeout 900000 2024-09-21T05:49:09.445 INFO:teuthology.orchestra.run.smithi117.stdout:mgr advanced client_mount_timeout 30 2024-09-21T05:49:09.445 INFO:teuthology.orchestra.run.smithi117.stdout:mgr advanced debug_cephsqlite 20/20 2024-09-21T05:49:09.445 INFO:teuthology.orchestra.run.smithi117.stdout:mgr advanced debug_client 20/20 2024-09-21T05:49:09.445 INFO:teuthology.orchestra.run.smithi117.stdout:mgr advanced debug_mgr 20/20 2024-09-21T05:49:09.445 INFO:teuthology.orchestra.run.smithi117.stdout:mgr advanced debug_ms 1/1 2024-09-21T05:49:09.445 INFO:teuthology.orchestra.run.smithi117.stdout:mgr advanced mgr/cephadm/allow_ptrace true * 2024-09-21T05:49:09.445 INFO:teuthology.orchestra.run.smithi117.stdout:mgr advanced mgr/cephadm/container_init True * 2024-09-21T05:49:09.445 INFO:teuthology.orchestra.run.smithi117.stdout:mgr advanced mgr/cephadm/migration_current 7 * 2024-09-21T05:49:09.445 INFO:teuthology.orchestra.run.smithi117.stdout:mgr advanced mgr/dashboard/ssl_server_port 8443 * 2024-09-21T05:49:09.445 INFO:teuthology.orchestra.run.smithi117.stdout:mgr advanced mgr/orchestrator/orchestrator cephadm 2024-09-21T05:49:09.445 INFO:teuthology.orchestra.run.smithi117.stdout:mgr advanced mgr/snap_schedule/allow_m_granularity true 2024-09-21T05:49:09.445 INFO:teuthology.orchestra.run.smithi117.stdout:mgr advanced mgr/snap_schedule/dump_on_update true 2024-09-21T05:49:09.445 INFO:teuthology.orchestra.run.smithi117.stdout:mgr advanced mgr/snap_schedule/log_level debug 2024-09-21T05:49:09.445 INFO:teuthology.orchestra.run.smithi117.stdout:mgr advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T05:49:09.446 INFO:teuthology.orchestra.run.smithi117.stdout:mgr advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T05:49:09.446 INFO:teuthology.orchestra.run.smithi117.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T05:49:09.446 INFO:teuthology.orchestra.run.smithi117.stdout:osd dev bdev_debug_aio true 2024-09-21T05:49:09.446 INFO:teuthology.orchestra.run.smithi117.stdout:osd dev bluestore_block_size 96636764160 * 2024-09-21T05:49:09.446 INFO:teuthology.orchestra.run.smithi117.stdout:osd advanced bluestore_compression_mode aggressive 2024-09-21T05:49:09.446 INFO:teuthology.orchestra.run.smithi117.stdout:osd dev bluestore_fsck_on_mount true 2024-09-21T05:49:09.446 INFO:teuthology.orchestra.run.smithi117.stdout:osd advanced debug_bluefs 20/20 2024-09-21T05:49:09.446 INFO:teuthology.orchestra.run.smithi117.stdout:osd advanced debug_bluestore 20/20 2024-09-21T05:49:09.446 INFO:teuthology.orchestra.run.smithi117.stdout:osd advanced debug_ms 1/1 2024-09-21T05:49:09.446 INFO:teuthology.orchestra.run.smithi117.stdout:osd advanced debug_osd 20/20 2024-09-21T05:49:09.446 INFO:teuthology.orchestra.run.smithi117.stdout:osd advanced debug_rocksdb 10/10 2024-09-21T05:49:09.446 INFO:teuthology.orchestra.run.smithi117.stdout:osd dev osd_debug_misdirected_ops true 2024-09-21T05:49:09.446 INFO:teuthology.orchestra.run.smithi117.stdout:osd dev osd_debug_op_order true 2024-09-21T05:49:09.446 INFO:teuthology.orchestra.run.smithi117.stdout:osd dev osd_debug_pg_log_writeout true 2024-09-21T05:49:09.447 INFO:teuthology.orchestra.run.smithi117.stdout:osd dev osd_debug_shutdown true 2024-09-21T05:49:09.447 INFO:teuthology.orchestra.run.smithi117.stdout:osd dev osd_debug_verify_cached_snaps true 2024-09-21T05:49:09.447 INFO:teuthology.orchestra.run.smithi117.stdout:osd dev osd_debug_verify_missing_on_start true 2024-09-21T05:49:09.447 INFO:teuthology.orchestra.run.smithi117.stdout:osd dev osd_debug_verify_stray_on_activate true 2024-09-21T05:49:09.447 INFO:teuthology.orchestra.run.smithi117.stdout:osd advanced osd_deep_scrub_update_digest_min_age 30 2024-09-21T05:49:09.447 INFO:teuthology.orchestra.run.smithi117.stdout:osd advanced osd_failsafe_full_ratio 0.950000 2024-09-21T05:49:09.447 INFO:teuthology.orchestra.run.smithi117.stdout:osd advanced osd_max_omap_entries_per_request 10 2024-09-21T05:49:09.447 INFO:teuthology.orchestra.run.smithi117.stdout:osd advanced osd_mclock_profile high_recovery_ops 2024-09-21T05:49:09.447 INFO:teuthology.orchestra.run.smithi117.stdout:osd advanced osd_memory_target_autotune true 2024-09-21T05:49:09.447 INFO:teuthology.orchestra.run.smithi117.stdout:osd advanced osd_objectstore bluestore * 2024-09-21T05:49:09.447 INFO:teuthology.orchestra.run.smithi117.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T05:49:09.447 INFO:teuthology.orchestra.run.smithi117.stdout:osd advanced osd_op_queue debug_random * 2024-09-21T05:49:09.447 INFO:teuthology.orchestra.run.smithi117.stdout:osd advanced osd_op_queue_cut_off debug_random * 2024-09-21T05:49:09.447 INFO:teuthology.orchestra.run.smithi117.stdout:osd advanced osd_recover_clone_overlap true 2024-09-21T05:49:09.448 INFO:teuthology.orchestra.run.smithi117.stdout:osd advanced osd_recovery_max_chunk 1048576 2024-09-21T05:49:09.448 INFO:teuthology.orchestra.run.smithi117.stdout:osd advanced osd_scrub_load_threshold 5.000000 2024-09-21T05:49:09.448 INFO:teuthology.orchestra.run.smithi117.stdout:osd advanced osd_scrub_max_interval 600.000000 2024-09-21T05:49:09.448 INFO:teuthology.orchestra.run.smithi117.stdout:osd advanced osd_shutdown_pgref_assert true 2024-09-21T05:49:09.448 INFO:teuthology.orchestra.run.smithi117.stdout:mds advanced debug_mds 20/20 2024-09-21T05:49:09.448 INFO:teuthology.orchestra.run.smithi117.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T05:49:09.448 INFO:teuthology.orchestra.run.smithi117.stdout:mds advanced debug_ms 2/2 2024-09-21T05:49:09.448 INFO:teuthology.orchestra.run.smithi117.stdout:mds advanced mds_bal_fragment_size_max 10000 2024-09-21T05:49:09.448 INFO:teuthology.orchestra.run.smithi117.stdout:mds advanced mds_bal_merge_size 5 2024-09-21T05:49:09.448 INFO:teuthology.orchestra.run.smithi117.stdout:mds advanced mds_bal_replicate_threshold 1.000000 2024-09-21T05:49:09.448 INFO:teuthology.orchestra.run.smithi117.stdout:mds advanced mds_bal_split_bits 3 2024-09-21T05:49:09.448 INFO:teuthology.orchestra.run.smithi117.stdout:mds advanced mds_bal_split_size 100 2024-09-21T05:49:09.448 INFO:teuthology.orchestra.run.smithi117.stdout:mds dev mds_debug_frag true 2024-09-21T05:49:09.448 INFO:teuthology.orchestra.run.smithi117.stdout:mds dev mds_debug_scatterstat true 2024-09-21T05:49:09.448 INFO:teuthology.orchestra.run.smithi117.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T05:49:09.449 INFO:teuthology.orchestra.run.smithi117.stdout:mds dev mds_verify_scatter true 2024-09-21T05:49:09.449 INFO:teuthology.orchestra.run.smithi117.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T05:49:09.449 INFO:teuthology.orchestra.run.smithi117.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T05:49:09.449 INFO:teuthology.orchestra.run.smithi117.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T05:49:09.449 INFO:teuthology.orchestra.run.smithi117.stdout:client advanced client_mount_timeout 600 2024-09-21T05:49:09.449 INFO:teuthology.orchestra.run.smithi117.stdout:client advanced debug_client 20/20 2024-09-21T05:49:09.449 INFO:teuthology.orchestra.run.smithi117.stdout:client advanced debug_ms 1/1 2024-09-21T05:49:09.449 INFO:teuthology.orchestra.run.smithi117.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T05:49:09.449 INFO:teuthology.orchestra.run.smithi117.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T05:49:09.449 INFO:teuthology.orchestra.run.smithi117.stdout:client.rgw advanced rgw_cache_enabled true 2024-09-21T05:49:09.449 INFO:teuthology.orchestra.run.smithi117.stdout:client.rgw advanced rgw_enable_ops_log true 2024-09-21T05:49:09.449 INFO:teuthology.orchestra.run.smithi117.stdout:client.rgw advanced rgw_enable_usage_log true 2024-09-21T05:49:09.450 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.442+0000 7f40fb5c3640 1 -- 172.21.15.117:0/3780941486 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f40e00b64d0 msgr2=0x7f40e00b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:09.450 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.442+0000 7f40fb5c3640 1 --2- 172.21.15.117:0/3780941486 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f40e00b64d0 0x7f40e00b89e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f40f0005c40 tx=0x7f40f000a3b0 comp rx=0 tx=0).stop 2024-09-21T05:49:09.450 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.442+0000 7f40fb5c3640 1 -- 172.21.15.117:0/3780941486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40f41083d0 msgr2=0x7f40f4199610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:09.450 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.442+0000 7f40fb5c3640 1 --2- 172.21.15.117:0/3780941486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40f41083d0 0x7f40f4199610 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f40e4032fd0 tx=0x7f40e4004270 comp rx=0 tx=0).stop 2024-09-21T05:49:09.450 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.442+0000 7f40fb5c3640 1 -- 172.21.15.117:0/3780941486 shutdown_connections 2024-09-21T05:49:09.450 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.442+0000 7f40fb5c3640 1 --2- 172.21.15.117:0/3780941486 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f40e00b64d0 0x7f40e00b89e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:09.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.442+0000 7f40fb5c3640 1 --2- 172.21.15.117:0/3780941486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40f41083d0 0x7f40f4199610 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:09.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.442+0000 7f40fb5c3640 1 --2- 172.21.15.117:0/3780941486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40f4108d20 0x7f40f4199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:09.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.442+0000 7f40fb5c3640 1 --2- 172.21.15.117:0/3780941486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40f4102ae0 0x7f40f41990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:09.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.442+0000 7f40fb5c3640 1 -- 172.21.15.117:0/3780941486 >> 172.21.15.117:0/3780941486 conn(0x7f40f4079760 msgr2=0x7f40f4104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:49:09.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.442+0000 7f40fb5c3640 1 -- 172.21.15.117:0/3780941486 shutdown_connections 2024-09-21T05:49:09.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:09.442+0000 7f40fb5c3640 1 -- 172.21.15.117:0/3780941486 wait complete. 2024-09-21T05:49:09.526 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:09 smithi117 ceph-mon[30947]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:09.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:09 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3780941486' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T05:49:09.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:09 smithi192 ceph-mon[34700]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:09.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:09 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3780941486' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T05:49:09.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:09 smithi138 ceph-mon[34391]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:09.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:09 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3780941486' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T05:49:10.194 INFO:tasks.cephadm:Deploying OSDs... 2024-09-21T05:49:10.194 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-21T05:49:10.194 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T05:49:10.210 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T05:49:10.210 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_1 2024-09-21T05:49:10.266 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T05:49:10.267 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T05:49:10.267 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-21T05:49:10.267 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T05:49:10.267 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T05:49:10.267 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-09-21 05:47:47.080692165 +0000 2024-09-21T05:49:10.267 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-09-21 05:42:47.534734740 +0000 2024-09-21T05:49:10.267 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-09-21 05:42:47.534734740 +0000 2024-09-21T05:49:10.267 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-09-21 05:42:47.534734740 +0000 2024-09-21T05:49:10.267 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T05:49:10.330 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-09-21T05:49:10.331 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-09-21T05:49:10.331 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000273476 s, 1.9 MB/s 2024-09-21T05:49:10.332 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T05:49:10.388 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_2 2024-09-21T05:49:10.443 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T05:49:10.443 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T05:49:10.443 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-21T05:49:10.443 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T05:49:10.443 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T05:49:10.443 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-09-21 05:47:47.107691631 +0000 2024-09-21T05:49:10.443 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-09-21 05:42:47.856728380 +0000 2024-09-21T05:49:10.443 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-09-21 05:42:47.856728380 +0000 2024-09-21T05:49:10.444 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-09-21 05:42:47.856728380 +0000 2024-09-21T05:49:10.444 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T05:49:10.507 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-09-21T05:49:10.507 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-09-21T05:49:10.508 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000239378 s, 2.1 MB/s 2024-09-21T05:49:10.508 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T05:49:10.565 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_3 2024-09-21T05:49:10.620 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T05:49:10.620 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T05:49:10.620 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-21T05:49:10.620 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T05:49:10.620 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T05:49:10.620 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-09-21 05:47:47.132691137 +0000 2024-09-21T05:49:10.620 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-09-21 05:42:48.182721941 +0000 2024-09-21T05:49:10.620 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-09-21 05:42:48.182721941 +0000 2024-09-21T05:49:10.620 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-09-21 05:42:48.182721941 +0000 2024-09-21T05:49:10.621 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T05:49:10.683 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-09-21T05:49:10.683 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-09-21T05:49:10.683 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000240793 s, 2.1 MB/s 2024-09-21T05:49:10.684 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T05:49:10.740 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_4 2024-09-21T05:49:10.796 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T05:49:10.796 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T05:49:10.796 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-21T05:49:10.796 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T05:49:10.796 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T05:49:10.796 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-09-21 05:47:47.158690624 +0000 2024-09-21T05:49:10.796 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-09-21 05:42:48.508715502 +0000 2024-09-21T05:49:10.796 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-09-21 05:42:48.508715502 +0000 2024-09-21T05:49:10.797 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-09-21 05:42:48.508715502 +0000 2024-09-21T05:49:10.797 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T05:49:10.859 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-09-21T05:49:10.859 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-09-21T05:49:10.859 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000217325 s, 2.4 MB/s 2024-09-21T05:49:10.860 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T05:49:10.916 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-21T05:49:10.916 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T05:49:10.936 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T05:49:10.936 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_1 2024-09-21T05:49:10.995 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T05:49:10.995 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T05:49:10.995 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-21T05:49:10.995 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T05:49:10.995 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T05:49:10.995 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-09-21 05:48:40.325124342 +0000 2024-09-21T05:49:10.995 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-09-21 05:43:01.746610292 +0000 2024-09-21T05:49:10.995 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-09-21 05:43:01.746610292 +0000 2024-09-21T05:49:10.995 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-09-21 05:43:01.746610292 +0000 2024-09-21T05:49:10.995 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T05:49:11.065 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-09-21T05:49:11.065 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-09-21T05:49:11.065 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000322508 s, 1.6 MB/s 2024-09-21T05:49:11.066 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T05:49:11.125 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_2 2024-09-21T05:49:11.183 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T05:49:11.183 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T05:49:11.183 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-21T05:49:11.183 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T05:49:11.184 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T05:49:11.184 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-09-21 05:48:40.325124342 +0000 2024-09-21T05:49:11.184 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-09-21 05:43:02.072603125 +0000 2024-09-21T05:49:11.184 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-09-21 05:43:02.072603125 +0000 2024-09-21T05:49:11.184 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-09-21 05:43:02.072603125 +0000 2024-09-21T05:49:11.184 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T05:49:11.249 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-09-21T05:49:11.249 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-09-21T05:49:11.249 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000207224 s, 2.5 MB/s 2024-09-21T05:49:11.250 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T05:49:11.308 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_3 2024-09-21T05:49:11.364 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T05:49:11.364 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T05:49:11.364 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-21T05:49:11.364 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T05:49:11.364 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T05:49:11.365 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-09-21 05:48:40.326124321 +0000 2024-09-21T05:49:11.365 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-09-21 05:43:02.406595781 +0000 2024-09-21T05:49:11.365 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-09-21 05:43:02.406595781 +0000 2024-09-21T05:49:11.365 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-09-21 05:43:02.406595781 +0000 2024-09-21T05:49:11.365 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T05:49:11.429 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-09-21T05:49:11.429 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-09-21T05:49:11.429 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000213067 s, 2.4 MB/s 2024-09-21T05:49:11.430 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T05:49:11.487 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_4 2024-09-21T05:49:11.544 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T05:49:11.544 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T05:49:11.545 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-21T05:49:11.545 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T05:49:11.545 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T05:49:11.545 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-09-21 05:48:40.326124321 +0000 2024-09-21T05:49:11.545 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-09-21 05:43:02.727588723 +0000 2024-09-21T05:49:11.545 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-09-21 05:43:02.727588723 +0000 2024-09-21T05:49:11.545 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-09-21 05:43:02.727588723 +0000 2024-09-21T05:49:11.545 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T05:49:11.608 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-09-21T05:49:11.608 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-09-21T05:49:11.608 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000202319 s, 2.5 MB/s 2024-09-21T05:49:11.610 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T05:49:11.666 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-21T05:49:11.666 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T05:49:11.686 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T05:49:11.686 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_1 2024-09-21T05:49:11.729 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:11 smithi192 ceph-mon[34700]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:11.744 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T05:49:11.744 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T05:49:11.744 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-21T05:49:11.744 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T05:49:11.744 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T05:49:11.744 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-09-21 05:48:38.093786837 +0000 2024-09-21T05:49:11.744 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-09-21 05:43:00.280800813 +0000 2024-09-21T05:49:11.744 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-09-21 05:43:00.280800813 +0000 2024-09-21T05:49:11.744 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-09-21 05:43:00.280800813 +0000 2024-09-21T05:49:11.745 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T05:49:11.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:11 smithi117 ceph-mon[30947]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:11.813 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-09-21T05:49:11.814 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-09-21T05:49:11.814 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000262346 s, 2.0 MB/s 2024-09-21T05:49:11.815 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T05:49:11.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:11 smithi138 ceph-mon[34391]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:11.874 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_2 2024-09-21T05:49:11.931 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T05:49:11.931 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T05:49:11.931 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-21T05:49:11.931 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T05:49:11.931 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T05:49:11.931 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-09-21 05:48:38.093786837 +0000 2024-09-21T05:49:11.931 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-09-21 05:43:00.614794619 +0000 2024-09-21T05:49:11.931 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-09-21 05:43:00.614794619 +0000 2024-09-21T05:49:11.931 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-09-21 05:43:00.614794619 +0000 2024-09-21T05:49:11.931 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T05:49:11.996 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-09-21T05:49:11.996 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-09-21T05:49:11.996 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000229978 s, 2.2 MB/s 2024-09-21T05:49:11.997 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T05:49:12.055 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_3 2024-09-21T05:49:12.112 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T05:49:12.112 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T05:49:12.112 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-21T05:49:12.112 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T05:49:12.112 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T05:49:12.112 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-09-21 05:48:38.094786818 +0000 2024-09-21T05:49:12.112 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-09-21 05:43:00.936788648 +0000 2024-09-21T05:49:12.112 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-09-21 05:43:00.936788648 +0000 2024-09-21T05:49:12.112 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-09-21 05:43:00.936788648 +0000 2024-09-21T05:49:12.113 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T05:49:12.176 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-09-21T05:49:12.176 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-09-21T05:49:12.176 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000215901 s, 2.4 MB/s 2024-09-21T05:49:12.177 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T05:49:12.234 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_4 2024-09-21T05:49:12.289 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T05:49:12.289 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T05:49:12.290 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-21T05:49:12.290 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T05:49:12.290 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T05:49:12.290 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-09-21 05:48:38.094786818 +0000 2024-09-21T05:49:12.290 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-09-21 05:43:01.285782177 +0000 2024-09-21T05:49:12.290 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-09-21 05:43:01.285782177 +0000 2024-09-21T05:49:12.290 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-09-21 05:43:01.285782177 +0000 2024-09-21T05:49:12.290 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T05:49:12.354 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-09-21T05:49:12.354 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-09-21T05:49:12.354 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000219238 s, 2.3 MB/s 2024-09-21T05:49:12.355 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T05:49:12.412 INFO:tasks.cephadm:Deploying osd.0 on smithi117 with /dev/vg_nvme/lv_4... 2024-09-21T05:49:12.412 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T05:49:12.663 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:49:13.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:13 smithi192 ceph-mon[34700]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:13.737 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:13 smithi117 ceph-mon[30947]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:13.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:13 smithi138 ceph-mon[34391]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:15.181 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:49:15.207 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi117:vg_nvme/lv_4 2024-09-21T05:49:15.452 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:49:15.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:15 smithi192 ceph-mon[34700]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:15.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:15 smithi117 ceph-mon[30947]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:15.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:15 smithi138 ceph-mon[34391]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:15.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.914+0000 7f3682c73640 1 -- 172.21.15.117:0/4222432767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f367c108d20 msgr2=0x7f367c10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:15.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.914+0000 7f3682c73640 1 --2- 172.21.15.117:0/4222432767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f367c108d20 0x7f367c10f550 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3674009ca0 tx=0x7f367402a8c0 comp rx=0 tx=0).stop 2024-09-21T05:49:15.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.914+0000 7f3682c73640 1 -- 172.21.15.117:0/4222432767 shutdown_connections 2024-09-21T05:49:15.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.914+0000 7f3682c73640 1 --2- 172.21.15.117:0/4222432767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f367c108d20 0x7f367c10f550 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:15.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.914+0000 7f3682c73640 1 --2- 172.21.15.117:0/4222432767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f367c1083d0 0x7f367c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:15.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.914+0000 7f3682c73640 1 --2- 172.21.15.117:0/4222432767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f367c102ae0 0x7f367c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:15.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.914+0000 7f3682c73640 1 -- 172.21.15.117:0/4222432767 >> 172.21.15.117:0/4222432767 conn(0x7f367c079760 msgr2=0x7f367c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:49:15.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.914+0000 7f3682c73640 1 -- 172.21.15.117:0/4222432767 shutdown_connections 2024-09-21T05:49:15.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.914+0000 7f3682c73640 1 -- 172.21.15.117:0/4222432767 wait complete. 2024-09-21T05:49:15.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.914+0000 7f3682c73640 1 Processor -- start 2024-09-21T05:49:15.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.915+0000 7f3682c73640 1 -- start start 2024-09-21T05:49:15.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.915+0000 7f3682c73640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f367c102ae0 0x7f367c199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:15.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.915+0000 7f3682c73640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f367c1083d0 0x7f367c1998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:15.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.915+0000 7f3682c73640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f367c108d20 0x7f367c19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:15.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.915+0000 7f3682c73640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f367c112990 con 0x7f367c108d20 2024-09-21T05:49:15.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.915+0000 7f3682c73640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f367c112810 con 0x7f367c102ae0 2024-09-21T05:49:15.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.915+0000 7f3682c73640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f367c112b10 con 0x7f367c1083d0 2024-09-21T05:49:15.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.915+0000 7f3680a0f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f367c102ae0 0x7f367c199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:15.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.915+0000 7f3681210640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f367c108d20 0x7f367c19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:15.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.915+0000 7f3680a0f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f367c102ae0 0x7f367c199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:34768/0 (socket says 172.21.15.117:34768) 2024-09-21T05:49:15.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.915+0000 7f3680a0f640 1 -- 172.21.15.117:0/2767326450 learned_addr learned my addr 172.21.15.117:0/2767326450 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:49:15.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.915+0000 7f367bfff640 1 --2- 172.21.15.117:0/2767326450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f367c1083d0 0x7f367c1998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:15.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.915+0000 7f3680a0f640 1 -- 172.21.15.117:0/2767326450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f367c1083d0 msgr2=0x7f367c1998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:15.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.915+0000 7f3680a0f640 1 --2- 172.21.15.117:0/2767326450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f367c1083d0 0x7f367c1998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:15.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.915+0000 7f3680a0f640 1 -- 172.21.15.117:0/2767326450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f367c108d20 msgr2=0x7f367c19eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:15.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.915+0000 7f3680a0f640 1 --2- 172.21.15.117:0/2767326450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f367c108d20 0x7f367c19eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:15.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.915+0000 7f3680a0f640 1 -- 172.21.15.117:0/2767326450 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f367c19f2b0 con 0x7f367c102ae0 2024-09-21T05:49:15.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.915+0000 7f3681210640 1 --2- 172.21.15.117:0/2767326450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f367c108d20 0x7f367c19eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:49:15.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.916+0000 7f3680a0f640 1 --2- 172.21.15.117:0/2767326450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f367c102ae0 0x7f367c199340 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f366c00d950 tx=0x7f366c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:49:15.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.916+0000 7f3679ffb640 1 -- 172.21.15.117:0/2767326450 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366c017030 con 0x7f367c102ae0 2024-09-21T05:49:15.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.916+0000 7f3682c73640 1 -- 172.21.15.117:0/2767326450 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f367c19f5a0 con 0x7f367c102ae0 2024-09-21T05:49:15.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.916+0000 7f3679ffb640 1 -- 172.21.15.117:0/2767326450 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f366c004020 con 0x7f367c102ae0 2024-09-21T05:49:15.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.916+0000 7f3679ffb640 1 -- 172.21.15.117:0/2767326450 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366c0047a0 con 0x7f367c102ae0 2024-09-21T05:49:15.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.916+0000 7f3682c73640 1 -- 172.21.15.117:0/2767326450 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f367c1a5d00 con 0x7f367c102ae0 2024-09-21T05:49:15.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.917+0000 7f3682c73640 1 -- 172.21.15.117:0/2767326450 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f367c103d10 con 0x7f367c102ae0 2024-09-21T05:49:15.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.918+0000 7f3679ffb640 1 -- 172.21.15.117:0/2767326450 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7f366c009a30 con 0x7f367c102ae0 2024-09-21T05:49:15.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.919+0000 7f3679ffb640 1 --2- 172.21.15.117:0/2767326450 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f36640b64d0 0x7f36640b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:15.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.919+0000 7f3679ffb640 1 -- 172.21.15.117:0/2767326450 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f366c0e5250 con 0x7f367c102ae0 2024-09-21T05:49:15.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.919+0000 7f367bfff640 1 --2- 172.21.15.117:0/2767326450 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f36640b64d0 0x7f36640b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:15.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.919+0000 7f367bfff640 1 --2- 172.21.15.117:0/2767326450 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f36640b64d0 0x7f36640b89e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f367c105f20 tx=0x7f367000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:49:15.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:15.924+0000 7f3679ffb640 1 -- 172.21.15.117:0/2767326450 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f366c0ad640 con 0x7f367c102ae0 2024-09-21T05:49:16.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:16.108+0000 7f3682c73640 1 -- 172.21.15.117:0/2767326450 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f367c063800 con 0x7f36640b64d0 2024-09-21T05:49:16.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:16 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:49:16.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:16 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:49:16.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:16 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:49:16.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:16 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:49:16.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:16 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:49:16.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:16 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:49:16.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:16 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:49:16.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:16 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:49:16.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:16 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:49:17.722 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:17 smithi117 ceph-mon[30947]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:49:17.722 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:17 smithi117 ceph-mon[30947]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:17.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:17 smithi192 ceph-mon[34700]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:49:17.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:17 smithi192 ceph-mon[34700]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:17.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:17 smithi138 ceph-mon[34391]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:49:17.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:17 smithi138 ceph-mon[34391]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:18.761 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:18 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2230866260' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "68315502-fa6c-4a6e-a2d4-a7dde8071c02"} : dispatch 2024-09-21T05:49:18.761 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:18 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2230866260' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68315502-fa6c-4a6e-a2d4-a7dde8071c02"}]': finished 2024-09-21T05:49:18.761 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:18 smithi117 ceph-mon[30947]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T05:49:18.762 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:18 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T05:49:18.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:18 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2230866260' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "68315502-fa6c-4a6e-a2d4-a7dde8071c02"} : dispatch 2024-09-21T05:49:18.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:18 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2230866260' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68315502-fa6c-4a6e-a2d4-a7dde8071c02"}]': finished 2024-09-21T05:49:18.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:18 smithi138 ceph-mon[34391]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T05:49:18.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:18 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T05:49:18.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:18 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2230866260' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "68315502-fa6c-4a6e-a2d4-a7dde8071c02"} : dispatch 2024-09-21T05:49:18.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:18 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2230866260' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68315502-fa6c-4a6e-a2d4-a7dde8071c02"}]': finished 2024-09-21T05:49:18.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:18 smithi192 ceph-mon[34700]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T05:49:18.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:18 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T05:49:19.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:19 smithi117 ceph-mon[30947]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:19.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:19 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/485690245' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:49:19.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:19 smithi138 ceph-mon[34391]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:19.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:19 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/485690245' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:49:19.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:19 smithi192 ceph-mon[34700]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:19.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:19 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/485690245' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:49:21.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:21 smithi117 ceph-mon[30947]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:21.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:21 smithi138 ceph-mon[34391]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:21.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:21 smithi192 ceph-mon[34700]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:23.629 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:23 smithi117 ceph-mon[30947]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:23.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:23 smithi138 ceph-mon[34391]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:23.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:23 smithi192 ceph-mon[34700]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:25.615 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:25 smithi117 ceph-mon[30947]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:25.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:25 smithi138 ceph-mon[34391]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:25.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:25 smithi192 ceph-mon[34700]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:27.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:27 smithi117 ceph-mon[30947]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:27.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:27 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T05:49:27.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:27 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:49:27.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:27 smithi138 ceph-mon[34391]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:27.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:27 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T05:49:27.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:27 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:49:27.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:27 smithi192 ceph-mon[34700]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:27.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:27 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T05:49:27.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:27 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:49:28.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:28 smithi117 ceph-mon[30947]: Deploying daemon osd.0 on smithi117 2024-09-21T05:49:28.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:28 smithi138 ceph-mon[34391]: Deploying daemon osd.0 on smithi117 2024-09-21T05:49:28.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:28 smithi192 ceph-mon[34700]: Deploying daemon osd.0 on smithi117 2024-09-21T05:49:29.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:29 smithi117 ceph-mon[30947]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:29.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:29 smithi138 ceph-mon[34391]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:29.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:29 smithi192 ceph-mon[34700]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:31.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:31 smithi117 ceph-mon[30947]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:31.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:31 smithi138 ceph-mon[34391]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:31.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:31 smithi192 ceph-mon[34700]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:33.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:33 smithi117 ceph-mon[30947]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:33.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:33 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:49:33.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:33 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:33.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:33 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:33.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:33 smithi138 ceph-mon[34391]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:33.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:33 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:49:33.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:33 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:33.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:33 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:33.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:33 smithi192 ceph-mon[34700]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:33.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:33 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:49:33.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:33 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:33.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:33 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:35.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:35 smithi117 ceph-mon[30947]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:35.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:35 smithi138 ceph-mon[34391]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:35.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:35 smithi192 ceph-mon[34700]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:37.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:37.000+0000 7f3679ffb640 1 -- 172.21.15.117:0/2767326450 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f367c063800 con 0x7f36640b64d0 2024-09-21T05:49:37.002 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 0 on host 'smithi117' 2024-09-21T05:49:37.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:37.002+0000 7f3682c73640 1 -- 172.21.15.117:0/2767326450 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f36640b64d0 msgr2=0x7f36640b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:37.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:37.003+0000 7f3682c73640 1 --2- 172.21.15.117:0/2767326450 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f36640b64d0 0x7f36640b89e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f367c105f20 tx=0x7f367000b390 comp rx=0 tx=0).stop 2024-09-21T05:49:37.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:37.003+0000 7f3682c73640 1 -- 172.21.15.117:0/2767326450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f367c102ae0 msgr2=0x7f367c199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:37.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:37.003+0000 7f3682c73640 1 --2- 172.21.15.117:0/2767326450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f367c102ae0 0x7f367c199340 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f366c00d950 tx=0x7f366c00de10 comp rx=0 tx=0).stop 2024-09-21T05:49:37.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:37.003+0000 7f3682c73640 1 -- 172.21.15.117:0/2767326450 shutdown_connections 2024-09-21T05:49:37.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:37.003+0000 7f3682c73640 1 --2- 172.21.15.117:0/2767326450 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f36640b64d0 0x7f36640b89e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:37.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:37.003+0000 7f3682c73640 1 --2- 172.21.15.117:0/2767326450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f367c108d20 0x7f367c19eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:37.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:37.003+0000 7f3682c73640 1 --2- 172.21.15.117:0/2767326450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f367c1083d0 0x7f367c1998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:37.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:37.003+0000 7f3682c73640 1 --2- 172.21.15.117:0/2767326450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f367c102ae0 0x7f367c199340 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:37.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:37.003+0000 7f3682c73640 1 -- 172.21.15.117:0/2767326450 >> 172.21.15.117:0/2767326450 conn(0x7f367c079760 msgr2=0x7f367c104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:49:37.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:37.003+0000 7f3682c73640 1 -- 172.21.15.117:0/2767326450 shutdown_connections 2024-09-21T05:49:37.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:49:37.003+0000 7f3682c73640 1 -- 172.21.15.117:0/2767326450 wait complete. 2024-09-21T05:49:37.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:37 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:37.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:37 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:37.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:37 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:49:37.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:37 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:49:37.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:37 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:37.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:37 smithi117 ceph-mon[30947]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:37.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:37 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:49:37.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:37 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:37.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:37 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:37.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:37 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:37.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:37 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:49:37.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:37 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:49:37.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:37 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:37.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:37 smithi138 ceph-mon[34391]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:37.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:37 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:49:37.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:37 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:37.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:37 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:37.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:37 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:37.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:37 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:49:37.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:37 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:49:37.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:37 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:37.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:37 smithi192 ceph-mon[34700]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:37.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:37 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:49:37.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:37 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:37.557 DEBUG:teuthology.orchestra.run.smithi117:osd.0> sudo journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.0.service 2024-09-21T05:49:37.560 INFO:tasks.cephadm:Deploying osd.1 on smithi138 with /dev/vg_nvme/lv_4... 2024-09-21T05:49:37.560 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T05:49:37.799 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T05:49:38.239 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:38 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:38.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:38 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:38.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:38 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:39.091 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:39 smithi138 ceph-mon[34391]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:39.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:39 smithi117 ceph-mon[30947]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:39.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:39 smithi192 ceph-mon[34700]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:40.010 INFO:journalctl@ceph.osd.0.smithi117.stdout:Sep 21 05:49:39 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-0[47501]: 2024-09-21T05:49:39.766+0000 7f258bb5c640 -1 osd.0 0 log_to_monitors true 2024-09-21T05:49:40.135 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-21T05:49:40.163 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi138:vg_nvme/lv_4 2024-09-21T05:49:40.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:40 smithi117 ceph-mon[30947]: from='osd.0 [v2:172.21.15.117:6802/3761478035,v1:172.21.15.117:6803/3761478035]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T05:49:40.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:40 smithi138 ceph-mon[34391]: from='osd.0 [v2:172.21.15.117:6802/3761478035,v1:172.21.15.117:6803/3761478035]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T05:49:40.413 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T05:49:40.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:40 smithi192 ceph-mon[34700]: from='osd.0 [v2:172.21.15.117:6802/3761478035,v1:172.21.15.117:6803/3761478035]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T05:49:40.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.954+0000 7fb578102640 1 -- 172.21.15.138:0/1148273197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb570103bc0 msgr2=0x7fb570103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:40.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.954+0000 7fb578102640 1 --2- 172.21.15.138:0/1148273197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb570103bc0 0x7fb570103fd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb56c009900 tx=0x7fb56c02a410 comp rx=0 tx=0).stop 2024-09-21T05:49:40.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.954+0000 7fb578102640 1 -- 172.21.15.138:0/1148273197 shutdown_connections 2024-09-21T05:49:40.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.954+0000 7fb578102640 1 --2- 172.21.15.138:0/1148273197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb570109e00 0x7fb570110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:40.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.954+0000 7fb578102640 1 --2- 172.21.15.138:0/1148273197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5701094b0 0x7fb5701098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:40.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.954+0000 7fb578102640 1 --2- 172.21.15.138:0/1148273197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb570103bc0 0x7fb570103fd0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:40.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.954+0000 7fb578102640 1 -- 172.21.15.138:0/1148273197 >> 172.21.15.138:0/1148273197 conn(0x7fb57007b910 msgr2=0x7fb57007bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:49:40.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.954+0000 7fb578102640 1 -- 172.21.15.138:0/1148273197 shutdown_connections 2024-09-21T05:49:40.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.954+0000 7fb578102640 1 -- 172.21.15.138:0/1148273197 wait complete. 2024-09-21T05:49:40.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.955+0000 7fb578102640 1 Processor -- start 2024-09-21T05:49:40.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.955+0000 7fb578102640 1 -- start start 2024-09-21T05:49:40.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.955+0000 7fb578102640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb570103bc0 0x7fb570075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:40.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.955+0000 7fb578102640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5701094b0 0x7fb570075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:40.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.955+0000 7fb578102640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb570109e00 0x7fb570076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:40.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.955+0000 7fb578102640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb570113ac0 con 0x7fb570109e00 2024-09-21T05:49:40.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.955+0000 7fb578102640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb570113940 con 0x7fb5701094b0 2024-09-21T05:49:40.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.955+0000 7fb578102640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb570113c40 con 0x7fb570103bc0 2024-09-21T05:49:40.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.955+0000 7fb57569d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5701094b0 0x7fb570075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:40.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.955+0000 7fb57569d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5701094b0 0x7fb570075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40968/0 (socket says 172.21.15.138:40968) 2024-09-21T05:49:40.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.955+0000 7fb57569d640 1 -- 172.21.15.138:0/73348330 learned_addr learned my addr 172.21.15.138:0/73348330 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-09-21T05:49:40.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.955+0000 7fb57669f640 1 --2- 172.21.15.138:0/73348330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb570109e00 0x7fb570076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:40.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.955+0000 7fb575e9e640 1 --2- 172.21.15.138:0/73348330 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb570103bc0 0x7fb570075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:40.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.956+0000 7fb57569d640 1 -- 172.21.15.138:0/73348330 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb570103bc0 msgr2=0x7fb570075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:40.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.956+0000 7fb57569d640 1 --2- 172.21.15.138:0/73348330 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb570103bc0 0x7fb570075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:40.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.956+0000 7fb57569d640 1 -- 172.21.15.138:0/73348330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb570109e00 msgr2=0x7fb570076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:40.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.956+0000 7fb57569d640 1 --2- 172.21.15.138:0/73348330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb570109e00 0x7fb570076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:40.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.956+0000 7fb57569d640 1 -- 172.21.15.138:0/73348330 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5701a8770 con 0x7fb5701094b0 2024-09-21T05:49:40.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.956+0000 7fb57569d640 1 --2- 172.21.15.138:0/73348330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5701094b0 0x7fb570075c50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb56000d740 tx=0x7fb56000dc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:49:40.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.956+0000 7fb566ffd640 1 -- 172.21.15.138:0/73348330 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb560017070 con 0x7fb5701094b0 2024-09-21T05:49:40.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.956+0000 7fb566ffd640 1 -- 172.21.15.138:0/73348330 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb56000de50 con 0x7fb5701094b0 2024-09-21T05:49:40.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.957+0000 7fb566ffd640 1 -- 172.21.15.138:0/73348330 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56000f620 con 0x7fb5701094b0 2024-09-21T05:49:40.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.957+0000 7fb578102640 1 -- 172.21.15.138:0/73348330 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5701a8a60 con 0x7fb5701094b0 2024-09-21T05:49:40.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.957+0000 7fb578102640 1 -- 172.21.15.138:0/73348330 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5701a8ff0 con 0x7fb5701094b0 2024-09-21T05:49:40.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.958+0000 7fb578102640 1 -- 172.21.15.138:0/73348330 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb570103fd0 con 0x7fb5701094b0 2024-09-21T05:49:40.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.961+0000 7fb566ffd640 1 -- 172.21.15.138:0/73348330 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7fb56001a070 con 0x7fb5701094b0 2024-09-21T05:49:40.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.961+0000 7fb566ffd640 1 --2- 172.21.15.138:0/73348330 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb55c0b6550 0x7fb55c0b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:40.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.961+0000 7fb566ffd640 1 -- 172.21.15.138:0/73348330 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1576+0+0 (secure 0 0 0) 0x7fb5600e50e0 con 0x7fb5701094b0 2024-09-21T05:49:40.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.962+0000 7fb575e9e640 1 --2- 172.21.15.138:0/73348330 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb55c0b6550 0x7fb55c0b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:40.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.962+0000 7fb575e9e640 1 --2- 172.21.15.138:0/73348330 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb55c0b6550 0x7fb55c0b8a60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb56c000c00 tx=0x7fb56c031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:49:40.966 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:40.965+0000 7fb566ffd640 1 -- 172.21.15.138:0/73348330 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb5600b1900 con 0x7fb5701094b0 2024-09-21T05:49:41.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:49:41.163+0000 7fb578102640 1 -- 172.21.15.138:0/73348330 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fb570063800 con 0x7fb55c0b6550 2024-09-21T05:49:41.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:41 smithi117 ceph-mon[30947]: from='osd.0 [v2:172.21.15.117:6802/3761478035,v1:172.21.15.117:6803/3761478035]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T05:49:41.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:41 smithi117 ceph-mon[30947]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T05:49:41.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:41 smithi117 ceph-mon[30947]: from='osd.0 [v2:172.21.15.117:6802/3761478035,v1:172.21.15.117:6803/3761478035]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-21T05:49:41.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:41 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T05:49:41.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:41 smithi117 ceph-mon[30947]: Detected new or changed devices on smithi117 2024-09-21T05:49:41.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:41 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:41.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:41 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:41.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:41 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T05:49:41.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:41 smithi117 ceph-mon[30947]: Adjusting osd_memory_target on smithi117 to 17088M 2024-09-21T05:49:41.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:41 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:41.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:41 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:49:41.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:41 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:49:41.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:41 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:41.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:41 smithi117 ceph-mon[30947]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:41.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:41 smithi138 ceph-mon[34391]: from='osd.0 [v2:172.21.15.117:6802/3761478035,v1:172.21.15.117:6803/3761478035]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T05:49:41.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:41 smithi138 ceph-mon[34391]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T05:49:41.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:41 smithi138 ceph-mon[34391]: from='osd.0 [v2:172.21.15.117:6802/3761478035,v1:172.21.15.117:6803/3761478035]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-21T05:49:41.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:41 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T05:49:41.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:41 smithi138 ceph-mon[34391]: Detected new or changed devices on smithi117 2024-09-21T05:49:41.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:41 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:41.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:41 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:41.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:41 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T05:49:41.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:41 smithi138 ceph-mon[34391]: Adjusting osd_memory_target on smithi117 to 17088M 2024-09-21T05:49:41.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:41 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:41.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:41 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:49:41.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:41 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:49:41.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:41 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:41.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:41 smithi138 ceph-mon[34391]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:41.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:41 smithi192 ceph-mon[34700]: from='osd.0 [v2:172.21.15.117:6802/3761478035,v1:172.21.15.117:6803/3761478035]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T05:49:41.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:41 smithi192 ceph-mon[34700]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T05:49:41.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:41 smithi192 ceph-mon[34700]: from='osd.0 [v2:172.21.15.117:6802/3761478035,v1:172.21.15.117:6803/3761478035]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-21T05:49:41.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:41 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T05:49:41.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:41 smithi192 ceph-mon[34700]: Detected new or changed devices on smithi117 2024-09-21T05:49:41.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:41 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:41.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:41 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:41.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:41 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T05:49:41.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:41 smithi192 ceph-mon[34700]: Adjusting osd_memory_target on smithi117 to 17088M 2024-09-21T05:49:41.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:41 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:41.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:41 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:49:41.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:41 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:49:41.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:41 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:41.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:41 smithi192 ceph-mon[34700]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:41.777 INFO:journalctl@ceph.osd.0.smithi117.stdout:Sep 21 05:49:41 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-0[47501]: 2024-09-21T05:49:41.449+0000 7f2587ade640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T05:49:41.777 INFO:journalctl@ceph.osd.0.smithi117.stdout:Sep 21 05:49:41 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-0[47501]: 2024-09-21T05:49:41.453+0000 7f257f8d4640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T05:49:42.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:42 smithi138 ceph-mon[34391]: from='osd.0 [v2:172.21.15.117:6802/3761478035,v1:172.21.15.117:6803/3761478035]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-21T05:49:42.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:42 smithi138 ceph-mon[34391]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T05:49:42.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:42 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T05:49:42.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:42 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T05:49:42.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:42 smithi138 ceph-mon[34391]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:49:42.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:42 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:49:42.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:42 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:49:42.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:42 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:49:42.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:42 smithi138 ceph-mon[34391]: from='osd.0 [v2:172.21.15.117:6802/3761478035,v1:172.21.15.117:6803/3761478035]' entity='osd.0' 2024-09-21T05:49:42.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:42 smithi117 ceph-mon[30947]: from='osd.0 [v2:172.21.15.117:6802/3761478035,v1:172.21.15.117:6803/3761478035]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-21T05:49:42.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:42 smithi117 ceph-mon[30947]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T05:49:42.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:42 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T05:49:42.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:42 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T05:49:42.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:42 smithi117 ceph-mon[30947]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:49:42.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:42 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:49:42.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:42 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:49:42.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:42 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:49:42.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:42 smithi117 ceph-mon[30947]: from='osd.0 [v2:172.21.15.117:6802/3761478035,v1:172.21.15.117:6803/3761478035]' entity='osd.0' 2024-09-21T05:49:42.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:42 smithi192 ceph-mon[34700]: from='osd.0 [v2:172.21.15.117:6802/3761478035,v1:172.21.15.117:6803/3761478035]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-21T05:49:42.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:42 smithi192 ceph-mon[34700]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T05:49:42.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:42 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T05:49:42.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:42 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T05:49:42.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:42 smithi192 ceph-mon[34700]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:49:42.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:42 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:49:42.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:42 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:49:42.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:42 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:49:42.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:42 smithi192 ceph-mon[34700]: from='osd.0 [v2:172.21.15.117:6802/3761478035,v1:172.21.15.117:6803/3761478035]' entity='osd.0' 2024-09-21T05:49:43.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:43 smithi138 ceph-mon[34391]: purged_snaps scrub starts 2024-09-21T05:49:43.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:43 smithi138 ceph-mon[34391]: purged_snaps scrub ok 2024-09-21T05:49:43.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:43 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T05:49:43.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:43 smithi138 ceph-mon[34391]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:43.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:43 smithi138 ceph-mon[34391]: osd.0 [v2:172.21.15.117:6802/3761478035,v1:172.21.15.117:6803/3761478035] boot 2024-09-21T05:49:43.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:43 smithi138 ceph-mon[34391]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T05:49:43.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:43 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T05:49:43.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:43 smithi192 ceph-mon[34700]: purged_snaps scrub starts 2024-09-21T05:49:43.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:43 smithi192 ceph-mon[34700]: purged_snaps scrub ok 2024-09-21T05:49:43.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:43 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T05:49:43.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:43 smithi192 ceph-mon[34700]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:43.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:43 smithi192 ceph-mon[34700]: osd.0 [v2:172.21.15.117:6802/3761478035,v1:172.21.15.117:6803/3761478035] boot 2024-09-21T05:49:43.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:43 smithi192 ceph-mon[34700]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T05:49:43.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:43 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T05:49:43.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:43 smithi117 ceph-mon[30947]: purged_snaps scrub starts 2024-09-21T05:49:43.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:43 smithi117 ceph-mon[30947]: purged_snaps scrub ok 2024-09-21T05:49:43.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:43 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T05:49:43.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:43 smithi117 ceph-mon[30947]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T05:49:43.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:43 smithi117 ceph-mon[30947]: osd.0 [v2:172.21.15.117:6802/3761478035,v1:172.21.15.117:6803/3761478035] boot 2024-09-21T05:49:43.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:43 smithi117 ceph-mon[30947]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T05:49:43.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:43 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T05:49:44.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:44 smithi192 ceph-mon[34700]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T05:49:44.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:44 smithi192 ceph-mon[34700]: from='client.? 172.21.15.138:0/2215360576' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d8910a08-4ef3-49f2-a4db-e3588d85d12b"} : dispatch 2024-09-21T05:49:44.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:44 smithi192 ceph-mon[34700]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d8910a08-4ef3-49f2-a4db-e3588d85d12b"} : dispatch 2024-09-21T05:49:44.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:44 smithi192 ceph-mon[34700]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d8910a08-4ef3-49f2-a4db-e3588d85d12b"}]': finished 2024-09-21T05:49:44.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:44 smithi192 ceph-mon[34700]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T05:49:44.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:44 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T05:49:44.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:44 smithi192 ceph-mon[34700]: from='client.? 172.21.15.138:0/2338282446' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:49:44.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:44 smithi117 ceph-mon[30947]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T05:49:44.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:44 smithi117 ceph-mon[30947]: from='client.? 172.21.15.138:0/2215360576' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d8910a08-4ef3-49f2-a4db-e3588d85d12b"} : dispatch 2024-09-21T05:49:44.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:44 smithi117 ceph-mon[30947]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d8910a08-4ef3-49f2-a4db-e3588d85d12b"} : dispatch 2024-09-21T05:49:44.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:44 smithi117 ceph-mon[30947]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d8910a08-4ef3-49f2-a4db-e3588d85d12b"}]': finished 2024-09-21T05:49:44.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:44 smithi117 ceph-mon[30947]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T05:49:44.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:44 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T05:49:44.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:44 smithi117 ceph-mon[30947]: from='client.? 172.21.15.138:0/2338282446' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:49:44.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:44 smithi138 ceph-mon[34391]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T05:49:44.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:44 smithi138 ceph-mon[34391]: from='client.? 172.21.15.138:0/2215360576' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d8910a08-4ef3-49f2-a4db-e3588d85d12b"} : dispatch 2024-09-21T05:49:44.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:44 smithi138 ceph-mon[34391]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d8910a08-4ef3-49f2-a4db-e3588d85d12b"} : dispatch 2024-09-21T05:49:44.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:44 smithi138 ceph-mon[34391]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d8910a08-4ef3-49f2-a4db-e3588d85d12b"}]': finished 2024-09-21T05:49:44.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:44 smithi138 ceph-mon[34391]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T05:49:44.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:44 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T05:49:44.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:44 smithi138 ceph-mon[34391]: from='client.? 172.21.15.138:0/2338282446' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:49:45.767 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:45 smithi138 ceph-mon[34391]: pgmap v35: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:49:45.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:45 smithi117 ceph-mon[30947]: pgmap v35: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:49:45.902 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:45 smithi192 ceph-mon[34700]: pgmap v35: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:49:47.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:47 smithi117 ceph-mon[30947]: pgmap v36: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:49:47.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:47 smithi138 ceph-mon[34391]: pgmap v36: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:49:47.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:47 smithi192 ceph-mon[34700]: pgmap v36: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:49:49.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:49 smithi138 ceph-mon[34391]: pgmap v37: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:49:49.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:49 smithi117 ceph-mon[30947]: pgmap v37: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:49:49.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:49 smithi192 ceph-mon[34700]: pgmap v37: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:49:51.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:51 smithi117 ceph-mon[30947]: pgmap v38: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:49:51.779 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:51 smithi138 ceph-mon[34391]: pgmap v38: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:49:51.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:51 smithi192 ceph-mon[34700]: pgmap v38: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:49:52.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:52 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T05:49:52.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:52 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:49:52.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:52 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T05:49:52.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:52 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:49:52.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:52 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T05:49:52.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:52 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:49:53.678 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:53 smithi138 ceph-mon[34391]: Deploying daemon osd.1 on smithi138 2024-09-21T05:49:53.678 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:53 smithi138 ceph-mon[34391]: pgmap v39: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:49:53.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:53 smithi117 ceph-mon[30947]: Deploying daemon osd.1 on smithi138 2024-09-21T05:49:53.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:53 smithi117 ceph-mon[30947]: pgmap v39: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:49:53.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:53 smithi192 ceph-mon[34700]: Deploying daemon osd.1 on smithi138 2024-09-21T05:49:53.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:53 smithi192 ceph-mon[34700]: pgmap v39: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:49:55.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:55 smithi117 ceph-mon[30947]: pgmap v40: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:49:55.816 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:55 smithi138 ceph-mon[34391]: pgmap v40: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:49:55.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:55 smithi192 ceph-mon[34700]: pgmap v40: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:49:57.600 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:57 smithi138 ceph-mon[34391]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:49:57.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:57 smithi117 ceph-mon[30947]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:49:57.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:57 smithi192 ceph-mon[34700]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:49:58.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:58 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:49:58.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:58 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:58.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:58 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:58.823 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:58 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:49:58.823 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:58 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:58.823 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:58 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:58.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:58 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:49:58.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:58 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:58.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:58 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:49:59.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:49:59 smithi138 ceph-mon[34391]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:49:59.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:49:59 smithi117 ceph-mon[30947]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:49:59.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:49:59 smithi192 ceph-mon[34700]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:50:00.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:00 smithi117 ceph-mon[30947]: overall HEALTH_OK 2024-09-21T05:50:00.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:00 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:00.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:00 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:00.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:00 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:50:00.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:00 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:50:00.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:00 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:00.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:00 smithi138 ceph-mon[34391]: overall HEALTH_OK 2024-09-21T05:50:00.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:00 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:00.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:00 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:00.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:00 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:50:00.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:00 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:50:00.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:00 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:00.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:00 smithi192 ceph-mon[34700]: overall HEALTH_OK 2024-09-21T05:50:00.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:00 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:00.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:00 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:00.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:00 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:50:00.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:00 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:50:00.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:00 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:01.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:01 smithi117 ceph-mon[30947]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:50:01.781 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:01 smithi138 ceph-mon[34391]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:50:01.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:01 smithi192 ceph-mon[34700]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:50:02.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:02.397+0000 7fb566ffd640 1 -- 172.21.15.138:0/73348330 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb570063800 con 0x7fb55c0b6550 2024-09-21T05:50:02.400 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 1 on host 'smithi138' 2024-09-21T05:50:02.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:02.400+0000 7fb578102640 1 -- 172.21.15.138:0/73348330 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb55c0b6550 msgr2=0x7fb55c0b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:50:02.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:02.400+0000 7fb578102640 1 --2- 172.21.15.138:0/73348330 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb55c0b6550 0x7fb55c0b8a60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb56c000c00 tx=0x7fb56c031400 comp rx=0 tx=0).stop 2024-09-21T05:50:02.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:02.400+0000 7fb578102640 1 -- 172.21.15.138:0/73348330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5701094b0 msgr2=0x7fb570075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:50:02.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:02.400+0000 7fb578102640 1 --2- 172.21.15.138:0/73348330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5701094b0 0x7fb570075c50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb56000d740 tx=0x7fb56000dc00 comp rx=0 tx=0).stop 2024-09-21T05:50:02.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:02.400+0000 7fb578102640 1 -- 172.21.15.138:0/73348330 shutdown_connections 2024-09-21T05:50:02.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:02.400+0000 7fb578102640 1 --2- 172.21.15.138:0/73348330 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb55c0b6550 0x7fb55c0b8a60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:50:02.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:02.400+0000 7fb578102640 1 --2- 172.21.15.138:0/73348330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb570109e00 0x7fb570076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:50:02.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:02.400+0000 7fb578102640 1 --2- 172.21.15.138:0/73348330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5701094b0 0x7fb570075c50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:50:02.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:02.400+0000 7fb578102640 1 --2- 172.21.15.138:0/73348330 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb570103bc0 0x7fb570075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:50:02.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:02.400+0000 7fb578102640 1 -- 172.21.15.138:0/73348330 >> 172.21.15.138:0/73348330 conn(0x7fb57007b910 msgr2=0x7fb5701075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:50:02.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:02.400+0000 7fb578102640 1 -- 172.21.15.138:0/73348330 shutdown_connections 2024-09-21T05:50:02.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:02.400+0000 7fb578102640 1 -- 172.21.15.138:0/73348330 wait complete. 2024-09-21T05:50:02.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:02 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:50:02.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:02 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:02.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:02 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:02.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:02 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:50:02.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:02 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:02.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:02 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:02.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:02 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:50:02.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:02 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:02.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:02 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:03.110 DEBUG:teuthology.orchestra.run.smithi138:osd.1> sudo journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.1.service 2024-09-21T05:50:03.112 INFO:tasks.cephadm:Deploying osd.2 on smithi192 with /dev/vg_nvme/lv_4... 2024-09-21T05:50:03.112 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T05:50:03.369 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T05:50:03.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:03 smithi138 ceph-mon[34391]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:50:03.821 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:03 smithi192 ceph-mon[34700]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:50:04.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:03 smithi117 ceph-mon[30947]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:50:04.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:04 smithi192 ceph-mon[34700]: from='osd.1 [v2:172.21.15.138:6800/1737365588,v1:172.21.15.138:6801/1737365588]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T05:50:04.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:04 smithi192 ceph-mon[34700]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T05:50:04.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:04 smithi138 ceph-mon[34391]: from='osd.1 [v2:172.21.15.138:6800/1737365588,v1:172.21.15.138:6801/1737365588]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T05:50:04.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:04 smithi138 ceph-mon[34391]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T05:50:04.866 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 21 05:50:04 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-1[39125]: 2024-09-21T05:50:04.512+0000 7f25313dc640 -1 osd.1 0 log_to_monitors true 2024-09-21T05:50:05.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:04 smithi117 ceph-mon[30947]: from='osd.1 [v2:172.21.15.138:6800/1737365588,v1:172.21.15.138:6801/1737365588]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T05:50:05.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:04 smithi117 ceph-mon[30947]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T05:50:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-21T05:50:05.520 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi192:vg_nvme/lv_4 2024-09-21T05:50:05.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:05 smithi192 ceph-mon[34700]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:50:05.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:05 smithi192 ceph-mon[34700]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T05:50:05.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:05 smithi192 ceph-mon[34700]: from='osd.1 [v2:172.21.15.138:6800/1737365588,v1:172.21.15.138:6801/1737365588]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-21T05:50:05.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:05 smithi192 ceph-mon[34700]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T05:50:05.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:05 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T05:50:05.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:05 smithi192 ceph-mon[34700]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-21T05:50:05.771 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T05:50:05.855 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:05 smithi138 ceph-mon[34391]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:50:05.855 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:05 smithi138 ceph-mon[34391]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T05:50:05.855 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:05 smithi138 ceph-mon[34391]: from='osd.1 [v2:172.21.15.138:6800/1737365588,v1:172.21.15.138:6801/1737365588]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-21T05:50:05.855 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:05 smithi138 ceph-mon[34391]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T05:50:05.855 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:05 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T05:50:05.855 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:05 smithi138 ceph-mon[34391]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-21T05:50:06.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:05 smithi117 ceph-mon[30947]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:50:06.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:05 smithi117 ceph-mon[30947]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T05:50:06.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:05 smithi117 ceph-mon[30947]: from='osd.1 [v2:172.21.15.138:6800/1737365588,v1:172.21.15.138:6801/1737365588]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-21T05:50:06.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:05 smithi117 ceph-mon[30947]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T05:50:06.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:05 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T05:50:06.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:05 smithi117 ceph-mon[30947]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-21T05:50:06.115 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 21 05:50:05 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-1[39125]: 2024-09-21T05:50:05.962+0000 7f252db73640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T05:50:06.115 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 21 05:50:05 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-1[39125]: 2024-09-21T05:50:05.966+0000 7f2525154640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T05:50:06.215 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.211+0000 7f5532c7d640 1 -- 172.21.15.192:0/192781733 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f552c103b80 msgr2=0x7f552c103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:50:06.215 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.211+0000 7f5532c7d640 1 --2- 172.21.15.192:0/192781733 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f552c103b80 0x7f552c103f90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5520009900 tx=0x7f552002a410 comp rx=0 tx=0).stop 2024-09-21T05:50:06.215 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.211+0000 7f5532c7d640 1 -- 172.21.15.192:0/192781733 shutdown_connections 2024-09-21T05:50:06.215 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.211+0000 7f5532c7d640 1 --2- 172.21.15.192:0/192781733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f552c109e10 0x7f552c110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:50:06.215 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.211+0000 7f5532c7d640 1 --2- 172.21.15.192:0/192781733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f552c1094c0 0x7f552c1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:50:06.215 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.211+0000 7f5532c7d640 1 --2- 172.21.15.192:0/192781733 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f552c103b80 0x7f552c103f90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:50:06.215 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.211+0000 7f5532c7d640 1 -- 172.21.15.192:0/192781733 >> 172.21.15.192:0/192781733 conn(0x7f552c0ff610 msgr2=0x7f552c101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:50:06.215 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.211+0000 7f5532c7d640 1 -- 172.21.15.192:0/192781733 shutdown_connections 2024-09-21T05:50:06.215 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.211+0000 7f5532c7d640 1 -- 172.21.15.192:0/192781733 wait complete. 2024-09-21T05:50:06.215 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.212+0000 7f5532c7d640 1 Processor -- start 2024-09-21T05:50:06.215 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.212+0000 7f5532c7d640 1 -- start start 2024-09-21T05:50:06.216 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.212+0000 7f5532c7d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f552c103b80 0x7f552c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:50:06.216 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.212+0000 7f5532c7d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f552c1094c0 0x7f552c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:50:06.216 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.212+0000 7f5532c7d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f552c109e10 0x7f552c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:50:06.216 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.212+0000 7f5532c7d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f552c113b30 con 0x7f552c1094c0 2024-09-21T05:50:06.216 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.212+0000 7f5532c7d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f552c1139b0 con 0x7f552c109e10 2024-09-21T05:50:06.216 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.212+0000 7f5532c7d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f552c113cb0 con 0x7f552c103b80 2024-09-21T05:50:06.216 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.212+0000 7f5530a19640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f552c103b80 0x7f552c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:50:06.216 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.212+0000 7f5530a19640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f552c103b80 0x7f552c075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:50166/0 (socket says 172.21.15.192:50166) 2024-09-21T05:50:06.216 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.212+0000 7f5530a19640 1 -- 172.21.15.192:0/1500565493 learned_addr learned my addr 172.21.15.192:0/1500565493 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-09-21T05:50:06.216 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.212+0000 7f552bfff640 1 --2- 172.21.15.192:0/1500565493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f552c1094c0 0x7f552c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:50:06.216 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.212+0000 7f553121a640 1 --2- 172.21.15.192:0/1500565493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f552c109e10 0x7f552c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:50:06.216 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.212+0000 7f5530a19640 1 -- 172.21.15.192:0/1500565493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f552c109e10 msgr2=0x7f552c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:50:06.216 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.212+0000 7f5530a19640 1 --2- 172.21.15.192:0/1500565493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f552c109e10 0x7f552c076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:50:06.216 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.212+0000 7f5530a19640 1 -- 172.21.15.192:0/1500565493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f552c1094c0 msgr2=0x7f552c075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:50:06.217 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.212+0000 7f5530a19640 1 --2- 172.21.15.192:0/1500565493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f552c1094c0 0x7f552c075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:50:06.217 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.212+0000 7f5530a19640 1 -- 172.21.15.192:0/1500565493 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f552c1a87a0 con 0x7f552c103b80 2024-09-21T05:50:06.217 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.212+0000 7f552bfff640 1 --2- 172.21.15.192:0/1500565493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f552c1094c0 0x7f552c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:50:06.217 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.213+0000 7f5530a19640 1 --2- 172.21.15.192:0/1500565493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f552c103b80 0x7f552c075710 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5520005940 tx=0x7f55200042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:50:06.217 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.213+0000 7f5529ffb640 1 -- 172.21.15.192:0/1500565493 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552003f030 con 0x7f552c103b80 2024-09-21T05:50:06.218 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.213+0000 7f5532c7d640 1 -- 172.21.15.192:0/1500565493 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f552c1a8a30 con 0x7f552c103b80 2024-09-21T05:50:06.218 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.213+0000 7f5529ffb640 1 -- 172.21.15.192:0/1500565493 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5520003cb0 con 0x7f552c103b80 2024-09-21T05:50:06.218 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.213+0000 7f5529ffb640 1 -- 172.21.15.192:0/1500565493 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5520035070 con 0x7f552c103b80 2024-09-21T05:50:06.218 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.213+0000 7f5532c7d640 1 -- 172.21.15.192:0/1500565493 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f552c1a8f90 con 0x7f552c103b80 2024-09-21T05:50:06.218 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.214+0000 7f5532c7d640 1 -- 172.21.15.192:0/1500565493 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f552c103f90 con 0x7f552c103b80 2024-09-21T05:50:06.218 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.216+0000 7f5529ffb640 1 -- 172.21.15.192:0/1500565493 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7f5520042030 con 0x7f552c103b80 2024-09-21T05:50:06.218 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.216+0000 7f5529ffb640 1 --2- 172.21.15.192:0/1500565493 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f551c0b64d0 0x7f551c0b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:50:06.219 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.217+0000 7f552bfff640 1 --2- 172.21.15.192:0/1500565493 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f551c0b64d0 0x7f551c0b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:50:06.219 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.217+0000 7f5529ffb640 1 -- 172.21.15.192:0/1500565493 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2244+0+0 (secure 0 0 0) 0x7f5520106400 con 0x7f552c103b80 2024-09-21T05:50:06.219 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.217+0000 7f552bfff640 1 --2- 172.21.15.192:0/1500565493 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f551c0b64d0 0x7f551c0b89e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f552c108f50 tx=0x7f5518005e50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:50:06.223 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.221+0000 7f5529ffb640 1 -- 172.21.15.192:0/1500565493 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f55200ce520 con 0x7f552c103b80 2024-09-21T05:50:06.418 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:06.416+0000 7f5532c7d640 1 -- 172.21.15.192:0/1500565493 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f552c063800 con 0x7f551c0b64d0 2024-09-21T05:50:06.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:06 smithi192 ceph-mon[34700]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-21T05:50:06.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:06 smithi192 ceph-mon[34700]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T05:50:06.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:06 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T05:50:06.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:06 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T05:50:06.810 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:06 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:06.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:06 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:06.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:06 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T05:50:06.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:06 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:06.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:06 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:50:06.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:06 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:50:06.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:06 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:06.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:06 smithi192 ceph-mon[34700]: from='osd.1 ' entity='osd.1' 2024-09-21T05:50:06.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:06 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:50:06.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:06 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:50:06.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:06 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:50:06.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:06 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T05:50:06.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:06 smithi138 ceph-mon[34391]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-21T05:50:06.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:06 smithi138 ceph-mon[34391]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T05:50:06.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:06 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T05:50:06.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:06 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T05:50:06.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:06 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:06.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:06 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:06.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:06 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T05:50:06.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:06 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:06.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:06 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:50:06.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:06 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:50:06.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:06 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:06.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:06 smithi138 ceph-mon[34391]: from='osd.1 ' entity='osd.1' 2024-09-21T05:50:06.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:06 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:50:06.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:06 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:50:06.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:06 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:50:06.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:06 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T05:50:07.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:06 smithi117 ceph-mon[30947]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-21T05:50:07.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:06 smithi117 ceph-mon[30947]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T05:50:07.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:06 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T05:50:07.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:06 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T05:50:07.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:06 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:07.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:06 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:07.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:06 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T05:50:07.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:06 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:07.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:06 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:50:07.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:06 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:50:07.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:06 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:07.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:06 smithi117 ceph-mon[30947]: from='osd.1 ' entity='osd.1' 2024-09-21T05:50:07.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:06 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:50:07.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:06 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:50:07.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:06 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:50:07.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:06 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T05:50:07.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:07 smithi192 ceph-mon[34700]: Detected new or changed devices on smithi138 2024-09-21T05:50:07.813 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:07 smithi192 ceph-mon[34700]: Adjusting osd_memory_target on smithi138 to 17088M 2024-09-21T05:50:07.813 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:07 smithi192 ceph-mon[34700]: purged_snaps scrub starts 2024-09-21T05:50:07.813 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:07 smithi192 ceph-mon[34700]: purged_snaps scrub ok 2024-09-21T05:50:07.813 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:07 smithi192 ceph-mon[34700]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:50:07.813 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:07 smithi192 ceph-mon[34700]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:50:07.813 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:07 smithi192 ceph-mon[34700]: osd.1 [v2:172.21.15.138:6800/1737365588,v1:172.21.15.138:6801/1737365588] boot 2024-09-21T05:50:07.813 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:07 smithi192 ceph-mon[34700]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T05:50:07.813 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:07 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T05:50:07.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:07 smithi138 ceph-mon[34391]: Detected new or changed devices on smithi138 2024-09-21T05:50:07.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:07 smithi138 ceph-mon[34391]: Adjusting osd_memory_target on smithi138 to 17088M 2024-09-21T05:50:07.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:07 smithi138 ceph-mon[34391]: purged_snaps scrub starts 2024-09-21T05:50:07.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:07 smithi138 ceph-mon[34391]: purged_snaps scrub ok 2024-09-21T05:50:07.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:07 smithi138 ceph-mon[34391]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:50:07.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:07 smithi138 ceph-mon[34391]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:50:07.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:07 smithi138 ceph-mon[34391]: osd.1 [v2:172.21.15.138:6800/1737365588,v1:172.21.15.138:6801/1737365588] boot 2024-09-21T05:50:07.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:07 smithi138 ceph-mon[34391]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T05:50:07.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:07 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T05:50:08.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:07 smithi117 ceph-mon[30947]: Detected new or changed devices on smithi138 2024-09-21T05:50:08.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:07 smithi117 ceph-mon[30947]: Adjusting osd_memory_target on smithi138 to 17088M 2024-09-21T05:50:08.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:07 smithi117 ceph-mon[30947]: purged_snaps scrub starts 2024-09-21T05:50:08.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:07 smithi117 ceph-mon[30947]: purged_snaps scrub ok 2024-09-21T05:50:08.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:07 smithi117 ceph-mon[30947]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:50:08.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:07 smithi117 ceph-mon[30947]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-21T05:50:08.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:07 smithi117 ceph-mon[30947]: osd.1 [v2:172.21.15.138:6800/1737365588,v1:172.21.15.138:6801/1737365588] boot 2024-09-21T05:50:08.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:07 smithi117 ceph-mon[30947]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T05:50:08.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:07 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T05:50:09.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:08 smithi192 ceph-mon[34700]: osdmap e15: 2 total, 2 up, 2 in 2024-09-21T05:50:09.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:08 smithi192 ceph-mon[34700]: pgmap v51: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:09.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:08 smithi192 ceph-mon[34700]: from='client.? 172.21.15.192:0/2592401424' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "94972378-a11f-4129-9e40-ad3fdec1d189"} : dispatch 2024-09-21T05:50:09.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:08 smithi192 ceph-mon[34700]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "94972378-a11f-4129-9e40-ad3fdec1d189"} : dispatch 2024-09-21T05:50:09.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:08 smithi192 ceph-mon[34700]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94972378-a11f-4129-9e40-ad3fdec1d189"}]': finished 2024-09-21T05:50:09.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:08 smithi192 ceph-mon[34700]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T05:50:09.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:08 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T05:50:09.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:08 smithi117 ceph-mon[30947]: osdmap e15: 2 total, 2 up, 2 in 2024-09-21T05:50:09.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:08 smithi117 ceph-mon[30947]: pgmap v51: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:09.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:08 smithi117 ceph-mon[30947]: from='client.? 172.21.15.192:0/2592401424' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "94972378-a11f-4129-9e40-ad3fdec1d189"} : dispatch 2024-09-21T05:50:09.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:08 smithi117 ceph-mon[30947]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "94972378-a11f-4129-9e40-ad3fdec1d189"} : dispatch 2024-09-21T05:50:09.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:08 smithi117 ceph-mon[30947]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94972378-a11f-4129-9e40-ad3fdec1d189"}]': finished 2024-09-21T05:50:09.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:08 smithi117 ceph-mon[30947]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T05:50:09.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:08 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T05:50:09.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:08 smithi138 ceph-mon[34391]: osdmap e15: 2 total, 2 up, 2 in 2024-09-21T05:50:09.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:08 smithi138 ceph-mon[34391]: pgmap v51: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:09.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:08 smithi138 ceph-mon[34391]: from='client.? 172.21.15.192:0/2592401424' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "94972378-a11f-4129-9e40-ad3fdec1d189"} : dispatch 2024-09-21T05:50:09.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:08 smithi138 ceph-mon[34391]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "94972378-a11f-4129-9e40-ad3fdec1d189"} : dispatch 2024-09-21T05:50:09.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:08 smithi138 ceph-mon[34391]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94972378-a11f-4129-9e40-ad3fdec1d189"}]': finished 2024-09-21T05:50:09.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:08 smithi138 ceph-mon[34391]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T05:50:09.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:08 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T05:50:10.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:09 smithi117 ceph-mon[30947]: from='client.? 172.21.15.192:0/2633332118' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:50:10.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:09 smithi138 ceph-mon[34391]: from='client.? 172.21.15.192:0/2633332118' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:50:10.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:09 smithi192 ceph-mon[34700]: from='client.? 172.21.15.192:0/2633332118' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:50:11.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:10 smithi117 ceph-mon[30947]: pgmap v53: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:11.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:10 smithi138 ceph-mon[34391]: pgmap v53: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:11.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:10 smithi192 ceph-mon[34700]: pgmap v53: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:13.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:13 smithi117 ceph-mon[30947]: pgmap v54: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:13.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:13 smithi138 ceph-mon[34391]: pgmap v54: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:13.892 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:13 smithi192 ceph-mon[34700]: pgmap v54: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:15.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:15 smithi117 ceph-mon[30947]: pgmap v55: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:15.788 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:15 smithi192 ceph-mon[34700]: pgmap v55: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:15.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:15 smithi138 ceph-mon[34391]: pgmap v55: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:17.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:17 smithi192 ceph-mon[34700]: pgmap v56: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:17.563 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:17 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T05:50:17.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:17 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:50:17.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:17 smithi117 ceph-mon[30947]: pgmap v56: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:17.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:17 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T05:50:17.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:17 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:50:17.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:17 smithi138 ceph-mon[34391]: pgmap v56: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:17.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:17 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T05:50:17.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:17 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:50:18.769 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:18 smithi192 ceph-mon[34700]: Deploying daemon osd.2 on smithi192 2024-09-21T05:50:18.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:18 smithi117 ceph-mon[30947]: Deploying daemon osd.2 on smithi192 2024-09-21T05:50:18.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:18 smithi138 ceph-mon[34391]: Deploying daemon osd.2 on smithi192 2024-09-21T05:50:19.667 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:19 smithi192 ceph-mon[34700]: pgmap v57: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:19.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:19 smithi117 ceph-mon[30947]: pgmap v57: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:19.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:19 smithi138 ceph-mon[34391]: pgmap v57: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:21.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:21 smithi117 ceph-mon[30947]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:21.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:21 smithi138 ceph-mon[34391]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:21.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:21 smithi192 ceph-mon[34700]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:23.766 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:23 smithi192 ceph-mon[34700]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:23.766 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:50:23.766 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:23.766 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:23.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:23 smithi117 ceph-mon[30947]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:23.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:50:23.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:23.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:23.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:23 smithi138 ceph-mon[34391]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:23.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:50:23.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:23.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:25.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:25 smithi192 ceph-mon[34700]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:25.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:25 smithi117 ceph-mon[30947]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:25.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:25 smithi138 ceph-mon[34391]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:26.961 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:26 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:26.962 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:26 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:26.962 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:26 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:50:26.962 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:26 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:50:26.962 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:26 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:26.962 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:26 smithi138 ceph-mon[34391]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:26.974 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:26 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:26.974 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:26 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:26.974 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:26 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:50:26.974 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:26 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:50:26.974 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:26 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:26.974 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:26 smithi192 ceph-mon[34700]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:27.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:26 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:27.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:26 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:27.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:26 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:50:27.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:26 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:50:27.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:26 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:27.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:26 smithi117 ceph-mon[30947]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:27.275 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:27.272+0000 7f5529ffb640 1 -- 172.21.15.192:0/1500565493 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f552c063800 con 0x7f551c0b64d0 2024-09-21T05:50:27.275 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 2 on host 'smithi192' 2024-09-21T05:50:27.277 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:27.275+0000 7f5532c7d640 1 -- 172.21.15.192:0/1500565493 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f551c0b64d0 msgr2=0x7f551c0b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:50:27.277 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:27.275+0000 7f5532c7d640 1 --2- 172.21.15.192:0/1500565493 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f551c0b64d0 0x7f551c0b89e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f552c108f50 tx=0x7f5518005e50 comp rx=0 tx=0).stop 2024-09-21T05:50:27.277 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:27.275+0000 7f5532c7d640 1 -- 172.21.15.192:0/1500565493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f552c103b80 msgr2=0x7f552c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:50:27.277 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:27.275+0000 7f5532c7d640 1 --2- 172.21.15.192:0/1500565493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f552c103b80 0x7f552c075710 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5520005940 tx=0x7f55200042f0 comp rx=0 tx=0).stop 2024-09-21T05:50:27.277 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:27.275+0000 7f5532c7d640 1 -- 172.21.15.192:0/1500565493 shutdown_connections 2024-09-21T05:50:27.277 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:27.275+0000 7f5532c7d640 1 --2- 172.21.15.192:0/1500565493 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f551c0b64d0 0x7f551c0b89e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:50:27.278 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:27.275+0000 7f5532c7d640 1 --2- 172.21.15.192:0/1500565493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f552c1094c0 0x7f552c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:50:27.278 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:27.275+0000 7f5532c7d640 1 --2- 172.21.15.192:0/1500565493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f552c109e10 0x7f552c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:50:27.278 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:27.275+0000 7f5532c7d640 1 --2- 172.21.15.192:0/1500565493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f552c103b80 0x7f552c075710 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:50:27.278 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:27.275+0000 7f5532c7d640 1 -- 172.21.15.192:0/1500565493 >> 172.21.15.192:0/1500565493 conn(0x7f552c0ff610 msgr2=0x7f552c101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:50:27.278 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:27.275+0000 7f5532c7d640 1 -- 172.21.15.192:0/1500565493 shutdown_connections 2024-09-21T05:50:27.278 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:50:27.275+0000 7f5532c7d640 1 -- 172.21.15.192:0/1500565493 wait complete. 2024-09-21T05:50:27.846 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:27 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:50:27.846 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:27 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:27.846 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:27 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:27.873 DEBUG:teuthology.orchestra.run.smithi192:osd.2> sudo journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.2.service 2024-09-21T05:50:27.875 INFO:tasks.cephadm:Deploying osd.3 on smithi117 with /dev/vg_nvme/lv_3... 2024-09-21T05:50:27.875 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T05:50:28.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:27 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:50:28.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:27 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:28.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:27 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:28.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:27 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:50:28.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:27 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:28.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:27 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:28.135 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:50:28.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:28 smithi117 ceph-mon[30947]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:28.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:28 smithi192 ceph-mon[34700]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:29.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:28 smithi138 ceph-mon[34391]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:29.262 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 21 05:50:29 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-2[39872]: 2024-09-21T05:50:29.196+0000 7f2b72d5c640 -1 osd.2 0 log_to_monitors true 2024-09-21T05:50:29.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:29 smithi192 ceph-mon[34700]: from='osd.2 [v2:172.21.15.192:6800/3002534992,v1:172.21.15.192:6801/3002534992]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T05:50:29.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:29 smithi192 ceph-mon[34700]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T05:50:29.988 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:29 smithi117 ceph-mon[30947]: from='osd.2 [v2:172.21.15.192:6800/3002534992,v1:172.21.15.192:6801/3002534992]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T05:50:29.988 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:29 smithi117 ceph-mon[30947]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T05:50:30.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:29 smithi138 ceph-mon[34391]: from='osd.2 [v2:172.21.15.192:6800/3002534992,v1:172.21.15.192:6801/3002534992]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T05:50:30.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:29 smithi138 ceph-mon[34391]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T05:50:30.774 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:50:30.802 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi117:vg_nvme/lv_3 2024-09-21T05:50:30.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:30 smithi192 ceph-mon[34700]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T05:50:30.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:30 smithi192 ceph-mon[34700]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T05:50:30.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:30 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T05:50:30.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:30 smithi192 ceph-mon[34700]: from='osd.2 [v2:172.21.15.192:6800/3002534992,v1:172.21.15.192:6801/3002534992]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]} : dispatch 2024-09-21T05:50:30.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:30 smithi192 ceph-mon[34700]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]} : dispatch 2024-09-21T05:50:30.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:30 smithi192 ceph-mon[34700]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:30.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:30 smithi192 ceph-mon[34700]: Detected new or changed devices on smithi192 2024-09-21T05:50:30.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:30 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:30.981 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:30 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:30.981 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:30 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T05:50:30.981 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:30 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:30.981 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:30 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:50:30.981 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:30 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:50:30.981 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:30 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:31.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:30 smithi117 ceph-mon[30947]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T05:50:31.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:30 smithi117 ceph-mon[30947]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T05:50:31.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:30 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T05:50:31.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:30 smithi117 ceph-mon[30947]: from='osd.2 [v2:172.21.15.192:6800/3002534992,v1:172.21.15.192:6801/3002534992]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]} : dispatch 2024-09-21T05:50:31.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:30 smithi117 ceph-mon[30947]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]} : dispatch 2024-09-21T05:50:31.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:30 smithi117 ceph-mon[30947]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:31.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:30 smithi117 ceph-mon[30947]: Detected new or changed devices on smithi192 2024-09-21T05:50:31.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:30 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:31.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:30 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:31.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:30 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T05:50:31.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:30 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:31.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:30 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:50:31.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:30 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:50:31.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:30 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:31.063 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:50:31.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:30 smithi138 ceph-mon[34391]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T05:50:31.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:30 smithi138 ceph-mon[34391]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T05:50:31.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:30 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T05:50:31.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:30 smithi138 ceph-mon[34391]: from='osd.2 [v2:172.21.15.192:6800/3002534992,v1:172.21.15.192:6801/3002534992]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]} : dispatch 2024-09-21T05:50:31.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:30 smithi138 ceph-mon[34391]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]} : dispatch 2024-09-21T05:50:31.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:30 smithi138 ceph-mon[34391]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:31.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:30 smithi138 ceph-mon[34391]: Detected new or changed devices on smithi192 2024-09-21T05:50:31.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:30 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:31.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:30 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:31.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:30 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T05:50:31.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:30 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:31.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:30 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:50:31.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:30 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:50:31.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:30 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:31.480 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 21 05:50:31 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-2[39872]: 2024-09-21T05:50:31.184+0000 7f2b6ecde640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T05:50:31.480 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 21 05:50:31 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-2[39872]: 2024-09-21T05:50:31.188+0000 7f2b66ad4640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T05:50:31.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.580+0000 7f4616d10640 1 -- 172.21.15.117:0/1831559855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4610109df0 msgr2=0x7f4610110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:50:31.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.580+0000 7f4616d10640 1 --2- 172.21.15.117:0/1831559855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4610109df0 0x7f4610110620 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4608009900 tx=0x7f460802a410 comp rx=0 tx=0).stop 2024-09-21T05:50:31.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.581+0000 7f4616d10640 1 -- 172.21.15.117:0/1831559855 shutdown_connections 2024-09-21T05:50:31.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.581+0000 7f4616d10640 1 --2- 172.21.15.117:0/1831559855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4610109df0 0x7f4610110620 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:50:31.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.581+0000 7f4616d10640 1 --2- 172.21.15.117:0/1831559855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46101094a0 0x7f46101098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:50:31.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.581+0000 7f4616d10640 1 --2- 172.21.15.117:0/1831559855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4610103b60 0x7f4610103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:50:31.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.581+0000 7f4616d10640 1 -- 172.21.15.117:0/1831559855 >> 172.21.15.117:0/1831559855 conn(0x7f461007b850 msgr2=0x7f461007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:50:31.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.581+0000 7f4616d10640 1 -- 172.21.15.117:0/1831559855 shutdown_connections 2024-09-21T05:50:31.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.581+0000 7f4616d10640 1 -- 172.21.15.117:0/1831559855 wait complete. 2024-09-21T05:50:31.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.581+0000 7f4616d10640 1 Processor -- start 2024-09-21T05:50:31.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.581+0000 7f4616d10640 1 -- start start 2024-09-21T05:50:31.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.582+0000 7f4616d10640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4610103b60 0x7f461019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:50:31.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.582+0000 7f4616d10640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46101094a0 0x7f461019a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:50:31.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.582+0000 7f4616d10640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4610109df0 0x7f461019fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:50:31.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.582+0000 7f4616d10640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4610113ab0 con 0x7f4610103b60 2024-09-21T05:50:31.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.582+0000 7f4616d10640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4610113930 con 0x7f4610109df0 2024-09-21T05:50:31.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.582+0000 7f4616d10640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4610113c30 con 0x7f46101094a0 2024-09-21T05:50:31.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.582+0000 7f4614aac640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4610103b60 0x7f461019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:50:31.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.582+0000 7f4614aac640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4610103b60 0x7f461019a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57292/0 (socket says 172.21.15.117:57292) 2024-09-21T05:50:31.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.582+0000 7f4614aac640 1 -- 172.21.15.117:0/3818846182 learned_addr learned my addr 172.21.15.117:0/3818846182 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:50:31.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.582+0000 7f4607fff640 1 --2- 172.21.15.117:0/3818846182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46101094a0 0x7f461019a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:50:31.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.582+0000 7f4614aac640 1 -- 172.21.15.117:0/3818846182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46101094a0 msgr2=0x7f461019a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:50:31.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.582+0000 7f4614aac640 1 --2- 172.21.15.117:0/3818846182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46101094a0 0x7f461019a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:50:31.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.582+0000 7f4614aac640 1 -- 172.21.15.117:0/3818846182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4610109df0 msgr2=0x7f461019fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:50:31.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.582+0000 7f4614aac640 1 --2- 172.21.15.117:0/3818846182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4610109df0 0x7f461019fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:50:31.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.582+0000 7f4614aac640 1 -- 172.21.15.117:0/3818846182 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46101a02b0 con 0x7f4610103b60 2024-09-21T05:50:31.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.582+0000 7f4614aac640 1 --2- 172.21.15.117:0/3818846182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4610103b60 0x7f461019a400 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f460c00cce0 tx=0x7f460c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:50:31.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.582+0000 7f4605ffb640 1 -- 172.21.15.117:0/3818846182 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c004b20 con 0x7f4610103b60 2024-09-21T05:50:31.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.582+0000 7f4605ffb640 1 -- 172.21.15.117:0/3818846182 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f460c004cc0 con 0x7f4610103b60 2024-09-21T05:50:31.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.582+0000 7f4605ffb640 1 -- 172.21.15.117:0/3818846182 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c00f6b0 con 0x7f4610103b60 2024-09-21T05:50:31.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.582+0000 7f4616d10640 1 -- 172.21.15.117:0/3818846182 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46101a05a0 con 0x7f4610103b60 2024-09-21T05:50:31.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.583+0000 7f4616d10640 1 -- 172.21.15.117:0/3818846182 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46101a6dc0 con 0x7f4610103b60 2024-09-21T05:50:31.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.583+0000 7f4616d10640 1 -- 172.21.15.117:0/3818846182 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4610103f70 con 0x7f4610103b60 2024-09-21T05:50:31.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.585+0000 7f4605ffb640 1 -- 172.21.15.117:0/3818846182 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 19) ==== 154474+0+0 (secure 0 0 0) 0x7f460c00f850 con 0x7f4610103b60 2024-09-21T05:50:31.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.586+0000 7f4605ffb640 1 --2- 172.21.15.117:0/3818846182 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f45f80b6580 0x7f45f80b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:50:31.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.586+0000 7f4607fff640 1 --2- 172.21.15.117:0/3818846182 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f45f80b6580 0x7f45f80b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:50:31.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.586+0000 7f4605ffb640 1 -- 172.21.15.117:0/3818846182 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2786+0+0 (secure 0 0 0) 0x7f460c0e6100 con 0x7f4610103b60 2024-09-21T05:50:31.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.586+0000 7f4607fff640 1 --2- 172.21.15.117:0/3818846182 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f45f80b6580 0x7f45f80b8a90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4600005e10 tx=0x7f4600005da0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:50:31.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.589+0000 7f4605ffb640 1 -- 172.21.15.117:0/3818846182 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f460c014030 con 0x7f4610103b60 2024-09-21T05:50:31.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:31.779+0000 7f4616d10640 1 -- 172.21.15.117:0/3818846182 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4610063800 con 0x7f45f80b6580 2024-09-21T05:50:32.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:31 smithi117 ceph-mon[30947]: Adjusting osd_memory_target on smithi192 to 17088M 2024-09-21T05:50:32.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:31 smithi117 ceph-mon[30947]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-09-21T05:50:32.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:31 smithi117 ceph-mon[30947]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T05:50:32.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:31 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T05:50:32.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:31 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T05:50:32.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:31 smithi117 ceph-mon[30947]: from='osd.2 ' entity='osd.2' 2024-09-21T05:50:32.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:31 smithi138 ceph-mon[34391]: Adjusting osd_memory_target on smithi192 to 17088M 2024-09-21T05:50:32.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:31 smithi138 ceph-mon[34391]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-09-21T05:50:32.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:31 smithi138 ceph-mon[34391]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T05:50:32.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:31 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T05:50:32.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:31 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T05:50:32.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:31 smithi138 ceph-mon[34391]: from='osd.2 ' entity='osd.2' 2024-09-21T05:50:32.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:31 smithi192 ceph-mon[34700]: Adjusting osd_memory_target on smithi192 to 17088M 2024-09-21T05:50:32.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:31 smithi192 ceph-mon[34700]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-09-21T05:50:32.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:31 smithi192 ceph-mon[34700]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T05:50:32.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:31 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T05:50:32.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:31 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T05:50:32.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:31 smithi192 ceph-mon[34700]: from='osd.2 ' entity='osd.2' 2024-09-21T05:50:33.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:32 smithi117 ceph-mon[30947]: purged_snaps scrub starts 2024-09-21T05:50:33.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:32 smithi117 ceph-mon[30947]: purged_snaps scrub ok 2024-09-21T05:50:33.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:32 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T05:50:33.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:32 smithi117 ceph-mon[30947]: from='client.14505 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:50:33.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:32 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:50:33.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:32 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:50:33.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:32 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:50:33.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:32 smithi117 ceph-mon[30947]: osd.2 [v2:172.21.15.192:6800/3002534992,v1:172.21.15.192:6801/3002534992] boot 2024-09-21T05:50:33.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:32 smithi117 ceph-mon[30947]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T05:50:33.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:32 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T05:50:33.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:32 smithi117 ceph-mon[30947]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:33.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:32 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T05:50:33.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:32 smithi138 ceph-mon[34391]: purged_snaps scrub starts 2024-09-21T05:50:33.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:32 smithi138 ceph-mon[34391]: purged_snaps scrub ok 2024-09-21T05:50:33.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:32 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T05:50:33.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:32 smithi138 ceph-mon[34391]: from='client.14505 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:50:33.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:32 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:50:33.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:32 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:50:33.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:32 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:50:33.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:32 smithi138 ceph-mon[34391]: osd.2 [v2:172.21.15.192:6800/3002534992,v1:172.21.15.192:6801/3002534992] boot 2024-09-21T05:50:33.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:32 smithi138 ceph-mon[34391]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T05:50:33.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:32 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T05:50:33.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:32 smithi138 ceph-mon[34391]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:33.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:32 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T05:50:33.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:32 smithi192 ceph-mon[34700]: purged_snaps scrub starts 2024-09-21T05:50:33.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:32 smithi192 ceph-mon[34700]: purged_snaps scrub ok 2024-09-21T05:50:33.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:32 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T05:50:33.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:32 smithi192 ceph-mon[34700]: from='client.14505 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:50:33.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:32 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:50:33.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:32 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:50:33.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:32 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:50:33.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:32 smithi192 ceph-mon[34700]: osd.2 [v2:172.21.15.192:6800/3002534992,v1:172.21.15.192:6801/3002534992] boot 2024-09-21T05:50:33.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:32 smithi192 ceph-mon[34700]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T05:50:33.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:32 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T05:50:33.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:32 smithi192 ceph-mon[34700]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-21T05:50:33.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:32 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T05:50:34.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:34 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T05:50:34.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:34 smithi192 ceph-mon[34700]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T05:50:34.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:34 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T05:50:34.480 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 21 05:50:34 smithi192 sudo[43534]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T05:50:34.481 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 21 05:50:34 smithi192 sudo[43534]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T05:50:34.481 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 21 05:50:34 smithi192 sudo[43534]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T05:50:34.481 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 21 05:50:34 smithi192 sudo[43534]: pam_unix(sudo:session): session closed for user root 2024-09-21T05:50:34.481 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 21 05:50:34 smithi192 sudo[43537]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T05:50:34.481 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 21 05:50:34 smithi192 sudo[43537]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T05:50:34.481 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 21 05:50:34 smithi192 sudo[43537]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T05:50:34.481 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 21 05:50:34 smithi192 sudo[43537]: pam_unix(sudo:session): session closed for user root 2024-09-21T05:50:34.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:34 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T05:50:34.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:34 smithi117 ceph-mon[30947]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T05:50:34.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:34 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T05:50:34.527 INFO:journalctl@ceph.osd.0.smithi117.stdout:Sep 21 05:50:34 smithi117 sudo[51657]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T05:50:34.527 INFO:journalctl@ceph.osd.0.smithi117.stdout:Sep 21 05:50:34 smithi117 sudo[51657]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T05:50:34.527 INFO:journalctl@ceph.osd.0.smithi117.stdout:Sep 21 05:50:34 smithi117 sudo[51657]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T05:50:34.527 INFO:journalctl@ceph.osd.0.smithi117.stdout:Sep 21 05:50:34 smithi117 sudo[51657]: pam_unix(sudo:session): session closed for user root 2024-09-21T05:50:34.527 INFO:journalctl@ceph.osd.0.smithi117.stdout:Sep 21 05:50:34 smithi117 sudo[51660]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T05:50:34.528 INFO:journalctl@ceph.osd.0.smithi117.stdout:Sep 21 05:50:34 smithi117 sudo[51660]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T05:50:34.528 INFO:journalctl@ceph.osd.0.smithi117.stdout:Sep 21 05:50:34 smithi117 sudo[51660]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T05:50:34.528 INFO:journalctl@ceph.osd.0.smithi117.stdout:Sep 21 05:50:34 smithi117 sudo[51660]: pam_unix(sudo:session): session closed for user root 2024-09-21T05:50:34.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:34 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T05:50:34.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:34 smithi138 ceph-mon[34391]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T05:50:34.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:34 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T05:50:34.615 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 21 05:50:34 smithi138 sudo[42666]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T05:50:34.616 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 21 05:50:34 smithi138 sudo[42666]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T05:50:34.616 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 21 05:50:34 smithi138 sudo[42666]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T05:50:34.616 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 21 05:50:34 smithi138 sudo[42666]: pam_unix(sudo:session): session closed for user root 2024-09-21T05:50:34.616 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 21 05:50:34 smithi138 sudo[42669]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T05:50:34.616 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 21 05:50:34 smithi138 sudo[42669]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T05:50:34.616 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 21 05:50:34 smithi138 sudo[42669]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T05:50:34.616 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 21 05:50:34 smithi138 sudo[42669]: pam_unix(sudo:session): session closed for user root 2024-09-21T05:50:34.929 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:34 smithi117 sudo[51669]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T05:50:34.929 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:34 smithi117 sudo[51669]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T05:50:34.929 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:34 smithi117 sudo[51669]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T05:50:34.929 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:34 smithi117 sudo[51669]: pam_unix(sudo:session): session closed for user root 2024-09-21T05:50:34.929 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:34 smithi117 sudo[51672]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T05:50:34.930 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:34 smithi117 sudo[51672]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T05:50:34.930 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:34 smithi117 sudo[51672]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T05:50:34.930 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:34 smithi117 sudo[51672]: pam_unix(sudo:session): session closed for user root 2024-09-21T05:50:34.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:34 smithi192 sudo[43540]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T05:50:34.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:34 smithi192 sudo[43540]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T05:50:34.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:34 smithi192 sudo[43540]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T05:50:34.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:34 smithi192 sudo[43540]: pam_unix(sudo:session): session closed for user root 2024-09-21T05:50:34.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:34 smithi192 sudo[43543]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T05:50:34.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:34 smithi192 sudo[43543]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T05:50:34.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:34 smithi192 sudo[43543]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T05:50:34.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:34 smithi192 sudo[43543]: pam_unix(sudo:session): session closed for user root 2024-09-21T05:50:35.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:34 smithi138 sudo[42672]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T05:50:35.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:34 smithi138 sudo[42672]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T05:50:35.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:34 smithi138 sudo[42672]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T05:50:35.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:34 smithi138 sudo[42672]: pam_unix(sudo:session): session closed for user root 2024-09-21T05:50:35.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:34 smithi138 sudo[42675]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T05:50:35.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:34 smithi138 sudo[42675]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T05:50:35.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:34 smithi138 sudo[42675]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T05:50:35.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:34 smithi138 sudo[42675]: pam_unix(sudo:session): session closed for user root 2024-09-21T05:50:35.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:35.405+0000 7f4605ffb640 1 -- 172.21.15.117:0/3818846182 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f460c0a9bf0 con 0x7f4610103b60 2024-09-21T05:50:35.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T05:50:35.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:35 smithi192 ceph-mon[34700]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T05:50:35.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:35 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2524142948' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d4168c43-f0de-4445-945c-92d1cf270c33"} : dispatch 2024-09-21T05:50:35.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:35 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2524142948' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4168c43-f0de-4445-945c-92d1cf270c33"}]': finished 2024-09-21T05:50:35.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:35 smithi192 ceph-mon[34700]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T05:50:35.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T05:50:35.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:35.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:35.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:35 smithi192 ceph-mon[34700]: pgmap v71: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:35.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:35.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:35 smithi192 ceph-mon[34700]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T05:50:35.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:35 smithi192 ceph-mon[34700]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T05:50:35.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T05:50:35.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T05:50:35.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:50:35.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:35 smithi192 ceph-mon[34700]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T05:50:35.482 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T05:50:35.482 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T05:50:35.482 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:50:35.482 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:35 smithi192 ceph-mon[34700]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T05:50:35.482 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:35 smithi192 ceph-mon[34700]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T05:50:35.482 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T05:50:35.482 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T05:50:35.482 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:50:35.482 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:35 smithi192 ceph-mon[34700]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T05:50:35.482 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:35 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/511216211' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:50:35.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T05:50:35.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:35 smithi117 ceph-mon[30947]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T05:50:35.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:35 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2524142948' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d4168c43-f0de-4445-945c-92d1cf270c33"} : dispatch 2024-09-21T05:50:35.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:35 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2524142948' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4168c43-f0de-4445-945c-92d1cf270c33"}]': finished 2024-09-21T05:50:35.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:35 smithi117 ceph-mon[30947]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T05:50:35.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T05:50:35.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:35.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:35.528 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:35 smithi117 ceph-mon[30947]: pgmap v71: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:35.528 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:35.528 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:35 smithi117 ceph-mon[30947]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T05:50:35.528 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:35 smithi117 ceph-mon[30947]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T05:50:35.528 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T05:50:35.528 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T05:50:35.528 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:50:35.528 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:35 smithi117 ceph-mon[30947]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T05:50:35.528 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T05:50:35.528 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T05:50:35.528 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:50:35.528 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:35 smithi117 ceph-mon[30947]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T05:50:35.528 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:35 smithi117 ceph-mon[30947]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T05:50:35.528 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T05:50:35.528 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T05:50:35.529 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:50:35.529 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:35 smithi117 ceph-mon[30947]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T05:50:35.529 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:35 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/511216211' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:50:35.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T05:50:35.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:35 smithi138 ceph-mon[34391]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T05:50:35.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:35 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2524142948' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "d4168c43-f0de-4445-945c-92d1cf270c33"} : dispatch 2024-09-21T05:50:35.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:35 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2524142948' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d4168c43-f0de-4445-945c-92d1cf270c33"}]': finished 2024-09-21T05:50:35.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:35 smithi138 ceph-mon[34391]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T05:50:35.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T05:50:35.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:35.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:35.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:35 smithi138 ceph-mon[34391]: pgmap v71: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:35.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:35.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:35 smithi138 ceph-mon[34391]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T05:50:35.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:35 smithi138 ceph-mon[34391]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T05:50:35.617 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T05:50:35.617 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T05:50:35.617 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:50:35.617 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:35 smithi138 ceph-mon[34391]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T05:50:35.617 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T05:50:35.617 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T05:50:35.617 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:50:35.617 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:35 smithi138 ceph-mon[34391]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T05:50:35.617 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:35 smithi138 ceph-mon[34391]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T05:50:35.617 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T05:50:35.617 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T05:50:35.617 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T05:50:35.617 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:35 smithi138 ceph-mon[34391]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T05:50:35.617 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:35 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/511216211' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:50:36.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:36 smithi192 ceph-mon[34700]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-09-21T05:50:36.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:36 smithi117 ceph-mon[30947]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-09-21T05:50:36.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:36 smithi138 ceph-mon[34391]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-09-21T05:50:37.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:37 smithi192 ceph-mon[34700]: pgmap v72: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:37.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:37 smithi117 ceph-mon[30947]: pgmap v72: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:37.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:37 smithi138 ceph-mon[34391]: pgmap v72: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:39.739 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:39 smithi117 ceph-mon[30947]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:39.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:39 smithi138 ceph-mon[34391]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:39.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:39 smithi192 ceph-mon[34700]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:41.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:41 smithi117 ceph-mon[30947]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:41.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:41 smithi138 ceph-mon[34391]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:41.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:41 smithi192 ceph-mon[34700]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:43.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:43 smithi117 ceph-mon[30947]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:43.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:43 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T05:50:43.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:43 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:50:43.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:43 smithi138 ceph-mon[34391]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:43.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:43 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T05:50:43.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:43 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:50:43.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:43 smithi192 ceph-mon[34700]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:43.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:43 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T05:50:43.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:43 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:50:44.654 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:44 smithi117 ceph-mon[30947]: Deploying daemon osd.3 on smithi117 2024-09-21T05:50:44.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:44 smithi138 ceph-mon[34391]: Deploying daemon osd.3 on smithi117 2024-09-21T05:50:44.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:44 smithi192 ceph-mon[34700]: Deploying daemon osd.3 on smithi117 2024-09-21T05:50:45.645 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:45 smithi117 ceph-mon[30947]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:45.766 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:45 smithi138 ceph-mon[34391]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:45.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:45 smithi192 ceph-mon[34700]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:47.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:47 smithi117 ceph-mon[30947]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:47.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:47 smithi138 ceph-mon[34391]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:47.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:47 smithi192 ceph-mon[34700]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:49.755 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:49 smithi117 ceph-mon[30947]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:49.755 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:49 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:50:49.755 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:49 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:49.755 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:49 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:49.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:49 smithi138 ceph-mon[34391]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:49.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:49 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:50:49.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:49 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:49.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:49 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:49.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:49 smithi192 ceph-mon[34700]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:49.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:49 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:50:49.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:49 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:49.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:49 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:51.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:51 smithi117 ceph-mon[30947]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:51.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:51 smithi138 ceph-mon[34391]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:51.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:51 smithi192 ceph-mon[34700]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:53.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:52 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:53.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:52 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:53.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:52 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:50:53.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:52 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:50:53.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:52 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:53.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:52 smithi138 ceph-mon[34391]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:53.127 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:52 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:53.127 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:52 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:53.127 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:52 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:50:53.127 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:52 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:50:53.127 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:52 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:53.128 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:52 smithi117 ceph-mon[30947]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:53.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:53.165+0000 7f4605ffb640 1 -- 172.21.15.117:0/3818846182 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4610063800 con 0x7f45f80b6580 2024-09-21T05:50:53.167 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 3 on host 'smithi117' 2024-09-21T05:50:53.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:53.167+0000 7f4616d10640 1 -- 172.21.15.117:0/3818846182 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f45f80b6580 msgr2=0x7f45f80b8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:50:53.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:53.167+0000 7f4616d10640 1 --2- 172.21.15.117:0/3818846182 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f45f80b6580 0x7f45f80b8a90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4600005e10 tx=0x7f4600005da0 comp rx=0 tx=0).stop 2024-09-21T05:50:53.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:53.167+0000 7f4616d10640 1 -- 172.21.15.117:0/3818846182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4610103b60 msgr2=0x7f461019a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:50:53.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:53.167+0000 7f4616d10640 1 --2- 172.21.15.117:0/3818846182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4610103b60 0x7f461019a400 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f460c00cce0 tx=0x7f460c00a620 comp rx=0 tx=0).stop 2024-09-21T05:50:53.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:53.167+0000 7f4616d10640 1 -- 172.21.15.117:0/3818846182 shutdown_connections 2024-09-21T05:50:53.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:53.167+0000 7f4616d10640 1 --2- 172.21.15.117:0/3818846182 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f45f80b6580 0x7f45f80b8a90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:50:53.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:53.167+0000 7f4616d10640 1 --2- 172.21.15.117:0/3818846182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4610109df0 0x7f461019fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:50:53.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:53.167+0000 7f4616d10640 1 --2- 172.21.15.117:0/3818846182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46101094a0 0x7f461019a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:50:53.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:53.167+0000 7f4616d10640 1 --2- 172.21.15.117:0/3818846182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4610103b60 0x7f461019a400 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:50:53.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:53.167+0000 7f4616d10640 1 -- 172.21.15.117:0/3818846182 >> 172.21.15.117:0/3818846182 conn(0x7f461007b850 msgr2=0x7f4610105430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:50:53.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:53.167+0000 7f4616d10640 1 -- 172.21.15.117:0/3818846182 shutdown_connections 2024-09-21T05:50:53.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:50:53.167+0000 7f4616d10640 1 -- 172.21.15.117:0/3818846182 wait complete. 2024-09-21T05:50:53.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:52 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:53.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:52 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:53.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:52 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:50:53.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:52 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:50:53.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:52 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:53.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:52 smithi192 ceph-mon[34700]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:53.895 DEBUG:teuthology.orchestra.run.smithi117:osd.3> sudo journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.3.service 2024-09-21T05:50:53.897 INFO:tasks.cephadm:Deploying osd.4 on smithi138 with /dev/vg_nvme/lv_3... 2024-09-21T05:50:53.898 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T05:50:54.150 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T05:50:54.271 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:54 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:50:54.271 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:54 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:54.271 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:54 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:54.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:54 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:50:54.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:54 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:54.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:54 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:54.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:54 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:50:54.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:54 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:54.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:54 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:55.383 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:55 smithi117 ceph-mon[30947]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:55.383 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:55 smithi117 ceph-mon[30947]: from='osd.3 [v2:172.21.15.117:6810/1222400293,v1:172.21.15.117:6811/1222400293]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T05:50:55.383 INFO:journalctl@ceph.osd.3.smithi117.stdout:Sep 21 05:50:55 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-3[53652]: 2024-09-21T05:50:55.139+0000 7f1ae6ee2640 -1 osd.3 0 log_to_monitors true 2024-09-21T05:50:55.423 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:55 smithi138 ceph-mon[34391]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:55.423 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:55 smithi138 ceph-mon[34391]: from='osd.3 [v2:172.21.15.117:6810/1222400293,v1:172.21.15.117:6811/1222400293]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T05:50:55.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:55 smithi192 ceph-mon[34700]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:55.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:55 smithi192 ceph-mon[34700]: from='osd.3 [v2:172.21.15.117:6810/1222400293,v1:172.21.15.117:6811/1222400293]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T05:50:56.331 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:56 smithi138 ceph-mon[34391]: from='osd.3 [v2:172.21.15.117:6810/1222400293,v1:172.21.15.117:6811/1222400293]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T05:50:56.331 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:56 smithi138 ceph-mon[34391]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T05:50:56.331 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:56 smithi138 ceph-mon[34391]: from='osd.3 [v2:172.21.15.117:6810/1222400293,v1:172.21.15.117:6811/1222400293]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-21T05:50:56.331 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:56 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T05:50:56.462 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:56 smithi117 ceph-mon[30947]: from='osd.3 [v2:172.21.15.117:6810/1222400293,v1:172.21.15.117:6811/1222400293]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T05:50:56.462 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:56 smithi117 ceph-mon[30947]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T05:50:56.462 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:56 smithi117 ceph-mon[30947]: from='osd.3 [v2:172.21.15.117:6810/1222400293,v1:172.21.15.117:6811/1222400293]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-21T05:50:56.462 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:56 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T05:50:56.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:56 smithi192 ceph-mon[34700]: from='osd.3 [v2:172.21.15.117:6810/1222400293,v1:172.21.15.117:6811/1222400293]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T05:50:56.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:56 smithi192 ceph-mon[34700]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T05:50:56.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:56 smithi192 ceph-mon[34700]: from='osd.3 [v2:172.21.15.117:6810/1222400293,v1:172.21.15.117:6811/1222400293]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-21T05:50:56.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:56 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T05:50:56.677 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-21T05:50:56.703 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi138:vg_nvme/lv_3 2024-09-21T05:50:56.777 INFO:journalctl@ceph.osd.3.smithi117.stdout:Sep 21 05:50:56 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-3[53652]: 2024-09-21T05:50:56.597+0000 7f1ae3679640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T05:50:56.777 INFO:journalctl@ceph.osd.3.smithi117.stdout:Sep 21 05:50:56 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-3[53652]: 2024-09-21T05:50:56.604+0000 7f1adac5a640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T05:50:56.948 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T05:50:57.353 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:57 smithi138 ceph-mon[34391]: from='osd.3 [v2:172.21.15.117:6810/1222400293,v1:172.21.15.117:6811/1222400293]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-21T05:50:57.353 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:57 smithi138 ceph-mon[34391]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T05:50:57.353 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:57 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T05:50:57.353 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:57 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T05:50:57.353 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:57 smithi138 ceph-mon[34391]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:57.353 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:57 smithi138 ceph-mon[34391]: from='osd.3 [v2:172.21.15.117:6810/1222400293,v1:172.21.15.117:6811/1222400293]' entity='osd.3' 2024-09-21T05:50:57.353 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:57 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:57.353 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:57 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:57.353 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:57 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T05:50:57.353 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:57 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T05:50:57.354 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:57 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:57.354 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:57 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:50:57.354 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:57 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:50:57.354 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:57 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:57.354 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:57 smithi138 ceph-mon[34391]: osd.3 [v2:172.21.15.117:6810/1222400293,v1:172.21.15.117:6811/1222400293] boot 2024-09-21T05:50:57.354 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:57 smithi138 ceph-mon[34391]: osdmap e25: 4 total, 4 up, 4 in 2024-09-21T05:50:57.354 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:57 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T05:50:57.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.443+0000 7f6afe249640 1 -- 172.21.15.138:0/1090082304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af81083b0 msgr2=0x7f6af81087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:50:57.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.443+0000 7f6afe249640 1 --2- 172.21.15.138:0/1090082304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af81083b0 0x7f6af81087c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6aec009900 tx=0x7f6aec02a410 comp rx=0 tx=0).stop 2024-09-21T05:50:57.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.443+0000 7f6afe249640 1 -- 172.21.15.138:0/1090082304 shutdown_connections 2024-09-21T05:50:57.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.443+0000 7f6afe249640 1 --2- 172.21.15.138:0/1090082304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6af8108d00 0x7f6af810f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:50:57.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.443+0000 7f6afe249640 1 --2- 172.21.15.138:0/1090082304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af81083b0 0x7f6af81087c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:50:57.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.443+0000 7f6afe249640 1 --2- 172.21.15.138:0/1090082304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6af8102a70 0x7f6af8102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:50:57.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.443+0000 7f6afe249640 1 -- 172.21.15.138:0/1090082304 >> 172.21.15.138:0/1090082304 conn(0x7f6af8079710 msgr2=0x7f6af8079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:50:57.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.443+0000 7f6afe249640 1 -- 172.21.15.138:0/1090082304 shutdown_connections 2024-09-21T05:50:57.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.443+0000 7f6afe249640 1 -- 172.21.15.138:0/1090082304 wait complete. 2024-09-21T05:50:57.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.444+0000 7f6afe249640 1 Processor -- start 2024-09-21T05:50:57.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.444+0000 7f6afe249640 1 -- start start 2024-09-21T05:50:57.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.444+0000 7f6afe249640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af8102a70 0x7f6af819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:50:57.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.444+0000 7f6afe249640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6af81083b0 0x7f6af819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:50:57.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.444+0000 7f6afe249640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6af8108d00 0x7f6af819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:50:57.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.444+0000 7f6afe249640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6af81129c0 con 0x7f6af81083b0 2024-09-21T05:50:57.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.444+0000 7f6afe249640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6af8112840 con 0x7f6af8102a70 2024-09-21T05:50:57.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.444+0000 7f6afe249640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6af8112b40 con 0x7f6af8108d00 2024-09-21T05:50:57.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.444+0000 7f6af77fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af8102a70 0x7f6af819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:50:57.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.444+0000 7f6af77fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af8102a70 0x7f6af819a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34856/0 (socket says 172.21.15.138:34856) 2024-09-21T05:50:57.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.444+0000 7f6af77fe640 1 -- 172.21.15.138:0/1573195541 learned_addr learned my addr 172.21.15.138:0/1573195541 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-09-21T05:50:57.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.444+0000 7f6af6ffd640 1 --2- 172.21.15.138:0/1573195541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6af81083b0 0x7f6af819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:50:57.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.444+0000 7f6af7fff640 1 --2- 172.21.15.138:0/1573195541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6af8108d00 0x7f6af819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:50:57.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.444+0000 7f6af77fe640 1 -- 172.21.15.138:0/1573195541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6af8108d00 msgr2=0x7f6af819fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:50:57.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.444+0000 7f6af77fe640 1 --2- 172.21.15.138:0/1573195541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6af8108d00 0x7f6af819fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:50:57.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.444+0000 7f6af77fe640 1 -- 172.21.15.138:0/1573195541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6af81083b0 msgr2=0x7f6af819aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:50:57.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.444+0000 7f6af77fe640 1 --2- 172.21.15.138:0/1573195541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6af81083b0 0x7f6af819aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:50:57.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.444+0000 7f6af77fe640 1 -- 172.21.15.138:0/1573195541 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af81a0430 con 0x7f6af8102a70 2024-09-21T05:50:57.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.444+0000 7f6af7fff640 1 --2- 172.21.15.138:0/1573195541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6af8108d00 0x7f6af819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:50:57.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.445+0000 7f6af77fe640 1 --2- 172.21.15.138:0/1573195541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af8102a70 0x7f6af819a4c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6ae800d950 tx=0x7f6ae800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:50:57.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.445+0000 7f6af4ff9640 1 -- 172.21.15.138:0/1573195541 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8017030 con 0x7f6af8102a70 2024-09-21T05:50:57.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.445+0000 7f6af4ff9640 1 -- 172.21.15.138:0/1573195541 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ae8004020 con 0x7f6af8102a70 2024-09-21T05:50:57.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.445+0000 7f6afe249640 1 -- 172.21.15.138:0/1573195541 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af81a0720 con 0x7f6af8102a70 2024-09-21T05:50:57.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.445+0000 7f6af4ff9640 1 -- 172.21.15.138:0/1573195541 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae80047a0 con 0x7f6af8102a70 2024-09-21T05:50:57.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.445+0000 7f6afe249640 1 -- 172.21.15.138:0/1573195541 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af81a6e80 con 0x7f6af8102a70 2024-09-21T05:50:57.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.446+0000 7f6afe249640 1 -- 172.21.15.138:0/1573195541 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6af8103cf0 con 0x7f6af8102a70 2024-09-21T05:50:57.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.447+0000 7f6af4ff9640 1 -- 172.21.15.138:0/1573195541 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f6ae8009a30 con 0x7f6af8102a70 2024-09-21T05:50:57.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.448+0000 7f6af4ff9640 1 --2- 172.21.15.138:0/1573195541 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6ae00b6560 0x7f6ae00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:50:57.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.448+0000 7f6af4ff9640 1 -- 172.21.15.138:0/1573195541 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 3917+0+0 (secure 0 0 0) 0x7f6ae80e5dc0 con 0x7f6af8102a70 2024-09-21T05:50:57.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.448+0000 7f6af6ffd640 1 --2- 172.21.15.138:0/1573195541 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6ae00b6560 0x7f6ae00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:50:57.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.449+0000 7f6af6ffd640 1 --2- 172.21.15.138:0/1573195541 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6ae00b6560 0x7f6ae00b8a70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6aec009230 tx=0x7f6aec031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:50:57.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.452+0000 7f6af4ff9640 1 -- 172.21.15.138:0/1573195541 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6ae80ad780 con 0x7f6af8102a70 2024-09-21T05:50:57.464 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:57 smithi117 ceph-mon[30947]: from='osd.3 [v2:172.21.15.117:6810/1222400293,v1:172.21.15.117:6811/1222400293]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-21T05:50:57.464 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:57 smithi117 ceph-mon[30947]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T05:50:57.465 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:57 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T05:50:57.465 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:57 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T05:50:57.465 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:57 smithi117 ceph-mon[30947]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:57.465 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:57 smithi117 ceph-mon[30947]: from='osd.3 [v2:172.21.15.117:6810/1222400293,v1:172.21.15.117:6811/1222400293]' entity='osd.3' 2024-09-21T05:50:57.465 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:57 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:57.465 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:57 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:57.465 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:57 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T05:50:57.465 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:57 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T05:50:57.465 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:57 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:57.465 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:57 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:50:57.465 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:57 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:50:57.465 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:57 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:57.465 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:57 smithi117 ceph-mon[30947]: osd.3 [v2:172.21.15.117:6810/1222400293,v1:172.21.15.117:6811/1222400293] boot 2024-09-21T05:50:57.466 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:57 smithi117 ceph-mon[30947]: osdmap e25: 4 total, 4 up, 4 in 2024-09-21T05:50:57.466 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:57 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T05:50:57.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:57 smithi192 ceph-mon[34700]: from='osd.3 [v2:172.21.15.117:6810/1222400293,v1:172.21.15.117:6811/1222400293]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-21T05:50:57.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:57 smithi192 ceph-mon[34700]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T05:50:57.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:57 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T05:50:57.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:57 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T05:50:57.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:57 smithi192 ceph-mon[34700]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-21T05:50:57.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:57 smithi192 ceph-mon[34700]: from='osd.3 [v2:172.21.15.117:6810/1222400293,v1:172.21.15.117:6811/1222400293]' entity='osd.3' 2024-09-21T05:50:57.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:57 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:57.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:57 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:57.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:57 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T05:50:57.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:57 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T05:50:57.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:57 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:57.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:57 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:50:57.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:57 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:50:57.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:57 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:50:57.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:57 smithi192 ceph-mon[34700]: osd.3 [v2:172.21.15.117:6810/1222400293,v1:172.21.15.117:6811/1222400293] boot 2024-09-21T05:50:57.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:57 smithi192 ceph-mon[34700]: osdmap e25: 4 total, 4 up, 4 in 2024-09-21T05:50:57.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:57 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T05:50:57.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:50:57.638+0000 7f6afe249640 1 -- 172.21.15.138:0/1573195541 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f6af810bb30 con 0x7f6ae00b6560 2024-09-21T05:50:58.322 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:58 smithi138 ceph-mon[34391]: purged_snaps scrub starts 2024-09-21T05:50:58.323 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:58 smithi138 ceph-mon[34391]: purged_snaps scrub ok 2024-09-21T05:50:58.323 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:58 smithi138 ceph-mon[34391]: Detected new or changed devices on smithi117 2024-09-21T05:50:58.323 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:58 smithi138 ceph-mon[34391]: Adjusting osd_memory_target on smithi117 to 8544M 2024-09-21T05:50:58.323 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:58 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:50:58.323 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:58 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:50:58.323 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:58 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:50:58.323 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:58 smithi138 ceph-mon[34391]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T05:50:58.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:58 smithi192 ceph-mon[34700]: purged_snaps scrub starts 2024-09-21T05:50:58.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:58 smithi192 ceph-mon[34700]: purged_snaps scrub ok 2024-09-21T05:50:58.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:58 smithi192 ceph-mon[34700]: Detected new or changed devices on smithi117 2024-09-21T05:50:58.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:58 smithi192 ceph-mon[34700]: Adjusting osd_memory_target on smithi117 to 8544M 2024-09-21T05:50:58.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:58 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:50:58.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:58 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:50:58.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:58 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:50:58.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:58 smithi192 ceph-mon[34700]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T05:50:58.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:58 smithi117 ceph-mon[30947]: purged_snaps scrub starts 2024-09-21T05:50:58.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:58 smithi117 ceph-mon[30947]: purged_snaps scrub ok 2024-09-21T05:50:58.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:58 smithi117 ceph-mon[30947]: Detected new or changed devices on smithi117 2024-09-21T05:50:58.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:58 smithi117 ceph-mon[30947]: Adjusting osd_memory_target on smithi117 to 8544M 2024-09-21T05:50:58.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:58 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:50:58.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:58 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:50:58.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:58 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:50:58.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:58 smithi117 ceph-mon[30947]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T05:50:59.247 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:59 smithi138 ceph-mon[34391]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:50:59.247 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:59 smithi138 ceph-mon[34391]: pgmap v87: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-21T05:50:59.247 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:50:59 smithi138 ceph-mon[34391]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T05:50:59.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:59 smithi192 ceph-mon[34700]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:50:59.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:59 smithi192 ceph-mon[34700]: pgmap v87: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-21T05:50:59.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:50:59 smithi192 ceph-mon[34700]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T05:50:59.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:59 smithi117 ceph-mon[30947]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:50:59.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:59 smithi117 ceph-mon[30947]: pgmap v87: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-21T05:50:59.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:50:59 smithi117 ceph-mon[30947]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T05:51:00.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:00 smithi192 ceph-mon[34700]: from='client.? 172.21.15.138:0/2988227575' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "28f1d7c0-8392-4a33-85c0-0de611cdd4b1"} : dispatch 2024-09-21T05:51:00.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:00 smithi192 ceph-mon[34700]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "28f1d7c0-8392-4a33-85c0-0de611cdd4b1"} : dispatch 2024-09-21T05:51:00.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:00 smithi192 ceph-mon[34700]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28f1d7c0-8392-4a33-85c0-0de611cdd4b1"}]': finished 2024-09-21T05:51:00.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:00 smithi192 ceph-mon[34700]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T05:51:00.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:00 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T05:51:00.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:00 smithi117 ceph-mon[30947]: from='client.? 172.21.15.138:0/2988227575' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "28f1d7c0-8392-4a33-85c0-0de611cdd4b1"} : dispatch 2024-09-21T05:51:00.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:00 smithi117 ceph-mon[30947]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "28f1d7c0-8392-4a33-85c0-0de611cdd4b1"} : dispatch 2024-09-21T05:51:00.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:00 smithi117 ceph-mon[30947]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28f1d7c0-8392-4a33-85c0-0de611cdd4b1"}]': finished 2024-09-21T05:51:00.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:00 smithi117 ceph-mon[30947]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T05:51:00.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:00 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T05:51:00.563 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:00 smithi138 ceph-mon[34391]: from='client.? 172.21.15.138:0/2988227575' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "28f1d7c0-8392-4a33-85c0-0de611cdd4b1"} : dispatch 2024-09-21T05:51:00.563 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:00 smithi138 ceph-mon[34391]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "28f1d7c0-8392-4a33-85c0-0de611cdd4b1"} : dispatch 2024-09-21T05:51:00.563 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:00 smithi138 ceph-mon[34391]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28f1d7c0-8392-4a33-85c0-0de611cdd4b1"}]': finished 2024-09-21T05:51:00.563 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:00 smithi138 ceph-mon[34391]: osdmap e28: 5 total, 4 up, 5 in 2024-09-21T05:51:00.563 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:00 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T05:51:01.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:01 smithi192 ceph-mon[34700]: pgmap v90: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-21T05:51:01.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:01 smithi192 ceph-mon[34700]: from='client.? 172.21.15.138:0/959183511' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:51:01.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:01 smithi117 ceph-mon[30947]: pgmap v90: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-21T05:51:01.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:01 smithi117 ceph-mon[30947]: from='client.? 172.21.15.138:0/959183511' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:51:01.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:01 smithi138 ceph-mon[34391]: pgmap v90: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-21T05:51:01.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:01 smithi138 ceph-mon[34391]: from='client.? 172.21.15.138:0/959183511' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:51:03.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:03 smithi117 ceph-mon[30947]: pgmap v91: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-21T05:51:03.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:03 smithi138 ceph-mon[34391]: pgmap v91: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-21T05:51:03.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:03 smithi192 ceph-mon[34700]: pgmap v91: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-21T05:51:05.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:05 smithi117 ceph-mon[30947]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-21T05:51:05.777 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:05 smithi138 ceph-mon[34391]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-21T05:51:05.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:05 smithi192 ceph-mon[34700]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-21T05:51:07.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:07 smithi117 ceph-mon[30947]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T05:51:07.848 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:07 smithi138 ceph-mon[34391]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T05:51:07.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:07 smithi192 ceph-mon[34700]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T05:51:09.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:09 smithi117 ceph-mon[30947]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-21T05:51:09.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:09 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T05:51:09.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:09 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:51:09.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:09 smithi117 ceph-mon[30947]: Deploying daemon osd.4 on smithi138 2024-09-21T05:51:09.838 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:09 smithi138 ceph-mon[34391]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-21T05:51:09.839 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:09 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T05:51:09.839 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:09 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:51:09.839 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:09 smithi138 ceph-mon[34391]: Deploying daemon osd.4 on smithi138 2024-09-21T05:51:09.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:09 smithi192 ceph-mon[34700]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-21T05:51:09.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:09 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T05:51:09.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:09 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:51:09.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:09 smithi192 ceph-mon[34700]: Deploying daemon osd.4 on smithi138 2024-09-21T05:51:11.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:11 smithi138 ceph-mon[34391]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-21T05:51:11.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:11 smithi117 ceph-mon[30947]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-21T05:51:11.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:11 smithi192 ceph-mon[34700]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-21T05:51:13.680 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:13 smithi138 ceph-mon[34391]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T05:51:13.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:13 smithi117 ceph-mon[30947]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T05:51:13.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:13 smithi192 ceph-mon[34700]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T05:51:15.678 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:15 smithi138 ceph-mon[34391]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T05:51:15.678 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:15 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:51:15.678 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:15 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:15.678 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:15 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:15.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:15 smithi117 ceph-mon[30947]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T05:51:15.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:15 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:51:15.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:15 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:15.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:15 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:15.782 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:15 smithi192 ceph-mon[34700]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T05:51:15.782 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:15 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:51:15.782 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:15 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:15.782 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:15 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:17.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:17 smithi117 ceph-mon[30947]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-21T05:51:17.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:17 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:17.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:17 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:17.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:17 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:51:17.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:17 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:51:17.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:17 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:17.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:17 smithi138 ceph-mon[34391]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-21T05:51:17.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:17 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:17.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:17 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:17.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:17 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:51:17.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:17 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:51:17.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:17 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:17.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:17 smithi192 ceph-mon[34700]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-21T05:51:17.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:17 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:17.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:17 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:17.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:17 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:51:17.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:17 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:51:17.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:17 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:19.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:51:19.121+0000 7f6af4ff9640 1 -- 172.21.15.138:0/1573195541 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6af810bb30 con 0x7f6ae00b6560 2024-09-21T05:51:19.123 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 4 on host 'smithi138' 2024-09-21T05:51:19.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:51:19.124+0000 7f6afe249640 1 -- 172.21.15.138:0/1573195541 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6ae00b6560 msgr2=0x7f6ae00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:51:19.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:51:19.124+0000 7f6afe249640 1 --2- 172.21.15.138:0/1573195541 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6ae00b6560 0x7f6ae00b8a70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6aec009230 tx=0x7f6aec031400 comp rx=0 tx=0).stop 2024-09-21T05:51:19.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:51:19.124+0000 7f6afe249640 1 -- 172.21.15.138:0/1573195541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af8102a70 msgr2=0x7f6af819a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:51:19.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:51:19.124+0000 7f6afe249640 1 --2- 172.21.15.138:0/1573195541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af8102a70 0x7f6af819a4c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6ae800d950 tx=0x7f6ae800de10 comp rx=0 tx=0).stop 2024-09-21T05:51:19.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:51:19.124+0000 7f6afe249640 1 -- 172.21.15.138:0/1573195541 shutdown_connections 2024-09-21T05:51:19.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:51:19.124+0000 7f6afe249640 1 --2- 172.21.15.138:0/1573195541 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6ae00b6560 0x7f6ae00b8a70 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:51:19.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:51:19.124+0000 7f6afe249640 1 --2- 172.21.15.138:0/1573195541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6af81083b0 0x7f6af819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:51:19.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:51:19.124+0000 7f6afe249640 1 --2- 172.21.15.138:0/1573195541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6af8108d00 0x7f6af819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:51:19.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:51:19.124+0000 7f6afe249640 1 --2- 172.21.15.138:0/1573195541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af8102a70 0x7f6af819a4c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:51:19.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:51:19.124+0000 7f6afe249640 1 -- 172.21.15.138:0/1573195541 >> 172.21.15.138:0/1573195541 conn(0x7f6af8079710 msgr2=0x7f6af8101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:51:19.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:51:19.124+0000 7f6afe249640 1 -- 172.21.15.138:0/1573195541 shutdown_connections 2024-09-21T05:51:19.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:51:19.124+0000 7f6afe249640 1 -- 172.21.15.138:0/1573195541 wait complete. 2024-09-21T05:51:19.797 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:19 smithi138 ceph-mon[34391]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-21T05:51:19.798 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:19 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:51:19.798 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:19 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:19.798 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:19 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:19.826 DEBUG:teuthology.orchestra.run.smithi138:osd.4> sudo journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.4.service 2024-09-21T05:51:19.828 INFO:tasks.cephadm:Deploying osd.5 on smithi192 with /dev/vg_nvme/lv_3... 2024-09-21T05:51:19.829 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T05:51:19.856 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:19 smithi192 ceph-mon[34700]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-21T05:51:19.856 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:19 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:51:19.856 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:19 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:19.856 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:19 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:20.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:19 smithi117 ceph-mon[30947]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-21T05:51:20.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:19 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:51:20.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:19 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:20.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:19 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:20.081 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T05:51:21.308 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 21 05:51:20 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-4[45287]: 2024-09-21T05:51:20.956+0000 7fab36abb640 -1 osd.4 0 log_to_monitors true 2024-09-21T05:51:21.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:21 smithi138 ceph-mon[34391]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-21T05:51:21.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:21 smithi138 ceph-mon[34391]: from='osd.4 [v2:172.21.15.138:6808/4294892870,v1:172.21.15.138:6809/4294892870]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T05:51:21.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:21 smithi138 ceph-mon[34391]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T05:51:21.671 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:21 smithi192 ceph-mon[34700]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-21T05:51:21.671 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:21 smithi192 ceph-mon[34700]: from='osd.4 [v2:172.21.15.138:6808/4294892870,v1:172.21.15.138:6809/4294892870]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T05:51:21.671 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:21 smithi192 ceph-mon[34700]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T05:51:22.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:21 smithi117 ceph-mon[30947]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-21T05:51:22.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:21 smithi117 ceph-mon[30947]: from='osd.4 [v2:172.21.15.138:6808/4294892870,v1:172.21.15.138:6809/4294892870]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T05:51:22.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:21 smithi117 ceph-mon[30947]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T05:51:22.406 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-21T05:51:22.432 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi192:vg_nvme/lv_3 2024-09-21T05:51:22.706 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:22 smithi192 ceph-mon[34700]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T05:51:22.706 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:22 smithi192 ceph-mon[34700]: from='osd.4 [v2:172.21.15.138:6808/4294892870,v1:172.21.15.138:6809/4294892870]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-21T05:51:22.707 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:22 smithi192 ceph-mon[34700]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T05:51:22.707 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:22 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T05:51:22.707 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:22 smithi192 ceph-mon[34700]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-21T05:51:22.707 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:22 smithi192 ceph-mon[34700]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-21T05:51:22.743 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T05:51:22.829 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:22 smithi138 ceph-mon[34391]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T05:51:22.830 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:22 smithi138 ceph-mon[34391]: from='osd.4 [v2:172.21.15.138:6808/4294892870,v1:172.21.15.138:6809/4294892870]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-21T05:51:22.830 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:22 smithi138 ceph-mon[34391]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T05:51:22.830 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:22 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T05:51:22.830 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:22 smithi138 ceph-mon[34391]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-21T05:51:22.830 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:22 smithi138 ceph-mon[34391]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-21T05:51:22.830 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 21 05:51:22 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-4[45287]: 2024-09-21T05:51:22.540+0000 7fab32a3d640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T05:51:22.830 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 21 05:51:22 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-4[45287]: 2024-09-21T05:51:22.548+0000 7fab2e83b640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T05:51:23.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:22 smithi117 ceph-mon[30947]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T05:51:23.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:22 smithi117 ceph-mon[30947]: from='osd.4 [v2:172.21.15.138:6808/4294892870,v1:172.21.15.138:6809/4294892870]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-21T05:51:23.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:22 smithi117 ceph-mon[30947]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T05:51:23.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:22 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T05:51:23.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:22 smithi117 ceph-mon[30947]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-21T05:51:23.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:22 smithi117 ceph-mon[30947]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-21T05:51:23.176 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.174+0000 7f4d75bb0640 1 -- 172.21.15.192:0/3290345486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d70107b30 msgr2=0x7f4d70107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:51:23.177 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.174+0000 7f4d75bb0640 1 --2- 172.21.15.192:0/3290345486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d70107b30 0x7f4d70107f40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4d64009900 tx=0x7f4d6402a410 comp rx=0 tx=0).stop 2024-09-21T05:51:23.177 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.174+0000 7f4d75bb0640 1 -- 172.21.15.192:0/3290345486 shutdown_connections 2024-09-21T05:51:23.177 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.174+0000 7f4d75bb0640 1 --2- 172.21.15.192:0/3290345486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d70108480 0x7f4d7010ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:51:23.177 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.174+0000 7f4d75bb0640 1 --2- 172.21.15.192:0/3290345486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d70107b30 0x7f4d70107f40 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:51:23.177 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.174+0000 7f4d75bb0640 1 --2- 172.21.15.192:0/3290345486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d701021f0 0x7f4d70102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:51:23.177 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.174+0000 7f4d75bb0640 1 -- 172.21.15.192:0/3290345486 >> 172.21.15.192:0/3290345486 conn(0x7f4d70077c60 msgr2=0x7f4d700780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:51:23.177 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.174+0000 7f4d75bb0640 1 -- 172.21.15.192:0/3290345486 shutdown_connections 2024-09-21T05:51:23.177 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.174+0000 7f4d75bb0640 1 -- 172.21.15.192:0/3290345486 wait complete. 2024-09-21T05:51:23.179 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.175+0000 7f4d75bb0640 1 Processor -- start 2024-09-21T05:51:23.179 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.175+0000 7f4d75bb0640 1 -- start start 2024-09-21T05:51:23.179 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.175+0000 7f4d75bb0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d701021f0 0x7f4d7019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:51:23.179 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.175+0000 7f4d75bb0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d70107b30 0x7f4d7019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:51:23.179 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.175+0000 7f4d75bb0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d70108480 0x7f4d7019fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:51:23.179 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.175+0000 7f4d75bb0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4d70113ab0 con 0x7f4d70107b30 2024-09-21T05:51:23.179 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.175+0000 7f4d75bb0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4d70113930 con 0x7f4d701021f0 2024-09-21T05:51:23.179 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.175+0000 7f4d75bb0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4d70113c30 con 0x7f4d70108480 2024-09-21T05:51:23.179 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.176+0000 7f4d6ffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d70108480 0x7f4d7019fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:51:23.179 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.176+0000 7f4d6ffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d70108480 0x7f4d7019fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:43866/0 (socket says 172.21.15.192:43866) 2024-09-21T05:51:23.180 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.176+0000 7f4d6ffff640 1 -- 172.21.15.192:0/585456701 learned_addr learned my addr 172.21.15.192:0/585456701 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-09-21T05:51:23.180 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.176+0000 7f4d6effd640 1 --2- 172.21.15.192:0/585456701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d70107b30 0x7f4d7019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:51:23.180 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.176+0000 7f4d6f7fe640 1 --2- 172.21.15.192:0/585456701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d701021f0 0x7f4d7019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:51:23.180 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.176+0000 7f4d6ffff640 1 -- 172.21.15.192:0/585456701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d701021f0 msgr2=0x7f4d7019a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:51:23.180 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.176+0000 7f4d6ffff640 1 --2- 172.21.15.192:0/585456701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d701021f0 0x7f4d7019a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:51:23.180 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.176+0000 7f4d6ffff640 1 -- 172.21.15.192:0/585456701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d70107b30 msgr2=0x7f4d7019aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:51:23.180 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.176+0000 7f4d6ffff640 1 --2- 172.21.15.192:0/585456701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d70107b30 0x7f4d7019aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:51:23.180 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.176+0000 7f4d6ffff640 1 -- 172.21.15.192:0/585456701 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d701a0490 con 0x7f4d70108480 2024-09-21T05:51:23.180 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.176+0000 7f4d6f7fe640 1 --2- 172.21.15.192:0/585456701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d701021f0 0x7f4d7019a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:51:23.180 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.176+0000 7f4d6ffff640 1 --2- 172.21.15.192:0/585456701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d70108480 0x7f4d7019fd60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4d6800eaa0 tx=0x7f4d6800ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:51:23.180 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.176+0000 7f4d6cff9640 1 -- 172.21.15.192:0/585456701 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6800cdb0 con 0x7f4d70108480 2024-09-21T05:51:23.180 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.176+0000 7f4d6cff9640 1 -- 172.21.15.192:0/585456701 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d68005020 con 0x7f4d70108480 2024-09-21T05:51:23.180 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.176+0000 7f4d75bb0640 1 -- 172.21.15.192:0/585456701 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d701a0780 con 0x7f4d70108480 2024-09-21T05:51:23.180 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.176+0000 7f4d6cff9640 1 -- 172.21.15.192:0/585456701 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d680116d0 con 0x7f4d70108480 2024-09-21T05:51:23.180 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.176+0000 7f4d75bb0640 1 -- 172.21.15.192:0/585456701 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d701a6ee0 con 0x7f4d70108480 2024-09-21T05:51:23.181 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.178+0000 7f4d75bb0640 1 -- 172.21.15.192:0/585456701 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d70103470 con 0x7f4d70108480 2024-09-21T05:51:23.182 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.180+0000 7f4d6cff9640 1 -- 172.21.15.192:0/585456701 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f4d680051d0 con 0x7f4d70108480 2024-09-21T05:51:23.183 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.181+0000 7f4d6cff9640 1 --2- 172.21.15.192:0/585456701 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4d580b6560 0x7f4d580b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:51:23.183 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.181+0000 7f4d6cff9640 1 -- 172.21.15.192:0/585456701 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4036+0+0 (secure 0 0 0) 0x7f4d680e6830 con 0x7f4d70108480 2024-09-21T05:51:23.183 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.181+0000 7f4d6f7fe640 1 --2- 172.21.15.192:0/585456701 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4d580b6560 0x7f4d580b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:51:23.184 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.182+0000 7f4d6f7fe640 1 --2- 172.21.15.192:0/585456701 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4d580b6560 0x7f4d580b8a70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4d600058c0 tx=0x7f4d60005ec0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:51:23.186 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.183+0000 7f4d6cff9640 1 -- 172.21.15.192:0/585456701 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4d68016030 con 0x7f4d70108480 2024-09-21T05:51:23.380 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:23.378+0000 7f4d75bb0640 1 -- 172.21.15.192:0/585456701 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4d70063800 con 0x7f4d580b6560 2024-09-21T05:51:23.708 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:23 smithi192 ceph-mon[34700]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-21T05:51:23.708 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:23 smithi192 ceph-mon[34700]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T05:51:23.708 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T05:51:23.708 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T05:51:23.708 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:23.708 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:23.708 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T05:51:23.708 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T05:51:23.708 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:23.709 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:51:23.709 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:51:23.709 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:23.709 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:51:23.709 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:51:23.709 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:51:23.709 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T05:51:23.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:23 smithi138 ceph-mon[34391]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-21T05:51:23.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:23 smithi138 ceph-mon[34391]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T05:51:23.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T05:51:23.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T05:51:23.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:23.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:23.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T05:51:23.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T05:51:23.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:23.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:51:23.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:51:23.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:23.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:51:23.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:51:23.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:51:23.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T05:51:24.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:23 smithi117 ceph-mon[30947]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-21T05:51:24.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:23 smithi117 ceph-mon[30947]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T05:51:24.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T05:51:24.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T05:51:24.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:24.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:24.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T05:51:24.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T05:51:24.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:24.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:51:24.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:51:24.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:24.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:51:24.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:51:24.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:51:24.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T05:51:24.729 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:24 smithi192 ceph-mon[34700]: purged_snaps scrub starts 2024-09-21T05:51:24.729 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:24 smithi192 ceph-mon[34700]: purged_snaps scrub ok 2024-09-21T05:51:24.729 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:24 smithi192 ceph-mon[34700]: Detected new or changed devices on smithi138 2024-09-21T05:51:24.729 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:24 smithi192 ceph-mon[34700]: Adjusting osd_memory_target on smithi138 to 8544M 2024-09-21T05:51:24.729 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:24 smithi192 ceph-mon[34700]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:51:24.729 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:24 smithi192 ceph-mon[34700]: osd.4 [v2:172.21.15.138:6808/4294892870,v1:172.21.15.138:6809/4294892870] boot 2024-09-21T05:51:24.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:24 smithi192 ceph-mon[34700]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T05:51:24.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:24 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T05:51:24.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:24 smithi138 ceph-mon[34391]: purged_snaps scrub starts 2024-09-21T05:51:24.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:24 smithi138 ceph-mon[34391]: purged_snaps scrub ok 2024-09-21T05:51:24.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:24 smithi138 ceph-mon[34391]: Detected new or changed devices on smithi138 2024-09-21T05:51:24.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:24 smithi138 ceph-mon[34391]: Adjusting osd_memory_target on smithi138 to 8544M 2024-09-21T05:51:24.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:24 smithi138 ceph-mon[34391]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:51:24.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:24 smithi138 ceph-mon[34391]: osd.4 [v2:172.21.15.138:6808/4294892870,v1:172.21.15.138:6809/4294892870] boot 2024-09-21T05:51:24.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:24 smithi138 ceph-mon[34391]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T05:51:24.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:24 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T05:51:25.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:24 smithi117 ceph-mon[30947]: purged_snaps scrub starts 2024-09-21T05:51:25.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:24 smithi117 ceph-mon[30947]: purged_snaps scrub ok 2024-09-21T05:51:25.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:24 smithi117 ceph-mon[30947]: Detected new or changed devices on smithi138 2024-09-21T05:51:25.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:24 smithi117 ceph-mon[30947]: Adjusting osd_memory_target on smithi138 to 8544M 2024-09-21T05:51:25.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:24 smithi117 ceph-mon[30947]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:51:25.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:24 smithi117 ceph-mon[30947]: osd.4 [v2:172.21.15.138:6808/4294892870,v1:172.21.15.138:6809/4294892870] boot 2024-09-21T05:51:25.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:24 smithi117 ceph-mon[30947]: osdmap e31: 5 total, 5 up, 5 in 2024-09-21T05:51:25.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:24 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T05:51:25.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:25 smithi192 ceph-mon[34700]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-21T05:51:25.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:25 smithi192 ceph-mon[34700]: osdmap e32: 5 total, 5 up, 5 in 2024-09-21T05:51:25.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:25 smithi138 ceph-mon[34391]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-21T05:51:25.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:25 smithi138 ceph-mon[34391]: osdmap e32: 5 total, 5 up, 5 in 2024-09-21T05:51:26.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:25 smithi117 ceph-mon[30947]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-09-21T05:51:26.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:25 smithi117 ceph-mon[30947]: osdmap e32: 5 total, 5 up, 5 in 2024-09-21T05:51:26.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:26 smithi192 ceph-mon[34700]: from='client.? 172.21.15.192:0/2989870005' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ca902407-45e8-4bc5-bdbd-a729c1de8322"} : dispatch 2024-09-21T05:51:26.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:26 smithi192 ceph-mon[34700]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ca902407-45e8-4bc5-bdbd-a729c1de8322"} : dispatch 2024-09-21T05:51:26.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:26 smithi192 ceph-mon[34700]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca902407-45e8-4bc5-bdbd-a729c1de8322"}]': finished 2024-09-21T05:51:26.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:26 smithi192 ceph-mon[34700]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T05:51:26.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:26 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T05:51:26.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:26 smithi192 ceph-mon[34700]: from='client.? 172.21.15.192:0/2373290801' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:51:26.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:26 smithi138 ceph-mon[34391]: from='client.? 172.21.15.192:0/2989870005' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ca902407-45e8-4bc5-bdbd-a729c1de8322"} : dispatch 2024-09-21T05:51:26.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:26 smithi138 ceph-mon[34391]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ca902407-45e8-4bc5-bdbd-a729c1de8322"} : dispatch 2024-09-21T05:51:26.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:26 smithi138 ceph-mon[34391]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca902407-45e8-4bc5-bdbd-a729c1de8322"}]': finished 2024-09-21T05:51:26.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:26 smithi138 ceph-mon[34391]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T05:51:26.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:26 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T05:51:26.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:26 smithi138 ceph-mon[34391]: from='client.? 172.21.15.192:0/2373290801' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:51:27.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:26 smithi117 ceph-mon[30947]: from='client.? 172.21.15.192:0/2989870005' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ca902407-45e8-4bc5-bdbd-a729c1de8322"} : dispatch 2024-09-21T05:51:27.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:26 smithi117 ceph-mon[30947]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ca902407-45e8-4bc5-bdbd-a729c1de8322"} : dispatch 2024-09-21T05:51:27.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:26 smithi117 ceph-mon[30947]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca902407-45e8-4bc5-bdbd-a729c1de8322"}]': finished 2024-09-21T05:51:27.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:26 smithi117 ceph-mon[30947]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T05:51:27.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:26 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T05:51:27.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:26 smithi117 ceph-mon[30947]: from='client.? 172.21.15.192:0/2373290801' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:51:27.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:27 smithi117 ceph-mon[30947]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:27.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:27 smithi138 ceph-mon[34391]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:27.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:27 smithi192 ceph-mon[34700]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:29.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:29 smithi138 ceph-mon[34391]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:29.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:29 smithi192 ceph-mon[34700]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:30.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:29 smithi117 ceph-mon[30947]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:31.605 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:31 smithi192 ceph-mon[34700]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:31.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:31 smithi138 ceph-mon[34391]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:32.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:31 smithi117 ceph-mon[30947]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:33.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:33 smithi138 ceph-mon[34391]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:33.909 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:33 smithi192 ceph-mon[34700]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:34.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:33 smithi117 ceph-mon[30947]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:34.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:34 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T05:51:34.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:34 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:51:34.904 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:34 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T05:51:34.904 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:34 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:51:35.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:34 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T05:51:35.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:34 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:51:35.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:35 smithi138 ceph-mon[34391]: Deploying daemon osd.5 on smithi192 2024-09-21T05:51:35.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:35 smithi138 ceph-mon[34391]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:35.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:35 smithi192 ceph-mon[34700]: Deploying daemon osd.5 on smithi192 2024-09-21T05:51:35.869 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:35 smithi192 ceph-mon[34700]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:36.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:35 smithi117 ceph-mon[30947]: Deploying daemon osd.5 on smithi192 2024-09-21T05:51:36.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:35 smithi117 ceph-mon[30947]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:37.861 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:37 smithi192 ceph-mon[34700]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:37.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:37 smithi138 ceph-mon[34391]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:38.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:37 smithi117 ceph-mon[30947]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:39.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:39 smithi138 ceph-mon[34391]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:39.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:39 smithi192 ceph-mon[34700]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:40.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:39 smithi117 ceph-mon[30947]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:40.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:40 smithi192 ceph-mon[34700]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:40.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:40 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:51:40.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:40 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:40.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:40 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:41.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:40 smithi117 ceph-mon[30947]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:41.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:40 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:51:41.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:40 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:41.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:40 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:41.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:40 smithi138 ceph-mon[34391]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:41.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:40 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:51:41.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:40 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:41.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:40 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:43.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:43 smithi192 ceph-mon[34700]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:43.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:43 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:43.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:43 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:43.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:43 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:51:43.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:43 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:51:43.618 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:43 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:43.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:43 smithi117 ceph-mon[30947]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:43.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:43 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:43.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:43 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:43.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:43 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:51:43.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:43 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:51:43.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:43 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:43.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:43 smithi138 ceph-mon[34391]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:43.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:43 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:43.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:43 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:43.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:43 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:51:43.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:43 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:51:43.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:43 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:43.993 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 5 on host 'smithi192' 2024-09-21T05:51:43.994 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:43.991+0000 7f4d6cff9640 1 -- 172.21.15.192:0/585456701 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4d70063800 con 0x7f4d580b6560 2024-09-21T05:51:43.996 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:43.994+0000 7f4d75bb0640 1 -- 172.21.15.192:0/585456701 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4d580b6560 msgr2=0x7f4d580b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:51:43.996 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:43.994+0000 7f4d75bb0640 1 --2- 172.21.15.192:0/585456701 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4d580b6560 0x7f4d580b8a70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4d600058c0 tx=0x7f4d60005ec0 comp rx=0 tx=0).stop 2024-09-21T05:51:43.996 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:43.994+0000 7f4d75bb0640 1 -- 172.21.15.192:0/585456701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d70108480 msgr2=0x7f4d7019fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:51:43.996 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:43.994+0000 7f4d75bb0640 1 --2- 172.21.15.192:0/585456701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d70108480 0x7f4d7019fd60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4d6800eaa0 tx=0x7f4d6800ef60 comp rx=0 tx=0).stop 2024-09-21T05:51:43.996 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:43.994+0000 7f4d75bb0640 1 -- 172.21.15.192:0/585456701 shutdown_connections 2024-09-21T05:51:43.996 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:43.994+0000 7f4d75bb0640 1 --2- 172.21.15.192:0/585456701 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4d580b6560 0x7f4d580b8a70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:51:43.996 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:43.994+0000 7f4d75bb0640 1 --2- 172.21.15.192:0/585456701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d70107b30 0x7f4d7019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:51:43.996 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:43.994+0000 7f4d75bb0640 1 --2- 172.21.15.192:0/585456701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d70108480 0x7f4d7019fd60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:51:43.997 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:43.994+0000 7f4d75bb0640 1 --2- 172.21.15.192:0/585456701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d701021f0 0x7f4d7019a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:51:43.997 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:43.994+0000 7f4d75bb0640 1 -- 172.21.15.192:0/585456701 >> 172.21.15.192:0/585456701 conn(0x7f4d70077c60 msgr2=0x7f4d70103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:51:43.997 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:43.994+0000 7f4d75bb0640 1 -- 172.21.15.192:0/585456701 shutdown_connections 2024-09-21T05:51:43.997 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:51:43.994+0000 7f4d75bb0640 1 -- 172.21.15.192:0/585456701 wait complete. 2024-09-21T05:51:44.589 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:44 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:51:44.590 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:44 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:44.590 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:44 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:44.615 DEBUG:teuthology.orchestra.run.smithi192:osd.5> sudo journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.5.service 2024-09-21T05:51:44.618 INFO:tasks.cephadm:Deploying osd.6 on smithi117 with /dev/vg_nvme/lv_2... 2024-09-21T05:51:44.618 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T05:51:44.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:44 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:51:44.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:44 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:44.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:44 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:44.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:44 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:51:44.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:44 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:44.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:44 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:44.877 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:51:45.767 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:45 smithi138 ceph-mon[34391]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:45.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:45 smithi117 ceph-mon[30947]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:45.889 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:45 smithi192 ceph-mon[34700]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:47.128 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 05:51:46 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T05:51:46.740+0000 7f3d78122640 -1 osd.5 0 log_to_monitors true 2024-09-21T05:51:47.526 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:47 smithi117 ceph-mon[30947]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:47.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:47 smithi117 ceph-mon[30947]: from='osd.5 [v2:172.21.15.192:6808/1776840019,v1:172.21.15.192:6809/1776840019]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T05:51:47.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:47 smithi117 ceph-mon[30947]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T05:51:47.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:47 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:47.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:47 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:47.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:47 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T05:51:47.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:47 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T05:51:47.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:47 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:47.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:47 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:51:47.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:47 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:51:47.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:47 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:47.700 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:51:47.725 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi117:vg_nvme/lv_2 2024-09-21T05:51:47.725 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:47 smithi192 ceph-mon[34700]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:47.726 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:47 smithi192 ceph-mon[34700]: from='osd.5 [v2:172.21.15.192:6808/1776840019,v1:172.21.15.192:6809/1776840019]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T05:51:47.726 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:47 smithi192 ceph-mon[34700]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T05:51:47.726 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:47 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:47.726 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:47 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:47.726 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:47 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T05:51:47.726 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:47 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T05:51:47.726 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:47 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:47.726 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:47 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:51:47.726 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:47 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:51:47.726 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:47 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:47.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:47 smithi138 ceph-mon[34391]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:47.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:47 smithi138 ceph-mon[34391]: from='osd.5 [v2:172.21.15.192:6808/1776840019,v1:172.21.15.192:6809/1776840019]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T05:51:47.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:47 smithi138 ceph-mon[34391]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T05:51:47.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:47 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:47.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:47 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:47.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:47 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T05:51:47.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:47 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T05:51:47.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:47 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:47.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:47 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:51:47.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:47 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:51:47.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:47 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:51:47.971 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:51:48.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.467+0000 7f8b17cc3640 1 -- 172.21.15.117:0/1504508663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b10102ae0 msgr2=0x7f8b10102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:51:48.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.467+0000 7f8b17cc3640 1 --2- 172.21.15.117:0/1504508663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b10102ae0 0x7f8b10102ef0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c009900 tx=0x7f8b0c02a410 comp rx=0 tx=0).stop 2024-09-21T05:51:48.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.467+0000 7f8b17cc3640 1 -- 172.21.15.117:0/1504508663 shutdown_connections 2024-09-21T05:51:48.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.467+0000 7f8b17cc3640 1 --2- 172.21.15.117:0/1504508663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b10108d70 0x7f8b1010f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:51:48.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.467+0000 7f8b17cc3640 1 --2- 172.21.15.117:0/1504508663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b10108420 0x7f8b10108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:51:48.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.467+0000 7f8b17cc3640 1 --2- 172.21.15.117:0/1504508663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b10102ae0 0x7f8b10102ef0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:51:48.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.467+0000 7f8b17cc3640 1 -- 172.21.15.117:0/1504508663 >> 172.21.15.117:0/1504508663 conn(0x7f8b10079770 msgr2=0x7f8b10079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:51:48.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.467+0000 7f8b17cc3640 1 -- 172.21.15.117:0/1504508663 shutdown_connections 2024-09-21T05:51:48.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.467+0000 7f8b17cc3640 1 -- 172.21.15.117:0/1504508663 wait complete. 2024-09-21T05:51:48.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.468+0000 7f8b17cc3640 1 Processor -- start 2024-09-21T05:51:48.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.468+0000 7f8b17cc3640 1 -- start start 2024-09-21T05:51:48.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.468+0000 7f8b17cc3640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b10102ae0 0x7f8b1019b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:51:48.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.468+0000 7f8b17cc3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b10108420 0x7f8b1019b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:51:48.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.468+0000 7f8b17cc3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b10108d70 0x7f8b1019bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:51:48.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.468+0000 7f8b17cc3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8b10112a30 con 0x7f8b10108420 2024-09-21T05:51:48.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.468+0000 7f8b17cc3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8b101128b0 con 0x7f8b10108d70 2024-09-21T05:51:48.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.468+0000 7f8b17cc3640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8b10112bb0 con 0x7f8b10102ae0 2024-09-21T05:51:48.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.468+0000 7f8b1525e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b10108420 0x7f8b1019b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:51:48.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.468+0000 7f8b1525e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b10108420 0x7f8b1019b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:47816/0 (socket says 172.21.15.117:47816) 2024-09-21T05:51:48.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.468+0000 7f8b1525e640 1 -- 172.21.15.117:0/2834282490 learned_addr learned my addr 172.21.15.117:0/2834282490 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:51:48.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.468+0000 7f8b15a5f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b10102ae0 0x7f8b1019b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:51:48.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.468+0000 7f8b16260640 1 --2- 172.21.15.117:0/2834282490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b10108d70 0x7f8b1019bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:51:48.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.468+0000 7f8b1525e640 1 -- 172.21.15.117:0/2834282490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b10102ae0 msgr2=0x7f8b1019b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:51:48.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.468+0000 7f8b1525e640 1 --2- 172.21.15.117:0/2834282490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b10102ae0 0x7f8b1019b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:51:48.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.468+0000 7f8b1525e640 1 -- 172.21.15.117:0/2834282490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b10108d70 msgr2=0x7f8b1019bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:51:48.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.468+0000 7f8b1525e640 1 --2- 172.21.15.117:0/2834282490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b10108d70 0x7f8b1019bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:51:48.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.468+0000 7f8b1525e640 1 -- 172.21.15.117:0/2834282490 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b101a65b0 con 0x7f8b10108420 2024-09-21T05:51:48.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.468+0000 7f8b16260640 1 --2- 172.21.15.117:0/2834282490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b10108d70 0x7f8b1019bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:51:48.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.469+0000 7f8b1525e640 1 --2- 172.21.15.117:0/2834282490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b10108420 0x7f8b1019b860 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8b0000c970 tx=0x7f8b0000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:51:48.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.469+0000 7f8b06ffd640 1 -- 172.21.15.117:0/2834282490 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b00007bb0 con 0x7f8b10108420 2024-09-21T05:51:48.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.469+0000 7f8b06ffd640 1 -- 172.21.15.117:0/2834282490 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b00007d50 con 0x7f8b10108420 2024-09-21T05:51:48.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.469+0000 7f8b17cc3640 1 -- 172.21.15.117:0/2834282490 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b101a68a0 con 0x7f8b10108420 2024-09-21T05:51:48.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.469+0000 7f8b06ffd640 1 -- 172.21.15.117:0/2834282490 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0000f450 con 0x7f8b10108420 2024-09-21T05:51:48.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.469+0000 7f8b17cc3640 1 -- 172.21.15.117:0/2834282490 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b101a6de0 con 0x7f8b10108420 2024-09-21T05:51:48.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.470+0000 7f8b17cc3640 1 -- 172.21.15.117:0/2834282490 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b10103d60 con 0x7f8b10108420 2024-09-21T05:51:48.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.471+0000 7f8b06ffd640 1 -- 172.21.15.117:0/2834282490 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f8b00004eb0 con 0x7f8b10108420 2024-09-21T05:51:48.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.472+0000 7f8b06ffd640 1 --2- 172.21.15.117:0/2834282490 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8afc0b6590 0x7f8afc0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:51:48.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.472+0000 7f8b15a5f640 1 --2- 172.21.15.117:0/2834282490 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8afc0b6590 0x7f8afc0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:51:48.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.473+0000 7f8b06ffd640 1 -- 172.21.15.117:0/2834282490 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4452+0+0 (secure 0 0 0) 0x7f8b000e6590 con 0x7f8b10108420 2024-09-21T05:51:48.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.473+0000 7f8b15a5f640 1 --2- 172.21.15.117:0/2834282490 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8afc0b6590 0x7f8afc0b8aa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c000c00 tx=0x7f8b0c031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:51:48.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.476+0000 7f8b06ffd640 1 -- 172.21.15.117:0/2834282490 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8b00014030 con 0x7f8b10108420 2024-09-21T05:51:48.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:51:48.666+0000 7f8b17cc3640 1 -- 172.21.15.117:0/2834282490 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8b10063800 con 0x7f8afc0b6590 2024-09-21T05:51:48.670 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:48 smithi117 ceph-mon[30947]: Detected new or changed devices on smithi192 2024-09-21T05:51:48.670 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:48 smithi117 ceph-mon[30947]: Adjusting osd_memory_target on smithi192 to 8544M 2024-09-21T05:51:48.670 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:48 smithi117 ceph-mon[30947]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T05:51:48.670 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:48 smithi117 ceph-mon[30947]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T05:51:48.670 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:48 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T05:51:48.670 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:48 smithi117 ceph-mon[30947]: from='osd.5 [v2:172.21.15.192:6808/1776840019,v1:172.21.15.192:6809/1776840019]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]} : dispatch 2024-09-21T05:51:48.670 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:48 smithi117 ceph-mon[30947]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]} : dispatch 2024-09-21T05:51:48.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:48 smithi138 ceph-mon[34391]: Detected new or changed devices on smithi192 2024-09-21T05:51:48.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:48 smithi138 ceph-mon[34391]: Adjusting osd_memory_target on smithi192 to 8544M 2024-09-21T05:51:48.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:48 smithi138 ceph-mon[34391]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T05:51:48.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:48 smithi138 ceph-mon[34391]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T05:51:48.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:48 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T05:51:48.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:48 smithi138 ceph-mon[34391]: from='osd.5 [v2:172.21.15.192:6808/1776840019,v1:172.21.15.192:6809/1776840019]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]} : dispatch 2024-09-21T05:51:48.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:48 smithi138 ceph-mon[34391]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]} : dispatch 2024-09-21T05:51:48.960 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:48 smithi192 ceph-mon[34700]: Detected new or changed devices on smithi192 2024-09-21T05:51:48.961 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:48 smithi192 ceph-mon[34700]: Adjusting osd_memory_target on smithi192 to 8544M 2024-09-21T05:51:48.961 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:48 smithi192 ceph-mon[34700]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T05:51:48.961 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:48 smithi192 ceph-mon[34700]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T05:51:48.961 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:48 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T05:51:48.961 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:48 smithi192 ceph-mon[34700]: from='osd.5 [v2:172.21.15.192:6808/1776840019,v1:172.21.15.192:6809/1776840019]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]} : dispatch 2024-09-21T05:51:48.961 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:48 smithi192 ceph-mon[34700]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]} : dispatch 2024-09-21T05:51:49.230 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 05:51:48 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T05:51:48.958+0000 7f3d740a4640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T05:51:49.230 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 05:51:48 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T05:51:48.967+0000 7f3d6be9a640 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T05:51:49.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:49 smithi117 ceph-mon[30947]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:49.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:49 smithi117 ceph-mon[30947]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-09-21T05:51:49.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:49 smithi117 ceph-mon[30947]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T05:51:49.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:49 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T05:51:49.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:49 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T05:51:49.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:49 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:51:49.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:49 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:51:49.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:49 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:51:49.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:49 smithi117 ceph-mon[30947]: from='osd.5 ' entity='osd.5' 2024-09-21T05:51:49.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:49 smithi138 ceph-mon[34391]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:49.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:49 smithi138 ceph-mon[34391]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-09-21T05:51:49.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:49 smithi138 ceph-mon[34391]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T05:51:49.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:49 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T05:51:49.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:49 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T05:51:49.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:49 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:51:49.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:49 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:51:49.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:49 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:51:49.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:49 smithi138 ceph-mon[34391]: from='osd.5 ' entity='osd.5' 2024-09-21T05:51:49.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:49 smithi192 ceph-mon[34700]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-09-21T05:51:49.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:49 smithi192 ceph-mon[34700]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-09-21T05:51:49.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:49 smithi192 ceph-mon[34700]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T05:51:49.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:49 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T05:51:49.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:49 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T05:51:49.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:49 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:51:49.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:49 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:51:49.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:49 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:51:49.981 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:49 smithi192 ceph-mon[34700]: from='osd.5 ' entity='osd.5' 2024-09-21T05:51:50.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:50 smithi117 ceph-mon[30947]: purged_snaps scrub starts 2024-09-21T05:51:50.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:50 smithi117 ceph-mon[30947]: purged_snaps scrub ok 2024-09-21T05:51:50.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:50 smithi117 ceph-mon[30947]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:51:50.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:50 smithi117 ceph-mon[30947]: osd.5 [v2:172.21.15.192:6808/1776840019,v1:172.21.15.192:6809/1776840019] boot 2024-09-21T05:51:50.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:50 smithi117 ceph-mon[30947]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T05:51:50.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:50 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T05:51:50.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:50 smithi138 ceph-mon[34391]: purged_snaps scrub starts 2024-09-21T05:51:50.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:50 smithi138 ceph-mon[34391]: purged_snaps scrub ok 2024-09-21T05:51:50.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:50 smithi138 ceph-mon[34391]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:51:50.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:50 smithi138 ceph-mon[34391]: osd.5 [v2:172.21.15.192:6808/1776840019,v1:172.21.15.192:6809/1776840019] boot 2024-09-21T05:51:50.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:50 smithi138 ceph-mon[34391]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T05:51:50.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:50 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T05:51:50.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:50 smithi192 ceph-mon[34700]: purged_snaps scrub starts 2024-09-21T05:51:50.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:50 smithi192 ceph-mon[34700]: purged_snaps scrub ok 2024-09-21T05:51:50.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:50 smithi192 ceph-mon[34700]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:51:50.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:50 smithi192 ceph-mon[34700]: osd.5 [v2:172.21.15.192:6808/1776840019,v1:172.21.15.192:6809/1776840019] boot 2024-09-21T05:51:50.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:50 smithi192 ceph-mon[34700]: osdmap e36: 6 total, 6 up, 6 in 2024-09-21T05:51:50.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:50 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T05:51:51.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:51 smithi138 ceph-mon[34391]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 207 MiB used, 536 GiB / 536 GiB avail 2024-09-21T05:51:51.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:51 smithi138 ceph-mon[34391]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T05:51:51.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:51 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2781296077' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6e507a91-ed09-4a33-bf0e-c4b593b3c5aa"} : dispatch 2024-09-21T05:51:51.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:51 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2781296077' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6e507a91-ed09-4a33-bf0e-c4b593b3c5aa"}]': finished 2024-09-21T05:51:51.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:51 smithi138 ceph-mon[34391]: osdmap e38: 7 total, 6 up, 7 in 2024-09-21T05:51:51.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:51 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T05:51:51.973 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:51 smithi117 ceph-mon[30947]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 207 MiB used, 536 GiB / 536 GiB avail 2024-09-21T05:51:51.974 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:51 smithi117 ceph-mon[30947]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T05:51:51.974 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:51 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2781296077' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6e507a91-ed09-4a33-bf0e-c4b593b3c5aa"} : dispatch 2024-09-21T05:51:51.974 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:51 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2781296077' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6e507a91-ed09-4a33-bf0e-c4b593b3c5aa"}]': finished 2024-09-21T05:51:51.974 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:51 smithi117 ceph-mon[30947]: osdmap e38: 7 total, 6 up, 7 in 2024-09-21T05:51:51.974 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:51 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T05:51:51.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:51 smithi192 ceph-mon[34700]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 207 MiB used, 536 GiB / 536 GiB avail 2024-09-21T05:51:51.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:51 smithi192 ceph-mon[34700]: osdmap e37: 6 total, 6 up, 6 in 2024-09-21T05:51:51.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:51 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2781296077' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6e507a91-ed09-4a33-bf0e-c4b593b3c5aa"} : dispatch 2024-09-21T05:51:51.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:51 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2781296077' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6e507a91-ed09-4a33-bf0e-c4b593b3c5aa"}]': finished 2024-09-21T05:51:51.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:51 smithi192 ceph-mon[34700]: osdmap e38: 7 total, 6 up, 7 in 2024-09-21T05:51:51.981 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:51 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T05:51:52.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:52 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/4149774560' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:51:52.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:52 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/4149774560' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:51:53.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:52 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/4149774560' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:51:53.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:53 smithi138 ceph-mon[34391]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 207 MiB used, 536 GiB / 536 GiB avail 2024-09-21T05:51:53.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:53 smithi192 ceph-mon[34700]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 207 MiB used, 536 GiB / 536 GiB avail 2024-09-21T05:51:54.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:53 smithi117 ceph-mon[30947]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 207 MiB used, 536 GiB / 536 GiB avail 2024-09-21T05:51:55.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:55 smithi117 ceph-mon[30947]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-21T05:51:55.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:55 smithi138 ceph-mon[34391]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-21T05:51:55.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:55 smithi192 ceph-mon[34700]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-21T05:51:57.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:57 smithi117 ceph-mon[30947]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-21T05:51:57.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:57 smithi138 ceph-mon[34391]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-21T05:51:57.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:57 smithi192 ceph-mon[34700]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-21T05:51:59.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:51:59 smithi117 ceph-mon[30947]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T05:51:59.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:51:59 smithi138 ceph-mon[34391]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T05:51:59.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:51:59 smithi192 ceph-mon[34700]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T05:52:00.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:00 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T05:52:00.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:00 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:52:00.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:00 smithi138 ceph-mon[34391]: Deploying daemon osd.6 on smithi117 2024-09-21T05:52:00.965 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:00 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T05:52:00.965 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:00 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:52:00.965 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:00 smithi117 ceph-mon[30947]: Deploying daemon osd.6 on smithi117 2024-09-21T05:52:00.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:00 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T05:52:00.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:00 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:52:00.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:00 smithi192 ceph-mon[34700]: Deploying daemon osd.6 on smithi117 2024-09-21T05:52:01.610 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:01 smithi117 ceph-mon[30947]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T05:52:01.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:01 smithi138 ceph-mon[34391]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T05:52:01.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:01 smithi192 ceph-mon[34700]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T05:52:03.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:03 smithi117 ceph-mon[30947]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-21T05:52:03.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:03 smithi138 ceph-mon[34391]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-21T05:52:03.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:03 smithi192 ceph-mon[34700]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-21T05:52:05.748 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:05 smithi117 ceph-mon[30947]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T05:52:05.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:05 smithi138 ceph-mon[34391]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T05:52:05.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:05 smithi192 ceph-mon[34700]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T05:52:06.659 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:06 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:52:06.659 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:06 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:06.659 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:06 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:06.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:06 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:52:06.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:06 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:06.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:06 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:06.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:06 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:52:06.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:06 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:06.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:06 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:07.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:07 smithi117 ceph-mon[30947]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-21T05:52:07.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:07 smithi138 ceph-mon[34391]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-21T05:52:07.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:07 smithi192 ceph-mon[34700]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-21T05:52:09.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:09 smithi117 ceph-mon[30947]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-21T05:52:09.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:09 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:09.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:09 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:09.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:09 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:52:09.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:09 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:52:09.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:09 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:09.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:52:09.854+0000 7f8b06ffd640 1 -- 172.21.15.117:0/2834282490 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8b10063800 con 0x7f8afc0b6590 2024-09-21T05:52:09.856 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 6 on host 'smithi117' 2024-09-21T05:52:09.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:52:09.857+0000 7f8b17cc3640 1 -- 172.21.15.117:0/2834282490 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8afc0b6590 msgr2=0x7f8afc0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:52:09.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:52:09.857+0000 7f8b17cc3640 1 --2- 172.21.15.117:0/2834282490 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8afc0b6590 0x7f8afc0b8aa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c000c00 tx=0x7f8b0c031400 comp rx=0 tx=0).stop 2024-09-21T05:52:09.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:52:09.857+0000 7f8b17cc3640 1 -- 172.21.15.117:0/2834282490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b10108420 msgr2=0x7f8b1019b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:52:09.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:52:09.857+0000 7f8b17cc3640 1 --2- 172.21.15.117:0/2834282490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b10108420 0x7f8b1019b860 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8b0000c970 tx=0x7f8b0000ce30 comp rx=0 tx=0).stop 2024-09-21T05:52:09.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:52:09.857+0000 7f8b17cc3640 1 -- 172.21.15.117:0/2834282490 shutdown_connections 2024-09-21T05:52:09.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:52:09.857+0000 7f8b17cc3640 1 --2- 172.21.15.117:0/2834282490 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8afc0b6590 0x7f8afc0b8aa0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:52:09.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:52:09.857+0000 7f8b17cc3640 1 --2- 172.21.15.117:0/2834282490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b10108420 0x7f8b1019b860 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:52:09.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:52:09.857+0000 7f8b17cc3640 1 --2- 172.21.15.117:0/2834282490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b10108d70 0x7f8b1019bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:52:09.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:52:09.857+0000 7f8b17cc3640 1 --2- 172.21.15.117:0/2834282490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b10102ae0 0x7f8b1019b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:52:09.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:52:09.857+0000 7f8b17cc3640 1 -- 172.21.15.117:0/2834282490 >> 172.21.15.117:0/2834282490 conn(0x7f8b10079770 msgr2=0x7f8b10106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:52:09.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:52:09.857+0000 7f8b17cc3640 1 -- 172.21.15.117:0/2834282490 shutdown_connections 2024-09-21T05:52:09.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:52:09.857+0000 7f8b17cc3640 1 -- 172.21.15.117:0/2834282490 wait complete. 2024-09-21T05:52:09.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:09 smithi138 ceph-mon[34391]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-21T05:52:09.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:09 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:09.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:09 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:09.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:09 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:52:09.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:09 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:52:09.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:09 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:09.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:09 smithi192 ceph-mon[34700]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-21T05:52:09.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:09 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:09.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:09 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:09.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:09 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:52:09.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:09 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:52:09.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:09 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:10.663 DEBUG:teuthology.orchestra.run.smithi117:osd.6> sudo journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.6.service 2024-09-21T05:52:10.666 INFO:tasks.cephadm:Deploying osd.7 on smithi138 with /dev/vg_nvme/lv_2... 2024-09-21T05:52:10.666 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T05:52:10.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:10 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:52:10.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:10 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:10.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:10 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:10.919 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:10 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:52:10.919 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:10 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:10.919 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:10 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:10.919 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T05:52:10.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:10 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:52:10.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:10 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:10.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:10 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:11.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:11 smithi117 ceph-mon[30947]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-21T05:52:11.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:11 smithi138 ceph-mon[34391]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-21T05:52:11.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:11 smithi192 ceph-mon[34700]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-21T05:52:12.120 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 05:52:11 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T05:52:11.844+0000 7fbdefbca640 -1 osd.6 0 log_to_monitors true 2024-09-21T05:52:12.737 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:12 smithi138 ceph-mon[34391]: from='osd.6 [v2:172.21.15.117:6818/1093341590,v1:172.21.15.117:6819/1093341590]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T05:52:12.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:12 smithi117 ceph-mon[30947]: from='osd.6 [v2:172.21.15.117:6818/1093341590,v1:172.21.15.117:6819/1093341590]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T05:52:12.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:12 smithi192 ceph-mon[34700]: from='osd.6 [v2:172.21.15.117:6818/1093341590,v1:172.21.15.117:6819/1093341590]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T05:52:13.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:13 smithi138 ceph-mon[34391]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-21T05:52:13.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:13 smithi138 ceph-mon[34391]: from='osd.6 [v2:172.21.15.117:6818/1093341590,v1:172.21.15.117:6819/1093341590]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T05:52:13.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:13 smithi138 ceph-mon[34391]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T05:52:13.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:13 smithi138 ceph-mon[34391]: from='osd.6 [v2:172.21.15.117:6818/1093341590,v1:172.21.15.117:6819/1093341590]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-21T05:52:13.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:13 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T05:52:13.737 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-21T05:52:13.765 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi138:vg_nvme/lv_2 2024-09-21T05:52:13.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 05:52:13 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T05:52:13.599+0000 7fbdebb4c640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T05:52:13.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 05:52:13 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T05:52:13.610+0000 7fbde794a640 -1 osd.6 40 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T05:52:13.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:13 smithi117 ceph-mon[30947]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-21T05:52:13.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:13 smithi117 ceph-mon[30947]: from='osd.6 [v2:172.21.15.117:6818/1093341590,v1:172.21.15.117:6819/1093341590]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T05:52:13.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:13 smithi117 ceph-mon[30947]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T05:52:13.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:13 smithi117 ceph-mon[30947]: from='osd.6 [v2:172.21.15.117:6818/1093341590,v1:172.21.15.117:6819/1093341590]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-21T05:52:13.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:13 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T05:52:13.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:13 smithi192 ceph-mon[34700]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-21T05:52:13.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:13 smithi192 ceph-mon[34700]: from='osd.6 [v2:172.21.15.117:6818/1093341590,v1:172.21.15.117:6819/1093341590]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T05:52:13.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:13 smithi192 ceph-mon[34700]: osdmap e39: 7 total, 6 up, 7 in 2024-09-21T05:52:13.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:13 smithi192 ceph-mon[34700]: from='osd.6 [v2:172.21.15.117:6818/1093341590,v1:172.21.15.117:6819/1093341590]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-21T05:52:13.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:13 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T05:52:14.027 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T05:52:14.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.664+0000 7fec53a19640 1 -- 172.21.15.138:0/1449432541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec4c102a60 msgr2=0x7fec4c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:52:14.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.664+0000 7fec53a19640 1 --2- 172.21.15.138:0/1449432541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec4c102a60 0x7fec4c102e70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fec48009ca0 tx=0x7fec4802a8c0 comp rx=0 tx=0).stop 2024-09-21T05:52:14.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.664+0000 7fec53a19640 1 -- 172.21.15.138:0/1449432541 shutdown_connections 2024-09-21T05:52:14.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.664+0000 7fec53a19640 1 --2- 172.21.15.138:0/1449432541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec4c1083a0 0x7fec4c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:52:14.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.664+0000 7fec53a19640 1 --2- 172.21.15.138:0/1449432541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec4c108cf0 0x7fec4c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:52:14.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.664+0000 7fec53a19640 1 --2- 172.21.15.138:0/1449432541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec4c102a60 0x7fec4c102e70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:52:14.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.664+0000 7fec53a19640 1 -- 172.21.15.138:0/1449432541 >> 172.21.15.138:0/1449432541 conn(0x7fec4c079730 msgr2=0x7fec4c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:52:14.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.664+0000 7fec53a19640 1 -- 172.21.15.138:0/1449432541 shutdown_connections 2024-09-21T05:52:14.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.664+0000 7fec53a19640 1 -- 172.21.15.138:0/1449432541 wait complete. 2024-09-21T05:52:14.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.665+0000 7fec53a19640 1 Processor -- start 2024-09-21T05:52:14.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.665+0000 7fec53a19640 1 -- start start 2024-09-21T05:52:14.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.665+0000 7fec53a19640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec4c102a60 0x7fec4c199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:52:14.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.665+0000 7fec53a19640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec4c1083a0 0x7fec4c1998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:52:14.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.665+0000 7fec53a19640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec4c108cf0 0x7fec4c19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:52:14.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.665+0000 7fec53a19640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c112960 con 0x7fec4c102a60 2024-09-21T05:52:14.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.665+0000 7fec53a19640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c1127e0 con 0x7fec4c1083a0 2024-09-21T05:52:14.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.665+0000 7fec53a19640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c112ae0 con 0x7fec4c108cf0 2024-09-21T05:52:14.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.665+0000 7fec50fb4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec4c1083a0 0x7fec4c1998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:52:14.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.665+0000 7fec50fb4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec4c1083a0 0x7fec4c1998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:36498/0 (socket says 172.21.15.138:36498) 2024-09-21T05:52:14.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.665+0000 7fec50fb4640 1 -- 172.21.15.138:0/3883060342 learned_addr learned my addr 172.21.15.138:0/3883060342 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-09-21T05:52:14.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.665+0000 7fec517b5640 1 --2- 172.21.15.138:0/3883060342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec4c102a60 0x7fec4c199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:52:14.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.666+0000 7fec51fb6640 1 --2- 172.21.15.138:0/3883060342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec4c108cf0 0x7fec4c19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:52:14.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.666+0000 7fec50fb4640 1 -- 172.21.15.138:0/3883060342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec4c108cf0 msgr2=0x7fec4c19eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:52:14.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.666+0000 7fec50fb4640 1 --2- 172.21.15.138:0/3883060342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec4c108cf0 0x7fec4c19eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:52:14.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.666+0000 7fec50fb4640 1 -- 172.21.15.138:0/3883060342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec4c102a60 msgr2=0x7fec4c199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:52:14.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.666+0000 7fec50fb4640 1 --2- 172.21.15.138:0/3883060342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec4c102a60 0x7fec4c199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:52:14.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.666+0000 7fec50fb4640 1 -- 172.21.15.138:0/3883060342 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec4c19f2a0 con 0x7fec4c1083a0 2024-09-21T05:52:14.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.666+0000 7fec51fb6640 1 --2- 172.21.15.138:0/3883060342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec4c108cf0 0x7fec4c19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:52:14.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.666+0000 7fec50fb4640 1 --2- 172.21.15.138:0/3883060342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec4c1083a0 0x7fec4c1998d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fec3c005fd0 tx=0x7fec3c00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:52:14.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.666+0000 7fec427fc640 1 -- 172.21.15.138:0/3883060342 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c017070 con 0x7fec4c1083a0 2024-09-21T05:52:14.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.666+0000 7fec427fc640 1 -- 172.21.15.138:0/3883060342 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec3c009c30 con 0x7fec4c1083a0 2024-09-21T05:52:14.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.666+0000 7fec53a19640 1 -- 172.21.15.138:0/3883060342 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec4c19f590 con 0x7fec4c1083a0 2024-09-21T05:52:14.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.666+0000 7fec427fc640 1 -- 172.21.15.138:0/3883060342 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c003d30 con 0x7fec4c1083a0 2024-09-21T05:52:14.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.666+0000 7fec53a19640 1 -- 172.21.15.138:0/3883060342 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec4c1a5cf0 con 0x7fec4c1083a0 2024-09-21T05:52:14.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.667+0000 7fec53a19640 1 -- 172.21.15.138:0/3883060342 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec4c102e70 con 0x7fec4c1083a0 2024-09-21T05:52:14.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.669+0000 7fec427fc640 1 -- 172.21.15.138:0/3883060342 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7fec3c003ed0 con 0x7fec4c1083a0 2024-09-21T05:52:14.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.669+0000 7fec427fc640 1 --2- 172.21.15.138:0/3883060342 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fec380b6560 0x7fec380b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:52:14.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.670+0000 7fec427fc640 1 -- 172.21.15.138:0/3883060342 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5180+0+0 (secure 0 0 0) 0x7fec3c0e6c10 con 0x7fec4c1083a0 2024-09-21T05:52:14.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.670+0000 7fec517b5640 1 --2- 172.21.15.138:0/3883060342 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fec380b6560 0x7fec380b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:52:14.675 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:14 smithi138 ceph-mon[34391]: from='osd.6 [v2:172.21.15.117:6818/1093341590,v1:172.21.15.117:6819/1093341590]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-21T05:52:14.675 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:14 smithi138 ceph-mon[34391]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T05:52:14.675 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:14 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T05:52:14.675 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:14 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T05:52:14.675 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:14 smithi138 ceph-mon[34391]: Detected new or changed devices on smithi117 2024-09-21T05:52:14.675 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:14 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:14.675 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:14 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:14.675 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:14 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T05:52:14.675 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:14 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T05:52:14.675 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:14 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T05:52:14.675 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:14 smithi138 ceph-mon[34391]: Adjusting osd_memory_target on smithi117 to 5696M 2024-09-21T05:52:14.676 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:14 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:14.676 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:14 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:52:14.676 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:14 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:52:14.676 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:14 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:14.676 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:14 smithi138 ceph-mon[34391]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-21T05:52:14.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.670+0000 7fec517b5640 1 --2- 172.21.15.138:0/3883060342 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fec380b6560 0x7fec380b8a70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fec48005e30 tx=0x7fec48009360 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:52:14.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.673+0000 7fec427fc640 1 -- 172.21.15.138:0/3883060342 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fec3c020030 con 0x7fec4c1083a0 2024-09-21T05:52:14.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:14.859+0000 7fec53a19640 1 -- 172.21.15.138:0/3883060342 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fec4c063800 con 0x7fec380b6560 2024-09-21T05:52:14.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:14 smithi192 ceph-mon[34700]: from='osd.6 [v2:172.21.15.117:6818/1093341590,v1:172.21.15.117:6819/1093341590]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-21T05:52:14.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:14 smithi192 ceph-mon[34700]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T05:52:14.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:14 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T05:52:14.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:14 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T05:52:14.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:14 smithi192 ceph-mon[34700]: Detected new or changed devices on smithi117 2024-09-21T05:52:14.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:14 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:14.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:14 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:14.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:14 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T05:52:14.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:14 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T05:52:14.981 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:14 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T05:52:14.981 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:14 smithi192 ceph-mon[34700]: Adjusting osd_memory_target on smithi117 to 5696M 2024-09-21T05:52:14.981 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:14 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:14.981 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:14 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:52:14.981 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:14 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:52:14.981 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:14 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:14.981 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:14 smithi192 ceph-mon[34700]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-21T05:52:15.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:14 smithi117 ceph-mon[30947]: from='osd.6 [v2:172.21.15.117:6818/1093341590,v1:172.21.15.117:6819/1093341590]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-21T05:52:15.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:14 smithi117 ceph-mon[30947]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T05:52:15.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:14 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T05:52:15.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:14 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T05:52:15.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:14 smithi117 ceph-mon[30947]: Detected new or changed devices on smithi117 2024-09-21T05:52:15.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:14 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:15.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:14 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:15.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:14 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T05:52:15.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:14 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T05:52:15.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:14 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T05:52:15.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:14 smithi117 ceph-mon[30947]: Adjusting osd_memory_target on smithi117 to 5696M 2024-09-21T05:52:15.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:14 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:15.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:14 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:52:15.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:14 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:52:15.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:14 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:15.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:14 smithi117 ceph-mon[30947]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-09-21T05:52:15.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:15 smithi138 ceph-mon[34391]: purged_snaps scrub starts 2024-09-21T05:52:15.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:15 smithi138 ceph-mon[34391]: purged_snaps scrub ok 2024-09-21T05:52:15.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:15 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T05:52:15.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:15 smithi138 ceph-mon[34391]: osd.6 [v2:172.21.15.117:6818/1093341590,v1:172.21.15.117:6819/1093341590] boot 2024-09-21T05:52:15.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:15 smithi138 ceph-mon[34391]: osdmap e41: 7 total, 7 up, 7 in 2024-09-21T05:52:15.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:15 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T05:52:15.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:15 smithi138 ceph-mon[34391]: from='client.24413 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:52:15.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:15 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:52:15.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:15 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:52:15.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:15 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:52:15.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:15 smithi192 ceph-mon[34700]: purged_snaps scrub starts 2024-09-21T05:52:15.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:15 smithi192 ceph-mon[34700]: purged_snaps scrub ok 2024-09-21T05:52:15.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:15 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T05:52:15.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:15 smithi192 ceph-mon[34700]: osd.6 [v2:172.21.15.117:6818/1093341590,v1:172.21.15.117:6819/1093341590] boot 2024-09-21T05:52:15.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:15 smithi192 ceph-mon[34700]: osdmap e41: 7 total, 7 up, 7 in 2024-09-21T05:52:15.881 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:15 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T05:52:15.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:15 smithi192 ceph-mon[34700]: from='client.24413 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:52:15.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:15 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:52:15.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:15 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:52:15.882 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:15 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:52:16.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:15 smithi117 ceph-mon[30947]: purged_snaps scrub starts 2024-09-21T05:52:16.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:15 smithi117 ceph-mon[30947]: purged_snaps scrub ok 2024-09-21T05:52:16.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:15 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T05:52:16.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:15 smithi117 ceph-mon[30947]: osd.6 [v2:172.21.15.117:6818/1093341590,v1:172.21.15.117:6819/1093341590] boot 2024-09-21T05:52:16.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:15 smithi117 ceph-mon[30947]: osdmap e41: 7 total, 7 up, 7 in 2024-09-21T05:52:16.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:15 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T05:52:16.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:15 smithi117 ceph-mon[30947]: from='client.24413 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:52:16.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:15 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:52:16.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:15 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:52:16.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:15 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:52:16.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:16 smithi138 ceph-mon[34391]: osdmap e42: 7 total, 7 up, 7 in 2024-09-21T05:52:16.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:16 smithi138 ceph-mon[34391]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:16.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:16 smithi192 ceph-mon[34700]: osdmap e42: 7 total, 7 up, 7 in 2024-09-21T05:52:16.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:16 smithi192 ceph-mon[34700]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:17.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:16 smithi117 ceph-mon[30947]: osdmap e42: 7 total, 7 up, 7 in 2024-09-21T05:52:17.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:16 smithi117 ceph-mon[30947]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:17.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:17 smithi138 ceph-mon[34391]: from='client.? 172.21.15.138:0/3459931290' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "36a47dd7-67f7-4ebe-8759-3a8bb6bd2ea3"} : dispatch 2024-09-21T05:52:17.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:17 smithi138 ceph-mon[34391]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "36a47dd7-67f7-4ebe-8759-3a8bb6bd2ea3"} : dispatch 2024-09-21T05:52:17.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:17 smithi138 ceph-mon[34391]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36a47dd7-67f7-4ebe-8759-3a8bb6bd2ea3"}]': finished 2024-09-21T05:52:17.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:17 smithi138 ceph-mon[34391]: osdmap e43: 8 total, 7 up, 8 in 2024-09-21T05:52:17.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:17 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T05:52:17.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:17 smithi192 ceph-mon[34700]: from='client.? 172.21.15.138:0/3459931290' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "36a47dd7-67f7-4ebe-8759-3a8bb6bd2ea3"} : dispatch 2024-09-21T05:52:17.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:17 smithi192 ceph-mon[34700]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "36a47dd7-67f7-4ebe-8759-3a8bb6bd2ea3"} : dispatch 2024-09-21T05:52:17.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:17 smithi192 ceph-mon[34700]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36a47dd7-67f7-4ebe-8759-3a8bb6bd2ea3"}]': finished 2024-09-21T05:52:17.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:17 smithi192 ceph-mon[34700]: osdmap e43: 8 total, 7 up, 8 in 2024-09-21T05:52:17.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:17 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T05:52:18.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:17 smithi117 ceph-mon[30947]: from='client.? 172.21.15.138:0/3459931290' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "36a47dd7-67f7-4ebe-8759-3a8bb6bd2ea3"} : dispatch 2024-09-21T05:52:18.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:17 smithi117 ceph-mon[30947]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "36a47dd7-67f7-4ebe-8759-3a8bb6bd2ea3"} : dispatch 2024-09-21T05:52:18.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:17 smithi117 ceph-mon[30947]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36a47dd7-67f7-4ebe-8759-3a8bb6bd2ea3"}]': finished 2024-09-21T05:52:18.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:17 smithi117 ceph-mon[30947]: osdmap e43: 8 total, 7 up, 8 in 2024-09-21T05:52:18.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:17 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T05:52:18.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:18 smithi192 ceph-mon[34700]: from='client.? 172.21.15.138:0/3124070920' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:52:18.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:18 smithi192 ceph-mon[34700]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:19.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:18 smithi117 ceph-mon[30947]: from='client.? 172.21.15.138:0/3124070920' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:52:19.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:18 smithi117 ceph-mon[30947]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:19.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:18 smithi138 ceph-mon[34391]: from='client.? 172.21.15.138:0/3124070920' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:52:19.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:18 smithi138 ceph-mon[34391]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:21.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:21 smithi117 ceph-mon[30947]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:21.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:21 smithi138 ceph-mon[34391]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:21.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:21 smithi192 ceph-mon[34700]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:23.627 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:23 smithi138 ceph-mon[34391]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:23.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:23 smithi117 ceph-mon[30947]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:23.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:23 smithi192 ceph-mon[34700]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:25.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:25 smithi117 ceph-mon[30947]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:25.781 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:25 smithi138 ceph-mon[34391]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:25.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:25 smithi192 ceph-mon[34700]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:26.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:26 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T05:52:26.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:26 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:52:26.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:26 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T05:52:26.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:26 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:52:26.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:26 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T05:52:26.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:26 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:52:27.834 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:27 smithi138 ceph-mon[34391]: Deploying daemon osd.7 on smithi138 2024-09-21T05:52:27.834 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:27 smithi138 ceph-mon[34391]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:27.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:27 smithi192 ceph-mon[34700]: Deploying daemon osd.7 on smithi138 2024-09-21T05:52:27.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:27 smithi192 ceph-mon[34700]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:28.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:27 smithi117 ceph-mon[30947]: Deploying daemon osd.7 on smithi138 2024-09-21T05:52:28.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:27 smithi117 ceph-mon[30947]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:29.593 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:29 smithi138 ceph-mon[34391]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:29.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:29 smithi192 ceph-mon[34700]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:30.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:29 smithi117 ceph-mon[30947]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:31.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:31 smithi138 ceph-mon[34391]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:31.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:31 smithi192 ceph-mon[34700]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:32.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:31 smithi117 ceph-mon[30947]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:32.796 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:32 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:52:32.796 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:32 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:32.796 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:32 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:32.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:32 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:52:32.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:32 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:32.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:32 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:33.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:32 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:52:33.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:32 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:33.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:32 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:33.803 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:33 smithi138 ceph-mon[34391]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:33.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:33 smithi192 ceph-mon[34700]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:34.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:33 smithi117 ceph-mon[30947]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:35.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:35 smithi138 ceph-mon[34391]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:35.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:35.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:35.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:52:35.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:52:35.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:35.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:35 smithi192 ceph-mon[34700]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:35.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:35.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:35.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:52:35.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:52:35.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:35.982 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 7 on host 'smithi138' 2024-09-21T05:52:35.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:35.980+0000 7fec427fc640 1 -- 172.21.15.138:0/3883060342 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fec4c063800 con 0x7fec380b6560 2024-09-21T05:52:35.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:35.982+0000 7fec53a19640 1 -- 172.21.15.138:0/3883060342 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fec380b6560 msgr2=0x7fec380b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:52:35.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:35.982+0000 7fec53a19640 1 --2- 172.21.15.138:0/3883060342 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fec380b6560 0x7fec380b8a70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fec48005e30 tx=0x7fec48009360 comp rx=0 tx=0).stop 2024-09-21T05:52:35.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:35.982+0000 7fec53a19640 1 -- 172.21.15.138:0/3883060342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec4c1083a0 msgr2=0x7fec4c1998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:52:35.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:35.982+0000 7fec53a19640 1 --2- 172.21.15.138:0/3883060342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec4c1083a0 0x7fec4c1998d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fec3c005fd0 tx=0x7fec3c00dde0 comp rx=0 tx=0).stop 2024-09-21T05:52:35.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:35.982+0000 7fec53a19640 1 -- 172.21.15.138:0/3883060342 shutdown_connections 2024-09-21T05:52:35.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:35.982+0000 7fec53a19640 1 --2- 172.21.15.138:0/3883060342 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fec380b6560 0x7fec380b8a70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:52:35.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:35.982+0000 7fec53a19640 1 --2- 172.21.15.138:0/3883060342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec4c108cf0 0x7fec4c19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:52:35.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:35.982+0000 7fec53a19640 1 --2- 172.21.15.138:0/3883060342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec4c1083a0 0x7fec4c1998d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:52:35.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:35.982+0000 7fec53a19640 1 --2- 172.21.15.138:0/3883060342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec4c102a60 0x7fec4c199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:52:35.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:35.982+0000 7fec53a19640 1 -- 172.21.15.138:0/3883060342 >> 172.21.15.138:0/3883060342 conn(0x7fec4c079730 msgr2=0x7fec4c101330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:52:35.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:35.983+0000 7fec53a19640 1 -- 172.21.15.138:0/3883060342 shutdown_connections 2024-09-21T05:52:35.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:52:35.983+0000 7fec53a19640 1 -- 172.21.15.138:0/3883060342 wait complete. 2024-09-21T05:52:36.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:35 smithi117 ceph-mon[30947]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:36.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:36.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:36.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:52:36.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:52:36.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:36.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:36 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:52:36.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:36 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:36.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:36 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:36.654 DEBUG:teuthology.orchestra.run.smithi138:osd.7> sudo journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.7.service 2024-09-21T05:52:36.657 INFO:tasks.cephadm:Deploying osd.8 on smithi192 with /dev/vg_nvme/lv_2... 2024-09-21T05:52:36.657 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T05:52:36.912 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T05:52:36.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:36 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:52:36.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:36 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:36.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:36 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:37.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:36 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:52:37.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:36 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:37.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:36 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:37.609 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:37 smithi192 ceph-mon[34700]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:37.837 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:37 smithi138 ceph-mon[34391]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:38.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:37 smithi117 ceph-mon[30947]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:38.556 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 05:52:38 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T05:52:38.427+0000 7f5c6cf73640 -1 osd.7 0 log_to_monitors true 2024-09-21T05:52:38.819 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:38 smithi138 ceph-mon[34391]: from='osd.7 [v2:172.21.15.138:6816/1448179714,v1:172.21.15.138:6817/1448179714]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T05:52:38.819 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:38 smithi138 ceph-mon[34391]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T05:52:38.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:38 smithi192 ceph-mon[34700]: from='osd.7 [v2:172.21.15.138:6816/1448179714,v1:172.21.15.138:6817/1448179714]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T05:52:38.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:38 smithi192 ceph-mon[34700]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T05:52:39.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:38 smithi117 ceph-mon[30947]: from='osd.7 [v2:172.21.15.138:6816/1448179714,v1:172.21.15.138:6817/1448179714]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T05:52:39.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:38 smithi117 ceph-mon[30947]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T05:52:39.305 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-21T05:52:39.331 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi192:vg_nvme/lv_2 2024-09-21T05:52:39.603 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T05:52:39.825 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:39 smithi192 ceph-mon[34700]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:39.825 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:39 smithi192 ceph-mon[34700]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T05:52:39.825 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:39 smithi192 ceph-mon[34700]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T05:52:39.825 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:39 smithi192 ceph-mon[34700]: from='osd.7 [v2:172.21.15.138:6816/1448179714,v1:172.21.15.138:6817/1448179714]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-21T05:52:39.825 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:39 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T05:52:39.826 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:39 smithi192 ceph-mon[34700]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-21T05:52:39.826 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:39 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:39.826 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:39 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:39.826 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:39 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T05:52:39.826 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:39 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T05:52:39.826 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:39 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T05:52:39.826 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:39 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:39.826 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:39 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:52:39.826 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:39 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:52:39.826 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:39 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:39.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 05:52:39 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T05:52:39.564+0000 7f5c68ef5640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T05:52:39.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 05:52:39 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T05:52:39.573+0000 7f5c64cf3640 -1 osd.7 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T05:52:39.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:39 smithi138 ceph-mon[34391]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:39.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:39 smithi138 ceph-mon[34391]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T05:52:39.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:39 smithi138 ceph-mon[34391]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T05:52:39.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:39 smithi138 ceph-mon[34391]: from='osd.7 [v2:172.21.15.138:6816/1448179714,v1:172.21.15.138:6817/1448179714]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-21T05:52:39.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:39 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T05:52:39.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:39 smithi138 ceph-mon[34391]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-21T05:52:39.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:39 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:39.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:39 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:39.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:39 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T05:52:39.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:39 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T05:52:39.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:39 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T05:52:39.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:39 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:39.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:39 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:52:39.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:39 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:52:39.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:39 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:40.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:39 smithi117 ceph-mon[30947]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:40.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:39 smithi117 ceph-mon[30947]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T05:52:40.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:39 smithi117 ceph-mon[30947]: osdmap e44: 8 total, 7 up, 8 in 2024-09-21T05:52:40.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:39 smithi117 ceph-mon[30947]: from='osd.7 [v2:172.21.15.138:6816/1448179714,v1:172.21.15.138:6817/1448179714]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-21T05:52:40.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:39 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T05:52:40.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:39 smithi117 ceph-mon[30947]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-21T05:52:40.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:39 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:40.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:39 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:40.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:39 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T05:52:40.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:39 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T05:52:40.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:39 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T05:52:40.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:39 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:40.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:39 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:52:40.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:39 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:52:40.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:39 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:40.042 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.040+0000 7fccde2b4640 1 -- 172.21.15.192:0/866565096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccd810de10 msgr2=0x7fccd8110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:52:40.042 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.040+0000 7fccde2b4640 1 --2- 172.21.15.192:0/866565096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccd810de10 0x7fccd8110300 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fccd0009ca0 tx=0x7fccd002a8c0 comp rx=0 tx=0).stop 2024-09-21T05:52:40.042 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.040+0000 7fccde2b4640 1 -- 172.21.15.192:0/866565096 shutdown_connections 2024-09-21T05:52:40.042 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.040+0000 7fccde2b4640 1 --2- 172.21.15.192:0/866565096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccd810de10 0x7fccd8110300 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:52:40.042 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.040+0000 7fccde2b4640 1 --2- 172.21.15.192:0/866565096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccd8105300 0x7fccd810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:52:40.042 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.040+0000 7fccde2b4640 1 --2- 172.21.15.192:0/866565096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd81049b0 0x7fccd8104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:52:40.043 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.040+0000 7fccde2b4640 1 -- 172.21.15.192:0/866565096 >> 172.21.15.192:0/866565096 conn(0x7fccd8079730 msgr2=0x7fccd8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:52:40.043 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.040+0000 7fccde2b4640 1 -- 172.21.15.192:0/866565096 shutdown_connections 2024-09-21T05:52:40.043 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.040+0000 7fccde2b4640 1 -- 172.21.15.192:0/866565096 wait complete. 2024-09-21T05:52:40.044 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.041+0000 7fccde2b4640 1 Processor -- start 2024-09-21T05:52:40.044 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.041+0000 7fccde2b4640 1 -- start start 2024-09-21T05:52:40.044 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.041+0000 7fccde2b4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd81049b0 0x7fccd819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:52:40.044 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.041+0000 7fccde2b4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccd8105300 0x7fccd819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:52:40.045 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.041+0000 7fccde2b4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccd810de10 0x7fccd819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:52:40.045 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.041+0000 7fccde2b4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fccd8113a90 con 0x7fccd81049b0 2024-09-21T05:52:40.045 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.041+0000 7fccde2b4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fccd8113910 con 0x7fccd810de10 2024-09-21T05:52:40.045 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.041+0000 7fccde2b4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fccd8113c10 con 0x7fccd8105300 2024-09-21T05:52:40.045 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.041+0000 7fccd77fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccd8105300 0x7fccd819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:52:40.045 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.041+0000 7fccd77fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccd8105300 0x7fccd819aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:58424/0 (socket says 172.21.15.192:58424) 2024-09-21T05:52:40.045 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.041+0000 7fccd77fe640 1 -- 172.21.15.192:0/2265250056 learned_addr learned my addr 172.21.15.192:0/2265250056 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-09-21T05:52:40.045 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.041+0000 7fccd7fff640 1 --2- 172.21.15.192:0/2265250056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd81049b0 0x7fccd819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:52:40.045 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.042+0000 7fccdc851640 1 --2- 172.21.15.192:0/2265250056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccd810de10 0x7fccd819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:52:40.045 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.042+0000 7fccd77fe640 1 -- 172.21.15.192:0/2265250056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccd810de10 msgr2=0x7fccd819fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:52:40.045 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.042+0000 7fccd77fe640 1 --2- 172.21.15.192:0/2265250056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccd810de10 0x7fccd819fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:52:40.045 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.042+0000 7fccd77fe640 1 -- 172.21.15.192:0/2265250056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd81049b0 msgr2=0x7fccd819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:52:40.045 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.042+0000 7fccd77fe640 1 --2- 172.21.15.192:0/2265250056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd81049b0 0x7fccd819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:52:40.045 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.042+0000 7fccd77fe640 1 -- 172.21.15.192:0/2265250056 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd81a03f0 con 0x7fccd8105300 2024-09-21T05:52:40.046 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.042+0000 7fccd77fe640 1 --2- 172.21.15.192:0/2265250056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccd8105300 0x7fccd819aa20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fccc000d750 tx=0x7fccc000dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:52:40.046 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.042+0000 7fccd57fa640 1 -- 172.21.15.192:0/2265250056 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0017070 con 0x7fccd8105300 2024-09-21T05:52:40.046 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.042+0000 7fccd57fa640 1 -- 172.21.15.192:0/2265250056 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccc00054d0 con 0x7fccd8105300 2024-09-21T05:52:40.046 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.042+0000 7fccde2b4640 1 -- 172.21.15.192:0/2265250056 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd81a06e0 con 0x7fccd8105300 2024-09-21T05:52:40.046 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.042+0000 7fccd57fa640 1 -- 172.21.15.192:0/2265250056 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0004c30 con 0x7fccd8105300 2024-09-21T05:52:40.046 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.042+0000 7fccde2b4640 1 -- 172.21.15.192:0/2265250056 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd81a6e40 con 0x7fccd8105300 2024-09-21T05:52:40.046 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.043+0000 7fccde2b4640 1 -- 172.21.15.192:0/2265250056 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccd806b560 con 0x7fccd8105300 2024-09-21T05:52:40.047 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.045+0000 7fccd57fa640 1 -- 172.21.15.192:0/2265250056 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7fccc0004dd0 con 0x7fccd8105300 2024-09-21T05:52:40.048 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.046+0000 7fccd57fa640 1 --2- 172.21.15.192:0/2265250056 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fccc40b6560 0x7fccc40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:52:40.048 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.046+0000 7fccd57fa640 1 -- 172.21.15.192:0/2265250056 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5332+0+0 (secure 0 0 0) 0x7fccc00e64e0 con 0x7fccd8105300 2024-09-21T05:52:40.048 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.046+0000 7fccd7fff640 1 --2- 172.21.15.192:0/2265250056 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fccc40b6560 0x7fccc40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:52:40.048 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.047+0000 7fccd7fff640 1 --2- 172.21.15.192:0/2265250056 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fccc40b6560 0x7fccc40b8a70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fccd806bc60 tx=0x7fccc800b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:52:40.051 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.049+0000 7fccd57fa640 1 -- 172.21.15.192:0/2265250056 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fccc00b1cf0 con 0x7fccd8105300 2024-09-21T05:52:40.258 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:52:40.256+0000 7fccde2b4640 1 -- 172.21.15.192:0/2265250056 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fccd8100c70 con 0x7fccc40b6560 2024-09-21T05:52:40.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:40 smithi138 ceph-mon[34391]: Detected new or changed devices on smithi138 2024-09-21T05:52:40.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:40 smithi138 ceph-mon[34391]: Adjusting osd_memory_target on smithi138 to 5696M 2024-09-21T05:52:40.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:40 smithi138 ceph-mon[34391]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-21T05:52:40.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:40 smithi138 ceph-mon[34391]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T05:52:40.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:40 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T05:52:40.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:40 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T05:52:40.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:40 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:52:40.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:40 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:52:40.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:40 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:52:40.875 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:40 smithi192 ceph-mon[34700]: Detected new or changed devices on smithi138 2024-09-21T05:52:40.875 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:40 smithi192 ceph-mon[34700]: Adjusting osd_memory_target on smithi138 to 5696M 2024-09-21T05:52:40.875 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:40 smithi192 ceph-mon[34700]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-21T05:52:40.875 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:40 smithi192 ceph-mon[34700]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T05:52:40.875 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:40 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T05:52:40.875 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:40 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T05:52:40.875 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:40 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:52:40.875 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:40 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:52:40.875 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:40 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:52:41.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:40 smithi117 ceph-mon[30947]: Detected new or changed devices on smithi138 2024-09-21T05:52:41.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:40 smithi117 ceph-mon[30947]: Adjusting osd_memory_target on smithi138 to 5696M 2024-09-21T05:52:41.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:40 smithi117 ceph-mon[30947]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-21T05:52:41.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:40 smithi117 ceph-mon[30947]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T05:52:41.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:40 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T05:52:41.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:40 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T05:52:41.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:40 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:52:41.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:40 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:52:41.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:40 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:52:41.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:41 smithi138 ceph-mon[34391]: purged_snaps scrub starts 2024-09-21T05:52:41.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:41 smithi138 ceph-mon[34391]: purged_snaps scrub ok 2024-09-21T05:52:41.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:41 smithi138 ceph-mon[34391]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:52:41.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:41 smithi138 ceph-mon[34391]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:41.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:41 smithi138 ceph-mon[34391]: osd.7 [v2:172.21.15.138:6816/1448179714,v1:172.21.15.138:6817/1448179714] boot 2024-09-21T05:52:41.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:41 smithi138 ceph-mon[34391]: osdmap e46: 8 total, 8 up, 8 in 2024-09-21T05:52:41.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:41 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T05:52:41.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:41 smithi138 ceph-mon[34391]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T05:52:41.891 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:41 smithi192 ceph-mon[34700]: purged_snaps scrub starts 2024-09-21T05:52:41.891 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:41 smithi192 ceph-mon[34700]: purged_snaps scrub ok 2024-09-21T05:52:41.891 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:41 smithi192 ceph-mon[34700]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:52:41.891 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:41 smithi192 ceph-mon[34700]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:41.891 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:41 smithi192 ceph-mon[34700]: osd.7 [v2:172.21.15.138:6816/1448179714,v1:172.21.15.138:6817/1448179714] boot 2024-09-21T05:52:41.891 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:41 smithi192 ceph-mon[34700]: osdmap e46: 8 total, 8 up, 8 in 2024-09-21T05:52:41.891 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:41 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T05:52:41.892 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:41 smithi192 ceph-mon[34700]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T05:52:42.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:41 smithi117 ceph-mon[30947]: purged_snaps scrub starts 2024-09-21T05:52:42.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:41 smithi117 ceph-mon[30947]: purged_snaps scrub ok 2024-09-21T05:52:42.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:41 smithi117 ceph-mon[30947]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:52:42.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:41 smithi117 ceph-mon[30947]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-09-21T05:52:42.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:41 smithi117 ceph-mon[30947]: osd.7 [v2:172.21.15.138:6816/1448179714,v1:172.21.15.138:6817/1448179714] boot 2024-09-21T05:52:42.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:41 smithi117 ceph-mon[30947]: osdmap e46: 8 total, 8 up, 8 in 2024-09-21T05:52:42.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:41 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T05:52:42.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:41 smithi117 ceph-mon[30947]: osdmap e47: 8 total, 8 up, 8 in 2024-09-21T05:52:43.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:43 smithi138 ceph-mon[34391]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T05:52:43.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:43 smithi138 ceph-mon[34391]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T05:52:43.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:43 smithi138 ceph-mon[34391]: from='client.? 172.21.15.192:0/802933610' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "118e0b5d-2619-4512-94c5-8d07a0d3a28f"} : dispatch 2024-09-21T05:52:43.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:43 smithi138 ceph-mon[34391]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "118e0b5d-2619-4512-94c5-8d07a0d3a28f"} : dispatch 2024-09-21T05:52:43.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:43 smithi138 ceph-mon[34391]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "118e0b5d-2619-4512-94c5-8d07a0d3a28f"}]': finished 2024-09-21T05:52:43.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:43 smithi138 ceph-mon[34391]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T05:52:43.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:43 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T05:52:43.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:43 smithi138 ceph-mon[34391]: from='client.? 172.21.15.192:0/2926003686' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:52:43.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:43 smithi192 ceph-mon[34700]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T05:52:43.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:43 smithi192 ceph-mon[34700]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T05:52:43.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:43 smithi192 ceph-mon[34700]: from='client.? 172.21.15.192:0/802933610' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "118e0b5d-2619-4512-94c5-8d07a0d3a28f"} : dispatch 2024-09-21T05:52:43.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:43 smithi192 ceph-mon[34700]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "118e0b5d-2619-4512-94c5-8d07a0d3a28f"} : dispatch 2024-09-21T05:52:43.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:43 smithi192 ceph-mon[34700]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "118e0b5d-2619-4512-94c5-8d07a0d3a28f"}]': finished 2024-09-21T05:52:43.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:43 smithi192 ceph-mon[34700]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T05:52:43.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:43 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T05:52:43.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:43 smithi192 ceph-mon[34700]: from='client.? 172.21.15.192:0/2926003686' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:52:44.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:43 smithi117 ceph-mon[30947]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T05:52:44.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:43 smithi117 ceph-mon[30947]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T05:52:44.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:43 smithi117 ceph-mon[30947]: from='client.? 172.21.15.192:0/802933610' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "118e0b5d-2619-4512-94c5-8d07a0d3a28f"} : dispatch 2024-09-21T05:52:44.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:43 smithi117 ceph-mon[30947]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "118e0b5d-2619-4512-94c5-8d07a0d3a28f"} : dispatch 2024-09-21T05:52:44.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:43 smithi117 ceph-mon[30947]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "118e0b5d-2619-4512-94c5-8d07a0d3a28f"}]': finished 2024-09-21T05:52:44.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:43 smithi117 ceph-mon[30947]: osdmap e49: 9 total, 8 up, 9 in 2024-09-21T05:52:44.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:43 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T05:52:44.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:43 smithi117 ceph-mon[30947]: from='client.? 172.21.15.192:0/2926003686' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:52:45.829 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:45 smithi117 ceph-mon[30947]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T05:52:45.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:45 smithi138 ceph-mon[34391]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T05:52:45.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:45 smithi192 ceph-mon[34700]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T05:52:47.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:47 smithi138 ceph-mon[34391]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-21T05:52:47.878 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:47 smithi192 ceph-mon[34700]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-21T05:52:48.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:47 smithi117 ceph-mon[30947]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-09-21T05:52:49.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:49 smithi192 ceph-mon[34700]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-21T05:52:49.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:49 smithi138 ceph-mon[34391]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-21T05:52:50.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:49 smithi117 ceph-mon[30947]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-21T05:52:51.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:51 smithi192 ceph-mon[34700]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T05:52:51.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:51 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T05:52:51.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:51 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:52:51.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:51 smithi138 ceph-mon[34391]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T05:52:51.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:51 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T05:52:51.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:51 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:52:52.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:51 smithi117 ceph-mon[30947]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T05:52:52.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:51 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T05:52:52.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:51 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:52:52.685 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:52 smithi192 ceph-mon[34700]: Deploying daemon osd.8 on smithi192 2024-09-21T05:52:52.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:52 smithi138 ceph-mon[34391]: Deploying daemon osd.8 on smithi192 2024-09-21T05:52:53.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:52 smithi117 ceph-mon[30947]: Deploying daemon osd.8 on smithi192 2024-09-21T05:52:53.825 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:53 smithi192 ceph-mon[34700]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T05:52:53.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:53 smithi138 ceph-mon[34391]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T05:52:54.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:53 smithi117 ceph-mon[30947]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T05:52:55.859 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:55 smithi192 ceph-mon[34700]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-21T05:52:55.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:55 smithi138 ceph-mon[34391]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-21T05:52:56.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:55 smithi117 ceph-mon[30947]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-21T05:52:57.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:57 smithi138 ceph-mon[34391]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T05:52:57.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:57 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:52:57.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:57 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:57.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:57 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:57.916 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:57 smithi192 ceph-mon[34700]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T05:52:57.916 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:57 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:52:57.916 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:57 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:57.916 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:57 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:58.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:57 smithi117 ceph-mon[30947]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T05:52:58.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:57 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:52:58.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:57 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:58.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:57 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:52:59.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:52:59 smithi138 ceph-mon[34391]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T05:52:59.877 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:52:59 smithi192 ceph-mon[34700]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T05:53:00.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:52:59 smithi117 ceph-mon[30947]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T05:53:01.066 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:01.064+0000 7fccd57fa640 1 -- 172.21.15.192:0/2265250056 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fccd8100c70 con 0x7fccc40b6560 2024-09-21T05:53:01.067 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 8 on host 'smithi192' 2024-09-21T05:53:01.069 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:01.067+0000 7fccde2b4640 1 -- 172.21.15.192:0/2265250056 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fccc40b6560 msgr2=0x7fccc40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:53:01.069 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:01.067+0000 7fccde2b4640 1 --2- 172.21.15.192:0/2265250056 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fccc40b6560 0x7fccc40b8a70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fccd806bc60 tx=0x7fccc800b390 comp rx=0 tx=0).stop 2024-09-21T05:53:01.069 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:01.067+0000 7fccde2b4640 1 -- 172.21.15.192:0/2265250056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccd8105300 msgr2=0x7fccd819aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:53:01.069 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:01.067+0000 7fccde2b4640 1 --2- 172.21.15.192:0/2265250056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccd8105300 0x7fccd819aa20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fccc000d750 tx=0x7fccc000dc10 comp rx=0 tx=0).stop 2024-09-21T05:53:01.069 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:01.067+0000 7fccde2b4640 1 -- 172.21.15.192:0/2265250056 shutdown_connections 2024-09-21T05:53:01.069 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:01.067+0000 7fccde2b4640 1 --2- 172.21.15.192:0/2265250056 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fccc40b6560 0x7fccc40b8a70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:53:01.070 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:01.067+0000 7fccde2b4640 1 --2- 172.21.15.192:0/2265250056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccd810de10 0x7fccd819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:53:01.070 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:01.067+0000 7fccde2b4640 1 --2- 172.21.15.192:0/2265250056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccd8105300 0x7fccd819aa20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:53:01.070 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:01.067+0000 7fccde2b4640 1 --2- 172.21.15.192:0/2265250056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd81049b0 0x7fccd819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:53:01.070 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:01.067+0000 7fccde2b4640 1 -- 172.21.15.192:0/2265250056 >> 172.21.15.192:0/2265250056 conn(0x7fccd8079730 msgr2=0x7fccd80ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:53:01.070 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:01.067+0000 7fccde2b4640 1 -- 172.21.15.192:0/2265250056 shutdown_connections 2024-09-21T05:53:01.070 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:01.067+0000 7fccde2b4640 1 -- 172.21.15.192:0/2265250056 wait complete. 2024-09-21T05:53:01.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:01 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:01.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:01 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:01.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:01 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:53:01.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:01 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:53:01.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:01 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:01.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:01 smithi117 ceph-mon[30947]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T05:53:01.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:01 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:01.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:01 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:01.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:01 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:53:01.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:01 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:53:01.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:01 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:01.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:01 smithi138 ceph-mon[34391]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T05:53:01.376 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:01 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:01.376 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:01 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:01.376 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:01 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:53:01.376 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:01 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:53:01.376 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:01 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:01.376 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:01 smithi192 ceph-mon[34700]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T05:53:01.772 DEBUG:teuthology.orchestra.run.smithi192:osd.8> sudo journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.8.service 2024-09-21T05:53:01.775 INFO:tasks.cephadm:Deploying osd.9 on smithi117 with /dev/vg_nvme/lv_1... 2024-09-21T05:53:01.775 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T05:53:02.036 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:53:02.157 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:02 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:53:02.157 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:02 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:02.157 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:02 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:02.298 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:02 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:53:02.298 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:02 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:02.298 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:02 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:02.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:02 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:53:02.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:02 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:02.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:02 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:02.927 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 05:53:02 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T05:53:02.608+0000 7feed3a2b640 -1 osd.8 0 log_to_monitors true 2024-09-21T05:53:03.207 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:03 smithi192 ceph-mon[34700]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T05:53:03.207 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:03 smithi192 ceph-mon[34700]: from='osd.8 [v2:172.21.15.192:6816/3960902885,v1:172.21.15.192:6817/3960902885]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T05:53:03.207 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:03 smithi192 ceph-mon[34700]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T05:53:03.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:03 smithi117 ceph-mon[30947]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T05:53:03.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:03 smithi117 ceph-mon[30947]: from='osd.8 [v2:172.21.15.192:6816/3960902885,v1:172.21.15.192:6817/3960902885]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T05:53:03.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:03 smithi117 ceph-mon[30947]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T05:53:03.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:03 smithi138 ceph-mon[34391]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T05:53:03.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:03 smithi138 ceph-mon[34391]: from='osd.8 [v2:172.21.15.192:6816/3960902885,v1:172.21.15.192:6817/3960902885]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T05:53:03.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:03 smithi138 ceph-mon[34391]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T05:53:04.159 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:04 smithi117 ceph-mon[30947]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T05:53:04.159 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:04 smithi117 ceph-mon[30947]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T05:53:04.159 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:04 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T05:53:04.159 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:04 smithi117 ceph-mon[30947]: from='osd.8 [v2:172.21.15.192:6816/3960902885,v1:172.21.15.192:6817/3960902885]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]} : dispatch 2024-09-21T05:53:04.159 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:04 smithi117 ceph-mon[30947]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]} : dispatch 2024-09-21T05:53:04.208 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:04 smithi192 ceph-mon[34700]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T05:53:04.209 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:04 smithi192 ceph-mon[34700]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T05:53:04.209 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:04 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T05:53:04.209 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:04 smithi192 ceph-mon[34700]: from='osd.8 [v2:172.21.15.192:6816/3960902885,v1:172.21.15.192:6817/3960902885]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]} : dispatch 2024-09-21T05:53:04.209 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:04 smithi192 ceph-mon[34700]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]} : dispatch 2024-09-21T05:53:04.209 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 05:53:04 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T05:53:04.032+0000 7feecf9ad640 -1 osd.8 0 waiting for initial osdmap 2024-09-21T05:53:04.209 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 05:53:04 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T05:53:04.041+0000 7feecb7ab640 -1 osd.8 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T05:53:04.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:04 smithi138 ceph-mon[34391]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T05:53:04.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:04 smithi138 ceph-mon[34391]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T05:53:04.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:04 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T05:53:04.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:04 smithi138 ceph-mon[34391]: from='osd.8 [v2:172.21.15.192:6816/3960902885,v1:172.21.15.192:6817/3960902885]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]} : dispatch 2024-09-21T05:53:04.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:04 smithi138 ceph-mon[34391]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]} : dispatch 2024-09-21T05:53:04.884 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:53:04.911 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi117:vg_nvme/lv_1 2024-09-21T05:53:05.164 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:53:05.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:05 smithi117 ceph-mon[30947]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-09-21T05:53:05.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:05 smithi117 ceph-mon[30947]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T05:53:05.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:05 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T05:53:05.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:05 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T05:53:05.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:05 smithi117 ceph-mon[30947]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T05:53:05.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:05 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:05.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:05 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:05.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:05 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T05:53:05.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:05 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T05:53:05.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:05 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T05:53:05.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:05 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:05.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:05 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:53:05.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:05 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:53:05.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:05 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:05.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:05 smithi138 ceph-mon[34391]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-09-21T05:53:05.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:05 smithi138 ceph-mon[34391]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T05:53:05.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:05 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T05:53:05.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:05 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T05:53:05.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:05 smithi138 ceph-mon[34391]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T05:53:05.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:05 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:05.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:05 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:05.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:05 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T05:53:05.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:05 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T05:53:05.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:05 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T05:53:05.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:05 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:05.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:05 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:53:05.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:05 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:53:05.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:05 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:05.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:05 smithi192 ceph-mon[34700]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-09-21T05:53:05.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:05 smithi192 ceph-mon[34700]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T05:53:05.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:05 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T05:53:05.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:05 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T05:53:05.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:05 smithi192 ceph-mon[34700]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-09-21T05:53:05.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:05 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:05.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:05 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:05.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:05 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T05:53:05.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:05 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T05:53:05.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:05 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T05:53:05.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:05 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:05.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:05 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:53:05.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:05 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:53:05.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:05 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:05.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.676+0000 7f77c1baa640 1 -- 172.21.15.117:0/2169005642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77bc1094b0 msgr2=0x7f77bc1098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:53:05.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.676+0000 7f77c1baa640 1 --2- 172.21.15.117:0/2169005642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77bc1094b0 0x7f77bc1098c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f77b0009900 tx=0x7f77b002a410 comp rx=0 tx=0).stop 2024-09-21T05:53:05.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.676+0000 7f77c1baa640 1 -- 172.21.15.117:0/2169005642 shutdown_connections 2024-09-21T05:53:05.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.676+0000 7f77c1baa640 1 --2- 172.21.15.117:0/2169005642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77bc1094b0 0x7f77bc1098c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:53:05.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.676+0000 7f77c1baa640 1 --2- 172.21.15.117:0/2169005642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77bc109e00 0x7f77bc110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:53:05.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.676+0000 7f77c1baa640 1 --2- 172.21.15.117:0/2169005642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77bc103b70 0x7f77bc103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:53:05.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.676+0000 7f77c1baa640 1 -- 172.21.15.117:0/2169005642 >> 172.21.15.117:0/2169005642 conn(0x7f77bc0ff5e0 msgr2=0x7f77bc101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:53:05.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.676+0000 7f77c1baa640 1 -- 172.21.15.117:0/2169005642 shutdown_connections 2024-09-21T05:53:05.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.676+0000 7f77c1baa640 1 -- 172.21.15.117:0/2169005642 wait complete. 2024-09-21T05:53:05.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.677+0000 7f77c1baa640 1 Processor -- start 2024-09-21T05:53:05.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.677+0000 7f77c1baa640 1 -- start start 2024-09-21T05:53:05.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.677+0000 7f77c1baa640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77bc103b70 0x7f77bc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:53:05.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.677+0000 7f77c1baa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77bc1094b0 0x7f77bc075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:53:05.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.677+0000 7f77c1baa640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77bc109e00 0x7f77bc076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:53:05.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.677+0000 7f77c1baa640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc113ac0 con 0x7f77bc109e00 2024-09-21T05:53:05.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.677+0000 7f77c1baa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc113940 con 0x7f77bc1094b0 2024-09-21T05:53:05.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.677+0000 7f77c1baa640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc113c40 con 0x7f77bc103b70 2024-09-21T05:53:05.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.677+0000 7f77bbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77bc109e00 0x7f77bc076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:53:05.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.677+0000 7f77bbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77bc109e00 0x7f77bc076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57596/0 (socket says 172.21.15.117:57596) 2024-09-21T05:53:05.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.677+0000 7f77bbfff640 1 -- 172.21.15.117:0/1357585612 learned_addr learned my addr 172.21.15.117:0/1357585612 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:53:05.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.678+0000 7f77baffd640 1 --2- 172.21.15.117:0/1357585612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77bc1094b0 0x7f77bc075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:53:05.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.678+0000 7f77bb7fe640 1 --2- 172.21.15.117:0/1357585612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77bc103b70 0x7f77bc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:53:05.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.678+0000 7f77bbfff640 1 -- 172.21.15.117:0/1357585612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77bc103b70 msgr2=0x7f77bc075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:53:05.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.678+0000 7f77bbfff640 1 --2- 172.21.15.117:0/1357585612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77bc103b70 0x7f77bc075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:53:05.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.678+0000 7f77bbfff640 1 -- 172.21.15.117:0/1357585612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77bc1094b0 msgr2=0x7f77bc075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:53:05.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.678+0000 7f77bbfff640 1 --2- 172.21.15.117:0/1357585612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77bc1094b0 0x7f77bc075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:53:05.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.678+0000 7f77bbfff640 1 -- 172.21.15.117:0/1357585612 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77bc1a8790 con 0x7f77bc109e00 2024-09-21T05:53:05.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.678+0000 7f77bb7fe640 1 --2- 172.21.15.117:0/1357585612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77bc103b70 0x7f77bc075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:53:05.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.678+0000 7f77bbfff640 1 --2- 172.21.15.117:0/1357585612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77bc109e00 0x7f77bc076190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f77b400c970 tx=0x7f77b400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:53:05.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.678+0000 7f77b8ff9640 1 -- 172.21.15.117:0/1357585612 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b4007bb0 con 0x7f77bc109e00 2024-09-21T05:53:05.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.678+0000 7f77b8ff9640 1 -- 172.21.15.117:0/1357585612 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77b4007d50 con 0x7f77bc109e00 2024-09-21T05:53:05.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.678+0000 7f77c1baa640 1 -- 172.21.15.117:0/1357585612 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77bc1a8a80 con 0x7f77bc109e00 2024-09-21T05:53:05.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.678+0000 7f77b8ff9640 1 -- 172.21.15.117:0/1357585612 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b4004e60 con 0x7f77bc109e00 2024-09-21T05:53:05.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.678+0000 7f77c1baa640 1 -- 172.21.15.117:0/1357585612 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77bc1a8f40 con 0x7f77bc109e00 2024-09-21T05:53:05.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.679+0000 7f77c1baa640 1 -- 172.21.15.117:0/1357585612 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77bc103f80 con 0x7f77bc109e00 2024-09-21T05:53:05.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.681+0000 7f77b8ff9640 1 -- 172.21.15.117:0/1357585612 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f77b4005000 con 0x7f77bc109e00 2024-09-21T05:53:05.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.681+0000 7f77b8ff9640 1 --2- 172.21.15.117:0/1357585612 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f77a40b6560 0x7f77a40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:53:05.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.681+0000 7f77bb7fe640 1 --2- 172.21.15.117:0/1357585612 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f77a40b6560 0x7f77a40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:53:05.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.682+0000 7f77b8ff9640 1 -- 172.21.15.117:0/1357585612 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6077+0+0 (secure 0 0 0) 0x7f77b40e6b40 con 0x7f77bc109e00 2024-09-21T05:53:05.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.682+0000 7f77bb7fe640 1 --2- 172.21.15.117:0/1357585612 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f77a40b6560 0x7f77a40b8a70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f77bc077160 tx=0x7f77ac009520 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:53:05.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.685+0000 7f77b8ff9640 1 -- 172.21.15.117:0/1357585612 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f77b4014030 con 0x7f77bc109e00 2024-09-21T05:53:05.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:05.872+0000 7f77c1baa640 1 -- 172.21.15.117:0/1357585612 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f77bc063800 con 0x7f77a40b6560 2024-09-21T05:53:06.252 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:06 smithi117 ceph-mon[30947]: purged_snaps scrub starts 2024-09-21T05:53:06.252 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:06 smithi117 ceph-mon[30947]: purged_snaps scrub ok 2024-09-21T05:53:06.252 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:06 smithi117 ceph-mon[30947]: Detected new or changed devices on smithi192 2024-09-21T05:53:06.252 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:06 smithi117 ceph-mon[30947]: Adjusting osd_memory_target on smithi192 to 5696M 2024-09-21T05:53:06.252 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:06 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T05:53:06.252 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:06 smithi117 ceph-mon[30947]: osd.8 [v2:172.21.15.192:6816/3960902885,v1:172.21.15.192:6817/3960902885] boot 2024-09-21T05:53:06.252 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:06 smithi117 ceph-mon[30947]: osdmap e52: 9 total, 9 up, 9 in 2024-09-21T05:53:06.252 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:06 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T05:53:06.252 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:06 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:53:06.252 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:06 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:53:06.253 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:06 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:53:06.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:06 smithi138 ceph-mon[34391]: purged_snaps scrub starts 2024-09-21T05:53:06.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:06 smithi138 ceph-mon[34391]: purged_snaps scrub ok 2024-09-21T05:53:06.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:06 smithi138 ceph-mon[34391]: Detected new or changed devices on smithi192 2024-09-21T05:53:06.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:06 smithi138 ceph-mon[34391]: Adjusting osd_memory_target on smithi192 to 5696M 2024-09-21T05:53:06.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:06 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T05:53:06.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:06 smithi138 ceph-mon[34391]: osd.8 [v2:172.21.15.192:6816/3960902885,v1:172.21.15.192:6817/3960902885] boot 2024-09-21T05:53:06.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:06 smithi138 ceph-mon[34391]: osdmap e52: 9 total, 9 up, 9 in 2024-09-21T05:53:06.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:06 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T05:53:06.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:06 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:53:06.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:06 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:53:06.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:06 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:53:06.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:06 smithi192 ceph-mon[34700]: purged_snaps scrub starts 2024-09-21T05:53:06.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:06 smithi192 ceph-mon[34700]: purged_snaps scrub ok 2024-09-21T05:53:06.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:06 smithi192 ceph-mon[34700]: Detected new or changed devices on smithi192 2024-09-21T05:53:06.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:06 smithi192 ceph-mon[34700]: Adjusting osd_memory_target on smithi192 to 5696M 2024-09-21T05:53:06.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:06 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T05:53:06.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:06 smithi192 ceph-mon[34700]: osd.8 [v2:172.21.15.192:6816/3960902885,v1:172.21.15.192:6817/3960902885] boot 2024-09-21T05:53:06.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:06 smithi192 ceph-mon[34700]: osdmap e52: 9 total, 9 up, 9 in 2024-09-21T05:53:06.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:06 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T05:53:06.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:06 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:53:06.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:06 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:53:06.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:06 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:53:07.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:07 smithi117 ceph-mon[30947]: from='client.14655 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:53:07.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:07 smithi117 ceph-mon[30947]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T05:53:07.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:07 smithi117 ceph-mon[30947]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail 2024-09-21T05:53:07.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:07 smithi138 ceph-mon[34391]: from='client.14655 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:53:07.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:07 smithi138 ceph-mon[34391]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T05:53:07.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:07 smithi138 ceph-mon[34391]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail 2024-09-21T05:53:07.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:07 smithi192 ceph-mon[34700]: from='client.14655 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:53:07.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:07 smithi192 ceph-mon[34700]: osdmap e53: 9 total, 9 up, 9 in 2024-09-21T05:53:07.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:07 smithi192 ceph-mon[34700]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail 2024-09-21T05:53:08.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:08 smithi117 ceph-mon[30947]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T05:53:08.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:08 smithi138 ceph-mon[34391]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T05:53:08.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:08 smithi192 ceph-mon[34700]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T05:53:09.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:09 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/1186705453' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "47543bc6-fa0b-4e72-9742-97dcda0b7f8d"} : dispatch 2024-09-21T05:53:09.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:09 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/1186705453' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47543bc6-fa0b-4e72-9742-97dcda0b7f8d"}]': finished 2024-09-21T05:53:09.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:09 smithi117 ceph-mon[30947]: osdmap e55: 10 total, 9 up, 10 in 2024-09-21T05:53:09.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:09 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T05:53:09.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:09 smithi117 ceph-mon[30947]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T05:53:09.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:09 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3719214509' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:53:09.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:09 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/1186705453' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "47543bc6-fa0b-4e72-9742-97dcda0b7f8d"} : dispatch 2024-09-21T05:53:09.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:09 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/1186705453' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47543bc6-fa0b-4e72-9742-97dcda0b7f8d"}]': finished 2024-09-21T05:53:09.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:09 smithi138 ceph-mon[34391]: osdmap e55: 10 total, 9 up, 10 in 2024-09-21T05:53:09.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:09 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T05:53:09.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:09 smithi138 ceph-mon[34391]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T05:53:09.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:09 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3719214509' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:53:09.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:09 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/1186705453' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "47543bc6-fa0b-4e72-9742-97dcda0b7f8d"} : dispatch 2024-09-21T05:53:09.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:09 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/1186705453' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47543bc6-fa0b-4e72-9742-97dcda0b7f8d"}]': finished 2024-09-21T05:53:09.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:09 smithi192 ceph-mon[34700]: osdmap e55: 10 total, 9 up, 10 in 2024-09-21T05:53:09.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:09 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T05:53:09.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:09 smithi192 ceph-mon[34700]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T05:53:09.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:09 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3719214509' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:53:11.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:11 smithi138 ceph-mon[34391]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T05:53:11.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:11 smithi192 ceph-mon[34700]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T05:53:12.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:11 smithi117 ceph-mon[30947]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T05:53:13.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:13 smithi138 ceph-mon[34391]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-21T05:53:13.908 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:13 smithi117 ceph-mon[30947]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-21T05:53:13.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:13 smithi192 ceph-mon[34700]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-21T05:53:15.796 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:15 smithi192 ceph-mon[34700]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T05:53:15.845 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:15 smithi117 ceph-mon[30947]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T05:53:15.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:15 smithi138 ceph-mon[34391]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-21T05:53:17.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:17 smithi117 ceph-mon[30947]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-21T05:53:17.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:17 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T05:53:17.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:17 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:53:17.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:17 smithi138 ceph-mon[34391]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-21T05:53:17.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:17 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T05:53:17.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:17 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:53:17.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:17 smithi192 ceph-mon[34700]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-21T05:53:17.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:17 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T05:53:17.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:17 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:53:18.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:18 smithi117 ceph-mon[30947]: Deploying daemon osd.9 on smithi117 2024-09-21T05:53:18.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:18 smithi138 ceph-mon[34391]: Deploying daemon osd.9 on smithi117 2024-09-21T05:53:18.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:18 smithi192 ceph-mon[34700]: Deploying daemon osd.9 on smithi117 2024-09-21T05:53:19.683 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:19 smithi117 ceph-mon[30947]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-21T05:53:19.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:19 smithi138 ceph-mon[34391]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-21T05:53:19.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:19 smithi192 ceph-mon[34700]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-21T05:53:21.691 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:21 smithi117 ceph-mon[30947]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T05:53:21.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:21 smithi138 ceph-mon[34391]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T05:53:21.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:21 smithi192 ceph-mon[34700]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T05:53:23.686 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:23 smithi117 ceph-mon[30947]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T05:53:23.686 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:53:23.686 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:23.686 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:23.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:23 smithi138 ceph-mon[34391]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T05:53:23.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:53:23.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:23.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:23.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:23 smithi192 ceph-mon[34700]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T05:53:23.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:53:23.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:23.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:25.829 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:25 smithi117 ceph-mon[30947]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T05:53:25.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:25 smithi138 ceph-mon[34391]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T05:53:25.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:25 smithi192 ceph-mon[34700]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T05:53:26.958 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 9 on host 'smithi117' 2024-09-21T05:53:26.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:26.954+0000 7f77b8ff9640 1 -- 172.21.15.117:0/1357585612 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f77bc063800 con 0x7f77a40b6560 2024-09-21T05:53:26.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:26.957+0000 7f77c1baa640 1 -- 172.21.15.117:0/1357585612 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f77a40b6560 msgr2=0x7f77a40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:53:26.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:26.957+0000 7f77c1baa640 1 --2- 172.21.15.117:0/1357585612 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f77a40b6560 0x7f77a40b8a70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f77bc077160 tx=0x7f77ac009520 comp rx=0 tx=0).stop 2024-09-21T05:53:26.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:26.957+0000 7f77c1baa640 1 -- 172.21.15.117:0/1357585612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77bc109e00 msgr2=0x7f77bc076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:53:26.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:26.957+0000 7f77c1baa640 1 --2- 172.21.15.117:0/1357585612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77bc109e00 0x7f77bc076190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f77b400c970 tx=0x7f77b400ce30 comp rx=0 tx=0).stop 2024-09-21T05:53:26.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:26.957+0000 7f77c1baa640 1 -- 172.21.15.117:0/1357585612 shutdown_connections 2024-09-21T05:53:26.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:26.957+0000 7f77c1baa640 1 --2- 172.21.15.117:0/1357585612 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f77a40b6560 0x7f77a40b8a70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:53:26.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:26.957+0000 7f77c1baa640 1 --2- 172.21.15.117:0/1357585612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77bc109e00 0x7f77bc076190 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:53:26.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:26.957+0000 7f77c1baa640 1 --2- 172.21.15.117:0/1357585612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77bc1094b0 0x7f77bc075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:53:26.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:26.957+0000 7f77c1baa640 1 --2- 172.21.15.117:0/1357585612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77bc103b70 0x7f77bc075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:53:26.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:26.957+0000 7f77c1baa640 1 -- 172.21.15.117:0/1357585612 >> 172.21.15.117:0/1357585612 conn(0x7f77bc0ff5e0 msgr2=0x7f77bc101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:53:26.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:26.957+0000 7f77c1baa640 1 -- 172.21.15.117:0/1357585612 shutdown_connections 2024-09-21T05:53:26.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:53:26.957+0000 7f77c1baa640 1 -- 172.21.15.117:0/1357585612 wait complete. 2024-09-21T05:53:27.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:27 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:27.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:27 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:27.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:27 smithi117 ceph-mon[30947]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T05:53:27.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:27 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:53:27.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:27 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:53:27.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:27 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:27.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:27 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:53:27.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:27 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:27.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:27 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:27.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:27 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:27.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:27 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:27.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:27 smithi138 ceph-mon[34391]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T05:53:27.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:27 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:53:27.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:27 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:53:27.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:27 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:27.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:27 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:53:27.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:27 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:27.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:27 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:27.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:27 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:27.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:27 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:27.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:27 smithi192 ceph-mon[34700]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T05:53:27.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:27 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:53:27.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:27 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:53:27.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:27 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:27.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:27 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:53:27.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:27 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:27.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:27 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:27.765 DEBUG:teuthology.orchestra.run.smithi117:osd.9> sudo journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.9.service 2024-09-21T05:53:27.768 INFO:tasks.cephadm:Deploying osd.10 on smithi138 with /dev/vg_nvme/lv_1... 2024-09-21T05:53:27.768 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T05:53:28.025 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T05:53:29.075 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 05:53:28 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T05:53:28.778+0000 7f65b9436640 -1 osd.9 0 log_to_monitors true 2024-09-21T05:53:29.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:29 smithi117 ceph-mon[30947]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T05:53:29.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:29 smithi117 ceph-mon[30947]: from='osd.9 [v2:172.21.15.117:6826/3453855195,v1:172.21.15.117:6827/3453855195]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T05:53:29.792 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:29 smithi138 ceph-mon[34391]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T05:53:29.792 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:29 smithi138 ceph-mon[34391]: from='osd.9 [v2:172.21.15.117:6826/3453855195,v1:172.21.15.117:6827/3453855195]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T05:53:29.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:29 smithi192 ceph-mon[34700]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T05:53:29.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:29 smithi192 ceph-mon[34700]: from='osd.9 [v2:172.21.15.117:6826/3453855195,v1:172.21.15.117:6827/3453855195]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T05:53:30.707 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:30 smithi117 ceph-mon[30947]: from='osd.9 [v2:172.21.15.117:6826/3453855195,v1:172.21.15.117:6827/3453855195]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T05:53:30.707 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:30 smithi117 ceph-mon[30947]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T05:53:30.708 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:30 smithi117 ceph-mon[30947]: from='osd.9 [v2:172.21.15.117:6826/3453855195,v1:172.21.15.117:6827/3453855195]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-21T05:53:30.708 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:30 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T05:53:30.708 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 05:53:30 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T05:53:30.551+0000 7f65b53b8640 -1 osd.9 0 waiting for initial osdmap 2024-09-21T05:53:30.708 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 05:53:30 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T05:53:30.561+0000 7f65b11b6640 -1 osd.9 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T05:53:30.772 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:30 smithi138 ceph-mon[34391]: from='osd.9 [v2:172.21.15.117:6826/3453855195,v1:172.21.15.117:6827/3453855195]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T05:53:30.772 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:30 smithi138 ceph-mon[34391]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T05:53:30.772 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:30 smithi138 ceph-mon[34391]: from='osd.9 [v2:172.21.15.117:6826/3453855195,v1:172.21.15.117:6827/3453855195]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-21T05:53:30.772 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:30 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T05:53:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-21T05:53:30.803 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi138:vg_nvme/lv_1 2024-09-21T05:53:30.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:30 smithi192 ceph-mon[34700]: from='osd.9 [v2:172.21.15.117:6826/3453855195,v1:172.21.15.117:6827/3453855195]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T05:53:30.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:30 smithi192 ceph-mon[34700]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T05:53:30.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:30 smithi192 ceph-mon[34700]: from='osd.9 [v2:172.21.15.117:6826/3453855195,v1:172.21.15.117:6827/3453855195]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-21T05:53:30.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:30 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T05:53:31.053 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T05:53:31.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.526+0000 7fb48185b640 1 -- 172.21.15.138:0/976260988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb47c103b90 msgr2=0x7fb47c103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:53:31.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.526+0000 7fb48185b640 1 --2- 172.21.15.138:0/976260988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb47c103b90 0x7fb47c103fa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb464009900 tx=0x7fb46402a410 comp rx=0 tx=0).stop 2024-09-21T05:53:31.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.526+0000 7fb48185b640 1 -- 172.21.15.138:0/976260988 shutdown_connections 2024-09-21T05:53:31.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.526+0000 7fb48185b640 1 --2- 172.21.15.138:0/976260988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb47c1094d0 0x7fb47c1098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:53:31.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.526+0000 7fb48185b640 1 --2- 172.21.15.138:0/976260988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb47c109e20 0x7fb47c110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:53:31.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.526+0000 7fb48185b640 1 --2- 172.21.15.138:0/976260988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb47c103b90 0x7fb47c103fa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:53:31.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.526+0000 7fb48185b640 1 -- 172.21.15.138:0/976260988 >> 172.21.15.138:0/976260988 conn(0x7fb47c0ff600 msgr2=0x7fb47c101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:53:31.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.526+0000 7fb48185b640 1 -- 172.21.15.138:0/976260988 shutdown_connections 2024-09-21T05:53:31.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.526+0000 7fb48185b640 1 -- 172.21.15.138:0/976260988 wait complete. 2024-09-21T05:53:31.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.526+0000 7fb48185b640 1 Processor -- start 2024-09-21T05:53:31.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.527+0000 7fb48185b640 1 -- start start 2024-09-21T05:53:31.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.527+0000 7fb48185b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb47c103b90 0x7fb47c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:53:31.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.527+0000 7fb48185b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb47c1094d0 0x7fb47c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:53:31.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.527+0000 7fb48185b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb47c109e20 0x7fb47c076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:53:31.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.527+0000 7fb48185b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c113b40 con 0x7fb47c1094d0 2024-09-21T05:53:31.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.527+0000 7fb48185b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c1139c0 con 0x7fb47c103b90 2024-09-21T05:53:31.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.527+0000 7fb48185b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb47c113cc0 con 0x7fb47c109e20 2024-09-21T05:53:31.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.527+0000 7fb47affd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb47c103b90 0x7fb47c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:53:31.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.527+0000 7fb47affd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb47c103b90 0x7fb47c075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54350/0 (socket says 172.21.15.138:54350) 2024-09-21T05:53:31.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.527+0000 7fb47affd640 1 -- 172.21.15.138:0/3497340641 learned_addr learned my addr 172.21.15.138:0/3497340641 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-09-21T05:53:31.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.527+0000 7fb47b7fe640 1 --2- 172.21.15.138:0/3497340641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb47c109e20 0x7fb47c076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:53:31.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.527+0000 7fb47a7fc640 1 --2- 172.21.15.138:0/3497340641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb47c1094d0 0x7fb47c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:53:31.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.527+0000 7fb47affd640 1 -- 172.21.15.138:0/3497340641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb47c109e20 msgr2=0x7fb47c076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:53:31.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.527+0000 7fb47affd640 1 --2- 172.21.15.138:0/3497340641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb47c109e20 0x7fb47c076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:53:31.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.527+0000 7fb47affd640 1 -- 172.21.15.138:0/3497340641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb47c1094d0 msgr2=0x7fb47c075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:53:31.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.527+0000 7fb47affd640 1 --2- 172.21.15.138:0/3497340641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb47c1094d0 0x7fb47c075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:53:31.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.527+0000 7fb47affd640 1 -- 172.21.15.138:0/3497340641 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb47c1a87b0 con 0x7fb47c103b90 2024-09-21T05:53:31.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.528+0000 7fb47affd640 1 --2- 172.21.15.138:0/3497340641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb47c103b90 0x7fb47c075710 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb464005940 tx=0x7fb4640042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:53:31.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.528+0000 7fb480859640 1 -- 172.21.15.138:0/3497340641 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46403f030 con 0x7fb47c103b90 2024-09-21T05:53:31.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.528+0000 7fb480859640 1 -- 172.21.15.138:0/3497340641 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb464005d10 con 0x7fb47c103b90 2024-09-21T05:53:31.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.528+0000 7fb48185b640 1 -- 172.21.15.138:0/3497340641 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb47c1a8a40 con 0x7fb47c103b90 2024-09-21T05:53:31.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.528+0000 7fb480859640 1 -- 172.21.15.138:0/3497340641 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb464035070 con 0x7fb47c103b90 2024-09-21T05:53:31.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.528+0000 7fb48185b640 1 -- 172.21.15.138:0/3497340641 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb47c1a8fa0 con 0x7fb47c103b90 2024-09-21T05:53:31.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.529+0000 7fb48185b640 1 -- 172.21.15.138:0/3497340641 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb47c104e10 con 0x7fb47c103b90 2024-09-21T05:53:31.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.530+0000 7fb480859640 1 -- 172.21.15.138:0/3497340641 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7fb464042030 con 0x7fb47c103b90 2024-09-21T05:53:31.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.531+0000 7fb480859640 1 --2- 172.21.15.138:0/3497340641 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb4680b6560 0x7fb4680b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:53:31.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.531+0000 7fb480859640 1 -- 172.21.15.138:0/3497340641 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 6196+0+0 (secure 0 0 0) 0x7fb4641072b0 con 0x7fb47c103b90 2024-09-21T05:53:31.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.531+0000 7fb47a7fc640 1 --2- 172.21.15.138:0/3497340641 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb4680b6560 0x7fb4680b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:53:31.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.532+0000 7fb47a7fc640 1 --2- 172.21.15.138:0/3497340641 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb4680b6560 0x7fb4680b8a70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb47c108ff0 tx=0x7fb46c009110 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:53:31.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.535+0000 7fb480859640 1 -- 172.21.15.138:0/3497340641 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb4640ce380 con 0x7fb47c103b90 2024-09-21T05:53:31.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:31.722+0000 7fb48185b640 1 -- 172.21.15.138:0/3497340641 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fb47c063800 con 0x7fb4680b6560 2024-09-21T05:53:31.726 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:31 smithi138 ceph-mon[34391]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T05:53:31.726 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:31 smithi138 ceph-mon[34391]: from='osd.9 [v2:172.21.15.117:6826/3453855195,v1:172.21.15.117:6827/3453855195]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-21T05:53:31.726 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:31 smithi138 ceph-mon[34391]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T05:53:31.726 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:31 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T05:53:31.726 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:31 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T05:53:31.726 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:31 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:31.726 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:31 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:31.726 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:31 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T05:53:31.726 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:31 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T05:53:31.727 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:31 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T05:53:31.727 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:31 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T05:53:31.727 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:31 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:31.727 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:31 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:53:31.727 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:31 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:53:31.727 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:31 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:31.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:31 smithi192 ceph-mon[34700]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T05:53:31.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:31 smithi192 ceph-mon[34700]: from='osd.9 [v2:172.21.15.117:6826/3453855195,v1:172.21.15.117:6827/3453855195]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-21T05:53:31.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:31 smithi192 ceph-mon[34700]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T05:53:31.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:31 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T05:53:31.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:31 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T05:53:31.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:31 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:31.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:31 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:31.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:31 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T05:53:31.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:31 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T05:53:31.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:31 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T05:53:31.981 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:31 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T05:53:31.981 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:31 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:31.981 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:31 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:53:31.981 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:31 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:53:31.981 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:31 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:32.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:31 smithi117 ceph-mon[30947]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-09-21T05:53:32.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:31 smithi117 ceph-mon[30947]: from='osd.9 [v2:172.21.15.117:6826/3453855195,v1:172.21.15.117:6827/3453855195]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-21T05:53:32.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:31 smithi117 ceph-mon[30947]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T05:53:32.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:31 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T05:53:32.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:31 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T05:53:32.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:31 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:32.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:31 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:32.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:31 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T05:53:32.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:31 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T05:53:32.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:31 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T05:53:32.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:31 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T05:53:32.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:31 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:32.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:31 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:53:32.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:31 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:53:32.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:31 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:32.716 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:32 smithi138 ceph-mon[34391]: purged_snaps scrub starts 2024-09-21T05:53:32.716 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:32 smithi138 ceph-mon[34391]: purged_snaps scrub ok 2024-09-21T05:53:32.716 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:32 smithi138 ceph-mon[34391]: Detected new or changed devices on smithi117 2024-09-21T05:53:32.716 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:32 smithi138 ceph-mon[34391]: Adjusting osd_memory_target on smithi117 to 4272M 2024-09-21T05:53:32.716 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:32 smithi138 ceph-mon[34391]: osd.9 [v2:172.21.15.117:6826/3453855195,v1:172.21.15.117:6827/3453855195] boot 2024-09-21T05:53:32.717 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:32 smithi138 ceph-mon[34391]: osdmap e58: 10 total, 10 up, 10 in 2024-09-21T05:53:32.717 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:32 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T05:53:32.717 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:32 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:53:32.717 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:32 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:53:32.717 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:32 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:53:32.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:32 smithi192 ceph-mon[34700]: purged_snaps scrub starts 2024-09-21T05:53:32.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:32 smithi192 ceph-mon[34700]: purged_snaps scrub ok 2024-09-21T05:53:32.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:32 smithi192 ceph-mon[34700]: Detected new or changed devices on smithi117 2024-09-21T05:53:32.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:32 smithi192 ceph-mon[34700]: Adjusting osd_memory_target on smithi117 to 4272M 2024-09-21T05:53:32.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:32 smithi192 ceph-mon[34700]: osd.9 [v2:172.21.15.117:6826/3453855195,v1:172.21.15.117:6827/3453855195] boot 2024-09-21T05:53:32.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:32 smithi192 ceph-mon[34700]: osdmap e58: 10 total, 10 up, 10 in 2024-09-21T05:53:32.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:32 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T05:53:32.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:32 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:53:32.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:32 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:53:32.981 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:32 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:53:33.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:32 smithi117 ceph-mon[30947]: purged_snaps scrub starts 2024-09-21T05:53:33.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:32 smithi117 ceph-mon[30947]: purged_snaps scrub ok 2024-09-21T05:53:33.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:32 smithi117 ceph-mon[30947]: Detected new or changed devices on smithi117 2024-09-21T05:53:33.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:32 smithi117 ceph-mon[30947]: Adjusting osd_memory_target on smithi117 to 4272M 2024-09-21T05:53:33.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:32 smithi117 ceph-mon[30947]: osd.9 [v2:172.21.15.117:6826/3453855195,v1:172.21.15.117:6827/3453855195] boot 2024-09-21T05:53:33.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:32 smithi117 ceph-mon[30947]: osdmap e58: 10 total, 10 up, 10 in 2024-09-21T05:53:33.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:32 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T05:53:33.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:32 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:53:33.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:32 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:53:33.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:32 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:53:33.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:33 smithi138 ceph-mon[34391]: from='client.24482 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:53:33.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:33 smithi138 ceph-mon[34391]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:33.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:33 smithi138 ceph-mon[34391]: osdmap e59: 10 total, 10 up, 10 in 2024-09-21T05:53:33.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:33 smithi192 ceph-mon[34700]: from='client.24482 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:53:33.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:33 smithi192 ceph-mon[34700]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:33.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:33 smithi192 ceph-mon[34700]: osdmap e59: 10 total, 10 up, 10 in 2024-09-21T05:53:34.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:33 smithi117 ceph-mon[30947]: from='client.24482 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:53:34.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:33 smithi117 ceph-mon[30947]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:34.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:33 smithi117 ceph-mon[30947]: osdmap e59: 10 total, 10 up, 10 in 2024-09-21T05:53:34.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:34 smithi138 ceph-mon[34391]: from='client.? 172.21.15.138:0/3828933779' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5e6708b4-7a85-4fa5-a03c-27d99f7d749f"} : dispatch 2024-09-21T05:53:34.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:34 smithi138 ceph-mon[34391]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5e6708b4-7a85-4fa5-a03c-27d99f7d749f"} : dispatch 2024-09-21T05:53:34.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:34 smithi138 ceph-mon[34391]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5e6708b4-7a85-4fa5-a03c-27d99f7d749f"}]': finished 2024-09-21T05:53:34.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:34 smithi138 ceph-mon[34391]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T05:53:34.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:34 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T05:53:34.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:34 smithi192 ceph-mon[34700]: from='client.? 172.21.15.138:0/3828933779' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5e6708b4-7a85-4fa5-a03c-27d99f7d749f"} : dispatch 2024-09-21T05:53:34.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:34 smithi192 ceph-mon[34700]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5e6708b4-7a85-4fa5-a03c-27d99f7d749f"} : dispatch 2024-09-21T05:53:34.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:34 smithi192 ceph-mon[34700]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5e6708b4-7a85-4fa5-a03c-27d99f7d749f"}]': finished 2024-09-21T05:53:34.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:34 smithi192 ceph-mon[34700]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T05:53:34.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:34 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T05:53:35.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:34 smithi117 ceph-mon[30947]: from='client.? 172.21.15.138:0/3828933779' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5e6708b4-7a85-4fa5-a03c-27d99f7d749f"} : dispatch 2024-09-21T05:53:35.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:34 smithi117 ceph-mon[30947]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5e6708b4-7a85-4fa5-a03c-27d99f7d749f"} : dispatch 2024-09-21T05:53:35.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:34 smithi117 ceph-mon[30947]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5e6708b4-7a85-4fa5-a03c-27d99f7d749f"}]': finished 2024-09-21T05:53:35.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:34 smithi117 ceph-mon[30947]: osdmap e60: 11 total, 10 up, 11 in 2024-09-21T05:53:35.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:34 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T05:53:35.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:35 smithi138 ceph-mon[34391]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:35.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:35 smithi138 ceph-mon[34391]: from='client.? 172.21.15.138:0/2278641069' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:53:35.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:35 smithi192 ceph-mon[34700]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:35.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:35 smithi192 ceph-mon[34700]: from='client.? 172.21.15.138:0/2278641069' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:53:36.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:35 smithi117 ceph-mon[30947]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:36.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:35 smithi117 ceph-mon[30947]: from='client.? 172.21.15.138:0/2278641069' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:53:37.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:37 smithi138 ceph-mon[34391]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:37.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:37 smithi192 ceph-mon[34700]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:38.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:37 smithi117 ceph-mon[30947]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:39.854 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:39 smithi138 ceph-mon[34391]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:39.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:39 smithi192 ceph-mon[34700]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:40.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:39 smithi117 ceph-mon[30947]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:41.824 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:41 smithi138 ceph-mon[34391]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:41.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:41 smithi192 ceph-mon[34700]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:42.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:41 smithi117 ceph-mon[30947]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:42.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:42 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T05:53:42.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:42 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:53:42.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:42 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T05:53:42.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:42 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:53:43.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:42 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T05:53:43.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:42 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:53:43.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:43 smithi138 ceph-mon[34391]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:43.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:43 smithi138 ceph-mon[34391]: Deploying daemon osd.10 on smithi138 2024-09-21T05:53:43.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:43 smithi192 ceph-mon[34700]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:43.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:43 smithi192 ceph-mon[34700]: Deploying daemon osd.10 on smithi138 2024-09-21T05:53:44.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:43 smithi117 ceph-mon[30947]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:44.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:43 smithi117 ceph-mon[30947]: Deploying daemon osd.10 on smithi138 2024-09-21T05:53:45.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:45 smithi138 ceph-mon[34391]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:45.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:45 smithi192 ceph-mon[34700]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:46.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:45 smithi117 ceph-mon[30947]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:47.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:47 smithi138 ceph-mon[34391]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:47.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:47 smithi192 ceph-mon[34700]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:48.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:47 smithi117 ceph-mon[30947]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:48.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:48 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:53:48.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:48 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:48.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:48 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:48.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:48 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:53:48.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:48 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:48.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:48 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:49.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:48 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:53:49.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:48 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:49.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:48 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:49.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:49 smithi138 ceph-mon[34391]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:49.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:49 smithi192 ceph-mon[34700]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:50.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:49 smithi117 ceph-mon[30947]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:51.771 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:51 smithi138 ceph-mon[34391]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:51.812 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 10 on host 'smithi138' 2024-09-21T05:53:51.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:51.809+0000 7fb480859640 1 -- 172.21.15.138:0/3497340641 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fb47c063800 con 0x7fb4680b6560 2024-09-21T05:53:51.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:51.812+0000 7fb48185b640 1 -- 172.21.15.138:0/3497340641 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb4680b6560 msgr2=0x7fb4680b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:53:51.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:51.812+0000 7fb48185b640 1 --2- 172.21.15.138:0/3497340641 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb4680b6560 0x7fb4680b8a70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb47c108ff0 tx=0x7fb46c009110 comp rx=0 tx=0).stop 2024-09-21T05:53:51.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:51.812+0000 7fb48185b640 1 -- 172.21.15.138:0/3497340641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb47c103b90 msgr2=0x7fb47c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:53:51.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:51.812+0000 7fb48185b640 1 --2- 172.21.15.138:0/3497340641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb47c103b90 0x7fb47c075710 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb464005940 tx=0x7fb4640042f0 comp rx=0 tx=0).stop 2024-09-21T05:53:51.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:51.812+0000 7fb48185b640 1 -- 172.21.15.138:0/3497340641 shutdown_connections 2024-09-21T05:53:51.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:51.812+0000 7fb48185b640 1 --2- 172.21.15.138:0/3497340641 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb4680b6560 0x7fb4680b8a70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:53:51.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:51.812+0000 7fb48185b640 1 --2- 172.21.15.138:0/3497340641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb47c1094d0 0x7fb47c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:53:51.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:51.812+0000 7fb48185b640 1 --2- 172.21.15.138:0/3497340641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb47c109e20 0x7fb47c076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:53:51.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:51.812+0000 7fb48185b640 1 --2- 172.21.15.138:0/3497340641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb47c103b90 0x7fb47c075710 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:53:51.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:51.812+0000 7fb48185b640 1 -- 172.21.15.138:0/3497340641 >> 172.21.15.138:0/3497340641 conn(0x7fb47c0ff600 msgr2=0x7fb47c101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:53:51.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:51.812+0000 7fb48185b640 1 -- 172.21.15.138:0/3497340641 shutdown_connections 2024-09-21T05:53:51.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-21T05:53:51.812+0000 7fb48185b640 1 -- 172.21.15.138:0/3497340641 wait complete. 2024-09-21T05:53:51.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:51 smithi192 ceph-mon[34700]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:52.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:51 smithi117 ceph-mon[30947]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:52.484 DEBUG:teuthology.orchestra.run.smithi138:osd.10> sudo journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.10.service 2024-09-21T05:53:52.486 INFO:tasks.cephadm:Deploying osd.11 on smithi192 with /dev/vg_nvme/lv_1... 2024-09-21T05:53:52.486 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T05:53:52.742 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T05:53:53.056 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:52 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:53.056 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:52 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:53.056 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:52 smithi138 ceph-mon[34391]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:53.057 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:52 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:53.057 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:52 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:53.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:52 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:53.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:52 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:53.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:52 smithi192 ceph-mon[34700]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:53.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:52 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:53.092 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:52 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:53.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:52 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:53.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:52 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:53.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:52 smithi117 ceph-mon[30947]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:53.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:52 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:53.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:52 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:55.116 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 05:53:54 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T05:53:54.806+0000 7fab572ef640 -1 osd.10 0 log_to_monitors true 2024-09-21T05:53:55.519 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-21T05:53:55.546 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi192:vg_nvme/lv_1 2024-09-21T05:53:55.797 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T05:53:55.818 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:55 smithi192 ceph-mon[34700]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:55.818 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:55 smithi192 ceph-mon[34700]: from='osd.10 [v2:172.21.15.138:6824/903392451,v1:172.21.15.138:6825/903392451]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T05:53:55.818 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:55 smithi192 ceph-mon[34700]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T05:53:55.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:55 smithi138 ceph-mon[34391]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:55.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:55 smithi138 ceph-mon[34391]: from='osd.10 [v2:172.21.15.138:6824/903392451,v1:172.21.15.138:6825/903392451]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T05:53:55.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:55 smithi138 ceph-mon[34391]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T05:53:56.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:55 smithi117 ceph-mon[30947]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:56.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:55 smithi117 ceph-mon[30947]: from='osd.10 [v2:172.21.15.138:6824/903392451,v1:172.21.15.138:6825/903392451]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T05:53:56.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:55 smithi117 ceph-mon[30947]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T05:53:56.253 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.246+0000 7f6fc7670640 1 -- 172.21.15.192:0/3464261463 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6fc01049b0 msgr2=0x7f6fc0104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:53:56.253 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.246+0000 7f6fc7670640 1 --2- 172.21.15.192:0/3464261463 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6fc01049b0 0x7f6fc0104dc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4009900 tx=0x7f6fb402a410 comp rx=0 tx=0).stop 2024-09-21T05:53:56.253 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.246+0000 7f6fc7670640 1 -- 172.21.15.192:0/3464261463 shutdown_connections 2024-09-21T05:53:56.253 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.246+0000 7f6fc7670640 1 --2- 172.21.15.192:0/3464261463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fc0105300 0x7f6fc010d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:53:56.253 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.246+0000 7f6fc7670640 1 --2- 172.21.15.192:0/3464261463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fc010de10 0x7f6fc0110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:53:56.253 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.246+0000 7f6fc7670640 1 --2- 172.21.15.192:0/3464261463 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6fc01049b0 0x7f6fc0104dc0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:53:56.253 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.246+0000 7f6fc7670640 1 -- 172.21.15.192:0/3464261463 >> 172.21.15.192:0/3464261463 conn(0x7f6fc0079730 msgr2=0x7f6fc0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:53:56.254 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.246+0000 7f6fc7670640 1 -- 172.21.15.192:0/3464261463 shutdown_connections 2024-09-21T05:53:56.254 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.246+0000 7f6fc7670640 1 -- 172.21.15.192:0/3464261463 wait complete. 2024-09-21T05:53:56.254 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.247+0000 7f6fc7670640 1 Processor -- start 2024-09-21T05:53:56.254 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.247+0000 7f6fc7670640 1 -- start start 2024-09-21T05:53:56.254 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.247+0000 7f6fc7670640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6fc01049b0 0x7f6fc019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:53:56.254 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.247+0000 7f6fc7670640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fc0105300 0x7f6fc019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:53:56.254 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.247+0000 7f6fc7670640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fc010de10 0x7f6fc019fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:53:56.254 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.247+0000 7f6fc7670640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc0113a90 con 0x7f6fc010de10 2024-09-21T05:53:56.254 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.247+0000 7f6fc7670640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc0113910 con 0x7f6fc0105300 2024-09-21T05:53:56.254 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.247+0000 7f6fc7670640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc0113c10 con 0x7f6fc01049b0 2024-09-21T05:53:56.254 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.248+0000 7f6fc540c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6fc01049b0 0x7f6fc019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:53:56.254 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.248+0000 7f6fc540c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6fc01049b0 0x7f6fc019a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:33874/0 (socket says 172.21.15.192:33874) 2024-09-21T05:53:56.254 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.248+0000 7f6fc540c640 1 -- 172.21.15.192:0/1874988606 learned_addr learned my addr 172.21.15.192:0/1874988606 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-09-21T05:53:56.254 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.248+0000 7f6fc5c0d640 1 --2- 172.21.15.192:0/1874988606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fc010de10 0x7f6fc019fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:53:56.255 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.248+0000 7f6fc4c0b640 1 --2- 172.21.15.192:0/1874988606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fc0105300 0x7f6fc019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:53:56.255 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.248+0000 7f6fc540c640 1 -- 172.21.15.192:0/1874988606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fc0105300 msgr2=0x7f6fc019aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:53:56.255 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.248+0000 7f6fc540c640 1 --2- 172.21.15.192:0/1874988606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fc0105300 0x7f6fc019aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:53:56.255 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.248+0000 7f6fc540c640 1 -- 172.21.15.192:0/1874988606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fc010de10 msgr2=0x7f6fc019fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:53:56.255 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.248+0000 7f6fc540c640 1 --2- 172.21.15.192:0/1874988606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fc010de10 0x7f6fc019fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:53:56.255 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.248+0000 7f6fc540c640 1 -- 172.21.15.192:0/1874988606 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc01a0470 con 0x7f6fc01049b0 2024-09-21T05:53:56.255 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.248+0000 7f6fc5c0d640 1 --2- 172.21.15.192:0/1874988606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fc010de10 0x7f6fc019fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:53:56.255 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.248+0000 7f6fc540c640 1 --2- 172.21.15.192:0/1874988606 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6fc01049b0 0x7f6fc019a500 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4006010 tx=0x7f6fb4004060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:53:56.255 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.248+0000 7f6fb27fc640 1 -- 172.21.15.192:0/1874988606 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb403f030 con 0x7f6fc01049b0 2024-09-21T05:53:56.255 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.248+0000 7f6fb27fc640 1 -- 172.21.15.192:0/1874988606 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fb4003aa0 con 0x7f6fc01049b0 2024-09-21T05:53:56.255 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.248+0000 7f6fc7670640 1 -- 172.21.15.192:0/1874988606 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fc01a0700 con 0x7f6fc01049b0 2024-09-21T05:53:56.255 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.248+0000 7f6fb27fc640 1 -- 172.21.15.192:0/1874988606 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4035070 con 0x7f6fc01049b0 2024-09-21T05:53:56.255 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.248+0000 7f6fc7670640 1 -- 172.21.15.192:0/1874988606 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fc01a6ec0 con 0x7f6fc01049b0 2024-09-21T05:53:56.255 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.249+0000 7f6fc7670640 1 -- 172.21.15.192:0/1874988606 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fc006b560 con 0x7f6fc01049b0 2024-09-21T05:53:56.255 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.251+0000 7f6fb27fc640 1 -- 172.21.15.192:0/1874988606 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f6fb4042030 con 0x7f6fc01049b0 2024-09-21T05:53:56.256 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.252+0000 7f6fb27fc640 1 --2- 172.21.15.192:0/1874988606 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6fac0b65e0 0x7f6fac0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:53:56.256 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.252+0000 7f6fb27fc640 1 -- 172.21.15.192:0/1874988606 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6613+0+0 (secure 0 0 0) 0x7f6fb41075a0 con 0x7f6fc01049b0 2024-09-21T05:53:56.256 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.252+0000 7f6fc4c0b640 1 --2- 172.21.15.192:0/1874988606 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6fac0b65e0 0x7f6fac0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:53:56.256 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.252+0000 7f6fc4c0b640 1 --2- 172.21.15.192:0/1874988606 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6fac0b65e0 0x7f6fac0b8af0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6fc007a0a0 tx=0x7f6fbc005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:53:56.257 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.255+0000 7f6fb27fc640 1 -- 172.21.15.192:0/1874988606 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6fb40ce4d0 con 0x7f6fc01049b0 2024-09-21T05:53:56.452 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:53:56.450+0000 7f6fc7670640 1 -- 172.21.15.192:0/1874988606 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f6fc0063800 con 0x7f6fac0b65e0 2024-09-21T05:53:56.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:56 smithi192 ceph-mon[34700]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T05:53:56.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:56 smithi192 ceph-mon[34700]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T05:53:56.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:56 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T05:53:56.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:56 smithi192 ceph-mon[34700]: from='osd.10 [v2:172.21.15.138:6824/903392451,v1:172.21.15.138:6825/903392451]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-21T05:53:56.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:56 smithi192 ceph-mon[34700]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-21T05:53:56.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:56 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:56.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:56 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:56.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:56 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T05:53:56.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:56 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T05:53:56.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:56 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T05:53:56.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:56 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T05:53:56.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:56 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:56.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:56 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:53:56.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:56 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:53:56.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:56 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:56.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:56 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:53:56.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:56 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:53:56.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:56 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:53:56.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:56 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:56.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:56 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:53:56.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:56 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:53:56.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:56 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:53:56.822 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:56 smithi138 ceph-mon[34391]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T05:53:56.822 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:56 smithi138 ceph-mon[34391]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T05:53:56.822 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:56 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T05:53:56.822 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:56 smithi138 ceph-mon[34391]: from='osd.10 [v2:172.21.15.138:6824/903392451,v1:172.21.15.138:6825/903392451]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-21T05:53:56.822 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:56 smithi138 ceph-mon[34391]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-21T05:53:56.822 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:56 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:56.822 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:56 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:56.822 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:56 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T05:53:56.822 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:56 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T05:53:56.823 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:56 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T05:53:56.823 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:56 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T05:53:56.823 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:56 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:56.823 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:56 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:53:56.823 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:56 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:53:56.823 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:56 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:56.823 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:56 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:53:56.823 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:56 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:53:56.823 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:56 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:53:56.823 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:56 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:56.823 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:56 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:53:56.823 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:56 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:53:56.823 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:56 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:53:57.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:56 smithi117 ceph-mon[30947]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T05:53:57.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:56 smithi117 ceph-mon[30947]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T05:53:57.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:56 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T05:53:57.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:56 smithi117 ceph-mon[30947]: from='osd.10 [v2:172.21.15.138:6824/903392451,v1:172.21.15.138:6825/903392451]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-21T05:53:57.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:56 smithi117 ceph-mon[30947]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-09-21T05:53:57.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:56 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:57.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:56 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:57.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:56 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T05:53:57.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:56 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T05:53:57.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:56 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T05:53:57.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:56 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T05:53:57.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:56 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:57.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:56 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:53:57.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:56 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:53:57.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:56 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:57.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:56 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:53:57.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:56 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:53:57.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:56 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:53:57.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:56 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:53:57.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:56 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T05:53:57.029 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:56 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T05:53:57.029 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:56 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:53:57.115 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 05:53:56 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T05:53:56.990+0000 7fab53a86640 -1 osd.10 0 waiting for initial osdmap 2024-09-21T05:53:57.115 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 05:53:57 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T05:53:57.006+0000 7fab4b067640 -1 osd.10 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T05:53:57.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:57 smithi138 ceph-mon[34391]: Detected new or changed devices on smithi138 2024-09-21T05:53:57.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:57 smithi138 ceph-mon[34391]: Adjusting osd_memory_target on smithi138 to 4272M 2024-09-21T05:53:57.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:57 smithi138 ceph-mon[34391]: from='client.24523 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:53:57.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:57 smithi138 ceph-mon[34391]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:57.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:57 smithi138 ceph-mon[34391]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-21T05:53:57.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:57 smithi138 ceph-mon[34391]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T05:53:57.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:57 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T05:53:57.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:57 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T05:53:57.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:57 smithi138 ceph-mon[34391]: from='osd.10 ' entity='osd.10' 2024-09-21T05:53:57.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:57 smithi138 ceph-mon[34391]: osd.10 [v2:172.21.15.138:6824/903392451,v1:172.21.15.138:6825/903392451] boot 2024-09-21T05:53:57.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:57 smithi138 ceph-mon[34391]: osdmap e63: 11 total, 11 up, 11 in 2024-09-21T05:53:57.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:57 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T05:53:57.877 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:57 smithi192 ceph-mon[34700]: Detected new or changed devices on smithi138 2024-09-21T05:53:57.878 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:57 smithi192 ceph-mon[34700]: Adjusting osd_memory_target on smithi138 to 4272M 2024-09-21T05:53:57.878 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:57 smithi192 ceph-mon[34700]: from='client.24523 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:53:57.878 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:57 smithi192 ceph-mon[34700]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:57.878 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:57 smithi192 ceph-mon[34700]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-21T05:53:57.878 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:57 smithi192 ceph-mon[34700]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T05:53:57.878 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:57 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T05:53:57.878 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:57 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T05:53:57.878 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:57 smithi192 ceph-mon[34700]: from='osd.10 ' entity='osd.10' 2024-09-21T05:53:57.878 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:57 smithi192 ceph-mon[34700]: osd.10 [v2:172.21.15.138:6824/903392451,v1:172.21.15.138:6825/903392451] boot 2024-09-21T05:53:57.878 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:57 smithi192 ceph-mon[34700]: osdmap e63: 11 total, 11 up, 11 in 2024-09-21T05:53:57.878 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:57 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T05:53:58.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:57 smithi117 ceph-mon[30947]: Detected new or changed devices on smithi138 2024-09-21T05:53:58.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:57 smithi117 ceph-mon[30947]: Adjusting osd_memory_target on smithi138 to 4272M 2024-09-21T05:53:58.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:57 smithi117 ceph-mon[30947]: from='client.24523 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:53:58.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:57 smithi117 ceph-mon[30947]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-09-21T05:53:58.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:57 smithi117 ceph-mon[30947]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-09-21T05:53:58.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:57 smithi117 ceph-mon[30947]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T05:53:58.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:57 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T05:53:58.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:57 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T05:53:58.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:57 smithi117 ceph-mon[30947]: from='osd.10 ' entity='osd.10' 2024-09-21T05:53:58.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:57 smithi117 ceph-mon[30947]: osd.10 [v2:172.21.15.138:6824/903392451,v1:172.21.15.138:6825/903392451] boot 2024-09-21T05:53:58.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:57 smithi117 ceph-mon[30947]: osdmap e63: 11 total, 11 up, 11 in 2024-09-21T05:53:58.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:57 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T05:53:58.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:58 smithi138 ceph-mon[34391]: purged_snaps scrub starts 2024-09-21T05:53:58.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:58 smithi138 ceph-mon[34391]: purged_snaps scrub ok 2024-09-21T05:53:58.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:58 smithi138 ceph-mon[34391]: osdmap e64: 11 total, 11 up, 11 in 2024-09-21T05:53:58.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:58 smithi192 ceph-mon[34700]: purged_snaps scrub starts 2024-09-21T05:53:58.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:58 smithi192 ceph-mon[34700]: purged_snaps scrub ok 2024-09-21T05:53:58.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:58 smithi192 ceph-mon[34700]: osdmap e64: 11 total, 11 up, 11 in 2024-09-21T05:53:59.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:58 smithi117 ceph-mon[30947]: purged_snaps scrub starts 2024-09-21T05:53:59.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:58 smithi117 ceph-mon[30947]: purged_snaps scrub ok 2024-09-21T05:53:59.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:58 smithi117 ceph-mon[30947]: osdmap e64: 11 total, 11 up, 11 in 2024-09-21T05:53:59.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:59 smithi138 ceph-mon[34391]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-21T05:53:59.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:59 smithi138 ceph-mon[34391]: from='client.? 172.21.15.192:0/2306649540' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "24bc2ef7-e2b4-43c6-9e6c-43ae02390415"} : dispatch 2024-09-21T05:53:59.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:59 smithi138 ceph-mon[34391]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "24bc2ef7-e2b4-43c6-9e6c-43ae02390415"} : dispatch 2024-09-21T05:53:59.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:59 smithi138 ceph-mon[34391]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "24bc2ef7-e2b4-43c6-9e6c-43ae02390415"}]': finished 2024-09-21T05:53:59.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:59 smithi138 ceph-mon[34391]: osdmap e65: 12 total, 11 up, 12 in 2024-09-21T05:53:59.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:59 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T05:53:59.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:53:59 smithi138 ceph-mon[34391]: from='client.? 172.21.15.192:0/2075411856' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:53:59.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:59 smithi192 ceph-mon[34700]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-21T05:53:59.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:59 smithi192 ceph-mon[34700]: from='client.? 172.21.15.192:0/2306649540' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "24bc2ef7-e2b4-43c6-9e6c-43ae02390415"} : dispatch 2024-09-21T05:53:59.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:59 smithi192 ceph-mon[34700]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "24bc2ef7-e2b4-43c6-9e6c-43ae02390415"} : dispatch 2024-09-21T05:53:59.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:59 smithi192 ceph-mon[34700]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "24bc2ef7-e2b4-43c6-9e6c-43ae02390415"}]': finished 2024-09-21T05:53:59.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:59 smithi192 ceph-mon[34700]: osdmap e65: 12 total, 11 up, 12 in 2024-09-21T05:53:59.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:59 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T05:53:59.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:53:59 smithi192 ceph-mon[34700]: from='client.? 172.21.15.192:0/2075411856' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:54:00.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:59 smithi117 ceph-mon[30947]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-21T05:54:00.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:59 smithi117 ceph-mon[30947]: from='client.? 172.21.15.192:0/2306649540' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "24bc2ef7-e2b4-43c6-9e6c-43ae02390415"} : dispatch 2024-09-21T05:54:00.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:59 smithi117 ceph-mon[30947]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "24bc2ef7-e2b4-43c6-9e6c-43ae02390415"} : dispatch 2024-09-21T05:54:00.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:59 smithi117 ceph-mon[30947]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "24bc2ef7-e2b4-43c6-9e6c-43ae02390415"}]': finished 2024-09-21T05:54:00.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:59 smithi117 ceph-mon[30947]: osdmap e65: 12 total, 11 up, 12 in 2024-09-21T05:54:00.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:59 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T05:54:00.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:53:59 smithi117 ceph-mon[30947]: from='client.? 172.21.15.192:0/2075411856' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T05:54:01.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:01 smithi138 ceph-mon[34391]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-21T05:54:01.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:01 smithi192 ceph-mon[34700]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-21T05:54:02.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:01 smithi117 ceph-mon[30947]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail 2024-09-21T05:54:03.827 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:03 smithi192 ceph-mon[34700]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-21T05:54:03.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:03 smithi138 ceph-mon[34391]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-21T05:54:04.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:03 smithi117 ceph-mon[30947]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-21T05:54:05.853 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:05 smithi192 ceph-mon[34700]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-21T05:54:05.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:05 smithi138 ceph-mon[34391]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-21T05:54:06.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:05 smithi117 ceph-mon[30947]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-21T05:54:07.729 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:07 smithi192 ceph-mon[34700]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-21T05:54:07.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:07 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T05:54:07.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:07 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:07.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:07 smithi138 ceph-mon[34391]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-21T05:54:07.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:07 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T05:54:07.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:07 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:08.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:07 smithi117 ceph-mon[30947]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-21T05:54:08.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:07 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T05:54:08.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:07 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:08.802 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:08 smithi192 ceph-mon[34700]: Deploying daemon osd.11 on smithi192 2024-09-21T05:54:08.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:08 smithi138 ceph-mon[34391]: Deploying daemon osd.11 on smithi192 2024-09-21T05:54:09.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:08 smithi117 ceph-mon[30947]: Deploying daemon osd.11 on smithi192 2024-09-21T05:54:09.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:09 smithi192 ceph-mon[34700]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T05:54:09.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:09 smithi138 ceph-mon[34391]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T05:54:10.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:09 smithi117 ceph-mon[30947]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T05:54:11.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:11 smithi138 ceph-mon[34391]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-21T05:54:11.875 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:11 smithi192 ceph-mon[34700]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-21T05:54:12.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:11 smithi117 ceph-mon[30947]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-21T05:54:13.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:13 smithi138 ceph-mon[34391]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T05:54:13.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:13 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:54:13.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:13 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:13.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:13 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:13.918 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:13 smithi192 ceph-mon[34700]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T05:54:13.918 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:13 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:54:13.918 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:13 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:13.918 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:13 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:14.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:13 smithi117 ceph-mon[30947]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T05:54:14.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:13 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:54:14.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:13 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:14.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:13 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:15.856 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:15 smithi117 ceph-mon[30947]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T05:54:15.865 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:15 smithi192 ceph-mon[34700]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T05:54:15.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:15 smithi138 ceph-mon[34391]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T05:54:16.927 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:16.925+0000 7f6fb27fc640 1 -- 172.21.15.192:0/1874988606 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f6fc0063800 con 0x7f6fac0b65e0 2024-09-21T05:54:16.928 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 11 on host 'smithi192' 2024-09-21T05:54:16.930 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:16.928+0000 7f6fc7670640 1 -- 172.21.15.192:0/1874988606 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6fac0b65e0 msgr2=0x7f6fac0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:16.931 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:16.928+0000 7f6fc7670640 1 --2- 172.21.15.192:0/1874988606 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6fac0b65e0 0x7f6fac0b8af0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6fc007a0a0 tx=0x7f6fbc005f90 comp rx=0 tx=0).stop 2024-09-21T05:54:16.931 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:16.928+0000 7f6fc7670640 1 -- 172.21.15.192:0/1874988606 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6fc01049b0 msgr2=0x7f6fc019a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:16.931 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:16.928+0000 7f6fc7670640 1 --2- 172.21.15.192:0/1874988606 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6fc01049b0 0x7f6fc019a500 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4006010 tx=0x7f6fb4004060 comp rx=0 tx=0).stop 2024-09-21T05:54:16.931 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:16.928+0000 7f6fc7670640 1 -- 172.21.15.192:0/1874988606 shutdown_connections 2024-09-21T05:54:16.931 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:16.928+0000 7f6fc7670640 1 --2- 172.21.15.192:0/1874988606 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6fac0b65e0 0x7f6fac0b8af0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:16.931 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:16.928+0000 7f6fc7670640 1 --2- 172.21.15.192:0/1874988606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6fc010de10 0x7f6fc019fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:16.931 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:16.928+0000 7f6fc7670640 1 --2- 172.21.15.192:0/1874988606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fc0105300 0x7f6fc019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:16.931 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:16.928+0000 7f6fc7670640 1 --2- 172.21.15.192:0/1874988606 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6fc01049b0 0x7f6fc019a500 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:16.931 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:16.928+0000 7f6fc7670640 1 -- 172.21.15.192:0/1874988606 >> 172.21.15.192:0/1874988606 conn(0x7f6fc0079730 msgr2=0x7f6fc00ff730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:54:16.931 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:16.928+0000 7f6fc7670640 1 -- 172.21.15.192:0/1874988606 shutdown_connections 2024-09-21T05:54:16.931 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:16.928+0000 7f6fc7670640 1 -- 172.21.15.192:0/1874988606 wait complete. 2024-09-21T05:54:17.503 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:17 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:17.504 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:17 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:17.504 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:17 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:17.504 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:17 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:54:17.504 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:17 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:17.504 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:17 smithi192 ceph-mon[34700]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T05:54:17.504 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:17 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:54:17.504 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:17 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:17.504 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:17 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:17.526 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:17 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:17.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:17 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:17.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:17 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:17.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:17 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:54:17.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:17 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:17.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:17 smithi117 ceph-mon[30947]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T05:54:17.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:17 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:54:17.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:17 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:17.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:17 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:17.575 DEBUG:teuthology.orchestra.run.smithi192:osd.11> sudo journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.11.service 2024-09-21T05:54:17.577 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-21T05:54:17.578 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T05:54:17.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:17 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:17.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:17 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:17.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:17 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:17.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:17 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:54:17.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:17 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:17.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:17 smithi138 ceph-mon[34391]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T05:54:17.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:17 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:54:17.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:17 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:17.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:17 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:17.823 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:54:18.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.318+0000 7fbe926ea640 1 -- 172.21.15.117:0/3891199382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe8c108cf0 msgr2=0x7fbe8c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:18.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.318+0000 7fbe926ea640 1 --2- 172.21.15.117:0/3891199382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe8c108cf0 0x7fbe8c10f520 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009ca0 tx=0x7fbe8402a8c0 comp rx=0 tx=0).stop 2024-09-21T05:54:18.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.318+0000 7fbe926ea640 1 -- 172.21.15.117:0/3891199382 shutdown_connections 2024-09-21T05:54:18.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.318+0000 7fbe926ea640 1 --2- 172.21.15.117:0/3891199382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe8c108cf0 0x7fbe8c10f520 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:18.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.318+0000 7fbe926ea640 1 --2- 172.21.15.117:0/3891199382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe8c1083a0 0x7fbe8c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:18.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.318+0000 7fbe926ea640 1 --2- 172.21.15.117:0/3891199382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe8c102a60 0x7fbe8c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:18.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.318+0000 7fbe926ea640 1 -- 172.21.15.117:0/3891199382 >> 172.21.15.117:0/3891199382 conn(0x7fbe8c079730 msgr2=0x7fbe8c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:54:18.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.318+0000 7fbe926ea640 1 -- 172.21.15.117:0/3891199382 shutdown_connections 2024-09-21T05:54:18.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.318+0000 7fbe926ea640 1 -- 172.21.15.117:0/3891199382 wait complete. 2024-09-21T05:54:18.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.319+0000 7fbe926ea640 1 Processor -- start 2024-09-21T05:54:18.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.319+0000 7fbe926ea640 1 -- start start 2024-09-21T05:54:18.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.319+0000 7fbe926ea640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe8c102a60 0x7fbe8c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:18.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.319+0000 7fbe926ea640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe8c1083a0 0x7fbe8c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:18.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.319+0000 7fbe926ea640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe8c108cf0 0x7fbe8c19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:18.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.319+0000 7fbe926ea640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c112960 con 0x7fbe8c102a60 2024-09-21T05:54:18.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.319+0000 7fbe926ea640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c1127e0 con 0x7fbe8c1083a0 2024-09-21T05:54:18.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.319+0000 7fbe926ea640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c112ae0 con 0x7fbe8c108cf0 2024-09-21T05:54:18.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.319+0000 7fbe8bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe8c102a60 0x7fbe8c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:18.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.319+0000 7fbe8bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe8c102a60 0x7fbe8c19a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42048/0 (socket says 172.21.15.117:42048) 2024-09-21T05:54:18.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.319+0000 7fbe8bfff640 1 -- 172.21.15.117:0/48844126 learned_addr learned my addr 172.21.15.117:0/48844126 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:54:18.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.319+0000 7fbe90c87640 1 --2- 172.21.15.117:0/48844126 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe8c108cf0 0x7fbe8c19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:18.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.320+0000 7fbe83fff640 1 --2- 172.21.15.117:0/48844126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe8c1083a0 0x7fbe8c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:18.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.320+0000 7fbe8bfff640 1 -- 172.21.15.117:0/48844126 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe8c108cf0 msgr2=0x7fbe8c19fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:18.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.320+0000 7fbe8bfff640 1 --2- 172.21.15.117:0/48844126 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe8c108cf0 0x7fbe8c19fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:18.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.320+0000 7fbe8bfff640 1 -- 172.21.15.117:0/48844126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe8c1083a0 msgr2=0x7fbe8c19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:18.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.320+0000 7fbe8bfff640 1 --2- 172.21.15.117:0/48844126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe8c1083a0 0x7fbe8c19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:18.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.320+0000 7fbe8bfff640 1 -- 172.21.15.117:0/48844126 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe8c1a03d0 con 0x7fbe8c102a60 2024-09-21T05:54:18.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.320+0000 7fbe8bfff640 1 --2- 172.21.15.117:0/48844126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe8c102a60 0x7fbe8c19a460 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00d770 tx=0x7fbe7c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:54:18.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.320+0000 7fbe89ffb640 1 -- 172.21.15.117:0/48844126 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c017070 con 0x7fbe8c102a60 2024-09-21T05:54:18.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.320+0000 7fbe89ffb640 1 -- 172.21.15.117:0/48844126 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe7c0054d0 con 0x7fbe8c102a60 2024-09-21T05:54:18.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.320+0000 7fbe926ea640 1 -- 172.21.15.117:0/48844126 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe8c1a06c0 con 0x7fbe8c102a60 2024-09-21T05:54:18.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.320+0000 7fbe89ffb640 1 -- 172.21.15.117:0/48844126 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c004c30 con 0x7fbe8c102a60 2024-09-21T05:54:18.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.320+0000 7fbe926ea640 1 -- 172.21.15.117:0/48844126 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe8c1a6e20 con 0x7fbe8c102a60 2024-09-21T05:54:18.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.321+0000 7fbe926ea640 1 -- 172.21.15.117:0/48844126 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe8c102e70 con 0x7fbe8c102a60 2024-09-21T05:54:18.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.323+0000 7fbe89ffb640 1 -- 172.21.15.117:0/48844126 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7fbe7c004dd0 con 0x7fbe8c102a60 2024-09-21T05:54:18.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.323+0000 7fbe89ffb640 1 --2- 172.21.15.117:0/48844126 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbe780b65e0 0x7fbe780b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:18.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.323+0000 7fbe83fff640 1 --2- 172.21.15.117:0/48844126 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbe780b65e0 0x7fbe780b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:18.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.323+0000 7fbe89ffb640 1 -- 172.21.15.117:0/48844126 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 7024+0+0 (secure 0 0 0) 0x7fbe7c0e6bb0 con 0x7fbe8c102a60 2024-09-21T05:54:18.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.324+0000 7fbe83fff640 1 --2- 172.21.15.117:0/48844126 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbe780b65e0 0x7fbe780b8af0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c06a2a0 tx=0x7fbe7400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:54:18.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.326+0000 7fbe89ffb640 1 -- 172.21.15.117:0/48844126 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbe7c0b1db0 con 0x7fbe8c102a60 2024-09-21T05:54:18.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.505+0000 7fbe926ea640 1 -- 172.21.15.117:0/48844126 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fbe8c19b590 con 0x7fbe8c102a60 2024-09-21T05:54:18.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.505+0000 7fbe89ffb640 1 -- 172.21.15.117:0/48844126 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v65) ==== 74+0+133 (secure 0 0 0) 0x7fbe8c19b590 con 0x7fbe8c102a60 2024-09-21T05:54:18.509 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:54:18.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.507+0000 7fbe926ea640 1 -- 172.21.15.117:0/48844126 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbe780b65e0 msgr2=0x7fbe780b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:18.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.507+0000 7fbe926ea640 1 --2- 172.21.15.117:0/48844126 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbe780b65e0 0x7fbe780b8af0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c06a2a0 tx=0x7fbe7400b390 comp rx=0 tx=0).stop 2024-09-21T05:54:18.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.507+0000 7fbe926ea640 1 -- 172.21.15.117:0/48844126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe8c102a60 msgr2=0x7fbe8c19a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:18.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.507+0000 7fbe926ea640 1 --2- 172.21.15.117:0/48844126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe8c102a60 0x7fbe8c19a460 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00d770 tx=0x7fbe7c00dc30 comp rx=0 tx=0).stop 2024-09-21T05:54:18.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.508+0000 7fbe926ea640 1 -- 172.21.15.117:0/48844126 shutdown_connections 2024-09-21T05:54:18.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.508+0000 7fbe926ea640 1 --2- 172.21.15.117:0/48844126 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbe780b65e0 0x7fbe780b8af0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:18.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.508+0000 7fbe926ea640 1 --2- 172.21.15.117:0/48844126 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe8c108cf0 0x7fbe8c19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:18.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.508+0000 7fbe926ea640 1 --2- 172.21.15.117:0/48844126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe8c1083a0 0x7fbe8c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:18.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.508+0000 7fbe926ea640 1 --2- 172.21.15.117:0/48844126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe8c102a60 0x7fbe8c19a460 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:18.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.508+0000 7fbe926ea640 1 -- 172.21.15.117:0/48844126 >> 172.21.15.117:0/48844126 conn(0x7fbe8c079730 msgr2=0x7fbe8c06b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:54:18.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.508+0000 7fbe926ea640 1 -- 172.21.15.117:0/48844126 shutdown_connections 2024-09-21T05:54:18.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:18.508+0000 7fbe926ea640 1 -- 172.21.15.117:0/48844126 wait complete. 2024-09-21T05:54:18.740 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:18 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/48844126' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T05:54:18.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:18 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/48844126' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T05:54:18.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:18 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/48844126' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T05:54:19.315 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":65,"num_osds":12,"num_up_osds":11,"osd_up_since":1726898037,"num_in_osds":12,"osd_in_since":1726898038,"num_remapped_pgs":0} 2024-09-21T05:54:19.751 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:19 smithi192 ceph-mon[34700]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T05:54:19.751 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:19 smithi192 ceph-mon[34700]: from='osd.11 [v2:172.21.15.192:6824/899090479,v1:172.21.15.192:6825/899090479]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T05:54:19.751 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:19 smithi192 ceph-mon[34700]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T05:54:19.751 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 05:54:19 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T05:54:19.478+0000 7f5548b7d640 -1 osd.11 0 log_to_monitors true 2024-09-21T05:54:19.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:19 smithi138 ceph-mon[34391]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T05:54:19.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:19 smithi138 ceph-mon[34391]: from='osd.11 [v2:172.21.15.192:6824/899090479,v1:172.21.15.192:6825/899090479]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T05:54:19.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:19 smithi138 ceph-mon[34391]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T05:54:20.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:19 smithi117 ceph-mon[30947]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T05:54:20.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:19 smithi117 ceph-mon[30947]: from='osd.11 [v2:172.21.15.192:6824/899090479,v1:172.21.15.192:6825/899090479]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T05:54:20.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:19 smithi117 ceph-mon[30947]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T05:54:20.316 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T05:54:20.586 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:54:20.707 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:20 smithi117 ceph-mon[30947]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T05:54:20.707 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:20 smithi117 ceph-mon[30947]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T05:54:20.707 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:20 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T05:54:20.707 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:20 smithi117 ceph-mon[30947]: from='osd.11 [v2:172.21.15.192:6824/899090479,v1:172.21.15.192:6825/899090479]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]} : dispatch 2024-09-21T05:54:20.707 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:20 smithi117 ceph-mon[30947]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]} : dispatch 2024-09-21T05:54:20.707 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:20 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:20.707 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:20 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:20.707 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:20 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T05:54:20.708 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:20 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T05:54:20.708 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:20 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T05:54:20.708 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:20 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T05:54:20.708 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:20 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:20.708 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:20 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:20.708 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:20 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:54:20.708 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:20 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:20.729 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 05:54:20 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T05:54:20.576+0000 7f5544aff640 -1 osd.11 0 waiting for initial osdmap 2024-09-21T05:54:20.730 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 05:54:20 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T05:54:20.587+0000 7f55408fd640 -1 osd.11 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T05:54:20.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:20 smithi192 ceph-mon[34700]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T05:54:20.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:20 smithi192 ceph-mon[34700]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T05:54:20.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:20 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T05:54:20.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:20 smithi192 ceph-mon[34700]: from='osd.11 [v2:172.21.15.192:6824/899090479,v1:172.21.15.192:6825/899090479]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]} : dispatch 2024-09-21T05:54:20.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:20 smithi192 ceph-mon[34700]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]} : dispatch 2024-09-21T05:54:20.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:20 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:20.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:20 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:20.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:20 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T05:54:20.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:20 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T05:54:20.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:20 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T05:54:20.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:20 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T05:54:20.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:20 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:20.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:20 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:20.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:20 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:54:20.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:20 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:20.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:20 smithi138 ceph-mon[34391]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T05:54:20.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:20 smithi138 ceph-mon[34391]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T05:54:20.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:20 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T05:54:20.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:20 smithi138 ceph-mon[34391]: from='osd.11 [v2:172.21.15.192:6824/899090479,v1:172.21.15.192:6825/899090479]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]} : dispatch 2024-09-21T05:54:20.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:20 smithi138 ceph-mon[34391]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]} : dispatch 2024-09-21T05:54:20.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:20 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:20.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:20 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:20.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:20 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T05:54:20.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:20 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T05:54:20.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:20 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T05:54:20.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:20 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T05:54:20.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:20 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:20.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:20 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:20.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:20 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:54:20.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:20 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:21.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.090+0000 7f5fe74e1640 1 -- 172.21.15.117:0/4017269387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fe0102a60 msgr2=0x7f5fe0102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:21.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.090+0000 7f5fe74e1640 1 --2- 172.21.15.117:0/4017269387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fe0102a60 0x7f5fe0102e70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc009900 tx=0x7f5fdc02a410 comp rx=0 tx=0).stop 2024-09-21T05:54:21.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.090+0000 7f5fe74e1640 1 -- 172.21.15.117:0/4017269387 shutdown_connections 2024-09-21T05:54:21.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.090+0000 7f5fe74e1640 1 --2- 172.21.15.117:0/4017269387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fe0108cf0 0x7f5fe010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:21.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.090+0000 7f5fe74e1640 1 --2- 172.21.15.117:0/4017269387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fe01083a0 0x7f5fe01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:21.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.090+0000 7f5fe74e1640 1 --2- 172.21.15.117:0/4017269387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fe0102a60 0x7f5fe0102e70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:21.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.090+0000 7f5fe74e1640 1 -- 172.21.15.117:0/4017269387 >> 172.21.15.117:0/4017269387 conn(0x7f5fe0079730 msgr2=0x7f5fe0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:54:21.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.090+0000 7f5fe74e1640 1 -- 172.21.15.117:0/4017269387 shutdown_connections 2024-09-21T05:54:21.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.090+0000 7f5fe74e1640 1 -- 172.21.15.117:0/4017269387 wait complete. 2024-09-21T05:54:21.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.091+0000 7f5fe74e1640 1 Processor -- start 2024-09-21T05:54:21.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.091+0000 7f5fe74e1640 1 -- start start 2024-09-21T05:54:21.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.091+0000 7f5fe74e1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fe0102a60 0x7f5fe019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:21.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.091+0000 7f5fe74e1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fe01083a0 0x7f5fe019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:21.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.091+0000 7f5fe74e1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fe0108cf0 0x7f5fe019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:21.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.091+0000 7f5fe74e1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe0113a80 con 0x7f5fe01083a0 2024-09-21T05:54:21.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.091+0000 7f5fe74e1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe0113900 con 0x7f5fe0108cf0 2024-09-21T05:54:21.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.091+0000 7f5fe74e1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe0113c00 con 0x7f5fe0102a60 2024-09-21T05:54:21.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.091+0000 7f5fe4a7c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fe01083a0 0x7f5fe019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:21.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.091+0000 7f5fe4a7c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fe01083a0 0x7f5fe019aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42056/0 (socket says 172.21.15.117:42056) 2024-09-21T05:54:21.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.091+0000 7f5fe4a7c640 1 -- 172.21.15.117:0/3992734022 learned_addr learned my addr 172.21.15.117:0/3992734022 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:54:21.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.091+0000 7f5fe527d640 1 --2- 172.21.15.117:0/3992734022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fe0102a60 0x7f5fe019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:21.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.092+0000 7f5fe5a7e640 1 --2- 172.21.15.117:0/3992734022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fe0108cf0 0x7f5fe019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:21.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.092+0000 7f5fe4a7c640 1 -- 172.21.15.117:0/3992734022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fe0102a60 msgr2=0x7f5fe019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:21.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.092+0000 7f5fe4a7c640 1 --2- 172.21.15.117:0/3992734022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fe0102a60 0x7f5fe019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:21.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.092+0000 7f5fe4a7c640 1 -- 172.21.15.117:0/3992734022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fe0108cf0 msgr2=0x7f5fe019fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:21.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.092+0000 7f5fe4a7c640 1 --2- 172.21.15.117:0/3992734022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fe0108cf0 0x7f5fe019fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:21.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.092+0000 7f5fe4a7c640 1 -- 172.21.15.117:0/3992734022 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe01a0400 con 0x7f5fe01083a0 2024-09-21T05:54:21.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.092+0000 7f5fe4a7c640 1 --2- 172.21.15.117:0/3992734022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fe01083a0 0x7f5fe019aa30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400c970 tx=0x7f5fd400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:54:21.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.092+0000 7f5fd27fc640 1 -- 172.21.15.117:0/3992734022 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4007bb0 con 0x7f5fe01083a0 2024-09-21T05:54:21.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.092+0000 7f5fd27fc640 1 -- 172.21.15.117:0/3992734022 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fd4007d50 con 0x7f5fe01083a0 2024-09-21T05:54:21.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.092+0000 7f5fe74e1640 1 -- 172.21.15.117:0/3992734022 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe01a06f0 con 0x7f5fe01083a0 2024-09-21T05:54:21.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.092+0000 7f5fd27fc640 1 -- 172.21.15.117:0/3992734022 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd400f450 con 0x7f5fe01083a0 2024-09-21T05:54:21.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.092+0000 7f5fe74e1640 1 -- 172.21.15.117:0/3992734022 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe01a6e50 con 0x7f5fe01083a0 2024-09-21T05:54:21.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.093+0000 7f5fe74e1640 1 -- 172.21.15.117:0/3992734022 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fe0102e70 con 0x7f5fe01083a0 2024-09-21T05:54:21.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.095+0000 7f5fd27fc640 1 -- 172.21.15.117:0/3992734022 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f5fd4004f20 con 0x7f5fe01083a0 2024-09-21T05:54:21.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.095+0000 7f5fd27fc640 1 --2- 172.21.15.117:0/3992734022 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5fcc0b6560 0x7f5fcc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:21.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.095+0000 7f5fe527d640 1 --2- 172.21.15.117:0/3992734022 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5fcc0b6560 0x7f5fcc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:21.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.095+0000 7f5fd27fc640 1 -- 172.21.15.117:0/3992734022 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 7062+0+0 (secure 0 0 0) 0x7f5fd40e6f60 con 0x7f5fe01083a0 2024-09-21T05:54:21.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.096+0000 7f5fe527d640 1 --2- 172.21.15.117:0/3992734022 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5fcc0b6560 0x7f5fcc0b8a70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc031440 tx=0x7f5fdc005d50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:54:21.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.099+0000 7f5fd27fc640 1 -- 172.21.15.117:0/3992734022 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5fd4014030 con 0x7f5fe01083a0 2024-09-21T05:54:21.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.276+0000 7f5fe74e1640 1 -- 172.21.15.117:0/3992734022 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f5fe019b5c0 con 0x7f5fe01083a0 2024-09-21T05:54:21.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.277+0000 7f5fd27fc640 1 -- 172.21.15.117:0/3992734022 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v67) ==== 74+0+133 (secure 0 0 0) 0x7f5fe019b5c0 con 0x7f5fe01083a0 2024-09-21T05:54:21.280 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:54:21.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.279+0000 7f5fe74e1640 1 -- 172.21.15.117:0/3992734022 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5fcc0b6560 msgr2=0x7f5fcc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:21.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.279+0000 7f5fe74e1640 1 --2- 172.21.15.117:0/3992734022 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5fcc0b6560 0x7f5fcc0b8a70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc031440 tx=0x7f5fdc005d50 comp rx=0 tx=0).stop 2024-09-21T05:54:21.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.279+0000 7f5fe74e1640 1 -- 172.21.15.117:0/3992734022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fe01083a0 msgr2=0x7f5fe019aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:21.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.279+0000 7f5fe74e1640 1 --2- 172.21.15.117:0/3992734022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fe01083a0 0x7f5fe019aa30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400c970 tx=0x7f5fd400ce30 comp rx=0 tx=0).stop 2024-09-21T05:54:21.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.279+0000 7f5fe74e1640 1 -- 172.21.15.117:0/3992734022 shutdown_connections 2024-09-21T05:54:21.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.279+0000 7f5fe74e1640 1 --2- 172.21.15.117:0/3992734022 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5fcc0b6560 0x7f5fcc0b8a70 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:21.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.279+0000 7f5fe74e1640 1 --2- 172.21.15.117:0/3992734022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fe01083a0 0x7f5fe019aa30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:21.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.279+0000 7f5fe74e1640 1 --2- 172.21.15.117:0/3992734022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fe0108cf0 0x7f5fe019fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:21.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.279+0000 7f5fe74e1640 1 --2- 172.21.15.117:0/3992734022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5fe0102a60 0x7f5fe019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:21.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.279+0000 7f5fe74e1640 1 -- 172.21.15.117:0/3992734022 >> 172.21.15.117:0/3992734022 conn(0x7f5fe0079730 msgr2=0x7f5fe0101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:54:21.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.279+0000 7f5fe74e1640 1 -- 172.21.15.117:0/3992734022 shutdown_connections 2024-09-21T05:54:21.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:21.279+0000 7f5fe74e1640 1 -- 172.21.15.117:0/3992734022 wait complete. 2024-09-21T05:54:21.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:21 smithi138 ceph-mon[34391]: Detected new or changed devices on smithi192 2024-09-21T05:54:21.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:21 smithi138 ceph-mon[34391]: Adjusting osd_memory_target on smithi192 to 4272M 2024-09-21T05:54:21.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:21 smithi138 ceph-mon[34391]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T05:54:21.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:21 smithi138 ceph-mon[34391]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-09-21T05:54:21.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:21 smithi138 ceph-mon[34391]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T05:54:21.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:21 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T05:54:21.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:21 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T05:54:21.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:21 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3992734022' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T05:54:21.961 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:21 smithi117 ceph-mon[30947]: Detected new or changed devices on smithi192 2024-09-21T05:54:21.961 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:21 smithi117 ceph-mon[30947]: Adjusting osd_memory_target on smithi192 to 4272M 2024-09-21T05:54:21.961 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:21 smithi117 ceph-mon[30947]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T05:54:21.961 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:21 smithi117 ceph-mon[30947]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-09-21T05:54:21.961 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:21 smithi117 ceph-mon[30947]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T05:54:21.961 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:21 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T05:54:21.961 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:21 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T05:54:21.961 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:21 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3992734022' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T05:54:21.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:21 smithi192 ceph-mon[34700]: Detected new or changed devices on smithi192 2024-09-21T05:54:21.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:21 smithi192 ceph-mon[34700]: Adjusting osd_memory_target on smithi192 to 4272M 2024-09-21T05:54:21.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:21 smithi192 ceph-mon[34700]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T05:54:21.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:21 smithi192 ceph-mon[34700]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-09-21T05:54:21.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:21 smithi192 ceph-mon[34700]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T05:54:21.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:21 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T05:54:21.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:21 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T05:54:21.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:21 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3992734022' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T05:54:21.988 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":67,"num_osds":12,"num_up_osds":11,"osd_up_since":1726898037,"num_in_osds":12,"osd_in_since":1726898038,"num_remapped_pgs":0} 2024-09-21T05:54:22.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:22 smithi138 ceph-mon[34391]: purged_snaps scrub starts 2024-09-21T05:54:22.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:22 smithi138 ceph-mon[34391]: purged_snaps scrub ok 2024-09-21T05:54:22.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:22 smithi138 ceph-mon[34391]: osd.11 [v2:172.21.15.192:6824/899090479,v1:172.21.15.192:6825/899090479] boot 2024-09-21T05:54:22.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:22 smithi138 ceph-mon[34391]: osdmap e68: 12 total, 12 up, 12 in 2024-09-21T05:54:22.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:22 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T05:54:22.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:22 smithi192 ceph-mon[34700]: purged_snaps scrub starts 2024-09-21T05:54:22.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:22 smithi192 ceph-mon[34700]: purged_snaps scrub ok 2024-09-21T05:54:22.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:22 smithi192 ceph-mon[34700]: osd.11 [v2:172.21.15.192:6824/899090479,v1:172.21.15.192:6825/899090479] boot 2024-09-21T05:54:22.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:22 smithi192 ceph-mon[34700]: osdmap e68: 12 total, 12 up, 12 in 2024-09-21T05:54:22.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:22 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T05:54:22.988 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T05:54:23.014 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:22 smithi117 ceph-mon[30947]: purged_snaps scrub starts 2024-09-21T05:54:23.014 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:22 smithi117 ceph-mon[30947]: purged_snaps scrub ok 2024-09-21T05:54:23.014 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:22 smithi117 ceph-mon[30947]: osd.11 [v2:172.21.15.192:6824/899090479,v1:172.21.15.192:6825/899090479] boot 2024-09-21T05:54:23.014 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:22 smithi117 ceph-mon[30947]: osdmap e68: 12 total, 12 up, 12 in 2024-09-21T05:54:23.014 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:22 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T05:54:23.236 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:54:23.665 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:23 smithi117 ceph-mon[30947]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T05:54:23.665 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:23 smithi117 ceph-mon[30947]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T05:54:23.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.751+0000 7fb7437a9640 1 -- 172.21.15.117:0/728421202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb73c109df0 msgr2=0x7fb73c110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:23.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.751+0000 7fb7437a9640 1 --2- 172.21.15.117:0/728421202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb73c109df0 0x7fb73c110620 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb734009ca0 tx=0x7fb73402a8c0 comp rx=0 tx=0).stop 2024-09-21T05:54:23.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.751+0000 7fb7437a9640 1 -- 172.21.15.117:0/728421202 shutdown_connections 2024-09-21T05:54:23.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.751+0000 7fb7437a9640 1 --2- 172.21.15.117:0/728421202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb73c109df0 0x7fb73c110620 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:23.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.751+0000 7fb7437a9640 1 --2- 172.21.15.117:0/728421202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb73c1094a0 0x7fb73c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:23.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.751+0000 7fb7437a9640 1 --2- 172.21.15.117:0/728421202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb73c103b60 0x7fb73c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:23.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.751+0000 7fb7437a9640 1 -- 172.21.15.117:0/728421202 >> 172.21.15.117:0/728421202 conn(0x7fb73c07b850 msgr2=0x7fb73c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:54:23.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.752+0000 7fb7437a9640 1 -- 172.21.15.117:0/728421202 shutdown_connections 2024-09-21T05:54:23.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.752+0000 7fb7437a9640 1 -- 172.21.15.117:0/728421202 wait complete. 2024-09-21T05:54:23.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.752+0000 7fb7437a9640 1 Processor -- start 2024-09-21T05:54:23.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.752+0000 7fb7437a9640 1 -- start start 2024-09-21T05:54:23.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.753+0000 7fb7437a9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb73c103b60 0x7fb73c19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:23.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.753+0000 7fb7437a9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb73c1094a0 0x7fb73c19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:23.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.753+0000 7fb7437a9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb73c109df0 0x7fb73c19fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:23.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.753+0000 7fb7437a9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb73c113ab0 con 0x7fb73c109df0 2024-09-21T05:54:23.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.753+0000 7fb7437a9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb73c113930 con 0x7fb73c1094a0 2024-09-21T05:54:23.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.753+0000 7fb7437a9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb73c113c30 con 0x7fb73c103b60 2024-09-21T05:54:23.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.753+0000 7fb741d46640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb73c109df0 0x7fb73c19fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:23.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.753+0000 7fb741d46640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb73c109df0 0x7fb73c19fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42080/0 (socket says 172.21.15.117:42080) 2024-09-21T05:54:23.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.753+0000 7fb741d46640 1 -- 172.21.15.117:0/494899389 learned_addr learned my addr 172.21.15.117:0/494899389 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:54:23.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.753+0000 7fb741545640 1 --2- 172.21.15.117:0/494899389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb73c103b60 0x7fb73c19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:23.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.753+0000 7fb740d44640 1 --2- 172.21.15.117:0/494899389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb73c1094a0 0x7fb73c19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:23.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.753+0000 7fb741d46640 1 -- 172.21.15.117:0/494899389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb73c103b60 msgr2=0x7fb73c19a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:23.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.753+0000 7fb741d46640 1 --2- 172.21.15.117:0/494899389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb73c103b60 0x7fb73c19a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:23.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.753+0000 7fb741d46640 1 -- 172.21.15.117:0/494899389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb73c1094a0 msgr2=0x7fb73c19aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:23.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.753+0000 7fb741d46640 1 --2- 172.21.15.117:0/494899389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb73c1094a0 0x7fb73c19aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:23.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.753+0000 7fb741d46640 1 -- 172.21.15.117:0/494899389 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb73c1a03f0 con 0x7fb73c109df0 2024-09-21T05:54:23.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.753+0000 7fb740d44640 1 --2- 172.21.15.117:0/494899389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb73c1094a0 0x7fb73c19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:54:23.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.753+0000 7fb741d46640 1 --2- 172.21.15.117:0/494899389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb73c109df0 0x7fb73c19fcc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb7340098d0 tx=0x7fb7340317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:54:23.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.753+0000 7fb7327fc640 1 -- 172.21.15.117:0/494899389 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb734003b70 con 0x7fb73c109df0 2024-09-21T05:54:23.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.753+0000 7fb7437a9640 1 -- 172.21.15.117:0/494899389 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb73c1a0680 con 0x7fb73c109df0 2024-09-21T05:54:23.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.753+0000 7fb7327fc640 1 -- 172.21.15.117:0/494899389 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb734003d10 con 0x7fb73c109df0 2024-09-21T05:54:23.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.753+0000 7fb7327fc640 1 -- 172.21.15.117:0/494899389 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73402fa20 con 0x7fb73c109df0 2024-09-21T05:54:23.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.753+0000 7fb7437a9640 1 -- 172.21.15.117:0/494899389 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb73c1a6e40 con 0x7fb73c109df0 2024-09-21T05:54:23.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.754+0000 7fb7437a9640 1 -- 172.21.15.117:0/494899389 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb73c103f70 con 0x7fb73c109df0 2024-09-21T05:54:23.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.756+0000 7fb7327fc640 1 -- 172.21.15.117:0/494899389 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7fb73402fbe0 con 0x7fb73c109df0 2024-09-21T05:54:23.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.757+0000 7fb7327fc640 1 --2- 172.21.15.117:0/494899389 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb7280b6560 0x7fb7280b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:23.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.757+0000 7fb741545640 1 --2- 172.21.15.117:0/494899389 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb7280b6560 0x7fb7280b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:23.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.757+0000 7fb7327fc640 1 -- 172.21.15.117:0/494899389 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7342+0+0 (secure 0 0 0) 0x7fb734108510 con 0x7fb73c109df0 2024-09-21T05:54:23.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.757+0000 7fb741545640 1 --2- 172.21.15.117:0/494899389 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb7280b6560 0x7fb7280b8a70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb738009b60 tx=0x7fb738005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:54:23.776 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.760+0000 7fb7327fc640 1 -- 172.21.15.117:0/494899389 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb7340cf170 con 0x7fb73c109df0 2024-09-21T05:54:23.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:23 smithi138 ceph-mon[34391]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T05:54:23.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:23 smithi138 ceph-mon[34391]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T05:54:23.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.939+0000 7fb7437a9640 1 -- 172.21.15.117:0/494899389 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fb73c19b5b0 con 0x7fb73c109df0 2024-09-21T05:54:23.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.939+0000 7fb7327fc640 1 -- 172.21.15.117:0/494899389 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v69) ==== 74+0+133 (secure 0 0 0) 0x7fb73c19b5b0 con 0x7fb73c109df0 2024-09-21T05:54:23.943 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:54:23.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.942+0000 7fb7437a9640 1 -- 172.21.15.117:0/494899389 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb7280b6560 msgr2=0x7fb7280b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:23.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.942+0000 7fb7437a9640 1 --2- 172.21.15.117:0/494899389 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb7280b6560 0x7fb7280b8a70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb738009b60 tx=0x7fb738005f90 comp rx=0 tx=0).stop 2024-09-21T05:54:23.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.942+0000 7fb7437a9640 1 -- 172.21.15.117:0/494899389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb73c109df0 msgr2=0x7fb73c19fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:23.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.942+0000 7fb7437a9640 1 --2- 172.21.15.117:0/494899389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb73c109df0 0x7fb73c19fcc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb7340098d0 tx=0x7fb7340317b0 comp rx=0 tx=0).stop 2024-09-21T05:54:23.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.942+0000 7fb7437a9640 1 -- 172.21.15.117:0/494899389 shutdown_connections 2024-09-21T05:54:23.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.942+0000 7fb7437a9640 1 --2- 172.21.15.117:0/494899389 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb7280b6560 0x7fb7280b8a70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:23.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.942+0000 7fb7437a9640 1 --2- 172.21.15.117:0/494899389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb73c109df0 0x7fb73c19fcc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:23.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.942+0000 7fb7437a9640 1 --2- 172.21.15.117:0/494899389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb73c1094a0 0x7fb73c19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:23.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.942+0000 7fb7437a9640 1 --2- 172.21.15.117:0/494899389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb73c103b60 0x7fb73c19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:23.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.942+0000 7fb7437a9640 1 -- 172.21.15.117:0/494899389 >> 172.21.15.117:0/494899389 conn(0x7fb73c07b850 msgr2=0x7fb73c107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:54:23.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.942+0000 7fb7437a9640 1 -- 172.21.15.117:0/494899389 shutdown_connections 2024-09-21T05:54:23.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:23.942+0000 7fb7437a9640 1 -- 172.21.15.117:0/494899389 wait complete. 2024-09-21T05:54:23.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:23 smithi192 ceph-mon[34700]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-09-21T05:54:23.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:23 smithi192 ceph-mon[34700]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T05:54:24.712 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:24 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/494899389' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T05:54:24.745 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":69,"num_osds":12,"num_up_osds":12,"osd_up_since":1726898061,"num_in_osds":12,"osd_in_since":1726898038,"num_remapped_pgs":0} 2024-09-21T05:54:24.746 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T05:54:24.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:24 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/494899389' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T05:54:24.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:24 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/494899389' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T05:54:25.029 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:54:25.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.526+0000 7f80be9f9640 1 -- 172.21.15.117:0/3090990918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b81083a0 msgr2=0x7f80b81087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:25.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.526+0000 7f80be9f9640 1 --2- 172.21.15.117:0/3090990918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b81083a0 0x7f80b81087b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f80a0009900 tx=0x7f80a002a410 comp rx=0 tx=0).stop 2024-09-21T05:54:25.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.526+0000 7f80be9f9640 1 -- 172.21.15.117:0/3090990918 shutdown_connections 2024-09-21T05:54:25.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.526+0000 7f80be9f9640 1 --2- 172.21.15.117:0/3090990918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b81083a0 0x7f80b81087b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:25.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.526+0000 7f80be9f9640 1 --2- 172.21.15.117:0/3090990918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80b8108cf0 0x7f80b810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:25.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.526+0000 7f80be9f9640 1 --2- 172.21.15.117:0/3090990918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80b8102a60 0x7f80b8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:25.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.526+0000 7f80be9f9640 1 -- 172.21.15.117:0/3090990918 >> 172.21.15.117:0/3090990918 conn(0x7f80b8079730 msgr2=0x7f80b8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:54:25.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.526+0000 7f80be9f9640 1 -- 172.21.15.117:0/3090990918 shutdown_connections 2024-09-21T05:54:25.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.526+0000 7f80be9f9640 1 -- 172.21.15.117:0/3090990918 wait complete. 2024-09-21T05:54:25.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.527+0000 7f80be9f9640 1 Processor -- start 2024-09-21T05:54:25.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.527+0000 7f80be9f9640 1 -- start start 2024-09-21T05:54:25.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.527+0000 7f80be9f9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b8102a60 0x7f80b81993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:25.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.527+0000 7f80be9f9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80b81083a0 0x7f80b8199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:25.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.527+0000 7f80be9f9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80b8108cf0 0x7f80b819ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:25.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.527+0000 7f80be9f9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f80b8112960 con 0x7f80b8102a60 2024-09-21T05:54:25.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.527+0000 7f80be9f9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f80b81127e0 con 0x7f80b81083a0 2024-09-21T05:54:25.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.527+0000 7f80be9f9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f80b8112ae0 con 0x7f80b8108cf0 2024-09-21T05:54:25.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.527+0000 7f80b7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b8102a60 0x7f80b81993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:25.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.527+0000 7f80b7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b8102a60 0x7f80b81993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37254/0 (socket says 172.21.15.117:37254) 2024-09-21T05:54:25.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.527+0000 7f80b7fff640 1 -- 172.21.15.117:0/923520339 learned_addr learned my addr 172.21.15.117:0/923520339 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:54:25.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.528+0000 7f80bcf96640 1 --2- 172.21.15.117:0/923520339 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80b8108cf0 0x7f80b819ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:25.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.528+0000 7f80b7fff640 1 -- 172.21.15.117:0/923520339 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80b8108cf0 msgr2=0x7f80b819ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:25.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.528+0000 7f80b7fff640 1 --2- 172.21.15.117:0/923520339 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80b8108cf0 0x7f80b819ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:25.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.528+0000 7f80b7fff640 1 -- 172.21.15.117:0/923520339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80b81083a0 msgr2=0x7f80b8199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:25.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.528+0000 7f80b7fff640 1 --2- 172.21.15.117:0/923520339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80b81083a0 0x7f80b8199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:25.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.528+0000 7f80b7fff640 1 -- 172.21.15.117:0/923520339 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b819f340 con 0x7f80b8102a60 2024-09-21T05:54:25.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.528+0000 7f80bcf96640 1 --2- 172.21.15.117:0/923520339 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80b8108cf0 0x7f80b819ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:54:25.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.528+0000 7f80b7fff640 1 --2- 172.21.15.117:0/923520339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b8102a60 0x7f80b81993d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f80a800c890 tx=0x7f80a800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:54:25.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.528+0000 7f80b5ffb640 1 -- 172.21.15.117:0/923520339 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8004220 con 0x7f80b8102a60 2024-09-21T05:54:25.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.528+0000 7f80b5ffb640 1 -- 172.21.15.117:0/923520339 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80a80043c0 con 0x7f80b8102a60 2024-09-21T05:54:25.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.528+0000 7f80be9f9640 1 -- 172.21.15.117:0/923520339 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80b819f630 con 0x7f80b8102a60 2024-09-21T05:54:25.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.528+0000 7f80b5ffb640 1 -- 172.21.15.117:0/923520339 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8004a00 con 0x7f80b8102a60 2024-09-21T05:54:25.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.528+0000 7f80be9f9640 1 -- 172.21.15.117:0/923520339 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80b81a5d90 con 0x7f80b8102a60 2024-09-21T05:54:25.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.529+0000 7f80be9f9640 1 -- 172.21.15.117:0/923520339 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80b8102e70 con 0x7f80b8102a60 2024-09-21T05:54:25.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.531+0000 7f80b5ffb640 1 -- 172.21.15.117:0/923520339 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f80a80079b0 con 0x7f80b8102a60 2024-09-21T05:54:25.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.531+0000 7f80b5ffb640 1 --2- 172.21.15.117:0/923520339 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f80a40b6560 0x7f80a40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:25.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.531+0000 7f80affff640 1 --2- 172.21.15.117:0/923520339 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f80a40b6560 0x7f80a40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:25.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.532+0000 7f80b5ffb640 1 -- 172.21.15.117:0/923520339 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7342+0+0 (secure 0 0 0) 0x7f80a80e71d0 con 0x7f80b8102a60 2024-09-21T05:54:25.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.532+0000 7f80affff640 1 --2- 172.21.15.117:0/923520339 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f80a40b6560 0x7f80a40b8a70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f80a002a930 tx=0x7f80a0031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:54:25.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.534+0000 7f80b5ffb640 1 -- 172.21.15.117:0/923520339 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f80a8014030 con 0x7f80b8102a60 2024-09-21T05:54:25.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.713+0000 7f80be9f9640 1 -- 172.21.15.117:0/923520339 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f80b819a500 con 0x7f80b8102a60 2024-09-21T05:54:25.716 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:25 smithi117 ceph-mon[30947]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:25.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.714+0000 7f80b5ffb640 1 -- 172.21.15.117:0/923520339 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+18869 (secure 0 0 0) 0x7f80a80ade30 con 0x7f80b8102a60 2024-09-21T05:54:25.717 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:54:25.717 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":69,"fsid":"ac8581a2-77dc-11ef-baf5-efdc52797490","created":"2024-09-21T05:46:47.021846+0000","modified":"2024-09-21T05:54:22.580237+0000","last_up_change":"2024-09-21T05:54:21.572849+0000","last_in_change":"2024-09-21T05:53:58.892597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:50:32.717389+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"68315502-fa6c-4a6e-a2d4-a7dde8071c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6803","nonce":3761478035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6805","nonce":3761478035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6809","nonce":3761478035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6807","nonce":3761478035}]},"public_addr":"172.21.15.117:6803/3761478035","cluster_addr":"172.21.15.117:6805/3761478035","heartbeat_back_addr":"172.21.15.117:6809/3761478035","heartbeat_front_addr":"172.21.15.117:6807/3761478035","state":["exists","up"]},{"osd":1,"uuid":"d8910a08-4ef3-49f2-a4db-e3588d85d12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6801","nonce":1737365588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6803","nonce":1737365588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6807","nonce":1737365588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6805","nonce":1737365588}]},"public_addr":"172.21.15.138:6801/1737365588","cluster_addr":"172.21.15.138:6803/1737365588","heartbeat_back_addr":"172.21.15.138:6807/1737365588","heartbeat_front_addr":"172.21.15.138:6805/1737365588","state":["exists","up"]},{"osd":2,"uuid":"94972378-a11f-4129-9e40-ad3fdec1d189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6801","nonce":3002534992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6803","nonce":3002534992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6807","nonce":3002534992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6805","nonce":3002534992}]},"public_addr":"172.21.15.192:6801/3002534992","cluster_addr":"172.21.15.192:6803/3002534992","heartbeat_back_addr":"172.21.15.192:6807/3002534992","heartbeat_front_addr":"172.21.15.192:6805/3002534992","state":["exists","up"]},{"osd":3,"uuid":"d4168c43-f0de-4445-945c-92d1cf270c33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6811","nonce":1222400293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6813","nonce":1222400293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6817","nonce":1222400293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6815","nonce":1222400293}]},"public_addr":"172.21.15.117:6811/1222400293","cluster_addr":"172.21.15.117:6813/1222400293","heartbeat_back_addr":"172.21.15.117:6817/1222400293","heartbeat_front_addr":"172.21.15.117:6815/1222400293","state":["exists","up"]},{"osd":4,"uuid":"28f1d7c0-8392-4a33-85c0-0de611cdd4b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6809","nonce":4294892870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6811","nonce":4294892870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6815","nonce":4294892870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6813","nonce":4294892870}]},"public_addr":"172.21.15.138:6809/4294892870","cluster_addr":"172.21.15.138:6811/4294892870","heartbeat_back_addr":"172.21.15.138:6815/4294892870","heartbeat_front_addr":"172.21.15.138:6813/4294892870","state":["exists","up"]},{"osd":5,"uuid":"ca902407-45e8-4bc5-bdbd-a729c1de8322","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6809","nonce":1776840019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6811","nonce":1776840019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6815","nonce":1776840019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6813","nonce":1776840019}]},"public_addr":"172.21.15.192:6809/1776840019","cluster_addr":"172.21.15.192:6811/1776840019","heartbeat_back_addr":"172.21.15.192:6815/1776840019","heartbeat_front_addr":"172.21.15.192:6813/1776840019","state":["exists","up"]},{"osd":6,"uuid":"6e507a91-ed09-4a33-bf0e-c4b593b3c5aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6819","nonce":1093341590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6821","nonce":1093341590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6825","nonce":1093341590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6823","nonce":1093341590}]},"public_addr":"172.21.15.117:6819/1093341590","cluster_addr":"172.21.15.117:6821/1093341590","heartbeat_back_addr":"172.21.15.117:6825/1093341590","heartbeat_front_addr":"172.21.15.117:6823/1093341590","state":["exists","up"]},{"osd":7,"uuid":"36a47dd7-67f7-4ebe-8759-3a8bb6bd2ea3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6817","nonce":1448179714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6819","nonce":1448179714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6823","nonce":1448179714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6821","nonce":1448179714}]},"public_addr":"172.21.15.138:6817/1448179714","cluster_addr":"172.21.15.138:6819/1448179714","heartbeat_back_addr":"172.21.15.138:6823/1448179714","heartbeat_front_addr":"172.21.15.138:6821/1448179714","state":["exists","up"]},{"osd":8,"uuid":"118e0b5d-2619-4512-94c5-8d07a0d3a28f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6817","nonce":3960902885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6819","nonce":3960902885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6823","nonce":3960902885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6821","nonce":3960902885}]},"public_addr":"172.21.15.192:6817/3960902885","cluster_addr":"172.21.15.192:6819/3960902885","heartbeat_back_addr":"172.21.15.192:6823/3960902885","heartbeat_front_addr":"172.21.15.192:6821/3960902885","state":["exists","up"]},{"osd":9,"uuid":"47543bc6-fa0b-4e72-9742-97dcda0b7f8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6827","nonce":3453855195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6829","nonce":3453855195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6833","nonce":3453855195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6831","nonce":3453855195}]},"public_addr":"172.21.15.117:6827/3453855195","cluster_addr":"172.21.15.117:6829/3453855195","heartbeat_back_addr":"172.21.15.117:6833/3453855195","heartbeat_front_addr":"172.21.15.117:6831/3453855195","state":["exists","up"]},{"osd":10,"uuid":"5e6708b4-7a85-4fa5-a03c-27d99f7d749f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6825","nonce":903392451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6827","nonce":903392451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6831","nonce":903392451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6829","nonce":903392451}]},"public_addr":"172.21.15.138:6825/903392451","cluster_addr":"172.21.15.138:6827/903392451","heartbeat_back_addr":"172.21.15.138:6831/903392451","heartbeat_front_addr":"172.21.15.138:6829/903392451","state":["exists","up"]},{"osd":11,"uuid":"24bc2ef7-e2b4-43c6-9e6c-43ae02390415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6825","nonce":899090479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6827","nonce":899090479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6831","nonce":899090479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6829","nonce":899090479}]},"public_addr":"172.21.15.192:6825/899090479","cluster_addr":"172.21.15.192:6827/899090479","heartbeat_back_addr":"172.21.15.192:6831/899090479","heartbeat_front_addr":"172.21.15.192:6829/899090479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:49:40.742217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:05.963655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:30.164495+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:56.598899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:51:21.940331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:51:47.777445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:52:12.869007+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:52:39.454054+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:03.569226+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:29.760863+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:55.788604+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:54:20.510184+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:0/2989394419":"2024-09-22T05:48:32.406654+0000","172.21.15.117:6801/1702676172":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/1766406059":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6800/2337036514":"2024-09-22T05:47:09.112435+0000","172.21.15.117:0/985798343":"2024-09-22T05:47:09.112435+0000","172.21.15.117:0/1083273222":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/473284565":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6801/2337036514":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6800/816635165":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/2642940197":"2024-09-22T05:48:32.406654+0000","172.21.15.117:6801/816635165":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/2225537433":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/3651584169":"2024-09-22T05:47:33.789146+0000","172.21.15.117:6800/1702676172":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/1284403882":"2024-09-22T05:47:33.789146+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:54:25.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.716+0000 7f80be9f9640 1 -- 172.21.15.117:0/923520339 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f80a40b6560 msgr2=0x7f80a40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:25.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.716+0000 7f80be9f9640 1 --2- 172.21.15.117:0/923520339 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f80a40b6560 0x7f80a40b8a70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f80a002a930 tx=0x7f80a0031400 comp rx=0 tx=0).stop 2024-09-21T05:54:25.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.716+0000 7f80be9f9640 1 -- 172.21.15.117:0/923520339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b8102a60 msgr2=0x7f80b81993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:25.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.716+0000 7f80be9f9640 1 --2- 172.21.15.117:0/923520339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b8102a60 0x7f80b81993d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f80a800c890 tx=0x7f80a800cd50 comp rx=0 tx=0).stop 2024-09-21T05:54:25.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.716+0000 7f80be9f9640 1 -- 172.21.15.117:0/923520339 shutdown_connections 2024-09-21T05:54:25.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.716+0000 7f80be9f9640 1 --2- 172.21.15.117:0/923520339 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f80a40b6560 0x7f80a40b8a70 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:25.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.716+0000 7f80be9f9640 1 --2- 172.21.15.117:0/923520339 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80b8108cf0 0x7f80b819ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:25.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.716+0000 7f80be9f9640 1 --2- 172.21.15.117:0/923520339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80b81083a0 0x7f80b8199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:25.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.716+0000 7f80be9f9640 1 --2- 172.21.15.117:0/923520339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b8102a60 0x7f80b81993d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:25.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.716+0000 7f80be9f9640 1 -- 172.21.15.117:0/923520339 >> 172.21.15.117:0/923520339 conn(0x7f80b8079730 msgr2=0x7f80b81044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:54:25.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.716+0000 7f80be9f9640 1 -- 172.21.15.117:0/923520339 shutdown_connections 2024-09-21T05:54:25.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:25.716+0000 7f80be9f9640 1 -- 172.21.15.117:0/923520339 wait complete. 2024-09-21T05:54:25.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:25 smithi138 ceph-mon[34391]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:25.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:25 smithi192 ceph-mon[34700]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:26.559 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T05:50:32.717389+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T05:54:26.559 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph osd pool get .mgr pg_num 2024-09-21T05:54:26.804 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:54:26.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:26 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/923520339' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T05:54:26.924 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:26 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/923520339' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T05:54:26.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:26 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/923520339' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T05:54:27.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.296+0000 7f8b97737640 1 -- 172.21.15.117:0/2315137272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b901049d0 msgr2=0x7f8b90104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:27.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.296+0000 7f8b97737640 1 --2- 172.21.15.117:0/2315137272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b901049d0 0x7f8b90104de0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8b80009ca0 tx=0x7f8b8002a8c0 comp rx=0 tx=0).stop 2024-09-21T05:54:27.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.296+0000 7f8b97737640 1 -- 172.21.15.117:0/2315137272 shutdown_connections 2024-09-21T05:54:27.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.296+0000 7f8b97737640 1 --2- 172.21.15.117:0/2315137272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b9010de30 0x7f8b90110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:27.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.296+0000 7f8b97737640 1 --2- 172.21.15.117:0/2315137272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b90105320 0x7f8b9010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:27.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.296+0000 7f8b97737640 1 --2- 172.21.15.117:0/2315137272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b901049d0 0x7f8b90104de0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:27.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.296+0000 7f8b97737640 1 -- 172.21.15.117:0/2315137272 >> 172.21.15.117:0/2315137272 conn(0x7f8b90079740 msgr2=0x7f8b90079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:54:27.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.296+0000 7f8b97737640 1 -- 172.21.15.117:0/2315137272 shutdown_connections 2024-09-21T05:54:27.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.296+0000 7f8b97737640 1 -- 172.21.15.117:0/2315137272 wait complete. 2024-09-21T05:54:27.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.297+0000 7f8b97737640 1 Processor -- start 2024-09-21T05:54:27.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.297+0000 7f8b97737640 1 -- start start 2024-09-21T05:54:27.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.297+0000 7f8b97737640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b901049d0 0x7f8b9019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:27.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.297+0000 7f8b97737640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b90105320 0x7f8b9019aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:27.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.297+0000 7f8b97737640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b9010de30 0x7f8b9019fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:27.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.297+0000 7f8b97737640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8b90113b30 con 0x7f8b9010de30 2024-09-21T05:54:27.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.297+0000 7f8b97737640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8b901139b0 con 0x7f8b901049d0 2024-09-21T05:54:27.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.297+0000 7f8b97737640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8b90113cb0 con 0x7f8b90105320 2024-09-21T05:54:27.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.297+0000 7f8b95cd4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b9010de30 0x7f8b9019fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:27.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.297+0000 7f8b95cd4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b9010de30 0x7f8b9019fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37266/0 (socket says 172.21.15.117:37266) 2024-09-21T05:54:27.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.297+0000 7f8b95cd4640 1 -- 172.21.15.117:0/2198232411 learned_addr learned my addr 172.21.15.117:0/2198232411 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:54:27.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.297+0000 7f8b94cd2640 1 --2- 172.21.15.117:0/2198232411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b90105320 0x7f8b9019aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:27.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.297+0000 7f8b954d3640 1 --2- 172.21.15.117:0/2198232411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b901049d0 0x7f8b9019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:27.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.297+0000 7f8b95cd4640 1 -- 172.21.15.117:0/2198232411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b90105320 msgr2=0x7f8b9019aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:27.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.297+0000 7f8b95cd4640 1 --2- 172.21.15.117:0/2198232411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b90105320 0x7f8b9019aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:27.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.297+0000 7f8b95cd4640 1 -- 172.21.15.117:0/2198232411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b901049d0 msgr2=0x7f8b9019a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:27.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.297+0000 7f8b95cd4640 1 --2- 172.21.15.117:0/2198232411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b901049d0 0x7f8b9019a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:27.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.297+0000 7f8b95cd4640 1 -- 172.21.15.117:0/2198232411 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b901a04b0 con 0x7f8b9010de30 2024-09-21T05:54:27.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.297+0000 7f8b94cd2640 1 --2- 172.21.15.117:0/2198232411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b90105320 0x7f8b9019aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:54:27.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.298+0000 7f8b95cd4640 1 --2- 172.21.15.117:0/2198232411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b9010de30 0x7f8b9019fd80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8b88009fd0 tx=0x7f8b8800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:54:27.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.298+0000 7f8b8e7fc640 1 -- 172.21.15.117:0/2198232411 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b88011830 con 0x7f8b9010de30 2024-09-21T05:54:27.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.298+0000 7f8b8e7fc640 1 -- 172.21.15.117:0/2198232411 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b88016070 con 0x7f8b9010de30 2024-09-21T05:54:27.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.298+0000 7f8b97737640 1 -- 172.21.15.117:0/2198232411 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b901a07a0 con 0x7f8b9010de30 2024-09-21T05:54:27.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.298+0000 7f8b8e7fc640 1 -- 172.21.15.117:0/2198232411 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b88010de0 con 0x7f8b9010de30 2024-09-21T05:54:27.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.298+0000 7f8b97737640 1 -- 172.21.15.117:0/2198232411 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b901a6f00 con 0x7f8b9010de30 2024-09-21T05:54:27.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.299+0000 7f8b97737640 1 -- 172.21.15.117:0/2198232411 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b9006bf20 con 0x7f8b9010de30 2024-09-21T05:54:27.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.300+0000 7f8b8e7fc640 1 -- 172.21.15.117:0/2198232411 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f8b880119d0 con 0x7f8b9010de30 2024-09-21T05:54:27.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.301+0000 7f8b8e7fc640 1 --2- 172.21.15.117:0/2198232411 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8b780b6560 0x7f8b780b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:27.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.301+0000 7f8b954d3640 1 --2- 172.21.15.117:0/2198232411 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8b780b6560 0x7f8b780b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:27.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.301+0000 7f8b8e7fc640 1 -- 172.21.15.117:0/2198232411 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7342+0+0 (secure 0 0 0) 0x7f8b880e6cc0 con 0x7f8b9010de30 2024-09-21T05:54:27.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.302+0000 7f8b954d3640 1 --2- 172.21.15.117:0/2198232411 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8b780b6560 0x7f8b780b8a70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8b80009ca0 tx=0x7f8b80005fd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:54:27.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.304+0000 7f8b8e7fc640 1 -- 172.21.15.117:0/2198232411 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8b880ad920 con 0x7f8b9010de30 2024-09-21T05:54:27.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.482+0000 7f8b97737640 1 -- 172.21.15.117:0/2198232411 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8b9019b670 con 0x7f8b9010de30 2024-09-21T05:54:27.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.483+0000 7f8b8e7fc640 1 -- 172.21.15.117:0/2198232411 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) ==== 93+0+10 (secure 0 0 0) 0x7f8b880b1d00 con 0x7f8b9010de30 2024-09-21T05:54:27.486 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 1 2024-09-21T05:54:27.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.485+0000 7f8b97737640 1 -- 172.21.15.117:0/2198232411 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8b780b6560 msgr2=0x7f8b780b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:27.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.485+0000 7f8b97737640 1 --2- 172.21.15.117:0/2198232411 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8b780b6560 0x7f8b780b8a70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8b80009ca0 tx=0x7f8b80005fd0 comp rx=0 tx=0).stop 2024-09-21T05:54:27.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.485+0000 7f8b97737640 1 -- 172.21.15.117:0/2198232411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b9010de30 msgr2=0x7f8b9019fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:27.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.485+0000 7f8b97737640 1 --2- 172.21.15.117:0/2198232411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b9010de30 0x7f8b9019fd80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8b88009fd0 tx=0x7f8b8800b8e0 comp rx=0 tx=0).stop 2024-09-21T05:54:27.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.485+0000 7f8b97737640 1 -- 172.21.15.117:0/2198232411 shutdown_connections 2024-09-21T05:54:27.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.485+0000 7f8b97737640 1 --2- 172.21.15.117:0/2198232411 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8b780b6560 0x7f8b780b8a70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:27.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.485+0000 7f8b97737640 1 --2- 172.21.15.117:0/2198232411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8b9010de30 0x7f8b9019fd80 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:27.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.485+0000 7f8b97737640 1 --2- 172.21.15.117:0/2198232411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b90105320 0x7f8b9019aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:27.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.485+0000 7f8b97737640 1 --2- 172.21.15.117:0/2198232411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b901049d0 0x7f8b9019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:27.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.485+0000 7f8b97737640 1 -- 172.21.15.117:0/2198232411 >> 172.21.15.117:0/2198232411 conn(0x7f8b90079740 msgr2=0x7f8b901007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:54:27.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.486+0000 7f8b97737640 1 -- 172.21.15.117:0/2198232411 shutdown_connections 2024-09-21T05:54:27.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:27.486+0000 7f8b97737640 1 -- 172.21.15.117:0/2198232411 wait complete. 2024-09-21T05:54:27.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:27 smithi117 ceph-mon[30947]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:27.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:27 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2198232411' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T05:54:27.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:27 smithi138 ceph-mon[34391]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:27.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:27 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2198232411' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T05:54:27.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:27 smithi192 ceph-mon[34700]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:27.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:27 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2198232411' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T05:54:28.328 INFO:tasks.cephadm:Adding mds.a on smithi117 2024-09-21T05:54:28.328 INFO:tasks.cephadm:Adding mds.d on smithi117 2024-09-21T05:54:28.328 INFO:tasks.cephadm:Adding mds.g on smithi117 2024-09-21T05:54:28.328 INFO:tasks.cephadm:Adding mds.j on smithi117 2024-09-21T05:54:28.328 INFO:tasks.cephadm:Adding mds.b on smithi138 2024-09-21T05:54:28.328 INFO:tasks.cephadm:Adding mds.e on smithi138 2024-09-21T05:54:28.328 INFO:tasks.cephadm:Adding mds.h on smithi138 2024-09-21T05:54:28.328 INFO:tasks.cephadm:Adding mds.k on smithi138 2024-09-21T05:54:28.329 INFO:tasks.cephadm:Adding mds.c on smithi192 2024-09-21T05:54:28.329 INFO:tasks.cephadm:Adding mds.f on smithi192 2024-09-21T05:54:28.329 INFO:tasks.cephadm:Adding mds.i on smithi192 2024-09-21T05:54:28.329 INFO:tasks.cephadm:Adding mds.l on smithi192 2024-09-21T05:54:28.329 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph orch apply mds all '12;smithi117=a;smithi117=d;smithi117=g;smithi117=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l' 2024-09-21T05:54:28.590 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T05:54:29.055 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.052+0000 7f8907f72640 1 -- 172.21.15.192:0/247439009 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f890010de10 msgr2=0x7f8900110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:29.055 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.052+0000 7f8907f72640 1 --2- 172.21.15.192:0/247439009 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f890010de10 0x7f8900110300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f88f8009900 tx=0x7f88f802a410 comp rx=0 tx=0).stop 2024-09-21T05:54:29.055 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.052+0000 7f8907f72640 1 -- 172.21.15.192:0/247439009 shutdown_connections 2024-09-21T05:54:29.055 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.052+0000 7f8907f72640 1 --2- 172.21.15.192:0/247439009 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f890010de10 0x7f8900110300 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:29.055 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.052+0000 7f8907f72640 1 --2- 172.21.15.192:0/247439009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8900105300 0x7f890010d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:29.056 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.052+0000 7f8907f72640 1 --2- 172.21.15.192:0/247439009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89001049b0 0x7f8900104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:29.056 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.052+0000 7f8907f72640 1 -- 172.21.15.192:0/247439009 >> 172.21.15.192:0/247439009 conn(0x7f8900079730 msgr2=0x7f8900079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:54:29.056 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.052+0000 7f8907f72640 1 -- 172.21.15.192:0/247439009 shutdown_connections 2024-09-21T05:54:29.056 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.052+0000 7f8907f72640 1 -- 172.21.15.192:0/247439009 wait complete. 2024-09-21T05:54:29.056 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.053+0000 7f8907f72640 1 Processor -- start 2024-09-21T05:54:29.056 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.053+0000 7f8907f72640 1 -- start start 2024-09-21T05:54:29.057 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.054+0000 7f8907f72640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89001049b0 0x7f890019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:29.057 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.054+0000 7f8907f72640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8900105300 0x7f890019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:29.057 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.054+0000 7f8907f72640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f890010de10 0x7f890019fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:29.058 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.054+0000 7f8907f72640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8900113b30 con 0x7f8900105300 2024-09-21T05:54:29.058 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.054+0000 7f8907f72640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f89001139b0 con 0x7f89001049b0 2024-09-21T05:54:29.058 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.054+0000 7f8907f72640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8900113cb0 con 0x7f890010de10 2024-09-21T05:54:29.058 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.054+0000 7f890650f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f890010de10 0x7f890019fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:29.058 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.054+0000 7f890650f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f890010de10 0x7f890019fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:36522/0 (socket says 172.21.15.192:36522) 2024-09-21T05:54:29.058 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.054+0000 7f890650f640 1 -- 172.21.15.192:0/780523133 learned_addr learned my addr 172.21.15.192:0/780523133 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-09-21T05:54:29.058 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.054+0000 7f890550d640 1 --2- 172.21.15.192:0/780523133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8900105300 0x7f890019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:29.058 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.054+0000 7f8905d0e640 1 --2- 172.21.15.192:0/780523133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89001049b0 0x7f890019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:29.058 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.054+0000 7f890650f640 1 -- 172.21.15.192:0/780523133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89001049b0 msgr2=0x7f890019a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:29.058 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.054+0000 7f890650f640 1 --2- 172.21.15.192:0/780523133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89001049b0 0x7f890019a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:29.058 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.054+0000 7f890650f640 1 -- 172.21.15.192:0/780523133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8900105300 msgr2=0x7f890019aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:29.058 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.054+0000 7f890650f640 1 --2- 172.21.15.192:0/780523133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8900105300 0x7f890019aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:29.058 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.054+0000 7f890650f640 1 -- 172.21.15.192:0/780523133 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89001a0490 con 0x7f890010de10 2024-09-21T05:54:29.059 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.054+0000 7f890650f640 1 --2- 172.21.15.192:0/780523133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f890010de10 0x7f890019fd60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f88f8009a20 tx=0x7f88f8004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:54:29.059 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.054+0000 7f88feffd640 1 -- 172.21.15.192:0/780523133 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f803f030 con 0x7f890010de10 2024-09-21T05:54:29.059 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.054+0000 7f8907f72640 1 -- 172.21.15.192:0/780523133 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89001a0720 con 0x7f890010de10 2024-09-21T05:54:29.059 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.054+0000 7f88feffd640 1 -- 172.21.15.192:0/780523133 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88f8003aa0 con 0x7f890010de10 2024-09-21T05:54:29.059 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.054+0000 7f88feffd640 1 -- 172.21.15.192:0/780523133 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f8035070 con 0x7f890010de10 2024-09-21T05:54:29.059 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.055+0000 7f8907f72640 1 -- 172.21.15.192:0/780523133 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89001a6ee0 con 0x7f890010de10 2024-09-21T05:54:29.059 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.055+0000 7f8907f72640 1 -- 172.21.15.192:0/780523133 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8900104dc0 con 0x7f890010de10 2024-09-21T05:54:29.059 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.057+0000 7f88feffd640 1 -- 172.21.15.192:0/780523133 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f88f8042030 con 0x7f890010de10 2024-09-21T05:54:29.059 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.058+0000 7f88feffd640 1 --2- 172.21.15.192:0/780523133 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f88e80b6560 0x7f88e80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:29.060 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.058+0000 7f88feffd640 1 -- 172.21.15.192:0/780523133 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7342+0+0 (secure 0 0 0) 0x7f88f81079e0 con 0x7f890010de10 2024-09-21T05:54:29.060 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.058+0000 7f8905d0e640 1 --2- 172.21.15.192:0/780523133 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f88e80b6560 0x7f88e80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:29.060 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.059+0000 7f8905d0e640 1 --2- 172.21.15.192:0/780523133 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f88e80b6560 0x7f88e80b8a70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f890006bc60 tx=0x7f88f000d040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:54:29.063 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.061+0000 7f88feffd640 1 -- 172.21.15.192:0/780523133 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f88f803be20 con 0x7f890010de10 2024-09-21T05:54:29.254 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.252+0000 7f8907f72640 1 -- 172.21.15.192:0/780523133 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi117=a;smithi117=d;smithi117=g;smithi117=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}) -- 0x7f8900100c70 con 0x7f88e80b6560 2024-09-21T05:54:29.263 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.261+0000 7f88feffd640 1 -- 172.21.15.192:0/780523133 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f8900100c70 con 0x7f88e80b6560 2024-09-21T05:54:29.263 INFO:teuthology.orchestra.run.smithi192.stdout:Scheduled mds.all update... 2024-09-21T05:54:29.265 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.263+0000 7f8907f72640 1 -- 172.21.15.192:0/780523133 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f88e80b6560 msgr2=0x7f88e80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:29.265 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.263+0000 7f8907f72640 1 --2- 172.21.15.192:0/780523133 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f88e80b6560 0x7f88e80b8a70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f890006bc60 tx=0x7f88f000d040 comp rx=0 tx=0).stop 2024-09-21T05:54:29.265 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.263+0000 7f8907f72640 1 -- 172.21.15.192:0/780523133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f890010de10 msgr2=0x7f890019fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:29.266 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.263+0000 7f8907f72640 1 --2- 172.21.15.192:0/780523133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f890010de10 0x7f890019fd60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f88f8009a20 tx=0x7f88f8004270 comp rx=0 tx=0).stop 2024-09-21T05:54:29.266 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.263+0000 7f8907f72640 1 -- 172.21.15.192:0/780523133 shutdown_connections 2024-09-21T05:54:29.266 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.263+0000 7f8907f72640 1 --2- 172.21.15.192:0/780523133 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f88e80b6560 0x7f88e80b8a70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:29.266 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.263+0000 7f8907f72640 1 --2- 172.21.15.192:0/780523133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8900105300 0x7f890019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:29.266 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.263+0000 7f8907f72640 1 --2- 172.21.15.192:0/780523133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f890010de10 0x7f890019fd60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:29.266 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.263+0000 7f8907f72640 1 --2- 172.21.15.192:0/780523133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89001049b0 0x7f890019a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:29.266 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.263+0000 7f8907f72640 1 -- 172.21.15.192:0/780523133 >> 172.21.15.192:0/780523133 conn(0x7f8900079730 msgr2=0x7f89000ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:54:29.266 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.263+0000 7f8907f72640 1 -- 172.21.15.192:0/780523133 shutdown_connections 2024-09-21T05:54:29.266 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-21T05:54:29.263+0000 7f8907f72640 1 -- 172.21.15.192:0/780523133 wait complete. 2024-09-21T05:54:29.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:29 smithi138 ceph-mon[34391]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:29.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:29 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:29.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:29 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:54:29.927 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:29 smithi192 ceph-mon[34700]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:29.927 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:29 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:29.927 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:29 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:54:29.967 DEBUG:teuthology.orchestra.run.smithi117:mds.a> sudo journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.a.service 2024-09-21T05:54:29.970 DEBUG:teuthology.orchestra.run.smithi117:mds.d> sudo journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.d.service 2024-09-21T05:54:29.989 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:29 smithi117 ceph-mon[30947]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:29.989 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:29 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:29.990 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:29 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:54:29.990 DEBUG:teuthology.orchestra.run.smithi117:mds.g> sudo journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.g.service 2024-09-21T05:54:30.033 DEBUG:teuthology.orchestra.run.smithi117:mds.j> sudo journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.j.service 2024-09-21T05:54:30.076 DEBUG:teuthology.orchestra.run.smithi138:mds.b> sudo journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.b.service 2024-09-21T05:54:30.079 DEBUG:teuthology.orchestra.run.smithi138:mds.e> sudo journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.e.service 2024-09-21T05:54:30.122 DEBUG:teuthology.orchestra.run.smithi138:mds.h> sudo journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.h.service 2024-09-21T05:54:30.165 DEBUG:teuthology.orchestra.run.smithi138:mds.k> sudo journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.k.service 2024-09-21T05:54:30.208 DEBUG:teuthology.orchestra.run.smithi192:mds.c> sudo journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.c.service 2024-09-21T05:54:30.211 DEBUG:teuthology.orchestra.run.smithi192:mds.f> sudo journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.f.service 2024-09-21T05:54:30.255 DEBUG:teuthology.orchestra.run.smithi192:mds.i> sudo journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.i.service 2024-09-21T05:54:30.298 DEBUG:teuthology.orchestra.run.smithi192:mds.l> sudo journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.l.service 2024-09-21T05:54:30.341 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-21T05:54:30.342 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T05:54:30.587 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:54:30.708 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:30 smithi117 ceph-mon[30947]: from='client.24577 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi117=a;smithi117=d;smithi117=g;smithi117=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:54:30.708 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:30 smithi117 ceph-mon[30947]: Saving service mds.all spec with placement smithi117=a;smithi117=d;smithi117=g;smithi117=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2024-09-21T05:54:30.708 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:30 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:30.708 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:30 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:54:30.708 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:30 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:30.708 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:30 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:54:30.708 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:30 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:54:30.708 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:30 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:30.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:30 smithi138 ceph-mon[34391]: from='client.24577 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi117=a;smithi117=d;smithi117=g;smithi117=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:54:30.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:30 smithi138 ceph-mon[34391]: Saving service mds.all spec with placement smithi117=a;smithi117=d;smithi117=g;smithi117=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2024-09-21T05:54:30.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:30 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:30.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:30 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:54:30.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:30 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:30.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:30 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:54:30.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:30 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:54:30.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:30 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:30.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:30 smithi192 ceph-mon[34700]: from='client.24577 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi117=a;smithi117=d;smithi117=g;smithi117=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:54:30.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:30 smithi192 ceph-mon[34700]: Saving service mds.all spec with placement smithi117=a;smithi117=d;smithi117=g;smithi117=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2024-09-21T05:54:30.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:30 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:30.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:30 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:54:30.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:30 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:30.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:30 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:54:30.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:30 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:54:30.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:30 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:31.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.096+0000 7f91bf141640 1 -- 172.21.15.117:0/441616625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91b8105ac0 msgr2=0x7f91b810f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:31.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.096+0000 7f91bf141640 1 --2- 172.21.15.117:0/441616625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91b8105ac0 0x7f91b810f530 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f91b0009ca0 tx=0x7f91b002a8c0 comp rx=0 tx=0).stop 2024-09-21T05:54:31.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.096+0000 7f91bf141640 1 -- 172.21.15.117:0/441616625 shutdown_connections 2024-09-21T05:54:31.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.096+0000 7f91bf141640 1 --2- 172.21.15.117:0/441616625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91b8105ac0 0x7f91b810f530 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:31.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.096+0000 7f91bf141640 1 --2- 172.21.15.117:0/441616625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91b8105170 0x7f91b8105580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:31.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.096+0000 7f91bf141640 1 --2- 172.21.15.117:0/441616625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91b8103160 0x7f91b8103570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:31.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.096+0000 7f91bf141640 1 -- 172.21.15.117:0/441616625 >> 172.21.15.117:0/441616625 conn(0x7f91b8077c60 msgr2=0x7f91b80780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:54:31.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.096+0000 7f91bf141640 1 -- 172.21.15.117:0/441616625 shutdown_connections 2024-09-21T05:54:31.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.096+0000 7f91bf141640 1 -- 172.21.15.117:0/441616625 wait complete. 2024-09-21T05:54:31.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.097+0000 7f91bf141640 1 Processor -- start 2024-09-21T05:54:31.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.097+0000 7f91bf141640 1 -- start start 2024-09-21T05:54:31.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.097+0000 7f91bf141640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91b8103160 0x7f91b8199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:31.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.097+0000 7f91bf141640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91b8105170 0x7f91b8199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:31.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.097+0000 7f91bf141640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91b8105ac0 0x7f91b819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:31.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.097+0000 7f91bf141640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f91b81129c0 con 0x7f91b8103160 2024-09-21T05:54:31.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.097+0000 7f91bf141640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f91b8112840 con 0x7f91b8105170 2024-09-21T05:54:31.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.097+0000 7f91bf141640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f91b8112b40 con 0x7f91b8105ac0 2024-09-21T05:54:31.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.097+0000 7f91bcedd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91b8103160 0x7f91b8199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:31.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.097+0000 7f91bcedd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91b8103160 0x7f91b8199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37280/0 (socket says 172.21.15.117:37280) 2024-09-21T05:54:31.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.097+0000 7f91bcedd640 1 -- 172.21.15.117:0/1521701603 learned_addr learned my addr 172.21.15.117:0/1521701603 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:54:31.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.097+0000 7f91bcedd640 1 -- 172.21.15.117:0/1521701603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91b8105ac0 msgr2=0x7f91b819ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:31.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.097+0000 7f91bcedd640 1 --2- 172.21.15.117:0/1521701603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91b8105ac0 0x7f91b819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:31.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.097+0000 7f91bcedd640 1 -- 172.21.15.117:0/1521701603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91b8105170 msgr2=0x7f91b8199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:31.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.097+0000 7f91bcedd640 1 --2- 172.21.15.117:0/1521701603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91b8105170 0x7f91b8199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:31.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.097+0000 7f91bcedd640 1 -- 172.21.15.117:0/1521701603 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b819f220 con 0x7f91b8103160 2024-09-21T05:54:31.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.098+0000 7f91bcedd640 1 --2- 172.21.15.117:0/1521701603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91b8103160 0x7f91b8199370 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f91b400c920 tx=0x7f91b400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:54:31.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.098+0000 7f91adffb640 1 -- 172.21.15.117:0/1521701603 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b4007c70 con 0x7f91b8103160 2024-09-21T05:54:31.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.098+0000 7f91bf141640 1 -- 172.21.15.117:0/1521701603 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b819f510 con 0x7f91b8103160 2024-09-21T05:54:31.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.098+0000 7f91adffb640 1 -- 172.21.15.117:0/1521701603 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91b4007e10 con 0x7f91b8103160 2024-09-21T05:54:31.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.098+0000 7f91adffb640 1 -- 172.21.15.117:0/1521701603 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b4018820 con 0x7f91b8103160 2024-09-21T05:54:31.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.098+0000 7f91bf141640 1 -- 172.21.15.117:0/1521701603 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b81a5d30 con 0x7f91b8103160 2024-09-21T05:54:31.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.099+0000 7f91bf141640 1 -- 172.21.15.117:0/1521701603 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91b8103570 con 0x7f91b8103160 2024-09-21T05:54:31.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.100+0000 7f91adffb640 1 -- 172.21.15.117:0/1521701603 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f91b400f450 con 0x7f91b8103160 2024-09-21T05:54:31.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.101+0000 7f91adffb640 1 --2- 172.21.15.117:0/1521701603 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f91a00b6590 0x7f91a00b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:31.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.101+0000 7f91affff640 1 --2- 172.21.15.117:0/1521701603 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f91a00b6590 0x7f91a00b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:31.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.101+0000 7f91adffb640 1 -- 172.21.15.117:0/1521701603 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7342+0+0 (secure 0 0 0) 0x7f91b40e7430 con 0x7f91b8103160 2024-09-21T05:54:31.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.102+0000 7f91affff640 1 --2- 172.21.15.117:0/1521701603 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f91a00b6590 0x7f91a00b8aa0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f91a8005fd0 tx=0x7f91a8005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:54:31.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.104+0000 7f91adffb640 1 -- 172.21.15.117:0/1521701603 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f91b4014030 con 0x7f91b8103160 2024-09-21T05:54:31.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.320+0000 7f91bf141640 1 -- 172.21.15.117:0/1521701603 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91b819a4a0 con 0x7f91b8103160 2024-09-21T05:54:31.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.321+0000 7f91adffb640 1 -- 172.21.15.117:0/1521701603 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f91b40ae120 con 0x7f91b8103160 2024-09-21T05:54:31.323 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:54:31.323 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"btime":"2024-09-21T05:46:47:021637+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-21T05:54:31.323 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 1 2024-09-21T05:54:31.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.323+0000 7f91bf141640 1 -- 172.21.15.117:0/1521701603 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f91a00b6590 msgr2=0x7f91a00b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:31.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.323+0000 7f91bf141640 1 --2- 172.21.15.117:0/1521701603 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f91a00b6590 0x7f91a00b8aa0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f91a8005fd0 tx=0x7f91a8005850 comp rx=0 tx=0).stop 2024-09-21T05:54:31.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.323+0000 7f91bf141640 1 -- 172.21.15.117:0/1521701603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91b8103160 msgr2=0x7f91b8199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:31.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.323+0000 7f91bf141640 1 --2- 172.21.15.117:0/1521701603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91b8103160 0x7f91b8199370 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f91b400c920 tx=0x7f91b400cde0 comp rx=0 tx=0).stop 2024-09-21T05:54:31.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.323+0000 7f91bf141640 1 -- 172.21.15.117:0/1521701603 shutdown_connections 2024-09-21T05:54:31.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.323+0000 7f91bf141640 1 --2- 172.21.15.117:0/1521701603 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f91a00b6590 0x7f91a00b8aa0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:31.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.323+0000 7f91bf141640 1 --2- 172.21.15.117:0/1521701603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f91b8105ac0 0x7f91b819ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:31.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.323+0000 7f91bf141640 1 --2- 172.21.15.117:0/1521701603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91b8105170 0x7f91b8199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:31.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.323+0000 7f91bf141640 1 --2- 172.21.15.117:0/1521701603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91b8103160 0x7f91b8199370 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:31.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.323+0000 7f91bf141640 1 -- 172.21.15.117:0/1521701603 >> 172.21.15.117:0/1521701603 conn(0x7f91b8077c60 msgr2=0x7f91b81063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:54:31.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.323+0000 7f91bf141640 1 -- 172.21.15.117:0/1521701603 shutdown_connections 2024-09-21T05:54:31.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:31.323+0000 7f91bf141640 1 -- 172.21.15.117:0/1521701603 wait complete. 2024-09-21T05:54:31.612 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 21 05:54:31 smithi192 systemd[1]: Starting Ceph mds.l for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T05:54:31.862 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:31 smithi192 ceph-mon[34700]: Deploying daemon mds.l on smithi192 2024-09-21T05:54:31.862 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:31 smithi192 ceph-mon[34700]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:31.862 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:31 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/1521701603' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:54:31.862 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 21 05:54:31 smithi192 podman[63453]: 2024-09-21 05:54:31.632114785 +0000 UTC m=+0.016092110 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T05:54:31.862 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 21 05:54:31 smithi192 podman[63453]: 2024-09-21 05:54:31.73824423 +0000 UTC m=+0.122221588 container create c9511689c622fce7ed2c07507624afacea5e9389c93eb6e7825fddacc98efcbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-l, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-21T05:54:31.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:31 smithi138 ceph-mon[34391]: Deploying daemon mds.l on smithi192 2024-09-21T05:54:31.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:31 smithi138 ceph-mon[34391]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:31.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:31 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/1521701603' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:54:32.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:31 smithi117 ceph-mon[30947]: Deploying daemon mds.l on smithi192 2024-09-21T05:54:32.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:31 smithi117 ceph-mon[30947]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:32.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:31 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/1521701603' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:54:32.480 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 21 05:54:32 smithi192 podman[63453]: 2024-09-21 05:54:32.008290706 +0000 UTC m=+0.392268019 container init c9511689c622fce7ed2c07507624afacea5e9389c93eb6e7825fddacc98efcbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-l, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0) 2024-09-21T05:54:32.480 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 21 05:54:32 smithi192 podman[63453]: 2024-09-21 05:54:32.011052308 +0000 UTC m=+0.395029624 container start c9511689c622fce7ed2c07507624afacea5e9389c93eb6e7825fddacc98efcbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-l, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-21T05:54:32.480 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 21 05:54:32 smithi192 ceph-mds[63472]: -- 172.21.15.192:0/1156416194 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5651df895860 con 0x5651e3a3da80 2024-09-21T05:54:32.480 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 21 05:54:32 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-l[63468]: starting mds.l at 2024-09-21T05:54:32.480 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 21 05:54:32 smithi192 bash[63453]: c9511689c622fce7ed2c07507624afacea5e9389c93eb6e7825fddacc98efcbf 2024-09-21T05:54:32.481 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 21 05:54:32 smithi192 systemd[1]: Started Ceph mds.l for ac8581a2-77dc-11ef-baf5-efdc52797490. 2024-09-21T05:54:33.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:33 smithi138 ceph-mon[34391]: mds.? [v2:172.21.15.192:6832/2224910879,v1:172.21.15.192:6833/2224910879] up:boot 2024-09-21T05:54:33.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:33 smithi138 ceph-mon[34391]: fsmap 1 up:standby 2024-09-21T05:54:33.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:33 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T05:54:33.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:33 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:33.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:33 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:33.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:33 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:33.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:33 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:54:33.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:33 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:54:33.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:33 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:33.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:33 smithi138 ceph-mon[34391]: Deploying daemon mds.k on smithi138 2024-09-21T05:54:33.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:33 smithi138 ceph-mon[34391]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:33.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:33 smithi192 ceph-mon[34700]: mds.? [v2:172.21.15.192:6832/2224910879,v1:172.21.15.192:6833/2224910879] up:boot 2024-09-21T05:54:33.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:33 smithi192 ceph-mon[34700]: fsmap 1 up:standby 2024-09-21T05:54:33.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:33 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T05:54:33.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:33 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:33.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:33 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:33.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:33 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:33.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:33 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:54:33.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:33 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:54:33.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:33 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:33.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:33 smithi192 ceph-mon[34700]: Deploying daemon mds.k on smithi138 2024-09-21T05:54:33.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:33 smithi192 ceph-mon[34700]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:33.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:33 smithi117 ceph-mon[30947]: mds.? [v2:172.21.15.192:6832/2224910879,v1:172.21.15.192:6833/2224910879] up:boot 2024-09-21T05:54:33.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:33 smithi117 ceph-mon[30947]: fsmap 1 up:standby 2024-09-21T05:54:33.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:33 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T05:54:33.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:33 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:33.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:33 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:33.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:33 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:33.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:33 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:54:33.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:33 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:54:33.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:33 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:33.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:33 smithi117 ceph-mon[30947]: Deploying daemon mds.k on smithi138 2024-09-21T05:54:33.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:33 smithi117 ceph-mon[30947]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:34.062 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T05:54:34.317 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:54:34.366 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 21 05:54:34 smithi138 systemd[1]: Starting Ceph mds.k for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T05:54:34.673 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 21 05:54:34 smithi138 podman[62958]: 2024-09-21 05:54:34.279946234 +0000 UTC m=+0.015754551 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T05:54:34.673 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 21 05:54:34 smithi138 podman[62958]: 2024-09-21 05:54:34.410788803 +0000 UTC m=+0.146597149 container create 0a96226c7fb44f0458ed8f2ebf2c40c009be5d87d1ff3fea75525f79e2b2f6c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-k, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True) 2024-09-21T05:54:34.673 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 21 05:54:34 smithi138 podman[62958]: 2024-09-21 05:54:34.671966668 +0000 UTC m=+0.407774973 container init 0a96226c7fb44f0458ed8f2ebf2c40c009be5d87d1ff3fea75525f79e2b2f6c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-k, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-21T05:54:34.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.808+0000 7f96657bf640 1 -- 172.21.15.117:0/2718668999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9660108d40 msgr2=0x7f966010f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:34.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.808+0000 7f96657bf640 1 --2- 172.21.15.117:0/2718668999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9660108d40 0x7f966010f570 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9658009900 tx=0x7f965802a410 comp rx=0 tx=0).stop 2024-09-21T05:54:34.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.808+0000 7f96657bf640 1 -- 172.21.15.117:0/2718668999 shutdown_connections 2024-09-21T05:54:34.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.808+0000 7f96657bf640 1 --2- 172.21.15.117:0/2718668999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9660108d40 0x7f966010f570 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:34.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.808+0000 7f96657bf640 1 --2- 172.21.15.117:0/2718668999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96601083f0 0x7f9660108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:34.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.808+0000 7f96657bf640 1 --2- 172.21.15.117:0/2718668999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96601031c0 0x7f96601035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:34.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.808+0000 7f96657bf640 1 -- 172.21.15.117:0/2718668999 >> 172.21.15.117:0/2718668999 conn(0x7f9660079e10 msgr2=0x7f966007a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:54:34.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.808+0000 7f96657bf640 1 -- 172.21.15.117:0/2718668999 shutdown_connections 2024-09-21T05:54:34.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.808+0000 7f96657bf640 1 -- 172.21.15.117:0/2718668999 wait complete. 2024-09-21T05:54:34.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.809+0000 7f96657bf640 1 Processor -- start 2024-09-21T05:54:34.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.809+0000 7f96657bf640 1 -- start start 2024-09-21T05:54:34.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.809+0000 7f96657bf640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96601031c0 0x7f966019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:34.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.809+0000 7f96657bf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96601083f0 0x7f966019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:34.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.809+0000 7f96657bf640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9660108d40 0x7f966019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:34.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.809+0000 7f96657bf640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9660112a00 con 0x7f96601083f0 2024-09-21T05:54:34.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.809+0000 7f96657bf640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9660112880 con 0x7f96601031c0 2024-09-21T05:54:34.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.809+0000 7f96657bf640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9660112b80 con 0x7f9660108d40 2024-09-21T05:54:34.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.809+0000 7f965e7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96601083f0 0x7f966019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:34.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.809+0000 7f965e7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96601083f0 0x7f966019aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59334/0 (socket says 172.21.15.117:59334) 2024-09-21T05:54:34.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.809+0000 7f965e7fc640 1 -- 172.21.15.117:0/3573901352 learned_addr learned my addr 172.21.15.117:0/3573901352 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:54:34.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.809+0000 7f965effd640 1 --2- 172.21.15.117:0/3573901352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96601031c0 0x7f966019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:34.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.809+0000 7f965f7fe640 1 --2- 172.21.15.117:0/3573901352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9660108d40 0x7f966019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:34.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.809+0000 7f965e7fc640 1 -- 172.21.15.117:0/3573901352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9660108d40 msgr2=0x7f966019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:34.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.809+0000 7f965e7fc640 1 --2- 172.21.15.117:0/3573901352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9660108d40 0x7f966019fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:34.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.809+0000 7f965e7fc640 1 -- 172.21.15.117:0/3573901352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96601031c0 msgr2=0x7f966019a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:34.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.809+0000 7f965e7fc640 1 --2- 172.21.15.117:0/3573901352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96601031c0 0x7f966019a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:34.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.809+0000 7f965e7fc640 1 -- 172.21.15.117:0/3573901352 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96601a03d0 con 0x7f96601083f0 2024-09-21T05:54:34.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.810+0000 7f965e7fc640 1 --2- 172.21.15.117:0/3573901352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96601083f0 0x7f966019aa00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f964800d950 tx=0x7f964800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:54:34.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.810+0000 7f9657fff640 1 -- 172.21.15.117:0/3573901352 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9648017070 con 0x7f96601083f0 2024-09-21T05:54:34.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.810+0000 7f9657fff640 1 -- 172.21.15.117:0/3573901352 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9648009c30 con 0x7f96601083f0 2024-09-21T05:54:34.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.810+0000 7f96657bf640 1 -- 172.21.15.117:0/3573901352 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96601a06c0 con 0x7f96601083f0 2024-09-21T05:54:34.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.810+0000 7f9657fff640 1 -- 172.21.15.117:0/3573901352 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96480047a0 con 0x7f96601083f0 2024-09-21T05:54:34.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.810+0000 7f96657bf640 1 -- 172.21.15.117:0/3573901352 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96601a6e20 con 0x7f96601083f0 2024-09-21T05:54:34.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.811+0000 7f96657bf640 1 -- 172.21.15.117:0/3573901352 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96601035d0 con 0x7f96601083f0 2024-09-21T05:54:34.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.812+0000 7f9657fff640 1 -- 172.21.15.117:0/3573901352 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f9648004020 con 0x7f96601083f0 2024-09-21T05:54:34.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.813+0000 7f9657fff640 1 --2- 172.21.15.117:0/3573901352 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f964c0b6560 0x7f964c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:34.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.813+0000 7f965effd640 1 --2- 172.21.15.117:0/3573901352 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f964c0b6560 0x7f964c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:34.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.813+0000 7f9657fff640 1 -- 172.21.15.117:0/3573901352 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7342+0+0 (secure 0 0 0) 0x7f96480e6c70 con 0x7f96601083f0 2024-09-21T05:54:34.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.813+0000 7f965effd640 1 --2- 172.21.15.117:0/3573901352 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f964c0b6560 0x7f964c0b8a70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9650005fd0 tx=0x7f9650009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:54:34.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:34.817+0000 7f9657fff640 1 -- 172.21.15.117:0/3573901352 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9648020020 con 0x7f96601083f0 2024-09-21T05:54:35.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:35.042+0000 7f96657bf640 1 -- 172.21.15.117:0/3573901352 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f966002e640 con 0x7f96601083f0 2024-09-21T05:54:35.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:35.043+0000 7f9657fff640 1 -- 172.21.15.117:0/3573901352 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f96480ad8d0 con 0x7f96601083f0 2024-09-21T05:54:35.045 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:54:35.045 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":2,"btime":"2024-09-21T05:54:32:049387+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24586,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/2224910879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2224910879},{"type":"v1","addr":"172.21.15.192:6833","nonce":2224910879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T05:54:35.045 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 2 2024-09-21T05:54:35.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:35.046+0000 7f96657bf640 1 -- 172.21.15.117:0/3573901352 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f964c0b6560 msgr2=0x7f964c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:35.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:35.046+0000 7f96657bf640 1 --2- 172.21.15.117:0/3573901352 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f964c0b6560 0x7f964c0b8a70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9650005fd0 tx=0x7f9650009500 comp rx=0 tx=0).stop 2024-09-21T05:54:35.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:35.046+0000 7f96657bf640 1 -- 172.21.15.117:0/3573901352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96601083f0 msgr2=0x7f966019aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:35.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:35.046+0000 7f96657bf640 1 --2- 172.21.15.117:0/3573901352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96601083f0 0x7f966019aa00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f964800d950 tx=0x7f964800de10 comp rx=0 tx=0).stop 2024-09-21T05:54:35.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:35.047+0000 7f96657bf640 1 -- 172.21.15.117:0/3573901352 shutdown_connections 2024-09-21T05:54:35.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:35.047+0000 7f96657bf640 1 --2- 172.21.15.117:0/3573901352 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f964c0b6560 0x7f964c0b8a70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:35.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:35.047+0000 7f96657bf640 1 --2- 172.21.15.117:0/3573901352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96601083f0 0x7f966019aa00 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:35.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:35.047+0000 7f96657bf640 1 --2- 172.21.15.117:0/3573901352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9660108d40 0x7f966019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:35.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:35.047+0000 7f96657bf640 1 --2- 172.21.15.117:0/3573901352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96601031c0 0x7f966019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:35.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:35.047+0000 7f96657bf640 1 -- 172.21.15.117:0/3573901352 >> 172.21.15.117:0/3573901352 conn(0x7f9660079e10 msgr2=0x7f9660101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:54:35.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:35.047+0000 7f96657bf640 1 -- 172.21.15.117:0/3573901352 shutdown_connections 2024-09-21T05:54:35.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:35.047+0000 7f96657bf640 1 -- 172.21.15.117:0/3573901352 wait complete. 2024-09-21T05:54:35.115 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 21 05:54:34 smithi138 podman[62958]: 2024-09-21 05:54:34.67480287 +0000 UTC m=+0.410611175 container start 0a96226c7fb44f0458ed8f2ebf2c40c009be5d87d1ff3fea75525f79e2b2f6c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-k, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0) 2024-09-21T05:54:35.115 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 21 05:54:34 smithi138 ceph-mds[62977]: -- 172.21.15.138:0/660600653 <== mon.2 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5585c1307860 con 0x5585c54af600 2024-09-21T05:54:35.115 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 21 05:54:34 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-k[62973]: starting mds.k at 2024-09-21T05:54:35.115 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 21 05:54:34 smithi138 bash[62958]: 0a96226c7fb44f0458ed8f2ebf2c40c009be5d87d1ff3fea75525f79e2b2f6c3 2024-09-21T05:54:35.115 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 21 05:54:34 smithi138 systemd[1]: Started Ceph mds.k for ac8581a2-77dc-11ef-baf5-efdc52797490. 2024-09-21T05:54:35.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:35 smithi117 ceph-mon[30947]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:35.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:35.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:35.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:35.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:54:35.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:54:35.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:35.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:35 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3573901352' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:54:35.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:35 smithi138 ceph-mon[34391]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:35.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:35.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:35.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:35.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:54:35.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:54:35.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:35.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:35 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3573901352' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:54:35.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:35 smithi192 ceph-mon[34700]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:35.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:35.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:35.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:35.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:54:35.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:54:35.981 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:35.981 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:35 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3573901352' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:54:36.842 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:36 smithi138 ceph-mon[34391]: Deploying daemon mds.j on smithi117 2024-09-21T05:54:36.842 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:36 smithi138 ceph-mon[34391]: mds.? [v2:172.21.15.138:6832/3725629862,v1:172.21.15.138:6833/3725629862] up:boot 2024-09-21T05:54:36.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:36 smithi138 ceph-mon[34391]: fsmap 2 up:standby 2024-09-21T05:54:36.843 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:36 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T05:54:36.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:36 smithi192 ceph-mon[34700]: Deploying daemon mds.j on smithi117 2024-09-21T05:54:36.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:36 smithi192 ceph-mon[34700]: mds.? [v2:172.21.15.138:6832/3725629862,v1:172.21.15.138:6833/3725629862] up:boot 2024-09-21T05:54:36.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:36 smithi192 ceph-mon[34700]: fsmap 2 up:standby 2024-09-21T05:54:36.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:36 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T05:54:37.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:36 smithi117 ceph-mon[30947]: Deploying daemon mds.j on smithi117 2024-09-21T05:54:37.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:36 smithi117 ceph-mon[30947]: mds.? [v2:172.21.15.138:6832/3725629862,v1:172.21.15.138:6833/3725629862] up:boot 2024-09-21T05:54:37.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:36 smithi117 ceph-mon[30947]: fsmap 2 up:standby 2024-09-21T05:54:37.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:36 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T05:54:37.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:37 smithi117 ceph-mon[30947]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:37.777 INFO:journalctl@ceph.mds.j.smithi117.stdout:Sep 21 05:54:37 smithi117 systemd[1]: Starting Ceph mds.j for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T05:54:37.863 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T05:54:37.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:37 smithi138 ceph-mon[34391]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:37.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:37 smithi192 ceph-mon[34700]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:38.082 INFO:journalctl@ceph.mds.j.smithi117.stdout:Sep 21 05:54:37 smithi117 podman[71831]: 2024-09-21 05:54:37.712163887 +0000 UTC m=+0.015656193 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T05:54:38.083 INFO:journalctl@ceph.mds.j.smithi117.stdout:Sep 21 05:54:37 smithi117 podman[71831]: 2024-09-21 05:54:37.844550038 +0000 UTC m=+0.148042377 container create 43bc510c58d61b55515c7363b4a3eb9e2ecab073a968cc37587628ebca25fba4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-21T05:54:38.083 INFO:journalctl@ceph.mds.j.smithi117.stdout:Sep 21 05:54:38 smithi117 podman[71831]: 2024-09-21 05:54:38.081028619 +0000 UTC m=+0.384520926 container init 43bc510c58d61b55515c7363b4a3eb9e2ecab073a968cc37587628ebca25fba4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-j, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-21T05:54:38.254 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:54:38.375 INFO:journalctl@ceph.mds.j.smithi117.stdout:Sep 21 05:54:38 smithi117 podman[71831]: 2024-09-21 05:54:38.084267263 +0000 UTC m=+0.387759555 container start 43bc510c58d61b55515c7363b4a3eb9e2ecab073a968cc37587628ebca25fba4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-j, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-21T05:54:38.376 INFO:journalctl@ceph.mds.j.smithi117.stdout:Sep 21 05:54:38 smithi117 ceph-mds[71880]: -- 172.21.15.117:0/601299568 <== mon.0 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b9b4075860 con 0x55b9b821d600 2024-09-21T05:54:38.376 INFO:journalctl@ceph.mds.j.smithi117.stdout:Sep 21 05:54:38 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-j[71864]: starting mds.j at 2024-09-21T05:54:38.376 INFO:journalctl@ceph.mds.j.smithi117.stdout:Sep 21 05:54:38 smithi117 bash[71831]: 43bc510c58d61b55515c7363b4a3eb9e2ecab073a968cc37587628ebca25fba4 2024-09-21T05:54:38.376 INFO:journalctl@ceph.mds.j.smithi117.stdout:Sep 21 05:54:38 smithi117 systemd[1]: Started Ceph mds.j for ac8581a2-77dc-11ef-baf5-efdc52797490. 2024-09-21T05:54:38.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.741+0000 7f1525fac640 1 -- 172.21.15.117:0/2281081054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15201083a0 msgr2=0x7f15201087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:38.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.741+0000 7f1525fac640 1 --2- 172.21.15.117:0/2281081054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15201083a0 0x7f15201087b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1514009ca0 tx=0x7f151402a8c0 comp rx=0 tx=0).stop 2024-09-21T05:54:38.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.741+0000 7f1525fac640 1 -- 172.21.15.117:0/2281081054 shutdown_connections 2024-09-21T05:54:38.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.741+0000 7f1525fac640 1 --2- 172.21.15.117:0/2281081054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15201083a0 0x7f15201087b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:38.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.741+0000 7f1525fac640 1 --2- 172.21.15.117:0/2281081054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1520108cf0 0x7f152010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:38.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.741+0000 7f1525fac640 1 --2- 172.21.15.117:0/2281081054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1520102a60 0x7f1520102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:38.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.741+0000 7f1525fac640 1 -- 172.21.15.117:0/2281081054 >> 172.21.15.117:0/2281081054 conn(0x7f1520079720 msgr2=0x7f1520079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:54:38.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.741+0000 7f1525fac640 1 -- 172.21.15.117:0/2281081054 shutdown_connections 2024-09-21T05:54:38.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.741+0000 7f1525fac640 1 -- 172.21.15.117:0/2281081054 wait complete. 2024-09-21T05:54:38.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.742+0000 7f1525fac640 1 Processor -- start 2024-09-21T05:54:38.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.742+0000 7f1525fac640 1 -- start start 2024-09-21T05:54:38.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.743+0000 7f1525fac640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1520102a60 0x7f152019b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:38.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.743+0000 7f1525fac640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15201083a0 0x7f152019b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:38.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.743+0000 7f1525fac640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1520108cf0 0x7f152019bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:38.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.743+0000 7f1525fac640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1520112960 con 0x7f1520102a60 2024-09-21T05:54:38.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.743+0000 7f1525fac640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f15201127e0 con 0x7f1520108cf0 2024-09-21T05:54:38.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.743+0000 7f1525fac640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1520112ae0 con 0x7f15201083a0 2024-09-21T05:54:38.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.743+0000 7f151f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1520102a60 0x7f152019b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:38.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.743+0000 7f151f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1520102a60 0x7f152019b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59366/0 (socket says 172.21.15.117:59366) 2024-09-21T05:54:38.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.743+0000 7f151f7fe640 1 -- 172.21.15.117:0/2416234408 learned_addr learned my addr 172.21.15.117:0/2416234408 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:54:38.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.743+0000 7f151ffff640 1 --2- 172.21.15.117:0/2416234408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1520108cf0 0x7f152019bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:38.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.743+0000 7f151effd640 1 --2- 172.21.15.117:0/2416234408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15201083a0 0x7f152019b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:38.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.743+0000 7f151f7fe640 1 -- 172.21.15.117:0/2416234408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15201083a0 msgr2=0x7f152019b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:38.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.743+0000 7f151f7fe640 1 --2- 172.21.15.117:0/2416234408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15201083a0 0x7f152019b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:38.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.743+0000 7f151f7fe640 1 -- 172.21.15.117:0/2416234408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1520108cf0 msgr2=0x7f152019bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:38.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.743+0000 7f151f7fe640 1 --2- 172.21.15.117:0/2416234408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1520108cf0 0x7f152019bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:38.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.743+0000 7f151f7fe640 1 -- 172.21.15.117:0/2416234408 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15201a6500 con 0x7f1520102a60 2024-09-21T05:54:38.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.743+0000 7f151ffff640 1 --2- 172.21.15.117:0/2416234408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1520108cf0 0x7f152019bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:54:38.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.743+0000 7f151f7fe640 1 --2- 172.21.15.117:0/2416234408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1520102a60 0x7f152019b270 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f151000d770 tx=0x7f151000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:54:38.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.743+0000 7f151cff9640 1 -- 172.21.15.117:0/2416234408 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1510017070 con 0x7f1520102a60 2024-09-21T05:54:38.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.743+0000 7f1525fac640 1 -- 172.21.15.117:0/2416234408 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15201a67f0 con 0x7f1520102a60 2024-09-21T05:54:38.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.743+0000 7f151cff9640 1 -- 172.21.15.117:0/2416234408 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15100054d0 con 0x7f1520102a60 2024-09-21T05:54:38.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.743+0000 7f151cff9640 1 -- 172.21.15.117:0/2416234408 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1510004c30 con 0x7f1520102a60 2024-09-21T05:54:38.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.743+0000 7f1525fac640 1 -- 172.21.15.117:0/2416234408 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15201a6d30 con 0x7f1520102a60 2024-09-21T05:54:38.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.745+0000 7f1525fac640 1 -- 172.21.15.117:0/2416234408 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1520103ce0 con 0x7f1520102a60 2024-09-21T05:54:38.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.746+0000 7f151cff9640 1 -- 172.21.15.117:0/2416234408 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f1510004dd0 con 0x7f1520102a60 2024-09-21T05:54:38.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.747+0000 7f151cff9640 1 --2- 172.21.15.117:0/2416234408 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f15080b6590 0x7f15080b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:38.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.747+0000 7f151effd640 1 --2- 172.21.15.117:0/2416234408 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f15080b6590 0x7f15080b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:38.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.747+0000 7f151cff9640 1 -- 172.21.15.117:0/2416234408 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7342+0+0 (secure 0 0 0) 0x7f15100e6b30 con 0x7f1520102a60 2024-09-21T05:54:38.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.747+0000 7f151effd640 1 --2- 172.21.15.117:0/2416234408 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f15080b6590 0x7f15080b8aa0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1520107960 tx=0x7f1514009360 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:54:38.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.751+0000 7f151cff9640 1 -- 172.21.15.117:0/2416234408 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f15100ad790 con 0x7f1520102a60 2024-09-21T05:54:38.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.974+0000 7f1525fac640 1 -- 172.21.15.117:0/2416234408 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f152002e640 con 0x7f1520102a60 2024-09-21T05:54:38.976 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.974+0000 7f151cff9640 1 -- 172.21.15.117:0/2416234408 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3094 (secure 0 0 0) 0x7f15100e4e20 con 0x7f1520102a60 2024-09-21T05:54:38.976 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:54:38.977 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":4,"btime":"2024-09-21T05:54:38:559831+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3227057504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3227057504},{"type":"v1","addr":"172.21.15.117:6835","nonce":3227057504}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24545,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/3725629862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3725629862},{"type":"v1","addr":"172.21.15.138:6833","nonce":3725629862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24586,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/2224910879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2224910879},{"type":"v1","addr":"172.21.15.192:6833","nonce":2224910879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T05:54:38.977 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 4 2024-09-21T05:54:38.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.976+0000 7f1525fac640 1 -- 172.21.15.117:0/2416234408 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f15080b6590 msgr2=0x7f15080b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:38.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.976+0000 7f1525fac640 1 --2- 172.21.15.117:0/2416234408 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f15080b6590 0x7f15080b8aa0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1520107960 tx=0x7f1514009360 comp rx=0 tx=0).stop 2024-09-21T05:54:38.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.976+0000 7f1525fac640 1 -- 172.21.15.117:0/2416234408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1520102a60 msgr2=0x7f152019b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:38.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.976+0000 7f1525fac640 1 --2- 172.21.15.117:0/2416234408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1520102a60 0x7f152019b270 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f151000d770 tx=0x7f151000dc30 comp rx=0 tx=0).stop 2024-09-21T05:54:38.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.976+0000 7f1525fac640 1 -- 172.21.15.117:0/2416234408 shutdown_connections 2024-09-21T05:54:38.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.976+0000 7f1525fac640 1 --2- 172.21.15.117:0/2416234408 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f15080b6590 0x7f15080b8aa0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:38.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.976+0000 7f1525fac640 1 --2- 172.21.15.117:0/2416234408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1520108cf0 0x7f152019bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:38.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.976+0000 7f1525fac640 1 --2- 172.21.15.117:0/2416234408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15201083a0 0x7f152019b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:38.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.976+0000 7f1525fac640 1 --2- 172.21.15.117:0/2416234408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1520102a60 0x7f152019b270 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:38.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.977+0000 7f1525fac640 1 -- 172.21.15.117:0/2416234408 >> 172.21.15.117:0/2416234408 conn(0x7f1520079720 msgr2=0x7f1520104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:54:38.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.977+0000 7f1525fac640 1 -- 172.21.15.117:0/2416234408 shutdown_connections 2024-09-21T05:54:38.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:38.977+0000 7f1525fac640 1 -- 172.21.15.117:0/2416234408 wait complete. 2024-09-21T05:54:39.492 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:39 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:39.492 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:39 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:39.492 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:39 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:39.492 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:39 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:54:39.492 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:39 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:54:39.492 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:39 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:39.492 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:39 smithi117 ceph-mon[30947]: Deploying daemon mds.d on smithi117 2024-09-21T05:54:39.492 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:39 smithi117 ceph-mon[30947]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:39.492 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:39 smithi117 ceph-mon[30947]: mds.? [v2:172.21.15.117:6834/3227057504,v1:172.21.15.117:6835/3227057504] up:boot 2024-09-21T05:54:39.493 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:39 smithi117 ceph-mon[30947]: fsmap 3 up:standby 2024-09-21T05:54:39.493 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:39 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T05:54:39.493 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:39 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2416234408' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:54:39.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:39 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:39.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:39 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:39.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:39 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:39.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:39 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:54:39.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:39 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:54:39.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:39 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:39.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:39 smithi138 ceph-mon[34391]: Deploying daemon mds.d on smithi117 2024-09-21T05:54:39.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:39 smithi138 ceph-mon[34391]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:39.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:39 smithi138 ceph-mon[34391]: mds.? [v2:172.21.15.117:6834/3227057504,v1:172.21.15.117:6835/3227057504] up:boot 2024-09-21T05:54:39.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:39 smithi138 ceph-mon[34391]: fsmap 3 up:standby 2024-09-21T05:54:39.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:39 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T05:54:39.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:39 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2416234408' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:54:39.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:39 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:39.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:39 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:39.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:39 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:39.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:39 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:54:39.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:39 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:54:39.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:39 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:39.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:39 smithi192 ceph-mon[34700]: Deploying daemon mds.d on smithi117 2024-09-21T05:54:39.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:39 smithi192 ceph-mon[34700]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:39.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:39 smithi192 ceph-mon[34700]: mds.? [v2:172.21.15.117:6834/3227057504,v1:172.21.15.117:6835/3227057504] up:boot 2024-09-21T05:54:39.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:39 smithi192 ceph-mon[34700]: fsmap 3 up:standby 2024-09-21T05:54:39.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:39 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T05:54:39.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:39 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2416234408' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:54:40.926 INFO:journalctl@ceph.mds.d.smithi117.stdout:Sep 21 05:54:40 smithi117 systemd[1]: Starting Ceph mds.d for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T05:54:41.193 INFO:journalctl@ceph.mds.d.smithi117.stdout:Sep 21 05:54:41 smithi117 podman[72211]: 2024-09-21 05:54:41.05908409 +0000 UTC m=+0.015860704 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T05:54:41.193 INFO:journalctl@ceph.mds.d.smithi117.stdout:Sep 21 05:54:41 smithi117 podman[72211]: 2024-09-21 05:54:41.192096939 +0000 UTC m=+0.148873545 container create a6a2bad078691008151e7720d9bb2b1869e819633d7a7d1d9699b4cc7557ca5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-d, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-21T05:54:41.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:41 smithi117 ceph-mon[30947]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:41.777 INFO:journalctl@ceph.mds.d.smithi117.stdout:Sep 21 05:54:41 smithi117 podman[72211]: 2024-09-21 05:54:41.445130705 +0000 UTC m=+0.401907315 container init a6a2bad078691008151e7720d9bb2b1869e819633d7a7d1d9699b4cc7557ca5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-d, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-21T05:54:41.777 INFO:journalctl@ceph.mds.d.smithi117.stdout:Sep 21 05:54:41 smithi117 podman[72211]: 2024-09-21 05:54:41.448012307 +0000 UTC m=+0.404788921 container start a6a2bad078691008151e7720d9bb2b1869e819633d7a7d1d9699b4cc7557ca5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-21T05:54:41.777 INFO:journalctl@ceph.mds.d.smithi117.stdout:Sep 21 05:54:41 smithi117 ceph-mds[72230]: -- 172.21.15.117:0/2481913504 <== mon.0 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55875ce73860 con 0x55876101b600 2024-09-21T05:54:41.777 INFO:journalctl@ceph.mds.d.smithi117.stdout:Sep 21 05:54:41 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-d[72226]: starting mds.d at 2024-09-21T05:54:41.777 INFO:journalctl@ceph.mds.d.smithi117.stdout:Sep 21 05:54:41 smithi117 bash[72211]: a6a2bad078691008151e7720d9bb2b1869e819633d7a7d1d9699b4cc7557ca5e 2024-09-21T05:54:41.777 INFO:journalctl@ceph.mds.d.smithi117.stdout:Sep 21 05:54:41 smithi117 systemd[1]: Started Ceph mds.d for ac8581a2-77dc-11ef-baf5-efdc52797490. 2024-09-21T05:54:41.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:41 smithi138 ceph-mon[34391]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:41.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:41 smithi192 ceph-mon[34700]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:42.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:42 smithi138 ceph-mon[34391]: mds.? [v2:172.21.15.117:6836/3303829183,v1:172.21.15.117:6837/3303829183] up:boot 2024-09-21T05:54:42.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:42 smithi138 ceph-mon[34391]: fsmap 4 up:standby 2024-09-21T05:54:42.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:42 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T05:54:42.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:42 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:42.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:42 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:42.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:42 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:42.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:42 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:54:42.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:42 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:54:42.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:42 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:42.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:42 smithi138 ceph-mon[34391]: Deploying daemon mds.e on smithi138 2024-09-21T05:54:42.955 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T05:54:42.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:42 smithi192 ceph-mon[34700]: mds.? [v2:172.21.15.117:6836/3303829183,v1:172.21.15.117:6837/3303829183] up:boot 2024-09-21T05:54:42.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:42 smithi192 ceph-mon[34700]: fsmap 4 up:standby 2024-09-21T05:54:42.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:42 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T05:54:42.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:42 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:42.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:42 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:42.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:42 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:42.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:42 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:54:42.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:42 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:54:42.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:42 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:42.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:42 smithi192 ceph-mon[34700]: Deploying daemon mds.e on smithi138 2024-09-21T05:54:42.984 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:42 smithi117 ceph-mon[30947]: mds.? [v2:172.21.15.117:6836/3303829183,v1:172.21.15.117:6837/3303829183] up:boot 2024-09-21T05:54:42.984 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:42 smithi117 ceph-mon[30947]: fsmap 4 up:standby 2024-09-21T05:54:42.984 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:42 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T05:54:42.985 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:42 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:42.985 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:42 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:42.985 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:42 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:42.985 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:42 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:54:42.985 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:42 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:54:42.985 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:42 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:42.985 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:42 smithi117 ceph-mon[30947]: Deploying daemon mds.e on smithi138 2024-09-21T05:54:43.218 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:54:43.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.705+0000 7f2b52fa4640 1 -- 172.21.15.117:0/1366364322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b4c108d00 msgr2=0x7f2b4c10f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:43.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.705+0000 7f2b52fa4640 1 --2- 172.21.15.117:0/1366364322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b4c108d00 0x7f2b4c10f530 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2b44009ca0 tx=0x7f2b4402a8c0 comp rx=0 tx=0).stop 2024-09-21T05:54:43.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.705+0000 7f2b52fa4640 1 -- 172.21.15.117:0/1366364322 shutdown_connections 2024-09-21T05:54:43.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.705+0000 7f2b52fa4640 1 --2- 172.21.15.117:0/1366364322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b4c108d00 0x7f2b4c10f530 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:43.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.705+0000 7f2b52fa4640 1 --2- 172.21.15.117:0/1366364322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b4c1083b0 0x7f2b4c1087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:43.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.705+0000 7f2b52fa4640 1 --2- 172.21.15.117:0/1366364322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b4c102a70 0x7f2b4c102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:43.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.705+0000 7f2b52fa4640 1 -- 172.21.15.117:0/1366364322 >> 172.21.15.117:0/1366364322 conn(0x7f2b4c079710 msgr2=0x7f2b4c079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:54:43.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.705+0000 7f2b52fa4640 1 -- 172.21.15.117:0/1366364322 shutdown_connections 2024-09-21T05:54:43.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.705+0000 7f2b52fa4640 1 -- 172.21.15.117:0/1366364322 wait complete. 2024-09-21T05:54:43.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.706+0000 7f2b52fa4640 1 Processor -- start 2024-09-21T05:54:43.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.706+0000 7f2b52fa4640 1 -- start start 2024-09-21T05:54:43.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.706+0000 7f2b52fa4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b4c102a70 0x7f2b4c19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:43.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.706+0000 7f2b52fa4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b4c1083b0 0x7f2b4c19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:43.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.706+0000 7f2b52fa4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b4c108d00 0x7f2b4c19fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:43.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.706+0000 7f2b52fa4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4c113b20 con 0x7f2b4c108d00 2024-09-21T05:54:43.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.706+0000 7f2b52fa4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4c1139a0 con 0x7f2b4c1083b0 2024-09-21T05:54:43.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.706+0000 7f2b52fa4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4c113ca0 con 0x7f2b4c102a70 2024-09-21T05:54:43.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.706+0000 7f2b51541640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b4c108d00 0x7f2b4c19fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:43.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.706+0000 7f2b51541640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b4c108d00 0x7f2b4c19fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59414/0 (socket says 172.21.15.117:59414) 2024-09-21T05:54:43.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.706+0000 7f2b51541640 1 -- 172.21.15.117:0/2727415954 learned_addr learned my addr 172.21.15.117:0/2727415954 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:54:43.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.706+0000 7f2b4bfff640 1 --2- 172.21.15.117:0/2727415954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b4c1083b0 0x7f2b4c19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:43.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.707+0000 7f2b50d40640 1 --2- 172.21.15.117:0/2727415954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b4c102a70 0x7f2b4c19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:43.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.707+0000 7f2b51541640 1 -- 172.21.15.117:0/2727415954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b4c102a70 msgr2=0x7f2b4c19a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:43.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.707+0000 7f2b51541640 1 --2- 172.21.15.117:0/2727415954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b4c102a70 0x7f2b4c19a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:43.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.707+0000 7f2b51541640 1 -- 172.21.15.117:0/2727415954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b4c1083b0 msgr2=0x7f2b4c19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:43.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.707+0000 7f2b51541640 1 --2- 172.21.15.117:0/2727415954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b4c1083b0 0x7f2b4c19aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:43.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.707+0000 7f2b51541640 1 -- 172.21.15.117:0/2727415954 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b4c1a04c0 con 0x7f2b4c108d00 2024-09-21T05:54:43.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.707+0000 7f2b51541640 1 --2- 172.21.15.117:0/2727415954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b4c108d00 0x7f2b4c19fd90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2b44005cc0 tx=0x7f2b4402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:54:43.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.707+0000 7f2b49ffb640 1 -- 172.21.15.117:0/2727415954 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b44003bf0 con 0x7f2b4c108d00 2024-09-21T05:54:43.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.707+0000 7f2b52fa4640 1 -- 172.21.15.117:0/2727415954 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b4c1a0750 con 0x7f2b4c108d00 2024-09-21T05:54:43.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.707+0000 7f2b49ffb640 1 -- 172.21.15.117:0/2727415954 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b44003d90 con 0x7f2b4c108d00 2024-09-21T05:54:43.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.707+0000 7f2b49ffb640 1 -- 172.21.15.117:0/2727415954 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4402fbc0 con 0x7f2b4c108d00 2024-09-21T05:54:43.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.707+0000 7f2b52fa4640 1 -- 172.21.15.117:0/2727415954 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b4c1a6f10 con 0x7f2b4c108d00 2024-09-21T05:54:43.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.708+0000 7f2b52fa4640 1 -- 172.21.15.117:0/2727415954 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b4c103cf0 con 0x7f2b4c108d00 2024-09-21T05:54:43.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.710+0000 7f2b49ffb640 1 -- 172.21.15.117:0/2727415954 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f2b4402fd60 con 0x7f2b4c108d00 2024-09-21T05:54:43.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.711+0000 7f2b49ffb640 1 --2- 172.21.15.117:0/2727415954 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2b340b6560 0x7f2b340b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:43.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.711+0000 7f2b50d40640 1 --2- 172.21.15.117:0/2727415954 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2b340b6560 0x7f2b340b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:43.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.711+0000 7f2b49ffb640 1 -- 172.21.15.117:0/2727415954 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7342+0+0 (secure 0 0 0) 0x7f2b44108700 con 0x7f2b4c108d00 2024-09-21T05:54:43.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.711+0000 7f2b50d40640 1 --2- 172.21.15.117:0/2727415954 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2b340b6560 0x7f2b340b8a70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c009890 tx=0x7f2b3c008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:54:43.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.714+0000 7f2b49ffb640 1 -- 172.21.15.117:0/2727415954 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2b440cf2c0 con 0x7f2b4c108d00 2024-09-21T05:54:43.859 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:43 smithi138 ceph-mon[34391]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:43.859 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 21 05:54:43 smithi138 systemd[1]: Starting Ceph mds.e for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T05:54:43.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.930+0000 7f2b52fa4640 1 -- 172.21.15.117:0/2727415954 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b4c19b680 con 0x7f2b4c108d00 2024-09-21T05:54:43.932 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:43 smithi117 ceph-mon[30947]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:43.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.931+0000 7f2b49ffb640 1 -- 172.21.15.117:0/2727415954 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3922 (secure 0 0 0) 0x7f2b4c19b680 con 0x7f2b4c108d00 2024-09-21T05:54:43.933 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:54:43.933 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":5,"btime":"2024-09-21T05:54:41:548714+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3227057504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3227057504},{"type":"v1","addr":"172.21.15.117:6835","nonce":3227057504}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3303829183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3303829183},{"type":"v1","addr":"172.21.15.117:6837","nonce":3303829183}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24545,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/3725629862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3725629862},{"type":"v1","addr":"172.21.15.138:6833","nonce":3725629862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24586,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/2224910879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2224910879},{"type":"v1","addr":"172.21.15.192:6833","nonce":2224910879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T05:54:43.933 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 5 2024-09-21T05:54:43.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.934+0000 7f2b52fa4640 1 -- 172.21.15.117:0/2727415954 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2b340b6560 msgr2=0x7f2b340b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:43.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.934+0000 7f2b52fa4640 1 --2- 172.21.15.117:0/2727415954 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2b340b6560 0x7f2b340b8a70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c009890 tx=0x7f2b3c008040 comp rx=0 tx=0).stop 2024-09-21T05:54:43.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.934+0000 7f2b52fa4640 1 -- 172.21.15.117:0/2727415954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b4c108d00 msgr2=0x7f2b4c19fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:43.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.934+0000 7f2b52fa4640 1 --2- 172.21.15.117:0/2727415954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b4c108d00 0x7f2b4c19fd90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2b44005cc0 tx=0x7f2b4402f380 comp rx=0 tx=0).stop 2024-09-21T05:54:43.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.934+0000 7f2b52fa4640 1 -- 172.21.15.117:0/2727415954 shutdown_connections 2024-09-21T05:54:43.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.934+0000 7f2b52fa4640 1 --2- 172.21.15.117:0/2727415954 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2b340b6560 0x7f2b340b8a70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:43.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.934+0000 7f2b52fa4640 1 --2- 172.21.15.117:0/2727415954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b4c108d00 0x7f2b4c19fd90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:43.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.934+0000 7f2b52fa4640 1 --2- 172.21.15.117:0/2727415954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b4c1083b0 0x7f2b4c19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:43.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.934+0000 7f2b52fa4640 1 --2- 172.21.15.117:0/2727415954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b4c102a70 0x7f2b4c19a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:43.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.934+0000 7f2b52fa4640 1 -- 172.21.15.117:0/2727415954 >> 172.21.15.117:0/2727415954 conn(0x7f2b4c079710 msgr2=0x7f2b4c1064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:54:43.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.934+0000 7f2b52fa4640 1 -- 172.21.15.117:0/2727415954 shutdown_connections 2024-09-21T05:54:43.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:43.934+0000 7f2b52fa4640 1 -- 172.21.15.117:0/2727415954 wait complete. 2024-09-21T05:54:43.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:43 smithi192 ceph-mon[34700]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:44.115 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 21 05:54:43 smithi138 podman[63240]: 2024-09-21 05:54:43.759823972 +0000 UTC m=+0.015787367 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T05:54:44.115 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 21 05:54:43 smithi138 podman[63240]: 2024-09-21 05:54:43.88438042 +0000 UTC m=+0.140343803 container create 4f0d649c20a1bfe499fe7634a6b660c94798087ae20d9e02cb2d02f0d5a8422f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-21T05:54:44.615 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 21 05:54:44 smithi138 podman[63240]: 2024-09-21 05:54:44.153769515 +0000 UTC m=+0.409732897 container init 4f0d649c20a1bfe499fe7634a6b660c94798087ae20d9e02cb2d02f0d5a8422f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-e, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-21T05:54:44.615 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 21 05:54:44 smithi138 podman[63240]: 2024-09-21 05:54:44.157225134 +0000 UTC m=+0.413188509 container start 4f0d649c20a1bfe499fe7634a6b660c94798087ae20d9e02cb2d02f0d5a8422f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-e, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-21T05:54:44.615 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 21 05:54:44 smithi138 ceph-mds[63259]: -- 172.21.15.138:0/2558713890 <== mon.2 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560cd7575c20 con 0x560cdb71d600 2024-09-21T05:54:44.615 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 21 05:54:44 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-e[63255]: starting mds.e at 2024-09-21T05:54:44.615 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 21 05:54:44 smithi138 bash[63240]: 4f0d649c20a1bfe499fe7634a6b660c94798087ae20d9e02cb2d02f0d5a8422f 2024-09-21T05:54:44.615 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 21 05:54:44 smithi138 systemd[1]: Started Ceph mds.e for ac8581a2-77dc-11ef-baf5-efdc52797490. 2024-09-21T05:54:44.937 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:44 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2727415954' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:54:44.938 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:44 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:44.938 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:44 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:44.938 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:44 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:44.938 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:44 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:54:44.938 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:44 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:54:44.938 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:44 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:45.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:44 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2727415954' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:54:45.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:44 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:45.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:44 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:45.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:44 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:45.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:44 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:54:45.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:44 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:54:45.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:44 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:45.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:44 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2727415954' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:54:45.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:44 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:45.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:44 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:45.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:44 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:45.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:44 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:54:45.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:44 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:54:45.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:44 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:45.958 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:45 smithi192 ceph-mon[34700]: Deploying daemon mds.f on smithi192 2024-09-21T05:54:45.959 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:45 smithi192 ceph-mon[34700]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:45.959 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:45 smithi192 ceph-mon[34700]: mds.? [v2:172.21.15.138:6834/4276801396,v1:172.21.15.138:6835/4276801396] up:boot 2024-09-21T05:54:45.959 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:45 smithi192 ceph-mon[34700]: fsmap 5 up:standby 2024-09-21T05:54:45.959 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:45 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T05:54:46.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:45 smithi117 ceph-mon[30947]: Deploying daemon mds.f on smithi192 2024-09-21T05:54:46.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:45 smithi117 ceph-mon[30947]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:46.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:45 smithi117 ceph-mon[30947]: mds.? [v2:172.21.15.138:6834/4276801396,v1:172.21.15.138:6835/4276801396] up:boot 2024-09-21T05:54:46.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:45 smithi117 ceph-mon[30947]: fsmap 5 up:standby 2024-09-21T05:54:46.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:45 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T05:54:46.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:45 smithi138 ceph-mon[34391]: Deploying daemon mds.f on smithi192 2024-09-21T05:54:46.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:45 smithi138 ceph-mon[34391]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:46.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:45 smithi138 ceph-mon[34391]: mds.? [v2:172.21.15.138:6834/4276801396,v1:172.21.15.138:6835/4276801396] up:boot 2024-09-21T05:54:46.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:45 smithi138 ceph-mon[34391]: fsmap 5 up:standby 2024-09-21T05:54:46.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:45 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T05:54:46.230 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 21 05:54:46 smithi192 systemd[1]: Starting Ceph mds.f for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T05:54:46.608 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 21 05:54:46 smithi192 podman[63722]: 2024-09-21 05:54:46.249174964 +0000 UTC m=+0.018095629 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T05:54:46.608 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 21 05:54:46 smithi192 podman[63722]: 2024-09-21 05:54:46.369881598 +0000 UTC m=+0.138802262 container create 18bdd104f97496f12bef7509217d98faddfae85ca7b300fbf89f473fbd25c67c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-f, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-21T05:54:46.657 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T05:54:46.910 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:54:46.980 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 21 05:54:46 smithi192 podman[63722]: 2024-09-21 05:54:46.631300156 +0000 UTC m=+0.400220803 container init 18bdd104f97496f12bef7509217d98faddfae85ca7b300fbf89f473fbd25c67c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T05:54:46.980 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 21 05:54:46 smithi192 podman[63722]: 2024-09-21 05:54:46.634560405 +0000 UTC m=+0.403481066 container start 18bdd104f97496f12bef7509217d98faddfae85ca7b300fbf89f473fbd25c67c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-f, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T05:54:46.980 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 21 05:54:46 smithi192 ceph-mds[63750]: -- 172.21.15.192:0/1950014105 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563ca8cb5860 con 0x563cace5d600 2024-09-21T05:54:46.980 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 21 05:54:46 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-f[63738]: starting mds.f at 2024-09-21T05:54:46.980 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 21 05:54:46 smithi192 bash[63722]: 18bdd104f97496f12bef7509217d98faddfae85ca7b300fbf89f473fbd25c67c 2024-09-21T05:54:46.980 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 21 05:54:46 smithi192 systemd[1]: Started Ceph mds.f for ac8581a2-77dc-11ef-baf5-efdc52797490. 2024-09-21T05:54:47.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.421+0000 7fdfe7a57640 1 -- 172.21.15.117:0/3507314523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfe0075dd0 msgr2=0x7fdfe00761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:47.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.421+0000 7fdfe7a57640 1 --2- 172.21.15.117:0/3507314523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfe0075dd0 0x7fdfe00761e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0009900 tx=0x7fdfd002a410 comp rx=0 tx=0).stop 2024-09-21T05:54:47.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.421+0000 7fdfe7a57640 1 -- 172.21.15.117:0/3507314523 shutdown_connections 2024-09-21T05:54:47.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.421+0000 7fdfe7a57640 1 --2- 172.21.15.117:0/3507314523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfe0075dd0 0x7fdfe00761e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:47.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.421+0000 7fdfe7a57640 1 --2- 172.21.15.117:0/3507314523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfe0114480 0x7fdfe0116950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:47.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.421+0000 7fdfe7a57640 1 --2- 172.21.15.117:0/3507314523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfe0077380 0x7fdfe00757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:47.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.421+0000 7fdfe7a57640 1 -- 172.21.15.117:0/3507314523 >> 172.21.15.117:0/3507314523 conn(0x7fdfe00ff600 msgr2=0x7fdfe0101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:54:47.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.421+0000 7fdfe7a57640 1 -- 172.21.15.117:0/3507314523 shutdown_connections 2024-09-21T05:54:47.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.421+0000 7fdfe7a57640 1 -- 172.21.15.117:0/3507314523 wait complete. 2024-09-21T05:54:47.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.422+0000 7fdfe7a57640 1 Processor -- start 2024-09-21T05:54:47.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.422+0000 7fdfe7a57640 1 -- start start 2024-09-21T05:54:47.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.422+0000 7fdfe7a57640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfe0075dd0 0x7fdfe01a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:47.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.422+0000 7fdfe7a57640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfe0077380 0x7fdfe01a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:47.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.422+0000 7fdfe7a57640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfe0114480 0x7fdfe01a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:47.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.422+0000 7fdfe7a57640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe011a0b0 con 0x7fdfe0114480 2024-09-21T05:54:47.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.422+0000 7fdfe7a57640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe0119f30 con 0x7fdfe0077380 2024-09-21T05:54:47.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.422+0000 7fdfe7a57640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe011a230 con 0x7fdfe0075dd0 2024-09-21T05:54:47.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.422+0000 7fdfe5ff4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfe0114480 0x7fdfe01a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:47.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.422+0000 7fdfe5ff4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfe0114480 0x7fdfe01a62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37992/0 (socket says 172.21.15.117:37992) 2024-09-21T05:54:47.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.422+0000 7fdfe5ff4640 1 -- 172.21.15.117:0/1314817837 learned_addr learned my addr 172.21.15.117:0/1314817837 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:54:47.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.422+0000 7fdfe57f3640 1 --2- 172.21.15.117:0/1314817837 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfe0075dd0 0x7fdfe01a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:47.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.422+0000 7fdfe4ff2640 1 --2- 172.21.15.117:0/1314817837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfe0077380 0x7fdfe01a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:47.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.423+0000 7fdfe5ff4640 1 -- 172.21.15.117:0/1314817837 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfe0075dd0 msgr2=0x7fdfe01a0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:47.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.423+0000 7fdfe5ff4640 1 --2- 172.21.15.117:0/1314817837 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfe0075dd0 0x7fdfe01a0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:47.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.423+0000 7fdfe5ff4640 1 -- 172.21.15.117:0/1314817837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfe0077380 msgr2=0x7fdfe01a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:47.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.423+0000 7fdfe5ff4640 1 --2- 172.21.15.117:0/1314817837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfe0077380 0x7fdfe01a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:47.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.423+0000 7fdfe5ff4640 1 -- 172.21.15.117:0/1314817837 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe01a69f0 con 0x7fdfe0114480 2024-09-21T05:54:47.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.423+0000 7fdfe4ff2640 1 --2- 172.21.15.117:0/1314817837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfe0077380 0x7fdfe01a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:54:47.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.423+0000 7fdfe57f3640 1 --2- 172.21.15.117:0/1314817837 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfe0075dd0 0x7fdfe01a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:54:47.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.423+0000 7fdfe5ff4640 1 --2- 172.21.15.117:0/1314817837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfe0114480 0x7fdfe01a62c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800eb10 tx=0x7fdfd800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:54:47.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.423+0000 7fdfd67fc640 1 -- 172.21.15.117:0/1314817837 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8004b20 con 0x7fdfe0114480 2024-09-21T05:54:47.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.423+0000 7fdfe7a57640 1 -- 172.21.15.117:0/1314817837 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfe01a6ce0 con 0x7fdfe0114480 2024-09-21T05:54:47.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.423+0000 7fdfd67fc640 1 -- 172.21.15.117:0/1314817837 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfd8004cc0 con 0x7fdfe0114480 2024-09-21T05:54:47.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.423+0000 7fdfd67fc640 1 -- 172.21.15.117:0/1314817837 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8010690 con 0x7fdfe0114480 2024-09-21T05:54:47.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.423+0000 7fdfe7a57640 1 -- 172.21.15.117:0/1314817837 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfe01ad440 con 0x7fdfe0114480 2024-09-21T05:54:47.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.424+0000 7fdfe7a57640 1 -- 172.21.15.117:0/1314817837 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfe00761e0 con 0x7fdfe0114480 2024-09-21T05:54:47.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.426+0000 7fdfd67fc640 1 -- 172.21.15.117:0/1314817837 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7fdfd8005000 con 0x7fdfe0114480 2024-09-21T05:54:47.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.426+0000 7fdfd67fc640 1 --2- 172.21.15.117:0/1314817837 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fdfcc0b6560 0x7fdfcc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:47.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.426+0000 7fdfe57f3640 1 --2- 172.21.15.117:0/1314817837 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fdfcc0b6560 0x7fdfcc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:47.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.427+0000 7fdfd67fc640 1 -- 172.21.15.117:0/1314817837 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7342+0+0 (secure 0 0 0) 0x7fdfd80e7780 con 0x7fdfe0114480 2024-09-21T05:54:47.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.427+0000 7fdfe57f3640 1 --2- 172.21.15.117:0/1314817837 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fdfcc0b6560 0x7fdfcc0b8a70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc00bfd0 tx=0x7fdfdc00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:54:47.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.429+0000 7fdfd67fc640 1 -- 172.21.15.117:0/1314817837 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdfd8056710 con 0x7fdfe0114480 2024-09-21T05:54:47.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.647+0000 7fdfe7a57640 1 -- 172.21.15.117:0/1314817837 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdfe01a1bb0 con 0x7fdfe0114480 2024-09-21T05:54:47.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.647+0000 7fdfd67fc640 1 -- 172.21.15.117:0/1314817837 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5578 (secure 0 0 0) 0x7fdfd80ae3e0 con 0x7fdfe0114480 2024-09-21T05:54:47.650 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:54:47.650 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":7,"btime":"2024-09-21T05:54:47:621394+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3227057504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3227057504},{"type":"v1","addr":"172.21.15.117:6835","nonce":3227057504}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3303829183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3303829183},{"type":"v1","addr":"172.21.15.117:6837","nonce":3303829183}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24545,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/3725629862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3725629862},{"type":"v1","addr":"172.21.15.138:6833","nonce":3725629862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24572,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/4276801396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4276801396},{"type":"v1","addr":"172.21.15.138:6835","nonce":4276801396}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24586,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/2224910879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2224910879},{"type":"v1","addr":"172.21.15.192:6833","nonce":2224910879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3530688860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3530688860},{"type":"v1","addr":"172.21.15.192:6835","nonce":3530688860}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-09-21T05:54:47.650 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 7 2024-09-21T05:54:47.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.650+0000 7fdfe7a57640 1 -- 172.21.15.117:0/1314817837 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fdfcc0b6560 msgr2=0x7fdfcc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:47.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.650+0000 7fdfe7a57640 1 --2- 172.21.15.117:0/1314817837 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fdfcc0b6560 0x7fdfcc0b8a70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc00bfd0 tx=0x7fdfdc00b390 comp rx=0 tx=0).stop 2024-09-21T05:54:47.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.650+0000 7fdfe7a57640 1 -- 172.21.15.117:0/1314817837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfe0114480 msgr2=0x7fdfe01a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:47.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.650+0000 7fdfe7a57640 1 --2- 172.21.15.117:0/1314817837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfe0114480 0x7fdfe01a62c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800eb10 tx=0x7fdfd800c2d0 comp rx=0 tx=0).stop 2024-09-21T05:54:47.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.650+0000 7fdfe7a57640 1 -- 172.21.15.117:0/1314817837 shutdown_connections 2024-09-21T05:54:47.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.650+0000 7fdfe7a57640 1 --2- 172.21.15.117:0/1314817837 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fdfcc0b6560 0x7fdfcc0b8a70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:47.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.650+0000 7fdfe7a57640 1 --2- 172.21.15.117:0/1314817837 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfe0114480 0x7fdfe01a62c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:47.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.650+0000 7fdfe7a57640 1 --2- 172.21.15.117:0/1314817837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfe0077380 0x7fdfe01a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:47.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.650+0000 7fdfe7a57640 1 --2- 172.21.15.117:0/1314817837 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfe0075dd0 0x7fdfe01a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:47.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.650+0000 7fdfe7a57640 1 -- 172.21.15.117:0/1314817837 >> 172.21.15.117:0/1314817837 conn(0x7fdfe00ff600 msgr2=0x7fdfe0110380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:54:47.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.650+0000 7fdfe7a57640 1 -- 172.21.15.117:0/1314817837 shutdown_connections 2024-09-21T05:54:47.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:47.650+0000 7fdfe7a57640 1 -- 172.21.15.117:0/1314817837 wait complete. 2024-09-21T05:54:47.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:47 smithi192 ceph-mon[34700]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:47.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:47 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:47.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:47 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:47.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:47 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:47.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:47 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:54:47.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:47 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:54:47.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:47 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:48.021 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:47 smithi117 ceph-mon[30947]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:48.021 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:47 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:48.021 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:47 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:48.022 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:47 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:48.022 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:47 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:54:48.022 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:47 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:54:48.022 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:47 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:48.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:47 smithi138 ceph-mon[34391]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:48.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:47 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:48.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:47 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:48.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:47 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:48.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:47 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:54:48.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:47 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:54:48.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:47 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:48.923 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:48 smithi192 ceph-mon[34700]: Deploying daemon mds.c on smithi192 2024-09-21T05:54:48.923 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:48 smithi192 ceph-mon[34700]: mds.? [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] up:boot 2024-09-21T05:54:48.923 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:48 smithi192 ceph-mon[34700]: fsmap 6 up:standby 2024-09-21T05:54:48.923 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:48 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T05:54:48.923 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:48 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/1314817837' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:54:48.923 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:48 smithi192 ceph-mon[34700]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:48.924 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 21 05:54:48 smithi192 systemd[1]: Starting Ceph mds.c for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T05:54:49.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:48 smithi117 ceph-mon[30947]: Deploying daemon mds.c on smithi192 2024-09-21T05:54:49.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:48 smithi117 ceph-mon[30947]: mds.? [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] up:boot 2024-09-21T05:54:49.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:48 smithi117 ceph-mon[30947]: fsmap 6 up:standby 2024-09-21T05:54:49.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:48 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T05:54:49.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:48 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/1314817837' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:54:49.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:48 smithi117 ceph-mon[30947]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:49.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:48 smithi138 ceph-mon[34391]: Deploying daemon mds.c on smithi192 2024-09-21T05:54:49.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:48 smithi138 ceph-mon[34391]: mds.? [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] up:boot 2024-09-21T05:54:49.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:48 smithi138 ceph-mon[34391]: fsmap 6 up:standby 2024-09-21T05:54:49.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:48 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T05:54:49.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:48 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/1314817837' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:54:49.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:48 smithi138 ceph-mon[34391]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:49.176 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 21 05:54:48 smithi192 podman[64005]: 2024-09-21 05:54:48.823993498 +0000 UTC m=+0.015472136 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T05:54:49.176 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 21 05:54:48 smithi192 podman[64005]: 2024-09-21 05:54:48.943131963 +0000 UTC m=+0.134610592 container create ad46b1b9886e93c90e8d2048ac7e48c423fd3dfe4614c4f7303233fb6823befe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-21T05:54:49.176 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 21 05:54:49 smithi192 podman[64005]: 2024-09-21 05:54:49.171319166 +0000 UTC m=+0.362797800 container init ad46b1b9886e93c90e8d2048ac7e48c423fd3dfe4614c4f7303233fb6823befe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-c, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-21T05:54:49.176 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 21 05:54:49 smithi192 podman[64005]: 2024-09-21 05:54:49.174481667 +0000 UTC m=+0.365960297 container start ad46b1b9886e93c90e8d2048ac7e48c423fd3dfe4614c4f7303233fb6823befe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-c, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-21T05:54:49.480 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 21 05:54:49 smithi192 ceph-mds[64024]: -- 172.21.15.192:0/3755163136 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b348a97c20 con 0x55b34cc3fa80 2024-09-21T05:54:49.480 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 21 05:54:49 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-c[64020]: starting mds.c at 2024-09-21T05:54:49.480 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 21 05:54:49 smithi192 bash[64005]: ad46b1b9886e93c90e8d2048ac7e48c423fd3dfe4614c4f7303233fb6823befe 2024-09-21T05:54:49.480 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 21 05:54:49 smithi192 systemd[1]: Started Ceph mds.c for ac8581a2-77dc-11ef-baf5-efdc52797490. 2024-09-21T05:54:50.388 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T05:54:50.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:50 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:50.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:50 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:50.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:50 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:50.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:50 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:54:50.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:50 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:54:50.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:50 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:50.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:50 smithi138 ceph-mon[34391]: Deploying daemon mds.h on smithi138 2024-09-21T05:54:50.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:50 smithi138 ceph-mon[34391]: mds.? [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] up:boot 2024-09-21T05:54:50.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:50 smithi138 ceph-mon[34391]: fsmap 7 up:standby 2024-09-21T05:54:50.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:50 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T05:54:50.649 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:54:50.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:50 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:50.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:50 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:50.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:50 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:50.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:50 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:54:50.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:50 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:54:50.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:50 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:50.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:50 smithi192 ceph-mon[34700]: Deploying daemon mds.h on smithi138 2024-09-21T05:54:50.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:50 smithi192 ceph-mon[34700]: mds.? [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] up:boot 2024-09-21T05:54:50.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:50 smithi192 ceph-mon[34700]: fsmap 7 up:standby 2024-09-21T05:54:50.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:50 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T05:54:50.770 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:50 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:50.770 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:50 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:50.770 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:50 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:50.770 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:50 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:54:50.770 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:50 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:54:50.770 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:50 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:50.770 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:50 smithi117 ceph-mon[30947]: Deploying daemon mds.h on smithi138 2024-09-21T05:54:50.770 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:50 smithi117 ceph-mon[30947]: mds.? [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] up:boot 2024-09-21T05:54:50.770 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:50 smithi117 ceph-mon[30947]: fsmap 7 up:standby 2024-09-21T05:54:50.771 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:50 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T05:54:51.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.150+0000 7f5116056640 1 -- 172.21.15.117:0/2375999021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51101094a0 msgr2=0x7f51101098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:51.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.150+0000 7f5116056640 1 --2- 172.21.15.117:0/2375999021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51101094a0 0x7f51101098b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5104009900 tx=0x7f510402a410 comp rx=0 tx=0).stop 2024-09-21T05:54:51.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.150+0000 7f5116056640 1 -- 172.21.15.117:0/2375999021 shutdown_connections 2024-09-21T05:54:51.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.150+0000 7f5116056640 1 --2- 172.21.15.117:0/2375999021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51101094a0 0x7f51101098b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:51.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.150+0000 7f5116056640 1 --2- 172.21.15.117:0/2375999021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5110109df0 0x7f5110110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:51.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.150+0000 7f5116056640 1 --2- 172.21.15.117:0/2375999021 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5110103b60 0x7f5110103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:51.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.150+0000 7f5116056640 1 -- 172.21.15.117:0/2375999021 >> 172.21.15.117:0/2375999021 conn(0x7f511007b850 msgr2=0x7f511007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:54:51.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.150+0000 7f5116056640 1 -- 172.21.15.117:0/2375999021 shutdown_connections 2024-09-21T05:54:51.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.150+0000 7f5116056640 1 -- 172.21.15.117:0/2375999021 wait complete. 2024-09-21T05:54:51.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.151+0000 7f5116056640 1 Processor -- start 2024-09-21T05:54:51.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.151+0000 7f5116056640 1 -- start start 2024-09-21T05:54:51.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.151+0000 7f5116056640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5110103b60 0x7f511019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:51.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.151+0000 7f5116056640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51101094a0 0x7f511019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:51.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.151+0000 7f5116056640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5110109df0 0x7f511019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:51.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.151+0000 7f5116056640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5110113ad0 con 0x7f5110103b60 2024-09-21T05:54:51.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.151+0000 7f5116056640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5110113950 con 0x7f5110109df0 2024-09-21T05:54:51.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.151+0000 7f5116056640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5110113c50 con 0x7f51101094a0 2024-09-21T05:54:51.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.151+0000 7f510f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5110103b60 0x7f511019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:51.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.151+0000 7f510f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5110103b60 0x7f511019a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38012/0 (socket says 172.21.15.117:38012) 2024-09-21T05:54:51.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.151+0000 7f510f7fe640 1 -- 172.21.15.117:0/519122600 learned_addr learned my addr 172.21.15.117:0/519122600 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:54:51.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.151+0000 7f510ffff640 1 --2- 172.21.15.117:0/519122600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5110109df0 0x7f511019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:51.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.151+0000 7f510f7fe640 1 -- 172.21.15.117:0/519122600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51101094a0 msgr2=0x7f511019ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:51.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.151+0000 7f510f7fe640 1 --2- 172.21.15.117:0/519122600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51101094a0 0x7f511019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:51.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.151+0000 7f510f7fe640 1 -- 172.21.15.117:0/519122600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5110109df0 msgr2=0x7f511019fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:51.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.151+0000 7f510f7fe640 1 --2- 172.21.15.117:0/519122600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5110109df0 0x7f511019fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:51.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.151+0000 7f510f7fe640 1 -- 172.21.15.117:0/519122600 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51101a04d0 con 0x7f5110103b60 2024-09-21T05:54:51.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.152+0000 7f510f7fe640 1 --2- 172.21.15.117:0/519122600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5110103b60 0x7f511019a560 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f510000b4a0 tx=0x7f510000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:54:51.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.152+0000 7f510cff9640 1 -- 172.21.15.117:0/519122600 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5100014070 con 0x7f5110103b60 2024-09-21T05:54:51.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.152+0000 7f5116056640 1 -- 172.21.15.117:0/519122600 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51101a07c0 con 0x7f5110103b60 2024-09-21T05:54:51.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.152+0000 7f510cff9640 1 -- 172.21.15.117:0/519122600 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5100010ac0 con 0x7f5110103b60 2024-09-21T05:54:51.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.152+0000 7f510cff9640 1 -- 172.21.15.117:0/519122600 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5100017c30 con 0x7f5110103b60 2024-09-21T05:54:51.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.152+0000 7f5116056640 1 -- 172.21.15.117:0/519122600 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51101a6f20 con 0x7f5110103b60 2024-09-21T05:54:51.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.153+0000 7f5116056640 1 -- 172.21.15.117:0/519122600 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5110103f70 con 0x7f5110103b60 2024-09-21T05:54:51.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.154+0000 7f510cff9640 1 -- 172.21.15.117:0/519122600 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f5100017070 con 0x7f5110103b60 2024-09-21T05:54:51.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.155+0000 7f510cff9640 1 --2- 172.21.15.117:0/519122600 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f50f80b6560 0x7f50f80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:51.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.155+0000 7f510effd640 1 --2- 172.21.15.117:0/519122600 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f50f80b6560 0x7f50f80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:51.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.155+0000 7f510cff9640 1 -- 172.21.15.117:0/519122600 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7342+0+0 (secure 0 0 0) 0x7f51000e65c0 con 0x7f5110103b60 2024-09-21T05:54:51.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.155+0000 7f510effd640 1 --2- 172.21.15.117:0/519122600 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f50f80b6560 0x7f50f80b8a70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f510402a930 tx=0x7f5104031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:54:51.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.159+0000 7f510cff9640 1 -- 172.21.15.117:0/519122600 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f51000ad220 con 0x7f5110103b60 2024-09-21T05:54:51.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.375+0000 7f5116056640 1 -- 172.21.15.117:0/519122600 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f511002e640 con 0x7f5110103b60 2024-09-21T05:54:51.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.376+0000 7f510cff9640 1 -- 172.21.15.117:0/519122600 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6406 (secure 0 0 0) 0x7f51000b1600 con 0x7f5110103b60 2024-09-21T05:54:51.378 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:54:51.378 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":8,"btime":"2024-09-21T05:54:49:633180+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3227057504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3227057504},{"type":"v1","addr":"172.21.15.117:6835","nonce":3227057504}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3303829183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3303829183},{"type":"v1","addr":"172.21.15.117:6837","nonce":3303829183}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24545,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/3725629862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3725629862},{"type":"v1","addr":"172.21.15.138:6833","nonce":3725629862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24572,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/4276801396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4276801396},{"type":"v1","addr":"172.21.15.138:6835","nonce":4276801396}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24586,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/2224910879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2224910879},{"type":"v1","addr":"172.21.15.192:6833","nonce":2224910879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3530688860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3530688860},{"type":"v1","addr":"172.21.15.192:6835","nonce":3530688860}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24628,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2683860842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2683860842},{"type":"v1","addr":"172.21.15.192:6837","nonce":2683860842}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-21T05:54:51.378 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 8 2024-09-21T05:54:51.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.378+0000 7f5116056640 1 -- 172.21.15.117:0/519122600 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f50f80b6560 msgr2=0x7f50f80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:51.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.378+0000 7f5116056640 1 --2- 172.21.15.117:0/519122600 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f50f80b6560 0x7f50f80b8a70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f510402a930 tx=0x7f5104031400 comp rx=0 tx=0).stop 2024-09-21T05:54:51.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.378+0000 7f5116056640 1 -- 172.21.15.117:0/519122600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5110103b60 msgr2=0x7f511019a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:51.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.378+0000 7f5116056640 1 --2- 172.21.15.117:0/519122600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5110103b60 0x7f511019a560 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f510000b4a0 tx=0x7f510000b960 comp rx=0 tx=0).stop 2024-09-21T05:54:51.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.378+0000 7f5116056640 1 -- 172.21.15.117:0/519122600 shutdown_connections 2024-09-21T05:54:51.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.378+0000 7f5116056640 1 --2- 172.21.15.117:0/519122600 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f50f80b6560 0x7f50f80b8a70 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:51.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.378+0000 7f5116056640 1 --2- 172.21.15.117:0/519122600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5110109df0 0x7f511019fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:51.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.378+0000 7f5116056640 1 --2- 172.21.15.117:0/519122600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51101094a0 0x7f511019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:51.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.378+0000 7f5116056640 1 --2- 172.21.15.117:0/519122600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5110103b60 0x7f511019a560 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:51.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.378+0000 7f5116056640 1 -- 172.21.15.117:0/519122600 >> 172.21.15.117:0/519122600 conn(0x7f511007b850 msgr2=0x7f51101075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:54:51.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.378+0000 7f5116056640 1 -- 172.21.15.117:0/519122600 shutdown_connections 2024-09-21T05:54:51.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:51.378+0000 7f5116056640 1 -- 172.21.15.117:0/519122600 wait complete. 2024-09-21T05:54:51.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:51 smithi138 ceph-mon[34391]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:51.615 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 21 05:54:51 smithi138 systemd[1]: Starting Ceph mds.h for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T05:54:51.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:51 smithi192 ceph-mon[34700]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:51.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:51 smithi117 ceph-mon[30947]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:51.879 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 21 05:54:51 smithi138 podman[63512]: 2024-09-21 05:54:51.516464006 +0000 UTC m=+0.015823290 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T05:54:51.879 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 21 05:54:51 smithi138 podman[63512]: 2024-09-21 05:54:51.625215659 +0000 UTC m=+0.124574946 container create 6bb618c6362c22f3c5d6ee1d26eb6c24a46d3e27ab056ad08b7d7ac39793f340 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-h, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T05:54:51.879 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 21 05:54:51 smithi138 podman[63512]: 2024-09-21 05:54:51.878150502 +0000 UTC m=+0.377509788 container init 6bb618c6362c22f3c5d6ee1d26eb6c24a46d3e27ab056ad08b7d7ac39793f340 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-h, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True) 2024-09-21T05:54:52.336 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 21 05:54:51 smithi138 podman[63512]: 2024-09-21 05:54:51.881464938 +0000 UTC m=+0.380824231 container start 6bb618c6362c22f3c5d6ee1d26eb6c24a46d3e27ab056ad08b7d7ac39793f340 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-h, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-21T05:54:52.337 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 21 05:54:51 smithi138 ceph-mds[63531]: -- 172.21.15.138:0/3288786006 <== mon.2 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f65a7a1860 con 0x55f65e949a80 2024-09-21T05:54:52.337 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 21 05:54:51 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-h[63527]: starting mds.h at 2024-09-21T05:54:52.337 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 21 05:54:51 smithi138 bash[63512]: 6bb618c6362c22f3c5d6ee1d26eb6c24a46d3e27ab056ad08b7d7ac39793f340 2024-09-21T05:54:52.337 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 21 05:54:51 smithi138 systemd[1]: Started Ceph mds.h for ac8581a2-77dc-11ef-baf5-efdc52797490. 2024-09-21T05:54:52.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:52 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/519122600' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:54:52.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:52 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:52.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:52 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:52.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:52 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:52.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:52 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:54:52.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:52 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:54:52.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:52 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:52.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:52 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/519122600' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:54:52.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:52 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:52.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:52 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:52.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:52 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:52.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:52 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:54:52.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:52 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:54:52.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:52 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:52.659 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:52 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/519122600' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:54:52.659 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:52 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:52.659 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:52 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:52.659 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:52 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:52.660 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:52 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:54:52.660 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:52 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:54:52.660 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:52 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:53.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:53 smithi138 ceph-mon[34391]: Deploying daemon mds.i on smithi192 2024-09-21T05:54:53.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:53 smithi138 ceph-mon[34391]: mds.? [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] up:boot 2024-09-21T05:54:53.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:53 smithi138 ceph-mon[34391]: fsmap 8 up:standby 2024-09-21T05:54:53.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:53 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T05:54:53.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:53 smithi138 ceph-mon[34391]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:53.727 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:53 smithi192 ceph-mon[34700]: Deploying daemon mds.i on smithi192 2024-09-21T05:54:53.728 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:53 smithi192 ceph-mon[34700]: mds.? [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] up:boot 2024-09-21T05:54:53.728 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:53 smithi192 ceph-mon[34700]: fsmap 8 up:standby 2024-09-21T05:54:53.728 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:53 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T05:54:53.728 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:53 smithi192 ceph-mon[34700]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:53.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:53 smithi117 ceph-mon[30947]: Deploying daemon mds.i on smithi192 2024-09-21T05:54:53.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:53 smithi117 ceph-mon[30947]: mds.? [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] up:boot 2024-09-21T05:54:53.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:53 smithi117 ceph-mon[30947]: fsmap 8 up:standby 2024-09-21T05:54:53.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:53 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T05:54:53.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:53 smithi117 ceph-mon[30947]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:53.980 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 21 05:54:53 smithi192 systemd[1]: Starting Ceph mds.i for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T05:54:54.095 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T05:54:54.352 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:54:54.378 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 21 05:54:54 smithi192 podman[64270]: 2024-09-21 05:54:54.022434073 +0000 UTC m=+0.015868223 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T05:54:54.378 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 21 05:54:54 smithi192 podman[64270]: 2024-09-21 05:54:54.133013805 +0000 UTC m=+0.126447942 container create 308b7b43d596d1903d754f87cf2c2a96d4a1e99d34e85f11c48eb3fcfe979d8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-i, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-21T05:54:54.378 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 21 05:54:54 smithi192 podman[64270]: 2024-09-21 05:54:54.377312761 +0000 UTC m=+0.370746897 container init 308b7b43d596d1903d754f87cf2c2a96d4a1e99d34e85f11c48eb3fcfe979d8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-i, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True) 2024-09-21T05:54:54.730 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 21 05:54:54 smithi192 podman[64270]: 2024-09-21 05:54:54.380361562 +0000 UTC m=+0.373795707 container start 308b7b43d596d1903d754f87cf2c2a96d4a1e99d34e85f11c48eb3fcfe979d8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-i, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T05:54:54.730 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 21 05:54:54 smithi192 ceph-mds[64289]: -- 172.21.15.192:0/2500025137 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aafe937860 con 0x55ab02adfa80 2024-09-21T05:54:54.730 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 21 05:54:54 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-i[64285]: starting mds.i at 2024-09-21T05:54:54.730 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 21 05:54:54 smithi192 bash[64270]: 308b7b43d596d1903d754f87cf2c2a96d4a1e99d34e85f11c48eb3fcfe979d8e 2024-09-21T05:54:54.730 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 21 05:54:54 smithi192 systemd[1]: Started Ceph mds.i for ac8581a2-77dc-11ef-baf5-efdc52797490. 2024-09-21T05:54:54.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.857+0000 7f1911ff7640 1 -- 172.21.15.117:0/4138969546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f190c104840 msgr2=0x7f190c104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:54.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.857+0000 7f1911ff7640 1 --2- 172.21.15.117:0/4138969546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f190c104840 0x7f190c104c50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f18fc009900 tx=0x7f18fc02a410 comp rx=0 tx=0).stop 2024-09-21T05:54:54.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.857+0000 7f1911ff7640 1 -- 172.21.15.117:0/4138969546 shutdown_connections 2024-09-21T05:54:54.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.857+0000 7f1911ff7640 1 --2- 172.21.15.117:0/4138969546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f190c10ddf0 0x7f190c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:54.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.857+0000 7f1911ff7640 1 --2- 172.21.15.117:0/4138969546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f190c105270 0x7f190c10d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:54.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.857+0000 7f1911ff7640 1 --2- 172.21.15.117:0/4138969546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f190c104840 0x7f190c104c50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:54.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.857+0000 7f1911ff7640 1 -- 172.21.15.117:0/4138969546 >> 172.21.15.117:0/4138969546 conn(0x7f190c079760 msgr2=0x7f190c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:54:54.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.857+0000 7f1911ff7640 1 -- 172.21.15.117:0/4138969546 shutdown_connections 2024-09-21T05:54:54.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.857+0000 7f1911ff7640 1 -- 172.21.15.117:0/4138969546 wait complete. 2024-09-21T05:54:54.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.858+0000 7f1911ff7640 1 Processor -- start 2024-09-21T05:54:54.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.858+0000 7f1911ff7640 1 -- start start 2024-09-21T05:54:54.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.859+0000 7f1911ff7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f190c104840 0x7f190c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:54.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.859+0000 7f1911ff7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f190c105270 0x7f190c19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:54.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.859+0000 7f1911ff7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f190c10ddf0 0x7f190c19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:54.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.859+0000 7f1911ff7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f190c113a90 con 0x7f190c105270 2024-09-21T05:54:54.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.859+0000 7f1911ff7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f190c113910 con 0x7f190c10ddf0 2024-09-21T05:54:54.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.859+0000 7f1911ff7640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f190c113c10 con 0x7f190c104840 2024-09-21T05:54:54.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.859+0000 7f190affd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f190c105270 0x7f190c19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:54.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.859+0000 7f190affd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f190c105270 0x7f190c19aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57210/0 (socket says 172.21.15.117:57210) 2024-09-21T05:54:54.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.859+0000 7f190affd640 1 -- 172.21.15.117:0/2925545770 learned_addr learned my addr 172.21.15.117:0/2925545770 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:54:54.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.859+0000 7f190bfff640 1 --2- 172.21.15.117:0/2925545770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f190c10ddf0 0x7f190c19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:54.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.859+0000 7f190b7fe640 1 --2- 172.21.15.117:0/2925545770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f190c104840 0x7f190c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:54.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.859+0000 7f190affd640 1 -- 172.21.15.117:0/2925545770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f190c104840 msgr2=0x7f190c19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:54.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.859+0000 7f190affd640 1 --2- 172.21.15.117:0/2925545770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f190c104840 0x7f190c19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:54.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.859+0000 7f190affd640 1 -- 172.21.15.117:0/2925545770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f190c10ddf0 msgr2=0x7f190c19fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:54.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.859+0000 7f190affd640 1 --2- 172.21.15.117:0/2925545770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f190c10ddf0 0x7f190c19fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:54.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.859+0000 7f190affd640 1 -- 172.21.15.117:0/2925545770 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f190c1a0410 con 0x7f190c105270 2024-09-21T05:54:54.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.859+0000 7f190affd640 1 --2- 172.21.15.117:0/2925545770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f190c105270 0x7f190c19aa40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f190000d750 tx=0x7f190000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:54:54.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.859+0000 7f1908ff9640 1 -- 172.21.15.117:0/2925545770 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1900017070 con 0x7f190c105270 2024-09-21T05:54:54.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.859+0000 7f1908ff9640 1 -- 172.21.15.117:0/2925545770 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19000054d0 con 0x7f190c105270 2024-09-21T05:54:54.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.859+0000 7f1911ff7640 1 -- 172.21.15.117:0/2925545770 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f190c1a0700 con 0x7f190c105270 2024-09-21T05:54:54.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.859+0000 7f1908ff9640 1 -- 172.21.15.117:0/2925545770 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1900004c30 con 0x7f190c105270 2024-09-21T05:54:54.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.859+0000 7f1911ff7640 1 -- 172.21.15.117:0/2925545770 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f190c1a6e60 con 0x7f190c105270 2024-09-21T05:54:54.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.860+0000 7f1911ff7640 1 -- 172.21.15.117:0/2925545770 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f190c06c240 con 0x7f190c105270 2024-09-21T05:54:54.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.862+0000 7f1908ff9640 1 -- 172.21.15.117:0/2925545770 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f1900004dd0 con 0x7f190c105270 2024-09-21T05:54:54.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.863+0000 7f1908ff9640 1 --2- 172.21.15.117:0/2925545770 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f18f40b6560 0x7f18f40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:54.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.863+0000 7f190b7fe640 1 --2- 172.21.15.117:0/2925545770 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f18f40b6560 0x7f18f40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:54.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.863+0000 7f1908ff9640 1 -- 172.21.15.117:0/2925545770 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7342+0+0 (secure 0 0 0) 0x7f19000e6dc0 con 0x7f190c105270 2024-09-21T05:54:54.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.863+0000 7f190b7fe640 1 --2- 172.21.15.117:0/2925545770 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f18f40b6560 0x7f18f40b8a70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f18fc031440 tx=0x7f18fc005d50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:54:54.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:54.866+0000 7f1908ff9640 1 -- 172.21.15.117:0/2925545770 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f19000ad9c0 con 0x7f190c105270 2024-09-21T05:54:55.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:55.092+0000 7f1911ff7640 1 -- 172.21.15.117:0/2925545770 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f190c19b5d0 con 0x7f190c105270 2024-09-21T05:54:55.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:55.093+0000 7f1908ff9640 1 -- 172.21.15.117:0/2925545770 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8064 (secure 0 0 0) 0x7f190c19b5d0 con 0x7f190c105270 2024-09-21T05:54:55.095 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:54:55.095 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":10,"btime":"2024-09-21T05:54:54:468892+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3227057504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3227057504},{"type":"v1","addr":"172.21.15.117:6835","nonce":3227057504}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3303829183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3303829183},{"type":"v1","addr":"172.21.15.117:6837","nonce":3303829183}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24545,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/3725629862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3725629862},{"type":"v1","addr":"172.21.15.138:6833","nonce":3725629862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24572,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/4276801396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4276801396},{"type":"v1","addr":"172.21.15.138:6835","nonce":4276801396}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24586,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/2224910879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2224910879},{"type":"v1","addr":"172.21.15.192:6833","nonce":2224910879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24602,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2540335374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2540335374},{"type":"v1","addr":"172.21.15.138:6837","nonce":2540335374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3530688860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3530688860},{"type":"v1","addr":"172.21.15.192:6835","nonce":3530688860}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24628,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2683860842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2683860842},{"type":"v1","addr":"172.21.15.192:6837","nonce":2683860842}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/1727623231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1727623231},{"type":"v1","addr":"172.21.15.192:6839","nonce":1727623231}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-21T05:54:55.095 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 10 2024-09-21T05:54:55.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:55.095+0000 7f1911ff7640 1 -- 172.21.15.117:0/2925545770 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f18f40b6560 msgr2=0x7f18f40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:55.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:55.095+0000 7f1911ff7640 1 --2- 172.21.15.117:0/2925545770 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f18f40b6560 0x7f18f40b8a70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f18fc031440 tx=0x7f18fc005d50 comp rx=0 tx=0).stop 2024-09-21T05:54:55.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:55.095+0000 7f1911ff7640 1 -- 172.21.15.117:0/2925545770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f190c105270 msgr2=0x7f190c19aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:55.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:55.095+0000 7f1911ff7640 1 --2- 172.21.15.117:0/2925545770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f190c105270 0x7f190c19aa40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f190000d750 tx=0x7f190000dc10 comp rx=0 tx=0).stop 2024-09-21T05:54:55.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:55.095+0000 7f1911ff7640 1 -- 172.21.15.117:0/2925545770 shutdown_connections 2024-09-21T05:54:55.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:55.095+0000 7f1911ff7640 1 --2- 172.21.15.117:0/2925545770 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f18f40b6560 0x7f18f40b8a70 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:55.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:55.095+0000 7f1911ff7640 1 --2- 172.21.15.117:0/2925545770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f190c105270 0x7f190c19aa40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:55.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:55.095+0000 7f1911ff7640 1 --2- 172.21.15.117:0/2925545770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f190c10ddf0 0x7f190c19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:55.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:55.095+0000 7f1911ff7640 1 --2- 172.21.15.117:0/2925545770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f190c104840 0x7f190c19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:55.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:55.095+0000 7f1911ff7640 1 -- 172.21.15.117:0/2925545770 >> 172.21.15.117:0/2925545770 conn(0x7f190c079760 msgr2=0x7f190c100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:54:55.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:55.095+0000 7f1911ff7640 1 -- 172.21.15.117:0/2925545770 shutdown_connections 2024-09-21T05:54:55.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:55.095+0000 7f1911ff7640 1 -- 172.21.15.117:0/2925545770 wait complete. 2024-09-21T05:54:55.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:55 smithi192 ceph-mon[34700]: mds.? [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] up:boot 2024-09-21T05:54:55.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:55 smithi192 ceph-mon[34700]: fsmap 9 up:standby 2024-09-21T05:54:55.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:55 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T05:54:55.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:55 smithi192 ceph-mon[34700]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:55.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:55 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:55.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:55 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:55.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:55 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:55.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:55 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:54:55.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:55 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:54:55.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:55 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:55.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:55 smithi192 ceph-mon[34700]: Deploying daemon mds.g on smithi117 2024-09-21T05:54:55.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:55 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2925545770' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:54:55.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:55 smithi117 ceph-mon[30947]: mds.? [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] up:boot 2024-09-21T05:54:55.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:55 smithi117 ceph-mon[30947]: fsmap 9 up:standby 2024-09-21T05:54:55.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:55 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T05:54:55.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:55 smithi117 ceph-mon[30947]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:55.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:55 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:55.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:55 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:55.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:55 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:55.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:55 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:54:55.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:55 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:54:55.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:55 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:55.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:55 smithi117 ceph-mon[30947]: Deploying daemon mds.g on smithi117 2024-09-21T05:54:55.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:55 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2925545770' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:54:55.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:55 smithi138 ceph-mon[34391]: mds.? [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] up:boot 2024-09-21T05:54:55.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:55 smithi138 ceph-mon[34391]: fsmap 9 up:standby 2024-09-21T05:54:55.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:55 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T05:54:55.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:55 smithi138 ceph-mon[34391]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:55.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:55 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:55.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:55 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:55.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:55 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:55.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:55 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:54:55.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:55 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:54:55.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:55 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:55.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:55 smithi138 ceph-mon[34391]: Deploying daemon mds.g on smithi117 2024-09-21T05:54:55.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:55 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2925545770' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:54:57.527 INFO:journalctl@ceph.mds.g.smithi117.stdout:Sep 21 05:54:57 smithi117 systemd[1]: Starting Ceph mds.g for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T05:54:57.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:57 smithi192 ceph-mon[34700]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:57.837 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:57 smithi117 ceph-mon[30947]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:57.838 INFO:journalctl@ceph.mds.g.smithi117.stdout:Sep 21 05:54:57 smithi117 podman[72944]: 2024-09-21 05:54:57.451524181 +0000 UTC m=+0.016114736 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T05:54:57.838 INFO:journalctl@ceph.mds.g.smithi117.stdout:Sep 21 05:54:57 smithi117 podman[72944]: 2024-09-21 05:54:57.583078054 +0000 UTC m=+0.147668645 container create ad0a6a2f1e72a3a0cd0aebaa7886f332975bf9f0a19f4c144ea78fa69cf4a1ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-g, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T05:54:57.838 INFO:journalctl@ceph.mds.g.smithi117.stdout:Sep 21 05:54:57 smithi117 podman[72944]: 2024-09-21 05:54:57.836590788 +0000 UTC m=+0.401181342 container init ad0a6a2f1e72a3a0cd0aebaa7886f332975bf9f0a19f4c144ea78fa69cf4a1ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-g, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-21T05:54:57.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:57 smithi138 ceph-mon[34391]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:57.966 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T05:54:58.228 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:54:58.277 INFO:journalctl@ceph.mds.g.smithi117.stdout:Sep 21 05:54:57 smithi117 podman[72944]: 2024-09-21 05:54:57.839549009 +0000 UTC m=+0.404139558 container start ad0a6a2f1e72a3a0cd0aebaa7886f332975bf9f0a19f4c144ea78fa69cf4a1ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-g, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T05:54:58.277 INFO:journalctl@ceph.mds.g.smithi117.stdout:Sep 21 05:54:57 smithi117 ceph-mds[72963]: -- 172.21.15.117:0/33763169 <== mon.0 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f84cd49860 con 0x55f850ef1180 2024-09-21T05:54:58.277 INFO:journalctl@ceph.mds.g.smithi117.stdout:Sep 21 05:54:57 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-g[72959]: starting mds.g at 2024-09-21T05:54:58.277 INFO:journalctl@ceph.mds.g.smithi117.stdout:Sep 21 05:54:57 smithi117 bash[72944]: ad0a6a2f1e72a3a0cd0aebaa7886f332975bf9f0a19f4c144ea78fa69cf4a1ca 2024-09-21T05:54:58.277 INFO:journalctl@ceph.mds.g.smithi117.stdout:Sep 21 05:54:57 smithi117 systemd[1]: Started Ceph mds.g for ac8581a2-77dc-11ef-baf5-efdc52797490. 2024-09-21T05:54:58.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.719+0000 7f7edc9a1640 1 -- 172.21.15.117:0/1778833976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ed8102080 msgr2=0x7f7ed8102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:58.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.719+0000 7f7edc9a1640 1 --2- 172.21.15.117:0/1778833976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ed8102080 0x7f7ed8102490 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009900 tx=0x7f7ec802a410 comp rx=0 tx=0).stop 2024-09-21T05:54:58.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.719+0000 7f7edc9a1640 1 -- 172.21.15.117:0/1778833976 shutdown_connections 2024-09-21T05:54:58.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.719+0000 7f7edc9a1640 1 --2- 172.21.15.117:0/1778833976 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ed8109eb0 0x7f7ed81106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:58.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.719+0000 7f7edc9a1640 1 --2- 172.21.15.117:0/1778833976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ed8109560 0x7f7ed8109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:58.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.719+0000 7f7edc9a1640 1 --2- 172.21.15.117:0/1778833976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ed8102080 0x7f7ed8102490 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:58.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.719+0000 7f7edc9a1640 1 -- 172.21.15.117:0/1778833976 >> 172.21.15.117:0/1778833976 conn(0x7f7ed8079ec0 msgr2=0x7f7ed807a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:54:58.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.719+0000 7f7edc9a1640 1 -- 172.21.15.117:0/1778833976 shutdown_connections 2024-09-21T05:54:58.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.719+0000 7f7edc9a1640 1 -- 172.21.15.117:0/1778833976 wait complete. 2024-09-21T05:54:58.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.720+0000 7f7edc9a1640 1 Processor -- start 2024-09-21T05:54:58.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.720+0000 7f7edc9a1640 1 -- start start 2024-09-21T05:54:58.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.720+0000 7f7edc9a1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ed8102080 0x7f7ed819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:58.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.720+0000 7f7edc9a1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ed8109560 0x7f7ed819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:58.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.720+0000 7f7edc9a1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ed8109eb0 0x7f7ed819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:58.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.721+0000 7f7edc9a1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed8113b70 con 0x7f7ed8109eb0 2024-09-21T05:54:58.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.721+0000 7f7edc9a1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed81139f0 con 0x7f7ed8102080 2024-09-21T05:54:58.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.721+0000 7f7edc9a1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed8113cf0 con 0x7f7ed8109560 2024-09-21T05:54:58.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.721+0000 7f7ed6d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ed8109eb0 0x7f7ed819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:58.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.721+0000 7f7ed6d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ed8109eb0 0x7f7ed819fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57264/0 (socket says 172.21.15.117:57264) 2024-09-21T05:54:58.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.721+0000 7f7ed6d9d640 1 -- 172.21.15.117:0/2122739264 learned_addr learned my addr 172.21.15.117:0/2122739264 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:54:58.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.721+0000 7f7ed659c640 1 --2- 172.21.15.117:0/2122739264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ed8102080 0x7f7ed819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:58.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.721+0000 7f7ecdd9b640 1 --2- 172.21.15.117:0/2122739264 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ed8109560 0x7f7ed819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:58.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.721+0000 7f7ed6d9d640 1 -- 172.21.15.117:0/2122739264 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ed8109560 msgr2=0x7f7ed819aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:58.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.721+0000 7f7ed6d9d640 1 --2- 172.21.15.117:0/2122739264 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ed8109560 0x7f7ed819aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:58.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.721+0000 7f7ed6d9d640 1 -- 172.21.15.117:0/2122739264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ed8102080 msgr2=0x7f7ed819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:58.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.721+0000 7f7ed6d9d640 1 --2- 172.21.15.117:0/2122739264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ed8102080 0x7f7ed819a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:58.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.721+0000 7f7ed6d9d640 1 -- 172.21.15.117:0/2122739264 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed81a0450 con 0x7f7ed8109eb0 2024-09-21T05:54:58.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.721+0000 7f7ed6d9d640 1 --2- 172.21.15.117:0/2122739264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ed8109eb0 0x7f7ed819fd20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000eaa0 tx=0x7f7ed000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:54:58.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.721+0000 7f7ecffff640 1 -- 172.21.15.117:0/2122739264 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed000cdb0 con 0x7f7ed8109eb0 2024-09-21T05:54:58.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.721+0000 7f7edc9a1640 1 -- 172.21.15.117:0/2122739264 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed81a0740 con 0x7f7ed8109eb0 2024-09-21T05:54:58.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.721+0000 7f7ecffff640 1 -- 172.21.15.117:0/2122739264 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ed0005020 con 0x7f7ed8109eb0 2024-09-21T05:54:58.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.721+0000 7f7ecffff640 1 -- 172.21.15.117:0/2122739264 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed00116d0 con 0x7f7ed8109eb0 2024-09-21T05:54:58.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.721+0000 7f7edc9a1640 1 -- 172.21.15.117:0/2122739264 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed81a6ea0 con 0x7f7ed8109eb0 2024-09-21T05:54:58.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.722+0000 7f7edc9a1640 1 -- 172.21.15.117:0/2122739264 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ed8102490 con 0x7f7ed8109eb0 2024-09-21T05:54:58.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.724+0000 7f7ecffff640 1 -- 172.21.15.117:0/2122739264 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f7ed00054a0 con 0x7f7ed8109eb0 2024-09-21T05:54:58.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.725+0000 7f7ecffff640 1 --2- 172.21.15.117:0/2122739264 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7ec40b6560 0x7f7ec40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:54:58.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.725+0000 7f7ed659c640 1 --2- 172.21.15.117:0/2122739264 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7ec40b6560 0x7f7ec40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:54:58.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.725+0000 7f7ecffff640 1 -- 172.21.15.117:0/2122739264 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7342+0+0 (secure 0 0 0) 0x7f7ed00e7550 con 0x7f7ed8109eb0 2024-09-21T05:54:58.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.725+0000 7f7ed659c640 1 --2- 172.21.15.117:0/2122739264 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7ec40b6560 0x7f7ec40b8a70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8031440 tx=0x7f7ec8009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:54:58.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.728+0000 7f7ecffff640 1 -- 172.21.15.117:0/2122739264 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7ed0016030 con 0x7f7ed8109eb0 2024-09-21T05:54:58.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.947+0000 7f7edc9a1640 1 -- 172.21.15.117:0/2122739264 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ed819b610 con 0x7f7ed8109eb0 2024-09-21T05:54:58.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.948+0000 7f7ecffff640 1 -- 172.21.15.117:0/2122739264 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8893 (secure 0 0 0) 0x7f7ed00a9030 con 0x7f7ed8109eb0 2024-09-21T05:54:58.950 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:54:58.950 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":11,"btime":"2024-09-21T05:54:58:553724+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3227057504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3227057504},{"type":"v1","addr":"172.21.15.117:6835","nonce":3227057504}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3303829183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3303829183},{"type":"v1","addr":"172.21.15.117:6837","nonce":3303829183}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/2314572411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2314572411},{"type":"v1","addr":"172.21.15.117:6839","nonce":2314572411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24545,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/3725629862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3725629862},{"type":"v1","addr":"172.21.15.138:6833","nonce":3725629862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24572,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/4276801396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4276801396},{"type":"v1","addr":"172.21.15.138:6835","nonce":4276801396}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24586,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/2224910879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2224910879},{"type":"v1","addr":"172.21.15.192:6833","nonce":2224910879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24602,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2540335374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2540335374},{"type":"v1","addr":"172.21.15.138:6837","nonce":2540335374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3530688860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3530688860},{"type":"v1","addr":"172.21.15.192:6835","nonce":3530688860}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24628,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2683860842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2683860842},{"type":"v1","addr":"172.21.15.192:6837","nonce":2683860842}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/1727623231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1727623231},{"type":"v1","addr":"172.21.15.192:6839","nonce":1727623231}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-21T05:54:58.950 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 11 2024-09-21T05:54:58.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.950+0000 7f7edc9a1640 1 -- 172.21.15.117:0/2122739264 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7ec40b6560 msgr2=0x7f7ec40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:58.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.950+0000 7f7edc9a1640 1 --2- 172.21.15.117:0/2122739264 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7ec40b6560 0x7f7ec40b8a70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8031440 tx=0x7f7ec8009560 comp rx=0 tx=0).stop 2024-09-21T05:54:58.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.950+0000 7f7edc9a1640 1 -- 172.21.15.117:0/2122739264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ed8109eb0 msgr2=0x7f7ed819fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:54:58.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.950+0000 7f7edc9a1640 1 --2- 172.21.15.117:0/2122739264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ed8109eb0 0x7f7ed819fd20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000eaa0 tx=0x7f7ed000ef60 comp rx=0 tx=0).stop 2024-09-21T05:54:58.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.951+0000 7f7edc9a1640 1 -- 172.21.15.117:0/2122739264 shutdown_connections 2024-09-21T05:54:58.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.951+0000 7f7edc9a1640 1 --2- 172.21.15.117:0/2122739264 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7ec40b6560 0x7f7ec40b8a70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:58.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.951+0000 7f7edc9a1640 1 --2- 172.21.15.117:0/2122739264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ed8109eb0 0x7f7ed819fd20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:58.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.951+0000 7f7edc9a1640 1 --2- 172.21.15.117:0/2122739264 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ed8109560 0x7f7ed819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:58.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.951+0000 7f7edc9a1640 1 --2- 172.21.15.117:0/2122739264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ed8102080 0x7f7ed819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:54:58.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.951+0000 7f7edc9a1640 1 -- 172.21.15.117:0/2122739264 >> 172.21.15.117:0/2122739264 conn(0x7f7ed8079ec0 msgr2=0x7f7ed81075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:54:58.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.951+0000 7f7edc9a1640 1 -- 172.21.15.117:0/2122739264 shutdown_connections 2024-09-21T05:54:58.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:54:58.951+0000 7f7edc9a1640 1 -- 172.21.15.117:0/2122739264 wait complete. 2024-09-21T05:54:59.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:58 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:59.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:58 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:59.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:58 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:59.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:58 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:54:59.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:58 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:54:59.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:58 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:59.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:58 smithi117 ceph-mon[30947]: Deploying daemon mds.a on smithi117 2024-09-21T05:54:59.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:58 smithi117 ceph-mon[30947]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:59.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:58 smithi117 ceph-mon[30947]: mds.? [v2:172.21.15.117:6838/2314572411,v1:172.21.15.117:6839/2314572411] up:boot 2024-09-21T05:54:59.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:58 smithi117 ceph-mon[30947]: fsmap 10 up:standby 2024-09-21T05:54:59.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:58 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T05:54:59.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:54:58 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2122739264' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:54:59.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:58 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:59.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:58 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:59.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:58 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:59.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:58 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:54:59.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:58 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:54:59.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:58 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:59.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:58 smithi138 ceph-mon[34391]: Deploying daemon mds.a on smithi117 2024-09-21T05:54:59.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:58 smithi138 ceph-mon[34391]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:59.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:58 smithi138 ceph-mon[34391]: mds.? [v2:172.21.15.117:6838/2314572411,v1:172.21.15.117:6839/2314572411] up:boot 2024-09-21T05:54:59.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:58 smithi138 ceph-mon[34391]: fsmap 10 up:standby 2024-09-21T05:54:59.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:58 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T05:54:59.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:54:58 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2122739264' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:54:59.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:58 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:59.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:58 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:59.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:58 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:54:59.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:58 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:54:59.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:58 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:54:59.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:58 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:54:59.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:58 smithi192 ceph-mon[34700]: Deploying daemon mds.a on smithi117 2024-09-21T05:54:59.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:58 smithi192 ceph-mon[34700]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:54:59.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:58 smithi192 ceph-mon[34700]: mds.? [v2:172.21.15.117:6838/2314572411,v1:172.21.15.117:6839/2314572411] up:boot 2024-09-21T05:54:59.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:58 smithi192 ceph-mon[34700]: fsmap 10 up:standby 2024-09-21T05:54:59.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:58 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T05:54:59.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:54:58 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2122739264' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:55:01.027 INFO:journalctl@ceph.mds.a.smithi117.stdout:Sep 21 05:55:00 smithi117 systemd[1]: Starting Ceph mds.a for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T05:55:01.402 INFO:journalctl@ceph.mds.a.smithi117.stdout:Sep 21 05:55:01 smithi117 podman[73328]: 2024-09-21 05:55:01.042474061 +0000 UTC m=+0.017770422 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T05:55:01.402 INFO:journalctl@ceph.mds.a.smithi117.stdout:Sep 21 05:55:01 smithi117 podman[73328]: 2024-09-21 05:55:01.173340368 +0000 UTC m=+0.148636760 container create 5c3fa963f450cd44f1ec6fda57ec5b020bfbea35433dec61991d7b2eb048807f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-a, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T05:55:01.402 INFO:journalctl@ceph.mds.a.smithi117.stdout:Sep 21 05:55:01 smithi117 podman[73328]: 2024-09-21 05:55:01.400857881 +0000 UTC m=+0.376154239 container init 5c3fa963f450cd44f1ec6fda57ec5b020bfbea35433dec61991d7b2eb048807f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-a, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T05:55:01.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:01 smithi117 ceph-mon[30947]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:55:01.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:01 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:01.777 INFO:journalctl@ceph.mds.a.smithi117.stdout:Sep 21 05:55:01 smithi117 podman[73328]: 2024-09-21 05:55:01.403595536 +0000 UTC m=+0.378891889 container start 5c3fa963f450cd44f1ec6fda57ec5b020bfbea35433dec61991d7b2eb048807f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-a, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T05:55:01.777 INFO:journalctl@ceph.mds.a.smithi117.stdout:Sep 21 05:55:01 smithi117 ceph-mds[73348]: -- 172.21.15.117:0/1270309164 <== mon.0 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5567a174b860 con 0x5567a58f3180 2024-09-21T05:55:01.777 INFO:journalctl@ceph.mds.a.smithi117.stdout:Sep 21 05:55:01 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-a[73344]: starting mds.a at 2024-09-21T05:55:01.777 INFO:journalctl@ceph.mds.a.smithi117.stdout:Sep 21 05:55:01 smithi117 bash[73328]: 5c3fa963f450cd44f1ec6fda57ec5b020bfbea35433dec61991d7b2eb048807f 2024-09-21T05:55:01.777 INFO:journalctl@ceph.mds.a.smithi117.stdout:Sep 21 05:55:01 smithi117 systemd[1]: Started Ceph mds.a for ac8581a2-77dc-11ef-baf5-efdc52797490. 2024-09-21T05:55:01.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:01 smithi138 ceph-mon[34391]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:55:01.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:01 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:01.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:01 smithi192 ceph-mon[34700]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:55:01.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:01 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:02.058 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T05:55:02.312 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:02.723 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:02 smithi117 ceph-mon[30947]: mds.? [v2:172.21.15.117:6840/3378096561,v1:172.21.15.117:6841/3378096561] up:boot 2024-09-21T05:55:02.723 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:02 smithi117 ceph-mon[30947]: fsmap 11 up:standby 2024-09-21T05:55:02.723 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:02 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T05:55:02.723 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:02 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:02.723 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:02 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:02.724 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:02 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:55:02.724 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:02 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:55:02.724 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:02 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:55:02.724 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:02 smithi117 ceph-mon[30947]: Deploying daemon mds.b on smithi138 2024-09-21T05:55:02.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.812+0000 7faa17122640 1 -- 172.21.15.117:0/697847376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa10108530 msgr2=0x7faa1010ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:02.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.812+0000 7faa17122640 1 --2- 172.21.15.117:0/697847376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa10108530 0x7faa1010ed60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7faa08009ca0 tx=0x7faa0802a8c0 comp rx=0 tx=0).stop 2024-09-21T05:55:02.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.812+0000 7faa17122640 1 -- 172.21.15.117:0/697847376 shutdown_connections 2024-09-21T05:55:02.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.812+0000 7faa17122640 1 --2- 172.21.15.117:0/697847376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa10108530 0x7faa1010ed60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:02.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.812+0000 7faa17122640 1 --2- 172.21.15.117:0/697847376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa10107be0 0x7faa10107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:02.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.812+0000 7faa17122640 1 --2- 172.21.15.117:0/697847376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa101022a0 0x7faa101026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:02.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.812+0000 7faa17122640 1 -- 172.21.15.117:0/697847376 >> 172.21.15.117:0/697847376 conn(0x7faa10079ec0 msgr2=0x7faa1007a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:02.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.812+0000 7faa17122640 1 -- 172.21.15.117:0/697847376 shutdown_connections 2024-09-21T05:55:02.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.812+0000 7faa17122640 1 -- 172.21.15.117:0/697847376 wait complete. 2024-09-21T05:55:02.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.813+0000 7faa17122640 1 Processor -- start 2024-09-21T05:55:02.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.813+0000 7faa17122640 1 -- start start 2024-09-21T05:55:02.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.813+0000 7faa17122640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa101022a0 0x7faa1019b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:02.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.813+0000 7faa17122640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa10107be0 0x7faa1019b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:02.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.813+0000 7faa17122640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa10108530 0x7faa1019bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:02.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.813+0000 7faa17122640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faa101121e0 con 0x7faa10108530 2024-09-21T05:55:02.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.813+0000 7faa17122640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faa10112060 con 0x7faa10107be0 2024-09-21T05:55:02.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.813+0000 7faa17122640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7faa10112360 con 0x7faa101022a0 2024-09-21T05:55:02.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.813+0000 7faa156bf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa10108530 0x7faa1019bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:02.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.813+0000 7faa156bf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa10108530 0x7faa1019bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57314/0 (socket says 172.21.15.117:57314) 2024-09-21T05:55:02.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.813+0000 7faa156bf640 1 -- 172.21.15.117:0/1208108402 learned_addr learned my addr 172.21.15.117:0/1208108402 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:55:02.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.813+0000 7faa0ffff640 1 --2- 172.21.15.117:0/1208108402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa10107be0 0x7faa1019b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:02.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.813+0000 7faa14ebe640 1 --2- 172.21.15.117:0/1208108402 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa101022a0 0x7faa1019b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:02.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.813+0000 7faa156bf640 1 -- 172.21.15.117:0/1208108402 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa101022a0 msgr2=0x7faa1019b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:02.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.813+0000 7faa156bf640 1 --2- 172.21.15.117:0/1208108402 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa101022a0 0x7faa1019b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:02.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.813+0000 7faa156bf640 1 -- 172.21.15.117:0/1208108402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa10107be0 msgr2=0x7faa1019b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:02.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.813+0000 7faa156bf640 1 --2- 172.21.15.117:0/1208108402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa10107be0 0x7faa1019b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:02.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.813+0000 7faa156bf640 1 -- 172.21.15.117:0/1208108402 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa101a64c0 con 0x7faa10108530 2024-09-21T05:55:02.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.813+0000 7faa14ebe640 1 --2- 172.21.15.117:0/1208108402 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa101022a0 0x7faa1019b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:55:02.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.814+0000 7faa156bf640 1 --2- 172.21.15.117:0/1208108402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa10108530 0x7faa1019bd40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7faa08005cc0 tx=0x7faa0802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:02.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.814+0000 7faa0dffb640 1 -- 172.21.15.117:0/1208108402 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa08003bf0 con 0x7faa10108530 2024-09-21T05:55:02.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.814+0000 7faa0dffb640 1 -- 172.21.15.117:0/1208108402 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa08003d90 con 0x7faa10108530 2024-09-21T05:55:02.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.814+0000 7faa17122640 1 -- 172.21.15.117:0/1208108402 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa101a6750 con 0x7faa10108530 2024-09-21T05:55:02.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.814+0000 7faa0dffb640 1 -- 172.21.15.117:0/1208108402 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0802fbc0 con 0x7faa10108530 2024-09-21T05:55:02.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.814+0000 7faa17122640 1 -- 172.21.15.117:0/1208108402 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa101a6ce0 con 0x7faa10108530 2024-09-21T05:55:02.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.815+0000 7faa17122640 1 -- 172.21.15.117:0/1208108402 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa101026b0 con 0x7faa10108530 2024-09-21T05:55:02.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.817+0000 7faa0dffb640 1 -- 172.21.15.117:0/1208108402 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7faa0802fd60 con 0x7faa10108530 2024-09-21T05:55:02.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.817+0000 7faa0dffb640 1 --2- 172.21.15.117:0/1208108402 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa9f80b6610 0x7fa9f80b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:02.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.818+0000 7faa14ebe640 1 --2- 172.21.15.117:0/1208108402 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa9f80b6610 0x7fa9f80b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:02.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.818+0000 7faa0dffb640 1 -- 172.21.15.117:0/1208108402 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7342+0+0 (secure 0 0 0) 0x7faa081083e0 con 0x7faa10108530 2024-09-21T05:55:02.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.818+0000 7faa14ebe640 1 --2- 172.21.15.117:0/1208108402 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa9f80b6610 0x7fa9f80b8b20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7faa00005fd0 tx=0x7faa00005f40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:02.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:02.821+0000 7faa0dffb640 1 -- 172.21.15.117:0/1208108402 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faa080cf040 con 0x7faa10108530 2024-09-21T05:55:02.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:02 smithi138 ceph-mon[34391]: mds.? [v2:172.21.15.117:6840/3378096561,v1:172.21.15.117:6841/3378096561] up:boot 2024-09-21T05:55:02.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:02 smithi138 ceph-mon[34391]: fsmap 11 up:standby 2024-09-21T05:55:02.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:02 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T05:55:02.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:02 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:02.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:02 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:02.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:02 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:55:02.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:02 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:55:02.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:02 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:55:02.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:02 smithi138 ceph-mon[34391]: Deploying daemon mds.b on smithi138 2024-09-21T05:55:02.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:02 smithi192 ceph-mon[34700]: mds.? [v2:172.21.15.117:6840/3378096561,v1:172.21.15.117:6841/3378096561] up:boot 2024-09-21T05:55:02.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:02 smithi192 ceph-mon[34700]: fsmap 11 up:standby 2024-09-21T05:55:02.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:02 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T05:55:02.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:02 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:02.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:02 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:02.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:02 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T05:55:02.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:02 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T05:55:02.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:02 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:55:02.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:02 smithi192 ceph-mon[34700]: Deploying daemon mds.b on smithi138 2024-09-21T05:55:03.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:03.033+0000 7faa17122640 1 -- 172.21.15.117:0/1208108402 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa10105e00 con 0x7faa10108530 2024-09-21T05:55:03.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:03.034+0000 7faa0dffb640 1 -- 172.21.15.117:0/1208108402 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9722 (secure 0 0 0) 0x7faa080098b0 con 0x7faa10108530 2024-09-21T05:55:03.036 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:55:03.036 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":12,"btime":"2024-09-21T05:55:01:552930+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3227057504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3227057504},{"type":"v1","addr":"172.21.15.117:6835","nonce":3227057504}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3303829183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3303829183},{"type":"v1","addr":"172.21.15.117:6837","nonce":3303829183}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/2314572411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2314572411},{"type":"v1","addr":"172.21.15.117:6839","nonce":2314572411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6841/3378096561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":3378096561},{"type":"v1","addr":"172.21.15.117:6841","nonce":3378096561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24545,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/3725629862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3725629862},{"type":"v1","addr":"172.21.15.138:6833","nonce":3725629862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24572,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/4276801396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4276801396},{"type":"v1","addr":"172.21.15.138:6835","nonce":4276801396}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24586,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/2224910879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2224910879},{"type":"v1","addr":"172.21.15.192:6833","nonce":2224910879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24602,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2540335374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2540335374},{"type":"v1","addr":"172.21.15.138:6837","nonce":2540335374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3530688860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3530688860},{"type":"v1","addr":"172.21.15.192:6835","nonce":3530688860}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24628,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2683860842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2683860842},{"type":"v1","addr":"172.21.15.192:6837","nonce":2683860842}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/1727623231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1727623231},{"type":"v1","addr":"172.21.15.192:6839","nonce":1727623231}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-21T05:55:03.036 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 12 2024-09-21T05:55:03.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:03.036+0000 7faa17122640 1 -- 172.21.15.117:0/1208108402 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa9f80b6610 msgr2=0x7fa9f80b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:03.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:03.036+0000 7faa17122640 1 --2- 172.21.15.117:0/1208108402 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa9f80b6610 0x7fa9f80b8b20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7faa00005fd0 tx=0x7faa00005f40 comp rx=0 tx=0).stop 2024-09-21T05:55:03.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:03.036+0000 7faa17122640 1 -- 172.21.15.117:0/1208108402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa10108530 msgr2=0x7faa1019bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:03.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:03.036+0000 7faa17122640 1 --2- 172.21.15.117:0/1208108402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa10108530 0x7faa1019bd40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7faa08005cc0 tx=0x7faa0802f380 comp rx=0 tx=0).stop 2024-09-21T05:55:03.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:03.037+0000 7faa17122640 1 -- 172.21.15.117:0/1208108402 shutdown_connections 2024-09-21T05:55:03.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:03.037+0000 7faa17122640 1 --2- 172.21.15.117:0/1208108402 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa9f80b6610 0x7fa9f80b8b20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:03.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:03.037+0000 7faa17122640 1 --2- 172.21.15.117:0/1208108402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa10108530 0x7faa1019bd40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:03.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:03.037+0000 7faa17122640 1 --2- 172.21.15.117:0/1208108402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa10107be0 0x7faa1019b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:03.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:03.037+0000 7faa17122640 1 --2- 172.21.15.117:0/1208108402 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa101022a0 0x7faa1019b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:03.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:03.037+0000 7faa17122640 1 -- 172.21.15.117:0/1208108402 >> 172.21.15.117:0/1208108402 conn(0x7faa10079ec0 msgr2=0x7faa10100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:03.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:03.037+0000 7faa17122640 1 -- 172.21.15.117:0/1208108402 shutdown_connections 2024-09-21T05:55:03.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:03.037+0000 7faa17122640 1 -- 172.21.15.117:0/1208108402 wait complete. 2024-09-21T05:55:03.688 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:03 smithi117 ceph-mon[30947]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:55:03.688 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:03 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/1208108402' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:55:03.817 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:03 smithi138 ceph-mon[34391]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:55:03.817 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:03 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/1208108402' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:55:03.817 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 21 05:55:03 smithi138 systemd[1]: Starting Ceph mds.b for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T05:55:03.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:03 smithi192 ceph-mon[34700]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:55:03.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:03 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/1208108402' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:55:04.075 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 21 05:55:03 smithi138 podman[63779]: 2024-09-21 05:55:03.71785982 +0000 UTC m=+0.016065064 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T05:55:04.076 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 21 05:55:03 smithi138 podman[63779]: 2024-09-21 05:55:03.830014864 +0000 UTC m=+0.128220164 container create 76f90b8f9d6932cd517acf6a6fee540b9de95a82a2094a9889cbf9aa691732e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-b, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-09-21T05:55:04.076 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 21 05:55:04 smithi138 podman[63779]: 2024-09-21 05:55:04.074392252 +0000 UTC m=+0.372597487 container init 76f90b8f9d6932cd517acf6a6fee540b9de95a82a2094a9889cbf9aa691732e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-b, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD) 2024-09-21T05:55:04.365 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 21 05:55:04 smithi138 podman[63779]: 2024-09-21 05:55:04.07763359 +0000 UTC m=+0.375838826 container start 76f90b8f9d6932cd517acf6a6fee540b9de95a82a2094a9889cbf9aa691732e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-21T05:55:04.365 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 21 05:55:04 smithi138 ceph-mds[63798]: -- 172.21.15.138:0/3699480793 <== mon.2 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ba41645c20 con 0x55ba457eda80 2024-09-21T05:55:04.365 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 21 05:55:04 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-b[63794]: starting mds.b at 2024-09-21T05:55:04.365 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 21 05:55:04 smithi138 bash[63779]: 76f90b8f9d6932cd517acf6a6fee540b9de95a82a2094a9889cbf9aa691732e9 2024-09-21T05:55:04.365 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 21 05:55:04 smithi138 systemd[1]: Started Ceph mds.b for ac8581a2-77dc-11ef-baf5-efdc52797490. 2024-09-21T05:55:05.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:05 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:05.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:05 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:05.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:05 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:05.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:05 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:05.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:05 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:05.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:05 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:55:05.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:05 smithi192 ceph-mon[34700]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:55:05.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:05 smithi192 ceph-mon[34700]: mds.? [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] up:boot 2024-09-21T05:55:05.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:05 smithi192 ceph-mon[34700]: fsmap 12 up:standby 2024-09-21T05:55:05.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:05 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T05:55:05.526 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:05 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:05.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:05 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:05.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:05 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:05.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:05 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:05.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:05 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:05.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:05 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:55:05.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:05 smithi117 ceph-mon[30947]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:55:05.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:05 smithi117 ceph-mon[30947]: mds.? [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] up:boot 2024-09-21T05:55:05.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:05 smithi117 ceph-mon[30947]: fsmap 12 up:standby 2024-09-21T05:55:05.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:05 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T05:55:05.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:05 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:05.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:05 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:05.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:05 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:05.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:05 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:05.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:05 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:05.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:05 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:55:05.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:05 smithi138 ceph-mon[34391]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:55:05.616 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:05 smithi138 ceph-mon[34391]: mds.? [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] up:boot 2024-09-21T05:55:05.617 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:05 smithi138 ceph-mon[34391]: fsmap 12 up:standby 2024-09-21T05:55:05.617 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:05 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T05:55:05.719 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T05:55:05.984 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:06.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.510+0000 7faa5ed85640 1 -- 172.21.15.117:0/1501954331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa581083c0 msgr2=0x7faa581087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:06.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.510+0000 7faa5ed85640 1 --2- 172.21.15.117:0/1501954331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa581083c0 0x7faa581087d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7faa44009900 tx=0x7faa4402a410 comp rx=0 tx=0).stop 2024-09-21T05:55:06.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.511+0000 7faa5ed85640 1 -- 172.21.15.117:0/1501954331 shutdown_connections 2024-09-21T05:55:06.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.511+0000 7faa5ed85640 1 --2- 172.21.15.117:0/1501954331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa581083c0 0x7faa581087d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:06.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.511+0000 7faa5ed85640 1 --2- 172.21.15.117:0/1501954331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa58108d10 0x7faa5810f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:06.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.511+0000 7faa5ed85640 1 --2- 172.21.15.117:0/1501954331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa58102a80 0x7faa58102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:06.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.511+0000 7faa5ed85640 1 -- 172.21.15.117:0/1501954331 >> 172.21.15.117:0/1501954331 conn(0x7faa58079740 msgr2=0x7faa58079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:06.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.511+0000 7faa5ed85640 1 -- 172.21.15.117:0/1501954331 shutdown_connections 2024-09-21T05:55:06.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.511+0000 7faa5ed85640 1 -- 172.21.15.117:0/1501954331 wait complete. 2024-09-21T05:55:06.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.511+0000 7faa5ed85640 1 Processor -- start 2024-09-21T05:55:06.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.511+0000 7faa5ed85640 1 -- start start 2024-09-21T05:55:06.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.511+0000 7faa5ed85640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa58102a80 0x7faa5819c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:06.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.511+0000 7faa5ed85640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa581083c0 0x7faa5819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:06.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.511+0000 7faa5ed85640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa58108d10 0x7faa581a1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:06.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.512+0000 7faa5ed85640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faa58113aa0 con 0x7faa58108d10 2024-09-21T05:55:06.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.512+0000 7faa5ed85640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faa58113920 con 0x7faa58102a80 2024-09-21T05:55:06.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.512+0000 7faa5ed85640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7faa58113c20 con 0x7faa581083c0 2024-09-21T05:55:06.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.512+0000 7faa5d322640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa58108d10 0x7faa581a1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:06.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.512+0000 7faa5d322640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa58108d10 0x7faa581a1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51256/0 (socket says 172.21.15.117:51256) 2024-09-21T05:55:06.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.512+0000 7faa5d322640 1 -- 172.21.15.117:0/3112797668 learned_addr learned my addr 172.21.15.117:0/3112797668 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:55:06.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.512+0000 7faa57fff640 1 --2- 172.21.15.117:0/3112797668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa581083c0 0x7faa5819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:06.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.512+0000 7faa5cb21640 1 --2- 172.21.15.117:0/3112797668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa58102a80 0x7faa5819c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:06.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.512+0000 7faa5d322640 1 -- 172.21.15.117:0/3112797668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa581083c0 msgr2=0x7faa5819ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:06.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.512+0000 7faa5d322640 1 --2- 172.21.15.117:0/3112797668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa581083c0 0x7faa5819ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:06.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.512+0000 7faa5d322640 1 -- 172.21.15.117:0/3112797668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa58102a80 msgr2=0x7faa5819c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:06.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.512+0000 7faa5d322640 1 --2- 172.21.15.117:0/3112797668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa58102a80 0x7faa5819c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:06.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.512+0000 7faa5d322640 1 -- 172.21.15.117:0/3112797668 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa581a26a0 con 0x7faa58108d10 2024-09-21T05:55:06.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.512+0000 7faa5d322640 1 --2- 172.21.15.117:0/3112797668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa58108d10 0x7faa581a1f70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7faa5000eaa0 tx=0x7faa5000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:06.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.512+0000 7faa55ffb640 1 -- 172.21.15.117:0/3112797668 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5000cdb0 con 0x7faa58108d10 2024-09-21T05:55:06.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.512+0000 7faa5ed85640 1 -- 172.21.15.117:0/3112797668 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa581a2990 con 0x7faa58108d10 2024-09-21T05:55:06.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.512+0000 7faa55ffb640 1 -- 172.21.15.117:0/3112797668 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa50005020 con 0x7faa58108d10 2024-09-21T05:55:06.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.512+0000 7faa55ffb640 1 -- 172.21.15.117:0/3112797668 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa500116d0 con 0x7faa58108d10 2024-09-21T05:55:06.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.512+0000 7faa5ed85640 1 -- 172.21.15.117:0/3112797668 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa581a90f0 con 0x7faa58108d10 2024-09-21T05:55:06.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.513+0000 7faa5ed85640 1 -- 172.21.15.117:0/3112797668 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa58103d00 con 0x7faa58108d10 2024-09-21T05:55:06.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.515+0000 7faa55ffb640 1 -- 172.21.15.117:0/3112797668 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7faa500054a0 con 0x7faa58108d10 2024-09-21T05:55:06.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.516+0000 7faa55ffb640 1 --2- 172.21.15.117:0/3112797668 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7faa480b6560 0x7faa480b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:06.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.516+0000 7faa5cb21640 1 --2- 172.21.15.117:0/3112797668 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7faa480b6560 0x7faa480b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:06.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.516+0000 7faa55ffb640 1 -- 172.21.15.117:0/3112797668 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7342+0+0 (secure 0 0 0) 0x7faa500e7440 con 0x7faa58108d10 2024-09-21T05:55:06.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.516+0000 7faa5cb21640 1 --2- 172.21.15.117:0/3112797668 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7faa480b6560 0x7faa480b8a70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7faa4c009bd0 tx=0x7faa4c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:06.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.519+0000 7faa55ffb640 1 -- 172.21.15.117:0/3112797668 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faa50016030 con 0x7faa58108d10 2024-09-21T05:55:06.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.750+0000 7faa5ed85640 1 -- 172.21.15.117:0/3112797668 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa58102e90 con 0x7faa58108d10 2024-09-21T05:55:06.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.752+0000 7faa55ffb640 1 -- 172.21.15.117:0/3112797668 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10551 (secure 0 0 0) 0x7faa58102e90 con 0x7faa58108d10 2024-09-21T05:55:06.754 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:55:06.754 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":13,"btime":"2024-09-21T05:55:04:582011+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3227057504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3227057504},{"type":"v1","addr":"172.21.15.117:6835","nonce":3227057504}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3303829183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3303829183},{"type":"v1","addr":"172.21.15.117:6837","nonce":3303829183}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14838,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/2314572411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2314572411},{"type":"v1","addr":"172.21.15.117:6839","nonce":2314572411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6841/3378096561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":3378096561},{"type":"v1","addr":"172.21.15.117:6841","nonce":3378096561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24545,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/3725629862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3725629862},{"type":"v1","addr":"172.21.15.138:6833","nonce":3725629862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24572,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/4276801396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4276801396},{"type":"v1","addr":"172.21.15.138:6835","nonce":4276801396}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24586,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/2224910879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2224910879},{"type":"v1","addr":"172.21.15.192:6833","nonce":2224910879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24602,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2540335374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2540335374},{"type":"v1","addr":"172.21.15.138:6837","nonce":2540335374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24616,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3530688860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3530688860},{"type":"v1","addr":"172.21.15.192:6835","nonce":3530688860}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24628,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2683860842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2683860842},{"type":"v1","addr":"172.21.15.192:6837","nonce":2683860842}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24637,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/1727623231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1727623231},{"type":"v1","addr":"172.21.15.192:6839","nonce":1727623231}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6839/3230372576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":3230372576},{"type":"v1","addr":"172.21.15.138:6839","nonce":3230372576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-21T05:55:06.754 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 13 2024-09-21T05:55:06.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.754+0000 7faa5ed85640 1 -- 172.21.15.117:0/3112797668 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7faa480b6560 msgr2=0x7faa480b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:06.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.754+0000 7faa5ed85640 1 --2- 172.21.15.117:0/3112797668 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7faa480b6560 0x7faa480b8a70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7faa4c009bd0 tx=0x7faa4c005f90 comp rx=0 tx=0).stop 2024-09-21T05:55:06.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.754+0000 7faa5ed85640 1 -- 172.21.15.117:0/3112797668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa58108d10 msgr2=0x7faa581a1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:06.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.754+0000 7faa5ed85640 1 --2- 172.21.15.117:0/3112797668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa58108d10 0x7faa581a1f70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7faa5000eaa0 tx=0x7faa5000ef60 comp rx=0 tx=0).stop 2024-09-21T05:55:06.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.754+0000 7faa5ed85640 1 -- 172.21.15.117:0/3112797668 shutdown_connections 2024-09-21T05:55:06.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.754+0000 7faa5ed85640 1 --2- 172.21.15.117:0/3112797668 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7faa480b6560 0x7faa480b8a70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:06.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.754+0000 7faa5ed85640 1 --2- 172.21.15.117:0/3112797668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa58108d10 0x7faa581a1f70 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:06.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.754+0000 7faa5ed85640 1 --2- 172.21.15.117:0/3112797668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa581083c0 0x7faa5819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:06.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.754+0000 7faa5ed85640 1 --2- 172.21.15.117:0/3112797668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa58102a80 0x7faa5819c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:06.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.754+0000 7faa5ed85640 1 -- 172.21.15.117:0/3112797668 >> 172.21.15.117:0/3112797668 conn(0x7faa58079740 msgr2=0x7faa58104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:06.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.754+0000 7faa5ed85640 1 -- 172.21.15.117:0/3112797668 shutdown_connections 2024-09-21T05:55:06.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:06.754+0000 7faa5ed85640 1 -- 172.21.15.117:0/3112797668 wait complete. 2024-09-21T05:55:07.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:06 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:07.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:06 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:07.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:06 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:07.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:06 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:07.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:06 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:07.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:06 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:07.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:06 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:55:07.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:06 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:55:07.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:06 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:07.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:06 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:55:07.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:06 smithi117 ceph-mon[30947]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:55:07.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:06 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:07.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:06 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:07.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:06 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:07.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:06 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:07.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:06 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:07.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:06 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:07.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:06 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:55:07.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:06 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:55:07.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:06 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:07.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:06 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:55:07.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:06 smithi138 ceph-mon[34391]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:55:07.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:06 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:07.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:06 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:07.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:06 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:07.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:06 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:07.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:06 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:07.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:06 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:07.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:06 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:55:07.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:06 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:55:07.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:06 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:07.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:06 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:55:07.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:06 smithi192 ceph-mon[34700]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:55:08.015 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:07 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3112797668' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:55:08.016 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:07 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:08.016 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:07 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:08.016 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:07 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:08.016 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:07 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:08.016 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:07 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:08.043 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph --format=json-pretty osd lspools 2024-09-21T05:55:08.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:07 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3112797668' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:55:08.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:07 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:08.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:07 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:08.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:07 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:08.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:07 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:08.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:07 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:08.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:07 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3112797668' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:55:08.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:07 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:08.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:07 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:08.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:07 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:08.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:07 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:08.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:07 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:08.310 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:08.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.807+0000 7f7b8f135640 1 -- 172.21.15.117:0/1930727220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b8810de10 msgr2=0x7f7b881102f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:08.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.807+0000 7f7b8f135640 1 --2- 172.21.15.117:0/1930727220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b8810de10 0x7f7b881102f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7b70009900 tx=0x7f7b7002a410 comp rx=0 tx=0).stop 2024-09-21T05:55:08.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.807+0000 7f7b8f135640 1 -- 172.21.15.117:0/1930727220 shutdown_connections 2024-09-21T05:55:08.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.807+0000 7f7b8f135640 1 --2- 172.21.15.117:0/1930727220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b8810de10 0x7f7b881102f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:08.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.807+0000 7f7b8f135640 1 --2- 172.21.15.117:0/1930727220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b88105290 0x7f7b8810d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:08.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.807+0000 7f7b8f135640 1 --2- 172.21.15.117:0/1930727220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b88104860 0x7f7b88104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:08.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.807+0000 7f7b8f135640 1 -- 172.21.15.117:0/1930727220 >> 172.21.15.117:0/1930727220 conn(0x7f7b88079730 msgr2=0x7f7b88079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:08.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.807+0000 7f7b8f135640 1 -- 172.21.15.117:0/1930727220 shutdown_connections 2024-09-21T05:55:08.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.807+0000 7f7b8f135640 1 -- 172.21.15.117:0/1930727220 wait complete. 2024-09-21T05:55:08.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.808+0000 7f7b8f135640 1 Processor -- start 2024-09-21T05:55:08.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.808+0000 7f7b8f135640 1 -- start start 2024-09-21T05:55:08.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.808+0000 7f7b8f135640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b88104860 0x7f7b8819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:08.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.808+0000 7f7b8f135640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b88105290 0x7f7b8819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:08.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.808+0000 7f7b8f135640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b8810de10 0x7f7b8819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:08.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.808+0000 7f7b8f135640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7b88113ac0 con 0x7f7b88104860 2024-09-21T05:55:08.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.808+0000 7f7b8f135640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7b88113940 con 0x7f7b8810de10 2024-09-21T05:55:08.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.808+0000 7f7b8f135640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7b88113c40 con 0x7f7b88105290 2024-09-21T05:55:08.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.808+0000 7f7b8ced1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b88104860 0x7f7b8819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:08.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.808+0000 7f7b8ced1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b88104860 0x7f7b8819a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51274/0 (socket says 172.21.15.117:51274) 2024-09-21T05:55:08.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.808+0000 7f7b8ced1640 1 -- 172.21.15.117:0/3350818156 learned_addr learned my addr 172.21.15.117:0/3350818156 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:55:08.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.808+0000 7f7b87fff640 1 --2- 172.21.15.117:0/3350818156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b88105290 0x7f7b8819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:08.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.809+0000 7f7b8ced1640 1 -- 172.21.15.117:0/3350818156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b88105290 msgr2=0x7f7b8819aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:08.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.809+0000 7f7b8d6d2640 1 --2- 172.21.15.117:0/3350818156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b8810de10 0x7f7b8819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:08.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.809+0000 7f7b8ced1640 1 --2- 172.21.15.117:0/3350818156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b88105290 0x7f7b8819aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:08.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.809+0000 7f7b8ced1640 1 -- 172.21.15.117:0/3350818156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b8810de10 msgr2=0x7f7b8819fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:08.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.809+0000 7f7b8ced1640 1 --2- 172.21.15.117:0/3350818156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b8810de10 0x7f7b8819fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:08.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.809+0000 7f7b8ced1640 1 -- 172.21.15.117:0/3350818156 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b881a0460 con 0x7f7b88104860 2024-09-21T05:55:08.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.809+0000 7f7b8ced1640 1 --2- 172.21.15.117:0/3350818156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b88104860 0x7f7b8819a4f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000d950 tx=0x7f7b8000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:08.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.809+0000 7f7b85ffb640 1 -- 172.21.15.117:0/3350818156 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80017030 con 0x7f7b88104860 2024-09-21T05:55:08.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.809+0000 7f7b85ffb640 1 -- 172.21.15.117:0/3350818156 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b80004020 con 0x7f7b88104860 2024-09-21T05:55:08.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.809+0000 7f7b85ffb640 1 -- 172.21.15.117:0/3350818156 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b800047a0 con 0x7f7b88104860 2024-09-21T05:55:08.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.809+0000 7f7b8f135640 1 -- 172.21.15.117:0/3350818156 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b881a0750 con 0x7f7b88104860 2024-09-21T05:55:08.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.809+0000 7f7b8f135640 1 -- 172.21.15.117:0/3350818156 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b881a6eb0 con 0x7f7b88104860 2024-09-21T05:55:08.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.810+0000 7f7b8f135640 1 -- 172.21.15.117:0/3350818156 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b88104c70 con 0x7f7b88104860 2024-09-21T05:55:08.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.812+0000 7f7b85ffb640 1 -- 172.21.15.117:0/3350818156 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f7b80009a30 con 0x7f7b88104860 2024-09-21T05:55:08.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.812+0000 7f7b85ffb640 1 --2- 172.21.15.117:0/3350818156 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7b740b65e0 0x7f7b740b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:08.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.812+0000 7f7b87fff640 1 --2- 172.21.15.117:0/3350818156 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7b740b65e0 0x7f7b740b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:08.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.813+0000 7f7b85ffb640 1 -- 172.21.15.117:0/3350818156 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7342+0+0 (secure 0 0 0) 0x7f7b800e6fb0 con 0x7f7b88104860 2024-09-21T05:55:08.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.813+0000 7f7b87fff640 1 --2- 172.21.15.117:0/3350818156 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7b740b65e0 0x7f7b740b8af0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7b8810fa90 tx=0x7f7b7800d040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:08.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:08.815+0000 7f7b85ffb640 1 -- 172.21.15.117:0/3350818156 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7b800adc90 con 0x7f7b88104860 2024-09-21T05:55:09.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:09.008+0000 7f7b8f135640 1 -- 172.21.15.117:0/3350818156 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7b8819b620 con 0x7f7b88104860 2024-09-21T05:55:09.012 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:08 smithi117 ceph-mon[30947]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:55:09.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:09.009+0000 7f7b85ffb640 1 -- 172.21.15.117:0/3350818156 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v69) ==== 84+0+65 (secure 0 0 0) 0x7f7b800b2070 con 0x7f7b88104860 2024-09-21T05:55:09.012 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:55:09.012 INFO:teuthology.orchestra.run.smithi117.stdout:[ 2024-09-21T05:55:09.012 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T05:55:09.012 INFO:teuthology.orchestra.run.smithi117.stdout: "poolnum": 1, 2024-09-21T05:55:09.012 INFO:teuthology.orchestra.run.smithi117.stdout: "poolname": ".mgr" 2024-09-21T05:55:09.012 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T05:55:09.013 INFO:teuthology.orchestra.run.smithi117.stdout:] 2024-09-21T05:55:09.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:09.012+0000 7f7b8f135640 1 -- 172.21.15.117:0/3350818156 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7b740b65e0 msgr2=0x7f7b740b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:09.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:09.012+0000 7f7b8f135640 1 --2- 172.21.15.117:0/3350818156 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7b740b65e0 0x7f7b740b8af0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7b8810fa90 tx=0x7f7b7800d040 comp rx=0 tx=0).stop 2024-09-21T05:55:09.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:09.012+0000 7f7b8f135640 1 -- 172.21.15.117:0/3350818156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b88104860 msgr2=0x7f7b8819a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:09.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:09.012+0000 7f7b8f135640 1 --2- 172.21.15.117:0/3350818156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b88104860 0x7f7b8819a4f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000d950 tx=0x7f7b8000de10 comp rx=0 tx=0).stop 2024-09-21T05:55:09.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:09.012+0000 7f7b8f135640 1 -- 172.21.15.117:0/3350818156 shutdown_connections 2024-09-21T05:55:09.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:09.012+0000 7f7b8f135640 1 --2- 172.21.15.117:0/3350818156 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7b740b65e0 0x7f7b740b8af0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:09.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:09.012+0000 7f7b8f135640 1 --2- 172.21.15.117:0/3350818156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b8810de10 0x7f7b8819fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:09.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:09.012+0000 7f7b8f135640 1 --2- 172.21.15.117:0/3350818156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b88105290 0x7f7b8819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:09.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:09.012+0000 7f7b8f135640 1 --2- 172.21.15.117:0/3350818156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b88104860 0x7f7b8819a4f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:09.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:09.012+0000 7f7b8f135640 1 -- 172.21.15.117:0/3350818156 >> 172.21.15.117:0/3350818156 conn(0x7f7b88079730 msgr2=0x7f7b88107980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:09.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:09.012+0000 7f7b8f135640 1 -- 172.21.15.117:0/3350818156 shutdown_connections 2024-09-21T05:55:09.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:09.012+0000 7f7b8f135640 1 -- 172.21.15.117:0/3350818156 wait complete. 2024-09-21T05:55:09.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:08 smithi138 ceph-mon[34391]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:55:09.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:08 smithi192 ceph-mon[34700]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:55:09.635 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T05:55:09.635 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T05:55:09.882 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:10.207 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:09 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:10.207 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:09 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:10.207 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:09 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:55:10.208 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:09 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:55:10.208 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:09 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:10.208 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:09 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3350818156' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T05:55:10.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:09 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:10.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:09 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:10.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:09 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:55:10.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:09 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:55:10.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:09 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:10.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:09 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3350818156' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T05:55:10.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:09 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:10.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:09 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:10.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:09 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:55:10.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:09 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:55:10.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:09 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:55:10.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:09 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3350818156' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T05:55:10.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.413+0000 7f1e1ea69640 1 -- 172.21.15.117:0/2539926771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e1810de20 msgr2=0x7f1e18110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:10.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.413+0000 7f1e1ea69640 1 --2- 172.21.15.117:0/2539926771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e1810de20 0x7f1e18110330 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1e10009ca0 tx=0x7f1e1002a8c0 comp rx=0 tx=0).stop 2024-09-21T05:55:10.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.413+0000 7f1e1ea69640 1 -- 172.21.15.117:0/2539926771 shutdown_connections 2024-09-21T05:55:10.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.413+0000 7f1e1ea69640 1 --2- 172.21.15.117:0/2539926771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e1810de20 0x7f1e18110330 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:10.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.413+0000 7f1e1ea69640 1 --2- 172.21.15.117:0/2539926771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e181052a0 0x7f1e1810d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:10.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.413+0000 7f1e1ea69640 1 --2- 172.21.15.117:0/2539926771 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e18104870 0x7f1e18104c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:10.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.413+0000 7f1e1ea69640 1 -- 172.21.15.117:0/2539926771 >> 172.21.15.117:0/2539926771 conn(0x7f1e18079750 msgr2=0x7f1e18079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:10.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.413+0000 7f1e1ea69640 1 -- 172.21.15.117:0/2539926771 shutdown_connections 2024-09-21T05:55:10.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.413+0000 7f1e1ea69640 1 -- 172.21.15.117:0/2539926771 wait complete. 2024-09-21T05:55:10.418 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.413+0000 7f1e1ea69640 1 Processor -- start 2024-09-21T05:55:10.418 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.413+0000 7f1e1ea69640 1 -- start start 2024-09-21T05:55:10.418 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.414+0000 7f1e1ea69640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e18104870 0x7f1e1819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:10.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.414+0000 7f1e1ea69640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e181052a0 0x7f1e1819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:10.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.414+0000 7f1e1ea69640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e1810de20 0x7f1e1819fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:10.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.414+0000 7f1e1ea69640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1e18113b40 con 0x7f1e181052a0 2024-09-21T05:55:10.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.414+0000 7f1e1ea69640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1e181139c0 con 0x7f1e1810de20 2024-09-21T05:55:10.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.414+0000 7f1e1ea69640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1e18113cc0 con 0x7f1e18104870 2024-09-21T05:55:10.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.414+0000 7f1e17fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e181052a0 0x7f1e1819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:10.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.414+0000 7f1e17fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e181052a0 0x7f1e1819ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51304/0 (socket says 172.21.15.117:51304) 2024-09-21T05:55:10.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.414+0000 7f1e17fff640 1 -- 172.21.15.117:0/3906628273 learned_addr learned my addr 172.21.15.117:0/3906628273 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:55:10.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.414+0000 7f1e1d006640 1 --2- 172.21.15.117:0/3906628273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e1810de20 0x7f1e1819fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:10.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.414+0000 7f1e1c805640 1 --2- 172.21.15.117:0/3906628273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e18104870 0x7f1e1819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:10.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.414+0000 7f1e17fff640 1 -- 172.21.15.117:0/3906628273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e18104870 msgr2=0x7f1e1819a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:10.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.414+0000 7f1e17fff640 1 --2- 172.21.15.117:0/3906628273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e18104870 0x7f1e1819a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:10.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.414+0000 7f1e17fff640 1 -- 172.21.15.117:0/3906628273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e1810de20 msgr2=0x7f1e1819fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:10.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.414+0000 7f1e17fff640 1 --2- 172.21.15.117:0/3906628273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e1810de20 0x7f1e1819fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:10.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.414+0000 7f1e17fff640 1 -- 172.21.15.117:0/3906628273 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e181a04e0 con 0x7f1e181052a0 2024-09-21T05:55:10.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.414+0000 7f1e1c805640 1 --2- 172.21.15.117:0/3906628273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e18104870 0x7f1e1819a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:55:10.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.414+0000 7f1e17fff640 1 --2- 172.21.15.117:0/3906628273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e181052a0 0x7f1e1819ab10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1e0400d740 tx=0x7f1e0400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:10.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.414+0000 7f1e15ffb640 1 -- 172.21.15.117:0/3906628273 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e04017070 con 0x7f1e181052a0 2024-09-21T05:55:10.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.414+0000 7f1e15ffb640 1 -- 172.21.15.117:0/3906628273 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e0400de50 con 0x7f1e181052a0 2024-09-21T05:55:10.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.414+0000 7f1e1ea69640 1 -- 172.21.15.117:0/3906628273 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e181a07d0 con 0x7f1e181052a0 2024-09-21T05:55:10.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.415+0000 7f1e15ffb640 1 -- 172.21.15.117:0/3906628273 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0400f620 con 0x7f1e181052a0 2024-09-21T05:55:10.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.415+0000 7f1e1ea69640 1 -- 172.21.15.117:0/3906628273 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e181a6f30 con 0x7f1e181052a0 2024-09-21T05:55:10.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.416+0000 7f1e1ea69640 1 -- 172.21.15.117:0/3906628273 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e1806c1c0 con 0x7f1e181052a0 2024-09-21T05:55:10.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.417+0000 7f1e15ffb640 1 -- 172.21.15.117:0/3906628273 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f1e0401a070 con 0x7f1e181052a0 2024-09-21T05:55:10.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.418+0000 7f1e15ffb640 1 --2- 172.21.15.117:0/3906628273 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1e080b6560 0x7f1e080b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:10.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.418+0000 7f1e1c805640 1 --2- 172.21.15.117:0/3906628273 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1e080b6560 0x7f1e080b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:10.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.418+0000 7f1e15ffb640 1 -- 172.21.15.117:0/3906628273 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7342+0+0 (secure 0 0 0) 0x7f1e040e6970 con 0x7f1e181052a0 2024-09-21T05:55:10.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.419+0000 7f1e1c805640 1 --2- 172.21.15.117:0/3906628273 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1e080b6560 0x7f1e080b8a70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c006fd0 tx=0x7f1e0c008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:10.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.421+0000 7f1e15ffb640 1 -- 172.21.15.117:0/3906628273 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1e040ad5d0 con 0x7f1e181052a0 2024-09-21T05:55:10.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.598+0000 7f1e1ea69640 1 -- 172.21.15.117:0/3906628273 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f1e1819b6a0 con 0x7f1e181052a0 2024-09-21T05:55:10.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.958+0000 7f1e15ffb640 1 -- 172.21.15.117:0/3906628273 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v70) ==== 138+0+0 (secure 0 0 0) 0x7f1e040b19b0 con 0x7f1e181052a0 2024-09-21T05:55:10.961 INFO:teuthology.orchestra.run.smithi117.stderr:pool 'cephfs_metadata' created 2024-09-21T05:55:10.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.961+0000 7f1e1ea69640 1 -- 172.21.15.117:0/3906628273 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1e080b6560 msgr2=0x7f1e080b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:10.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.961+0000 7f1e1ea69640 1 --2- 172.21.15.117:0/3906628273 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1e080b6560 0x7f1e080b8a70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c006fd0 tx=0x7f1e0c008040 comp rx=0 tx=0).stop 2024-09-21T05:55:10.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.961+0000 7f1e1ea69640 1 -- 172.21.15.117:0/3906628273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e181052a0 msgr2=0x7f1e1819ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:10.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.961+0000 7f1e1ea69640 1 --2- 172.21.15.117:0/3906628273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e181052a0 0x7f1e1819ab10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1e0400d740 tx=0x7f1e0400dc00 comp rx=0 tx=0).stop 2024-09-21T05:55:10.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.961+0000 7f1e1ea69640 1 -- 172.21.15.117:0/3906628273 shutdown_connections 2024-09-21T05:55:10.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.961+0000 7f1e1ea69640 1 --2- 172.21.15.117:0/3906628273 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1e080b6560 0x7f1e080b8a70 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:10.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.961+0000 7f1e1ea69640 1 --2- 172.21.15.117:0/3906628273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e181052a0 0x7f1e1819ab10 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:10.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.961+0000 7f1e1ea69640 1 --2- 172.21.15.117:0/3906628273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e1810de20 0x7f1e1819fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:10.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.961+0000 7f1e1ea69640 1 --2- 172.21.15.117:0/3906628273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e18104870 0x7f1e1819a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:10.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.961+0000 7f1e1ea69640 1 -- 172.21.15.117:0/3906628273 >> 172.21.15.117:0/3906628273 conn(0x7f1e18079750 msgr2=0x7f1e18107930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:10.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.961+0000 7f1e1ea69640 1 -- 172.21.15.117:0/3906628273 shutdown_connections 2024-09-21T05:55:10.965 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:10.961+0000 7f1e1ea69640 1 -- 172.21.15.117:0/3906628273 wait complete. 2024-09-21T05:55:11.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:10 smithi192 ceph-mon[34700]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:55:11.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:10 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3906628273' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T05:55:11.275 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:10 smithi117 ceph-mon[30947]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:55:11.275 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:10 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3906628273' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T05:55:11.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:10 smithi138 ceph-mon[34391]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:55:11.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:10 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3906628273' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T05:55:11.613 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T05:55:11.858 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:12.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:11 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3906628273' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T05:55:12.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:11 smithi192 ceph-mon[34700]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T05:55:12.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:11 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3906628273' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T05:55:12.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:11 smithi117 ceph-mon[30947]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T05:55:12.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:11 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3906628273' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T05:55:12.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:11 smithi138 ceph-mon[34391]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T05:55:12.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.389+0000 7f78d01e8640 1 -- 172.21.15.117:0/4201674665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78c81083c0 msgr2=0x7f78c81087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:12.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.389+0000 7f78d01e8640 1 --2- 172.21.15.117:0/4201674665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78c81083c0 0x7f78c81087d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f78b8009900 tx=0x7f78b802a410 comp rx=0 tx=0).stop 2024-09-21T05:55:12.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.389+0000 7f78d01e8640 1 -- 172.21.15.117:0/4201674665 shutdown_connections 2024-09-21T05:55:12.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.389+0000 7f78d01e8640 1 --2- 172.21.15.117:0/4201674665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78c81083c0 0x7f78c81087d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:12.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.389+0000 7f78d01e8640 1 --2- 172.21.15.117:0/4201674665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78c8108d10 0x7f78c810f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:12.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.389+0000 7f78d01e8640 1 --2- 172.21.15.117:0/4201674665 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78c8102a80 0x7f78c8102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:12.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.389+0000 7f78d01e8640 1 -- 172.21.15.117:0/4201674665 >> 172.21.15.117:0/4201674665 conn(0x7f78c8079740 msgr2=0x7f78c8079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:12.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.390+0000 7f78d01e8640 1 -- 172.21.15.117:0/4201674665 shutdown_connections 2024-09-21T05:55:12.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.390+0000 7f78d01e8640 1 -- 172.21.15.117:0/4201674665 wait complete. 2024-09-21T05:55:12.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.390+0000 7f78d01e8640 1 Processor -- start 2024-09-21T05:55:12.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.390+0000 7f78d01e8640 1 -- start start 2024-09-21T05:55:12.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.390+0000 7f78d01e8640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78c8102a80 0x7f78c819b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:12.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.390+0000 7f78d01e8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78c81083c0 0x7f78c819b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:12.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.390+0000 7f78d01e8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78c8108d10 0x7f78c819bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:12.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.390+0000 7f78d01e8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f78c8112980 con 0x7f78c8108d10 2024-09-21T05:55:12.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.390+0000 7f78d01e8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f78c8112800 con 0x7f78c81083c0 2024-09-21T05:55:12.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.390+0000 7f78d01e8640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f78c8112b00 con 0x7f78c8102a80 2024-09-21T05:55:12.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.391+0000 7f78ce785640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78c8108d10 0x7f78c819bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:12.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.391+0000 7f78ce785640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78c8108d10 0x7f78c819bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51330/0 (socket says 172.21.15.117:51330) 2024-09-21T05:55:12.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.391+0000 7f78ce785640 1 -- 172.21.15.117:0/1443009087 learned_addr learned my addr 172.21.15.117:0/1443009087 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:55:12.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.391+0000 7f78cdf84640 1 --2- 172.21.15.117:0/1443009087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78c8102a80 0x7f78c819b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:12.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.391+0000 7f78cd783640 1 --2- 172.21.15.117:0/1443009087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78c81083c0 0x7f78c819b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:12.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.391+0000 7f78ce785640 1 -- 172.21.15.117:0/1443009087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78c8102a80 msgr2=0x7f78c819b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:12.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.391+0000 7f78ce785640 1 --2- 172.21.15.117:0/1443009087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78c8102a80 0x7f78c819b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:12.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.391+0000 7f78ce785640 1 -- 172.21.15.117:0/1443009087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78c81083c0 msgr2=0x7f78c819b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:12.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.391+0000 7f78ce785640 1 --2- 172.21.15.117:0/1443009087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78c81083c0 0x7f78c819b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:12.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.391+0000 7f78ce785640 1 -- 172.21.15.117:0/1443009087 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c81a65b0 con 0x7f78c8108d10 2024-09-21T05:55:12.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.391+0000 7f78ce785640 1 --2- 172.21.15.117:0/1443009087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78c8108d10 0x7f78c819bd70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f78c000eaa0 tx=0x7f78c000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:12.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.391+0000 7f78beffd640 1 -- 172.21.15.117:0/1443009087 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c000cdb0 con 0x7f78c8108d10 2024-09-21T05:55:12.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.391+0000 7f78beffd640 1 -- 172.21.15.117:0/1443009087 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78c0005020 con 0x7f78c8108d10 2024-09-21T05:55:12.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.391+0000 7f78beffd640 1 -- 172.21.15.117:0/1443009087 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c00116d0 con 0x7f78c8108d10 2024-09-21T05:55:12.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.391+0000 7f78d01e8640 1 -- 172.21.15.117:0/1443009087 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c81a68a0 con 0x7f78c8108d10 2024-09-21T05:55:12.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.391+0000 7f78d01e8640 1 -- 172.21.15.117:0/1443009087 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c81a6d60 con 0x7f78c8108d10 2024-09-21T05:55:12.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.392+0000 7f78d01e8640 1 -- 172.21.15.117:0/1443009087 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78c8103d00 con 0x7f78c8108d10 2024-09-21T05:55:12.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.394+0000 7f78beffd640 1 -- 172.21.15.117:0/1443009087 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f78c00051d0 con 0x7f78c8108d10 2024-09-21T05:55:12.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.395+0000 7f78beffd640 1 --2- 172.21.15.117:0/1443009087 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f78b40b6560 0x7f78b40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:12.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.395+0000 7f78cdf84640 1 --2- 172.21.15.117:0/1443009087 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f78b40b6560 0x7f78b40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:12.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.395+0000 7f78beffd640 1 -- 172.21.15.117:0/1443009087 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7712+0+0 (secure 0 0 0) 0x7f78c00e75e0 con 0x7f78c8108d10 2024-09-21T05:55:12.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.395+0000 7f78cdf84640 1 --2- 172.21.15.117:0/1443009087 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f78b40b6560 0x7f78b40b8a70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f78c806c3a0 tx=0x7f78c400b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:12.401 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.398+0000 7f78beffd640 1 -- 172.21.15.117:0/1443009087 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f78c0016030 con 0x7f78c8108d10 2024-09-21T05:55:12.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.575+0000 7f78d01e8640 1 -- 172.21.15.117:0/1443009087 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f78c8102e90 con 0x7f78c8108d10 2024-09-21T05:55:12.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.971+0000 7f78beffd640 1 -- 172.21.15.117:0/1443009087 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v72) ==== 170+0+0 (secure 0 0 0) 0x7f78c00b24b0 con 0x7f78c8108d10 2024-09-21T05:55:12.974 INFO:teuthology.orchestra.run.smithi117.stderr:pool 'cephfs_data' created 2024-09-21T05:55:12.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.974+0000 7f78d01e8640 1 -- 172.21.15.117:0/1443009087 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f78b40b6560 msgr2=0x7f78b40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:12.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.974+0000 7f78d01e8640 1 --2- 172.21.15.117:0/1443009087 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f78b40b6560 0x7f78b40b8a70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f78c806c3a0 tx=0x7f78c400b410 comp rx=0 tx=0).stop 2024-09-21T05:55:12.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.974+0000 7f78d01e8640 1 -- 172.21.15.117:0/1443009087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78c8108d10 msgr2=0x7f78c819bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:12.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.974+0000 7f78d01e8640 1 --2- 172.21.15.117:0/1443009087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78c8108d10 0x7f78c819bd70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f78c000eaa0 tx=0x7f78c000ef60 comp rx=0 tx=0).stop 2024-09-21T05:55:12.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.974+0000 7f78d01e8640 1 -- 172.21.15.117:0/1443009087 shutdown_connections 2024-09-21T05:55:12.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.974+0000 7f78d01e8640 1 --2- 172.21.15.117:0/1443009087 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f78b40b6560 0x7f78b40b8a70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:12.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.974+0000 7f78d01e8640 1 --2- 172.21.15.117:0/1443009087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78c8108d10 0x7f78c819bd70 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:12.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.974+0000 7f78d01e8640 1 --2- 172.21.15.117:0/1443009087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78c81083c0 0x7f78c819b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:12.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.974+0000 7f78d01e8640 1 --2- 172.21.15.117:0/1443009087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78c8102a80 0x7f78c819b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:12.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.974+0000 7f78d01e8640 1 -- 172.21.15.117:0/1443009087 >> 172.21.15.117:0/1443009087 conn(0x7f78c8079740 msgr2=0x7f78c81044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:12.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.974+0000 7f78d01e8640 1 -- 172.21.15.117:0/1443009087 shutdown_connections 2024-09-21T05:55:12.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:12.974+0000 7f78d01e8640 1 -- 172.21.15.117:0/1443009087 wait complete. 2024-09-21T05:55:13.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:12 smithi192 ceph-mon[34700]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T05:55:13.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:12 smithi192 ceph-mon[34700]: pgmap v259: 33 pgs: 9 creating+peering, 21 unknown, 3 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:55:13.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:12 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/1443009087' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T05:55:13.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:12 smithi117 ceph-mon[30947]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T05:55:13.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:12 smithi117 ceph-mon[30947]: pgmap v259: 33 pgs: 9 creating+peering, 21 unknown, 3 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:55:13.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:12 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/1443009087' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T05:55:13.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:12 smithi138 ceph-mon[34391]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T05:55:13.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:12 smithi138 ceph-mon[34391]: pgmap v259: 33 pgs: 9 creating+peering, 21 unknown, 3 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:55:13.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:12 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/1443009087' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T05:55:13.622 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T05:55:13.868 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:13.984 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:13 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/1443009087' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T05:55:13.984 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:13 smithi117 ceph-mon[30947]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T05:55:14.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.329+0000 7fd914471640 1 -- 172.21.15.117:0/2380382104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd90c1083c0 msgr2=0x7fd90c1087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:14.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.329+0000 7fd914471640 1 --2- 172.21.15.117:0/2380382104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd90c1083c0 0x7fd90c1087d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc009900 tx=0x7fd8fc02a410 comp rx=0 tx=0).stop 2024-09-21T05:55:14.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.329+0000 7fd914471640 1 -- 172.21.15.117:0/2380382104 shutdown_connections 2024-09-21T05:55:14.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.329+0000 7fd914471640 1 --2- 172.21.15.117:0/2380382104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd90c1083c0 0x7fd90c1087d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:14.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.329+0000 7fd914471640 1 --2- 172.21.15.117:0/2380382104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd90c108d10 0x7fd90c10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:14.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.329+0000 7fd914471640 1 --2- 172.21.15.117:0/2380382104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd90c102a80 0x7fd90c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:14.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.329+0000 7fd914471640 1 -- 172.21.15.117:0/2380382104 >> 172.21.15.117:0/2380382104 conn(0x7fd90c079740 msgr2=0x7fd90c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:14.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.329+0000 7fd914471640 1 -- 172.21.15.117:0/2380382104 shutdown_connections 2024-09-21T05:55:14.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.329+0000 7fd914471640 1 -- 172.21.15.117:0/2380382104 wait complete. 2024-09-21T05:55:14.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.330+0000 7fd914471640 1 Processor -- start 2024-09-21T05:55:14.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.330+0000 7fd914471640 1 -- start start 2024-09-21T05:55:14.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.330+0000 7fd914471640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd90c102a80 0x7fd90c199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:14.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.330+0000 7fd914471640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd90c1083c0 0x7fd90c1997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:14.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.330+0000 7fd914471640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd90c108d10 0x7fd90c199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:14.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.330+0000 7fd914471640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd90c112a40 con 0x7fd90c108d10 2024-09-21T05:55:14.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.330+0000 7fd914471640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd90c1128c0 con 0x7fd90c102a80 2024-09-21T05:55:14.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.330+0000 7fd914471640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd90c112bc0 con 0x7fd90c1083c0 2024-09-21T05:55:14.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.330+0000 7fd912a0e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd90c108d10 0x7fd90c199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:14.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.330+0000 7fd912a0e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd90c108d10 0x7fd90c199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51358/0 (socket says 172.21.15.117:51358) 2024-09-21T05:55:14.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.330+0000 7fd912a0e640 1 -- 172.21.15.117:0/705561228 learned_addr learned my addr 172.21.15.117:0/705561228 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:55:14.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.330+0000 7fd91220d640 1 --2- 172.21.15.117:0/705561228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd90c102a80 0x7fd90c199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:14.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.331+0000 7fd912a0e640 1 -- 172.21.15.117:0/705561228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd90c1083c0 msgr2=0x7fd90c1997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:14.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.331+0000 7fd912a0e640 1 --2- 172.21.15.117:0/705561228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd90c1083c0 0x7fd90c1997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:14.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.331+0000 7fd912a0e640 1 -- 172.21.15.117:0/705561228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd90c102a80 msgr2=0x7fd90c199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:14.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.331+0000 7fd912a0e640 1 --2- 172.21.15.117:0/705561228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd90c102a80 0x7fd90c199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:14.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.331+0000 7fd912a0e640 1 -- 172.21.15.117:0/705561228 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd90c1a8930 con 0x7fd90c108d10 2024-09-21T05:55:14.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.331+0000 7fd912a0e640 1 --2- 172.21.15.117:0/705561228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd90c108d10 0x7fd90c199d00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd90400b970 tx=0x7fd90400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:14.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.331+0000 7fd9037fe640 1 -- 172.21.15.117:0/705561228 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd904019070 con 0x7fd90c108d10 2024-09-21T05:55:14.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.331+0000 7fd914471640 1 -- 172.21.15.117:0/705561228 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd90c1a8c20 con 0x7fd90c108d10 2024-09-21T05:55:14.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.331+0000 7fd9037fe640 1 -- 172.21.15.117:0/705561228 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd904014440 con 0x7fd90c108d10 2024-09-21T05:55:14.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.331+0000 7fd9037fe640 1 -- 172.21.15.117:0/705561228 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9040075d0 con 0x7fd90c108d10 2024-09-21T05:55:14.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.331+0000 7fd914471640 1 -- 172.21.15.117:0/705561228 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd90c1a9160 con 0x7fd90c108d10 2024-09-21T05:55:14.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.332+0000 7fd914471640 1 -- 172.21.15.117:0/705561228 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd90c103d00 con 0x7fd90c108d10 2024-09-21T05:55:14.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.333+0000 7fd9037fe640 1 -- 172.21.15.117:0/705561228 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7fd90401c070 con 0x7fd90c108d10 2024-09-21T05:55:14.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.334+0000 7fd9037fe640 1 --2- 172.21.15.117:0/705561228 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd8f80b6560 0x7fd8f80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:14.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.334+0000 7fd91220d640 1 --2- 172.21.15.117:0/705561228 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd8f80b6560 0x7fd8f80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:14.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.334+0000 7fd9037fe640 1 -- 172.21.15.117:0/705561228 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8094+0+0 (secure 0 0 0) 0x7fd9040e6c50 con 0x7fd90c108d10 2024-09-21T05:55:14.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.335+0000 7fd91220d640 1 --2- 172.21.15.117:0/705561228 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd8f80b6560 0x7fd8f80b8a70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd90c07a120 tx=0x7fd908008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:14.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.337+0000 7fd9037fe640 1 -- 172.21.15.117:0/705561228 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd9040ad5c0 con 0x7fd90c108d10 2024-09-21T05:55:14.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:13 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/1443009087' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T05:55:14.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:13 smithi138 ceph-mon[34391]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T05:55:14.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:13 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/1443009087' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T05:55:14.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:13 smithi192 ceph-mon[34700]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T05:55:14.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.553+0000 7fd914471640 1 -- 172.21.15.117:0/705561228 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd90c102f10 con 0x7fd90c108d10 2024-09-21T05:55:14.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.981+0000 7fd9037fe640 1 -- 172.21.15.117:0/705561228 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fd90c102f10 con 0x7fd90c108d10 2024-09-21T05:55:14.985 INFO:teuthology.orchestra.run.smithi117.stderr:new fs with metadata pool 2 and data pool 3 2024-09-21T05:55:14.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.984+0000 7fd914471640 1 -- 172.21.15.117:0/705561228 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd8f80b6560 msgr2=0x7fd8f80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:14.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.984+0000 7fd914471640 1 --2- 172.21.15.117:0/705561228 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd8f80b6560 0x7fd8f80b8a70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd90c07a120 tx=0x7fd908008040 comp rx=0 tx=0).stop 2024-09-21T05:55:14.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.984+0000 7fd914471640 1 -- 172.21.15.117:0/705561228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd90c108d10 msgr2=0x7fd90c199d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:14.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.984+0000 7fd914471640 1 --2- 172.21.15.117:0/705561228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd90c108d10 0x7fd90c199d00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd90400b970 tx=0x7fd90400be30 comp rx=0 tx=0).stop 2024-09-21T05:55:14.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.984+0000 7fd914471640 1 -- 172.21.15.117:0/705561228 shutdown_connections 2024-09-21T05:55:14.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.984+0000 7fd914471640 1 --2- 172.21.15.117:0/705561228 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd8f80b6560 0x7fd8f80b8a70 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:14.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.984+0000 7fd914471640 1 --2- 172.21.15.117:0/705561228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd90c108d10 0x7fd90c199d00 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:14.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.984+0000 7fd914471640 1 --2- 172.21.15.117:0/705561228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd90c1083c0 0x7fd90c1997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:14.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.984+0000 7fd914471640 1 --2- 172.21.15.117:0/705561228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd90c102a80 0x7fd90c199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:14.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.984+0000 7fd914471640 1 -- 172.21.15.117:0/705561228 >> 172.21.15.117:0/705561228 conn(0x7fd90c079740 msgr2=0x7fd90c1053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:14.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.984+0000 7fd914471640 1 -- 172.21.15.117:0/705561228 shutdown_connections 2024-09-21T05:55:14.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:14.985+0000 7fd914471640 1 -- 172.21.15.117:0/705561228 wait complete. 2024-09-21T05:55:15.266 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:14 smithi138 ceph-mon[34391]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T05:55:15.266 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:14 smithi138 ceph-mon[34391]: pgmap v262: 97 pgs: 17 creating+peering, 70 unknown, 10 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:55:15.266 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:14 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/705561228' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T05:55:15.266 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:14 smithi138 ceph-mon[34391]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T05:55:15.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:14 smithi117 ceph-mon[30947]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T05:55:15.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:14 smithi117 ceph-mon[30947]: pgmap v262: 97 pgs: 17 creating+peering, 70 unknown, 10 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:55:15.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:14 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/705561228' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T05:55:15.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:14 smithi117 ceph-mon[30947]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T05:55:15.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:14 smithi192 ceph-mon[34700]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T05:55:15.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:14 smithi192 ceph-mon[34700]: pgmap v262: 97 pgs: 17 creating+peering, 70 unknown, 10 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T05:55:15.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:14 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/705561228' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T05:55:15.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:14 smithi192 ceph-mon[34700]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T05:55:15.701 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T05:55:15.949 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:16.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:15 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/705561228' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T05:55:16.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:15 smithi117 ceph-mon[30947]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T05:55:16.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:15 smithi117 ceph-mon[30947]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T05:55:16.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:15 smithi117 ceph-mon[30947]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T05:55:16.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:15 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/705561228' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T05:55:16.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:15 smithi138 ceph-mon[34391]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T05:55:16.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:15 smithi138 ceph-mon[34391]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T05:55:16.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:15 smithi138 ceph-mon[34391]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T05:55:16.401 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:15 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/705561228' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T05:55:16.401 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:15 smithi192 ceph-mon[34700]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T05:55:16.401 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:15 smithi192 ceph-mon[34700]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T05:55:16.401 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:15 smithi192 ceph-mon[34700]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T05:55:16.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.412+0000 7ff82606d640 1 -- 172.21.15.117:0/1475693245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff82010de90 msgr2=0x7ff820110380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:16.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.412+0000 7ff82606d640 1 --2- 172.21.15.117:0/1475693245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff82010de90 0x7ff820110380 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff818009ca0 tx=0x7ff81802a8c0 comp rx=0 tx=0).stop 2024-09-21T05:55:16.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.412+0000 7ff82606d640 1 -- 172.21.15.117:0/1475693245 shutdown_connections 2024-09-21T05:55:16.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.412+0000 7ff82606d640 1 --2- 172.21.15.117:0/1475693245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff82010de90 0x7ff820110380 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:16.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.412+0000 7ff82606d640 1 --2- 172.21.15.117:0/1475693245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff820105380 0x7ff82010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:16.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.412+0000 7ff82606d640 1 --2- 172.21.15.117:0/1475693245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff820104a30 0x7ff820104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:16.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.412+0000 7ff82606d640 1 -- 172.21.15.117:0/1475693245 >> 172.21.15.117:0/1475693245 conn(0x7ff820079770 msgr2=0x7ff820079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:16.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.412+0000 7ff82606d640 1 -- 172.21.15.117:0/1475693245 shutdown_connections 2024-09-21T05:55:16.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.412+0000 7ff82606d640 1 -- 172.21.15.117:0/1475693245 wait complete. 2024-09-21T05:55:16.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.413+0000 7ff82606d640 1 Processor -- start 2024-09-21T05:55:16.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.413+0000 7ff82606d640 1 -- start start 2024-09-21T05:55:16.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.414+0000 7ff82606d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff820104a30 0x7ff82019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:16.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.414+0000 7ff82606d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff820105380 0x7ff82019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:16.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.414+0000 7ff82606d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff82010de90 0x7ff82019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:16.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.414+0000 7ff82606d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff820113b10 con 0x7ff82010de90 2024-09-21T05:55:16.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.414+0000 7ff82606d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff820113990 con 0x7ff820104a30 2024-09-21T05:55:16.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.414+0000 7ff82606d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff820113c90 con 0x7ff820105380 2024-09-21T05:55:16.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.414+0000 7ff81ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff82010de90 0x7ff82019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:16.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.414+0000 7ff81ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff82010de90 0x7ff82019fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40442/0 (socket says 172.21.15.117:40442) 2024-09-21T05:55:16.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.414+0000 7ff81ffff640 1 -- 172.21.15.117:0/3601499172 learned_addr learned my addr 172.21.15.117:0/3601499172 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:55:16.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.414+0000 7ff81effd640 1 --2- 172.21.15.117:0/3601499172 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff820105380 0x7ff82019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:16.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.414+0000 7ff81f7fe640 1 --2- 172.21.15.117:0/3601499172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff820104a30 0x7ff82019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:16.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.414+0000 7ff81ffff640 1 -- 172.21.15.117:0/3601499172 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff820105380 msgr2=0x7ff82019aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:16.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.414+0000 7ff81ffff640 1 --2- 172.21.15.117:0/3601499172 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff820105380 0x7ff82019aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:16.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.414+0000 7ff81ffff640 1 -- 172.21.15.117:0/3601499172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff820104a30 msgr2=0x7ff82019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:16.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.414+0000 7ff81ffff640 1 --2- 172.21.15.117:0/3601499172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff820104a30 0x7ff82019a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:16.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.414+0000 7ff81ffff640 1 -- 172.21.15.117:0/3601499172 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8201a0410 con 0x7ff82010de90 2024-09-21T05:55:16.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.414+0000 7ff81f7fe640 1 --2- 172.21.15.117:0/3601499172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff820104a30 0x7ff82019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:55:16.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.414+0000 7ff81ffff640 1 --2- 172.21.15.117:0/3601499172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff82010de90 0x7ff82019fce0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff818005cc0 tx=0x7ff81802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:16.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.414+0000 7ff81cff9640 1 -- 172.21.15.117:0/3601499172 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818003bf0 con 0x7ff82010de90 2024-09-21T05:55:16.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.414+0000 7ff81cff9640 1 -- 172.21.15.117:0/3601499172 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff818003d90 con 0x7ff82010de90 2024-09-21T05:55:16.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.414+0000 7ff82606d640 1 -- 172.21.15.117:0/3601499172 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8201a06a0 con 0x7ff82010de90 2024-09-21T05:55:16.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.414+0000 7ff81cff9640 1 -- 172.21.15.117:0/3601499172 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81802fbc0 con 0x7ff82010de90 2024-09-21T05:55:16.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.415+0000 7ff82606d640 1 -- 172.21.15.117:0/3601499172 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8201a6e60 con 0x7ff82010de90 2024-09-21T05:55:16.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.415+0000 7ff82606d640 1 -- 172.21.15.117:0/3601499172 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff82006c260 con 0x7ff82010de90 2024-09-21T05:55:16.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.418+0000 7ff81cff9640 1 -- 172.21.15.117:0/3601499172 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7ff81802fd60 con 0x7ff82010de90 2024-09-21T05:55:16.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.419+0000 7ff81cff9640 1 --2- 172.21.15.117:0/3601499172 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff8080b6560 0x7ff8080b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:16.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.419+0000 7ff81f7fe640 1 --2- 172.21.15.117:0/3601499172 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff8080b6560 0x7ff8080b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:16.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.419+0000 7ff81cff9640 1 -- 172.21.15.117:0/3601499172 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 8194+0+0 (secure 0 0 0) 0x7ff8181089e0 con 0x7ff82010de90 2024-09-21T05:55:16.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.419+0000 7ff81f7fe640 1 --2- 172.21.15.117:0/3601499172 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff8080b6560 0x7ff8080b8a70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff8100058c0 tx=0x7ff810005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:16.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.422+0000 7ff81cff9640 1 -- 172.21.15.117:0/3601499172 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff8180cf290 con 0x7ff82010de90 2024-09-21T05:55:16.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.600+0000 7ff82606d640 1 -- 172.21.15.117:0/3601499172 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff8201a7030 con 0x7ff82010de90 2024-09-21T05:55:16.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.601+0000 7ff81cff9640 1 -- 172.21.15.117:0/3601499172 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+22680 (secure 0 0 0) 0x7ff8180d3670 con 0x7ff82010de90 2024-09-21T05:55:16.605 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:55:16.605 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":74,"fsid":"ac8581a2-77dc-11ef-baf5-efdc52797490","created":"2024-09-21T05:46:47.021846+0000","modified":"2024-09-21T05:55:14.555939+0000","last_up_change":"2024-09-21T05:54:21.572849+0000","last_in_change":"2024-09-21T05:53:58.892597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:50:32.717389+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:55:10.604159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T05:55:12.581258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"68315502-fa6c-4a6e-a2d4-a7dde8071c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6803","nonce":3761478035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6805","nonce":3761478035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6809","nonce":3761478035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6807","nonce":3761478035}]},"public_addr":"172.21.15.117:6803/3761478035","cluster_addr":"172.21.15.117:6805/3761478035","heartbeat_back_addr":"172.21.15.117:6809/3761478035","heartbeat_front_addr":"172.21.15.117:6807/3761478035","state":["exists","up"]},{"osd":1,"uuid":"d8910a08-4ef3-49f2-a4db-e3588d85d12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6801","nonce":1737365588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6803","nonce":1737365588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6807","nonce":1737365588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6805","nonce":1737365588}]},"public_addr":"172.21.15.138:6801/1737365588","cluster_addr":"172.21.15.138:6803/1737365588","heartbeat_back_addr":"172.21.15.138:6807/1737365588","heartbeat_front_addr":"172.21.15.138:6805/1737365588","state":["exists","up"]},{"osd":2,"uuid":"94972378-a11f-4129-9e40-ad3fdec1d189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6801","nonce":3002534992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6803","nonce":3002534992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6807","nonce":3002534992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6805","nonce":3002534992}]},"public_addr":"172.21.15.192:6801/3002534992","cluster_addr":"172.21.15.192:6803/3002534992","heartbeat_back_addr":"172.21.15.192:6807/3002534992","heartbeat_front_addr":"172.21.15.192:6805/3002534992","state":["exists","up"]},{"osd":3,"uuid":"d4168c43-f0de-4445-945c-92d1cf270c33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6811","nonce":1222400293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6813","nonce":1222400293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6817","nonce":1222400293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6815","nonce":1222400293}]},"public_addr":"172.21.15.117:6811/1222400293","cluster_addr":"172.21.15.117:6813/1222400293","heartbeat_back_addr":"172.21.15.117:6817/1222400293","heartbeat_front_addr":"172.21.15.117:6815/1222400293","state":["exists","up"]},{"osd":4,"uuid":"28f1d7c0-8392-4a33-85c0-0de611cdd4b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6809","nonce":4294892870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6811","nonce":4294892870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6815","nonce":4294892870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6813","nonce":4294892870}]},"public_addr":"172.21.15.138:6809/4294892870","cluster_addr":"172.21.15.138:6811/4294892870","heartbeat_back_addr":"172.21.15.138:6815/4294892870","heartbeat_front_addr":"172.21.15.138:6813/4294892870","state":["exists","up"]},{"osd":5,"uuid":"ca902407-45e8-4bc5-bdbd-a729c1de8322","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6809","nonce":1776840019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6811","nonce":1776840019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6815","nonce":1776840019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6813","nonce":1776840019}]},"public_addr":"172.21.15.192:6809/1776840019","cluster_addr":"172.21.15.192:6811/1776840019","heartbeat_back_addr":"172.21.15.192:6815/1776840019","heartbeat_front_addr":"172.21.15.192:6813/1776840019","state":["exists","up"]},{"osd":6,"uuid":"6e507a91-ed09-4a33-bf0e-c4b593b3c5aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6819","nonce":1093341590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6821","nonce":1093341590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6825","nonce":1093341590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6823","nonce":1093341590}]},"public_addr":"172.21.15.117:6819/1093341590","cluster_addr":"172.21.15.117:6821/1093341590","heartbeat_back_addr":"172.21.15.117:6825/1093341590","heartbeat_front_addr":"172.21.15.117:6823/1093341590","state":["exists","up"]},{"osd":7,"uuid":"36a47dd7-67f7-4ebe-8759-3a8bb6bd2ea3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6817","nonce":1448179714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6819","nonce":1448179714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6823","nonce":1448179714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6821","nonce":1448179714}]},"public_addr":"172.21.15.138:6817/1448179714","cluster_addr":"172.21.15.138:6819/1448179714","heartbeat_back_addr":"172.21.15.138:6823/1448179714","heartbeat_front_addr":"172.21.15.138:6821/1448179714","state":["exists","up"]},{"osd":8,"uuid":"118e0b5d-2619-4512-94c5-8d07a0d3a28f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6817","nonce":3960902885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6819","nonce":3960902885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6823","nonce":3960902885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6821","nonce":3960902885}]},"public_addr":"172.21.15.192:6817/3960902885","cluster_addr":"172.21.15.192:6819/3960902885","heartbeat_back_addr":"172.21.15.192:6823/3960902885","heartbeat_front_addr":"172.21.15.192:6821/3960902885","state":["exists","up"]},{"osd":9,"uuid":"47543bc6-fa0b-4e72-9742-97dcda0b7f8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6827","nonce":3453855195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6829","nonce":3453855195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6833","nonce":3453855195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6831","nonce":3453855195}]},"public_addr":"172.21.15.117:6827/3453855195","cluster_addr":"172.21.15.117:6829/3453855195","heartbeat_back_addr":"172.21.15.117:6833/3453855195","heartbeat_front_addr":"172.21.15.117:6831/3453855195","state":["exists","up"]},{"osd":10,"uuid":"5e6708b4-7a85-4fa5-a03c-27d99f7d749f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6825","nonce":903392451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6827","nonce":903392451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6831","nonce":903392451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6829","nonce":903392451}]},"public_addr":"172.21.15.138:6825/903392451","cluster_addr":"172.21.15.138:6827/903392451","heartbeat_back_addr":"172.21.15.138:6831/903392451","heartbeat_front_addr":"172.21.15.138:6829/903392451","state":["exists","up"]},{"osd":11,"uuid":"24bc2ef7-e2b4-43c6-9e6c-43ae02390415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6825","nonce":899090479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6827","nonce":899090479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6831","nonce":899090479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6829","nonce":899090479}]},"public_addr":"172.21.15.192:6825/899090479","cluster_addr":"172.21.15.192:6827/899090479","heartbeat_back_addr":"172.21.15.192:6831/899090479","heartbeat_front_addr":"172.21.15.192:6829/899090479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:49:40.742217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:05.963655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:30.164495+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:56.598899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:51:21.940331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:51:47.777445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:52:12.869007+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:52:39.454054+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:03.569226+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:29.760863+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:55.788604+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:54:20.510184+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:0/2989394419":"2024-09-22T05:48:32.406654+0000","172.21.15.117:6801/1702676172":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/1766406059":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6800/2337036514":"2024-09-22T05:47:09.112435+0000","172.21.15.117:0/985798343":"2024-09-22T05:47:09.112435+0000","172.21.15.117:0/1083273222":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/473284565":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6801/2337036514":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6800/816635165":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/2642940197":"2024-09-22T05:48:32.406654+0000","172.21.15.117:6801/816635165":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/2225537433":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/3651584169":"2024-09-22T05:47:33.789146+0000","172.21.15.117:6800/1702676172":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/1284403882":"2024-09-22T05:47:33.789146+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:55:16.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.604+0000 7ff82606d640 1 -- 172.21.15.117:0/3601499172 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff8080b6560 msgr2=0x7ff8080b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:16.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.604+0000 7ff82606d640 1 --2- 172.21.15.117:0/3601499172 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff8080b6560 0x7ff8080b8a70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff8100058c0 tx=0x7ff810005850 comp rx=0 tx=0).stop 2024-09-21T05:55:16.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.604+0000 7ff82606d640 1 -- 172.21.15.117:0/3601499172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff82010de90 msgr2=0x7ff82019fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:16.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.604+0000 7ff82606d640 1 --2- 172.21.15.117:0/3601499172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff82010de90 0x7ff82019fce0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff818005cc0 tx=0x7ff81802f380 comp rx=0 tx=0).stop 2024-09-21T05:55:16.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.604+0000 7ff82606d640 1 -- 172.21.15.117:0/3601499172 shutdown_connections 2024-09-21T05:55:16.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.604+0000 7ff82606d640 1 --2- 172.21.15.117:0/3601499172 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff8080b6560 0x7ff8080b8a70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:16.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.604+0000 7ff82606d640 1 --2- 172.21.15.117:0/3601499172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff82010de90 0x7ff82019fce0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:16.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.604+0000 7ff82606d640 1 --2- 172.21.15.117:0/3601499172 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff820105380 0x7ff82019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:16.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.604+0000 7ff82606d640 1 --2- 172.21.15.117:0/3601499172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff820104a30 0x7ff82019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:16.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.604+0000 7ff82606d640 1 -- 172.21.15.117:0/3601499172 >> 172.21.15.117:0/3601499172 conn(0x7ff820079770 msgr2=0x7ff820100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:16.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.604+0000 7ff82606d640 1 -- 172.21.15.117:0/3601499172 shutdown_connections 2024-09-21T05:55:16.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:16.604+0000 7ff82606d640 1 -- 172.21.15.117:0/3601499172 wait complete. 2024-09-21T05:55:17.198 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:17 smithi117 ceph-mon[30947]: mds.? [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] up:active 2024-09-21T05:55:17.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:17 smithi117 ceph-mon[30947]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T05:55:17.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:17 smithi117 ceph-mon[30947]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T05:55:17.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:17 smithi117 ceph-mon[30947]: pgmap v264: 97 pgs: 11 creating+peering, 31 unknown, 55 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 225 B/s wr, 5 op/s 2024-09-21T05:55:17.199 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:17 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3601499172' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T05:55:17.226 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T05:55:17.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:17 smithi138 ceph-mon[34391]: mds.? [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] up:active 2024-09-21T05:55:17.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:17 smithi138 ceph-mon[34391]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T05:55:17.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:17 smithi138 ceph-mon[34391]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T05:55:17.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:17 smithi138 ceph-mon[34391]: pgmap v264: 97 pgs: 11 creating+peering, 31 unknown, 55 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 225 B/s wr, 5 op/s 2024-09-21T05:55:17.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:17 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3601499172' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T05:55:17.479 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:17.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:17 smithi192 ceph-mon[34700]: mds.? [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] up:active 2024-09-21T05:55:17.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:17 smithi192 ceph-mon[34700]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T05:55:17.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:17 smithi192 ceph-mon[34700]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T05:55:17.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:17 smithi192 ceph-mon[34700]: pgmap v264: 97 pgs: 11 creating+peering, 31 unknown, 55 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 225 B/s wr, 5 op/s 2024-09-21T05:55:17.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:17 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3601499172' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T05:55:17.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.960+0000 7fb1dc30f640 1 -- 172.21.15.117:0/4243095518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1d4109e00 msgr2=0x7fb1d4110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:17.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.960+0000 7fb1dc30f640 1 --2- 172.21.15.117:0/4243095518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1d4109e00 0x7fb1d4110630 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4009900 tx=0x7fb1c402a410 comp rx=0 tx=0).stop 2024-09-21T05:55:17.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.960+0000 7fb1dc30f640 1 -- 172.21.15.117:0/4243095518 shutdown_connections 2024-09-21T05:55:17.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.960+0000 7fb1dc30f640 1 --2- 172.21.15.117:0/4243095518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1d4109e00 0x7fb1d4110630 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:17.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.960+0000 7fb1dc30f640 1 --2- 172.21.15.117:0/4243095518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1d41094b0 0x7fb1d41098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:17.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.960+0000 7fb1dc30f640 1 --2- 172.21.15.117:0/4243095518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1d4103b70 0x7fb1d4103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:17.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.960+0000 7fb1dc30f640 1 -- 172.21.15.117:0/4243095518 >> 172.21.15.117:0/4243095518 conn(0x7fb1d40ff5e0 msgr2=0x7fb1d4101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:17.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.960+0000 7fb1dc30f640 1 -- 172.21.15.117:0/4243095518 shutdown_connections 2024-09-21T05:55:17.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.960+0000 7fb1dc30f640 1 -- 172.21.15.117:0/4243095518 wait complete. 2024-09-21T05:55:17.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.961+0000 7fb1dc30f640 1 Processor -- start 2024-09-21T05:55:17.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.961+0000 7fb1dc30f640 1 -- start start 2024-09-21T05:55:17.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.961+0000 7fb1dc30f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1d4103b70 0x7fb1d419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:17.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.961+0000 7fb1dc30f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1d41094b0 0x7fb1d419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:17.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.961+0000 7fb1dc30f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1d4109e00 0x7fb1d419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:17.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.961+0000 7fb1dc30f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d4113ac0 con 0x7fb1d41094b0 2024-09-21T05:55:17.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.961+0000 7fb1dc30f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d4113940 con 0x7fb1d4103b70 2024-09-21T05:55:17.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.961+0000 7fb1dc30f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d4113c40 con 0x7fb1d4109e00 2024-09-21T05:55:17.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.961+0000 7fb1d98aa640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1d41094b0 0x7fb1d419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:17.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.961+0000 7fb1d98aa640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1d41094b0 0x7fb1d419aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40468/0 (socket says 172.21.15.117:40468) 2024-09-21T05:55:17.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.961+0000 7fb1d98aa640 1 -- 172.21.15.117:0/2062164389 learned_addr learned my addr 172.21.15.117:0/2062164389 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:55:17.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.961+0000 7fb1da8ac640 1 --2- 172.21.15.117:0/2062164389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1d4109e00 0x7fb1d419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:17.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.961+0000 7fb1da0ab640 1 --2- 172.21.15.117:0/2062164389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1d4103b70 0x7fb1d419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:17.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.962+0000 7fb1d98aa640 1 -- 172.21.15.117:0/2062164389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1d4109e00 msgr2=0x7fb1d419fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:17.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.962+0000 7fb1d98aa640 1 --2- 172.21.15.117:0/2062164389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1d4109e00 0x7fb1d419fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:17.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.962+0000 7fb1d98aa640 1 -- 172.21.15.117:0/2062164389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1d4103b70 msgr2=0x7fb1d419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:17.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.962+0000 7fb1d98aa640 1 --2- 172.21.15.117:0/2062164389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1d4103b70 0x7fb1d419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:17.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.962+0000 7fb1d98aa640 1 -- 172.21.15.117:0/2062164389 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d41a0400 con 0x7fb1d41094b0 2024-09-21T05:55:17.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.962+0000 7fb1d98aa640 1 --2- 172.21.15.117:0/2062164389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1d41094b0 0x7fb1d419aa30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb1c800cce0 tx=0x7fb1c800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:17.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.962+0000 7fb1d37fe640 1 -- 172.21.15.117:0/2062164389 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c8004b20 con 0x7fb1d41094b0 2024-09-21T05:55:17.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.962+0000 7fb1dc30f640 1 -- 172.21.15.117:0/2062164389 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1d41a06f0 con 0x7fb1d41094b0 2024-09-21T05:55:17.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.962+0000 7fb1d37fe640 1 -- 172.21.15.117:0/2062164389 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1c8004cc0 con 0x7fb1d41094b0 2024-09-21T05:55:17.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.962+0000 7fb1d37fe640 1 -- 172.21.15.117:0/2062164389 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c800f6b0 con 0x7fb1d41094b0 2024-09-21T05:55:17.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.962+0000 7fb1dc30f640 1 -- 172.21.15.117:0/2062164389 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1d41a6e50 con 0x7fb1d41094b0 2024-09-21T05:55:17.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.963+0000 7fb1dc30f640 1 -- 172.21.15.117:0/2062164389 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1d4103f80 con 0x7fb1d41094b0 2024-09-21T05:55:17.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.965+0000 7fb1d37fe640 1 -- 172.21.15.117:0/2062164389 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7fb1c800f850 con 0x7fb1d41094b0 2024-09-21T05:55:17.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.966+0000 7fb1d37fe640 1 --2- 172.21.15.117:0/2062164389 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb1bc0b6560 0x7fb1bc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:17.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.966+0000 7fb1da0ab640 1 --2- 172.21.15.117:0/2062164389 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb1bc0b6560 0x7fb1bc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:17.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.966+0000 7fb1d37fe640 1 -- 172.21.15.117:0/2062164389 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 8194+0+0 (secure 0 0 0) 0x7fb1c80e7770 con 0x7fb1d41094b0 2024-09-21T05:55:17.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.966+0000 7fb1da0ab640 1 --2- 172.21.15.117:0/2062164389 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb1bc0b6560 0x7fb1bc0b8a70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc009b60 tx=0x7fb1cc005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:17.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:17.969+0000 7fb1d37fe640 1 -- 172.21.15.117:0/2062164389 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb1c8014030 con 0x7fb1d41094b0 2024-09-21T05:55:18.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:18.154+0000 7fb1dc30f640 1 -- 172.21.15.117:0/2062164389 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb1d402e640 con 0x7fb1d41094b0 2024-09-21T05:55:18.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:18.155+0000 7fb1d37fe640 1 -- 172.21.15.117:0/2062164389 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+22680 (secure 0 0 0) 0x7fb1c80ae020 con 0x7fb1d41094b0 2024-09-21T05:55:18.159 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:55:18.159 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":74,"fsid":"ac8581a2-77dc-11ef-baf5-efdc52797490","created":"2024-09-21T05:46:47.021846+0000","modified":"2024-09-21T05:55:14.555939+0000","last_up_change":"2024-09-21T05:54:21.572849+0000","last_in_change":"2024-09-21T05:53:58.892597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:50:32.717389+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:55:10.604159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T05:55:12.581258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"68315502-fa6c-4a6e-a2d4-a7dde8071c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6803","nonce":3761478035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6805","nonce":3761478035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6809","nonce":3761478035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6807","nonce":3761478035}]},"public_addr":"172.21.15.117:6803/3761478035","cluster_addr":"172.21.15.117:6805/3761478035","heartbeat_back_addr":"172.21.15.117:6809/3761478035","heartbeat_front_addr":"172.21.15.117:6807/3761478035","state":["exists","up"]},{"osd":1,"uuid":"d8910a08-4ef3-49f2-a4db-e3588d85d12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6801","nonce":1737365588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6803","nonce":1737365588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6807","nonce":1737365588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6805","nonce":1737365588}]},"public_addr":"172.21.15.138:6801/1737365588","cluster_addr":"172.21.15.138:6803/1737365588","heartbeat_back_addr":"172.21.15.138:6807/1737365588","heartbeat_front_addr":"172.21.15.138:6805/1737365588","state":["exists","up"]},{"osd":2,"uuid":"94972378-a11f-4129-9e40-ad3fdec1d189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6801","nonce":3002534992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6803","nonce":3002534992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6807","nonce":3002534992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6805","nonce":3002534992}]},"public_addr":"172.21.15.192:6801/3002534992","cluster_addr":"172.21.15.192:6803/3002534992","heartbeat_back_addr":"172.21.15.192:6807/3002534992","heartbeat_front_addr":"172.21.15.192:6805/3002534992","state":["exists","up"]},{"osd":3,"uuid":"d4168c43-f0de-4445-945c-92d1cf270c33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6811","nonce":1222400293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6813","nonce":1222400293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6817","nonce":1222400293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6815","nonce":1222400293}]},"public_addr":"172.21.15.117:6811/1222400293","cluster_addr":"172.21.15.117:6813/1222400293","heartbeat_back_addr":"172.21.15.117:6817/1222400293","heartbeat_front_addr":"172.21.15.117:6815/1222400293","state":["exists","up"]},{"osd":4,"uuid":"28f1d7c0-8392-4a33-85c0-0de611cdd4b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6809","nonce":4294892870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6811","nonce":4294892870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6815","nonce":4294892870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6813","nonce":4294892870}]},"public_addr":"172.21.15.138:6809/4294892870","cluster_addr":"172.21.15.138:6811/4294892870","heartbeat_back_addr":"172.21.15.138:6815/4294892870","heartbeat_front_addr":"172.21.15.138:6813/4294892870","state":["exists","up"]},{"osd":5,"uuid":"ca902407-45e8-4bc5-bdbd-a729c1de8322","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6809","nonce":1776840019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6811","nonce":1776840019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6815","nonce":1776840019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6813","nonce":1776840019}]},"public_addr":"172.21.15.192:6809/1776840019","cluster_addr":"172.21.15.192:6811/1776840019","heartbeat_back_addr":"172.21.15.192:6815/1776840019","heartbeat_front_addr":"172.21.15.192:6813/1776840019","state":["exists","up"]},{"osd":6,"uuid":"6e507a91-ed09-4a33-bf0e-c4b593b3c5aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6819","nonce":1093341590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6821","nonce":1093341590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6825","nonce":1093341590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6823","nonce":1093341590}]},"public_addr":"172.21.15.117:6819/1093341590","cluster_addr":"172.21.15.117:6821/1093341590","heartbeat_back_addr":"172.21.15.117:6825/1093341590","heartbeat_front_addr":"172.21.15.117:6823/1093341590","state":["exists","up"]},{"osd":7,"uuid":"36a47dd7-67f7-4ebe-8759-3a8bb6bd2ea3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6817","nonce":1448179714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6819","nonce":1448179714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6823","nonce":1448179714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6821","nonce":1448179714}]},"public_addr":"172.21.15.138:6817/1448179714","cluster_addr":"172.21.15.138:6819/1448179714","heartbeat_back_addr":"172.21.15.138:6823/1448179714","heartbeat_front_addr":"172.21.15.138:6821/1448179714","state":["exists","up"]},{"osd":8,"uuid":"118e0b5d-2619-4512-94c5-8d07a0d3a28f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6817","nonce":3960902885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6819","nonce":3960902885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6823","nonce":3960902885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6821","nonce":3960902885}]},"public_addr":"172.21.15.192:6817/3960902885","cluster_addr":"172.21.15.192:6819/3960902885","heartbeat_back_addr":"172.21.15.192:6823/3960902885","heartbeat_front_addr":"172.21.15.192:6821/3960902885","state":["exists","up"]},{"osd":9,"uuid":"47543bc6-fa0b-4e72-9742-97dcda0b7f8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6827","nonce":3453855195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6829","nonce":3453855195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6833","nonce":3453855195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6831","nonce":3453855195}]},"public_addr":"172.21.15.117:6827/3453855195","cluster_addr":"172.21.15.117:6829/3453855195","heartbeat_back_addr":"172.21.15.117:6833/3453855195","heartbeat_front_addr":"172.21.15.117:6831/3453855195","state":["exists","up"]},{"osd":10,"uuid":"5e6708b4-7a85-4fa5-a03c-27d99f7d749f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6825","nonce":903392451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6827","nonce":903392451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6831","nonce":903392451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6829","nonce":903392451}]},"public_addr":"172.21.15.138:6825/903392451","cluster_addr":"172.21.15.138:6827/903392451","heartbeat_back_addr":"172.21.15.138:6831/903392451","heartbeat_front_addr":"172.21.15.138:6829/903392451","state":["exists","up"]},{"osd":11,"uuid":"24bc2ef7-e2b4-43c6-9e6c-43ae02390415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6825","nonce":899090479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6827","nonce":899090479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6831","nonce":899090479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6829","nonce":899090479}]},"public_addr":"172.21.15.192:6825/899090479","cluster_addr":"172.21.15.192:6827/899090479","heartbeat_back_addr":"172.21.15.192:6831/899090479","heartbeat_front_addr":"172.21.15.192:6829/899090479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:49:40.742217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:05.963655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:30.164495+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:56.598899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:51:21.940331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:51:47.777445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:52:12.869007+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:52:39.454054+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:03.569226+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:29.760863+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:55.788604+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:54:20.510184+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:0/2989394419":"2024-09-22T05:48:32.406654+0000","172.21.15.117:6801/1702676172":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/1766406059":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6800/2337036514":"2024-09-22T05:47:09.112435+0000","172.21.15.117:0/985798343":"2024-09-22T05:47:09.112435+0000","172.21.15.117:0/1083273222":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/473284565":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6801/2337036514":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6800/816635165":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/2642940197":"2024-09-22T05:48:32.406654+0000","172.21.15.117:6801/816635165":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/2225537433":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/3651584169":"2024-09-22T05:47:33.789146+0000","172.21.15.117:6800/1702676172":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/1284403882":"2024-09-22T05:47:33.789146+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:55:18.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:18.157+0000 7fb1dc30f640 1 -- 172.21.15.117:0/2062164389 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb1bc0b6560 msgr2=0x7fb1bc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:18.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:18.157+0000 7fb1dc30f640 1 --2- 172.21.15.117:0/2062164389 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb1bc0b6560 0x7fb1bc0b8a70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc009b60 tx=0x7fb1cc005f90 comp rx=0 tx=0).stop 2024-09-21T05:55:18.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:18.157+0000 7fb1dc30f640 1 -- 172.21.15.117:0/2062164389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1d41094b0 msgr2=0x7fb1d419aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:18.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:18.157+0000 7fb1dc30f640 1 --2- 172.21.15.117:0/2062164389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1d41094b0 0x7fb1d419aa30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb1c800cce0 tx=0x7fb1c800a620 comp rx=0 tx=0).stop 2024-09-21T05:55:18.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:18.158+0000 7fb1dc30f640 1 -- 172.21.15.117:0/2062164389 shutdown_connections 2024-09-21T05:55:18.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:18.158+0000 7fb1dc30f640 1 --2- 172.21.15.117:0/2062164389 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb1bc0b6560 0x7fb1bc0b8a70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:18.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:18.158+0000 7fb1dc30f640 1 --2- 172.21.15.117:0/2062164389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1d41094b0 0x7fb1d419aa30 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:18.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:18.158+0000 7fb1dc30f640 1 --2- 172.21.15.117:0/2062164389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1d4109e00 0x7fb1d419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:18.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:18.158+0000 7fb1dc30f640 1 --2- 172.21.15.117:0/2062164389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1d4103b70 0x7fb1d419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:18.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:18.158+0000 7fb1dc30f640 1 -- 172.21.15.117:0/2062164389 >> 172.21.15.117:0/2062164389 conn(0x7fb1d40ff5e0 msgr2=0x7fb1d4106aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:18.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:18.158+0000 7fb1dc30f640 1 -- 172.21.15.117:0/2062164389 shutdown_connections 2024-09-21T05:55:18.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:18.158+0000 7fb1dc30f640 1 -- 172.21.15.117:0/2062164389 wait complete. 2024-09-21T05:55:18.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:18 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2062164389' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T05:55:18.494 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:18 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2062164389' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T05:55:18.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:18 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2062164389' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T05:55:18.802 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs set cephfs standby_count_wanted 0 2024-09-21T05:55:19.051 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:19.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:19 smithi192 ceph-mon[34700]: pgmap v265: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-09-21T05:55:19.486 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:19 smithi117 ceph-mon[30947]: pgmap v265: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-09-21T05:55:19.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.578+0000 7f303e907640 1 -- 172.21.15.117:0/3360141905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30381094a0 msgr2=0x7f30381098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:19.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.578+0000 7f303e907640 1 --2- 172.21.15.117:0/3360141905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30381094a0 0x7f30381098b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f302c009900 tx=0x7f302c02a410 comp rx=0 tx=0).stop 2024-09-21T05:55:19.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.579+0000 7f303e907640 1 -- 172.21.15.117:0/3360141905 shutdown_connections 2024-09-21T05:55:19.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.579+0000 7f303e907640 1 --2- 172.21.15.117:0/3360141905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30381094a0 0x7f30381098b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:19.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.579+0000 7f303e907640 1 --2- 172.21.15.117:0/3360141905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3038109df0 0x7f3038110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:19.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.579+0000 7f303e907640 1 --2- 172.21.15.117:0/3360141905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3038103b60 0x7f3038103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:19.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.579+0000 7f303e907640 1 -- 172.21.15.117:0/3360141905 >> 172.21.15.117:0/3360141905 conn(0x7f303807b850 msgr2=0x7f303807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:19.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.579+0000 7f303e907640 1 -- 172.21.15.117:0/3360141905 shutdown_connections 2024-09-21T05:55:19.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.579+0000 7f303e907640 1 -- 172.21.15.117:0/3360141905 wait complete. 2024-09-21T05:55:19.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.579+0000 7f303e907640 1 Processor -- start 2024-09-21T05:55:19.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.579+0000 7f303e907640 1 -- start start 2024-09-21T05:55:19.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.580+0000 7f303e907640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3038103b60 0x7f3038075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:19.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.580+0000 7f303e907640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30381094a0 0x7f3038075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:19.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.580+0000 7f303e907640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3038109df0 0x7f3038076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:19.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.580+0000 7f303e907640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3038113ab0 con 0x7f30381094a0 2024-09-21T05:55:19.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.580+0000 7f303e907640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3038113930 con 0x7f3038103b60 2024-09-21T05:55:19.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.580+0000 7f303e907640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3038113c30 con 0x7f3038109df0 2024-09-21T05:55:19.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.580+0000 7f30377fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30381094a0 0x7f3038075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:19.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.580+0000 7f30377fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30381094a0 0x7f3038075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40484/0 (socket says 172.21.15.117:40484) 2024-09-21T05:55:19.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.580+0000 7f30377fe640 1 -- 172.21.15.117:0/3610551083 learned_addr learned my addr 172.21.15.117:0/3610551083 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:55:19.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.580+0000 7f3037fff640 1 --2- 172.21.15.117:0/3610551083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3038103b60 0x7f3038075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:19.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.580+0000 7f303cea4640 1 --2- 172.21.15.117:0/3610551083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3038109df0 0x7f3038076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:19.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.580+0000 7f30377fe640 1 -- 172.21.15.117:0/3610551083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3038109df0 msgr2=0x7f3038076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:19.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.580+0000 7f30377fe640 1 --2- 172.21.15.117:0/3610551083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3038109df0 0x7f3038076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:19.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.580+0000 7f30377fe640 1 -- 172.21.15.117:0/3610551083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3038103b60 msgr2=0x7f3038075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:19.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.580+0000 7f30377fe640 1 --2- 172.21.15.117:0/3610551083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3038103b60 0x7f3038075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:19.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.580+0000 7f30377fe640 1 -- 172.21.15.117:0/3610551083 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30381a8740 con 0x7f30381094a0 2024-09-21T05:55:19.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.580+0000 7f30377fe640 1 --2- 172.21.15.117:0/3610551083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30381094a0 0x7f3038075c50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f302c02a930 tx=0x7f302c031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:19.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.581+0000 7f30357fa640 1 -- 172.21.15.117:0/3610551083 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c03f030 con 0x7f30381094a0 2024-09-21T05:55:19.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.581+0000 7f30357fa640 1 -- 172.21.15.117:0/3610551083 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f302c004490 con 0x7f30381094a0 2024-09-21T05:55:19.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.581+0000 7f303e907640 1 -- 172.21.15.117:0/3610551083 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30381a89d0 con 0x7f30381094a0 2024-09-21T05:55:19.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.581+0000 7f30357fa640 1 -- 172.21.15.117:0/3610551083 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c035070 con 0x7f30381094a0 2024-09-21T05:55:19.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.581+0000 7f303e907640 1 -- 172.21.15.117:0/3610551083 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30381a8eb0 con 0x7f30381094a0 2024-09-21T05:55:19.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.582+0000 7f303e907640 1 -- 172.21.15.117:0/3610551083 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3038103f70 con 0x7f30381094a0 2024-09-21T05:55:19.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.583+0000 7f30357fa640 1 -- 172.21.15.117:0/3610551083 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f302c042030 con 0x7f30381094a0 2024-09-21T05:55:19.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.584+0000 7f30357fa640 1 --2- 172.21.15.117:0/3610551083 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f30200b6560 0x7f30200b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:19.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.584+0000 7f3037fff640 1 --2- 172.21.15.117:0/3610551083 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f30200b6560 0x7f30200b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:19.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.584+0000 7f30357fa640 1 -- 172.21.15.117:0/3610551083 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 8194+0+0 (secure 0 0 0) 0x7f302c107a10 con 0x7f30381094a0 2024-09-21T05:55:19.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.584+0000 7f3037fff640 1 --2- 172.21.15.117:0/3610551083 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f30200b6560 0x7f30200b8a70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3038077160 tx=0x7f3028005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:19.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.587+0000 7f30357fa640 1 -- 172.21.15.117:0/3610551083 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f302c0d26a0 con 0x7f30381094a0 2024-09-21T05:55:19.595 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:19 smithi138 ceph-mon[34391]: pgmap v265: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-09-21T05:55:19.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:19.804+0000 7f303e907640 1 -- 172.21.15.117:0/3610551083 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f303802e640 con 0x7f30381094a0 2024-09-21T05:55:19.865 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 21 05:55:19 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-b[63794]: 2024-09-21T05:55:19.592+0000 7f5099011640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T05:55:20.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:20.221+0000 7f30357fa640 1 -- 172.21.15.117:0/3610551083 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f302c0d75e0 con 0x7f30381094a0 2024-09-21T05:55:20.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:20.223+0000 7f303e907640 1 -- 172.21.15.117:0/3610551083 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f30200b6560 msgr2=0x7f30200b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:20.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:20.223+0000 7f303e907640 1 --2- 172.21.15.117:0/3610551083 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f30200b6560 0x7f30200b8a70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3038077160 tx=0x7f3028005f90 comp rx=0 tx=0).stop 2024-09-21T05:55:20.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:20.223+0000 7f303e907640 1 -- 172.21.15.117:0/3610551083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30381094a0 msgr2=0x7f3038075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:20.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:20.223+0000 7f303e907640 1 --2- 172.21.15.117:0/3610551083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30381094a0 0x7f3038075c50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f302c02a930 tx=0x7f302c031a20 comp rx=0 tx=0).stop 2024-09-21T05:55:20.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:20.223+0000 7f303e907640 1 -- 172.21.15.117:0/3610551083 shutdown_connections 2024-09-21T05:55:20.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:20.223+0000 7f303e907640 1 --2- 172.21.15.117:0/3610551083 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f30200b6560 0x7f30200b8a70 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:20.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:20.223+0000 7f303e907640 1 --2- 172.21.15.117:0/3610551083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30381094a0 0x7f3038075c50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:20.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:20.223+0000 7f303e907640 1 --2- 172.21.15.117:0/3610551083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3038109df0 0x7f3038076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:20.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:20.223+0000 7f303e907640 1 --2- 172.21.15.117:0/3610551083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3038103b60 0x7f3038075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:20.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:20.224+0000 7f303e907640 1 -- 172.21.15.117:0/3610551083 >> 172.21.15.117:0/3610551083 conn(0x7f303807b850 msgr2=0x7f3038105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:20.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:20.224+0000 7f303e907640 1 -- 172.21.15.117:0/3610551083 shutdown_connections 2024-09-21T05:55:20.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:20.224+0000 7f303e907640 1 -- 172.21.15.117:0/3610551083 wait complete. 2024-09-21T05:55:20.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:20 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3610551083' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T05:55:20.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:20 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3610551083' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T05:55:20.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:20 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3610551083' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T05:55:20.930 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-21T05:55:20.930 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs set cephfs max_mds 5 2024-09-21T05:55:21.181 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:21.526 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:21 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3610551083' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T05:55:21.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:21 smithi117 ceph-mon[30947]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T05:55:21.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:21 smithi117 ceph-mon[30947]: pgmap v266: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-09-21T05:55:21.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:21 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3610551083' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T05:55:21.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:21 smithi138 ceph-mon[34391]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T05:55:21.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:21 smithi138 ceph-mon[34391]: pgmap v266: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-09-21T05:55:21.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:21 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3610551083' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T05:55:21.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:21 smithi192 ceph-mon[34700]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T05:55:21.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:21 smithi192 ceph-mon[34700]: pgmap v266: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-09-21T05:55:21.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.729+0000 7f59979a8640 1 -- 172.21.15.117:0/874172049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5990103b60 msgr2=0x7f5990103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:21.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.729+0000 7f59979a8640 1 --2- 172.21.15.117:0/874172049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5990103b60 0x7f5990103f70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5980009ca0 tx=0x7f598002a8c0 comp rx=0 tx=0).stop 2024-09-21T05:55:21.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.729+0000 7f59979a8640 1 -- 172.21.15.117:0/874172049 shutdown_connections 2024-09-21T05:55:21.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.729+0000 7f59979a8640 1 --2- 172.21.15.117:0/874172049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5990109df0 0x7f5990110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:21.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.729+0000 7f59979a8640 1 --2- 172.21.15.117:0/874172049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59901094a0 0x7f59901098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:21.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.729+0000 7f59979a8640 1 --2- 172.21.15.117:0/874172049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5990103b60 0x7f5990103f70 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:21.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.729+0000 7f59979a8640 1 -- 172.21.15.117:0/874172049 >> 172.21.15.117:0/874172049 conn(0x7f599007b850 msgr2=0x7f599007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:21.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.729+0000 7f59979a8640 1 -- 172.21.15.117:0/874172049 shutdown_connections 2024-09-21T05:55:21.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.729+0000 7f59979a8640 1 -- 172.21.15.117:0/874172049 wait complete. 2024-09-21T05:55:21.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.730+0000 7f59979a8640 1 Processor -- start 2024-09-21T05:55:21.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.730+0000 7f59979a8640 1 -- start start 2024-09-21T05:55:21.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.730+0000 7f59979a8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5990103b60 0x7f599019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:21.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.730+0000 7f59979a8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59901094a0 0x7f599019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:21.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.730+0000 7f59979a8640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5990109df0 0x7f599019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:21.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.730+0000 7f59979a8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5990113ab0 con 0x7f59901094a0 2024-09-21T05:55:21.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.730+0000 7f59979a8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5990113930 con 0x7f5990103b60 2024-09-21T05:55:21.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.730+0000 7f59979a8640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5990113c30 con 0x7f5990109df0 2024-09-21T05:55:21.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.730+0000 7f5994f43640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59901094a0 0x7f599019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:21.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.730+0000 7f5994f43640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59901094a0 0x7f599019aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40502/0 (socket says 172.21.15.117:40502) 2024-09-21T05:55:21.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.730+0000 7f5994f43640 1 -- 172.21.15.117:0/4162015459 learned_addr learned my addr 172.21.15.117:0/4162015459 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:55:21.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.730+0000 7f5995744640 1 --2- 172.21.15.117:0/4162015459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5990103b60 0x7f599019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:21.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.730+0000 7f5995f45640 1 --2- 172.21.15.117:0/4162015459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5990109df0 0x7f599019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:21.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.730+0000 7f5994f43640 1 -- 172.21.15.117:0/4162015459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5990109df0 msgr2=0x7f599019fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:21.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.730+0000 7f5994f43640 1 --2- 172.21.15.117:0/4162015459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5990109df0 0x7f599019fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:21.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.730+0000 7f5994f43640 1 -- 172.21.15.117:0/4162015459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5990103b60 msgr2=0x7f599019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:21.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.730+0000 7f5994f43640 1 --2- 172.21.15.117:0/4162015459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5990103b60 0x7f599019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:21.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.730+0000 7f5994f43640 1 -- 172.21.15.117:0/4162015459 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59901a0330 con 0x7f59901094a0 2024-09-21T05:55:21.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.730+0000 7f5995f45640 1 --2- 172.21.15.117:0/4162015459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5990109df0 0x7f599019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:55:21.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.731+0000 7f5994f43640 1 --2- 172.21.15.117:0/4162015459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59901094a0 0x7f599019aa20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f598400c8c0 tx=0x7f598400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:21.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.731+0000 7f598e7fc640 1 -- 172.21.15.117:0/4162015459 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5984007bb0 con 0x7f59901094a0 2024-09-21T05:55:21.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.731+0000 7f598e7fc640 1 -- 172.21.15.117:0/4162015459 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5984007d50 con 0x7f59901094a0 2024-09-21T05:55:21.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.731+0000 7f598e7fc640 1 -- 172.21.15.117:0/4162015459 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59840187c0 con 0x7f59901094a0 2024-09-21T05:55:21.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.731+0000 7f59979a8640 1 -- 172.21.15.117:0/4162015459 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59901a0620 con 0x7f59901094a0 2024-09-21T05:55:21.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.731+0000 7f59979a8640 1 -- 172.21.15.117:0/4162015459 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59900755f0 con 0x7f59901094a0 2024-09-21T05:55:21.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.732+0000 7f59979a8640 1 -- 172.21.15.117:0/4162015459 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5990104de0 con 0x7f59901094a0 2024-09-21T05:55:21.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.733+0000 7f598e7fc640 1 -- 172.21.15.117:0/4162015459 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f598400fa20 con 0x7f59901094a0 2024-09-21T05:55:21.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.734+0000 7f598e7fc640 1 --2- 172.21.15.117:0/4162015459 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f59780b6590 0x7f59780b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:21.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.734+0000 7f5995744640 1 --2- 172.21.15.117:0/4162015459 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f59780b6590 0x7f59780b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:21.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.734+0000 7f598e7fc640 1 -- 172.21.15.117:0/4162015459 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 8194+0+0 (secure 0 0 0) 0x7f59840e7680 con 0x7f59901094a0 2024-09-21T05:55:21.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.734+0000 7f5995744640 1 --2- 172.21.15.117:0/4162015459 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f59780b6590 0x7f59780b8aa0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5980005e30 tx=0x7f5980009360 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:21.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.737+0000 7f598e7fc640 1 -- 172.21.15.117:0/4162015459 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5984014030 con 0x7f59901094a0 2024-09-21T05:55:21.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:21.956+0000 7f59979a8640 1 -- 172.21.15.117:0/4162015459 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f599010bbc0 con 0x7f59901094a0 2024-09-21T05:55:22.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:22.180+0000 7f598e7fc640 1 -- 172.21.15.117:0/4162015459 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f59840adfc0 con 0x7f59901094a0 2024-09-21T05:55:22.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:22.182+0000 7f59979a8640 1 -- 172.21.15.117:0/4162015459 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f59780b6590 msgr2=0x7f59780b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:22.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:22.182+0000 7f59979a8640 1 --2- 172.21.15.117:0/4162015459 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f59780b6590 0x7f59780b8aa0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5980005e30 tx=0x7f5980009360 comp rx=0 tx=0).stop 2024-09-21T05:55:22.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:22.182+0000 7f59979a8640 1 -- 172.21.15.117:0/4162015459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59901094a0 msgr2=0x7f599019aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:22.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:22.182+0000 7f59979a8640 1 --2- 172.21.15.117:0/4162015459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59901094a0 0x7f599019aa20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f598400c8c0 tx=0x7f598400cd80 comp rx=0 tx=0).stop 2024-09-21T05:55:22.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:22.182+0000 7f59979a8640 1 -- 172.21.15.117:0/4162015459 shutdown_connections 2024-09-21T05:55:22.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:22.182+0000 7f59979a8640 1 --2- 172.21.15.117:0/4162015459 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f59780b6590 0x7f59780b8aa0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:22.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:22.182+0000 7f59979a8640 1 --2- 172.21.15.117:0/4162015459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59901094a0 0x7f599019aa20 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:22.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:22.182+0000 7f59979a8640 1 --2- 172.21.15.117:0/4162015459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5990109df0 0x7f599019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:22.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:22.182+0000 7f59979a8640 1 --2- 172.21.15.117:0/4162015459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5990103b60 0x7f599019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:22.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:22.182+0000 7f59979a8640 1 -- 172.21.15.117:0/4162015459 >> 172.21.15.117:0/4162015459 conn(0x7f599007b850 msgr2=0x7f5990107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:22.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:22.182+0000 7f59979a8640 1 -- 172.21.15.117:0/4162015459 shutdown_connections 2024-09-21T05:55:22.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:22.182+0000 7f59979a8640 1 -- 172.21.15.117:0/4162015459 wait complete. 2024-09-21T05:55:22.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:22 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/4162015459' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-21T05:55:22.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:22 smithi117 ceph-mon[30947]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T05:55:22.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:22 smithi117 ceph-mon[30947]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T05:55:22.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:22 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/4162015459' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-21T05:55:22.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:22 smithi117 ceph-mon[30947]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-21T05:55:22.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:22 smithi117 ceph-mon[30947]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-21T05:55:22.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:22 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/4162015459' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-21T05:55:22.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:22 smithi138 ceph-mon[34391]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T05:55:22.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:22 smithi138 ceph-mon[34391]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T05:55:22.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:22 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/4162015459' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-21T05:55:22.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:22 smithi138 ceph-mon[34391]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-21T05:55:22.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:22 smithi138 ceph-mon[34391]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-21T05:55:22.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:22 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/4162015459' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-21T05:55:22.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:22 smithi192 ceph-mon[34700]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-21T05:55:22.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:22 smithi192 ceph-mon[34700]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-21T05:55:22.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:22 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/4162015459' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-21T05:55:22.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:22 smithi192 ceph-mon[34700]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-21T05:55:22.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:22 smithi192 ceph-mon[34700]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-21T05:55:22.897 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs set cephfs allow_standby_replay true 2024-09-21T05:55:23.144 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:23.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:23 smithi117 ceph-mon[30947]: pgmap v267: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-21T05:55:23.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:23 smithi117 ceph-mon[30947]: mds.? [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] up:active 2024-09-21T05:55:23.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:23 smithi117 ceph-mon[30947]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T05:55:23.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:23 smithi117 ceph-mon[30947]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-21T05:55:23.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:23 smithi117 ceph-mon[30947]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-21T05:55:23.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:23 smithi117 ceph-mon[30947]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-21T05:55:23.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:23 smithi138 ceph-mon[34391]: pgmap v267: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-21T05:55:23.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:23 smithi138 ceph-mon[34391]: mds.? [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] up:active 2024-09-21T05:55:23.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:23 smithi138 ceph-mon[34391]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T05:55:23.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:23 smithi138 ceph-mon[34391]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-21T05:55:23.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:23 smithi138 ceph-mon[34391]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-21T05:55:23.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:23 smithi138 ceph-mon[34391]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-21T05:55:23.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.630+0000 7f9a30dd1640 1 -- 172.21.15.117:0/19714588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a2c1094a0 msgr2=0x7f9a2c1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:23.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.630+0000 7f9a30dd1640 1 --2- 172.21.15.117:0/19714588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a2c1094a0 0x7f9a2c1098b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009900 tx=0x7f9a1402a410 comp rx=0 tx=0).stop 2024-09-21T05:55:23.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.630+0000 7f9a30dd1640 1 -- 172.21.15.117:0/19714588 shutdown_connections 2024-09-21T05:55:23.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.630+0000 7f9a30dd1640 1 --2- 172.21.15.117:0/19714588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a2c1094a0 0x7f9a2c1098b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:23.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.630+0000 7f9a30dd1640 1 --2- 172.21.15.117:0/19714588 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a2c109df0 0x7f9a2c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:23.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.630+0000 7f9a30dd1640 1 --2- 172.21.15.117:0/19714588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a2c103b60 0x7f9a2c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:23.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.630+0000 7f9a30dd1640 1 -- 172.21.15.117:0/19714588 >> 172.21.15.117:0/19714588 conn(0x7f9a2c07b850 msgr2=0x7f9a2c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:23.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.630+0000 7f9a30dd1640 1 -- 172.21.15.117:0/19714588 shutdown_connections 2024-09-21T05:55:23.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.630+0000 7f9a30dd1640 1 -- 172.21.15.117:0/19714588 wait complete. 2024-09-21T05:55:23.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.631+0000 7f9a30dd1640 1 Processor -- start 2024-09-21T05:55:23.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.631+0000 7f9a30dd1640 1 -- start start 2024-09-21T05:55:23.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.631+0000 7f9a30dd1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a2c103b60 0x7f9a2c19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:23.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.631+0000 7f9a30dd1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a2c1094a0 0x7f9a2c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:23.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.631+0000 7f9a30dd1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a2c109df0 0x7f9a2c19fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:23.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.631+0000 7f9a30dd1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c113ab0 con 0x7f9a2c1094a0 2024-09-21T05:55:23.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.631+0000 7f9a30dd1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c113930 con 0x7f9a2c109df0 2024-09-21T05:55:23.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.631+0000 7f9a30dd1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c113c30 con 0x7f9a2c103b60 2024-09-21T05:55:23.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.631+0000 7f9a29d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a2c1094a0 0x7f9a2c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:23.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.631+0000 7f9a29d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a2c1094a0 0x7f9a2c19ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40524/0 (socket says 172.21.15.117:40524) 2024-09-21T05:55:23.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.631+0000 7f9a29d9b640 1 -- 172.21.15.117:0/132347520 learned_addr learned my addr 172.21.15.117:0/132347520 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:55:23.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.631+0000 7f9a2a59c640 1 --2- 172.21.15.117:0/132347520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a2c103b60 0x7f9a2c19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:23.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.631+0000 7f9a2ad9d640 1 --2- 172.21.15.117:0/132347520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a2c109df0 0x7f9a2c19fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:23.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.632+0000 7f9a29d9b640 1 -- 172.21.15.117:0/132347520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a2c103b60 msgr2=0x7f9a2c19a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:23.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.632+0000 7f9a29d9b640 1 --2- 172.21.15.117:0/132347520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a2c103b60 0x7f9a2c19a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:23.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.632+0000 7f9a29d9b640 1 -- 172.21.15.117:0/132347520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a2c109df0 msgr2=0x7f9a2c19fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:23.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.632+0000 7f9a29d9b640 1 --2- 172.21.15.117:0/132347520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a2c109df0 0x7f9a2c19fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:23.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.632+0000 7f9a29d9b640 1 -- 172.21.15.117:0/132347520 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a2c1a0430 con 0x7f9a2c1094a0 2024-09-21T05:55:23.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.632+0000 7f9a29d9b640 1 --2- 172.21.15.117:0/132347520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a2c1094a0 0x7f9a2c19ab20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9a1402a930 tx=0x7f9a14004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:23.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.632+0000 7f9a237fe640 1 -- 172.21.15.117:0/132347520 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1403f030 con 0x7f9a2c1094a0 2024-09-21T05:55:23.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.632+0000 7f9a30dd1640 1 -- 172.21.15.117:0/132347520 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a2c1a06c0 con 0x7f9a2c1094a0 2024-09-21T05:55:23.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.632+0000 7f9a237fe640 1 -- 172.21.15.117:0/132347520 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a14003cb0 con 0x7f9a2c1094a0 2024-09-21T05:55:23.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.632+0000 7f9a237fe640 1 -- 172.21.15.117:0/132347520 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a14035070 con 0x7f9a2c1094a0 2024-09-21T05:55:23.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.632+0000 7f9a30dd1640 1 -- 172.21.15.117:0/132347520 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a2c1a0ba0 con 0x7f9a2c1094a0 2024-09-21T05:55:23.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.633+0000 7f9a30dd1640 1 -- 172.21.15.117:0/132347520 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a2c103f70 con 0x7f9a2c1094a0 2024-09-21T05:55:23.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.635+0000 7f9a237fe640 1 -- 172.21.15.117:0/132347520 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f9a14042030 con 0x7f9a2c1094a0 2024-09-21T05:55:23.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.636+0000 7f9a237fe640 1 --2- 172.21.15.117:0/132347520 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9a180b6590 0x7f9a180b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:23.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.636+0000 7f9a2a59c640 1 --2- 172.21.15.117:0/132347520 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9a180b6590 0x7f9a180b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:23.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.636+0000 7f9a237fe640 1 -- 172.21.15.117:0/132347520 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 8194+0+0 (secure 0 0 0) 0x7f9a14107940 con 0x7f9a2c1094a0 2024-09-21T05:55:23.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.636+0000 7f9a2a59c640 1 --2- 172.21.15.117:0/132347520 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9a180b6590 0x7f9a180b8aa0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0058c0 tx=0x7f9a1c00a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:23.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.639+0000 7f9a237fe640 1 -- 172.21.15.117:0/132347520 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9a140d25d0 con 0x7f9a2c1094a0 2024-09-21T05:55:23.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:23 smithi192 ceph-mon[34700]: pgmap v267: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-21T05:55:23.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:23 smithi192 ceph-mon[34700]: mds.? [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] up:active 2024-09-21T05:55:23.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:23 smithi192 ceph-mon[34700]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-21T05:55:23.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:23 smithi192 ceph-mon[34700]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-21T05:55:23.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:23 smithi192 ceph-mon[34700]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-21T05:55:23.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:23 smithi192 ceph-mon[34700]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-21T05:55:23.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:23.854+0000 7f9a30dd1640 1 -- 172.21.15.117:0/132347520 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f9a2c19b6b0 con 0x7f9a2c1094a0 2024-09-21T05:55:24.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:24.189+0000 7f9a237fe640 1 -- 172.21.15.117:0/132347520 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7f9a140d7510 con 0x7f9a2c1094a0 2024-09-21T05:55:24.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:24.192+0000 7f9a30dd1640 1 -- 172.21.15.117:0/132347520 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9a180b6590 msgr2=0x7f9a180b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:24.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:24.192+0000 7f9a30dd1640 1 --2- 172.21.15.117:0/132347520 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9a180b6590 0x7f9a180b8aa0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0058c0 tx=0x7f9a1c00a2d0 comp rx=0 tx=0).stop 2024-09-21T05:55:24.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:24.192+0000 7f9a30dd1640 1 -- 172.21.15.117:0/132347520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a2c1094a0 msgr2=0x7f9a2c19ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:24.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:24.192+0000 7f9a30dd1640 1 --2- 172.21.15.117:0/132347520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a2c1094a0 0x7f9a2c19ab20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9a1402a930 tx=0x7f9a14004270 comp rx=0 tx=0).stop 2024-09-21T05:55:24.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:24.192+0000 7f9a30dd1640 1 -- 172.21.15.117:0/132347520 shutdown_connections 2024-09-21T05:55:24.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:24.192+0000 7f9a30dd1640 1 --2- 172.21.15.117:0/132347520 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9a180b6590 0x7f9a180b8aa0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:24.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:24.192+0000 7f9a30dd1640 1 --2- 172.21.15.117:0/132347520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a2c1094a0 0x7f9a2c19ab20 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:24.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:24.192+0000 7f9a30dd1640 1 --2- 172.21.15.117:0/132347520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a2c109df0 0x7f9a2c19fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:24.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:24.192+0000 7f9a30dd1640 1 --2- 172.21.15.117:0/132347520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a2c103b60 0x7f9a2c19a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:24.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:24.192+0000 7f9a30dd1640 1 -- 172.21.15.117:0/132347520 >> 172.21.15.117:0/132347520 conn(0x7f9a2c07b850 msgr2=0x7f9a2c105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:24.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:24.192+0000 7f9a30dd1640 1 -- 172.21.15.117:0/132347520 shutdown_connections 2024-09-21T05:55:24.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:24.192+0000 7f9a30dd1640 1 -- 172.21.15.117:0/132347520 wait complete. 2024-09-21T05:55:24.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:24 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/132347520' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T05:55:24.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:24 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/132347520' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T05:55:24.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:24 smithi117 ceph-mon[30947]: mds.? [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] up:active 2024-09-21T05:55:24.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:24 smithi117 ceph-mon[30947]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-21T05:55:24.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:24 smithi117 ceph-mon[30947]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-21T05:55:24.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:24 smithi117 ceph-mon[30947]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-09-21T05:55:24.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:24 smithi117 ceph-mon[30947]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-09-21T05:55:24.594 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:24 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/132347520' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T05:55:24.595 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:24 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/132347520' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T05:55:24.595 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:24 smithi138 ceph-mon[34391]: mds.? [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] up:active 2024-09-21T05:55:24.595 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:24 smithi138 ceph-mon[34391]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-21T05:55:24.595 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:24 smithi138 ceph-mon[34391]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-21T05:55:24.595 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:24 smithi138 ceph-mon[34391]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-09-21T05:55:24.595 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:24 smithi138 ceph-mon[34391]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-09-21T05:55:24.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:24 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/132347520' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T05:55:24.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:24 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/132347520' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T05:55:24.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:24 smithi192 ceph-mon[34700]: mds.? [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] up:active 2024-09-21T05:55:24.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:24 smithi192 ceph-mon[34700]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-21T05:55:24.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:24 smithi192 ceph-mon[34700]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-21T05:55:24.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:24 smithi192 ceph-mon[34700]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:creating} 8 up:standby 2024-09-21T05:55:24.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:24 smithi192 ceph-mon[34700]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-09-21T05:55:24.865 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 21 05:55:24 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-b[63794]: 2024-09-21T05:55:24.593+0000 7f5099011640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-21T05:55:24.865 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 21 05:55:24 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-b[63794]: 2024-09-21T05:55:24.593+0000 7f5099011640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-21T05:55:24.891 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs set cephfs session_timeout 300 2024-09-21T05:55:25.136 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:25.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:25 smithi117 ceph-mon[30947]: pgmap v268: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 6 op/s 2024-09-21T05:55:25.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:25 smithi117 ceph-mon[30947]: mds.? [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] up:active 2024-09-21T05:55:25.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:25 smithi117 ceph-mon[30947]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-21T05:55:25.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:25 smithi117 ceph-mon[30947]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T05:55:25.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:25 smithi117 ceph-mon[30947]: Cluster is now healthy 2024-09-21T05:55:25.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:25 smithi117 ceph-mon[30947]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-09-21T05:55:25.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:25 smithi117 ceph-mon[30947]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-21T05:55:25.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:25 smithi117 ceph-mon[30947]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-09-21T05:55:25.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:25 smithi138 ceph-mon[34391]: pgmap v268: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 6 op/s 2024-09-21T05:55:25.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:25 smithi138 ceph-mon[34391]: mds.? [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] up:active 2024-09-21T05:55:25.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:25 smithi138 ceph-mon[34391]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-21T05:55:25.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:25 smithi138 ceph-mon[34391]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T05:55:25.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:25 smithi138 ceph-mon[34391]: Cluster is now healthy 2024-09-21T05:55:25.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:25 smithi138 ceph-mon[34391]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-09-21T05:55:25.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:25 smithi138 ceph-mon[34391]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-21T05:55:25.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:25 smithi138 ceph-mon[34391]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-09-21T05:55:25.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.623+0000 7f820024e640 1 -- 172.21.15.117:0/3688761507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81f8103c30 msgr2=0x7f81f8104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:25.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.623+0000 7f820024e640 1 --2- 172.21.15.117:0/3688761507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81f8103c30 0x7f81f8104040 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f81e8009ca0 tx=0x7f81e802a8c0 comp rx=0 tx=0).stop 2024-09-21T05:55:25.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.623+0000 7f820024e640 1 -- 172.21.15.117:0/3688761507 shutdown_connections 2024-09-21T05:55:25.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.623+0000 7f820024e640 1 --2- 172.21.15.117:0/3688761507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81f8109ec0 0x7f81f81106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:25.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.623+0000 7f820024e640 1 --2- 172.21.15.117:0/3688761507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81f8109570 0x7f81f8109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:25.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.623+0000 7f820024e640 1 --2- 172.21.15.117:0/3688761507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81f8103c30 0x7f81f8104040 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:25.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.623+0000 7f820024e640 1 -- 172.21.15.117:0/3688761507 >> 172.21.15.117:0/3688761507 conn(0x7f81f807b8d0 msgr2=0x7f81f807bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:25.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.623+0000 7f820024e640 1 -- 172.21.15.117:0/3688761507 shutdown_connections 2024-09-21T05:55:25.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.623+0000 7f820024e640 1 -- 172.21.15.117:0/3688761507 wait complete. 2024-09-21T05:55:25.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.624+0000 7f820024e640 1 Processor -- start 2024-09-21T05:55:25.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.624+0000 7f820024e640 1 -- start start 2024-09-21T05:55:25.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.624+0000 7f820024e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81f8103c30 0x7f81f819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:25.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.624+0000 7f820024e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81f8109570 0x7f81f819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:25.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.624+0000 7f820024e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81f8109ec0 0x7f81f819fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:25.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.624+0000 7f820024e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f81f8113ae0 con 0x7f81f8103c30 2024-09-21T05:55:25.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.624+0000 7f820024e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81f8113960 con 0x7f81f8109ec0 2024-09-21T05:55:25.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.624+0000 7f820024e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f81f8113c60 con 0x7f81f8109570 2024-09-21T05:55:25.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.624+0000 7f81fdfea640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81f8103c30 0x7f81f819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:25.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.624+0000 7f81fdfea640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81f8103c30 0x7f81f819a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49842/0 (socket says 172.21.15.117:49842) 2024-09-21T05:55:25.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.624+0000 7f81fdfea640 1 -- 172.21.15.117:0/1949241200 learned_addr learned my addr 172.21.15.117:0/1949241200 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:55:25.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.624+0000 7f81fe7eb640 1 --2- 172.21.15.117:0/1949241200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81f8109ec0 0x7f81f819fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:25.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.624+0000 7f81fd7e9640 1 --2- 172.21.15.117:0/1949241200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81f8109570 0x7f81f819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:25.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.624+0000 7f81fdfea640 1 -- 172.21.15.117:0/1949241200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81f8109570 msgr2=0x7f81f819ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:25.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.624+0000 7f81fdfea640 1 --2- 172.21.15.117:0/1949241200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81f8109570 0x7f81f819ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:25.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.624+0000 7f81fdfea640 1 -- 172.21.15.117:0/1949241200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81f8109ec0 msgr2=0x7f81f819fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:25.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.624+0000 7f81fdfea640 1 --2- 172.21.15.117:0/1949241200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81f8109ec0 0x7f81f819fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:25.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.625+0000 7f81fdfea640 1 -- 172.21.15.117:0/1949241200 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f81a04e0 con 0x7f81f8103c30 2024-09-21T05:55:25.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.625+0000 7f81fe7eb640 1 --2- 172.21.15.117:0/1949241200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81f8109ec0 0x7f81f819fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:55:25.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.625+0000 7f81fdfea640 1 --2- 172.21.15.117:0/1949241200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81f8103c30 0x7f81f819a570 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f81e802ade0 tx=0x7f81e8031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:25.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.625+0000 7f81f6ffd640 1 -- 172.21.15.117:0/1949241200 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e8003b70 con 0x7f81f8103c30 2024-09-21T05:55:25.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.625+0000 7f81f6ffd640 1 -- 172.21.15.117:0/1949241200 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81e8003d10 con 0x7f81f8103c30 2024-09-21T05:55:25.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.625+0000 7f81f6ffd640 1 -- 172.21.15.117:0/1949241200 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e802fa50 con 0x7f81f8103c30 2024-09-21T05:55:25.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.625+0000 7f820024e640 1 -- 172.21.15.117:0/1949241200 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81f81a0770 con 0x7f81f8103c30 2024-09-21T05:55:25.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.625+0000 7f820024e640 1 -- 172.21.15.117:0/1949241200 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81f81a6f30 con 0x7f81f8103c30 2024-09-21T05:55:25.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.626+0000 7f820024e640 1 -- 172.21.15.117:0/1949241200 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81f8104040 con 0x7f81f8103c30 2024-09-21T05:55:25.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.627+0000 7f81f6ffd640 1 -- 172.21.15.117:0/1949241200 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7f81e8004580 con 0x7f81f8103c30 2024-09-21T05:55:25.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.628+0000 7f81f6ffd640 1 --2- 172.21.15.117:0/1949241200 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f81e00b6560 0x7f81e00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:25.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.628+0000 7f81fd7e9640 1 --2- 172.21.15.117:0/1949241200 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f81e00b6560 0x7f81e00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:25.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.628+0000 7f81f6ffd640 1 -- 172.21.15.117:0/1949241200 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 8194+0+0 (secure 0 0 0) 0x7f81e8108680 con 0x7f81f8103c30 2024-09-21T05:55:25.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.629+0000 7f81fd7e9640 1 --2- 172.21.15.117:0/1949241200 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f81e00b6560 0x7f81e00b8a70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f81f8106590 tx=0x7f81ec005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:25.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.633+0000 7f81f6ffd640 1 -- 172.21.15.117:0/1949241200 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f81e80cef30 con 0x7f81f8103c30 2024-09-21T05:55:25.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:25 smithi192 ceph-mon[34700]: pgmap v268: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 6 op/s 2024-09-21T05:55:25.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:25 smithi192 ceph-mon[34700]: mds.? [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] up:active 2024-09-21T05:55:25.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:25 smithi192 ceph-mon[34700]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-21T05:55:25.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:25 smithi192 ceph-mon[34700]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-21T05:55:25.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:25 smithi192 ceph-mon[34700]: Cluster is now healthy 2024-09-21T05:55:25.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:25 smithi192 ceph-mon[34700]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=f=up:active} 8 up:standby 2024-09-21T05:55:25.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:25 smithi192 ceph-mon[34700]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-21T05:55:25.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:25 smithi192 ceph-mon[34700]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-09-21T05:55:25.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:25.848+0000 7f820024e640 1 -- 172.21.15.117:0/1949241200 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f81f810bcc0 con 0x7f81f8103c30 2024-09-21T05:55:26.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:26.204+0000 7f81f6ffd640 1 -- 172.21.15.117:0/1949241200 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7f81e80d3310 con 0x7f81f8103c30 2024-09-21T05:55:26.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:26.207+0000 7f820024e640 1 -- 172.21.15.117:0/1949241200 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f81e00b6560 msgr2=0x7f81e00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:26.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:26.207+0000 7f820024e640 1 --2- 172.21.15.117:0/1949241200 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f81e00b6560 0x7f81e00b8a70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f81f8106590 tx=0x7f81ec005f90 comp rx=0 tx=0).stop 2024-09-21T05:55:26.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:26.207+0000 7f820024e640 1 -- 172.21.15.117:0/1949241200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81f8103c30 msgr2=0x7f81f819a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:26.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:26.207+0000 7f820024e640 1 --2- 172.21.15.117:0/1949241200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81f8103c30 0x7f81f819a570 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f81e802ade0 tx=0x7f81e8031770 comp rx=0 tx=0).stop 2024-09-21T05:55:26.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:26.207+0000 7f820024e640 1 -- 172.21.15.117:0/1949241200 shutdown_connections 2024-09-21T05:55:26.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:26.207+0000 7f820024e640 1 --2- 172.21.15.117:0/1949241200 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f81e00b6560 0x7f81e00b8a70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:26.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:26.207+0000 7f820024e640 1 --2- 172.21.15.117:0/1949241200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81f8109ec0 0x7f81f819fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:26.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:26.207+0000 7f820024e640 1 --2- 172.21.15.117:0/1949241200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81f8109570 0x7f81f819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:26.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:26.207+0000 7f820024e640 1 --2- 172.21.15.117:0/1949241200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81f8103c30 0x7f81f819a570 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:26.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:26.207+0000 7f820024e640 1 -- 172.21.15.117:0/1949241200 >> 172.21.15.117:0/1949241200 conn(0x7f81f807b8d0 msgr2=0x7f81f8107710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:26.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:26.207+0000 7f820024e640 1 -- 172.21.15.117:0/1949241200 shutdown_connections 2024-09-21T05:55:26.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:26.207+0000 7f820024e640 1 -- 172.21.15.117:0/1949241200 wait complete. 2024-09-21T05:55:26.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:26 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/1949241200' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T05:55:26.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:26 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/1949241200' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T05:55:26.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:26 smithi117 ceph-mon[30947]: mds.? [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] up:active 2024-09-21T05:55:26.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:26 smithi117 ceph-mon[30947]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-21T05:55:26.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:26 smithi117 ceph-mon[30947]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T05:55:26.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:26 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/1949241200' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T05:55:26.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:26 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/1949241200' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T05:55:26.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:26 smithi138 ceph-mon[34391]: mds.? [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] up:active 2024-09-21T05:55:26.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:26 smithi138 ceph-mon[34391]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-21T05:55:26.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:26 smithi138 ceph-mon[34391]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T05:55:26.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:26 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/1949241200' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T05:55:26.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:26 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/1949241200' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T05:55:26.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:26 smithi192 ceph-mon[34700]: mds.? [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] up:active 2024-09-21T05:55:26.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:26 smithi192 ceph-mon[34700]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-21T05:55:26.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:26 smithi192 ceph-mon[34700]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T05:55:26.943 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-09-21T05:55:26.943 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs subvolumegroup create cephfs qa 2024-09-21T05:55:27.191 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:27.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:27 smithi192 ceph-mon[34700]: pgmap v269: 97 pgs: 97 active+clean; 583 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s wr, 12 op/s 2024-09-21T05:55:27.526 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:27 smithi117 ceph-mon[30947]: pgmap v269: 97 pgs: 97 active+clean; 583 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s wr, 12 op/s 2024-09-21T05:55:27.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:27 smithi138 ceph-mon[34391]: pgmap v269: 97 pgs: 97 active+clean; 583 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s wr, 12 op/s 2024-09-21T05:55:27.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.666+0000 7fb253a17640 1 -- 172.21.15.117:0/2784380699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb24c1049d0 msgr2=0x7fb24c104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:27.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.666+0000 7fb253a17640 1 --2- 172.21.15.117:0/2784380699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb24c1049d0 0x7fb24c104de0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb23c009900 tx=0x7fb23c02a410 comp rx=0 tx=0).stop 2024-09-21T05:55:27.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.666+0000 7fb253a17640 1 -- 172.21.15.117:0/2784380699 shutdown_connections 2024-09-21T05:55:27.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.666+0000 7fb253a17640 1 --2- 172.21.15.117:0/2784380699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb24c10de30 0x7fb24c110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:27.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.666+0000 7fb253a17640 1 --2- 172.21.15.117:0/2784380699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb24c105320 0x7fb24c10d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:27.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.666+0000 7fb253a17640 1 --2- 172.21.15.117:0/2784380699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb24c1049d0 0x7fb24c104de0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:27.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.666+0000 7fb253a17640 1 -- 172.21.15.117:0/2784380699 >> 172.21.15.117:0/2784380699 conn(0x7fb24c079740 msgr2=0x7fb24c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:27.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.666+0000 7fb253a17640 1 -- 172.21.15.117:0/2784380699 shutdown_connections 2024-09-21T05:55:27.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.666+0000 7fb253a17640 1 -- 172.21.15.117:0/2784380699 wait complete. 2024-09-21T05:55:27.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.667+0000 7fb253a17640 1 Processor -- start 2024-09-21T05:55:27.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.667+0000 7fb253a17640 1 -- start start 2024-09-21T05:55:27.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.667+0000 7fb253a17640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb24c1049d0 0x7fb24c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:27.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.667+0000 7fb253a17640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb24c105320 0x7fb24c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:27.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.667+0000 7fb253a17640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb24c10de30 0x7fb24c19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:27.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.667+0000 7fb253a17640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb24c113a90 con 0x7fb24c1049d0 2024-09-21T05:55:27.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.667+0000 7fb253a17640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb24c113910 con 0x7fb24c105320 2024-09-21T05:55:27.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.667+0000 7fb253a17640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb24c113c10 con 0x7fb24c10de30 2024-09-21T05:55:27.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.667+0000 7fb2517b3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb24c1049d0 0x7fb24c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:27.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.667+0000 7fb2517b3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb24c1049d0 0x7fb24c19a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49862/0 (socket says 172.21.15.117:49862) 2024-09-21T05:55:27.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.667+0000 7fb2517b3640 1 -- 172.21.15.117:0/2735131116 learned_addr learned my addr 172.21.15.117:0/2735131116 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:55:27.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.667+0000 7fb251fb4640 1 --2- 172.21.15.117:0/2735131116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb24c10de30 0x7fb24c19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:27.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.667+0000 7fb2517b3640 1 -- 172.21.15.117:0/2735131116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb24c10de30 msgr2=0x7fb24c19fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:27.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.667+0000 7fb2517b3640 1 --2- 172.21.15.117:0/2735131116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb24c10de30 0x7fb24c19fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:27.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.667+0000 7fb2517b3640 1 -- 172.21.15.117:0/2735131116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb24c105320 msgr2=0x7fb24c19aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:27.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.667+0000 7fb250fb2640 1 --2- 172.21.15.117:0/2735131116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb24c105320 0x7fb24c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:27.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.667+0000 7fb2517b3640 1 --2- 172.21.15.117:0/2735131116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb24c105320 0x7fb24c19aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:27.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.667+0000 7fb2517b3640 1 -- 172.21.15.117:0/2735131116 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb24c1a03b0 con 0x7fb24c1049d0 2024-09-21T05:55:27.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.668+0000 7fb2517b3640 1 --2- 172.21.15.117:0/2735131116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb24c1049d0 0x7fb24c19a500 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb23c031a90 tx=0x7fb23c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:27.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.668+0000 7fb24a7fc640 1 -- 172.21.15.117:0/2735131116 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c03f030 con 0x7fb24c1049d0 2024-09-21T05:55:27.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.668+0000 7fb24a7fc640 1 -- 172.21.15.117:0/2735131116 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb23c003cb0 con 0x7fb24c1049d0 2024-09-21T05:55:27.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.668+0000 7fb24a7fc640 1 -- 172.21.15.117:0/2735131116 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23c035070 con 0x7fb24c1049d0 2024-09-21T05:55:27.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.668+0000 7fb253a17640 1 -- 172.21.15.117:0/2735131116 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb24c1a0640 con 0x7fb24c1049d0 2024-09-21T05:55:27.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.668+0000 7fb253a17640 1 -- 172.21.15.117:0/2735131116 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb24c1a0b70 con 0x7fb24c1049d0 2024-09-21T05:55:27.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.669+0000 7fb253a17640 1 -- 172.21.15.117:0/2735131116 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb24c06c240 con 0x7fb24c1049d0 2024-09-21T05:55:27.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.670+0000 7fb24a7fc640 1 -- 172.21.15.117:0/2735131116 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 20) ==== 154530+0+0 (secure 0 0 0) 0x7fb23c042030 con 0x7fb24c1049d0 2024-09-21T05:55:27.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.671+0000 7fb24a7fc640 1 --2- 172.21.15.117:0/2735131116 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb2340b6610 0x7fb2340b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:27.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.671+0000 7fb250fb2640 1 --2- 172.21.15.117:0/2735131116 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb2340b6610 0x7fb2340b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:27.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.671+0000 7fb24a7fc640 1 -- 172.21.15.117:0/2735131116 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 8194+0+0 (secure 0 0 0) 0x7fb23c107d30 con 0x7fb24c1049d0 2024-09-21T05:55:27.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.672+0000 7fb250fb2640 1 --2- 172.21.15.117:0/2735131116 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb2340b6610 0x7fb2340b8b20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb24c079ef0 tx=0x7fb240005cc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:27.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.675+0000 7fb24a7fc640 1 -- 172.21.15.117:0/2735131116 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb24c1a0b70 con 0x7fb24c1049d0 2024-09-21T05:55:27.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.875+0000 7fb253a17640 1 -- 172.21.15.117:0/2735131116 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fb24c063800 con 0x7fb2340b6610 2024-09-21T05:55:27.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.907+0000 7fb24a7fc640 1 -- 172.21.15.117:0/2735131116 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb24c063800 con 0x7fb2340b6610 2024-09-21T05:55:27.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.909+0000 7fb253a17640 1 -- 172.21.15.117:0/2735131116 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb2340b6610 msgr2=0x7fb2340b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:27.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.909+0000 7fb253a17640 1 --2- 172.21.15.117:0/2735131116 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb2340b6610 0x7fb2340b8b20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb24c079ef0 tx=0x7fb240005cc0 comp rx=0 tx=0).stop 2024-09-21T05:55:27.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.909+0000 7fb253a17640 1 -- 172.21.15.117:0/2735131116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb24c1049d0 msgr2=0x7fb24c19a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:27.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.909+0000 7fb253a17640 1 --2- 172.21.15.117:0/2735131116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb24c1049d0 0x7fb24c19a500 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb23c031a90 tx=0x7fb23c004270 comp rx=0 tx=0).stop 2024-09-21T05:55:27.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.909+0000 7fb253a17640 1 -- 172.21.15.117:0/2735131116 shutdown_connections 2024-09-21T05:55:27.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.909+0000 7fb253a17640 1 --2- 172.21.15.117:0/2735131116 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb2340b6610 0x7fb2340b8b20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:27.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.909+0000 7fb253a17640 1 --2- 172.21.15.117:0/2735131116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb24c10de30 0x7fb24c19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:27.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.909+0000 7fb253a17640 1 --2- 172.21.15.117:0/2735131116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb24c105320 0x7fb24c19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:27.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.909+0000 7fb253a17640 1 --2- 172.21.15.117:0/2735131116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb24c1049d0 0x7fb24c19a500 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:27.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.909+0000 7fb253a17640 1 -- 172.21.15.117:0/2735131116 >> 172.21.15.117:0/2735131116 conn(0x7fb24c079740 msgr2=0x7fb24c0ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:27.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.909+0000 7fb253a17640 1 -- 172.21.15.117:0/2735131116 shutdown_connections 2024-09-21T05:55:27.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:27.909+0000 7fb253a17640 1 -- 172.21.15.117:0/2735131116 wait complete. 2024-09-21T05:55:28.271 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:55:27 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: 2024-09-21T05:55:27.880+0000 7f95aed1b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:55:28.272 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:55:27 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: 2024-09-21T05:55:27.880+0000 7f95aed1b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:55:28.272 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:55:27 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: 2024-09-21T05:55:27.880+0000 7f95aed1b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:55:28.272 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:55:27 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: 2024-09-21T05:55:27.880+0000 7f95aed1b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:55:28.272 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:55:27 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: 2024-09-21T05:55:27.880+0000 7f95aed1b640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:55:28.667 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-21T05:55:28.912 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:29.403 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.400+0000 7fb21be1e640 1 -- 172.21.15.117:0/3596102852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb214102a90 msgr2=0x7fb214102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:29.403 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.400+0000 7fb21be1e640 1 --2- 172.21.15.117:0/3596102852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb214102a90 0x7fb214102ea0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb210009900 tx=0x7fb21002a410 comp rx=0 tx=0).stop 2024-09-21T05:55:29.403 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.400+0000 7fb21be1e640 1 -- 172.21.15.117:0/3596102852 shutdown_connections 2024-09-21T05:55:29.403 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.400+0000 7fb21be1e640 1 --2- 172.21.15.117:0/3596102852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb214108d20 0x7fb21410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:29.403 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.400+0000 7fb21be1e640 1 --2- 172.21.15.117:0/3596102852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2141083d0 0x7fb2141087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:29.403 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.400+0000 7fb21be1e640 1 --2- 172.21.15.117:0/3596102852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb214102a90 0x7fb214102ea0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:29.403 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.400+0000 7fb21be1e640 1 -- 172.21.15.117:0/3596102852 >> 172.21.15.117:0/3596102852 conn(0x7fb214079750 msgr2=0x7fb214079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:29.403 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.401+0000 7fb21be1e640 1 -- 172.21.15.117:0/3596102852 shutdown_connections 2024-09-21T05:55:29.403 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.401+0000 7fb21be1e640 1 -- 172.21.15.117:0/3596102852 wait complete. 2024-09-21T05:55:29.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.401+0000 7fb21be1e640 1 Processor -- start 2024-09-21T05:55:29.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.401+0000 7fb21be1e640 1 -- start start 2024-09-21T05:55:29.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.402+0000 7fb21be1e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb214102a90 0x7fb214199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:29.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.402+0000 7fb21be1e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2141083d0 0x7fb2141998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:29.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.402+0000 7fb21be1e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb214108d20 0x7fb21419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:29.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.402+0000 7fb21be1e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb214112990 con 0x7fb214102a90 2024-09-21T05:55:29.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.402+0000 7fb21be1e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb214112810 con 0x7fb2141083d0 2024-09-21T05:55:29.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.402+0000 7fb21be1e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb214112b10 con 0x7fb214108d20 2024-09-21T05:55:29.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.402+0000 7fb219bba640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb214102a90 0x7fb214199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:29.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.402+0000 7fb219bba640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb214102a90 0x7fb214199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49902/0 (socket says 172.21.15.117:49902) 2024-09-21T05:55:29.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.402+0000 7fb219bba640 1 -- 172.21.15.117:0/1188553560 learned_addr learned my addr 172.21.15.117:0/1188553560 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:55:29.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.402+0000 7fb2193b9640 1 --2- 172.21.15.117:0/1188553560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2141083d0 0x7fb2141998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:29.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.402+0000 7fb21a3bb640 1 --2- 172.21.15.117:0/1188553560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb214108d20 0x7fb21419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:29.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.402+0000 7fb219bba640 1 -- 172.21.15.117:0/1188553560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb214108d20 msgr2=0x7fb21419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:29.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.402+0000 7fb219bba640 1 --2- 172.21.15.117:0/1188553560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb214108d20 0x7fb21419eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:29.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.402+0000 7fb219bba640 1 -- 172.21.15.117:0/1188553560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2141083d0 msgr2=0x7fb2141998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:29.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.402+0000 7fb219bba640 1 --2- 172.21.15.117:0/1188553560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2141083d0 0x7fb2141998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:29.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.402+0000 7fb219bba640 1 -- 172.21.15.117:0/1188553560 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb21419f1d0 con 0x7fb214102a90 2024-09-21T05:55:29.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.403+0000 7fb219bba640 1 --2- 172.21.15.117:0/1188553560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb214102a90 0x7fb214199320 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb210005940 tx=0x7fb2100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:29.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.403+0000 7fb20affd640 1 -- 172.21.15.117:0/1188553560 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21003f030 con 0x7fb214102a90 2024-09-21T05:55:29.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.403+0000 7fb21be1e640 1 -- 172.21.15.117:0/1188553560 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb21419f460 con 0x7fb214102a90 2024-09-21T05:55:29.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.403+0000 7fb20affd640 1 -- 172.21.15.117:0/1188553560 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb210003cb0 con 0x7fb214102a90 2024-09-21T05:55:29.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.403+0000 7fb20affd640 1 -- 172.21.15.117:0/1188553560 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb210035070 con 0x7fb214102a90 2024-09-21T05:55:29.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.403+0000 7fb21be1e640 1 -- 172.21.15.117:0/1188553560 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb21419f990 con 0x7fb214102a90 2024-09-21T05:55:29.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.404+0000 7fb21be1e640 1 -- 172.21.15.117:0/1188553560 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb214103d10 con 0x7fb214102a90 2024-09-21T05:55:29.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.405+0000 7fb20affd640 1 -- 172.21.15.117:0/1188553560 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7fb210042030 con 0x7fb214102a90 2024-09-21T05:55:29.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.406+0000 7fb20affd640 1 --2- 172.21.15.117:0/1188553560 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb2000b6620 0x7fb2000b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:29.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.406+0000 7fb2193b9640 1 --2- 172.21.15.117:0/1188553560 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb2000b6620 0x7fb2000b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:29.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.406+0000 7fb20affd640 1 -- 172.21.15.117:0/1188553560 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 8194+0+0 (secure 0 0 0) 0x7fb210107ca0 con 0x7fb214102a90 2024-09-21T05:55:29.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.406+0000 7fb2193b9640 1 --2- 172.21.15.117:0/1188553560 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb2000b6620 0x7fb2000b8b30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb214079cf0 tx=0x7fb20400a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:29.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.409+0000 7fb20affd640 1 -- 172.21.15.117:0/1188553560 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb21419f990 con 0x7fb214102a90 2024-09-21T05:55:29.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.614+0000 7fb21be1e640 1 -- 172.21.15.117:0/1188553560 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fb21410ab20 con 0x7fb2000b6620 2024-09-21T05:55:29.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.645+0000 7fb20affd640 1 -- 172.21.15.117:0/1188553560 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb21410ab20 con 0x7fb2000b6620 2024-09-21T05:55:29.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.647+0000 7fb21be1e640 1 -- 172.21.15.117:0/1188553560 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb2000b6620 msgr2=0x7fb2000b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:29.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.647+0000 7fb21be1e640 1 --2- 172.21.15.117:0/1188553560 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb2000b6620 0x7fb2000b8b30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb214079cf0 tx=0x7fb20400a2d0 comp rx=0 tx=0).stop 2024-09-21T05:55:29.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.647+0000 7fb21be1e640 1 -- 172.21.15.117:0/1188553560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb214102a90 msgr2=0x7fb214199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:29.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.647+0000 7fb21be1e640 1 --2- 172.21.15.117:0/1188553560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb214102a90 0x7fb214199320 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb210005940 tx=0x7fb2100042f0 comp rx=0 tx=0).stop 2024-09-21T05:55:29.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.647+0000 7fb21be1e640 1 -- 172.21.15.117:0/1188553560 shutdown_connections 2024-09-21T05:55:29.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.647+0000 7fb21be1e640 1 --2- 172.21.15.117:0/1188553560 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb2000b6620 0x7fb2000b8b30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:29.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.647+0000 7fb21be1e640 1 --2- 172.21.15.117:0/1188553560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb214108d20 0x7fb21419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:29.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.647+0000 7fb21be1e640 1 --2- 172.21.15.117:0/1188553560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2141083d0 0x7fb2141998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:29.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.647+0000 7fb21be1e640 1 --2- 172.21.15.117:0/1188553560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb214102a90 0x7fb214199320 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:29.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.647+0000 7fb21be1e640 1 -- 172.21.15.117:0/1188553560 >> 172.21.15.117:0/1188553560 conn(0x7fb214079750 msgr2=0x7fb2141013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:29.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.647+0000 7fb21be1e640 1 -- 172.21.15.117:0/1188553560 shutdown_connections 2024-09-21T05:55:29.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:29.647+0000 7fb21be1e640 1 -- 172.21.15.117:0/1188553560 wait complete. 2024-09-21T05:55:29.865 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 21 05:55:29 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-b[63794]: 2024-09-21T05:55:29.593+0000 7f5099011640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-21T05:55:29.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:29 smithi138 ceph-mon[34391]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:55:29.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:29 smithi138 ceph-mon[34391]: pgmap v270: 97 pgs: 97 active+clean; 584 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 16 op/s 2024-09-21T05:55:29.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:29 smithi138 ceph-mon[34391]: mgrmap e21: x(active, since 6m), standbys: z, y 2024-09-21T05:55:29.954 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:29 smithi117 ceph-mon[30947]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:55:29.954 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:29 smithi117 ceph-mon[30947]: pgmap v270: 97 pgs: 97 active+clean; 584 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 16 op/s 2024-09-21T05:55:29.955 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:29 smithi117 ceph-mon[30947]: mgrmap e21: x(active, since 6m), standbys: z, y 2024-09-21T05:55:29.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:29 smithi192 ceph-mon[34700]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:55:29.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:29 smithi192 ceph-mon[34700]: pgmap v270: 97 pgs: 97 active+clean; 584 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 16 op/s 2024-09-21T05:55:29.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:29 smithi192 ceph-mon[34700]: mgrmap e21: x(active, since 6m), standbys: z, y 2024-09-21T05:55:30.391 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-21T05:55:30.640 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:30.912 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:30 smithi117 ceph-mon[30947]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:55:30.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:30 smithi192 ceph-mon[34700]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:55:31.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:30 smithi138 ceph-mon[34391]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:55:31.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.145+0000 7f3df334d640 1 -- 172.21.15.117:0/4116905120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3dec106c30 msgr2=0x7f3dec110680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:31.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.145+0000 7f3df334d640 1 --2- 172.21.15.117:0/4116905120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3dec106c30 0x7f3dec110680 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3de4009900 tx=0x7f3de402a410 comp rx=0 tx=0).stop 2024-09-21T05:55:31.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.145+0000 7f3df334d640 1 -- 172.21.15.117:0/4116905120 shutdown_connections 2024-09-21T05:55:31.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.145+0000 7f3df334d640 1 --2- 172.21.15.117:0/4116905120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3dec106c30 0x7f3dec110680 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:31.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.145+0000 7f3df334d640 1 --2- 172.21.15.117:0/4116905120 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3dec1062e0 0x7f3dec1066f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:31.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.145+0000 7f3df334d640 1 --2- 172.21.15.117:0/4116905120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3dec102000 0x7f3dec102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:31.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.145+0000 7f3df334d640 1 -- 172.21.15.117:0/4116905120 >> 172.21.15.117:0/4116905120 conn(0x7f3dec079e40 msgr2=0x7f3dec07a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:31.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.145+0000 7f3df334d640 1 -- 172.21.15.117:0/4116905120 shutdown_connections 2024-09-21T05:55:31.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.145+0000 7f3df334d640 1 -- 172.21.15.117:0/4116905120 wait complete. 2024-09-21T05:55:31.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.146+0000 7f3df334d640 1 Processor -- start 2024-09-21T05:55:31.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.146+0000 7f3df334d640 1 -- start start 2024-09-21T05:55:31.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.147+0000 7f3df334d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3dec102000 0x7f3dec19c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:31.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.147+0000 7f3df334d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3dec1062e0 0x7f3dec19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:31.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.147+0000 7f3df334d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3dec106c30 0x7f3dec1a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:31.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.147+0000 7f3df334d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3dec113b30 con 0x7f3dec106c30 2024-09-21T05:55:31.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.147+0000 7f3df334d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3dec1139b0 con 0x7f3dec102000 2024-09-21T05:55:31.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.147+0000 7f3df334d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3dec113cb0 con 0x7f3dec1062e0 2024-09-21T05:55:31.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.147+0000 7f3df18ea640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3dec106c30 0x7f3dec1a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:31.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.147+0000 7f3df18ea640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3dec106c30 0x7f3dec1a1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49916/0 (socket says 172.21.15.117:49916) 2024-09-21T05:55:31.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.147+0000 7f3df18ea640 1 -- 172.21.15.117:0/1872849056 learned_addr learned my addr 172.21.15.117:0/1872849056 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:55:31.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.147+0000 7f3df10e9640 1 --2- 172.21.15.117:0/1872849056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3dec102000 0x7f3dec19c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:31.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.147+0000 7f3df08e8640 1 --2- 172.21.15.117:0/1872849056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3dec1062e0 0x7f3dec19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:31.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.147+0000 7f3df18ea640 1 -- 172.21.15.117:0/1872849056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3dec1062e0 msgr2=0x7f3dec19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:31.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.147+0000 7f3df18ea640 1 --2- 172.21.15.117:0/1872849056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3dec1062e0 0x7f3dec19cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:31.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.147+0000 7f3df18ea640 1 -- 172.21.15.117:0/1872849056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3dec102000 msgr2=0x7f3dec19c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:31.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.147+0000 7f3df18ea640 1 --2- 172.21.15.117:0/1872849056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3dec102000 0x7f3dec19c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:31.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.147+0000 7f3df18ea640 1 -- 172.21.15.117:0/1872849056 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dec1a2550 con 0x7f3dec106c30 2024-09-21T05:55:31.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.147+0000 7f3df18ea640 1 --2- 172.21.15.117:0/1872849056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3dec106c30 0x7f3dec1a1ee0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3de4009a20 tx=0x7f3de4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:31.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.147+0000 7f3dea7fc640 1 -- 172.21.15.117:0/1872849056 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de403f030 con 0x7f3dec106c30 2024-09-21T05:55:31.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.147+0000 7f3dea7fc640 1 -- 172.21.15.117:0/1872849056 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3de4003aa0 con 0x7f3dec106c30 2024-09-21T05:55:31.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.147+0000 7f3dea7fc640 1 -- 172.21.15.117:0/1872849056 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de4035070 con 0x7f3dec106c30 2024-09-21T05:55:31.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.147+0000 7f3df334d640 1 -- 172.21.15.117:0/1872849056 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dec1a27e0 con 0x7f3dec106c30 2024-09-21T05:55:31.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.147+0000 7f3df334d640 1 -- 172.21.15.117:0/1872849056 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dec1a9060 con 0x7f3dec106c30 2024-09-21T05:55:31.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.149+0000 7f3df334d640 1 -- 172.21.15.117:0/1872849056 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dec102410 con 0x7f3dec106c30 2024-09-21T05:55:31.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.150+0000 7f3dea7fc640 1 -- 172.21.15.117:0/1872849056 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f3de4042030 con 0x7f3dec106c30 2024-09-21T05:55:31.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.150+0000 7f3dea7fc640 1 --2- 172.21.15.117:0/1872849056 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3dd40b66a0 0x7f3dd40b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:31.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.151+0000 7f3df10e9640 1 --2- 172.21.15.117:0/1872849056 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3dd40b66a0 0x7f3dd40b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:31.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.151+0000 7f3dea7fc640 1 -- 172.21.15.117:0/1872849056 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 8194+0+0 (secure 0 0 0) 0x7f3de4107d20 con 0x7f3dec106c30 2024-09-21T05:55:31.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.151+0000 7f3df10e9640 1 --2- 172.21.15.117:0/1872849056 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3dd40b66a0 0x7f3dd40b8bb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc007ec0 tx=0x7f3ddc0075b0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:31.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.155+0000 7f3dea7fc640 1 -- 172.21.15.117:0/1872849056 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3de40ce410 con 0x7f3dec106c30 2024-09-21T05:55:31.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.361+0000 7f3df334d640 1 -- 172.21.15.117:0/1872849056 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f3dec1087c0 con 0x7f3dd40b66a0 2024-09-21T05:55:31.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.366+0000 7f3dea7fc640 1 -- 172.21.15.117:0/1872849056 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f3dec1087c0 con 0x7f3dd40b66a0 2024-09-21T05:55:31.368 INFO:teuthology.orchestra.run.smithi117.stdout:/volumes/qa/sv_0/3db33fef-7cd8-4287-8f46-601ef6b84a5d 2024-09-21T05:55:31.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.368+0000 7f3df334d640 1 -- 172.21.15.117:0/1872849056 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3dd40b66a0 msgr2=0x7f3dd40b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:31.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.368+0000 7f3df334d640 1 --2- 172.21.15.117:0/1872849056 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3dd40b66a0 0x7f3dd40b8bb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc007ec0 tx=0x7f3ddc0075b0 comp rx=0 tx=0).stop 2024-09-21T05:55:31.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.368+0000 7f3df334d640 1 -- 172.21.15.117:0/1872849056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3dec106c30 msgr2=0x7f3dec1a1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:31.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.368+0000 7f3df334d640 1 --2- 172.21.15.117:0/1872849056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3dec106c30 0x7f3dec1a1ee0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3de4009a20 tx=0x7f3de4004270 comp rx=0 tx=0).stop 2024-09-21T05:55:31.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.368+0000 7f3df334d640 1 -- 172.21.15.117:0/1872849056 shutdown_connections 2024-09-21T05:55:31.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.368+0000 7f3df334d640 1 --2- 172.21.15.117:0/1872849056 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3dd40b66a0 0x7f3dd40b8bb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:31.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.368+0000 7f3df334d640 1 --2- 172.21.15.117:0/1872849056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3dec106c30 0x7f3dec1a1ee0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:31.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.368+0000 7f3df334d640 1 --2- 172.21.15.117:0/1872849056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3dec1062e0 0x7f3dec19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:31.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.368+0000 7f3df334d640 1 --2- 172.21.15.117:0/1872849056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3dec102000 0x7f3dec19c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:31.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.368+0000 7f3df334d640 1 -- 172.21.15.117:0/1872849056 >> 172.21.15.117:0/1872849056 conn(0x7f3dec079e40 msgr2=0x7f3dec1080b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:31.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.368+0000 7f3df334d640 1 -- 172.21.15.117:0/1872849056 shutdown_connections 2024-09-21T05:55:31.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:31.368+0000 7f3df334d640 1 -- 172.21.15.117:0/1872849056 wait complete. 2024-09-21T05:55:31.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:31 smithi192 ceph-mon[34700]: pgmap v271: 97 pgs: 97 active+clean; 585 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4.3 KiB/s wr, 17 op/s 2024-09-21T05:55:31.998 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:31 smithi117 ceph-mon[30947]: pgmap v271: 97 pgs: 97 active+clean; 585 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4.3 KiB/s wr, 17 op/s 2024-09-21T05:55:32.026 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-21T05:55:32.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:31 smithi138 ceph-mon[34391]: pgmap v271: 97 pgs: 97 active+clean; 585 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4.3 KiB/s wr, 17 op/s 2024-09-21T05:55:32.276 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:32.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.752+0000 7f00f0f47640 1 -- 172.21.15.117:0/3537250565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00ec103270 msgr2=0x7f00ec103680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:32.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.752+0000 7f00f0f47640 1 --2- 172.21.15.117:0/3537250565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00ec103270 0x7f00ec103680 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f00dc009900 tx=0x7f00dc02a410 comp rx=0 tx=0).stop 2024-09-21T05:55:32.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.752+0000 7f00f0f47640 1 -- 172.21.15.117:0/3537250565 shutdown_connections 2024-09-21T05:55:32.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.752+0000 7f00f0f47640 1 --2- 172.21.15.117:0/3537250565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00ec105b90 0x7f00ec10f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:32.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.752+0000 7f00f0f47640 1 --2- 172.21.15.117:0/3537250565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00ec105240 0x7f00ec105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:32.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.752+0000 7f00f0f47640 1 --2- 172.21.15.117:0/3537250565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00ec103270 0x7f00ec103680 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:32.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.753+0000 7f00f0f47640 1 -- 172.21.15.117:0/3537250565 >> 172.21.15.117:0/3537250565 conn(0x7f00ec079ec0 msgr2=0x7f00ec07a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:32.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.753+0000 7f00f0f47640 1 -- 172.21.15.117:0/3537250565 shutdown_connections 2024-09-21T05:55:32.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.753+0000 7f00f0f47640 1 -- 172.21.15.117:0/3537250565 wait complete. 2024-09-21T05:55:32.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.753+0000 7f00f0f47640 1 Processor -- start 2024-09-21T05:55:32.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.754+0000 7f00f0f47640 1 -- start start 2024-09-21T05:55:32.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.754+0000 7f00f0f47640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00ec103270 0x7f00ec1100c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:32.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.754+0000 7f00f0f47640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00ec105240 0x7f00ec110600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:32.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.754+0000 7f00f0f47640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00ec105b90 0x7f00ec110b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:32.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.754+0000 7f00f0f47640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec113c10 con 0x7f00ec105b90 2024-09-21T05:55:32.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.754+0000 7f00f0f47640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec113a90 con 0x7f00ec105240 2024-09-21T05:55:32.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.754+0000 7f00f0f47640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec113d90 con 0x7f00ec103270 2024-09-21T05:55:32.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.754+0000 7f00ead9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00ec105b90 0x7f00ec110b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:32.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.754+0000 7f00ea59c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00ec103270 0x7f00ec1100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:32.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.754+0000 7f00ead9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00ec105b90 0x7f00ec110b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49934/0 (socket says 172.21.15.117:49934) 2024-09-21T05:55:32.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.754+0000 7f00ead9d640 1 -- 172.21.15.117:0/1110911533 learned_addr learned my addr 172.21.15.117:0/1110911533 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:55:32.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.754+0000 7f00e1d9b640 1 --2- 172.21.15.117:0/1110911533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00ec105240 0x7f00ec110600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:32.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.755+0000 7f00ead9d640 1 -- 172.21.15.117:0/1110911533 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00ec103270 msgr2=0x7f00ec1100c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:32.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.755+0000 7f00ead9d640 1 --2- 172.21.15.117:0/1110911533 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00ec103270 0x7f00ec1100c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:32.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.755+0000 7f00ead9d640 1 -- 172.21.15.117:0/1110911533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00ec105240 msgr2=0x7f00ec110600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:32.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.755+0000 7f00ead9d640 1 --2- 172.21.15.117:0/1110911533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00ec105240 0x7f00ec110600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:32.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.755+0000 7f00ead9d640 1 -- 172.21.15.117:0/1110911533 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00ec1a88c0 con 0x7f00ec105b90 2024-09-21T05:55:32.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.755+0000 7f00e1d9b640 1 --2- 172.21.15.117:0/1110911533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00ec105240 0x7f00ec110600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:55:32.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.755+0000 7f00ead9d640 1 --2- 172.21.15.117:0/1110911533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00ec105b90 0x7f00ec110b40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f00e400b970 tx=0x7f00e400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:32.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.755+0000 7f00e3fff640 1 -- 172.21.15.117:0/1110911533 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e4019070 con 0x7f00ec105b90 2024-09-21T05:55:32.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.755+0000 7f00e3fff640 1 -- 172.21.15.117:0/1110911533 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00e4014440 con 0x7f00ec105b90 2024-09-21T05:55:32.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.755+0000 7f00f0f47640 1 -- 172.21.15.117:0/1110911533 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00ec1a8bb0 con 0x7f00ec105b90 2024-09-21T05:55:32.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.755+0000 7f00e3fff640 1 -- 172.21.15.117:0/1110911533 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e40075d0 con 0x7f00ec105b90 2024-09-21T05:55:32.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.755+0000 7f00f0f47640 1 -- 172.21.15.117:0/1110911533 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00ec1a90a0 con 0x7f00ec105b90 2024-09-21T05:55:32.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.756+0000 7f00f0f47640 1 -- 172.21.15.117:0/1110911533 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00ec1113c0 con 0x7f00ec105b90 2024-09-21T05:55:32.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.758+0000 7f00e3fff640 1 -- 172.21.15.117:0/1110911533 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f00e401c070 con 0x7f00ec105b90 2024-09-21T05:55:32.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.758+0000 7f00e3fff640 1 --2- 172.21.15.117:0/1110911533 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f00d80b65f0 0x7f00d80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:32.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.759+0000 7f00ea59c640 1 --2- 172.21.15.117:0/1110911533 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f00d80b65f0 0x7f00d80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:32.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.759+0000 7f00e3fff640 1 -- 172.21.15.117:0/1110911533 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8397+0+0 (secure 0 0 0) 0x7f00e40e6ed0 con 0x7f00ec105b90 2024-09-21T05:55:32.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.759+0000 7f00ea59c640 1 --2- 172.21.15.117:0/1110911533 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f00d80b65f0 0x7f00d80b8b00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f00dc000c00 tx=0x7f00dc031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:32.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.762+0000 7f00e3fff640 1 -- 172.21.15.117:0/1110911533 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f00e40b1940 con 0x7f00ec105b90 2024-09-21T05:55:32.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.965+0000 7f00f0f47640 1 -- 172.21.15.117:0/1110911533 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f00ec063800 con 0x7f00d80b65f0 2024-09-21T05:55:32.967 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:32 smithi117 ceph-mon[30947]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:55:32.967 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:32 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]} : dispatch 2024-09-21T05:55:32.967 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:32 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]} : dispatch 2024-09-21T05:55:32.967 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:32 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]} : dispatch 2024-09-21T05:55:32.968 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:32 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]} : dispatch 2024-09-21T05:55:32.968 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:32 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]} : dispatch 2024-09-21T05:55:32.968 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:32 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]} : dispatch 2024-09-21T05:55:32.968 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:32 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]} : dispatch 2024-09-21T05:55:32.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:32 smithi192 ceph-mon[34700]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:55:32.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:32 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]} : dispatch 2024-09-21T05:55:32.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:32 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]} : dispatch 2024-09-21T05:55:32.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:32 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]} : dispatch 2024-09-21T05:55:32.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:32 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]} : dispatch 2024-09-21T05:55:32.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:32 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]} : dispatch 2024-09-21T05:55:32.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:32 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]} : dispatch 2024-09-21T05:55:32.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:32 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]} : dispatch 2024-09-21T05:55:32.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.994+0000 7f00e3fff640 1 -- 172.21.15.117:0/1110911533 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f00ec063800 con 0x7f00d80b65f0 2024-09-21T05:55:32.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.996+0000 7f00f0f47640 1 -- 172.21.15.117:0/1110911533 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f00d80b65f0 msgr2=0x7f00d80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:32.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.996+0000 7f00f0f47640 1 --2- 172.21.15.117:0/1110911533 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f00d80b65f0 0x7f00d80b8b00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f00dc000c00 tx=0x7f00dc031400 comp rx=0 tx=0).stop 2024-09-21T05:55:32.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.996+0000 7f00f0f47640 1 -- 172.21.15.117:0/1110911533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00ec105b90 msgr2=0x7f00ec110b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:32.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.996+0000 7f00f0f47640 1 --2- 172.21.15.117:0/1110911533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00ec105b90 0x7f00ec110b40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f00e400b970 tx=0x7f00e400be30 comp rx=0 tx=0).stop 2024-09-21T05:55:32.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.996+0000 7f00f0f47640 1 -- 172.21.15.117:0/1110911533 shutdown_connections 2024-09-21T05:55:32.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.996+0000 7f00f0f47640 1 --2- 172.21.15.117:0/1110911533 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f00d80b65f0 0x7f00d80b8b00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:32.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.996+0000 7f00f0f47640 1 --2- 172.21.15.117:0/1110911533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00ec105b90 0x7f00ec110b40 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:32.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.996+0000 7f00f0f47640 1 --2- 172.21.15.117:0/1110911533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00ec105240 0x7f00ec110600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:32.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.996+0000 7f00f0f47640 1 --2- 172.21.15.117:0/1110911533 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00ec103270 0x7f00ec1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:32.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.996+0000 7f00f0f47640 1 -- 172.21.15.117:0/1110911533 >> 172.21.15.117:0/1110911533 conn(0x7f00ec079ec0 msgr2=0x7f00ec1073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:32.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.996+0000 7f00f0f47640 1 -- 172.21.15.117:0/1110911533 shutdown_connections 2024-09-21T05:55:32.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:32.996+0000 7f00f0f47640 1 -- 172.21.15.117:0/1110911533 wait complete. 2024-09-21T05:55:33.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:32 smithi138 ceph-mon[34391]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:55:33.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:32 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]} : dispatch 2024-09-21T05:55:33.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:32 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]} : dispatch 2024-09-21T05:55:33.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:32 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]} : dispatch 2024-09-21T05:55:33.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:32 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]} : dispatch 2024-09-21T05:55:33.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:32 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]} : dispatch 2024-09-21T05:55:33.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:32 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]} : dispatch 2024-09-21T05:55:33.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:32 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]} : dispatch 2024-09-21T05:55:33.778 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-21T05:55:33.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:33 smithi192 ceph-mon[34700]: pgmap v272: 97 pgs: 97 active+clean; 585 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.3 KiB/s wr, 19 op/s 2024-09-21T05:55:33.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:33 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-21T05:55:33.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:33 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-09-21T05:55:33.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:33 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-21T05:55:33.981 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:33 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-09-21T05:55:33.981 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:33 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-21T05:55:33.981 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:33 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-21T05:55:33.981 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:33 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-09-21T05:55:33.981 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:33 smithi192 ceph-mon[34700]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T05:55:34.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:33 smithi117 ceph-mon[30947]: pgmap v272: 97 pgs: 97 active+clean; 585 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.3 KiB/s wr, 19 op/s 2024-09-21T05:55:34.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:33 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-21T05:55:34.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:33 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-09-21T05:55:34.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:33 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-21T05:55:34.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:33 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-09-21T05:55:34.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:33 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-21T05:55:34.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:33 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-21T05:55:34.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:33 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-09-21T05:55:34.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:33 smithi117 ceph-mon[30947]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T05:55:34.028 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:34.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:33 smithi138 ceph-mon[34391]: pgmap v272: 97 pgs: 97 active+clean; 585 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 4.3 KiB/s wr, 19 op/s 2024-09-21T05:55:34.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:33 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-21T05:55:34.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:33 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-09-21T05:55:34.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:33 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-21T05:55:34.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:33 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-09-21T05:55:34.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:33 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-21T05:55:34.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:33 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-21T05:55:34.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:33 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-09-21T05:55:34.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:33 smithi138 ceph-mon[34391]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T05:55:34.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.528+0000 7f806bd20640 1 -- 172.21.15.117:0/263567057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8064105380 msgr2=0x7f806410d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:34.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.528+0000 7f806bd20640 1 --2- 172.21.15.117:0/263567057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8064105380 0x7f806410d8d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8054009900 tx=0x7f805402a410 comp rx=0 tx=0).stop 2024-09-21T05:55:34.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.528+0000 7f806bd20640 1 -- 172.21.15.117:0/263567057 shutdown_connections 2024-09-21T05:55:34.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.529+0000 7f806bd20640 1 --2- 172.21.15.117:0/263567057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8064105380 0x7f806410d8d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:34.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.529+0000 7f806bd20640 1 --2- 172.21.15.117:0/263567057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f806410de60 0x7f8064110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:34.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.529+0000 7f806bd20640 1 --2- 172.21.15.117:0/263567057 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8064104a30 0x7f8064104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:34.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.529+0000 7f806bd20640 1 -- 172.21.15.117:0/263567057 >> 172.21.15.117:0/263567057 conn(0x7f8064079760 msgr2=0x7f8064079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:34.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.529+0000 7f806bd20640 1 -- 172.21.15.117:0/263567057 shutdown_connections 2024-09-21T05:55:34.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.529+0000 7f806bd20640 1 -- 172.21.15.117:0/263567057 wait complete. 2024-09-21T05:55:34.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.529+0000 7f806bd20640 1 Processor -- start 2024-09-21T05:55:34.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.529+0000 7f806bd20640 1 -- start start 2024-09-21T05:55:34.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.530+0000 7f806bd20640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8064104a30 0x7f806419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:34.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.530+0000 7f806bd20640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8064105380 0x7f806419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:34.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.530+0000 7f806bd20640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f806410de60 0x7f806419fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:34.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.530+0000 7f806bd20640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8064113b00 con 0x7f806410de60 2024-09-21T05:55:34.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.530+0000 7f806bd20640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8064113980 con 0x7f8064105380 2024-09-21T05:55:34.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.530+0000 7f806bd20640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8064113c80 con 0x7f8064104a30 2024-09-21T05:55:34.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.530+0000 7f806a2bd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f806410de60 0x7f806419fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:34.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.530+0000 7f806a2bd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f806410de60 0x7f806419fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37340/0 (socket says 172.21.15.117:37340) 2024-09-21T05:55:34.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.530+0000 7f806a2bd640 1 -- 172.21.15.117:0/2224450761 learned_addr learned my addr 172.21.15.117:0/2224450761 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:55:34.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.530+0000 7f80692bb640 1 --2- 172.21.15.117:0/2224450761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8064105380 0x7f806419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:34.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.530+0000 7f8069abc640 1 --2- 172.21.15.117:0/2224450761 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8064104a30 0x7f806419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:34.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.530+0000 7f806a2bd640 1 -- 172.21.15.117:0/2224450761 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8064104a30 msgr2=0x7f806419a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:34.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.530+0000 7f806a2bd640 1 --2- 172.21.15.117:0/2224450761 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8064104a30 0x7f806419a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:34.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.530+0000 7f806a2bd640 1 -- 172.21.15.117:0/2224450761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8064105380 msgr2=0x7f806419ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:34.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.530+0000 7f806a2bd640 1 --2- 172.21.15.117:0/2224450761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8064105380 0x7f806419ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:34.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.530+0000 7f806a2bd640 1 -- 172.21.15.117:0/2224450761 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80641a0540 con 0x7f806410de60 2024-09-21T05:55:34.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.530+0000 7f80692bb640 1 --2- 172.21.15.117:0/2224450761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8064105380 0x7f806419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:55:34.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.530+0000 7f806a2bd640 1 --2- 172.21.15.117:0/2224450761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f806410de60 0x7f806419fe10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f805c00b970 tx=0x7f805c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:34.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.530+0000 7f805affd640 1 -- 172.21.15.117:0/2224450761 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c019070 con 0x7f806410de60 2024-09-21T05:55:34.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.530+0000 7f805affd640 1 -- 172.21.15.117:0/2224450761 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f805c014440 con 0x7f806410de60 2024-09-21T05:55:34.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.530+0000 7f806bd20640 1 -- 172.21.15.117:0/2224450761 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80641a0830 con 0x7f806410de60 2024-09-21T05:55:34.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.531+0000 7f805affd640 1 -- 172.21.15.117:0/2224450761 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805c0075d0 con 0x7f806410de60 2024-09-21T05:55:34.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.531+0000 7f806bd20640 1 -- 172.21.15.117:0/2224450761 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80641a6f90 con 0x7f806410de60 2024-09-21T05:55:34.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.532+0000 7f806bd20640 1 -- 172.21.15.117:0/2224450761 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f806406c1f0 con 0x7f806410de60 2024-09-21T05:55:34.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.533+0000 7f805affd640 1 -- 172.21.15.117:0/2224450761 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f805c01c070 con 0x7f806410de60 2024-09-21T05:55:34.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.534+0000 7f805affd640 1 --2- 172.21.15.117:0/2224450761 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f80500b6670 0x7f80500b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:34.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.534+0000 7f8069abc640 1 --2- 172.21.15.117:0/2224450761 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f80500b6670 0x7f80500b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:34.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.534+0000 7f805affd640 1 -- 172.21.15.117:0/2224450761 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f805c0e6e80 con 0x7f806410de60 2024-09-21T05:55:34.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.534+0000 7f8069abc640 1 --2- 172.21.15.117:0/2224450761 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f80500b6670 0x7f80500b8b80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8060005fd0 tx=0x7f806000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:34.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.537+0000 7f805affd640 1 -- 172.21.15.117:0/2224450761 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f805c0b18f0 con 0x7f806410de60 2024-09-21T05:55:34.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.743+0000 7f806bd20640 1 -- 172.21.15.117:0/2224450761 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f8064063800 con 0x7f80500b6670 2024-09-21T05:55:34.745 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:34 smithi117 ceph-mon[30947]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:55:34.745 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:34 smithi117 ceph-mon[30947]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T05:55:34.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.748+0000 7f805affd640 1 -- 172.21.15.117:0/2224450761 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f8064063800 con 0x7f80500b6670 2024-09-21T05:55:34.751 INFO:teuthology.orchestra.run.smithi117.stdout:/volumes/qa/sv_1/16a073f7-1cc5-454e-a604-c30c5a2cdf02 2024-09-21T05:55:34.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.750+0000 7f806bd20640 1 -- 172.21.15.117:0/2224450761 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f80500b6670 msgr2=0x7f80500b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:34.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.750+0000 7f806bd20640 1 --2- 172.21.15.117:0/2224450761 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f80500b6670 0x7f80500b8b80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8060005fd0 tx=0x7f806000b390 comp rx=0 tx=0).stop 2024-09-21T05:55:34.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.750+0000 7f806bd20640 1 -- 172.21.15.117:0/2224450761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f806410de60 msgr2=0x7f806419fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:34.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.750+0000 7f806bd20640 1 --2- 172.21.15.117:0/2224450761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f806410de60 0x7f806419fe10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f805c00b970 tx=0x7f805c00be30 comp rx=0 tx=0).stop 2024-09-21T05:55:34.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.751+0000 7f806bd20640 1 -- 172.21.15.117:0/2224450761 shutdown_connections 2024-09-21T05:55:34.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.751+0000 7f806bd20640 1 --2- 172.21.15.117:0/2224450761 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f80500b6670 0x7f80500b8b80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:34.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.751+0000 7f806bd20640 1 --2- 172.21.15.117:0/2224450761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f806410de60 0x7f806419fe10 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:34.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.751+0000 7f806bd20640 1 --2- 172.21.15.117:0/2224450761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8064105380 0x7f806419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:34.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.751+0000 7f806bd20640 1 --2- 172.21.15.117:0/2224450761 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8064104a30 0x7f806419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:34.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.751+0000 7f806bd20640 1 -- 172.21.15.117:0/2224450761 >> 172.21.15.117:0/2224450761 conn(0x7f8064079760 msgr2=0x7f80641008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:34.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.751+0000 7f806bd20640 1 -- 172.21.15.117:0/2224450761 shutdown_connections 2024-09-21T05:55:34.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:34.751+0000 7f806bd20640 1 -- 172.21.15.117:0/2224450761 wait complete. 2024-09-21T05:55:34.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:34 smithi192 ceph-mon[34700]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:55:34.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:34 smithi192 ceph-mon[34700]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T05:55:35.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:34 smithi138 ceph-mon[34391]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:55:35.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:34 smithi138 ceph-mon[34391]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T05:55:35.424 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T05:55:35.673 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:35.928 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:35 smithi117 ceph-mon[30947]: pgmap v275: 97 pgs: 97 active+clean; 608 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.6 KiB/s wr, 21 op/s 2024-09-21T05:55:35.928 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:35 smithi117 ceph-mon[30947]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:55:35.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:35 smithi192 ceph-mon[34700]: pgmap v275: 97 pgs: 97 active+clean; 608 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.6 KiB/s wr, 21 op/s 2024-09-21T05:55:35.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:35 smithi192 ceph-mon[34700]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:55:36.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:35 smithi138 ceph-mon[34391]: pgmap v275: 97 pgs: 97 active+clean; 608 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.6 KiB/s wr, 21 op/s 2024-09-21T05:55:36.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:35 smithi138 ceph-mon[34391]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:55:36.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.148+0000 7fa14a1e6640 1 -- 172.21.15.117:0/4128577835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa144109df0 msgr2=0x7fa144110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:36.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.148+0000 7fa14a1e6640 1 --2- 172.21.15.117:0/4128577835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa144109df0 0x7fa144110620 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa13c009ca0 tx=0x7fa13c02a8c0 comp rx=0 tx=0).stop 2024-09-21T05:55:36.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.148+0000 7fa14a1e6640 1 -- 172.21.15.117:0/4128577835 shutdown_connections 2024-09-21T05:55:36.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.148+0000 7fa14a1e6640 1 --2- 172.21.15.117:0/4128577835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa144109df0 0x7fa144110620 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:36.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.148+0000 7fa14a1e6640 1 --2- 172.21.15.117:0/4128577835 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1441094a0 0x7fa1441098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:36.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.148+0000 7fa14a1e6640 1 --2- 172.21.15.117:0/4128577835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa144103b60 0x7fa144103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:36.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.148+0000 7fa14a1e6640 1 -- 172.21.15.117:0/4128577835 >> 172.21.15.117:0/4128577835 conn(0x7fa14407b850 msgr2=0x7fa14407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:36.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.148+0000 7fa14a1e6640 1 -- 172.21.15.117:0/4128577835 shutdown_connections 2024-09-21T05:55:36.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.148+0000 7fa14a1e6640 1 -- 172.21.15.117:0/4128577835 wait complete. 2024-09-21T05:55:36.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.149+0000 7fa14a1e6640 1 Processor -- start 2024-09-21T05:55:36.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.149+0000 7fa14a1e6640 1 -- start start 2024-09-21T05:55:36.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.150+0000 7fa14a1e6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa144103b60 0x7fa14419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:36.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.150+0000 7fa14a1e6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1441094a0 0x7fa14419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:36.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.150+0000 7fa14a1e6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa144109df0 0x7fa14419fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:36.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.150+0000 7fa14a1e6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa144113ab0 con 0x7fa144103b60 2024-09-21T05:55:36.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.150+0000 7fa14a1e6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa144113930 con 0x7fa144109df0 2024-09-21T05:55:36.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.150+0000 7fa14a1e6640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa144113c30 con 0x7fa1441094a0 2024-09-21T05:55:36.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.150+0000 7fa1437fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa144103b60 0x7fa14419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:36.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.150+0000 7fa1437fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa144103b60 0x7fa14419a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37366/0 (socket says 172.21.15.117:37366) 2024-09-21T05:55:36.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.150+0000 7fa1437fe640 1 -- 172.21.15.117:0/3441462906 learned_addr learned my addr 172.21.15.117:0/3441462906 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:55:36.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.150+0000 7fa142ffd640 1 --2- 172.21.15.117:0/3441462906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1441094a0 0x7fa14419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:36.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.150+0000 7fa143fff640 1 --2- 172.21.15.117:0/3441462906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa144109df0 0x7fa14419fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:36.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.150+0000 7fa1437fe640 1 -- 172.21.15.117:0/3441462906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1441094a0 msgr2=0x7fa14419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:36.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.150+0000 7fa1437fe640 1 --2- 172.21.15.117:0/3441462906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1441094a0 0x7fa14419a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:36.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.150+0000 7fa1437fe640 1 -- 172.21.15.117:0/3441462906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa144109df0 msgr2=0x7fa14419fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:36.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.150+0000 7fa1437fe640 1 --2- 172.21.15.117:0/3441462906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa144109df0 0x7fa14419fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:36.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.150+0000 7fa1437fe640 1 -- 172.21.15.117:0/3441462906 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1441a02f0 con 0x7fa144103b60 2024-09-21T05:55:36.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.150+0000 7fa143fff640 1 --2- 172.21.15.117:0/3441462906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa144109df0 0x7fa14419fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:55:36.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.150+0000 7fa1437fe640 1 --2- 172.21.15.117:0/3441462906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa144103b60 0x7fa14419a440 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa13400c890 tx=0x7fa13400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:36.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.151+0000 7fa140ff9640 1 -- 172.21.15.117:0/3441462906 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa134004220 con 0x7fa144103b60 2024-09-21T05:55:36.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.151+0000 7fa14a1e6640 1 -- 172.21.15.117:0/3441462906 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1441a05e0 con 0x7fa144103b60 2024-09-21T05:55:36.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.151+0000 7fa140ff9640 1 -- 172.21.15.117:0/3441462906 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1340043c0 con 0x7fa144103b60 2024-09-21T05:55:36.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.151+0000 7fa140ff9640 1 -- 172.21.15.117:0/3441462906 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa134004a00 con 0x7fa144103b60 2024-09-21T05:55:36.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.151+0000 7fa14a1e6640 1 -- 172.21.15.117:0/3441462906 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1441a6e00 con 0x7fa144103b60 2024-09-21T05:55:36.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.152+0000 7fa14a1e6640 1 -- 172.21.15.117:0/3441462906 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa144103f70 con 0x7fa144103b60 2024-09-21T05:55:36.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.153+0000 7fa140ff9640 1 -- 172.21.15.117:0/3441462906 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7fa1340079b0 con 0x7fa144103b60 2024-09-21T05:55:36.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.154+0000 7fa140ff9640 1 --2- 172.21.15.117:0/3441462906 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa1300b66a0 0x7fa1300b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:36.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.154+0000 7fa142ffd640 1 --2- 172.21.15.117:0/3441462906 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa1300b66a0 0x7fa1300b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:36.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.154+0000 7fa140ff9640 1 -- 172.21.15.117:0/3441462906 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fa1340e76d0 con 0x7fa144103b60 2024-09-21T05:55:36.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.155+0000 7fa142ffd640 1 --2- 172.21.15.117:0/3441462906 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa1300b66a0 0x7fa1300b8bb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa144106fc0 tx=0x7fa12c00a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:36.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.157+0000 7fa140ff9640 1 -- 172.21.15.117:0/3441462906 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa134014030 con 0x7fa144103b60 2024-09-21T05:55:36.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.372+0000 7fa14a1e6640 1 -- 172.21.15.117:0/3441462906 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa14419b570 con 0x7fa144103b60 2024-09-21T05:55:36.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.373+0000 7fa140ff9640 1 -- 172.21.15.117:0/3441462906 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+12001 (secure 0 0 0) 0x7fa1340add00 con 0x7fa144103b60 2024-09-21T05:55:36.375 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:55:36.375 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":26,"btime":"2024-09-21T05:55:26:209435+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3227057504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3227057504},{"type":"v1","addr":"172.21.15.117:6835","nonce":3227057504}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3303829183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3303829183},{"type":"v1","addr":"172.21.15.117:6837","nonce":3303829183}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T05:55:14.555897+0000","modified":"2024-09-21T05:55:26.209411+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24638,"mds_1":24637,"mds_2":24628,"mds_3":24616,"mds_4":24602},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6839/2314572411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2314572411},{"type":"v1","addr":"172.21.15.117:6839","nonce":2314572411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6841/3378096561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":3378096561},{"type":"v1","addr":"172.21.15.117:6841","nonce":3378096561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/3725629862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3725629862},{"type":"v1","addr":"172.21.15.138:6833","nonce":3725629862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/4276801396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4276801396},{"type":"v1","addr":"172.21.15.138:6835","nonce":4276801396}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6833/2224910879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2224910879},{"type":"v1","addr":"172.21.15.192:6833","nonce":2224910879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.138:6837/2540335374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2540335374},{"type":"v1","addr":"172.21.15.138:6837","nonce":2540335374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24616":{"gid":24616,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.192:6835/3530688860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3530688860},{"type":"v1","addr":"172.21.15.192:6835","nonce":3530688860}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.192:6837/2683860842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2683860842},{"type":"v1","addr":"172.21.15.192:6837","nonce":2683860842}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.192:6839/1727623231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1727623231},{"type":"v1","addr":"172.21.15.192:6839","nonce":1727623231}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/3230372576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":3230372576},{"type":"v1","addr":"172.21.15.138:6839","nonce":3230372576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24638,24637,24616,24628,24602]},"id":1}]} 2024-09-21T05:55:36.376 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 26 2024-09-21T05:55:36.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.375+0000 7fa14a1e6640 1 -- 172.21.15.117:0/3441462906 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa1300b66a0 msgr2=0x7fa1300b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:36.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.375+0000 7fa14a1e6640 1 --2- 172.21.15.117:0/3441462906 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa1300b66a0 0x7fa1300b8bb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa144106fc0 tx=0x7fa12c00a2d0 comp rx=0 tx=0).stop 2024-09-21T05:55:36.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.375+0000 7fa14a1e6640 1 -- 172.21.15.117:0/3441462906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa144103b60 msgr2=0x7fa14419a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:36.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.375+0000 7fa14a1e6640 1 --2- 172.21.15.117:0/3441462906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa144103b60 0x7fa14419a440 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa13400c890 tx=0x7fa13400cc90 comp rx=0 tx=0).stop 2024-09-21T05:55:36.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.375+0000 7fa14a1e6640 1 -- 172.21.15.117:0/3441462906 shutdown_connections 2024-09-21T05:55:36.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.375+0000 7fa14a1e6640 1 --2- 172.21.15.117:0/3441462906 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa1300b66a0 0x7fa1300b8bb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:36.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.375+0000 7fa14a1e6640 1 --2- 172.21.15.117:0/3441462906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa144109df0 0x7fa14419fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:36.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.375+0000 7fa14a1e6640 1 --2- 172.21.15.117:0/3441462906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1441094a0 0x7fa14419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:36.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.375+0000 7fa14a1e6640 1 --2- 172.21.15.117:0/3441462906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa144103b60 0x7fa14419a440 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:36.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.375+0000 7fa14a1e6640 1 -- 172.21.15.117:0/3441462906 >> 172.21.15.117:0/3441462906 conn(0x7fa14407b850 msgr2=0x7fa144105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:36.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.375+0000 7fa14a1e6640 1 -- 172.21.15.117:0/3441462906 shutdown_connections 2024-09-21T05:55:36.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:36.375+0000 7fa14a1e6640 1 -- 172.21.15.117:0/3441462906 wait complete. 2024-09-21T05:55:36.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:36 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3441462906' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:55:36.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:36 smithi192 ceph-mon[34700]: pgmap v276: 97 pgs: 97 active+clean; 608 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.1 KiB/s wr, 14 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T05:55:37.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:36 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3441462906' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:55:37.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:36 smithi117 ceph-mon[30947]: pgmap v276: 97 pgs: 97 active+clean; 608 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.1 KiB/s wr, 14 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T05:55:37.113 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T05:55:37.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:36 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3441462906' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:55:37.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:36 smithi138 ceph-mon[34391]: pgmap v276: 97 pgs: 97 active+clean; 608 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.1 KiB/s wr, 14 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T05:55:37.362 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:37.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.834+0000 7fd031f8a640 1 -- 172.21.15.117:0/2351351721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd02c102a60 msgr2=0x7fd02c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:37.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.834+0000 7fd031f8a640 1 --2- 172.21.15.117:0/2351351721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd02c102a60 0x7fd02c102e70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd01c009900 tx=0x7fd01c02a410 comp rx=0 tx=0).stop 2024-09-21T05:55:37.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.834+0000 7fd031f8a640 1 -- 172.21.15.117:0/2351351721 shutdown_connections 2024-09-21T05:55:37.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.834+0000 7fd031f8a640 1 --2- 172.21.15.117:0/2351351721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd02c108cf0 0x7fd02c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:37.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.834+0000 7fd031f8a640 1 --2- 172.21.15.117:0/2351351721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd02c1083a0 0x7fd02c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:37.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.834+0000 7fd031f8a640 1 --2- 172.21.15.117:0/2351351721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd02c102a60 0x7fd02c102e70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:37.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.834+0000 7fd031f8a640 1 -- 172.21.15.117:0/2351351721 >> 172.21.15.117:0/2351351721 conn(0x7fd02c079730 msgr2=0x7fd02c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:37.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.834+0000 7fd031f8a640 1 -- 172.21.15.117:0/2351351721 shutdown_connections 2024-09-21T05:55:37.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.834+0000 7fd031f8a640 1 -- 172.21.15.117:0/2351351721 wait complete. 2024-09-21T05:55:37.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.834+0000 7fd031f8a640 1 Processor -- start 2024-09-21T05:55:37.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.835+0000 7fd031f8a640 1 -- start start 2024-09-21T05:55:37.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.835+0000 7fd031f8a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd02c102a60 0x7fd02c19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:37.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.835+0000 7fd031f8a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd02c1083a0 0x7fd02c19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:37.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.835+0000 7fd031f8a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd02c108cf0 0x7fd02c19fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:37.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.835+0000 7fd031f8a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c113ae0 con 0x7fd02c102a60 2024-09-21T05:55:37.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.835+0000 7fd031f8a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c113960 con 0x7fd02c108cf0 2024-09-21T05:55:37.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.835+0000 7fd031f8a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd02c113c60 con 0x7fd02c1083a0 2024-09-21T05:55:37.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.835+0000 7fd02b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd02c102a60 0x7fd02c19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:37.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.835+0000 7fd02b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd02c102a60 0x7fd02c19a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37370/0 (socket says 172.21.15.117:37370) 2024-09-21T05:55:37.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.835+0000 7fd02b7fe640 1 -- 172.21.15.117:0/346982613 learned_addr learned my addr 172.21.15.117:0/346982613 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:55:37.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.835+0000 7fd02b7fe640 1 -- 172.21.15.117:0/346982613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd02c1083a0 msgr2=0x7fd02c19aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:37.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.835+0000 7fd02affd640 1 --2- 172.21.15.117:0/346982613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd02c1083a0 0x7fd02c19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:37.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.835+0000 7fd02b7fe640 1 --2- 172.21.15.117:0/346982613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd02c1083a0 0x7fd02c19aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:37.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.835+0000 7fd02bfff640 1 --2- 172.21.15.117:0/346982613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd02c108cf0 0x7fd02c19fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:37.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.835+0000 7fd02b7fe640 1 -- 172.21.15.117:0/346982613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd02c108cf0 msgr2=0x7fd02c19fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:37.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.835+0000 7fd02b7fe640 1 --2- 172.21.15.117:0/346982613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd02c108cf0 0x7fd02c19fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:37.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.835+0000 7fd02b7fe640 1 -- 172.21.15.117:0/346982613 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd02c1a0440 con 0x7fd02c102a60 2024-09-21T05:55:37.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.835+0000 7fd02affd640 1 --2- 172.21.15.117:0/346982613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd02c1083a0 0x7fd02c19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:55:37.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.835+0000 7fd02bfff640 1 --2- 172.21.15.117:0/346982613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd02c108cf0 0x7fd02c19fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:55:37.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.836+0000 7fd02b7fe640 1 --2- 172.21.15.117:0/346982613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd02c102a60 0x7fd02c19a4d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd01c006010 tx=0x7fd01c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:37.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.836+0000 7fd028ff9640 1 -- 172.21.15.117:0/346982613 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c03f030 con 0x7fd02c102a60 2024-09-21T05:55:37.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.836+0000 7fd031f8a640 1 -- 172.21.15.117:0/346982613 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd02c1a06d0 con 0x7fd02c102a60 2024-09-21T05:55:37.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.836+0000 7fd028ff9640 1 -- 172.21.15.117:0/346982613 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd01c003800 con 0x7fd02c102a60 2024-09-21T05:55:37.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.836+0000 7fd028ff9640 1 -- 172.21.15.117:0/346982613 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c035070 con 0x7fd02c102a60 2024-09-21T05:55:37.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.836+0000 7fd031f8a640 1 -- 172.21.15.117:0/346982613 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd02c1a6e90 con 0x7fd02c102a60 2024-09-21T05:55:37.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.837+0000 7fd031f8a640 1 -- 172.21.15.117:0/346982613 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd02c102e70 con 0x7fd02c102a60 2024-09-21T05:55:37.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.838+0000 7fd028ff9640 1 -- 172.21.15.117:0/346982613 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7fd01c003cb0 con 0x7fd02c102a60 2024-09-21T05:55:37.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.839+0000 7fd028ff9640 1 --2- 172.21.15.117:0/346982613 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd0140b65f0 0x7fd0140b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:37.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.839+0000 7fd02affd640 1 --2- 172.21.15.117:0/346982613 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd0140b65f0 0x7fd0140b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:37.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.839+0000 7fd028ff9640 1 -- 172.21.15.117:0/346982613 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fd01c108220 con 0x7fd02c102a60 2024-09-21T05:55:37.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.840+0000 7fd02affd640 1 --2- 172.21.15.117:0/346982613 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd0140b65f0 0x7fd0140b8b00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd02c100cc0 tx=0x7fd020005d30 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:37.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:37.842+0000 7fd028ff9640 1 -- 172.21.15.117:0/346982613 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd01c0ce8b0 con 0x7fd02c102a60 2024-09-21T05:55:38.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:38.025+0000 7fd031f8a640 1 -- 172.21.15.117:0/346982613 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd02c19b600 con 0x7fd02c102a60 2024-09-21T05:55:38.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:38.026+0000 7fd028ff9640 1 -- 172.21.15.117:0/346982613 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+23066 (secure 0 0 0) 0x7fd01c0d2c90 con 0x7fd02c102a60 2024-09-21T05:55:38.028 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:55:38.029 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":76,"fsid":"ac8581a2-77dc-11ef-baf5-efdc52797490","created":"2024-09-21T05:46:47.021846+0000","modified":"2024-09-21T05:55:33.628948+0000","last_up_change":"2024-09-21T05:54:21.572849+0000","last_in_change":"2024-09-21T05:53:58.892597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:50:32.717389+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:55:10.604159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T05:55:12.581258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"68315502-fa6c-4a6e-a2d4-a7dde8071c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6803","nonce":3761478035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6805","nonce":3761478035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6809","nonce":3761478035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6807","nonce":3761478035}]},"public_addr":"172.21.15.117:6803/3761478035","cluster_addr":"172.21.15.117:6805/3761478035","heartbeat_back_addr":"172.21.15.117:6809/3761478035","heartbeat_front_addr":"172.21.15.117:6807/3761478035","state":["exists","up"]},{"osd":1,"uuid":"d8910a08-4ef3-49f2-a4db-e3588d85d12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6801","nonce":1737365588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6803","nonce":1737365588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6807","nonce":1737365588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6805","nonce":1737365588}]},"public_addr":"172.21.15.138:6801/1737365588","cluster_addr":"172.21.15.138:6803/1737365588","heartbeat_back_addr":"172.21.15.138:6807/1737365588","heartbeat_front_addr":"172.21.15.138:6805/1737365588","state":["exists","up"]},{"osd":2,"uuid":"94972378-a11f-4129-9e40-ad3fdec1d189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6801","nonce":3002534992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6803","nonce":3002534992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6807","nonce":3002534992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6805","nonce":3002534992}]},"public_addr":"172.21.15.192:6801/3002534992","cluster_addr":"172.21.15.192:6803/3002534992","heartbeat_back_addr":"172.21.15.192:6807/3002534992","heartbeat_front_addr":"172.21.15.192:6805/3002534992","state":["exists","up"]},{"osd":3,"uuid":"d4168c43-f0de-4445-945c-92d1cf270c33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6811","nonce":1222400293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6813","nonce":1222400293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6817","nonce":1222400293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6815","nonce":1222400293}]},"public_addr":"172.21.15.117:6811/1222400293","cluster_addr":"172.21.15.117:6813/1222400293","heartbeat_back_addr":"172.21.15.117:6817/1222400293","heartbeat_front_addr":"172.21.15.117:6815/1222400293","state":["exists","up"]},{"osd":4,"uuid":"28f1d7c0-8392-4a33-85c0-0de611cdd4b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6809","nonce":4294892870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6811","nonce":4294892870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6815","nonce":4294892870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6813","nonce":4294892870}]},"public_addr":"172.21.15.138:6809/4294892870","cluster_addr":"172.21.15.138:6811/4294892870","heartbeat_back_addr":"172.21.15.138:6815/4294892870","heartbeat_front_addr":"172.21.15.138:6813/4294892870","state":["exists","up"]},{"osd":5,"uuid":"ca902407-45e8-4bc5-bdbd-a729c1de8322","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6809","nonce":1776840019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6811","nonce":1776840019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6815","nonce":1776840019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6813","nonce":1776840019}]},"public_addr":"172.21.15.192:6809/1776840019","cluster_addr":"172.21.15.192:6811/1776840019","heartbeat_back_addr":"172.21.15.192:6815/1776840019","heartbeat_front_addr":"172.21.15.192:6813/1776840019","state":["exists","up"]},{"osd":6,"uuid":"6e507a91-ed09-4a33-bf0e-c4b593b3c5aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6819","nonce":1093341590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6821","nonce":1093341590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6825","nonce":1093341590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6823","nonce":1093341590}]},"public_addr":"172.21.15.117:6819/1093341590","cluster_addr":"172.21.15.117:6821/1093341590","heartbeat_back_addr":"172.21.15.117:6825/1093341590","heartbeat_front_addr":"172.21.15.117:6823/1093341590","state":["exists","up"]},{"osd":7,"uuid":"36a47dd7-67f7-4ebe-8759-3a8bb6bd2ea3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6817","nonce":1448179714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6819","nonce":1448179714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6823","nonce":1448179714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6821","nonce":1448179714}]},"public_addr":"172.21.15.138:6817/1448179714","cluster_addr":"172.21.15.138:6819/1448179714","heartbeat_back_addr":"172.21.15.138:6823/1448179714","heartbeat_front_addr":"172.21.15.138:6821/1448179714","state":["exists","up"]},{"osd":8,"uuid":"118e0b5d-2619-4512-94c5-8d07a0d3a28f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6817","nonce":3960902885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6819","nonce":3960902885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6823","nonce":3960902885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6821","nonce":3960902885}]},"public_addr":"172.21.15.192:6817/3960902885","cluster_addr":"172.21.15.192:6819/3960902885","heartbeat_back_addr":"172.21.15.192:6823/3960902885","heartbeat_front_addr":"172.21.15.192:6821/3960902885","state":["exists","up"]},{"osd":9,"uuid":"47543bc6-fa0b-4e72-9742-97dcda0b7f8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6827","nonce":3453855195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6829","nonce":3453855195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6833","nonce":3453855195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6831","nonce":3453855195}]},"public_addr":"172.21.15.117:6827/3453855195","cluster_addr":"172.21.15.117:6829/3453855195","heartbeat_back_addr":"172.21.15.117:6833/3453855195","heartbeat_front_addr":"172.21.15.117:6831/3453855195","state":["exists","up"]},{"osd":10,"uuid":"5e6708b4-7a85-4fa5-a03c-27d99f7d749f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6825","nonce":903392451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6827","nonce":903392451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6831","nonce":903392451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6829","nonce":903392451}]},"public_addr":"172.21.15.138:6825/903392451","cluster_addr":"172.21.15.138:6827/903392451","heartbeat_back_addr":"172.21.15.138:6831/903392451","heartbeat_front_addr":"172.21.15.138:6829/903392451","state":["exists","up"]},{"osd":11,"uuid":"24bc2ef7-e2b4-43c6-9e6c-43ae02390415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6825","nonce":899090479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6827","nonce":899090479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6831","nonce":899090479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6829","nonce":899090479}]},"public_addr":"172.21.15.192:6825/899090479","cluster_addr":"172.21.15.192:6827/899090479","heartbeat_back_addr":"172.21.15.192:6831/899090479","heartbeat_front_addr":"172.21.15.192:6829/899090479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:49:40.742217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:05.963655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:30.164495+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:56.598899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:51:21.940331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:51:47.777445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:52:12.869007+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:52:39.454054+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:03.569226+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:29.760863+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:55.788604+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:54:20.510184+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:0/2989394419":"2024-09-22T05:48:32.406654+0000","172.21.15.117:6801/1702676172":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/1766406059":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6800/2337036514":"2024-09-22T05:47:09.112435+0000","172.21.15.117:0/985798343":"2024-09-22T05:47:09.112435+0000","172.21.15.117:0/1083273222":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/473284565":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6801/2337036514":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6800/816635165":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/2642940197":"2024-09-22T05:48:32.406654+0000","172.21.15.117:6801/816635165":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/2225537433":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/3651584169":"2024-09-22T05:47:33.789146+0000","172.21.15.117:6800/1702676172":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/1284403882":"2024-09-22T05:47:33.789146+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:55:38.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:38.029+0000 7fd031f8a640 1 -- 172.21.15.117:0/346982613 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd0140b65f0 msgr2=0x7fd0140b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:38.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:38.029+0000 7fd031f8a640 1 --2- 172.21.15.117:0/346982613 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd0140b65f0 0x7fd0140b8b00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd02c100cc0 tx=0x7fd020005d30 comp rx=0 tx=0).stop 2024-09-21T05:55:38.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:38.029+0000 7fd031f8a640 1 -- 172.21.15.117:0/346982613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd02c102a60 msgr2=0x7fd02c19a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:38.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:38.029+0000 7fd031f8a640 1 --2- 172.21.15.117:0/346982613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd02c102a60 0x7fd02c19a4d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd01c006010 tx=0x7fd01c0042f0 comp rx=0 tx=0).stop 2024-09-21T05:55:38.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:38.029+0000 7fd031f8a640 1 -- 172.21.15.117:0/346982613 shutdown_connections 2024-09-21T05:55:38.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:38.029+0000 7fd031f8a640 1 --2- 172.21.15.117:0/346982613 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd0140b65f0 0x7fd0140b8b00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:38.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:38.029+0000 7fd031f8a640 1 --2- 172.21.15.117:0/346982613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd02c108cf0 0x7fd02c19fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:38.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:38.029+0000 7fd031f8a640 1 --2- 172.21.15.117:0/346982613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd02c1083a0 0x7fd02c19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:38.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:38.029+0000 7fd031f8a640 1 --2- 172.21.15.117:0/346982613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd02c102a60 0x7fd02c19a4d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:38.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:38.029+0000 7fd031f8a640 1 -- 172.21.15.117:0/346982613 >> 172.21.15.117:0/346982613 conn(0x7fd02c079730 msgr2=0x7fd02c101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:38.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:38.029+0000 7fd031f8a640 1 -- 172.21.15.117:0/346982613 shutdown_connections 2024-09-21T05:55:38.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:38.029+0000 7fd031f8a640 1 -- 172.21.15.117:0/346982613 wait complete. 2024-09-21T05:55:38.349 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:38 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/346982613' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T05:55:38.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:38 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/346982613' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T05:55:38.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:38 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/346982613' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T05:55:38.661 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T05:55:38.661 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T05:55:38.906 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:39.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:39 smithi117 ceph-mon[30947]: pgmap v277: 97 pgs: 97 active+clean; 634 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.9 KiB/s wr, 15 op/s; 83 B/s, 1 objects/s recovering 2024-09-21T05:55:39.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:39 smithi138 ceph-mon[34391]: pgmap v277: 97 pgs: 97 active+clean; 634 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.9 KiB/s wr, 15 op/s; 83 B/s, 1 objects/s recovering 2024-09-21T05:55:39.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.366+0000 7fcf05c20640 1 -- 172.21.15.117:0/4115001135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf00109df0 msgr2=0x7fcf00110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:39.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.366+0000 7fcf05c20640 1 --2- 172.21.15.117:0/4115001135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf00109df0 0x7fcf00110620 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009ca0 tx=0x7fcef802a8c0 comp rx=0 tx=0).stop 2024-09-21T05:55:39.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.366+0000 7fcf05c20640 1 -- 172.21.15.117:0/4115001135 shutdown_connections 2024-09-21T05:55:39.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.366+0000 7fcf05c20640 1 --2- 172.21.15.117:0/4115001135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf00109df0 0x7fcf00110620 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:39.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.366+0000 7fcf05c20640 1 --2- 172.21.15.117:0/4115001135 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf001094a0 0x7fcf001098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:39.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.366+0000 7fcf05c20640 1 --2- 172.21.15.117:0/4115001135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf00103b60 0x7fcf00103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:39.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.366+0000 7fcf05c20640 1 -- 172.21.15.117:0/4115001135 >> 172.21.15.117:0/4115001135 conn(0x7fcf0007b850 msgr2=0x7fcf0007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:39.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.366+0000 7fcf05c20640 1 -- 172.21.15.117:0/4115001135 shutdown_connections 2024-09-21T05:55:39.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.366+0000 7fcf05c20640 1 -- 172.21.15.117:0/4115001135 wait complete. 2024-09-21T05:55:39.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.367+0000 7fcf05c20640 1 Processor -- start 2024-09-21T05:55:39.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.367+0000 7fcf05c20640 1 -- start start 2024-09-21T05:55:39.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.367+0000 7fcf05c20640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf00103b60 0x7fcf0019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:39.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.367+0000 7fcf05c20640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf001094a0 0x7fcf0019a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:39.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.367+0000 7fcf05c20640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf00109df0 0x7fcf0019fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:39.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.367+0000 7fcf05c20640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcf00113ab0 con 0x7fcf00109df0 2024-09-21T05:55:39.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.367+0000 7fcf05c20640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcf00113930 con 0x7fcf00103b60 2024-09-21T05:55:39.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.367+0000 7fcf05c20640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcf00113c30 con 0x7fcf001094a0 2024-09-21T05:55:39.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.368+0000 7fcefffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf00109df0 0x7fcf0019fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:39.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.368+0000 7fcefffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf00109df0 0x7fcf0019fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37386/0 (socket says 172.21.15.117:37386) 2024-09-21T05:55:39.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.368+0000 7fcefffff640 1 -- 172.21.15.117:0/1396685249 learned_addr learned my addr 172.21.15.117:0/1396685249 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:55:39.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.368+0000 7fcef7fff640 1 --2- 172.21.15.117:0/1396685249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf001094a0 0x7fcf0019a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:39.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.368+0000 7fceff7fe640 1 --2- 172.21.15.117:0/1396685249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf00103b60 0x7fcf0019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:39.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.368+0000 7fcefffff640 1 -- 172.21.15.117:0/1396685249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf001094a0 msgr2=0x7fcf0019a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:39.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.368+0000 7fcefffff640 1 --2- 172.21.15.117:0/1396685249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf001094a0 0x7fcf0019a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:39.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.368+0000 7fcefffff640 1 -- 172.21.15.117:0/1396685249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf00103b60 msgr2=0x7fcf0019a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:39.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.368+0000 7fcefffff640 1 --2- 172.21.15.117:0/1396685249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf00103b60 0x7fcf0019a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:39.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.368+0000 7fcefffff640 1 -- 172.21.15.117:0/1396685249 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf001a0370 con 0x7fcf00109df0 2024-09-21T05:55:39.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.368+0000 7fceff7fe640 1 --2- 172.21.15.117:0/1396685249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf00103b60 0x7fcf0019a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:55:39.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.368+0000 7fcefffff640 1 --2- 172.21.15.117:0/1396685249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf00109df0 0x7fcf0019fc40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcef8005cc0 tx=0x7fcef802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:39.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.368+0000 7fcefd7fa640 1 -- 172.21.15.117:0/1396685249 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef8003bf0 con 0x7fcf00109df0 2024-09-21T05:55:39.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.368+0000 7fcefd7fa640 1 -- 172.21.15.117:0/1396685249 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcef8003d90 con 0x7fcf00109df0 2024-09-21T05:55:39.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.368+0000 7fcf05c20640 1 -- 172.21.15.117:0/1396685249 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf001a0600 con 0x7fcf00109df0 2024-09-21T05:55:39.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.368+0000 7fcefd7fa640 1 -- 172.21.15.117:0/1396685249 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef802fbc0 con 0x7fcf00109df0 2024-09-21T05:55:39.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.368+0000 7fcf05c20640 1 -- 172.21.15.117:0/1396685249 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf001a6dc0 con 0x7fcf00109df0 2024-09-21T05:55:39.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.369+0000 7fcf05c20640 1 -- 172.21.15.117:0/1396685249 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf00103f70 con 0x7fcf00109df0 2024-09-21T05:55:39.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.371+0000 7fcefd7fa640 1 -- 172.21.15.117:0/1396685249 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7fcef802fd60 con 0x7fcf00109df0 2024-09-21T05:55:39.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.372+0000 7fcefd7fa640 1 --2- 172.21.15.117:0/1396685249 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fceec0b6670 0x7fceec0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:39.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.372+0000 7fceff7fe640 1 --2- 172.21.15.117:0/1396685249 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fceec0b6670 0x7fceec0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:39.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.372+0000 7fcefd7fa640 1 -- 172.21.15.117:0/1396685249 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fcef8108bf0 con 0x7fcf00109df0 2024-09-21T05:55:39.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.372+0000 7fceff7fe640 1 --2- 172.21.15.117:0/1396685249 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fceec0b6670 0x7fceec0b8b80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcef00058c0 tx=0x7fcef0005ec0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:39.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.375+0000 7fcefd7fa640 1 -- 172.21.15.117:0/1396685249 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcef80cf220 con 0x7fcf00109df0 2024-09-21T05:55:39.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:39 smithi192 ceph-mon[34700]: pgmap v277: 97 pgs: 97 active+clean; 634 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.9 KiB/s wr, 15 op/s; 83 B/s, 1 objects/s recovering 2024-09-21T05:55:39.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.550+0000 7fcf05c20640 1 -- 172.21.15.117:0/1396685249 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcf00063800 con 0x7fceec0b6670 2024-09-21T05:55:39.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.555+0000 7fcefd7fa640 1 -- 172.21.15.117:0/1396685249 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259067 (secure 0 0 0) 0x7fcf00063800 con 0x7fceec0b6670 2024-09-21T05:55:39.559 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:55:39.560 INFO:teuthology.orchestra.run.smithi117.stderr:dumped all 2024-09-21T05:55:39.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.558+0000 7fcf05c20640 1 -- 172.21.15.117:0/1396685249 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fceec0b6670 msgr2=0x7fceec0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:39.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.558+0000 7fcf05c20640 1 --2- 172.21.15.117:0/1396685249 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fceec0b6670 0x7fceec0b8b80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcef00058c0 tx=0x7fcef0005ec0 comp rx=0 tx=0).stop 2024-09-21T05:55:39.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.558+0000 7fcf05c20640 1 -- 172.21.15.117:0/1396685249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf00109df0 msgr2=0x7fcf0019fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:39.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.558+0000 7fcf05c20640 1 --2- 172.21.15.117:0/1396685249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf00109df0 0x7fcf0019fc40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcef8005cc0 tx=0x7fcef802f380 comp rx=0 tx=0).stop 2024-09-21T05:55:39.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.558+0000 7fcf05c20640 1 -- 172.21.15.117:0/1396685249 shutdown_connections 2024-09-21T05:55:39.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.558+0000 7fcf05c20640 1 --2- 172.21.15.117:0/1396685249 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fceec0b6670 0x7fceec0b8b80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:39.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.558+0000 7fcf05c20640 1 --2- 172.21.15.117:0/1396685249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf00109df0 0x7fcf0019fc40 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:39.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.558+0000 7fcf05c20640 1 --2- 172.21.15.117:0/1396685249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf001094a0 0x7fcf0019a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:39.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.558+0000 7fcf05c20640 1 --2- 172.21.15.117:0/1396685249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf00103b60 0x7fcf0019a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:39.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.558+0000 7fcf05c20640 1 -- 172.21.15.117:0/1396685249 >> 172.21.15.117:0/1396685249 conn(0x7fcf0007b850 msgr2=0x7fcf00107490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:39.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.558+0000 7fcf05c20640 1 -- 172.21.15.117:0/1396685249 shutdown_connections 2024-09-21T05:55:39.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:39.558+0000 7fcf05c20640 1 -- 172.21.15.117:0/1396685249 wait complete. 2024-09-21T05:55:40.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:40 smithi117 ceph-mon[30947]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T05:55:40.357 INFO:teuthology.orchestra.run.smithi117.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2024-09-21T05:55:38.509673+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649137,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":220,"num_read_kb":203,"num_write":358,"num_write_kb":2277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":2952512,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":295,"ondisk_log_size":295,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":293,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":363748,"kb_used_data":10284,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124578076,"statfs":{"total":1151940427776,"available":1151567949824,"internally_reserved":0,"allocated":10530816,"data_stored":6871059,"data_compressed":182193,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":49953,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":96,"num_read_kb":103,"num_write":29,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001779"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636773+0000","last_change":"2024-09-21T05:55:14.007642+0000","last_active":"2024-09-21T05:55:33.636773+0000","last_peered":"2024-09-21T05:55:33.636773+0000","last_clean":"2024-09-21T05:55:33.636773+0000","last_became_active":"2024-09-21T05:55:14.006943+0000","last_became_peered":"2024-09-21T05:55:14.006943+0000","last_unstale":"2024-09-21T05:55:33.636773+0000","last_undegraded":"2024-09-21T05:55:33.636773+0000","last_fullsized":"2024-09-21T05:55:33.636773+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:23:53.816079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767047+0000","last_change":"2024-09-21T05:55:14.002034+0000","last_active":"2024-09-21T05:55:33.767047+0000","last_peered":"2024-09-21T05:55:33.767047+0000","last_clean":"2024-09-21T05:55:33.767047+0000","last_became_active":"2024-09-21T05:55:14.001658+0000","last_became_peered":"2024-09-21T05:55:14.001658+0000","last_unstale":"2024-09-21T05:55:33.767047+0000","last_undegraded":"2024-09-21T05:55:33.767047+0000","last_fullsized":"2024-09-21T05:55:33.767047+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:01.568362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.766840+0000","last_change":"2024-09-21T05:55:13.996901+0000","last_active":"2024-09-21T05:55:33.766840+0000","last_peered":"2024-09-21T05:55:33.766840+0000","last_clean":"2024-09-21T05:55:33.766840+0000","last_became_active":"2024-09-21T05:55:13.996020+0000","last_became_peered":"2024-09-21T05:55:13.996020+0000","last_unstale":"2024-09-21T05:55:33.766840+0000","last_undegraded":"2024-09-21T05:55:33.766840+0000","last_fullsized":"2024-09-21T05:55:33.766840+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:38.771926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.696132+0000","last_change":"2024-09-21T05:55:13.995968+0000","last_active":"2024-09-21T05:55:33.696132+0000","last_peered":"2024-09-21T05:55:33.696132+0000","last_clean":"2024-09-21T05:55:33.696132+0000","last_became_active":"2024-09-21T05:55:13.995259+0000","last_became_peered":"2024-09-21T05:55:13.995259+0000","last_unstale":"2024-09-21T05:55:33.696132+0000","last_undegraded":"2024-09-21T05:55:33.696132+0000","last_fullsized":"2024-09-21T05:55:33.696132+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:48.953342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635823+0000","last_change":"2024-09-21T05:55:13.995778+0000","last_active":"2024-09-21T05:55:33.635823+0000","last_peered":"2024-09-21T05:55:33.635823+0000","last_clean":"2024-09-21T05:55:33.635823+0000","last_became_active":"2024-09-21T05:55:13.995415+0000","last_became_peered":"2024-09-21T05:55:13.995415+0000","last_unstale":"2024-09-21T05:55:33.635823+0000","last_undegraded":"2024-09-21T05:55:33.635823+0000","last_fullsized":"2024-09-21T05:55:33.635823+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:52.562120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.634110+0000","last_change":"2024-09-21T05:55:14.000005+0000","last_active":"2024-09-21T05:55:33.634110+0000","last_peered":"2024-09-21T05:55:33.634110+0000","last_clean":"2024-09-21T05:55:33.634110+0000","last_became_active":"2024-09-21T05:55:13.999493+0000","last_became_peered":"2024-09-21T05:55:13.999493+0000","last_unstale":"2024-09-21T05:55:33.634110+0000","last_undegraded":"2024-09-21T05:55:33.634110+0000","last_fullsized":"2024-09-21T05:55:33.634110+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:29.426368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635816+0000","last_change":"2024-09-21T05:55:13.993004+0000","last_active":"2024-09-21T05:55:33.635816+0000","last_peered":"2024-09-21T05:55:33.635816+0000","last_clean":"2024-09-21T05:55:33.635816+0000","last_became_active":"2024-09-21T05:55:13.992615+0000","last_became_peered":"2024-09-21T05:55:13.992615+0000","last_unstale":"2024-09-21T05:55:33.635816+0000","last_undegraded":"2024-09-21T05:55:33.635816+0000","last_fullsized":"2024-09-21T05:55:33.635816+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:48.324618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.769380+0000","last_change":"2024-09-21T05:55:33.769006+0000","last_active":"2024-09-21T05:55:33.769380+0000","last_peered":"2024-09-21T05:55:33.769380+0000","last_clean":"2024-09-21T05:55:33.769380+0000","last_became_active":"2024-09-21T05:55:33.768705+0000","last_became_peered":"2024-09-21T05:55:33.768705+0000","last_unstale":"2024-09-21T05:55:33.769380+0000","last_undegraded":"2024-09-21T05:55:33.769380+0000","last_fullsized":"2024-09-21T05:55:33.769380+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:14.013300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639798+0000","last_change":"2024-09-21T05:55:14.004131+0000","last_active":"2024-09-21T05:55:33.639798+0000","last_peered":"2024-09-21T05:55:33.639798+0000","last_clean":"2024-09-21T05:55:33.639798+0000","last_became_active":"2024-09-21T05:55:14.003621+0000","last_became_peered":"2024-09-21T05:55:14.003621+0000","last_unstale":"2024-09-21T05:55:33.639798+0000","last_undegraded":"2024-09-21T05:55:33.639798+0000","last_fullsized":"2024-09-21T05:55:33.639798+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:00.694211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.643388+0000","last_change":"2024-09-21T05:55:33.643289+0000","last_active":"2024-09-21T05:55:33.643388+0000","last_peered":"2024-09-21T05:55:33.643388+0000","last_clean":"2024-09-21T05:55:33.643388+0000","last_became_active":"2024-09-21T05:55:33.642805+0000","last_became_peered":"2024-09-21T05:55:33.642805+0000","last_unstale":"2024-09-21T05:55:33.643388+0000","last_undegraded":"2024-09-21T05:55:33.643388+0000","last_fullsized":"2024-09-21T05:55:33.643388+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:00.854712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636225+0000","last_change":"2024-09-21T05:55:14.006467+0000","last_active":"2024-09-21T05:55:33.636225+0000","last_peered":"2024-09-21T05:55:33.636225+0000","last_clean":"2024-09-21T05:55:33.636225+0000","last_became_active":"2024-09-21T05:55:14.005978+0000","last_became_peered":"2024-09-21T05:55:14.005978+0000","last_unstale":"2024-09-21T05:55:33.636225+0000","last_undegraded":"2024-09-21T05:55:33.636225+0000","last_fullsized":"2024-09-21T05:55:33.636225+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:37.391152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.696241+0000","last_change":"2024-09-21T05:55:14.003972+0000","last_active":"2024-09-21T05:55:33.696241+0000","last_peered":"2024-09-21T05:55:33.696241+0000","last_clean":"2024-09-21T05:55:33.696241+0000","last_became_active":"2024-09-21T05:55:14.003601+0000","last_became_peered":"2024-09-21T05:55:14.003601+0000","last_unstale":"2024-09-21T05:55:33.696241+0000","last_undegraded":"2024-09-21T05:55:33.696241+0000","last_fullsized":"2024-09-21T05:55:33.696241+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:16.201087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636463+0000","last_change":"2024-09-21T05:55:13.996051+0000","last_active":"2024-09-21T05:55:33.636463+0000","last_peered":"2024-09-21T05:55:33.636463+0000","last_clean":"2024-09-21T05:55:33.636463+0000","last_became_active":"2024-09-21T05:55:13.995679+0000","last_became_peered":"2024-09-21T05:55:13.995679+0000","last_unstale":"2024-09-21T05:55:33.636463+0000","last_undegraded":"2024-09-21T05:55:33.636463+0000","last_fullsized":"2024-09-21T05:55:33.636463+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:45:44.372029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.766681+0000","last_change":"2024-09-21T05:55:13.995421+0000","last_active":"2024-09-21T05:55:33.766681+0000","last_peered":"2024-09-21T05:55:33.766681+0000","last_clean":"2024-09-21T05:55:33.766681+0000","last_became_active":"2024-09-21T05:55:13.995006+0000","last_became_peered":"2024-09-21T05:55:13.995006+0000","last_unstale":"2024-09-21T05:55:33.766681+0000","last_undegraded":"2024-09-21T05:55:33.766681+0000","last_fullsized":"2024-09-21T05:55:33.766681+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:00.636555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.766677+0000","last_change":"2024-09-21T05:55:13.993466+0000","last_active":"2024-09-21T05:55:33.766677+0000","last_peered":"2024-09-21T05:55:33.766677+0000","last_clean":"2024-09-21T05:55:33.766677+0000","last_became_active":"2024-09-21T05:55:13.993137+0000","last_became_peered":"2024-09-21T05:55:13.993137+0000","last_unstale":"2024-09-21T05:55:33.766677+0000","last_undegraded":"2024-09-21T05:55:33.766677+0000","last_fullsized":"2024-09-21T05:55:33.766677+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:33.218868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639111+0000","last_change":"2024-09-21T05:55:13.996316+0000","last_active":"2024-09-21T05:55:33.639111+0000","last_peered":"2024-09-21T05:55:33.639111+0000","last_clean":"2024-09-21T05:55:33.639111+0000","last_became_active":"2024-09-21T05:55:13.995918+0000","last_became_peered":"2024-09-21T05:55:13.995918+0000","last_unstale":"2024-09-21T05:55:33.639111+0000","last_undegraded":"2024-09-21T05:55:33.639111+0000","last_fullsized":"2024-09-21T05:55:33.639111+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:45:54.408504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636962+0000","last_change":"2024-09-21T05:55:14.007198+0000","last_active":"2024-09-21T05:55:33.636962+0000","last_peered":"2024-09-21T05:55:33.636962+0000","last_clean":"2024-09-21T05:55:33.636962+0000","last_became_active":"2024-09-21T05:55:14.006700+0000","last_became_peered":"2024-09-21T05:55:14.006700+0000","last_unstale":"2024-09-21T05:55:33.636962+0000","last_undegraded":"2024-09-21T05:55:33.636962+0000","last_fullsized":"2024-09-21T05:55:33.636962+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:45.343778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.633333+0000","last_change":"2024-09-21T05:55:14.000152+0000","last_active":"2024-09-21T05:55:33.633333+0000","last_peered":"2024-09-21T05:55:33.633333+0000","last_clean":"2024-09-21T05:55:33.633333+0000","last_became_active":"2024-09-21T05:55:13.999786+0000","last_became_peered":"2024-09-21T05:55:13.999786+0000","last_unstale":"2024-09-21T05:55:33.633333+0000","last_undegraded":"2024-09-21T05:55:33.633333+0000","last_fullsized":"2024-09-21T05:55:33.633333+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:43:25.713410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.640106+0000","last_change":"2024-09-21T05:55:13.988367+0000","last_active":"2024-09-21T05:55:33.640106+0000","last_peered":"2024-09-21T05:55:33.640106+0000","last_clean":"2024-09-21T05:55:33.640106+0000","last_became_active":"2024-09-21T05:55:13.987982+0000","last_became_peered":"2024-09-21T05:55:13.987982+0000","last_unstale":"2024-09-21T05:55:33.640106+0000","last_undegraded":"2024-09-21T05:55:33.640106+0000","last_fullsized":"2024-09-21T05:55:33.640106+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:15.395155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636251+0000","last_change":"2024-09-21T05:55:14.004962+0000","last_active":"2024-09-21T05:55:33.636251+0000","last_peered":"2024-09-21T05:55:33.636251+0000","last_clean":"2024-09-21T05:55:33.636251+0000","last_became_active":"2024-09-21T05:55:14.004573+0000","last_became_peered":"2024-09-21T05:55:14.004573+0000","last_unstale":"2024-09-21T05:55:33.636251+0000","last_undegraded":"2024-09-21T05:55:33.636251+0000","last_fullsized":"2024-09-21T05:55:33.636251+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:59.014017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636468+0000","last_change":"2024-09-21T05:55:13.991267+0000","last_active":"2024-09-21T05:55:33.636468+0000","last_peered":"2024-09-21T05:55:33.636468+0000","last_clean":"2024-09-21T05:55:33.636468+0000","last_became_active":"2024-09-21T05:55:13.990916+0000","last_became_peered":"2024-09-21T05:55:13.990916+0000","last_unstale":"2024-09-21T05:55:33.636468+0000","last_undegraded":"2024-09-21T05:55:33.636468+0000","last_fullsized":"2024-09-21T05:55:33.636468+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:06:03.904632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639504+0000","last_change":"2024-09-21T05:55:13.994147+0000","last_active":"2024-09-21T05:55:33.639504+0000","last_peered":"2024-09-21T05:55:33.639504+0000","last_clean":"2024-09-21T05:55:33.639504+0000","last_became_active":"2024-09-21T05:55:13.993794+0000","last_became_peered":"2024-09-21T05:55:13.993794+0000","last_unstale":"2024-09-21T05:55:33.639504+0000","last_undegraded":"2024-09-21T05:55:33.639504+0000","last_fullsized":"2024-09-21T05:55:33.639504+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:19.970853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.638668+0000","last_change":"2024-09-21T05:55:14.001119+0000","last_active":"2024-09-21T05:55:33.638668+0000","last_peered":"2024-09-21T05:55:33.638668+0000","last_clean":"2024-09-21T05:55:33.638668+0000","last_became_active":"2024-09-21T05:55:14.000799+0000","last_became_peered":"2024-09-21T05:55:14.000799+0000","last_unstale":"2024-09-21T05:55:33.638668+0000","last_undegraded":"2024-09-21T05:55:33.638668+0000","last_fullsized":"2024-09-21T05:55:33.638668+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:28.259188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767914+0000","last_change":"2024-09-21T05:55:13.992043+0000","last_active":"2024-09-21T05:55:33.767914+0000","last_peered":"2024-09-21T05:55:33.767914+0000","last_clean":"2024-09-21T05:55:33.767914+0000","last_became_active":"2024-09-21T05:55:13.991344+0000","last_became_peered":"2024-09-21T05:55:13.991344+0000","last_unstale":"2024-09-21T05:55:33.767914+0000","last_undegraded":"2024-09-21T05:55:33.767914+0000","last_fullsized":"2024-09-21T05:55:33.767914+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:42:55.674333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639242+0000","last_change":"2024-09-21T05:55:13.993114+0000","last_active":"2024-09-21T05:55:33.639242+0000","last_peered":"2024-09-21T05:55:33.639242+0000","last_clean":"2024-09-21T05:55:33.639242+0000","last_became_active":"2024-09-21T05:55:13.992749+0000","last_became_peered":"2024-09-21T05:55:13.992749+0000","last_unstale":"2024-09-21T05:55:33.639242+0000","last_undegraded":"2024-09-21T05:55:33.639242+0000","last_fullsized":"2024-09-21T05:55:33.639242+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:15.926967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.640614+0000","last_change":"2024-09-21T05:55:13.997199+0000","last_active":"2024-09-21T05:55:33.640614+0000","last_peered":"2024-09-21T05:55:33.640614+0000","last_clean":"2024-09-21T05:55:33.640614+0000","last_became_active":"2024-09-21T05:55:13.996809+0000","last_became_peered":"2024-09-21T05:55:13.996809+0000","last_unstale":"2024-09-21T05:55:33.640614+0000","last_undegraded":"2024-09-21T05:55:33.640614+0000","last_fullsized":"2024-09-21T05:55:33.640614+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:16.681430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636906+0000","last_change":"2024-09-21T05:55:14.000879+0000","last_active":"2024-09-21T05:55:33.636906+0000","last_peered":"2024-09-21T05:55:33.636906+0000","last_clean":"2024-09-21T05:55:33.636906+0000","last_became_active":"2024-09-21T05:55:14.000259+0000","last_became_peered":"2024-09-21T05:55:14.000259+0000","last_unstale":"2024-09-21T05:55:33.636906+0000","last_undegraded":"2024-09-21T05:55:33.636906+0000","last_fullsized":"2024-09-21T05:55:33.636906+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:01:08.938417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636509+0000","last_change":"2024-09-21T05:55:13.989433+0000","last_active":"2024-09-21T05:55:33.636509+0000","last_peered":"2024-09-21T05:55:33.636509+0000","last_clean":"2024-09-21T05:55:33.636509+0000","last_became_active":"2024-09-21T05:55:13.989097+0000","last_became_peered":"2024-09-21T05:55:13.989097+0000","last_unstale":"2024-09-21T05:55:33.636509+0000","last_undegraded":"2024-09-21T05:55:33.636509+0000","last_fullsized":"2024-09-21T05:55:33.636509+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:37.835298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767185+0000","last_change":"2024-09-21T05:55:13.993911+0000","last_active":"2024-09-21T05:55:33.767185+0000","last_peered":"2024-09-21T05:55:33.767185+0000","last_clean":"2024-09-21T05:55:33.767185+0000","last_became_active":"2024-09-21T05:55:13.993570+0000","last_became_peered":"2024-09-21T05:55:13.993570+0000","last_unstale":"2024-09-21T05:55:33.767185+0000","last_undegraded":"2024-09-21T05:55:33.767185+0000","last_fullsized":"2024-09-21T05:55:33.767185+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:08:16.588457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639566+0000","last_change":"2024-09-21T05:55:13.992764+0000","last_active":"2024-09-21T05:55:33.639566+0000","last_peered":"2024-09-21T05:55:33.639566+0000","last_clean":"2024-09-21T05:55:33.639566+0000","last_became_active":"2024-09-21T05:55:13.992338+0000","last_became_peered":"2024-09-21T05:55:13.992338+0000","last_unstale":"2024-09-21T05:55:33.639566+0000","last_undegraded":"2024-09-21T05:55:33.639566+0000","last_fullsized":"2024-09-21T05:55:33.639566+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:25:29.997353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639811+0000","last_change":"2024-09-21T05:55:13.987119+0000","last_active":"2024-09-21T05:55:33.639811+0000","last_peered":"2024-09-21T05:55:33.639811+0000","last_clean":"2024-09-21T05:55:33.639811+0000","last_became_active":"2024-09-21T05:55:13.986663+0000","last_became_peered":"2024-09-21T05:55:13.986663+0000","last_unstale":"2024-09-21T05:55:33.639811+0000","last_undegraded":"2024-09-21T05:55:33.639811+0000","last_fullsized":"2024-09-21T05:55:33.639811+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:02.089833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.637197+0000","last_change":"2024-09-21T05:55:14.003577+0000","last_active":"2024-09-21T05:55:33.637197+0000","last_peered":"2024-09-21T05:55:33.637197+0000","last_clean":"2024-09-21T05:55:33.637197+0000","last_became_active":"2024-09-21T05:55:14.003162+0000","last_became_peered":"2024-09-21T05:55:14.003162+0000","last_unstale":"2024-09-21T05:55:33.637197+0000","last_undegraded":"2024-09-21T05:55:33.637197+0000","last_fullsized":"2024-09-21T05:55:33.637197+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:58.630843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635758+0000","last_change":"2024-09-21T05:55:13.992078+0000","last_active":"2024-09-21T05:55:33.635758+0000","last_peered":"2024-09-21T05:55:33.635758+0000","last_clean":"2024-09-21T05:55:33.635758+0000","last_became_active":"2024-09-21T05:55:13.991680+0000","last_became_peered":"2024-09-21T05:55:13.991680+0000","last_unstale":"2024-09-21T05:55:33.635758+0000","last_undegraded":"2024-09-21T05:55:33.635758+0000","last_fullsized":"2024-09-21T05:55:33.635758+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:52.886755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"74'6","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767838+0000","last_change":"2024-09-21T05:55:11.982560+0000","last_active":"2024-09-21T05:55:33.767838+0000","last_peered":"2024-09-21T05:55:33.767838+0000","last_clean":"2024-09-21T05:55:33.767838+0000","last_became_active":"2024-09-21T05:55:11.982179+0000","last_became_peered":"2024-09-21T05:55:11.982179+0000","last_unstale":"2024-09-21T05:55:33.767838+0000","last_undegraded":"2024-09-21T05:55:33.767838+0000","last_fullsized":"2024-09-21T05:55:33.767838+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:06.012194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767640+0000","last_change":"2024-09-21T05:55:13.993994+0000","last_active":"2024-09-21T05:55:33.767640+0000","last_peered":"2024-09-21T05:55:33.767640+0000","last_clean":"2024-09-21T05:55:33.767640+0000","last_became_active":"2024-09-21T05:55:13.993585+0000","last_became_peered":"2024-09-21T05:55:13.993585+0000","last_unstale":"2024-09-21T05:55:33.767640+0000","last_undegraded":"2024-09-21T05:55:33.767640+0000","last_fullsized":"2024-09-21T05:55:33.767640+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:11.608616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"74'7","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.634233+0000","last_change":"2024-09-21T05:55:11.984397+0000","last_active":"2024-09-21T05:55:33.634233+0000","last_peered":"2024-09-21T05:55:33.634233+0000","last_clean":"2024-09-21T05:55:33.634233+0000","last_became_active":"2024-09-21T05:55:11.983888+0000","last_became_peered":"2024-09-21T05:55:11.983888+0000","last_unstale":"2024-09-21T05:55:33.634233+0000","last_undegraded":"2024-09-21T05:55:33.634233+0000","last_fullsized":"2024-09-21T05:55:33.634233+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:59.012710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636187+0000","last_change":"2024-09-21T05:55:11.974547+0000","last_active":"2024-09-21T05:55:33.636187+0000","last_peered":"2024-09-21T05:55:33.636187+0000","last_clean":"2024-09-21T05:55:33.636187+0000","last_became_active":"2024-09-21T05:55:11.974151+0000","last_became_peered":"2024-09-21T05:55:11.974151+0000","last_unstale":"2024-09-21T05:55:33.636187+0000","last_undegraded":"2024-09-21T05:55:33.636187+0000","last_fullsized":"2024-09-21T05:55:33.636187+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:51.489548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.638587+0000","last_change":"2024-09-21T05:55:13.999287+0000","last_active":"2024-09-21T05:55:33.638587+0000","last_peered":"2024-09-21T05:55:33.638587+0000","last_clean":"2024-09-21T05:55:33.638587+0000","last_became_active":"2024-09-21T05:55:13.998742+0000","last_became_peered":"2024-09-21T05:55:13.998742+0000","last_unstale":"2024-09-21T05:55:33.638587+0000","last_undegraded":"2024-09-21T05:55:33.638587+0000","last_fullsized":"2024-09-21T05:55:33.638587+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:33.842810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"74'3","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.768113+0000","last_change":"2024-09-21T05:55:11.981354+0000","last_active":"2024-09-21T05:55:33.768113+0000","last_peered":"2024-09-21T05:55:33.768113+0000","last_clean":"2024-09-21T05:55:33.768113+0000","last_became_active":"2024-09-21T05:55:11.980993+0000","last_became_peered":"2024-09-21T05:55:11.980993+0000","last_unstale":"2024-09-21T05:55:33.768113+0000","last_undegraded":"2024-09-21T05:55:33.768113+0000","last_fullsized":"2024-09-21T05:55:33.768113+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:05.037669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.633042+0000","last_change":"2024-09-21T05:55:14.001041+0000","last_active":"2024-09-21T05:55:33.633042+0000","last_peered":"2024-09-21T05:55:33.633042+0000","last_clean":"2024-09-21T05:55:33.633042+0000","last_became_active":"2024-09-21T05:55:14.000688+0000","last_became_peered":"2024-09-21T05:55:14.000688+0000","last_unstale":"2024-09-21T05:55:33.633042+0000","last_undegraded":"2024-09-21T05:55:33.633042+0000","last_fullsized":"2024-09-21T05:55:33.633042+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:51.190495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"74'4","reported_seq":51,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:37.239608+0000","last_change":"2024-09-21T05:55:11.984097+0000","last_active":"2024-09-21T05:55:37.239608+0000","last_peered":"2024-09-21T05:55:37.239608+0000","last_clean":"2024-09-21T05:55:37.239608+0000","last_became_active":"2024-09-21T05:55:11.982969+0000","last_became_peered":"2024-09-21T05:55:11.982969+0000","last_unstale":"2024-09-21T05:55:37.239608+0000","last_undegraded":"2024-09-21T05:55:37.239608+0000","last_fullsized":"2024-09-21T05:55:37.239608+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:41.372225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639897+0000","last_change":"2024-09-21T05:55:13.998433+0000","last_active":"2024-09-21T05:55:33.639897+0000","last_peered":"2024-09-21T05:55:33.639897+0000","last_clean":"2024-09-21T05:55:33.639897+0000","last_became_active":"2024-09-21T05:55:13.997890+0000","last_became_peered":"2024-09-21T05:55:13.997890+0000","last_unstale":"2024-09-21T05:55:33.639897+0000","last_undegraded":"2024-09-21T05:55:33.639897+0000","last_fullsized":"2024-09-21T05:55:33.639897+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:24.396789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"74'4","reported_seq":40,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.701709+0000","last_change":"2024-09-21T05:55:33.701599+0000","last_active":"2024-09-21T05:55:33.701709+0000","last_peered":"2024-09-21T05:55:33.701709+0000","last_clean":"2024-09-21T05:55:33.701709+0000","last_became_active":"2024-09-21T05:55:33.697385+0000","last_became_peered":"2024-09-21T05:55:33.697385+0000","last_unstale":"2024-09-21T05:55:33.701709+0000","last_undegraded":"2024-09-21T05:55:33.701709+0000","last_fullsized":"2024-09-21T05:55:33.701709+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:18.653991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767050+0000","last_change":"2024-09-21T05:55:13.995111+0000","last_active":"2024-09-21T05:55:33.767050+0000","last_peered":"2024-09-21T05:55:33.767050+0000","last_clean":"2024-09-21T05:55:33.767050+0000","last_became_active":"2024-09-21T05:55:13.994691+0000","last_became_peered":"2024-09-21T05:55:13.994691+0000","last_unstale":"2024-09-21T05:55:33.767050+0000","last_undegraded":"2024-09-21T05:55:33.767050+0000","last_fullsized":"2024-09-21T05:55:33.767050+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:30.167649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"74'5","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767919+0000","last_change":"2024-09-21T05:55:12.013961+0000","last_active":"2024-09-21T05:55:33.767919+0000","last_peered":"2024-09-21T05:55:33.767919+0000","last_clean":"2024-09-21T05:55:33.767919+0000","last_became_active":"2024-09-21T05:55:12.013573+0000","last_became_peered":"2024-09-21T05:55:12.013573+0000","last_unstale":"2024-09-21T05:55:33.767919+0000","last_undegraded":"2024-09-21T05:55:33.767919+0000","last_fullsized":"2024-09-21T05:55:33.767919+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:12.119605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639617+0000","last_change":"2024-09-21T05:55:13.996093+0000","last_active":"2024-09-21T05:55:33.639617+0000","last_peered":"2024-09-21T05:55:33.639617+0000","last_clean":"2024-09-21T05:55:33.639617+0000","last_became_active":"2024-09-21T05:55:13.995342+0000","last_became_peered":"2024-09-21T05:55:13.995342+0000","last_unstale":"2024-09-21T05:55:33.639617+0000","last_undegraded":"2024-09-21T05:55:33.639617+0000","last_fullsized":"2024-09-21T05:55:33.639617+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:20.335330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"76'24","reported_seq":77,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:37.242700+0000","last_change":"2024-09-21T05:55:11.986193+0000","last_active":"2024-09-21T05:55:37.242700+0000","last_peered":"2024-09-21T05:55:37.242700+0000","last_clean":"2024-09-21T05:55:37.242700+0000","last_became_active":"2024-09-21T05:55:11.985736+0000","last_became_peered":"2024-09-21T05:55:11.985736+0000","last_unstale":"2024-09-21T05:55:37.242700+0000","last_undegraded":"2024-09-21T05:55:37.242700+0000","last_fullsized":"2024-09-21T05:55:37.242700+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.503419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50940,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":30,"num_read_kb":64,"num_write":26,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636170+0000","last_change":"2024-09-21T05:55:14.001123+0000","last_active":"2024-09-21T05:55:33.636170+0000","last_peered":"2024-09-21T05:55:33.636170+0000","last_clean":"2024-09-21T05:55:33.636170+0000","last_became_active":"2024-09-21T05:55:14.000742+0000","last_became_peered":"2024-09-21T05:55:14.000742+0000","last_unstale":"2024-09-21T05:55:33.636170+0000","last_undegraded":"2024-09-21T05:55:33.636170+0000","last_fullsized":"2024-09-21T05:55:33.636170+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:15.277992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"74'9","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639957+0000","last_change":"2024-09-21T05:55:11.981396+0000","last_active":"2024-09-21T05:55:33.639957+0000","last_peered":"2024-09-21T05:55:33.639957+0000","last_clean":"2024-09-21T05:55:33.639957+0000","last_became_active":"2024-09-21T05:55:11.980976+0000","last_became_peered":"2024-09-21T05:55:11.980976+0000","last_unstale":"2024-09-21T05:55:33.639957+0000","last_undegraded":"2024-09-21T05:55:33.639957+0000","last_fullsized":"2024-09-21T05:55:33.639957+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:43.600453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636282+0000","last_change":"2024-09-21T05:55:13.991354+0000","last_active":"2024-09-21T05:55:33.636282+0000","last_peered":"2024-09-21T05:55:33.636282+0000","last_clean":"2024-09-21T05:55:33.636282+0000","last_became_active":"2024-09-21T05:55:13.990967+0000","last_became_peered":"2024-09-21T05:55:13.990967+0000","last_unstale":"2024-09-21T05:55:33.636282+0000","last_undegraded":"2024-09-21T05:55:33.636282+0000","last_fullsized":"2024-09-21T05:55:33.636282+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:04.889461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"74'5","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767638+0000","last_change":"2024-09-21T05:55:11.981019+0000","last_active":"2024-09-21T05:55:33.767638+0000","last_peered":"2024-09-21T05:55:33.767638+0000","last_clean":"2024-09-21T05:55:33.767638+0000","last_became_active":"2024-09-21T05:55:11.980619+0000","last_became_peered":"2024-09-21T05:55:11.980619+0000","last_unstale":"2024-09-21T05:55:33.767638+0000","last_undegraded":"2024-09-21T05:55:33.767638+0000","last_fullsized":"2024-09-21T05:55:33.767638+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:41.005092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635917+0000","last_change":"2024-09-21T05:55:13.994535+0000","last_active":"2024-09-21T05:55:33.635917+0000","last_peered":"2024-09-21T05:55:33.635917+0000","last_clean":"2024-09-21T05:55:33.635917+0000","last_became_active":"2024-09-21T05:55:13.994153+0000","last_became_peered":"2024-09-21T05:55:13.994153+0000","last_unstale":"2024-09-21T05:55:33.635917+0000","last_undegraded":"2024-09-21T05:55:33.635917+0000","last_fullsized":"2024-09-21T05:55:33.635917+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:02.301263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"74'6","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.696142+0000","last_change":"2024-09-21T05:55:11.977694+0000","last_active":"2024-09-21T05:55:33.696142+0000","last_peered":"2024-09-21T05:55:33.696142+0000","last_clean":"2024-09-21T05:55:33.696142+0000","last_became_active":"2024-09-21T05:55:11.977146+0000","last_became_peered":"2024-09-21T05:55:11.977146+0000","last_unstale":"2024-09-21T05:55:33.696142+0000","last_undegraded":"2024-09-21T05:55:33.696142+0000","last_fullsized":"2024-09-21T05:55:33.696142+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:29.736564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635721+0000","last_change":"2024-09-21T05:55:13.998032+0000","last_active":"2024-09-21T05:55:33.635721+0000","last_peered":"2024-09-21T05:55:33.635721+0000","last_clean":"2024-09-21T05:55:33.635721+0000","last_became_active":"2024-09-21T05:55:13.997617+0000","last_became_peered":"2024-09-21T05:55:13.997617+0000","last_unstale":"2024-09-21T05:55:33.635721+0000","last_undegraded":"2024-09-21T05:55:33.635721+0000","last_fullsized":"2024-09-21T05:55:33.635721+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:46.552726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"74'17","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635659+0000","last_change":"2024-09-21T05:55:12.013540+0000","last_active":"2024-09-21T05:55:33.635659+0000","last_peered":"2024-09-21T05:55:33.635659+0000","last_clean":"2024-09-21T05:55:33.635659+0000","last_became_active":"2024-09-21T05:55:12.013151+0000","last_became_peered":"2024-09-21T05:55:12.013151+0000","last_unstale":"2024-09-21T05:55:33.635659+0000","last_undegraded":"2024-09-21T05:55:33.635659+0000","last_fullsized":"2024-09-21T05:55:33.635659+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:53.705904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767640+0000","last_change":"2024-09-21T05:55:13.999501+0000","last_active":"2024-09-21T05:55:33.767640+0000","last_peered":"2024-09-21T05:55:33.767640+0000","last_clean":"2024-09-21T05:55:33.767640+0000","last_became_active":"2024-09-21T05:55:13.997819+0000","last_became_peered":"2024-09-21T05:55:13.997819+0000","last_unstale":"2024-09-21T05:55:33.767640+0000","last_undegraded":"2024-09-21T05:55:33.767640+0000","last_fullsized":"2024-09-21T05:55:33.767640+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:54.334755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"74'2","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636760+0000","last_change":"2024-09-21T05:55:11.984428+0000","last_active":"2024-09-21T05:55:33.636760+0000","last_peered":"2024-09-21T05:55:33.636760+0000","last_clean":"2024-09-21T05:55:33.636760+0000","last_became_active":"2024-09-21T05:55:11.983543+0000","last_became_peered":"2024-09-21T05:55:11.983543+0000","last_unstale":"2024-09-21T05:55:33.636760+0000","last_undegraded":"2024-09-21T05:55:33.636760+0000","last_fullsized":"2024-09-21T05:55:33.636760+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:39.636408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.637304+0000","last_change":"2024-09-21T05:55:14.003851+0000","last_active":"2024-09-21T05:55:33.637304+0000","last_peered":"2024-09-21T05:55:33.637304+0000","last_clean":"2024-09-21T05:55:33.637304+0000","last_became_active":"2024-09-21T05:55:14.003354+0000","last_became_peered":"2024-09-21T05:55:14.003354+0000","last_unstale":"2024-09-21T05:55:33.637304+0000","last_undegraded":"2024-09-21T05:55:33.637304+0000","last_fullsized":"2024-09-21T05:55:33.637304+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:01.321612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"74'5","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.645976+0000","last_change":"2024-09-21T05:55:33.645825+0000","last_active":"2024-09-21T05:55:33.645976+0000","last_peered":"2024-09-21T05:55:33.645976+0000","last_clean":"2024-09-21T05:55:33.645976+0000","last_became_active":"2024-09-21T05:55:33.640707+0000","last_became_peered":"2024-09-21T05:55:33.640707+0000","last_unstale":"2024-09-21T05:55:33.645976+0000","last_undegraded":"2024-09-21T05:55:33.645976+0000","last_fullsized":"2024-09-21T05:55:33.645976+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:33.665707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636452+0000","last_change":"2024-09-21T05:55:14.001855+0000","last_active":"2024-09-21T05:55:33.636452+0000","last_peered":"2024-09-21T05:55:33.636452+0000","last_clean":"2024-09-21T05:55:33.636452+0000","last_became_active":"2024-09-21T05:55:14.001484+0000","last_became_peered":"2024-09-21T05:55:14.001484+0000","last_unstale":"2024-09-21T05:55:33.636452+0000","last_undegraded":"2024-09-21T05:55:33.636452+0000","last_fullsized":"2024-09-21T05:55:33.636452+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:57.615334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'3","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767928+0000","last_change":"2024-09-21T05:55:11.979716+0000","last_active":"2024-09-21T05:55:33.767928+0000","last_peered":"2024-09-21T05:55:33.767928+0000","last_clean":"2024-09-21T05:55:33.767928+0000","last_became_active":"2024-09-21T05:55:11.979307+0000","last_became_peered":"2024-09-21T05:55:11.979307+0000","last_unstale":"2024-09-21T05:55:33.767928+0000","last_undegraded":"2024-09-21T05:55:33.767928+0000","last_fullsized":"2024-09-21T05:55:33.767928+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:30.019040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636060+0000","last_change":"2024-09-21T05:55:13.991769+0000","last_active":"2024-09-21T05:55:33.636060+0000","last_peered":"2024-09-21T05:55:33.636060+0000","last_clean":"2024-09-21T05:55:33.636060+0000","last_became_active":"2024-09-21T05:55:13.991345+0000","last_became_peered":"2024-09-21T05:55:13.991345+0000","last_unstale":"2024-09-21T05:55:33.636060+0000","last_undegraded":"2024-09-21T05:55:33.636060+0000","last_fullsized":"2024-09-21T05:55:33.636060+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:52.227126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"74'6","reported_seq":44,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:36.238721+0000","last_change":"2024-09-21T05:55:12.013836+0000","last_active":"2024-09-21T05:55:36.238721+0000","last_peered":"2024-09-21T05:55:36.238721+0000","last_clean":"2024-09-21T05:55:36.238721+0000","last_became_active":"2024-09-21T05:55:12.013436+0000","last_became_peered":"2024-09-21T05:55:12.013436+0000","last_unstale":"2024-09-21T05:55:36.238721+0000","last_undegraded":"2024-09-21T05:55:36.238721+0000","last_fullsized":"2024-09-21T05:55:36.238721+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:40:02.767620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636197+0000","last_change":"2024-09-21T05:55:14.001156+0000","last_active":"2024-09-21T05:55:33.636197+0000","last_peered":"2024-09-21T05:55:33.636197+0000","last_clean":"2024-09-21T05:55:33.636197+0000","last_became_active":"2024-09-21T05:55:14.000777+0000","last_became_peered":"2024-09-21T05:55:14.000777+0000","last_unstale":"2024-09-21T05:55:33.636197+0000","last_undegraded":"2024-09-21T05:55:33.636197+0000","last_fullsized":"2024-09-21T05:55:33.636197+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:46:08.000612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":97,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636093+0000","last_change":"2024-09-21T05:53:58.971355+0000","last_active":"2024-09-21T05:55:33.636093+0000","last_peered":"2024-09-21T05:55:33.636093+0000","last_clean":"2024-09-21T05:55:33.636093+0000","last_became_active":"2024-09-21T05:53:58.965825+0000","last_became_peered":"2024-09-21T05:53:58.965825+0000","last_unstale":"2024-09-21T05:55:33.636093+0000","last_undegraded":"2024-09-21T05:55:33.636093+0000","last_fullsized":"2024-09-21T05:55:33.636093+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:50:33.190160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:50:33.190160+0000","last_clean_scrub_stamp":"2024-09-21T05:50:33.190160+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:31.387339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"74'6","reported_seq":32,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.696020+0000","last_change":"2024-09-21T05:55:11.976092+0000","last_active":"2024-09-21T05:55:33.696020+0000","last_peered":"2024-09-21T05:55:33.696020+0000","last_clean":"2024-09-21T05:55:33.696020+0000","last_became_active":"2024-09-21T05:55:11.975657+0000","last_became_peered":"2024-09-21T05:55:11.975657+0000","last_unstale":"2024-09-21T05:55:33.696020+0000","last_undegraded":"2024-09-21T05:55:33.696020+0000","last_fullsized":"2024-09-21T05:55:33.696020+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:46.792054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639547+0000","last_change":"2024-09-21T05:55:13.997621+0000","last_active":"2024-09-21T05:55:33.639547+0000","last_peered":"2024-09-21T05:55:33.639547+0000","last_clean":"2024-09-21T05:55:33.639547+0000","last_became_active":"2024-09-21T05:55:13.997127+0000","last_became_peered":"2024-09-21T05:55:13.997127+0000","last_unstale":"2024-09-21T05:55:33.639547+0000","last_undegraded":"2024-09-21T05:55:33.639547+0000","last_fullsized":"2024-09-21T05:55:33.639547+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:32.399696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"74'4","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635928+0000","last_change":"2024-09-21T05:55:11.982387+0000","last_active":"2024-09-21T05:55:33.635928+0000","last_peered":"2024-09-21T05:55:33.635928+0000","last_clean":"2024-09-21T05:55:33.635928+0000","last_became_active":"2024-09-21T05:55:11.981851+0000","last_became_peered":"2024-09-21T05:55:11.981851+0000","last_unstale":"2024-09-21T05:55:33.635928+0000","last_undegraded":"2024-09-21T05:55:33.635928+0000","last_fullsized":"2024-09-21T05:55:33.635928+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:45.345738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636236+0000","last_change":"2024-09-21T05:55:13.981065+0000","last_active":"2024-09-21T05:55:33.636236+0000","last_peered":"2024-09-21T05:55:33.636236+0000","last_clean":"2024-09-21T05:55:33.636236+0000","last_became_active":"2024-09-21T05:55:13.980524+0000","last_became_peered":"2024-09-21T05:55:13.980524+0000","last_unstale":"2024-09-21T05:55:33.636236+0000","last_undegraded":"2024-09-21T05:55:33.636236+0000","last_fullsized":"2024-09-21T05:55:33.636236+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:43.322429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"74'5","reported_seq":39,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:34.234599+0000","last_change":"2024-09-21T05:55:12.013434+0000","last_active":"2024-09-21T05:55:34.234599+0000","last_peered":"2024-09-21T05:55:34.234599+0000","last_clean":"2024-09-21T05:55:34.234599+0000","last_became_active":"2024-09-21T05:55:12.013087+0000","last_became_peered":"2024-09-21T05:55:12.013087+0000","last_unstale":"2024-09-21T05:55:34.234599+0000","last_undegraded":"2024-09-21T05:55:34.234599+0000","last_fullsized":"2024-09-21T05:55:34.234599+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:19.876288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767927+0000","last_change":"2024-09-21T05:55:13.996102+0000","last_active":"2024-09-21T05:55:33.767927+0000","last_peered":"2024-09-21T05:55:33.767927+0000","last_clean":"2024-09-21T05:55:33.767927+0000","last_became_active":"2024-09-21T05:55:13.994900+0000","last_became_peered":"2024-09-21T05:55:13.994900+0000","last_unstale":"2024-09-21T05:55:33.767927+0000","last_undegraded":"2024-09-21T05:55:33.767927+0000","last_fullsized":"2024-09-21T05:55:33.767927+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:04.998577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"74'5","reported_seq":38,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.640273+0000","last_change":"2024-09-21T05:55:11.981184+0000","last_active":"2024-09-21T05:55:33.640273+0000","last_peered":"2024-09-21T05:55:33.640273+0000","last_clean":"2024-09-21T05:55:33.640273+0000","last_became_active":"2024-09-21T05:55:11.979581+0000","last_became_peered":"2024-09-21T05:55:11.979581+0000","last_unstale":"2024-09-21T05:55:33.640273+0000","last_undegraded":"2024-09-21T05:55:33.640273+0000","last_fullsized":"2024-09-21T05:55:33.640273+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:53.833860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.768241+0000","last_change":"2024-09-21T05:55:33.768136+0000","last_active":"2024-09-21T05:55:33.768241+0000","last_peered":"2024-09-21T05:55:33.768241+0000","last_clean":"2024-09-21T05:55:33.768241+0000","last_became_active":"2024-09-21T05:55:33.767697+0000","last_became_peered":"2024-09-21T05:55:33.767697+0000","last_unstale":"2024-09-21T05:55:33.768241+0000","last_undegraded":"2024-09-21T05:55:33.768241+0000","last_fullsized":"2024-09-21T05:55:33.768241+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:48.146453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"74'1","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.633171+0000","last_change":"2024-09-21T05:55:12.013121+0000","last_active":"2024-09-21T05:55:33.633171+0000","last_peered":"2024-09-21T05:55:33.633171+0000","last_clean":"2024-09-21T05:55:33.633171+0000","last_became_active":"2024-09-21T05:55:12.012657+0000","last_became_peered":"2024-09-21T05:55:12.012657+0000","last_unstale":"2024-09-21T05:55:33.633171+0000","last_undegraded":"2024-09-21T05:55:33.633171+0000","last_fullsized":"2024-09-21T05:55:33.633171+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:25:48.002938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"74'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.634506+0000","last_change":"2024-09-21T05:55:14.004660+0000","last_active":"2024-09-21T05:55:33.634506+0000","last_peered":"2024-09-21T05:55:33.634506+0000","last_clean":"2024-09-21T05:55:33.634506+0000","last_became_active":"2024-09-21T05:55:14.004297+0000","last_became_peered":"2024-09-21T05:55:14.004297+0000","last_unstale":"2024-09-21T05:55:33.634506+0000","last_undegraded":"2024-09-21T05:55:33.634506+0000","last_fullsized":"2024-09-21T05:55:33.634506+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:24.399891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"74'4","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636395+0000","last_change":"2024-09-21T05:55:12.015717+0000","last_active":"2024-09-21T05:55:33.636395+0000","last_peered":"2024-09-21T05:55:33.636395+0000","last_clean":"2024-09-21T05:55:33.636395+0000","last_became_active":"2024-09-21T05:55:12.015419+0000","last_became_peered":"2024-09-21T05:55:12.015419+0000","last_unstale":"2024-09-21T05:55:33.636395+0000","last_undegraded":"2024-09-21T05:55:33.636395+0000","last_fullsized":"2024-09-21T05:55:33.636395+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:57.988860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636635+0000","last_change":"2024-09-21T05:55:13.992460+0000","last_active":"2024-09-21T05:55:33.636635+0000","last_peered":"2024-09-21T05:55:33.636635+0000","last_clean":"2024-09-21T05:55:33.636635+0000","last_became_active":"2024-09-21T05:55:13.991887+0000","last_became_peered":"2024-09-21T05:55:13.991887+0000","last_unstale":"2024-09-21T05:55:33.636635+0000","last_undegraded":"2024-09-21T05:55:33.636635+0000","last_fullsized":"2024-09-21T05:55:33.636635+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:37.718050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"76'12","reported_seq":35,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.994291+0000","last_change":"2024-09-21T05:55:11.982235+0000","last_active":"2024-09-21T05:55:33.994291+0000","last_peered":"2024-09-21T05:55:33.994291+0000","last_clean":"2024-09-21T05:55:33.994291+0000","last_became_active":"2024-09-21T05:55:11.981846+0000","last_became_peered":"2024-09-21T05:55:11.981846+0000","last_unstale":"2024-09-21T05:55:33.994291+0000","last_undegraded":"2024-09-21T05:55:33.994291+0000","last_fullsized":"2024-09-21T05:55:33.994291+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:40:04.330346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639253+0000","last_change":"2024-09-21T05:55:13.999249+0000","last_active":"2024-09-21T05:55:33.639253+0000","last_peered":"2024-09-21T05:55:33.639253+0000","last_clean":"2024-09-21T05:55:33.639253+0000","last_became_active":"2024-09-21T05:55:13.998452+0000","last_became_peered":"2024-09-21T05:55:13.998452+0000","last_unstale":"2024-09-21T05:55:33.639253+0000","last_undegraded":"2024-09-21T05:55:33.639253+0000","last_fullsized":"2024-09-21T05:55:33.639253+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:59.364684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"74'7","reported_seq":31,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.696351+0000","last_change":"2024-09-21T05:55:11.971850+0000","last_active":"2024-09-21T05:55:33.696351+0000","last_peered":"2024-09-21T05:55:33.696351+0000","last_clean":"2024-09-21T05:55:33.696351+0000","last_became_active":"2024-09-21T05:55:11.971415+0000","last_became_peered":"2024-09-21T05:55:11.971415+0000","last_unstale":"2024-09-21T05:55:33.696351+0000","last_undegraded":"2024-09-21T05:55:33.696351+0000","last_fullsized":"2024-09-21T05:55:33.696351+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:51.990745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.766931+0000","last_change":"2024-09-21T05:55:13.994500+0000","last_active":"2024-09-21T05:55:33.766931+0000","last_peered":"2024-09-21T05:55:33.766931+0000","last_clean":"2024-09-21T05:55:33.766931+0000","last_became_active":"2024-09-21T05:55:13.994146+0000","last_became_peered":"2024-09-21T05:55:13.994146+0000","last_unstale":"2024-09-21T05:55:33.766931+0000","last_undegraded":"2024-09-21T05:55:33.766931+0000","last_fullsized":"2024-09-21T05:55:33.766931+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:42.934234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639268+0000","last_change":"2024-09-21T05:55:12.013479+0000","last_active":"2024-09-21T05:55:33.639268+0000","last_peered":"2024-09-21T05:55:33.639268+0000","last_clean":"2024-09-21T05:55:33.639268+0000","last_became_active":"2024-09-21T05:55:12.013180+0000","last_became_peered":"2024-09-21T05:55:12.013180+0000","last_unstale":"2024-09-21T05:55:33.639268+0000","last_undegraded":"2024-09-21T05:55:33.639268+0000","last_fullsized":"2024-09-21T05:55:33.639268+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:28.800072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635548+0000","last_change":"2024-09-21T05:55:14.001795+0000","last_active":"2024-09-21T05:55:33.635548+0000","last_peered":"2024-09-21T05:55:33.635548+0000","last_clean":"2024-09-21T05:55:33.635548+0000","last_became_active":"2024-09-21T05:55:14.001430+0000","last_became_peered":"2024-09-21T05:55:14.001430+0000","last_unstale":"2024-09-21T05:55:33.635548+0000","last_undegraded":"2024-09-21T05:55:33.635548+0000","last_fullsized":"2024-09-21T05:55:33.635548+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:10.339939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"74'7","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.638793+0000","last_change":"2024-09-21T05:55:11.976894+0000","last_active":"2024-09-21T05:55:33.638793+0000","last_peered":"2024-09-21T05:55:33.638793+0000","last_clean":"2024-09-21T05:55:33.638793+0000","last_became_active":"2024-09-21T05:55:11.976516+0000","last_became_peered":"2024-09-21T05:55:11.976516+0000","last_unstale":"2024-09-21T05:55:33.638793+0000","last_undegraded":"2024-09-21T05:55:33.638793+0000","last_fullsized":"2024-09-21T05:55:33.638793+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:58.726311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.637297+0000","last_change":"2024-09-21T05:55:14.003760+0000","last_active":"2024-09-21T05:55:33.637297+0000","last_peered":"2024-09-21T05:55:33.637297+0000","last_clean":"2024-09-21T05:55:33.637297+0000","last_became_active":"2024-09-21T05:55:14.003180+0000","last_became_peered":"2024-09-21T05:55:14.003180+0000","last_unstale":"2024-09-21T05:55:33.637297+0000","last_undegraded":"2024-09-21T05:55:33.637297+0000","last_fullsized":"2024-09-21T05:55:33.637297+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:25.077765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639713+0000","last_change":"2024-09-21T05:55:11.983865+0000","last_active":"2024-09-21T05:55:33.639713+0000","last_peered":"2024-09-21T05:55:33.639713+0000","last_clean":"2024-09-21T05:55:33.639713+0000","last_became_active":"2024-09-21T05:55:11.983473+0000","last_became_peered":"2024-09-21T05:55:11.983473+0000","last_unstale":"2024-09-21T05:55:33.639713+0000","last_undegraded":"2024-09-21T05:55:33.639713+0000","last_fullsized":"2024-09-21T05:55:33.639713+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:18.767317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639006+0000","last_change":"2024-09-21T05:55:13.999001+0000","last_active":"2024-09-21T05:55:33.639006+0000","last_peered":"2024-09-21T05:55:33.639006+0000","last_clean":"2024-09-21T05:55:33.639006+0000","last_became_active":"2024-09-21T05:55:13.998656+0000","last_became_peered":"2024-09-21T05:55:13.998656+0000","last_unstale":"2024-09-21T05:55:33.639006+0000","last_undegraded":"2024-09-21T05:55:33.639006+0000","last_fullsized":"2024-09-21T05:55:33.639006+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:50.913635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"74'4","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.646791+0000","last_change":"2024-09-21T05:55:33.646124+0000","last_active":"2024-09-21T05:55:33.646791+0000","last_peered":"2024-09-21T05:55:33.646791+0000","last_clean":"2024-09-21T05:55:33.646791+0000","last_became_active":"2024-09-21T05:55:33.641142+0000","last_became_peered":"2024-09-21T05:55:33.641142+0000","last_unstale":"2024-09-21T05:55:33.646791+0000","last_undegraded":"2024-09-21T05:55:33.646791+0000","last_fullsized":"2024-09-21T05:55:33.646791+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:36.341473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635915+0000","last_change":"2024-09-21T05:55:13.997494+0000","last_active":"2024-09-21T05:55:33.635915+0000","last_peered":"2024-09-21T05:55:33.635915+0000","last_clean":"2024-09-21T05:55:33.635915+0000","last_became_active":"2024-09-21T05:55:13.997138+0000","last_became_peered":"2024-09-21T05:55:13.997138+0000","last_unstale":"2024-09-21T05:55:33.635915+0000","last_undegraded":"2024-09-21T05:55:33.635915+0000","last_fullsized":"2024-09-21T05:55:33.635915+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:02:01.777126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767713+0000","last_change":"2024-09-21T05:55:14.002750+0000","last_active":"2024-09-21T05:55:33.767713+0000","last_peered":"2024-09-21T05:55:33.767713+0000","last_clean":"2024-09-21T05:55:33.767713+0000","last_became_active":"2024-09-21T05:55:14.002150+0000","last_became_peered":"2024-09-21T05:55:14.002150+0000","last_unstale":"2024-09-21T05:55:33.767713+0000","last_undegraded":"2024-09-21T05:55:33.767713+0000","last_fullsized":"2024-09-21T05:55:33.767713+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:49.603133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"74'5","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636032+0000","last_change":"2024-09-21T05:55:11.982366+0000","last_active":"2024-09-21T05:55:33.636032+0000","last_peered":"2024-09-21T05:55:33.636032+0000","last_clean":"2024-09-21T05:55:33.636032+0000","last_became_active":"2024-09-21T05:55:11.981994+0000","last_became_peered":"2024-09-21T05:55:11.981994+0000","last_unstale":"2024-09-21T05:55:33.636032+0000","last_undegraded":"2024-09-21T05:55:33.636032+0000","last_fullsized":"2024-09-21T05:55:33.636032+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:14.626758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639407+0000","last_change":"2024-09-21T05:55:14.008204+0000","last_active":"2024-09-21T05:55:33.639407+0000","last_peered":"2024-09-21T05:55:33.639407+0000","last_clean":"2024-09-21T05:55:33.639407+0000","last_became_active":"2024-09-21T05:55:14.007800+0000","last_became_peered":"2024-09-21T05:55:14.007800+0000","last_unstale":"2024-09-21T05:55:33.639407+0000","last_undegraded":"2024-09-21T05:55:33.639407+0000","last_fullsized":"2024-09-21T05:55:33.639407+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:49.760850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"74'12","reported_seq":36,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636283+0000","last_change":"2024-09-21T05:55:11.986168+0000","last_active":"2024-09-21T05:55:33.636283+0000","last_peered":"2024-09-21T05:55:33.636283+0000","last_clean":"2024-09-21T05:55:33.636283+0000","last_became_active":"2024-09-21T05:55:11.985694+0000","last_became_peered":"2024-09-21T05:55:11.985694+0000","last_unstale":"2024-09-21T05:55:33.636283+0000","last_undegraded":"2024-09-21T05:55:33.636283+0000","last_fullsized":"2024-09-21T05:55:33.636283+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:44.863935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767026+0000","last_change":"2024-09-21T05:55:14.002459+0000","last_active":"2024-09-21T05:55:33.767026+0000","last_peered":"2024-09-21T05:55:33.767026+0000","last_clean":"2024-09-21T05:55:33.767026+0000","last_became_active":"2024-09-21T05:55:14.001962+0000","last_became_peered":"2024-09-21T05:55:14.001962+0000","last_unstale":"2024-09-21T05:55:33.767026+0000","last_undegraded":"2024-09-21T05:55:33.767026+0000","last_fullsized":"2024-09-21T05:55:33.767026+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:53:26.892772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639340+0000","last_change":"2024-09-21T05:55:12.014149+0000","last_active":"2024-09-21T05:55:33.639340+0000","last_peered":"2024-09-21T05:55:33.639340+0000","last_clean":"2024-09-21T05:55:33.639340+0000","last_became_active":"2024-09-21T05:55:12.013789+0000","last_became_peered":"2024-09-21T05:55:12.013789+0000","last_unstale":"2024-09-21T05:55:33.639340+0000","last_undegraded":"2024-09-21T05:55:33.639340+0000","last_fullsized":"2024-09-21T05:55:33.639340+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:18.304023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639186+0000","last_change":"2024-09-21T05:55:13.993500+0000","last_active":"2024-09-21T05:55:33.639186+0000","last_peered":"2024-09-21T05:55:33.639186+0000","last_clean":"2024-09-21T05:55:33.639186+0000","last_became_active":"2024-09-21T05:55:13.993104+0000","last_became_peered":"2024-09-21T05:55:13.993104+0000","last_unstale":"2024-09-21T05:55:33.639186+0000","last_undegraded":"2024-09-21T05:55:33.639186+0000","last_fullsized":"2024-09-21T05:55:33.639186+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:01.445119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"74'13","reported_seq":68,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:34.640963+0000","last_change":"2024-09-21T05:55:33.650207+0000","last_active":"2024-09-21T05:55:34.640963+0000","last_peered":"2024-09-21T05:55:34.640963+0000","last_clean":"2024-09-21T05:55:34.640963+0000","last_became_active":"2024-09-21T05:55:33.638388+0000","last_became_peered":"2024-09-21T05:55:33.638388+0000","last_unstale":"2024-09-21T05:55:34.640963+0000","last_undegraded":"2024-09-21T05:55:34.640963+0000","last_fullsized":"2024-09-21T05:55:34.640963+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:51.478063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":58535,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":139,"num_read_kb":134,"num_write":233,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":528384,"data_stored":175605,"data_compressed":5682,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":201,"ondisk_log_size":201,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281769,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":68,"seq":292057776145,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":839680,"data_stored":412559,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939670,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":974848,"data_stored":1053856,"data_compressed":43475,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103195,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":856064,"data_stored":411426,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299425,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":851968,"data_stored":411804,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495653,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":897024,"data_stored":1004041,"data_compressed":41581,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659179,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":839680,"data_stored":411143,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822703,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":917504,"data_stored":1004000,"data_compressed":41581,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986229,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":839680,"data_stored":411395,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":856064,"data_stored":410968,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378688,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":909312,"data_stored":463998,"data_compressed":7646,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542211,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":888832,"data_stored":462996,"data_compressed":7646,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705738,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":860160,"data_stored":412873,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34208,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34208,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34208,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35829,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35829,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35829,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35829,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35829,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":52692,"data_compressed":1894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":53694,"data_compressed":1894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":53067,"data_compressed":1894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:55:40.362 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T05:55:40.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:40 smithi138 ceph-mon[34391]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T05:55:40.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:40 smithi192 ceph-mon[34700]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T05:55:40.615 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:41.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.092+0000 7f97a8d83640 1 -- 172.21.15.117:0/503448411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97a4103950 msgr2=0x7f97a410f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:41.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.092+0000 7f97a8d83640 1 --2- 172.21.15.117:0/503448411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97a4103950 0x7f97a410f500 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f979c009ca0 tx=0x7f979c02a8c0 comp rx=0 tx=0).stop 2024-09-21T05:55:41.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.092+0000 7f97a8d83640 1 -- 172.21.15.117:0/503448411 shutdown_connections 2024-09-21T05:55:41.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.092+0000 7f97a8d83640 1 --2- 172.21.15.117:0/503448411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97a4103950 0x7f97a410f500 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:41.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.092+0000 7f97a8d83640 1 --2- 172.21.15.117:0/503448411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97a4103000 0x7f97a4103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:41.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.092+0000 7f97a8d83640 1 --2- 172.21.15.117:0/503448411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97a4107690 0x7f97a4107aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:41.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.092+0000 7f97a8d83640 1 -- 172.21.15.117:0/503448411 >> 172.21.15.117:0/503448411 conn(0x7f97a4079ea0 msgr2=0x7f97a407a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:41.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.092+0000 7f97a8d83640 1 -- 172.21.15.117:0/503448411 shutdown_connections 2024-09-21T05:55:41.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.092+0000 7f97a8d83640 1 -- 172.21.15.117:0/503448411 wait complete. 2024-09-21T05:55:41.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.093+0000 7f97a8d83640 1 Processor -- start 2024-09-21T05:55:41.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.093+0000 7f97a8d83640 1 -- start start 2024-09-21T05:55:41.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.093+0000 7f97a8d83640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97a4103000 0x7f97a419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:41.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.093+0000 7f97a8d83640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97a4103950 0x7f97a419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:41.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.093+0000 7f97a8d83640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97a4107690 0x7f97a419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:41.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.093+0000 7f97a8d83640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f97a4112a40 con 0x7f97a4103000 2024-09-21T05:55:41.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.093+0000 7f97a8d83640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f97a41128c0 con 0x7f97a4107690 2024-09-21T05:55:41.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.093+0000 7f97a8d83640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f97a4112bc0 con 0x7f97a4103950 2024-09-21T05:55:41.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.093+0000 7f97a259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97a4103000 0x7f97a419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:41.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.093+0000 7f97a259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97a4103000 0x7f97a419a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37398/0 (socket says 172.21.15.117:37398) 2024-09-21T05:55:41.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.093+0000 7f97a259c640 1 -- 172.21.15.117:0/3249132574 learned_addr learned my addr 172.21.15.117:0/3249132574 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:55:41.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.093+0000 7f979bfff640 1 --2- 172.21.15.117:0/3249132574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97a4103950 0x7f97a419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:41.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.093+0000 7f97a2d9d640 1 --2- 172.21.15.117:0/3249132574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97a4107690 0x7f97a419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:41.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.094+0000 7f97a259c640 1 -- 172.21.15.117:0/3249132574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97a4103950 msgr2=0x7f97a419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:41.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.094+0000 7f97a259c640 1 --2- 172.21.15.117:0/3249132574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97a4103950 0x7f97a419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:41.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.094+0000 7f97a259c640 1 -- 172.21.15.117:0/3249132574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97a4107690 msgr2=0x7f97a419fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:41.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.094+0000 7f97a259c640 1 --2- 172.21.15.117:0/3249132574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97a4107690 0x7f97a419fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:41.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.094+0000 7f97a259c640 1 -- 172.21.15.117:0/3249132574 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a41a0450 con 0x7f97a4103000 2024-09-21T05:55:41.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.094+0000 7f97a259c640 1 --2- 172.21.15.117:0/3249132574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97a4103000 0x7f97a419a4e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f979400d950 tx=0x7f979400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:41.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.096+0000 7f979b7fe640 1 -- 172.21.15.117:0/3249132574 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9794017070 con 0x7f97a4103000 2024-09-21T05:55:41.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.096+0000 7f979b7fe640 1 -- 172.21.15.117:0/3249132574 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9794009c30 con 0x7f97a4103000 2024-09-21T05:55:41.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.096+0000 7f97a8d83640 1 -- 172.21.15.117:0/3249132574 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97a41a0740 con 0x7f97a4103000 2024-09-21T05:55:41.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.096+0000 7f979b7fe640 1 -- 172.21.15.117:0/3249132574 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979400f620 con 0x7f97a4103000 2024-09-21T05:55:41.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.096+0000 7f97a8d83640 1 -- 172.21.15.117:0/3249132574 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97a41a6ea0 con 0x7f97a4103000 2024-09-21T05:55:41.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.097+0000 7f97a8d83640 1 -- 172.21.15.117:0/3249132574 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97a4103410 con 0x7f97a4103000 2024-09-21T05:55:41.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.098+0000 7f979b7fe640 1 -- 172.21.15.117:0/3249132574 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f9794010460 con 0x7f97a4103000 2024-09-21T05:55:41.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.099+0000 7f979b7fe640 1 --2- 172.21.15.117:0/3249132574 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f97900b65f0 0x7f97900b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:41.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.099+0000 7f979bfff640 1 --2- 172.21.15.117:0/3249132574 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f97900b65f0 0x7f97900b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:41.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.099+0000 7f979b7fe640 1 -- 172.21.15.117:0/3249132574 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f97940e7160 con 0x7f97a4103000 2024-09-21T05:55:41.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.099+0000 7f979bfff640 1 --2- 172.21.15.117:0/3249132574 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f97900b65f0 0x7f97900b8b00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f97a4105070 tx=0x7f978c00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:41.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.103+0000 7f979b7fe640 1 -- 172.21.15.117:0/3249132574 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9794009de0 con 0x7f97a4103000 2024-09-21T05:55:41.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:41 smithi117 ceph-mon[30947]: pgmap v278: 97 pgs: 97 active+clean; 634 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 KiB/s wr, 13 op/s; 83 B/s, 1 objects/s recovering 2024-09-21T05:55:41.279 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.278+0000 7f97a8d83640 1 -- 172.21.15.117:0/3249132574 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f97a410bba0 con 0x7f97900b65f0 2024-09-21T05:55:41.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.282+0000 7f979b7fe640 1 -- 172.21.15.117:0/3249132574 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259066 (secure 0 0 0) 0x7f97a410bba0 con 0x7f97900b65f0 2024-09-21T05:55:41.286 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:55:41.287 INFO:teuthology.orchestra.run.smithi117.stderr:dumped all 2024-09-21T05:55:41.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.286+0000 7f97a8d83640 1 -- 172.21.15.117:0/3249132574 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f97900b65f0 msgr2=0x7f97900b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:41.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.286+0000 7f97a8d83640 1 --2- 172.21.15.117:0/3249132574 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f97900b65f0 0x7f97900b8b00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f97a4105070 tx=0x7f978c00b390 comp rx=0 tx=0).stop 2024-09-21T05:55:41.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.286+0000 7f97a8d83640 1 -- 172.21.15.117:0/3249132574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97a4103000 msgr2=0x7f97a419a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:41.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.286+0000 7f97a8d83640 1 --2- 172.21.15.117:0/3249132574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97a4103000 0x7f97a419a4e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f979400d950 tx=0x7f979400de10 comp rx=0 tx=0).stop 2024-09-21T05:55:41.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.286+0000 7f97a8d83640 1 -- 172.21.15.117:0/3249132574 shutdown_connections 2024-09-21T05:55:41.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.286+0000 7f97a8d83640 1 --2- 172.21.15.117:0/3249132574 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f97900b65f0 0x7f97900b8b00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:41.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.286+0000 7f97a8d83640 1 --2- 172.21.15.117:0/3249132574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97a4107690 0x7f97a419fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:41.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.286+0000 7f97a8d83640 1 --2- 172.21.15.117:0/3249132574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97a4103950 0x7f97a419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:41.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.286+0000 7f97a8d83640 1 --2- 172.21.15.117:0/3249132574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97a4103000 0x7f97a419a4e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:41.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.286+0000 7f97a8d83640 1 -- 172.21.15.117:0/3249132574 >> 172.21.15.117:0/3249132574 conn(0x7f97a4079ea0 msgr2=0x7f97a41054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:41.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.286+0000 7f97a8d83640 1 -- 172.21.15.117:0/3249132574 shutdown_connections 2024-09-21T05:55:41.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:41.286+0000 7f97a8d83640 1 -- 172.21.15.117:0/3249132574 wait complete. 2024-09-21T05:55:41.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:41 smithi138 ceph-mon[34391]: pgmap v278: 97 pgs: 97 active+clean; 634 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 KiB/s wr, 13 op/s; 83 B/s, 1 objects/s recovering 2024-09-21T05:55:41.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:41 smithi192 ceph-mon[34700]: pgmap v278: 97 pgs: 97 active+clean; 634 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 KiB/s wr, 13 op/s; 83 B/s, 1 objects/s recovering 2024-09-21T05:55:42.116 INFO:teuthology.orchestra.run.smithi117.stdout:{"pg_ready":true,"pg_map":{"version":278,"stamp":"2024-09-21T05:55:40.510213+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649137,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":230,"num_read_kb":203,"num_write":358,"num_write_kb":2277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":2952512,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":295,"ondisk_log_size":295,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":292,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":363748,"kb_used_data":10284,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124578076,"statfs":{"total":1151940427776,"available":1151567949824,"internally_reserved":0,"allocated":10530816,"data_stored":6871059,"data_compressed":182193,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":49953,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":83,"num_write":24,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001887"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636773+0000","last_change":"2024-09-21T05:55:14.007642+0000","last_active":"2024-09-21T05:55:33.636773+0000","last_peered":"2024-09-21T05:55:33.636773+0000","last_clean":"2024-09-21T05:55:33.636773+0000","last_became_active":"2024-09-21T05:55:14.006943+0000","last_became_peered":"2024-09-21T05:55:14.006943+0000","last_unstale":"2024-09-21T05:55:33.636773+0000","last_undegraded":"2024-09-21T05:55:33.636773+0000","last_fullsized":"2024-09-21T05:55:33.636773+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:23:53.816079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767047+0000","last_change":"2024-09-21T05:55:14.002034+0000","last_active":"2024-09-21T05:55:33.767047+0000","last_peered":"2024-09-21T05:55:33.767047+0000","last_clean":"2024-09-21T05:55:33.767047+0000","last_became_active":"2024-09-21T05:55:14.001658+0000","last_became_peered":"2024-09-21T05:55:14.001658+0000","last_unstale":"2024-09-21T05:55:33.767047+0000","last_undegraded":"2024-09-21T05:55:33.767047+0000","last_fullsized":"2024-09-21T05:55:33.767047+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:01.568362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.766840+0000","last_change":"2024-09-21T05:55:13.996901+0000","last_active":"2024-09-21T05:55:33.766840+0000","last_peered":"2024-09-21T05:55:33.766840+0000","last_clean":"2024-09-21T05:55:33.766840+0000","last_became_active":"2024-09-21T05:55:13.996020+0000","last_became_peered":"2024-09-21T05:55:13.996020+0000","last_unstale":"2024-09-21T05:55:33.766840+0000","last_undegraded":"2024-09-21T05:55:33.766840+0000","last_fullsized":"2024-09-21T05:55:33.766840+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:38.771926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.696132+0000","last_change":"2024-09-21T05:55:13.995968+0000","last_active":"2024-09-21T05:55:33.696132+0000","last_peered":"2024-09-21T05:55:33.696132+0000","last_clean":"2024-09-21T05:55:33.696132+0000","last_became_active":"2024-09-21T05:55:13.995259+0000","last_became_peered":"2024-09-21T05:55:13.995259+0000","last_unstale":"2024-09-21T05:55:33.696132+0000","last_undegraded":"2024-09-21T05:55:33.696132+0000","last_fullsized":"2024-09-21T05:55:33.696132+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:48.953342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635823+0000","last_change":"2024-09-21T05:55:13.995778+0000","last_active":"2024-09-21T05:55:33.635823+0000","last_peered":"2024-09-21T05:55:33.635823+0000","last_clean":"2024-09-21T05:55:33.635823+0000","last_became_active":"2024-09-21T05:55:13.995415+0000","last_became_peered":"2024-09-21T05:55:13.995415+0000","last_unstale":"2024-09-21T05:55:33.635823+0000","last_undegraded":"2024-09-21T05:55:33.635823+0000","last_fullsized":"2024-09-21T05:55:33.635823+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:52.562120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.634110+0000","last_change":"2024-09-21T05:55:14.000005+0000","last_active":"2024-09-21T05:55:33.634110+0000","last_peered":"2024-09-21T05:55:33.634110+0000","last_clean":"2024-09-21T05:55:33.634110+0000","last_became_active":"2024-09-21T05:55:13.999493+0000","last_became_peered":"2024-09-21T05:55:13.999493+0000","last_unstale":"2024-09-21T05:55:33.634110+0000","last_undegraded":"2024-09-21T05:55:33.634110+0000","last_fullsized":"2024-09-21T05:55:33.634110+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:29.426368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635816+0000","last_change":"2024-09-21T05:55:13.993004+0000","last_active":"2024-09-21T05:55:33.635816+0000","last_peered":"2024-09-21T05:55:33.635816+0000","last_clean":"2024-09-21T05:55:33.635816+0000","last_became_active":"2024-09-21T05:55:13.992615+0000","last_became_peered":"2024-09-21T05:55:13.992615+0000","last_unstale":"2024-09-21T05:55:33.635816+0000","last_undegraded":"2024-09-21T05:55:33.635816+0000","last_fullsized":"2024-09-21T05:55:33.635816+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:48.324618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.769380+0000","last_change":"2024-09-21T05:55:33.769006+0000","last_active":"2024-09-21T05:55:33.769380+0000","last_peered":"2024-09-21T05:55:33.769380+0000","last_clean":"2024-09-21T05:55:33.769380+0000","last_became_active":"2024-09-21T05:55:33.768705+0000","last_became_peered":"2024-09-21T05:55:33.768705+0000","last_unstale":"2024-09-21T05:55:33.769380+0000","last_undegraded":"2024-09-21T05:55:33.769380+0000","last_fullsized":"2024-09-21T05:55:33.769380+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:14.013300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639798+0000","last_change":"2024-09-21T05:55:14.004131+0000","last_active":"2024-09-21T05:55:33.639798+0000","last_peered":"2024-09-21T05:55:33.639798+0000","last_clean":"2024-09-21T05:55:33.639798+0000","last_became_active":"2024-09-21T05:55:14.003621+0000","last_became_peered":"2024-09-21T05:55:14.003621+0000","last_unstale":"2024-09-21T05:55:33.639798+0000","last_undegraded":"2024-09-21T05:55:33.639798+0000","last_fullsized":"2024-09-21T05:55:33.639798+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:00.694211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.643388+0000","last_change":"2024-09-21T05:55:33.643289+0000","last_active":"2024-09-21T05:55:33.643388+0000","last_peered":"2024-09-21T05:55:33.643388+0000","last_clean":"2024-09-21T05:55:33.643388+0000","last_became_active":"2024-09-21T05:55:33.642805+0000","last_became_peered":"2024-09-21T05:55:33.642805+0000","last_unstale":"2024-09-21T05:55:33.643388+0000","last_undegraded":"2024-09-21T05:55:33.643388+0000","last_fullsized":"2024-09-21T05:55:33.643388+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:00.854712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636225+0000","last_change":"2024-09-21T05:55:14.006467+0000","last_active":"2024-09-21T05:55:33.636225+0000","last_peered":"2024-09-21T05:55:33.636225+0000","last_clean":"2024-09-21T05:55:33.636225+0000","last_became_active":"2024-09-21T05:55:14.005978+0000","last_became_peered":"2024-09-21T05:55:14.005978+0000","last_unstale":"2024-09-21T05:55:33.636225+0000","last_undegraded":"2024-09-21T05:55:33.636225+0000","last_fullsized":"2024-09-21T05:55:33.636225+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:37.391152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.696241+0000","last_change":"2024-09-21T05:55:14.003972+0000","last_active":"2024-09-21T05:55:33.696241+0000","last_peered":"2024-09-21T05:55:33.696241+0000","last_clean":"2024-09-21T05:55:33.696241+0000","last_became_active":"2024-09-21T05:55:14.003601+0000","last_became_peered":"2024-09-21T05:55:14.003601+0000","last_unstale":"2024-09-21T05:55:33.696241+0000","last_undegraded":"2024-09-21T05:55:33.696241+0000","last_fullsized":"2024-09-21T05:55:33.696241+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:16.201087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636463+0000","last_change":"2024-09-21T05:55:13.996051+0000","last_active":"2024-09-21T05:55:33.636463+0000","last_peered":"2024-09-21T05:55:33.636463+0000","last_clean":"2024-09-21T05:55:33.636463+0000","last_became_active":"2024-09-21T05:55:13.995679+0000","last_became_peered":"2024-09-21T05:55:13.995679+0000","last_unstale":"2024-09-21T05:55:33.636463+0000","last_undegraded":"2024-09-21T05:55:33.636463+0000","last_fullsized":"2024-09-21T05:55:33.636463+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:45:44.372029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.766681+0000","last_change":"2024-09-21T05:55:13.995421+0000","last_active":"2024-09-21T05:55:33.766681+0000","last_peered":"2024-09-21T05:55:33.766681+0000","last_clean":"2024-09-21T05:55:33.766681+0000","last_became_active":"2024-09-21T05:55:13.995006+0000","last_became_peered":"2024-09-21T05:55:13.995006+0000","last_unstale":"2024-09-21T05:55:33.766681+0000","last_undegraded":"2024-09-21T05:55:33.766681+0000","last_fullsized":"2024-09-21T05:55:33.766681+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:00.636555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.766677+0000","last_change":"2024-09-21T05:55:13.993466+0000","last_active":"2024-09-21T05:55:33.766677+0000","last_peered":"2024-09-21T05:55:33.766677+0000","last_clean":"2024-09-21T05:55:33.766677+0000","last_became_active":"2024-09-21T05:55:13.993137+0000","last_became_peered":"2024-09-21T05:55:13.993137+0000","last_unstale":"2024-09-21T05:55:33.766677+0000","last_undegraded":"2024-09-21T05:55:33.766677+0000","last_fullsized":"2024-09-21T05:55:33.766677+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:33.218868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639111+0000","last_change":"2024-09-21T05:55:13.996316+0000","last_active":"2024-09-21T05:55:33.639111+0000","last_peered":"2024-09-21T05:55:33.639111+0000","last_clean":"2024-09-21T05:55:33.639111+0000","last_became_active":"2024-09-21T05:55:13.995918+0000","last_became_peered":"2024-09-21T05:55:13.995918+0000","last_unstale":"2024-09-21T05:55:33.639111+0000","last_undegraded":"2024-09-21T05:55:33.639111+0000","last_fullsized":"2024-09-21T05:55:33.639111+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:45:54.408504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636962+0000","last_change":"2024-09-21T05:55:14.007198+0000","last_active":"2024-09-21T05:55:33.636962+0000","last_peered":"2024-09-21T05:55:33.636962+0000","last_clean":"2024-09-21T05:55:33.636962+0000","last_became_active":"2024-09-21T05:55:14.006700+0000","last_became_peered":"2024-09-21T05:55:14.006700+0000","last_unstale":"2024-09-21T05:55:33.636962+0000","last_undegraded":"2024-09-21T05:55:33.636962+0000","last_fullsized":"2024-09-21T05:55:33.636962+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:45.343778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.633333+0000","last_change":"2024-09-21T05:55:14.000152+0000","last_active":"2024-09-21T05:55:33.633333+0000","last_peered":"2024-09-21T05:55:33.633333+0000","last_clean":"2024-09-21T05:55:33.633333+0000","last_became_active":"2024-09-21T05:55:13.999786+0000","last_became_peered":"2024-09-21T05:55:13.999786+0000","last_unstale":"2024-09-21T05:55:33.633333+0000","last_undegraded":"2024-09-21T05:55:33.633333+0000","last_fullsized":"2024-09-21T05:55:33.633333+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:43:25.713410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.640106+0000","last_change":"2024-09-21T05:55:13.988367+0000","last_active":"2024-09-21T05:55:33.640106+0000","last_peered":"2024-09-21T05:55:33.640106+0000","last_clean":"2024-09-21T05:55:33.640106+0000","last_became_active":"2024-09-21T05:55:13.987982+0000","last_became_peered":"2024-09-21T05:55:13.987982+0000","last_unstale":"2024-09-21T05:55:33.640106+0000","last_undegraded":"2024-09-21T05:55:33.640106+0000","last_fullsized":"2024-09-21T05:55:33.640106+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:15.395155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636251+0000","last_change":"2024-09-21T05:55:14.004962+0000","last_active":"2024-09-21T05:55:33.636251+0000","last_peered":"2024-09-21T05:55:33.636251+0000","last_clean":"2024-09-21T05:55:33.636251+0000","last_became_active":"2024-09-21T05:55:14.004573+0000","last_became_peered":"2024-09-21T05:55:14.004573+0000","last_unstale":"2024-09-21T05:55:33.636251+0000","last_undegraded":"2024-09-21T05:55:33.636251+0000","last_fullsized":"2024-09-21T05:55:33.636251+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:59.014017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636468+0000","last_change":"2024-09-21T05:55:13.991267+0000","last_active":"2024-09-21T05:55:33.636468+0000","last_peered":"2024-09-21T05:55:33.636468+0000","last_clean":"2024-09-21T05:55:33.636468+0000","last_became_active":"2024-09-21T05:55:13.990916+0000","last_became_peered":"2024-09-21T05:55:13.990916+0000","last_unstale":"2024-09-21T05:55:33.636468+0000","last_undegraded":"2024-09-21T05:55:33.636468+0000","last_fullsized":"2024-09-21T05:55:33.636468+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:06:03.904632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639504+0000","last_change":"2024-09-21T05:55:13.994147+0000","last_active":"2024-09-21T05:55:33.639504+0000","last_peered":"2024-09-21T05:55:33.639504+0000","last_clean":"2024-09-21T05:55:33.639504+0000","last_became_active":"2024-09-21T05:55:13.993794+0000","last_became_peered":"2024-09-21T05:55:13.993794+0000","last_unstale":"2024-09-21T05:55:33.639504+0000","last_undegraded":"2024-09-21T05:55:33.639504+0000","last_fullsized":"2024-09-21T05:55:33.639504+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:19.970853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.638668+0000","last_change":"2024-09-21T05:55:14.001119+0000","last_active":"2024-09-21T05:55:33.638668+0000","last_peered":"2024-09-21T05:55:33.638668+0000","last_clean":"2024-09-21T05:55:33.638668+0000","last_became_active":"2024-09-21T05:55:14.000799+0000","last_became_peered":"2024-09-21T05:55:14.000799+0000","last_unstale":"2024-09-21T05:55:33.638668+0000","last_undegraded":"2024-09-21T05:55:33.638668+0000","last_fullsized":"2024-09-21T05:55:33.638668+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:28.259188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767914+0000","last_change":"2024-09-21T05:55:13.992043+0000","last_active":"2024-09-21T05:55:33.767914+0000","last_peered":"2024-09-21T05:55:33.767914+0000","last_clean":"2024-09-21T05:55:33.767914+0000","last_became_active":"2024-09-21T05:55:13.991344+0000","last_became_peered":"2024-09-21T05:55:13.991344+0000","last_unstale":"2024-09-21T05:55:33.767914+0000","last_undegraded":"2024-09-21T05:55:33.767914+0000","last_fullsized":"2024-09-21T05:55:33.767914+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:42:55.674333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639242+0000","last_change":"2024-09-21T05:55:13.993114+0000","last_active":"2024-09-21T05:55:33.639242+0000","last_peered":"2024-09-21T05:55:33.639242+0000","last_clean":"2024-09-21T05:55:33.639242+0000","last_became_active":"2024-09-21T05:55:13.992749+0000","last_became_peered":"2024-09-21T05:55:13.992749+0000","last_unstale":"2024-09-21T05:55:33.639242+0000","last_undegraded":"2024-09-21T05:55:33.639242+0000","last_fullsized":"2024-09-21T05:55:33.639242+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:15.926967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.640614+0000","last_change":"2024-09-21T05:55:13.997199+0000","last_active":"2024-09-21T05:55:33.640614+0000","last_peered":"2024-09-21T05:55:33.640614+0000","last_clean":"2024-09-21T05:55:33.640614+0000","last_became_active":"2024-09-21T05:55:13.996809+0000","last_became_peered":"2024-09-21T05:55:13.996809+0000","last_unstale":"2024-09-21T05:55:33.640614+0000","last_undegraded":"2024-09-21T05:55:33.640614+0000","last_fullsized":"2024-09-21T05:55:33.640614+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:16.681430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636906+0000","last_change":"2024-09-21T05:55:14.000879+0000","last_active":"2024-09-21T05:55:33.636906+0000","last_peered":"2024-09-21T05:55:33.636906+0000","last_clean":"2024-09-21T05:55:33.636906+0000","last_became_active":"2024-09-21T05:55:14.000259+0000","last_became_peered":"2024-09-21T05:55:14.000259+0000","last_unstale":"2024-09-21T05:55:33.636906+0000","last_undegraded":"2024-09-21T05:55:33.636906+0000","last_fullsized":"2024-09-21T05:55:33.636906+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:01:08.938417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636509+0000","last_change":"2024-09-21T05:55:13.989433+0000","last_active":"2024-09-21T05:55:33.636509+0000","last_peered":"2024-09-21T05:55:33.636509+0000","last_clean":"2024-09-21T05:55:33.636509+0000","last_became_active":"2024-09-21T05:55:13.989097+0000","last_became_peered":"2024-09-21T05:55:13.989097+0000","last_unstale":"2024-09-21T05:55:33.636509+0000","last_undegraded":"2024-09-21T05:55:33.636509+0000","last_fullsized":"2024-09-21T05:55:33.636509+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:37.835298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767185+0000","last_change":"2024-09-21T05:55:13.993911+0000","last_active":"2024-09-21T05:55:33.767185+0000","last_peered":"2024-09-21T05:55:33.767185+0000","last_clean":"2024-09-21T05:55:33.767185+0000","last_became_active":"2024-09-21T05:55:13.993570+0000","last_became_peered":"2024-09-21T05:55:13.993570+0000","last_unstale":"2024-09-21T05:55:33.767185+0000","last_undegraded":"2024-09-21T05:55:33.767185+0000","last_fullsized":"2024-09-21T05:55:33.767185+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:08:16.588457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639566+0000","last_change":"2024-09-21T05:55:13.992764+0000","last_active":"2024-09-21T05:55:33.639566+0000","last_peered":"2024-09-21T05:55:33.639566+0000","last_clean":"2024-09-21T05:55:33.639566+0000","last_became_active":"2024-09-21T05:55:13.992338+0000","last_became_peered":"2024-09-21T05:55:13.992338+0000","last_unstale":"2024-09-21T05:55:33.639566+0000","last_undegraded":"2024-09-21T05:55:33.639566+0000","last_fullsized":"2024-09-21T05:55:33.639566+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:25:29.997353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639811+0000","last_change":"2024-09-21T05:55:13.987119+0000","last_active":"2024-09-21T05:55:33.639811+0000","last_peered":"2024-09-21T05:55:33.639811+0000","last_clean":"2024-09-21T05:55:33.639811+0000","last_became_active":"2024-09-21T05:55:13.986663+0000","last_became_peered":"2024-09-21T05:55:13.986663+0000","last_unstale":"2024-09-21T05:55:33.639811+0000","last_undegraded":"2024-09-21T05:55:33.639811+0000","last_fullsized":"2024-09-21T05:55:33.639811+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:02.089833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.637197+0000","last_change":"2024-09-21T05:55:14.003577+0000","last_active":"2024-09-21T05:55:33.637197+0000","last_peered":"2024-09-21T05:55:33.637197+0000","last_clean":"2024-09-21T05:55:33.637197+0000","last_became_active":"2024-09-21T05:55:14.003162+0000","last_became_peered":"2024-09-21T05:55:14.003162+0000","last_unstale":"2024-09-21T05:55:33.637197+0000","last_undegraded":"2024-09-21T05:55:33.637197+0000","last_fullsized":"2024-09-21T05:55:33.637197+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:58.630843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635758+0000","last_change":"2024-09-21T05:55:13.992078+0000","last_active":"2024-09-21T05:55:33.635758+0000","last_peered":"2024-09-21T05:55:33.635758+0000","last_clean":"2024-09-21T05:55:33.635758+0000","last_became_active":"2024-09-21T05:55:13.991680+0000","last_became_peered":"2024-09-21T05:55:13.991680+0000","last_unstale":"2024-09-21T05:55:33.635758+0000","last_undegraded":"2024-09-21T05:55:33.635758+0000","last_fullsized":"2024-09-21T05:55:33.635758+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:52.886755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"74'6","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:38.241965+0000","last_change":"2024-09-21T05:55:11.982560+0000","last_active":"2024-09-21T05:55:38.241965+0000","last_peered":"2024-09-21T05:55:38.241965+0000","last_clean":"2024-09-21T05:55:38.241965+0000","last_became_active":"2024-09-21T05:55:11.982179+0000","last_became_peered":"2024-09-21T05:55:11.982179+0000","last_unstale":"2024-09-21T05:55:38.241965+0000","last_undegraded":"2024-09-21T05:55:38.241965+0000","last_fullsized":"2024-09-21T05:55:38.241965+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:06.012194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767640+0000","last_change":"2024-09-21T05:55:13.993994+0000","last_active":"2024-09-21T05:55:33.767640+0000","last_peered":"2024-09-21T05:55:33.767640+0000","last_clean":"2024-09-21T05:55:33.767640+0000","last_became_active":"2024-09-21T05:55:13.993585+0000","last_became_peered":"2024-09-21T05:55:13.993585+0000","last_unstale":"2024-09-21T05:55:33.767640+0000","last_undegraded":"2024-09-21T05:55:33.767640+0000","last_fullsized":"2024-09-21T05:55:33.767640+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:11.608616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"74'7","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.634233+0000","last_change":"2024-09-21T05:55:11.984397+0000","last_active":"2024-09-21T05:55:33.634233+0000","last_peered":"2024-09-21T05:55:33.634233+0000","last_clean":"2024-09-21T05:55:33.634233+0000","last_became_active":"2024-09-21T05:55:11.983888+0000","last_became_peered":"2024-09-21T05:55:11.983888+0000","last_unstale":"2024-09-21T05:55:33.634233+0000","last_undegraded":"2024-09-21T05:55:33.634233+0000","last_fullsized":"2024-09-21T05:55:33.634233+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:59.012710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636187+0000","last_change":"2024-09-21T05:55:11.974547+0000","last_active":"2024-09-21T05:55:33.636187+0000","last_peered":"2024-09-21T05:55:33.636187+0000","last_clean":"2024-09-21T05:55:33.636187+0000","last_became_active":"2024-09-21T05:55:11.974151+0000","last_became_peered":"2024-09-21T05:55:11.974151+0000","last_unstale":"2024-09-21T05:55:33.636187+0000","last_undegraded":"2024-09-21T05:55:33.636187+0000","last_fullsized":"2024-09-21T05:55:33.636187+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:51.489548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.638587+0000","last_change":"2024-09-21T05:55:13.999287+0000","last_active":"2024-09-21T05:55:33.638587+0000","last_peered":"2024-09-21T05:55:33.638587+0000","last_clean":"2024-09-21T05:55:33.638587+0000","last_became_active":"2024-09-21T05:55:13.998742+0000","last_became_peered":"2024-09-21T05:55:13.998742+0000","last_unstale":"2024-09-21T05:55:33.638587+0000","last_undegraded":"2024-09-21T05:55:33.638587+0000","last_fullsized":"2024-09-21T05:55:33.638587+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:33.842810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"74'3","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.768113+0000","last_change":"2024-09-21T05:55:11.981354+0000","last_active":"2024-09-21T05:55:33.768113+0000","last_peered":"2024-09-21T05:55:33.768113+0000","last_clean":"2024-09-21T05:55:33.768113+0000","last_became_active":"2024-09-21T05:55:11.980993+0000","last_became_peered":"2024-09-21T05:55:11.980993+0000","last_unstale":"2024-09-21T05:55:33.768113+0000","last_undegraded":"2024-09-21T05:55:33.768113+0000","last_fullsized":"2024-09-21T05:55:33.768113+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:05.037669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.633042+0000","last_change":"2024-09-21T05:55:14.001041+0000","last_active":"2024-09-21T05:55:33.633042+0000","last_peered":"2024-09-21T05:55:33.633042+0000","last_clean":"2024-09-21T05:55:33.633042+0000","last_became_active":"2024-09-21T05:55:14.000688+0000","last_became_peered":"2024-09-21T05:55:14.000688+0000","last_unstale":"2024-09-21T05:55:33.633042+0000","last_undegraded":"2024-09-21T05:55:33.633042+0000","last_fullsized":"2024-09-21T05:55:33.633042+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:51.190495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"74'4","reported_seq":51,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:37.239608+0000","last_change":"2024-09-21T05:55:11.984097+0000","last_active":"2024-09-21T05:55:37.239608+0000","last_peered":"2024-09-21T05:55:37.239608+0000","last_clean":"2024-09-21T05:55:37.239608+0000","last_became_active":"2024-09-21T05:55:11.982969+0000","last_became_peered":"2024-09-21T05:55:11.982969+0000","last_unstale":"2024-09-21T05:55:37.239608+0000","last_undegraded":"2024-09-21T05:55:37.239608+0000","last_fullsized":"2024-09-21T05:55:37.239608+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:41.372225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639897+0000","last_change":"2024-09-21T05:55:13.998433+0000","last_active":"2024-09-21T05:55:33.639897+0000","last_peered":"2024-09-21T05:55:33.639897+0000","last_clean":"2024-09-21T05:55:33.639897+0000","last_became_active":"2024-09-21T05:55:13.997890+0000","last_became_peered":"2024-09-21T05:55:13.997890+0000","last_unstale":"2024-09-21T05:55:33.639897+0000","last_undegraded":"2024-09-21T05:55:33.639897+0000","last_fullsized":"2024-09-21T05:55:33.639897+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:24.396789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"74'4","reported_seq":40,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.701709+0000","last_change":"2024-09-21T05:55:33.701599+0000","last_active":"2024-09-21T05:55:33.701709+0000","last_peered":"2024-09-21T05:55:33.701709+0000","last_clean":"2024-09-21T05:55:33.701709+0000","last_became_active":"2024-09-21T05:55:33.697385+0000","last_became_peered":"2024-09-21T05:55:33.697385+0000","last_unstale":"2024-09-21T05:55:33.701709+0000","last_undegraded":"2024-09-21T05:55:33.701709+0000","last_fullsized":"2024-09-21T05:55:33.701709+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:18.653991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767050+0000","last_change":"2024-09-21T05:55:13.995111+0000","last_active":"2024-09-21T05:55:33.767050+0000","last_peered":"2024-09-21T05:55:33.767050+0000","last_clean":"2024-09-21T05:55:33.767050+0000","last_became_active":"2024-09-21T05:55:13.994691+0000","last_became_peered":"2024-09-21T05:55:13.994691+0000","last_unstale":"2024-09-21T05:55:33.767050+0000","last_undegraded":"2024-09-21T05:55:33.767050+0000","last_fullsized":"2024-09-21T05:55:33.767050+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:30.167649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"74'5","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767919+0000","last_change":"2024-09-21T05:55:12.013961+0000","last_active":"2024-09-21T05:55:33.767919+0000","last_peered":"2024-09-21T05:55:33.767919+0000","last_clean":"2024-09-21T05:55:33.767919+0000","last_became_active":"2024-09-21T05:55:12.013573+0000","last_became_peered":"2024-09-21T05:55:12.013573+0000","last_unstale":"2024-09-21T05:55:33.767919+0000","last_undegraded":"2024-09-21T05:55:33.767919+0000","last_fullsized":"2024-09-21T05:55:33.767919+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:12.119605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639617+0000","last_change":"2024-09-21T05:55:13.996093+0000","last_active":"2024-09-21T05:55:33.639617+0000","last_peered":"2024-09-21T05:55:33.639617+0000","last_clean":"2024-09-21T05:55:33.639617+0000","last_became_active":"2024-09-21T05:55:13.995342+0000","last_became_peered":"2024-09-21T05:55:13.995342+0000","last_unstale":"2024-09-21T05:55:33.639617+0000","last_undegraded":"2024-09-21T05:55:33.639617+0000","last_fullsized":"2024-09-21T05:55:33.639617+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:20.335330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"76'24","reported_seq":77,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:37.242700+0000","last_change":"2024-09-21T05:55:11.986193+0000","last_active":"2024-09-21T05:55:37.242700+0000","last_peered":"2024-09-21T05:55:37.242700+0000","last_clean":"2024-09-21T05:55:37.242700+0000","last_became_active":"2024-09-21T05:55:11.985736+0000","last_became_peered":"2024-09-21T05:55:11.985736+0000","last_unstale":"2024-09-21T05:55:37.242700+0000","last_undegraded":"2024-09-21T05:55:37.242700+0000","last_fullsized":"2024-09-21T05:55:37.242700+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.503419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50940,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":30,"num_read_kb":64,"num_write":26,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636170+0000","last_change":"2024-09-21T05:55:14.001123+0000","last_active":"2024-09-21T05:55:33.636170+0000","last_peered":"2024-09-21T05:55:33.636170+0000","last_clean":"2024-09-21T05:55:33.636170+0000","last_became_active":"2024-09-21T05:55:14.000742+0000","last_became_peered":"2024-09-21T05:55:14.000742+0000","last_unstale":"2024-09-21T05:55:33.636170+0000","last_undegraded":"2024-09-21T05:55:33.636170+0000","last_fullsized":"2024-09-21T05:55:33.636170+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:15.277992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"74'9","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639957+0000","last_change":"2024-09-21T05:55:11.981396+0000","last_active":"2024-09-21T05:55:33.639957+0000","last_peered":"2024-09-21T05:55:33.639957+0000","last_clean":"2024-09-21T05:55:33.639957+0000","last_became_active":"2024-09-21T05:55:11.980976+0000","last_became_peered":"2024-09-21T05:55:11.980976+0000","last_unstale":"2024-09-21T05:55:33.639957+0000","last_undegraded":"2024-09-21T05:55:33.639957+0000","last_fullsized":"2024-09-21T05:55:33.639957+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:43.600453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636282+0000","last_change":"2024-09-21T05:55:13.991354+0000","last_active":"2024-09-21T05:55:33.636282+0000","last_peered":"2024-09-21T05:55:33.636282+0000","last_clean":"2024-09-21T05:55:33.636282+0000","last_became_active":"2024-09-21T05:55:13.990967+0000","last_became_peered":"2024-09-21T05:55:13.990967+0000","last_unstale":"2024-09-21T05:55:33.636282+0000","last_undegraded":"2024-09-21T05:55:33.636282+0000","last_fullsized":"2024-09-21T05:55:33.636282+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:04.889461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"74'5","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767638+0000","last_change":"2024-09-21T05:55:11.981019+0000","last_active":"2024-09-21T05:55:33.767638+0000","last_peered":"2024-09-21T05:55:33.767638+0000","last_clean":"2024-09-21T05:55:33.767638+0000","last_became_active":"2024-09-21T05:55:11.980619+0000","last_became_peered":"2024-09-21T05:55:11.980619+0000","last_unstale":"2024-09-21T05:55:33.767638+0000","last_undegraded":"2024-09-21T05:55:33.767638+0000","last_fullsized":"2024-09-21T05:55:33.767638+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:41.005092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635917+0000","last_change":"2024-09-21T05:55:13.994535+0000","last_active":"2024-09-21T05:55:33.635917+0000","last_peered":"2024-09-21T05:55:33.635917+0000","last_clean":"2024-09-21T05:55:33.635917+0000","last_became_active":"2024-09-21T05:55:13.994153+0000","last_became_peered":"2024-09-21T05:55:13.994153+0000","last_unstale":"2024-09-21T05:55:33.635917+0000","last_undegraded":"2024-09-21T05:55:33.635917+0000","last_fullsized":"2024-09-21T05:55:33.635917+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:02.301263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"74'6","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:38.241026+0000","last_change":"2024-09-21T05:55:11.977694+0000","last_active":"2024-09-21T05:55:38.241026+0000","last_peered":"2024-09-21T05:55:38.241026+0000","last_clean":"2024-09-21T05:55:38.241026+0000","last_became_active":"2024-09-21T05:55:11.977146+0000","last_became_peered":"2024-09-21T05:55:11.977146+0000","last_unstale":"2024-09-21T05:55:38.241026+0000","last_undegraded":"2024-09-21T05:55:38.241026+0000","last_fullsized":"2024-09-21T05:55:38.241026+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:29.736564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635721+0000","last_change":"2024-09-21T05:55:13.998032+0000","last_active":"2024-09-21T05:55:33.635721+0000","last_peered":"2024-09-21T05:55:33.635721+0000","last_clean":"2024-09-21T05:55:33.635721+0000","last_became_active":"2024-09-21T05:55:13.997617+0000","last_became_peered":"2024-09-21T05:55:13.997617+0000","last_unstale":"2024-09-21T05:55:33.635721+0000","last_undegraded":"2024-09-21T05:55:33.635721+0000","last_fullsized":"2024-09-21T05:55:33.635721+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:46.552726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"74'17","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635659+0000","last_change":"2024-09-21T05:55:12.013540+0000","last_active":"2024-09-21T05:55:33.635659+0000","last_peered":"2024-09-21T05:55:33.635659+0000","last_clean":"2024-09-21T05:55:33.635659+0000","last_became_active":"2024-09-21T05:55:12.013151+0000","last_became_peered":"2024-09-21T05:55:12.013151+0000","last_unstale":"2024-09-21T05:55:33.635659+0000","last_undegraded":"2024-09-21T05:55:33.635659+0000","last_fullsized":"2024-09-21T05:55:33.635659+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:53.705904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767640+0000","last_change":"2024-09-21T05:55:13.999501+0000","last_active":"2024-09-21T05:55:33.767640+0000","last_peered":"2024-09-21T05:55:33.767640+0000","last_clean":"2024-09-21T05:55:33.767640+0000","last_became_active":"2024-09-21T05:55:13.997819+0000","last_became_peered":"2024-09-21T05:55:13.997819+0000","last_unstale":"2024-09-21T05:55:33.767640+0000","last_undegraded":"2024-09-21T05:55:33.767640+0000","last_fullsized":"2024-09-21T05:55:33.767640+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:54.334755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"74'2","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636760+0000","last_change":"2024-09-21T05:55:11.984428+0000","last_active":"2024-09-21T05:55:33.636760+0000","last_peered":"2024-09-21T05:55:33.636760+0000","last_clean":"2024-09-21T05:55:33.636760+0000","last_became_active":"2024-09-21T05:55:11.983543+0000","last_became_peered":"2024-09-21T05:55:11.983543+0000","last_unstale":"2024-09-21T05:55:33.636760+0000","last_undegraded":"2024-09-21T05:55:33.636760+0000","last_fullsized":"2024-09-21T05:55:33.636760+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:39.636408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.637304+0000","last_change":"2024-09-21T05:55:14.003851+0000","last_active":"2024-09-21T05:55:33.637304+0000","last_peered":"2024-09-21T05:55:33.637304+0000","last_clean":"2024-09-21T05:55:33.637304+0000","last_became_active":"2024-09-21T05:55:14.003354+0000","last_became_peered":"2024-09-21T05:55:14.003354+0000","last_unstale":"2024-09-21T05:55:33.637304+0000","last_undegraded":"2024-09-21T05:55:33.637304+0000","last_fullsized":"2024-09-21T05:55:33.637304+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:01.321612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"74'5","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.645976+0000","last_change":"2024-09-21T05:55:33.645825+0000","last_active":"2024-09-21T05:55:33.645976+0000","last_peered":"2024-09-21T05:55:33.645976+0000","last_clean":"2024-09-21T05:55:33.645976+0000","last_became_active":"2024-09-21T05:55:33.640707+0000","last_became_peered":"2024-09-21T05:55:33.640707+0000","last_unstale":"2024-09-21T05:55:33.645976+0000","last_undegraded":"2024-09-21T05:55:33.645976+0000","last_fullsized":"2024-09-21T05:55:33.645976+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:33.665707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636452+0000","last_change":"2024-09-21T05:55:14.001855+0000","last_active":"2024-09-21T05:55:33.636452+0000","last_peered":"2024-09-21T05:55:33.636452+0000","last_clean":"2024-09-21T05:55:33.636452+0000","last_became_active":"2024-09-21T05:55:14.001484+0000","last_became_peered":"2024-09-21T05:55:14.001484+0000","last_unstale":"2024-09-21T05:55:33.636452+0000","last_undegraded":"2024-09-21T05:55:33.636452+0000","last_fullsized":"2024-09-21T05:55:33.636452+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:57.615334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'3","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767928+0000","last_change":"2024-09-21T05:55:11.979716+0000","last_active":"2024-09-21T05:55:33.767928+0000","last_peered":"2024-09-21T05:55:33.767928+0000","last_clean":"2024-09-21T05:55:33.767928+0000","last_became_active":"2024-09-21T05:55:11.979307+0000","last_became_peered":"2024-09-21T05:55:11.979307+0000","last_unstale":"2024-09-21T05:55:33.767928+0000","last_undegraded":"2024-09-21T05:55:33.767928+0000","last_fullsized":"2024-09-21T05:55:33.767928+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:30.019040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636060+0000","last_change":"2024-09-21T05:55:13.991769+0000","last_active":"2024-09-21T05:55:33.636060+0000","last_peered":"2024-09-21T05:55:33.636060+0000","last_clean":"2024-09-21T05:55:33.636060+0000","last_became_active":"2024-09-21T05:55:13.991345+0000","last_became_peered":"2024-09-21T05:55:13.991345+0000","last_unstale":"2024-09-21T05:55:33.636060+0000","last_undegraded":"2024-09-21T05:55:33.636060+0000","last_fullsized":"2024-09-21T05:55:33.636060+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:52.227126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"74'6","reported_seq":44,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:36.238721+0000","last_change":"2024-09-21T05:55:12.013836+0000","last_active":"2024-09-21T05:55:36.238721+0000","last_peered":"2024-09-21T05:55:36.238721+0000","last_clean":"2024-09-21T05:55:36.238721+0000","last_became_active":"2024-09-21T05:55:12.013436+0000","last_became_peered":"2024-09-21T05:55:12.013436+0000","last_unstale":"2024-09-21T05:55:36.238721+0000","last_undegraded":"2024-09-21T05:55:36.238721+0000","last_fullsized":"2024-09-21T05:55:36.238721+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:40:02.767620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636197+0000","last_change":"2024-09-21T05:55:14.001156+0000","last_active":"2024-09-21T05:55:33.636197+0000","last_peered":"2024-09-21T05:55:33.636197+0000","last_clean":"2024-09-21T05:55:33.636197+0000","last_became_active":"2024-09-21T05:55:14.000777+0000","last_became_peered":"2024-09-21T05:55:14.000777+0000","last_unstale":"2024-09-21T05:55:33.636197+0000","last_undegraded":"2024-09-21T05:55:33.636197+0000","last_fullsized":"2024-09-21T05:55:33.636197+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:46:08.000612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":97,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636093+0000","last_change":"2024-09-21T05:53:58.971355+0000","last_active":"2024-09-21T05:55:33.636093+0000","last_peered":"2024-09-21T05:55:33.636093+0000","last_clean":"2024-09-21T05:55:33.636093+0000","last_became_active":"2024-09-21T05:53:58.965825+0000","last_became_peered":"2024-09-21T05:53:58.965825+0000","last_unstale":"2024-09-21T05:55:33.636093+0000","last_undegraded":"2024-09-21T05:55:33.636093+0000","last_fullsized":"2024-09-21T05:55:33.636093+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:50:33.190160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:50:33.190160+0000","last_clean_scrub_stamp":"2024-09-21T05:50:33.190160+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:31.387339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"74'6","reported_seq":32,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.696020+0000","last_change":"2024-09-21T05:55:11.976092+0000","last_active":"2024-09-21T05:55:33.696020+0000","last_peered":"2024-09-21T05:55:33.696020+0000","last_clean":"2024-09-21T05:55:33.696020+0000","last_became_active":"2024-09-21T05:55:11.975657+0000","last_became_peered":"2024-09-21T05:55:11.975657+0000","last_unstale":"2024-09-21T05:55:33.696020+0000","last_undegraded":"2024-09-21T05:55:33.696020+0000","last_fullsized":"2024-09-21T05:55:33.696020+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:46.792054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639547+0000","last_change":"2024-09-21T05:55:13.997621+0000","last_active":"2024-09-21T05:55:33.639547+0000","last_peered":"2024-09-21T05:55:33.639547+0000","last_clean":"2024-09-21T05:55:33.639547+0000","last_became_active":"2024-09-21T05:55:13.997127+0000","last_became_peered":"2024-09-21T05:55:13.997127+0000","last_unstale":"2024-09-21T05:55:33.639547+0000","last_undegraded":"2024-09-21T05:55:33.639547+0000","last_fullsized":"2024-09-21T05:55:33.639547+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:32.399696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"74'4","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635928+0000","last_change":"2024-09-21T05:55:11.982387+0000","last_active":"2024-09-21T05:55:33.635928+0000","last_peered":"2024-09-21T05:55:33.635928+0000","last_clean":"2024-09-21T05:55:33.635928+0000","last_became_active":"2024-09-21T05:55:11.981851+0000","last_became_peered":"2024-09-21T05:55:11.981851+0000","last_unstale":"2024-09-21T05:55:33.635928+0000","last_undegraded":"2024-09-21T05:55:33.635928+0000","last_fullsized":"2024-09-21T05:55:33.635928+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:45.345738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636236+0000","last_change":"2024-09-21T05:55:13.981065+0000","last_active":"2024-09-21T05:55:33.636236+0000","last_peered":"2024-09-21T05:55:33.636236+0000","last_clean":"2024-09-21T05:55:33.636236+0000","last_became_active":"2024-09-21T05:55:13.980524+0000","last_became_peered":"2024-09-21T05:55:13.980524+0000","last_unstale":"2024-09-21T05:55:33.636236+0000","last_undegraded":"2024-09-21T05:55:33.636236+0000","last_fullsized":"2024-09-21T05:55:33.636236+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:43.322429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"74'5","reported_seq":39,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:34.234599+0000","last_change":"2024-09-21T05:55:12.013434+0000","last_active":"2024-09-21T05:55:34.234599+0000","last_peered":"2024-09-21T05:55:34.234599+0000","last_clean":"2024-09-21T05:55:34.234599+0000","last_became_active":"2024-09-21T05:55:12.013087+0000","last_became_peered":"2024-09-21T05:55:12.013087+0000","last_unstale":"2024-09-21T05:55:34.234599+0000","last_undegraded":"2024-09-21T05:55:34.234599+0000","last_fullsized":"2024-09-21T05:55:34.234599+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:19.876288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767927+0000","last_change":"2024-09-21T05:55:13.996102+0000","last_active":"2024-09-21T05:55:33.767927+0000","last_peered":"2024-09-21T05:55:33.767927+0000","last_clean":"2024-09-21T05:55:33.767927+0000","last_became_active":"2024-09-21T05:55:13.994900+0000","last_became_peered":"2024-09-21T05:55:13.994900+0000","last_unstale":"2024-09-21T05:55:33.767927+0000","last_undegraded":"2024-09-21T05:55:33.767927+0000","last_fullsized":"2024-09-21T05:55:33.767927+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:04.998577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"74'5","reported_seq":38,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.640273+0000","last_change":"2024-09-21T05:55:11.981184+0000","last_active":"2024-09-21T05:55:33.640273+0000","last_peered":"2024-09-21T05:55:33.640273+0000","last_clean":"2024-09-21T05:55:33.640273+0000","last_became_active":"2024-09-21T05:55:11.979581+0000","last_became_peered":"2024-09-21T05:55:11.979581+0000","last_unstale":"2024-09-21T05:55:33.640273+0000","last_undegraded":"2024-09-21T05:55:33.640273+0000","last_fullsized":"2024-09-21T05:55:33.640273+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:53.833860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.768241+0000","last_change":"2024-09-21T05:55:33.768136+0000","last_active":"2024-09-21T05:55:33.768241+0000","last_peered":"2024-09-21T05:55:33.768241+0000","last_clean":"2024-09-21T05:55:33.768241+0000","last_became_active":"2024-09-21T05:55:33.767697+0000","last_became_peered":"2024-09-21T05:55:33.767697+0000","last_unstale":"2024-09-21T05:55:33.768241+0000","last_undegraded":"2024-09-21T05:55:33.768241+0000","last_fullsized":"2024-09-21T05:55:33.768241+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:48.146453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"74'1","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.633171+0000","last_change":"2024-09-21T05:55:12.013121+0000","last_active":"2024-09-21T05:55:33.633171+0000","last_peered":"2024-09-21T05:55:33.633171+0000","last_clean":"2024-09-21T05:55:33.633171+0000","last_became_active":"2024-09-21T05:55:12.012657+0000","last_became_peered":"2024-09-21T05:55:12.012657+0000","last_unstale":"2024-09-21T05:55:33.633171+0000","last_undegraded":"2024-09-21T05:55:33.633171+0000","last_fullsized":"2024-09-21T05:55:33.633171+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:25:48.002938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"74'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.634506+0000","last_change":"2024-09-21T05:55:14.004660+0000","last_active":"2024-09-21T05:55:33.634506+0000","last_peered":"2024-09-21T05:55:33.634506+0000","last_clean":"2024-09-21T05:55:33.634506+0000","last_became_active":"2024-09-21T05:55:14.004297+0000","last_became_peered":"2024-09-21T05:55:14.004297+0000","last_unstale":"2024-09-21T05:55:33.634506+0000","last_undegraded":"2024-09-21T05:55:33.634506+0000","last_fullsized":"2024-09-21T05:55:33.634506+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:24.399891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"74'4","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636395+0000","last_change":"2024-09-21T05:55:12.015717+0000","last_active":"2024-09-21T05:55:33.636395+0000","last_peered":"2024-09-21T05:55:33.636395+0000","last_clean":"2024-09-21T05:55:33.636395+0000","last_became_active":"2024-09-21T05:55:12.015419+0000","last_became_peered":"2024-09-21T05:55:12.015419+0000","last_unstale":"2024-09-21T05:55:33.636395+0000","last_undegraded":"2024-09-21T05:55:33.636395+0000","last_fullsized":"2024-09-21T05:55:33.636395+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:57.988860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636635+0000","last_change":"2024-09-21T05:55:13.992460+0000","last_active":"2024-09-21T05:55:33.636635+0000","last_peered":"2024-09-21T05:55:33.636635+0000","last_clean":"2024-09-21T05:55:33.636635+0000","last_became_active":"2024-09-21T05:55:13.991887+0000","last_became_peered":"2024-09-21T05:55:13.991887+0000","last_unstale":"2024-09-21T05:55:33.636635+0000","last_undegraded":"2024-09-21T05:55:33.636635+0000","last_fullsized":"2024-09-21T05:55:33.636635+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:37.718050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"76'12","reported_seq":35,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.994291+0000","last_change":"2024-09-21T05:55:11.982235+0000","last_active":"2024-09-21T05:55:33.994291+0000","last_peered":"2024-09-21T05:55:33.994291+0000","last_clean":"2024-09-21T05:55:33.994291+0000","last_became_active":"2024-09-21T05:55:11.981846+0000","last_became_peered":"2024-09-21T05:55:11.981846+0000","last_unstale":"2024-09-21T05:55:33.994291+0000","last_undegraded":"2024-09-21T05:55:33.994291+0000","last_fullsized":"2024-09-21T05:55:33.994291+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:40:04.330346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639253+0000","last_change":"2024-09-21T05:55:13.999249+0000","last_active":"2024-09-21T05:55:33.639253+0000","last_peered":"2024-09-21T05:55:33.639253+0000","last_clean":"2024-09-21T05:55:33.639253+0000","last_became_active":"2024-09-21T05:55:13.998452+0000","last_became_peered":"2024-09-21T05:55:13.998452+0000","last_unstale":"2024-09-21T05:55:33.639253+0000","last_undegraded":"2024-09-21T05:55:33.639253+0000","last_fullsized":"2024-09-21T05:55:33.639253+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:59.364684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"74'7","reported_seq":31,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.696351+0000","last_change":"2024-09-21T05:55:11.971850+0000","last_active":"2024-09-21T05:55:33.696351+0000","last_peered":"2024-09-21T05:55:33.696351+0000","last_clean":"2024-09-21T05:55:33.696351+0000","last_became_active":"2024-09-21T05:55:11.971415+0000","last_became_peered":"2024-09-21T05:55:11.971415+0000","last_unstale":"2024-09-21T05:55:33.696351+0000","last_undegraded":"2024-09-21T05:55:33.696351+0000","last_fullsized":"2024-09-21T05:55:33.696351+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:51.990745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.766931+0000","last_change":"2024-09-21T05:55:13.994500+0000","last_active":"2024-09-21T05:55:33.766931+0000","last_peered":"2024-09-21T05:55:33.766931+0000","last_clean":"2024-09-21T05:55:33.766931+0000","last_became_active":"2024-09-21T05:55:13.994146+0000","last_became_peered":"2024-09-21T05:55:13.994146+0000","last_unstale":"2024-09-21T05:55:33.766931+0000","last_undegraded":"2024-09-21T05:55:33.766931+0000","last_fullsized":"2024-09-21T05:55:33.766931+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:42.934234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639268+0000","last_change":"2024-09-21T05:55:12.013479+0000","last_active":"2024-09-21T05:55:33.639268+0000","last_peered":"2024-09-21T05:55:33.639268+0000","last_clean":"2024-09-21T05:55:33.639268+0000","last_became_active":"2024-09-21T05:55:12.013180+0000","last_became_peered":"2024-09-21T05:55:12.013180+0000","last_unstale":"2024-09-21T05:55:33.639268+0000","last_undegraded":"2024-09-21T05:55:33.639268+0000","last_fullsized":"2024-09-21T05:55:33.639268+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:28.800072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635548+0000","last_change":"2024-09-21T05:55:14.001795+0000","last_active":"2024-09-21T05:55:33.635548+0000","last_peered":"2024-09-21T05:55:33.635548+0000","last_clean":"2024-09-21T05:55:33.635548+0000","last_became_active":"2024-09-21T05:55:14.001430+0000","last_became_peered":"2024-09-21T05:55:14.001430+0000","last_unstale":"2024-09-21T05:55:33.635548+0000","last_undegraded":"2024-09-21T05:55:33.635548+0000","last_fullsized":"2024-09-21T05:55:33.635548+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:10.339939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"74'7","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.638793+0000","last_change":"2024-09-21T05:55:11.976894+0000","last_active":"2024-09-21T05:55:33.638793+0000","last_peered":"2024-09-21T05:55:33.638793+0000","last_clean":"2024-09-21T05:55:33.638793+0000","last_became_active":"2024-09-21T05:55:11.976516+0000","last_became_peered":"2024-09-21T05:55:11.976516+0000","last_unstale":"2024-09-21T05:55:33.638793+0000","last_undegraded":"2024-09-21T05:55:33.638793+0000","last_fullsized":"2024-09-21T05:55:33.638793+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:58.726311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.637297+0000","last_change":"2024-09-21T05:55:14.003760+0000","last_active":"2024-09-21T05:55:33.637297+0000","last_peered":"2024-09-21T05:55:33.637297+0000","last_clean":"2024-09-21T05:55:33.637297+0000","last_became_active":"2024-09-21T05:55:14.003180+0000","last_became_peered":"2024-09-21T05:55:14.003180+0000","last_unstale":"2024-09-21T05:55:33.637297+0000","last_undegraded":"2024-09-21T05:55:33.637297+0000","last_fullsized":"2024-09-21T05:55:33.637297+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:25.077765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639713+0000","last_change":"2024-09-21T05:55:11.983865+0000","last_active":"2024-09-21T05:55:33.639713+0000","last_peered":"2024-09-21T05:55:33.639713+0000","last_clean":"2024-09-21T05:55:33.639713+0000","last_became_active":"2024-09-21T05:55:11.983473+0000","last_became_peered":"2024-09-21T05:55:11.983473+0000","last_unstale":"2024-09-21T05:55:33.639713+0000","last_undegraded":"2024-09-21T05:55:33.639713+0000","last_fullsized":"2024-09-21T05:55:33.639713+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:18.767317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639006+0000","last_change":"2024-09-21T05:55:13.999001+0000","last_active":"2024-09-21T05:55:33.639006+0000","last_peered":"2024-09-21T05:55:33.639006+0000","last_clean":"2024-09-21T05:55:33.639006+0000","last_became_active":"2024-09-21T05:55:13.998656+0000","last_became_peered":"2024-09-21T05:55:13.998656+0000","last_unstale":"2024-09-21T05:55:33.639006+0000","last_undegraded":"2024-09-21T05:55:33.639006+0000","last_fullsized":"2024-09-21T05:55:33.639006+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:50.913635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"74'4","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.646791+0000","last_change":"2024-09-21T05:55:33.646124+0000","last_active":"2024-09-21T05:55:33.646791+0000","last_peered":"2024-09-21T05:55:33.646791+0000","last_clean":"2024-09-21T05:55:33.646791+0000","last_became_active":"2024-09-21T05:55:33.641142+0000","last_became_peered":"2024-09-21T05:55:33.641142+0000","last_unstale":"2024-09-21T05:55:33.646791+0000","last_undegraded":"2024-09-21T05:55:33.646791+0000","last_fullsized":"2024-09-21T05:55:33.646791+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:36.341473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635915+0000","last_change":"2024-09-21T05:55:13.997494+0000","last_active":"2024-09-21T05:55:33.635915+0000","last_peered":"2024-09-21T05:55:33.635915+0000","last_clean":"2024-09-21T05:55:33.635915+0000","last_became_active":"2024-09-21T05:55:13.997138+0000","last_became_peered":"2024-09-21T05:55:13.997138+0000","last_unstale":"2024-09-21T05:55:33.635915+0000","last_undegraded":"2024-09-21T05:55:33.635915+0000","last_fullsized":"2024-09-21T05:55:33.635915+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:02:01.777126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767713+0000","last_change":"2024-09-21T05:55:14.002750+0000","last_active":"2024-09-21T05:55:33.767713+0000","last_peered":"2024-09-21T05:55:33.767713+0000","last_clean":"2024-09-21T05:55:33.767713+0000","last_became_active":"2024-09-21T05:55:14.002150+0000","last_became_peered":"2024-09-21T05:55:14.002150+0000","last_unstale":"2024-09-21T05:55:33.767713+0000","last_undegraded":"2024-09-21T05:55:33.767713+0000","last_fullsized":"2024-09-21T05:55:33.767713+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:49.603133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"74'5","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636032+0000","last_change":"2024-09-21T05:55:11.982366+0000","last_active":"2024-09-21T05:55:33.636032+0000","last_peered":"2024-09-21T05:55:33.636032+0000","last_clean":"2024-09-21T05:55:33.636032+0000","last_became_active":"2024-09-21T05:55:11.981994+0000","last_became_peered":"2024-09-21T05:55:11.981994+0000","last_unstale":"2024-09-21T05:55:33.636032+0000","last_undegraded":"2024-09-21T05:55:33.636032+0000","last_fullsized":"2024-09-21T05:55:33.636032+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:14.626758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639407+0000","last_change":"2024-09-21T05:55:14.008204+0000","last_active":"2024-09-21T05:55:33.639407+0000","last_peered":"2024-09-21T05:55:33.639407+0000","last_clean":"2024-09-21T05:55:33.639407+0000","last_became_active":"2024-09-21T05:55:14.007800+0000","last_became_peered":"2024-09-21T05:55:14.007800+0000","last_unstale":"2024-09-21T05:55:33.639407+0000","last_undegraded":"2024-09-21T05:55:33.639407+0000","last_fullsized":"2024-09-21T05:55:33.639407+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:49.760850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"74'12","reported_seq":36,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636283+0000","last_change":"2024-09-21T05:55:11.986168+0000","last_active":"2024-09-21T05:55:33.636283+0000","last_peered":"2024-09-21T05:55:33.636283+0000","last_clean":"2024-09-21T05:55:33.636283+0000","last_became_active":"2024-09-21T05:55:11.985694+0000","last_became_peered":"2024-09-21T05:55:11.985694+0000","last_unstale":"2024-09-21T05:55:33.636283+0000","last_undegraded":"2024-09-21T05:55:33.636283+0000","last_fullsized":"2024-09-21T05:55:33.636283+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:44.863935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767026+0000","last_change":"2024-09-21T05:55:14.002459+0000","last_active":"2024-09-21T05:55:33.767026+0000","last_peered":"2024-09-21T05:55:33.767026+0000","last_clean":"2024-09-21T05:55:33.767026+0000","last_became_active":"2024-09-21T05:55:14.001962+0000","last_became_peered":"2024-09-21T05:55:14.001962+0000","last_unstale":"2024-09-21T05:55:33.767026+0000","last_undegraded":"2024-09-21T05:55:33.767026+0000","last_fullsized":"2024-09-21T05:55:33.767026+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:53:26.892772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639340+0000","last_change":"2024-09-21T05:55:12.014149+0000","last_active":"2024-09-21T05:55:33.639340+0000","last_peered":"2024-09-21T05:55:33.639340+0000","last_clean":"2024-09-21T05:55:33.639340+0000","last_became_active":"2024-09-21T05:55:12.013789+0000","last_became_peered":"2024-09-21T05:55:12.013789+0000","last_unstale":"2024-09-21T05:55:33.639340+0000","last_undegraded":"2024-09-21T05:55:33.639340+0000","last_fullsized":"2024-09-21T05:55:33.639340+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:18.304023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639186+0000","last_change":"2024-09-21T05:55:13.993500+0000","last_active":"2024-09-21T05:55:33.639186+0000","last_peered":"2024-09-21T05:55:33.639186+0000","last_clean":"2024-09-21T05:55:33.639186+0000","last_became_active":"2024-09-21T05:55:13.993104+0000","last_became_peered":"2024-09-21T05:55:13.993104+0000","last_unstale":"2024-09-21T05:55:33.639186+0000","last_undegraded":"2024-09-21T05:55:33.639186+0000","last_fullsized":"2024-09-21T05:55:33.639186+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:01.445119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"74'13","reported_seq":68,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:34.640963+0000","last_change":"2024-09-21T05:55:33.650207+0000","last_active":"2024-09-21T05:55:34.640963+0000","last_peered":"2024-09-21T05:55:34.640963+0000","last_clean":"2024-09-21T05:55:34.640963+0000","last_became_active":"2024-09-21T05:55:33.638388+0000","last_became_peered":"2024-09-21T05:55:33.638388+0000","last_unstale":"2024-09-21T05:55:34.640963+0000","last_undegraded":"2024-09-21T05:55:34.640963+0000","last_fullsized":"2024-09-21T05:55:34.640963+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:51.478063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":58535,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":149,"num_read_kb":134,"num_write":233,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":528384,"data_stored":175605,"data_compressed":5682,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":201,"ondisk_log_size":201,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281769,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":68,"seq":292057776145,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":839680,"data_stored":412559,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939670,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":974848,"data_stored":1053856,"data_compressed":43475,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103195,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":856064,"data_stored":411426,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299425,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":851968,"data_stored":411804,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495653,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":897024,"data_stored":1004041,"data_compressed":41581,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659179,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":839680,"data_stored":411143,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822704,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":917504,"data_stored":1004000,"data_compressed":41581,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986230,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":839680,"data_stored":411395,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":856064,"data_stored":410968,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378688,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":909312,"data_stored":463998,"data_compressed":7646,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542211,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":888832,"data_stored":462996,"data_compressed":7646,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705738,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":860160,"data_stored":412873,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34208,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34208,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34208,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35829,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35829,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35829,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35829,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35829,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":52692,"data_compressed":1894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":53694,"data_compressed":1894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":53067,"data_compressed":1894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:55:42.120 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T05:55:42.120 INFO:tasks.cephadm:Setting up client nodes... 2024-09-21T05:55:42.121 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-21T05:55:42.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:42 smithi138 ceph-mon[34391]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T05:55:42.367 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:42.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:42 smithi192 ceph-mon[34700]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T05:55:42.487 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:42 smithi117 ceph-mon[30947]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T05:55:42.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.847+0000 7f1d58505640 1 -- 172.21.15.117:0/3349800543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d50102aa0 msgr2=0x7f1d50102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:42.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.847+0000 7f1d58505640 1 --2- 172.21.15.117:0/3349800543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d50102aa0 0x7f1d50102eb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c009900 tx=0x7f1d4c02a410 comp rx=0 tx=0).stop 2024-09-21T05:55:42.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.847+0000 7f1d58505640 1 -- 172.21.15.117:0/3349800543 shutdown_connections 2024-09-21T05:55:42.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.847+0000 7f1d58505640 1 --2- 172.21.15.117:0/3349800543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d50108d30 0x7f1d5010f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:42.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.847+0000 7f1d58505640 1 --2- 172.21.15.117:0/3349800543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d501083e0 0x7f1d501087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:42.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.847+0000 7f1d58505640 1 --2- 172.21.15.117:0/3349800543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d50102aa0 0x7f1d50102eb0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:42.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.847+0000 7f1d58505640 1 -- 172.21.15.117:0/3349800543 >> 172.21.15.117:0/3349800543 conn(0x7f1d50079750 msgr2=0x7f1d50079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:42.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.847+0000 7f1d58505640 1 -- 172.21.15.117:0/3349800543 shutdown_connections 2024-09-21T05:55:42.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.847+0000 7f1d58505640 1 -- 172.21.15.117:0/3349800543 wait complete. 2024-09-21T05:55:42.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.848+0000 7f1d58505640 1 Processor -- start 2024-09-21T05:55:42.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.848+0000 7f1d58505640 1 -- start start 2024-09-21T05:55:42.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.848+0000 7f1d58505640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d50102aa0 0x7f1d5019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:42.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.848+0000 7f1d58505640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d501083e0 0x7f1d5019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:42.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.848+0000 7f1d58505640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d50108d30 0x7f1d5019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:42.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.848+0000 7f1d58505640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1d50113b50 con 0x7f1d501083e0 2024-09-21T05:55:42.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.848+0000 7f1d58505640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1d501139d0 con 0x7f1d50108d30 2024-09-21T05:55:42.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.848+0000 7f1d58505640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1d50113cd0 con 0x7f1d50102aa0 2024-09-21T05:55:42.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.848+0000 7f1d55aa0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d501083e0 0x7f1d5019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:42.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.848+0000 7f1d55aa0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d501083e0 0x7f1d5019ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37414/0 (socket says 172.21.15.117:37414) 2024-09-21T05:55:42.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.848+0000 7f1d55aa0640 1 -- 172.21.15.117:0/3206036543 learned_addr learned my addr 172.21.15.117:0/3206036543 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:55:42.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.849+0000 7f1d562a1640 1 --2- 172.21.15.117:0/3206036543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d50102aa0 0x7f1d5019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:42.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.849+0000 7f1d56aa2640 1 --2- 172.21.15.117:0/3206036543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d50108d30 0x7f1d5019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:42.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.849+0000 7f1d55aa0640 1 -- 172.21.15.117:0/3206036543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d50102aa0 msgr2=0x7f1d5019a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:42.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.849+0000 7f1d55aa0640 1 --2- 172.21.15.117:0/3206036543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d50102aa0 0x7f1d5019a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:42.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.849+0000 7f1d55aa0640 1 -- 172.21.15.117:0/3206036543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d50108d30 msgr2=0x7f1d5019fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:42.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.849+0000 7f1d55aa0640 1 --2- 172.21.15.117:0/3206036543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d50108d30 0x7f1d5019fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:42.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.849+0000 7f1d55aa0640 1 -- 172.21.15.117:0/3206036543 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d501a0410 con 0x7f1d501083e0 2024-09-21T05:55:42.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.849+0000 7f1d562a1640 1 --2- 172.21.15.117:0/3206036543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d50102aa0 0x7f1d5019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:55:42.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.849+0000 7f1d55aa0640 1 --2- 172.21.15.117:0/3206036543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d501083e0 0x7f1d5019ab00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1d4000c8c0 tx=0x7f1d4000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:42.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.849+0000 7f1d477fe640 1 -- 172.21.15.117:0/3206036543 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d40007bb0 con 0x7f1d501083e0 2024-09-21T05:55:42.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.849+0000 7f1d58505640 1 -- 172.21.15.117:0/3206036543 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d501a0700 con 0x7f1d501083e0 2024-09-21T05:55:42.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.849+0000 7f1d477fe640 1 -- 172.21.15.117:0/3206036543 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d40007d50 con 0x7f1d501083e0 2024-09-21T05:55:42.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.849+0000 7f1d477fe640 1 -- 172.21.15.117:0/3206036543 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d400187c0 con 0x7f1d501083e0 2024-09-21T05:55:42.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.849+0000 7f1d58505640 1 -- 172.21.15.117:0/3206036543 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d501a6f20 con 0x7f1d501083e0 2024-09-21T05:55:42.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.850+0000 7f1d58505640 1 -- 172.21.15.117:0/3206036543 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d50102eb0 con 0x7f1d501083e0 2024-09-21T05:55:42.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.852+0000 7f1d477fe640 1 -- 172.21.15.117:0/3206036543 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f1d4000fa20 con 0x7f1d501083e0 2024-09-21T05:55:42.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.852+0000 7f1d477fe640 1 --2- 172.21.15.117:0/3206036543 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1d3c0b6620 0x7f1d3c0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:42.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.852+0000 7f1d562a1640 1 --2- 172.21.15.117:0/3206036543 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1d3c0b6620 0x7f1d3c0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:42.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.852+0000 7f1d477fe640 1 -- 172.21.15.117:0/3206036543 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f1d400e79f0 con 0x7f1d501083e0 2024-09-21T05:55:42.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.853+0000 7f1d562a1640 1 --2- 172.21.15.117:0/3206036543 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1d3c0b6620 0x7f1d3c0b8b30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c031440 tx=0x7f1d4c005d50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:42.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:42.856+0000 7f1d477fe640 1 -- 172.21.15.117:0/3206036543 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1d40014030 con 0x7f1d501083e0 2024-09-21T05:55:43.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:43.104+0000 7f1d58505640 1 -- 172.21.15.117:0/3206036543 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f1d5019b690 con 0x7f1d501083e0 2024-09-21T05:55:43.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:43.107+0000 7f1d477fe640 1 -- 172.21.15.117:0/3206036543 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f1d5019b690 con 0x7f1d501083e0 2024-09-21T05:55:43.109 INFO:teuthology.orchestra.run.smithi117.stdout:[client.0] 2024-09-21T05:55:43.109 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDfX+5mbDdPBhAAk8OKspZ/CyjOB7/7BOZiGA== 2024-09-21T05:55:43.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:43.109+0000 7f1d58505640 1 -- 172.21.15.117:0/3206036543 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1d3c0b6620 msgr2=0x7f1d3c0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:43.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:43.109+0000 7f1d58505640 1 --2- 172.21.15.117:0/3206036543 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1d3c0b6620 0x7f1d3c0b8b30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c031440 tx=0x7f1d4c005d50 comp rx=0 tx=0).stop 2024-09-21T05:55:43.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:43.109+0000 7f1d58505640 1 -- 172.21.15.117:0/3206036543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d501083e0 msgr2=0x7f1d5019ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:43.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:43.109+0000 7f1d58505640 1 --2- 172.21.15.117:0/3206036543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d501083e0 0x7f1d5019ab00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1d4000c8c0 tx=0x7f1d4000cd80 comp rx=0 tx=0).stop 2024-09-21T05:55:43.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:43.109+0000 7f1d58505640 1 -- 172.21.15.117:0/3206036543 shutdown_connections 2024-09-21T05:55:43.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:43.109+0000 7f1d58505640 1 --2- 172.21.15.117:0/3206036543 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1d3c0b6620 0x7f1d3c0b8b30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:43.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:43.109+0000 7f1d58505640 1 --2- 172.21.15.117:0/3206036543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d501083e0 0x7f1d5019ab00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:43.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:43.109+0000 7f1d58505640 1 --2- 172.21.15.117:0/3206036543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d50108d30 0x7f1d5019fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:43.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:43.109+0000 7f1d58505640 1 --2- 172.21.15.117:0/3206036543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d50102aa0 0x7f1d5019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:43.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:43.109+0000 7f1d58505640 1 -- 172.21.15.117:0/3206036543 >> 172.21.15.117:0/3206036543 conn(0x7f1d50079750 msgr2=0x7f1d50106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:43.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:43.109+0000 7f1d58505640 1 -- 172.21.15.117:0/3206036543 shutdown_connections 2024-09-21T05:55:43.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:43.109+0000 7f1d58505640 1 -- 172.21.15.117:0/3206036543 wait complete. 2024-09-21T05:55:43.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:43 smithi138 ceph-mon[34391]: pgmap v279: 97 pgs: 97 active+clean; 634 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.5 KiB/s wr, 9 op/s; 68 B/s, 1 objects/s recovering 2024-09-21T05:55:43.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:43 smithi192 ceph-mon[34700]: pgmap v279: 97 pgs: 97 active+clean; 634 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.5 KiB/s wr, 9 op/s; 68 B/s, 1 objects/s recovering 2024-09-21T05:55:43.496 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:43 smithi117 ceph-mon[30947]: pgmap v279: 97 pgs: 97 active+clean; 634 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.5 KiB/s wr, 9 op/s; 68 B/s, 1 objects/s recovering 2024-09-21T05:55:43.943 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-21T05:55:43.943 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-21T05:55:43.943 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T05:55:43.980 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph config log 1 --format=json 2024-09-21T05:55:44.267 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:44.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:44 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3206036543' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T05:55:44.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:44 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3206036543' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T05:55:44.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:44 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3206036543' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T05:55:44.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:44 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3206036543' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T05:55:44.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:44 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3206036543' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T05:55:44.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:44 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3206036543' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T05:55:44.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.782+0000 7f26a32b1640 1 -- 172.21.15.117:0/2483131739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f269c1083c0 msgr2=0x7f269c1087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:44.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.782+0000 7f26a32b1640 1 --2- 172.21.15.117:0/2483131739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f269c1083c0 0x7f269c1087d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f268c009900 tx=0x7f268c02a410 comp rx=0 tx=0).stop 2024-09-21T05:55:44.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.782+0000 7f26a32b1640 1 -- 172.21.15.117:0/2483131739 shutdown_connections 2024-09-21T05:55:44.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.782+0000 7f26a32b1640 1 --2- 172.21.15.117:0/2483131739 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f269c1083c0 0x7f269c1087d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:44.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.782+0000 7f26a32b1640 1 --2- 172.21.15.117:0/2483131739 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f269c108d10 0x7f269c10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:44.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.782+0000 7f26a32b1640 1 --2- 172.21.15.117:0/2483131739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f269c102a80 0x7f269c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:44.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.782+0000 7f26a32b1640 1 -- 172.21.15.117:0/2483131739 >> 172.21.15.117:0/2483131739 conn(0x7f269c079740 msgr2=0x7f269c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:44.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.783+0000 7f26a32b1640 1 -- 172.21.15.117:0/2483131739 shutdown_connections 2024-09-21T05:55:44.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.783+0000 7f26a32b1640 1 -- 172.21.15.117:0/2483131739 wait complete. 2024-09-21T05:55:44.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.783+0000 7f26a32b1640 1 Processor -- start 2024-09-21T05:55:44.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.783+0000 7f26a32b1640 1 -- start start 2024-09-21T05:55:44.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.784+0000 7f26a32b1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f269c102a80 0x7f269c19b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:44.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.784+0000 7f26a32b1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f269c1083c0 0x7f269c19b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:44.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.784+0000 7f26a32b1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f269c108d10 0x7f269c19be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:44.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.784+0000 7f26a32b1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f269c112980 con 0x7f269c108d10 2024-09-21T05:55:44.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.784+0000 7f26a32b1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f269c112800 con 0x7f269c1083c0 2024-09-21T05:55:44.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.784+0000 7f26a32b1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f269c112b00 con 0x7f269c102a80 2024-09-21T05:55:44.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.784+0000 7f26a184e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f269c108d10 0x7f269c19be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:44.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.784+0000 7f26a184e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f269c108d10 0x7f269c19be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43126/0 (socket says 172.21.15.117:43126) 2024-09-21T05:55:44.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.784+0000 7f26a184e640 1 -- 172.21.15.117:0/2408045669 learned_addr learned my addr 172.21.15.117:0/2408045669 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:55:44.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.784+0000 7f26a084c640 1 --2- 172.21.15.117:0/2408045669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f269c1083c0 0x7f269c19b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:44.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.784+0000 7f26a104d640 1 --2- 172.21.15.117:0/2408045669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f269c102a80 0x7f269c19b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:44.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.784+0000 7f26a184e640 1 -- 172.21.15.117:0/2408045669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f269c102a80 msgr2=0x7f269c19b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:44.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.784+0000 7f26a184e640 1 --2- 172.21.15.117:0/2408045669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f269c102a80 0x7f269c19b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:44.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.784+0000 7f26a184e640 1 -- 172.21.15.117:0/2408045669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f269c1083c0 msgr2=0x7f269c19b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:44.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.784+0000 7f26a184e640 1 --2- 172.21.15.117:0/2408045669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f269c1083c0 0x7f269c19b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:44.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.784+0000 7f26a184e640 1 -- 172.21.15.117:0/2408045669 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f269c1a66a0 con 0x7f269c108d10 2024-09-21T05:55:44.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.784+0000 7f26a184e640 1 --2- 172.21.15.117:0/2408045669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f269c108d10 0x7f269c19be90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f269400cad0 tx=0x7f269400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:44.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.784+0000 7f26927fc640 1 -- 172.21.15.117:0/2408045669 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269400ae20 con 0x7f269c108d10 2024-09-21T05:55:44.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.785+0000 7f26a32b1640 1 -- 172.21.15.117:0/2408045669 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f269c1a6990 con 0x7f269c108d10 2024-09-21T05:55:44.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.785+0000 7f26927fc640 1 -- 172.21.15.117:0/2408045669 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2694005020 con 0x7f269c108d10 2024-09-21T05:55:44.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.785+0000 7f26927fc640 1 -- 172.21.15.117:0/2408045669 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269400f6b0 con 0x7f269c108d10 2024-09-21T05:55:44.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.785+0000 7f26a32b1640 1 -- 172.21.15.117:0/2408045669 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f269c1a6ef0 con 0x7f269c108d10 2024-09-21T05:55:44.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.786+0000 7f26a32b1640 1 -- 172.21.15.117:0/2408045669 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f269c103d00 con 0x7f269c108d10 2024-09-21T05:55:44.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.787+0000 7f26927fc640 1 -- 172.21.15.117:0/2408045669 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f26940051d0 con 0x7f269c108d10 2024-09-21T05:55:44.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.788+0000 7f26927fc640 1 --2- 172.21.15.117:0/2408045669 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f26880b66a0 0x7f26880b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:44.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.788+0000 7f26a104d640 1 --2- 172.21.15.117:0/2408045669 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f26880b66a0 0x7f26880b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:44.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.788+0000 7f26927fc640 1 -- 172.21.15.117:0/2408045669 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f26940e77a0 con 0x7f269c108d10 2024-09-21T05:55:44.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.789+0000 7f26a104d640 1 --2- 172.21.15.117:0/2408045669 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f26880b66a0 0x7f26880b8bb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f26980058c0 tx=0x7f269800a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:44.793 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.791+0000 7f26927fc640 1 -- 172.21.15.117:0/2408045669 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2694014030 con 0x7f269c108d10 2024-09-21T05:55:44.971 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.970+0000 7f26a32b1640 1 -- 172.21.15.117:0/2408045669 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f269c102e90 con 0x7f269c108d10 2024-09-21T05:55:44.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.970+0000 7f26927fc640 1 -- 172.21.15.117:0/2408045669 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v56) ==== 86+0+133 (secure 0 0 0) 0x7f269c102e90 con 0x7f269c108d10 2024-09-21T05:55:44.972 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:55:44.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.972+0000 7f26a32b1640 1 -- 172.21.15.117:0/2408045669 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f26880b66a0 msgr2=0x7f26880b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:44.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.972+0000 7f26a32b1640 1 --2- 172.21.15.117:0/2408045669 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f26880b66a0 0x7f26880b8bb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f26980058c0 tx=0x7f269800a2d0 comp rx=0 tx=0).stop 2024-09-21T05:55:44.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.972+0000 7f26a32b1640 1 -- 172.21.15.117:0/2408045669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f269c108d10 msgr2=0x7f269c19be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:44.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.972+0000 7f26a32b1640 1 --2- 172.21.15.117:0/2408045669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f269c108d10 0x7f269c19be90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f269400cad0 tx=0x7f269400cf90 comp rx=0 tx=0).stop 2024-09-21T05:55:44.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.973+0000 7f26a32b1640 1 -- 172.21.15.117:0/2408045669 shutdown_connections 2024-09-21T05:55:44.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.973+0000 7f26a32b1640 1 --2- 172.21.15.117:0/2408045669 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f26880b66a0 0x7f26880b8bb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:44.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.973+0000 7f26a32b1640 1 --2- 172.21.15.117:0/2408045669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f269c108d10 0x7f269c19be90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:44.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.973+0000 7f26a32b1640 1 --2- 172.21.15.117:0/2408045669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f269c1083c0 0x7f269c19b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:44.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.973+0000 7f26a32b1640 1 --2- 172.21.15.117:0/2408045669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f269c102a80 0x7f269c19b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:44.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.973+0000 7f26a32b1640 1 -- 172.21.15.117:0/2408045669 >> 172.21.15.117:0/2408045669 conn(0x7f269c079740 msgr2=0x7f269c1045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:44.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.973+0000 7f26a32b1640 1 -- 172.21.15.117:0/2408045669 shutdown_connections 2024-09-21T05:55:44.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:44.973+0000 7f26a32b1640 1 -- 172.21.15.117:0/2408045669 wait complete. 2024-09-21T05:55:45.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:45 smithi117 ceph-mon[30947]: pgmap v280: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.0 KiB/s wr, 10 op/s; 61 B/s, 1 objects/s recovering 2024-09-21T05:55:45.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:45 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2408045669' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T05:55:45.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:45 smithi138 ceph-mon[34391]: pgmap v280: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.0 KiB/s wr, 10 op/s; 61 B/s, 1 objects/s recovering 2024-09-21T05:55:45.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:45 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2408045669' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T05:55:45.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:45 smithi192 ceph-mon[34700]: pgmap v280: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.0 KiB/s wr, 10 op/s; 61 B/s, 1 objects/s recovering 2024-09-21T05:55:45.381 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:45 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2408045669' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T05:55:45.736 INFO:teuthology.orchestra.run.smithi117.stdout:[{"version":56,"timestamp":"2024-09-21T05:55:04.244155+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-09-21T05:55:45.736 INFO:tasks.ceph_manager:config epoch is 56 2024-09-21T05:55:45.736 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T05:55:45.737 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-21T05:55:45.737 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph mgr dump --format=json 2024-09-21T05:55:45.984 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:46.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.433+0000 7f2044a4b640 1 -- 172.21.15.117:0/1199069476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2040108cf0 msgr2=0x7f204010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:46.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.433+0000 7f2044a4b640 1 --2- 172.21.15.117:0/1199069476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2040108cf0 0x7f204010f520 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2038009ca0 tx=0x7f203802a8c0 comp rx=0 tx=0).stop 2024-09-21T05:55:46.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.433+0000 7f2044a4b640 1 -- 172.21.15.117:0/1199069476 shutdown_connections 2024-09-21T05:55:46.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.433+0000 7f2044a4b640 1 --2- 172.21.15.117:0/1199069476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2040108cf0 0x7f204010f520 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:46.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.433+0000 7f2044a4b640 1 --2- 172.21.15.117:0/1199069476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20401083a0 0x7f20401087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:46.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.433+0000 7f2044a4b640 1 --2- 172.21.15.117:0/1199069476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2040102a60 0x7f2040102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:46.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.433+0000 7f2044a4b640 1 -- 172.21.15.117:0/1199069476 >> 172.21.15.117:0/1199069476 conn(0x7f2040079730 msgr2=0x7f2040079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:46.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.433+0000 7f2044a4b640 1 -- 172.21.15.117:0/1199069476 shutdown_connections 2024-09-21T05:55:46.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.433+0000 7f2044a4b640 1 -- 172.21.15.117:0/1199069476 wait complete. 2024-09-21T05:55:46.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.434+0000 7f2044a4b640 1 Processor -- start 2024-09-21T05:55:46.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.435+0000 7f2044a4b640 1 -- start start 2024-09-21T05:55:46.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.435+0000 7f2044a4b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2040102a60 0x7f20400ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:46.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.435+0000 7f2044a4b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20401083a0 0x7f20400ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:46.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.435+0000 7f2044a4b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2040108cf0 0x7f20401000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:46.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.435+0000 7f2044a4b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f20401129c0 con 0x7f2040102a60 2024-09-21T05:55:46.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.435+0000 7f2044a4b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2040112840 con 0x7f20401083a0 2024-09-21T05:55:46.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.435+0000 7f2044a4b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2040112b40 con 0x7f2040108cf0 2024-09-21T05:55:46.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.435+0000 7f203e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2040102a60 0x7f20400ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:46.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.435+0000 7f203e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2040102a60 0x7f20400ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43142/0 (socket says 172.21.15.117:43142) 2024-09-21T05:55:46.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.435+0000 7f203e59c640 1 -- 172.21.15.117:0/935338908 learned_addr learned my addr 172.21.15.117:0/935338908 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:55:46.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.435+0000 7f203dd9b640 1 --2- 172.21.15.117:0/935338908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20401083a0 0x7f20400ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:46.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.435+0000 7f203e59c640 1 -- 172.21.15.117:0/935338908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2040108cf0 msgr2=0x7f20401000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:46.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.435+0000 7f203e59c640 1 --2- 172.21.15.117:0/935338908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2040108cf0 0x7f20401000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:46.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.435+0000 7f203e59c640 1 -- 172.21.15.117:0/935338908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20401083a0 msgr2=0x7f20400ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:46.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.435+0000 7f203e59c640 1 --2- 172.21.15.117:0/935338908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20401083a0 0x7f20400ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:46.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.435+0000 7f203e59c640 1 -- 172.21.15.117:0/935338908 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20401a8760 con 0x7f2040102a60 2024-09-21T05:55:46.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.436+0000 7f203e59c640 1 --2- 172.21.15.117:0/935338908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2040102a60 0x7f20400ff670 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f203000b4a0 tx=0x7f203000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:46.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.436+0000 7f202f7fe640 1 -- 172.21.15.117:0/935338908 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2030014070 con 0x7f2040102a60 2024-09-21T05:55:46.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.436+0000 7f2044a4b640 1 -- 172.21.15.117:0/935338908 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20401a8a50 con 0x7f2040102a60 2024-09-21T05:55:46.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.436+0000 7f202f7fe640 1 -- 172.21.15.117:0/935338908 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2030010ac0 con 0x7f2040102a60 2024-09-21T05:55:46.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.436+0000 7f202f7fe640 1 -- 172.21.15.117:0/935338908 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2030017c30 con 0x7f2040102a60 2024-09-21T05:55:46.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.436+0000 7f2044a4b640 1 -- 172.21.15.117:0/935338908 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20401a9010 con 0x7f2040102a60 2024-09-21T05:55:46.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.437+0000 7f2044a4b640 1 -- 172.21.15.117:0/935338908 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2040102e70 con 0x7f2040102a60 2024-09-21T05:55:46.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.438+0000 7f202f7fe640 1 -- 172.21.15.117:0/935338908 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f2030017070 con 0x7f2040102a60 2024-09-21T05:55:46.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.439+0000 7f202f7fe640 1 --2- 172.21.15.117:0/935338908 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f20280b65f0 0x7f20280b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:46.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.439+0000 7f203dd9b640 1 --2- 172.21.15.117:0/935338908 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f20280b65f0 0x7f20280b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:46.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.439+0000 7f202f7fe640 1 -- 172.21.15.117:0/935338908 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f20300e6860 con 0x7f2040102a60 2024-09-21T05:55:46.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.440+0000 7f203dd9b640 1 --2- 172.21.15.117:0/935338908 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f20280b65f0 0x7f20280b8b00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2040106fb0 tx=0x7f203400a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:46.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.444+0000 7f202f7fe640 1 -- 172.21.15.117:0/935338908 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f20300acef0 con 0x7f2040102a60 2024-09-21T05:55:46.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.691+0000 7f2044a4b640 1 -- 172.21.15.117:0/935338908 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f20401a9400 con 0x7f2040102a60 2024-09-21T05:55:46.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.696+0000 7f202f7fe640 1 -- 172.21.15.117:0/935338908 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v21) ==== 74+0+296462 (secure 0 0 0) 0x7f20300b12d0 con 0x7f2040102a60 2024-09-21T05:55:46.702 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:55:46.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.699+0000 7f2044a4b640 1 -- 172.21.15.117:0/935338908 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f20280b65f0 msgr2=0x7f20280b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:46.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.699+0000 7f2044a4b640 1 --2- 172.21.15.117:0/935338908 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f20280b65f0 0x7f20280b8b00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2040106fb0 tx=0x7f203400a2d0 comp rx=0 tx=0).stop 2024-09-21T05:55:46.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.699+0000 7f2044a4b640 1 -- 172.21.15.117:0/935338908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2040102a60 msgr2=0x7f20400ff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:46.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.699+0000 7f2044a4b640 1 --2- 172.21.15.117:0/935338908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2040102a60 0x7f20400ff670 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f203000b4a0 tx=0x7f203000b960 comp rx=0 tx=0).stop 2024-09-21T05:55:46.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.699+0000 7f2044a4b640 1 -- 172.21.15.117:0/935338908 shutdown_connections 2024-09-21T05:55:46.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.699+0000 7f2044a4b640 1 --2- 172.21.15.117:0/935338908 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f20280b65f0 0x7f20280b8b00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:46.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.699+0000 7f2044a4b640 1 --2- 172.21.15.117:0/935338908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2040108cf0 0x7f20401000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:46.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.699+0000 7f2044a4b640 1 --2- 172.21.15.117:0/935338908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20401083a0 0x7f20400ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:46.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.699+0000 7f2044a4b640 1 --2- 172.21.15.117:0/935338908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2040102a60 0x7f20400ff670 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:46.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.699+0000 7f2044a4b640 1 -- 172.21.15.117:0/935338908 >> 172.21.15.117:0/935338908 conn(0x7f2040079730 msgr2=0x7f2040101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:46.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.700+0000 7f2044a4b640 1 -- 172.21.15.117:0/935338908 shutdown_connections 2024-09-21T05:55:46.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:46.700+0000 7f2044a4b640 1 -- 172.21.15.117:0/935338908 wait complete. 2024-09-21T05:55:47.535 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":21,"flags":0,"active_gid":14223,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":4224731851},{"type":"v1","addr":"172.21.15.117:6801","nonce":4224731851}]},"active_addr":"172.21.15.117:6801/4224731851","active_change":"2024-09-21T05:48:32.406821+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24136,"name":"z","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24140,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.117:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":465263213}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":1446309389}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":2927815512}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":1222785116}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":3979577666}]}]} 2024-09-21T05:55:47.542 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-21T05:55:47.542 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-21T05:55:47.542 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T05:55:47.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:47 smithi117 ceph-mon[30947]: pgmap v281: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.7 KiB/s wr, 11 op/s; 55 B/s, 0 objects/s recovering 2024-09-21T05:55:47.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:47 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/935338908' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T05:55:47.791 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:47.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:47 smithi138 ceph-mon[34391]: pgmap v281: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.7 KiB/s wr, 11 op/s; 55 B/s, 0 objects/s recovering 2024-09-21T05:55:47.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:47 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/935338908' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T05:55:47.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:47 smithi192 ceph-mon[34700]: pgmap v281: 97 pgs: 97 active+clean; 642 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.7 KiB/s wr, 11 op/s; 55 B/s, 0 objects/s recovering 2024-09-21T05:55:47.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:47 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/935338908' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T05:55:48.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.299+0000 7f680bb99640 1 -- 172.21.15.117:0/1664377075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6804103160 msgr2=0x7f6804103570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:48.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.299+0000 7f680bb99640 1 --2- 172.21.15.117:0/1664377075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6804103160 0x7f6804103570 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f67f4009ca0 tx=0x7f67f402a8c0 comp rx=0 tx=0).stop 2024-09-21T05:55:48.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.299+0000 7f680bb99640 1 -- 172.21.15.117:0/1664377075 shutdown_connections 2024-09-21T05:55:48.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.299+0000 7f680bb99640 1 --2- 172.21.15.117:0/1664377075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6804105ac0 0x7f680410f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:48.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.299+0000 7f680bb99640 1 --2- 172.21.15.117:0/1664377075 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6804105170 0x7f6804105580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:48.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.299+0000 7f680bb99640 1 --2- 172.21.15.117:0/1664377075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6804103160 0x7f6804103570 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:48.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.299+0000 7f680bb99640 1 -- 172.21.15.117:0/1664377075 >> 172.21.15.117:0/1664377075 conn(0x7f6804077c60 msgr2=0x7f68040780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:48.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.299+0000 7f680bb99640 1 -- 172.21.15.117:0/1664377075 shutdown_connections 2024-09-21T05:55:48.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.299+0000 7f680bb99640 1 -- 172.21.15.117:0/1664377075 wait complete. 2024-09-21T05:55:48.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.300+0000 7f680bb99640 1 Processor -- start 2024-09-21T05:55:48.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.300+0000 7f680bb99640 1 -- start start 2024-09-21T05:55:48.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.300+0000 7f680bb99640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6804103160 0x7f680419b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:48.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.300+0000 7f680bb99640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6804105170 0x7f680419b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:48.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.300+0000 7f680bb99640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6804105ac0 0x7f680419bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:48.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.300+0000 7f680bb99640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f68041129c0 con 0x7f6804103160 2024-09-21T05:55:48.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.300+0000 7f680bb99640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6804112840 con 0x7f6804105ac0 2024-09-21T05:55:48.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.300+0000 7f680bb99640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6804112b40 con 0x7f6804105170 2024-09-21T05:55:48.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.300+0000 7f6809935640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6804103160 0x7f680419b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:48.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.300+0000 7f6809935640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6804103160 0x7f680419b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43154/0 (socket says 172.21.15.117:43154) 2024-09-21T05:55:48.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.300+0000 7f6809935640 1 -- 172.21.15.117:0/3169192050 learned_addr learned my addr 172.21.15.117:0/3169192050 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:55:48.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.300+0000 7f6809134640 1 --2- 172.21.15.117:0/3169192050 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6804105170 0x7f680419b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:48.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.300+0000 7f6809935640 1 -- 172.21.15.117:0/3169192050 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6804105170 msgr2=0x7f680419b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:48.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.300+0000 7f6809935640 1 --2- 172.21.15.117:0/3169192050 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6804105170 0x7f680419b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:48.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.300+0000 7f6809935640 1 -- 172.21.15.117:0/3169192050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6804105ac0 msgr2=0x7f680419bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:48.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.300+0000 7f680a136640 1 --2- 172.21.15.117:0/3169192050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6804105ac0 0x7f680419bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:48.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.300+0000 7f6809935640 1 --2- 172.21.15.117:0/3169192050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6804105ac0 0x7f680419bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:48.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.300+0000 7f6809935640 1 -- 172.21.15.117:0/3169192050 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68041a65c0 con 0x7f6804103160 2024-09-21T05:55:48.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.300+0000 7f6809134640 1 --2- 172.21.15.117:0/3169192050 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6804105170 0x7f680419b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:55:48.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.301+0000 7f6809935640 1 --2- 172.21.15.117:0/3169192050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6804103160 0x7f680419b2b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f67f402ade0 tx=0x7f67f4003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:48.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.301+0000 7f6802ffd640 1 -- 172.21.15.117:0/3169192050 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f4003d60 con 0x7f6804103160 2024-09-21T05:55:48.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.301+0000 7f680bb99640 1 -- 172.21.15.117:0/3169192050 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68041a6850 con 0x7f6804103160 2024-09-21T05:55:48.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.301+0000 7f6802ffd640 1 -- 172.21.15.117:0/3169192050 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67f40044f0 con 0x7f6804103160 2024-09-21T05:55:48.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.301+0000 7f6802ffd640 1 -- 172.21.15.117:0/3169192050 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f402fd10 con 0x7f6804103160 2024-09-21T05:55:48.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.301+0000 7f680bb99640 1 -- 172.21.15.117:0/3169192050 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68041a6de0 con 0x7f6804103160 2024-09-21T05:55:48.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.302+0000 7f680bb99640 1 -- 172.21.15.117:0/3169192050 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6804103570 con 0x7f6804103160 2024-09-21T05:55:48.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.303+0000 7f6802ffd640 1 -- 172.21.15.117:0/3169192050 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f67f40046a0 con 0x7f6804103160 2024-09-21T05:55:48.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.304+0000 7f6802ffd640 1 --2- 172.21.15.117:0/3169192050 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f67ec0b65f0 0x7f67ec0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:48.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.304+0000 7f6809134640 1 --2- 172.21.15.117:0/3169192050 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f67ec0b65f0 0x7f67ec0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:48.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.304+0000 7f6802ffd640 1 -- 172.21.15.117:0/3169192050 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f67f4108d10 con 0x7f6804103160 2024-09-21T05:55:48.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.304+0000 7f6809134640 1 --2- 172.21.15.117:0/3169192050 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f67ec0b65f0 0x7f67ec0b8b00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f67f8005fd0 tx=0x7f67f8005ea0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:48.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.307+0000 7f6802ffd640 1 -- 172.21.15.117:0/3169192050 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f67f403b410 con 0x7f6804103160 2024-09-21T05:55:48.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.485+0000 7f680bb99640 1 -- 172.21.15.117:0/3169192050 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6804102230 con 0x7f6804103160 2024-09-21T05:55:48.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.486+0000 7f6802ffd640 1 -- 172.21.15.117:0/3169192050 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+23066 (secure 0 0 0) 0x7f6804102230 con 0x7f6804103160 2024-09-21T05:55:48.488 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:55:48.488 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":76,"fsid":"ac8581a2-77dc-11ef-baf5-efdc52797490","created":"2024-09-21T05:46:47.021846+0000","modified":"2024-09-21T05:55:33.628948+0000","last_up_change":"2024-09-21T05:54:21.572849+0000","last_in_change":"2024-09-21T05:53:58.892597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:50:32.717389+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:55:10.604159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T05:55:12.581258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"68315502-fa6c-4a6e-a2d4-a7dde8071c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6803","nonce":3761478035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6805","nonce":3761478035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6809","nonce":3761478035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6807","nonce":3761478035}]},"public_addr":"172.21.15.117:6803/3761478035","cluster_addr":"172.21.15.117:6805/3761478035","heartbeat_back_addr":"172.21.15.117:6809/3761478035","heartbeat_front_addr":"172.21.15.117:6807/3761478035","state":["exists","up"]},{"osd":1,"uuid":"d8910a08-4ef3-49f2-a4db-e3588d85d12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6801","nonce":1737365588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6803","nonce":1737365588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6807","nonce":1737365588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6805","nonce":1737365588}]},"public_addr":"172.21.15.138:6801/1737365588","cluster_addr":"172.21.15.138:6803/1737365588","heartbeat_back_addr":"172.21.15.138:6807/1737365588","heartbeat_front_addr":"172.21.15.138:6805/1737365588","state":["exists","up"]},{"osd":2,"uuid":"94972378-a11f-4129-9e40-ad3fdec1d189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6801","nonce":3002534992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6803","nonce":3002534992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6807","nonce":3002534992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6805","nonce":3002534992}]},"public_addr":"172.21.15.192:6801/3002534992","cluster_addr":"172.21.15.192:6803/3002534992","heartbeat_back_addr":"172.21.15.192:6807/3002534992","heartbeat_front_addr":"172.21.15.192:6805/3002534992","state":["exists","up"]},{"osd":3,"uuid":"d4168c43-f0de-4445-945c-92d1cf270c33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6811","nonce":1222400293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6813","nonce":1222400293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6817","nonce":1222400293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6815","nonce":1222400293}]},"public_addr":"172.21.15.117:6811/1222400293","cluster_addr":"172.21.15.117:6813/1222400293","heartbeat_back_addr":"172.21.15.117:6817/1222400293","heartbeat_front_addr":"172.21.15.117:6815/1222400293","state":["exists","up"]},{"osd":4,"uuid":"28f1d7c0-8392-4a33-85c0-0de611cdd4b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6809","nonce":4294892870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6811","nonce":4294892870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6815","nonce":4294892870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6813","nonce":4294892870}]},"public_addr":"172.21.15.138:6809/4294892870","cluster_addr":"172.21.15.138:6811/4294892870","heartbeat_back_addr":"172.21.15.138:6815/4294892870","heartbeat_front_addr":"172.21.15.138:6813/4294892870","state":["exists","up"]},{"osd":5,"uuid":"ca902407-45e8-4bc5-bdbd-a729c1de8322","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6809","nonce":1776840019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6811","nonce":1776840019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6815","nonce":1776840019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6813","nonce":1776840019}]},"public_addr":"172.21.15.192:6809/1776840019","cluster_addr":"172.21.15.192:6811/1776840019","heartbeat_back_addr":"172.21.15.192:6815/1776840019","heartbeat_front_addr":"172.21.15.192:6813/1776840019","state":["exists","up"]},{"osd":6,"uuid":"6e507a91-ed09-4a33-bf0e-c4b593b3c5aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6819","nonce":1093341590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6821","nonce":1093341590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6825","nonce":1093341590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6823","nonce":1093341590}]},"public_addr":"172.21.15.117:6819/1093341590","cluster_addr":"172.21.15.117:6821/1093341590","heartbeat_back_addr":"172.21.15.117:6825/1093341590","heartbeat_front_addr":"172.21.15.117:6823/1093341590","state":["exists","up"]},{"osd":7,"uuid":"36a47dd7-67f7-4ebe-8759-3a8bb6bd2ea3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6817","nonce":1448179714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6819","nonce":1448179714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6823","nonce":1448179714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6821","nonce":1448179714}]},"public_addr":"172.21.15.138:6817/1448179714","cluster_addr":"172.21.15.138:6819/1448179714","heartbeat_back_addr":"172.21.15.138:6823/1448179714","heartbeat_front_addr":"172.21.15.138:6821/1448179714","state":["exists","up"]},{"osd":8,"uuid":"118e0b5d-2619-4512-94c5-8d07a0d3a28f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6817","nonce":3960902885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6819","nonce":3960902885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6823","nonce":3960902885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6821","nonce":3960902885}]},"public_addr":"172.21.15.192:6817/3960902885","cluster_addr":"172.21.15.192:6819/3960902885","heartbeat_back_addr":"172.21.15.192:6823/3960902885","heartbeat_front_addr":"172.21.15.192:6821/3960902885","state":["exists","up"]},{"osd":9,"uuid":"47543bc6-fa0b-4e72-9742-97dcda0b7f8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6827","nonce":3453855195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6829","nonce":3453855195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6833","nonce":3453855195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6831","nonce":3453855195}]},"public_addr":"172.21.15.117:6827/3453855195","cluster_addr":"172.21.15.117:6829/3453855195","heartbeat_back_addr":"172.21.15.117:6833/3453855195","heartbeat_front_addr":"172.21.15.117:6831/3453855195","state":["exists","up"]},{"osd":10,"uuid":"5e6708b4-7a85-4fa5-a03c-27d99f7d749f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6825","nonce":903392451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6827","nonce":903392451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6831","nonce":903392451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6829","nonce":903392451}]},"public_addr":"172.21.15.138:6825/903392451","cluster_addr":"172.21.15.138:6827/903392451","heartbeat_back_addr":"172.21.15.138:6831/903392451","heartbeat_front_addr":"172.21.15.138:6829/903392451","state":["exists","up"]},{"osd":11,"uuid":"24bc2ef7-e2b4-43c6-9e6c-43ae02390415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6825","nonce":899090479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6827","nonce":899090479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6831","nonce":899090479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6829","nonce":899090479}]},"public_addr":"172.21.15.192:6825/899090479","cluster_addr":"172.21.15.192:6827/899090479","heartbeat_back_addr":"172.21.15.192:6831/899090479","heartbeat_front_addr":"172.21.15.192:6829/899090479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:49:40.742217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:05.963655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:30.164495+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:56.598899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:51:21.940331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:51:47.777445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:52:12.869007+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:52:39.454054+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:03.569226+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:29.760863+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:55.788604+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:54:20.510184+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:0/2989394419":"2024-09-22T05:48:32.406654+0000","172.21.15.117:6801/1702676172":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/1766406059":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6800/2337036514":"2024-09-22T05:47:09.112435+0000","172.21.15.117:0/985798343":"2024-09-22T05:47:09.112435+0000","172.21.15.117:0/1083273222":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/473284565":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6801/2337036514":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6800/816635165":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/2642940197":"2024-09-22T05:48:32.406654+0000","172.21.15.117:6801/816635165":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/2225537433":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/3651584169":"2024-09-22T05:47:33.789146+0000","172.21.15.117:6800/1702676172":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/1284403882":"2024-09-22T05:47:33.789146+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:55:48.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.489+0000 7f680bb99640 1 -- 172.21.15.117:0/3169192050 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f67ec0b65f0 msgr2=0x7f67ec0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:48.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.489+0000 7f680bb99640 1 --2- 172.21.15.117:0/3169192050 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f67ec0b65f0 0x7f67ec0b8b00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f67f8005fd0 tx=0x7f67f8005ea0 comp rx=0 tx=0).stop 2024-09-21T05:55:48.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.489+0000 7f680bb99640 1 -- 172.21.15.117:0/3169192050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6804103160 msgr2=0x7f680419b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:48.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.489+0000 7f680bb99640 1 --2- 172.21.15.117:0/3169192050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6804103160 0x7f680419b2b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f67f402ade0 tx=0x7f67f4003970 comp rx=0 tx=0).stop 2024-09-21T05:55:48.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.489+0000 7f680bb99640 1 -- 172.21.15.117:0/3169192050 shutdown_connections 2024-09-21T05:55:48.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.489+0000 7f680bb99640 1 --2- 172.21.15.117:0/3169192050 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f67ec0b65f0 0x7f67ec0b8b00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:48.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.489+0000 7f680bb99640 1 --2- 172.21.15.117:0/3169192050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6804105ac0 0x7f680419bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:48.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.489+0000 7f680bb99640 1 --2- 172.21.15.117:0/3169192050 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6804105170 0x7f680419b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:48.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.489+0000 7f680bb99640 1 --2- 172.21.15.117:0/3169192050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6804103160 0x7f680419b2b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:48.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.489+0000 7f680bb99640 1 -- 172.21.15.117:0/3169192050 >> 172.21.15.117:0/3169192050 conn(0x7f6804077c60 msgr2=0x7f6804106f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:48.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.489+0000 7f680bb99640 1 -- 172.21.15.117:0/3169192050 shutdown_connections 2024-09-21T05:55:48.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:48.489+0000 7f680bb99640 1 -- 172.21.15.117:0/3169192050 wait complete. 2024-09-21T05:55:48.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:48 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3169192050' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T05:55:48.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:48 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3169192050' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T05:55:48.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:48 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3169192050' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T05:55:49.147 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-21T05:55:49.148 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T05:55:49.395 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:49.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:49 smithi117 ceph-mon[30947]: pgmap v282: 97 pgs: 97 active+clean; 645 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.9 KiB/s wr, 12 op/s; 55 B/s, 0 objects/s recovering 2024-09-21T05:55:49.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:49 smithi138 ceph-mon[34391]: pgmap v282: 97 pgs: 97 active+clean; 645 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.9 KiB/s wr, 12 op/s; 55 B/s, 0 objects/s recovering 2024-09-21T05:55:49.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.888+0000 7f378d927640 1 -- 172.21.15.117:0/863116847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37881049d0 msgr2=0x7f3788104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:49.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.888+0000 7f378d927640 1 --2- 172.21.15.117:0/863116847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37881049d0 0x7f3788104de0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3778009900 tx=0x7f377802a410 comp rx=0 tx=0).stop 2024-09-21T05:55:49.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.889+0000 7f378d927640 1 -- 172.21.15.117:0/863116847 shutdown_connections 2024-09-21T05:55:49.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.889+0000 7f378d927640 1 --2- 172.21.15.117:0/863116847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f378810de30 0x7f3788110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:49.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.889+0000 7f378d927640 1 --2- 172.21.15.117:0/863116847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3788105320 0x7f378810d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:49.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.889+0000 7f378d927640 1 --2- 172.21.15.117:0/863116847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37881049d0 0x7f3788104de0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:49.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.889+0000 7f378d927640 1 -- 172.21.15.117:0/863116847 >> 172.21.15.117:0/863116847 conn(0x7f3788079740 msgr2=0x7f3788079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:49.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.889+0000 7f378d927640 1 -- 172.21.15.117:0/863116847 shutdown_connections 2024-09-21T05:55:49.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.889+0000 7f378d927640 1 -- 172.21.15.117:0/863116847 wait complete. 2024-09-21T05:55:49.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.889+0000 7f378d927640 1 Processor -- start 2024-09-21T05:55:49.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.889+0000 7f378d927640 1 -- start start 2024-09-21T05:55:49.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.890+0000 7f378d927640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37881049d0 0x7f378819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:49.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.890+0000 7f378d927640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3788105320 0x7f378819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:49.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.890+0000 7f378d927640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f378810de30 0x7f378819fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:49.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.890+0000 7f378d927640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3788113af0 con 0x7f378810de30 2024-09-21T05:55:49.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.890+0000 7f378d927640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3788113970 con 0x7f37881049d0 2024-09-21T05:55:49.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.890+0000 7f378d927640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3788113c70 con 0x7f3788105320 2024-09-21T05:55:49.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.890+0000 7f37877fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f378810de30 0x7f378819fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:49.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.890+0000 7f37877fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f378810de30 0x7f378819fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43168/0 (socket says 172.21.15.117:43168) 2024-09-21T05:55:49.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.890+0000 7f37877fe640 1 -- 172.21.15.117:0/3727525377 learned_addr learned my addr 172.21.15.117:0/3727525377 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:55:49.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.890+0000 7f3786ffd640 1 --2- 172.21.15.117:0/3727525377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37881049d0 0x7f378819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:49.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.890+0000 7f37867fc640 1 --2- 172.21.15.117:0/3727525377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3788105320 0x7f378819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:49.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.890+0000 7f37877fe640 1 -- 172.21.15.117:0/3727525377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3788105320 msgr2=0x7f378819aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:49.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.890+0000 7f37877fe640 1 --2- 172.21.15.117:0/3727525377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3788105320 0x7f378819aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:49.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.890+0000 7f37877fe640 1 -- 172.21.15.117:0/3727525377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37881049d0 msgr2=0x7f378819a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:49.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.890+0000 7f37877fe640 1 --2- 172.21.15.117:0/3727525377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37881049d0 0x7f378819a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:49.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.890+0000 7f37877fe640 1 -- 172.21.15.117:0/3727525377 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37881a04b0 con 0x7f378810de30 2024-09-21T05:55:49.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.890+0000 7f3786ffd640 1 --2- 172.21.15.117:0/3727525377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37881049d0 0x7f378819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:55:49.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.890+0000 7f37877fe640 1 --2- 172.21.15.117:0/3727525377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f378810de30 0x7f378819fd80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f378000eb10 tx=0x7f378000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:49.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.891+0000 7f378c925640 1 -- 172.21.15.117:0/3727525377 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378000ce20 con 0x7f378810de30 2024-09-21T05:55:49.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.891+0000 7f378c925640 1 -- 172.21.15.117:0/3727525377 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3780005020 con 0x7f378810de30 2024-09-21T05:55:49.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.891+0000 7f378c925640 1 -- 172.21.15.117:0/3727525377 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3780010690 con 0x7f378810de30 2024-09-21T05:55:49.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.891+0000 7f378d927640 1 -- 172.21.15.117:0/3727525377 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37881a07a0 con 0x7f378810de30 2024-09-21T05:55:49.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.891+0000 7f378d927640 1 -- 172.21.15.117:0/3727525377 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37881a6f00 con 0x7f378810de30 2024-09-21T05:55:49.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.892+0000 7f378d927640 1 -- 172.21.15.117:0/3727525377 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f378806c240 con 0x7f378810de30 2024-09-21T05:55:49.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.893+0000 7f378c925640 1 -- 172.21.15.117:0/3727525377 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f3780005510 con 0x7f378810de30 2024-09-21T05:55:49.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.894+0000 7f378c925640 1 --2- 172.21.15.117:0/3727525377 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f37700b65f0 0x7f37700b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:49.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.894+0000 7f3786ffd640 1 --2- 172.21.15.117:0/3727525377 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f37700b65f0 0x7f37700b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:49.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.894+0000 7f378c925640 1 -- 172.21.15.117:0/3727525377 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f37800e7de0 con 0x7f378810de30 2024-09-21T05:55:49.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.894+0000 7f3786ffd640 1 --2- 172.21.15.117:0/3727525377 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f37700b65f0 0x7f37700b8b00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3778006010 tx=0x7f3778000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:49.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:49.897+0000 7f378c925640 1 -- 172.21.15.117:0/3727525377 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3780056710 con 0x7f378810de30 2024-09-21T05:55:49.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:49 smithi192 ceph-mon[34700]: pgmap v282: 97 pgs: 97 active+clean; 645 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.9 KiB/s wr, 12 op/s; 55 B/s, 0 objects/s recovering 2024-09-21T05:55:50.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:50.076+0000 7f378d927640 1 -- 172.21.15.117:0/3727525377 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3788104de0 con 0x7f378810de30 2024-09-21T05:55:50.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:50.078+0000 7f378c925640 1 -- 172.21.15.117:0/3727525377 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+23066 (secure 0 0 0) 0x7f3788104de0 con 0x7f378810de30 2024-09-21T05:55:50.080 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:55:50.080 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":76,"fsid":"ac8581a2-77dc-11ef-baf5-efdc52797490","created":"2024-09-21T05:46:47.021846+0000","modified":"2024-09-21T05:55:33.628948+0000","last_up_change":"2024-09-21T05:54:21.572849+0000","last_in_change":"2024-09-21T05:53:58.892597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:50:32.717389+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:55:10.604159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T05:55:12.581258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"68315502-fa6c-4a6e-a2d4-a7dde8071c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6803","nonce":3761478035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6805","nonce":3761478035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6809","nonce":3761478035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6807","nonce":3761478035}]},"public_addr":"172.21.15.117:6803/3761478035","cluster_addr":"172.21.15.117:6805/3761478035","heartbeat_back_addr":"172.21.15.117:6809/3761478035","heartbeat_front_addr":"172.21.15.117:6807/3761478035","state":["exists","up"]},{"osd":1,"uuid":"d8910a08-4ef3-49f2-a4db-e3588d85d12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6801","nonce":1737365588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6803","nonce":1737365588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6807","nonce":1737365588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6805","nonce":1737365588}]},"public_addr":"172.21.15.138:6801/1737365588","cluster_addr":"172.21.15.138:6803/1737365588","heartbeat_back_addr":"172.21.15.138:6807/1737365588","heartbeat_front_addr":"172.21.15.138:6805/1737365588","state":["exists","up"]},{"osd":2,"uuid":"94972378-a11f-4129-9e40-ad3fdec1d189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6801","nonce":3002534992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6803","nonce":3002534992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6807","nonce":3002534992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6805","nonce":3002534992}]},"public_addr":"172.21.15.192:6801/3002534992","cluster_addr":"172.21.15.192:6803/3002534992","heartbeat_back_addr":"172.21.15.192:6807/3002534992","heartbeat_front_addr":"172.21.15.192:6805/3002534992","state":["exists","up"]},{"osd":3,"uuid":"d4168c43-f0de-4445-945c-92d1cf270c33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6811","nonce":1222400293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6813","nonce":1222400293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6817","nonce":1222400293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6815","nonce":1222400293}]},"public_addr":"172.21.15.117:6811/1222400293","cluster_addr":"172.21.15.117:6813/1222400293","heartbeat_back_addr":"172.21.15.117:6817/1222400293","heartbeat_front_addr":"172.21.15.117:6815/1222400293","state":["exists","up"]},{"osd":4,"uuid":"28f1d7c0-8392-4a33-85c0-0de611cdd4b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6809","nonce":4294892870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6811","nonce":4294892870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6815","nonce":4294892870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6813","nonce":4294892870}]},"public_addr":"172.21.15.138:6809/4294892870","cluster_addr":"172.21.15.138:6811/4294892870","heartbeat_back_addr":"172.21.15.138:6815/4294892870","heartbeat_front_addr":"172.21.15.138:6813/4294892870","state":["exists","up"]},{"osd":5,"uuid":"ca902407-45e8-4bc5-bdbd-a729c1de8322","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6809","nonce":1776840019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6811","nonce":1776840019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6815","nonce":1776840019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6813","nonce":1776840019}]},"public_addr":"172.21.15.192:6809/1776840019","cluster_addr":"172.21.15.192:6811/1776840019","heartbeat_back_addr":"172.21.15.192:6815/1776840019","heartbeat_front_addr":"172.21.15.192:6813/1776840019","state":["exists","up"]},{"osd":6,"uuid":"6e507a91-ed09-4a33-bf0e-c4b593b3c5aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6819","nonce":1093341590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6821","nonce":1093341590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6825","nonce":1093341590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6823","nonce":1093341590}]},"public_addr":"172.21.15.117:6819/1093341590","cluster_addr":"172.21.15.117:6821/1093341590","heartbeat_back_addr":"172.21.15.117:6825/1093341590","heartbeat_front_addr":"172.21.15.117:6823/1093341590","state":["exists","up"]},{"osd":7,"uuid":"36a47dd7-67f7-4ebe-8759-3a8bb6bd2ea3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6817","nonce":1448179714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6819","nonce":1448179714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6823","nonce":1448179714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6821","nonce":1448179714}]},"public_addr":"172.21.15.138:6817/1448179714","cluster_addr":"172.21.15.138:6819/1448179714","heartbeat_back_addr":"172.21.15.138:6823/1448179714","heartbeat_front_addr":"172.21.15.138:6821/1448179714","state":["exists","up"]},{"osd":8,"uuid":"118e0b5d-2619-4512-94c5-8d07a0d3a28f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6817","nonce":3960902885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6819","nonce":3960902885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6823","nonce":3960902885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6821","nonce":3960902885}]},"public_addr":"172.21.15.192:6817/3960902885","cluster_addr":"172.21.15.192:6819/3960902885","heartbeat_back_addr":"172.21.15.192:6823/3960902885","heartbeat_front_addr":"172.21.15.192:6821/3960902885","state":["exists","up"]},{"osd":9,"uuid":"47543bc6-fa0b-4e72-9742-97dcda0b7f8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6827","nonce":3453855195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6829","nonce":3453855195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6833","nonce":3453855195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6831","nonce":3453855195}]},"public_addr":"172.21.15.117:6827/3453855195","cluster_addr":"172.21.15.117:6829/3453855195","heartbeat_back_addr":"172.21.15.117:6833/3453855195","heartbeat_front_addr":"172.21.15.117:6831/3453855195","state":["exists","up"]},{"osd":10,"uuid":"5e6708b4-7a85-4fa5-a03c-27d99f7d749f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6825","nonce":903392451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6827","nonce":903392451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6831","nonce":903392451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6829","nonce":903392451}]},"public_addr":"172.21.15.138:6825/903392451","cluster_addr":"172.21.15.138:6827/903392451","heartbeat_back_addr":"172.21.15.138:6831/903392451","heartbeat_front_addr":"172.21.15.138:6829/903392451","state":["exists","up"]},{"osd":11,"uuid":"24bc2ef7-e2b4-43c6-9e6c-43ae02390415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6825","nonce":899090479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6827","nonce":899090479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6831","nonce":899090479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6829","nonce":899090479}]},"public_addr":"172.21.15.192:6825/899090479","cluster_addr":"172.21.15.192:6827/899090479","heartbeat_back_addr":"172.21.15.192:6831/899090479","heartbeat_front_addr":"172.21.15.192:6829/899090479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:49:40.742217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:05.963655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:30.164495+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:56.598899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:51:21.940331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:51:47.777445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:52:12.869007+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:52:39.454054+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:03.569226+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:29.760863+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:55.788604+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:54:20.510184+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:0/2989394419":"2024-09-22T05:48:32.406654+0000","172.21.15.117:6801/1702676172":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/1766406059":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6800/2337036514":"2024-09-22T05:47:09.112435+0000","172.21.15.117:0/985798343":"2024-09-22T05:47:09.112435+0000","172.21.15.117:0/1083273222":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/473284565":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6801/2337036514":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6800/816635165":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/2642940197":"2024-09-22T05:48:32.406654+0000","172.21.15.117:6801/816635165":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/2225537433":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/3651584169":"2024-09-22T05:47:33.789146+0000","172.21.15.117:6800/1702676172":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/1284403882":"2024-09-22T05:47:33.789146+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:55:50.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:50.080+0000 7f378d927640 1 -- 172.21.15.117:0/3727525377 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f37700b65f0 msgr2=0x7f37700b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:50.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:50.080+0000 7f378d927640 1 --2- 172.21.15.117:0/3727525377 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f37700b65f0 0x7f37700b8b00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3778006010 tx=0x7f3778000bc0 comp rx=0 tx=0).stop 2024-09-21T05:55:50.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:50.080+0000 7f378d927640 1 -- 172.21.15.117:0/3727525377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f378810de30 msgr2=0x7f378819fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:50.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:50.080+0000 7f378d927640 1 --2- 172.21.15.117:0/3727525377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f378810de30 0x7f378819fd80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f378000eb10 tx=0x7f378000ef10 comp rx=0 tx=0).stop 2024-09-21T05:55:50.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:50.081+0000 7f378d927640 1 -- 172.21.15.117:0/3727525377 shutdown_connections 2024-09-21T05:55:50.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:50.081+0000 7f378d927640 1 --2- 172.21.15.117:0/3727525377 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f37700b65f0 0x7f37700b8b00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:50.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:50.081+0000 7f378d927640 1 --2- 172.21.15.117:0/3727525377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f378810de30 0x7f378819fd80 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:50.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:50.081+0000 7f378d927640 1 --2- 172.21.15.117:0/3727525377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3788105320 0x7f378819aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:50.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:50.081+0000 7f378d927640 1 --2- 172.21.15.117:0/3727525377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37881049d0 0x7f378819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:50.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:50.081+0000 7f378d927640 1 -- 172.21.15.117:0/3727525377 >> 172.21.15.117:0/3727525377 conn(0x7f3788079740 msgr2=0x7f37880ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:50.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:50.081+0000 7f378d927640 1 -- 172.21.15.117:0/3727525377 shutdown_connections 2024-09-21T05:55:50.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:50.081+0000 7f378d927640 1 -- 172.21.15.117:0/3727525377 wait complete. 2024-09-21T05:55:50.793 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:50 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3727525377' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T05:55:50.822 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell osd.0 flush_pg_stats 2024-09-21T05:55:50.822 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell osd.1 flush_pg_stats 2024-09-21T05:55:50.823 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell osd.2 flush_pg_stats 2024-09-21T05:55:50.823 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell osd.3 flush_pg_stats 2024-09-21T05:55:50.823 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell osd.4 flush_pg_stats 2024-09-21T05:55:50.824 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell osd.5 flush_pg_stats 2024-09-21T05:55:50.824 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell osd.6 flush_pg_stats 2024-09-21T05:55:50.825 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell osd.7 flush_pg_stats 2024-09-21T05:55:50.825 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell osd.8 flush_pg_stats 2024-09-21T05:55:50.826 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell osd.9 flush_pg_stats 2024-09-21T05:55:50.826 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell osd.10 flush_pg_stats 2024-09-21T05:55:50.826 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell osd.11 flush_pg_stats 2024-09-21T05:55:50.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:50 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3727525377' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T05:55:50.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:50 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3727525377' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T05:55:51.571 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:51.586 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:51.588 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:51.594 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:51.594 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:51.594 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:51.598 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:51.598 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:51.602 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:51.604 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:51.615 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:51.619 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:55:51.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:51 smithi138 ceph-mon[34391]: pgmap v283: 97 pgs: 97 active+clean; 645 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-21T05:55:51.869 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:51 smithi117 ceph-mon[30947]: pgmap v283: 97 pgs: 97 active+clean; 645 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-21T05:55:51.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:51 smithi192 ceph-mon[34700]: pgmap v283: 97 pgs: 97 active+clean; 645 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-21T05:55:52.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.630+0000 7fc72917f640 1 -- 172.21.15.117:0/421610065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc72410de30 msgr2=0x7fc724110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:52.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.630+0000 7fc72917f640 1 --2- 172.21.15.117:0/421610065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc72410de30 0x7fc724110320 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc71c009ca0 tx=0x7fc71c02a8c0 comp rx=0 tx=0).stop 2024-09-21T05:55:52.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.631+0000 7fc72917f640 1 -- 172.21.15.117:0/421610065 shutdown_connections 2024-09-21T05:55:52.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.631+0000 7fc72917f640 1 --2- 172.21.15.117:0/421610065 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc72410de30 0x7fc724110320 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:52.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.631+0000 7fc72917f640 1 --2- 172.21.15.117:0/421610065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc724105320 0x7fc72410d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:52.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.631+0000 7fc72917f640 1 --2- 172.21.15.117:0/421610065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7241049d0 0x7fc724104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:52.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.631+0000 7fc72917f640 1 -- 172.21.15.117:0/421610065 >> 172.21.15.117:0/421610065 conn(0x7fc724079740 msgr2=0x7fc724079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:52.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.631+0000 7fc72917f640 1 -- 172.21.15.117:0/421610065 shutdown_connections 2024-09-21T05:55:52.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.631+0000 7fc72917f640 1 -- 172.21.15.117:0/421610065 wait complete. 2024-09-21T05:55:52.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.632+0000 7fc72917f640 1 Processor -- start 2024-09-21T05:55:52.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.632+0000 7fc72917f640 1 -- start start 2024-09-21T05:55:52.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.632+0000 7fc72917f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7241049d0 0x7fc72419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:52.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.632+0000 7fc72917f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc724105320 0x7fc72419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:52.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.632+0000 7fc72917f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc72410de30 0x7fc72419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:52.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.632+0000 7fc72917f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc724113ab0 con 0x7fc72410de30 2024-09-21T05:55:52.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.632+0000 7fc72917f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc724113930 con 0x7fc7241049d0 2024-09-21T05:55:52.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.632+0000 7fc72917f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc724113c30 con 0x7fc724105320 2024-09-21T05:55:52.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.632+0000 7fc72359e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc72410de30 0x7fc72419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:52.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.632+0000 7fc72359e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc72410de30 0x7fc72419fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43178/0 (socket says 172.21.15.117:43178) 2024-09-21T05:55:52.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.632+0000 7fc72359e640 1 -- 172.21.15.117:0/3159725782 learned_addr learned my addr 172.21.15.117:0/3159725782 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:55:52.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.632+0000 7fc722d9d640 1 --2- 172.21.15.117:0/3159725782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7241049d0 0x7fc72419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:52.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.632+0000 7fc72259c640 1 --2- 172.21.15.117:0/3159725782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc724105320 0x7fc72419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:52.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.633+0000 7fc72359e640 1 -- 172.21.15.117:0/3159725782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc724105320 msgr2=0x7fc72419aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:52.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.633+0000 7fc72359e640 1 --2- 172.21.15.117:0/3159725782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc724105320 0x7fc72419aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:52.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.633+0000 7fc72359e640 1 -- 172.21.15.117:0/3159725782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7241049d0 msgr2=0x7fc72419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:52.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.633+0000 7fc72359e640 1 --2- 172.21.15.117:0/3159725782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7241049d0 0x7fc72419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:52.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.633+0000 7fc72359e640 1 -- 172.21.15.117:0/3159725782 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7241a03f0 con 0x7fc72410de30 2024-09-21T05:55:52.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.633+0000 7fc72359e640 1 --2- 172.21.15.117:0/3159725782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc72410de30 0x7fc72419fcc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc71c005cc0 tx=0x7fc71c02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:52.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.633+0000 7fc70bfff640 1 -- 172.21.15.117:0/3159725782 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c003bf0 con 0x7fc72410de30 2024-09-21T05:55:52.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.633+0000 7fc70bfff640 1 -- 172.21.15.117:0/3159725782 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc71c003d90 con 0x7fc72410de30 2024-09-21T05:55:52.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.633+0000 7fc72917f640 1 -- 172.21.15.117:0/3159725782 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7241a0680 con 0x7fc72410de30 2024-09-21T05:55:52.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.633+0000 7fc70bfff640 1 -- 172.21.15.117:0/3159725782 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c02fbc0 con 0x7fc72410de30 2024-09-21T05:55:52.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.633+0000 7fc72917f640 1 -- 172.21.15.117:0/3159725782 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7241a6e40 con 0x7fc72410de30 2024-09-21T05:55:52.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.634+0000 7fc72917f640 1 -- 172.21.15.117:0/3159725782 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc724100fe0 con 0x7fc72410de30 2024-09-21T05:55:52.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.635+0000 7fc70bfff640 1 -- 172.21.15.117:0/3159725782 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7fc71c02fd60 con 0x7fc72410de30 2024-09-21T05:55:52.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.636+0000 7fc70bfff640 1 --2- 172.21.15.117:0/3159725782 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc70c0b65f0 0x7fc70c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:52.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.636+0000 7fc722d9d640 1 --2- 172.21.15.117:0/3159725782 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc70c0b65f0 0x7fc70c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:52.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.636+0000 7fc70bfff640 1 -- 172.21.15.117:0/3159725782 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fc71c108c40 con 0x7fc72410de30 2024-09-21T05:55:52.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.637+0000 7fc70bfff640 1 --2- 172.21.15.117:0/3159725782 >> [v2:172.21.15.192:6808/1776840019,v1:172.21.15.192:6809/1776840019] conn(0x7fc70c0c02e0 0x7fc70c0c27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:52.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.637+0000 7fc70bfff640 1 -- 172.21.15.117:0/3159725782 --> [v2:172.21.15.192:6808/1776840019,v1:172.21.15.192:6809/1776840019] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc70c0c2ea0 con 0x7fc70c0c02e0 2024-09-21T05:55:52.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.637+0000 7fc70bfff640 1 -- 172.21.15.117:0/3159725782 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fc71c109090 con 0x7fc72410de30 2024-09-21T05:55:52.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.637+0000 7fc722d9d640 1 --2- 172.21.15.117:0/3159725782 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc70c0b65f0 0x7fc70c0b8b00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc72406c620 tx=0x7fc71400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:52.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.637+0000 7fc72259c640 1 --2- 172.21.15.117:0/3159725782 >> [v2:172.21.15.192:6808/1776840019,v1:172.21.15.192:6809/1776840019] conn(0x7fc70c0c02e0 0x7fc70c0c27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:52.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.637+0000 7fc72259c640 1 --2- 172.21.15.117:0/3159725782 >> [v2:172.21.15.192:6808/1776840019,v1:172.21.15.192:6809/1776840019] conn(0x7fc70c0c02e0 0x7fc70c0c27d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:52.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.638+0000 7fc70bfff640 1 -- 172.21.15.117:0/3159725782 <== osd.5 v2:172.21.15.192:6808/1776840019 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc70c0c2ea0 con 0x7fc70c0c02e0 2024-09-21T05:55:52.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.652+0000 7fc72917f640 1 -- 172.21.15.117:0/3159725782 --> [v2:172.21.15.192:6808/1776840019,v1:172.21.15.192:6809/1776840019] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc72406bf20 con 0x7fc70c0c02e0 2024-09-21T05:55:52.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.653+0000 7fc70bfff640 1 -- 172.21.15.117:0/3159725782 <== osd.5 v2:172.21.15.192:6808/1776840019 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc72406bf20 con 0x7fc70c0c02e0 2024-09-21T05:55:52.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.653+0000 7fc72917f640 1 -- 172.21.15.117:0/3159725782 >> [v2:172.21.15.192:6808/1776840019,v1:172.21.15.192:6809/1776840019] conn(0x7fc70c0c02e0 msgr2=0x7fc70c0c27d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:52.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.653+0000 7fc72917f640 1 --2- 172.21.15.117:0/3159725782 >> [v2:172.21.15.192:6808/1776840019,v1:172.21.15.192:6809/1776840019] conn(0x7fc70c0c02e0 0x7fc70c0c27d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:52.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.653+0000 7fc72917f640 1 -- 172.21.15.117:0/3159725782 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc70c0b65f0 msgr2=0x7fc70c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:52.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.653+0000 7fc72917f640 1 --2- 172.21.15.117:0/3159725782 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc70c0b65f0 0x7fc70c0b8b00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc72406c620 tx=0x7fc71400b390 comp rx=0 tx=0).stop 2024-09-21T05:55:52.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.653+0000 7fc72917f640 1 -- 172.21.15.117:0/3159725782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc72410de30 msgr2=0x7fc72419fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:52.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.653+0000 7fc72917f640 1 --2- 172.21.15.117:0/3159725782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc72410de30 0x7fc72419fcc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc71c005cc0 tx=0x7fc71c02f380 comp rx=0 tx=0).stop 2024-09-21T05:55:52.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.653+0000 7fc72359e640 1 -- 172.21.15.117:0/3159725782 reap_dead start 2024-09-21T05:55:52.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.653+0000 7fc72917f640 1 -- 172.21.15.117:0/3159725782 shutdown_connections 2024-09-21T05:55:52.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.653+0000 7fc72917f640 1 -- 172.21.15.117:0/3159725782 >> 172.21.15.117:0/3159725782 conn(0x7fc724079740 msgr2=0x7fc7240ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:52.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.653+0000 7fc72917f640 1 -- 172.21.15.117:0/3159725782 shutdown_connections 2024-09-21T05:55:52.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:52.653+0000 7fc72917f640 1 -- 172.21.15.117:0/3159725782 wait complete. 2024-09-21T05:55:53.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.302+0000 7f580528b640 1 -- 172.21.15.117:0/3040970695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f580010de30 msgr2=0x7f5800110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:53.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.302+0000 7f580528b640 1 --2- 172.21.15.117:0/3040970695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f580010de30 0x7f5800110320 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f57f8009900 tx=0x7f57f802a410 comp rx=0 tx=0).stop 2024-09-21T05:55:53.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.302+0000 7f580528b640 1 -- 172.21.15.117:0/3040970695 shutdown_connections 2024-09-21T05:55:53.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.302+0000 7f580528b640 1 --2- 172.21.15.117:0/3040970695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f580010de30 0x7f5800110320 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:53.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.302+0000 7f580528b640 1 --2- 172.21.15.117:0/3040970695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5800105320 0x7f580010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:53.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.302+0000 7f580528b640 1 --2- 172.21.15.117:0/3040970695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58001049d0 0x7f5800104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:53.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.302+0000 7f580528b640 1 -- 172.21.15.117:0/3040970695 >> 172.21.15.117:0/3040970695 conn(0x7f5800079740 msgr2=0x7f5800079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:53.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.302+0000 7f580528b640 1 -- 172.21.15.117:0/3040970695 shutdown_connections 2024-09-21T05:55:53.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.302+0000 7f580528b640 1 -- 172.21.15.117:0/3040970695 wait complete. 2024-09-21T05:55:53.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.303+0000 7f580528b640 1 Processor -- start 2024-09-21T05:55:53.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.303+0000 7f580528b640 1 -- start start 2024-09-21T05:55:53.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.304+0000 7f580528b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58001049d0 0x7f580019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:53.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.304+0000 7f580528b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5800105320 0x7f580019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:53.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.304+0000 7f580528b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f580010de30 0x7f580019fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:53.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.304+0000 7f580528b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5800113b30 con 0x7f5800105320 2024-09-21T05:55:53.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.304+0000 7f580528b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f58001139b0 con 0x7f580010de30 2024-09-21T05:55:53.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.304+0000 7f580528b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5800113cb0 con 0x7f58001049d0 2024-09-21T05:55:53.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.304+0000 7f57fe7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5800105320 0x7f580019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:53.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.304+0000 7f57fe7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5800105320 0x7f580019ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43198/0 (socket says 172.21.15.117:43198) 2024-09-21T05:55:53.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.304+0000 7f57fe7fc640 1 -- 172.21.15.117:0/1104479084 learned_addr learned my addr 172.21.15.117:0/1104479084 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:55:53.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.304+0000 7f57ff7fe640 1 --2- 172.21.15.117:0/1104479084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f580010de30 0x7f580019fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:53.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.304+0000 7f57feffd640 1 --2- 172.21.15.117:0/1104479084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58001049d0 0x7f580019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:53.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.304+0000 7f57fe7fc640 1 -- 172.21.15.117:0/1104479084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58001049d0 msgr2=0x7f580019a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:53.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.304+0000 7f57fe7fc640 1 --2- 172.21.15.117:0/1104479084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58001049d0 0x7f580019a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:53.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.304+0000 7f57fe7fc640 1 -- 172.21.15.117:0/1104479084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f580010de30 msgr2=0x7f580019fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:53.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.304+0000 7f57fe7fc640 1 --2- 172.21.15.117:0/1104479084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f580010de30 0x7f580019fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:53.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.304+0000 7f57fe7fc640 1 -- 172.21.15.117:0/1104479084 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58001a0530 con 0x7f5800105320 2024-09-21T05:55:53.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.304+0000 7f57feffd640 1 --2- 172.21.15.117:0/1104479084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58001049d0 0x7f580019a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:55:53.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.304+0000 7f57fe7fc640 1 --2- 172.21.15.117:0/1104479084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5800105320 0x7f580019ab60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f57e800d740 tx=0x7f57e800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:53.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.304+0000 7f57f7fff640 1 -- 172.21.15.117:0/1104479084 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e8017070 con 0x7f5800105320 2024-09-21T05:55:53.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.304+0000 7f580528b640 1 -- 172.21.15.117:0/1104479084 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58001a0820 con 0x7f5800105320 2024-09-21T05:55:53.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.304+0000 7f57f7fff640 1 -- 172.21.15.117:0/1104479084 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57e800de50 con 0x7f5800105320 2024-09-21T05:55:53.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.304+0000 7f57f7fff640 1 -- 172.21.15.117:0/1104479084 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e800f620 con 0x7f5800105320 2024-09-21T05:55:53.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.305+0000 7f580528b640 1 -- 172.21.15.117:0/1104479084 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58001a6f80 con 0x7f5800105320 2024-09-21T05:55:53.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.306+0000 7f580528b640 1 -- 172.21.15.117:0/1104479084 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5800100f60 con 0x7f5800105320 2024-09-21T05:55:53.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.307+0000 7f57f7fff640 1 -- 172.21.15.117:0/1104479084 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f57e801a070 con 0x7f5800105320 2024-09-21T05:55:53.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.308+0000 7f57f7fff640 1 --2- 172.21.15.117:0/1104479084 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f57ec0b6670 0x7f57ec0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:53.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.308+0000 7f57feffd640 1 --2- 172.21.15.117:0/1104479084 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f57ec0b6670 0x7f57ec0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:53.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.308+0000 7f57f7fff640 1 -- 172.21.15.117:0/1104479084 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f57e80e7380 con 0x7f5800105320 2024-09-21T05:55:53.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.308+0000 7f57f7fff640 1 --2- 172.21.15.117:0/1104479084 >> [v2:172.21.15.138:6816/1448179714,v1:172.21.15.138:6817/1448179714] conn(0x7f57ec0c0360 0x7f57ec0c2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:53.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.308+0000 7f57f7fff640 1 -- 172.21.15.117:0/1104479084 --> [v2:172.21.15.138:6816/1448179714,v1:172.21.15.138:6817/1448179714] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f57ec0c2f20 con 0x7f57ec0c0360 2024-09-21T05:55:53.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.308+0000 7f57f7fff640 1 -- 172.21.15.117:0/1104479084 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f57e80e7730 con 0x7f5800105320 2024-09-21T05:55:53.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.308+0000 7f57feffd640 1 --2- 172.21.15.117:0/1104479084 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f57ec0b6670 0x7f57ec0b8b80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f580006c620 tx=0x7f57f00073c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:53.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.308+0000 7f57ff7fe640 1 --2- 172.21.15.117:0/1104479084 >> [v2:172.21.15.138:6816/1448179714,v1:172.21.15.138:6817/1448179714] conn(0x7f57ec0c0360 0x7f57ec0c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:53.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.309+0000 7f57ff7fe640 1 --2- 172.21.15.117:0/1104479084 >> [v2:172.21.15.138:6816/1448179714,v1:172.21.15.138:6817/1448179714] conn(0x7f57ec0c0360 0x7f57ec0c2850 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:53.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.310+0000 7f57f7fff640 1 -- 172.21.15.117:0/1104479084 <== osd.7 v2:172.21.15.138:6816/1448179714 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f57ec0c2f20 con 0x7f57ec0c0360 2024-09-21T05:55:53.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.324+0000 7f580528b640 1 -- 172.21.15.117:0/1104479084 --> [v2:172.21.15.138:6816/1448179714,v1:172.21.15.138:6817/1448179714] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f580006bf20 con 0x7f57ec0c0360 2024-09-21T05:55:53.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.325+0000 7f57f7fff640 1 -- 172.21.15.117:0/1104479084 <== osd.7 v2:172.21.15.138:6816/1448179714 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f580006bf20 con 0x7f57ec0c0360 2024-09-21T05:55:53.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.325+0000 7f580528b640 1 -- 172.21.15.117:0/1104479084 >> [v2:172.21.15.138:6816/1448179714,v1:172.21.15.138:6817/1448179714] conn(0x7f57ec0c0360 msgr2=0x7f57ec0c2850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:53.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.325+0000 7f580528b640 1 --2- 172.21.15.117:0/1104479084 >> [v2:172.21.15.138:6816/1448179714,v1:172.21.15.138:6817/1448179714] conn(0x7f57ec0c0360 0x7f57ec0c2850 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:53.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.325+0000 7f580528b640 1 -- 172.21.15.117:0/1104479084 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f57ec0b6670 msgr2=0x7f57ec0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:53.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.325+0000 7f580528b640 1 --2- 172.21.15.117:0/1104479084 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f57ec0b6670 0x7f57ec0b8b80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f580006c620 tx=0x7f57f00073c0 comp rx=0 tx=0).stop 2024-09-21T05:55:53.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.325+0000 7f580528b640 1 -- 172.21.15.117:0/1104479084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5800105320 msgr2=0x7f580019ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:53.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.325+0000 7f580528b640 1 --2- 172.21.15.117:0/1104479084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5800105320 0x7f580019ab60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f57e800d740 tx=0x7f57e800dc00 comp rx=0 tx=0).stop 2024-09-21T05:55:53.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.325+0000 7f57ff7fe640 1 -- 172.21.15.117:0/1104479084 reap_dead start 2024-09-21T05:55:53.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.325+0000 7f580528b640 1 -- 172.21.15.117:0/1104479084 shutdown_connections 2024-09-21T05:55:53.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.325+0000 7f580528b640 1 -- 172.21.15.117:0/1104479084 >> 172.21.15.117:0/1104479084 conn(0x7f5800079740 msgr2=0x7f58000ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:53.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.325+0000 7f580528b640 1 -- 172.21.15.117:0/1104479084 shutdown_connections 2024-09-21T05:55:53.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.325+0000 7f580528b640 1 -- 172.21.15.117:0/1104479084 wait complete. 2024-09-21T05:55:53.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.496+0000 7fb26734a640 1 -- 172.21.15.117:0/791656019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb260102a60 msgr2=0x7fb260102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:53.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.496+0000 7fb26734a640 1 --2- 172.21.15.117:0/791656019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb260102a60 0x7fb260102e70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009900 tx=0x7fb25c02a410 comp rx=0 tx=0).stop 2024-09-21T05:55:53.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.496+0000 7fb26734a640 1 -- 172.21.15.117:0/791656019 shutdown_connections 2024-09-21T05:55:53.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.496+0000 7fb26734a640 1 --2- 172.21.15.117:0/791656019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb260108cf0 0x7fb26010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:53.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.496+0000 7fb26734a640 1 --2- 172.21.15.117:0/791656019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2601083a0 0x7fb2601087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:53.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.496+0000 7fb26734a640 1 --2- 172.21.15.117:0/791656019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb260102a60 0x7fb260102e70 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:53.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.496+0000 7fb26734a640 1 -- 172.21.15.117:0/791656019 >> 172.21.15.117:0/791656019 conn(0x7fb260079730 msgr2=0x7fb260079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:53.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.496+0000 7fb26734a640 1 -- 172.21.15.117:0/791656019 shutdown_connections 2024-09-21T05:55:53.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.496+0000 7fb26734a640 1 -- 172.21.15.117:0/791656019 wait complete. 2024-09-21T05:55:53.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.497+0000 7fb26734a640 1 Processor -- start 2024-09-21T05:55:53.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.497+0000 7fb26734a640 1 -- start start 2024-09-21T05:55:53.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.497+0000 7fb26734a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb260102a60 0x7fb260199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:53.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.497+0000 7fb26734a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2601083a0 0x7fb260199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:53.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.497+0000 7fb26734a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb260108cf0 0x7fb26019ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:53.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.497+0000 7fb26734a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb260112960 con 0x7fb260102a60 2024-09-21T05:55:53.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.497+0000 7fb26734a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2601127e0 con 0x7fb260108cf0 2024-09-21T05:55:53.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.497+0000 7fb26734a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb260112ae0 con 0x7fb2601083a0 2024-09-21T05:55:53.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.497+0000 7fb2650e6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb260102a60 0x7fb260199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:53.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.497+0000 7fb2650e6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb260102a60 0x7fb260199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43216/0 (socket says 172.21.15.117:43216) 2024-09-21T05:55:53.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.497+0000 7fb2650e6640 1 -- 172.21.15.117:0/1606487696 learned_addr learned my addr 172.21.15.117:0/1606487696 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:55:53.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.497+0000 7fb2648e5640 1 --2- 172.21.15.117:0/1606487696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2601083a0 0x7fb260199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:53.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.498+0000 7fb2658e7640 1 --2- 172.21.15.117:0/1606487696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb260108cf0 0x7fb26019ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:53.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.498+0000 7fb2650e6640 1 -- 172.21.15.117:0/1606487696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2601083a0 msgr2=0x7fb260199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:53.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.498+0000 7fb2650e6640 1 --2- 172.21.15.117:0/1606487696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2601083a0 0x7fb260199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:53.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.498+0000 7fb2650e6640 1 -- 172.21.15.117:0/1606487696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb260108cf0 msgr2=0x7fb26019ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:53.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.498+0000 7fb2650e6640 1 --2- 172.21.15.117:0/1606487696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb260108cf0 0x7fb26019ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:53.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.498+0000 7fb2650e6640 1 -- 172.21.15.117:0/1606487696 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb26019f220 con 0x7fb260102a60 2024-09-21T05:55:53.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.498+0000 7fb2658e7640 1 --2- 172.21.15.117:0/1606487696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb260108cf0 0x7fb26019ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:55:53.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.498+0000 7fb2650e6640 1 --2- 172.21.15.117:0/1606487696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb260102a60 0x7fb260199370 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb25c006010 tx=0x7fb25c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:53.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.498+0000 7fb2567fc640 1 -- 172.21.15.117:0/1606487696 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c03f030 con 0x7fb260102a60 2024-09-21T05:55:53.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.498+0000 7fb2567fc640 1 -- 172.21.15.117:0/1606487696 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb25c003800 con 0x7fb260102a60 2024-09-21T05:55:53.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.498+0000 7fb26734a640 1 -- 172.21.15.117:0/1606487696 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb26019f4b0 con 0x7fb260102a60 2024-09-21T05:55:53.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.498+0000 7fb2567fc640 1 -- 172.21.15.117:0/1606487696 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c035070 con 0x7fb260102a60 2024-09-21T05:55:53.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.498+0000 7fb26734a640 1 -- 172.21.15.117:0/1606487696 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb26019f990 con 0x7fb260102a60 2024-09-21T05:55:53.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.499+0000 7fb26734a640 1 -- 172.21.15.117:0/1606487696 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb260063b50 con 0x7fb260102a60 2024-09-21T05:55:53.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.501+0000 7fb2567fc640 1 -- 172.21.15.117:0/1606487696 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7fb25c003cb0 con 0x7fb260102a60 2024-09-21T05:55:53.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.501+0000 7fb2567fc640 1 --2- 172.21.15.117:0/1606487696 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb24c0b6620 0x7fb24c0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:53.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.501+0000 7fb2648e5640 1 --2- 172.21.15.117:0/1606487696 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb24c0b6620 0x7fb24c0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:53.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.501+0000 7fb2567fc640 1 -- 172.21.15.117:0/1606487696 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fb25c108660 con 0x7fb260102a60 2024-09-21T05:55:53.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.501+0000 7fb2567fc640 1 --2- 172.21.15.117:0/1606487696 >> [v2:172.21.15.117:6818/1093341590,v1:172.21.15.117:6819/1093341590] conn(0x7fb24c0c0310 0x7fb24c0c2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:53.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.501+0000 7fb2567fc640 1 -- 172.21.15.117:0/1606487696 --> [v2:172.21.15.117:6818/1093341590,v1:172.21.15.117:6819/1093341590] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb24c0c2ed0 con 0x7fb24c0c0310 2024-09-21T05:55:53.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.501+0000 7fb2567fc640 1 -- 172.21.15.117:0/1606487696 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fb25c108ab0 con 0x7fb260102a60 2024-09-21T05:55:53.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.502+0000 7fb2658e7640 1 --2- 172.21.15.117:0/1606487696 >> [v2:172.21.15.117:6818/1093341590,v1:172.21.15.117:6819/1093341590] conn(0x7fb24c0c0310 0x7fb24c0c2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:53.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.502+0000 7fb2648e5640 1 --2- 172.21.15.117:0/1606487696 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb24c0b6620 0x7fb24c0b8b30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb250005fd0 tx=0x7fb250005e20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:53.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.502+0000 7fb2658e7640 1 --2- 172.21.15.117:0/1606487696 >> [v2:172.21.15.117:6818/1093341590,v1:172.21.15.117:6819/1093341590] conn(0x7fb24c0c0310 0x7fb24c0c2800 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:53.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.503+0000 7fb2567fc640 1 -- 172.21.15.117:0/1606487696 <== osd.6 v2:172.21.15.117:6818/1093341590 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb24c0c2ed0 con 0x7fb24c0c0310 2024-09-21T05:55:53.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.516+0000 7fb26734a640 1 -- 172.21.15.117:0/1606487696 --> [v2:172.21.15.117:6818/1093341590,v1:172.21.15.117:6819/1093341590] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb260063fd0 con 0x7fb24c0c0310 2024-09-21T05:55:53.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.517+0000 7fb2567fc640 1 -- 172.21.15.117:0/1606487696 <== osd.6 v2:172.21.15.117:6818/1093341590 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb260063fd0 con 0x7fb24c0c0310 2024-09-21T05:55:53.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.517+0000 7fb26734a640 1 -- 172.21.15.117:0/1606487696 >> [v2:172.21.15.117:6818/1093341590,v1:172.21.15.117:6819/1093341590] conn(0x7fb24c0c0310 msgr2=0x7fb24c0c2800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:53.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.517+0000 7fb26734a640 1 --2- 172.21.15.117:0/1606487696 >> [v2:172.21.15.117:6818/1093341590,v1:172.21.15.117:6819/1093341590] conn(0x7fb24c0c0310 0x7fb24c0c2800 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:53.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.517+0000 7fb26734a640 1 -- 172.21.15.117:0/1606487696 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb24c0b6620 msgr2=0x7fb24c0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:53.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.517+0000 7fb26734a640 1 --2- 172.21.15.117:0/1606487696 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb24c0b6620 0x7fb24c0b8b30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb250005fd0 tx=0x7fb250005e20 comp rx=0 tx=0).stop 2024-09-21T05:55:53.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.517+0000 7fb26734a640 1 -- 172.21.15.117:0/1606487696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb260102a60 msgr2=0x7fb260199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:53.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.517+0000 7fb26734a640 1 --2- 172.21.15.117:0/1606487696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb260102a60 0x7fb260199370 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb25c006010 tx=0x7fb25c0042f0 comp rx=0 tx=0).stop 2024-09-21T05:55:53.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.517+0000 7fb2658e7640 1 -- 172.21.15.117:0/1606487696 reap_dead start 2024-09-21T05:55:53.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.517+0000 7fb26734a640 1 -- 172.21.15.117:0/1606487696 shutdown_connections 2024-09-21T05:55:53.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.517+0000 7fb26734a640 1 -- 172.21.15.117:0/1606487696 >> 172.21.15.117:0/1606487696 conn(0x7fb260079730 msgr2=0x7fb260101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:53.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.517+0000 7fb26734a640 1 -- 172.21.15.117:0/1606487696 shutdown_connections 2024-09-21T05:55:53.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:53.517+0000 7fb26734a640 1 -- 172.21.15.117:0/1606487696 wait complete. 2024-09-21T05:55:53.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:53 smithi117 ceph-mon[30947]: pgmap v284: 97 pgs: 97 active+clean; 645 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-21T05:55:53.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:53 smithi138 ceph-mon[34391]: pgmap v284: 97 pgs: 97 active+clean; 645 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-21T05:55:53.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:53 smithi192 ceph-mon[34700]: pgmap v284: 97 pgs: 97 active+clean; 645 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-21T05:55:54.232 INFO:teuthology.orchestra.run.smithi117.stdout:197568495657 2024-09-21T05:55:54.232 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.7 2024-09-21T05:55:55.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.022+0000 7f571bbb1640 1 -- 172.21.15.117:0/1246752025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57141083b0 msgr2=0x7f57141087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:55.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.022+0000 7f571bbb1640 1 --2- 172.21.15.117:0/1246752025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57141083b0 0x7f57141087c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5704009900 tx=0x7f570402a410 comp rx=0 tx=0).stop 2024-09-21T05:55:55.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.022+0000 7f571bbb1640 1 -- 172.21.15.117:0/1246752025 shutdown_connections 2024-09-21T05:55:55.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.022+0000 7f571bbb1640 1 --2- 172.21.15.117:0/1246752025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57141083b0 0x7f57141087c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:55.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.022+0000 7f571bbb1640 1 --2- 172.21.15.117:0/1246752025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5714108d00 0x7f571410f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:55.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.022+0000 7f571bbb1640 1 --2- 172.21.15.117:0/1246752025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5714103180 0x7f5714103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:55.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.022+0000 7f571bbb1640 1 -- 172.21.15.117:0/1246752025 >> 172.21.15.117:0/1246752025 conn(0x7f5714077c60 msgr2=0x7f57140780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:55.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.022+0000 7f571bbb1640 1 -- 172.21.15.117:0/1246752025 shutdown_connections 2024-09-21T05:55:55.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.022+0000 7f571bbb1640 1 -- 172.21.15.117:0/1246752025 wait complete. 2024-09-21T05:55:55.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.023+0000 7f571bbb1640 1 Processor -- start 2024-09-21T05:55:55.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.023+0000 7f571bbb1640 1 -- start start 2024-09-21T05:55:55.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.023+0000 7f571bbb1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5714103180 0x7f5714199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:55.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.023+0000 7f571bbb1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57141083b0 0x7f5714199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:55.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.023+0000 7f571bbb1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5714108d00 0x7f571419ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:55.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.023+0000 7f571bbb1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f57141129c0 con 0x7f57141083b0 2024-09-21T05:55:55.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.023+0000 7f571bbb1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5714112840 con 0x7f5714103180 2024-09-21T05:55:55.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.023+0000 7f571bbb1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5714112b40 con 0x7f5714108d00 2024-09-21T05:55:55.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.024+0000 7f571914c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57141083b0 0x7f5714199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:55.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.024+0000 7f571914c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57141083b0 0x7f5714199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36280/0 (socket says 172.21.15.117:36280) 2024-09-21T05:55:55.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.024+0000 7f571914c640 1 -- 172.21.15.117:0/77791473 learned_addr learned my addr 172.21.15.117:0/77791473 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:55:55.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.024+0000 7f571994d640 1 --2- 172.21.15.117:0/77791473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5714103180 0x7f5714199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:55.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.024+0000 7f571a14e640 1 --2- 172.21.15.117:0/77791473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5714108d00 0x7f571419ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:55.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.024+0000 7f571914c640 1 -- 172.21.15.117:0/77791473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5714108d00 msgr2=0x7f571419ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:55.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.024+0000 7f571914c640 1 --2- 172.21.15.117:0/77791473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5714108d00 0x7f571419ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:55.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.024+0000 7f571914c640 1 -- 172.21.15.117:0/77791473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5714103180 msgr2=0x7f5714199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:55.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.024+0000 7f571914c640 1 --2- 172.21.15.117:0/77791473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5714103180 0x7f5714199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:55.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.024+0000 7f571914c640 1 -- 172.21.15.117:0/77791473 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f571419f2e0 con 0x7f57141083b0 2024-09-21T05:55:55.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.024+0000 7f571994d640 1 --2- 172.21.15.117:0/77791473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5714103180 0x7f5714199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:55:55.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.024+0000 7f571914c640 1 --2- 172.21.15.117:0/77791473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57141083b0 0x7f5714199910 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f570402a930 tx=0x7f5704031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:55.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.024+0000 7f570affd640 1 -- 172.21.15.117:0/77791473 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5704046070 con 0x7f57141083b0 2024-09-21T05:55:55.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.024+0000 7f571bbb1640 1 -- 172.21.15.117:0/77791473 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f571419f570 con 0x7f57141083b0 2024-09-21T05:55:55.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.024+0000 7f570affd640 1 -- 172.21.15.117:0/77791473 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5704004410 con 0x7f57141083b0 2024-09-21T05:55:55.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.024+0000 7f570affd640 1 -- 172.21.15.117:0/77791473 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5704035050 con 0x7f57141083b0 2024-09-21T05:55:55.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.024+0000 7f571bbb1640 1 -- 172.21.15.117:0/77791473 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5714101de0 con 0x7f57141083b0 2024-09-21T05:55:55.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.025+0000 7f571bbb1640 1 -- 172.21.15.117:0/77791473 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5714102410 con 0x7f57141083b0 2024-09-21T05:55:55.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.027+0000 7f570affd640 1 -- 172.21.15.117:0/77791473 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f5704049070 con 0x7f57141083b0 2024-09-21T05:55:55.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.027+0000 7f570affd640 1 --2- 172.21.15.117:0/77791473 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f57000b65f0 0x7f57000b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:55.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.028+0000 7f571994d640 1 --2- 172.21.15.117:0/77791473 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f57000b65f0 0x7f57000b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:55.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.028+0000 7f570affd640 1 -- 172.21.15.117:0/77791473 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f5704109700 con 0x7f57141083b0 2024-09-21T05:55:55.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.028+0000 7f570affd640 1 --2- 172.21.15.117:0/77791473 >> [v2:172.21.15.117:6802/3761478035,v1:172.21.15.117:6803/3761478035] conn(0x7f57000c02e0 0x7f57000c27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:55.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.028+0000 7f570affd640 1 -- 172.21.15.117:0/77791473 --> [v2:172.21.15.117:6802/3761478035,v1:172.21.15.117:6803/3761478035] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f57000c2ea0 con 0x7f57000c02e0 2024-09-21T05:55:55.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.028+0000 7f570affd640 1 -- 172.21.15.117:0/77791473 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f5704109b50 con 0x7f57141083b0 2024-09-21T05:55:55.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.028+0000 7f571a14e640 1 --2- 172.21.15.117:0/77791473 >> [v2:172.21.15.117:6802/3761478035,v1:172.21.15.117:6803/3761478035] conn(0x7f57000c02e0 0x7f57000c27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:55.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.028+0000 7f571994d640 1 --2- 172.21.15.117:0/77791473 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f57000b65f0 0x7f57000b8b00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5710009c20 tx=0x7f5710005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:55.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.028+0000 7f571a14e640 1 --2- 172.21.15.117:0/77791473 >> [v2:172.21.15.117:6802/3761478035,v1:172.21.15.117:6803/3761478035] conn(0x7f57000c02e0 0x7f57000c27d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:55.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.029+0000 7f570affd640 1 -- 172.21.15.117:0/77791473 <== osd.0 v2:172.21.15.117:6802/3761478035 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f57000c2ea0 con 0x7f57000c02e0 2024-09-21T05:55:55.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.042+0000 7f571bbb1640 1 -- 172.21.15.117:0/77791473 --> [v2:172.21.15.117:6802/3761478035,v1:172.21.15.117:6803/3761478035] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5714102810 con 0x7f57000c02e0 2024-09-21T05:55:55.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.042+0000 7f570affd640 1 -- 172.21.15.117:0/77791473 <== osd.0 v2:172.21.15.117:6802/3761478035 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5714102810 con 0x7f57000c02e0 2024-09-21T05:55:55.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.043+0000 7f571bbb1640 1 -- 172.21.15.117:0/77791473 >> [v2:172.21.15.117:6802/3761478035,v1:172.21.15.117:6803/3761478035] conn(0x7f57000c02e0 msgr2=0x7f57000c27d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:55.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.043+0000 7f571bbb1640 1 --2- 172.21.15.117:0/77791473 >> [v2:172.21.15.117:6802/3761478035,v1:172.21.15.117:6803/3761478035] conn(0x7f57000c02e0 0x7f57000c27d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:55.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.043+0000 7f571bbb1640 1 -- 172.21.15.117:0/77791473 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f57000b65f0 msgr2=0x7f57000b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:55.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.043+0000 7f571bbb1640 1 --2- 172.21.15.117:0/77791473 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f57000b65f0 0x7f57000b8b00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5710009c20 tx=0x7f5710005f90 comp rx=0 tx=0).stop 2024-09-21T05:55:55.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.043+0000 7f571bbb1640 1 -- 172.21.15.117:0/77791473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57141083b0 msgr2=0x7f5714199910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:55.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.043+0000 7f571bbb1640 1 --2- 172.21.15.117:0/77791473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57141083b0 0x7f5714199910 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f570402a930 tx=0x7f5704031a20 comp rx=0 tx=0).stop 2024-09-21T05:55:55.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.043+0000 7f571a14e640 1 -- 172.21.15.117:0/77791473 reap_dead start 2024-09-21T05:55:55.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.043+0000 7f571bbb1640 1 -- 172.21.15.117:0/77791473 shutdown_connections 2024-09-21T05:55:55.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.043+0000 7f571bbb1640 1 -- 172.21.15.117:0/77791473 >> 172.21.15.117:0/77791473 conn(0x7f5714077c60 msgr2=0x7f5714106440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:55.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.043+0000 7f571bbb1640 1 -- 172.21.15.117:0/77791473 shutdown_connections 2024-09-21T05:55:55.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:55.043+0000 7f571bbb1640 1 -- 172.21.15.117:0/77791473 wait complete. 2024-09-21T05:55:55.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:55 smithi117 ceph-mon[30947]: pgmap v285: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-21T05:55:55.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:55 smithi138 ceph-mon[34391]: pgmap v285: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-21T05:55:55.945 INFO:teuthology.orchestra.run.smithi117.stdout:38654705742 2024-09-21T05:55:55.945 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.0 2024-09-21T05:55:55.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:55 smithi192 ceph-mon[34700]: pgmap v285: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-21T05:55:56.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.461+0000 7f746627b640 1 -- 172.21.15.117:0/3584594705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7460108d30 msgr2=0x7f746010f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:56.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.461+0000 7f746627b640 1 --2- 172.21.15.117:0/3584594705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7460108d30 0x7f746010f560 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7458009ca0 tx=0x7f745802a8c0 comp rx=0 tx=0).stop 2024-09-21T05:55:56.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.461+0000 7f746627b640 1 -- 172.21.15.117:0/3584594705 shutdown_connections 2024-09-21T05:55:56.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.461+0000 7f746627b640 1 --2- 172.21.15.117:0/3584594705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7460108d30 0x7f746010f560 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:56.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.461+0000 7f746627b640 1 --2- 172.21.15.117:0/3584594705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74601083e0 0x7f74601087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:56.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.461+0000 7f746627b640 1 --2- 172.21.15.117:0/3584594705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7460102aa0 0x7f7460102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:56.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.461+0000 7f746627b640 1 -- 172.21.15.117:0/3584594705 >> 172.21.15.117:0/3584594705 conn(0x7f74600797e0 msgr2=0x7f7460079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:56.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.461+0000 7f746627b640 1 -- 172.21.15.117:0/3584594705 shutdown_connections 2024-09-21T05:55:56.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.461+0000 7f746627b640 1 -- 172.21.15.117:0/3584594705 wait complete. 2024-09-21T05:55:56.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.462+0000 7f746627b640 1 Processor -- start 2024-09-21T05:55:56.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.462+0000 7f746627b640 1 -- start start 2024-09-21T05:55:56.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.462+0000 7f746627b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7460102aa0 0x7f746019b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:56.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.462+0000 7f746627b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74601083e0 0x7f746019b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:56.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.462+0000 7f746627b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7460108d30 0x7f746019bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:56.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.462+0000 7f746627b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f74601129a0 con 0x7f7460108d30 2024-09-21T05:55:56.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.462+0000 7f746627b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7460112820 con 0x7f7460102aa0 2024-09-21T05:55:56.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.462+0000 7f746627b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7460112b20 con 0x7f74601083e0 2024-09-21T05:55:56.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.462+0000 7f7464818640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7460108d30 0x7f746019bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:56.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.462+0000 7f7464818640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7460108d30 0x7f746019bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36300/0 (socket says 172.21.15.117:36300) 2024-09-21T05:55:56.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.462+0000 7f7464818640 1 -- 172.21.15.117:0/3647450599 learned_addr learned my addr 172.21.15.117:0/3647450599 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:55:56.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.462+0000 7f745ffff640 1 --2- 172.21.15.117:0/3647450599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7460102aa0 0x7f746019b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:56.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.462+0000 7f7457fff640 1 --2- 172.21.15.117:0/3647450599 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74601083e0 0x7f746019b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:56.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.463+0000 7f7464818640 1 -- 172.21.15.117:0/3647450599 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74601083e0 msgr2=0x7f746019b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:56.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.463+0000 7f7464818640 1 --2- 172.21.15.117:0/3647450599 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74601083e0 0x7f746019b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:56.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.463+0000 7f7464818640 1 -- 172.21.15.117:0/3647450599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7460102aa0 msgr2=0x7f746019b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:56.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.463+0000 7f7464818640 1 --2- 172.21.15.117:0/3647450599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7460102aa0 0x7f746019b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:56.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.463+0000 7f7464818640 1 -- 172.21.15.117:0/3647450599 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74601a6420 con 0x7f7460108d30 2024-09-21T05:55:56.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.463+0000 7f7464818640 1 --2- 172.21.15.117:0/3647450599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7460108d30 0x7f746019bc70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7458005cc0 tx=0x7f745802f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:56.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.463+0000 7f745dffb640 1 -- 172.21.15.117:0/3647450599 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7458003cf0 con 0x7f7460108d30 2024-09-21T05:55:56.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.463+0000 7f746627b640 1 -- 172.21.15.117:0/3647450599 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7460198fe0 con 0x7f7460108d30 2024-09-21T05:55:56.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.463+0000 7f745dffb640 1 -- 172.21.15.117:0/3647450599 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74580044f0 con 0x7f7460108d30 2024-09-21T05:55:56.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.463+0000 7f745dffb640 1 -- 172.21.15.117:0/3647450599 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f745802fd10 con 0x7f7460108d30 2024-09-21T05:55:56.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.463+0000 7f746627b640 1 -- 172.21.15.117:0/3647450599 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7460199570 con 0x7f7460108d30 2024-09-21T05:55:56.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.464+0000 7f746627b640 1 -- 172.21.15.117:0/3647450599 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7460063b50 con 0x7f7460108d30 2024-09-21T05:55:56.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.466+0000 7f745dffb640 1 -- 172.21.15.117:0/3647450599 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f7458004040 con 0x7f7460108d30 2024-09-21T05:55:56.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.466+0000 7f745dffb640 1 --2- 172.21.15.117:0/3647450599 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f744c0b6670 0x7f744c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:56.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.467+0000 7f745ffff640 1 --2- 172.21.15.117:0/3647450599 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f744c0b6670 0x7f744c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:56.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.467+0000 7f745dffb640 1 -- 172.21.15.117:0/3647450599 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f7458108e60 con 0x7f7460108d30 2024-09-21T05:55:56.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.467+0000 7f745dffb640 1 --2- 172.21.15.117:0/3647450599 >> [v2:172.21.15.138:6800/1737365588,v1:172.21.15.138:6801/1737365588] conn(0x7f744c0c0360 0x7f744c0c2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:56.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.467+0000 7f745dffb640 1 -- 172.21.15.117:0/3647450599 --> [v2:172.21.15.138:6800/1737365588,v1:172.21.15.138:6801/1737365588] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f744c0c2f20 con 0x7f744c0c0360 2024-09-21T05:55:56.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.467+0000 7f745dffb640 1 -- 172.21.15.117:0/3647450599 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f74581092b0 con 0x7f7460108d30 2024-09-21T05:55:56.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.467+0000 7f745ffff640 1 --2- 172.21.15.117:0/3647450599 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f744c0b6670 0x7f744c0b8b80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7450005e10 tx=0x7f7450005da0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:56.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.467+0000 7f7457fff640 1 --2- 172.21.15.117:0/3647450599 >> [v2:172.21.15.138:6800/1737365588,v1:172.21.15.138:6801/1737365588] conn(0x7f744c0c0360 0x7f744c0c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:56.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.468+0000 7f7457fff640 1 --2- 172.21.15.117:0/3647450599 >> [v2:172.21.15.138:6800/1737365588,v1:172.21.15.138:6801/1737365588] conn(0x7f744c0c0360 0x7f744c0c2850 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:56.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.468+0000 7f745dffb640 1 -- 172.21.15.117:0/3647450599 <== osd.1 v2:172.21.15.138:6800/1737365588 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f744c0c2f20 con 0x7f744c0c0360 2024-09-21T05:55:56.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.482+0000 7f746627b640 1 -- 172.21.15.117:0/3647450599 --> [v2:172.21.15.138:6800/1737365588,v1:172.21.15.138:6801/1737365588] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7460063f80 con 0x7f744c0c0360 2024-09-21T05:55:56.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.482+0000 7f745dffb640 1 -- 172.21.15.117:0/3647450599 <== osd.1 v2:172.21.15.138:6800/1737365588 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7460063f80 con 0x7f744c0c0360 2024-09-21T05:55:56.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.483+0000 7f746627b640 1 -- 172.21.15.117:0/3647450599 >> [v2:172.21.15.138:6800/1737365588,v1:172.21.15.138:6801/1737365588] conn(0x7f744c0c0360 msgr2=0x7f744c0c2850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:56.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.483+0000 7f746627b640 1 --2- 172.21.15.117:0/3647450599 >> [v2:172.21.15.138:6800/1737365588,v1:172.21.15.138:6801/1737365588] conn(0x7f744c0c0360 0x7f744c0c2850 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:56.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.483+0000 7f746627b640 1 -- 172.21.15.117:0/3647450599 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f744c0b6670 msgr2=0x7f744c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:56.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.483+0000 7f746627b640 1 --2- 172.21.15.117:0/3647450599 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f744c0b6670 0x7f744c0b8b80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7450005e10 tx=0x7f7450005da0 comp rx=0 tx=0).stop 2024-09-21T05:55:56.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.483+0000 7f746627b640 1 -- 172.21.15.117:0/3647450599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7460108d30 msgr2=0x7f746019bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:56.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.483+0000 7f746627b640 1 --2- 172.21.15.117:0/3647450599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7460108d30 0x7f746019bc70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7458005cc0 tx=0x7f745802f470 comp rx=0 tx=0).stop 2024-09-21T05:55:56.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.483+0000 7f7464818640 1 -- 172.21.15.117:0/3647450599 reap_dead start 2024-09-21T05:55:56.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.483+0000 7f746627b640 1 -- 172.21.15.117:0/3647450599 shutdown_connections 2024-09-21T05:55:56.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.483+0000 7f746627b640 1 -- 172.21.15.117:0/3647450599 >> 172.21.15.117:0/3647450599 conn(0x7f74600797e0 msgr2=0x7f74601043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:56.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.483+0000 7f746627b640 1 -- 172.21.15.117:0/3647450599 shutdown_connections 2024-09-21T05:55:56.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.483+0000 7f746627b640 1 -- 172.21.15.117:0/3647450599 wait complete. 2024-09-21T05:55:56.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.921+0000 7fb025055640 1 -- 172.21.15.117:0/1680939029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0201049b0 msgr2=0x7fb020104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:56.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.921+0000 7fb025055640 1 --2- 172.21.15.117:0/1680939029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0201049b0 0x7fb020104dc0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb010009ca0 tx=0x7fb01002a8c0 comp rx=0 tx=0).stop 2024-09-21T05:55:56.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.921+0000 7fb025055640 1 -- 172.21.15.117:0/1680939029 shutdown_connections 2024-09-21T05:55:56.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.921+0000 7fb025055640 1 --2- 172.21.15.117:0/1680939029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb02010de10 0x7fb020110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:56.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.921+0000 7fb025055640 1 --2- 172.21.15.117:0/1680939029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb020105300 0x7fb02010d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:56.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.921+0000 7fb025055640 1 --2- 172.21.15.117:0/1680939029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0201049b0 0x7fb020104dc0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:56.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.921+0000 7fb025055640 1 -- 172.21.15.117:0/1680939029 >> 172.21.15.117:0/1680939029 conn(0x7fb020079730 msgr2=0x7fb020079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:56.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.921+0000 7fb025055640 1 -- 172.21.15.117:0/1680939029 shutdown_connections 2024-09-21T05:55:56.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.921+0000 7fb025055640 1 -- 172.21.15.117:0/1680939029 wait complete. 2024-09-21T05:55:56.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.922+0000 7fb025055640 1 Processor -- start 2024-09-21T05:55:56.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.922+0000 7fb025055640 1 -- start start 2024-09-21T05:55:56.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.922+0000 7fb025055640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0201049b0 0x7fb02019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:56.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.922+0000 7fb025055640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb020105300 0x7fb02019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:56.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.922+0000 7fb025055640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb02010de10 0x7fb02019fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:56.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.922+0000 7fb025055640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb020113a70 con 0x7fb020105300 2024-09-21T05:55:56.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.922+0000 7fb025055640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb0201138f0 con 0x7fb02010de10 2024-09-21T05:55:56.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.922+0000 7fb025055640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb020113bf0 con 0x7fb0201049b0 2024-09-21T05:55:56.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.922+0000 7fb01e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb020105300 0x7fb02019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:56.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.922+0000 7fb01e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb020105300 0x7fb02019aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36316/0 (socket says 172.21.15.117:36316) 2024-09-21T05:55:56.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.922+0000 7fb01e59c640 1 -- 172.21.15.117:0/2858161873 learned_addr learned my addr 172.21.15.117:0/2858161873 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:55:56.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.922+0000 7fb01ed9d640 1 --2- 172.21.15.117:0/2858161873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0201049b0 0x7fb02019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:56.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.922+0000 7fb01f59e640 1 --2- 172.21.15.117:0/2858161873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb02010de10 0x7fb02019fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:56.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.923+0000 7fb01e59c640 1 -- 172.21.15.117:0/2858161873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0201049b0 msgr2=0x7fb02019a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:56.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.923+0000 7fb01e59c640 1 --2- 172.21.15.117:0/2858161873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0201049b0 0x7fb02019a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:56.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.923+0000 7fb01e59c640 1 -- 172.21.15.117:0/2858161873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb02010de10 msgr2=0x7fb02019fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:56.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.923+0000 7fb01e59c640 1 --2- 172.21.15.117:0/2858161873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb02010de10 0x7fb02019fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:56.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.923+0000 7fb01e59c640 1 -- 172.21.15.117:0/2858161873 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0201a0450 con 0x7fb020105300 2024-09-21T05:55:56.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.923+0000 7fb01e59c640 1 --2- 172.21.15.117:0/2858161873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb020105300 0x7fb02019aa80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb01400e9f0 tx=0x7fb01400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:56.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.923+0000 7fb007fff640 1 -- 172.21.15.117:0/2858161873 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01400ccc0 con 0x7fb020105300 2024-09-21T05:55:56.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.923+0000 7fb025055640 1 -- 172.21.15.117:0/2858161873 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0201a0740 con 0x7fb020105300 2024-09-21T05:55:56.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.923+0000 7fb007fff640 1 -- 172.21.15.117:0/2858161873 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0140108e0 con 0x7fb020105300 2024-09-21T05:55:56.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.923+0000 7fb007fff640 1 -- 172.21.15.117:0/2858161873 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014018860 con 0x7fb020105300 2024-09-21T05:55:56.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.923+0000 7fb025055640 1 -- 172.21.15.117:0/2858161873 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0201a6ea0 con 0x7fb020105300 2024-09-21T05:55:56.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.924+0000 7fb025055640 1 -- 172.21.15.117:0/2858161873 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb020107280 con 0x7fb020105300 2024-09-21T05:55:56.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.926+0000 7fb007fff640 1 -- 172.21.15.117:0/2858161873 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7fb014010430 con 0x7fb020105300 2024-09-21T05:55:56.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.926+0000 7fb007fff640 1 --2- 172.21.15.117:0/2858161873 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb0080b65f0 0x7fb0080b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:56.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.926+0000 7fb01ed9d640 1 --2- 172.21.15.117:0/2858161873 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb0080b65f0 0x7fb0080b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:56.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.926+0000 7fb007fff640 1 -- 172.21.15.117:0/2858161873 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fb0140e7fe0 con 0x7fb020105300 2024-09-21T05:55:56.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.927+0000 7fb007fff640 1 --2- 172.21.15.117:0/2858161873 >> [v2:172.21.15.138:6808/4294892870,v1:172.21.15.138:6809/4294892870] conn(0x7fb0080c02e0 0x7fb0080c27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:56.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.927+0000 7fb007fff640 1 -- 172.21.15.117:0/2858161873 --> [v2:172.21.15.138:6808/4294892870,v1:172.21.15.138:6809/4294892870] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb0080c2ea0 con 0x7fb0080c02e0 2024-09-21T05:55:56.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.927+0000 7fb007fff640 1 -- 172.21.15.117:0/2858161873 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fb0140e8430 con 0x7fb020105300 2024-09-21T05:55:56.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.927+0000 7fb01ed9d640 1 --2- 172.21.15.117:0/2858161873 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb0080b65f0 0x7fb0080b8b00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb010005e30 tx=0x7fb010009360 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:56.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.927+0000 7fb01f59e640 1 --2- 172.21.15.117:0/2858161873 >> [v2:172.21.15.138:6808/4294892870,v1:172.21.15.138:6809/4294892870] conn(0x7fb0080c02e0 0x7fb0080c27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:56.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.927+0000 7fb01f59e640 1 --2- 172.21.15.117:0/2858161873 >> [v2:172.21.15.138:6808/4294892870,v1:172.21.15.138:6809/4294892870] conn(0x7fb0080c02e0 0x7fb0080c27d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:56.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.928+0000 7fb007fff640 1 -- 172.21.15.117:0/2858161873 <== osd.4 v2:172.21.15.138:6808/4294892870 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb0080c2ea0 con 0x7fb0080c02e0 2024-09-21T05:55:56.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.943+0000 7fb025055640 1 -- 172.21.15.117:0/2858161873 --> [v2:172.21.15.138:6808/4294892870,v1:172.21.15.138:6809/4294892870] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb02006b560 con 0x7fb0080c02e0 2024-09-21T05:55:56.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.944+0000 7fb007fff640 1 -- 172.21.15.117:0/2858161873 <== osd.4 v2:172.21.15.138:6808/4294892870 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb02006b560 con 0x7fb0080c02e0 2024-09-21T05:55:56.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.944+0000 7fb025055640 1 -- 172.21.15.117:0/2858161873 >> [v2:172.21.15.138:6808/4294892870,v1:172.21.15.138:6809/4294892870] conn(0x7fb0080c02e0 msgr2=0x7fb0080c27d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:56.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.944+0000 7fb025055640 1 --2- 172.21.15.117:0/2858161873 >> [v2:172.21.15.138:6808/4294892870,v1:172.21.15.138:6809/4294892870] conn(0x7fb0080c02e0 0x7fb0080c27d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:56.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.945+0000 7fb025055640 1 -- 172.21.15.117:0/2858161873 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb0080b65f0 msgr2=0x7fb0080b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:56.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.945+0000 7fb025055640 1 --2- 172.21.15.117:0/2858161873 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb0080b65f0 0x7fb0080b8b00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb010005e30 tx=0x7fb010009360 comp rx=0 tx=0).stop 2024-09-21T05:55:56.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.945+0000 7fb025055640 1 -- 172.21.15.117:0/2858161873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb020105300 msgr2=0x7fb02019aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:56.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.945+0000 7fb025055640 1 --2- 172.21.15.117:0/2858161873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb020105300 0x7fb02019aa80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb01400e9f0 tx=0x7fb01400eeb0 comp rx=0 tx=0).stop 2024-09-21T05:55:56.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.945+0000 7fb01f59e640 1 -- 172.21.15.117:0/2858161873 reap_dead start 2024-09-21T05:55:56.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.945+0000 7fb025055640 1 -- 172.21.15.117:0/2858161873 shutdown_connections 2024-09-21T05:55:56.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.945+0000 7fb025055640 1 -- 172.21.15.117:0/2858161873 >> 172.21.15.117:0/2858161873 conn(0x7fb020079730 msgr2=0x7fb020106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:56.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.945+0000 7fb025055640 1 -- 172.21.15.117:0/2858161873 shutdown_connections 2024-09-21T05:55:56.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.945+0000 7fb025055640 1 -- 172.21.15.117:0/2858161873 wait complete. 2024-09-21T05:55:57.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.526+0000 7f4d0ad54640 1 -- 172.21.15.117:0/3143232713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d04103bb0 msgr2=0x7f4d04103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:57.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.526+0000 7f4d0ad54640 1 --2- 172.21.15.117:0/3143232713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d04103bb0 0x7f4d04103fc0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8009900 tx=0x7f4cf802a410 comp rx=0 tx=0).stop 2024-09-21T05:55:57.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.527+0000 7f4d0ad54640 1 -- 172.21.15.117:0/3143232713 shutdown_connections 2024-09-21T05:55:57.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.527+0000 7f4d0ad54640 1 --2- 172.21.15.117:0/3143232713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d04109e40 0x7f4d04110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:57.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.527+0000 7f4d0ad54640 1 --2- 172.21.15.117:0/3143232713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d041094f0 0x7f4d04109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:57.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.527+0000 7f4d0ad54640 1 --2- 172.21.15.117:0/3143232713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d04103bb0 0x7f4d04103fc0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:57.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.527+0000 7f4d0ad54640 1 -- 172.21.15.117:0/3143232713 >> 172.21.15.117:0/3143232713 conn(0x7f4d040ff600 msgr2=0x7f4d04101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:57.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.527+0000 7f4d0ad54640 1 -- 172.21.15.117:0/3143232713 shutdown_connections 2024-09-21T05:55:57.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.527+0000 7f4d0ad54640 1 -- 172.21.15.117:0/3143232713 wait complete. 2024-09-21T05:55:57.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.527+0000 7f4d0ad54640 1 Processor -- start 2024-09-21T05:55:57.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.527+0000 7f4d0ad54640 1 -- start start 2024-09-21T05:55:57.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.528+0000 7f4d0ad54640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d04103bb0 0x7f4d0419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:57.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.528+0000 7f4d0ad54640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d041094f0 0x7f4d0419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:57.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.528+0000 7f4d0ad54640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d04109e40 0x7f4d0419fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:57.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.528+0000 7f4d0ad54640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4d04113b00 con 0x7f4d041094f0 2024-09-21T05:55:57.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.528+0000 7f4d0ad54640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4d04113980 con 0x7f4d04103bb0 2024-09-21T05:55:57.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.528+0000 7f4d0ad54640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4d04113c80 con 0x7f4d04109e40 2024-09-21T05:55:57.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.528+0000 7f4d03fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d041094f0 0x7f4d0419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:57.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.528+0000 7f4d03fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d041094f0 0x7f4d0419aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36336/0 (socket says 172.21.15.117:36336) 2024-09-21T05:55:57.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.528+0000 7f4d03fff640 1 -- 172.21.15.117:0/3759488506 learned_addr learned my addr 172.21.15.117:0/3759488506 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:55:57.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.528+0000 7f4d092f1640 1 --2- 172.21.15.117:0/3759488506 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d04109e40 0x7f4d0419fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:57.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.528+0000 7f4d08af0640 1 --2- 172.21.15.117:0/3759488506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d04103bb0 0x7f4d0419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:57.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.528+0000 7f4d03fff640 1 -- 172.21.15.117:0/3759488506 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d04109e40 msgr2=0x7f4d0419fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:57.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.528+0000 7f4d03fff640 1 --2- 172.21.15.117:0/3759488506 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d04109e40 0x7f4d0419fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:57.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.528+0000 7f4d03fff640 1 -- 172.21.15.117:0/3759488506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d04103bb0 msgr2=0x7f4d0419a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:57.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.528+0000 7f4d03fff640 1 --2- 172.21.15.117:0/3759488506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d04103bb0 0x7f4d0419a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:57.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.528+0000 7f4d03fff640 1 -- 172.21.15.117:0/3759488506 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d041a0480 con 0x7f4d041094f0 2024-09-21T05:55:57.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.528+0000 7f4d08af0640 1 --2- 172.21.15.117:0/3759488506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d04103bb0 0x7f4d0419a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:55:57.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.528+0000 7f4d03fff640 1 --2- 172.21.15.117:0/3759488506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d041094f0 0x7f4d0419aab0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4cf000d750 tx=0x7f4cf000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:57.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.528+0000 7f4d01ffb640 1 -- 172.21.15.117:0/3759488506 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0017070 con 0x7f4d041094f0 2024-09-21T05:55:57.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.528+0000 7f4d0ad54640 1 -- 172.21.15.117:0/3759488506 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d041a0770 con 0x7f4d041094f0 2024-09-21T05:55:57.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.528+0000 7f4d01ffb640 1 -- 172.21.15.117:0/3759488506 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cf00054d0 con 0x7f4d041094f0 2024-09-21T05:55:57.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.528+0000 7f4d01ffb640 1 -- 172.21.15.117:0/3759488506 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0004c30 con 0x7f4d041094f0 2024-09-21T05:55:57.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.529+0000 7f4d0ad54640 1 -- 172.21.15.117:0/3759488506 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d041a6ed0 con 0x7f4d041094f0 2024-09-21T05:55:57.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.530+0000 7f4d0ad54640 1 -- 172.21.15.117:0/3759488506 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4d04063ad0 con 0x7f4d041094f0 2024-09-21T05:55:57.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.532+0000 7f4d01ffb640 1 -- 172.21.15.117:0/3759488506 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f4cf0004dd0 con 0x7f4d041094f0 2024-09-21T05:55:57.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.533+0000 7f4d01ffb640 1 --2- 172.21.15.117:0/3759488506 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4cf40b65f0 0x7f4cf40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:57.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.533+0000 7f4d08af0640 1 --2- 172.21.15.117:0/3759488506 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4cf40b65f0 0x7f4cf40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:57.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.533+0000 7f4d01ffb640 1 -- 172.21.15.117:0/3759488506 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f4cf00e74f0 con 0x7f4d041094f0 2024-09-21T05:55:57.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.533+0000 7f4d01ffb640 1 --2- 172.21.15.117:0/3759488506 >> [v2:172.21.15.138:6824/903392451,v1:172.21.15.138:6825/903392451] conn(0x7f4cf40c02e0 0x7f4cf40c27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:57.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.533+0000 7f4d01ffb640 1 -- 172.21.15.117:0/3759488506 --> [v2:172.21.15.138:6824/903392451,v1:172.21.15.138:6825/903392451] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4cf0021e50 con 0x7f4cf40c02e0 2024-09-21T05:55:57.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.533+0000 7f4d01ffb640 1 -- 172.21.15.117:0/3759488506 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f4cf00e7940 con 0x7f4d041094f0 2024-09-21T05:55:57.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.533+0000 7f4d08af0640 1 --2- 172.21.15.117:0/3759488506 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4cf40b65f0 0x7f4cf40b8b00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4cf80096c0 tx=0x7f4cf8031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:57.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.533+0000 7f4d092f1640 1 --2- 172.21.15.117:0/3759488506 >> [v2:172.21.15.138:6824/903392451,v1:172.21.15.138:6825/903392451] conn(0x7f4cf40c02e0 0x7f4cf40c27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:57.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.534+0000 7f4d092f1640 1 --2- 172.21.15.117:0/3759488506 >> [v2:172.21.15.138:6824/903392451,v1:172.21.15.138:6825/903392451] conn(0x7f4cf40c02e0 0x7f4cf40c27d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:57.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.535+0000 7f4d01ffb640 1 -- 172.21.15.117:0/3759488506 <== osd.10 v2:172.21.15.138:6824/903392451 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4cf0021e50 con 0x7f4cf40c02e0 2024-09-21T05:55:57.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.548+0000 7f4d0ad54640 1 -- 172.21.15.117:0/3759488506 --> [v2:172.21.15.138:6824/903392451,v1:172.21.15.138:6825/903392451] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4d04063f50 con 0x7f4cf40c02e0 2024-09-21T05:55:57.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.549+0000 7f4d01ffb640 1 -- 172.21.15.117:0/3759488506 <== osd.10 v2:172.21.15.138:6824/903392451 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4d04063f50 con 0x7f4cf40c02e0 2024-09-21T05:55:57.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.549+0000 7f4d0ad54640 1 -- 172.21.15.117:0/3759488506 >> [v2:172.21.15.138:6824/903392451,v1:172.21.15.138:6825/903392451] conn(0x7f4cf40c02e0 msgr2=0x7f4cf40c27d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:57.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.549+0000 7f4d0ad54640 1 --2- 172.21.15.117:0/3759488506 >> [v2:172.21.15.138:6824/903392451,v1:172.21.15.138:6825/903392451] conn(0x7f4cf40c02e0 0x7f4cf40c27d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:57.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.549+0000 7f4d0ad54640 1 -- 172.21.15.117:0/3759488506 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4cf40b65f0 msgr2=0x7f4cf40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:57.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.549+0000 7f4d0ad54640 1 --2- 172.21.15.117:0/3759488506 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4cf40b65f0 0x7f4cf40b8b00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4cf80096c0 tx=0x7f4cf8031400 comp rx=0 tx=0).stop 2024-09-21T05:55:57.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.549+0000 7f4d0ad54640 1 -- 172.21.15.117:0/3759488506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d041094f0 msgr2=0x7f4d0419aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:57.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.549+0000 7f4d0ad54640 1 --2- 172.21.15.117:0/3759488506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d041094f0 0x7f4d0419aab0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4cf000d750 tx=0x7f4cf000dc10 comp rx=0 tx=0).stop 2024-09-21T05:55:57.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.549+0000 7f4d092f1640 1 -- 172.21.15.117:0/3759488506 reap_dead start 2024-09-21T05:55:57.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.549+0000 7f4d0ad54640 1 -- 172.21.15.117:0/3759488506 shutdown_connections 2024-09-21T05:55:57.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.549+0000 7f4d0ad54640 1 -- 172.21.15.117:0/3759488506 >> 172.21.15.117:0/3759488506 conn(0x7f4d040ff600 msgr2=0x7f4d04101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:57.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.550+0000 7f4d0ad54640 1 -- 172.21.15.117:0/3759488506 shutdown_connections 2024-09-21T05:55:57.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:57.550+0000 7f4d0ad54640 1 -- 172.21.15.117:0/3759488506 wait complete. 2024-09-21T05:55:57.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:57 smithi138 ceph-mon[34391]: pgmap v286: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-21T05:55:57.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:57 smithi192 ceph-mon[34700]: pgmap v286: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-21T05:55:58.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:57 smithi117 ceph-mon[30947]: pgmap v286: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-21T05:55:58.469 INFO:teuthology.orchestra.run.smithi117.stdout:154618822707 2024-09-21T05:55:58.469 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.5 2024-09-21T05:55:58.876 INFO:teuthology.orchestra.run.smithi117.stdout:270582939674 2024-09-21T05:55:58.876 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.10 2024-09-21T05:55:59.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.565+0000 7fac1a160640 1 -- 172.21.15.117:0/2965640064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac14109df0 msgr2=0x7fac14110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:59.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.565+0000 7fac1a160640 1 --2- 172.21.15.117:0/2965640064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac14109df0 0x7fac14110620 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fac0c009ca0 tx=0x7fac0c02a8c0 comp rx=0 tx=0).stop 2024-09-21T05:55:59.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.566+0000 7fac1a160640 1 -- 172.21.15.117:0/2965640064 shutdown_connections 2024-09-21T05:55:59.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.566+0000 7fac1a160640 1 --2- 172.21.15.117:0/2965640064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac14109df0 0x7fac14110620 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:59.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.566+0000 7fac1a160640 1 --2- 172.21.15.117:0/2965640064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac141094a0 0x7fac141098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:59.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.566+0000 7fac1a160640 1 --2- 172.21.15.117:0/2965640064 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac14103b60 0x7fac14103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:59.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.566+0000 7fac1a160640 1 -- 172.21.15.117:0/2965640064 >> 172.21.15.117:0/2965640064 conn(0x7fac1407b850 msgr2=0x7fac1407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:59.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.566+0000 7fac1a160640 1 -- 172.21.15.117:0/2965640064 shutdown_connections 2024-09-21T05:55:59.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.566+0000 7fac1a160640 1 -- 172.21.15.117:0/2965640064 wait complete. 2024-09-21T05:55:59.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.566+0000 7fac1a160640 1 Processor -- start 2024-09-21T05:55:59.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.566+0000 7fac1a160640 1 -- start start 2024-09-21T05:55:59.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.567+0000 7fac1a160640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac14103b60 0x7fac1419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:59.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.567+0000 7fac1a160640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac141094a0 0x7fac1419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:59.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.567+0000 7fac1a160640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac14109df0 0x7fac1419fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:59.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.567+0000 7fac1a160640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fac14113ab0 con 0x7fac14109df0 2024-09-21T05:55:59.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.567+0000 7fac1a160640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fac14113930 con 0x7fac141094a0 2024-09-21T05:55:59.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.567+0000 7fac1a160640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fac14113c30 con 0x7fac14103b60 2024-09-21T05:55:59.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.567+0000 7fac13fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac14109df0 0x7fac1419fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:59.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.567+0000 7fac13fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac14109df0 0x7fac1419fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36362/0 (socket says 172.21.15.117:36362) 2024-09-21T05:55:59.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.567+0000 7fac13fff640 1 -- 172.21.15.117:0/1206903683 learned_addr learned my addr 172.21.15.117:0/1206903683 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:55:59.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.567+0000 7fac137fe640 1 --2- 172.21.15.117:0/1206903683 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac14103b60 0x7fac1419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:59.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.567+0000 7fac12ffd640 1 --2- 172.21.15.117:0/1206903683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac141094a0 0x7fac1419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:59.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.567+0000 7fac13fff640 1 -- 172.21.15.117:0/1206903683 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac14103b60 msgr2=0x7fac1419a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:59.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.567+0000 7fac13fff640 1 --2- 172.21.15.117:0/1206903683 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac14103b60 0x7fac1419a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:59.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.567+0000 7fac13fff640 1 -- 172.21.15.117:0/1206903683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac141094a0 msgr2=0x7fac1419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:59.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.567+0000 7fac13fff640 1 --2- 172.21.15.117:0/1206903683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac141094a0 0x7fac1419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:59.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.567+0000 7fac13fff640 1 -- 172.21.15.117:0/1206903683 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac141a0390 con 0x7fac14109df0 2024-09-21T05:55:59.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.567+0000 7fac13fff640 1 --2- 172.21.15.117:0/1206903683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac14109df0 0x7fac1419fc60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fac0c005cc0 tx=0x7fac0c02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:59.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.568+0000 7fac10ff9640 1 -- 172.21.15.117:0/1206903683 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c003bf0 con 0x7fac14109df0 2024-09-21T05:55:59.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.568+0000 7fac10ff9640 1 -- 172.21.15.117:0/1206903683 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac0c003d90 con 0x7fac14109df0 2024-09-21T05:55:59.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.568+0000 7fac1a160640 1 -- 172.21.15.117:0/1206903683 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac141a0620 con 0x7fac14109df0 2024-09-21T05:55:59.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.568+0000 7fac10ff9640 1 -- 172.21.15.117:0/1206903683 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c02fbc0 con 0x7fac14109df0 2024-09-21T05:55:59.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.568+0000 7fac1a160640 1 -- 172.21.15.117:0/1206903683 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac141a6de0 con 0x7fac14109df0 2024-09-21T05:55:59.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.569+0000 7fac1a160640 1 -- 172.21.15.117:0/1206903683 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fac14063b50 con 0x7fac14109df0 2024-09-21T05:55:59.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.570+0000 7fac10ff9640 1 -- 172.21.15.117:0/1206903683 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7fac0c02fd60 con 0x7fac14109df0 2024-09-21T05:55:59.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.571+0000 7fac10ff9640 1 --2- 172.21.15.117:0/1206903683 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fac000b65f0 0x7fac000b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:59.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.571+0000 7fac137fe640 1 --2- 172.21.15.117:0/1206903683 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fac000b65f0 0x7fac000b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:59.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.571+0000 7fac10ff9640 1 -- 172.21.15.117:0/1206903683 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fac0c108b90 con 0x7fac14109df0 2024-09-21T05:55:59.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.571+0000 7fac10ff9640 1 --2- 172.21.15.117:0/1206903683 >> [v2:172.21.15.192:6824/899090479,v1:172.21.15.192:6825/899090479] conn(0x7fac000c02e0 0x7fac000c27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:55:59.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.571+0000 7fac10ff9640 1 -- 172.21.15.117:0/1206903683 --> [v2:172.21.15.192:6824/899090479,v1:172.21.15.192:6825/899090479] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fac000c2ea0 con 0x7fac000c02e0 2024-09-21T05:55:59.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.571+0000 7fac10ff9640 1 -- 172.21.15.117:0/1206903683 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fac0c108fe0 con 0x7fac14109df0 2024-09-21T05:55:59.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.572+0000 7fac137fe640 1 --2- 172.21.15.117:0/1206903683 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fac000b65f0 0x7fac000b8b00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fac04005fd0 tx=0x7fac0400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:59.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.572+0000 7fac12ffd640 1 --2- 172.21.15.117:0/1206903683 >> [v2:172.21.15.192:6824/899090479,v1:172.21.15.192:6825/899090479] conn(0x7fac000c02e0 0x7fac000c27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:55:59.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.572+0000 7fac12ffd640 1 --2- 172.21.15.117:0/1206903683 >> [v2:172.21.15.192:6824/899090479,v1:172.21.15.192:6825/899090479] conn(0x7fac000c02e0 0x7fac000c27d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:55:59.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.573+0000 7fac10ff9640 1 -- 172.21.15.117:0/1206903683 <== osd.11 v2:172.21.15.192:6824/899090479 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fac000c2ea0 con 0x7fac000c02e0 2024-09-21T05:55:59.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.586+0000 7fac1a160640 1 -- 172.21.15.117:0/1206903683 --> [v2:172.21.15.192:6824/899090479,v1:172.21.15.192:6825/899090479] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fac14063fd0 con 0x7fac000c02e0 2024-09-21T05:55:59.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.587+0000 7fac10ff9640 1 -- 172.21.15.117:0/1206903683 <== osd.11 v2:172.21.15.192:6824/899090479 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fac14063fd0 con 0x7fac000c02e0 2024-09-21T05:55:59.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.587+0000 7fac1a160640 1 -- 172.21.15.117:0/1206903683 >> [v2:172.21.15.192:6824/899090479,v1:172.21.15.192:6825/899090479] conn(0x7fac000c02e0 msgr2=0x7fac000c27d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:59.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.587+0000 7fac1a160640 1 --2- 172.21.15.117:0/1206903683 >> [v2:172.21.15.192:6824/899090479,v1:172.21.15.192:6825/899090479] conn(0x7fac000c02e0 0x7fac000c27d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:55:59.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.587+0000 7fac1a160640 1 -- 172.21.15.117:0/1206903683 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fac000b65f0 msgr2=0x7fac000b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:59.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.588+0000 7fac1a160640 1 --2- 172.21.15.117:0/1206903683 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fac000b65f0 0x7fac000b8b00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fac04005fd0 tx=0x7fac0400b390 comp rx=0 tx=0).stop 2024-09-21T05:55:59.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.588+0000 7fac1a160640 1 -- 172.21.15.117:0/1206903683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac14109df0 msgr2=0x7fac1419fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:55:59.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.588+0000 7fac1a160640 1 --2- 172.21.15.117:0/1206903683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac14109df0 0x7fac1419fc60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fac0c005cc0 tx=0x7fac0c02f380 comp rx=0 tx=0).stop 2024-09-21T05:55:59.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.588+0000 7fac13fff640 1 -- 172.21.15.117:0/1206903683 reap_dead start 2024-09-21T05:55:59.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.588+0000 7fac1a160640 1 -- 172.21.15.117:0/1206903683 shutdown_connections 2024-09-21T05:55:59.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.588+0000 7fac1a160640 1 -- 172.21.15.117:0/1206903683 >> 172.21.15.117:0/1206903683 conn(0x7fac1407b850 msgr2=0x7fac14107490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:55:59.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.588+0000 7fac1a160640 1 -- 172.21.15.117:0/1206903683 shutdown_connections 2024-09-21T05:55:59.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:59.588+0000 7fac1a160640 1 -- 172.21.15.117:0/1206903683 wait complete. 2024-09-21T05:55:59.737 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:55:59 smithi117 ceph-mon[30947]: pgmap v287: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-21T05:55:59.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:55:59 smithi138 ceph-mon[34391]: pgmap v287: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-21T05:55:59.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:55:59 smithi192 ceph-mon[34700]: pgmap v287: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-21T05:56:00.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.366+0000 7fb64c5d7640 1 -- 172.21.15.117:0/3564154601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6441094d0 msgr2=0x7fb6441098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:00.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.366+0000 7fb64c5d7640 1 --2- 172.21.15.117:0/3564154601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6441094d0 0x7fb6441098e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb634009900 tx=0x7fb63402a410 comp rx=0 tx=0).stop 2024-09-21T05:56:00.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.366+0000 7fb64c5d7640 1 -- 172.21.15.117:0/3564154601 shutdown_connections 2024-09-21T05:56:00.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.366+0000 7fb64c5d7640 1 --2- 172.21.15.117:0/3564154601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6441094d0 0x7fb6441098e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:00.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.366+0000 7fb64c5d7640 1 --2- 172.21.15.117:0/3564154601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb644109e20 0x7fb644110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:00.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.366+0000 7fb64c5d7640 1 --2- 172.21.15.117:0/3564154601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb644103b90 0x7fb644103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:00.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.366+0000 7fb64c5d7640 1 -- 172.21.15.117:0/3564154601 >> 172.21.15.117:0/3564154601 conn(0x7fb6440ff5e0 msgr2=0x7fb644101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:00.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.367+0000 7fb64c5d7640 1 -- 172.21.15.117:0/3564154601 shutdown_connections 2024-09-21T05:56:00.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.367+0000 7fb64c5d7640 1 -- 172.21.15.117:0/3564154601 wait complete. 2024-09-21T05:56:00.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.367+0000 7fb64c5d7640 1 Processor -- start 2024-09-21T05:56:00.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.367+0000 7fb64c5d7640 1 -- start start 2024-09-21T05:56:00.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.367+0000 7fb64c5d7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb644103b90 0x7fb644075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:00.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.367+0000 7fb64c5d7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6441094d0 0x7fb644075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:00.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.367+0000 7fb64c5d7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb644109e20 0x7fb644076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:00.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.368+0000 7fb64c5d7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb644113b20 con 0x7fb644109e20 2024-09-21T05:56:00.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.368+0000 7fb64c5d7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb6441139a0 con 0x7fb6441094d0 2024-09-21T05:56:00.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.368+0000 7fb64c5d7640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb644113ca0 con 0x7fb644103b90 2024-09-21T05:56:00.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.368+0000 7fb64ab74640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb644109e20 0x7fb644076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:00.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.368+0000 7fb64ab74640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb644109e20 0x7fb644076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36294/0 (socket says 172.21.15.117:36294) 2024-09-21T05:56:00.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.368+0000 7fb64ab74640 1 -- 172.21.15.117:0/1281344477 learned_addr learned my addr 172.21.15.117:0/1281344477 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:00.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.368+0000 7fb64a373640 1 --2- 172.21.15.117:0/1281344477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb644103b90 0x7fb644075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:00.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.368+0000 7fb649b72640 1 --2- 172.21.15.117:0/1281344477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6441094d0 0x7fb644075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:00.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.368+0000 7fb64ab74640 1 -- 172.21.15.117:0/1281344477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb644103b90 msgr2=0x7fb644075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:00.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.368+0000 7fb64ab74640 1 --2- 172.21.15.117:0/1281344477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb644103b90 0x7fb644075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:00.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.368+0000 7fb64ab74640 1 -- 172.21.15.117:0/1281344477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6441094d0 msgr2=0x7fb644075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:00.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.368+0000 7fb64ab74640 1 --2- 172.21.15.117:0/1281344477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6441094d0 0x7fb644075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:00.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.368+0000 7fb64ab74640 1 -- 172.21.15.117:0/1281344477 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6441a87d0 con 0x7fb644109e20 2024-09-21T05:56:00.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.368+0000 7fb64ab74640 1 --2- 172.21.15.117:0/1281344477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb644109e20 0x7fb644076190 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00b970 tx=0x7fb63c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:00.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.368+0000 7fb63b7fe640 1 -- 172.21.15.117:0/1281344477 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c019070 con 0x7fb644109e20 2024-09-21T05:56:00.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.368+0000 7fb63b7fe640 1 -- 172.21.15.117:0/1281344477 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb63c014440 con 0x7fb644109e20 2024-09-21T05:56:00.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.368+0000 7fb64c5d7640 1 -- 172.21.15.117:0/1281344477 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6441a8ac0 con 0x7fb644109e20 2024-09-21T05:56:00.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.368+0000 7fb63b7fe640 1 -- 172.21.15.117:0/1281344477 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c0075d0 con 0x7fb644109e20 2024-09-21T05:56:00.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.368+0000 7fb64c5d7640 1 -- 172.21.15.117:0/1281344477 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6441a9000 con 0x7fb644109e20 2024-09-21T05:56:00.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.369+0000 7fb64c5d7640 1 -- 172.21.15.117:0/1281344477 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb644063b50 con 0x7fb644109e20 2024-09-21T05:56:00.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.371+0000 7fb63b7fe640 1 -- 172.21.15.117:0/1281344477 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7fb63c01c070 con 0x7fb644109e20 2024-09-21T05:56:00.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.372+0000 7fb63b7fe640 1 --2- 172.21.15.117:0/1281344477 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6300b65f0 0x7fb6300b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:00.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.372+0000 7fb64a373640 1 --2- 172.21.15.117:0/1281344477 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6300b65f0 0x7fb6300b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:00.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.372+0000 7fb63b7fe640 1 -- 172.21.15.117:0/1281344477 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fb63c0e7250 con 0x7fb644109e20 2024-09-21T05:56:00.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.372+0000 7fb63b7fe640 1 --2- 172.21.15.117:0/1281344477 >> [v2:172.21.15.192:6800/3002534992,v1:172.21.15.192:6801/3002534992] conn(0x7fb6300c02e0 0x7fb6300c27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:00.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.372+0000 7fb63b7fe640 1 -- 172.21.15.117:0/1281344477 --> [v2:172.21.15.192:6800/3002534992,v1:172.21.15.192:6801/3002534992] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb6300c2ea0 con 0x7fb6300c02e0 2024-09-21T05:56:00.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.372+0000 7fb63b7fe640 1 -- 172.21.15.117:0/1281344477 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fb63c0e76a0 con 0x7fb644109e20 2024-09-21T05:56:00.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.372+0000 7fb64a373640 1 --2- 172.21.15.117:0/1281344477 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6300b65f0 0x7fb6300b8b00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb644077160 tx=0x7fb64000a250 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:00.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.372+0000 7fb649b72640 1 --2- 172.21.15.117:0/1281344477 >> [v2:172.21.15.192:6800/3002534992,v1:172.21.15.192:6801/3002534992] conn(0x7fb6300c02e0 0x7fb6300c27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:00.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.373+0000 7fb649b72640 1 --2- 172.21.15.117:0/1281344477 >> [v2:172.21.15.192:6800/3002534992,v1:172.21.15.192:6801/3002534992] conn(0x7fb6300c02e0 0x7fb6300c27d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:00.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.374+0000 7fb63b7fe640 1 -- 172.21.15.117:0/1281344477 <== osd.2 v2:172.21.15.192:6800/3002534992 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb6300c2ea0 con 0x7fb6300c02e0 2024-09-21T05:56:00.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.387+0000 7fb64c5d7640 1 -- 172.21.15.117:0/1281344477 --> [v2:172.21.15.192:6800/3002534992,v1:172.21.15.192:6801/3002534992] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb644063fd0 con 0x7fb6300c02e0 2024-09-21T05:56:00.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.388+0000 7fb63b7fe640 1 -- 172.21.15.117:0/1281344477 <== osd.2 v2:172.21.15.192:6800/3002534992 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb644063fd0 con 0x7fb6300c02e0 2024-09-21T05:56:00.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.388+0000 7fb64c5d7640 1 -- 172.21.15.117:0/1281344477 >> [v2:172.21.15.192:6800/3002534992,v1:172.21.15.192:6801/3002534992] conn(0x7fb6300c02e0 msgr2=0x7fb6300c27d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:00.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.388+0000 7fb64c5d7640 1 --2- 172.21.15.117:0/1281344477 >> [v2:172.21.15.192:6800/3002534992,v1:172.21.15.192:6801/3002534992] conn(0x7fb6300c02e0 0x7fb6300c27d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:00.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.388+0000 7fb64c5d7640 1 -- 172.21.15.117:0/1281344477 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6300b65f0 msgr2=0x7fb6300b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:00.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.388+0000 7fb64c5d7640 1 --2- 172.21.15.117:0/1281344477 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6300b65f0 0x7fb6300b8b00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb644077160 tx=0x7fb64000a250 comp rx=0 tx=0).stop 2024-09-21T05:56:00.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.388+0000 7fb64c5d7640 1 -- 172.21.15.117:0/1281344477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb644109e20 msgr2=0x7fb644076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:00.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.388+0000 7fb64c5d7640 1 --2- 172.21.15.117:0/1281344477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb644109e20 0x7fb644076190 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00b970 tx=0x7fb63c00be30 comp rx=0 tx=0).stop 2024-09-21T05:56:00.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.388+0000 7fb64ab74640 1 -- 172.21.15.117:0/1281344477 reap_dead start 2024-09-21T05:56:00.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.388+0000 7fb64c5d7640 1 -- 172.21.15.117:0/1281344477 shutdown_connections 2024-09-21T05:56:00.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.388+0000 7fb64c5d7640 1 -- 172.21.15.117:0/1281344477 >> 172.21.15.117:0/1281344477 conn(0x7fb6440ff5e0 msgr2=0x7fb644101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:00.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.388+0000 7fb64c5d7640 1 -- 172.21.15.117:0/1281344477 shutdown_connections 2024-09-21T05:56:00.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:55:56.388+0000 7fb64c5d7640 1 -- 172.21.15.117:0/1281344477 wait complete. 2024-09-21T05:56:00.652 INFO:teuthology.orchestra.run.smithi117.stdout:81604378692 2024-09-21T05:56:00.653 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.2 2024-09-21T05:56:01.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.008+0000 7f5de0814640 1 -- 172.21.15.117:0/3008138257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ddc1052a0 msgr2=0x7f5ddc10d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:01.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.008+0000 7f5de0814640 1 --2- 172.21.15.117:0/3008138257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ddc1052a0 0x7f5ddc10d930 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0009900 tx=0x7f5dd002a410 comp rx=0 tx=0).stop 2024-09-21T05:56:01.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.008+0000 7f5de0814640 1 -- 172.21.15.117:0/3008138257 shutdown_connections 2024-09-21T05:56:01.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.008+0000 7f5de0814640 1 --2- 172.21.15.117:0/3008138257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ddc1052a0 0x7f5ddc10d930 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:01.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.008+0000 7f5de0814640 1 --2- 172.21.15.117:0/3008138257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ddc10de70 0x7f5ddc110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:01.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.008+0000 7f5de0814640 1 --2- 172.21.15.117:0/3008138257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ddc1048c0 0x7f5ddc104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:01.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.008+0000 7f5de0814640 1 -- 172.21.15.117:0/3008138257 >> 172.21.15.117:0/3008138257 conn(0x7f5ddc0797b0 msgr2=0x7f5ddc079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:01.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.008+0000 7f5de0814640 1 -- 172.21.15.117:0/3008138257 shutdown_connections 2024-09-21T05:56:01.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.008+0000 7f5de0814640 1 -- 172.21.15.117:0/3008138257 wait complete. 2024-09-21T05:56:01.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.009+0000 7f5de0814640 1 Processor -- start 2024-09-21T05:56:01.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.009+0000 7f5de0814640 1 -- start start 2024-09-21T05:56:01.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.009+0000 7f5de0814640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ddc1048c0 0x7f5ddc19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:01.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.009+0000 7f5de0814640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ddc1052a0 0x7f5ddc19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:01.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.009+0000 7f5de0814640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ddc10de70 0x7f5ddc19fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:01.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.009+0000 7f5de0814640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc113b70 con 0x7f5ddc1052a0 2024-09-21T05:56:01.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.009+0000 7f5de0814640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc1139f0 con 0x7f5ddc10de70 2024-09-21T05:56:01.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.009+0000 7f5de0814640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc113cf0 con 0x7f5ddc1048c0 2024-09-21T05:56:01.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.010+0000 7f5dd9d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ddc1052a0 0x7f5ddc19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:01.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.010+0000 7f5dd9d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ddc1052a0 0x7f5ddc19ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36378/0 (socket says 172.21.15.117:36378) 2024-09-21T05:56:01.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.010+0000 7f5dd9d9b640 1 -- 172.21.15.117:0/2907819000 learned_addr learned my addr 172.21.15.117:0/2907819000 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:01.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.010+0000 7f5ddad9d640 1 --2- 172.21.15.117:0/2907819000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ddc10de70 0x7f5ddc19fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:01.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.010+0000 7f5dda59c640 1 --2- 172.21.15.117:0/2907819000 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ddc1048c0 0x7f5ddc19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:01.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.010+0000 7f5dd9d9b640 1 -- 172.21.15.117:0/2907819000 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ddc1048c0 msgr2=0x7f5ddc19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:01.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.010+0000 7f5dd9d9b640 1 --2- 172.21.15.117:0/2907819000 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ddc1048c0 0x7f5ddc19a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:01.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.010+0000 7f5dd9d9b640 1 -- 172.21.15.117:0/2907819000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ddc10de70 msgr2=0x7f5ddc19fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:01.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.010+0000 7f5dd9d9b640 1 --2- 172.21.15.117:0/2907819000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ddc10de70 0x7f5ddc19fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:01.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.010+0000 7f5dd9d9b640 1 -- 172.21.15.117:0/2907819000 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ddc1a0530 con 0x7f5ddc1052a0 2024-09-21T05:56:01.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.010+0000 7f5dd9d9b640 1 --2- 172.21.15.117:0/2907819000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ddc1052a0 0x7f5ddc19ab60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5dd002a930 tx=0x7f5dd0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:01.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.010+0000 7f5dcb7fe640 1 -- 172.21.15.117:0/2907819000 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd003f030 con 0x7f5ddc1052a0 2024-09-21T05:56:01.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.010+0000 7f5dcb7fe640 1 -- 172.21.15.117:0/2907819000 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dd0003aa0 con 0x7f5ddc1052a0 2024-09-21T05:56:01.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.010+0000 7f5de0814640 1 -- 172.21.15.117:0/2907819000 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ddc1a07c0 con 0x7f5ddc1052a0 2024-09-21T05:56:01.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.011+0000 7f5dcb7fe640 1 -- 172.21.15.117:0/2907819000 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0035070 con 0x7f5ddc1052a0 2024-09-21T05:56:01.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.011+0000 7f5de0814640 1 -- 172.21.15.117:0/2907819000 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ddc1a6f80 con 0x7f5ddc1052a0 2024-09-21T05:56:01.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.012+0000 7f5de0814640 1 -- 172.21.15.117:0/2907819000 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5ddc063b10 con 0x7f5ddc1052a0 2024-09-21T05:56:01.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.013+0000 7f5dcb7fe640 1 -- 172.21.15.117:0/2907819000 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f5dd0042030 con 0x7f5ddc1052a0 2024-09-21T05:56:01.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.014+0000 7f5dcb7fe640 1 --2- 172.21.15.117:0/2907819000 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5dc40b65f0 0x7f5dc40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:01.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.014+0000 7f5dda59c640 1 --2- 172.21.15.117:0/2907819000 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5dc40b65f0 0x7f5dc40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:01.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.014+0000 7f5dcb7fe640 1 -- 172.21.15.117:0/2907819000 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f5dd01080c0 con 0x7f5ddc1052a0 2024-09-21T05:56:01.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.014+0000 7f5dcb7fe640 1 --2- 172.21.15.117:0/2907819000 >> [v2:172.21.15.192:6816/3960902885,v1:172.21.15.192:6817/3960902885] conn(0x7f5dc40c02e0 0x7f5dc40c27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:01.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.014+0000 7f5dcb7fe640 1 -- 172.21.15.117:0/2907819000 --> [v2:172.21.15.192:6816/3960902885,v1:172.21.15.192:6817/3960902885] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5dc40c2ea0 con 0x7f5dc40c02e0 2024-09-21T05:56:01.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.014+0000 7f5dcb7fe640 1 -- 172.21.15.117:0/2907819000 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f5dd0108510 con 0x7f5ddc1052a0 2024-09-21T05:56:01.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.015+0000 7f5dda59c640 1 --2- 172.21.15.117:0/2907819000 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5dc40b65f0 0x7f5dc40b8b00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc009990 tx=0x7f5dcc009110 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:01.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.015+0000 7f5ddad9d640 1 --2- 172.21.15.117:0/2907819000 >> [v2:172.21.15.192:6816/3960902885,v1:172.21.15.192:6817/3960902885] conn(0x7f5dc40c02e0 0x7f5dc40c27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:01.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.015+0000 7f5ddad9d640 1 --2- 172.21.15.117:0/2907819000 >> [v2:172.21.15.192:6816/3960902885,v1:172.21.15.192:6817/3960902885] conn(0x7f5dc40c02e0 0x7f5dc40c27d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:01.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.016+0000 7f5dcb7fe640 1 -- 172.21.15.117:0/2907819000 <== osd.8 v2:172.21.15.192:6816/3960902885 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5dc40c2ea0 con 0x7f5dc40c02e0 2024-09-21T05:56:01.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.034+0000 7f5de0814640 1 -- 172.21.15.117:0/2907819000 --> [v2:172.21.15.192:6816/3960902885,v1:172.21.15.192:6817/3960902885] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5ddc063f10 con 0x7f5dc40c02e0 2024-09-21T05:56:01.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.035+0000 7f5dcb7fe640 1 -- 172.21.15.117:0/2907819000 <== osd.8 v2:172.21.15.192:6816/3960902885 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5ddc063f10 con 0x7f5dc40c02e0 2024-09-21T05:56:01.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.036+0000 7f5de0814640 1 -- 172.21.15.117:0/2907819000 >> [v2:172.21.15.192:6816/3960902885,v1:172.21.15.192:6817/3960902885] conn(0x7f5dc40c02e0 msgr2=0x7f5dc40c27d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:01.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.036+0000 7f5de0814640 1 --2- 172.21.15.117:0/2907819000 >> [v2:172.21.15.192:6816/3960902885,v1:172.21.15.192:6817/3960902885] conn(0x7f5dc40c02e0 0x7f5dc40c27d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:01.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.036+0000 7f5de0814640 1 -- 172.21.15.117:0/2907819000 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5dc40b65f0 msgr2=0x7f5dc40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:01.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.036+0000 7f5de0814640 1 --2- 172.21.15.117:0/2907819000 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5dc40b65f0 0x7f5dc40b8b00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc009990 tx=0x7f5dcc009110 comp rx=0 tx=0).stop 2024-09-21T05:56:01.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.036+0000 7f5de0814640 1 -- 172.21.15.117:0/2907819000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ddc1052a0 msgr2=0x7f5ddc19ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:01.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.036+0000 7f5de0814640 1 --2- 172.21.15.117:0/2907819000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ddc1052a0 0x7f5ddc19ab60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5dd002a930 tx=0x7f5dd0004270 comp rx=0 tx=0).stop 2024-09-21T05:56:01.040 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.036+0000 7f5ddad9d640 1 -- 172.21.15.117:0/2907819000 reap_dead start 2024-09-21T05:56:01.040 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.036+0000 7f5de0814640 1 -- 172.21.15.117:0/2907819000 shutdown_connections 2024-09-21T05:56:01.040 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.036+0000 7f5de0814640 1 -- 172.21.15.117:0/2907819000 >> 172.21.15.117:0/2907819000 conn(0x7f5ddc0797b0 msgr2=0x7f5ddc0ff730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:01.040 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.036+0000 7f5de0814640 1 -- 172.21.15.117:0/2907819000 shutdown_connections 2024-09-21T05:56:01.040 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:01.036+0000 7f5de0814640 1 -- 172.21.15.117:0/2907819000 wait complete. 2024-09-21T05:56:01.498 INFO:teuthology.orchestra.run.smithi117.stdout:176093659183 2024-09-21T05:56:01.499 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.6 2024-09-21T05:56:01.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:01 smithi117 ceph-mon[30947]: pgmap v288: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T05:56:01.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:01 smithi138 ceph-mon[34391]: pgmap v288: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T05:56:01.891 INFO:teuthology.orchestra.run.smithi117.stdout:133143986234 2024-09-21T05:56:01.892 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.4 2024-09-21T05:56:01.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:01 smithi192 ceph-mon[34700]: pgmap v288: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T05:56:02.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.160+0000 7f6c010ca640 1 -- 172.21.15.117:0/926315722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bfc1049b0 msgr2=0x7f6bfc104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:02.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.160+0000 7f6c010ca640 1 --2- 172.21.15.117:0/926315722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bfc1049b0 0x7f6bfc104dc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6be4009900 tx=0x7f6be402a410 comp rx=0 tx=0).stop 2024-09-21T05:56:02.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.160+0000 7f6c010ca640 1 -- 172.21.15.117:0/926315722 shutdown_connections 2024-09-21T05:56:02.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.160+0000 7f6c010ca640 1 --2- 172.21.15.117:0/926315722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bfc10de10 0x7f6bfc110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:02.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.160+0000 7f6c010ca640 1 --2- 172.21.15.117:0/926315722 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bfc105300 0x7f6bfc10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:02.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.160+0000 7f6c010ca640 1 --2- 172.21.15.117:0/926315722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bfc1049b0 0x7f6bfc104dc0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:02.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.160+0000 7f6c010ca640 1 -- 172.21.15.117:0/926315722 >> 172.21.15.117:0/926315722 conn(0x7f6bfc079730 msgr2=0x7f6bfc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:02.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.160+0000 7f6c010ca640 1 -- 172.21.15.117:0/926315722 shutdown_connections 2024-09-21T05:56:02.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.160+0000 7f6c010ca640 1 -- 172.21.15.117:0/926315722 wait complete. 2024-09-21T05:56:02.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.161+0000 7f6c010ca640 1 Processor -- start 2024-09-21T05:56:02.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.161+0000 7f6c010ca640 1 -- start start 2024-09-21T05:56:02.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.162+0000 7f6c010ca640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bfc1049b0 0x7f6bfc19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:02.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.162+0000 7f6c010ca640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bfc105300 0x7f6bfc19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:02.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.162+0000 7f6c010ca640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bfc10de10 0x7f6bfc19fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:02.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.162+0000 7f6c010ca640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6bfc113a90 con 0x7f6bfc10de10 2024-09-21T05:56:02.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.162+0000 7f6c010ca640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6bfc113910 con 0x7f6bfc105300 2024-09-21T05:56:02.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.162+0000 7f6c010ca640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6bfc113c10 con 0x7f6bfc1049b0 2024-09-21T05:56:02.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.162+0000 7f6bfb59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bfc10de10 0x7f6bfc19fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:02.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.162+0000 7f6bfb59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bfc10de10 0x7f6bfc19fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36406/0 (socket says 172.21.15.117:36406) 2024-09-21T05:56:02.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.162+0000 7f6bfb59e640 1 -- 172.21.15.117:0/2286907278 learned_addr learned my addr 172.21.15.117:0/2286907278 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:02.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.162+0000 7f6bfa59c640 1 --2- 172.21.15.117:0/2286907278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bfc105300 0x7f6bfc19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:02.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.162+0000 7f6bfad9d640 1 --2- 172.21.15.117:0/2286907278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bfc1049b0 0x7f6bfc19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:02.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.162+0000 7f6bfa59c640 1 -- 172.21.15.117:0/2286907278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bfc1049b0 msgr2=0x7f6bfc19a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:02.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.162+0000 7f6bfa59c640 1 --2- 172.21.15.117:0/2286907278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6bfc1049b0 0x7f6bfc19a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:02.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.162+0000 7f6bfa59c640 1 -- 172.21.15.117:0/2286907278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bfc10de10 msgr2=0x7f6bfc19fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:02.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.162+0000 7f6bfa59c640 1 --2- 172.21.15.117:0/2286907278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6bfc10de10 0x7f6bfc19fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:02.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.162+0000 7f6bfa59c640 1 -- 172.21.15.117:0/2286907278 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bfc1a04d0 con 0x7f6bfc105300 2024-09-21T05:56:02.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.163+0000 7f6bfa59c640 1 --2- 172.21.15.117:0/2286907278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bfc105300 0x7f6bfc19ab00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc06ad30 tx=0x7f6bec00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:02.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.163+0000 7f6bf3fff640 1 -- 172.21.15.117:0/2286907278 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bec00cc40 con 0x7f6bfc105300 2024-09-21T05:56:02.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.163+0000 7f6c010ca640 1 -- 172.21.15.117:0/2286907278 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bfc1a07c0 con 0x7f6bfc105300 2024-09-21T05:56:02.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.163+0000 7f6bf3fff640 1 -- 172.21.15.117:0/2286907278 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bec00cde0 con 0x7f6bfc105300 2024-09-21T05:56:02.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.163+0000 7f6bf3fff640 1 -- 172.21.15.117:0/2286907278 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bec004a00 con 0x7f6bfc105300 2024-09-21T05:56:02.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.163+0000 7f6c010ca640 1 -- 172.21.15.117:0/2286907278 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bfc1025d0 con 0x7f6bfc105300 2024-09-21T05:56:02.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.164+0000 7f6c010ca640 1 -- 172.21.15.117:0/2286907278 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6bfc063b10 con 0x7f6bfc105300 2024-09-21T05:56:02.169 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.167+0000 7f6bf3fff640 1 -- 172.21.15.117:0/2286907278 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f6bec004ba0 con 0x7f6bfc105300 2024-09-21T05:56:02.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.167+0000 7f6bf3fff640 1 --2- 172.21.15.117:0/2286907278 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6be80b6670 0x7f6be80b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:02.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.167+0000 7f6bfad9d640 1 --2- 172.21.15.117:0/2286907278 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6be80b6670 0x7f6be80b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:02.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.168+0000 7f6bf3fff640 1 -- 172.21.15.117:0/2286907278 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f6bec0e7750 con 0x7f6bfc105300 2024-09-21T05:56:02.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.168+0000 7f6bf3fff640 1 --2- 172.21.15.117:0/2286907278 >> [v2:172.21.15.117:6810/1222400293,v1:172.21.15.117:6811/1222400293] conn(0x7f6be80c0360 0x7f6be80c2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:02.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.168+0000 7f6bf3fff640 1 -- 172.21.15.117:0/2286907278 --> [v2:172.21.15.117:6810/1222400293,v1:172.21.15.117:6811/1222400293] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6bec009030 con 0x7f6be80c0360 2024-09-21T05:56:02.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.168+0000 7f6bf3fff640 1 -- 172.21.15.117:0/2286907278 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f6bec0e7ba0 con 0x7f6bfc105300 2024-09-21T05:56:02.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.168+0000 7f6bfb59e640 1 --2- 172.21.15.117:0/2286907278 >> [v2:172.21.15.117:6810/1222400293,v1:172.21.15.117:6811/1222400293] conn(0x7f6be80c0360 0x7f6be80c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:02.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.168+0000 7f6bfad9d640 1 --2- 172.21.15.117:0/2286907278 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6be80b6670 0x7f6be80b8b80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6be4000c00 tx=0x7f6be4031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:02.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.168+0000 7f6bfb59e640 1 --2- 172.21.15.117:0/2286907278 >> [v2:172.21.15.117:6810/1222400293,v1:172.21.15.117:6811/1222400293] conn(0x7f6be80c0360 0x7f6be80c2850 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:02.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.169+0000 7f6bf3fff640 1 -- 172.21.15.117:0/2286907278 <== osd.3 v2:172.21.15.117:6810/1222400293 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6bec009030 con 0x7f6be80c0360 2024-09-21T05:56:02.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.183+0000 7f6c010ca640 1 -- 172.21.15.117:0/2286907278 --> [v2:172.21.15.117:6810/1222400293,v1:172.21.15.117:6811/1222400293] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6bfc063f90 con 0x7f6be80c0360 2024-09-21T05:56:02.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.184+0000 7f6bf3fff640 1 -- 172.21.15.117:0/2286907278 <== osd.3 v2:172.21.15.117:6810/1222400293 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6bfc063f90 con 0x7f6be80c0360 2024-09-21T05:56:02.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.184+0000 7f6c010ca640 1 -- 172.21.15.117:0/2286907278 >> [v2:172.21.15.117:6810/1222400293,v1:172.21.15.117:6811/1222400293] conn(0x7f6be80c0360 msgr2=0x7f6be80c2850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:02.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.184+0000 7f6c010ca640 1 --2- 172.21.15.117:0/2286907278 >> [v2:172.21.15.117:6810/1222400293,v1:172.21.15.117:6811/1222400293] conn(0x7f6be80c0360 0x7f6be80c2850 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:02.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.184+0000 7f6c010ca640 1 -- 172.21.15.117:0/2286907278 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6be80b6670 msgr2=0x7f6be80b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:02.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.184+0000 7f6c010ca640 1 --2- 172.21.15.117:0/2286907278 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6be80b6670 0x7f6be80b8b80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6be4000c00 tx=0x7f6be4031400 comp rx=0 tx=0).stop 2024-09-21T05:56:02.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.184+0000 7f6c010ca640 1 -- 172.21.15.117:0/2286907278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bfc105300 msgr2=0x7f6bfc19ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:02.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.184+0000 7f6c010ca640 1 --2- 172.21.15.117:0/2286907278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bfc105300 0x7f6bfc19ab00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc06ad30 tx=0x7f6bec00ee50 comp rx=0 tx=0).stop 2024-09-21T05:56:02.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.184+0000 7f6bfb59e640 1 -- 172.21.15.117:0/2286907278 reap_dead start 2024-09-21T05:56:02.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.185+0000 7f6c010ca640 1 -- 172.21.15.117:0/2286907278 shutdown_connections 2024-09-21T05:56:02.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.185+0000 7f6c010ca640 1 -- 172.21.15.117:0/2286907278 >> 172.21.15.117:0/2286907278 conn(0x7f6bfc079730 msgr2=0x7f6bfc0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:02.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.185+0000 7f6c010ca640 1 -- 172.21.15.117:0/2286907278 shutdown_connections 2024-09-21T05:56:02.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.185+0000 7f6c010ca640 1 -- 172.21.15.117:0/2286907278 wait complete. 2024-09-21T05:56:02.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.059+0000 7fa651088640 1 -- 172.21.15.117:0/141771882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa64c102a60 msgr2=0x7fa64c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:02.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.059+0000 7fa651088640 1 --2- 172.21.15.117:0/141771882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa64c102a60 0x7fa64c102e70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa63c009900 tx=0x7fa63c02a410 comp rx=0 tx=0).stop 2024-09-21T05:56:02.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.060+0000 7fa651088640 1 -- 172.21.15.117:0/141771882 shutdown_connections 2024-09-21T05:56:02.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.060+0000 7fa651088640 1 --2- 172.21.15.117:0/141771882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa64c108cf0 0x7fa64c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:02.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.060+0000 7fa651088640 1 --2- 172.21.15.117:0/141771882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa64c1083a0 0x7fa64c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:02.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.060+0000 7fa651088640 1 --2- 172.21.15.117:0/141771882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa64c102a60 0x7fa64c102e70 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:02.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.060+0000 7fa651088640 1 -- 172.21.15.117:0/141771882 >> 172.21.15.117:0/141771882 conn(0x7fa64c079730 msgr2=0x7fa64c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:02.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.060+0000 7fa651088640 1 -- 172.21.15.117:0/141771882 shutdown_connections 2024-09-21T05:56:02.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.060+0000 7fa651088640 1 -- 172.21.15.117:0/141771882 wait complete. 2024-09-21T05:56:02.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.061+0000 7fa651088640 1 Processor -- start 2024-09-21T05:56:02.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.061+0000 7fa651088640 1 -- start start 2024-09-21T05:56:02.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.061+0000 7fa651088640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa64c102a60 0x7fa64c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:02.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.061+0000 7fa651088640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa64c1083a0 0x7fa64c19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:02.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.061+0000 7fa651088640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa64c108cf0 0x7fa64c19fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:02.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.061+0000 7fa651088640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c113ac0 con 0x7fa64c102a60 2024-09-21T05:56:02.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.061+0000 7fa651088640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c113940 con 0x7fa64c108cf0 2024-09-21T05:56:02.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.061+0000 7fa651088640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa64c113c40 con 0x7fa64c1083a0 2024-09-21T05:56:02.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.062+0000 7fa64ad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa64c102a60 0x7fa64c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:02.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.062+0000 7fa64ad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa64c102a60 0x7fa64c19a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36388/0 (socket says 172.21.15.117:36388) 2024-09-21T05:56:02.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.062+0000 7fa64ad9d640 1 -- 172.21.15.117:0/541225702 learned_addr learned my addr 172.21.15.117:0/541225702 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:02.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.062+0000 7fa64b59e640 1 --2- 172.21.15.117:0/541225702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa64c108cf0 0x7fa64c19fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:02.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.062+0000 7fa64a59c640 1 --2- 172.21.15.117:0/541225702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa64c1083a0 0x7fa64c19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:02.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.062+0000 7fa64ad9d640 1 -- 172.21.15.117:0/541225702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa64c1083a0 msgr2=0x7fa64c19aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:02.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.062+0000 7fa64ad9d640 1 --2- 172.21.15.117:0/541225702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa64c1083a0 0x7fa64c19aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:02.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.062+0000 7fa64ad9d640 1 -- 172.21.15.117:0/541225702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa64c108cf0 msgr2=0x7fa64c19fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:02.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.062+0000 7fa64ad9d640 1 --2- 172.21.15.117:0/541225702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa64c108cf0 0x7fa64c19fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:02.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.062+0000 7fa64ad9d640 1 -- 172.21.15.117:0/541225702 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa64c1a0360 con 0x7fa64c102a60 2024-09-21T05:56:02.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.062+0000 7fa64ad9d640 1 --2- 172.21.15.117:0/541225702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa64c102a60 0x7fa64c19a4b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa63c006010 tx=0x7fa63c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:02.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.062+0000 7fa633fff640 1 -- 172.21.15.117:0/541225702 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c03f030 con 0x7fa64c102a60 2024-09-21T05:56:02.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.062+0000 7fa633fff640 1 -- 172.21.15.117:0/541225702 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa63c003800 con 0x7fa64c102a60 2024-09-21T05:56:02.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.062+0000 7fa651088640 1 -- 172.21.15.117:0/541225702 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa64c1a05f0 con 0x7fa64c102a60 2024-09-21T05:56:02.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.062+0000 7fa633fff640 1 -- 172.21.15.117:0/541225702 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c035070 con 0x7fa64c102a60 2024-09-21T05:56:02.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.062+0000 7fa651088640 1 -- 172.21.15.117:0/541225702 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa64c1a6e70 con 0x7fa64c102a60 2024-09-21T05:56:02.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.063+0000 7fa651088640 1 -- 172.21.15.117:0/541225702 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa64c063b50 con 0x7fa64c102a60 2024-09-21T05:56:02.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.065+0000 7fa633fff640 1 -- 172.21.15.117:0/541225702 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7fa63c003cb0 con 0x7fa64c102a60 2024-09-21T05:56:02.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.066+0000 7fa633fff640 1 --2- 172.21.15.117:0/541225702 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa6340b6620 0x7fa6340b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:02.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.066+0000 7fa64a59c640 1 --2- 172.21.15.117:0/541225702 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa6340b6620 0x7fa6340b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:02.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.066+0000 7fa633fff640 1 -- 172.21.15.117:0/541225702 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fa63c108340 con 0x7fa64c102a60 2024-09-21T05:56:02.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.067+0000 7fa633fff640 1 --2- 172.21.15.117:0/541225702 >> [v2:172.21.15.117:6826/3453855195,v1:172.21.15.117:6827/3453855195] conn(0x7fa6340c0310 0x7fa6340c2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:02.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.067+0000 7fa633fff640 1 -- 172.21.15.117:0/541225702 --> [v2:172.21.15.117:6826/3453855195,v1:172.21.15.117:6827/3453855195] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa63c03be50 con 0x7fa6340c0310 2024-09-21T05:56:02.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.067+0000 7fa633fff640 1 -- 172.21.15.117:0/541225702 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fa63c108790 con 0x7fa64c102a60 2024-09-21T05:56:02.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.067+0000 7fa64a59c640 1 --2- 172.21.15.117:0/541225702 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa6340b6620 0x7fa6340b8b30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa64c079cd0 tx=0x7fa640008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:02.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.067+0000 7fa64b59e640 1 --2- 172.21.15.117:0/541225702 >> [v2:172.21.15.117:6826/3453855195,v1:172.21.15.117:6827/3453855195] conn(0x7fa6340c0310 0x7fa6340c2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:02.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.067+0000 7fa64b59e640 1 --2- 172.21.15.117:0/541225702 >> [v2:172.21.15.117:6826/3453855195,v1:172.21.15.117:6827/3453855195] conn(0x7fa6340c0310 0x7fa6340c2800 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:02.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.068+0000 7fa633fff640 1 -- 172.21.15.117:0/541225702 <== osd.9 v2:172.21.15.117:6826/3453855195 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa63c03be50 con 0x7fa6340c0310 2024-09-21T05:56:02.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.084+0000 7fa651088640 1 -- 172.21.15.117:0/541225702 --> [v2:172.21.15.117:6826/3453855195,v1:172.21.15.117:6827/3453855195] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa64c063f50 con 0x7fa6340c0310 2024-09-21T05:56:02.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.084+0000 7fa633fff640 1 -- 172.21.15.117:0/541225702 <== osd.9 v2:172.21.15.117:6826/3453855195 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa64c063f50 con 0x7fa6340c0310 2024-09-21T05:56:02.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.085+0000 7fa651088640 1 -- 172.21.15.117:0/541225702 >> [v2:172.21.15.117:6826/3453855195,v1:172.21.15.117:6827/3453855195] conn(0x7fa6340c0310 msgr2=0x7fa6340c2800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:02.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.085+0000 7fa651088640 1 --2- 172.21.15.117:0/541225702 >> [v2:172.21.15.117:6826/3453855195,v1:172.21.15.117:6827/3453855195] conn(0x7fa6340c0310 0x7fa6340c2800 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:02.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.085+0000 7fa651088640 1 -- 172.21.15.117:0/541225702 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa6340b6620 msgr2=0x7fa6340b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:02.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.085+0000 7fa651088640 1 --2- 172.21.15.117:0/541225702 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa6340b6620 0x7fa6340b8b30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa64c079cd0 tx=0x7fa640008040 comp rx=0 tx=0).stop 2024-09-21T05:56:02.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.085+0000 7fa651088640 1 -- 172.21.15.117:0/541225702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa64c102a60 msgr2=0x7fa64c19a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:02.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.085+0000 7fa651088640 1 --2- 172.21.15.117:0/541225702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa64c102a60 0x7fa64c19a4b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa63c006010 tx=0x7fa63c0042f0 comp rx=0 tx=0).stop 2024-09-21T05:56:02.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.085+0000 7fa64b59e640 1 -- 172.21.15.117:0/541225702 reap_dead start 2024-09-21T05:56:02.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.085+0000 7fa651088640 1 -- 172.21.15.117:0/541225702 shutdown_connections 2024-09-21T05:56:02.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.085+0000 7fa651088640 1 -- 172.21.15.117:0/541225702 >> 172.21.15.117:0/541225702 conn(0x7fa64c079730 msgr2=0x7fa64c1014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:02.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.085+0000 7fa651088640 1 -- 172.21.15.117:0/541225702 shutdown_connections 2024-09-21T05:56:02.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:02.085+0000 7fa651088640 1 -- 172.21.15.117:0/541225702 wait complete. 2024-09-21T05:56:03.105 INFO:teuthology.orchestra.run.smithi117.stdout:249108103201 2024-09-21T05:56:03.106 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.9 2024-09-21T05:56:03.732 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:03 smithi117 ceph-mon[30947]: pgmap v289: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T05:56:03.758 INFO:teuthology.orchestra.run.smithi117.stdout:107374182463 2024-09-21T05:56:03.758 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.3 2024-09-21T05:56:03.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:03 smithi138 ceph-mon[34391]: pgmap v289: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T05:56:03.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:03 smithi192 ceph-mon[34700]: pgmap v289: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-21T05:56:04.404 INFO:teuthology.orchestra.run.smithi117.stdout:223338299430 2024-09-21T05:56:04.405 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.8 2024-09-21T05:56:04.798 INFO:teuthology.orchestra.run.smithi117.stdout:60129542215 2024-09-21T05:56:04.799 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.1 2024-09-21T05:56:04.804 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:04.818 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:04.856 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:05.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.550+0000 7f389ac8c640 1 -- 172.21.15.117:0/75061484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3894102a80 msgr2=0x7f3894102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:05.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.550+0000 7f389ac8c640 1 --2- 172.21.15.117:0/75061484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3894102a80 0x7f3894102e90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3888009900 tx=0x7f388802a410 comp rx=0 tx=0).stop 2024-09-21T05:56:05.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.550+0000 7f389ac8c640 1 -- 172.21.15.117:0/75061484 shutdown_connections 2024-09-21T05:56:05.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.550+0000 7f389ac8c640 1 --2- 172.21.15.117:0/75061484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3894108d10 0x7f389410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.550+0000 7f389ac8c640 1 --2- 172.21.15.117:0/75061484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38941083c0 0x7f38941087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.550+0000 7f389ac8c640 1 --2- 172.21.15.117:0/75061484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3894102a80 0x7f3894102e90 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.550+0000 7f389ac8c640 1 -- 172.21.15.117:0/75061484 >> 172.21.15.117:0/75061484 conn(0x7f3894079740 msgr2=0x7f3894079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:05.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.550+0000 7f389ac8c640 1 -- 172.21.15.117:0/75061484 shutdown_connections 2024-09-21T05:56:05.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.550+0000 7f389ac8c640 1 -- 172.21.15.117:0/75061484 wait complete. 2024-09-21T05:56:05.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.551+0000 7f389ac8c640 1 Processor -- start 2024-09-21T05:56:05.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.551+0000 7f389ac8c640 1 -- start start 2024-09-21T05:56:05.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.551+0000 7f389ac8c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3894102a80 0x7f389419b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:05.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.551+0000 7f389ac8c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38941083c0 0x7f389419b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:05.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.551+0000 7f389ac8c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3894108d10 0x7f389419bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:05.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.551+0000 7f389ac8c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3894112980 con 0x7f3894102a80 2024-09-21T05:56:05.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.551+0000 7f389ac8c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3894112800 con 0x7f3894108d10 2024-09-21T05:56:05.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.551+0000 7f389ac8c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3894112b00 con 0x7f38941083c0 2024-09-21T05:56:05.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.551+0000 7f3898a28640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3894102a80 0x7f389419b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:05.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.551+0000 7f3898a28640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3894102a80 0x7f389419b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37866/0 (socket says 172.21.15.117:37866) 2024-09-21T05:56:05.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.551+0000 7f3898a28640 1 -- 172.21.15.117:0/2727592689 learned_addr learned my addr 172.21.15.117:0/2727592689 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:05.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.552+0000 7f3893fff640 1 --2- 172.21.15.117:0/2727592689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38941083c0 0x7f389419b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:05.560 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.552+0000 7f3898a28640 1 -- 172.21.15.117:0/2727592689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38941083c0 msgr2=0x7f389419b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:05.560 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.552+0000 7f3898a28640 1 --2- 172.21.15.117:0/2727592689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38941083c0 0x7f389419b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.560 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.552+0000 7f3898a28640 1 -- 172.21.15.117:0/2727592689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3894108d10 msgr2=0x7f389419bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:05.560 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.552+0000 7f3899229640 1 --2- 172.21.15.117:0/2727592689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3894108d10 0x7f389419bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:05.560 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.552+0000 7f3898a28640 1 --2- 172.21.15.117:0/2727592689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3894108d10 0x7f389419bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.560 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.552+0000 7f3898a28640 1 -- 172.21.15.117:0/2727592689 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38941a64b0 con 0x7f3894102a80 2024-09-21T05:56:05.560 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.552+0000 7f3899229640 1 --2- 172.21.15.117:0/2727592689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3894108d10 0x7f389419bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:05.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.552+0000 7f3898a28640 1 --2- 172.21.15.117:0/2727592689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3894102a80 0x7f389419b230 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3888006010 tx=0x7f38880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:05.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.552+0000 7f3891ffb640 1 -- 172.21.15.117:0/2727592689 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388803f030 con 0x7f3894102a80 2024-09-21T05:56:05.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.552+0000 7f3891ffb640 1 -- 172.21.15.117:0/2727592689 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3888003800 con 0x7f3894102a80 2024-09-21T05:56:05.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.552+0000 7f389ac8c640 1 -- 172.21.15.117:0/2727592689 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38941a6740 con 0x7f3894102a80 2024-09-21T05:56:05.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.552+0000 7f3891ffb640 1 -- 172.21.15.117:0/2727592689 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3888035070 con 0x7f3894102a80 2024-09-21T05:56:05.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.552+0000 7f389ac8c640 1 -- 172.21.15.117:0/2727592689 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38941a6c20 con 0x7f3894102a80 2024-09-21T05:56:05.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.553+0000 7f389ac8c640 1 -- 172.21.15.117:0/2727592689 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3894103d00 con 0x7f3894102a80 2024-09-21T05:56:05.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.555+0000 7f3891ffb640 1 -- 172.21.15.117:0/2727592689 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f3888003cb0 con 0x7f3894102a80 2024-09-21T05:56:05.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.556+0000 7f3891ffb640 1 --2- 172.21.15.117:0/2727592689 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f38840b65f0 0x7f38840b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:05.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.556+0000 7f3893fff640 1 --2- 172.21.15.117:0/2727592689 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f38840b65f0 0x7f38840b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:05.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.556+0000 7f3891ffb640 1 -- 172.21.15.117:0/2727592689 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f38881081d0 con 0x7f3894102a80 2024-09-21T05:56:05.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.556+0000 7f3893fff640 1 --2- 172.21.15.117:0/2727592689 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f38840b65f0 0x7f38840b8b00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3880005e10 tx=0x7f3880005da0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:05.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.559+0000 7f3891ffb640 1 -- 172.21.15.117:0/2727592689 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f38880ce860 con 0x7f3894102a80 2024-09-21T05:56:05.664 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:05 smithi117 ceph-mon[30947]: pgmap v290: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-21T05:56:05.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.748+0000 7f389ac8c640 1 -- 172.21.15.117:0/2727592689 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f389410a550 con 0x7f3894102a80 2024-09-21T05:56:05.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.749+0000 7f3891ffb640 1 -- 172.21.15.117:0/2727592689 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f38880d2c40 con 0x7f3894102a80 2024-09-21T05:56:05.752 INFO:teuthology.orchestra.run.smithi117.stdout:270582939676 2024-09-21T05:56:05.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.751+0000 7f389ac8c640 1 -- 172.21.15.117:0/2727592689 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f38840b65f0 msgr2=0x7f38840b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:05.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.751+0000 7f389ac8c640 1 --2- 172.21.15.117:0/2727592689 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f38840b65f0 0x7f38840b8b00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3880005e10 tx=0x7f3880005da0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.751+0000 7f389ac8c640 1 -- 172.21.15.117:0/2727592689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3894102a80 msgr2=0x7f389419b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:05.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.751+0000 7f389ac8c640 1 --2- 172.21.15.117:0/2727592689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3894102a80 0x7f389419b230 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3888006010 tx=0x7f38880042f0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.751+0000 7f389ac8c640 1 -- 172.21.15.117:0/2727592689 shutdown_connections 2024-09-21T05:56:05.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.751+0000 7f389ac8c640 1 --2- 172.21.15.117:0/2727592689 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f38840b65f0 0x7f38840b8b00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.751+0000 7f389ac8c640 1 --2- 172.21.15.117:0/2727592689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3894108d10 0x7f389419bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.751+0000 7f389ac8c640 1 --2- 172.21.15.117:0/2727592689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38941083c0 0x7f389419b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.751+0000 7f389ac8c640 1 --2- 172.21.15.117:0/2727592689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3894102a80 0x7f389419b230 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.751+0000 7f389ac8c640 1 -- 172.21.15.117:0/2727592689 >> 172.21.15.117:0/2727592689 conn(0x7f3894079740 msgr2=0x7f3894101480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:05.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.751+0000 7f389ac8c640 1 -- 172.21.15.117:0/2727592689 shutdown_connections 2024-09-21T05:56:05.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.751+0000 7f389ac8c640 1 -- 172.21.15.117:0/2727592689 wait complete. 2024-09-21T05:56:05.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.754+0000 7f4206c61640 1 -- 172.21.15.117:0/445576041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4200108cf0 msgr2=0x7f420010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:05.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.754+0000 7f4206c61640 1 --2- 172.21.15.117:0/445576041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4200108cf0 0x7f420010f520 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f41f8009ca0 tx=0x7f41f802a8c0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.754+0000 7f4206c61640 1 -- 172.21.15.117:0/445576041 shutdown_connections 2024-09-21T05:56:05.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.754+0000 7f4206c61640 1 --2- 172.21.15.117:0/445576041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4200108cf0 0x7f420010f520 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.754+0000 7f4206c61640 1 --2- 172.21.15.117:0/445576041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f42001083a0 0x7f42001087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.754+0000 7f4206c61640 1 --2- 172.21.15.117:0/445576041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4200102a60 0x7f4200102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.754+0000 7f4206c61640 1 -- 172.21.15.117:0/445576041 >> 172.21.15.117:0/445576041 conn(0x7f4200079730 msgr2=0x7f4200079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:05.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.754+0000 7f4206c61640 1 -- 172.21.15.117:0/445576041 shutdown_connections 2024-09-21T05:56:05.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.754+0000 7f4206c61640 1 -- 172.21.15.117:0/445576041 wait complete. 2024-09-21T05:56:05.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.755+0000 7f4206c61640 1 Processor -- start 2024-09-21T05:56:05.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.755+0000 7f4206c61640 1 -- start start 2024-09-21T05:56:05.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.755+0000 7f4206c61640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4200102a60 0x7f420019a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:05.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.755+0000 7f4206c61640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42001083a0 0x7f420019a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:05.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.755+0000 7f4206c61640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4200108cf0 0x7f420019fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:05.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.755+0000 7f4206c61640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4200112960 con 0x7f4200108cf0 2024-09-21T05:56:05.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.755+0000 7f4206c61640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f42001127e0 con 0x7f4200102a60 2024-09-21T05:56:05.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.755+0000 7f4206c61640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4200112ae0 con 0x7f42001083a0 2024-09-21T05:56:05.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.755+0000 7f42051fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4200108cf0 0x7f420019fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:05.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.755+0000 7f41fffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42001083a0 0x7f420019a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:05.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.755+0000 7f42051fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4200108cf0 0x7f420019fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37886/0 (socket says 172.21.15.117:37886) 2024-09-21T05:56:05.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.755+0000 7f42051fe640 1 -- 172.21.15.117:0/1097324828 learned_addr learned my addr 172.21.15.117:0/1097324828 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:05.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.755+0000 7f42049fd640 1 --2- 172.21.15.117:0/1097324828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4200102a60 0x7f420019a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:05.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.755+0000 7f42051fe640 1 -- 172.21.15.117:0/1097324828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42001083a0 msgr2=0x7f420019a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:05.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.755+0000 7f42051fe640 1 --2- 172.21.15.117:0/1097324828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42001083a0 0x7f420019a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.755+0000 7f42051fe640 1 -- 172.21.15.117:0/1097324828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4200102a60 msgr2=0x7f420019a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:05.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.755+0000 7f42051fe640 1 --2- 172.21.15.117:0/1097324828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4200102a60 0x7f420019a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.755+0000 7f42051fe640 1 -- 172.21.15.117:0/1097324828 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42001a0270 con 0x7f4200108cf0 2024-09-21T05:56:05.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.755+0000 7f41fffff640 1 --2- 172.21.15.117:0/1097324828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42001083a0 0x7f420019a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:56:05.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.756+0000 7f42051fe640 1 --2- 172.21.15.117:0/1097324828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4200108cf0 0x7f420019fc00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f41f8005cc0 tx=0x7f41f802f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:05.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.756+0000 7f41fdffb640 1 -- 172.21.15.117:0/1097324828 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f8003cf0 con 0x7f4200108cf0 2024-09-21T05:56:05.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.756+0000 7f4206c61640 1 -- 172.21.15.117:0/1097324828 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42001a0500 con 0x7f4200108cf0 2024-09-21T05:56:05.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.756+0000 7f41fdffb640 1 -- 172.21.15.117:0/1097324828 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41f80044f0 con 0x7f4200108cf0 2024-09-21T05:56:05.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.756+0000 7f41fdffb640 1 -- 172.21.15.117:0/1097324828 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f802fc30 con 0x7f4200108cf0 2024-09-21T05:56:05.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.756+0000 7f4206c61640 1 -- 172.21.15.117:0/1097324828 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42001a0a40 con 0x7f4200108cf0 2024-09-21T05:56:05.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.757+0000 7f4206c61640 1 -- 172.21.15.117:0/1097324828 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4200102e70 con 0x7f4200108cf0 2024-09-21T05:56:05.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.760+0000 7f41fdffb640 1 -- 172.21.15.117:0/1097324828 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f41f8004040 con 0x7f4200108cf0 2024-09-21T05:56:05.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.760+0000 7f41fdffb640 1 --2- 172.21.15.117:0/1097324828 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f41f00b6620 0x7f41f00b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:05.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.761+0000 7f41fdffb640 1 -- 172.21.15.117:0/1097324828 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f41f81089d0 con 0x7f4200108cf0 2024-09-21T05:56:05.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.761+0000 7f42049fd640 1 --2- 172.21.15.117:0/1097324828 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f41f00b6620 0x7f41f00b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:05.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.761+0000 7f42049fd640 1 --2- 172.21.15.117:0/1097324828 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f41f00b6620 0x7f41f00b8b30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f41f400abe0 tx=0x7f41f400a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:05.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.765+0000 7f41fdffb640 1 -- 172.21.15.117:0/1097324828 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f41f80cf060 con 0x7f4200108cf0 2024-09-21T05:56:05.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:05 smithi138 ceph-mon[34391]: pgmap v290: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-21T05:56:05.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.950+0000 7f4206c61640 1 -- 172.21.15.117:0/1097324828 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f420002e640 con 0x7f4200108cf0 2024-09-21T05:56:05.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.951+0000 7f41fdffb640 1 -- 172.21.15.117:0/1097324828 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f41f80d3440 con 0x7f4200108cf0 2024-09-21T05:56:05.954 INFO:teuthology.orchestra.run.smithi117.stdout:38654705744 2024-09-21T05:56:05.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.953+0000 7f4206c61640 1 -- 172.21.15.117:0/1097324828 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f41f00b6620 msgr2=0x7f41f00b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:05.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.953+0000 7f4206c61640 1 --2- 172.21.15.117:0/1097324828 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f41f00b6620 0x7f41f00b8b30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f41f400abe0 tx=0x7f41f400a2d0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.953+0000 7f4206c61640 1 -- 172.21.15.117:0/1097324828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4200108cf0 msgr2=0x7f420019fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:05.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.953+0000 7f4206c61640 1 --2- 172.21.15.117:0/1097324828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4200108cf0 0x7f420019fc00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f41f8005cc0 tx=0x7f41f802f470 comp rx=0 tx=0).stop 2024-09-21T05:56:05.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.953+0000 7f4206c61640 1 -- 172.21.15.117:0/1097324828 shutdown_connections 2024-09-21T05:56:05.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.953+0000 7f4206c61640 1 --2- 172.21.15.117:0/1097324828 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f41f00b6620 0x7f41f00b8b30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.953+0000 7f4206c61640 1 --2- 172.21.15.117:0/1097324828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4200108cf0 0x7f420019fc00 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.953+0000 7f4206c61640 1 --2- 172.21.15.117:0/1097324828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42001083a0 0x7f420019a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.953+0000 7f4206c61640 1 --2- 172.21.15.117:0/1097324828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4200102a60 0x7f420019a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:05.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.953+0000 7f4206c61640 1 -- 172.21.15.117:0/1097324828 >> 172.21.15.117:0/1097324828 conn(0x7f4200079730 msgr2=0x7f420006b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:05.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.953+0000 7f4206c61640 1 -- 172.21.15.117:0/1097324828 shutdown_connections 2024-09-21T05:56:05.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:05.953+0000 7f4206c61640 1 -- 172.21.15.117:0/1097324828 wait complete. 2024-09-21T05:56:05.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:05 smithi192 ceph-mon[34700]: pgmap v290: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-21T05:56:06.616 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705742 got 38654705744 for osd.0 2024-09-21T05:56:06.616 DEBUG:teuthology.parallel:result is None 2024-09-21T05:56:06.699 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:06 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2727592689' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T05:56:06.699 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:06 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/1097324828' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T05:56:06.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:06 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2727592689' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T05:56:06.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:06 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/1097324828' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T05:56:07.035 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939674 got 270582939676 for osd.10 2024-09-21T05:56:07.035 DEBUG:teuthology.parallel:result is None 2024-09-21T05:56:07.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:06 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2727592689' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T05:56:07.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:06 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/1097324828' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T05:56:07.292 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:07.296 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:07.297 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:07.297 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:07.297 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:07.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.472+0000 7f681ea95640 1 -- 172.21.15.117:0/692112102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68181094a0 msgr2=0x7f68181098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:07.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.472+0000 7f681ea95640 1 --2- 172.21.15.117:0/692112102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68181094a0 0x7f68181098b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6808009900 tx=0x7f680802a410 comp rx=0 tx=0).stop 2024-09-21T05:56:07.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.472+0000 7f681ea95640 1 -- 172.21.15.117:0/692112102 shutdown_connections 2024-09-21T05:56:07.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.472+0000 7f681ea95640 1 --2- 172.21.15.117:0/692112102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f68181094a0 0x7f68181098b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:07.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.472+0000 7f681ea95640 1 --2- 172.21.15.117:0/692112102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6818109df0 0x7f6818110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:07.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.472+0000 7f681ea95640 1 --2- 172.21.15.117:0/692112102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6818103b60 0x7f6818103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:07.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.472+0000 7f681ea95640 1 -- 172.21.15.117:0/692112102 >> 172.21.15.117:0/692112102 conn(0x7f681807b850 msgr2=0x7f681807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:07.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.472+0000 7f681ea95640 1 -- 172.21.15.117:0/692112102 shutdown_connections 2024-09-21T05:56:07.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.472+0000 7f681ea95640 1 -- 172.21.15.117:0/692112102 wait complete. 2024-09-21T05:56:07.477 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.473+0000 7f681ea95640 1 Processor -- start 2024-09-21T05:56:07.477 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.473+0000 7f681ea95640 1 -- start start 2024-09-21T05:56:07.477 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.473+0000 7f681ea95640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6818103b60 0x7f681819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:07.477 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.473+0000 7f681ea95640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68181094a0 0x7f681819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:07.477 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.473+0000 7f681ea95640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6818109df0 0x7f68181a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:07.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.473+0000 7f681ea95640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6818113ab0 con 0x7f6818103b60 2024-09-21T05:56:07.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.473+0000 7f681ea95640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6818113930 con 0x7f68181094a0 2024-09-21T05:56:07.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.473+0000 7f681ea95640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6818113c30 con 0x7f6818109df0 2024-09-21T05:56:07.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.473+0000 7f681c831640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6818103b60 0x7f681819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:07.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.473+0000 7f681c831640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6818103b60 0x7f681819c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37910/0 (socket says 172.21.15.117:37910) 2024-09-21T05:56:07.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.473+0000 7f681c831640 1 -- 172.21.15.117:0/1058090681 learned_addr learned my addr 172.21.15.117:0/1058090681 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:07.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.473+0000 7f681c831640 1 -- 172.21.15.117:0/1058090681 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6818109df0 msgr2=0x7f68181a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:07.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.473+0000 7f681c831640 1 --2- 172.21.15.117:0/1058090681 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6818109df0 0x7f68181a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:07.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.473+0000 7f680ffff640 1 --2- 172.21.15.117:0/1058090681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68181094a0 0x7f681819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:07.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.473+0000 7f681c831640 1 -- 172.21.15.117:0/1058090681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68181094a0 msgr2=0x7f681819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:07.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.473+0000 7f681c831640 1 --2- 172.21.15.117:0/1058090681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68181094a0 0x7f681819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:07.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.473+0000 7f681c831640 1 -- 172.21.15.117:0/1058090681 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68181a2610 con 0x7f6818103b60 2024-09-21T05:56:07.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.473+0000 7f680ffff640 1 --2- 172.21.15.117:0/1058090681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68181094a0 0x7f681819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:07.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.474+0000 7f681c831640 1 --2- 172.21.15.117:0/1058090681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6818103b60 0x7f681819c6a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f681400d950 tx=0x7f681400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:07.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.474+0000 7f680dffb640 1 -- 172.21.15.117:0/1058090681 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6814017070 con 0x7f6818103b60 2024-09-21T05:56:07.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.474+0000 7f681ea95640 1 -- 172.21.15.117:0/1058090681 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68181a2900 con 0x7f6818103b60 2024-09-21T05:56:07.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.474+0000 7f680dffb640 1 -- 172.21.15.117:0/1058090681 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6814009c30 con 0x7f6818103b60 2024-09-21T05:56:07.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.474+0000 7f680dffb640 1 -- 172.21.15.117:0/1058090681 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681400f620 con 0x7f6818103b60 2024-09-21T05:56:07.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.474+0000 7f681ea95640 1 -- 172.21.15.117:0/1058090681 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68181a9060 con 0x7f6818103b60 2024-09-21T05:56:07.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.475+0000 7f681ea95640 1 -- 172.21.15.117:0/1058090681 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6818103f70 con 0x7f6818103b60 2024-09-21T05:56:07.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.476+0000 7f680dffb640 1 -- 172.21.15.117:0/1058090681 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f6814010460 con 0x7f6818103b60 2024-09-21T05:56:07.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.477+0000 7f680dffb640 1 --2- 172.21.15.117:0/1058090681 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f68000b65f0 0x7f68000b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:07.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.477+0000 7f680ffff640 1 --2- 172.21.15.117:0/1058090681 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f68000b65f0 0x7f68000b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:07.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.477+0000 7f680dffb640 1 -- 172.21.15.117:0/1058090681 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f68140e72c0 con 0x7f6818103b60 2024-09-21T05:56:07.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.478+0000 7f680ffff640 1 --2- 172.21.15.117:0/1058090681 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f68000b65f0 0x7f68000b8b00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6808000c00 tx=0x7f6808031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:07.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.483+0000 7f680dffb640 1 -- 172.21.15.117:0/1058090681 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f68140ad950 con 0x7f6818103b60 2024-09-21T05:56:07.670 INFO:teuthology.orchestra.run.smithi117.stdout:197568495660 2024-09-21T05:56:07.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.667+0000 7f681ea95640 1 -- 172.21.15.117:0/1058090681 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f68181a93d0 con 0x7f6818103b60 2024-09-21T05:56:07.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.667+0000 7f680dffb640 1 -- 172.21.15.117:0/1058090681 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6814009de0 con 0x7f6818103b60 2024-09-21T05:56:07.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.669+0000 7f681ea95640 1 -- 172.21.15.117:0/1058090681 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f68000b65f0 msgr2=0x7f68000b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:07.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.669+0000 7f681ea95640 1 --2- 172.21.15.117:0/1058090681 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f68000b65f0 0x7f68000b8b00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6808000c00 tx=0x7f6808031400 comp rx=0 tx=0).stop 2024-09-21T05:56:07.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.669+0000 7f681ea95640 1 -- 172.21.15.117:0/1058090681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6818103b60 msgr2=0x7f681819c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:07.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.669+0000 7f681ea95640 1 --2- 172.21.15.117:0/1058090681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6818103b60 0x7f681819c6a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f681400d950 tx=0x7f681400de10 comp rx=0 tx=0).stop 2024-09-21T05:56:07.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.669+0000 7f681ea95640 1 -- 172.21.15.117:0/1058090681 shutdown_connections 2024-09-21T05:56:07.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.669+0000 7f681ea95640 1 --2- 172.21.15.117:0/1058090681 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f68000b65f0 0x7f68000b8b00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:07.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.669+0000 7f681ea95640 1 --2- 172.21.15.117:0/1058090681 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6818109df0 0x7f68181a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:07.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.669+0000 7f681ea95640 1 --2- 172.21.15.117:0/1058090681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68181094a0 0x7f681819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:07.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.669+0000 7f681ea95640 1 --2- 172.21.15.117:0/1058090681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6818103b60 0x7f681819c6a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:07.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.669+0000 7f681ea95640 1 -- 172.21.15.117:0/1058090681 >> 172.21.15.117:0/1058090681 conn(0x7f681807b850 msgr2=0x7f6818107310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:07.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.669+0000 7f681ea95640 1 -- 172.21.15.117:0/1058090681 shutdown_connections 2024-09-21T05:56:07.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:07.669+0000 7f681ea95640 1 -- 172.21.15.117:0/1058090681 wait complete. 2024-09-21T05:56:07.898 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:07 smithi117 ceph-mon[30947]: pgmap v291: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-09-21T05:56:07.925 INFO:teuthology.orchestra.run.smithi117.stdout:292057776150 2024-09-21T05:56:07.925 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.11 2024-09-21T05:56:07.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:07 smithi192 ceph-mon[34700]: pgmap v291: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-09-21T05:56:08.104 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:08.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:07 smithi138 ceph-mon[34391]: pgmap v291: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-09-21T05:56:08.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:08 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/1058090681' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T05:56:08.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.776+0000 7fef0bd96640 1 -- 172.21.15.117:0/566149590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef04103b60 msgr2=0x7fef04103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:08.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.776+0000 7fef0bd96640 1 --2- 172.21.15.117:0/566149590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef04103b60 0x7fef04103f70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7feef4009ca0 tx=0x7feef402a8c0 comp rx=0 tx=0).stop 2024-09-21T05:56:08.789 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.777+0000 7fef0bd96640 1 -- 172.21.15.117:0/566149590 shutdown_connections 2024-09-21T05:56:08.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.777+0000 7fef0bd96640 1 --2- 172.21.15.117:0/566149590 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef04109df0 0x7fef04110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:08.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.777+0000 7fef0bd96640 1 --2- 172.21.15.117:0/566149590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef041094a0 0x7fef041098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:08.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.777+0000 7fef0bd96640 1 --2- 172.21.15.117:0/566149590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef04103b60 0x7fef04103f70 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:08.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.777+0000 7fef0bd96640 1 -- 172.21.15.117:0/566149590 >> 172.21.15.117:0/566149590 conn(0x7fef0407b850 msgr2=0x7fef0407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:08.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.777+0000 7fef0bd96640 1 -- 172.21.15.117:0/566149590 shutdown_connections 2024-09-21T05:56:08.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.777+0000 7fef0bd96640 1 -- 172.21.15.117:0/566149590 wait complete. 2024-09-21T05:56:08.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.777+0000 7fef0bd96640 1 Processor -- start 2024-09-21T05:56:08.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.777+0000 7fef0bd96640 1 -- start start 2024-09-21T05:56:08.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.778+0000 7fef0bd96640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef04103b60 0x7fef0419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:08.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.778+0000 7fef0bd96640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef041094a0 0x7fef0419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:08.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.778+0000 7fef0bd96640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef04109df0 0x7fef0419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:08.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.778+0000 7fef0bd96640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fef04113b00 con 0x7fef04103b60 2024-09-21T05:56:08.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.778+0000 7fef0bd96640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fef04113980 con 0x7fef041094a0 2024-09-21T05:56:08.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.778+0000 7fef0bd96640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fef04113c80 con 0x7fef04109df0 2024-09-21T05:56:08.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.778+0000 7fef09b32640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef04103b60 0x7fef0419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:08.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.778+0000 7fef09b32640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef04103b60 0x7fef0419a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37926/0 (socket says 172.21.15.117:37926) 2024-09-21T05:56:08.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.778+0000 7fef09b32640 1 -- 172.21.15.117:0/778454413 learned_addr learned my addr 172.21.15.117:0/778454413 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:08.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.778+0000 7fef09331640 1 --2- 172.21.15.117:0/778454413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef041094a0 0x7fef0419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:08.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.778+0000 7fef0a333640 1 --2- 172.21.15.117:0/778454413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef04109df0 0x7fef0419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:08.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.778+0000 7fef09b32640 1 -- 172.21.15.117:0/778454413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef04109df0 msgr2=0x7fef0419fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:08.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.778+0000 7fef09b32640 1 --2- 172.21.15.117:0/778454413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef04109df0 0x7fef0419fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:08.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.778+0000 7fef09b32640 1 -- 172.21.15.117:0/778454413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef041094a0 msgr2=0x7fef0419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:08.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.778+0000 7fef09b32640 1 --2- 172.21.15.117:0/778454413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef041094a0 0x7fef0419aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:08.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.778+0000 7fef09b32640 1 -- 172.21.15.117:0/778454413 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef041a0440 con 0x7fef04103b60 2024-09-21T05:56:08.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.778+0000 7fef0a333640 1 --2- 172.21.15.117:0/778454413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef04109df0 0x7fef0419fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:08.791 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.778+0000 7fef09331640 1 --2- 172.21.15.117:0/778454413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef041094a0 0x7fef0419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:56:08.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.778+0000 7fef09b32640 1 --2- 172.21.15.117:0/778454413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef04103b60 0x7fef0419a4d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7feef4005e30 tx=0x7feef402f3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:08.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.778+0000 7fef02ffd640 1 -- 172.21.15.117:0/778454413 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef4003bf0 con 0x7fef04103b60 2024-09-21T05:56:08.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.779+0000 7fef0bd96640 1 -- 172.21.15.117:0/778454413 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef041a06d0 con 0x7fef04103b60 2024-09-21T05:56:08.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.779+0000 7fef02ffd640 1 -- 172.21.15.117:0/778454413 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feef4003d90 con 0x7fef04103b60 2024-09-21T05:56:08.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.779+0000 7fef02ffd640 1 -- 172.21.15.117:0/778454413 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef402fb10 con 0x7fef04103b60 2024-09-21T05:56:08.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.779+0000 7fef0bd96640 1 -- 172.21.15.117:0/778454413 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef041a6e90 con 0x7fef04103b60 2024-09-21T05:56:08.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.780+0000 7fef0bd96640 1 -- 172.21.15.117:0/778454413 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef04103f70 con 0x7fef04103b60 2024-09-21T05:56:08.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.781+0000 7fef02ffd640 1 -- 172.21.15.117:0/778454413 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7feef4004680 con 0x7fef04103b60 2024-09-21T05:56:08.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.782+0000 7fef02ffd640 1 --2- 172.21.15.117:0/778454413 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7feeec0b65f0 0x7feeec0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:08.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.782+0000 7fef09331640 1 --2- 172.21.15.117:0/778454413 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7feeec0b65f0 0x7feeec0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:08.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.782+0000 7fef02ffd640 1 -- 172.21.15.117:0/778454413 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7feef41089c0 con 0x7fef04103b60 2024-09-21T05:56:08.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.782+0000 7fef09331640 1 --2- 172.21.15.117:0/778454413 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7feeec0b65f0 0x7feeec0b8b00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fef04109170 tx=0x7feef8005ec0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:08.792 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.786+0000 7fef02ffd640 1 -- 172.21.15.117:0/778454413 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7feef40cf050 con 0x7fef04103b60 2024-09-21T05:56:08.805 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:08.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.974+0000 7fef0bd96640 1 -- 172.21.15.117:0/778454413 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fef0419b600 con 0x7fef04103b60 2024-09-21T05:56:08.979 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.974+0000 7fef02ffd640 1 -- 172.21.15.117:0/778454413 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7feef40d3430 con 0x7fef04103b60 2024-09-21T05:56:08.979 INFO:teuthology.orchestra.run.smithi117.stdout:176093659186 2024-09-21T05:56:08.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.976+0000 7fef0bd96640 1 -- 172.21.15.117:0/778454413 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7feeec0b65f0 msgr2=0x7feeec0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:08.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.976+0000 7fef0bd96640 1 --2- 172.21.15.117:0/778454413 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7feeec0b65f0 0x7feeec0b8b00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fef04109170 tx=0x7feef8005ec0 comp rx=0 tx=0).stop 2024-09-21T05:56:08.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.976+0000 7fef0bd96640 1 -- 172.21.15.117:0/778454413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef04103b60 msgr2=0x7fef0419a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:08.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.976+0000 7fef0bd96640 1 --2- 172.21.15.117:0/778454413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef04103b60 0x7fef0419a4d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7feef4005e30 tx=0x7feef402f3e0 comp rx=0 tx=0).stop 2024-09-21T05:56:08.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.977+0000 7fef0bd96640 1 -- 172.21.15.117:0/778454413 shutdown_connections 2024-09-21T05:56:08.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.977+0000 7fef0bd96640 1 --2- 172.21.15.117:0/778454413 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7feeec0b65f0 0x7feeec0b8b00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:08.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.977+0000 7fef0bd96640 1 --2- 172.21.15.117:0/778454413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef04109df0 0x7fef0419fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:08.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.977+0000 7fef0bd96640 1 --2- 172.21.15.117:0/778454413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef041094a0 0x7fef0419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:08.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.977+0000 7fef0bd96640 1 --2- 172.21.15.117:0/778454413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef04103b60 0x7fef0419a4d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:08.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.977+0000 7fef0bd96640 1 -- 172.21.15.117:0/778454413 >> 172.21.15.117:0/778454413 conn(0x7fef0407b850 msgr2=0x7fef04107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:08.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.977+0000 7fef0bd96640 1 -- 172.21.15.117:0/778454413 shutdown_connections 2024-09-21T05:56:08.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:08.977+0000 7fef0bd96640 1 -- 172.21.15.117:0/778454413 wait complete. 2024-09-21T05:56:08.981 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:08 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/1058090681' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T05:56:09.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:08 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/1058090681' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T05:56:09.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.466+0000 7fb013246640 1 -- 172.21.15.117:0/2558356588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb00c108d70 msgr2=0x7fb00c10f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:09.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.466+0000 7fb013246640 1 --2- 172.21.15.117:0/2558356588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb00c108d70 0x7fb00c10f5a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7faff4009900 tx=0x7faff402a410 comp rx=0 tx=0).stop 2024-09-21T05:56:09.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.466+0000 7fb013246640 1 -- 172.21.15.117:0/2558356588 shutdown_connections 2024-09-21T05:56:09.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.466+0000 7fb013246640 1 --2- 172.21.15.117:0/2558356588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb00c108d70 0x7fb00c10f5a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:09.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.466+0000 7fb013246640 1 --2- 172.21.15.117:0/2558356588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb00c108420 0x7fb00c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:09.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.466+0000 7fb013246640 1 --2- 172.21.15.117:0/2558356588 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb00c102ae0 0x7fb00c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:09.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.466+0000 7fb013246640 1 -- 172.21.15.117:0/2558356588 >> 172.21.15.117:0/2558356588 conn(0x7fb00c079770 msgr2=0x7fb00c079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:09.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.466+0000 7fb013246640 1 -- 172.21.15.117:0/2558356588 shutdown_connections 2024-09-21T05:56:09.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.466+0000 7fb013246640 1 -- 172.21.15.117:0/2558356588 wait complete. 2024-09-21T05:56:09.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.467+0000 7fb013246640 1 Processor -- start 2024-09-21T05:56:09.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.467+0000 7fb013246640 1 -- start start 2024-09-21T05:56:09.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.467+0000 7fb013246640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb00c102ae0 0x7fb00c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:09.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.467+0000 7fb013246640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb00c108420 0x7fb00c19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:09.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.467+0000 7fb013246640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb00c108d70 0x7fb00c19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:09.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.468+0000 7fb013246640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c113b30 con 0x7fb00c108d70 2024-09-21T05:56:09.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.468+0000 7fb013246640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c1139b0 con 0x7fb00c102ae0 2024-09-21T05:56:09.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.468+0000 7fb013246640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c113cb0 con 0x7fb00c108420 2024-09-21T05:56:09.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.468+0000 7fb0117e3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb00c108d70 0x7fb00c19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:09.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.468+0000 7fb0117e3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb00c108d70 0x7fb00c19fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37944/0 (socket says 172.21.15.117:37944) 2024-09-21T05:56:09.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.468+0000 7fb00bfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb00c108420 0x7fb00c19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:09.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.468+0000 7fb0117e3640 1 -- 172.21.15.117:0/879335842 learned_addr learned my addr 172.21.15.117:0/879335842 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:09.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.468+0000 7fb010fe2640 1 --2- 172.21.15.117:0/879335842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb00c102ae0 0x7fb00c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:09.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.468+0000 7fb0117e3640 1 -- 172.21.15.117:0/879335842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb00c108420 msgr2=0x7fb00c19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:09.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.468+0000 7fb0117e3640 1 --2- 172.21.15.117:0/879335842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb00c108420 0x7fb00c19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:09.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.468+0000 7fb0117e3640 1 -- 172.21.15.117:0/879335842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb00c102ae0 msgr2=0x7fb00c19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:09.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.468+0000 7fb0117e3640 1 --2- 172.21.15.117:0/879335842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb00c102ae0 0x7fb00c19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:09.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.468+0000 7fb0117e3640 1 -- 172.21.15.117:0/879335842 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb00c1a0410 con 0x7fb00c108d70 2024-09-21T05:56:09.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.468+0000 7fb0117e3640 1 --2- 172.21.15.117:0/879335842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb00c108d70 0x7fb00c19fce0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7faff4004770 tx=0x7faff4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:09.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.468+0000 7fb009ffb640 1 -- 172.21.15.117:0/879335842 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff403f030 con 0x7fb00c108d70 2024-09-21T05:56:09.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.468+0000 7fb009ffb640 1 -- 172.21.15.117:0/879335842 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faff4003aa0 con 0x7fb00c108d70 2024-09-21T05:56:09.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.468+0000 7fb009ffb640 1 -- 172.21.15.117:0/879335842 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff4035070 con 0x7fb00c108d70 2024-09-21T05:56:09.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.468+0000 7fb013246640 1 -- 172.21.15.117:0/879335842 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb00c1a06a0 con 0x7fb00c108d70 2024-09-21T05:56:09.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.469+0000 7fb013246640 1 -- 172.21.15.117:0/879335842 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb00c1a6e60 con 0x7fb00c108d70 2024-09-21T05:56:09.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.470+0000 7fb013246640 1 -- 172.21.15.117:0/879335842 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb00c103d60 con 0x7fb00c108d70 2024-09-21T05:56:09.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.472+0000 7fb009ffb640 1 -- 172.21.15.117:0/879335842 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7faff4042030 con 0x7fb00c108d70 2024-09-21T05:56:09.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.473+0000 7fb009ffb640 1 --2- 172.21.15.117:0/879335842 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7faff80b65f0 0x7faff80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:09.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.473+0000 7fb010fe2640 1 --2- 172.21.15.117:0/879335842 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7faff80b65f0 0x7faff80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:09.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.473+0000 7fb009ffb640 1 -- 172.21.15.117:0/879335842 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7faff4107fe0 con 0x7fb00c108d70 2024-09-21T05:56:09.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.473+0000 7fb010fe2640 1 --2- 172.21.15.117:0/879335842 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7faff80b65f0 0x7faff80b8b00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb004005fd0 tx=0x7fb004005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:09.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.476+0000 7fb009ffb640 1 -- 172.21.15.117:0/879335842 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7faff40ce670 con 0x7fb00c108d70 2024-09-21T05:56:09.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.663+0000 7fb013246640 1 -- 172.21.15.117:0/879335842 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fb00c102ef0 con 0x7fb00c108d70 2024-09-21T05:56:09.666 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:09 smithi117 ceph-mon[30947]: pgmap v292: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T05:56:09.666 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:09 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/778454413' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T05:56:09.666 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:09 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:56:09.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.663+0000 7fb009ffb640 1 -- 172.21.15.117:0/879335842 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7faff40d2a50 con 0x7fb00c108d70 2024-09-21T05:56:09.667 INFO:teuthology.orchestra.run.smithi117.stdout:81604378695 2024-09-21T05:56:09.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.665+0000 7fb013246640 1 -- 172.21.15.117:0/879335842 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7faff80b65f0 msgr2=0x7faff80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:09.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.665+0000 7fb013246640 1 --2- 172.21.15.117:0/879335842 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7faff80b65f0 0x7faff80b8b00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb004005fd0 tx=0x7fb004005850 comp rx=0 tx=0).stop 2024-09-21T05:56:09.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.665+0000 7fb013246640 1 -- 172.21.15.117:0/879335842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb00c108d70 msgr2=0x7fb00c19fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:09.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.665+0000 7fb013246640 1 --2- 172.21.15.117:0/879335842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb00c108d70 0x7fb00c19fce0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7faff4004770 tx=0x7faff4004270 comp rx=0 tx=0).stop 2024-09-21T05:56:09.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.666+0000 7fb013246640 1 -- 172.21.15.117:0/879335842 shutdown_connections 2024-09-21T05:56:09.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.666+0000 7fb013246640 1 --2- 172.21.15.117:0/879335842 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7faff80b65f0 0x7faff80b8b00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:09.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.666+0000 7fb013246640 1 --2- 172.21.15.117:0/879335842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb00c108d70 0x7fb00c19fce0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:09.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.666+0000 7fb013246640 1 --2- 172.21.15.117:0/879335842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb00c108420 0x7fb00c19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:09.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.666+0000 7fb013246640 1 --2- 172.21.15.117:0/879335842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb00c102ae0 0x7fb00c19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:09.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.666+0000 7fb013246640 1 -- 172.21.15.117:0/879335842 >> 172.21.15.117:0/879335842 conn(0x7fb00c079770 msgr2=0x7fb00c106410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:09.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.666+0000 7fb013246640 1 -- 172.21.15.117:0/879335842 shutdown_connections 2024-09-21T05:56:09.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:09.666+0000 7fb013246640 1 -- 172.21.15.117:0/879335842 wait complete. 2024-09-21T05:56:09.877 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:09 smithi138 ceph-mon[34391]: pgmap v292: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T05:56:09.877 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:09 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/778454413' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T05:56:09.877 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:09 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:56:09.969 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495657 got 197568495660 for osd.7 2024-09-21T05:56:09.969 DEBUG:teuthology.parallel:result is None 2024-09-21T05:56:09.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:09 smithi192 ceph-mon[34700]: pgmap v292: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T05:56:09.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:09 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/778454413' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T05:56:09.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:09 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:56:10.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.111+0000 7f7c2c02b640 1 -- 172.21.15.117:0/1330143991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c241083c0 msgr2=0x7f7c241087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:10.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.111+0000 7f7c2c02b640 1 --2- 172.21.15.117:0/1330143991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c241083c0 0x7f7c241087d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7c18009900 tx=0x7f7c1802a410 comp rx=0 tx=0).stop 2024-09-21T05:56:10.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.112+0000 7f7c2c02b640 1 -- 172.21.15.117:0/1330143991 shutdown_connections 2024-09-21T05:56:10.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.112+0000 7f7c2c02b640 1 --2- 172.21.15.117:0/1330143991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c241083c0 0x7f7c241087d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:10.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.112+0000 7f7c2c02b640 1 --2- 172.21.15.117:0/1330143991 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c24108d10 0x7f7c2410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:10.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.112+0000 7f7c2c02b640 1 --2- 172.21.15.117:0/1330143991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c24102a80 0x7f7c24102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:10.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.112+0000 7f7c2c02b640 1 -- 172.21.15.117:0/1330143991 >> 172.21.15.117:0/1330143991 conn(0x7f7c24079740 msgr2=0x7f7c24079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:10.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.112+0000 7f7c2c02b640 1 -- 172.21.15.117:0/1330143991 shutdown_connections 2024-09-21T05:56:10.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.112+0000 7f7c2c02b640 1 -- 172.21.15.117:0/1330143991 wait complete. 2024-09-21T05:56:10.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.112+0000 7f7c2c02b640 1 Processor -- start 2024-09-21T05:56:10.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.112+0000 7f7c2c02b640 1 -- start start 2024-09-21T05:56:10.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.113+0000 7f7c2c02b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c24102a80 0x7f7c2419b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:10.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.113+0000 7f7c2c02b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c241083c0 0x7f7c2419b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:10.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.113+0000 7f7c2c02b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c24108d10 0x7f7c2419bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:10.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.113+0000 7f7c2c02b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7c24112980 con 0x7f7c241083c0 2024-09-21T05:56:10.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.113+0000 7f7c2c02b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7c24112800 con 0x7f7c24108d10 2024-09-21T05:56:10.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.113+0000 7f7c2c02b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7c24112b00 con 0x7f7c24102a80 2024-09-21T05:56:10.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.113+0000 7f7c295c6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c241083c0 0x7f7c2419b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:10.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.113+0000 7f7c295c6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c241083c0 0x7f7c2419b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37962/0 (socket says 172.21.15.117:37962) 2024-09-21T05:56:10.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.113+0000 7f7c295c6640 1 -- 172.21.15.117:0/3331310957 learned_addr learned my addr 172.21.15.117:0/3331310957 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:10.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.113+0000 7f7c29dc7640 1 --2- 172.21.15.117:0/3331310957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c24102a80 0x7f7c2419b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:10.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.113+0000 7f7c2a5c8640 1 --2- 172.21.15.117:0/3331310957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c24108d10 0x7f7c2419bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:10.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.113+0000 7f7c295c6640 1 -- 172.21.15.117:0/3331310957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c24102a80 msgr2=0x7f7c2419b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:10.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.113+0000 7f7c295c6640 1 --2- 172.21.15.117:0/3331310957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c24102a80 0x7f7c2419b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:10.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.113+0000 7f7c295c6640 1 -- 172.21.15.117:0/3331310957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c24108d10 msgr2=0x7f7c2419bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:10.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.113+0000 7f7c295c6640 1 --2- 172.21.15.117:0/3331310957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c24108d10 0x7f7c2419bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:10.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.113+0000 7f7c295c6640 1 -- 172.21.15.117:0/3331310957 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c241a65e0 con 0x7f7c241083c0 2024-09-21T05:56:10.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.113+0000 7f7c295c6640 1 --2- 172.21.15.117:0/3331310957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c241083c0 0x7f7c2419b890 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7c18006010 tx=0x7f7c18004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:10.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.113+0000 7f7c22ffd640 1 -- 172.21.15.117:0/3331310957 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1803f030 con 0x7f7c241083c0 2024-09-21T05:56:10.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.113+0000 7f7c22ffd640 1 -- 172.21.15.117:0/3331310957 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c18003aa0 con 0x7f7c241083c0 2024-09-21T05:56:10.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.113+0000 7f7c22ffd640 1 -- 172.21.15.117:0/3331310957 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c18035070 con 0x7f7c241083c0 2024-09-21T05:56:10.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.113+0000 7f7c2c02b640 1 -- 172.21.15.117:0/3331310957 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c241a6870 con 0x7f7c241083c0 2024-09-21T05:56:10.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.114+0000 7f7c2c02b640 1 -- 172.21.15.117:0/3331310957 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c241a6d50 con 0x7f7c241083c0 2024-09-21T05:56:10.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.114+0000 7f7c2c02b640 1 -- 172.21.15.117:0/3331310957 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c24103d00 con 0x7f7c241083c0 2024-09-21T05:56:10.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.116+0000 7f7c22ffd640 1 -- 172.21.15.117:0/3331310957 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f7c18042030 con 0x7f7c241083c0 2024-09-21T05:56:10.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.117+0000 7f7c22ffd640 1 --2- 172.21.15.117:0/3331310957 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7c0c0b6620 0x7f7c0c0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:10.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.117+0000 7f7c29dc7640 1 --2- 172.21.15.117:0/3331310957 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7c0c0b6620 0x7f7c0c0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:10.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.117+0000 7f7c22ffd640 1 -- 172.21.15.117:0/3331310957 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f7c18107d80 con 0x7f7c241083c0 2024-09-21T05:56:10.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.117+0000 7f7c29dc7640 1 --2- 172.21.15.117:0/3331310957 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7c0c0b6620 0x7f7c0c0b8b30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7c2406c6a0 tx=0x7f7c14009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:10.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.120+0000 7f7c22ffd640 1 -- 172.21.15.117:0/3331310957 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7c180d27f0 con 0x7f7c241083c0 2024-09-21T05:56:10.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.308+0000 7f7c2c02b640 1 -- 172.21.15.117:0/3331310957 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f7c24102e90 con 0x7f7c241083c0 2024-09-21T05:56:10.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.308+0000 7f7c22ffd640 1 -- 172.21.15.117:0/3331310957 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7c180d7730 con 0x7f7c241083c0 2024-09-21T05:56:10.311 INFO:teuthology.orchestra.run.smithi117.stdout:223338299432 2024-09-21T05:56:10.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.311+0000 7f7c2c02b640 1 -- 172.21.15.117:0/3331310957 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7c0c0b6620 msgr2=0x7f7c0c0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:10.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.311+0000 7f7c2c02b640 1 --2- 172.21.15.117:0/3331310957 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7c0c0b6620 0x7f7c0c0b8b30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7c2406c6a0 tx=0x7f7c14009500 comp rx=0 tx=0).stop 2024-09-21T05:56:10.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.311+0000 7f7c2c02b640 1 -- 172.21.15.117:0/3331310957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c241083c0 msgr2=0x7f7c2419b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:10.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.311+0000 7f7c2c02b640 1 --2- 172.21.15.117:0/3331310957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c241083c0 0x7f7c2419b890 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7c18006010 tx=0x7f7c18004060 comp rx=0 tx=0).stop 2024-09-21T05:56:10.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.311+0000 7f7c2c02b640 1 -- 172.21.15.117:0/3331310957 shutdown_connections 2024-09-21T05:56:10.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.311+0000 7f7c2c02b640 1 --2- 172.21.15.117:0/3331310957 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7c0c0b6620 0x7f7c0c0b8b30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:10.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.311+0000 7f7c2c02b640 1 --2- 172.21.15.117:0/3331310957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c241083c0 0x7f7c2419b890 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:10.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.311+0000 7f7c2c02b640 1 --2- 172.21.15.117:0/3331310957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c24108d10 0x7f7c2419bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:10.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.311+0000 7f7c2c02b640 1 --2- 172.21.15.117:0/3331310957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c24102a80 0x7f7c2419b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:10.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.311+0000 7f7c2c02b640 1 -- 172.21.15.117:0/3331310957 >> 172.21.15.117:0/3331310957 conn(0x7f7c24079740 msgr2=0x7f7c241044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:10.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.311+0000 7f7c2c02b640 1 -- 172.21.15.117:0/3331310957 shutdown_connections 2024-09-21T05:56:10.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:10.311+0000 7f7c2c02b640 1 -- 172.21.15.117:0/3331310957 wait complete. 2024-09-21T05:56:10.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:10 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/879335842' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T05:56:10.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:10 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3331310957' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T05:56:10.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:10 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:56:10.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:10 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:56:10.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:10 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:56:10.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:10 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:56:10.992 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:10 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/879335842' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T05:56:10.992 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:10 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3331310957' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T05:56:10.993 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:10 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:56:10.993 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:10 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:56:10.993 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:10 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:56:10.993 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:10 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:56:11.019 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299430 got 223338299432 for osd.8 2024-09-21T05:56:11.019 DEBUG:teuthology.parallel:result is None 2024-09-21T05:56:11.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:10 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/879335842' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T05:56:11.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:10 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3331310957' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T05:56:11.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:10 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:56:11.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:10 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:56:11.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:10 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:56:11.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:10 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:56:11.557 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378692 got 81604378695 for osd.2 2024-09-21T05:56:11.558 DEBUG:teuthology.parallel:result is None 2024-09-21T05:56:11.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.736+0000 7f8ab7a34640 1 -- 172.21.15.117:0/1000462364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ab0109e10 msgr2=0x7f8ab0110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:11.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.736+0000 7f8ab7a34640 1 --2- 172.21.15.117:0/1000462364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ab0109e10 0x7f8ab0110640 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8009ca0 tx=0x7f8aa802a8c0 comp rx=0 tx=0).stop 2024-09-21T05:56:11.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.736+0000 7f8ab7a34640 1 -- 172.21.15.117:0/1000462364 shutdown_connections 2024-09-21T05:56:11.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.736+0000 7f8ab7a34640 1 --2- 172.21.15.117:0/1000462364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ab0109e10 0x7f8ab0110640 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:11.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.736+0000 7f8ab7a34640 1 --2- 172.21.15.117:0/1000462364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ab01094c0 0x7f8ab01098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:11.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.736+0000 7f8ab7a34640 1 --2- 172.21.15.117:0/1000462364 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab0103b80 0x7f8ab0103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:11.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.736+0000 7f8ab7a34640 1 -- 172.21.15.117:0/1000462364 >> 172.21.15.117:0/1000462364 conn(0x7f8ab00ff610 msgr2=0x7f8ab0101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:11.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.736+0000 7f8ab7a34640 1 -- 172.21.15.117:0/1000462364 shutdown_connections 2024-09-21T05:56:11.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.736+0000 7f8ab7a34640 1 -- 172.21.15.117:0/1000462364 wait complete. 2024-09-21T05:56:11.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.737+0000 7f8ab7a34640 1 Processor -- start 2024-09-21T05:56:11.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.737+0000 7f8ab7a34640 1 -- start start 2024-09-21T05:56:11.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.737+0000 7f8ab7a34640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ab0103b80 0x7f8ab019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:11.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.737+0000 7f8ab7a34640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ab01094c0 0x7f8ab019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:11.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.737+0000 7f8ab7a34640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab0109e10 0x7f8ab019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:11.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.737+0000 7f8ab7a34640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab0113ad0 con 0x7f8ab0103b80 2024-09-21T05:56:11.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.737+0000 7f8ab7a34640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab0113950 con 0x7f8ab01094c0 2024-09-21T05:56:11.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.737+0000 7f8ab7a34640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab0113c50 con 0x7f8ab0109e10 2024-09-21T05:56:11.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.737+0000 7f8ab57d0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ab0103b80 0x7f8ab019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:11.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.737+0000 7f8ab57d0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ab0103b80 0x7f8ab019a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37976/0 (socket says 172.21.15.117:37976) 2024-09-21T05:56:11.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.737+0000 7f8ab57d0640 1 -- 172.21.15.117:0/2300718257 learned_addr learned my addr 172.21.15.117:0/2300718257 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:11.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.737+0000 7f8ab5fd1640 1 --2- 172.21.15.117:0/2300718257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab0109e10 0x7f8ab019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:11.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.737+0000 7f8ab4fcf640 1 --2- 172.21.15.117:0/2300718257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ab01094c0 0x7f8ab019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:11.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.737+0000 7f8ab57d0640 1 -- 172.21.15.117:0/2300718257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab0109e10 msgr2=0x7f8ab019fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:11.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.737+0000 7f8ab57d0640 1 --2- 172.21.15.117:0/2300718257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab0109e10 0x7f8ab019fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:11.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.737+0000 7f8ab57d0640 1 -- 172.21.15.117:0/2300718257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ab01094c0 msgr2=0x7f8ab019a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:11.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.737+0000 7f8ab57d0640 1 --2- 172.21.15.117:0/2300718257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ab01094c0 0x7f8ab019a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:11.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.737+0000 7f8ab57d0640 1 -- 172.21.15.117:0/2300718257 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab01a02d0 con 0x7f8ab0103b80 2024-09-21T05:56:11.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.738+0000 7f8ab57d0640 1 --2- 172.21.15.117:0/2300718257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ab0103b80 0x7f8ab019a420 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00c920 tx=0x7f8aac00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:11.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.738+0000 7f8aa67fc640 1 -- 172.21.15.117:0/2300718257 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac007c70 con 0x7f8ab0103b80 2024-09-21T05:56:11.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.738+0000 7f8aa67fc640 1 -- 172.21.15.117:0/2300718257 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8aac007e10 con 0x7f8ab0103b80 2024-09-21T05:56:11.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.738+0000 7f8ab7a34640 1 -- 172.21.15.117:0/2300718257 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ab01a05c0 con 0x7f8ab0103b80 2024-09-21T05:56:11.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.738+0000 7f8aa67fc640 1 -- 172.21.15.117:0/2300718257 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac018820 con 0x7f8ab0103b80 2024-09-21T05:56:11.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.738+0000 7f8ab7a34640 1 -- 172.21.15.117:0/2300718257 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ab01a6de0 con 0x7f8ab0103b80 2024-09-21T05:56:11.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.739+0000 7f8ab7a34640 1 -- 172.21.15.117:0/2300718257 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ab0103f90 con 0x7f8ab0103b80 2024-09-21T05:56:11.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.740+0000 7f8aa67fc640 1 -- 172.21.15.117:0/2300718257 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f8aac00f450 con 0x7f8ab0103b80 2024-09-21T05:56:11.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.742+0000 7f8aa67fc640 1 --2- 172.21.15.117:0/2300718257 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8a9c0b66a0 0x7f8a9c0b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:11.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.742+0000 7f8ab4fcf640 1 --2- 172.21.15.117:0/2300718257 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8a9c0b66a0 0x7f8a9c0b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:11.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.742+0000 7f8aa67fc640 1 -- 172.21.15.117:0/2300718257 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f8aac0e79a0 con 0x7f8ab0103b80 2024-09-21T05:56:11.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.742+0000 7f8ab4fcf640 1 --2- 172.21.15.117:0/2300718257 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8a9c0b66a0 0x7f8a9c0b8bb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0005fd0 tx=0x7f8aa0009470 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:11.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.745+0000 7f8aa67fc640 1 -- 172.21.15.117:0/2300718257 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8aac014030 con 0x7f8ab0103b80 2024-09-21T05:56:11.890 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:11 smithi117 ceph-mon[30947]: pgmap v293: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T05:56:11.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.931+0000 7f8ab7a34640 1 -- 172.21.15.117:0/2300718257 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f8ab019b550 con 0x7f8ab0103b80 2024-09-21T05:56:11.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.931+0000 7f8aa67fc640 1 -- 172.21.15.117:0/2300718257 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8aac0ae140 con 0x7f8ab0103b80 2024-09-21T05:56:11.933 INFO:teuthology.orchestra.run.smithi117.stdout:60129542218 2024-09-21T05:56:11.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.933+0000 7f8ab7a34640 1 -- 172.21.15.117:0/2300718257 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8a9c0b66a0 msgr2=0x7f8a9c0b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:11.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.933+0000 7f8ab7a34640 1 --2- 172.21.15.117:0/2300718257 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8a9c0b66a0 0x7f8a9c0b8bb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0005fd0 tx=0x7f8aa0009470 comp rx=0 tx=0).stop 2024-09-21T05:56:11.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.933+0000 7f8ab7a34640 1 -- 172.21.15.117:0/2300718257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ab0103b80 msgr2=0x7f8ab019a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:11.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.933+0000 7f8ab7a34640 1 --2- 172.21.15.117:0/2300718257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ab0103b80 0x7f8ab019a420 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00c920 tx=0x7f8aac00cde0 comp rx=0 tx=0).stop 2024-09-21T05:56:11.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.933+0000 7f8ab7a34640 1 -- 172.21.15.117:0/2300718257 shutdown_connections 2024-09-21T05:56:11.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.933+0000 7f8ab7a34640 1 --2- 172.21.15.117:0/2300718257 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8a9c0b66a0 0x7f8a9c0b8bb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:11.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.933+0000 7f8ab7a34640 1 --2- 172.21.15.117:0/2300718257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab0109e10 0x7f8ab019fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:11.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.933+0000 7f8ab7a34640 1 --2- 172.21.15.117:0/2300718257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ab01094c0 0x7f8ab019a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:11.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.933+0000 7f8ab7a34640 1 --2- 172.21.15.117:0/2300718257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ab0103b80 0x7f8ab019a420 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:11.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.933+0000 7f8ab7a34640 1 -- 172.21.15.117:0/2300718257 >> 172.21.15.117:0/2300718257 conn(0x7f8ab00ff610 msgr2=0x7f8ab0106a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:11.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.934+0000 7f8ab7a34640 1 -- 172.21.15.117:0/2300718257 shutdown_connections 2024-09-21T05:56:11.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.934+0000 7f8ab7a34640 1 -- 172.21.15.117:0/2300718257 wait complete. 2024-09-21T05:56:11.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:11 smithi192 ceph-mon[34700]: pgmap v293: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T05:56:11.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.983+0000 7f7011537640 1 -- 172.21.15.117:0/3693457566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f700c1083b0 msgr2=0x7f700c1087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:11.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.983+0000 7f7011537640 1 --2- 172.21.15.117:0/3693457566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f700c1083b0 0x7f700c1087c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4009ca0 tx=0x7f6ff402a8c0 comp rx=0 tx=0).stop 2024-09-21T05:56:11.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.983+0000 7f7011537640 1 -- 172.21.15.117:0/3693457566 shutdown_connections 2024-09-21T05:56:11.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.983+0000 7f7011537640 1 --2- 172.21.15.117:0/3693457566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f700c1083b0 0x7f700c1087c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:11.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.983+0000 7f7011537640 1 --2- 172.21.15.117:0/3693457566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f700c108d00 0x7f700c10f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:11.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.983+0000 7f7011537640 1 --2- 172.21.15.117:0/3693457566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f700c102ac0 0x7f700c102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:11.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.983+0000 7f7011537640 1 -- 172.21.15.117:0/3693457566 >> 172.21.15.117:0/3693457566 conn(0x7f700c079790 msgr2=0x7f700c079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:11.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.983+0000 7f7011537640 1 -- 172.21.15.117:0/3693457566 shutdown_connections 2024-09-21T05:56:11.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.983+0000 7f7011537640 1 -- 172.21.15.117:0/3693457566 wait complete. 2024-09-21T05:56:11.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.983+0000 7f7011537640 1 Processor -- start 2024-09-21T05:56:11.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.984+0000 7f7011537640 1 -- start start 2024-09-21T05:56:11.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.984+0000 7f7011537640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f700c102ac0 0x7f700c19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:11.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.984+0000 7f7011537640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f700c1083b0 0x7f700c19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:11.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.984+0000 7f7011537640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f700c108d00 0x7f700c19fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:11.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.984+0000 7f7011537640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f700c113b00 con 0x7f700c102ac0 2024-09-21T05:56:11.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.984+0000 7f7011537640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f700c113980 con 0x7f700c1083b0 2024-09-21T05:56:11.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.984+0000 7f7011537640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f700c113c80 con 0x7f700c108d00 2024-09-21T05:56:11.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.984+0000 7f700affd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f700c102ac0 0x7f700c19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:11.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.984+0000 7f700affd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f700c102ac0 0x7f700c19a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37982/0 (socket says 172.21.15.117:37982) 2024-09-21T05:56:11.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.984+0000 7f700affd640 1 -- 172.21.15.117:0/1308294928 learned_addr learned my addr 172.21.15.117:0/1308294928 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:11.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.984+0000 7f700a7fc640 1 --2- 172.21.15.117:0/1308294928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f700c1083b0 0x7f700c19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:11.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.984+0000 7f700b7fe640 1 --2- 172.21.15.117:0/1308294928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f700c108d00 0x7f700c19fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:11.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.984+0000 7f700affd640 1 -- 172.21.15.117:0/1308294928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f700c108d00 msgr2=0x7f700c19fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:11.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.984+0000 7f700affd640 1 --2- 172.21.15.117:0/1308294928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f700c108d00 0x7f700c19fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:11.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.984+0000 7f700affd640 1 -- 172.21.15.117:0/1308294928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f700c1083b0 msgr2=0x7f700c19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:11.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.984+0000 7f700affd640 1 --2- 172.21.15.117:0/1308294928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f700c1083b0 0x7f700c19aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:11.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.984+0000 7f700affd640 1 -- 172.21.15.117:0/1308294928 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f700c1a04c0 con 0x7f700c102ac0 2024-09-21T05:56:11.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.985+0000 7f700affd640 1 --2- 172.21.15.117:0/1308294928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f700c102ac0 0x7f700c19a550 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f700400b4a0 tx=0x7f700400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:11.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.985+0000 7f7003fff640 1 -- 172.21.15.117:0/1308294928 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7004014070 con 0x7f700c102ac0 2024-09-21T05:56:11.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.985+0000 7f7003fff640 1 -- 172.21.15.117:0/1308294928 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7004010ac0 con 0x7f700c102ac0 2024-09-21T05:56:11.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.985+0000 7f7011537640 1 -- 172.21.15.117:0/1308294928 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f700c1a07b0 con 0x7f700c102ac0 2024-09-21T05:56:11.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.985+0000 7f7003fff640 1 -- 172.21.15.117:0/1308294928 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7004017c30 con 0x7f700c102ac0 2024-09-21T05:56:11.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.985+0000 7f7011537640 1 -- 172.21.15.117:0/1308294928 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f700c1a6f10 con 0x7f700c102ac0 2024-09-21T05:56:11.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.986+0000 7f7011537640 1 -- 172.21.15.117:0/1308294928 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f700c102ed0 con 0x7f700c102ac0 2024-09-21T05:56:11.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.988+0000 7f7003fff640 1 -- 172.21.15.117:0/1308294928 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f7004017070 con 0x7f700c102ac0 2024-09-21T05:56:11.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.989+0000 7f7003fff640 1 --2- 172.21.15.117:0/1308294928 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6ff80b65f0 0x7f6ff80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:11.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.989+0000 7f700a7fc640 1 --2- 172.21.15.117:0/1308294928 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6ff80b65f0 0x7f6ff80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:11.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.989+0000 7f7003fff640 1 -- 172.21.15.117:0/1308294928 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f70040e6b00 con 0x7f700c102ac0 2024-09-21T05:56:11.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.989+0000 7f700a7fc640 1 --2- 172.21.15.117:0/1308294928 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6ff80b65f0 0x7f6ff80b8b00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4000f80 tx=0x7f6ff40093e0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:11.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:11.992+0000 7f7003fff640 1 -- 172.21.15.117:0/1308294928 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f70040ad190 con 0x7f700c102ac0 2024-09-21T05:56:12.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:11 smithi138 ceph-mon[34391]: pgmap v293: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T05:56:12.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:12.175+0000 7f7011537640 1 -- 172.21.15.117:0/1308294928 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f700c19b680 con 0x7f700c102ac0 2024-09-21T05:56:12.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:12.176+0000 7f7003fff640 1 -- 172.21.15.117:0/1308294928 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f70040b1570 con 0x7f700c102ac0 2024-09-21T05:56:12.178 INFO:teuthology.orchestra.run.smithi117.stdout:249108103202 2024-09-21T05:56:12.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:12.178+0000 7f7011537640 1 -- 172.21.15.117:0/1308294928 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6ff80b65f0 msgr2=0x7f6ff80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:12.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:12.178+0000 7f7011537640 1 --2- 172.21.15.117:0/1308294928 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6ff80b65f0 0x7f6ff80b8b00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4000f80 tx=0x7f6ff40093e0 comp rx=0 tx=0).stop 2024-09-21T05:56:12.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:12.178+0000 7f7011537640 1 -- 172.21.15.117:0/1308294928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f700c102ac0 msgr2=0x7f700c19a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:12.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:12.178+0000 7f7011537640 1 --2- 172.21.15.117:0/1308294928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f700c102ac0 0x7f700c19a550 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f700400b4a0 tx=0x7f700400b960 comp rx=0 tx=0).stop 2024-09-21T05:56:12.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:12.178+0000 7f7011537640 1 -- 172.21.15.117:0/1308294928 shutdown_connections 2024-09-21T05:56:12.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:12.178+0000 7f7011537640 1 --2- 172.21.15.117:0/1308294928 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6ff80b65f0 0x7f6ff80b8b00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:12.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:12.178+0000 7f7011537640 1 --2- 172.21.15.117:0/1308294928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f700c108d00 0x7f700c19fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:12.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:12.178+0000 7f7011537640 1 --2- 172.21.15.117:0/1308294928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f700c1083b0 0x7f700c19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:12.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:12.178+0000 7f7011537640 1 --2- 172.21.15.117:0/1308294928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f700c102ac0 0x7f700c19a550 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:12.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:12.178+0000 7f7011537640 1 -- 172.21.15.117:0/1308294928 >> 172.21.15.117:0/1308294928 conn(0x7f700c079790 msgr2=0x7f700c106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:12.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:12.178+0000 7f7011537640 1 -- 172.21.15.117:0/1308294928 shutdown_connections 2024-09-21T05:56:12.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:12.178+0000 7f7011537640 1 -- 172.21.15.117:0/1308294928 wait complete. 2024-09-21T05:56:12.618 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:12.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:12 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2300718257' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T05:56:12.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:12 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/1308294928' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T05:56:13.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:12 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2300718257' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T05:56:13.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:12 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/1308294928' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T05:56:13.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:12 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2300718257' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T05:56:13.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:12 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/1308294928' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T05:56:13.289 INFO:tasks.cephadm.ceph_manager.ceph:need seq 176093659183 got 176093659186 for osd.6 2024-09-21T05:56:13.289 DEBUG:teuthology.parallel:result is None 2024-09-21T05:56:13.923 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:13 smithi117 ceph-mon[30947]: pgmap v294: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T05:56:13.951 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103201 got 249108103202 for osd.9 2024-09-21T05:56:13.951 DEBUG:teuthology.parallel:result is None 2024-09-21T05:56:13.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:13 smithi192 ceph-mon[34700]: pgmap v294: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T05:56:14.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:13 smithi138 ceph-mon[34391]: pgmap v294: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T05:56:14.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.315+0000 7f4a0d309640 1 -- 172.21.15.117:0/3162297055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a081094c0 msgr2=0x7f4a081098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:14.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.315+0000 7f4a0d309640 1 --2- 172.21.15.117:0/3162297055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a081094c0 0x7f4a081098d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f49f0009900 tx=0x7f49f002a410 comp rx=0 tx=0).stop 2024-09-21T05:56:14.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.315+0000 7f4a0d309640 1 -- 172.21.15.117:0/3162297055 shutdown_connections 2024-09-21T05:56:14.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.315+0000 7f4a0d309640 1 --2- 172.21.15.117:0/3162297055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a081094c0 0x7f4a081098d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:14.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.315+0000 7f4a0d309640 1 --2- 172.21.15.117:0/3162297055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a08109e10 0x7f4a08110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:14.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.315+0000 7f4a0d309640 1 --2- 172.21.15.117:0/3162297055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a08103b80 0x7f4a08103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:14.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.315+0000 7f4a0d309640 1 -- 172.21.15.117:0/3162297055 >> 172.21.15.117:0/3162297055 conn(0x7f4a080ff5b0 msgr2=0x7f4a08101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:14.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.315+0000 7f4a0d309640 1 -- 172.21.15.117:0/3162297055 shutdown_connections 2024-09-21T05:56:14.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.315+0000 7f4a0d309640 1 -- 172.21.15.117:0/3162297055 wait complete. 2024-09-21T05:56:14.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.316+0000 7f4a0d309640 1 Processor -- start 2024-09-21T05:56:14.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.316+0000 7f4a0d309640 1 -- start start 2024-09-21T05:56:14.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.316+0000 7f4a0d309640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a08103b80 0x7f4a0819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:14.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.316+0000 7f4a0d309640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a081094c0 0x7f4a0819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:14.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.316+0000 7f4a0d309640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a08109e10 0x7f4a0819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:14.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.316+0000 7f4a0d309640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4a08113ad0 con 0x7f4a081094c0 2024-09-21T05:56:14.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.316+0000 7f4a0d309640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4a08113950 con 0x7f4a08103b80 2024-09-21T05:56:14.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.316+0000 7f4a0d309640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4a08113c50 con 0x7f4a08109e10 2024-09-21T05:56:14.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.316+0000 7f4a067fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a081094c0 0x7f4a0819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:14.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.316+0000 7f4a067fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a081094c0 0x7f4a0819aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37990/0 (socket says 172.21.15.117:37990) 2024-09-21T05:56:14.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.316+0000 7f4a067fc640 1 -- 172.21.15.117:0/1831510945 learned_addr learned my addr 172.21.15.117:0/1831510945 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:14.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.316+0000 7f4a077fe640 1 --2- 172.21.15.117:0/1831510945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a08109e10 0x7f4a0819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:14.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.317+0000 7f4a06ffd640 1 --2- 172.21.15.117:0/1831510945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a08103b80 0x7f4a0819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:14.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.317+0000 7f4a067fc640 1 -- 172.21.15.117:0/1831510945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a08109e10 msgr2=0x7f4a0819fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:14.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.317+0000 7f4a067fc640 1 --2- 172.21.15.117:0/1831510945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a08109e10 0x7f4a0819fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:14.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.317+0000 7f4a067fc640 1 -- 172.21.15.117:0/1831510945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a08103b80 msgr2=0x7f4a0819a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:14.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.317+0000 7f4a067fc640 1 --2- 172.21.15.117:0/1831510945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a08103b80 0x7f4a0819a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:14.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.317+0000 7f4a067fc640 1 -- 172.21.15.117:0/1831510945 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a081a0430 con 0x7f4a081094c0 2024-09-21T05:56:14.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.317+0000 7f4a067fc640 1 --2- 172.21.15.117:0/1831510945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a081094c0 0x7f4a0819aa60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f49f0006010 tx=0x7f49f0004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:14.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.317+0000 7f49fffff640 1 -- 172.21.15.117:0/1831510945 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f003f030 con 0x7f4a081094c0 2024-09-21T05:56:14.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.317+0000 7f49fffff640 1 -- 172.21.15.117:0/1831510945 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49f0003aa0 con 0x7f4a081094c0 2024-09-21T05:56:14.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.317+0000 7f4a0d309640 1 -- 172.21.15.117:0/1831510945 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a081a06c0 con 0x7f4a081094c0 2024-09-21T05:56:14.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.317+0000 7f49fffff640 1 -- 172.21.15.117:0/1831510945 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f0035070 con 0x7f4a081094c0 2024-09-21T05:56:14.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.317+0000 7f4a0d309640 1 -- 172.21.15.117:0/1831510945 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a081a6e80 con 0x7f4a081094c0 2024-09-21T05:56:14.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.318+0000 7f4a0d309640 1 -- 172.21.15.117:0/1831510945 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a08104e00 con 0x7f4a081094c0 2024-09-21T05:56:14.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.321+0000 7f49fffff640 1 -- 172.21.15.117:0/1831510945 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f49f0042030 con 0x7f4a081094c0 2024-09-21T05:56:14.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.321+0000 7f49fffff640 1 --2- 172.21.15.117:0/1831510945 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f49f40b65f0 0x7f49f40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:14.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.322+0000 7f4a06ffd640 1 --2- 172.21.15.117:0/1831510945 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f49f40b65f0 0x7f49f40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:14.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.322+0000 7f49fffff640 1 -- 172.21.15.117:0/1831510945 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f49f0107e30 con 0x7f4a081094c0 2024-09-21T05:56:14.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.322+0000 7f4a06ffd640 1 --2- 172.21.15.117:0/1831510945 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f49f40b65f0 0x7f49f40b8b00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009b60 tx=0x7f4a00005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:14.325 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:14.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.324+0000 7f49fffff640 1 -- 172.21.15.117:0/1831510945 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f49f00ce4c0 con 0x7f4a081094c0 2024-09-21T05:56:14.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.510+0000 7f4a0d309640 1 -- 172.21.15.117:0/1831510945 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f4a0810bbe0 con 0x7f4a081094c0 2024-09-21T05:56:14.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.511+0000 7f49fffff640 1 -- 172.21.15.117:0/1831510945 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f49f00d28a0 con 0x7f4a081094c0 2024-09-21T05:56:14.513 INFO:teuthology.orchestra.run.smithi117.stdout:133143986237 2024-09-21T05:56:14.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.513+0000 7f4a0d309640 1 -- 172.21.15.117:0/1831510945 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f49f40b65f0 msgr2=0x7f49f40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:14.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.513+0000 7f4a0d309640 1 --2- 172.21.15.117:0/1831510945 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f49f40b65f0 0x7f49f40b8b00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009b60 tx=0x7f4a00005f90 comp rx=0 tx=0).stop 2024-09-21T05:56:14.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.513+0000 7f4a0d309640 1 -- 172.21.15.117:0/1831510945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a081094c0 msgr2=0x7f4a0819aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:14.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.513+0000 7f4a0d309640 1 --2- 172.21.15.117:0/1831510945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a081094c0 0x7f4a0819aa60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f49f0006010 tx=0x7f49f0004060 comp rx=0 tx=0).stop 2024-09-21T05:56:14.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.513+0000 7f4a0d309640 1 -- 172.21.15.117:0/1831510945 shutdown_connections 2024-09-21T05:56:14.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.513+0000 7f4a0d309640 1 --2- 172.21.15.117:0/1831510945 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f49f40b65f0 0x7f49f40b8b00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:14.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.513+0000 7f4a0d309640 1 --2- 172.21.15.117:0/1831510945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a081094c0 0x7f4a0819aa60 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:14.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.513+0000 7f4a0d309640 1 --2- 172.21.15.117:0/1831510945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a08109e10 0x7f4a0819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:14.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.513+0000 7f4a0d309640 1 --2- 172.21.15.117:0/1831510945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a08103b80 0x7f4a0819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:14.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.513+0000 7f4a0d309640 1 -- 172.21.15.117:0/1831510945 >> 172.21.15.117:0/1831510945 conn(0x7f4a080ff5b0 msgr2=0x7f4a08106ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:14.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.513+0000 7f4a0d309640 1 -- 172.21.15.117:0/1831510945 shutdown_connections 2024-09-21T05:56:14.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.513+0000 7f4a0d309640 1 -- 172.21.15.117:0/1831510945 wait complete. 2024-09-21T05:56:14.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.586+0000 7f466845a640 1 -- 172.21.15.117:0/1671722309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4660102050 msgr2=0x7f4660102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:14.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.586+0000 7f466845a640 1 --2- 172.21.15.117:0/1671722309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4660102050 0x7f4660102460 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f465c009ca0 tx=0x7f465c02a8c0 comp rx=0 tx=0).stop 2024-09-21T05:56:14.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.586+0000 7f466845a640 1 -- 172.21.15.117:0/1671722309 shutdown_connections 2024-09-21T05:56:14.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.586+0000 7f466845a640 1 --2- 172.21.15.117:0/1671722309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4660106bf0 0x7f4660110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:14.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.586+0000 7f466845a640 1 --2- 172.21.15.117:0/1671722309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46601062a0 0x7f46601066b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:14.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.586+0000 7f466845a640 1 --2- 172.21.15.117:0/1671722309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4660102050 0x7f4660102460 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:14.595 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.586+0000 7f466845a640 1 -- 172.21.15.117:0/1671722309 >> 172.21.15.117:0/1671722309 conn(0x7f4660077c60 msgr2=0x7f46600780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:14.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.586+0000 7f466845a640 1 -- 172.21.15.117:0/1671722309 shutdown_connections 2024-09-21T05:56:14.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.586+0000 7f466845a640 1 -- 172.21.15.117:0/1671722309 wait complete. 2024-09-21T05:56:14.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.586+0000 7f466845a640 1 Processor -- start 2024-09-21T05:56:14.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.586+0000 7f466845a640 1 -- start start 2024-09-21T05:56:14.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.587+0000 7f466845a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4660102050 0x7f466019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:14.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.587+0000 7f466845a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46601062a0 0x7f466019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:14.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.587+0000 7f466845a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4660106bf0 0x7f466019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:14.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.587+0000 7f466845a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4660113aa0 con 0x7f4660106bf0 2024-09-21T05:56:14.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.587+0000 7f466845a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4660113920 con 0x7f46601062a0 2024-09-21T05:56:14.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.587+0000 7f466845a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4660113c20 con 0x7f4660102050 2024-09-21T05:56:14.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.587+0000 7f46669f7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4660106bf0 0x7f466019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:14.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.587+0000 7f46669f7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4660106bf0 0x7f466019fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40078/0 (socket says 172.21.15.117:40078) 2024-09-21T05:56:14.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.587+0000 7f46669f7640 1 -- 172.21.15.117:0/3534229662 learned_addr learned my addr 172.21.15.117:0/3534229662 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:14.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.587+0000 7f46661f6640 1 --2- 172.21.15.117:0/3534229662 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4660102050 0x7f466019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:14.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.587+0000 7f46659f5640 1 --2- 172.21.15.117:0/3534229662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46601062a0 0x7f466019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:14.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.587+0000 7f46669f7640 1 -- 172.21.15.117:0/3534229662 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4660102050 msgr2=0x7f466019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:14.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.587+0000 7f46669f7640 1 --2- 172.21.15.117:0/3534229662 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4660102050 0x7f466019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:14.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.587+0000 7f46669f7640 1 -- 172.21.15.117:0/3534229662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46601062a0 msgr2=0x7f466019aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:14.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.587+0000 7f46669f7640 1 --2- 172.21.15.117:0/3534229662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46601062a0 0x7f466019aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:14.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.587+0000 7f46669f7640 1 -- 172.21.15.117:0/3534229662 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46601a0400 con 0x7f4660106bf0 2024-09-21T05:56:14.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.587+0000 7f46669f7640 1 --2- 172.21.15.117:0/3534229662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4660106bf0 0x7f466019fcd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f465800d750 tx=0x7f465800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:14.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.587+0000 7f46577fe640 1 -- 172.21.15.117:0/3534229662 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4658017070 con 0x7f4660106bf0 2024-09-21T05:56:14.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.588+0000 7f466845a640 1 -- 172.21.15.117:0/3534229662 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46601a06f0 con 0x7f4660106bf0 2024-09-21T05:56:14.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.588+0000 7f46577fe640 1 -- 172.21.15.117:0/3534229662 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46580054d0 con 0x7f4660106bf0 2024-09-21T05:56:14.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.588+0000 7f46577fe640 1 -- 172.21.15.117:0/3534229662 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4658004c60 con 0x7f4660106bf0 2024-09-21T05:56:14.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.588+0000 7f466845a640 1 -- 172.21.15.117:0/3534229662 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46601a6e50 con 0x7f4660106bf0 2024-09-21T05:56:14.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.589+0000 7f466845a640 1 -- 172.21.15.117:0/3534229662 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4660102460 con 0x7f4660106bf0 2024-09-21T05:56:14.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.590+0000 7f46577fe640 1 -- 172.21.15.117:0/3534229662 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f4658009bd0 con 0x7f4660106bf0 2024-09-21T05:56:14.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.591+0000 7f46577fe640 1 --2- 172.21.15.117:0/3534229662 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f464c0b65f0 0x7f464c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:14.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.591+0000 7f46661f6640 1 --2- 172.21.15.117:0/3534229662 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f464c0b65f0 0x7f464c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:14.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.591+0000 7f46577fe640 1 -- 172.21.15.117:0/3534229662 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f46580e73f0 con 0x7f4660106bf0 2024-09-21T05:56:14.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.592+0000 7f46661f6640 1 --2- 172.21.15.117:0/3534229662 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f464c0b65f0 0x7f464c0b8b00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f465c000f80 tx=0x7f465c009360 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:14.598 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.595+0000 7f46577fe640 1 -- 172.21.15.117:0/3534229662 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f46580ada80 con 0x7f4660106bf0 2024-09-21T05:56:14.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:14 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/1831510945' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T05:56:14.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.779+0000 7f466845a640 1 -- 172.21.15.117:0/3534229662 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f466019b5c0 con 0x7f4660106bf0 2024-09-21T05:56:14.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.779+0000 7f46577fe640 1 -- 172.21.15.117:0/3534229662 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f466019b5c0 con 0x7f4660106bf0 2024-09-21T05:56:14.781 INFO:teuthology.orchestra.run.smithi117.stdout:154618822711 2024-09-21T05:56:14.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.781+0000 7f466845a640 1 -- 172.21.15.117:0/3534229662 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f464c0b65f0 msgr2=0x7f464c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:14.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.781+0000 7f466845a640 1 --2- 172.21.15.117:0/3534229662 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f464c0b65f0 0x7f464c0b8b00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f465c000f80 tx=0x7f465c009360 comp rx=0 tx=0).stop 2024-09-21T05:56:14.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.781+0000 7f466845a640 1 -- 172.21.15.117:0/3534229662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4660106bf0 msgr2=0x7f466019fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:14.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.781+0000 7f466845a640 1 --2- 172.21.15.117:0/3534229662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4660106bf0 0x7f466019fcd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f465800d750 tx=0x7f465800dc10 comp rx=0 tx=0).stop 2024-09-21T05:56:14.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.781+0000 7f466845a640 1 -- 172.21.15.117:0/3534229662 shutdown_connections 2024-09-21T05:56:14.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.781+0000 7f466845a640 1 --2- 172.21.15.117:0/3534229662 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f464c0b65f0 0x7f464c0b8b00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:14.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.781+0000 7f466845a640 1 --2- 172.21.15.117:0/3534229662 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4660106bf0 0x7f466019fcd0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:14.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.781+0000 7f466845a640 1 --2- 172.21.15.117:0/3534229662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46601062a0 0x7f466019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:14.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.781+0000 7f466845a640 1 --2- 172.21.15.117:0/3534229662 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4660102050 0x7f466019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:14.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.781+0000 7f466845a640 1 -- 172.21.15.117:0/3534229662 >> 172.21.15.117:0/3534229662 conn(0x7f4660077c60 msgr2=0x7f4660107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:14.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.781+0000 7f466845a640 1 -- 172.21.15.117:0/3534229662 shutdown_connections 2024-09-21T05:56:14.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:14.781+0000 7f466845a640 1 -- 172.21.15.117:0/3534229662 wait complete. 2024-09-21T05:56:14.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:14 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/1831510945' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T05:56:15.016 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:14 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/1831510945' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T05:56:15.491 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822707 got 154618822711 for osd.5 2024-09-21T05:56:15.491 DEBUG:teuthology.parallel:result is None 2024-09-21T05:56:15.739 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:15 smithi117 ceph-mon[30947]: pgmap v295: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-09-21T05:56:15.739 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:15 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3534229662' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T05:56:15.898 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:15 smithi192 ceph-mon[34700]: pgmap v295: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-09-21T05:56:15.898 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:15 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3534229662' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T05:56:16.067 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542215 got 60129542218 for osd.1 2024-09-21T05:56:16.067 DEBUG:teuthology.parallel:result is None 2024-09-21T05:56:16.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:15 smithi138 ceph-mon[34391]: pgmap v295: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-09-21T05:56:16.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:15 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3534229662' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T05:56:16.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.232+0000 7fb97d31e640 1 -- 172.21.15.117:0/363899514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb978102050 msgr2=0x7fb978102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:16.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.232+0000 7fb97d31e640 1 --2- 172.21.15.117:0/363899514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb978102050 0x7fb978102460 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb968009900 tx=0x7fb96802a410 comp rx=0 tx=0).stop 2024-09-21T05:56:16.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.232+0000 7fb97d31e640 1 -- 172.21.15.117:0/363899514 shutdown_connections 2024-09-21T05:56:16.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.232+0000 7fb97d31e640 1 --2- 172.21.15.117:0/363899514 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb978106bf0 0x7fb978110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:16.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.232+0000 7fb97d31e640 1 --2- 172.21.15.117:0/363899514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9781062a0 0x7fb9781066b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:16.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.232+0000 7fb97d31e640 1 --2- 172.21.15.117:0/363899514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb978102050 0x7fb978102460 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:16.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.232+0000 7fb97d31e640 1 -- 172.21.15.117:0/363899514 >> 172.21.15.117:0/363899514 conn(0x7fb978077c60 msgr2=0x7fb9780780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:16.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.232+0000 7fb97d31e640 1 -- 172.21.15.117:0/363899514 shutdown_connections 2024-09-21T05:56:16.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.232+0000 7fb97d31e640 1 -- 172.21.15.117:0/363899514 wait complete. 2024-09-21T05:56:16.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.232+0000 7fb97d31e640 1 Processor -- start 2024-09-21T05:56:16.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.233+0000 7fb97d31e640 1 -- start start 2024-09-21T05:56:16.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.233+0000 7fb97d31e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb978102050 0x7fb978102ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:16.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.233+0000 7fb97d31e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9781062a0 0x7fb978103530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:16.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.233+0000 7fb97d31e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb978106bf0 0x7fb978103a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:16.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.233+0000 7fb97d31e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb978113ac0 con 0x7fb9781062a0 2024-09-21T05:56:16.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.233+0000 7fb97d31e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb978113940 con 0x7fb978102050 2024-09-21T05:56:16.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.233+0000 7fb97d31e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb978113c40 con 0x7fb978106bf0 2024-09-21T05:56:16.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.233+0000 7fb9767fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9781062a0 0x7fb978103530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:16.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.233+0000 7fb9767fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9781062a0 0x7fb978103530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40104/0 (socket says 172.21.15.117:40104) 2024-09-21T05:56:16.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.233+0000 7fb9767fc640 1 -- 172.21.15.117:0/1502630425 learned_addr learned my addr 172.21.15.117:0/1502630425 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:16.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.233+0000 7fb9777fe640 1 --2- 172.21.15.117:0/1502630425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb978106bf0 0x7fb978103a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:16.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.233+0000 7fb976ffd640 1 --2- 172.21.15.117:0/1502630425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb978102050 0x7fb978102ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:16.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.233+0000 7fb9767fc640 1 -- 172.21.15.117:0/1502630425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb978106bf0 msgr2=0x7fb978103a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:16.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.234+0000 7fb9767fc640 1 --2- 172.21.15.117:0/1502630425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb978106bf0 0x7fb978103a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:16.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.234+0000 7fb9767fc640 1 -- 172.21.15.117:0/1502630425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb978102050 msgr2=0x7fb978102ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:16.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.234+0000 7fb9767fc640 1 --2- 172.21.15.117:0/1502630425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb978102050 0x7fb978102ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:16.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.234+0000 7fb9767fc640 1 -- 172.21.15.117:0/1502630425 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9781a8750 con 0x7fb9781062a0 2024-09-21T05:56:16.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.234+0000 7fb9767fc640 1 --2- 172.21.15.117:0/1502630425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9781062a0 0x7fb978103530 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb96000e9f0 tx=0x7fb96000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:16.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.234+0000 7fb96ffff640 1 -- 172.21.15.117:0/1502630425 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96000ccc0 con 0x7fb9781062a0 2024-09-21T05:56:16.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.234+0000 7fb97d31e640 1 -- 172.21.15.117:0/1502630425 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9781a8a40 con 0x7fb9781062a0 2024-09-21T05:56:16.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.234+0000 7fb96ffff640 1 -- 172.21.15.117:0/1502630425 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9600108e0 con 0x7fb9781062a0 2024-09-21T05:56:16.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.234+0000 7fb96ffff640 1 -- 172.21.15.117:0/1502630425 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb960018860 con 0x7fb9781062a0 2024-09-21T05:56:16.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.234+0000 7fb97d31e640 1 -- 172.21.15.117:0/1502630425 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9781a8fa0 con 0x7fb9781062a0 2024-09-21T05:56:16.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.235+0000 7fb97d31e640 1 -- 172.21.15.117:0/1502630425 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb978102460 con 0x7fb9781062a0 2024-09-21T05:56:16.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.237+0000 7fb96ffff640 1 -- 172.21.15.117:0/1502630425 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7fb960010430 con 0x7fb9781062a0 2024-09-21T05:56:16.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.237+0000 7fb96ffff640 1 --2- 172.21.15.117:0/1502630425 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb9640b65f0 0x7fb9640b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:16.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.238+0000 7fb976ffd640 1 --2- 172.21.15.117:0/1502630425 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb9640b65f0 0x7fb9640b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:16.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.238+0000 7fb96ffff640 1 -- 172.21.15.117:0/1502630425 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fb9600e7d00 con 0x7fb9781062a0 2024-09-21T05:56:16.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.238+0000 7fb976ffd640 1 --2- 172.21.15.117:0/1502630425 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb9640b65f0 0x7fb9640b8b00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb968004770 tx=0x7fb9680058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:16.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.245+0000 7fb96ffff640 1 -- 172.21.15.117:0/1502630425 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb9600b2770 con 0x7fb9781062a0 2024-09-21T05:56:16.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.434+0000 7fb97d31e640 1 -- 172.21.15.117:0/1502630425 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fb9781042f0 con 0x7fb9781062a0 2024-09-21T05:56:16.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.434+0000 7fb96ffff640 1 -- 172.21.15.117:0/1502630425 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb9600b76b0 con 0x7fb9781062a0 2024-09-21T05:56:16.436 INFO:teuthology.orchestra.run.smithi117.stdout:107374182466 2024-09-21T05:56:16.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.436+0000 7fb97d31e640 1 -- 172.21.15.117:0/1502630425 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb9640b65f0 msgr2=0x7fb9640b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:16.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.436+0000 7fb97d31e640 1 --2- 172.21.15.117:0/1502630425 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb9640b65f0 0x7fb9640b8b00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb968004770 tx=0x7fb9680058d0 comp rx=0 tx=0).stop 2024-09-21T05:56:16.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.436+0000 7fb97d31e640 1 -- 172.21.15.117:0/1502630425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9781062a0 msgr2=0x7fb978103530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:16.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.436+0000 7fb97d31e640 1 --2- 172.21.15.117:0/1502630425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9781062a0 0x7fb978103530 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb96000e9f0 tx=0x7fb96000eeb0 comp rx=0 tx=0).stop 2024-09-21T05:56:16.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.436+0000 7fb97d31e640 1 -- 172.21.15.117:0/1502630425 shutdown_connections 2024-09-21T05:56:16.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.436+0000 7fb97d31e640 1 --2- 172.21.15.117:0/1502630425 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb9640b65f0 0x7fb9640b8b00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:16.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.436+0000 7fb97d31e640 1 --2- 172.21.15.117:0/1502630425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9781062a0 0x7fb978103530 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:16.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.436+0000 7fb97d31e640 1 --2- 172.21.15.117:0/1502630425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb978106bf0 0x7fb978103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:16.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.436+0000 7fb97d31e640 1 --2- 172.21.15.117:0/1502630425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb978102050 0x7fb978102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:16.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.436+0000 7fb97d31e640 1 -- 172.21.15.117:0/1502630425 >> 172.21.15.117:0/1502630425 conn(0x7fb978077c60 msgr2=0x7fb9781074b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:16.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.437+0000 7fb97d31e640 1 -- 172.21.15.117:0/1502630425 shutdown_connections 2024-09-21T05:56:16.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.437+0000 7fb97d31e640 1 -- 172.21.15.117:0/1502630425 wait complete. 2024-09-21T05:56:16.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.609+0000 7f124e11a640 1 -- 172.21.15.117:0/1529108801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12481084b0 msgr2=0x7f124810ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:16.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.609+0000 7f124e11a640 1 --2- 172.21.15.117:0/1529108801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12481084b0 0x7f124810ece0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1240009ca0 tx=0x7f124002a8c0 comp rx=0 tx=0).stop 2024-09-21T05:56:16.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.609+0000 7f124e11a640 1 -- 172.21.15.117:0/1529108801 shutdown_connections 2024-09-21T05:56:16.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.609+0000 7f124e11a640 1 --2- 172.21.15.117:0/1529108801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12481084b0 0x7f124810ece0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:16.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.609+0000 7f124e11a640 1 --2- 172.21.15.117:0/1529108801 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1248107b60 0x7f1248107f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:16.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.609+0000 7f124e11a640 1 --2- 172.21.15.117:0/1529108801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1248102270 0x7f1248102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:16.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.609+0000 7f124e11a640 1 -- 172.21.15.117:0/1529108801 >> 172.21.15.117:0/1529108801 conn(0x7f1248077cc0 msgr2=0x7f1248078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:16.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.610+0000 7f124e11a640 1 -- 172.21.15.117:0/1529108801 shutdown_connections 2024-09-21T05:56:16.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.610+0000 7f124e11a640 1 -- 172.21.15.117:0/1529108801 wait complete. 2024-09-21T05:56:16.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.610+0000 7f124e11a640 1 Processor -- start 2024-09-21T05:56:16.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.610+0000 7f124e11a640 1 -- start start 2024-09-21T05:56:16.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.611+0000 7f124e11a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1248102270 0x7f12480fee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:16.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.611+0000 7f124e11a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1248107b60 0x7f12480ff340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:16.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.611+0000 7f124e11a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12481084b0 0x7f12480ff880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:16.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.611+0000 7f124e11a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1248112160 con 0x7f1248107b60 2024-09-21T05:56:16.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.611+0000 7f124e11a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1248111fe0 con 0x7f12481084b0 2024-09-21T05:56:16.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.611+0000 7f124e11a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f12481122e0 con 0x7f1248102270 2024-09-21T05:56:16.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.611+0000 7f1246ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1248107b60 0x7f12480ff340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:16.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.611+0000 7f12477fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1248102270 0x7f12480fee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:16.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.611+0000 7f1246ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1248107b60 0x7f12480ff340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40126/0 (socket says 172.21.15.117:40126) 2024-09-21T05:56:16.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.611+0000 7f1246ffd640 1 -- 172.21.15.117:0/3211047741 learned_addr learned my addr 172.21.15.117:0/3211047741 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:16.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.611+0000 7f1247fff640 1 --2- 172.21.15.117:0/3211047741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12481084b0 0x7f12480ff880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:16.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.611+0000 7f12477fe640 1 -- 172.21.15.117:0/3211047741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12481084b0 msgr2=0x7f12480ff880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:16.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.611+0000 7f12477fe640 1 --2- 172.21.15.117:0/3211047741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12481084b0 0x7f12480ff880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:16.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.611+0000 7f12477fe640 1 -- 172.21.15.117:0/3211047741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1248107b60 msgr2=0x7f12480ff340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:16.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.611+0000 7f12477fe640 1 --2- 172.21.15.117:0/3211047741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1248107b60 0x7f12480ff340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:16.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.611+0000 7f12477fe640 1 -- 172.21.15.117:0/3211047741 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12481a8720 con 0x7f1248102270 2024-09-21T05:56:16.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.611+0000 7f1246ffd640 1 --2- 172.21.15.117:0/3211047741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1248107b60 0x7f12480ff340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:56:16.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.612+0000 7f12477fe640 1 --2- 172.21.15.117:0/3211047741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1248102270 0x7f12480fee00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f123800e9f0 tx=0x7f123800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:16.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.612+0000 7f1244ff9640 1 -- 172.21.15.117:0/3211047741 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123800cd80 con 0x7f1248102270 2024-09-21T05:56:16.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.612+0000 7f1244ff9640 1 -- 172.21.15.117:0/3211047741 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12380035f0 con 0x7f1248102270 2024-09-21T05:56:16.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.612+0000 7f124e11a640 1 -- 172.21.15.117:0/3211047741 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12481a8a10 con 0x7f1248102270 2024-09-21T05:56:16.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.612+0000 7f1244ff9640 1 -- 172.21.15.117:0/3211047741 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1238010670 con 0x7f1248102270 2024-09-21T05:56:16.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.612+0000 7f124e11a640 1 -- 172.21.15.117:0/3211047741 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12481a8f50 con 0x7f1248102270 2024-09-21T05:56:16.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.613+0000 7f124e11a640 1 -- 172.21.15.117:0/3211047741 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1248102680 con 0x7f1248102270 2024-09-21T05:56:16.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.615+0000 7f1244ff9640 1 -- 172.21.15.117:0/3211047741 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f12380037a0 con 0x7f1248102270 2024-09-21T05:56:16.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.616+0000 7f1244ff9640 1 --2- 172.21.15.117:0/3211047741 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f12300b65f0 0x7f12300b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:16.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.616+0000 7f1246ffd640 1 --2- 172.21.15.117:0/3211047741 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f12300b65f0 0x7f12300b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:16.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.616+0000 7f1244ff9640 1 -- 172.21.15.117:0/3211047741 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f12380e7a20 con 0x7f1248102270 2024-09-21T05:56:16.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.617+0000 7f1246ffd640 1 --2- 172.21.15.117:0/3211047741 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f12300b65f0 0x7f12300b8b00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f124806b910 tx=0x7f123c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:16.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.618+0000 7f1244ff9640 1 -- 172.21.15.117:0/3211047741 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1238056710 con 0x7f1248102270 2024-09-21T05:56:16.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.810+0000 7f124e11a640 1 -- 172.21.15.117:0/3211047741 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f124802e640 con 0x7f1248102270 2024-09-21T05:56:16.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.810+0000 7f1244ff9640 1 -- 172.21.15.117:0/3211047741 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f12380ae0b0 con 0x7f1248102270 2024-09-21T05:56:16.814 INFO:teuthology.orchestra.run.smithi117.stdout:292057776153 2024-09-21T05:56:16.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.813+0000 7f124e11a640 1 -- 172.21.15.117:0/3211047741 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f12300b65f0 msgr2=0x7f12300b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:16.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.813+0000 7f124e11a640 1 --2- 172.21.15.117:0/3211047741 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f12300b65f0 0x7f12300b8b00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f124806b910 tx=0x7f123c005f90 comp rx=0 tx=0).stop 2024-09-21T05:56:16.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.813+0000 7f124e11a640 1 -- 172.21.15.117:0/3211047741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1248102270 msgr2=0x7f12480fee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:16.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.813+0000 7f124e11a640 1 --2- 172.21.15.117:0/3211047741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1248102270 0x7f12480fee00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f123800e9f0 tx=0x7f123800eeb0 comp rx=0 tx=0).stop 2024-09-21T05:56:16.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.813+0000 7f124e11a640 1 -- 172.21.15.117:0/3211047741 shutdown_connections 2024-09-21T05:56:16.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.813+0000 7f124e11a640 1 --2- 172.21.15.117:0/3211047741 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f12300b65f0 0x7f12300b8b00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:16.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.813+0000 7f124e11a640 1 --2- 172.21.15.117:0/3211047741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1248107b60 0x7f12480ff340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:16.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.813+0000 7f124e11a640 1 --2- 172.21.15.117:0/3211047741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12481084b0 0x7f12480ff880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:16.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.813+0000 7f124e11a640 1 --2- 172.21.15.117:0/3211047741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1248102270 0x7f12480fee00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:16.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.813+0000 7f124e11a640 1 -- 172.21.15.117:0/3211047741 >> 172.21.15.117:0/3211047741 conn(0x7f1248077cc0 msgr2=0x7f1248108b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:16.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.813+0000 7f124e11a640 1 -- 172.21.15.117:0/3211047741 shutdown_connections 2024-09-21T05:56:16.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:16.813+0000 7f124e11a640 1 -- 172.21.15.117:0/3211047741 wait complete. 2024-09-21T05:56:16.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:16 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/1502630425' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T05:56:17.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:16 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/1502630425' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T05:56:17.082 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986234 got 133143986237 for osd.4 2024-09-21T05:56:17.082 DEBUG:teuthology.parallel:result is None 2024-09-21T05:56:17.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:16 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/1502630425' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T05:56:17.875 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776150 got 292057776153 for osd.11 2024-09-21T05:56:17.875 DEBUG:teuthology.parallel:result is None 2024-09-21T05:56:17.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:17 smithi192 ceph-mon[34700]: pgmap v296: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T05:56:17.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:17 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3211047741' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T05:56:18.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:17 smithi117 ceph-mon[30947]: pgmap v296: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T05:56:18.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:17 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3211047741' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T05:56:18.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:17 smithi138 ceph-mon[34391]: pgmap v296: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-21T05:56:18.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:17 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3211047741' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T05:56:18.318 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182463 got 107374182466 for osd.3 2024-09-21T05:56:18.318 DEBUG:teuthology.parallel:result is None 2024-09-21T05:56:18.318 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T05:56:18.319 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T05:56:18.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:18 smithi192 ceph-mon[34700]: pgmap v297: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-21T05:56:19.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:18 smithi117 ceph-mon[30947]: pgmap v297: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-21T05:56:19.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:18 smithi138 ceph-mon[34391]: pgmap v297: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-21T05:56:19.134 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:19.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.686+0000 7fbb09794640 1 -- 172.21.15.117:0/202890907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb04108d70 msgr2=0x7fbb0410f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:19.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.686+0000 7fbb09794640 1 --2- 172.21.15.117:0/202890907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb04108d70 0x7fbb0410f5a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4009900 tx=0x7fbaf402a410 comp rx=0 tx=0).stop 2024-09-21T05:56:19.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.686+0000 7fbb09794640 1 -- 172.21.15.117:0/202890907 shutdown_connections 2024-09-21T05:56:19.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.686+0000 7fbb09794640 1 --2- 172.21.15.117:0/202890907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb04108d70 0x7fbb0410f5a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:19.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.686+0000 7fbb09794640 1 --2- 172.21.15.117:0/202890907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb04108420 0x7fbb04108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:19.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.686+0000 7fbb09794640 1 --2- 172.21.15.117:0/202890907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb04102ae0 0x7fbb04102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:19.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.686+0000 7fbb09794640 1 -- 172.21.15.117:0/202890907 >> 172.21.15.117:0/202890907 conn(0x7fbb04079770 msgr2=0x7fbb04079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:19.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.686+0000 7fbb09794640 1 -- 172.21.15.117:0/202890907 shutdown_connections 2024-09-21T05:56:19.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.686+0000 7fbb09794640 1 -- 172.21.15.117:0/202890907 wait complete. 2024-09-21T05:56:19.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.687+0000 7fbb09794640 1 Processor -- start 2024-09-21T05:56:19.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.687+0000 7fbb09794640 1 -- start start 2024-09-21T05:56:19.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.687+0000 7fbb09794640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb04102ae0 0x7fbb040ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:19.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.687+0000 7fbb09794640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb04108420 0x7fbb040ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:19.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.687+0000 7fbb09794640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb04108d70 0x7fbb041000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:19.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.687+0000 7fbb09794640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbb04112a30 con 0x7fbb04108d70 2024-09-21T05:56:19.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.687+0000 7fbb09794640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbb041128b0 con 0x7fbb04108420 2024-09-21T05:56:19.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.687+0000 7fbb09794640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbb04112bb0 con 0x7fbb04102ae0 2024-09-21T05:56:19.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.688+0000 7fbb037fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb04108d70 0x7fbb041000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:19.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.688+0000 7fbb037fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb04108d70 0x7fbb041000f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40134/0 (socket says 172.21.15.117:40134) 2024-09-21T05:56:19.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.688+0000 7fbb037fe640 1 -- 172.21.15.117:0/198567634 learned_addr learned my addr 172.21.15.117:0/198567634 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:19.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.688+0000 7fbb027fc640 1 --2- 172.21.15.117:0/198567634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb04108420 0x7fbb040ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:19.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.688+0000 7fbb02ffd640 1 --2- 172.21.15.117:0/198567634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb04102ae0 0x7fbb040ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:19.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.688+0000 7fbb037fe640 1 -- 172.21.15.117:0/198567634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb04102ae0 msgr2=0x7fbb040ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:19.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.688+0000 7fbb037fe640 1 --2- 172.21.15.117:0/198567634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb04102ae0 0x7fbb040ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:19.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.688+0000 7fbb037fe640 1 -- 172.21.15.117:0/198567634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb04108420 msgr2=0x7fbb040ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:19.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.688+0000 7fbb037fe640 1 --2- 172.21.15.117:0/198567634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb04108420 0x7fbb040ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:19.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.688+0000 7fbb037fe640 1 -- 172.21.15.117:0/198567634 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb041a87d0 con 0x7fbb04108d70 2024-09-21T05:56:19.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.688+0000 7fbb02ffd640 1 --2- 172.21.15.117:0/198567634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb04102ae0 0x7fbb040ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:19.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.688+0000 7fbb027fc640 1 --2- 172.21.15.117:0/198567634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb04108420 0x7fbb040ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:56:19.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.688+0000 7fbb037fe640 1 --2- 172.21.15.117:0/198567634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb04108d70 0x7fbb041000f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4004770 tx=0x7fbaf4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:19.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.688+0000 7fbafbfff640 1 -- 172.21.15.117:0/198567634 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf403f030 con 0x7fbb04108d70 2024-09-21T05:56:19.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.688+0000 7fbb09794640 1 -- 172.21.15.117:0/198567634 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb041a8a60 con 0x7fbb04108d70 2024-09-21T05:56:19.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.688+0000 7fbafbfff640 1 -- 172.21.15.117:0/198567634 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbaf4003aa0 con 0x7fbb04108d70 2024-09-21T05:56:19.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.688+0000 7fbafbfff640 1 -- 172.21.15.117:0/198567634 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4035070 con 0x7fbb04108d70 2024-09-21T05:56:19.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.689+0000 7fbb09794640 1 -- 172.21.15.117:0/198567634 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb041a8f90 con 0x7fbb04108d70 2024-09-21T05:56:19.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.690+0000 7fbb09794640 1 -- 172.21.15.117:0/198567634 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb04103d60 con 0x7fbb04108d70 2024-09-21T05:56:19.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.691+0000 7fbafbfff640 1 -- 172.21.15.117:0/198567634 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7fbaf4042030 con 0x7fbb04108d70 2024-09-21T05:56:19.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.692+0000 7fbafbfff640 1 --2- 172.21.15.117:0/198567634 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbaf00b65f0 0x7fbaf00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:19.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.692+0000 7fbb02ffd640 1 --2- 172.21.15.117:0/198567634 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbaf00b65f0 0x7fbaf00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:19.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.692+0000 7fbafbfff640 1 -- 172.21.15.117:0/198567634 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fbaf4107ab0 con 0x7fbb04108d70 2024-09-21T05:56:19.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.693+0000 7fbb02ffd640 1 --2- 172.21.15.117:0/198567634 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbaf00b65f0 0x7fbaf00b8b00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbb041010c0 tx=0x7fbafc005e70 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:19.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.697+0000 7fbafbfff640 1 -- 172.21.15.117:0/198567634 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbaf40ce140 con 0x7fbb04108d70 2024-09-21T05:56:19.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.894+0000 7fbb09794640 1 -- 172.21.15.117:0/198567634 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbb04101740 con 0x7fbaf00b65f0 2024-09-21T05:56:19.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.899+0000 7fbafbfff640 1 -- 172.21.15.117:0/198567634 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259045 (secure 0 0 0) 0x7fbb04101740 con 0x7fbaf00b65f0 2024-09-21T05:56:19.903 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:56:19.904 INFO:teuthology.orchestra.run.smithi117.stderr:dumped all 2024-09-21T05:56:19.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.903+0000 7fbb09794640 1 -- 172.21.15.117:0/198567634 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbaf00b65f0 msgr2=0x7fbaf00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:19.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.903+0000 7fbb09794640 1 --2- 172.21.15.117:0/198567634 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbaf00b65f0 0x7fbaf00b8b00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbb041010c0 tx=0x7fbafc005e70 comp rx=0 tx=0).stop 2024-09-21T05:56:19.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.903+0000 7fbb09794640 1 -- 172.21.15.117:0/198567634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb04108d70 msgr2=0x7fbb041000f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:19.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.903+0000 7fbb09794640 1 --2- 172.21.15.117:0/198567634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb04108d70 0x7fbb041000f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4004770 tx=0x7fbaf4004270 comp rx=0 tx=0).stop 2024-09-21T05:56:19.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.903+0000 7fbb09794640 1 -- 172.21.15.117:0/198567634 shutdown_connections 2024-09-21T05:56:19.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.903+0000 7fbb09794640 1 --2- 172.21.15.117:0/198567634 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbaf00b65f0 0x7fbaf00b8b00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:19.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.903+0000 7fbb09794640 1 --2- 172.21.15.117:0/198567634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb04108d70 0x7fbb041000f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:19.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.903+0000 7fbb09794640 1 --2- 172.21.15.117:0/198567634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb04108420 0x7fbb040ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:19.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.903+0000 7fbb09794640 1 --2- 172.21.15.117:0/198567634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb04102ae0 0x7fbb040ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:19.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.903+0000 7fbb09794640 1 -- 172.21.15.117:0/198567634 >> 172.21.15.117:0/198567634 conn(0x7fbb04079770 msgr2=0x7fbb0406be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:19.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.903+0000 7fbb09794640 1 -- 172.21.15.117:0/198567634 shutdown_connections 2024-09-21T05:56:19.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:19.903+0000 7fbb09794640 1 -- 172.21.15.117:0/198567634 wait complete. 2024-09-21T05:56:20.573 INFO:teuthology.orchestra.run.smithi117.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2024-09-21T05:56:18.520849+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662099,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":623,"num_read_kb":417,"num_write":370,"num_write_kb":2294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":2952512,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":363760,"kb_used_data":10320,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124578064,"statfs":{"total":1151940427776,"available":1151567937536,"internally_reserved":0,"allocated":10567680,"data_stored":6909945,"data_compressed":182193,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":129,"num_read_kb":69,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003422"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636773+0000","last_change":"2024-09-21T05:55:14.007642+0000","last_active":"2024-09-21T05:55:33.636773+0000","last_peered":"2024-09-21T05:55:33.636773+0000","last_clean":"2024-09-21T05:55:33.636773+0000","last_became_active":"2024-09-21T05:55:14.006943+0000","last_became_peered":"2024-09-21T05:55:14.006943+0000","last_unstale":"2024-09-21T05:55:33.636773+0000","last_undegraded":"2024-09-21T05:55:33.636773+0000","last_fullsized":"2024-09-21T05:55:33.636773+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:23:53.816079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767047+0000","last_change":"2024-09-21T05:55:14.002034+0000","last_active":"2024-09-21T05:55:33.767047+0000","last_peered":"2024-09-21T05:55:33.767047+0000","last_clean":"2024-09-21T05:55:33.767047+0000","last_became_active":"2024-09-21T05:55:14.001658+0000","last_became_peered":"2024-09-21T05:55:14.001658+0000","last_unstale":"2024-09-21T05:55:33.767047+0000","last_undegraded":"2024-09-21T05:55:33.767047+0000","last_fullsized":"2024-09-21T05:55:33.767047+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:01.568362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.766840+0000","last_change":"2024-09-21T05:55:13.996901+0000","last_active":"2024-09-21T05:55:33.766840+0000","last_peered":"2024-09-21T05:55:33.766840+0000","last_clean":"2024-09-21T05:55:33.766840+0000","last_became_active":"2024-09-21T05:55:13.996020+0000","last_became_peered":"2024-09-21T05:55:13.996020+0000","last_unstale":"2024-09-21T05:55:33.766840+0000","last_undegraded":"2024-09-21T05:55:33.766840+0000","last_fullsized":"2024-09-21T05:55:33.766840+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:38.771926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.696132+0000","last_change":"2024-09-21T05:55:13.995968+0000","last_active":"2024-09-21T05:55:33.696132+0000","last_peered":"2024-09-21T05:55:33.696132+0000","last_clean":"2024-09-21T05:55:33.696132+0000","last_became_active":"2024-09-21T05:55:13.995259+0000","last_became_peered":"2024-09-21T05:55:13.995259+0000","last_unstale":"2024-09-21T05:55:33.696132+0000","last_undegraded":"2024-09-21T05:55:33.696132+0000","last_fullsized":"2024-09-21T05:55:33.696132+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:48.953342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635823+0000","last_change":"2024-09-21T05:55:13.995778+0000","last_active":"2024-09-21T05:55:33.635823+0000","last_peered":"2024-09-21T05:55:33.635823+0000","last_clean":"2024-09-21T05:55:33.635823+0000","last_became_active":"2024-09-21T05:55:13.995415+0000","last_became_peered":"2024-09-21T05:55:13.995415+0000","last_unstale":"2024-09-21T05:55:33.635823+0000","last_undegraded":"2024-09-21T05:55:33.635823+0000","last_fullsized":"2024-09-21T05:55:33.635823+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:52.562120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.634110+0000","last_change":"2024-09-21T05:55:14.000005+0000","last_active":"2024-09-21T05:55:33.634110+0000","last_peered":"2024-09-21T05:55:33.634110+0000","last_clean":"2024-09-21T05:55:33.634110+0000","last_became_active":"2024-09-21T05:55:13.999493+0000","last_became_peered":"2024-09-21T05:55:13.999493+0000","last_unstale":"2024-09-21T05:55:33.634110+0000","last_undegraded":"2024-09-21T05:55:33.634110+0000","last_fullsized":"2024-09-21T05:55:33.634110+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:29.426368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635816+0000","last_change":"2024-09-21T05:55:13.993004+0000","last_active":"2024-09-21T05:55:33.635816+0000","last_peered":"2024-09-21T05:55:33.635816+0000","last_clean":"2024-09-21T05:55:33.635816+0000","last_became_active":"2024-09-21T05:55:13.992615+0000","last_became_peered":"2024-09-21T05:55:13.992615+0000","last_unstale":"2024-09-21T05:55:33.635816+0000","last_undegraded":"2024-09-21T05:55:33.635816+0000","last_fullsized":"2024-09-21T05:55:33.635816+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:48.324618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.769380+0000","last_change":"2024-09-21T05:55:33.769006+0000","last_active":"2024-09-21T05:55:33.769380+0000","last_peered":"2024-09-21T05:55:33.769380+0000","last_clean":"2024-09-21T05:55:33.769380+0000","last_became_active":"2024-09-21T05:55:33.768705+0000","last_became_peered":"2024-09-21T05:55:33.768705+0000","last_unstale":"2024-09-21T05:55:33.769380+0000","last_undegraded":"2024-09-21T05:55:33.769380+0000","last_fullsized":"2024-09-21T05:55:33.769380+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:14.013300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639798+0000","last_change":"2024-09-21T05:55:14.004131+0000","last_active":"2024-09-21T05:55:33.639798+0000","last_peered":"2024-09-21T05:55:33.639798+0000","last_clean":"2024-09-21T05:55:33.639798+0000","last_became_active":"2024-09-21T05:55:14.003621+0000","last_became_peered":"2024-09-21T05:55:14.003621+0000","last_unstale":"2024-09-21T05:55:33.639798+0000","last_undegraded":"2024-09-21T05:55:33.639798+0000","last_fullsized":"2024-09-21T05:55:33.639798+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:00.694211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.643388+0000","last_change":"2024-09-21T05:55:33.643289+0000","last_active":"2024-09-21T05:55:33.643388+0000","last_peered":"2024-09-21T05:55:33.643388+0000","last_clean":"2024-09-21T05:55:33.643388+0000","last_became_active":"2024-09-21T05:55:33.642805+0000","last_became_peered":"2024-09-21T05:55:33.642805+0000","last_unstale":"2024-09-21T05:55:33.643388+0000","last_undegraded":"2024-09-21T05:55:33.643388+0000","last_fullsized":"2024-09-21T05:55:33.643388+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:00.854712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636225+0000","last_change":"2024-09-21T05:55:14.006467+0000","last_active":"2024-09-21T05:55:33.636225+0000","last_peered":"2024-09-21T05:55:33.636225+0000","last_clean":"2024-09-21T05:55:33.636225+0000","last_became_active":"2024-09-21T05:55:14.005978+0000","last_became_peered":"2024-09-21T05:55:14.005978+0000","last_unstale":"2024-09-21T05:55:33.636225+0000","last_undegraded":"2024-09-21T05:55:33.636225+0000","last_fullsized":"2024-09-21T05:55:33.636225+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:37.391152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.696241+0000","last_change":"2024-09-21T05:55:14.003972+0000","last_active":"2024-09-21T05:55:33.696241+0000","last_peered":"2024-09-21T05:55:33.696241+0000","last_clean":"2024-09-21T05:55:33.696241+0000","last_became_active":"2024-09-21T05:55:14.003601+0000","last_became_peered":"2024-09-21T05:55:14.003601+0000","last_unstale":"2024-09-21T05:55:33.696241+0000","last_undegraded":"2024-09-21T05:55:33.696241+0000","last_fullsized":"2024-09-21T05:55:33.696241+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:16.201087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636463+0000","last_change":"2024-09-21T05:55:13.996051+0000","last_active":"2024-09-21T05:55:33.636463+0000","last_peered":"2024-09-21T05:55:33.636463+0000","last_clean":"2024-09-21T05:55:33.636463+0000","last_became_active":"2024-09-21T05:55:13.995679+0000","last_became_peered":"2024-09-21T05:55:13.995679+0000","last_unstale":"2024-09-21T05:55:33.636463+0000","last_undegraded":"2024-09-21T05:55:33.636463+0000","last_fullsized":"2024-09-21T05:55:33.636463+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:45:44.372029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.766681+0000","last_change":"2024-09-21T05:55:13.995421+0000","last_active":"2024-09-21T05:55:33.766681+0000","last_peered":"2024-09-21T05:55:33.766681+0000","last_clean":"2024-09-21T05:55:33.766681+0000","last_became_active":"2024-09-21T05:55:13.995006+0000","last_became_peered":"2024-09-21T05:55:13.995006+0000","last_unstale":"2024-09-21T05:55:33.766681+0000","last_undegraded":"2024-09-21T05:55:33.766681+0000","last_fullsized":"2024-09-21T05:55:33.766681+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:00.636555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.766677+0000","last_change":"2024-09-21T05:55:13.993466+0000","last_active":"2024-09-21T05:55:33.766677+0000","last_peered":"2024-09-21T05:55:33.766677+0000","last_clean":"2024-09-21T05:55:33.766677+0000","last_became_active":"2024-09-21T05:55:13.993137+0000","last_became_peered":"2024-09-21T05:55:13.993137+0000","last_unstale":"2024-09-21T05:55:33.766677+0000","last_undegraded":"2024-09-21T05:55:33.766677+0000","last_fullsized":"2024-09-21T05:55:33.766677+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:33.218868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639111+0000","last_change":"2024-09-21T05:55:13.996316+0000","last_active":"2024-09-21T05:55:33.639111+0000","last_peered":"2024-09-21T05:55:33.639111+0000","last_clean":"2024-09-21T05:55:33.639111+0000","last_became_active":"2024-09-21T05:55:13.995918+0000","last_became_peered":"2024-09-21T05:55:13.995918+0000","last_unstale":"2024-09-21T05:55:33.639111+0000","last_undegraded":"2024-09-21T05:55:33.639111+0000","last_fullsized":"2024-09-21T05:55:33.639111+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:45:54.408504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636962+0000","last_change":"2024-09-21T05:55:14.007198+0000","last_active":"2024-09-21T05:55:33.636962+0000","last_peered":"2024-09-21T05:55:33.636962+0000","last_clean":"2024-09-21T05:55:33.636962+0000","last_became_active":"2024-09-21T05:55:14.006700+0000","last_became_peered":"2024-09-21T05:55:14.006700+0000","last_unstale":"2024-09-21T05:55:33.636962+0000","last_undegraded":"2024-09-21T05:55:33.636962+0000","last_fullsized":"2024-09-21T05:55:33.636962+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:45.343778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.633333+0000","last_change":"2024-09-21T05:55:14.000152+0000","last_active":"2024-09-21T05:55:33.633333+0000","last_peered":"2024-09-21T05:55:33.633333+0000","last_clean":"2024-09-21T05:55:33.633333+0000","last_became_active":"2024-09-21T05:55:13.999786+0000","last_became_peered":"2024-09-21T05:55:13.999786+0000","last_unstale":"2024-09-21T05:55:33.633333+0000","last_undegraded":"2024-09-21T05:55:33.633333+0000","last_fullsized":"2024-09-21T05:55:33.633333+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:43:25.713410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.640106+0000","last_change":"2024-09-21T05:55:13.988367+0000","last_active":"2024-09-21T05:55:33.640106+0000","last_peered":"2024-09-21T05:55:33.640106+0000","last_clean":"2024-09-21T05:55:33.640106+0000","last_became_active":"2024-09-21T05:55:13.987982+0000","last_became_peered":"2024-09-21T05:55:13.987982+0000","last_unstale":"2024-09-21T05:55:33.640106+0000","last_undegraded":"2024-09-21T05:55:33.640106+0000","last_fullsized":"2024-09-21T05:55:33.640106+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:15.395155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636251+0000","last_change":"2024-09-21T05:55:14.004962+0000","last_active":"2024-09-21T05:55:33.636251+0000","last_peered":"2024-09-21T05:55:33.636251+0000","last_clean":"2024-09-21T05:55:33.636251+0000","last_became_active":"2024-09-21T05:55:14.004573+0000","last_became_peered":"2024-09-21T05:55:14.004573+0000","last_unstale":"2024-09-21T05:55:33.636251+0000","last_undegraded":"2024-09-21T05:55:33.636251+0000","last_fullsized":"2024-09-21T05:55:33.636251+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:59.014017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636468+0000","last_change":"2024-09-21T05:55:13.991267+0000","last_active":"2024-09-21T05:55:33.636468+0000","last_peered":"2024-09-21T05:55:33.636468+0000","last_clean":"2024-09-21T05:55:33.636468+0000","last_became_active":"2024-09-21T05:55:13.990916+0000","last_became_peered":"2024-09-21T05:55:13.990916+0000","last_unstale":"2024-09-21T05:55:33.636468+0000","last_undegraded":"2024-09-21T05:55:33.636468+0000","last_fullsized":"2024-09-21T05:55:33.636468+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:06:03.904632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639504+0000","last_change":"2024-09-21T05:55:13.994147+0000","last_active":"2024-09-21T05:55:33.639504+0000","last_peered":"2024-09-21T05:55:33.639504+0000","last_clean":"2024-09-21T05:55:33.639504+0000","last_became_active":"2024-09-21T05:55:13.993794+0000","last_became_peered":"2024-09-21T05:55:13.993794+0000","last_unstale":"2024-09-21T05:55:33.639504+0000","last_undegraded":"2024-09-21T05:55:33.639504+0000","last_fullsized":"2024-09-21T05:55:33.639504+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:19.970853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.638668+0000","last_change":"2024-09-21T05:55:14.001119+0000","last_active":"2024-09-21T05:55:33.638668+0000","last_peered":"2024-09-21T05:55:33.638668+0000","last_clean":"2024-09-21T05:55:33.638668+0000","last_became_active":"2024-09-21T05:55:14.000799+0000","last_became_peered":"2024-09-21T05:55:14.000799+0000","last_unstale":"2024-09-21T05:55:33.638668+0000","last_undegraded":"2024-09-21T05:55:33.638668+0000","last_fullsized":"2024-09-21T05:55:33.638668+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:28.259188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767914+0000","last_change":"2024-09-21T05:55:13.992043+0000","last_active":"2024-09-21T05:55:33.767914+0000","last_peered":"2024-09-21T05:55:33.767914+0000","last_clean":"2024-09-21T05:55:33.767914+0000","last_became_active":"2024-09-21T05:55:13.991344+0000","last_became_peered":"2024-09-21T05:55:13.991344+0000","last_unstale":"2024-09-21T05:55:33.767914+0000","last_undegraded":"2024-09-21T05:55:33.767914+0000","last_fullsized":"2024-09-21T05:55:33.767914+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:42:55.674333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639242+0000","last_change":"2024-09-21T05:55:13.993114+0000","last_active":"2024-09-21T05:55:33.639242+0000","last_peered":"2024-09-21T05:55:33.639242+0000","last_clean":"2024-09-21T05:55:33.639242+0000","last_became_active":"2024-09-21T05:55:13.992749+0000","last_became_peered":"2024-09-21T05:55:13.992749+0000","last_unstale":"2024-09-21T05:55:33.639242+0000","last_undegraded":"2024-09-21T05:55:33.639242+0000","last_fullsized":"2024-09-21T05:55:33.639242+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:15.926967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.640614+0000","last_change":"2024-09-21T05:55:13.997199+0000","last_active":"2024-09-21T05:55:33.640614+0000","last_peered":"2024-09-21T05:55:33.640614+0000","last_clean":"2024-09-21T05:55:33.640614+0000","last_became_active":"2024-09-21T05:55:13.996809+0000","last_became_peered":"2024-09-21T05:55:13.996809+0000","last_unstale":"2024-09-21T05:55:33.640614+0000","last_undegraded":"2024-09-21T05:55:33.640614+0000","last_fullsized":"2024-09-21T05:55:33.640614+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:16.681430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636906+0000","last_change":"2024-09-21T05:55:14.000879+0000","last_active":"2024-09-21T05:55:33.636906+0000","last_peered":"2024-09-21T05:55:33.636906+0000","last_clean":"2024-09-21T05:55:33.636906+0000","last_became_active":"2024-09-21T05:55:14.000259+0000","last_became_peered":"2024-09-21T05:55:14.000259+0000","last_unstale":"2024-09-21T05:55:33.636906+0000","last_undegraded":"2024-09-21T05:55:33.636906+0000","last_fullsized":"2024-09-21T05:55:33.636906+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:01:08.938417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636509+0000","last_change":"2024-09-21T05:55:13.989433+0000","last_active":"2024-09-21T05:55:33.636509+0000","last_peered":"2024-09-21T05:55:33.636509+0000","last_clean":"2024-09-21T05:55:33.636509+0000","last_became_active":"2024-09-21T05:55:13.989097+0000","last_became_peered":"2024-09-21T05:55:13.989097+0000","last_unstale":"2024-09-21T05:55:33.636509+0000","last_undegraded":"2024-09-21T05:55:33.636509+0000","last_fullsized":"2024-09-21T05:55:33.636509+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:37.835298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767185+0000","last_change":"2024-09-21T05:55:13.993911+0000","last_active":"2024-09-21T05:55:33.767185+0000","last_peered":"2024-09-21T05:55:33.767185+0000","last_clean":"2024-09-21T05:55:33.767185+0000","last_became_active":"2024-09-21T05:55:13.993570+0000","last_became_peered":"2024-09-21T05:55:13.993570+0000","last_unstale":"2024-09-21T05:55:33.767185+0000","last_undegraded":"2024-09-21T05:55:33.767185+0000","last_fullsized":"2024-09-21T05:55:33.767185+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:08:16.588457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639566+0000","last_change":"2024-09-21T05:55:13.992764+0000","last_active":"2024-09-21T05:55:33.639566+0000","last_peered":"2024-09-21T05:55:33.639566+0000","last_clean":"2024-09-21T05:55:33.639566+0000","last_became_active":"2024-09-21T05:55:13.992338+0000","last_became_peered":"2024-09-21T05:55:13.992338+0000","last_unstale":"2024-09-21T05:55:33.639566+0000","last_undegraded":"2024-09-21T05:55:33.639566+0000","last_fullsized":"2024-09-21T05:55:33.639566+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:25:29.997353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639811+0000","last_change":"2024-09-21T05:55:13.987119+0000","last_active":"2024-09-21T05:55:33.639811+0000","last_peered":"2024-09-21T05:55:33.639811+0000","last_clean":"2024-09-21T05:55:33.639811+0000","last_became_active":"2024-09-21T05:55:13.986663+0000","last_became_peered":"2024-09-21T05:55:13.986663+0000","last_unstale":"2024-09-21T05:55:33.639811+0000","last_undegraded":"2024-09-21T05:55:33.639811+0000","last_fullsized":"2024-09-21T05:55:33.639811+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:02.089833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.637197+0000","last_change":"2024-09-21T05:55:14.003577+0000","last_active":"2024-09-21T05:55:33.637197+0000","last_peered":"2024-09-21T05:55:33.637197+0000","last_clean":"2024-09-21T05:55:33.637197+0000","last_became_active":"2024-09-21T05:55:14.003162+0000","last_became_peered":"2024-09-21T05:55:14.003162+0000","last_unstale":"2024-09-21T05:55:33.637197+0000","last_undegraded":"2024-09-21T05:55:33.637197+0000","last_fullsized":"2024-09-21T05:55:33.637197+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:58.630843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635758+0000","last_change":"2024-09-21T05:55:13.992078+0000","last_active":"2024-09-21T05:55:33.635758+0000","last_peered":"2024-09-21T05:55:33.635758+0000","last_clean":"2024-09-21T05:55:33.635758+0000","last_became_active":"2024-09-21T05:55:13.991680+0000","last_became_peered":"2024-09-21T05:55:13.991680+0000","last_unstale":"2024-09-21T05:55:33.635758+0000","last_undegraded":"2024-09-21T05:55:33.635758+0000","last_fullsized":"2024-09-21T05:55:33.635758+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:52.886755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"74'6","reported_seq":81,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:56:13.295550+0000","last_change":"2024-09-21T05:55:11.982560+0000","last_active":"2024-09-21T05:56:13.295550+0000","last_peered":"2024-09-21T05:56:13.295550+0000","last_clean":"2024-09-21T05:56:13.295550+0000","last_became_active":"2024-09-21T05:55:11.982179+0000","last_became_peered":"2024-09-21T05:55:11.982179+0000","last_unstale":"2024-09-21T05:56:13.295550+0000","last_undegraded":"2024-09-21T05:56:13.295550+0000","last_fullsized":"2024-09-21T05:56:13.295550+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:06.012194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":52,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767640+0000","last_change":"2024-09-21T05:55:13.993994+0000","last_active":"2024-09-21T05:55:33.767640+0000","last_peered":"2024-09-21T05:55:33.767640+0000","last_clean":"2024-09-21T05:55:33.767640+0000","last_became_active":"2024-09-21T05:55:13.993585+0000","last_became_peered":"2024-09-21T05:55:13.993585+0000","last_unstale":"2024-09-21T05:55:33.767640+0000","last_undegraded":"2024-09-21T05:55:33.767640+0000","last_fullsized":"2024-09-21T05:55:33.767640+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:11.608616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"74'7","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.634233+0000","last_change":"2024-09-21T05:55:11.984397+0000","last_active":"2024-09-21T05:55:33.634233+0000","last_peered":"2024-09-21T05:55:33.634233+0000","last_clean":"2024-09-21T05:55:33.634233+0000","last_became_active":"2024-09-21T05:55:11.983888+0000","last_became_peered":"2024-09-21T05:55:11.983888+0000","last_unstale":"2024-09-21T05:55:33.634233+0000","last_undegraded":"2024-09-21T05:55:33.634233+0000","last_fullsized":"2024-09-21T05:55:33.634233+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:59.012710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636187+0000","last_change":"2024-09-21T05:55:11.974547+0000","last_active":"2024-09-21T05:55:33.636187+0000","last_peered":"2024-09-21T05:55:33.636187+0000","last_clean":"2024-09-21T05:55:33.636187+0000","last_became_active":"2024-09-21T05:55:11.974151+0000","last_became_peered":"2024-09-21T05:55:11.974151+0000","last_unstale":"2024-09-21T05:55:33.636187+0000","last_undegraded":"2024-09-21T05:55:33.636187+0000","last_fullsized":"2024-09-21T05:55:33.636187+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:51.489548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.638587+0000","last_change":"2024-09-21T05:55:13.999287+0000","last_active":"2024-09-21T05:55:33.638587+0000","last_peered":"2024-09-21T05:55:33.638587+0000","last_clean":"2024-09-21T05:55:33.638587+0000","last_became_active":"2024-09-21T05:55:13.998742+0000","last_became_peered":"2024-09-21T05:55:13.998742+0000","last_unstale":"2024-09-21T05:55:33.638587+0000","last_undegraded":"2024-09-21T05:55:33.638587+0000","last_fullsized":"2024-09-21T05:55:33.638587+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:33.842810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"74'3","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.768113+0000","last_change":"2024-09-21T05:55:11.981354+0000","last_active":"2024-09-21T05:55:33.768113+0000","last_peered":"2024-09-21T05:55:33.768113+0000","last_clean":"2024-09-21T05:55:33.768113+0000","last_became_active":"2024-09-21T05:55:11.980993+0000","last_became_peered":"2024-09-21T05:55:11.980993+0000","last_unstale":"2024-09-21T05:55:33.768113+0000","last_undegraded":"2024-09-21T05:55:33.768113+0000","last_fullsized":"2024-09-21T05:55:33.768113+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:05.037669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.633042+0000","last_change":"2024-09-21T05:55:14.001041+0000","last_active":"2024-09-21T05:55:33.633042+0000","last_peered":"2024-09-21T05:55:33.633042+0000","last_clean":"2024-09-21T05:55:33.633042+0000","last_became_active":"2024-09-21T05:55:14.000688+0000","last_became_peered":"2024-09-21T05:55:14.000688+0000","last_unstale":"2024-09-21T05:55:33.633042+0000","last_undegraded":"2024-09-21T05:55:33.633042+0000","last_fullsized":"2024-09-21T05:55:33.633042+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:51.190495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"74'4","reported_seq":131,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:56:17.301119+0000","last_change":"2024-09-21T05:55:11.984097+0000","last_active":"2024-09-21T05:56:17.301119+0000","last_peered":"2024-09-21T05:56:17.301119+0000","last_clean":"2024-09-21T05:56:17.301119+0000","last_became_active":"2024-09-21T05:55:11.982969+0000","last_became_peered":"2024-09-21T05:55:11.982969+0000","last_unstale":"2024-09-21T05:56:17.301119+0000","last_undegraded":"2024-09-21T05:56:17.301119+0000","last_fullsized":"2024-09-21T05:56:17.301119+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:41.372225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639897+0000","last_change":"2024-09-21T05:55:13.998433+0000","last_active":"2024-09-21T05:55:33.639897+0000","last_peered":"2024-09-21T05:55:33.639897+0000","last_clean":"2024-09-21T05:55:33.639897+0000","last_became_active":"2024-09-21T05:55:13.997890+0000","last_became_peered":"2024-09-21T05:55:13.997890+0000","last_unstale":"2024-09-21T05:55:33.639897+0000","last_undegraded":"2024-09-21T05:55:33.639897+0000","last_fullsized":"2024-09-21T05:55:33.639897+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:24.396789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"74'4","reported_seq":40,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.701709+0000","last_change":"2024-09-21T05:55:33.701599+0000","last_active":"2024-09-21T05:55:33.701709+0000","last_peered":"2024-09-21T05:55:33.701709+0000","last_clean":"2024-09-21T05:55:33.701709+0000","last_became_active":"2024-09-21T05:55:33.697385+0000","last_became_peered":"2024-09-21T05:55:33.697385+0000","last_unstale":"2024-09-21T05:55:33.701709+0000","last_undegraded":"2024-09-21T05:55:33.701709+0000","last_fullsized":"2024-09-21T05:55:33.701709+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:18.653991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767050+0000","last_change":"2024-09-21T05:55:13.995111+0000","last_active":"2024-09-21T05:55:33.767050+0000","last_peered":"2024-09-21T05:55:33.767050+0000","last_clean":"2024-09-21T05:55:33.767050+0000","last_became_active":"2024-09-21T05:55:13.994691+0000","last_became_peered":"2024-09-21T05:55:13.994691+0000","last_unstale":"2024-09-21T05:55:33.767050+0000","last_undegraded":"2024-09-21T05:55:33.767050+0000","last_fullsized":"2024-09-21T05:55:33.767050+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:30.167649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"74'5","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767919+0000","last_change":"2024-09-21T05:55:12.013961+0000","last_active":"2024-09-21T05:55:33.767919+0000","last_peered":"2024-09-21T05:55:33.767919+0000","last_clean":"2024-09-21T05:55:33.767919+0000","last_became_active":"2024-09-21T05:55:12.013573+0000","last_became_peered":"2024-09-21T05:55:12.013573+0000","last_unstale":"2024-09-21T05:55:33.767919+0000","last_undegraded":"2024-09-21T05:55:33.767919+0000","last_fullsized":"2024-09-21T05:55:33.767919+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:12.119605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639617+0000","last_change":"2024-09-21T05:55:13.996093+0000","last_active":"2024-09-21T05:55:33.639617+0000","last_peered":"2024-09-21T05:55:33.639617+0000","last_clean":"2024-09-21T05:55:33.639617+0000","last_became_active":"2024-09-21T05:55:13.995342+0000","last_became_peered":"2024-09-21T05:55:13.995342+0000","last_unstale":"2024-09-21T05:55:33.639617+0000","last_undegraded":"2024-09-21T05:55:33.639617+0000","last_fullsized":"2024-09-21T05:55:33.639617+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:20.335330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"76'30","reported_seq":166,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:56:17.300955+0000","last_change":"2024-09-21T05:55:11.986193+0000","last_active":"2024-09-21T05:56:17.300955+0000","last_peered":"2024-09-21T05:56:17.300955+0000","last_clean":"2024-09-21T05:56:17.300955+0000","last_became_active":"2024-09-21T05:55:11.985736+0000","last_became_peered":"2024-09-21T05:55:11.985736+0000","last_unstale":"2024-09-21T05:56:17.300955+0000","last_undegraded":"2024-09-21T05:56:17.300955+0000","last_fullsized":"2024-09-21T05:56:17.300955+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.503419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63902,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":113,"num_read_kb":118,"num_write":32,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636170+0000","last_change":"2024-09-21T05:55:14.001123+0000","last_active":"2024-09-21T05:55:33.636170+0000","last_peered":"2024-09-21T05:55:33.636170+0000","last_clean":"2024-09-21T05:55:33.636170+0000","last_became_active":"2024-09-21T05:55:14.000742+0000","last_became_peered":"2024-09-21T05:55:14.000742+0000","last_unstale":"2024-09-21T05:55:33.636170+0000","last_undegraded":"2024-09-21T05:55:33.636170+0000","last_fullsized":"2024-09-21T05:55:33.636170+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:15.277992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"74'9","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639957+0000","last_change":"2024-09-21T05:55:11.981396+0000","last_active":"2024-09-21T05:55:33.639957+0000","last_peered":"2024-09-21T05:55:33.639957+0000","last_clean":"2024-09-21T05:55:33.639957+0000","last_became_active":"2024-09-21T05:55:11.980976+0000","last_became_peered":"2024-09-21T05:55:11.980976+0000","last_unstale":"2024-09-21T05:55:33.639957+0000","last_undegraded":"2024-09-21T05:55:33.639957+0000","last_fullsized":"2024-09-21T05:55:33.639957+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:43.600453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636282+0000","last_change":"2024-09-21T05:55:13.991354+0000","last_active":"2024-09-21T05:55:33.636282+0000","last_peered":"2024-09-21T05:55:33.636282+0000","last_clean":"2024-09-21T05:55:33.636282+0000","last_became_active":"2024-09-21T05:55:13.990967+0000","last_became_peered":"2024-09-21T05:55:13.990967+0000","last_unstale":"2024-09-21T05:55:33.636282+0000","last_undegraded":"2024-09-21T05:55:33.636282+0000","last_fullsized":"2024-09-21T05:55:33.636282+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:04.889461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"74'5","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767638+0000","last_change":"2024-09-21T05:55:11.981019+0000","last_active":"2024-09-21T05:55:33.767638+0000","last_peered":"2024-09-21T05:55:33.767638+0000","last_clean":"2024-09-21T05:55:33.767638+0000","last_became_active":"2024-09-21T05:55:11.980619+0000","last_became_peered":"2024-09-21T05:55:11.980619+0000","last_unstale":"2024-09-21T05:55:33.767638+0000","last_undegraded":"2024-09-21T05:55:33.767638+0000","last_fullsized":"2024-09-21T05:55:33.767638+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:41.005092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635917+0000","last_change":"2024-09-21T05:55:13.994535+0000","last_active":"2024-09-21T05:55:33.635917+0000","last_peered":"2024-09-21T05:55:33.635917+0000","last_clean":"2024-09-21T05:55:33.635917+0000","last_became_active":"2024-09-21T05:55:13.994153+0000","last_became_peered":"2024-09-21T05:55:13.994153+0000","last_unstale":"2024-09-21T05:55:33.635917+0000","last_undegraded":"2024-09-21T05:55:33.635917+0000","last_fullsized":"2024-09-21T05:55:33.635917+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:02.301263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"74'6","reported_seq":81,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:56:13.295552+0000","last_change":"2024-09-21T05:55:11.977694+0000","last_active":"2024-09-21T05:56:13.295552+0000","last_peered":"2024-09-21T05:56:13.295552+0000","last_clean":"2024-09-21T05:56:13.295552+0000","last_became_active":"2024-09-21T05:55:11.977146+0000","last_became_peered":"2024-09-21T05:55:11.977146+0000","last_unstale":"2024-09-21T05:56:13.295552+0000","last_undegraded":"2024-09-21T05:56:13.295552+0000","last_fullsized":"2024-09-21T05:56:13.295552+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:29.736564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635721+0000","last_change":"2024-09-21T05:55:13.998032+0000","last_active":"2024-09-21T05:55:33.635721+0000","last_peered":"2024-09-21T05:55:33.635721+0000","last_clean":"2024-09-21T05:55:33.635721+0000","last_became_active":"2024-09-21T05:55:13.997617+0000","last_became_peered":"2024-09-21T05:55:13.997617+0000","last_unstale":"2024-09-21T05:55:33.635721+0000","last_undegraded":"2024-09-21T05:55:33.635721+0000","last_fullsized":"2024-09-21T05:55:33.635721+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:46.552726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"74'17","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635659+0000","last_change":"2024-09-21T05:55:12.013540+0000","last_active":"2024-09-21T05:55:33.635659+0000","last_peered":"2024-09-21T05:55:33.635659+0000","last_clean":"2024-09-21T05:55:33.635659+0000","last_became_active":"2024-09-21T05:55:12.013151+0000","last_became_peered":"2024-09-21T05:55:12.013151+0000","last_unstale":"2024-09-21T05:55:33.635659+0000","last_undegraded":"2024-09-21T05:55:33.635659+0000","last_fullsized":"2024-09-21T05:55:33.635659+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:53.705904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767640+0000","last_change":"2024-09-21T05:55:13.999501+0000","last_active":"2024-09-21T05:55:33.767640+0000","last_peered":"2024-09-21T05:55:33.767640+0000","last_clean":"2024-09-21T05:55:33.767640+0000","last_became_active":"2024-09-21T05:55:13.997819+0000","last_became_peered":"2024-09-21T05:55:13.997819+0000","last_unstale":"2024-09-21T05:55:33.767640+0000","last_undegraded":"2024-09-21T05:55:33.767640+0000","last_fullsized":"2024-09-21T05:55:33.767640+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:54.334755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"74'2","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636760+0000","last_change":"2024-09-21T05:55:11.984428+0000","last_active":"2024-09-21T05:55:33.636760+0000","last_peered":"2024-09-21T05:55:33.636760+0000","last_clean":"2024-09-21T05:55:33.636760+0000","last_became_active":"2024-09-21T05:55:11.983543+0000","last_became_peered":"2024-09-21T05:55:11.983543+0000","last_unstale":"2024-09-21T05:55:33.636760+0000","last_undegraded":"2024-09-21T05:55:33.636760+0000","last_fullsized":"2024-09-21T05:55:33.636760+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:39.636408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.637304+0000","last_change":"2024-09-21T05:55:14.003851+0000","last_active":"2024-09-21T05:55:33.637304+0000","last_peered":"2024-09-21T05:55:33.637304+0000","last_clean":"2024-09-21T05:55:33.637304+0000","last_became_active":"2024-09-21T05:55:14.003354+0000","last_became_peered":"2024-09-21T05:55:14.003354+0000","last_unstale":"2024-09-21T05:55:33.637304+0000","last_undegraded":"2024-09-21T05:55:33.637304+0000","last_fullsized":"2024-09-21T05:55:33.637304+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:01.321612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"74'5","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.645976+0000","last_change":"2024-09-21T05:55:33.645825+0000","last_active":"2024-09-21T05:55:33.645976+0000","last_peered":"2024-09-21T05:55:33.645976+0000","last_clean":"2024-09-21T05:55:33.645976+0000","last_became_active":"2024-09-21T05:55:33.640707+0000","last_became_peered":"2024-09-21T05:55:33.640707+0000","last_unstale":"2024-09-21T05:55:33.645976+0000","last_undegraded":"2024-09-21T05:55:33.645976+0000","last_fullsized":"2024-09-21T05:55:33.645976+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:33.665707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636452+0000","last_change":"2024-09-21T05:55:14.001855+0000","last_active":"2024-09-21T05:55:33.636452+0000","last_peered":"2024-09-21T05:55:33.636452+0000","last_clean":"2024-09-21T05:55:33.636452+0000","last_became_active":"2024-09-21T05:55:14.001484+0000","last_became_peered":"2024-09-21T05:55:14.001484+0000","last_unstale":"2024-09-21T05:55:33.636452+0000","last_undegraded":"2024-09-21T05:55:33.636452+0000","last_fullsized":"2024-09-21T05:55:33.636452+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:57.615334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'3","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767928+0000","last_change":"2024-09-21T05:55:11.979716+0000","last_active":"2024-09-21T05:55:33.767928+0000","last_peered":"2024-09-21T05:55:33.767928+0000","last_clean":"2024-09-21T05:55:33.767928+0000","last_became_active":"2024-09-21T05:55:11.979307+0000","last_became_peered":"2024-09-21T05:55:11.979307+0000","last_unstale":"2024-09-21T05:55:33.767928+0000","last_undegraded":"2024-09-21T05:55:33.767928+0000","last_fullsized":"2024-09-21T05:55:33.767928+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:30.019040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636060+0000","last_change":"2024-09-21T05:55:13.991769+0000","last_active":"2024-09-21T05:55:33.636060+0000","last_peered":"2024-09-21T05:55:33.636060+0000","last_clean":"2024-09-21T05:55:33.636060+0000","last_became_active":"2024-09-21T05:55:13.991345+0000","last_became_peered":"2024-09-21T05:55:13.991345+0000","last_unstale":"2024-09-21T05:55:33.636060+0000","last_undegraded":"2024-09-21T05:55:33.636060+0000","last_fullsized":"2024-09-21T05:55:33.636060+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:52.227126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"74'6","reported_seq":84,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:56:16.310634+0000","last_change":"2024-09-21T05:55:12.013836+0000","last_active":"2024-09-21T05:56:16.310634+0000","last_peered":"2024-09-21T05:56:16.310634+0000","last_clean":"2024-09-21T05:56:16.310634+0000","last_became_active":"2024-09-21T05:55:12.013436+0000","last_became_peered":"2024-09-21T05:55:12.013436+0000","last_unstale":"2024-09-21T05:56:16.310634+0000","last_undegraded":"2024-09-21T05:56:16.310634+0000","last_fullsized":"2024-09-21T05:56:16.310634+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:40:02.767620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636197+0000","last_change":"2024-09-21T05:55:14.001156+0000","last_active":"2024-09-21T05:55:33.636197+0000","last_peered":"2024-09-21T05:55:33.636197+0000","last_clean":"2024-09-21T05:55:33.636197+0000","last_became_active":"2024-09-21T05:55:14.000777+0000","last_became_peered":"2024-09-21T05:55:14.000777+0000","last_unstale":"2024-09-21T05:55:33.636197+0000","last_undegraded":"2024-09-21T05:55:33.636197+0000","last_fullsized":"2024-09-21T05:55:33.636197+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:46:08.000612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":97,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636093+0000","last_change":"2024-09-21T05:53:58.971355+0000","last_active":"2024-09-21T05:55:33.636093+0000","last_peered":"2024-09-21T05:55:33.636093+0000","last_clean":"2024-09-21T05:55:33.636093+0000","last_became_active":"2024-09-21T05:53:58.965825+0000","last_became_peered":"2024-09-21T05:53:58.965825+0000","last_unstale":"2024-09-21T05:55:33.636093+0000","last_undegraded":"2024-09-21T05:55:33.636093+0000","last_fullsized":"2024-09-21T05:55:33.636093+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:50:33.190160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:50:33.190160+0000","last_clean_scrub_stamp":"2024-09-21T05:50:33.190160+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:31.387339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"74'6","reported_seq":32,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.696020+0000","last_change":"2024-09-21T05:55:11.976092+0000","last_active":"2024-09-21T05:55:33.696020+0000","last_peered":"2024-09-21T05:55:33.696020+0000","last_clean":"2024-09-21T05:55:33.696020+0000","last_became_active":"2024-09-21T05:55:11.975657+0000","last_became_peered":"2024-09-21T05:55:11.975657+0000","last_unstale":"2024-09-21T05:55:33.696020+0000","last_undegraded":"2024-09-21T05:55:33.696020+0000","last_fullsized":"2024-09-21T05:55:33.696020+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:46.792054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639547+0000","last_change":"2024-09-21T05:55:13.997621+0000","last_active":"2024-09-21T05:55:33.639547+0000","last_peered":"2024-09-21T05:55:33.639547+0000","last_clean":"2024-09-21T05:55:33.639547+0000","last_became_active":"2024-09-21T05:55:13.997127+0000","last_became_peered":"2024-09-21T05:55:13.997127+0000","last_unstale":"2024-09-21T05:55:33.639547+0000","last_undegraded":"2024-09-21T05:55:33.639547+0000","last_fullsized":"2024-09-21T05:55:33.639547+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:32.399696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"74'4","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635928+0000","last_change":"2024-09-21T05:55:11.982387+0000","last_active":"2024-09-21T05:55:33.635928+0000","last_peered":"2024-09-21T05:55:33.635928+0000","last_clean":"2024-09-21T05:55:33.635928+0000","last_became_active":"2024-09-21T05:55:11.981851+0000","last_became_peered":"2024-09-21T05:55:11.981851+0000","last_unstale":"2024-09-21T05:55:33.635928+0000","last_undegraded":"2024-09-21T05:55:33.635928+0000","last_fullsized":"2024-09-21T05:55:33.635928+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:45.345738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636236+0000","last_change":"2024-09-21T05:55:13.981065+0000","last_active":"2024-09-21T05:55:33.636236+0000","last_peered":"2024-09-21T05:55:33.636236+0000","last_clean":"2024-09-21T05:55:33.636236+0000","last_became_active":"2024-09-21T05:55:13.980524+0000","last_became_peered":"2024-09-21T05:55:13.980524+0000","last_unstale":"2024-09-21T05:55:33.636236+0000","last_undegraded":"2024-09-21T05:55:33.636236+0000","last_fullsized":"2024-09-21T05:55:33.636236+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:43.322429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"74'5","reported_seq":79,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:56:14.296457+0000","last_change":"2024-09-21T05:55:12.013434+0000","last_active":"2024-09-21T05:56:14.296457+0000","last_peered":"2024-09-21T05:56:14.296457+0000","last_clean":"2024-09-21T05:56:14.296457+0000","last_became_active":"2024-09-21T05:55:12.013087+0000","last_became_peered":"2024-09-21T05:55:12.013087+0000","last_unstale":"2024-09-21T05:56:14.296457+0000","last_undegraded":"2024-09-21T05:56:14.296457+0000","last_fullsized":"2024-09-21T05:56:14.296457+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:19.876288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767927+0000","last_change":"2024-09-21T05:55:13.996102+0000","last_active":"2024-09-21T05:55:33.767927+0000","last_peered":"2024-09-21T05:55:33.767927+0000","last_clean":"2024-09-21T05:55:33.767927+0000","last_became_active":"2024-09-21T05:55:13.994900+0000","last_became_peered":"2024-09-21T05:55:13.994900+0000","last_unstale":"2024-09-21T05:55:33.767927+0000","last_undegraded":"2024-09-21T05:55:33.767927+0000","last_fullsized":"2024-09-21T05:55:33.767927+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:04.998577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"74'5","reported_seq":78,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:56:13.710364+0000","last_change":"2024-09-21T05:55:11.981184+0000","last_active":"2024-09-21T05:56:13.710364+0000","last_peered":"2024-09-21T05:56:13.710364+0000","last_clean":"2024-09-21T05:56:13.710364+0000","last_became_active":"2024-09-21T05:55:11.979581+0000","last_became_peered":"2024-09-21T05:55:11.979581+0000","last_unstale":"2024-09-21T05:56:13.710364+0000","last_undegraded":"2024-09-21T05:56:13.710364+0000","last_fullsized":"2024-09-21T05:56:13.710364+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:53.833860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.768241+0000","last_change":"2024-09-21T05:55:33.768136+0000","last_active":"2024-09-21T05:55:33.768241+0000","last_peered":"2024-09-21T05:55:33.768241+0000","last_clean":"2024-09-21T05:55:33.768241+0000","last_became_active":"2024-09-21T05:55:33.767697+0000","last_became_peered":"2024-09-21T05:55:33.767697+0000","last_unstale":"2024-09-21T05:55:33.768241+0000","last_undegraded":"2024-09-21T05:55:33.768241+0000","last_fullsized":"2024-09-21T05:55:33.768241+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:48.146453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"74'1","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.633171+0000","last_change":"2024-09-21T05:55:12.013121+0000","last_active":"2024-09-21T05:55:33.633171+0000","last_peered":"2024-09-21T05:55:33.633171+0000","last_clean":"2024-09-21T05:55:33.633171+0000","last_became_active":"2024-09-21T05:55:12.012657+0000","last_became_peered":"2024-09-21T05:55:12.012657+0000","last_unstale":"2024-09-21T05:55:33.633171+0000","last_undegraded":"2024-09-21T05:55:33.633171+0000","last_fullsized":"2024-09-21T05:55:33.633171+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:25:48.002938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"74'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.634506+0000","last_change":"2024-09-21T05:55:14.004660+0000","last_active":"2024-09-21T05:55:33.634506+0000","last_peered":"2024-09-21T05:55:33.634506+0000","last_clean":"2024-09-21T05:55:33.634506+0000","last_became_active":"2024-09-21T05:55:14.004297+0000","last_became_peered":"2024-09-21T05:55:14.004297+0000","last_unstale":"2024-09-21T05:55:33.634506+0000","last_undegraded":"2024-09-21T05:55:33.634506+0000","last_fullsized":"2024-09-21T05:55:33.634506+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:24.399891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"74'4","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636395+0000","last_change":"2024-09-21T05:55:12.015717+0000","last_active":"2024-09-21T05:55:33.636395+0000","last_peered":"2024-09-21T05:55:33.636395+0000","last_clean":"2024-09-21T05:55:33.636395+0000","last_became_active":"2024-09-21T05:55:12.015419+0000","last_became_peered":"2024-09-21T05:55:12.015419+0000","last_unstale":"2024-09-21T05:55:33.636395+0000","last_undegraded":"2024-09-21T05:55:33.636395+0000","last_fullsized":"2024-09-21T05:55:33.636395+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:57.988860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636635+0000","last_change":"2024-09-21T05:55:13.992460+0000","last_active":"2024-09-21T05:55:33.636635+0000","last_peered":"2024-09-21T05:55:33.636635+0000","last_clean":"2024-09-21T05:55:33.636635+0000","last_became_active":"2024-09-21T05:55:13.991887+0000","last_became_peered":"2024-09-21T05:55:13.991887+0000","last_unstale":"2024-09-21T05:55:33.636635+0000","last_undegraded":"2024-09-21T05:55:33.636635+0000","last_fullsized":"2024-09-21T05:55:33.636635+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:37.718050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"76'15","reported_seq":38,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:38.995033+0000","last_change":"2024-09-21T05:55:11.982235+0000","last_active":"2024-09-21T05:55:38.995033+0000","last_peered":"2024-09-21T05:55:38.995033+0000","last_clean":"2024-09-21T05:55:38.995033+0000","last_became_active":"2024-09-21T05:55:11.981846+0000","last_became_peered":"2024-09-21T05:55:11.981846+0000","last_unstale":"2024-09-21T05:55:38.995033+0000","last_undegraded":"2024-09-21T05:55:38.995033+0000","last_fullsized":"2024-09-21T05:55:38.995033+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:40:04.330346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639253+0000","last_change":"2024-09-21T05:55:13.999249+0000","last_active":"2024-09-21T05:55:33.639253+0000","last_peered":"2024-09-21T05:55:33.639253+0000","last_clean":"2024-09-21T05:55:33.639253+0000","last_became_active":"2024-09-21T05:55:13.998452+0000","last_became_peered":"2024-09-21T05:55:13.998452+0000","last_unstale":"2024-09-21T05:55:33.639253+0000","last_undegraded":"2024-09-21T05:55:33.639253+0000","last_fullsized":"2024-09-21T05:55:33.639253+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:59.364684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"74'7","reported_seq":31,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.696351+0000","last_change":"2024-09-21T05:55:11.971850+0000","last_active":"2024-09-21T05:55:33.696351+0000","last_peered":"2024-09-21T05:55:33.696351+0000","last_clean":"2024-09-21T05:55:33.696351+0000","last_became_active":"2024-09-21T05:55:11.971415+0000","last_became_peered":"2024-09-21T05:55:11.971415+0000","last_unstale":"2024-09-21T05:55:33.696351+0000","last_undegraded":"2024-09-21T05:55:33.696351+0000","last_fullsized":"2024-09-21T05:55:33.696351+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:51.990745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.766931+0000","last_change":"2024-09-21T05:55:13.994500+0000","last_active":"2024-09-21T05:55:33.766931+0000","last_peered":"2024-09-21T05:55:33.766931+0000","last_clean":"2024-09-21T05:55:33.766931+0000","last_became_active":"2024-09-21T05:55:13.994146+0000","last_became_peered":"2024-09-21T05:55:13.994146+0000","last_unstale":"2024-09-21T05:55:33.766931+0000","last_undegraded":"2024-09-21T05:55:33.766931+0000","last_fullsized":"2024-09-21T05:55:33.766931+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:42.934234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639268+0000","last_change":"2024-09-21T05:55:12.013479+0000","last_active":"2024-09-21T05:55:33.639268+0000","last_peered":"2024-09-21T05:55:33.639268+0000","last_clean":"2024-09-21T05:55:33.639268+0000","last_became_active":"2024-09-21T05:55:12.013180+0000","last_became_peered":"2024-09-21T05:55:12.013180+0000","last_unstale":"2024-09-21T05:55:33.639268+0000","last_undegraded":"2024-09-21T05:55:33.639268+0000","last_fullsized":"2024-09-21T05:55:33.639268+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:28.800072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635548+0000","last_change":"2024-09-21T05:55:14.001795+0000","last_active":"2024-09-21T05:55:33.635548+0000","last_peered":"2024-09-21T05:55:33.635548+0000","last_clean":"2024-09-21T05:55:33.635548+0000","last_became_active":"2024-09-21T05:55:14.001430+0000","last_became_peered":"2024-09-21T05:55:14.001430+0000","last_unstale":"2024-09-21T05:55:33.635548+0000","last_undegraded":"2024-09-21T05:55:33.635548+0000","last_fullsized":"2024-09-21T05:55:33.635548+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:10.339939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"74'7","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.638793+0000","last_change":"2024-09-21T05:55:11.976894+0000","last_active":"2024-09-21T05:55:33.638793+0000","last_peered":"2024-09-21T05:55:33.638793+0000","last_clean":"2024-09-21T05:55:33.638793+0000","last_became_active":"2024-09-21T05:55:11.976516+0000","last_became_peered":"2024-09-21T05:55:11.976516+0000","last_unstale":"2024-09-21T05:55:33.638793+0000","last_undegraded":"2024-09-21T05:55:33.638793+0000","last_fullsized":"2024-09-21T05:55:33.638793+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:58.726311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.637297+0000","last_change":"2024-09-21T05:55:14.003760+0000","last_active":"2024-09-21T05:55:33.637297+0000","last_peered":"2024-09-21T05:55:33.637297+0000","last_clean":"2024-09-21T05:55:33.637297+0000","last_became_active":"2024-09-21T05:55:14.003180+0000","last_became_peered":"2024-09-21T05:55:14.003180+0000","last_unstale":"2024-09-21T05:55:33.637297+0000","last_undegraded":"2024-09-21T05:55:33.637297+0000","last_fullsized":"2024-09-21T05:55:33.637297+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:25.077765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639713+0000","last_change":"2024-09-21T05:55:11.983865+0000","last_active":"2024-09-21T05:55:33.639713+0000","last_peered":"2024-09-21T05:55:33.639713+0000","last_clean":"2024-09-21T05:55:33.639713+0000","last_became_active":"2024-09-21T05:55:11.983473+0000","last_became_peered":"2024-09-21T05:55:11.983473+0000","last_unstale":"2024-09-21T05:55:33.639713+0000","last_undegraded":"2024-09-21T05:55:33.639713+0000","last_fullsized":"2024-09-21T05:55:33.639713+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:18.767317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639006+0000","last_change":"2024-09-21T05:55:13.999001+0000","last_active":"2024-09-21T05:55:33.639006+0000","last_peered":"2024-09-21T05:55:33.639006+0000","last_clean":"2024-09-21T05:55:33.639006+0000","last_became_active":"2024-09-21T05:55:13.998656+0000","last_became_peered":"2024-09-21T05:55:13.998656+0000","last_unstale":"2024-09-21T05:55:33.639006+0000","last_undegraded":"2024-09-21T05:55:33.639006+0000","last_fullsized":"2024-09-21T05:55:33.639006+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:50.913635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"74'4","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.646791+0000","last_change":"2024-09-21T05:55:33.646124+0000","last_active":"2024-09-21T05:55:33.646791+0000","last_peered":"2024-09-21T05:55:33.646791+0000","last_clean":"2024-09-21T05:55:33.646791+0000","last_became_active":"2024-09-21T05:55:33.641142+0000","last_became_peered":"2024-09-21T05:55:33.641142+0000","last_unstale":"2024-09-21T05:55:33.646791+0000","last_undegraded":"2024-09-21T05:55:33.646791+0000","last_fullsized":"2024-09-21T05:55:33.646791+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:36.341473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635915+0000","last_change":"2024-09-21T05:55:13.997494+0000","last_active":"2024-09-21T05:55:33.635915+0000","last_peered":"2024-09-21T05:55:33.635915+0000","last_clean":"2024-09-21T05:55:33.635915+0000","last_became_active":"2024-09-21T05:55:13.997138+0000","last_became_peered":"2024-09-21T05:55:13.997138+0000","last_unstale":"2024-09-21T05:55:33.635915+0000","last_undegraded":"2024-09-21T05:55:33.635915+0000","last_fullsized":"2024-09-21T05:55:33.635915+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:02:01.777126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767713+0000","last_change":"2024-09-21T05:55:14.002750+0000","last_active":"2024-09-21T05:55:33.767713+0000","last_peered":"2024-09-21T05:55:33.767713+0000","last_clean":"2024-09-21T05:55:33.767713+0000","last_became_active":"2024-09-21T05:55:14.002150+0000","last_became_peered":"2024-09-21T05:55:14.002150+0000","last_unstale":"2024-09-21T05:55:33.767713+0000","last_undegraded":"2024-09-21T05:55:33.767713+0000","last_fullsized":"2024-09-21T05:55:33.767713+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:49.603133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"74'5","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636032+0000","last_change":"2024-09-21T05:55:11.982366+0000","last_active":"2024-09-21T05:55:33.636032+0000","last_peered":"2024-09-21T05:55:33.636032+0000","last_clean":"2024-09-21T05:55:33.636032+0000","last_became_active":"2024-09-21T05:55:11.981994+0000","last_became_peered":"2024-09-21T05:55:11.981994+0000","last_unstale":"2024-09-21T05:55:33.636032+0000","last_undegraded":"2024-09-21T05:55:33.636032+0000","last_fullsized":"2024-09-21T05:55:33.636032+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:14.626758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639407+0000","last_change":"2024-09-21T05:55:14.008204+0000","last_active":"2024-09-21T05:55:33.639407+0000","last_peered":"2024-09-21T05:55:33.639407+0000","last_clean":"2024-09-21T05:55:33.639407+0000","last_became_active":"2024-09-21T05:55:14.007800+0000","last_became_peered":"2024-09-21T05:55:14.007800+0000","last_unstale":"2024-09-21T05:55:33.639407+0000","last_undegraded":"2024-09-21T05:55:33.639407+0000","last_fullsized":"2024-09-21T05:55:33.639407+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:49.760850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"74'12","reported_seq":36,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636283+0000","last_change":"2024-09-21T05:55:11.986168+0000","last_active":"2024-09-21T05:55:33.636283+0000","last_peered":"2024-09-21T05:55:33.636283+0000","last_clean":"2024-09-21T05:55:33.636283+0000","last_became_active":"2024-09-21T05:55:11.985694+0000","last_became_peered":"2024-09-21T05:55:11.985694+0000","last_unstale":"2024-09-21T05:55:33.636283+0000","last_undegraded":"2024-09-21T05:55:33.636283+0000","last_fullsized":"2024-09-21T05:55:33.636283+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:44.863935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767026+0000","last_change":"2024-09-21T05:55:14.002459+0000","last_active":"2024-09-21T05:55:33.767026+0000","last_peered":"2024-09-21T05:55:33.767026+0000","last_clean":"2024-09-21T05:55:33.767026+0000","last_became_active":"2024-09-21T05:55:14.001962+0000","last_became_peered":"2024-09-21T05:55:14.001962+0000","last_unstale":"2024-09-21T05:55:33.767026+0000","last_undegraded":"2024-09-21T05:55:33.767026+0000","last_fullsized":"2024-09-21T05:55:33.767026+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:53:26.892772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639340+0000","last_change":"2024-09-21T05:55:12.014149+0000","last_active":"2024-09-21T05:55:33.639340+0000","last_peered":"2024-09-21T05:55:33.639340+0000","last_clean":"2024-09-21T05:55:33.639340+0000","last_became_active":"2024-09-21T05:55:12.013789+0000","last_became_peered":"2024-09-21T05:55:12.013789+0000","last_unstale":"2024-09-21T05:55:33.639340+0000","last_undegraded":"2024-09-21T05:55:33.639340+0000","last_fullsized":"2024-09-21T05:55:33.639340+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:18.304023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639186+0000","last_change":"2024-09-21T05:55:13.993500+0000","last_active":"2024-09-21T05:55:33.639186+0000","last_peered":"2024-09-21T05:55:33.639186+0000","last_clean":"2024-09-21T05:55:33.639186+0000","last_became_active":"2024-09-21T05:55:13.993104+0000","last_became_peered":"2024-09-21T05:55:13.993104+0000","last_unstale":"2024-09-21T05:55:33.639186+0000","last_undegraded":"2024-09-21T05:55:33.639186+0000","last_fullsized":"2024-09-21T05:55:33.639186+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:01.445119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"74'13","reported_seq":108,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:56:14.711079+0000","last_change":"2024-09-21T05:55:33.650207+0000","last_active":"2024-09-21T05:56:14.711079+0000","last_peered":"2024-09-21T05:56:14.711079+0000","last_clean":"2024-09-21T05:56:14.711079+0000","last_became_active":"2024-09-21T05:55:33.638388+0000","last_became_peered":"2024-09-21T05:55:33.638388+0000","last_unstale":"2024-09-21T05:56:14.711079+0000","last_undegraded":"2024-09-21T05:56:14.711079+0000","last_fullsized":"2024-09-21T05:56:14.711079+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:51.478063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":46,"num_read_kb":46,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71497,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":542,"num_read_kb":348,"num_write":245,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":214491,"data_compressed":5682,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281769,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":68,"seq":292057776154,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":839680,"data_stored":412559,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939679,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":1066818,"data_compressed":43475,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103204,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":856064,"data_stored":411426,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299434,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":411804,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495662,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":897024,"data_stored":1004041,"data_compressed":41581,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659188,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":839680,"data_stored":411143,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822712,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":917504,"data_stored":1004000,"data_compressed":41581,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986238,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":839680,"data_stored":411395,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":856064,"data_stored":410968,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378697,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":476960,"data_compressed":7646,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542220,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":475958,"data_compressed":7646,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705747,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":860160,"data_stored":412873,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34208,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34208,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34208,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35829,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35829,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35829,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35829,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35829,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":65654,"data_compressed":1894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":66656,"data_compressed":1894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":66029,"data_compressed":1894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:56:20.580 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T05:56:20.836 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:21.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.403+0000 7fe4eee8d640 1 -- 172.21.15.117:0/3046571633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4e8108420 msgr2=0x7fe4e8108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:21.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.403+0000 7fe4eee8d640 1 --2- 172.21.15.117:0/3046571633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4e8108420 0x7fe4e8108830 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009900 tx=0x7fe4d402a410 comp rx=0 tx=0).stop 2024-09-21T05:56:21.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.404+0000 7fe4eee8d640 1 -- 172.21.15.117:0/3046571633 shutdown_connections 2024-09-21T05:56:21.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.404+0000 7fe4eee8d640 1 --2- 172.21.15.117:0/3046571633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4e8108420 0x7fe4e8108830 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:21.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.404+0000 7fe4eee8d640 1 --2- 172.21.15.117:0/3046571633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4e8108d70 0x7fe4e810f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:21.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.404+0000 7fe4eee8d640 1 --2- 172.21.15.117:0/3046571633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4e8102ae0 0x7fe4e8102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:21.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.404+0000 7fe4eee8d640 1 -- 172.21.15.117:0/3046571633 >> 172.21.15.117:0/3046571633 conn(0x7fe4e8079770 msgr2=0x7fe4e8079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:21.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.404+0000 7fe4eee8d640 1 -- 172.21.15.117:0/3046571633 shutdown_connections 2024-09-21T05:56:21.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.404+0000 7fe4eee8d640 1 -- 172.21.15.117:0/3046571633 wait complete. 2024-09-21T05:56:21.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.404+0000 7fe4eee8d640 1 Processor -- start 2024-09-21T05:56:21.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.404+0000 7fe4eee8d640 1 -- start start 2024-09-21T05:56:21.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.404+0000 7fe4eee8d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4e8102ae0 0x7fe4e819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:21.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.404+0000 7fe4eee8d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4e8108420 0x7fe4e819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:21.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.404+0000 7fe4eee8d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4e8108d70 0x7fe4e819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:21.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.405+0000 7fe4eee8d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e8112a30 con 0x7fe4e8108d70 2024-09-21T05:56:21.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.405+0000 7fe4eee8d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e81128b0 con 0x7fe4e8102ae0 2024-09-21T05:56:21.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.405+0000 7fe4eee8d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e8112bb0 con 0x7fe4e8108420 2024-09-21T05:56:21.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.405+0000 7fe4ed42a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4e8108d70 0x7fe4e819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:21.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.405+0000 7fe4ed42a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4e8108d70 0x7fe4e819fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40160/0 (socket says 172.21.15.117:40160) 2024-09-21T05:56:21.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.405+0000 7fe4ed42a640 1 -- 172.21.15.117:0/544448026 learned_addr learned my addr 172.21.15.117:0/544448026 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:21.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.405+0000 7fe4ecc29640 1 --2- 172.21.15.117:0/544448026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4e8102ae0 0x7fe4e819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:21.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.405+0000 7fe4e7fff640 1 --2- 172.21.15.117:0/544448026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4e8108420 0x7fe4e819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:21.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.405+0000 7fe4ed42a640 1 -- 172.21.15.117:0/544448026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4e8108420 msgr2=0x7fe4e819aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:21.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.405+0000 7fe4ed42a640 1 --2- 172.21.15.117:0/544448026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4e8108420 0x7fe4e819aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:21.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.405+0000 7fe4ed42a640 1 -- 172.21.15.117:0/544448026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4e8102ae0 msgr2=0x7fe4e819a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:21.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.405+0000 7fe4ed42a640 1 --2- 172.21.15.117:0/544448026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4e8102ae0 0x7fe4e819a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:21.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.405+0000 7fe4ed42a640 1 -- 172.21.15.117:0/544448026 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e81a03a0 con 0x7fe4e8108d70 2024-09-21T05:56:21.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.405+0000 7fe4ed42a640 1 --2- 172.21.15.117:0/544448026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4e8108d70 0x7fe4e819fd30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe4e000cce0 tx=0x7fe4e000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:21.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.405+0000 7fe4e5ffb640 1 -- 172.21.15.117:0/544448026 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e000f450 con 0x7fe4e8108d70 2024-09-21T05:56:21.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.405+0000 7fe4eee8d640 1 -- 172.21.15.117:0/544448026 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4e81a0690 con 0x7fe4e8108d70 2024-09-21T05:56:21.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.405+0000 7fe4e5ffb640 1 -- 172.21.15.117:0/544448026 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4e000faa0 con 0x7fe4e8108d70 2024-09-21T05:56:21.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.405+0000 7fe4e5ffb640 1 -- 172.21.15.117:0/544448026 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0018a80 con 0x7fe4e8108d70 2024-09-21T05:56:21.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.406+0000 7fe4eee8d640 1 -- 172.21.15.117:0/544448026 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4e81a0b50 con 0x7fe4e8108d70 2024-09-21T05:56:21.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.407+0000 7fe4eee8d640 1 -- 172.21.15.117:0/544448026 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4e8103d60 con 0x7fe4e8108d70 2024-09-21T05:56:21.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.408+0000 7fe4e5ffb640 1 -- 172.21.15.117:0/544448026 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7fe4e000fc50 con 0x7fe4e8108d70 2024-09-21T05:56:21.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.409+0000 7fe4e5ffb640 1 --2- 172.21.15.117:0/544448026 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe4d80b6620 0x7fe4d80b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:21.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.409+0000 7fe4ecc29640 1 --2- 172.21.15.117:0/544448026 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe4d80b6620 0x7fe4d80b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:21.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.409+0000 7fe4e5ffb640 1 -- 172.21.15.117:0/544448026 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fe4e00e79e0 con 0x7fe4e8108d70 2024-09-21T05:56:21.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.409+0000 7fe4ecc29640 1 --2- 172.21.15.117:0/544448026 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe4d80b6620 0x7fe4d80b8b30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00abe0 tx=0x7fe4dc00a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:21.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.413+0000 7fe4e5ffb640 1 -- 172.21.15.117:0/544448026 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe4e0014030 con 0x7fe4e8108d70 2024-09-21T05:56:21.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.589+0000 7fe4eee8d640 1 -- 172.21.15.117:0/544448026 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe4e8063800 con 0x7fe4d80b6620 2024-09-21T05:56:21.594 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.593+0000 7fe4e5ffb640 1 -- 172.21.15.117:0/544448026 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259045 (secure 0 0 0) 0x7fe4e8063800 con 0x7fe4d80b6620 2024-09-21T05:56:21.597 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:56:21.598 INFO:teuthology.orchestra.run.smithi117.stderr:dumped all 2024-09-21T05:56:21.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.597+0000 7fe4eee8d640 1 -- 172.21.15.117:0/544448026 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe4d80b6620 msgr2=0x7fe4d80b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:21.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.597+0000 7fe4eee8d640 1 --2- 172.21.15.117:0/544448026 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe4d80b6620 0x7fe4d80b8b30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00abe0 tx=0x7fe4dc00a2d0 comp rx=0 tx=0).stop 2024-09-21T05:56:21.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.597+0000 7fe4eee8d640 1 -- 172.21.15.117:0/544448026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4e8108d70 msgr2=0x7fe4e819fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:21.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.597+0000 7fe4eee8d640 1 --2- 172.21.15.117:0/544448026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4e8108d70 0x7fe4e819fd30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe4e000cce0 tx=0x7fe4e000a620 comp rx=0 tx=0).stop 2024-09-21T05:56:21.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.597+0000 7fe4eee8d640 1 -- 172.21.15.117:0/544448026 shutdown_connections 2024-09-21T05:56:21.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.597+0000 7fe4eee8d640 1 --2- 172.21.15.117:0/544448026 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe4d80b6620 0x7fe4d80b8b30 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:21.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.597+0000 7fe4eee8d640 1 --2- 172.21.15.117:0/544448026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4e8108d70 0x7fe4e819fd30 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:21.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.597+0000 7fe4eee8d640 1 --2- 172.21.15.117:0/544448026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4e8108420 0x7fe4e819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:21.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.597+0000 7fe4eee8d640 1 --2- 172.21.15.117:0/544448026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4e8102ae0 0x7fe4e819a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:21.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.597+0000 7fe4eee8d640 1 -- 172.21.15.117:0/544448026 >> 172.21.15.117:0/544448026 conn(0x7fe4e8079770 msgr2=0x7fe4e8101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:21.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.597+0000 7fe4eee8d640 1 -- 172.21.15.117:0/544448026 shutdown_connections 2024-09-21T05:56:21.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:21.597+0000 7fe4eee8d640 1 -- 172.21.15.117:0/544448026 wait complete. 2024-09-21T05:56:21.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:21 smithi192 ceph-mon[34700]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T05:56:21.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:21 smithi192 ceph-mon[34700]: pgmap v298: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T05:56:21.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:21 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:56:21.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:21 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:56:21.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:21 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:56:21.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:21 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:56:21.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:21 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:56:22.016 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:21 smithi117 ceph-mon[30947]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T05:56:22.016 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:21 smithi117 ceph-mon[30947]: pgmap v298: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T05:56:22.016 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:21 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:56:22.016 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:21 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:56:22.016 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:21 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:56:22.016 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:21 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:56:22.017 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:21 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:56:22.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:21 smithi138 ceph-mon[34391]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T05:56:22.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:21 smithi138 ceph-mon[34391]: pgmap v298: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T05:56:22.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:21 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:56:22.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:21 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:56:22.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:21 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:56:22.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:21 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:56:22.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:21 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:56:22.300 INFO:teuthology.orchestra.run.smithi117.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2024-09-21T05:56:20.521612+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662099,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":633,"num_read_kb":417,"num_write":370,"num_write_kb":2294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":2952512,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":363760,"kb_used_data":10320,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124578064,"statfs":{"total":1151940427776,"available":1151567937536,"internally_reserved":0,"allocated":10567680,"data_stored":6909945,"data_compressed":182193,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003560"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636773+0000","last_change":"2024-09-21T05:55:14.007642+0000","last_active":"2024-09-21T05:55:33.636773+0000","last_peered":"2024-09-21T05:55:33.636773+0000","last_clean":"2024-09-21T05:55:33.636773+0000","last_became_active":"2024-09-21T05:55:14.006943+0000","last_became_peered":"2024-09-21T05:55:14.006943+0000","last_unstale":"2024-09-21T05:55:33.636773+0000","last_undegraded":"2024-09-21T05:55:33.636773+0000","last_fullsized":"2024-09-21T05:55:33.636773+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:23:53.816079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767047+0000","last_change":"2024-09-21T05:55:14.002034+0000","last_active":"2024-09-21T05:55:33.767047+0000","last_peered":"2024-09-21T05:55:33.767047+0000","last_clean":"2024-09-21T05:55:33.767047+0000","last_became_active":"2024-09-21T05:55:14.001658+0000","last_became_peered":"2024-09-21T05:55:14.001658+0000","last_unstale":"2024-09-21T05:55:33.767047+0000","last_undegraded":"2024-09-21T05:55:33.767047+0000","last_fullsized":"2024-09-21T05:55:33.767047+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:01.568362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.766840+0000","last_change":"2024-09-21T05:55:13.996901+0000","last_active":"2024-09-21T05:55:33.766840+0000","last_peered":"2024-09-21T05:55:33.766840+0000","last_clean":"2024-09-21T05:55:33.766840+0000","last_became_active":"2024-09-21T05:55:13.996020+0000","last_became_peered":"2024-09-21T05:55:13.996020+0000","last_unstale":"2024-09-21T05:55:33.766840+0000","last_undegraded":"2024-09-21T05:55:33.766840+0000","last_fullsized":"2024-09-21T05:55:33.766840+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:38.771926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.696132+0000","last_change":"2024-09-21T05:55:13.995968+0000","last_active":"2024-09-21T05:55:33.696132+0000","last_peered":"2024-09-21T05:55:33.696132+0000","last_clean":"2024-09-21T05:55:33.696132+0000","last_became_active":"2024-09-21T05:55:13.995259+0000","last_became_peered":"2024-09-21T05:55:13.995259+0000","last_unstale":"2024-09-21T05:55:33.696132+0000","last_undegraded":"2024-09-21T05:55:33.696132+0000","last_fullsized":"2024-09-21T05:55:33.696132+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:48.953342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635823+0000","last_change":"2024-09-21T05:55:13.995778+0000","last_active":"2024-09-21T05:55:33.635823+0000","last_peered":"2024-09-21T05:55:33.635823+0000","last_clean":"2024-09-21T05:55:33.635823+0000","last_became_active":"2024-09-21T05:55:13.995415+0000","last_became_peered":"2024-09-21T05:55:13.995415+0000","last_unstale":"2024-09-21T05:55:33.635823+0000","last_undegraded":"2024-09-21T05:55:33.635823+0000","last_fullsized":"2024-09-21T05:55:33.635823+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:52.562120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.634110+0000","last_change":"2024-09-21T05:55:14.000005+0000","last_active":"2024-09-21T05:55:33.634110+0000","last_peered":"2024-09-21T05:55:33.634110+0000","last_clean":"2024-09-21T05:55:33.634110+0000","last_became_active":"2024-09-21T05:55:13.999493+0000","last_became_peered":"2024-09-21T05:55:13.999493+0000","last_unstale":"2024-09-21T05:55:33.634110+0000","last_undegraded":"2024-09-21T05:55:33.634110+0000","last_fullsized":"2024-09-21T05:55:33.634110+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:29.426368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635816+0000","last_change":"2024-09-21T05:55:13.993004+0000","last_active":"2024-09-21T05:55:33.635816+0000","last_peered":"2024-09-21T05:55:33.635816+0000","last_clean":"2024-09-21T05:55:33.635816+0000","last_became_active":"2024-09-21T05:55:13.992615+0000","last_became_peered":"2024-09-21T05:55:13.992615+0000","last_unstale":"2024-09-21T05:55:33.635816+0000","last_undegraded":"2024-09-21T05:55:33.635816+0000","last_fullsized":"2024-09-21T05:55:33.635816+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:48.324618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.769380+0000","last_change":"2024-09-21T05:55:33.769006+0000","last_active":"2024-09-21T05:55:33.769380+0000","last_peered":"2024-09-21T05:55:33.769380+0000","last_clean":"2024-09-21T05:55:33.769380+0000","last_became_active":"2024-09-21T05:55:33.768705+0000","last_became_peered":"2024-09-21T05:55:33.768705+0000","last_unstale":"2024-09-21T05:55:33.769380+0000","last_undegraded":"2024-09-21T05:55:33.769380+0000","last_fullsized":"2024-09-21T05:55:33.769380+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:14.013300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639798+0000","last_change":"2024-09-21T05:55:14.004131+0000","last_active":"2024-09-21T05:55:33.639798+0000","last_peered":"2024-09-21T05:55:33.639798+0000","last_clean":"2024-09-21T05:55:33.639798+0000","last_became_active":"2024-09-21T05:55:14.003621+0000","last_became_peered":"2024-09-21T05:55:14.003621+0000","last_unstale":"2024-09-21T05:55:33.639798+0000","last_undegraded":"2024-09-21T05:55:33.639798+0000","last_fullsized":"2024-09-21T05:55:33.639798+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:00.694211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.643388+0000","last_change":"2024-09-21T05:55:33.643289+0000","last_active":"2024-09-21T05:55:33.643388+0000","last_peered":"2024-09-21T05:55:33.643388+0000","last_clean":"2024-09-21T05:55:33.643388+0000","last_became_active":"2024-09-21T05:55:33.642805+0000","last_became_peered":"2024-09-21T05:55:33.642805+0000","last_unstale":"2024-09-21T05:55:33.643388+0000","last_undegraded":"2024-09-21T05:55:33.643388+0000","last_fullsized":"2024-09-21T05:55:33.643388+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:00.854712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636225+0000","last_change":"2024-09-21T05:55:14.006467+0000","last_active":"2024-09-21T05:55:33.636225+0000","last_peered":"2024-09-21T05:55:33.636225+0000","last_clean":"2024-09-21T05:55:33.636225+0000","last_became_active":"2024-09-21T05:55:14.005978+0000","last_became_peered":"2024-09-21T05:55:14.005978+0000","last_unstale":"2024-09-21T05:55:33.636225+0000","last_undegraded":"2024-09-21T05:55:33.636225+0000","last_fullsized":"2024-09-21T05:55:33.636225+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:37.391152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.696241+0000","last_change":"2024-09-21T05:55:14.003972+0000","last_active":"2024-09-21T05:55:33.696241+0000","last_peered":"2024-09-21T05:55:33.696241+0000","last_clean":"2024-09-21T05:55:33.696241+0000","last_became_active":"2024-09-21T05:55:14.003601+0000","last_became_peered":"2024-09-21T05:55:14.003601+0000","last_unstale":"2024-09-21T05:55:33.696241+0000","last_undegraded":"2024-09-21T05:55:33.696241+0000","last_fullsized":"2024-09-21T05:55:33.696241+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:16.201087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636463+0000","last_change":"2024-09-21T05:55:13.996051+0000","last_active":"2024-09-21T05:55:33.636463+0000","last_peered":"2024-09-21T05:55:33.636463+0000","last_clean":"2024-09-21T05:55:33.636463+0000","last_became_active":"2024-09-21T05:55:13.995679+0000","last_became_peered":"2024-09-21T05:55:13.995679+0000","last_unstale":"2024-09-21T05:55:33.636463+0000","last_undegraded":"2024-09-21T05:55:33.636463+0000","last_fullsized":"2024-09-21T05:55:33.636463+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:45:44.372029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.766681+0000","last_change":"2024-09-21T05:55:13.995421+0000","last_active":"2024-09-21T05:55:33.766681+0000","last_peered":"2024-09-21T05:55:33.766681+0000","last_clean":"2024-09-21T05:55:33.766681+0000","last_became_active":"2024-09-21T05:55:13.995006+0000","last_became_peered":"2024-09-21T05:55:13.995006+0000","last_unstale":"2024-09-21T05:55:33.766681+0000","last_undegraded":"2024-09-21T05:55:33.766681+0000","last_fullsized":"2024-09-21T05:55:33.766681+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:00.636555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.766677+0000","last_change":"2024-09-21T05:55:13.993466+0000","last_active":"2024-09-21T05:55:33.766677+0000","last_peered":"2024-09-21T05:55:33.766677+0000","last_clean":"2024-09-21T05:55:33.766677+0000","last_became_active":"2024-09-21T05:55:13.993137+0000","last_became_peered":"2024-09-21T05:55:13.993137+0000","last_unstale":"2024-09-21T05:55:33.766677+0000","last_undegraded":"2024-09-21T05:55:33.766677+0000","last_fullsized":"2024-09-21T05:55:33.766677+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:33.218868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639111+0000","last_change":"2024-09-21T05:55:13.996316+0000","last_active":"2024-09-21T05:55:33.639111+0000","last_peered":"2024-09-21T05:55:33.639111+0000","last_clean":"2024-09-21T05:55:33.639111+0000","last_became_active":"2024-09-21T05:55:13.995918+0000","last_became_peered":"2024-09-21T05:55:13.995918+0000","last_unstale":"2024-09-21T05:55:33.639111+0000","last_undegraded":"2024-09-21T05:55:33.639111+0000","last_fullsized":"2024-09-21T05:55:33.639111+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:45:54.408504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636962+0000","last_change":"2024-09-21T05:55:14.007198+0000","last_active":"2024-09-21T05:55:33.636962+0000","last_peered":"2024-09-21T05:55:33.636962+0000","last_clean":"2024-09-21T05:55:33.636962+0000","last_became_active":"2024-09-21T05:55:14.006700+0000","last_became_peered":"2024-09-21T05:55:14.006700+0000","last_unstale":"2024-09-21T05:55:33.636962+0000","last_undegraded":"2024-09-21T05:55:33.636962+0000","last_fullsized":"2024-09-21T05:55:33.636962+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:45.343778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.633333+0000","last_change":"2024-09-21T05:55:14.000152+0000","last_active":"2024-09-21T05:55:33.633333+0000","last_peered":"2024-09-21T05:55:33.633333+0000","last_clean":"2024-09-21T05:55:33.633333+0000","last_became_active":"2024-09-21T05:55:13.999786+0000","last_became_peered":"2024-09-21T05:55:13.999786+0000","last_unstale":"2024-09-21T05:55:33.633333+0000","last_undegraded":"2024-09-21T05:55:33.633333+0000","last_fullsized":"2024-09-21T05:55:33.633333+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:43:25.713410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.640106+0000","last_change":"2024-09-21T05:55:13.988367+0000","last_active":"2024-09-21T05:55:33.640106+0000","last_peered":"2024-09-21T05:55:33.640106+0000","last_clean":"2024-09-21T05:55:33.640106+0000","last_became_active":"2024-09-21T05:55:13.987982+0000","last_became_peered":"2024-09-21T05:55:13.987982+0000","last_unstale":"2024-09-21T05:55:33.640106+0000","last_undegraded":"2024-09-21T05:55:33.640106+0000","last_fullsized":"2024-09-21T05:55:33.640106+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:15.395155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636251+0000","last_change":"2024-09-21T05:55:14.004962+0000","last_active":"2024-09-21T05:55:33.636251+0000","last_peered":"2024-09-21T05:55:33.636251+0000","last_clean":"2024-09-21T05:55:33.636251+0000","last_became_active":"2024-09-21T05:55:14.004573+0000","last_became_peered":"2024-09-21T05:55:14.004573+0000","last_unstale":"2024-09-21T05:55:33.636251+0000","last_undegraded":"2024-09-21T05:55:33.636251+0000","last_fullsized":"2024-09-21T05:55:33.636251+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:59.014017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636468+0000","last_change":"2024-09-21T05:55:13.991267+0000","last_active":"2024-09-21T05:55:33.636468+0000","last_peered":"2024-09-21T05:55:33.636468+0000","last_clean":"2024-09-21T05:55:33.636468+0000","last_became_active":"2024-09-21T05:55:13.990916+0000","last_became_peered":"2024-09-21T05:55:13.990916+0000","last_unstale":"2024-09-21T05:55:33.636468+0000","last_undegraded":"2024-09-21T05:55:33.636468+0000","last_fullsized":"2024-09-21T05:55:33.636468+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:06:03.904632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639504+0000","last_change":"2024-09-21T05:55:13.994147+0000","last_active":"2024-09-21T05:55:33.639504+0000","last_peered":"2024-09-21T05:55:33.639504+0000","last_clean":"2024-09-21T05:55:33.639504+0000","last_became_active":"2024-09-21T05:55:13.993794+0000","last_became_peered":"2024-09-21T05:55:13.993794+0000","last_unstale":"2024-09-21T05:55:33.639504+0000","last_undegraded":"2024-09-21T05:55:33.639504+0000","last_fullsized":"2024-09-21T05:55:33.639504+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:19.970853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.638668+0000","last_change":"2024-09-21T05:55:14.001119+0000","last_active":"2024-09-21T05:55:33.638668+0000","last_peered":"2024-09-21T05:55:33.638668+0000","last_clean":"2024-09-21T05:55:33.638668+0000","last_became_active":"2024-09-21T05:55:14.000799+0000","last_became_peered":"2024-09-21T05:55:14.000799+0000","last_unstale":"2024-09-21T05:55:33.638668+0000","last_undegraded":"2024-09-21T05:55:33.638668+0000","last_fullsized":"2024-09-21T05:55:33.638668+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:28.259188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767914+0000","last_change":"2024-09-21T05:55:13.992043+0000","last_active":"2024-09-21T05:55:33.767914+0000","last_peered":"2024-09-21T05:55:33.767914+0000","last_clean":"2024-09-21T05:55:33.767914+0000","last_became_active":"2024-09-21T05:55:13.991344+0000","last_became_peered":"2024-09-21T05:55:13.991344+0000","last_unstale":"2024-09-21T05:55:33.767914+0000","last_undegraded":"2024-09-21T05:55:33.767914+0000","last_fullsized":"2024-09-21T05:55:33.767914+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:42:55.674333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639242+0000","last_change":"2024-09-21T05:55:13.993114+0000","last_active":"2024-09-21T05:55:33.639242+0000","last_peered":"2024-09-21T05:55:33.639242+0000","last_clean":"2024-09-21T05:55:33.639242+0000","last_became_active":"2024-09-21T05:55:13.992749+0000","last_became_peered":"2024-09-21T05:55:13.992749+0000","last_unstale":"2024-09-21T05:55:33.639242+0000","last_undegraded":"2024-09-21T05:55:33.639242+0000","last_fullsized":"2024-09-21T05:55:33.639242+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:15.926967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.640614+0000","last_change":"2024-09-21T05:55:13.997199+0000","last_active":"2024-09-21T05:55:33.640614+0000","last_peered":"2024-09-21T05:55:33.640614+0000","last_clean":"2024-09-21T05:55:33.640614+0000","last_became_active":"2024-09-21T05:55:13.996809+0000","last_became_peered":"2024-09-21T05:55:13.996809+0000","last_unstale":"2024-09-21T05:55:33.640614+0000","last_undegraded":"2024-09-21T05:55:33.640614+0000","last_fullsized":"2024-09-21T05:55:33.640614+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:16.681430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636906+0000","last_change":"2024-09-21T05:55:14.000879+0000","last_active":"2024-09-21T05:55:33.636906+0000","last_peered":"2024-09-21T05:55:33.636906+0000","last_clean":"2024-09-21T05:55:33.636906+0000","last_became_active":"2024-09-21T05:55:14.000259+0000","last_became_peered":"2024-09-21T05:55:14.000259+0000","last_unstale":"2024-09-21T05:55:33.636906+0000","last_undegraded":"2024-09-21T05:55:33.636906+0000","last_fullsized":"2024-09-21T05:55:33.636906+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:01:08.938417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636509+0000","last_change":"2024-09-21T05:55:13.989433+0000","last_active":"2024-09-21T05:55:33.636509+0000","last_peered":"2024-09-21T05:55:33.636509+0000","last_clean":"2024-09-21T05:55:33.636509+0000","last_became_active":"2024-09-21T05:55:13.989097+0000","last_became_peered":"2024-09-21T05:55:13.989097+0000","last_unstale":"2024-09-21T05:55:33.636509+0000","last_undegraded":"2024-09-21T05:55:33.636509+0000","last_fullsized":"2024-09-21T05:55:33.636509+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:37.835298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767185+0000","last_change":"2024-09-21T05:55:13.993911+0000","last_active":"2024-09-21T05:55:33.767185+0000","last_peered":"2024-09-21T05:55:33.767185+0000","last_clean":"2024-09-21T05:55:33.767185+0000","last_became_active":"2024-09-21T05:55:13.993570+0000","last_became_peered":"2024-09-21T05:55:13.993570+0000","last_unstale":"2024-09-21T05:55:33.767185+0000","last_undegraded":"2024-09-21T05:55:33.767185+0000","last_fullsized":"2024-09-21T05:55:33.767185+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:08:16.588457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639566+0000","last_change":"2024-09-21T05:55:13.992764+0000","last_active":"2024-09-21T05:55:33.639566+0000","last_peered":"2024-09-21T05:55:33.639566+0000","last_clean":"2024-09-21T05:55:33.639566+0000","last_became_active":"2024-09-21T05:55:13.992338+0000","last_became_peered":"2024-09-21T05:55:13.992338+0000","last_unstale":"2024-09-21T05:55:33.639566+0000","last_undegraded":"2024-09-21T05:55:33.639566+0000","last_fullsized":"2024-09-21T05:55:33.639566+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:25:29.997353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639811+0000","last_change":"2024-09-21T05:55:13.987119+0000","last_active":"2024-09-21T05:55:33.639811+0000","last_peered":"2024-09-21T05:55:33.639811+0000","last_clean":"2024-09-21T05:55:33.639811+0000","last_became_active":"2024-09-21T05:55:13.986663+0000","last_became_peered":"2024-09-21T05:55:13.986663+0000","last_unstale":"2024-09-21T05:55:33.639811+0000","last_undegraded":"2024-09-21T05:55:33.639811+0000","last_fullsized":"2024-09-21T05:55:33.639811+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:02.089833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.637197+0000","last_change":"2024-09-21T05:55:14.003577+0000","last_active":"2024-09-21T05:55:33.637197+0000","last_peered":"2024-09-21T05:55:33.637197+0000","last_clean":"2024-09-21T05:55:33.637197+0000","last_became_active":"2024-09-21T05:55:14.003162+0000","last_became_peered":"2024-09-21T05:55:14.003162+0000","last_unstale":"2024-09-21T05:55:33.637197+0000","last_undegraded":"2024-09-21T05:55:33.637197+0000","last_fullsized":"2024-09-21T05:55:33.637197+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:58.630843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635758+0000","last_change":"2024-09-21T05:55:13.992078+0000","last_active":"2024-09-21T05:55:33.635758+0000","last_peered":"2024-09-21T05:55:33.635758+0000","last_clean":"2024-09-21T05:55:33.635758+0000","last_became_active":"2024-09-21T05:55:13.991680+0000","last_became_peered":"2024-09-21T05:55:13.991680+0000","last_unstale":"2024-09-21T05:55:33.635758+0000","last_undegraded":"2024-09-21T05:55:33.635758+0000","last_fullsized":"2024-09-21T05:55:33.635758+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:52.886755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"74'6","reported_seq":86,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:56:18.302864+0000","last_change":"2024-09-21T05:55:11.982560+0000","last_active":"2024-09-21T05:56:18.302864+0000","last_peered":"2024-09-21T05:56:18.302864+0000","last_clean":"2024-09-21T05:56:18.302864+0000","last_became_active":"2024-09-21T05:55:11.982179+0000","last_became_peered":"2024-09-21T05:55:11.982179+0000","last_unstale":"2024-09-21T05:56:18.302864+0000","last_undegraded":"2024-09-21T05:56:18.302864+0000","last_fullsized":"2024-09-21T05:56:18.302864+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:06.012194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767640+0000","last_change":"2024-09-21T05:55:13.993994+0000","last_active":"2024-09-21T05:55:33.767640+0000","last_peered":"2024-09-21T05:55:33.767640+0000","last_clean":"2024-09-21T05:55:33.767640+0000","last_became_active":"2024-09-21T05:55:13.993585+0000","last_became_peered":"2024-09-21T05:55:13.993585+0000","last_unstale":"2024-09-21T05:55:33.767640+0000","last_undegraded":"2024-09-21T05:55:33.767640+0000","last_fullsized":"2024-09-21T05:55:33.767640+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:11.608616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"74'7","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.634233+0000","last_change":"2024-09-21T05:55:11.984397+0000","last_active":"2024-09-21T05:55:33.634233+0000","last_peered":"2024-09-21T05:55:33.634233+0000","last_clean":"2024-09-21T05:55:33.634233+0000","last_became_active":"2024-09-21T05:55:11.983888+0000","last_became_peered":"2024-09-21T05:55:11.983888+0000","last_unstale":"2024-09-21T05:55:33.634233+0000","last_undegraded":"2024-09-21T05:55:33.634233+0000","last_fullsized":"2024-09-21T05:55:33.634233+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:59.012710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636187+0000","last_change":"2024-09-21T05:55:11.974547+0000","last_active":"2024-09-21T05:55:33.636187+0000","last_peered":"2024-09-21T05:55:33.636187+0000","last_clean":"2024-09-21T05:55:33.636187+0000","last_became_active":"2024-09-21T05:55:11.974151+0000","last_became_peered":"2024-09-21T05:55:11.974151+0000","last_unstale":"2024-09-21T05:55:33.636187+0000","last_undegraded":"2024-09-21T05:55:33.636187+0000","last_fullsized":"2024-09-21T05:55:33.636187+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:51.489548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.638587+0000","last_change":"2024-09-21T05:55:13.999287+0000","last_active":"2024-09-21T05:55:33.638587+0000","last_peered":"2024-09-21T05:55:33.638587+0000","last_clean":"2024-09-21T05:55:33.638587+0000","last_became_active":"2024-09-21T05:55:13.998742+0000","last_became_peered":"2024-09-21T05:55:13.998742+0000","last_unstale":"2024-09-21T05:55:33.638587+0000","last_undegraded":"2024-09-21T05:55:33.638587+0000","last_fullsized":"2024-09-21T05:55:33.638587+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:33.842810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"74'3","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.768113+0000","last_change":"2024-09-21T05:55:11.981354+0000","last_active":"2024-09-21T05:55:33.768113+0000","last_peered":"2024-09-21T05:55:33.768113+0000","last_clean":"2024-09-21T05:55:33.768113+0000","last_became_active":"2024-09-21T05:55:11.980993+0000","last_became_peered":"2024-09-21T05:55:11.980993+0000","last_unstale":"2024-09-21T05:55:33.768113+0000","last_undegraded":"2024-09-21T05:55:33.768113+0000","last_fullsized":"2024-09-21T05:55:33.768113+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:05.037669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.633042+0000","last_change":"2024-09-21T05:55:14.001041+0000","last_active":"2024-09-21T05:55:33.633042+0000","last_peered":"2024-09-21T05:55:33.633042+0000","last_clean":"2024-09-21T05:55:33.633042+0000","last_became_active":"2024-09-21T05:55:14.000688+0000","last_became_peered":"2024-09-21T05:55:14.000688+0000","last_unstale":"2024-09-21T05:55:33.633042+0000","last_undegraded":"2024-09-21T05:55:33.633042+0000","last_fullsized":"2024-09-21T05:55:33.633042+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:51.190495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"74'4","reported_seq":131,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:56:17.301119+0000","last_change":"2024-09-21T05:55:11.984097+0000","last_active":"2024-09-21T05:56:17.301119+0000","last_peered":"2024-09-21T05:56:17.301119+0000","last_clean":"2024-09-21T05:56:17.301119+0000","last_became_active":"2024-09-21T05:55:11.982969+0000","last_became_peered":"2024-09-21T05:55:11.982969+0000","last_unstale":"2024-09-21T05:56:17.301119+0000","last_undegraded":"2024-09-21T05:56:17.301119+0000","last_fullsized":"2024-09-21T05:56:17.301119+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:41.372225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639897+0000","last_change":"2024-09-21T05:55:13.998433+0000","last_active":"2024-09-21T05:55:33.639897+0000","last_peered":"2024-09-21T05:55:33.639897+0000","last_clean":"2024-09-21T05:55:33.639897+0000","last_became_active":"2024-09-21T05:55:13.997890+0000","last_became_peered":"2024-09-21T05:55:13.997890+0000","last_unstale":"2024-09-21T05:55:33.639897+0000","last_undegraded":"2024-09-21T05:55:33.639897+0000","last_fullsized":"2024-09-21T05:55:33.639897+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:24.396789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"74'4","reported_seq":40,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.701709+0000","last_change":"2024-09-21T05:55:33.701599+0000","last_active":"2024-09-21T05:55:33.701709+0000","last_peered":"2024-09-21T05:55:33.701709+0000","last_clean":"2024-09-21T05:55:33.701709+0000","last_became_active":"2024-09-21T05:55:33.697385+0000","last_became_peered":"2024-09-21T05:55:33.697385+0000","last_unstale":"2024-09-21T05:55:33.701709+0000","last_undegraded":"2024-09-21T05:55:33.701709+0000","last_fullsized":"2024-09-21T05:55:33.701709+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:18.653991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767050+0000","last_change":"2024-09-21T05:55:13.995111+0000","last_active":"2024-09-21T05:55:33.767050+0000","last_peered":"2024-09-21T05:55:33.767050+0000","last_clean":"2024-09-21T05:55:33.767050+0000","last_became_active":"2024-09-21T05:55:13.994691+0000","last_became_peered":"2024-09-21T05:55:13.994691+0000","last_unstale":"2024-09-21T05:55:33.767050+0000","last_undegraded":"2024-09-21T05:55:33.767050+0000","last_fullsized":"2024-09-21T05:55:33.767050+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:30.167649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"74'5","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767919+0000","last_change":"2024-09-21T05:55:12.013961+0000","last_active":"2024-09-21T05:55:33.767919+0000","last_peered":"2024-09-21T05:55:33.767919+0000","last_clean":"2024-09-21T05:55:33.767919+0000","last_became_active":"2024-09-21T05:55:12.013573+0000","last_became_peered":"2024-09-21T05:55:12.013573+0000","last_unstale":"2024-09-21T05:55:33.767919+0000","last_undegraded":"2024-09-21T05:55:33.767919+0000","last_fullsized":"2024-09-21T05:55:33.767919+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:12.119605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639617+0000","last_change":"2024-09-21T05:55:13.996093+0000","last_active":"2024-09-21T05:55:33.639617+0000","last_peered":"2024-09-21T05:55:33.639617+0000","last_clean":"2024-09-21T05:55:33.639617+0000","last_became_active":"2024-09-21T05:55:13.995342+0000","last_became_peered":"2024-09-21T05:55:13.995342+0000","last_unstale":"2024-09-21T05:55:33.639617+0000","last_undegraded":"2024-09-21T05:55:33.639617+0000","last_fullsized":"2024-09-21T05:55:33.639617+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:20.335330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"76'30","reported_seq":166,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:56:17.300955+0000","last_change":"2024-09-21T05:55:11.986193+0000","last_active":"2024-09-21T05:56:17.300955+0000","last_peered":"2024-09-21T05:56:17.300955+0000","last_clean":"2024-09-21T05:56:17.300955+0000","last_became_active":"2024-09-21T05:55:11.985736+0000","last_became_peered":"2024-09-21T05:55:11.985736+0000","last_unstale":"2024-09-21T05:56:17.300955+0000","last_undegraded":"2024-09-21T05:56:17.300955+0000","last_fullsized":"2024-09-21T05:56:17.300955+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.503419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63902,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":113,"num_read_kb":118,"num_write":32,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636170+0000","last_change":"2024-09-21T05:55:14.001123+0000","last_active":"2024-09-21T05:55:33.636170+0000","last_peered":"2024-09-21T05:55:33.636170+0000","last_clean":"2024-09-21T05:55:33.636170+0000","last_became_active":"2024-09-21T05:55:14.000742+0000","last_became_peered":"2024-09-21T05:55:14.000742+0000","last_unstale":"2024-09-21T05:55:33.636170+0000","last_undegraded":"2024-09-21T05:55:33.636170+0000","last_fullsized":"2024-09-21T05:55:33.636170+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:15.277992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"74'9","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639957+0000","last_change":"2024-09-21T05:55:11.981396+0000","last_active":"2024-09-21T05:55:33.639957+0000","last_peered":"2024-09-21T05:55:33.639957+0000","last_clean":"2024-09-21T05:55:33.639957+0000","last_became_active":"2024-09-21T05:55:11.980976+0000","last_became_peered":"2024-09-21T05:55:11.980976+0000","last_unstale":"2024-09-21T05:55:33.639957+0000","last_undegraded":"2024-09-21T05:55:33.639957+0000","last_fullsized":"2024-09-21T05:55:33.639957+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:43.600453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636282+0000","last_change":"2024-09-21T05:55:13.991354+0000","last_active":"2024-09-21T05:55:33.636282+0000","last_peered":"2024-09-21T05:55:33.636282+0000","last_clean":"2024-09-21T05:55:33.636282+0000","last_became_active":"2024-09-21T05:55:13.990967+0000","last_became_peered":"2024-09-21T05:55:13.990967+0000","last_unstale":"2024-09-21T05:55:33.636282+0000","last_undegraded":"2024-09-21T05:55:33.636282+0000","last_fullsized":"2024-09-21T05:55:33.636282+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:04.889461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"74'5","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767638+0000","last_change":"2024-09-21T05:55:11.981019+0000","last_active":"2024-09-21T05:55:33.767638+0000","last_peered":"2024-09-21T05:55:33.767638+0000","last_clean":"2024-09-21T05:55:33.767638+0000","last_became_active":"2024-09-21T05:55:11.980619+0000","last_became_peered":"2024-09-21T05:55:11.980619+0000","last_unstale":"2024-09-21T05:55:33.767638+0000","last_undegraded":"2024-09-21T05:55:33.767638+0000","last_fullsized":"2024-09-21T05:55:33.767638+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:41.005092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635917+0000","last_change":"2024-09-21T05:55:13.994535+0000","last_active":"2024-09-21T05:55:33.635917+0000","last_peered":"2024-09-21T05:55:33.635917+0000","last_clean":"2024-09-21T05:55:33.635917+0000","last_became_active":"2024-09-21T05:55:13.994153+0000","last_became_peered":"2024-09-21T05:55:13.994153+0000","last_unstale":"2024-09-21T05:55:33.635917+0000","last_undegraded":"2024-09-21T05:55:33.635917+0000","last_fullsized":"2024-09-21T05:55:33.635917+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:02.301263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"74'6","reported_seq":86,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:56:18.302938+0000","last_change":"2024-09-21T05:55:11.977694+0000","last_active":"2024-09-21T05:56:18.302938+0000","last_peered":"2024-09-21T05:56:18.302938+0000","last_clean":"2024-09-21T05:56:18.302938+0000","last_became_active":"2024-09-21T05:55:11.977146+0000","last_became_peered":"2024-09-21T05:55:11.977146+0000","last_unstale":"2024-09-21T05:56:18.302938+0000","last_undegraded":"2024-09-21T05:56:18.302938+0000","last_fullsized":"2024-09-21T05:56:18.302938+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:29.736564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":58,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635721+0000","last_change":"2024-09-21T05:55:13.998032+0000","last_active":"2024-09-21T05:55:33.635721+0000","last_peered":"2024-09-21T05:55:33.635721+0000","last_clean":"2024-09-21T05:55:33.635721+0000","last_became_active":"2024-09-21T05:55:13.997617+0000","last_became_peered":"2024-09-21T05:55:13.997617+0000","last_unstale":"2024-09-21T05:55:33.635721+0000","last_undegraded":"2024-09-21T05:55:33.635721+0000","last_fullsized":"2024-09-21T05:55:33.635721+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:46.552726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"74'17","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635659+0000","last_change":"2024-09-21T05:55:12.013540+0000","last_active":"2024-09-21T05:55:33.635659+0000","last_peered":"2024-09-21T05:55:33.635659+0000","last_clean":"2024-09-21T05:55:33.635659+0000","last_became_active":"2024-09-21T05:55:12.013151+0000","last_became_peered":"2024-09-21T05:55:12.013151+0000","last_unstale":"2024-09-21T05:55:33.635659+0000","last_undegraded":"2024-09-21T05:55:33.635659+0000","last_fullsized":"2024-09-21T05:55:33.635659+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:53.705904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767640+0000","last_change":"2024-09-21T05:55:13.999501+0000","last_active":"2024-09-21T05:55:33.767640+0000","last_peered":"2024-09-21T05:55:33.767640+0000","last_clean":"2024-09-21T05:55:33.767640+0000","last_became_active":"2024-09-21T05:55:13.997819+0000","last_became_peered":"2024-09-21T05:55:13.997819+0000","last_unstale":"2024-09-21T05:55:33.767640+0000","last_undegraded":"2024-09-21T05:55:33.767640+0000","last_fullsized":"2024-09-21T05:55:33.767640+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:54.334755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"74'2","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636760+0000","last_change":"2024-09-21T05:55:11.984428+0000","last_active":"2024-09-21T05:55:33.636760+0000","last_peered":"2024-09-21T05:55:33.636760+0000","last_clean":"2024-09-21T05:55:33.636760+0000","last_became_active":"2024-09-21T05:55:11.983543+0000","last_became_peered":"2024-09-21T05:55:11.983543+0000","last_unstale":"2024-09-21T05:55:33.636760+0000","last_undegraded":"2024-09-21T05:55:33.636760+0000","last_fullsized":"2024-09-21T05:55:33.636760+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:39.636408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.637304+0000","last_change":"2024-09-21T05:55:14.003851+0000","last_active":"2024-09-21T05:55:33.637304+0000","last_peered":"2024-09-21T05:55:33.637304+0000","last_clean":"2024-09-21T05:55:33.637304+0000","last_became_active":"2024-09-21T05:55:14.003354+0000","last_became_peered":"2024-09-21T05:55:14.003354+0000","last_unstale":"2024-09-21T05:55:33.637304+0000","last_undegraded":"2024-09-21T05:55:33.637304+0000","last_fullsized":"2024-09-21T05:55:33.637304+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:01.321612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"74'5","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.645976+0000","last_change":"2024-09-21T05:55:33.645825+0000","last_active":"2024-09-21T05:55:33.645976+0000","last_peered":"2024-09-21T05:55:33.645976+0000","last_clean":"2024-09-21T05:55:33.645976+0000","last_became_active":"2024-09-21T05:55:33.640707+0000","last_became_peered":"2024-09-21T05:55:33.640707+0000","last_unstale":"2024-09-21T05:55:33.645976+0000","last_undegraded":"2024-09-21T05:55:33.645976+0000","last_fullsized":"2024-09-21T05:55:33.645976+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:33.665707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636452+0000","last_change":"2024-09-21T05:55:14.001855+0000","last_active":"2024-09-21T05:55:33.636452+0000","last_peered":"2024-09-21T05:55:33.636452+0000","last_clean":"2024-09-21T05:55:33.636452+0000","last_became_active":"2024-09-21T05:55:14.001484+0000","last_became_peered":"2024-09-21T05:55:14.001484+0000","last_unstale":"2024-09-21T05:55:33.636452+0000","last_undegraded":"2024-09-21T05:55:33.636452+0000","last_fullsized":"2024-09-21T05:55:33.636452+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:57.615334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'3","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767928+0000","last_change":"2024-09-21T05:55:11.979716+0000","last_active":"2024-09-21T05:55:33.767928+0000","last_peered":"2024-09-21T05:55:33.767928+0000","last_clean":"2024-09-21T05:55:33.767928+0000","last_became_active":"2024-09-21T05:55:11.979307+0000","last_became_peered":"2024-09-21T05:55:11.979307+0000","last_unstale":"2024-09-21T05:55:33.767928+0000","last_undegraded":"2024-09-21T05:55:33.767928+0000","last_fullsized":"2024-09-21T05:55:33.767928+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:30.019040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636060+0000","last_change":"2024-09-21T05:55:13.991769+0000","last_active":"2024-09-21T05:55:33.636060+0000","last_peered":"2024-09-21T05:55:33.636060+0000","last_clean":"2024-09-21T05:55:33.636060+0000","last_became_active":"2024-09-21T05:55:13.991345+0000","last_became_peered":"2024-09-21T05:55:13.991345+0000","last_unstale":"2024-09-21T05:55:33.636060+0000","last_undegraded":"2024-09-21T05:55:33.636060+0000","last_fullsized":"2024-09-21T05:55:33.636060+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:52.227126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"74'6","reported_seq":84,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:56:16.310634+0000","last_change":"2024-09-21T05:55:12.013836+0000","last_active":"2024-09-21T05:56:16.310634+0000","last_peered":"2024-09-21T05:56:16.310634+0000","last_clean":"2024-09-21T05:56:16.310634+0000","last_became_active":"2024-09-21T05:55:12.013436+0000","last_became_peered":"2024-09-21T05:55:12.013436+0000","last_unstale":"2024-09-21T05:56:16.310634+0000","last_undegraded":"2024-09-21T05:56:16.310634+0000","last_fullsized":"2024-09-21T05:56:16.310634+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:40:02.767620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636197+0000","last_change":"2024-09-21T05:55:14.001156+0000","last_active":"2024-09-21T05:55:33.636197+0000","last_peered":"2024-09-21T05:55:33.636197+0000","last_clean":"2024-09-21T05:55:33.636197+0000","last_became_active":"2024-09-21T05:55:14.000777+0000","last_became_peered":"2024-09-21T05:55:14.000777+0000","last_unstale":"2024-09-21T05:55:33.636197+0000","last_undegraded":"2024-09-21T05:55:33.636197+0000","last_fullsized":"2024-09-21T05:55:33.636197+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:46:08.000612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":97,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636093+0000","last_change":"2024-09-21T05:53:58.971355+0000","last_active":"2024-09-21T05:55:33.636093+0000","last_peered":"2024-09-21T05:55:33.636093+0000","last_clean":"2024-09-21T05:55:33.636093+0000","last_became_active":"2024-09-21T05:53:58.965825+0000","last_became_peered":"2024-09-21T05:53:58.965825+0000","last_unstale":"2024-09-21T05:55:33.636093+0000","last_undegraded":"2024-09-21T05:55:33.636093+0000","last_fullsized":"2024-09-21T05:55:33.636093+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:50:33.190160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:50:33.190160+0000","last_clean_scrub_stamp":"2024-09-21T05:50:33.190160+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:31.387339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"74'6","reported_seq":32,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.696020+0000","last_change":"2024-09-21T05:55:11.976092+0000","last_active":"2024-09-21T05:55:33.696020+0000","last_peered":"2024-09-21T05:55:33.696020+0000","last_clean":"2024-09-21T05:55:33.696020+0000","last_became_active":"2024-09-21T05:55:11.975657+0000","last_became_peered":"2024-09-21T05:55:11.975657+0000","last_unstale":"2024-09-21T05:55:33.696020+0000","last_undegraded":"2024-09-21T05:55:33.696020+0000","last_fullsized":"2024-09-21T05:55:33.696020+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:46.792054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639547+0000","last_change":"2024-09-21T05:55:13.997621+0000","last_active":"2024-09-21T05:55:33.639547+0000","last_peered":"2024-09-21T05:55:33.639547+0000","last_clean":"2024-09-21T05:55:33.639547+0000","last_became_active":"2024-09-21T05:55:13.997127+0000","last_became_peered":"2024-09-21T05:55:13.997127+0000","last_unstale":"2024-09-21T05:55:33.639547+0000","last_undegraded":"2024-09-21T05:55:33.639547+0000","last_fullsized":"2024-09-21T05:55:33.639547+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:32.399696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"74'4","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635928+0000","last_change":"2024-09-21T05:55:11.982387+0000","last_active":"2024-09-21T05:55:33.635928+0000","last_peered":"2024-09-21T05:55:33.635928+0000","last_clean":"2024-09-21T05:55:33.635928+0000","last_became_active":"2024-09-21T05:55:11.981851+0000","last_became_peered":"2024-09-21T05:55:11.981851+0000","last_unstale":"2024-09-21T05:55:33.635928+0000","last_undegraded":"2024-09-21T05:55:33.635928+0000","last_fullsized":"2024-09-21T05:55:33.635928+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:45.345738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636236+0000","last_change":"2024-09-21T05:55:13.981065+0000","last_active":"2024-09-21T05:55:33.636236+0000","last_peered":"2024-09-21T05:55:33.636236+0000","last_clean":"2024-09-21T05:55:33.636236+0000","last_became_active":"2024-09-21T05:55:13.980524+0000","last_became_peered":"2024-09-21T05:55:13.980524+0000","last_unstale":"2024-09-21T05:55:33.636236+0000","last_undegraded":"2024-09-21T05:55:33.636236+0000","last_fullsized":"2024-09-21T05:55:33.636236+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:43.322429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"74'5","reported_seq":79,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:56:14.296457+0000","last_change":"2024-09-21T05:55:12.013434+0000","last_active":"2024-09-21T05:56:14.296457+0000","last_peered":"2024-09-21T05:56:14.296457+0000","last_clean":"2024-09-21T05:56:14.296457+0000","last_became_active":"2024-09-21T05:55:12.013087+0000","last_became_peered":"2024-09-21T05:55:12.013087+0000","last_unstale":"2024-09-21T05:56:14.296457+0000","last_undegraded":"2024-09-21T05:56:14.296457+0000","last_fullsized":"2024-09-21T05:56:14.296457+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:19.876288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767927+0000","last_change":"2024-09-21T05:55:13.996102+0000","last_active":"2024-09-21T05:55:33.767927+0000","last_peered":"2024-09-21T05:55:33.767927+0000","last_clean":"2024-09-21T05:55:33.767927+0000","last_became_active":"2024-09-21T05:55:13.994900+0000","last_became_peered":"2024-09-21T05:55:13.994900+0000","last_unstale":"2024-09-21T05:55:33.767927+0000","last_undegraded":"2024-09-21T05:55:33.767927+0000","last_fullsized":"2024-09-21T05:55:33.767927+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:04.998577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"74'5","reported_seq":78,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:56:13.710364+0000","last_change":"2024-09-21T05:55:11.981184+0000","last_active":"2024-09-21T05:56:13.710364+0000","last_peered":"2024-09-21T05:56:13.710364+0000","last_clean":"2024-09-21T05:56:13.710364+0000","last_became_active":"2024-09-21T05:55:11.979581+0000","last_became_peered":"2024-09-21T05:55:11.979581+0000","last_unstale":"2024-09-21T05:56:13.710364+0000","last_undegraded":"2024-09-21T05:56:13.710364+0000","last_fullsized":"2024-09-21T05:56:13.710364+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:53.833860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.768241+0000","last_change":"2024-09-21T05:55:33.768136+0000","last_active":"2024-09-21T05:55:33.768241+0000","last_peered":"2024-09-21T05:55:33.768241+0000","last_clean":"2024-09-21T05:55:33.768241+0000","last_became_active":"2024-09-21T05:55:33.767697+0000","last_became_peered":"2024-09-21T05:55:33.767697+0000","last_unstale":"2024-09-21T05:55:33.768241+0000","last_undegraded":"2024-09-21T05:55:33.768241+0000","last_fullsized":"2024-09-21T05:55:33.768241+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:48.146453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"74'1","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.633171+0000","last_change":"2024-09-21T05:55:12.013121+0000","last_active":"2024-09-21T05:55:33.633171+0000","last_peered":"2024-09-21T05:55:33.633171+0000","last_clean":"2024-09-21T05:55:33.633171+0000","last_became_active":"2024-09-21T05:55:12.012657+0000","last_became_peered":"2024-09-21T05:55:12.012657+0000","last_unstale":"2024-09-21T05:55:33.633171+0000","last_undegraded":"2024-09-21T05:55:33.633171+0000","last_fullsized":"2024-09-21T05:55:33.633171+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:25:48.002938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"74'1","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.634506+0000","last_change":"2024-09-21T05:55:14.004660+0000","last_active":"2024-09-21T05:55:33.634506+0000","last_peered":"2024-09-21T05:55:33.634506+0000","last_clean":"2024-09-21T05:55:33.634506+0000","last_became_active":"2024-09-21T05:55:14.004297+0000","last_became_peered":"2024-09-21T05:55:14.004297+0000","last_unstale":"2024-09-21T05:55:33.634506+0000","last_undegraded":"2024-09-21T05:55:33.634506+0000","last_fullsized":"2024-09-21T05:55:33.634506+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:24.399891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"74'4","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636395+0000","last_change":"2024-09-21T05:55:12.015717+0000","last_active":"2024-09-21T05:55:33.636395+0000","last_peered":"2024-09-21T05:55:33.636395+0000","last_clean":"2024-09-21T05:55:33.636395+0000","last_became_active":"2024-09-21T05:55:12.015419+0000","last_became_peered":"2024-09-21T05:55:12.015419+0000","last_unstale":"2024-09-21T05:55:33.636395+0000","last_undegraded":"2024-09-21T05:55:33.636395+0000","last_fullsized":"2024-09-21T05:55:33.636395+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:57.988860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636635+0000","last_change":"2024-09-21T05:55:13.992460+0000","last_active":"2024-09-21T05:55:33.636635+0000","last_peered":"2024-09-21T05:55:33.636635+0000","last_clean":"2024-09-21T05:55:33.636635+0000","last_became_active":"2024-09-21T05:55:13.991887+0000","last_became_peered":"2024-09-21T05:55:13.991887+0000","last_unstale":"2024-09-21T05:55:33.636635+0000","last_undegraded":"2024-09-21T05:55:33.636635+0000","last_fullsized":"2024-09-21T05:55:33.636635+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:37.718050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"76'15","reported_seq":38,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:38.995033+0000","last_change":"2024-09-21T05:55:11.982235+0000","last_active":"2024-09-21T05:55:38.995033+0000","last_peered":"2024-09-21T05:55:38.995033+0000","last_clean":"2024-09-21T05:55:38.995033+0000","last_became_active":"2024-09-21T05:55:11.981846+0000","last_became_peered":"2024-09-21T05:55:11.981846+0000","last_unstale":"2024-09-21T05:55:38.995033+0000","last_undegraded":"2024-09-21T05:55:38.995033+0000","last_fullsized":"2024-09-21T05:55:38.995033+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:40:04.330346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639253+0000","last_change":"2024-09-21T05:55:13.999249+0000","last_active":"2024-09-21T05:55:33.639253+0000","last_peered":"2024-09-21T05:55:33.639253+0000","last_clean":"2024-09-21T05:55:33.639253+0000","last_became_active":"2024-09-21T05:55:13.998452+0000","last_became_peered":"2024-09-21T05:55:13.998452+0000","last_unstale":"2024-09-21T05:55:33.639253+0000","last_undegraded":"2024-09-21T05:55:33.639253+0000","last_fullsized":"2024-09-21T05:55:33.639253+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:59.364684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"74'7","reported_seq":31,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.696351+0000","last_change":"2024-09-21T05:55:11.971850+0000","last_active":"2024-09-21T05:55:33.696351+0000","last_peered":"2024-09-21T05:55:33.696351+0000","last_clean":"2024-09-21T05:55:33.696351+0000","last_became_active":"2024-09-21T05:55:11.971415+0000","last_became_peered":"2024-09-21T05:55:11.971415+0000","last_unstale":"2024-09-21T05:55:33.696351+0000","last_undegraded":"2024-09-21T05:55:33.696351+0000","last_fullsized":"2024-09-21T05:55:33.696351+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:51.990745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.766931+0000","last_change":"2024-09-21T05:55:13.994500+0000","last_active":"2024-09-21T05:55:33.766931+0000","last_peered":"2024-09-21T05:55:33.766931+0000","last_clean":"2024-09-21T05:55:33.766931+0000","last_became_active":"2024-09-21T05:55:13.994146+0000","last_became_peered":"2024-09-21T05:55:13.994146+0000","last_unstale":"2024-09-21T05:55:33.766931+0000","last_undegraded":"2024-09-21T05:55:33.766931+0000","last_fullsized":"2024-09-21T05:55:33.766931+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:42.934234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639268+0000","last_change":"2024-09-21T05:55:12.013479+0000","last_active":"2024-09-21T05:55:33.639268+0000","last_peered":"2024-09-21T05:55:33.639268+0000","last_clean":"2024-09-21T05:55:33.639268+0000","last_became_active":"2024-09-21T05:55:12.013180+0000","last_became_peered":"2024-09-21T05:55:12.013180+0000","last_unstale":"2024-09-21T05:55:33.639268+0000","last_undegraded":"2024-09-21T05:55:33.639268+0000","last_fullsized":"2024-09-21T05:55:33.639268+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:28.800072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635548+0000","last_change":"2024-09-21T05:55:14.001795+0000","last_active":"2024-09-21T05:55:33.635548+0000","last_peered":"2024-09-21T05:55:33.635548+0000","last_clean":"2024-09-21T05:55:33.635548+0000","last_became_active":"2024-09-21T05:55:14.001430+0000","last_became_peered":"2024-09-21T05:55:14.001430+0000","last_unstale":"2024-09-21T05:55:33.635548+0000","last_undegraded":"2024-09-21T05:55:33.635548+0000","last_fullsized":"2024-09-21T05:55:33.635548+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:10.339939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"74'7","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.638793+0000","last_change":"2024-09-21T05:55:11.976894+0000","last_active":"2024-09-21T05:55:33.638793+0000","last_peered":"2024-09-21T05:55:33.638793+0000","last_clean":"2024-09-21T05:55:33.638793+0000","last_became_active":"2024-09-21T05:55:11.976516+0000","last_became_peered":"2024-09-21T05:55:11.976516+0000","last_unstale":"2024-09-21T05:55:33.638793+0000","last_undegraded":"2024-09-21T05:55:33.638793+0000","last_fullsized":"2024-09-21T05:55:33.638793+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:58.726311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.637297+0000","last_change":"2024-09-21T05:55:14.003760+0000","last_active":"2024-09-21T05:55:33.637297+0000","last_peered":"2024-09-21T05:55:33.637297+0000","last_clean":"2024-09-21T05:55:33.637297+0000","last_became_active":"2024-09-21T05:55:14.003180+0000","last_became_peered":"2024-09-21T05:55:14.003180+0000","last_unstale":"2024-09-21T05:55:33.637297+0000","last_undegraded":"2024-09-21T05:55:33.637297+0000","last_fullsized":"2024-09-21T05:55:33.637297+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:25.077765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639713+0000","last_change":"2024-09-21T05:55:11.983865+0000","last_active":"2024-09-21T05:55:33.639713+0000","last_peered":"2024-09-21T05:55:33.639713+0000","last_clean":"2024-09-21T05:55:33.639713+0000","last_became_active":"2024-09-21T05:55:11.983473+0000","last_became_peered":"2024-09-21T05:55:11.983473+0000","last_unstale":"2024-09-21T05:55:33.639713+0000","last_undegraded":"2024-09-21T05:55:33.639713+0000","last_fullsized":"2024-09-21T05:55:33.639713+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:18.767317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639006+0000","last_change":"2024-09-21T05:55:13.999001+0000","last_active":"2024-09-21T05:55:33.639006+0000","last_peered":"2024-09-21T05:55:33.639006+0000","last_clean":"2024-09-21T05:55:33.639006+0000","last_became_active":"2024-09-21T05:55:13.998656+0000","last_became_peered":"2024-09-21T05:55:13.998656+0000","last_unstale":"2024-09-21T05:55:33.639006+0000","last_undegraded":"2024-09-21T05:55:33.639006+0000","last_fullsized":"2024-09-21T05:55:33.639006+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:50.913635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"74'4","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.646791+0000","last_change":"2024-09-21T05:55:33.646124+0000","last_active":"2024-09-21T05:55:33.646791+0000","last_peered":"2024-09-21T05:55:33.646791+0000","last_clean":"2024-09-21T05:55:33.646791+0000","last_became_active":"2024-09-21T05:55:33.641142+0000","last_became_peered":"2024-09-21T05:55:33.641142+0000","last_unstale":"2024-09-21T05:55:33.646791+0000","last_undegraded":"2024-09-21T05:55:33.646791+0000","last_fullsized":"2024-09-21T05:55:33.646791+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:36.341473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635915+0000","last_change":"2024-09-21T05:55:13.997494+0000","last_active":"2024-09-21T05:55:33.635915+0000","last_peered":"2024-09-21T05:55:33.635915+0000","last_clean":"2024-09-21T05:55:33.635915+0000","last_became_active":"2024-09-21T05:55:13.997138+0000","last_became_peered":"2024-09-21T05:55:13.997138+0000","last_unstale":"2024-09-21T05:55:33.635915+0000","last_undegraded":"2024-09-21T05:55:33.635915+0000","last_fullsized":"2024-09-21T05:55:33.635915+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:02:01.777126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767713+0000","last_change":"2024-09-21T05:55:14.002750+0000","last_active":"2024-09-21T05:55:33.767713+0000","last_peered":"2024-09-21T05:55:33.767713+0000","last_clean":"2024-09-21T05:55:33.767713+0000","last_became_active":"2024-09-21T05:55:14.002150+0000","last_became_peered":"2024-09-21T05:55:14.002150+0000","last_unstale":"2024-09-21T05:55:33.767713+0000","last_undegraded":"2024-09-21T05:55:33.767713+0000","last_fullsized":"2024-09-21T05:55:33.767713+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:49.603133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"74'5","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636032+0000","last_change":"2024-09-21T05:55:11.982366+0000","last_active":"2024-09-21T05:55:33.636032+0000","last_peered":"2024-09-21T05:55:33.636032+0000","last_clean":"2024-09-21T05:55:33.636032+0000","last_became_active":"2024-09-21T05:55:11.981994+0000","last_became_peered":"2024-09-21T05:55:11.981994+0000","last_unstale":"2024-09-21T05:55:33.636032+0000","last_undegraded":"2024-09-21T05:55:33.636032+0000","last_fullsized":"2024-09-21T05:55:33.636032+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:14.626758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639407+0000","last_change":"2024-09-21T05:55:14.008204+0000","last_active":"2024-09-21T05:55:33.639407+0000","last_peered":"2024-09-21T05:55:33.639407+0000","last_clean":"2024-09-21T05:55:33.639407+0000","last_became_active":"2024-09-21T05:55:14.007800+0000","last_became_peered":"2024-09-21T05:55:14.007800+0000","last_unstale":"2024-09-21T05:55:33.639407+0000","last_undegraded":"2024-09-21T05:55:33.639407+0000","last_fullsized":"2024-09-21T05:55:33.639407+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:49.760850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"74'12","reported_seq":36,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636283+0000","last_change":"2024-09-21T05:55:11.986168+0000","last_active":"2024-09-21T05:55:33.636283+0000","last_peered":"2024-09-21T05:55:33.636283+0000","last_clean":"2024-09-21T05:55:33.636283+0000","last_became_active":"2024-09-21T05:55:11.985694+0000","last_became_peered":"2024-09-21T05:55:11.985694+0000","last_unstale":"2024-09-21T05:55:33.636283+0000","last_undegraded":"2024-09-21T05:55:33.636283+0000","last_fullsized":"2024-09-21T05:55:33.636283+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:44.863935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767026+0000","last_change":"2024-09-21T05:55:14.002459+0000","last_active":"2024-09-21T05:55:33.767026+0000","last_peered":"2024-09-21T05:55:33.767026+0000","last_clean":"2024-09-21T05:55:33.767026+0000","last_became_active":"2024-09-21T05:55:14.001962+0000","last_became_peered":"2024-09-21T05:55:14.001962+0000","last_unstale":"2024-09-21T05:55:33.767026+0000","last_undegraded":"2024-09-21T05:55:33.767026+0000","last_fullsized":"2024-09-21T05:55:33.767026+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:53:26.892772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639340+0000","last_change":"2024-09-21T05:55:12.014149+0000","last_active":"2024-09-21T05:55:33.639340+0000","last_peered":"2024-09-21T05:55:33.639340+0000","last_clean":"2024-09-21T05:55:33.639340+0000","last_became_active":"2024-09-21T05:55:12.013789+0000","last_became_peered":"2024-09-21T05:55:12.013789+0000","last_unstale":"2024-09-21T05:55:33.639340+0000","last_undegraded":"2024-09-21T05:55:33.639340+0000","last_fullsized":"2024-09-21T05:55:33.639340+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:18.304023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639186+0000","last_change":"2024-09-21T05:55:13.993500+0000","last_active":"2024-09-21T05:55:33.639186+0000","last_peered":"2024-09-21T05:55:33.639186+0000","last_clean":"2024-09-21T05:55:33.639186+0000","last_became_active":"2024-09-21T05:55:13.993104+0000","last_became_peered":"2024-09-21T05:55:13.993104+0000","last_unstale":"2024-09-21T05:55:33.639186+0000","last_undegraded":"2024-09-21T05:55:33.639186+0000","last_fullsized":"2024-09-21T05:55:33.639186+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:01.445119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"74'13","reported_seq":108,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:56:14.711079+0000","last_change":"2024-09-21T05:55:33.650207+0000","last_active":"2024-09-21T05:56:14.711079+0000","last_peered":"2024-09-21T05:56:14.711079+0000","last_clean":"2024-09-21T05:56:14.711079+0000","last_became_active":"2024-09-21T05:55:33.638388+0000","last_became_peered":"2024-09-21T05:55:33.638388+0000","last_unstale":"2024-09-21T05:56:14.711079+0000","last_undegraded":"2024-09-21T05:56:14.711079+0000","last_fullsized":"2024-09-21T05:56:14.711079+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:51.478063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":46,"num_read_kb":46,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71497,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":552,"num_read_kb":348,"num_write":245,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":214491,"data_compressed":5682,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281769,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":68,"seq":292057776154,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":839680,"data_stored":412559,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939679,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":1066818,"data_compressed":43475,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103204,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":856064,"data_stored":411426,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299434,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":411804,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495662,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":897024,"data_stored":1004041,"data_compressed":41581,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659188,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":839680,"data_stored":411143,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822713,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":917504,"data_stored":1004000,"data_compressed":41581,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986239,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":839680,"data_stored":411395,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":856064,"data_stored":410968,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378697,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":476960,"data_compressed":7646,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542220,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":475958,"data_compressed":7646,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705747,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32028,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713124,"statfs":{"total":95995035648,"available":95962238976,"internally_reserved":0,"allocated":860160,"data_stored":412873,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34208,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34208,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34208,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35829,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35829,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35829,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35829,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35829,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":65654,"data_compressed":1894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":66656,"data_compressed":1894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":66029,"data_compressed":1894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:56:22.304 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T05:56:22.304 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T05:56:22.304 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-21T05:56:22.304 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph health --format=json 2024-09-21T05:56:22.555 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:22.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:22 smithi192 ceph-mon[34700]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T05:56:23.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:22 smithi117 ceph-mon[30947]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T05:56:23.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:22 smithi138 ceph-mon[34391]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T05:56:23.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.122+0000 7f08b68cc640 1 -- 172.21.15.117:0/581927108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08b0109e10 msgr2=0x7f08b0110640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:23.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.122+0000 7f08b68cc640 1 --2- 172.21.15.117:0/581927108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08b0109e10 0x7f08b0110640 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f08a8009ca0 tx=0x7f08a802a8e0 comp rx=0 tx=0).stop 2024-09-21T05:56:23.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.122+0000 7f08b68cc640 1 -- 172.21.15.117:0/581927108 shutdown_connections 2024-09-21T05:56:23.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.122+0000 7f08b68cc640 1 --2- 172.21.15.117:0/581927108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08b0109e10 0x7f08b0110640 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:23.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.122+0000 7f08b68cc640 1 --2- 172.21.15.117:0/581927108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08b01094c0 0x7f08b01098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:23.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.122+0000 7f08b68cc640 1 --2- 172.21.15.117:0/581927108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08b0103b80 0x7f08b0103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:23.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.122+0000 7f08b68cc640 1 -- 172.21.15.117:0/581927108 >> 172.21.15.117:0/581927108 conn(0x7f08b00ff5d0 msgr2=0x7f08b0101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:23.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.122+0000 7f08b68cc640 1 -- 172.21.15.117:0/581927108 shutdown_connections 2024-09-21T05:56:23.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.122+0000 7f08b68cc640 1 -- 172.21.15.117:0/581927108 wait complete. 2024-09-21T05:56:23.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.123+0000 7f08b68cc640 1 Processor -- start 2024-09-21T05:56:23.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.123+0000 7f08b68cc640 1 -- start start 2024-09-21T05:56:23.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.123+0000 7f08b68cc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08b0103b80 0x7f08b0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:23.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.123+0000 7f08b68cc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08b01094c0 0x7f08b0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:23.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.123+0000 7f08b68cc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08b0109e10 0x7f08b0076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:23.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.123+0000 7f08b68cc640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f08b0113b90 con 0x7f08b0109e10 2024-09-21T05:56:23.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.123+0000 7f08b68cc640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f08b0113a10 con 0x7f08b0103b80 2024-09-21T05:56:23.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.123+0000 7f08b68cc640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f08b0113d10 con 0x7f08b01094c0 2024-09-21T05:56:23.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.123+0000 7f08b4e69640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08b0109e10 0x7f08b0076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:23.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.123+0000 7f08b4e69640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08b0109e10 0x7f08b0076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40178/0 (socket says 172.21.15.117:40178) 2024-09-21T05:56:23.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.123+0000 7f08b4e69640 1 -- 172.21.15.117:0/3056198350 learned_addr learned my addr 172.21.15.117:0/3056198350 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:23.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.124+0000 7f08affff640 1 --2- 172.21.15.117:0/3056198350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08b0103b80 0x7f08b0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:23.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.124+0000 7f08af7fe640 1 --2- 172.21.15.117:0/3056198350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08b01094c0 0x7f08b0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:23.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.124+0000 7f08b4e69640 1 -- 172.21.15.117:0/3056198350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08b01094c0 msgr2=0x7f08b0075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:23.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.124+0000 7f08b4e69640 1 --2- 172.21.15.117:0/3056198350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08b01094c0 0x7f08b0075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:23.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.124+0000 7f08b4e69640 1 -- 172.21.15.117:0/3056198350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08b0103b80 msgr2=0x7f08b0075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:23.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.124+0000 7f08b4e69640 1 --2- 172.21.15.117:0/3056198350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08b0103b80 0x7f08b0075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:23.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.124+0000 7f08b4e69640 1 -- 172.21.15.117:0/3056198350 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b01a8840 con 0x7f08b0109e10 2024-09-21T05:56:23.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.124+0000 7f08af7fe640 1 --2- 172.21.15.117:0/3056198350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08b01094c0 0x7f08b0075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:23.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.124+0000 7f08b4e69640 1 --2- 172.21.15.117:0/3056198350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08b0109e10 0x7f08b0076190 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f08a8005cc0 tx=0x7f08a8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:23.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.124+0000 7f08ad7fa640 1 -- 172.21.15.117:0/3056198350 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a8004040 con 0x7f08b0109e10 2024-09-21T05:56:23.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.124+0000 7f08b68cc640 1 -- 172.21.15.117:0/3056198350 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08b01a8ad0 con 0x7f08b0109e10 2024-09-21T05:56:23.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.124+0000 7f08ad7fa640 1 -- 172.21.15.117:0/3056198350 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08a8004690 con 0x7f08b0109e10 2024-09-21T05:56:23.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.124+0000 7f08ad7fa640 1 -- 172.21.15.117:0/3056198350 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a8030e20 con 0x7f08b0109e10 2024-09-21T05:56:23.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.124+0000 7f08b68cc640 1 -- 172.21.15.117:0/3056198350 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08b01a9010 con 0x7f08b0109e10 2024-09-21T05:56:23.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.125+0000 7f08b68cc640 1 -- 172.21.15.117:0/3056198350 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08b0104010 con 0x7f08b0109e10 2024-09-21T05:56:23.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.127+0000 7f08ad7fa640 1 -- 172.21.15.117:0/3056198350 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f08a80305e0 con 0x7f08b0109e10 2024-09-21T05:56:23.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.127+0000 7f08ad7fa640 1 --2- 172.21.15.117:0/3056198350 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f08980b65f0 0x7f08980b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:23.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.127+0000 7f08affff640 1 --2- 172.21.15.117:0/3056198350 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f08980b65f0 0x7f08980b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:23.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.127+0000 7f08ad7fa640 1 -- 172.21.15.117:0/3056198350 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f08a8108bc0 con 0x7f08b0109e10 2024-09-21T05:56:23.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.128+0000 7f08affff640 1 --2- 172.21.15.117:0/3056198350 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f08980b65f0 0x7f08980b8b00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f08b0077160 tx=0x7f08a0008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:23.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.131+0000 7f08ad7fa640 1 -- 172.21.15.117:0/3056198350 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f08a80cf250 con 0x7f08b0109e10 2024-09-21T05:56:23.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.345+0000 7f08b68cc640 1 -- 172.21.15.117:0/3056198350 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f08b0076a10 con 0x7f08b0109e10 2024-09-21T05:56:23.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.346+0000 7f08ad7fa640 1 -- 172.21.15.117:0/3056198350 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f08b0076a10 con 0x7f08b0109e10 2024-09-21T05:56:23.348 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:56:23.348 INFO:teuthology.orchestra.run.smithi117.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T05:56:23.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.348+0000 7f08b68cc640 1 -- 172.21.15.117:0/3056198350 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f08980b65f0 msgr2=0x7f08980b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:23.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.348+0000 7f08b68cc640 1 --2- 172.21.15.117:0/3056198350 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f08980b65f0 0x7f08980b8b00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f08b0077160 tx=0x7f08a0008040 comp rx=0 tx=0).stop 2024-09-21T05:56:23.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.348+0000 7f08b68cc640 1 -- 172.21.15.117:0/3056198350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08b0109e10 msgr2=0x7f08b0076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:23.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.348+0000 7f08b68cc640 1 --2- 172.21.15.117:0/3056198350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08b0109e10 0x7f08b0076190 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f08a8005cc0 tx=0x7f08a8003ba0 comp rx=0 tx=0).stop 2024-09-21T05:56:23.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.348+0000 7f08b68cc640 1 -- 172.21.15.117:0/3056198350 shutdown_connections 2024-09-21T05:56:23.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.348+0000 7f08b68cc640 1 --2- 172.21.15.117:0/3056198350 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f08980b65f0 0x7f08980b8b00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:23.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.348+0000 7f08b68cc640 1 --2- 172.21.15.117:0/3056198350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08b0109e10 0x7f08b0076190 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:23.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.348+0000 7f08b68cc640 1 --2- 172.21.15.117:0/3056198350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08b01094c0 0x7f08b0075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:23.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.348+0000 7f08b68cc640 1 --2- 172.21.15.117:0/3056198350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08b0103b80 0x7f08b0075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:23.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.348+0000 7f08b68cc640 1 -- 172.21.15.117:0/3056198350 >> 172.21.15.117:0/3056198350 conn(0x7f08b00ff5d0 msgr2=0x7f08b01019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:23.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.349+0000 7f08b68cc640 1 -- 172.21.15.117:0/3056198350 shutdown_connections 2024-09-21T05:56:23.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:23.349+0000 7f08b68cc640 1 -- 172.21.15.117:0/3056198350 wait complete. 2024-09-21T05:56:23.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:23 smithi192 ceph-mon[34700]: pgmap v299: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T05:56:23.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:23 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3056198350' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T05:56:24.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:23 smithi117 ceph-mon[30947]: pgmap v299: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T05:56:24.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:23 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3056198350' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T05:56:24.107 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-21T05:56:24.107 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T05:56:24.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:23 smithi138 ceph-mon[34391]: pgmap v299: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T05:56:24.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:23 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3056198350' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T05:56:24.352 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:24.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.847+0000 7f9daea51640 1 -- 172.21.15.117:0/197746427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9da8102290 msgr2=0x7f9da81026a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:24.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.847+0000 7f9daea51640 1 --2- 172.21.15.117:0/197746427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9da8102290 0x7f9da81026a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9d98009900 tx=0x7f9d9802a3f0 comp rx=0 tx=0).stop 2024-09-21T05:56:24.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.848+0000 7f9daea51640 1 -- 172.21.15.117:0/197746427 shutdown_connections 2024-09-21T05:56:24.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.848+0000 7f9daea51640 1 --2- 172.21.15.117:0/197746427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9da8108520 0x7f9da810ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:24.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.848+0000 7f9daea51640 1 --2- 172.21.15.117:0/197746427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9da8107bd0 0x7f9da8107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:24.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.848+0000 7f9daea51640 1 --2- 172.21.15.117:0/197746427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9da8102290 0x7f9da81026a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:24.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.848+0000 7f9daea51640 1 -- 172.21.15.117:0/197746427 >> 172.21.15.117:0/197746427 conn(0x7f9da8079ea0 msgr2=0x7f9da807a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:24.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.848+0000 7f9daea51640 1 -- 172.21.15.117:0/197746427 shutdown_connections 2024-09-21T05:56:24.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.848+0000 7f9daea51640 1 -- 172.21.15.117:0/197746427 wait complete. 2024-09-21T05:56:24.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.848+0000 7f9daea51640 1 Processor -- start 2024-09-21T05:56:24.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.848+0000 7f9daea51640 1 -- start start 2024-09-21T05:56:24.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.849+0000 7f9daea51640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9da8102290 0x7f9da819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:24.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.849+0000 7f9daea51640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9da8107bd0 0x7f9da819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:24.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.849+0000 7f9daea51640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9da8108520 0x7f9da819fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:24.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.849+0000 7f9daea51640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9da8111960 con 0x7f9da8102290 2024-09-21T05:56:24.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.849+0000 7f9daea51640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9da81117e0 con 0x7f9da8108520 2024-09-21T05:56:24.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.849+0000 7f9daea51640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9da8111ae0 con 0x7f9da8107bd0 2024-09-21T05:56:24.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.849+0000 7f9da7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9da8102290 0x7f9da819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:24.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.849+0000 7f9da7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9da8102290 0x7f9da819a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40504/0 (socket says 172.21.15.117:40504) 2024-09-21T05:56:24.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.849+0000 7f9da7fff640 1 -- 172.21.15.117:0/231449576 learned_addr learned my addr 172.21.15.117:0/231449576 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:24.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.849+0000 7f9da77fe640 1 --2- 172.21.15.117:0/231449576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9da8107bd0 0x7f9da819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:24.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.849+0000 7f9dacfee640 1 --2- 172.21.15.117:0/231449576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9da8108520 0x7f9da819fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:24.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.849+0000 7f9da7fff640 1 -- 172.21.15.117:0/231449576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9da8107bd0 msgr2=0x7f9da819aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:24.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.849+0000 7f9da7fff640 1 --2- 172.21.15.117:0/231449576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9da8107bd0 0x7f9da819aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:24.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.849+0000 7f9da7fff640 1 -- 172.21.15.117:0/231449576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9da8108520 msgr2=0x7f9da819fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:24.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.849+0000 7f9da7fff640 1 --2- 172.21.15.117:0/231449576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9da8108520 0x7f9da819fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:24.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.849+0000 7f9da7fff640 1 -- 172.21.15.117:0/231449576 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da81a0400 con 0x7f9da8102290 2024-09-21T05:56:24.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.850+0000 7f9da7fff640 1 --2- 172.21.15.117:0/231449576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9da8102290 0x7f9da819a550 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9d98005940 tx=0x7f9d980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:24.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.850+0000 7f9da57fa640 1 -- 172.21.15.117:0/231449576 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9803f030 con 0x7f9da8102290 2024-09-21T05:56:24.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.850+0000 7f9da57fa640 1 -- 172.21.15.117:0/231449576 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d98003cb0 con 0x7f9da8102290 2024-09-21T05:56:24.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.850+0000 7f9da57fa640 1 -- 172.21.15.117:0/231449576 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98035070 con 0x7f9da8102290 2024-09-21T05:56:24.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.850+0000 7f9daea51640 1 -- 172.21.15.117:0/231449576 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da81a0690 con 0x7f9da8102290 2024-09-21T05:56:24.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.850+0000 7f9daea51640 1 -- 172.21.15.117:0/231449576 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da81a0b20 con 0x7f9da8102290 2024-09-21T05:56:24.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.851+0000 7f9daea51640 1 -- 172.21.15.117:0/231449576 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9da81026a0 con 0x7f9da8102290 2024-09-21T05:56:24.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.853+0000 7f9da57fa640 1 -- 172.21.15.117:0/231449576 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f9d98042030 con 0x7f9da8102290 2024-09-21T05:56:24.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.854+0000 7f9da57fa640 1 --2- 172.21.15.117:0/231449576 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9d900b66a0 0x7f9d900b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:24.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.854+0000 7f9da77fe640 1 --2- 172.21.15.117:0/231449576 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9d900b66a0 0x7f9d900b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:24.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.854+0000 7f9da57fa640 1 -- 172.21.15.117:0/231449576 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f9d98107cb0 con 0x7f9da8102290 2024-09-21T05:56:24.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.854+0000 7f9da77fe640 1 --2- 172.21.15.117:0/231449576 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9d900b66a0 0x7f9d900b8bb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9da8104cf0 tx=0x7f9d9c00a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:24.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:24.857+0000 7f9da57fa640 1 -- 172.21.15.117:0/231449576 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9d980ce340 con 0x7f9da8102290 2024-09-21T05:56:25.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:25.074+0000 7f9daea51640 1 -- 172.21.15.117:0/231449576 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9da819b680 con 0x7f9da8102290 2024-09-21T05:56:25.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:25.075+0000 7f9da57fa640 1 -- 172.21.15.117:0/231449576 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+12001 (secure 0 0 0) 0x7f9d980d2720 con 0x7f9da8102290 2024-09-21T05:56:25.077 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:56:25.077 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":26,"btime":"2024-09-21T05:55:26:209435+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3227057504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3227057504},{"type":"v1","addr":"172.21.15.117:6835","nonce":3227057504}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3303829183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3303829183},{"type":"v1","addr":"172.21.15.117:6837","nonce":3303829183}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T05:55:14.555897+0000","modified":"2024-09-21T05:55:26.209411+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24638,"mds_1":24637,"mds_2":24628,"mds_3":24616,"mds_4":24602},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6839/2314572411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2314572411},{"type":"v1","addr":"172.21.15.117:6839","nonce":2314572411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6841/3378096561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":3378096561},{"type":"v1","addr":"172.21.15.117:6841","nonce":3378096561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/3725629862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3725629862},{"type":"v1","addr":"172.21.15.138:6833","nonce":3725629862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/4276801396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4276801396},{"type":"v1","addr":"172.21.15.138:6835","nonce":4276801396}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6833/2224910879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2224910879},{"type":"v1","addr":"172.21.15.192:6833","nonce":2224910879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.138:6837/2540335374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2540335374},{"type":"v1","addr":"172.21.15.138:6837","nonce":2540335374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24616":{"gid":24616,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.192:6835/3530688860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3530688860},{"type":"v1","addr":"172.21.15.192:6835","nonce":3530688860}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.192:6837/2683860842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2683860842},{"type":"v1","addr":"172.21.15.192:6837","nonce":2683860842}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.192:6839/1727623231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1727623231},{"type":"v1","addr":"172.21.15.192:6839","nonce":1727623231}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/3230372576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":3230372576},{"type":"v1","addr":"172.21.15.138:6839","nonce":3230372576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24638,24637,24616,24628,24602]},"id":1}]} 2024-09-21T05:56:25.078 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 26 2024-09-21T05:56:25.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:25.078+0000 7f9daea51640 1 -- 172.21.15.117:0/231449576 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9d900b66a0 msgr2=0x7f9d900b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:25.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:25.078+0000 7f9daea51640 1 --2- 172.21.15.117:0/231449576 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9d900b66a0 0x7f9d900b8bb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9da8104cf0 tx=0x7f9d9c00a2d0 comp rx=0 tx=0).stop 2024-09-21T05:56:25.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:25.078+0000 7f9daea51640 1 -- 172.21.15.117:0/231449576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9da8102290 msgr2=0x7f9da819a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:25.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:25.078+0000 7f9daea51640 1 --2- 172.21.15.117:0/231449576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9da8102290 0x7f9da819a550 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9d98005940 tx=0x7f9d980042f0 comp rx=0 tx=0).stop 2024-09-21T05:56:25.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:25.078+0000 7f9daea51640 1 -- 172.21.15.117:0/231449576 shutdown_connections 2024-09-21T05:56:25.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:25.078+0000 7f9daea51640 1 --2- 172.21.15.117:0/231449576 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9d900b66a0 0x7f9d900b8bb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:25.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:25.078+0000 7f9daea51640 1 --2- 172.21.15.117:0/231449576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9da8108520 0x7f9da819fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:25.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:25.078+0000 7f9daea51640 1 --2- 172.21.15.117:0/231449576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9da8107bd0 0x7f9da819aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:25.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:25.078+0000 7f9daea51640 1 --2- 172.21.15.117:0/231449576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9da8102290 0x7f9da819a550 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:25.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:25.078+0000 7f9daea51640 1 -- 172.21.15.117:0/231449576 >> 172.21.15.117:0/231449576 conn(0x7f9da8079ea0 msgr2=0x7f9da8105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:25.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:25.078+0000 7f9daea51640 1 -- 172.21.15.117:0/231449576 shutdown_connections 2024-09-21T05:56:25.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:25.078+0000 7f9daea51640 1 -- 172.21.15.117:0/231449576 wait complete. 2024-09-21T05:56:25.757 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:25 smithi117 ceph-mon[30947]: pgmap v300: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T05:56:25.757 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:25 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/231449576' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:56:25.788 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T05:56:25.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:25 smithi192 ceph-mon[34700]: pgmap v300: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T05:56:25.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:25 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/231449576' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:56:26.039 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:26.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:25 smithi138 ceph-mon[34391]: pgmap v300: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T05:56:26.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:25 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/231449576' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:56:26.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.488+0000 7f55ec3f4640 1 -- 172.21.15.117:0/2247171719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55e4108d00 msgr2=0x7f55e410f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:26.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.488+0000 7f55ec3f4640 1 --2- 172.21.15.117:0/2247171719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55e4108d00 0x7f55e410f530 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009900 tx=0x7f55dc02a410 comp rx=0 tx=0).stop 2024-09-21T05:56:26.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.488+0000 7f55ec3f4640 1 -- 172.21.15.117:0/2247171719 shutdown_connections 2024-09-21T05:56:26.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.488+0000 7f55ec3f4640 1 --2- 172.21.15.117:0/2247171719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55e4108d00 0x7f55e410f530 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:26.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.488+0000 7f55ec3f4640 1 --2- 172.21.15.117:0/2247171719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55e41083b0 0x7f55e41087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:26.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.488+0000 7f55ec3f4640 1 --2- 172.21.15.117:0/2247171719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f55e4102a70 0x7f55e4102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:26.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.488+0000 7f55ec3f4640 1 -- 172.21.15.117:0/2247171719 >> 172.21.15.117:0/2247171719 conn(0x7f55e4079730 msgr2=0x7f55e4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:26.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.488+0000 7f55ec3f4640 1 -- 172.21.15.117:0/2247171719 shutdown_connections 2024-09-21T05:56:26.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.488+0000 7f55ec3f4640 1 -- 172.21.15.117:0/2247171719 wait complete. 2024-09-21T05:56:26.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.489+0000 7f55ec3f4640 1 Processor -- start 2024-09-21T05:56:26.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.489+0000 7f55ec3f4640 1 -- start start 2024-09-21T05:56:26.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.489+0000 7f55ec3f4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55e4102a70 0x7f55e4199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:26.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.489+0000 7f55ec3f4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f55e41083b0 0x7f55e4199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:26.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.489+0000 7f55ec3f4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55e4108d00 0x7f55e419ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:26.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.489+0000 7f55ec3f4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f55e41129d0 con 0x7f55e4108d00 2024-09-21T05:56:26.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.489+0000 7f55ec3f4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f55e4112850 con 0x7f55e4102a70 2024-09-21T05:56:26.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.489+0000 7f55ec3f4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f55e4112b50 con 0x7f55e41083b0 2024-09-21T05:56:26.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.489+0000 7f55ea991640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55e4108d00 0x7f55e419ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:26.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.489+0000 7f55ea991640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55e4108d00 0x7f55e419ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40528/0 (socket says 172.21.15.117:40528) 2024-09-21T05:56:26.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.489+0000 7f55ea991640 1 -- 172.21.15.117:0/3878644024 learned_addr learned my addr 172.21.15.117:0/3878644024 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:26.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.489+0000 7f55e998f640 1 --2- 172.21.15.117:0/3878644024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f55e41083b0 0x7f55e4199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:26.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.489+0000 7f55ea190640 1 --2- 172.21.15.117:0/3878644024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55e4102a70 0x7f55e4199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:26.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.490+0000 7f55ea991640 1 -- 172.21.15.117:0/3878644024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f55e41083b0 msgr2=0x7f55e4199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:26.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.490+0000 7f55ea991640 1 --2- 172.21.15.117:0/3878644024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f55e41083b0 0x7f55e4199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:26.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.490+0000 7f55ea991640 1 -- 172.21.15.117:0/3878644024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55e4102a70 msgr2=0x7f55e4199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:26.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.490+0000 7f55ea991640 1 --2- 172.21.15.117:0/3878644024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55e4102a70 0x7f55e4199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:26.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.490+0000 7f55ea991640 1 -- 172.21.15.117:0/3878644024 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e419f230 con 0x7f55e4108d00 2024-09-21T05:56:26.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.490+0000 7f55ea991640 1 --2- 172.21.15.117:0/3878644024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55e4108d00 0x7f55e419ebc0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009a20 tx=0x7f55dc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:26.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.490+0000 7f55e37fe640 1 -- 172.21.15.117:0/3878644024 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55dc046070 con 0x7f55e4108d00 2024-09-21T05:56:26.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.490+0000 7f55ec3f4640 1 -- 172.21.15.117:0/3878644024 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55e419f4c0 con 0x7f55e4108d00 2024-09-21T05:56:26.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.490+0000 7f55e37fe640 1 -- 172.21.15.117:0/3878644024 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55dc003aa0 con 0x7f55e4108d00 2024-09-21T05:56:26.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.490+0000 7f55e37fe640 1 -- 172.21.15.117:0/3878644024 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55dc035050 con 0x7f55e4108d00 2024-09-21T05:56:26.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.490+0000 7f55ec3f4640 1 -- 172.21.15.117:0/3878644024 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55e406bb80 con 0x7f55e4108d00 2024-09-21T05:56:26.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.492+0000 7f55ec3f4640 1 -- 172.21.15.117:0/3878644024 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55e4103cf0 con 0x7f55e4108d00 2024-09-21T05:56:26.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.493+0000 7f55e37fe640 1 -- 172.21.15.117:0/3878644024 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f55dc049070 con 0x7f55e4108d00 2024-09-21T05:56:26.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.494+0000 7f55e37fe640 1 --2- 172.21.15.117:0/3878644024 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f55cc0b6620 0x7f55cc0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:26.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.494+0000 7f55ea190640 1 --2- 172.21.15.117:0/3878644024 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f55cc0b6620 0x7f55cc0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:26.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.494+0000 7f55e37fe640 1 -- 172.21.15.117:0/3878644024 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f55dc1093a0 con 0x7f55e4108d00 2024-09-21T05:56:26.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.494+0000 7f55ea190640 1 --2- 172.21.15.117:0/3878644024 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f55cc0b6620 0x7f55cc0b8b30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f55d400a400 tx=0x7f55d400d040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:26.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.498+0000 7f55e37fe640 1 -- 172.21.15.117:0/3878644024 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f55dc0cfa30 con 0x7f55e4108d00 2024-09-21T05:56:26.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.715+0000 7f55ec3f4640 1 -- 172.21.15.117:0/3878644024 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55e4102e80 con 0x7f55e4108d00 2024-09-21T05:56:26.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.716+0000 7f55e37fe640 1 -- 172.21.15.117:0/3878644024 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+12001 (secure 0 0 0) 0x7f55e4102e80 con 0x7f55e4108d00 2024-09-21T05:56:26.718 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:56:26.718 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":26,"btime":"2024-09-21T05:55:26:209435+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3227057504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3227057504},{"type":"v1","addr":"172.21.15.117:6835","nonce":3227057504}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3303829183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3303829183},{"type":"v1","addr":"172.21.15.117:6837","nonce":3303829183}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T05:55:14.555897+0000","modified":"2024-09-21T05:55:26.209411+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24638,"mds_1":24637,"mds_2":24628,"mds_3":24616,"mds_4":24602},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6839/2314572411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2314572411},{"type":"v1","addr":"172.21.15.117:6839","nonce":2314572411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6841/3378096561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":3378096561},{"type":"v1","addr":"172.21.15.117:6841","nonce":3378096561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/3725629862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3725629862},{"type":"v1","addr":"172.21.15.138:6833","nonce":3725629862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/4276801396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4276801396},{"type":"v1","addr":"172.21.15.138:6835","nonce":4276801396}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6833/2224910879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2224910879},{"type":"v1","addr":"172.21.15.192:6833","nonce":2224910879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.138:6837/2540335374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2540335374},{"type":"v1","addr":"172.21.15.138:6837","nonce":2540335374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24616":{"gid":24616,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.192:6835/3530688860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3530688860},{"type":"v1","addr":"172.21.15.192:6835","nonce":3530688860}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.192:6837/2683860842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2683860842},{"type":"v1","addr":"172.21.15.192:6837","nonce":2683860842}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.192:6839/1727623231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1727623231},{"type":"v1","addr":"172.21.15.192:6839","nonce":1727623231}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/3230372576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":3230372576},{"type":"v1","addr":"172.21.15.138:6839","nonce":3230372576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24638,24637,24616,24628,24602]},"id":1}]} 2024-09-21T05:56:26.718 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 26 2024-09-21T05:56:26.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.718+0000 7f55ec3f4640 1 -- 172.21.15.117:0/3878644024 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f55cc0b6620 msgr2=0x7f55cc0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:26.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.718+0000 7f55ec3f4640 1 --2- 172.21.15.117:0/3878644024 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f55cc0b6620 0x7f55cc0b8b30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f55d400a400 tx=0x7f55d400d040 comp rx=0 tx=0).stop 2024-09-21T05:56:26.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.718+0000 7f55ec3f4640 1 -- 172.21.15.117:0/3878644024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55e4108d00 msgr2=0x7f55e419ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:26.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.718+0000 7f55ec3f4640 1 --2- 172.21.15.117:0/3878644024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55e4108d00 0x7f55e419ebc0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009a20 tx=0x7f55dc004270 comp rx=0 tx=0).stop 2024-09-21T05:56:26.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.718+0000 7f55ec3f4640 1 -- 172.21.15.117:0/3878644024 shutdown_connections 2024-09-21T05:56:26.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.718+0000 7f55ec3f4640 1 --2- 172.21.15.117:0/3878644024 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f55cc0b6620 0x7f55cc0b8b30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:26.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.718+0000 7f55ec3f4640 1 --2- 172.21.15.117:0/3878644024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f55e4108d00 0x7f55e419ebc0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:26.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.718+0000 7f55ec3f4640 1 --2- 172.21.15.117:0/3878644024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f55e41083b0 0x7f55e4199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:26.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.718+0000 7f55ec3f4640 1 --2- 172.21.15.117:0/3878644024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55e4102a70 0x7f55e4199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:26.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.718+0000 7f55ec3f4640 1 -- 172.21.15.117:0/3878644024 >> 172.21.15.117:0/3878644024 conn(0x7f55e4079730 msgr2=0x7f55e4105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:26.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.718+0000 7f55ec3f4640 1 -- 172.21.15.117:0/3878644024 shutdown_connections 2024-09-21T05:56:26.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:26.718+0000 7f55ec3f4640 1 -- 172.21.15.117:0/3878644024 wait complete. 2024-09-21T05:56:27.417 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T05:56:27.664 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:27.765 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:27 smithi117 ceph-mon[30947]: pgmap v301: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T05:56:27.765 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:27 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3878644024' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:56:27.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:27 smithi192 ceph-mon[34700]: pgmap v301: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T05:56:27.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:27 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3878644024' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:56:28.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:27 smithi138 ceph-mon[34391]: pgmap v301: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T05:56:28.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:27 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3878644024' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:56:28.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.158+0000 7f81b49ef640 1 -- 172.21.15.117:0/679134049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81b0108600 msgr2=0x7f81b0108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:28.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.158+0000 7f81b49ef640 1 --2- 172.21.15.117:0/679134049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81b0108600 0x7f81b0108a10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f81a4009900 tx=0x7f81a402a410 comp rx=0 tx=0).stop 2024-09-21T05:56:28.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.158+0000 7f81b49ef640 1 -- 172.21.15.117:0/679134049 shutdown_connections 2024-09-21T05:56:28.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.158+0000 7f81b49ef640 1 --2- 172.21.15.117:0/679134049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81b0108600 0x7f81b0108a10 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:28.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.158+0000 7f81b49ef640 1 --2- 172.21.15.117:0/679134049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81b0108f50 0x7f81b010f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:28.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.158+0000 7f81b49ef640 1 --2- 172.21.15.117:0/679134049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81b01031c0 0x7f81b01035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:28.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.158+0000 7f81b49ef640 1 -- 172.21.15.117:0/679134049 >> 172.21.15.117:0/679134049 conn(0x7f81b0079e30 msgr2=0x7f81b007a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:28.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.158+0000 7f81b49ef640 1 -- 172.21.15.117:0/679134049 shutdown_connections 2024-09-21T05:56:28.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.158+0000 7f81b49ef640 1 -- 172.21.15.117:0/679134049 wait complete. 2024-09-21T05:56:28.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.159+0000 7f81b49ef640 1 Processor -- start 2024-09-21T05:56:28.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.159+0000 7f81b49ef640 1 -- start start 2024-09-21T05:56:28.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.159+0000 7f81b49ef640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81b01031c0 0x7f81b019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:28.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.159+0000 7f81b49ef640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81b0108600 0x7f81b019ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:28.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.159+0000 7f81b49ef640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81b0108f50 0x7f81b019ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:28.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.159+0000 7f81b49ef640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f81b0112c10 con 0x7f81b01031c0 2024-09-21T05:56:28.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.159+0000 7f81b49ef640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81b0112a90 con 0x7f81b0108600 2024-09-21T05:56:28.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.159+0000 7f81b49ef640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f81b0112d90 con 0x7f81b0108f50 2024-09-21T05:56:28.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.159+0000 7f81ae59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81b01031c0 0x7f81b019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:28.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.159+0000 7f81ae59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81b01031c0 0x7f81b019a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40550/0 (socket says 172.21.15.117:40550) 2024-09-21T05:56:28.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.159+0000 7f81ae59c640 1 -- 172.21.15.117:0/804512994 learned_addr learned my addr 172.21.15.117:0/804512994 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:28.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.159+0000 7f81ae59c640 1 -- 172.21.15.117:0/804512994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81b0108f50 msgr2=0x7f81b019ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:28.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.159+0000 7f81ae59c640 1 --2- 172.21.15.117:0/804512994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81b0108f50 0x7f81b019ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:28.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.159+0000 7f81ae59c640 1 -- 172.21.15.117:0/804512994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81b0108600 msgr2=0x7f81b019ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:28.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.159+0000 7f81ae59c640 1 --2- 172.21.15.117:0/804512994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81b0108600 0x7f81b019ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:28.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.159+0000 7f81ae59c640 1 -- 172.21.15.117:0/804512994 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b01a0620 con 0x7f81b01031c0 2024-09-21T05:56:28.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.160+0000 7f81ae59c640 1 --2- 172.21.15.117:0/804512994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81b01031c0 0x7f81b019a770 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f81a000c920 tx=0x7f81a000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:28.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.160+0000 7f819f7fe640 1 -- 172.21.15.117:0/804512994 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a0007c70 con 0x7f81b01031c0 2024-09-21T05:56:28.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.160+0000 7f81b49ef640 1 -- 172.21.15.117:0/804512994 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b01a0910 con 0x7f81b01031c0 2024-09-21T05:56:28.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.160+0000 7f819f7fe640 1 -- 172.21.15.117:0/804512994 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81a0007e10 con 0x7f81b01031c0 2024-09-21T05:56:28.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.160+0000 7f819f7fe640 1 -- 172.21.15.117:0/804512994 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a0018820 con 0x7f81b01031c0 2024-09-21T05:56:28.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.160+0000 7f81b49ef640 1 -- 172.21.15.117:0/804512994 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b01a7130 con 0x7f81b01031c0 2024-09-21T05:56:28.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.161+0000 7f81b49ef640 1 -- 172.21.15.117:0/804512994 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81b01035d0 con 0x7f81b01031c0 2024-09-21T05:56:28.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.162+0000 7f819f7fe640 1 -- 172.21.15.117:0/804512994 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f81a000f450 con 0x7f81b01031c0 2024-09-21T05:56:28.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.163+0000 7f819f7fe640 1 --2- 172.21.15.117:0/804512994 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f81980b6620 0x7f81980b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:28.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.163+0000 7f81add9b640 1 --2- 172.21.15.117:0/804512994 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f81980b6620 0x7f81980b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:28.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.163+0000 7f819f7fe640 1 -- 172.21.15.117:0/804512994 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f81a00e7790 con 0x7f81b01031c0 2024-09-21T05:56:28.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.164+0000 7f81add9b640 1 --2- 172.21.15.117:0/804512994 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f81980b6620 0x7f81980b8b30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f81a402a930 tx=0x7f81a4031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:28.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.167+0000 7f819f7fe640 1 -- 172.21.15.117:0/804512994 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f81a0014030 con 0x7f81b01031c0 2024-09-21T05:56:28.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.348+0000 7f81b49ef640 1 -- 172.21.15.117:0/804512994 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f81b019b8a0 con 0x7f81b01031c0 2024-09-21T05:56:28.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.350+0000 7f819f7fe640 1 -- 172.21.15.117:0/804512994 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+23066 (secure 0 0 0) 0x7f81a00a7e20 con 0x7f81b01031c0 2024-09-21T05:56:28.352 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:56:28.352 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":76,"fsid":"ac8581a2-77dc-11ef-baf5-efdc52797490","created":"2024-09-21T05:46:47.021846+0000","modified":"2024-09-21T05:55:33.628948+0000","last_up_change":"2024-09-21T05:54:21.572849+0000","last_in_change":"2024-09-21T05:53:58.892597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:50:32.717389+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:55:10.604159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T05:55:12.581258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"68315502-fa6c-4a6e-a2d4-a7dde8071c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6803","nonce":3761478035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6805","nonce":3761478035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6809","nonce":3761478035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6807","nonce":3761478035}]},"public_addr":"172.21.15.117:6803/3761478035","cluster_addr":"172.21.15.117:6805/3761478035","heartbeat_back_addr":"172.21.15.117:6809/3761478035","heartbeat_front_addr":"172.21.15.117:6807/3761478035","state":["exists","up"]},{"osd":1,"uuid":"d8910a08-4ef3-49f2-a4db-e3588d85d12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6801","nonce":1737365588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6803","nonce":1737365588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6807","nonce":1737365588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6805","nonce":1737365588}]},"public_addr":"172.21.15.138:6801/1737365588","cluster_addr":"172.21.15.138:6803/1737365588","heartbeat_back_addr":"172.21.15.138:6807/1737365588","heartbeat_front_addr":"172.21.15.138:6805/1737365588","state":["exists","up"]},{"osd":2,"uuid":"94972378-a11f-4129-9e40-ad3fdec1d189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6801","nonce":3002534992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6803","nonce":3002534992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6807","nonce":3002534992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6805","nonce":3002534992}]},"public_addr":"172.21.15.192:6801/3002534992","cluster_addr":"172.21.15.192:6803/3002534992","heartbeat_back_addr":"172.21.15.192:6807/3002534992","heartbeat_front_addr":"172.21.15.192:6805/3002534992","state":["exists","up"]},{"osd":3,"uuid":"d4168c43-f0de-4445-945c-92d1cf270c33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6811","nonce":1222400293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6813","nonce":1222400293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6817","nonce":1222400293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6815","nonce":1222400293}]},"public_addr":"172.21.15.117:6811/1222400293","cluster_addr":"172.21.15.117:6813/1222400293","heartbeat_back_addr":"172.21.15.117:6817/1222400293","heartbeat_front_addr":"172.21.15.117:6815/1222400293","state":["exists","up"]},{"osd":4,"uuid":"28f1d7c0-8392-4a33-85c0-0de611cdd4b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6809","nonce":4294892870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6811","nonce":4294892870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6815","nonce":4294892870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6813","nonce":4294892870}]},"public_addr":"172.21.15.138:6809/4294892870","cluster_addr":"172.21.15.138:6811/4294892870","heartbeat_back_addr":"172.21.15.138:6815/4294892870","heartbeat_front_addr":"172.21.15.138:6813/4294892870","state":["exists","up"]},{"osd":5,"uuid":"ca902407-45e8-4bc5-bdbd-a729c1de8322","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6809","nonce":1776840019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6811","nonce":1776840019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6815","nonce":1776840019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6813","nonce":1776840019}]},"public_addr":"172.21.15.192:6809/1776840019","cluster_addr":"172.21.15.192:6811/1776840019","heartbeat_back_addr":"172.21.15.192:6815/1776840019","heartbeat_front_addr":"172.21.15.192:6813/1776840019","state":["exists","up"]},{"osd":6,"uuid":"6e507a91-ed09-4a33-bf0e-c4b593b3c5aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6819","nonce":1093341590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6821","nonce":1093341590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6825","nonce":1093341590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6823","nonce":1093341590}]},"public_addr":"172.21.15.117:6819/1093341590","cluster_addr":"172.21.15.117:6821/1093341590","heartbeat_back_addr":"172.21.15.117:6825/1093341590","heartbeat_front_addr":"172.21.15.117:6823/1093341590","state":["exists","up"]},{"osd":7,"uuid":"36a47dd7-67f7-4ebe-8759-3a8bb6bd2ea3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6817","nonce":1448179714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6819","nonce":1448179714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6823","nonce":1448179714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6821","nonce":1448179714}]},"public_addr":"172.21.15.138:6817/1448179714","cluster_addr":"172.21.15.138:6819/1448179714","heartbeat_back_addr":"172.21.15.138:6823/1448179714","heartbeat_front_addr":"172.21.15.138:6821/1448179714","state":["exists","up"]},{"osd":8,"uuid":"118e0b5d-2619-4512-94c5-8d07a0d3a28f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6817","nonce":3960902885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6819","nonce":3960902885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6823","nonce":3960902885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6821","nonce":3960902885}]},"public_addr":"172.21.15.192:6817/3960902885","cluster_addr":"172.21.15.192:6819/3960902885","heartbeat_back_addr":"172.21.15.192:6823/3960902885","heartbeat_front_addr":"172.21.15.192:6821/3960902885","state":["exists","up"]},{"osd":9,"uuid":"47543bc6-fa0b-4e72-9742-97dcda0b7f8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6827","nonce":3453855195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6829","nonce":3453855195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6833","nonce":3453855195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6831","nonce":3453855195}]},"public_addr":"172.21.15.117:6827/3453855195","cluster_addr":"172.21.15.117:6829/3453855195","heartbeat_back_addr":"172.21.15.117:6833/3453855195","heartbeat_front_addr":"172.21.15.117:6831/3453855195","state":["exists","up"]},{"osd":10,"uuid":"5e6708b4-7a85-4fa5-a03c-27d99f7d749f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6825","nonce":903392451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6827","nonce":903392451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6831","nonce":903392451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6829","nonce":903392451}]},"public_addr":"172.21.15.138:6825/903392451","cluster_addr":"172.21.15.138:6827/903392451","heartbeat_back_addr":"172.21.15.138:6831/903392451","heartbeat_front_addr":"172.21.15.138:6829/903392451","state":["exists","up"]},{"osd":11,"uuid":"24bc2ef7-e2b4-43c6-9e6c-43ae02390415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6825","nonce":899090479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6827","nonce":899090479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6831","nonce":899090479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6829","nonce":899090479}]},"public_addr":"172.21.15.192:6825/899090479","cluster_addr":"172.21.15.192:6827/899090479","heartbeat_back_addr":"172.21.15.192:6831/899090479","heartbeat_front_addr":"172.21.15.192:6829/899090479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:49:40.742217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:05.963655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:30.164495+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:56.598899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:51:21.940331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:51:47.777445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:52:12.869007+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:52:39.454054+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:03.569226+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:29.760863+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:55.788604+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:54:20.510184+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:0/2989394419":"2024-09-22T05:48:32.406654+0000","172.21.15.117:6801/1702676172":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/1766406059":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6800/2337036514":"2024-09-22T05:47:09.112435+0000","172.21.15.117:0/985798343":"2024-09-22T05:47:09.112435+0000","172.21.15.117:0/1083273222":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/473284565":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6801/2337036514":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6800/816635165":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/2642940197":"2024-09-22T05:48:32.406654+0000","172.21.15.117:6801/816635165":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/2225537433":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/3651584169":"2024-09-22T05:47:33.789146+0000","172.21.15.117:6800/1702676172":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/1284403882":"2024-09-22T05:47:33.789146+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:56:28.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.352+0000 7f81b49ef640 1 -- 172.21.15.117:0/804512994 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f81980b6620 msgr2=0x7f81980b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:28.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.352+0000 7f81b49ef640 1 --2- 172.21.15.117:0/804512994 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f81980b6620 0x7f81980b8b30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f81a402a930 tx=0x7f81a4031400 comp rx=0 tx=0).stop 2024-09-21T05:56:28.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.352+0000 7f81b49ef640 1 -- 172.21.15.117:0/804512994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81b01031c0 msgr2=0x7f81b019a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:28.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.352+0000 7f81b49ef640 1 --2- 172.21.15.117:0/804512994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81b01031c0 0x7f81b019a770 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f81a000c920 tx=0x7f81a000cde0 comp rx=0 tx=0).stop 2024-09-21T05:56:28.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.352+0000 7f81b49ef640 1 -- 172.21.15.117:0/804512994 shutdown_connections 2024-09-21T05:56:28.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.352+0000 7f81b49ef640 1 --2- 172.21.15.117:0/804512994 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f81980b6620 0x7f81980b8b30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:28.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.352+0000 7f81b49ef640 1 --2- 172.21.15.117:0/804512994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81b0108f50 0x7f81b019ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:28.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.352+0000 7f81b49ef640 1 --2- 172.21.15.117:0/804512994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81b0108600 0x7f81b019ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:28.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.352+0000 7f81b49ef640 1 --2- 172.21.15.117:0/804512994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81b01031c0 0x7f81b019a770 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:28.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.352+0000 7f81b49ef640 1 -- 172.21.15.117:0/804512994 >> 172.21.15.117:0/804512994 conn(0x7f81b0079e30 msgr2=0x7f81b0101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:28.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.353+0000 7f81b49ef640 1 -- 172.21.15.117:0/804512994 shutdown_connections 2024-09-21T05:56:28.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:28.353+0000 7f81b49ef640 1 -- 172.21.15.117:0/804512994 wait complete. 2024-09-21T05:56:28.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:28 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/804512994' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T05:56:28.990 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:28 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/804512994' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T05:56:29.018 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T05:56:29.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:28 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/804512994' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T05:56:29.261 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:29.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.723+0000 7f3c3abc7640 1 -- 172.21.15.117:0/2687062498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c34102aa0 msgr2=0x7f3c34102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:29.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.723+0000 7f3c3abc7640 1 --2- 172.21.15.117:0/2687062498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c34102aa0 0x7f3c34102eb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3c30009ca0 tx=0x7f3c3002a8c0 comp rx=0 tx=0).stop 2024-09-21T05:56:29.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.724+0000 7f3c3abc7640 1 -- 172.21.15.117:0/2687062498 shutdown_connections 2024-09-21T05:56:29.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.724+0000 7f3c3abc7640 1 --2- 172.21.15.117:0/2687062498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c34108d30 0x7f3c3410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:29.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.724+0000 7f3c3abc7640 1 --2- 172.21.15.117:0/2687062498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c341083e0 0x7f3c341087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:29.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.724+0000 7f3c3abc7640 1 --2- 172.21.15.117:0/2687062498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c34102aa0 0x7f3c34102eb0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:29.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.724+0000 7f3c3abc7640 1 -- 172.21.15.117:0/2687062498 >> 172.21.15.117:0/2687062498 conn(0x7f3c34079750 msgr2=0x7f3c34079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:29.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.724+0000 7f3c3abc7640 1 -- 172.21.15.117:0/2687062498 shutdown_connections 2024-09-21T05:56:29.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.724+0000 7f3c3abc7640 1 -- 172.21.15.117:0/2687062498 wait complete. 2024-09-21T05:56:29.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.724+0000 7f3c3abc7640 1 Processor -- start 2024-09-21T05:56:29.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.725+0000 7f3c3abc7640 1 -- start start 2024-09-21T05:56:29.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.725+0000 7f3c3abc7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c34102aa0 0x7f3c3419b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:29.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.725+0000 7f3c3abc7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c341083e0 0x7f3c3419b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:29.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.725+0000 7f3c3abc7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c34108d30 0x7f3c3419bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:29.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.725+0000 7f3c3abc7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3c341129f0 con 0x7f3c34102aa0 2024-09-21T05:56:29.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.725+0000 7f3c3abc7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3c34112870 con 0x7f3c34108d30 2024-09-21T05:56:29.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.725+0000 7f3c3abc7640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3c34112b70 con 0x7f3c341083e0 2024-09-21T05:56:29.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.725+0000 7f3c38963640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c34102aa0 0x7f3c3419b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:29.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.725+0000 7f3c38963640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c34102aa0 0x7f3c3419b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40566/0 (socket says 172.21.15.117:40566) 2024-09-21T05:56:29.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.725+0000 7f3c38963640 1 -- 172.21.15.117:0/4254178683 learned_addr learned my addr 172.21.15.117:0/4254178683 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:29.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.725+0000 7f3c2bfff640 1 --2- 172.21.15.117:0/4254178683 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c341083e0 0x7f3c3419b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:29.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.725+0000 7f3c39164640 1 --2- 172.21.15.117:0/4254178683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c34108d30 0x7f3c3419bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:29.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.725+0000 7f3c38963640 1 -- 172.21.15.117:0/4254178683 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c341083e0 msgr2=0x7f3c3419b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:29.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.725+0000 7f3c38963640 1 --2- 172.21.15.117:0/4254178683 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c341083e0 0x7f3c3419b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:29.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.725+0000 7f3c38963640 1 -- 172.21.15.117:0/4254178683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c34108d30 msgr2=0x7f3c3419bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:29.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.725+0000 7f3c38963640 1 --2- 172.21.15.117:0/4254178683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c34108d30 0x7f3c3419bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:29.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.725+0000 7f3c38963640 1 -- 172.21.15.117:0/4254178683 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c341a65a0 con 0x7f3c34102aa0 2024-09-21T05:56:29.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.725+0000 7f3c38963640 1 --2- 172.21.15.117:0/4254178683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c34102aa0 0x7f3c3419b2e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3c3002ade0 tx=0x7f3c30003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:29.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.726+0000 7f3c29ffb640 1 -- 172.21.15.117:0/4254178683 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c30003d60 con 0x7f3c34102aa0 2024-09-21T05:56:29.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.726+0000 7f3c29ffb640 1 -- 172.21.15.117:0/4254178683 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c300044f0 con 0x7f3c34102aa0 2024-09-21T05:56:29.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.726+0000 7f3c3abc7640 1 -- 172.21.15.117:0/4254178683 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c341a6830 con 0x7f3c34102aa0 2024-09-21T05:56:29.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.726+0000 7f3c29ffb640 1 -- 172.21.15.117:0/4254178683 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3002fd10 con 0x7f3c34102aa0 2024-09-21T05:56:29.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.726+0000 7f3c3abc7640 1 -- 172.21.15.117:0/4254178683 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c341a6d70 con 0x7f3c34102aa0 2024-09-21T05:56:29.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.727+0000 7f3c3abc7640 1 -- 172.21.15.117:0/4254178683 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c34103d20 con 0x7f3c34102aa0 2024-09-21T05:56:29.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.728+0000 7f3c29ffb640 1 -- 172.21.15.117:0/4254178683 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f3c300046a0 con 0x7f3c34102aa0 2024-09-21T05:56:29.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.729+0000 7f3c29ffb640 1 --2- 172.21.15.117:0/4254178683 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3c1c0b65f0 0x7f3c1c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:29.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.729+0000 7f3c2bfff640 1 --2- 172.21.15.117:0/4254178683 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3c1c0b65f0 0x7f3c1c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:29.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.729+0000 7f3c29ffb640 1 -- 172.21.15.117:0/4254178683 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f3c30108c50 con 0x7f3c34102aa0 2024-09-21T05:56:29.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.729+0000 7f3c2bfff640 1 --2- 172.21.15.117:0/4254178683 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3c1c0b65f0 0x7f3c1c0b8b00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3c34107f70 tx=0x7f3c2400a250 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:29.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.732+0000 7f3c29ffb640 1 -- 172.21.15.117:0/4254178683 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3c300cf2e0 con 0x7f3c34102aa0 2024-09-21T05:56:29.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.948+0000 7f3c3abc7640 1 -- 172.21.15.117:0/4254178683 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c34102eb0 con 0x7f3c34102aa0 2024-09-21T05:56:29.950 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:29 smithi117 ceph-mon[30947]: pgmap v302: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T05:56:29.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.949+0000 7f3c29ffb640 1 -- 172.21.15.117:0/4254178683 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+12001 (secure 0 0 0) 0x7f3c34102eb0 con 0x7f3c34102aa0 2024-09-21T05:56:29.951 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:56:29.951 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":26,"btime":"2024-09-21T05:55:26:209435+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3227057504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3227057504},{"type":"v1","addr":"172.21.15.117:6835","nonce":3227057504}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3303829183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3303829183},{"type":"v1","addr":"172.21.15.117:6837","nonce":3303829183}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T05:55:14.555897+0000","modified":"2024-09-21T05:55:26.209411+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24638,"mds_1":24637,"mds_2":24628,"mds_3":24616,"mds_4":24602},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6839/2314572411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2314572411},{"type":"v1","addr":"172.21.15.117:6839","nonce":2314572411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6841/3378096561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":3378096561},{"type":"v1","addr":"172.21.15.117:6841","nonce":3378096561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/3725629862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3725629862},{"type":"v1","addr":"172.21.15.138:6833","nonce":3725629862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/4276801396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4276801396},{"type":"v1","addr":"172.21.15.138:6835","nonce":4276801396}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6833/2224910879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2224910879},{"type":"v1","addr":"172.21.15.192:6833","nonce":2224910879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.138:6837/2540335374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2540335374},{"type":"v1","addr":"172.21.15.138:6837","nonce":2540335374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24616":{"gid":24616,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.192:6835/3530688860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3530688860},{"type":"v1","addr":"172.21.15.192:6835","nonce":3530688860}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.192:6837/2683860842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2683860842},{"type":"v1","addr":"172.21.15.192:6837","nonce":2683860842}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.192:6839/1727623231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1727623231},{"type":"v1","addr":"172.21.15.192:6839","nonce":1727623231}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/3230372576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":3230372576},{"type":"v1","addr":"172.21.15.138:6839","nonce":3230372576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24638,24637,24616,24628,24602]},"id":1}]} 2024-09-21T05:56:29.951 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 26 2024-09-21T05:56:29.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.951+0000 7f3c3abc7640 1 -- 172.21.15.117:0/4254178683 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3c1c0b65f0 msgr2=0x7f3c1c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:29.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.951+0000 7f3c3abc7640 1 --2- 172.21.15.117:0/4254178683 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3c1c0b65f0 0x7f3c1c0b8b00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3c34107f70 tx=0x7f3c2400a250 comp rx=0 tx=0).stop 2024-09-21T05:56:29.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.951+0000 7f3c3abc7640 1 -- 172.21.15.117:0/4254178683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c34102aa0 msgr2=0x7f3c3419b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:29.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.951+0000 7f3c3abc7640 1 --2- 172.21.15.117:0/4254178683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c34102aa0 0x7f3c3419b2e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3c3002ade0 tx=0x7f3c30003970 comp rx=0 tx=0).stop 2024-09-21T05:56:29.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.951+0000 7f3c3abc7640 1 -- 172.21.15.117:0/4254178683 shutdown_connections 2024-09-21T05:56:29.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.951+0000 7f3c3abc7640 1 --2- 172.21.15.117:0/4254178683 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3c1c0b65f0 0x7f3c1c0b8b00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:29.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.951+0000 7f3c3abc7640 1 --2- 172.21.15.117:0/4254178683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c34108d30 0x7f3c3419bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:29.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.951+0000 7f3c3abc7640 1 --2- 172.21.15.117:0/4254178683 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c341083e0 0x7f3c3419b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:29.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.951+0000 7f3c3abc7640 1 --2- 172.21.15.117:0/4254178683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c34102aa0 0x7f3c3419b2e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:29.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.951+0000 7f3c3abc7640 1 -- 172.21.15.117:0/4254178683 >> 172.21.15.117:0/4254178683 conn(0x7f3c34079750 msgr2=0x7f3c34106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:29.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.951+0000 7f3c3abc7640 1 -- 172.21.15.117:0/4254178683 shutdown_connections 2024-09-21T05:56:29.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:29.951+0000 7f3c3abc7640 1 -- 172.21.15.117:0/4254178683 wait complete. 2024-09-21T05:56:29.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:29 smithi192 ceph-mon[34700]: pgmap v302: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T05:56:30.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:29 smithi138 ceph-mon[34391]: pgmap v302: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T05:56:30.626 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T05:55:14.555897+0000', 'modified': '2024-09-21T05:55:26.209411+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24638, 'mds_1': 24637, 'mds_2': 24628, 'mds_3': 24616, 'mds_4': 24602}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14838': {'gid': 14838, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.117:6839/2314572411', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6838', 'nonce': 2314572411}, {'type': 'v1', 'addr': '172.21.15.117:6839', 'nonce': 2314572411}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14850': {'gid': 14850, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.117:6841/3378096561', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6840', 'nonce': 3378096561}, {'type': 'v1', 'addr': '172.21.15.117:6841', 'nonce': 3378096561}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24545': {'gid': 24545, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.138:6833/3725629862', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6832', 'nonce': 3725629862}, {'type': 'v1', 'addr': '172.21.15.138:6833', 'nonce': 3725629862}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24572': {'gid': 24572, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.138:6835/4276801396', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6834', 'nonce': 4276801396}, {'type': 'v1', 'addr': '172.21.15.138:6835', 'nonce': 4276801396}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24586': {'gid': 24586, 'name': 'l', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.192:6833/2224910879', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6832', 'nonce': 2224910879}, {'type': 'v1', 'addr': '172.21.15.192:6833', 'nonce': 2224910879}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24602': {'gid': 24602, 'name': 'h', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.138:6837/2540335374', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6836', 'nonce': 2540335374}, {'type': 'v1', 'addr': '172.21.15.138:6837', 'nonce': 2540335374}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24616': {'gid': 24616, 'name': 'f', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.192:6835/3530688860', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6834', 'nonce': 3530688860}, {'type': 'v1', 'addr': '172.21.15.192:6835', 'nonce': 3530688860}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24628': {'gid': 24628, 'name': 'c', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.192:6837/2683860842', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6836', 'nonce': 2683860842}, {'type': 'v1', 'addr': '172.21.15.192:6837', 'nonce': 2683860842}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24637': {'gid': 24637, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.192:6839/1727623231', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6838', 'nonce': 1727623231}, {'type': 'v1', 'addr': '172.21.15.192:6839', 'nonce': 1727623231}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24638': {'gid': 24638, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.138:6839/3230372576', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6838', 'nonce': 3230372576}, {'type': 'v1', 'addr': '172.21.15.138:6839', 'nonce': 3230372576}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24638, 'qdb_cluster': [24638, 24637, 24616, 24628, 24602]} 2024-09-21T05:56:30.627 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-21T05:56:30.627 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.h status 2024-09-21T05:56:30.887 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:30.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:30 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/4254178683' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:56:30.983 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:30 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/4254178683' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:56:31.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:30 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/4254178683' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:56:31.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.313+0000 7f2afba34640 1 -- 172.21.15.117:0/3161805245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2af4103b60 msgr2=0x7f2af4103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:31.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.313+0000 7f2afba34640 1 --2- 172.21.15.117:0/3161805245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2af4103b60 0x7f2af4103f70 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4009900 tx=0x7f2ae402a410 comp rx=0 tx=0).stop 2024-09-21T05:56:31.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.314+0000 7f2afba34640 1 -- 172.21.15.117:0/3161805245 shutdown_connections 2024-09-21T05:56:31.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.314+0000 7f2afba34640 1 --2- 172.21.15.117:0/3161805245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2af4109df0 0x7f2af4110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.314+0000 7f2afba34640 1 --2- 172.21.15.117:0/3161805245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2af41094a0 0x7f2af41098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.314+0000 7f2afba34640 1 --2- 172.21.15.117:0/3161805245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2af4103b60 0x7f2af4103f70 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.314+0000 7f2afba34640 1 -- 172.21.15.117:0/3161805245 >> 172.21.15.117:0/3161805245 conn(0x7f2af407b850 msgr2=0x7f2af407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:31.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.314+0000 7f2afba34640 1 -- 172.21.15.117:0/3161805245 shutdown_connections 2024-09-21T05:56:31.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.314+0000 7f2afba34640 1 -- 172.21.15.117:0/3161805245 wait complete. 2024-09-21T05:56:31.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.314+0000 7f2afba34640 1 Processor -- start 2024-09-21T05:56:31.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.314+0000 7f2afba34640 1 -- start start 2024-09-21T05:56:31.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.315+0000 7f2afba34640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2af4103b60 0x7f2af419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:31.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.315+0000 7f2afba34640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2af41094a0 0x7f2af419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:31.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.315+0000 7f2afba34640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2af4109df0 0x7f2af419fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:31.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.315+0000 7f2afba34640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2af4113ab0 con 0x7f2af41094a0 2024-09-21T05:56:31.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.315+0000 7f2afba34640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2af4113930 con 0x7f2af4109df0 2024-09-21T05:56:31.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.315+0000 7f2afba34640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2af4113c30 con 0x7f2af4103b60 2024-09-21T05:56:31.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.315+0000 7f2af8fcf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2af41094a0 0x7f2af419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:31.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.315+0000 7f2af8fcf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2af41094a0 0x7f2af419aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40580/0 (socket says 172.21.15.117:40580) 2024-09-21T05:56:31.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.315+0000 7f2af8fcf640 1 -- 172.21.15.117:0/2813456456 learned_addr learned my addr 172.21.15.117:0/2813456456 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:31.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.315+0000 7f2af97d0640 1 --2- 172.21.15.117:0/2813456456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2af4103b60 0x7f2af419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:31.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.315+0000 7f2af8fcf640 1 -- 172.21.15.117:0/2813456456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2af4103b60 msgr2=0x7f2af419a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:31.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.315+0000 7f2af8fcf640 1 --2- 172.21.15.117:0/2813456456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2af4103b60 0x7f2af419a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.315+0000 7f2af8fcf640 1 -- 172.21.15.117:0/2813456456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2af4109df0 msgr2=0x7f2af419fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:31.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.315+0000 7f2af8fcf640 1 --2- 172.21.15.117:0/2813456456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2af4109df0 0x7f2af419fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.315+0000 7f2af8fcf640 1 -- 172.21.15.117:0/2813456456 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af41a0490 con 0x7f2af41094a0 2024-09-21T05:56:31.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.315+0000 7f2af8fcf640 1 --2- 172.21.15.117:0/2813456456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2af41094a0 0x7f2af419aac0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2ae800d950 tx=0x7f2ae800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:31.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.316+0000 7f2af27fc640 1 -- 172.21.15.117:0/2813456456 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8017070 con 0x7f2af41094a0 2024-09-21T05:56:31.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.316+0000 7f2af27fc640 1 -- 172.21.15.117:0/2813456456 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ae8009c30 con 0x7f2af41094a0 2024-09-21T05:56:31.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.316+0000 7f2afba34640 1 -- 172.21.15.117:0/2813456456 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2af41a0780 con 0x7f2af41094a0 2024-09-21T05:56:31.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.316+0000 7f2af27fc640 1 -- 172.21.15.117:0/2813456456 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae80047a0 con 0x7f2af41094a0 2024-09-21T05:56:31.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.316+0000 7f2afba34640 1 -- 172.21.15.117:0/2813456456 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2af41a6ee0 con 0x7f2af41094a0 2024-09-21T05:56:31.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.318+0000 7f2af27fc640 1 -- 172.21.15.117:0/2813456456 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f2ae8004020 con 0x7f2af41094a0 2024-09-21T05:56:31.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.318+0000 7f2b01fa8740 1 Processor -- start 2024-09-21T05:56:31.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.318+0000 7f2b01fa8740 1 -- start start 2024-09-21T05:56:31.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.319+0000 7f2b01fa8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a547af60 0x55f6a5820f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:31.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.319+0000 7f2b01fa8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6a54520a0 0x55f6a5821440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:31.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.319+0000 7f2b01fa8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6a545a280 0x55f6a5821980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:31.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.319+0000 7f2b01fa8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f6a5798c10 con 0x55f6a545a280 2024-09-21T05:56:31.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.319+0000 7f2b01fa8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f6a5788040 con 0x55f6a547af60 2024-09-21T05:56:31.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.319+0000 7f2b01fa8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f6a578be50 con 0x55f6a54520a0 2024-09-21T05:56:31.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.319+0000 7f2af8fcf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6a545a280 0x55f6a5821980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:31.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.319+0000 7f2af8fcf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6a545a280 0x55f6a5821980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40596/0 (socket says 172.21.15.117:40596) 2024-09-21T05:56:31.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.319+0000 7f2af8fcf640 1 -- 172.21.15.117:0/3323092369 learned_addr learned my addr 172.21.15.117:0/3323092369 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:31.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.319+0000 7f2af97d0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6a54520a0 0x55f6a5821440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:31.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.319+0000 7f2af9fd1640 1 --2- 172.21.15.117:0/3323092369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a547af60 0x55f6a5820f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:31.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.319+0000 7f2af27fc640 1 --2- 172.21.15.117:0/2813456456 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2adc0b65f0 0x7f2adc0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:31.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.319+0000 7f2af8fcf640 1 -- 172.21.15.117:0/3323092369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6a54520a0 msgr2=0x55f6a5821440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:31.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.319+0000 7f2af8fcf640 1 --2- 172.21.15.117:0/3323092369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6a54520a0 0x55f6a5821440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.319+0000 7f2af8fcf640 1 -- 172.21.15.117:0/3323092369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a547af60 msgr2=0x55f6a5820f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:31.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.319+0000 7f2af8fcf640 1 --2- 172.21.15.117:0/3323092369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a547af60 0x55f6a5820f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.319+0000 7f2af8fcf640 1 -- 172.21.15.117:0/3323092369 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6a54e6540 con 0x55f6a545a280 2024-09-21T05:56:31.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.319+0000 7f2af9fd1640 1 --2- 172.21.15.117:0/2813456456 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2adc0b65f0 0x7f2adc0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:31.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.320+0000 7f2af27fc640 1 -- 172.21.15.117:0/2813456456 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f2ae80e72e0 con 0x7f2af41094a0 2024-09-21T05:56:31.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.320+0000 7f2af8fcf640 1 --2- 172.21.15.117:0/3323092369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6a545a280 0x55f6a5821980 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80e4b30 tx=0x7f2ae8075d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff41481030f5616 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:31.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.320+0000 7f2adb7fe640 1 -- 172.21.15.117:0/3323092369 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae80acbc0 con 0x55f6a545a280 2024-09-21T05:56:31.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.320+0000 7f2adb7fe640 1 -- 172.21.15.117:0/3323092369 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ae8075ba0 con 0x55f6a545a280 2024-09-21T05:56:31.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.320+0000 7f2adb7fe640 1 -- 172.21.15.117:0/3323092369 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae80eab40 con 0x55f6a545a280 2024-09-21T05:56:31.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.320+0000 7f2af9fd1640 1 --2- 172.21.15.117:0/2813456456 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2adc0b65f0 0x7f2adc0b8b00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2aec005c40 tx=0x7f2aec004a40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:31.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.321+0000 7f2b01fa8740 1 -- 172.21.15.117:0/3323092369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6a545a280 msgr2=0x55f6a5821980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:31.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.321+0000 7f2b01fa8740 1 --2- 172.21.15.117:0/3323092369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6a545a280 0x55f6a5821980 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80e4b30 tx=0x7f2ae8075d90 comp rx=0 tx=0).stop 2024-09-21T05:56:31.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.321+0000 7f2b01fa8740 1 -- 172.21.15.117:0/3323092369 shutdown_connections 2024-09-21T05:56:31.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.321+0000 7f2b01fa8740 1 --2- 172.21.15.117:0/3323092369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6a545a280 0x55f6a5821980 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.321+0000 7f2b01fa8740 1 --2- 172.21.15.117:0/3323092369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6a54520a0 0x55f6a5821440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.321+0000 7f2b01fa8740 1 --2- 172.21.15.117:0/3323092369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a547af60 0x55f6a5820f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.321+0000 7f2b01fa8740 1 -- 172.21.15.117:0/3323092369 >> 172.21.15.117:0/3323092369 conn(0x55f6a546f350 msgr2=0x55f6a581edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:31.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.321+0000 7f2b01fa8740 1 -- 172.21.15.117:0/3323092369 shutdown_connections 2024-09-21T05:56:31.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.321+0000 7f2b01fa8740 1 -- 172.21.15.117:0/3323092369 wait complete. 2024-09-21T05:56:31.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.321+0000 7f2b01fa8740 1 -- start start 2024-09-21T05:56:31.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.321+0000 7f2b01fa8740 1 Processor -- start 2024-09-21T05:56:31.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.322+0000 7f2b01fa8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a54520a0 0x55f6a582c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:31.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.322+0000 7f2b01fa8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6a545a280 0x55f6a582cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:31.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.322+0000 7f2b01fa8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6a547af60 0x55f6a5838970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:31.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.322+0000 7f2b01fa8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f6a55e95f0 con 0x55f6a545a280 2024-09-21T05:56:31.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.322+0000 7f2b01fa8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f6a5723010 con 0x55f6a54520a0 2024-09-21T05:56:31.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.322+0000 7f2b01fa8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f6a582f680 con 0x55f6a547af60 2024-09-21T05:56:31.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.322+0000 7f2b01fa8740 10 client.0 _ll_register_callbacks cb 0x55f6a5782340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:56:31.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.322+0000 7f2af8fcf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6a545a280 0x55f6a582cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:31.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.322+0000 7f2af8fcf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6a545a280 0x55f6a582cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40598/0 (socket says 172.21.15.117:40598) 2024-09-21T05:56:31.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.322+0000 7f2af8fcf640 1 -- 172.21.15.117:0/3973795448 learned_addr learned my addr 172.21.15.117:0/3973795448 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:31.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.322+0000 7f2af97d0640 1 --2- 172.21.15.117:0/3973795448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a54520a0 0x55f6a582c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:31.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.322+0000 7f2af9fd1640 1 --2- 172.21.15.117:0/3973795448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6a547af60 0x55f6a5838970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:31.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.322+0000 7f2af8fcf640 1 -- 172.21.15.117:0/3973795448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6a547af60 msgr2=0x55f6a5838970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:31.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.322+0000 7f2af8fcf640 1 --2- 172.21.15.117:0/3973795448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6a547af60 0x55f6a5838970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.322+0000 7f2af8fcf640 1 -- 172.21.15.117:0/3973795448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a54520a0 msgr2=0x55f6a582c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:31.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.322+0000 7f2af8fcf640 1 --2- 172.21.15.117:0/3973795448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a54520a0 0x55f6a582c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.322+0000 7f2af8fcf640 1 -- 172.21.15.117:0/3973795448 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6a5750a00 con 0x55f6a545a280 2024-09-21T05:56:31.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.322+0000 7f2af8fcf640 1 --2- 172.21.15.117:0/3973795448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6a545a280 0x55f6a582cef0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80e4c70 tx=0x7f2ae80e5070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ef2ba0ba796b79 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:31.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.322+0000 7f2ad8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:56:31.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.323+0000 7f2ad8ff9640 1 -- 172.21.15.117:0/3973795448 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae80ac9e0 con 0x55f6a545a280 2024-09-21T05:56:31.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.323+0000 7f2ad8ff9640 1 -- 172.21.15.117:0/3973795448 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f2ae80a8c40 con 0x55f6a545a280 2024-09-21T05:56:31.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.323+0000 7f2b01fa8740 1 -- 172.21.15.117:0/3973795448 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f6a579ab60 con 0x55f6a545a280 2024-09-21T05:56:31.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.323+0000 7f2ad8ff9640 1 -- 172.21.15.117:0/3973795448 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2ae80acbc0 con 0x55f6a545a280 2024-09-21T05:56:31.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.323+0000 7f2b01fa8740 10 client.15216 fetch_fsmap learned FSMap version 26 2024-09-21T05:56:31.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.323+0000 7f2b01fa8740 1 -- 172.21.15.117:0/3973795448 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f6a583b580 con 0x55f6a545a280 2024-09-21T05:56:31.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.323+0000 7f2ad8ff9640 1 -- 172.21.15.117:0/3973795448 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f6a579ab60 con 0x55f6a545a280 2024-09-21T05:56:31.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.323+0000 7f2ad8ff9640 1 -- 172.21.15.117:0/3973795448 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f2ae80a94e0 con 0x55f6a545a280 2024-09-21T05:56:31.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.323+0000 7f2b01fa8740 10 client.15216 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T05:56:31.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.323+0000 7f2b01fa8740 10 client.15216 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T05:56:31.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.323+0000 7f2b01fa8740 20 client.15216 populate_metadata read hostname 'smithi117' 2024-09-21T05:56:31.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.323+0000 7f2b01fa8740 1 --2- 172.21.15.117:0/3973795448 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x55f6a54398e0 0x55f6a583eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:31.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.323+0000 7f2b01fa8740 4 client.15216 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:56:31.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.323+0000 7f2b01fa8740 1 -- 172.21.15.117:0/3973795448 --> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6a5796610 con 0x55f6a54398e0 2024-09-21T05:56:31.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.323+0000 7f2af97d0640 1 --2- 172.21.15.117:0/3973795448 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x55f6a54398e0 0x55f6a583eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:31.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.324+0000 7f2ad8ff9640 10 client.15216 ms_handle_connect on v2:172.21.15.138:6836/2540335374 2024-09-21T05:56:31.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.324+0000 7f2af97d0640 1 --2- 172.21.15.117:0/3973795448 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x55f6a54398e0 0x55f6a583eb70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fd24f7ab17702fe0 server_cookie=535f3fe0d739f628 in_seq=0 out_seq=0 2024-09-21T05:56:31.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.325+0000 7f2ad8ff9640 1 -- 172.21.15.117:0/3973795448 <== mds.4 v2:172.21.15.138:6836/2540335374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f6a5796610 con 0x55f6a54398e0 2024-09-21T05:56:31.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.325+0000 7f2ad8ff9640 10 client.15216 handle_command_reply: tid=0 2024-09-21T05:56:31.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.325+0000 7f2b01fa8740 1 client.15216 shutdown 2024-09-21T05:56:31.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.325+0000 7f2b01fa8740 1 -- 172.21.15.117:0/3973795448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6a545a280 msgr2=0x55f6a582cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:31.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.325+0000 7f2b01fa8740 1 --2- 172.21.15.117:0/3973795448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6a545a280 0x55f6a582cef0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80e4c70 tx=0x7f2ae80e5070 comp rx=0 tx=0).stop 2024-09-21T05:56:31.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.325+0000 7f2b01fa8740 1 -- 172.21.15.117:0/3973795448 shutdown_connections 2024-09-21T05:56:31.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.325+0000 7f2b01fa8740 1 --2- 172.21.15.117:0/3973795448 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x55f6a54398e0 0x55f6a583eb70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.325+0000 7f2ad8ff9640 0 client.15216 ms_handle_reset on v2:172.21.15.138:6836/2540335374 2024-09-21T05:56:31.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.325+0000 7f2b01fa8740 1 --2- 172.21.15.117:0/3973795448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6a545a280 0x55f6a582cef0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.325+0000 7f2b01fa8740 1 --2- 172.21.15.117:0/3973795448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6a547af60 0x55f6a5838970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.325+0000 7f2b01fa8740 1 --2- 172.21.15.117:0/3973795448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a54520a0 0x55f6a582c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.325+0000 7f2b01fa8740 1 -- 172.21.15.117:0/3973795448 >> 172.21.15.117:0/3973795448 conn(0x55f6a546f350 msgr2=0x55f6a5821f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:31.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.325+0000 7f2b01fa8740 1 -- 172.21.15.117:0/3973795448 shutdown_connections 2024-09-21T05:56:31.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.325+0000 7f2b01fa8740 1 -- 172.21.15.117:0/3973795448 wait complete. 2024-09-21T05:56:31.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.325+0000 7f2b01fa8740 20 client.15216 trim_cache size 0 max 16384 2024-09-21T05:56:31.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.344+0000 7f2b01fa8740 1 Processor -- start 2024-09-21T05:56:31.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.344+0000 7f2b01fa8740 1 -- start start 2024-09-21T05:56:31.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.344+0000 7f2b01fa8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6a54398e0 0x55f6a5827020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:31.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.344+0000 7f2b01fa8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a54520a0 0x55f6a582e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:31.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.344+0000 7f2b01fa8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6a545a280 0x55f6a58388e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:31.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.344+0000 7f2b01fa8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f6a583ceb0 con 0x55f6a545a280 2024-09-21T05:56:31.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.344+0000 7f2b01fa8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f6a583d580 con 0x55f6a54520a0 2024-09-21T05:56:31.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.344+0000 7f2b01fa8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f6a583dc50 con 0x55f6a54398e0 2024-09-21T05:56:31.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.344+0000 7f2af9fd1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6a545a280 0x55f6a58388e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:31.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.344+0000 7f2af9fd1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6a545a280 0x55f6a58388e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40608/0 (socket says 172.21.15.117:40608) 2024-09-21T05:56:31.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.344+0000 7f2af9fd1640 1 -- 172.21.15.117:0/3738649631 learned_addr learned my addr 172.21.15.117:0/3738649631 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:31.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.344+0000 7f2af97d0640 1 --2- 172.21.15.117:0/3738649631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6a54398e0 0x55f6a5827020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:31.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.344+0000 7f2af8fcf640 1 --2- 172.21.15.117:0/3738649631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a54520a0 0x55f6a582e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:31.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.344+0000 7f2af9fd1640 1 -- 172.21.15.117:0/3738649631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6a54398e0 msgr2=0x55f6a5827020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:31.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.344+0000 7f2af9fd1640 1 --2- 172.21.15.117:0/3738649631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6a54398e0 0x55f6a5827020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.344+0000 7f2af9fd1640 1 -- 172.21.15.117:0/3738649631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a54520a0 msgr2=0x55f6a582e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:31.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.344+0000 7f2af9fd1640 1 --2- 172.21.15.117:0/3738649631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a54520a0 0x55f6a582e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.344+0000 7f2af9fd1640 1 -- 172.21.15.117:0/3738649631 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6a582da70 con 0x55f6a545a280 2024-09-21T05:56:31.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.344+0000 7f2af9fd1640 1 --2- 172.21.15.117:0/3738649631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6a545a280 0x55f6a58388e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2aec0049b0 tx=0x7f2aec014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520b3d77f7b58771 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:31.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.345+0000 7f2adaffd640 1 -- 172.21.15.117:0/3738649631 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec01f630 con 0x55f6a545a280 2024-09-21T05:56:31.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.345+0000 7f2adaffd640 1 -- 172.21.15.117:0/3738649631 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2aec016070 con 0x55f6a545a280 2024-09-21T05:56:31.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.345+0000 7f2adaffd640 1 -- 172.21.15.117:0/3738649631 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec01eaa0 con 0x55f6a545a280 2024-09-21T05:56:31.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.346+0000 7f2b01fa8740 1 -- 172.21.15.117:0/3738649631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6a545a280 msgr2=0x55f6a58388e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:31.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.346+0000 7f2b01fa8740 1 --2- 172.21.15.117:0/3738649631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6a545a280 0x55f6a58388e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2aec0049b0 tx=0x7f2aec014000 comp rx=0 tx=0).stop 2024-09-21T05:56:31.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.346+0000 7f2b01fa8740 1 -- 172.21.15.117:0/3738649631 shutdown_connections 2024-09-21T05:56:31.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.346+0000 7f2b01fa8740 1 --2- 172.21.15.117:0/3738649631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6a545a280 0x55f6a58388e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.346+0000 7f2b01fa8740 1 --2- 172.21.15.117:0/3738649631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a54520a0 0x55f6a582e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.346+0000 7f2b01fa8740 1 --2- 172.21.15.117:0/3738649631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6a54398e0 0x55f6a5827020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.346+0000 7f2b01fa8740 1 -- 172.21.15.117:0/3738649631 >> 172.21.15.117:0/3738649631 conn(0x55f6a546f350 msgr2=0x55f6a582dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:31.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.346+0000 7f2b01fa8740 1 -- 172.21.15.117:0/3738649631 shutdown_connections 2024-09-21T05:56:31.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.346+0000 7f2b01fa8740 1 -- 172.21.15.117:0/3738649631 wait complete. 2024-09-21T05:56:31.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.346+0000 7f2b01fa8740 1 -- start start 2024-09-21T05:56:31.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.346+0000 7f2b01fa8740 1 Processor -- start 2024-09-21T05:56:31.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.347+0000 7f2b01fa8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6a54398e0 0x55f6a58411e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:31.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.347+0000 7f2b01fa8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6a54520a0 0x55f6a5841720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:31.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.347+0000 7f2b01fa8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a545a280 0x55f6a5846920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:31.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.347+0000 7f2b01fa8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f6a583e9f0 con 0x55f6a54520a0 2024-09-21T05:56:31.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.347+0000 7f2b01fa8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f6a583e320 con 0x55f6a545a280 2024-09-21T05:56:31.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.347+0000 7f2b01fa8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f6a583f0c0 con 0x55f6a54398e0 2024-09-21T05:56:31.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.347+0000 7f2b01fa8740 10 client.0 _ll_register_callbacks cb 0x7f2aec009480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:56:31.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.347+0000 7f2af8fcf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6a54520a0 0x55f6a5841720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:31.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.347+0000 7f2af8fcf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6a54520a0 0x55f6a5841720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40616/0 (socket says 172.21.15.117:40616) 2024-09-21T05:56:31.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.347+0000 7f2af8fcf640 1 -- 172.21.15.117:0/2112427689 learned_addr learned my addr 172.21.15.117:0/2112427689 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:31.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.347+0000 7f2af9fd1640 1 --2- 172.21.15.117:0/2112427689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a545a280 0x55f6a5846920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:31.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.347+0000 7f2af97d0640 1 --2- 172.21.15.117:0/2112427689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6a54398e0 0x55f6a58411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:31.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.347+0000 7f2af8fcf640 1 -- 172.21.15.117:0/2112427689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6a54398e0 msgr2=0x55f6a58411e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:31.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.347+0000 7f2af8fcf640 1 --2- 172.21.15.117:0/2112427689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6a54398e0 0x55f6a58411e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.347+0000 7f2af8fcf640 1 -- 172.21.15.117:0/2112427689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a545a280 msgr2=0x55f6a5846920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:31.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.347+0000 7f2af8fcf640 1 --2- 172.21.15.117:0/2112427689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a545a280 0x55f6a5846920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.347+0000 7f2af8fcf640 1 -- 172.21.15.117:0/2112427689 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6a5846e60 con 0x55f6a54520a0 2024-09-21T05:56:31.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.347+0000 7f2af97d0640 1 --2- 172.21.15.117:0/2112427689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6a54398e0 0x55f6a58411e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:31.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.347+0000 7f2af8fcf640 1 --2- 172.21.15.117:0/2112427689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6a54520a0 0x55f6a5841720 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80e5170 tx=0x7f2ae80a9220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a810ad051d63f05 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:31.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.347+0000 7f2ad8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:56:31.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.347+0000 7f2ad8ff9640 1 -- 172.21.15.117:0/2112427689 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae80ac9e0 con 0x55f6a54520a0 2024-09-21T05:56:31.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.347+0000 7f2ad8ff9640 1 -- 172.21.15.117:0/2112427689 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f2ae80c3130 con 0x55f6a54520a0 2024-09-21T05:56:31.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.347+0000 7f2b01fa8740 1 -- 172.21.15.117:0/2112427689 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2aec016070 con 0x55f6a54520a0 2024-09-21T05:56:31.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.348+0000 7f2ad8ff9640 1 -- 172.21.15.117:0/2112427689 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2ae80b6ce0 con 0x55f6a54520a0 2024-09-21T05:56:31.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.348+0000 7f2b01fa8740 10 client.15222 fetch_fsmap learned FSMap version 26 2024-09-21T05:56:31.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.348+0000 7f2b01fa8740 1 -- 172.21.15.117:0/2112427689 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f6a58aa680 con 0x55f6a54520a0 2024-09-21T05:56:31.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.348+0000 7f2ad8ff9640 1 -- 172.21.15.117:0/2112427689 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2aec016070 con 0x55f6a54520a0 2024-09-21T05:56:31.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.348+0000 7f2ad8ff9640 1 -- 172.21.15.117:0/2112427689 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f2ae80ccbd0 con 0x55f6a54520a0 2024-09-21T05:56:31.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.348+0000 7f2b01fa8740 10 client.15222 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T05:56:31.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.348+0000 7f2b01fa8740 10 client.15222 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T05:56:31.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.348+0000 7f2b01fa8740 20 client.15222 populate_metadata read hostname 'smithi117' 2024-09-21T05:56:31.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.348+0000 7f2b01fa8740 1 --2- 172.21.15.117:0/2112427689 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x55f6a547af60 0x55f6a58add20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:31.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.348+0000 7f2b01fa8740 4 client.15222 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T05:56:31.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.348+0000 7f2b01fa8740 1 -- 172.21.15.117:0/2112427689 --> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] -- command(tid 0: {"prefix": "status"}) -- 0x55f6a5793a40 con 0x55f6a547af60 2024-09-21T05:56:31.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.348+0000 7f2af97d0640 1 --2- 172.21.15.117:0/2112427689 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x55f6a547af60 0x55f6a58add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:31.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.349+0000 7f2af97d0640 1 --2- 172.21.15.117:0/2112427689 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x55f6a547af60 0x55f6a58add20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e161e0da3c5e4656 server_cookie=9fe7b22f492a3fcc in_seq=0 out_seq=0 2024-09-21T05:56:31.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.349+0000 7f2ad8ff9640 10 client.15222 ms_handle_connect on v2:172.21.15.138:6836/2540335374 2024-09-21T05:56:31.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.349+0000 7f2ad8ff9640 1 -- 172.21.15.117:0/2112427689 <== mds.4 v2:172.21.15.138:6836/2540335374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f6a5793a40 con 0x55f6a547af60 2024-09-21T05:56:31.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.349+0000 7f2ad8ff9640 10 client.15222 handle_command_reply: tid=0 2024-09-21T05:56:31.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.349+0000 7f2b01fa8740 1 client.15222 shutdown 2024-09-21T05:56:31.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.349+0000 7f2b01fa8740 1 -- 172.21.15.117:0/2112427689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6a54520a0 msgr2=0x55f6a5841720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:31.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.349+0000 7f2b01fa8740 1 --2- 172.21.15.117:0/2112427689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6a54520a0 0x55f6a5841720 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80e5170 tx=0x7f2ae80a9220 comp rx=0 tx=0).stop 2024-09-21T05:56:31.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.349+0000 7f2b01fa8740 1 -- 172.21.15.117:0/2112427689 shutdown_connections 2024-09-21T05:56:31.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.349+0000 7f2b01fa8740 1 --2- 172.21.15.117:0/2112427689 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x55f6a547af60 0x55f6a58add20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.349+0000 7f2b01fa8740 1 --2- 172.21.15.117:0/2112427689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6a54520a0 0x55f6a5841720 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.349+0000 7f2b01fa8740 1 --2- 172.21.15.117:0/2112427689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6a545a280 0x55f6a5846920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.349+0000 7f2ad8ff9640 0 client.15222 ms_handle_reset on v2:172.21.15.138:6836/2540335374 2024-09-21T05:56:31.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.349+0000 7f2b01fa8740 1 --2- 172.21.15.117:0/2112427689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6a54398e0 0x55f6a58411e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.349+0000 7f2b01fa8740 1 -- 172.21.15.117:0/2112427689 >> 172.21.15.117:0/2112427689 conn(0x55f6a546f350 msgr2=0x55f6a582dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:31.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.349+0000 7f2b01fa8740 1 -- 172.21.15.117:0/2112427689 shutdown_connections 2024-09-21T05:56:31.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.349+0000 7f2b01fa8740 1 -- 172.21.15.117:0/2112427689 wait complete. 2024-09-21T05:56:31.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.349+0000 7f2b01fa8740 20 client.15222 trim_cache size 0 max 16384 2024-09-21T05:56:31.358 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T05:56:31.358 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_fsid": "ac8581a2-77dc-11ef-baf5-efdc52797490", 2024-09-21T05:56:31.358 INFO:teuthology.orchestra.run.smithi117.stdout: "whoami": 4, 2024-09-21T05:56:31.359 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 24602, 2024-09-21T05:56:31.359 INFO:teuthology.orchestra.run.smithi117.stdout: "want_state": "up:active", 2024-09-21T05:56:31.359 INFO:teuthology.orchestra.run.smithi117.stdout: "state": "up:active", 2024-09-21T05:56:31.359 INFO:teuthology.orchestra.run.smithi117.stdout: "fs_name": "cephfs", 2024-09-21T05:56:31.359 INFO:teuthology.orchestra.run.smithi117.stdout: "rank_uptime": 66.144908317000002, 2024-09-21T05:56:31.359 INFO:teuthology.orchestra.run.smithi117.stdout: "mdsmap_epoch": 26, 2024-09-21T05:56:31.359 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch": 76, 2024-09-21T05:56:31.359 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch_barrier": 74, 2024-09-21T05:56:31.359 INFO:teuthology.orchestra.run.smithi117.stdout: "uptime": 99.433112136000005 2024-09-21T05:56:31.359 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T05:56:31.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.350+0000 7f2afba34640 1 -- 172.21.15.117:0/2813456456 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2adc0b65f0 msgr2=0x7f2adc0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:31.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.350+0000 7f2afba34640 1 --2- 172.21.15.117:0/2813456456 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2adc0b65f0 0x7f2adc0b8b00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2aec005c40 tx=0x7f2aec004a40 comp rx=0 tx=0).stop 2024-09-21T05:56:31.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.350+0000 7f2afba34640 1 -- 172.21.15.117:0/2813456456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2af41094a0 msgr2=0x7f2af419aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:31.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.350+0000 7f2afba34640 1 --2- 172.21.15.117:0/2813456456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2af41094a0 0x7f2af419aac0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2ae800d950 tx=0x7f2ae800de10 comp rx=0 tx=0).stop 2024-09-21T05:56:31.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.350+0000 7f2afba34640 1 -- 172.21.15.117:0/2813456456 shutdown_connections 2024-09-21T05:56:31.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.350+0000 7f2afba34640 1 --2- 172.21.15.117:0/2813456456 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2adc0b65f0 0x7f2adc0b8b00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.350+0000 7f2afba34640 1 --2- 172.21.15.117:0/2813456456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2af41094a0 0x7f2af419aac0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.350+0000 7f2afba34640 1 --2- 172.21.15.117:0/2813456456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2af4109df0 0x7f2af419fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.350+0000 7f2afba34640 1 --2- 172.21.15.117:0/2813456456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2af4103b60 0x7f2af419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:31.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.350+0000 7f2afba34640 1 -- 172.21.15.117:0/2813456456 >> 172.21.15.117:0/2813456456 conn(0x7f2af407b850 msgr2=0x7f2af4107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:31.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.350+0000 7f2afba34640 1 -- 172.21.15.117:0/2813456456 shutdown_connections 2024-09-21T05:56:31.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:31.350+0000 7f2afba34640 1 -- 172.21.15.117:0/2813456456 wait complete. 2024-09-21T05:56:31.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:31 smithi192 ceph-mon[34700]: pgmap v303: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T05:56:32.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:31 smithi117 ceph-mon[30947]: pgmap v303: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T05:56:32.097 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.f status 2024-09-21T05:56:32.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:31 smithi138 ceph-mon[34391]: pgmap v303: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T05:56:32.345 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:32.750 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:32 smithi117 ceph-mon[30947]: pgmap v304: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T05:56:32.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.839+0000 7fbee6161640 1 -- 172.21.15.117:0/328166108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbee0102a60 msgr2=0x7fbee0102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:32.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.839+0000 7fbee6161640 1 --2- 172.21.15.117:0/328166108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbee0102a60 0x7fbee0102e70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbed0009900 tx=0x7fbed002a410 comp rx=0 tx=0).stop 2024-09-21T05:56:32.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.839+0000 7fbee6161640 1 -- 172.21.15.117:0/328166108 shutdown_connections 2024-09-21T05:56:32.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.839+0000 7fbee6161640 1 --2- 172.21.15.117:0/328166108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbee0108cf0 0x7fbee010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.839+0000 7fbee6161640 1 --2- 172.21.15.117:0/328166108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbee01083a0 0x7fbee01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.839+0000 7fbee6161640 1 --2- 172.21.15.117:0/328166108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbee0102a60 0x7fbee0102e70 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.839+0000 7fbee6161640 1 -- 172.21.15.117:0/328166108 >> 172.21.15.117:0/328166108 conn(0x7fbee0079730 msgr2=0x7fbee0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:32.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.839+0000 7fbee6161640 1 -- 172.21.15.117:0/328166108 shutdown_connections 2024-09-21T05:56:32.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.839+0000 7fbee6161640 1 -- 172.21.15.117:0/328166108 wait complete. 2024-09-21T05:56:32.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.840+0000 7fbee6161640 1 Processor -- start 2024-09-21T05:56:32.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.840+0000 7fbee6161640 1 -- start start 2024-09-21T05:56:32.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.840+0000 7fbee6161640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbee0102a60 0x7fbee019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:32.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.840+0000 7fbee6161640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbee01083a0 0x7fbee019a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:32.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.840+0000 7fbee6161640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbee0108cf0 0x7fbee019fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:32.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.840+0000 7fbee6161640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbee0112960 con 0x7fbee0108cf0 2024-09-21T05:56:32.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.840+0000 7fbee6161640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbee01127e0 con 0x7fbee01083a0 2024-09-21T05:56:32.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.840+0000 7fbee6161640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbee0112ae0 con 0x7fbee0102a60 2024-09-21T05:56:32.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.841+0000 7fbedffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbee0108cf0 0x7fbee019fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:32.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.841+0000 7fbedffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbee0108cf0 0x7fbee019fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40628/0 (socket says 172.21.15.117:40628) 2024-09-21T05:56:32.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.841+0000 7fbedffff640 1 -- 172.21.15.117:0/456934806 learned_addr learned my addr 172.21.15.117:0/456934806 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:32.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.841+0000 7fbedf7fe640 1 --2- 172.21.15.117:0/456934806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbee0102a60 0x7fbee019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:32.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.841+0000 7fbedeffd640 1 --2- 172.21.15.117:0/456934806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbee01083a0 0x7fbee019a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:32.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.841+0000 7fbedffff640 1 -- 172.21.15.117:0/456934806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbee0102a60 msgr2=0x7fbee019a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:32.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.841+0000 7fbedffff640 1 --2- 172.21.15.117:0/456934806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbee0102a60 0x7fbee019a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.841+0000 7fbedffff640 1 -- 172.21.15.117:0/456934806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbee01083a0 msgr2=0x7fbee019a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:32.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.841+0000 7fbedffff640 1 --2- 172.21.15.117:0/456934806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbee01083a0 0x7fbee019a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.841+0000 7fbedffff640 1 -- 172.21.15.117:0/456934806 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee01a02b0 con 0x7fbee0108cf0 2024-09-21T05:56:32.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.841+0000 7fbedf7fe640 1 --2- 172.21.15.117:0/456934806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbee0102a60 0x7fbee019a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:56:32.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.841+0000 7fbedffff640 1 --2- 172.21.15.117:0/456934806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbee0108cf0 0x7fbee019fc40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbed800d750 tx=0x7fbed800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:32.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.841+0000 7fbedcff9640 1 -- 172.21.15.117:0/456934806 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed8017070 con 0x7fbee0108cf0 2024-09-21T05:56:32.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.841+0000 7fbedcff9640 1 -- 172.21.15.117:0/456934806 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbed80054d0 con 0x7fbee0108cf0 2024-09-21T05:56:32.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.841+0000 7fbee6161640 1 -- 172.21.15.117:0/456934806 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee01a05a0 con 0x7fbee0108cf0 2024-09-21T05:56:32.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.841+0000 7fbedcff9640 1 -- 172.21.15.117:0/456934806 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed8004c60 con 0x7fbee0108cf0 2024-09-21T05:56:32.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.841+0000 7fbee6161640 1 -- 172.21.15.117:0/456934806 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee01a6dc0 con 0x7fbee0108cf0 2024-09-21T05:56:32.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.844+0000 7fbeec6d5740 1 Processor -- start 2024-09-21T05:56:32.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.844+0000 7fbedcff9640 1 -- 172.21.15.117:0/456934806 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7fbed8009bd0 con 0x7fbee0108cf0 2024-09-21T05:56:32.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.844+0000 7fbeec6d5740 1 -- start start 2024-09-21T05:56:32.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.844+0000 7fbeec6d5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c16ba24f60 0x55c16bdcaf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:32.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.844+0000 7fbeec6d5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c16b9fc0a0 0x55c16bdcb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:32.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.844+0000 7fbeec6d5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c16ba04280 0x55c16bdcb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:32.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.844+0000 7fbeec6d5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c16bd42c10 con 0x55c16ba24f60 2024-09-21T05:56:32.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.845+0000 7fbeec6d5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c16bd32040 con 0x55c16ba04280 2024-09-21T05:56:32.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.845+0000 7fbeec6d5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c16bd35e50 con 0x55c16b9fc0a0 2024-09-21T05:56:32.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.845+0000 7fbedeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c16ba24f60 0x55c16bdcaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:32.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.845+0000 7fbedeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c16ba24f60 0x55c16bdcaf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40636/0 (socket says 172.21.15.117:40636) 2024-09-21T05:56:32.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.845+0000 7fbedeffd640 1 -- 172.21.15.117:0/4087630026 learned_addr learned my addr 172.21.15.117:0/4087630026 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:32.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.845+0000 7fbedeffd640 1 --2- 172.21.15.117:0/4087630026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c16ba04280 0x55c16bdcb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:32.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.845+0000 7fbedf7fe640 1 --2- 172.21.15.117:0/4087630026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c16b9fc0a0 0x55c16bdcb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:32.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.845+0000 7fbedeffd640 1 -- 172.21.15.117:0/4087630026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c16b9fc0a0 msgr2=0x55c16bdcb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:32.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.845+0000 7fbedeffd640 1 --2- 172.21.15.117:0/4087630026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c16b9fc0a0 0x55c16bdcb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.845+0000 7fbedeffd640 1 -- 172.21.15.117:0/4087630026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c16ba04280 msgr2=0x55c16bdcb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:32.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.845+0000 7fbedeffd640 1 --2- 172.21.15.117:0/4087630026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c16ba04280 0x55c16bdcb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.845+0000 7fbedcff9640 1 --2- 172.21.15.117:0/456934806 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbec80b6620 0x7fbec80b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:32.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.845+0000 7fbedeffd640 1 -- 172.21.15.117:0/4087630026 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c16ba90540 con 0x55c16ba24f60 2024-09-21T05:56:32.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.845+0000 7fbedffff640 1 --2- 172.21.15.117:0/456934806 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbec80b6620 0x7fbec80b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:32.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.845+0000 7fbedeffd640 1 --2- 172.21.15.117:0/4087630026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c16ba24f60 0x55c16bdcaf00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbed4008f20 tx=0x7fbed40087c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6219a8397cb60a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:32.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.845+0000 7fbedcff9640 1 -- 172.21.15.117:0/456934806 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fbed80e6fc0 con 0x7fbee0108cf0 2024-09-21T05:56:32.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.845+0000 7fbec5ffb640 1 -- 172.21.15.117:0/4087630026 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4004b20 con 0x55c16ba24f60 2024-09-21T05:56:32.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.845+0000 7fbec5ffb640 1 -- 172.21.15.117:0/4087630026 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbed400fd10 con 0x55c16ba24f60 2024-09-21T05:56:32.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.845+0000 7fbec5ffb640 1 -- 172.21.15.117:0/4087630026 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed40052f0 con 0x55c16ba24f60 2024-09-21T05:56:32.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.846+0000 7fbeec6d5740 1 -- 172.21.15.117:0/4087630026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c16ba24f60 msgr2=0x55c16bdcaf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:32.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.846+0000 7fbedffff640 1 --2- 172.21.15.117:0/456934806 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbec80b6620 0x7fbec80b8b30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbed80aa1c0 tx=0x7fbed8094040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:32.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.846+0000 7fbeec6d5740 1 --2- 172.21.15.117:0/4087630026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c16ba24f60 0x55c16bdcaf00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbed4008f20 tx=0x7fbed40087c0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.846+0000 7fbeec6d5740 1 -- 172.21.15.117:0/4087630026 shutdown_connections 2024-09-21T05:56:32.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.846+0000 7fbeec6d5740 1 --2- 172.21.15.117:0/4087630026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c16ba04280 0x55c16bdcb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.846+0000 7fbeec6d5740 1 --2- 172.21.15.117:0/4087630026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c16b9fc0a0 0x55c16bdcb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.846+0000 7fbeec6d5740 1 --2- 172.21.15.117:0/4087630026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c16ba24f60 0x55c16bdcaf00 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.846+0000 7fbeec6d5740 1 -- 172.21.15.117:0/4087630026 >> 172.21.15.117:0/4087630026 conn(0x55c16ba19350 msgr2=0x55c16bdc8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:32.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.846+0000 7fbeec6d5740 1 -- 172.21.15.117:0/4087630026 shutdown_connections 2024-09-21T05:56:32.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.846+0000 7fbeec6d5740 1 -- 172.21.15.117:0/4087630026 wait complete. 2024-09-21T05:56:32.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.847+0000 7fbeec6d5740 1 -- start start 2024-09-21T05:56:32.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.847+0000 7fbeec6d5740 1 Processor -- start 2024-09-21T05:56:32.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.847+0000 7fbeec6d5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c16b9fc0a0 0x55c16bdd69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:32.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.847+0000 7fbeec6d5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c16ba04280 0x55c16bdd6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:32.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.847+0000 7fbeec6d5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c16ba24f60 0x55c16bde2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:32.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.847+0000 7fbeec6d5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c16bb935f0 con 0x55c16b9fc0a0 2024-09-21T05:56:32.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.847+0000 7fbeec6d5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c16bccd010 con 0x55c16ba24f60 2024-09-21T05:56:32.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.847+0000 7fbeec6d5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c16bdd9680 con 0x55c16ba04280 2024-09-21T05:56:32.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.848+0000 7fbedeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c16b9fc0a0 0x55c16bdd69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:32.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.847+0000 7fbeec6d5740 10 client.0 _ll_register_callbacks cb 0x55c16bd2c340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:56:32.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.848+0000 7fbedeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c16b9fc0a0 0x55c16bdd69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40648/0 (socket says 172.21.15.117:40648) 2024-09-21T05:56:32.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.848+0000 7fbedeffd640 1 -- 172.21.15.117:0/3265430986 learned_addr learned my addr 172.21.15.117:0/3265430986 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:32.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.848+0000 7fbedeffd640 1 --2- 172.21.15.117:0/3265430986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c16ba24f60 0x55c16bde2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:32.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.848+0000 7fbedf7fe640 1 --2- 172.21.15.117:0/3265430986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c16ba04280 0x55c16bdd6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:32.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.848+0000 7fbedeffd640 1 -- 172.21.15.117:0/3265430986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c16ba04280 msgr2=0x55c16bdd6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:32.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.848+0000 7fbedeffd640 1 --2- 172.21.15.117:0/3265430986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c16ba04280 0x55c16bdd6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.848+0000 7fbedeffd640 1 -- 172.21.15.117:0/3265430986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c16ba24f60 msgr2=0x55c16bde2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:32.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.848+0000 7fbedeffd640 1 --2- 172.21.15.117:0/3265430986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c16ba24f60 0x55c16bde2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.848+0000 7fbedeffd640 1 -- 172.21.15.117:0/3265430986 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c16bcfaa00 con 0x55c16b9fc0a0 2024-09-21T05:56:32.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.848+0000 7fbedf7fe640 1 --2- 172.21.15.117:0/3265430986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c16ba04280 0x55c16bdd6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:32.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.848+0000 7fbedeffd640 1 --2- 172.21.15.117:0/3265430986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c16b9fc0a0 0x55c16bdd69c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbed4006010 tx=0x7fbed4008f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23eec90413e3cf50 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:32.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.848+0000 7fbebeffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:56:32.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.848+0000 7fbebeffd640 1 -- 172.21.15.117:0/3265430986 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed400fa70 con 0x55c16b9fc0a0 2024-09-21T05:56:32.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.848+0000 7fbebeffd640 1 -- 172.21.15.117:0/3265430986 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fbed4005090 con 0x55c16b9fc0a0 2024-09-21T05:56:32.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.848+0000 7fbeec6d5740 1 -- 172.21.15.117:0/3265430986 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbed400fd10 con 0x55c16b9fc0a0 2024-09-21T05:56:32.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.848+0000 7fbebeffd640 1 -- 172.21.15.117:0/3265430986 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbed4005290 con 0x55c16b9fc0a0 2024-09-21T05:56:32.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.848+0000 7fbeec6d5740 10 client.15234 fetch_fsmap learned FSMap version 26 2024-09-21T05:56:32.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.848+0000 7fbeec6d5740 1 -- 172.21.15.117:0/3265430986 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c16bde55a0 con 0x55c16b9fc0a0 2024-09-21T05:56:32.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.849+0000 7fbebeffd640 1 -- 172.21.15.117:0/3265430986 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbed400fd10 con 0x55c16b9fc0a0 2024-09-21T05:56:32.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.849+0000 7fbebeffd640 1 -- 172.21.15.117:0/3265430986 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fbed40226a0 con 0x55c16b9fc0a0 2024-09-21T05:56:32.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.849+0000 7fbeec6d5740 10 client.15234 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T05:56:32.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.849+0000 7fbeec6d5740 10 client.15234 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-21T05:56:32.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.849+0000 7fbeec6d5740 20 client.15234 populate_metadata read hostname 'smithi117' 2024-09-21T05:56:32.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.849+0000 7fbeec6d5740 1 --2- 172.21.15.117:0/3265430986 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55c16b9e38e0 0x55c16bde8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:32.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.849+0000 7fbeec6d5740 4 client.15234 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:56:32.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.849+0000 7fbeec6d5740 1 -- 172.21.15.117:0/3265430986 --> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c16bd44b60 con 0x55c16b9e38e0 2024-09-21T05:56:32.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.849+0000 7fbedf7fe640 1 --2- 172.21.15.117:0/3265430986 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55c16b9e38e0 0x55c16bde8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:32.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.850+0000 7fbebeffd640 10 client.15234 ms_handle_connect on v2:172.21.15.192:6834/3530688860 2024-09-21T05:56:32.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.850+0000 7fbedf7fe640 1 --2- 172.21.15.117:0/3265430986 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55c16b9e38e0 0x55c16bde8ce0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e847b2c7d5398490 server_cookie=df431f4c626ea71d in_seq=0 out_seq=0 2024-09-21T05:56:32.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.850+0000 7fbebeffd640 1 -- 172.21.15.117:0/3265430986 <== mds.3 v2:172.21.15.192:6834/3530688860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c16bd44b60 con 0x55c16b9e38e0 2024-09-21T05:56:32.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.850+0000 7fbebeffd640 10 client.15234 handle_command_reply: tid=0 2024-09-21T05:56:32.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.851+0000 7fbeec6d5740 1 client.15234 shutdown 2024-09-21T05:56:32.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.851+0000 7fbeec6d5740 1 -- 172.21.15.117:0/3265430986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c16b9fc0a0 msgr2=0x55c16bdd69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:32.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.851+0000 7fbeec6d5740 1 --2- 172.21.15.117:0/3265430986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c16b9fc0a0 0x55c16bdd69c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fbed4006010 tx=0x7fbed4008f50 comp rx=0 tx=0).stop 2024-09-21T05:56:32.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.851+0000 7fbeec6d5740 1 -- 172.21.15.117:0/3265430986 shutdown_connections 2024-09-21T05:56:32.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.851+0000 7fbeec6d5740 1 --2- 172.21.15.117:0/3265430986 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55c16b9e38e0 0x55c16bde8ce0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.851+0000 7fbeec6d5740 1 --2- 172.21.15.117:0/3265430986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c16ba24f60 0x55c16bde2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.851+0000 7fbeec6d5740 1 --2- 172.21.15.117:0/3265430986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c16ba04280 0x55c16bdd6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.851+0000 7fbebeffd640 0 client.15234 ms_handle_reset on v2:172.21.15.192:6834/3530688860 2024-09-21T05:56:32.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.851+0000 7fbeec6d5740 1 --2- 172.21.15.117:0/3265430986 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c16b9fc0a0 0x55c16bdd69c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.851+0000 7fbeec6d5740 1 -- 172.21.15.117:0/3265430986 >> 172.21.15.117:0/3265430986 conn(0x55c16ba19350 msgr2=0x55c16bdce0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:32.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.851+0000 7fbeec6d5740 1 -- 172.21.15.117:0/3265430986 shutdown_connections 2024-09-21T05:56:32.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.851+0000 7fbeec6d5740 1 -- 172.21.15.117:0/3265430986 wait complete. 2024-09-21T05:56:32.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.851+0000 7fbeec6d5740 20 client.15234 trim_cache size 0 max 16384 2024-09-21T05:56:32.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.869+0000 7fbeec6d5740 1 Processor -- start 2024-09-21T05:56:32.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.869+0000 7fbeec6d5740 1 -- start start 2024-09-21T05:56:32.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.869+0000 7fbeec6d5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c16b9e38e0 0x55c16bdd19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:32.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.869+0000 7fbeec6d5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c16b9fc0a0 0x55c16bdd1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:32.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.869+0000 7fbeec6d5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c16ba04280 0x55c16bde28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:32.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.869+0000 7fbeec6d5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c16bde7020 con 0x55c16b9e38e0 2024-09-21T05:56:32.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.869+0000 7fbeec6d5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c16bde76f0 con 0x55c16b9fc0a0 2024-09-21T05:56:32.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.869+0000 7fbeec6d5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c16bde7dc0 con 0x55c16ba04280 2024-09-21T05:56:32.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.869+0000 7fbedeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c16b9e38e0 0x55c16bdd19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:32.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.869+0000 7fbedeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c16b9e38e0 0x55c16bdd19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40660/0 (socket says 172.21.15.117:40660) 2024-09-21T05:56:32.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.869+0000 7fbedeffd640 1 -- 172.21.15.117:0/3244581980 learned_addr learned my addr 172.21.15.117:0/3244581980 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:32.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.869+0000 7fbedf7fe640 1 --2- 172.21.15.117:0/3244581980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c16b9fc0a0 0x55c16bdd1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:32.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.869+0000 7fbedeffd640 1 -- 172.21.15.117:0/3244581980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c16ba04280 msgr2=0x55c16bde28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:32.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.869+0000 7fbedeffd640 1 --2- 172.21.15.117:0/3244581980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c16ba04280 0x55c16bde28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.869+0000 7fbedeffd640 1 -- 172.21.15.117:0/3244581980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c16b9fc0a0 msgr2=0x55c16bdd1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:32.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.869+0000 7fbedeffd640 1 --2- 172.21.15.117:0/3244581980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c16b9fc0a0 0x55c16bdd1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.869+0000 7fbedeffd640 1 -- 172.21.15.117:0/3244581980 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c16bdc4520 con 0x55c16b9e38e0 2024-09-21T05:56:32.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.870+0000 7fbedeffd640 1 --2- 172.21.15.117:0/3244581980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c16b9e38e0 0x55c16bdd19e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbed40146d0 tx=0x7fbed4005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b77679d4aa6c35 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:32.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.870+0000 7fbec57fa640 1 -- 172.21.15.117:0/3244581980 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed403c070 con 0x55c16b9e38e0 2024-09-21T05:56:32.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.870+0000 7fbec57fa640 1 -- 172.21.15.117:0/3244581980 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbed4005000 con 0x55c16b9e38e0 2024-09-21T05:56:32.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.870+0000 7fbec57fa640 1 -- 172.21.15.117:0/3244581980 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4005690 con 0x55c16b9e38e0 2024-09-21T05:56:32.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.871+0000 7fbeec6d5740 1 -- 172.21.15.117:0/3244581980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c16b9e38e0 msgr2=0x55c16bdd19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:32.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.871+0000 7fbeec6d5740 1 --2- 172.21.15.117:0/3244581980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c16b9e38e0 0x55c16bdd19e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fbed40146d0 tx=0x7fbed4005e90 comp rx=0 tx=0).stop 2024-09-21T05:56:32.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.871+0000 7fbeec6d5740 1 -- 172.21.15.117:0/3244581980 shutdown_connections 2024-09-21T05:56:32.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.871+0000 7fbeec6d5740 1 --2- 172.21.15.117:0/3244581980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c16ba04280 0x55c16bde28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.871+0000 7fbeec6d5740 1 --2- 172.21.15.117:0/3244581980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c16b9fc0a0 0x55c16bdd1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.871+0000 7fbeec6d5740 1 --2- 172.21.15.117:0/3244581980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c16b9e38e0 0x55c16bdd19e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.871+0000 7fbeec6d5740 1 -- 172.21.15.117:0/3244581980 >> 172.21.15.117:0/3244581980 conn(0x55c16ba19350 msgr2=0x55c16bdd69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:32.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.871+0000 7fbeec6d5740 1 -- 172.21.15.117:0/3244581980 shutdown_connections 2024-09-21T05:56:32.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.871+0000 7fbeec6d5740 1 -- 172.21.15.117:0/3244581980 wait complete. 2024-09-21T05:56:32.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.871+0000 7fbeec6d5740 1 -- start start 2024-09-21T05:56:32.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.871+0000 7fbeec6d5740 1 Processor -- start 2024-09-21T05:56:32.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.872+0000 7fbeec6d5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c16b9e38e0 0x55c16bdea510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:32.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.872+0000 7fbeec6d5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c16b9fc0a0 0x55c16bdeaad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:32.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.872+0000 7fbeec6d5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c16ba04280 0x55c16bdefab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:32.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.872+0000 7fbeec6d5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c16bde8b60 con 0x55c16ba04280 2024-09-21T05:56:32.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.872+0000 7fbeec6d5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c16bde8490 con 0x55c16b9e38e0 2024-09-21T05:56:32.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.872+0000 7fbeec6d5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c16bde9230 con 0x55c16b9fc0a0 2024-09-21T05:56:32.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.872+0000 7fbeec6d5740 10 client.0 _ll_register_callbacks cb 0x7fbed4014720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:56:32.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.872+0000 7fbedeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c16ba04280 0x55c16bdefab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:32.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.872+0000 7fbedeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c16ba04280 0x55c16bdefab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40672/0 (socket says 172.21.15.117:40672) 2024-09-21T05:56:32.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.872+0000 7fbedeffd640 1 -- 172.21.15.117:0/2784027226 learned_addr learned my addr 172.21.15.117:0/2784027226 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:32.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.872+0000 7fbedf7fe640 1 --2- 172.21.15.117:0/2784027226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c16b9fc0a0 0x55c16bdeaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:32.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.872+0000 7fbedeffd640 1 --2- 172.21.15.117:0/2784027226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c16b9e38e0 0x55c16bdea510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:32.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.872+0000 7fbedeffd640 1 -- 172.21.15.117:0/2784027226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c16b9fc0a0 msgr2=0x55c16bdeaad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:32.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.872+0000 7fbedeffd640 1 --2- 172.21.15.117:0/2784027226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c16b9fc0a0 0x55c16bdeaad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.872+0000 7fbedeffd640 1 -- 172.21.15.117:0/2784027226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c16b9e38e0 msgr2=0x55c16bdea510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:32.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.872+0000 7fbedeffd640 1 --2- 172.21.15.117:0/2784027226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c16b9e38e0 0x55c16bdea510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.872+0000 7fbedeffd640 1 -- 172.21.15.117:0/2784027226 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c16bdf0120 con 0x55c16ba04280 2024-09-21T05:56:32.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.872+0000 7fbedeffd640 1 --2- 172.21.15.117:0/2784027226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c16ba04280 0x55c16bdefab0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbed4008680 tx=0x7fbed4031040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07bea8d4cdaa590 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:32.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.872+0000 7fbebeffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:56:32.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.873+0000 7fbebeffd640 1 -- 172.21.15.117:0/2784027226 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4049060 con 0x55c16ba04280 2024-09-21T05:56:32.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.873+0000 7fbeec6d5740 1 -- 172.21.15.117:0/2784027226 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbed4005000 con 0x55c16ba04280 2024-09-21T05:56:32.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.873+0000 7fbebeffd640 1 -- 172.21.15.117:0/2784027226 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fbed4005640 con 0x55c16ba04280 2024-09-21T05:56:32.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.873+0000 7fbebeffd640 1 -- 172.21.15.117:0/2784027226 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbed403c040 con 0x55c16ba04280 2024-09-21T05:56:32.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.873+0000 7fbeec6d5740 10 client.15240 fetch_fsmap learned FSMap version 26 2024-09-21T05:56:32.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.873+0000 7fbeec6d5740 1 -- 172.21.15.117:0/2784027226 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c16bdf14e0 con 0x55c16ba04280 2024-09-21T05:56:32.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.873+0000 7fbebeffd640 1 -- 172.21.15.117:0/2784027226 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbed4005000 con 0x55c16ba04280 2024-09-21T05:56:32.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.873+0000 7fbebeffd640 1 -- 172.21.15.117:0/2784027226 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fbed4022ce0 con 0x55c16ba04280 2024-09-21T05:56:32.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.873+0000 7fbeec6d5740 10 client.15240 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T05:56:32.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.873+0000 7fbeec6d5740 10 client.15240 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-21T05:56:32.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.873+0000 7fbeec6d5740 20 client.15240 populate_metadata read hostname 'smithi117' 2024-09-21T05:56:32.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.873+0000 7fbeec6d5740 1 --2- 172.21.15.117:0/2784027226 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55c16ba24f60 0x55c16be56c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:32.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.873+0000 7fbeec6d5740 4 client.15240 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T05:56:32.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.873+0000 7fbeec6d5740 1 -- 172.21.15.117:0/2784027226 --> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] -- command(tid 0: {"prefix": "status"}) -- 0x55c16bd40610 con 0x55c16ba24f60 2024-09-21T05:56:32.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.874+0000 7fbedf7fe640 1 --2- 172.21.15.117:0/2784027226 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55c16ba24f60 0x55c16be56c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:32.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.874+0000 7fbebeffd640 10 client.15240 ms_handle_connect on v2:172.21.15.192:6834/3530688860 2024-09-21T05:56:32.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.874+0000 7fbedf7fe640 1 --2- 172.21.15.117:0/2784027226 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55c16ba24f60 0x55c16be56c20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=95f061a1d974842c server_cookie=34e58505bd6b3f4b in_seq=0 out_seq=0 2024-09-21T05:56:32.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.874+0000 7fbebeffd640 1 -- 172.21.15.117:0/2784027226 <== mds.3 v2:172.21.15.192:6834/3530688860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55c16bd40610 con 0x55c16ba24f60 2024-09-21T05:56:32.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.874+0000 7fbebeffd640 10 client.15240 handle_command_reply: tid=0 2024-09-21T05:56:32.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.874+0000 7fbeec6d5740 1 client.15240 shutdown 2024-09-21T05:56:32.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.874+0000 7fbeec6d5740 1 -- 172.21.15.117:0/2784027226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c16ba04280 msgr2=0x55c16bdefab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:32.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.874+0000 7fbeec6d5740 1 --2- 172.21.15.117:0/2784027226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c16ba04280 0x55c16bdefab0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbed4008680 tx=0x7fbed4031040 comp rx=0 tx=0).stop 2024-09-21T05:56:32.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.875+0000 7fbeec6d5740 1 -- 172.21.15.117:0/2784027226 shutdown_connections 2024-09-21T05:56:32.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.875+0000 7fbeec6d5740 1 --2- 172.21.15.117:0/2784027226 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55c16ba24f60 0x55c16be56c20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.875+0000 7fbeec6d5740 1 --2- 172.21.15.117:0/2784027226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c16ba04280 0x55c16bdefab0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.875+0000 7fbeec6d5740 1 --2- 172.21.15.117:0/2784027226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c16b9fc0a0 0x55c16bdeaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.875+0000 7fbeec6d5740 1 --2- 172.21.15.117:0/2784027226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c16b9e38e0 0x55c16bdea510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.875+0000 7fbeec6d5740 1 -- 172.21.15.117:0/2784027226 >> 172.21.15.117:0/2784027226 conn(0x55c16ba19350 msgr2=0x55c16bdd19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:32.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.875+0000 7fbebeffd640 0 client.15240 ms_handle_reset on v2:172.21.15.192:6834/3530688860 2024-09-21T05:56:32.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.875+0000 7fbeec6d5740 1 -- 172.21.15.117:0/2784027226 shutdown_connections 2024-09-21T05:56:32.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.875+0000 7fbeec6d5740 1 -- 172.21.15.117:0/2784027226 wait complete. 2024-09-21T05:56:32.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.875+0000 7fbeec6d5740 20 client.15240 trim_cache size 0 max 16384 2024-09-21T05:56:32.882 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T05:56:32.882 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_fsid": "ac8581a2-77dc-11ef-baf5-efdc52797490", 2024-09-21T05:56:32.882 INFO:teuthology.orchestra.run.smithi117.stdout: "whoami": 3, 2024-09-21T05:56:32.882 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 24616, 2024-09-21T05:56:32.882 INFO:teuthology.orchestra.run.smithi117.stdout: "want_state": "up:active", 2024-09-21T05:56:32.882 INFO:teuthology.orchestra.run.smithi117.stdout: "state": "up:active", 2024-09-21T05:56:32.883 INFO:teuthology.orchestra.run.smithi117.stdout: "fs_name": "cephfs", 2024-09-21T05:56:32.883 INFO:teuthology.orchestra.run.smithi117.stdout: "rank_uptime": 68.67839961, 2024-09-21T05:56:32.883 INFO:teuthology.orchestra.run.smithi117.stdout: "mdsmap_epoch": 26, 2024-09-21T05:56:32.883 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch": 76, 2024-09-21T05:56:32.883 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch_barrier": 74, 2024-09-21T05:56:32.883 INFO:teuthology.orchestra.run.smithi117.stdout: "uptime": 106.20428687499999 2024-09-21T05:56:32.883 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T05:56:32.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.875+0000 7fbee6161640 1 -- 172.21.15.117:0/456934806 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbec80b6620 msgr2=0x7fbec80b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:32.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.875+0000 7fbee6161640 1 --2- 172.21.15.117:0/456934806 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbec80b6620 0x7fbec80b8b30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbed80aa1c0 tx=0x7fbed8094040 comp rx=0 tx=0).stop 2024-09-21T05:56:32.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.875+0000 7fbee6161640 1 -- 172.21.15.117:0/456934806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbee0108cf0 msgr2=0x7fbee019fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:32.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.875+0000 7fbee6161640 1 --2- 172.21.15.117:0/456934806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbee0108cf0 0x7fbee019fc40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbed800d750 tx=0x7fbed800dc10 comp rx=0 tx=0).stop 2024-09-21T05:56:32.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.875+0000 7fbee6161640 1 -- 172.21.15.117:0/456934806 shutdown_connections 2024-09-21T05:56:32.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.875+0000 7fbee6161640 1 --2- 172.21.15.117:0/456934806 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbec80b6620 0x7fbec80b8b30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.875+0000 7fbee6161640 1 --2- 172.21.15.117:0/456934806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbee0108cf0 0x7fbee019fc40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.875+0000 7fbee6161640 1 --2- 172.21.15.117:0/456934806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbee01083a0 0x7fbee019a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.875+0000 7fbee6161640 1 --2- 172.21.15.117:0/456934806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbee0102a60 0x7fbee019a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:32.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.875+0000 7fbee6161640 1 -- 172.21.15.117:0/456934806 >> 172.21.15.117:0/456934806 conn(0x7fbee0079730 msgr2=0x7fbee01066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:32.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.875+0000 7fbee6161640 1 -- 172.21.15.117:0/456934806 shutdown_connections 2024-09-21T05:56:32.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:32.875+0000 7fbee6161640 1 -- 172.21.15.117:0/456934806 wait complete. 2024-09-21T05:56:32.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:32 smithi192 ceph-mon[34700]: pgmap v304: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T05:56:33.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:32 smithi138 ceph-mon[34391]: pgmap v304: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T05:56:33.635 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.c status 2024-09-21T05:56:33.881 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:34.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.368+0000 7f6bab2c2640 1 -- 172.21.15.117:0/2213346586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ba4105300 msgr2=0x7f6ba410d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:34.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.368+0000 7f6bab2c2640 1 --2- 172.21.15.117:0/2213346586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ba4105300 0x7f6ba410d850 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009900 tx=0x7f6b9802a410 comp rx=0 tx=0).stop 2024-09-21T05:56:34.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.368+0000 7f6bab2c2640 1 -- 172.21.15.117:0/2213346586 shutdown_connections 2024-09-21T05:56:34.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.368+0000 7f6bab2c2640 1 --2- 172.21.15.117:0/2213346586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ba4105300 0x7f6ba410d850 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.368+0000 7f6bab2c2640 1 --2- 172.21.15.117:0/2213346586 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ba410de10 0x7f6ba4110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.368+0000 7f6bab2c2640 1 --2- 172.21.15.117:0/2213346586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ba41049b0 0x7f6ba4104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.368+0000 7f6bab2c2640 1 -- 172.21.15.117:0/2213346586 >> 172.21.15.117:0/2213346586 conn(0x7f6ba4079730 msgr2=0x7f6ba4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:34.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.368+0000 7f6bab2c2640 1 -- 172.21.15.117:0/2213346586 shutdown_connections 2024-09-21T05:56:34.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.368+0000 7f6bab2c2640 1 -- 172.21.15.117:0/2213346586 wait complete. 2024-09-21T05:56:34.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.369+0000 7f6bab2c2640 1 Processor -- start 2024-09-21T05:56:34.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.369+0000 7f6bab2c2640 1 -- start start 2024-09-21T05:56:34.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.369+0000 7f6bab2c2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ba41049b0 0x7f6ba419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:34.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.369+0000 7f6bab2c2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ba4105300 0x7f6ba419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:34.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.369+0000 7f6bab2c2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ba410de10 0x7f6ba419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:34.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.369+0000 7f6bab2c2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba4113a70 con 0x7f6ba41049b0 2024-09-21T05:56:34.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.369+0000 7f6bab2c2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba41138f0 con 0x7f6ba4105300 2024-09-21T05:56:34.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.369+0000 7f6bab2c2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba4113bf0 con 0x7f6ba410de10 2024-09-21T05:56:34.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.369+0000 7f6ba905e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ba41049b0 0x7f6ba419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:34.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.369+0000 7f6ba905e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ba41049b0 0x7f6ba419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40692/0 (socket says 172.21.15.117:40692) 2024-09-21T05:56:34.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.369+0000 7f6ba905e640 1 -- 172.21.15.117:0/2098071887 learned_addr learned my addr 172.21.15.117:0/2098071887 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:34.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.369+0000 7f6ba885d640 1 --2- 172.21.15.117:0/2098071887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ba4105300 0x7f6ba419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:34.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.369+0000 7f6ba985f640 1 --2- 172.21.15.117:0/2098071887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ba410de10 0x7f6ba419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:34.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.370+0000 7f6ba905e640 1 -- 172.21.15.117:0/2098071887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ba410de10 msgr2=0x7f6ba419fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:34.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.370+0000 7f6ba905e640 1 --2- 172.21.15.117:0/2098071887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ba410de10 0x7f6ba419fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.370+0000 7f6ba905e640 1 -- 172.21.15.117:0/2098071887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ba4105300 msgr2=0x7f6ba419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:34.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.370+0000 7f6ba905e640 1 --2- 172.21.15.117:0/2098071887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ba4105300 0x7f6ba419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.370+0000 7f6ba905e640 1 -- 172.21.15.117:0/2098071887 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba41a03d0 con 0x7f6ba41049b0 2024-09-21T05:56:34.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.370+0000 7f6ba905e640 1 --2- 172.21.15.117:0/2098071887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ba41049b0 0x7f6ba419a460 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6b9400ccb0 tx=0x7f6b9400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:34.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.370+0000 7f6ba27fc640 1 -- 172.21.15.117:0/2098071887 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b94003990 con 0x7f6ba41049b0 2024-09-21T05:56:34.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.370+0000 7f6ba27fc640 1 -- 172.21.15.117:0/2098071887 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b94004020 con 0x7f6ba41049b0 2024-09-21T05:56:34.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.370+0000 7f6ba27fc640 1 -- 172.21.15.117:0/2098071887 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9400f6b0 con 0x7f6ba41049b0 2024-09-21T05:56:34.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.370+0000 7f6bab2c2640 1 -- 172.21.15.117:0/2098071887 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba41a06c0 con 0x7f6ba41049b0 2024-09-21T05:56:34.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.370+0000 7f6bab2c2640 1 -- 172.21.15.117:0/2098071887 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba41a6e20 con 0x7f6ba41049b0 2024-09-21T05:56:34.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.373+0000 7f6bb1836740 1 Processor -- start 2024-09-21T05:56:34.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.373+0000 7f6bb1836740 1 -- start start 2024-09-21T05:56:34.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.373+0000 7f6ba27fc640 1 -- 172.21.15.117:0/2098071887 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f6b940044b0 con 0x7f6ba41049b0 2024-09-21T05:56:34.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.373+0000 7f6bb1836740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fbbc3bf60 0x557fbbfe1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:34.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.373+0000 7f6bb1836740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fbbc130a0 0x557fbbfe2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:34.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.373+0000 7f6bb1836740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fbbc1b280 0x557fbbfe2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:34.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.373+0000 7f6bb1836740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557fbbf59c10 con 0x557fbbc3bf60 2024-09-21T05:56:34.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.373+0000 7f6bb1836740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557fbbf49040 con 0x557fbbc130a0 2024-09-21T05:56:34.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.373+0000 7f6bb1836740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557fbbf4ce50 con 0x557fbbc1b280 2024-09-21T05:56:34.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.373+0000 7f6ba985f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fbbc3bf60 0x557fbbfe1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:34.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.373+0000 7f6ba985f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fbbc3bf60 0x557fbbfe1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40702/0 (socket says 172.21.15.117:40702) 2024-09-21T05:56:34.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.373+0000 7f6ba985f640 1 -- 172.21.15.117:0/515023368 learned_addr learned my addr 172.21.15.117:0/515023368 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:34.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.373+0000 7f6ba885d640 1 --2- 172.21.15.117:0/515023368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fbbc1b280 0x557fbbfe2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:34.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.373+0000 7f6ba905e640 1 --2- 172.21.15.117:0/515023368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fbbc130a0 0x557fbbfe2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:34.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.373+0000 7f6ba985f640 1 -- 172.21.15.117:0/515023368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fbbc1b280 msgr2=0x557fbbfe2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:34.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.374+0000 7f6ba985f640 1 --2- 172.21.15.117:0/515023368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fbbc1b280 0x557fbbfe2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.374+0000 7f6ba985f640 1 -- 172.21.15.117:0/515023368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fbbc130a0 msgr2=0x557fbbfe2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:34.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.374+0000 7f6ba985f640 1 --2- 172.21.15.117:0/515023368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fbbc130a0 0x557fbbfe2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.374+0000 7f6ba985f640 1 -- 172.21.15.117:0/515023368 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fbbca7540 con 0x557fbbc3bf60 2024-09-21T05:56:34.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.374+0000 7f6ba985f640 1 --2- 172.21.15.117:0/515023368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fbbc3bf60 0x557fbbfe1f00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c00bc70 tx=0x7f6b9c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=253f397e1eb2c1ed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:34.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.374+0000 7f6b8b7fe640 1 -- 172.21.15.117:0/515023368 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c015070 con 0x557fbbc3bf60 2024-09-21T05:56:34.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.374+0000 7f6ba27fc640 1 --2- 172.21.15.117:0/2098071887 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6b8c0b6670 0x7f6b8c0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:34.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.374+0000 7f6b8b7fe640 1 -- 172.21.15.117:0/515023368 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b9c01a910 con 0x557fbbc3bf60 2024-09-21T05:56:34.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.374+0000 7f6b8b7fe640 1 -- 172.21.15.117:0/515023368 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c0118e0 con 0x557fbbc3bf60 2024-09-21T05:56:34.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.374+0000 7f6ba905e640 1 --2- 172.21.15.117:0/2098071887 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6b8c0b6670 0x7f6b8c0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:34.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.374+0000 7f6ba27fc640 1 -- 172.21.15.117:0/2098071887 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f6b940e7900 con 0x7f6ba41049b0 2024-09-21T05:56:34.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.375+0000 7f6bb1836740 1 -- 172.21.15.117:0/515023368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fbbc3bf60 msgr2=0x557fbbfe1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:34.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.375+0000 7f6ba905e640 1 --2- 172.21.15.117:0/2098071887 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6b8c0b6670 0x7f6b8c0b8b80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x557fbbfe0b80 tx=0x7f6b940ed4b0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:34.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.375+0000 7f6bb1836740 1 --2- 172.21.15.117:0/515023368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fbbc3bf60 0x557fbbfe1f00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c00bc70 tx=0x7f6b9c00f730 comp rx=0 tx=0).stop 2024-09-21T05:56:34.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.375+0000 7f6bb1836740 1 -- 172.21.15.117:0/515023368 shutdown_connections 2024-09-21T05:56:34.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.375+0000 7f6bb1836740 1 --2- 172.21.15.117:0/515023368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fbbc1b280 0x557fbbfe2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.375+0000 7f6bb1836740 1 --2- 172.21.15.117:0/515023368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fbbc130a0 0x557fbbfe2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.375+0000 7f6bb1836740 1 --2- 172.21.15.117:0/515023368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fbbc3bf60 0x557fbbfe1f00 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.375+0000 7f6bb1836740 1 -- 172.21.15.117:0/515023368 >> 172.21.15.117:0/515023368 conn(0x557fbbc30350 msgr2=0x557fbbfdfdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:34.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.375+0000 7f6bb1836740 1 -- 172.21.15.117:0/515023368 shutdown_connections 2024-09-21T05:56:34.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.375+0000 7f6bb1836740 1 -- 172.21.15.117:0/515023368 wait complete. 2024-09-21T05:56:34.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.375+0000 7f6bb1836740 1 -- start start 2024-09-21T05:56:34.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.376+0000 7f6bb1836740 1 Processor -- start 2024-09-21T05:56:34.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.376+0000 7f6bb1836740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fbbc130a0 0x557fbbfed9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:34.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.376+0000 7f6bb1836740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fbbc1b280 0x557fbbfedf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:34.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.376+0000 7f6bb1836740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fbbc3bf60 0x557fbbff9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:34.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.376+0000 7f6bb1836740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557fbbdaa5f0 con 0x557fbbc3bf60 2024-09-21T05:56:34.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.376+0000 7f6bb1836740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557fbbee4010 con 0x557fbbc1b280 2024-09-21T05:56:34.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.376+0000 7f6bb1836740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557fbbff06e0 con 0x557fbbc130a0 2024-09-21T05:56:34.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.376+0000 7f6bb1836740 10 client.0 _ll_register_callbacks cb 0x557fbbf43340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:56:34.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.376+0000 7f6ba985f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fbbc3bf60 0x557fbbff9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:34.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.376+0000 7f6ba985f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fbbc3bf60 0x557fbbff9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40710/0 (socket says 172.21.15.117:40710) 2024-09-21T05:56:34.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.376+0000 7f6ba985f640 1 -- 172.21.15.117:0/2967777817 learned_addr learned my addr 172.21.15.117:0/2967777817 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:34.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.376+0000 7f6ba885d640 1 --2- 172.21.15.117:0/2967777817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fbbc1b280 0x557fbbfedf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:34.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.376+0000 7f6ba985f640 1 --2- 172.21.15.117:0/2967777817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fbbc130a0 0x557fbbfed9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:34.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.376+0000 7f6ba985f640 1 -- 172.21.15.117:0/2967777817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fbbc130a0 msgr2=0x557fbbfed9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:34.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.376+0000 7f6ba985f640 1 --2- 172.21.15.117:0/2967777817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fbbc130a0 0x557fbbfed9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.376+0000 7f6ba985f640 1 -- 172.21.15.117:0/2967777817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fbbc1b280 msgr2=0x557fbbfedf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:34.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.376+0000 7f6ba985f640 1 --2- 172.21.15.117:0/2967777817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fbbc1b280 0x557fbbfedf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.377+0000 7f6ba985f640 1 -- 172.21.15.117:0/2967777817 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fbbf11a00 con 0x557fbbc3bf60 2024-09-21T05:56:34.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.377+0000 7f6ba985f640 1 --2- 172.21.15.117:0/2967777817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fbbc3bf60 0x557fbbff9a00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c005bc0 tx=0x7f6b9c011580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57ce464defd8224 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:34.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.377+0000 7f6b88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:56:34.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.377+0000 7f6b88ff9640 1 -- 172.21.15.117:0/2967777817 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c015030 con 0x557fbbc3bf60 2024-09-21T05:56:34.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.377+0000 7f6b88ff9640 1 -- 172.21.15.117:0/2967777817 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f6b9c01a460 con 0x557fbbc3bf60 2024-09-21T05:56:34.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.377+0000 7f6bb1836740 1 -- 172.21.15.117:0/2967777817 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b9c01a910 con 0x557fbbc3bf60 2024-09-21T05:56:34.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.377+0000 7f6b88ff9640 1 -- 172.21.15.117:0/2967777817 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6b9c01a660 con 0x557fbbc3bf60 2024-09-21T05:56:34.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.377+0000 7f6bb1836740 10 client.15252 fetch_fsmap learned FSMap version 26 2024-09-21T05:56:34.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.377+0000 7f6bb1836740 1 -- 172.21.15.117:0/2967777817 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557fbbffc600 con 0x557fbbc3bf60 2024-09-21T05:56:34.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.377+0000 7f6b88ff9640 1 -- 172.21.15.117:0/2967777817 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b9c01a910 con 0x557fbbc3bf60 2024-09-21T05:56:34.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.377+0000 7f6b88ff9640 1 -- 172.21.15.117:0/2967777817 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f6b9c030070 con 0x557fbbc3bf60 2024-09-21T05:56:34.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.377+0000 7f6bb1836740 10 client.15252 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T05:56:34.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.378+0000 7f6bb1836740 10 client.15252 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T05:56:34.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.378+0000 7f6bb1836740 20 client.15252 populate_metadata read hostname 'smithi117' 2024-09-21T05:56:34.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.378+0000 7f6bb1836740 1 --2- 172.21.15.117:0/2967777817 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x557fbbbfa8e0 0x557fbbfffe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:34.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.378+0000 7f6bb1836740 4 client.15252 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:56:34.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.378+0000 7f6bb1836740 1 -- 172.21.15.117:0/2967777817 --> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557fbbf5bb60 con 0x557fbbbfa8e0 2024-09-21T05:56:34.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.378+0000 7f6ba885d640 1 --2- 172.21.15.117:0/2967777817 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x557fbbbfa8e0 0x557fbbfffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:34.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.378+0000 7f6b88ff9640 10 client.15252 ms_handle_connect on v2:172.21.15.192:6836/2683860842 2024-09-21T05:56:34.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.378+0000 7f6ba885d640 1 --2- 172.21.15.117:0/2967777817 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x557fbbbfa8e0 0x557fbbfffe10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4d306403132bc7a4 server_cookie=3a1cd0478ae98a13 in_seq=0 out_seq=0 2024-09-21T05:56:34.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.379+0000 7f6b88ff9640 1 -- 172.21.15.117:0/2967777817 <== mds.2 v2:172.21.15.192:6836/2683860842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557fbbf5bb60 con 0x557fbbbfa8e0 2024-09-21T05:56:34.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.379+0000 7f6b88ff9640 10 client.15252 handle_command_reply: tid=0 2024-09-21T05:56:34.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.379+0000 7f6bb1836740 1 client.15252 shutdown 2024-09-21T05:56:34.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.379+0000 7f6bb1836740 1 -- 172.21.15.117:0/2967777817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fbbc3bf60 msgr2=0x557fbbff9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:34.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.379+0000 7f6bb1836740 1 --2- 172.21.15.117:0/2967777817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fbbc3bf60 0x557fbbff9a00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c005bc0 tx=0x7f6b9c011580 comp rx=0 tx=0).stop 2024-09-21T05:56:34.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.379+0000 7f6bb1836740 1 -- 172.21.15.117:0/2967777817 shutdown_connections 2024-09-21T05:56:34.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.379+0000 7f6bb1836740 1 --2- 172.21.15.117:0/2967777817 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x557fbbbfa8e0 0x557fbbfffe10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.379+0000 7f6bb1836740 1 --2- 172.21.15.117:0/2967777817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fbbc3bf60 0x557fbbff9a00 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.379+0000 7f6bb1836740 1 --2- 172.21.15.117:0/2967777817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fbbc1b280 0x557fbbfedf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.379+0000 7f6b88ff9640 0 client.15252 ms_handle_reset on v2:172.21.15.192:6836/2683860842 2024-09-21T05:56:34.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.379+0000 7f6bb1836740 1 --2- 172.21.15.117:0/2967777817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fbbc130a0 0x557fbbfed9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.379+0000 7f6bb1836740 1 -- 172.21.15.117:0/2967777817 >> 172.21.15.117:0/2967777817 conn(0x557fbbc30350 msgr2=0x557fbbfe50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:34.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.380+0000 7f6bb1836740 1 -- 172.21.15.117:0/2967777817 shutdown_connections 2024-09-21T05:56:34.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.380+0000 7f6bb1836740 1 -- 172.21.15.117:0/2967777817 wait complete. 2024-09-21T05:56:34.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.380+0000 7f6bb1836740 20 client.15252 trim_cache size 0 max 16384 2024-09-21T05:56:34.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.397+0000 7f6bb1836740 1 Processor -- start 2024-09-21T05:56:34.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.397+0000 7f6bb1836740 1 -- start start 2024-09-21T05:56:34.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.398+0000 7f6bb1836740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fbbc130a0 0x557fbbfe8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:34.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.398+0000 7f6bb1836740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fbbc1b280 0x557fbbff9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:34.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.398+0000 7f6bb1836740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fbbc3bf60 0x557fbbff9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:34.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.398+0000 7f6bb1836740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557fbbffe150 con 0x557fbbc130a0 2024-09-21T05:56:34.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.398+0000 7f6bb1836740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557fbbffe820 con 0x557fbbc3bf60 2024-09-21T05:56:34.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.398+0000 7f6bb1836740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557fbbffeef0 con 0x557fbbc1b280 2024-09-21T05:56:34.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.398+0000 7f6ba985f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fbbc130a0 0x557fbbfe8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:34.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.398+0000 7f6ba985f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fbbc130a0 0x557fbbfe8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40718/0 (socket says 172.21.15.117:40718) 2024-09-21T05:56:34.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.398+0000 7f6ba985f640 1 -- 172.21.15.117:0/2756247005 learned_addr learned my addr 172.21.15.117:0/2756247005 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:34.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.398+0000 7f6ba985f640 1 -- 172.21.15.117:0/2756247005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fbbc1b280 msgr2=0x557fbbff9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:34.400 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.398+0000 7f6ba985f640 1 --2- 172.21.15.117:0/2756247005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fbbc1b280 0x557fbbff9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.401 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.398+0000 7f6ba985f640 1 -- 172.21.15.117:0/2756247005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fbbc3bf60 msgr2=0x557fbbff9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:34.401 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.398+0000 7f6ba985f640 1 --2- 172.21.15.117:0/2756247005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fbbc3bf60 0x557fbbff9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.401 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.398+0000 7f6ba985f640 1 -- 172.21.15.117:0/2756247005 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fbbfebcf0 con 0x557fbbc130a0 2024-09-21T05:56:34.403 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.398+0000 7f6ba985f640 1 --2- 172.21.15.117:0/2756247005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fbbc130a0 0x557fbbfe8020 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c00fcb0 tx=0x7f6b9c00b440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6de0e36fd7a1042 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:34.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.398+0000 7f6b8affd640 1 -- 172.21.15.117:0/2756247005 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c036030 con 0x557fbbc130a0 2024-09-21T05:56:34.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.398+0000 7f6b8affd640 1 -- 172.21.15.117:0/2756247005 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b9c011e20 con 0x557fbbc130a0 2024-09-21T05:56:34.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.398+0000 7f6b8affd640 1 -- 172.21.15.117:0/2756247005 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c015070 con 0x557fbbc130a0 2024-09-21T05:56:34.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.399+0000 7f6bb1836740 1 -- 172.21.15.117:0/2756247005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fbbc130a0 msgr2=0x557fbbfe8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:34.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.399+0000 7f6bb1836740 1 --2- 172.21.15.117:0/2756247005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fbbc130a0 0x557fbbfe8020 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c00fcb0 tx=0x7f6b9c00b440 comp rx=0 tx=0).stop 2024-09-21T05:56:34.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.399+0000 7f6bb1836740 1 -- 172.21.15.117:0/2756247005 shutdown_connections 2024-09-21T05:56:34.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.399+0000 7f6bb1836740 1 --2- 172.21.15.117:0/2756247005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fbbc3bf60 0x557fbbff9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.399+0000 7f6bb1836740 1 --2- 172.21.15.117:0/2756247005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fbbc1b280 0x557fbbff9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.399+0000 7f6bb1836740 1 --2- 172.21.15.117:0/2756247005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fbbc130a0 0x557fbbfe8020 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.399+0000 7f6bb1836740 1 -- 172.21.15.117:0/2756247005 >> 172.21.15.117:0/2756247005 conn(0x557fbbc30350 msgr2=0x557fbbfe89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:34.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.399+0000 7f6bb1836740 1 -- 172.21.15.117:0/2756247005 shutdown_connections 2024-09-21T05:56:34.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.399+0000 7f6bb1836740 1 -- 172.21.15.117:0/2756247005 wait complete. 2024-09-21T05:56:34.404 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.400+0000 7f6bb1836740 1 -- start start 2024-09-21T05:56:34.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.400+0000 7f6bb1836740 1 Processor -- start 2024-09-21T05:56:34.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.400+0000 7f6bb1836740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fbbc130a0 0x557fbc008770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:34.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.400+0000 7f6bb1836740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fbbc1b280 0x557fbc001c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:34.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.400+0000 7f6bb1836740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fbbc3bf60 0x557fbc002160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:34.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.400+0000 7f6bb1836740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557fbbfffc90 con 0x557fbbc3bf60 2024-09-21T05:56:34.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.400+0000 7f6bb1836740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557fbbfff5c0 con 0x557fbbc130a0 2024-09-21T05:56:34.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.400+0000 7f6bb1836740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557fbc000360 con 0x557fbbc1b280 2024-09-21T05:56:34.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.400+0000 7f6bb1836740 10 client.0 _ll_register_callbacks cb 0x7f6b940ed380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:56:34.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.400+0000 7f6ba985f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fbbc3bf60 0x557fbc002160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:34.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.400+0000 7f6ba985f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fbbc3bf60 0x557fbc002160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40726/0 (socket says 172.21.15.117:40726) 2024-09-21T05:56:34.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.400+0000 7f6ba985f640 1 -- 172.21.15.117:0/3711900631 learned_addr learned my addr 172.21.15.117:0/3711900631 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:34.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.400+0000 7f6ba985f640 1 --2- 172.21.15.117:0/3711900631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fbbc130a0 0x557fbc008770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:34.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.401+0000 7f6ba985f640 1 -- 172.21.15.117:0/3711900631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fbbc1b280 msgr2=0x557fbc001c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:34.405 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.401+0000 7f6ba985f640 1 --2- 172.21.15.117:0/3711900631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fbbc1b280 0x557fbc001c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.401+0000 7f6ba985f640 1 -- 172.21.15.117:0/3711900631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fbbc130a0 msgr2=0x557fbc008770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:34.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.401+0000 7f6ba985f640 1 --2- 172.21.15.117:0/3711900631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fbbc130a0 0x557fbc008770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.401+0000 7f6ba985f640 1 -- 172.21.15.117:0/3711900631 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fbc0027d0 con 0x557fbbc3bf60 2024-09-21T05:56:34.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.401+0000 7f6ba985f640 1 --2- 172.21.15.117:0/3711900631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fbbc3bf60 0x557fbc002160 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c00bb90 tx=0x7f6b9c028960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df136fb76dcca6da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:34.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.401+0000 7f6b88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:56:34.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.401+0000 7f6b88ff9640 1 -- 172.21.15.117:0/3711900631 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c036070 con 0x557fbbc3bf60 2024-09-21T05:56:34.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.401+0000 7f6b88ff9640 1 -- 172.21.15.117:0/3711900631 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f6b9c00be00 con 0x557fbbc3bf60 2024-09-21T05:56:34.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.401+0000 7f6bb1836740 1 -- 172.21.15.117:0/3711900631 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b9c011e20 con 0x557fbbc3bf60 2024-09-21T05:56:34.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.401+0000 7f6b88ff9640 1 -- 172.21.15.117:0/3711900631 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6b9c015040 con 0x557fbbc3bf60 2024-09-21T05:56:34.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.401+0000 7f6bb1836740 10 client.15258 fetch_fsmap learned FSMap version 26 2024-09-21T05:56:34.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.401+0000 7f6bb1836740 1 -- 172.21.15.117:0/3711900631 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557fbc0081e0 con 0x557fbbc3bf60 2024-09-21T05:56:34.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.401+0000 7f6b88ff9640 1 -- 172.21.15.117:0/3711900631 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b9c011e20 con 0x557fbbc3bf60 2024-09-21T05:56:34.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.401+0000 7f6b88ff9640 1 -- 172.21.15.117:0/3711900631 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f6b9c030040 con 0x557fbbc3bf60 2024-09-21T05:56:34.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.402+0000 7f6bb1836740 10 client.15258 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T05:56:34.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.402+0000 7f6bb1836740 10 client.15258 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T05:56:34.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.402+0000 7f6bb1836740 20 client.15258 populate_metadata read hostname 'smithi117' 2024-09-21T05:56:34.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.402+0000 7f6bb1836740 1 --2- 172.21.15.117:0/3711900631 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x557fbbbcfc90 0x557fbc06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:34.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.402+0000 7f6bb1836740 4 client.15258 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T05:56:34.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.402+0000 7f6bb1836740 1 -- 172.21.15.117:0/3711900631 --> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] -- command(tid 0: {"prefix": "status"}) -- 0x557fbbf57610 con 0x557fbbbcfc90 2024-09-21T05:56:34.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.402+0000 7f6ba885d640 1 --2- 172.21.15.117:0/3711900631 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x557fbbbcfc90 0x557fbc06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:34.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.402+0000 7f6b88ff9640 10 client.15258 ms_handle_connect on v2:172.21.15.192:6836/2683860842 2024-09-21T05:56:34.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.402+0000 7f6ba885d640 1 --2- 172.21.15.117:0/3711900631 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x557fbbbcfc90 0x557fbc06e7a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4b1e32a9aaa8b525 server_cookie=de19c34f626797ca in_seq=0 out_seq=0 2024-09-21T05:56:34.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.403+0000 7f6b88ff9640 1 -- 172.21.15.117:0/3711900631 <== mds.2 v2:172.21.15.192:6836/2683860842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x557fbbf57610 con 0x557fbbbcfc90 2024-09-21T05:56:34.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.403+0000 7f6b88ff9640 10 client.15258 handle_command_reply: tid=0 2024-09-21T05:56:34.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.403+0000 7f6bb1836740 1 client.15258 shutdown 2024-09-21T05:56:34.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.403+0000 7f6bb1836740 1 -- 172.21.15.117:0/3711900631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fbbc3bf60 msgr2=0x557fbc002160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:34.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.403+0000 7f6bb1836740 1 --2- 172.21.15.117:0/3711900631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fbbc3bf60 0x557fbc002160 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c00bb90 tx=0x7f6b9c028960 comp rx=0 tx=0).stop 2024-09-21T05:56:34.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.403+0000 7f6bb1836740 1 -- 172.21.15.117:0/3711900631 shutdown_connections 2024-09-21T05:56:34.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.403+0000 7f6bb1836740 1 --2- 172.21.15.117:0/3711900631 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x557fbbbcfc90 0x557fbc06e7a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.403+0000 7f6bb1836740 1 --2- 172.21.15.117:0/3711900631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557fbbc3bf60 0x557fbc002160 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.403+0000 7f6bb1836740 1 --2- 172.21.15.117:0/3711900631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557fbbc1b280 0x557fbc001c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.403+0000 7f6b88ff9640 0 client.15258 ms_handle_reset on v2:172.21.15.192:6836/2683860842 2024-09-21T05:56:34.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.403+0000 7f6bb1836740 1 --2- 172.21.15.117:0/3711900631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557fbbc130a0 0x557fbc008770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.403+0000 7f6bb1836740 1 -- 172.21.15.117:0/3711900631 >> 172.21.15.117:0/3711900631 conn(0x557fbbc30350 msgr2=0x557fbbfdf730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:34.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.403+0000 7f6bb1836740 1 -- 172.21.15.117:0/3711900631 shutdown_connections 2024-09-21T05:56:34.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.403+0000 7f6bb1836740 1 -- 172.21.15.117:0/3711900631 wait complete. 2024-09-21T05:56:34.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.403+0000 7f6bb1836740 20 client.15258 trim_cache size 0 max 16384 2024-09-21T05:56:34.409 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T05:56:34.409 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_fsid": "ac8581a2-77dc-11ef-baf5-efdc52797490", 2024-09-21T05:56:34.410 INFO:teuthology.orchestra.run.smithi117.stdout: "whoami": 2, 2024-09-21T05:56:34.410 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 24628, 2024-09-21T05:56:34.410 INFO:teuthology.orchestra.run.smithi117.stdout: "want_state": "up:active", 2024-09-21T05:56:34.410 INFO:teuthology.orchestra.run.smithi117.stdout: "state": "up:active", 2024-09-21T05:56:34.410 INFO:teuthology.orchestra.run.smithi117.stdout: "fs_name": "cephfs", 2024-09-21T05:56:34.410 INFO:teuthology.orchestra.run.smithi117.stdout: "rank_uptime": 71.213772367000004, 2024-09-21T05:56:34.410 INFO:teuthology.orchestra.run.smithi117.stdout: "mdsmap_epoch": 26, 2024-09-21T05:56:34.410 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch": 76, 2024-09-21T05:56:34.410 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch_barrier": 74, 2024-09-21T05:56:34.410 INFO:teuthology.orchestra.run.smithi117.stdout: "uptime": 105.192847661 2024-09-21T05:56:34.410 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T05:56:34.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.403+0000 7f6bab2c2640 1 -- 172.21.15.117:0/2098071887 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6b8c0b6670 msgr2=0x7f6b8c0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:34.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.403+0000 7f6bab2c2640 1 --2- 172.21.15.117:0/2098071887 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6b8c0b6670 0x7f6b8c0b8b80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x557fbbfe0b80 tx=0x7f6b940ed4b0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.404+0000 7f6bab2c2640 1 -- 172.21.15.117:0/2098071887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ba41049b0 msgr2=0x7f6ba419a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:34.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.404+0000 7f6bab2c2640 1 --2- 172.21.15.117:0/2098071887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ba41049b0 0x7f6ba419a460 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6b9400ccb0 tx=0x7f6b9400a620 comp rx=0 tx=0).stop 2024-09-21T05:56:34.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.404+0000 7f6bab2c2640 1 -- 172.21.15.117:0/2098071887 shutdown_connections 2024-09-21T05:56:34.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.404+0000 7f6bab2c2640 1 --2- 172.21.15.117:0/2098071887 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6b8c0b6670 0x7f6b8c0b8b80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.404+0000 7f6bab2c2640 1 --2- 172.21.15.117:0/2098071887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ba410de10 0x7f6ba419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.404+0000 7f6bab2c2640 1 --2- 172.21.15.117:0/2098071887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ba4105300 0x7f6ba419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.404+0000 7f6bab2c2640 1 --2- 172.21.15.117:0/2098071887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ba41049b0 0x7f6ba419a460 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:34.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.404+0000 7f6bab2c2640 1 -- 172.21.15.117:0/2098071887 >> 172.21.15.117:0/2098071887 conn(0x7f6ba4079730 msgr2=0x7f6ba4100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:34.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.404+0000 7f6bab2c2640 1 -- 172.21.15.117:0/2098071887 shutdown_connections 2024-09-21T05:56:34.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:34.404+0000 7f6bab2c2640 1 -- 172.21.15.117:0/2098071887 wait complete. 2024-09-21T05:56:35.068 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.i status 2024-09-21T05:56:35.315 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:35.751 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:35 smithi117 ceph-mon[30947]: pgmap v305: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T05:56:35.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.838+0000 7f4c00173640 1 -- 172.21.15.117:0/3036185611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bf8076080 msgr2=0x7f4bf810fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:35.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.838+0000 7f4c00173640 1 --2- 172.21.15.117:0/3036185611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bf8076080 0x7f4bf810fc60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4bec009900 tx=0x7f4bec02a410 comp rx=0 tx=0).stop 2024-09-21T05:56:35.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.838+0000 7f4c00173640 1 -- 172.21.15.117:0/3036185611 shutdown_connections 2024-09-21T05:56:35.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.838+0000 7f4c00173640 1 --2- 172.21.15.117:0/3036185611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bf8076080 0x7f4bf810fc60 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.838+0000 7f4c00173640 1 --2- 172.21.15.117:0/3036185611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bf81101a0 0x7f4bf81125c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.838+0000 7f4c00173640 1 --2- 172.21.15.117:0/3036185611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bf8075730 0x7f4bf8075b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.838+0000 7f4c00173640 1 -- 172.21.15.117:0/3036185611 >> 172.21.15.117:0/3036185611 conn(0x7f4bf807b910 msgr2=0x7f4bf807bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:35.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.838+0000 7f4c00173640 1 -- 172.21.15.117:0/3036185611 shutdown_connections 2024-09-21T05:56:35.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.838+0000 7f4c00173640 1 -- 172.21.15.117:0/3036185611 wait complete. 2024-09-21T05:56:35.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.839+0000 7f4c00173640 1 Processor -- start 2024-09-21T05:56:35.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.839+0000 7f4c00173640 1 -- start start 2024-09-21T05:56:35.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.839+0000 7f4c00173640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bf8075730 0x7f4bf819c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:35.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.839+0000 7f4c00173640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bf8076080 0x7f4bf819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:35.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.839+0000 7f4c00173640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bf81101a0 0x7f4bf81a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:35.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.839+0000 7f4c00173640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf8115cd0 con 0x7f4bf8076080 2024-09-21T05:56:35.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.839+0000 7f4c00173640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf8115b50 con 0x7f4bf81101a0 2024-09-21T05:56:35.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.839+0000 7f4c00173640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf8115e50 con 0x7f4bf8075730 2024-09-21T05:56:35.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.839+0000 7f4bfd70e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bf8076080 0x7f4bf819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:35.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.839+0000 7f4bfd70e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bf8076080 0x7f4bf819cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54300/0 (socket says 172.21.15.117:54300) 2024-09-21T05:56:35.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.839+0000 7f4bfd70e640 1 -- 172.21.15.117:0/3337923099 learned_addr learned my addr 172.21.15.117:0/3337923099 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:35.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.839+0000 7f4bfdf0f640 1 --2- 172.21.15.117:0/3337923099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bf8075730 0x7f4bf819c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:35.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.839+0000 7f4bfd70e640 1 -- 172.21.15.117:0/3337923099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bf8075730 msgr2=0x7f4bf819c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:35.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.839+0000 7f4bfd70e640 1 --2- 172.21.15.117:0/3337923099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bf8075730 0x7f4bf819c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.839+0000 7f4bfd70e640 1 -- 172.21.15.117:0/3337923099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bf81101a0 msgr2=0x7f4bf81a1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:35.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.839+0000 7f4bfd70e640 1 --2- 172.21.15.117:0/3337923099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bf81101a0 0x7f4bf81a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.839+0000 7f4bfd70e640 1 -- 172.21.15.117:0/3337923099 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bf81a25f0 con 0x7f4bf8076080 2024-09-21T05:56:35.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.840+0000 7f4bfd70e640 1 --2- 172.21.15.117:0/3337923099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bf8076080 0x7f4bf819cce0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4bec02a930 tx=0x7f4bec004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:35.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.840+0000 7f4bf6ffd640 1 -- 172.21.15.117:0/3337923099 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bec03f030 con 0x7f4bf8076080 2024-09-21T05:56:35.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.840+0000 7f4bf6ffd640 1 -- 172.21.15.117:0/3337923099 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bec003cb0 con 0x7f4bf8076080 2024-09-21T05:56:35.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.840+0000 7f4c00173640 1 -- 172.21.15.117:0/3337923099 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bf81a2880 con 0x7f4bf8076080 2024-09-21T05:56:35.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.840+0000 7f4bf6ffd640 1 -- 172.21.15.117:0/3337923099 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bec035070 con 0x7f4bf8076080 2024-09-21T05:56:35.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.840+0000 7f4c00173640 1 -- 172.21.15.117:0/3337923099 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bf81a9100 con 0x7f4bf8076080 2024-09-21T05:56:35.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.842+0000 7f4c066e7740 1 Processor -- start 2024-09-21T05:56:35.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.842+0000 7f4c066e7740 1 -- start start 2024-09-21T05:56:35.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.842+0000 7f4bf6ffd640 1 -- 172.21.15.117:0/3337923099 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f4bec042030 con 0x7f4bf8076080 2024-09-21T05:56:35.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.843+0000 7f4c066e7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb7c263f60 0x55bb7c609f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:35.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.843+0000 7f4c066e7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb7c23b0a0 0x55bb7c60a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:35.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.843+0000 7f4c066e7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb7c243280 0x55bb7c60a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:35.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.843+0000 7f4c066e7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bb7c581c10 con 0x55bb7c23b0a0 2024-09-21T05:56:35.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.843+0000 7f4c066e7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bb7c571040 con 0x55bb7c263f60 2024-09-21T05:56:35.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.843+0000 7f4c066e7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55bb7c574e50 con 0x55bb7c243280 2024-09-21T05:56:35.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.843+0000 7f4bfdf0f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb7c23b0a0 0x55bb7c60a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:35.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.843+0000 7f4bfdf0f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb7c23b0a0 0x55bb7c60a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54316/0 (socket says 172.21.15.117:54316) 2024-09-21T05:56:35.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.843+0000 7f4bfdf0f640 1 -- 172.21.15.117:0/2528315194 learned_addr learned my addr 172.21.15.117:0/2528315194 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:35.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.843+0000 7f4bfd70e640 1 --2- 172.21.15.117:0/2528315194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb7c243280 0x55bb7c60a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:35.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.843+0000 7f4bfe710640 1 --2- 172.21.15.117:0/2528315194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb7c263f60 0x55bb7c609f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:35.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.843+0000 7f4bfdf0f640 1 -- 172.21.15.117:0/2528315194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb7c243280 msgr2=0x55bb7c60a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:35.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.843+0000 7f4bfdf0f640 1 --2- 172.21.15.117:0/2528315194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb7c243280 0x55bb7c60a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.843+0000 7f4bfdf0f640 1 -- 172.21.15.117:0/2528315194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb7c263f60 msgr2=0x55bb7c609f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:35.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.843+0000 7f4bfdf0f640 1 --2- 172.21.15.117:0/2528315194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb7c263f60 0x55bb7c609f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.843+0000 7f4bfdf0f640 1 -- 172.21.15.117:0/2528315194 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb7c2cf540 con 0x55bb7c23b0a0 2024-09-21T05:56:35.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.843+0000 7f4bfd70e640 1 --2- 172.21.15.117:0/2528315194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb7c243280 0x55bb7c60a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:56:35.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.843+0000 7f4bf6ffd640 1 --2- 172.21.15.117:0/3337923099 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4be00b66a0 0x7f4be00b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:35.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.843+0000 7f4bfdf0f640 1 --2- 172.21.15.117:0/2528315194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb7c23b0a0 0x55bb7c60a440 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4be8009fa0 tx=0x7f4be800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d075e6a0971c3236 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:35.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.843+0000 7f4bfe710640 1 --2- 172.21.15.117:0/3337923099 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4be00b66a0 0x7f4be00b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:35.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.843+0000 7f4bf6ffd640 1 -- 172.21.15.117:0/3337923099 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f4bec107c10 con 0x7f4bf8076080 2024-09-21T05:56:35.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.844+0000 7f4bdffff640 1 -- 172.21.15.117:0/2528315194 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be801a070 con 0x55bb7c23b0a0 2024-09-21T05:56:35.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.844+0000 7f4bdffff640 1 -- 172.21.15.117:0/2528315194 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4be80158c0 con 0x55bb7c23b0a0 2024-09-21T05:56:35.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.844+0000 7f4bdffff640 1 -- 172.21.15.117:0/2528315194 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be800a910 con 0x55bb7c23b0a0 2024-09-21T05:56:35.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.844+0000 7f4c066e7740 1 -- 172.21.15.117:0/2528315194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb7c23b0a0 msgr2=0x55bb7c60a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:35.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.844+0000 7f4bfe710640 1 --2- 172.21.15.117:0/3337923099 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4be00b66a0 0x7f4be00b8bb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4bf000cad0 tx=0x7f4bf0005e30 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:35.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.844+0000 7f4c066e7740 1 --2- 172.21.15.117:0/2528315194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb7c23b0a0 0x55bb7c60a440 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4be8009fa0 tx=0x7f4be800c8b0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.845+0000 7f4c066e7740 1 -- 172.21.15.117:0/2528315194 shutdown_connections 2024-09-21T05:56:35.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.845+0000 7f4c066e7740 1 --2- 172.21.15.117:0/2528315194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb7c23b0a0 0x55bb7c60a440 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.845+0000 7f4c066e7740 1 --2- 172.21.15.117:0/2528315194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb7c243280 0x55bb7c60a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.845+0000 7f4c066e7740 1 --2- 172.21.15.117:0/2528315194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb7c263f60 0x55bb7c609f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.845+0000 7f4c066e7740 1 -- 172.21.15.117:0/2528315194 >> 172.21.15.117:0/2528315194 conn(0x55bb7c258350 msgr2=0x55bb7c607df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:35.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.845+0000 7f4c066e7740 1 -- 172.21.15.117:0/2528315194 shutdown_connections 2024-09-21T05:56:35.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.845+0000 7f4c066e7740 1 -- 172.21.15.117:0/2528315194 wait complete. 2024-09-21T05:56:35.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.845+0000 7f4c066e7740 1 -- start start 2024-09-21T05:56:35.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.845+0000 7f4c066e7740 1 Processor -- start 2024-09-21T05:56:35.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.846+0000 7f4c066e7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb7c23b0a0 0x55bb7c6159f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:35.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.846+0000 7f4c066e7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb7c243280 0x55bb7c615f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:35.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.846+0000 7f4c066e7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb7c263f60 0x55bb7c6219d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:35.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.846+0000 7f4c066e7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bb7c3d25f0 con 0x55bb7c243280 2024-09-21T05:56:35.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.846+0000 7f4c066e7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bb7c50c010 con 0x55bb7c263f60 2024-09-21T05:56:35.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.846+0000 7f4c066e7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55bb7c6186e0 con 0x55bb7c23b0a0 2024-09-21T05:56:35.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.846+0000 7f4c066e7740 10 client.0 _ll_register_callbacks cb 0x55bb7c56b340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:56:35.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.846+0000 7f4bfd70e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb7c243280 0x55bb7c615f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:35.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.846+0000 7f4bfd70e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb7c243280 0x55bb7c615f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54320/0 (socket says 172.21.15.117:54320) 2024-09-21T05:56:35.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.846+0000 7f4bfd70e640 1 -- 172.21.15.117:0/1038803736 learned_addr learned my addr 172.21.15.117:0/1038803736 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:35.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.846+0000 7f4bfdf0f640 1 --2- 172.21.15.117:0/1038803736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb7c23b0a0 0x55bb7c6159f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:35.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.846+0000 7f4bfe710640 1 --2- 172.21.15.117:0/1038803736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb7c263f60 0x55bb7c6219d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:35.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.846+0000 7f4bfd70e640 1 -- 172.21.15.117:0/1038803736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb7c23b0a0 msgr2=0x55bb7c6159f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:35.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.846+0000 7f4bfd70e640 1 --2- 172.21.15.117:0/1038803736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb7c23b0a0 0x55bb7c6159f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.846+0000 7f4bfd70e640 1 -- 172.21.15.117:0/1038803736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb7c263f60 msgr2=0x55bb7c6219d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:35.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.846+0000 7f4bfd70e640 1 --2- 172.21.15.117:0/1038803736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb7c263f60 0x55bb7c6219d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.846+0000 7f4bfd70e640 1 -- 172.21.15.117:0/1038803736 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb7c539a00 con 0x55bb7c243280 2024-09-21T05:56:35.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.846+0000 7f4bfd70e640 1 --2- 172.21.15.117:0/1038803736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb7c243280 0x55bb7c615f30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4bec0cd8a0 tx=0x7f4bec0c9a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fbb72d6b7f3a9c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:35.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.846+0000 7f4bdd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:56:35.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.846+0000 7f4bdd7fa640 1 -- 172.21.15.117:0/1038803736 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bec049030 con 0x55bb7c243280 2024-09-21T05:56:35.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.847+0000 7f4bdd7fa640 1 -- 172.21.15.117:0/1038803736 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f4bec03bbe0 con 0x55bb7c243280 2024-09-21T05:56:35.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.847+0000 7f4c066e7740 1 -- 172.21.15.117:0/1038803736 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4be80158c0 con 0x55bb7c243280 2024-09-21T05:56:35.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.847+0000 7f4bdd7fa640 1 -- 172.21.15.117:0/1038803736 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4bec0cd310 con 0x55bb7c243280 2024-09-21T05:56:35.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.847+0000 7f4c066e7740 10 client.15270 fetch_fsmap learned FSMap version 26 2024-09-21T05:56:35.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.847+0000 7f4c066e7740 1 -- 172.21.15.117:0/1038803736 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bb7c6245d0 con 0x55bb7c243280 2024-09-21T05:56:35.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.847+0000 7f4bdd7fa640 1 -- 172.21.15.117:0/1038803736 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4be80158c0 con 0x55bb7c243280 2024-09-21T05:56:35.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.847+0000 7f4bdd7fa640 1 -- 172.21.15.117:0/1038803736 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f4bec0d7130 con 0x55bb7c243280 2024-09-21T05:56:35.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.847+0000 7f4c066e7740 10 client.15270 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T05:56:35.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.847+0000 7f4c066e7740 10 client.15270 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T05:56:35.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.847+0000 7f4c066e7740 20 client.15270 populate_metadata read hostname 'smithi117' 2024-09-21T05:56:35.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.847+0000 7f4c066e7740 1 --2- 172.21.15.117:0/1038803736 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55bb7c2228e0 0x55bb7c628010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:35.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.847+0000 7f4c066e7740 4 client.15270 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:56:35.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.847+0000 7f4c066e7740 1 -- 172.21.15.117:0/1038803736 --> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb7c583b60 con 0x55bb7c2228e0 2024-09-21T05:56:35.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.848+0000 7f4bfdf0f640 1 --2- 172.21.15.117:0/1038803736 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55bb7c2228e0 0x55bb7c628010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:35.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.848+0000 7f4bfdf0f640 1 --2- 172.21.15.117:0/1038803736 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55bb7c2228e0 0x55bb7c628010 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3e6fb045b37d8994 server_cookie=cc3b2705ce147a11 in_seq=0 out_seq=0 2024-09-21T05:56:35.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.848+0000 7f4bdd7fa640 10 client.15270 ms_handle_connect on v2:172.21.15.192:6838/1727623231 2024-09-21T05:56:35.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.849+0000 7f4bdd7fa640 1 -- 172.21.15.117:0/1038803736 <== mds.1 v2:172.21.15.192:6838/1727623231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bb7c583b60 con 0x55bb7c2228e0 2024-09-21T05:56:35.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.849+0000 7f4bdd7fa640 10 client.15270 handle_command_reply: tid=0 2024-09-21T05:56:35.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.849+0000 7f4c066e7740 1 client.15270 shutdown 2024-09-21T05:56:35.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.849+0000 7f4c066e7740 1 -- 172.21.15.117:0/1038803736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb7c243280 msgr2=0x55bb7c615f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:35.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.849+0000 7f4c066e7740 1 --2- 172.21.15.117:0/1038803736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb7c243280 0x55bb7c615f30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4bec0cd8a0 tx=0x7f4bec0c9a40 comp rx=0 tx=0).stop 2024-09-21T05:56:35.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.849+0000 7f4c066e7740 1 -- 172.21.15.117:0/1038803736 shutdown_connections 2024-09-21T05:56:35.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.849+0000 7f4c066e7740 1 --2- 172.21.15.117:0/1038803736 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55bb7c2228e0 0x55bb7c628010 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.849+0000 7f4c066e7740 1 --2- 172.21.15.117:0/1038803736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb7c243280 0x55bb7c615f30 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.849+0000 7f4c066e7740 1 --2- 172.21.15.117:0/1038803736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb7c263f60 0x55bb7c6219d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.849+0000 7f4bdd7fa640 0 client.15270 ms_handle_reset on v2:172.21.15.192:6838/1727623231 2024-09-21T05:56:35.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.849+0000 7f4c066e7740 1 --2- 172.21.15.117:0/1038803736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb7c23b0a0 0x55bb7c6159f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.849+0000 7f4c066e7740 1 -- 172.21.15.117:0/1038803736 >> 172.21.15.117:0/1038803736 conn(0x55bb7c258350 msgr2=0x55bb7c60d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:35.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.849+0000 7f4c066e7740 1 -- 172.21.15.117:0/1038803736 shutdown_connections 2024-09-21T05:56:35.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.849+0000 7f4c066e7740 1 -- 172.21.15.117:0/1038803736 wait complete. 2024-09-21T05:56:35.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.849+0000 7f4c066e7740 20 client.15270 trim_cache size 0 max 16384 2024-09-21T05:56:35.869 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:35 smithi138 ceph-mon[34391]: pgmap v305: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T05:56:35.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.867+0000 7f4c066e7740 1 Processor -- start 2024-09-21T05:56:35.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.867+0000 7f4c066e7740 1 -- start start 2024-09-21T05:56:35.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.867+0000 7f4c066e7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb7c23b0a0 0x55bb7c607d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:35.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.867+0000 7f4c066e7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb7c243280 0x55bb7c621940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:35.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.867+0000 7f4c066e7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb7c263f60 0x55bb7c621e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:35.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.867+0000 7f4c066e7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bb7c626350 con 0x55bb7c23b0a0 2024-09-21T05:56:35.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.867+0000 7f4c066e7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bb7c626a20 con 0x55bb7c263f60 2024-09-21T05:56:35.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.867+0000 7f4c066e7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55bb7c6270f0 con 0x55bb7c243280 2024-09-21T05:56:35.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.867+0000 7f4bfdf0f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb7c23b0a0 0x55bb7c607d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:35.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.868+0000 7f4bfdf0f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb7c23b0a0 0x55bb7c607d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54322/0 (socket says 172.21.15.117:54322) 2024-09-21T05:56:35.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.868+0000 7f4bfdf0f640 1 -- 172.21.15.117:0/2419083629 learned_addr learned my addr 172.21.15.117:0/2419083629 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:35.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.868+0000 7f4bfe710640 1 --2- 172.21.15.117:0/2419083629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb7c263f60 0x55bb7c621e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:35.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.868+0000 7f4bfdf0f640 1 -- 172.21.15.117:0/2419083629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb7c243280 msgr2=0x55bb7c621940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:35.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.868+0000 7f4bfdf0f640 1 --2- 172.21.15.117:0/2419083629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb7c243280 0x55bb7c621940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.868+0000 7f4bfdf0f640 1 -- 172.21.15.117:0/2419083629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb7c263f60 msgr2=0x55bb7c621e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:35.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.868+0000 7f4bfdf0f640 1 --2- 172.21.15.117:0/2419083629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb7c263f60 0x55bb7c621e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.868+0000 7f4bfdf0f640 1 -- 172.21.15.117:0/2419083629 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb7c613cf0 con 0x55bb7c23b0a0 2024-09-21T05:56:35.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.868+0000 7f4bfe710640 1 --2- 172.21.15.117:0/2419083629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb7c263f60 0x55bb7c621e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:35.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.868+0000 7f4bfdf0f640 1 --2- 172.21.15.117:0/2419083629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb7c23b0a0 0x55bb7c607d10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4be8009ba0 tx=0x7f4be8004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b3f3faa5de507f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:35.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.868+0000 7f4bdf7fe640 1 -- 172.21.15.117:0/2419083629 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be802f070 con 0x55bb7c23b0a0 2024-09-21T05:56:35.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.868+0000 7f4bdf7fe640 1 -- 172.21.15.117:0/2419083629 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4be8015cf0 con 0x55bb7c23b0a0 2024-09-21T05:56:35.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.868+0000 7f4bdf7fe640 1 -- 172.21.15.117:0/2419083629 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be801a040 con 0x55bb7c23b0a0 2024-09-21T05:56:35.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.869+0000 7f4c066e7740 1 -- 172.21.15.117:0/2419083629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb7c23b0a0 msgr2=0x55bb7c607d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:35.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.869+0000 7f4c066e7740 1 --2- 172.21.15.117:0/2419083629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb7c23b0a0 0x55bb7c607d10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4be8009ba0 tx=0x7f4be8004b50 comp rx=0 tx=0).stop 2024-09-21T05:56:35.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.869+0000 7f4c066e7740 1 -- 172.21.15.117:0/2419083629 shutdown_connections 2024-09-21T05:56:35.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.869+0000 7f4c066e7740 1 --2- 172.21.15.117:0/2419083629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb7c263f60 0x55bb7c621e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.869+0000 7f4c066e7740 1 --2- 172.21.15.117:0/2419083629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb7c243280 0x55bb7c621940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.869+0000 7f4c066e7740 1 --2- 172.21.15.117:0/2419083629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb7c23b0a0 0x55bb7c607d10 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.869+0000 7f4c066e7740 1 -- 172.21.15.117:0/2419083629 >> 172.21.15.117:0/2419083629 conn(0x55bb7c258350 msgr2=0x55bb7c607600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:35.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.869+0000 7f4c066e7740 1 -- 172.21.15.117:0/2419083629 shutdown_connections 2024-09-21T05:56:35.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.869+0000 7f4c066e7740 1 -- 172.21.15.117:0/2419083629 wait complete. 2024-09-21T05:56:35.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.869+0000 7f4c066e7740 1 -- start start 2024-09-21T05:56:35.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.869+0000 7f4c066e7740 1 Processor -- start 2024-09-21T05:56:35.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.870+0000 7f4c066e7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb7c23b0a0 0x55bb7c62a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:35.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.870+0000 7f4c066e7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb7c243280 0x55bb7c62a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:35.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.870+0000 7f4c066e7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb7c263f60 0x55bb7c62af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:35.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.870+0000 7f4c066e7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bb7c627e90 con 0x55bb7c243280 2024-09-21T05:56:35.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.870+0000 7f4c066e7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bb7c6277c0 con 0x55bb7c263f60 2024-09-21T05:56:35.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.870+0000 7f4c066e7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55bb7c628560 con 0x55bb7c23b0a0 2024-09-21T05:56:35.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.870+0000 7f4c066e7740 10 client.0 _ll_register_callbacks cb 0x7f4bec005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:56:35.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.870+0000 7f4bfd70e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb7c243280 0x55bb7c62a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:35.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.870+0000 7f4bfd70e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb7c243280 0x55bb7c62a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54334/0 (socket says 172.21.15.117:54334) 2024-09-21T05:56:35.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.870+0000 7f4bfd70e640 1 -- 172.21.15.117:0/2901727161 learned_addr learned my addr 172.21.15.117:0/2901727161 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:35.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.870+0000 7f4bfe710640 1 --2- 172.21.15.117:0/2901727161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb7c263f60 0x55bb7c62af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:35.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.870+0000 7f4bfdf0f640 1 --2- 172.21.15.117:0/2901727161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb7c23b0a0 0x55bb7c62a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:35.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.870+0000 7f4bfd70e640 1 -- 172.21.15.117:0/2901727161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb7c23b0a0 msgr2=0x55bb7c62a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:35.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.870+0000 7f4bfd70e640 1 --2- 172.21.15.117:0/2901727161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb7c23b0a0 0x55bb7c62a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.870+0000 7f4bfd70e640 1 -- 172.21.15.117:0/2901727161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb7c263f60 msgr2=0x55bb7c62af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:35.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.870+0000 7f4bfd70e640 1 --2- 172.21.15.117:0/2901727161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb7c263f60 0x55bb7c62af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.870+0000 7f4bfd70e640 1 -- 172.21.15.117:0/2901727161 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb7c62fde0 con 0x55bb7c243280 2024-09-21T05:56:35.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.870+0000 7f4bfd70e640 1 --2- 172.21.15.117:0/2901727161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb7c243280 0x55bb7c62a9e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4bec006010 tx=0x7f4bec03bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77bdcad6bbaa6a38 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:35.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.871+0000 7f4bdd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:56:35.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.871+0000 7f4bdd7fa640 1 -- 172.21.15.117:0/2901727161 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bec049050 con 0x55bb7c243280 2024-09-21T05:56:35.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.871+0000 7f4bdd7fa640 1 -- 172.21.15.117:0/2901727161 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f4bec0efdb0 con 0x55bb7c243280 2024-09-21T05:56:35.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.871+0000 7f4c066e7740 1 -- 172.21.15.117:0/2901727161 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4be8015cf0 con 0x55bb7c243280 2024-09-21T05:56:35.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.871+0000 7f4bdd7fa640 1 -- 172.21.15.117:0/2901727161 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4bec0cd970 con 0x55bb7c243280 2024-09-21T05:56:35.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.871+0000 7f4c066e7740 10 client.15276 fetch_fsmap learned FSMap version 26 2024-09-21T05:56:35.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.871+0000 7f4c066e7740 1 -- 172.21.15.117:0/2901727161 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bb7c693880 con 0x55bb7c243280 2024-09-21T05:56:35.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.871+0000 7f4bdd7fa640 1 -- 172.21.15.117:0/2901727161 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4be8015cf0 con 0x55bb7c243280 2024-09-21T05:56:35.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.871+0000 7f4bdd7fa640 1 -- 172.21.15.117:0/2901727161 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f4bec105b30 con 0x55bb7c243280 2024-09-21T05:56:35.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.871+0000 7f4c066e7740 10 client.15276 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T05:56:35.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.871+0000 7f4c066e7740 10 client.15276 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T05:56:35.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.871+0000 7f4c066e7740 20 client.15276 populate_metadata read hostname 'smithi117' 2024-09-21T05:56:35.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.871+0000 7f4c066e7740 1 --2- 172.21.15.117:0/2901727161 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55bb7c1f7c90 0x55bb7c696ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:35.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.871+0000 7f4c066e7740 4 client.15276 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T05:56:35.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.871+0000 7f4c066e7740 1 -- 172.21.15.117:0/2901727161 --> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] -- command(tid 0: {"prefix": "status"}) -- 0x55bb7c57f610 con 0x55bb7c1f7c90 2024-09-21T05:56:35.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.872+0000 7f4bfdf0f640 1 --2- 172.21.15.117:0/2901727161 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55bb7c1f7c90 0x55bb7c696ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:35.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.872+0000 7f4bdd7fa640 10 client.15276 ms_handle_connect on v2:172.21.15.192:6838/1727623231 2024-09-21T05:56:35.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.872+0000 7f4bfdf0f640 1 --2- 172.21.15.117:0/2901727161 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55bb7c1f7c90 0x55bb7c696ee0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7066d5181cc69c7a server_cookie=4f3213888479d6ca in_seq=0 out_seq=0 2024-09-21T05:56:35.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.872+0000 7f4bdd7fa640 1 -- 172.21.15.117:0/2901727161 <== mds.1 v2:172.21.15.192:6838/1727623231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55bb7c57f610 con 0x55bb7c1f7c90 2024-09-21T05:56:35.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.872+0000 7f4bdd7fa640 10 client.15276 handle_command_reply: tid=0 2024-09-21T05:56:35.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.872+0000 7f4c066e7740 1 client.15276 shutdown 2024-09-21T05:56:35.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.873+0000 7f4c066e7740 1 -- 172.21.15.117:0/2901727161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb7c243280 msgr2=0x55bb7c62a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:35.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.873+0000 7f4c066e7740 1 --2- 172.21.15.117:0/2901727161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb7c243280 0x55bb7c62a9e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4bec006010 tx=0x7f4bec03bdf0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.873+0000 7f4c066e7740 1 -- 172.21.15.117:0/2901727161 shutdown_connections 2024-09-21T05:56:35.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.873+0000 7f4c066e7740 1 --2- 172.21.15.117:0/2901727161 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55bb7c1f7c90 0x55bb7c696ee0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.873+0000 7f4c066e7740 1 --2- 172.21.15.117:0/2901727161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb7c243280 0x55bb7c62a9e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.873+0000 7f4bdd7fa640 0 client.15276 ms_handle_reset on v2:172.21.15.192:6838/1727623231 2024-09-21T05:56:35.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.873+0000 7f4c066e7740 1 --2- 172.21.15.117:0/2901727161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb7c263f60 0x55bb7c62af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.873+0000 7f4c066e7740 1 --2- 172.21.15.117:0/2901727161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb7c23b0a0 0x55bb7c62a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.873+0000 7f4c066e7740 1 -- 172.21.15.117:0/2901727161 >> 172.21.15.117:0/2901727161 conn(0x55bb7c258350 msgr2=0x55bb7c60d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:35.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.873+0000 7f4c066e7740 1 -- 172.21.15.117:0/2901727161 shutdown_connections 2024-09-21T05:56:35.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.873+0000 7f4c066e7740 1 -- 172.21.15.117:0/2901727161 wait complete. 2024-09-21T05:56:35.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.873+0000 7f4c066e7740 20 client.15276 trim_cache size 0 max 16384 2024-09-21T05:56:35.879 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T05:56:35.880 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_fsid": "ac8581a2-77dc-11ef-baf5-efdc52797490", 2024-09-21T05:56:35.880 INFO:teuthology.orchestra.run.smithi117.stdout: "whoami": 1, 2024-09-21T05:56:35.880 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 24637, 2024-09-21T05:56:35.880 INFO:teuthology.orchestra.run.smithi117.stdout: "want_state": "up:active", 2024-09-21T05:56:35.880 INFO:teuthology.orchestra.run.smithi117.stdout: "state": "up:active", 2024-09-21T05:56:35.880 INFO:teuthology.orchestra.run.smithi117.stdout: "fs_name": "cephfs", 2024-09-21T05:56:35.880 INFO:teuthology.orchestra.run.smithi117.stdout: "rank_uptime": 73.690565207999995, 2024-09-21T05:56:35.880 INFO:teuthology.orchestra.run.smithi117.stdout: "mdsmap_epoch": 26, 2024-09-21T05:56:35.880 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch": 76, 2024-09-21T05:56:35.880 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch_barrier": 74, 2024-09-21T05:56:35.880 INFO:teuthology.orchestra.run.smithi117.stdout: "uptime": 101.457178172 2024-09-21T05:56:35.880 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T05:56:35.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.873+0000 7f4c00173640 1 -- 172.21.15.117:0/3337923099 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4be00b66a0 msgr2=0x7f4be00b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:35.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.873+0000 7f4c00173640 1 --2- 172.21.15.117:0/3337923099 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4be00b66a0 0x7f4be00b8bb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4bf000cad0 tx=0x7f4bf0005e30 comp rx=0 tx=0).stop 2024-09-21T05:56:35.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.873+0000 7f4c00173640 1 -- 172.21.15.117:0/3337923099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bf8076080 msgr2=0x7f4bf819cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:35.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.873+0000 7f4c00173640 1 --2- 172.21.15.117:0/3337923099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bf8076080 0x7f4bf819cce0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4bec02a930 tx=0x7f4bec004270 comp rx=0 tx=0).stop 2024-09-21T05:56:35.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.874+0000 7f4c00173640 1 -- 172.21.15.117:0/3337923099 shutdown_connections 2024-09-21T05:56:35.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.874+0000 7f4c00173640 1 --2- 172.21.15.117:0/3337923099 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4be00b66a0 0x7f4be00b8bb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.874+0000 7f4c00173640 1 --2- 172.21.15.117:0/3337923099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bf8076080 0x7f4bf819cce0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.874+0000 7f4c00173640 1 --2- 172.21.15.117:0/3337923099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bf81101a0 0x7f4bf81a1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.874+0000 7f4c00173640 1 --2- 172.21.15.117:0/3337923099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bf8075730 0x7f4bf819c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:35.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.874+0000 7f4c00173640 1 -- 172.21.15.117:0/3337923099 >> 172.21.15.117:0/3337923099 conn(0x7f4bf807b910 msgr2=0x7f4bf8108c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:35.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.874+0000 7f4c00173640 1 -- 172.21.15.117:0/3337923099 shutdown_connections 2024-09-21T05:56:35.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:35.874+0000 7f4c00173640 1 -- 172.21.15.117:0/3337923099 wait complete. 2024-09-21T05:56:35.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:35 smithi192 ceph-mon[34700]: pgmap v305: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T05:56:36.552 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T05:56:36.798 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:37.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.326+0000 7fb684671640 1 -- 172.21.15.117:0/3297388637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb67c104860 msgr2=0x7fb67c104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:37.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.326+0000 7fb684671640 1 --2- 172.21.15.117:0/3297388637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb67c104860 0x7fb67c104c70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb66c009ca0 tx=0x7fb66c02a8e0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.326+0000 7fb684671640 1 -- 172.21.15.117:0/3297388637 shutdown_connections 2024-09-21T05:56:37.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.326+0000 7fb684671640 1 --2- 172.21.15.117:0/3297388637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb67c10de10 0x7fb67c1102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.326+0000 7fb684671640 1 --2- 172.21.15.117:0/3297388637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb67c105290 0x7fb67c10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.326+0000 7fb684671640 1 --2- 172.21.15.117:0/3297388637 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb67c104860 0x7fb67c104c70 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.326+0000 7fb684671640 1 -- 172.21.15.117:0/3297388637 >> 172.21.15.117:0/3297388637 conn(0x7fb67c079730 msgr2=0x7fb67c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:37.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.326+0000 7fb684671640 1 -- 172.21.15.117:0/3297388637 shutdown_connections 2024-09-21T05:56:37.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.326+0000 7fb684671640 1 -- 172.21.15.117:0/3297388637 wait complete. 2024-09-21T05:56:37.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.326+0000 7fb684671640 1 Processor -- start 2024-09-21T05:56:37.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.327+0000 7fb684671640 1 -- start start 2024-09-21T05:56:37.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.327+0000 7fb684671640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb67c104860 0x7fb67c102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:37.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.327+0000 7fb684671640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb67c105290 0x7fb67c102cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:37.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.327+0000 7fb684671640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb67c10de10 0x7fb67c1031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:37.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.327+0000 7fb684671640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb67c113ac0 con 0x7fb67c10de10 2024-09-21T05:56:37.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.327+0000 7fb684671640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb67c113940 con 0x7fb67c105290 2024-09-21T05:56:37.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.327+0000 7fb684671640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb67c113c40 con 0x7fb67c104860 2024-09-21T05:56:37.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.327+0000 7fb682c0e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb67c10de10 0x7fb67c1031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:37.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.327+0000 7fb682c0e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb67c10de10 0x7fb67c1031f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54352/0 (socket says 172.21.15.117:54352) 2024-09-21T05:56:37.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.327+0000 7fb682c0e640 1 -- 172.21.15.117:0/3044667717 learned_addr learned my addr 172.21.15.117:0/3044667717 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:37.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.327+0000 7fb68240d640 1 --2- 172.21.15.117:0/3044667717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb67c104860 0x7fb67c102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:37.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.327+0000 7fb681c0c640 1 --2- 172.21.15.117:0/3044667717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb67c105290 0x7fb67c102cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:37.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.327+0000 7fb682c0e640 1 -- 172.21.15.117:0/3044667717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb67c104860 msgr2=0x7fb67c102770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:37.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.327+0000 7fb682c0e640 1 --2- 172.21.15.117:0/3044667717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb67c104860 0x7fb67c102770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.327+0000 7fb682c0e640 1 -- 172.21.15.117:0/3044667717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb67c105290 msgr2=0x7fb67c102cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:37.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.327+0000 7fb682c0e640 1 --2- 172.21.15.117:0/3044667717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb67c105290 0x7fb67c102cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.327+0000 7fb682c0e640 1 -- 172.21.15.117:0/3044667717 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb67c1a8810 con 0x7fb67c10de10 2024-09-21T05:56:37.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.327+0000 7fb681c0c640 1 --2- 172.21.15.117:0/3044667717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb67c105290 0x7fb67c102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:37.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.327+0000 7fb682c0e640 1 --2- 172.21.15.117:0/3044667717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb67c10de10 0x7fb67c1031f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb67400c970 tx=0x7fb67400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:37.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.328+0000 7fb67b7fe640 1 -- 172.21.15.117:0/3044667717 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb674007bb0 con 0x7fb67c10de10 2024-09-21T05:56:37.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.328+0000 7fb67b7fe640 1 -- 172.21.15.117:0/3044667717 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb674007d50 con 0x7fb67c10de10 2024-09-21T05:56:37.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.328+0000 7fb684671640 1 -- 172.21.15.117:0/3044667717 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb67c1a8b00 con 0x7fb67c10de10 2024-09-21T05:56:37.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.328+0000 7fb67b7fe640 1 -- 172.21.15.117:0/3044667717 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb674004e60 con 0x7fb67c10de10 2024-09-21T05:56:37.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.328+0000 7fb684671640 1 -- 172.21.15.117:0/3044667717 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb67c1a8ff0 con 0x7fb67c10de10 2024-09-21T05:56:37.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.331+0000 7fb68abe5740 1 Processor -- start 2024-09-21T05:56:37.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.331+0000 7fb67b7fe640 1 -- 172.21.15.117:0/3044667717 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7fb674005020 con 0x7fb67c10de10 2024-09-21T05:56:37.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.331+0000 7fb68abe5740 1 -- start start 2024-09-21T05:56:37.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.332+0000 7fb68abe5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f9cb86f60 0x557f9cf2cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:37.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.332+0000 7fb68abe5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f9cb5e0a0 0x557f9cf2d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:37.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.332+0000 7fb68abe5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f9cb66280 0x557f9cf2d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:37.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.332+0000 7fb68abe5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557f9cea4c10 con 0x557f9cb5e0a0 2024-09-21T05:56:37.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.332+0000 7fb68abe5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557f9ce94040 con 0x557f9cb86f60 2024-09-21T05:56:37.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.332+0000 7fb68abe5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557f9ce97e50 con 0x557f9cb66280 2024-09-21T05:56:37.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.332+0000 7fb68240d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f9cb5e0a0 0x557f9cf2d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:37.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.332+0000 7fb68240d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f9cb5e0a0 0x557f9cf2d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54368/0 (socket says 172.21.15.117:54368) 2024-09-21T05:56:37.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.332+0000 7fb68240d640 1 -- 172.21.15.117:0/3647344928 learned_addr learned my addr 172.21.15.117:0/3647344928 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:37.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.332+0000 7fb681c0c640 1 --2- 172.21.15.117:0/3647344928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f9cb86f60 0x557f9cf2cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:37.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.332+0000 7fb681c0c640 1 --2- 172.21.15.117:0/3647344928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f9cb66280 0x557f9cf2d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:37.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.332+0000 7fb68240d640 1 -- 172.21.15.117:0/3647344928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f9cb66280 msgr2=0x557f9cf2d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:37.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.332+0000 7fb68240d640 1 --2- 172.21.15.117:0/3647344928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f9cb66280 0x557f9cf2d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.332+0000 7fb68240d640 1 -- 172.21.15.117:0/3647344928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f9cb86f60 msgr2=0x557f9cf2cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:37.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.332+0000 7fb68240d640 1 --2- 172.21.15.117:0/3647344928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f9cb86f60 0x557f9cf2cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.332+0000 7fb68240d640 1 -- 172.21.15.117:0/3647344928 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f9cbf2540 con 0x557f9cb5e0a0 2024-09-21T05:56:37.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.332+0000 7fb67b7fe640 1 --2- 172.21.15.117:0/3044667717 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6640b65f0 0x7fb6640b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:37.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.332+0000 7fb68240d640 1 --2- 172.21.15.117:0/3647344928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f9cb5e0a0 0x557f9cf2d440 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb66c009ca0 tx=0x7fb66c0317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f4d8f64dcff5ad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:37.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.332+0000 7fb681c0c640 1 --2- 172.21.15.117:0/3044667717 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6640b65f0 0x7fb6640b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:37.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.332+0000 7fb67b7fe640 1 -- 172.21.15.117:0/3044667717 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fb6740e7260 con 0x7fb67c10de10 2024-09-21T05:56:37.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.332+0000 7fb678ff9640 1 -- 172.21.15.117:0/3647344928 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66c031ad0 con 0x557f9cb5e0a0 2024-09-21T05:56:37.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.333+0000 7fb678ff9640 1 -- 172.21.15.117:0/3647344928 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb66c036030 con 0x557f9cb5e0a0 2024-09-21T05:56:37.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.333+0000 7fb678ff9640 1 -- 172.21.15.117:0/3647344928 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66c0045d0 con 0x557f9cb5e0a0 2024-09-21T05:56:37.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.333+0000 7fb68abe5740 1 -- 172.21.15.117:0/3647344928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f9cb5e0a0 msgr2=0x557f9cf2d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:37.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.334+0000 7fb68abe5740 1 --2- 172.21.15.117:0/3647344928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f9cb5e0a0 0x557f9cf2d440 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb66c009ca0 tx=0x7fb66c0317b0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.333+0000 7fb681c0c640 1 --2- 172.21.15.117:0/3044667717 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6640b65f0 0x7fb6640b8b00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb67c102450 tx=0x7fb67000c190 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:37.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.334+0000 7fb68abe5740 1 -- 172.21.15.117:0/3647344928 shutdown_connections 2024-09-21T05:56:37.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.334+0000 7fb68abe5740 1 --2- 172.21.15.117:0/3647344928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f9cb5e0a0 0x557f9cf2d440 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.334+0000 7fb68abe5740 1 --2- 172.21.15.117:0/3647344928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f9cb66280 0x557f9cf2d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.334+0000 7fb68abe5740 1 --2- 172.21.15.117:0/3647344928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f9cb86f60 0x557f9cf2cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.334+0000 7fb68abe5740 1 -- 172.21.15.117:0/3647344928 >> 172.21.15.117:0/3647344928 conn(0x557f9cb7b350 msgr2=0x557f9cf2adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:37.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.334+0000 7fb68abe5740 1 -- 172.21.15.117:0/3647344928 shutdown_connections 2024-09-21T05:56:37.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.334+0000 7fb68abe5740 1 -- 172.21.15.117:0/3647344928 wait complete. 2024-09-21T05:56:37.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.334+0000 7fb68abe5740 1 -- start start 2024-09-21T05:56:37.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.334+0000 7fb68abe5740 1 Processor -- start 2024-09-21T05:56:37.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.334+0000 7fb68abe5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f9cb5e0a0 0x557f9cf389e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:37.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.335+0000 7fb68abe5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f9cb66280 0x557f9cf38f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:37.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.335+0000 7fb68abe5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f9cb86f60 0x557f9cf449a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:37.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.335+0000 7fb68abe5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557f9ccf55f0 con 0x557f9cb5e0a0 2024-09-21T05:56:37.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.335+0000 7fb68abe5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557f9ce2f010 con 0x557f9cb86f60 2024-09-21T05:56:37.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.335+0000 7fb68abe5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557f9cf3b680 con 0x557f9cb66280 2024-09-21T05:56:37.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.335+0000 7fb68abe5740 10 client.0 _ll_register_callbacks cb 0x557f9ce8e340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:56:37.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.335+0000 7fb68240d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f9cb5e0a0 0x557f9cf389e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:37.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.335+0000 7fb68240d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f9cb5e0a0 0x557f9cf389e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54372/0 (socket says 172.21.15.117:54372) 2024-09-21T05:56:37.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.335+0000 7fb68240d640 1 -- 172.21.15.117:0/833599712 learned_addr learned my addr 172.21.15.117:0/833599712 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:37.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.335+0000 7fb682c0e640 1 --2- 172.21.15.117:0/833599712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f9cb86f60 0x557f9cf449a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:37.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.335+0000 7fb681c0c640 1 --2- 172.21.15.117:0/833599712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f9cb66280 0x557f9cf38f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:37.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.335+0000 7fb68240d640 1 -- 172.21.15.117:0/833599712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f9cb66280 msgr2=0x557f9cf38f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:37.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.335+0000 7fb68240d640 1 --2- 172.21.15.117:0/833599712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f9cb66280 0x557f9cf38f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.335+0000 7fb68240d640 1 -- 172.21.15.117:0/833599712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f9cb86f60 msgr2=0x557f9cf449a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:37.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.335+0000 7fb68240d640 1 --2- 172.21.15.117:0/833599712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f9cb86f60 0x557f9cf449a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.335+0000 7fb68240d640 1 -- 172.21.15.117:0/833599712 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f9ce5ca00 con 0x557f9cb5e0a0 2024-09-21T05:56:37.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.335+0000 7fb68240d640 1 --2- 172.21.15.117:0/833599712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f9cb5e0a0 0x557f9cf389e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb66c0045a0 tx=0x7fb66c0045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e58179121864f2c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:37.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.335+0000 7fb661ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:56:37.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.335+0000 7fb661ffb640 1 -- 172.21.15.117:0/833599712 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66c049040 con 0x557f9cb5e0a0 2024-09-21T05:56:37.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.335+0000 7fb661ffb640 1 -- 172.21.15.117:0/833599712 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fb66c0047f0 con 0x557f9cb5e0a0 2024-09-21T05:56:37.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.335+0000 7fb68abe5740 1 -- 172.21.15.117:0/833599712 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb66c036030 con 0x557f9cb5e0a0 2024-09-21T05:56:37.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.336+0000 7fb661ffb640 1 -- 172.21.15.117:0/833599712 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb66c031c90 con 0x557f9cb5e0a0 2024-09-21T05:56:37.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.336+0000 7fb68abe5740 10 client.15288 fetch_fsmap learned FSMap version 26 2024-09-21T05:56:37.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.336+0000 7fb68abe5740 1 -- 172.21.15.117:0/833599712 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557f9cf47580 con 0x557f9cb5e0a0 2024-09-21T05:56:37.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.336+0000 7fb661ffb640 1 -- 172.21.15.117:0/833599712 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb66c036030 con 0x557f9cb5e0a0 2024-09-21T05:56:37.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.336+0000 7fb661ffb640 1 -- 172.21.15.117:0/833599712 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fb66c04c030 con 0x557f9cb5e0a0 2024-09-21T05:56:37.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.336+0000 7fb68abe5740 10 client.15288 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T05:56:37.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.336+0000 7fb68abe5740 10 client.15288 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T05:56:37.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.336+0000 7fb68abe5740 20 client.15288 populate_metadata read hostname 'smithi117' 2024-09-21T05:56:37.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.336+0000 7fb68abe5740 1 --2- 172.21.15.117:0/833599712 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x557f9cb458e0 0x557f9cf4aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:37.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.336+0000 7fb68abe5740 4 client.15288 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:56:37.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.336+0000 7fb68abe5740 1 -- 172.21.15.117:0/833599712 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f9cea6b60 con 0x557f9cb458e0 2024-09-21T05:56:37.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.336+0000 7fb681c0c640 1 --2- 172.21.15.117:0/833599712 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x557f9cb458e0 0x557f9cf4aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:37.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.337+0000 7fb661ffb640 10 client.15288 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:56:37.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.337+0000 7fb681c0c640 1 --2- 172.21.15.117:0/833599712 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x557f9cb458e0 0x557f9cf4aaf0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=345feb9ab94b24d4 server_cookie=56da47c54dd42f47 in_seq=0 out_seq=0 2024-09-21T05:56:37.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.338+0000 7fb661ffb640 1 -- 172.21.15.117:0/833599712 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557f9cea6b60 con 0x557f9cb458e0 2024-09-21T05:56:37.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.338+0000 7fb661ffb640 10 client.15288 handle_command_reply: tid=0 2024-09-21T05:56:37.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.338+0000 7fb68abe5740 1 client.15288 shutdown 2024-09-21T05:56:37.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.338+0000 7fb68abe5740 1 -- 172.21.15.117:0/833599712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f9cb5e0a0 msgr2=0x557f9cf389e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:37.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.338+0000 7fb68abe5740 1 --2- 172.21.15.117:0/833599712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f9cb5e0a0 0x557f9cf389e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb66c0045a0 tx=0x7fb66c0045d0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.338+0000 7fb68abe5740 1 -- 172.21.15.117:0/833599712 shutdown_connections 2024-09-21T05:56:37.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.338+0000 7fb68abe5740 1 --2- 172.21.15.117:0/833599712 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x557f9cb458e0 0x557f9cf4aaf0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.338+0000 7fb68abe5740 1 --2- 172.21.15.117:0/833599712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f9cb86f60 0x557f9cf449a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.338+0000 7fb68abe5740 1 --2- 172.21.15.117:0/833599712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f9cb66280 0x557f9cf38f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.338+0000 7fb68abe5740 1 --2- 172.21.15.117:0/833599712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f9cb5e0a0 0x557f9cf389e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.338+0000 7fb661ffb640 0 client.15288 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:56:37.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.338+0000 7fb68abe5740 1 -- 172.21.15.117:0/833599712 >> 172.21.15.117:0/833599712 conn(0x557f9cb7b350 msgr2=0x557f9cf30090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:37.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.338+0000 7fb68abe5740 1 -- 172.21.15.117:0/833599712 shutdown_connections 2024-09-21T05:56:37.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.338+0000 7fb68abe5740 1 -- 172.21.15.117:0/833599712 wait complete. 2024-09-21T05:56:37.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.338+0000 7fb68abe5740 20 client.15288 trim_cache size 0 max 16384 2024-09-21T05:56:37.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.356+0000 7fb68abe5740 1 Processor -- start 2024-09-21T05:56:37.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.356+0000 7fb68abe5740 1 -- start start 2024-09-21T05:56:37.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.356+0000 7fb68abe5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f9cb5e0a0 0x557f9cf44910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:37.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.356+0000 7fb68abe5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f9cb66280 0x557f9cf44e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:37.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.356+0000 7fb68abe5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f9cb86f60 0x557f9cf2a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:37.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.356+0000 7fb68abe5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557f9cf48e30 con 0x557f9cb86f60 2024-09-21T05:56:37.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.356+0000 7fb68abe5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557f9cf49500 con 0x557f9cb5e0a0 2024-09-21T05:56:37.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.356+0000 7fb68abe5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557f9cf49bd0 con 0x557f9cb66280 2024-09-21T05:56:37.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.356+0000 7fb682c0e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f9cb86f60 0x557f9cf2a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:37.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.356+0000 7fb682c0e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f9cb86f60 0x557f9cf2a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54378/0 (socket says 172.21.15.117:54378) 2024-09-21T05:56:37.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.356+0000 7fb682c0e640 1 -- 172.21.15.117:0/406776988 learned_addr learned my addr 172.21.15.117:0/406776988 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:37.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.356+0000 7fb68240d640 1 --2- 172.21.15.117:0/406776988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f9cb5e0a0 0x557f9cf44910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:37.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.357+0000 7fb682c0e640 1 -- 172.21.15.117:0/406776988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f9cb66280 msgr2=0x557f9cf44e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:37.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.357+0000 7fb682c0e640 1 --2- 172.21.15.117:0/406776988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f9cb66280 0x557f9cf44e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.357+0000 7fb682c0e640 1 -- 172.21.15.117:0/406776988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f9cb5e0a0 msgr2=0x557f9cf44910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:37.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.357+0000 7fb682c0e640 1 --2- 172.21.15.117:0/406776988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f9cb5e0a0 0x557f9cf44910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.357+0000 7fb682c0e640 1 -- 172.21.15.117:0/406776988 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f9cf25080 con 0x557f9cb86f60 2024-09-21T05:56:37.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.357+0000 7fb68240d640 1 --2- 172.21.15.117:0/406776988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f9cb5e0a0 0x557f9cf44910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:56:37.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.357+0000 7fb682c0e640 1 --2- 172.21.15.117:0/406776988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f9cb86f60 0x557f9cf2a730 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb674007590 tx=0x7fb6740ad0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e436f27895f97c8f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:37.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.357+0000 7fb663fff640 1 -- 172.21.15.117:0/406776988 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67401d040 con 0x557f9cb86f60 2024-09-21T05:56:37.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.357+0000 7fb663fff640 1 -- 172.21.15.117:0/406776988 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6740e6070 con 0x557f9cb86f60 2024-09-21T05:56:37.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.357+0000 7fb663fff640 1 -- 172.21.15.117:0/406776988 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6740a8da0 con 0x557f9cb86f60 2024-09-21T05:56:37.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.358+0000 7fb68abe5740 1 -- 172.21.15.117:0/406776988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f9cb86f60 msgr2=0x557f9cf2a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:37.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.358+0000 7fb68abe5740 1 --2- 172.21.15.117:0/406776988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f9cb86f60 0x557f9cf2a730 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb674007590 tx=0x7fb6740ad0f0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.358+0000 7fb68abe5740 1 -- 172.21.15.117:0/406776988 shutdown_connections 2024-09-21T05:56:37.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.358+0000 7fb68abe5740 1 --2- 172.21.15.117:0/406776988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f9cb86f60 0x557f9cf2a730 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.358+0000 7fb68abe5740 1 --2- 172.21.15.117:0/406776988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f9cb66280 0x557f9cf44e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.358+0000 7fb68abe5740 1 --2- 172.21.15.117:0/406776988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f9cb5e0a0 0x557f9cf44910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.358+0000 7fb68abe5740 1 -- 172.21.15.117:0/406776988 >> 172.21.15.117:0/406776988 conn(0x557f9cb7b350 msgr2=0x557f9cf389c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:37.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.358+0000 7fb68abe5740 1 -- 172.21.15.117:0/406776988 shutdown_connections 2024-09-21T05:56:37.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.358+0000 7fb68abe5740 1 -- 172.21.15.117:0/406776988 wait complete. 2024-09-21T05:56:37.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.358+0000 7fb68abe5740 1 -- start start 2024-09-21T05:56:37.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.358+0000 7fb68abe5740 1 Processor -- start 2024-09-21T05:56:37.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.359+0000 7fb68abe5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f9cb5e0a0 0x557f9cf537d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:37.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.359+0000 7fb68abe5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f9cb66280 0x557f9cf4cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:37.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.359+0000 7fb68abe5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f9cb86f60 0x557f9cf4d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:37.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.359+0000 7fb68abe5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557f9cf4a970 con 0x557f9cb66280 2024-09-21T05:56:37.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.359+0000 7fb68abe5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557f9cf4a2a0 con 0x557f9cb5e0a0 2024-09-21T05:56:37.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.359+0000 7fb68abe5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557f9cf4b040 con 0x557f9cb86f60 2024-09-21T05:56:37.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.359+0000 7fb68abe5740 10 client.0 _ll_register_callbacks cb 0x7fb67000c750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:56:37.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.359+0000 7fb681c0c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f9cb66280 0x557f9cf4cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:37.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.359+0000 7fb681c0c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f9cb66280 0x557f9cf4cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54382/0 (socket says 172.21.15.117:54382) 2024-09-21T05:56:37.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.359+0000 7fb681c0c640 1 -- 172.21.15.117:0/1219887807 learned_addr learned my addr 172.21.15.117:0/1219887807 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:37.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.359+0000 7fb68240d640 1 --2- 172.21.15.117:0/1219887807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f9cb5e0a0 0x557f9cf537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:37.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.359+0000 7fb682c0e640 1 --2- 172.21.15.117:0/1219887807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f9cb86f60 0x557f9cf4d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:37.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.359+0000 7fb681c0c640 1 -- 172.21.15.117:0/1219887807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f9cb86f60 msgr2=0x557f9cf4d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:37.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.359+0000 7fb681c0c640 1 --2- 172.21.15.117:0/1219887807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f9cb86f60 0x557f9cf4d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.359+0000 7fb681c0c640 1 -- 172.21.15.117:0/1219887807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f9cb5e0a0 msgr2=0x557f9cf537d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:37.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.359+0000 7fb681c0c640 1 --2- 172.21.15.117:0/1219887807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f9cb5e0a0 0x557f9cf537d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.359+0000 7fb681c0c640 1 -- 172.21.15.117:0/1219887807 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f9cf52410 con 0x557f9cb66280 2024-09-21T05:56:37.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.359+0000 7fb682c0e640 1 --2- 172.21.15.117:0/1219887807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f9cb86f60 0x557f9cf4d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:56:37.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.359+0000 7fb681c0c640 1 --2- 172.21.15.117:0/1219887807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f9cb66280 0x557f9cf4cd00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x557f9cf4b440 tx=0x7fb67000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab2b36ae9796408 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:37.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.359+0000 7fb661ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:56:37.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.359+0000 7fb661ffb640 1 -- 172.21.15.117:0/1219887807 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb670033060 con 0x557f9cb66280 2024-09-21T05:56:37.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.359+0000 7fb661ffb640 1 -- 172.21.15.117:0/1219887807 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fb67002e480 con 0x557f9cb66280 2024-09-21T05:56:37.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.360+0000 7fb68abe5740 1 -- 172.21.15.117:0/1219887807 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6740e6070 con 0x557f9cb66280 2024-09-21T05:56:37.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.360+0000 7fb661ffb640 1 -- 172.21.15.117:0/1219887807 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb67000f040 con 0x557f9cb66280 2024-09-21T05:56:37.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.360+0000 7fb68abe5740 10 client.15294 fetch_fsmap learned FSMap version 26 2024-09-21T05:56:37.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.360+0000 7fb68abe5740 1 -- 172.21.15.117:0/1219887807 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557f9cfb6210 con 0x557f9cb66280 2024-09-21T05:56:37.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.360+0000 7fb661ffb640 1 -- 172.21.15.117:0/1219887807 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6740e6070 con 0x557f9cb66280 2024-09-21T05:56:37.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.360+0000 7fb661ffb640 1 -- 172.21.15.117:0/1219887807 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fb670015040 con 0x557f9cb66280 2024-09-21T05:56:37.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.360+0000 7fb68abe5740 10 client.15294 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T05:56:37.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.360+0000 7fb68abe5740 10 client.15294 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T05:56:37.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.360+0000 7fb68abe5740 20 client.15294 populate_metadata read hostname 'smithi117' 2024-09-21T05:56:37.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.360+0000 7fb68abe5740 1 --2- 172.21.15.117:0/1219887807 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x557f9cb1ac90 0x557f9cfb9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:37.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.360+0000 7fb68abe5740 4 client.15294 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T05:56:37.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.360+0000 7fb68abe5740 1 -- 172.21.15.117:0/1219887807 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "status"}) -- 0x557f9cea2610 con 0x557f9cb1ac90 2024-09-21T05:56:37.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.360+0000 7fb68240d640 1 --2- 172.21.15.117:0/1219887807 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x557f9cb1ac90 0x557f9cfb9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:37.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.361+0000 7fb661ffb640 10 client.15294 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:56:37.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.361+0000 7fb68240d640 1 --2- 172.21.15.117:0/1219887807 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x557f9cb1ac90 0x557f9cfb9a90 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d68b21b4367a883c server_cookie=4d9d4202683db26f in_seq=0 out_seq=0 2024-09-21T05:56:37.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.361+0000 7fb661ffb640 1 -- 172.21.15.117:0/1219887807 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x557f9cea2610 con 0x557f9cb1ac90 2024-09-21T05:56:37.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.361+0000 7fb661ffb640 10 client.15294 handle_command_reply: tid=0 2024-09-21T05:56:37.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.361+0000 7fb68abe5740 1 client.15294 shutdown 2024-09-21T05:56:37.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.361+0000 7fb68abe5740 1 -- 172.21.15.117:0/1219887807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f9cb66280 msgr2=0x557f9cf4cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:37.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.361+0000 7fb68abe5740 1 --2- 172.21.15.117:0/1219887807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f9cb66280 0x557f9cf4cd00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x557f9cf4b440 tx=0x7fb67000c8b0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.361+0000 7fb68abe5740 1 -- 172.21.15.117:0/1219887807 shutdown_connections 2024-09-21T05:56:37.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.361+0000 7fb68abe5740 1 --2- 172.21.15.117:0/1219887807 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x557f9cb1ac90 0x557f9cfb9a90 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.361+0000 7fb68abe5740 1 --2- 172.21.15.117:0/1219887807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f9cb66280 0x557f9cf4cd00 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.361+0000 7fb68abe5740 1 --2- 172.21.15.117:0/1219887807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f9cb86f60 0x557f9cf4d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.361+0000 7fb68abe5740 1 --2- 172.21.15.117:0/1219887807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f9cb5e0a0 0x557f9cf537d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.361+0000 7fb68abe5740 1 -- 172.21.15.117:0/1219887807 >> 172.21.15.117:0/1219887807 conn(0x557f9cb7b350 msgr2=0x557f9cf2a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:37.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.361+0000 7fb661ffb640 0 client.15294 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:56:37.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.361+0000 7fb68abe5740 1 -- 172.21.15.117:0/1219887807 shutdown_connections 2024-09-21T05:56:37.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.361+0000 7fb68abe5740 1 -- 172.21.15.117:0/1219887807 wait complete. 2024-09-21T05:56:37.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.361+0000 7fb68abe5740 20 client.15294 trim_cache size 0 max 16384 2024-09-21T05:56:37.368 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T05:56:37.369 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_fsid": "ac8581a2-77dc-11ef-baf5-efdc52797490", 2024-09-21T05:56:37.369 INFO:teuthology.orchestra.run.smithi117.stdout: "whoami": 0, 2024-09-21T05:56:37.369 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 24638, 2024-09-21T05:56:37.369 INFO:teuthology.orchestra.run.smithi117.stdout: "want_state": "up:active", 2024-09-21T05:56:37.369 INFO:teuthology.orchestra.run.smithi117.stdout: "state": "up:active", 2024-09-21T05:56:37.369 INFO:teuthology.orchestra.run.smithi117.stdout: "fs_name": "cephfs", 2024-09-21T05:56:37.369 INFO:teuthology.orchestra.run.smithi117.stdout: "rank_uptime": 82.376854707999996, 2024-09-21T05:56:37.369 INFO:teuthology.orchestra.run.smithi117.stdout: "mdsmap_epoch": 26, 2024-09-21T05:56:37.369 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch": 76, 2024-09-21T05:56:37.369 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch_barrier": 74, 2024-09-21T05:56:37.369 INFO:teuthology.orchestra.run.smithi117.stdout: "uptime": 93.247851741999995 2024-09-21T05:56:37.369 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T05:56:37.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.362+0000 7fb684671640 1 -- 172.21.15.117:0/3044667717 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6640b65f0 msgr2=0x7fb6640b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:37.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.362+0000 7fb684671640 1 --2- 172.21.15.117:0/3044667717 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6640b65f0 0x7fb6640b8b00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb67c102450 tx=0x7fb67000c190 comp rx=0 tx=0).stop 2024-09-21T05:56:37.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.362+0000 7fb684671640 1 -- 172.21.15.117:0/3044667717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb67c10de10 msgr2=0x7fb67c1031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:37.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.362+0000 7fb684671640 1 --2- 172.21.15.117:0/3044667717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb67c10de10 0x7fb67c1031f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb67400c970 tx=0x7fb67400ce30 comp rx=0 tx=0).stop 2024-09-21T05:56:37.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.362+0000 7fb684671640 1 -- 172.21.15.117:0/3044667717 shutdown_connections 2024-09-21T05:56:37.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.362+0000 7fb684671640 1 --2- 172.21.15.117:0/3044667717 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6640b65f0 0x7fb6640b8b00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.362+0000 7fb684671640 1 --2- 172.21.15.117:0/3044667717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb67c10de10 0x7fb67c1031f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.362+0000 7fb684671640 1 --2- 172.21.15.117:0/3044667717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb67c105290 0x7fb67c102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.362+0000 7fb684671640 1 --2- 172.21.15.117:0/3044667717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb67c104860 0x7fb67c102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:37.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.362+0000 7fb684671640 1 -- 172.21.15.117:0/3044667717 >> 172.21.15.117:0/3044667717 conn(0x7fb67c079730 msgr2=0x7fb67c1008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:37.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.362+0000 7fb684671640 1 -- 172.21.15.117:0/3044667717 shutdown_connections 2024-09-21T05:56:37.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:37.362+0000 7fb684671640 1 -- 172.21.15.117:0/3044667717 wait complete. 2024-09-21T05:56:37.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:37 smithi138 ceph-mon[34391]: pgmap v306: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T05:56:37.918 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:37 smithi117 ceph-mon[30947]: pgmap v306: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T05:56:37.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:37 smithi192 ceph-mon[34700]: pgmap v306: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T05:56:38.051 INFO:tasks.cephadm:Setup complete, yielding 2024-09-21T05:56:38.051 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T05:56:38.064 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi117.front.sepia.ceph.com 2024-09-21T05:56:38.065 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- bash -c 'ceph orch status' 2024-09-21T05:56:38.310 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:38.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.816+0000 7f9fcfd27640 1 -- 172.21.15.117:0/1661269329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fc8105b10 msgr2=0x7f9fc810f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:38.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.816+0000 7f9fcfd27640 1 --2- 172.21.15.117:0/1661269329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fc8105b10 0x7f9fc810f5a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0009ca0 tx=0x7f9fc002a8c0 comp rx=0 tx=0).stop 2024-09-21T05:56:38.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.816+0000 7f9fcfd27640 1 -- 172.21.15.117:0/1661269329 shutdown_connections 2024-09-21T05:56:38.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.816+0000 7f9fcfd27640 1 --2- 172.21.15.117:0/1661269329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fc8105b10 0x7f9fc810f5a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:38.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.816+0000 7f9fcfd27640 1 --2- 172.21.15.117:0/1661269329 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fc81051c0 0x7f9fc81055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:38.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.816+0000 7f9fcfd27640 1 --2- 172.21.15.117:0/1661269329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fc81031b0 0x7f9fc81035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:38.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.816+0000 7f9fcfd27640 1 -- 172.21.15.117:0/1661269329 >> 172.21.15.117:0/1661269329 conn(0x7f9fc8079e40 msgr2=0x7f9fc807a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:38.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.816+0000 7f9fcfd27640 1 -- 172.21.15.117:0/1661269329 shutdown_connections 2024-09-21T05:56:38.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.816+0000 7f9fcfd27640 1 -- 172.21.15.117:0/1661269329 wait complete. 2024-09-21T05:56:38.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.817+0000 7f9fcfd27640 1 Processor -- start 2024-09-21T05:56:38.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.817+0000 7f9fcfd27640 1 -- start start 2024-09-21T05:56:38.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.817+0000 7f9fcfd27640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fc81031b0 0x7f9fc819b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:38.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.817+0000 7f9fcfd27640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fc81051c0 0x7f9fc819b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:38.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.817+0000 7f9fcfd27640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fc8105b10 0x7f9fc819bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:38.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.817+0000 7f9fcfd27640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc81129e0 con 0x7f9fc8105b10 2024-09-21T05:56:38.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.817+0000 7f9fcfd27640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc8112860 con 0x7f9fc81031b0 2024-09-21T05:56:38.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.817+0000 7f9fcfd27640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc8112b60 con 0x7f9fc81051c0 2024-09-21T05:56:38.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.818+0000 7f9fce2c4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fc8105b10 0x7f9fc819bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:38.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.818+0000 7f9fce2c4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fc8105b10 0x7f9fc819bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54406/0 (socket says 172.21.15.117:54406) 2024-09-21T05:56:38.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.818+0000 7f9fce2c4640 1 -- 172.21.15.117:0/2224188412 learned_addr learned my addr 172.21.15.117:0/2224188412 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:38.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.818+0000 7f9fcdac3640 1 --2- 172.21.15.117:0/2224188412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fc81031b0 0x7f9fc819b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:38.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.818+0000 7f9fcd2c2640 1 --2- 172.21.15.117:0/2224188412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fc81051c0 0x7f9fc819b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:38.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.818+0000 7f9fce2c4640 1 -- 172.21.15.117:0/2224188412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fc81051c0 msgr2=0x7f9fc819b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:38.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.818+0000 7f9fce2c4640 1 --2- 172.21.15.117:0/2224188412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fc81051c0 0x7f9fc819b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:38.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.818+0000 7f9fce2c4640 1 -- 172.21.15.117:0/2224188412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fc81031b0 msgr2=0x7f9fc819b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:38.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.818+0000 7f9fce2c4640 1 --2- 172.21.15.117:0/2224188412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fc81031b0 0x7f9fc819b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:38.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.818+0000 7f9fce2c4640 1 -- 172.21.15.117:0/2224188412 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc81a6530 con 0x7f9fc8105b10 2024-09-21T05:56:38.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.818+0000 7f9fce2c4640 1 --2- 172.21.15.117:0/2224188412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fc8105b10 0x7f9fc819bdb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0005cc0 tx=0x7f9fc0031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:38.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.818+0000 7f9fbeffd640 1 -- 172.21.15.117:0/2224188412 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0003b70 con 0x7f9fc8105b10 2024-09-21T05:56:38.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.818+0000 7f9fcfd27640 1 -- 172.21.15.117:0/2224188412 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fc81a67c0 con 0x7f9fc8105b10 2024-09-21T05:56:38.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.818+0000 7f9fbeffd640 1 -- 172.21.15.117:0/2224188412 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fc0003d10 con 0x7f9fc8105b10 2024-09-21T05:56:38.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.818+0000 7f9fbeffd640 1 -- 172.21.15.117:0/2224188412 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc002fad0 con 0x7f9fc8105b10 2024-09-21T05:56:38.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.818+0000 7f9fcfd27640 1 -- 172.21.15.117:0/2224188412 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fc81a6d00 con 0x7f9fc8105b10 2024-09-21T05:56:38.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.819+0000 7f9fcfd27640 1 -- 172.21.15.117:0/2224188412 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fc81035c0 con 0x7f9fc8105b10 2024-09-21T05:56:38.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.822+0000 7f9fbeffd640 1 -- 172.21.15.117:0/2224188412 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f9fc002fc70 con 0x7f9fc8105b10 2024-09-21T05:56:38.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.822+0000 7f9fbeffd640 1 --2- 172.21.15.117:0/2224188412 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9fb40b6620 0x7f9fb40b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:38.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.823+0000 7f9fcdac3640 1 --2- 172.21.15.117:0/2224188412 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9fb40b6620 0x7f9fb40b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:38.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.823+0000 7f9fbeffd640 1 -- 172.21.15.117:0/2224188412 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f9fc0108a40 con 0x7f9fc8105b10 2024-09-21T05:56:38.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.823+0000 7f9fcdac3640 1 --2- 172.21.15.117:0/2224188412 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9fb40b6620 0x7f9fb40b8b30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8102970 tx=0x7f9fc400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:38.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:38.826+0000 7f9fbeffd640 1 -- 172.21.15.117:0/2224188412 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9fc00cf070 con 0x7f9fc8105b10 2024-09-21T05:56:39.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:39.020+0000 7f9fcfd27640 1 -- 172.21.15.117:0/2224188412 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f9fc810ac40 con 0x7f9fb40b6620 2024-09-21T05:56:39.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:39.021+0000 7f9fbeffd640 1 -- 172.21.15.117:0/2224188412 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f9fc4010bb0 con 0x7f9fb40b6620 2024-09-21T05:56:39.023 INFO:teuthology.orchestra.run.smithi117.stdout:Backend: cephadm 2024-09-21T05:56:39.023 INFO:teuthology.orchestra.run.smithi117.stdout:Available: Yes 2024-09-21T05:56:39.023 INFO:teuthology.orchestra.run.smithi117.stdout:Paused: No 2024-09-21T05:56:39.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:39.024+0000 7f9fcfd27640 1 -- 172.21.15.117:0/2224188412 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9fb40b6620 msgr2=0x7f9fb40b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:39.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:39.024+0000 7f9fcfd27640 1 --2- 172.21.15.117:0/2224188412 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9fb40b6620 0x7f9fb40b8b30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8102970 tx=0x7f9fc400b390 comp rx=0 tx=0).stop 2024-09-21T05:56:39.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:39.024+0000 7f9fcfd27640 1 -- 172.21.15.117:0/2224188412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fc8105b10 msgr2=0x7f9fc819bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:39.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:39.024+0000 7f9fcfd27640 1 --2- 172.21.15.117:0/2224188412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fc8105b10 0x7f9fc819bdb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0005cc0 tx=0x7f9fc0031840 comp rx=0 tx=0).stop 2024-09-21T05:56:39.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:39.024+0000 7f9fcfd27640 1 -- 172.21.15.117:0/2224188412 shutdown_connections 2024-09-21T05:56:39.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:39.024+0000 7f9fcfd27640 1 --2- 172.21.15.117:0/2224188412 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9fb40b6620 0x7f9fb40b8b30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:39.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:39.024+0000 7f9fcfd27640 1 --2- 172.21.15.117:0/2224188412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fc8105b10 0x7f9fc819bdb0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:39.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:39.024+0000 7f9fcfd27640 1 --2- 172.21.15.117:0/2224188412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fc81051c0 0x7f9fc819b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:39.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:39.024+0000 7f9fcfd27640 1 --2- 172.21.15.117:0/2224188412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fc81031b0 0x7f9fc819b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:39.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:39.024+0000 7f9fcfd27640 1 -- 172.21.15.117:0/2224188412 >> 172.21.15.117:0/2224188412 conn(0x7f9fc8079e40 msgr2=0x7f9fc81063f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:39.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:39.024+0000 7f9fcfd27640 1 -- 172.21.15.117:0/2224188412 shutdown_connections 2024-09-21T05:56:39.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:39.024+0000 7f9fcfd27640 1 -- 172.21.15.117:0/2224188412 wait complete. 2024-09-21T05:56:39.649 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:39 smithi117 ceph-mon[30947]: pgmap v307: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T05:56:39.678 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ps' 2024-09-21T05:56:39.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:39 smithi138 ceph-mon[34391]: pgmap v307: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T05:56:39.925 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:39.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:39 smithi192 ceph-mon[34700]: pgmap v307: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T05:56:40.413 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.409+0000 7fae9bfdb640 1 -- 172.21.15.117:0/2746084685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae94105b90 msgr2=0x7fae9410f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.413 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.409+0000 7fae9bfdb640 1 --2- 172.21.15.117:0/2746084685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae94105b90 0x7fae9410f5e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fae8c009900 tx=0x7fae8c02a410 comp rx=0 tx=0).stop 2024-09-21T05:56:40.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.409+0000 7fae9bfdb640 1 -- 172.21.15.117:0/2746084685 shutdown_connections 2024-09-21T05:56:40.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.409+0000 7fae9bfdb640 1 --2- 172.21.15.117:0/2746084685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae94105b90 0x7fae9410f5e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.409+0000 7fae9bfdb640 1 --2- 172.21.15.117:0/2746084685 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae94105240 0x7fae94105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.409+0000 7fae9bfdb640 1 --2- 172.21.15.117:0/2746084685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae94103270 0x7fae94103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.409+0000 7fae9bfdb640 1 -- 172.21.15.117:0/2746084685 >> 172.21.15.117:0/2746084685 conn(0x7fae94079ec0 msgr2=0x7fae9407a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:40.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.409+0000 7fae9bfdb640 1 -- 172.21.15.117:0/2746084685 shutdown_connections 2024-09-21T05:56:40.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.409+0000 7fae9bfdb640 1 -- 172.21.15.117:0/2746084685 wait complete. 2024-09-21T05:56:40.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.410+0000 7fae9bfdb640 1 Processor -- start 2024-09-21T05:56:40.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.410+0000 7fae9bfdb640 1 -- start start 2024-09-21T05:56:40.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.410+0000 7fae9bfdb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae94103270 0x7fae941990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.410+0000 7fae9bfdb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae94105240 0x7fae941995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.410+0000 7fae9bfdb640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae94105b90 0x7fae94199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.410+0000 7fae9bfdb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fae94112ab0 con 0x7fae94105240 2024-09-21T05:56:40.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.410+0000 7fae9bfdb640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fae94112930 con 0x7fae94103270 2024-09-21T05:56:40.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.410+0000 7fae9bfdb640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fae94112c30 con 0x7fae94105b90 2024-09-21T05:56:40.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.410+0000 7fae99576640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae94105240 0x7fae941995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.410+0000 7fae99576640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae94105240 0x7fae941995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54426/0 (socket says 172.21.15.117:54426) 2024-09-21T05:56:40.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.410+0000 7fae99576640 1 -- 172.21.15.117:0/4294203993 learned_addr learned my addr 172.21.15.117:0/4294203993 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:40.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.410+0000 7fae9a578640 1 --2- 172.21.15.117:0/4294203993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae94105b90 0x7fae94199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.411+0000 7fae99d77640 1 --2- 172.21.15.117:0/4294203993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae94103270 0x7fae941990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.411+0000 7fae99576640 1 -- 172.21.15.117:0/4294203993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae94105b90 msgr2=0x7fae94199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.411+0000 7fae99576640 1 --2- 172.21.15.117:0/4294203993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae94105b90 0x7fae94199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.411+0000 7fae99576640 1 -- 172.21.15.117:0/4294203993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae94103270 msgr2=0x7fae941990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.411+0000 7fae99576640 1 --2- 172.21.15.117:0/4294203993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae94103270 0x7fae941990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.411+0000 7fae99576640 1 -- 172.21.15.117:0/4294203993 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae9419e390 con 0x7fae94105240 2024-09-21T05:56:40.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.411+0000 7fae99576640 1 --2- 172.21.15.117:0/4294203993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae94105240 0x7fae941995f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fae8400d740 tx=0x7fae8400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:40.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.411+0000 7fae8affd640 1 -- 172.21.15.117:0/4294203993 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae84017070 con 0x7fae94105240 2024-09-21T05:56:40.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.411+0000 7fae9bfdb640 1 -- 172.21.15.117:0/4294203993 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae9419e680 con 0x7fae94105240 2024-09-21T05:56:40.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.411+0000 7fae8affd640 1 -- 172.21.15.117:0/4294203993 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae8400de50 con 0x7fae94105240 2024-09-21T05:56:40.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.411+0000 7fae8affd640 1 -- 172.21.15.117:0/4294203993 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae84004ce0 con 0x7fae94105240 2024-09-21T05:56:40.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.411+0000 7fae9bfdb640 1 -- 172.21.15.117:0/4294203993 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae9419ebc0 con 0x7fae94105240 2024-09-21T05:56:40.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.412+0000 7fae9bfdb640 1 -- 172.21.15.117:0/4294203993 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae94103680 con 0x7fae94105240 2024-09-21T05:56:40.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.414+0000 7fae8affd640 1 -- 172.21.15.117:0/4294203993 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7fae84004e80 con 0x7fae94105240 2024-09-21T05:56:40.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.415+0000 7fae8affd640 1 --2- 172.21.15.117:0/4294203993 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fae800b65f0 0x7fae800b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:40.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.415+0000 7fae99d77640 1 --2- 172.21.15.117:0/4294203993 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fae800b65f0 0x7fae800b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:40.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.415+0000 7fae8affd640 1 -- 172.21.15.117:0/4294203993 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fae840e7380 con 0x7fae94105240 2024-09-21T05:56:40.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.415+0000 7fae99d77640 1 --2- 172.21.15.117:0/4294203993 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fae800b65f0 0x7fae800b8b00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fae94102a90 tx=0x7fae900094f0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:40.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.418+0000 7fae8affd640 1 -- 172.21.15.117:0/4294203993 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fae840ada10 con 0x7fae94105240 2024-09-21T05:56:40.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.606+0000 7fae9bfdb640 1 -- 172.21.15.117:0/4294203993 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fae9419ee20 con 0x7fae800b65f0 2024-09-21T05:56:40.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.618+0000 7fae8affd640 1 -- 172.21.15.117:0/4294203993 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7fae9419ee20 con 0x7fae800b65f0 2024-09-21T05:56:40.620 INFO:teuthology.orchestra.run.smithi117.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T05:56:40.621 INFO:teuthology.orchestra.run.smithi117.stdout:mds.a smithi117 running (99s) 19s ago 99s 85.3M - 19.3.0-5120-g18432218 ddce307e1f32 5c3fa963f450 2024-09-21T05:56:40.621 INFO:teuthology.orchestra.run.smithi117.stdout:mds.b smithi138 running (96s) 30s ago 96s 94.1M - 19.3.0-5120-g18432218 ddce307e1f32 76f90b8f9d69 2024-09-21T05:56:40.621 INFO:teuthology.orchestra.run.smithi117.stdout:mds.c smithi192 running (111s) 30s ago 111s 85.5M - 19.3.0-5120-g18432218 ddce307e1f32 ad46b1b9886e 2024-09-21T05:56:40.621 INFO:teuthology.orchestra.run.smithi117.stdout:mds.d smithi117 running (119s) 19s ago 119s 81.8M - 19.3.0-5120-g18432218 ddce307e1f32 a6a2bad07869 2024-09-21T05:56:40.621 INFO:teuthology.orchestra.run.smithi117.stdout:mds.e smithi138 running (116s) 30s ago 116s 84.7M - 19.3.0-5120-g18432218 ddce307e1f32 4f0d649c20a1 2024-09-21T05:56:40.621 INFO:teuthology.orchestra.run.smithi117.stdout:mds.f smithi192 running (114s) 30s ago 113s 85.4M - 19.3.0-5120-g18432218 ddce307e1f32 18bdd104f974 2024-09-21T05:56:40.621 INFO:teuthology.orchestra.run.smithi117.stdout:mds.g smithi117 running (103s) 19s ago 102s 85.3M - 19.3.0-5120-g18432218 ddce307e1f32 ad0a6a2f1e72 2024-09-21T05:56:40.621 INFO:teuthology.orchestra.run.smithi117.stdout:mds.h smithi138 running (109s) 30s ago 108s 85.9M - 19.3.0-5120-g18432218 ddce307e1f32 6bb618c6362c 2024-09-21T05:56:40.621 INFO:teuthology.orchestra.run.smithi117.stdout:mds.i smithi192 running (106s) 30s ago 106s 88.8M - 19.3.0-5120-g18432218 ddce307e1f32 308b7b43d596 2024-09-21T05:56:40.621 INFO:teuthology.orchestra.run.smithi117.stdout:mds.j smithi117 running (2m) 19s ago 2m 81.7M - 19.3.0-5120-g18432218 ddce307e1f32 43bc510c58d6 2024-09-21T05:56:40.621 INFO:teuthology.orchestra.run.smithi117.stdout:mds.k smithi138 running (2m) 30s ago 2m 84.0M - 19.3.0-5120-g18432218 ddce307e1f32 0a96226c7fb4 2024-09-21T05:56:40.621 INFO:teuthology.orchestra.run.smithi117.stdout:mds.l smithi192 running (2m) 30s ago 2m 85.3M - 19.3.0-5120-g18432218 ddce307e1f32 c9511689c622 2024-09-21T05:56:40.621 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.x smithi117 *:9283,8765 running (9m) 19s ago 9m 585M - 19.3.0-5120-g18432218 ddce307e1f32 0d059746aa48 2024-09-21T05:56:40.621 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.y smithi138 *:8443,8765 running (8m) 30s ago 7m 527M - 19.3.0-5120-g18432218 ddce307e1f32 d9b6fffec4d1 2024-09-21T05:56:40.622 INFO:teuthology.orchestra.run.smithi117.stdout:mgr.z smithi192 *:8443,8765 running (8m) 30s ago 8m 524M - 19.3.0-5120-g18432218 ddce307e1f32 544ea82b8168 2024-09-21T05:56:40.622 INFO:teuthology.orchestra.run.smithi117.stdout:mon.a smithi117 running (9m) 19s ago 9m 121M 2048M 19.3.0-5120-g18432218 ddce307e1f32 9aed4354fc16 2024-09-21T05:56:40.622 INFO:teuthology.orchestra.run.smithi117.stdout:mon.b smithi138 running (8m) 30s ago 8m 113M 2048M 19.3.0-5120-g18432218 ddce307e1f32 01a69c7125fc 2024-09-21T05:56:40.622 INFO:teuthology.orchestra.run.smithi117.stdout:mon.c smithi192 running (8m) 30s ago 8m 110M 2048M 19.3.0-5120-g18432218 ddce307e1f32 bbceae20b5a2 2024-09-21T05:56:40.622 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 smithi117 running (7m) 19s ago 7m 124M 4272M 19.3.0-5120-g18432218 ddce307e1f32 246c25bc5dcc 2024-09-21T05:56:40.622 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 smithi138 running (6m) 30s ago 6m 124M 4272M 19.3.0-5120-g18432218 ddce307e1f32 ecd97c2ee7bb 2024-09-21T05:56:40.622 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 smithi192 running (6m) 30s ago 6m 127M 4272M 19.3.0-5120-g18432218 ddce307e1f32 1cbc4820ef8b 2024-09-21T05:56:40.622 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 smithi117 running (5m) 19s ago 5m 121M 4272M 19.3.0-5120-g18432218 ddce307e1f32 94749b8fdcb9 2024-09-21T05:56:40.622 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 smithi138 running (5m) 30s ago 5m 113M 4272M 19.3.0-5120-g18432218 ddce307e1f32 ed15ef1f9298 2024-09-21T05:56:40.622 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 smithi192 running (5m) 30s ago 5m 121M 4272M 19.3.0-5120-g18432218 ddce307e1f32 0b3c306df5d1 2024-09-21T05:56:40.622 INFO:teuthology.orchestra.run.smithi117.stdout:osd.6 smithi117 running (4m) 19s ago 4m 109M 4272M 19.3.0-5120-g18432218 ddce307e1f32 32e2348a9ea4 2024-09-21T05:56:40.622 INFO:teuthology.orchestra.run.smithi117.stdout:osd.7 smithi138 running (4m) 30s ago 4m 114M 4272M 19.3.0-5120-g18432218 ddce307e1f32 9dfa83dbeb4a 2024-09-21T05:56:40.623 INFO:teuthology.orchestra.run.smithi117.stdout:osd.8 smithi192 running (3m) 30s ago 3m 111M 4272M 19.3.0-5120-g18432218 ddce307e1f32 276e48ec3175 2024-09-21T05:56:40.623 INFO:teuthology.orchestra.run.smithi117.stdout:osd.9 smithi117 running (3m) 19s ago 3m 108M 4272M 19.3.0-5120-g18432218 ddce307e1f32 bb1a36e66a81 2024-09-21T05:56:40.623 INFO:teuthology.orchestra.run.smithi117.stdout:osd.10 smithi138 running (2m) 30s ago 2m 122M 4272M 19.3.0-5120-g18432218 ddce307e1f32 1668a7780193 2024-09-21T05:56:40.623 INFO:teuthology.orchestra.run.smithi117.stdout:osd.11 smithi192 running (2m) 30s ago 2m 109M 4272M 19.3.0-5120-g18432218 ddce307e1f32 9c7f7ae14393 2024-09-21T05:56:40.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.621+0000 7fae9bfdb640 1 -- 172.21.15.117:0/4294203993 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fae800b65f0 msgr2=0x7fae800b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.621+0000 7fae9bfdb640 1 --2- 172.21.15.117:0/4294203993 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fae800b65f0 0x7fae800b8b00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fae94102a90 tx=0x7fae900094f0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.621+0000 7fae9bfdb640 1 -- 172.21.15.117:0/4294203993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae94105240 msgr2=0x7fae941995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:40.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.621+0000 7fae9bfdb640 1 --2- 172.21.15.117:0/4294203993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae94105240 0x7fae941995f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fae8400d740 tx=0x7fae8400dc00 comp rx=0 tx=0).stop 2024-09-21T05:56:40.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.621+0000 7fae9bfdb640 1 -- 172.21.15.117:0/4294203993 shutdown_connections 2024-09-21T05:56:40.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.621+0000 7fae9bfdb640 1 --2- 172.21.15.117:0/4294203993 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fae800b65f0 0x7fae800b8b00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.621+0000 7fae9bfdb640 1 --2- 172.21.15.117:0/4294203993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae94105240 0x7fae941995f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.621+0000 7fae9bfdb640 1 --2- 172.21.15.117:0/4294203993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae94105b90 0x7fae94199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.621+0000 7fae9bfdb640 1 --2- 172.21.15.117:0/4294203993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae94103270 0x7fae941990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:40.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.621+0000 7fae9bfdb640 1 -- 172.21.15.117:0/4294203993 >> 172.21.15.117:0/4294203993 conn(0x7fae94079ec0 msgr2=0x7fae94106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:40.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.621+0000 7fae9bfdb640 1 -- 172.21.15.117:0/4294203993 shutdown_connections 2024-09-21T05:56:40.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:40.621+0000 7fae9bfdb640 1 -- 172.21.15.117:0/4294203993 wait complete. 2024-09-21T05:56:40.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:40 smithi138 ceph-mon[34391]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:56:40.974 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:40 smithi117 ceph-mon[30947]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:56:40.981 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:40 smithi192 ceph-mon[34700]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:56:41.319 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ls' 2024-09-21T05:56:41.564 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:41.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:41 smithi138 ceph-mon[34391]: pgmap v308: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T05:56:41.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:41 smithi138 ceph-mon[34391]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:56:41.883 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:41 smithi117 ceph-mon[30947]: pgmap v308: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T05:56:41.883 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:41 smithi117 ceph-mon[30947]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:56:41.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:41 smithi192 ceph-mon[34700]: pgmap v308: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T05:56:41.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:41 smithi192 ceph-mon[34700]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:56:42.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.117+0000 7f46c9764640 1 -- 172.21.15.117:0/684136317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46c4109570 msgr2=0x7f46c4109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:42.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.117+0000 7f46c9764640 1 --2- 172.21.15.117:0/684136317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46c4109570 0x7f46c4109980 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f46ac009900 tx=0x7f46ac02a410 comp rx=0 tx=0).stop 2024-09-21T05:56:42.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.117+0000 7f46c9764640 1 -- 172.21.15.117:0/684136317 shutdown_connections 2024-09-21T05:56:42.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.117+0000 7f46c9764640 1 --2- 172.21.15.117:0/684136317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46c4109570 0x7f46c4109980 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:42.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.117+0000 7f46c9764640 1 --2- 172.21.15.117:0/684136317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46c4109ec0 0x7f46c41106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:42.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.117+0000 7f46c9764640 1 --2- 172.21.15.117:0/684136317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46c4103c30 0x7f46c4104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:42.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.117+0000 7f46c9764640 1 -- 172.21.15.117:0/684136317 >> 172.21.15.117:0/684136317 conn(0x7f46c407b8d0 msgr2=0x7f46c407bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:42.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.117+0000 7f46c9764640 1 -- 172.21.15.117:0/684136317 shutdown_connections 2024-09-21T05:56:42.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.117+0000 7f46c9764640 1 -- 172.21.15.117:0/684136317 wait complete. 2024-09-21T05:56:42.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.118+0000 7f46c9764640 1 Processor -- start 2024-09-21T05:56:42.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.118+0000 7f46c9764640 1 -- start start 2024-09-21T05:56:42.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.118+0000 7f46c9764640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46c4103c30 0x7f46c419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:42.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.118+0000 7f46c9764640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46c4109570 0x7f46c419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:42.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.118+0000 7f46c9764640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46c4109ec0 0x7f46c419fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:42.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.118+0000 7f46c9764640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f46c4113b80 con 0x7f46c4103c30 2024-09-21T05:56:42.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.118+0000 7f46c9764640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f46c4113a00 con 0x7f46c4109ec0 2024-09-21T05:56:42.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.118+0000 7f46c9764640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f46c4113d00 con 0x7f46c4109570 2024-09-21T05:56:42.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.118+0000 7f46c2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46c4103c30 0x7f46c419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:42.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.118+0000 7f46c2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46c4103c30 0x7f46c419a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54444/0 (socket says 172.21.15.117:54444) 2024-09-21T05:56:42.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.118+0000 7f46c2ffd640 1 -- 172.21.15.117:0/926983844 learned_addr learned my addr 172.21.15.117:0/926983844 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:42.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.118+0000 7f46c37fe640 1 --2- 172.21.15.117:0/926983844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46c4109ec0 0x7f46c419fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:42.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.119+0000 7f46c2ffd640 1 -- 172.21.15.117:0/926983844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46c4109570 msgr2=0x7f46c419abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:42.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.119+0000 7f46c27fc640 1 --2- 172.21.15.117:0/926983844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46c4109570 0x7f46c419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:42.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.119+0000 7f46c2ffd640 1 --2- 172.21.15.117:0/926983844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46c4109570 0x7f46c419abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:42.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.119+0000 7f46c2ffd640 1 -- 172.21.15.117:0/926983844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46c4109ec0 msgr2=0x7f46c419fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:42.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.119+0000 7f46c2ffd640 1 --2- 172.21.15.117:0/926983844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46c4109ec0 0x7f46c419fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:42.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.119+0000 7f46c2ffd640 1 -- 172.21.15.117:0/926983844 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c41a0580 con 0x7f46c4103c30 2024-09-21T05:56:42.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.119+0000 7f46c37fe640 1 --2- 172.21.15.117:0/926983844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46c4109ec0 0x7f46c419fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:56:42.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.119+0000 7f46c27fc640 1 --2- 172.21.15.117:0/926983844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46c4109570 0x7f46c419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:42.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.119+0000 7f46c2ffd640 1 --2- 172.21.15.117:0/926983844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46c4103c30 0x7f46c419a610 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00ba50 tx=0x7f46bc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:42.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.119+0000 7f46bbfff640 1 -- 172.21.15.117:0/926983844 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc019070 con 0x7f46c4103c30 2024-09-21T05:56:42.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.119+0000 7f46c9764640 1 -- 172.21.15.117:0/926983844 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46c41a0870 con 0x7f46c4103c30 2024-09-21T05:56:42.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.119+0000 7f46bbfff640 1 -- 172.21.15.117:0/926983844 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46bc00cbe0 con 0x7f46c4103c30 2024-09-21T05:56:42.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.119+0000 7f46bbfff640 1 -- 172.21.15.117:0/926983844 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc0075d0 con 0x7f46c4103c30 2024-09-21T05:56:42.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.119+0000 7f46c9764640 1 -- 172.21.15.117:0/926983844 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46c41a6fd0 con 0x7f46c4103c30 2024-09-21T05:56:42.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.120+0000 7f46c9764640 1 -- 172.21.15.117:0/926983844 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46c41040c0 con 0x7f46c4103c30 2024-09-21T05:56:42.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.122+0000 7f46bbfff640 1 -- 172.21.15.117:0/926983844 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f46bc01c070 con 0x7f46c4103c30 2024-09-21T05:56:42.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.122+0000 7f46bbfff640 1 --2- 172.21.15.117:0/926983844 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f46b00b65f0 0x7f46b00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:42.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.122+0000 7f46c27fc640 1 --2- 172.21.15.117:0/926983844 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f46b00b65f0 0x7f46b00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:42.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.122+0000 7f46bbfff640 1 -- 172.21.15.117:0/926983844 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f46bc0e7020 con 0x7f46c4103c30 2024-09-21T05:56:42.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.123+0000 7f46c27fc640 1 --2- 172.21.15.117:0/926983844 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f46b00b65f0 0x7f46b00b8b00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f46ac000c00 tx=0x7f46ac031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:42.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.125+0000 7f46bbfff640 1 -- 172.21.15.117:0/926983844 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f46bc0ad6b0 con 0x7f46c4103c30 2024-09-21T05:56:42.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.313+0000 7f46c9764640 1 -- 172.21.15.117:0/926983844 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f46c4063800 con 0x7f46b00b65f0 2024-09-21T05:56:42.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.318+0000 7f46bbfff640 1 -- 172.21.15.117:0/926983844 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f46c4063800 con 0x7f46b00b65f0 2024-09-21T05:56:42.320 INFO:teuthology.orchestra.run.smithi117.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-21T05:56:42.320 INFO:teuthology.orchestra.run.smithi117.stdout:mds.all 12/12 31s ago 2m smithi117=a;smithi117=d;smithi117=g;smithi117=j;smithi138=b;smithi138=e;smithi138=h;smithi138=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2024-09-21T05:56:42.320 INFO:teuthology.orchestra.run.smithi117.stdout:mgr 3/3 31s ago 8m smithi117=x;smithi138=y;smithi192=z;count:3 2024-09-21T05:56:42.320 INFO:teuthology.orchestra.run.smithi117.stdout:mon 3/3 31s ago 8m smithi117:172.21.15.117=a;smithi138:172.21.15.138=b;smithi192:172.21.15.192=c;count:3 2024-09-21T05:56:42.321 INFO:teuthology.orchestra.run.smithi117.stdout:osd 12 31s ago - 2024-09-21T05:56:42.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.320+0000 7f46c9764640 1 -- 172.21.15.117:0/926983844 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f46b00b65f0 msgr2=0x7f46b00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:42.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.320+0000 7f46c9764640 1 --2- 172.21.15.117:0/926983844 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f46b00b65f0 0x7f46b00b8b00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f46ac000c00 tx=0x7f46ac031400 comp rx=0 tx=0).stop 2024-09-21T05:56:42.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.320+0000 7f46c9764640 1 -- 172.21.15.117:0/926983844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46c4103c30 msgr2=0x7f46c419a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:42.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.320+0000 7f46c9764640 1 --2- 172.21.15.117:0/926983844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46c4103c30 0x7f46c419a610 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00ba50 tx=0x7f46bc00bf10 comp rx=0 tx=0).stop 2024-09-21T05:56:42.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.321+0000 7f46c9764640 1 -- 172.21.15.117:0/926983844 shutdown_connections 2024-09-21T05:56:42.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.321+0000 7f46c9764640 1 --2- 172.21.15.117:0/926983844 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f46b00b65f0 0x7f46b00b8b00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:42.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.321+0000 7f46c9764640 1 --2- 172.21.15.117:0/926983844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46c4109ec0 0x7f46c419fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:42.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.321+0000 7f46c9764640 1 --2- 172.21.15.117:0/926983844 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46c4109570 0x7f46c419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:42.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.321+0000 7f46c9764640 1 --2- 172.21.15.117:0/926983844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46c4103c30 0x7f46c419a610 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:42.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.321+0000 7f46c9764640 1 -- 172.21.15.117:0/926983844 >> 172.21.15.117:0/926983844 conn(0x7f46c407b8d0 msgr2=0x7f46c4107670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:42.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.321+0000 7f46c9764640 1 -- 172.21.15.117:0/926983844 shutdown_connections 2024-09-21T05:56:42.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:42.321+0000 7f46c9764640 1 -- 172.21.15.117:0/926983844 wait complete. 2024-09-21T05:56:43.007 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- bash -c 'ceph orch host ls' 2024-09-21T05:56:43.256 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:43.713 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:43 smithi117 ceph-mon[30947]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:56:43.713 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:43 smithi117 ceph-mon[30947]: pgmap v309: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T05:56:43.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.710+0000 7f149c6c8640 1 -- 172.21.15.117:0/1748538712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1494102940 msgr2=0x7f14941125c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.710+0000 7f149c6c8640 1 --2- 172.21.15.117:0/1748538712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1494102940 0x7f14941125c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f148c009ca0 tx=0x7f148c02a950 comp rx=0 tx=0).stop 2024-09-21T05:56:43.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.711+0000 7f149c6c8640 1 -- 172.21.15.117:0/1748538712 shutdown_connections 2024-09-21T05:56:43.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.711+0000 7f149c6c8640 1 --2- 172.21.15.117:0/1748538712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1494102940 0x7f14941125c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.711+0000 7f149c6c8640 1 --2- 172.21.15.117:0/1748538712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1494101df0 0x7f1494102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.711+0000 7f149c6c8640 1 --2- 172.21.15.117:0/1748538712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1494104710 0x7f1494104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.711+0000 7f149c6c8640 1 -- 172.21.15.117:0/1748538712 >> 172.21.15.117:0/1748538712 conn(0x7f1494079ec0 msgr2=0x7f149407a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:43.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.711+0000 7f149c6c8640 1 -- 172.21.15.117:0/1748538712 shutdown_connections 2024-09-21T05:56:43.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.711+0000 7f149c6c8640 1 -- 172.21.15.117:0/1748538712 wait complete. 2024-09-21T05:56:43.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.711+0000 7f149c6c8640 1 Processor -- start 2024-09-21T05:56:43.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.712+0000 7f149c6c8640 1 -- start start 2024-09-21T05:56:43.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.712+0000 7f149c6c8640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1494101df0 0x7f1494108f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.712+0000 7f149c6c8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1494102940 0x7f1494105230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.712+0000 7f149c6c8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1494104710 0x7f1494105950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.712+0000 7f149c6c8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1494115de0 con 0x7f1494104710 2024-09-21T05:56:43.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.712+0000 7f149c6c8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1494115c60 con 0x7f1494102940 2024-09-21T05:56:43.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.712+0000 7f149c6c8640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1494115f60 con 0x7f1494101df0 2024-09-21T05:56:43.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.712+0000 7f149a464640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1494101df0 0x7f1494108f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.712+0000 7f149ac65640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1494104710 0x7f1494105950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.712+0000 7f149a464640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1494101df0 0x7f1494108f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:52352/0 (socket says 172.21.15.117:52352) 2024-09-21T05:56:43.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.712+0000 7f149a464640 1 -- 172.21.15.117:0/1158760426 learned_addr learned my addr 172.21.15.117:0/1158760426 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:43.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.713+0000 7f1499c63640 1 --2- 172.21.15.117:0/1158760426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1494102940 0x7f1494105230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.713+0000 7f149a464640 1 -- 172.21.15.117:0/1158760426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1494102940 msgr2=0x7f1494105230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.713+0000 7f149a464640 1 --2- 172.21.15.117:0/1158760426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1494102940 0x7f1494105230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.713+0000 7f149a464640 1 -- 172.21.15.117:0/1158760426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1494104710 msgr2=0x7f1494105950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.713+0000 7f149a464640 1 --2- 172.21.15.117:0/1158760426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1494104710 0x7f1494105950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.713+0000 7f149a464640 1 -- 172.21.15.117:0/1158760426 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f149419c460 con 0x7f1494101df0 2024-09-21T05:56:43.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.713+0000 7f149a464640 1 --2- 172.21.15.117:0/1158760426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1494101df0 0x7f1494108f50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f149000b4a0 tx=0x7f149000b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:43.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.713+0000 7f148b7fe640 1 -- 172.21.15.117:0/1158760426 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1490014070 con 0x7f1494101df0 2024-09-21T05:56:43.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.713+0000 7f148b7fe640 1 -- 172.21.15.117:0/1158760426 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1490010ac0 con 0x7f1494101df0 2024-09-21T05:56:43.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.713+0000 7f149c6c8640 1 -- 172.21.15.117:0/1158760426 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f149419c750 con 0x7f1494101df0 2024-09-21T05:56:43.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.713+0000 7f148b7fe640 1 -- 172.21.15.117:0/1158760426 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1490017c30 con 0x7f1494101df0 2024-09-21T05:56:43.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.714+0000 7f149c6c8640 1 -- 172.21.15.117:0/1158760426 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f149419cc40 con 0x7f1494101df0 2024-09-21T05:56:43.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.714+0000 7f149c6c8640 1 -- 172.21.15.117:0/1158760426 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1494102e00 con 0x7f1494101df0 2024-09-21T05:56:43.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.716+0000 7f148b7fe640 1 -- 172.21.15.117:0/1158760426 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f1490017070 con 0x7f1494101df0 2024-09-21T05:56:43.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.717+0000 7f148b7fe640 1 --2- 172.21.15.117:0/1158760426 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f14800b65f0 0x7f14800b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:43.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.717+0000 7f1499c63640 1 --2- 172.21.15.117:0/1158760426 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f14800b65f0 0x7f14800b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:43.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.717+0000 7f148b7fe640 1 -- 172.21.15.117:0/1158760426 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f14900e6850 con 0x7f1494101df0 2024-09-21T05:56:43.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.717+0000 7f1499c63640 1 --2- 172.21.15.117:0/1158760426 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f14800b65f0 0x7f14800b8b00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1494101d60 tx=0x7f148400a4c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:43.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.720+0000 7f148b7fe640 1 -- 172.21.15.117:0/1158760426 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f14900acee0 con 0x7f1494101df0 2024-09-21T05:56:43.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:43 smithi138 ceph-mon[34391]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:56:43.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:43 smithi138 ceph-mon[34391]: pgmap v309: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T05:56:43.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.905+0000 7f149c6c8640 1 -- 172.21.15.117:0/1158760426 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f149410ccf0 con 0x7f14800b65f0 2024-09-21T05:56:43.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.907+0000 7f148b7fe640 1 -- 172.21.15.117:0/1158760426 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f149410ccf0 con 0x7f14800b65f0 2024-09-21T05:56:43.909 INFO:teuthology.orchestra.run.smithi117.stdout:HOST ADDR LABELS STATUS 2024-09-21T05:56:43.909 INFO:teuthology.orchestra.run.smithi117.stdout:smithi117 172.21.15.117 2024-09-21T05:56:43.909 INFO:teuthology.orchestra.run.smithi117.stdout:smithi138 172.21.15.138 2024-09-21T05:56:43.909 INFO:teuthology.orchestra.run.smithi117.stdout:smithi192 172.21.15.192 2024-09-21T05:56:43.909 INFO:teuthology.orchestra.run.smithi117.stdout:3 hosts in cluster 2024-09-21T05:56:43.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.909+0000 7f149c6c8640 1 -- 172.21.15.117:0/1158760426 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f14800b65f0 msgr2=0x7f14800b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.909+0000 7f149c6c8640 1 --2- 172.21.15.117:0/1158760426 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f14800b65f0 0x7f14800b8b00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1494101d60 tx=0x7f148400a4c0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.909+0000 7f149c6c8640 1 -- 172.21.15.117:0/1158760426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1494101df0 msgr2=0x7f1494108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:43.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.909+0000 7f149c6c8640 1 --2- 172.21.15.117:0/1158760426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1494101df0 0x7f1494108f50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f149000b4a0 tx=0x7f149000b960 comp rx=0 tx=0).stop 2024-09-21T05:56:43.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.909+0000 7f149c6c8640 1 -- 172.21.15.117:0/1158760426 shutdown_connections 2024-09-21T05:56:43.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.909+0000 7f149c6c8640 1 --2- 172.21.15.117:0/1158760426 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f14800b65f0 0x7f14800b8b00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.909+0000 7f149c6c8640 1 --2- 172.21.15.117:0/1158760426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1494104710 0x7f1494105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.909+0000 7f149c6c8640 1 --2- 172.21.15.117:0/1158760426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1494102940 0x7f1494105230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.909+0000 7f149c6c8640 1 --2- 172.21.15.117:0/1158760426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1494101df0 0x7f1494108f50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:43.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.909+0000 7f149c6c8640 1 -- 172.21.15.117:0/1158760426 >> 172.21.15.117:0/1158760426 conn(0x7f1494079ec0 msgr2=0x7f149410c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:43.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.909+0000 7f149c6c8640 1 -- 172.21.15.117:0/1158760426 shutdown_connections 2024-09-21T05:56:43.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:43.909+0000 7f149c6c8640 1 -- 172.21.15.117:0/1158760426 wait complete. 2024-09-21T05:56:43.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:43 smithi192 ceph-mon[34700]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:56:43.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:43 smithi192 ceph-mon[34700]: pgmap v309: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T05:56:44.631 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- bash -c 'ceph orch device ls' 2024-09-21T05:56:44.877 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:45.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.347+0000 7fe0c78b5640 1 -- 172.21.15.117:0/1426880341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0c0109ec0 msgr2=0x7fe0c01106f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:45.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.347+0000 7fe0c78b5640 1 --2- 172.21.15.117:0/1426880341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0c0109ec0 0x7fe0c01106f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8009ca0 tx=0x7fe0b802a8c0 comp rx=0 tx=0).stop 2024-09-21T05:56:45.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.347+0000 7fe0c78b5640 1 -- 172.21.15.117:0/1426880341 shutdown_connections 2024-09-21T05:56:45.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.347+0000 7fe0c78b5640 1 --2- 172.21.15.117:0/1426880341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0c0109ec0 0x7fe0c01106f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:45.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.347+0000 7fe0c78b5640 1 --2- 172.21.15.117:0/1426880341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0c0109570 0x7fe0c0109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:45.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.347+0000 7fe0c78b5640 1 --2- 172.21.15.117:0/1426880341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0c0103c30 0x7fe0c0104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:45.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.347+0000 7fe0c78b5640 1 -- 172.21.15.117:0/1426880341 >> 172.21.15.117:0/1426880341 conn(0x7fe0c007b8d0 msgr2=0x7fe0c007bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:45.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.347+0000 7fe0c78b5640 1 -- 172.21.15.117:0/1426880341 shutdown_connections 2024-09-21T05:56:45.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.347+0000 7fe0c78b5640 1 -- 172.21.15.117:0/1426880341 wait complete. 2024-09-21T05:56:45.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.348+0000 7fe0c78b5640 1 Processor -- start 2024-09-21T05:56:45.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.348+0000 7fe0c78b5640 1 -- start start 2024-09-21T05:56:45.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.349+0000 7fe0c78b5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0c0103c30 0x7fe0c019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:45.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.349+0000 7fe0c78b5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0c0109570 0x7fe0c019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:45.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.349+0000 7fe0c78b5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0c0109ec0 0x7fe0c019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:45.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.349+0000 7fe0c78b5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c0113ae0 con 0x7fe0c0103c30 2024-09-21T05:56:45.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.349+0000 7fe0c78b5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c0113960 con 0x7fe0c0109570 2024-09-21T05:56:45.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.349+0000 7fe0c78b5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c0113c60 con 0x7fe0c0109ec0 2024-09-21T05:56:45.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.349+0000 7fe0c5651640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0c0103c30 0x7fe0c019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:45.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.349+0000 7fe0c5651640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0c0103c30 0x7fe0c019a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50930/0 (socket says 172.21.15.117:50930) 2024-09-21T05:56:45.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.349+0000 7fe0c5651640 1 -- 172.21.15.117:0/1597883311 learned_addr learned my addr 172.21.15.117:0/1597883311 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:45.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.349+0000 7fe0c4e50640 1 --2- 172.21.15.117:0/1597883311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0c0109570 0x7fe0c019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:45.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.349+0000 7fe0c5e52640 1 --2- 172.21.15.117:0/1597883311 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0c0109ec0 0x7fe0c019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:45.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.349+0000 7fe0c5651640 1 -- 172.21.15.117:0/1597883311 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0c0109ec0 msgr2=0x7fe0c019fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:45.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.349+0000 7fe0c5651640 1 --2- 172.21.15.117:0/1597883311 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0c0109ec0 0x7fe0c019fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:45.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.349+0000 7fe0c5651640 1 -- 172.21.15.117:0/1597883311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0c0109570 msgr2=0x7fe0c019a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:45.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.349+0000 7fe0c5651640 1 --2- 172.21.15.117:0/1597883311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0c0109570 0x7fe0c019a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:45.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.349+0000 7fe0c5651640 1 -- 172.21.15.117:0/1597883311 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c01a0360 con 0x7fe0c0103c30 2024-09-21T05:56:45.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.349+0000 7fe0c5651640 1 --2- 172.21.15.117:0/1597883311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0c0103c30 0x7fe0c019a3f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc00e9f0 tx=0x7fe0bc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:45.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.349+0000 7fe0b67fc640 1 -- 172.21.15.117:0/1597883311 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc00cd80 con 0x7fe0c0103c30 2024-09-21T05:56:45.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.349+0000 7fe0b67fc640 1 -- 172.21.15.117:0/1597883311 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0bc0054d0 con 0x7fe0c0103c30 2024-09-21T05:56:45.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.349+0000 7fe0c78b5640 1 -- 172.21.15.117:0/1597883311 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0c01a0650 con 0x7fe0c0103c30 2024-09-21T05:56:45.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.349+0000 7fe0b67fc640 1 -- 172.21.15.117:0/1597883311 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc010690 con 0x7fe0c0103c30 2024-09-21T05:56:45.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.350+0000 7fe0c78b5640 1 -- 172.21.15.117:0/1597883311 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0c01a6db0 con 0x7fe0c0103c30 2024-09-21T05:56:45.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.351+0000 7fe0c78b5640 1 -- 172.21.15.117:0/1597883311 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0c0104040 con 0x7fe0c0103c30 2024-09-21T05:56:45.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.352+0000 7fe0b67fc640 1 -- 172.21.15.117:0/1597883311 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7fe0bc004e90 con 0x7fe0c0103c30 2024-09-21T05:56:45.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.353+0000 7fe0b67fc640 1 --2- 172.21.15.117:0/1597883311 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe0ac0b65f0 0x7fe0ac0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:45.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.353+0000 7fe0c4e50640 1 --2- 172.21.15.117:0/1597883311 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe0ac0b65f0 0x7fe0ac0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:45.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.353+0000 7fe0b67fc640 1 -- 172.21.15.117:0/1597883311 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fe0bc0e7970 con 0x7fe0c0103c30 2024-09-21T05:56:45.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.353+0000 7fe0c4e50640 1 --2- 172.21.15.117:0/1597883311 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe0ac0b65f0 0x7fe0ac0b8b00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0107150 tx=0x7fe0b0005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:45.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.357+0000 7fe0b67fc640 1 -- 172.21.15.117:0/1597883311 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe0bc0ae000 con 0x7fe0c0103c30 2024-09-21T05:56:45.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.564+0000 7fe0c78b5640 1 -- 172.21.15.117:0/1597883311 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fe0c0063800 con 0x7fe0ac0b65f0 2024-09-21T05:56:45.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.566+0000 7fe0b67fc640 1 -- 172.21.15.117:0/1597883311 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fe0c0063800 con 0x7fe0ac0b65f0 2024-09-21T05:56:45.568 INFO:teuthology.orchestra.run.smithi117.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-21T05:56:45.568 INFO:teuthology.orchestra.run.smithi117.stdout:smithi117 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400V6400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-21T05:56:45.568 INFO:teuthology.orchestra.run.smithi117.stdout:smithi138 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6161003C400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-21T05:56:45.568 INFO:teuthology.orchestra.run.smithi117.stdout:smithi192 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500NN400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-21T05:56:45.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.568+0000 7fe0c78b5640 1 -- 172.21.15.117:0/1597883311 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe0ac0b65f0 msgr2=0x7fe0ac0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:45.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.568+0000 7fe0c78b5640 1 --2- 172.21.15.117:0/1597883311 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe0ac0b65f0 0x7fe0ac0b8b00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0107150 tx=0x7fe0b0005f90 comp rx=0 tx=0).stop 2024-09-21T05:56:45.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.568+0000 7fe0c78b5640 1 -- 172.21.15.117:0/1597883311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0c0103c30 msgr2=0x7fe0c019a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:45.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.568+0000 7fe0c78b5640 1 --2- 172.21.15.117:0/1597883311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0c0103c30 0x7fe0c019a3f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc00e9f0 tx=0x7fe0bc00edf0 comp rx=0 tx=0).stop 2024-09-21T05:56:45.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.568+0000 7fe0c78b5640 1 -- 172.21.15.117:0/1597883311 shutdown_connections 2024-09-21T05:56:45.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.568+0000 7fe0c78b5640 1 --2- 172.21.15.117:0/1597883311 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe0ac0b65f0 0x7fe0ac0b8b00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:45.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.568+0000 7fe0c78b5640 1 --2- 172.21.15.117:0/1597883311 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0c0109ec0 0x7fe0c019fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:45.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.568+0000 7fe0c78b5640 1 --2- 172.21.15.117:0/1597883311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0c0109570 0x7fe0c019a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:45.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.568+0000 7fe0c78b5640 1 --2- 172.21.15.117:0/1597883311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0c0103c30 0x7fe0c019a3f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:45.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.568+0000 7fe0c78b5640 1 -- 172.21.15.117:0/1597883311 >> 172.21.15.117:0/1597883311 conn(0x7fe0c007b8d0 msgr2=0x7fe0c01054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:45.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.569+0000 7fe0c78b5640 1 -- 172.21.15.117:0/1597883311 shutdown_connections 2024-09-21T05:56:45.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:45.569+0000 7fe0c78b5640 1 -- 172.21.15.117:0/1597883311 wait complete. 2024-09-21T05:56:45.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:45 smithi138 ceph-mon[34391]: from='client.24940 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:56:45.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:45 smithi138 ceph-mon[34391]: pgmap v310: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T05:56:45.919 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:45 smithi117 ceph-mon[30947]: from='client.24940 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:56:45.919 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:45 smithi117 ceph-mon[30947]: pgmap v310: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T05:56:45.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:45 smithi192 ceph-mon[34700]: from='client.24940 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:56:45.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:45 smithi192 ceph-mon[34700]: pgmap v310: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T05:56:46.347 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T05:56:46.364 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi117.front.sepia.ceph.com 2024-09-21T05:56:46.364 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- bash -c 'ceph fs dump' 2024-09-21T05:56:46.619 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:46.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:46 smithi138 ceph-mon[34391]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:56:46.966 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:46 smithi117 ceph-mon[30947]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:56:46.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:46 smithi192 ceph-mon[34700]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:56:47.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.196+0000 7f6315e22640 1 -- 172.21.15.117:0/213784562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63101049b0 msgr2=0x7f6310104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:47.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.196+0000 7f6315e22640 1 --2- 172.21.15.117:0/213784562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63101049b0 0x7f6310104dc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6300009900 tx=0x7f630002a3f0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.196+0000 7f6315e22640 1 -- 172.21.15.117:0/213784562 shutdown_connections 2024-09-21T05:56:47.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.196+0000 7f6315e22640 1 --2- 172.21.15.117:0/213784562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f631010de10 0x7f6310110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.196+0000 7f6315e22640 1 --2- 172.21.15.117:0/213784562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6310105300 0x7f631010d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.196+0000 7f6315e22640 1 --2- 172.21.15.117:0/213784562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63101049b0 0x7f6310104dc0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.196+0000 7f6315e22640 1 -- 172.21.15.117:0/213784562 >> 172.21.15.117:0/213784562 conn(0x7f6310079730 msgr2=0x7f6310079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:47.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.196+0000 7f6315e22640 1 -- 172.21.15.117:0/213784562 shutdown_connections 2024-09-21T05:56:47.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.196+0000 7f6315e22640 1 -- 172.21.15.117:0/213784562 wait complete. 2024-09-21T05:56:47.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.196+0000 7f6315e22640 1 Processor -- start 2024-09-21T05:56:47.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.197+0000 7f6315e22640 1 -- start start 2024-09-21T05:56:47.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.197+0000 7f6315e22640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63101049b0 0x7f631019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:47.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.197+0000 7f6315e22640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6310105300 0x7f631019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:47.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.197+0000 7f6315e22640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f631010de10 0x7f631019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:47.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.197+0000 7f6315e22640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6310113a70 con 0x7f63101049b0 2024-09-21T05:56:47.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.197+0000 7f6315e22640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f63101138f0 con 0x7f6310105300 2024-09-21T05:56:47.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.197+0000 7f6315e22640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6310113bf0 con 0x7f631010de10 2024-09-21T05:56:47.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.197+0000 7f630f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63101049b0 0x7f631019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:47.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.197+0000 7f630f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63101049b0 0x7f631019a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50944/0 (socket says 172.21.15.117:50944) 2024-09-21T05:56:47.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.197+0000 7f630f7fe640 1 -- 172.21.15.117:0/583813932 learned_addr learned my addr 172.21.15.117:0/583813932 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:47.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.198+0000 7f630ffff640 1 --2- 172.21.15.117:0/583813932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f631010de10 0x7f631019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:47.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.198+0000 7f630effd640 1 --2- 172.21.15.117:0/583813932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6310105300 0x7f631019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:47.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.198+0000 7f630f7fe640 1 -- 172.21.15.117:0/583813932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f631010de10 msgr2=0x7f631019fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:47.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.198+0000 7f630f7fe640 1 --2- 172.21.15.117:0/583813932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f631010de10 0x7f631019fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.198+0000 7f630f7fe640 1 -- 172.21.15.117:0/583813932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6310105300 msgr2=0x7f631019aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:47.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.198+0000 7f630f7fe640 1 --2- 172.21.15.117:0/583813932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6310105300 0x7f631019aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.198+0000 7f630f7fe640 1 -- 172.21.15.117:0/583813932 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63101a0370 con 0x7f63101049b0 2024-09-21T05:56:47.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.198+0000 7f630f7fe640 1 --2- 172.21.15.117:0/583813932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63101049b0 0x7f631019a4c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6300031a90 tx=0x7f6300004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:47.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.199+0000 7f630cff9640 1 -- 172.21.15.117:0/583813932 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630003f030 con 0x7f63101049b0 2024-09-21T05:56:47.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.199+0000 7f630cff9640 1 -- 172.21.15.117:0/583813932 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6300003cb0 con 0x7f63101049b0 2024-09-21T05:56:47.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.199+0000 7f6315e22640 1 -- 172.21.15.117:0/583813932 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63101a0600 con 0x7f63101049b0 2024-09-21T05:56:47.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.199+0000 7f630cff9640 1 -- 172.21.15.117:0/583813932 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6300035070 con 0x7f63101049b0 2024-09-21T05:56:47.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.199+0000 7f6315e22640 1 -- 172.21.15.117:0/583813932 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63101a0ae0 con 0x7f63101049b0 2024-09-21T05:56:47.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.200+0000 7f6315e22640 1 -- 172.21.15.117:0/583813932 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6310104dc0 con 0x7f63101049b0 2024-09-21T05:56:47.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.201+0000 7f630cff9640 1 -- 172.21.15.117:0/583813932 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f6300042030 con 0x7f63101049b0 2024-09-21T05:56:47.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.202+0000 7f630cff9640 1 --2- 172.21.15.117:0/583813932 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f62f80b6620 0x7f62f80b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:47.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.202+0000 7f630effd640 1 --2- 172.21.15.117:0/583813932 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f62f80b6620 0x7f62f80b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:47.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.202+0000 7f630cff9640 1 -- 172.21.15.117:0/583813932 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f6300107f90 con 0x7f63101049b0 2024-09-21T05:56:47.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.203+0000 7f630effd640 1 --2- 172.21.15.117:0/583813932 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f62f80b6620 0x7f62f80b8b30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6304007350 tx=0x7f63040072e0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:47.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.206+0000 7f630cff9640 1 -- 172.21.15.117:0/583813932 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f63000ce620 con 0x7f63101049b0 2024-09-21T05:56:47.424 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.422+0000 7f6315e22640 1 -- 172.21.15.117:0/583813932 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f6310102750 con 0x7f63101049b0 2024-09-21T05:56:47.425 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.423+0000 7f630cff9640 1 -- 172.21.15.117:0/583813932 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3147 (secure 0 0 0) 0x7f63000d2a00 con 0x7f63101049b0 2024-09-21T05:56:47.425 INFO:teuthology.orchestra.run.smithi117.stdout:e26 2024-09-21T05:56:47.425 INFO:teuthology.orchestra.run.smithi117.stdout:btime 2024-09-21T05:55:26:209435+0000 2024-09-21T05:56:47.425 INFO:teuthology.orchestra.run.smithi117.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T05:56:47.425 INFO:teuthology.orchestra.run.smithi117.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:56:47.425 INFO:teuthology.orchestra.run.smithi117.stdout:legacy client fscid: 1 2024-09-21T05:56:47.425 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:56:47.425 INFO:teuthology.orchestra.run.smithi117.stdout:Filesystem 'cephfs' (1) 2024-09-21T05:56:47.425 INFO:teuthology.orchestra.run.smithi117.stdout:fs_name cephfs 2024-09-21T05:56:47.425 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 26 2024-09-21T05:56:47.426 INFO:teuthology.orchestra.run.smithi117.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-21T05:56:47.426 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-09-21T05:55:14.555897+0000 2024-09-21T05:56:47.426 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-09-21T05:55:26.209411+0000 2024-09-21T05:56:47.426 INFO:teuthology.orchestra.run.smithi117.stdout:tableserver 0 2024-09-21T05:56:47.426 INFO:teuthology.orchestra.run.smithi117.stdout:root 0 2024-09-21T05:56:47.426 INFO:teuthology.orchestra.run.smithi117.stdout:session_timeout 300 2024-09-21T05:56:47.426 INFO:teuthology.orchestra.run.smithi117.stdout:session_autoclose 300 2024-09-21T05:56:47.426 INFO:teuthology.orchestra.run.smithi117.stdout:max_file_size 1099511627776 2024-09-21T05:56:47.426 INFO:teuthology.orchestra.run.smithi117.stdout:max_xattr_size 65536 2024-09-21T05:56:47.426 INFO:teuthology.orchestra.run.smithi117.stdout:required_client_features {} 2024-09-21T05:56:47.426 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure 0 2024-09-21T05:56:47.426 INFO:teuthology.orchestra.run.smithi117.stdout:last_failure_osd_epoch 0 2024-09-21T05:56:47.426 INFO:teuthology.orchestra.run.smithi117.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T05:56:47.426 INFO:teuthology.orchestra.run.smithi117.stdout:max_mds 5 2024-09-21T05:56:47.427 INFO:teuthology.orchestra.run.smithi117.stdout:in 0,1,2,3,4 2024-09-21T05:56:47.427 INFO:teuthology.orchestra.run.smithi117.stdout:up {0=24638,1=24637,2=24628,3=24616,4=24602} 2024-09-21T05:56:47.427 INFO:teuthology.orchestra.run.smithi117.stdout:failed 2024-09-21T05:56:47.427 INFO:teuthology.orchestra.run.smithi117.stdout:damaged 2024-09-21T05:56:47.427 INFO:teuthology.orchestra.run.smithi117.stdout:stopped 2024-09-21T05:56:47.427 INFO:teuthology.orchestra.run.smithi117.stdout:data_pools [3] 2024-09-21T05:56:47.427 INFO:teuthology.orchestra.run.smithi117.stdout:metadata_pool 2 2024-09-21T05:56:47.427 INFO:teuthology.orchestra.run.smithi117.stdout:inline_data disabled 2024-09-21T05:56:47.427 INFO:teuthology.orchestra.run.smithi117.stdout:balancer 2024-09-21T05:56:47.427 INFO:teuthology.orchestra.run.smithi117.stdout:bal_rank_mask -1 2024-09-21T05:56:47.427 INFO:teuthology.orchestra.run.smithi117.stdout:standby_count_wanted 0 2024-09-21T05:56:47.427 INFO:teuthology.orchestra.run.smithi117.stdout:qdb_cluster leader: 24638 members: 24638,24637,24616,24628,24602 2024-09-21T05:56:47.427 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.b{0:24638} state up:active seq 4 addr [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:56:47.427 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.l{0:24586} state up:standby-replay seq 1 addr [v2:172.21.15.192:6832/2224910879,v1:172.21.15.192:6833/2224910879] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:56:47.427 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.i{1:24637} state up:active seq 8 addr [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:56:47.428 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.e{1:24572} state up:standby-replay seq 1 addr [v2:172.21.15.138:6834/4276801396,v1:172.21.15.138:6835/4276801396] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:56:47.428 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.c{2:24628} state up:active seq 10 addr [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:56:47.428 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.k{2:24545} state up:standby-replay seq 1 addr [v2:172.21.15.138:6832/3725629862,v1:172.21.15.138:6833/3725629862] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:56:47.428 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.f{3:24616} state up:active seq 11 addr [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:56:47.428 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.a{3:14850} state up:standby-replay seq 1 addr [v2:172.21.15.117:6840/3378096561,v1:172.21.15.117:6841/3378096561] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:56:47.428 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.h{4:24602} state up:active seq 10 addr [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:56:47.428 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.g{4:14838} state up:standby-replay seq 1 addr [v2:172.21.15.117:6838/2314572411,v1:172.21.15.117:6839/2314572411] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:56:47.428 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:56:47.428 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:56:47.428 INFO:teuthology.orchestra.run.smithi117.stdout:Standby daemons: 2024-09-21T05:56:47.428 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:56:47.428 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.j{-1:14775} state up:standby seq 1 addr [v2:172.21.15.117:6834/3227057504,v1:172.21.15.117:6835/3227057504] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:56:47.428 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.d{-1:14787} state up:standby seq 1 addr [v2:172.21.15.117:6836/3303829183,v1:172.21.15.117:6837/3303829183] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T05:56:47.429 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 26 2024-09-21T05:56:47.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.426+0000 7f6315e22640 1 -- 172.21.15.117:0/583813932 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f62f80b6620 msgr2=0x7f62f80b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:47.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.426+0000 7f6315e22640 1 --2- 172.21.15.117:0/583813932 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f62f80b6620 0x7f62f80b8b30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6304007350 tx=0x7f63040072e0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.426+0000 7f6315e22640 1 -- 172.21.15.117:0/583813932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63101049b0 msgr2=0x7f631019a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:47.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.426+0000 7f6315e22640 1 --2- 172.21.15.117:0/583813932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63101049b0 0x7f631019a4c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6300031a90 tx=0x7f6300004270 comp rx=0 tx=0).stop 2024-09-21T05:56:47.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.426+0000 7f6315e22640 1 -- 172.21.15.117:0/583813932 shutdown_connections 2024-09-21T05:56:47.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.426+0000 7f6315e22640 1 --2- 172.21.15.117:0/583813932 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f62f80b6620 0x7f62f80b8b30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.429 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.426+0000 7f6315e22640 1 --2- 172.21.15.117:0/583813932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f631010de10 0x7f631019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.426+0000 7f6315e22640 1 --2- 172.21.15.117:0/583813932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6310105300 0x7f631019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.426+0000 7f6315e22640 1 --2- 172.21.15.117:0/583813932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63101049b0 0x7f631019a4c0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:47.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.426+0000 7f6315e22640 1 -- 172.21.15.117:0/583813932 >> 172.21.15.117:0/583813932 conn(0x7f6310079730 msgr2=0x7f63100ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:47.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.426+0000 7f6315e22640 1 -- 172.21.15.117:0/583813932 shutdown_connections 2024-09-21T05:56:47.430 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:47.426+0000 7f6315e22640 1 -- 172.21.15.117:0/583813932 wait complete. 2024-09-21T05:56:47.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:47 smithi117 ceph-mon[30947]: pgmap v311: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T05:56:47.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:47 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/583813932' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:56:47.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:47 smithi138 ceph-mon[34391]: pgmap v311: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T05:56:47.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:47 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/583813932' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:56:47.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:47 smithi192 ceph-mon[34700]: pgmap v311: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T05:56:47.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:47 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/583813932' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T05:56:48.131 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- bash -c 'ceph osd dump' 2024-09-21T05:56:48.378 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:48.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.871+0000 7f3d3fdac640 1 -- 172.21.15.117:0/802751393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d38108d70 msgr2=0x7f3d3810f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:48.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.871+0000 7f3d3fdac640 1 --2- 172.21.15.117:0/802751393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d38108d70 0x7f3d3810f5a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3d30009900 tx=0x7f3d3002a410 comp rx=0 tx=0).stop 2024-09-21T05:56:48.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.871+0000 7f3d3fdac640 1 -- 172.21.15.117:0/802751393 shutdown_connections 2024-09-21T05:56:48.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.871+0000 7f3d3fdac640 1 --2- 172.21.15.117:0/802751393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d38108d70 0x7f3d3810f5a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:48.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.871+0000 7f3d3fdac640 1 --2- 172.21.15.117:0/802751393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d38108420 0x7f3d38108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:48.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.871+0000 7f3d3fdac640 1 --2- 172.21.15.117:0/802751393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d38102ae0 0x7f3d38102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:48.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.871+0000 7f3d3fdac640 1 -- 172.21.15.117:0/802751393 >> 172.21.15.117:0/802751393 conn(0x7f3d38079770 msgr2=0x7f3d38079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:48.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.871+0000 7f3d3fdac640 1 -- 172.21.15.117:0/802751393 shutdown_connections 2024-09-21T05:56:48.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.871+0000 7f3d3fdac640 1 -- 172.21.15.117:0/802751393 wait complete. 2024-09-21T05:56:48.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.872+0000 7f3d3fdac640 1 Processor -- start 2024-09-21T05:56:48.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.872+0000 7f3d3fdac640 1 -- start start 2024-09-21T05:56:48.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.872+0000 7f3d3fdac640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d38102ae0 0x7f3d3819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:48.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.872+0000 7f3d3fdac640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d38108420 0x7f3d3819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:48.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.872+0000 7f3d3fdac640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d38108d70 0x7f3d3819fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:48.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.872+0000 7f3d3fdac640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3d38112af0 con 0x7f3d38102ae0 2024-09-21T05:56:48.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.872+0000 7f3d3fdac640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3d38112970 con 0x7f3d38108d70 2024-09-21T05:56:48.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.872+0000 7f3d3fdac640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3d38112c70 con 0x7f3d38108420 2024-09-21T05:56:48.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.872+0000 7f3d3db48640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d38102ae0 0x7f3d3819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:48.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.872+0000 7f3d3db48640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d38102ae0 0x7f3d3819a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50952/0 (socket says 172.21.15.117:50952) 2024-09-21T05:56:48.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.872+0000 7f3d3db48640 1 -- 172.21.15.117:0/3090370925 learned_addr learned my addr 172.21.15.117:0/3090370925 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:48.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.873+0000 7f3d3d347640 1 --2- 172.21.15.117:0/3090370925 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d38108420 0x7f3d3819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:48.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.873+0000 7f3d3db48640 1 -- 172.21.15.117:0/3090370925 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d38108420 msgr2=0x7f3d3819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:48.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.873+0000 7f3d3db48640 1 --2- 172.21.15.117:0/3090370925 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d38108420 0x7f3d3819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:48.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.873+0000 7f3d3db48640 1 -- 172.21.15.117:0/3090370925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d38108d70 msgr2=0x7f3d3819fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:48.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.873+0000 7f3d3e349640 1 --2- 172.21.15.117:0/3090370925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d38108d70 0x7f3d3819fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:48.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.873+0000 7f3d3db48640 1 --2- 172.21.15.117:0/3090370925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d38108d70 0x7f3d3819fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:48.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.873+0000 7f3d3db48640 1 -- 172.21.15.117:0/3090370925 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d381a0500 con 0x7f3d38102ae0 2024-09-21T05:56:48.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.873+0000 7f3d3d347640 1 --2- 172.21.15.117:0/3090370925 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d38108420 0x7f3d3819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:48.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.873+0000 7f3d3e349640 1 --2- 172.21.15.117:0/3090370925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d38108d70 0x7f3d3819fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:48.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.873+0000 7f3d3db48640 1 --2- 172.21.15.117:0/3090370925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d38102ae0 0x7f3d3819a590 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800ba50 tx=0x7f3d2800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:48.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.873+0000 7f3d36ffd640 1 -- 172.21.15.117:0/3090370925 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d28019070 con 0x7f3d38102ae0 2024-09-21T05:56:48.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.873+0000 7f3d3fdac640 1 -- 172.21.15.117:0/3090370925 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d381a07f0 con 0x7f3d38102ae0 2024-09-21T05:56:48.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.873+0000 7f3d36ffd640 1 -- 172.21.15.117:0/3090370925 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d2800cbe0 con 0x7f3d38102ae0 2024-09-21T05:56:48.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.873+0000 7f3d36ffd640 1 -- 172.21.15.117:0/3090370925 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d280075d0 con 0x7f3d38102ae0 2024-09-21T05:56:48.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.873+0000 7f3d3fdac640 1 -- 172.21.15.117:0/3090370925 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d381a6fd0 con 0x7f3d38102ae0 2024-09-21T05:56:48.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.874+0000 7f3d3fdac640 1 -- 172.21.15.117:0/3090370925 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d38103d60 con 0x7f3d38102ae0 2024-09-21T05:56:48.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.876+0000 7f3d36ffd640 1 -- 172.21.15.117:0/3090370925 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f3d2801c070 con 0x7f3d38102ae0 2024-09-21T05:56:48.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.876+0000 7f3d36ffd640 1 --2- 172.21.15.117:0/3090370925 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3d200b65f0 0x7f3d200b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:48.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.876+0000 7f3d3d347640 1 --2- 172.21.15.117:0/3090370925 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3d200b65f0 0x7f3d200b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:48.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.877+0000 7f3d36ffd640 1 -- 172.21.15.117:0/3090370925 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f3d280e7020 con 0x7f3d38102ae0 2024-09-21T05:56:48.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.877+0000 7f3d3d347640 1 --2- 172.21.15.117:0/3090370925 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3d200b65f0 0x7f3d200b8b00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c006fd0 tx=0x7f3d2c008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:48.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:48.880+0000 7f3d36ffd640 1 -- 172.21.15.117:0/3090370925 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3d280ad6b0 con 0x7f3d38102ae0 2024-09-21T05:56:49.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:49.059+0000 7f3d3fdac640 1 -- 172.21.15.117:0/3090370925 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f3d3802e640 con 0x7f3d38102ae0 2024-09-21T05:56:49.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:49.060+0000 7f3d36ffd640 1 -- 172.21.15.117:0/3090370925 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v76) ==== 56+0+5676 (secure 0 0 0) 0x7f3d280b1a90 con 0x7f3d38102ae0 2024-09-21T05:56:49.063 INFO:teuthology.orchestra.run.smithi117.stdout:epoch 76 2024-09-21T05:56:49.063 INFO:teuthology.orchestra.run.smithi117.stdout:fsid ac8581a2-77dc-11ef-baf5-efdc52797490 2024-09-21T05:56:49.063 INFO:teuthology.orchestra.run.smithi117.stdout:created 2024-09-21T05:46:47.021846+0000 2024-09-21T05:56:49.063 INFO:teuthology.orchestra.run.smithi117.stdout:modified 2024-09-21T05:55:33.628948+0000 2024-09-21T05:56:49.064 INFO:teuthology.orchestra.run.smithi117.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-21T05:56:49.064 INFO:teuthology.orchestra.run.smithi117.stdout:crush_version 26 2024-09-21T05:56:49.064 INFO:teuthology.orchestra.run.smithi117.stdout:full_ratio 0.95 2024-09-21T05:56:49.064 INFO:teuthology.orchestra.run.smithi117.stdout:backfillfull_ratio 0.9 2024-09-21T05:56:49.064 INFO:teuthology.orchestra.run.smithi117.stdout:nearfull_ratio 0.85 2024-09-21T05:56:49.064 INFO:teuthology.orchestra.run.smithi117.stdout:require_min_compat_client luminous 2024-09-21T05:56:49.064 INFO:teuthology.orchestra.run.smithi117.stdout:min_compat_client luminous 2024-09-21T05:56:49.064 INFO:teuthology.orchestra.run.smithi117.stdout:require_osd_release squid 2024-09-21T05:56:49.064 INFO:teuthology.orchestra.run.smithi117.stdout:stretch_mode_enabled false 2024-09-21T05:56:49.064 INFO:teuthology.orchestra.run.smithi117.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-21T05:56:49.064 INFO:teuthology.orchestra.run.smithi117.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 74 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-21T05:56:49.064 INFO:teuthology.orchestra.run.smithi117.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 74 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-21T05:56:49.064 INFO:teuthology.orchestra.run.smithi117.stdout:max_osd 12 2024-09-21T05:56:49.064 INFO:teuthology.orchestra.run.smithi117.stdout:osd.0 up in weight 1 up_from 9 up_thru 72 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6802/3761478035,v1:172.21.15.117:6803/3761478035] [v2:172.21.15.117:6804/3761478035,v1:172.21.15.117:6805/3761478035] exists,up 68315502-fa6c-4a6e-a2d4-a7dde8071c02 2024-09-21T05:56:49.065 INFO:teuthology.orchestra.run.smithi117.stdout:osd.1 up in weight 1 up_from 14 up_thru 72 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6800/1737365588,v1:172.21.15.138:6801/1737365588] [v2:172.21.15.138:6802/1737365588,v1:172.21.15.138:6803/1737365588] exists,up d8910a08-4ef3-49f2-a4db-e3588d85d12b 2024-09-21T05:56:49.065 INFO:teuthology.orchestra.run.smithi117.stdout:osd.2 up in weight 1 up_from 19 up_thru 72 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6800/3002534992,v1:172.21.15.192:6801/3002534992] [v2:172.21.15.192:6802/3002534992,v1:172.21.15.192:6803/3002534992] exists,up 94972378-a11f-4129-9e40-ad3fdec1d189 2024-09-21T05:56:49.065 INFO:teuthology.orchestra.run.smithi117.stdout:osd.3 up in weight 1 up_from 25 up_thru 72 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6810/1222400293,v1:172.21.15.117:6811/1222400293] [v2:172.21.15.117:6812/1222400293,v1:172.21.15.117:6813/1222400293] exists,up d4168c43-f0de-4445-945c-92d1cf270c33 2024-09-21T05:56:49.065 INFO:teuthology.orchestra.run.smithi117.stdout:osd.4 up in weight 1 up_from 31 up_thru 72 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6808/4294892870,v1:172.21.15.138:6809/4294892870] [v2:172.21.15.138:6810/4294892870,v1:172.21.15.138:6811/4294892870] exists,up 28f1d7c0-8392-4a33-85c0-0de611cdd4b1 2024-09-21T05:56:49.065 INFO:teuthology.orchestra.run.smithi117.stdout:osd.5 up in weight 1 up_from 36 up_thru 72 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6808/1776840019,v1:172.21.15.192:6809/1776840019] [v2:172.21.15.192:6810/1776840019,v1:172.21.15.192:6811/1776840019] exists,up ca902407-45e8-4bc5-bdbd-a729c1de8322 2024-09-21T05:56:49.065 INFO:teuthology.orchestra.run.smithi117.stdout:osd.6 up in weight 1 up_from 41 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6818/1093341590,v1:172.21.15.117:6819/1093341590] [v2:172.21.15.117:6820/1093341590,v1:172.21.15.117:6821/1093341590] exists,up 6e507a91-ed09-4a33-bf0e-c4b593b3c5aa 2024-09-21T05:56:49.065 INFO:teuthology.orchestra.run.smithi117.stdout:osd.7 up in weight 1 up_from 46 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6816/1448179714,v1:172.21.15.138:6817/1448179714] [v2:172.21.15.138:6818/1448179714,v1:172.21.15.138:6819/1448179714] exists,up 36a47dd7-67f7-4ebe-8759-3a8bb6bd2ea3 2024-09-21T05:56:49.065 INFO:teuthology.orchestra.run.smithi117.stdout:osd.8 up in weight 1 up_from 52 up_thru 72 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6816/3960902885,v1:172.21.15.192:6817/3960902885] [v2:172.21.15.192:6818/3960902885,v1:172.21.15.192:6819/3960902885] exists,up 118e0b5d-2619-4512-94c5-8d07a0d3a28f 2024-09-21T05:56:49.065 INFO:teuthology.orchestra.run.smithi117.stdout:osd.9 up in weight 1 up_from 58 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6826/3453855195,v1:172.21.15.117:6827/3453855195] [v2:172.21.15.117:6828/3453855195,v1:172.21.15.117:6829/3453855195] exists,up 47543bc6-fa0b-4e72-9742-97dcda0b7f8d 2024-09-21T05:56:49.065 INFO:teuthology.orchestra.run.smithi117.stdout:osd.10 up in weight 1 up_from 63 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6824/903392451,v1:172.21.15.138:6825/903392451] [v2:172.21.15.138:6826/903392451,v1:172.21.15.138:6827/903392451] exists,up 5e6708b4-7a85-4fa5-a03c-27d99f7d749f 2024-09-21T05:56:49.065 INFO:teuthology.orchestra.run.smithi117.stdout:osd.11 up in weight 1 up_from 68 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6824/899090479,v1:172.21.15.192:6825/899090479] [v2:172.21.15.192:6826/899090479,v1:172.21.15.192:6827/899090479] exists,up 24bc2ef7-e2b4-43c6-9e6c-43ae02390415 2024-09-21T05:56:49.065 INFO:teuthology.orchestra.run.smithi117.stdout:pg_upmap_items 2.b [8,11] 2024-09-21T05:56:49.065 INFO:teuthology.orchestra.run.smithi117.stdout:pg_upmap_items 2.e [2,1] 2024-09-21T05:56:49.066 INFO:teuthology.orchestra.run.smithi117.stdout:pg_upmap_items 2.16 [2,4] 2024-09-21T05:56:49.066 INFO:teuthology.orchestra.run.smithi117.stdout:pg_upmap_items 2.1f [7,11] 2024-09-21T05:56:49.066 INFO:teuthology.orchestra.run.smithi117.stdout:pg_upmap_items 3.3 [4,9] 2024-09-21T05:56:49.066 INFO:teuthology.orchestra.run.smithi117.stdout:pg_upmap_items 3.36 [4,9] 2024-09-21T05:56:49.066 INFO:teuthology.orchestra.run.smithi117.stdout:pg_upmap_items 3.38 [3,9] 2024-09-21T05:56:49.066 INFO:teuthology.orchestra.run.smithi117.stdout:blocklist 172.21.15.117:0/2989394419 expires 2024-09-22T05:48:32.406654+0000 2024-09-21T05:56:49.066 INFO:teuthology.orchestra.run.smithi117.stdout:blocklist 172.21.15.117:6801/1702676172 expires 2024-09-22T05:48:32.406654+0000 2024-09-21T05:56:49.066 INFO:teuthology.orchestra.run.smithi117.stdout:blocklist 172.21.15.117:0/1766406059 expires 2024-09-22T05:47:09.112435+0000 2024-09-21T05:56:49.066 INFO:teuthology.orchestra.run.smithi117.stdout:blocklist 172.21.15.117:6800/2337036514 expires 2024-09-22T05:47:09.112435+0000 2024-09-21T05:56:49.066 INFO:teuthology.orchestra.run.smithi117.stdout:blocklist 172.21.15.117:0/985798343 expires 2024-09-22T05:47:09.112435+0000 2024-09-21T05:56:49.066 INFO:teuthology.orchestra.run.smithi117.stdout:blocklist 172.21.15.117:0/1083273222 expires 2024-09-22T05:48:32.406654+0000 2024-09-21T05:56:49.066 INFO:teuthology.orchestra.run.smithi117.stdout:blocklist 172.21.15.117:0/473284565 expires 2024-09-22T05:47:09.112435+0000 2024-09-21T05:56:49.066 INFO:teuthology.orchestra.run.smithi117.stdout:blocklist 172.21.15.117:6801/2337036514 expires 2024-09-22T05:47:09.112435+0000 2024-09-21T05:56:49.066 INFO:teuthology.orchestra.run.smithi117.stdout:blocklist 172.21.15.117:6800/816635165 expires 2024-09-22T05:47:33.789146+0000 2024-09-21T05:56:49.067 INFO:teuthology.orchestra.run.smithi117.stdout:blocklist 172.21.15.117:0/2642940197 expires 2024-09-22T05:48:32.406654+0000 2024-09-21T05:56:49.067 INFO:teuthology.orchestra.run.smithi117.stdout:blocklist 172.21.15.117:6801/816635165 expires 2024-09-22T05:47:33.789146+0000 2024-09-21T05:56:49.067 INFO:teuthology.orchestra.run.smithi117.stdout:blocklist 172.21.15.117:0/2225537433 expires 2024-09-22T05:47:33.789146+0000 2024-09-21T05:56:49.067 INFO:teuthology.orchestra.run.smithi117.stdout:blocklist 172.21.15.117:0/3651584169 expires 2024-09-22T05:47:33.789146+0000 2024-09-21T05:56:49.067 INFO:teuthology.orchestra.run.smithi117.stdout:blocklist 172.21.15.117:6800/1702676172 expires 2024-09-22T05:48:32.406654+0000 2024-09-21T05:56:49.067 INFO:teuthology.orchestra.run.smithi117.stdout:blocklist 172.21.15.117:0/1284403882 expires 2024-09-22T05:47:33.789146+0000 2024-09-21T05:56:49.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:49.063+0000 7f3d3fdac640 1 -- 172.21.15.117:0/3090370925 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3d200b65f0 msgr2=0x7f3d200b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:49.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:49.063+0000 7f3d3fdac640 1 --2- 172.21.15.117:0/3090370925 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3d200b65f0 0x7f3d200b8b00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c006fd0 tx=0x7f3d2c008040 comp rx=0 tx=0).stop 2024-09-21T05:56:49.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:49.063+0000 7f3d3fdac640 1 -- 172.21.15.117:0/3090370925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d38102ae0 msgr2=0x7f3d3819a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:49.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:49.063+0000 7f3d3fdac640 1 --2- 172.21.15.117:0/3090370925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d38102ae0 0x7f3d3819a590 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800ba50 tx=0x7f3d2800bf10 comp rx=0 tx=0).stop 2024-09-21T05:56:49.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:49.063+0000 7f3d3fdac640 1 -- 172.21.15.117:0/3090370925 shutdown_connections 2024-09-21T05:56:49.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:49.063+0000 7f3d3fdac640 1 --2- 172.21.15.117:0/3090370925 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3d200b65f0 0x7f3d200b8b00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:49.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:49.063+0000 7f3d3fdac640 1 --2- 172.21.15.117:0/3090370925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d38108d70 0x7f3d3819fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:49.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:49.063+0000 7f3d3fdac640 1 --2- 172.21.15.117:0/3090370925 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d38108420 0x7f3d3819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:49.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:49.063+0000 7f3d3fdac640 1 --2- 172.21.15.117:0/3090370925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d38102ae0 0x7f3d3819a590 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:49.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:49.063+0000 7f3d3fdac640 1 -- 172.21.15.117:0/3090370925 >> 172.21.15.117:0/3090370925 conn(0x7f3d38079770 msgr2=0x7f3d38101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:49.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:49.063+0000 7f3d3fdac640 1 -- 172.21.15.117:0/3090370925 shutdown_connections 2024-09-21T05:56:49.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:49.063+0000 7f3d3fdac640 1 -- 172.21.15.117:0/3090370925 wait complete. 2024-09-21T05:56:49.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:49 smithi117 ceph-mon[30947]: pgmap v312: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T05:56:49.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:49 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3090370925' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T05:56:49.800 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-21T05:56:49.811 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T05:56:49.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:49 smithi138 ceph-mon[34391]: pgmap v312: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T05:56:49.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:49 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3090370925' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T05:56:49.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:49 smithi192 ceph-mon[34700]: pgmap v312: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T05:56:49.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:49 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3090370925' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T05:56:50.057 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:50.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.501+0000 7f5f98d0a640 1 -- 172.21.15.117:0/3500908376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f941083a0 msgr2=0x7f5f941087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:50.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.501+0000 7f5f98d0a640 1 --2- 172.21.15.117:0/3500908376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f941083a0 0x7f5f941087b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5f84009900 tx=0x7f5f8402a410 comp rx=0 tx=0).stop 2024-09-21T05:56:50.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.501+0000 7f5f98d0a640 1 -- 172.21.15.117:0/3500908376 shutdown_connections 2024-09-21T05:56:50.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.501+0000 7f5f98d0a640 1 --2- 172.21.15.117:0/3500908376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f941083a0 0x7f5f941087b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:50.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.501+0000 7f5f98d0a640 1 --2- 172.21.15.117:0/3500908376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f94108cf0 0x7f5f9410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:50.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.501+0000 7f5f98d0a640 1 --2- 172.21.15.117:0/3500908376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f94102a60 0x7f5f94102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:50.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.501+0000 7f5f98d0a640 1 -- 172.21.15.117:0/3500908376 >> 172.21.15.117:0/3500908376 conn(0x7f5f94079730 msgr2=0x7f5f94079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:50.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.501+0000 7f5f98d0a640 1 -- 172.21.15.117:0/3500908376 shutdown_connections 2024-09-21T05:56:50.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.502+0000 7f5f98d0a640 1 -- 172.21.15.117:0/3500908376 wait complete. 2024-09-21T05:56:50.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.502+0000 7f5f98d0a640 1 Processor -- start 2024-09-21T05:56:50.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.502+0000 7f5f98d0a640 1 -- start start 2024-09-21T05:56:50.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.503+0000 7f5f98d0a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f94102a60 0x7f5f9419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:50.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.503+0000 7f5f98d0a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f941083a0 0x7f5f9419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:50.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.503+0000 7f5f98d0a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f94108cf0 0x7f5f9419fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:50.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.503+0000 7f5f98d0a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5f94113b20 con 0x7f5f94102a60 2024-09-21T05:56:50.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.503+0000 7f5f98d0a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5f941139a0 con 0x7f5f941083a0 2024-09-21T05:56:50.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.503+0000 7f5f98d0a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5f94113ca0 con 0x7f5f94108cf0 2024-09-21T05:56:50.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.503+0000 7f5f9259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f94102a60 0x7f5f9419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:50.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.503+0000 7f5f9259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f94102a60 0x7f5f9419a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50974/0 (socket says 172.21.15.117:50974) 2024-09-21T05:56:50.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.503+0000 7f5f9259c640 1 -- 172.21.15.117:0/2337945477 learned_addr learned my addr 172.21.15.117:0/2337945477 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:50.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.503+0000 7f5f92d9d640 1 --2- 172.21.15.117:0/2337945477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f94108cf0 0x7f5f9419fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:50.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.503+0000 7f5f91d9b640 1 --2- 172.21.15.117:0/2337945477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f941083a0 0x7f5f9419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:50.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.503+0000 7f5f9259c640 1 -- 172.21.15.117:0/2337945477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f94108cf0 msgr2=0x7f5f9419fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:50.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.503+0000 7f5f9259c640 1 --2- 172.21.15.117:0/2337945477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f94108cf0 0x7f5f9419fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:50.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.503+0000 7f5f9259c640 1 -- 172.21.15.117:0/2337945477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f941083a0 msgr2=0x7f5f9419abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:50.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.503+0000 7f5f9259c640 1 --2- 172.21.15.117:0/2337945477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f941083a0 0x7f5f9419abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:50.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.503+0000 7f5f9259c640 1 -- 172.21.15.117:0/2337945477 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f941a05c0 con 0x7f5f94102a60 2024-09-21T05:56:50.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.503+0000 7f5f91d9b640 1 --2- 172.21.15.117:0/2337945477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f941083a0 0x7f5f9419abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:56:50.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.503+0000 7f5f9259c640 1 --2- 172.21.15.117:0/2337945477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f94102a60 0x7f5f9419a650 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5f8800ba50 tx=0x7f5f8800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:50.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.503+0000 7f5f837fe640 1 -- 172.21.15.117:0/2337945477 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f88019070 con 0x7f5f94102a60 2024-09-21T05:56:50.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.504+0000 7f5f837fe640 1 -- 172.21.15.117:0/2337945477 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f8800cbe0 con 0x7f5f94102a60 2024-09-21T05:56:50.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.504+0000 7f5f98d0a640 1 -- 172.21.15.117:0/2337945477 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f941a08b0 con 0x7f5f94102a60 2024-09-21T05:56:50.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.504+0000 7f5f837fe640 1 -- 172.21.15.117:0/2337945477 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f880075d0 con 0x7f5f94102a60 2024-09-21T05:56:50.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.504+0000 7f5f98d0a640 1 -- 172.21.15.117:0/2337945477 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f941a7010 con 0x7f5f94102a60 2024-09-21T05:56:50.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.505+0000 7f5f98d0a640 1 -- 172.21.15.117:0/2337945477 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f94102ef0 con 0x7f5f94102a60 2024-09-21T05:56:50.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.506+0000 7f5f837fe640 1 -- 172.21.15.117:0/2337945477 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f5f8801c070 con 0x7f5f94102a60 2024-09-21T05:56:50.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.507+0000 7f5f837fe640 1 --2- 172.21.15.117:0/2337945477 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5f7c0b65f0 0x7f5f7c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:50.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.507+0000 7f5f91d9b640 1 --2- 172.21.15.117:0/2337945477 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5f7c0b65f0 0x7f5f7c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:50.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.507+0000 7f5f837fe640 1 -- 172.21.15.117:0/2337945477 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f5f880e6f70 con 0x7f5f94102a60 2024-09-21T05:56:50.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.508+0000 7f5f91d9b640 1 --2- 172.21.15.117:0/2337945477 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5f7c0b65f0 0x7f5f7c0b8b00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5f84000c00 tx=0x7f5f84031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:50.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.510+0000 7f5f837fe640 1 -- 172.21.15.117:0/2337945477 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5f880b19e0 con 0x7f5f94102a60 2024-09-21T05:56:50.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.723+0000 7f5f98d0a640 1 -- 172.21.15.117:0/2337945477 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f9419b780 con 0x7f5f94102a60 2024-09-21T05:56:50.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.724+0000 7f5f837fe640 1 -- 172.21.15.117:0/2337945477 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+12001 (secure 0 0 0) 0x7f5f9419b780 con 0x7f5f94102a60 2024-09-21T05:56:50.727 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:56:50.727 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":26,"btime":"2024-09-21T05:55:26:209435+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3227057504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3227057504},{"type":"v1","addr":"172.21.15.117:6835","nonce":3227057504}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3303829183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3303829183},{"type":"v1","addr":"172.21.15.117:6837","nonce":3303829183}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T05:55:14.555897+0000","modified":"2024-09-21T05:55:26.209411+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24638,"mds_1":24637,"mds_2":24628,"mds_3":24616,"mds_4":24602},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6839/2314572411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2314572411},{"type":"v1","addr":"172.21.15.117:6839","nonce":2314572411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6841/3378096561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":3378096561},{"type":"v1","addr":"172.21.15.117:6841","nonce":3378096561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/3725629862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3725629862},{"type":"v1","addr":"172.21.15.138:6833","nonce":3725629862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/4276801396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4276801396},{"type":"v1","addr":"172.21.15.138:6835","nonce":4276801396}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6833/2224910879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2224910879},{"type":"v1","addr":"172.21.15.192:6833","nonce":2224910879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.138:6837/2540335374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2540335374},{"type":"v1","addr":"172.21.15.138:6837","nonce":2540335374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24616":{"gid":24616,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.192:6835/3530688860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3530688860},{"type":"v1","addr":"172.21.15.192:6835","nonce":3530688860}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.192:6837/2683860842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2683860842},{"type":"v1","addr":"172.21.15.192:6837","nonce":2683860842}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.192:6839/1727623231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1727623231},{"type":"v1","addr":"172.21.15.192:6839","nonce":1727623231}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/3230372576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":3230372576},{"type":"v1","addr":"172.21.15.138:6839","nonce":3230372576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24638,24637,24616,24628,24602]},"id":1}]} 2024-09-21T05:56:50.727 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 26 2024-09-21T05:56:50.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.726+0000 7f5f98d0a640 1 -- 172.21.15.117:0/2337945477 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5f7c0b65f0 msgr2=0x7f5f7c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:50.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.726+0000 7f5f98d0a640 1 --2- 172.21.15.117:0/2337945477 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5f7c0b65f0 0x7f5f7c0b8b00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5f84000c00 tx=0x7f5f84031400 comp rx=0 tx=0).stop 2024-09-21T05:56:50.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.726+0000 7f5f98d0a640 1 -- 172.21.15.117:0/2337945477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f94102a60 msgr2=0x7f5f9419a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:50.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.726+0000 7f5f98d0a640 1 --2- 172.21.15.117:0/2337945477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f94102a60 0x7f5f9419a650 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5f8800ba50 tx=0x7f5f8800bf10 comp rx=0 tx=0).stop 2024-09-21T05:56:50.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.726+0000 7f5f98d0a640 1 -- 172.21.15.117:0/2337945477 shutdown_connections 2024-09-21T05:56:50.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.726+0000 7f5f98d0a640 1 --2- 172.21.15.117:0/2337945477 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5f7c0b65f0 0x7f5f7c0b8b00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:50.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.726+0000 7f5f98d0a640 1 --2- 172.21.15.117:0/2337945477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f94108cf0 0x7f5f9419fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:50.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.726+0000 7f5f98d0a640 1 --2- 172.21.15.117:0/2337945477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f941083a0 0x7f5f9419abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:50.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.726+0000 7f5f98d0a640 1 --2- 172.21.15.117:0/2337945477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f94102a60 0x7f5f9419a650 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:50.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.726+0000 7f5f98d0a640 1 -- 172.21.15.117:0/2337945477 >> 172.21.15.117:0/2337945477 conn(0x7f5f94079730 msgr2=0x7f5f94105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:50.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.726+0000 7f5f98d0a640 1 -- 172.21.15.117:0/2337945477 shutdown_connections 2024-09-21T05:56:50.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:50.726+0000 7f5f98d0a640 1 -- 172.21.15.117:0/2337945477 wait complete. 2024-09-21T05:56:51.408 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T05:56:51.655 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:51.775 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:51 smithi117 ceph-mon[30947]: pgmap v313: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T05:56:51.775 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:51 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2337945477' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:56:51.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:51 smithi138 ceph-mon[34391]: pgmap v313: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T05:56:51.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:51 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2337945477' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:56:51.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:51 smithi192 ceph-mon[34700]: pgmap v313: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T05:56:51.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:51 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2337945477' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:56:52.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.137+0000 7f9643c61640 1 -- 172.21.15.117:0/3049876676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f963c105300 msgr2=0x7f963c10d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:52.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.137+0000 7f9643c61640 1 --2- 172.21.15.117:0/3049876676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f963c105300 0x7f963c10d850 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9630009900 tx=0x7f963002a410 comp rx=0 tx=0).stop 2024-09-21T05:56:52.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.137+0000 7f9643c61640 1 -- 172.21.15.117:0/3049876676 shutdown_connections 2024-09-21T05:56:52.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.137+0000 7f9643c61640 1 --2- 172.21.15.117:0/3049876676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f963c105300 0x7f963c10d850 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:52.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.137+0000 7f9643c61640 1 --2- 172.21.15.117:0/3049876676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f963c10de10 0x7f963c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:52.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.137+0000 7f9643c61640 1 --2- 172.21.15.117:0/3049876676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f963c1049b0 0x7f963c104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:52.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.137+0000 7f9643c61640 1 -- 172.21.15.117:0/3049876676 >> 172.21.15.117:0/3049876676 conn(0x7f963c079730 msgr2=0x7f963c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:52.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.137+0000 7f9643c61640 1 -- 172.21.15.117:0/3049876676 shutdown_connections 2024-09-21T05:56:52.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.137+0000 7f9643c61640 1 -- 172.21.15.117:0/3049876676 wait complete. 2024-09-21T05:56:52.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.138+0000 7f9643c61640 1 Processor -- start 2024-09-21T05:56:52.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.138+0000 7f9643c61640 1 -- start start 2024-09-21T05:56:52.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.138+0000 7f9643c61640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f963c1049b0 0x7f963c19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:52.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.138+0000 7f9643c61640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f963c105300 0x7f963c19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:52.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.138+0000 7f9643c61640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f963c10de10 0x7f963c19fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:52.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.138+0000 7f9643c61640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f963c113b50 con 0x7f963c1049b0 2024-09-21T05:56:52.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.138+0000 7f9643c61640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f963c1139d0 con 0x7f963c10de10 2024-09-21T05:56:52.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.138+0000 7f9643c61640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f963c113cd0 con 0x7f963c105300 2024-09-21T05:56:52.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.138+0000 7f96419fd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f963c1049b0 0x7f963c19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:52.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.138+0000 7f96419fd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f963c1049b0 0x7f963c19a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50994/0 (socket says 172.21.15.117:50994) 2024-09-21T05:56:52.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.138+0000 7f96419fd640 1 -- 172.21.15.117:0/2424124526 learned_addr learned my addr 172.21.15.117:0/2424124526 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:52.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.138+0000 7f96411fc640 1 --2- 172.21.15.117:0/2424124526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f963c105300 0x7f963c19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:52.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.138+0000 7f96421fe640 1 --2- 172.21.15.117:0/2424124526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f963c10de10 0x7f963c19fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:52.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.138+0000 7f96419fd640 1 -- 172.21.15.117:0/2424124526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f963c105300 msgr2=0x7f963c19ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:52.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.138+0000 7f96419fd640 1 --2- 172.21.15.117:0/2424124526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f963c105300 0x7f963c19ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:52.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.138+0000 7f96419fd640 1 -- 172.21.15.117:0/2424124526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f963c10de10 msgr2=0x7f963c19fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:52.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.138+0000 7f96419fd640 1 --2- 172.21.15.117:0/2424124526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f963c10de10 0x7f963c19fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:52.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.138+0000 7f96419fd640 1 -- 172.21.15.117:0/2424124526 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f963c1a0550 con 0x7f963c1049b0 2024-09-21T05:56:52.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.138+0000 7f96421fe640 1 --2- 172.21.15.117:0/2424124526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f963c10de10 0x7f963c19fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:52.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.138+0000 7f96411fc640 1 --2- 172.21.15.117:0/2424124526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f963c105300 0x7f963c19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:56:52.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.139+0000 7f96419fd640 1 --2- 172.21.15.117:0/2424124526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f963c1049b0 0x7f963c19a5e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f962c00b9c0 tx=0x7f962c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:52.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.139+0000 7f963affd640 1 -- 172.21.15.117:0/2424124526 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c019070 con 0x7f963c1049b0 2024-09-21T05:56:52.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.139+0000 7f963affd640 1 -- 172.21.15.117:0/2424124526 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f962c00cba0 con 0x7f963c1049b0 2024-09-21T05:56:52.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.139+0000 7f9643c61640 1 -- 172.21.15.117:0/2424124526 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f963c1a0840 con 0x7f963c1049b0 2024-09-21T05:56:52.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.139+0000 7f963affd640 1 -- 172.21.15.117:0/2424124526 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c0125a0 con 0x7f963c1049b0 2024-09-21T05:56:52.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.139+0000 7f9643c61640 1 -- 172.21.15.117:0/2424124526 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f963c1a6fa0 con 0x7f963c1049b0 2024-09-21T05:56:52.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.140+0000 7f9643c61640 1 -- 172.21.15.117:0/2424124526 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f963c104e40 con 0x7f963c1049b0 2024-09-21T05:56:52.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.141+0000 7f963affd640 1 -- 172.21.15.117:0/2424124526 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f962c01c070 con 0x7f963c1049b0 2024-09-21T05:56:52.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.142+0000 7f963affd640 1 --2- 172.21.15.117:0/2424124526 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f96240b6670 0x7f96240b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:52.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.142+0000 7f96411fc640 1 --2- 172.21.15.117:0/2424124526 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f96240b6670 0x7f96240b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:52.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.142+0000 7f963affd640 1 -- 172.21.15.117:0/2424124526 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f962c0e7000 con 0x7f963c1049b0 2024-09-21T05:56:52.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.143+0000 7f96411fc640 1 --2- 172.21.15.117:0/2424124526 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f96240b6670 0x7f96240b8b80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9630009230 tx=0x7f9630000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:52.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.145+0000 7f963affd640 1 -- 172.21.15.117:0/2424124526 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f962c0ad690 con 0x7f963c1049b0 2024-09-21T05:56:52.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.360+0000 7f9643c61640 1 -- 172.21.15.117:0/2424124526 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f963c19b710 con 0x7f963c1049b0 2024-09-21T05:56:52.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.361+0000 7f963affd640 1 -- 172.21.15.117:0/2424124526 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+12001 (secure 0 0 0) 0x7f963c19b710 con 0x7f963c1049b0 2024-09-21T05:56:52.364 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:56:52.364 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":26,"btime":"2024-09-21T05:55:26:209435+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3227057504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3227057504},{"type":"v1","addr":"172.21.15.117:6835","nonce":3227057504}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3303829183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3303829183},{"type":"v1","addr":"172.21.15.117:6837","nonce":3303829183}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T05:55:14.555897+0000","modified":"2024-09-21T05:55:26.209411+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24638,"mds_1":24637,"mds_2":24628,"mds_3":24616,"mds_4":24602},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6839/2314572411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2314572411},{"type":"v1","addr":"172.21.15.117:6839","nonce":2314572411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6841/3378096561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":3378096561},{"type":"v1","addr":"172.21.15.117:6841","nonce":3378096561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/3725629862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3725629862},{"type":"v1","addr":"172.21.15.138:6833","nonce":3725629862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/4276801396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4276801396},{"type":"v1","addr":"172.21.15.138:6835","nonce":4276801396}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6833/2224910879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2224910879},{"type":"v1","addr":"172.21.15.192:6833","nonce":2224910879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.138:6837/2540335374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2540335374},{"type":"v1","addr":"172.21.15.138:6837","nonce":2540335374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24616":{"gid":24616,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.192:6835/3530688860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3530688860},{"type":"v1","addr":"172.21.15.192:6835","nonce":3530688860}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.192:6837/2683860842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2683860842},{"type":"v1","addr":"172.21.15.192:6837","nonce":2683860842}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.192:6839/1727623231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1727623231},{"type":"v1","addr":"172.21.15.192:6839","nonce":1727623231}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/3230372576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":3230372576},{"type":"v1","addr":"172.21.15.138:6839","nonce":3230372576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24638,24637,24616,24628,24602]},"id":1}]} 2024-09-21T05:56:52.364 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 26 2024-09-21T05:56:52.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.363+0000 7f9643c61640 1 -- 172.21.15.117:0/2424124526 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f96240b6670 msgr2=0x7f96240b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:52.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.363+0000 7f9643c61640 1 --2- 172.21.15.117:0/2424124526 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f96240b6670 0x7f96240b8b80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9630009230 tx=0x7f9630000bc0 comp rx=0 tx=0).stop 2024-09-21T05:56:52.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.363+0000 7f9643c61640 1 -- 172.21.15.117:0/2424124526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f963c1049b0 msgr2=0x7f963c19a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:52.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.363+0000 7f9643c61640 1 --2- 172.21.15.117:0/2424124526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f963c1049b0 0x7f963c19a5e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f962c00b9c0 tx=0x7f962c00be80 comp rx=0 tx=0).stop 2024-09-21T05:56:52.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.363+0000 7f9643c61640 1 -- 172.21.15.117:0/2424124526 shutdown_connections 2024-09-21T05:56:52.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.363+0000 7f9643c61640 1 --2- 172.21.15.117:0/2424124526 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f96240b6670 0x7f96240b8b80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:52.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.363+0000 7f9643c61640 1 --2- 172.21.15.117:0/2424124526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f963c10de10 0x7f963c19fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:52.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.363+0000 7f9643c61640 1 --2- 172.21.15.117:0/2424124526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f963c105300 0x7f963c19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:52.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.363+0000 7f9643c61640 1 --2- 172.21.15.117:0/2424124526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f963c1049b0 0x7f963c19a5e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:52.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.363+0000 7f9643c61640 1 -- 172.21.15.117:0/2424124526 >> 172.21.15.117:0/2424124526 conn(0x7f963c079730 msgr2=0x7f963c1008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:52.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.364+0000 7f9643c61640 1 -- 172.21.15.117:0/2424124526 shutdown_connections 2024-09-21T05:56:52.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:52.364+0000 7f9643c61640 1 -- 172.21.15.117:0/2424124526 wait complete. 2024-09-21T05:56:52.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:52 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2424124526' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:56:53.014 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:52 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2424124526' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:56:53.043 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T05:56:53.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:52 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2424124526' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:56:53.292 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:53.768 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:53 smithi117 ceph-mon[30947]: pgmap v314: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T05:56:53.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.856+0000 7f07da1f6640 1 -- 172.21.15.117:0/4218672428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07d4108d00 msgr2=0x7f07d410f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:53.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.856+0000 7f07da1f6640 1 --2- 172.21.15.117:0/4218672428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07d4108d00 0x7f07d410f530 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f07cc009ca0 tx=0x7f07cc02a8e0 comp rx=0 tx=0).stop 2024-09-21T05:56:53.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.856+0000 7f07da1f6640 1 -- 172.21.15.117:0/4218672428 shutdown_connections 2024-09-21T05:56:53.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.856+0000 7f07da1f6640 1 --2- 172.21.15.117:0/4218672428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07d4108d00 0x7f07d410f530 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:53.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.856+0000 7f07da1f6640 1 --2- 172.21.15.117:0/4218672428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07d41083b0 0x7f07d41087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:53.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.856+0000 7f07da1f6640 1 --2- 172.21.15.117:0/4218672428 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07d4102a70 0x7f07d4102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:53.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.856+0000 7f07da1f6640 1 -- 172.21.15.117:0/4218672428 >> 172.21.15.117:0/4218672428 conn(0x7f07d4079710 msgr2=0x7f07d4079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:53.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.856+0000 7f07da1f6640 1 -- 172.21.15.117:0/4218672428 shutdown_connections 2024-09-21T05:56:53.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.856+0000 7f07da1f6640 1 -- 172.21.15.117:0/4218672428 wait complete. 2024-09-21T05:56:53.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.856+0000 7f07da1f6640 1 Processor -- start 2024-09-21T05:56:53.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.856+0000 7f07da1f6640 1 -- start start 2024-09-21T05:56:53.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.857+0000 7f07da1f6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07d4102a70 0x7f07d419c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:53.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.857+0000 7f07da1f6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07d41083b0 0x7f07d419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:53.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.857+0000 7f07da1f6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07d4108d00 0x7f07d41a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:53.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.857+0000 7f07da1f6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f07d4112a00 con 0x7f07d41083b0 2024-09-21T05:56:53.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.857+0000 7f07da1f6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f07d4112880 con 0x7f07d4108d00 2024-09-21T05:56:53.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.857+0000 7f07da1f6640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f07d4112b80 con 0x7f07d4102a70 2024-09-21T05:56:53.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.857+0000 7f07d2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07d41083b0 0x7f07d419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:53.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.857+0000 7f07d2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07d41083b0 0x7f07d419cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51016/0 (socket says 172.21.15.117:51016) 2024-09-21T05:56:53.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.857+0000 7f07d2ffd640 1 -- 172.21.15.117:0/1991634648 learned_addr learned my addr 172.21.15.117:0/1991634648 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:53.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.857+0000 7f07d3fff640 1 --2- 172.21.15.117:0/1991634648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07d4108d00 0x7f07d41a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:53.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.857+0000 7f07d37fe640 1 --2- 172.21.15.117:0/1991634648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07d4102a70 0x7f07d419c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:53.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.857+0000 7f07d2ffd640 1 -- 172.21.15.117:0/1991634648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07d4102a70 msgr2=0x7f07d419c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:53.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.857+0000 7f07d2ffd640 1 --2- 172.21.15.117:0/1991634648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07d4102a70 0x7f07d419c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:53.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.857+0000 7f07d2ffd640 1 -- 172.21.15.117:0/1991634648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07d4108d00 msgr2=0x7f07d41a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:53.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.857+0000 7f07d2ffd640 1 --2- 172.21.15.117:0/1991634648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07d4108d00 0x7f07d41a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:53.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.857+0000 7f07d2ffd640 1 -- 172.21.15.117:0/1991634648 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07d41a2560 con 0x7f07d41083b0 2024-09-21T05:56:53.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.857+0000 7f07d2ffd640 1 --2- 172.21.15.117:0/1991634648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07d41083b0 0x7f07d419cc50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f07c8006fd0 tx=0x7f07c800b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:53.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.858+0000 7f07d0ff9640 1 -- 172.21.15.117:0/1991634648 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c8015020 con 0x7f07d41083b0 2024-09-21T05:56:53.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.858+0000 7f07da1f6640 1 -- 172.21.15.117:0/1991634648 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07d41a2850 con 0x7f07d41083b0 2024-09-21T05:56:53.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.858+0000 7f07d0ff9640 1 -- 172.21.15.117:0/1991634648 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07c8011ce0 con 0x7f07d41083b0 2024-09-21T05:56:53.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.858+0000 7f07d0ff9640 1 -- 172.21.15.117:0/1991634648 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c8010d90 con 0x7f07d41083b0 2024-09-21T05:56:53.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.858+0000 7f07da1f6640 1 -- 172.21.15.117:0/1991634648 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07d41a9070 con 0x7f07d41083b0 2024-09-21T05:56:53.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.859+0000 7f07da1f6640 1 -- 172.21.15.117:0/1991634648 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07d4103cf0 con 0x7f07d41083b0 2024-09-21T05:56:53.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.861+0000 7f07d0ff9640 1 -- 172.21.15.117:0/1991634648 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f07c8018020 con 0x7f07d41083b0 2024-09-21T05:56:53.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.862+0000 7f07d0ff9640 1 --2- 172.21.15.117:0/1991634648 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f07bc0b6620 0x7f07bc0b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:53.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.862+0000 7f07d37fe640 1 --2- 172.21.15.117:0/1991634648 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f07bc0b6620 0x7f07bc0b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:53.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.862+0000 7f07d0ff9640 1 -- 172.21.15.117:0/1991634648 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f07c80e71b0 con 0x7f07d41083b0 2024-09-21T05:56:53.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.862+0000 7f07d37fe640 1 --2- 172.21.15.117:0/1991634648 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f07bc0b6620 0x7f07bc0b8b30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f07c400abe0 tx=0x7f07c400a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:53.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:53.866+0000 7f07d0ff9640 1 -- 172.21.15.117:0/1991634648 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f07c801f020 con 0x7f07d41083b0 2024-09-21T05:56:53.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:53 smithi192 ceph-mon[34700]: pgmap v314: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T05:56:54.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:54.046+0000 7f07da1f6640 1 -- 172.21.15.117:0/1991634648 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f07d4102e80 con 0x7f07d41083b0 2024-09-21T05:56:54.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:54.048+0000 7f07d0ff9640 1 -- 172.21.15.117:0/1991634648 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+23066 (secure 0 0 0) 0x7f07d4102e80 con 0x7f07d41083b0 2024-09-21T05:56:54.051 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:56:54.051 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":76,"fsid":"ac8581a2-77dc-11ef-baf5-efdc52797490","created":"2024-09-21T05:46:47.021846+0000","modified":"2024-09-21T05:55:33.628948+0000","last_up_change":"2024-09-21T05:54:21.572849+0000","last_in_change":"2024-09-21T05:53:58.892597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:50:32.717389+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:55:10.604159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T05:55:12.581258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"68315502-fa6c-4a6e-a2d4-a7dde8071c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6803","nonce":3761478035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6805","nonce":3761478035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6809","nonce":3761478035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6807","nonce":3761478035}]},"public_addr":"172.21.15.117:6803/3761478035","cluster_addr":"172.21.15.117:6805/3761478035","heartbeat_back_addr":"172.21.15.117:6809/3761478035","heartbeat_front_addr":"172.21.15.117:6807/3761478035","state":["exists","up"]},{"osd":1,"uuid":"d8910a08-4ef3-49f2-a4db-e3588d85d12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6801","nonce":1737365588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6803","nonce":1737365588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6807","nonce":1737365588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6805","nonce":1737365588}]},"public_addr":"172.21.15.138:6801/1737365588","cluster_addr":"172.21.15.138:6803/1737365588","heartbeat_back_addr":"172.21.15.138:6807/1737365588","heartbeat_front_addr":"172.21.15.138:6805/1737365588","state":["exists","up"]},{"osd":2,"uuid":"94972378-a11f-4129-9e40-ad3fdec1d189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6801","nonce":3002534992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6803","nonce":3002534992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6807","nonce":3002534992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6805","nonce":3002534992}]},"public_addr":"172.21.15.192:6801/3002534992","cluster_addr":"172.21.15.192:6803/3002534992","heartbeat_back_addr":"172.21.15.192:6807/3002534992","heartbeat_front_addr":"172.21.15.192:6805/3002534992","state":["exists","up"]},{"osd":3,"uuid":"d4168c43-f0de-4445-945c-92d1cf270c33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6811","nonce":1222400293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6813","nonce":1222400293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6817","nonce":1222400293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6815","nonce":1222400293}]},"public_addr":"172.21.15.117:6811/1222400293","cluster_addr":"172.21.15.117:6813/1222400293","heartbeat_back_addr":"172.21.15.117:6817/1222400293","heartbeat_front_addr":"172.21.15.117:6815/1222400293","state":["exists","up"]},{"osd":4,"uuid":"28f1d7c0-8392-4a33-85c0-0de611cdd4b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6809","nonce":4294892870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6811","nonce":4294892870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6815","nonce":4294892870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6813","nonce":4294892870}]},"public_addr":"172.21.15.138:6809/4294892870","cluster_addr":"172.21.15.138:6811/4294892870","heartbeat_back_addr":"172.21.15.138:6815/4294892870","heartbeat_front_addr":"172.21.15.138:6813/4294892870","state":["exists","up"]},{"osd":5,"uuid":"ca902407-45e8-4bc5-bdbd-a729c1de8322","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6809","nonce":1776840019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6811","nonce":1776840019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6815","nonce":1776840019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6813","nonce":1776840019}]},"public_addr":"172.21.15.192:6809/1776840019","cluster_addr":"172.21.15.192:6811/1776840019","heartbeat_back_addr":"172.21.15.192:6815/1776840019","heartbeat_front_addr":"172.21.15.192:6813/1776840019","state":["exists","up"]},{"osd":6,"uuid":"6e507a91-ed09-4a33-bf0e-c4b593b3c5aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6819","nonce":1093341590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6821","nonce":1093341590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6825","nonce":1093341590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6823","nonce":1093341590}]},"public_addr":"172.21.15.117:6819/1093341590","cluster_addr":"172.21.15.117:6821/1093341590","heartbeat_back_addr":"172.21.15.117:6825/1093341590","heartbeat_front_addr":"172.21.15.117:6823/1093341590","state":["exists","up"]},{"osd":7,"uuid":"36a47dd7-67f7-4ebe-8759-3a8bb6bd2ea3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6817","nonce":1448179714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6819","nonce":1448179714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6823","nonce":1448179714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6821","nonce":1448179714}]},"public_addr":"172.21.15.138:6817/1448179714","cluster_addr":"172.21.15.138:6819/1448179714","heartbeat_back_addr":"172.21.15.138:6823/1448179714","heartbeat_front_addr":"172.21.15.138:6821/1448179714","state":["exists","up"]},{"osd":8,"uuid":"118e0b5d-2619-4512-94c5-8d07a0d3a28f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6817","nonce":3960902885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6819","nonce":3960902885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6823","nonce":3960902885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6821","nonce":3960902885}]},"public_addr":"172.21.15.192:6817/3960902885","cluster_addr":"172.21.15.192:6819/3960902885","heartbeat_back_addr":"172.21.15.192:6823/3960902885","heartbeat_front_addr":"172.21.15.192:6821/3960902885","state":["exists","up"]},{"osd":9,"uuid":"47543bc6-fa0b-4e72-9742-97dcda0b7f8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6827","nonce":3453855195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6829","nonce":3453855195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6833","nonce":3453855195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6831","nonce":3453855195}]},"public_addr":"172.21.15.117:6827/3453855195","cluster_addr":"172.21.15.117:6829/3453855195","heartbeat_back_addr":"172.21.15.117:6833/3453855195","heartbeat_front_addr":"172.21.15.117:6831/3453855195","state":["exists","up"]},{"osd":10,"uuid":"5e6708b4-7a85-4fa5-a03c-27d99f7d749f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6825","nonce":903392451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6827","nonce":903392451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6831","nonce":903392451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6829","nonce":903392451}]},"public_addr":"172.21.15.138:6825/903392451","cluster_addr":"172.21.15.138:6827/903392451","heartbeat_back_addr":"172.21.15.138:6831/903392451","heartbeat_front_addr":"172.21.15.138:6829/903392451","state":["exists","up"]},{"osd":11,"uuid":"24bc2ef7-e2b4-43c6-9e6c-43ae02390415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6825","nonce":899090479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6827","nonce":899090479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6831","nonce":899090479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6829","nonce":899090479}]},"public_addr":"172.21.15.192:6825/899090479","cluster_addr":"172.21.15.192:6827/899090479","heartbeat_back_addr":"172.21.15.192:6831/899090479","heartbeat_front_addr":"172.21.15.192:6829/899090479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:49:40.742217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:05.963655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:30.164495+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:56.598899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:51:21.940331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:51:47.777445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:52:12.869007+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:52:39.454054+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:03.569226+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:29.760863+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:55.788604+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:54:20.510184+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:0/2989394419":"2024-09-22T05:48:32.406654+0000","172.21.15.117:6801/1702676172":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/1766406059":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6800/2337036514":"2024-09-22T05:47:09.112435+0000","172.21.15.117:0/985798343":"2024-09-22T05:47:09.112435+0000","172.21.15.117:0/1083273222":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/473284565":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6801/2337036514":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6800/816635165":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/2642940197":"2024-09-22T05:48:32.406654+0000","172.21.15.117:6801/816635165":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/2225537433":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/3651584169":"2024-09-22T05:47:33.789146+0000","172.21.15.117:6800/1702676172":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/1284403882":"2024-09-22T05:47:33.789146+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:56:54.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:54.050+0000 7f07da1f6640 1 -- 172.21.15.117:0/1991634648 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f07bc0b6620 msgr2=0x7f07bc0b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:54.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:54.050+0000 7f07da1f6640 1 --2- 172.21.15.117:0/1991634648 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f07bc0b6620 0x7f07bc0b8b30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f07c400abe0 tx=0x7f07c400a2d0 comp rx=0 tx=0).stop 2024-09-21T05:56:54.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:54.050+0000 7f07da1f6640 1 -- 172.21.15.117:0/1991634648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07d41083b0 msgr2=0x7f07d419cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:54.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:54.050+0000 7f07da1f6640 1 --2- 172.21.15.117:0/1991634648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07d41083b0 0x7f07d419cc50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f07c8006fd0 tx=0x7f07c800b820 comp rx=0 tx=0).stop 2024-09-21T05:56:54.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:54.050+0000 7f07da1f6640 1 -- 172.21.15.117:0/1991634648 shutdown_connections 2024-09-21T05:56:54.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:54.050+0000 7f07da1f6640 1 --2- 172.21.15.117:0/1991634648 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f07bc0b6620 0x7f07bc0b8b30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:54.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:54.050+0000 7f07da1f6640 1 --2- 172.21.15.117:0/1991634648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07d41083b0 0x7f07d419cc50 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:54.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:54.050+0000 7f07da1f6640 1 --2- 172.21.15.117:0/1991634648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07d4108d00 0x7f07d41a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:54.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:54.050+0000 7f07da1f6640 1 --2- 172.21.15.117:0/1991634648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07d4102a70 0x7f07d419c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:54.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:54.050+0000 7f07da1f6640 1 -- 172.21.15.117:0/1991634648 >> 172.21.15.117:0/1991634648 conn(0x7f07d4079710 msgr2=0x7f07d4106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:54.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:54.051+0000 7f07da1f6640 1 -- 172.21.15.117:0/1991634648 shutdown_connections 2024-09-21T05:56:54.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:54.051+0000 7f07da1f6640 1 -- 172.21.15.117:0/1991634648 wait complete. 2024-09-21T05:56:54.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:53 smithi138 ceph-mon[34391]: pgmap v314: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T05:56:54.699 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:54 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/1991634648' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T05:56:54.727 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T05:55:14.555897+0000', 'modified': '2024-09-21T05:55:26.209411+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24638, 'mds_1': 24637, 'mds_2': 24628, 'mds_3': 24616, 'mds_4': 24602}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14838': {'gid': 14838, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.117:6839/2314572411', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6838', 'nonce': 2314572411}, {'type': 'v1', 'addr': '172.21.15.117:6839', 'nonce': 2314572411}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14850': {'gid': 14850, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.117:6841/3378096561', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6840', 'nonce': 3378096561}, {'type': 'v1', 'addr': '172.21.15.117:6841', 'nonce': 3378096561}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24545': {'gid': 24545, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.138:6833/3725629862', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6832', 'nonce': 3725629862}, {'type': 'v1', 'addr': '172.21.15.138:6833', 'nonce': 3725629862}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24572': {'gid': 24572, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.138:6835/4276801396', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6834', 'nonce': 4276801396}, {'type': 'v1', 'addr': '172.21.15.138:6835', 'nonce': 4276801396}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24586': {'gid': 24586, 'name': 'l', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.192:6833/2224910879', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6832', 'nonce': 2224910879}, {'type': 'v1', 'addr': '172.21.15.192:6833', 'nonce': 2224910879}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24602': {'gid': 24602, 'name': 'h', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.138:6837/2540335374', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6836', 'nonce': 2540335374}, {'type': 'v1', 'addr': '172.21.15.138:6837', 'nonce': 2540335374}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24616': {'gid': 24616, 'name': 'f', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.192:6835/3530688860', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6834', 'nonce': 3530688860}, {'type': 'v1', 'addr': '172.21.15.192:6835', 'nonce': 3530688860}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24628': {'gid': 24628, 'name': 'c', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.192:6837/2683860842', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6836', 'nonce': 2683860842}, {'type': 'v1', 'addr': '172.21.15.192:6837', 'nonce': 2683860842}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24637': {'gid': 24637, 'name': 'i', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.192:6839/1727623231', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6838', 'nonce': 1727623231}, {'type': 'v1', 'addr': '172.21.15.192:6839', 'nonce': 1727623231}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24638': {'gid': 24638, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.138:6839/3230372576', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6838', 'nonce': 3230372576}, {'type': 'v1', 'addr': '172.21.15.138:6839', 'nonce': 3230372576}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24638, 'qdb_cluster': [24638, 24637, 24616, 24628, 24602]} 2024-09-21T05:56:54.727 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-21T05:56:54.727 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.h status 2024-09-21T05:56:54.973 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:54.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:54 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/1991634648' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T05:56:55.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:54 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/1991634648' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T05:56:55.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.465+0000 7f5b52408640 1 -- 172.21.15.117:0/2167888171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b4c1083d0 msgr2=0x7f5b4c1087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:55.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.465+0000 7f5b52408640 1 --2- 172.21.15.117:0/2167888171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b4c1083d0 0x7f5b4c1087e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5b40009900 tx=0x7f5b4002a3f0 comp rx=0 tx=0).stop 2024-09-21T05:56:55.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.465+0000 7f5b52408640 1 -- 172.21.15.117:0/2167888171 shutdown_connections 2024-09-21T05:56:55.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.465+0000 7f5b52408640 1 --2- 172.21.15.117:0/2167888171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b4c1083d0 0x7f5b4c1087e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:55.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.465+0000 7f5b52408640 1 --2- 172.21.15.117:0/2167888171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b4c108d20 0x7f5b4c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:55.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.465+0000 7f5b52408640 1 --2- 172.21.15.117:0/2167888171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b4c102ae0 0x7f5b4c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:55.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.465+0000 7f5b52408640 1 -- 172.21.15.117:0/2167888171 >> 172.21.15.117:0/2167888171 conn(0x7f5b4c079760 msgr2=0x7f5b4c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:55.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.465+0000 7f5b52408640 1 -- 172.21.15.117:0/2167888171 shutdown_connections 2024-09-21T05:56:55.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.465+0000 7f5b52408640 1 -- 172.21.15.117:0/2167888171 wait complete. 2024-09-21T05:56:55.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.466+0000 7f5b52408640 1 Processor -- start 2024-09-21T05:56:55.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.466+0000 7f5b52408640 1 -- start start 2024-09-21T05:56:55.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.466+0000 7f5b52408640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b4c102ae0 0x7f5b4c19b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:55.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.466+0000 7f5b52408640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b4c1083d0 0x7f5b4c19b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:55.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.466+0000 7f5b52408640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b4c108d20 0x7f5b4c19bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:55.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.466+0000 7f5b52408640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c112990 con 0x7f5b4c108d20 2024-09-21T05:56:55.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.466+0000 7f5b52408640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c112810 con 0x7f5b4c1083d0 2024-09-21T05:56:55.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.466+0000 7f5b52408640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c112b10 con 0x7f5b4c102ae0 2024-09-21T05:56:55.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.466+0000 7f5b509a5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b4c108d20 0x7f5b4c19bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:55.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.466+0000 7f5b509a5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b4c108d20 0x7f5b4c19bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44448/0 (socket says 172.21.15.117:44448) 2024-09-21T05:56:55.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.466+0000 7f5b509a5640 1 -- 172.21.15.117:0/853891814 learned_addr learned my addr 172.21.15.117:0/853891814 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:55.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.466+0000 7f5b4bfff640 1 --2- 172.21.15.117:0/853891814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b4c102ae0 0x7f5b4c19b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:55.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.467+0000 7f5b509a5640 1 -- 172.21.15.117:0/853891814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b4c102ae0 msgr2=0x7f5b4c19b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:55.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.467+0000 7f5b509a5640 1 --2- 172.21.15.117:0/853891814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b4c102ae0 0x7f5b4c19b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:55.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.467+0000 7f5b4b7fe640 1 --2- 172.21.15.117:0/853891814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b4c1083d0 0x7f5b4c19b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:55.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.467+0000 7f5b509a5640 1 -- 172.21.15.117:0/853891814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b4c1083d0 msgr2=0x7f5b4c19b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:55.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.467+0000 7f5b509a5640 1 --2- 172.21.15.117:0/853891814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b4c1083d0 0x7f5b4c19b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:55.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.467+0000 7f5b509a5640 1 -- 172.21.15.117:0/853891814 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4c1a63f0 con 0x7f5b4c108d20 2024-09-21T05:56:55.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.467+0000 7f5b4bfff640 1 --2- 172.21.15.117:0/853891814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b4c102ae0 0x7f5b4c19b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:56:55.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.467+0000 7f5b509a5640 1 --2- 172.21.15.117:0/853891814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b4c108d20 0x7f5b4c19bc40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5b4400cce0 tx=0x7f5b4400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:55.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.467+0000 7f5b497fa640 1 -- 172.21.15.117:0/853891814 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b4400f450 con 0x7f5b4c108d20 2024-09-21T05:56:55.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.467+0000 7f5b52408640 1 -- 172.21.15.117:0/853891814 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b4c1a66e0 con 0x7f5b4c108d20 2024-09-21T05:56:55.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.467+0000 7f5b497fa640 1 -- 172.21.15.117:0/853891814 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b4400faa0 con 0x7f5b4c108d20 2024-09-21T05:56:55.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.467+0000 7f5b497fa640 1 -- 172.21.15.117:0/853891814 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b44018b70 con 0x7f5b4c108d20 2024-09-21T05:56:55.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.467+0000 7f5b52408640 1 -- 172.21.15.117:0/853891814 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b4c1a6c20 con 0x7f5b4c108d20 2024-09-21T05:56:55.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.470+0000 7f5b5897c740 1 Processor -- start 2024-09-21T05:56:55.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.470+0000 7f5b497fa640 1 -- 172.21.15.117:0/853891814 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f5b44018d10 con 0x7f5b4c108d20 2024-09-21T05:56:55.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.470+0000 7f5b5897c740 1 -- start start 2024-09-21T05:56:55.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.470+0000 7f5b5897c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0deddef60 0x55e0df184f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:55.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.470+0000 7f5b5897c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0dedb60a0 0x55e0df185440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:55.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.470+0000 7f5b5897c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0dedbe280 0x55e0df185980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:55.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.470+0000 7f5b5897c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e0df0fcc10 con 0x55e0dedb60a0 2024-09-21T05:56:55.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.470+0000 7f5b5897c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e0df0ec040 con 0x55e0dedbe280 2024-09-21T05:56:55.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.470+0000 7f5b5897c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55e0df0efe50 con 0x55e0deddef60 2024-09-21T05:56:55.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.470+0000 7f5b4bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0dedb60a0 0x55e0df185440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:55.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.470+0000 7f5b4bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0dedb60a0 0x55e0df185440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44454/0 (socket says 172.21.15.117:44454) 2024-09-21T05:56:55.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.470+0000 7f5b4bfff640 1 -- 172.21.15.117:0/601493715 learned_addr learned my addr 172.21.15.117:0/601493715 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:55.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.471+0000 7f5b4b7fe640 1 --2- 172.21.15.117:0/601493715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0deddef60 0x55e0df184f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:55.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.471+0000 7f5b4bfff640 1 -- 172.21.15.117:0/601493715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0deddef60 msgr2=0x55e0df184f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:55.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.471+0000 7f5b4bfff640 1 --2- 172.21.15.117:0/601493715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0deddef60 0x55e0df184f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:55.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.471+0000 7f5b4b7fe640 1 --2- 172.21.15.117:0/601493715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0dedbe280 0x55e0df185980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:55.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.471+0000 7f5b4bfff640 1 -- 172.21.15.117:0/601493715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0dedbe280 msgr2=0x55e0df185980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:55.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.471+0000 7f5b4bfff640 1 --2- 172.21.15.117:0/601493715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0dedbe280 0x55e0df185980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:55.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.471+0000 7f5b4bfff640 1 -- 172.21.15.117:0/601493715 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0dee4a540 con 0x55e0dedb60a0 2024-09-21T05:56:55.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.471+0000 7f5b497fa640 1 --2- 172.21.15.117:0/853891814 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5b340b65f0 0x7f5b340b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:55.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.471+0000 7f5b4b7fe640 1 --2- 172.21.15.117:0/853891814 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5b340b65f0 0x7f5b340b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:55.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.471+0000 7f5b4bfff640 1 --2- 172.21.15.117:0/601493715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0dedb60a0 0x55e0df185440 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c008c30 tx=0x7f5b3c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba50ccf93469fbc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:55.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.471+0000 7f5b497fa640 1 -- 172.21.15.117:0/853891814 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f5b440e7af0 con 0x7f5b4c108d20 2024-09-21T05:56:55.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.471+0000 7f5b327fc640 1 -- 172.21.15.117:0/601493715 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c014070 con 0x55e0dedb60a0 2024-09-21T05:56:55.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.471+0000 7f5b327fc640 1 -- 172.21.15.117:0/601493715 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b3c0047a0 con 0x55e0dedb60a0 2024-09-21T05:56:55.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.471+0000 7f5b327fc640 1 -- 172.21.15.117:0/601493715 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c010670 con 0x55e0dedb60a0 2024-09-21T05:56:55.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.472+0000 7f5b5897c740 1 -- 172.21.15.117:0/601493715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0dedb60a0 msgr2=0x55e0df185440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:55.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.472+0000 7f5b5897c740 1 --2- 172.21.15.117:0/601493715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0dedb60a0 0x55e0df185440 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c008c30 tx=0x7f5b3c00e720 comp rx=0 tx=0).stop 2024-09-21T05:56:55.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.472+0000 7f5b4b7fe640 1 --2- 172.21.15.117:0/853891814 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5b340b65f0 0x7f5b340b8b00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5b400096c0 tx=0x7f5b40005fd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:55.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.472+0000 7f5b5897c740 1 -- 172.21.15.117:0/601493715 shutdown_connections 2024-09-21T05:56:55.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.472+0000 7f5b5897c740 1 --2- 172.21.15.117:0/601493715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0dedb60a0 0x55e0df185440 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:55.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.472+0000 7f5b5897c740 1 --2- 172.21.15.117:0/601493715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0dedbe280 0x55e0df185980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:55.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.472+0000 7f5b5897c740 1 --2- 172.21.15.117:0/601493715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0deddef60 0x55e0df184f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:55.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.472+0000 7f5b5897c740 1 -- 172.21.15.117:0/601493715 >> 172.21.15.117:0/601493715 conn(0x55e0dedd3350 msgr2=0x55e0df182df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:55.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.472+0000 7f5b5897c740 1 -- 172.21.15.117:0/601493715 shutdown_connections 2024-09-21T05:56:55.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.472+0000 7f5b5897c740 1 -- 172.21.15.117:0/601493715 wait complete. 2024-09-21T05:56:55.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.472+0000 7f5b5897c740 1 -- start start 2024-09-21T05:56:55.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.473+0000 7f5b5897c740 1 Processor -- start 2024-09-21T05:56:55.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.473+0000 7f5b5897c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0dedb60a0 0x55e0df190a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:55.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.473+0000 7f5b5897c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0dedbe280 0x55e0df190f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:55.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.473+0000 7f5b5897c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0deddef60 0x55e0df1a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:55.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.473+0000 7f5b5897c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e0def4d5f0 con 0x55e0dedb60a0 2024-09-21T05:56:55.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.473+0000 7f5b5897c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e0df087010 con 0x55e0deddef60 2024-09-21T05:56:55.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.473+0000 7f5b5897c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55e0df197a90 con 0x55e0dedbe280 2024-09-21T05:56:55.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.473+0000 7f5b4bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0dedb60a0 0x55e0df190a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:55.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.473+0000 7f5b4bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0dedb60a0 0x55e0df190a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44462/0 (socket says 172.21.15.117:44462) 2024-09-21T05:56:55.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.473+0000 7f5b5897c740 10 client.0 _ll_register_callbacks cb 0x55e0df0e6340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:56:55.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.473+0000 7f5b4bfff640 1 -- 172.21.15.117:0/1738714102 learned_addr learned my addr 172.21.15.117:0/1738714102 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:55.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.473+0000 7f5b4b7fe640 1 --2- 172.21.15.117:0/1738714102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0dedbe280 0x55e0df190f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:55.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.473+0000 7f5b509a5640 1 --2- 172.21.15.117:0/1738714102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0deddef60 0x55e0df1a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:55.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.473+0000 7f5b4bfff640 1 -- 172.21.15.117:0/1738714102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0dedbe280 msgr2=0x55e0df190f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:55.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.473+0000 7f5b4bfff640 1 --2- 172.21.15.117:0/1738714102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0dedbe280 0x55e0df190f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:55.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.473+0000 7f5b4bfff640 1 -- 172.21.15.117:0/1738714102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0deddef60 msgr2=0x55e0df1a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:55.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.473+0000 7f5b4bfff640 1 --2- 172.21.15.117:0/1738714102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0deddef60 0x55e0df1a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:55.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.473+0000 7f5b4bfff640 1 -- 172.21.15.117:0/1738714102 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0df0b4a00 con 0x55e0dedb60a0 2024-09-21T05:56:55.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.474+0000 7f5b4bfff640 1 --2- 172.21.15.117:0/1738714102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0dedb60a0 0x55e0df190a40 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c000c00 tx=0x7f5b3c005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba686c22d187f784 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:55.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.474+0000 7f5b2b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:56:55.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.474+0000 7f5b2b7fe640 1 -- 172.21.15.117:0/1738714102 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c027060 con 0x55e0dedb60a0 2024-09-21T05:56:55.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.474+0000 7f5b2b7fe640 1 -- 172.21.15.117:0/1738714102 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f5b3c005380 con 0x55e0dedb60a0 2024-09-21T05:56:55.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.474+0000 7f5b5897c740 1 -- 172.21.15.117:0/1738714102 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b3c0047a0 con 0x55e0dedb60a0 2024-09-21T05:56:55.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.474+0000 7f5b2b7fe640 1 -- 172.21.15.117:0/1738714102 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5b3c014070 con 0x55e0dedb60a0 2024-09-21T05:56:55.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.474+0000 7f5b5897c740 10 client.15366 fetch_fsmap learned FSMap version 26 2024-09-21T05:56:55.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.474+0000 7f5b5897c740 1 -- 172.21.15.117:0/1738714102 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e0df1a3a20 con 0x55e0dedb60a0 2024-09-21T05:56:55.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.474+0000 7f5b2b7fe640 1 -- 172.21.15.117:0/1738714102 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b3c0047a0 con 0x55e0dedb60a0 2024-09-21T05:56:55.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.475+0000 7f5b2b7fe640 1 -- 172.21.15.117:0/1738714102 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f5b3c018430 con 0x55e0dedb60a0 2024-09-21T05:56:55.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.475+0000 7f5b5897c740 10 client.15366 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T05:56:55.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.475+0000 7f5b5897c740 10 client.15366 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T05:56:55.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.475+0000 7f5b5897c740 20 client.15366 populate_metadata read hostname 'smithi117' 2024-09-21T05:56:55.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.475+0000 7f5b5897c740 1 --2- 172.21.15.117:0/1738714102 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x55e0ded9d8e0 0x55e0df1a7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:55.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.475+0000 7f5b5897c740 4 client.15366 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:56:55.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.475+0000 7f5b5897c740 1 -- 172.21.15.117:0/1738714102 --> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0df0feb60 con 0x55e0ded9d8e0 2024-09-21T05:56:55.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.475+0000 7f5b4b7fe640 1 --2- 172.21.15.117:0/1738714102 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x55e0ded9d8e0 0x55e0df1a7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:55.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.476+0000 7f5b2b7fe640 10 client.15366 ms_handle_connect on v2:172.21.15.138:6836/2540335374 2024-09-21T05:56:55.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.476+0000 7f5b4b7fe640 1 --2- 172.21.15.117:0/1738714102 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x55e0ded9d8e0 0x55e0df1a7040 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=71a382f30547f00d server_cookie=32688c1011e563a1 in_seq=0 out_seq=0 2024-09-21T05:56:55.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.476+0000 7f5b2b7fe640 1 -- 172.21.15.117:0/1738714102 <== mds.4 v2:172.21.15.138:6836/2540335374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e0df0feb60 con 0x55e0ded9d8e0 2024-09-21T05:56:55.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.476+0000 7f5b2b7fe640 10 client.15366 handle_command_reply: tid=0 2024-09-21T05:56:55.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.476+0000 7f5b5897c740 1 client.15366 shutdown 2024-09-21T05:56:55.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.477+0000 7f5b5897c740 1 -- 172.21.15.117:0/1738714102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0dedb60a0 msgr2=0x55e0df190a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:55.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.477+0000 7f5b5897c740 1 --2- 172.21.15.117:0/1738714102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0dedb60a0 0x55e0df190a40 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c000c00 tx=0x7f5b3c005000 comp rx=0 tx=0).stop 2024-09-21T05:56:55.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.477+0000 7f5b5897c740 1 -- 172.21.15.117:0/1738714102 shutdown_connections 2024-09-21T05:56:55.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.477+0000 7f5b5897c740 1 --2- 172.21.15.117:0/1738714102 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x55e0ded9d8e0 0x55e0df1a7040 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:55.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.477+0000 7f5b5897c740 1 --2- 172.21.15.117:0/1738714102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0deddef60 0x55e0df1a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:55.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.477+0000 7f5b2b7fe640 0 client.15366 ms_handle_reset on v2:172.21.15.138:6836/2540335374 2024-09-21T05:56:55.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.477+0000 7f5b5897c740 1 --2- 172.21.15.117:0/1738714102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0dedbe280 0x55e0df190f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:55.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.477+0000 7f5b5897c740 1 --2- 172.21.15.117:0/1738714102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0dedb60a0 0x55e0df190a40 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:55.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.477+0000 7f5b5897c740 1 -- 172.21.15.117:0/1738714102 >> 172.21.15.117:0/1738714102 conn(0x55e0dedd3350 msgr2=0x55e0df188090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:55.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.477+0000 7f5b5897c740 1 -- 172.21.15.117:0/1738714102 shutdown_connections 2024-09-21T05:56:55.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.477+0000 7f5b5897c740 1 -- 172.21.15.117:0/1738714102 wait complete. 2024-09-21T05:56:55.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.477+0000 7f5b5897c740 20 client.15366 trim_cache size 0 max 16384 2024-09-21T05:56:55.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.495+0000 7f5b5897c740 1 Processor -- start 2024-09-21T05:56:55.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.495+0000 7f5b5897c740 1 -- start start 2024-09-21T05:56:55.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.496+0000 7f5b5897c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0dedbe280 0x55e0df18b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:55.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.496+0000 7f5b5897c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0deddef60 0x55e0df1a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:55.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.496+0000 7f5b5897c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0ded72c90 0x55e0df1a1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:55.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.496+0000 7f5b5897c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e0df1a5380 con 0x55e0ded72c90 2024-09-21T05:56:55.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.496+0000 7f5b5897c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e0df1a5a50 con 0x55e0dedbe280 2024-09-21T05:56:55.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.496+0000 7f5b5897c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55e0df1a6120 con 0x55e0deddef60 2024-09-21T05:56:55.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.496+0000 7f5b509a5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0ded72c90 0x55e0df1a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:55.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.496+0000 7f5b509a5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0ded72c90 0x55e0df1a1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44472/0 (socket says 172.21.15.117:44472) 2024-09-21T05:56:55.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.496+0000 7f5b509a5640 1 -- 172.21.15.117:0/4251892864 learned_addr learned my addr 172.21.15.117:0/4251892864 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:55.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.496+0000 7f5b4b7fe640 1 --2- 172.21.15.117:0/4251892864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0deddef60 0x55e0df1a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:55.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.496+0000 7f5b4bfff640 1 --2- 172.21.15.117:0/4251892864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0dedbe280 0x55e0df18b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:55.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.496+0000 7f5b509a5640 1 -- 172.21.15.117:0/4251892864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0deddef60 msgr2=0x55e0df1a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:55.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.496+0000 7f5b509a5640 1 --2- 172.21.15.117:0/4251892864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0deddef60 0x55e0df1a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:55.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.496+0000 7f5b509a5640 1 -- 172.21.15.117:0/4251892864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0dedbe280 msgr2=0x55e0df18b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:55.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.496+0000 7f5b509a5640 1 --2- 172.21.15.117:0/4251892864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0dedbe280 0x55e0df18b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:55.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.496+0000 7f5b509a5640 1 -- 172.21.15.117:0/4251892864 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0df187d60 con 0x55e0ded72c90 2024-09-21T05:56:55.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.496+0000 7f5b509a5640 1 --2- 172.21.15.117:0/4251892864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0ded72c90 0x55e0df1a1230 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5b440ad780 tx=0x7f5b44018610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b832be39361c060 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:55.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.496+0000 7f5b31ffb640 1 -- 172.21.15.117:0/4251892864 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b440b2560 con 0x55e0ded72c90 2024-09-21T05:56:55.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.496+0000 7f5b31ffb640 1 -- 172.21.15.117:0/4251892864 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b440a9050 con 0x55e0ded72c90 2024-09-21T05:56:55.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.496+0000 7f5b31ffb640 1 -- 172.21.15.117:0/4251892864 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b440b2330 con 0x55e0ded72c90 2024-09-21T05:56:55.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.497+0000 7f5b5897c740 1 -- 172.21.15.117:0/4251892864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0ded72c90 msgr2=0x55e0df1a1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:55.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.497+0000 7f5b5897c740 1 --2- 172.21.15.117:0/4251892864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0ded72c90 0x55e0df1a1230 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5b440ad780 tx=0x7f5b44018610 comp rx=0 tx=0).stop 2024-09-21T05:56:55.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.497+0000 7f5b5897c740 1 -- 172.21.15.117:0/4251892864 shutdown_connections 2024-09-21T05:56:55.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.497+0000 7f5b5897c740 1 --2- 172.21.15.117:0/4251892864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0ded72c90 0x55e0df1a1230 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:55.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.497+0000 7f5b5897c740 1 --2- 172.21.15.117:0/4251892864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0deddef60 0x55e0df1a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:55.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.497+0000 7f5b5897c740 1 --2- 172.21.15.117:0/4251892864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0dedbe280 0x55e0df18b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:55.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.497+0000 7f5b5897c740 1 -- 172.21.15.117:0/4251892864 >> 172.21.15.117:0/4251892864 conn(0x55e0dedd3350 msgr2=0x55e0df18b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:55.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.497+0000 7f5b5897c740 1 -- 172.21.15.117:0/4251892864 shutdown_connections 2024-09-21T05:56:55.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.497+0000 7f5b5897c740 1 -- 172.21.15.117:0/4251892864 wait complete. 2024-09-21T05:56:55.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.498+0000 7f5b5897c740 1 -- start start 2024-09-21T05:56:55.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.498+0000 7f5b5897c740 1 Processor -- start 2024-09-21T05:56:55.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.498+0000 7f5b5897c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0ded72c90 0x55e0df1a92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:55.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.498+0000 7f5b5897c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0dedbe280 0x55e0df1a9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:55.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.498+0000 7f5b5897c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0deddef60 0x55e0df1a9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:55.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.498+0000 7f5b5897c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e0df1a6ec0 con 0x55e0deddef60 2024-09-21T05:56:55.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.498+0000 7f5b5897c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e0df1a67f0 con 0x55e0dedbe280 2024-09-21T05:56:55.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.498+0000 7f5b509a5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0deddef60 0x55e0df1a9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:55.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.498+0000 7f5b5897c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55e0df1a7590 con 0x55e0ded72c90 2024-09-21T05:56:55.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.498+0000 7f5b509a5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0deddef60 0x55e0df1a9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44482/0 (socket says 172.21.15.117:44482) 2024-09-21T05:56:55.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.498+0000 7f5b509a5640 1 -- 172.21.15.117:0/1143726794 learned_addr learned my addr 172.21.15.117:0/1143726794 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:55.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.498+0000 7f5b5897c740 10 client.0 _ll_register_callbacks cb 0x7f5b3c008ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:56:55.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.499+0000 7f5b4bfff640 1 --2- 172.21.15.117:0/1143726794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0ded72c90 0x55e0df1a92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:55.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.499+0000 7f5b4b7fe640 1 --2- 172.21.15.117:0/1143726794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0dedbe280 0x55e0df1a9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:55.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.499+0000 7f5b509a5640 1 -- 172.21.15.117:0/1143726794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0ded72c90 msgr2=0x55e0df1a92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:55.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.499+0000 7f5b509a5640 1 --2- 172.21.15.117:0/1143726794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0ded72c90 0x55e0df1a92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:55.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.499+0000 7f5b509a5640 1 -- 172.21.15.117:0/1143726794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0dedbe280 msgr2=0x55e0df1a9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:55.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.499+0000 7f5b509a5640 1 --2- 172.21.15.117:0/1143726794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0dedbe280 0x55e0df1a9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:55.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.499+0000 7f5b509a5640 1 -- 172.21.15.117:0/1143726794 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0df1aea90 con 0x55e0deddef60 2024-09-21T05:56:55.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.499+0000 7f5b509a5640 1 --2- 172.21.15.117:0/1143726794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0deddef60 0x55e0df1a9d50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5b4400ab20 tx=0x7f5b440aa1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc7f02effb6f09e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:55.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.499+0000 7f5b2b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:56:55.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.499+0000 7f5b2b7fe640 1 -- 172.21.15.117:0/1143726794 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b440c8060 con 0x55e0deddef60 2024-09-21T05:56:55.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.499+0000 7f5b2b7fe640 1 -- 172.21.15.117:0/1143726794 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f5b440b22d0 con 0x55e0deddef60 2024-09-21T05:56:55.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.499+0000 7f5b5897c740 1 -- 172.21.15.117:0/1143726794 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b440a9050 con 0x55e0deddef60 2024-09-21T05:56:55.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.499+0000 7f5b2b7fe640 1 -- 172.21.15.117:0/1143726794 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5b4400f5f0 con 0x55e0deddef60 2024-09-21T05:56:55.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.499+0000 7f5b5897c740 10 client.15372 fetch_fsmap learned FSMap version 26 2024-09-21T05:56:55.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.499+0000 7f5b5897c740 1 -- 172.21.15.117:0/1143726794 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e0df212790 con 0x55e0deddef60 2024-09-21T05:56:55.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.499+0000 7f5b2b7fe640 1 -- 172.21.15.117:0/1143726794 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b440a9050 con 0x55e0deddef60 2024-09-21T05:56:55.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.500+0000 7f5b2b7fe640 1 -- 172.21.15.117:0/1143726794 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f5b440ac750 con 0x55e0deddef60 2024-09-21T05:56:55.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.500+0000 7f5b5897c740 10 client.15372 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T05:56:55.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.500+0000 7f5b5897c740 10 client.15372 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-21T05:56:55.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.500+0000 7f5b5897c740 20 client.15372 populate_metadata read hostname 'smithi117' 2024-09-21T05:56:55.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.500+0000 7f5b5897c740 1 --2- 172.21.15.117:0/1143726794 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x55e0df214680 0x55e0df215af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:55.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.500+0000 7f5b5897c740 4 client.15372 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T05:56:55.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.500+0000 7f5b5897c740 1 -- 172.21.15.117:0/1143726794 --> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] -- command(tid 0: {"prefix": "status"}) -- 0x55e0df0fa610 con 0x55e0df214680 2024-09-21T05:56:55.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.500+0000 7f5b4bfff640 1 --2- 172.21.15.117:0/1143726794 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x55e0df214680 0x55e0df215af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:55.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.501+0000 7f5b2b7fe640 10 client.15372 ms_handle_connect on v2:172.21.15.138:6836/2540335374 2024-09-21T05:56:55.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.501+0000 7f5b4bfff640 1 --2- 172.21.15.117:0/1143726794 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x55e0df214680 0x55e0df215af0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=380e40b9ff3ea090 server_cookie=5ef7e4fd4dd751ca in_seq=0 out_seq=0 2024-09-21T05:56:55.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.501+0000 7f5b2b7fe640 1 -- 172.21.15.117:0/1143726794 <== mds.4 v2:172.21.15.138:6836/2540335374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e0df0fa610 con 0x55e0df214680 2024-09-21T05:56:55.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.501+0000 7f5b2b7fe640 10 client.15372 handle_command_reply: tid=0 2024-09-21T05:56:55.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.501+0000 7f5b5897c740 1 client.15372 shutdown 2024-09-21T05:56:55.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.501+0000 7f5b5897c740 1 -- 172.21.15.117:0/1143726794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0deddef60 msgr2=0x55e0df1a9d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:55.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.501+0000 7f5b5897c740 1 --2- 172.21.15.117:0/1143726794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0deddef60 0x55e0df1a9d50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5b4400ab20 tx=0x7f5b440aa1f0 comp rx=0 tx=0).stop 2024-09-21T05:56:55.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.501+0000 7f5b5897c740 1 -- 172.21.15.117:0/1143726794 shutdown_connections 2024-09-21T05:56:55.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.501+0000 7f5b5897c740 1 --2- 172.21.15.117:0/1143726794 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x55e0df214680 0x55e0df215af0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:55.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.501+0000 7f5b5897c740 1 --2- 172.21.15.117:0/1143726794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e0deddef60 0x55e0df1a9d50 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:55.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.501+0000 7f5b5897c740 1 --2- 172.21.15.117:0/1143726794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e0dedbe280 0x55e0df1a9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:55.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.501+0000 7f5b2b7fe640 0 client.15372 ms_handle_reset on v2:172.21.15.138:6836/2540335374 2024-09-21T05:56:55.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.501+0000 7f5b5897c740 1 --2- 172.21.15.117:0/1143726794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e0ded72c90 0x55e0df1a92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:55.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.501+0000 7f5b5897c740 1 -- 172.21.15.117:0/1143726794 >> 172.21.15.117:0/1143726794 conn(0x55e0dedd3350 msgr2=0x55e0df182600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:55.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.501+0000 7f5b5897c740 1 -- 172.21.15.117:0/1143726794 shutdown_connections 2024-09-21T05:56:55.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.501+0000 7f5b5897c740 1 -- 172.21.15.117:0/1143726794 wait complete. 2024-09-21T05:56:55.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.501+0000 7f5b5897c740 20 client.15372 trim_cache size 0 max 16384 2024-09-21T05:56:55.511 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T05:56:55.511 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_fsid": "ac8581a2-77dc-11ef-baf5-efdc52797490", 2024-09-21T05:56:55.511 INFO:teuthology.orchestra.run.smithi117.stdout: "whoami": 4, 2024-09-21T05:56:55.511 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 24602, 2024-09-21T05:56:55.511 INFO:teuthology.orchestra.run.smithi117.stdout: "want_state": "up:active", 2024-09-21T05:56:55.511 INFO:teuthology.orchestra.run.smithi117.stdout: "state": "up:active", 2024-09-21T05:56:55.511 INFO:teuthology.orchestra.run.smithi117.stdout: "fs_name": "cephfs", 2024-09-21T05:56:55.512 INFO:teuthology.orchestra.run.smithi117.stdout: "rank_uptime": 90.297865092999999, 2024-09-21T05:56:55.512 INFO:teuthology.orchestra.run.smithi117.stdout: "mdsmap_epoch": 26, 2024-09-21T05:56:55.512 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch": 76, 2024-09-21T05:56:55.512 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch_barrier": 74, 2024-09-21T05:56:55.512 INFO:teuthology.orchestra.run.smithi117.stdout: "uptime": 123.586069123 2024-09-21T05:56:55.512 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T05:56:55.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.502+0000 7f5b52408640 1 -- 172.21.15.117:0/853891814 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5b340b65f0 msgr2=0x7f5b340b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:55.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.502+0000 7f5b52408640 1 --2- 172.21.15.117:0/853891814 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5b340b65f0 0x7f5b340b8b00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5b400096c0 tx=0x7f5b40005fd0 comp rx=0 tx=0).stop 2024-09-21T05:56:55.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.502+0000 7f5b52408640 1 -- 172.21.15.117:0/853891814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b4c108d20 msgr2=0x7f5b4c19bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:55.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.502+0000 7f5b52408640 1 --2- 172.21.15.117:0/853891814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b4c108d20 0x7f5b4c19bc40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5b4400cce0 tx=0x7f5b4400a620 comp rx=0 tx=0).stop 2024-09-21T05:56:55.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.502+0000 7f5b52408640 1 -- 172.21.15.117:0/853891814 shutdown_connections 2024-09-21T05:56:55.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.502+0000 7f5b52408640 1 --2- 172.21.15.117:0/853891814 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5b340b65f0 0x7f5b340b8b00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:55.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.502+0000 7f5b52408640 1 --2- 172.21.15.117:0/853891814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b4c108d20 0x7f5b4c19bc40 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:55.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.502+0000 7f5b52408640 1 --2- 172.21.15.117:0/853891814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b4c1083d0 0x7f5b4c19b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:55.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.502+0000 7f5b52408640 1 --2- 172.21.15.117:0/853891814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b4c102ae0 0x7f5b4c19b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:55.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.502+0000 7f5b52408640 1 -- 172.21.15.117:0/853891814 >> 172.21.15.117:0/853891814 conn(0x7f5b4c079760 msgr2=0x7f5b4c1043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:55.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.502+0000 7f5b52408640 1 -- 172.21.15.117:0/853891814 shutdown_connections 2024-09-21T05:56:55.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:55.502+0000 7f5b52408640 1 -- 172.21.15.117:0/853891814 wait complete. 2024-09-21T05:56:55.973 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:55 smithi117 ceph-mon[30947]: pgmap v315: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T05:56:55.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:55 smithi192 ceph-mon[34700]: pgmap v315: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T05:56:56.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:55 smithi138 ceph-mon[34391]: pgmap v315: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T05:56:56.420 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.f status 2024-09-21T05:56:56.666 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:57.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.173+0000 7f797e08d640 1 -- 172.21.15.117:0/3929409906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7978103b60 msgr2=0x7f7978103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:57.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.173+0000 7f797e08d640 1 --2- 172.21.15.117:0/3929409906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7978103b60 0x7f7978103f70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7968009900 tx=0x7f796802a410 comp rx=0 tx=0).stop 2024-09-21T05:56:57.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.173+0000 7f797e08d640 1 -- 172.21.15.117:0/3929409906 shutdown_connections 2024-09-21T05:56:57.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.173+0000 7f797e08d640 1 --2- 172.21.15.117:0/3929409906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7978109df0 0x7f7978110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.173+0000 7f797e08d640 1 --2- 172.21.15.117:0/3929409906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79781094a0 0x7f79781098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.173+0000 7f797e08d640 1 --2- 172.21.15.117:0/3929409906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7978103b60 0x7f7978103f70 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.173+0000 7f797e08d640 1 -- 172.21.15.117:0/3929409906 >> 172.21.15.117:0/3929409906 conn(0x7f797807b850 msgr2=0x7f797807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:57.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.173+0000 7f797e08d640 1 -- 172.21.15.117:0/3929409906 shutdown_connections 2024-09-21T05:56:57.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.173+0000 7f797e08d640 1 -- 172.21.15.117:0/3929409906 wait complete. 2024-09-21T05:56:57.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.174+0000 7f797e08d640 1 Processor -- start 2024-09-21T05:56:57.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.174+0000 7f797e08d640 1 -- start start 2024-09-21T05:56:57.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.174+0000 7f797e08d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7978103b60 0x7f797819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:57.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.174+0000 7f797e08d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79781094a0 0x7f797819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:57.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.175+0000 7f797e08d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7978109df0 0x7f797819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:57.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.175+0000 7f797e08d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7978113b10 con 0x7f7978109df0 2024-09-21T05:56:57.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.175+0000 7f797e08d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7978113990 con 0x7f7978103b60 2024-09-21T05:56:57.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.175+0000 7f797e08d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7978113c90 con 0x7f79781094a0 2024-09-21T05:56:57.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.175+0000 7f7977fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7978109df0 0x7f797819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:57.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.175+0000 7f7977fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7978109df0 0x7f797819fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44492/0 (socket says 172.21.15.117:44492) 2024-09-21T05:56:57.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.175+0000 7f7977fff640 1 -- 172.21.15.117:0/3705279875 learned_addr learned my addr 172.21.15.117:0/3705279875 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:57.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.175+0000 7f79777fe640 1 --2- 172.21.15.117:0/3705279875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7978103b60 0x7f797819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:57.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.175+0000 7f7977fff640 1 -- 172.21.15.117:0/3705279875 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79781094a0 msgr2=0x7f797819aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:57.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.175+0000 7f7977fff640 1 --2- 172.21.15.117:0/3705279875 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79781094a0 0x7f797819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.175+0000 7f7977fff640 1 -- 172.21.15.117:0/3705279875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7978103b60 msgr2=0x7f797819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:57.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.175+0000 7f7977fff640 1 --2- 172.21.15.117:0/3705279875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7978103b60 0x7f797819a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.175+0000 7f7977fff640 1 -- 172.21.15.117:0/3705279875 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79781a0450 con 0x7f7978109df0 2024-09-21T05:56:57.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.175+0000 7f7977fff640 1 --2- 172.21.15.117:0/3705279875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7978109df0 0x7f797819fd20 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f797000c970 tx=0x7f797000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:57.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.175+0000 7f7974ff9640 1 -- 172.21.15.117:0/3705279875 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7970007bb0 con 0x7f7978109df0 2024-09-21T05:56:57.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.175+0000 7f7974ff9640 1 -- 172.21.15.117:0/3705279875 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7970007d50 con 0x7f7978109df0 2024-09-21T05:56:57.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.175+0000 7f797e08d640 1 -- 172.21.15.117:0/3705279875 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79781a0740 con 0x7f7978109df0 2024-09-21T05:56:57.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.175+0000 7f7974ff9640 1 -- 172.21.15.117:0/3705279875 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7970004e60 con 0x7f7978109df0 2024-09-21T05:56:57.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.175+0000 7f797e08d640 1 -- 172.21.15.117:0/3705279875 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79781a6ea0 con 0x7f7978109df0 2024-09-21T05:56:57.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.178+0000 7f7984601740 1 Processor -- start 2024-09-21T05:56:57.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.178+0000 7f7974ff9640 1 -- 172.21.15.117:0/3705279875 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f7970005020 con 0x7f7978109df0 2024-09-21T05:56:57.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.178+0000 7f7984601740 1 -- start start 2024-09-21T05:56:57.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.178+0000 7f7984601740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e86a6f60 0x55f9e8a4cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:57.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.178+0000 7f7984601740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9e867e0a0 0x55f9e8a4d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:57.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.178+0000 7f7984601740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9e8686280 0x55f9e8a4d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:57.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.178+0000 7f7984601740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f9e89c4c10 con 0x55f9e867e0a0 2024-09-21T05:56:57.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.178+0000 7f7984601740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f9e89b4040 con 0x55f9e86a6f60 2024-09-21T05:56:57.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.178+0000 7f7984601740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f9e89b7e50 con 0x55f9e8686280 2024-09-21T05:56:57.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.178+0000 7f79777fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9e867e0a0 0x55f9e8a4d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:57.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.178+0000 7f79777fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9e867e0a0 0x55f9e8a4d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44496/0 (socket says 172.21.15.117:44496) 2024-09-21T05:56:57.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.178+0000 7f79777fe640 1 -- 172.21.15.117:0/41422267 learned_addr learned my addr 172.21.15.117:0/41422267 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:57.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.178+0000 7f7976ffd640 1 --2- 172.21.15.117:0/41422267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e86a6f60 0x55f9e8a4cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:57.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.179+0000 7f7974ff9640 1 --2- 172.21.15.117:0/3705279875 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f79600b65f0 0x7f79600b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:57.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.179+0000 7f7976ffd640 1 --2- 172.21.15.117:0/41422267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9e8686280 0x55f9e8a4d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:57.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.179+0000 7f79777fe640 1 -- 172.21.15.117:0/41422267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9e8686280 msgr2=0x55f9e8a4d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:57.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.179+0000 7f79777fe640 1 --2- 172.21.15.117:0/41422267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9e8686280 0x55f9e8a4d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.179+0000 7f79777fe640 1 -- 172.21.15.117:0/41422267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e86a6f60 msgr2=0x55f9e8a4cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:57.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.179+0000 7f79777fe640 1 --2- 172.21.15.117:0/41422267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e86a6f60 0x55f9e8a4cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.179+0000 7f79777fe640 1 -- 172.21.15.117:0/41422267 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9e8712540 con 0x55f9e867e0a0 2024-09-21T05:56:57.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.179+0000 7f7976ffd640 1 --2- 172.21.15.117:0/41422267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9e8686280 0x55f9e8a4d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:57.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.179+0000 7f7977fff640 1 --2- 172.21.15.117:0/3705279875 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f79600b65f0 0x7f79600b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:57.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.179+0000 7f7974ff9640 1 -- 172.21.15.117:0/3705279875 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f79700e72e0 con 0x7f7978109df0 2024-09-21T05:56:57.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.179+0000 7f79777fe640 1 --2- 172.21.15.117:0/41422267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9e867e0a0 0x55f9e8a4d440 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f796802a930 tx=0x7f7968031cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805c053431456c3d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:57.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.179+0000 7f795dffb640 1 -- 172.21.15.117:0/41422267 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796803f070 con 0x55f9e867e0a0 2024-09-21T05:56:57.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.179+0000 7f795dffb640 1 -- 172.21.15.117:0/41422267 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79680316a0 con 0x55f9e867e0a0 2024-09-21T05:56:57.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.179+0000 7f795dffb640 1 -- 172.21.15.117:0/41422267 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7968035070 con 0x55f9e867e0a0 2024-09-21T05:56:57.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.180+0000 7f7984601740 1 -- 172.21.15.117:0/41422267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9e867e0a0 msgr2=0x55f9e8a4d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:57.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.180+0000 7f7977fff640 1 --2- 172.21.15.117:0/3705279875 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f79600b65f0 0x7f79600b8b00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f797000c940 tx=0x7f7970014400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:57.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.180+0000 7f7984601740 1 --2- 172.21.15.117:0/41422267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9e867e0a0 0x55f9e8a4d440 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f796802a930 tx=0x7f7968031cd0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.180+0000 7f7984601740 1 -- 172.21.15.117:0/41422267 shutdown_connections 2024-09-21T05:56:57.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.180+0000 7f7984601740 1 --2- 172.21.15.117:0/41422267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9e867e0a0 0x55f9e8a4d440 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.180+0000 7f7984601740 1 --2- 172.21.15.117:0/41422267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9e8686280 0x55f9e8a4d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.180+0000 7f7984601740 1 --2- 172.21.15.117:0/41422267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e86a6f60 0x55f9e8a4cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.180+0000 7f7984601740 1 -- 172.21.15.117:0/41422267 >> 172.21.15.117:0/41422267 conn(0x55f9e869b350 msgr2=0x55f9e8a4adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:57.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.180+0000 7f7984601740 1 -- 172.21.15.117:0/41422267 shutdown_connections 2024-09-21T05:56:57.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.180+0000 7f7984601740 1 -- 172.21.15.117:0/41422267 wait complete. 2024-09-21T05:56:57.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.181+0000 7f7984601740 1 -- start start 2024-09-21T05:56:57.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.181+0000 7f7984601740 1 Processor -- start 2024-09-21T05:56:57.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.181+0000 7f7984601740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e867e0a0 0x55f9e8a587b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:57.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.181+0000 7f7984601740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9e8686280 0x55f9e8a58d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:57.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.181+0000 7f7984601740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9e86a6f60 0x55f9e8a649a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:57.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.181+0000 7f7984601740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f9e88155f0 con 0x55f9e8686280 2024-09-21T05:56:57.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.181+0000 7f7984601740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f9e894f010 con 0x55f9e867e0a0 2024-09-21T05:56:57.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.181+0000 7f7984601740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f9e8a5b680 con 0x55f9e86a6f60 2024-09-21T05:56:57.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.181+0000 7f7984601740 10 client.0 _ll_register_callbacks cb 0x55f9e89ae340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:56:57.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.181+0000 7f79777fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9e8686280 0x55f9e8a58d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:57.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.181+0000 7f79777fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9e8686280 0x55f9e8a58d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44508/0 (socket says 172.21.15.117:44508) 2024-09-21T05:56:57.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.181+0000 7f79777fe640 1 -- 172.21.15.117:0/3392411066 learned_addr learned my addr 172.21.15.117:0/3392411066 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:57.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.182+0000 7f7976ffd640 1 --2- 172.21.15.117:0/3392411066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9e86a6f60 0x55f9e8a649a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:57.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.182+0000 7f79777fe640 1 -- 172.21.15.117:0/3392411066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9e86a6f60 msgr2=0x55f9e8a649a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:57.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.182+0000 7f79777fe640 1 --2- 172.21.15.117:0/3392411066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9e86a6f60 0x55f9e8a649a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.182+0000 7f79777fe640 1 -- 172.21.15.117:0/3392411066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e867e0a0 msgr2=0x55f9e8a587b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:57.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.182+0000 7f79777fe640 1 --2- 172.21.15.117:0/3392411066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e867e0a0 0x55f9e8a587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.182+0000 7f79777fe640 1 -- 172.21.15.117:0/3392411066 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9e897ca00 con 0x55f9e8686280 2024-09-21T05:56:57.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.182+0000 7f79777fe640 1 --2- 172.21.15.117:0/3392411066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9e8686280 0x55f9e8a58d70 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7968031440 tx=0x7f7968031590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4522b7e0c0f45c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:57.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.182+0000 7f7956ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:56:57.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.182+0000 7f7956ffd640 1 -- 172.21.15.117:0/3392411066 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7968035030 con 0x55f9e8686280 2024-09-21T05:56:57.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.182+0000 7f7984601740 1 -- 172.21.15.117:0/3392411066 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79680316a0 con 0x55f9e8686280 2024-09-21T05:56:57.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.182+0000 7f7956ffd640 1 -- 172.21.15.117:0/3392411066 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f7968004250 con 0x55f9e8686280 2024-09-21T05:56:57.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.182+0000 7f7956ffd640 1 -- 172.21.15.117:0/3392411066 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f796803f040 con 0x55f9e8686280 2024-09-21T05:56:57.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.182+0000 7f7984601740 10 client.15384 fetch_fsmap learned FSMap version 26 2024-09-21T05:56:57.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.182+0000 7f7984601740 1 -- 172.21.15.117:0/3392411066 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f9e8a674c0 con 0x55f9e8686280 2024-09-21T05:56:57.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.182+0000 7f7956ffd640 1 -- 172.21.15.117:0/3392411066 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79680316a0 con 0x55f9e8686280 2024-09-21T05:56:57.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.183+0000 7f7956ffd640 1 -- 172.21.15.117:0/3392411066 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f7968003860 con 0x55f9e8686280 2024-09-21T05:56:57.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.183+0000 7f7984601740 10 client.15384 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T05:56:57.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.183+0000 7f7984601740 10 client.15384 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-21T05:56:57.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.183+0000 7f7984601740 20 client.15384 populate_metadata read hostname 'smithi117' 2024-09-21T05:56:57.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.183+0000 7f7984601740 1 --2- 172.21.15.117:0/3392411066 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55f9e86658e0 0x55f9e8a6aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:57.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.183+0000 7f7984601740 4 client.15384 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:56:57.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.183+0000 7f7984601740 1 -- 172.21.15.117:0/3392411066 --> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9e89c6b60 con 0x55f9e86658e0 2024-09-21T05:56:57.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.183+0000 7f7976ffd640 1 --2- 172.21.15.117:0/3392411066 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55f9e86658e0 0x55f9e8a6aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:57.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.183+0000 7f7956ffd640 10 client.15384 ms_handle_connect on v2:172.21.15.192:6834/3530688860 2024-09-21T05:56:57.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.183+0000 7f7976ffd640 1 --2- 172.21.15.117:0/3392411066 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55f9e86658e0 0x55f9e8a6aa30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1bb033382b2515e3 server_cookie=dd2b08e5f401b7d6 in_seq=0 out_seq=0 2024-09-21T05:56:57.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.184+0000 7f7956ffd640 1 -- 172.21.15.117:0/3392411066 <== mds.3 v2:172.21.15.192:6834/3530688860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f9e89c6b60 con 0x55f9e86658e0 2024-09-21T05:56:57.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.184+0000 7f7956ffd640 10 client.15384 handle_command_reply: tid=0 2024-09-21T05:56:57.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.184+0000 7f7984601740 1 client.15384 shutdown 2024-09-21T05:56:57.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.185+0000 7f7984601740 1 -- 172.21.15.117:0/3392411066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9e8686280 msgr2=0x55f9e8a58d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:57.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.185+0000 7f7984601740 1 --2- 172.21.15.117:0/3392411066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9e8686280 0x55f9e8a58d70 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7968031440 tx=0x7f7968031590 comp rx=0 tx=0).stop 2024-09-21T05:56:57.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.185+0000 7f7984601740 1 -- 172.21.15.117:0/3392411066 shutdown_connections 2024-09-21T05:56:57.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.185+0000 7f7984601740 1 --2- 172.21.15.117:0/3392411066 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55f9e86658e0 0x55f9e8a6aa30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.185+0000 7f7984601740 1 --2- 172.21.15.117:0/3392411066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9e8686280 0x55f9e8a58d70 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.185+0000 7f7984601740 1 --2- 172.21.15.117:0/3392411066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9e86a6f60 0x55f9e8a649a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.185+0000 7f7956ffd640 0 client.15384 ms_handle_reset on v2:172.21.15.192:6834/3530688860 2024-09-21T05:56:57.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.185+0000 7f7984601740 1 --2- 172.21.15.117:0/3392411066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e867e0a0 0x55f9e8a587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.185+0000 7f7984601740 1 -- 172.21.15.117:0/3392411066 >> 172.21.15.117:0/3392411066 conn(0x55f9e869b350 msgr2=0x55f9e8a50090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:57.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.185+0000 7f7984601740 1 -- 172.21.15.117:0/3392411066 shutdown_connections 2024-09-21T05:56:57.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.185+0000 7f7984601740 1 -- 172.21.15.117:0/3392411066 wait complete. 2024-09-21T05:56:57.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.185+0000 7f7984601740 20 client.15384 trim_cache size 0 max 16384 2024-09-21T05:56:57.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.206+0000 7f7984601740 1 Processor -- start 2024-09-21T05:56:57.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.206+0000 7f7984601740 1 -- start start 2024-09-21T05:56:57.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.206+0000 7f7984601740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9e867e0a0 0x55f9e8a53020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:57.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.206+0000 7f7984601740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e8686280 0x55f9e8a4ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:57.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.206+0000 7f7984601740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9e86a6f60 0x55f9e8a64910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:57.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.206+0000 7f7984601740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f9e8a68d70 con 0x55f9e867e0a0 2024-09-21T05:56:57.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.206+0000 7f7984601740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f9e8a69440 con 0x55f9e8686280 2024-09-21T05:56:57.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.206+0000 7f7984601740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f9e8a69b10 con 0x55f9e86a6f60 2024-09-21T05:56:57.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.207+0000 7f7976ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9e867e0a0 0x55f9e8a53020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:57.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.207+0000 7f7976ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9e867e0a0 0x55f9e8a53020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44518/0 (socket says 172.21.15.117:44518) 2024-09-21T05:56:57.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.207+0000 7f7976ffd640 1 -- 172.21.15.117:0/68108500 learned_addr learned my addr 172.21.15.117:0/68108500 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:57.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.207+0000 7f79777fe640 1 --2- 172.21.15.117:0/68108500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e8686280 0x55f9e8a4ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:57.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.207+0000 7f7976ffd640 1 --2- 172.21.15.117:0/68108500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9e86a6f60 0x55f9e8a64910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:57.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.207+0000 7f7976ffd640 1 -- 172.21.15.117:0/68108500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9e86a6f60 msgr2=0x55f9e8a64910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:57.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.207+0000 7f7976ffd640 1 --2- 172.21.15.117:0/68108500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9e86a6f60 0x55f9e8a64910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.207+0000 7f7976ffd640 1 -- 172.21.15.117:0/68108500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e8686280 msgr2=0x55f9e8a4ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:57.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.207+0000 7f7976ffd640 1 --2- 172.21.15.117:0/68108500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e8686280 0x55f9e8a4ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.207+0000 7f7976ffd640 1 -- 172.21.15.117:0/68108500 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9e8a54320 con 0x55f9e867e0a0 2024-09-21T05:56:57.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.207+0000 7f79777fe640 1 --2- 172.21.15.117:0/68108500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e8686280 0x55f9e8a4ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:56:57.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.207+0000 7f7976ffd640 1 --2- 172.21.15.117:0/68108500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9e867e0a0 0x55f9e8a53020 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f797806c760 tx=0x7f796c002080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ac9551564c4347 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:57.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.209+0000 7f795d7fa640 1 -- 172.21.15.117:0/68108500 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796c0193d0 con 0x55f9e867e0a0 2024-09-21T05:56:57.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.209+0000 7f795d7fa640 1 -- 172.21.15.117:0/68108500 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f796c019a20 con 0x55f9e867e0a0 2024-09-21T05:56:57.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.209+0000 7f795d7fa640 1 -- 172.21.15.117:0/68108500 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796c01eaf0 con 0x55f9e867e0a0 2024-09-21T05:56:57.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.209+0000 7f7984601740 1 -- 172.21.15.117:0/68108500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9e867e0a0 msgr2=0x55f9e8a53020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:57.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.209+0000 7f7984601740 1 --2- 172.21.15.117:0/68108500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9e867e0a0 0x55f9e8a53020 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f797806c760 tx=0x7f796c002080 comp rx=0 tx=0).stop 2024-09-21T05:56:57.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.210+0000 7f7984601740 1 -- 172.21.15.117:0/68108500 shutdown_connections 2024-09-21T05:56:57.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.210+0000 7f7984601740 1 --2- 172.21.15.117:0/68108500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9e86a6f60 0x55f9e8a64910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.210+0000 7f7984601740 1 --2- 172.21.15.117:0/68108500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e8686280 0x55f9e8a4ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.210+0000 7f7984601740 1 --2- 172.21.15.117:0/68108500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9e867e0a0 0x55f9e8a53020 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.210+0000 7f7984601740 1 -- 172.21.15.117:0/68108500 >> 172.21.15.117:0/68108500 conn(0x55f9e869b350 msgr2=0x55f9e8a4a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:57.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.210+0000 7f7984601740 1 -- 172.21.15.117:0/68108500 shutdown_connections 2024-09-21T05:56:57.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.210+0000 7f7984601740 1 -- 172.21.15.117:0/68108500 wait complete. 2024-09-21T05:56:57.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.210+0000 7f7984601740 1 -- start start 2024-09-21T05:56:57.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.210+0000 7f7984601740 1 Processor -- start 2024-09-21T05:56:57.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.211+0000 7f7984601740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9e867e0a0 0x55f9e8a733f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:57.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.211+0000 7f7984601740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e8686280 0x55f9e8a6c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:57.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.211+0000 7f7984601740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9e86a6f60 0x55f9e8a6ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:57.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.211+0000 7f7984601740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f9e8a6a8b0 con 0x55f9e86a6f60 2024-09-21T05:56:57.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.211+0000 7f7984601740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f9e8a6a1e0 con 0x55f9e8686280 2024-09-21T05:56:57.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.211+0000 7f7984601740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f9e8a6af80 con 0x55f9e867e0a0 2024-09-21T05:56:57.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.211+0000 7f7984601740 10 client.0 _ll_register_callbacks cb 0x7f796c0019b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:56:57.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.211+0000 7f7976ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9e86a6f60 0x55f9e8a6ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:57.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.211+0000 7f7976ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9e86a6f60 0x55f9e8a6ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44520/0 (socket says 172.21.15.117:44520) 2024-09-21T05:56:57.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.211+0000 7f7976ffd640 1 -- 172.21.15.117:0/994956335 learned_addr learned my addr 172.21.15.117:0/994956335 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:57.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.211+0000 7f79777fe640 1 --2- 172.21.15.117:0/994956335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e8686280 0x55f9e8a6c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:57.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.211+0000 7f7976ffd640 1 --2- 172.21.15.117:0/994956335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9e867e0a0 0x55f9e8a733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:57.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.211+0000 7f7976ffd640 1 -- 172.21.15.117:0/994956335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9e867e0a0 msgr2=0x55f9e8a733f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:57.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.211+0000 7f7976ffd640 1 --2- 172.21.15.117:0/994956335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9e867e0a0 0x55f9e8a733f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.211+0000 7f7976ffd640 1 -- 172.21.15.117:0/994956335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e8686280 msgr2=0x55f9e8a6c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:57.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.211+0000 7f7976ffd640 1 --2- 172.21.15.117:0/994956335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e8686280 0x55f9e8a6c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.211+0000 7f7976ffd640 1 -- 172.21.15.117:0/994956335 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9e8a6d540 con 0x55f9e86a6f60 2024-09-21T05:56:57.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.211+0000 7f7976ffd640 1 --2- 172.21.15.117:0/994956335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9e86a6f60 0x55f9e8a6ce80 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f796c019c80 tx=0x7f796c01e690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=287b8ee96dd978d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:57.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.211+0000 7f7956ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:56:57.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.212+0000 7f7956ffd640 1 -- 172.21.15.117:0/994956335 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796c01e890 con 0x55f9e86a6f60 2024-09-21T05:56:57.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.212+0000 7f7956ffd640 1 -- 172.21.15.117:0/994956335 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f796c01ea30 con 0x55f9e86a6f60 2024-09-21T05:56:57.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.212+0000 7f7984601740 1 -- 172.21.15.117:0/994956335 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f796c019a20 con 0x55f9e86a6f60 2024-09-21T05:56:57.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.212+0000 7f7956ffd640 1 -- 172.21.15.117:0/994956335 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f796c0193d0 con 0x55f9e86a6f60 2024-09-21T05:56:57.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.212+0000 7f7984601740 10 client.15390 fetch_fsmap learned FSMap version 26 2024-09-21T05:56:57.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.212+0000 7f7984601740 1 -- 172.21.15.117:0/994956335 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f9e8ad5f00 con 0x55f9e86a6f60 2024-09-21T05:56:57.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.212+0000 7f7956ffd640 1 -- 172.21.15.117:0/994956335 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f796c019a20 con 0x55f9e86a6f60 2024-09-21T05:56:57.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.212+0000 7f7956ffd640 1 -- 172.21.15.117:0/994956335 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f796c02c040 con 0x55f9e86a6f60 2024-09-21T05:56:57.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.212+0000 7f7984601740 10 client.15390 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T05:56:57.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.212+0000 7f7984601740 10 client.15390 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-21T05:56:57.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.212+0000 7f7984601740 20 client.15390 populate_metadata read hostname 'smithi117' 2024-09-21T05:56:57.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.212+0000 7f7984601740 1 --2- 172.21.15.117:0/994956335 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55f9e863ac90 0x55f9e8ad9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:57.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.212+0000 7f7984601740 4 client.15390 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T05:56:57.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.212+0000 7f7984601740 1 -- 172.21.15.117:0/994956335 --> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] -- command(tid 0: {"prefix": "status"}) -- 0x55f9e89c2610 con 0x55f9e863ac90 2024-09-21T05:56:57.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.213+0000 7f79777fe640 1 --2- 172.21.15.117:0/994956335 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55f9e863ac90 0x55f9e8ad9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:57.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.213+0000 7f7956ffd640 10 client.15390 ms_handle_connect on v2:172.21.15.192:6834/3530688860 2024-09-21T05:56:57.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.213+0000 7f79777fe640 1 --2- 172.21.15.117:0/994956335 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55f9e863ac90 0x55f9e8ad9290 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=271eb408050464e server_cookie=34407f311bbb900f in_seq=0 out_seq=0 2024-09-21T05:56:57.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.213+0000 7f7956ffd640 1 -- 172.21.15.117:0/994956335 <== mds.3 v2:172.21.15.192:6834/3530688860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f9e89c2610 con 0x55f9e863ac90 2024-09-21T05:56:57.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.213+0000 7f7956ffd640 10 client.15390 handle_command_reply: tid=0 2024-09-21T05:56:57.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.213+0000 7f7984601740 1 client.15390 shutdown 2024-09-21T05:56:57.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.214+0000 7f7984601740 1 -- 172.21.15.117:0/994956335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9e86a6f60 msgr2=0x55f9e8a6ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:57.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.214+0000 7f7984601740 1 --2- 172.21.15.117:0/994956335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9e86a6f60 0x55f9e8a6ce80 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f796c019c80 tx=0x7f796c01e690 comp rx=0 tx=0).stop 2024-09-21T05:56:57.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.214+0000 7f7984601740 1 -- 172.21.15.117:0/994956335 shutdown_connections 2024-09-21T05:56:57.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.214+0000 7f7984601740 1 --2- 172.21.15.117:0/994956335 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55f9e863ac90 0x55f9e8ad9290 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.214+0000 7f7984601740 1 --2- 172.21.15.117:0/994956335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f9e86a6f60 0x55f9e8a6ce80 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.214+0000 7f7984601740 1 --2- 172.21.15.117:0/994956335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9e8686280 0x55f9e8a6c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.214+0000 7f7956ffd640 0 client.15390 ms_handle_reset on v2:172.21.15.192:6834/3530688860 2024-09-21T05:56:57.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.214+0000 7f7984601740 1 --2- 172.21.15.117:0/994956335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f9e867e0a0 0x55f9e8a733f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.214+0000 7f7984601740 1 -- 172.21.15.117:0/994956335 >> 172.21.15.117:0/994956335 conn(0x55f9e869b350 msgr2=0x55f9e8a4df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:57.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.214+0000 7f7984601740 1 -- 172.21.15.117:0/994956335 shutdown_connections 2024-09-21T05:56:57.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.214+0000 7f7984601740 1 -- 172.21.15.117:0/994956335 wait complete. 2024-09-21T05:56:57.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.214+0000 7f7984601740 20 client.15390 trim_cache size 0 max 16384 2024-09-21T05:56:57.220 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T05:56:57.220 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_fsid": "ac8581a2-77dc-11ef-baf5-efdc52797490", 2024-09-21T05:56:57.221 INFO:teuthology.orchestra.run.smithi117.stdout: "whoami": 3, 2024-09-21T05:56:57.221 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 24616, 2024-09-21T05:56:57.221 INFO:teuthology.orchestra.run.smithi117.stdout: "want_state": "up:active", 2024-09-21T05:56:57.221 INFO:teuthology.orchestra.run.smithi117.stdout: "state": "up:active", 2024-09-21T05:56:57.221 INFO:teuthology.orchestra.run.smithi117.stdout: "fs_name": "cephfs", 2024-09-21T05:56:57.221 INFO:teuthology.orchestra.run.smithi117.stdout: "rank_uptime": 93.018419515000005, 2024-09-21T05:56:57.221 INFO:teuthology.orchestra.run.smithi117.stdout: "mdsmap_epoch": 26, 2024-09-21T05:56:57.221 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch": 76, 2024-09-21T05:56:57.221 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch_barrier": 74, 2024-09-21T05:56:57.221 INFO:teuthology.orchestra.run.smithi117.stdout: "uptime": 130.54430578200001 2024-09-21T05:56:57.221 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T05:56:57.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.215+0000 7f797e08d640 1 -- 172.21.15.117:0/3705279875 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f79600b65f0 msgr2=0x7f79600b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:57.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.215+0000 7f797e08d640 1 --2- 172.21.15.117:0/3705279875 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f79600b65f0 0x7f79600b8b00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f797000c940 tx=0x7f7970014400 comp rx=0 tx=0).stop 2024-09-21T05:56:57.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.215+0000 7f797e08d640 1 -- 172.21.15.117:0/3705279875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7978109df0 msgr2=0x7f797819fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:57.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.215+0000 7f797e08d640 1 --2- 172.21.15.117:0/3705279875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7978109df0 0x7f797819fd20 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f797000c970 tx=0x7f797000ce30 comp rx=0 tx=0).stop 2024-09-21T05:56:57.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.215+0000 7f797e08d640 1 -- 172.21.15.117:0/3705279875 shutdown_connections 2024-09-21T05:56:57.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.215+0000 7f797e08d640 1 --2- 172.21.15.117:0/3705279875 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f79600b65f0 0x7f79600b8b00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.215+0000 7f797e08d640 1 --2- 172.21.15.117:0/3705279875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7978109df0 0x7f797819fd20 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.215+0000 7f797e08d640 1 --2- 172.21.15.117:0/3705279875 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79781094a0 0x7f797819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.215+0000 7f797e08d640 1 --2- 172.21.15.117:0/3705279875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7978103b60 0x7f797819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:57.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.215+0000 7f797e08d640 1 -- 172.21.15.117:0/3705279875 >> 172.21.15.117:0/3705279875 conn(0x7f797807b850 msgr2=0x7f7978107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:57.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.215+0000 7f797e08d640 1 -- 172.21.15.117:0/3705279875 shutdown_connections 2024-09-21T05:56:57.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:57.215+0000 7f797e08d640 1 -- 172.21.15.117:0/3705279875 wait complete. 2024-09-21T05:56:57.976 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:57 smithi117 ceph-mon[30947]: pgmap v316: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T05:56:57.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:57 smithi192 ceph-mon[34700]: pgmap v316: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T05:56:58.006 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.c status 2024-09-21T05:56:58.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:57 smithi138 ceph-mon[34391]: pgmap v316: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T05:56:58.252 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:58.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.735+0000 7fc0ca670640 1 -- 172.21.15.117:0/2939495824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0c4108d70 msgr2=0x7fc0c410f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:58.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.735+0000 7fc0ca670640 1 --2- 172.21.15.117:0/2939495824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0c4108d70 0x7fc0c410f5a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009ca0 tx=0x7fc0bc02a8c0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.735+0000 7fc0ca670640 1 -- 172.21.15.117:0/2939495824 shutdown_connections 2024-09-21T05:56:58.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.735+0000 7fc0ca670640 1 --2- 172.21.15.117:0/2939495824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0c4108d70 0x7fc0c410f5a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.735+0000 7fc0ca670640 1 --2- 172.21.15.117:0/2939495824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0c4108420 0x7fc0c4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.735+0000 7fc0ca670640 1 --2- 172.21.15.117:0/2939495824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0c4102ae0 0x7fc0c4102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.735+0000 7fc0ca670640 1 -- 172.21.15.117:0/2939495824 >> 172.21.15.117:0/2939495824 conn(0x7fc0c4079770 msgr2=0x7fc0c4079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:58.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.735+0000 7fc0ca670640 1 -- 172.21.15.117:0/2939495824 shutdown_connections 2024-09-21T05:56:58.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.735+0000 7fc0ca670640 1 -- 172.21.15.117:0/2939495824 wait complete. 2024-09-21T05:56:58.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.736+0000 7fc0ca670640 1 Processor -- start 2024-09-21T05:56:58.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.736+0000 7fc0ca670640 1 -- start start 2024-09-21T05:56:58.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.736+0000 7fc0ca670640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0c4102ae0 0x7fc0c419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:58.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.736+0000 7fc0ca670640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0c4108420 0x7fc0c419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:58.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.736+0000 7fc0ca670640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0c4108d70 0x7fc0c419fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:58.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.736+0000 7fc0ca670640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c4112a30 con 0x7fc0c4108420 2024-09-21T05:56:58.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.736+0000 7fc0ca670640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c41128b0 con 0x7fc0c4102ae0 2024-09-21T05:56:58.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.736+0000 7fc0ca670640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c4112bb0 con 0x7fc0c4108d70 2024-09-21T05:56:58.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.736+0000 7fc0bbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0c4108420 0x7fc0c419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:58.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.736+0000 7fc0bbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0c4108420 0x7fc0c419aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44542/0 (socket says 172.21.15.117:44542) 2024-09-21T05:56:58.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.736+0000 7fc0bbfff640 1 -- 172.21.15.117:0/203558966 learned_addr learned my addr 172.21.15.117:0/203558966 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:58.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.736+0000 7fc0c8c0d640 1 --2- 172.21.15.117:0/203558966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0c4108d70 0x7fc0c419fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:58.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.736+0000 7fc0c3fff640 1 --2- 172.21.15.117:0/203558966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0c4102ae0 0x7fc0c419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:58.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.736+0000 7fc0bbfff640 1 -- 172.21.15.117:0/203558966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0c4108d70 msgr2=0x7fc0c419fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:58.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.736+0000 7fc0bbfff640 1 --2- 172.21.15.117:0/203558966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0c4108d70 0x7fc0c419fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.736+0000 7fc0bbfff640 1 -- 172.21.15.117:0/203558966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0c4102ae0 msgr2=0x7fc0c419a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:58.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.736+0000 7fc0bbfff640 1 --2- 172.21.15.117:0/203558966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0c4102ae0 0x7fc0c419a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.736+0000 7fc0bbfff640 1 -- 172.21.15.117:0/203558966 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c41a0480 con 0x7fc0c4108420 2024-09-21T05:56:58.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.736+0000 7fc0c3fff640 1 --2- 172.21.15.117:0/203558966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0c4102ae0 0x7fc0c419a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:56:58.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.736+0000 7fc0bbfff640 1 --2- 172.21.15.117:0/203558966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0c4108420 0x7fc0c419aab0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac00c970 tx=0x7fc0ac00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:58.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.737+0000 7fc0c1ffb640 1 -- 172.21.15.117:0/203558966 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac004220 con 0x7fc0c4108420 2024-09-21T05:56:58.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.737+0000 7fc0ca670640 1 -- 172.21.15.117:0/203558966 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0c41a0770 con 0x7fc0c4108420 2024-09-21T05:56:58.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.737+0000 7fc0c1ffb640 1 -- 172.21.15.117:0/203558966 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0ac0043c0 con 0x7fc0c4108420 2024-09-21T05:56:58.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.737+0000 7fc0c1ffb640 1 -- 172.21.15.117:0/203558966 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac004a00 con 0x7fc0c4108420 2024-09-21T05:56:58.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.737+0000 7fc0ca670640 1 -- 172.21.15.117:0/203558966 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0c41a6ed0 con 0x7fc0c4108420 2024-09-21T05:56:58.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.739+0000 7fc0c1ffb640 1 -- 172.21.15.117:0/203558966 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7fc0ac0079b0 con 0x7fc0c4108420 2024-09-21T05:56:58.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.739+0000 7fc0d0be4740 1 Processor -- start 2024-09-21T05:56:58.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.739+0000 7fc0d0be4740 1 -- start start 2024-09-21T05:56:58.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.740+0000 7fc0d0be4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583b9baef60 0x5583b9f59310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:58.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.740+0000 7fc0d0be4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583b9b860a0 0x5583b9f59850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:58.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.740+0000 7fc0d0be4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583b9b8e280 0x5583b9f59d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:58.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.740+0000 7fc0d0be4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5583b9eccc10 con 0x5583b9baef60 2024-09-21T05:56:58.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.740+0000 7fc0d0be4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5583b9ebc040 con 0x5583b9b860a0 2024-09-21T05:56:58.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.740+0000 7fc0d0be4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5583b9ebfe50 con 0x5583b9b8e280 2024-09-21T05:56:58.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.740+0000 7fc0c8c0d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583b9baef60 0x5583b9f59310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:58.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.740+0000 7fc0c8c0d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583b9baef60 0x5583b9f59310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44558/0 (socket says 172.21.15.117:44558) 2024-09-21T05:56:58.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.740+0000 7fc0c8c0d640 1 -- 172.21.15.117:0/3206150426 learned_addr learned my addr 172.21.15.117:0/3206150426 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:58.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.740+0000 7fc0c3fff640 1 --2- 172.21.15.117:0/3206150426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583b9b860a0 0x5583b9f59850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:58.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.740+0000 7fc0bbfff640 1 --2- 172.21.15.117:0/3206150426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583b9b8e280 0x5583b9f59d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:58.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.740+0000 7fc0c8c0d640 1 -- 172.21.15.117:0/3206150426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583b9b8e280 msgr2=0x5583b9f59d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:58.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.740+0000 7fc0c8c0d640 1 --2- 172.21.15.117:0/3206150426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583b9b8e280 0x5583b9f59d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.740+0000 7fc0c8c0d640 1 -- 172.21.15.117:0/3206150426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583b9b860a0 msgr2=0x5583b9f59850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:58.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.740+0000 7fc0c8c0d640 1 --2- 172.21.15.117:0/3206150426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583b9b860a0 0x5583b9f59850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.740+0000 7fc0c8c0d640 1 -- 172.21.15.117:0/3206150426 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583b9c1a540 con 0x5583b9baef60 2024-09-21T05:56:58.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.740+0000 7fc0bbfff640 1 --2- 172.21.15.117:0/3206150426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583b9b8e280 0x5583b9f59d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:56:58.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.740+0000 7fc0c3fff640 1 --2- 172.21.15.117:0/3206150426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583b9b860a0 0x5583b9f59850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:56:58.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.740+0000 7fc0c1ffb640 1 --2- 172.21.15.117:0/203558966 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc0b00b65f0 0x7fc0b00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:58.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.740+0000 7fc0c8c0d640 1 --2- 172.21.15.117:0/3206150426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583b9baef60 0x5583b9f59310 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009920 tx=0x7fc0bc009950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb67a1ac33b28d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:58.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.740+0000 7fc0c3fff640 1 --2- 172.21.15.117:0/203558966 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc0b00b65f0 0x7fc0b00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:58.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.741+0000 7fc0c1ffb640 1 -- 172.21.15.117:0/203558966 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fc0ac0e7320 con 0x7fc0c4108420 2024-09-21T05:56:58.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.741+0000 7fc0ba7fc640 1 -- 172.21.15.117:0/3206150426 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc031e30 con 0x5583b9baef60 2024-09-21T05:56:58.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.741+0000 7fc0ba7fc640 1 -- 172.21.15.117:0/3206150426 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0bc036030 con 0x5583b9baef60 2024-09-21T05:56:58.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.741+0000 7fc0ba7fc640 1 -- 172.21.15.117:0/3206150426 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc0042a0 con 0x5583b9baef60 2024-09-21T05:56:58.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.741+0000 7fc0d0be4740 1 -- 172.21.15.117:0/3206150426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583b9baef60 msgr2=0x5583b9f59310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:58.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.741+0000 7fc0d0be4740 1 --2- 172.21.15.117:0/3206150426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583b9baef60 0x5583b9f59310 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009920 tx=0x7fc0bc009950 comp rx=0 tx=0).stop 2024-09-21T05:56:58.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.741+0000 7fc0c3fff640 1 --2- 172.21.15.117:0/203558966 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc0b00b65f0 0x7fc0b00b8b00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc0b40058c0 tx=0x7fc0b4005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:58.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.742+0000 7fc0d0be4740 1 -- 172.21.15.117:0/3206150426 shutdown_connections 2024-09-21T05:56:58.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.742+0000 7fc0d0be4740 1 --2- 172.21.15.117:0/3206150426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583b9b8e280 0x5583b9f59d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.742+0000 7fc0d0be4740 1 --2- 172.21.15.117:0/3206150426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583b9b860a0 0x5583b9f59850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.742+0000 7fc0d0be4740 1 --2- 172.21.15.117:0/3206150426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583b9baef60 0x5583b9f59310 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.742+0000 7fc0d0be4740 1 -- 172.21.15.117:0/3206150426 >> 172.21.15.117:0/3206150426 conn(0x5583b9ba3350 msgr2=0x5583b9f57200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:58.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.742+0000 7fc0d0be4740 1 -- 172.21.15.117:0/3206150426 shutdown_connections 2024-09-21T05:56:58.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.742+0000 7fc0d0be4740 1 -- 172.21.15.117:0/3206150426 wait complete. 2024-09-21T05:56:58.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.742+0000 7fc0d0be4740 1 -- start start 2024-09-21T05:56:58.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.742+0000 7fc0d0be4740 1 Processor -- start 2024-09-21T05:56:58.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.743+0000 7fc0d0be4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583b9b860a0 0x5583b9f64e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:58.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.743+0000 7fc0d0be4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583b9b8e280 0x5583b9f653d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:58.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.743+0000 7fc0d0be4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583b9baef60 0x5583b9f70c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:58.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.743+0000 7fc0d0be4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5583b9d1d5f0 con 0x5583b9baef60 2024-09-21T05:56:58.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.743+0000 7fc0d0be4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5583b9e57010 con 0x5583b9b8e280 2024-09-21T05:56:58.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.743+0000 7fc0d0be4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5583b9f67910 con 0x5583b9b860a0 2024-09-21T05:56:58.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.743+0000 7fc0d0be4740 10 client.0 _ll_register_callbacks cb 0x5583b9eb6340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:56:58.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.743+0000 7fc0c8c0d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583b9baef60 0x5583b9f70c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:58.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.743+0000 7fc0c8c0d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583b9baef60 0x5583b9f70c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44566/0 (socket says 172.21.15.117:44566) 2024-09-21T05:56:58.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.743+0000 7fc0c8c0d640 1 -- 172.21.15.117:0/1189250433 learned_addr learned my addr 172.21.15.117:0/1189250433 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:58.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.743+0000 7fc0bbfff640 1 --2- 172.21.15.117:0/1189250433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583b9b8e280 0x5583b9f653d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:58.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.743+0000 7fc0c3fff640 1 --2- 172.21.15.117:0/1189250433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583b9b860a0 0x5583b9f64e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:58.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.743+0000 7fc0c8c0d640 1 -- 172.21.15.117:0/1189250433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583b9b860a0 msgr2=0x5583b9f64e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:58.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.743+0000 7fc0c8c0d640 1 --2- 172.21.15.117:0/1189250433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583b9b860a0 0x5583b9f64e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.743+0000 7fc0c8c0d640 1 -- 172.21.15.117:0/1189250433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583b9b8e280 msgr2=0x5583b9f653d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:58.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.743+0000 7fc0c8c0d640 1 --2- 172.21.15.117:0/1189250433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583b9b8e280 0x5583b9f653d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.743+0000 7fc0c8c0d640 1 -- 172.21.15.117:0/1189250433 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583b9e84a00 con 0x5583b9baef60 2024-09-21T05:56:58.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.743+0000 7fc0c3fff640 1 --2- 172.21.15.117:0/1189250433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583b9b860a0 0x5583b9f64e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:56:58.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.743+0000 7fc0c8c0d640 1 --2- 172.21.15.117:0/1189250433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583b9baef60 0x5583b9f70c30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc031680 tx=0x7fc0bc0316b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d72a92fafbae4ec server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:58.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.743+0000 7fc0a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:56:58.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.744+0000 7fc0a37fe640 1 -- 172.21.15.117:0/1189250433 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc04a040 con 0x5583b9baef60 2024-09-21T05:56:58.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.744+0000 7fc0a37fe640 1 -- 172.21.15.117:0/1189250433 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fc0bc004200 con 0x5583b9baef60 2024-09-21T05:56:58.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.744+0000 7fc0d0be4740 1 -- 172.21.15.117:0/1189250433 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0bc036030 con 0x5583b9baef60 2024-09-21T05:56:58.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.744+0000 7fc0a37fe640 1 -- 172.21.15.117:0/1189250433 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc0bc0318d0 con 0x5583b9baef60 2024-09-21T05:56:58.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.744+0000 7fc0d0be4740 10 client.15402 fetch_fsmap learned FSMap version 26 2024-09-21T05:56:58.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.744+0000 7fc0d0be4740 1 -- 172.21.15.117:0/1189250433 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5583b9f73820 con 0x5583b9baef60 2024-09-21T05:56:58.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.744+0000 7fc0a37fe640 1 -- 172.21.15.117:0/1189250433 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0bc036030 con 0x5583b9baef60 2024-09-21T05:56:58.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.744+0000 7fc0a37fe640 1 -- 172.21.15.117:0/1189250433 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fc0bc04d030 con 0x5583b9baef60 2024-09-21T05:56:58.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.744+0000 7fc0d0be4740 10 client.15402 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T05:56:58.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.744+0000 7fc0d0be4740 10 client.15402 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T05:56:58.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.744+0000 7fc0d0be4740 20 client.15402 populate_metadata read hostname 'smithi117' 2024-09-21T05:56:58.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.744+0000 7fc0d0be4740 1 --2- 172.21.15.117:0/1189250433 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x5583b9b6d8e0 0x5583b9f76f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:58.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.744+0000 7fc0d0be4740 4 client.15402 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:56:58.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.744+0000 7fc0d0be4740 1 -- 172.21.15.117:0/1189250433 --> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583b9eceb60 con 0x5583b9b6d8e0 2024-09-21T05:56:58.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.745+0000 7fc0c3fff640 1 --2- 172.21.15.117:0/1189250433 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x5583b9b6d8e0 0x5583b9f76f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:58.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.745+0000 7fc0a37fe640 10 client.15402 ms_handle_connect on v2:172.21.15.192:6836/2683860842 2024-09-21T05:56:58.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.745+0000 7fc0c3fff640 1 --2- 172.21.15.117:0/1189250433 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x5583b9b6d8e0 0x5583b9f76f10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=132cab5166948f61 server_cookie=8c4cdfdce4c92857 in_seq=0 out_seq=0 2024-09-21T05:56:58.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.746+0000 7fc0a37fe640 1 -- 172.21.15.117:0/1189250433 <== mds.2 v2:172.21.15.192:6836/2683860842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5583b9eceb60 con 0x5583b9b6d8e0 2024-09-21T05:56:58.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.746+0000 7fc0a37fe640 10 client.15402 handle_command_reply: tid=0 2024-09-21T05:56:58.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.746+0000 7fc0d0be4740 1 client.15402 shutdown 2024-09-21T05:56:58.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.746+0000 7fc0d0be4740 1 -- 172.21.15.117:0/1189250433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583b9baef60 msgr2=0x5583b9f70c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:58.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.746+0000 7fc0d0be4740 1 --2- 172.21.15.117:0/1189250433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583b9baef60 0x5583b9f70c30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc031680 tx=0x7fc0bc0316b0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.746+0000 7fc0d0be4740 1 -- 172.21.15.117:0/1189250433 shutdown_connections 2024-09-21T05:56:58.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.746+0000 7fc0d0be4740 1 --2- 172.21.15.117:0/1189250433 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x5583b9b6d8e0 0x5583b9f76f10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.746+0000 7fc0d0be4740 1 --2- 172.21.15.117:0/1189250433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583b9baef60 0x5583b9f70c30 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.746+0000 7fc0d0be4740 1 --2- 172.21.15.117:0/1189250433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583b9b8e280 0x5583b9f653d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.746+0000 7fc0a37fe640 0 client.15402 ms_handle_reset on v2:172.21.15.192:6836/2683860842 2024-09-21T05:56:58.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.746+0000 7fc0d0be4740 1 --2- 172.21.15.117:0/1189250433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583b9b860a0 0x5583b9f64e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.746+0000 7fc0d0be4740 1 -- 172.21.15.117:0/1189250433 >> 172.21.15.117:0/1189250433 conn(0x5583b9ba3350 msgr2=0x5583b9f5c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:58.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.746+0000 7fc0d0be4740 1 -- 172.21.15.117:0/1189250433 shutdown_connections 2024-09-21T05:56:58.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.746+0000 7fc0d0be4740 1 -- 172.21.15.117:0/1189250433 wait complete. 2024-09-21T05:56:58.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.746+0000 7fc0d0be4740 20 client.15402 trim_cache size 0 max 16384 2024-09-21T05:56:58.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.764+0000 7fc0d0be4740 1 Processor -- start 2024-09-21T05:56:58.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.764+0000 7fc0d0be4740 1 -- start start 2024-09-21T05:56:58.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.765+0000 7fc0d0be4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583b9b6d8e0 0x5583b9f5f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:58.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.765+0000 7fc0d0be4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583b9b860a0 0x5583b9f65580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:58.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.765+0000 7fc0d0be4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583b9b8e280 0x5583b9f70ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:58.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.765+0000 7fc0d0be4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5583b9f75250 con 0x5583b9b6d8e0 2024-09-21T05:56:58.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.765+0000 7fc0d0be4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5583b9f75920 con 0x5583b9b8e280 2024-09-21T05:56:58.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.765+0000 7fc0d0be4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5583b9f75ff0 con 0x5583b9b860a0 2024-09-21T05:56:58.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.765+0000 7fc0c3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583b9b6d8e0 0x5583b9f5f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:58.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.765+0000 7fc0c3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583b9b6d8e0 0x5583b9f5f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44578/0 (socket says 172.21.15.117:44578) 2024-09-21T05:56:58.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.765+0000 7fc0c3fff640 1 -- 172.21.15.117:0/40959527 learned_addr learned my addr 172.21.15.117:0/40959527 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:58.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.765+0000 7fc0bbfff640 1 --2- 172.21.15.117:0/40959527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583b9b860a0 0x5583b9f65580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:58.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.765+0000 7fc0c8c0d640 1 --2- 172.21.15.117:0/40959527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583b9b8e280 0x5583b9f70ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:58.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.765+0000 7fc0c3fff640 1 -- 172.21.15.117:0/40959527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583b9b860a0 msgr2=0x5583b9f65580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:58.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.765+0000 7fc0c3fff640 1 --2- 172.21.15.117:0/40959527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583b9b860a0 0x5583b9f65580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.765+0000 7fc0c3fff640 1 -- 172.21.15.117:0/40959527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583b9b8e280 msgr2=0x5583b9f70ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:58.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.765+0000 7fc0c3fff640 1 --2- 172.21.15.117:0/40959527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583b9b8e280 0x5583b9f70ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.765+0000 7fc0c3fff640 1 -- 172.21.15.117:0/40959527 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583b9f57b70 con 0x5583b9b6d8e0 2024-09-21T05:56:58.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.765+0000 7fc0c3fff640 1 --2- 172.21.15.117:0/40959527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583b9b6d8e0 0x5583b9f5f430 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4012470 tx=0x7fc0b4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e87867e0d00b1a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:58.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.766+0000 7fc0b9ffb640 1 -- 172.21.15.117:0/40959527 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4004230 con 0x5583b9b6d8e0 2024-09-21T05:56:58.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.766+0000 7fc0b9ffb640 1 -- 172.21.15.117:0/40959527 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0b402c030 con 0x5583b9b6d8e0 2024-09-21T05:56:58.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.766+0000 7fc0b9ffb640 1 -- 172.21.15.117:0/40959527 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4003a60 con 0x5583b9b6d8e0 2024-09-21T05:56:58.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.766+0000 7fc0d0be4740 1 -- 172.21.15.117:0/40959527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583b9b6d8e0 msgr2=0x5583b9f5f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:58.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.766+0000 7fc0d0be4740 1 --2- 172.21.15.117:0/40959527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583b9b6d8e0 0x5583b9f5f430 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc0b4012470 tx=0x7fc0b4014000 comp rx=0 tx=0).stop 2024-09-21T05:56:58.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.766+0000 7fc0d0be4740 1 -- 172.21.15.117:0/40959527 shutdown_connections 2024-09-21T05:56:58.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.766+0000 7fc0d0be4740 1 --2- 172.21.15.117:0/40959527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583b9b8e280 0x5583b9f70ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.766+0000 7fc0d0be4740 1 --2- 172.21.15.117:0/40959527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583b9b860a0 0x5583b9f65580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.766+0000 7fc0d0be4740 1 --2- 172.21.15.117:0/40959527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583b9b6d8e0 0x5583b9f5f430 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.767+0000 7fc0d0be4740 1 -- 172.21.15.117:0/40959527 >> 172.21.15.117:0/40959527 conn(0x5583b9ba3350 msgr2=0x5583b9f64e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:58.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.767+0000 7fc0d0be4740 1 -- 172.21.15.117:0/40959527 shutdown_connections 2024-09-21T05:56:58.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.767+0000 7fc0d0be4740 1 -- 172.21.15.117:0/40959527 wait complete. 2024-09-21T05:56:58.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.767+0000 7fc0d0be4740 1 -- start start 2024-09-21T05:56:58.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.767+0000 7fc0d0be4740 1 Processor -- start 2024-09-21T05:56:58.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.768+0000 7fc0d0be4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583b9b6d8e0 0x5583b9f79440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:58.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.768+0000 7fc0d0be4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583b9b860a0 0x5583b9f79a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:58.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.768+0000 7fc0d0be4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583b9b8e280 0x5583b9f7f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:58.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.768+0000 7fc0d0be4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5583b9f76d90 con 0x5583b9b6d8e0 2024-09-21T05:56:58.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.768+0000 7fc0d0be4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5583b9f766c0 con 0x5583b9b8e280 2024-09-21T05:56:58.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.768+0000 7fc0d0be4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5583b9f77460 con 0x5583b9b860a0 2024-09-21T05:56:58.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.768+0000 7fc0d0be4740 10 client.0 _ll_register_callbacks cb 0x7fc0bc009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:56:58.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.768+0000 7fc0c3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583b9b6d8e0 0x5583b9f79440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:58.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.768+0000 7fc0c3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583b9b6d8e0 0x5583b9f79440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44584/0 (socket says 172.21.15.117:44584) 2024-09-21T05:56:58.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.768+0000 7fc0c3fff640 1 -- 172.21.15.117:0/2329118935 learned_addr learned my addr 172.21.15.117:0/2329118935 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:56:58.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.768+0000 7fc0bbfff640 1 --2- 172.21.15.117:0/2329118935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583b9b860a0 0x5583b9f79a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:58.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.768+0000 7fc0c8c0d640 1 --2- 172.21.15.117:0/2329118935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583b9b8e280 0x5583b9f7f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:58.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.768+0000 7fc0bbfff640 1 -- 172.21.15.117:0/2329118935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583b9b8e280 msgr2=0x5583b9f7f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:58.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.768+0000 7fc0bbfff640 1 --2- 172.21.15.117:0/2329118935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583b9b8e280 0x5583b9f7f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.768+0000 7fc0bbfff640 1 -- 172.21.15.117:0/2329118935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583b9b6d8e0 msgr2=0x5583b9f79440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:56:58.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.768+0000 7fc0bbfff640 1 --2- 172.21.15.117:0/2329118935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583b9b6d8e0 0x5583b9f79440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.768+0000 7fc0bbfff640 1 -- 172.21.15.117:0/2329118935 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583b9f7fac0 con 0x5583b9b860a0 2024-09-21T05:56:58.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.768+0000 7fc0c3fff640 1 --2- 172.21.15.117:0/2329118935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583b9b6d8e0 0x5583b9f79440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:56:58.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.768+0000 7fc0bbfff640 1 --2- 172.21.15.117:0/2329118935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583b9b860a0 0x5583b9f79a00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac0e4c30 tx=0x7fc0ac00a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40e9c8b1e54820fd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:56:58.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.768+0000 7fc0a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-21T05:56:58.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.768+0000 7fc0a37fe640 1 -- 172.21.15.117:0/2329118935 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac01d040 con 0x5583b9b860a0 2024-09-21T05:56:58.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.769+0000 7fc0a37fe640 1 -- 172.21.15.117:0/2329118935 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fc0ac0abf80 con 0x5583b9b860a0 2024-09-21T05:56:58.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.769+0000 7fc0d0be4740 1 -- 172.21.15.117:0/2329118935 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0b402c030 con 0x5583b9b860a0 2024-09-21T05:56:58.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.769+0000 7fc0a37fe640 1 -- 172.21.15.117:0/2329118935 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc0ac0acdc0 con 0x5583b9b860a0 2024-09-21T05:56:58.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.769+0000 7fc0d0be4740 10 client.24988 fetch_fsmap learned FSMap version 26 2024-09-21T05:56:58.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.769+0000 7fc0d0be4740 1 -- 172.21.15.117:0/2329118935 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5583b9f54e70 con 0x5583b9b860a0 2024-09-21T05:56:58.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.769+0000 7fc0a37fe640 1 -- 172.21.15.117:0/2329118935 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0b402c030 con 0x5583b9b860a0 2024-09-21T05:56:58.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.769+0000 7fc0a37fe640 1 -- 172.21.15.117:0/2329118935 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fc0ac0b2f20 con 0x5583b9b860a0 2024-09-21T05:56:58.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.769+0000 7fc0d0be4740 10 client.24988 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T05:56:58.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.769+0000 7fc0d0be4740 10 client.24988 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T05:56:58.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.769+0000 7fc0d0be4740 20 client.24988 populate_metadata read hostname 'smithi117' 2024-09-21T05:56:58.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.769+0000 7fc0d0be4740 1 --2- 172.21.15.117:0/2329118935 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x5583b9baef60 0x5583b9fea0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:56:58.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.769+0000 7fc0d0be4740 4 client.24988 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T05:56:58.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.769+0000 7fc0d0be4740 1 -- 172.21.15.117:0/2329118935 --> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] -- command(tid 0: {"prefix": "status"}) -- 0x5583b9eca610 con 0x5583b9baef60 2024-09-21T05:56:58.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.770+0000 7fc0c3fff640 1 --2- 172.21.15.117:0/2329118935 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x5583b9baef60 0x5583b9fea0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:56:58.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.770+0000 7fc0a37fe640 10 client.24988 ms_handle_connect on v2:172.21.15.192:6836/2683860842 2024-09-21T05:56:58.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.770+0000 7fc0c3fff640 1 --2- 172.21.15.117:0/2329118935 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x5583b9baef60 0x5583b9fea0b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bfa9beda43bdcc73 server_cookie=39cb01019632cd83 in_seq=0 out_seq=0 2024-09-21T05:56:58.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.770+0000 7fc0a37fe640 1 -- 172.21.15.117:0/2329118935 <== mds.2 v2:172.21.15.192:6836/2683860842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5583b9eca610 con 0x5583b9baef60 2024-09-21T05:56:58.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.770+0000 7fc0a37fe640 10 client.24988 handle_command_reply: tid=0 2024-09-21T05:56:58.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.770+0000 7fc0d0be4740 1 client.24988 shutdown 2024-09-21T05:56:58.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.771+0000 7fc0d0be4740 1 -- 172.21.15.117:0/2329118935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583b9b860a0 msgr2=0x5583b9f79a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:58.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.771+0000 7fc0d0be4740 1 --2- 172.21.15.117:0/2329118935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583b9b860a0 0x5583b9f79a00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac0e4c30 tx=0x7fc0ac00a340 comp rx=0 tx=0).stop 2024-09-21T05:56:58.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.771+0000 7fc0d0be4740 1 -- 172.21.15.117:0/2329118935 shutdown_connections 2024-09-21T05:56:58.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.771+0000 7fc0d0be4740 1 --2- 172.21.15.117:0/2329118935 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x5583b9baef60 0x5583b9fea0b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.771+0000 7fc0d0be4740 1 --2- 172.21.15.117:0/2329118935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5583b9b8e280 0x5583b9f7f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.771+0000 7fc0d0be4740 1 --2- 172.21.15.117:0/2329118935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583b9b860a0 0x5583b9f79a00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.771+0000 7fc0a37fe640 0 client.24988 ms_handle_reset on v2:172.21.15.192:6836/2683860842 2024-09-21T05:56:58.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.771+0000 7fc0d0be4740 1 --2- 172.21.15.117:0/2329118935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5583b9b6d8e0 0x5583b9f79440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.771+0000 7fc0d0be4740 1 -- 172.21.15.117:0/2329118935 >> 172.21.15.117:0/2329118935 conn(0x5583b9ba3350 msgr2=0x5583b9f65600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:58.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.771+0000 7fc0d0be4740 1 -- 172.21.15.117:0/2329118935 shutdown_connections 2024-09-21T05:56:58.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.771+0000 7fc0d0be4740 1 -- 172.21.15.117:0/2329118935 wait complete. 2024-09-21T05:56:58.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.771+0000 7fc0d0be4740 20 client.24988 trim_cache size 0 max 16384 2024-09-21T05:56:58.785 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T05:56:58.785 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_fsid": "ac8581a2-77dc-11ef-baf5-efdc52797490", 2024-09-21T05:56:58.785 INFO:teuthology.orchestra.run.smithi117.stdout: "whoami": 2, 2024-09-21T05:56:58.785 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 24628, 2024-09-21T05:56:58.785 INFO:teuthology.orchestra.run.smithi117.stdout: "want_state": "up:active", 2024-09-21T05:56:58.785 INFO:teuthology.orchestra.run.smithi117.stdout: "state": "up:active", 2024-09-21T05:56:58.785 INFO:teuthology.orchestra.run.smithi117.stdout: "fs_name": "cephfs", 2024-09-21T05:56:58.786 INFO:teuthology.orchestra.run.smithi117.stdout: "rank_uptime": 95.582684857000004, 2024-09-21T05:56:58.786 INFO:teuthology.orchestra.run.smithi117.stdout: "mdsmap_epoch": 26, 2024-09-21T05:56:58.786 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch": 76, 2024-09-21T05:56:58.786 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch_barrier": 74, 2024-09-21T05:56:58.786 INFO:teuthology.orchestra.run.smithi117.stdout: "uptime": 129.56176070199999 2024-09-21T05:56:58.786 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T05:56:58.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.772+0000 7fc0ca670640 1 -- 172.21.15.117:0/203558966 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc0b00b65f0 msgr2=0x7fc0b00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:58.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.772+0000 7fc0ca670640 1 --2- 172.21.15.117:0/203558966 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc0b00b65f0 0x7fc0b00b8b00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc0b40058c0 tx=0x7fc0b4005850 comp rx=0 tx=0).stop 2024-09-21T05:56:58.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.772+0000 7fc0ca670640 1 -- 172.21.15.117:0/203558966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0c4108420 msgr2=0x7fc0c419aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:56:58.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.772+0000 7fc0ca670640 1 --2- 172.21.15.117:0/203558966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0c4108420 0x7fc0c419aab0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac00c970 tx=0x7fc0ac00ce30 comp rx=0 tx=0).stop 2024-09-21T05:56:58.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.772+0000 7fc0ca670640 1 -- 172.21.15.117:0/203558966 shutdown_connections 2024-09-21T05:56:58.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.772+0000 7fc0ca670640 1 --2- 172.21.15.117:0/203558966 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc0b00b65f0 0x7fc0b00b8b00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.772+0000 7fc0ca670640 1 --2- 172.21.15.117:0/203558966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0c4108420 0x7fc0c419aab0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.772+0000 7fc0ca670640 1 --2- 172.21.15.117:0/203558966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0c4108d70 0x7fc0c419fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.772+0000 7fc0ca670640 1 --2- 172.21.15.117:0/203558966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0c4102ae0 0x7fc0c419a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:56:58.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.772+0000 7fc0ca670640 1 -- 172.21.15.117:0/203558966 >> 172.21.15.117:0/203558966 conn(0x7fc0c4079770 msgr2=0x7fc0c4106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:56:58.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.772+0000 7fc0ca670640 1 -- 172.21.15.117:0/203558966 shutdown_connections 2024-09-21T05:56:58.788 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:56:58.772+0000 7fc0ca670640 1 -- 172.21.15.117:0/203558966 wait complete. 2024-09-21T05:56:59.572 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.i status 2024-09-21T05:56:59.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:56:59 smithi117 ceph-mon[30947]: pgmap v317: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T05:56:59.817 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:56:59.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:56:59 smithi192 ceph-mon[34700]: pgmap v317: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T05:57:00.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:56:59 smithi138 ceph-mon[34391]: pgmap v317: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-21T05:57:00.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.278+0000 7feb25816640 1 -- 172.21.15.117:0/3512614734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb20102080 msgr2=0x7feb20102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:00.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.278+0000 7feb25816640 1 --2- 172.21.15.117:0/3512614734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb20102080 0x7feb20102490 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7feb10009ca0 tx=0x7feb1002a8c0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.278+0000 7feb25816640 1 -- 172.21.15.117:0/3512614734 shutdown_connections 2024-09-21T05:57:00.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.278+0000 7feb25816640 1 --2- 172.21.15.117:0/3512614734 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb20109eb0 0x7feb201106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.278+0000 7feb25816640 1 --2- 172.21.15.117:0/3512614734 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb20109560 0x7feb20109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.278+0000 7feb25816640 1 --2- 172.21.15.117:0/3512614734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb20102080 0x7feb20102490 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.278+0000 7feb25816640 1 -- 172.21.15.117:0/3512614734 >> 172.21.15.117:0/3512614734 conn(0x7feb20079ec0 msgr2=0x7feb2007a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:00.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.278+0000 7feb25816640 1 -- 172.21.15.117:0/3512614734 shutdown_connections 2024-09-21T05:57:00.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.278+0000 7feb25816640 1 -- 172.21.15.117:0/3512614734 wait complete. 2024-09-21T05:57:00.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.279+0000 7feb25816640 1 Processor -- start 2024-09-21T05:57:00.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.279+0000 7feb25816640 1 -- start start 2024-09-21T05:57:00.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.279+0000 7feb25816640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb20102080 0x7feb2019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:00.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.279+0000 7feb25816640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb20109560 0x7feb2019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:00.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.279+0000 7feb25816640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb20109eb0 0x7feb2019fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:00.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.279+0000 7feb25816640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7feb20113b70 con 0x7feb20102080 2024-09-21T05:57:00.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.279+0000 7feb25816640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feb201139f0 con 0x7feb20109560 2024-09-21T05:57:00.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.279+0000 7feb25816640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7feb20113cf0 con 0x7feb20109eb0 2024-09-21T05:57:00.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.279+0000 7feb1effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb20102080 0x7feb2019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:00.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.279+0000 7feb1effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb20102080 0x7feb2019a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44594/0 (socket says 172.21.15.117:44594) 2024-09-21T05:57:00.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.279+0000 7feb1effd640 1 -- 172.21.15.117:0/3835939140 learned_addr learned my addr 172.21.15.117:0/3835939140 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:00.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.279+0000 7feb1e7fc640 1 --2- 172.21.15.117:0/3835939140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb20109560 0x7feb2019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:00.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.279+0000 7feb1f7fe640 1 --2- 172.21.15.117:0/3835939140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb20109eb0 0x7feb2019fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:00.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.279+0000 7feb1effd640 1 -- 172.21.15.117:0/3835939140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb20109eb0 msgr2=0x7feb2019fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:00.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.279+0000 7feb1effd640 1 --2- 172.21.15.117:0/3835939140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb20109eb0 0x7feb2019fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.279+0000 7feb1effd640 1 -- 172.21.15.117:0/3835939140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb20109560 msgr2=0x7feb2019aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:00.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.279+0000 7feb1effd640 1 --2- 172.21.15.117:0/3835939140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb20109560 0x7feb2019aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.279+0000 7feb1effd640 1 -- 172.21.15.117:0/3835939140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb201a03d0 con 0x7feb20102080 2024-09-21T05:57:00.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.280+0000 7feb1e7fc640 1 --2- 172.21.15.117:0/3835939140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb20109560 0x7feb2019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:57:00.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.280+0000 7feb1effd640 1 --2- 172.21.15.117:0/3835939140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb20102080 0x7feb2019a520 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7feb10006010 tx=0x7feb100319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:00.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.280+0000 7feb24814640 1 -- 172.21.15.117:0/3835939140 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb10003bf0 con 0x7feb20102080 2024-09-21T05:57:00.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.280+0000 7feb24814640 1 -- 172.21.15.117:0/3835939140 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb10003d90 con 0x7feb20102080 2024-09-21T05:57:00.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.280+0000 7feb25816640 1 -- 172.21.15.117:0/3835939140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb201a0660 con 0x7feb20102080 2024-09-21T05:57:00.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.280+0000 7feb24814640 1 -- 172.21.15.117:0/3835939140 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1002fc20 con 0x7feb20102080 2024-09-21T05:57:00.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.280+0000 7feb25816640 1 -- 172.21.15.117:0/3835939140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb201a0ba0 con 0x7feb20102080 2024-09-21T05:57:00.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.282+0000 7feb2bd8a740 1 Processor -- start 2024-09-21T05:57:00.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.282+0000 7feb2bd8a740 1 -- start start 2024-09-21T05:57:00.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.283+0000 7feb2bd8a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576a374ef60 0x5576a3af4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:00.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.283+0000 7feb2bd8a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576a37260a0 0x5576a3af5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:00.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.283+0000 7feb2bd8a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5576a372e280 0x5576a3af5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:00.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.283+0000 7feb2bd8a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5576a3a6cc10 con 0x5576a37260a0 2024-09-21T05:57:00.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.283+0000 7feb2bd8a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5576a3a5c040 con 0x5576a374ef60 2024-09-21T05:57:00.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.283+0000 7feb2bd8a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5576a3a5fe50 con 0x5576a372e280 2024-09-21T05:57:00.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.283+0000 7feb1e7fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5576a372e280 0x5576a3af5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:00.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.283+0000 7feb24814640 1 -- 172.21.15.117:0/3835939140 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7feb10004040 con 0x7feb20102080 2024-09-21T05:57:00.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.283+0000 7feb1e7fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5576a372e280 0x5576a3af5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:60796/0 (socket says 172.21.15.117:60796) 2024-09-21T05:57:00.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.283+0000 7feb1e7fc640 1 -- 172.21.15.117:0/576579004 learned_addr learned my addr 172.21.15.117:0/576579004 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:00.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.283+0000 7feb1f7fe640 1 --2- 172.21.15.117:0/576579004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576a374ef60 0x5576a3af4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:00.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.283+0000 7feb1effd640 1 --2- 172.21.15.117:0/576579004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576a37260a0 0x5576a3af5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:00.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.283+0000 7feb1e7fc640 1 -- 172.21.15.117:0/576579004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576a374ef60 msgr2=0x5576a3af4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:00.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.283+0000 7feb1e7fc640 1 --2- 172.21.15.117:0/576579004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576a374ef60 0x5576a3af4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.283+0000 7feb1e7fc640 1 -- 172.21.15.117:0/576579004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576a37260a0 msgr2=0x5576a3af5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:00.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.283+0000 7feb1e7fc640 1 --2- 172.21.15.117:0/576579004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576a37260a0 0x5576a3af5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.283+0000 7feb1e7fc640 1 -- 172.21.15.117:0/576579004 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576a37ba540 con 0x5576a372e280 2024-09-21T05:57:00.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.284+0000 7feb1e7fc640 1 --2- 172.21.15.117:0/576579004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5576a372e280 0x5576a3af5980 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7feb1400bcd0 tx=0x7feb1400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd3eba210d64659e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:00.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.284+0000 7feb24814640 1 --2- 172.21.15.117:0/3835939140 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7feb080b6620 0x7feb080b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:00.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.284+0000 7feb05ffb640 1 -- 172.21.15.117:0/576579004 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb14015070 con 0x5576a372e280 2024-09-21T05:57:00.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.284+0000 7feb05ffb640 1 -- 172.21.15.117:0/576579004 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb1401a910 con 0x5576a372e280 2024-09-21T05:57:00.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.284+0000 7feb05ffb640 1 -- 172.21.15.117:0/576579004 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb140118c0 con 0x5576a372e280 2024-09-21T05:57:00.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.284+0000 7feb1f7fe640 1 --2- 172.21.15.117:0/3835939140 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7feb080b6620 0x7feb080b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:00.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.284+0000 7feb24814640 1 -- 172.21.15.117:0/3835939140 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7feb10108970 con 0x7feb20102080 2024-09-21T05:57:00.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.285+0000 7feb2bd8a740 1 -- 172.21.15.117:0/576579004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5576a372e280 msgr2=0x5576a3af5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:00.320 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.285+0000 7feb1f7fe640 1 --2- 172.21.15.117:0/3835939140 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7feb080b6620 0x7feb080b8b30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7feb1800cc60 tx=0x7feb1800c430 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:00.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.285+0000 7feb2bd8a740 1 --2- 172.21.15.117:0/576579004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5576a372e280 0x5576a3af5980 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7feb1400bcd0 tx=0x7feb1400f730 comp rx=0 tx=0).stop 2024-09-21T05:57:00.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.285+0000 7feb2bd8a740 1 -- 172.21.15.117:0/576579004 shutdown_connections 2024-09-21T05:57:00.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.285+0000 7feb2bd8a740 1 --2- 172.21.15.117:0/576579004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576a37260a0 0x5576a3af5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.285+0000 7feb2bd8a740 1 --2- 172.21.15.117:0/576579004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5576a372e280 0x5576a3af5980 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.285+0000 7feb2bd8a740 1 --2- 172.21.15.117:0/576579004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576a374ef60 0x5576a3af4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.285+0000 7feb2bd8a740 1 -- 172.21.15.117:0/576579004 >> 172.21.15.117:0/576579004 conn(0x5576a3743350 msgr2=0x5576a3af2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:00.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.285+0000 7feb2bd8a740 1 -- 172.21.15.117:0/576579004 shutdown_connections 2024-09-21T05:57:00.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.285+0000 7feb2bd8a740 1 -- 172.21.15.117:0/576579004 wait complete. 2024-09-21T05:57:00.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.285+0000 7feb2bd8a740 1 -- start start 2024-09-21T05:57:00.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.285+0000 7feb2bd8a740 1 Processor -- start 2024-09-21T05:57:00.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.286+0000 7feb2bd8a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576a37260a0 0x5576a3b009e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:00.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.286+0000 7feb2bd8a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5576a372e280 0x5576a3b00f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:00.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.286+0000 7feb2bd8a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576a374ef60 0x5576a3b0ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:00.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.286+0000 7feb2bd8a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5576a38bd5f0 con 0x5576a37260a0 2024-09-21T05:57:00.321 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.286+0000 7feb2bd8a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5576a39f7010 con 0x5576a374ef60 2024-09-21T05:57:00.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.286+0000 7feb2bd8a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5576a3b036e0 con 0x5576a372e280 2024-09-21T05:57:00.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.286+0000 7feb2bd8a740 10 client.0 _ll_register_callbacks cb 0x5576a3a56340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:57:00.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.286+0000 7feb1effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576a37260a0 0x5576a3b009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:00.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.286+0000 7feb1effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576a37260a0 0x5576a3b009e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44610/0 (socket says 172.21.15.117:44610) 2024-09-21T05:57:00.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.286+0000 7feb1effd640 1 -- 172.21.15.117:0/3386797990 learned_addr learned my addr 172.21.15.117:0/3386797990 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:00.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.286+0000 7feb1e7fc640 1 --2- 172.21.15.117:0/3386797990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5576a372e280 0x5576a3b00f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:00.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.286+0000 7feb1effd640 1 -- 172.21.15.117:0/3386797990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5576a372e280 msgr2=0x5576a3b00f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:00.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.286+0000 7feb1effd640 1 --2- 172.21.15.117:0/3386797990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5576a372e280 0x5576a3b00f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.286+0000 7feb1effd640 1 -- 172.21.15.117:0/3386797990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576a374ef60 msgr2=0x5576a3b0ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:00.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.286+0000 7feb1f7fe640 1 --2- 172.21.15.117:0/3386797990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576a374ef60 0x5576a3b0ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:00.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.286+0000 7feb1effd640 1 --2- 172.21.15.117:0/3386797990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576a374ef60 0x5576a3b0ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.286+0000 7feb1effd640 1 -- 172.21.15.117:0/3386797990 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576a3a24a00 con 0x5576a37260a0 2024-09-21T05:57:00.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.286+0000 7feb1effd640 1 --2- 172.21.15.117:0/3386797990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576a37260a0 0x5576a3b009e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7feb100cbb70 tx=0x7feb100099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4702f29808ff3760 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:00.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.286+0000 7feafeffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:57:00.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.286+0000 7feafeffd640 1 -- 172.21.15.117:0/3386797990 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb100ce6d0 con 0x5576a37260a0 2024-09-21T05:57:00.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.287+0000 7feafeffd640 1 -- 172.21.15.117:0/3386797990 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7feb100046a0 con 0x5576a37260a0 2024-09-21T05:57:00.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.287+0000 7feb2bd8a740 1 -- 172.21.15.117:0/3386797990 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb1401a910 con 0x5576a37260a0 2024-09-21T05:57:00.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.287+0000 7feafeffd640 1 -- 172.21.15.117:0/3386797990 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7feb100ca070 con 0x5576a37260a0 2024-09-21T05:57:00.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.287+0000 7feb2bd8a740 10 client.15420 fetch_fsmap learned FSMap version 26 2024-09-21T05:57:00.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.287+0000 7feb2bd8a740 1 -- 172.21.15.117:0/3386797990 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5576a3b0f5e0 con 0x5576a37260a0 2024-09-21T05:57:00.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.287+0000 7feafeffd640 1 -- 172.21.15.117:0/3386797990 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb1401a910 con 0x5576a37260a0 2024-09-21T05:57:00.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.287+0000 7feafeffd640 1 -- 172.21.15.117:0/3386797990 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7feb100d7e90 con 0x5576a37260a0 2024-09-21T05:57:00.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.287+0000 7feb2bd8a740 10 client.15420 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T05:57:00.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.287+0000 7feb2bd8a740 10 client.15420 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T05:57:00.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.287+0000 7feb2bd8a740 20 client.15420 populate_metadata read hostname 'smithi117' 2024-09-21T05:57:00.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.287+0000 7feb2bd8a740 1 --2- 172.21.15.117:0/3386797990 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x5576a370d8e0 0x5576a3b13090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:00.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.287+0000 7feb2bd8a740 4 client.15420 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:57:00.323 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.287+0000 7feb2bd8a740 1 -- 172.21.15.117:0/3386797990 --> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576a3a6eb60 con 0x5576a370d8e0 2024-09-21T05:57:00.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.287+0000 7feb1e7fc640 1 --2- 172.21.15.117:0/3386797990 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x5576a370d8e0 0x5576a3b13090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:00.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.288+0000 7feb1e7fc640 1 --2- 172.21.15.117:0/3386797990 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x5576a370d8e0 0x5576a3b13090 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6dec8e57d33cb8a server_cookie=ea6c519131108982 in_seq=0 out_seq=0 2024-09-21T05:57:00.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.288+0000 7feafeffd640 10 client.15420 ms_handle_connect on v2:172.21.15.192:6838/1727623231 2024-09-21T05:57:00.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.289+0000 7feafeffd640 1 -- 172.21.15.117:0/3386797990 <== mds.1 v2:172.21.15.192:6838/1727623231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5576a3a6eb60 con 0x5576a370d8e0 2024-09-21T05:57:00.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.289+0000 7feafeffd640 10 client.15420 handle_command_reply: tid=0 2024-09-21T05:57:00.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.289+0000 7feb2bd8a740 1 client.15420 shutdown 2024-09-21T05:57:00.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.289+0000 7feb2bd8a740 1 -- 172.21.15.117:0/3386797990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576a37260a0 msgr2=0x5576a3b009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:00.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.289+0000 7feb2bd8a740 1 --2- 172.21.15.117:0/3386797990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576a37260a0 0x5576a3b009e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7feb100cbb70 tx=0x7feb100099c0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.289+0000 7feb2bd8a740 1 -- 172.21.15.117:0/3386797990 shutdown_connections 2024-09-21T05:57:00.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.289+0000 7feb2bd8a740 1 --2- 172.21.15.117:0/3386797990 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x5576a370d8e0 0x5576a3b13090 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.289+0000 7feb2bd8a740 1 --2- 172.21.15.117:0/3386797990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576a374ef60 0x5576a3b0ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.289+0000 7feafeffd640 0 client.15420 ms_handle_reset on v2:172.21.15.192:6838/1727623231 2024-09-21T05:57:00.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.289+0000 7feb2bd8a740 1 --2- 172.21.15.117:0/3386797990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5576a372e280 0x5576a3b00f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.289+0000 7feb2bd8a740 1 --2- 172.21.15.117:0/3386797990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576a37260a0 0x5576a3b009e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.289+0000 7feb2bd8a740 1 -- 172.21.15.117:0/3386797990 >> 172.21.15.117:0/3386797990 conn(0x5576a3743350 msgr2=0x5576a3af8090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:00.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.289+0000 7feb2bd8a740 1 -- 172.21.15.117:0/3386797990 shutdown_connections 2024-09-21T05:57:00.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.289+0000 7feb2bd8a740 1 -- 172.21.15.117:0/3386797990 wait complete. 2024-09-21T05:57:00.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.289+0000 7feb2bd8a740 20 client.15420 trim_cache size 0 max 16384 2024-09-21T05:57:00.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.307+0000 7feb2bd8a740 1 Processor -- start 2024-09-21T05:57:00.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.307+0000 7feb2bd8a740 1 -- start start 2024-09-21T05:57:00.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.307+0000 7feb2bd8a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5576a37260a0 0x5576a3b1b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:00.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.307+0000 7feb2bd8a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576a372e280 0x5576a3b1bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:00.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.307+0000 7feb2bd8a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576a374ef60 0x5576a3b0c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:00.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.307+0000 7feb2bd8a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5576a3b113d0 con 0x5576a372e280 2024-09-21T05:57:00.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.307+0000 7feb2bd8a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5576a3b11aa0 con 0x5576a374ef60 2024-09-21T05:57:00.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.307+0000 7feb2bd8a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5576a3b12170 con 0x5576a37260a0 2024-09-21T05:57:00.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.307+0000 7feb1e7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576a372e280 0x5576a3b1bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:00.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.307+0000 7feb1e7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576a372e280 0x5576a3b1bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44626/0 (socket says 172.21.15.117:44626) 2024-09-21T05:57:00.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.307+0000 7feb1e7fc640 1 -- 172.21.15.117:0/283000507 learned_addr learned my addr 172.21.15.117:0/283000507 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:00.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.307+0000 7feb1effd640 1 --2- 172.21.15.117:0/283000507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5576a37260a0 0x5576a3b1b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:00.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.307+0000 7feb1e7fc640 1 -- 172.21.15.117:0/283000507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5576a37260a0 msgr2=0x5576a3b1b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:00.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.307+0000 7feb1e7fc640 1 --2- 172.21.15.117:0/283000507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5576a37260a0 0x5576a3b1b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.307+0000 7feb1e7fc640 1 -- 172.21.15.117:0/283000507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576a374ef60 msgr2=0x5576a3b0c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:00.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.307+0000 7feb1e7fc640 1 --2- 172.21.15.117:0/283000507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576a374ef60 0x5576a3b0c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.307+0000 7feb1e7fc640 1 -- 172.21.15.117:0/283000507 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576a3b01aa0 con 0x5576a372e280 2024-09-21T05:57:00.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.308+0000 7feb1e7fc640 1 --2- 172.21.15.117:0/283000507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576a372e280 0x5576a3b1bca0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7feb14011480 tx=0x7feb1400bbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7405e9ce905c0072 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:00.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.308+0000 7feb057fa640 1 -- 172.21.15.117:0/283000507 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb14015040 con 0x5576a372e280 2024-09-21T05:57:00.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.308+0000 7feb057fa640 1 -- 172.21.15.117:0/283000507 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb14004680 con 0x5576a372e280 2024-09-21T05:57:00.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.308+0000 7feb057fa640 1 -- 172.21.15.117:0/283000507 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb14011830 con 0x5576a372e280 2024-09-21T05:57:00.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.308+0000 7feb2bd8a740 1 -- 172.21.15.117:0/283000507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576a372e280 msgr2=0x5576a3b1bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:00.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.308+0000 7feb2bd8a740 1 --2- 172.21.15.117:0/283000507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576a372e280 0x5576a3b1bca0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7feb14011480 tx=0x7feb1400bbd0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.309+0000 7feb2bd8a740 1 -- 172.21.15.117:0/283000507 shutdown_connections 2024-09-21T05:57:00.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.309+0000 7feb2bd8a740 1 --2- 172.21.15.117:0/283000507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576a372e280 0x5576a3b1bca0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.309+0000 7feb2bd8a740 1 --2- 172.21.15.117:0/283000507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576a374ef60 0x5576a3b0c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.309+0000 7feb2bd8a740 1 --2- 172.21.15.117:0/283000507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5576a37260a0 0x5576a3b1b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.309+0000 7feb2bd8a740 1 -- 172.21.15.117:0/283000507 >> 172.21.15.117:0/283000507 conn(0x5576a3743350 msgr2=0x5576a3aff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:00.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.309+0000 7feb2bd8a740 1 -- 172.21.15.117:0/283000507 shutdown_connections 2024-09-21T05:57:00.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.309+0000 7feb2bd8a740 1 -- 172.21.15.117:0/283000507 wait complete. 2024-09-21T05:57:00.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.309+0000 7feb2bd8a740 1 -- start start 2024-09-21T05:57:00.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.309+0000 7feb2bd8a740 1 Processor -- start 2024-09-21T05:57:00.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.310+0000 7feb2bd8a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576a37260a0 0x5576a3b19da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:00.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.310+0000 7feb2bd8a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576a372e280 0x5576a3b1a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:00.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.310+0000 7feb2bd8a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5576a374ef60 0x5576a3b7a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:00.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.310+0000 7feb2bd8a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5576a3b12f10 con 0x5576a37260a0 2024-09-21T05:57:00.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.310+0000 7feb2bd8a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5576a3b12840 con 0x5576a372e280 2024-09-21T05:57:00.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.310+0000 7feb2bd8a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5576a3b135e0 con 0x5576a374ef60 2024-09-21T05:57:00.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.310+0000 7feb2bd8a740 10 client.0 _ll_register_callbacks cb 0x7feb10009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:57:00.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.310+0000 7feb1effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576a37260a0 0x5576a3b19da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:00.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.310+0000 7feb1effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576a37260a0 0x5576a3b19da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44628/0 (socket says 172.21.15.117:44628) 2024-09-21T05:57:00.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.310+0000 7feb1effd640 1 -- 172.21.15.117:0/2370886022 learned_addr learned my addr 172.21.15.117:0/2370886022 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:00.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.310+0000 7feb1e7fc640 1 --2- 172.21.15.117:0/2370886022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576a372e280 0x5576a3b1a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:00.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.310+0000 7feb1f7fe640 1 --2- 172.21.15.117:0/2370886022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5576a374ef60 0x5576a3b7a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:00.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.310+0000 7feb1effd640 1 -- 172.21.15.117:0/2370886022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5576a374ef60 msgr2=0x5576a3b7a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:00.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.310+0000 7feb1effd640 1 --2- 172.21.15.117:0/2370886022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5576a374ef60 0x5576a3b7a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.310+0000 7feb1effd640 1 -- 172.21.15.117:0/2370886022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576a372e280 msgr2=0x5576a3b1a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:00.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.310+0000 7feb1effd640 1 --2- 172.21.15.117:0/2370886022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576a372e280 0x5576a3b1a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.310+0000 7feb1effd640 1 -- 172.21.15.117:0/2370886022 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576a3b7abc0 con 0x5576a37260a0 2024-09-21T05:57:00.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.310+0000 7feb1effd640 1 --2- 172.21.15.117:0/2370886022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576a37260a0 0x5576a3b19da0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7feb10004af0 tx=0x7feb100d32e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6b2a952bc8a7d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:00.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.310+0000 7feafeffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:57:00.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.310+0000 7feafeffd640 1 -- 172.21.15.117:0/2370886022 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb100ce070 con 0x5576a37260a0 2024-09-21T05:57:00.328 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.310+0000 7feafeffd640 1 -- 172.21.15.117:0/2370886022 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7feb10004b40 con 0x5576a37260a0 2024-09-21T05:57:00.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.310+0000 7feb2bd8a740 1 -- 172.21.15.117:0/2370886022 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb14004680 con 0x5576a37260a0 2024-09-21T05:57:00.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.310+0000 7feafeffd640 1 -- 172.21.15.117:0/2370886022 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7feb100ce250 con 0x5576a37260a0 2024-09-21T05:57:00.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.311+0000 7feb2bd8a740 10 client.15426 fetch_fsmap learned FSMap version 26 2024-09-21T05:57:00.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.311+0000 7feb2bd8a740 1 -- 172.21.15.117:0/2370886022 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5576a3b83120 con 0x5576a37260a0 2024-09-21T05:57:00.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.311+0000 7feafeffd640 1 -- 172.21.15.117:0/2370886022 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb14004680 con 0x5576a37260a0 2024-09-21T05:57:00.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.311+0000 7feafeffd640 1 -- 172.21.15.117:0/2370886022 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7feb100ca250 con 0x5576a37260a0 2024-09-21T05:57:00.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.311+0000 7feb2bd8a740 10 client.15426 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T05:57:00.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.311+0000 7feb2bd8a740 10 client.15426 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-21T05:57:00.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.311+0000 7feb2bd8a740 20 client.15426 populate_metadata read hostname 'smithi117' 2024-09-21T05:57:00.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.311+0000 7feb2bd8a740 1 --2- 172.21.15.117:0/2370886022 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x5576a36e2c90 0x5576a3b867b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:00.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.311+0000 7feb2bd8a740 4 client.15426 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T05:57:00.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.311+0000 7feb2bd8a740 1 -- 172.21.15.117:0/2370886022 --> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] -- command(tid 0: {"prefix": "status"}) -- 0x5576a3a6a610 con 0x5576a36e2c90 2024-09-21T05:57:00.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.311+0000 7feb1e7fc640 1 --2- 172.21.15.117:0/2370886022 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x5576a36e2c90 0x5576a3b867b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:00.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.312+0000 7feafeffd640 10 client.15426 ms_handle_connect on v2:172.21.15.192:6838/1727623231 2024-09-21T05:57:00.329 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.312+0000 7feb1e7fc640 1 --2- 172.21.15.117:0/2370886022 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x5576a36e2c90 0x5576a3b867b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=de0826fd5a527204 server_cookie=5d6da2e41c0d69fc in_seq=0 out_seq=0 2024-09-21T05:57:00.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.312+0000 7feafeffd640 1 -- 172.21.15.117:0/2370886022 <== mds.1 v2:172.21.15.192:6838/1727623231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5576a3a6a610 con 0x5576a36e2c90 2024-09-21T05:57:00.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.312+0000 7feafeffd640 10 client.15426 handle_command_reply: tid=0 2024-09-21T05:57:00.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.312+0000 7feb2bd8a740 1 client.15426 shutdown 2024-09-21T05:57:00.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.312+0000 7feb2bd8a740 1 -- 172.21.15.117:0/2370886022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576a37260a0 msgr2=0x5576a3b19da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:00.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.312+0000 7feb2bd8a740 1 --2- 172.21.15.117:0/2370886022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576a37260a0 0x5576a3b19da0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7feb10004af0 tx=0x7feb100d32e0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.312+0000 7feb2bd8a740 1 -- 172.21.15.117:0/2370886022 shutdown_connections 2024-09-21T05:57:00.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.312+0000 7feb2bd8a740 1 --2- 172.21.15.117:0/2370886022 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x5576a36e2c90 0x5576a3b867b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.312+0000 7feb2bd8a740 1 --2- 172.21.15.117:0/2370886022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5576a374ef60 0x5576a3b7a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.312+0000 7feafeffd640 0 client.15426 ms_handle_reset on v2:172.21.15.192:6838/1727623231 2024-09-21T05:57:00.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.312+0000 7feb2bd8a740 1 --2- 172.21.15.117:0/2370886022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576a372e280 0x5576a3b1a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.312+0000 7feb2bd8a740 1 --2- 172.21.15.117:0/2370886022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5576a37260a0 0x5576a3b19da0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.312+0000 7feb2bd8a740 1 -- 172.21.15.117:0/2370886022 >> 172.21.15.117:0/2370886022 conn(0x5576a3743350 msgr2=0x5576a3af2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:00.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.312+0000 7feb2bd8a740 1 -- 172.21.15.117:0/2370886022 shutdown_connections 2024-09-21T05:57:00.330 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.312+0000 7feb2bd8a740 1 -- 172.21.15.117:0/2370886022 wait complete. 2024-09-21T05:57:00.331 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T05:57:00.331 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_fsid": "ac8581a2-77dc-11ef-baf5-efdc52797490", 2024-09-21T05:57:00.331 INFO:teuthology.orchestra.run.smithi117.stdout: "whoami": 1, 2024-09-21T05:57:00.331 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 24637, 2024-09-21T05:57:00.331 INFO:teuthology.orchestra.run.smithi117.stdout: "want_state": "up:active", 2024-09-21T05:57:00.331 INFO:teuthology.orchestra.run.smithi117.stdout: "state": "up:active", 2024-09-21T05:57:00.331 INFO:teuthology.orchestra.run.smithi117.stdout: "fs_name": "cephfs", 2024-09-21T05:57:00.331 INFO:teuthology.orchestra.run.smithi117.stdout: "rank_uptime": 98.131039956999999, 2024-09-21T05:57:00.331 INFO:teuthology.orchestra.run.smithi117.stdout: "mdsmap_epoch": 26, 2024-09-21T05:57:00.331 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch": 76, 2024-09-21T05:57:00.331 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch_barrier": 74, 2024-09-21T05:57:00.331 INFO:teuthology.orchestra.run.smithi117.stdout: "uptime": 125.897652056 2024-09-21T05:57:00.332 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T05:57:00.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.312+0000 7feb2bd8a740 20 client.15426 trim_cache size 0 max 16384 2024-09-21T05:57:00.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.313+0000 7feb25816640 1 -- 172.21.15.117:0/3835939140 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7feb080b6620 msgr2=0x7feb080b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:00.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.313+0000 7feb25816640 1 --2- 172.21.15.117:0/3835939140 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7feb080b6620 0x7feb080b8b30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7feb1800cc60 tx=0x7feb1800c430 comp rx=0 tx=0).stop 2024-09-21T05:57:00.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.313+0000 7feb25816640 1 -- 172.21.15.117:0/3835939140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb20102080 msgr2=0x7feb2019a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:00.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.313+0000 7feb25816640 1 --2- 172.21.15.117:0/3835939140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb20102080 0x7feb2019a520 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7feb10006010 tx=0x7feb100319f0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.313+0000 7feb25816640 1 -- 172.21.15.117:0/3835939140 shutdown_connections 2024-09-21T05:57:00.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.313+0000 7feb25816640 1 --2- 172.21.15.117:0/3835939140 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7feb080b6620 0x7feb080b8b30 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.313+0000 7feb25816640 1 --2- 172.21.15.117:0/3835939140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb20109eb0 0x7feb2019fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.313+0000 7feb25816640 1 --2- 172.21.15.117:0/3835939140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb20109560 0x7feb2019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.313+0000 7feb25816640 1 --2- 172.21.15.117:0/3835939140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb20102080 0x7feb2019a520 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:00.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.313+0000 7feb25816640 1 -- 172.21.15.117:0/3835939140 >> 172.21.15.117:0/3835939140 conn(0x7feb20079ec0 msgr2=0x7feb201075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:00.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.313+0000 7feb25816640 1 -- 172.21.15.117:0/3835939140 shutdown_connections 2024-09-21T05:57:00.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:00.313+0000 7feb25816640 1 -- 172.21.15.117:0/3835939140 wait complete. 2024-09-21T05:57:01.124 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T05:57:01.370 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:57:01.750 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:01 smithi117 ceph-mon[30947]: pgmap v318: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T05:57:01.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.838+0000 7f5b3bd74640 1 -- 172.21.15.117:0/247860244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b34075dd0 msgr2=0x7f5b340761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:01.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.838+0000 7f5b3bd74640 1 --2- 172.21.15.117:0/247860244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b34075dd0 0x7f5b340761e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5b28009900 tx=0x7f5b2802a410 comp rx=0 tx=0).stop 2024-09-21T05:57:01.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.838+0000 7f5b3bd74640 1 -- 172.21.15.117:0/247860244 shutdown_connections 2024-09-21T05:57:01.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.838+0000 7f5b3bd74640 1 --2- 172.21.15.117:0/247860244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b34075dd0 0x7f5b340761e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.838+0000 7f5b3bd74640 1 --2- 172.21.15.117:0/247860244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b34114430 0x7f5b34116900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.838+0000 7f5b3bd74640 1 --2- 172.21.15.117:0/247860244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b34077380 0x7f5b340757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.838+0000 7f5b3bd74640 1 -- 172.21.15.117:0/247860244 >> 172.21.15.117:0/247860244 conn(0x7f5b3407b850 msgr2=0x7f5b3407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:01.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.838+0000 7f5b3bd74640 1 -- 172.21.15.117:0/247860244 shutdown_connections 2024-09-21T05:57:01.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.838+0000 7f5b3bd74640 1 -- 172.21.15.117:0/247860244 wait complete. 2024-09-21T05:57:01.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.838+0000 7f5b3bd74640 1 Processor -- start 2024-09-21T05:57:01.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.839+0000 7f5b3bd74640 1 -- start start 2024-09-21T05:57:01.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.839+0000 7f5b3bd74640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b34075dd0 0x7f5b341a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:01.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.839+0000 7f5b3bd74640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b34077380 0x7f5b341a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:01.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.839+0000 7f5b3bd74640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b34114430 0x7f5b341a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:01.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.839+0000 7f5b3bd74640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3411a0b0 con 0x7f5b34075dd0 2024-09-21T05:57:01.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.839+0000 7f5b3bd74640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5b34119f30 con 0x7f5b34114430 2024-09-21T05:57:01.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.839+0000 7f5b3bd74640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3411a230 con 0x7f5b34077380 2024-09-21T05:57:01.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.839+0000 7f5b39b10640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b34075dd0 0x7f5b341a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:01.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.839+0000 7f5b39b10640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b34075dd0 0x7f5b341a0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44640/0 (socket says 172.21.15.117:44640) 2024-09-21T05:57:01.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.839+0000 7f5b39b10640 1 -- 172.21.15.117:0/1656470600 learned_addr learned my addr 172.21.15.117:0/1656470600 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:01.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.839+0000 7f5b3930f640 1 --2- 172.21.15.117:0/1656470600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b34077380 0x7f5b341a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:01.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.839+0000 7f5b3a311640 1 --2- 172.21.15.117:0/1656470600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b34114430 0x7f5b341a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:01.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.839+0000 7f5b39b10640 1 -- 172.21.15.117:0/1656470600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b34077380 msgr2=0x7f5b341a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:01.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.839+0000 7f5b39b10640 1 --2- 172.21.15.117:0/1656470600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b34077380 0x7f5b341a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.839+0000 7f5b39b10640 1 -- 172.21.15.117:0/1656470600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b34114430 msgr2=0x7f5b341a6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:01.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.839+0000 7f5b39b10640 1 --2- 172.21.15.117:0/1656470600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b34114430 0x7f5b341a6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.839+0000 7f5b39b10640 1 -- 172.21.15.117:0/1656470600 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b341a6a30 con 0x7f5b34075dd0 2024-09-21T05:57:01.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.839+0000 7f5b3a311640 1 --2- 172.21.15.117:0/1656470600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b34114430 0x7f5b341a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:57:01.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.839+0000 7f5b39b10640 1 --2- 172.21.15.117:0/1656470600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b34075dd0 0x7f5b341a0ac0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400b4a0 tx=0x7f5b2400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:01.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.840+0000 7f5b32ffd640 1 -- 172.21.15.117:0/1656470600 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b24014070 con 0x7f5b34075dd0 2024-09-21T05:57:01.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.840+0000 7f5b32ffd640 1 -- 172.21.15.117:0/1656470600 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b24010ac0 con 0x7f5b34075dd0 2024-09-21T05:57:01.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.840+0000 7f5b3bd74640 1 -- 172.21.15.117:0/1656470600 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b341a6d20 con 0x7f5b34075dd0 2024-09-21T05:57:01.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.840+0000 7f5b32ffd640 1 -- 172.21.15.117:0/1656470600 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b24017c30 con 0x7f5b34075dd0 2024-09-21T05:57:01.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.840+0000 7f5b3bd74640 1 -- 172.21.15.117:0/1656470600 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b341ad480 con 0x7f5b34075dd0 2024-09-21T05:57:01.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.842+0000 7f5b32ffd640 1 -- 172.21.15.117:0/1656470600 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7f5b24017070 con 0x7f5b34075dd0 2024-09-21T05:57:01.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.843+0000 7f5b422e8740 1 Processor -- start 2024-09-21T05:57:01.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.843+0000 7f5b422e8740 1 -- start start 2024-09-21T05:57:01.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.843+0000 7f5b422e8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b37070f60 0x555b37416f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:01.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.843+0000 7f5b422e8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b370480a0 0x555b37417440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:01.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.843+0000 7f5b422e8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b37050280 0x555b37417980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:01.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.843+0000 7f5b422e8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555b3738ec10 con 0x555b370480a0 2024-09-21T05:57:01.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.843+0000 7f5b422e8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555b3737e040 con 0x555b37070f60 2024-09-21T05:57:01.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.843+0000 7f5b422e8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555b37381e50 con 0x555b37050280 2024-09-21T05:57:01.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.843+0000 7f5b39b10640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b370480a0 0x555b37417440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:01.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.843+0000 7f5b39b10640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b370480a0 0x555b37417440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44646/0 (socket says 172.21.15.117:44646) 2024-09-21T05:57:01.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.843+0000 7f5b39b10640 1 -- 172.21.15.117:0/50746141 learned_addr learned my addr 172.21.15.117:0/50746141 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:01.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.843+0000 7f5b3a311640 1 --2- 172.21.15.117:0/50746141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b37070f60 0x555b37416f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:01.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.844+0000 7f5b3930f640 1 --2- 172.21.15.117:0/50746141 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b37050280 0x555b37417980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:01.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.844+0000 7f5b39b10640 1 -- 172.21.15.117:0/50746141 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b37050280 msgr2=0x555b37417980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:01.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.844+0000 7f5b39b10640 1 --2- 172.21.15.117:0/50746141 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b37050280 0x555b37417980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.844+0000 7f5b39b10640 1 -- 172.21.15.117:0/50746141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b37070f60 msgr2=0x555b37416f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:01.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.844+0000 7f5b39b10640 1 --2- 172.21.15.117:0/50746141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b37070f60 0x555b37416f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.844+0000 7f5b39b10640 1 -- 172.21.15.117:0/50746141 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b370dc540 con 0x555b370480a0 2024-09-21T05:57:01.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.844+0000 7f5b32ffd640 1 --2- 172.21.15.117:0/1656470600 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5b1c0b65f0 0x7f5b1c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:01.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.844+0000 7f5b3a311640 1 --2- 172.21.15.117:0/1656470600 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5b1c0b65f0 0x7f5b1c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:01.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.844+0000 7f5b39b10640 1 --2- 172.21.15.117:0/50746141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b370480a0 0x555b37417440 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5b240e40b0 tx=0x7f5b240ee670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ddb2a1647e40e48 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:01.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.844+0000 7f5b32ffd640 1 -- 172.21.15.117:0/1656470600 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f5b240e6860 con 0x7f5b34075dd0 2024-09-21T05:57:01.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.844+0000 7f5b1bfff640 1 -- 172.21.15.117:0/50746141 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2401e070 con 0x555b370480a0 2024-09-21T05:57:01.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.844+0000 7f5b1bfff640 1 -- 172.21.15.117:0/50746141 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b24017360 con 0x555b370480a0 2024-09-21T05:57:01.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.844+0000 7f5b1bfff640 1 -- 172.21.15.117:0/50746141 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b240ac5c0 con 0x555b370480a0 2024-09-21T05:57:01.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.845+0000 7f5b422e8740 1 -- 172.21.15.117:0/50746141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b370480a0 msgr2=0x555b37417440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:01.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.845+0000 7f5b422e8740 1 --2- 172.21.15.117:0/50746141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b370480a0 0x555b37417440 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5b240e40b0 tx=0x7f5b240ee670 comp rx=0 tx=0).stop 2024-09-21T05:57:01.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.845+0000 7f5b3a311640 1 --2- 172.21.15.117:0/1656470600 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5b1c0b65f0 0x7f5b1c0b8b00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c005fd0 tx=0x7f5b2c009800 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:01.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.845+0000 7f5b422e8740 1 -- 172.21.15.117:0/50746141 shutdown_connections 2024-09-21T05:57:01.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.845+0000 7f5b422e8740 1 --2- 172.21.15.117:0/50746141 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b370480a0 0x555b37417440 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.845+0000 7f5b422e8740 1 --2- 172.21.15.117:0/50746141 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b37050280 0x555b37417980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.845+0000 7f5b422e8740 1 --2- 172.21.15.117:0/50746141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b37070f60 0x555b37416f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.845+0000 7f5b422e8740 1 -- 172.21.15.117:0/50746141 >> 172.21.15.117:0/50746141 conn(0x555b37065350 msgr2=0x555b37414df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:01.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.845+0000 7f5b422e8740 1 -- 172.21.15.117:0/50746141 shutdown_connections 2024-09-21T05:57:01.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.845+0000 7f5b422e8740 1 -- 172.21.15.117:0/50746141 wait complete. 2024-09-21T05:57:01.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.846+0000 7f5b422e8740 1 -- start start 2024-09-21T05:57:01.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.846+0000 7f5b422e8740 1 Processor -- start 2024-09-21T05:57:01.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.846+0000 7f5b422e8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b370480a0 0x555b37422a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:01.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.846+0000 7f5b422e8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b37050280 0x555b37422f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:01.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.846+0000 7f5b422e8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b37070f60 0x555b3742e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:01.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.846+0000 7f5b422e8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555b371df5f0 con 0x555b370480a0 2024-09-21T05:57:01.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.846+0000 7f5b422e8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555b37319010 con 0x555b37070f60 2024-09-21T05:57:01.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.846+0000 7f5b422e8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555b374256e0 con 0x555b37050280 2024-09-21T05:57:01.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.846+0000 7f5b422e8740 10 client.0 _ll_register_callbacks cb 0x555b37378340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:57:01.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.846+0000 7f5b39b10640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b370480a0 0x555b37422a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:01.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.846+0000 7f5b39b10640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b370480a0 0x555b37422a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44654/0 (socket says 172.21.15.117:44654) 2024-09-21T05:57:01.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.846+0000 7f5b39b10640 1 -- 172.21.15.117:0/1426801812 learned_addr learned my addr 172.21.15.117:0/1426801812 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:01.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.847+0000 7f5b3930f640 1 --2- 172.21.15.117:0/1426801812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b37050280 0x555b37422f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:01.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.847+0000 7f5b3a311640 1 --2- 172.21.15.117:0/1426801812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b37070f60 0x555b3742e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:01.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.847+0000 7f5b39b10640 1 -- 172.21.15.117:0/1426801812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b37050280 msgr2=0x555b37422f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:01.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.847+0000 7f5b39b10640 1 --2- 172.21.15.117:0/1426801812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b37050280 0x555b37422f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.847+0000 7f5b39b10640 1 -- 172.21.15.117:0/1426801812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b37070f60 msgr2=0x555b3742e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:01.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.847+0000 7f5b39b10640 1 --2- 172.21.15.117:0/1426801812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b37070f60 0x555b3742e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.847+0000 7f5b39b10640 1 -- 172.21.15.117:0/1426801812 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b37346a00 con 0x555b370480a0 2024-09-21T05:57:01.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.847+0000 7f5b3a311640 1 --2- 172.21.15.117:0/1426801812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b37070f60 0x555b3742e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:57:01.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.847+0000 7f5b39b10640 1 --2- 172.21.15.117:0/1426801812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b370480a0 0x555b37422a30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5b240e41f0 tx=0x7f5b240e45f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c2402774fada4a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:01.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.847+0000 7f5b197fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:57:01.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.847+0000 7f5b197fa640 1 -- 172.21.15.117:0/1426801812 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2401e050 con 0x555b370480a0 2024-09-21T05:57:01.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.847+0000 7f5b197fa640 1 -- 172.21.15.117:0/1426801812 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f5b240eed20 con 0x555b370480a0 2024-09-21T05:57:01.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.847+0000 7f5b422e8740 1 -- 172.21.15.117:0/1426801812 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555b37390b60 con 0x555b370480a0 2024-09-21T05:57:01.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.847+0000 7f5b197fa640 1 -- 172.21.15.117:0/1426801812 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5b240abf60 con 0x555b370480a0 2024-09-21T05:57:01.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.847+0000 7f5b422e8740 10 client.15438 fetch_fsmap learned FSMap version 26 2024-09-21T05:57:01.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.847+0000 7f5b422e8740 1 -- 172.21.15.117:0/1426801812 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555b37431600 con 0x555b370480a0 2024-09-21T05:57:01.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.847+0000 7f5b197fa640 1 -- 172.21.15.117:0/1426801812 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555b37390b60 con 0x555b370480a0 2024-09-21T05:57:01.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.847+0000 7f5b197fa640 1 -- 172.21.15.117:0/1426801812 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f5b24084030 con 0x555b370480a0 2024-09-21T05:57:01.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.848+0000 7f5b422e8740 10 client.15438 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T05:57:01.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.848+0000 7f5b422e8740 10 client.15438 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T05:57:01.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.848+0000 7f5b422e8740 20 client.15438 populate_metadata read hostname 'smithi117' 2024-09-21T05:57:01.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.848+0000 7f5b422e8740 1 --2- 172.21.15.117:0/1426801812 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555b3702f8e0 0x555b37434ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:01.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.848+0000 7f5b422e8740 4 client.15438 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:57:01.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.848+0000 7f5b422e8740 1 -- 172.21.15.117:0/1426801812 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b3738c610 con 0x555b3702f8e0 2024-09-21T05:57:01.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.848+0000 7f5b3930f640 1 --2- 172.21.15.117:0/1426801812 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555b3702f8e0 0x555b37434ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:01.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.848+0000 7f5b197fa640 10 client.15438 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:57:01.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.848+0000 7f5b3930f640 1 --2- 172.21.15.117:0/1426801812 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555b3702f8e0 0x555b37434ad0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec38c894d711fe83 server_cookie=d242ee4e074e34b6 in_seq=0 out_seq=0 2024-09-21T05:57:01.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.849+0000 7f5b197fa640 1 -- 172.21.15.117:0/1426801812 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555b3738c610 con 0x555b3702f8e0 2024-09-21T05:57:01.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.849+0000 7f5b197fa640 10 client.15438 handle_command_reply: tid=0 2024-09-21T05:57:01.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.849+0000 7f5b422e8740 1 client.15438 shutdown 2024-09-21T05:57:01.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.849+0000 7f5b422e8740 1 -- 172.21.15.117:0/1426801812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b370480a0 msgr2=0x555b37422a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:01.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.849+0000 7f5b422e8740 1 --2- 172.21.15.117:0/1426801812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b370480a0 0x555b37422a30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5b240e41f0 tx=0x7f5b240e45f0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.849+0000 7f5b422e8740 1 -- 172.21.15.117:0/1426801812 shutdown_connections 2024-09-21T05:57:01.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.849+0000 7f5b422e8740 1 --2- 172.21.15.117:0/1426801812 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555b3702f8e0 0x555b37434ad0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.849+0000 7f5b422e8740 1 --2- 172.21.15.117:0/1426801812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b37070f60 0x555b3742e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.849+0000 7f5b422e8740 1 --2- 172.21.15.117:0/1426801812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b37050280 0x555b37422f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.849+0000 7f5b422e8740 1 --2- 172.21.15.117:0/1426801812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b370480a0 0x555b37422a30 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.849+0000 7f5b197fa640 0 client.15438 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:57:01.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.849+0000 7f5b422e8740 1 -- 172.21.15.117:0/1426801812 >> 172.21.15.117:0/1426801812 conn(0x555b37065350 msgr2=0x555b3741a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:01.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.850+0000 7f5b422e8740 1 -- 172.21.15.117:0/1426801812 shutdown_connections 2024-09-21T05:57:01.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.850+0000 7f5b422e8740 1 -- 172.21.15.117:0/1426801812 wait complete. 2024-09-21T05:57:01.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.850+0000 7f5b422e8740 20 client.15438 trim_cache size 0 max 16384 2024-09-21T05:57:01.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.867+0000 7f5b422e8740 1 Processor -- start 2024-09-21T05:57:01.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.867+0000 7f5b422e8740 1 -- start start 2024-09-21T05:57:01.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.868+0000 7f5b422e8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b370480a0 0x555b37414e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:01.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.868+0000 7f5b422e8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b37050280 0x555b3741d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:01.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.868+0000 7f5b422e8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b37070f60 0x555b3742e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:01.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.868+0000 7f5b422e8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555b37432e10 con 0x555b370480a0 2024-09-21T05:57:01.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.868+0000 7f5b422e8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555b374334e0 con 0x555b37070f60 2024-09-21T05:57:01.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.868+0000 7f5b422e8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555b37433bb0 con 0x555b37050280 2024-09-21T05:57:01.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.868+0000 7f5b39b10640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b370480a0 0x555b37414e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:01.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.868+0000 7f5b39b10640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b370480a0 0x555b37414e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44662/0 (socket says 172.21.15.117:44662) 2024-09-21T05:57:01.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.868+0000 7f5b39b10640 1 -- 172.21.15.117:0/1187684477 learned_addr learned my addr 172.21.15.117:0/1187684477 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:01.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.868+0000 7f5b39b10640 1 -- 172.21.15.117:0/1187684477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b37050280 msgr2=0x555b3741d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:01.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.868+0000 7f5b39b10640 1 --2- 172.21.15.117:0/1187684477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b37050280 0x555b3741d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.868+0000 7f5b39b10640 1 -- 172.21.15.117:0/1187684477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b37070f60 msgr2=0x555b3742e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:01.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.868+0000 7f5b39b10640 1 --2- 172.21.15.117:0/1187684477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b37070f60 0x555b3742e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.868+0000 7f5b39b10640 1 -- 172.21.15.117:0/1187684477 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b37419d60 con 0x555b370480a0 2024-09-21T05:57:01.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.868+0000 7f5b39b10640 1 --2- 172.21.15.117:0/1187684477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b370480a0 0x555b37414e40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400be60 tx=0x7f5b24010f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84859b32ad44724 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:01.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.868+0000 7f5b1b7fe640 1 -- 172.21.15.117:0/1187684477 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2401e070 con 0x555b370480a0 2024-09-21T05:57:01.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.868+0000 7f5b1b7fe640 1 -- 172.21.15.117:0/1187684477 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b240ac300 con 0x555b370480a0 2024-09-21T05:57:01.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.868+0000 7f5b1b7fe640 1 -- 172.21.15.117:0/1187684477 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b240ac140 con 0x555b370480a0 2024-09-21T05:57:01.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.869+0000 7f5b422e8740 1 -- 172.21.15.117:0/1187684477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b370480a0 msgr2=0x555b37414e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:01.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.869+0000 7f5b422e8740 1 --2- 172.21.15.117:0/1187684477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b370480a0 0x555b37414e40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400be60 tx=0x7f5b24010f00 comp rx=0 tx=0).stop 2024-09-21T05:57:01.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.869+0000 7f5b422e8740 1 -- 172.21.15.117:0/1187684477 shutdown_connections 2024-09-21T05:57:01.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.869+0000 7f5b422e8740 1 --2- 172.21.15.117:0/1187684477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b37070f60 0x555b3742e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.869+0000 7f5b422e8740 1 --2- 172.21.15.117:0/1187684477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b37050280 0x555b3741d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.869+0000 7f5b422e8740 1 --2- 172.21.15.117:0/1187684477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b370480a0 0x555b37414e40 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.869+0000 7f5b422e8740 1 -- 172.21.15.117:0/1187684477 >> 172.21.15.117:0/1187684477 conn(0x555b37065350 msgr2=0x555b37414730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:01.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.869+0000 7f5b422e8740 1 -- 172.21.15.117:0/1187684477 shutdown_connections 2024-09-21T05:57:01.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.869+0000 7f5b422e8740 1 -- 172.21.15.117:0/1187684477 wait complete. 2024-09-21T05:57:01.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.870+0000 7f5b422e8740 1 -- start start 2024-09-21T05:57:01.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.870+0000 7f5b422e8740 1 Processor -- start 2024-09-21T05:57:01.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.870+0000 7f5b422e8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b370480a0 0x555b3743d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:01.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.870+0000 7f5b422e8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b37050280 0x555b374367a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:01.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.870+0000 7f5b422e8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b37070f60 0x555b37436ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:01.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.870+0000 7f5b422e8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555b37434950 con 0x555b37070f60 2024-09-21T05:57:01.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.870+0000 7f5b422e8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555b37434280 con 0x555b37050280 2024-09-21T05:57:01.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.870+0000 7f5b422e8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555b37435020 con 0x555b370480a0 2024-09-21T05:57:01.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.870+0000 7f5b422e8740 10 client.0 _ll_register_callbacks cb 0x7f5b2c0096a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:57:01.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.870+0000 7f5b3a311640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b37070f60 0x555b37436ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:01.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.870+0000 7f5b3a311640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b37070f60 0x555b37436ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44672/0 (socket says 172.21.15.117:44672) 2024-09-21T05:57:01.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.870+0000 7f5b3a311640 1 -- 172.21.15.117:0/855587308 learned_addr learned my addr 172.21.15.117:0/855587308 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:01.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.870+0000 7f5b39b10640 1 --2- 172.21.15.117:0/855587308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b370480a0 0x555b3743d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:01.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.870+0000 7f5b3930f640 1 --2- 172.21.15.117:0/855587308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b37050280 0x555b374367a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:01.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.871+0000 7f5b3a311640 1 -- 172.21.15.117:0/855587308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b370480a0 msgr2=0x555b3743d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:01.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.871+0000 7f5b3a311640 1 --2- 172.21.15.117:0/855587308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b370480a0 0x555b3743d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.871+0000 7f5b3a311640 1 -- 172.21.15.117:0/855587308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b37050280 msgr2=0x555b374367a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:01.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.871+0000 7f5b3a311640 1 --2- 172.21.15.117:0/855587308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b37050280 0x555b374367a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.871+0000 7f5b3a311640 1 -- 172.21.15.117:0/855587308 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b37437220 con 0x555b37070f60 2024-09-21T05:57:01.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.871+0000 7f5b39b10640 1 --2- 172.21.15.117:0/855587308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b370480a0 0x555b3743d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:57:01.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.871+0000 7f5b3930f640 1 --2- 172.21.15.117:0/855587308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b37050280 0x555b374367a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:57:01.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.871+0000 7f5b3a311640 1 --2- 172.21.15.117:0/855587308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b37070f60 0x555b37436ce0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c011d90 tx=0x7f5b2c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a1093b52493849 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:01.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.871+0000 7f5b197fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:57:01.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.871+0000 7f5b197fa640 1 -- 172.21.15.117:0/855587308 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c01fe60 con 0x555b37070f60 2024-09-21T05:57:01.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.871+0000 7f5b197fa640 1 -- 172.21.15.117:0/855587308 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f5b2c004d20 con 0x555b37070f60 2024-09-21T05:57:01.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.871+0000 7f5b422e8740 1 -- 172.21.15.117:0/855587308 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b240ac300 con 0x555b37070f60 2024-09-21T05:57:01.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.871+0000 7f5b197fa640 1 -- 172.21.15.117:0/855587308 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5b2c005020 con 0x555b37070f60 2024-09-21T05:57:01.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.871+0000 7f5b422e8740 10 client.15444 fetch_fsmap learned FSMap version 26 2024-09-21T05:57:01.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.871+0000 7f5b422e8740 1 -- 172.21.15.117:0/855587308 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555b3743cc00 con 0x555b37070f60 2024-09-21T05:57:01.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.871+0000 7f5b197fa640 1 -- 172.21.15.117:0/855587308 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b240ac300 con 0x555b37070f60 2024-09-21T05:57:01.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.871+0000 7f5b197fa640 1 -- 172.21.15.117:0/855587308 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f5b2c026030 con 0x555b37070f60 2024-09-21T05:57:01.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.872+0000 7f5b422e8740 10 client.15444 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T05:57:01.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.872+0000 7f5b422e8740 10 client.15444 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T05:57:01.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.872+0000 7f5b422e8740 20 client.15444 populate_metadata read hostname 'smithi117' 2024-09-21T05:57:01.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.872+0000 7f5b422e8740 1 --2- 172.21.15.117:0/855587308 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555b37004c90 0x555b374a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:01.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.872+0000 7f5b422e8740 4 client.15444 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T05:57:01.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.872+0000 7f5b422e8740 1 -- 172.21.15.117:0/855587308 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "status"}) -- 0x555b37389a40 con 0x555b37004c90 2024-09-21T05:57:01.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.872+0000 7f5b39b10640 1 --2- 172.21.15.117:0/855587308 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555b37004c90 0x555b374a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:01.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.872+0000 7f5b39b10640 1 --2- 172.21.15.117:0/855587308 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555b37004c90 0x555b374a35e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6f19b8ac4728426 server_cookie=8f10b2b4e648a59d in_seq=0 out_seq=0 2024-09-21T05:57:01.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.872+0000 7f5b197fa640 10 client.15444 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:57:01.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.873+0000 7f5b197fa640 1 -- 172.21.15.117:0/855587308 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x555b37389a40 con 0x555b37004c90 2024-09-21T05:57:01.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.873+0000 7f5b197fa640 10 client.15444 handle_command_reply: tid=0 2024-09-21T05:57:01.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.873+0000 7f5b422e8740 1 client.15444 shutdown 2024-09-21T05:57:01.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.873+0000 7f5b422e8740 1 -- 172.21.15.117:0/855587308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b37070f60 msgr2=0x555b37436ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:01.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.873+0000 7f5b422e8740 1 --2- 172.21.15.117:0/855587308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b37070f60 0x555b37436ce0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c011d90 tx=0x7f5b2c004ab0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.873+0000 7f5b422e8740 1 -- 172.21.15.117:0/855587308 shutdown_connections 2024-09-21T05:57:01.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.873+0000 7f5b422e8740 1 --2- 172.21.15.117:0/855587308 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555b37004c90 0x555b374a35e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.873+0000 7f5b422e8740 1 --2- 172.21.15.117:0/855587308 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b37070f60 0x555b37436ce0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.873+0000 7f5b197fa640 0 client.15444 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:57:01.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.873+0000 7f5b422e8740 1 --2- 172.21.15.117:0/855587308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b37050280 0x555b374367a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.873+0000 7f5b422e8740 1 --2- 172.21.15.117:0/855587308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b370480a0 0x555b3743d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.873+0000 7f5b422e8740 1 -- 172.21.15.117:0/855587308 >> 172.21.15.117:0/855587308 conn(0x555b37065350 msgr2=0x555b374133b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:01.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.873+0000 7f5b422e8740 1 -- 172.21.15.117:0/855587308 shutdown_connections 2024-09-21T05:57:01.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.873+0000 7f5b422e8740 1 -- 172.21.15.117:0/855587308 wait complete. 2024-09-21T05:57:01.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.873+0000 7f5b422e8740 20 client.15444 trim_cache size 0 max 16384 2024-09-21T05:57:01.880 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T05:57:01.880 INFO:teuthology.orchestra.run.smithi117.stdout: "cluster_fsid": "ac8581a2-77dc-11ef-baf5-efdc52797490", 2024-09-21T05:57:01.880 INFO:teuthology.orchestra.run.smithi117.stdout: "whoami": 0, 2024-09-21T05:57:01.881 INFO:teuthology.orchestra.run.smithi117.stdout: "id": 24638, 2024-09-21T05:57:01.881 INFO:teuthology.orchestra.run.smithi117.stdout: "want_state": "up:active", 2024-09-21T05:57:01.881 INFO:teuthology.orchestra.run.smithi117.stdout: "state": "up:active", 2024-09-21T05:57:01.881 INFO:teuthology.orchestra.run.smithi117.stdout: "fs_name": "cephfs", 2024-09-21T05:57:01.881 INFO:teuthology.orchestra.run.smithi117.stdout: "rank_uptime": 106.888356426, 2024-09-21T05:57:01.881 INFO:teuthology.orchestra.run.smithi117.stdout: "mdsmap_epoch": 26, 2024-09-21T05:57:01.881 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch": 76, 2024-09-21T05:57:01.881 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_epoch_barrier": 74, 2024-09-21T05:57:01.881 INFO:teuthology.orchestra.run.smithi117.stdout: "uptime": 117.759353346 2024-09-21T05:57:01.881 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T05:57:01.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.874+0000 7f5b3bd74640 1 -- 172.21.15.117:0/1656470600 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5b1c0b65f0 msgr2=0x7f5b1c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:01.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.874+0000 7f5b3bd74640 1 --2- 172.21.15.117:0/1656470600 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5b1c0b65f0 0x7f5b1c0b8b00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c005fd0 tx=0x7f5b2c009800 comp rx=0 tx=0).stop 2024-09-21T05:57:01.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.874+0000 7f5b3bd74640 1 -- 172.21.15.117:0/1656470600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b34075dd0 msgr2=0x7f5b341a0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:01.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.874+0000 7f5b3bd74640 1 --2- 172.21.15.117:0/1656470600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b34075dd0 0x7f5b341a0ac0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400b4a0 tx=0x7f5b2400b960 comp rx=0 tx=0).stop 2024-09-21T05:57:01.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.874+0000 7f5b3bd74640 1 -- 172.21.15.117:0/1656470600 shutdown_connections 2024-09-21T05:57:01.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.874+0000 7f5b3bd74640 1 --2- 172.21.15.117:0/1656470600 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5b1c0b65f0 0x7f5b1c0b8b00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.874+0000 7f5b3bd74640 1 --2- 172.21.15.117:0/1656470600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b34114430 0x7f5b341a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.874+0000 7f5b3bd74640 1 --2- 172.21.15.117:0/1656470600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b34077380 0x7f5b341a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.874+0000 7f5b3bd74640 1 --2- 172.21.15.117:0/1656470600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b34075dd0 0x7f5b341a0ac0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:01.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.874+0000 7f5b3bd74640 1 -- 172.21.15.117:0/1656470600 >> 172.21.15.117:0/1656470600 conn(0x7f5b3407b850 msgr2=0x7f5b341130e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:01.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.874+0000 7f5b3bd74640 1 -- 172.21.15.117:0/1656470600 shutdown_connections 2024-09-21T05:57:01.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:01.874+0000 7f5b3bd74640 1 -- 172.21.15.117:0/1656470600 wait complete. 2024-09-21T05:57:01.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:01 smithi192 ceph-mon[34700]: pgmap v318: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T05:57:02.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:01 smithi138 ceph-mon[34391]: pgmap v318: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-21T05:57:02.653 INFO:teuthology.run_tasks:Running task kclient... 2024-09-21T05:57:02.683 INFO:tasks.kclient:Mounting kernel clients... 2024-09-21T05:57:02.683 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-21T05:57:02.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:57:02.683 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list 2024-09-21T05:57:02.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:57:02.700 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link delete ceph-brx 2024-09-21T05:57:02.772 INFO:teuthology.orchestra.run.smithi117.stderr:Cannot find device "ceph-brx" 2024-09-21T05:57:02.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:57:02.774 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-09-21T05:57:02.774 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T05:57:02.774 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T05:57:02.774 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs ls 2024-09-21T05:57:03.060 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:57:03.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.548+0000 7fddbef3d640 1 -- 172.21.15.117:0/1162873731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb8108cf0 msgr2=0x7fddb810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:03.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.548+0000 7fddbef3d640 1 --2- 172.21.15.117:0/1162873731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb8108cf0 0x7fddb810f520 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fddb0009900 tx=0x7fddb002a410 comp rx=0 tx=0).stop 2024-09-21T05:57:03.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.548+0000 7fddbef3d640 1 -- 172.21.15.117:0/1162873731 shutdown_connections 2024-09-21T05:57:03.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.548+0000 7fddbef3d640 1 --2- 172.21.15.117:0/1162873731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb8108cf0 0x7fddb810f520 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:03.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.548+0000 7fddbef3d640 1 --2- 172.21.15.117:0/1162873731 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fddb81083a0 0x7fddb81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:03.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.548+0000 7fddbef3d640 1 --2- 172.21.15.117:0/1162873731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddb8102a60 0x7fddb8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:03.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.548+0000 7fddbef3d640 1 -- 172.21.15.117:0/1162873731 >> 172.21.15.117:0/1162873731 conn(0x7fddb8079730 msgr2=0x7fddb8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:03.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.548+0000 7fddbef3d640 1 -- 172.21.15.117:0/1162873731 shutdown_connections 2024-09-21T05:57:03.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.548+0000 7fddbef3d640 1 -- 172.21.15.117:0/1162873731 wait complete. 2024-09-21T05:57:03.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.549+0000 7fddbef3d640 1 Processor -- start 2024-09-21T05:57:03.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.549+0000 7fddbef3d640 1 -- start start 2024-09-21T05:57:03.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.549+0000 7fddbef3d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb8102a60 0x7fddb819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:03.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.549+0000 7fddbef3d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fddb81083a0 0x7fddb819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:03.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.549+0000 7fddbef3d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddb8108cf0 0x7fddb819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:03.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.549+0000 7fddbef3d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fddb8113ab0 con 0x7fddb8102a60 2024-09-21T05:57:03.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.549+0000 7fddbef3d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fddb8113930 con 0x7fddb8108cf0 2024-09-21T05:57:03.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.549+0000 7fddbef3d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fddb8113c30 con 0x7fddb81083a0 2024-09-21T05:57:03.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.549+0000 7fddbccd9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb8102a60 0x7fddb819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:03.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.549+0000 7fddbccd9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb8102a60 0x7fddb819a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44688/0 (socket says 172.21.15.117:44688) 2024-09-21T05:57:03.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.549+0000 7fddbccd9640 1 -- 172.21.15.117:0/3496838211 learned_addr learned my addr 172.21.15.117:0/3496838211 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:03.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.549+0000 7fddb7fff640 1 --2- 172.21.15.117:0/3496838211 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fddb81083a0 0x7fddb819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:03.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.549+0000 7fddbd4da640 1 --2- 172.21.15.117:0/3496838211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddb8108cf0 0x7fddb819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:03.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.549+0000 7fddbccd9640 1 -- 172.21.15.117:0/3496838211 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fddb81083a0 msgr2=0x7fddb819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:03.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.549+0000 7fddbccd9640 1 --2- 172.21.15.117:0/3496838211 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fddb81083a0 0x7fddb819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:03.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.549+0000 7fddbccd9640 1 -- 172.21.15.117:0/3496838211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddb8108cf0 msgr2=0x7fddb819fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:03.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.549+0000 7fddbccd9640 1 --2- 172.21.15.117:0/3496838211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddb8108cf0 0x7fddb819fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:03.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.550+0000 7fddbccd9640 1 -- 172.21.15.117:0/3496838211 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb81a03b0 con 0x7fddb8102a60 2024-09-21T05:57:03.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.550+0000 7fddbd4da640 1 --2- 172.21.15.117:0/3496838211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddb8108cf0 0x7fddb819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:57:03.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.550+0000 7fddbccd9640 1 --2- 172.21.15.117:0/3496838211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb8102a60 0x7fddb819a440 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdda000eab0 tx=0x7fdda000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:03.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.550+0000 7fddb5ffb640 1 -- 172.21.15.117:0/3496838211 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda000ce40 con 0x7fddb8102a60 2024-09-21T05:57:03.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.550+0000 7fddb5ffb640 1 -- 172.21.15.117:0/3496838211 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdda0011920 con 0x7fddb8102a60 2024-09-21T05:57:03.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.550+0000 7fddbef3d640 1 -- 172.21.15.117:0/3496838211 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddb81a06a0 con 0x7fddb8102a60 2024-09-21T05:57:03.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.550+0000 7fddb5ffb640 1 -- 172.21.15.117:0/3496838211 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda0010a30 con 0x7fddb8102a60 2024-09-21T05:57:03.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.550+0000 7fddbef3d640 1 -- 172.21.15.117:0/3496838211 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddb81a6e00 con 0x7fddb8102a60 2024-09-21T05:57:03.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.551+0000 7fddbef3d640 1 -- 172.21.15.117:0/3496838211 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddb8102e70 con 0x7fddb8102a60 2024-09-21T05:57:03.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.552+0000 7fddb5ffb640 1 -- 172.21.15.117:0/3496838211 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) ==== 154581+0+0 (secure 0 0 0) 0x7fdda0010bd0 con 0x7fddb8102a60 2024-09-21T05:57:03.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.553+0000 7fddb5ffb640 1 --2- 172.21.15.117:0/3496838211 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fdda40b65f0 0x7fdda40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:03.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.553+0000 7fddb7fff640 1 --2- 172.21.15.117:0/3496838211 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fdda40b65f0 0x7fdda40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:03.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.553+0000 7fddb5ffb640 1 -- 172.21.15.117:0/3496838211 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fdda00e7bb0 con 0x7fddb8102a60 2024-09-21T05:57:03.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.554+0000 7fddb7fff640 1 --2- 172.21.15.117:0/3496838211 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fdda40b65f0 0x7fdda40b8b00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fddb8105ec0 tx=0x7fdda800b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:03.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.556+0000 7fddb5ffb640 1 -- 172.21.15.117:0/3496838211 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdda0016030 con 0x7fddb8102a60 2024-09-21T05:57:03.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:03 smithi117 ceph-mon[30947]: pgmap v319: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T05:57:03.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.775+0000 7fddbef3d640 1 -- 172.21.15.117:0/3496838211 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fddb802e640 con 0x7fddb8102a60 2024-09-21T05:57:03.777 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.775+0000 7fddb5ffb640 1 -- 172.21.15.117:0/3496838211 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7fdda00ae240 con 0x7fddb8102a60 2024-09-21T05:57:03.777 INFO:teuthology.orchestra.run.smithi117.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T05:57:03.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.778+0000 7fddbef3d640 1 -- 172.21.15.117:0/3496838211 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fdda40b65f0 msgr2=0x7fdda40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:03.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.778+0000 7fddbef3d640 1 --2- 172.21.15.117:0/3496838211 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fdda40b65f0 0x7fdda40b8b00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fddb8105ec0 tx=0x7fdda800b390 comp rx=0 tx=0).stop 2024-09-21T05:57:03.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.778+0000 7fddbef3d640 1 -- 172.21.15.117:0/3496838211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb8102a60 msgr2=0x7fddb819a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:03.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.778+0000 7fddbef3d640 1 --2- 172.21.15.117:0/3496838211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb8102a60 0x7fddb819a440 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdda000eab0 tx=0x7fdda000ef70 comp rx=0 tx=0).stop 2024-09-21T05:57:03.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.778+0000 7fddbef3d640 1 -- 172.21.15.117:0/3496838211 shutdown_connections 2024-09-21T05:57:03.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.778+0000 7fddbef3d640 1 --2- 172.21.15.117:0/3496838211 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fdda40b65f0 0x7fdda40b8b00 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:03.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.778+0000 7fddbef3d640 1 --2- 172.21.15.117:0/3496838211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddb8108cf0 0x7fddb819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:03.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.778+0000 7fddbef3d640 1 --2- 172.21.15.117:0/3496838211 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fddb81083a0 0x7fddb819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:03.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.778+0000 7fddbef3d640 1 --2- 172.21.15.117:0/3496838211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddb8102a60 0x7fddb819a440 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:03.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.778+0000 7fddbef3d640 1 -- 172.21.15.117:0/3496838211 >> 172.21.15.117:0/3496838211 conn(0x7fddb8079730 msgr2=0x7fddb8104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:03.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.778+0000 7fddbef3d640 1 -- 172.21.15.117:0/3496838211 shutdown_connections 2024-09-21T05:57:03.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:03.778+0000 7fddbef3d640 1 -- 172.21.15.117:0/3496838211 wait complete. 2024-09-21T05:57:03.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:03 smithi192 ceph-mon[34700]: pgmap v319: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T05:57:04.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:03 smithi138 ceph-mon[34391]: pgmap v319: 97 pgs: 97 active+clean; 647 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T05:57:04.569 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T05:57:04.569 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T05:57:04.570 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi117.front.sepia.ceph.com 2024-09-21T05:57:04.570 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T05:57:04.570 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T05:57:04.570 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T05:57:04.570 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T05:57:04.570 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T05:57:04.570 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:57:04.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:57:04.571 DEBUG:teuthology.orchestra.run.smithi117:> ip addr 2024-09-21T05:57:04.586 INFO:teuthology.orchestra.run.smithi117.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T05:57:04.586 INFO:teuthology.orchestra.run.smithi117.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T05:57:04.587 INFO:teuthology.orchestra.run.smithi117.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T05:57:04.587 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:57:04.587 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 ::1/128 scope host 2024-09-21T05:57:04.587 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:57:04.587 INFO:teuthology.orchestra.run.smithi117.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:57:04.587 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:d8:a8:8a brd ff:ff:ff:ff:ff:ff 2024-09-21T05:57:04.587 INFO:teuthology.orchestra.run.smithi117.stdout: altname enp7s0f0 2024-09-21T05:57:04.587 INFO:teuthology.orchestra.run.smithi117.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:57:04.587 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:d8:a8:8b brd ff:ff:ff:ff:ff:ff 2024-09-21T05:57:04.587 INFO:teuthology.orchestra.run.smithi117.stdout: altname enp7s0f1 2024-09-21T05:57:04.587 INFO:teuthology.orchestra.run.smithi117.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:57:04.587 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:8f:44:62 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:57:04.587 INFO:teuthology.orchestra.run.smithi117.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T05:57:04.587 INFO:teuthology.orchestra.run.smithi117.stdout: link/ether 0c:c4:7a:8f:44:63 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:57:04.588 INFO:teuthology.orchestra.run.smithi117.stdout: inet 172.21.15.117/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T05:57:04.588 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft 42124sec preferred_lft 42124sec 2024-09-21T05:57:04.588 INFO:teuthology.orchestra.run.smithi117.stdout: inet6 fe80::ec4:7aff:fe8f:4463/64 scope link noprefixroute 2024-09-21T05:57:04.588 INFO:teuthology.orchestra.run.smithi117.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:57:04.588 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T05:57:04.588 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:57:04.588 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-09-21T05:57:04.588 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link add name ceph-brx type bridge 2024-09-21T05:57:04.588 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip addr flush dev ceph-brx 2024-09-21T05:57:04.589 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set ceph-brx up 2024-09-21T05:57:04.589 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T05:57:04.589 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-09-21T05:57:04.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:57:04.793 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:04 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3496838211' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T05:57:04.796 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:57:04.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:57:04.803 DEBUG:teuthology.orchestra.run.smithi117:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T05:57:04.828 INFO:teuthology.orchestra.run.smithi117.stdout:1 2024-09-21T05:57:04.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:57:04.830 DEBUG:teuthology.orchestra.run.smithi117:> route 2024-09-21T05:57:04.896 INFO:teuthology.orchestra.run.smithi117.stdout:Kernel IP routing table 2024-09-21T05:57:04.896 INFO:teuthology.orchestra.run.smithi117.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T05:57:04.896 INFO:teuthology.orchestra.run.smithi117.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T05:57:04.896 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T05:57:04.896 INFO:teuthology.orchestra.run.smithi117.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T05:57:04.897 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:57:04.897 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-09-21T05:57:04.897 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T05:57:04.897 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T05:57:04.897 DEBUG:teuthology.orchestra.run.smithi117:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T05:57:04.897 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-09-21T05:57:04.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:04 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3496838211' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T05:57:04.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:57:05.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:04 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3496838211' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T05:57:05.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:57:05.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:57:05.208 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list 2024-09-21T05:57:05.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:57:05.224 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list-id 2024-09-21T05:57:05.279 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:57:05.279 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-09-21T05:57:05.279 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T05:57:05.280 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-21T05:57:05.280 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-09-21T05:57:05.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:57:05.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:57:05.396 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-21T05:57:05.397 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:57:05.397 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-09-21T05:57:05.397 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-21T05:57:05.397 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T05:57:05.397 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-21T05:57:05.397 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-21T05:57:05.397 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-21T05:57:05.397 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-09-21T05:57:05.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:57:05.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:57:05.523 DEBUG:teuthology.orchestra.run.smithi117:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:57:05.523 DEBUG:teuthology.orchestra.run.smithi117:> set -e 2024-09-21T05:57:05.523 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set brx.0 up 2024-09-21T05:57:05.523 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T05:57:05.524 DEBUG:teuthology.orchestra.run.smithi117:> ') 2024-09-21T05:57:05.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:57:05.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:57:05.599 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:57:05.599 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T05:57:05.614 INFO:teuthology.orchestra.run.smithi117.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:57:05.615 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:57:05.615 DEBUG:teuthology.orchestra.run.smithi117:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:57:05.664 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-21T05:57:05.664 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-21T05:57:05.664 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/16a073f7-1cc5-454e-a604-c30c5a2cdf02 2024-09-21T05:57:05.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:57:05.665 DEBUG:teuthology.orchestra.run.smithi117:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/16a073f7-1cc5-454e-a604-c30c5a2cdf02 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-21T05:57:05.769 INFO:teuthology.orchestra.run.smithi117.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-21T05:57:05.769 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-09-21T05:57:05.769 INFO:teuthology.orchestra.run.smithi117.stdout:invalid new device string format 2024-09-21T05:57:05.889 INFO:teuthology.orchestra.run.smithi117.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-21T05:57:05.889 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-09-21T05:57:05.890 INFO:teuthology.orchestra.run.smithi117.stdout:invalid new device string format 2024-09-21T05:57:05.890 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: resolved to: "172.21.15.117:3300,172.21.15.138:3300,172.21.15.192:3300" 2024-09-21T05:57:05.890 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.117:3300,172.21.15.138:3300,172.21.15.192:3300:/volumes/qa/sv_1/16a073f7-1cc5-454e-a604-c30c5a2cdf02 2024-09-21T05:57:05.890 INFO:teuthology.orchestra.run.smithi117.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=ac8581a2-77dc-11ef-baf5-efdc52797490" will pass to kernel 2024-09-21T05:57:05.890 INFO:teuthology.orchestra.run.smithi117.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T05:57:05.890 INFO:teuthology.orchestra.run.smithi117.stdout: You just mounted a file system that supports labels which does not 2024-09-21T05:57:05.890 INFO:teuthology.orchestra.run.smithi117.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T05:57:05.891 INFO:teuthology.orchestra.run.smithi117.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T05:57:05.891 INFO:teuthology.orchestra.run.smithi117.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T05:57:05.892 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T05:57:05.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:57:05.892 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:57:05.913 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:05 smithi117 ceph-mon[30947]: pgmap v320: 97 pgs: 97 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-21T05:57:05.913 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:05 smithi117 ceph-mon[30947]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-09-21T05:57:05.918 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:57:05.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:05 smithi192 ceph-mon[34700]: pgmap v320: 97 pgs: 97 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-21T05:57:05.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:05 smithi192 ceph-mon[34700]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-09-21T05:57:06.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:57:06.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:57:06.018 INFO:teuthology.orchestra.run.smithi117.stdout:client25007 2024-09-21T05:57:06.018 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-21T05:57:06.019 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:57:06.035 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:57:06.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:05 smithi138 ceph-mon[34391]: pgmap v320: 97 pgs: 97 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-21T05:57:06.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:05 smithi138 ceph-mon[34391]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-09-21T05:57:06.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:57:06.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:57:06.131 INFO:teuthology.orchestra.run.smithi117.stdout:client25007 2024-09-21T05:57:06.131 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:57:06.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:57:06.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:57:06.187 INFO:teuthology.orchestra.run.smithi117.stdout:client25007 2024-09-21T05:57:06.187 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490.client25007/status') 2024-09-21T05:57:06.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:57:06.246 INFO:teuthology.orchestra.run.smithi117.stdout:instance: client.25007 (3)172.21.15.117:0/932646808 2024-09-21T05:57:06.246 INFO:teuthology.orchestra.run.smithi117.stdout:blocklisted: false 2024-09-21T05:57:06.246 INFO:teuthology.orchestra.run.smithi117.stderr:0+1 records in 2024-09-21T05:57:06.246 INFO:teuthology.orchestra.run.smithi117.stderr:0+1 records out 2024-09-21T05:57:06.246 INFO:teuthology.orchestra.run.smithi117.stderr:71 bytes copied, 2.3389e-05 s, 3.0 MB/s 2024-09-21T05:57:06.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:57:06.253 INFO:teuthology.run_tasks:Running task exec... 2024-09-21T05:57:06.263 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T05:57:06.264 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi117.front.sepia.ceph.com 2024-09-21T05:57:06.264 DEBUG:teuthology.orchestra.run.smithi117:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-21T05:57:06.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.372+0000 7f3ebfb39640 1 -- 172.21.15.117:0/432814559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3eb8109430 msgr2=0x7f3eb8109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:06.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.372+0000 7f3ebfb39640 1 --2- 172.21.15.117:0/432814559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3eb8109430 0x7f3eb8109840 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8009900 tx=0x7f3ea802a410 comp rx=0 tx=0).stop 2024-09-21T05:57:06.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.372+0000 7f3ebfb39640 1 -- 172.21.15.117:0/432814559 shutdown_connections 2024-09-21T05:57:06.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.372+0000 7f3ebfb39640 1 --2- 172.21.15.117:0/432814559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3eb8109430 0x7f3eb8109840 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:06.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.372+0000 7f3ebfb39640 1 --2- 172.21.15.117:0/432814559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3eb8109d80 0x7f3eb81105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:06.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.372+0000 7f3ebfb39640 1 --2- 172.21.15.117:0/432814559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3eb8103b00 0x7f3eb8103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:06.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.372+0000 7f3ebfb39640 1 -- 172.21.15.117:0/432814559 >> 172.21.15.117:0/432814559 conn(0x7f3eb80ff500 msgr2=0x7f3eb8101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:06.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.372+0000 7f3ebfb39640 1 -- 172.21.15.117:0/432814559 shutdown_connections 2024-09-21T05:57:06.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.372+0000 7f3ebfb39640 1 -- 172.21.15.117:0/432814559 wait complete. 2024-09-21T05:57:06.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.372+0000 7f3ebfb39640 1 Processor -- start 2024-09-21T05:57:06.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.373+0000 7f3ebfb39640 1 -- start start 2024-09-21T05:57:06.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.373+0000 7f3ebfb39640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3eb8103b00 0x7f3eb819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:06.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.373+0000 7f3ebfb39640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3eb8109430 0x7f3eb819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:06.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.373+0000 7f3ebfb39640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3eb8109d80 0x7f3eb819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:06.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.373+0000 7f3ebfb39640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb8113a40 con 0x7f3eb8109430 2024-09-21T05:57:06.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.373+0000 7f3ebfb39640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb81138c0 con 0x7f3eb8109d80 2024-09-21T05:57:06.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.373+0000 7f3ebfb39640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb8113bc0 con 0x7f3eb8103b00 2024-09-21T05:57:06.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.373+0000 7f3ebd0d4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3eb8109430 0x7f3eb819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:06.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.373+0000 7f3ebd0d4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3eb8109430 0x7f3eb819aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60226/0 (socket says 172.21.15.117:60226) 2024-09-21T05:57:06.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.373+0000 7f3ebd0d4640 1 -- 172.21.15.117:0/3324929392 learned_addr learned my addr 172.21.15.117:0/3324929392 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:06.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.373+0000 7f3ebd8d5640 1 --2- 172.21.15.117:0/3324929392 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3eb8103b00 0x7f3eb819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:06.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.373+0000 7f3ebe0d6640 1 --2- 172.21.15.117:0/3324929392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3eb8109d80 0x7f3eb819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:06.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.373+0000 7f3ebd0d4640 1 -- 172.21.15.117:0/3324929392 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3eb8103b00 msgr2=0x7f3eb819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:06.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.373+0000 7f3ebd0d4640 1 --2- 172.21.15.117:0/3324929392 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3eb8103b00 0x7f3eb819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:06.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.373+0000 7f3ebd0d4640 1 -- 172.21.15.117:0/3324929392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3eb8109d80 msgr2=0x7f3eb819fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:06.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.373+0000 7f3ebd0d4640 1 --2- 172.21.15.117:0/3324929392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3eb8109d80 0x7f3eb819fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:06.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.373+0000 7f3ebd0d4640 1 -- 172.21.15.117:0/3324929392 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb81a03d0 con 0x7f3eb8109430 2024-09-21T05:57:06.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.373+0000 7f3ebd0d4640 1 --2- 172.21.15.117:0/3324929392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3eb8109430 0x7f3eb819aa40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3ea802a930 tx=0x7f3ea8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:06.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.374+0000 7f3eaeffd640 1 -- 172.21.15.117:0/3324929392 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea803f030 con 0x7f3eb8109430 2024-09-21T05:57:06.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.374+0000 7f3ebfb39640 1 -- 172.21.15.117:0/3324929392 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb81a0660 con 0x7f3eb8109430 2024-09-21T05:57:06.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.374+0000 7f3eaeffd640 1 -- 172.21.15.117:0/3324929392 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ea8003aa0 con 0x7f3eb8109430 2024-09-21T05:57:06.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.374+0000 7f3eaeffd640 1 -- 172.21.15.117:0/3324929392 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8035070 con 0x7f3eb8109430 2024-09-21T05:57:06.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.374+0000 7f3ebfb39640 1 -- 172.21.15.117:0/3324929392 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb81a6e50 con 0x7f3eb8109430 2024-09-21T05:57:06.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.375+0000 7f3ebfb39640 1 -- 172.21.15.117:0/3324929392 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eb8103f10 con 0x7f3eb8109430 2024-09-21T05:57:06.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.376+0000 7f3eaeffd640 1 -- 172.21.15.117:0/3324929392 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 22) ==== 154530+0+0 (secure 0 0 0) 0x7f3ea8042030 con 0x7f3eb8109430 2024-09-21T05:57:06.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.377+0000 7f3eaeffd640 1 --2- 172.21.15.117:0/3324929392 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3ea40b6560 0x7f3ea40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:06.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.377+0000 7f3ebd8d5640 1 --2- 172.21.15.117:0/3324929392 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3ea40b6560 0x7f3ea40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:06.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.377+0000 7f3eaeffd640 1 -- 172.21.15.117:0/3324929392 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f3ea8107f30 con 0x7f3eb8109430 2024-09-21T05:57:06.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.377+0000 7f3ebd8d5640 1 --2- 172.21.15.117:0/3324929392 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3ea40b6560 0x7f3ea40b8a70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3eb400bbe0 tx=0x7f3eb400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:06.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.380+0000 7f3eaeffd640 1 -- 172.21.15.117:0/3324929392 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3ea80ce720 con 0x7f3eb8109430 2024-09-21T05:57:06.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.598+0000 7f3ebfb39640 1 -- 172.21.15.117:0/3324929392 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f3eb810bb70 con 0x7f3eb8109430 2024-09-21T05:57:06.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.657+0000 7f3eaeffd640 1 -- 172.21.15.117:0/3324929392 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v27) ==== 117+0+0 (secure 0 0 0) 0x7f3ea80d2b00 con 0x7f3eb8109430 2024-09-21T05:57:06.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.659+0000 7f3ebfb39640 1 -- 172.21.15.117:0/3324929392 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3ea40b6560 msgr2=0x7f3ea40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:06.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.659+0000 7f3ebfb39640 1 --2- 172.21.15.117:0/3324929392 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3ea40b6560 0x7f3ea40b8a70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3eb400bbe0 tx=0x7f3eb400b390 comp rx=0 tx=0).stop 2024-09-21T05:57:06.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.659+0000 7f3ebfb39640 1 -- 172.21.15.117:0/3324929392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3eb8109430 msgr2=0x7f3eb819aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:06.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.659+0000 7f3ebfb39640 1 --2- 172.21.15.117:0/3324929392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3eb8109430 0x7f3eb819aa40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3ea802a930 tx=0x7f3ea8004270 comp rx=0 tx=0).stop 2024-09-21T05:57:06.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.659+0000 7f3ebfb39640 1 -- 172.21.15.117:0/3324929392 shutdown_connections 2024-09-21T05:57:06.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.659+0000 7f3ebfb39640 1 --2- 172.21.15.117:0/3324929392 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3ea40b6560 0x7f3ea40b8a70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:06.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.659+0000 7f3ebfb39640 1 --2- 172.21.15.117:0/3324929392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3eb8109430 0x7f3eb819aa40 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:06.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.659+0000 7f3ebfb39640 1 --2- 172.21.15.117:0/3324929392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3eb8109d80 0x7f3eb819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:06.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.659+0000 7f3ebfb39640 1 --2- 172.21.15.117:0/3324929392 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3eb8103b00 0x7f3eb819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:06.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.659+0000 7f3ebfb39640 1 -- 172.21.15.117:0/3324929392 >> 172.21.15.117:0/3324929392 conn(0x7f3eb80ff500 msgr2=0x7f3eb8106a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:06.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.659+0000 7f3ebfb39640 1 -- 172.21.15.117:0/3324929392 shutdown_connections 2024-09-21T05:57:06.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.659+0000 7f3ebfb39640 1 -- 172.21.15.117:0/3324929392 wait complete. 2024-09-21T05:57:06.672 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-21T05:57:06.685 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-21T05:57:06.707 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:57:06.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.816+0000 7fe327dcf640 1 -- 172.21.15.117:0/1846245541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3201045e0 msgr2=0x7fe3201049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:06.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.816+0000 7fe327dcf640 1 --2- 172.21.15.117:0/1846245541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3201045e0 0x7fe3201049f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe31c009900 tx=0x7fe31c02a480 comp rx=0 tx=0).stop 2024-09-21T05:57:06.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.816+0000 7fe327dcf640 1 -- 172.21.15.117:0/1846245541 shutdown_connections 2024-09-21T05:57:06.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.816+0000 7fe327dcf640 1 --2- 172.21.15.117:0/1846245541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe320102840 0x7fe3201124c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:06.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.816+0000 7fe327dcf640 1 --2- 172.21.15.117:0/1846245541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe320101c70 0x7fe3201020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:06.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.816+0000 7fe327dcf640 1 --2- 172.21.15.117:0/1846245541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3201045e0 0x7fe3201049f0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:06.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.816+0000 7fe327dcf640 1 -- 172.21.15.117:0/1846245541 >> 172.21.15.117:0/1846245541 conn(0x7fe320079ce0 msgr2=0x7fe32007a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:06.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.817+0000 7fe327dcf640 1 -- 172.21.15.117:0/1846245541 shutdown_connections 2024-09-21T05:57:06.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.817+0000 7fe327dcf640 1 -- 172.21.15.117:0/1846245541 wait complete. 2024-09-21T05:57:06.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.817+0000 7fe327dcf640 1 Processor -- start 2024-09-21T05:57:06.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.817+0000 7fe327dcf640 1 -- start start 2024-09-21T05:57:06.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.817+0000 7fe327dcf640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe320101c70 0x7fe32019c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:06.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.818+0000 7fe327dcf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe320102840 0x7fe32019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:06.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.818+0000 7fe327dcf640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3201045e0 0x7fe3201a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:06.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.818+0000 7fe327dcf640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe320115bd0 con 0x7fe320102840 2024-09-21T05:57:06.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.818+0000 7fe327dcf640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe320115a50 con 0x7fe3201045e0 2024-09-21T05:57:06.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.818+0000 7fe327dcf640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe320115d50 con 0x7fe320101c70 2024-09-21T05:57:06.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.818+0000 7fe32536a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe320102840 0x7fe32019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:06.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.818+0000 7fe32536a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe320102840 0x7fe32019cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60248/0 (socket says 172.21.15.117:60248) 2024-09-21T05:57:06.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.818+0000 7fe32536a640 1 -- 172.21.15.117:0/745687472 learned_addr learned my addr 172.21.15.117:0/745687472 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:06.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.818+0000 7fe32636c640 1 --2- 172.21.15.117:0/745687472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3201045e0 0x7fe3201a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:06.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.818+0000 7fe325b6b640 1 --2- 172.21.15.117:0/745687472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe320101c70 0x7fe32019c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:06.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.818+0000 7fe32536a640 1 -- 172.21.15.117:0/745687472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe320101c70 msgr2=0x7fe32019c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:06.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.818+0000 7fe32536a640 1 --2- 172.21.15.117:0/745687472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe320101c70 0x7fe32019c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:06.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.818+0000 7fe32536a640 1 -- 172.21.15.117:0/745687472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3201045e0 msgr2=0x7fe3201a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:06.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.818+0000 7fe32536a640 1 --2- 172.21.15.117:0/745687472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3201045e0 0x7fe3201a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:06.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.818+0000 7fe32536a640 1 -- 172.21.15.117:0/745687472 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3201a2580 con 0x7fe320102840 2024-09-21T05:57:06.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.818+0000 7fe32536a640 1 --2- 172.21.15.117:0/745687472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe320102840 0x7fe32019cbf0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe31000d740 tx=0x7fe31000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:06.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.818+0000 7fe316ffd640 1 -- 172.21.15.117:0/745687472 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe310017070 con 0x7fe320102840 2024-09-21T05:57:06.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.818+0000 7fe316ffd640 1 -- 172.21.15.117:0/745687472 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe31000de50 con 0x7fe320102840 2024-09-21T05:57:06.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.818+0000 7fe316ffd640 1 -- 172.21.15.117:0/745687472 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe310004ce0 con 0x7fe320102840 2024-09-21T05:57:06.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.818+0000 7fe327dcf640 1 -- 172.21.15.117:0/745687472 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3201a2870 con 0x7fe320102840 2024-09-21T05:57:06.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.818+0000 7fe327dcf640 1 -- 172.21.15.117:0/745687472 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3201a9000 con 0x7fe320102840 2024-09-21T05:57:06.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.819+0000 7fe327dcf640 1 -- 172.21.15.117:0/745687472 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3201020e0 con 0x7fe320102840 2024-09-21T05:57:06.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.821+0000 7fe316ffd640 1 -- 172.21.15.117:0/745687472 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 22) ==== 154530+0+0 (secure 0 0 0) 0x7fe310004e80 con 0x7fe320102840 2024-09-21T05:57:06.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.821+0000 7fe316ffd640 1 --2- 172.21.15.117:0/745687472 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe30c0b65e0 0x7fe30c0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:06.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.822+0000 7fe325b6b640 1 --2- 172.21.15.117:0/745687472 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe30c0b65e0 0x7fe30c0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:06.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.822+0000 7fe316ffd640 1 -- 172.21.15.117:0/745687472 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fe3100e6d70 con 0x7fe320102840 2024-09-21T05:57:06.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.822+0000 7fe325b6b640 1 --2- 172.21.15.117:0/745687472 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe30c0b65e0 0x7fe30c0b8af0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe31c031440 tx=0x7fe31c005d50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:06.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:06.826+0000 7fe316ffd640 1 -- 172.21.15.117:0/745687472 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe3100ad560 con 0x7fe320102840 2024-09-21T05:57:06.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:06 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3324929392' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-21T05:57:07.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.002+0000 7fe327dcf640 1 -- 172.21.15.117:0/745687472 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe3201a9330 con 0x7fe320102840 2024-09-21T05:57:07.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.004+0000 7fe316ffd640 1 -- 172.21.15.117:0/745687472 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+23066 (secure 0 0 0) 0x7fe3100b1940 con 0x7fe320102840 2024-09-21T05:57:07.006 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:57:07.006 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":76,"fsid":"ac8581a2-77dc-11ef-baf5-efdc52797490","created":"2024-09-21T05:46:47.021846+0000","modified":"2024-09-21T05:55:33.628948+0000","last_up_change":"2024-09-21T05:54:21.572849+0000","last_in_change":"2024-09-21T05:53:58.892597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:50:32.717389+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:55:10.604159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T05:55:12.581258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"68315502-fa6c-4a6e-a2d4-a7dde8071c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6803","nonce":3761478035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6805","nonce":3761478035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6809","nonce":3761478035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6807","nonce":3761478035}]},"public_addr":"172.21.15.117:6803/3761478035","cluster_addr":"172.21.15.117:6805/3761478035","heartbeat_back_addr":"172.21.15.117:6809/3761478035","heartbeat_front_addr":"172.21.15.117:6807/3761478035","state":["exists","up"]},{"osd":1,"uuid":"d8910a08-4ef3-49f2-a4db-e3588d85d12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6801","nonce":1737365588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6803","nonce":1737365588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6807","nonce":1737365588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6805","nonce":1737365588}]},"public_addr":"172.21.15.138:6801/1737365588","cluster_addr":"172.21.15.138:6803/1737365588","heartbeat_back_addr":"172.21.15.138:6807/1737365588","heartbeat_front_addr":"172.21.15.138:6805/1737365588","state":["exists","up"]},{"osd":2,"uuid":"94972378-a11f-4129-9e40-ad3fdec1d189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6801","nonce":3002534992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6803","nonce":3002534992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6807","nonce":3002534992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6805","nonce":3002534992}]},"public_addr":"172.21.15.192:6801/3002534992","cluster_addr":"172.21.15.192:6803/3002534992","heartbeat_back_addr":"172.21.15.192:6807/3002534992","heartbeat_front_addr":"172.21.15.192:6805/3002534992","state":["exists","up"]},{"osd":3,"uuid":"d4168c43-f0de-4445-945c-92d1cf270c33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6811","nonce":1222400293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6813","nonce":1222400293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6817","nonce":1222400293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6815","nonce":1222400293}]},"public_addr":"172.21.15.117:6811/1222400293","cluster_addr":"172.21.15.117:6813/1222400293","heartbeat_back_addr":"172.21.15.117:6817/1222400293","heartbeat_front_addr":"172.21.15.117:6815/1222400293","state":["exists","up"]},{"osd":4,"uuid":"28f1d7c0-8392-4a33-85c0-0de611cdd4b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6809","nonce":4294892870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6811","nonce":4294892870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6815","nonce":4294892870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6813","nonce":4294892870}]},"public_addr":"172.21.15.138:6809/4294892870","cluster_addr":"172.21.15.138:6811/4294892870","heartbeat_back_addr":"172.21.15.138:6815/4294892870","heartbeat_front_addr":"172.21.15.138:6813/4294892870","state":["exists","up"]},{"osd":5,"uuid":"ca902407-45e8-4bc5-bdbd-a729c1de8322","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6809","nonce":1776840019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6811","nonce":1776840019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6815","nonce":1776840019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6813","nonce":1776840019}]},"public_addr":"172.21.15.192:6809/1776840019","cluster_addr":"172.21.15.192:6811/1776840019","heartbeat_back_addr":"172.21.15.192:6815/1776840019","heartbeat_front_addr":"172.21.15.192:6813/1776840019","state":["exists","up"]},{"osd":6,"uuid":"6e507a91-ed09-4a33-bf0e-c4b593b3c5aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6819","nonce":1093341590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6821","nonce":1093341590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6825","nonce":1093341590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6823","nonce":1093341590}]},"public_addr":"172.21.15.117:6819/1093341590","cluster_addr":"172.21.15.117:6821/1093341590","heartbeat_back_addr":"172.21.15.117:6825/1093341590","heartbeat_front_addr":"172.21.15.117:6823/1093341590","state":["exists","up"]},{"osd":7,"uuid":"36a47dd7-67f7-4ebe-8759-3a8bb6bd2ea3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6817","nonce":1448179714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6819","nonce":1448179714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6823","nonce":1448179714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6821","nonce":1448179714}]},"public_addr":"172.21.15.138:6817/1448179714","cluster_addr":"172.21.15.138:6819/1448179714","heartbeat_back_addr":"172.21.15.138:6823/1448179714","heartbeat_front_addr":"172.21.15.138:6821/1448179714","state":["exists","up"]},{"osd":8,"uuid":"118e0b5d-2619-4512-94c5-8d07a0d3a28f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6817","nonce":3960902885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6819","nonce":3960902885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6823","nonce":3960902885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6821","nonce":3960902885}]},"public_addr":"172.21.15.192:6817/3960902885","cluster_addr":"172.21.15.192:6819/3960902885","heartbeat_back_addr":"172.21.15.192:6823/3960902885","heartbeat_front_addr":"172.21.15.192:6821/3960902885","state":["exists","up"]},{"osd":9,"uuid":"47543bc6-fa0b-4e72-9742-97dcda0b7f8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6827","nonce":3453855195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6829","nonce":3453855195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6833","nonce":3453855195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6831","nonce":3453855195}]},"public_addr":"172.21.15.117:6827/3453855195","cluster_addr":"172.21.15.117:6829/3453855195","heartbeat_back_addr":"172.21.15.117:6833/3453855195","heartbeat_front_addr":"172.21.15.117:6831/3453855195","state":["exists","up"]},{"osd":10,"uuid":"5e6708b4-7a85-4fa5-a03c-27d99f7d749f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6825","nonce":903392451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6827","nonce":903392451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6831","nonce":903392451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6829","nonce":903392451}]},"public_addr":"172.21.15.138:6825/903392451","cluster_addr":"172.21.15.138:6827/903392451","heartbeat_back_addr":"172.21.15.138:6831/903392451","heartbeat_front_addr":"172.21.15.138:6829/903392451","state":["exists","up"]},{"osd":11,"uuid":"24bc2ef7-e2b4-43c6-9e6c-43ae02390415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6825","nonce":899090479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6827","nonce":899090479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6831","nonce":899090479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6829","nonce":899090479}]},"public_addr":"172.21.15.192:6825/899090479","cluster_addr":"172.21.15.192:6827/899090479","heartbeat_back_addr":"172.21.15.192:6831/899090479","heartbeat_front_addr":"172.21.15.192:6829/899090479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:49:40.742217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:05.963655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:30.164495+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:56.598899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:51:21.940331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:51:47.777445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:52:12.869007+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:52:39.454054+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:03.569226+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:29.760863+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:55.788604+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:54:20.510184+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:0/2989394419":"2024-09-22T05:48:32.406654+0000","172.21.15.117:6801/1702676172":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/1766406059":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6800/2337036514":"2024-09-22T05:47:09.112435+0000","172.21.15.117:0/985798343":"2024-09-22T05:47:09.112435+0000","172.21.15.117:0/1083273222":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/473284565":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6801/2337036514":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6800/816635165":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/2642940197":"2024-09-22T05:48:32.406654+0000","172.21.15.117:6801/816635165":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/2225537433":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/3651584169":"2024-09-22T05:47:33.789146+0000","172.21.15.117:6800/1702676172":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/1284403882":"2024-09-22T05:47:33.789146+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:57:07.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.006+0000 7fe327dcf640 1 -- 172.21.15.117:0/745687472 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe30c0b65e0 msgr2=0x7fe30c0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:07.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.006+0000 7fe327dcf640 1 --2- 172.21.15.117:0/745687472 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe30c0b65e0 0x7fe30c0b8af0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe31c031440 tx=0x7fe31c005d50 comp rx=0 tx=0).stop 2024-09-21T05:57:07.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.006+0000 7fe327dcf640 1 -- 172.21.15.117:0/745687472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe320102840 msgr2=0x7fe32019cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:07.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.006+0000 7fe327dcf640 1 --2- 172.21.15.117:0/745687472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe320102840 0x7fe32019cbf0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe31000d740 tx=0x7fe31000dc00 comp rx=0 tx=0).stop 2024-09-21T05:57:07.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.006+0000 7fe327dcf640 1 -- 172.21.15.117:0/745687472 shutdown_connections 2024-09-21T05:57:07.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.006+0000 7fe327dcf640 1 --2- 172.21.15.117:0/745687472 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe30c0b65e0 0x7fe30c0b8af0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.006+0000 7fe327dcf640 1 --2- 172.21.15.117:0/745687472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe320102840 0x7fe32019cbf0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.006+0000 7fe327dcf640 1 --2- 172.21.15.117:0/745687472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3201045e0 0x7fe3201a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.006+0000 7fe327dcf640 1 --2- 172.21.15.117:0/745687472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe320101c70 0x7fe32019c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.006+0000 7fe327dcf640 1 -- 172.21.15.117:0/745687472 >> 172.21.15.117:0/745687472 conn(0x7fe320079ce0 msgr2=0x7fe320100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:07.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.006+0000 7fe327dcf640 1 -- 172.21.15.117:0/745687472 shutdown_connections 2024-09-21T05:57:07.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.006+0000 7fe327dcf640 1 -- 172.21.15.117:0/745687472 wait complete. 2024-09-21T05:57:07.018 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:06 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3324929392' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-21T05:57:07.020 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T05:50:32.717389+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T05:55:10.604159+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T05:55:12.581258+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T05:57:07.020 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T05:57:07.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:06 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3324929392' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-21T05:57:07.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.131+0000 7f866ec3f640 1 -- 172.21.15.117:0/3033591993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8668075c70 msgr2=0x7f8668076080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:07.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.131+0000 7f866ec3f640 1 --2- 172.21.15.117:0/3033591993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8668075c70 0x7f8668076080 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8658009900 tx=0x7f865802a410 comp rx=0 tx=0).stop 2024-09-21T05:57:07.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.132+0000 7f866ec3f640 1 -- 172.21.15.117:0/3033591993 shutdown_connections 2024-09-21T05:57:07.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.132+0000 7f866ec3f640 1 --2- 172.21.15.117:0/3033591993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8668075c70 0x7f8668076080 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.132+0000 7f866ec3f640 1 --2- 172.21.15.117:0/3033591993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8668114390 0x7f8668116880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.132+0000 7f866ec3f640 1 --2- 172.21.15.117:0/3033591993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8668077310 0x7f8668075730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.132+0000 7f866ec3f640 1 -- 172.21.15.117:0/3033591993 >> 172.21.15.117:0/3033591993 conn(0x7f866807a8c0 msgr2=0x7f866807acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:07.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.132+0000 7f866ec3f640 1 -- 172.21.15.117:0/3033591993 shutdown_connections 2024-09-21T05:57:07.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.132+0000 7f866ec3f640 1 -- 172.21.15.117:0/3033591993 wait complete. 2024-09-21T05:57:07.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.132+0000 7f866ec3f640 1 Processor -- start 2024-09-21T05:57:07.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.132+0000 7f866ec3f640 1 -- start start 2024-09-21T05:57:07.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.132+0000 7f866ec3f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8668075c70 0x7f86681a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:07.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.132+0000 7f866ec3f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8668077310 0x7f86681a0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:07.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.132+0000 7f866ec3f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8668114390 0x7f86681a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:07.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.132+0000 7f866ec3f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8668119fd0 con 0x7f8668077310 2024-09-21T05:57:07.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.133+0000 7f866ec3f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8668119e50 con 0x7f8668114390 2024-09-21T05:57:07.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.133+0000 7f866ec3f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f866811a150 con 0x7f8668075c70 2024-09-21T05:57:07.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.133+0000 7f865ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8668077310 0x7f86681a0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:07.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.133+0000 7f865ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8668077310 0x7f86681a0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60252/0 (socket says 172.21.15.117:60252) 2024-09-21T05:57:07.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.133+0000 7f865ffff640 1 -- 172.21.15.117:0/2173730241 learned_addr learned my addr 172.21.15.117:0/2173730241 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:07.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.133+0000 7f866c9db640 1 --2- 172.21.15.117:0/2173730241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8668075c70 0x7f86681a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:07.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.133+0000 7f866d1dc640 1 --2- 172.21.15.117:0/2173730241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8668114390 0x7f86681a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:07.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.133+0000 7f865ffff640 1 -- 172.21.15.117:0/2173730241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8668075c70 msgr2=0x7f86681a09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:07.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.133+0000 7f865ffff640 1 --2- 172.21.15.117:0/2173730241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8668075c70 0x7f86681a09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.133+0000 7f865ffff640 1 -- 172.21.15.117:0/2173730241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8668114390 msgr2=0x7f86681a6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:07.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.133+0000 7f865ffff640 1 --2- 172.21.15.117:0/2173730241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8668114390 0x7f86681a6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.133+0000 7f865ffff640 1 -- 172.21.15.117:0/2173730241 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86681a6900 con 0x7f8668077310 2024-09-21T05:57:07.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.133+0000 7f865ffff640 1 --2- 172.21.15.117:0/2173730241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8668077310 0x7f86681a0f70 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8658032fd0 tx=0x7f8658004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:07.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.133+0000 7f865dffb640 1 -- 172.21.15.117:0/2173730241 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865803f030 con 0x7f8668077310 2024-09-21T05:57:07.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.133+0000 7f865dffb640 1 -- 172.21.15.117:0/2173730241 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8658004490 con 0x7f8668077310 2024-09-21T05:57:07.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.133+0000 7f866ec3f640 1 -- 172.21.15.117:0/2173730241 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86681a6b90 con 0x7f8668077310 2024-09-21T05:57:07.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.133+0000 7f865dffb640 1 -- 172.21.15.117:0/2173730241 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8658035070 con 0x7f8668077310 2024-09-21T05:57:07.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.134+0000 7f866ec3f640 1 -- 172.21.15.117:0/2173730241 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86681ad380 con 0x7f8668077310 2024-09-21T05:57:07.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.135+0000 7f866ec3f640 1 -- 172.21.15.117:0/2173730241 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8668076c40 con 0x7f8668077310 2024-09-21T05:57:07.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.136+0000 7f865dffb640 1 -- 172.21.15.117:0/2173730241 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 22) ==== 154530+0+0 (secure 0 0 0) 0x7f8658042030 con 0x7f8668077310 2024-09-21T05:57:07.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.137+0000 7f865dffb640 1 --2- 172.21.15.117:0/2173730241 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f86500b6560 0x7f86500b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:07.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.137+0000 7f866c9db640 1 --2- 172.21.15.117:0/2173730241 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f86500b6560 0x7f86500b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:07.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.137+0000 7f865dffb640 1 -- 172.21.15.117:0/2173730241 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f8658107ce0 con 0x7f8668077310 2024-09-21T05:57:07.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.138+0000 7f866c9db640 1 --2- 172.21.15.117:0/2173730241 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f86500b6560 0x7f86500b8a70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8664005fd0 tx=0x7f866400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:07.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.141+0000 7f865dffb640 1 -- 172.21.15.117:0/2173730241 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f86580d28b0 con 0x7f8668077310 2024-09-21T05:57:07.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.320+0000 7f866ec3f640 1 -- 172.21.15.117:0/2173730241 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f86681ad550 con 0x7f8668077310 2024-09-21T05:57:07.322 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.321+0000 7f865dffb640 1 -- 172.21.15.117:0/2173730241 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) ==== 93+0+10 (secure 0 0 0) 0x7f86580d77f0 con 0x7f8668077310 2024-09-21T05:57:07.323 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 1 2024-09-21T05:57:07.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.323+0000 7f866ec3f640 1 -- 172.21.15.117:0/2173730241 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f86500b6560 msgr2=0x7f86500b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:07.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.323+0000 7f866ec3f640 1 --2- 172.21.15.117:0/2173730241 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f86500b6560 0x7f86500b8a70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8664005fd0 tx=0x7f866400b390 comp rx=0 tx=0).stop 2024-09-21T05:57:07.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.323+0000 7f866ec3f640 1 -- 172.21.15.117:0/2173730241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8668077310 msgr2=0x7f86681a0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:07.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.323+0000 7f866ec3f640 1 --2- 172.21.15.117:0/2173730241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8668077310 0x7f86681a0f70 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8658032fd0 tx=0x7f8658004270 comp rx=0 tx=0).stop 2024-09-21T05:57:07.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.323+0000 7f866ec3f640 1 -- 172.21.15.117:0/2173730241 shutdown_connections 2024-09-21T05:57:07.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.323+0000 7f866ec3f640 1 --2- 172.21.15.117:0/2173730241 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f86500b6560 0x7f86500b8a70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.323+0000 7f866ec3f640 1 --2- 172.21.15.117:0/2173730241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8668077310 0x7f86681a0f70 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.325 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.323+0000 7f866ec3f640 1 --2- 172.21.15.117:0/2173730241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8668114390 0x7f86681a6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.323+0000 7f866ec3f640 1 --2- 172.21.15.117:0/2173730241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8668075c70 0x7f86681a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.323+0000 7f866ec3f640 1 -- 172.21.15.117:0/2173730241 >> 172.21.15.117:0/2173730241 conn(0x7f866807a8c0 msgr2=0x7f8668112cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:07.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.323+0000 7f866ec3f640 1 -- 172.21.15.117:0/2173730241 shutdown_connections 2024-09-21T05:57:07.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.323+0000 7f866ec3f640 1 -- 172.21.15.117:0/2173730241 wait complete. 2024-09-21T05:57:07.336 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T05:57:07.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.446+0000 7fa198694640 1 -- 172.21.15.117:0/2657343178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1901050b0 msgr2=0x7fa1901054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:07.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.446+0000 7fa198694640 1 --2- 172.21.15.117:0/2657343178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1901050b0 0x7fa1901054c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa184009900 tx=0x7fa18402a410 comp rx=0 tx=0).stop 2024-09-21T05:57:07.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.446+0000 7fa198694640 1 -- 172.21.15.117:0/2657343178 shutdown_connections 2024-09-21T05:57:07.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.446+0000 7fa198694640 1 --2- 172.21.15.117:0/2657343178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1901050b0 0x7fa1901054c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.446+0000 7fa198694640 1 --2- 172.21.15.117:0/2657343178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa190105a00 0x7fa19010f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.446+0000 7fa198694640 1 --2- 172.21.15.117:0/2657343178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1901030f0 0x7fa190103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.446+0000 7fa198694640 1 -- 172.21.15.117:0/2657343178 >> 172.21.15.117:0/2657343178 conn(0x7fa190079e10 msgr2=0x7fa19007a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:07.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.446+0000 7fa198694640 1 -- 172.21.15.117:0/2657343178 shutdown_connections 2024-09-21T05:57:07.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.446+0000 7fa198694640 1 -- 172.21.15.117:0/2657343178 wait complete. 2024-09-21T05:57:07.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.447+0000 7fa198694640 1 Processor -- start 2024-09-21T05:57:07.449 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.447+0000 7fa198694640 1 -- start start 2024-09-21T05:57:07.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.447+0000 7fa198694640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1901030f0 0x7fa19019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:07.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.447+0000 7fa198694640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1901050b0 0x7fa19019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:07.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.447+0000 7fa198694640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa190105a00 0x7fa19019fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:07.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.447+0000 7fa198694640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa190112900 con 0x7fa190105a00 2024-09-21T05:57:07.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.447+0000 7fa198694640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa190112780 con 0x7fa1901030f0 2024-09-21T05:57:07.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.447+0000 7fa198694640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa190112a80 con 0x7fa1901050b0 2024-09-21T05:57:07.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.447+0000 7fa196c31640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa190105a00 0x7fa19019fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:07.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.447+0000 7fa196c31640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa190105a00 0x7fa19019fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60272/0 (socket says 172.21.15.117:60272) 2024-09-21T05:57:07.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.448+0000 7fa196c31640 1 -- 172.21.15.117:0/1946106118 learned_addr learned my addr 172.21.15.117:0/1946106118 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:07.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.448+0000 7fa195c2f640 1 --2- 172.21.15.117:0/1946106118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1901050b0 0x7fa19019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:07.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.448+0000 7fa196430640 1 --2- 172.21.15.117:0/1946106118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1901030f0 0x7fa19019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:07.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.448+0000 7fa196c31640 1 -- 172.21.15.117:0/1946106118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1901050b0 msgr2=0x7fa19019a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:07.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.448+0000 7fa196c31640 1 --2- 172.21.15.117:0/1946106118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1901050b0 0x7fa19019a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.448+0000 7fa196c31640 1 -- 172.21.15.117:0/1946106118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1901030f0 msgr2=0x7fa19019a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:07.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.448+0000 7fa196c31640 1 --2- 172.21.15.117:0/1946106118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1901030f0 0x7fa19019a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.448+0000 7fa196c31640 1 -- 172.21.15.117:0/1946106118 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1901a0340 con 0x7fa190105a00 2024-09-21T05:57:07.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.448+0000 7fa196c31640 1 --2- 172.21.15.117:0/1946106118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa190105a00 0x7fa19019fc40 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa18800eaa0 tx=0x7fa18800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:07.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.448+0000 7fa1837fe640 1 -- 172.21.15.117:0/1946106118 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18800cdb0 con 0x7fa190105a00 2024-09-21T05:57:07.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.448+0000 7fa1837fe640 1 -- 172.21.15.117:0/1946106118 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa188005020 con 0x7fa190105a00 2024-09-21T05:57:07.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.448+0000 7fa198694640 1 -- 172.21.15.117:0/1946106118 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1901a0630 con 0x7fa190105a00 2024-09-21T05:57:07.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.448+0000 7fa1837fe640 1 -- 172.21.15.117:0/1946106118 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1880116d0 con 0x7fa190105a00 2024-09-21T05:57:07.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.448+0000 7fa198694640 1 -- 172.21.15.117:0/1946106118 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1901a6dc0 con 0x7fa190105a00 2024-09-21T05:57:07.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.449+0000 7fa198694640 1 -- 172.21.15.117:0/1946106118 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa190103500 con 0x7fa190105a00 2024-09-21T05:57:07.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.451+0000 7fa1837fe640 1 -- 172.21.15.117:0/1946106118 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 22) ==== 154530+0+0 (secure 0 0 0) 0x7fa1880051d0 con 0x7fa190105a00 2024-09-21T05:57:07.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.452+0000 7fa1837fe640 1 --2- 172.21.15.117:0/1946106118 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa17c0b6560 0x7fa17c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:07.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.452+0000 7fa196430640 1 --2- 172.21.15.117:0/1946106118 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa17c0b6560 0x7fa17c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:07.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.452+0000 7fa1837fe640 1 -- 172.21.15.117:0/1946106118 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fa1880e79a0 con 0x7fa190105a00 2024-09-21T05:57:07.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.452+0000 7fa196430640 1 --2- 172.21.15.117:0/1946106118 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa17c0b6560 0x7fa17c0b8a70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa18c009b60 tx=0x7fa18c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:07.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.455+0000 7fa1837fe640 1 -- 172.21.15.117:0/1946106118 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa188016030 con 0x7fa190105a00 2024-09-21T05:57:07.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.633+0000 7fa198694640 1 -- 172.21.15.117:0/1946106118 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa1901a7130 con 0x7fa190105a00 2024-09-21T05:57:07.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.633+0000 7fa1837fe640 1 -- 172.21.15.117:0/1946106118 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v76) ==== 104+0+11 (secure 0 0 0) 0x7fa1880ae190 con 0x7fa190105a00 2024-09-21T05:57:07.635 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 32 2024-09-21T05:57:07.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.636+0000 7fa198694640 1 -- 172.21.15.117:0/1946106118 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa17c0b6560 msgr2=0x7fa17c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:07.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.636+0000 7fa198694640 1 --2- 172.21.15.117:0/1946106118 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa17c0b6560 0x7fa17c0b8a70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa18c009b60 tx=0x7fa18c005f90 comp rx=0 tx=0).stop 2024-09-21T05:57:07.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.636+0000 7fa198694640 1 -- 172.21.15.117:0/1946106118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa190105a00 msgr2=0x7fa19019fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:07.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.636+0000 7fa198694640 1 --2- 172.21.15.117:0/1946106118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa190105a00 0x7fa19019fc40 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa18800eaa0 tx=0x7fa18800ef60 comp rx=0 tx=0).stop 2024-09-21T05:57:07.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.636+0000 7fa198694640 1 -- 172.21.15.117:0/1946106118 shutdown_connections 2024-09-21T05:57:07.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.636+0000 7fa198694640 1 --2- 172.21.15.117:0/1946106118 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa17c0b6560 0x7fa17c0b8a70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.636+0000 7fa198694640 1 --2- 172.21.15.117:0/1946106118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa190105a00 0x7fa19019fc40 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.636+0000 7fa198694640 1 --2- 172.21.15.117:0/1946106118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1901050b0 0x7fa19019a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.636+0000 7fa198694640 1 --2- 172.21.15.117:0/1946106118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1901030f0 0x7fa19019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.636+0000 7fa198694640 1 -- 172.21.15.117:0/1946106118 >> 172.21.15.117:0/1946106118 conn(0x7fa190079e10 msgr2=0x7fa190100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:07.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.636+0000 7fa198694640 1 -- 172.21.15.117:0/1946106118 shutdown_connections 2024-09-21T05:57:07.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.636+0000 7fa198694640 1 -- 172.21.15.117:0/1946106118 wait complete. 2024-09-21T05:57:07.649 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T05:57:07.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.760+0000 7f25edf5f640 1 -- 172.21.15.117:0/1297789565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25e81048e0 msgr2=0x7f25e8104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:07.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.760+0000 7f25edf5f640 1 --2- 172.21.15.117:0/1297789565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25e81048e0 0x7f25e8104cf0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f25d8009900 tx=0x7f25d802a410 comp rx=0 tx=0).stop 2024-09-21T05:57:07.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.760+0000 7f25edf5f640 1 -- 172.21.15.117:0/1297789565 shutdown_connections 2024-09-21T05:57:07.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.760+0000 7f25edf5f640 1 --2- 172.21.15.117:0/1297789565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25e810dd40 0x7f25e8110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.760+0000 7f25edf5f640 1 --2- 172.21.15.117:0/1297789565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25e8105230 0x7f25e810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.760+0000 7f25edf5f640 1 --2- 172.21.15.117:0/1297789565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25e81048e0 0x7f25e8104cf0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.760+0000 7f25edf5f640 1 -- 172.21.15.117:0/1297789565 >> 172.21.15.117:0/1297789565 conn(0x7f25e80fe3b0 msgr2=0x7f25e8100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:07.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.760+0000 7f25edf5f640 1 -- 172.21.15.117:0/1297789565 shutdown_connections 2024-09-21T05:57:07.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.760+0000 7f25edf5f640 1 -- 172.21.15.117:0/1297789565 wait complete. 2024-09-21T05:57:07.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.761+0000 7f25edf5f640 1 Processor -- start 2024-09-21T05:57:07.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.761+0000 7f25edf5f640 1 -- start start 2024-09-21T05:57:07.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.761+0000 7f25edf5f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25e81048e0 0x7f25e819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:07.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.761+0000 7f25edf5f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25e8105230 0x7f25e819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:07.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.761+0000 7f25edf5f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25e810dd40 0x7f25e819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:07.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.761+0000 7f25edf5f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f25e81139c0 con 0x7f25e810dd40 2024-09-21T05:57:07.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.761+0000 7f25edf5f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f25e8113840 con 0x7f25e81048e0 2024-09-21T05:57:07.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.761+0000 7f25edf5f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f25e8113b40 con 0x7f25e8105230 2024-09-21T05:57:07.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.761+0000 7f25e7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25e810dd40 0x7f25e819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:07.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.761+0000 7f25e7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25e810dd40 0x7f25e819fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60286/0 (socket says 172.21.15.117:60286) 2024-09-21T05:57:07.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.761+0000 7f25e7fff640 1 -- 172.21.15.117:0/2344018081 learned_addr learned my addr 172.21.15.117:0/2344018081 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:07.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.761+0000 7f25e77fe640 1 --2- 172.21.15.117:0/2344018081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25e81048e0 0x7f25e819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:07.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.761+0000 7f25e6ffd640 1 --2- 172.21.15.117:0/2344018081 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25e8105230 0x7f25e819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:07.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.762+0000 7f25e7fff640 1 -- 172.21.15.117:0/2344018081 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25e8105230 msgr2=0x7f25e819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:07.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.762+0000 7f25e7fff640 1 --2- 172.21.15.117:0/2344018081 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25e8105230 0x7f25e819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.762+0000 7f25e7fff640 1 -- 172.21.15.117:0/2344018081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25e81048e0 msgr2=0x7f25e819a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:07.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.762+0000 7f25e7fff640 1 --2- 172.21.15.117:0/2344018081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25e81048e0 0x7f25e819a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.762+0000 7f25e7fff640 1 -- 172.21.15.117:0/2344018081 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e81a02e0 con 0x7f25e810dd40 2024-09-21T05:57:07.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.762+0000 7f25e6ffd640 1 --2- 172.21.15.117:0/2344018081 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25e8105230 0x7f25e819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:57:07.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.762+0000 7f25e7fff640 1 --2- 172.21.15.117:0/2344018081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25e810dd40 0x7f25e819fc70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f25e000cad0 tx=0x7f25e000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:07.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.762+0000 7f25e4ff9640 1 -- 172.21.15.117:0/2344018081 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e000ae20 con 0x7f25e810dd40 2024-09-21T05:57:07.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.762+0000 7f25edf5f640 1 -- 172.21.15.117:0/2344018081 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25e81a05d0 con 0x7f25e810dd40 2024-09-21T05:57:07.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.762+0000 7f25e4ff9640 1 -- 172.21.15.117:0/2344018081 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25e0005020 con 0x7f25e810dd40 2024-09-21T05:57:07.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.762+0000 7f25e4ff9640 1 -- 172.21.15.117:0/2344018081 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e000f6b0 con 0x7f25e810dd40 2024-09-21T05:57:07.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.762+0000 7f25edf5f640 1 -- 172.21.15.117:0/2344018081 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25e81a6df0 con 0x7f25e810dd40 2024-09-21T05:57:07.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.763+0000 7f25edf5f640 1 -- 172.21.15.117:0/2344018081 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25e8104cf0 con 0x7f25e810dd40 2024-09-21T05:57:07.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.765+0000 7f25e4ff9640 1 -- 172.21.15.117:0/2344018081 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 22) ==== 154530+0+0 (secure 0 0 0) 0x7f25e00051d0 con 0x7f25e810dd40 2024-09-21T05:57:07.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.766+0000 7f25e4ff9640 1 --2- 172.21.15.117:0/2344018081 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f25d00b6560 0x7f25d00b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:07.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.766+0000 7f25e77fe640 1 --2- 172.21.15.117:0/2344018081 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f25d00b6560 0x7f25d00b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:07.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.766+0000 7f25e4ff9640 1 -- 172.21.15.117:0/2344018081 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f25e00e7820 con 0x7f25e810dd40 2024-09-21T05:57:07.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.766+0000 7f25e77fe640 1 --2- 172.21.15.117:0/2344018081 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f25d00b6560 0x7f25d00b8a70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f25d8031440 tx=0x7f25d8005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:07.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.769+0000 7f25e4ff9640 1 -- 172.21.15.117:0/2344018081 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f25e0014030 con 0x7f25e810dd40 2024-09-21T05:57:07.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.945+0000 7f25edf5f640 1 -- 172.21.15.117:0/2344018081 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f25e819b560 con 0x7f25e810dd40 2024-09-21T05:57:07.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.946+0000 7f25e4ff9640 1 -- 172.21.15.117:0/2344018081 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v76) ==== 100+0+11 (secure 0 0 0) 0x7f25e00ae010 con 0x7f25e810dd40 2024-09-21T05:57:07.948 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 64 2024-09-21T05:57:07.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.948+0000 7f25edf5f640 1 -- 172.21.15.117:0/2344018081 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f25d00b6560 msgr2=0x7f25d00b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:07.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.948+0000 7f25edf5f640 1 --2- 172.21.15.117:0/2344018081 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f25d00b6560 0x7f25d00b8a70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f25d8031440 tx=0x7f25d8005e80 comp rx=0 tx=0).stop 2024-09-21T05:57:07.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.948+0000 7f25edf5f640 1 -- 172.21.15.117:0/2344018081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25e810dd40 msgr2=0x7f25e819fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:07.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.948+0000 7f25edf5f640 1 --2- 172.21.15.117:0/2344018081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25e810dd40 0x7f25e819fc70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f25e000cad0 tx=0x7f25e000cf90 comp rx=0 tx=0).stop 2024-09-21T05:57:07.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.948+0000 7f25edf5f640 1 -- 172.21.15.117:0/2344018081 shutdown_connections 2024-09-21T05:57:07.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.948+0000 7f25edf5f640 1 --2- 172.21.15.117:0/2344018081 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f25d00b6560 0x7f25d00b8a70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.948+0000 7f25edf5f640 1 --2- 172.21.15.117:0/2344018081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25e810dd40 0x7f25e819fc70 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.948+0000 7f25edf5f640 1 --2- 172.21.15.117:0/2344018081 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25e8105230 0x7f25e819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.948+0000 7f25edf5f640 1 --2- 172.21.15.117:0/2344018081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25e81048e0 0x7f25e819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:07.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.948+0000 7f25edf5f640 1 -- 172.21.15.117:0/2344018081 >> 172.21.15.117:0/2344018081 conn(0x7f25e80fe3b0 msgr2=0x7f25e8107850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:07.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.948+0000 7f25edf5f640 1 -- 172.21.15.117:0/2344018081 shutdown_connections 2024-09-21T05:57:07.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:07.948+0000 7f25edf5f640 1 -- 172.21.15.117:0/2344018081 wait complete. 2024-09-21T05:57:07.960 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:07 smithi117 ceph-mon[30947]: pgmap v321: 97 pgs: 97 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-21T05:57:07.960 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:07 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3324929392' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-21T05:57:07.961 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:07 smithi117 ceph-mon[30947]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T05:57:07.961 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:07 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/745687472' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T05:57:07.961 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:07 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2173730241' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T05:57:07.961 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:07 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/1946106118' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T05:57:07.961 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-21T05:57:07.962 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T05:57:07.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:07 smithi192 ceph-mon[34700]: pgmap v321: 97 pgs: 97 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-21T05:57:07.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:07 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3324929392' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-21T05:57:07.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:07 smithi192 ceph-mon[34700]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T05:57:07.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:07 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/745687472' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T05:57:07.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:07 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2173730241' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T05:57:07.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:07 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/1946106118' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T05:57:08.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:07 smithi138 ceph-mon[34391]: pgmap v321: 97 pgs: 97 active+clean; 647 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-21T05:57:08.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:07 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3324929392' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-21T05:57:08.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:07 smithi138 ceph-mon[34391]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T05:57:08.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:07 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/745687472' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T05:57:08.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:07 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2173730241' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T05:57:08.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:07 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/1946106118' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T05:57:08.208 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:57:08.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.663+0000 7f17a39ae640 1 -- 172.21.15.117:0/2779049538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f179c1083a0 msgr2=0x7f179c1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:08.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.663+0000 7f17a39ae640 1 --2- 172.21.15.117:0/2779049538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f179c1083a0 0x7f179c1087b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f178c009900 tx=0x7f178c02a410 comp rx=0 tx=0).stop 2024-09-21T05:57:08.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.663+0000 7f17a39ae640 1 -- 172.21.15.117:0/2779049538 shutdown_connections 2024-09-21T05:57:08.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.663+0000 7f17a39ae640 1 --2- 172.21.15.117:0/2779049538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f179c1083a0 0x7f179c1087b0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:08.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.663+0000 7f17a39ae640 1 --2- 172.21.15.117:0/2779049538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f179c108cf0 0x7f179c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:08.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.663+0000 7f17a39ae640 1 --2- 172.21.15.117:0/2779049538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f179c102a60 0x7f179c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:08.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.663+0000 7f17a39ae640 1 -- 172.21.15.117:0/2779049538 >> 172.21.15.117:0/2779049538 conn(0x7f179c079730 msgr2=0x7f179c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:08.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.664+0000 7f17a39ae640 1 -- 172.21.15.117:0/2779049538 shutdown_connections 2024-09-21T05:57:08.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.664+0000 7f17a39ae640 1 -- 172.21.15.117:0/2779049538 wait complete. 2024-09-21T05:57:08.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.664+0000 7f17a39ae640 1 Processor -- start 2024-09-21T05:57:08.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.664+0000 7f17a39ae640 1 -- start start 2024-09-21T05:57:08.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.665+0000 7f17a39ae640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f179c102a60 0x7f179c19c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:08.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.665+0000 7f17a39ae640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f179c1083a0 0x7f179c19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:08.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.665+0000 7f17a39ae640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f179c108cf0 0x7f179c1a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:08.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.665+0000 7f17a39ae640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f179c1129b0 con 0x7f179c108cf0 2024-09-21T05:57:08.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.665+0000 7f17a39ae640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f179c112830 con 0x7f179c1083a0 2024-09-21T05:57:08.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.665+0000 7f17a39ae640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f179c112b30 con 0x7f179c102a60 2024-09-21T05:57:08.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.665+0000 7f17a1f4b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f179c108cf0 0x7f179c1a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:08.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.665+0000 7f17a1f4b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f179c108cf0 0x7f179c1a1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60318/0 (socket says 172.21.15.117:60318) 2024-09-21T05:57:08.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.665+0000 7f17a0f49640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f179c1083a0 0x7f179c19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:08.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.665+0000 7f17a1f4b640 1 -- 172.21.15.117:0/1050515444 learned_addr learned my addr 172.21.15.117:0/1050515444 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:08.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.665+0000 7f17a174a640 1 --2- 172.21.15.117:0/1050515444 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f179c102a60 0x7f179c19c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:08.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.665+0000 7f17a1f4b640 1 -- 172.21.15.117:0/1050515444 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f179c102a60 msgr2=0x7f179c19c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:08.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.665+0000 7f17a1f4b640 1 --2- 172.21.15.117:0/1050515444 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f179c102a60 0x7f179c19c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:08.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.665+0000 7f17a1f4b640 1 -- 172.21.15.117:0/1050515444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f179c1083a0 msgr2=0x7f179c19cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:08.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.665+0000 7f17a1f4b640 1 --2- 172.21.15.117:0/1050515444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f179c1083a0 0x7f179c19cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:08.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.665+0000 7f17a1f4b640 1 -- 172.21.15.117:0/1050515444 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f179c1a2660 con 0x7f179c108cf0 2024-09-21T05:57:08.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.665+0000 7f17a1f4b640 1 --2- 172.21.15.117:0/1050515444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f179c108cf0 0x7f179c1a1f30 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f179400eb10 tx=0x7f179400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:08.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.667+0000 7f17927fc640 1 -- 172.21.15.117:0/1050515444 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1794004b20 con 0x7f179c108cf0 2024-09-21T05:57:08.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.667+0000 7f17927fc640 1 -- 172.21.15.117:0/1050515444 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1794004cc0 con 0x7f179c108cf0 2024-09-21T05:57:08.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.667+0000 7f17a39ae640 1 -- 172.21.15.117:0/1050515444 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f179c1a2950 con 0x7f179c108cf0 2024-09-21T05:57:08.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.667+0000 7f17927fc640 1 -- 172.21.15.117:0/1050515444 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1794010670 con 0x7f179c108cf0 2024-09-21T05:57:08.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.667+0000 7f17a39ae640 1 -- 172.21.15.117:0/1050515444 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f179c1a90b0 con 0x7f179c108cf0 2024-09-21T05:57:08.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.668+0000 7f17a39ae640 1 -- 172.21.15.117:0/1050515444 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f179c102e70 con 0x7f179c108cf0 2024-09-21T05:57:08.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.670+0000 7f17927fc640 1 -- 172.21.15.117:0/1050515444 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 22) ==== 154530+0+0 (secure 0 0 0) 0x7f1794005000 con 0x7f179c108cf0 2024-09-21T05:57:08.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.671+0000 7f17927fc640 1 --2- 172.21.15.117:0/1050515444 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f17880b65e0 0x7f17880b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:08.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.671+0000 7f17a174a640 1 --2- 172.21.15.117:0/1050515444 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f17880b65e0 0x7f17880b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:08.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.671+0000 7f17927fc640 1 -- 172.21.15.117:0/1050515444 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f17940e7ac0 con 0x7f179c108cf0 2024-09-21T05:57:08.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.671+0000 7f17a174a640 1 --2- 172.21.15.117:0/1050515444 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f17880b65e0 0x7f17880b8af0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1798009bd0 tx=0x7f1798005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:08.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.674+0000 7f17927fc640 1 -- 172.21.15.117:0/1050515444 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1794056710 con 0x7f179c108cf0 2024-09-21T05:57:08.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.886+0000 7f17a39ae640 1 -- 172.21.15.117:0/1050515444 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f179c19d820 con 0x7f179c108cf0 2024-09-21T05:57:08.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.887+0000 7f17927fc640 1 -- 172.21.15.117:0/1050515444 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+12001 (secure 0 0 0) 0x7f179c19d820 con 0x7f179c108cf0 2024-09-21T05:57:08.889 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:57:08.889 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":27,"btime":"2024-09-21T05:57:06:651813+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3227057504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3227057504},{"type":"v1","addr":"172.21.15.117:6835","nonce":3227057504}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3303829183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3303829183},{"type":"v1","addr":"172.21.15.117:6837","nonce":3303829183}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T05:55:14.555897+0000","modified":"2024-09-21T05:57:06.599449+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24638,"mds_1":24637,"mds_2":24628,"mds_3":24616,"mds_4":24602},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6839/2314572411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2314572411},{"type":"v1","addr":"172.21.15.117:6839","nonce":2314572411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6841/3378096561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":3378096561},{"type":"v1","addr":"172.21.15.117:6841","nonce":3378096561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/3725629862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3725629862},{"type":"v1","addr":"172.21.15.138:6833","nonce":3725629862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/4276801396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4276801396},{"type":"v1","addr":"172.21.15.138:6835","nonce":4276801396}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6833/2224910879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2224910879},{"type":"v1","addr":"172.21.15.192:6833","nonce":2224910879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.138:6837/2540335374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2540335374},{"type":"v1","addr":"172.21.15.138:6837","nonce":2540335374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24616":{"gid":24616,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.192:6835/3530688860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3530688860},{"type":"v1","addr":"172.21.15.192:6835","nonce":3530688860}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.192:6837/2683860842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2683860842},{"type":"v1","addr":"172.21.15.192:6837","nonce":2683860842}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.192:6839/1727623231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1727623231},{"type":"v1","addr":"172.21.15.192:6839","nonce":1727623231}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/3230372576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":3230372576},{"type":"v1","addr":"172.21.15.138:6839","nonce":3230372576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24602,24628,24616,24637,24638]},"id":1}]} 2024-09-21T05:57:08.889 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 27 2024-09-21T05:57:08.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.889+0000 7f17a39ae640 1 -- 172.21.15.117:0/1050515444 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f17880b65e0 msgr2=0x7f17880b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:08.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.889+0000 7f17a39ae640 1 --2- 172.21.15.117:0/1050515444 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f17880b65e0 0x7f17880b8af0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1798009bd0 tx=0x7f1798005f90 comp rx=0 tx=0).stop 2024-09-21T05:57:08.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.889+0000 7f17a39ae640 1 -- 172.21.15.117:0/1050515444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f179c108cf0 msgr2=0x7f179c1a1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:08.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.889+0000 7f17a39ae640 1 --2- 172.21.15.117:0/1050515444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f179c108cf0 0x7f179c1a1f30 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f179400eb10 tx=0x7f179400c2d0 comp rx=0 tx=0).stop 2024-09-21T05:57:08.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.889+0000 7f17a39ae640 1 -- 172.21.15.117:0/1050515444 shutdown_connections 2024-09-21T05:57:08.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.889+0000 7f17a39ae640 1 --2- 172.21.15.117:0/1050515444 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f17880b65e0 0x7f17880b8af0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:08.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.889+0000 7f17a39ae640 1 --2- 172.21.15.117:0/1050515444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f179c108cf0 0x7f179c1a1f30 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:08.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.889+0000 7f17a39ae640 1 --2- 172.21.15.117:0/1050515444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f179c1083a0 0x7f179c19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:08.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.889+0000 7f17a39ae640 1 --2- 172.21.15.117:0/1050515444 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f179c102a60 0x7f179c19c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:08.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.889+0000 7f17a39ae640 1 -- 172.21.15.117:0/1050515444 >> 172.21.15.117:0/1050515444 conn(0x7f179c079730 msgr2=0x7f179c06af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:08.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.889+0000 7f17a39ae640 1 -- 172.21.15.117:0/1050515444 shutdown_connections 2024-09-21T05:57:08.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:08.889+0000 7f17a39ae640 1 -- 172.21.15.117:0/1050515444 wait complete. 2024-09-21T05:57:08.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:08 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2344018081' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T05:57:09.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:08 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2344018081' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T05:57:09.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:08 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2344018081' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T05:57:09.742 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-21T05:57:09.742 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-21T05:57:09.742 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:57:09.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.853+0000 7f2620539640 1 -- 172.21.15.117:0/961847938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2618103ad0 msgr2=0x7f2618103ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:09.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.853+0000 7f2620539640 1 --2- 172.21.15.117:0/961847938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2618103ad0 0x7f2618103ee0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2608009900 tx=0x7f260802a410 comp rx=0 tx=0).stop 2024-09-21T05:57:09.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.854+0000 7f2620539640 1 -- 172.21.15.117:0/961847938 shutdown_connections 2024-09-21T05:57:09.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.854+0000 7f2620539640 1 --2- 172.21.15.117:0/961847938 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2618109d50 0x7f2618110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:09.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.854+0000 7f2620539640 1 --2- 172.21.15.117:0/961847938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2618109400 0x7f2618109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:09.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.854+0000 7f2620539640 1 --2- 172.21.15.117:0/961847938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2618103ad0 0x7f2618103ee0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:09.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.854+0000 7f2620539640 1 -- 172.21.15.117:0/961847938 >> 172.21.15.117:0/961847938 conn(0x7f26180ff510 msgr2=0x7f2618101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:09.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.854+0000 7f2620539640 1 -- 172.21.15.117:0/961847938 shutdown_connections 2024-09-21T05:57:09.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.854+0000 7f2620539640 1 -- 172.21.15.117:0/961847938 wait complete. 2024-09-21T05:57:09.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.854+0000 7f2620539640 1 Processor -- start 2024-09-21T05:57:09.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.854+0000 7f2620539640 1 -- start start 2024-09-21T05:57:09.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.855+0000 7f2620539640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2618103ad0 0x7f261819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:09.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.855+0000 7f2620539640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2618109400 0x7f261819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:09.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.855+0000 7f2620539640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2618109d50 0x7f261819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:09.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.855+0000 7f2620539640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2618113a10 con 0x7f2618103ad0 2024-09-21T05:57:09.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.855+0000 7f2620539640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2618113890 con 0x7f2618109d50 2024-09-21T05:57:09.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.855+0000 7f2620539640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2618113b90 con 0x7f2618109400 2024-09-21T05:57:09.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.855+0000 7f261e2d5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2618103ad0 0x7f261819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:09.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.855+0000 7f261e2d5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2618103ad0 0x7f261819a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60348/0 (socket says 172.21.15.117:60348) 2024-09-21T05:57:09.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.855+0000 7f261e2d5640 1 -- 172.21.15.117:0/3401415890 learned_addr learned my addr 172.21.15.117:0/3401415890 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:09.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.855+0000 7f261dad4640 1 --2- 172.21.15.117:0/3401415890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2618109400 0x7f261819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:09.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.855+0000 7f261ead6640 1 --2- 172.21.15.117:0/3401415890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2618109d50 0x7f261819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:09.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.855+0000 7f261e2d5640 1 -- 172.21.15.117:0/3401415890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2618109400 msgr2=0x7f261819a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:09.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.855+0000 7f261e2d5640 1 --2- 172.21.15.117:0/3401415890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2618109400 0x7f261819a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:09.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.855+0000 7f261e2d5640 1 -- 172.21.15.117:0/3401415890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2618109d50 msgr2=0x7f261819fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:09.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.855+0000 7f261e2d5640 1 --2- 172.21.15.117:0/3401415890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2618109d50 0x7f261819fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:09.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.855+0000 7f261e2d5640 1 -- 172.21.15.117:0/3401415890 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26181a02d0 con 0x7f2618103ad0 2024-09-21T05:57:09.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.855+0000 7f261ead6640 1 --2- 172.21.15.117:0/3401415890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2618109d50 0x7f261819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:57:09.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.855+0000 7f261dad4640 1 --2- 172.21.15.117:0/3401415890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2618109400 0x7f261819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:57:09.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.855+0000 7f261e2d5640 1 --2- 172.21.15.117:0/3401415890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2618103ad0 0x7f261819a430 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2608005940 tx=0x7f26080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:09.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.855+0000 7f260f7fe640 1 -- 172.21.15.117:0/3401415890 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260803f030 con 0x7f2618103ad0 2024-09-21T05:57:09.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.855+0000 7f2620539640 1 -- 172.21.15.117:0/3401415890 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26181a0560 con 0x7f2618103ad0 2024-09-21T05:57:09.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.855+0000 7f260f7fe640 1 -- 172.21.15.117:0/3401415890 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2608003cb0 con 0x7f2618103ad0 2024-09-21T05:57:09.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.855+0000 7f260f7fe640 1 -- 172.21.15.117:0/3401415890 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2608035070 con 0x7f2618103ad0 2024-09-21T05:57:09.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.855+0000 7f2620539640 1 -- 172.21.15.117:0/3401415890 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26181a0aa0 con 0x7f2618103ad0 2024-09-21T05:57:09.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.856+0000 7f2620539640 1 -- 172.21.15.117:0/3401415890 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2618104d40 con 0x7f2618103ad0 2024-09-21T05:57:09.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.858+0000 7f260f7fe640 1 -- 172.21.15.117:0/3401415890 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 22) ==== 154530+0+0 (secure 0 0 0) 0x7f2608042030 con 0x7f2618103ad0 2024-09-21T05:57:09.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.859+0000 7f260f7fe640 1 --2- 172.21.15.117:0/3401415890 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f26040b6560 0x7f26040b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:09.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.859+0000 7f261dad4640 1 --2- 172.21.15.117:0/3401415890 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f26040b6560 0x7f26040b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:09.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.859+0000 7f260f7fe640 1 -- 172.21.15.117:0/3401415890 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f26081080b0 con 0x7f2618103ad0 2024-09-21T05:57:09.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.859+0000 7f261dad4640 1 --2- 172.21.15.117:0/3401415890 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f26040b6560 0x7f26040b8a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f26140058c0 tx=0x7f2614005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:09.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:09.862+0000 7f260f7fe640 1 -- 172.21.15.117:0/3401415890 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f26080ce8a0 con 0x7f2618103ad0 2024-09-21T05:57:09.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:09 smithi192 ceph-mon[34700]: pgmap v322: 97 pgs: 97 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 12 op/s 2024-09-21T05:57:09.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:09 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/1050515444' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:57:10.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:09 smithi117 ceph-mon[30947]: pgmap v322: 97 pgs: 97 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 12 op/s 2024-09-21T05:57:10.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:09 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/1050515444' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:57:10.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.037+0000 7f2620539640 1 -- 172.21.15.117:0/3401415890 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2618063800 con 0x7f26040b6560 2024-09-21T05:57:10.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.042+0000 7f260f7fe640 1 -- 172.21.15.117:0/3401415890 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259322 (secure 0 0 0) 0x7f2618063800 con 0x7f26040b6560 2024-09-21T05:57:10.046 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:57:10.048 INFO:teuthology.orchestra.run.smithi117.stderr:dumped all 2024-09-21T05:57:10.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.045+0000 7f2620539640 1 -- 172.21.15.117:0/3401415890 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f26040b6560 msgr2=0x7f26040b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:10.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.045+0000 7f2620539640 1 --2- 172.21.15.117:0/3401415890 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f26040b6560 0x7f26040b8a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f26140058c0 tx=0x7f2614005850 comp rx=0 tx=0).stop 2024-09-21T05:57:10.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.045+0000 7f2620539640 1 -- 172.21.15.117:0/3401415890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2618103ad0 msgr2=0x7f261819a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:10.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.045+0000 7f2620539640 1 --2- 172.21.15.117:0/3401415890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2618103ad0 0x7f261819a430 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2608005940 tx=0x7f26080042f0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.045+0000 7f2620539640 1 -- 172.21.15.117:0/3401415890 shutdown_connections 2024-09-21T05:57:10.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.045+0000 7f2620539640 1 --2- 172.21.15.117:0/3401415890 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f26040b6560 0x7f26040b8a70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.045+0000 7f2620539640 1 --2- 172.21.15.117:0/3401415890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2618109d50 0x7f261819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.045+0000 7f2620539640 1 --2- 172.21.15.117:0/3401415890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2618109400 0x7f261819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.045+0000 7f2620539640 1 --2- 172.21.15.117:0/3401415890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2618103ad0 0x7f261819a430 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.045+0000 7f2620539640 1 -- 172.21.15.117:0/3401415890 >> 172.21.15.117:0/3401415890 conn(0x7f26180ff510 msgr2=0x7f2618101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:10.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.045+0000 7f2620539640 1 -- 172.21.15.117:0/3401415890 shutdown_connections 2024-09-21T05:57:10.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.045+0000 7f2620539640 1 -- 172.21.15.117:0/3401415890 wait complete. 2024-09-21T05:57:10.059 INFO:teuthology.orchestra.run.smithi117.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-09-21T05:57:08.534958+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663079,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1128,"num_read_kb":672,"num_write":385,"num_write_kb":2306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":2952512,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":817,"ondisk_log_size":817,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376208,"kb_used_data":10320,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565616,"statfs":{"total":1151940427776,"available":1151555190784,"internally_reserved":0,"allocated":10567680,"data_stored":6913193,"data_compressed":182193,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,0,0,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":980,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":75,"num_write":15,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002924"},"pg_stats":[{"pgid":"3.3f","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.767187+0000","last_change":"2024-09-21T05:55:14.007642+0000","last_active":"2024-09-21T05:57:02.767187+0000","last_peered":"2024-09-21T05:57:02.767187+0000","last_clean":"2024-09-21T05:57:02.767187+0000","last_became_active":"2024-09-21T05:55:14.006943+0000","last_became_peered":"2024-09-21T05:55:14.006943+0000","last_unstale":"2024-09-21T05:57:02.767187+0000","last_undegraded":"2024-09-21T05:57:02.767187+0000","last_fullsized":"2024-09-21T05:57:02.767187+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:23:53.816079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"76'4","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.748660+0000","last_change":"2024-09-21T05:55:14.002034+0000","last_active":"2024-09-21T05:57:02.748660+0000","last_peered":"2024-09-21T05:57:02.748660+0000","last_clean":"2024-09-21T05:57:02.748660+0000","last_became_active":"2024-09-21T05:55:14.001658+0000","last_became_peered":"2024-09-21T05:55:14.001658+0000","last_unstale":"2024-09-21T05:57:02.748660+0000","last_undegraded":"2024-09-21T05:57:02.748660+0000","last_fullsized":"2024-09-21T05:57:02.748660+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:01.568362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.746220+0000","last_change":"2024-09-21T05:55:13.996901+0000","last_active":"2024-09-21T05:57:02.746220+0000","last_peered":"2024-09-21T05:57:02.746220+0000","last_clean":"2024-09-21T05:57:02.746220+0000","last_became_active":"2024-09-21T05:55:13.996020+0000","last_became_peered":"2024-09-21T05:55:13.996020+0000","last_unstale":"2024-09-21T05:57:02.746220+0000","last_undegraded":"2024-09-21T05:57:02.746220+0000","last_fullsized":"2024-09-21T05:57:02.746220+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:38.771926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.750182+0000","last_change":"2024-09-21T05:55:13.995968+0000","last_active":"2024-09-21T05:57:02.750182+0000","last_peered":"2024-09-21T05:57:02.750182+0000","last_clean":"2024-09-21T05:57:02.750182+0000","last_became_active":"2024-09-21T05:55:13.995259+0000","last_became_peered":"2024-09-21T05:55:13.995259+0000","last_unstale":"2024-09-21T05:57:02.750182+0000","last_undegraded":"2024-09-21T05:57:02.750182+0000","last_fullsized":"2024-09-21T05:57:02.750182+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:48.953342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"76'4","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.726754+0000","last_change":"2024-09-21T05:55:13.995778+0000","last_active":"2024-09-21T05:57:02.726754+0000","last_peered":"2024-09-21T05:57:02.726754+0000","last_clean":"2024-09-21T05:57:02.726754+0000","last_became_active":"2024-09-21T05:55:13.995415+0000","last_became_peered":"2024-09-21T05:55:13.995415+0000","last_unstale":"2024-09-21T05:57:02.726754+0000","last_undegraded":"2024-09-21T05:57:02.726754+0000","last_fullsized":"2024-09-21T05:57:02.726754+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:52.562120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.737104+0000","last_change":"2024-09-21T05:55:14.000005+0000","last_active":"2024-09-21T05:57:02.737104+0000","last_peered":"2024-09-21T05:57:02.737104+0000","last_clean":"2024-09-21T05:57:02.737104+0000","last_became_active":"2024-09-21T05:55:13.999493+0000","last_became_peered":"2024-09-21T05:55:13.999493+0000","last_unstale":"2024-09-21T05:57:02.737104+0000","last_undegraded":"2024-09-21T05:57:02.737104+0000","last_fullsized":"2024-09-21T05:57:02.737104+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:29.426368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.756030+0000","last_change":"2024-09-21T05:55:13.993004+0000","last_active":"2024-09-21T05:57:02.756030+0000","last_peered":"2024-09-21T05:57:02.756030+0000","last_clean":"2024-09-21T05:57:02.756030+0000","last_became_active":"2024-09-21T05:55:13.992615+0000","last_became_peered":"2024-09-21T05:55:13.992615+0000","last_unstale":"2024-09-21T05:57:02.756030+0000","last_undegraded":"2024-09-21T05:57:02.756030+0000","last_fullsized":"2024-09-21T05:57:02.756030+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:48.324618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"76'9","reported_seq":21,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.745963+0000","last_change":"2024-09-21T05:55:33.769006+0000","last_active":"2024-09-21T05:57:02.745963+0000","last_peered":"2024-09-21T05:57:02.745963+0000","last_clean":"2024-09-21T05:57:02.745963+0000","last_became_active":"2024-09-21T05:55:33.768705+0000","last_became_peered":"2024-09-21T05:55:33.768705+0000","last_unstale":"2024-09-21T05:57:02.745963+0000","last_undegraded":"2024-09-21T05:57:02.745963+0000","last_fullsized":"2024-09-21T05:57:02.745963+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:14.013300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.765471+0000","last_change":"2024-09-21T05:55:14.004131+0000","last_active":"2024-09-21T05:57:02.765471+0000","last_peered":"2024-09-21T05:57:02.765471+0000","last_clean":"2024-09-21T05:57:02.765471+0000","last_became_active":"2024-09-21T05:55:14.003621+0000","last_became_peered":"2024-09-21T05:55:14.003621+0000","last_unstale":"2024-09-21T05:57:02.765471+0000","last_undegraded":"2024-09-21T05:57:02.765471+0000","last_fullsized":"2024-09-21T05:57:02.765471+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:00.694211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"76'16","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.750873+0000","last_change":"2024-09-21T05:55:33.643289+0000","last_active":"2024-09-21T05:57:02.750873+0000","last_peered":"2024-09-21T05:57:02.750873+0000","last_clean":"2024-09-21T05:57:02.750873+0000","last_became_active":"2024-09-21T05:55:33.642805+0000","last_became_peered":"2024-09-21T05:55:33.642805+0000","last_unstale":"2024-09-21T05:57:02.750873+0000","last_undegraded":"2024-09-21T05:57:02.750873+0000","last_fullsized":"2024-09-21T05:57:02.750873+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:00.854712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.766867+0000","last_change":"2024-09-21T05:55:14.006467+0000","last_active":"2024-09-21T05:57:02.766867+0000","last_peered":"2024-09-21T05:57:02.766867+0000","last_clean":"2024-09-21T05:57:02.766867+0000","last_became_active":"2024-09-21T05:55:14.005978+0000","last_became_peered":"2024-09-21T05:55:14.005978+0000","last_unstale":"2024-09-21T05:57:02.766867+0000","last_undegraded":"2024-09-21T05:57:02.766867+0000","last_fullsized":"2024-09-21T05:57:02.766867+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:37.391152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.747861+0000","last_change":"2024-09-21T05:55:14.003972+0000","last_active":"2024-09-21T05:57:02.747861+0000","last_peered":"2024-09-21T05:57:02.747861+0000","last_clean":"2024-09-21T05:57:02.747861+0000","last_became_active":"2024-09-21T05:55:14.003601+0000","last_became_peered":"2024-09-21T05:55:14.003601+0000","last_unstale":"2024-09-21T05:57:02.747861+0000","last_undegraded":"2024-09-21T05:57:02.747861+0000","last_fullsized":"2024-09-21T05:57:02.747861+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:16.201087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.751561+0000","last_change":"2024-09-21T05:55:13.996051+0000","last_active":"2024-09-21T05:57:02.751561+0000","last_peered":"2024-09-21T05:57:02.751561+0000","last_clean":"2024-09-21T05:57:02.751561+0000","last_became_active":"2024-09-21T05:55:13.995679+0000","last_became_peered":"2024-09-21T05:55:13.995679+0000","last_unstale":"2024-09-21T05:57:02.751561+0000","last_undegraded":"2024-09-21T05:57:02.751561+0000","last_fullsized":"2024-09-21T05:57:02.751561+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:45:44.372029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.745041+0000","last_change":"2024-09-21T05:55:13.995421+0000","last_active":"2024-09-21T05:57:02.745041+0000","last_peered":"2024-09-21T05:57:02.745041+0000","last_clean":"2024-09-21T05:57:02.745041+0000","last_became_active":"2024-09-21T05:55:13.995006+0000","last_became_peered":"2024-09-21T05:55:13.995006+0000","last_unstale":"2024-09-21T05:57:02.745041+0000","last_undegraded":"2024-09-21T05:57:02.745041+0000","last_fullsized":"2024-09-21T05:57:02.745041+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:00.636555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.745532+0000","last_change":"2024-09-21T05:55:13.993466+0000","last_active":"2024-09-21T05:57:02.745532+0000","last_peered":"2024-09-21T05:57:02.745532+0000","last_clean":"2024-09-21T05:57:02.745532+0000","last_became_active":"2024-09-21T05:55:13.993137+0000","last_became_peered":"2024-09-21T05:55:13.993137+0000","last_unstale":"2024-09-21T05:57:02.745532+0000","last_undegraded":"2024-09-21T05:57:02.745532+0000","last_fullsized":"2024-09-21T05:57:02.745532+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:33.218868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.765648+0000","last_change":"2024-09-21T05:55:13.996316+0000","last_active":"2024-09-21T05:57:02.765648+0000","last_peered":"2024-09-21T05:57:02.765648+0000","last_clean":"2024-09-21T05:57:02.765648+0000","last_became_active":"2024-09-21T05:55:13.995918+0000","last_became_peered":"2024-09-21T05:55:13.995918+0000","last_unstale":"2024-09-21T05:57:02.765648+0000","last_undegraded":"2024-09-21T05:57:02.765648+0000","last_fullsized":"2024-09-21T05:57:02.765648+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:45:54.408504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.767326+0000","last_change":"2024-09-21T05:55:14.007198+0000","last_active":"2024-09-21T05:57:02.767326+0000","last_peered":"2024-09-21T05:57:02.767326+0000","last_clean":"2024-09-21T05:57:02.767326+0000","last_became_active":"2024-09-21T05:55:14.006700+0000","last_became_peered":"2024-09-21T05:55:14.006700+0000","last_unstale":"2024-09-21T05:57:02.767326+0000","last_undegraded":"2024-09-21T05:57:02.767326+0000","last_fullsized":"2024-09-21T05:57:02.767326+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:45.343778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.765593+0000","last_change":"2024-09-21T05:55:14.000152+0000","last_active":"2024-09-21T05:57:02.765593+0000","last_peered":"2024-09-21T05:57:02.765593+0000","last_clean":"2024-09-21T05:57:02.765593+0000","last_became_active":"2024-09-21T05:55:13.999786+0000","last_became_peered":"2024-09-21T05:55:13.999786+0000","last_unstale":"2024-09-21T05:57:02.765593+0000","last_undegraded":"2024-09-21T05:57:02.765593+0000","last_fullsized":"2024-09-21T05:57:02.765593+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:43:25.713410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.749050+0000","last_change":"2024-09-21T05:55:13.988367+0000","last_active":"2024-09-21T05:57:02.749050+0000","last_peered":"2024-09-21T05:57:02.749050+0000","last_clean":"2024-09-21T05:57:02.749050+0000","last_became_active":"2024-09-21T05:55:13.987982+0000","last_became_peered":"2024-09-21T05:55:13.987982+0000","last_unstale":"2024-09-21T05:57:02.749050+0000","last_undegraded":"2024-09-21T05:57:02.749050+0000","last_fullsized":"2024-09-21T05:57:02.749050+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:15.395155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.748991+0000","last_change":"2024-09-21T05:55:14.004962+0000","last_active":"2024-09-21T05:57:02.748991+0000","last_peered":"2024-09-21T05:57:02.748991+0000","last_clean":"2024-09-21T05:57:02.748991+0000","last_became_active":"2024-09-21T05:55:14.004573+0000","last_became_peered":"2024-09-21T05:55:14.004573+0000","last_unstale":"2024-09-21T05:57:02.748991+0000","last_undegraded":"2024-09-21T05:57:02.748991+0000","last_fullsized":"2024-09-21T05:57:02.748991+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:59.014017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.756283+0000","last_change":"2024-09-21T05:55:13.991267+0000","last_active":"2024-09-21T05:57:02.756283+0000","last_peered":"2024-09-21T05:57:02.756283+0000","last_clean":"2024-09-21T05:57:02.756283+0000","last_became_active":"2024-09-21T05:55:13.990916+0000","last_became_peered":"2024-09-21T05:55:13.990916+0000","last_unstale":"2024-09-21T05:57:02.756283+0000","last_undegraded":"2024-09-21T05:57:02.756283+0000","last_fullsized":"2024-09-21T05:57:02.756283+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:06:03.904632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.766270+0000","last_change":"2024-09-21T05:55:13.994147+0000","last_active":"2024-09-21T05:57:02.766270+0000","last_peered":"2024-09-21T05:57:02.766270+0000","last_clean":"2024-09-21T05:57:02.766270+0000","last_became_active":"2024-09-21T05:55:13.993794+0000","last_became_peered":"2024-09-21T05:55:13.993794+0000","last_unstale":"2024-09-21T05:57:02.766270+0000","last_undegraded":"2024-09-21T05:57:02.766270+0000","last_fullsized":"2024-09-21T05:57:02.766270+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:19.970853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.750316+0000","last_change":"2024-09-21T05:55:14.001119+0000","last_active":"2024-09-21T05:57:02.750316+0000","last_peered":"2024-09-21T05:57:02.750316+0000","last_clean":"2024-09-21T05:57:02.750316+0000","last_became_active":"2024-09-21T05:55:14.000799+0000","last_became_peered":"2024-09-21T05:55:14.000799+0000","last_unstale":"2024-09-21T05:57:02.750316+0000","last_undegraded":"2024-09-21T05:57:02.750316+0000","last_fullsized":"2024-09-21T05:57:02.750316+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:28.259188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.739740+0000","last_change":"2024-09-21T05:55:13.992043+0000","last_active":"2024-09-21T05:57:02.739740+0000","last_peered":"2024-09-21T05:57:02.739740+0000","last_clean":"2024-09-21T05:57:02.739740+0000","last_became_active":"2024-09-21T05:55:13.991344+0000","last_became_peered":"2024-09-21T05:55:13.991344+0000","last_unstale":"2024-09-21T05:57:02.739740+0000","last_undegraded":"2024-09-21T05:57:02.739740+0000","last_fullsized":"2024-09-21T05:57:02.739740+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:42:55.674333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"76'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.749506+0000","last_change":"2024-09-21T05:55:13.993114+0000","last_active":"2024-09-21T05:57:02.749506+0000","last_peered":"2024-09-21T05:57:02.749506+0000","last_clean":"2024-09-21T05:57:02.749506+0000","last_became_active":"2024-09-21T05:55:13.992749+0000","last_became_peered":"2024-09-21T05:55:13.992749+0000","last_unstale":"2024-09-21T05:57:02.749506+0000","last_undegraded":"2024-09-21T05:57:02.749506+0000","last_fullsized":"2024-09-21T05:57:02.749506+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:15.926967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"76'15","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.763355+0000","last_change":"2024-09-21T05:55:13.997199+0000","last_active":"2024-09-21T05:57:02.763355+0000","last_peered":"2024-09-21T05:57:02.763355+0000","last_clean":"2024-09-21T05:57:02.763355+0000","last_became_active":"2024-09-21T05:55:13.996809+0000","last_became_peered":"2024-09-21T05:55:13.996809+0000","last_unstale":"2024-09-21T05:57:02.763355+0000","last_undegraded":"2024-09-21T05:57:02.763355+0000","last_fullsized":"2024-09-21T05:57:02.763355+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:16.681430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.751124+0000","last_change":"2024-09-21T05:55:14.000879+0000","last_active":"2024-09-21T05:57:02.751124+0000","last_peered":"2024-09-21T05:57:02.751124+0000","last_clean":"2024-09-21T05:57:02.751124+0000","last_became_active":"2024-09-21T05:55:14.000259+0000","last_became_peered":"2024-09-21T05:55:14.000259+0000","last_unstale":"2024-09-21T05:57:02.751124+0000","last_undegraded":"2024-09-21T05:57:02.751124+0000","last_fullsized":"2024-09-21T05:57:02.751124+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:01:08.938417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"76'4","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.744616+0000","last_change":"2024-09-21T05:55:13.989433+0000","last_active":"2024-09-21T05:57:02.744616+0000","last_peered":"2024-09-21T05:57:02.744616+0000","last_clean":"2024-09-21T05:57:02.744616+0000","last_became_active":"2024-09-21T05:55:13.989097+0000","last_became_peered":"2024-09-21T05:55:13.989097+0000","last_unstale":"2024-09-21T05:57:02.744616+0000","last_undegraded":"2024-09-21T05:57:02.744616+0000","last_fullsized":"2024-09-21T05:57:02.744616+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:37.835298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.742763+0000","last_change":"2024-09-21T05:55:13.993911+0000","last_active":"2024-09-21T05:57:02.742763+0000","last_peered":"2024-09-21T05:57:02.742763+0000","last_clean":"2024-09-21T05:57:02.742763+0000","last_became_active":"2024-09-21T05:55:13.993570+0000","last_became_peered":"2024-09-21T05:55:13.993570+0000","last_unstale":"2024-09-21T05:57:02.742763+0000","last_undegraded":"2024-09-21T05:57:02.742763+0000","last_fullsized":"2024-09-21T05:57:02.742763+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:08:16.588457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.751759+0000","last_change":"2024-09-21T05:55:13.992764+0000","last_active":"2024-09-21T05:57:02.751759+0000","last_peered":"2024-09-21T05:57:02.751759+0000","last_clean":"2024-09-21T05:57:02.751759+0000","last_became_active":"2024-09-21T05:55:13.992338+0000","last_became_peered":"2024-09-21T05:55:13.992338+0000","last_unstale":"2024-09-21T05:57:02.751759+0000","last_undegraded":"2024-09-21T05:57:02.751759+0000","last_fullsized":"2024-09-21T05:57:02.751759+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:25:29.997353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"76'12","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.752225+0000","last_change":"2024-09-21T05:55:13.987119+0000","last_active":"2024-09-21T05:57:02.752225+0000","last_peered":"2024-09-21T05:57:02.752225+0000","last_clean":"2024-09-21T05:57:02.752225+0000","last_became_active":"2024-09-21T05:55:13.986663+0000","last_became_peered":"2024-09-21T05:55:13.986663+0000","last_unstale":"2024-09-21T05:57:02.752225+0000","last_undegraded":"2024-09-21T05:57:02.752225+0000","last_fullsized":"2024-09-21T05:57:02.752225+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:02.089833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.751424+0000","last_change":"2024-09-21T05:55:14.003577+0000","last_active":"2024-09-21T05:57:02.751424+0000","last_peered":"2024-09-21T05:57:02.751424+0000","last_clean":"2024-09-21T05:57:02.751424+0000","last_became_active":"2024-09-21T05:55:14.003162+0000","last_became_peered":"2024-09-21T05:55:14.003162+0000","last_unstale":"2024-09-21T05:57:02.751424+0000","last_undegraded":"2024-09-21T05:57:02.751424+0000","last_fullsized":"2024-09-21T05:57:02.751424+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:58.630843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.730878+0000","last_change":"2024-09-21T05:55:13.992078+0000","last_active":"2024-09-21T05:57:02.730878+0000","last_peered":"2024-09-21T05:57:02.730878+0000","last_clean":"2024-09-21T05:57:02.730878+0000","last_became_active":"2024-09-21T05:55:13.991680+0000","last_became_peered":"2024-09-21T05:55:13.991680+0000","last_unstale":"2024-09-21T05:57:02.730878+0000","last_undegraded":"2024-09-21T05:57:02.730878+0000","last_fullsized":"2024-09-21T05:57:02.730878+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:52.886755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"76'7","reported_seq":133,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:03.373713+0000","last_change":"2024-09-21T05:55:11.982560+0000","last_active":"2024-09-21T05:57:03.373713+0000","last_peered":"2024-09-21T05:57:03.373713+0000","last_clean":"2024-09-21T05:57:03.373713+0000","last_became_active":"2024-09-21T05:55:11.982179+0000","last_became_peered":"2024-09-21T05:55:11.982179+0000","last_unstale":"2024-09-21T05:57:03.373713+0000","last_undegraded":"2024-09-21T05:57:03.373713+0000","last_fullsized":"2024-09-21T05:57:03.373713+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:06.012194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.740537+0000","last_change":"2024-09-21T05:55:13.993994+0000","last_active":"2024-09-21T05:57:02.740537+0000","last_peered":"2024-09-21T05:57:02.740537+0000","last_clean":"2024-09-21T05:57:02.740537+0000","last_became_active":"2024-09-21T05:55:13.993585+0000","last_became_peered":"2024-09-21T05:55:13.993585+0000","last_unstale":"2024-09-21T05:57:02.740537+0000","last_undegraded":"2024-09-21T05:57:02.740537+0000","last_fullsized":"2024-09-21T05:57:02.740537+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:11.608616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"74'7","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.634233+0000","last_change":"2024-09-21T05:55:11.984397+0000","last_active":"2024-09-21T05:55:33.634233+0000","last_peered":"2024-09-21T05:55:33.634233+0000","last_clean":"2024-09-21T05:55:33.634233+0000","last_became_active":"2024-09-21T05:55:11.983888+0000","last_became_peered":"2024-09-21T05:55:11.983888+0000","last_unstale":"2024-09-21T05:55:33.634233+0000","last_undegraded":"2024-09-21T05:55:33.634233+0000","last_fullsized":"2024-09-21T05:55:33.634233+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:59.012710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636187+0000","last_change":"2024-09-21T05:55:11.974547+0000","last_active":"2024-09-21T05:55:33.636187+0000","last_peered":"2024-09-21T05:55:33.636187+0000","last_clean":"2024-09-21T05:55:33.636187+0000","last_became_active":"2024-09-21T05:55:11.974151+0000","last_became_peered":"2024-09-21T05:55:11.974151+0000","last_unstale":"2024-09-21T05:55:33.636187+0000","last_undegraded":"2024-09-21T05:55:33.636187+0000","last_fullsized":"2024-09-21T05:55:33.636187+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:51.489548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.746384+0000","last_change":"2024-09-21T05:55:13.999287+0000","last_active":"2024-09-21T05:57:02.746384+0000","last_peered":"2024-09-21T05:57:02.746384+0000","last_clean":"2024-09-21T05:57:02.746384+0000","last_became_active":"2024-09-21T05:55:13.998742+0000","last_became_peered":"2024-09-21T05:55:13.998742+0000","last_unstale":"2024-09-21T05:57:02.746384+0000","last_undegraded":"2024-09-21T05:57:02.746384+0000","last_fullsized":"2024-09-21T05:57:02.746384+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:33.842810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"74'3","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.768113+0000","last_change":"2024-09-21T05:55:11.981354+0000","last_active":"2024-09-21T05:55:33.768113+0000","last_peered":"2024-09-21T05:55:33.768113+0000","last_clean":"2024-09-21T05:55:33.768113+0000","last_became_active":"2024-09-21T05:55:11.980993+0000","last_became_peered":"2024-09-21T05:55:11.980993+0000","last_unstale":"2024-09-21T05:55:33.768113+0000","last_undegraded":"2024-09-21T05:55:33.768113+0000","last_fullsized":"2024-09-21T05:55:33.768113+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:05.037669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.746795+0000","last_change":"2024-09-21T05:55:14.001041+0000","last_active":"2024-09-21T05:57:02.746795+0000","last_peered":"2024-09-21T05:57:02.746795+0000","last_clean":"2024-09-21T05:57:02.746795+0000","last_became_active":"2024-09-21T05:55:14.000688+0000","last_became_peered":"2024-09-21T05:55:14.000688+0000","last_unstale":"2024-09-21T05:57:02.746795+0000","last_undegraded":"2024-09-21T05:57:02.746795+0000","last_fullsized":"2024-09-21T05:57:02.746795+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:51.190495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"76'6","reported_seq":233,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:07.379709+0000","last_change":"2024-09-21T05:55:11.984097+0000","last_active":"2024-09-21T05:57:07.379709+0000","last_peered":"2024-09-21T05:57:07.379709+0000","last_clean":"2024-09-21T05:57:07.379709+0000","last_became_active":"2024-09-21T05:55:11.982969+0000","last_became_peered":"2024-09-21T05:55:11.982969+0000","last_unstale":"2024-09-21T05:57:07.379709+0000","last_undegraded":"2024-09-21T05:57:07.379709+0000","last_fullsized":"2024-09-21T05:57:07.379709+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:41.372225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":204,"num_read_kb":204,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.768826+0000","last_change":"2024-09-21T05:55:13.998433+0000","last_active":"2024-09-21T05:57:02.768826+0000","last_peered":"2024-09-21T05:57:02.768826+0000","last_clean":"2024-09-21T05:57:02.768826+0000","last_became_active":"2024-09-21T05:55:13.997890+0000","last_became_peered":"2024-09-21T05:55:13.997890+0000","last_unstale":"2024-09-21T05:57:02.768826+0000","last_undegraded":"2024-09-21T05:57:02.768826+0000","last_fullsized":"2024-09-21T05:57:02.768826+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:24.396789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"74'4","reported_seq":40,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.701709+0000","last_change":"2024-09-21T05:55:33.701599+0000","last_active":"2024-09-21T05:55:33.701709+0000","last_peered":"2024-09-21T05:55:33.701709+0000","last_clean":"2024-09-21T05:55:33.701709+0000","last_became_active":"2024-09-21T05:55:33.697385+0000","last_became_peered":"2024-09-21T05:55:33.697385+0000","last_unstale":"2024-09-21T05:55:33.701709+0000","last_undegraded":"2024-09-21T05:55:33.701709+0000","last_fullsized":"2024-09-21T05:55:33.701709+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:18.653991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.741741+0000","last_change":"2024-09-21T05:55:13.995111+0000","last_active":"2024-09-21T05:57:02.741741+0000","last_peered":"2024-09-21T05:57:02.741741+0000","last_clean":"2024-09-21T05:57:02.741741+0000","last_became_active":"2024-09-21T05:55:13.994691+0000","last_became_peered":"2024-09-21T05:55:13.994691+0000","last_unstale":"2024-09-21T05:57:02.741741+0000","last_undegraded":"2024-09-21T05:57:02.741741+0000","last_fullsized":"2024-09-21T05:57:02.741741+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:30.167649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"74'5","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767919+0000","last_change":"2024-09-21T05:55:12.013961+0000","last_active":"2024-09-21T05:55:33.767919+0000","last_peered":"2024-09-21T05:55:33.767919+0000","last_clean":"2024-09-21T05:55:33.767919+0000","last_became_active":"2024-09-21T05:55:12.013573+0000","last_became_peered":"2024-09-21T05:55:12.013573+0000","last_unstale":"2024-09-21T05:55:33.767919+0000","last_undegraded":"2024-09-21T05:55:33.767919+0000","last_fullsized":"2024-09-21T05:55:33.767919+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:12.119605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.768347+0000","last_change":"2024-09-21T05:55:13.996093+0000","last_active":"2024-09-21T05:57:02.768347+0000","last_peered":"2024-09-21T05:57:02.768347+0000","last_clean":"2024-09-21T05:57:02.768347+0000","last_became_active":"2024-09-21T05:55:13.995342+0000","last_became_peered":"2024-09-21T05:55:13.995342+0000","last_unstale":"2024-09-21T05:57:02.768347+0000","last_undegraded":"2024-09-21T05:57:02.768347+0000","last_fullsized":"2024-09-21T05:57:02.768347+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:20.335330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"76'34","reported_seq":272,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:07.380341+0000","last_change":"2024-09-21T05:55:11.986193+0000","last_active":"2024-09-21T05:57:07.380341+0000","last_peered":"2024-09-21T05:57:07.380341+0000","last_clean":"2024-09-21T05:57:07.380341+0000","last_became_active":"2024-09-21T05:55:11.985736+0000","last_became_peered":"2024-09-21T05:55:11.985736+0000","last_unstale":"2024-09-21T05:57:07.380341+0000","last_undegraded":"2024-09-21T05:57:07.380341+0000","last_fullsized":"2024-09-21T05:57:07.380341+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.503419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64459,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":215,"num_read_kb":170,"num_write":36,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.756535+0000","last_change":"2024-09-21T05:55:14.001123+0000","last_active":"2024-09-21T05:57:02.756535+0000","last_peered":"2024-09-21T05:57:02.756535+0000","last_clean":"2024-09-21T05:57:02.756535+0000","last_became_active":"2024-09-21T05:55:14.000742+0000","last_became_peered":"2024-09-21T05:55:14.000742+0000","last_unstale":"2024-09-21T05:57:02.756535+0000","last_undegraded":"2024-09-21T05:57:02.756535+0000","last_fullsized":"2024-09-21T05:57:02.756535+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:15.277992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"74'9","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639957+0000","last_change":"2024-09-21T05:55:11.981396+0000","last_active":"2024-09-21T05:55:33.639957+0000","last_peered":"2024-09-21T05:55:33.639957+0000","last_clean":"2024-09-21T05:55:33.639957+0000","last_became_active":"2024-09-21T05:55:11.980976+0000","last_became_peered":"2024-09-21T05:55:11.980976+0000","last_unstale":"2024-09-21T05:55:33.639957+0000","last_undegraded":"2024-09-21T05:55:33.639957+0000","last_fullsized":"2024-09-21T05:55:33.639957+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:43.600453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.755984+0000","last_change":"2024-09-21T05:55:13.991354+0000","last_active":"2024-09-21T05:57:02.755984+0000","last_peered":"2024-09-21T05:57:02.755984+0000","last_clean":"2024-09-21T05:57:02.755984+0000","last_became_active":"2024-09-21T05:55:13.990967+0000","last_became_peered":"2024-09-21T05:55:13.990967+0000","last_unstale":"2024-09-21T05:57:02.755984+0000","last_undegraded":"2024-09-21T05:57:02.755984+0000","last_fullsized":"2024-09-21T05:57:02.755984+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:04.889461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"74'5","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767638+0000","last_change":"2024-09-21T05:55:11.981019+0000","last_active":"2024-09-21T05:55:33.767638+0000","last_peered":"2024-09-21T05:55:33.767638+0000","last_clean":"2024-09-21T05:55:33.767638+0000","last_became_active":"2024-09-21T05:55:11.980619+0000","last_became_peered":"2024-09-21T05:55:11.980619+0000","last_unstale":"2024-09-21T05:55:33.767638+0000","last_undegraded":"2024-09-21T05:55:33.767638+0000","last_fullsized":"2024-09-21T05:55:33.767638+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:41.005092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"76'15","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.756751+0000","last_change":"2024-09-21T05:55:13.994535+0000","last_active":"2024-09-21T05:57:02.756751+0000","last_peered":"2024-09-21T05:57:02.756751+0000","last_clean":"2024-09-21T05:57:02.756751+0000","last_became_active":"2024-09-21T05:55:13.994153+0000","last_became_peered":"2024-09-21T05:55:13.994153+0000","last_unstale":"2024-09-21T05:57:02.756751+0000","last_undegraded":"2024-09-21T05:57:02.756751+0000","last_fullsized":"2024-09-21T05:57:02.756751+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:02.301263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"76'7","reported_seq":133,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:03.385021+0000","last_change":"2024-09-21T05:55:11.977694+0000","last_active":"2024-09-21T05:57:03.385021+0000","last_peered":"2024-09-21T05:57:03.385021+0000","last_clean":"2024-09-21T05:57:03.385021+0000","last_became_active":"2024-09-21T05:55:11.977146+0000","last_became_peered":"2024-09-21T05:55:11.977146+0000","last_unstale":"2024-09-21T05:57:03.385021+0000","last_undegraded":"2024-09-21T05:57:03.385021+0000","last_fullsized":"2024-09-21T05:57:03.385021+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:29.736564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.755975+0000","last_change":"2024-09-21T05:55:13.998032+0000","last_active":"2024-09-21T05:57:02.755975+0000","last_peered":"2024-09-21T05:57:02.755975+0000","last_clean":"2024-09-21T05:57:02.755975+0000","last_became_active":"2024-09-21T05:55:13.997617+0000","last_became_peered":"2024-09-21T05:55:13.997617+0000","last_unstale":"2024-09-21T05:57:02.755975+0000","last_undegraded":"2024-09-21T05:57:02.755975+0000","last_fullsized":"2024-09-21T05:57:02.755975+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:46.552726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"74'17","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635659+0000","last_change":"2024-09-21T05:55:12.013540+0000","last_active":"2024-09-21T05:55:33.635659+0000","last_peered":"2024-09-21T05:55:33.635659+0000","last_clean":"2024-09-21T05:55:33.635659+0000","last_became_active":"2024-09-21T05:55:12.013151+0000","last_became_peered":"2024-09-21T05:55:12.013151+0000","last_unstale":"2024-09-21T05:55:33.635659+0000","last_undegraded":"2024-09-21T05:55:33.635659+0000","last_fullsized":"2024-09-21T05:55:33.635659+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:53.705904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.747391+0000","last_change":"2024-09-21T05:55:13.999501+0000","last_active":"2024-09-21T05:57:02.747391+0000","last_peered":"2024-09-21T05:57:02.747391+0000","last_clean":"2024-09-21T05:57:02.747391+0000","last_became_active":"2024-09-21T05:55:13.997819+0000","last_became_peered":"2024-09-21T05:55:13.997819+0000","last_unstale":"2024-09-21T05:57:02.747391+0000","last_undegraded":"2024-09-21T05:57:02.747391+0000","last_fullsized":"2024-09-21T05:57:02.747391+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:54.334755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"74'2","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636760+0000","last_change":"2024-09-21T05:55:11.984428+0000","last_active":"2024-09-21T05:55:33.636760+0000","last_peered":"2024-09-21T05:55:33.636760+0000","last_clean":"2024-09-21T05:55:33.636760+0000","last_became_active":"2024-09-21T05:55:11.983543+0000","last_became_peered":"2024-09-21T05:55:11.983543+0000","last_unstale":"2024-09-21T05:55:33.636760+0000","last_undegraded":"2024-09-21T05:55:33.636760+0000","last_fullsized":"2024-09-21T05:55:33.636760+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:39.636408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.749459+0000","last_change":"2024-09-21T05:55:14.003851+0000","last_active":"2024-09-21T05:57:02.749459+0000","last_peered":"2024-09-21T05:57:02.749459+0000","last_clean":"2024-09-21T05:57:02.749459+0000","last_became_active":"2024-09-21T05:55:14.003354+0000","last_became_peered":"2024-09-21T05:55:14.003354+0000","last_unstale":"2024-09-21T05:57:02.749459+0000","last_undegraded":"2024-09-21T05:57:02.749459+0000","last_fullsized":"2024-09-21T05:57:02.749459+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:01.321612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"74'5","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.645976+0000","last_change":"2024-09-21T05:55:33.645825+0000","last_active":"2024-09-21T05:55:33.645976+0000","last_peered":"2024-09-21T05:55:33.645976+0000","last_clean":"2024-09-21T05:55:33.645976+0000","last_became_active":"2024-09-21T05:55:33.640707+0000","last_became_peered":"2024-09-21T05:55:33.640707+0000","last_unstale":"2024-09-21T05:55:33.645976+0000","last_undegraded":"2024-09-21T05:55:33.645976+0000","last_fullsized":"2024-09-21T05:55:33.645976+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:33.665707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.756593+0000","last_change":"2024-09-21T05:55:14.001855+0000","last_active":"2024-09-21T05:57:02.756593+0000","last_peered":"2024-09-21T05:57:02.756593+0000","last_clean":"2024-09-21T05:57:02.756593+0000","last_became_active":"2024-09-21T05:55:14.001484+0000","last_became_peered":"2024-09-21T05:55:14.001484+0000","last_unstale":"2024-09-21T05:57:02.756593+0000","last_undegraded":"2024-09-21T05:57:02.756593+0000","last_fullsized":"2024-09-21T05:57:02.756593+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:57.615334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'3","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767928+0000","last_change":"2024-09-21T05:55:11.979716+0000","last_active":"2024-09-21T05:55:33.767928+0000","last_peered":"2024-09-21T05:55:33.767928+0000","last_clean":"2024-09-21T05:55:33.767928+0000","last_became_active":"2024-09-21T05:55:11.979307+0000","last_became_peered":"2024-09-21T05:55:11.979307+0000","last_unstale":"2024-09-21T05:55:33.767928+0000","last_undegraded":"2024-09-21T05:55:33.767928+0000","last_fullsized":"2024-09-21T05:55:33.767928+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:30.019040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.755924+0000","last_change":"2024-09-21T05:55:13.991769+0000","last_active":"2024-09-21T05:57:02.755924+0000","last_peered":"2024-09-21T05:57:02.755924+0000","last_clean":"2024-09-21T05:57:02.755924+0000","last_became_active":"2024-09-21T05:55:13.991345+0000","last_became_peered":"2024-09-21T05:55:13.991345+0000","last_unstale":"2024-09-21T05:57:02.755924+0000","last_undegraded":"2024-09-21T05:57:02.755924+0000","last_fullsized":"2024-09-21T05:57:02.755924+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:52.227126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"76'7","reported_seq":136,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:06.406951+0000","last_change":"2024-09-21T05:55:12.013836+0000","last_active":"2024-09-21T05:57:06.406951+0000","last_peered":"2024-09-21T05:57:06.406951+0000","last_clean":"2024-09-21T05:57:06.406951+0000","last_became_active":"2024-09-21T05:55:12.013436+0000","last_became_peered":"2024-09-21T05:55:12.013436+0000","last_unstale":"2024-09-21T05:57:06.406951+0000","last_undegraded":"2024-09-21T05:57:06.406951+0000","last_fullsized":"2024-09-21T05:57:06.406951+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:40:02.767620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.766968+0000","last_change":"2024-09-21T05:55:14.001156+0000","last_active":"2024-09-21T05:57:02.766968+0000","last_peered":"2024-09-21T05:57:02.766968+0000","last_clean":"2024-09-21T05:57:02.766968+0000","last_became_active":"2024-09-21T05:55:14.000777+0000","last_became_peered":"2024-09-21T05:55:14.000777+0000","last_unstale":"2024-09-21T05:57:02.766968+0000","last_undegraded":"2024-09-21T05:57:02.766968+0000","last_fullsized":"2024-09-21T05:57:02.766968+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:46:08.000612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":98,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636154+0000","last_change":"2024-09-21T05:53:58.971355+0000","last_active":"2024-09-21T05:55:33.636154+0000","last_peered":"2024-09-21T05:55:33.636154+0000","last_clean":"2024-09-21T05:55:33.636154+0000","last_became_active":"2024-09-21T05:53:58.965825+0000","last_became_peered":"2024-09-21T05:53:58.965825+0000","last_unstale":"2024-09-21T05:55:33.636154+0000","last_undegraded":"2024-09-21T05:55:33.636154+0000","last_fullsized":"2024-09-21T05:55:33.636154+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:50:33.190160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:50:33.190160+0000","last_clean_scrub_stamp":"2024-09-21T05:50:33.190160+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:31.387339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"74'6","reported_seq":32,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.696020+0000","last_change":"2024-09-21T05:55:11.976092+0000","last_active":"2024-09-21T05:55:33.696020+0000","last_peered":"2024-09-21T05:55:33.696020+0000","last_clean":"2024-09-21T05:55:33.696020+0000","last_became_active":"2024-09-21T05:55:11.975657+0000","last_became_peered":"2024-09-21T05:55:11.975657+0000","last_unstale":"2024-09-21T05:55:33.696020+0000","last_undegraded":"2024-09-21T05:55:33.696020+0000","last_fullsized":"2024-09-21T05:55:33.696020+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:46.792054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.745844+0000","last_change":"2024-09-21T05:55:13.997621+0000","last_active":"2024-09-21T05:57:02.745844+0000","last_peered":"2024-09-21T05:57:02.745844+0000","last_clean":"2024-09-21T05:57:02.745844+0000","last_became_active":"2024-09-21T05:55:13.997127+0000","last_became_peered":"2024-09-21T05:55:13.997127+0000","last_unstale":"2024-09-21T05:57:02.745844+0000","last_undegraded":"2024-09-21T05:57:02.745844+0000","last_fullsized":"2024-09-21T05:57:02.745844+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:32.399696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"74'4","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635928+0000","last_change":"2024-09-21T05:55:11.982387+0000","last_active":"2024-09-21T05:55:33.635928+0000","last_peered":"2024-09-21T05:55:33.635928+0000","last_clean":"2024-09-21T05:55:33.635928+0000","last_became_active":"2024-09-21T05:55:11.981851+0000","last_became_peered":"2024-09-21T05:55:11.981851+0000","last_unstale":"2024-09-21T05:55:33.635928+0000","last_undegraded":"2024-09-21T05:55:33.635928+0000","last_fullsized":"2024-09-21T05:55:33.635928+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:45.345738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.748575+0000","last_change":"2024-09-21T05:55:13.981065+0000","last_active":"2024-09-21T05:57:02.748575+0000","last_peered":"2024-09-21T05:57:02.748575+0000","last_clean":"2024-09-21T05:57:02.748575+0000","last_became_active":"2024-09-21T05:55:13.980524+0000","last_became_peered":"2024-09-21T05:55:13.980524+0000","last_unstale":"2024-09-21T05:57:02.748575+0000","last_undegraded":"2024-09-21T05:57:02.748575+0000","last_fullsized":"2024-09-21T05:57:02.748575+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:43.322429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"76'6","reported_seq":130,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:04.385849+0000","last_change":"2024-09-21T05:55:12.013434+0000","last_active":"2024-09-21T05:57:04.385849+0000","last_peered":"2024-09-21T05:57:04.385849+0000","last_clean":"2024-09-21T05:57:04.385849+0000","last_became_active":"2024-09-21T05:55:12.013087+0000","last_became_peered":"2024-09-21T05:55:12.013087+0000","last_unstale":"2024-09-21T05:57:04.385849+0000","last_undegraded":"2024-09-21T05:57:04.385849+0000","last_fullsized":"2024-09-21T05:57:04.385849+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:19.876288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.744907+0000","last_change":"2024-09-21T05:55:13.996102+0000","last_active":"2024-09-21T05:57:02.744907+0000","last_peered":"2024-09-21T05:57:02.744907+0000","last_clean":"2024-09-21T05:57:02.744907+0000","last_became_active":"2024-09-21T05:55:13.994900+0000","last_became_peered":"2024-09-21T05:55:13.994900+0000","last_unstale":"2024-09-21T05:57:02.744907+0000","last_undegraded":"2024-09-21T05:57:02.744907+0000","last_fullsized":"2024-09-21T05:57:02.744907+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:04.998577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"74'5","reported_seq":128,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:03.782035+0000","last_change":"2024-09-21T05:55:11.981184+0000","last_active":"2024-09-21T05:57:03.782035+0000","last_peered":"2024-09-21T05:57:03.782035+0000","last_clean":"2024-09-21T05:57:03.782035+0000","last_became_active":"2024-09-21T05:55:11.979581+0000","last_became_peered":"2024-09-21T05:55:11.979581+0000","last_unstale":"2024-09-21T05:57:03.782035+0000","last_undegraded":"2024-09-21T05:57:03.782035+0000","last_fullsized":"2024-09-21T05:57:03.782035+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:53.833860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"76'7","reported_seq":32,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.736365+0000","last_change":"2024-09-21T05:55:33.768136+0000","last_active":"2024-09-21T05:57:02.736365+0000","last_peered":"2024-09-21T05:57:02.736365+0000","last_clean":"2024-09-21T05:57:02.736365+0000","last_became_active":"2024-09-21T05:55:33.767697+0000","last_became_peered":"2024-09-21T05:55:33.767697+0000","last_unstale":"2024-09-21T05:57:02.736365+0000","last_undegraded":"2024-09-21T05:57:02.736365+0000","last_fullsized":"2024-09-21T05:57:02.736365+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:48.146453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"74'1","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.633171+0000","last_change":"2024-09-21T05:55:12.013121+0000","last_active":"2024-09-21T05:55:33.633171+0000","last_peered":"2024-09-21T05:55:33.633171+0000","last_clean":"2024-09-21T05:55:33.633171+0000","last_became_active":"2024-09-21T05:55:12.012657+0000","last_became_peered":"2024-09-21T05:55:12.012657+0000","last_unstale":"2024-09-21T05:55:33.633171+0000","last_undegraded":"2024-09-21T05:55:33.633171+0000","last_fullsized":"2024-09-21T05:55:33.633171+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:25:48.002938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.747160+0000","last_change":"2024-09-21T05:55:14.004660+0000","last_active":"2024-09-21T05:57:02.747160+0000","last_peered":"2024-09-21T05:57:02.747160+0000","last_clean":"2024-09-21T05:57:02.747160+0000","last_became_active":"2024-09-21T05:55:14.004297+0000","last_became_peered":"2024-09-21T05:55:14.004297+0000","last_unstale":"2024-09-21T05:57:02.747160+0000","last_undegraded":"2024-09-21T05:57:02.747160+0000","last_fullsized":"2024-09-21T05:57:02.747160+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:24.399891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"74'4","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636395+0000","last_change":"2024-09-21T05:55:12.015717+0000","last_active":"2024-09-21T05:55:33.636395+0000","last_peered":"2024-09-21T05:55:33.636395+0000","last_clean":"2024-09-21T05:55:33.636395+0000","last_became_active":"2024-09-21T05:55:12.015419+0000","last_became_peered":"2024-09-21T05:55:12.015419+0000","last_unstale":"2024-09-21T05:55:33.636395+0000","last_undegraded":"2024-09-21T05:55:33.636395+0000","last_fullsized":"2024-09-21T05:55:33.636395+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:57.988860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.747609+0000","last_change":"2024-09-21T05:55:13.992460+0000","last_active":"2024-09-21T05:57:02.747609+0000","last_peered":"2024-09-21T05:57:02.747609+0000","last_clean":"2024-09-21T05:57:02.747609+0000","last_became_active":"2024-09-21T05:55:13.991887+0000","last_became_peered":"2024-09-21T05:55:13.991887+0000","last_unstale":"2024-09-21T05:57:02.747609+0000","last_undegraded":"2024-09-21T05:57:02.747609+0000","last_fullsized":"2024-09-21T05:57:02.747609+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:37.718050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"76'17","reported_seq":40,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:03.008081+0000","last_change":"2024-09-21T05:55:11.982235+0000","last_active":"2024-09-21T05:57:03.008081+0000","last_peered":"2024-09-21T05:57:03.008081+0000","last_clean":"2024-09-21T05:57:03.008081+0000","last_became_active":"2024-09-21T05:55:11.981846+0000","last_became_peered":"2024-09-21T05:55:11.981846+0000","last_unstale":"2024-09-21T05:57:03.008081+0000","last_undegraded":"2024-09-21T05:57:03.008081+0000","last_fullsized":"2024-09-21T05:57:03.008081+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:40:04.330346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.744826+0000","last_change":"2024-09-21T05:55:13.999249+0000","last_active":"2024-09-21T05:57:02.744826+0000","last_peered":"2024-09-21T05:57:02.744826+0000","last_clean":"2024-09-21T05:57:02.744826+0000","last_became_active":"2024-09-21T05:55:13.998452+0000","last_became_peered":"2024-09-21T05:55:13.998452+0000","last_unstale":"2024-09-21T05:57:02.744826+0000","last_undegraded":"2024-09-21T05:57:02.744826+0000","last_fullsized":"2024-09-21T05:57:02.744826+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:59.364684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"74'7","reported_seq":31,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.696351+0000","last_change":"2024-09-21T05:55:11.971850+0000","last_active":"2024-09-21T05:55:33.696351+0000","last_peered":"2024-09-21T05:55:33.696351+0000","last_clean":"2024-09-21T05:55:33.696351+0000","last_became_active":"2024-09-21T05:55:11.971415+0000","last_became_peered":"2024-09-21T05:55:11.971415+0000","last_unstale":"2024-09-21T05:55:33.696351+0000","last_undegraded":"2024-09-21T05:55:33.696351+0000","last_fullsized":"2024-09-21T05:55:33.696351+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:51.990745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.743839+0000","last_change":"2024-09-21T05:55:13.994500+0000","last_active":"2024-09-21T05:57:02.743839+0000","last_peered":"2024-09-21T05:57:02.743839+0000","last_clean":"2024-09-21T05:57:02.743839+0000","last_became_active":"2024-09-21T05:55:13.994146+0000","last_became_peered":"2024-09-21T05:55:13.994146+0000","last_unstale":"2024-09-21T05:57:02.743839+0000","last_undegraded":"2024-09-21T05:57:02.743839+0000","last_fullsized":"2024-09-21T05:57:02.743839+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:42.934234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639268+0000","last_change":"2024-09-21T05:55:12.013479+0000","last_active":"2024-09-21T05:55:33.639268+0000","last_peered":"2024-09-21T05:55:33.639268+0000","last_clean":"2024-09-21T05:55:33.639268+0000","last_became_active":"2024-09-21T05:55:12.013180+0000","last_became_peered":"2024-09-21T05:55:12.013180+0000","last_unstale":"2024-09-21T05:55:33.639268+0000","last_undegraded":"2024-09-21T05:55:33.639268+0000","last_fullsized":"2024-09-21T05:55:33.639268+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:28.800072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"76'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.709028+0000","last_change":"2024-09-21T05:55:14.001795+0000","last_active":"2024-09-21T05:57:02.709028+0000","last_peered":"2024-09-21T05:57:02.709028+0000","last_clean":"2024-09-21T05:57:02.709028+0000","last_became_active":"2024-09-21T05:55:14.001430+0000","last_became_peered":"2024-09-21T05:55:14.001430+0000","last_unstale":"2024-09-21T05:57:02.709028+0000","last_undegraded":"2024-09-21T05:57:02.709028+0000","last_fullsized":"2024-09-21T05:57:02.709028+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:10.339939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"74'7","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.638793+0000","last_change":"2024-09-21T05:55:11.976894+0000","last_active":"2024-09-21T05:55:33.638793+0000","last_peered":"2024-09-21T05:55:33.638793+0000","last_clean":"2024-09-21T05:55:33.638793+0000","last_became_active":"2024-09-21T05:55:11.976516+0000","last_became_peered":"2024-09-21T05:55:11.976516+0000","last_unstale":"2024-09-21T05:55:33.638793+0000","last_undegraded":"2024-09-21T05:55:33.638793+0000","last_fullsized":"2024-09-21T05:55:33.638793+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:58.726311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.746513+0000","last_change":"2024-09-21T05:55:14.003760+0000","last_active":"2024-09-21T05:57:02.746513+0000","last_peered":"2024-09-21T05:57:02.746513+0000","last_clean":"2024-09-21T05:57:02.746513+0000","last_became_active":"2024-09-21T05:55:14.003180+0000","last_became_peered":"2024-09-21T05:55:14.003180+0000","last_unstale":"2024-09-21T05:57:02.746513+0000","last_undegraded":"2024-09-21T05:57:02.746513+0000","last_fullsized":"2024-09-21T05:57:02.746513+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:25.077765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639713+0000","last_change":"2024-09-21T05:55:11.983865+0000","last_active":"2024-09-21T05:55:33.639713+0000","last_peered":"2024-09-21T05:55:33.639713+0000","last_clean":"2024-09-21T05:55:33.639713+0000","last_became_active":"2024-09-21T05:55:11.983473+0000","last_became_peered":"2024-09-21T05:55:11.983473+0000","last_unstale":"2024-09-21T05:55:33.639713+0000","last_undegraded":"2024-09-21T05:55:33.639713+0000","last_fullsized":"2024-09-21T05:55:33.639713+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:18.767317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.741938+0000","last_change":"2024-09-21T05:55:13.999001+0000","last_active":"2024-09-21T05:57:02.741938+0000","last_peered":"2024-09-21T05:57:02.741938+0000","last_clean":"2024-09-21T05:57:02.741938+0000","last_became_active":"2024-09-21T05:55:13.998656+0000","last_became_peered":"2024-09-21T05:55:13.998656+0000","last_unstale":"2024-09-21T05:57:02.741938+0000","last_undegraded":"2024-09-21T05:57:02.741938+0000","last_fullsized":"2024-09-21T05:57:02.741938+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:50.913635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"74'4","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.646791+0000","last_change":"2024-09-21T05:55:33.646124+0000","last_active":"2024-09-21T05:55:33.646791+0000","last_peered":"2024-09-21T05:55:33.646791+0000","last_clean":"2024-09-21T05:55:33.646791+0000","last_became_active":"2024-09-21T05:55:33.641142+0000","last_became_peered":"2024-09-21T05:55:33.641142+0000","last_unstale":"2024-09-21T05:55:33.646791+0000","last_undegraded":"2024-09-21T05:55:33.646791+0000","last_fullsized":"2024-09-21T05:55:33.646791+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:36.341473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.767041+0000","last_change":"2024-09-21T05:55:13.997494+0000","last_active":"2024-09-21T05:57:02.767041+0000","last_peered":"2024-09-21T05:57:02.767041+0000","last_clean":"2024-09-21T05:57:02.767041+0000","last_became_active":"2024-09-21T05:55:13.997138+0000","last_became_peered":"2024-09-21T05:55:13.997138+0000","last_unstale":"2024-09-21T05:57:02.767041+0000","last_undegraded":"2024-09-21T05:57:02.767041+0000","last_fullsized":"2024-09-21T05:57:02.767041+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:02:01.777126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"76'15","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.747951+0000","last_change":"2024-09-21T05:55:14.002750+0000","last_active":"2024-09-21T05:57:02.747951+0000","last_peered":"2024-09-21T05:57:02.747951+0000","last_clean":"2024-09-21T05:57:02.747951+0000","last_became_active":"2024-09-21T05:55:14.002150+0000","last_became_peered":"2024-09-21T05:55:14.002150+0000","last_unstale":"2024-09-21T05:57:02.747951+0000","last_undegraded":"2024-09-21T05:57:02.747951+0000","last_fullsized":"2024-09-21T05:57:02.747951+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:49.603133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"74'5","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636032+0000","last_change":"2024-09-21T05:55:11.982366+0000","last_active":"2024-09-21T05:55:33.636032+0000","last_peered":"2024-09-21T05:55:33.636032+0000","last_clean":"2024-09-21T05:55:33.636032+0000","last_became_active":"2024-09-21T05:55:11.981994+0000","last_became_peered":"2024-09-21T05:55:11.981994+0000","last_unstale":"2024-09-21T05:55:33.636032+0000","last_undegraded":"2024-09-21T05:55:33.636032+0000","last_fullsized":"2024-09-21T05:55:33.636032+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:14.626758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.769310+0000","last_change":"2024-09-21T05:55:14.008204+0000","last_active":"2024-09-21T05:57:02.769310+0000","last_peered":"2024-09-21T05:57:02.769310+0000","last_clean":"2024-09-21T05:57:02.769310+0000","last_became_active":"2024-09-21T05:55:14.007800+0000","last_became_peered":"2024-09-21T05:55:14.007800+0000","last_unstale":"2024-09-21T05:57:02.769310+0000","last_undegraded":"2024-09-21T05:57:02.769310+0000","last_fullsized":"2024-09-21T05:57:02.769310+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:49.760850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"74'12","reported_seq":36,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636283+0000","last_change":"2024-09-21T05:55:11.986168+0000","last_active":"2024-09-21T05:55:33.636283+0000","last_peered":"2024-09-21T05:55:33.636283+0000","last_clean":"2024-09-21T05:55:33.636283+0000","last_became_active":"2024-09-21T05:55:11.985694+0000","last_became_peered":"2024-09-21T05:55:11.985694+0000","last_unstale":"2024-09-21T05:55:33.636283+0000","last_undegraded":"2024-09-21T05:55:33.636283+0000","last_fullsized":"2024-09-21T05:55:33.636283+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:44.863935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.748453+0000","last_change":"2024-09-21T05:55:14.002459+0000","last_active":"2024-09-21T05:57:02.748453+0000","last_peered":"2024-09-21T05:57:02.748453+0000","last_clean":"2024-09-21T05:57:02.748453+0000","last_became_active":"2024-09-21T05:55:14.001962+0000","last_became_peered":"2024-09-21T05:55:14.001962+0000","last_unstale":"2024-09-21T05:57:02.748453+0000","last_undegraded":"2024-09-21T05:57:02.748453+0000","last_fullsized":"2024-09-21T05:57:02.748453+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:53:26.892772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639340+0000","last_change":"2024-09-21T05:55:12.014149+0000","last_active":"2024-09-21T05:55:33.639340+0000","last_peered":"2024-09-21T05:55:33.639340+0000","last_clean":"2024-09-21T05:55:33.639340+0000","last_became_active":"2024-09-21T05:55:12.013789+0000","last_became_peered":"2024-09-21T05:55:12.013789+0000","last_unstale":"2024-09-21T05:55:33.639340+0000","last_undegraded":"2024-09-21T05:55:33.639340+0000","last_fullsized":"2024-09-21T05:55:33.639340+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:18.304023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.749464+0000","last_change":"2024-09-21T05:55:13.993500+0000","last_active":"2024-09-21T05:57:02.749464+0000","last_peered":"2024-09-21T05:57:02.749464+0000","last_clean":"2024-09-21T05:57:02.749464+0000","last_became_active":"2024-09-21T05:55:13.993104+0000","last_became_peered":"2024-09-21T05:55:13.993104+0000","last_unstale":"2024-09-21T05:57:02.749464+0000","last_undegraded":"2024-09-21T05:57:02.749464+0000","last_fullsized":"2024-09-21T05:57:02.749464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:01.445119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"76'14","reported_seq":159,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:05.784296+0000","last_change":"2024-09-21T05:55:33.650207+0000","last_active":"2024-09-21T05:57:05.784296+0000","last_peered":"2024-09-21T05:57:05.784296+0000","last_clean":"2024-09-21T05:57:05.784296+0000","last_became_active":"2024-09-21T05:55:33.638388+0000","last_became_peered":"2024-09-21T05:55:33.638388+0000","last_unstale":"2024-09-21T05:57:05.784296+0000","last_undegraded":"2024-09-21T05:57:05.784296+0000","last_fullsized":"2024-09-21T05:57:05.784296+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:51.478063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72477,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1047,"num_read_kb":603,"num_write":260,"num_write_kb":154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":217739,"data_compressed":5682,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281769,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":68,"seq":292057776164,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":839680,"data_stored":412700,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939689,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":1067516,"data_compressed":43475,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103214,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":856064,"data_stored":411426,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299444,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":411804,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,0,0,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495672,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":1004323,"data_compressed":41581,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659198,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":839680,"data_stored":411143,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822722,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":1004141,"data_compressed":41581,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986248,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":839680,"data_stored":411536,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36180,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708972,"statfs":{"total":95995035648,"available":95957987328,"internally_reserved":0,"allocated":856064,"data_stored":410968,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378707,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":477799,"data_compressed":7646,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542230,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":901120,"data_stored":476823,"data_compressed":7646,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705757,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36184,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708968,"statfs":{"total":95995035648,"available":95957983232,"internally_reserved":0,"allocated":860160,"data_stored":413014,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34208,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34208,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34208,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35829,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35829,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35829,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35829,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35829,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":66519,"data_compressed":1894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":67495,"data_compressed":1894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":66727,"data_compressed":1894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:57:10.063 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:57:10.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:09 smithi138 ceph-mon[34391]: pgmap v322: 97 pgs: 97 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1023 B/s wr, 12 op/s 2024-09-21T05:57:10.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:09 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/1050515444' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:57:10.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.205+0000 7f2098313640 1 -- 172.21.15.117:0/3342363345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2090105290 msgr2=0x7f209010d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:10.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.205+0000 7f2098313640 1 --2- 172.21.15.117:0/3342363345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2090105290 0x7f209010d7d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f208c009900 tx=0x7f208c02a410 comp rx=0 tx=0).stop 2024-09-21T05:57:10.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.206+0000 7f2098313640 1 -- 172.21.15.117:0/3342363345 shutdown_connections 2024-09-21T05:57:10.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.206+0000 7f2098313640 1 --2- 172.21.15.117:0/3342363345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2090105290 0x7f209010d7d0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.206+0000 7f2098313640 1 --2- 172.21.15.117:0/3342363345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f209010dd60 0x7f2090110240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.206+0000 7f2098313640 1 --2- 172.21.15.117:0/3342363345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2090104940 0x7f2090104d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.206+0000 7f2098313640 1 -- 172.21.15.117:0/3342363345 >> 172.21.15.117:0/3342363345 conn(0x7f20900fe350 msgr2=0x7f20901007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:10.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.206+0000 7f2098313640 1 -- 172.21.15.117:0/3342363345 shutdown_connections 2024-09-21T05:57:10.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.206+0000 7f2098313640 1 -- 172.21.15.117:0/3342363345 wait complete. 2024-09-21T05:57:10.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.206+0000 7f2098313640 1 Processor -- start 2024-09-21T05:57:10.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.206+0000 7f2098313640 1 -- start start 2024-09-21T05:57:10.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.206+0000 7f2098313640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2090104940 0x7f209019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:10.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.206+0000 7f2098313640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2090105290 0x7f209019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:10.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.206+0000 7f2098313640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f209010dd60 0x7f209019fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:10.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.206+0000 7f2098313640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f20901139b0 con 0x7f209010dd60 2024-09-21T05:57:10.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.206+0000 7f2098313640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2090113830 con 0x7f2090105290 2024-09-21T05:57:10.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.206+0000 7f2098313640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2090113b30 con 0x7f2090104940 2024-09-21T05:57:10.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.207+0000 7f20968b0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f209010dd60 0x7f209019fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:10.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.207+0000 7f20968b0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f209010dd60 0x7f209019fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60362/0 (socket says 172.21.15.117:60362) 2024-09-21T05:57:10.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.207+0000 7f20968b0640 1 -- 172.21.15.117:0/315146293 learned_addr learned my addr 172.21.15.117:0/315146293 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:10.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.207+0000 7f20960af640 1 --2- 172.21.15.117:0/315146293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2090104940 0x7f209019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:10.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.207+0000 7f20958ae640 1 --2- 172.21.15.117:0/315146293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2090105290 0x7f209019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:10.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.207+0000 7f20968b0640 1 -- 172.21.15.117:0/315146293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2090104940 msgr2=0x7f209019a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:10.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.207+0000 7f20968b0640 1 --2- 172.21.15.117:0/315146293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2090104940 0x7f209019a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.207+0000 7f20968b0640 1 -- 172.21.15.117:0/315146293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2090105290 msgr2=0x7f209019a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:10.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.207+0000 7f20968b0640 1 --2- 172.21.15.117:0/315146293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2090105290 0x7f209019a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.207+0000 7f20968b0640 1 -- 172.21.15.117:0/315146293 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20901a0340 con 0x7f209010dd60 2024-09-21T05:57:10.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.207+0000 7f20958ae640 1 --2- 172.21.15.117:0/315146293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2090105290 0x7f209019a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:57:10.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.207+0000 7f20968b0640 1 --2- 172.21.15.117:0/315146293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f209010dd60 0x7f209019fc40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f208800eaa0 tx=0x7f208800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:10.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.207+0000 7f20837fe640 1 -- 172.21.15.117:0/315146293 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208800cdb0 con 0x7f209010dd60 2024-09-21T05:57:10.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.207+0000 7f20837fe640 1 -- 172.21.15.117:0/315146293 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2088005020 con 0x7f209010dd60 2024-09-21T05:57:10.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.207+0000 7f20837fe640 1 -- 172.21.15.117:0/315146293 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20880116d0 con 0x7f209010dd60 2024-09-21T05:57:10.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.207+0000 7f2098313640 1 -- 172.21.15.117:0/315146293 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20901a0630 con 0x7f209010dd60 2024-09-21T05:57:10.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.207+0000 7f2098313640 1 -- 172.21.15.117:0/315146293 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20901a6dc0 con 0x7f209010dd60 2024-09-21T05:57:10.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.208+0000 7f2098313640 1 -- 172.21.15.117:0/315146293 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2090104d50 con 0x7f209010dd60 2024-09-21T05:57:10.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.210+0000 7f20837fe640 1 -- 172.21.15.117:0/315146293 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 22) ==== 154530+0+0 (secure 0 0 0) 0x7f20880051d0 con 0x7f209010dd60 2024-09-21T05:57:10.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.211+0000 7f20837fe640 1 --2- 172.21.15.117:0/315146293 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f207c0b6560 0x7f207c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:10.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.211+0000 7f20960af640 1 --2- 172.21.15.117:0/315146293 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f207c0b6560 0x7f207c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:10.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.211+0000 7f20837fe640 1 -- 172.21.15.117:0/315146293 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f20880e7a20 con 0x7f209010dd60 2024-09-21T05:57:10.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.211+0000 7f20960af640 1 --2- 172.21.15.117:0/315146293 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f207c0b6560 0x7f207c0b8a70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2084005fd0 tx=0x7f208400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:10.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.214+0000 7f20837fe640 1 -- 172.21.15.117:0/315146293 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2088016030 con 0x7f209010dd60 2024-09-21T05:57:10.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.387+0000 7f2098313640 1 -- 172.21.15.117:0/315146293 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2090107fc0 con 0x7f207c0b6560 2024-09-21T05:57:10.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.391+0000 7f20837fe640 1 -- 172.21.15.117:0/315146293 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259322 (secure 0 0 0) 0x7f2090107fc0 con 0x7f207c0b6560 2024-09-21T05:57:10.395 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:57:10.396 INFO:teuthology.orchestra.run.smithi117.stderr:dumped all 2024-09-21T05:57:10.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.394+0000 7f2098313640 1 -- 172.21.15.117:0/315146293 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f207c0b6560 msgr2=0x7f207c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:10.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.394+0000 7f2098313640 1 --2- 172.21.15.117:0/315146293 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f207c0b6560 0x7f207c0b8a70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2084005fd0 tx=0x7f208400b390 comp rx=0 tx=0).stop 2024-09-21T05:57:10.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.394+0000 7f2098313640 1 -- 172.21.15.117:0/315146293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f209010dd60 msgr2=0x7f209019fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:10.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.394+0000 7f2098313640 1 --2- 172.21.15.117:0/315146293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f209010dd60 0x7f209019fc40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f208800eaa0 tx=0x7f208800ef60 comp rx=0 tx=0).stop 2024-09-21T05:57:10.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.394+0000 7f2098313640 1 -- 172.21.15.117:0/315146293 shutdown_connections 2024-09-21T05:57:10.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.394+0000 7f2098313640 1 --2- 172.21.15.117:0/315146293 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f207c0b6560 0x7f207c0b8a70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.394+0000 7f2098313640 1 --2- 172.21.15.117:0/315146293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f209010dd60 0x7f209019fc40 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.394+0000 7f2098313640 1 --2- 172.21.15.117:0/315146293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2090105290 0x7f209019a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.394+0000 7f2098313640 1 --2- 172.21.15.117:0/315146293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2090104940 0x7f209019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.394+0000 7f2098313640 1 -- 172.21.15.117:0/315146293 >> 172.21.15.117:0/315146293 conn(0x7f20900fe350 msgr2=0x7f20901078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:10.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.394+0000 7f2098313640 1 -- 172.21.15.117:0/315146293 shutdown_connections 2024-09-21T05:57:10.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.394+0000 7f2098313640 1 -- 172.21.15.117:0/315146293 wait complete. 2024-09-21T05:57:10.408 INFO:teuthology.orchestra.run.smithi117.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-09-21T05:57:08.534958+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663079,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1128,"num_read_kb":672,"num_write":385,"num_write_kb":2306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":2952512,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":817,"ondisk_log_size":817,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376208,"kb_used_data":10320,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565616,"statfs":{"total":1151940427776,"available":1151555190784,"internally_reserved":0,"allocated":10567680,"data_stored":6913193,"data_compressed":182193,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,0,0,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":980,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":75,"num_write":15,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002924"},"pg_stats":[{"pgid":"3.3f","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.767187+0000","last_change":"2024-09-21T05:55:14.007642+0000","last_active":"2024-09-21T05:57:02.767187+0000","last_peered":"2024-09-21T05:57:02.767187+0000","last_clean":"2024-09-21T05:57:02.767187+0000","last_became_active":"2024-09-21T05:55:14.006943+0000","last_became_peered":"2024-09-21T05:55:14.006943+0000","last_unstale":"2024-09-21T05:57:02.767187+0000","last_undegraded":"2024-09-21T05:57:02.767187+0000","last_fullsized":"2024-09-21T05:57:02.767187+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:23:53.816079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"76'4","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.748660+0000","last_change":"2024-09-21T05:55:14.002034+0000","last_active":"2024-09-21T05:57:02.748660+0000","last_peered":"2024-09-21T05:57:02.748660+0000","last_clean":"2024-09-21T05:57:02.748660+0000","last_became_active":"2024-09-21T05:55:14.001658+0000","last_became_peered":"2024-09-21T05:55:14.001658+0000","last_unstale":"2024-09-21T05:57:02.748660+0000","last_undegraded":"2024-09-21T05:57:02.748660+0000","last_fullsized":"2024-09-21T05:57:02.748660+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:01.568362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.746220+0000","last_change":"2024-09-21T05:55:13.996901+0000","last_active":"2024-09-21T05:57:02.746220+0000","last_peered":"2024-09-21T05:57:02.746220+0000","last_clean":"2024-09-21T05:57:02.746220+0000","last_became_active":"2024-09-21T05:55:13.996020+0000","last_became_peered":"2024-09-21T05:55:13.996020+0000","last_unstale":"2024-09-21T05:57:02.746220+0000","last_undegraded":"2024-09-21T05:57:02.746220+0000","last_fullsized":"2024-09-21T05:57:02.746220+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:38.771926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.750182+0000","last_change":"2024-09-21T05:55:13.995968+0000","last_active":"2024-09-21T05:57:02.750182+0000","last_peered":"2024-09-21T05:57:02.750182+0000","last_clean":"2024-09-21T05:57:02.750182+0000","last_became_active":"2024-09-21T05:55:13.995259+0000","last_became_peered":"2024-09-21T05:55:13.995259+0000","last_unstale":"2024-09-21T05:57:02.750182+0000","last_undegraded":"2024-09-21T05:57:02.750182+0000","last_fullsized":"2024-09-21T05:57:02.750182+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:48.953342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"76'4","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.726754+0000","last_change":"2024-09-21T05:55:13.995778+0000","last_active":"2024-09-21T05:57:02.726754+0000","last_peered":"2024-09-21T05:57:02.726754+0000","last_clean":"2024-09-21T05:57:02.726754+0000","last_became_active":"2024-09-21T05:55:13.995415+0000","last_became_peered":"2024-09-21T05:55:13.995415+0000","last_unstale":"2024-09-21T05:57:02.726754+0000","last_undegraded":"2024-09-21T05:57:02.726754+0000","last_fullsized":"2024-09-21T05:57:02.726754+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:52.562120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.737104+0000","last_change":"2024-09-21T05:55:14.000005+0000","last_active":"2024-09-21T05:57:02.737104+0000","last_peered":"2024-09-21T05:57:02.737104+0000","last_clean":"2024-09-21T05:57:02.737104+0000","last_became_active":"2024-09-21T05:55:13.999493+0000","last_became_peered":"2024-09-21T05:55:13.999493+0000","last_unstale":"2024-09-21T05:57:02.737104+0000","last_undegraded":"2024-09-21T05:57:02.737104+0000","last_fullsized":"2024-09-21T05:57:02.737104+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:29.426368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.756030+0000","last_change":"2024-09-21T05:55:13.993004+0000","last_active":"2024-09-21T05:57:02.756030+0000","last_peered":"2024-09-21T05:57:02.756030+0000","last_clean":"2024-09-21T05:57:02.756030+0000","last_became_active":"2024-09-21T05:55:13.992615+0000","last_became_peered":"2024-09-21T05:55:13.992615+0000","last_unstale":"2024-09-21T05:57:02.756030+0000","last_undegraded":"2024-09-21T05:57:02.756030+0000","last_fullsized":"2024-09-21T05:57:02.756030+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:48.324618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"76'9","reported_seq":21,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.745963+0000","last_change":"2024-09-21T05:55:33.769006+0000","last_active":"2024-09-21T05:57:02.745963+0000","last_peered":"2024-09-21T05:57:02.745963+0000","last_clean":"2024-09-21T05:57:02.745963+0000","last_became_active":"2024-09-21T05:55:33.768705+0000","last_became_peered":"2024-09-21T05:55:33.768705+0000","last_unstale":"2024-09-21T05:57:02.745963+0000","last_undegraded":"2024-09-21T05:57:02.745963+0000","last_fullsized":"2024-09-21T05:57:02.745963+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:14.013300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.765471+0000","last_change":"2024-09-21T05:55:14.004131+0000","last_active":"2024-09-21T05:57:02.765471+0000","last_peered":"2024-09-21T05:57:02.765471+0000","last_clean":"2024-09-21T05:57:02.765471+0000","last_became_active":"2024-09-21T05:55:14.003621+0000","last_became_peered":"2024-09-21T05:55:14.003621+0000","last_unstale":"2024-09-21T05:57:02.765471+0000","last_undegraded":"2024-09-21T05:57:02.765471+0000","last_fullsized":"2024-09-21T05:57:02.765471+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:00.694211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"76'16","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.750873+0000","last_change":"2024-09-21T05:55:33.643289+0000","last_active":"2024-09-21T05:57:02.750873+0000","last_peered":"2024-09-21T05:57:02.750873+0000","last_clean":"2024-09-21T05:57:02.750873+0000","last_became_active":"2024-09-21T05:55:33.642805+0000","last_became_peered":"2024-09-21T05:55:33.642805+0000","last_unstale":"2024-09-21T05:57:02.750873+0000","last_undegraded":"2024-09-21T05:57:02.750873+0000","last_fullsized":"2024-09-21T05:57:02.750873+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:00.854712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.766867+0000","last_change":"2024-09-21T05:55:14.006467+0000","last_active":"2024-09-21T05:57:02.766867+0000","last_peered":"2024-09-21T05:57:02.766867+0000","last_clean":"2024-09-21T05:57:02.766867+0000","last_became_active":"2024-09-21T05:55:14.005978+0000","last_became_peered":"2024-09-21T05:55:14.005978+0000","last_unstale":"2024-09-21T05:57:02.766867+0000","last_undegraded":"2024-09-21T05:57:02.766867+0000","last_fullsized":"2024-09-21T05:57:02.766867+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:37.391152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.747861+0000","last_change":"2024-09-21T05:55:14.003972+0000","last_active":"2024-09-21T05:57:02.747861+0000","last_peered":"2024-09-21T05:57:02.747861+0000","last_clean":"2024-09-21T05:57:02.747861+0000","last_became_active":"2024-09-21T05:55:14.003601+0000","last_became_peered":"2024-09-21T05:55:14.003601+0000","last_unstale":"2024-09-21T05:57:02.747861+0000","last_undegraded":"2024-09-21T05:57:02.747861+0000","last_fullsized":"2024-09-21T05:57:02.747861+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:16.201087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.751561+0000","last_change":"2024-09-21T05:55:13.996051+0000","last_active":"2024-09-21T05:57:02.751561+0000","last_peered":"2024-09-21T05:57:02.751561+0000","last_clean":"2024-09-21T05:57:02.751561+0000","last_became_active":"2024-09-21T05:55:13.995679+0000","last_became_peered":"2024-09-21T05:55:13.995679+0000","last_unstale":"2024-09-21T05:57:02.751561+0000","last_undegraded":"2024-09-21T05:57:02.751561+0000","last_fullsized":"2024-09-21T05:57:02.751561+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:45:44.372029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.745041+0000","last_change":"2024-09-21T05:55:13.995421+0000","last_active":"2024-09-21T05:57:02.745041+0000","last_peered":"2024-09-21T05:57:02.745041+0000","last_clean":"2024-09-21T05:57:02.745041+0000","last_became_active":"2024-09-21T05:55:13.995006+0000","last_became_peered":"2024-09-21T05:55:13.995006+0000","last_unstale":"2024-09-21T05:57:02.745041+0000","last_undegraded":"2024-09-21T05:57:02.745041+0000","last_fullsized":"2024-09-21T05:57:02.745041+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:00.636555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.745532+0000","last_change":"2024-09-21T05:55:13.993466+0000","last_active":"2024-09-21T05:57:02.745532+0000","last_peered":"2024-09-21T05:57:02.745532+0000","last_clean":"2024-09-21T05:57:02.745532+0000","last_became_active":"2024-09-21T05:55:13.993137+0000","last_became_peered":"2024-09-21T05:55:13.993137+0000","last_unstale":"2024-09-21T05:57:02.745532+0000","last_undegraded":"2024-09-21T05:57:02.745532+0000","last_fullsized":"2024-09-21T05:57:02.745532+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:33.218868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.765648+0000","last_change":"2024-09-21T05:55:13.996316+0000","last_active":"2024-09-21T05:57:02.765648+0000","last_peered":"2024-09-21T05:57:02.765648+0000","last_clean":"2024-09-21T05:57:02.765648+0000","last_became_active":"2024-09-21T05:55:13.995918+0000","last_became_peered":"2024-09-21T05:55:13.995918+0000","last_unstale":"2024-09-21T05:57:02.765648+0000","last_undegraded":"2024-09-21T05:57:02.765648+0000","last_fullsized":"2024-09-21T05:57:02.765648+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:45:54.408504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.767326+0000","last_change":"2024-09-21T05:55:14.007198+0000","last_active":"2024-09-21T05:57:02.767326+0000","last_peered":"2024-09-21T05:57:02.767326+0000","last_clean":"2024-09-21T05:57:02.767326+0000","last_became_active":"2024-09-21T05:55:14.006700+0000","last_became_peered":"2024-09-21T05:55:14.006700+0000","last_unstale":"2024-09-21T05:57:02.767326+0000","last_undegraded":"2024-09-21T05:57:02.767326+0000","last_fullsized":"2024-09-21T05:57:02.767326+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:45.343778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.765593+0000","last_change":"2024-09-21T05:55:14.000152+0000","last_active":"2024-09-21T05:57:02.765593+0000","last_peered":"2024-09-21T05:57:02.765593+0000","last_clean":"2024-09-21T05:57:02.765593+0000","last_became_active":"2024-09-21T05:55:13.999786+0000","last_became_peered":"2024-09-21T05:55:13.999786+0000","last_unstale":"2024-09-21T05:57:02.765593+0000","last_undegraded":"2024-09-21T05:57:02.765593+0000","last_fullsized":"2024-09-21T05:57:02.765593+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:43:25.713410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.749050+0000","last_change":"2024-09-21T05:55:13.988367+0000","last_active":"2024-09-21T05:57:02.749050+0000","last_peered":"2024-09-21T05:57:02.749050+0000","last_clean":"2024-09-21T05:57:02.749050+0000","last_became_active":"2024-09-21T05:55:13.987982+0000","last_became_peered":"2024-09-21T05:55:13.987982+0000","last_unstale":"2024-09-21T05:57:02.749050+0000","last_undegraded":"2024-09-21T05:57:02.749050+0000","last_fullsized":"2024-09-21T05:57:02.749050+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:15.395155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.748991+0000","last_change":"2024-09-21T05:55:14.004962+0000","last_active":"2024-09-21T05:57:02.748991+0000","last_peered":"2024-09-21T05:57:02.748991+0000","last_clean":"2024-09-21T05:57:02.748991+0000","last_became_active":"2024-09-21T05:55:14.004573+0000","last_became_peered":"2024-09-21T05:55:14.004573+0000","last_unstale":"2024-09-21T05:57:02.748991+0000","last_undegraded":"2024-09-21T05:57:02.748991+0000","last_fullsized":"2024-09-21T05:57:02.748991+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:59.014017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.756283+0000","last_change":"2024-09-21T05:55:13.991267+0000","last_active":"2024-09-21T05:57:02.756283+0000","last_peered":"2024-09-21T05:57:02.756283+0000","last_clean":"2024-09-21T05:57:02.756283+0000","last_became_active":"2024-09-21T05:55:13.990916+0000","last_became_peered":"2024-09-21T05:55:13.990916+0000","last_unstale":"2024-09-21T05:57:02.756283+0000","last_undegraded":"2024-09-21T05:57:02.756283+0000","last_fullsized":"2024-09-21T05:57:02.756283+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:06:03.904632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.766270+0000","last_change":"2024-09-21T05:55:13.994147+0000","last_active":"2024-09-21T05:57:02.766270+0000","last_peered":"2024-09-21T05:57:02.766270+0000","last_clean":"2024-09-21T05:57:02.766270+0000","last_became_active":"2024-09-21T05:55:13.993794+0000","last_became_peered":"2024-09-21T05:55:13.993794+0000","last_unstale":"2024-09-21T05:57:02.766270+0000","last_undegraded":"2024-09-21T05:57:02.766270+0000","last_fullsized":"2024-09-21T05:57:02.766270+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:19.970853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.750316+0000","last_change":"2024-09-21T05:55:14.001119+0000","last_active":"2024-09-21T05:57:02.750316+0000","last_peered":"2024-09-21T05:57:02.750316+0000","last_clean":"2024-09-21T05:57:02.750316+0000","last_became_active":"2024-09-21T05:55:14.000799+0000","last_became_peered":"2024-09-21T05:55:14.000799+0000","last_unstale":"2024-09-21T05:57:02.750316+0000","last_undegraded":"2024-09-21T05:57:02.750316+0000","last_fullsized":"2024-09-21T05:57:02.750316+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:28.259188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.739740+0000","last_change":"2024-09-21T05:55:13.992043+0000","last_active":"2024-09-21T05:57:02.739740+0000","last_peered":"2024-09-21T05:57:02.739740+0000","last_clean":"2024-09-21T05:57:02.739740+0000","last_became_active":"2024-09-21T05:55:13.991344+0000","last_became_peered":"2024-09-21T05:55:13.991344+0000","last_unstale":"2024-09-21T05:57:02.739740+0000","last_undegraded":"2024-09-21T05:57:02.739740+0000","last_fullsized":"2024-09-21T05:57:02.739740+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:42:55.674333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"76'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.749506+0000","last_change":"2024-09-21T05:55:13.993114+0000","last_active":"2024-09-21T05:57:02.749506+0000","last_peered":"2024-09-21T05:57:02.749506+0000","last_clean":"2024-09-21T05:57:02.749506+0000","last_became_active":"2024-09-21T05:55:13.992749+0000","last_became_peered":"2024-09-21T05:55:13.992749+0000","last_unstale":"2024-09-21T05:57:02.749506+0000","last_undegraded":"2024-09-21T05:57:02.749506+0000","last_fullsized":"2024-09-21T05:57:02.749506+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:15.926967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"76'15","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.763355+0000","last_change":"2024-09-21T05:55:13.997199+0000","last_active":"2024-09-21T05:57:02.763355+0000","last_peered":"2024-09-21T05:57:02.763355+0000","last_clean":"2024-09-21T05:57:02.763355+0000","last_became_active":"2024-09-21T05:55:13.996809+0000","last_became_peered":"2024-09-21T05:55:13.996809+0000","last_unstale":"2024-09-21T05:57:02.763355+0000","last_undegraded":"2024-09-21T05:57:02.763355+0000","last_fullsized":"2024-09-21T05:57:02.763355+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:16.681430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.751124+0000","last_change":"2024-09-21T05:55:14.000879+0000","last_active":"2024-09-21T05:57:02.751124+0000","last_peered":"2024-09-21T05:57:02.751124+0000","last_clean":"2024-09-21T05:57:02.751124+0000","last_became_active":"2024-09-21T05:55:14.000259+0000","last_became_peered":"2024-09-21T05:55:14.000259+0000","last_unstale":"2024-09-21T05:57:02.751124+0000","last_undegraded":"2024-09-21T05:57:02.751124+0000","last_fullsized":"2024-09-21T05:57:02.751124+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:01:08.938417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"76'4","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.744616+0000","last_change":"2024-09-21T05:55:13.989433+0000","last_active":"2024-09-21T05:57:02.744616+0000","last_peered":"2024-09-21T05:57:02.744616+0000","last_clean":"2024-09-21T05:57:02.744616+0000","last_became_active":"2024-09-21T05:55:13.989097+0000","last_became_peered":"2024-09-21T05:55:13.989097+0000","last_unstale":"2024-09-21T05:57:02.744616+0000","last_undegraded":"2024-09-21T05:57:02.744616+0000","last_fullsized":"2024-09-21T05:57:02.744616+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:37.835298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.742763+0000","last_change":"2024-09-21T05:55:13.993911+0000","last_active":"2024-09-21T05:57:02.742763+0000","last_peered":"2024-09-21T05:57:02.742763+0000","last_clean":"2024-09-21T05:57:02.742763+0000","last_became_active":"2024-09-21T05:55:13.993570+0000","last_became_peered":"2024-09-21T05:55:13.993570+0000","last_unstale":"2024-09-21T05:57:02.742763+0000","last_undegraded":"2024-09-21T05:57:02.742763+0000","last_fullsized":"2024-09-21T05:57:02.742763+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:08:16.588457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.751759+0000","last_change":"2024-09-21T05:55:13.992764+0000","last_active":"2024-09-21T05:57:02.751759+0000","last_peered":"2024-09-21T05:57:02.751759+0000","last_clean":"2024-09-21T05:57:02.751759+0000","last_became_active":"2024-09-21T05:55:13.992338+0000","last_became_peered":"2024-09-21T05:55:13.992338+0000","last_unstale":"2024-09-21T05:57:02.751759+0000","last_undegraded":"2024-09-21T05:57:02.751759+0000","last_fullsized":"2024-09-21T05:57:02.751759+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:25:29.997353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"76'12","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.752225+0000","last_change":"2024-09-21T05:55:13.987119+0000","last_active":"2024-09-21T05:57:02.752225+0000","last_peered":"2024-09-21T05:57:02.752225+0000","last_clean":"2024-09-21T05:57:02.752225+0000","last_became_active":"2024-09-21T05:55:13.986663+0000","last_became_peered":"2024-09-21T05:55:13.986663+0000","last_unstale":"2024-09-21T05:57:02.752225+0000","last_undegraded":"2024-09-21T05:57:02.752225+0000","last_fullsized":"2024-09-21T05:57:02.752225+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:02.089833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.751424+0000","last_change":"2024-09-21T05:55:14.003577+0000","last_active":"2024-09-21T05:57:02.751424+0000","last_peered":"2024-09-21T05:57:02.751424+0000","last_clean":"2024-09-21T05:57:02.751424+0000","last_became_active":"2024-09-21T05:55:14.003162+0000","last_became_peered":"2024-09-21T05:55:14.003162+0000","last_unstale":"2024-09-21T05:57:02.751424+0000","last_undegraded":"2024-09-21T05:57:02.751424+0000","last_fullsized":"2024-09-21T05:57:02.751424+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:58.630843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.730878+0000","last_change":"2024-09-21T05:55:13.992078+0000","last_active":"2024-09-21T05:57:02.730878+0000","last_peered":"2024-09-21T05:57:02.730878+0000","last_clean":"2024-09-21T05:57:02.730878+0000","last_became_active":"2024-09-21T05:55:13.991680+0000","last_became_peered":"2024-09-21T05:55:13.991680+0000","last_unstale":"2024-09-21T05:57:02.730878+0000","last_undegraded":"2024-09-21T05:57:02.730878+0000","last_fullsized":"2024-09-21T05:57:02.730878+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:52.886755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"76'7","reported_seq":133,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:03.373713+0000","last_change":"2024-09-21T05:55:11.982560+0000","last_active":"2024-09-21T05:57:03.373713+0000","last_peered":"2024-09-21T05:57:03.373713+0000","last_clean":"2024-09-21T05:57:03.373713+0000","last_became_active":"2024-09-21T05:55:11.982179+0000","last_became_peered":"2024-09-21T05:55:11.982179+0000","last_unstale":"2024-09-21T05:57:03.373713+0000","last_undegraded":"2024-09-21T05:57:03.373713+0000","last_fullsized":"2024-09-21T05:57:03.373713+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:06.012194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.740537+0000","last_change":"2024-09-21T05:55:13.993994+0000","last_active":"2024-09-21T05:57:02.740537+0000","last_peered":"2024-09-21T05:57:02.740537+0000","last_clean":"2024-09-21T05:57:02.740537+0000","last_became_active":"2024-09-21T05:55:13.993585+0000","last_became_peered":"2024-09-21T05:55:13.993585+0000","last_unstale":"2024-09-21T05:57:02.740537+0000","last_undegraded":"2024-09-21T05:57:02.740537+0000","last_fullsized":"2024-09-21T05:57:02.740537+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:11.608616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"74'7","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.634233+0000","last_change":"2024-09-21T05:55:11.984397+0000","last_active":"2024-09-21T05:55:33.634233+0000","last_peered":"2024-09-21T05:55:33.634233+0000","last_clean":"2024-09-21T05:55:33.634233+0000","last_became_active":"2024-09-21T05:55:11.983888+0000","last_became_peered":"2024-09-21T05:55:11.983888+0000","last_unstale":"2024-09-21T05:55:33.634233+0000","last_undegraded":"2024-09-21T05:55:33.634233+0000","last_fullsized":"2024-09-21T05:55:33.634233+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:59.012710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636187+0000","last_change":"2024-09-21T05:55:11.974547+0000","last_active":"2024-09-21T05:55:33.636187+0000","last_peered":"2024-09-21T05:55:33.636187+0000","last_clean":"2024-09-21T05:55:33.636187+0000","last_became_active":"2024-09-21T05:55:11.974151+0000","last_became_peered":"2024-09-21T05:55:11.974151+0000","last_unstale":"2024-09-21T05:55:33.636187+0000","last_undegraded":"2024-09-21T05:55:33.636187+0000","last_fullsized":"2024-09-21T05:55:33.636187+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:51.489548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.746384+0000","last_change":"2024-09-21T05:55:13.999287+0000","last_active":"2024-09-21T05:57:02.746384+0000","last_peered":"2024-09-21T05:57:02.746384+0000","last_clean":"2024-09-21T05:57:02.746384+0000","last_became_active":"2024-09-21T05:55:13.998742+0000","last_became_peered":"2024-09-21T05:55:13.998742+0000","last_unstale":"2024-09-21T05:57:02.746384+0000","last_undegraded":"2024-09-21T05:57:02.746384+0000","last_fullsized":"2024-09-21T05:57:02.746384+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:33.842810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"74'3","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.768113+0000","last_change":"2024-09-21T05:55:11.981354+0000","last_active":"2024-09-21T05:55:33.768113+0000","last_peered":"2024-09-21T05:55:33.768113+0000","last_clean":"2024-09-21T05:55:33.768113+0000","last_became_active":"2024-09-21T05:55:11.980993+0000","last_became_peered":"2024-09-21T05:55:11.980993+0000","last_unstale":"2024-09-21T05:55:33.768113+0000","last_undegraded":"2024-09-21T05:55:33.768113+0000","last_fullsized":"2024-09-21T05:55:33.768113+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:05.037669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.746795+0000","last_change":"2024-09-21T05:55:14.001041+0000","last_active":"2024-09-21T05:57:02.746795+0000","last_peered":"2024-09-21T05:57:02.746795+0000","last_clean":"2024-09-21T05:57:02.746795+0000","last_became_active":"2024-09-21T05:55:14.000688+0000","last_became_peered":"2024-09-21T05:55:14.000688+0000","last_unstale":"2024-09-21T05:57:02.746795+0000","last_undegraded":"2024-09-21T05:57:02.746795+0000","last_fullsized":"2024-09-21T05:57:02.746795+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:51.190495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"76'6","reported_seq":233,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:07.379709+0000","last_change":"2024-09-21T05:55:11.984097+0000","last_active":"2024-09-21T05:57:07.379709+0000","last_peered":"2024-09-21T05:57:07.379709+0000","last_clean":"2024-09-21T05:57:07.379709+0000","last_became_active":"2024-09-21T05:55:11.982969+0000","last_became_peered":"2024-09-21T05:55:11.982969+0000","last_unstale":"2024-09-21T05:57:07.379709+0000","last_undegraded":"2024-09-21T05:57:07.379709+0000","last_fullsized":"2024-09-21T05:57:07.379709+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:41.372225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":204,"num_read_kb":204,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.768826+0000","last_change":"2024-09-21T05:55:13.998433+0000","last_active":"2024-09-21T05:57:02.768826+0000","last_peered":"2024-09-21T05:57:02.768826+0000","last_clean":"2024-09-21T05:57:02.768826+0000","last_became_active":"2024-09-21T05:55:13.997890+0000","last_became_peered":"2024-09-21T05:55:13.997890+0000","last_unstale":"2024-09-21T05:57:02.768826+0000","last_undegraded":"2024-09-21T05:57:02.768826+0000","last_fullsized":"2024-09-21T05:57:02.768826+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:24.396789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"74'4","reported_seq":40,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.701709+0000","last_change":"2024-09-21T05:55:33.701599+0000","last_active":"2024-09-21T05:55:33.701709+0000","last_peered":"2024-09-21T05:55:33.701709+0000","last_clean":"2024-09-21T05:55:33.701709+0000","last_became_active":"2024-09-21T05:55:33.697385+0000","last_became_peered":"2024-09-21T05:55:33.697385+0000","last_unstale":"2024-09-21T05:55:33.701709+0000","last_undegraded":"2024-09-21T05:55:33.701709+0000","last_fullsized":"2024-09-21T05:55:33.701709+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:18.653991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.741741+0000","last_change":"2024-09-21T05:55:13.995111+0000","last_active":"2024-09-21T05:57:02.741741+0000","last_peered":"2024-09-21T05:57:02.741741+0000","last_clean":"2024-09-21T05:57:02.741741+0000","last_became_active":"2024-09-21T05:55:13.994691+0000","last_became_peered":"2024-09-21T05:55:13.994691+0000","last_unstale":"2024-09-21T05:57:02.741741+0000","last_undegraded":"2024-09-21T05:57:02.741741+0000","last_fullsized":"2024-09-21T05:57:02.741741+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:30.167649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"74'5","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767919+0000","last_change":"2024-09-21T05:55:12.013961+0000","last_active":"2024-09-21T05:55:33.767919+0000","last_peered":"2024-09-21T05:55:33.767919+0000","last_clean":"2024-09-21T05:55:33.767919+0000","last_became_active":"2024-09-21T05:55:12.013573+0000","last_became_peered":"2024-09-21T05:55:12.013573+0000","last_unstale":"2024-09-21T05:55:33.767919+0000","last_undegraded":"2024-09-21T05:55:33.767919+0000","last_fullsized":"2024-09-21T05:55:33.767919+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:12.119605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.768347+0000","last_change":"2024-09-21T05:55:13.996093+0000","last_active":"2024-09-21T05:57:02.768347+0000","last_peered":"2024-09-21T05:57:02.768347+0000","last_clean":"2024-09-21T05:57:02.768347+0000","last_became_active":"2024-09-21T05:55:13.995342+0000","last_became_peered":"2024-09-21T05:55:13.995342+0000","last_unstale":"2024-09-21T05:57:02.768347+0000","last_undegraded":"2024-09-21T05:57:02.768347+0000","last_fullsized":"2024-09-21T05:57:02.768347+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:20.335330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"76'34","reported_seq":272,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:07.380341+0000","last_change":"2024-09-21T05:55:11.986193+0000","last_active":"2024-09-21T05:57:07.380341+0000","last_peered":"2024-09-21T05:57:07.380341+0000","last_clean":"2024-09-21T05:57:07.380341+0000","last_became_active":"2024-09-21T05:55:11.985736+0000","last_became_peered":"2024-09-21T05:55:11.985736+0000","last_unstale":"2024-09-21T05:57:07.380341+0000","last_undegraded":"2024-09-21T05:57:07.380341+0000","last_fullsized":"2024-09-21T05:57:07.380341+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.503419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64459,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":215,"num_read_kb":170,"num_write":36,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.756535+0000","last_change":"2024-09-21T05:55:14.001123+0000","last_active":"2024-09-21T05:57:02.756535+0000","last_peered":"2024-09-21T05:57:02.756535+0000","last_clean":"2024-09-21T05:57:02.756535+0000","last_became_active":"2024-09-21T05:55:14.000742+0000","last_became_peered":"2024-09-21T05:55:14.000742+0000","last_unstale":"2024-09-21T05:57:02.756535+0000","last_undegraded":"2024-09-21T05:57:02.756535+0000","last_fullsized":"2024-09-21T05:57:02.756535+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:15.277992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"74'9","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639957+0000","last_change":"2024-09-21T05:55:11.981396+0000","last_active":"2024-09-21T05:55:33.639957+0000","last_peered":"2024-09-21T05:55:33.639957+0000","last_clean":"2024-09-21T05:55:33.639957+0000","last_became_active":"2024-09-21T05:55:11.980976+0000","last_became_peered":"2024-09-21T05:55:11.980976+0000","last_unstale":"2024-09-21T05:55:33.639957+0000","last_undegraded":"2024-09-21T05:55:33.639957+0000","last_fullsized":"2024-09-21T05:55:33.639957+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:43.600453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.755984+0000","last_change":"2024-09-21T05:55:13.991354+0000","last_active":"2024-09-21T05:57:02.755984+0000","last_peered":"2024-09-21T05:57:02.755984+0000","last_clean":"2024-09-21T05:57:02.755984+0000","last_became_active":"2024-09-21T05:55:13.990967+0000","last_became_peered":"2024-09-21T05:55:13.990967+0000","last_unstale":"2024-09-21T05:57:02.755984+0000","last_undegraded":"2024-09-21T05:57:02.755984+0000","last_fullsized":"2024-09-21T05:57:02.755984+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:04.889461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"74'5","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767638+0000","last_change":"2024-09-21T05:55:11.981019+0000","last_active":"2024-09-21T05:55:33.767638+0000","last_peered":"2024-09-21T05:55:33.767638+0000","last_clean":"2024-09-21T05:55:33.767638+0000","last_became_active":"2024-09-21T05:55:11.980619+0000","last_became_peered":"2024-09-21T05:55:11.980619+0000","last_unstale":"2024-09-21T05:55:33.767638+0000","last_undegraded":"2024-09-21T05:55:33.767638+0000","last_fullsized":"2024-09-21T05:55:33.767638+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:41.005092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"76'15","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.756751+0000","last_change":"2024-09-21T05:55:13.994535+0000","last_active":"2024-09-21T05:57:02.756751+0000","last_peered":"2024-09-21T05:57:02.756751+0000","last_clean":"2024-09-21T05:57:02.756751+0000","last_became_active":"2024-09-21T05:55:13.994153+0000","last_became_peered":"2024-09-21T05:55:13.994153+0000","last_unstale":"2024-09-21T05:57:02.756751+0000","last_undegraded":"2024-09-21T05:57:02.756751+0000","last_fullsized":"2024-09-21T05:57:02.756751+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:02.301263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"76'7","reported_seq":133,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:03.385021+0000","last_change":"2024-09-21T05:55:11.977694+0000","last_active":"2024-09-21T05:57:03.385021+0000","last_peered":"2024-09-21T05:57:03.385021+0000","last_clean":"2024-09-21T05:57:03.385021+0000","last_became_active":"2024-09-21T05:55:11.977146+0000","last_became_peered":"2024-09-21T05:55:11.977146+0000","last_unstale":"2024-09-21T05:57:03.385021+0000","last_undegraded":"2024-09-21T05:57:03.385021+0000","last_fullsized":"2024-09-21T05:57:03.385021+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:29.736564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.755975+0000","last_change":"2024-09-21T05:55:13.998032+0000","last_active":"2024-09-21T05:57:02.755975+0000","last_peered":"2024-09-21T05:57:02.755975+0000","last_clean":"2024-09-21T05:57:02.755975+0000","last_became_active":"2024-09-21T05:55:13.997617+0000","last_became_peered":"2024-09-21T05:55:13.997617+0000","last_unstale":"2024-09-21T05:57:02.755975+0000","last_undegraded":"2024-09-21T05:57:02.755975+0000","last_fullsized":"2024-09-21T05:57:02.755975+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:46.552726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"74'17","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635659+0000","last_change":"2024-09-21T05:55:12.013540+0000","last_active":"2024-09-21T05:55:33.635659+0000","last_peered":"2024-09-21T05:55:33.635659+0000","last_clean":"2024-09-21T05:55:33.635659+0000","last_became_active":"2024-09-21T05:55:12.013151+0000","last_became_peered":"2024-09-21T05:55:12.013151+0000","last_unstale":"2024-09-21T05:55:33.635659+0000","last_undegraded":"2024-09-21T05:55:33.635659+0000","last_fullsized":"2024-09-21T05:55:33.635659+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:53.705904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.747391+0000","last_change":"2024-09-21T05:55:13.999501+0000","last_active":"2024-09-21T05:57:02.747391+0000","last_peered":"2024-09-21T05:57:02.747391+0000","last_clean":"2024-09-21T05:57:02.747391+0000","last_became_active":"2024-09-21T05:55:13.997819+0000","last_became_peered":"2024-09-21T05:55:13.997819+0000","last_unstale":"2024-09-21T05:57:02.747391+0000","last_undegraded":"2024-09-21T05:57:02.747391+0000","last_fullsized":"2024-09-21T05:57:02.747391+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:54.334755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"74'2","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636760+0000","last_change":"2024-09-21T05:55:11.984428+0000","last_active":"2024-09-21T05:55:33.636760+0000","last_peered":"2024-09-21T05:55:33.636760+0000","last_clean":"2024-09-21T05:55:33.636760+0000","last_became_active":"2024-09-21T05:55:11.983543+0000","last_became_peered":"2024-09-21T05:55:11.983543+0000","last_unstale":"2024-09-21T05:55:33.636760+0000","last_undegraded":"2024-09-21T05:55:33.636760+0000","last_fullsized":"2024-09-21T05:55:33.636760+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:39.636408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.749459+0000","last_change":"2024-09-21T05:55:14.003851+0000","last_active":"2024-09-21T05:57:02.749459+0000","last_peered":"2024-09-21T05:57:02.749459+0000","last_clean":"2024-09-21T05:57:02.749459+0000","last_became_active":"2024-09-21T05:55:14.003354+0000","last_became_peered":"2024-09-21T05:55:14.003354+0000","last_unstale":"2024-09-21T05:57:02.749459+0000","last_undegraded":"2024-09-21T05:57:02.749459+0000","last_fullsized":"2024-09-21T05:57:02.749459+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:01.321612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"74'5","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.645976+0000","last_change":"2024-09-21T05:55:33.645825+0000","last_active":"2024-09-21T05:55:33.645976+0000","last_peered":"2024-09-21T05:55:33.645976+0000","last_clean":"2024-09-21T05:55:33.645976+0000","last_became_active":"2024-09-21T05:55:33.640707+0000","last_became_peered":"2024-09-21T05:55:33.640707+0000","last_unstale":"2024-09-21T05:55:33.645976+0000","last_undegraded":"2024-09-21T05:55:33.645976+0000","last_fullsized":"2024-09-21T05:55:33.645976+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:33.665707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.756593+0000","last_change":"2024-09-21T05:55:14.001855+0000","last_active":"2024-09-21T05:57:02.756593+0000","last_peered":"2024-09-21T05:57:02.756593+0000","last_clean":"2024-09-21T05:57:02.756593+0000","last_became_active":"2024-09-21T05:55:14.001484+0000","last_became_peered":"2024-09-21T05:55:14.001484+0000","last_unstale":"2024-09-21T05:57:02.756593+0000","last_undegraded":"2024-09-21T05:57:02.756593+0000","last_fullsized":"2024-09-21T05:57:02.756593+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:57.615334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'3","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767928+0000","last_change":"2024-09-21T05:55:11.979716+0000","last_active":"2024-09-21T05:55:33.767928+0000","last_peered":"2024-09-21T05:55:33.767928+0000","last_clean":"2024-09-21T05:55:33.767928+0000","last_became_active":"2024-09-21T05:55:11.979307+0000","last_became_peered":"2024-09-21T05:55:11.979307+0000","last_unstale":"2024-09-21T05:55:33.767928+0000","last_undegraded":"2024-09-21T05:55:33.767928+0000","last_fullsized":"2024-09-21T05:55:33.767928+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:30.019040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.755924+0000","last_change":"2024-09-21T05:55:13.991769+0000","last_active":"2024-09-21T05:57:02.755924+0000","last_peered":"2024-09-21T05:57:02.755924+0000","last_clean":"2024-09-21T05:57:02.755924+0000","last_became_active":"2024-09-21T05:55:13.991345+0000","last_became_peered":"2024-09-21T05:55:13.991345+0000","last_unstale":"2024-09-21T05:57:02.755924+0000","last_undegraded":"2024-09-21T05:57:02.755924+0000","last_fullsized":"2024-09-21T05:57:02.755924+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:52.227126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"76'7","reported_seq":136,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:06.406951+0000","last_change":"2024-09-21T05:55:12.013836+0000","last_active":"2024-09-21T05:57:06.406951+0000","last_peered":"2024-09-21T05:57:06.406951+0000","last_clean":"2024-09-21T05:57:06.406951+0000","last_became_active":"2024-09-21T05:55:12.013436+0000","last_became_peered":"2024-09-21T05:55:12.013436+0000","last_unstale":"2024-09-21T05:57:06.406951+0000","last_undegraded":"2024-09-21T05:57:06.406951+0000","last_fullsized":"2024-09-21T05:57:06.406951+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:40:02.767620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.766968+0000","last_change":"2024-09-21T05:55:14.001156+0000","last_active":"2024-09-21T05:57:02.766968+0000","last_peered":"2024-09-21T05:57:02.766968+0000","last_clean":"2024-09-21T05:57:02.766968+0000","last_became_active":"2024-09-21T05:55:14.000777+0000","last_became_peered":"2024-09-21T05:55:14.000777+0000","last_unstale":"2024-09-21T05:57:02.766968+0000","last_undegraded":"2024-09-21T05:57:02.766968+0000","last_fullsized":"2024-09-21T05:57:02.766968+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:46:08.000612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":98,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636154+0000","last_change":"2024-09-21T05:53:58.971355+0000","last_active":"2024-09-21T05:55:33.636154+0000","last_peered":"2024-09-21T05:55:33.636154+0000","last_clean":"2024-09-21T05:55:33.636154+0000","last_became_active":"2024-09-21T05:53:58.965825+0000","last_became_peered":"2024-09-21T05:53:58.965825+0000","last_unstale":"2024-09-21T05:55:33.636154+0000","last_undegraded":"2024-09-21T05:55:33.636154+0000","last_fullsized":"2024-09-21T05:55:33.636154+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:50:33.190160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:50:33.190160+0000","last_clean_scrub_stamp":"2024-09-21T05:50:33.190160+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:31.387339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"74'6","reported_seq":32,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.696020+0000","last_change":"2024-09-21T05:55:11.976092+0000","last_active":"2024-09-21T05:55:33.696020+0000","last_peered":"2024-09-21T05:55:33.696020+0000","last_clean":"2024-09-21T05:55:33.696020+0000","last_became_active":"2024-09-21T05:55:11.975657+0000","last_became_peered":"2024-09-21T05:55:11.975657+0000","last_unstale":"2024-09-21T05:55:33.696020+0000","last_undegraded":"2024-09-21T05:55:33.696020+0000","last_fullsized":"2024-09-21T05:55:33.696020+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:46.792054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.745844+0000","last_change":"2024-09-21T05:55:13.997621+0000","last_active":"2024-09-21T05:57:02.745844+0000","last_peered":"2024-09-21T05:57:02.745844+0000","last_clean":"2024-09-21T05:57:02.745844+0000","last_became_active":"2024-09-21T05:55:13.997127+0000","last_became_peered":"2024-09-21T05:55:13.997127+0000","last_unstale":"2024-09-21T05:57:02.745844+0000","last_undegraded":"2024-09-21T05:57:02.745844+0000","last_fullsized":"2024-09-21T05:57:02.745844+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:32.399696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"74'4","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635928+0000","last_change":"2024-09-21T05:55:11.982387+0000","last_active":"2024-09-21T05:55:33.635928+0000","last_peered":"2024-09-21T05:55:33.635928+0000","last_clean":"2024-09-21T05:55:33.635928+0000","last_became_active":"2024-09-21T05:55:11.981851+0000","last_became_peered":"2024-09-21T05:55:11.981851+0000","last_unstale":"2024-09-21T05:55:33.635928+0000","last_undegraded":"2024-09-21T05:55:33.635928+0000","last_fullsized":"2024-09-21T05:55:33.635928+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:45.345738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.748575+0000","last_change":"2024-09-21T05:55:13.981065+0000","last_active":"2024-09-21T05:57:02.748575+0000","last_peered":"2024-09-21T05:57:02.748575+0000","last_clean":"2024-09-21T05:57:02.748575+0000","last_became_active":"2024-09-21T05:55:13.980524+0000","last_became_peered":"2024-09-21T05:55:13.980524+0000","last_unstale":"2024-09-21T05:57:02.748575+0000","last_undegraded":"2024-09-21T05:57:02.748575+0000","last_fullsized":"2024-09-21T05:57:02.748575+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:43.322429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"76'6","reported_seq":130,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:04.385849+0000","last_change":"2024-09-21T05:55:12.013434+0000","last_active":"2024-09-21T05:57:04.385849+0000","last_peered":"2024-09-21T05:57:04.385849+0000","last_clean":"2024-09-21T05:57:04.385849+0000","last_became_active":"2024-09-21T05:55:12.013087+0000","last_became_peered":"2024-09-21T05:55:12.013087+0000","last_unstale":"2024-09-21T05:57:04.385849+0000","last_undegraded":"2024-09-21T05:57:04.385849+0000","last_fullsized":"2024-09-21T05:57:04.385849+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:19.876288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.744907+0000","last_change":"2024-09-21T05:55:13.996102+0000","last_active":"2024-09-21T05:57:02.744907+0000","last_peered":"2024-09-21T05:57:02.744907+0000","last_clean":"2024-09-21T05:57:02.744907+0000","last_became_active":"2024-09-21T05:55:13.994900+0000","last_became_peered":"2024-09-21T05:55:13.994900+0000","last_unstale":"2024-09-21T05:57:02.744907+0000","last_undegraded":"2024-09-21T05:57:02.744907+0000","last_fullsized":"2024-09-21T05:57:02.744907+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:04.998577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"74'5","reported_seq":128,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:03.782035+0000","last_change":"2024-09-21T05:55:11.981184+0000","last_active":"2024-09-21T05:57:03.782035+0000","last_peered":"2024-09-21T05:57:03.782035+0000","last_clean":"2024-09-21T05:57:03.782035+0000","last_became_active":"2024-09-21T05:55:11.979581+0000","last_became_peered":"2024-09-21T05:55:11.979581+0000","last_unstale":"2024-09-21T05:57:03.782035+0000","last_undegraded":"2024-09-21T05:57:03.782035+0000","last_fullsized":"2024-09-21T05:57:03.782035+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:53.833860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"76'7","reported_seq":32,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.736365+0000","last_change":"2024-09-21T05:55:33.768136+0000","last_active":"2024-09-21T05:57:02.736365+0000","last_peered":"2024-09-21T05:57:02.736365+0000","last_clean":"2024-09-21T05:57:02.736365+0000","last_became_active":"2024-09-21T05:55:33.767697+0000","last_became_peered":"2024-09-21T05:55:33.767697+0000","last_unstale":"2024-09-21T05:57:02.736365+0000","last_undegraded":"2024-09-21T05:57:02.736365+0000","last_fullsized":"2024-09-21T05:57:02.736365+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:48.146453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"74'1","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.633171+0000","last_change":"2024-09-21T05:55:12.013121+0000","last_active":"2024-09-21T05:55:33.633171+0000","last_peered":"2024-09-21T05:55:33.633171+0000","last_clean":"2024-09-21T05:55:33.633171+0000","last_became_active":"2024-09-21T05:55:12.012657+0000","last_became_peered":"2024-09-21T05:55:12.012657+0000","last_unstale":"2024-09-21T05:55:33.633171+0000","last_undegraded":"2024-09-21T05:55:33.633171+0000","last_fullsized":"2024-09-21T05:55:33.633171+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:25:48.002938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.747160+0000","last_change":"2024-09-21T05:55:14.004660+0000","last_active":"2024-09-21T05:57:02.747160+0000","last_peered":"2024-09-21T05:57:02.747160+0000","last_clean":"2024-09-21T05:57:02.747160+0000","last_became_active":"2024-09-21T05:55:14.004297+0000","last_became_peered":"2024-09-21T05:55:14.004297+0000","last_unstale":"2024-09-21T05:57:02.747160+0000","last_undegraded":"2024-09-21T05:57:02.747160+0000","last_fullsized":"2024-09-21T05:57:02.747160+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:24.399891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"74'4","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636395+0000","last_change":"2024-09-21T05:55:12.015717+0000","last_active":"2024-09-21T05:55:33.636395+0000","last_peered":"2024-09-21T05:55:33.636395+0000","last_clean":"2024-09-21T05:55:33.636395+0000","last_became_active":"2024-09-21T05:55:12.015419+0000","last_became_peered":"2024-09-21T05:55:12.015419+0000","last_unstale":"2024-09-21T05:55:33.636395+0000","last_undegraded":"2024-09-21T05:55:33.636395+0000","last_fullsized":"2024-09-21T05:55:33.636395+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:57.988860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.747609+0000","last_change":"2024-09-21T05:55:13.992460+0000","last_active":"2024-09-21T05:57:02.747609+0000","last_peered":"2024-09-21T05:57:02.747609+0000","last_clean":"2024-09-21T05:57:02.747609+0000","last_became_active":"2024-09-21T05:55:13.991887+0000","last_became_peered":"2024-09-21T05:55:13.991887+0000","last_unstale":"2024-09-21T05:57:02.747609+0000","last_undegraded":"2024-09-21T05:57:02.747609+0000","last_fullsized":"2024-09-21T05:57:02.747609+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:37.718050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"76'17","reported_seq":40,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:03.008081+0000","last_change":"2024-09-21T05:55:11.982235+0000","last_active":"2024-09-21T05:57:03.008081+0000","last_peered":"2024-09-21T05:57:03.008081+0000","last_clean":"2024-09-21T05:57:03.008081+0000","last_became_active":"2024-09-21T05:55:11.981846+0000","last_became_peered":"2024-09-21T05:55:11.981846+0000","last_unstale":"2024-09-21T05:57:03.008081+0000","last_undegraded":"2024-09-21T05:57:03.008081+0000","last_fullsized":"2024-09-21T05:57:03.008081+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:40:04.330346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.744826+0000","last_change":"2024-09-21T05:55:13.999249+0000","last_active":"2024-09-21T05:57:02.744826+0000","last_peered":"2024-09-21T05:57:02.744826+0000","last_clean":"2024-09-21T05:57:02.744826+0000","last_became_active":"2024-09-21T05:55:13.998452+0000","last_became_peered":"2024-09-21T05:55:13.998452+0000","last_unstale":"2024-09-21T05:57:02.744826+0000","last_undegraded":"2024-09-21T05:57:02.744826+0000","last_fullsized":"2024-09-21T05:57:02.744826+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:59.364684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"74'7","reported_seq":31,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.696351+0000","last_change":"2024-09-21T05:55:11.971850+0000","last_active":"2024-09-21T05:55:33.696351+0000","last_peered":"2024-09-21T05:55:33.696351+0000","last_clean":"2024-09-21T05:55:33.696351+0000","last_became_active":"2024-09-21T05:55:11.971415+0000","last_became_peered":"2024-09-21T05:55:11.971415+0000","last_unstale":"2024-09-21T05:55:33.696351+0000","last_undegraded":"2024-09-21T05:55:33.696351+0000","last_fullsized":"2024-09-21T05:55:33.696351+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:51.990745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.743839+0000","last_change":"2024-09-21T05:55:13.994500+0000","last_active":"2024-09-21T05:57:02.743839+0000","last_peered":"2024-09-21T05:57:02.743839+0000","last_clean":"2024-09-21T05:57:02.743839+0000","last_became_active":"2024-09-21T05:55:13.994146+0000","last_became_peered":"2024-09-21T05:55:13.994146+0000","last_unstale":"2024-09-21T05:57:02.743839+0000","last_undegraded":"2024-09-21T05:57:02.743839+0000","last_fullsized":"2024-09-21T05:57:02.743839+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:42.934234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639268+0000","last_change":"2024-09-21T05:55:12.013479+0000","last_active":"2024-09-21T05:55:33.639268+0000","last_peered":"2024-09-21T05:55:33.639268+0000","last_clean":"2024-09-21T05:55:33.639268+0000","last_became_active":"2024-09-21T05:55:12.013180+0000","last_became_peered":"2024-09-21T05:55:12.013180+0000","last_unstale":"2024-09-21T05:55:33.639268+0000","last_undegraded":"2024-09-21T05:55:33.639268+0000","last_fullsized":"2024-09-21T05:55:33.639268+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:28.800072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"76'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.709028+0000","last_change":"2024-09-21T05:55:14.001795+0000","last_active":"2024-09-21T05:57:02.709028+0000","last_peered":"2024-09-21T05:57:02.709028+0000","last_clean":"2024-09-21T05:57:02.709028+0000","last_became_active":"2024-09-21T05:55:14.001430+0000","last_became_peered":"2024-09-21T05:55:14.001430+0000","last_unstale":"2024-09-21T05:57:02.709028+0000","last_undegraded":"2024-09-21T05:57:02.709028+0000","last_fullsized":"2024-09-21T05:57:02.709028+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:10.339939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"74'7","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.638793+0000","last_change":"2024-09-21T05:55:11.976894+0000","last_active":"2024-09-21T05:55:33.638793+0000","last_peered":"2024-09-21T05:55:33.638793+0000","last_clean":"2024-09-21T05:55:33.638793+0000","last_became_active":"2024-09-21T05:55:11.976516+0000","last_became_peered":"2024-09-21T05:55:11.976516+0000","last_unstale":"2024-09-21T05:55:33.638793+0000","last_undegraded":"2024-09-21T05:55:33.638793+0000","last_fullsized":"2024-09-21T05:55:33.638793+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:58.726311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.746513+0000","last_change":"2024-09-21T05:55:14.003760+0000","last_active":"2024-09-21T05:57:02.746513+0000","last_peered":"2024-09-21T05:57:02.746513+0000","last_clean":"2024-09-21T05:57:02.746513+0000","last_became_active":"2024-09-21T05:55:14.003180+0000","last_became_peered":"2024-09-21T05:55:14.003180+0000","last_unstale":"2024-09-21T05:57:02.746513+0000","last_undegraded":"2024-09-21T05:57:02.746513+0000","last_fullsized":"2024-09-21T05:57:02.746513+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:25.077765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639713+0000","last_change":"2024-09-21T05:55:11.983865+0000","last_active":"2024-09-21T05:55:33.639713+0000","last_peered":"2024-09-21T05:55:33.639713+0000","last_clean":"2024-09-21T05:55:33.639713+0000","last_became_active":"2024-09-21T05:55:11.983473+0000","last_became_peered":"2024-09-21T05:55:11.983473+0000","last_unstale":"2024-09-21T05:55:33.639713+0000","last_undegraded":"2024-09-21T05:55:33.639713+0000","last_fullsized":"2024-09-21T05:55:33.639713+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:18.767317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.741938+0000","last_change":"2024-09-21T05:55:13.999001+0000","last_active":"2024-09-21T05:57:02.741938+0000","last_peered":"2024-09-21T05:57:02.741938+0000","last_clean":"2024-09-21T05:57:02.741938+0000","last_became_active":"2024-09-21T05:55:13.998656+0000","last_became_peered":"2024-09-21T05:55:13.998656+0000","last_unstale":"2024-09-21T05:57:02.741938+0000","last_undegraded":"2024-09-21T05:57:02.741938+0000","last_fullsized":"2024-09-21T05:57:02.741938+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:50.913635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"74'4","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.646791+0000","last_change":"2024-09-21T05:55:33.646124+0000","last_active":"2024-09-21T05:55:33.646791+0000","last_peered":"2024-09-21T05:55:33.646791+0000","last_clean":"2024-09-21T05:55:33.646791+0000","last_became_active":"2024-09-21T05:55:33.641142+0000","last_became_peered":"2024-09-21T05:55:33.641142+0000","last_unstale":"2024-09-21T05:55:33.646791+0000","last_undegraded":"2024-09-21T05:55:33.646791+0000","last_fullsized":"2024-09-21T05:55:33.646791+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:36.341473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.767041+0000","last_change":"2024-09-21T05:55:13.997494+0000","last_active":"2024-09-21T05:57:02.767041+0000","last_peered":"2024-09-21T05:57:02.767041+0000","last_clean":"2024-09-21T05:57:02.767041+0000","last_became_active":"2024-09-21T05:55:13.997138+0000","last_became_peered":"2024-09-21T05:55:13.997138+0000","last_unstale":"2024-09-21T05:57:02.767041+0000","last_undegraded":"2024-09-21T05:57:02.767041+0000","last_fullsized":"2024-09-21T05:57:02.767041+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:02:01.777126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"76'15","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.747951+0000","last_change":"2024-09-21T05:55:14.002750+0000","last_active":"2024-09-21T05:57:02.747951+0000","last_peered":"2024-09-21T05:57:02.747951+0000","last_clean":"2024-09-21T05:57:02.747951+0000","last_became_active":"2024-09-21T05:55:14.002150+0000","last_became_peered":"2024-09-21T05:55:14.002150+0000","last_unstale":"2024-09-21T05:57:02.747951+0000","last_undegraded":"2024-09-21T05:57:02.747951+0000","last_fullsized":"2024-09-21T05:57:02.747951+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:49.603133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"74'5","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636032+0000","last_change":"2024-09-21T05:55:11.982366+0000","last_active":"2024-09-21T05:55:33.636032+0000","last_peered":"2024-09-21T05:55:33.636032+0000","last_clean":"2024-09-21T05:55:33.636032+0000","last_became_active":"2024-09-21T05:55:11.981994+0000","last_became_peered":"2024-09-21T05:55:11.981994+0000","last_unstale":"2024-09-21T05:55:33.636032+0000","last_undegraded":"2024-09-21T05:55:33.636032+0000","last_fullsized":"2024-09-21T05:55:33.636032+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:14.626758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.769310+0000","last_change":"2024-09-21T05:55:14.008204+0000","last_active":"2024-09-21T05:57:02.769310+0000","last_peered":"2024-09-21T05:57:02.769310+0000","last_clean":"2024-09-21T05:57:02.769310+0000","last_became_active":"2024-09-21T05:55:14.007800+0000","last_became_peered":"2024-09-21T05:55:14.007800+0000","last_unstale":"2024-09-21T05:57:02.769310+0000","last_undegraded":"2024-09-21T05:57:02.769310+0000","last_fullsized":"2024-09-21T05:57:02.769310+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:49.760850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"74'12","reported_seq":36,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636283+0000","last_change":"2024-09-21T05:55:11.986168+0000","last_active":"2024-09-21T05:55:33.636283+0000","last_peered":"2024-09-21T05:55:33.636283+0000","last_clean":"2024-09-21T05:55:33.636283+0000","last_became_active":"2024-09-21T05:55:11.985694+0000","last_became_peered":"2024-09-21T05:55:11.985694+0000","last_unstale":"2024-09-21T05:55:33.636283+0000","last_undegraded":"2024-09-21T05:55:33.636283+0000","last_fullsized":"2024-09-21T05:55:33.636283+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:44.863935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.748453+0000","last_change":"2024-09-21T05:55:14.002459+0000","last_active":"2024-09-21T05:57:02.748453+0000","last_peered":"2024-09-21T05:57:02.748453+0000","last_clean":"2024-09-21T05:57:02.748453+0000","last_became_active":"2024-09-21T05:55:14.001962+0000","last_became_peered":"2024-09-21T05:55:14.001962+0000","last_unstale":"2024-09-21T05:57:02.748453+0000","last_undegraded":"2024-09-21T05:57:02.748453+0000","last_fullsized":"2024-09-21T05:57:02.748453+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:53:26.892772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639340+0000","last_change":"2024-09-21T05:55:12.014149+0000","last_active":"2024-09-21T05:55:33.639340+0000","last_peered":"2024-09-21T05:55:33.639340+0000","last_clean":"2024-09-21T05:55:33.639340+0000","last_became_active":"2024-09-21T05:55:12.013789+0000","last_became_peered":"2024-09-21T05:55:12.013789+0000","last_unstale":"2024-09-21T05:55:33.639340+0000","last_undegraded":"2024-09-21T05:55:33.639340+0000","last_fullsized":"2024-09-21T05:55:33.639340+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:18.304023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.749464+0000","last_change":"2024-09-21T05:55:13.993500+0000","last_active":"2024-09-21T05:57:02.749464+0000","last_peered":"2024-09-21T05:57:02.749464+0000","last_clean":"2024-09-21T05:57:02.749464+0000","last_became_active":"2024-09-21T05:55:13.993104+0000","last_became_peered":"2024-09-21T05:55:13.993104+0000","last_unstale":"2024-09-21T05:57:02.749464+0000","last_undegraded":"2024-09-21T05:57:02.749464+0000","last_fullsized":"2024-09-21T05:57:02.749464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:01.445119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"76'14","reported_seq":159,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:05.784296+0000","last_change":"2024-09-21T05:55:33.650207+0000","last_active":"2024-09-21T05:57:05.784296+0000","last_peered":"2024-09-21T05:57:05.784296+0000","last_clean":"2024-09-21T05:57:05.784296+0000","last_became_active":"2024-09-21T05:55:33.638388+0000","last_became_peered":"2024-09-21T05:55:33.638388+0000","last_unstale":"2024-09-21T05:57:05.784296+0000","last_undegraded":"2024-09-21T05:57:05.784296+0000","last_fullsized":"2024-09-21T05:57:05.784296+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:51.478063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72477,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1047,"num_read_kb":603,"num_write":260,"num_write_kb":154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":217739,"data_compressed":5682,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281769,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":68,"seq":292057776164,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":839680,"data_stored":412700,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939689,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":1067516,"data_compressed":43475,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103214,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":856064,"data_stored":411426,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299444,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":411804,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,0,0,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495672,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":1004323,"data_compressed":41581,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659198,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":839680,"data_stored":411143,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822722,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":1004141,"data_compressed":41581,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986248,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":839680,"data_stored":411536,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36180,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708972,"statfs":{"total":95995035648,"available":95957987328,"internally_reserved":0,"allocated":856064,"data_stored":410968,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378707,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":477799,"data_compressed":7646,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542230,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":901120,"data_stored":476823,"data_compressed":7646,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705757,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36184,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708968,"statfs":{"total":95995035648,"available":95957983232,"internally_reserved":0,"allocated":860160,"data_stored":413014,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34208,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34208,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34208,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35829,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35829,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35829,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35829,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35829,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":66519,"data_compressed":1894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":67495,"data_compressed":1894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":66727,"data_compressed":1894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:57:10.412 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-21T05:57:10.412 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:57:10.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.556+0000 7f5411bda640 1 -- 172.21.15.117:0/2785405842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f540c1061f0 msgr2=0x7f540c106600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:10.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.556+0000 7f5411bda640 1 --2- 172.21.15.117:0/2785405842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f540c1061f0 0x7f540c106600 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5400009900 tx=0x7f540002a410 comp rx=0 tx=0).stop 2024-09-21T05:57:10.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.556+0000 7f5411bda640 1 -- 172.21.15.117:0/2785405842 shutdown_connections 2024-09-21T05:57:10.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.556+0000 7f5411bda640 1 --2- 172.21.15.117:0/2785405842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f540c1061f0 0x7f540c106600 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.556+0000 7f5411bda640 1 --2- 172.21.15.117:0/2785405842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f540c106b40 0x7f540c110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.556+0000 7f5411bda640 1 --2- 172.21.15.117:0/2785405842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f540c102010 0x7f540c102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.556+0000 7f5411bda640 1 -- 172.21.15.117:0/2785405842 >> 172.21.15.117:0/2785405842 conn(0x7f540c079d00 msgr2=0x7f540c07a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:10.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.556+0000 7f5411bda640 1 -- 172.21.15.117:0/2785405842 shutdown_connections 2024-09-21T05:57:10.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.556+0000 7f5411bda640 1 -- 172.21.15.117:0/2785405842 wait complete. 2024-09-21T05:57:10.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.557+0000 7f5411bda640 1 Processor -- start 2024-09-21T05:57:10.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.557+0000 7f5411bda640 1 -- start start 2024-09-21T05:57:10.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.557+0000 7f5411bda640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f540c102010 0x7f540c19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:10.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.557+0000 7f5411bda640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f540c1061f0 0x7f540c19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:10.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.557+0000 7f5411bda640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f540c106b40 0x7f540c19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:10.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.557+0000 7f5411bda640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f540c113a60 con 0x7f540c106b40 2024-09-21T05:57:10.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.557+0000 7f5411bda640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1138e0 con 0x7f540c1061f0 2024-09-21T05:57:10.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.557+0000 7f5411bda640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f540c113be0 con 0x7f540c102010 2024-09-21T05:57:10.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.557+0000 7f540bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f540c106b40 0x7f540c19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:10.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.557+0000 7f540bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f540c106b40 0x7f540c19fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60382/0 (socket says 172.21.15.117:60382) 2024-09-21T05:57:10.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.557+0000 7f540bfff640 1 -- 172.21.15.117:0/3004545577 learned_addr learned my addr 172.21.15.117:0/3004545577 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:10.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.557+0000 7f540b7fe640 1 --2- 172.21.15.117:0/3004545577 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f540c102010 0x7f540c19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:10.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.557+0000 7f540affd640 1 --2- 172.21.15.117:0/3004545577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f540c1061f0 0x7f540c19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:10.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.558+0000 7f540bfff640 1 -- 172.21.15.117:0/3004545577 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f540c102010 msgr2=0x7f540c19a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:10.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.558+0000 7f540bfff640 1 --2- 172.21.15.117:0/3004545577 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f540c102010 0x7f540c19a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.558+0000 7f540bfff640 1 -- 172.21.15.117:0/3004545577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f540c1061f0 msgr2=0x7f540c19aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:10.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.558+0000 7f540bfff640 1 --2- 172.21.15.117:0/3004545577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f540c1061f0 0x7f540c19aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.558+0000 7f540bfff640 1 -- 172.21.15.117:0/3004545577 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540c1a03b0 con 0x7f540c106b40 2024-09-21T05:57:10.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.558+0000 7f540affd640 1 --2- 172.21.15.117:0/3004545577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f540c1061f0 0x7f540c19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:57:10.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.558+0000 7f540bfff640 1 --2- 172.21.15.117:0/3004545577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f540c106b40 0x7f540c19fcb0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5404009fd0 tx=0x7f540400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:10.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.558+0000 7f5408ff9640 1 -- 172.21.15.117:0/3004545577 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5404017070 con 0x7f540c106b40 2024-09-21T05:57:10.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.558+0000 7f5408ff9640 1 -- 172.21.15.117:0/3004545577 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5404003e20 con 0x7f540c106b40 2024-09-21T05:57:10.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.558+0000 7f5411bda640 1 -- 172.21.15.117:0/3004545577 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f540c1a06a0 con 0x7f540c106b40 2024-09-21T05:57:10.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.558+0000 7f5408ff9640 1 -- 172.21.15.117:0/3004545577 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5404010460 con 0x7f540c106b40 2024-09-21T05:57:10.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.558+0000 7f5411bda640 1 -- 172.21.15.117:0/3004545577 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f540c1a6e30 con 0x7f540c106b40 2024-09-21T05:57:10.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.559+0000 7f5411bda640 1 -- 172.21.15.117:0/3004545577 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f540c102420 con 0x7f540c106b40 2024-09-21T05:57:10.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.561+0000 7f5408ff9640 1 -- 172.21.15.117:0/3004545577 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 22) ==== 154530+0+0 (secure 0 0 0) 0x7f5404003970 con 0x7f540c106b40 2024-09-21T05:57:10.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.561+0000 7f5408ff9640 1 --2- 172.21.15.117:0/3004545577 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f53f40b6560 0x7f53f40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:10.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.561+0000 7f540b7fe640 1 --2- 172.21.15.117:0/3004545577 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f53f40b6560 0x7f53f40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:10.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.561+0000 7f5408ff9640 1 -- 172.21.15.117:0/3004545577 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f54040e7170 con 0x7f540c106b40 2024-09-21T05:57:10.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.562+0000 7f540b7fe640 1 --2- 172.21.15.117:0/3004545577 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f53f40b6560 0x7f53f40b8a70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f53fc005fd0 tx=0x7f53fc00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:10.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.564+0000 7f5408ff9640 1 -- 172.21.15.117:0/3004545577 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f54040ad960 con 0x7f540c106b40 2024-09-21T05:57:10.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.741+0000 7f5411bda640 1 -- 172.21.15.117:0/3004545577 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f540c10bc60 con 0x7f53f40b6560 2024-09-21T05:57:10.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.745+0000 7f5408ff9640 1 -- 172.21.15.117:0/3004545577 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259298 (secure 0 0 0) 0x7f540c10bc60 con 0x7f53f40b6560 2024-09-21T05:57:10.749 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:57:10.751 INFO:teuthology.orchestra.run.smithi117.stderr:dumped all 2024-09-21T05:57:10.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.748+0000 7f5411bda640 1 -- 172.21.15.117:0/3004545577 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f53f40b6560 msgr2=0x7f53f40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:10.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.748+0000 7f5411bda640 1 --2- 172.21.15.117:0/3004545577 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f53f40b6560 0x7f53f40b8a70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f53fc005fd0 tx=0x7f53fc00b390 comp rx=0 tx=0).stop 2024-09-21T05:57:10.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.748+0000 7f5411bda640 1 -- 172.21.15.117:0/3004545577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f540c106b40 msgr2=0x7f540c19fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:10.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.748+0000 7f5411bda640 1 --2- 172.21.15.117:0/3004545577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f540c106b40 0x7f540c19fcb0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5404009fd0 tx=0x7f540400b260 comp rx=0 tx=0).stop 2024-09-21T05:57:10.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.748+0000 7f5411bda640 1 -- 172.21.15.117:0/3004545577 shutdown_connections 2024-09-21T05:57:10.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.748+0000 7f5411bda640 1 --2- 172.21.15.117:0/3004545577 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f53f40b6560 0x7f53f40b8a70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.748+0000 7f5411bda640 1 --2- 172.21.15.117:0/3004545577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f540c106b40 0x7f540c19fcb0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.748+0000 7f5411bda640 1 --2- 172.21.15.117:0/3004545577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f540c1061f0 0x7f540c19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.748+0000 7f5411bda640 1 --2- 172.21.15.117:0/3004545577 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f540c102010 0x7f540c19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:10.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.748+0000 7f5411bda640 1 -- 172.21.15.117:0/3004545577 >> 172.21.15.117:0/3004545577 conn(0x7f540c079d00 msgr2=0x7f540c108470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:10.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.748+0000 7f5411bda640 1 -- 172.21.15.117:0/3004545577 shutdown_connections 2024-09-21T05:57:10.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:10.748+0000 7f5411bda640 1 -- 172.21.15.117:0/3004545577 wait complete. 2024-09-21T05:57:10.761 INFO:teuthology.orchestra.run.smithi117.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-09-21T05:57:10.535507+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663079,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1138,"num_read_kb":672,"num_write":385,"num_write_kb":2306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":2952512,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":817,"ondisk_log_size":817,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376208,"kb_used_data":10320,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565616,"statfs":{"total":1151940427776,"available":1151555190784,"internally_reserved":0,"allocated":10567680,"data_stored":6913193,"data_compressed":182193,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,0,0,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":980,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":115,"num_read_kb":55,"num_write":15,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003014"},"pg_stats":[{"pgid":"3.3f","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.767187+0000","last_change":"2024-09-21T05:55:14.007642+0000","last_active":"2024-09-21T05:57:02.767187+0000","last_peered":"2024-09-21T05:57:02.767187+0000","last_clean":"2024-09-21T05:57:02.767187+0000","last_became_active":"2024-09-21T05:55:14.006943+0000","last_became_peered":"2024-09-21T05:55:14.006943+0000","last_unstale":"2024-09-21T05:57:02.767187+0000","last_undegraded":"2024-09-21T05:57:02.767187+0000","last_fullsized":"2024-09-21T05:57:02.767187+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:23:53.816079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"76'4","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.748660+0000","last_change":"2024-09-21T05:55:14.002034+0000","last_active":"2024-09-21T05:57:02.748660+0000","last_peered":"2024-09-21T05:57:02.748660+0000","last_clean":"2024-09-21T05:57:02.748660+0000","last_became_active":"2024-09-21T05:55:14.001658+0000","last_became_peered":"2024-09-21T05:55:14.001658+0000","last_unstale":"2024-09-21T05:57:02.748660+0000","last_undegraded":"2024-09-21T05:57:02.748660+0000","last_fullsized":"2024-09-21T05:57:02.748660+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:09:01.568362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.746220+0000","last_change":"2024-09-21T05:55:13.996901+0000","last_active":"2024-09-21T05:57:02.746220+0000","last_peered":"2024-09-21T05:57:02.746220+0000","last_clean":"2024-09-21T05:57:02.746220+0000","last_became_active":"2024-09-21T05:55:13.996020+0000","last_became_peered":"2024-09-21T05:55:13.996020+0000","last_unstale":"2024-09-21T05:57:02.746220+0000","last_undegraded":"2024-09-21T05:57:02.746220+0000","last_fullsized":"2024-09-21T05:57:02.746220+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:38.771926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.750182+0000","last_change":"2024-09-21T05:55:13.995968+0000","last_active":"2024-09-21T05:57:02.750182+0000","last_peered":"2024-09-21T05:57:02.750182+0000","last_clean":"2024-09-21T05:57:02.750182+0000","last_became_active":"2024-09-21T05:55:13.995259+0000","last_became_peered":"2024-09-21T05:55:13.995259+0000","last_unstale":"2024-09-21T05:57:02.750182+0000","last_undegraded":"2024-09-21T05:57:02.750182+0000","last_fullsized":"2024-09-21T05:57:02.750182+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:48.953342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"76'4","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.726754+0000","last_change":"2024-09-21T05:55:13.995778+0000","last_active":"2024-09-21T05:57:02.726754+0000","last_peered":"2024-09-21T05:57:02.726754+0000","last_clean":"2024-09-21T05:57:02.726754+0000","last_became_active":"2024-09-21T05:55:13.995415+0000","last_became_peered":"2024-09-21T05:55:13.995415+0000","last_unstale":"2024-09-21T05:57:02.726754+0000","last_undegraded":"2024-09-21T05:57:02.726754+0000","last_fullsized":"2024-09-21T05:57:02.726754+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:52.562120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.737104+0000","last_change":"2024-09-21T05:55:14.000005+0000","last_active":"2024-09-21T05:57:02.737104+0000","last_peered":"2024-09-21T05:57:02.737104+0000","last_clean":"2024-09-21T05:57:02.737104+0000","last_became_active":"2024-09-21T05:55:13.999493+0000","last_became_peered":"2024-09-21T05:55:13.999493+0000","last_unstale":"2024-09-21T05:57:02.737104+0000","last_undegraded":"2024-09-21T05:57:02.737104+0000","last_fullsized":"2024-09-21T05:57:02.737104+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:29.426368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.756030+0000","last_change":"2024-09-21T05:55:13.993004+0000","last_active":"2024-09-21T05:57:02.756030+0000","last_peered":"2024-09-21T05:57:02.756030+0000","last_clean":"2024-09-21T05:57:02.756030+0000","last_became_active":"2024-09-21T05:55:13.992615+0000","last_became_peered":"2024-09-21T05:55:13.992615+0000","last_unstale":"2024-09-21T05:57:02.756030+0000","last_undegraded":"2024-09-21T05:57:02.756030+0000","last_fullsized":"2024-09-21T05:57:02.756030+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:48.324618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"76'9","reported_seq":21,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.745963+0000","last_change":"2024-09-21T05:55:33.769006+0000","last_active":"2024-09-21T05:57:02.745963+0000","last_peered":"2024-09-21T05:57:02.745963+0000","last_clean":"2024-09-21T05:57:02.745963+0000","last_became_active":"2024-09-21T05:55:33.768705+0000","last_became_peered":"2024-09-21T05:55:33.768705+0000","last_unstale":"2024-09-21T05:57:02.745963+0000","last_undegraded":"2024-09-21T05:57:02.745963+0000","last_fullsized":"2024-09-21T05:57:02.745963+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:14.013300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.765471+0000","last_change":"2024-09-21T05:55:14.004131+0000","last_active":"2024-09-21T05:57:02.765471+0000","last_peered":"2024-09-21T05:57:02.765471+0000","last_clean":"2024-09-21T05:57:02.765471+0000","last_became_active":"2024-09-21T05:55:14.003621+0000","last_became_peered":"2024-09-21T05:55:14.003621+0000","last_unstale":"2024-09-21T05:57:02.765471+0000","last_undegraded":"2024-09-21T05:57:02.765471+0000","last_fullsized":"2024-09-21T05:57:02.765471+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:00.694211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"76'16","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.750873+0000","last_change":"2024-09-21T05:55:33.643289+0000","last_active":"2024-09-21T05:57:02.750873+0000","last_peered":"2024-09-21T05:57:02.750873+0000","last_clean":"2024-09-21T05:57:02.750873+0000","last_became_active":"2024-09-21T05:55:33.642805+0000","last_became_peered":"2024-09-21T05:55:33.642805+0000","last_unstale":"2024-09-21T05:57:02.750873+0000","last_undegraded":"2024-09-21T05:57:02.750873+0000","last_fullsized":"2024-09-21T05:57:02.750873+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:13:00.854712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.766867+0000","last_change":"2024-09-21T05:55:14.006467+0000","last_active":"2024-09-21T05:57:02.766867+0000","last_peered":"2024-09-21T05:57:02.766867+0000","last_clean":"2024-09-21T05:57:02.766867+0000","last_became_active":"2024-09-21T05:55:14.005978+0000","last_became_peered":"2024-09-21T05:55:14.005978+0000","last_unstale":"2024-09-21T05:57:02.766867+0000","last_undegraded":"2024-09-21T05:57:02.766867+0000","last_fullsized":"2024-09-21T05:57:02.766867+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:37.391152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.747861+0000","last_change":"2024-09-21T05:55:14.003972+0000","last_active":"2024-09-21T05:57:02.747861+0000","last_peered":"2024-09-21T05:57:02.747861+0000","last_clean":"2024-09-21T05:57:02.747861+0000","last_became_active":"2024-09-21T05:55:14.003601+0000","last_became_peered":"2024-09-21T05:55:14.003601+0000","last_unstale":"2024-09-21T05:57:02.747861+0000","last_undegraded":"2024-09-21T05:57:02.747861+0000","last_fullsized":"2024-09-21T05:57:02.747861+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:28:16.201087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.751561+0000","last_change":"2024-09-21T05:55:13.996051+0000","last_active":"2024-09-21T05:57:02.751561+0000","last_peered":"2024-09-21T05:57:02.751561+0000","last_clean":"2024-09-21T05:57:02.751561+0000","last_became_active":"2024-09-21T05:55:13.995679+0000","last_became_peered":"2024-09-21T05:55:13.995679+0000","last_unstale":"2024-09-21T05:57:02.751561+0000","last_undegraded":"2024-09-21T05:57:02.751561+0000","last_fullsized":"2024-09-21T05:57:02.751561+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:45:44.372029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.745041+0000","last_change":"2024-09-21T05:55:13.995421+0000","last_active":"2024-09-21T05:57:02.745041+0000","last_peered":"2024-09-21T05:57:02.745041+0000","last_clean":"2024-09-21T05:57:02.745041+0000","last_became_active":"2024-09-21T05:55:13.995006+0000","last_became_peered":"2024-09-21T05:55:13.995006+0000","last_unstale":"2024-09-21T05:57:02.745041+0000","last_undegraded":"2024-09-21T05:57:02.745041+0000","last_fullsized":"2024-09-21T05:57:02.745041+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:00.636555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.745532+0000","last_change":"2024-09-21T05:55:13.993466+0000","last_active":"2024-09-21T05:57:02.745532+0000","last_peered":"2024-09-21T05:57:02.745532+0000","last_clean":"2024-09-21T05:57:02.745532+0000","last_became_active":"2024-09-21T05:55:13.993137+0000","last_became_peered":"2024-09-21T05:55:13.993137+0000","last_unstale":"2024-09-21T05:57:02.745532+0000","last_undegraded":"2024-09-21T05:57:02.745532+0000","last_fullsized":"2024-09-21T05:57:02.745532+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:33.218868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.765648+0000","last_change":"2024-09-21T05:55:13.996316+0000","last_active":"2024-09-21T05:57:02.765648+0000","last_peered":"2024-09-21T05:57:02.765648+0000","last_clean":"2024-09-21T05:57:02.765648+0000","last_became_active":"2024-09-21T05:55:13.995918+0000","last_became_peered":"2024-09-21T05:55:13.995918+0000","last_unstale":"2024-09-21T05:57:02.765648+0000","last_undegraded":"2024-09-21T05:57:02.765648+0000","last_fullsized":"2024-09-21T05:57:02.765648+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:45:54.408504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.767326+0000","last_change":"2024-09-21T05:55:14.007198+0000","last_active":"2024-09-21T05:57:02.767326+0000","last_peered":"2024-09-21T05:57:02.767326+0000","last_clean":"2024-09-21T05:57:02.767326+0000","last_became_active":"2024-09-21T05:55:14.006700+0000","last_became_peered":"2024-09-21T05:55:14.006700+0000","last_unstale":"2024-09-21T05:57:02.767326+0000","last_undegraded":"2024-09-21T05:57:02.767326+0000","last_fullsized":"2024-09-21T05:57:02.767326+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:45.343778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.765593+0000","last_change":"2024-09-21T05:55:14.000152+0000","last_active":"2024-09-21T05:57:02.765593+0000","last_peered":"2024-09-21T05:57:02.765593+0000","last_clean":"2024-09-21T05:57:02.765593+0000","last_became_active":"2024-09-21T05:55:13.999786+0000","last_became_peered":"2024-09-21T05:55:13.999786+0000","last_unstale":"2024-09-21T05:57:02.765593+0000","last_undegraded":"2024-09-21T05:57:02.765593+0000","last_fullsized":"2024-09-21T05:57:02.765593+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:43:25.713410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.749050+0000","last_change":"2024-09-21T05:55:13.988367+0000","last_active":"2024-09-21T05:57:02.749050+0000","last_peered":"2024-09-21T05:57:02.749050+0000","last_clean":"2024-09-21T05:57:02.749050+0000","last_became_active":"2024-09-21T05:55:13.987982+0000","last_became_peered":"2024-09-21T05:55:13.987982+0000","last_unstale":"2024-09-21T05:57:02.749050+0000","last_undegraded":"2024-09-21T05:57:02.749050+0000","last_fullsized":"2024-09-21T05:57:02.749050+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:15.395155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.748991+0000","last_change":"2024-09-21T05:55:14.004962+0000","last_active":"2024-09-21T05:57:02.748991+0000","last_peered":"2024-09-21T05:57:02.748991+0000","last_clean":"2024-09-21T05:57:02.748991+0000","last_became_active":"2024-09-21T05:55:14.004573+0000","last_became_peered":"2024-09-21T05:55:14.004573+0000","last_unstale":"2024-09-21T05:57:02.748991+0000","last_undegraded":"2024-09-21T05:57:02.748991+0000","last_fullsized":"2024-09-21T05:57:02.748991+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:59.014017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.756283+0000","last_change":"2024-09-21T05:55:13.991267+0000","last_active":"2024-09-21T05:57:02.756283+0000","last_peered":"2024-09-21T05:57:02.756283+0000","last_clean":"2024-09-21T05:57:02.756283+0000","last_became_active":"2024-09-21T05:55:13.990916+0000","last_became_peered":"2024-09-21T05:55:13.990916+0000","last_unstale":"2024-09-21T05:57:02.756283+0000","last_undegraded":"2024-09-21T05:57:02.756283+0000","last_fullsized":"2024-09-21T05:57:02.756283+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:06:03.904632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.766270+0000","last_change":"2024-09-21T05:55:13.994147+0000","last_active":"2024-09-21T05:57:02.766270+0000","last_peered":"2024-09-21T05:57:02.766270+0000","last_clean":"2024-09-21T05:57:02.766270+0000","last_became_active":"2024-09-21T05:55:13.993794+0000","last_became_peered":"2024-09-21T05:55:13.993794+0000","last_unstale":"2024-09-21T05:57:02.766270+0000","last_undegraded":"2024-09-21T05:57:02.766270+0000","last_fullsized":"2024-09-21T05:57:02.766270+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:19.970853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.750316+0000","last_change":"2024-09-21T05:55:14.001119+0000","last_active":"2024-09-21T05:57:02.750316+0000","last_peered":"2024-09-21T05:57:02.750316+0000","last_clean":"2024-09-21T05:57:02.750316+0000","last_became_active":"2024-09-21T05:55:14.000799+0000","last_became_peered":"2024-09-21T05:55:14.000799+0000","last_unstale":"2024-09-21T05:57:02.750316+0000","last_undegraded":"2024-09-21T05:57:02.750316+0000","last_fullsized":"2024-09-21T05:57:02.750316+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:28.259188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.739740+0000","last_change":"2024-09-21T05:55:13.992043+0000","last_active":"2024-09-21T05:57:02.739740+0000","last_peered":"2024-09-21T05:57:02.739740+0000","last_clean":"2024-09-21T05:57:02.739740+0000","last_became_active":"2024-09-21T05:55:13.991344+0000","last_became_peered":"2024-09-21T05:55:13.991344+0000","last_unstale":"2024-09-21T05:57:02.739740+0000","last_undegraded":"2024-09-21T05:57:02.739740+0000","last_fullsized":"2024-09-21T05:57:02.739740+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:42:55.674333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"76'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.749506+0000","last_change":"2024-09-21T05:55:13.993114+0000","last_active":"2024-09-21T05:57:02.749506+0000","last_peered":"2024-09-21T05:57:02.749506+0000","last_clean":"2024-09-21T05:57:02.749506+0000","last_became_active":"2024-09-21T05:55:13.992749+0000","last_became_peered":"2024-09-21T05:55:13.992749+0000","last_unstale":"2024-09-21T05:57:02.749506+0000","last_undegraded":"2024-09-21T05:57:02.749506+0000","last_fullsized":"2024-09-21T05:57:02.749506+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:15.926967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"76'15","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.763355+0000","last_change":"2024-09-21T05:55:13.997199+0000","last_active":"2024-09-21T05:57:02.763355+0000","last_peered":"2024-09-21T05:57:02.763355+0000","last_clean":"2024-09-21T05:57:02.763355+0000","last_became_active":"2024-09-21T05:55:13.996809+0000","last_became_peered":"2024-09-21T05:55:13.996809+0000","last_unstale":"2024-09-21T05:57:02.763355+0000","last_undegraded":"2024-09-21T05:57:02.763355+0000","last_fullsized":"2024-09-21T05:57:02.763355+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:16.681430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.751124+0000","last_change":"2024-09-21T05:55:14.000879+0000","last_active":"2024-09-21T05:57:02.751124+0000","last_peered":"2024-09-21T05:57:02.751124+0000","last_clean":"2024-09-21T05:57:02.751124+0000","last_became_active":"2024-09-21T05:55:14.000259+0000","last_became_peered":"2024-09-21T05:55:14.000259+0000","last_unstale":"2024-09-21T05:57:02.751124+0000","last_undegraded":"2024-09-21T05:57:02.751124+0000","last_fullsized":"2024-09-21T05:57:02.751124+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:01:08.938417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"76'4","reported_seq":22,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.744616+0000","last_change":"2024-09-21T05:55:13.989433+0000","last_active":"2024-09-21T05:57:02.744616+0000","last_peered":"2024-09-21T05:57:02.744616+0000","last_clean":"2024-09-21T05:57:02.744616+0000","last_became_active":"2024-09-21T05:55:13.989097+0000","last_became_peered":"2024-09-21T05:55:13.989097+0000","last_unstale":"2024-09-21T05:57:02.744616+0000","last_undegraded":"2024-09-21T05:57:02.744616+0000","last_fullsized":"2024-09-21T05:57:02.744616+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:37.835298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.742763+0000","last_change":"2024-09-21T05:55:13.993911+0000","last_active":"2024-09-21T05:57:02.742763+0000","last_peered":"2024-09-21T05:57:02.742763+0000","last_clean":"2024-09-21T05:57:02.742763+0000","last_became_active":"2024-09-21T05:55:13.993570+0000","last_became_peered":"2024-09-21T05:55:13.993570+0000","last_unstale":"2024-09-21T05:57:02.742763+0000","last_undegraded":"2024-09-21T05:57:02.742763+0000","last_fullsized":"2024-09-21T05:57:02.742763+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:08:16.588457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.751759+0000","last_change":"2024-09-21T05:55:13.992764+0000","last_active":"2024-09-21T05:57:02.751759+0000","last_peered":"2024-09-21T05:57:02.751759+0000","last_clean":"2024-09-21T05:57:02.751759+0000","last_became_active":"2024-09-21T05:55:13.992338+0000","last_became_peered":"2024-09-21T05:55:13.992338+0000","last_unstale":"2024-09-21T05:57:02.751759+0000","last_undegraded":"2024-09-21T05:57:02.751759+0000","last_fullsized":"2024-09-21T05:57:02.751759+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:25:29.997353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"76'12","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.752225+0000","last_change":"2024-09-21T05:55:13.987119+0000","last_active":"2024-09-21T05:57:02.752225+0000","last_peered":"2024-09-21T05:57:02.752225+0000","last_clean":"2024-09-21T05:57:02.752225+0000","last_became_active":"2024-09-21T05:55:13.986663+0000","last_became_peered":"2024-09-21T05:55:13.986663+0000","last_unstale":"2024-09-21T05:57:02.752225+0000","last_undegraded":"2024-09-21T05:57:02.752225+0000","last_fullsized":"2024-09-21T05:57:02.752225+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:02.089833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.751424+0000","last_change":"2024-09-21T05:55:14.003577+0000","last_active":"2024-09-21T05:57:02.751424+0000","last_peered":"2024-09-21T05:57:02.751424+0000","last_clean":"2024-09-21T05:57:02.751424+0000","last_became_active":"2024-09-21T05:55:14.003162+0000","last_became_peered":"2024-09-21T05:55:14.003162+0000","last_unstale":"2024-09-21T05:57:02.751424+0000","last_undegraded":"2024-09-21T05:57:02.751424+0000","last_fullsized":"2024-09-21T05:57:02.751424+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:58.630843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.730878+0000","last_change":"2024-09-21T05:55:13.992078+0000","last_active":"2024-09-21T05:57:02.730878+0000","last_peered":"2024-09-21T05:57:02.730878+0000","last_clean":"2024-09-21T05:57:02.730878+0000","last_became_active":"2024-09-21T05:55:13.991680+0000","last_became_peered":"2024-09-21T05:55:13.991680+0000","last_unstale":"2024-09-21T05:57:02.730878+0000","last_undegraded":"2024-09-21T05:57:02.730878+0000","last_fullsized":"2024-09-21T05:57:02.730878+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:52.886755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"76'7","reported_seq":138,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:08.381811+0000","last_change":"2024-09-21T05:55:11.982560+0000","last_active":"2024-09-21T05:57:08.381811+0000","last_peered":"2024-09-21T05:57:08.381811+0000","last_clean":"2024-09-21T05:57:08.381811+0000","last_became_active":"2024-09-21T05:55:11.982179+0000","last_became_peered":"2024-09-21T05:55:11.982179+0000","last_unstale":"2024-09-21T05:57:08.381811+0000","last_undegraded":"2024-09-21T05:57:08.381811+0000","last_fullsized":"2024-09-21T05:57:08.381811+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:06.012194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.740537+0000","last_change":"2024-09-21T05:55:13.993994+0000","last_active":"2024-09-21T05:57:02.740537+0000","last_peered":"2024-09-21T05:57:02.740537+0000","last_clean":"2024-09-21T05:57:02.740537+0000","last_became_active":"2024-09-21T05:55:13.993585+0000","last_became_peered":"2024-09-21T05:55:13.993585+0000","last_unstale":"2024-09-21T05:57:02.740537+0000","last_undegraded":"2024-09-21T05:57:02.740537+0000","last_fullsized":"2024-09-21T05:57:02.740537+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:11.608616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"74'7","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.634233+0000","last_change":"2024-09-21T05:55:11.984397+0000","last_active":"2024-09-21T05:55:33.634233+0000","last_peered":"2024-09-21T05:55:33.634233+0000","last_clean":"2024-09-21T05:55:33.634233+0000","last_became_active":"2024-09-21T05:55:11.983888+0000","last_became_peered":"2024-09-21T05:55:11.983888+0000","last_unstale":"2024-09-21T05:55:33.634233+0000","last_undegraded":"2024-09-21T05:55:33.634233+0000","last_fullsized":"2024-09-21T05:55:33.634233+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:59.012710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636187+0000","last_change":"2024-09-21T05:55:11.974547+0000","last_active":"2024-09-21T05:55:33.636187+0000","last_peered":"2024-09-21T05:55:33.636187+0000","last_clean":"2024-09-21T05:55:33.636187+0000","last_became_active":"2024-09-21T05:55:11.974151+0000","last_became_peered":"2024-09-21T05:55:11.974151+0000","last_unstale":"2024-09-21T05:55:33.636187+0000","last_undegraded":"2024-09-21T05:55:33.636187+0000","last_fullsized":"2024-09-21T05:55:33.636187+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:51.489548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.746384+0000","last_change":"2024-09-21T05:55:13.999287+0000","last_active":"2024-09-21T05:57:02.746384+0000","last_peered":"2024-09-21T05:57:02.746384+0000","last_clean":"2024-09-21T05:57:02.746384+0000","last_became_active":"2024-09-21T05:55:13.998742+0000","last_became_peered":"2024-09-21T05:55:13.998742+0000","last_unstale":"2024-09-21T05:57:02.746384+0000","last_undegraded":"2024-09-21T05:57:02.746384+0000","last_fullsized":"2024-09-21T05:57:02.746384+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:33.842810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"74'3","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.768113+0000","last_change":"2024-09-21T05:55:11.981354+0000","last_active":"2024-09-21T05:55:33.768113+0000","last_peered":"2024-09-21T05:55:33.768113+0000","last_clean":"2024-09-21T05:55:33.768113+0000","last_became_active":"2024-09-21T05:55:11.980993+0000","last_became_peered":"2024-09-21T05:55:11.980993+0000","last_unstale":"2024-09-21T05:55:33.768113+0000","last_undegraded":"2024-09-21T05:55:33.768113+0000","last_fullsized":"2024-09-21T05:55:33.768113+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:05.037669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.746795+0000","last_change":"2024-09-21T05:55:14.001041+0000","last_active":"2024-09-21T05:57:02.746795+0000","last_peered":"2024-09-21T05:57:02.746795+0000","last_clean":"2024-09-21T05:57:02.746795+0000","last_became_active":"2024-09-21T05:55:14.000688+0000","last_became_peered":"2024-09-21T05:55:14.000688+0000","last_unstale":"2024-09-21T05:57:02.746795+0000","last_undegraded":"2024-09-21T05:57:02.746795+0000","last_fullsized":"2024-09-21T05:57:02.746795+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:51.190495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"76'6","reported_seq":233,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:07.379709+0000","last_change":"2024-09-21T05:55:11.984097+0000","last_active":"2024-09-21T05:57:07.379709+0000","last_peered":"2024-09-21T05:57:07.379709+0000","last_clean":"2024-09-21T05:57:07.379709+0000","last_became_active":"2024-09-21T05:55:11.982969+0000","last_became_peered":"2024-09-21T05:55:11.982969+0000","last_unstale":"2024-09-21T05:57:07.379709+0000","last_undegraded":"2024-09-21T05:57:07.379709+0000","last_fullsized":"2024-09-21T05:57:07.379709+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:41.372225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":204,"num_read_kb":204,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"76'10","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.768826+0000","last_change":"2024-09-21T05:55:13.998433+0000","last_active":"2024-09-21T05:57:02.768826+0000","last_peered":"2024-09-21T05:57:02.768826+0000","last_clean":"2024-09-21T05:57:02.768826+0000","last_became_active":"2024-09-21T05:55:13.997890+0000","last_became_peered":"2024-09-21T05:55:13.997890+0000","last_unstale":"2024-09-21T05:57:02.768826+0000","last_undegraded":"2024-09-21T05:57:02.768826+0000","last_fullsized":"2024-09-21T05:57:02.768826+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:24.396789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"74'4","reported_seq":40,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.701709+0000","last_change":"2024-09-21T05:55:33.701599+0000","last_active":"2024-09-21T05:55:33.701709+0000","last_peered":"2024-09-21T05:55:33.701709+0000","last_clean":"2024-09-21T05:55:33.701709+0000","last_became_active":"2024-09-21T05:55:33.697385+0000","last_became_peered":"2024-09-21T05:55:33.697385+0000","last_unstale":"2024-09-21T05:55:33.701709+0000","last_undegraded":"2024-09-21T05:55:33.701709+0000","last_fullsized":"2024-09-21T05:55:33.701709+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:18.653991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.741741+0000","last_change":"2024-09-21T05:55:13.995111+0000","last_active":"2024-09-21T05:57:02.741741+0000","last_peered":"2024-09-21T05:57:02.741741+0000","last_clean":"2024-09-21T05:57:02.741741+0000","last_became_active":"2024-09-21T05:55:13.994691+0000","last_became_peered":"2024-09-21T05:55:13.994691+0000","last_unstale":"2024-09-21T05:57:02.741741+0000","last_undegraded":"2024-09-21T05:57:02.741741+0000","last_fullsized":"2024-09-21T05:57:02.741741+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:33:30.167649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"74'5","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767919+0000","last_change":"2024-09-21T05:55:12.013961+0000","last_active":"2024-09-21T05:55:33.767919+0000","last_peered":"2024-09-21T05:55:33.767919+0000","last_clean":"2024-09-21T05:55:33.767919+0000","last_became_active":"2024-09-21T05:55:12.013573+0000","last_became_peered":"2024-09-21T05:55:12.013573+0000","last_unstale":"2024-09-21T05:55:33.767919+0000","last_undegraded":"2024-09-21T05:55:33.767919+0000","last_fullsized":"2024-09-21T05:55:33.767919+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:12.119605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"76'11","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.768347+0000","last_change":"2024-09-21T05:55:13.996093+0000","last_active":"2024-09-21T05:57:02.768347+0000","last_peered":"2024-09-21T05:57:02.768347+0000","last_clean":"2024-09-21T05:57:02.768347+0000","last_became_active":"2024-09-21T05:55:13.995342+0000","last_became_peered":"2024-09-21T05:55:13.995342+0000","last_unstale":"2024-09-21T05:57:02.768347+0000","last_undegraded":"2024-09-21T05:57:02.768347+0000","last_fullsized":"2024-09-21T05:57:02.768347+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:20.335330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"76'34","reported_seq":272,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:07.380341+0000","last_change":"2024-09-21T05:55:11.986193+0000","last_active":"2024-09-21T05:57:07.380341+0000","last_peered":"2024-09-21T05:57:07.380341+0000","last_clean":"2024-09-21T05:57:07.380341+0000","last_became_active":"2024-09-21T05:55:11.985736+0000","last_became_peered":"2024-09-21T05:55:11.985736+0000","last_unstale":"2024-09-21T05:57:07.380341+0000","last_undegraded":"2024-09-21T05:57:07.380341+0000","last_fullsized":"2024-09-21T05:57:07.380341+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:31.503419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64459,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":215,"num_read_kb":170,"num_write":36,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.756535+0000","last_change":"2024-09-21T05:55:14.001123+0000","last_active":"2024-09-21T05:57:02.756535+0000","last_peered":"2024-09-21T05:57:02.756535+0000","last_clean":"2024-09-21T05:57:02.756535+0000","last_became_active":"2024-09-21T05:55:14.000742+0000","last_became_peered":"2024-09-21T05:55:14.000742+0000","last_unstale":"2024-09-21T05:57:02.756535+0000","last_undegraded":"2024-09-21T05:57:02.756535+0000","last_fullsized":"2024-09-21T05:57:02.756535+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:15.277992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"74'9","reported_seq":34,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639957+0000","last_change":"2024-09-21T05:55:11.981396+0000","last_active":"2024-09-21T05:55:33.639957+0000","last_peered":"2024-09-21T05:55:33.639957+0000","last_clean":"2024-09-21T05:55:33.639957+0000","last_became_active":"2024-09-21T05:55:11.980976+0000","last_became_peered":"2024-09-21T05:55:11.980976+0000","last_unstale":"2024-09-21T05:55:33.639957+0000","last_undegraded":"2024-09-21T05:55:33.639957+0000","last_fullsized":"2024-09-21T05:55:33.639957+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:43.600453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.755984+0000","last_change":"2024-09-21T05:55:13.991354+0000","last_active":"2024-09-21T05:57:02.755984+0000","last_peered":"2024-09-21T05:57:02.755984+0000","last_clean":"2024-09-21T05:57:02.755984+0000","last_became_active":"2024-09-21T05:55:13.990967+0000","last_became_peered":"2024-09-21T05:55:13.990967+0000","last_unstale":"2024-09-21T05:57:02.755984+0000","last_undegraded":"2024-09-21T05:57:02.755984+0000","last_fullsized":"2024-09-21T05:57:02.755984+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:04.889461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"74'5","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767638+0000","last_change":"2024-09-21T05:55:11.981019+0000","last_active":"2024-09-21T05:55:33.767638+0000","last_peered":"2024-09-21T05:55:33.767638+0000","last_clean":"2024-09-21T05:55:33.767638+0000","last_became_active":"2024-09-21T05:55:11.980619+0000","last_became_peered":"2024-09-21T05:55:11.980619+0000","last_unstale":"2024-09-21T05:55:33.767638+0000","last_undegraded":"2024-09-21T05:55:33.767638+0000","last_fullsized":"2024-09-21T05:55:33.767638+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:41.005092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"76'15","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.756751+0000","last_change":"2024-09-21T05:55:13.994535+0000","last_active":"2024-09-21T05:57:02.756751+0000","last_peered":"2024-09-21T05:57:02.756751+0000","last_clean":"2024-09-21T05:57:02.756751+0000","last_became_active":"2024-09-21T05:55:13.994153+0000","last_became_peered":"2024-09-21T05:55:13.994153+0000","last_unstale":"2024-09-21T05:57:02.756751+0000","last_undegraded":"2024-09-21T05:57:02.756751+0000","last_fullsized":"2024-09-21T05:57:02.756751+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:02.301263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"76'7","reported_seq":138,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:08.392792+0000","last_change":"2024-09-21T05:55:11.977694+0000","last_active":"2024-09-21T05:57:08.392792+0000","last_peered":"2024-09-21T05:57:08.392792+0000","last_clean":"2024-09-21T05:57:08.392792+0000","last_became_active":"2024-09-21T05:55:11.977146+0000","last_became_peered":"2024-09-21T05:55:11.977146+0000","last_unstale":"2024-09-21T05:57:08.392792+0000","last_undegraded":"2024-09-21T05:57:08.392792+0000","last_fullsized":"2024-09-21T05:57:08.392792+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:29.736564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":109,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.755975+0000","last_change":"2024-09-21T05:55:13.998032+0000","last_active":"2024-09-21T05:57:02.755975+0000","last_peered":"2024-09-21T05:57:02.755975+0000","last_clean":"2024-09-21T05:57:02.755975+0000","last_became_active":"2024-09-21T05:55:13.997617+0000","last_became_peered":"2024-09-21T05:55:13.997617+0000","last_unstale":"2024-09-21T05:57:02.755975+0000","last_undegraded":"2024-09-21T05:57:02.755975+0000","last_fullsized":"2024-09-21T05:57:02.755975+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:46.552726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"74'17","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635659+0000","last_change":"2024-09-21T05:55:12.013540+0000","last_active":"2024-09-21T05:55:33.635659+0000","last_peered":"2024-09-21T05:55:33.635659+0000","last_clean":"2024-09-21T05:55:33.635659+0000","last_became_active":"2024-09-21T05:55:12.013151+0000","last_became_peered":"2024-09-21T05:55:12.013151+0000","last_unstale":"2024-09-21T05:55:33.635659+0000","last_undegraded":"2024-09-21T05:55:33.635659+0000","last_fullsized":"2024-09-21T05:55:33.635659+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:53.705904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.747391+0000","last_change":"2024-09-21T05:55:13.999501+0000","last_active":"2024-09-21T05:57:02.747391+0000","last_peered":"2024-09-21T05:57:02.747391+0000","last_clean":"2024-09-21T05:57:02.747391+0000","last_became_active":"2024-09-21T05:55:13.997819+0000","last_became_peered":"2024-09-21T05:55:13.997819+0000","last_unstale":"2024-09-21T05:57:02.747391+0000","last_undegraded":"2024-09-21T05:57:02.747391+0000","last_fullsized":"2024-09-21T05:57:02.747391+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:54.334755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"74'2","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636760+0000","last_change":"2024-09-21T05:55:11.984428+0000","last_active":"2024-09-21T05:55:33.636760+0000","last_peered":"2024-09-21T05:55:33.636760+0000","last_clean":"2024-09-21T05:55:33.636760+0000","last_became_active":"2024-09-21T05:55:11.983543+0000","last_became_peered":"2024-09-21T05:55:11.983543+0000","last_unstale":"2024-09-21T05:55:33.636760+0000","last_undegraded":"2024-09-21T05:55:33.636760+0000","last_fullsized":"2024-09-21T05:55:33.636760+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:39.636408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.749459+0000","last_change":"2024-09-21T05:55:14.003851+0000","last_active":"2024-09-21T05:57:02.749459+0000","last_peered":"2024-09-21T05:57:02.749459+0000","last_clean":"2024-09-21T05:57:02.749459+0000","last_became_active":"2024-09-21T05:55:14.003354+0000","last_became_peered":"2024-09-21T05:55:14.003354+0000","last_unstale":"2024-09-21T05:57:02.749459+0000","last_undegraded":"2024-09-21T05:57:02.749459+0000","last_fullsized":"2024-09-21T05:57:02.749459+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:01.321612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"74'5","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.645976+0000","last_change":"2024-09-21T05:55:33.645825+0000","last_active":"2024-09-21T05:55:33.645976+0000","last_peered":"2024-09-21T05:55:33.645976+0000","last_clean":"2024-09-21T05:55:33.645976+0000","last_became_active":"2024-09-21T05:55:33.640707+0000","last_became_peered":"2024-09-21T05:55:33.640707+0000","last_unstale":"2024-09-21T05:55:33.645976+0000","last_undegraded":"2024-09-21T05:55:33.645976+0000","last_fullsized":"2024-09-21T05:55:33.645976+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:33.665707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.756593+0000","last_change":"2024-09-21T05:55:14.001855+0000","last_active":"2024-09-21T05:57:02.756593+0000","last_peered":"2024-09-21T05:57:02.756593+0000","last_clean":"2024-09-21T05:57:02.756593+0000","last_became_active":"2024-09-21T05:55:14.001484+0000","last_became_peered":"2024-09-21T05:55:14.001484+0000","last_unstale":"2024-09-21T05:57:02.756593+0000","last_undegraded":"2024-09-21T05:57:02.756593+0000","last_fullsized":"2024-09-21T05:57:02.756593+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:57.615334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'3","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.767928+0000","last_change":"2024-09-21T05:55:11.979716+0000","last_active":"2024-09-21T05:55:33.767928+0000","last_peered":"2024-09-21T05:55:33.767928+0000","last_clean":"2024-09-21T05:55:33.767928+0000","last_became_active":"2024-09-21T05:55:11.979307+0000","last_became_peered":"2024-09-21T05:55:11.979307+0000","last_unstale":"2024-09-21T05:55:33.767928+0000","last_undegraded":"2024-09-21T05:55:33.767928+0000","last_fullsized":"2024-09-21T05:55:33.767928+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:30.019040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.755924+0000","last_change":"2024-09-21T05:55:13.991769+0000","last_active":"2024-09-21T05:57:02.755924+0000","last_peered":"2024-09-21T05:57:02.755924+0000","last_clean":"2024-09-21T05:57:02.755924+0000","last_became_active":"2024-09-21T05:55:13.991345+0000","last_became_peered":"2024-09-21T05:55:13.991345+0000","last_unstale":"2024-09-21T05:57:02.755924+0000","last_undegraded":"2024-09-21T05:57:02.755924+0000","last_fullsized":"2024-09-21T05:57:02.755924+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:52.227126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"76'7","reported_seq":136,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:06.406951+0000","last_change":"2024-09-21T05:55:12.013836+0000","last_active":"2024-09-21T05:57:06.406951+0000","last_peered":"2024-09-21T05:57:06.406951+0000","last_clean":"2024-09-21T05:57:06.406951+0000","last_became_active":"2024-09-21T05:55:12.013436+0000","last_became_peered":"2024-09-21T05:55:12.013436+0000","last_unstale":"2024-09-21T05:57:06.406951+0000","last_undegraded":"2024-09-21T05:57:06.406951+0000","last_fullsized":"2024-09-21T05:57:06.406951+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:40:02.767620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.766968+0000","last_change":"2024-09-21T05:55:14.001156+0000","last_active":"2024-09-21T05:57:02.766968+0000","last_peered":"2024-09-21T05:57:02.766968+0000","last_clean":"2024-09-21T05:57:02.766968+0000","last_became_active":"2024-09-21T05:55:14.000777+0000","last_became_peered":"2024-09-21T05:55:14.000777+0000","last_unstale":"2024-09-21T05:57:02.766968+0000","last_undegraded":"2024-09-21T05:57:02.766968+0000","last_fullsized":"2024-09-21T05:57:02.766968+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:46:08.000612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":98,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636154+0000","last_change":"2024-09-21T05:53:58.971355+0000","last_active":"2024-09-21T05:55:33.636154+0000","last_peered":"2024-09-21T05:55:33.636154+0000","last_clean":"2024-09-21T05:55:33.636154+0000","last_became_active":"2024-09-21T05:53:58.965825+0000","last_became_peered":"2024-09-21T05:53:58.965825+0000","last_unstale":"2024-09-21T05:55:33.636154+0000","last_undegraded":"2024-09-21T05:55:33.636154+0000","last_fullsized":"2024-09-21T05:55:33.636154+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:50:33.190160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:50:33.190160+0000","last_clean_scrub_stamp":"2024-09-21T05:50:33.190160+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:31.387339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"74'6","reported_seq":32,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.696020+0000","last_change":"2024-09-21T05:55:11.976092+0000","last_active":"2024-09-21T05:55:33.696020+0000","last_peered":"2024-09-21T05:55:33.696020+0000","last_clean":"2024-09-21T05:55:33.696020+0000","last_became_active":"2024-09-21T05:55:11.975657+0000","last_became_peered":"2024-09-21T05:55:11.975657+0000","last_unstale":"2024-09-21T05:55:33.696020+0000","last_undegraded":"2024-09-21T05:55:33.696020+0000","last_fullsized":"2024-09-21T05:55:33.696020+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:46.792054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.745844+0000","last_change":"2024-09-21T05:55:13.997621+0000","last_active":"2024-09-21T05:57:02.745844+0000","last_peered":"2024-09-21T05:57:02.745844+0000","last_clean":"2024-09-21T05:57:02.745844+0000","last_became_active":"2024-09-21T05:55:13.997127+0000","last_became_peered":"2024-09-21T05:55:13.997127+0000","last_unstale":"2024-09-21T05:57:02.745844+0000","last_undegraded":"2024-09-21T05:57:02.745844+0000","last_fullsized":"2024-09-21T05:57:02.745844+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:32.399696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"74'4","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.635928+0000","last_change":"2024-09-21T05:55:11.982387+0000","last_active":"2024-09-21T05:55:33.635928+0000","last_peered":"2024-09-21T05:55:33.635928+0000","last_clean":"2024-09-21T05:55:33.635928+0000","last_became_active":"2024-09-21T05:55:11.981851+0000","last_became_peered":"2024-09-21T05:55:11.981851+0000","last_unstale":"2024-09-21T05:55:33.635928+0000","last_undegraded":"2024-09-21T05:55:33.635928+0000","last_fullsized":"2024-09-21T05:55:33.635928+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:43:45.345738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.748575+0000","last_change":"2024-09-21T05:55:13.981065+0000","last_active":"2024-09-21T05:57:02.748575+0000","last_peered":"2024-09-21T05:57:02.748575+0000","last_clean":"2024-09-21T05:57:02.748575+0000","last_became_active":"2024-09-21T05:55:13.980524+0000","last_became_peered":"2024-09-21T05:55:13.980524+0000","last_unstale":"2024-09-21T05:57:02.748575+0000","last_undegraded":"2024-09-21T05:57:02.748575+0000","last_fullsized":"2024-09-21T05:57:02.748575+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:13:43.322429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"76'6","reported_seq":130,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:04.385849+0000","last_change":"2024-09-21T05:55:12.013434+0000","last_active":"2024-09-21T05:57:04.385849+0000","last_peered":"2024-09-21T05:57:04.385849+0000","last_clean":"2024-09-21T05:57:04.385849+0000","last_became_active":"2024-09-21T05:55:12.013087+0000","last_became_peered":"2024-09-21T05:55:12.013087+0000","last_unstale":"2024-09-21T05:57:04.385849+0000","last_undegraded":"2024-09-21T05:57:04.385849+0000","last_fullsized":"2024-09-21T05:57:04.385849+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:19.876288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.744907+0000","last_change":"2024-09-21T05:55:13.996102+0000","last_active":"2024-09-21T05:57:02.744907+0000","last_peered":"2024-09-21T05:57:02.744907+0000","last_clean":"2024-09-21T05:57:02.744907+0000","last_became_active":"2024-09-21T05:55:13.994900+0000","last_became_peered":"2024-09-21T05:55:13.994900+0000","last_unstale":"2024-09-21T05:57:02.744907+0000","last_undegraded":"2024-09-21T05:57:02.744907+0000","last_fullsized":"2024-09-21T05:57:02.744907+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:04.998577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"74'5","reported_seq":128,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:03.782035+0000","last_change":"2024-09-21T05:55:11.981184+0000","last_active":"2024-09-21T05:57:03.782035+0000","last_peered":"2024-09-21T05:57:03.782035+0000","last_clean":"2024-09-21T05:57:03.782035+0000","last_became_active":"2024-09-21T05:55:11.979581+0000","last_became_peered":"2024-09-21T05:55:11.979581+0000","last_unstale":"2024-09-21T05:57:03.782035+0000","last_undegraded":"2024-09-21T05:57:03.782035+0000","last_fullsized":"2024-09-21T05:57:03.782035+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:53.833860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"76'7","reported_seq":32,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.736365+0000","last_change":"2024-09-21T05:55:33.768136+0000","last_active":"2024-09-21T05:57:02.736365+0000","last_peered":"2024-09-21T05:57:02.736365+0000","last_clean":"2024-09-21T05:57:02.736365+0000","last_became_active":"2024-09-21T05:55:33.767697+0000","last_became_peered":"2024-09-21T05:55:33.767697+0000","last_unstale":"2024-09-21T05:57:02.736365+0000","last_undegraded":"2024-09-21T05:57:02.736365+0000","last_fullsized":"2024-09-21T05:57:02.736365+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:48.146453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"74'1","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.633171+0000","last_change":"2024-09-21T05:55:12.013121+0000","last_active":"2024-09-21T05:55:33.633171+0000","last_peered":"2024-09-21T05:55:33.633171+0000","last_clean":"2024-09-21T05:55:33.633171+0000","last_became_active":"2024-09-21T05:55:12.012657+0000","last_became_peered":"2024-09-21T05:55:12.012657+0000","last_unstale":"2024-09-21T05:55:33.633171+0000","last_undegraded":"2024-09-21T05:55:33.633171+0000","last_fullsized":"2024-09-21T05:55:33.633171+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:25:48.002938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.747160+0000","last_change":"2024-09-21T05:55:14.004660+0000","last_active":"2024-09-21T05:57:02.747160+0000","last_peered":"2024-09-21T05:57:02.747160+0000","last_clean":"2024-09-21T05:57:02.747160+0000","last_became_active":"2024-09-21T05:55:14.004297+0000","last_became_peered":"2024-09-21T05:55:14.004297+0000","last_unstale":"2024-09-21T05:57:02.747160+0000","last_undegraded":"2024-09-21T05:57:02.747160+0000","last_fullsized":"2024-09-21T05:57:02.747160+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:24.399891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"74'4","reported_seq":28,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636395+0000","last_change":"2024-09-21T05:55:12.015717+0000","last_active":"2024-09-21T05:55:33.636395+0000","last_peered":"2024-09-21T05:55:33.636395+0000","last_clean":"2024-09-21T05:55:33.636395+0000","last_became_active":"2024-09-21T05:55:12.015419+0000","last_became_peered":"2024-09-21T05:55:12.015419+0000","last_unstale":"2024-09-21T05:55:33.636395+0000","last_undegraded":"2024-09-21T05:55:33.636395+0000","last_fullsized":"2024-09-21T05:55:33.636395+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:57.988860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.747609+0000","last_change":"2024-09-21T05:55:13.992460+0000","last_active":"2024-09-21T05:57:02.747609+0000","last_peered":"2024-09-21T05:57:02.747609+0000","last_clean":"2024-09-21T05:57:02.747609+0000","last_became_active":"2024-09-21T05:55:13.991887+0000","last_became_peered":"2024-09-21T05:55:13.991887+0000","last_unstale":"2024-09-21T05:57:02.747609+0000","last_undegraded":"2024-09-21T05:57:02.747609+0000","last_fullsized":"2024-09-21T05:57:02.747609+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:37.718050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"76'17","reported_seq":40,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:03.008081+0000","last_change":"2024-09-21T05:55:11.982235+0000","last_active":"2024-09-21T05:57:03.008081+0000","last_peered":"2024-09-21T05:57:03.008081+0000","last_clean":"2024-09-21T05:57:03.008081+0000","last_became_active":"2024-09-21T05:55:11.981846+0000","last_became_peered":"2024-09-21T05:55:11.981846+0000","last_unstale":"2024-09-21T05:57:03.008081+0000","last_undegraded":"2024-09-21T05:57:03.008081+0000","last_fullsized":"2024-09-21T05:57:03.008081+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:40:04.330346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.744826+0000","last_change":"2024-09-21T05:55:13.999249+0000","last_active":"2024-09-21T05:57:02.744826+0000","last_peered":"2024-09-21T05:57:02.744826+0000","last_clean":"2024-09-21T05:57:02.744826+0000","last_became_active":"2024-09-21T05:55:13.998452+0000","last_became_peered":"2024-09-21T05:55:13.998452+0000","last_unstale":"2024-09-21T05:57:02.744826+0000","last_undegraded":"2024-09-21T05:57:02.744826+0000","last_fullsized":"2024-09-21T05:57:02.744826+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:59.364684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"74'7","reported_seq":31,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.696351+0000","last_change":"2024-09-21T05:55:11.971850+0000","last_active":"2024-09-21T05:55:33.696351+0000","last_peered":"2024-09-21T05:55:33.696351+0000","last_clean":"2024-09-21T05:55:33.696351+0000","last_became_active":"2024-09-21T05:55:11.971415+0000","last_became_peered":"2024-09-21T05:55:11.971415+0000","last_unstale":"2024-09-21T05:55:33.696351+0000","last_undegraded":"2024-09-21T05:55:33.696351+0000","last_fullsized":"2024-09-21T05:55:33.696351+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:51.990745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"76'7","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.743839+0000","last_change":"2024-09-21T05:55:13.994500+0000","last_active":"2024-09-21T05:57:02.743839+0000","last_peered":"2024-09-21T05:57:02.743839+0000","last_clean":"2024-09-21T05:57:02.743839+0000","last_became_active":"2024-09-21T05:55:13.994146+0000","last_became_peered":"2024-09-21T05:55:13.994146+0000","last_unstale":"2024-09-21T05:57:02.743839+0000","last_undegraded":"2024-09-21T05:57:02.743839+0000","last_fullsized":"2024-09-21T05:57:02.743839+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:42.934234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639268+0000","last_change":"2024-09-21T05:55:12.013479+0000","last_active":"2024-09-21T05:55:33.639268+0000","last_peered":"2024-09-21T05:55:33.639268+0000","last_clean":"2024-09-21T05:55:33.639268+0000","last_became_active":"2024-09-21T05:55:12.013180+0000","last_became_peered":"2024-09-21T05:55:12.013180+0000","last_unstale":"2024-09-21T05:55:33.639268+0000","last_undegraded":"2024-09-21T05:55:33.639268+0000","last_fullsized":"2024-09-21T05:55:33.639268+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:28.800072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"76'2","reported_seq":20,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.709028+0000","last_change":"2024-09-21T05:55:14.001795+0000","last_active":"2024-09-21T05:57:02.709028+0000","last_peered":"2024-09-21T05:57:02.709028+0000","last_clean":"2024-09-21T05:57:02.709028+0000","last_became_active":"2024-09-21T05:55:14.001430+0000","last_became_peered":"2024-09-21T05:55:14.001430+0000","last_unstale":"2024-09-21T05:57:02.709028+0000","last_undegraded":"2024-09-21T05:57:02.709028+0000","last_fullsized":"2024-09-21T05:57:02.709028+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:10.339939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"74'7","reported_seq":30,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.638793+0000","last_change":"2024-09-21T05:55:11.976894+0000","last_active":"2024-09-21T05:55:33.638793+0000","last_peered":"2024-09-21T05:55:33.638793+0000","last_clean":"2024-09-21T05:55:33.638793+0000","last_became_active":"2024-09-21T05:55:11.976516+0000","last_became_peered":"2024-09-21T05:55:11.976516+0000","last_unstale":"2024-09-21T05:55:33.638793+0000","last_undegraded":"2024-09-21T05:55:33.638793+0000","last_fullsized":"2024-09-21T05:55:33.638793+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:58.726311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.746513+0000","last_change":"2024-09-21T05:55:14.003760+0000","last_active":"2024-09-21T05:57:02.746513+0000","last_peered":"2024-09-21T05:57:02.746513+0000","last_clean":"2024-09-21T05:57:02.746513+0000","last_became_active":"2024-09-21T05:55:14.003180+0000","last_became_peered":"2024-09-21T05:55:14.003180+0000","last_unstale":"2024-09-21T05:57:02.746513+0000","last_undegraded":"2024-09-21T05:57:02.746513+0000","last_fullsized":"2024-09-21T05:57:02.746513+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:25.077765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"74'3","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639713+0000","last_change":"2024-09-21T05:55:11.983865+0000","last_active":"2024-09-21T05:55:33.639713+0000","last_peered":"2024-09-21T05:55:33.639713+0000","last_clean":"2024-09-21T05:55:33.639713+0000","last_became_active":"2024-09-21T05:55:11.983473+0000","last_became_peered":"2024-09-21T05:55:11.983473+0000","last_unstale":"2024-09-21T05:55:33.639713+0000","last_undegraded":"2024-09-21T05:55:33.639713+0000","last_fullsized":"2024-09-21T05:55:33.639713+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:18.767317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"76'8","reported_seq":26,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.741938+0000","last_change":"2024-09-21T05:55:13.999001+0000","last_active":"2024-09-21T05:57:02.741938+0000","last_peered":"2024-09-21T05:57:02.741938+0000","last_clean":"2024-09-21T05:57:02.741938+0000","last_became_active":"2024-09-21T05:55:13.998656+0000","last_became_peered":"2024-09-21T05:55:13.998656+0000","last_unstale":"2024-09-21T05:57:02.741938+0000","last_undegraded":"2024-09-21T05:57:02.741938+0000","last_fullsized":"2024-09-21T05:57:02.741938+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:50.913635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"74'4","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.646791+0000","last_change":"2024-09-21T05:55:33.646124+0000","last_active":"2024-09-21T05:55:33.646791+0000","last_peered":"2024-09-21T05:55:33.646791+0000","last_clean":"2024-09-21T05:55:33.646791+0000","last_became_active":"2024-09-21T05:55:33.641142+0000","last_became_peered":"2024-09-21T05:55:33.641142+0000","last_unstale":"2024-09-21T05:55:33.646791+0000","last_undegraded":"2024-09-21T05:55:33.646791+0000","last_fullsized":"2024-09-21T05:55:33.646791+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:29:36.341473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.767041+0000","last_change":"2024-09-21T05:55:13.997494+0000","last_active":"2024-09-21T05:57:02.767041+0000","last_peered":"2024-09-21T05:57:02.767041+0000","last_clean":"2024-09-21T05:57:02.767041+0000","last_became_active":"2024-09-21T05:55:13.997138+0000","last_became_peered":"2024-09-21T05:55:13.997138+0000","last_unstale":"2024-09-21T05:57:02.767041+0000","last_undegraded":"2024-09-21T05:57:02.767041+0000","last_fullsized":"2024-09-21T05:57:02.767041+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:02:01.777126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"76'15","reported_seq":33,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.747951+0000","last_change":"2024-09-21T05:55:14.002750+0000","last_active":"2024-09-21T05:57:02.747951+0000","last_peered":"2024-09-21T05:57:02.747951+0000","last_clean":"2024-09-21T05:57:02.747951+0000","last_became_active":"2024-09-21T05:55:14.002150+0000","last_became_peered":"2024-09-21T05:55:14.002150+0000","last_unstale":"2024-09-21T05:57:02.747951+0000","last_undegraded":"2024-09-21T05:57:02.747951+0000","last_fullsized":"2024-09-21T05:57:02.747951+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:26:49.603133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"74'5","reported_seq":29,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636032+0000","last_change":"2024-09-21T05:55:11.982366+0000","last_active":"2024-09-21T05:55:33.636032+0000","last_peered":"2024-09-21T05:55:33.636032+0000","last_clean":"2024-09-21T05:55:33.636032+0000","last_became_active":"2024-09-21T05:55:11.981994+0000","last_became_peered":"2024-09-21T05:55:11.981994+0000","last_unstale":"2024-09-21T05:55:33.636032+0000","last_undegraded":"2024-09-21T05:55:33.636032+0000","last_fullsized":"2024-09-21T05:55:33.636032+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:14.626758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"76'9","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.769310+0000","last_change":"2024-09-21T05:55:14.008204+0000","last_active":"2024-09-21T05:57:02.769310+0000","last_peered":"2024-09-21T05:57:02.769310+0000","last_clean":"2024-09-21T05:57:02.769310+0000","last_became_active":"2024-09-21T05:55:14.007800+0000","last_became_peered":"2024-09-21T05:55:14.007800+0000","last_unstale":"2024-09-21T05:57:02.769310+0000","last_undegraded":"2024-09-21T05:57:02.769310+0000","last_fullsized":"2024-09-21T05:57:02.769310+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:49.760850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"74'12","reported_seq":36,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.636283+0000","last_change":"2024-09-21T05:55:11.986168+0000","last_active":"2024-09-21T05:55:33.636283+0000","last_peered":"2024-09-21T05:55:33.636283+0000","last_clean":"2024-09-21T05:55:33.636283+0000","last_became_active":"2024-09-21T05:55:11.985694+0000","last_became_peered":"2024-09-21T05:55:11.985694+0000","last_unstale":"2024-09-21T05:55:33.636283+0000","last_undegraded":"2024-09-21T05:55:33.636283+0000","last_fullsized":"2024-09-21T05:55:33.636283+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:44.863935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"76'6","reported_seq":24,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.748453+0000","last_change":"2024-09-21T05:55:14.002459+0000","last_active":"2024-09-21T05:57:02.748453+0000","last_peered":"2024-09-21T05:57:02.748453+0000","last_clean":"2024-09-21T05:57:02.748453+0000","last_became_active":"2024-09-21T05:55:14.001962+0000","last_became_peered":"2024-09-21T05:55:14.001962+0000","last_unstale":"2024-09-21T05:57:02.748453+0000","last_undegraded":"2024-09-21T05:57:02.748453+0000","last_fullsized":"2024-09-21T05:57:02.748453+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:53:26.892772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"74'2","reported_seq":25,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:55:33.639340+0000","last_change":"2024-09-21T05:55:12.014149+0000","last_active":"2024-09-21T05:55:33.639340+0000","last_peered":"2024-09-21T05:55:33.639340+0000","last_clean":"2024-09-21T05:55:33.639340+0000","last_became_active":"2024-09-21T05:55:12.013789+0000","last_became_peered":"2024-09-21T05:55:12.013789+0000","last_unstale":"2024-09-21T05:55:33.639340+0000","last_undegraded":"2024-09-21T05:55:33.639340+0000","last_fullsized":"2024-09-21T05:55:33.639340+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:18.304023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"76'5","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:02.749464+0000","last_change":"2024-09-21T05:55:13.993500+0000","last_active":"2024-09-21T05:57:02.749464+0000","last_peered":"2024-09-21T05:57:02.749464+0000","last_clean":"2024-09-21T05:57:02.749464+0000","last_became_active":"2024-09-21T05:55:13.993104+0000","last_became_peered":"2024-09-21T05:55:13.993104+0000","last_unstale":"2024-09-21T05:57:02.749464+0000","last_undegraded":"2024-09-21T05:57:02.749464+0000","last_fullsized":"2024-09-21T05:57:02.749464+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:12.967277+0000","last_clean_scrub_stamp":"2024-09-21T05:55:12.967277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:12:01.445119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"76'14","reported_seq":159,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-21T05:57:05.784296+0000","last_change":"2024-09-21T05:55:33.650207+0000","last_active":"2024-09-21T05:57:05.784296+0000","last_peered":"2024-09-21T05:57:05.784296+0000","last_clean":"2024-09-21T05:57:05.784296+0000","last_became_active":"2024-09-21T05:55:33.638388+0000","last_became_peered":"2024-09-21T05:55:33.638388+0000","last_unstale":"2024-09-21T05:57:05.784296+0000","last_undegraded":"2024-09-21T05:57:05.784296+0000","last_fullsized":"2024-09-21T05:57:05.784296+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:55:10.954278+0000","last_clean_scrub_stamp":"2024-09-21T05:55:10.954278+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:51.478063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72477,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1057,"num_read_kb":603,"num_write":260,"num_write_kb":154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":217739,"data_compressed":5682,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281769,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":68,"seq":292057776164,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":839680,"data_stored":412700,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939689,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":1067516,"data_compressed":43475,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103214,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":856064,"data_stored":411426,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299444,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":411804,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,0,0,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495672,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":1004323,"data_compressed":41581,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659198,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":839680,"data_stored":411143,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822723,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":1004141,"data_compressed":41581,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986249,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":839680,"data_stored":411536,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36180,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708972,"statfs":{"total":95995035648,"available":95957987328,"internally_reserved":0,"allocated":856064,"data_stored":410968,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378707,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":477799,"data_compressed":7646,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542230,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":901120,"data_stored":476823,"data_compressed":7646,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705757,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36184,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708968,"statfs":{"total":95995035648,"available":95957983232,"internally_reserved":0,"allocated":860160,"data_stored":413014,"data_compressed":5752,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34208,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34208,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34208,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35829,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35829,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35829,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35829,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35829,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":66519,"data_compressed":1894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":67495,"data_compressed":1894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":66727,"data_compressed":1894,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:57:10.765 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T05:57:11.047 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:57:11.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.513+0000 7f560aa2e640 1 -- 172.21.15.117:0/266971493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5604108cf0 msgr2=0x7f560410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:11.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.513+0000 7f560aa2e640 1 --2- 172.21.15.117:0/266971493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5604108cf0 0x7f560410f520 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f55fc009ca0 tx=0x7f55fc02a8c0 comp rx=0 tx=0).stop 2024-09-21T05:57:11.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.513+0000 7f560aa2e640 1 -- 172.21.15.117:0/266971493 shutdown_connections 2024-09-21T05:57:11.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.513+0000 7f560aa2e640 1 --2- 172.21.15.117:0/266971493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5604108cf0 0x7f560410f520 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:11.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.513+0000 7f560aa2e640 1 --2- 172.21.15.117:0/266971493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56041083a0 0x7f56041087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:11.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.513+0000 7f560aa2e640 1 --2- 172.21.15.117:0/266971493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5604102a60 0x7f5604102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:11.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.513+0000 7f560aa2e640 1 -- 172.21.15.117:0/266971493 >> 172.21.15.117:0/266971493 conn(0x7f5604079730 msgr2=0x7f5604079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:11.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.513+0000 7f560aa2e640 1 -- 172.21.15.117:0/266971493 shutdown_connections 2024-09-21T05:57:11.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.513+0000 7f560aa2e640 1 -- 172.21.15.117:0/266971493 wait complete. 2024-09-21T05:57:11.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.514+0000 7f560aa2e640 1 Processor -- start 2024-09-21T05:57:11.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.514+0000 7f560aa2e640 1 -- start start 2024-09-21T05:57:11.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.514+0000 7f560aa2e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5604102a60 0x7f560419b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:11.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.514+0000 7f560aa2e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56041083a0 0x7f560419b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:11.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.514+0000 7f560aa2e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5604108cf0 0x7f560419bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:11.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.514+0000 7f560aa2e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5604112960 con 0x7f5604108cf0 2024-09-21T05:57:11.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.514+0000 7f560aa2e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f56041127e0 con 0x7f56041083a0 2024-09-21T05:57:11.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.514+0000 7f560aa2e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5604112ae0 con 0x7f5604102a60 2024-09-21T05:57:11.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.514+0000 7f5608fcb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5604108cf0 0x7f560419bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:11.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.514+0000 7f5608fcb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5604108cf0 0x7f560419bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60394/0 (socket says 172.21.15.117:60394) 2024-09-21T05:57:11.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.514+0000 7f5608fcb640 1 -- 172.21.15.117:0/969636385 learned_addr learned my addr 172.21.15.117:0/969636385 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:11.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.514+0000 7f5603fff640 1 --2- 172.21.15.117:0/969636385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5604102a60 0x7f560419b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:11.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.514+0000 7f55fbfff640 1 --2- 172.21.15.117:0/969636385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56041083a0 0x7f560419b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:11.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.515+0000 7f5608fcb640 1 -- 172.21.15.117:0/969636385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5604102a60 msgr2=0x7f560419b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:11.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.515+0000 7f5608fcb640 1 --2- 172.21.15.117:0/969636385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5604102a60 0x7f560419b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:11.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.515+0000 7f5608fcb640 1 -- 172.21.15.117:0/969636385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56041083a0 msgr2=0x7f560419b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:11.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.515+0000 7f5608fcb640 1 --2- 172.21.15.117:0/969636385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56041083a0 0x7f560419b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:11.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.515+0000 7f5608fcb640 1 -- 172.21.15.117:0/969636385 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56041a6580 con 0x7f5604108cf0 2024-09-21T05:57:11.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.515+0000 7f55fbfff640 1 --2- 172.21.15.117:0/969636385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56041083a0 0x7f560419b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:57:11.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.515+0000 7f5608fcb640 1 --2- 172.21.15.117:0/969636385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5604108cf0 0x7f560419bcf0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f55fc005cc0 tx=0x7f55fc02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:11.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.515+0000 7f5601ffb640 1 -- 172.21.15.117:0/969636385 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc003bf0 con 0x7f5604108cf0 2024-09-21T05:57:11.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.515+0000 7f5601ffb640 1 -- 172.21.15.117:0/969636385 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55fc003d90 con 0x7f5604108cf0 2024-09-21T05:57:11.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.515+0000 7f560aa2e640 1 -- 172.21.15.117:0/969636385 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56041a6810 con 0x7f5604108cf0 2024-09-21T05:57:11.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.515+0000 7f5601ffb640 1 -- 172.21.15.117:0/969636385 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc02fbc0 con 0x7f5604108cf0 2024-09-21T05:57:11.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.515+0000 7f560aa2e640 1 -- 172.21.15.117:0/969636385 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56041a6da0 con 0x7f5604108cf0 2024-09-21T05:57:11.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.516+0000 7f560aa2e640 1 -- 172.21.15.117:0/969636385 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5604102e70 con 0x7f5604108cf0 2024-09-21T05:57:11.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.517+0000 7f5601ffb640 1 -- 172.21.15.117:0/969636385 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 22) ==== 154530+0+0 (secure 0 0 0) 0x7f55fc02fd60 con 0x7f5604108cf0 2024-09-21T05:57:11.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.518+0000 7f5601ffb640 1 --2- 172.21.15.117:0/969636385 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f55f00b65e0 0x7f55f00b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:11.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.518+0000 7f5603fff640 1 --2- 172.21.15.117:0/969636385 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f55f00b65e0 0x7f55f00b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:11.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.518+0000 7f5601ffb640 1 -- 172.21.15.117:0/969636385 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f55fc1088a0 con 0x7f5604108cf0 2024-09-21T05:57:11.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.519+0000 7f5603fff640 1 --2- 172.21.15.117:0/969636385 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f55f00b65e0 0x7f55f00b8af0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f560406ba20 tx=0x7f55f400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:11.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.523+0000 7f5601ffb640 1 -- 172.21.15.117:0/969636385 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f55fc0cf090 con 0x7f5604108cf0 2024-09-21T05:57:11.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.738+0000 7f560aa2e640 1 -- 172.21.15.117:0/969636385 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f560402e640 con 0x7f5604108cf0 2024-09-21T05:57:11.740 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:11 smithi117 ceph-mon[30947]: from='client.15492 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T05:57:11.740 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:11 smithi117 ceph-mon[30947]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T05:57:11.740 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:11 smithi117 ceph-mon[30947]: pgmap v323: 97 pgs: 97 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-21T05:57:11.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.738+0000 7f5601ffb640 1 -- 172.21.15.117:0/969636385 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+12001 (secure 0 0 0) 0x7f55fc0098b0 con 0x7f5604108cf0 2024-09-21T05:57:11.741 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:57:11.741 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":27,"btime":"2024-09-21T05:57:06:651813+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3227057504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3227057504},{"type":"v1","addr":"172.21.15.117:6835","nonce":3227057504}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3303829183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3303829183},{"type":"v1","addr":"172.21.15.117:6837","nonce":3303829183}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T05:55:14.555897+0000","modified":"2024-09-21T05:57:06.599449+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24638,"mds_1":24637,"mds_2":24628,"mds_3":24616,"mds_4":24602},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6839/2314572411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2314572411},{"type":"v1","addr":"172.21.15.117:6839","nonce":2314572411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6841/3378096561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":3378096561},{"type":"v1","addr":"172.21.15.117:6841","nonce":3378096561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/3725629862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3725629862},{"type":"v1","addr":"172.21.15.138:6833","nonce":3725629862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/4276801396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4276801396},{"type":"v1","addr":"172.21.15.138:6835","nonce":4276801396}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6833/2224910879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2224910879},{"type":"v1","addr":"172.21.15.192:6833","nonce":2224910879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.138:6837/2540335374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2540335374},{"type":"v1","addr":"172.21.15.138:6837","nonce":2540335374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24616":{"gid":24616,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.192:6835/3530688860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3530688860},{"type":"v1","addr":"172.21.15.192:6835","nonce":3530688860}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.192:6837/2683860842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2683860842},{"type":"v1","addr":"172.21.15.192:6837","nonce":2683860842}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.192:6839/1727623231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1727623231},{"type":"v1","addr":"172.21.15.192:6839","nonce":1727623231}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/3230372576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":3230372576},{"type":"v1","addr":"172.21.15.138:6839","nonce":3230372576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24602,24628,24616,24637,24638]},"id":1}]} 2024-09-21T05:57:11.741 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 27 2024-09-21T05:57:11.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.741+0000 7f560aa2e640 1 -- 172.21.15.117:0/969636385 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f55f00b65e0 msgr2=0x7f55f00b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:11.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.741+0000 7f560aa2e640 1 --2- 172.21.15.117:0/969636385 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f55f00b65e0 0x7f55f00b8af0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f560406ba20 tx=0x7f55f400b390 comp rx=0 tx=0).stop 2024-09-21T05:57:11.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.741+0000 7f560aa2e640 1 -- 172.21.15.117:0/969636385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5604108cf0 msgr2=0x7f560419bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:11.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.741+0000 7f560aa2e640 1 --2- 172.21.15.117:0/969636385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5604108cf0 0x7f560419bcf0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f55fc005cc0 tx=0x7f55fc02f380 comp rx=0 tx=0).stop 2024-09-21T05:57:11.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.741+0000 7f560aa2e640 1 -- 172.21.15.117:0/969636385 shutdown_connections 2024-09-21T05:57:11.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.741+0000 7f560aa2e640 1 --2- 172.21.15.117:0/969636385 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f55f00b65e0 0x7f55f00b8af0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:11.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.741+0000 7f560aa2e640 1 --2- 172.21.15.117:0/969636385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5604108cf0 0x7f560419bcf0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:11.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.741+0000 7f560aa2e640 1 --2- 172.21.15.117:0/969636385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56041083a0 0x7f560419b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:11.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.741+0000 7f560aa2e640 1 --2- 172.21.15.117:0/969636385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5604102a60 0x7f560419b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:11.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.741+0000 7f560aa2e640 1 -- 172.21.15.117:0/969636385 >> 172.21.15.117:0/969636385 conn(0x7f5604079730 msgr2=0x7f5604104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:11.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.741+0000 7f560aa2e640 1 -- 172.21.15.117:0/969636385 shutdown_connections 2024-09-21T05:57:11.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:11.741+0000 7f560aa2e640 1 -- 172.21.15.117:0/969636385 wait complete. 2024-09-21T05:57:11.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:11 smithi192 ceph-mon[34700]: from='client.15492 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T05:57:11.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:11 smithi192 ceph-mon[34700]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T05:57:11.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:11 smithi192 ceph-mon[34700]: pgmap v323: 97 pgs: 97 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-21T05:57:12.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:11 smithi138 ceph-mon[34391]: from='client.15492 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T05:57:12.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:11 smithi138 ceph-mon[34391]: from='client.15498 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T05:57:12.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:11 smithi138 ceph-mon[34391]: pgmap v323: 97 pgs: 97 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-21T05:57:12.542 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T05:57:12.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:12 smithi117 ceph-mon[30947]: from='client.15504 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T05:57:12.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:12 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/969636385' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:57:12.790 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:57:12.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:12 smithi192 ceph-mon[34700]: from='client.15504 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T05:57:12.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:12 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/969636385' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:57:13.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:12 smithi138 ceph-mon[34391]: from='client.15504 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T05:57:13.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:12 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/969636385' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:57:13.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.271+0000 7fda9e783640 1 -- 172.21.15.117:0/3906268516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda98102080 msgr2=0x7fda98102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:13.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.271+0000 7fda9e783640 1 --2- 172.21.15.117:0/3906268516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda98102080 0x7fda98102490 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fda88009900 tx=0x7fda8802a410 comp rx=0 tx=0).stop 2024-09-21T05:57:13.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.272+0000 7fda9e783640 1 -- 172.21.15.117:0/3906268516 shutdown_connections 2024-09-21T05:57:13.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.272+0000 7fda9e783640 1 --2- 172.21.15.117:0/3906268516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda98109eb0 0x7fda981106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:13.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.272+0000 7fda9e783640 1 --2- 172.21.15.117:0/3906268516 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda98109560 0x7fda98109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:13.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.272+0000 7fda9e783640 1 --2- 172.21.15.117:0/3906268516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda98102080 0x7fda98102490 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:13.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.272+0000 7fda9e783640 1 -- 172.21.15.117:0/3906268516 >> 172.21.15.117:0/3906268516 conn(0x7fda98079ec0 msgr2=0x7fda9807a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:13.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.272+0000 7fda9e783640 1 -- 172.21.15.117:0/3906268516 shutdown_connections 2024-09-21T05:57:13.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.272+0000 7fda9e783640 1 -- 172.21.15.117:0/3906268516 wait complete. 2024-09-21T05:57:13.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.272+0000 7fda9e783640 1 Processor -- start 2024-09-21T05:57:13.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.272+0000 7fda9e783640 1 -- start start 2024-09-21T05:57:13.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.273+0000 7fda9e783640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda98102080 0x7fda9819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:13.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.273+0000 7fda9e783640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda98109560 0x7fda9819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:13.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.273+0000 7fda9e783640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda98109eb0 0x7fda9819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:13.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.273+0000 7fda9e783640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fda98113b70 con 0x7fda98109560 2024-09-21T05:57:13.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.273+0000 7fda9e783640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fda981139f0 con 0x7fda98102080 2024-09-21T05:57:13.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.273+0000 7fda9e783640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fda98113cf0 con 0x7fda98109eb0 2024-09-21T05:57:13.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.273+0000 7fda8ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda98109560 0x7fda9819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:13.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.273+0000 7fda8ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda98109560 0x7fda9819aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60408/0 (socket says 172.21.15.117:60408) 2024-09-21T05:57:13.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.273+0000 7fda8ffff640 1 -- 172.21.15.117:0/2364988671 learned_addr learned my addr 172.21.15.117:0/2364988671 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:13.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.273+0000 7fda97fff640 1 --2- 172.21.15.117:0/2364988671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda98102080 0x7fda9819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:13.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.273+0000 7fda9cd20640 1 --2- 172.21.15.117:0/2364988671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda98109eb0 0x7fda9819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:13.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.273+0000 7fda8ffff640 1 -- 172.21.15.117:0/2364988671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda98109eb0 msgr2=0x7fda9819fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:13.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.273+0000 7fda8ffff640 1 --2- 172.21.15.117:0/2364988671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda98109eb0 0x7fda9819fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:13.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.273+0000 7fda8ffff640 1 -- 172.21.15.117:0/2364988671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda98102080 msgr2=0x7fda9819a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:13.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.273+0000 7fda8ffff640 1 --2- 172.21.15.117:0/2364988671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda98102080 0x7fda9819a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:13.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.273+0000 7fda8ffff640 1 -- 172.21.15.117:0/2364988671 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda981a0470 con 0x7fda98109560 2024-09-21T05:57:13.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.273+0000 7fda8ffff640 1 --2- 172.21.15.117:0/2364988671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda98109560 0x7fda9819aaa0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fda8000e9c0 tx=0x7fda8000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:13.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.273+0000 7fda95ffb640 1 -- 172.21.15.117:0/2364988671 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8000ccc0 con 0x7fda98109560 2024-09-21T05:57:13.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.273+0000 7fda95ffb640 1 -- 172.21.15.117:0/2364988671 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda800054d0 con 0x7fda98109560 2024-09-21T05:57:13.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.273+0000 7fda9e783640 1 -- 172.21.15.117:0/2364988671 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda981a0760 con 0x7fda98109560 2024-09-21T05:57:13.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.273+0000 7fda95ffb640 1 -- 172.21.15.117:0/2364988671 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda80010430 con 0x7fda98109560 2024-09-21T05:57:13.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.273+0000 7fda9e783640 1 -- 172.21.15.117:0/2364988671 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda981a6ec0 con 0x7fda98109560 2024-09-21T05:57:13.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.275+0000 7fda9e783640 1 -- 172.21.15.117:0/2364988671 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda9806b8f0 con 0x7fda98109560 2024-09-21T05:57:13.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.276+0000 7fda95ffb640 1 -- 172.21.15.117:0/2364988671 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 22) ==== 154530+0+0 (secure 0 0 0) 0x7fda80005000 con 0x7fda98109560 2024-09-21T05:57:13.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.277+0000 7fda95ffb640 1 --2- 172.21.15.117:0/2364988671 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fda840b6560 0x7fda840b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:13.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.277+0000 7fda97fff640 1 --2- 172.21.15.117:0/2364988671 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fda840b6560 0x7fda840b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:13.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.277+0000 7fda95ffb640 1 -- 172.21.15.117:0/2364988671 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fda800e7a70 con 0x7fda98109560 2024-09-21T05:57:13.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.277+0000 7fda97fff640 1 --2- 172.21.15.117:0/2364988671 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fda840b6560 0x7fda840b8a70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fda88031440 tx=0x7fda88005d50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:13.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.280+0000 7fda95ffb640 1 -- 172.21.15.117:0/2364988671 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fda80056710 con 0x7fda98109560 2024-09-21T05:57:13.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.459+0000 7fda9e783640 1 -- 172.21.15.117:0/2364988671 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda98102490 con 0x7fda98109560 2024-09-21T05:57:13.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.460+0000 7fda95ffb640 1 -- 172.21.15.117:0/2364988671 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+23066 (secure 0 0 0) 0x7fda98102490 con 0x7fda98109560 2024-09-21T05:57:13.462 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:57:13.463 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":76,"fsid":"ac8581a2-77dc-11ef-baf5-efdc52797490","created":"2024-09-21T05:46:47.021846+0000","modified":"2024-09-21T05:55:33.628948+0000","last_up_change":"2024-09-21T05:54:21.572849+0000","last_in_change":"2024-09-21T05:53:58.892597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:50:32.717389+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:55:10.604159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T05:55:12.581258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"68315502-fa6c-4a6e-a2d4-a7dde8071c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6803","nonce":3761478035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6805","nonce":3761478035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6809","nonce":3761478035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6807","nonce":3761478035}]},"public_addr":"172.21.15.117:6803/3761478035","cluster_addr":"172.21.15.117:6805/3761478035","heartbeat_back_addr":"172.21.15.117:6809/3761478035","heartbeat_front_addr":"172.21.15.117:6807/3761478035","state":["exists","up"]},{"osd":1,"uuid":"d8910a08-4ef3-49f2-a4db-e3588d85d12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6801","nonce":1737365588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6803","nonce":1737365588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6807","nonce":1737365588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6805","nonce":1737365588}]},"public_addr":"172.21.15.138:6801/1737365588","cluster_addr":"172.21.15.138:6803/1737365588","heartbeat_back_addr":"172.21.15.138:6807/1737365588","heartbeat_front_addr":"172.21.15.138:6805/1737365588","state":["exists","up"]},{"osd":2,"uuid":"94972378-a11f-4129-9e40-ad3fdec1d189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6801","nonce":3002534992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6803","nonce":3002534992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6807","nonce":3002534992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6805","nonce":3002534992}]},"public_addr":"172.21.15.192:6801/3002534992","cluster_addr":"172.21.15.192:6803/3002534992","heartbeat_back_addr":"172.21.15.192:6807/3002534992","heartbeat_front_addr":"172.21.15.192:6805/3002534992","state":["exists","up"]},{"osd":3,"uuid":"d4168c43-f0de-4445-945c-92d1cf270c33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6811","nonce":1222400293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6813","nonce":1222400293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6817","nonce":1222400293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6815","nonce":1222400293}]},"public_addr":"172.21.15.117:6811/1222400293","cluster_addr":"172.21.15.117:6813/1222400293","heartbeat_back_addr":"172.21.15.117:6817/1222400293","heartbeat_front_addr":"172.21.15.117:6815/1222400293","state":["exists","up"]},{"osd":4,"uuid":"28f1d7c0-8392-4a33-85c0-0de611cdd4b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6809","nonce":4294892870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6811","nonce":4294892870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6815","nonce":4294892870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6813","nonce":4294892870}]},"public_addr":"172.21.15.138:6809/4294892870","cluster_addr":"172.21.15.138:6811/4294892870","heartbeat_back_addr":"172.21.15.138:6815/4294892870","heartbeat_front_addr":"172.21.15.138:6813/4294892870","state":["exists","up"]},{"osd":5,"uuid":"ca902407-45e8-4bc5-bdbd-a729c1de8322","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6809","nonce":1776840019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6811","nonce":1776840019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6815","nonce":1776840019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6813","nonce":1776840019}]},"public_addr":"172.21.15.192:6809/1776840019","cluster_addr":"172.21.15.192:6811/1776840019","heartbeat_back_addr":"172.21.15.192:6815/1776840019","heartbeat_front_addr":"172.21.15.192:6813/1776840019","state":["exists","up"]},{"osd":6,"uuid":"6e507a91-ed09-4a33-bf0e-c4b593b3c5aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6819","nonce":1093341590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6821","nonce":1093341590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6825","nonce":1093341590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6823","nonce":1093341590}]},"public_addr":"172.21.15.117:6819/1093341590","cluster_addr":"172.21.15.117:6821/1093341590","heartbeat_back_addr":"172.21.15.117:6825/1093341590","heartbeat_front_addr":"172.21.15.117:6823/1093341590","state":["exists","up"]},{"osd":7,"uuid":"36a47dd7-67f7-4ebe-8759-3a8bb6bd2ea3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6817","nonce":1448179714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6819","nonce":1448179714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6823","nonce":1448179714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6821","nonce":1448179714}]},"public_addr":"172.21.15.138:6817/1448179714","cluster_addr":"172.21.15.138:6819/1448179714","heartbeat_back_addr":"172.21.15.138:6823/1448179714","heartbeat_front_addr":"172.21.15.138:6821/1448179714","state":["exists","up"]},{"osd":8,"uuid":"118e0b5d-2619-4512-94c5-8d07a0d3a28f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6817","nonce":3960902885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6819","nonce":3960902885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6823","nonce":3960902885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6821","nonce":3960902885}]},"public_addr":"172.21.15.192:6817/3960902885","cluster_addr":"172.21.15.192:6819/3960902885","heartbeat_back_addr":"172.21.15.192:6823/3960902885","heartbeat_front_addr":"172.21.15.192:6821/3960902885","state":["exists","up"]},{"osd":9,"uuid":"47543bc6-fa0b-4e72-9742-97dcda0b7f8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6827","nonce":3453855195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6829","nonce":3453855195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6833","nonce":3453855195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6831","nonce":3453855195}]},"public_addr":"172.21.15.117:6827/3453855195","cluster_addr":"172.21.15.117:6829/3453855195","heartbeat_back_addr":"172.21.15.117:6833/3453855195","heartbeat_front_addr":"172.21.15.117:6831/3453855195","state":["exists","up"]},{"osd":10,"uuid":"5e6708b4-7a85-4fa5-a03c-27d99f7d749f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6825","nonce":903392451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6827","nonce":903392451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6831","nonce":903392451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6829","nonce":903392451}]},"public_addr":"172.21.15.138:6825/903392451","cluster_addr":"172.21.15.138:6827/903392451","heartbeat_back_addr":"172.21.15.138:6831/903392451","heartbeat_front_addr":"172.21.15.138:6829/903392451","state":["exists","up"]},{"osd":11,"uuid":"24bc2ef7-e2b4-43c6-9e6c-43ae02390415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6825","nonce":899090479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6827","nonce":899090479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6831","nonce":899090479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6829","nonce":899090479}]},"public_addr":"172.21.15.192:6825/899090479","cluster_addr":"172.21.15.192:6827/899090479","heartbeat_back_addr":"172.21.15.192:6831/899090479","heartbeat_front_addr":"172.21.15.192:6829/899090479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:49:40.742217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:05.963655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:30.164495+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:56.598899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:51:21.940331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:51:47.777445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:52:12.869007+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:52:39.454054+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:03.569226+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:29.760863+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:55.788604+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:54:20.510184+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:0/2989394419":"2024-09-22T05:48:32.406654+0000","172.21.15.117:6801/1702676172":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/1766406059":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6800/2337036514":"2024-09-22T05:47:09.112435+0000","172.21.15.117:0/985798343":"2024-09-22T05:47:09.112435+0000","172.21.15.117:0/1083273222":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/473284565":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6801/2337036514":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6800/816635165":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/2642940197":"2024-09-22T05:48:32.406654+0000","172.21.15.117:6801/816635165":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/2225537433":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/3651584169":"2024-09-22T05:47:33.789146+0000","172.21.15.117:6800/1702676172":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/1284403882":"2024-09-22T05:47:33.789146+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:57:13.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.463+0000 7fda9e783640 1 -- 172.21.15.117:0/2364988671 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fda840b6560 msgr2=0x7fda840b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:13.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.463+0000 7fda9e783640 1 --2- 172.21.15.117:0/2364988671 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fda840b6560 0x7fda840b8a70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fda88031440 tx=0x7fda88005d50 comp rx=0 tx=0).stop 2024-09-21T05:57:13.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.463+0000 7fda9e783640 1 -- 172.21.15.117:0/2364988671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda98109560 msgr2=0x7fda9819aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:13.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.463+0000 7fda9e783640 1 --2- 172.21.15.117:0/2364988671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda98109560 0x7fda9819aaa0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fda8000e9c0 tx=0x7fda8000ee80 comp rx=0 tx=0).stop 2024-09-21T05:57:13.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.463+0000 7fda9e783640 1 -- 172.21.15.117:0/2364988671 shutdown_connections 2024-09-21T05:57:13.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.463+0000 7fda9e783640 1 --2- 172.21.15.117:0/2364988671 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fda840b6560 0x7fda840b8a70 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:13.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.463+0000 7fda9e783640 1 --2- 172.21.15.117:0/2364988671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda98109560 0x7fda9819aaa0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:13.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.463+0000 7fda9e783640 1 --2- 172.21.15.117:0/2364988671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda98109eb0 0x7fda9819fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:13.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.463+0000 7fda9e783640 1 --2- 172.21.15.117:0/2364988671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda98102080 0x7fda9819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:13.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.463+0000 7fda9e783640 1 -- 172.21.15.117:0/2364988671 >> 172.21.15.117:0/2364988671 conn(0x7fda98079ec0 msgr2=0x7fda981075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:13.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.463+0000 7fda9e783640 1 -- 172.21.15.117:0/2364988671 shutdown_connections 2024-09-21T05:57:13.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:13.463+0000 7fda9e783640 1 -- 172.21.15.117:0/2364988671 wait complete. 2024-09-21T05:57:13.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:13 smithi117 ceph-mon[30947]: pgmap v324: 97 pgs: 97 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-21T05:57:13.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:13 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2364988671' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T05:57:13.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:13 smithi192 ceph-mon[34700]: pgmap v324: 97 pgs: 97 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-21T05:57:13.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:13 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2364988671' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T05:57:14.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:13 smithi138 ceph-mon[34391]: pgmap v324: 97 pgs: 97 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-21T05:57:14.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:13 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2364988671' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T05:57:14.152 DEBUG:tasks.fwd_scrub:Yielding 2024-09-21T05:57:14.153 INFO:teuthology.run_tasks:Running task exec... 2024-09-21T05:57:14.162 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T05:57:14.162 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi117.front.sepia.ceph.com 2024-09-21T05:57:14.163 DEBUG:teuthology.orchestra.run.smithi117:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-21T05:57:14.163 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-21T05:57:14.163 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T05:57:14.163 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T05:57:14.280 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.278+0000 7f1af5aa4640 1 -- 172.21.15.117:0/3832494067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1af0108300 msgr2=0x7f1af0108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:14.280 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.278+0000 7f1af5aa4640 1 --2- 172.21.15.117:0/3832494067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1af0108300 0x7f1af0108710 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4009900 tx=0x7f1ae402a410 comp rx=0 tx=0).stop 2024-09-21T05:57:14.280 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.278+0000 7f1af5aa4640 1 -- 172.21.15.117:0/3832494067 shutdown_connections 2024-09-21T05:57:14.280 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.278+0000 7f1af5aa4640 1 --2- 172.21.15.117:0/3832494067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1af0108300 0x7f1af0108710 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.280 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.278+0000 7f1af5aa4640 1 --2- 172.21.15.117:0/3832494067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1af0108c50 0x7f1af010f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.278+0000 7f1af5aa4640 1 --2- 172.21.15.117:0/3832494067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1af01029d0 0x7f1af0102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.278+0000 7f1af5aa4640 1 -- 172.21.15.117:0/3832494067 >> 172.21.15.117:0/3832494067 conn(0x7f1af00fe410 msgr2=0x7f1af0100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:14.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.278+0000 7f1af5aa4640 1 -- 172.21.15.117:0/3832494067 shutdown_connections 2024-09-21T05:57:14.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.278+0000 7f1af5aa4640 1 -- 172.21.15.117:0/3832494067 wait complete. 2024-09-21T05:57:14.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.278+0000 7f1af5aa4640 1 Processor -- start 2024-09-21T05:57:14.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.279+0000 7f1af5aa4640 1 -- start start 2024-09-21T05:57:14.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.279+0000 7f1af5aa4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1af01029d0 0x7f1af0199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:14.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.279+0000 7f1af5aa4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1af0108300 0x7f1af01998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:14.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.279+0000 7f1af5aa4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1af0108c50 0x7f1af019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:14.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.279+0000 7f1af5aa4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1af0112910 con 0x7f1af0108c50 2024-09-21T05:57:14.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.279+0000 7f1af5aa4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1af0112790 con 0x7f1af0108300 2024-09-21T05:57:14.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.279+0000 7f1af5aa4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1af0112a90 con 0x7f1af01029d0 2024-09-21T05:57:14.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.279+0000 7f1aeffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1af0108c50 0x7f1af019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:14.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.279+0000 7f1aeffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1af0108c50 0x7f1af019eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60428/0 (socket says 172.21.15.117:60428) 2024-09-21T05:57:14.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.279+0000 7f1aeffff640 1 -- 172.21.15.117:0/2183670910 learned_addr learned my addr 172.21.15.117:0/2183670910 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:14.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.279+0000 7f1aef7fe640 1 --2- 172.21.15.117:0/2183670910 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1af01029d0 0x7f1af0199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:14.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.279+0000 7f1aeeffd640 1 --2- 172.21.15.117:0/2183670910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1af0108300 0x7f1af01998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:14.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.279+0000 7f1aeffff640 1 -- 172.21.15.117:0/2183670910 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1af01029d0 msgr2=0x7f1af0199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:14.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.279+0000 7f1aeffff640 1 --2- 172.21.15.117:0/2183670910 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1af01029d0 0x7f1af0199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.279+0000 7f1aeffff640 1 -- 172.21.15.117:0/2183670910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1af0108300 msgr2=0x7f1af01998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:14.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.279+0000 7f1aeffff640 1 --2- 172.21.15.117:0/2183670910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1af0108300 0x7f1af01998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.279+0000 7f1aeffff640 1 -- 172.21.15.117:0/2183670910 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af019f280 con 0x7f1af0108c50 2024-09-21T05:57:14.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.279+0000 7f1aeeffd640 1 --2- 172.21.15.117:0/2183670910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1af0108300 0x7f1af01998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:57:14.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.280+0000 7f1aeffff640 1 --2- 172.21.15.117:0/2183670910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1af0108c50 0x7f1af019eb80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1ae800eaa0 tx=0x7f1ae800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:14.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.280+0000 7f1aecff9640 1 -- 172.21.15.117:0/2183670910 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae800cdb0 con 0x7f1af0108c50 2024-09-21T05:57:14.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.280+0000 7f1aecff9640 1 -- 172.21.15.117:0/2183670910 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ae8005020 con 0x7f1af0108c50 2024-09-21T05:57:14.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.280+0000 7f1af5aa4640 1 -- 172.21.15.117:0/2183670910 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1af019f570 con 0x7f1af0108c50 2024-09-21T05:57:14.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.280+0000 7f1aecff9640 1 -- 172.21.15.117:0/2183670910 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae80116d0 con 0x7f1af0108c50 2024-09-21T05:57:14.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.280+0000 7f1af5aa4640 1 -- 172.21.15.117:0/2183670910 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1af01a5d00 con 0x7f1af0108c50 2024-09-21T05:57:14.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.281+0000 7f1af5aa4640 1 -- 172.21.15.117:0/2183670910 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1af0102de0 con 0x7f1af0108c50 2024-09-21T05:57:14.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.283+0000 7f1aecff9640 1 -- 172.21.15.117:0/2183670910 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 22) ==== 154530+0+0 (secure 0 0 0) 0x7f1ae80051d0 con 0x7f1af0108c50 2024-09-21T05:57:14.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.284+0000 7f1aecff9640 1 --2- 172.21.15.117:0/2183670910 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1ad80b6560 0x7f1ad80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:14.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.284+0000 7f1aef7fe640 1 --2- 172.21.15.117:0/2183670910 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1ad80b6560 0x7f1ad80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:14.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.284+0000 7f1aecff9640 1 -- 172.21.15.117:0/2183670910 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f1ae80e7920 con 0x7f1af0108c50 2024-09-21T05:57:14.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.284+0000 7f1aef7fe640 1 --2- 172.21.15.117:0/2183670910 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1ad80b6560 0x7f1ad80b8a70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0005d00 tx=0x7f1ae000b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:14.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.287+0000 7f1aecff9640 1 -- 172.21.15.117:0/2183670910 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1ae8016030 con 0x7f1af0108c50 2024-09-21T05:57:14.419 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:57:14.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.499+0000 7f1af5aa4640 1 -- 172.21.15.117:0/2183670910 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f1af0063800 con 0x7f1ad80b6560 2024-09-21T05:57:14.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.664+0000 7f1aecff9640 1 -- 172.21.15.117:0/2183670910 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1af0063800 con 0x7f1ad80b6560 2024-09-21T05:57:14.666 INFO:teuthology.orchestra.run.smithi117.stdout:Schedule set for path / 2024-09-21T05:57:14.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.667+0000 7f1af5aa4640 1 -- 172.21.15.117:0/2183670910 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1ad80b6560 msgr2=0x7f1ad80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:14.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.667+0000 7f1af5aa4640 1 --2- 172.21.15.117:0/2183670910 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1ad80b6560 0x7f1ad80b8a70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0005d00 tx=0x7f1ae000b410 comp rx=0 tx=0).stop 2024-09-21T05:57:14.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.667+0000 7f1af5aa4640 1 -- 172.21.15.117:0/2183670910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1af0108c50 msgr2=0x7f1af019eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:14.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.667+0000 7f1af5aa4640 1 --2- 172.21.15.117:0/2183670910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1af0108c50 0x7f1af019eb80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1ae800eaa0 tx=0x7f1ae800ef60 comp rx=0 tx=0).stop 2024-09-21T05:57:14.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.668+0000 7f1af5aa4640 1 -- 172.21.15.117:0/2183670910 shutdown_connections 2024-09-21T05:57:14.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.668+0000 7f1af5aa4640 1 --2- 172.21.15.117:0/2183670910 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1ad80b6560 0x7f1ad80b8a70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.668+0000 7f1af5aa4640 1 --2- 172.21.15.117:0/2183670910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1af0108c50 0x7f1af019eb80 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.668+0000 7f1af5aa4640 1 --2- 172.21.15.117:0/2183670910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1af0108300 0x7f1af01998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.668+0000 7f1af5aa4640 1 --2- 172.21.15.117:0/2183670910 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1af01029d0 0x7f1af0199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.668+0000 7f1af5aa4640 1 -- 172.21.15.117:0/2183670910 >> 172.21.15.117:0/2183670910 conn(0x7f1af00fe410 msgr2=0x7f1af0100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:14.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.668+0000 7f1af5aa4640 1 -- 172.21.15.117:0/2183670910 shutdown_connections 2024-09-21T05:57:14.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.668+0000 7f1af5aa4640 1 -- 172.21.15.117:0/2183670910 wait complete. 2024-09-21T05:57:14.682 DEBUG:teuthology.orchestra.run.smithi117:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-21T05:57:14.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.802+0000 7f23f27da640 1 -- 172.21.15.117:0/251579036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23ec1083e0 msgr2=0x7f23ec10ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:14.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.802+0000 7f23f27da640 1 --2- 172.21.15.117:0/251579036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23ec1083e0 0x7f23ec10ec10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f23e4009ca0 tx=0x7f23e402a8c0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.802+0000 7f23f27da640 1 -- 172.21.15.117:0/251579036 shutdown_connections 2024-09-21T05:57:14.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.802+0000 7f23f27da640 1 --2- 172.21.15.117:0/251579036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23ec1083e0 0x7f23ec10ec10 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.802+0000 7f23f27da640 1 --2- 172.21.15.117:0/251579036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23ec107a90 0x7f23ec107ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.802+0000 7f23f27da640 1 --2- 172.21.15.117:0/251579036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23ec102160 0x7f23ec102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.802+0000 7f23f27da640 1 -- 172.21.15.117:0/251579036 >> 172.21.15.117:0/251579036 conn(0x7f23ec079ec0 msgr2=0x7f23ec07a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:14.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.802+0000 7f23f27da640 1 -- 172.21.15.117:0/251579036 shutdown_connections 2024-09-21T05:57:14.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.802+0000 7f23f27da640 1 -- 172.21.15.117:0/251579036 wait complete. 2024-09-21T05:57:14.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.803+0000 7f23f27da640 1 Processor -- start 2024-09-21T05:57:14.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.803+0000 7f23f27da640 1 -- start start 2024-09-21T05:57:14.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.804+0000 7f23f27da640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23ec102160 0x7f23ec19b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:14.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.804+0000 7f23f27da640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23ec107a90 0x7f23ec19b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:14.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.804+0000 7f23f27da640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23ec1083e0 0x7f23ec19bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:14.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.804+0000 7f23f27da640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec1120b0 con 0x7f23ec102160 2024-09-21T05:57:14.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.804+0000 7f23f27da640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec111f30 con 0x7f23ec1083e0 2024-09-21T05:57:14.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.804+0000 7f23f27da640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec112230 con 0x7f23ec107a90 2024-09-21T05:57:14.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.804+0000 7f23ebfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23ec102160 0x7f23ec19b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:14.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.804+0000 7f23ebfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23ec102160 0x7f23ec19b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34698/0 (socket says 172.21.15.117:34698) 2024-09-21T05:57:14.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.804+0000 7f23ebfff640 1 -- 172.21.15.117:0/4155385568 learned_addr learned my addr 172.21.15.117:0/4155385568 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:14.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.804+0000 7f23eb7fe640 1 --2- 172.21.15.117:0/4155385568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23ec107a90 0x7f23ec19b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:14.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.804+0000 7f23f0d77640 1 --2- 172.21.15.117:0/4155385568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23ec1083e0 0x7f23ec19bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:14.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.804+0000 7f23ebfff640 1 -- 172.21.15.117:0/4155385568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23ec107a90 msgr2=0x7f23ec19b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:14.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.804+0000 7f23ebfff640 1 --2- 172.21.15.117:0/4155385568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23ec107a90 0x7f23ec19b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.804+0000 7f23ebfff640 1 -- 172.21.15.117:0/4155385568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23ec1083e0 msgr2=0x7f23ec19bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:14.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.804+0000 7f23ebfff640 1 --2- 172.21.15.117:0/4155385568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23ec1083e0 0x7f23ec19bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.804+0000 7f23ebfff640 1 -- 172.21.15.117:0/4155385568 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23ec1a6570 con 0x7f23ec102160 2024-09-21T05:57:14.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.804+0000 7f23f0d77640 1 --2- 172.21.15.117:0/4155385568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23ec1083e0 0x7f23ec19bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:57:14.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.804+0000 7f23ebfff640 1 --2- 172.21.15.117:0/4155385568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23ec102160 0x7f23ec19b2b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f23dc00e9f0 tx=0x7f23dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:14.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.804+0000 7f23e97fa640 1 -- 172.21.15.117:0/4155385568 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23dc00cd80 con 0x7f23ec102160 2024-09-21T05:57:14.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.804+0000 7f23e97fa640 1 -- 172.21.15.117:0/4155385568 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23dc0108e0 con 0x7f23ec102160 2024-09-21T05:57:14.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.804+0000 7f23f27da640 1 -- 172.21.15.117:0/4155385568 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23ec1a6860 con 0x7f23ec102160 2024-09-21T05:57:14.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.804+0000 7f23e97fa640 1 -- 172.21.15.117:0/4155385568 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23dc018980 con 0x7f23ec102160 2024-09-21T05:57:14.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.804+0000 7f23f27da640 1 -- 172.21.15.117:0/4155385568 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23ec1a6db0 con 0x7f23ec102160 2024-09-21T05:57:14.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.805+0000 7f23f27da640 1 -- 172.21.15.117:0/4155385568 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23ec102570 con 0x7f23ec102160 2024-09-21T05:57:14.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.807+0000 7f23e97fa640 1 -- 172.21.15.117:0/4155385568 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 23) ==== 154587+0+0 (secure 0 0 0) 0x7f23dc010430 con 0x7f23ec102160 2024-09-21T05:57:14.810 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:14 smithi117 ceph-mon[30947]: from='client.15522 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:57:14.810 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:14 smithi117 ceph-mon[30947]: pgmap v325: 97 pgs: 97 active+clean; 653 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-09-21T05:57:14.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.808+0000 7f23e97fa640 1 --2- 172.21.15.117:0/4155385568 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f23d40b65f0 0x7f23d40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:14.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.808+0000 7f23eb7fe640 1 --2- 172.21.15.117:0/4155385568 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f23d40b65f0 0x7f23d40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:14.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.808+0000 7f23e97fa640 1 -- 172.21.15.117:0/4155385568 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f23dc0e7e60 con 0x7f23ec102160 2024-09-21T05:57:14.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.808+0000 7f23eb7fe640 1 --2- 172.21.15.117:0/4155385568 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f23d40b65f0 0x7f23d40b8b00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f23ec06b550 tx=0x7f23e0005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:14.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.812+0000 7f23e97fa640 1 -- 172.21.15.117:0/4155385568 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f23dc0ae4f0 con 0x7f23ec102160 2024-09-21T05:57:14.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.903+0000 7fd8325e5640 1 -- 172.21.15.117:0/3546068117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd82c10de60 msgr2=0x7fd82c110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:14.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.903+0000 7fd8325e5640 1 --2- 172.21.15.117:0/3546068117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd82c10de60 0x7fd82c110330 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd824009ca0 tx=0x7fd82402a8c0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.903+0000 7fd8325e5640 1 -- 172.21.15.117:0/3546068117 shutdown_connections 2024-09-21T05:57:14.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.903+0000 7fd8325e5640 1 --2- 172.21.15.117:0/3546068117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd82c10de60 0x7fd82c110330 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.903+0000 7fd8325e5640 1 --2- 172.21.15.117:0/3546068117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd82c105380 0x7fd82c10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.903+0000 7fd8325e5640 1 --2- 172.21.15.117:0/3546068117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd82c104a30 0x7fd82c104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.903+0000 7fd8325e5640 1 -- 172.21.15.117:0/3546068117 >> 172.21.15.117:0/3546068117 conn(0x7fd82c0797b0 msgr2=0x7fd82c079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:14.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.904+0000 7fd8325e5640 1 -- 172.21.15.117:0/3546068117 shutdown_connections 2024-09-21T05:57:14.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.904+0000 7fd8325e5640 1 -- 172.21.15.117:0/3546068117 wait complete. 2024-09-21T05:57:14.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.904+0000 7fd8325e5640 1 Processor -- start 2024-09-21T05:57:14.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.905+0000 7fd8325e5640 1 -- start start 2024-09-21T05:57:14.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.905+0000 7fd8325e5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd82c104a30 0x7fd82c102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:14.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.905+0000 7fd8325e5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd82c105380 0x7fd82c102cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:14.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.905+0000 7fd8325e5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd82c10de60 0x7fd82c1031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:14.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.905+0000 7fd8325e5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd82c113ac0 con 0x7fd82c104a30 2024-09-21T05:57:14.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.905+0000 7fd8325e5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd82c113940 con 0x7fd82c105380 2024-09-21T05:57:14.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.905+0000 7fd8325e5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd82c113c40 con 0x7fd82c10de60 2024-09-21T05:57:14.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.905+0000 7fd82bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd82c104a30 0x7fd82c102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:14.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.905+0000 7fd82bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd82c104a30 0x7fd82c102770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34712/0 (socket says 172.21.15.117:34712) 2024-09-21T05:57:14.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.905+0000 7fd82bfff640 1 -- 172.21.15.117:0/3875142444 learned_addr learned my addr 172.21.15.117:0/3875142444 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:14.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.905+0000 7fd82b7fe640 1 --2- 172.21.15.117:0/3875142444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd82c105380 0x7fd82c102cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:14.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.906+0000 7fd830b82640 1 --2- 172.21.15.117:0/3875142444 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd82c10de60 0x7fd82c1031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:14.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.906+0000 7fd82bfff640 1 -- 172.21.15.117:0/3875142444 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd82c10de60 msgr2=0x7fd82c1031f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:14.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.906+0000 7fd82bfff640 1 --2- 172.21.15.117:0/3875142444 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd82c10de60 0x7fd82c1031f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.906+0000 7fd82bfff640 1 -- 172.21.15.117:0/3875142444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd82c105380 msgr2=0x7fd82c102cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:14.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.906+0000 7fd82bfff640 1 --2- 172.21.15.117:0/3875142444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd82c105380 0x7fd82c102cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.906+0000 7fd82bfff640 1 -- 172.21.15.117:0/3875142444 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd82c1a8730 con 0x7fd82c104a30 2024-09-21T05:57:14.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.906+0000 7fd82b7fe640 1 --2- 172.21.15.117:0/3875142444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd82c105380 0x7fd82c102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:57:14.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.906+0000 7fd82bfff640 1 --2- 172.21.15.117:0/3875142444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd82c104a30 0x7fd82c102770 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd81c00c970 tx=0x7fd81c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:14.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.906+0000 7fd8297fa640 1 -- 172.21.15.117:0/3875142444 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81c007bb0 con 0x7fd82c104a30 2024-09-21T05:57:14.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.906+0000 7fd8325e5640 1 -- 172.21.15.117:0/3875142444 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd82c1a8a20 con 0x7fd82c104a30 2024-09-21T05:57:14.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.906+0000 7fd8297fa640 1 -- 172.21.15.117:0/3875142444 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd81c007d50 con 0x7fd82c104a30 2024-09-21T05:57:14.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.906+0000 7fd8297fa640 1 -- 172.21.15.117:0/3875142444 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81c00f450 con 0x7fd82c104a30 2024-09-21T05:57:14.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.906+0000 7fd8325e5640 1 -- 172.21.15.117:0/3875142444 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd82c1a8f10 con 0x7fd82c104a30 2024-09-21T05:57:14.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.909+0000 7fd8297fa640 1 -- 172.21.15.117:0/3875142444 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 23) ==== 154587+0+0 (secure 0 0 0) 0x7fd81c004f20 con 0x7fd82c104a30 2024-09-21T05:57:14.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.910+0000 7fd8297fa640 1 --2- 172.21.15.117:0/3875142444 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd8140b65f0 0x7fd8140b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:14.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.910+0000 7fd82b7fe640 1 --2- 172.21.15.117:0/3875142444 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd8140b65f0 0x7fd8140b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:14.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.910+0000 7fd8297fa640 1 -- 172.21.15.117:0/3875142444 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fd81c0e72b0 con 0x7fd82c104a30 2024-09-21T05:57:14.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.910+0000 7fd838b59740 1 Processor -- start 2024-09-21T05:57:14.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.910+0000 7fd838b59740 1 -- start start 2024-09-21T05:57:14.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.910+0000 7fd82b7fe640 1 --2- 172.21.15.117:0/3875142444 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd8140b65f0 0x7fd8140b8b00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd82c10fa50 tx=0x7fd820005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:14.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.910+0000 7fd838b59740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56091ce04350 0x56091d1ab430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:14.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.910+0000 7fd838b59740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56091cddb490 0x56091d1ab970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:14.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.910+0000 7fd838b59740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56091cde3670 0x56091d1abeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:14.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.910+0000 7fd838b59740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56091d121e60 con 0x56091cde3670 2024-09-21T05:57:14.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.910+0000 7fd838b59740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56091d111290 con 0x56091cddb490 2024-09-21T05:57:14.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.910+0000 7fd838b59740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56091d1150a0 con 0x56091ce04350 2024-09-21T05:57:14.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.911+0000 7fd830b82640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56091cde3670 0x56091d1abeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:14.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.911+0000 7fd830b82640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56091cde3670 0x56091d1abeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34728/0 (socket says 172.21.15.117:34728) 2024-09-21T05:57:14.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.911+0000 7fd830b82640 1 -- 172.21.15.117:0/438529126 learned_addr learned my addr 172.21.15.117:0/438529126 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:14.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.911+0000 7fd82b7fe640 1 --2- 172.21.15.117:0/438529126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56091cddb490 0x56091d1ab970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:14.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.911+0000 7fd82bfff640 1 --2- 172.21.15.117:0/438529126 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56091ce04350 0x56091d1ab430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:14.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.911+0000 7fd830b82640 1 -- 172.21.15.117:0/438529126 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56091ce04350 msgr2=0x56091d1ab430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:14.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.911+0000 7fd830b82640 1 --2- 172.21.15.117:0/438529126 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56091ce04350 0x56091d1ab430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.911+0000 7fd830b82640 1 -- 172.21.15.117:0/438529126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56091cddb490 msgr2=0x56091d1ab970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:14.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.911+0000 7fd830b82640 1 --2- 172.21.15.117:0/438529126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56091cddb490 0x56091d1ab970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.911+0000 7fd830b82640 1 -- 172.21.15.117:0/438529126 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56091ce6f930 con 0x56091cde3670 2024-09-21T05:57:14.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.911+0000 7fd830b82640 1 --2- 172.21.15.117:0/438529126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56091cde3670 0x56091d1abeb0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd824009880 tx=0x7fd8240098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7a8551eda6f259 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:14.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.911+0000 7fd8127fc640 1 -- 172.21.15.117:0/438529126 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd824031d70 con 0x56091cde3670 2024-09-21T05:57:14.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.911+0000 7fd8127fc640 1 -- 172.21.15.117:0/438529126 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd824036040 con 0x56091cde3670 2024-09-21T05:57:14.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.911+0000 7fd8127fc640 1 -- 172.21.15.117:0/438529126 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd824004240 con 0x56091cde3670 2024-09-21T05:57:14.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.912+0000 7fd838b59740 1 -- 172.21.15.117:0/438529126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56091cde3670 msgr2=0x56091d1abeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:14.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.912+0000 7fd838b59740 1 --2- 172.21.15.117:0/438529126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56091cde3670 0x56091d1abeb0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd824009880 tx=0x7fd8240098b0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.912+0000 7fd838b59740 1 -- 172.21.15.117:0/438529126 shutdown_connections 2024-09-21T05:57:14.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.912+0000 7fd838b59740 1 --2- 172.21.15.117:0/438529126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56091cde3670 0x56091d1abeb0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.912+0000 7fd838b59740 1 --2- 172.21.15.117:0/438529126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56091cddb490 0x56091d1ab970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.912+0000 7fd838b59740 1 --2- 172.21.15.117:0/438529126 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56091ce04350 0x56091d1ab430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.912+0000 7fd838b59740 1 -- 172.21.15.117:0/438529126 >> 172.21.15.117:0/438529126 conn(0x56091cdf8740 msgr2=0x56091d1a9320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:14.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.912+0000 7fd838b59740 1 -- 172.21.15.117:0/438529126 shutdown_connections 2024-09-21T05:57:14.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.912+0000 7fd838b59740 1 -- 172.21.15.117:0/438529126 wait complete. 2024-09-21T05:57:14.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.913+0000 7fd838b59740 1 -- start start 2024-09-21T05:57:14.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.913+0000 7fd838b59740 1 Processor -- start 2024-09-21T05:57:14.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.913+0000 7fd838b59740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56091cddb490 0x56091d1b6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:14.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.913+0000 7fd838b59740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56091cde3670 0x56091d1b7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:14.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.913+0000 7fd838b59740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56091ce04350 0x56091d1c2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:14.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.913+0000 7fd838b59740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56091cf72890 con 0x56091cde3670 2024-09-21T05:57:14.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.913+0000 7fd838b59740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56091d0abd40 con 0x56091ce04350 2024-09-21T05:57:14.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.913+0000 7fd838b59740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56091d1b9bb0 con 0x56091cddb490 2024-09-21T05:57:14.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.913+0000 7fd82b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56091cde3670 0x56091d1b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:14.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.913+0000 7fd82b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56091cde3670 0x56091d1b7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34736/0 (socket says 172.21.15.117:34736) 2024-09-21T05:57:14.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.913+0000 7fd82b7fe640 1 -- 172.21.15.117:0/1594911257 learned_addr learned my addr 172.21.15.117:0/1594911257 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:14.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.913+0000 7fd82bfff640 1 --2- 172.21.15.117:0/1594911257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56091cddb490 0x56091d1b6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:14.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.913+0000 7fd838b59740 10 client.0 _ll_register_callbacks cb 0x56091d10fc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:57:14.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.914+0000 7fd830b82640 1 --2- 172.21.15.117:0/1594911257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56091ce04350 0x56091d1c2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:14.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.914+0000 7fd82b7fe640 1 -- 172.21.15.117:0/1594911257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56091cddb490 msgr2=0x56091d1b6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:14.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.914+0000 7fd82b7fe640 1 --2- 172.21.15.117:0/1594911257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56091cddb490 0x56091d1b6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.914+0000 7fd82b7fe640 1 -- 172.21.15.117:0/1594911257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56091ce04350 msgr2=0x56091d1c2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:14.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.914+0000 7fd82b7fe640 1 --2- 172.21.15.117:0/1594911257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56091ce04350 0x56091d1c2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.914+0000 7fd82b7fe640 1 -- 172.21.15.117:0/1594911257 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56091d1c34c0 con 0x56091cde3670 2024-09-21T05:57:14.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.914+0000 7fd82bfff640 1 --2- 172.21.15.117:0/1594911257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56091cddb490 0x56091d1b6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:57:14.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.914+0000 7fd82b7fe640 1 --2- 172.21.15.117:0/1594911257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56091cde3670 0x56091d1b7530 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd820003970 tx=0x7fd820004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d10ac3c0ef63576 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:14.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.914+0000 7fd80b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:57:14.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.914+0000 7fd80b7fe640 1 -- 172.21.15.117:0/1594911257 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82000fe10 con 0x56091cde3670 2024-09-21T05:57:14.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.914+0000 7fd838b59740 1 -- 172.21.15.117:0/1594911257 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd824036040 con 0x56091cde3670 2024-09-21T05:57:14.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.914+0000 7fd80b7fe640 1 -- 172.21.15.117:0/1594911257 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fd820028440 con 0x56091cde3670 2024-09-21T05:57:14.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.914+0000 7fd80b7fe640 1 -- 172.21.15.117:0/1594911257 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd820028640 con 0x56091cde3670 2024-09-21T05:57:14.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.914+0000 7fd838b59740 10 client.15546 fetch_fsmap learned FSMap version 27 2024-09-21T05:57:14.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.914+0000 7fd838b59740 1 -- 172.21.15.117:0/1594911257 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56091d1c5e10 con 0x56091cde3670 2024-09-21T05:57:14.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.915+0000 7fd80b7fe640 1 -- 172.21.15.117:0/1594911257 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd824036040 con 0x56091cde3670 2024-09-21T05:57:14.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.915+0000 7fd80b7fe640 1 -- 172.21.15.117:0/1594911257 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd820026020 con 0x56091cde3670 2024-09-21T05:57:14.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.915+0000 7fd838b59740 10 client.15546 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:57:14.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.915+0000 7fd838b59740 10 client.15546 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T05:57:14.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.915+0000 7fd838b59740 20 client.15546 populate_metadata read hostname 'smithi117' 2024-09-21T05:57:14.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.915+0000 7fd838b59740 1 --2- 172.21.15.117:0/1594911257 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56091cdc2cd0 0x56091d1c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:14.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.915+0000 7fd838b59740 4 client.15546 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:57:14.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.915+0000 7fd838b59740 1 -- 172.21.15.117:0/1594911257 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56091d123db0 con 0x56091cdc2cd0 2024-09-21T05:57:14.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.915+0000 7fd82bfff640 1 --2- 172.21.15.117:0/1594911257 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56091cdc2cd0 0x56091d1c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:14.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.916+0000 7fd80b7fe640 10 client.15546 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:57:14.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.916+0000 7fd82bfff640 1 --2- 172.21.15.117:0/1594911257 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56091cdc2cd0 0x56091d1c93b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e188713d8366edc1 server_cookie=5655d951e30726dc in_seq=0 out_seq=0 2024-09-21T05:57:14.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.916+0000 7fd80b7fe640 1 -- 172.21.15.117:0/1594911257 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56091d123db0 con 0x56091cdc2cd0 2024-09-21T05:57:14.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.916+0000 7fd80b7fe640 10 client.15546 handle_command_reply: tid=0 2024-09-21T05:57:14.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.916+0000 7fd838b59740 1 client.15546 shutdown 2024-09-21T05:57:14.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.916+0000 7fd838b59740 1 -- 172.21.15.117:0/1594911257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56091cde3670 msgr2=0x56091d1b7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:14.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.916+0000 7fd838b59740 1 --2- 172.21.15.117:0/1594911257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56091cde3670 0x56091d1b7530 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd820003970 tx=0x7fd820004730 comp rx=0 tx=0).stop 2024-09-21T05:57:14.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.916+0000 7fd838b59740 1 -- 172.21.15.117:0/1594911257 shutdown_connections 2024-09-21T05:57:14.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.916+0000 7fd838b59740 1 --2- 172.21.15.117:0/1594911257 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56091cdc2cd0 0x56091d1c93b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.916+0000 7fd838b59740 1 --2- 172.21.15.117:0/1594911257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56091cde3670 0x56091d1b7530 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.916+0000 7fd838b59740 1 --2- 172.21.15.117:0/1594911257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56091ce04350 0x56091d1c2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.916+0000 7fd838b59740 1 --2- 172.21.15.117:0/1594911257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56091cddb490 0x56091d1b6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.916+0000 7fd838b59740 1 -- 172.21.15.117:0/1594911257 >> 172.21.15.117:0/1594911257 conn(0x56091cdf8740 msgr2=0x56091d1ae5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:14.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.917+0000 7fd80b7fe640 0 client.15546 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:57:14.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.917+0000 7fd838b59740 1 -- 172.21.15.117:0/1594911257 shutdown_connections 2024-09-21T05:57:14.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.917+0000 7fd838b59740 1 -- 172.21.15.117:0/1594911257 wait complete. 2024-09-21T05:57:14.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.917+0000 7fd838b59740 20 client.15546 trim_cache size 0 max 16384 2024-09-21T05:57:14.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.934+0000 7fd838b59740 1 Processor -- start 2024-09-21T05:57:14.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.934+0000 7fd838b59740 1 -- start start 2024-09-21T05:57:14.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.934+0000 7fd838b59740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56091cddb490 0x56091d1b1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:14.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.934+0000 7fd838b59740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56091cde3670 0x56091d1c2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:14.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.934+0000 7fd838b59740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56091ce04350 0x56091d1b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:14.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.934+0000 7fd838b59740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd82c1b17b0 con 0x56091cddb490 2024-09-21T05:57:14.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.934+0000 7fd838b59740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd82c1b32f0 con 0x56091ce04350 2024-09-21T05:57:14.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.934+0000 7fd838b59740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd82c1b1e80 con 0x56091cde3670 2024-09-21T05:57:14.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.934+0000 7fd82bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56091cddb490 0x56091d1b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:14.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.934+0000 7fd82bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56091cddb490 0x56091d1b1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34750/0 (socket says 172.21.15.117:34750) 2024-09-21T05:57:14.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.934+0000 7fd82bfff640 1 -- 172.21.15.117:0/3520790713 learned_addr learned my addr 172.21.15.117:0/3520790713 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:14.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.935+0000 7fd830b82640 1 --2- 172.21.15.117:0/3520790713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56091ce04350 0x56091d1b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:14.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.935+0000 7fd82b7fe640 1 --2- 172.21.15.117:0/3520790713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56091cde3670 0x56091d1c2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:14.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.935+0000 7fd82bfff640 1 -- 172.21.15.117:0/3520790713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56091cde3670 msgr2=0x56091d1c2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:14.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.935+0000 7fd82bfff640 1 --2- 172.21.15.117:0/3520790713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56091cde3670 0x56091d1c2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.935+0000 7fd82bfff640 1 -- 172.21.15.117:0/3520790713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56091ce04350 msgr2=0x56091d1b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:14.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.935+0000 7fd82bfff640 1 --2- 172.21.15.117:0/3520790713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56091ce04350 0x56091d1b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.935+0000 7fd82bfff640 1 -- 172.21.15.117:0/3520790713 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56091d1a3520 con 0x56091cddb490 2024-09-21T05:57:14.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.935+0000 7fd82b7fe640 1 --2- 172.21.15.117:0/3520790713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56091cde3670 0x56091d1c2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:57:14.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.935+0000 7fd82bfff640 1 --2- 172.21.15.117:0/3520790713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56091cddb490 0x56091d1b1550 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd81c0e5140 tx=0x7fd81c01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d65f7466d0456c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:14.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.935+0000 7fd811ffb640 1 -- 172.21.15.117:0/3520790713 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81c01d040 con 0x56091cddb490 2024-09-21T05:57:14.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.935+0000 7fd811ffb640 1 -- 172.21.15.117:0/3520790713 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd81c0ad940 con 0x56091cddb490 2024-09-21T05:57:14.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.935+0000 7fd811ffb640 1 -- 172.21.15.117:0/3520790713 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81c0e4f00 con 0x56091cddb490 2024-09-21T05:57:14.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.936+0000 7fd838b59740 1 -- 172.21.15.117:0/3520790713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56091cddb490 msgr2=0x56091d1b1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:14.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.936+0000 7fd838b59740 1 --2- 172.21.15.117:0/3520790713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56091cddb490 0x56091d1b1550 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd81c0e5140 tx=0x7fd81c01a040 comp rx=0 tx=0).stop 2024-09-21T05:57:14.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.936+0000 7fd838b59740 1 -- 172.21.15.117:0/3520790713 shutdown_connections 2024-09-21T05:57:14.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.936+0000 7fd838b59740 1 --2- 172.21.15.117:0/3520790713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56091ce04350 0x56091d1b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.936+0000 7fd838b59740 1 --2- 172.21.15.117:0/3520790713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56091cde3670 0x56091d1c2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.936+0000 7fd838b59740 1 --2- 172.21.15.117:0/3520790713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56091cddb490 0x56091d1b1550 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.936+0000 7fd838b59740 1 -- 172.21.15.117:0/3520790713 >> 172.21.15.117:0/3520790713 conn(0x56091cdf8740 msgr2=0x56091d1b5670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:14.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.936+0000 7fd838b59740 1 -- 172.21.15.117:0/3520790713 shutdown_connections 2024-09-21T05:57:14.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.936+0000 7fd838b59740 1 -- 172.21.15.117:0/3520790713 wait complete. 2024-09-21T05:57:14.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.936+0000 7fd838b59740 1 -- start start 2024-09-21T05:57:14.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.937+0000 7fd838b59740 1 Processor -- start 2024-09-21T05:57:14.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.937+0000 7fd838b59740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56091cddb490 0x56091d1baec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:14.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.937+0000 7fd838b59740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56091cde3670 0x56091d1bb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:14.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.937+0000 7fd838b59740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56091ce04350 0x56091d1bb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:14.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.937+0000 7fd838b59740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd82c1b2c20 con 0x56091cde3670 2024-09-21T05:57:14.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.937+0000 7fd838b59740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd82c1b2550 con 0x56091cddb490 2024-09-21T05:57:14.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.937+0000 7fd838b59740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd82c1af790 con 0x56091ce04350 2024-09-21T05:57:14.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.937+0000 7fd838b59740 10 client.0 _ll_register_callbacks cb 0x7fd8200040d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:57:14.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.937+0000 7fd82b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56091cde3670 0x56091d1bb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:14.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.937+0000 7fd82b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56091cde3670 0x56091d1bb400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34766/0 (socket says 172.21.15.117:34766) 2024-09-21T05:57:14.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.937+0000 7fd82b7fe640 1 -- 172.21.15.117:0/2217927961 learned_addr learned my addr 172.21.15.117:0/2217927961 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:14.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.937+0000 7fd82bfff640 1 --2- 172.21.15.117:0/2217927961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56091cddb490 0x56091d1baec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:14.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.937+0000 7fd830b82640 1 --2- 172.21.15.117:0/2217927961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56091ce04350 0x56091d1bb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:14.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.937+0000 7fd82b7fe640 1 -- 172.21.15.117:0/2217927961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56091ce04350 msgr2=0x56091d1bb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:14.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.937+0000 7fd82b7fe640 1 --2- 172.21.15.117:0/2217927961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56091ce04350 0x56091d1bb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.937+0000 7fd82b7fe640 1 -- 172.21.15.117:0/2217927961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56091cddb490 msgr2=0x56091d1baec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:14.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.937+0000 7fd82b7fe640 1 --2- 172.21.15.117:0/2217927961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56091cddb490 0x56091d1baec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.937+0000 7fd82b7fe640 1 -- 172.21.15.117:0/2217927961 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56091d232e60 con 0x56091cde3670 2024-09-21T05:57:14.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.937+0000 7fd82bfff640 1 --2- 172.21.15.117:0/2217927961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56091cddb490 0x56091d1baec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:57:14.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.938+0000 7fd82b7fe640 1 --2- 172.21.15.117:0/2217927961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56091cde3670 0x56091d1bb400 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd820009e80 tx=0x7fd820011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0c46afc1baed90 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:14.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.938+0000 7fd80b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:57:14.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.938+0000 7fd80b7fe640 1 -- 172.21.15.117:0/2217927961 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82000fc10 con 0x56091cde3670 2024-09-21T05:57:14.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.938+0000 7fd80b7fe640 1 -- 172.21.15.117:0/2217927961 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fd82000fdb0 con 0x56091cde3670 2024-09-21T05:57:14.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.938+0000 7fd838b59740 1 -- 172.21.15.117:0/2217927961 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd81c0ad940 con 0x56091cde3670 2024-09-21T05:57:14.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.938+0000 7fd80b7fe640 1 -- 172.21.15.117:0/2217927961 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd820041050 con 0x56091cde3670 2024-09-21T05:57:14.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.938+0000 7fd838b59740 10 client.15552 fetch_fsmap learned FSMap version 27 2024-09-21T05:57:14.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.938+0000 7fd838b59740 1 -- 172.21.15.117:0/2217927961 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56091d2340f0 con 0x56091cde3670 2024-09-21T05:57:14.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.938+0000 7fd80b7fe640 1 -- 172.21.15.117:0/2217927961 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd81c0ad940 con 0x56091cde3670 2024-09-21T05:57:14.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.938+0000 7fd80b7fe640 1 -- 172.21.15.117:0/2217927961 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd820026040 con 0x56091cde3670 2024-09-21T05:57:14.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.938+0000 7fd838b59740 10 client.15552 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:57:14.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.938+0000 7fd838b59740 10 client.15552 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T05:57:14.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.938+0000 7fd838b59740 20 client.15552 populate_metadata read hostname 'smithi117' 2024-09-21T05:57:14.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.938+0000 7fd838b59740 1 --2- 172.21.15.117:0/2217927961 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56091cd98080 0x56091d236900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:14.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.938+0000 7fd838b59740 4 client.15552 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T05:57:14.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.939+0000 7fd838b59740 1 -- 172.21.15.117:0/2217927961 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56091d11f860 con 0x56091cd98080 2024-09-21T05:57:14.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.939+0000 7fd82bfff640 1 --2- 172.21.15.117:0/2217927961 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56091cd98080 0x56091d236900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:14.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.939+0000 7fd80b7fe640 10 client.15552 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:57:14.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.939+0000 7fd82bfff640 1 --2- 172.21.15.117:0/2217927961 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56091cd98080 0x56091d236900 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e81ffb2576eaf3b server_cookie=f911ee7e2c3d7db0 in_seq=0 out_seq=0 2024-09-21T05:57:14.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.941+0000 7fd80b7fe640 1 -- 172.21.15.117:0/2217927961 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56091d11f860 con 0x56091cd98080 2024-09-21T05:57:14.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.941+0000 7fd80b7fe640 10 client.15552 handle_command_reply: tid=0 2024-09-21T05:57:14.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.941+0000 7fd838b59740 1 client.15552 shutdown 2024-09-21T05:57:14.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.941+0000 7fd838b59740 1 -- 172.21.15.117:0/2217927961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56091cde3670 msgr2=0x56091d1bb400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:14.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.941+0000 7fd838b59740 1 --2- 172.21.15.117:0/2217927961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56091cde3670 0x56091d1bb400 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd820009e80 tx=0x7fd820011cf0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.941+0000 7fd838b59740 1 -- 172.21.15.117:0/2217927961 shutdown_connections 2024-09-21T05:57:14.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.941+0000 7fd838b59740 1 --2- 172.21.15.117:0/2217927961 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56091cd98080 0x56091d236900 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.941+0000 7fd838b59740 1 --2- 172.21.15.117:0/2217927961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56091cde3670 0x56091d1bb400 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.941+0000 7fd838b59740 1 --2- 172.21.15.117:0/2217927961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56091ce04350 0x56091d1bb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.941+0000 7fd80b7fe640 0 client.15552 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:57:14.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.941+0000 7fd838b59740 1 --2- 172.21.15.117:0/2217927961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56091cddb490 0x56091d1baec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.941+0000 7fd838b59740 1 -- 172.21.15.117:0/2217927961 >> 172.21.15.117:0/2217927961 conn(0x56091cdf8740 msgr2=0x56091d1ac460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:14.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.941+0000 7fd838b59740 1 -- 172.21.15.117:0/2217927961 shutdown_connections 2024-09-21T05:57:14.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.941+0000 7fd838b59740 1 -- 172.21.15.117:0/2217927961 wait complete. 2024-09-21T05:57:14.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.942+0000 7fd838b59740 20 client.15552 trim_cache size 0 max 16384 2024-09-21T05:57:14.946 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T05:57:14.946 INFO:teuthology.orchestra.run.smithi117.stdout: "return_code": 0, 2024-09-21T05:57:14.946 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_tag": "b30bfff0-0f6b-4e30-8467-ff7db421eba6", 2024-09-21T05:57:14.946 INFO:teuthology.orchestra.run.smithi117.stdout: "mode": "asynchronous" 2024-09-21T05:57:14.947 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T05:57:14.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.942+0000 7fd8325e5640 1 -- 172.21.15.117:0/3875142444 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd8140b65f0 msgr2=0x7fd8140b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:14.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.942+0000 7fd8325e5640 1 --2- 172.21.15.117:0/3875142444 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd8140b65f0 0x7fd8140b8b00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd82c10fa50 tx=0x7fd820005f90 comp rx=0 tx=0).stop 2024-09-21T05:57:14.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.942+0000 7fd8325e5640 1 -- 172.21.15.117:0/3875142444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd82c104a30 msgr2=0x7fd82c102770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:14.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.942+0000 7fd8325e5640 1 --2- 172.21.15.117:0/3875142444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd82c104a30 0x7fd82c102770 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd81c00c970 tx=0x7fd81c00ce30 comp rx=0 tx=0).stop 2024-09-21T05:57:14.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.942+0000 7fd8325e5640 1 -- 172.21.15.117:0/3875142444 shutdown_connections 2024-09-21T05:57:14.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.942+0000 7fd8325e5640 1 --2- 172.21.15.117:0/3875142444 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd8140b65f0 0x7fd8140b8b00 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.942+0000 7fd8325e5640 1 --2- 172.21.15.117:0/3875142444 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd82c10de60 0x7fd82c1031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.942+0000 7fd8325e5640 1 --2- 172.21.15.117:0/3875142444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd82c105380 0x7fd82c102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.942+0000 7fd8325e5640 1 --2- 172.21.15.117:0/3875142444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd82c104a30 0x7fd82c102770 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:14.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.942+0000 7fd8325e5640 1 -- 172.21.15.117:0/3875142444 >> 172.21.15.117:0/3875142444 conn(0x7fd82c0797b0 msgr2=0x7fd82c0ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:14.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.942+0000 7fd8325e5640 1 -- 172.21.15.117:0/3875142444 shutdown_connections 2024-09-21T05:57:14.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:14.942+0000 7fd8325e5640 1 -- 172.21.15.117:0/3875142444 wait complete. 2024-09-21T05:57:14.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:14 smithi192 ceph-mon[34700]: from='client.15522 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:57:14.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:14 smithi192 ceph-mon[34700]: pgmap v325: 97 pgs: 97 active+clean; 653 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-09-21T05:57:15.016 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:14 smithi138 ceph-mon[34391]: from='client.15522 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:57:15.016 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:14 smithi138 ceph-mon[34391]: pgmap v325: 97 pgs: 97 active+clean; 653 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-09-21T05:57:15.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.032+0000 7f23f27da640 1 -- 172.21.15.117:0/4155385568 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f23ec101590 con 0x7f23d40b65f0 2024-09-21T05:57:15.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.096+0000 7f23e97fa640 1 -- 172.21.15.117:0/4155385568 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f23ec101590 con 0x7f23d40b65f0 2024-09-21T05:57:15.098 INFO:teuthology.orchestra.run.smithi117.stdout:Retention added to path / 2024-09-21T05:57:15.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.099+0000 7f23f27da640 1 -- 172.21.15.117:0/4155385568 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f23d40b65f0 msgr2=0x7f23d40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:15.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.099+0000 7f23f27da640 1 --2- 172.21.15.117:0/4155385568 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f23d40b65f0 0x7f23d40b8b00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f23ec06b550 tx=0x7f23e0005f90 comp rx=0 tx=0).stop 2024-09-21T05:57:15.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.099+0000 7f23f27da640 1 -- 172.21.15.117:0/4155385568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23ec102160 msgr2=0x7f23ec19b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:15.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.099+0000 7f23f27da640 1 --2- 172.21.15.117:0/4155385568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23ec102160 0x7f23ec19b2b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f23dc00e9f0 tx=0x7f23dc00eeb0 comp rx=0 tx=0).stop 2024-09-21T05:57:15.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.099+0000 7f23f27da640 1 -- 172.21.15.117:0/4155385568 shutdown_connections 2024-09-21T05:57:15.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.099+0000 7f23f27da640 1 --2- 172.21.15.117:0/4155385568 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f23d40b65f0 0x7f23d40b8b00 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:15.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.099+0000 7f23f27da640 1 --2- 172.21.15.117:0/4155385568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23ec1083e0 0x7f23ec19bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:15.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.099+0000 7f23f27da640 1 --2- 172.21.15.117:0/4155385568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23ec107a90 0x7f23ec19b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:15.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.099+0000 7f23f27da640 1 --2- 172.21.15.117:0/4155385568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23ec102160 0x7f23ec19b2b0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:15.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.099+0000 7f23f27da640 1 -- 172.21.15.117:0/4155385568 >> 172.21.15.117:0/4155385568 conn(0x7f23ec079ec0 msgr2=0x7f23ec100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:15.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.099+0000 7f23f27da640 1 -- 172.21.15.117:0/4155385568 shutdown_connections 2024-09-21T05:57:15.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.099+0000 7f23f27da640 1 -- 172.21.15.117:0/4155385568 wait complete. 2024-09-21T05:57:15.112 DEBUG:teuthology.orchestra.run.smithi117:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-21T05:57:15.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.223+0000 7f386a694640 1 -- 172.21.15.117:0/2227908594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3864101f00 msgr2=0x7f3864102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:15.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.223+0000 7f386a694640 1 --2- 172.21.15.117:0/2227908594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3864101f00 0x7f3864102310 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3854009900 tx=0x7f385402a410 comp rx=0 tx=0).stop 2024-09-21T05:57:15.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.223+0000 7f386a694640 1 -- 172.21.15.117:0/2227908594 shutdown_connections 2024-09-21T05:57:15.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.223+0000 7f386a694640 1 --2- 172.21.15.117:0/2227908594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3864109d80 0x7f38641105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:15.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.223+0000 7f386a694640 1 --2- 172.21.15.117:0/2227908594 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3864109430 0x7f3864109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:15.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.223+0000 7f386a694640 1 --2- 172.21.15.117:0/2227908594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3864101f00 0x7f3864102310 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:15.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.223+0000 7f386a694640 1 -- 172.21.15.117:0/2227908594 >> 172.21.15.117:0/2227908594 conn(0x7f3864077ad0 msgr2=0x7f3864077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:15.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.223+0000 7f386a694640 1 -- 172.21.15.117:0/2227908594 shutdown_connections 2024-09-21T05:57:15.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.223+0000 7f386a694640 1 -- 172.21.15.117:0/2227908594 wait complete. 2024-09-21T05:57:15.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.224+0000 7f386a694640 1 Processor -- start 2024-09-21T05:57:15.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.224+0000 7f386a694640 1 -- start start 2024-09-21T05:57:15.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.224+0000 7f386a694640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3864101f00 0x7f386419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:15.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.224+0000 7f386a694640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3864109430 0x7f386419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:15.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.224+0000 7f386a694640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3864109d80 0x7f386419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:15.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.224+0000 7f386a694640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3864113a80 con 0x7f3864109d80 2024-09-21T05:57:15.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.224+0000 7f386a694640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3864113900 con 0x7f3864109430 2024-09-21T05:57:15.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.224+0000 7f386a694640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3864113c00 con 0x7f3864101f00 2024-09-21T05:57:15.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.224+0000 7f3868c31640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3864109d80 0x7f386419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:15.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.224+0000 7f3868c31640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3864109d80 0x7f386419fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34788/0 (socket says 172.21.15.117:34788) 2024-09-21T05:57:15.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.225+0000 7f3868c31640 1 -- 172.21.15.117:0/3605184660 learned_addr learned my addr 172.21.15.117:0/3605184660 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:15.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.225+0000 7f3863fff640 1 --2- 172.21.15.117:0/3605184660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3864101f00 0x7f386419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:15.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.225+0000 7f38637fe640 1 --2- 172.21.15.117:0/3605184660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3864109430 0x7f386419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:15.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.225+0000 7f3868c31640 1 -- 172.21.15.117:0/3605184660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3864101f00 msgr2=0x7f386419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:15.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.225+0000 7f3868c31640 1 --2- 172.21.15.117:0/3605184660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3864101f00 0x7f386419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:15.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.225+0000 7f3868c31640 1 -- 172.21.15.117:0/3605184660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3864109430 msgr2=0x7f386419aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:15.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.225+0000 7f3868c31640 1 --2- 172.21.15.117:0/3605184660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3864109430 0x7f386419aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:15.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.225+0000 7f3868c31640 1 -- 172.21.15.117:0/3605184660 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38641a03d0 con 0x7f3864109d80 2024-09-21T05:57:15.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.225+0000 7f3868c31640 1 --2- 172.21.15.117:0/3605184660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3864109d80 0x7f386419fcd0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f385c00eaa0 tx=0x7f385c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:15.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.225+0000 7f38617fa640 1 -- 172.21.15.117:0/3605184660 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385c004d20 con 0x7f3864109d80 2024-09-21T05:57:15.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.225+0000 7f38617fa640 1 -- 172.21.15.117:0/3605184660 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f385c005020 con 0x7f3864109d80 2024-09-21T05:57:15.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.225+0000 7f386a694640 1 -- 172.21.15.117:0/3605184660 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38641a06c0 con 0x7f3864109d80 2024-09-21T05:57:15.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.225+0000 7f38617fa640 1 -- 172.21.15.117:0/3605184660 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385c0116b0 con 0x7f3864109d80 2024-09-21T05:57:15.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.225+0000 7f386a694640 1 -- 172.21.15.117:0/3605184660 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38641a6e50 con 0x7f3864109d80 2024-09-21T05:57:15.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.226+0000 7f386a694640 1 -- 172.21.15.117:0/3605184660 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3864102310 con 0x7f3864109d80 2024-09-21T05:57:15.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.228+0000 7f38617fa640 1 -- 172.21.15.117:0/3605184660 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 23) ==== 154587+0+0 (secure 0 0 0) 0x7f385c0051d0 con 0x7f3864109d80 2024-09-21T05:57:15.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.229+0000 7f38617fa640 1 --2- 172.21.15.117:0/3605184660 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f384c0b65f0 0x7f384c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:15.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.229+0000 7f3863fff640 1 --2- 172.21.15.117:0/3605184660 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f384c0b65f0 0x7f384c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:15.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.229+0000 7f38617fa640 1 -- 172.21.15.117:0/3605184660 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f385c0e7740 con 0x7f3864109d80 2024-09-21T05:57:15.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.229+0000 7f3863fff640 1 --2- 172.21.15.117:0/3605184660 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f384c0b65f0 0x7f384c0b8b00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3854031440 tx=0x7f3854009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:15.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.233+0000 7f38617fa640 1 -- 172.21.15.117:0/3605184660 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f385c016030 con 0x7f3864109d80 2024-09-21T05:57:15.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.435+0000 7f386a694640 1 -- 172.21.15.117:0/3605184660 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f3864063800 con 0x7f384c0b65f0 2024-09-21T05:57:15.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.442+0000 7f38617fa640 1 -- 172.21.15.117:0/3605184660 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f3864063800 con 0x7f384c0b65f0 2024-09-21T05:57:15.444 INFO:teuthology.orchestra.run.smithi117.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-21T00:00:00", "created": "2024-09-21T05:57:14", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-21T05:57:15.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.444+0000 7f386a694640 1 -- 172.21.15.117:0/3605184660 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f384c0b65f0 msgr2=0x7f384c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:15.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.444+0000 7f386a694640 1 --2- 172.21.15.117:0/3605184660 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f384c0b65f0 0x7f384c0b8b00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3854031440 tx=0x7f3854009560 comp rx=0 tx=0).stop 2024-09-21T05:57:15.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.444+0000 7f386a694640 1 -- 172.21.15.117:0/3605184660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3864109d80 msgr2=0x7f386419fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:15.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.444+0000 7f386a694640 1 --2- 172.21.15.117:0/3605184660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3864109d80 0x7f386419fcd0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f385c00eaa0 tx=0x7f385c00eea0 comp rx=0 tx=0).stop 2024-09-21T05:57:15.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.445+0000 7f386a694640 1 -- 172.21.15.117:0/3605184660 shutdown_connections 2024-09-21T05:57:15.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.445+0000 7f386a694640 1 --2- 172.21.15.117:0/3605184660 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f384c0b65f0 0x7f384c0b8b00 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:15.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.445+0000 7f386a694640 1 --2- 172.21.15.117:0/3605184660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3864109d80 0x7f386419fcd0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:15.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.445+0000 7f386a694640 1 --2- 172.21.15.117:0/3605184660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3864109430 0x7f386419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:15.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.445+0000 7f386a694640 1 --2- 172.21.15.117:0/3605184660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3864101f00 0x7f386419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:15.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.445+0000 7f386a694640 1 -- 172.21.15.117:0/3605184660 >> 172.21.15.117:0/3605184660 conn(0x7f3864077ad0 msgr2=0x7f38641074e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:15.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.445+0000 7f386a694640 1 -- 172.21.15.117:0/3605184660 shutdown_connections 2024-09-21T05:57:15.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.445+0000 7f386a694640 1 -- 172.21.15.117:0/3605184660 wait complete. 2024-09-21T05:57:15.458 DEBUG:teuthology.orchestra.run.smithi117:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-21T05:57:15.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.568+0000 7f5f5d5c6640 1 -- 172.21.15.117:0/2483189325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f58102060 msgr2=0x7f5f58102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:15.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.568+0000 7f5f5d5c6640 1 --2- 172.21.15.117:0/2483189325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f58102060 0x7f5f58102470 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5f48009900 tx=0x7f5f4802a410 comp rx=0 tx=0).stop 2024-09-21T05:57:15.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.568+0000 7f5f5d5c6640 1 -- 172.21.15.117:0/2483189325 shutdown_connections 2024-09-21T05:57:15.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.568+0000 7f5f5d5c6640 1 --2- 172.21.15.117:0/2483189325 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f5810ae50 0x7f5f581127b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:15.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.568+0000 7f5f5d5c6640 1 --2- 172.21.15.117:0/2483189325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f5810a500 0x7f5f5810a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:15.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.568+0000 7f5f5d5c6640 1 --2- 172.21.15.117:0/2483189325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f58102060 0x7f5f58102470 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:15.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.568+0000 7f5f5d5c6640 1 -- 172.21.15.117:0/2483189325 >> 172.21.15.117:0/2483189325 conn(0x7f5f58079e90 msgr2=0x7f5f5807a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:15.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.568+0000 7f5f5d5c6640 1 -- 172.21.15.117:0/2483189325 shutdown_connections 2024-09-21T05:57:15.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.568+0000 7f5f5d5c6640 1 -- 172.21.15.117:0/2483189325 wait complete. 2024-09-21T05:57:15.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.569+0000 7f5f5d5c6640 1 Processor -- start 2024-09-21T05:57:15.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.569+0000 7f5f5d5c6640 1 -- start start 2024-09-21T05:57:15.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.569+0000 7f5f5d5c6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f58102060 0x7f5f5819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:15.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.569+0000 7f5f5d5c6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f5810a500 0x7f5f5819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:15.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.569+0000 7f5f5d5c6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f5810ae50 0x7f5f581a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:15.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.570+0000 7f5f5d5c6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5f58115c40 con 0x7f5f5810a500 2024-09-21T05:57:15.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.570+0000 7f5f5d5c6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5f58115ac0 con 0x7f5f58102060 2024-09-21T05:57:15.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.570+0000 7f5f5d5c6640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5f58115dc0 con 0x7f5f5810ae50 2024-09-21T05:57:15.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.570+0000 7f5f567fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f5810a500 0x7f5f5819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:15.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.570+0000 7f5f567fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f5810a500 0x7f5f5819cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34810/0 (socket says 172.21.15.117:34810) 2024-09-21T05:57:15.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.570+0000 7f5f567fc640 1 -- 172.21.15.117:0/975516242 learned_addr learned my addr 172.21.15.117:0/975516242 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:15.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.570+0000 7f5f56ffd640 1 --2- 172.21.15.117:0/975516242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f58102060 0x7f5f5819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:15.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.570+0000 7f5f577fe640 1 --2- 172.21.15.117:0/975516242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f5810ae50 0x7f5f581a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:15.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.570+0000 7f5f567fc640 1 -- 172.21.15.117:0/975516242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f5810ae50 msgr2=0x7f5f581a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:15.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.570+0000 7f5f567fc640 1 --2- 172.21.15.117:0/975516242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f5810ae50 0x7f5f581a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:15.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.570+0000 7f5f567fc640 1 -- 172.21.15.117:0/975516242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f58102060 msgr2=0x7f5f5819c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:15.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.570+0000 7f5f567fc640 1 --2- 172.21.15.117:0/975516242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f58102060 0x7f5f5819c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:15.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.570+0000 7f5f567fc640 1 -- 172.21.15.117:0/975516242 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f581a25d0 con 0x7f5f5810a500 2024-09-21T05:57:15.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.570+0000 7f5f577fe640 1 --2- 172.21.15.117:0/975516242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f5810ae50 0x7f5f581a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:57:15.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.570+0000 7f5f567fc640 1 --2- 172.21.15.117:0/975516242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f5810a500 0x7f5f5819cc40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c00e9f0 tx=0x7f5f4c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:15.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.570+0000 7f5f3ffff640 1 -- 172.21.15.117:0/975516242 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c00ccc0 con 0x7f5f5810a500 2024-09-21T05:57:15.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.570+0000 7f5f3ffff640 1 -- 172.21.15.117:0/975516242 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f4c0108e0 con 0x7f5f5810a500 2024-09-21T05:57:15.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.570+0000 7f5f5d5c6640 1 -- 172.21.15.117:0/975516242 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f581a28c0 con 0x7f5f5810a500 2024-09-21T05:57:15.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.570+0000 7f5f3ffff640 1 -- 172.21.15.117:0/975516242 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c018860 con 0x7f5f5810a500 2024-09-21T05:57:15.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.570+0000 7f5f5d5c6640 1 -- 172.21.15.117:0/975516242 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f581a9050 con 0x7f5f5810a500 2024-09-21T05:57:15.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.572+0000 7f5f5d5c6640 1 -- 172.21.15.117:0/975516242 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f58102470 con 0x7f5f5810a500 2024-09-21T05:57:15.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.573+0000 7f5f3ffff640 1 -- 172.21.15.117:0/975516242 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 23) ==== 154587+0+0 (secure 0 0 0) 0x7f5f4c010430 con 0x7f5f5810a500 2024-09-21T05:57:15.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.574+0000 7f5f3ffff640 1 --2- 172.21.15.117:0/975516242 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5f400b65f0 0x7f5f400b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:15.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.575+0000 7f5f56ffd640 1 --2- 172.21.15.117:0/975516242 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5f400b65f0 0x7f5f400b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:15.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.575+0000 7f5f3ffff640 1 -- 172.21.15.117:0/975516242 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f5f4c0e7e10 con 0x7f5f5810a500 2024-09-21T05:57:15.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.575+0000 7f5f56ffd640 1 --2- 172.21.15.117:0/975516242 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5f400b65f0 0x7f5f400b8b00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5f48031440 tx=0x7f5f48009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:15.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.578+0000 7f5f3ffff640 1 -- 172.21.15.117:0/975516242 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5f4c0ae440 con 0x7f5f5810a500 2024-09-21T05:57:15.602 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T05:57:15.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:15 smithi117 ceph-mon[30947]: mgrmap e23: x(active, since 8m), standbys: z, y 2024-09-21T05:57:15.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:15 smithi117 ceph-mon[30947]: scrub queued for path: / 2024-09-21T05:57:15.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:15 smithi117 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-21T05:57:15.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:15 smithi117 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-21T05:57:15.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:15 smithi117 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-21T05:57:15.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:15 smithi117 ceph-mon[30947]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:57:15.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:15 smithi117 ceph-mon[30947]: from='client.15558 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:57:15.790 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.788+0000 7f5f5d5c6640 1 -- 172.21.15.117:0/975516242 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f5f5810dd40 con 0x7f5f400b65f0 2024-09-21T05:57:15.798 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.796+0000 7f5f3ffff640 1 -- 172.21.15.117:0/975516242 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f5f5810dd40 con 0x7f5f400b65f0 2024-09-21T05:57:15.798 INFO:teuthology.orchestra.run.smithi117.stdout:/ 1m 6m3h 2024-09-21T05:57:15.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.798+0000 7f5f5d5c6640 1 -- 172.21.15.117:0/975516242 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5f400b65f0 msgr2=0x7f5f400b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:15.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.798+0000 7f5f5d5c6640 1 --2- 172.21.15.117:0/975516242 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5f400b65f0 0x7f5f400b8b00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5f48031440 tx=0x7f5f48009560 comp rx=0 tx=0).stop 2024-09-21T05:57:15.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.798+0000 7f5f5d5c6640 1 -- 172.21.15.117:0/975516242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f5810a500 msgr2=0x7f5f5819cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:15.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.798+0000 7f5f5d5c6640 1 --2- 172.21.15.117:0/975516242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f5810a500 0x7f5f5819cc40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c00e9f0 tx=0x7f5f4c00eeb0 comp rx=0 tx=0).stop 2024-09-21T05:57:15.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.798+0000 7f5f5d5c6640 1 -- 172.21.15.117:0/975516242 shutdown_connections 2024-09-21T05:57:15.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.798+0000 7f5f5d5c6640 1 --2- 172.21.15.117:0/975516242 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5f400b65f0 0x7f5f400b8b00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:15.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.798+0000 7f5f5d5c6640 1 --2- 172.21.15.117:0/975516242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f5810a500 0x7f5f5819cc40 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:15.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.798+0000 7f5f5d5c6640 1 --2- 172.21.15.117:0/975516242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f5810ae50 0x7f5f581a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:15.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.798+0000 7f5f5d5c6640 1 --2- 172.21.15.117:0/975516242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f58102060 0x7f5f5819c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:15.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.798+0000 7f5f5d5c6640 1 -- 172.21.15.117:0/975516242 >> 172.21.15.117:0/975516242 conn(0x7f5f58079e90 msgr2=0x7f5f58104310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:15.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.798+0000 7f5f5d5c6640 1 -- 172.21.15.117:0/975516242 shutdown_connections 2024-09-21T05:57:15.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:15.798+0000 7f5f5d5c6640 1 -- 172.21.15.117:0/975516242 wait complete. 2024-09-21T05:57:15.813 DEBUG:teuthology.orchestra.run.smithi117:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-21T05:57:15.839 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-21T05:57:15.850 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-21T05:57:15.859 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi117.front.sepia.ceph.com 2024-09-21T05:57:15.859 DEBUG:teuthology.orchestra.run.smithi117:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-21T05:57:15.860 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:57:15.862 INFO:teuthology.run_tasks:Running task workunit... 2024-09-21T05:57:15.874 INFO:tasks.workunit:Pulling workunits from ref 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T05:57:15.875 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-21T05:57:15.875 INFO:tasks.workunit:timeout=3h 2024-09-21T05:57:15.875 INFO:tasks.workunit:cleanup=True 2024-09-21T05:57:15.875 DEBUG:teuthology.orchestra.run.smithi117:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-21T05:57:15.890 INFO:teuthology.orchestra.run.smithi117.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-21T05:57:15.890 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-21T05:57:15.890 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-21T05:57:15.890 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T05:57:15.891 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-21T05:57:15.891 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-09-21 05:55:32.971556265 +0000 2024-09-21T05:57:15.891 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-09-21 05:55:32.971556265 +0000 2024-09-21T05:57:15.891 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-09-21 05:57:05.913653571 +0000 2024-09-21T05:57:15.891 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-09-21 05:55:32.971556265 +0000 2024-09-21T05:57:15.891 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-21T05:57:15.891 DEBUG:teuthology.orchestra.run.smithi117:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-21T05:57:15.966 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T05:57:15.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:15 smithi192 ceph-mon[34700]: mgrmap e23: x(active, since 8m), standbys: z, y 2024-09-21T05:57:15.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:15 smithi192 ceph-mon[34700]: scrub queued for path: / 2024-09-21T05:57:15.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:15 smithi192 ceph-mon[34700]: scrub summary: idle+waiting paths [/] 2024-09-21T05:57:15.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:15 smithi192 ceph-mon[34700]: scrub summary: active paths [/] 2024-09-21T05:57:15.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:15 smithi192 ceph-mon[34700]: scrub summary: idle+waiting paths [/] 2024-09-21T05:57:15.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:15 smithi192 ceph-mon[34700]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:57:15.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:15 smithi192 ceph-mon[34700]: from='client.15558 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:57:16.022 INFO:tasks.workunit.client.0.smithi117.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-21T05:57:16.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:15 smithi138 ceph-mon[34391]: mgrmap e23: x(active, since 8m), standbys: z, y 2024-09-21T05:57:16.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:15 smithi138 ceph-mon[34391]: scrub queued for path: / 2024-09-21T05:57:16.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:15 smithi138 ceph-mon[34391]: scrub summary: idle+waiting paths [/] 2024-09-21T05:57:16.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:15 smithi138 ceph-mon[34391]: scrub summary: active paths [/] 2024-09-21T05:57:16.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:15 smithi138 ceph-mon[34391]: scrub summary: idle+waiting paths [/] 2024-09-21T05:57:16.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:15 smithi138 ceph-mon[34391]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:57:16.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:15 smithi138 ceph-mon[34391]: from='client.15558 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:57:16.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.403+0000 7fd38d526640 1 -- 172.21.15.117:0/1418417510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd38810de50 msgr2=0x7fd388110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:16.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.403+0000 7fd38d526640 1 --2- 172.21.15.117:0/1418417510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd38810de50 0x7fd388110320 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd380009ca0 tx=0x7fd38002a8c0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.403+0000 7fd38d526640 1 -- 172.21.15.117:0/1418417510 shutdown_connections 2024-09-21T05:57:16.406 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.403+0000 7fd38d526640 1 --2- 172.21.15.117:0/1418417510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd38810de50 0x7fd388110320 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.403+0000 7fd38d526640 1 --2- 172.21.15.117:0/1418417510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd388105370 0x7fd38810d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.403+0000 7fd38d526640 1 --2- 172.21.15.117:0/1418417510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd388104a20 0x7fd388104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.403+0000 7fd38d526640 1 -- 172.21.15.117:0/1418417510 >> 172.21.15.117:0/1418417510 conn(0x7fd3880796d0 msgr2=0x7fd388079ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:16.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.403+0000 7fd38d526640 1 -- 172.21.15.117:0/1418417510 shutdown_connections 2024-09-21T05:57:16.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.403+0000 7fd38d526640 1 -- 172.21.15.117:0/1418417510 wait complete. 2024-09-21T05:57:16.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.404+0000 7fd38d526640 1 Processor -- start 2024-09-21T05:57:16.407 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.404+0000 7fd38d526640 1 -- start start 2024-09-21T05:57:16.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.405+0000 7fd38d526640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd388104a20 0x7fd38819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:16.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.405+0000 7fd38d526640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd388105370 0x7fd38819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:16.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.405+0000 7fd38d526640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd38810de50 0x7fd38819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:16.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.405+0000 7fd38d526640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd388113ab0 con 0x7fd38810de50 2024-09-21T05:57:16.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.405+0000 7fd38d526640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd388113930 con 0x7fd388104a20 2024-09-21T05:57:16.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.405+0000 7fd38d526640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd388113c30 con 0x7fd388105370 2024-09-21T05:57:16.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.405+0000 7fd3877fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd38810de50 0x7fd38819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:16.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.405+0000 7fd386ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd388104a20 0x7fd38819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:16.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.405+0000 7fd3877fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd38810de50 0x7fd38819fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34828/0 (socket says 172.21.15.117:34828) 2024-09-21T05:57:16.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.405+0000 7fd3877fe640 1 -- 172.21.15.117:0/3925079521 learned_addr learned my addr 172.21.15.117:0/3925079521 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:16.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.405+0000 7fd3867fc640 1 --2- 172.21.15.117:0/3925079521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd388105370 0x7fd38819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:16.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.405+0000 7fd386ffd640 1 -- 172.21.15.117:0/3925079521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd388105370 msgr2=0x7fd38819a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:16.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.405+0000 7fd386ffd640 1 --2- 172.21.15.117:0/3925079521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd388105370 0x7fd38819a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.405+0000 7fd386ffd640 1 -- 172.21.15.117:0/3925079521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd38810de50 msgr2=0x7fd38819fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:16.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.405+0000 7fd386ffd640 1 --2- 172.21.15.117:0/3925079521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd38810de50 0x7fd38819fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.405+0000 7fd386ffd640 1 -- 172.21.15.117:0/3925079521 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3881a0390 con 0x7fd388104a20 2024-09-21T05:57:16.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.406+0000 7fd386ffd640 1 --2- 172.21.15.117:0/3925079521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd388104a20 0x7fd38819a420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd37800e9f0 tx=0x7fd37800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:16.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.406+0000 7fd37ffff640 1 -- 172.21.15.117:0/3925079521 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37800ccc0 con 0x7fd388104a20 2024-09-21T05:57:16.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.406+0000 7fd38d526640 1 -- 172.21.15.117:0/3925079521 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3881a0680 con 0x7fd388104a20 2024-09-21T05:57:16.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.406+0000 7fd37ffff640 1 -- 172.21.15.117:0/3925079521 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3780044d0 con 0x7fd388104a20 2024-09-21T05:57:16.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.406+0000 7fd37ffff640 1 -- 172.21.15.117:0/3925079521 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37801c070 con 0x7fd388104a20 2024-09-21T05:57:16.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.406+0000 7fd38d526640 1 -- 172.21.15.117:0/3925079521 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3881a6de0 con 0x7fd388104a20 2024-09-21T05:57:16.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.408+0000 7fd393a9a740 1 Processor -- start 2024-09-21T05:57:16.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.408+0000 7fd393a9a740 1 -- start start 2024-09-21T05:57:16.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.409+0000 7fd393a9a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a69215b070 0x55a692502270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:16.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.409+0000 7fd393a9a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6921321b0 0x55a6925027b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:16.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.409+0000 7fd393a9a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a69213a390 0x55a692502cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:16.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.409+0000 7fd393a9a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a692478e50 con 0x55a69215b070 2024-09-21T05:57:16.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.409+0000 7fd393a9a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a692468280 con 0x55a6921321b0 2024-09-21T05:57:16.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.409+0000 7fd393a9a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a69246c090 con 0x55a69213a390 2024-09-21T05:57:16.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.409+0000 7fd37ffff640 1 -- 172.21.15.117:0/3925079521 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 23) ==== 154587+0+0 (secure 0 0 0) 0x7fd378004a00 con 0x7fd388104a20 2024-09-21T05:57:16.411 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.409+0000 7fd3877fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a69215b070 0x55a692502270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:16.412 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.409+0000 7fd3877fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a69215b070 0x55a692502270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34840/0 (socket says 172.21.15.117:34840) 2024-09-21T05:57:16.412 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.409+0000 7fd3877fe640 1 -- 172.21.15.117:0/2232683335 learned_addr learned my addr 172.21.15.117:0/2232683335 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:16.413 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.409+0000 7fd386ffd640 1 --2- 172.21.15.117:0/2232683335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6921321b0 0x55a6925027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:16.413 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.409+0000 7fd3867fc640 1 --2- 172.21.15.117:0/2232683335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a69213a390 0x55a692502cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:16.413 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.409+0000 7fd3877fe640 1 -- 172.21.15.117:0/2232683335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a69213a390 msgr2=0x55a692502cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:16.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.409+0000 7fd3877fe640 1 --2- 172.21.15.117:0/2232683335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a69213a390 0x55a692502cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.409+0000 7fd3877fe640 1 -- 172.21.15.117:0/2232683335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6921321b0 msgr2=0x55a6925027b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:16.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.409+0000 7fd3877fe640 1 --2- 172.21.15.117:0/2232683335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6921321b0 0x55a6925027b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.410+0000 7fd3877fe640 1 -- 172.21.15.117:0/2232683335 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6921c6650 con 0x55a69215b070 2024-09-21T05:57:16.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.410+0000 7fd386ffd640 1 --2- 172.21.15.117:0/2232683335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6921321b0 0x55a6925027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:57:16.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.410+0000 7fd3877fe640 1 --2- 172.21.15.117:0/2232683335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a69215b070 0x55a692502270 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd38002f450 tx=0x7fd38002f480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ee850939070deb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:16.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.410+0000 7fd37d7fa640 1 -- 172.21.15.117:0/2232683335 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd380031cf0 con 0x55a69215b070 2024-09-21T05:57:16.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.410+0000 7fd37d7fa640 1 -- 172.21.15.117:0/2232683335 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd380036070 con 0x55a69215b070 2024-09-21T05:57:16.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.410+0000 7fd37d7fa640 1 -- 172.21.15.117:0/2232683335 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3800042a0 con 0x55a69215b070 2024-09-21T05:57:16.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.410+0000 7fd37ffff640 1 --2- 172.21.15.117:0/3925079521 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd3740b65f0 0x7fd3740b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:16.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.410+0000 7fd386ffd640 1 --2- 172.21.15.117:0/3925079521 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd3740b65f0 0x7fd3740b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:16.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.411+0000 7fd37ffff640 1 -- 172.21.15.117:0/3925079521 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fd3780e75f0 con 0x7fd388104a20 2024-09-21T05:57:16.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.411+0000 7fd393a9a740 1 -- 172.21.15.117:0/2232683335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a69215b070 msgr2=0x55a692502270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:16.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.411+0000 7fd386ffd640 1 --2- 172.21.15.117:0/3925079521 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd3740b65f0 0x7fd3740b8b00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x55a692500ef0 tx=0x7fd37801ede0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:16.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.411+0000 7fd393a9a740 1 --2- 172.21.15.117:0/2232683335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a69215b070 0x55a692502270 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd38002f450 tx=0x7fd38002f480 comp rx=0 tx=0).stop 2024-09-21T05:57:16.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.411+0000 7fd393a9a740 1 -- 172.21.15.117:0/2232683335 shutdown_connections 2024-09-21T05:57:16.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.411+0000 7fd393a9a740 1 --2- 172.21.15.117:0/2232683335 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a69213a390 0x55a692502cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.411+0000 7fd393a9a740 1 --2- 172.21.15.117:0/2232683335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6921321b0 0x55a6925027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.411+0000 7fd393a9a740 1 --2- 172.21.15.117:0/2232683335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a69215b070 0x55a692502270 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.411+0000 7fd393a9a740 1 -- 172.21.15.117:0/2232683335 >> 172.21.15.117:0/2232683335 conn(0x55a69214f460 msgr2=0x55a692500160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:16.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.411+0000 7fd393a9a740 1 -- 172.21.15.117:0/2232683335 shutdown_connections 2024-09-21T05:57:16.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.411+0000 7fd393a9a740 1 -- 172.21.15.117:0/2232683335 wait complete. 2024-09-21T05:57:16.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.412+0000 7fd393a9a740 1 -- start start 2024-09-21T05:57:16.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.412+0000 7fd393a9a740 1 Processor -- start 2024-09-21T05:57:16.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.412+0000 7fd393a9a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6921321b0 0x55a69250d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:16.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.412+0000 7fd393a9a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a69213a390 0x55a69250df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:16.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.412+0000 7fd393a9a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a69215b070 0x55a692519b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:16.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.412+0000 7fd393a9a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a6922c9700 con 0x55a69215b070 2024-09-21T05:57:16.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.412+0000 7fd393a9a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a692403110 con 0x55a6921321b0 2024-09-21T05:57:16.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.412+0000 7fd393a9a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a692510870 con 0x55a69213a390 2024-09-21T05:57:16.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.412+0000 7fd393a9a740 10 client.0 _ll_register_callbacks cb 0x55a69241c990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:57:16.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.412+0000 7fd3877fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a69215b070 0x55a692519b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:16.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.412+0000 7fd3877fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a69215b070 0x55a692519b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34852/0 (socket says 172.21.15.117:34852) 2024-09-21T05:57:16.419 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.412+0000 7fd3877fe640 1 -- 172.21.15.117:0/2461237634 learned_addr learned my addr 172.21.15.117:0/2461237634 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:16.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.412+0000 7fd3867fc640 1 --2- 172.21.15.117:0/2461237634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a69213a390 0x55a69250df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:16.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.412+0000 7fd3877fe640 1 --2- 172.21.15.117:0/2461237634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6921321b0 0x55a69250d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:16.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.413+0000 7fd3867fc640 1 -- 172.21.15.117:0/2461237634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6921321b0 msgr2=0x55a69250d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:16.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.413+0000 7fd3867fc640 1 --2- 172.21.15.117:0/2461237634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6921321b0 0x55a69250d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.413+0000 7fd3867fc640 1 -- 172.21.15.117:0/2461237634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a69215b070 msgr2=0x55a692519b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:16.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.413+0000 7fd3867fc640 1 --2- 172.21.15.117:0/2461237634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a69215b070 0x55a692519b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.413+0000 7fd3867fc640 1 -- 172.21.15.117:0/2461237634 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a692430c40 con 0x55a69213a390 2024-09-21T05:57:16.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.413+0000 7fd3877fe640 1 --2- 172.21.15.117:0/2461237634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a69215b070 0x55a692519b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:57:16.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.413+0000 7fd3867fc640 1 --2- 172.21.15.117:0/2461237634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a69213a390 0x55a69250df10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd38806b6c0 tx=0x7fd370013b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44d433747bad8f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:16.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.413+0000 7fd36e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-21T05:57:16.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.413+0000 7fd36e7fc640 1 -- 172.21.15.117:0/2461237634 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd370011910 con 0x55a69213a390 2024-09-21T05:57:16.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.413+0000 7fd393a9a740 1 -- 172.21.15.117:0/2461237634 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd380036070 con 0x55a69213a390 2024-09-21T05:57:16.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.413+0000 7fd36e7fc640 1 -- 172.21.15.117:0/2461237634 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fd370011ab0 con 0x55a69213a390 2024-09-21T05:57:16.420 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.413+0000 7fd36e7fc640 1 -- 172.21.15.117:0/2461237634 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd370011cb0 con 0x55a69213a390 2024-09-21T05:57:16.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.413+0000 7fd393a9a740 10 client.25117 fetch_fsmap learned FSMap version 27 2024-09-21T05:57:16.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.413+0000 7fd393a9a740 1 -- 172.21.15.117:0/2461237634 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a69251c610 con 0x55a69213a390 2024-09-21T05:57:16.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.414+0000 7fd36e7fc640 1 -- 172.21.15.117:0/2461237634 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd380036070 con 0x55a69213a390 2024-09-21T05:57:16.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.414+0000 7fd36e7fc640 1 -- 172.21.15.117:0/2461237634 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd37001a070 con 0x55a69213a390 2024-09-21T05:57:16.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.414+0000 7fd393a9a740 10 client.25117 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:57:16.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.414+0000 7fd393a9a740 10 client.25117 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T05:57:16.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.414+0000 7fd393a9a740 20 client.25117 populate_metadata read hostname 'smithi117' 2024-09-21T05:57:16.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.414+0000 7fd393a9a740 1 --2- 172.21.15.117:0/2461237634 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55a6921199f0 0x55a69251fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:16.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.414+0000 7fd393a9a740 4 client.25117 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:57:16.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.414+0000 7fd393a9a740 1 -- 172.21.15.117:0/2461237634 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a69247ada0 con 0x55a6921199f0 2024-09-21T05:57:16.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.414+0000 7fd3877fe640 1 --2- 172.21.15.117:0/2461237634 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55a6921199f0 0x55a69251fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:16.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.415+0000 7fd36e7fc640 10 client.25117 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:57:16.421 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.415+0000 7fd3877fe640 1 --2- 172.21.15.117:0/2461237634 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55a6921199f0 0x55a69251fb20 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63cfc4172b682246 server_cookie=6902b8bebf549686 in_seq=0 out_seq=0 2024-09-21T05:57:16.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.416+0000 7fd36e7fc640 1 -- 172.21.15.117:0/2461237634 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a69247ada0 con 0x55a6921199f0 2024-09-21T05:57:16.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.416+0000 7fd36e7fc640 10 client.25117 handle_command_reply: tid=0 2024-09-21T05:57:16.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.416+0000 7fd393a9a740 1 client.25117 shutdown 2024-09-21T05:57:16.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.416+0000 7fd393a9a740 1 -- 172.21.15.117:0/2461237634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a69213a390 msgr2=0x55a69250df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:16.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.416+0000 7fd393a9a740 1 --2- 172.21.15.117:0/2461237634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a69213a390 0x55a69250df10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd38806b6c0 tx=0x7fd370013b70 comp rx=0 tx=0).stop 2024-09-21T05:57:16.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.416+0000 7fd393a9a740 1 -- 172.21.15.117:0/2461237634 shutdown_connections 2024-09-21T05:57:16.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.416+0000 7fd393a9a740 1 --2- 172.21.15.117:0/2461237634 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55a6921199f0 0x55a69251fb20 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.416+0000 7fd393a9a740 1 --2- 172.21.15.117:0/2461237634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a69215b070 0x55a692519b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.416+0000 7fd393a9a740 1 --2- 172.21.15.117:0/2461237634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a69213a390 0x55a69250df10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.416+0000 7fd393a9a740 1 --2- 172.21.15.117:0/2461237634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6921321b0 0x55a69250d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.416+0000 7fd393a9a740 1 -- 172.21.15.117:0/2461237634 >> 172.21.15.117:0/2461237634 conn(0x55a69214f460 msgr2=0x55a692505450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:16.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.416+0000 7fd36e7fc640 0 client.25117 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:57:16.422 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.416+0000 7fd393a9a740 1 -- 172.21.15.117:0/2461237634 shutdown_connections 2024-09-21T05:57:16.423 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.416+0000 7fd393a9a740 1 -- 172.21.15.117:0/2461237634 wait complete. 2024-09-21T05:57:16.423 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.416+0000 7fd393a9a740 20 client.25117 trim_cache size 0 max 16384 2024-09-21T05:57:16.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.434+0000 7fd393a9a740 1 Processor -- start 2024-09-21T05:57:16.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.434+0000 7fd393a9a740 1 -- start start 2024-09-21T05:57:16.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.434+0000 7fd393a9a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6921321b0 0x55a692505a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:16.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.434+0000 7fd393a9a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a69213a390 0x55a69250d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:16.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.434+0000 7fd393a9a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a69215b070 0x55a692519a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:16.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.434+0000 7fd393a9a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881af440 con 0x55a69215b070 2024-09-21T05:57:16.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.434+0000 7fd393a9a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881b0f80 con 0x55a6921321b0 2024-09-21T05:57:16.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.434+0000 7fd393a9a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881afb10 con 0x55a69213a390 2024-09-21T05:57:16.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.435+0000 7fd3877fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a69215b070 0x55a692519a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:16.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.435+0000 7fd3877fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a69215b070 0x55a692519a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34866/0 (socket says 172.21.15.117:34866) 2024-09-21T05:57:16.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.435+0000 7fd3877fe640 1 -- 172.21.15.117:0/1391822651 learned_addr learned my addr 172.21.15.117:0/1391822651 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:16.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.435+0000 7fd3867fc640 1 --2- 172.21.15.117:0/1391822651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a69213a390 0x55a69250d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:16.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.435+0000 7fd3877fe640 1 --2- 172.21.15.117:0/1391822651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6921321b0 0x55a692505a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:16.437 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.435+0000 7fd3877fe640 1 -- 172.21.15.117:0/1391822651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a69213a390 msgr2=0x55a69250d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:16.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.435+0000 7fd3877fe640 1 --2- 172.21.15.117:0/1391822651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a69213a390 0x55a69250d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.435+0000 7fd3877fe640 1 -- 172.21.15.117:0/1391822651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6921321b0 msgr2=0x55a692505a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:16.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.435+0000 7fd3877fe640 1 --2- 172.21.15.117:0/1391822651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6921321b0 0x55a692505a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.435+0000 7fd3877fe640 1 -- 172.21.15.117:0/1391822651 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a69250c930 con 0x55a69215b070 2024-09-21T05:57:16.438 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.435+0000 7fd3877fe640 1 --2- 172.21.15.117:0/1391822651 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a69215b070 0x55a692519a90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd380009620 tx=0x7fd380003db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c3bae4ab394433 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:16.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.435+0000 7fd37cff9640 1 -- 172.21.15.117:0/1391822651 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd380031cf0 con 0x55a69215b070 2024-09-21T05:57:16.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.435+0000 7fd37cff9640 1 -- 172.21.15.117:0/1391822651 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd380036070 con 0x55a69215b070 2024-09-21T05:57:16.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.435+0000 7fd37cff9640 1 -- 172.21.15.117:0/1391822651 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd380031880 con 0x55a69215b070 2024-09-21T05:57:16.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.437+0000 7fd393a9a740 1 -- 172.21.15.117:0/1391822651 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a69215b070 msgr2=0x55a692519a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:16.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.437+0000 7fd393a9a740 1 --2- 172.21.15.117:0/1391822651 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a69215b070 0x55a692519a90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd380009620 tx=0x7fd380003db0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.437+0000 7fd393a9a740 1 -- 172.21.15.117:0/1391822651 shutdown_connections 2024-09-21T05:57:16.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.437+0000 7fd393a9a740 1 --2- 172.21.15.117:0/1391822651 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a69215b070 0x55a692519a90 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.437+0000 7fd393a9a740 1 --2- 172.21.15.117:0/1391822651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a69213a390 0x55a69250d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.437+0000 7fd393a9a740 1 --2- 172.21.15.117:0/1391822651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a6921321b0 0x55a692505a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.437+0000 7fd393a9a740 1 -- 172.21.15.117:0/1391822651 >> 172.21.15.117:0/1391822651 conn(0x55a69214f460 msgr2=0x55a692505370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:16.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.437+0000 7fd393a9a740 1 -- 172.21.15.117:0/1391822651 shutdown_connections 2024-09-21T05:57:16.440 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.437+0000 7fd393a9a740 1 -- 172.21.15.117:0/1391822651 wait complete. 2024-09-21T05:57:16.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.438+0000 7fd393a9a740 1 -- start start 2024-09-21T05:57:16.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.438+0000 7fd393a9a740 1 Processor -- start 2024-09-21T05:57:16.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.438+0000 7fd393a9a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a6921321b0 0x55a692511ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:16.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.438+0000 7fd393a9a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a69213a390 0x55a692511ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:16.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.438+0000 7fd393a9a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a69215b070 0x55a692512530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:16.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.438+0000 7fd393a9a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881b08b0 con 0x55a69215b070 2024-09-21T05:57:16.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.438+0000 7fd393a9a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881b01e0 con 0x55a69213a390 2024-09-21T05:57:16.442 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.438+0000 7fd393a9a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881ad420 con 0x55a6921321b0 2024-09-21T05:57:16.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.438+0000 7fd393a9a740 10 client.0 _ll_register_callbacks cb 0x7fd3700097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:57:16.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.438+0000 7fd3877fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a69215b070 0x55a692512530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:16.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.438+0000 7fd3877fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a69215b070 0x55a692512530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34882/0 (socket says 172.21.15.117:34882) 2024-09-21T05:57:16.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.438+0000 7fd3877fe640 1 -- 172.21.15.117:0/2343220486 learned_addr learned my addr 172.21.15.117:0/2343220486 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:16.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.438+0000 7fd3867fc640 1 --2- 172.21.15.117:0/2343220486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a69213a390 0x55a692511ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:16.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.439+0000 7fd3877fe640 1 --2- 172.21.15.117:0/2343220486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a6921321b0 0x55a692511ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:16.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.439+0000 7fd3877fe640 1 -- 172.21.15.117:0/2343220486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a6921321b0 msgr2=0x55a692511ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:16.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.439+0000 7fd3877fe640 1 --2- 172.21.15.117:0/2343220486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a6921321b0 0x55a692511ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.439+0000 7fd3877fe640 1 -- 172.21.15.117:0/2343220486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a69213a390 msgr2=0x55a692511ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:16.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.439+0000 7fd3877fe640 1 --2- 172.21.15.117:0/2343220486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a69213a390 0x55a692511ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.443 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.439+0000 7fd3877fe640 1 -- 172.21.15.117:0/2343220486 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a69258d8d0 con 0x55a69215b070 2024-09-21T05:57:16.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.439+0000 7fd3867fc640 1 --2- 172.21.15.117:0/2343220486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a69213a390 0x55a692511ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:57:16.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.439+0000 7fd3877fe640 1 --2- 172.21.15.117:0/2343220486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a69215b070 0x55a692512530 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x55a69251ac20 tx=0x7fd38002b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db654e0b423be35c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:16.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.439+0000 7fd36e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:57:16.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.439+0000 7fd36e7fc640 1 -- 172.21.15.117:0/2343220486 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd380031550 con 0x55a69215b070 2024-09-21T05:57:16.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.439+0000 7fd36e7fc640 1 -- 172.21.15.117:0/2343220486 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fd380004450 con 0x55a69215b070 2024-09-21T05:57:16.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.439+0000 7fd393a9a740 1 -- 172.21.15.117:0/2343220486 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd380036070 con 0x55a69215b070 2024-09-21T05:57:16.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.440+0000 7fd36e7fc640 1 -- 172.21.15.117:0/2343220486 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd380031e10 con 0x55a69215b070 2024-09-21T05:57:16.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.440+0000 7fd393a9a740 10 client.15582 fetch_fsmap learned FSMap version 27 2024-09-21T05:57:16.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.440+0000 7fd393a9a740 1 -- 172.21.15.117:0/2343220486 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a69258eb40 con 0x55a69215b070 2024-09-21T05:57:16.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.440+0000 7fd36e7fc640 1 -- 172.21.15.117:0/2343220486 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd380036070 con 0x55a69215b070 2024-09-21T05:57:16.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.440+0000 7fd36e7fc640 1 -- 172.21.15.117:0/2343220486 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd380043020 con 0x55a69215b070 2024-09-21T05:57:16.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.440+0000 7fd393a9a740 10 client.15582 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:57:16.444 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.440+0000 7fd393a9a740 10 client.15582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T05:57:16.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.440+0000 7fd393a9a740 20 client.15582 populate_metadata read hostname 'smithi117' 2024-09-21T05:57:16.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.440+0000 7fd393a9a740 1 --2- 172.21.15.117:0/2343220486 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55a6920eeda0 0x55a692590f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:16.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.440+0000 7fd393a9a740 4 client.15582 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T05:57:16.445 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.440+0000 7fd393a9a740 1 -- 172.21.15.117:0/2343220486 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a692476850 con 0x55a6920eeda0 2024-09-21T05:57:16.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.441+0000 7fd3867fc640 1 --2- 172.21.15.117:0/2343220486 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55a6920eeda0 0x55a692590f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:16.446 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.441+0000 7fd36e7fc640 10 client.15582 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:57:16.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.441+0000 7fd3867fc640 1 --2- 172.21.15.117:0/2343220486 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55a6920eeda0 0x55a692590f70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95b43312fc3be198 server_cookie=18c001db687436fd in_seq=0 out_seq=0 2024-09-21T05:57:16.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.441+0000 7fd36e7fc640 1 -- 172.21.15.117:0/2343220486 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+570 (crc 0 0 0) 0x55a692476850 con 0x55a6920eeda0 2024-09-21T05:57:16.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.441+0000 7fd36e7fc640 10 client.15582 handle_command_reply: tid=0 2024-09-21T05:57:16.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.441+0000 7fd393a9a740 1 client.15582 shutdown 2024-09-21T05:57:16.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.442+0000 7fd393a9a740 1 -- 172.21.15.117:0/2343220486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a69215b070 msgr2=0x55a692512530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:16.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.442+0000 7fd393a9a740 1 --2- 172.21.15.117:0/2343220486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a69215b070 0x55a692512530 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x55a69251ac20 tx=0x7fd38002b040 comp rx=0 tx=0).stop 2024-09-21T05:57:16.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.442+0000 7fd393a9a740 1 -- 172.21.15.117:0/2343220486 shutdown_connections 2024-09-21T05:57:16.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.442+0000 7fd393a9a740 1 --2- 172.21.15.117:0/2343220486 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55a6920eeda0 0x55a692590f70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.442+0000 7fd393a9a740 1 --2- 172.21.15.117:0/2343220486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a69215b070 0x55a692512530 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.442+0000 7fd393a9a740 1 --2- 172.21.15.117:0/2343220486 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a69213a390 0x55a692511ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.442+0000 7fd36e7fc640 0 client.15582 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:57:16.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.442+0000 7fd393a9a740 1 --2- 172.21.15.117:0/2343220486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a6921321b0 0x55a692511ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.447 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.442+0000 7fd393a9a740 1 -- 172.21.15.117:0/2343220486 >> 172.21.15.117:0/2343220486 conn(0x55a69214f460 msgr2=0x55a6924ff920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:16.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.442+0000 7fd393a9a740 1 -- 172.21.15.117:0/2343220486 shutdown_connections 2024-09-21T05:57:16.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.442+0000 7fd393a9a740 1 -- 172.21.15.117:0/2343220486 wait complete. 2024-09-21T05:57:16.448 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.442+0000 7fd393a9a740 20 client.15582 trim_cache size 0 max 16384 2024-09-21T05:57:16.448 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T05:57:16.448 INFO:teuthology.orchestra.run.smithi117.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T05:57:16.448 INFO:teuthology.orchestra.run.smithi117.stdout: "scrubs": { 2024-09-21T05:57:16.448 INFO:teuthology.orchestra.run.smithi117.stdout: "b30bfff0-0f6b-4e30-8467-ff7db421eba6": { 2024-09-21T05:57:16.448 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T05:57:16.448 INFO:teuthology.orchestra.run.smithi117.stdout: "tag": "b30bfff0-0f6b-4e30-8467-ff7db421eba6", 2024-09-21T05:57:16.448 INFO:teuthology.orchestra.run.smithi117.stdout: "options": "recursive,force" 2024-09-21T05:57:16.448 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T05:57:16.449 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T05:57:16.449 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_stats": { 2024-09-21T05:57:16.449 INFO:teuthology.orchestra.run.smithi117.stdout: "b30bfff0-0f6b-4e30-8467-ff7db421eba6": { 2024-09-21T05:57:16.449 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:57:14.939082+0000", 2024-09-21T05:57:16.449 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T05:57:16.449 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T05:57:16.449 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T05:57:16.449 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T05:57:16.449 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T05:57:16.449 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T05:57:16.449 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T05:57:16.449 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T05:57:16.450 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.443+0000 7fd38d526640 1 -- 172.21.15.117:0/3925079521 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd3740b65f0 msgr2=0x7fd3740b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:16.450 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.443+0000 7fd38d526640 1 --2- 172.21.15.117:0/3925079521 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd3740b65f0 0x7fd3740b8b00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x55a692500ef0 tx=0x7fd37801ede0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.443+0000 7fd38d526640 1 -- 172.21.15.117:0/3925079521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd388104a20 msgr2=0x7fd38819a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:16.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.443+0000 7fd38d526640 1 --2- 172.21.15.117:0/3925079521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd388104a20 0x7fd38819a420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd37800e9f0 tx=0x7fd37800eeb0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.443+0000 7fd38d526640 1 -- 172.21.15.117:0/3925079521 shutdown_connections 2024-09-21T05:57:16.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.443+0000 7fd38d526640 1 --2- 172.21.15.117:0/3925079521 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd3740b65f0 0x7fd3740b8b00 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.443+0000 7fd38d526640 1 --2- 172.21.15.117:0/3925079521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd38810de50 0x7fd38819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.443+0000 7fd38d526640 1 --2- 172.21.15.117:0/3925079521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd388105370 0x7fd38819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.443+0000 7fd38d526640 1 --2- 172.21.15.117:0/3925079521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd388104a20 0x7fd38819a420 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:16.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.443+0000 7fd38d526640 1 -- 172.21.15.117:0/3925079521 >> 172.21.15.117:0/3925079521 conn(0x7fd3880796d0 msgr2=0x7fd3880ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:16.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.443+0000 7fd38d526640 1 -- 172.21.15.117:0/3925079521 shutdown_connections 2024-09-21T05:57:16.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:16.443+0000 7fd38d526640 1 -- 172.21.15.117:0/3925079521 wait complete. 2024-09-21T05:57:16.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:16 smithi192 ceph-mon[34700]: from='client.15564 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:57:16.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:16 smithi192 ceph-mon[34700]: pgmap v326: 97 pgs: 97 active+clean; 782 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-09-21T05:57:17.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:16 smithi117 ceph-mon[30947]: from='client.15564 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:57:17.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:16 smithi117 ceph-mon[30947]: pgmap v326: 97 pgs: 97 active+clean; 782 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-09-21T05:57:17.111 INFO:tasks.cephfs.filesystem:scrub status for tag:b30bfff0-0f6b-4e30-8467-ff7db421eba6 - {'path': '/', 'tag': 'b30bfff0-0f6b-4e30-8467-ff7db421eba6', 'options': 'recursive,force'} 2024-09-21T05:57:17.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:16 smithi138 ceph-mon[34391]: from='client.15564 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T05:57:17.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:16 smithi138 ceph-mon[34391]: pgmap v326: 97 pgs: 97 active+clean; 782 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-09-21T05:57:19.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:19 smithi138 ceph-mon[34391]: pgmap v327: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 126 KiB/s wr, 54 op/s 2024-09-21T05:57:19.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:19 smithi192 ceph-mon[34700]: pgmap v327: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 126 KiB/s wr, 54 op/s 2024-09-21T05:57:20.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:19 smithi117 ceph-mon[30947]: pgmap v327: 97 pgs: 97 active+clean; 1.1 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 126 KiB/s wr, 54 op/s 2024-09-21T05:57:21.848 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:21 smithi117 ceph-mon[30947]: pgmap v328: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 56 op/s 2024-09-21T05:57:21.848 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:21 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:57:21.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:21 smithi138 ceph-mon[34391]: pgmap v328: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 56 op/s 2024-09-21T05:57:21.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:21 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:57:21.886 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:21 smithi192 ceph-mon[34700]: pgmap v328: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 56 op/s 2024-09-21T05:57:21.886 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:21 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:57:23.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:57:23.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:57:23.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:57:23.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:57:23.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:57:23.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:57:23.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:23 smithi192 ceph-mon[34700]: pgmap v329: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 175 KiB/s wr, 57 op/s 2024-09-21T05:57:23.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:57:23.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:57:23.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:57:23.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:57:23.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:57:23.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:57:23.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:57:23.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:57:23.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:57:23.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:23 smithi117 ceph-mon[30947]: pgmap v329: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 175 KiB/s wr, 57 op/s 2024-09-21T05:57:23.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:57:23.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:57:23.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:57:23.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:57:23.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:57:23.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:57:23.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:57:23.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:57:23.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:57:23.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:23 smithi138 ceph-mon[34391]: pgmap v329: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 175 KiB/s wr, 57 op/s 2024-09-21T05:57:23.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:57:23.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:57:23.866 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:57:25.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:25 smithi138 ceph-mon[34391]: pgmap v330: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 175 KiB/s wr, 58 op/s 2024-09-21T05:57:25.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:25 smithi192 ceph-mon[34700]: pgmap v330: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 175 KiB/s wr, 58 op/s 2024-09-21T05:57:26.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:25 smithi117 ceph-mon[30947]: pgmap v330: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 175 KiB/s wr, 58 op/s 2024-09-21T05:57:27.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:27 smithi117 ceph-mon[30947]: pgmap v331: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 57 op/s 2024-09-21T05:57:27.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:27 smithi138 ceph-mon[34391]: pgmap v331: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 57 op/s 2024-09-21T05:57:27.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:27 smithi192 ceph-mon[34700]: pgmap v331: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 57 op/s 2024-09-21T05:57:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.009+0000 7f0ee54d5640 1 -- 172.21.15.117:0/631648570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ee0109d50 msgr2=0x7f0ee0110580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.009+0000 7f0ee54d5640 1 --2- 172.21.15.117:0/631648570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ee0109d50 0x7f0ee0110580 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8009ca0 tx=0x7f0ed802a8c0 comp rx=0 tx=0).stop 2024-09-21T05:57:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.010+0000 7f0ee54d5640 1 -- 172.21.15.117:0/631648570 shutdown_connections 2024-09-21T05:57:29.012 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.010+0000 7f0ee54d5640 1 --2- 172.21.15.117:0/631648570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ee0109d50 0x7f0ee0110580 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.010+0000 7f0ee54d5640 1 --2- 172.21.15.117:0/631648570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ee0109400 0x7f0ee0109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.010+0000 7f0ee54d5640 1 --2- 172.21.15.117:0/631648570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ee0103ad0 0x7f0ee0103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.010+0000 7f0ee54d5640 1 -- 172.21.15.117:0/631648570 >> 172.21.15.117:0/631648570 conn(0x7f0ee00ff510 msgr2=0x7f0ee0101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.010+0000 7f0ee54d5640 1 -- 172.21.15.117:0/631648570 shutdown_connections 2024-09-21T05:57:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.010+0000 7f0ee54d5640 1 -- 172.21.15.117:0/631648570 wait complete. 2024-09-21T05:57:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.010+0000 7f0ee54d5640 1 Processor -- start 2024-09-21T05:57:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.010+0000 7f0ee54d5640 1 -- start start 2024-09-21T05:57:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.011+0000 7f0ee54d5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ee0103ad0 0x7f0ee019a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.011+0000 7f0ee54d5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ee0109400 0x7f0ee019a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:29.019 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.011+0000 7f0ee54d5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ee0109d50 0x7f0ee019fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.011+0000 7f0ee54d5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee0113a10 con 0x7f0ee0103ad0 2024-09-21T05:57:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.011+0000 7f0ee54d5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee0113890 con 0x7f0ee0109d50 2024-09-21T05:57:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.011+0000 7f0ee54d5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee0113b90 con 0x7f0ee0109400 2024-09-21T05:57:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.011+0000 7f0edeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ee0103ad0 0x7f0ee019a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.011+0000 7f0edeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ee0103ad0 0x7f0ee019a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48900/0 (socket says 172.21.15.117:48900) 2024-09-21T05:57:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.011+0000 7f0edeffd640 1 -- 172.21.15.117:0/3731620824 learned_addr learned my addr 172.21.15.117:0/3731620824 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.011+0000 7f0edf7fe640 1 --2- 172.21.15.117:0/3731620824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ee0109d50 0x7f0ee019fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.011+0000 7f0ed7fff640 1 --2- 172.21.15.117:0/3731620824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ee0109400 0x7f0ee019a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.011+0000 7f0edeffd640 1 -- 172.21.15.117:0/3731620824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ee0109400 msgr2=0x7f0ee019a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.011+0000 7f0edeffd640 1 --2- 172.21.15.117:0/3731620824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ee0109400 0x7f0ee019a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.011+0000 7f0edeffd640 1 -- 172.21.15.117:0/3731620824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ee0109d50 msgr2=0x7f0ee019fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.011+0000 7f0edeffd640 1 --2- 172.21.15.117:0/3731620824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ee0109d50 0x7f0ee019fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:29.020 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.011+0000 7f0edeffd640 1 -- 172.21.15.117:0/3731620824 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ee01a0270 con 0x7f0ee0103ad0 2024-09-21T05:57:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.011+0000 7f0edeffd640 1 --2- 172.21.15.117:0/3731620824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ee0103ad0 0x7f0ee019a3d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0ed000c920 tx=0x7f0ed000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.011+0000 7f0edcff9640 1 -- 172.21.15.117:0/3731620824 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0007c70 con 0x7f0ee0103ad0 2024-09-21T05:57:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.011+0000 7f0ee54d5640 1 -- 172.21.15.117:0/3731620824 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ee01a0560 con 0x7f0ee0103ad0 2024-09-21T05:57:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.012+0000 7f0edcff9640 1 -- 172.21.15.117:0/3731620824 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ed0007e10 con 0x7f0ee0103ad0 2024-09-21T05:57:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.012+0000 7f0edcff9640 1 -- 172.21.15.117:0/3731620824 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0018820 con 0x7f0ee0103ad0 2024-09-21T05:57:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.012+0000 7f0ee54d5640 1 -- 172.21.15.117:0/3731620824 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ee01a6d80 con 0x7f0ee0103ad0 2024-09-21T05:57:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.014+0000 7f0eec440740 1 Processor -- start 2024-09-21T05:57:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.015+0000 7f0eec440740 1 -- start start 2024-09-21T05:57:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.015+0000 7f0eec440740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dc99231b0 0x558dc9cc89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.015+0000 7f0eec440740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dc98fa2f0 0x558dc9cc8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.015+0000 7f0eec440740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc99024d0 0x558dc9cc9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.015+0000 7f0eec440740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558dc9c3d6f0 con 0x558dc98fa2f0 2024-09-21T05:57:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.015+0000 7f0eec440740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558dc9c2cb20 con 0x558dc99231b0 2024-09-21T05:57:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.015+0000 7f0eec440740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558dc9c30930 con 0x558dc99024d0 2024-09-21T05:57:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.015+0000 7f0edcff9640 1 -- 172.21.15.117:0/3731620824 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 23) ==== 154587+0+0 (secure 0 0 0) 0x7f0ed00189c0 con 0x7f0ee0103ad0 2024-09-21T05:57:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.015+0000 7f0edf7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dc99231b0 0x558dc9cc89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.015+0000 7f0edf7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dc99231b0 0x558dc9cc89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:55386/0 (socket says 172.21.15.117:55386) 2024-09-21T05:57:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.015+0000 7f0edf7fe640 1 -- 172.21.15.117:0/2191757465 learned_addr learned my addr 172.21.15.117:0/2191757465 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.015+0000 7f0ed7fff640 1 --2- 172.21.15.117:0/2191757465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc99024d0 0x558dc9cc9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.016+0000 7f0edeffd640 1 --2- 172.21.15.117:0/2191757465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dc98fa2f0 0x558dc9cc8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.016+0000 7f0edf7fe640 1 -- 172.21.15.117:0/2191757465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc99024d0 msgr2=0x558dc9cc9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.016+0000 7f0edf7fe640 1 --2- 172.21.15.117:0/2191757465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc99024d0 0x558dc9cc9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.016+0000 7f0edf7fe640 1 -- 172.21.15.117:0/2191757465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dc98fa2f0 msgr2=0x558dc9cc8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.016+0000 7f0edf7fe640 1 --2- 172.21.15.117:0/2191757465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dc98fa2f0 0x558dc9cc8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.016+0000 7f0edf7fe640 1 -- 172.21.15.117:0/2191757465 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dc9bf2530 con 0x558dc99231b0 2024-09-21T05:57:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.016+0000 7f0edf7fe640 1 --2- 172.21.15.117:0/2191757465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dc99231b0 0x558dc9cc89b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8009950 tx=0x7f0ed8009980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86ec9e6b95792384 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.016+0000 7f0ed57fa640 1 -- 172.21.15.117:0/2191757465 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8031710 con 0x558dc99231b0 2024-09-21T05:57:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.016+0000 7f0ed57fa640 1 -- 172.21.15.117:0/2191757465 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ed8036070 con 0x558dc99231b0 2024-09-21T05:57:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.016+0000 7f0edcff9640 1 --2- 172.21.15.117:0/3731620824 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0ecc0b65f0 0x7f0ecc0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.016+0000 7f0ed57fa640 1 -- 172.21.15.117:0/2191757465 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8004730 con 0x558dc99231b0 2024-09-21T05:57:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.017+0000 7f0edeffd640 1 --2- 172.21.15.117:0/3731620824 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0ecc0b65f0 0x7f0ecc0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.017+0000 7f0edcff9640 1 -- 172.21.15.117:0/3731620824 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f0ed00e7680 con 0x7f0ee0103ad0 2024-09-21T05:57:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.017+0000 7f0eec440740 1 -- 172.21.15.117:0/2191757465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dc99231b0 msgr2=0x558dc9cc89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.017+0000 7f0edeffd640 1 --2- 172.21.15.117:0/3731620824 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0ecc0b65f0 0x7f0ecc0b8b00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x558dc9cc7630 tx=0x7f0ed00ed530 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.017+0000 7f0eec440740 1 --2- 172.21.15.117:0/2191757465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dc99231b0 0x558dc9cc89b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8009950 tx=0x7f0ed8009980 comp rx=0 tx=0).stop 2024-09-21T05:57:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.017+0000 7f0eec440740 1 -- 172.21.15.117:0/2191757465 shutdown_connections 2024-09-21T05:57:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.017+0000 7f0eec440740 1 --2- 172.21.15.117:0/2191757465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dc98fa2f0 0x558dc9cc8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.017+0000 7f0eec440740 1 --2- 172.21.15.117:0/2191757465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc99024d0 0x558dc9cc9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.017+0000 7f0eec440740 1 --2- 172.21.15.117:0/2191757465 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dc99231b0 0x558dc9cc89b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.017+0000 7f0eec440740 1 -- 172.21.15.117:0/2191757465 >> 172.21.15.117:0/2191757465 conn(0x558dc9917300 msgr2=0x558dc9cc68a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.018+0000 7f0eec440740 1 -- 172.21.15.117:0/2191757465 shutdown_connections 2024-09-21T05:57:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.018+0000 7f0eec440740 1 -- 172.21.15.117:0/2191757465 wait complete. 2024-09-21T05:57:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.018+0000 7f0eec440740 1 -- start start 2024-09-21T05:57:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.018+0000 7f0eec440740 1 Processor -- start 2024-09-21T05:57:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.018+0000 7f0eec440740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dc98fa2f0 0x558dc9cd40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.018+0000 7f0eec440740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dc99024d0 0x558dc9cd45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.018+0000 7f0eec440740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc99231b0 0x558dc9ce0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.018+0000 7f0eec440740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558dc9b64ab0 con 0x558dc99024d0 2024-09-21T05:57:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.018+0000 7f0eec440740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558dc9bd2730 con 0x558dc98fa2f0 2024-09-21T05:57:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.018+0000 7f0eec440740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558dc9cd6f50 con 0x558dc99231b0 2024-09-21T05:57:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.018+0000 7f0eec440740 10 client.0 _ll_register_callbacks cb 0x558dc9c266b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:57:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.018+0000 7f0ed7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dc99024d0 0x558dc9cd45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.018+0000 7f0ed7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dc99024d0 0x558dc9cd45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48916/0 (socket says 172.21.15.117:48916) 2024-09-21T05:57:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.018+0000 7f0ed7fff640 1 -- 172.21.15.117:0/4206149607 learned_addr learned my addr 172.21.15.117:0/4206149607 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.019+0000 7f0edf7fe640 1 --2- 172.21.15.117:0/4206149607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dc98fa2f0 0x558dc9cd40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.019+0000 7f0edf7fe640 1 --2- 172.21.15.117:0/4206149607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc99231b0 0x558dc9ce0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.019+0000 7f0ed7fff640 1 -- 172.21.15.117:0/4206149607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc99231b0 msgr2=0x558dc9ce0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.019+0000 7f0ed7fff640 1 --2- 172.21.15.117:0/4206149607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc99231b0 0x558dc9ce0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.019+0000 7f0ed7fff640 1 -- 172.21.15.117:0/4206149607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dc98fa2f0 msgr2=0x558dc9cd40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.019+0000 7f0ed7fff640 1 --2- 172.21.15.117:0/4206149607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dc98fa2f0 0x558dc9cd40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.019+0000 7f0ed7fff640 1 -- 172.21.15.117:0/4206149607 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dc998e790 con 0x558dc99024d0 2024-09-21T05:57:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.019+0000 7f0edf7fe640 1 --2- 172.21.15.117:0/4206149607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dc98fa2f0 0x558dc9cd40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:57:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.019+0000 7f0ed7fff640 1 --2- 172.21.15.117:0/4206149607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dc99024d0 0x558dc9cd45f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800bad0 tx=0x7f0ec800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3393f3679e530a9f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.019+0000 7f0ec67fc640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:57:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.019+0000 7f0ec67fc640 1 -- 172.21.15.117:0/4206149607 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8003e00 con 0x558dc99024d0 2024-09-21T05:57:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.019+0000 7f0ec67fc640 1 -- 172.21.15.117:0/4206149607 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f0ec8011440 con 0x558dc99024d0 2024-09-21T05:57:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.019+0000 7f0eec440740 1 -- 172.21.15.117:0/4206149607 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ed8036070 con 0x558dc99024d0 2024-09-21T05:57:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.019+0000 7f0ec67fc640 1 -- 172.21.15.117:0/4206149607 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0ec8011680 con 0x558dc99024d0 2024-09-21T05:57:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.019+0000 7f0eec440740 10 client.15594 fetch_fsmap learned FSMap version 27 2024-09-21T05:57:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.019+0000 7f0eec440740 1 -- 172.21.15.117:0/4206149607 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558dc9a07620 con 0x558dc99024d0 2024-09-21T05:57:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.019+0000 7f0ec67fc640 1 -- 172.21.15.117:0/4206149607 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ed8036070 con 0x558dc99024d0 2024-09-21T05:57:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.020+0000 7f0ec67fc640 1 -- 172.21.15.117:0/4206149607 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0ec801ba70 con 0x558dc99024d0 2024-09-21T05:57:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.020+0000 7f0eec440740 10 client.15594 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:57:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.020+0000 7f0eec440740 10 client.15594 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T05:57:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.020+0000 7f0eec440740 20 client.15594 populate_metadata read hostname 'smithi117' 2024-09-21T05:57:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.020+0000 7f0eec440740 1 --2- 172.21.15.117:0/4206149607 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x558dc98e1d80 0x558dc9ce60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.020+0000 7f0eec440740 4 client.15594 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:57:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.020+0000 7f0eec440740 1 -- 172.21.15.117:0/4206149607 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dc9c3f640 con 0x558dc98e1d80 2024-09-21T05:57:29.029 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.020+0000 7f0edf7fe640 1 --2- 172.21.15.117:0/4206149607 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x558dc98e1d80 0x558dc9ce60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.020+0000 7f0ec67fc640 10 client.15594 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:57:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.020+0000 7f0edf7fe640 1 --2- 172.21.15.117:0/4206149607 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x558dc98e1d80 0x558dc9ce60f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6050f36543f4b53c server_cookie=a2f8e8ca7b159e65 in_seq=0 out_seq=0 2024-09-21T05:57:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.021+0000 7f0ec67fc640 1 -- 172.21.15.117:0/4206149607 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558dc9c3f640 con 0x558dc98e1d80 2024-09-21T05:57:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.021+0000 7f0ec67fc640 10 client.15594 handle_command_reply: tid=0 2024-09-21T05:57:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.021+0000 7f0eec440740 1 client.15594 shutdown 2024-09-21T05:57:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.021+0000 7f0eec440740 1 -- 172.21.15.117:0/4206149607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dc99024d0 msgr2=0x558dc9cd45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.021+0000 7f0eec440740 1 --2- 172.21.15.117:0/4206149607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dc99024d0 0x558dc9cd45f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800bad0 tx=0x7f0ec800bf90 comp rx=0 tx=0).stop 2024-09-21T05:57:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.021+0000 7f0eec440740 1 -- 172.21.15.117:0/4206149607 shutdown_connections 2024-09-21T05:57:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.021+0000 7f0eec440740 1 --2- 172.21.15.117:0/4206149607 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x558dc98e1d80 0x558dc9ce60f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.021+0000 7f0eec440740 1 --2- 172.21.15.117:0/4206149607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dc99024d0 0x558dc9cd45f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.021+0000 7f0eec440740 1 --2- 172.21.15.117:0/4206149607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc99231b0 0x558dc9ce0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.021+0000 7f0ec67fc640 0 client.15594 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:57:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.021+0000 7f0eec440740 1 --2- 172.21.15.117:0/4206149607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dc98fa2f0 0x558dc9cd40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:29.030 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.021+0000 7f0eec440740 1 -- 172.21.15.117:0/4206149607 >> 172.21.15.117:0/4206149607 conn(0x558dc9917300 msgr2=0x558dc9cc99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.021+0000 7f0eec440740 1 -- 172.21.15.117:0/4206149607 shutdown_connections 2024-09-21T05:57:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.021+0000 7f0eec440740 1 -- 172.21.15.117:0/4206149607 wait complete. 2024-09-21T05:57:29.031 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.021+0000 7f0eec440740 20 client.15594 trim_cache size 0 max 16384 2024-09-21T05:57:29.037 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.036+0000 7f0eec440740 1 Processor -- start 2024-09-21T05:57:29.037 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.036+0000 7f0eec440740 1 -- start start 2024-09-21T05:57:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.036+0000 7f0eec440740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc98fa2f0 0x558dc9cd4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.036+0000 7f0eec440740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dc99024d0 0x558dc9cca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.036+0000 7f0eec440740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dc99231b0 0x558dc9cd5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.036+0000 7f0eec440740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee01af390 con 0x558dc99231b0 2024-09-21T05:57:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.036+0000 7f0eec440740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee01b0ed0 con 0x558dc99024d0 2024-09-21T05:57:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.036+0000 7f0eec440740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee01afa60 con 0x558dc98fa2f0 2024-09-21T05:57:29.039 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.036+0000 7f0edf7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dc99231b0 0x558dc9cd5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.036+0000 7f0edf7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dc99231b0 0x558dc9cd5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48926/0 (socket says 172.21.15.117:48926) 2024-09-21T05:57:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.036+0000 7f0edf7fe640 1 -- 172.21.15.117:0/4241980498 learned_addr learned my addr 172.21.15.117:0/4241980498 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.036+0000 7f0ed7fff640 1 --2- 172.21.15.117:0/4241980498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dc99024d0 0x558dc9cca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.036+0000 7f0edf7fe640 1 --2- 172.21.15.117:0/4241980498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc98fa2f0 0x558dc9cd4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.036+0000 7f0edf7fe640 1 -- 172.21.15.117:0/4241980498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc98fa2f0 msgr2=0x558dc9cd4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.036+0000 7f0edf7fe640 1 --2- 172.21.15.117:0/4241980498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc98fa2f0 0x558dc9cd4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.036+0000 7f0edf7fe640 1 -- 172.21.15.117:0/4241980498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dc99024d0 msgr2=0x558dc9cca3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.036+0000 7f0edf7fe640 1 --2- 172.21.15.117:0/4241980498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dc99024d0 0x558dc9cca3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.037+0000 7f0edf7fe640 1 -- 172.21.15.117:0/4241980498 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dc9ccb7c0 con 0x558dc99231b0 2024-09-21T05:57:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.037+0000 7f0edf7fe640 1 --2- 172.21.15.117:0/4241980498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dc99231b0 0x558dc9cd5ce0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8006010 tx=0x7f0ed803e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a082984139bdeb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.037+0000 7f0ed4ff9640 1 -- 172.21.15.117:0/4241980498 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed80047b0 con 0x558dc99231b0 2024-09-21T05:57:29.041 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.037+0000 7f0ed4ff9640 1 -- 172.21.15.117:0/4241980498 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ed8036070 con 0x558dc99231b0 2024-09-21T05:57:29.041 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.037+0000 7f0ed4ff9640 1 -- 172.21.15.117:0/4241980498 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed802f660 con 0x558dc99231b0 2024-09-21T05:57:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.038+0000 7f0eec440740 1 -- 172.21.15.117:0/4241980498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dc99231b0 msgr2=0x558dc9cd5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.038+0000 7f0eec440740 1 --2- 172.21.15.117:0/4241980498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dc99231b0 0x558dc9cd5ce0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8006010 tx=0x7f0ed803e040 comp rx=0 tx=0).stop 2024-09-21T05:57:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.038+0000 7f0eec440740 1 -- 172.21.15.117:0/4241980498 shutdown_connections 2024-09-21T05:57:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.038+0000 7f0eec440740 1 --2- 172.21.15.117:0/4241980498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dc99231b0 0x558dc9cd5ce0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.038+0000 7f0eec440740 1 --2- 172.21.15.117:0/4241980498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dc99024d0 0x558dc9cca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.038+0000 7f0eec440740 1 --2- 172.21.15.117:0/4241980498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc98fa2f0 0x558dc9cd4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.038+0000 7f0eec440740 1 -- 172.21.15.117:0/4241980498 >> 172.21.15.117:0/4241980498 conn(0x558dc9917300 msgr2=0x558dc9cd55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.038+0000 7f0eec440740 1 -- 172.21.15.117:0/4241980498 shutdown_connections 2024-09-21T05:57:29.044 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.038+0000 7f0eec440740 1 -- 172.21.15.117:0/4241980498 wait complete. 2024-09-21T05:57:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.038+0000 7f0eec440740 1 -- start start 2024-09-21T05:57:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.038+0000 7f0eec440740 1 Processor -- start 2024-09-21T05:57:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.039+0000 7f0eec440740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dc98fa2f0 0x558dc9ceb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.039+0000 7f0eec440740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc99024d0 0x558dc9cebfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.039+0000 7f0eec440740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dc99231b0 0x558dc9d49fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.039+0000 7f0eec440740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee01b0800 con 0x558dc99231b0 2024-09-21T05:57:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.039+0000 7f0eec440740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee01b0130 con 0x558dc98fa2f0 2024-09-21T05:57:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.039+0000 7f0eec440740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee01ad370 con 0x558dc99024d0 2024-09-21T05:57:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.039+0000 7f0eec440740 10 client.0 _ll_register_callbacks cb 0x7f0ec8004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:57:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.039+0000 7f0edf7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dc99231b0 0x558dc9d49fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.039+0000 7f0edf7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dc99231b0 0x558dc9d49fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48940/0 (socket says 172.21.15.117:48940) 2024-09-21T05:57:29.045 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.039+0000 7f0edf7fe640 1 -- 172.21.15.117:0/3537120860 learned_addr learned my addr 172.21.15.117:0/3537120860 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.039+0000 7f0edf7fe640 1 --2- 172.21.15.117:0/3537120860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dc98fa2f0 0x558dc9ceb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.039+0000 7f0ed7fff640 1 --2- 172.21.15.117:0/3537120860 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc99024d0 0x558dc9cebfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.039+0000 7f0edf7fe640 1 -- 172.21.15.117:0/3537120860 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc99024d0 msgr2=0x558dc9cebfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.039+0000 7f0edf7fe640 1 --2- 172.21.15.117:0/3537120860 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc99024d0 0x558dc9cebfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.039+0000 7f0edf7fe640 1 -- 172.21.15.117:0/3537120860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dc98fa2f0 msgr2=0x558dc9ceb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.039+0000 7f0edf7fe640 1 --2- 172.21.15.117:0/3537120860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dc98fa2f0 0x558dc9ceb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.039+0000 7f0edf7fe640 1 -- 172.21.15.117:0/3537120860 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dc9d4a670 con 0x558dc99231b0 2024-09-21T05:57:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.039+0000 7f0edf7fe640 1 --2- 172.21.15.117:0/3537120860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dc99231b0 0x558dc9d49fd0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8009a60 tx=0x7f0ed8003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb164cab7db0b92b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.039+0000 7f0ec67fc640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:57:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.039+0000 7f0ec67fc640 1 -- 172.21.15.117:0/3537120860 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8049070 con 0x558dc99231b0 2024-09-21T05:57:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.039+0000 7f0ec67fc640 1 -- 172.21.15.117:0/3537120860 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f0ed8003c70 con 0x558dc99231b0 2024-09-21T05:57:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.039+0000 7f0eec440740 1 -- 172.21.15.117:0/3537120860 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ed8036070 con 0x558dc99231b0 2024-09-21T05:57:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.039+0000 7f0ec67fc640 1 -- 172.21.15.117:0/3537120860 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0ed80041e0 con 0x558dc99231b0 2024-09-21T05:57:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.039+0000 7f0eec440740 10 client.15600 fetch_fsmap learned FSMap version 27 2024-09-21T05:57:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.039+0000 7f0eec440740 1 -- 172.21.15.117:0/3537120860 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558dc9d54ca0 con 0x558dc99231b0 2024-09-21T05:57:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.040+0000 7f0ec67fc640 1 -- 172.21.15.117:0/3537120860 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ed8036070 con 0x558dc99231b0 2024-09-21T05:57:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.040+0000 7f0ec67fc640 1 -- 172.21.15.117:0/3537120860 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0ed8053070 con 0x558dc99231b0 2024-09-21T05:57:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.040+0000 7f0eec440740 10 client.15600 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:57:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.040+0000 7f0eec440740 10 client.15600 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T05:57:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.040+0000 7f0eec440740 20 client.15600 populate_metadata read hostname 'smithi117' 2024-09-21T05:57:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.040+0000 7f0eec440740 1 --2- 172.21.15.117:0/3537120860 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x558dc98b71c0 0x558dc9d570c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.040+0000 7f0eec440740 4 client.15600 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T05:57:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.040+0000 7f0eec440740 1 -- 172.21.15.117:0/3537120860 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558dc9c3b0f0 con 0x558dc98b71c0 2024-09-21T05:57:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.040+0000 7f0ed7fff640 1 --2- 172.21.15.117:0/3537120860 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x558dc98b71c0 0x558dc9d570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.041+0000 7f0ed7fff640 1 --2- 172.21.15.117:0/3537120860 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x558dc98b71c0 0x558dc9d570c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6a00349abf299e9 server_cookie=f82a1f8e4a56db62 in_seq=0 out_seq=0 2024-09-21T05:57:29.047 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.041+0000 7f0ec67fc640 10 client.15600 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:57:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.052+0000 7f0ec67fc640 1 -- 172.21.15.117:0/3537120860 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558dc9c3b0f0 con 0x558dc98b71c0 2024-09-21T05:57:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.052+0000 7f0ec67fc640 10 client.15600 handle_command_reply: tid=0 2024-09-21T05:57:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.052+0000 7f0eec440740 1 client.15600 shutdown 2024-09-21T05:57:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.053+0000 7f0eec440740 1 -- 172.21.15.117:0/3537120860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dc99231b0 msgr2=0x558dc9d49fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.053+0000 7f0eec440740 1 --2- 172.21.15.117:0/3537120860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dc99231b0 0x558dc9d49fd0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8009a60 tx=0x7f0ed8003b90 comp rx=0 tx=0).stop 2024-09-21T05:57:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.053+0000 7f0eec440740 1 -- 172.21.15.117:0/3537120860 shutdown_connections 2024-09-21T05:57:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.053+0000 7f0eec440740 1 --2- 172.21.15.117:0/3537120860 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x558dc98b71c0 0x558dc9d570c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.053+0000 7f0eec440740 1 --2- 172.21.15.117:0/3537120860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558dc99231b0 0x558dc9d49fd0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.053+0000 7f0ec67fc640 0 client.15600 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:57:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.053+0000 7f0eec440740 1 --2- 172.21.15.117:0/3537120860 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558dc99024d0 0x558dc9cebfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.053+0000 7f0eec440740 1 --2- 172.21.15.117:0/3537120860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558dc98fa2f0 0x558dc9ceb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.053+0000 7f0eec440740 1 -- 172.21.15.117:0/3537120860 >> 172.21.15.117:0/3537120860 conn(0x558dc9917300 msgr2=0x558dc9cd5f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.053+0000 7f0eec440740 1 -- 172.21.15.117:0/3537120860 shutdown_connections 2024-09-21T05:57:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.053+0000 7f0eec440740 1 -- 172.21.15.117:0/3537120860 wait complete. 2024-09-21T05:57:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.053+0000 7f0eec440740 20 client.15600 trim_cache size 0 max 16384 2024-09-21T05:57:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T05:57:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T05:57:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T05:57:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T05:57:29.057 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.053+0000 7f0ee54d5640 1 -- 172.21.15.117:0/3731620824 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0ecc0b65f0 msgr2=0x7f0ecc0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:29.057 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.053+0000 7f0ee54d5640 1 --2- 172.21.15.117:0/3731620824 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0ecc0b65f0 0x7f0ecc0b8b00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x558dc9cc7630 tx=0x7f0ed00ed530 comp rx=0 tx=0).stop 2024-09-21T05:57:29.057 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.053+0000 7f0ee54d5640 1 -- 172.21.15.117:0/3731620824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ee0103ad0 msgr2=0x7f0ee019a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:29.057 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.053+0000 7f0ee54d5640 1 --2- 172.21.15.117:0/3731620824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ee0103ad0 0x7f0ee019a3d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0ed000c920 tx=0x7f0ed000cde0 comp rx=0 tx=0).stop 2024-09-21T05:57:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.053+0000 7f0ee54d5640 1 -- 172.21.15.117:0/3731620824 shutdown_connections 2024-09-21T05:57:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.053+0000 7f0ee54d5640 1 --2- 172.21.15.117:0/3731620824 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0ecc0b65f0 0x7f0ecc0b8b00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.053+0000 7f0ee54d5640 1 --2- 172.21.15.117:0/3731620824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ee0109d50 0x7f0ee019fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.053+0000 7f0ee54d5640 1 --2- 172.21.15.117:0/3731620824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ee0109400 0x7f0ee019a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.053+0000 7f0ee54d5640 1 --2- 172.21.15.117:0/3731620824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ee0103ad0 0x7f0ee019a3d0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.053+0000 7f0ee54d5640 1 -- 172.21.15.117:0/3731620824 >> 172.21.15.117:0/3731620824 conn(0x7f0ee00ff510 msgr2=0x7f0ee0101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.054+0000 7f0ee54d5640 1 -- 172.21.15.117:0/3731620824 shutdown_connections 2024-09-21T05:57:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:29.054+0000 7f0ee54d5640 1 -- 172.21.15.117:0/3731620824 wait complete. 2024-09-21T05:57:29.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:29 smithi138 ceph-mon[34391]: pgmap v332: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 152 KiB/s wr, 54 op/s 2024-09-21T05:57:29.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:29 smithi138 ceph-mon[34391]: scrub summary: idle 2024-09-21T05:57:29.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:29 smithi192 ceph-mon[34700]: pgmap v332: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 152 KiB/s wr, 54 op/s 2024-09-21T05:57:29.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:29 smithi192 ceph-mon[34700]: scrub summary: idle 2024-09-21T05:57:30.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:29 smithi117 ceph-mon[30947]: pgmap v332: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 152 KiB/s wr, 54 op/s 2024-09-21T05:57:30.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:29 smithi117 ceph-mon[30947]: scrub summary: idle 2024-09-21T05:57:31.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:31 smithi138 ceph-mon[34391]: pgmap v333: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-09-21T05:57:31.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:31 smithi192 ceph-mon[34700]: pgmap v333: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-09-21T05:57:32.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:31 smithi117 ceph-mon[30947]: pgmap v333: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-09-21T05:57:33.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:33 smithi138 ceph-mon[34391]: pgmap v334: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-21T05:57:33.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:33 smithi192 ceph-mon[34700]: pgmap v334: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-21T05:57:34.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:33 smithi117 ceph-mon[30947]: pgmap v334: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-21T05:57:35.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:35 smithi138 ceph-mon[34391]: pgmap v335: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.6 KiB/s wr, 14 op/s 2024-09-21T05:57:35.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:35 smithi192 ceph-mon[34700]: pgmap v335: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.6 KiB/s wr, 14 op/s 2024-09-21T05:57:36.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:35 smithi117 ceph-mon[30947]: pgmap v335: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.6 KiB/s wr, 14 op/s 2024-09-21T05:57:37.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:37 smithi138 ceph-mon[34391]: pgmap v336: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.6 KiB/s wr, 14 op/s 2024-09-21T05:57:37.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:37 smithi192 ceph-mon[34700]: pgmap v336: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.6 KiB/s wr, 14 op/s 2024-09-21T05:57:38.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:37 smithi117 ceph-mon[30947]: pgmap v336: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.6 KiB/s wr, 14 op/s 2024-09-21T05:57:39.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:39 smithi192 ceph-mon[34700]: pgmap v337: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.6 KiB/s wr, 15 op/s 2024-09-21T05:57:40.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:39 smithi117 ceph-mon[30947]: pgmap v337: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.6 KiB/s wr, 15 op/s 2024-09-21T05:57:40.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:39 smithi138 ceph-mon[34391]: pgmap v337: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.6 KiB/s wr, 15 op/s 2024-09-21T05:57:41.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:41 smithi192 ceph-mon[34700]: pgmap v338: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-09-21T05:57:42.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:41 smithi117 ceph-mon[30947]: pgmap v338: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-09-21T05:57:42.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:41 smithi138 ceph-mon[34391]: pgmap v338: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-09-21T05:57:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.154+0000 7f8f9f119640 1 -- 172.21.15.117:0/3630334580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f98108c30 msgr2=0x7f8f9810f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.154+0000 7f8f9f119640 1 --2- 172.21.15.117:0/3630334580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f98108c30 0x7f8f9810f460 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8f90009ca0 tx=0x7f8f9002a8c0 comp rx=0 tx=0).stop 2024-09-21T05:57:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.154+0000 7f8f9f119640 1 -- 172.21.15.117:0/3630334580 shutdown_connections 2024-09-21T05:57:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.154+0000 7f8f9f119640 1 --2- 172.21.15.117:0/3630334580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f98108c30 0x7f8f9810f460 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.154+0000 7f8f9f119640 1 --2- 172.21.15.117:0/3630334580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f981082e0 0x7f8f981086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.154+0000 7f8f9f119640 1 --2- 172.21.15.117:0/3630334580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f981029b0 0x7f8f98102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.154+0000 7f8f9f119640 1 -- 172.21.15.117:0/3630334580 >> 172.21.15.117:0/3630334580 conn(0x7f8f9807a740 msgr2=0x7f8f9807ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.154+0000 7f8f9f119640 1 -- 172.21.15.117:0/3630334580 shutdown_connections 2024-09-21T05:57:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.154+0000 7f8f9f119640 1 -- 172.21.15.117:0/3630334580 wait complete. 2024-09-21T05:57:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.155+0000 7f8f9f119640 1 Processor -- start 2024-09-21T05:57:42.158 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.155+0000 7f8f9f119640 1 -- start start 2024-09-21T05:57:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.155+0000 7f8f9f119640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f981029b0 0x7f8f9819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.155+0000 7f8f9f119640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f981082e0 0x7f8f9819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.155+0000 7f8f9f119640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f98108c30 0x7f8f9819fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:42.160 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.155+0000 7f8f9f119640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8f981128f0 con 0x7f8f98108c30 2024-09-21T05:57:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.155+0000 7f8f9f119640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8f98112770 con 0x7f8f981029b0 2024-09-21T05:57:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.155+0000 7f8f9f119640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8f98112a70 con 0x7f8f981082e0 2024-09-21T05:57:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.156+0000 7f8f9d6b6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f98108c30 0x7f8f9819fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.156+0000 7f8f9d6b6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f98108c30 0x7f8f9819fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33234/0 (socket says 172.21.15.117:33234) 2024-09-21T05:57:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.156+0000 7f8f9d6b6640 1 -- 172.21.15.117:0/3714834602 learned_addr learned my addr 172.21.15.117:0/3714834602 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.156+0000 7f8f9ceb5640 1 --2- 172.21.15.117:0/3714834602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f981029b0 0x7f8f9819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.156+0000 7f8f97fff640 1 --2- 172.21.15.117:0/3714834602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f981082e0 0x7f8f9819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.156+0000 7f8f9d6b6640 1 -- 172.21.15.117:0/3714834602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f981082e0 msgr2=0x7f8f9819aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.156+0000 7f8f9d6b6640 1 --2- 172.21.15.117:0/3714834602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f981082e0 0x7f8f9819aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.156+0000 7f8f9d6b6640 1 -- 172.21.15.117:0/3714834602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f981029b0 msgr2=0x7f8f9819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.156+0000 7f8f9d6b6640 1 --2- 172.21.15.117:0/3714834602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f981029b0 0x7f8f9819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.156+0000 7f8f9d6b6640 1 -- 172.21.15.117:0/3714834602 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f981a0300 con 0x7f8f98108c30 2024-09-21T05:57:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.156+0000 7f8f9ceb5640 1 --2- 172.21.15.117:0/3714834602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f981029b0 0x7f8f9819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:57:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.156+0000 7f8f97fff640 1 --2- 172.21.15.117:0/3714834602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f981082e0 0x7f8f9819aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:57:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.156+0000 7f8f9d6b6640 1 --2- 172.21.15.117:0/3714834602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f98108c30 0x7f8f9819fc90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8f900098d0 tx=0x7f8f900317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.156+0000 7f8f95ffb640 1 -- 172.21.15.117:0/3714834602 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f90003b70 con 0x7f8f98108c30 2024-09-21T05:57:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.156+0000 7f8f95ffb640 1 -- 172.21.15.117:0/3714834602 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f90003d10 con 0x7f8f98108c30 2024-09-21T05:57:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.156+0000 7f8f9f119640 1 -- 172.21.15.117:0/3714834602 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f981a0590 con 0x7f8f98108c30 2024-09-21T05:57:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.156+0000 7f8f95ffb640 1 -- 172.21.15.117:0/3714834602 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9002fa20 con 0x7f8f98108c30 2024-09-21T05:57:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.157+0000 7f8f9f119640 1 -- 172.21.15.117:0/3714834602 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f981a6e10 con 0x7f8f98108c30 2024-09-21T05:57:42.162 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.159+0000 7f8fa6086740 1 Processor -- start 2024-09-21T05:57:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.160+0000 7f8f95ffb640 1 -- 172.21.15.117:0/3714834602 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 23) ==== 154587+0+0 (secure 0 0 0) 0x7f8f9002fc40 con 0x7f8f98108c30 2024-09-21T05:57:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.161+0000 7f8fa6086740 1 -- start start 2024-09-21T05:57:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.161+0000 7f8fa6086740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f1f6c91b0 0x557f1fa6e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.161+0000 7f8fa6086740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1f6a02f0 0x557f1fa6eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.161+0000 7f8fa6086740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f1f6a84d0 0x557f1fa6f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.161+0000 7f8fa6086740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557f1f9e36f0 con 0x557f1f6a84d0 2024-09-21T05:57:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.161+0000 7f8fa6086740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557f1f9d2b20 con 0x557f1f6c91b0 2024-09-21T05:57:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.161+0000 7f8fa6086740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557f1f9d6930 con 0x557f1f6a02f0 2024-09-21T05:57:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.161+0000 7f8f97fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f1f6a84d0 0x557f1fa6f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.161+0000 7f8f97fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f1f6a84d0 0x557f1fa6f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33240/0 (socket says 172.21.15.117:33240) 2024-09-21T05:57:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.161+0000 7f8f97fff640 1 -- 172.21.15.117:0/651444439 learned_addr learned my addr 172.21.15.117:0/651444439 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.161+0000 7f8f9ceb5640 1 --2- 172.21.15.117:0/651444439 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1f6a02f0 0x557f1fa6eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.161+0000 7f8f97fff640 1 --2- 172.21.15.117:0/651444439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f1f6c91b0 0x557f1fa6e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.161+0000 7f8f95ffb640 1 --2- 172.21.15.117:0/3714834602 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8f880b65f0 0x7f8f880b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:42.165 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.161+0000 7f8f97fff640 1 -- 172.21.15.117:0/651444439 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1f6a02f0 msgr2=0x557f1fa6eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.161+0000 7f8f97fff640 1 --2- 172.21.15.117:0/651444439 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1f6a02f0 0x557f1fa6eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.161+0000 7f8f97fff640 1 -- 172.21.15.117:0/651444439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f1f6c91b0 msgr2=0x557f1fa6e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.161+0000 7f8f97fff640 1 --2- 172.21.15.117:0/651444439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f1f6c91b0 0x557f1fa6e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.161+0000 7f8f97fff640 1 -- 172.21.15.117:0/651444439 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f1f998530 con 0x557f1f6a84d0 2024-09-21T05:57:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.161+0000 7f8f9d6b6640 1 --2- 172.21.15.117:0/3714834602 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8f880b65f0 0x7f8f880b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.162+0000 7f8f97fff640 1 --2- 172.21.15.117:0/651444439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f1f6a84d0 0x557f1fa6f430 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8f84012520 tx=0x7f8f8400b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c5405e8ef8c71a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.162+0000 7f8f95ffb640 1 -- 172.21.15.117:0/3714834602 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f8f90108940 con 0x7f8f98108c30 2024-09-21T05:57:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.162+0000 7f8f82ffd640 1 -- 172.21.15.117:0/651444439 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f84018030 con 0x557f1f6a84d0 2024-09-21T05:57:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.162+0000 7f8f82ffd640 1 -- 172.21.15.117:0/651444439 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f84010b50 con 0x557f1f6a84d0 2024-09-21T05:57:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.162+0000 7f8f82ffd640 1 -- 172.21.15.117:0/651444439 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8400eba0 con 0x557f1f6a84d0 2024-09-21T05:57:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.163+0000 7f8fa6086740 1 -- 172.21.15.117:0/651444439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f1f6a84d0 msgr2=0x557f1fa6f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:42.169 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.163+0000 7f8f9d6b6640 1 --2- 172.21.15.117:0/3714834602 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8f880b65f0 0x7f8f880b8b00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8f900c8fd0 tx=0x7f8f9009b690 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.163+0000 7f8fa6086740 1 --2- 172.21.15.117:0/651444439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f1f6a84d0 0x557f1fa6f430 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8f84012520 tx=0x7f8f8400b600 comp rx=0 tx=0).stop 2024-09-21T05:57:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.163+0000 7f8fa6086740 1 -- 172.21.15.117:0/651444439 shutdown_connections 2024-09-21T05:57:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.163+0000 7f8fa6086740 1 --2- 172.21.15.117:0/651444439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f1f6a84d0 0x557f1fa6f430 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.163+0000 7f8fa6086740 1 --2- 172.21.15.117:0/651444439 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1f6a02f0 0x557f1fa6eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.163+0000 7f8fa6086740 1 --2- 172.21.15.117:0/651444439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f1f6c91b0 0x557f1fa6e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.163+0000 7f8fa6086740 1 -- 172.21.15.117:0/651444439 >> 172.21.15.117:0/651444439 conn(0x557f1f6bd300 msgr2=0x557f1fa6c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.163+0000 7f8fa6086740 1 -- 172.21.15.117:0/651444439 shutdown_connections 2024-09-21T05:57:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.163+0000 7f8fa6086740 1 -- 172.21.15.117:0/651444439 wait complete. 2024-09-21T05:57:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.163+0000 7f8fa6086740 1 -- start start 2024-09-21T05:57:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.163+0000 7f8fa6086740 1 Processor -- start 2024-09-21T05:57:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.164+0000 7f8fa6086740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1f6a02f0 0x557f1fa7a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.164+0000 7f8fa6086740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f1f6a84d0 0x557f1fa7a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:42.170 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.164+0000 7f8fa6086740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f1f6c91b0 0x557f1fa86420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.164+0000 7f8fa6086740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557f1f90aab0 con 0x557f1f6c91b0 2024-09-21T05:57:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.164+0000 7f8fa6086740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557f1f978730 con 0x557f1f6a84d0 2024-09-21T05:57:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.164+0000 7f8fa6086740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557f1fa7d130 con 0x557f1f6a02f0 2024-09-21T05:57:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.164+0000 7f8fa6086740 10 client.0 _ll_register_callbacks cb 0x557f1f9cc6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:57:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.164+0000 7f8f97fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f1f6c91b0 0x557f1fa86420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.164+0000 7f8f97fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f1f6c91b0 0x557f1fa86420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33254/0 (socket says 172.21.15.117:33254) 2024-09-21T05:57:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.164+0000 7f8f97fff640 1 -- 172.21.15.117:0/1528035610 learned_addr learned my addr 172.21.15.117:0/1528035610 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.164+0000 7f8f97fff640 1 --2- 172.21.15.117:0/1528035610 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1f6a02f0 0x557f1fa7a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.164+0000 7f8f9ceb5640 1 --2- 172.21.15.117:0/1528035610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f1f6a84d0 0x557f1fa7a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.164+0000 7f8f97fff640 1 -- 172.21.15.117:0/1528035610 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1f6a02f0 msgr2=0x557f1fa7a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.164+0000 7f8f97fff640 1 --2- 172.21.15.117:0/1528035610 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1f6a02f0 0x557f1fa7a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.164+0000 7f8f97fff640 1 -- 172.21.15.117:0/1528035610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f1f6a84d0 msgr2=0x557f1fa7a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.164+0000 7f8f97fff640 1 --2- 172.21.15.117:0/1528035610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f1f6a84d0 0x557f1fa7a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.164+0000 7f8f97fff640 1 -- 172.21.15.117:0/1528035610 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f1f734790 con 0x557f1f6c91b0 2024-09-21T05:57:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.165+0000 7f8f97fff640 1 --2- 172.21.15.117:0/1528035610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f1f6c91b0 0x557f1fa86420 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400e4e0 tx=0x7f8f8400e780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f98dc3cfafb2c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.165+0000 7f8f77fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:57:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.165+0000 7f8f77fff640 1 -- 172.21.15.117:0/1528035610 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f84018070 con 0x557f1f6c91b0 2024-09-21T05:57:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.165+0000 7f8f77fff640 1 -- 172.21.15.117:0/1528035610 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f8f8400eb30 con 0x557f1f6c91b0 2024-09-21T05:57:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.165+0000 7f8fa6086740 1 -- 172.21.15.117:0/1528035610 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f84010b50 con 0x557f1f6c91b0 2024-09-21T05:57:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.165+0000 7f8f77fff640 1 -- 172.21.15.117:0/1528035610 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8f84031050 con 0x557f1f6c91b0 2024-09-21T05:57:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.165+0000 7f8fa6086740 10 client.15612 fetch_fsmap learned FSMap version 27 2024-09-21T05:57:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.165+0000 7f8fa6086740 1 -- 172.21.15.117:0/1528035610 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557f1f7ad620 con 0x557f1f6c91b0 2024-09-21T05:57:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.165+0000 7f8f77fff640 1 -- 172.21.15.117:0/1528035610 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f84010b50 con 0x557f1f6c91b0 2024-09-21T05:57:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.165+0000 7f8f77fff640 1 -- 172.21.15.117:0/1528035610 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8f84029020 con 0x557f1f6c91b0 2024-09-21T05:57:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.165+0000 7f8fa6086740 10 client.15612 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:57:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.165+0000 7f8fa6086740 10 client.15612 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T05:57:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.165+0000 7f8fa6086740 20 client.15612 populate_metadata read hostname 'smithi117' 2024-09-21T05:57:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.165+0000 7f8fa6086740 1 --2- 172.21.15.117:0/1528035610 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x557f1f687d80 0x557f1fa8c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.166+0000 7f8fa6086740 4 client.15612 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:57:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.166+0000 7f8fa6086740 1 -- 172.21.15.117:0/1528035610 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f1f9e5640 con 0x557f1f687d80 2024-09-21T05:57:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.166+0000 7f8f9ceb5640 1 --2- 172.21.15.117:0/1528035610 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x557f1f687d80 0x557f1fa8c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.166+0000 7f8f77fff640 10 client.15612 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:57:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.166+0000 7f8f9ceb5640 1 --2- 172.21.15.117:0/1528035610 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x557f1f687d80 0x557f1fa8c140 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da7b2abc2e7c4096 server_cookie=f73be7739ec4d19f in_seq=0 out_seq=0 2024-09-21T05:57:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.167+0000 7f8f77fff640 1 -- 172.21.15.117:0/1528035610 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557f1f9e5640 con 0x557f1f687d80 2024-09-21T05:57:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.167+0000 7f8f77fff640 10 client.15612 handle_command_reply: tid=0 2024-09-21T05:57:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.167+0000 7f8fa6086740 1 client.15612 shutdown 2024-09-21T05:57:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.167+0000 7f8fa6086740 1 -- 172.21.15.117:0/1528035610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f1f6c91b0 msgr2=0x557f1fa86420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.167+0000 7f8fa6086740 1 --2- 172.21.15.117:0/1528035610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f1f6c91b0 0x557f1fa86420 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400e4e0 tx=0x7f8f8400e780 comp rx=0 tx=0).stop 2024-09-21T05:57:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.167+0000 7f8fa6086740 1 -- 172.21.15.117:0/1528035610 shutdown_connections 2024-09-21T05:57:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.167+0000 7f8fa6086740 1 --2- 172.21.15.117:0/1528035610 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x557f1f687d80 0x557f1fa8c140 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.167+0000 7f8fa6086740 1 --2- 172.21.15.117:0/1528035610 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f1f6c91b0 0x557f1fa86420 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.167+0000 7f8f77fff640 0 client.15612 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:57:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.167+0000 7f8fa6086740 1 --2- 172.21.15.117:0/1528035610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f1f6a84d0 0x557f1fa7a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.167+0000 7f8fa6086740 1 --2- 172.21.15.117:0/1528035610 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1f6a02f0 0x557f1fa7a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.167+0000 7f8fa6086740 1 -- 172.21.15.117:0/1528035610 >> 172.21.15.117:0/1528035610 conn(0x557f1f6bd300 msgr2=0x557f1fa6f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.168+0000 7f8fa6086740 1 -- 172.21.15.117:0/1528035610 shutdown_connections 2024-09-21T05:57:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.168+0000 7f8fa6086740 1 -- 172.21.15.117:0/1528035610 wait complete. 2024-09-21T05:57:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.168+0000 7f8fa6086740 20 client.15612 trim_cache size 0 max 16384 2024-09-21T05:57:42.186 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.182+0000 7f8fa6086740 1 Processor -- start 2024-09-21T05:57:42.186 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.182+0000 7f8fa6086740 1 -- start start 2024-09-21T05:57:42.186 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.182+0000 7f8fa6086740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1f687d80 0x557f1fa7b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:42.186 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.182+0000 7f8fa6086740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f1f6a02f0 0x557f1fa7a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:42.186 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.182+0000 7f8fa6086740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f1f6a84d0 0x557f1fa6beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.182+0000 7f8fa6086740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8f981af490 con 0x557f1f6a02f0 2024-09-21T05:57:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.182+0000 7f8fa6086740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8f981b0fd0 con 0x557f1f6a84d0 2024-09-21T05:57:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.182+0000 7f8fa6086740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8f981afb60 con 0x557f1f687d80 2024-09-21T05:57:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.182+0000 7f8f9ceb5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f1f6a02f0 0x557f1fa7a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.183+0000 7f8f9ceb5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f1f6a02f0 0x557f1fa7a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33264/0 (socket says 172.21.15.117:33264) 2024-09-21T05:57:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.183+0000 7f8f9ceb5640 1 -- 172.21.15.117:0/1854311150 learned_addr learned my addr 172.21.15.117:0/1854311150 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.183+0000 7f8f97fff640 1 --2- 172.21.15.117:0/1854311150 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1f687d80 0x557f1fa7b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.183+0000 7f8f97fff640 1 --2- 172.21.15.117:0/1854311150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f1f6a84d0 0x557f1fa6beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.183+0000 7f8f9ceb5640 1 -- 172.21.15.117:0/1854311150 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1f687d80 msgr2=0x557f1fa7b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.183+0000 7f8f9ceb5640 1 --2- 172.21.15.117:0/1854311150 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1f687d80 0x557f1fa7b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.183+0000 7f8f9ceb5640 1 -- 172.21.15.117:0/1854311150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f1f6a84d0 msgr2=0x557f1fa6beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.183+0000 7f8f9ceb5640 1 --2- 172.21.15.117:0/1854311150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f1f6a84d0 0x557f1fa6beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.183+0000 7f8f9ceb5640 1 -- 172.21.15.117:0/1854311150 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f1fa7c500 con 0x557f1f6a02f0 2024-09-21T05:57:42.187 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.183+0000 7f8f97fff640 1 --2- 172.21.15.117:0/1854311150 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1f687d80 0x557f1fa7b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:57:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.183+0000 7f8f9ceb5640 1 --2- 172.21.15.117:0/1854311150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f1f6a02f0 0x557f1fa7a610 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c004700 tx=0x7f8f8c0129e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d01d2ba715d0ae0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.183+0000 7f8f827fc640 1 -- 172.21.15.117:0/1854311150 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c005220 con 0x557f1f6a02f0 2024-09-21T05:57:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.183+0000 7f8f827fc640 1 -- 172.21.15.117:0/1854311150 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f8c004c50 con 0x557f1f6a02f0 2024-09-21T05:57:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.183+0000 7f8f827fc640 1 -- 172.21.15.117:0/1854311150 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c011be0 con 0x557f1f6a02f0 2024-09-21T05:57:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.184+0000 7f8fa6086740 1 -- 172.21.15.117:0/1854311150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f1f6a02f0 msgr2=0x557f1fa7a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.184+0000 7f8fa6086740 1 --2- 172.21.15.117:0/1854311150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f1f6a02f0 0x557f1fa7a610 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c004700 tx=0x7f8f8c0129e0 comp rx=0 tx=0).stop 2024-09-21T05:57:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.184+0000 7f8fa6086740 1 -- 172.21.15.117:0/1854311150 shutdown_connections 2024-09-21T05:57:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.184+0000 7f8fa6086740 1 --2- 172.21.15.117:0/1854311150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f1f6a02f0 0x557f1fa7a610 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.185+0000 7f8fa6086740 1 --2- 172.21.15.117:0/1854311150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f1f6a84d0 0x557f1fa6beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:42.191 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.185+0000 7f8fa6086740 1 --2- 172.21.15.117:0/1854311150 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1f687d80 0x557f1fa7b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.185+0000 7f8fa6086740 1 -- 172.21.15.117:0/1854311150 >> 172.21.15.117:0/1854311150 conn(0x557f1f6bd300 msgr2=0x557f1fa703a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.185+0000 7f8fa6086740 1 -- 172.21.15.117:0/1854311150 shutdown_connections 2024-09-21T05:57:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.185+0000 7f8fa6086740 1 -- 172.21.15.117:0/1854311150 wait complete. 2024-09-21T05:57:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.185+0000 7f8fa6086740 1 -- start start 2024-09-21T05:57:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.185+0000 7f8fa6086740 1 Processor -- start 2024-09-21T05:57:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.186+0000 7f8fa6086740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f1f687d80 0x557f1fa82c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.186+0000 7f8fa6086740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1f6a02f0 0x557f1fa7e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.186+0000 7f8fa6086740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f1f6a84d0 0x557f1fa7e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.186+0000 7f8fa6086740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8f981b0900 con 0x557f1f687d80 2024-09-21T05:57:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.186+0000 7f8fa6086740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8f981b0230 con 0x557f1f6a84d0 2024-09-21T05:57:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.186+0000 7f8fa6086740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8f981ad470 con 0x557f1f6a02f0 2024-09-21T05:57:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.186+0000 7f8fa6086740 10 client.0 _ll_register_callbacks cb 0x7f8f8400bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:57:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.186+0000 7f8f97fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f1f687d80 0x557f1fa82c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.186+0000 7f8f97fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f1f687d80 0x557f1fa82c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33272/0 (socket says 172.21.15.117:33272) 2024-09-21T05:57:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.186+0000 7f8f97fff640 1 -- 172.21.15.117:0/1409512353 learned_addr learned my addr 172.21.15.117:0/1409512353 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.186+0000 7f8f9ceb5640 1 --2- 172.21.15.117:0/1409512353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1f6a02f0 0x557f1fa7e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.186+0000 7f8f97fff640 1 --2- 172.21.15.117:0/1409512353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f1f6a84d0 0x557f1fa7e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.186+0000 7f8f97fff640 1 -- 172.21.15.117:0/1409512353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1f6a02f0 msgr2=0x557f1fa7e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.186+0000 7f8f97fff640 1 --2- 172.21.15.117:0/1409512353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1f6a02f0 0x557f1fa7e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.186+0000 7f8f97fff640 1 -- 172.21.15.117:0/1409512353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f1f6a84d0 msgr2=0x557f1fa7e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.186+0000 7f8f97fff640 1 --2- 172.21.15.117:0/1409512353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f1f6a84d0 0x557f1fa7e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.186+0000 7f8f97fff640 1 -- 172.21.15.117:0/1409512353 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f1fa7ef10 con 0x557f1f687d80 2024-09-21T05:57:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.186+0000 7f8f97fff640 1 --2- 172.21.15.117:0/1409512353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f1f687d80 0x557f1fa82c60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400e7f0 tx=0x7f8f84004890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6601428ea16dd991 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.186+0000 7f8f77fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:57:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.187+0000 7f8f77fff640 1 -- 172.21.15.117:0/1409512353 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8403c060 con 0x557f1f687d80 2024-09-21T05:57:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.187+0000 7f8f77fff640 1 -- 172.21.15.117:0/1409512353 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f8f8400eb30 con 0x557f1f687d80 2024-09-21T05:57:42.193 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.187+0000 7f8fa6086740 1 -- 172.21.15.117:0/1409512353 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f8c004c50 con 0x557f1f687d80 2024-09-21T05:57:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.187+0000 7f8f77fff640 1 -- 172.21.15.117:0/1409512353 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8f84018040 con 0x557f1f687d80 2024-09-21T05:57:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.187+0000 7f8fa6086740 10 client.15618 fetch_fsmap learned FSMap version 27 2024-09-21T05:57:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.187+0000 7f8fa6086740 1 -- 172.21.15.117:0/1409512353 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557f1faf6960 con 0x557f1f687d80 2024-09-21T05:57:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.187+0000 7f8f77fff640 1 -- 172.21.15.117:0/1409512353 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f8c004c50 con 0x557f1f687d80 2024-09-21T05:57:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.187+0000 7f8f77fff640 1 -- 172.21.15.117:0/1409512353 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8f84029050 con 0x557f1f687d80 2024-09-21T05:57:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.187+0000 7f8fa6086740 10 client.15618 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:57:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.187+0000 7f8fa6086740 10 client.15618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T05:57:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.187+0000 7f8fa6086740 20 client.15618 populate_metadata read hostname 'smithi117' 2024-09-21T05:57:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.187+0000 7f8fa6086740 1 --2- 172.21.15.117:0/1409512353 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x557f1f6c91b0 0x557f1faf9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.187+0000 7f8fa6086740 4 client.15618 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T05:57:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.187+0000 7f8fa6086740 1 -- 172.21.15.117:0/1409512353 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557f1f9e10f0 con 0x557f1f6c91b0 2024-09-21T05:57:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.188+0000 7f8f9ceb5640 1 --2- 172.21.15.117:0/1409512353 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x557f1f6c91b0 0x557f1faf9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:42.194 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.188+0000 7f8f77fff640 10 client.15618 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:57:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.188+0000 7f8f9ceb5640 1 --2- 172.21.15.117:0/1409512353 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x557f1f6c91b0 0x557f1faf9150 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3201c92b5ea406c server_cookie=239b34acb6d2272f in_seq=0 out_seq=0 2024-09-21T05:57:42.197 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.194+0000 7f8f77fff640 1 -- 172.21.15.117:0/1409512353 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557f1f9e10f0 con 0x557f1f6c91b0 2024-09-21T05:57:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.194+0000 7f8f77fff640 10 client.15618 handle_command_reply: tid=0 2024-09-21T05:57:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.194+0000 7f8fa6086740 1 client.15618 shutdown 2024-09-21T05:57:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.194+0000 7f8fa6086740 1 -- 172.21.15.117:0/1409512353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f1f687d80 msgr2=0x557f1fa82c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.194+0000 7f8fa6086740 1 --2- 172.21.15.117:0/1409512353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f1f687d80 0x557f1fa82c60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400e7f0 tx=0x7f8f84004890 comp rx=0 tx=0).stop 2024-09-21T05:57:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.194+0000 7f8fa6086740 1 -- 172.21.15.117:0/1409512353 shutdown_connections 2024-09-21T05:57:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.194+0000 7f8fa6086740 1 --2- 172.21.15.117:0/1409512353 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x557f1f6c91b0 0x557f1faf9150 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.194+0000 7f8fa6086740 1 --2- 172.21.15.117:0/1409512353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557f1f6a84d0 0x557f1fa7e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.194+0000 7f8fa6086740 1 --2- 172.21.15.117:0/1409512353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f1f6a02f0 0x557f1fa7e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.194+0000 7f8fa6086740 1 --2- 172.21.15.117:0/1409512353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557f1f687d80 0x557f1fa82c60 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.194+0000 7f8f77fff640 0 client.15618 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:57:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.194+0000 7f8fa6086740 1 -- 172.21.15.117:0/1409512353 >> 172.21.15.117:0/1409512353 conn(0x557f1f6bd300 msgr2=0x557f1fa70170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.195+0000 7f8fa6086740 1 -- 172.21.15.117:0/1409512353 shutdown_connections 2024-09-21T05:57:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.195+0000 7f8fa6086740 1 -- 172.21.15.117:0/1409512353 wait complete. 2024-09-21T05:57:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.195+0000 7f8fa6086740 20 client.15618 trim_cache size 0 max 16384 2024-09-21T05:57:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T05:57:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T05:57:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T05:57:42.199 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T05:57:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.195+0000 7f8f9f119640 1 -- 172.21.15.117:0/3714834602 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8f880b65f0 msgr2=0x7f8f880b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.195+0000 7f8f9f119640 1 --2- 172.21.15.117:0/3714834602 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8f880b65f0 0x7f8f880b8b00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8f900c8fd0 tx=0x7f8f9009b690 comp rx=0 tx=0).stop 2024-09-21T05:57:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.195+0000 7f8f9f119640 1 -- 172.21.15.117:0/3714834602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f98108c30 msgr2=0x7f8f9819fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.195+0000 7f8f9f119640 1 --2- 172.21.15.117:0/3714834602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f98108c30 0x7f8f9819fc90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8f900098d0 tx=0x7f8f900317b0 comp rx=0 tx=0).stop 2024-09-21T05:57:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.195+0000 7f8f9f119640 1 -- 172.21.15.117:0/3714834602 shutdown_connections 2024-09-21T05:57:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.195+0000 7f8f9f119640 1 --2- 172.21.15.117:0/3714834602 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8f880b65f0 0x7f8f880b8b00 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.195+0000 7f8f9f119640 1 --2- 172.21.15.117:0/3714834602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f98108c30 0x7f8f9819fc90 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.195+0000 7f8f9f119640 1 --2- 172.21.15.117:0/3714834602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f981082e0 0x7f8f9819aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.195+0000 7f8f9f119640 1 --2- 172.21.15.117:0/3714834602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f981029b0 0x7f8f9819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.195+0000 7f8f9f119640 1 -- 172.21.15.117:0/3714834602 >> 172.21.15.117:0/3714834602 conn(0x7f8f9807a740 msgr2=0x7f8f981019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.196+0000 7f8f9f119640 1 -- 172.21.15.117:0/3714834602 shutdown_connections 2024-09-21T05:57:42.200 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:42.196+0000 7f8f9f119640 1 -- 172.21.15.117:0/3714834602 wait complete. 2024-09-21T05:57:43.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:43 smithi192 ceph-mon[34700]: pgmap v339: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-21T05:57:44.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:43 smithi117 ceph-mon[30947]: pgmap v339: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-21T05:57:44.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:43 smithi138 ceph-mon[34391]: pgmap v339: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-21T05:57:45.917 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:45 smithi138 ceph-mon[34391]: pgmap v340: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-09-21T05:57:45.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:45 smithi192 ceph-mon[34700]: pgmap v340: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-09-21T05:57:46.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:45 smithi117 ceph-mon[30947]: pgmap v340: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-09-21T05:57:47.112 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T05:57:47.464 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:57:47.742 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:47 smithi117 ceph-mon[30947]: pgmap v341: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-21T05:57:47.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:47 smithi192 ceph-mon[34700]: pgmap v341: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-21T05:57:48.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.005+0000 7fbf59d9b640 1 -- 172.21.15.117:0/2131792108 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c0308f0 con 0x7fbf5c1021f0 2024-09-21T05:57:48.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.005+0000 7fbf6106e640 1 -- 172.21.15.117:0/2131792108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf5c1021f0 msgr2=0x7fbf5c102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:48.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.005+0000 7fbf6106e640 1 --2- 172.21.15.117:0/2131792108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf5c1021f0 0x7fbf5c102600 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c009860 tx=0x7fbf4c02a360 comp rx=0 tx=0).stop 2024-09-21T05:57:48.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.005+0000 7fbf6106e640 1 -- 172.21.15.117:0/2131792108 shutdown_connections 2024-09-21T05:57:48.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.005+0000 7fbf6106e640 1 --2- 172.21.15.117:0/2131792108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf5c108480 0x7fbf5c10ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:48.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.005+0000 7fbf6106e640 1 --2- 172.21.15.117:0/2131792108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf5c107b30 0x7fbf5c107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:48.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.005+0000 7fbf6106e640 1 --2- 172.21.15.117:0/2131792108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf5c1021f0 0x7fbf5c102600 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:48.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.005+0000 7fbf6106e640 1 -- 172.21.15.117:0/2131792108 >> 172.21.15.117:0/2131792108 conn(0x7fbf5c077c60 msgr2=0x7fbf5c0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:48.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.005+0000 7fbf6106e640 1 -- 172.21.15.117:0/2131792108 shutdown_connections 2024-09-21T05:57:48.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.005+0000 7fbf6106e640 1 -- 172.21.15.117:0/2131792108 wait complete. 2024-09-21T05:57:48.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.006+0000 7fbf6106e640 1 Processor -- start 2024-09-21T05:57:48.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.007+0000 7fbf6106e640 1 -- start start 2024-09-21T05:57:48.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.007+0000 7fbf6106e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf5c1021f0 0x7fbf5c19a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:48.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.007+0000 7fbf6106e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf5c107b30 0x7fbf5c19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:48.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.007+0000 7fbf6106e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf5c108480 0x7fbf5c19fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:48.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.007+0000 7fbf6106e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c113b10 con 0x7fbf5c108480 2024-09-21T05:57:48.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.007+0000 7fbf6106e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c113990 con 0x7fbf5c107b30 2024-09-21T05:57:48.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.007+0000 7fbf6106e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c113c90 con 0x7fbf5c1021f0 2024-09-21T05:57:48.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.007+0000 7fbf5b59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf5c108480 0x7fbf5c19fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:48.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.007+0000 7fbf5b59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf5c108480 0x7fbf5c19fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34530/0 (socket says 172.21.15.117:34530) 2024-09-21T05:57:48.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.007+0000 7fbf5b59e640 1 -- 172.21.15.117:0/754525055 learned_addr learned my addr 172.21.15.117:0/754525055 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:48.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.007+0000 7fbf5a59c640 1 --2- 172.21.15.117:0/754525055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf5c107b30 0x7fbf5c19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:48.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.007+0000 7fbf5ad9d640 1 --2- 172.21.15.117:0/754525055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf5c1021f0 0x7fbf5c19a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:48.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.008+0000 7fbf5ad9d640 1 -- 172.21.15.117:0/754525055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf5c107b30 msgr2=0x7fbf5c19aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:48.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.008+0000 7fbf5ad9d640 1 --2- 172.21.15.117:0/754525055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf5c107b30 0x7fbf5c19aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:48.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.008+0000 7fbf5ad9d640 1 -- 172.21.15.117:0/754525055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf5c108480 msgr2=0x7fbf5c19fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:48.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.008+0000 7fbf5ad9d640 1 --2- 172.21.15.117:0/754525055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf5c108480 0x7fbf5c19fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:48.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.008+0000 7fbf5ad9d640 1 -- 172.21.15.117:0/754525055 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf5c1a0490 con 0x7fbf5c1021f0 2024-09-21T05:57:48.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.008+0000 7fbf5ad9d640 1 --2- 172.21.15.117:0/754525055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf5c1021f0 0x7fbf5c19a520 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c005940 tx=0x7fbf4c032cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:48.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.008+0000 7fbf43fff640 1 -- 172.21.15.117:0/754525055 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c037050 con 0x7fbf5c1021f0 2024-09-21T05:57:48.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.009+0000 7fbf6106e640 1 -- 172.21.15.117:0/754525055 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf5c1a0720 con 0x7fbf5c1021f0 2024-09-21T05:57:48.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.009+0000 7fbf43fff640 1 -- 172.21.15.117:0/754525055 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf4c038040 con 0x7fbf5c1021f0 2024-09-21T05:57:48.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.009+0000 7fbf43fff640 1 -- 172.21.15.117:0/754525055 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c0039d0 con 0x7fbf5c1021f0 2024-09-21T05:57:48.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.009+0000 7fbf6106e640 1 -- 172.21.15.117:0/754525055 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf5c110300 con 0x7fbf5c1021f0 2024-09-21T05:57:48.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.012+0000 7fbf43fff640 1 -- 172.21.15.117:0/754525055 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 23) ==== 154587+0+0 (secure 0 0 0) 0x7fbf4c045030 con 0x7fbf5c1021f0 2024-09-21T05:57:48.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.012+0000 7fbf675e2740 1 Processor -- start 2024-09-21T05:57:48.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.012+0000 7fbf675e2740 1 -- start start 2024-09-21T05:57:48.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.013+0000 7fbf675e2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ecba1d070 0x555ecbdc4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:48.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.013+0000 7fbf675e2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ecb9f41b0 0x555ecbdc47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:48.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.013+0000 7fbf675e2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555ecb9fc390 0x555ecbdc4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:48.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.013+0000 7fbf675e2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555ecbd3ae50 con 0x555ecb9f41b0 2024-09-21T05:57:48.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.013+0000 7fbf675e2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555ecbd2a280 con 0x555ecba1d070 2024-09-21T05:57:48.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.013+0000 7fbf675e2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555ecbd2e090 con 0x555ecb9fc390 2024-09-21T05:57:48.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.013+0000 7fbf5b59e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ecba1d070 0x555ecbdc4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:48.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.013+0000 7fbf5b59e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ecba1d070 0x555ecbdc4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:60354/0 (socket says 172.21.15.117:60354) 2024-09-21T05:57:48.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.013+0000 7fbf5b59e640 1 -- 172.21.15.117:0/889847297 learned_addr learned my addr 172.21.15.117:0/889847297 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:48.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.013+0000 7fbf43fff640 1 --2- 172.21.15.117:0/754525055 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbf440b6670 0x7fbf440b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:48.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.013+0000 7fbf5ad9d640 1 --2- 172.21.15.117:0/889847297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ecb9f41b0 0x555ecbdc47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:48.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.013+0000 7fbf5b59e640 1 -- 172.21.15.117:0/889847297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555ecb9fc390 msgr2=0x555ecbdc4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:48.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.013+0000 7fbf5b59e640 1 --2- 172.21.15.117:0/889847297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555ecb9fc390 0x555ecbdc4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:48.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.013+0000 7fbf5b59e640 1 -- 172.21.15.117:0/889847297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ecb9f41b0 msgr2=0x555ecbdc47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:48.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.013+0000 7fbf5b59e640 1 --2- 172.21.15.117:0/889847297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ecb9f41b0 0x555ecbdc47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:48.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.013+0000 7fbf5b59e640 1 -- 172.21.15.117:0/889847297 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ecba88650 con 0x555ecba1d070 2024-09-21T05:57:48.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.014+0000 7fbf43fff640 1 -- 172.21.15.117:0/754525055 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fbf4c106f70 con 0x7fbf5c1021f0 2024-09-21T05:57:48.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.014+0000 7fbf5b59e640 1 --2- 172.21.15.117:0/754525055 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbf440b6670 0x7fbf440b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:48.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.014+0000 7fbf5b59e640 1 --2- 172.21.15.117:0/889847297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ecba1d070 0x555ecbdc4270 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbf5400e660 tx=0x7fbf5400eb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35deec5aa5635c37 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:48.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.014+0000 7fbf417fa640 1 -- 172.21.15.117:0/889847297 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf54019440 con 0x555ecba1d070 2024-09-21T05:57:48.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.014+0000 7fbf417fa640 1 -- 172.21.15.117:0/889847297 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf54019b10 con 0x555ecba1d070 2024-09-21T05:57:48.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.016+0000 7fbf417fa640 1 -- 172.21.15.117:0/889847297 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf54024cb0 con 0x555ecba1d070 2024-09-21T05:57:48.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.016+0000 7fbf675e2740 1 -- 172.21.15.117:0/889847297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ecba1d070 msgr2=0x555ecbdc4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:48.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.016+0000 7fbf675e2740 1 --2- 172.21.15.117:0/889847297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ecba1d070 0x555ecbdc4270 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbf5400e660 tx=0x7fbf5400eb20 comp rx=0 tx=0).stop 2024-09-21T05:57:48.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.016+0000 7fbf675e2740 1 -- 172.21.15.117:0/889847297 shutdown_connections 2024-09-21T05:57:48.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.016+0000 7fbf675e2740 1 --2- 172.21.15.117:0/889847297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ecb9f41b0 0x555ecbdc47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:48.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.016+0000 7fbf675e2740 1 --2- 172.21.15.117:0/889847297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555ecb9fc390 0x555ecbdc4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:48.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.016+0000 7fbf675e2740 1 --2- 172.21.15.117:0/889847297 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ecba1d070 0x555ecbdc4270 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:48.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.016+0000 7fbf675e2740 1 -- 172.21.15.117:0/889847297 >> 172.21.15.117:0/889847297 conn(0x555ecba11460 msgr2=0x555ecbdc2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:48.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.016+0000 7fbf675e2740 1 -- 172.21.15.117:0/889847297 shutdown_connections 2024-09-21T05:57:48.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.016+0000 7fbf5b59e640 1 --2- 172.21.15.117:0/754525055 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbf440b6670 0x7fbf440b8b80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbf54023000 tx=0x7fbf54024f70 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:48.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.016+0000 7fbf675e2740 1 -- 172.21.15.117:0/889847297 wait complete. 2024-09-21T05:57:48.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.017+0000 7fbf675e2740 1 -- start start 2024-09-21T05:57:48.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.017+0000 7fbf675e2740 1 Processor -- start 2024-09-21T05:57:48.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.017+0000 7fbf675e2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ecb9f41b0 0x555ecbdd8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:48.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.017+0000 7fbf675e2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ecb9fc390 0x555ecbdd8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:48.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.017+0000 7fbf675e2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555ecba1d070 0x555ecbdd8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:48.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.017+0000 7fbf5ad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ecb9f41b0 0x555ecbdd8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:48.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.017+0000 7fbf675e2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555ecbb8b700 con 0x555ecb9f41b0 2024-09-21T05:57:48.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.017+0000 7fbf5a59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ecb9fc390 0x555ecbdd8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:48.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.017+0000 7fbf5ad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ecb9f41b0 0x555ecbdd8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34556/0 (socket says 172.21.15.117:34556) 2024-09-21T05:57:48.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.017+0000 7fbf5a59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ecb9fc390 0x555ecbdd8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:60364/0 (socket says 172.21.15.117:60364) 2024-09-21T05:57:48.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.017+0000 7fbf5ad9d640 1 -- 172.21.15.117:0/520562676 learned_addr learned my addr 172.21.15.117:0/520562676 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:48.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.018+0000 7fbf675e2740 1 -- 172.21.15.117:0/520562676 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555ecbcc5110 con 0x555ecb9fc390 2024-09-21T05:57:48.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.018+0000 7fbf675e2740 1 -- 172.21.15.117:0/520562676 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555ecbddb220 con 0x555ecba1d070 2024-09-21T05:57:48.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.018+0000 7fbf5b59e640 1 --2- 172.21.15.117:0/520562676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555ecba1d070 0x555ecbdd8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:48.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.018+0000 7fbf675e2740 10 client.0 _ll_register_callbacks cb 0x555ecbcde990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:57:48.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.018+0000 7fbf5ad9d640 1 -- 172.21.15.117:0/520562676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555ecba1d070 msgr2=0x555ecbdd8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:48.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.018+0000 7fbf5ad9d640 1 --2- 172.21.15.117:0/520562676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555ecba1d070 0x555ecbdd8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:48.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.018+0000 7fbf5ad9d640 1 -- 172.21.15.117:0/520562676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ecb9fc390 msgr2=0x555ecbdd8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:48.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.018+0000 7fbf5ad9d640 1 --2- 172.21.15.117:0/520562676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ecb9fc390 0x555ecbdd8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:48.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.018+0000 7fbf5ad9d640 1 -- 172.21.15.117:0/520562676 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ecbcf2c40 con 0x555ecb9f41b0 2024-09-21T05:57:48.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.018+0000 7fbf5a59c640 1 --2- 172.21.15.117:0/520562676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ecb9fc390 0x555ecbdd8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:57:48.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.018+0000 7fbf5b59e640 1 --2- 172.21.15.117:0/520562676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555ecba1d070 0x555ecbdd8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:57:48.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.018+0000 7fbf5ad9d640 1 --2- 172.21.15.117:0/520562676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ecb9f41b0 0x555ecbdd8050 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c030490 tx=0x7fbf4c0ccd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b95bc0ef8855d77 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:48.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.018+0000 7fbf3eff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:57:48.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.019+0000 7fbf3eff5640 1 -- 172.21.15.117:0/520562676 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c0cc700 con 0x555ecb9f41b0 2024-09-21T05:57:48.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.019+0000 7fbf3eff5640 1 -- 172.21.15.117:0/520562676 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fbf4c0cc1f0 con 0x555ecb9f41b0 2024-09-21T05:57:48.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.019+0000 7fbf675e2740 1 -- 172.21.15.117:0/520562676 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf54019b10 con 0x555ecb9f41b0 2024-09-21T05:57:48.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.019+0000 7fbf3eff5640 1 -- 172.21.15.117:0/520562676 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbf4c032700 con 0x555ecb9f41b0 2024-09-21T05:57:48.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.019+0000 7fbf675e2740 10 client.15627 fetch_fsmap learned FSMap version 27 2024-09-21T05:57:48.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.019+0000 7fbf675e2740 1 -- 172.21.15.117:0/520562676 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555ecbde6ec0 con 0x555ecb9f41b0 2024-09-21T05:57:48.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.020+0000 7fbf3eff5640 1 -- 172.21.15.117:0/520562676 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf54019b10 con 0x555ecb9f41b0 2024-09-21T05:57:48.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.020+0000 7fbf3eff5640 1 -- 172.21.15.117:0/520562676 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbf4c0b81a0 con 0x555ecb9f41b0 2024-09-21T05:57:48.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.020+0000 7fbf675e2740 10 client.15627 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:57:48.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.020+0000 7fbf675e2740 10 client.15627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T05:57:48.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.020+0000 7fbf675e2740 20 client.15627 populate_metadata read hostname 'smithi117' 2024-09-21T05:57:48.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.020+0000 7fbf675e2740 1 --2- 172.21.15.117:0/520562676 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555ecb9db9f0 0x555ecbdea390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:48.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.020+0000 7fbf675e2740 4 client.15627 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:57:48.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.020+0000 7fbf675e2740 1 -- 172.21.15.117:0/520562676 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ecbd3cda0 con 0x555ecb9db9f0 2024-09-21T05:57:48.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.020+0000 7fbf5a59c640 1 --2- 172.21.15.117:0/520562676 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555ecb9db9f0 0x555ecbdea390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:48.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.021+0000 7fbf5a59c640 1 --2- 172.21.15.117:0/520562676 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555ecb9db9f0 0x555ecbdea390 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25179ee5cf24cd3 server_cookie=48e2388bd4550c2e in_seq=0 out_seq=0 2024-09-21T05:57:48.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.021+0000 7fbf3eff5640 10 client.15627 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:57:48.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.021+0000 7fbf3eff5640 1 -- 172.21.15.117:0/520562676 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555ecbd3cda0 con 0x555ecb9db9f0 2024-09-21T05:57:48.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.021+0000 7fbf3eff5640 10 client.15627 handle_command_reply: tid=0 2024-09-21T05:57:48.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.022+0000 7fbf675e2740 1 client.15627 shutdown 2024-09-21T05:57:48.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.022+0000 7fbf675e2740 1 -- 172.21.15.117:0/520562676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ecb9f41b0 msgr2=0x555ecbdd8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:48.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.022+0000 7fbf675e2740 1 --2- 172.21.15.117:0/520562676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ecb9f41b0 0x555ecbdd8050 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c030490 tx=0x7fbf4c0ccd60 comp rx=0 tx=0).stop 2024-09-21T05:57:48.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.022+0000 7fbf675e2740 1 -- 172.21.15.117:0/520562676 shutdown_connections 2024-09-21T05:57:48.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.022+0000 7fbf675e2740 1 --2- 172.21.15.117:0/520562676 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555ecb9db9f0 0x555ecbdea390 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:48.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.022+0000 7fbf675e2740 1 --2- 172.21.15.117:0/520562676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555ecba1d070 0x555ecbdd8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:48.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.022+0000 7fbf675e2740 1 --2- 172.21.15.117:0/520562676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ecb9fc390 0x555ecbdd8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:48.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.022+0000 7fbf675e2740 1 --2- 172.21.15.117:0/520562676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ecb9f41b0 0x555ecbdd8050 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:48.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.022+0000 7fbf675e2740 1 -- 172.21.15.117:0/520562676 >> 172.21.15.117:0/520562676 conn(0x555ecba11460 msgr2=0x555ecbdc0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:48.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.022+0000 7fbf3eff5640 0 client.15627 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:57:48.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.022+0000 7fbf675e2740 1 -- 172.21.15.117:0/520562676 shutdown_connections 2024-09-21T05:57:48.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.022+0000 7fbf675e2740 1 -- 172.21.15.117:0/520562676 wait complete. 2024-09-21T05:57:48.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.023+0000 7fbf675e2740 20 client.15627 trim_cache size 0 max 16384 2024-09-21T05:57:48.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.040+0000 7fbf675e2740 1 Processor -- start 2024-09-21T05:57:48.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.040+0000 7fbf675e2740 1 -- start start 2024-09-21T05:57:48.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.041+0000 7fbf675e2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ecb9f41b0 0x555ecbdc2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:48.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.041+0000 7fbf675e2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ecb9fc390 0x555ecbdc59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:48.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.041+0000 7fbf675e2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555ecba1d070 0x555ecbdd5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:48.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.041+0000 7fbf675e2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c1ad2a0 con 0x555ecb9f41b0 2024-09-21T05:57:48.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.041+0000 7fbf675e2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c1aede0 con 0x555ecb9fc390 2024-09-21T05:57:48.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.041+0000 7fbf675e2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c1ad970 con 0x555ecba1d070 2024-09-21T05:57:48.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.041+0000 7fbf5ad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ecb9f41b0 0x555ecbdc2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:48.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.041+0000 7fbf5ad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ecb9f41b0 0x555ecbdc2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34568/0 (socket says 172.21.15.117:34568) 2024-09-21T05:57:48.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.041+0000 7fbf5ad9d640 1 -- 172.21.15.117:0/4008132295 learned_addr learned my addr 172.21.15.117:0/4008132295 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:48.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.041+0000 7fbf5a59c640 1 --2- 172.21.15.117:0/4008132295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ecb9fc390 0x555ecbdc59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:48.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.041+0000 7fbf5b59e640 1 --2- 172.21.15.117:0/4008132295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555ecba1d070 0x555ecbdd5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:48.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.041+0000 7fbf5ad9d640 1 -- 172.21.15.117:0/4008132295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555ecba1d070 msgr2=0x555ecbdd5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:48.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.041+0000 7fbf5ad9d640 1 --2- 172.21.15.117:0/4008132295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555ecba1d070 0x555ecbdd5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:48.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.041+0000 7fbf5ad9d640 1 -- 172.21.15.117:0/4008132295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ecb9fc390 msgr2=0x555ecbdc59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:48.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.041+0000 7fbf5ad9d640 1 --2- 172.21.15.117:0/4008132295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ecb9fc390 0x555ecbdc59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:48.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.041+0000 7fbf5ad9d640 1 -- 172.21.15.117:0/4008132295 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ecbdbc360 con 0x555ecb9f41b0 2024-09-21T05:57:48.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.041+0000 7fbf5a59c640 1 --2- 172.21.15.117:0/4008132295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ecb9fc390 0x555ecbdc59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:57:48.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.042+0000 7fbf5ad9d640 1 --2- 172.21.15.117:0/4008132295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ecb9f41b0 0x555ecbdc2100 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0ccc90 tx=0x7fbf4c0cce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6414349ec4c60037 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:48.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.042+0000 7fbf40ff9640 1 -- 172.21.15.117:0/4008132295 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c0cc700 con 0x555ecb9f41b0 2024-09-21T05:57:48.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.042+0000 7fbf40ff9640 1 -- 172.21.15.117:0/4008132295 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf4c0cd690 con 0x555ecb9f41b0 2024-09-21T05:57:48.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.042+0000 7fbf40ff9640 1 -- 172.21.15.117:0/4008132295 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c0cc8e0 con 0x555ecb9f41b0 2024-09-21T05:57:48.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.043+0000 7fbf675e2740 1 -- 172.21.15.117:0/4008132295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ecb9f41b0 msgr2=0x555ecbdc2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:48.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.043+0000 7fbf675e2740 1 --2- 172.21.15.117:0/4008132295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ecb9f41b0 0x555ecbdc2100 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0ccc90 tx=0x7fbf4c0cce90 comp rx=0 tx=0).stop 2024-09-21T05:57:48.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.043+0000 7fbf675e2740 1 -- 172.21.15.117:0/4008132295 shutdown_connections 2024-09-21T05:57:48.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.043+0000 7fbf675e2740 1 --2- 172.21.15.117:0/4008132295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555ecba1d070 0x555ecbdd5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:48.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.043+0000 7fbf675e2740 1 --2- 172.21.15.117:0/4008132295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ecb9fc390 0x555ecbdc59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:48.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.043+0000 7fbf675e2740 1 --2- 172.21.15.117:0/4008132295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ecb9f41b0 0x555ecbdc2100 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:48.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.043+0000 7fbf675e2740 1 -- 172.21.15.117:0/4008132295 >> 172.21.15.117:0/4008132295 conn(0x555ecba11460 msgr2=0x555ecbdc52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:48.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.043+0000 7fbf675e2740 1 -- 172.21.15.117:0/4008132295 shutdown_connections 2024-09-21T05:57:48.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.043+0000 7fbf675e2740 1 -- 172.21.15.117:0/4008132295 wait complete. 2024-09-21T05:57:48.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.044+0000 7fbf675e2740 1 -- start start 2024-09-21T05:57:48.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.044+0000 7fbf675e2740 1 Processor -- start 2024-09-21T05:57:48.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.045+0000 7fbf675e2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555ecb9f41b0 0x555ecbddc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:48.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.045+0000 7fbf675e2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ecb9fc390 0x555ecbddcca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:48.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.045+0000 7fbf675e2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ecba1d070 0x555ecbddd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:48.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.045+0000 7fbf5ad9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555ecb9f41b0 0x555ecbddc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:48.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.045+0000 7fbf5a59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ecb9fc390 0x555ecbddcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:48.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.045+0000 7fbf5a59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ecb9fc390 0x555ecbddcca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:60382/0 (socket says 172.21.15.117:60382) 2024-09-21T05:57:48.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.045+0000 7fbf5ad9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555ecb9f41b0 0x555ecbddc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:52002/0 (socket says 172.21.15.117:52002) 2024-09-21T05:57:48.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.046+0000 7fbf5b59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ecba1d070 0x555ecbddd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:48.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.046+0000 7fbf5b59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ecba1d070 0x555ecbddd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34570/0 (socket says 172.21.15.117:34570) 2024-09-21T05:57:48.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.046+0000 7fbf5a59c640 1 -- 172.21.15.117:0/3929279037 learned_addr learned my addr 172.21.15.117:0/3929279037 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:48.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.046+0000 7fbf675e2740 1 -- 172.21.15.117:0/3929279037 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c1ae710 con 0x555ecba1d070 2024-09-21T05:57:48.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.046+0000 7fbf675e2740 1 -- 172.21.15.117:0/3929279037 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c1ae040 con 0x555ecb9fc390 2024-09-21T05:57:48.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.046+0000 7fbf675e2740 1 -- 172.21.15.117:0/3929279037 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c1ab2a0 con 0x555ecb9f41b0 2024-09-21T05:57:48.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.046+0000 7fbf5ad9d640 1 -- 172.21.15.117:0/3929279037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ecb9fc390 msgr2=0x555ecbddcca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:48.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.047+0000 7fbf5ad9d640 1 --2- 172.21.15.117:0/3929279037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ecb9fc390 0x555ecbddcca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:48.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.047+0000 7fbf5ad9d640 1 -- 172.21.15.117:0/3929279037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ecba1d070 msgr2=0x555ecbddd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:48.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.047+0000 7fbf5ad9d640 1 --2- 172.21.15.117:0/3929279037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ecba1d070 0x555ecbddd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:48.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.047+0000 7fbf5ad9d640 1 -- 172.21.15.117:0/3929279037 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ecbe53fa0 con 0x555ecb9f41b0 2024-09-21T05:57:48.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.047+0000 7fbf5a59c640 1 --2- 172.21.15.117:0/3929279037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ecb9fc390 0x555ecbddcca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:57:48.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.047+0000 7fbf5b59e640 1 --2- 172.21.15.117:0/3929279037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ecba1d070 0x555ecbddd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:57:48.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.047+0000 7fbf675e2740 10 client.0 _ll_register_callbacks cb 0x7fbf4c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:57:48.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.047+0000 7fbf675e2740 1 -- 172.21.15.117:0/3929279037 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf4c0cd690 con 0x555ecb9f41b0 2024-09-21T05:57:48.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.047+0000 7fbf5ad9d640 1 --2- 172.21.15.117:0/3929279037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555ecb9f41b0 0x555ecbddc760 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c009830 tx=0x7fbf4c0ccd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5249ff4a27080465 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:48.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.047+0000 7fbf3eff5640 10 client.25168 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-21T05:57:48.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.047+0000 7fbf3eff5640 1 -- 172.21.15.117:0/3929279037 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c0c8e30 con 0x555ecb9f41b0 2024-09-21T05:57:48.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.047+0000 7fbf3eff5640 1 -- 172.21.15.117:0/3929279037 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fbf4c10a310 con 0x555ecb9f41b0 2024-09-21T05:57:48.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.048+0000 7fbf3eff5640 1 -- 172.21.15.117:0/3929279037 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbf4c0cc8e0 con 0x555ecb9f41b0 2024-09-21T05:57:48.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.048+0000 7fbf675e2740 10 client.25168 fetch_fsmap learned FSMap version 27 2024-09-21T05:57:48.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.048+0000 7fbf675e2740 1 -- 172.21.15.117:0/3929279037 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({fsmap=27}) -- 0x555ecbe53fa0 con 0x555ecb9f41b0 2024-09-21T05:57:48.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.048+0000 7fbf3eff5640 1 -- 172.21.15.117:0/3929279037 <== mon.1 v2:172.21.15.192:3300/0 4 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbf4c0f4170 con 0x555ecb9f41b0 2024-09-21T05:57:48.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.048+0000 7fbf675e2740 10 client.25168 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:57:48.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.048+0000 7fbf675e2740 10 client.25168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T05:57:48.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.048+0000 7fbf675e2740 20 client.25168 populate_metadata read hostname 'smithi117' 2024-09-21T05:57:48.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.048+0000 7fbf675e2740 1 --2- 172.21.15.117:0/3929279037 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555ecb9b0da0 0x555ecbe57200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:48.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.048+0000 7fbf675e2740 4 client.25168 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T05:57:48.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.048+0000 7fbf675e2740 1 -- 172.21.15.117:0/3929279037 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555ecbd38850 con 0x555ecb9b0da0 2024-09-21T05:57:48.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.048+0000 7fbf5a59c640 1 --2- 172.21.15.117:0/3929279037 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555ecb9b0da0 0x555ecbe57200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:48.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.049+0000 7fbf3eff5640 10 client.25168 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:57:48.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.049+0000 7fbf5a59c640 1 --2- 172.21.15.117:0/3929279037 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555ecb9b0da0 0x555ecbe57200 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d0bc0fce929acc4 server_cookie=39f9c2331092fa82 in_seq=0 out_seq=0 2024-09-21T05:57:48.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.049+0000 7fbf3eff5640 1 -- 172.21.15.117:0/3929279037 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+374 (crc 0 0 0) 0x555ecbd38850 con 0x555ecb9b0da0 2024-09-21T05:57:48.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.049+0000 7fbf3eff5640 10 client.25168 handle_command_reply: tid=0 2024-09-21T05:57:48.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.049+0000 7fbf675e2740 1 client.25168 shutdown 2024-09-21T05:57:48.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.049+0000 7fbf675e2740 1 -- 172.21.15.117:0/3929279037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555ecb9f41b0 msgr2=0x555ecbddc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:48.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.049+0000 7fbf675e2740 1 --2- 172.21.15.117:0/3929279037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555ecb9f41b0 0x555ecbddc760 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c009830 tx=0x7fbf4c0ccd60 comp rx=0 tx=0).stop 2024-09-21T05:57:48.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.049+0000 7fbf675e2740 1 -- 172.21.15.117:0/3929279037 shutdown_connections 2024-09-21T05:57:48.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.049+0000 7fbf675e2740 1 --2- 172.21.15.117:0/3929279037 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555ecb9b0da0 0x555ecbe57200 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:48.055 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.049+0000 7fbf675e2740 1 --2- 172.21.15.117:0/3929279037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555ecba1d070 0x555ecbddd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:48.055 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T05:57:48.055 INFO:teuthology.orchestra.run.smithi117.stdout: "status": "no active scrubs running", 2024-09-21T05:57:48.055 INFO:teuthology.orchestra.run.smithi117.stdout: "scrubs": {}, 2024-09-21T05:57:48.055 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_stats": { 2024-09-21T05:57:48.055 INFO:teuthology.orchestra.run.smithi117.stdout: "b30bfff0-0f6b-4e30-8467-ff7db421eba6": { 2024-09-21T05:57:48.055 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:57:14.939082+0000", 2024-09-21T05:57:48.055 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T05:57:48.056 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T05:57:48.056 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T05:57:48.056 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T05:57:48.056 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T05:57:48.056 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T05:57:48.056 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T05:57:48.056 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T05:57:48.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.049+0000 7fbf675e2740 1 --2- 172.21.15.117:0/3929279037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ecb9fc390 0x555ecbddcca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:48.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.049+0000 7fbf3eff5640 0 client.25168 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:57:48.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.049+0000 7fbf675e2740 1 --2- 172.21.15.117:0/3929279037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555ecb9f41b0 0x555ecbddc760 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:48.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.049+0000 7fbf675e2740 1 -- 172.21.15.117:0/3929279037 >> 172.21.15.117:0/3929279037 conn(0x555ecba11460 msgr2=0x555ecbdc51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:48.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.050+0000 7fbf675e2740 1 -- 172.21.15.117:0/3929279037 shutdown_connections 2024-09-21T05:57:48.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.050+0000 7fbf675e2740 1 -- 172.21.15.117:0/3929279037 wait complete. 2024-09-21T05:57:48.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.050+0000 7fbf675e2740 20 client.25168 trim_cache size 0 max 16384 2024-09-21T05:57:48.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.050+0000 7fbf6106e640 1 -- 172.21.15.117:0/754525055 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbf440b6670 msgr2=0x7fbf440b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:48.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.050+0000 7fbf6106e640 1 --2- 172.21.15.117:0/754525055 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbf440b6670 0x7fbf440b8b80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbf54023000 tx=0x7fbf54024f70 comp rx=0 tx=0).stop 2024-09-21T05:57:48.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.050+0000 7fbf6106e640 1 -- 172.21.15.117:0/754525055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf5c1021f0 msgr2=0x7fbf5c19a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:48.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.050+0000 7fbf6106e640 1 --2- 172.21.15.117:0/754525055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf5c1021f0 0x7fbf5c19a520 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c005940 tx=0x7fbf4c032cd0 comp rx=0 tx=0).stop 2024-09-21T05:57:48.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.051+0000 7fbf6106e640 1 -- 172.21.15.117:0/754525055 shutdown_connections 2024-09-21T05:57:48.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.051+0000 7fbf6106e640 1 --2- 172.21.15.117:0/754525055 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbf440b6670 0x7fbf440b8b80 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:48.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.051+0000 7fbf6106e640 1 --2- 172.21.15.117:0/754525055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf5c108480 0x7fbf5c19fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:48.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.051+0000 7fbf6106e640 1 --2- 172.21.15.117:0/754525055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf5c107b30 0x7fbf5c19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:48.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.051+0000 7fbf6106e640 1 --2- 172.21.15.117:0/754525055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf5c1021f0 0x7fbf5c19a520 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:48.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.051+0000 7fbf6106e640 1 -- 172.21.15.117:0/754525055 >> 172.21.15.117:0/754525055 conn(0x7fbf5c077c60 msgr2=0x7fbf5c105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:48.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.051+0000 7fbf6106e640 1 -- 172.21.15.117:0/754525055 shutdown_connections 2024-09-21T05:57:48.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:48.051+0000 7fbf6106e640 1 -- 172.21.15.117:0/754525055 wait complete. 2024-09-21T05:57:48.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:47 smithi138 ceph-mon[34391]: pgmap v341: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-21T05:57:48.790 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T05:57:48.790 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T05:57:49.052 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:57:49.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.572+0000 7f4220d33640 1 -- 172.21.15.117:0/3085338467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f421c107b30 msgr2=0x7f421c107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:49.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.572+0000 7f4220d33640 1 --2- 172.21.15.117:0/3085338467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f421c107b30 0x7f421c107f40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4210009900 tx=0x7f421002a410 comp rx=0 tx=0).stop 2024-09-21T05:57:49.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.572+0000 7f4220d33640 1 -- 172.21.15.117:0/3085338467 shutdown_connections 2024-09-21T05:57:49.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.573+0000 7f4220d33640 1 --2- 172.21.15.117:0/3085338467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f421c107b30 0x7f421c107f40 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:49.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.573+0000 7f4220d33640 1 --2- 172.21.15.117:0/3085338467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f421c108480 0x7f421c10ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:49.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.573+0000 7f4220d33640 1 --2- 172.21.15.117:0/3085338467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f421c1021f0 0x7f421c102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:49.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.573+0000 7f4220d33640 1 -- 172.21.15.117:0/3085338467 >> 172.21.15.117:0/3085338467 conn(0x7f421c077c60 msgr2=0x7f421c0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:49.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.573+0000 7f4220d33640 1 -- 172.21.15.117:0/3085338467 shutdown_connections 2024-09-21T05:57:49.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.573+0000 7f4220d33640 1 -- 172.21.15.117:0/3085338467 wait complete. 2024-09-21T05:57:49.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.574+0000 7f4220d33640 1 Processor -- start 2024-09-21T05:57:49.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.574+0000 7f4220d33640 1 -- start start 2024-09-21T05:57:49.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.575+0000 7f4220d33640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f421c1021f0 0x7f421c19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:49.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.575+0000 7f4220d33640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f421c107b30 0x7f421c19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:49.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.575+0000 7f4220d33640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f421c108480 0x7f421c19fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:49.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.575+0000 7f421a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f421c1021f0 0x7f421c19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:49.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.575+0000 7f421a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f421c1021f0 0x7f421c19a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34580/0 (socket says 172.21.15.117:34580) 2024-09-21T05:57:49.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.575+0000 7f421a59c640 1 -- 172.21.15.117:0/223735724 learned_addr learned my addr 172.21.15.117:0/223735724 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:49.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.575+0000 7f4220d33640 1 -- 172.21.15.117:0/223735724 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f421c1118c0 con 0x7f421c1021f0 2024-09-21T05:57:49.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.575+0000 7f4220d33640 1 -- 172.21.15.117:0/223735724 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f421c111740 con 0x7f421c108480 2024-09-21T05:57:49.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.575+0000 7f4220d33640 1 -- 172.21.15.117:0/223735724 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f421c111a40 con 0x7f421c107b30 2024-09-21T05:57:49.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.575+0000 7f421ad9d640 1 --2- 172.21.15.117:0/223735724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f421c108480 0x7f421c19fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:49.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.575+0000 7f421a59c640 1 -- 172.21.15.117:0/223735724 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f421c107b30 msgr2=0x7f421c19ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:49.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.575+0000 7f421a59c640 1 --2- 172.21.15.117:0/223735724 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f421c107b30 0x7f421c19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:49.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.575+0000 7f421a59c640 1 -- 172.21.15.117:0/223735724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f421c108480 msgr2=0x7f421c19fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:49.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.575+0000 7f421a59c640 1 --2- 172.21.15.117:0/223735724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f421c108480 0x7f421c19fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:49.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.575+0000 7f421a59c640 1 -- 172.21.15.117:0/223735724 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f421c1a04f0 con 0x7f421c1021f0 2024-09-21T05:57:49.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.575+0000 7f421ad9d640 1 --2- 172.21.15.117:0/223735724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f421c108480 0x7f421c19fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:57:49.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.575+0000 7f421a59c640 1 --2- 172.21.15.117:0/223735724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f421c1021f0 0x7f421c19a580 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f420c00ddc0 tx=0x7f420c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:49.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.576+0000 7f420b7fe640 1 -- 172.21.15.117:0/223735724 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c01b070 con 0x7f421c1021f0 2024-09-21T05:57:49.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.576+0000 7f4220d33640 1 -- 172.21.15.117:0/223735724 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f421c1a07e0 con 0x7f421c1021f0 2024-09-21T05:57:49.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.576+0000 7f4220d33640 1 -- 172.21.15.117:0/223735724 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f421c198080 con 0x7f421c1021f0 2024-09-21T05:57:49.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.576+0000 7f420b7fe640 1 -- 172.21.15.117:0/223735724 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f420c00be40 con 0x7f421c1021f0 2024-09-21T05:57:49.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.576+0000 7f420b7fe640 1 -- 172.21.15.117:0/223735724 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c016a80 con 0x7f421c1021f0 2024-09-21T05:57:49.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.577+0000 7f42097fa640 1 -- 172.21.15.117:0/223735724 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4200005290 con 0x7f421c1021f0 2024-09-21T05:57:49.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.580+0000 7f420b7fe640 1 -- 172.21.15.117:0/223735724 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 23) ==== 154587+0+0 (secure 0 0 0) 0x7f420c01e070 con 0x7f421c1021f0 2024-09-21T05:57:49.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.581+0000 7f420b7fe640 1 --2- 172.21.15.117:0/223735724 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f42040b6670 0x7f42040b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:49.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.581+0000 7f4219d9b640 1 --2- 172.21.15.117:0/223735724 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f42040b6670 0x7f42040b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:49.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.581+0000 7f420b7fe640 1 -- 172.21.15.117:0/223735724 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f420c0e7610 con 0x7f421c1021f0 2024-09-21T05:57:49.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.582+0000 7f4219d9b640 1 --2- 172.21.15.117:0/223735724 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f42040b6670 0x7f42040b8b80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f421002a930 tx=0x7f4210031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:49.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.587+0000 7f420b7fe640 1 -- 172.21.15.117:0/223735724 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f420c0adca0 con 0x7f421c1021f0 2024-09-21T05:57:49.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:49 smithi117 ceph-mon[30947]: pgmap v342: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-21T05:57:49.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.899+0000 7f42097fa640 1 -- 172.21.15.117:0/223735724 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4200005580 con 0x7f421c1021f0 2024-09-21T05:57:49.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.900+0000 7f420b7fe640 1 -- 172.21.15.117:0/223735724 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+12001 (secure 0 0 0) 0x7f420c0b2080 con 0x7f421c1021f0 2024-09-21T05:57:49.904 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:57:49.904 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":27,"btime":"2024-09-21T05:57:06:651813+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3227057504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3227057504},{"type":"v1","addr":"172.21.15.117:6835","nonce":3227057504}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3303829183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3303829183},{"type":"v1","addr":"172.21.15.117:6837","nonce":3303829183}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T05:55:14.555897+0000","modified":"2024-09-21T05:57:06.599449+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24638,"mds_1":24637,"mds_2":24628,"mds_3":24616,"mds_4":24602},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6839/2314572411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2314572411},{"type":"v1","addr":"172.21.15.117:6839","nonce":2314572411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6841/3378096561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":3378096561},{"type":"v1","addr":"172.21.15.117:6841","nonce":3378096561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/3725629862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3725629862},{"type":"v1","addr":"172.21.15.138:6833","nonce":3725629862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/4276801396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4276801396},{"type":"v1","addr":"172.21.15.138:6835","nonce":4276801396}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6833/2224910879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2224910879},{"type":"v1","addr":"172.21.15.192:6833","nonce":2224910879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.138:6837/2540335374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2540335374},{"type":"v1","addr":"172.21.15.138:6837","nonce":2540335374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24616":{"gid":24616,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.192:6835/3530688860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3530688860},{"type":"v1","addr":"172.21.15.192:6835","nonce":3530688860}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.192:6837/2683860842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2683860842},{"type":"v1","addr":"172.21.15.192:6837","nonce":2683860842}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.192:6839/1727623231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1727623231},{"type":"v1","addr":"172.21.15.192:6839","nonce":1727623231}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/3230372576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":3230372576},{"type":"v1","addr":"172.21.15.138:6839","nonce":3230372576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24602,24628,24616,24637,24638]},"id":1}]} 2024-09-21T05:57:49.904 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 27 2024-09-21T05:57:49.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.902+0000 7f42097fa640 1 -- 172.21.15.117:0/223735724 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f42040b6670 msgr2=0x7f42040b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:49.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.902+0000 7f42097fa640 1 --2- 172.21.15.117:0/223735724 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f42040b6670 0x7f42040b8b80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f421002a930 tx=0x7f4210031400 comp rx=0 tx=0).stop 2024-09-21T05:57:49.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.902+0000 7f42097fa640 1 -- 172.21.15.117:0/223735724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f421c1021f0 msgr2=0x7f421c19a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:49.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.902+0000 7f42097fa640 1 --2- 172.21.15.117:0/223735724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f421c1021f0 0x7f421c19a580 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f420c00ddc0 tx=0x7f420c00a620 comp rx=0 tx=0).stop 2024-09-21T05:57:49.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.902+0000 7f42097fa640 1 -- 172.21.15.117:0/223735724 shutdown_connections 2024-09-21T05:57:49.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.902+0000 7f42097fa640 1 --2- 172.21.15.117:0/223735724 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f42040b6670 0x7f42040b8b80 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:49.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.902+0000 7f42097fa640 1 --2- 172.21.15.117:0/223735724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f421c108480 0x7f421c19fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:49.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.902+0000 7f42097fa640 1 --2- 172.21.15.117:0/223735724 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f421c107b30 0x7f421c19ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:49.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.902+0000 7f42097fa640 1 --2- 172.21.15.117:0/223735724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f421c1021f0 0x7f421c19a580 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:49.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.902+0000 7f42097fa640 1 -- 172.21.15.117:0/223735724 >> 172.21.15.117:0/223735724 conn(0x7f421c077c60 msgr2=0x7f421c105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:49.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.903+0000 7f42097fa640 1 -- 172.21.15.117:0/223735724 shutdown_connections 2024-09-21T05:57:49.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:49.903+0000 7f42097fa640 1 -- 172.21.15.117:0/223735724 wait complete. 2024-09-21T05:57:49.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:49 smithi192 ceph-mon[34700]: pgmap v342: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-21T05:57:50.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:49 smithi138 ceph-mon[34391]: pgmap v342: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-21T05:57:50.650 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T05:57:50.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:50 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/223735724' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:57:51.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:50 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/223735724' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:57:51.037 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:57:51.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:50 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/223735724' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:57:51.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.566+0000 7f820da11640 1 -- 172.21.15.117:0/2544581829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8208108480 msgr2=0x7f820810ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:51.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.566+0000 7f820da11640 1 --2- 172.21.15.117:0/2544581829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8208108480 0x7f820810ecb0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f8200009ca0 tx=0x7f820002a8c0 comp rx=0 tx=0).stop 2024-09-21T05:57:51.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.567+0000 7f820da11640 1 -- 172.21.15.117:0/2544581829 shutdown_connections 2024-09-21T05:57:51.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.567+0000 7f820da11640 1 --2- 172.21.15.117:0/2544581829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8208108480 0x7f820810ecb0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:51.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.567+0000 7f820da11640 1 --2- 172.21.15.117:0/2544581829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8208107b30 0x7f8208107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:51.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.567+0000 7f820da11640 1 --2- 172.21.15.117:0/2544581829 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82081021f0 0x7f8208102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:51.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.567+0000 7f820da11640 1 -- 172.21.15.117:0/2544581829 >> 172.21.15.117:0/2544581829 conn(0x7f8208077c60 msgr2=0x7f82080780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:51.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.567+0000 7f820da11640 1 -- 172.21.15.117:0/2544581829 shutdown_connections 2024-09-21T05:57:51.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.567+0000 7f820da11640 1 -- 172.21.15.117:0/2544581829 wait complete. 2024-09-21T05:57:51.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.568+0000 7f820da11640 1 Processor -- start 2024-09-21T05:57:51.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.568+0000 7f820da11640 1 -- start start 2024-09-21T05:57:51.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.568+0000 7f820da11640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82081021f0 0x7f820819b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:51.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.568+0000 7f820da11640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8208107b30 0x7f820819b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:51.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.568+0000 7f820da11640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8208108480 0x7f820819bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:51.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.568+0000 7f820da11640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8208112150 con 0x7f8208107b30 2024-09-21T05:57:51.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.568+0000 7f820da11640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8208111fd0 con 0x7f8208108480 2024-09-21T05:57:51.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.568+0000 7f820da11640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f82081122d0 con 0x7f82081021f0 2024-09-21T05:57:51.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.568+0000 7f81fe7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8208107b30 0x7f820819b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:51.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.568+0000 7f81fe7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8208107b30 0x7f820819b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34602/0 (socket says 172.21.15.117:34602) 2024-09-21T05:57:51.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.568+0000 7f81fe7fc640 1 -- 172.21.15.117:0/375159882 learned_addr learned my addr 172.21.15.117:0/375159882 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:51.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.569+0000 7f8206ffd640 1 --2- 172.21.15.117:0/375159882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82081021f0 0x7f820819b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:51.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.569+0000 7f82077fe640 1 --2- 172.21.15.117:0/375159882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8208108480 0x7f820819bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:51.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.569+0000 7f82077fe640 1 -- 172.21.15.117:0/375159882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82081021f0 msgr2=0x7f820819b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:51.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.569+0000 7f82077fe640 1 --2- 172.21.15.117:0/375159882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82081021f0 0x7f820819b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:51.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.569+0000 7f82077fe640 1 -- 172.21.15.117:0/375159882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8208107b30 msgr2=0x7f820819b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:51.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.569+0000 7f82077fe640 1 --2- 172.21.15.117:0/375159882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8208107b30 0x7f820819b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:51.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.569+0000 7f82077fe640 1 -- 172.21.15.117:0/375159882 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82081a65b0 con 0x7f8208108480 2024-09-21T05:57:51.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.569+0000 7f81fe7fc640 1 --2- 172.21.15.117:0/375159882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8208107b30 0x7f820819b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:57:51.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.569+0000 7f82077fe640 1 --2- 172.21.15.117:0/375159882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8208108480 0x7f820819bd70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f820002ade0 tx=0x7f8200031660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:51.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.570+0000 7f820ca0f640 1 -- 172.21.15.117:0/375159882 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8200003b70 con 0x7f8208108480 2024-09-21T05:57:51.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.570+0000 7f820ca0f640 1 -- 172.21.15.117:0/375159882 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8200031910 con 0x7f8208108480 2024-09-21T05:57:51.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.570+0000 7f820da11640 1 -- 172.21.15.117:0/375159882 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82081a6840 con 0x7f8208108480 2024-09-21T05:57:51.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.570+0000 7f820ca0f640 1 -- 172.21.15.117:0/375159882 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820002f950 con 0x7f8208108480 2024-09-21T05:57:51.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.570+0000 7f820da11640 1 -- 172.21.15.117:0/375159882 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82081a6cc0 con 0x7f8208108480 2024-09-21T05:57:51.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.572+0000 7f8213f85740 1 Processor -- start 2024-09-21T05:57:51.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.574+0000 7f820ca0f640 1 -- 172.21.15.117:0/375159882 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 23) ==== 154587+0+0 (secure 0 0 0) 0x7f8200004040 con 0x7f8208108480 2024-09-21T05:57:51.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.574+0000 7f8213f85740 1 -- start start 2024-09-21T05:57:51.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.574+0000 7f8213f85740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56145becf070 0x56145c2762c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:51.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.574+0000 7f8213f85740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145bea61b0 0x56145c276800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:51.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.574+0000 7f8213f85740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56145beae390 0x56145c276d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:51.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.574+0000 7f8213f85740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56145c1ecea0 con 0x56145beae390 2024-09-21T05:57:51.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.574+0000 7f8213f85740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56145c1dc2d0 con 0x56145becf070 2024-09-21T05:57:51.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.574+0000 7f8213f85740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56145c1e00e0 con 0x56145bea61b0 2024-09-21T05:57:51.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.575+0000 7f81fe7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56145beae390 0x56145c276d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:51.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.575+0000 7f8206ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145bea61b0 0x56145c276800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:51.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.575+0000 7f8206ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145bea61b0 0x56145c276800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:52048/0 (socket says 172.21.15.117:52048) 2024-09-21T05:57:51.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.575+0000 7f8206ffd640 1 -- 172.21.15.117:0/294163875 learned_addr learned my addr 172.21.15.117:0/294163875 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:51.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.575+0000 7f820ca0f640 1 --2- 172.21.15.117:0/375159882 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f81f40b65f0 0x7f81f40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:51.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.575+0000 7f81fe7fc640 1 --2- 172.21.15.117:0/294163875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56145becf070 0x56145c2762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:51.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.575+0000 7f81fe7fc640 1 -- 172.21.15.117:0/294163875 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145bea61b0 msgr2=0x56145c276800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:51.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.575+0000 7f820ca0f640 1 -- 172.21.15.117:0/375159882 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f82001085e0 con 0x7f8208108480 2024-09-21T05:57:51.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.575+0000 7f82077fe640 1 --2- 172.21.15.117:0/375159882 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f81f40b65f0 0x7f81f40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:51.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.575+0000 7f81fe7fc640 1 --2- 172.21.15.117:0/294163875 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145bea61b0 0x56145c276800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:51.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.575+0000 7f81fe7fc640 1 -- 172.21.15.117:0/294163875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56145becf070 msgr2=0x56145c2762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:51.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.575+0000 7f81fe7fc640 1 --2- 172.21.15.117:0/294163875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56145becf070 0x56145c2762c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:51.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.575+0000 7f81fe7fc640 1 -- 172.21.15.117:0/294163875 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56145bf3a650 con 0x56145beae390 2024-09-21T05:57:51.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.576+0000 7f81fe7fc640 1 --2- 172.21.15.117:0/294163875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56145beae390 0x56145c276d40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f81f0011640 tx=0x7f81f0009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9842005bc971f38 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:51.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.576+0000 7f82077fe640 1 --2- 172.21.15.117:0/375159882 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f81f40b65f0 0x7f81f40b8b00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f820002f550 tx=0x7f82000047b0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:51.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.576+0000 7f81fdffb640 1 -- 172.21.15.117:0/294163875 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f00042f0 con 0x56145beae390 2024-09-21T05:57:51.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.576+0000 7f81fdffb640 1 -- 172.21.15.117:0/294163875 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81f0018070 con 0x56145beae390 2024-09-21T05:57:51.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.576+0000 7f81fdffb640 1 -- 172.21.15.117:0/294163875 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f00042f0 con 0x56145beae390 2024-09-21T05:57:51.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.577+0000 7f8213f85740 1 -- 172.21.15.117:0/294163875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56145beae390 msgr2=0x56145c276d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:51.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.577+0000 7f8213f85740 1 --2- 172.21.15.117:0/294163875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56145beae390 0x56145c276d40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f81f0011640 tx=0x7f81f0009c40 comp rx=0 tx=0).stop 2024-09-21T05:57:51.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.577+0000 7f8213f85740 1 -- 172.21.15.117:0/294163875 shutdown_connections 2024-09-21T05:57:51.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.577+0000 7f8213f85740 1 --2- 172.21.15.117:0/294163875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56145beae390 0x56145c276d40 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:51.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.577+0000 7f8213f85740 1 --2- 172.21.15.117:0/294163875 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145bea61b0 0x56145c276800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:51.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.577+0000 7f8213f85740 1 --2- 172.21.15.117:0/294163875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56145becf070 0x56145c2762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:51.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.577+0000 7f8213f85740 1 -- 172.21.15.117:0/294163875 >> 172.21.15.117:0/294163875 conn(0x56145bec3460 msgr2=0x56145c2741b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:51.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.577+0000 7f8213f85740 1 -- 172.21.15.117:0/294163875 shutdown_connections 2024-09-21T05:57:51.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.577+0000 7f8213f85740 1 -- 172.21.15.117:0/294163875 wait complete. 2024-09-21T05:57:51.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.577+0000 7f8213f85740 1 -- start start 2024-09-21T05:57:51.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.578+0000 7f8213f85740 1 Processor -- start 2024-09-21T05:57:51.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.578+0000 7f8213f85740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56145bea61b0 0x56145c281b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:51.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.578+0000 7f8213f85740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56145beae390 0x56145c282080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:51.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.578+0000 7f8213f85740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145becf070 0x56145c2825c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:51.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.578+0000 7f8213f85740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56145c03d700 con 0x56145beae390 2024-09-21T05:57:51.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.578+0000 7f8213f85740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56145c177160 con 0x56145bea61b0 2024-09-21T05:57:51.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.578+0000 7f8213f85740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56145c27c5f0 con 0x56145becf070 2024-09-21T05:57:51.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.578+0000 7f8213f85740 10 client.0 _ll_register_callbacks cb 0x56145c1909e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:57:51.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.579+0000 7f8206ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56145beae390 0x56145c282080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:51.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.579+0000 7f81fe7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56145bea61b0 0x56145c281b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:51.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.579+0000 7f81fe7fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145becf070 0x56145c2825c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:51.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.579+0000 7f8206ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56145beae390 0x56145c282080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34614/0 (socket says 172.21.15.117:34614) 2024-09-21T05:57:51.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.579+0000 7f8206ffd640 1 -- 172.21.15.117:0/3848095629 learned_addr learned my addr 172.21.15.117:0/3848095629 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:51.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.579+0000 7f8206ffd640 1 -- 172.21.15.117:0/3848095629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145becf070 msgr2=0x56145c2825c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:51.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.579+0000 7f8206ffd640 1 --2- 172.21.15.117:0/3848095629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145becf070 0x56145c2825c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:51.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.579+0000 7f8206ffd640 1 -- 172.21.15.117:0/3848095629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56145bea61b0 msgr2=0x56145c281b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:51.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.579+0000 7f8206ffd640 1 --2- 172.21.15.117:0/3848095629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56145bea61b0 0x56145c281b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:51.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.579+0000 7f8206ffd640 1 -- 172.21.15.117:0/3848095629 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56145c1a4c90 con 0x56145beae390 2024-09-21T05:57:51.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.579+0000 7f8206ffd640 1 --2- 172.21.15.117:0/3848095629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56145beae390 0x56145c282080 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f81f8012a70 tx=0x7f81f8012f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190289be0738663a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:51.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.579+0000 7f81e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:57:51.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.580+0000 7f81e6ffd640 1 -- 172.21.15.117:0/3848095629 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f80120d0 con 0x56145beae390 2024-09-21T05:57:51.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.580+0000 7f81e6ffd640 1 -- 172.21.15.117:0/3848095629 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f81f8004020 con 0x56145beae390 2024-09-21T05:57:51.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.580+0000 7f8213f85740 1 -- 172.21.15.117:0/3848095629 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81f0018070 con 0x56145beae390 2024-09-21T05:57:51.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.580+0000 7f81e6ffd640 1 -- 172.21.15.117:0/3848095629 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f81f8012270 con 0x56145beae390 2024-09-21T05:57:51.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.580+0000 7f8213f85740 10 client.15651 fetch_fsmap learned FSMap version 27 2024-09-21T05:57:51.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.580+0000 7f8213f85740 1 -- 172.21.15.117:0/3848095629 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56145c290660 con 0x56145beae390 2024-09-21T05:57:51.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.580+0000 7f81e6ffd640 1 -- 172.21.15.117:0/3848095629 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81f0018070 con 0x56145beae390 2024-09-21T05:57:51.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.581+0000 7f81e6ffd640 1 -- 172.21.15.117:0/3848095629 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f81f80038c0 con 0x56145beae390 2024-09-21T05:57:51.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.581+0000 7f8213f85740 10 client.15651 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:57:51.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.581+0000 7f8213f85740 10 client.15651 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T05:57:51.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.581+0000 7f8213f85740 20 client.15651 populate_metadata read hostname 'smithi117' 2024-09-21T05:57:51.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.581+0000 7f8213f85740 1 --2- 172.21.15.117:0/3848095629 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x56145be8d9f0 0x56145c293b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:51.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.581+0000 7f8213f85740 4 client.15651 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:57:51.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.581+0000 7f8213f85740 1 -- 172.21.15.117:0/3848095629 --> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56145c1eedf0 con 0x56145be8d9f0 2024-09-21T05:57:51.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.581+0000 7f81fe7fc640 1 --2- 172.21.15.117:0/3848095629 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x56145be8d9f0 0x56145c293b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:51.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.582+0000 7f81e6ffd640 10 client.15651 ms_handle_connect on v2:172.21.15.138:6836/2540335374 2024-09-21T05:57:51.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.582+0000 7f81fe7fc640 1 --2- 172.21.15.117:0/3848095629 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x56145be8d9f0 0x56145c293b10 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=79d92de8db474595 server_cookie=9d9e22307b3720bd in_seq=0 out_seq=0 2024-09-21T05:57:51.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.582+0000 7f81e6ffd640 1 -- 172.21.15.117:0/3848095629 <== mds.4 v2:172.21.15.138:6836/2540335374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56145c1eedf0 con 0x56145be8d9f0 2024-09-21T05:57:51.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.582+0000 7f81e6ffd640 10 client.15651 handle_command_reply: tid=0 2024-09-21T05:57:51.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.583+0000 7f8213f85740 1 client.15651 shutdown 2024-09-21T05:57:51.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.583+0000 7f8213f85740 1 -- 172.21.15.117:0/3848095629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56145beae390 msgr2=0x56145c282080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:51.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.583+0000 7f8213f85740 1 --2- 172.21.15.117:0/3848095629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56145beae390 0x56145c282080 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f81f8012a70 tx=0x7f81f8012f30 comp rx=0 tx=0).stop 2024-09-21T05:57:51.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.583+0000 7f8213f85740 1 -- 172.21.15.117:0/3848095629 shutdown_connections 2024-09-21T05:57:51.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.583+0000 7f8213f85740 1 --2- 172.21.15.117:0/3848095629 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x56145be8d9f0 0x56145c293b10 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:51.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.583+0000 7f81e6ffd640 0 client.15651 ms_handle_reset on v2:172.21.15.138:6836/2540335374 2024-09-21T05:57:51.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.583+0000 7f8213f85740 1 --2- 172.21.15.117:0/3848095629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56145beae390 0x56145c282080 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:51.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.583+0000 7f8213f85740 1 --2- 172.21.15.117:0/3848095629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145becf070 0x56145c2825c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:51.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.583+0000 7f8213f85740 1 --2- 172.21.15.117:0/3848095629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56145bea61b0 0x56145c281b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:51.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.583+0000 7f8213f85740 1 -- 172.21.15.117:0/3848095629 >> 172.21.15.117:0/3848095629 conn(0x56145bec3460 msgr2=0x56145c272750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:51.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.583+0000 7f8213f85740 1 -- 172.21.15.117:0/3848095629 shutdown_connections 2024-09-21T05:57:51.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.583+0000 7f8213f85740 1 -- 172.21.15.117:0/3848095629 wait complete. 2024-09-21T05:57:51.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.583+0000 7f8213f85740 20 client.15651 trim_cache size 0 max 16384 2024-09-21T05:57:51.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.609+0000 7f8213f85740 1 Processor -- start 2024-09-21T05:57:51.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.609+0000 7f8213f85740 1 -- start start 2024-09-21T05:57:51.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.609+0000 7f8213f85740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145be8d9f0 0x56145c285b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:51.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.609+0000 7f8213f85740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56145bea61b0 0x56145c2868b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:51.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.609+0000 7f8213f85740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56145beae390 0x56145c273760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:51.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.609+0000 7f8213f85740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f82081b5ba0 con 0x56145beae390 2024-09-21T05:57:51.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.609+0000 7f8213f85740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82081b76e0 con 0x56145bea61b0 2024-09-21T05:57:51.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.609+0000 7f8213f85740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f82081b6270 con 0x56145be8d9f0 2024-09-21T05:57:51.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.609+0000 7f81fe7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56145beae390 0x56145c273760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:51.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.609+0000 7f81fe7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56145beae390 0x56145c273760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34616/0 (socket says 172.21.15.117:34616) 2024-09-21T05:57:51.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.609+0000 7f81fe7fc640 1 -- 172.21.15.117:0/1069788089 learned_addr learned my addr 172.21.15.117:0/1069788089 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:51.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.609+0000 7f81fe7fc640 1 --2- 172.21.15.117:0/1069788089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145be8d9f0 0x56145c285b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:51.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.609+0000 7f8206ffd640 1 --2- 172.21.15.117:0/1069788089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56145bea61b0 0x56145c2868b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:51.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.610+0000 7f81fe7fc640 1 -- 172.21.15.117:0/1069788089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145be8d9f0 msgr2=0x56145c285b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:51.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.610+0000 7f81fe7fc640 1 --2- 172.21.15.117:0/1069788089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145be8d9f0 0x56145c285b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:51.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.610+0000 7f81fe7fc640 1 -- 172.21.15.117:0/1069788089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56145bea61b0 msgr2=0x56145c2868b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:51.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.610+0000 7f81fe7fc640 1 --2- 172.21.15.117:0/1069788089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56145bea61b0 0x56145c2868b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:51.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.610+0000 7f81fe7fc640 1 -- 172.21.15.117:0/1069788089 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56145c274b30 con 0x56145beae390 2024-09-21T05:57:51.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.610+0000 7f81fe7fc640 1 --2- 172.21.15.117:0/1069788089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56145beae390 0x56145c273760 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f81f00049f0 tx=0x7f81f0004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ecd9f415ff057bf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:51.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.610+0000 7f81fd7fa640 1 -- 172.21.15.117:0/1069788089 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f0025040 con 0x56145beae390 2024-09-21T05:57:51.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.610+0000 7f81fd7fa640 1 -- 172.21.15.117:0/1069788089 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81f0018070 con 0x56145beae390 2024-09-21T05:57:51.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.610+0000 7f81fd7fa640 1 -- 172.21.15.117:0/1069788089 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f00042f0 con 0x56145beae390 2024-09-21T05:57:51.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.611+0000 7f8213f85740 1 -- 172.21.15.117:0/1069788089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56145beae390 msgr2=0x56145c273760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:51.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.611+0000 7f8213f85740 1 --2- 172.21.15.117:0/1069788089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56145beae390 0x56145c273760 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f81f00049f0 tx=0x7f81f0004a20 comp rx=0 tx=0).stop 2024-09-21T05:57:51.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.611+0000 7f8213f85740 1 -- 172.21.15.117:0/1069788089 shutdown_connections 2024-09-21T05:57:51.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.611+0000 7f8213f85740 1 --2- 172.21.15.117:0/1069788089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56145beae390 0x56145c273760 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:51.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.611+0000 7f8213f85740 1 --2- 172.21.15.117:0/1069788089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56145bea61b0 0x56145c2868b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:51.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.611+0000 7f8213f85740 1 --2- 172.21.15.117:0/1069788089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145be8d9f0 0x56145c285b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:51.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.611+0000 7f8213f85740 1 -- 172.21.15.117:0/1069788089 >> 172.21.15.117:0/1069788089 conn(0x56145bec3460 msgr2=0x56145c2861a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:51.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.612+0000 7f8213f85740 1 -- 172.21.15.117:0/1069788089 shutdown_connections 2024-09-21T05:57:51.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.612+0000 7f8213f85740 1 -- 172.21.15.117:0/1069788089 wait complete. 2024-09-21T05:57:51.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.612+0000 7f8213f85740 1 -- start start 2024-09-21T05:57:51.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.612+0000 7f8213f85740 1 Processor -- start 2024-09-21T05:57:51.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.613+0000 7f8213f85740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145be8d9f0 0x56145c28dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:51.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.613+0000 7f8213f85740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56145bea61b0 0x56145c28e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:51.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.613+0000 7f8213f85740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56145beae390 0x56145c28e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:51.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.613+0000 7f8213f85740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f82081b7010 con 0x56145beae390 2024-09-21T05:57:51.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.613+0000 7f8213f85740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82081b6940 con 0x56145bea61b0 2024-09-21T05:57:51.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.613+0000 7f8213f85740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f82081b3ba0 con 0x56145be8d9f0 2024-09-21T05:57:51.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.613+0000 7f8213f85740 10 client.0 _ll_register_callbacks cb 0x7f81f8012680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:57:51.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.613+0000 7f8206ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56145bea61b0 0x56145c28e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:51.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.613+0000 7f8206ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56145bea61b0 0x56145c28e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:60442/0 (socket says 172.21.15.117:60442) 2024-09-21T05:57:51.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.613+0000 7f8206ffd640 1 -- 172.21.15.117:0/719780789 learned_addr learned my addr 172.21.15.117:0/719780789 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:51.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.613+0000 7f81fe7fc640 1 --2- 172.21.15.117:0/719780789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145be8d9f0 0x56145c28dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:51.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.614+0000 7f8206ffd640 1 -- 172.21.15.117:0/719780789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145be8d9f0 msgr2=0x56145c28dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:51.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.614+0000 7f8206ffd640 1 --2- 172.21.15.117:0/719780789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145be8d9f0 0x56145c28dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:51.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.614+0000 7f8206ffd640 1 -- 172.21.15.117:0/719780789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56145beae390 msgr2=0x56145c28e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:51.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.614+0000 7f8206ffd640 1 --2- 172.21.15.117:0/719780789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56145beae390 0x56145c28e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:51.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.614+0000 7f8206ffd640 1 -- 172.21.15.117:0/719780789 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56145c288ce0 con 0x56145bea61b0 2024-09-21T05:57:51.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.614+0000 7f81fe7fc640 1 --2- 172.21.15.117:0/719780789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145be8d9f0 0x56145c28dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:57:51.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.614+0000 7f8206ffd640 1 --2- 172.21.15.117:0/719780789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56145bea61b0 0x56145c28e240 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f81f8003cc0 tx=0x7f81f801c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=938bad33200451a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:51.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.614+0000 7f81e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-21T05:57:51.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.614+0000 7f81e6ffd640 1 -- 172.21.15.117:0/719780789 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f8003a40 con 0x56145bea61b0 2024-09-21T05:57:51.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.614+0000 7f8213f85740 1 -- 172.21.15.117:0/719780789 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81f0018070 con 0x56145bea61b0 2024-09-21T05:57:51.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.614+0000 7f81e6ffd640 1 -- 172.21.15.117:0/719780789 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f81f800f980 con 0x56145bea61b0 2024-09-21T05:57:51.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.614+0000 7f81e6ffd640 1 -- 172.21.15.117:0/719780789 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f81f8034050 con 0x56145bea61b0 2024-09-21T05:57:51.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.614+0000 7f8213f85740 10 client.25199 fetch_fsmap learned FSMap version 27 2024-09-21T05:57:51.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.614+0000 7f8213f85740 1 -- 172.21.15.117:0/719780789 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56145c2fe750 con 0x56145bea61b0 2024-09-21T05:57:51.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.615+0000 7f81e6ffd640 1 -- 172.21.15.117:0/719780789 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81f0018070 con 0x56145bea61b0 2024-09-21T05:57:51.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.615+0000 7f81e6ffd640 1 -- 172.21.15.117:0/719780789 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f81f801c7a0 con 0x56145bea61b0 2024-09-21T05:57:51.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.615+0000 7f8213f85740 10 client.25199 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:57:51.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.615+0000 7f8213f85740 10 client.25199 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T05:57:51.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.615+0000 7f8213f85740 20 client.25199 populate_metadata read hostname 'smithi117' 2024-09-21T05:57:51.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.615+0000 7f8213f85740 1 --2- 172.21.15.117:0/719780789 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x56145becf070 0x56145c300ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:51.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.615+0000 7f8213f85740 4 client.25199 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T05:57:51.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.615+0000 7f8213f85740 1 -- 172.21.15.117:0/719780789 --> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56145c1ea8a0 con 0x56145becf070 2024-09-21T05:57:51.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.615+0000 7f81fe7fc640 1 --2- 172.21.15.117:0/719780789 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x56145becf070 0x56145c300ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:51.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.616+0000 7f81fe7fc640 1 --2- 172.21.15.117:0/719780789 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x56145becf070 0x56145c300ed0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dcb9dd45e9f83b22 server_cookie=844ab4a3f80a76a6 in_seq=0 out_seq=0 2024-09-21T05:57:51.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.616+0000 7f81e6ffd640 10 client.25199 ms_handle_connect on v2:172.21.15.138:6836/2540335374 2024-09-21T05:57:51.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.616+0000 7f81e6ffd640 1 -- 172.21.15.117:0/719780789 <== mds.4 v2:172.21.15.138:6836/2540335374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56145c1ea8a0 con 0x56145becf070 2024-09-21T05:57:51.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.616+0000 7f81e6ffd640 10 client.25199 handle_command_reply: tid=0 2024-09-21T05:57:51.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.616+0000 7f8213f85740 1 client.25199 shutdown 2024-09-21T05:57:51.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.616+0000 7f8213f85740 1 -- 172.21.15.117:0/719780789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56145bea61b0 msgr2=0x56145c28e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:51.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.616+0000 7f8213f85740 1 --2- 172.21.15.117:0/719780789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56145bea61b0 0x56145c28e240 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f81f8003cc0 tx=0x7f81f801c6a0 comp rx=0 tx=0).stop 2024-09-21T05:57:51.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.616+0000 7f8213f85740 1 -- 172.21.15.117:0/719780789 shutdown_connections 2024-09-21T05:57:51.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.616+0000 7f8213f85740 1 --2- 172.21.15.117:0/719780789 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x56145becf070 0x56145c300ed0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:51.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.616+0000 7f8213f85740 1 --2- 172.21.15.117:0/719780789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56145beae390 0x56145c28e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:51.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.616+0000 7f8213f85740 1 --2- 172.21.15.117:0/719780789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56145bea61b0 0x56145c28e240 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:51.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.616+0000 7f8213f85740 1 --2- 172.21.15.117:0/719780789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56145be8d9f0 0x56145c28dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:51.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.616+0000 7f8213f85740 1 -- 172.21.15.117:0/719780789 >> 172.21.15.117:0/719780789 conn(0x56145bec3460 msgr2=0x56145c282240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:51.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.616+0000 7f81e6ffd640 0 client.25199 ms_handle_reset on v2:172.21.15.138:6836/2540335374 2024-09-21T05:57:51.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.616+0000 7f8213f85740 1 -- 172.21.15.117:0/719780789 shutdown_connections 2024-09-21T05:57:51.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.616+0000 7f8213f85740 1 -- 172.21.15.117:0/719780789 wait complete. 2024-09-21T05:57:51.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.617+0000 7f8213f85740 20 client.25199 trim_cache size 0 max 16384 2024-09-21T05:57:51.621 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T05:57:51.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.617+0000 7f820da11640 1 -- 172.21.15.117:0/375159882 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f81f40b65f0 msgr2=0x7f81f40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:51.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.617+0000 7f820da11640 1 --2- 172.21.15.117:0/375159882 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f81f40b65f0 0x7f81f40b8b00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f820002f550 tx=0x7f82000047b0 comp rx=0 tx=0).stop 2024-09-21T05:57:51.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.617+0000 7f820da11640 1 -- 172.21.15.117:0/375159882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8208108480 msgr2=0x7f820819bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:51.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.617+0000 7f820da11640 1 --2- 172.21.15.117:0/375159882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8208108480 0x7f820819bd70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f820002ade0 tx=0x7f8200031660 comp rx=0 tx=0).stop 2024-09-21T05:57:51.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.617+0000 7f820da11640 1 -- 172.21.15.117:0/375159882 shutdown_connections 2024-09-21T05:57:51.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.617+0000 7f820da11640 1 --2- 172.21.15.117:0/375159882 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f81f40b65f0 0x7f81f40b8b00 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:51.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.617+0000 7f820da11640 1 --2- 172.21.15.117:0/375159882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8208107b30 0x7f820819b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:51.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.617+0000 7f820da11640 1 --2- 172.21.15.117:0/375159882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8208108480 0x7f820819bd70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:51.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.617+0000 7f820da11640 1 --2- 172.21.15.117:0/375159882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82081021f0 0x7f820819b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:51.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.617+0000 7f820da11640 1 -- 172.21.15.117:0/375159882 >> 172.21.15.117:0/375159882 conn(0x7f8208077c60 msgr2=0x7f82081038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:51.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.618+0000 7f820da11640 1 -- 172.21.15.117:0/375159882 shutdown_connections 2024-09-21T05:57:51.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:51.618+0000 7f820da11640 1 -- 172.21.15.117:0/375159882 wait complete. 2024-09-21T05:57:51.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:51 smithi117 ceph-mon[30947]: pgmap v343: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-21T05:57:51.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:51 smithi192 ceph-mon[34700]: pgmap v343: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-21T05:57:52.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:51 smithi138 ceph-mon[34391]: pgmap v343: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-21T05:57:52.354 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T05:57:52.727 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:57:53.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.339+0000 7fc7ef688640 1 -- 172.21.15.117:0/2683914851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7e81083a0 msgr2=0x7fc7e81087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:53.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.339+0000 7fc7ef688640 1 --2- 172.21.15.117:0/2683914851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7e81083a0 0x7fc7e81087b0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8009900 tx=0x7fc7d802a3f0 comp rx=0 tx=0).stop 2024-09-21T05:57:53.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.339+0000 7fc7ef688640 1 -- 172.21.15.117:0/2683914851 shutdown_connections 2024-09-21T05:57:53.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.339+0000 7fc7ef688640 1 --2- 172.21.15.117:0/2683914851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7e81083a0 0x7fc7e81087b0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:53.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.339+0000 7fc7ef688640 1 --2- 172.21.15.117:0/2683914851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e8108cf0 0x7fc7e810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:53.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.339+0000 7fc7ef688640 1 --2- 172.21.15.117:0/2683914851 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7e8102a60 0x7fc7e8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:53.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.339+0000 7fc7ef688640 1 -- 172.21.15.117:0/2683914851 >> 172.21.15.117:0/2683914851 conn(0x7fc7e8079730 msgr2=0x7fc7e8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:53.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.339+0000 7fc7ef688640 1 -- 172.21.15.117:0/2683914851 shutdown_connections 2024-09-21T05:57:53.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.340+0000 7fc7ef688640 1 -- 172.21.15.117:0/2683914851 wait complete. 2024-09-21T05:57:53.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.340+0000 7fc7ef688640 1 Processor -- start 2024-09-21T05:57:53.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.341+0000 7fc7ef688640 1 -- start start 2024-09-21T05:57:53.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.341+0000 7fc7ef688640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7e8102a60 0x7fc7e819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:53.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.341+0000 7fc7ed424640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7e8102a60 0x7fc7e819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:53.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.341+0000 7fc7ed424640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7e8102a60 0x7fc7e819a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34654/0 (socket says 172.21.15.117:34654) 2024-09-21T05:57:53.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.341+0000 7fc7ed424640 1 -- 172.21.15.117:0/478199419 learned_addr learned my addr 172.21.15.117:0/478199419 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:53.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.341+0000 7fc7ef688640 1 --2- 172.21.15.117:0/478199419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e8108cf0 0x7fc7e819abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:53.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.341+0000 7fc7ef688640 1 --2- 172.21.15.117:0/478199419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7e819fdd0 0x7fc7e81a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:53.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.341+0000 7fc7ef688640 1 -- 172.21.15.117:0/478199419 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e8112960 con 0x7fc7e8102a60 2024-09-21T05:57:53.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.341+0000 7fc7ef688640 1 -- 172.21.15.117:0/478199419 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e81127e0 con 0x7fc7e8108cf0 2024-09-21T05:57:53.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.341+0000 7fc7ef688640 1 -- 172.21.15.117:0/478199419 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e8112ae0 con 0x7fc7e819fdd0 2024-09-21T05:57:53.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.342+0000 7fc7ecc23640 1 --2- 172.21.15.117:0/478199419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e8108cf0 0x7fc7e819abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:53.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.342+0000 7fc7ed424640 1 -- 172.21.15.117:0/478199419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7e819fdd0 msgr2=0x7fc7e81a01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:53.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.342+0000 7fc7ed424640 1 --2- 172.21.15.117:0/478199419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7e819fdd0 0x7fc7e81a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:53.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.342+0000 7fc7ed424640 1 -- 172.21.15.117:0/478199419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e8108cf0 msgr2=0x7fc7e819abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:53.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.342+0000 7fc7ed424640 1 --2- 172.21.15.117:0/478199419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e8108cf0 0x7fc7e819abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:53.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.342+0000 7fc7ed424640 1 -- 172.21.15.117:0/478199419 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e81a09b0 con 0x7fc7e8102a60 2024-09-21T05:57:53.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.342+0000 7fc7ed424640 1 --2- 172.21.15.117:0/478199419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7e8102a60 0x7fc7e819a650 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc7e400d770 tx=0x7fc7e400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:53.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.342+0000 7fc7de7fc640 1 -- 172.21.15.117:0/478199419 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e4018070 con 0x7fc7e8102a60 2024-09-21T05:57:53.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.343+0000 7fc7de7fc640 1 -- 172.21.15.117:0/478199419 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7e40054d0 con 0x7fc7e8102a60 2024-09-21T05:57:53.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.343+0000 7fc7ef688640 1 -- 172.21.15.117:0/478199419 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e8113100 con 0x7fc7e8102a60 2024-09-21T05:57:53.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.343+0000 7fc7de7fc640 1 -- 172.21.15.117:0/478199419 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e4004c30 con 0x7fc7e8102a60 2024-09-21T05:57:53.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.343+0000 7fc7ef688640 1 -- 172.21.15.117:0/478199419 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e8113610 con 0x7fc7e8102a60 2024-09-21T05:57:53.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.346+0000 7fc7de7fc640 1 -- 172.21.15.117:0/478199419 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 23) ==== 154587+0+0 (secure 0 0 0) 0x7fc7e4004f10 con 0x7fc7e8102a60 2024-09-21T05:57:53.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.347+0000 7fc7f5bfc740 1 Processor -- start 2024-09-21T05:57:53.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.347+0000 7fc7de7fc640 1 --2- 172.21.15.117:0/478199419 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc7d40b6670 0x7fc7d40b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:53.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.347+0000 7fc7f5bfc740 1 -- start start 2024-09-21T05:57:53.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.347+0000 7fc7de7fc640 1 -- 172.21.15.117:0/478199419 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fc7e40eb290 con 0x7fc7e8102a60 2024-09-21T05:57:53.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.347+0000 7fc7edc25640 1 --2- 172.21.15.117:0/478199419 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc7d40b6670 0x7fc7d40b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:53.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.348+0000 7fc7f5bfc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d40f385070 0x55d40f72c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:53.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.348+0000 7fc7f5bfc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d40f35c1b0 0x55d40f72c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:53.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.348+0000 7fc7f5bfc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d40f364390 0x55d40f72cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:53.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.348+0000 7fc7f5bfc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d40f6a2ea0 con 0x55d40f364390 2024-09-21T05:57:53.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.348+0000 7fc7f5bfc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d40f6922d0 con 0x55d40f385070 2024-09-21T05:57:53.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.348+0000 7fc7f5bfc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55d40f6960e0 con 0x55d40f35c1b0 2024-09-21T05:57:53.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.348+0000 7fc7ed424640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d40f385070 0x55d40f72c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:53.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.348+0000 7fc7ed424640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d40f385070 0x55d40f72c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:60472/0 (socket says 172.21.15.117:60472) 2024-09-21T05:57:53.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.348+0000 7fc7ed424640 1 -- 172.21.15.117:0/2951871252 learned_addr learned my addr 172.21.15.117:0/2951871252 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:53.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.348+0000 7fc7ecc23640 1 --2- 172.21.15.117:0/2951871252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d40f35c1b0 0x55d40f72c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:53.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.348+0000 7fc7edc25640 1 --2- 172.21.15.117:0/2951871252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d40f364390 0x55d40f72cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:53.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.348+0000 7fc7ed424640 1 -- 172.21.15.117:0/2951871252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d40f35c1b0 msgr2=0x55d40f72c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:53.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.348+0000 7fc7ed424640 1 --2- 172.21.15.117:0/2951871252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d40f35c1b0 0x55d40f72c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:53.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.348+0000 7fc7edc25640 1 --2- 172.21.15.117:0/478199419 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc7d40b6670 0x7fc7d40b8b80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc7e00092f0 tx=0x7fc7e0009220 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:53.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.348+0000 7fc7ed424640 1 -- 172.21.15.117:0/2951871252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d40f364390 msgr2=0x55d40f72cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:53.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.348+0000 7fc7ed424640 1 --2- 172.21.15.117:0/2951871252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d40f364390 0x55d40f72cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:53.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.348+0000 7fc7ed424640 1 -- 172.21.15.117:0/2951871252 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d40f3f0650 con 0x55d40f385070 2024-09-21T05:57:53.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.348+0000 7fc7edc25640 1 --2- 172.21.15.117:0/2951871252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d40f364390 0x55d40f72cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:57:53.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.349+0000 7fc7ed424640 1 --2- 172.21.15.117:0/2951871252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d40f385070 0x55d40f72c2c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc7e40b0ec0 tx=0x7fc7e40ad890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aff95dd5f91c638e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:53.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.349+0000 7fc7d37fe640 1 -- 172.21.15.117:0/2951871252 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e40b18c0 con 0x55d40f385070 2024-09-21T05:57:53.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.349+0000 7fc7d37fe640 1 -- 172.21.15.117:0/2951871252 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7e40b5ca0 con 0x55d40f385070 2024-09-21T05:57:53.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.349+0000 7fc7d37fe640 1 -- 172.21.15.117:0/2951871252 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e40ba750 con 0x55d40f385070 2024-09-21T05:57:53.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.350+0000 7fc7f5bfc740 1 -- 172.21.15.117:0/2951871252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d40f385070 msgr2=0x55d40f72c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:53.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.350+0000 7fc7f5bfc740 1 --2- 172.21.15.117:0/2951871252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d40f385070 0x55d40f72c2c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc7e40b0ec0 tx=0x7fc7e40ad890 comp rx=0 tx=0).stop 2024-09-21T05:57:53.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.350+0000 7fc7f5bfc740 1 -- 172.21.15.117:0/2951871252 shutdown_connections 2024-09-21T05:57:53.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.350+0000 7fc7f5bfc740 1 --2- 172.21.15.117:0/2951871252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d40f364390 0x55d40f72cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:53.354 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.350+0000 7fc7f5bfc740 1 --2- 172.21.15.117:0/2951871252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d40f35c1b0 0x55d40f72c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:53.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.350+0000 7fc7f5bfc740 1 --2- 172.21.15.117:0/2951871252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d40f385070 0x55d40f72c2c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:53.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.350+0000 7fc7f5bfc740 1 -- 172.21.15.117:0/2951871252 >> 172.21.15.117:0/2951871252 conn(0x55d40f379460 msgr2=0x55d40f72a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:53.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.351+0000 7fc7f5bfc740 1 -- 172.21.15.117:0/2951871252 shutdown_connections 2024-09-21T05:57:53.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.351+0000 7fc7f5bfc740 1 -- 172.21.15.117:0/2951871252 wait complete. 2024-09-21T05:57:53.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.351+0000 7fc7f5bfc740 1 -- start start 2024-09-21T05:57:53.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.351+0000 7fc7f5bfc740 1 Processor -- start 2024-09-21T05:57:53.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.352+0000 7fc7f5bfc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d40f35c1b0 0x55d40f737980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:53.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.352+0000 7fc7ed424640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d40f35c1b0 0x55d40f737980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:53.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.352+0000 7fc7ed424640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d40f35c1b0 0x55d40f737980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34672/0 (socket says 172.21.15.117:34672) 2024-09-21T05:57:53.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.352+0000 7fc7ed424640 1 -- 172.21.15.117:0/2483519461 learned_addr learned my addr 172.21.15.117:0/2483519461 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:53.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.352+0000 7fc7f5bfc740 1 --2- 172.21.15.117:0/2483519461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d40f364390 0x55d40f737ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:53.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.352+0000 7fc7f5bfc740 1 --2- 172.21.15.117:0/2483519461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d40f385070 0x55d40f743de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:53.355 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.352+0000 7fc7f5bfc740 1 -- 172.21.15.117:0/2483519461 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d40f4f3700 con 0x55d40f35c1b0 2024-09-21T05:57:53.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.352+0000 7fc7f5bfc740 1 -- 172.21.15.117:0/2483519461 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d40f62d160 con 0x55d40f385070 2024-09-21T05:57:53.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.352+0000 7fc7f5bfc740 1 -- 172.21.15.117:0/2483519461 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55d40f73aaa0 con 0x55d40f364390 2024-09-21T05:57:53.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.353+0000 7fc7edc25640 1 --2- 172.21.15.117:0/2483519461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d40f385070 0x55d40f743de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:53.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.353+0000 7fc7ecc23640 1 --2- 172.21.15.117:0/2483519461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d40f364390 0x55d40f737ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:53.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.353+0000 7fc7ed424640 1 -- 172.21.15.117:0/2483519461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d40f364390 msgr2=0x55d40f737ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:53.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.353+0000 7fc7ed424640 1 --2- 172.21.15.117:0/2483519461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d40f364390 0x55d40f737ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:53.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.353+0000 7fc7ed424640 1 -- 172.21.15.117:0/2483519461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d40f385070 msgr2=0x55d40f743de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:53.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.353+0000 7fc7ed424640 1 --2- 172.21.15.117:0/2483519461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d40f385070 0x55d40f743de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:53.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.353+0000 7fc7ed424640 1 -- 172.21.15.117:0/2483519461 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d40f65ac90 con 0x55d40f35c1b0 2024-09-21T05:57:53.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.353+0000 7fc7f5bfc740 10 client.0 _ll_register_callbacks cb 0x55d40f6469e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:57:53.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.353+0000 7fc7ed424640 1 --2- 172.21.15.117:0/2483519461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d40f35c1b0 0x55d40f737980 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc7e40ae430 tx=0x7fc7e40b0e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8660ef4b727b23b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:53.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.353+0000 7fc7d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:57:53.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.353+0000 7fc7f5bfc740 1 -- 172.21.15.117:0/2483519461 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7e40b5ca0 con 0x55d40f35c1b0 2024-09-21T05:57:53.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.354+0000 7fc7d0ff9640 1 -- 172.21.15.117:0/2483519461 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e40ba950 con 0x55d40f35c1b0 2024-09-21T05:57:53.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.354+0000 7fc7d0ff9640 1 -- 172.21.15.117:0/2483519461 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fc7e40d4b40 con 0x55d40f35c1b0 2024-09-21T05:57:53.358 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.354+0000 7fc7d0ff9640 1 -- 172.21.15.117:0/2483519461 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc7e40d4d40 con 0x55d40f35c1b0 2024-09-21T05:57:53.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.354+0000 7fc7f5bfc740 10 client.15666 fetch_fsmap learned FSMap version 27 2024-09-21T05:57:53.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.354+0000 7fc7f5bfc740 1 -- 172.21.15.117:0/2483519461 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55d40f746640 con 0x55d40f35c1b0 2024-09-21T05:57:53.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.354+0000 7fc7d0ff9640 1 -- 172.21.15.117:0/2483519461 <== mon.0 v2:172.21.15.117:3300/0 4 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc7e40b0420 con 0x55d40f35c1b0 2024-09-21T05:57:53.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.354+0000 7fc7f5bfc740 10 client.15666 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:57:53.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.355+0000 7fc7f5bfc740 10 client.15666 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T05:57:53.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.355+0000 7fc7f5bfc740 20 client.15666 populate_metadata read hostname 'smithi117' 2024-09-21T05:57:53.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.355+0000 7fc7f5bfc740 1 --2- 172.21.15.117:0/2483519461 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55d40f3439f0 0x55d40f7499a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:53.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.355+0000 7fc7f5bfc740 4 client.15666 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:57:53.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.355+0000 7fc7f5bfc740 1 -- 172.21.15.117:0/2483519461 --> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc7e40b5ca0 con 0x55d40f3439f0 2024-09-21T05:57:53.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.355+0000 7fc7ecc23640 1 --2- 172.21.15.117:0/2483519461 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55d40f3439f0 0x55d40f7499a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:53.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.356+0000 7fc7d0ff9640 10 client.15666 ms_handle_connect on v2:172.21.15.192:6834/3530688860 2024-09-21T05:57:53.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.356+0000 7fc7ecc23640 1 --2- 172.21.15.117:0/2483519461 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55d40f3439f0 0x55d40f7499a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=afc68cb2e967063c server_cookie=bd50db32998d4520 in_seq=0 out_seq=0 2024-09-21T05:57:53.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.356+0000 7fc7d0ff9640 1 -- 172.21.15.117:0/2483519461 <== mds.3 v2:172.21.15.192:6834/3530688860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fc7e40b5ca0 con 0x55d40f3439f0 2024-09-21T05:57:53.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.356+0000 7fc7d0ff9640 10 client.15666 handle_command_reply: tid=0 2024-09-21T05:57:53.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.356+0000 7fc7f5bfc740 1 client.15666 shutdown 2024-09-21T05:57:53.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.357+0000 7fc7f5bfc740 1 -- 172.21.15.117:0/2483519461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d40f35c1b0 msgr2=0x55d40f737980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:53.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.357+0000 7fc7f5bfc740 1 --2- 172.21.15.117:0/2483519461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d40f35c1b0 0x55d40f737980 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc7e40ae430 tx=0x7fc7e40b0e00 comp rx=0 tx=0).stop 2024-09-21T05:57:53.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.357+0000 7fc7f5bfc740 1 -- 172.21.15.117:0/2483519461 shutdown_connections 2024-09-21T05:57:53.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.357+0000 7fc7f5bfc740 1 --2- 172.21.15.117:0/2483519461 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55d40f3439f0 0x55d40f7499a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:53.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.357+0000 7fc7f5bfc740 1 --2- 172.21.15.117:0/2483519461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d40f385070 0x55d40f743de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:53.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.357+0000 7fc7d0ff9640 0 client.15666 ms_handle_reset on v2:172.21.15.192:6834/3530688860 2024-09-21T05:57:53.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.357+0000 7fc7f5bfc740 1 --2- 172.21.15.117:0/2483519461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d40f364390 0x55d40f737ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:53.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.357+0000 7fc7f5bfc740 1 --2- 172.21.15.117:0/2483519461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d40f35c1b0 0x55d40f737980 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:53.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.357+0000 7fc7f5bfc740 1 -- 172.21.15.117:0/2483519461 >> 172.21.15.117:0/2483519461 conn(0x55d40f379460 msgr2=0x55d40f72f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:53.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.357+0000 7fc7f5bfc740 1 -- 172.21.15.117:0/2483519461 shutdown_connections 2024-09-21T05:57:53.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.357+0000 7fc7f5bfc740 1 -- 172.21.15.117:0/2483519461 wait complete. 2024-09-21T05:57:53.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.358+0000 7fc7f5bfc740 20 client.15666 trim_cache size 0 max 16384 2024-09-21T05:57:53.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.383+0000 7fc7f5bfc740 1 Processor -- start 2024-09-21T05:57:53.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.383+0000 7fc7f5bfc740 1 -- start start 2024-09-21T05:57:53.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.383+0000 7fc7f5bfc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d40f3439f0 0x55d40f739860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:53.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.383+0000 7fc7f5bfc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d40f35c1b0 0x55d40f72da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:53.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.383+0000 7fc7f5bfc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d40f364390 0x55d40f743d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:53.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.383+0000 7fc7f5bfc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e81adb60 con 0x55d40f35c1b0 2024-09-21T05:57:53.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.383+0000 7fc7f5bfc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e81af6a0 con 0x55d40f3439f0 2024-09-21T05:57:53.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.383+0000 7fc7f5bfc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e81ae230 con 0x55d40f364390 2024-09-21T05:57:53.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.383+0000 7fc7ecc23640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d40f35c1b0 0x55d40f72da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:53.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.383+0000 7fc7ecc23640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d40f35c1b0 0x55d40f72da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34680/0 (socket says 172.21.15.117:34680) 2024-09-21T05:57:53.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.384+0000 7fc7ecc23640 1 -- 172.21.15.117:0/2041155380 learned_addr learned my addr 172.21.15.117:0/2041155380 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:53.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.384+0000 7fc7ed424640 1 --2- 172.21.15.117:0/2041155380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d40f3439f0 0x55d40f739860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:53.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.384+0000 7fc7ed424640 1 -- 172.21.15.117:0/2041155380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d40f364390 msgr2=0x55d40f743d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:53.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.384+0000 7fc7ed424640 1 --2- 172.21.15.117:0/2041155380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d40f364390 0x55d40f743d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:53.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.384+0000 7fc7ed424640 1 -- 172.21.15.117:0/2041155380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d40f35c1b0 msgr2=0x55d40f72da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:53.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.384+0000 7fc7ed424640 1 --2- 172.21.15.117:0/2041155380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d40f35c1b0 0x55d40f72da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:53.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.384+0000 7fc7ed424640 1 -- 172.21.15.117:0/2041155380 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d40f724460 con 0x55d40f3439f0 2024-09-21T05:57:53.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.384+0000 7fc7ecc23640 1 --2- 172.21.15.117:0/2041155380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d40f35c1b0 0x55d40f72da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:57:53.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.384+0000 7fc7ed424640 1 --2- 172.21.15.117:0/2041155380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d40f3439f0 0x55d40f739860 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc7e40e8a80 tx=0x7fc7e4098070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4caa246d39fb21a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:53.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.385+0000 7fc7d2ffd640 1 -- 172.21.15.117:0/2041155380 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e40b18c0 con 0x55d40f3439f0 2024-09-21T05:57:53.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.385+0000 7fc7d2ffd640 1 -- 172.21.15.117:0/2041155380 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7e40b5ca0 con 0x55d40f3439f0 2024-09-21T05:57:53.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.385+0000 7fc7d2ffd640 1 -- 172.21.15.117:0/2041155380 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e40b0420 con 0x55d40f3439f0 2024-09-21T05:57:53.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.386+0000 7fc7f5bfc740 1 -- 172.21.15.117:0/2041155380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d40f3439f0 msgr2=0x55d40f739860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:53.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.386+0000 7fc7f5bfc740 1 --2- 172.21.15.117:0/2041155380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d40f3439f0 0x55d40f739860 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc7e40e8a80 tx=0x7fc7e4098070 comp rx=0 tx=0).stop 2024-09-21T05:57:53.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.386+0000 7fc7f5bfc740 1 -- 172.21.15.117:0/2041155380 shutdown_connections 2024-09-21T05:57:53.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.386+0000 7fc7f5bfc740 1 --2- 172.21.15.117:0/2041155380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d40f35c1b0 0x55d40f72da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:53.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.386+0000 7fc7f5bfc740 1 --2- 172.21.15.117:0/2041155380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d40f364390 0x55d40f743d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:53.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.386+0000 7fc7f5bfc740 1 --2- 172.21.15.117:0/2041155380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d40f3439f0 0x55d40f739860 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:53.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.386+0000 7fc7f5bfc740 1 -- 172.21.15.117:0/2041155380 >> 172.21.15.117:0/2041155380 conn(0x55d40f379460 msgr2=0x55d40f72d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:53.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.386+0000 7fc7f5bfc740 1 -- 172.21.15.117:0/2041155380 shutdown_connections 2024-09-21T05:57:53.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.386+0000 7fc7f5bfc740 1 -- 172.21.15.117:0/2041155380 wait complete. 2024-09-21T05:57:53.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.386+0000 7fc7f5bfc740 1 -- start start 2024-09-21T05:57:53.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.387+0000 7fc7f5bfc740 1 Processor -- start 2024-09-21T05:57:53.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.387+0000 7fc7f5bfc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d40f3439f0 0x55d40f74ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:53.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.387+0000 7fc7f5bfc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d40f35c1b0 0x55d40f74b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:53.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.387+0000 7fc7f5bfc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d40f364390 0x55d40f7b3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:53.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.388+0000 7fc7ed424640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d40f3439f0 0x55d40f74ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:53.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.388+0000 7fc7ed424640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d40f3439f0 0x55d40f74ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:52142/0 (socket says 172.21.15.117:52142) 2024-09-21T05:57:53.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.388+0000 7fc7ecc23640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d40f35c1b0 0x55d40f74b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:53.392 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.388+0000 7fc7edc25640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d40f364390 0x55d40f7b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:53.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.388+0000 7fc7ecc23640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d40f35c1b0 0x55d40f74b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:60504/0 (socket says 172.21.15.117:60504) 2024-09-21T05:57:53.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.388+0000 7fc7edc25640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d40f364390 0x55d40f7b3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34686/0 (socket says 172.21.15.117:34686) 2024-09-21T05:57:53.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.388+0000 7fc7ed424640 1 -- 172.21.15.117:0/2621960827 learned_addr learned my addr 172.21.15.117:0/2621960827 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:53.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.388+0000 7fc7f5bfc740 1 -- 172.21.15.117:0/2621960827 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e81aefd0 con 0x55d40f364390 2024-09-21T05:57:53.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.388+0000 7fc7f5bfc740 1 -- 172.21.15.117:0/2621960827 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e81ae900 con 0x55d40f35c1b0 2024-09-21T05:57:53.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.388+0000 7fc7f5bfc740 1 -- 172.21.15.117:0/2621960827 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e81abb60 con 0x55d40f3439f0 2024-09-21T05:57:53.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.388+0000 7fc7ecc23640 1 -- 172.21.15.117:0/2621960827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d40f3439f0 msgr2=0x55d40f74ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:53.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.388+0000 7fc7ecc23640 1 --2- 172.21.15.117:0/2621960827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d40f3439f0 0x55d40f74ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:53.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.388+0000 7fc7ecc23640 1 -- 172.21.15.117:0/2621960827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d40f364390 msgr2=0x55d40f7b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:53.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.388+0000 7fc7ecc23640 1 --2- 172.21.15.117:0/2621960827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d40f364390 0x55d40f7b3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:53.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.388+0000 7fc7ecc23640 1 -- 172.21.15.117:0/2621960827 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d40f74b950 con 0x55d40f35c1b0 2024-09-21T05:57:53.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.388+0000 7fc7edc25640 1 --2- 172.21.15.117:0/2621960827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d40f364390 0x55d40f7b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:57:53.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.388+0000 7fc7f5bfc740 10 client.0 _ll_register_callbacks cb 0x7fc7e40b5070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:57:53.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.388+0000 7fc7f5bfc740 1 -- 172.21.15.117:0/2621960827 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7e40b5ca0 con 0x55d40f35c1b0 2024-09-21T05:57:53.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.389+0000 7fc7ecc23640 1 --2- 172.21.15.117:0/2621960827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d40f35c1b0 0x55d40f74b160 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8005ab0 tx=0x7fc7d8032c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7e8c73f4742e749 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:53.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.389+0000 7fc7d0ff9640 10 client.25217 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-21T05:57:53.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.389+0000 7fc7d0ff9640 1 -- 172.21.15.117:0/2621960827 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8035040 con 0x55d40f35c1b0 2024-09-21T05:57:53.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.389+0000 7fc7d0ff9640 1 -- 172.21.15.117:0/2621960827 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fc7d8004030 con 0x55d40f35c1b0 2024-09-21T05:57:53.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.389+0000 7fc7d0ff9640 1 -- 172.21.15.117:0/2621960827 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc7d8004230 con 0x55d40f35c1b0 2024-09-21T05:57:53.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.389+0000 7fc7f5bfc740 10 client.25217 fetch_fsmap learned FSMap version 27 2024-09-21T05:57:53.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.389+0000 7fc7f5bfc740 1 -- 172.21.15.117:0/2621960827 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55d40f7b4420 con 0x55d40f35c1b0 2024-09-21T05:57:53.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.390+0000 7fc7d0ff9640 1 -- 172.21.15.117:0/2621960827 <== mon.2 v2:172.21.15.138:3300/0 4 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc7d8005040 con 0x55d40f35c1b0 2024-09-21T05:57:53.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.390+0000 7fc7f5bfc740 10 client.25217 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:57:53.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.390+0000 7fc7f5bfc740 10 client.25217 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T05:57:53.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.390+0000 7fc7f5bfc740 20 client.25217 populate_metadata read hostname 'smithi117' 2024-09-21T05:57:53.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.390+0000 7fc7f5bfc740 1 --2- 172.21.15.117:0/2621960827 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55d40f385070 0x55d40f7b6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:53.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.390+0000 7fc7f5bfc740 4 client.25217 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T05:57:53.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.390+0000 7fc7f5bfc740 1 -- 172.21.15.117:0/2621960827 --> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d40f6a4df0 con 0x55d40f385070 2024-09-21T05:57:53.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.391+0000 7fc7ed424640 1 --2- 172.21.15.117:0/2621960827 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55d40f385070 0x55d40f7b6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:53.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.391+0000 7fc7d0ff9640 10 client.25217 ms_handle_connect on v2:172.21.15.192:6834/3530688860 2024-09-21T05:57:53.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.391+0000 7fc7ed424640 1 --2- 172.21.15.117:0/2621960827 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55d40f385070 0x55d40f7b6b50 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6ebd8670fe966980 server_cookie=7030134507c39c44 in_seq=0 out_seq=0 2024-09-21T05:57:53.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.391+0000 7fc7d0ff9640 1 -- 172.21.15.117:0/2621960827 <== mds.3 v2:172.21.15.192:6834/3530688860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d40f6a4df0 con 0x55d40f385070 2024-09-21T05:57:53.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.391+0000 7fc7d0ff9640 10 client.25217 handle_command_reply: tid=0 2024-09-21T05:57:53.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.391+0000 7fc7f5bfc740 1 client.25217 shutdown 2024-09-21T05:57:53.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.392+0000 7fc7f5bfc740 1 -- 172.21.15.117:0/2621960827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d40f35c1b0 msgr2=0x55d40f74b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:53.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.392+0000 7fc7f5bfc740 1 --2- 172.21.15.117:0/2621960827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d40f35c1b0 0x55d40f74b160 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8005ab0 tx=0x7fc7d8032c60 comp rx=0 tx=0).stop 2024-09-21T05:57:53.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.392+0000 7fc7f5bfc740 1 -- 172.21.15.117:0/2621960827 shutdown_connections 2024-09-21T05:57:53.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.392+0000 7fc7f5bfc740 1 --2- 172.21.15.117:0/2621960827 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55d40f385070 0x55d40f7b6b50 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:53.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.392+0000 7fc7f5bfc740 1 --2- 172.21.15.117:0/2621960827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d40f364390 0x55d40f7b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:53.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.392+0000 7fc7f5bfc740 1 --2- 172.21.15.117:0/2621960827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d40f35c1b0 0x55d40f74b160 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:53.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.392+0000 7fc7f5bfc740 1 --2- 172.21.15.117:0/2621960827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d40f3439f0 0x55d40f74ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:53.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.392+0000 7fc7f5bfc740 1 -- 172.21.15.117:0/2621960827 >> 172.21.15.117:0/2621960827 conn(0x55d40f379460 msgr2=0x55d40f72da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:53.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.392+0000 7fc7d0ff9640 0 client.25217 ms_handle_reset on v2:172.21.15.192:6834/3530688860 2024-09-21T05:57:53.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.392+0000 7fc7f5bfc740 1 -- 172.21.15.117:0/2621960827 shutdown_connections 2024-09-21T05:57:53.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.392+0000 7fc7f5bfc740 1 -- 172.21.15.117:0/2621960827 wait complete. 2024-09-21T05:57:53.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.392+0000 7fc7f5bfc740 20 client.25217 trim_cache size 0 max 16384 2024-09-21T05:57:53.397 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T05:57:53.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.393+0000 7fc7ef688640 1 -- 172.21.15.117:0/478199419 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc7d40b6670 msgr2=0x7fc7d40b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:53.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.393+0000 7fc7ef688640 1 --2- 172.21.15.117:0/478199419 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc7d40b6670 0x7fc7d40b8b80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc7e00092f0 tx=0x7fc7e0009220 comp rx=0 tx=0).stop 2024-09-21T05:57:53.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.393+0000 7fc7ef688640 1 -- 172.21.15.117:0/478199419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7e8102a60 msgr2=0x7fc7e819a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:53.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.393+0000 7fc7ef688640 1 --2- 172.21.15.117:0/478199419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7e8102a60 0x7fc7e819a650 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc7e400d770 tx=0x7fc7e400dc30 comp rx=0 tx=0).stop 2024-09-21T05:57:53.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.393+0000 7fc7ef688640 1 -- 172.21.15.117:0/478199419 shutdown_connections 2024-09-21T05:57:53.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.393+0000 7fc7ef688640 1 --2- 172.21.15.117:0/478199419 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc7d40b6670 0x7fc7d40b8b80 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:53.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.393+0000 7fc7ef688640 1 --2- 172.21.15.117:0/478199419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7e819fdd0 0x7fc7e81a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:53.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.393+0000 7fc7ef688640 1 --2- 172.21.15.117:0/478199419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7e8108cf0 0x7fc7e819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:53.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.393+0000 7fc7ef688640 1 --2- 172.21.15.117:0/478199419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7e8102a60 0x7fc7e819a650 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:53.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.393+0000 7fc7ef688640 1 -- 172.21.15.117:0/478199419 >> 172.21.15.117:0/478199419 conn(0x7fc7e8079730 msgr2=0x7fc7e81067f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:53.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.393+0000 7fc7ef688640 1 -- 172.21.15.117:0/478199419 shutdown_connections 2024-09-21T05:57:53.399 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:53.393+0000 7fc7ef688640 1 -- 172.21.15.117:0/478199419 wait complete. 2024-09-21T05:57:53.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:53 smithi192 ceph-mon[34700]: pgmap v344: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-21T05:57:54.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:53 smithi117 ceph-mon[30947]: pgmap v344: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-21T05:57:54.097 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T05:57:54.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:53 smithi138 ceph-mon[34391]: pgmap v344: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-21T05:57:54.419 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:57:55.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.000+0000 7f93cb496640 1 -- 172.21.15.117:0/1721867329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93c41083a0 msgr2=0x7f93c41087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:55.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.000+0000 7f93cb496640 1 --2- 172.21.15.117:0/1721867329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93c41083a0 0x7f93c41087b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f93b4009900 tx=0x7f93b402a410 comp rx=0 tx=0).stop 2024-09-21T05:57:55.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.000+0000 7f93cb496640 1 -- 172.21.15.117:0/1721867329 shutdown_connections 2024-09-21T05:57:55.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.000+0000 7f93cb496640 1 --2- 172.21.15.117:0/1721867329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93c41083a0 0x7f93c41087b0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.000+0000 7f93cb496640 1 --2- 172.21.15.117:0/1721867329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93c4108cf0 0x7f93c410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.000+0000 7f93cb496640 1 --2- 172.21.15.117:0/1721867329 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93c4102a60 0x7f93c4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.000+0000 7f93cb496640 1 -- 172.21.15.117:0/1721867329 >> 172.21.15.117:0/1721867329 conn(0x7f93c4079730 msgr2=0x7f93c4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:55.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.001+0000 7f93cb496640 1 -- 172.21.15.117:0/1721867329 shutdown_connections 2024-09-21T05:57:55.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.001+0000 7f93cb496640 1 -- 172.21.15.117:0/1721867329 wait complete. 2024-09-21T05:57:55.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.002+0000 7f93cb496640 1 Processor -- start 2024-09-21T05:57:55.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.002+0000 7f93cb496640 1 -- start start 2024-09-21T05:57:55.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.002+0000 7f93cb496640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93c4102a60 0x7f93c40ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.002+0000 7f93cb496640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93c4108cf0 0x7f93c40ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.002+0000 7f93cb496640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93c41000f0 0x7f93c41a86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.002+0000 7f93c9232640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93c4102a60 0x7f93c40ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:55.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.002+0000 7f93c9232640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93c4102a60 0x7f93c40ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41318/0 (socket says 172.21.15.117:41318) 2024-09-21T05:57:55.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.002+0000 7f93c9232640 1 -- 172.21.15.117:0/3375986989 learned_addr learned my addr 172.21.15.117:0/3375986989 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:55.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.002+0000 7f93cb496640 1 -- 172.21.15.117:0/3375986989 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f93c41129b0 con 0x7f93c4102a60 2024-09-21T05:57:55.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.002+0000 7f93cb496640 1 -- 172.21.15.117:0/3375986989 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f93c4112830 con 0x7f93c4108cf0 2024-09-21T05:57:55.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.002+0000 7f93cb496640 1 -- 172.21.15.117:0/3375986989 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f93c4112b30 con 0x7f93c41000f0 2024-09-21T05:57:55.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.002+0000 7f93c9a33640 1 --2- 172.21.15.117:0/3375986989 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93c41000f0 0x7f93c41a86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:55.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.003+0000 7f93c8a31640 1 --2- 172.21.15.117:0/3375986989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93c4108cf0 0x7f93c40ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:55.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.003+0000 7f93c9232640 1 -- 172.21.15.117:0/3375986989 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93c41000f0 msgr2=0x7f93c41a86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:55.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.003+0000 7f93c9232640 1 --2- 172.21.15.117:0/3375986989 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93c41000f0 0x7f93c41a86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.003+0000 7f93c9232640 1 -- 172.21.15.117:0/3375986989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93c4108cf0 msgr2=0x7f93c40ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:55.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.003+0000 7f93c9232640 1 --2- 172.21.15.117:0/3375986989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93c4108cf0 0x7f93c40ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.003+0000 7f93c9232640 1 -- 172.21.15.117:0/3375986989 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c41a8c90 con 0x7f93c4102a60 2024-09-21T05:57:55.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.003+0000 7f93c9232640 1 --2- 172.21.15.117:0/3375986989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93c4102a60 0x7f93c40ff670 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f93c000d770 tx=0x7f93c000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:55.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.004+0000 7f93ba7fc640 1 -- 172.21.15.117:0/3375986989 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c0017070 con 0x7f93c4102a60 2024-09-21T05:57:55.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.004+0000 7f93ba7fc640 1 -- 172.21.15.117:0/3375986989 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93c00054d0 con 0x7f93c4102a60 2024-09-21T05:57:55.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.004+0000 7f93ba7fc640 1 -- 172.21.15.117:0/3375986989 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c0004d00 con 0x7f93c4102a60 2024-09-21T05:57:55.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.004+0000 7f93cb496640 1 -- 172.21.15.117:0/3375986989 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93c41a8f80 con 0x7f93c4102a60 2024-09-21T05:57:55.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.004+0000 7f93cb496640 1 -- 172.21.15.117:0/3375986989 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93c41a93e0 con 0x7f93c4102a60 2024-09-21T05:57:55.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.008+0000 7f93d1a0a740 1 Processor -- start 2024-09-21T05:57:55.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.008+0000 7f93d1a0a740 1 -- start start 2024-09-21T05:57:55.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.008+0000 7f93d1a0a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c08561070 0x556c08908290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.008+0000 7f93d1a0a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c085381b0 0x556c089087d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.008+0000 7f93d1a0a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c08540390 0x556c08908d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.008+0000 7f93d1a0a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556c0887eea0 con 0x556c085381b0 2024-09-21T05:57:55.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.008+0000 7f93d1a0a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556c0886e2d0 con 0x556c08540390 2024-09-21T05:57:55.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.008+0000 7f93d1a0a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556c088720e0 con 0x556c08561070 2024-09-21T05:57:55.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.008+0000 7f93ba7fc640 1 -- 172.21.15.117:0/3375986989 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 23) ==== 154587+0+0 (secure 0 0 0) 0x7f93c0010460 con 0x7f93c4102a60 2024-09-21T05:57:55.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.009+0000 7f93c9a33640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c08561070 0x556c08908290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:55.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.009+0000 7f93c8a31640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c08540390 0x556c08908d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:55.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.009+0000 7f93c8a31640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c08540390 0x556c08908d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:54608/0 (socket says 172.21.15.117:54608) 2024-09-21T05:57:55.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.009+0000 7f93c8a31640 1 -- 172.21.15.117:0/630044556 learned_addr learned my addr 172.21.15.117:0/630044556 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:55.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.009+0000 7f93c9232640 1 --2- 172.21.15.117:0/630044556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c085381b0 0x556c089087d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:55.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.009+0000 7f93c9232640 1 -- 172.21.15.117:0/630044556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c08561070 msgr2=0x556c08908290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:55.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.009+0000 7f93c9232640 1 --2- 172.21.15.117:0/630044556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c08561070 0x556c08908290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.009+0000 7f93c9232640 1 -- 172.21.15.117:0/630044556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c08540390 msgr2=0x556c08908d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:55.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.009+0000 7f93c9232640 1 --2- 172.21.15.117:0/630044556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c08540390 0x556c08908d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.009+0000 7f93c9232640 1 -- 172.21.15.117:0/630044556 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c085cc650 con 0x556c085381b0 2024-09-21T05:57:55.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.010+0000 7f93c9232640 1 --2- 172.21.15.117:0/630044556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c085381b0 0x556c089087d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f93c00e4870 tx=0x7f93c00ef6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3624f0938dc9bff9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:55.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.010+0000 7f93af7fe640 1 -- 172.21.15.117:0/630044556 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c00fa070 con 0x556c085381b0 2024-09-21T05:57:55.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.010+0000 7f93ba7fc640 1 --2- 172.21.15.117:0/3375986989 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f93b00b6670 0x7f93b00b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.010+0000 7f93af7fe640 1 -- 172.21.15.117:0/630044556 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93c00acab0 con 0x556c085381b0 2024-09-21T05:57:55.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.010+0000 7f93c9a33640 1 --2- 172.21.15.117:0/3375986989 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f93b00b6670 0x7f93b00b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:55.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.011+0000 7f93ba7fc640 1 -- 172.21.15.117:0/3375986989 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f93c00e6f60 con 0x7f93c4102a60 2024-09-21T05:57:55.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.011+0000 7f93af7fe640 1 -- 172.21.15.117:0/630044556 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c00ac8f0 con 0x556c085381b0 2024-09-21T05:57:55.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.012+0000 7f93c9a33640 1 --2- 172.21.15.117:0/3375986989 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f93b00b6670 0x7f93b00b8b80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f93bc005fd0 tx=0x7f93bc009600 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:55.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.012+0000 7f93d1a0a740 1 -- 172.21.15.117:0/630044556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c085381b0 msgr2=0x556c089087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:55.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.012+0000 7f93d1a0a740 1 --2- 172.21.15.117:0/630044556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c085381b0 0x556c089087d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f93c00e4870 tx=0x7f93c00ef6f0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.012+0000 7f93d1a0a740 1 -- 172.21.15.117:0/630044556 shutdown_connections 2024-09-21T05:57:55.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.012+0000 7f93d1a0a740 1 --2- 172.21.15.117:0/630044556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c085381b0 0x556c089087d0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.012+0000 7f93d1a0a740 1 --2- 172.21.15.117:0/630044556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c08540390 0x556c08908d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.012+0000 7f93d1a0a740 1 --2- 172.21.15.117:0/630044556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c08561070 0x556c08908290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.012+0000 7f93d1a0a740 1 -- 172.21.15.117:0/630044556 >> 172.21.15.117:0/630044556 conn(0x556c08555460 msgr2=0x556c08906180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:55.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.012+0000 7f93d1a0a740 1 -- 172.21.15.117:0/630044556 shutdown_connections 2024-09-21T05:57:55.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.012+0000 7f93d1a0a740 1 -- 172.21.15.117:0/630044556 wait complete. 2024-09-21T05:57:55.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.012+0000 7f93d1a0a740 1 -- start start 2024-09-21T05:57:55.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.013+0000 7f93d1a0a740 1 Processor -- start 2024-09-21T05:57:55.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.013+0000 7f93d1a0a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c085381b0 0x556c0891bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.013+0000 7f93d1a0a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c08540390 0x556c0891c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.013+0000 7f93d1a0a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c08561070 0x556c0891ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.013+0000 7f93d1a0a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556c086cf700 con 0x556c08540390 2024-09-21T05:57:55.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.014+0000 7f93d1a0a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556c08809160 con 0x556c085381b0 2024-09-21T05:57:55.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.014+0000 7f93c8a31640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c08540390 0x556c0891c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:55.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.014+0000 7f93c8a31640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c08540390 0x556c0891c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41326/0 (socket says 172.21.15.117:41326) 2024-09-21T05:57:55.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.014+0000 7f93c8a31640 1 -- 172.21.15.117:0/4247725527 learned_addr learned my addr 172.21.15.117:0/4247725527 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:55.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.014+0000 7f93d1a0a740 1 -- 172.21.15.117:0/4247725527 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556c0891f240 con 0x556c08561070 2024-09-21T05:57:55.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.014+0000 7f93c9232640 1 --2- 172.21.15.117:0/4247725527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c085381b0 0x556c0891bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:55.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.014+0000 7f93c8a31640 1 -- 172.21.15.117:0/4247725527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c08561070 msgr2=0x556c0891ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:55.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.014+0000 7f93c8a31640 1 --2- 172.21.15.117:0/4247725527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c08561070 0x556c0891ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.014+0000 7f93c8a31640 1 -- 172.21.15.117:0/4247725527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c085381b0 msgr2=0x556c0891bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:55.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.014+0000 7f93c8a31640 1 --2- 172.21.15.117:0/4247725527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c085381b0 0x556c0891bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.014+0000 7f93c8a31640 1 -- 172.21.15.117:0/4247725527 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c08836c90 con 0x556c08540390 2024-09-21T05:57:55.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.014+0000 7f93c9232640 1 --2- 172.21.15.117:0/4247725527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c085381b0 0x556c0891bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:57:55.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.014+0000 7f93d1a0a740 10 client.0 _ll_register_callbacks cb 0x556c088229e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:57:55.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.014+0000 7f93acff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:57:55.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.014+0000 7f93c8a31640 1 --2- 172.21.15.117:0/4247725527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c08540390 0x556c0891c500 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f93c406aaa0 tx=0x7f93b4004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b3abda0ae1fe0b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:55.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.015+0000 7f93d1a0a740 1 -- 172.21.15.117:0/4247725527 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93c00acab0 con 0x556c08540390 2024-09-21T05:57:55.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.015+0000 7f93acff9640 1 -- 172.21.15.117:0/4247725527 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b4035040 con 0x556c08540390 2024-09-21T05:57:55.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.015+0000 7f93acff9640 1 -- 172.21.15.117:0/4247725527 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f93b4005550 con 0x556c08540390 2024-09-21T05:57:55.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.015+0000 7f93acff9640 1 -- 172.21.15.117:0/4247725527 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f93b4004750 con 0x556c08540390 2024-09-21T05:57:55.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.015+0000 7f93d1a0a740 10 client.15684 fetch_fsmap learned FSMap version 27 2024-09-21T05:57:55.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.015+0000 7f93d1a0a740 1 -- 172.21.15.117:0/4247725527 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({fsmap=27}) -- 0x556c0893bd80 con 0x556c08540390 2024-09-21T05:57:55.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.016+0000 7f93acff9640 1 -- 172.21.15.117:0/4247725527 <== mon.0 v2:172.21.15.117:3300/0 4 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f93b40048f0 con 0x556c08540390 2024-09-21T05:57:55.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.016+0000 7f93d1a0a740 10 client.15684 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:57:55.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.016+0000 7f93d1a0a740 10 client.15684 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T05:57:55.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.016+0000 7f93d1a0a740 20 client.15684 populate_metadata read hostname 'smithi117' 2024-09-21T05:57:55.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.016+0000 7f93d1a0a740 1 --2- 172.21.15.117:0/4247725527 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x556c0851f9f0 0x556c0893f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.016+0000 7f93d1a0a740 4 client.15684 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:57:55.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.016+0000 7f93d1a0a740 1 -- 172.21.15.117:0/4247725527 --> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c08880df0 con 0x556c0851f9f0 2024-09-21T05:57:55.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.016+0000 7f93c9232640 1 --2- 172.21.15.117:0/4247725527 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x556c0851f9f0 0x556c0893f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:55.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.017+0000 7f93c9232640 1 --2- 172.21.15.117:0/4247725527 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x556c0851f9f0 0x556c0893f2d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a049a3d4042188f server_cookie=286fd204e8ccf7e in_seq=0 out_seq=0 2024-09-21T05:57:55.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.017+0000 7f93acff9640 10 client.15684 ms_handle_connect on v2:172.21.15.192:6836/2683860842 2024-09-21T05:57:55.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.017+0000 7f93acff9640 1 -- 172.21.15.117:0/4247725527 <== mds.2 v2:172.21.15.192:6836/2683860842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556c08880df0 con 0x556c0851f9f0 2024-09-21T05:57:55.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.017+0000 7f93acff9640 10 client.15684 handle_command_reply: tid=0 2024-09-21T05:57:55.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.018+0000 7f93d1a0a740 1 client.15684 shutdown 2024-09-21T05:57:55.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.018+0000 7f93d1a0a740 1 -- 172.21.15.117:0/4247725527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c08540390 msgr2=0x556c0891c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:55.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.018+0000 7f93d1a0a740 1 --2- 172.21.15.117:0/4247725527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c08540390 0x556c0891c500 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f93c406aaa0 tx=0x7f93b4004560 comp rx=0 tx=0).stop 2024-09-21T05:57:55.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.018+0000 7f93d1a0a740 1 -- 172.21.15.117:0/4247725527 shutdown_connections 2024-09-21T05:57:55.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.018+0000 7f93d1a0a740 1 --2- 172.21.15.117:0/4247725527 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x556c0851f9f0 0x556c0893f2d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.018+0000 7f93acff9640 0 client.15684 ms_handle_reset on v2:172.21.15.192:6836/2683860842 2024-09-21T05:57:55.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.018+0000 7f93d1a0a740 1 --2- 172.21.15.117:0/4247725527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c08540390 0x556c0891c500 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.018+0000 7f93d1a0a740 1 --2- 172.21.15.117:0/4247725527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c08561070 0x556c0891ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.018+0000 7f93d1a0a740 1 --2- 172.21.15.117:0/4247725527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c085381b0 0x556c0891bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.018+0000 7f93d1a0a740 1 -- 172.21.15.117:0/4247725527 >> 172.21.15.117:0/4247725527 conn(0x556c08555460 msgr2=0x556c08904720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:55.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.018+0000 7f93d1a0a740 1 -- 172.21.15.117:0/4247725527 shutdown_connections 2024-09-21T05:57:55.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.019+0000 7f93d1a0a740 1 -- 172.21.15.117:0/4247725527 wait complete. 2024-09-21T05:57:55.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.019+0000 7f93d1a0a740 20 client.15684 trim_cache size 0 max 16384 2024-09-21T05:57:55.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.044+0000 7f93d1a0a740 1 Processor -- start 2024-09-21T05:57:55.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.044+0000 7f93d1a0a740 1 -- start start 2024-09-21T05:57:55.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.044+0000 7f93d1a0a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c0851f9f0 0x556c089060c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.044+0000 7f93d1a0a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c085381b0 0x556c0891dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.044+0000 7f93d1a0a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c08540390 0x556c089191b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.044+0000 7f93d1a0a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f93c4070f80 con 0x556c08540390 2024-09-21T05:57:55.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.044+0000 7f93d1a0a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f93c4072ac0 con 0x556c085381b0 2024-09-21T05:57:55.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.044+0000 7f93d1a0a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f93c4071650 con 0x556c0851f9f0 2024-09-21T05:57:55.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.045+0000 7f93c9a33640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c08540390 0x556c089191b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:55.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.045+0000 7f93c9a33640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c08540390 0x556c089191b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41332/0 (socket says 172.21.15.117:41332) 2024-09-21T05:57:55.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.045+0000 7f93c9a33640 1 -- 172.21.15.117:0/2390672797 learned_addr learned my addr 172.21.15.117:0/2390672797 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:55.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.045+0000 7f93c8a31640 1 --2- 172.21.15.117:0/2390672797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c085381b0 0x556c0891dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:55.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.045+0000 7f93c9232640 1 --2- 172.21.15.117:0/2390672797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c0851f9f0 0x556c089060c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:55.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.045+0000 7f93c9a33640 1 -- 172.21.15.117:0/2390672797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c0851f9f0 msgr2=0x556c089060c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:55.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.045+0000 7f93c9a33640 1 --2- 172.21.15.117:0/2390672797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c0851f9f0 0x556c089060c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.045+0000 7f93c9a33640 1 -- 172.21.15.117:0/2390672797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c085381b0 msgr2=0x556c0891dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:55.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.045+0000 7f93c9a33640 1 --2- 172.21.15.117:0/2390672797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c085381b0 0x556c0891dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.045+0000 7f93c9a33640 1 -- 172.21.15.117:0/2390672797 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c0891cd80 con 0x556c08540390 2024-09-21T05:57:55.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.045+0000 7f93c9232640 1 --2- 172.21.15.117:0/2390672797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c0851f9f0 0x556c089060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:57:55.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.045+0000 7f93c9a33640 1 --2- 172.21.15.117:0/2390672797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c08540390 0x556c089191b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f93bc01d640 tx=0x7f93bc014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc293430b9c3752 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:55.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.045+0000 7f93aeffd640 1 -- 172.21.15.117:0/2390672797 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc01d310 con 0x556c08540390 2024-09-21T05:57:55.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.046+0000 7f93aeffd640 1 -- 172.21.15.117:0/2390672797 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93bc003970 con 0x556c08540390 2024-09-21T05:57:55.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.046+0000 7f93aeffd640 1 -- 172.21.15.117:0/2390672797 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc004730 con 0x556c08540390 2024-09-21T05:57:55.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.047+0000 7f93d1a0a740 1 -- 172.21.15.117:0/2390672797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c08540390 msgr2=0x556c089191b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:55.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.047+0000 7f93d1a0a740 1 --2- 172.21.15.117:0/2390672797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c08540390 0x556c089191b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f93bc01d640 tx=0x7f93bc014040 comp rx=0 tx=0).stop 2024-09-21T05:57:55.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.047+0000 7f93d1a0a740 1 -- 172.21.15.117:0/2390672797 shutdown_connections 2024-09-21T05:57:55.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.047+0000 7f93d1a0a740 1 --2- 172.21.15.117:0/2390672797 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c08540390 0x556c089191b0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.047+0000 7f93d1a0a740 1 --2- 172.21.15.117:0/2390672797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c085381b0 0x556c0891dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.047+0000 7f93d1a0a740 1 --2- 172.21.15.117:0/2390672797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c0851f9f0 0x556c089060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.047+0000 7f93d1a0a740 1 -- 172.21.15.117:0/2390672797 >> 172.21.15.117:0/2390672797 conn(0x556c08555460 msgr2=0x556c0891d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:55.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.047+0000 7f93d1a0a740 1 -- 172.21.15.117:0/2390672797 shutdown_connections 2024-09-21T05:57:55.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.047+0000 7f93d1a0a740 1 -- 172.21.15.117:0/2390672797 wait complete. 2024-09-21T05:57:55.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.048+0000 7f93d1a0a740 1 -- start start 2024-09-21T05:57:55.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.048+0000 7f93d1a0a740 1 Processor -- start 2024-09-21T05:57:55.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.048+0000 7f93d1a0a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c0851f9f0 0x556c08931d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.049+0000 7f93d1a0a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c085381b0 0x556c089322c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.049+0000 7f93c9232640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c0851f9f0 0x556c08931d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:55.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.049+0000 7f93c9232640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c0851f9f0 0x556c08931d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41344/0 (socket says 172.21.15.117:41344) 2024-09-21T05:57:55.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.049+0000 7f93c9232640 1 -- 172.21.15.117:0/1564286620 learned_addr learned my addr 172.21.15.117:0/1564286620 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:55.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.049+0000 7f93d1a0a740 1 --2- 172.21.15.117:0/1564286620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c08540390 0x556c089a8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.049+0000 7f93d1a0a740 1 -- 172.21.15.117:0/1564286620 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f93c40723f0 con 0x556c0851f9f0 2024-09-21T05:57:55.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.049+0000 7f93d1a0a740 1 -- 172.21.15.117:0/1564286620 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f93c4071d20 con 0x556c085381b0 2024-09-21T05:57:55.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.049+0000 7f93d1a0a740 1 -- 172.21.15.117:0/1564286620 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f93c406ef60 con 0x556c08540390 2024-09-21T05:57:55.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.049+0000 7f93c9a33640 1 --2- 172.21.15.117:0/1564286620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c08540390 0x556c089a8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:55.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.049+0000 7f93d1a0a740 10 client.0 _ll_register_callbacks cb 0x7f93b4009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:57:55.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.049+0000 7f93c9232640 1 -- 172.21.15.117:0/1564286620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c08540390 msgr2=0x556c089a8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:55.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.049+0000 7f93c9232640 1 --2- 172.21.15.117:0/1564286620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c08540390 0x556c089a8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.049+0000 7f93c9232640 1 -- 172.21.15.117:0/1564286620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c085381b0 msgr2=0x556c089322c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T05:57:55.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.049+0000 7f93c9232640 1 --2- 172.21.15.117:0/1564286620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c085381b0 0x556c089322c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.049+0000 7f93c9232640 1 -- 172.21.15.117:0/1564286620 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c089a9220 con 0x556c0851f9f0 2024-09-21T05:57:55.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.050+0000 7f93c9232640 1 --2- 172.21.15.117:0/1564286620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c0851f9f0 0x556c08931d80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f93c00a9570 tx=0x7f93c00ef190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18936a1f89eb2677 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:55.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.050+0000 7f93acff9640 10 client.15690 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:57:55.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.050+0000 7f93acff9640 1 -- 172.21.15.117:0/1564286620 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c00c8720 con 0x556c0851f9f0 2024-09-21T05:57:55.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.050+0000 7f93d1a0a740 1 -- 172.21.15.117:0/1564286620 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93bc003970 con 0x556c0851f9f0 2024-09-21T05:57:55.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.050+0000 7f93acff9640 1 -- 172.21.15.117:0/1564286620 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f93c00eac20 con 0x556c0851f9f0 2024-09-21T05:57:55.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.050+0000 7f93acff9640 1 -- 172.21.15.117:0/1564286620 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f93c00c8900 con 0x556c0851f9f0 2024-09-21T05:57:55.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.050+0000 7f93d1a0a740 10 client.15690 fetch_fsmap learned FSMap version 27 2024-09-21T05:57:55.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.050+0000 7f93d1a0a740 1 -- 172.21.15.117:0/1564286620 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({fsmap=27}) -- 0x556c089aa3b0 con 0x556c0851f9f0 2024-09-21T05:57:55.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.051+0000 7f93acff9640 1 -- 172.21.15.117:0/1564286620 <== mon.0 v2:172.21.15.117:3300/0 4 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f93c00a9210 con 0x556c0851f9f0 2024-09-21T05:57:55.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.051+0000 7f93d1a0a740 10 client.15690 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:57:55.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.051+0000 7f93d1a0a740 10 client.15690 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T05:57:55.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.051+0000 7f93d1a0a740 20 client.15690 populate_metadata read hostname 'smithi117' 2024-09-21T05:57:55.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.051+0000 7f93d1a0a740 1 --2- 172.21.15.117:0/1564286620 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x556c08561070 0x556c089ac640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.051+0000 7f93d1a0a740 4 client.15690 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T05:57:55.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.051+0000 7f93d1a0a740 1 -- 172.21.15.117:0/1564286620 --> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f93bc003970 con 0x556c08561070 2024-09-21T05:57:55.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.051+0000 7f93c8a31640 1 --2- 172.21.15.117:0/1564286620 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x556c08561070 0x556c089ac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:55.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.052+0000 7f93acff9640 10 client.15690 ms_handle_connect on v2:172.21.15.192:6836/2683860842 2024-09-21T05:57:55.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.052+0000 7f93c8a31640 1 --2- 172.21.15.117:0/1564286620 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x556c08561070 0x556c089ac640 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c418bfdb5957dff0 server_cookie=7aa2c4c8bdc18324 in_seq=0 out_seq=0 2024-09-21T05:57:55.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.052+0000 7f93acff9640 1 -- 172.21.15.117:0/1564286620 <== mds.2 v2:172.21.15.192:6836/2683860842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f93bc003970 con 0x556c08561070 2024-09-21T05:57:55.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.052+0000 7f93acff9640 10 client.15690 handle_command_reply: tid=0 2024-09-21T05:57:55.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.052+0000 7f93d1a0a740 1 client.15690 shutdown 2024-09-21T05:57:55.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.052+0000 7f93d1a0a740 1 -- 172.21.15.117:0/1564286620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c0851f9f0 msgr2=0x556c08931d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:55.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.052+0000 7f93d1a0a740 1 --2- 172.21.15.117:0/1564286620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c0851f9f0 0x556c08931d80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f93c00a9570 tx=0x7f93c00ef190 comp rx=0 tx=0).stop 2024-09-21T05:57:55.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.052+0000 7f93d1a0a740 1 -- 172.21.15.117:0/1564286620 shutdown_connections 2024-09-21T05:57:55.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.052+0000 7f93d1a0a740 1 --2- 172.21.15.117:0/1564286620 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x556c08561070 0x556c089ac640 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.052+0000 7f93d1a0a740 1 --2- 172.21.15.117:0/1564286620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556c08540390 0x556c089a8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.052+0000 7f93acff9640 0 client.15690 ms_handle_reset on v2:172.21.15.192:6836/2683860842 2024-09-21T05:57:55.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.052+0000 7f93d1a0a740 1 --2- 172.21.15.117:0/1564286620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556c085381b0 0x556c089322c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.052+0000 7f93d1a0a740 1 --2- 172.21.15.117:0/1564286620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556c0851f9f0 0x556c08931d80 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.052+0000 7f93d1a0a740 1 -- 172.21.15.117:0/1564286620 >> 172.21.15.117:0/1564286620 conn(0x556c08555460 msgr2=0x556c0891c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:55.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.052+0000 7f93d1a0a740 1 -- 172.21.15.117:0/1564286620 shutdown_connections 2024-09-21T05:57:55.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.053+0000 7f93d1a0a740 1 -- 172.21.15.117:0/1564286620 wait complete. 2024-09-21T05:57:55.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.053+0000 7f93d1a0a740 20 client.15690 trim_cache size 0 max 16384 2024-09-21T05:57:55.057 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T05:57:55.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.053+0000 7f93cb496640 1 -- 172.21.15.117:0/3375986989 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f93b00b6670 msgr2=0x7f93b00b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:55.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.053+0000 7f93cb496640 1 --2- 172.21.15.117:0/3375986989 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f93b00b6670 0x7f93b00b8b80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f93bc005fd0 tx=0x7f93bc009600 comp rx=0 tx=0).stop 2024-09-21T05:57:55.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.054+0000 7f93cb496640 1 -- 172.21.15.117:0/3375986989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93c4102a60 msgr2=0x7f93c40ff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:55.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.054+0000 7f93cb496640 1 --2- 172.21.15.117:0/3375986989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93c4102a60 0x7f93c40ff670 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f93c000d770 tx=0x7f93c000dc30 comp rx=0 tx=0).stop 2024-09-21T05:57:55.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.055+0000 7f93cb496640 1 -- 172.21.15.117:0/3375986989 shutdown_connections 2024-09-21T05:57:55.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.055+0000 7f93cb496640 1 --2- 172.21.15.117:0/3375986989 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f93b00b6670 0x7f93b00b8b80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.055+0000 7f93cb496640 1 --2- 172.21.15.117:0/3375986989 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93c41000f0 0x7f93c41a86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.055+0000 7f93cb496640 1 --2- 172.21.15.117:0/3375986989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93c4108cf0 0x7f93c40ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.055+0000 7f93cb496640 1 --2- 172.21.15.117:0/3375986989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93c4102a60 0x7f93c40ff670 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.055+0000 7f93cb496640 1 -- 172.21.15.117:0/3375986989 >> 172.21.15.117:0/3375986989 conn(0x7f93c4079730 msgr2=0x7f93c406b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:55.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.055+0000 7f93cb496640 1 -- 172.21.15.117:0/3375986989 shutdown_connections 2024-09-21T05:57:55.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:55.055+0000 7f93cb496640 1 -- 172.21.15.117:0/3375986989 wait complete. 2024-09-21T05:57:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.337+0000 7f0002884640 1 -- 172.21.15.117:0/789878847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efffc108400 msgr2=0x7efffc10ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.337+0000 7f0002884640 1 --2- 172.21.15.117:0/789878847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efffc108400 0x7efffc10ec30 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7efff400adb0 tx=0x7efff402b7a0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.337+0000 7f0002884640 1 -- 172.21.15.117:0/789878847 shutdown_connections 2024-09-21T05:57:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.337+0000 7f0002884640 1 --2- 172.21.15.117:0/789878847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efffc108400 0x7efffc10ec30 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.337+0000 7f0002884640 1 --2- 172.21.15.117:0/789878847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efffc107ab0 0x7efffc107ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.337+0000 7f0002884640 1 --2- 172.21.15.117:0/789878847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efffc102180 0x7efffc102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.338+0000 7f0002884640 1 -- 172.21.15.117:0/789878847 >> 172.21.15.117:0/789878847 conn(0x7efffc079e90 msgr2=0x7efffc07a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.338+0000 7f0002884640 1 -- 172.21.15.117:0/789878847 shutdown_connections 2024-09-21T05:57:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.338+0000 7f0002884640 1 -- 172.21.15.117:0/789878847 wait complete. 2024-09-21T05:57:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.338+0000 7f0002884640 1 Processor -- start 2024-09-21T05:57:55.341 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.339+0000 7f0002884640 1 -- start start 2024-09-21T05:57:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.339+0000 7f0002884640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efffc102180 0x7efffc199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.342 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.339+0000 7f0002884640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efffc107ab0 0x7efffc1980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.339+0000 7f0002884640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efffc108400 0x7efffc198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.339+0000 7f0002884640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efffc111840 con 0x7efffc108400 2024-09-21T05:57:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.339+0000 7f0002884640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efffc1116c0 con 0x7efffc102180 2024-09-21T05:57:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.339+0000 7f0002884640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efffc1119c0 con 0x7efffc107ab0 2024-09-21T05:57:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.339+0000 7f0000e21640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efffc108400 0x7efffc198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.339+0000 7f0000e21640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efffc108400 0x7efffc198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41366/0 (socket says 172.21.15.117:41366) 2024-09-21T05:57:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.339+0000 7f0000e21640 1 -- 172.21.15.117:0/223342269 learned_addr learned my addr 172.21.15.117:0/223342269 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.339+0000 7efffbfff640 1 --2- 172.21.15.117:0/223342269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efffc102180 0x7efffc199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.339+0000 7efffb7fe640 1 --2- 172.21.15.117:0/223342269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efffc107ab0 0x7efffc1980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.340+0000 7efffbfff640 1 -- 172.21.15.117:0/223342269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efffc107ab0 msgr2=0x7efffc1980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.340+0000 7efffbfff640 1 --2- 172.21.15.117:0/223342269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efffc107ab0 0x7efffc1980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.340+0000 7efffbfff640 1 -- 172.21.15.117:0/223342269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efffc108400 msgr2=0x7efffc198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.340+0000 7efffbfff640 1 --2- 172.21.15.117:0/223342269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efffc108400 0x7efffc198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.343 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.340+0000 7efffbfff640 1 -- 172.21.15.117:0/223342269 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efffc198e40 con 0x7efffc102180 2024-09-21T05:57:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.340+0000 7f0000e21640 1 --2- 172.21.15.117:0/223342269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efffc108400 0x7efffc198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:57:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.340+0000 7efffbfff640 1 --2- 172.21.15.117:0/223342269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efffc102180 0x7efffc199b00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7effec00e9f0 tx=0x7effec00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.340+0000 7efff97fa640 1 -- 172.21.15.117:0/223342269 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec00ccc0 con 0x7efffc102180 2024-09-21T05:57:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.340+0000 7efff97fa640 1 -- 172.21.15.117:0/223342269 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effec0108e0 con 0x7efffc102180 2024-09-21T05:57:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.340+0000 7f0002884640 1 -- 172.21.15.117:0/223342269 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efffc1acee0 con 0x7efffc102180 2024-09-21T05:57:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.340+0000 7efff97fa640 1 -- 172.21.15.117:0/223342269 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec018880 con 0x7efffc102180 2024-09-21T05:57:55.344 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.340+0000 7f0002884640 1 -- 172.21.15.117:0/223342269 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efffc1ad3a0 con 0x7efffc102180 2024-09-21T05:57:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.343+0000 7efff97fa640 1 -- 172.21.15.117:0/223342269 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 23) ==== 154587+0+0 (secure 0 0 0) 0x7effec010430 con 0x7efffc102180 2024-09-21T05:57:55.345 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.344+0000 7f00097f2740 1 Processor -- start 2024-09-21T05:57:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.344+0000 7f00097f2740 1 -- start start 2024-09-21T05:57:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.344+0000 7f00097f2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569a22d11b0 0x5569a26769b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.347 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.344+0000 7f00097f2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569a22a82f0 0x5569a2676ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.344+0000 7f00097f2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569a22b04d0 0x5569a2677430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.344+0000 7f00097f2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5569a25eb6f0 con 0x5569a22d11b0 2024-09-21T05:57:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.344+0000 7f00097f2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5569a25dab20 con 0x5569a22a82f0 2024-09-21T05:57:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.344+0000 7f00097f2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5569a25de930 con 0x5569a22b04d0 2024-09-21T05:57:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.344+0000 7efff97fa640 1 --2- 172.21.15.117:0/223342269 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7effe40b65f0 0x7effe40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.344+0000 7f0000e21640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569a22d11b0 0x5569a26769b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.344+0000 7f0000e21640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569a22d11b0 0x5569a26769b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41372/0 (socket says 172.21.15.117:41372) 2024-09-21T05:57:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.344+0000 7f0000e21640 1 -- 172.21.15.117:0/4184704934 learned_addr learned my addr 172.21.15.117:0/4184704934 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.345+0000 7efffb7fe640 1 --2- 172.21.15.117:0/4184704934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569a22b04d0 0x5569a2677430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.345+0000 7efffbfff640 1 --2- 172.21.15.117:0/4184704934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569a22a82f0 0x5569a2676ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.345+0000 7f0000e21640 1 --2- 172.21.15.117:0/223342269 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7effe40b65f0 0x7effe40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.345+0000 7f0000e21640 1 -- 172.21.15.117:0/4184704934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569a22b04d0 msgr2=0x5569a2677430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.345+0000 7efff97fa640 1 -- 172.21.15.117:0/223342269 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7effec0e7930 con 0x7efffc102180 2024-09-21T05:57:55.348 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.345+0000 7f0000e21640 1 --2- 172.21.15.117:0/4184704934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569a22b04d0 0x5569a2677430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.345+0000 7f0000e21640 1 -- 172.21.15.117:0/4184704934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569a22a82f0 msgr2=0x5569a2676ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.345+0000 7f0000e21640 1 --2- 172.21.15.117:0/4184704934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569a22a82f0 0x5569a2676ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.345+0000 7f0000e21640 1 -- 172.21.15.117:0/4184704934 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569a25a0530 con 0x5569a22d11b0 2024-09-21T05:57:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.345+0000 7f0000e21640 1 --2- 172.21.15.117:0/4184704934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569a22d11b0 0x5569a26769b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7efff400aed0 tx=0x7efff4003af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9736b8bab698c7b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.345+0000 7effe27fc640 1 -- 172.21.15.117:0/4184704934 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff4032600 con 0x5569a22d11b0 2024-09-21T05:57:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.346+0000 7effe27fc640 1 -- 172.21.15.117:0/4184704934 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff4050070 con 0x5569a22d11b0 2024-09-21T05:57:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.346+0000 7effe27fc640 1 -- 172.21.15.117:0/4184704934 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff4031610 con 0x5569a22d11b0 2024-09-21T05:57:55.349 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.346+0000 7f0000e21640 1 --2- 172.21.15.117:0/223342269 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7effe40b65f0 0x7effe40b8b00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efff404f000 tx=0x7efff404d000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:55.351 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.347+0000 7f00097f2740 1 -- 172.21.15.117:0/4184704934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569a22d11b0 msgr2=0x5569a26769b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.347+0000 7f00097f2740 1 --2- 172.21.15.117:0/4184704934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569a22d11b0 0x5569a26769b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7efff400aed0 tx=0x7efff4003af0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.347+0000 7f00097f2740 1 -- 172.21.15.117:0/4184704934 shutdown_connections 2024-09-21T05:57:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.347+0000 7f00097f2740 1 --2- 172.21.15.117:0/4184704934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569a22b04d0 0x5569a2677430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.347+0000 7f00097f2740 1 --2- 172.21.15.117:0/4184704934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569a22a82f0 0x5569a2676ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.347+0000 7f00097f2740 1 --2- 172.21.15.117:0/4184704934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569a22d11b0 0x5569a26769b0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.347+0000 7f00097f2740 1 -- 172.21.15.117:0/4184704934 >> 172.21.15.117:0/4184704934 conn(0x5569a22c5300 msgr2=0x5569a26748a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.347+0000 7f00097f2740 1 -- 172.21.15.117:0/4184704934 shutdown_connections 2024-09-21T05:57:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.347+0000 7f00097f2740 1 -- 172.21.15.117:0/4184704934 wait complete. 2024-09-21T05:57:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.347+0000 7f00097f2740 1 -- start start 2024-09-21T05:57:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.348+0000 7f00097f2740 1 Processor -- start 2024-09-21T05:57:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.348+0000 7f00097f2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569a22a82f0 0x5569a2682070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.348+0000 7f00097f2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569a22b04d0 0x5569a26825b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.348+0000 7f00097f2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569a22d11b0 0x5569a268e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.348+0000 7f00097f2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5569a2512ab0 con 0x5569a22a82f0 2024-09-21T05:57:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.348+0000 7efffbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569a22a82f0 0x5569a2682070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.348+0000 7f00097f2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5569a2580730 con 0x5569a22d11b0 2024-09-21T05:57:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.348+0000 7f00097f2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5569a2685130 con 0x5569a22b04d0 2024-09-21T05:57:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.348+0000 7efffbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569a22a82f0 0x5569a2682070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41388/0 (socket says 172.21.15.117:41388) 2024-09-21T05:57:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.348+0000 7efffbfff640 1 -- 172.21.15.117:0/3746460376 learned_addr learned my addr 172.21.15.117:0/3746460376 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.349+0000 7f00097f2740 10 client.0 _ll_register_callbacks cb 0x5569a25d46b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:57:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.349+0000 7efffb7fe640 1 --2- 172.21.15.117:0/3746460376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569a22b04d0 0x5569a26825b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.349+0000 7f0000e21640 1 --2- 172.21.15.117:0/3746460376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569a22d11b0 0x5569a268e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.349+0000 7f0000e21640 1 -- 172.21.15.117:0/3746460376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569a22b04d0 msgr2=0x5569a26825b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.349+0000 7f0000e21640 1 --2- 172.21.15.117:0/3746460376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569a22b04d0 0x5569a26825b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.353 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.349+0000 7f0000e21640 1 -- 172.21.15.117:0/3746460376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569a22a82f0 msgr2=0x5569a2682070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.349+0000 7f0000e21640 1 --2- 172.21.15.117:0/3746460376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569a22a82f0 0x5569a2682070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.349+0000 7f0000e21640 1 -- 172.21.15.117:0/3746460376 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569a233c790 con 0x5569a22d11b0 2024-09-21T05:57:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.349+0000 7efffbfff640 1 --2- 172.21.15.117:0/3746460376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569a22a82f0 0x5569a2682070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:57:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.349+0000 7f0000e21640 1 --2- 172.21.15.117:0/3746460376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569a22d11b0 0x5569a268e450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efff4006010 tx=0x7efff400a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1cd8bf4d40c3b84 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.349+0000 7effdb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-21T05:57:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.350+0000 7effdb7fe640 1 -- 172.21.15.117:0/3746460376 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff4032600 con 0x5569a22d11b0 2024-09-21T05:57:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.350+0000 7effdb7fe640 1 -- 172.21.15.117:0/3746460376 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7efff40327a0 con 0x5569a22d11b0 2024-09-21T05:57:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.350+0000 7f00097f2740 1 -- 172.21.15.117:0/3746460376 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efff4050070 con 0x5569a22d11b0 2024-09-21T05:57:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.350+0000 7effdb7fe640 1 -- 172.21.15.117:0/3746460376 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efff40318e0 con 0x5569a22d11b0 2024-09-21T05:57:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.350+0000 7f00097f2740 10 client.25241 fetch_fsmap learned FSMap version 27 2024-09-21T05:57:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.350+0000 7f00097f2740 1 -- 172.21.15.117:0/3746460376 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5569a23b5620 con 0x5569a22d11b0 2024-09-21T05:57:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.350+0000 7effdb7fe640 1 -- 172.21.15.117:0/3746460376 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff4050070 con 0x5569a22d11b0 2024-09-21T05:57:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.350+0000 7effdb7fe640 1 -- 172.21.15.117:0/3746460376 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7efff40308b0 con 0x5569a22d11b0 2024-09-21T05:57:55.356 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.350+0000 7f00097f2740 10 client.25241 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:57:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.350+0000 7f00097f2740 10 client.25241 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T05:57:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.350+0000 7f00097f2740 20 client.25241 populate_metadata read hostname 'smithi117' 2024-09-21T05:57:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.351+0000 7f00097f2740 1 --2- 172.21.15.117:0/3746460376 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5569a228fd80 0x5569a2694010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.351+0000 7f00097f2740 4 client.25241 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:57:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.351+0000 7f00097f2740 1 -- 172.21.15.117:0/3746460376 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569a25ed640 con 0x5569a228fd80 2024-09-21T05:57:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.351+0000 7efffbfff640 1 --2- 172.21.15.117:0/3746460376 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5569a228fd80 0x5569a2694010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.351+0000 7effdb7fe640 10 client.25241 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:57:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.351+0000 7efffbfff640 1 --2- 172.21.15.117:0/3746460376 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5569a228fd80 0x5569a2694010 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=864bcf329cf31165 server_cookie=322b11178a32ae0d in_seq=0 out_seq=0 2024-09-21T05:57:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.352+0000 7effdb7fe640 1 -- 172.21.15.117:0/3746460376 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5569a25ed640 con 0x5569a228fd80 2024-09-21T05:57:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.352+0000 7effdb7fe640 10 client.25241 handle_command_reply: tid=0 2024-09-21T05:57:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.352+0000 7f00097f2740 1 client.25241 shutdown 2024-09-21T05:57:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.352+0000 7f00097f2740 1 -- 172.21.15.117:0/3746460376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569a22d11b0 msgr2=0x5569a268e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:55.357 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.352+0000 7f00097f2740 1 --2- 172.21.15.117:0/3746460376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569a22d11b0 0x5569a268e450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efff4006010 tx=0x7efff400a6d0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.352+0000 7f00097f2740 1 -- 172.21.15.117:0/3746460376 shutdown_connections 2024-09-21T05:57:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.352+0000 7f00097f2740 1 --2- 172.21.15.117:0/3746460376 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5569a228fd80 0x5569a2694010 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.352+0000 7effdb7fe640 0 client.25241 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:57:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.352+0000 7f00097f2740 1 --2- 172.21.15.117:0/3746460376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569a22d11b0 0x5569a268e450 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.352+0000 7f00097f2740 1 --2- 172.21.15.117:0/3746460376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569a22b04d0 0x5569a26825b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.352+0000 7f00097f2740 1 --2- 172.21.15.117:0/3746460376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569a22a82f0 0x5569a2682070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.353+0000 7f00097f2740 1 -- 172.21.15.117:0/3746460376 >> 172.21.15.117:0/3746460376 conn(0x5569a22c5300 msgr2=0x5569a26779e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.353+0000 7f00097f2740 1 -- 172.21.15.117:0/3746460376 shutdown_connections 2024-09-21T05:57:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.353+0000 7f00097f2740 1 -- 172.21.15.117:0/3746460376 wait complete. 2024-09-21T05:57:55.358 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.353+0000 7f00097f2740 20 client.25241 trim_cache size 0 max 16384 2024-09-21T05:57:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.381+0000 7f00097f2740 1 Processor -- start 2024-09-21T05:57:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.381+0000 7f00097f2740 1 -- start start 2024-09-21T05:57:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.382+0000 7f00097f2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569a22a82f0 0x5569a26780f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.382+0000 7f00097f2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569a22b04d0 0x5569a268e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.382+0000 7f00097f2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569a22d11b0 0x5569a267f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.382+0000 7f00097f2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efffc1b5b90 con 0x5569a22d11b0 2024-09-21T05:57:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.382+0000 7f00097f2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efffc1b76d0 con 0x5569a22a82f0 2024-09-21T05:57:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.382+0000 7f00097f2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efffc1b6260 con 0x5569a22b04d0 2024-09-21T05:57:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.382+0000 7f0000e21640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569a22d11b0 0x5569a267f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.382+0000 7efffbfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569a22a82f0 0x5569a26780f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.382+0000 7f0000e21640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569a22d11b0 0x5569a267f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41400/0 (socket says 172.21.15.117:41400) 2024-09-21T05:57:55.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.382+0000 7f0000e21640 1 -- 172.21.15.117:0/877719898 learned_addr learned my addr 172.21.15.117:0/877719898 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:55.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.382+0000 7efffb7fe640 1 --2- 172.21.15.117:0/877719898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569a22b04d0 0x5569a268e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.382+0000 7f0000e21640 1 -- 172.21.15.117:0/877719898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569a22b04d0 msgr2=0x5569a268e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.382+0000 7f0000e21640 1 --2- 172.21.15.117:0/877719898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569a22b04d0 0x5569a268e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.382+0000 7f0000e21640 1 -- 172.21.15.117:0/877719898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569a22a82f0 msgr2=0x5569a26780f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.382+0000 7f0000e21640 1 --2- 172.21.15.117:0/877719898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569a22a82f0 0x5569a26780f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.382+0000 7f0000e21640 1 -- 172.21.15.117:0/877719898 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569a2685010 con 0x5569a22d11b0 2024-09-21T05:57:55.386 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.382+0000 7f0000e21640 1 --2- 172.21.15.117:0/877719898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569a22d11b0 0x5569a267f240 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7efff4003fd0 tx=0x7efff4004670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf947225d75542b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.383+0000 7effe1ffb640 1 -- 172.21.15.117:0/877719898 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff4031a80 con 0x5569a22d11b0 2024-09-21T05:57:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.383+0000 7effe1ffb640 1 -- 172.21.15.117:0/877719898 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff4050070 con 0x5569a22d11b0 2024-09-21T05:57:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.383+0000 7effe1ffb640 1 -- 172.21.15.117:0/877719898 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff4030770 con 0x5569a22d11b0 2024-09-21T05:57:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.384+0000 7f00097f2740 1 -- 172.21.15.117:0/877719898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569a22d11b0 msgr2=0x5569a267f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.384+0000 7f00097f2740 1 --2- 172.21.15.117:0/877719898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569a22d11b0 0x5569a267f240 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7efff4003fd0 tx=0x7efff4004670 comp rx=0 tx=0).stop 2024-09-21T05:57:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.384+0000 7f00097f2740 1 -- 172.21.15.117:0/877719898 shutdown_connections 2024-09-21T05:57:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.384+0000 7f00097f2740 1 --2- 172.21.15.117:0/877719898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569a22d11b0 0x5569a267f240 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.384+0000 7f00097f2740 1 --2- 172.21.15.117:0/877719898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569a22b04d0 0x5569a268e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.384+0000 7f00097f2740 1 --2- 172.21.15.117:0/877719898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569a22a82f0 0x5569a26780f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.384+0000 7f00097f2740 1 -- 172.21.15.117:0/877719898 >> 172.21.15.117:0/877719898 conn(0x5569a22c5300 msgr2=0x5569a26779e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.384+0000 7f00097f2740 1 -- 172.21.15.117:0/877719898 shutdown_connections 2024-09-21T05:57:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.384+0000 7f00097f2740 1 -- 172.21.15.117:0/877719898 wait complete. 2024-09-21T05:57:55.387 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.385+0000 7f00097f2740 1 -- start start 2024-09-21T05:57:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.385+0000 7f00097f2740 1 Processor -- start 2024-09-21T05:57:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.386+0000 7f00097f2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569a22a82f0 0x5569a2696890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.386+0000 7f00097f2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569a22b04d0 0x5569a2696e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.386+0000 7efffbfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569a22a82f0 0x5569a2696890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.386+0000 7efffbfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569a22a82f0 0x5569a2696890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:54692/0 (socket says 172.21.15.117:54692) 2024-09-21T05:57:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.386+0000 7efffb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569a22b04d0 0x5569a2696e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.386+0000 7efffb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569a22b04d0 0x5569a2696e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41416/0 (socket says 172.21.15.117:41416) 2024-09-21T05:57:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.386+0000 7efffbfff640 1 -- 172.21.15.117:0/2224888518 learned_addr learned my addr 172.21.15.117:0/2224888518 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.386+0000 7f00097f2740 1 --2- 172.21.15.117:0/2224888518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569a22d11b0 0x5569a26fc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.386+0000 7f00097f2740 1 -- 172.21.15.117:0/2224888518 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efffc1b7000 con 0x5569a22b04d0 2024-09-21T05:57:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.386+0000 7f00097f2740 1 -- 172.21.15.117:0/2224888518 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efffc1b6930 con 0x5569a22a82f0 2024-09-21T05:57:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.386+0000 7f00097f2740 1 -- 172.21.15.117:0/2224888518 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efffc1b3b70 con 0x5569a22d11b0 2024-09-21T05:57:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.386+0000 7f00097f2740 10 client.0 _ll_register_callbacks cb 0x7efff400ac30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:57:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.386+0000 7efffb7fe640 1 -- 172.21.15.117:0/2224888518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569a22d11b0 msgr2=0x5569a26fc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.386+0000 7efffb7fe640 1 --2- 172.21.15.117:0/2224888518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569a22d11b0 0x5569a26fc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.386+0000 7efffb7fe640 1 -- 172.21.15.117:0/2224888518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569a22a82f0 msgr2=0x5569a2696890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.386+0000 7efffb7fe640 1 --2- 172.21.15.117:0/2224888518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569a22a82f0 0x5569a2696890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.386+0000 7efffb7fe640 1 -- 172.21.15.117:0/2224888518 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569a26fcdd0 con 0x5569a22b04d0 2024-09-21T05:57:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.386+0000 7efffbfff640 1 --2- 172.21.15.117:0/2224888518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569a22a82f0 0x5569a2696890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:57:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.387+0000 7efffb7fe640 1 --2- 172.21.15.117:0/2224888518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569a22b04d0 0x5569a2696e30 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7efffc1076f0 tx=0x7efff0012bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677b1b6fe8a1f0c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.387+0000 7effdb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:57:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.387+0000 7effdb7fe640 1 -- 172.21.15.117:0/2224888518 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff0023060 con 0x5569a22b04d0 2024-09-21T05:57:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.387+0000 7effdb7fe640 1 -- 172.21.15.117:0/2224888518 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7efff0004230 con 0x5569a22b04d0 2024-09-21T05:57:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.387+0000 7f00097f2740 1 -- 172.21.15.117:0/2224888518 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efff4050070 con 0x5569a22b04d0 2024-09-21T05:57:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.387+0000 7effdb7fe640 1 -- 172.21.15.117:0/2224888518 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efff0004430 con 0x5569a22b04d0 2024-09-21T05:57:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.387+0000 7f00097f2740 10 client.15708 fetch_fsmap learned FSMap version 27 2024-09-21T05:57:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.387+0000 7f00097f2740 1 -- 172.21.15.117:0/2224888518 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5569a26fe0d0 con 0x5569a22b04d0 2024-09-21T05:57:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.388+0000 7effdb7fe640 1 -- 172.21.15.117:0/2224888518 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff4050070 con 0x5569a22b04d0 2024-09-21T05:57:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.388+0000 7effdb7fe640 1 -- 172.21.15.117:0/2224888518 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7efff0003ae0 con 0x5569a22b04d0 2024-09-21T05:57:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.388+0000 7f00097f2740 10 client.15708 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:57:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.388+0000 7f00097f2740 10 client.15708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T05:57:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.388+0000 7f00097f2740 20 client.15708 populate_metadata read hostname 'smithi117' 2024-09-21T05:57:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.388+0000 7f00097f2740 1 --2- 172.21.15.117:0/2224888518 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5569a22651c0 0x5569a2700890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:55.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.388+0000 7f00097f2740 4 client.15708 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T05:57:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.388+0000 7f00097f2740 1 -- 172.21.15.117:0/2224888518 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5569a25e90f0 con 0x5569a22651c0 2024-09-21T05:57:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.388+0000 7efffbfff640 1 --2- 172.21.15.117:0/2224888518 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5569a22651c0 0x5569a2700890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.389+0000 7effdb7fe640 10 client.15708 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:57:55.393 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.389+0000 7efffbfff640 1 --2- 172.21.15.117:0/2224888518 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5569a22651c0 0x5569a2700890 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e39dc52d88f9f server_cookie=4469bde662bee341 in_seq=0 out_seq=0 2024-09-21T05:57:55.398 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.396+0000 7effdb7fe640 1 -- 172.21.15.117:0/2224888518 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5569a25e90f0 con 0x5569a22651c0 2024-09-21T05:57:55.398 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.396+0000 7effdb7fe640 10 client.15708 handle_command_reply: tid=0 2024-09-21T05:57:55.398 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.396+0000 7f00097f2740 1 client.15708 shutdown 2024-09-21T05:57:55.399 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.396+0000 7f00097f2740 1 -- 172.21.15.117:0/2224888518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569a22b04d0 msgr2=0x5569a2696e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:55.399 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.396+0000 7f00097f2740 1 --2- 172.21.15.117:0/2224888518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569a22b04d0 0x5569a2696e30 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7efffc1076f0 tx=0x7efff0012bd0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.399 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.396+0000 7f00097f2740 1 -- 172.21.15.117:0/2224888518 shutdown_connections 2024-09-21T05:57:55.399 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.396+0000 7f00097f2740 1 --2- 172.21.15.117:0/2224888518 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5569a22651c0 0x5569a2700890 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.399 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.396+0000 7f00097f2740 1 --2- 172.21.15.117:0/2224888518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569a22b04d0 0x5569a2696e30 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.399 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.396+0000 7f00097f2740 1 --2- 172.21.15.117:0/2224888518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569a22d11b0 0x5569a26fc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.399 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.396+0000 7f00097f2740 1 --2- 172.21.15.117:0/2224888518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569a22a82f0 0x5569a2696890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.399 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.396+0000 7f00097f2740 1 -- 172.21.15.117:0/2224888518 >> 172.21.15.117:0/2224888518 conn(0x5569a22c5300 msgr2=0x5569a2674880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:55.399 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.396+0000 7effdb7fe640 0 client.15708 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:57:55.399 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.397+0000 7f00097f2740 1 -- 172.21.15.117:0/2224888518 shutdown_connections 2024-09-21T05:57:55.399 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.397+0000 7f00097f2740 1 -- 172.21.15.117:0/2224888518 wait complete. 2024-09-21T05:57:55.399 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.397+0000 7f00097f2740 20 client.15708 trim_cache size 0 max 16384 2024-09-21T05:57:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T05:57:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T05:57:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T05:57:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T05:57:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.397+0000 7f0002884640 1 -- 172.21.15.117:0/223342269 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7effe40b65f0 msgr2=0x7effe40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.397+0000 7f0002884640 1 --2- 172.21.15.117:0/223342269 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7effe40b65f0 0x7effe40b8b00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efff404f000 tx=0x7efff404d000 comp rx=0 tx=0).stop 2024-09-21T05:57:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.397+0000 7f0002884640 1 -- 172.21.15.117:0/223342269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efffc102180 msgr2=0x7efffc199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.397+0000 7f0002884640 1 --2- 172.21.15.117:0/223342269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efffc102180 0x7efffc199b00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7effec00e9f0 tx=0x7effec00eeb0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.397+0000 7f0002884640 1 -- 172.21.15.117:0/223342269 shutdown_connections 2024-09-21T05:57:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.397+0000 7f0002884640 1 --2- 172.21.15.117:0/223342269 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7effe40b65f0 0x7effe40b8b00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.397+0000 7f0002884640 1 --2- 172.21.15.117:0/223342269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efffc108400 0x7efffc198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.397+0000 7f0002884640 1 --2- 172.21.15.117:0/223342269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efffc107ab0 0x7efffc1980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.397+0000 7f0002884640 1 --2- 172.21.15.117:0/223342269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efffc102180 0x7efffc199b00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.398+0000 7f0002884640 1 -- 172.21.15.117:0/223342269 >> 172.21.15.117:0/223342269 conn(0x7efffc079e90 msgr2=0x7efffc105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.398+0000 7f0002884640 1 -- 172.21.15.117:0/223342269 shutdown_connections 2024-09-21T05:57:55.402 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:57:55.398+0000 7f0002884640 1 -- 172.21.15.117:0/223342269 wait complete. 2024-09-21T05:57:55.802 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T05:57:55.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:55 smithi192 ceph-mon[34700]: pgmap v345: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-21T05:57:56.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:55 smithi117 ceph-mon[30947]: pgmap v345: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-21T05:57:56.075 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:57:56.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:55 smithi138 ceph-mon[34391]: pgmap v345: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-21T05:57:56.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.648+0000 7fa845bc1640 1 -- 172.21.15.117:0/512288851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa840102010 msgr2=0x7fa840102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:56.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.648+0000 7fa845bc1640 1 --2- 172.21.15.117:0/512288851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa840102010 0x7fa840102420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa830009860 tx=0x7fa83002a340 comp rx=0 tx=0).stop 2024-09-21T05:57:56.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.648+0000 7fa845bc1640 1 -- 172.21.15.117:0/512288851 shutdown_connections 2024-09-21T05:57:56.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.648+0000 7fa845bc1640 1 --2- 172.21.15.117:0/512288851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8401094c0 0x7fa8401098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:56.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.648+0000 7fa845bc1640 1 --2- 172.21.15.117:0/512288851 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa840109e10 0x7fa8401106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:56.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.648+0000 7fa845bc1640 1 --2- 172.21.15.117:0/512288851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa840102010 0x7fa840102420 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:56.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.648+0000 7fa845bc1640 1 -- 172.21.15.117:0/512288851 >> 172.21.15.117:0/512288851 conn(0x7fa840077c60 msgr2=0x7fa8400780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:56.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.648+0000 7fa845bc1640 1 -- 172.21.15.117:0/512288851 shutdown_connections 2024-09-21T05:57:56.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.648+0000 7fa845bc1640 1 -- 172.21.15.117:0/512288851 wait complete. 2024-09-21T05:57:56.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.649+0000 7fa845bc1640 1 Processor -- start 2024-09-21T05:57:56.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.649+0000 7fa845bc1640 1 -- start start 2024-09-21T05:57:56.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.649+0000 7fa845bc1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa840102010 0x7fa840198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:56.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.649+0000 7fa845bc1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8401094c0 0x7fa8401988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:56.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.649+0000 7fa845bc1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa840109e10 0x7fa84019db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:56.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.650+0000 7fa845bc1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa8401118d0 con 0x7fa840109e10 2024-09-21T05:57:56.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.650+0000 7fa845bc1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa840111750 con 0x7fa840102010 2024-09-21T05:57:56.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.650+0000 7fa845bc1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa840111a50 con 0x7fa8401094c0 2024-09-21T05:57:56.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.650+0000 7fa83effd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8401094c0 0x7fa8401988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:56.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.650+0000 7fa83effd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8401094c0 0x7fa8401988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:33510/0 (socket says 172.21.15.117:33510) 2024-09-21T05:57:56.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.650+0000 7fa83effd640 1 -- 172.21.15.117:0/2110329053 learned_addr learned my addr 172.21.15.117:0/2110329053 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:56.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.650+0000 7fa83f7fe640 1 --2- 172.21.15.117:0/2110329053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa840102010 0x7fa840198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:56.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.650+0000 7fa83ffff640 1 --2- 172.21.15.117:0/2110329053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa840109e10 0x7fa84019db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:56.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.650+0000 7fa83effd640 1 -- 172.21.15.117:0/2110329053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa840102010 msgr2=0x7fa840198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:56.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.650+0000 7fa83effd640 1 --2- 172.21.15.117:0/2110329053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa840102010 0x7fa840198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:56.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.650+0000 7fa83effd640 1 -- 172.21.15.117:0/2110329053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa840109e10 msgr2=0x7fa84019db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:56.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.650+0000 7fa83effd640 1 --2- 172.21.15.117:0/2110329053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa840109e10 0x7fa84019db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:56.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.650+0000 7fa83effd640 1 -- 172.21.15.117:0/2110329053 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84019e290 con 0x7fa8401094c0 2024-09-21T05:57:56.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.650+0000 7fa83f7fe640 1 --2- 172.21.15.117:0/2110329053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa840102010 0x7fa840198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:57:56.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.651+0000 7fa83effd640 1 --2- 172.21.15.117:0/2110329053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8401094c0 0x7fa8401988c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa8340058c0 tx=0x7fa83400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:56.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.651+0000 7fa83cff9640 1 -- 172.21.15.117:0/2110329053 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83400cd30 con 0x7fa8401094c0 2024-09-21T05:57:56.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.651+0000 7fa83cff9640 1 -- 172.21.15.117:0/2110329053 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8340035f0 con 0x7fa8401094c0 2024-09-21T05:57:56.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.651+0000 7fa83cff9640 1 -- 172.21.15.117:0/2110329053 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8340044e0 con 0x7fa8401094c0 2024-09-21T05:57:56.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.651+0000 7fa845bc1640 1 -- 172.21.15.117:0/2110329053 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa84019e580 con 0x7fa8401094c0 2024-09-21T05:57:56.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.651+0000 7fa845bc1640 1 -- 172.21.15.117:0/2110329053 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8401a4ce0 con 0x7fa8401094c0 2024-09-21T05:57:56.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.654+0000 7fa83cff9640 1 -- 172.21.15.117:0/2110329053 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 23) ==== 154587+0+0 (secure 0 0 0) 0x7fa834004030 con 0x7fa8401094c0 2024-09-21T05:57:56.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.655+0000 7fa84c135740 1 Processor -- start 2024-09-21T05:57:56.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.655+0000 7fa84c135740 1 -- start start 2024-09-21T05:57:56.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.655+0000 7fa83cff9640 1 --2- 172.21.15.117:0/2110329053 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa8280b6670 0x7fa8280b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:56.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.655+0000 7fa84c135740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558df3566070 0x558df3915eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:56.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.655+0000 7fa84c135740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558df353d1b0 0x558df39163f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:56.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.655+0000 7fa84c135740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df3545390 0x558df3916930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:56.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.655+0000 7fa84c135740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558df3883ea0 con 0x558df353d1b0 2024-09-21T05:57:56.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.655+0000 7fa84c135740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558df38732d0 con 0x558df3545390 2024-09-21T05:57:56.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.655+0000 7fa84c135740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558df38770e0 con 0x558df3566070 2024-09-21T05:57:56.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.655+0000 7fa83ffff640 1 --2- 172.21.15.117:0/2110329053 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa8280b6670 0x7fa8280b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:56.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.655+0000 7fa83effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558df353d1b0 0x558df39163f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:56.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.655+0000 7fa83f7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558df3566070 0x558df3915eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:56.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.656+0000 7fa83f7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558df3566070 0x558df3915eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:33514/0 (socket says 172.21.15.117:33514) 2024-09-21T05:57:56.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.656+0000 7fa83f7fe640 1 -- 172.21.15.117:0/1969196119 learned_addr learned my addr 172.21.15.117:0/1969196119 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:56.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.656+0000 7fa83ffff640 1 --2- 172.21.15.117:0/1969196119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df3545390 0x558df3916930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:56.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.656+0000 7fa83cff9640 1 -- 172.21.15.117:0/2110329053 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fa8340e7910 con 0x7fa8401094c0 2024-09-21T05:57:56.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.656+0000 7fa83effd640 1 -- 172.21.15.117:0/1969196119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558df3566070 msgr2=0x558df3915eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:56.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.656+0000 7fa83effd640 1 --2- 172.21.15.117:0/1969196119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558df3566070 0x558df3915eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:56.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.656+0000 7fa83effd640 1 -- 172.21.15.117:0/1969196119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df3545390 msgr2=0x558df3916930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:56.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.656+0000 7fa83effd640 1 --2- 172.21.15.117:0/1969196119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df3545390 0x558df3916930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:56.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.656+0000 7fa83effd640 1 -- 172.21.15.117:0/1969196119 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558df35d1650 con 0x558df353d1b0 2024-09-21T05:57:56.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.656+0000 7fa83f7fe640 1 --2- 172.21.15.117:0/1969196119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558df3566070 0x558df3915eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:57:56.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.656+0000 7fa83ffff640 1 --2- 172.21.15.117:0/2110329053 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa8280b6670 0x7fa8280b8b80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa838005fd0 tx=0x7fa838004410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:56.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.657+0000 7fa83effd640 1 --2- 172.21.15.117:0/1969196119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558df353d1b0 0x558df39163f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa8340e51c0 tx=0x7fa83401b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87300e3b2b2ea444 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:56.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.657+0000 7fa825ffb640 1 -- 172.21.15.117:0/1969196119 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8340ad190 con 0x558df353d1b0 2024-09-21T05:57:56.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.657+0000 7fa825ffb640 1 -- 172.21.15.117:0/1969196119 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa834009530 con 0x558df353d1b0 2024-09-21T05:57:56.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.657+0000 7fa825ffb640 1 -- 172.21.15.117:0/1969196119 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8340acfb0 con 0x558df353d1b0 2024-09-21T05:57:56.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.658+0000 7fa84c135740 1 -- 172.21.15.117:0/1969196119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558df353d1b0 msgr2=0x558df39163f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:56.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.658+0000 7fa84c135740 1 --2- 172.21.15.117:0/1969196119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558df353d1b0 0x558df39163f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa8340e51c0 tx=0x7fa83401b720 comp rx=0 tx=0).stop 2024-09-21T05:57:56.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.658+0000 7fa84c135740 1 -- 172.21.15.117:0/1969196119 shutdown_connections 2024-09-21T05:57:56.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.658+0000 7fa84c135740 1 --2- 172.21.15.117:0/1969196119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558df353d1b0 0x558df39163f0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:56.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.658+0000 7fa84c135740 1 --2- 172.21.15.117:0/1969196119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df3545390 0x558df3916930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:56.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.658+0000 7fa84c135740 1 --2- 172.21.15.117:0/1969196119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558df3566070 0x558df3915eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:56.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.658+0000 7fa84c135740 1 -- 172.21.15.117:0/1969196119 >> 172.21.15.117:0/1969196119 conn(0x558df355a460 msgr2=0x558df3913da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:56.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.658+0000 7fa84c135740 1 -- 172.21.15.117:0/1969196119 shutdown_connections 2024-09-21T05:57:56.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.658+0000 7fa84c135740 1 -- 172.21.15.117:0/1969196119 wait complete. 2024-09-21T05:57:56.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.659+0000 7fa84c135740 1 -- start start 2024-09-21T05:57:56.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.659+0000 7fa84c135740 1 Processor -- start 2024-09-21T05:57:56.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.659+0000 7fa84c135740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df353d1b0 0x558df3921490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:56.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.659+0000 7fa84c135740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558df3545390 0x558df39219d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:56.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.659+0000 7fa84c135740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558df3566070 0x558df3921f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:56.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.659+0000 7fa84c135740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558df36d4700 con 0x558df3566070 2024-09-21T05:57:56.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.659+0000 7fa84c135740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558df380e160 con 0x558df353d1b0 2024-09-21T05:57:56.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.659+0000 7fa84c135740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558df3924690 con 0x558df3545390 2024-09-21T05:57:56.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.660+0000 7fa84c135740 10 client.0 _ll_register_callbacks cb 0x558df38279e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:57:56.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.660+0000 7fa83ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558df3566070 0x558df3921f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:56.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.660+0000 7fa83f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df353d1b0 0x558df3921490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:56.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.660+0000 7fa83effd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558df3545390 0x558df39219d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:56.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.660+0000 7fa83effd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558df3545390 0x558df39219d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:33528/0 (socket says 172.21.15.117:33528) 2024-09-21T05:57:56.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.660+0000 7fa83effd640 1 -- 172.21.15.117:0/3115984497 learned_addr learned my addr 172.21.15.117:0/3115984497 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:56.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.660+0000 7fa83ffff640 1 -- 172.21.15.117:0/3115984497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558df3545390 msgr2=0x558df39219d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:56.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.660+0000 7fa83ffff640 1 --2- 172.21.15.117:0/3115984497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558df3545390 0x558df39219d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:56.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.660+0000 7fa83ffff640 1 -- 172.21.15.117:0/3115984497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df353d1b0 msgr2=0x558df3921490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:56.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.660+0000 7fa83ffff640 1 --2- 172.21.15.117:0/3115984497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df353d1b0 0x558df3921490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:56.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.660+0000 7fa83ffff640 1 -- 172.21.15.117:0/3115984497 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558df383bc90 con 0x558df3566070 2024-09-21T05:57:56.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.660+0000 7fa83ffff640 1 --2- 172.21.15.117:0/3115984497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558df3566070 0x558df3921f10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa83801c000 tx=0x7fa838004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6dfd465e69bf8d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:56.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.660+0000 7fa81effd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:57:56.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.661+0000 7fa81effd640 1 -- 172.21.15.117:0/3115984497 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa838024440 con 0x558df3566070 2024-09-21T05:57:56.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.661+0000 7fa81effd640 1 -- 172.21.15.117:0/3115984497 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fa8380245e0 con 0x558df3566070 2024-09-21T05:57:56.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.661+0000 7fa84c135740 1 -- 172.21.15.117:0/3115984497 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558df3885df0 con 0x558df3566070 2024-09-21T05:57:56.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.661+0000 7fa81effd640 1 -- 172.21.15.117:0/3115984497 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa8380247e0 con 0x558df3566070 2024-09-21T05:57:56.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.661+0000 7fa84c135740 10 client.15720 fetch_fsmap learned FSMap version 27 2024-09-21T05:57:56.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.661+0000 7fa84c135740 1 -- 172.21.15.117:0/3115984497 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558df39302c0 con 0x558df3566070 2024-09-21T05:57:56.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.662+0000 7fa81effd640 1 -- 172.21.15.117:0/3115984497 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558df3885df0 con 0x558df3566070 2024-09-21T05:57:56.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.662+0000 7fa81effd640 1 -- 172.21.15.117:0/3115984497 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa83801b710 con 0x558df3566070 2024-09-21T05:57:56.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.662+0000 7fa84c135740 10 client.15720 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:57:56.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.662+0000 7fa84c135740 10 client.15720 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T05:57:56.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.662+0000 7fa84c135740 20 client.15720 populate_metadata read hostname 'smithi117' 2024-09-21T05:57:56.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.662+0000 7fa84c135740 1 --2- 172.21.15.117:0/3115984497 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x558df35249f0 0x558df39337a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:56.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.662+0000 7fa84c135740 4 client.15720 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:57:56.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.662+0000 7fa84c135740 1 -- 172.21.15.117:0/3115984497 --> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558df38818a0 con 0x558df35249f0 2024-09-21T05:57:56.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.662+0000 7fa83f7fe640 1 --2- 172.21.15.117:0/3115984497 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x558df35249f0 0x558df39337a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:56.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.663+0000 7fa81effd640 10 client.15720 ms_handle_connect on v2:172.21.15.192:6838/1727623231 2024-09-21T05:57:56.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.663+0000 7fa83f7fe640 1 --2- 172.21.15.117:0/3115984497 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x558df35249f0 0x558df39337a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6ba15115a921275c server_cookie=6e89c05f714433bc in_seq=0 out_seq=0 2024-09-21T05:57:56.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.664+0000 7fa81effd640 1 -- 172.21.15.117:0/3115984497 <== mds.1 v2:172.21.15.192:6838/1727623231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558df38818a0 con 0x558df35249f0 2024-09-21T05:57:56.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.664+0000 7fa81effd640 10 client.15720 handle_command_reply: tid=0 2024-09-21T05:57:56.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.664+0000 7fa84c135740 1 client.15720 shutdown 2024-09-21T05:57:56.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.664+0000 7fa84c135740 1 -- 172.21.15.117:0/3115984497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558df3566070 msgr2=0x558df3921f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:56.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.664+0000 7fa84c135740 1 --2- 172.21.15.117:0/3115984497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558df3566070 0x558df3921f10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa83801c000 tx=0x7fa838004ab0 comp rx=0 tx=0).stop 2024-09-21T05:57:56.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.664+0000 7fa84c135740 1 -- 172.21.15.117:0/3115984497 shutdown_connections 2024-09-21T05:57:56.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.664+0000 7fa84c135740 1 --2- 172.21.15.117:0/3115984497 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x558df35249f0 0x558df39337a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:56.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.664+0000 7fa81effd640 0 client.15720 ms_handle_reset on v2:172.21.15.192:6838/1727623231 2024-09-21T05:57:56.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.664+0000 7fa84c135740 1 --2- 172.21.15.117:0/3115984497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558df3566070 0x558df3921f10 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:56.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.664+0000 7fa84c135740 1 --2- 172.21.15.117:0/3115984497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558df3545390 0x558df39219d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:56.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.664+0000 7fa84c135740 1 --2- 172.21.15.117:0/3115984497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df353d1b0 0x558df3921490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:56.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.664+0000 7fa84c135740 1 -- 172.21.15.117:0/3115984497 >> 172.21.15.117:0/3115984497 conn(0x558df355a460 msgr2=0x558df3919040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:56.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.664+0000 7fa84c135740 1 -- 172.21.15.117:0/3115984497 shutdown_connections 2024-09-21T05:57:56.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.664+0000 7fa84c135740 1 -- 172.21.15.117:0/3115984497 wait complete. 2024-09-21T05:57:56.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.665+0000 7fa84c135740 20 client.15720 trim_cache size 0 max 16384 2024-09-21T05:57:56.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.678+0000 7fa84c135740 1 Processor -- start 2024-09-21T05:57:56.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.678+0000 7fa84c135740 1 -- start start 2024-09-21T05:57:56.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.678+0000 7fa84c135740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558df35249f0 0x558df3923480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:56.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.678+0000 7fa84c135740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558df353d1b0 0x558df392d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:56.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.678+0000 7fa84c135740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df3545390 0x558df391e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:56.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.678+0000 7fa84c135740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa8401ad4c0 con 0x558df353d1b0 2024-09-21T05:57:56.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.678+0000 7fa84c135740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa8401af000 con 0x558df3545390 2024-09-21T05:57:56.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.678+0000 7fa84c135740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa8401adb90 con 0x558df35249f0 2024-09-21T05:57:56.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.678+0000 7fa83effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558df353d1b0 0x558df392d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:56.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.678+0000 7fa83effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558df353d1b0 0x558df392d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41448/0 (socket says 172.21.15.117:41448) 2024-09-21T05:57:56.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.678+0000 7fa83effd640 1 -- 172.21.15.117:0/1071068225 learned_addr learned my addr 172.21.15.117:0/1071068225 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:56.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.679+0000 7fa83effd640 1 -- 172.21.15.117:0/1071068225 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558df35249f0 msgr2=0x558df3923480 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T05:57:56.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.679+0000 7fa83effd640 1 --2- 172.21.15.117:0/1071068225 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558df35249f0 0x558df3923480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:56.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.679+0000 7fa83effd640 1 -- 172.21.15.117:0/1071068225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df3545390 msgr2=0x558df391e680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T05:57:56.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.679+0000 7fa83effd640 1 --2- 172.21.15.117:0/1071068225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df3545390 0x558df391e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:56.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.679+0000 7fa83effd640 1 -- 172.21.15.117:0/1071068225 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558df39181a0 con 0x558df353d1b0 2024-09-21T05:57:56.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.679+0000 7fa83effd640 1 --2- 172.21.15.117:0/1071068225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558df353d1b0 0x558df392d880 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa8340a9e10 tx=0x7fa8340e51a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d459d48cd9bf0211 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:56.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.679+0000 7fa8257fa640 1 -- 172.21.15.117:0/1071068225 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8340b2020 con 0x558df353d1b0 2024-09-21T05:57:56.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.679+0000 7fa8257fa640 1 -- 172.21.15.117:0/1071068225 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa834004320 con 0x558df353d1b0 2024-09-21T05:57:56.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.679+0000 7fa8257fa640 1 -- 172.21.15.117:0/1071068225 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8340a9ab0 con 0x558df353d1b0 2024-09-21T05:57:56.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.680+0000 7fa84c135740 1 -- 172.21.15.117:0/1071068225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558df353d1b0 msgr2=0x558df392d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:56.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.680+0000 7fa84c135740 1 --2- 172.21.15.117:0/1071068225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558df353d1b0 0x558df392d880 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa8340a9e10 tx=0x7fa8340e51a0 comp rx=0 tx=0).stop 2024-09-21T05:57:56.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.680+0000 7fa84c135740 1 -- 172.21.15.117:0/1071068225 shutdown_connections 2024-09-21T05:57:56.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.680+0000 7fa84c135740 1 --2- 172.21.15.117:0/1071068225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558df353d1b0 0x558df392d880 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:56.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.680+0000 7fa84c135740 1 --2- 172.21.15.117:0/1071068225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df3545390 0x558df391e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:56.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.680+0000 7fa84c135740 1 --2- 172.21.15.117:0/1071068225 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558df35249f0 0x558df3923480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:56.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.680+0000 7fa84c135740 1 -- 172.21.15.117:0/1071068225 >> 172.21.15.117:0/1071068225 conn(0x558df355a460 msgr2=0x558df39178a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:56.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.681+0000 7fa84c135740 1 -- 172.21.15.117:0/1071068225 shutdown_connections 2024-09-21T05:57:56.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.681+0000 7fa84c135740 1 -- 172.21.15.117:0/1071068225 wait complete. 2024-09-21T05:57:56.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.681+0000 7fa84c135740 1 -- start start 2024-09-21T05:57:56.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.681+0000 7fa84c135740 1 Processor -- start 2024-09-21T05:57:56.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.682+0000 7fa84c135740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558df35249f0 0x558df3929f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:56.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.682+0000 7fa84c135740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df353d1b0 0x558df39253b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:56.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.682+0000 7fa84c135740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558df3545390 0x558df39258f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:56.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.682+0000 7fa83f7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558df35249f0 0x558df3929f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:56.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.682+0000 7fa83f7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558df35249f0 0x558df3929f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:33540/0 (socket says 172.21.15.117:33540) 2024-09-21T05:57:56.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.682+0000 7fa83ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558df3545390 0x558df39258f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:56.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.682+0000 7fa83ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558df3545390 0x558df39258f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41456/0 (socket says 172.21.15.117:41456) 2024-09-21T05:57:56.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.682+0000 7fa83effd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df353d1b0 0x558df39253b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:56.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.682+0000 7fa83f7fe640 1 -- 172.21.15.117:0/3576661574 learned_addr learned my addr 172.21.15.117:0/3576661574 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:56.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.682+0000 7fa84c135740 1 -- 172.21.15.117:0/3576661574 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa8401ae930 con 0x558df3545390 2024-09-21T05:57:56.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.682+0000 7fa84c135740 1 -- 172.21.15.117:0/3576661574 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa8401ae260 con 0x558df353d1b0 2024-09-21T05:57:56.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.682+0000 7fa84c135740 1 -- 172.21.15.117:0/3576661574 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa8401ab4a0 con 0x558df35249f0 2024-09-21T05:57:56.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.683+0000 7fa83ffff640 1 -- 172.21.15.117:0/3576661574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558df35249f0 msgr2=0x558df3929f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:56.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.683+0000 7fa83ffff640 1 --2- 172.21.15.117:0/3576661574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558df35249f0 0x558df3929f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:56.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.683+0000 7fa83ffff640 1 -- 172.21.15.117:0/3576661574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df353d1b0 msgr2=0x558df39253b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:56.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.683+0000 7fa83ffff640 1 --2- 172.21.15.117:0/3576661574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df353d1b0 0x558df39253b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:56.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.683+0000 7fa83ffff640 1 -- 172.21.15.117:0/3576661574 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558df3925eb0 con 0x558df3545390 2024-09-21T05:57:56.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.683+0000 7fa83f7fe640 1 --2- 172.21.15.117:0/3576661574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558df35249f0 0x558df3929f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:57:56.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.683+0000 7fa83ffff640 1 --2- 172.21.15.117:0/3576661574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558df3545390 0x558df39258f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa83801bbb0 tx=0x7fa838009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a51346e294f16af server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:56.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.683+0000 7fa81effd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:57:56.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.683+0000 7fa84c135740 10 client.0 _ll_register_callbacks cb 0x7fa83801c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:57:56.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.683+0000 7fa81effd640 1 -- 172.21.15.117:0/3576661574 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83803c020 con 0x558df3545390 2024-09-21T05:57:56.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.684+0000 7fa81effd640 1 -- 172.21.15.117:0/3576661574 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fa838024980 con 0x558df3545390 2024-09-21T05:57:56.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.684+0000 7fa84c135740 1 -- 172.21.15.117:0/3576661574 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558df387ecd0 con 0x558df3545390 2024-09-21T05:57:56.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.684+0000 7fa81effd640 1 -- 172.21.15.117:0/3576661574 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa838024b80 con 0x558df3545390 2024-09-21T05:57:56.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.684+0000 7fa84c135740 10 client.15726 fetch_fsmap learned FSMap version 27 2024-09-21T05:57:56.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.684+0000 7fa84c135740 1 -- 172.21.15.117:0/3576661574 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({fsmap=27}) -- 0x558df399dd30 con 0x558df3545390 2024-09-21T05:57:56.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.684+0000 7fa81effd640 1 -- 172.21.15.117:0/3576661574 <== mon.0 v2:172.21.15.117:3300/0 4 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fa83801b3c0 con 0x558df3545390 2024-09-21T05:57:56.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.684+0000 7fa84c135740 10 client.15726 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:57:56.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.684+0000 7fa84c135740 10 client.15726 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T05:57:56.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.684+0000 7fa84c135740 20 client.15726 populate_metadata read hostname 'smithi117' 2024-09-21T05:57:56.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.684+0000 7fa84c135740 1 --2- 172.21.15.117:0/3576661574 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x558df3566070 0x558df39a0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:56.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.684+0000 7fa84c135740 4 client.15726 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T05:57:56.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.684+0000 7fa84c135740 1 -- 172.21.15.117:0/3576661574 --> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558df387ecd0 con 0x558df3566070 2024-09-21T05:57:56.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.685+0000 7fa83f7fe640 1 --2- 172.21.15.117:0/3576661574 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x558df3566070 0x558df39a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:56.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.685+0000 7fa83f7fe640 1 --2- 172.21.15.117:0/3576661574 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x558df3566070 0x558df39a0360 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a4c3a7e5bf7baf09 server_cookie=1999d990a0ce72d4 in_seq=0 out_seq=0 2024-09-21T05:57:56.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.685+0000 7fa81effd640 10 client.15726 ms_handle_connect on v2:172.21.15.192:6838/1727623231 2024-09-21T05:57:56.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.685+0000 7fa81effd640 1 -- 172.21.15.117:0/3576661574 <== mds.1 v2:172.21.15.192:6838/1727623231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558df387ecd0 con 0x558df3566070 2024-09-21T05:57:56.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.685+0000 7fa81effd640 10 client.15726 handle_command_reply: tid=0 2024-09-21T05:57:56.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.685+0000 7fa84c135740 1 client.15726 shutdown 2024-09-21T05:57:56.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.686+0000 7fa84c135740 1 -- 172.21.15.117:0/3576661574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558df3545390 msgr2=0x558df39258f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:56.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.686+0000 7fa84c135740 1 --2- 172.21.15.117:0/3576661574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558df3545390 0x558df39258f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa83801bbb0 tx=0x7fa838009f70 comp rx=0 tx=0).stop 2024-09-21T05:57:56.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.686+0000 7fa84c135740 1 -- 172.21.15.117:0/3576661574 shutdown_connections 2024-09-21T05:57:56.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.686+0000 7fa84c135740 1 --2- 172.21.15.117:0/3576661574 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x558df3566070 0x558df39a0360 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:56.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.686+0000 7fa84c135740 1 --2- 172.21.15.117:0/3576661574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558df3545390 0x558df39258f0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:56.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.686+0000 7fa84c135740 1 --2- 172.21.15.117:0/3576661574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558df353d1b0 0x558df39253b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:56.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.686+0000 7fa84c135740 1 --2- 172.21.15.117:0/3576661574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558df35249f0 0x558df3929f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:56.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.686+0000 7fa84c135740 1 -- 172.21.15.117:0/3576661574 >> 172.21.15.117:0/3576661574 conn(0x558df355a460 msgr2=0x558df391b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:56.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.686+0000 7fa81effd640 0 client.15726 ms_handle_reset on v2:172.21.15.192:6838/1727623231 2024-09-21T05:57:56.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.686+0000 7fa84c135740 1 -- 172.21.15.117:0/3576661574 shutdown_connections 2024-09-21T05:57:56.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.686+0000 7fa84c135740 1 -- 172.21.15.117:0/3576661574 wait complete. 2024-09-21T05:57:56.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.686+0000 7fa84c135740 20 client.15726 trim_cache size 0 max 16384 2024-09-21T05:57:56.690 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T05:57:56.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.687+0000 7fa845bc1640 1 -- 172.21.15.117:0/2110329053 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa8280b6670 msgr2=0x7fa8280b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:56.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.687+0000 7fa845bc1640 1 --2- 172.21.15.117:0/2110329053 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa8280b6670 0x7fa8280b8b80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa838005fd0 tx=0x7fa838004410 comp rx=0 tx=0).stop 2024-09-21T05:57:56.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.687+0000 7fa845bc1640 1 -- 172.21.15.117:0/2110329053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8401094c0 msgr2=0x7fa8401988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:56.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.687+0000 7fa845bc1640 1 --2- 172.21.15.117:0/2110329053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8401094c0 0x7fa8401988c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa8340058c0 tx=0x7fa83400ee50 comp rx=0 tx=0).stop 2024-09-21T05:57:56.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.687+0000 7fa845bc1640 1 -- 172.21.15.117:0/2110329053 shutdown_connections 2024-09-21T05:57:56.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.687+0000 7fa845bc1640 1 --2- 172.21.15.117:0/2110329053 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa8280b6670 0x7fa8280b8b80 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:56.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.687+0000 7fa845bc1640 1 --2- 172.21.15.117:0/2110329053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa840109e10 0x7fa84019db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:56.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.687+0000 7fa845bc1640 1 --2- 172.21.15.117:0/2110329053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8401094c0 0x7fa8401988c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:56.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.687+0000 7fa845bc1640 1 --2- 172.21.15.117:0/2110329053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa840102010 0x7fa840198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:56.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.687+0000 7fa845bc1640 1 -- 172.21.15.117:0/2110329053 >> 172.21.15.117:0/2110329053 conn(0x7fa840077c60 msgr2=0x7fa8401075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:56.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.687+0000 7fa845bc1640 1 -- 172.21.15.117:0/2110329053 shutdown_connections 2024-09-21T05:57:56.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:56.687+0000 7fa845bc1640 1 -- 172.21.15.117:0/2110329053 wait complete. 2024-09-21T05:57:57.536 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T05:57:57.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:57 smithi117 ceph-mon[30947]: pgmap v346: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-21T05:57:57.931 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:57:57.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:57 smithi192 ceph-mon[34700]: pgmap v346: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-21T05:57:58.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:57 smithi138 ceph-mon[34391]: pgmap v346: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-21T05:57:58.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.504+0000 7f655f47d640 1 -- 172.21.15.117:0/118854811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6558108d00 msgr2=0x7f655810f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:58.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.504+0000 7f655f47d640 1 --2- 172.21.15.117:0/118854811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6558108d00 0x7f655810f530 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6550009ca0 tx=0x7f655002a8c0 comp rx=0 tx=0).stop 2024-09-21T05:57:58.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.504+0000 7f655f47d640 1 -- 172.21.15.117:0/118854811 shutdown_connections 2024-09-21T05:57:58.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.504+0000 7f655f47d640 1 --2- 172.21.15.117:0/118854811 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6558108d00 0x7f655810f530 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:58.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.504+0000 7f655f47d640 1 --2- 172.21.15.117:0/118854811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65581083b0 0x7f65581087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:58.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.504+0000 7f655f47d640 1 --2- 172.21.15.117:0/118854811 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6558102ac0 0x7f6558102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:58.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.504+0000 7f655f47d640 1 -- 172.21.15.117:0/118854811 >> 172.21.15.117:0/118854811 conn(0x7f6558079790 msgr2=0x7f6558079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:58.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.504+0000 7f655f47d640 1 -- 172.21.15.117:0/118854811 shutdown_connections 2024-09-21T05:57:58.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.504+0000 7f655f47d640 1 -- 172.21.15.117:0/118854811 wait complete. 2024-09-21T05:57:58.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.506+0000 7f655f47d640 1 Processor -- start 2024-09-21T05:57:58.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.506+0000 7f655f47d640 1 -- start start 2024-09-21T05:57:58.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.507+0000 7f655f47d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6558102ac0 0x7f6558198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:58.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.507+0000 7f655f47d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65581083b0 0x7f65581988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:58.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.507+0000 7f655ca18640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65581083b0 0x7f65581988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:58.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.507+0000 7f655ca18640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65581083b0 0x7f65581988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41478/0 (socket says 172.21.15.117:41478) 2024-09-21T05:57:58.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.507+0000 7f655d219640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6558102ac0 0x7f6558198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:58.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.507+0000 7f655d219640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6558102ac0 0x7f6558198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:33570/0 (socket says 172.21.15.117:33570) 2024-09-21T05:57:58.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.507+0000 7f655f47d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6558108d00 0x7f655819db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:58.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.507+0000 7f655ca18640 1 -- 172.21.15.117:0/4256437512 learned_addr learned my addr 172.21.15.117:0/4256437512 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:58.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.507+0000 7f655f47d640 1 -- 172.21.15.117:0/4256437512 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6558111900 con 0x7f65581083b0 2024-09-21T05:57:58.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.507+0000 7f655f47d640 1 -- 172.21.15.117:0/4256437512 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6558111780 con 0x7f6558108d00 2024-09-21T05:57:58.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.507+0000 7f655f47d640 1 -- 172.21.15.117:0/4256437512 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6558111a80 con 0x7f6558102ac0 2024-09-21T05:57:58.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.507+0000 7f655d219640 1 -- 172.21.15.117:0/4256437512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6558108d00 msgr2=0x7f655819db90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T05:57:58.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.507+0000 7f655d219640 1 --2- 172.21.15.117:0/4256437512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6558108d00 0x7f655819db90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:58.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.507+0000 7f655d219640 1 -- 172.21.15.117:0/4256437512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65581083b0 msgr2=0x7f65581988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:58.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.507+0000 7f655d219640 1 --2- 172.21.15.117:0/4256437512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65581083b0 0x7f65581988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:58.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.507+0000 7f655d219640 1 -- 172.21.15.117:0/4256437512 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f655819e2c0 con 0x7f6558102ac0 2024-09-21T05:57:58.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.508+0000 7f655d219640 1 --2- 172.21.15.117:0/4256437512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6558102ac0 0x7f6558198350 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6554017b40 tx=0x7f655400d520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:58.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.508+0000 7f654e7fc640 1 -- 172.21.15.117:0/4256437512 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6554005220 con 0x7f6558102ac0 2024-09-21T05:57:58.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.508+0000 7f654e7fc640 1 -- 172.21.15.117:0/4256437512 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6554004c50 con 0x7f6558102ac0 2024-09-21T05:57:58.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.508+0000 7f654e7fc640 1 -- 172.21.15.117:0/4256437512 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655401e8f0 con 0x7f6558102ac0 2024-09-21T05:57:58.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.508+0000 7f655f47d640 1 -- 172.21.15.117:0/4256437512 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f655819e5b0 con 0x7f6558102ac0 2024-09-21T05:57:58.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.508+0000 7f655f47d640 1 -- 172.21.15.117:0/4256437512 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65581a4d10 con 0x7f6558102ac0 2024-09-21T05:57:58.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.512+0000 7f65659f1740 1 Processor -- start 2024-09-21T05:57:58.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.512+0000 7f65659f1740 1 -- start start 2024-09-21T05:57:58.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.513+0000 7f654e7fc640 1 -- 172.21.15.117:0/4256437512 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 23) ==== 154587+0+0 (secure 0 0 0) 0x7f655400c030 con 0x7f6558102ac0 2024-09-21T05:57:58.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.513+0000 7f65659f1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb308c070 0x555bb34332c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:58.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.513+0000 7f65659f1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb30631b0 0x555bb3433800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:58.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.513+0000 7f65659f1740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bb306b390 0x555bb3433d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:58.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.513+0000 7f655d219640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb30631b0 0x555bb3433800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:58.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.513+0000 7f655da1a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb308c070 0x555bb34332c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:58.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.513+0000 7f655da1a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb308c070 0x555bb34332c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:54778/0 (socket says 172.21.15.117:54778) 2024-09-21T05:57:58.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.513+0000 7f655d219640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb30631b0 0x555bb3433800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41490/0 (socket says 172.21.15.117:41490) 2024-09-21T05:57:58.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.514+0000 7f655ca18640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bb306b390 0x555bb3433d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:58.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.514+0000 7f655ca18640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bb306b390 0x555bb3433d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:33572/0 (socket says 172.21.15.117:33572) 2024-09-21T05:57:58.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.514+0000 7f65659f1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555bb33a9ea0 con 0x555bb30631b0 2024-09-21T05:57:58.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.514+0000 7f65659f1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555bb33992d0 con 0x555bb308c070 2024-09-21T05:57:58.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.514+0000 7f65659f1740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555bb339d0e0 con 0x555bb306b390 2024-09-21T05:57:58.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.514+0000 7f655da1a640 1 -- 172.21.15.117:0/4021533425 learned_addr learned my addr 172.21.15.117:0/4021533425 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:58.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.514+0000 7f654e7fc640 1 --2- 172.21.15.117:0/4256437512 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f65440b66f0 0x7f65440b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:58.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.515+0000 7f655d219640 1 -- 172.21.15.117:0/4021533425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bb306b390 msgr2=0x555bb3433d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:58.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.515+0000 7f655d219640 1 --2- 172.21.15.117:0/4021533425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bb306b390 0x555bb3433d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:58.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.515+0000 7f655d219640 1 -- 172.21.15.117:0/4021533425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb308c070 msgr2=0x555bb34332c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:58.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.515+0000 7f655d219640 1 --2- 172.21.15.117:0/4021533425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb308c070 0x555bb34332c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:58.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.515+0000 7f655d219640 1 -- 172.21.15.117:0/4021533425 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bb30f7650 con 0x555bb30631b0 2024-09-21T05:57:58.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.515+0000 7f655da1a640 1 --2- 172.21.15.117:0/4256437512 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f65440b66f0 0x7f65440b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:58.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.515+0000 7f655d219640 1 --2- 172.21.15.117:0/4021533425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb30631b0 0x555bb3433800 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f65540ed1d0 tx=0x7f65540f5d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b68a2ffd7a836035 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:58.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.516+0000 7f65437fe640 1 -- 172.21.15.117:0/4021533425 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65540fe070 con 0x555bb30631b0 2024-09-21T05:57:58.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.516+0000 7f655da1a640 1 --2- 172.21.15.117:0/4256437512 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f65440b66f0 0x7f65440b8c00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f65500314a0 tx=0x7f65500094a0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:58.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.516+0000 7f65437fe640 1 -- 172.21.15.117:0/4021533425 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65540fa420 con 0x555bb30631b0 2024-09-21T05:57:58.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.516+0000 7f65437fe640 1 -- 172.21.15.117:0/4021533425 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65540f1330 con 0x555bb30631b0 2024-09-21T05:57:58.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.516+0000 7f654e7fc640 1 -- 172.21.15.117:0/4256437512 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f65540ef950 con 0x7f6558102ac0 2024-09-21T05:57:58.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.517+0000 7f65659f1740 1 -- 172.21.15.117:0/4021533425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb30631b0 msgr2=0x555bb3433800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:58.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.517+0000 7f65659f1740 1 --2- 172.21.15.117:0/4021533425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb30631b0 0x555bb3433800 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f65540ed1d0 tx=0x7f65540f5d80 comp rx=0 tx=0).stop 2024-09-21T05:57:58.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.517+0000 7f65659f1740 1 -- 172.21.15.117:0/4021533425 shutdown_connections 2024-09-21T05:57:58.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.517+0000 7f65659f1740 1 --2- 172.21.15.117:0/4021533425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb30631b0 0x555bb3433800 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:58.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.517+0000 7f65659f1740 1 --2- 172.21.15.117:0/4021533425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bb306b390 0x555bb3433d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:58.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.517+0000 7f65659f1740 1 --2- 172.21.15.117:0/4021533425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb308c070 0x555bb34332c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:58.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.517+0000 7f65659f1740 1 -- 172.21.15.117:0/4021533425 >> 172.21.15.117:0/4021533425 conn(0x555bb3080460 msgr2=0x555bb34311b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:58.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.517+0000 7f65659f1740 1 -- 172.21.15.117:0/4021533425 shutdown_connections 2024-09-21T05:57:58.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.517+0000 7f65659f1740 1 -- 172.21.15.117:0/4021533425 wait complete. 2024-09-21T05:57:58.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.518+0000 7f65659f1740 1 -- start start 2024-09-21T05:57:58.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.518+0000 7f65659f1740 1 Processor -- start 2024-09-21T05:57:58.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.519+0000 7f65659f1740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bb30631b0 0x555bb343eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:58.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.519+0000 7f65659f1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb306b390 0x555bb343f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:58.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.519+0000 7f65659f1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb308c070 0x555bb344ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:58.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.519+0000 7f65659f1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555bb31fa700 con 0x555bb308c070 2024-09-21T05:57:58.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.519+0000 7f65659f1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555bb3334160 con 0x555bb306b390 2024-09-21T05:57:58.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.519+0000 7f65659f1740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555bb3441aa0 con 0x555bb30631b0 2024-09-21T05:57:58.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.519+0000 7f65659f1740 10 client.0 _ll_register_callbacks cb 0x555bb334d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:57:58.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.519+0000 7f655ca18640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb306b390 0x555bb343f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:58.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.519+0000 7f655d219640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bb30631b0 0x555bb343eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:58.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.519+0000 7f655ca18640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb306b390 0x555bb343f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:54786/0 (socket says 172.21.15.117:54786) 2024-09-21T05:57:58.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.519+0000 7f655ca18640 1 -- 172.21.15.117:0/580632160 learned_addr learned my addr 172.21.15.117:0/580632160 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:58.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.519+0000 7f655da1a640 1 --2- 172.21.15.117:0/580632160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb308c070 0x555bb344ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:58.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.520+0000 7f655d219640 1 -- 172.21.15.117:0/580632160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb306b390 msgr2=0x555bb343f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:58.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.520+0000 7f655d219640 1 --2- 172.21.15.117:0/580632160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb306b390 0x555bb343f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:58.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.520+0000 7f655d219640 1 -- 172.21.15.117:0/580632160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb308c070 msgr2=0x555bb344ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:58.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.520+0000 7f655d219640 1 --2- 172.21.15.117:0/580632160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb308c070 0x555bb344ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:58.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.520+0000 7f655d219640 1 -- 172.21.15.117:0/580632160 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bb3361c90 con 0x555bb30631b0 2024-09-21T05:57:58.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.520+0000 7f655da1a640 1 --2- 172.21.15.117:0/580632160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb308c070 0x555bb344ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:57:58.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.520+0000 7f655d219640 1 --2- 172.21.15.117:0/580632160 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bb30631b0 0x555bb343eac0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6554017b10 tx=0x7f65540ed680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e70ea8b1ca9356f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:58.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.520+0000 7f6540ff9640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-21T05:57:58.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.521+0000 7f6540ff9640 1 -- 172.21.15.117:0/580632160 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65540fe020 con 0x555bb30631b0 2024-09-21T05:57:58.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.521+0000 7f6540ff9640 1 -- 172.21.15.117:0/580632160 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f65540bf010 con 0x555bb30631b0 2024-09-21T05:57:58.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.521+0000 7f65659f1740 1 -- 172.21.15.117:0/580632160 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65540fa420 con 0x555bb30631b0 2024-09-21T05:57:58.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.521+0000 7f6540ff9640 1 -- 172.21.15.117:0/580632160 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f65540b1ed0 con 0x555bb30631b0 2024-09-21T05:57:58.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.521+0000 7f65659f1740 10 client.25255 fetch_fsmap learned FSMap version 27 2024-09-21T05:57:58.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.521+0000 7f65659f1740 1 -- 172.21.15.117:0/580632160 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555bb344d8a0 con 0x555bb30631b0 2024-09-21T05:57:58.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.521+0000 7f6540ff9640 1 -- 172.21.15.117:0/580632160 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6554004e00 con 0x555bb30631b0 2024-09-21T05:57:58.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.522+0000 7f6540ff9640 1 -- 172.21.15.117:0/580632160 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f655401e070 con 0x555bb30631b0 2024-09-21T05:57:58.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.522+0000 7f65659f1740 10 client.25255 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:57:58.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.522+0000 7f65659f1740 10 client.25255 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T05:57:58.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.522+0000 7f65659f1740 20 client.25255 populate_metadata read hostname 'smithi117' 2024-09-21T05:57:58.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.522+0000 7f65659f1740 1 --2- 172.21.15.117:0/580632160 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555bb304a9f0 0x555bb3450c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:58.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.522+0000 7f65659f1740 4 client.25255 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:57:58.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.522+0000 7f65659f1740 1 -- 172.21.15.117:0/580632160 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f65540fa420 con 0x555bb304a9f0 2024-09-21T05:57:58.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.522+0000 7f655ca18640 1 --2- 172.21.15.117:0/580632160 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555bb304a9f0 0x555bb3450c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:58.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.522+0000 7f655ca18640 1 --2- 172.21.15.117:0/580632160 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555bb304a9f0 0x555bb3450c00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f29d0ccc7ef6239c server_cookie=bdc353a04e441dbc in_seq=0 out_seq=0 2024-09-21T05:57:58.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.523+0000 7f6540ff9640 10 client.25255 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:57:58.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.523+0000 7f6540ff9640 1 -- 172.21.15.117:0/580632160 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f65540fa420 con 0x555bb304a9f0 2024-09-21T05:57:58.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.523+0000 7f6540ff9640 10 client.25255 handle_command_reply: tid=0 2024-09-21T05:57:58.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.523+0000 7f65659f1740 1 client.25255 shutdown 2024-09-21T05:57:58.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.523+0000 7f65659f1740 1 -- 172.21.15.117:0/580632160 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bb30631b0 msgr2=0x555bb343eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:58.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.523+0000 7f65659f1740 1 --2- 172.21.15.117:0/580632160 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bb30631b0 0x555bb343eac0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6554017b10 tx=0x7f65540ed680 comp rx=0 tx=0).stop 2024-09-21T05:57:58.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.524+0000 7f65659f1740 1 -- 172.21.15.117:0/580632160 shutdown_connections 2024-09-21T05:57:58.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.524+0000 7f65659f1740 1 --2- 172.21.15.117:0/580632160 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555bb304a9f0 0x555bb3450c00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:58.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.524+0000 7f6540ff9640 0 client.25255 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:57:58.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.524+0000 7f65659f1740 1 --2- 172.21.15.117:0/580632160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb308c070 0x555bb344ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:58.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.524+0000 7f65659f1740 1 --2- 172.21.15.117:0/580632160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb306b390 0x555bb343f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:58.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.524+0000 7f65659f1740 1 --2- 172.21.15.117:0/580632160 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bb30631b0 0x555bb343eac0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:58.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.524+0000 7f65659f1740 1 -- 172.21.15.117:0/580632160 >> 172.21.15.117:0/580632160 conn(0x555bb3080460 msgr2=0x555bb34364a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:58.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.524+0000 7f65659f1740 1 -- 172.21.15.117:0/580632160 shutdown_connections 2024-09-21T05:57:58.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.524+0000 7f65659f1740 1 -- 172.21.15.117:0/580632160 wait complete. 2024-09-21T05:57:58.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.524+0000 7f65659f1740 20 client.25255 trim_cache size 0 max 16384 2024-09-21T05:57:58.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.550+0000 7f65659f1740 1 Processor -- start 2024-09-21T05:57:58.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.550+0000 7f65659f1740 1 -- start start 2024-09-21T05:57:58.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.550+0000 7f65659f1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb304a9f0 0x555bb3436ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:58.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.550+0000 7f65659f1740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bb30631b0 0x555bb343bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:58.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.550+0000 7f65659f1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb306b390 0x555bb343c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:58.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.551+0000 7f655d219640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb304a9f0 0x555bb3436ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:58.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.551+0000 7f655ca18640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bb30631b0 0x555bb343bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:58.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.551+0000 7f655ca18640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bb30631b0 0x555bb343bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:33598/0 (socket says 172.21.15.117:33598) 2024-09-21T05:57:58.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.551+0000 7f655d219640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb304a9f0 0x555bb3436ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41522/0 (socket says 172.21.15.117:41522) 2024-09-21T05:57:58.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.551+0000 7f655ca18640 1 -- 172.21.15.117:0/1669764887 learned_addr learned my addr 172.21.15.117:0/1669764887 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:58.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.551+0000 7f65659f1740 1 -- 172.21.15.117:0/1669764887 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f65581ad3c0 con 0x555bb304a9f0 2024-09-21T05:57:58.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.551+0000 7f65659f1740 1 -- 172.21.15.117:0/1669764887 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f65581aef00 con 0x555bb306b390 2024-09-21T05:57:58.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.551+0000 7f65659f1740 1 -- 172.21.15.117:0/1669764887 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f65581ada90 con 0x555bb30631b0 2024-09-21T05:57:58.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.551+0000 7f655da1a640 1 --2- 172.21.15.117:0/1669764887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb306b390 0x555bb343c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:58.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.551+0000 7f655d219640 1 -- 172.21.15.117:0/1669764887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bb30631b0 msgr2=0x555bb343bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:58.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.551+0000 7f655d219640 1 --2- 172.21.15.117:0/1669764887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bb30631b0 0x555bb343bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:58.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.551+0000 7f655d219640 1 -- 172.21.15.117:0/1669764887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb306b390 msgr2=0x555bb343c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:58.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.551+0000 7f655d219640 1 --2- 172.21.15.117:0/1669764887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb306b390 0x555bb343c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:58.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.551+0000 7f655d219640 1 -- 172.21.15.117:0/1669764887 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bb34355b0 con 0x555bb304a9f0 2024-09-21T05:57:58.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.551+0000 7f655ca18640 1 --2- 172.21.15.117:0/1669764887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bb30631b0 0x555bb343bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:57:58.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.552+0000 7f655d219640 1 --2- 172.21.15.117:0/1669764887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb304a9f0 0x555bb3436ad0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f65540b2af0 tx=0x7f6554004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea01397744c64390 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:58.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.552+0000 7f6542ffd640 1 -- 172.21.15.117:0/1669764887 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65540b4ff0 con 0x555bb304a9f0 2024-09-21T05:57:58.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.552+0000 7f6542ffd640 1 -- 172.21.15.117:0/1669764887 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6554004e00 con 0x555bb304a9f0 2024-09-21T05:57:58.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.552+0000 7f6542ffd640 1 -- 172.21.15.117:0/1669764887 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65540b51d0 con 0x555bb304a9f0 2024-09-21T05:57:58.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.553+0000 7f65659f1740 1 -- 172.21.15.117:0/1669764887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb304a9f0 msgr2=0x555bb3436ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:58.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.553+0000 7f65659f1740 1 --2- 172.21.15.117:0/1669764887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb304a9f0 0x555bb3436ad0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f65540b2af0 tx=0x7f6554004040 comp rx=0 tx=0).stop 2024-09-21T05:57:58.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.554+0000 7f65659f1740 1 -- 172.21.15.117:0/1669764887 shutdown_connections 2024-09-21T05:57:58.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.554+0000 7f65659f1740 1 --2- 172.21.15.117:0/1669764887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb306b390 0x555bb343c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:58.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.554+0000 7f65659f1740 1 --2- 172.21.15.117:0/1669764887 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bb30631b0 0x555bb343bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:58.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.554+0000 7f65659f1740 1 --2- 172.21.15.117:0/1669764887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb304a9f0 0x555bb3436ad0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:58.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.554+0000 7f65659f1740 1 -- 172.21.15.117:0/1669764887 >> 172.21.15.117:0/1669764887 conn(0x555bb3080460 msgr2=0x555bb34363c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:58.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.554+0000 7f65659f1740 1 -- 172.21.15.117:0/1669764887 shutdown_connections 2024-09-21T05:57:58.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.554+0000 7f65659f1740 1 -- 172.21.15.117:0/1669764887 wait complete. 2024-09-21T05:57:58.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.554+0000 7f65659f1740 1 -- start start 2024-09-21T05:57:58.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.554+0000 7f65659f1740 1 Processor -- start 2024-09-21T05:57:58.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.555+0000 7f65659f1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb304a9f0 0x555bb3454550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:58.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.555+0000 7f65659f1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb30631b0 0x555bb3451a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:58.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.555+0000 7f65659f1740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bb306b390 0x555bb34520c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:58.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.555+0000 7f65659f1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f65581ae830 con 0x555bb304a9f0 2024-09-21T05:57:58.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.555+0000 7f65659f1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f65581ae160 con 0x555bb30631b0 2024-09-21T05:57:58.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.555+0000 7f65659f1740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f65581ab3a0 con 0x555bb306b390 2024-09-21T05:57:58.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.555+0000 7f655d219640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb304a9f0 0x555bb3454550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:58.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.555+0000 7f655d219640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb304a9f0 0x555bb3454550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41528/0 (socket says 172.21.15.117:41528) 2024-09-21T05:57:58.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.555+0000 7f655d219640 1 -- 172.21.15.117:0/257547088 learned_addr learned my addr 172.21.15.117:0/257547088 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:57:58.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.555+0000 7f65659f1740 10 client.0 _ll_register_callbacks cb 0x7f6548005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:57:58.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.555+0000 7f655ca18640 1 --2- 172.21.15.117:0/257547088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb30631b0 0x555bb3451a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:58.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.555+0000 7f655da1a640 1 --2- 172.21.15.117:0/257547088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bb306b390 0x555bb34520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:58.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.556+0000 7f655d219640 1 -- 172.21.15.117:0/257547088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bb306b390 msgr2=0x555bb34520c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:58.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.556+0000 7f655d219640 1 --2- 172.21.15.117:0/257547088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bb306b390 0x555bb34520c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:58.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.556+0000 7f655d219640 1 -- 172.21.15.117:0/257547088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb30631b0 msgr2=0x555bb3451a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:57:58.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.556+0000 7f655d219640 1 --2- 172.21.15.117:0/257547088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb30631b0 0x555bb3451a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:58.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.556+0000 7f655d219640 1 -- 172.21.15.117:0/257547088 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bb3452830 con 0x555bb304a9f0 2024-09-21T05:57:58.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.556+0000 7f655d219640 1 --2- 172.21.15.117:0/257547088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb304a9f0 0x555bb3454550 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f65540b5580 tx=0x7f65540ed680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=610baddd075a466 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:57:58.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.556+0000 7f6540ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:57:58.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.556+0000 7f6540ff9640 1 -- 172.21.15.117:0/257547088 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65540b5650 con 0x555bb304a9f0 2024-09-21T05:57:58.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.556+0000 7f6540ff9640 1 -- 172.21.15.117:0/257547088 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f65540bf020 con 0x555bb304a9f0 2024-09-21T05:57:58.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.556+0000 7f65659f1740 1 -- 172.21.15.117:0/257547088 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555bb33abdf0 con 0x555bb304a9f0 2024-09-21T05:57:58.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.557+0000 7f6540ff9640 1 -- 172.21.15.117:0/257547088 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f65540b1720 con 0x555bb304a9f0 2024-09-21T05:57:58.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.557+0000 7f65659f1740 10 client.15744 fetch_fsmap learned FSMap version 27 2024-09-21T05:57:58.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.557+0000 7f65659f1740 1 -- 172.21.15.117:0/257547088 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555bb3452830 con 0x555bb304a9f0 2024-09-21T05:57:58.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.557+0000 7f6540ff9640 1 -- 172.21.15.117:0/257547088 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555bb33abdf0 con 0x555bb304a9f0 2024-09-21T05:57:58.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.558+0000 7f6540ff9640 1 -- 172.21.15.117:0/257547088 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f65540b6bb0 con 0x555bb304a9f0 2024-09-21T05:57:58.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.558+0000 7f65659f1740 10 client.15744 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:57:58.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.558+0000 7f65659f1740 10 client.15744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T05:57:58.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.558+0000 7f65659f1740 20 client.15744 populate_metadata read hostname 'smithi117' 2024-09-21T05:57:58.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.558+0000 7f65659f1740 1 --2- 172.21.15.117:0/257547088 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555bb308c070 0x555bb34bd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:57:58.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.558+0000 7f65659f1740 4 client.15744 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T05:57:58.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.558+0000 7f65659f1740 1 -- 172.21.15.117:0/257547088 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555bb33a78a0 con 0x555bb308c070 2024-09-21T05:57:58.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.558+0000 7f655ca18640 1 --2- 172.21.15.117:0/257547088 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555bb308c070 0x555bb34bd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:57:58.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.559+0000 7f6540ff9640 10 client.15744 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:57:58.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.559+0000 7f655ca18640 1 --2- 172.21.15.117:0/257547088 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555bb308c070 0x555bb34bd6c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25d3bee2c9e3bf12 server_cookie=e0beb8a78fa2cf96 in_seq=0 out_seq=0 2024-09-21T05:57:58.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.559+0000 7f6540ff9640 1 -- 172.21.15.117:0/257547088 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555bb33a78a0 con 0x555bb308c070 2024-09-21T05:57:58.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.559+0000 7f6540ff9640 10 client.15744 handle_command_reply: tid=0 2024-09-21T05:57:58.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.559+0000 7f65659f1740 1 client.15744 shutdown 2024-09-21T05:57:58.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.559+0000 7f65659f1740 1 -- 172.21.15.117:0/257547088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb304a9f0 msgr2=0x555bb3454550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:58.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.559+0000 7f65659f1740 1 --2- 172.21.15.117:0/257547088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb304a9f0 0x555bb3454550 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f65540b5580 tx=0x7f65540ed680 comp rx=0 tx=0).stop 2024-09-21T05:57:58.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.559+0000 7f65659f1740 1 -- 172.21.15.117:0/257547088 shutdown_connections 2024-09-21T05:57:58.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.559+0000 7f65659f1740 1 --2- 172.21.15.117:0/257547088 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555bb308c070 0x555bb34bd6c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:58.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.560+0000 7f65659f1740 1 --2- 172.21.15.117:0/257547088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bb306b390 0x555bb34520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:58.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.560+0000 7f65659f1740 1 --2- 172.21.15.117:0/257547088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555bb30631b0 0x555bb3451a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:58.569 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T05:57:58.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.560+0000 7f6540ff9640 0 client.15744 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:57:58.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.560+0000 7f65659f1740 1 --2- 172.21.15.117:0/257547088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555bb304a9f0 0x555bb3454550 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:58.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.560+0000 7f65659f1740 1 -- 172.21.15.117:0/257547088 >> 172.21.15.117:0/257547088 conn(0x555bb3080460 msgr2=0x555bb3434210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:58.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.560+0000 7f65659f1740 1 -- 172.21.15.117:0/257547088 shutdown_connections 2024-09-21T05:57:58.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.560+0000 7f65659f1740 1 -- 172.21.15.117:0/257547088 wait complete. 2024-09-21T05:57:58.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.560+0000 7f65659f1740 20 client.15744 trim_cache size 0 max 16384 2024-09-21T05:57:58.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.561+0000 7f655f47d640 1 -- 172.21.15.117:0/4256437512 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f65440b66f0 msgr2=0x7f65440b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:58.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.561+0000 7f655f47d640 1 --2- 172.21.15.117:0/4256437512 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f65440b66f0 0x7f65440b8c00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f65500314a0 tx=0x7f65500094a0 comp rx=0 tx=0).stop 2024-09-21T05:57:58.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.561+0000 7f655f47d640 1 -- 172.21.15.117:0/4256437512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6558102ac0 msgr2=0x7f6558198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:57:58.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.561+0000 7f655f47d640 1 --2- 172.21.15.117:0/4256437512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6558102ac0 0x7f6558198350 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6554017b40 tx=0x7f655400d520 comp rx=0 tx=0).stop 2024-09-21T05:57:58.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.561+0000 7f655f47d640 1 -- 172.21.15.117:0/4256437512 shutdown_connections 2024-09-21T05:57:58.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.561+0000 7f655f47d640 1 --2- 172.21.15.117:0/4256437512 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f65440b66f0 0x7f65440b8c00 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:58.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.561+0000 7f655f47d640 1 --2- 172.21.15.117:0/4256437512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65581083b0 0x7f65581988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:58.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.561+0000 7f655f47d640 1 --2- 172.21.15.117:0/4256437512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6558108d00 0x7f655819db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:58.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.561+0000 7f655f47d640 1 --2- 172.21.15.117:0/4256437512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6558102ac0 0x7f6558198350 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:57:58.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.561+0000 7f655f47d640 1 -- 172.21.15.117:0/4256437512 >> 172.21.15.117:0/4256437512 conn(0x7f6558079790 msgr2=0x7f6558104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:57:58.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.561+0000 7f655f47d640 1 -- 172.21.15.117:0/4256437512 shutdown_connections 2024-09-21T05:57:58.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:57:58.561+0000 7f655f47d640 1 -- 172.21.15.117:0/4256437512 wait complete. 2024-09-21T05:57:59.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:57:59 smithi192 ceph-mon[34700]: pgmap v347: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-09-21T05:58:00.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:57:59 smithi117 ceph-mon[30947]: pgmap v347: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-09-21T05:58:00.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:57:59 smithi138 ceph-mon[34391]: pgmap v347: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-09-21T05:58:00.292 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T05:58:00.292 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T05:58:00.527 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:58:00 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: 2024-09-21T05:58:00.109+0000 7f95941ac640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:58:00.527 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:58:00 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: 2024-09-21T05:58:00.109+0000 7f95941ac640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:58:00.527 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:58:00 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: 2024-09-21T05:58:00.109+0000 7f95941ac640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:58:00.527 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:58:00 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: 2024-09-21T05:58:00.109+0000 7f95941ac640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:58:00.527 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 05:58:00 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: 2024-09-21T05:58:00.109+0000 7f95941ac640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T05:58:00.605 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:58:01.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.189+0000 7fb524aec640 1 -- 172.21.15.117:0/179974217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb520105ac0 msgr2=0x7fb52010f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:01.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.189+0000 7fb524aec640 1 --2- 172.21.15.117:0/179974217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb520105ac0 0x7fb52010f530 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb518009ca0 tx=0x7fb51802a8c0 comp rx=0 tx=0).stop 2024-09-21T05:58:01.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.189+0000 7fb524aec640 1 -- 172.21.15.117:0/179974217 shutdown_connections 2024-09-21T05:58:01.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.189+0000 7fb524aec640 1 --2- 172.21.15.117:0/179974217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb520105ac0 0x7fb52010f530 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:01.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.189+0000 7fb524aec640 1 --2- 172.21.15.117:0/179974217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb520105170 0x7fb520105580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:01.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.189+0000 7fb524aec640 1 --2- 172.21.15.117:0/179974217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb520103080 0x7fb520103490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:01.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.189+0000 7fb524aec640 1 -- 172.21.15.117:0/179974217 >> 172.21.15.117:0/179974217 conn(0x7fb520077c60 msgr2=0x7fb5200780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:01.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.189+0000 7fb524aec640 1 -- 172.21.15.117:0/179974217 shutdown_connections 2024-09-21T05:58:01.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.189+0000 7fb524aec640 1 -- 172.21.15.117:0/179974217 wait complete. 2024-09-21T05:58:01.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.190+0000 7fb524aec640 1 Processor -- start 2024-09-21T05:58:01.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.190+0000 7fb524aec640 1 -- start start 2024-09-21T05:58:01.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.190+0000 7fb524aec640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb520103080 0x7fb520101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:01.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.190+0000 7fb524aec640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb520105170 0x7fb520102220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:01.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.191+0000 7fb524aec640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb520105ac0 0x7fb520102760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:01.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.191+0000 7fb524aec640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb520112a20 con 0x7fb520105ac0 2024-09-21T05:58:01.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.191+0000 7fb524aec640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5201128a0 con 0x7fb520105170 2024-09-21T05:58:01.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.191+0000 7fb524aec640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb520112ba0 con 0x7fb520103080 2024-09-21T05:58:01.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.191+0000 7fb51dd9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb520105170 0x7fb520102220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:01.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.191+0000 7fb51dd9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb520105170 0x7fb520102220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:54832/0 (socket says 172.21.15.117:54832) 2024-09-21T05:58:01.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.191+0000 7fb51dd9b640 1 -- 172.21.15.117:0/1432437317 learned_addr learned my addr 172.21.15.117:0/1432437317 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:01.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.191+0000 7fb51ed9d640 1 --2- 172.21.15.117:0/1432437317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb520105ac0 0x7fb520102760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:01.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.191+0000 7fb51dd9b640 1 -- 172.21.15.117:0/1432437317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb520103080 msgr2=0x7fb520101ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:01.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.191+0000 7fb51dd9b640 1 --2- 172.21.15.117:0/1432437317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb520103080 0x7fb520101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:01.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.191+0000 7fb51dd9b640 1 -- 172.21.15.117:0/1432437317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb520105ac0 msgr2=0x7fb520102760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:01.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.191+0000 7fb51dd9b640 1 --2- 172.21.15.117:0/1432437317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb520105ac0 0x7fb520102760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:01.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.191+0000 7fb51dd9b640 1 -- 172.21.15.117:0/1432437317 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5201b0f80 con 0x7fb520105170 2024-09-21T05:58:01.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.191+0000 7fb51ed9d640 1 --2- 172.21.15.117:0/1432437317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb520105ac0 0x7fb520102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:58:01.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.191+0000 7fb51dd9b640 1 --2- 172.21.15.117:0/1432437317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb520105170 0x7fb520102220 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb51400e9f0 tx=0x7fb51400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:01.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.192+0000 7fb50f7fe640 1 -- 172.21.15.117:0/1432437317 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51400cc60 con 0x7fb520105170 2024-09-21T05:58:01.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.192+0000 7fb524aec640 1 -- 172.21.15.117:0/1432437317 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5201b1270 con 0x7fb520105170 2024-09-21T05:58:01.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.192+0000 7fb50f7fe640 1 -- 172.21.15.117:0/1432437317 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb51400ce00 con 0x7fb520105170 2024-09-21T05:58:01.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.192+0000 7fb50f7fe640 1 -- 172.21.15.117:0/1432437317 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5140187c0 con 0x7fb520105170 2024-09-21T05:58:01.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.192+0000 7fb524aec640 1 -- 172.21.15.117:0/1432437317 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5201b1830 con 0x7fb520105170 2024-09-21T05:58:01.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.195+0000 7fb52b060740 1 Processor -- start 2024-09-21T05:58:01.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.195+0000 7fb52b060740 1 -- start start 2024-09-21T05:58:01.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.196+0000 7fb52b060740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5b42c350 0x55da5b7d3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:01.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.196+0000 7fb52b060740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da5b403490 0x55da5b7d3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:01.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.196+0000 7fb52b060740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da5b40b670 0x55da5b7d3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:01.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.196+0000 7fb52b060740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55da5b749e60 con 0x55da5b403490 2024-09-21T05:58:01.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.196+0000 7fb52b060740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55da5b739290 con 0x55da5b42c350 2024-09-21T05:58:01.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.196+0000 7fb52b060740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55da5b73d0a0 con 0x55da5b40b670 2024-09-21T05:58:01.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.196+0000 7fb50f7fe640 1 -- 172.21.15.117:0/1432437317 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fb514018960 con 0x7fb520105170 2024-09-21T05:58:01.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.196+0000 7fb51ed9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5b42c350 0x55da5b7d3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:01.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.196+0000 7fb51ed9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5b42c350 0x55da5b7d3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:54848/0 (socket says 172.21.15.117:54848) 2024-09-21T05:58:01.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.196+0000 7fb51ed9d640 1 -- 172.21.15.117:0/1890663240 learned_addr learned my addr 172.21.15.117:0/1890663240 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:01.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.196+0000 7fb51e59c640 1 --2- 172.21.15.117:0/1890663240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da5b403490 0x55da5b7d3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:01.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.196+0000 7fb51ed9d640 1 -- 172.21.15.117:0/1890663240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da5b40b670 msgr2=0x55da5b7d3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:01.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.196+0000 7fb51dd9b640 1 --2- 172.21.15.117:0/1890663240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da5b40b670 0x55da5b7d3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:01.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.196+0000 7fb51ed9d640 1 --2- 172.21.15.117:0/1890663240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da5b40b670 0x55da5b7d3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:01.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.196+0000 7fb51ed9d640 1 -- 172.21.15.117:0/1890663240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da5b403490 msgr2=0x55da5b7d3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:01.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.196+0000 7fb51ed9d640 1 --2- 172.21.15.117:0/1890663240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da5b403490 0x55da5b7d3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:01.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.196+0000 7fb51ed9d640 1 -- 172.21.15.117:0/1890663240 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da5b497930 con 0x55da5b42c350 2024-09-21T05:58:01.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.196+0000 7fb51e59c640 1 --2- 172.21.15.117:0/1890663240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da5b403490 0x55da5b7d3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:58:01.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.197+0000 7fb51ed9d640 1 --2- 172.21.15.117:0/1890663240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5b42c350 0x55da5b7d3430 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb518033040 tx=0x7fb51802f450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a1ee9bd7e8d3e60 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:01.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.197+0000 7fb50cff9640 1 -- 172.21.15.117:0/1890663240 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb518031650 con 0x55da5b42c350 2024-09-21T05:58:01.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.197+0000 7fb50cff9640 1 -- 172.21.15.117:0/1890663240 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb518036040 con 0x55da5b42c350 2024-09-21T05:58:01.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.197+0000 7fb50cff9640 1 -- 172.21.15.117:0/1890663240 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5180045d0 con 0x55da5b42c350 2024-09-21T05:58:01.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.197+0000 7fb50f7fe640 1 --2- 172.21.15.117:0/1432437317 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb5080bef10 0x7fb5080c1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:01.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.198+0000 7fb50f7fe640 1 -- 172.21.15.117:0/1432437317 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fb5140e7ba0 con 0x7fb520105170 2024-09-21T05:58:01.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.198+0000 7fb51e59c640 1 --2- 172.21.15.117:0/1432437317 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb5080bef10 0x7fb5080c1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:01.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.199+0000 7fb52b060740 1 -- 172.21.15.117:0/1890663240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5b42c350 msgr2=0x55da5b7d3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:01.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.199+0000 7fb52b060740 1 --2- 172.21.15.117:0/1890663240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5b42c350 0x55da5b7d3430 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb518033040 tx=0x7fb51802f450 comp rx=0 tx=0).stop 2024-09-21T05:58:01.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.199+0000 7fb52b060740 1 -- 172.21.15.117:0/1890663240 shutdown_connections 2024-09-21T05:58:01.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.199+0000 7fb52b060740 1 --2- 172.21.15.117:0/1890663240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da5b403490 0x55da5b7d3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:01.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.199+0000 7fb52b060740 1 --2- 172.21.15.117:0/1890663240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da5b40b670 0x55da5b7d3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:01.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.199+0000 7fb52b060740 1 --2- 172.21.15.117:0/1890663240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5b42c350 0x55da5b7d3430 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:01.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.199+0000 7fb52b060740 1 -- 172.21.15.117:0/1890663240 >> 172.21.15.117:0/1890663240 conn(0x55da5b420740 msgr2=0x55da5b7d1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:01.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.199+0000 7fb52b060740 1 -- 172.21.15.117:0/1890663240 shutdown_connections 2024-09-21T05:58:01.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.199+0000 7fb52b060740 1 -- 172.21.15.117:0/1890663240 wait complete. 2024-09-21T05:58:01.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.199+0000 7fb51e59c640 1 --2- 172.21.15.117:0/1432437317 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb5080bef10 0x7fb5080c1420 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55da5b7d20b0 tx=0x7fb510008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:01.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.199+0000 7fb52b060740 1 -- start start 2024-09-21T05:58:01.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.200+0000 7fb52b060740 1 Processor -- start 2024-09-21T05:58:01.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.200+0000 7fb52b060740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5b403490 0x55da5b7df010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:01.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.200+0000 7fb52b060740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da5b40b670 0x55da5b7df550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:01.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.200+0000 7fb52b060740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da5b42c350 0x55da5b7ead20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:01.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.200+0000 7fb52b060740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55da5b59a890 con 0x55da5b42c350 2024-09-21T05:58:01.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.200+0000 7fb52b060740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55da5b6d3d40 con 0x55da5b403490 2024-09-21T05:58:01.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.200+0000 7fb52b060740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55da5b7e1a30 con 0x55da5b40b670 2024-09-21T05:58:01.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.200+0000 7fb52b060740 10 client.0 _ll_register_callbacks cb 0x55da5b737c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:58:01.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.200+0000 7fb51ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da5b42c350 0x55da5b7ead20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:01.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.200+0000 7fb51ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da5b42c350 0x55da5b7ead20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41588/0 (socket says 172.21.15.117:41588) 2024-09-21T05:58:01.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.200+0000 7fb51ed9d640 1 -- 172.21.15.117:0/239003744 learned_addr learned my addr 172.21.15.117:0/239003744 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:01.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.200+0000 7fb51dd9b640 1 --2- 172.21.15.117:0/239003744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da5b40b670 0x55da5b7df550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:01.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.200+0000 7fb51e59c640 1 --2- 172.21.15.117:0/239003744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5b403490 0x55da5b7df010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:01.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.201+0000 7fb51e59c640 1 -- 172.21.15.117:0/239003744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da5b40b670 msgr2=0x55da5b7df550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:01.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.201+0000 7fb51e59c640 1 --2- 172.21.15.117:0/239003744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da5b40b670 0x55da5b7df550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:01.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.201+0000 7fb51e59c640 1 -- 172.21.15.117:0/239003744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da5b42c350 msgr2=0x55da5b7ead20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:01.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.201+0000 7fb51e59c640 1 --2- 172.21.15.117:0/239003744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da5b42c350 0x55da5b7ead20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:01.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.201+0000 7fb51e59c640 1 -- 172.21.15.117:0/239003744 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da5b7eb2b0 con 0x55da5b403490 2024-09-21T05:58:01.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.201+0000 7fb51e59c640 1 --2- 172.21.15.117:0/239003744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5b403490 0x55da5b7df010 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb51000a1e0 tx=0x7fb510004840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7f4975756284fad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:01.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.201+0000 7fb505ffb640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-21T05:58:01.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.201+0000 7fb505ffb640 1 -- 172.21.15.117:0/239003744 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510023060 con 0x55da5b403490 2024-09-21T05:58:01.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.201+0000 7fb52b060740 1 -- 172.21.15.117:0/239003744 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb518036040 con 0x55da5b403490 2024-09-21T05:58:01.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.201+0000 7fb505ffb640 1 -- 172.21.15.117:0/239003744 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fb51001e600 con 0x55da5b403490 2024-09-21T05:58:01.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.201+0000 7fb505ffb640 1 -- 172.21.15.117:0/239003744 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb51001e800 con 0x55da5b403490 2024-09-21T05:58:01.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.201+0000 7fb52b060740 10 client.25295 fetch_fsmap learned FSMap version 27 2024-09-21T05:58:01.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.201+0000 7fb52b060740 1 -- 172.21.15.117:0/239003744 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55da5b7edc00 con 0x55da5b403490 2024-09-21T05:58:01.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.202+0000 7fb505ffb640 1 -- 172.21.15.117:0/239003744 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb518036040 con 0x55da5b403490 2024-09-21T05:58:01.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.202+0000 7fb505ffb640 1 -- 172.21.15.117:0/239003744 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb51001d3a0 con 0x55da5b403490 2024-09-21T05:58:01.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.202+0000 7fb52b060740 10 client.25295 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:58:01.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.202+0000 7fb52b060740 10 client.25295 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T05:58:01.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.202+0000 7fb52b060740 20 client.25295 populate_metadata read hostname 'smithi117' 2024-09-21T05:58:01.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.202+0000 7fb52b060740 1 --2- 172.21.15.117:0/239003744 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55da5b3eacd0 0x55da5b7f1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:01.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.202+0000 7fb52b060740 4 client.25295 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:58:01.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.202+0000 7fb52b060740 1 -- 172.21.15.117:0/239003744 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da5b74bdb0 con 0x55da5b3eacd0 2024-09-21T05:58:01.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.202+0000 7fb51dd9b640 1 --2- 172.21.15.117:0/239003744 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55da5b3eacd0 0x55da5b7f1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:01.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.203+0000 7fb51dd9b640 1 --2- 172.21.15.117:0/239003744 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55da5b3eacd0 0x55da5b7f1570 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d14d36cf83616da9 server_cookie=3b966e06b3ca2372 in_seq=0 out_seq=0 2024-09-21T05:58:01.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.203+0000 7fb505ffb640 10 client.25295 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:58:01.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.203+0000 7fb505ffb640 1 -- 172.21.15.117:0/239003744 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55da5b74bdb0 con 0x55da5b3eacd0 2024-09-21T05:58:01.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.203+0000 7fb505ffb640 10 client.25295 handle_command_reply: tid=0 2024-09-21T05:58:01.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.203+0000 7fb52b060740 1 client.25295 shutdown 2024-09-21T05:58:01.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.204+0000 7fb52b060740 1 -- 172.21.15.117:0/239003744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5b403490 msgr2=0x55da5b7df010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:01.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.204+0000 7fb52b060740 1 --2- 172.21.15.117:0/239003744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5b403490 0x55da5b7df010 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb51000a1e0 tx=0x7fb510004840 comp rx=0 tx=0).stop 2024-09-21T05:58:01.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.204+0000 7fb52b060740 1 -- 172.21.15.117:0/239003744 shutdown_connections 2024-09-21T05:58:01.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.204+0000 7fb52b060740 1 --2- 172.21.15.117:0/239003744 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55da5b3eacd0 0x55da5b7f1570 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:01.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.204+0000 7fb52b060740 1 --2- 172.21.15.117:0/239003744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da5b42c350 0x55da5b7ead20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:01.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.204+0000 7fb52b060740 1 --2- 172.21.15.117:0/239003744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da5b40b670 0x55da5b7df550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:01.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.204+0000 7fb52b060740 1 --2- 172.21.15.117:0/239003744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5b403490 0x55da5b7df010 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:01.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.204+0000 7fb505ffb640 0 client.25295 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:58:01.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.204+0000 7fb52b060740 1 -- 172.21.15.117:0/239003744 >> 172.21.15.117:0/239003744 conn(0x55da5b420740 msgr2=0x55da5b7d6610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:01.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.204+0000 7fb52b060740 1 -- 172.21.15.117:0/239003744 shutdown_connections 2024-09-21T05:58:01.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.204+0000 7fb52b060740 1 -- 172.21.15.117:0/239003744 wait complete. 2024-09-21T05:58:01.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.204+0000 7fb52b060740 20 client.25295 trim_cache size 0 max 16384 2024-09-21T05:58:01.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.238+0000 7fb52b060740 1 Processor -- start 2024-09-21T05:58:01.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.238+0000 7fb52b060740 1 -- start start 2024-09-21T05:58:01.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.238+0000 7fb52b060740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da5b3eacd0 0x55da5b7d8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:01.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.238+0000 7fb52b060740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5b403490 0x55da5b7d8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:01.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.238+0000 7fb52b060740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da5b40b670 0x55da5b7d6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:01.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.238+0000 7fb52b060740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb5201b5b40 con 0x55da5b40b670 2024-09-21T05:58:01.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.238+0000 7fb52b060740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5201b7680 con 0x55da5b403490 2024-09-21T05:58:01.240 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.238+0000 7fb52b060740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb5201b6210 con 0x55da5b3eacd0 2024-09-21T05:58:01.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.238+0000 7fb51e59c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da5b3eacd0 0x55da5b7d8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:01.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.238+0000 7fb51ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da5b40b670 0x55da5b7d6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:01.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.238+0000 7fb51e59c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da5b3eacd0 0x55da5b7d8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:33656/0 (socket says 172.21.15.117:33656) 2024-09-21T05:58:01.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.238+0000 7fb51e59c640 1 -- 172.21.15.117:0/2604710043 learned_addr learned my addr 172.21.15.117:0/2604710043 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:01.242 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.239+0000 7fb51e59c640 1 -- 172.21.15.117:0/2604710043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5b403490 msgr2=0x55da5b7d8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:01.242 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.239+0000 7fb51e59c640 1 --2- 172.21.15.117:0/2604710043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5b403490 0x55da5b7d8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:01.242 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.239+0000 7fb51e59c640 1 -- 172.21.15.117:0/2604710043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da5b40b670 msgr2=0x55da5b7d6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:01.242 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.239+0000 7fb51e59c640 1 --2- 172.21.15.117:0/2604710043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da5b40b670 0x55da5b7d6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:01.242 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.239+0000 7fb51e59c640 1 -- 172.21.15.117:0/2604710043 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da5b7d1860 con 0x55da5b3eacd0 2024-09-21T05:58:01.242 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.239+0000 7fb51ed9d640 1 --2- 172.21.15.117:0/2604710043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da5b40b670 0x55da5b7d6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:58:01.242 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.239+0000 7fb51e59c640 1 --2- 172.21.15.117:0/2604710043 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da5b3eacd0 0x55da5b7d8510 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb51001d7f0 tx=0x7fb51000c690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6668573aff0be6a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:01.242 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.239+0000 7fb507fff640 1 -- 172.21.15.117:0/2604710043 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51001d9e0 con 0x55da5b3eacd0 2024-09-21T05:58:01.242 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.239+0000 7fb507fff640 1 -- 172.21.15.117:0/2604710043 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb51001ee50 con 0x55da5b3eacd0 2024-09-21T05:58:01.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.239+0000 7fb507fff640 1 -- 172.21.15.117:0/2604710043 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510031860 con 0x55da5b3eacd0 2024-09-21T05:58:01.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.241+0000 7fb52b060740 1 -- 172.21.15.117:0/2604710043 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da5b3eacd0 msgr2=0x55da5b7d8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:01.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.241+0000 7fb52b060740 1 --2- 172.21.15.117:0/2604710043 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da5b3eacd0 0x55da5b7d8510 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb51001d7f0 tx=0x7fb51000c690 comp rx=0 tx=0).stop 2024-09-21T05:58:01.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.241+0000 7fb52b060740 1 -- 172.21.15.117:0/2604710043 shutdown_connections 2024-09-21T05:58:01.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.241+0000 7fb52b060740 1 --2- 172.21.15.117:0/2604710043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da5b40b670 0x55da5b7d6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:01.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.241+0000 7fb52b060740 1 --2- 172.21.15.117:0/2604710043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5b403490 0x55da5b7d8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:01.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.241+0000 7fb52b060740 1 --2- 172.21.15.117:0/2604710043 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da5b3eacd0 0x55da5b7d8510 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:01.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.241+0000 7fb52b060740 1 -- 172.21.15.117:0/2604710043 >> 172.21.15.117:0/2604710043 conn(0x55da5b420740 msgr2=0x55da5b7dd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:01.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.241+0000 7fb52b060740 1 -- 172.21.15.117:0/2604710043 shutdown_connections 2024-09-21T05:58:01.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.241+0000 7fb52b060740 1 -- 172.21.15.117:0/2604710043 wait complete. 2024-09-21T05:58:01.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.241+0000 7fb52b060740 1 -- start start 2024-09-21T05:58:01.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.241+0000 7fb52b060740 1 Processor -- start 2024-09-21T05:58:01.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.242+0000 7fb52b060740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da5b3eacd0 0x55da5b7e3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:01.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.242+0000 7fb52b060740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da5b403490 0x55da5b7e3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:01.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.242+0000 7fb52b060740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5b40b670 0x55da5b85af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:01.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.242+0000 7fb52b060740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb5201b6fb0 con 0x55da5b403490 2024-09-21T05:58:01.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.242+0000 7fb52b060740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb5201b68e0 con 0x55da5b40b670 2024-09-21T05:58:01.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.242+0000 7fb52b060740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb5201b3b20 con 0x55da5b3eacd0 2024-09-21T05:58:01.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.242+0000 7fb52b060740 10 client.0 _ll_register_callbacks cb 0x7fb51000a4e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:58:01.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.242+0000 7fb51e59c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da5b3eacd0 0x55da5b7e3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:01.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.242+0000 7fb51ed9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5b40b670 0x55da5b85af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:01.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.242+0000 7fb51e59c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da5b3eacd0 0x55da5b7e3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:33662/0 (socket says 172.21.15.117:33662) 2024-09-21T05:58:01.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.242+0000 7fb51e59c640 1 -- 172.21.15.117:0/474188613 learned_addr learned my addr 172.21.15.117:0/474188613 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:01.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.242+0000 7fb51dd9b640 1 --2- 172.21.15.117:0/474188613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da5b403490 0x55da5b7e3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:01.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.243+0000 7fb51e59c640 1 -- 172.21.15.117:0/474188613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5b40b670 msgr2=0x55da5b85af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:01.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.243+0000 7fb51e59c640 1 --2- 172.21.15.117:0/474188613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5b40b670 0x55da5b85af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:01.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.243+0000 7fb51e59c640 1 -- 172.21.15.117:0/474188613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da5b403490 msgr2=0x55da5b7e3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:01.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.243+0000 7fb51e59c640 1 --2- 172.21.15.117:0/474188613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da5b403490 0x55da5b7e3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:01.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.243+0000 7fb51e59c640 1 -- 172.21.15.117:0/474188613 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da5b85b4b0 con 0x55da5b3eacd0 2024-09-21T05:58:01.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.243+0000 7fb51ed9d640 1 --2- 172.21.15.117:0/474188613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5b40b670 0x55da5b85af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:58:01.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.243+0000 7fb51dd9b640 1 --2- 172.21.15.117:0/474188613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da5b403490 0x55da5b7e3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:58:01.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.243+0000 7fb51e59c640 1 --2- 172.21.15.117:0/474188613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da5b3eacd0 0x55da5b7e3320 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb51000cb50 tx=0x7fb51001e700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=325efc61aa9b8fa6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:01.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.243+0000 7fb505ffb640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-21T05:58:01.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.243+0000 7fb505ffb640 1 -- 172.21.15.117:0/474188613 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510040060 con 0x55da5b3eacd0 2024-09-21T05:58:01.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.243+0000 7fb505ffb640 1 -- 172.21.15.117:0/474188613 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fb51001d960 con 0x55da5b3eacd0 2024-09-21T05:58:01.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.243+0000 7fb52b060740 1 -- 172.21.15.117:0/474188613 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb51001ee50 con 0x55da5b3eacd0 2024-09-21T05:58:01.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.244+0000 7fb505ffb640 1 -- 172.21.15.117:0/474188613 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb510023040 con 0x55da5b3eacd0 2024-09-21T05:58:01.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.244+0000 7fb52b060740 10 client.25273 fetch_fsmap learned FSMap version 27 2024-09-21T05:58:01.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.244+0000 7fb52b060740 1 -- 172.21.15.117:0/474188613 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55da5b85c700 con 0x55da5b3eacd0 2024-09-21T05:58:01.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.244+0000 7fb505ffb640 1 -- 172.21.15.117:0/474188613 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb51001ee50 con 0x55da5b3eacd0 2024-09-21T05:58:01.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.244+0000 7fb505ffb640 1 -- 172.21.15.117:0/474188613 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb51003a070 con 0x55da5b3eacd0 2024-09-21T05:58:01.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.244+0000 7fb52b060740 10 client.25273 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:58:01.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.244+0000 7fb52b060740 10 client.25273 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T05:58:01.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.244+0000 7fb52b060740 20 client.25273 populate_metadata read hostname 'smithi117' 2024-09-21T05:58:01.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.245+0000 7fb52b060740 1 --2- 172.21.15.117:0/474188613 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55da5b42c350 0x55da5b85edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:01.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.245+0000 7fb52b060740 4 client.25273 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T05:58:01.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.245+0000 7fb52b060740 1 -- 172.21.15.117:0/474188613 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55da5b747860 con 0x55da5b42c350 2024-09-21T05:58:01.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.245+0000 7fb51dd9b640 1 --2- 172.21.15.117:0/474188613 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55da5b42c350 0x55da5b85edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:01.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.245+0000 7fb505ffb640 10 client.25273 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:58:01.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.245+0000 7fb51dd9b640 1 --2- 172.21.15.117:0/474188613 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55da5b42c350 0x55da5b85edb0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b580047cf94f874 server_cookie=e92ef8cf70e9dbce in_seq=0 out_seq=0 2024-09-21T05:58:01.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.247+0000 7fb505ffb640 1 -- 172.21.15.117:0/474188613 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55da5b747860 con 0x55da5b42c350 2024-09-21T05:58:01.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.247+0000 7fb505ffb640 10 client.25273 handle_command_reply: tid=0 2024-09-21T05:58:01.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.247+0000 7fb52b060740 1 client.25273 shutdown 2024-09-21T05:58:01.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.247+0000 7fb52b060740 1 -- 172.21.15.117:0/474188613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da5b3eacd0 msgr2=0x55da5b7e3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:01.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.247+0000 7fb52b060740 1 --2- 172.21.15.117:0/474188613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da5b3eacd0 0x55da5b7e3320 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb51000cb50 tx=0x7fb51001e700 comp rx=0 tx=0).stop 2024-09-21T05:58:01.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.247+0000 7fb52b060740 1 -- 172.21.15.117:0/474188613 shutdown_connections 2024-09-21T05:58:01.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.247+0000 7fb52b060740 1 --2- 172.21.15.117:0/474188613 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55da5b42c350 0x55da5b85edb0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:01.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.248+0000 7fb52b060740 1 --2- 172.21.15.117:0/474188613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55da5b403490 0x55da5b7e3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:01.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.248+0000 7fb505ffb640 0 client.25273 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:58:01.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.248+0000 7fb52b060740 1 --2- 172.21.15.117:0/474188613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da5b40b670 0x55da5b85af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:01.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.248+0000 7fb52b060740 1 --2- 172.21.15.117:0/474188613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55da5b3eacd0 0x55da5b7e3320 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:01.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.248+0000 7fb52b060740 1 -- 172.21.15.117:0/474188613 >> 172.21.15.117:0/474188613 conn(0x55da5b420740 msgr2=0x55da5b7d6530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:01.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.248+0000 7fb52b060740 1 -- 172.21.15.117:0/474188613 shutdown_connections 2024-09-21T05:58:01.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.248+0000 7fb52b060740 1 -- 172.21.15.117:0/474188613 wait complete. 2024-09-21T05:58:01.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.248+0000 7fb52b060740 20 client.25273 trim_cache size 0 max 16384 2024-09-21T05:58:01.252 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T05:58:01.252 INFO:teuthology.orchestra.run.smithi117.stdout: "return_code": 0, 2024-09-21T05:58:01.252 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_tag": "8b3fe95a-b76b-4e94-afd0-72fe35e2dd92", 2024-09-21T05:58:01.252 INFO:teuthology.orchestra.run.smithi117.stdout: "mode": "asynchronous" 2024-09-21T05:58:01.252 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T05:58:01.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.249+0000 7fb524aec640 1 -- 172.21.15.117:0/1432437317 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb5080bef10 msgr2=0x7fb5080c1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:01.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.249+0000 7fb524aec640 1 --2- 172.21.15.117:0/1432437317 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb5080bef10 0x7fb5080c1420 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55da5b7d20b0 tx=0x7fb510008040 comp rx=0 tx=0).stop 2024-09-21T05:58:01.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.249+0000 7fb524aec640 1 -- 172.21.15.117:0/1432437317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb520105170 msgr2=0x7fb520102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:01.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.249+0000 7fb524aec640 1 --2- 172.21.15.117:0/1432437317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb520105170 0x7fb520102220 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb51400e9f0 tx=0x7fb51400eeb0 comp rx=0 tx=0).stop 2024-09-21T05:58:01.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.249+0000 7fb524aec640 1 -- 172.21.15.117:0/1432437317 shutdown_connections 2024-09-21T05:58:01.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.249+0000 7fb524aec640 1 --2- 172.21.15.117:0/1432437317 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb5080bef10 0x7fb5080c1420 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:01.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.249+0000 7fb524aec640 1 --2- 172.21.15.117:0/1432437317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb520105ac0 0x7fb520102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:01.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.249+0000 7fb524aec640 1 --2- 172.21.15.117:0/1432437317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb520105170 0x7fb520102220 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:01.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.249+0000 7fb524aec640 1 --2- 172.21.15.117:0/1432437317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb520103080 0x7fb520101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:01.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.249+0000 7fb524aec640 1 -- 172.21.15.117:0/1432437317 >> 172.21.15.117:0/1432437317 conn(0x7fb520077c60 msgr2=0x7fb5201063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:01.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.249+0000 7fb524aec640 1 -- 172.21.15.117:0/1432437317 shutdown_connections 2024-09-21T05:58:01.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:01.249+0000 7fb524aec640 1 -- 172.21.15.117:0/1432437317 wait complete. 2024-09-21T05:58:01.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:01 smithi192 ceph-mon[34700]: pgmap v348: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-09-21T05:58:01.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:01 smithi192 ceph-mon[34700]: mgrmap e24: x(active, since 9m), standbys: z, y 2024-09-21T05:58:02.019 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:01 smithi117 ceph-mon[30947]: pgmap v348: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-09-21T05:58:02.019 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:01 smithi117 ceph-mon[30947]: mgrmap e24: x(active, since 9m), standbys: z, y 2024-09-21T05:58:02.058 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T05:58:02.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:01 smithi138 ceph-mon[34391]: pgmap v348: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-09-21T05:58:02.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:01 smithi138 ceph-mon[34391]: mgrmap e24: x(active, since 9m), standbys: z, y 2024-09-21T05:58:02.441 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:58:02.891 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:02 smithi117 ceph-mon[30947]: scrub queued for path: / 2024-09-21T05:58:02.891 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:02 smithi117 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-21T05:58:02.891 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:02 smithi117 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-21T05:58:02.891 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:02 smithi117 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-21T05:58:02.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:02 smithi192 ceph-mon[34700]: scrub queued for path: / 2024-09-21T05:58:02.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:02 smithi192 ceph-mon[34700]: scrub summary: idle+waiting paths [/] 2024-09-21T05:58:02.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:02 smithi192 ceph-mon[34700]: scrub summary: active paths [/] 2024-09-21T05:58:02.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:02 smithi192 ceph-mon[34700]: scrub summary: idle+waiting paths [/] 2024-09-21T05:58:03.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.983+0000 7f45b14ef640 1 -- 172.21.15.117:0/859012324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45ac1094b0 msgr2=0x7f45ac1098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:03.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.983+0000 7f45b14ef640 1 --2- 172.21.15.117:0/859012324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45ac1094b0 0x7f45ac1098c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f45a0009900 tx=0x7f45a002a410 comp rx=0 tx=0).stop 2024-09-21T05:58:03.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.983+0000 7f45b14ef640 1 -- 172.21.15.117:0/859012324 shutdown_connections 2024-09-21T05:58:03.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.983+0000 7f45b14ef640 1 --2- 172.21.15.117:0/859012324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45ac1094b0 0x7f45ac1098c0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:03.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.983+0000 7f45b14ef640 1 --2- 172.21.15.117:0/859012324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45ac109e00 0x7f45ac110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:03.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.983+0000 7f45b14ef640 1 --2- 172.21.15.117:0/859012324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45ac101f70 0x7f45ac102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:03.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.983+0000 7f45b14ef640 1 -- 172.21.15.117:0/859012324 >> 172.21.15.117:0/859012324 conn(0x7f45ac077c60 msgr2=0x7f45ac0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:03.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.983+0000 7f45b14ef640 1 -- 172.21.15.117:0/859012324 shutdown_connections 2024-09-21T05:58:03.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.983+0000 7f45b14ef640 1 -- 172.21.15.117:0/859012324 wait complete. 2024-09-21T05:58:03.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.984+0000 7f45b14ef640 1 Processor -- start 2024-09-21T05:58:03.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.984+0000 7f45b14ef640 1 -- start start 2024-09-21T05:58:03.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.984+0000 7f45b14ef640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45ac101f70 0x7f45ac19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:03.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.985+0000 7f45b14ef640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45ac1094b0 0x7f45ac19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:03.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.985+0000 7f45b14ef640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45ac109e00 0x7f45ac19fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:03.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.985+0000 7f45b14ef640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac113b10 con 0x7f45ac109e00 2024-09-21T05:58:03.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.985+0000 7f45b14ef640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac113990 con 0x7f45ac101f70 2024-09-21T05:58:03.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.985+0000 7f45b14ef640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac113c90 con 0x7f45ac1094b0 2024-09-21T05:58:03.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.985+0000 7f45ab7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45ac109e00 0x7f45ac19fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:03.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.985+0000 7f45ab7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45ac109e00 0x7f45ac19fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41630/0 (socket says 172.21.15.117:41630) 2024-09-21T05:58:03.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.985+0000 7f45ab7fe640 1 -- 172.21.15.117:0/2244284437 learned_addr learned my addr 172.21.15.117:0/2244284437 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:03.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.985+0000 7f45aa7fc640 1 --2- 172.21.15.117:0/2244284437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45ac1094b0 0x7f45ac19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:03.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.985+0000 7f45aaffd640 1 --2- 172.21.15.117:0/2244284437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45ac101f70 0x7f45ac19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:03.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.985+0000 7f45ab7fe640 1 -- 172.21.15.117:0/2244284437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45ac1094b0 msgr2=0x7f45ac19aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:03.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.985+0000 7f45ab7fe640 1 --2- 172.21.15.117:0/2244284437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45ac1094b0 0x7f45ac19aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:03.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.985+0000 7f45ab7fe640 1 -- 172.21.15.117:0/2244284437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45ac101f70 msgr2=0x7f45ac19a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:03.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.985+0000 7f45ab7fe640 1 --2- 172.21.15.117:0/2244284437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45ac101f70 0x7f45ac19a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:03.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.985+0000 7f45ab7fe640 1 -- 172.21.15.117:0/2244284437 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45ac1a03f0 con 0x7f45ac109e00 2024-09-21T05:58:03.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.985+0000 7f45ab7fe640 1 --2- 172.21.15.117:0/2244284437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45ac109e00 0x7f45ac19fcc0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f45a400eb10 tx=0x7f45a400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:03.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.986+0000 7f4593fff640 1 -- 172.21.15.117:0/2244284437 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a4004b20 con 0x7f45ac109e00 2024-09-21T05:58:03.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.986+0000 7f4593fff640 1 -- 172.21.15.117:0/2244284437 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45a4004cc0 con 0x7f45ac109e00 2024-09-21T05:58:03.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.986+0000 7f4593fff640 1 -- 172.21.15.117:0/2244284437 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a4010670 con 0x7f45ac109e00 2024-09-21T05:58:03.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.986+0000 7f45b14ef640 1 -- 172.21.15.117:0/2244284437 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45ac1a06e0 con 0x7f45ac109e00 2024-09-21T05:58:03.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.986+0000 7f45b14ef640 1 -- 172.21.15.117:0/2244284437 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45ac102df0 con 0x7f45ac109e00 2024-09-21T05:58:03.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.989+0000 7f45b7a63740 1 Processor -- start 2024-09-21T05:58:03.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.990+0000 7f4593fff640 1 -- 172.21.15.117:0/2244284437 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f45a4005000 con 0x7f45ac109e00 2024-09-21T05:58:03.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.990+0000 7f45b7a63740 1 -- start start 2024-09-21T05:58:03.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.990+0000 7f45b7a63740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653dbbc6070 0x5653dbf6d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:03.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.990+0000 7f45b7a63740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653dbb9d1b0 0x5653dbf6d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:03.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.990+0000 7f45b7a63740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653dbba5390 0x5653dbf6dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:03.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.990+0000 7f45b7a63740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5653dbee3e50 con 0x5653dbbc6070 2024-09-21T05:58:03.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.990+0000 7f45b7a63740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5653dbed3280 con 0x5653dbb9d1b0 2024-09-21T05:58:03.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.990+0000 7f45b7a63740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5653dbed7090 con 0x5653dbba5390 2024-09-21T05:58:03.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.990+0000 7f45aa7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653dbbc6070 0x5653dbf6d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:03.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.990+0000 7f45aa7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653dbbc6070 0x5653dbf6d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41632/0 (socket says 172.21.15.117:41632) 2024-09-21T05:58:03.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.990+0000 7f45aa7fc640 1 -- 172.21.15.117:0/352230079 learned_addr learned my addr 172.21.15.117:0/352230079 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:03.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.991+0000 7f45aaffd640 1 --2- 172.21.15.117:0/352230079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653dbb9d1b0 0x5653dbf6d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:03.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.991+0000 7f45aa7fc640 1 --2- 172.21.15.117:0/352230079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653dbba5390 0x5653dbf6dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:03.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.991+0000 7f45aa7fc640 1 -- 172.21.15.117:0/352230079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653dbba5390 msgr2=0x5653dbf6dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:03.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.991+0000 7f45aa7fc640 1 --2- 172.21.15.117:0/352230079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653dbba5390 0x5653dbf6dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:03.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.991+0000 7f45aa7fc640 1 -- 172.21.15.117:0/352230079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653dbb9d1b0 msgr2=0x5653dbf6d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:03.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.991+0000 7f45aa7fc640 1 --2- 172.21.15.117:0/352230079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653dbb9d1b0 0x5653dbf6d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:03.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.991+0000 7f4593fff640 1 --2- 172.21.15.117:0/2244284437 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f45940b6780 0x7f45940b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:03.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.991+0000 7f45aa7fc640 1 -- 172.21.15.117:0/352230079 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653dbc31650 con 0x5653dbbc6070 2024-09-21T05:58:03.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.991+0000 7f45ab7fe640 1 --2- 172.21.15.117:0/2244284437 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f45940b6780 0x7f45940b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:03.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.991+0000 7f4593fff640 1 -- 172.21.15.117:0/2244284437 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f45a40e7d00 con 0x7f45ac109e00 2024-09-21T05:58:03.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.992+0000 7f45aa7fc640 1 --2- 172.21.15.117:0/352230079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653dbbc6070 0x5653dbf6d270 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f45a0005ab0 tx=0x7f45a0005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8a67c5b11f22e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:03.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.992+0000 7f45ab7fe640 1 --2- 172.21.15.117:0/2244284437 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f45940b6780 0x7f45940b8c90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f45a40aade0 tx=0x7f45a40139b0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:03.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.992+0000 7f45917fa640 1 -- 172.21.15.117:0/352230079 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0035070 con 0x5653dbbc6070 2024-09-21T05:58:03.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.992+0000 7f45917fa640 1 -- 172.21.15.117:0/352230079 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45a0004750 con 0x5653dbbc6070 2024-09-21T05:58:03.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.992+0000 7f45917fa640 1 -- 172.21.15.117:0/352230079 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0043040 con 0x5653dbbc6070 2024-09-21T05:58:03.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.993+0000 7f45b7a63740 1 -- 172.21.15.117:0/352230079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653dbbc6070 msgr2=0x5653dbf6d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:03.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.993+0000 7f45b7a63740 1 --2- 172.21.15.117:0/352230079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653dbbc6070 0x5653dbf6d270 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f45a0005ab0 tx=0x7f45a0005a20 comp rx=0 tx=0).stop 2024-09-21T05:58:03.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.993+0000 7f45b7a63740 1 -- 172.21.15.117:0/352230079 shutdown_connections 2024-09-21T05:58:03.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.993+0000 7f45b7a63740 1 --2- 172.21.15.117:0/352230079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653dbba5390 0x5653dbf6dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:03.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.993+0000 7f45b7a63740 1 --2- 172.21.15.117:0/352230079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653dbb9d1b0 0x5653dbf6d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:03.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.993+0000 7f45b7a63740 1 --2- 172.21.15.117:0/352230079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653dbbc6070 0x5653dbf6d270 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:03.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.993+0000 7f45b7a63740 1 -- 172.21.15.117:0/352230079 >> 172.21.15.117:0/352230079 conn(0x5653dbbba460 msgr2=0x5653dbf6b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:03.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.993+0000 7f45b7a63740 1 -- 172.21.15.117:0/352230079 shutdown_connections 2024-09-21T05:58:03.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.994+0000 7f45b7a63740 1 -- 172.21.15.117:0/352230079 wait complete. 2024-09-21T05:58:03.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.994+0000 7f45b7a63740 1 -- start start 2024-09-21T05:58:03.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.994+0000 7f45b7a63740 1 Processor -- start 2024-09-21T05:58:03.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.995+0000 7f45b7a63740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653dbb9d1b0 0x5653dbf78a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:03.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.995+0000 7f45b7a63740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653dbba5390 0x5653dbf78f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:03.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.995+0000 7f45b7a63740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653dbb849f0 0x5653dbf84ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:03.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.995+0000 7f45b7a63740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5653dbd34700 con 0x5653dbb9d1b0 2024-09-21T05:58:03.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.995+0000 7f45b7a63740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5653dbe6e110 con 0x5653dbba5390 2024-09-21T05:58:03.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.995+0000 7f45b7a63740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5653dbf7b9f0 con 0x5653dbb849f0 2024-09-21T05:58:03.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.995+0000 7f45aa7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653dbb9d1b0 0x5653dbf78a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:03.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.995+0000 7f45b7a63740 10 client.0 _ll_register_callbacks cb 0x5653dbe87990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:58:03.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.995+0000 7f45aaffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653dbba5390 0x5653dbf78f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:03.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.995+0000 7f45aa7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653dbb9d1b0 0x5653dbf78a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41648/0 (socket says 172.21.15.117:41648) 2024-09-21T05:58:03.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.995+0000 7f45aa7fc640 1 -- 172.21.15.117:0/134778322 learned_addr learned my addr 172.21.15.117:0/134778322 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:03.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.995+0000 7f45aaffd640 1 -- 172.21.15.117:0/134778322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653dbb849f0 msgr2=0x5653dbf84ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T05:58:03.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.995+0000 7f45aaffd640 1 --2- 172.21.15.117:0/134778322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653dbb849f0 0x5653dbf84ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:03.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.995+0000 7f45aaffd640 1 -- 172.21.15.117:0/134778322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653dbb9d1b0 msgr2=0x5653dbf78a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:03.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.995+0000 7f45aaffd640 1 --2- 172.21.15.117:0/134778322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653dbb9d1b0 0x5653dbf78a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:03.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.995+0000 7f45aaffd640 1 -- 172.21.15.117:0/134778322 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653dbe9bc40 con 0x5653dbba5390 2024-09-21T05:58:03.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.995+0000 7f45aa7fc640 1 --2- 172.21.15.117:0/134778322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653dbb9d1b0 0x5653dbf78a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:58:03.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.996+0000 7f45aaffd640 1 --2- 172.21.15.117:0/134778322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653dbba5390 0x5653dbf78f50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f459c009d10 tx=0x7f459c0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83ae0b73202d7adb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:03.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.996+0000 7f458eff5640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-21T05:58:03.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.996+0000 7f458eff5640 1 -- 172.21.15.117:0/134778322 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459c012660 con 0x5653dbba5390 2024-09-21T05:58:03.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.996+0000 7f458eff5640 1 -- 172.21.15.117:0/134778322 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f459c012800 con 0x5653dbba5390 2024-09-21T05:58:03.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.996+0000 7f45b7a63740 1 -- 172.21.15.117:0/134778322 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45a0004750 con 0x5653dbba5390 2024-09-21T05:58:03.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.996+0000 7f458eff5640 1 -- 172.21.15.117:0/134778322 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f459c012a00 con 0x5653dbba5390 2024-09-21T05:58:03.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.996+0000 7f45b7a63740 10 client.25307 fetch_fsmap learned FSMap version 27 2024-09-21T05:58:03.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.996+0000 7f45b7a63740 1 -- 172.21.15.117:0/134778322 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5653dbf87850 con 0x5653dbba5390 2024-09-21T05:58:03.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.996+0000 7f458eff5640 1 -- 172.21.15.117:0/134778322 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f459c012ba0 con 0x5653dbba5390 2024-09-21T05:58:03.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.997+0000 7f458eff5640 1 -- 172.21.15.117:0/134778322 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f459c00f040 con 0x5653dbba5390 2024-09-21T05:58:03.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.998+0000 7f45b7a63740 10 client.25307 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:58:03.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.998+0000 7f45b7a63740 10 client.25307 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T05:58:03.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.998+0000 7f45b7a63740 20 client.25307 populate_metadata read hostname 'smithi117' 2024-09-21T05:58:03.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.998+0000 7f45b7a63740 1 --2- 172.21.15.117:0/134778322 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5653dbb59da0 0x5653dbf8ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:03.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.998+0000 7f45b7a63740 4 client.25307 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:58:03.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.998+0000 7f45b7a63740 1 -- 172.21.15.117:0/134778322 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f45a0004750 con 0x5653dbb59da0 2024-09-21T05:58:03.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.998+0000 7f45aa7fc640 1 --2- 172.21.15.117:0/134778322 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5653dbb59da0 0x5653dbf8ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:03.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.998+0000 7f458eff5640 10 client.25307 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:58:03.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.998+0000 7f45aa7fc640 1 --2- 172.21.15.117:0/134778322 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5653dbb59da0 0x5653dbf8ae40 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b12fa25f17de4f66 server_cookie=a874df33000257ba in_seq=0 out_seq=0 2024-09-21T05:58:03.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.999+0000 7f458eff5640 1 -- 172.21.15.117:0/134778322 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f45a0004750 con 0x5653dbb59da0 2024-09-21T05:58:03.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.999+0000 7f458eff5640 10 client.25307 handle_command_reply: tid=0 2024-09-21T05:58:03.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.999+0000 7f45b7a63740 1 client.25307 shutdown 2024-09-21T05:58:03.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.999+0000 7f45b7a63740 1 -- 172.21.15.117:0/134778322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653dbba5390 msgr2=0x5653dbf78f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:03.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:02.999+0000 7f45b7a63740 1 --2- 172.21.15.117:0/134778322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653dbba5390 0x5653dbf78f50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f459c009d10 tx=0x7f459c0117b0 comp rx=0 tx=0).stop 2024-09-21T05:58:03.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.000+0000 7f45b7a63740 1 -- 172.21.15.117:0/134778322 shutdown_connections 2024-09-21T05:58:03.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.000+0000 7f45b7a63740 1 --2- 172.21.15.117:0/134778322 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5653dbb59da0 0x5653dbf8ae40 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:03.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.000+0000 7f45b7a63740 1 --2- 172.21.15.117:0/134778322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653dbb849f0 0x5653dbf84ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:03.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.000+0000 7f45b7a63740 1 --2- 172.21.15.117:0/134778322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653dbba5390 0x5653dbf78f50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:03.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.000+0000 7f45b7a63740 1 --2- 172.21.15.117:0/134778322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653dbb9d1b0 0x5653dbf78a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:03.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.000+0000 7f45b7a63740 1 -- 172.21.15.117:0/134778322 >> 172.21.15.117:0/134778322 conn(0x5653dbbba460 msgr2=0x5653dbf70450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:03.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.000+0000 7f458eff5640 0 client.25307 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:58:03.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.000+0000 7f45b7a63740 1 -- 172.21.15.117:0/134778322 shutdown_connections 2024-09-21T05:58:03.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.000+0000 7f45b7a63740 1 -- 172.21.15.117:0/134778322 wait complete. 2024-09-21T05:58:03.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.000+0000 7f45b7a63740 20 client.25307 trim_cache size 0 max 16384 2024-09-21T05:58:03.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.019+0000 7f45b7a63740 1 Processor -- start 2024-09-21T05:58:03.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.020+0000 7f45b7a63740 1 -- start start 2024-09-21T05:58:03.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.020+0000 7f45b7a63740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653dbb849f0 0x5653dbf6e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:03.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.020+0000 7f45b7a63740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653dbb9d1b0 0x5653dbf75c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:03.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.020+0000 7f45b7a63740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653dbba5390 0x5653dbf76140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:03.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.020+0000 7f45b7a63740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac1ad3b0 con 0x5653dbba5390 2024-09-21T05:58:03.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.020+0000 7f45b7a63740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac1aeef0 con 0x5653dbb9d1b0 2024-09-21T05:58:03.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.020+0000 7f45b7a63740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac1ada80 con 0x5653dbb849f0 2024-09-21T05:58:03.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.020+0000 7f45aa7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653dbba5390 0x5653dbf76140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:03.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.020+0000 7f45aa7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653dbba5390 0x5653dbf76140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41658/0 (socket says 172.21.15.117:41658) 2024-09-21T05:58:03.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.020+0000 7f45aa7fc640 1 -- 172.21.15.117:0/3954035856 learned_addr learned my addr 172.21.15.117:0/3954035856 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:03.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.020+0000 7f45aaffd640 1 --2- 172.21.15.117:0/3954035856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653dbb9d1b0 0x5653dbf75c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:03.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.020+0000 7f45aa7fc640 1 --2- 172.21.15.117:0/3954035856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653dbb849f0 0x5653dbf6e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:03.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.020+0000 7f45aa7fc640 1 -- 172.21.15.117:0/3954035856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653dbb849f0 msgr2=0x5653dbf6e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:03.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.020+0000 7f45aa7fc640 1 --2- 172.21.15.117:0/3954035856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653dbb849f0 0x5653dbf6e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:03.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.020+0000 7f45aa7fc640 1 -- 172.21.15.117:0/3954035856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653dbb9d1b0 msgr2=0x5653dbf75c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:03.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.020+0000 7f45aa7fc640 1 --2- 172.21.15.117:0/3954035856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653dbb9d1b0 0x5653dbf75c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:03.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.020+0000 7f45aa7fc640 1 -- 172.21.15.117:0/3954035856 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653dbb59fd0 con 0x5653dbba5390 2024-09-21T05:58:03.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.020+0000 7f45aaffd640 1 --2- 172.21.15.117:0/3954035856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653dbb9d1b0 0x5653dbf75c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:58:03.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.021+0000 7f45aa7fc640 1 --2- 172.21.15.117:0/3954035856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653dbba5390 0x5653dbf76140 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f45a0004320 tx=0x7f45a0005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab0ce7450c4078 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:03.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.021+0000 7f4590ff9640 1 -- 172.21.15.117:0/3954035856 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0035040 con 0x5653dbba5390 2024-09-21T05:58:03.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.021+0000 7f4590ff9640 1 -- 172.21.15.117:0/3954035856 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45a00053e0 con 0x5653dbba5390 2024-09-21T05:58:03.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.021+0000 7f4590ff9640 1 -- 172.21.15.117:0/3954035856 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0031820 con 0x5653dbba5390 2024-09-21T05:58:03.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.022+0000 7f45b7a63740 1 -- 172.21.15.117:0/3954035856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653dbba5390 msgr2=0x5653dbf76140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:03.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.022+0000 7f45b7a63740 1 --2- 172.21.15.117:0/3954035856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653dbba5390 0x5653dbf76140 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f45a0004320 tx=0x7f45a0005850 comp rx=0 tx=0).stop 2024-09-21T05:58:03.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.022+0000 7f45b7a63740 1 -- 172.21.15.117:0/3954035856 shutdown_connections 2024-09-21T05:58:03.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.022+0000 7f45b7a63740 1 --2- 172.21.15.117:0/3954035856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653dbba5390 0x5653dbf76140 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:03.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.022+0000 7f45b7a63740 1 --2- 172.21.15.117:0/3954035856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653dbb9d1b0 0x5653dbf75c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:03.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.022+0000 7f45b7a63740 1 --2- 172.21.15.117:0/3954035856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653dbb849f0 0x5653dbf6e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:03.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.022+0000 7f45b7a63740 1 -- 172.21.15.117:0/3954035856 >> 172.21.15.117:0/3954035856 conn(0x5653dbbba460 msgr2=0x5653dbf6e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:03.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.022+0000 7f45b7a63740 1 -- 172.21.15.117:0/3954035856 shutdown_connections 2024-09-21T05:58:03.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.022+0000 7f45b7a63740 1 -- 172.21.15.117:0/3954035856 wait complete. 2024-09-21T05:58:03.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.022+0000 7f45b7a63740 1 -- start start 2024-09-21T05:58:03.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.022+0000 7f45b7a63740 1 Processor -- start 2024-09-21T05:58:03.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.023+0000 7f45b7a63740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653dbb849f0 0x5653dbf7e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:03.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.023+0000 7f45b7a63740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653dbb9d1b0 0x5653dbf7ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:03.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.023+0000 7f45b7a63740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653dbfed980 0x5653dbfedd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:03.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.023+0000 7f45b7a63740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac1ae820 con 0x5653dbb849f0 2024-09-21T05:58:03.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.023+0000 7f45b7a63740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac1ae150 con 0x5653dbfed980 2024-09-21T05:58:03.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.023+0000 7f45b7a63740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac1ab3b0 con 0x5653dbb9d1b0 2024-09-21T05:58:03.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.023+0000 7f45b7a63740 10 client.0 _ll_register_callbacks cb 0x7f459c009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:58:03.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.023+0000 7f45aa7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653dbb849f0 0x5653dbf7e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:03.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.023+0000 7f45aa7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653dbb849f0 0x5653dbf7e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41670/0 (socket says 172.21.15.117:41670) 2024-09-21T05:58:03.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.023+0000 7f45aa7fc640 1 -- 172.21.15.117:0/3552704077 learned_addr learned my addr 172.21.15.117:0/3552704077 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:03.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.023+0000 7f45aaffd640 1 --2- 172.21.15.117:0/3552704077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653dbb9d1b0 0x5653dbf7ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:03.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.023+0000 7f45aa7fc640 1 -- 172.21.15.117:0/3552704077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653dbb9d1b0 msgr2=0x5653dbf7ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:03.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.023+0000 7f45aa7fc640 1 --2- 172.21.15.117:0/3552704077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653dbb9d1b0 0x5653dbf7ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:03.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.023+0000 7f45aa7fc640 1 -- 172.21.15.117:0/3552704077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653dbfed980 msgr2=0x5653dbfedd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:03.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.023+0000 7f45aa7fc640 1 --2- 172.21.15.117:0/3552704077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653dbfed980 0x5653dbfedd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:03.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.023+0000 7f45aa7fc640 1 -- 172.21.15.117:0/3552704077 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653dbfee400 con 0x5653dbb849f0 2024-09-21T05:58:03.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.023+0000 7f45aaffd640 1 --2- 172.21.15.117:0/3552704077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653dbb9d1b0 0x5653dbf7ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:58:03.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.023+0000 7f458eff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:58:03.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.023+0000 7f45aa7fc640 1 --2- 172.21.15.117:0/3552704077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653dbb849f0 0x5653dbf7e790 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f45a002a9d0 tx=0x7f45a0009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb359cff6cd9e3c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:03.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.024+0000 7f458eff5640 1 -- 172.21.15.117:0/3552704077 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0035030 con 0x5653dbb849f0 2024-09-21T05:58:03.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.024+0000 7f458eff5640 1 -- 172.21.15.117:0/3552704077 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f45a0043050 con 0x5653dbb849f0 2024-09-21T05:58:03.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.024+0000 7f45b7a63740 1 -- 172.21.15.117:0/3552704077 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5653dbf70f90 con 0x5653dbb849f0 2024-09-21T05:58:03.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.024+0000 7f458eff5640 1 -- 172.21.15.117:0/3552704077 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f45a00314a0 con 0x5653dbb849f0 2024-09-21T05:58:03.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.024+0000 7f45b7a63740 10 client.15783 fetch_fsmap learned FSMap version 27 2024-09-21T05:58:03.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.024+0000 7f45b7a63740 1 -- 172.21.15.117:0/3552704077 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5653dbff37f0 con 0x5653dbb849f0 2024-09-21T05:58:03.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.024+0000 7f458eff5640 1 -- 172.21.15.117:0/3552704077 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45a0031bb0 con 0x5653dbb849f0 2024-09-21T05:58:03.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.024+0000 7f458eff5640 1 -- 172.21.15.117:0/3552704077 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f45a00054f0 con 0x5653dbb849f0 2024-09-21T05:58:03.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.024+0000 7f45b7a63740 10 client.15783 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:58:03.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.024+0000 7f45b7a63740 10 client.15783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T05:58:03.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.024+0000 7f45b7a63740 20 client.15783 populate_metadata read hostname 'smithi117' 2024-09-21T05:58:03.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.024+0000 7f45b7a63740 1 --2- 172.21.15.117:0/3552704077 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5653dbff3a80 0x5653dc008ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:03.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.024+0000 7f45b7a63740 4 client.15783 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T05:58:03.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.024+0000 7f45b7a63740 1 -- 172.21.15.117:0/3552704077 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5653dbf70f90 con 0x5653dbff3a80 2024-09-21T05:58:03.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.025+0000 7f45aaffd640 1 --2- 172.21.15.117:0/3552704077 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5653dbff3a80 0x5653dc008ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:03.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.025+0000 7f458eff5640 10 client.15783 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:58:03.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.025+0000 7f45aaffd640 1 --2- 172.21.15.117:0/3552704077 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5653dbff3a80 0x5653dc008ff0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a24520e947d571fd server_cookie=c0a48b94e300ac84 in_seq=0 out_seq=0 2024-09-21T05:58:03.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.026+0000 7f458eff5640 1 -- 172.21.15.117:0/3552704077 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+854 (crc 0 0 0) 0x5653dbf70f90 con 0x5653dbff3a80 2024-09-21T05:58:03.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.026+0000 7f458eff5640 10 client.15783 handle_command_reply: tid=0 2024-09-21T05:58:03.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.026+0000 7f45b7a63740 1 client.15783 shutdown 2024-09-21T05:58:03.032 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T05:58:03.032 INFO:teuthology.orchestra.run.smithi117.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T05:58:03.032 INFO:teuthology.orchestra.run.smithi117.stdout: "scrubs": { 2024-09-21T05:58:03.032 INFO:teuthology.orchestra.run.smithi117.stdout: "8b3fe95a-b76b-4e94-afd0-72fe35e2dd92": { 2024-09-21T05:58:03.032 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T05:58:03.032 INFO:teuthology.orchestra.run.smithi117.stdout: "tag": "8b3fe95a-b76b-4e94-afd0-72fe35e2dd92", 2024-09-21T05:58:03.032 INFO:teuthology.orchestra.run.smithi117.stdout: "options": "recursive,force" 2024-09-21T05:58:03.032 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T05:58:03.032 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T05:58:03.032 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_stats": { 2024-09-21T05:58:03.032 INFO:teuthology.orchestra.run.smithi117.stdout: "8b3fe95a-b76b-4e94-afd0-72fe35e2dd92": { 2024-09-21T05:58:03.033 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:58:01.246076+0000", 2024-09-21T05:58:03.033 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T05:58:03.033 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T05:58:03.033 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T05:58:03.033 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T05:58:03.033 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T05:58:03.033 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T05:58:03.033 INFO:teuthology.orchestra.run.smithi117.stdout: "b30bfff0-0f6b-4e30-8467-ff7db421eba6": { 2024-09-21T05:58:03.033 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:57:14.939082+0000", 2024-09-21T05:58:03.033 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T05:58:03.033 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T05:58:03.033 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T05:58:03.033 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T05:58:03.033 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T05:58:03.033 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T05:58:03.034 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T05:58:03.034 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T05:58:03.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.026+0000 7f45b7a63740 1 -- 172.21.15.117:0/3552704077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653dbb849f0 msgr2=0x5653dbf7e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:03.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.026+0000 7f45b7a63740 1 --2- 172.21.15.117:0/3552704077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653dbb849f0 0x5653dbf7e790 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f45a002a9d0 tx=0x7f45a0009810 comp rx=0 tx=0).stop 2024-09-21T05:58:03.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.026+0000 7f45b7a63740 1 -- 172.21.15.117:0/3552704077 shutdown_connections 2024-09-21T05:58:03.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.026+0000 7f45b7a63740 1 --2- 172.21.15.117:0/3552704077 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5653dbff3a80 0x5653dc008ff0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:03.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.026+0000 7f45b7a63740 1 --2- 172.21.15.117:0/3552704077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5653dbfed980 0x5653dbfedd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:03.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.026+0000 7f45b7a63740 1 --2- 172.21.15.117:0/3552704077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653dbb9d1b0 0x5653dbf7ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:03.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.026+0000 7f45b7a63740 1 --2- 172.21.15.117:0/3552704077 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5653dbb849f0 0x5653dbf7e790 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:03.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.026+0000 7f458eff5640 0 client.15783 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:58:03.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.026+0000 7f45b7a63740 1 -- 172.21.15.117:0/3552704077 >> 172.21.15.117:0/3552704077 conn(0x5653dbbba460 msgr2=0x5653dbf79e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:03.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.026+0000 7f45b7a63740 1 -- 172.21.15.117:0/3552704077 shutdown_connections 2024-09-21T05:58:03.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.026+0000 7f45b7a63740 1 -- 172.21.15.117:0/3552704077 wait complete. 2024-09-21T05:58:03.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.026+0000 7f45b7a63740 20 client.15783 trim_cache size 0 max 16384 2024-09-21T05:58:03.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.027+0000 7f45b14ef640 1 -- 172.21.15.117:0/2244284437 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f45940b6780 msgr2=0x7f45940b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:03.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.027+0000 7f45b14ef640 1 --2- 172.21.15.117:0/2244284437 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f45940b6780 0x7f45940b8c90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f45a40aade0 tx=0x7f45a40139b0 comp rx=0 tx=0).stop 2024-09-21T05:58:03.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.027+0000 7f45b14ef640 1 -- 172.21.15.117:0/2244284437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45ac109e00 msgr2=0x7f45ac19fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:03.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.027+0000 7f45b14ef640 1 --2- 172.21.15.117:0/2244284437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45ac109e00 0x7f45ac19fcc0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f45a400eb10 tx=0x7f45a400c2d0 comp rx=0 tx=0).stop 2024-09-21T05:58:03.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.027+0000 7f45b14ef640 1 -- 172.21.15.117:0/2244284437 shutdown_connections 2024-09-21T05:58:03.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.027+0000 7f45b14ef640 1 --2- 172.21.15.117:0/2244284437 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f45940b6780 0x7f45940b8c90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:03.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.027+0000 7f45b14ef640 1 --2- 172.21.15.117:0/2244284437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45ac109e00 0x7f45ac19fcc0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:03.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.027+0000 7f45b14ef640 1 --2- 172.21.15.117:0/2244284437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45ac1094b0 0x7f45ac19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:03.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.027+0000 7f45b14ef640 1 --2- 172.21.15.117:0/2244284437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45ac101f70 0x7f45ac19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:03.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.027+0000 7f45b14ef640 1 -- 172.21.15.117:0/2244284437 >> 172.21.15.117:0/2244284437 conn(0x7f45ac077c60 msgr2=0x7f45ac1074a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:03.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.027+0000 7f45b14ef640 1 -- 172.21.15.117:0/2244284437 shutdown_connections 2024-09-21T05:58:03.035 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:03.027+0000 7f45b14ef640 1 -- 172.21.15.117:0/2244284437 wait complete. 2024-09-21T05:58:03.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:02 smithi138 ceph-mon[34391]: scrub queued for path: / 2024-09-21T05:58:03.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:02 smithi138 ceph-mon[34391]: scrub summary: idle+waiting paths [/] 2024-09-21T05:58:03.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:02 smithi138 ceph-mon[34391]: scrub summary: active paths [/] 2024-09-21T05:58:03.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:02 smithi138 ceph-mon[34391]: scrub summary: idle+waiting paths [/] 2024-09-21T05:58:03.820 INFO:tasks.cephfs.filesystem:scrub status for tag:8b3fe95a-b76b-4e94-afd0-72fe35e2dd92 - {'path': '/', 'tag': '8b3fe95a-b76b-4e94-afd0-72fe35e2dd92', 'options': 'recursive,force'} 2024-09-21T05:58:03.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:03 smithi192 ceph-mon[34700]: pgmap v349: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 24 op/s 2024-09-21T05:58:04.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:03 smithi117 ceph-mon[30947]: pgmap v349: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 24 op/s 2024-09-21T05:58:04.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:03 smithi138 ceph-mon[34391]: pgmap v349: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 24 op/s 2024-09-21T05:58:04.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:04 smithi192 ceph-mon[34700]: scrub summary: idle 2024-09-21T05:58:05.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:04 smithi117 ceph-mon[30947]: scrub summary: idle 2024-09-21T05:58:05.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:04 smithi138 ceph-mon[34391]: scrub summary: idle 2024-09-21T05:58:05.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:05 smithi192 ceph-mon[34700]: pgmap v350: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 26 op/s 2024-09-21T05:58:06.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:05 smithi117 ceph-mon[30947]: pgmap v350: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 26 op/s 2024-09-21T05:58:06.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:05 smithi138 ceph-mon[34391]: pgmap v350: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 26 op/s 2024-09-21T05:58:07.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:07 smithi192 ceph-mon[34700]: pgmap v351: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-21T05:58:08.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:07 smithi117 ceph-mon[30947]: pgmap v351: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-21T05:58:08.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:07 smithi138 ceph-mon[34391]: pgmap v351: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-21T05:58:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.544+0000 7f9f10e7e640 1 -- 172.21.15.117:0/737753026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f0c10dd80 msgr2=0x7f9f0c110260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.544+0000 7f9f10e7e640 1 --2- 172.21.15.117:0/737753026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f0c10dd80 0x7f9f0c110260 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f9f04009ca0 tx=0x7f9f0402a8c0 comp rx=0 tx=0).stop 2024-09-21T05:58:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.544+0000 7f9f10e7e640 1 -- 172.21.15.117:0/737753026 shutdown_connections 2024-09-21T05:58:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.544+0000 7f9f10e7e640 1 --2- 172.21.15.117:0/737753026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f0c10dd80 0x7f9f0c110260 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.544+0000 7f9f10e7e640 1 --2- 172.21.15.117:0/737753026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f0c1052b0 0x7f9f0c10d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.544+0000 7f9f10e7e640 1 --2- 172.21.15.117:0/737753026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f0c104960 0x7f9f0c104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.544+0000 7f9f10e7e640 1 -- 172.21.15.117:0/737753026 >> 172.21.15.117:0/737753026 conn(0x7f9f0c07a740 msgr2=0x7f9f0c07ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.545+0000 7f9f10e7e640 1 -- 172.21.15.117:0/737753026 shutdown_connections 2024-09-21T05:58:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.545+0000 7f9f10e7e640 1 -- 172.21.15.117:0/737753026 wait complete. 2024-09-21T05:58:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.546+0000 7f9f10e7e640 1 Processor -- start 2024-09-21T05:58:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.546+0000 7f9f10e7e640 1 -- start start 2024-09-21T05:58:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.546+0000 7f9f10e7e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f0c104960 0x7f9f0c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.546+0000 7f9f10e7e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f0c1052b0 0x7f9f0c19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.546+0000 7f9f10e7e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f0c10dd80 0x7f9f0c19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.546+0000 7f9f10e7e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c113a20 con 0x7f9f0c104960 2024-09-21T05:58:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.546+0000 7f9f10e7e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c1138a0 con 0x7f9f0c1052b0 2024-09-21T05:58:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.546+0000 7f9f10e7e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c113ba0 con 0x7f9f0c10dd80 2024-09-21T05:58:08.550 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.546+0000 7f9f0a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f0c104960 0x7f9f0c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:08.550 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.546+0000 7f9f0a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f0c104960 0x7f9f0c19a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34094/0 (socket says 172.21.15.117:34094) 2024-09-21T05:58:08.550 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.546+0000 7f9f0a59c640 1 -- 172.21.15.117:0/2481386023 learned_addr learned my addr 172.21.15.117:0/2481386023 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:08.550 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.546+0000 7f9f03fff640 1 --2- 172.21.15.117:0/2481386023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f0c1052b0 0x7f9f0c19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:08.550 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.547+0000 7f9f0ad9d640 1 --2- 172.21.15.117:0/2481386023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f0c10dd80 0x7f9f0c19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:08.550 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.547+0000 7f9f0a59c640 1 -- 172.21.15.117:0/2481386023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f0c10dd80 msgr2=0x7f9f0c19fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:08.550 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.547+0000 7f9f0a59c640 1 --2- 172.21.15.117:0/2481386023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f0c10dd80 0x7f9f0c19fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:08.550 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.547+0000 7f9f0a59c640 1 -- 172.21.15.117:0/2481386023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f0c1052b0 msgr2=0x7f9f0c19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:08.550 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.547+0000 7f9f0a59c640 1 --2- 172.21.15.117:0/2481386023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f0c1052b0 0x7f9f0c19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:08.550 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.547+0000 7f9f0a59c640 1 -- 172.21.15.117:0/2481386023 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f0c1a03d0 con 0x7f9f0c104960 2024-09-21T05:58:08.550 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.547+0000 7f9f0a59c640 1 --2- 172.21.15.117:0/2481386023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f0c104960 0x7f9f0c19a4a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00d950 tx=0x7f9efc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:08.550 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.547+0000 7f9f037fe640 1 -- 172.21.15.117:0/2481386023 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9efc017070 con 0x7f9f0c104960 2024-09-21T05:58:08.550 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.547+0000 7f9f037fe640 1 -- 172.21.15.117:0/2481386023 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9efc009c30 con 0x7f9f0c104960 2024-09-21T05:58:08.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.547+0000 7f9f10e7e640 1 -- 172.21.15.117:0/2481386023 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f0c1a06c0 con 0x7f9f0c104960 2024-09-21T05:58:08.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.547+0000 7f9f037fe640 1 -- 172.21.15.117:0/2481386023 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9efc00f620 con 0x7f9f0c104960 2024-09-21T05:58:08.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.549+0000 7f9f10e7e640 1 -- 172.21.15.117:0/2481386023 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f0c1a6e50 con 0x7f9f0c104960 2024-09-21T05:58:08.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.550+0000 7f9f037fe640 1 -- 172.21.15.117:0/2481386023 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f9efc010460 con 0x7f9f0c104960 2024-09-21T05:58:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.551+0000 7f9f037fe640 1 --2- 172.21.15.117:0/2481386023 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9ef80b6700 0x7f9ef80b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.552+0000 7f9f03fff640 1 --2- 172.21.15.117:0/2481386023 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9ef80b6700 0x7f9ef80b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.552+0000 7f9f037fe640 1 -- 172.21.15.117:0/2481386023 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f9efc0ea170 con 0x7f9f0c104960 2024-09-21T05:58:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.552+0000 7f9f03fff640 1 --2- 172.21.15.117:0/2481386023 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9ef80b6700 0x7f9ef80b8c10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c10fb50 tx=0x7f9ef400b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.553+0000 7f9f17de8740 1 Processor -- start 2024-09-21T05:58:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.553+0000 7f9f17de8740 1 -- start start 2024-09-21T05:58:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.553+0000 7f9f17de8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecb89561b0 0x55ecb8cfb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.554+0000 7f9f17de8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ecb892d2f0 0x55ecb8cfbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.554+0000 7f9f17de8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecb89354d0 0x55ecb8cfc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.554+0000 7f9f17de8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ecb8c706f0 con 0x55ecb89354d0 2024-09-21T05:58:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.554+0000 7f9f17de8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ecb8c5fb20 con 0x55ecb89561b0 2024-09-21T05:58:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.554+0000 7f9f17de8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55ecb8c63930 con 0x55ecb892d2f0 2024-09-21T05:58:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.554+0000 7f9f0a59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecb89561b0 0x55ecb8cfb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.554+0000 7f9f0ad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecb89354d0 0x55ecb8cfc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.554+0000 7f9f0a59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecb89561b0 0x55ecb8cfb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:53766/0 (socket says 172.21.15.117:53766) 2024-09-21T05:58:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.554+0000 7f9f0a59c640 1 -- 172.21.15.117:0/3691102776 learned_addr learned my addr 172.21.15.117:0/3691102776 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.554+0000 7f9f03fff640 1 --2- 172.21.15.117:0/3691102776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ecb892d2f0 0x55ecb8cfbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.554+0000 7f9f0a59c640 1 -- 172.21.15.117:0/3691102776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ecb892d2f0 msgr2=0x55ecb8cfbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.554+0000 7f9f0a59c640 1 --2- 172.21.15.117:0/3691102776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ecb892d2f0 0x55ecb8cfbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.554+0000 7f9f0a59c640 1 -- 172.21.15.117:0/3691102776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecb89354d0 msgr2=0x55ecb8cfc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.554+0000 7f9f0a59c640 1 --2- 172.21.15.117:0/3691102776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecb89354d0 0x55ecb8cfc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.554+0000 7f9f0a59c640 1 -- 172.21.15.117:0/3691102776 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecb8c25530 con 0x55ecb89561b0 2024-09-21T05:58:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.554+0000 7f9f0ad9d640 1 --2- 172.21.15.117:0/3691102776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecb89354d0 0x55ecb8cfc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:58:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.555+0000 7f9f0a59c640 1 --2- 172.21.15.117:0/3691102776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecb89561b0 0x55ecb8cfb9b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9efc0a9850 tx=0x7f9efc0abf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=183ffe48419ef4fc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.555+0000 7f9f00ff9640 1 -- 172.21.15.117:0/3691102776 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9efc0eb060 con 0x55ecb89561b0 2024-09-21T05:58:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.555+0000 7f9f00ff9640 1 -- 172.21.15.117:0/3691102776 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9efc009de0 con 0x55ecb89561b0 2024-09-21T05:58:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.555+0000 7f9f00ff9640 1 -- 172.21.15.117:0/3691102776 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9efc010750 con 0x55ecb89561b0 2024-09-21T05:58:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.556+0000 7f9f17de8740 1 -- 172.21.15.117:0/3691102776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecb89561b0 msgr2=0x55ecb8cfb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.556+0000 7f9f17de8740 1 --2- 172.21.15.117:0/3691102776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecb89561b0 0x55ecb8cfb9b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9efc0a9850 tx=0x7f9efc0abf30 comp rx=0 tx=0).stop 2024-09-21T05:58:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.556+0000 7f9f17de8740 1 -- 172.21.15.117:0/3691102776 shutdown_connections 2024-09-21T05:58:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.556+0000 7f9f17de8740 1 --2- 172.21.15.117:0/3691102776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecb89354d0 0x55ecb8cfc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.556+0000 7f9f17de8740 1 --2- 172.21.15.117:0/3691102776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ecb892d2f0 0x55ecb8cfbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.556+0000 7f9f17de8740 1 --2- 172.21.15.117:0/3691102776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecb89561b0 0x55ecb8cfb9b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.556+0000 7f9f17de8740 1 -- 172.21.15.117:0/3691102776 >> 172.21.15.117:0/3691102776 conn(0x55ecb894a300 msgr2=0x55ecb8cf98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.556+0000 7f9f17de8740 1 -- 172.21.15.117:0/3691102776 shutdown_connections 2024-09-21T05:58:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.556+0000 7f9f17de8740 1 -- 172.21.15.117:0/3691102776 wait complete. 2024-09-21T05:58:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.556+0000 7f9f17de8740 1 -- start start 2024-09-21T05:58:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.556+0000 7f9f17de8740 1 Processor -- start 2024-09-21T05:58:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.557+0000 7f9f17de8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ecb892d2f0 0x55ecb8d07060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.557+0000 7f9f17de8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecb89354d0 0x55ecb8d075a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.557+0000 7f9f17de8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecb89561b0 0x55ecb8d13360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.557+0000 7f9f17de8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ecb8b97ab0 con 0x55ecb89354d0 2024-09-21T05:58:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.557+0000 7f9f17de8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ecb8c05730 con 0x55ecb89561b0 2024-09-21T05:58:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.557+0000 7f9f17de8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55ecb8d0a070 con 0x55ecb892d2f0 2024-09-21T05:58:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.557+0000 7f9f17de8740 10 client.0 _ll_register_callbacks cb 0x55ecb8c596b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:58:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.557+0000 7f9f03fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecb89354d0 0x55ecb8d075a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.557+0000 7f9f0a59c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ecb892d2f0 0x55ecb8d07060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.557+0000 7f9f0a59c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ecb892d2f0 0x55ecb8d07060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:44362/0 (socket says 172.21.15.117:44362) 2024-09-21T05:58:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.557+0000 7f9f03fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecb89354d0 0x55ecb8d075a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34110/0 (socket says 172.21.15.117:34110) 2024-09-21T05:58:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.557+0000 7f9f0a59c640 1 -- 172.21.15.117:0/102318143 learned_addr learned my addr 172.21.15.117:0/102318143 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.557+0000 7f9f0a59c640 1 -- 172.21.15.117:0/102318143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecb89561b0 msgr2=0x55ecb8d13360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.557+0000 7f9f0a59c640 1 --2- 172.21.15.117:0/102318143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecb89561b0 0x55ecb8d13360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.557+0000 7f9f0a59c640 1 -- 172.21.15.117:0/102318143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecb89354d0 msgr2=0x55ecb8d075a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.557+0000 7f9f0a59c640 1 --2- 172.21.15.117:0/102318143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecb89354d0 0x55ecb8d075a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.557+0000 7f9f0a59c640 1 -- 172.21.15.117:0/102318143 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecb89c1790 con 0x55ecb892d2f0 2024-09-21T05:58:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.557+0000 7f9f03fff640 1 --2- 172.21.15.117:0/102318143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecb89354d0 0x55ecb8d075a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:58:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.558+0000 7f9f0a59c640 1 --2- 172.21.15.117:0/102318143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ecb892d2f0 0x55ecb8d07060 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9efc0ac2e0 tx=0x7f9efc0e6420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d3240a3a79d0832 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:08.562 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.558+0000 7f9ef1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-21T05:58:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.558+0000 7f9ef1ffb640 1 -- 172.21.15.117:0/102318143 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9efc05b020 con 0x55ecb892d2f0 2024-09-21T05:58:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.558+0000 7f9f17de8740 1 -- 172.21.15.117:0/102318143 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9efc009de0 con 0x55ecb892d2f0 2024-09-21T05:58:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.558+0000 7f9ef1ffb640 1 -- 172.21.15.117:0/102318143 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f9efc0a8130 con 0x55ecb892d2f0 2024-09-21T05:58:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.558+0000 7f9ef1ffb640 1 -- 172.21.15.117:0/102318143 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9efc0b1010 con 0x55ecb892d2f0 2024-09-21T05:58:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.558+0000 7f9f17de8740 10 client.25297 fetch_fsmap learned FSMap version 27 2024-09-21T05:58:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.558+0000 7f9f17de8740 1 -- 172.21.15.117:0/102318143 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ecb8a3a620 con 0x55ecb892d2f0 2024-09-21T05:58:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.558+0000 7f9ef1ffb640 1 -- 172.21.15.117:0/102318143 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9efc009de0 con 0x55ecb892d2f0 2024-09-21T05:58:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.559+0000 7f9ef1ffb640 1 -- 172.21.15.117:0/102318143 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9efc0a6ce0 con 0x55ecb892d2f0 2024-09-21T05:58:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.559+0000 7f9f17de8740 10 client.25297 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:58:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.559+0000 7f9f17de8740 10 client.25297 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T05:58:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.559+0000 7f9f17de8740 20 client.25297 populate_metadata read hostname 'smithi117' 2024-09-21T05:58:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.559+0000 7f9f17de8740 1 --2- 172.21.15.117:0/102318143 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ecb8914d80 0x55ecb8d1d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.559+0000 7f9f17de8740 4 client.25297 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:58:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.559+0000 7f9f17de8740 1 -- 172.21.15.117:0/102318143 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecb8c72640 con 0x55ecb8914d80 2024-09-21T05:58:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.560+0000 7f9f03fff640 1 --2- 172.21.15.117:0/102318143 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ecb8914d80 0x55ecb8d1d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.560+0000 7f9ef1ffb640 10 client.25297 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:58:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.560+0000 7f9f03fff640 1 --2- 172.21.15.117:0/102318143 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ecb8914d80 0x55ecb8d1d320 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54f044f700bfb21f server_cookie=58fcf34b29a0a08 in_seq=0 out_seq=0 2024-09-21T05:58:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.561+0000 7f9ef1ffb640 1 -- 172.21.15.117:0/102318143 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ecb8c72640 con 0x55ecb8914d80 2024-09-21T05:58:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.561+0000 7f9ef1ffb640 10 client.25297 handle_command_reply: tid=0 2024-09-21T05:58:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.561+0000 7f9f17de8740 1 client.25297 shutdown 2024-09-21T05:58:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.561+0000 7f9f17de8740 1 -- 172.21.15.117:0/102318143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ecb892d2f0 msgr2=0x55ecb8d07060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.561+0000 7f9f17de8740 1 --2- 172.21.15.117:0/102318143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ecb892d2f0 0x55ecb8d07060 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9efc0ac2e0 tx=0x7f9efc0e6420 comp rx=0 tx=0).stop 2024-09-21T05:58:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.561+0000 7f9f17de8740 1 -- 172.21.15.117:0/102318143 shutdown_connections 2024-09-21T05:58:08.565 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.561+0000 7f9f17de8740 1 --2- 172.21.15.117:0/102318143 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ecb8914d80 0x55ecb8d1d320 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:08.566 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.561+0000 7f9f17de8740 1 --2- 172.21.15.117:0/102318143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecb89354d0 0x55ecb8d075a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:08.566 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.561+0000 7f9ef1ffb640 0 client.25297 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:58:08.566 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.561+0000 7f9f17de8740 1 --2- 172.21.15.117:0/102318143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecb89561b0 0x55ecb8d13360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:08.566 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.561+0000 7f9f17de8740 1 --2- 172.21.15.117:0/102318143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ecb892d2f0 0x55ecb8d07060 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:08.566 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.561+0000 7f9f17de8740 1 -- 172.21.15.117:0/102318143 >> 172.21.15.117:0/102318143 conn(0x55ecb894a300 msgr2=0x55ecb8cfc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:08.566 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.561+0000 7f9f17de8740 1 -- 172.21.15.117:0/102318143 shutdown_connections 2024-09-21T05:58:08.566 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.561+0000 7f9f17de8740 1 -- 172.21.15.117:0/102318143 wait complete. 2024-09-21T05:58:08.566 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.561+0000 7f9f17de8740 20 client.25297 trim_cache size 0 max 16384 2024-09-21T05:58:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.576+0000 7f9f17de8740 1 Processor -- start 2024-09-21T05:58:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.576+0000 7f9f17de8740 1 -- start start 2024-09-21T05:58:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.576+0000 7f9f17de8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecb8914d80 0x55ecb8cfd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.576+0000 7f9f17de8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecb892d2f0 0x55ecb8d08c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.576+0000 7f9f17de8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ecb89354d0 0x55ecb8d228f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.576+0000 7f9f17de8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c1ad480 con 0x55ecb892d2f0 2024-09-21T05:58:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.576+0000 7f9f17de8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c1aefc0 con 0x55ecb8914d80 2024-09-21T05:58:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.576+0000 7f9f17de8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c1adb50 con 0x55ecb89354d0 2024-09-21T05:58:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.576+0000 7f9f03fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecb892d2f0 0x55ecb8d08c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.576+0000 7f9f03fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecb892d2f0 0x55ecb8d08c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34120/0 (socket says 172.21.15.117:34120) 2024-09-21T05:58:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.576+0000 7f9f03fff640 1 -- 172.21.15.117:0/3928688133 learned_addr learned my addr 172.21.15.117:0/3928688133 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.576+0000 7f9f0ad9d640 1 --2- 172.21.15.117:0/3928688133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ecb89354d0 0x55ecb8d228f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.576+0000 7f9f0a59c640 1 --2- 172.21.15.117:0/3928688133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecb8914d80 0x55ecb8cfd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.577+0000 7f9f03fff640 1 -- 172.21.15.117:0/3928688133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ecb89354d0 msgr2=0x55ecb8d228f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.577+0000 7f9f03fff640 1 --2- 172.21.15.117:0/3928688133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ecb89354d0 0x55ecb8d228f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.577+0000 7f9f03fff640 1 -- 172.21.15.117:0/3928688133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecb8914d80 msgr2=0x55ecb8cfd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.577+0000 7f9f03fff640 1 --2- 172.21.15.117:0/3928688133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecb8914d80 0x55ecb8cfd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.577+0000 7f9f03fff640 1 -- 172.21.15.117:0/3928688133 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecb8cfa1a0 con 0x55ecb892d2f0 2024-09-21T05:58:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.577+0000 7f9f03fff640 1 --2- 172.21.15.117:0/3928688133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecb892d2f0 0x55ecb8d08c90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c06c730 tx=0x7f9ef4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e10808bc2c66d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.577+0000 7f9ef3fff640 1 -- 172.21.15.117:0/3928688133 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4021190 con 0x55ecb892d2f0 2024-09-21T05:58:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.577+0000 7f9ef3fff640 1 -- 172.21.15.117:0/3928688133 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ef4021a80 con 0x55ecb892d2f0 2024-09-21T05:58:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.577+0000 7f9ef3fff640 1 -- 172.21.15.117:0/3928688133 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef402aa00 con 0x55ecb892d2f0 2024-09-21T05:58:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.578+0000 7f9f17de8740 1 -- 172.21.15.117:0/3928688133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecb892d2f0 msgr2=0x55ecb8d08c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.578+0000 7f9f17de8740 1 --2- 172.21.15.117:0/3928688133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecb892d2f0 0x55ecb8d08c90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c06c730 tx=0x7f9ef4013000 comp rx=0 tx=0).stop 2024-09-21T05:58:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.578+0000 7f9f17de8740 1 -- 172.21.15.117:0/3928688133 shutdown_connections 2024-09-21T05:58:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.578+0000 7f9f17de8740 1 --2- 172.21.15.117:0/3928688133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecb892d2f0 0x55ecb8d08c90 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.578+0000 7f9f17de8740 1 --2- 172.21.15.117:0/3928688133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ecb89354d0 0x55ecb8d228f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.578+0000 7f9f17de8740 1 --2- 172.21.15.117:0/3928688133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecb8914d80 0x55ecb8cfd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:08.583 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.578+0000 7f9f17de8740 1 -- 172.21.15.117:0/3928688133 >> 172.21.15.117:0/3928688133 conn(0x55ecb894a300 msgr2=0x55ecb8d08580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:08.583 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.578+0000 7f9f17de8740 1 -- 172.21.15.117:0/3928688133 shutdown_connections 2024-09-21T05:58:08.583 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.578+0000 7f9f17de8740 1 -- 172.21.15.117:0/3928688133 wait complete. 2024-09-21T05:58:08.583 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.579+0000 7f9f17de8740 1 -- start start 2024-09-21T05:58:08.583 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.579+0000 7f9f17de8740 1 Processor -- start 2024-09-21T05:58:08.583 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.579+0000 7f9f17de8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ecb8914d80 0x55ecb8d0f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:08.583 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.579+0000 7f9f17de8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecb89354d0 0x55ecb8d0cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:08.583 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.579+0000 7f9f17de8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecb89561b0 0x55ecb8d0d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:08.583 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.579+0000 7f9f17de8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c1ae8f0 con 0x55ecb89354d0 2024-09-21T05:58:08.583 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.579+0000 7f9f17de8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c1ae220 con 0x55ecb89561b0 2024-09-21T05:58:08.583 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.579+0000 7f9f17de8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0c1ab460 con 0x55ecb8914d80 2024-09-21T05:58:08.583 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.579+0000 7f9f17de8740 10 client.0 _ll_register_callbacks cb 0x7f9f04009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:58:08.583 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.579+0000 7f9f03fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecb89354d0 0x55ecb8d0cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:08.584 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.579+0000 7f9f03fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecb89354d0 0x55ecb8d0cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34130/0 (socket says 172.21.15.117:34130) 2024-09-21T05:58:08.584 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.579+0000 7f9f03fff640 1 -- 172.21.15.117:0/2796576003 learned_addr learned my addr 172.21.15.117:0/2796576003 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:08.584 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.579+0000 7f9f0ad9d640 1 --2- 172.21.15.117:0/2796576003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecb89561b0 0x55ecb8d0d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:08.584 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.579+0000 7f9f0a59c640 1 --2- 172.21.15.117:0/2796576003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ecb8914d80 0x55ecb8d0f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:08.584 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.580+0000 7f9f03fff640 1 -- 172.21.15.117:0/2796576003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ecb8914d80 msgr2=0x55ecb8d0f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:08.584 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.580+0000 7f9f03fff640 1 --2- 172.21.15.117:0/2796576003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ecb8914d80 0x55ecb8d0f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:08.584 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.580+0000 7f9f03fff640 1 -- 172.21.15.117:0/2796576003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecb89561b0 msgr2=0x55ecb8d0d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:08.584 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.580+0000 7f9f03fff640 1 --2- 172.21.15.117:0/2796576003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecb89561b0 0x55ecb8d0d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:08.584 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.580+0000 7f9f03fff640 1 -- 172.21.15.117:0/2796576003 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecb8d0d700 con 0x55ecb89354d0 2024-09-21T05:58:08.584 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.580+0000 7f9f03fff640 1 --2- 172.21.15.117:0/2796576003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecb89354d0 0x55ecb8d0cc80 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4011520 tx=0x7f9ef4005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aab9706e3e39037 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:08.584 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.580+0000 7f9ef1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:58:08.584 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.580+0000 7f9ef1ffb640 1 -- 172.21.15.117:0/2796576003 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4040060 con 0x55ecb89354d0 2024-09-21T05:58:08.584 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.580+0000 7f9ef1ffb640 1 -- 172.21.15.117:0/2796576003 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f9ef402d050 con 0x55ecb89354d0 2024-09-21T05:58:08.584 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.580+0000 7f9f17de8740 1 -- 172.21.15.117:0/2796576003 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ef4021a80 con 0x55ecb89354d0 2024-09-21T05:58:08.585 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.580+0000 7f9ef1ffb640 1 -- 172.21.15.117:0/2796576003 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9ef40215d0 con 0x55ecb89354d0 2024-09-21T05:58:08.585 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.580+0000 7f9f17de8740 10 client.15801 fetch_fsmap learned FSMap version 27 2024-09-21T05:58:08.585 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.580+0000 7f9f17de8740 1 -- 172.21.15.117:0/2796576003 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ecb8d87760 con 0x55ecb89354d0 2024-09-21T05:58:08.586 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.581+0000 7f9ef1ffb640 1 -- 172.21.15.117:0/2796576003 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ef4021a80 con 0x55ecb89354d0 2024-09-21T05:58:08.586 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.581+0000 7f9ef1ffb640 1 -- 172.21.15.117:0/2796576003 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9ef403b470 con 0x55ecb89354d0 2024-09-21T05:58:08.586 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.581+0000 7f9f17de8740 10 client.15801 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:58:08.586 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.581+0000 7f9f17de8740 10 client.15801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T05:58:08.586 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.581+0000 7f9f17de8740 20 client.15801 populate_metadata read hostname 'smithi117' 2024-09-21T05:58:08.586 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.581+0000 7f9f17de8740 1 --2- 172.21.15.117:0/2796576003 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ecb88ea1c0 0x55ecb8d89f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:08.586 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.581+0000 7f9f17de8740 4 client.15801 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T05:58:08.586 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.581+0000 7f9f17de8740 1 -- 172.21.15.117:0/2796576003 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ecb8c6e0f0 con 0x55ecb88ea1c0 2024-09-21T05:58:08.586 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.581+0000 7f9f0a59c640 1 --2- 172.21.15.117:0/2796576003 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ecb88ea1c0 0x55ecb8d89f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:08.586 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.582+0000 7f9ef1ffb640 10 client.15801 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:58:08.586 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.582+0000 7f9f0a59c640 1 --2- 172.21.15.117:0/2796576003 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ecb88ea1c0 0x55ecb8d89f20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebdc3621dd5ecfe6 server_cookie=5935166ef3d4ab30 in_seq=0 out_seq=0 2024-09-21T05:58:08.590 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.588+0000 7f9ef1ffb640 1 -- 172.21.15.117:0/2796576003 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ecb8c6e0f0 con 0x55ecb88ea1c0 2024-09-21T05:58:08.590 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.588+0000 7f9ef1ffb640 10 client.15801 handle_command_reply: tid=0 2024-09-21T05:58:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.588+0000 7f9f17de8740 1 client.15801 shutdown 2024-09-21T05:58:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.588+0000 7f9f17de8740 1 -- 172.21.15.117:0/2796576003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecb89354d0 msgr2=0x55ecb8d0cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.588+0000 7f9f17de8740 1 --2- 172.21.15.117:0/2796576003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecb89354d0 0x55ecb8d0cc80 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4011520 tx=0x7f9ef4005f60 comp rx=0 tx=0).stop 2024-09-21T05:58:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.588+0000 7f9f17de8740 1 -- 172.21.15.117:0/2796576003 shutdown_connections 2024-09-21T05:58:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.588+0000 7f9f17de8740 1 --2- 172.21.15.117:0/2796576003 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ecb88ea1c0 0x55ecb8d89f20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.588+0000 7f9f17de8740 1 --2- 172.21.15.117:0/2796576003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ecb89354d0 0x55ecb8d0cc80 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.588+0000 7f9f17de8740 1 --2- 172.21.15.117:0/2796576003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ecb89561b0 0x55ecb8d0d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.588+0000 7f9f17de8740 1 --2- 172.21.15.117:0/2796576003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ecb8914d80 0x55ecb8d0f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.588+0000 7f9f17de8740 1 -- 172.21.15.117:0/2796576003 >> 172.21.15.117:0/2796576003 conn(0x55ecb894a300 msgr2=0x55ecb8d08ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.588+0000 7f9ef1ffb640 0 client.15801 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:58:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.589+0000 7f9f17de8740 1 -- 172.21.15.117:0/2796576003 shutdown_connections 2024-09-21T05:58:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.589+0000 7f9f17de8740 1 -- 172.21.15.117:0/2796576003 wait complete. 2024-09-21T05:58:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.589+0000 7f9f17de8740 20 client.15801 trim_cache size 0 max 16384 2024-09-21T05:58:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T05:58:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T05:58:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T05:58:08.592 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T05:58:08.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.590+0000 7f9f10e7e640 1 -- 172.21.15.117:0/2481386023 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9ef80b6700 msgr2=0x7f9ef80b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:08.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.590+0000 7f9f10e7e640 1 --2- 172.21.15.117:0/2481386023 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9ef80b6700 0x7f9ef80b8c10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c10fb50 tx=0x7f9ef400b410 comp rx=0 tx=0).stop 2024-09-21T05:58:08.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.590+0000 7f9f10e7e640 1 -- 172.21.15.117:0/2481386023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f0c104960 msgr2=0x7f9f0c19a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:08.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.590+0000 7f9f10e7e640 1 --2- 172.21.15.117:0/2481386023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f0c104960 0x7f9f0c19a4a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00d950 tx=0x7f9efc00de10 comp rx=0 tx=0).stop 2024-09-21T05:58:08.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.590+0000 7f9f10e7e640 1 -- 172.21.15.117:0/2481386023 shutdown_connections 2024-09-21T05:58:08.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.590+0000 7f9f10e7e640 1 --2- 172.21.15.117:0/2481386023 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9ef80b6700 0x7f9ef80b8c10 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:08.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.590+0000 7f9f10e7e640 1 --2- 172.21.15.117:0/2481386023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f0c10dd80 0x7f9f0c19fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:08.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.590+0000 7f9f10e7e640 1 --2- 172.21.15.117:0/2481386023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f0c1052b0 0x7f9f0c19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:08.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.590+0000 7f9f10e7e640 1 --2- 172.21.15.117:0/2481386023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f0c104960 0x7f9f0c19a4a0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:08.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.590+0000 7f9f10e7e640 1 -- 172.21.15.117:0/2481386023 >> 172.21.15.117:0/2481386023 conn(0x7f9f0c07a740 msgr2=0x7f9f0c0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:08.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.590+0000 7f9f10e7e640 1 -- 172.21.15.117:0/2481386023 shutdown_connections 2024-09-21T05:58:08.594 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:08.590+0000 7f9f10e7e640 1 -- 172.21.15.117:0/2481386023 wait complete. 2024-09-21T05:58:09.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:08 smithi117 ceph-mon[30947]: pgmap v352: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-21T05:58:09.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:08 smithi138 ceph-mon[34391]: pgmap v352: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-21T05:58:09.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:08 smithi192 ceph-mon[34700]: pgmap v352: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-21T05:58:11.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:11 smithi138 ceph-mon[34391]: pgmap v353: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-09-21T05:58:11.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:11 smithi192 ceph-mon[34700]: pgmap v353: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-09-21T05:58:12.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:11 smithi117 ceph-mon[30947]: pgmap v353: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-09-21T05:58:13.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:13 smithi138 ceph-mon[34391]: pgmap v354: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-09-21T05:58:13.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:13 smithi192 ceph-mon[34700]: pgmap v354: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-09-21T05:58:14.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:13 smithi117 ceph-mon[30947]: pgmap v354: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-09-21T05:58:15.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:15 smithi138 ceph-mon[34391]: pgmap v355: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-21T05:58:15.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:15 smithi192 ceph-mon[34700]: pgmap v355: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-21T05:58:16.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:15 smithi117 ceph-mon[30947]: pgmap v355: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-21T05:58:17.464 INFO:tasks.workunit.client.0.smithi117.stderr:Note: switching to '1843221895adccfb3b692060251dd6a0183bc410'. 2024-09-21T05:58:17.464 INFO:tasks.workunit.client.0.smithi117.stderr: 2024-09-21T05:58:17.464 INFO:tasks.workunit.client.0.smithi117.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-21T05:58:17.464 INFO:tasks.workunit.client.0.smithi117.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-21T05:58:17.464 INFO:tasks.workunit.client.0.smithi117.stderr:state without impacting any branches by switching back to a branch. 2024-09-21T05:58:17.464 INFO:tasks.workunit.client.0.smithi117.stderr: 2024-09-21T05:58:17.464 INFO:tasks.workunit.client.0.smithi117.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-21T05:58:17.465 INFO:tasks.workunit.client.0.smithi117.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-21T05:58:17.465 INFO:tasks.workunit.client.0.smithi117.stderr: 2024-09-21T05:58:17.465 INFO:tasks.workunit.client.0.smithi117.stderr: git switch -c 2024-09-21T05:58:17.465 INFO:tasks.workunit.client.0.smithi117.stderr: 2024-09-21T05:58:17.465 INFO:tasks.workunit.client.0.smithi117.stderr:Or undo this operation with: 2024-09-21T05:58:17.465 INFO:tasks.workunit.client.0.smithi117.stderr: 2024-09-21T05:58:17.465 INFO:tasks.workunit.client.0.smithi117.stderr: git switch - 2024-09-21T05:58:17.465 INFO:tasks.workunit.client.0.smithi117.stderr: 2024-09-21T05:58:17.465 INFO:tasks.workunit.client.0.smithi117.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-21T05:58:17.465 INFO:tasks.workunit.client.0.smithi117.stderr: 2024-09-21T05:58:17.465 INFO:tasks.workunit.client.0.smithi117.stderr:HEAD is now at 1843221895a Merge PR #56776 into wip-pdonnell-testing-20240920.212106-debug 2024-09-21T05:58:17.469 DEBUG:teuthology.orchestra.run.smithi117:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-21T05:58:17.504 INFO:tasks.workunit.client.0.smithi117.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-21T05:58:17.507 INFO:tasks.workunit.client.0.smithi117.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T05:58:17.507 INFO:tasks.workunit.client.0.smithi117.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-21T05:58:17.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:17 smithi192 ceph-mon[34700]: pgmap v356: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-21T05:58:18.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:17 smithi117 ceph-mon[30947]: pgmap v356: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-21T05:58:18.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:17 smithi138 ceph-mon[34391]: pgmap v356: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-09-21T05:58:18.169 INFO:tasks.workunit.client.0.smithi117.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-21T05:58:18.236 INFO:tasks.workunit.client.0.smithi117.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-21T05:58:18.275 INFO:tasks.workunit.client.0.smithi117.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T05:58:18.277 INFO:tasks.workunit.client.0.smithi117.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T05:58:18.277 INFO:tasks.workunit.client.0.smithi117.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-21T05:58:18.316 INFO:tasks.workunit.client.0.smithi117.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T05:58:18.319 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-21T05:58:18.319 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-21T05:58:18.376 INFO:tasks.workunit:Running workunits matching suites/iogen.sh on client.0... 2024-09-21T05:58:18.378 INFO:tasks.workunit:Running workunit suites/iogen.sh... 2024-09-21T05:58:18.379 DEBUG:teuthology.orchestra.run.smithi117:workunit test suites/iogen.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1843221895adccfb3b692060251dd6a0183bc410 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iogen.sh 2024-09-21T05:58:18.437 INFO:tasks.workunit.client.0.smithi117.stdout:getting iogen 2024-09-21T05:58:18.438 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'getting iogen' 2024-09-21T05:58:18.438 INFO:tasks.workunit.client.0.smithi117.stderr:+ wget http://download.ceph.com/qa/iogen_3.1p0.tar 2024-09-21T05:58:18.455 INFO:tasks.workunit.client.0.smithi117.stderr:--2024-09-21 05:58:18-- http://download.ceph.com/qa/iogen_3.1p0.tar 2024-09-21T05:58:18.456 INFO:tasks.workunit.client.0.smithi117.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-09-21T05:58:18.477 INFO:tasks.workunit.client.0.smithi117.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-09-21T05:58:18.498 INFO:tasks.workunit.client.0.smithi117.stderr:HTTP request sent, awaiting response... 200 OK 2024-09-21T05:58:18.498 INFO:tasks.workunit.client.0.smithi117.stderr:Length: 8071 (7.9K) [application/octet-stream] 2024-09-21T05:58:18.499 INFO:tasks.workunit.client.0.smithi117.stderr:Saving to: ‘iogen_3.1p0.tar’ 2024-09-21T05:58:18.502 INFO:tasks.workunit.client.0.smithi117.stderr: 2024-09-21T05:58:18.503 INFO:tasks.workunit.client.0.smithi117.stderr: 0K ....... 100% 1.31G=0s 2024-09-21T05:58:18.503 INFO:tasks.workunit.client.0.smithi117.stderr: 2024-09-21T05:58:18.503 INFO:tasks.workunit.client.0.smithi117.stderr:2024-09-21 05:58:18 (1.31 GB/s) - ‘iogen_3.1p0.tar’ saved [8071/8071] 2024-09-21T05:58:18.503 INFO:tasks.workunit.client.0.smithi117.stderr: 2024-09-21T05:58:18.503 INFO:tasks.workunit.client.0.smithi117.stderr:+ tar -xvzf iogen_3.1p0.tar 2024-09-21T05:58:18.513 INFO:tasks.workunit.client.0.smithi117.stdout:iogen_3.1p0 2024-09-21T05:58:18.514 INFO:tasks.workunit.client.0.smithi117.stdout:iogen_3.1p0/src 2024-09-21T05:58:18.515 INFO:tasks.workunit.client.0.smithi117.stdout:iogen_3.1p0/src/iogen.8 2024-09-21T05:58:18.516 INFO:tasks.workunit.client.0.smithi117.stdout:iogen_3.1p0/src/iogen.c 2024-09-21T05:58:18.517 INFO:tasks.workunit.client.0.smithi117.stdout:iogen_3.1p0/README 2024-09-21T05:58:18.518 INFO:tasks.workunit.client.0.smithi117.stdout:iogen_3.1p0/Makefile.bsd 2024-09-21T05:58:18.519 INFO:tasks.workunit.client.0.smithi117.stdout:iogen_3.1p0/Makefile 2024-09-21T05:58:18.520 INFO:tasks.workunit.client.0.smithi117.stderr:+ cd iogen_3.1p0 2024-09-21T05:58:18.521 INFO:tasks.workunit.client.0.smithi117.stdout:making iogen 2024-09-21T05:58:18.521 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'making iogen' 2024-09-21T05:58:18.521 INFO:tasks.workunit.client.0.smithi117.stderr:+ make 2024-09-21T05:58:18.531 INFO:tasks.workunit.client.0.smithi117.stdout:gcc -D_FILE_OFFSET_BITS=64 -DVERSION=\"3.1p0\" -Dstrlcpy=strncpy -Darc4random=rand -O2 src/iogen.c -o iogen 2024-09-21T05:58:18.766 INFO:tasks.workunit.client.0.smithi117.stdout:nroff -mandoc -Tascii src/iogen.8 > iogen.cat8 2024-09-21T05:58:18.852 INFO:tasks.workunit.client.0.smithi117.stdout:running iogen 2024-09-21T05:58:18.852 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'running iogen' 2024-09-21T05:58:18.852 INFO:tasks.workunit.client.0.smithi117.stderr:+ ./iogen -n 5 -s 2g 2024-09-21T05:58:18.853 INFO:tasks.workunit.client.0.smithi117.stderr:child spawned 87140 2024-09-21T05:58:18.854 INFO:tasks.workunit.client.0.smithi117.stderr:child spawned 87141 2024-09-21T05:58:18.854 INFO:tasks.workunit.client.0.smithi117.stderr:child spawned 87142 2024-09-21T05:58:18.854 INFO:tasks.workunit.client.0.smithi117.stderr:child spawned 87143 2024-09-21T05:58:18.854 INFO:tasks.workunit.client.0.smithi117.stderr:child spawned 87144 2024-09-21T05:58:18.854 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'sleep for 10 min' 2024-09-21T05:58:18.854 INFO:tasks.workunit.client.0.smithi117.stdout:sleep for 10 min 2024-09-21T05:58:18.854 INFO:tasks.workunit.client.0.smithi117.stderr:+ sleep 600 2024-09-21T05:58:19.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:19 smithi192 ceph-mon[34700]: pgmap v357: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-21T05:58:20.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:19 smithi117 ceph-mon[30947]: pgmap v357: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-21T05:58:20.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:19 smithi138 ceph-mon[34391]: pgmap v357: 97 pgs: 97 active+clean; 1.4 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-21T05:58:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.760+0000 7fe7b315f640 1 -- 172.21.15.117:0/3516084033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7ac110140 msgr2=0x7fe7ac112560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.760+0000 7fe7b315f640 1 --2- 172.21.15.117:0/3516084033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7ac110140 0x7fe7ac112560 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4009ca0 tx=0x7fe7a402a980 comp rx=0 tx=0).stop 2024-09-21T05:58:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.760+0000 7fe7b315f640 1 -- 172.21.15.117:0/3516084033 shutdown_connections 2024-09-21T05:58:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.760+0000 7fe7b315f640 1 --2- 172.21.15.117:0/3516084033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7ac110140 0x7fe7ac112560 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.760+0000 7fe7b315f640 1 --2- 172.21.15.117:0/3516084033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7ac102ee0 0x7fe7ac10fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.760+0000 7fe7b315f640 1 --2- 172.21.15.117:0/3516084033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7ac102590 0x7fe7ac1029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.760+0000 7fe7b315f640 1 -- 172.21.15.117:0/3516084033 >> 172.21.15.117:0/3516084033 conn(0x7fe7ac0fe3f0 msgr2=0x7fe7ac100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.760+0000 7fe7b315f640 1 -- 172.21.15.117:0/3516084033 shutdown_connections 2024-09-21T05:58:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.760+0000 7fe7b315f640 1 -- 172.21.15.117:0/3516084033 wait complete. 2024-09-21T05:58:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.761+0000 7fe7b315f640 1 Processor -- start 2024-09-21T05:58:21.763 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.761+0000 7fe7b315f640 1 -- start start 2024-09-21T05:58:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.762+0000 7fe7b315f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7ac102590 0x7fe7ac1085f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.762+0000 7fe7b315f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7ac102ee0 0x7fe7ac1048c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.762+0000 7fe7b315f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7ac110140 0x7fe7ac104ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.762+0000 7fe7b315f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ac115d30 con 0x7fe7ac102590 2024-09-21T05:58:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.762+0000 7fe7b315f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ac115bb0 con 0x7fe7ac110140 2024-09-21T05:58:21.764 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.762+0000 7fe7b315f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ac115eb0 con 0x7fe7ac102ee0 2024-09-21T05:58:21.765 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.762+0000 7fe7b0efb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7ac102590 0x7fe7ac1085f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:21.765 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.762+0000 7fe7b0efb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7ac102590 0x7fe7ac1085f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44446/0 (socket says 172.21.15.117:44446) 2024-09-21T05:58:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.762+0000 7fe7b0efb640 1 -- 172.21.15.117:0/2235697507 learned_addr learned my addr 172.21.15.117:0/2235697507 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.762+0000 7fe7b16fc640 1 --2- 172.21.15.117:0/2235697507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7ac110140 0x7fe7ac104ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.763+0000 7fe7b0efb640 1 -- 172.21.15.117:0/2235697507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7ac102ee0 msgr2=0x7fe7ac1048c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T05:58:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.763+0000 7fe7b0efb640 1 --2- 172.21.15.117:0/2235697507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7ac102ee0 0x7fe7ac1048c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.763+0000 7fe7b0efb640 1 -- 172.21.15.117:0/2235697507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7ac110140 msgr2=0x7fe7ac104ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.763+0000 7fe7b0efb640 1 --2- 172.21.15.117:0/2235697507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7ac110140 0x7fe7ac104ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.763+0000 7fe7b0efb640 1 -- 172.21.15.117:0/2235697507 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ac1acc90 con 0x7fe7ac102590 2024-09-21T05:58:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.763+0000 7fe7b0efb640 1 --2- 172.21.15.117:0/2235697507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7ac102590 0x7fe7ac1085f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe7a000b450 tx=0x7fe7a000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.764+0000 7fe7a9ffb640 1 -- 172.21.15.117:0/2235697507 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a0015020 con 0x7fe7ac102590 2024-09-21T05:58:21.767 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.764+0000 7fe7a9ffb640 1 -- 172.21.15.117:0/2235697507 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7a0011d40 con 0x7fe7ac102590 2024-09-21T05:58:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.764+0000 7fe7a9ffb640 1 -- 172.21.15.117:0/2235697507 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a000f340 con 0x7fe7ac102590 2024-09-21T05:58:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.764+0000 7fe7b315f640 1 -- 172.21.15.117:0/2235697507 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7ac1acec0 con 0x7fe7ac102590 2024-09-21T05:58:21.768 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.764+0000 7fe7b315f640 1 -- 172.21.15.117:0/2235697507 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7ac1ad3b0 con 0x7fe7ac102590 2024-09-21T05:58:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.769+0000 7fe7ba0c8740 1 Processor -- start 2024-09-21T05:58:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.769+0000 7fe7a9ffb640 1 -- 172.21.15.117:0/2235697507 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fe7a0018020 con 0x7fe7ac102590 2024-09-21T05:58:21.771 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.769+0000 7fe7ba0c8740 1 -- start start 2024-09-21T05:58:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.770+0000 7fe7ba0c8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c62361a1b0 0x55c6239bf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.770+0000 7fe7ba0c8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6235f12f0 0x55c6239bfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.770+0000 7fe7ba0c8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6235f94d0 0x55c6239c0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:21.772 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.770+0000 7fe7ba0c8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c6239346f0 con 0x55c62361a1b0 2024-09-21T05:58:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.770+0000 7fe7ba0c8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c623923b20 con 0x55c6235f12f0 2024-09-21T05:58:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.770+0000 7fe7ba0c8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c623927930 con 0x55c6235f94d0 2024-09-21T05:58:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.770+0000 7fe7b16fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c62361a1b0 0x55c6239bf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.770+0000 7fe7b16fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c62361a1b0 0x55c6239bf9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44448/0 (socket says 172.21.15.117:44448) 2024-09-21T05:58:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.770+0000 7fe7b16fc640 1 -- 172.21.15.117:0/3065805562 learned_addr learned my addr 172.21.15.117:0/3065805562 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:21.773 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.771+0000 7fe7abfff640 1 --2- 172.21.15.117:0/3065805562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6235f94d0 0x55c6239c0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.771+0000 7fe7a9ffb640 1 --2- 172.21.15.117:0/2235697507 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe79c0b6700 0x7fe79c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.771+0000 7fe7b16fc640 1 -- 172.21.15.117:0/3065805562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6235f94d0 msgr2=0x55c6239c0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.771+0000 7fe7b16fc640 1 --2- 172.21.15.117:0/3065805562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6235f94d0 0x55c6239c0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.771+0000 7fe7b16fc640 1 -- 172.21.15.117:0/3065805562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6235f12f0 msgr2=0x55c6239bfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.772+0000 7fe7a9ffb640 1 -- 172.21.15.117:0/2235697507 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fe7a00e7060 con 0x7fe7ac102590 2024-09-21T05:58:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.772+0000 7fe7b0efb640 1 --2- 172.21.15.117:0/3065805562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6235f12f0 0x55c6239bfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.772+0000 7fe7b16fc640 1 --2- 172.21.15.117:0/3065805562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6235f12f0 0x55c6239bfef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.772+0000 7fe7b16fc640 1 -- 172.21.15.117:0/3065805562 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6238e9530 con 0x55c62361a1b0 2024-09-21T05:58:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.772+0000 7fe7b16fc640 1 --2- 172.21.15.117:0/2235697507 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe79c0b6700 0x7fe79c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.773+0000 7fe7b16fc640 1 --2- 172.21.15.117:0/3065805562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c62361a1b0 0x55c6239bf9b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4006010 tx=0x7fe7a402fdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfbe65d77b2a1c60 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.774+0000 7fe796ffd640 1 -- 172.21.15.117:0/3065805562 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4034070 con 0x55c62361a1b0 2024-09-21T05:58:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.774+0000 7fe796ffd640 1 -- 172.21.15.117:0/3065805562 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7a4003cb0 con 0x55c62361a1b0 2024-09-21T05:58:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.774+0000 7fe796ffd640 1 -- 172.21.15.117:0/3065805562 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a403a430 con 0x55c62361a1b0 2024-09-21T05:58:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.774+0000 7fe7b16fc640 1 --2- 172.21.15.117:0/2235697507 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe79c0b6700 0x7fe79c0b8c10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe7a404b000 tx=0x7fe7a404c000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.776+0000 7fe7ba0c8740 1 -- 172.21.15.117:0/3065805562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c62361a1b0 msgr2=0x55c6239bf9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.776+0000 7fe7ba0c8740 1 --2- 172.21.15.117:0/3065805562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c62361a1b0 0x55c6239bf9b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4006010 tx=0x7fe7a402fdb0 comp rx=0 tx=0).stop 2024-09-21T05:58:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.776+0000 7fe7ba0c8740 1 -- 172.21.15.117:0/3065805562 shutdown_connections 2024-09-21T05:58:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.776+0000 7fe7ba0c8740 1 --2- 172.21.15.117:0/3065805562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6235f94d0 0x55c6239c0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.776+0000 7fe7ba0c8740 1 --2- 172.21.15.117:0/3065805562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6235f12f0 0x55c6239bfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.776+0000 7fe7ba0c8740 1 --2- 172.21.15.117:0/3065805562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c62361a1b0 0x55c6239bf9b0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.777+0000 7fe7ba0c8740 1 -- 172.21.15.117:0/3065805562 >> 172.21.15.117:0/3065805562 conn(0x55c62360e300 msgr2=0x55c6239bd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.777+0000 7fe7ba0c8740 1 -- 172.21.15.117:0/3065805562 shutdown_connections 2024-09-21T05:58:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.777+0000 7fe7ba0c8740 1 -- 172.21.15.117:0/3065805562 wait complete. 2024-09-21T05:58:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.778+0000 7fe7ba0c8740 1 -- start start 2024-09-21T05:58:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.778+0000 7fe7ba0c8740 1 Processor -- start 2024-09-21T05:58:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.780+0000 7fe7ba0c8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6235f12f0 0x55c6239cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.781+0000 7fe7ba0c8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6235f94d0 0x55c6239cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.781+0000 7fe7abfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6235f94d0 0x55c6239cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.781+0000 7fe7abfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6235f94d0 0x55c6239cb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44458/0 (socket says 172.21.15.117:44458) 2024-09-21T05:58:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.782+0000 7fe7abfff640 1 -- 172.21.15.117:0/2995892277 learned_addr learned my addr 172.21.15.117:0/2995892277 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.782+0000 7fe7ba0c8740 1 --2- 172.21.15.117:0/2995892277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c62361a1b0 0x55c6239d7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.782+0000 7fe7ba0c8740 1 -- 172.21.15.117:0/2995892277 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c62385bab0 con 0x55c6235f94d0 2024-09-21T05:58:21.785 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.783+0000 7fe7ba0c8740 1 -- 172.21.15.117:0/2995892277 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c6238c9730 con 0x55c62361a1b0 2024-09-21T05:58:21.785 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.783+0000 7fe7ba0c8740 1 -- 172.21.15.117:0/2995892277 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c6239ce130 con 0x55c6235f12f0 2024-09-21T05:58:21.785 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.783+0000 7fe7ba0c8740 10 client.0 _ll_register_callbacks cb 0x55c62391d6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:58:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.784+0000 7fe7abfff640 1 -- 172.21.15.117:0/2995892277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6235f12f0 msgr2=0x55c6239cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.784+0000 7fe7abfff640 1 --2- 172.21.15.117:0/2995892277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6235f12f0 0x55c6239cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.784+0000 7fe7abfff640 1 -- 172.21.15.117:0/2995892277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c62361a1b0 msgr2=0x55c6239d7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.784+0000 7fe7abfff640 1 --2- 172.21.15.117:0/2995892277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c62361a1b0 0x55c6239d7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.784+0000 7fe7abfff640 1 -- 172.21.15.117:0/2995892277 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c623685790 con 0x55c6235f94d0 2024-09-21T05:58:21.787 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.784+0000 7fe7abfff640 1 --2- 172.21.15.117:0/2995892277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6235f94d0 0x55c6239cb5f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe79800af00 tx=0x7fe79800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dafcc083239661a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:21.787 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.784+0000 7fe78bfff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:58:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.785+0000 7fe78bfff640 1 -- 172.21.15.117:0/2995892277 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe798015070 con 0x55c6235f94d0 2024-09-21T05:58:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.785+0000 7fe78bfff640 1 -- 172.21.15.117:0/2995892277 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fe7980115f0 con 0x55c6235f94d0 2024-09-21T05:58:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.786+0000 7fe7ba0c8740 1 -- 172.21.15.117:0/2995892277 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7a4003cb0 con 0x55c6235f94d0 2024-09-21T05:58:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.786+0000 7fe78bfff640 1 -- 172.21.15.117:0/2995892277 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe79801f050 con 0x55c6235f94d0 2024-09-21T05:58:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.786+0000 7fe7ba0c8740 10 client.15813 fetch_fsmap learned FSMap version 27 2024-09-21T05:58:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.786+0000 7fe7ba0c8740 1 -- 172.21.15.117:0/2995892277 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c6236fe620 con 0x55c6235f94d0 2024-09-21T05:58:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.787+0000 7fe78bfff640 1 -- 172.21.15.117:0/2995892277 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe798011ca0 con 0x55c6235f94d0 2024-09-21T05:58:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.787+0000 7fe78bfff640 1 -- 172.21.15.117:0/2995892277 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe79801ac20 con 0x55c6235f94d0 2024-09-21T05:58:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.788+0000 7fe7ba0c8740 10 client.15813 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:58:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.788+0000 7fe7ba0c8740 10 client.15813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T05:58:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.788+0000 7fe7ba0c8740 20 client.15813 populate_metadata read hostname 'smithi117' 2024-09-21T05:58:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.788+0000 7fe7ba0c8740 1 --2- 172.21.15.117:0/2995892277 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c6235d8d80 0x55c6239dd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.788+0000 7fe7ba0c8740 4 client.15813 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:58:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.788+0000 7fe7ba0c8740 1 -- 172.21.15.117:0/2995892277 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe7a4003cb0 con 0x55c6235d8d80 2024-09-21T05:58:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.789+0000 7fe7b0efb640 1 --2- 172.21.15.117:0/2995892277 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c6235d8d80 0x55c6239dd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.795+0000 7fe78bfff640 10 client.15813 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:58:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.795+0000 7fe7b0efb640 1 --2- 172.21.15.117:0/2995892277 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c6235d8d80 0x55c6239dd2e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3248d703cf4dd17 server_cookie=b742c4f5241958a3 in_seq=0 out_seq=0 2024-09-21T05:58:21.799 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.797+0000 7fe78bfff640 1 -- 172.21.15.117:0/2995892277 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fe7a4003cb0 con 0x55c6235d8d80 2024-09-21T05:58:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.797+0000 7fe78bfff640 10 client.15813 handle_command_reply: tid=0 2024-09-21T05:58:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.797+0000 7fe7ba0c8740 1 client.15813 shutdown 2024-09-21T05:58:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.798+0000 7fe7ba0c8740 1 -- 172.21.15.117:0/2995892277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6235f94d0 msgr2=0x55c6239cb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.798+0000 7fe7ba0c8740 1 --2- 172.21.15.117:0/2995892277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6235f94d0 0x55c6239cb5f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe79800af00 tx=0x7fe79800c520 comp rx=0 tx=0).stop 2024-09-21T05:58:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.798+0000 7fe7ba0c8740 1 -- 172.21.15.117:0/2995892277 shutdown_connections 2024-09-21T05:58:21.800 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.798+0000 7fe7ba0c8740 1 --2- 172.21.15.117:0/2995892277 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c6235d8d80 0x55c6239dd2e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.798+0000 7fe7ba0c8740 1 --2- 172.21.15.117:0/2995892277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6235f94d0 0x55c6239cb5f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:21.801 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.798+0000 7fe78bfff640 0 client.15813 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:58:21.802 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.798+0000 7fe7ba0c8740 1 --2- 172.21.15.117:0/2995892277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c62361a1b0 0x55c6239d7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:21.802 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.798+0000 7fe7ba0c8740 1 --2- 172.21.15.117:0/2995892277 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6235f12f0 0x55c6239cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:21.802 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.798+0000 7fe7ba0c8740 1 -- 172.21.15.117:0/2995892277 >> 172.21.15.117:0/2995892277 conn(0x55c62360e300 msgr2=0x55c6239c09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:21.802 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.798+0000 7fe7ba0c8740 1 -- 172.21.15.117:0/2995892277 shutdown_connections 2024-09-21T05:58:21.802 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.798+0000 7fe7ba0c8740 1 -- 172.21.15.117:0/2995892277 wait complete. 2024-09-21T05:58:21.802 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.798+0000 7fe7ba0c8740 20 client.15813 trim_cache size 0 max 16384 2024-09-21T05:58:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.829+0000 7fe7ba0c8740 1 Processor -- start 2024-09-21T05:58:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.829+0000 7fe7ba0c8740 1 -- start start 2024-09-21T05:58:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.829+0000 7fe7ba0c8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6235f12f0 0x55c6239c67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.829+0000 7fe7ba0c8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6235f94d0 0x55c6239c13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:21.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.829+0000 7fe7ba0c8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c62361a1b0 0x55c6239ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.830+0000 7fe7ba0c8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ac1b5c30 con 0x55c62361a1b0 2024-09-21T05:58:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.830+0000 7fe7ba0c8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ac1b7770 con 0x55c6235f12f0 2024-09-21T05:58:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.830+0000 7fe7b16fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c62361a1b0 0x55c6239ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.830+0000 7fe7b16fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c62361a1b0 0x55c6239ccce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44466/0 (socket says 172.21.15.117:44466) 2024-09-21T05:58:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.830+0000 7fe7b16fc640 1 -- 172.21.15.117:0/3582310914 learned_addr learned my addr 172.21.15.117:0/3582310914 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.830+0000 7fe7ba0c8740 1 -- 172.21.15.117:0/3582310914 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ac1b6300 con 0x55c6235f94d0 2024-09-21T05:58:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.831+0000 7fe7b16fc640 1 -- 172.21.15.117:0/3582310914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6235f94d0 msgr2=0x55c6239c13a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T05:58:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.831+0000 7fe7b16fc640 1 --2- 172.21.15.117:0/3582310914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6235f94d0 0x55c6239c13a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.831+0000 7fe7b16fc640 1 -- 172.21.15.117:0/3582310914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6235f12f0 msgr2=0x55c6239c67f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T05:58:21.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.831+0000 7fe7b16fc640 1 --2- 172.21.15.117:0/3582310914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6235f12f0 0x55c6239c67f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:21.834 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.831+0000 7fe7b16fc640 1 -- 172.21.15.117:0/3582310914 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6239c55b0 con 0x55c62361a1b0 2024-09-21T05:58:21.834 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.831+0000 7fe7b16fc640 1 --2- 172.21.15.117:0/3582310914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c62361a1b0 0x55c6239ccce0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe7a402b0d0 tx=0x7fe7a4003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec3f88f8cd89f5f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.833+0000 7fe7967fc640 1 -- 172.21.15.117:0/3582310914 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4034040 con 0x55c62361a1b0 2024-09-21T05:58:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.833+0000 7fe7967fc640 1 -- 172.21.15.117:0/3582310914 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7a403a430 con 0x55c62361a1b0 2024-09-21T05:58:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.833+0000 7fe7967fc640 1 -- 172.21.15.117:0/3582310914 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4052470 con 0x55c62361a1b0 2024-09-21T05:58:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.835+0000 7fe7ba0c8740 1 -- 172.21.15.117:0/3582310914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c62361a1b0 msgr2=0x55c6239ccce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.835+0000 7fe7ba0c8740 1 --2- 172.21.15.117:0/3582310914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c62361a1b0 0x55c6239ccce0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe7a402b0d0 tx=0x7fe7a4003a90 comp rx=0 tx=0).stop 2024-09-21T05:58:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.835+0000 7fe7ba0c8740 1 -- 172.21.15.117:0/3582310914 shutdown_connections 2024-09-21T05:58:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.835+0000 7fe7ba0c8740 1 --2- 172.21.15.117:0/3582310914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c62361a1b0 0x55c6239ccce0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.835+0000 7fe7ba0c8740 1 --2- 172.21.15.117:0/3582310914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6235f94d0 0x55c6239c13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.835+0000 7fe7ba0c8740 1 --2- 172.21.15.117:0/3582310914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c6235f12f0 0x55c6239c67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.835+0000 7fe7ba0c8740 1 -- 172.21.15.117:0/3582310914 >> 172.21.15.117:0/3582310914 conn(0x55c62360e300 msgr2=0x55c6239cc5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.836+0000 7fe7ba0c8740 1 -- 172.21.15.117:0/3582310914 shutdown_connections 2024-09-21T05:58:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.836+0000 7fe7ba0c8740 1 -- 172.21.15.117:0/3582310914 wait complete. 2024-09-21T05:58:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.836+0000 7fe7ba0c8740 1 -- start start 2024-09-21T05:58:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.837+0000 7fe7ba0c8740 1 Processor -- start 2024-09-21T05:58:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.837+0000 7fe7ba0c8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6235f12f0 0x55c6239d3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.837+0000 7fe7ba0c8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6235f94d0 0x55c6239cee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.837+0000 7fe7ba0c8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c62361a1b0 0x55c6239cf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.837+0000 7fe7ba0c8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ac1b70a0 con 0x55c6235f94d0 2024-09-21T05:58:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.837+0000 7fe7ba0c8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ac1b69d0 con 0x55c62361a1b0 2024-09-21T05:58:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.837+0000 7fe7ba0c8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ac1b3c10 con 0x55c6235f12f0 2024-09-21T05:58:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.838+0000 7fe7ba0c8740 10 client.0 _ll_register_callbacks cb 0x7fe798006ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:58:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.838+0000 7fe7abfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6235f94d0 0x55c6239cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.838+0000 7fe7abfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6235f94d0 0x55c6239cee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44468/0 (socket says 172.21.15.117:44468) 2024-09-21T05:58:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.838+0000 7fe7abfff640 1 -- 172.21.15.117:0/233942578 learned_addr learned my addr 172.21.15.117:0/233942578 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.839+0000 7fe7abfff640 1 -- 172.21.15.117:0/233942578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6235f12f0 msgr2=0x55c6239d3980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T05:58:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.839+0000 7fe7abfff640 1 --2- 172.21.15.117:0/233942578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6235f12f0 0x55c6239d3980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.839+0000 7fe7abfff640 1 -- 172.21.15.117:0/233942578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c62361a1b0 msgr2=0x55c6239cf360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.839+0000 7fe7abfff640 1 --2- 172.21.15.117:0/233942578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c62361a1b0 0x55c6239cf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.839+0000 7fe7abfff640 1 -- 172.21.15.117:0/233942578 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6239cf8d0 con 0x55c6235f94d0 2024-09-21T05:58:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.839+0000 7fe7abfff640 1 --2- 172.21.15.117:0/233942578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6235f94d0 0x55c6239cee20 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe798019a40 tx=0x7fe79801a670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f290abe0a5c95b57 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.839+0000 7fe78bfff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:58:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.839+0000 7fe78bfff640 1 -- 172.21.15.117:0/233942578 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe798015040 con 0x55c6235f94d0 2024-09-21T05:58:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.840+0000 7fe78bfff640 1 -- 172.21.15.117:0/233942578 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fe798004440 con 0x55c6235f94d0 2024-09-21T05:58:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.841+0000 7fe7ba0c8740 1 -- 172.21.15.117:0/233942578 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7a403a430 con 0x55c6235f94d0 2024-09-21T05:58:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.841+0000 7fe78bfff640 1 -- 172.21.15.117:0/233942578 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe79801a790 con 0x55c6235f94d0 2024-09-21T05:58:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.841+0000 7fe7ba0c8740 10 client.15819 fetch_fsmap learned FSMap version 27 2024-09-21T05:58:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.841+0000 7fe7ba0c8740 1 -- 172.21.15.117:0/233942578 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c623a47a60 con 0x55c6235f94d0 2024-09-21T05:58:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.843+0000 7fe78bfff640 1 -- 172.21.15.117:0/233942578 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7a403a430 con 0x55c6235f94d0 2024-09-21T05:58:21.847 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.843+0000 7fe78bfff640 1 -- 172.21.15.117:0/233942578 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe7980039f0 con 0x55c6235f94d0 2024-09-21T05:58:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.844+0000 7fe7ba0c8740 10 client.15819 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:58:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.844+0000 7fe7ba0c8740 10 client.15819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T05:58:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.844+0000 7fe7ba0c8740 20 client.15819 populate_metadata read hostname 'smithi117' 2024-09-21T05:58:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.844+0000 7fe7ba0c8740 1 --2- 172.21.15.117:0/233942578 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c6235ae1c0 0x55c623a4a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.844+0000 7fe7ba0c8740 4 client.15819 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T05:58:21.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.844+0000 7fe7ba0c8740 1 -- 172.21.15.117:0/233942578 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c623936640 con 0x55c6235ae1c0 2024-09-21T05:58:21.849 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.845+0000 7fe7b0efb640 1 --2- 172.21.15.117:0/233942578 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c6235ae1c0 0x55c623a4a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:21.851 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.848+0000 7fe7b0efb640 1 --2- 172.21.15.117:0/233942578 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c6235ae1c0 0x55c623a4a130 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=178fc0135ed7d9f5 server_cookie=661a882017556a3a in_seq=0 out_seq=0 2024-09-21T05:58:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.848+0000 7fe78bfff640 10 client.15819 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:58:21.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:21 smithi192 ceph-mon[34700]: pgmap v358: 97 pgs: 97 active+clean; 33 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.7 MiB/s wr, 10 op/s 2024-09-21T05:58:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.985+0000 7fe78bfff640 1 -- 172.21.15.117:0/233942578 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c623936640 con 0x55c6235ae1c0 2024-09-21T05:58:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.985+0000 7fe78bfff640 10 client.15819 handle_command_reply: tid=0 2024-09-21T05:58:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.986+0000 7fe7ba0c8740 1 client.15819 shutdown 2024-09-21T05:58:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.986+0000 7fe7ba0c8740 1 -- 172.21.15.117:0/233942578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6235f94d0 msgr2=0x55c6239cee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.986+0000 7fe7ba0c8740 1 --2- 172.21.15.117:0/233942578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6235f94d0 0x55c6239cee20 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe798019a40 tx=0x7fe79801a670 comp rx=0 tx=0).stop 2024-09-21T05:58:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.986+0000 7fe7ba0c8740 1 -- 172.21.15.117:0/233942578 shutdown_connections 2024-09-21T05:58:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.986+0000 7fe7ba0c8740 1 --2- 172.21.15.117:0/233942578 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c6235ae1c0 0x55c623a4a130 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.987+0000 7fe7ba0c8740 1 --2- 172.21.15.117:0/233942578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c6235f94d0 0x55c6239cee20 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.987+0000 7fe7ba0c8740 1 --2- 172.21.15.117:0/233942578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c62361a1b0 0x55c6239cf360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.987+0000 7fe7ba0c8740 1 --2- 172.21.15.117:0/233942578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c6235f12f0 0x55c6239d3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.987+0000 7fe7ba0c8740 1 -- 172.21.15.117:0/233942578 >> 172.21.15.117:0/233942578 conn(0x55c62360e300 msgr2=0x55c6239c1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.987+0000 7fe78bfff640 0 client.15819 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:58:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.987+0000 7fe7ba0c8740 1 -- 172.21.15.117:0/233942578 shutdown_connections 2024-09-21T05:58:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.987+0000 7fe7ba0c8740 1 -- 172.21.15.117:0/233942578 wait complete. 2024-09-21T05:58:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.987+0000 7fe7ba0c8740 20 client.15819 trim_cache size 0 max 16384 2024-09-21T05:58:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T05:58:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T05:58:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T05:58:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T05:58:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.988+0000 7fe7b315f640 1 -- 172.21.15.117:0/2235697507 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe79c0b6700 msgr2=0x7fe79c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.988+0000 7fe7b315f640 1 --2- 172.21.15.117:0/2235697507 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe79c0b6700 0x7fe79c0b8c10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe7a404b000 tx=0x7fe7a404c000 comp rx=0 tx=0).stop 2024-09-21T05:58:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.988+0000 7fe7b315f640 1 -- 172.21.15.117:0/2235697507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7ac102590 msgr2=0x7fe7ac1085f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.988+0000 7fe7b315f640 1 --2- 172.21.15.117:0/2235697507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7ac102590 0x7fe7ac1085f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe7a000b450 tx=0x7fe7a000b910 comp rx=0 tx=0).stop 2024-09-21T05:58:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.988+0000 7fe7b315f640 1 -- 172.21.15.117:0/2235697507 shutdown_connections 2024-09-21T05:58:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.988+0000 7fe7b315f640 1 --2- 172.21.15.117:0/2235697507 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe79c0b6700 0x7fe79c0b8c10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.988+0000 7fe7b315f640 1 --2- 172.21.15.117:0/2235697507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7ac110140 0x7fe7ac104ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.988+0000 7fe7b315f640 1 --2- 172.21.15.117:0/2235697507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7ac102ee0 0x7fe7ac1048c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.989+0000 7fe7b315f640 1 --2- 172.21.15.117:0/2235697507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7ac102590 0x7fe7ac1085f0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.989+0000 7fe7b315f640 1 -- 172.21.15.117:0/2235697507 >> 172.21.15.117:0/2235697507 conn(0x7fe7ac0fe3f0 msgr2=0x7fe7ac1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.989+0000 7fe7b315f640 1 -- 172.21.15.117:0/2235697507 shutdown_connections 2024-09-21T05:58:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:21.989+0000 7fe7b315f640 1 -- 172.21.15.117:0/2235697507 wait complete. 2024-09-21T05:58:22.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:21 smithi117 ceph-mon[30947]: pgmap v358: 97 pgs: 97 active+clean; 33 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.7 MiB/s wr, 10 op/s 2024-09-21T05:58:22.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:21 smithi138 ceph-mon[34391]: pgmap v358: 97 pgs: 97 active+clean; 33 MiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.7 MiB/s wr, 10 op/s 2024-09-21T05:58:23.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:23 smithi192 ceph-mon[34700]: pgmap v359: 97 pgs: 97 active+clean; 777 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 65 MiB/s wr, 28 op/s 2024-09-21T05:58:23.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:58:23.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:58:23.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:58:23.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:58:23.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:58:23.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:58:23.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:23 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:58:24.029 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:23 smithi117 ceph-mon[30947]: pgmap v359: 97 pgs: 97 active+clean; 777 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 65 MiB/s wr, 28 op/s 2024-09-21T05:58:24.029 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:58:24.029 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:58:24.029 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:58:24.029 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:58:24.029 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:58:24.029 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:58:24.029 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:23 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:58:24.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:23 smithi138 ceph-mon[34391]: pgmap v359: 97 pgs: 97 active+clean; 777 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 65 MiB/s wr, 28 op/s 2024-09-21T05:58:24.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:58:24.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:58:24.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:58:24.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:58:24.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:58:24.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:58:24.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:23 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:58:24.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:24 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:58:24.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:24 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:58:24.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:24 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:58:25.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:24 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:58:25.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:24 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:58:25.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:24 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:58:25.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:24 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:58:25.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:24 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:58:25.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:24 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:58:25.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:25 smithi192 ceph-mon[34700]: pgmap v360: 97 pgs: 97 active+clean; 2.2 GiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 187 MiB/s wr, 138 op/s 2024-09-21T05:58:26.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:25 smithi117 ceph-mon[30947]: pgmap v360: 97 pgs: 97 active+clean; 2.2 GiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 187 MiB/s wr, 138 op/s 2024-09-21T05:58:26.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:25 smithi138 ceph-mon[34391]: pgmap v360: 97 pgs: 97 active+clean; 2.2 GiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 187 MiB/s wr, 138 op/s 2024-09-21T05:58:27.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:27 smithi192 ceph-mon[34700]: pgmap v361: 97 pgs: 97 active+clean; 3.8 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 324 MiB/s wr, 268 op/s 2024-09-21T05:58:28.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:27 smithi117 ceph-mon[30947]: pgmap v361: 97 pgs: 97 active+clean; 3.8 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 324 MiB/s wr, 268 op/s 2024-09-21T05:58:28.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:27 smithi138 ceph-mon[34391]: pgmap v361: 97 pgs: 97 active+clean; 3.8 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 324 MiB/s wr, 268 op/s 2024-09-21T05:58:29.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:29 smithi192 ceph-mon[34700]: pgmap v362: 97 pgs: 97 active+clean; 6.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 549 MiB/s wr, 431 op/s 2024-09-21T05:58:30.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:29 smithi117 ceph-mon[30947]: pgmap v362: 97 pgs: 97 active+clean; 6.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 549 MiB/s wr, 431 op/s 2024-09-21T05:58:30.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:29 smithi138 ceph-mon[34391]: pgmap v362: 97 pgs: 97 active+clean; 6.4 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 549 MiB/s wr, 431 op/s 2024-09-21T05:58:31.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:31 smithi192 ceph-mon[34700]: pgmap v363: 97 pgs: 97 active+clean; 6.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 588 MiB/s wr, 462 op/s 2024-09-21T05:58:32.029 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:31 smithi117 ceph-mon[30947]: pgmap v363: 97 pgs: 97 active+clean; 6.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 588 MiB/s wr, 462 op/s 2024-09-21T05:58:32.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:31 smithi138 ceph-mon[34391]: pgmap v363: 97 pgs: 97 active+clean; 6.9 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 588 MiB/s wr, 462 op/s 2024-09-21T05:58:33.821 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T05:58:33.954 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:33 smithi117 ceph-mon[30947]: pgmap v364: 97 pgs: 97 active+clean; 8.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 746 MiB/s wr, 743 op/s 2024-09-21T05:58:33.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:33 smithi192 ceph-mon[34700]: pgmap v364: 97 pgs: 97 active+clean; 8.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 746 MiB/s wr, 743 op/s 2024-09-21T05:58:34.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:33 smithi138 ceph-mon[34391]: pgmap v364: 97 pgs: 97 active+clean; 8.4 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 746 MiB/s wr, 743 op/s 2024-09-21T05:58:34.687 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:58:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.192+0000 7f88e3230640 1 -- 172.21.15.117:0/2174978901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88dc1029b0 msgr2=0x7f88dc102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.192+0000 7f88e3230640 1 --2- 172.21.15.117:0/2174978901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88dc1029b0 0x7f88dc102dc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f88d8009900 tx=0x7f88d802a440 comp rx=0 tx=0).stop 2024-09-21T05:58:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.192+0000 7f88e3230640 1 -- 172.21.15.117:0/2174978901 shutdown_connections 2024-09-21T05:58:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.192+0000 7f88e3230640 1 --2- 172.21.15.117:0/2174978901 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88dc108c30 0x7f88dc10f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:35.195 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.192+0000 7f88e3230640 1 --2- 172.21.15.117:0/2174978901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88dc1082e0 0x7f88dc1086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.192+0000 7f88e3230640 1 --2- 172.21.15.117:0/2174978901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88dc1029b0 0x7f88dc102dc0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.192+0000 7f88e3230640 1 -- 172.21.15.117:0/2174978901 >> 172.21.15.117:0/2174978901 conn(0x7f88dc07a740 msgr2=0x7f88dc07ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.192+0000 7f88e3230640 1 -- 172.21.15.117:0/2174978901 shutdown_connections 2024-09-21T05:58:35.196 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.192+0000 7f88e3230640 1 -- 172.21.15.117:0/2174978901 wait complete. 2024-09-21T05:58:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.194+0000 7f88e3230640 1 Processor -- start 2024-09-21T05:58:35.197 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.194+0000 7f88e3230640 1 -- start start 2024-09-21T05:58:35.198 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.194+0000 7f88e3230640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88dc1029b0 0x7f88dc100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:35.199 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.194+0000 7f88e3230640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88dc1082e0 0x7f88dc0ff550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:35.199 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.194+0000 7f88e3230640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88dc108c30 0x7f88dc0ffa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:35.199 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.194+0000 7f88e3230640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc112a20 con 0x7f88dc1082e0 2024-09-21T05:58:35.199 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.194+0000 7f88e3230640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc1128a0 con 0x7f88dc108c30 2024-09-21T05:58:35.199 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.194+0000 7f88e3230640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc112ba0 con 0x7f88dc1029b0 2024-09-21T05:58:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.195+0000 7f88d3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88dc1082e0 0x7f88dc0ff550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.195+0000 7f88d3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88dc1082e0 0x7f88dc0ff550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50422/0 (socket says 172.21.15.117:50422) 2024-09-21T05:58:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.195+0000 7f88d3fff640 1 -- 172.21.15.117:0/4226821243 learned_addr learned my addr 172.21.15.117:0/4226821243 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.195+0000 7f88d3fff640 1 -- 172.21.15.117:0/4226821243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88dc1029b0 msgr2=0x7f88dc100f70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T05:58:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.195+0000 7f88d3fff640 1 --2- 172.21.15.117:0/4226821243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88dc1029b0 0x7f88dc100f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.195+0000 7f88d3fff640 1 -- 172.21.15.117:0/4226821243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88dc108c30 msgr2=0x7f88dc0ffa90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T05:58:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.195+0000 7f88d3fff640 1 --2- 172.21.15.117:0/4226821243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88dc108c30 0x7f88dc0ffa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:35.200 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.195+0000 7f88d3fff640 1 -- 172.21.15.117:0/4226821243 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88dc1002b0 con 0x7f88dc1082e0 2024-09-21T05:58:35.201 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.197+0000 7f88d3fff640 1 --2- 172.21.15.117:0/4226821243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88dc1082e0 0x7f88dc0ff550 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f88cc00b450 tx=0x7f88cc00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.198+0000 7f88d1ffb640 1 -- 172.21.15.117:0/4226821243 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88cc015020 con 0x7f88dc1082e0 2024-09-21T05:58:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.198+0000 7f88d1ffb640 1 -- 172.21.15.117:0/4226821243 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88cc011d40 con 0x7f88dc1082e0 2024-09-21T05:58:35.202 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.198+0000 7f88d1ffb640 1 -- 172.21.15.117:0/4226821243 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88cc00f340 con 0x7f88dc1082e0 2024-09-21T05:58:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.198+0000 7f88e3230640 1 -- 172.21.15.117:0/4226821243 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88dc1acfc0 con 0x7f88dc1082e0 2024-09-21T05:58:35.203 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.198+0000 7f88e3230640 1 -- 172.21.15.117:0/4226821243 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88dc1ad430 con 0x7f88dc1082e0 2024-09-21T05:58:35.206 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.202+0000 7f88ea27b740 1 Processor -- start 2024-09-21T05:58:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.203+0000 7f88ea27b740 1 -- start start 2024-09-21T05:58:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.203+0000 7f88ea27b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55767bb0a1b0 0x55767beaf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.203+0000 7f88ea27b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55767bae12f0 0x55767beafef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.203+0000 7f88ea27b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55767bae94d0 0x55767beb0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.203+0000 7f88ea27b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55767be246f0 con 0x55767bae12f0 2024-09-21T05:58:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.203+0000 7f88ea27b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55767be13b20 con 0x55767bae94d0 2024-09-21T05:58:35.207 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.203+0000 7f88ea27b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55767be17930 con 0x55767bb0a1b0 2024-09-21T05:58:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.203+0000 7f88e0fcc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55767bae12f0 0x55767beafef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.203+0000 7f88e0fcc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55767bae12f0 0x55767beafef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50432/0 (socket says 172.21.15.117:50432) 2024-09-21T05:58:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.203+0000 7f88e0fcc640 1 -- 172.21.15.117:0/2450807608 learned_addr learned my addr 172.21.15.117:0/2450807608 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.204+0000 7f88e0fcc640 1 -- 172.21.15.117:0/2450807608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55767bb0a1b0 msgr2=0x55767beaf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.204+0000 7f88e0fcc640 1 --2- 172.21.15.117:0/2450807608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55767bb0a1b0 0x55767beaf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.204+0000 7f88e0fcc640 1 -- 172.21.15.117:0/2450807608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55767bae94d0 msgr2=0x55767beb0430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T05:58:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.204+0000 7f88e0fcc640 1 --2- 172.21.15.117:0/2450807608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55767bae94d0 0x55767beb0430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.204+0000 7f88e0fcc640 1 -- 172.21.15.117:0/2450807608 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55767bdd9530 con 0x55767bae12f0 2024-09-21T05:58:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.205+0000 7f88d1ffb640 1 -- 172.21.15.117:0/4226821243 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f88cc018020 con 0x7f88dc1082e0 2024-09-21T05:58:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.205+0000 7f88e0fcc640 1 --2- 172.21.15.117:0/2450807608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55767bae12f0 0x55767beafef0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f88d802aa30 tx=0x7f88d80040f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc7cc31e73ec648f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.206+0000 7f88c2ffd640 1 -- 172.21.15.117:0/2450807608 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d803e030 con 0x55767bae12f0 2024-09-21T05:58:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.206+0000 7f88d1ffb640 1 --2- 172.21.15.117:0/4226821243 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f88c40b67d0 0x7f88c40b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.206+0000 7f88c2ffd640 1 -- 172.21.15.117:0/2450807608 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88d8004260 con 0x55767bae12f0 2024-09-21T05:58:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.206+0000 7f88c2ffd640 1 -- 172.21.15.117:0/2450807608 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d8035040 con 0x55767bae12f0 2024-09-21T05:58:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.206+0000 7f88d1ffb640 1 -- 172.21.15.117:0/4226821243 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f88cc0e7070 con 0x7f88dc1082e0 2024-09-21T05:58:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.206+0000 7f88e17cd640 1 --2- 172.21.15.117:0/4226821243 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f88c40b67d0 0x7f88c40b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.208+0000 7f88ea27b740 1 -- 172.21.15.117:0/2450807608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55767bae12f0 msgr2=0x55767beafef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.208+0000 7f88e17cd640 1 --2- 172.21.15.117:0/4226821243 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f88c40b67d0 0x7f88c40b8ce0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f88d400aab0 tx=0x7f88d4009250 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.208+0000 7f88ea27b740 1 --2- 172.21.15.117:0/2450807608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55767bae12f0 0x55767beafef0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f88d802aa30 tx=0x7f88d80040f0 comp rx=0 tx=0).stop 2024-09-21T05:58:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.208+0000 7f88ea27b740 1 -- 172.21.15.117:0/2450807608 shutdown_connections 2024-09-21T05:58:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.208+0000 7f88ea27b740 1 --2- 172.21.15.117:0/2450807608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55767bae12f0 0x55767beafef0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.208+0000 7f88ea27b740 1 --2- 172.21.15.117:0/2450807608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55767bae94d0 0x55767beb0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.208+0000 7f88ea27b740 1 --2- 172.21.15.117:0/2450807608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55767bb0a1b0 0x55767beaf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.208+0000 7f88ea27b740 1 -- 172.21.15.117:0/2450807608 >> 172.21.15.117:0/2450807608 conn(0x55767bafe300 msgr2=0x55767bead8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.209+0000 7f88ea27b740 1 -- 172.21.15.117:0/2450807608 shutdown_connections 2024-09-21T05:58:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.209+0000 7f88ea27b740 1 -- 172.21.15.117:0/2450807608 wait complete. 2024-09-21T05:58:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.210+0000 7f88ea27b740 1 -- start start 2024-09-21T05:58:35.215 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.210+0000 7f88ea27b740 1 Processor -- start 2024-09-21T05:58:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.211+0000 7f88ea27b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55767bae94d0 0x55767bebb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.211+0000 7f88ea27b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55767bb0a1b0 0x55767bebb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.211+0000 7f88ea27b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55767bac8d80 0x55767bec73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.211+0000 7f88ea27b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55767bd4bab0 con 0x55767bac8d80 2024-09-21T05:58:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.211+0000 7f88ea27b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55767bdb9730 con 0x55767bb0a1b0 2024-09-21T05:58:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.211+0000 7f88e17cd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55767bac8d80 0x55767bec73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.211+0000 7f88ea27b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55767bebe0d0 con 0x55767bae94d0 2024-09-21T05:58:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.211+0000 7f88ea27b740 10 client.0 _ll_register_callbacks cb 0x55767be0d6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:58:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.211+0000 7f88e17cd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55767bac8d80 0x55767bec73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50444/0 (socket says 172.21.15.117:50444) 2024-09-21T05:58:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.211+0000 7f88e17cd640 1 -- 172.21.15.117:0/1508401497 learned_addr learned my addr 172.21.15.117:0/1508401497 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:35.217 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.212+0000 7f88e0fcc640 1 --2- 172.21.15.117:0/1508401497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55767bae94d0 0x55767bebb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:35.217 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.212+0000 7f88d3fff640 1 --2- 172.21.15.117:0/1508401497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55767bb0a1b0 0x55767bebb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:35.218 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.212+0000 7f88e17cd640 1 -- 172.21.15.117:0/1508401497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55767bae94d0 msgr2=0x55767bebb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:35.218 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.212+0000 7f88e17cd640 1 --2- 172.21.15.117:0/1508401497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55767bae94d0 0x55767bebb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:35.218 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.212+0000 7f88e17cd640 1 -- 172.21.15.117:0/1508401497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55767bb0a1b0 msgr2=0x55767bebb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:35.218 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.212+0000 7f88e17cd640 1 --2- 172.21.15.117:0/1508401497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55767bb0a1b0 0x55767bebb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:35.218 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.212+0000 7f88e17cd640 1 -- 172.21.15.117:0/1508401497 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55767bb75790 con 0x55767bac8d80 2024-09-21T05:58:35.218 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.212+0000 7f88e17cd640 1 --2- 172.21.15.117:0/1508401497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55767bac8d80 0x55767bec73c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f88d400da50 tx=0x7f88d4004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ea6bcfdaee8fc84 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:35.218 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.213+0000 7f88bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:58:35.218 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.213+0000 7f88bbfff640 1 -- 172.21.15.117:0/1508401497 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d4018d60 con 0x55767bac8d80 2024-09-21T05:58:35.218 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.213+0000 7f88bbfff640 1 -- 172.21.15.117:0/1508401497 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f88d4023480 con 0x55767bac8d80 2024-09-21T05:58:35.218 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.213+0000 7f88ea27b740 1 -- 172.21.15.117:0/1508401497 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88d8004260 con 0x55767bac8d80 2024-09-21T05:58:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.213+0000 7f88bbfff640 1 -- 172.21.15.117:0/1508401497 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f88d4023680 con 0x55767bac8d80 2024-09-21T05:58:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.214+0000 7f88ea27b740 10 client.15831 fetch_fsmap learned FSMap version 27 2024-09-21T05:58:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.214+0000 7f88ea27b740 1 -- 172.21.15.117:0/1508401497 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55767bbee620 con 0x55767bac8d80 2024-09-21T05:58:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.214+0000 7f88bbfff640 1 -- 172.21.15.117:0/1508401497 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88d8004260 con 0x55767bac8d80 2024-09-21T05:58:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.214+0000 7f88bbfff640 1 -- 172.21.15.117:0/1508401497 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f88d402d9c0 con 0x55767bac8d80 2024-09-21T05:58:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.215+0000 7f88ea27b740 10 client.15831 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:58:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.215+0000 7f88ea27b740 10 client.15831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T05:58:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.215+0000 7f88ea27b740 20 client.15831 populate_metadata read hostname 'smithi117' 2024-09-21T05:58:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.215+0000 7f88ea27b740 1 --2- 172.21.15.117:0/1508401497 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55767ba9e1c0 0x55767becd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.215+0000 7f88ea27b740 4 client.15831 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:58:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.215+0000 7f88ea27b740 1 -- 172.21.15.117:0/1508401497 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55767be26640 con 0x55767ba9e1c0 2024-09-21T05:58:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.216+0000 7f88e0fcc640 1 --2- 172.21.15.117:0/1508401497 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55767ba9e1c0 0x55767becd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.218+0000 7f88bbfff640 10 client.15831 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:58:35.223 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.218+0000 7f88e0fcc640 1 --2- 172.21.15.117:0/1508401497 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55767ba9e1c0 0x55767becd340 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd3916fb0946788d server_cookie=adc47d2346d7d48c in_seq=0 out_seq=0 2024-09-21T05:58:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.227+0000 7f88bbfff640 1 -- 172.21.15.117:0/1508401497 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55767be26640 con 0x55767ba9e1c0 2024-09-21T05:58:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.227+0000 7f88bbfff640 10 client.15831 handle_command_reply: tid=0 2024-09-21T05:58:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.228+0000 7f88ea27b740 1 client.15831 shutdown 2024-09-21T05:58:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.230+0000 7f88ea27b740 1 -- 172.21.15.117:0/1508401497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55767bac8d80 msgr2=0x55767bec73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.230+0000 7f88ea27b740 1 --2- 172.21.15.117:0/1508401497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55767bac8d80 0x55767bec73c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f88d400da50 tx=0x7f88d4004b40 comp rx=0 tx=0).stop 2024-09-21T05:58:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.230+0000 7f88ea27b740 1 -- 172.21.15.117:0/1508401497 shutdown_connections 2024-09-21T05:58:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.230+0000 7f88ea27b740 1 --2- 172.21.15.117:0/1508401497 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55767ba9e1c0 0x55767becd340 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.230+0000 7f88ea27b740 1 --2- 172.21.15.117:0/1508401497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55767bac8d80 0x55767bec73c0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.230+0000 7f88ea27b740 1 --2- 172.21.15.117:0/1508401497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55767bb0a1b0 0x55767bebb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.230+0000 7f88ea27b740 1 --2- 172.21.15.117:0/1508401497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55767bae94d0 0x55767bebb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.230+0000 7f88ea27b740 1 -- 172.21.15.117:0/1508401497 >> 172.21.15.117:0/1508401497 conn(0x55767bafe300 msgr2=0x55767beabea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.231+0000 7f88bbfff640 0 client.15831 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:58:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.232+0000 7f88ea27b740 1 -- 172.21.15.117:0/1508401497 shutdown_connections 2024-09-21T05:58:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.232+0000 7f88ea27b740 1 -- 172.21.15.117:0/1508401497 wait complete. 2024-09-21T05:58:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.242+0000 7f88ea27b740 20 client.15831 trim_cache size 0 max 16384 2024-09-21T05:58:35.338 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.330+0000 7f88ea27b740 1 Processor -- start 2024-09-21T05:58:35.338 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.331+0000 7f88ea27b740 1 -- start start 2024-09-21T05:58:35.338 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.331+0000 7f88ea27b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55767bac8d80 0x55767beba160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:35.338 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.331+0000 7f88ea27b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55767bae94d0 0x55767bead8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:35.338 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.331+0000 7f88ea27b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55767bb0a1b0 0x55767bec72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.331+0000 7f88ea27b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc1b5b90 con 0x55767bb0a1b0 2024-09-21T05:58:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.331+0000 7f88ea27b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc1b76d0 con 0x55767bae94d0 2024-09-21T05:58:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.331+0000 7f88ea27b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc1b6260 con 0x55767bac8d80 2024-09-21T05:58:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.332+0000 7f88d3fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55767bae94d0 0x55767bead8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.332+0000 7f88d3fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55767bae94d0 0x55767bead8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:44204/0 (socket says 172.21.15.117:44204) 2024-09-21T05:58:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.332+0000 7f88d3fff640 1 -- 172.21.15.117:0/1350648686 learned_addr learned my addr 172.21.15.117:0/1350648686 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.335+0000 7f88e0fcc640 1 --2- 172.21.15.117:0/1350648686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55767bac8d80 0x55767beba160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.335+0000 7f88d3fff640 1 -- 172.21.15.117:0/1350648686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55767bac8d80 msgr2=0x55767beba160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:35.339 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.335+0000 7f88d3fff640 1 --2- 172.21.15.117:0/1350648686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55767bac8d80 0x55767beba160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.335+0000 7f88d3fff640 1 -- 172.21.15.117:0/1350648686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55767bb0a1b0 msgr2=0x55767bec72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.335+0000 7f88d3fff640 1 --2- 172.21.15.117:0/1350648686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55767bb0a1b0 0x55767bec72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.335+0000 7f88d3fff640 1 -- 172.21.15.117:0/1350648686 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55767beacfc0 con 0x55767bae94d0 2024-09-21T05:58:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.336+0000 7f88d3fff640 1 --2- 172.21.15.117:0/1350648686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55767bae94d0 0x55767bead8d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f88cc0acbe0 tx=0x7f88cc0ac9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9d6e5994f9f08c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.355+0000 7f88c27fc640 1 -- 172.21.15.117:0/1350648686 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88cc0b19c0 con 0x55767bae94d0 2024-09-21T05:58:35.360 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.356+0000 7f88c27fc640 1 -- 172.21.15.117:0/1350648686 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88cc0b6900 con 0x55767bae94d0 2024-09-21T05:58:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.357+0000 7f88ea27b740 1 -- 172.21.15.117:0/1350648686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55767bae94d0 msgr2=0x55767bead8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.357+0000 7f88ea27b740 1 --2- 172.21.15.117:0/1350648686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55767bae94d0 0x55767bead8d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f88cc0acbe0 tx=0x7f88cc0ac9f0 comp rx=0 tx=0).stop 2024-09-21T05:58:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.358+0000 7f88ea27b740 1 -- 172.21.15.117:0/1350648686 shutdown_connections 2024-09-21T05:58:35.361 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.358+0000 7f88ea27b740 1 --2- 172.21.15.117:0/1350648686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55767bb0a1b0 0x55767bec72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.358+0000 7f88ea27b740 1 --2- 172.21.15.117:0/1350648686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55767bae94d0 0x55767bead8d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.358+0000 7f88ea27b740 1 --2- 172.21.15.117:0/1350648686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55767bac8d80 0x55767beba160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.358+0000 7f88ea27b740 1 -- 172.21.15.117:0/1350648686 >> 172.21.15.117:0/1350648686 conn(0x55767bafe300 msgr2=0x55767bead1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.360+0000 7f88ea27b740 1 -- 172.21.15.117:0/1350648686 shutdown_connections 2024-09-21T05:58:35.366 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.364+0000 7f88ea27b740 1 -- 172.21.15.117:0/1350648686 wait complete. 2024-09-21T05:58:35.367 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.364+0000 7f88ea27b740 1 -- start start 2024-09-21T05:58:35.367 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.365+0000 7f88ea27b740 1 Processor -- start 2024-09-21T05:58:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.365+0000 7f88ea27b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55767bac8d80 0x55767bf332a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.366+0000 7f88ea27b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55767bae94d0 0x55767bec3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.366+0000 7f88ea27b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55767bb0a1b0 0x55767bec3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.366+0000 7f88ea27b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc1b7000 con 0x55767bb0a1b0 2024-09-21T05:58:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.366+0000 7f88ea27b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc1b6930 con 0x55767bac8d80 2024-09-21T05:58:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.366+0000 7f88ea27b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc1b3b70 con 0x55767bae94d0 2024-09-21T05:58:35.369 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.366+0000 7f88ea27b740 10 client.0 _ll_register_callbacks cb 0x7f88d400a7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:58:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.367+0000 7f88e17cd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55767bb0a1b0 0x55767bec3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.367+0000 7f88e17cd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55767bb0a1b0 0x55767bec3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50452/0 (socket says 172.21.15.117:50452) 2024-09-21T05:58:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.367+0000 7f88e17cd640 1 -- 172.21.15.117:0/2738133 learned_addr learned my addr 172.21.15.117:0/2738133 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.367+0000 7f88e0fcc640 1 --2- 172.21.15.117:0/2738133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55767bac8d80 0x55767bf332a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:35.370 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.367+0000 7f88d3fff640 1 --2- 172.21.15.117:0/2738133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55767bae94d0 0x55767bec3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.368+0000 7f88e0fcc640 1 -- 172.21.15.117:0/2738133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55767bae94d0 msgr2=0x55767bec3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.368+0000 7f88e0fcc640 1 --2- 172.21.15.117:0/2738133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55767bae94d0 0x55767bec3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.368+0000 7f88e0fcc640 1 -- 172.21.15.117:0/2738133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55767bb0a1b0 msgr2=0x55767bec3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.368+0000 7f88e0fcc640 1 --2- 172.21.15.117:0/2738133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55767bb0a1b0 0x55767bec3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:35.371 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.368+0000 7f88e0fcc640 1 -- 172.21.15.117:0/2738133 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55767bec3da0 con 0x55767bac8d80 2024-09-21T05:58:35.375 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.372+0000 7f88e0fcc640 1 --2- 172.21.15.117:0/2738133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55767bac8d80 0x55767bf332a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55767beb2300 tx=0x7f88d8004580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a59f66b271ba0da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:35.380 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.375+0000 7f88bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-21T05:58:35.380 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.375+0000 7f88bbfff640 1 -- 172.21.15.117:0/2738133 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d8052040 con 0x55767bac8d80 2024-09-21T05:58:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.376+0000 7f88ea27b740 1 -- 172.21.15.117:0/2738133 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88cc0b6900 con 0x55767bac8d80 2024-09-21T05:58:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.376+0000 7f88bbfff640 1 -- 172.21.15.117:0/2738133 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f88d8003850 con 0x55767bac8d80 2024-09-21T05:58:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.379+0000 7f88bbfff640 1 -- 172.21.15.117:0/2738133 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f88d8035040 con 0x55767bac8d80 2024-09-21T05:58:35.400 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.392+0000 7f88ea27b740 10 client.25334 fetch_fsmap learned FSMap version 27 2024-09-21T05:58:35.400 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.392+0000 7f88ea27b740 1 -- 172.21.15.117:0/2738133 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55767bf31540 con 0x55767bac8d80 2024-09-21T05:58:35.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.402+0000 7f88bbfff640 1 -- 172.21.15.117:0/2738133 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88cc0b6900 con 0x55767bac8d80 2024-09-21T05:58:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.404+0000 7f88bbfff640 1 -- 172.21.15.117:0/2738133 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f88d8031c40 con 0x55767bac8d80 2024-09-21T05:58:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.404+0000 7f88ea27b740 10 client.25334 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:58:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.404+0000 7f88ea27b740 10 client.25334 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T05:58:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.404+0000 7f88ea27b740 20 client.25334 populate_metadata read hostname 'smithi117' 2024-09-21T05:58:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.404+0000 7f88ea27b740 1 --2- 172.21.15.117:0/2738133 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55767bf5a110 0x55767bf5c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.404+0000 7f88ea27b740 4 client.25334 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T05:58:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.404+0000 7f88ea27b740 1 -- 172.21.15.117:0/2738133 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55767be220f0 con 0x55767bf5a110 2024-09-21T05:58:35.416 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.412+0000 7f88d3fff640 1 --2- 172.21.15.117:0/2738133 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55767bf5a110 0x55767bf5c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.423+0000 7f88d3fff640 1 --2- 172.21.15.117:0/2738133 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55767bf5a110 0x55767bf5c530 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5919f41c096c4616 server_cookie=a0494f9dd8f14da1 in_seq=0 out_seq=0 2024-09-21T05:58:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:35.423+0000 7f88bbfff640 10 client.25334 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:58:35.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:35 smithi117 ceph-mon[30947]: pgmap v365: 97 pgs: 97 active+clean; 9.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 851 MiB/s wr, 1.15k op/s 2024-09-21T05:58:35.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:35 smithi192 ceph-mon[34700]: pgmap v365: 97 pgs: 97 active+clean; 9.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 851 MiB/s wr, 1.15k op/s 2024-09-21T05:58:36.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:35 smithi138 ceph-mon[34391]: pgmap v365: 97 pgs: 97 active+clean; 9.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 851 MiB/s wr, 1.15k op/s 2024-09-21T05:58:37.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.780+0000 7f3d88252640 1 -- 172.21.15.117:0/3796816216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d80106280 msgr2=0x7f3d80106690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:37.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.780+0000 7f3d88252640 1 --2- 172.21.15.117:0/3796816216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d80106280 0x7f3d80106690 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3d70009900 tx=0x7f3d7002a410 comp rx=0 tx=0).stop 2024-09-21T05:58:37.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.780+0000 7f3d88252640 1 -- 172.21.15.117:0/3796816216 shutdown_connections 2024-09-21T05:58:37.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.780+0000 7f3d88252640 1 --2- 172.21.15.117:0/3796816216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d80106280 0x7f3d80106690 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:37.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.780+0000 7f3d88252640 1 --2- 172.21.15.117:0/3796816216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d80106bd0 0x7f3d80110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:37.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.780+0000 7f3d88252640 1 --2- 172.21.15.117:0/3796816216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d80102030 0x7f3d80102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:37.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.780+0000 7f3d88252640 1 -- 172.21.15.117:0/3796816216 >> 172.21.15.117:0/3796816216 conn(0x7f3d80077c60 msgr2=0x7f3d800780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:37.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.782+0000 7f3d88252640 1 -- 172.21.15.117:0/3796816216 shutdown_connections 2024-09-21T05:58:37.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.782+0000 7f3d88252640 1 -- 172.21.15.117:0/3796816216 wait complete. 2024-09-21T05:58:37.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.783+0000 7f3d88252640 1 Processor -- start 2024-09-21T05:58:37.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.783+0000 7f3d88252640 1 -- start start 2024-09-21T05:58:37.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.783+0000 7f3d88252640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d80102030 0x7f3d8019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:37.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.783+0000 7f3d88252640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d80106280 0x7f3d8019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:37.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.783+0000 7f3d88252640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d80106bd0 0x7f3d8019fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:37.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.783+0000 7f3d88252640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3d80113b70 con 0x7f3d80106280 2024-09-21T05:58:37.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.783+0000 7f3d88252640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3d801139f0 con 0x7f3d80102030 2024-09-21T05:58:37.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.783+0000 7f3d88252640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3d80113cf0 con 0x7f3d80106bd0 2024-09-21T05:58:37.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.784+0000 7f3d857ed640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d80106280 0x7f3d8019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:37.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.784+0000 7f3d857ed640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d80106280 0x7f3d8019ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50466/0 (socket says 172.21.15.117:50466) 2024-09-21T05:58:37.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.784+0000 7f3d857ed640 1 -- 172.21.15.117:0/4164380759 learned_addr learned my addr 172.21.15.117:0/4164380759 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:37.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.784+0000 7f3d85fee640 1 --2- 172.21.15.117:0/4164380759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d80102030 0x7f3d8019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:37.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.784+0000 7f3d857ed640 1 -- 172.21.15.117:0/4164380759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d80106bd0 msgr2=0x7f3d8019fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:37.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.784+0000 7f3d857ed640 1 --2- 172.21.15.117:0/4164380759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d80106bd0 0x7f3d8019fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:37.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.784+0000 7f3d857ed640 1 -- 172.21.15.117:0/4164380759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d80102030 msgr2=0x7f3d8019a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:37.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.784+0000 7f3d857ed640 1 --2- 172.21.15.117:0/4164380759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d80102030 0x7f3d8019a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:37.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.784+0000 7f3d857ed640 1 -- 172.21.15.117:0/4164380759 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d801a04f0 con 0x7f3d80106280 2024-09-21T05:58:37.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.784+0000 7f3d857ed640 1 --2- 172.21.15.117:0/4164380759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d80106280 0x7f3d8019ab20 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3d7002a980 tx=0x7f3d70003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:37.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.784+0000 7f3d76ffd640 1 -- 172.21.15.117:0/4164380759 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7003f070 con 0x7f3d80106280 2024-09-21T05:58:37.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.785+0000 7f3d88252640 1 -- 172.21.15.117:0/4164380759 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d801a0780 con 0x7f3d80106280 2024-09-21T05:58:37.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.785+0000 7f3d76ffd640 1 -- 172.21.15.117:0/4164380759 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d7003bd20 con 0x7f3d80106280 2024-09-21T05:58:37.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.785+0000 7f3d76ffd640 1 -- 172.21.15.117:0/4164380759 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d70035030 con 0x7f3d80106280 2024-09-21T05:58:37.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.785+0000 7f3d88252640 1 -- 172.21.15.117:0/4164380759 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d801a6f40 con 0x7f3d80106280 2024-09-21T05:58:37.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.788+0000 7f3d8e7c6740 1 Processor -- start 2024-09-21T05:58:37.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.788+0000 7f3d8e7c6740 1 -- start start 2024-09-21T05:58:37.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.788+0000 7f3d76ffd640 1 -- 172.21.15.117:0/4164380759 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f3d70042070 con 0x7f3d80106280 2024-09-21T05:58:37.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.788+0000 7f3d8e7c6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5558de1cf070 0x5558de576650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:37.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.788+0000 7f3d8e7c6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558de1a61b0 0x5558de576b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:37.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.788+0000 7f3d8e7c6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558de1ae390 0x5558de5770d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:37.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.789+0000 7f3d8e7c6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5558de4ece50 con 0x5558de1a61b0 2024-09-21T05:58:37.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.789+0000 7f3d8e7c6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5558de4dc280 con 0x5558de1ae390 2024-09-21T05:58:37.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.789+0000 7f3d8e7c6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5558de4e0090 con 0x5558de1cf070 2024-09-21T05:58:37.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.789+0000 7f3d85fee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558de1a61b0 0x5558de576b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:37.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.789+0000 7f3d85fee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558de1a61b0 0x5558de576b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50474/0 (socket says 172.21.15.117:50474) 2024-09-21T05:58:37.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.789+0000 7f3d85fee640 1 -- 172.21.15.117:0/1223497685 learned_addr learned my addr 172.21.15.117:0/1223497685 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:37.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.789+0000 7f3d85fee640 1 -- 172.21.15.117:0/1223497685 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5558de1cf070 msgr2=0x5558de576650 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T05:58:37.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.789+0000 7f3d85fee640 1 --2- 172.21.15.117:0/1223497685 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5558de1cf070 0x5558de576650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:37.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.789+0000 7f3d85fee640 1 -- 172.21.15.117:0/1223497685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558de1ae390 msgr2=0x5558de5770d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T05:58:37.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.789+0000 7f3d85fee640 1 --2- 172.21.15.117:0/1223497685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558de1ae390 0x5558de5770d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:37.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.789+0000 7f3d85fee640 1 -- 172.21.15.117:0/1223497685 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558de23a650 con 0x5558de1a61b0 2024-09-21T05:58:37.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.789+0000 7f3d76ffd640 1 --2- 172.21.15.117:0/4164380759 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3d6c0b66b0 0x7f3d6c0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:37.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.789+0000 7f3d85fee640 1 --2- 172.21.15.117:0/1223497685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558de1a61b0 0x5558de576b90 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c00b7f0 tx=0x7f3d7c00bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b124ecd119dfb44 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:37.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.790+0000 7f3d867ef640 1 --2- 172.21.15.117:0/4164380759 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3d6c0b66b0 0x7f3d6c0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:37.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.790+0000 7f3d6bfff640 1 -- 172.21.15.117:0/1223497685 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c00d950 con 0x5558de1a61b0 2024-09-21T05:58:37.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.790+0000 7f3d6bfff640 1 -- 172.21.15.117:0/1223497685 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d7c018070 con 0x5558de1a61b0 2024-09-21T05:58:37.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.790+0000 7f3d6bfff640 1 -- 172.21.15.117:0/1223497685 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c0123e0 con 0x5558de1a61b0 2024-09-21T05:58:37.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.790+0000 7f3d76ffd640 1 -- 172.21.15.117:0/4164380759 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f3d70107c80 con 0x7f3d80106280 2024-09-21T05:58:37.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.791+0000 7f3d867ef640 1 --2- 172.21.15.117:0/4164380759 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3d6c0b66b0 0x7f3d6c0b8bc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3d78005fd0 tx=0x7f3d78009710 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:37.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.791+0000 7f3d8e7c6740 1 -- 172.21.15.117:0/1223497685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558de1a61b0 msgr2=0x5558de576b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:37.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.791+0000 7f3d8e7c6740 1 --2- 172.21.15.117:0/1223497685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558de1a61b0 0x5558de576b90 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c00b7f0 tx=0x7f3d7c00bcb0 comp rx=0 tx=0).stop 2024-09-21T05:58:37.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.791+0000 7f3d8e7c6740 1 -- 172.21.15.117:0/1223497685 shutdown_connections 2024-09-21T05:58:37.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.791+0000 7f3d8e7c6740 1 --2- 172.21.15.117:0/1223497685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558de1a61b0 0x5558de576b90 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:37.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.791+0000 7f3d8e7c6740 1 --2- 172.21.15.117:0/1223497685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558de1ae390 0x5558de5770d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:37.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.791+0000 7f3d8e7c6740 1 --2- 172.21.15.117:0/1223497685 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5558de1cf070 0x5558de576650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:37.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.791+0000 7f3d8e7c6740 1 -- 172.21.15.117:0/1223497685 >> 172.21.15.117:0/1223497685 conn(0x5558de1c3460 msgr2=0x5558de574540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:37.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.792+0000 7f3d8e7c6740 1 -- 172.21.15.117:0/1223497685 shutdown_connections 2024-09-21T05:58:37.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.792+0000 7f3d8e7c6740 1 -- 172.21.15.117:0/1223497685 wait complete. 2024-09-21T05:58:37.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.792+0000 7f3d8e7c6740 1 -- start start 2024-09-21T05:58:37.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.792+0000 7f3d8e7c6740 1 Processor -- start 2024-09-21T05:58:37.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.793+0000 7f3d8e7c6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558de1a61b0 0x5558de581df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:37.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.793+0000 7f3d8e7c6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558de1ae390 0x5558de582330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:37.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.793+0000 7f3d8e7c6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5558de1cf070 0x5558de58e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:37.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.793+0000 7f3d8e7c6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5558de33d700 con 0x5558de1ae390 2024-09-21T05:58:37.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.793+0000 7f3d8e7c6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5558de477110 con 0x5558de1a61b0 2024-09-21T05:58:37.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.793+0000 7f3d8e7c6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5558de584e30 con 0x5558de1cf070 2024-09-21T05:58:37.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.793+0000 7f3d857ed640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558de1ae390 0x5558de582330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:37.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.793+0000 7f3d857ed640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558de1ae390 0x5558de582330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50486/0 (socket says 172.21.15.117:50486) 2024-09-21T05:58:37.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.793+0000 7f3d857ed640 1 -- 172.21.15.117:0/2770172437 learned_addr learned my addr 172.21.15.117:0/2770172437 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:37.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.793+0000 7f3d8e7c6740 10 client.0 _ll_register_callbacks cb 0x5558de490990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:58:37.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.794+0000 7f3d867ef640 1 --2- 172.21.15.117:0/2770172437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5558de1cf070 0x5558de58e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:37.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.794+0000 7f3d857ed640 1 -- 172.21.15.117:0/2770172437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5558de1cf070 msgr2=0x5558de58e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:37.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.794+0000 7f3d857ed640 1 --2- 172.21.15.117:0/2770172437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5558de1cf070 0x5558de58e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:37.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.794+0000 7f3d857ed640 1 -- 172.21.15.117:0/2770172437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558de1a61b0 msgr2=0x5558de581df0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T05:58:37.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.794+0000 7f3d857ed640 1 --2- 172.21.15.117:0/2770172437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558de1a61b0 0x5558de581df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:37.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.794+0000 7f3d857ed640 1 -- 172.21.15.117:0/2770172437 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558de4a4c40 con 0x5558de1ae390 2024-09-21T05:58:37.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.794+0000 7f3d857ed640 1 --2- 172.21.15.117:0/2770172437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558de1ae390 0x5558de582330 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3d70105a50 tx=0x7f3d700ca2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e21b33d34a90cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:37.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.794+0000 7f3d697fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:58:37.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.795+0000 7f3d697fa640 1 -- 172.21.15.117:0/2770172437 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d70049070 con 0x5558de1ae390 2024-09-21T05:58:37.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.795+0000 7f3d8e7c6740 1 -- 172.21.15.117:0/2770172437 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d7c018070 con 0x5558de1ae390 2024-09-21T05:58:37.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.795+0000 7f3d697fa640 1 -- 172.21.15.117:0/2770172437 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f3d700314c0 con 0x5558de1ae390 2024-09-21T05:58:37.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.795+0000 7f3d697fa640 1 -- 172.21.15.117:0/2770172437 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3d700cd4a0 con 0x5558de1ae390 2024-09-21T05:58:37.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.795+0000 7f3d8e7c6740 10 client.15846 fetch_fsmap learned FSMap version 27 2024-09-21T05:58:37.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.795+0000 7f3d8e7c6740 1 -- 172.21.15.117:0/2770172437 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5558de590c20 con 0x5558de1ae390 2024-09-21T05:58:37.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.796+0000 7f3d697fa640 1 -- 172.21.15.117:0/2770172437 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d7c018070 con 0x5558de1ae390 2024-09-21T05:58:37.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.796+0000 7f3d697fa640 1 -- 172.21.15.117:0/2770172437 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3d700b7070 con 0x5558de1ae390 2024-09-21T05:58:37.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.796+0000 7f3d8e7c6740 10 client.15846 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:58:37.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.796+0000 7f3d8e7c6740 10 client.15846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T05:58:37.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.796+0000 7f3d8e7c6740 20 client.15846 populate_metadata read hostname 'smithi117' 2024-09-21T05:58:37.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.796+0000 7f3d8e7c6740 1 --2- 172.21.15.117:0/2770172437 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5558de18d9f0 0x5558de5940f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:37.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.796+0000 7f3d8e7c6740 4 client.15846 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:58:37.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.796+0000 7f3d8e7c6740 1 -- 172.21.15.117:0/2770172437 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558de4eeda0 con 0x5558de18d9f0 2024-09-21T05:58:37.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.799+0000 7f3d85fee640 1 --2- 172.21.15.117:0/2770172437 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5558de18d9f0 0x5558de5940f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:37.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.804+0000 7f3d85fee640 1 --2- 172.21.15.117:0/2770172437 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5558de18d9f0 0x5558de5940f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cbafc6afd38170d server_cookie=d9c5959c8eafc49b in_seq=0 out_seq=0 2024-09-21T05:58:37.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.804+0000 7f3d697fa640 10 client.15846 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:58:37.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.809+0000 7f3d697fa640 1 -- 172.21.15.117:0/2770172437 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5558de4eeda0 con 0x5558de18d9f0 2024-09-21T05:58:37.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.809+0000 7f3d697fa640 10 client.15846 handle_command_reply: tid=0 2024-09-21T05:58:37.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.809+0000 7f3d8e7c6740 1 client.15846 shutdown 2024-09-21T05:58:37.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.810+0000 7f3d8e7c6740 1 -- 172.21.15.117:0/2770172437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558de1ae390 msgr2=0x5558de582330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:37.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.810+0000 7f3d8e7c6740 1 --2- 172.21.15.117:0/2770172437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558de1ae390 0x5558de582330 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3d70105a50 tx=0x7f3d700ca2d0 comp rx=0 tx=0).stop 2024-09-21T05:58:37.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.810+0000 7f3d8e7c6740 1 -- 172.21.15.117:0/2770172437 shutdown_connections 2024-09-21T05:58:37.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.810+0000 7f3d8e7c6740 1 --2- 172.21.15.117:0/2770172437 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5558de18d9f0 0x5558de5940f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:37.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.810+0000 7f3d8e7c6740 1 --2- 172.21.15.117:0/2770172437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558de1ae390 0x5558de582330 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:37.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.810+0000 7f3d8e7c6740 1 --2- 172.21.15.117:0/2770172437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5558de1cf070 0x5558de58e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:37.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.810+0000 7f3d8e7c6740 1 --2- 172.21.15.117:0/2770172437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558de1a61b0 0x5558de581df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:37.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.810+0000 7f3d8e7c6740 1 -- 172.21.15.117:0/2770172437 >> 172.21.15.117:0/2770172437 conn(0x5558de1c3460 msgr2=0x5558de5797e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:37.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.810+0000 7f3d697fa640 0 client.15846 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:58:37.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.810+0000 7f3d8e7c6740 1 -- 172.21.15.117:0/2770172437 shutdown_connections 2024-09-21T05:58:37.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.810+0000 7f3d8e7c6740 1 -- 172.21.15.117:0/2770172437 wait complete. 2024-09-21T05:58:37.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.810+0000 7f3d8e7c6740 20 client.15846 trim_cache size 0 max 16384 2024-09-21T05:58:37.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.838+0000 7f3d8e7c6740 1 Processor -- start 2024-09-21T05:58:37.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.838+0000 7f3d8e7c6740 1 -- start start 2024-09-21T05:58:37.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.838+0000 7f3d8e7c6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558de1a61b0 0x5558de583a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:37.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.838+0000 7f3d8e7c6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5558de1ae390 0x5558de573b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:37.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.838+0000 7f3d8e7c6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558de1cf070 0x5558de5740b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:37.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.838+0000 7f3d8e7c6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3d801b1780 con 0x5558de1cf070 2024-09-21T05:58:37.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.838+0000 7f3d8e7c6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3d801b32c0 con 0x5558de1a61b0 2024-09-21T05:58:37.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.838+0000 7f3d8e7c6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3d801b1e50 con 0x5558de1ae390 2024-09-21T05:58:37.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.838+0000 7f3d867ef640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558de1cf070 0x5558de5740b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:37.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.839+0000 7f3d867ef640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558de1cf070 0x5558de5740b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50494/0 (socket says 172.21.15.117:50494) 2024-09-21T05:58:37.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.839+0000 7f3d867ef640 1 -- 172.21.15.117:0/1563199571 learned_addr learned my addr 172.21.15.117:0/1563199571 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:37.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.839+0000 7f3d867ef640 1 -- 172.21.15.117:0/1563199571 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5558de1ae390 msgr2=0x5558de573b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:37.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.839+0000 7f3d867ef640 1 --2- 172.21.15.117:0/1563199571 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5558de1ae390 0x5558de573b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:37.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.839+0000 7f3d867ef640 1 -- 172.21.15.117:0/1563199571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558de1a61b0 msgr2=0x5558de583a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:37.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.839+0000 7f3d867ef640 1 --2- 172.21.15.117:0/1563199571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558de1a61b0 0x5558de583a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:37.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.839+0000 7f3d867ef640 1 -- 172.21.15.117:0/1563199571 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558de5830b0 con 0x5558de1cf070 2024-09-21T05:58:37.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.839+0000 7f3d867ef640 1 --2- 172.21.15.117:0/1563199571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558de1cf070 0x5558de5740b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3d780114f0 tx=0x7f3d78011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d6a2352d0c41c01 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:37.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.839+0000 7f3d6b7fe640 1 -- 172.21.15.117:0/1563199571 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d78023070 con 0x5558de1cf070 2024-09-21T05:58:37.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.840+0000 7f3d6b7fe640 1 -- 172.21.15.117:0/1563199571 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d78017040 con 0x5558de1cf070 2024-09-21T05:58:37.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.840+0000 7f3d6b7fe640 1 -- 172.21.15.117:0/1563199571 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d780109a0 con 0x5558de1cf070 2024-09-21T05:58:37.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.841+0000 7f3d8e7c6740 1 -- 172.21.15.117:0/1563199571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558de1cf070 msgr2=0x5558de5740b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:37.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.841+0000 7f3d8e7c6740 1 --2- 172.21.15.117:0/1563199571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558de1cf070 0x5558de5740b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3d780114f0 tx=0x7f3d78011000 comp rx=0 tx=0).stop 2024-09-21T05:58:37.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.841+0000 7f3d8e7c6740 1 -- 172.21.15.117:0/1563199571 shutdown_connections 2024-09-21T05:58:37.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.841+0000 7f3d8e7c6740 1 --2- 172.21.15.117:0/1563199571 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558de1cf070 0x5558de5740b0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:37.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.841+0000 7f3d8e7c6740 1 --2- 172.21.15.117:0/1563199571 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5558de1ae390 0x5558de573b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:37.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.841+0000 7f3d8e7c6740 1 --2- 172.21.15.117:0/1563199571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558de1a61b0 0x5558de583a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:37.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.841+0000 7f3d8e7c6740 1 -- 172.21.15.117:0/1563199571 >> 172.21.15.117:0/1563199571 conn(0x5558de1c3460 msgr2=0x5558de583310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:37.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.841+0000 7f3d8e7c6740 1 -- 172.21.15.117:0/1563199571 shutdown_connections 2024-09-21T05:58:37.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.841+0000 7f3d8e7c6740 1 -- 172.21.15.117:0/1563199571 wait complete. 2024-09-21T05:58:37.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.842+0000 7f3d8e7c6740 1 -- start start 2024-09-21T05:58:37.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.842+0000 7f3d8e7c6740 1 Processor -- start 2024-09-21T05:58:37.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.843+0000 7f3d8e7c6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558de1a61b0 0x5558de58a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:37.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.843+0000 7f3d8e7c6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558de1ae390 0x5558de585bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:37.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.843+0000 7f3d8e7c6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5558de1cf070 0x5558de5860f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:37.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.843+0000 7f3d85fee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558de1a61b0 0x5558de58a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:37.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.843+0000 7f3d85fee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558de1a61b0 0x5558de58a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50508/0 (socket says 172.21.15.117:50508) 2024-09-21T05:58:37.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.843+0000 7f3d8e7c6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3d801b2bf0 con 0x5558de1a61b0 2024-09-21T05:58:37.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.843+0000 7f3d85fee640 1 -- 172.21.15.117:0/2505067349 learned_addr learned my addr 172.21.15.117:0/2505067349 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:37.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.843+0000 7f3d8e7c6740 1 -- 172.21.15.117:0/2505067349 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3d801b2520 con 0x5558de1ae390 2024-09-21T05:58:37.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.843+0000 7f3d8e7c6740 1 -- 172.21.15.117:0/2505067349 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3d801af780 con 0x5558de1cf070 2024-09-21T05:58:37.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.844+0000 7f3d8e7c6740 10 client.0 _ll_register_callbacks cb 0x7f3d70005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:58:37.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.844+0000 7f3d85fee640 1 -- 172.21.15.117:0/2505067349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5558de1cf070 msgr2=0x5558de5860f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:37.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.844+0000 7f3d85fee640 1 --2- 172.21.15.117:0/2505067349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5558de1cf070 0x5558de5860f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:37.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.844+0000 7f3d85fee640 1 -- 172.21.15.117:0/2505067349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558de1ae390 msgr2=0x5558de585bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:37.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.844+0000 7f3d85fee640 1 --2- 172.21.15.117:0/2505067349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558de1ae390 0x5558de585bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:37.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.844+0000 7f3d85fee640 1 -- 172.21.15.117:0/2505067349 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558de5866b0 con 0x5558de1a61b0 2024-09-21T05:58:37.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.844+0000 7f3d85fee640 1 --2- 172.21.15.117:0/2505067349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558de1a61b0 0x5558de58a700 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c008010 tx=0x7f3d7c013c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd8300bd5177d91 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:37.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.844+0000 7f3d697fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:58:37.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.845+0000 7f3d697fa640 1 -- 172.21.15.117:0/2505067349 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c00d950 con 0x5558de1a61b0 2024-09-21T05:58:37.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.845+0000 7f3d8e7c6740 1 -- 172.21.15.117:0/2505067349 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d78017040 con 0x5558de1a61b0 2024-09-21T05:58:37.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.846+0000 7f3d697fa640 1 -- 172.21.15.117:0/2505067349 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f3d7c00daf0 con 0x5558de1a61b0 2024-09-21T05:58:37.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.846+0000 7f3d697fa640 1 -- 172.21.15.117:0/2505067349 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3d7c00dd10 con 0x5558de1a61b0 2024-09-21T05:58:37.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.846+0000 7f3d8e7c6740 10 client.15852 fetch_fsmap learned FSMap version 27 2024-09-21T05:58:37.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.846+0000 7f3d8e7c6740 1 -- 172.21.15.117:0/2505067349 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5558de5fe830 con 0x5558de1a61b0 2024-09-21T05:58:37.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.847+0000 7f3d697fa640 1 -- 172.21.15.117:0/2505067349 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d78017040 con 0x5558de1a61b0 2024-09-21T05:58:37.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.848+0000 7f3d697fa640 1 -- 172.21.15.117:0/2505067349 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3d7c013ce0 con 0x5558de1a61b0 2024-09-21T05:58:37.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.848+0000 7f3d8e7c6740 10 client.15852 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:58:37.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.848+0000 7f3d8e7c6740 10 client.15852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T05:58:37.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.848+0000 7f3d8e7c6740 20 client.15852 populate_metadata read hostname 'smithi117' 2024-09-21T05:58:37.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.849+0000 7f3d8e7c6740 1 --2- 172.21.15.117:0/2505067349 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5558de162da0 0x5558de600f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:37.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.849+0000 7f3d8e7c6740 4 client.15852 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T05:58:37.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.849+0000 7f3d8e7c6740 1 -- 172.21.15.117:0/2505067349 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5558de4ea850 con 0x5558de162da0 2024-09-21T05:58:37.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.853+0000 7f3d857ed640 1 --2- 172.21.15.117:0/2505067349 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5558de162da0 0x5558de600f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:37.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.856+0000 7f3d697fa640 10 client.15852 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:58:37.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.856+0000 7f3d857ed640 1 --2- 172.21.15.117:0/2505067349 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5558de162da0 0x5558de600f00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e5933158aed778e server_cookie=440386abf6f03e15 in_seq=0 out_seq=0 2024-09-21T05:58:37.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.859+0000 7f3d697fa640 1 -- 172.21.15.117:0/2505067349 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+658 (crc 0 0 0) 0x5558de4ea850 con 0x5558de162da0 2024-09-21T05:58:37.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.860+0000 7f3d697fa640 10 client.15852 handle_command_reply: tid=0 2024-09-21T05:58:37.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.860+0000 7f3d8e7c6740 1 client.15852 shutdown 2024-09-21T05:58:37.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.860+0000 7f3d8e7c6740 1 -- 172.21.15.117:0/2505067349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558de1a61b0 msgr2=0x5558de58a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:37.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.860+0000 7f3d8e7c6740 1 --2- 172.21.15.117:0/2505067349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558de1a61b0 0x5558de58a700 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c008010 tx=0x7f3d7c013c20 comp rx=0 tx=0).stop 2024-09-21T05:58:37.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.860+0000 7f3d8e7c6740 1 -- 172.21.15.117:0/2505067349 shutdown_connections 2024-09-21T05:58:37.523 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T05:58:37.523 INFO:teuthology.orchestra.run.smithi117.stdout: "status": "no active scrubs running", 2024-09-21T05:58:37.523 INFO:teuthology.orchestra.run.smithi117.stdout: "scrubs": {}, 2024-09-21T05:58:37.523 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_stats": { 2024-09-21T05:58:37.523 INFO:teuthology.orchestra.run.smithi117.stdout: "8b3fe95a-b76b-4e94-afd0-72fe35e2dd92": { 2024-09-21T05:58:37.524 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:58:01.246076+0000", 2024-09-21T05:58:37.524 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T05:58:37.524 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T05:58:37.524 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T05:58:37.524 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T05:58:37.524 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T05:58:37.524 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T05:58:37.524 INFO:teuthology.orchestra.run.smithi117.stdout: "b30bfff0-0f6b-4e30-8467-ff7db421eba6": { 2024-09-21T05:58:37.524 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:57:14.939082+0000", 2024-09-21T05:58:37.524 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T05:58:37.524 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T05:58:37.524 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T05:58:37.524 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T05:58:37.524 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T05:58:37.524 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T05:58:37.525 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T05:58:37.525 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T05:58:37.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.860+0000 7f3d8e7c6740 1 --2- 172.21.15.117:0/2505067349 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5558de162da0 0x5558de600f00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:37.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.860+0000 7f3d8e7c6740 1 --2- 172.21.15.117:0/2505067349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5558de1cf070 0x5558de5860f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:37.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.860+0000 7f3d8e7c6740 1 --2- 172.21.15.117:0/2505067349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5558de1ae390 0x5558de585bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:37.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.860+0000 7f3d697fa640 0 client.15852 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:58:37.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.860+0000 7f3d8e7c6740 1 --2- 172.21.15.117:0/2505067349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5558de1a61b0 0x5558de58a700 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:37.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.861+0000 7f3d8e7c6740 1 -- 172.21.15.117:0/2505067349 >> 172.21.15.117:0/2505067349 conn(0x5558de1c3460 msgr2=0x5558de577680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:37.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.861+0000 7f3d8e7c6740 1 -- 172.21.15.117:0/2505067349 shutdown_connections 2024-09-21T05:58:37.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.861+0000 7f3d8e7c6740 1 -- 172.21.15.117:0/2505067349 wait complete. 2024-09-21T05:58:37.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.861+0000 7f3d8e7c6740 20 client.15852 trim_cache size 0 max 16384 2024-09-21T05:58:37.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.863+0000 7f3d88252640 1 -- 172.21.15.117:0/4164380759 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3d6c0b66b0 msgr2=0x7f3d6c0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:37.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.863+0000 7f3d88252640 1 --2- 172.21.15.117:0/4164380759 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3d6c0b66b0 0x7f3d6c0b8bc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3d78005fd0 tx=0x7f3d78009710 comp rx=0 tx=0).stop 2024-09-21T05:58:37.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.863+0000 7f3d88252640 1 -- 172.21.15.117:0/4164380759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d80106280 msgr2=0x7f3d8019ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:37.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.863+0000 7f3d88252640 1 --2- 172.21.15.117:0/4164380759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d80106280 0x7f3d8019ab20 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3d7002a980 tx=0x7f3d70003cd0 comp rx=0 tx=0).stop 2024-09-21T05:58:37.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.863+0000 7f3d88252640 1 -- 172.21.15.117:0/4164380759 shutdown_connections 2024-09-21T05:58:37.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.863+0000 7f3d88252640 1 --2- 172.21.15.117:0/4164380759 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3d6c0b66b0 0x7f3d6c0b8bc0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:37.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.864+0000 7f3d88252640 1 --2- 172.21.15.117:0/4164380759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d80106280 0x7f3d8019ab20 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:37.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.864+0000 7f3d88252640 1 --2- 172.21.15.117:0/4164380759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d80106bd0 0x7f3d8019fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:37.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.864+0000 7f3d88252640 1 --2- 172.21.15.117:0/4164380759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d80102030 0x7f3d8019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:37.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.864+0000 7f3d88252640 1 -- 172.21.15.117:0/4164380759 >> 172.21.15.117:0/4164380759 conn(0x7f3d80077c60 msgr2=0x7f3d801074b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:37.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.864+0000 7f3d88252640 1 -- 172.21.15.117:0/4164380759 shutdown_connections 2024-09-21T05:58:37.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:36.864+0000 7f3d88252640 1 -- 172.21.15.117:0/4164380759 wait complete. 2024-09-21T05:58:37.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:37 smithi117 ceph-mon[30947]: pgmap v366: 97 pgs: 97 active+clean; 9.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 868 MiB/s wr, 1.53k op/s 2024-09-21T05:58:37.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:37 smithi192 ceph-mon[34700]: pgmap v366: 97 pgs: 97 active+clean; 9.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 868 MiB/s wr, 1.53k op/s 2024-09-21T05:58:38.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:37 smithi138 ceph-mon[34391]: pgmap v366: 97 pgs: 97 active+clean; 9.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 868 MiB/s wr, 1.53k op/s 2024-09-21T05:58:38.212 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:38.209+0000 7f88bbfff640 1 -- 172.21.15.117:0/2738133 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55767be220f0 con 0x55767bf5a110 2024-09-21T05:58:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:38.209+0000 7f88bbfff640 10 client.25334 handle_command_reply: tid=0 2024-09-21T05:58:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:38.209+0000 7f88ea27b740 1 client.25334 shutdown 2024-09-21T05:58:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:38.210+0000 7f88ea27b740 1 -- 172.21.15.117:0/2738133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55767bac8d80 msgr2=0x55767bf332a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:38.210+0000 7f88ea27b740 1 --2- 172.21.15.117:0/2738133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55767bac8d80 0x55767bf332a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55767beb2300 tx=0x7f88d8004580 comp rx=0 tx=0).stop 2024-09-21T05:58:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:38.210+0000 7f88ea27b740 1 -- 172.21.15.117:0/2738133 shutdown_connections 2024-09-21T05:58:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:38.210+0000 7f88ea27b740 1 --2- 172.21.15.117:0/2738133 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55767bf5a110 0x55767bf5c530 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:38.210+0000 7f88ea27b740 1 --2- 172.21.15.117:0/2738133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55767bb0a1b0 0x55767bec3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:38.210+0000 7f88ea27b740 1 --2- 172.21.15.117:0/2738133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55767bae94d0 0x55767bec3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:38.210+0000 7f88ea27b740 1 --2- 172.21.15.117:0/2738133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55767bac8d80 0x55767bf332a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:38.211+0000 7f88ea27b740 1 -- 172.21.15.117:0/2738133 >> 172.21.15.117:0/2738133 conn(0x55767bafe300 msgr2=0x55767beb4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:38.211+0000 7f88bbfff640 0 client.25334 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:58:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:38.212+0000 7f88ea27b740 1 -- 172.21.15.117:0/2738133 shutdown_connections 2024-09-21T05:58:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:38.212+0000 7f88ea27b740 1 -- 172.21.15.117:0/2738133 wait complete. 2024-09-21T05:58:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:38.212+0000 7f88ea27b740 20 client.25334 trim_cache size 0 max 16384 2024-09-21T05:58:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T05:58:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T05:58:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T05:58:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T05:58:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:38.215+0000 7f88e3230640 1 -- 172.21.15.117:0/4226821243 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f88c40b67d0 msgr2=0x7f88c40b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:38.215+0000 7f88e3230640 1 --2- 172.21.15.117:0/4226821243 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f88c40b67d0 0x7f88c40b8ce0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f88d400aab0 tx=0x7f88d4009250 comp rx=0 tx=0).stop 2024-09-21T05:58:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:38.215+0000 7f88e3230640 1 -- 172.21.15.117:0/4226821243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88dc1082e0 msgr2=0x7f88dc0ff550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:38.215+0000 7f88e3230640 1 --2- 172.21.15.117:0/4226821243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88dc1082e0 0x7f88dc0ff550 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f88cc00b450 tx=0x7f88cc00b910 comp rx=0 tx=0).stop 2024-09-21T05:58:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:38.215+0000 7f88e3230640 1 -- 172.21.15.117:0/4226821243 shutdown_connections 2024-09-21T05:58:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:38.215+0000 7f88e3230640 1 --2- 172.21.15.117:0/4226821243 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f88c40b67d0 0x7f88c40b8ce0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:38.215+0000 7f88e3230640 1 --2- 172.21.15.117:0/4226821243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88dc1082e0 0x7f88dc0ff550 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:38.215+0000 7f88e3230640 1 --2- 172.21.15.117:0/4226821243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88dc108c30 0x7f88dc0ffa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:38.215+0000 7f88e3230640 1 --2- 172.21.15.117:0/4226821243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88dc1029b0 0x7f88dc100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:38.216+0000 7f88e3230640 1 -- 172.21.15.117:0/4226821243 >> 172.21.15.117:0/4226821243 conn(0x7f88dc07a740 msgr2=0x7f88dc1018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:38.217+0000 7f88e3230640 1 -- 172.21.15.117:0/4226821243 shutdown_connections 2024-09-21T05:58:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:38.217+0000 7f88e3230640 1 -- 172.21.15.117:0/4226821243 wait complete. 2024-09-21T05:58:39.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:38 smithi117 ceph-mon[30947]: pgmap v367: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 966 MiB/s wr, 2.14k op/s 2024-09-21T05:58:39.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:38 smithi138 ceph-mon[34391]: pgmap v367: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 966 MiB/s wr, 2.14k op/s 2024-09-21T05:58:39.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:38 smithi192 ceph-mon[34700]: pgmap v367: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 966 MiB/s wr, 2.14k op/s 2024-09-21T05:58:41.583 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T05:58:41.583 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T05:58:41.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:41 smithi117 ceph-mon[30947]: pgmap v368: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 782 MiB/s wr, 2.05k op/s 2024-09-21T05:58:41.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:41 smithi192 ceph-mon[34700]: pgmap v368: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 782 MiB/s wr, 2.05k op/s 2024-09-21T05:58:42.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:41 smithi138 ceph-mon[34391]: pgmap v368: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 782 MiB/s wr, 2.05k op/s 2024-09-21T05:58:42.400 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:58:43.778 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:43 smithi117 ceph-mon[30947]: pgmap v369: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 910 MiB/s wr, 2.37k op/s 2024-09-21T05:58:43.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:43 smithi192 ceph-mon[34700]: pgmap v369: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 910 MiB/s wr, 2.37k op/s 2024-09-21T05:58:44.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:43 smithi138 ceph-mon[34391]: pgmap v369: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 910 MiB/s wr, 2.37k op/s 2024-09-21T05:58:45.103 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T05:58:45.104 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":27,"btime":"2024-09-21T05:57:06:651813+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3227057504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3227057504},{"type":"v1","addr":"172.21.15.117:6835","nonce":3227057504}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3303829183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3303829183},{"type":"v1","addr":"172.21.15.117:6837","nonce":3303829183}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T05:55:14.555897+0000","modified":"2024-09-21T05:57:06.599449+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24638,"mds_1":24637,"mds_2":24628,"mds_3":24616,"mds_4":24602},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6839/2314572411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2314572411},{"type":"v1","addr":"172.21.15.117:6839","nonce":2314572411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6841/3378096561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":3378096561},{"type":"v1","addr":"172.21.15.117:6841","nonce":3378096561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/3725629862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3725629862},{"type":"v1","addr":"172.21.15.138:6833","nonce":3725629862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/4276801396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4276801396},{"type":"v1","addr":"172.21.15.138:6835","nonce":4276801396}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6833/2224910879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2224910879},{"type":"v1","addr":"172.21.15.192:6833","nonce":2224910879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.138:6837/2540335374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2540335374},{"type":"v1","addr":"172.21.15.138:6837","nonce":2540335374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24616":{"gid":24616,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.192:6835/3530688860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3530688860},{"type":"v1","addr":"172.21.15.192:6835","nonce":3530688860}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.192:6837/2683860842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2683860842},{"type":"v1","addr":"172.21.15.192:6837","nonce":2683860842}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.192:6839/1727623231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1727623231},{"type":"v1","addr":"172.21.15.192:6839","nonce":1727623231}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/3230372576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":3230372576},{"type":"v1","addr":"172.21.15.138:6839","nonce":3230372576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24602,24628,24616,24637,24638]},"id":1}]} 2024-09-21T05:58:45.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.461+0000 7fa437753640 1 -- 172.21.15.117:0/3421519473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa43011cc30 msgr2=0x7fa43011f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:45.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.463+0000 7fa437753640 1 --2- 172.21.15.117:0/3421519473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa43011cc30 0x7fa43011f140 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa428009ca0 tx=0x7fa42802a8d0 comp rx=0 tx=0).stop 2024-09-21T05:58:45.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.465+0000 7fa437753640 1 -- 172.21.15.117:0/3421519473 shutdown_connections 2024-09-21T05:58:45.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.465+0000 7fa437753640 1 --2- 172.21.15.117:0/3421519473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa43011cc30 0x7fa43011f140 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:45.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.465+0000 7fa437753640 1 --2- 172.21.15.117:0/3421519473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa43010a9e0 0x7fa43010adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:45.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.465+0000 7fa437753640 1 --2- 172.21.15.117:0/3421519473 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa430111f40 0x7fa43010a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:45.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.465+0000 7fa437753640 1 -- 172.21.15.117:0/3421519473 >> 172.21.15.117:0/3421519473 conn(0x7fa430070ec0 msgr2=0x7fa4300712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:45.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.470+0000 7fa437753640 1 -- 172.21.15.117:0/3421519473 shutdown_connections 2024-09-21T05:58:45.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.472+0000 7fa437753640 1 -- 172.21.15.117:0/3421519473 wait complete. 2024-09-21T05:58:45.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.473+0000 7fa437753640 1 Processor -- start 2024-09-21T05:58:45.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.473+0000 7fa437753640 1 -- start start 2024-09-21T05:58:45.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.474+0000 7fa437753640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa43010a9e0 0x7fa4301a9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:45.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.474+0000 7fa437753640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa430111f40 0x7fa4301a96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:45.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.474+0000 7fa437753640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa43011cc30 0x7fa4301ae940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:45.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.474+0000 7fa437753640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa430122910 con 0x7fa430111f40 2024-09-21T05:58:45.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.474+0000 7fa437753640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa430122790 con 0x7fa43010a9e0 2024-09-21T05:58:45.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.474+0000 7fa437753640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa430122a90 con 0x7fa43011cc30 2024-09-21T05:58:45.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.475+0000 7fa435cf0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa43011cc30 0x7fa4301ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:45.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.476+0000 7fa4354ef640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa43010a9e0 0x7fa4301a9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:45.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.476+0000 7fa435cf0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa43011cc30 0x7fa4301ae940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:42774/0 (socket says 172.21.15.117:42774) 2024-09-21T05:58:45.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.476+0000 7fa435cf0640 1 -- 172.21.15.117:0/1557959509 learned_addr learned my addr 172.21.15.117:0/1557959509 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:45.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.476+0000 7fa434cee640 1 --2- 172.21.15.117:0/1557959509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa430111f40 0x7fa4301a96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:45.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.477+0000 7fa434cee640 1 -- 172.21.15.117:0/1557959509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa43011cc30 msgr2=0x7fa4301ae940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:45.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.477+0000 7fa434cee640 1 --2- 172.21.15.117:0/1557959509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa43011cc30 0x7fa4301ae940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:45.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.477+0000 7fa434cee640 1 -- 172.21.15.117:0/1557959509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa43010a9e0 msgr2=0x7fa4301a9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:45.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.477+0000 7fa434cee640 1 --2- 172.21.15.117:0/1557959509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa43010a9e0 0x7fa4301a9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:45.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.477+0000 7fa434cee640 1 -- 172.21.15.117:0/1557959509 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4301af070 con 0x7fa430111f40 2024-09-21T05:58:45.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.477+0000 7fa434cee640 1 --2- 172.21.15.117:0/1557959509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa430111f40 0x7fa4301a96a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa43006c6e0 tx=0x7fa42000b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:45.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.478+0000 7fa4354ef640 1 --2- 172.21.15.117:0/1557959509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa43010a9e0 0x7fa4301a9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:58:45.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.484+0000 7fa4267fc640 1 -- 172.21.15.117:0/1557959509 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa420015020 con 0x7fa430111f40 2024-09-21T05:58:45.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.484+0000 7fa4267fc640 1 -- 172.21.15.117:0/1557959509 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa420011ce0 con 0x7fa430111f40 2024-09-21T05:58:45.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.484+0000 7fa4267fc640 1 -- 172.21.15.117:0/1557959509 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa420010e10 con 0x7fa430111f40 2024-09-21T05:58:45.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.484+0000 7fa437753640 1 -- 172.21.15.117:0/1557959509 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4301af360 con 0x7fa430111f40 2024-09-21T05:58:45.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.484+0000 7fa437753640 1 -- 172.21.15.117:0/1557959509 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4301b5ac0 con 0x7fa430111f40 2024-09-21T05:58:45.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.488+0000 7fa437753640 1 -- 172.21.15.117:0/1557959509 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4140052c0 con 0x7fa430111f40 2024-09-21T05:58:45.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.498+0000 7fa4267fc640 1 -- 172.21.15.117:0/1557959509 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fa420018020 con 0x7fa430111f40 2024-09-21T05:58:45.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.499+0000 7fa4267fc640 1 --2- 172.21.15.117:0/1557959509 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa41c0b6730 0x7fa41c0b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:45.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.503+0000 7fa4267fc640 1 -- 172.21.15.117:0/1557959509 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fa4200a92b0 con 0x7fa430111f40 2024-09-21T05:58:45.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.506+0000 7fa4354ef640 1 --2- 172.21.15.117:0/1557959509 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa41c0b6730 0x7fa41c0b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:45.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.514+0000 7fa4354ef640 1 --2- 172.21.15.117:0/1557959509 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa41c0b6730 0x7fa41c0b8c40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa42c0058c0 tx=0x7fa42c005ec0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:45.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:44.540+0000 7fa4267fc640 1 -- 172.21.15.117:0/1557959509 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa42001f020 con 0x7fa430111f40 2024-09-21T05:58:45.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:45.070+0000 7fa437753640 1 -- 172.21.15.117:0/1557959509 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4140055b0 con 0x7fa430111f40 2024-09-21T05:58:45.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:45.073+0000 7fa4267fc640 1 -- 172.21.15.117:0/1557959509 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+12001 (secure 0 0 0) 0x7fa4200ad740 con 0x7fa430111f40 2024-09-21T05:58:45.107 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 27 2024-09-21T05:58:45.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:45.079+0000 7fa437753640 1 -- 172.21.15.117:0/1557959509 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa41c0b6730 msgr2=0x7fa41c0b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:45.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:45.079+0000 7fa437753640 1 --2- 172.21.15.117:0/1557959509 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa41c0b6730 0x7fa41c0b8c40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa42c0058c0 tx=0x7fa42c005ec0 comp rx=0 tx=0).stop 2024-09-21T05:58:45.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:45.079+0000 7fa437753640 1 -- 172.21.15.117:0/1557959509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa430111f40 msgr2=0x7fa4301a96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:45.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:45.079+0000 7fa437753640 1 --2- 172.21.15.117:0/1557959509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa430111f40 0x7fa4301a96a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa43006c6e0 tx=0x7fa42000b850 comp rx=0 tx=0).stop 2024-09-21T05:58:45.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:45.079+0000 7fa437753640 1 -- 172.21.15.117:0/1557959509 shutdown_connections 2024-09-21T05:58:45.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:45.079+0000 7fa437753640 1 --2- 172.21.15.117:0/1557959509 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa41c0b6730 0x7fa41c0b8c40 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:45.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:45.079+0000 7fa437753640 1 --2- 172.21.15.117:0/1557959509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa430111f40 0x7fa4301a96a0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:45.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:45.079+0000 7fa437753640 1 --2- 172.21.15.117:0/1557959509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa43011cc30 0x7fa4301ae940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:45.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:45.079+0000 7fa437753640 1 --2- 172.21.15.117:0/1557959509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa43010a9e0 0x7fa4301a9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:45.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:45.079+0000 7fa437753640 1 -- 172.21.15.117:0/1557959509 >> 172.21.15.117:0/1557959509 conn(0x7fa430070ec0 msgr2=0x7fa430071e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:45.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:45.086+0000 7fa437753640 1 -- 172.21.15.117:0/1557959509 shutdown_connections 2024-09-21T05:58:45.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:45.091+0000 7fa437753640 1 -- 172.21.15.117:0/1557959509 wait complete. 2024-09-21T05:58:45.917 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:45 smithi138 ceph-mon[34391]: pgmap v370: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 927 MiB/s wr, 2.49k op/s 2024-09-21T05:58:45.917 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:45 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/1557959509' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:58:45.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:45 smithi192 ceph-mon[34700]: pgmap v370: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 927 MiB/s wr, 2.49k op/s 2024-09-21T05:58:45.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:45 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/1557959509' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:58:46.030 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:45 smithi117 ceph-mon[30947]: pgmap v370: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 927 MiB/s wr, 2.49k op/s 2024-09-21T05:58:46.030 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:45 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/1557959509' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T05:58:47.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:47 smithi117 ceph-mon[30947]: pgmap v371: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 901 MiB/s wr, 2.54k op/s 2024-09-21T05:58:47.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:47 smithi192 ceph-mon[34700]: pgmap v371: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 901 MiB/s wr, 2.54k op/s 2024-09-21T05:58:48.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:47 smithi138 ceph-mon[34391]: pgmap v371: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 901 MiB/s wr, 2.54k op/s 2024-09-21T05:58:49.207 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T05:58:49.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:49 smithi192 ceph-mon[34700]: pgmap v372: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 997 MiB/s wr, 2.92k op/s 2024-09-21T05:58:49.993 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:58:50.030 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:49 smithi117 ceph-mon[30947]: pgmap v372: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 997 MiB/s wr, 2.92k op/s 2024-09-21T05:58:50.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:49 smithi138 ceph-mon[34391]: pgmap v372: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 997 MiB/s wr, 2.92k op/s 2024-09-21T05:58:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.540+0000 7fb1faa39640 1 -- 172.21.15.117:0/383520344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1f410a900 msgr2=0x7fb1f410ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.540+0000 7fb1faa39640 1 --2- 172.21.15.117:0/383520344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1f410a900 0x7fb1f410ad10 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec009ca0 tx=0x7fb1ec02a8d0 comp rx=0 tx=0).stop 2024-09-21T05:58:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.541+0000 7fb1faa39640 1 -- 172.21.15.117:0/383520344 shutdown_connections 2024-09-21T05:58:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.541+0000 7fb1faa39640 1 --2- 172.21.15.117:0/383520344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1f410a900 0x7fb1f410ad10 secure :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec009ca0 tx=0x7fb1ec02a8d0 comp rx=0 tx=0).stop 2024-09-21T05:58:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.541+0000 7fb1faa39640 1 --2- 172.21.15.117:0/383520344 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1f411cba0 0x7fb1f411efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.541+0000 7fb1faa39640 1 --2- 172.21.15.117:0/383520344 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1f4111ca0 0x7fb1f410a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.541+0000 7fb1faa39640 1 -- 172.21.15.117:0/383520344 >> 172.21.15.117:0/383520344 conn(0x7fb1f4070c00 msgr2=0x7fb1f4071010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.543+0000 7fb1faa39640 1 -- 172.21.15.117:0/383520344 shutdown_connections 2024-09-21T05:58:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.548+0000 7fb1faa39640 1 -- 172.21.15.117:0/383520344 wait complete. 2024-09-21T05:58:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.549+0000 7fb1faa39640 1 Processor -- start 2024-09-21T05:58:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.549+0000 7fb1faa39640 1 -- start start 2024-09-21T05:58:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.550+0000 7fb1faa39640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1f4111ca0 0x7fb1f411b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.550+0000 7fb1faa39640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1f411cba0 0x7fb1f41147b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.550+0000 7fb1faa39640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1f4114de0 0x7fb1f411a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.550+0000 7fb1faa39640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f4122710 con 0x7fb1f4114de0 2024-09-21T05:58:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.550+0000 7fb1faa39640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f4122590 con 0x7fb1f411cba0 2024-09-21T05:58:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.550+0000 7fb1faa39640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f4122890 con 0x7fb1f4111ca0 2024-09-21T05:58:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.552+0000 7fb1f9a37640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1f4111ca0 0x7fb1f411b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.552+0000 7fb1f9a37640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1f4111ca0 0x7fb1f411b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:42800/0 (socket says 172.21.15.117:42800) 2024-09-21T05:58:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.552+0000 7fb1f9a37640 1 -- 172.21.15.117:0/1208685512 learned_addr learned my addr 172.21.15.117:0/1208685512 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.553+0000 7fb1fa238640 1 --2- 172.21.15.117:0/1208685512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1f4114de0 0x7fb1f411a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.555+0000 7fb1f9a37640 1 -- 172.21.15.117:0/1208685512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1f411cba0 msgr2=0x7fb1f41147b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.555+0000 7fb1f9a37640 1 --2- 172.21.15.117:0/1208685512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1f411cba0 0x7fb1f41147b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.555+0000 7fb1f9a37640 1 -- 172.21.15.117:0/1208685512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1f4114de0 msgr2=0x7fb1f411a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.555+0000 7fb1f9a37640 1 --2- 172.21.15.117:0/1208685512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1f4114de0 0x7fb1f411a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.555+0000 7fb1f9a37640 1 -- 172.21.15.117:0/1208685512 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f41155c0 con 0x7fb1f4111ca0 2024-09-21T05:58:51.561 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.559+0000 7fb1f9a37640 1 --2- 172.21.15.117:0/1208685512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1f4111ca0 0x7fb1f411b5b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb1f000c970 tx=0x7fb1f000ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.560+0000 7fb1eaffd640 1 -- 172.21.15.117:0/1208685512 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0007bb0 con 0x7fb1f4111ca0 2024-09-21T05:58:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.560+0000 7fb1eaffd640 1 -- 172.21.15.117:0/1208685512 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1f0007d50 con 0x7fb1f4111ca0 2024-09-21T05:58:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.562+0000 7fb1faa39640 1 -- 172.21.15.117:0/1208685512 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1f411a7b0 con 0x7fb1f4111ca0 2024-09-21T05:58:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.562+0000 7fb1faa39640 1 -- 172.21.15.117:0/1208685512 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1f411acd0 con 0x7fb1f4111ca0 2024-09-21T05:58:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.563+0000 7fb1eaffd640 1 -- 172.21.15.117:0/1208685512 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f000f450 con 0x7fb1f4111ca0 2024-09-21T05:58:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.571+0000 7fb202c0b740 1 Processor -- start 2024-09-21T05:58:51.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.572+0000 7fb202c0b740 1 -- start start 2024-09-21T05:58:51.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.572+0000 7fb202c0b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef6083f1b0 0x55ef60be4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:51.575 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.572+0000 7fb202c0b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef608162f0 0x55ef60be50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:51.575 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.573+0000 7fb202c0b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef6081e4d0 0x55ef60be5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:51.575 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.573+0000 7fb202c0b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ef60b596f0 con 0x55ef6081e4d0 2024-09-21T05:58:51.575 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.573+0000 7fb202c0b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ef60b48b20 con 0x55ef608162f0 2024-09-21T05:58:51.575 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.573+0000 7fb202c0b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55ef60b4c930 con 0x55ef6083f1b0 2024-09-21T05:58:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.574+0000 7fb1fa238640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef6083f1b0 0x55ef60be4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.574+0000 7fb1fa238640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef6083f1b0 0x55ef60be4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:42814/0 (socket says 172.21.15.117:42814) 2024-09-21T05:58:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.574+0000 7fb1fa238640 1 -- 172.21.15.117:0/3763309224 learned_addr learned my addr 172.21.15.117:0/3763309224 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.578+0000 7fb1fa238640 1 -- 172.21.15.117:0/3763309224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef608162f0 msgr2=0x55ef60be50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.578+0000 7fb1fa238640 1 --2- 172.21.15.117:0/3763309224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef608162f0 0x55ef60be50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.578+0000 7fb1fa238640 1 -- 172.21.15.117:0/3763309224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef6081e4d0 msgr2=0x55ef60be5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.578+0000 7fb1fa238640 1 --2- 172.21.15.117:0/3763309224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef6081e4d0 0x55ef60be5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.578+0000 7fb1fa238640 1 -- 172.21.15.117:0/3763309224 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef60b0e530 con 0x55ef6083f1b0 2024-09-21T05:58:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.579+0000 7fb1eaffd640 1 -- 172.21.15.117:0/1208685512 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fb1f0004f30 con 0x7fb1f4111ca0 2024-09-21T05:58:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.579+0000 7fb1fa238640 1 --2- 172.21.15.117:0/3763309224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef6083f1b0 0x55ef60be4ba0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40099f0 tx=0x7fb1e4009eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e58f5f67b9474806 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.581+0000 7fb1eaffd640 1 --2- 172.21.15.117:0/1208685512 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb1e00b66f0 0x7fb1e00b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.582+0000 7fb1eaffd640 1 -- 172.21.15.117:0/1208685512 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fb1f00e9730 con 0x7fb1f4111ca0 2024-09-21T05:58:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.583+0000 7fb1f9a37640 1 --2- 172.21.15.117:0/1208685512 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb1e00b66f0 0x7fb1e00b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.587+0000 7fb1f9a37640 1 --2- 172.21.15.117:0/1208685512 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb1e00b66f0 0x7fb1e00b8c00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55ef60be3820 tx=0x7fb1f00abd40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.588+0000 7fb1dffff640 1 -- 172.21.15.117:0/3763309224 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4017070 con 0x55ef6083f1b0 2024-09-21T05:58:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.589+0000 7fb1dffff640 1 -- 172.21.15.117:0/3763309224 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1e4013470 con 0x55ef6083f1b0 2024-09-21T05:58:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.589+0000 7fb1dffff640 1 -- 172.21.15.117:0/3763309224 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4012630 con 0x55ef6083f1b0 2024-09-21T05:58:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.590+0000 7fb202c0b740 1 -- 172.21.15.117:0/3763309224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef6083f1b0 msgr2=0x55ef60be4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.590+0000 7fb202c0b740 1 --2- 172.21.15.117:0/3763309224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef6083f1b0 0x55ef60be4ba0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40099f0 tx=0x7fb1e4009eb0 comp rx=0 tx=0).stop 2024-09-21T05:58:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.590+0000 7fb202c0b740 1 -- 172.21.15.117:0/3763309224 shutdown_connections 2024-09-21T05:58:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.590+0000 7fb202c0b740 1 --2- 172.21.15.117:0/3763309224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef6081e4d0 0x55ef60be5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.590+0000 7fb202c0b740 1 --2- 172.21.15.117:0/3763309224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef608162f0 0x55ef60be50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.590+0000 7fb202c0b740 1 --2- 172.21.15.117:0/3763309224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef6083f1b0 0x55ef60be4ba0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.590+0000 7fb202c0b740 1 -- 172.21.15.117:0/3763309224 >> 172.21.15.117:0/3763309224 conn(0x55ef60833300 msgr2=0x55ef60be2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.591+0000 7fb202c0b740 1 -- 172.21.15.117:0/3763309224 shutdown_connections 2024-09-21T05:58:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.591+0000 7fb202c0b740 1 -- 172.21.15.117:0/3763309224 wait complete. 2024-09-21T05:58:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.592+0000 7fb202c0b740 1 -- start start 2024-09-21T05:58:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.592+0000 7fb202c0b740 1 Processor -- start 2024-09-21T05:58:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.593+0000 7fb202c0b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef608162f0 0x55ef60bf0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.593+0000 7fb202c0b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef6081e4d0 0x55ef60bf07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.593+0000 7fb202c0b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef6083f1b0 0x55ef60bfc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.593+0000 7fb202c0b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ef60a80ab0 con 0x55ef6083f1b0 2024-09-21T05:58:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.593+0000 7fb202c0b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ef60aee730 con 0x55ef608162f0 2024-09-21T05:58:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.593+0000 7fb202c0b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55ef60bf3320 con 0x55ef6081e4d0 2024-09-21T05:58:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.594+0000 7fb202c0b740 10 client.0 _ll_register_callbacks cb 0x55ef60b426b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:58:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.594+0000 7fb1fa238640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef6083f1b0 0x55ef60bfc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.594+0000 7fb1fa238640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef6083f1b0 0x55ef60bfc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51114/0 (socket says 172.21.15.117:51114) 2024-09-21T05:58:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.594+0000 7fb1fa238640 1 -- 172.21.15.117:0/3120818810 learned_addr learned my addr 172.21.15.117:0/3120818810 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.594+0000 7fb1f9236640 1 --2- 172.21.15.117:0/3120818810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef6081e4d0 0x55ef60bf07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.594+0000 7fb1fa238640 1 -- 172.21.15.117:0/3120818810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef6081e4d0 msgr2=0x55ef60bf07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.594+0000 7fb1fa238640 1 --2- 172.21.15.117:0/3120818810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef6081e4d0 0x55ef60bf07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.594+0000 7fb1fa238640 1 -- 172.21.15.117:0/3120818810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef608162f0 msgr2=0x55ef60bf0290 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T05:58:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.594+0000 7fb1fa238640 1 --2- 172.21.15.117:0/3120818810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef608162f0 0x55ef60bf0290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.595+0000 7fb1fa238640 1 -- 172.21.15.117:0/3120818810 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef608aa790 con 0x55ef6083f1b0 2024-09-21T05:58:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.595+0000 7fb1fa238640 1 --2- 172.21.15.117:0/3120818810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef6083f1b0 0x55ef60bfc610 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4013e50 tx=0x7fb1e4013cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17071f334d37f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.595+0000 7fb1dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:58:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.595+0000 7fb1dd7fa640 1 -- 172.21.15.117:0/3120818810 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4017070 con 0x55ef6083f1b0 2024-09-21T05:58:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.595+0000 7fb202c0b740 1 -- 172.21.15.117:0/3120818810 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1e4013470 con 0x55ef6083f1b0 2024-09-21T05:58:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.595+0000 7fb1dd7fa640 1 -- 172.21.15.117:0/3120818810 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fb1e4013820 con 0x55ef6083f1b0 2024-09-21T05:58:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.595+0000 7fb1dd7fa640 1 -- 172.21.15.117:0/3120818810 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb1e4012850 con 0x55ef6083f1b0 2024-09-21T05:58:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.596+0000 7fb202c0b740 10 client.15867 fetch_fsmap learned FSMap version 27 2024-09-21T05:58:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.596+0000 7fb202c0b740 1 -- 172.21.15.117:0/3120818810 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ef60923620 con 0x55ef6083f1b0 2024-09-21T05:58:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.596+0000 7fb1dd7fa640 1 -- 172.21.15.117:0/3120818810 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1e4013470 con 0x55ef6083f1b0 2024-09-21T05:58:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.596+0000 7fb1dd7fa640 1 -- 172.21.15.117:0/3120818810 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb1e402ea40 con 0x55ef6083f1b0 2024-09-21T05:58:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.597+0000 7fb202c0b740 10 client.15867 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:58:51.600 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.597+0000 7fb202c0b740 10 client.15867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T05:58:51.600 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.597+0000 7fb202c0b740 20 client.15867 populate_metadata read hostname 'smithi117' 2024-09-21T05:58:51.600 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.597+0000 7fb202c0b740 1 --2- 172.21.15.117:0/3120818810 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ef607fdd80 0x55ef60c02280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:51.600 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.597+0000 7fb202c0b740 4 client.15867 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:58:51.600 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.597+0000 7fb202c0b740 1 -- 172.21.15.117:0/3120818810 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef60b5b640 con 0x55ef607fdd80 2024-09-21T05:58:51.600 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.599+0000 7fb1f9236640 1 --2- 172.21.15.117:0/3120818810 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ef607fdd80 0x55ef60c02280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.604+0000 7fb1dd7fa640 10 client.15867 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:58:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.604+0000 7fb1f9236640 1 --2- 172.21.15.117:0/3120818810 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ef607fdd80 0x55ef60c02280 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5184858860a4cd2c server_cookie=c67596a70a185d55 in_seq=0 out_seq=0 2024-09-21T05:58:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.611+0000 7fb1dd7fa640 1 -- 172.21.15.117:0/3120818810 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ef60b5b640 con 0x55ef607fdd80 2024-09-21T05:58:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.611+0000 7fb1dd7fa640 10 client.15867 handle_command_reply: tid=0 2024-09-21T05:58:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.611+0000 7fb202c0b740 1 client.15867 shutdown 2024-09-21T05:58:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.611+0000 7fb202c0b740 1 -- 172.21.15.117:0/3120818810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef6083f1b0 msgr2=0x55ef60bfc610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.611+0000 7fb202c0b740 1 --2- 172.21.15.117:0/3120818810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef6083f1b0 0x55ef60bfc610 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4013e50 tx=0x7fb1e4013cc0 comp rx=0 tx=0).stop 2024-09-21T05:58:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.611+0000 7fb202c0b740 1 -- 172.21.15.117:0/3120818810 shutdown_connections 2024-09-21T05:58:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.611+0000 7fb202c0b740 1 --2- 172.21.15.117:0/3120818810 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ef607fdd80 0x55ef60c02280 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.611+0000 7fb202c0b740 1 --2- 172.21.15.117:0/3120818810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef6083f1b0 0x55ef60bfc610 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.611+0000 7fb202c0b740 1 --2- 172.21.15.117:0/3120818810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef6081e4d0 0x55ef60bf07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.611+0000 7fb202c0b740 1 --2- 172.21.15.117:0/3120818810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef608162f0 0x55ef60bf0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.611+0000 7fb202c0b740 1 -- 172.21.15.117:0/3120818810 >> 172.21.15.117:0/3120818810 conn(0x55ef60833300 msgr2=0x55ef60be5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.611+0000 7fb1dd7fa640 0 client.15867 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:58:51.615 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.612+0000 7fb202c0b740 1 -- 172.21.15.117:0/3120818810 shutdown_connections 2024-09-21T05:58:51.615 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.612+0000 7fb202c0b740 1 -- 172.21.15.117:0/3120818810 wait complete. 2024-09-21T05:58:51.615 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.612+0000 7fb202c0b740 20 client.15867 trim_cache size 0 max 16384 2024-09-21T05:58:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.637+0000 7fb202c0b740 1 Processor -- start 2024-09-21T05:58:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.637+0000 7fb202c0b740 1 -- start start 2024-09-21T05:58:51.640 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.638+0000 7fb202c0b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef608162f0 0x55ef60beb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:51.640 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.638+0000 7fb202c0b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef6081e4d0 0x55ef60bf1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:51.641 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.638+0000 7fb202c0b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef6083f1b0 0x55ef60bed460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:51.641 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.639+0000 7fb202c0b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f41c6e60 con 0x55ef6083f1b0 2024-09-21T05:58:51.641 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.639+0000 7fb202c0b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f41c89a0 con 0x55ef6081e4d0 2024-09-21T05:58:51.641 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.639+0000 7fb202c0b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f41c7530 con 0x55ef608162f0 2024-09-21T05:58:51.641 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.639+0000 7fb1fa238640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef608162f0 0x55ef60beb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:51.641 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.639+0000 7fb1fa238640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef608162f0 0x55ef60beb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:42834/0 (socket says 172.21.15.117:42834) 2024-09-21T05:58:51.641 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.639+0000 7fb1fa238640 1 -- 172.21.15.117:0/1722494388 learned_addr learned my addr 172.21.15.117:0/1722494388 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:51.641 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.639+0000 7fb1fa238640 1 --2- 172.21.15.117:0/1722494388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef6083f1b0 0x55ef60bed460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:51.642 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.640+0000 7fb1f9236640 1 --2- 172.21.15.117:0/1722494388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef6081e4d0 0x55ef60bf1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:51.642 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.640+0000 7fb1fa238640 1 -- 172.21.15.117:0/1722494388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef6081e4d0 msgr2=0x55ef60bf1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:51.642 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.640+0000 7fb1fa238640 1 --2- 172.21.15.117:0/1722494388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef6081e4d0 0x55ef60bf1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:51.642 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.640+0000 7fb1fa238640 1 -- 172.21.15.117:0/1722494388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef6083f1b0 msgr2=0x55ef60bed460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:51.642 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.640+0000 7fb1fa238640 1 --2- 172.21.15.117:0/1722494388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef6083f1b0 0x55ef60bed460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.640+0000 7fb1fa238640 1 -- 172.21.15.117:0/1722494388 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef60bf1550 con 0x55ef608162f0 2024-09-21T05:58:51.643 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.641+0000 7fb1fa238640 1 --2- 172.21.15.117:0/1722494388 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef608162f0 0x55ef60beb9e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4000ea0 tx=0x7fb1e401e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d16c9f551f0ca05 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:51.644 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.642+0000 7fb1df7fe640 1 -- 172.21.15.117:0/1722494388 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4017050 con 0x55ef608162f0 2024-09-21T05:58:51.645 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.642+0000 7fb1df7fe640 1 -- 172.21.15.117:0/1722494388 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1e4035040 con 0x55ef608162f0 2024-09-21T05:58:51.645 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.642+0000 7fb1df7fe640 1 -- 172.21.15.117:0/1722494388 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4034070 con 0x55ef608162f0 2024-09-21T05:58:51.646 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.643+0000 7fb202c0b740 1 -- 172.21.15.117:0/1722494388 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef608162f0 msgr2=0x55ef60beb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:51.646 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.644+0000 7fb202c0b740 1 --2- 172.21.15.117:0/1722494388 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef608162f0 0x55ef60beb9e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4000ea0 tx=0x7fb1e401e040 comp rx=0 tx=0).stop 2024-09-21T05:58:51.646 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.644+0000 7fb202c0b740 1 -- 172.21.15.117:0/1722494388 shutdown_connections 2024-09-21T05:58:51.646 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.644+0000 7fb202c0b740 1 --2- 172.21.15.117:0/1722494388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef6083f1b0 0x55ef60bed460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:51.646 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.644+0000 7fb202c0b740 1 --2- 172.21.15.117:0/1722494388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef6081e4d0 0x55ef60bf1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:51.646 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.644+0000 7fb202c0b740 1 --2- 172.21.15.117:0/1722494388 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef608162f0 0x55ef60beb9e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:51.646 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.644+0000 7fb202c0b740 1 -- 172.21.15.117:0/1722494388 >> 172.21.15.117:0/1722494388 conn(0x55ef60833300 msgr2=0x55ef60bf17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:51.647 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.645+0000 7fb202c0b740 1 -- 172.21.15.117:0/1722494388 shutdown_connections 2024-09-21T05:58:51.648 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.646+0000 7fb202c0b740 1 -- 172.21.15.117:0/1722494388 wait complete. 2024-09-21T05:58:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.647+0000 7fb202c0b740 1 -- start start 2024-09-21T05:58:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.648+0000 7fb202c0b740 1 Processor -- start 2024-09-21T05:58:51.651 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.649+0000 7fb202c0b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef608162f0 0x55ef60c04aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:51.651 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.649+0000 7fb202c0b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef6081e4d0 0x55ef60c04fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:51.651 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.649+0000 7fb202c0b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef6083f1b0 0x55ef60c6aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:51.651 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.649+0000 7fb202c0b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f41c82d0 con 0x55ef6083f1b0 2024-09-21T05:58:51.651 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.649+0000 7fb202c0b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f41c7c00 con 0x55ef6081e4d0 2024-09-21T05:58:51.651 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.649+0000 7fb202c0b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f41c4e40 con 0x55ef608162f0 2024-09-21T05:58:51.652 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.649+0000 7fb202c0b740 10 client.0 _ll_register_callbacks cb 0x7fb1e4009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:58:51.652 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.650+0000 7fb1fa238640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef6083f1b0 0x55ef60c6aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:51.652 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.650+0000 7fb1fa238640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef6083f1b0 0x55ef60c6aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51136/0 (socket says 172.21.15.117:51136) 2024-09-21T05:58:51.652 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.650+0000 7fb1fa238640 1 -- 172.21.15.117:0/3200276006 learned_addr learned my addr 172.21.15.117:0/3200276006 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:51.652 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.650+0000 7fb1fa238640 1 -- 172.21.15.117:0/3200276006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef608162f0 msgr2=0x55ef60c04aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T05:58:51.653 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.650+0000 7fb1fa238640 1 --2- 172.21.15.117:0/3200276006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef608162f0 0x55ef60c04aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:51.653 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.650+0000 7fb1fa238640 1 -- 172.21.15.117:0/3200276006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef6081e4d0 msgr2=0x55ef60c04fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:51.653 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.650+0000 7fb1fa238640 1 --2- 172.21.15.117:0/3200276006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef6081e4d0 0x55ef60c04fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:51.653 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.650+0000 7fb1fa238640 1 -- 172.21.15.117:0/3200276006 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef60c6aff0 con 0x55ef6083f1b0 2024-09-21T05:58:51.653 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.651+0000 7fb1fa238640 1 --2- 172.21.15.117:0/3200276006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef6083f1b0 0x55ef60c6aa00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4006dc0 tx=0x7fb1e401cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98fea8c989fe4f39 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:51.653 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.651+0000 7fb1dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:58:51.653 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.651+0000 7fb1dd7fa640 1 -- 172.21.15.117:0/3200276006 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4034040 con 0x55ef6083f1b0 2024-09-21T05:58:51.654 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.652+0000 7fb1dd7fa640 1 -- 172.21.15.117:0/3200276006 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fb1e401cb20 con 0x55ef6083f1b0 2024-09-21T05:58:51.654 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.652+0000 7fb202c0b740 1 -- 172.21.15.117:0/3200276006 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1e4035040 con 0x55ef6083f1b0 2024-09-21T05:58:51.654 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.652+0000 7fb1dd7fa640 1 -- 172.21.15.117:0/3200276006 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb1e4017040 con 0x55ef6083f1b0 2024-09-21T05:58:51.654 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.652+0000 7fb202c0b740 10 client.15873 fetch_fsmap learned FSMap version 27 2024-09-21T05:58:51.654 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.652+0000 7fb202c0b740 1 -- 172.21.15.117:0/3200276006 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ef60c6c280 con 0x55ef6083f1b0 2024-09-21T05:58:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.653+0000 7fb1dd7fa640 1 -- 172.21.15.117:0/3200276006 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1e4035040 con 0x55ef6083f1b0 2024-09-21T05:58:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.654+0000 7fb1dd7fa640 1 -- 172.21.15.117:0/3200276006 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb1e4038020 con 0x55ef6083f1b0 2024-09-21T05:58:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.654+0000 7fb202c0b740 10 client.15873 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:58:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.654+0000 7fb202c0b740 10 client.15873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T05:58:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.654+0000 7fb202c0b740 20 client.15873 populate_metadata read hostname 'smithi117' 2024-09-21T05:58:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.655+0000 7fb202c0b740 1 --2- 172.21.15.117:0/3200276006 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ef607d31c0 0x55ef60c6ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.655+0000 7fb202c0b740 4 client.15873 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T05:58:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.655+0000 7fb202c0b740 1 -- 172.21.15.117:0/3200276006 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ef60b570f0 con 0x55ef607d31c0 2024-09-21T05:58:51.660 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.658+0000 7fb1f9236640 1 --2- 172.21.15.117:0/3200276006 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ef607d31c0 0x55ef60c6ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:51.666 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.665+0000 7fb1dd7fa640 10 client.15873 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:58:51.667 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:51.665+0000 7fb1f9236640 1 --2- 172.21.15.117:0/3200276006 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ef607d31c0 0x55ef60c6ea60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46a81e177400085e server_cookie=5d140e9c2b4879d in_seq=0 out_seq=0 2024-09-21T05:58:51.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:51 smithi192 ceph-mon[34700]: pgmap v373: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 801 MiB/s wr, 2.28k op/s 2024-09-21T05:58:52.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:51 smithi117 ceph-mon[30947]: pgmap v373: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 801 MiB/s wr, 2.28k op/s 2024-09-21T05:58:52.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:51 smithi138 ceph-mon[34391]: pgmap v373: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 801 MiB/s wr, 2.28k op/s 2024-09-21T05:58:52.741 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T05:58:52.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.370+0000 7f874eb10640 1 -- 172.21.15.117:0/2138575784 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f874811cc30 msgr2=0x7f874811f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:52.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.370+0000 7f874eb10640 1 --2- 172.21.15.117:0/2138575784 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f874811cc30 0x7f874811f140 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8740009860 tx=0x7f874002a380 comp rx=0 tx=0).stop 2024-09-21T05:58:52.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.374+0000 7f874eb10640 1 -- 172.21.15.117:0/2138575784 shutdown_connections 2024-09-21T05:58:52.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.374+0000 7f874eb10640 1 --2- 172.21.15.117:0/2138575784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f874810a9e0 0x7f874810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:52.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.374+0000 7f874eb10640 1 --2- 172.21.15.117:0/2138575784 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f874811cc30 0x7f874811f140 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:52.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.374+0000 7f874eb10640 1 --2- 172.21.15.117:0/2138575784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8748111f40 0x7f874810a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:52.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.374+0000 7f874eb10640 1 -- 172.21.15.117:0/2138575784 >> 172.21.15.117:0/2138575784 conn(0x7f8748070ec0 msgr2=0x7f87480712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:52.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.374+0000 7f874eb10640 1 -- 172.21.15.117:0/2138575784 shutdown_connections 2024-09-21T05:58:52.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.376+0000 7f874eb10640 1 -- 172.21.15.117:0/2138575784 wait complete. 2024-09-21T05:58:52.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.379+0000 7f874eb10640 1 Processor -- start 2024-09-21T05:58:52.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.379+0000 7f874eb10640 1 -- start start 2024-09-21T05:58:52.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.385+0000 7f874eb10640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f874810a9e0 0x7f87481a9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:52.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.385+0000 7f874eb10640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8748111f40 0x7f87481a97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:52.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.385+0000 7f874eb10640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f874811cc30 0x7f87481aea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:52.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.385+0000 7f874eb10640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f87481228b0 con 0x7f874811cc30 2024-09-21T05:58:52.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.385+0000 7f874eb10640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8748122730 con 0x7f8748111f40 2024-09-21T05:58:52.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.385+0000 7f874eb10640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8748122a30 con 0x7f874810a9e0 2024-09-21T05:58:52.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.386+0000 7f874d0ad640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f874811cc30 0x7f87481aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:52.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.386+0000 7f874d0ad640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f874811cc30 0x7f87481aea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51160/0 (socket says 172.21.15.117:51160) 2024-09-21T05:58:52.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.386+0000 7f874d0ad640 1 -- 172.21.15.117:0/800387445 learned_addr learned my addr 172.21.15.117:0/800387445 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:52.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.387+0000 7f874d0ad640 1 -- 172.21.15.117:0/800387445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f874810a9e0 msgr2=0x7f87481a9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:52.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.387+0000 7f874d0ad640 1 --2- 172.21.15.117:0/800387445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f874810a9e0 0x7f87481a9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:52.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.387+0000 7f874d0ad640 1 -- 172.21.15.117:0/800387445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8748111f40 msgr2=0x7f87481a97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:52.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.387+0000 7f874d0ad640 1 --2- 172.21.15.117:0/800387445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8748111f40 0x7f87481a97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:52.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.387+0000 7f874d0ad640 1 -- 172.21.15.117:0/800387445 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87481af1b0 con 0x7f874811cc30 2024-09-21T05:58:52.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.388+0000 7f874d0ad640 1 --2- 172.21.15.117:0/800387445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f874811cc30 0x7f87481aea80 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f874002a8f0 tx=0x7f8740005650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:52.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.389+0000 7f8745ffb640 1 -- 172.21.15.117:0/800387445 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8740040020 con 0x7f874811cc30 2024-09-21T05:58:52.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.389+0000 7f8745ffb640 1 -- 172.21.15.117:0/800387445 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8740004b90 con 0x7f874811cc30 2024-09-21T05:58:52.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.389+0000 7f8745ffb640 1 -- 172.21.15.117:0/800387445 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874003c6c0 con 0x7f874811cc30 2024-09-21T05:58:52.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.391+0000 7f874eb10640 1 -- 172.21.15.117:0/800387445 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87481af440 con 0x7f874811cc30 2024-09-21T05:58:52.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.391+0000 7f874eb10640 1 -- 172.21.15.117:0/800387445 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87481b5c00 con 0x7f874811cc30 2024-09-21T05:58:52.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.403+0000 7f8755084740 1 Processor -- start 2024-09-21T05:58:52.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.403+0000 7f8755084740 1 -- start start 2024-09-21T05:58:52.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.404+0000 7f8755084740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c6fe36070 0x560c701dd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:52.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.404+0000 7f8755084740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c6fe0d1b0 0x560c701ddbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:52.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.404+0000 7f8755084740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c6fe15390 0x560c701de120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:52.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.404+0000 7f8755084740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560c70153ea0 con 0x560c6fe0d1b0 2024-09-21T05:58:52.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.404+0000 7f8755084740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560c701432d0 con 0x560c6fe36070 2024-09-21T05:58:52.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.404+0000 7f8755084740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x560c701470e0 con 0x560c6fe15390 2024-09-21T05:58:52.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.404+0000 7f8745ffb640 1 -- 172.21.15.117:0/800387445 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f8740043020 con 0x7f874811cc30 2024-09-21T05:58:52.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.405+0000 7f8745ffb640 1 --2- 172.21.15.117:0/800387445 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f87340b6880 0x7f87340b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:52.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.406+0000 7f8745ffb640 1 -- 172.21.15.117:0/800387445 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f8740108040 con 0x7f874811cc30 2024-09-21T05:58:52.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.406+0000 7f8747fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c6fe36070 0x560c701dd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:52.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.406+0000 7f8747fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c6fe15390 0x560c701de120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:52.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.407+0000 7f874c8ac640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c6fe0d1b0 0x560c701ddbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:52.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.407+0000 7f874c8ac640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c6fe0d1b0 0x560c701ddbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51162/0 (socket says 172.21.15.117:51162) 2024-09-21T05:58:52.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.407+0000 7f874c8ac640 1 -- 172.21.15.117:0/4137067642 learned_addr learned my addr 172.21.15.117:0/4137067642 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:52.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.407+0000 7f874d0ad640 1 --2- 172.21.15.117:0/800387445 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f87340b6880 0x7f87340b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:52.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.407+0000 7f874c8ac640 1 -- 172.21.15.117:0/4137067642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c6fe15390 msgr2=0x560c701de120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:52.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.407+0000 7f874c8ac640 1 --2- 172.21.15.117:0/4137067642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c6fe15390 0x560c701de120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:52.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.407+0000 7f874c8ac640 1 -- 172.21.15.117:0/4137067642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c6fe36070 msgr2=0x560c701dd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:52.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.407+0000 7f874c8ac640 1 --2- 172.21.15.117:0/4137067642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c6fe36070 0x560c701dd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:52.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.408+0000 7f874c8ac640 1 -- 172.21.15.117:0/4137067642 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c6fea1650 con 0x560c6fe0d1b0 2024-09-21T05:58:52.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.408+0000 7f874c8ac640 1 --2- 172.21.15.117:0/4137067642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c6fe0d1b0 0x560c701ddbe0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8738007ce0 tx=0x7f873800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39c277fb35d64f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:52.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.408+0000 7f874d0ad640 1 --2- 172.21.15.117:0/800387445 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f87340b6880 0x7f87340b8d90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f87400ca480 tx=0x7f8740005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:52.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.409+0000 7f873effd640 1 -- 172.21.15.117:0/4137067642 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8738014070 con 0x560c6fe0d1b0 2024-09-21T05:58:52.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.410+0000 7f873effd640 1 -- 172.21.15.117:0/4137067642 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8738003b70 con 0x560c6fe0d1b0 2024-09-21T05:58:52.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.411+0000 7f8755084740 1 -- 172.21.15.117:0/4137067642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c6fe0d1b0 msgr2=0x560c701ddbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:52.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.411+0000 7f8755084740 1 --2- 172.21.15.117:0/4137067642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c6fe0d1b0 0x560c701ddbe0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8738007ce0 tx=0x7f873800c6a0 comp rx=0 tx=0).stop 2024-09-21T05:58:52.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.412+0000 7f8755084740 1 -- 172.21.15.117:0/4137067642 shutdown_connections 2024-09-21T05:58:52.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.412+0000 7f8755084740 1 --2- 172.21.15.117:0/4137067642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c6fe0d1b0 0x560c701ddbe0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:52.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.412+0000 7f8755084740 1 --2- 172.21.15.117:0/4137067642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c6fe15390 0x560c701de120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:52.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.412+0000 7f8755084740 1 --2- 172.21.15.117:0/4137067642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c6fe36070 0x560c701dd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:52.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.412+0000 7f8755084740 1 -- 172.21.15.117:0/4137067642 >> 172.21.15.117:0/4137067642 conn(0x560c6fe2a460 msgr2=0x560c701db590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:52.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.412+0000 7f8755084740 1 -- 172.21.15.117:0/4137067642 shutdown_connections 2024-09-21T05:58:52.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.412+0000 7f8755084740 1 -- 172.21.15.117:0/4137067642 wait complete. 2024-09-21T05:58:52.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.413+0000 7f8755084740 1 -- start start 2024-09-21T05:58:52.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.413+0000 7f8755084740 1 Processor -- start 2024-09-21T05:58:52.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.414+0000 7f8755084740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c6fe0d1b0 0x560c701e8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:52.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.414+0000 7f8755084740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c6fe15390 0x560c701e9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:52.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.414+0000 7f8755084740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c6fe36070 0x560c701e9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:52.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.414+0000 7f8755084740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560c6ffa4700 con 0x560c6fe15390 2024-09-21T05:58:52.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.414+0000 7f8755084740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560c700de160 con 0x560c6fe36070 2024-09-21T05:58:52.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.414+0000 7f8755084740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x560c701ebed0 con 0x560c6fe0d1b0 2024-09-21T05:58:52.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.414+0000 7f8755084740 10 client.0 _ll_register_callbacks cb 0x560c700f79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:58:52.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.416+0000 7f874c8ac640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c6fe15390 0x560c701e9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:52.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.417+0000 7f874c8ac640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c6fe15390 0x560c701e9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51168/0 (socket says 172.21.15.117:51168) 2024-09-21T05:58:52.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.417+0000 7f874c8ac640 1 -- 172.21.15.117:0/3191957900 learned_addr learned my addr 172.21.15.117:0/3191957900 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:52.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.417+0000 7f8747fff640 1 --2- 172.21.15.117:0/3191957900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c6fe36070 0x560c701e9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:52.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.417+0000 7f8747fff640 1 --2- 172.21.15.117:0/3191957900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c6fe0d1b0 0x560c701e8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:52.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.418+0000 7f874c8ac640 1 -- 172.21.15.117:0/3191957900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c6fe0d1b0 msgr2=0x560c701e8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:52.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.418+0000 7f874c8ac640 1 --2- 172.21.15.117:0/3191957900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c6fe0d1b0 0x560c701e8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:52.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.418+0000 7f874c8ac640 1 -- 172.21.15.117:0/3191957900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c6fe36070 msgr2=0x560c701e9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:52.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.418+0000 7f874c8ac640 1 --2- 172.21.15.117:0/3191957900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c6fe36070 0x560c701e9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:52.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.418+0000 7f874c8ac640 1 -- 172.21.15.117:0/3191957900 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c7010bc90 con 0x560c6fe15390 2024-09-21T05:58:52.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.419+0000 7f874c8ac640 1 --2- 172.21.15.117:0/3191957900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c6fe15390 0x560c701e9210 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8738004700 tx=0x7f8738007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769a7ae5558cb8be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:52.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.419+0000 7f8727fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:58:52.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.420+0000 7f8727fff640 1 -- 172.21.15.117:0/3191957900 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8738026060 con 0x560c6fe15390 2024-09-21T05:58:52.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.420+0000 7f8755084740 1 -- 172.21.15.117:0/3191957900 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8738003b70 con 0x560c6fe15390 2024-09-21T05:58:52.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.421+0000 7f8727fff640 1 -- 172.21.15.117:0/3191957900 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f8738003970 con 0x560c6fe15390 2024-09-21T05:58:52.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.426+0000 7f8727fff640 1 -- 172.21.15.117:0/3191957900 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8738014040 con 0x560c6fe15390 2024-09-21T05:58:52.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.426+0000 7f8755084740 10 client.15885 fetch_fsmap learned FSMap version 27 2024-09-21T05:58:52.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.426+0000 7f8755084740 1 -- 172.21.15.117:0/3191957900 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560c701f7bf0 con 0x560c6fe15390 2024-09-21T05:58:52.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.428+0000 7f8727fff640 1 -- 172.21.15.117:0/3191957900 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8738003b70 con 0x560c6fe15390 2024-09-21T05:58:52.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.429+0000 7f8727fff640 1 -- 172.21.15.117:0/3191957900 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8738018600 con 0x560c6fe15390 2024-09-21T05:58:52.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.429+0000 7f8755084740 10 client.15885 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:58:52.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.429+0000 7f8755084740 10 client.15885 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T05:58:52.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.429+0000 7f8755084740 20 client.15885 populate_metadata read hostname 'smithi117' 2024-09-21T05:58:52.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.429+0000 7f8755084740 1 --2- 172.21.15.117:0/3191957900 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x560c6fdf49f0 0x560c701fb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:52.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.429+0000 7f8755084740 4 client.15885 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:58:52.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.429+0000 7f8755084740 1 -- 172.21.15.117:0/3191957900 --> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c70155df0 con 0x560c6fdf49f0 2024-09-21T05:58:52.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.431+0000 7f8747fff640 1 --2- 172.21.15.117:0/3191957900 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x560c6fdf49f0 0x560c701fb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:52.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.437+0000 7f8727fff640 10 client.15885 ms_handle_connect on v2:172.21.15.138:6836/2540335374 2024-09-21T05:58:52.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.437+0000 7f8747fff640 1 --2- 172.21.15.117:0/3191957900 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x560c6fdf49f0 0x560c701fb390 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=60c7d0ec39acd3bc server_cookie=8ca67e075aa0a07c in_seq=0 out_seq=0 2024-09-21T05:58:52.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.441+0000 7f8727fff640 1 -- 172.21.15.117:0/3191957900 <== mds.4 v2:172.21.15.138:6836/2540335374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560c70155df0 con 0x560c6fdf49f0 2024-09-21T05:58:52.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.441+0000 7f8727fff640 10 client.15885 handle_command_reply: tid=0 2024-09-21T05:58:52.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.443+0000 7f8755084740 1 client.15885 shutdown 2024-09-21T05:58:52.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.445+0000 7f8755084740 1 -- 172.21.15.117:0/3191957900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c6fe15390 msgr2=0x560c701e9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:52.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.446+0000 7f8755084740 1 --2- 172.21.15.117:0/3191957900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c6fe15390 0x560c701e9210 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8738004700 tx=0x7f8738007a50 comp rx=0 tx=0).stop 2024-09-21T05:58:52.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.449+0000 7f8755084740 1 -- 172.21.15.117:0/3191957900 shutdown_connections 2024-09-21T05:58:52.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.449+0000 7f8755084740 1 --2- 172.21.15.117:0/3191957900 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x560c6fdf49f0 0x560c701fb390 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:52.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.449+0000 7f8755084740 1 --2- 172.21.15.117:0/3191957900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c6fe15390 0x560c701e9210 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:52.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.449+0000 7f8755084740 1 --2- 172.21.15.117:0/3191957900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c6fe36070 0x560c701e9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:52.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.449+0000 7f8755084740 1 --2- 172.21.15.117:0/3191957900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c6fe0d1b0 0x560c701e8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:52.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.449+0000 7f8755084740 1 -- 172.21.15.117:0/3191957900 >> 172.21.15.117:0/3191957900 conn(0x560c6fe2a460 msgr2=0x560c701e0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:52.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.450+0000 7f8727fff640 0 client.15885 ms_handle_reset on v2:172.21.15.138:6836/2540335374 2024-09-21T05:58:52.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.452+0000 7f8755084740 1 -- 172.21.15.117:0/3191957900 shutdown_connections 2024-09-21T05:58:52.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.453+0000 7f8755084740 1 -- 172.21.15.117:0/3191957900 wait complete. 2024-09-21T05:58:52.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.453+0000 7f8755084740 20 client.15885 trim_cache size 0 max 16384 2024-09-21T05:58:52.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.482+0000 7f8755084740 1 Processor -- start 2024-09-21T05:58:52.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.482+0000 7f8755084740 1 -- start start 2024-09-21T05:58:52.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.482+0000 7f8755084740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c6fe0d1b0 0x560c701e8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:52.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.482+0000 7f8755084740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c6fe15390 0x560c701e8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:52.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.483+0000 7f8747fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c6fe0d1b0 0x560c701e8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:52.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.483+0000 7f8747fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c6fe0d1b0 0x560c701e8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51170/0 (socket says 172.21.15.117:51170) 2024-09-21T05:58:52.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.483+0000 7f8747fff640 1 -- 172.21.15.117:0/372039507 learned_addr learned my addr 172.21.15.117:0/372039507 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:52.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.483+0000 7f8755084740 1 --2- 172.21.15.117:0/372039507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c6fe36070 0x560c701e5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:52.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.483+0000 7f8755084740 1 -- 172.21.15.117:0/372039507 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f87481be3b0 con 0x560c6fe0d1b0 2024-09-21T05:58:52.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.483+0000 7f8755084740 1 -- 172.21.15.117:0/372039507 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f87481bfef0 con 0x560c6fe15390 2024-09-21T05:58:52.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.483+0000 7f8755084740 1 -- 172.21.15.117:0/372039507 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f87481bea80 con 0x560c6fe36070 2024-09-21T05:58:52.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.484+0000 7f874c8ac640 1 --2- 172.21.15.117:0/372039507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c6fe15390 0x560c701e8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:52.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.484+0000 7f8747fff640 1 --2- 172.21.15.117:0/372039507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c6fe36070 0x560c701e5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:52.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.484+0000 7f8747fff640 1 -- 172.21.15.117:0/372039507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c6fe36070 msgr2=0x560c701e5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:52.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.484+0000 7f8747fff640 1 --2- 172.21.15.117:0/372039507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c6fe36070 0x560c701e5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:52.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.484+0000 7f8747fff640 1 -- 172.21.15.117:0/372039507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c6fe15390 msgr2=0x560c701e8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:52.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.484+0000 7f8747fff640 1 --2- 172.21.15.117:0/372039507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c6fe15390 0x560c701e8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:52.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.484+0000 7f8747fff640 1 -- 172.21.15.117:0/372039507 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c701e9d70 con 0x560c6fe0d1b0 2024-09-21T05:58:52.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.486+0000 7f8747fff640 1 --2- 172.21.15.117:0/372039507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c6fe0d1b0 0x560c701e8a80 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f874806c970 tx=0x7f87300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a59c8722fbbc95 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:52.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.487+0000 7f873e7fc640 1 -- 172.21.15.117:0/372039507 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873002d070 con 0x560c6fe0d1b0 2024-09-21T05:58:52.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.487+0000 7f873e7fc640 1 -- 172.21.15.117:0/372039507 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8730010c40 con 0x560c6fe0d1b0 2024-09-21T05:58:52.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.487+0000 7f873e7fc640 1 -- 172.21.15.117:0/372039507 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8730021610 con 0x560c6fe0d1b0 2024-09-21T05:58:52.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.489+0000 7f8755084740 1 -- 172.21.15.117:0/372039507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c6fe0d1b0 msgr2=0x560c701e8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:52.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.489+0000 7f8755084740 1 --2- 172.21.15.117:0/372039507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c6fe0d1b0 0x560c701e8a80 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f874806c970 tx=0x7f87300046c0 comp rx=0 tx=0).stop 2024-09-21T05:58:52.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.489+0000 7f8755084740 1 -- 172.21.15.117:0/372039507 shutdown_connections 2024-09-21T05:58:52.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.489+0000 7f8755084740 1 --2- 172.21.15.117:0/372039507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c6fe36070 0x560c701e5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:52.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.490+0000 7f8755084740 1 --2- 172.21.15.117:0/372039507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c6fe15390 0x560c701e8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:52.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.490+0000 7f8755084740 1 --2- 172.21.15.117:0/372039507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c6fe0d1b0 0x560c701e8a80 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:52.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.490+0000 7f8755084740 1 -- 172.21.15.117:0/372039507 >> 172.21.15.117:0/372039507 conn(0x560c6fe2a460 msgr2=0x560c701e7550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:52.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.490+0000 7f8755084740 1 -- 172.21.15.117:0/372039507 shutdown_connections 2024-09-21T05:58:52.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.490+0000 7f8755084740 1 -- 172.21.15.117:0/372039507 wait complete. 2024-09-21T05:58:52.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.491+0000 7f8755084740 1 -- start start 2024-09-21T05:58:52.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.491+0000 7f8755084740 1 Processor -- start 2024-09-21T05:58:52.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.492+0000 7f8755084740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c6fe0d1b0 0x560c701ede80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:52.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.493+0000 7f8755084740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c6fe15390 0x560c701ee3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:52.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.493+0000 7f8755084740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c6fe36070 0x560c70274d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:52.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.493+0000 7f8755084740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f87481bf820 con 0x560c6fe0d1b0 2024-09-21T05:58:52.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.493+0000 7f8755084740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f87481bf150 con 0x560c6fe36070 2024-09-21T05:58:52.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.493+0000 7f8755084740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f87481bc390 con 0x560c6fe15390 2024-09-21T05:58:52.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.493+0000 7f8755084740 10 client.0 _ll_register_callbacks cb 0x7f87300099a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:58:52.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.493+0000 7f8747fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c6fe0d1b0 0x560c701ede80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:52.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.493+0000 7f8747fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c6fe0d1b0 0x560c701ede80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51180/0 (socket says 172.21.15.117:51180) 2024-09-21T05:58:52.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.493+0000 7f8747fff640 1 -- 172.21.15.117:0/616415686 learned_addr learned my addr 172.21.15.117:0/616415686 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:58:52.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.494+0000 7f8747fff640 1 --2- 172.21.15.117:0/616415686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c6fe36070 0x560c70274d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:52.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.494+0000 7f8747fff640 1 -- 172.21.15.117:0/616415686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c6fe15390 msgr2=0x560c701ee3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:52.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.494+0000 7f8747fff640 1 --2- 172.21.15.117:0/616415686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c6fe15390 0x560c701ee3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:52.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.494+0000 7f8747fff640 1 -- 172.21.15.117:0/616415686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c6fe36070 msgr2=0x560c70274d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:58:52.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.494+0000 7f8747fff640 1 --2- 172.21.15.117:0/616415686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c6fe36070 0x560c70274d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:52.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.494+0000 7f8747fff640 1 -- 172.21.15.117:0/616415686 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c70275320 con 0x560c6fe0d1b0 2024-09-21T05:58:52.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.495+0000 7f8747fff640 1 --2- 172.21.15.117:0/616415686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c6fe0d1b0 0x560c701ede80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8730010c10 tx=0x7f8730009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c21907676a09305b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:58:52.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.495+0000 7f8727fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:58:52.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.495+0000 7f8727fff640 1 -- 172.21.15.117:0/616415686 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873002d040 con 0x560c6fe0d1b0 2024-09-21T05:58:52.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.496+0000 7f8755084740 1 -- 172.21.15.117:0/616415686 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8730010c40 con 0x560c6fe0d1b0 2024-09-21T05:58:52.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.496+0000 7f8727fff640 1 -- 172.21.15.117:0/616415686 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f8730013b50 con 0x560c6fe0d1b0 2024-09-21T05:58:52.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.496+0000 7f8727fff640 1 -- 172.21.15.117:0/616415686 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8730013d50 con 0x560c6fe0d1b0 2024-09-21T05:58:52.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.496+0000 7f8755084740 10 client.15891 fetch_fsmap learned FSMap version 27 2024-09-21T05:58:52.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.496+0000 7f8755084740 1 -- 172.21.15.117:0/616415686 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560c702765a0 con 0x560c6fe0d1b0 2024-09-21T05:58:52.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.497+0000 7f8727fff640 1 -- 172.21.15.117:0/616415686 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8730010c40 con 0x560c6fe0d1b0 2024-09-21T05:58:52.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.497+0000 7f8727fff640 1 -- 172.21.15.117:0/616415686 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8730027030 con 0x560c6fe0d1b0 2024-09-21T05:58:52.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.497+0000 7f8755084740 10 client.15891 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:58:52.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.497+0000 7f8755084740 10 client.15891 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T05:58:52.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.497+0000 7f8755084740 20 client.15891 populate_metadata read hostname 'smithi117' 2024-09-21T05:58:52.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.497+0000 7f8755084740 1 --2- 172.21.15.117:0/616415686 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x560c6fdc9da0 0x560c70278ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:58:52.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.498+0000 7f8755084740 4 client.15891 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T05:58:52.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.498+0000 7f8755084740 1 -- 172.21.15.117:0/616415686 --> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560c701518a0 con 0x560c6fdc9da0 2024-09-21T05:58:52.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.498+0000 7f874c8ac640 1 --2- 172.21.15.117:0/616415686 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x560c6fdc9da0 0x560c70278ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:58:52.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.501+0000 7f8727fff640 10 client.15891 ms_handle_connect on v2:172.21.15.138:6836/2540335374 2024-09-21T05:58:52.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.501+0000 7f874c8ac640 1 --2- 172.21.15.117:0/616415686 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x560c6fdc9da0 0x560c70278ce0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8de6644369e9e31c server_cookie=39f3edd113357028 in_seq=0 out_seq=0 2024-09-21T05:58:52.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.503+0000 7f8727fff640 1 -- 172.21.15.117:0/616415686 <== mds.4 v2:172.21.15.138:6836/2540335374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560c701518a0 con 0x560c6fdc9da0 2024-09-21T05:58:52.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.503+0000 7f8727fff640 10 client.15891 handle_command_reply: tid=0 2024-09-21T05:58:52.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.503+0000 7f8755084740 1 client.15891 shutdown 2024-09-21T05:58:52.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.503+0000 7f8755084740 1 -- 172.21.15.117:0/616415686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c6fe0d1b0 msgr2=0x560c701ede80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:52.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.503+0000 7f8755084740 1 --2- 172.21.15.117:0/616415686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c6fe0d1b0 0x560c701ede80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8730010c10 tx=0x7f8730009a70 comp rx=0 tx=0).stop 2024-09-21T05:58:52.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.503+0000 7f8755084740 1 -- 172.21.15.117:0/616415686 shutdown_connections 2024-09-21T05:58:52.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.503+0000 7f8755084740 1 --2- 172.21.15.117:0/616415686 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x560c6fdc9da0 0x560c70278ce0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:52.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.504+0000 7f8727fff640 0 client.15891 ms_handle_reset on v2:172.21.15.138:6836/2540335374 2024-09-21T05:58:52.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.504+0000 7f8755084740 1 --2- 172.21.15.117:0/616415686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c6fe36070 0x560c70274d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:52.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.504+0000 7f8755084740 1 --2- 172.21.15.117:0/616415686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c6fe15390 0x560c701ee3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:52.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.504+0000 7f8755084740 1 --2- 172.21.15.117:0/616415686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c6fe0d1b0 0x560c701ede80 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:52.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.504+0000 7f8755084740 1 -- 172.21.15.117:0/616415686 >> 172.21.15.117:0/616415686 conn(0x560c6fe2a460 msgr2=0x560c701e8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:52.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.504+0000 7f8755084740 1 -- 172.21.15.117:0/616415686 shutdown_connections 2024-09-21T05:58:52.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.504+0000 7f8755084740 1 -- 172.21.15.117:0/616415686 wait complete. 2024-09-21T05:58:52.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.504+0000 7f8755084740 20 client.15891 trim_cache size 0 max 16384 2024-09-21T05:58:52.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.505+0000 7f874eb10640 1 -- 172.21.15.117:0/800387445 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f87340b6880 msgr2=0x7f87340b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:52.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.505+0000 7f874eb10640 1 --2- 172.21.15.117:0/800387445 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f87340b6880 0x7f87340b8d90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f87400ca480 tx=0x7f8740005f90 comp rx=0 tx=0).stop 2024-09-21T05:58:52.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.505+0000 7f874eb10640 1 -- 172.21.15.117:0/800387445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f874811cc30 msgr2=0x7f87481aea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:52.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.505+0000 7f874eb10640 1 --2- 172.21.15.117:0/800387445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f874811cc30 0x7f87481aea80 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f874002a8f0 tx=0x7f8740005650 comp rx=0 tx=0).stop 2024-09-21T05:58:52.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.506+0000 7f874eb10640 1 -- 172.21.15.117:0/800387445 shutdown_connections 2024-09-21T05:58:52.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.506+0000 7f874eb10640 1 --2- 172.21.15.117:0/800387445 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f87340b6880 0x7f87340b8d90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:52.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.506+0000 7f874eb10640 1 --2- 172.21.15.117:0/800387445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f874811cc30 0x7f87481aea80 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:52.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.506+0000 7f874eb10640 1 --2- 172.21.15.117:0/800387445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8748111f40 0x7f87481a97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:52.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.506+0000 7f874eb10640 1 --2- 172.21.15.117:0/800387445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f874810a9e0 0x7f87481a9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:52.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.506+0000 7f874eb10640 1 -- 172.21.15.117:0/800387445 >> 172.21.15.117:0/800387445 conn(0x7f8748070ec0 msgr2=0x7f874811e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:52.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.506+0000 7f874eb10640 1 -- 172.21.15.117:0/800387445 shutdown_connections 2024-09-21T05:58:52.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:52.506+0000 7f874eb10640 1 -- 172.21.15.117:0/800387445 wait complete. 2024-09-21T05:58:53.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:53 smithi192 ceph-mon[34700]: pgmap v374: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 921 MiB/s wr, 2.78k op/s 2024-09-21T05:58:54.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:53 smithi117 ceph-mon[30947]: pgmap v374: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 921 MiB/s wr, 2.78k op/s 2024-09-21T05:58:54.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:53 smithi138 ceph-mon[34391]: pgmap v374: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 921 MiB/s wr, 2.78k op/s 2024-09-21T05:58:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:55.237+0000 7fb1dd7fa640 1 -- 172.21.15.117:0/3200276006 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ef60b570f0 con 0x55ef607d31c0 2024-09-21T05:58:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:55.237+0000 7fb1dd7fa640 10 client.15873 handle_command_reply: tid=0 2024-09-21T05:58:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:55.237+0000 7fb202c0b740 1 client.15873 shutdown 2024-09-21T05:58:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:55.238+0000 7fb202c0b740 1 -- 172.21.15.117:0/3200276006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef6083f1b0 msgr2=0x55ef60c6aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:55.238+0000 7fb202c0b740 1 --2- 172.21.15.117:0/3200276006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef6083f1b0 0x55ef60c6aa00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4006dc0 tx=0x7fb1e401cf00 comp rx=0 tx=0).stop 2024-09-21T05:58:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:55.238+0000 7fb202c0b740 1 -- 172.21.15.117:0/3200276006 shutdown_connections 2024-09-21T05:58:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:55.238+0000 7fb202c0b740 1 --2- 172.21.15.117:0/3200276006 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ef607d31c0 0x55ef60c6ea60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:55.238+0000 7fb202c0b740 1 --2- 172.21.15.117:0/3200276006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ef6083f1b0 0x55ef60c6aa00 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:55.238+0000 7fb202c0b740 1 --2- 172.21.15.117:0/3200276006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ef6081e4d0 0x55ef60c04fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:55.238+0000 7fb202c0b740 1 --2- 172.21.15.117:0/3200276006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ef608162f0 0x55ef60c04aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:55.238+0000 7fb202c0b740 1 -- 172.21.15.117:0/3200276006 >> 172.21.15.117:0/3200276006 conn(0x55ef60833300 msgr2=0x55ef60bf1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:55.238+0000 7fb1dd7fa640 0 client.15873 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:58:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:55.238+0000 7fb202c0b740 1 -- 172.21.15.117:0/3200276006 shutdown_connections 2024-09-21T05:58:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:55.238+0000 7fb202c0b740 1 -- 172.21.15.117:0/3200276006 wait complete. 2024-09-21T05:58:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:55.239+0000 7fb202c0b740 20 client.15873 trim_cache size 0 max 16384 2024-09-21T05:58:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T05:58:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T05:58:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T05:58:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T05:58:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:55.241+0000 7fb1faa39640 1 -- 172.21.15.117:0/1208685512 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb1e00b66f0 msgr2=0x7fb1e00b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:55.241+0000 7fb1faa39640 1 --2- 172.21.15.117:0/1208685512 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb1e00b66f0 0x7fb1e00b8c00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55ef60be3820 tx=0x7fb1f00abd40 comp rx=0 tx=0).stop 2024-09-21T05:58:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:55.241+0000 7fb1faa39640 1 -- 172.21.15.117:0/1208685512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1f4111ca0 msgr2=0x7fb1f411b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:58:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:55.241+0000 7fb1faa39640 1 --2- 172.21.15.117:0/1208685512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1f4111ca0 0x7fb1f411b5b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb1f000c970 tx=0x7fb1f000ce30 comp rx=0 tx=0).stop 2024-09-21T05:58:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:55.241+0000 7fb1faa39640 1 -- 172.21.15.117:0/1208685512 shutdown_connections 2024-09-21T05:58:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:55.242+0000 7fb1faa39640 1 --2- 172.21.15.117:0/1208685512 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb1e00b66f0 0x7fb1e00b8c00 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:55.242+0000 7fb1faa39640 1 --2- 172.21.15.117:0/1208685512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1f4114de0 0x7fb1f411a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:55.242+0000 7fb1faa39640 1 --2- 172.21.15.117:0/1208685512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1f411cba0 0x7fb1f41147b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:55.242+0000 7fb1faa39640 1 --2- 172.21.15.117:0/1208685512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1f4111ca0 0x7fb1f411b5b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:58:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:55.242+0000 7fb1faa39640 1 -- 172.21.15.117:0/1208685512 >> 172.21.15.117:0/1208685512 conn(0x7fb1f4070c00 msgr2=0x7fb1f4110920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:58:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:55.242+0000 7fb1faa39640 1 -- 172.21.15.117:0/1208685512 shutdown_connections 2024-09-21T05:58:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:58:55.242+0000 7fb1faa39640 1 -- 172.21.15.117:0/1208685512 wait complete. 2024-09-21T05:58:55.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:55 smithi192 ceph-mon[34700]: pgmap v375: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 929 MiB/s wr, 2.86k op/s 2024-09-21T05:58:56.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:55 smithi117 ceph-mon[30947]: pgmap v375: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 929 MiB/s wr, 2.86k op/s 2024-09-21T05:58:56.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:55 smithi138 ceph-mon[34391]: pgmap v375: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 929 MiB/s wr, 2.86k op/s 2024-09-21T05:58:56.556 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T05:58:57.123 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:58:57.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:57 smithi192 ceph-mon[34700]: pgmap v376: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 891 MiB/s wr, 2.67k op/s 2024-09-21T05:58:58.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:57 smithi117 ceph-mon[30947]: pgmap v376: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 891 MiB/s wr, 2.67k op/s 2024-09-21T05:58:58.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:57 smithi138 ceph-mon[34391]: pgmap v376: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 891 MiB/s wr, 2.67k op/s 2024-09-21T05:58:58.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:58:58 smithi117 ceph-mon[30947]: pgmap v377: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 983 MiB/s wr, 2.60k op/s 2024-09-21T05:58:58.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:58:58 smithi192 ceph-mon[34700]: pgmap v377: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 983 MiB/s wr, 2.60k op/s 2024-09-21T05:58:59.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:58:58 smithi138 ceph-mon[34391]: pgmap v377: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 983 MiB/s wr, 2.60k op/s 2024-09-21T05:59:00.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.429+0000 7f9e58808640 1 -- 172.21.15.117:0/3432014763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e54107b30 msgr2=0x7f9e54107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:00.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.429+0000 7f9e58808640 1 --2- 172.21.15.117:0/3432014763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e54107b30 0x7f9e54107f40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9e48009900 tx=0x7f9e4802a410 comp rx=0 tx=0).stop 2024-09-21T05:59:00.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.429+0000 7f9e58808640 1 -- 172.21.15.117:0/3432014763 shutdown_connections 2024-09-21T05:59:00.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.429+0000 7f9e58808640 1 --2- 172.21.15.117:0/3432014763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e54107b30 0x7f9e54107f40 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:00.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.429+0000 7f9e58808640 1 --2- 172.21.15.117:0/3432014763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e54108480 0x7f9e5410ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:00.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.429+0000 7f9e58808640 1 --2- 172.21.15.117:0/3432014763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e541021f0 0x7f9e54102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:00.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.429+0000 7f9e58808640 1 -- 172.21.15.117:0/3432014763 >> 172.21.15.117:0/3432014763 conn(0x7f9e54077c60 msgr2=0x7f9e540780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:00.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.430+0000 7f9e58808640 1 -- 172.21.15.117:0/3432014763 shutdown_connections 2024-09-21T05:59:00.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.430+0000 7f9e58808640 1 -- 172.21.15.117:0/3432014763 wait complete. 2024-09-21T05:59:00.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.431+0000 7f9e58808640 1 Processor -- start 2024-09-21T05:59:00.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.431+0000 7f9e58808640 1 -- start start 2024-09-21T05:59:00.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.431+0000 7f9e58808640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e541021f0 0x7f9e5419b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:00.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.431+0000 7f9e58808640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e54107b30 0x7f9e5419b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:00.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.432+0000 7f9e58808640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e54108480 0x7f9e5419bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:00.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.432+0000 7f9e58808640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9e54112ae0 con 0x7f9e541021f0 2024-09-21T05:59:00.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.432+0000 7f9e58808640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9e54112960 con 0x7f9e54107b30 2024-09-21T05:59:00.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.432+0000 7f9e58808640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9e54112c60 con 0x7f9e54108480 2024-09-21T05:59:00.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.432+0000 7f9e5259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e541021f0 0x7f9e5419b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:00.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.432+0000 7f9e5259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e541021f0 0x7f9e5419b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44476/0 (socket says 172.21.15.117:44476) 2024-09-21T05:59:00.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.432+0000 7f9e5259c640 1 -- 172.21.15.117:0/2665228370 learned_addr learned my addr 172.21.15.117:0/2665228370 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:00.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.432+0000 7f9e5259c640 1 -- 172.21.15.117:0/2665228370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e54108480 msgr2=0x7f9e5419bef0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T05:59:00.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.432+0000 7f9e5259c640 1 --2- 172.21.15.117:0/2665228370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e54108480 0x7f9e5419bef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:00.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.432+0000 7f9e5259c640 1 -- 172.21.15.117:0/2665228370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e54107b30 msgr2=0x7f9e5419b9b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T05:59:00.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.432+0000 7f9e5259c640 1 --2- 172.21.15.117:0/2665228370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e54107b30 0x7f9e5419b9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:00.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.432+0000 7f9e5259c640 1 -- 172.21.15.117:0/2665228370 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e541a6730 con 0x7f9e541021f0 2024-09-21T05:59:00.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.433+0000 7f9e5259c640 1 --2- 172.21.15.117:0/2665228370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e541021f0 0x7f9e5419b470 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9e4400b450 tx=0x7f9e4400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:00.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.433+0000 7f9e437fe640 1 -- 172.21.15.117:0/2665228370 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e44015020 con 0x7f9e541021f0 2024-09-21T05:59:00.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.433+0000 7f9e437fe640 1 -- 172.21.15.117:0/2665228370 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e44011d40 con 0x7f9e541021f0 2024-09-21T05:59:00.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.433+0000 7f9e437fe640 1 -- 172.21.15.117:0/2665228370 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4400f340 con 0x7f9e541021f0 2024-09-21T05:59:00.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.433+0000 7f9e58808640 1 -- 172.21.15.117:0/2665228370 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e541a6a20 con 0x7f9e541021f0 2024-09-21T05:59:00.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.433+0000 7f9e58808640 1 -- 172.21.15.117:0/2665228370 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e541a6f90 con 0x7f9e541021f0 2024-09-21T05:59:00.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.439+0000 7f9e5ed7c740 1 Processor -- start 2024-09-21T05:59:00.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.440+0000 7f9e437fe640 1 -- 172.21.15.117:0/2665228370 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f9e44018020 con 0x7f9e541021f0 2024-09-21T05:59:00.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.442+0000 7f9e5ed7c740 1 -- start start 2024-09-21T05:59:00.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.442+0000 7f9e437fe640 1 --2- 172.21.15.117:0/2665228370 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9e3c0b6780 0x7f9e3c0b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:00.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.443+0000 7f9e437fe640 1 -- 172.21.15.117:0/2665228370 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f9e440e7160 con 0x7f9e541021f0 2024-09-21T05:59:00.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.443+0000 7f9e5ed7c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf5b8ef070 0x55cf5bc966a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:00.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.443+0000 7f9e5ed7c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5b8c61b0 0x55cf5bc96be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:00.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.443+0000 7f9e5ed7c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf5b8ce390 0x55cf5bc97120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:00.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.443+0000 7f9e5ed7c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cf5bc0cea0 con 0x55cf5b8ef070 2024-09-21T05:59:00.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.443+0000 7f9e5ed7c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cf5bbfc2d0 con 0x55cf5b8c61b0 2024-09-21T05:59:00.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.443+0000 7f9e5ed7c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55cf5bc000e0 con 0x55cf5b8ce390 2024-09-21T05:59:00.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.443+0000 7f9e52d9d640 1 --2- 172.21.15.117:0/2665228370 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9e3c0b6780 0x7f9e3c0b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:00.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.444+0000 7f9e5259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf5b8ef070 0x55cf5bc966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:00.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.444+0000 7f9e5259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf5b8ef070 0x55cf5bc966a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44490/0 (socket says 172.21.15.117:44490) 2024-09-21T05:59:00.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.444+0000 7f9e5259c640 1 -- 172.21.15.117:0/1593006834 learned_addr learned my addr 172.21.15.117:0/1593006834 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:00.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.444+0000 7f9e51d9b640 1 --2- 172.21.15.117:0/1593006834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5b8c61b0 0x55cf5bc96be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:00.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.444+0000 7f9e52d9d640 1 --2- 172.21.15.117:0/1593006834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf5b8ce390 0x55cf5bc97120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:00.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.445+0000 7f9e5259c640 1 -- 172.21.15.117:0/1593006834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf5b8ce390 msgr2=0x55cf5bc97120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:00.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.445+0000 7f9e5259c640 1 --2- 172.21.15.117:0/1593006834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf5b8ce390 0x55cf5bc97120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:00.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.445+0000 7f9e5259c640 1 -- 172.21.15.117:0/1593006834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5b8c61b0 msgr2=0x55cf5bc96be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:00.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.445+0000 7f9e5259c640 1 --2- 172.21.15.117:0/1593006834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5b8c61b0 0x55cf5bc96be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:00.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.445+0000 7f9e5259c640 1 -- 172.21.15.117:0/1593006834 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf5b95a650 con 0x55cf5b8ef070 2024-09-21T05:59:00.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.446+0000 7f9e52d9d640 1 --2- 172.21.15.117:0/1593006834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf5b8ce390 0x55cf5bc97120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:59:00.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.446+0000 7f9e52d9d640 1 --2- 172.21.15.117:0/2665228370 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9e3c0b6780 0x7f9e3c0b8c90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c0104a0 tx=0x7f9e4c010960 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:00.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.446+0000 7f9e5259c640 1 --2- 172.21.15.117:0/1593006834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf5b8ef070 0x55cf5bc966a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9e440a95a0 tx=0x7f9e440ac800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a75dbf515a0a2ac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:00.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.448+0000 7f9e40ff9640 1 -- 172.21.15.117:0/1593006834 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e440b1ab0 con 0x55cf5b8ef070 2024-09-21T05:59:00.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.448+0000 7f9e40ff9640 1 -- 172.21.15.117:0/1593006834 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e440b69f0 con 0x55cf5b8ef070 2024-09-21T05:59:00.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.449+0000 7f9e40ff9640 1 -- 172.21.15.117:0/1593006834 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e44094370 con 0x55cf5b8ef070 2024-09-21T05:59:00.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.450+0000 7f9e5ed7c740 1 -- 172.21.15.117:0/1593006834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf5b8ef070 msgr2=0x55cf5bc966a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:00.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.450+0000 7f9e5ed7c740 1 --2- 172.21.15.117:0/1593006834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf5b8ef070 0x55cf5bc966a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9e440a95a0 tx=0x7f9e440ac800 comp rx=0 tx=0).stop 2024-09-21T05:59:00.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.451+0000 7f9e5ed7c740 1 -- 172.21.15.117:0/1593006834 shutdown_connections 2024-09-21T05:59:00.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.451+0000 7f9e5ed7c740 1 --2- 172.21.15.117:0/1593006834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf5b8ce390 0x55cf5bc97120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:00.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.451+0000 7f9e5ed7c740 1 --2- 172.21.15.117:0/1593006834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5b8c61b0 0x55cf5bc96be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:00.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.451+0000 7f9e5ed7c740 1 --2- 172.21.15.117:0/1593006834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf5b8ef070 0x55cf5bc966a0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:00.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.451+0000 7f9e5ed7c740 1 -- 172.21.15.117:0/1593006834 >> 172.21.15.117:0/1593006834 conn(0x55cf5b8e3460 msgr2=0x55cf5bc94590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:00.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.451+0000 7f9e5ed7c740 1 -- 172.21.15.117:0/1593006834 shutdown_connections 2024-09-21T05:59:00.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.451+0000 7f9e5ed7c740 1 -- 172.21.15.117:0/1593006834 wait complete. 2024-09-21T05:59:00.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.452+0000 7f9e5ed7c740 1 -- start start 2024-09-21T05:59:00.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.452+0000 7f9e5ed7c740 1 Processor -- start 2024-09-21T05:59:00.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.452+0000 7f9e5ed7c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5b8c61b0 0x55cf5bca2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:00.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.452+0000 7f9e5ed7c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf5b8ce390 0x55cf5bca2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:00.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.452+0000 7f9e5ed7c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf5b8ef070 0x55cf5bca7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:00.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.452+0000 7f9e5ed7c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cf5ba5d700 con 0x55cf5b8ef070 2024-09-21T05:59:00.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.452+0000 7f9e5ed7c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cf5bb97160 con 0x55cf5b8c61b0 2024-09-21T05:59:00.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.452+0000 7f9e5ed7c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55cf5bc9ce20 con 0x55cf5b8ce390 2024-09-21T05:59:00.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.452+0000 7f9e5ed7c740 10 client.0 _ll_register_callbacks cb 0x55cf5bbb09e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:59:00.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.453+0000 7f9e52d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf5b8ef070 0x55cf5bca7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:00.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.453+0000 7f9e52d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf5b8ef070 0x55cf5bca7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44494/0 (socket says 172.21.15.117:44494) 2024-09-21T05:59:00.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.453+0000 7f9e52d9d640 1 -- 172.21.15.117:0/1185946063 learned_addr learned my addr 172.21.15.117:0/1185946063 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:00.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.453+0000 7f9e52d9d640 1 -- 172.21.15.117:0/1185946063 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf5b8ce390 msgr2=0x55cf5bca2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:00.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.453+0000 7f9e51d9b640 1 --2- 172.21.15.117:0/1185946063 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf5b8ce390 0x55cf5bca2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:00.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.453+0000 7f9e52d9d640 1 --2- 172.21.15.117:0/1185946063 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf5b8ce390 0x55cf5bca2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:00.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.453+0000 7f9e52d9d640 1 -- 172.21.15.117:0/1185946063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5b8c61b0 msgr2=0x55cf5bca2620 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T05:59:00.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.453+0000 7f9e52d9d640 1 --2- 172.21.15.117:0/1185946063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5b8c61b0 0x55cf5bca2620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:00.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.453+0000 7f9e52d9d640 1 -- 172.21.15.117:0/1185946063 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf5bbc4c90 con 0x55cf5b8ef070 2024-09-21T05:59:00.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.453+0000 7f9e51d9b640 1 --2- 172.21.15.117:0/1185946063 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf5b8ce390 0x55cf5bca2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:59:00.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.453+0000 7f9e52d9d640 1 --2- 172.21.15.117:0/1185946063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf5b8ef070 0x55cf5bca7bb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00c970 tx=0x7f9e4c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a41026d5b0918bc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:00.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.453+0000 7f9e39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:59:00.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.454+0000 7f9e39ffb640 1 -- 172.21.15.117:0/1185946063 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c00edc0 con 0x55cf5b8ef070 2024-09-21T05:59:00.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.454+0000 7f9e5ed7c740 1 -- 172.21.15.117:0/1185946063 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e440b69f0 con 0x55cf5b8ef070 2024-09-21T05:59:00.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.454+0000 7f9e39ffb640 1 -- 172.21.15.117:0/1185946063 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f9e4c026410 con 0x55cf5b8ef070 2024-09-21T05:59:00.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.454+0000 7f9e39ffb640 1 -- 172.21.15.117:0/1185946063 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9e4c026610 con 0x55cf5b8ef070 2024-09-21T05:59:00.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.454+0000 7f9e5ed7c740 10 client.15903 fetch_fsmap learned FSMap version 27 2024-09-21T05:59:00.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.454+0000 7f9e5ed7c740 1 -- 172.21.15.117:0/1185946063 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cf5bcb93d0 con 0x55cf5b8ef070 2024-09-21T05:59:00.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.455+0000 7f9e39ffb640 1 -- 172.21.15.117:0/1185946063 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e440b69f0 con 0x55cf5b8ef070 2024-09-21T05:59:00.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.455+0000 7f9e39ffb640 1 -- 172.21.15.117:0/1185946063 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9e4c01c5f0 con 0x55cf5b8ef070 2024-09-21T05:59:00.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.455+0000 7f9e5ed7c740 10 client.15903 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:59:00.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.455+0000 7f9e5ed7c740 10 client.15903 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T05:59:00.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.455+0000 7f9e5ed7c740 20 client.15903 populate_metadata read hostname 'smithi117' 2024-09-21T05:59:00.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.455+0000 7f9e5ed7c740 1 --2- 172.21.15.117:0/1185946063 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55cf5b8ad9f0 0x55cf5bcbcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:00.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.455+0000 7f9e5ed7c740 4 client.15903 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:59:00.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.455+0000 7f9e5ed7c740 1 -- 172.21.15.117:0/1185946063 --> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf5bc0edf0 con 0x55cf5b8ad9f0 2024-09-21T05:59:00.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.456+0000 7f9e5259c640 1 --2- 172.21.15.117:0/1185946063 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55cf5b8ad9f0 0x55cf5bcbcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:00.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.457+0000 7f9e39ffb640 10 client.15903 ms_handle_connect on v2:172.21.15.192:6834/3530688860 2024-09-21T05:59:00.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.457+0000 7f9e5259c640 1 --2- 172.21.15.117:0/1185946063 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55cf5b8ad9f0 0x55cf5bcbcdb0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=caa00e4ef999f02b server_cookie=6c6d0e5e25e1f8a2 in_seq=0 out_seq=0 2024-09-21T05:59:00.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.458+0000 7f9e39ffb640 1 -- 172.21.15.117:0/1185946063 <== mds.3 v2:172.21.15.192:6834/3530688860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cf5bc0edf0 con 0x55cf5b8ad9f0 2024-09-21T05:59:00.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.458+0000 7f9e39ffb640 10 client.15903 handle_command_reply: tid=0 2024-09-21T05:59:00.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.459+0000 7f9e5ed7c740 1 client.15903 shutdown 2024-09-21T05:59:00.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.459+0000 7f9e5ed7c740 1 -- 172.21.15.117:0/1185946063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf5b8ef070 msgr2=0x55cf5bca7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:00.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.459+0000 7f9e5ed7c740 1 --2- 172.21.15.117:0/1185946063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf5b8ef070 0x55cf5bca7bb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00c970 tx=0x7f9e4c016000 comp rx=0 tx=0).stop 2024-09-21T05:59:00.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.459+0000 7f9e5ed7c740 1 -- 172.21.15.117:0/1185946063 shutdown_connections 2024-09-21T05:59:00.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.459+0000 7f9e5ed7c740 1 --2- 172.21.15.117:0/1185946063 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55cf5b8ad9f0 0x55cf5bcbcdb0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:00.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.459+0000 7f9e5ed7c740 1 --2- 172.21.15.117:0/1185946063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf5b8ef070 0x55cf5bca7bb0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:00.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.459+0000 7f9e5ed7c740 1 --2- 172.21.15.117:0/1185946063 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf5b8ce390 0x55cf5bca2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:00.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.459+0000 7f9e5ed7c740 1 --2- 172.21.15.117:0/1185946063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5b8c61b0 0x55cf5bca2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:00.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.459+0000 7f9e39ffb640 0 client.15903 ms_handle_reset on v2:172.21.15.192:6834/3530688860 2024-09-21T05:59:00.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.459+0000 7f9e5ed7c740 1 -- 172.21.15.117:0/1185946063 >> 172.21.15.117:0/1185946063 conn(0x55cf5b8e3460 msgr2=0x55cf5bc92bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:00.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.460+0000 7f9e5ed7c740 1 -- 172.21.15.117:0/1185946063 shutdown_connections 2024-09-21T05:59:00.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.460+0000 7f9e5ed7c740 1 -- 172.21.15.117:0/1185946063 wait complete. 2024-09-21T05:59:00.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.460+0000 7f9e5ed7c740 20 client.15903 trim_cache size 0 max 16384 2024-09-21T05:59:00.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.484+0000 7f9e5ed7c740 1 Processor -- start 2024-09-21T05:59:00.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.484+0000 7f9e5ed7c740 1 -- start start 2024-09-21T05:59:00.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.484+0000 7f9e5ed7c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf5b8c61b0 0x55cf5bca0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:00.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.484+0000 7f9e5ed7c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf5b8ce390 0x55cf5bca11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:00.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.484+0000 7f9e5ed7c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5b8ef070 0x55cf5bc9f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:00.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.484+0000 7f9e5ed7c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9e541ad590 con 0x55cf5b8ce390 2024-09-21T05:59:00.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.484+0000 7f9e5ed7c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9e541af0d0 con 0x55cf5b8ef070 2024-09-21T05:59:00.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.484+0000 7f9e5ed7c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9e541adc60 con 0x55cf5b8c61b0 2024-09-21T05:59:00.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.485+0000 7f9e51d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf5b8ce390 0x55cf5bca11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:00.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.485+0000 7f9e51d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf5b8ce390 0x55cf5bca11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44506/0 (socket says 172.21.15.117:44506) 2024-09-21T05:59:00.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.485+0000 7f9e51d9b640 1 -- 172.21.15.117:0/1610202401 learned_addr learned my addr 172.21.15.117:0/1610202401 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:00.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.485+0000 7f9e51d9b640 1 -- 172.21.15.117:0/1610202401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf5b8c61b0 msgr2=0x55cf5bca0ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T05:59:00.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.485+0000 7f9e51d9b640 1 --2- 172.21.15.117:0/1610202401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf5b8c61b0 0x55cf5bca0ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:00.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.485+0000 7f9e51d9b640 1 -- 172.21.15.117:0/1610202401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5b8ef070 msgr2=0x55cf5bc9f610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T05:59:00.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.485+0000 7f9e51d9b640 1 --2- 172.21.15.117:0/1610202401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5b8ef070 0x55cf5bc9f610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:00.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.485+0000 7f9e51d9b640 1 -- 172.21.15.117:0/1610202401 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf5bca2fc0 con 0x55cf5b8ce390 2024-09-21T05:59:00.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.485+0000 7f9e51d9b640 1 --2- 172.21.15.117:0/1610202401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf5b8ce390 0x55cf5bca11e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9e48006010 tx=0x7f9e48004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=252999f387cec44e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:00.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.486+0000 7f9e3bfff640 1 -- 172.21.15.117:0/1610202401 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e48041070 con 0x55cf5b8ce390 2024-09-21T05:59:00.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.486+0000 7f9e3bfff640 1 -- 172.21.15.117:0/1610202401 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e48003aa0 con 0x55cf5b8ce390 2024-09-21T05:59:00.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.486+0000 7f9e3bfff640 1 -- 172.21.15.117:0/1610202401 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e48035040 con 0x55cf5b8ce390 2024-09-21T05:59:00.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.487+0000 7f9e5ed7c740 1 -- 172.21.15.117:0/1610202401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf5b8ce390 msgr2=0x55cf5bca11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:00.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.487+0000 7f9e5ed7c740 1 --2- 172.21.15.117:0/1610202401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf5b8ce390 0x55cf5bca11e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9e48006010 tx=0x7f9e48004310 comp rx=0 tx=0).stop 2024-09-21T05:59:00.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.487+0000 7f9e5ed7c740 1 -- 172.21.15.117:0/1610202401 shutdown_connections 2024-09-21T05:59:00.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.487+0000 7f9e5ed7c740 1 --2- 172.21.15.117:0/1610202401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf5b8ce390 0x55cf5bca11e0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:00.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.487+0000 7f9e5ed7c740 1 --2- 172.21.15.117:0/1610202401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5b8ef070 0x55cf5bc9f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:00.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.487+0000 7f9e5ed7c740 1 --2- 172.21.15.117:0/1610202401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf5b8c61b0 0x55cf5bca0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:00.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.487+0000 7f9e5ed7c740 1 -- 172.21.15.117:0/1610202401 >> 172.21.15.117:0/1610202401 conn(0x55cf5b8e3460 msgr2=0x55cf5bc9d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:00.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.487+0000 7f9e5ed7c740 1 -- 172.21.15.117:0/1610202401 shutdown_connections 2024-09-21T05:59:00.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.487+0000 7f9e5ed7c740 1 -- 172.21.15.117:0/1610202401 wait complete. 2024-09-21T05:59:00.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.488+0000 7f9e5ed7c740 1 -- start start 2024-09-21T05:59:00.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.488+0000 7f9e5ed7c740 1 Processor -- start 2024-09-21T05:59:00.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.489+0000 7f9e5ed7c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5b8c61b0 0x55cf5bcb6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:00.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.489+0000 7f9e5ed7c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf5b8ce390 0x55cf5bcb40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:00.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.489+0000 7f9e5ed7c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf5b8ef070 0x55cf5bcb4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:00.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.489+0000 7f9e5ed7c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9e541aea00 con 0x55cf5b8ef070 2024-09-21T05:59:00.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.489+0000 7f9e5ed7c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9e541ae330 con 0x55cf5b8c61b0 2024-09-21T05:59:00.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.489+0000 7f9e5ed7c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9e541ab5b0 con 0x55cf5b8ce390 2024-09-21T05:59:00.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.489+0000 7f9e5ed7c740 10 client.0 _ll_register_callbacks cb 0x7f9e4c00c630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:59:00.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.489+0000 7f9e52d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf5b8ef070 0x55cf5bcb4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:00.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.489+0000 7f9e52d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf5b8ef070 0x55cf5bcb4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44512/0 (socket says 172.21.15.117:44512) 2024-09-21T05:59:00.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.489+0000 7f9e52d9d640 1 -- 172.21.15.117:0/3115500280 learned_addr learned my addr 172.21.15.117:0/3115500280 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:00.152 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T05:59:00.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.489+0000 7f9e51d9b640 1 --2- 172.21.15.117:0/3115500280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf5b8ce390 0x55cf5bcb40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:00.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.490+0000 7f9e52d9d640 1 -- 172.21.15.117:0/3115500280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf5b8ce390 msgr2=0x55cf5bcb40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:00.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.490+0000 7f9e52d9d640 1 --2- 172.21.15.117:0/3115500280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf5b8ce390 0x55cf5bcb40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:00.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.490+0000 7f9e5259c640 1 --2- 172.21.15.117:0/3115500280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5b8c61b0 0x55cf5bcb6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:00.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.490+0000 7f9e52d9d640 1 -- 172.21.15.117:0/3115500280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5b8c61b0 msgr2=0x55cf5bcb6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:00.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.490+0000 7f9e52d9d640 1 --2- 172.21.15.117:0/3115500280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5b8c61b0 0x55cf5bcb6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:00.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.490+0000 7f9e52d9d640 1 -- 172.21.15.117:0/3115500280 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf5bcb4b60 con 0x55cf5b8ef070 2024-09-21T05:59:00.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.490+0000 7f9e5259c640 1 --2- 172.21.15.117:0/3115500280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5b8c61b0 0x55cf5bcb6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:59:00.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.490+0000 7f9e52d9d640 1 --2- 172.21.15.117:0/3115500280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf5b8ef070 0x55cf5bcb4620 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c0092b0 tx=0x7f9e4c00e800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2f3262426db7fb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:00.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.490+0000 7f9e39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:59:00.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.491+0000 7f9e39ffb640 1 -- 172.21.15.117:0/3115500280 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c03f060 con 0x55cf5b8ef070 2024-09-21T05:59:00.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.491+0000 7f9e39ffb640 1 -- 172.21.15.117:0/3115500280 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f9e4c01c5f0 con 0x55cf5b8ef070 2024-09-21T05:59:00.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.491+0000 7f9e5ed7c740 1 -- 172.21.15.117:0/3115500280 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e48003aa0 con 0x55cf5b8ef070 2024-09-21T05:59:00.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.491+0000 7f9e39ffb640 1 -- 172.21.15.117:0/3115500280 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9e4c01c7f0 con 0x55cf5b8ef070 2024-09-21T05:59:00.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.491+0000 7f9e5ed7c740 10 client.15909 fetch_fsmap learned FSMap version 27 2024-09-21T05:59:00.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.491+0000 7f9e5ed7c740 1 -- 172.21.15.117:0/3115500280 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cf5bd27420 con 0x55cf5b8ef070 2024-09-21T05:59:00.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.492+0000 7f9e39ffb640 1 -- 172.21.15.117:0/3115500280 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e48003aa0 con 0x55cf5b8ef070 2024-09-21T05:59:00.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.492+0000 7f9e39ffb640 1 -- 172.21.15.117:0/3115500280 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9e4c03a800 con 0x55cf5b8ef070 2024-09-21T05:59:00.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.492+0000 7f9e5ed7c740 10 client.15909 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:59:00.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.492+0000 7f9e5ed7c740 10 client.15909 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T05:59:00.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.492+0000 7f9e5ed7c740 20 client.15909 populate_metadata read hostname 'smithi117' 2024-09-21T05:59:00.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.492+0000 7f9e5ed7c740 1 --2- 172.21.15.117:0/3115500280 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55cf5b882da0 0x55cf5bd29b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:00.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.492+0000 7f9e5ed7c740 4 client.15909 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T05:59:00.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.492+0000 7f9e5ed7c740 1 -- 172.21.15.117:0/3115500280 --> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cf5bc0a8a0 con 0x55cf5b882da0 2024-09-21T05:59:00.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.493+0000 7f9e5259c640 1 --2- 172.21.15.117:0/3115500280 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55cf5b882da0 0x55cf5bd29b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:00.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.494+0000 7f9e39ffb640 10 client.15909 ms_handle_connect on v2:172.21.15.192:6834/3530688860 2024-09-21T05:59:00.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.495+0000 7f9e5259c640 1 --2- 172.21.15.117:0/3115500280 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55cf5b882da0 0x55cf5bd29b80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6273626400b54a23 server_cookie=b702e6e4e300b61c in_seq=0 out_seq=0 2024-09-21T05:59:00.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.496+0000 7f9e39ffb640 1 -- 172.21.15.117:0/3115500280 <== mds.3 v2:172.21.15.192:6834/3530688860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cf5bc0a8a0 con 0x55cf5b882da0 2024-09-21T05:59:00.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.496+0000 7f9e39ffb640 10 client.15909 handle_command_reply: tid=0 2024-09-21T05:59:00.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.496+0000 7f9e5ed7c740 1 client.15909 shutdown 2024-09-21T05:59:00.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.497+0000 7f9e5ed7c740 1 -- 172.21.15.117:0/3115500280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf5b8ef070 msgr2=0x55cf5bcb4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:00.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.497+0000 7f9e5ed7c740 1 --2- 172.21.15.117:0/3115500280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf5b8ef070 0x55cf5bcb4620 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c0092b0 tx=0x7f9e4c00e800 comp rx=0 tx=0).stop 2024-09-21T05:59:00.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.497+0000 7f9e5ed7c740 1 -- 172.21.15.117:0/3115500280 shutdown_connections 2024-09-21T05:59:00.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.497+0000 7f9e5ed7c740 1 --2- 172.21.15.117:0/3115500280 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55cf5b882da0 0x55cf5bd29b80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:00.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.497+0000 7f9e39ffb640 0 client.15909 ms_handle_reset on v2:172.21.15.192:6834/3530688860 2024-09-21T05:59:00.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.497+0000 7f9e5ed7c740 1 --2- 172.21.15.117:0/3115500280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf5b8ef070 0x55cf5bcb4620 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:00.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.497+0000 7f9e5ed7c740 1 --2- 172.21.15.117:0/3115500280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf5b8ce390 0x55cf5bcb40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:00.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.497+0000 7f9e5ed7c740 1 --2- 172.21.15.117:0/3115500280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf5b8c61b0 0x55cf5bcb6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:00.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.497+0000 7f9e5ed7c740 1 -- 172.21.15.117:0/3115500280 >> 172.21.15.117:0/3115500280 conn(0x55cf5b8e3460 msgr2=0x55cf5bca0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:00.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.497+0000 7f9e5ed7c740 1 -- 172.21.15.117:0/3115500280 shutdown_connections 2024-09-21T05:59:00.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.497+0000 7f9e5ed7c740 1 -- 172.21.15.117:0/3115500280 wait complete. 2024-09-21T05:59:00.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.497+0000 7f9e5ed7c740 20 client.15909 trim_cache size 0 max 16384 2024-09-21T05:59:00.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.498+0000 7f9e58808640 1 -- 172.21.15.117:0/2665228370 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9e3c0b6780 msgr2=0x7f9e3c0b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:00.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.498+0000 7f9e58808640 1 --2- 172.21.15.117:0/2665228370 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9e3c0b6780 0x7f9e3c0b8c90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c0104a0 tx=0x7f9e4c010960 comp rx=0 tx=0).stop 2024-09-21T05:59:00.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.498+0000 7f9e58808640 1 -- 172.21.15.117:0/2665228370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e541021f0 msgr2=0x7f9e5419b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:00.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.498+0000 7f9e58808640 1 --2- 172.21.15.117:0/2665228370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e541021f0 0x7f9e5419b470 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9e4400b450 tx=0x7f9e4400b910 comp rx=0 tx=0).stop 2024-09-21T05:59:00.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.499+0000 7f9e58808640 1 -- 172.21.15.117:0/2665228370 shutdown_connections 2024-09-21T05:59:00.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.499+0000 7f9e58808640 1 --2- 172.21.15.117:0/2665228370 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9e3c0b6780 0x7f9e3c0b8c90 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:00.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.499+0000 7f9e58808640 1 --2- 172.21.15.117:0/2665228370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e54108480 0x7f9e5419bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:00.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.499+0000 7f9e58808640 1 --2- 172.21.15.117:0/2665228370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e54107b30 0x7f9e5419b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:00.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.499+0000 7f9e58808640 1 --2- 172.21.15.117:0/2665228370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e541021f0 0x7f9e5419b470 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:00.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.499+0000 7f9e58808640 1 -- 172.21.15.117:0/2665228370 >> 172.21.15.117:0/2665228370 conn(0x7f9e54077c60 msgr2=0x7f9e54105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:00.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.499+0000 7f9e58808640 1 -- 172.21.15.117:0/2665228370 shutdown_connections 2024-09-21T05:59:00.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:58:59.499+0000 7f9e58808640 1 -- 172.21.15.117:0/2665228370 wait complete. 2024-09-21T05:59:01.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:01 smithi192 ceph-mon[34700]: pgmap v378: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 786 MiB/s wr, 1.82k op/s 2024-09-21T05:59:02.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:01 smithi117 ceph-mon[30947]: pgmap v378: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 786 MiB/s wr, 1.82k op/s 2024-09-21T05:59:02.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:01 smithi138 ceph-mon[34391]: pgmap v378: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 786 MiB/s wr, 1.82k op/s 2024-09-21T05:59:03.927 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:03 smithi117 ceph-mon[30947]: pgmap v379: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 914 MiB/s wr, 2.08k op/s 2024-09-21T05:59:03.929 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T05:59:03.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:03 smithi192 ceph-mon[34700]: pgmap v379: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 914 MiB/s wr, 2.08k op/s 2024-09-21T05:59:04.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:03 smithi138 ceph-mon[34391]: pgmap v379: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 914 MiB/s wr, 2.08k op/s 2024-09-21T05:59:04.442 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:59:05.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:05 smithi192 ceph-mon[34700]: pgmap v380: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 936 MiB/s wr, 1.86k op/s 2024-09-21T05:59:06.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:05 smithi117 ceph-mon[30947]: pgmap v380: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 936 MiB/s wr, 1.86k op/s 2024-09-21T05:59:06.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:05 smithi138 ceph-mon[34391]: pgmap v380: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 936 MiB/s wr, 1.86k op/s 2024-09-21T05:59:06.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.699+0000 7f04061d7640 1 -- 172.21.15.117:0/3486270145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0400075750 msgr2=0x7f0400075b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:06.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.699+0000 7f04061d7640 1 --2- 172.21.15.117:0/3486270145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0400075750 0x7f0400075b60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009900 tx=0x7f03f002a410 comp rx=0 tx=0).stop 2024-09-21T05:59:06.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.699+0000 7f04061d7640 1 -- 172.21.15.117:0/3486270145 shutdown_connections 2024-09-21T05:59:06.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.699+0000 7f04061d7640 1 --2- 172.21.15.117:0/3486270145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0400110150 0x7f0400112570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.699+0000 7f04061d7640 1 --2- 172.21.15.117:0/3486270145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04000760a0 0x7f040010fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.699+0000 7f04061d7640 1 --2- 172.21.15.117:0/3486270145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0400075750 0x7f0400075b60 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.699+0000 7f04061d7640 1 -- 172.21.15.117:0/3486270145 >> 172.21.15.117:0/3486270145 conn(0x7f04000ff5e0 msgr2=0x7f0400101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:06.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.699+0000 7f04061d7640 1 -- 172.21.15.117:0/3486270145 shutdown_connections 2024-09-21T05:59:06.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.699+0000 7f04061d7640 1 -- 172.21.15.117:0/3486270145 wait complete. 2024-09-21T05:59:06.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.700+0000 7f04061d7640 1 Processor -- start 2024-09-21T05:59:06.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.700+0000 7f04061d7640 1 -- start start 2024-09-21T05:59:06.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.700+0000 7f04061d7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0400075750 0x7f04001075e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:06.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.700+0000 7f04061d7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04000760a0 0x7f04001038c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:06.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.701+0000 7f04061d7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0400110150 0x7f0400103fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:06.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.701+0000 7f04061d7640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0400115cc0 con 0x7f0400110150 2024-09-21T05:59:06.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.701+0000 7f04061d7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0400115b40 con 0x7f04000760a0 2024-09-21T05:59:06.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.701+0000 7f04061d7640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0400115e40 con 0x7f0400075750 2024-09-21T05:59:06.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.701+0000 7f03fffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0400110150 0x7f0400103fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:06.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.701+0000 7f03fffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0400110150 0x7f0400103fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45376/0 (socket says 172.21.15.117:45376) 2024-09-21T05:59:06.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.701+0000 7f03fffff640 1 -- 172.21.15.117:0/224167961 learned_addr learned my addr 172.21.15.117:0/224167961 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:06.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.701+0000 7f03ff7fe640 1 --2- 172.21.15.117:0/224167961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0400075750 0x7f04001075e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:06.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.702+0000 7f03f7fff640 1 --2- 172.21.15.117:0/224167961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04000760a0 0x7f04001038c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:06.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.702+0000 7f03fffff640 1 -- 172.21.15.117:0/224167961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0400075750 msgr2=0x7f04001075e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:06.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.702+0000 7f03fffff640 1 --2- 172.21.15.117:0/224167961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0400075750 0x7f04001075e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.702+0000 7f03fffff640 1 -- 172.21.15.117:0/224167961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04000760a0 msgr2=0x7f04001038c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:06.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.702+0000 7f03fffff640 1 --2- 172.21.15.117:0/224167961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04000760a0 0x7f04001038c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.702+0000 7f03fffff640 1 -- 172.21.15.117:0/224167961 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04001acc60 con 0x7f0400110150 2024-09-21T05:59:06.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.702+0000 7f03ff7fe640 1 --2- 172.21.15.117:0/224167961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0400075750 0x7f04001075e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:59:06.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.703+0000 7f03f7fff640 1 --2- 172.21.15.117:0/224167961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04000760a0 0x7f04001038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:59:06.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.703+0000 7f03fffff640 1 --2- 172.21.15.117:0/224167961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0400110150 0x7f0400103fe0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f03f800d750 tx=0x7f03f800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:06.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.703+0000 7f03fd7fa640 1 -- 172.21.15.117:0/224167961 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f8017070 con 0x7f0400110150 2024-09-21T05:59:06.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.703+0000 7f03fd7fa640 1 -- 172.21.15.117:0/224167961 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03f80054d0 con 0x7f0400110150 2024-09-21T05:59:06.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.703+0000 7f03fd7fa640 1 -- 172.21.15.117:0/224167961 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f8004c60 con 0x7f0400110150 2024-09-21T05:59:06.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.704+0000 7f04061d7640 1 -- 172.21.15.117:0/224167961 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04001ace90 con 0x7f0400110150 2024-09-21T05:59:06.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.704+0000 7f04061d7640 1 -- 172.21.15.117:0/224167961 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04001ad230 con 0x7f0400110150 2024-09-21T05:59:06.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.708+0000 7f040c74b740 1 Processor -- start 2024-09-21T05:59:06.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.710+0000 7f03fd7fa640 1 -- 172.21.15.117:0/224167961 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f03f8009bd0 con 0x7f0400110150 2024-09-21T05:59:06.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.710+0000 7f040c74b740 1 -- start start 2024-09-21T05:59:06.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.711+0000 7f040c74b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad75ed2070 0x55ad762792c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:06.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.711+0000 7f040c74b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad75ea91b0 0x55ad76279800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:06.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.711+0000 7f03fd7fa640 1 --2- 172.21.15.117:0/224167961 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f03e00b67d0 0x7f03e00b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:06.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.711+0000 7f040c74b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad75eb1390 0x55ad76279d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:06.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.711+0000 7f040c74b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ad761efea0 con 0x55ad75ed2070 2024-09-21T05:59:06.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.711+0000 7f040c74b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ad761df2d0 con 0x55ad75eb1390 2024-09-21T05:59:06.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.711+0000 7f040c74b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55ad761e30e0 con 0x55ad75ea91b0 2024-09-21T05:59:06.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.712+0000 7f03fd7fa640 1 -- 172.21.15.117:0/224167961 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f03f80e6f40 con 0x7f0400110150 2024-09-21T05:59:06.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.712+0000 7f03f7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad75ed2070 0x55ad762792c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:06.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.712+0000 7f03f7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad75ed2070 0x55ad762792c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45384/0 (socket says 172.21.15.117:45384) 2024-09-21T05:59:06.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.712+0000 7f03f7fff640 1 -- 172.21.15.117:0/2542610594 learned_addr learned my addr 172.21.15.117:0/2542610594 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:06.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.712+0000 7f03f7fff640 1 --2- 172.21.15.117:0/2542610594 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad75ea91b0 0x55ad76279800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:06.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.713+0000 7f03f7fff640 1 -- 172.21.15.117:0/2542610594 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad75ea91b0 msgr2=0x55ad76279800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:06.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.713+0000 7f03f7fff640 1 --2- 172.21.15.117:0/2542610594 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad75ea91b0 0x55ad76279800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.713+0000 7f03f7fff640 1 -- 172.21.15.117:0/2542610594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad75eb1390 msgr2=0x55ad76279d40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T05:59:06.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.713+0000 7f03f7fff640 1 --2- 172.21.15.117:0/2542610594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad75eb1390 0x55ad76279d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.713+0000 7f03f7fff640 1 -- 172.21.15.117:0/2542610594 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad75f3d650 con 0x55ad75ed2070 2024-09-21T05:59:06.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.713+0000 7f03ff7fe640 1 --2- 172.21.15.117:0/224167961 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f03e00b67d0 0x7f03e00b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:06.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.714+0000 7f03f7fff640 1 --2- 172.21.15.117:0/2542610594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad75ed2070 0x55ad762792c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f03e80128e0 tx=0x7f03e8009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97389fc3b05ea80 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:06.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.714+0000 7f03f5ffb640 1 -- 172.21.15.117:0/2542610594 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8003970 con 0x55ad75ed2070 2024-09-21T05:59:06.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.714+0000 7f03ff7fe640 1 --2- 172.21.15.117:0/224167961 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f03e00b67d0 0x7f03e00b8ce0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f03f0000c00 tx=0x7f03f0031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:06.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.714+0000 7f03f5ffb640 1 -- 172.21.15.117:0/2542610594 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03e800e8d0 con 0x55ad75ed2070 2024-09-21T05:59:06.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.714+0000 7f03f5ffb640 1 -- 172.21.15.117:0/2542610594 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8004450 con 0x55ad75ed2070 2024-09-21T05:59:06.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.715+0000 7f040c74b740 1 -- 172.21.15.117:0/2542610594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad75ed2070 msgr2=0x55ad762792c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:06.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.715+0000 7f040c74b740 1 --2- 172.21.15.117:0/2542610594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad75ed2070 0x55ad762792c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f03e80128e0 tx=0x7f03e8009b30 comp rx=0 tx=0).stop 2024-09-21T05:59:06.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.716+0000 7f040c74b740 1 -- 172.21.15.117:0/2542610594 shutdown_connections 2024-09-21T05:59:06.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.716+0000 7f040c74b740 1 --2- 172.21.15.117:0/2542610594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad75eb1390 0x55ad76279d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.716+0000 7f040c74b740 1 --2- 172.21.15.117:0/2542610594 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad75ea91b0 0x55ad76279800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.716+0000 7f040c74b740 1 --2- 172.21.15.117:0/2542610594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad75ed2070 0x55ad762792c0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.716+0000 7f040c74b740 1 -- 172.21.15.117:0/2542610594 >> 172.21.15.117:0/2542610594 conn(0x55ad75ec6460 msgr2=0x55ad762771b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:06.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.716+0000 7f040c74b740 1 -- 172.21.15.117:0/2542610594 shutdown_connections 2024-09-21T05:59:06.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.716+0000 7f040c74b740 1 -- 172.21.15.117:0/2542610594 wait complete. 2024-09-21T05:59:06.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.716+0000 7f040c74b740 1 -- start start 2024-09-21T05:59:06.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.717+0000 7f040c74b740 1 Processor -- start 2024-09-21T05:59:06.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.717+0000 7f040c74b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad75ea91b0 0x55ad76284a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:06.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.717+0000 7f040c74b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad75eb1390 0x55ad76284f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:06.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.717+0000 7f040c74b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad75ed2070 0x55ad76290dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:06.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.717+0000 7f040c74b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ad76040700 con 0x55ad75eb1390 2024-09-21T05:59:06.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.717+0000 7f040c74b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ad7617a160 con 0x55ad75ea91b0 2024-09-21T05:59:06.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.717+0000 7f040c74b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55ad76287aa0 con 0x55ad75ed2070 2024-09-21T05:59:06.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.717+0000 7f040c74b740 10 client.0 _ll_register_callbacks cb 0x55ad761939e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:59:06.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.718+0000 7f03f7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad75eb1390 0x55ad76284f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:06.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.718+0000 7f03f7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad75eb1390 0x55ad76284f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45398/0 (socket says 172.21.15.117:45398) 2024-09-21T05:59:06.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.718+0000 7f03f7fff640 1 -- 172.21.15.117:0/1111108704 learned_addr learned my addr 172.21.15.117:0/1111108704 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:06.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.718+0000 7f03fffff640 1 --2- 172.21.15.117:0/1111108704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad75ed2070 0x55ad76290dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:06.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.719+0000 7f03ff7fe640 1 --2- 172.21.15.117:0/1111108704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad75ea91b0 0x55ad76284a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:06.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.719+0000 7f03f7fff640 1 -- 172.21.15.117:0/1111108704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad75ed2070 msgr2=0x55ad76290dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:06.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.719+0000 7f03f7fff640 1 --2- 172.21.15.117:0/1111108704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad75ed2070 0x55ad76290dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.719+0000 7f03f7fff640 1 -- 172.21.15.117:0/1111108704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad75ea91b0 msgr2=0x55ad76284a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:06.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.719+0000 7f03f7fff640 1 --2- 172.21.15.117:0/1111108704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad75ea91b0 0x55ad76284a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.719+0000 7f03f7fff640 1 -- 172.21.15.117:0/1111108704 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad761a7c90 con 0x55ad75eb1390 2024-09-21T05:59:06.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.719+0000 7f03f7fff640 1 --2- 172.21.15.117:0/1111108704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad75eb1390 0x55ad76284f80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f03e8005ad0 tx=0x7f03e8009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=999efcdfc35e21df server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:06.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.719+0000 7f03eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:59:06.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.720+0000 7f03eeffd640 1 -- 172.21.15.117:0/1111108704 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8003d80 con 0x55ad75eb1390 2024-09-21T05:59:06.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.720+0000 7f040c74b740 1 -- 172.21.15.117:0/1111108704 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03e800e8d0 con 0x55ad75eb1390 2024-09-21T05:59:06.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.720+0000 7f03eeffd640 1 -- 172.21.15.117:0/1111108704 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f03e8003b80 con 0x55ad75eb1390 2024-09-21T05:59:06.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.720+0000 7f03eeffd640 1 -- 172.21.15.117:0/1111108704 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f03e8003d80 con 0x55ad75eb1390 2024-09-21T05:59:06.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.720+0000 7f040c74b740 10 client.15921 fetch_fsmap learned FSMap version 27 2024-09-21T05:59:06.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.721+0000 7f040c74b740 1 -- 172.21.15.117:0/1111108704 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ad76293840 con 0x55ad75eb1390 2024-09-21T05:59:06.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.721+0000 7f03eeffd640 1 -- 172.21.15.117:0/1111108704 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03e800e8d0 con 0x55ad75eb1390 2024-09-21T05:59:06.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.721+0000 7f03eeffd640 1 -- 172.21.15.117:0/1111108704 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f03e8027070 con 0x55ad75eb1390 2024-09-21T05:59:06.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.721+0000 7f040c74b740 10 client.15921 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:59:06.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.721+0000 7f040c74b740 10 client.15921 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T05:59:06.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.721+0000 7f040c74b740 20 client.15921 populate_metadata read hostname 'smithi117' 2024-09-21T05:59:06.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.721+0000 7f040c74b740 1 --2- 172.21.15.117:0/1111108704 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55ad75e909f0 0x55ad76296ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:06.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.721+0000 7f040c74b740 4 client.15921 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:59:06.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.722+0000 7f040c74b740 1 -- 172.21.15.117:0/1111108704 --> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad761f1df0 con 0x55ad75e909f0 2024-09-21T05:59:06.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.722+0000 7f03ff7fe640 1 --2- 172.21.15.117:0/1111108704 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55ad75e909f0 0x55ad76296ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:06.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.724+0000 7f03eeffd640 10 client.15921 ms_handle_connect on v2:172.21.15.192:6836/2683860842 2024-09-21T05:59:06.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.724+0000 7f03ff7fe640 1 --2- 172.21.15.117:0/1111108704 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55ad75e909f0 0x55ad76296ba0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=320359a62c0639 server_cookie=aa541863c4ccb32a in_seq=0 out_seq=0 2024-09-21T05:59:06.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.726+0000 7f03eeffd640 1 -- 172.21.15.117:0/1111108704 <== mds.2 v2:172.21.15.192:6836/2683860842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ad761f1df0 con 0x55ad75e909f0 2024-09-21T05:59:06.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.726+0000 7f03eeffd640 10 client.15921 handle_command_reply: tid=0 2024-09-21T05:59:06.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.726+0000 7f040c74b740 1 client.15921 shutdown 2024-09-21T05:59:06.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.727+0000 7f040c74b740 1 -- 172.21.15.117:0/1111108704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad75eb1390 msgr2=0x55ad76284f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:06.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.727+0000 7f040c74b740 1 --2- 172.21.15.117:0/1111108704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad75eb1390 0x55ad76284f80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f03e8005ad0 tx=0x7f03e8009500 comp rx=0 tx=0).stop 2024-09-21T05:59:06.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.728+0000 7f040c74b740 1 -- 172.21.15.117:0/1111108704 shutdown_connections 2024-09-21T05:59:06.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.728+0000 7f040c74b740 1 --2- 172.21.15.117:0/1111108704 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55ad75e909f0 0x55ad76296ba0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.728+0000 7f040c74b740 1 --2- 172.21.15.117:0/1111108704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad75eb1390 0x55ad76284f80 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.728+0000 7f040c74b740 1 --2- 172.21.15.117:0/1111108704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad75ed2070 0x55ad76290dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.728+0000 7f040c74b740 1 --2- 172.21.15.117:0/1111108704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad75ea91b0 0x55ad76284a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.728+0000 7f040c74b740 1 -- 172.21.15.117:0/1111108704 >> 172.21.15.117:0/1111108704 conn(0x55ad75ec6460 msgr2=0x55ad7627c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:06.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.728+0000 7f03eeffd640 0 client.15921 ms_handle_reset on v2:172.21.15.192:6836/2683860842 2024-09-21T05:59:06.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.728+0000 7f040c74b740 1 -- 172.21.15.117:0/1111108704 shutdown_connections 2024-09-21T05:59:06.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.729+0000 7f040c74b740 1 -- 172.21.15.117:0/1111108704 wait complete. 2024-09-21T05:59:06.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.729+0000 7f040c74b740 20 client.15921 trim_cache size 0 max 16384 2024-09-21T05:59:06.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.746+0000 7f040c74b740 1 Processor -- start 2024-09-21T05:59:06.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.747+0000 7f040c74b740 1 -- start start 2024-09-21T05:59:06.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.747+0000 7f040c74b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad75ea91b0 0x55ad76286920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:06.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.747+0000 7f040c74b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad75eb1390 0x55ad7627aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:06.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.747+0000 7f040c74b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad75ed2070 0x55ad76290cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:06.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.747+0000 7f040c74b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f04001b5b80 con 0x55ad75eb1390 2024-09-21T05:59:06.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.747+0000 7f040c74b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f04001b76c0 con 0x55ad75ea91b0 2024-09-21T05:59:06.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.747+0000 7f040c74b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f04001b6250 con 0x55ad75ed2070 2024-09-21T05:59:06.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.748+0000 7f03f7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad75eb1390 0x55ad7627aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:06.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.748+0000 7f03fffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad75ed2070 0x55ad76290cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:06.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.748+0000 7f03fffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad75ed2070 0x55ad76290cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:35964/0 (socket says 172.21.15.117:35964) 2024-09-21T05:59:06.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.748+0000 7f03fffff640 1 -- 172.21.15.117:0/3749493293 learned_addr learned my addr 172.21.15.117:0/3749493293 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:06.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.748+0000 7f03fffff640 1 -- 172.21.15.117:0/3749493293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad75ea91b0 msgr2=0x55ad76286920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:06.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.748+0000 7f03fffff640 1 --2- 172.21.15.117:0/3749493293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad75ea91b0 0x55ad76286920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.748+0000 7f03fffff640 1 -- 172.21.15.117:0/3749493293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad75eb1390 msgr2=0x55ad7627aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:06.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.748+0000 7f03fffff640 1 --2- 172.21.15.117:0/3749493293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad75eb1390 0x55ad7627aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.748+0000 7f03fffff640 1 -- 172.21.15.117:0/3749493293 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad76297000 con 0x55ad75ed2070 2024-09-21T05:59:06.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.749+0000 7f03fffff640 1 --2- 172.21.15.117:0/3749493293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad75ed2070 0x55ad76290cc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f03f8005d80 tx=0x7f03f80a8d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=753ca1c6ea515906 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:06.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.749+0000 7f03f57fa640 1 -- 172.21.15.117:0/3749493293 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f8020050 con 0x55ad75ed2070 2024-09-21T05:59:06.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.749+0000 7f03f57fa640 1 -- 172.21.15.117:0/3749493293 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03f8005680 con 0x55ad75ed2070 2024-09-21T05:59:06.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.749+0000 7f03f57fa640 1 -- 172.21.15.117:0/3749493293 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f80e4e60 con 0x55ad75ed2070 2024-09-21T05:59:06.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.750+0000 7f040c74b740 1 -- 172.21.15.117:0/3749493293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad75ed2070 msgr2=0x55ad76290cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:06.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.750+0000 7f040c74b740 1 --2- 172.21.15.117:0/3749493293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad75ed2070 0x55ad76290cc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f03f8005d80 tx=0x7f03f80a8d90 comp rx=0 tx=0).stop 2024-09-21T05:59:06.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.750+0000 7f040c74b740 1 -- 172.21.15.117:0/3749493293 shutdown_connections 2024-09-21T05:59:06.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.751+0000 7f040c74b740 1 --2- 172.21.15.117:0/3749493293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad75eb1390 0x55ad7627aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.751+0000 7f040c74b740 1 --2- 172.21.15.117:0/3749493293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad75ed2070 0x55ad76290cc0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.751+0000 7f040c74b740 1 --2- 172.21.15.117:0/3749493293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad75ea91b0 0x55ad76286920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.751+0000 7f040c74b740 1 -- 172.21.15.117:0/3749493293 >> 172.21.15.117:0/3749493293 conn(0x55ad75ec6460 msgr2=0x55ad7627a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:06.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.751+0000 7f040c74b740 1 -- 172.21.15.117:0/3749493293 shutdown_connections 2024-09-21T05:59:06.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.751+0000 7f040c74b740 1 -- 172.21.15.117:0/3749493293 wait complete. 2024-09-21T05:59:06.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.752+0000 7f040c74b740 1 -- start start 2024-09-21T05:59:06.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.752+0000 7f040c74b740 1 Processor -- start 2024-09-21T05:59:06.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.753+0000 7f040c74b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad75ea91b0 0x55ad7628b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:06.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.753+0000 7f040c74b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad75eb1390 0x55ad7628b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:06.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.753+0000 7f040c74b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad75ed2070 0x55ad76300440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:06.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.753+0000 7f040c74b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f04001b6ff0 con 0x55ad75ed2070 2024-09-21T05:59:06.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.753+0000 7f040c74b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f04001b6920 con 0x55ad75eb1390 2024-09-21T05:59:06.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.753+0000 7f040c74b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f04001b3b60 con 0x55ad75ea91b0 2024-09-21T05:59:06.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.753+0000 7f040c74b740 10 client.0 _ll_register_callbacks cb 0x7f03e80127c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:59:06.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.754+0000 7f03fffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad75ed2070 0x55ad76300440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:06.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.754+0000 7f03fffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad75ed2070 0x55ad76300440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45410/0 (socket says 172.21.15.117:45410) 2024-09-21T05:59:06.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.754+0000 7f03fffff640 1 -- 172.21.15.117:0/3084095477 learned_addr learned my addr 172.21.15.117:0/3084095477 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:06.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.754+0000 7f03ff7fe640 1 --2- 172.21.15.117:0/3084095477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad75ea91b0 0x55ad7628b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:06.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.754+0000 7f03f7fff640 1 --2- 172.21.15.117:0/3084095477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad75eb1390 0x55ad7628b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:06.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.754+0000 7f03fffff640 1 -- 172.21.15.117:0/3084095477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad75ea91b0 msgr2=0x55ad7628b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:06.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.754+0000 7f03fffff640 1 --2- 172.21.15.117:0/3084095477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad75ea91b0 0x55ad7628b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.754+0000 7f03fffff640 1 -- 172.21.15.117:0/3084095477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad75eb1390 msgr2=0x55ad7628b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:06.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.754+0000 7f03fffff640 1 --2- 172.21.15.117:0/3084095477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad75eb1390 0x55ad7628b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.754+0000 7f03fffff640 1 -- 172.21.15.117:0/3084095477 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad76300980 con 0x55ad75ed2070 2024-09-21T05:59:06.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.754+0000 7f03f7fff640 1 --2- 172.21.15.117:0/3084095477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad75eb1390 0x55ad7628b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:59:06.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.755+0000 7f03ff7fe640 1 --2- 172.21.15.117:0/3084095477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad75ea91b0 0x55ad7628b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:59:06.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.755+0000 7f03fffff640 1 --2- 172.21.15.117:0/3084095477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad75ed2070 0x55ad76300440 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f03f80acd50 tx=0x7f03f80a8940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78887054a914f7b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:06.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.755+0000 7f03eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:59:06.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.755+0000 7f03eeffd640 1 -- 172.21.15.117:0/3084095477 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f80e4e60 con 0x55ad75ed2070 2024-09-21T05:59:06.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.755+0000 7f040c74b740 1 -- 172.21.15.117:0/3084095477 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03f8005680 con 0x55ad75ed2070 2024-09-21T05:59:06.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.756+0000 7f03eeffd640 1 -- 172.21.15.117:0/3084095477 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f03f80aba90 con 0x55ad75ed2070 2024-09-21T05:59:06.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.756+0000 7f03eeffd640 1 -- 172.21.15.117:0/3084095477 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f03f80ad4c0 con 0x55ad75ed2070 2024-09-21T05:59:06.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.756+0000 7f040c74b740 10 client.15927 fetch_fsmap learned FSMap version 27 2024-09-21T05:59:06.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.756+0000 7f040c74b740 1 -- 172.21.15.117:0/3084095477 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ad76301b10 con 0x55ad75ed2070 2024-09-21T05:59:06.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.757+0000 7f03eeffd640 1 -- 172.21.15.117:0/3084095477 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03f8005680 con 0x55ad75ed2070 2024-09-21T05:59:06.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.757+0000 7f03eeffd640 1 -- 172.21.15.117:0/3084095477 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f03f80b1290 con 0x55ad75ed2070 2024-09-21T05:59:06.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.757+0000 7f040c74b740 10 client.15927 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:59:06.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.757+0000 7f040c74b740 10 client.15927 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T05:59:06.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.757+0000 7f040c74b740 20 client.15927 populate_metadata read hostname 'smithi117' 2024-09-21T05:59:06.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.758+0000 7f040c74b740 1 --2- 172.21.15.117:0/3084095477 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55ad75e65da0 0x55ad763041f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:06.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.758+0000 7f040c74b740 4 client.15927 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T05:59:06.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.758+0000 7f040c74b740 1 -- 172.21.15.117:0/3084095477 --> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ad761ed8a0 con 0x55ad75e65da0 2024-09-21T05:59:06.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.759+0000 7f03ff7fe640 1 --2- 172.21.15.117:0/3084095477 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55ad75e65da0 0x55ad763041f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:06.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.759+0000 7f03ff7fe640 1 --2- 172.21.15.117:0/3084095477 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55ad75e65da0 0x55ad763041f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aa1435b401555dae server_cookie=a45e3641b901e939 in_seq=0 out_seq=0 2024-09-21T05:59:06.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.759+0000 7f03eeffd640 10 client.15927 ms_handle_connect on v2:172.21.15.192:6836/2683860842 2024-09-21T05:59:06.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.759+0000 7f03eeffd640 1 -- 172.21.15.117:0/3084095477 <== mds.2 v2:172.21.15.192:6836/2683860842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ad761ed8a0 con 0x55ad75e65da0 2024-09-21T05:59:06.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.759+0000 7f03eeffd640 10 client.15927 handle_command_reply: tid=0 2024-09-21T05:59:06.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.759+0000 7f040c74b740 1 client.15927 shutdown 2024-09-21T05:59:06.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.760+0000 7f040c74b740 1 -- 172.21.15.117:0/3084095477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad75ed2070 msgr2=0x55ad76300440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:06.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.760+0000 7f040c74b740 1 --2- 172.21.15.117:0/3084095477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad75ed2070 0x55ad76300440 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f03f80acd50 tx=0x7f03f80a8940 comp rx=0 tx=0).stop 2024-09-21T05:59:06.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.760+0000 7f040c74b740 1 -- 172.21.15.117:0/3084095477 shutdown_connections 2024-09-21T05:59:06.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.760+0000 7f040c74b740 1 --2- 172.21.15.117:0/3084095477 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55ad75e65da0 0x55ad763041f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.760+0000 7f040c74b740 1 --2- 172.21.15.117:0/3084095477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ad75ed2070 0x55ad76300440 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.760+0000 7f03eeffd640 0 client.15927 ms_handle_reset on v2:172.21.15.192:6836/2683860842 2024-09-21T05:59:06.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.760+0000 7f040c74b740 1 --2- 172.21.15.117:0/3084095477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ad75eb1390 0x55ad7628b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.760+0000 7f040c74b740 1 --2- 172.21.15.117:0/3084095477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ad75ea91b0 0x55ad7628b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.760+0000 7f040c74b740 1 -- 172.21.15.117:0/3084095477 >> 172.21.15.117:0/3084095477 conn(0x55ad75ec6460 msgr2=0x55ad76286680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:06.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.760+0000 7f040c74b740 1 -- 172.21.15.117:0/3084095477 shutdown_connections 2024-09-21T05:59:06.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.760+0000 7f040c74b740 1 -- 172.21.15.117:0/3084095477 wait complete. 2024-09-21T05:59:06.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.761+0000 7f040c74b740 20 client.15927 trim_cache size 0 max 16384 2024-09-21T05:59:06.764 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T05:59:06.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.761+0000 7f04061d7640 1 -- 172.21.15.117:0/224167961 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f03e00b67d0 msgr2=0x7f03e00b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:06.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.761+0000 7f04061d7640 1 --2- 172.21.15.117:0/224167961 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f03e00b67d0 0x7f03e00b8ce0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f03f0000c00 tx=0x7f03f0031400 comp rx=0 tx=0).stop 2024-09-21T05:59:06.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.762+0000 7f04061d7640 1 -- 172.21.15.117:0/224167961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0400110150 msgr2=0x7f0400103fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:06.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.762+0000 7f04061d7640 1 --2- 172.21.15.117:0/224167961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0400110150 0x7f0400103fe0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f03f800d750 tx=0x7f03f800dc10 comp rx=0 tx=0).stop 2024-09-21T05:59:06.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.762+0000 7f04061d7640 1 -- 172.21.15.117:0/224167961 shutdown_connections 2024-09-21T05:59:06.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.762+0000 7f04061d7640 1 --2- 172.21.15.117:0/224167961 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f03e00b67d0 0x7f03e00b8ce0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.762+0000 7f04061d7640 1 --2- 172.21.15.117:0/224167961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0400110150 0x7f0400103fe0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.762+0000 7f04061d7640 1 --2- 172.21.15.117:0/224167961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04000760a0 0x7f04001038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.762+0000 7f04061d7640 1 --2- 172.21.15.117:0/224167961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0400075750 0x7f04001075e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:06.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.762+0000 7f04061d7640 1 -- 172.21.15.117:0/224167961 >> 172.21.15.117:0/224167961 conn(0x7f04000ff5e0 msgr2=0x7f04001021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:06.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.762+0000 7f04061d7640 1 -- 172.21.15.117:0/224167961 shutdown_connections 2024-09-21T05:59:06.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:06.762+0000 7f04061d7640 1 -- 172.21.15.117:0/224167961 wait complete. 2024-09-21T05:59:07.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:07 smithi192 ceph-mon[34700]: pgmap v381: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 906 MiB/s wr, 1.80k op/s 2024-09-21T05:59:08.039 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:07 smithi117 ceph-mon[30947]: pgmap v381: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 906 MiB/s wr, 1.80k op/s 2024-09-21T05:59:08.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:07 smithi138 ceph-mon[34391]: pgmap v381: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 906 MiB/s wr, 1.80k op/s 2024-09-21T05:59:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.417+0000 7fd22f4d3640 1 -- 172.21.15.117:0/3208037649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd22810fa00 msgr2=0x7fd22810fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:08.419 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.417+0000 7fd22f4d3640 1 --2- 172.21.15.117:0/3208037649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd22810fa00 0x7fd22810fe10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd22001acb0 tx=0x7fd22003b8e0 comp rx=0 tx=0).stop 2024-09-21T05:59:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.418+0000 7fd22f4d3640 1 -- 172.21.15.117:0/3208037649 shutdown_connections 2024-09-21T05:59:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.418+0000 7fd22f4d3640 1 --2- 172.21.15.117:0/3208037649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd228110350 0x7fd228110b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.418+0000 7fd22f4d3640 1 --2- 172.21.15.117:0/3208037649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd22810fa00 0x7fd22810fe10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.418+0000 7fd22f4d3640 1 --2- 172.21.15.117:0/3208037649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd22810a660 0x7fd22810aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.418+0000 7fd22f4d3640 1 -- 172.21.15.117:0/3208037649 >> 172.21.15.117:0/3208037649 conn(0x7fd228070dd0 msgr2=0x7fd2280711e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.418+0000 7fd22f4d3640 1 -- 172.21.15.117:0/3208037649 shutdown_connections 2024-09-21T05:59:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.419+0000 7fd22f4d3640 1 -- 172.21.15.117:0/3208037649 wait complete. 2024-09-21T05:59:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.419+0000 7fd22f4d3640 1 Processor -- start 2024-09-21T05:59:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.420+0000 7fd22f4d3640 1 -- start start 2024-09-21T05:59:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.420+0000 7fd22f4d3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd22810a660 0x7fd2281a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.420+0000 7fd22f4d3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd22810fa00 0x7fd2281a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.420+0000 7fd22f4d3640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd228110350 0x7fd2281a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.420+0000 7fd22f4d3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd228113fb0 con 0x7fd22810a660 2024-09-21T05:59:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.420+0000 7fd22f4d3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd228113e30 con 0x7fd22810fa00 2024-09-21T05:59:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.420+0000 7fd22f4d3640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd228114130 con 0x7fd228110350 2024-09-21T05:59:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.421+0000 7fd22d26f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd22810a660 0x7fd2281a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.421+0000 7fd22d26f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd22810a660 0x7fd2281a0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45430/0 (socket says 172.21.15.117:45430) 2024-09-21T05:59:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.421+0000 7fd22d26f640 1 -- 172.21.15.117:0/112446717 learned_addr learned my addr 172.21.15.117:0/112446717 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.421+0000 7fd22da70640 1 --2- 172.21.15.117:0/112446717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd228110350 0x7fd2281a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.422+0000 7fd22da70640 1 -- 172.21.15.117:0/112446717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd22810fa00 msgr2=0x7fd2281a0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.422+0000 7fd22da70640 1 --2- 172.21.15.117:0/112446717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd22810fa00 0x7fd2281a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.422+0000 7fd22da70640 1 -- 172.21.15.117:0/112446717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd22810a660 msgr2=0x7fd2281a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.422+0000 7fd22da70640 1 --2- 172.21.15.117:0/112446717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd22810a660 0x7fd2281a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.422+0000 7fd22da70640 1 -- 172.21.15.117:0/112446717 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2281a6970 con 0x7fd228110350 2024-09-21T05:59:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.422+0000 7fd22da70640 1 --2- 172.21.15.117:0/112446717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd228110350 0x7fd2281a6270 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd21c00d680 tx=0x7fd21c00db40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:08.426 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.424+0000 7fd21a7fc640 1 -- 172.21.15.117:0/112446717 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21c017070 con 0x7fd228110350 2024-09-21T05:59:08.426 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.424+0000 7fd21a7fc640 1 -- 172.21.15.117:0/112446717 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd21c009820 con 0x7fd228110350 2024-09-21T05:59:08.426 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.424+0000 7fd21a7fc640 1 -- 172.21.15.117:0/112446717 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21c004ce0 con 0x7fd228110350 2024-09-21T05:59:08.426 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.424+0000 7fd22f4d3640 1 -- 172.21.15.117:0/112446717 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2281a6c60 con 0x7fd228110350 2024-09-21T05:59:08.426 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.424+0000 7fd22f4d3640 1 -- 172.21.15.117:0/112446717 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2281ad3f0 con 0x7fd228110350 2024-09-21T05:59:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.431+0000 7fd21a7fc640 1 -- 172.21.15.117:0/112446717 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fd21c005220 con 0x7fd228110350 2024-09-21T05:59:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.432+0000 7fd21a7fc640 1 --2- 172.21.15.117:0/112446717 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd2140b6850 0x7fd2140b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.433+0000 7fd21a7fc640 1 -- 172.21.15.117:0/112446717 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fd21c0e6d30 con 0x7fd228110350 2024-09-21T05:59:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.433+0000 7fd22d26f640 1 --2- 172.21.15.117:0/112446717 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd2140b6850 0x7fd2140b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:08.436 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.434+0000 7fd22d26f640 1 --2- 172.21.15.117:0/112446717 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd2140b6850 0x7fd2140b8d60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd22400bca0 tx=0x7fd22400b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.437+0000 7fd236443740 1 Processor -- start 2024-09-21T05:59:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.437+0000 7fd236443740 1 -- start start 2024-09-21T05:59:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.438+0000 7fd236443740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f7f37e1b0 0x556f7f7239b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.438+0000 7fd236443740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f7f3552f0 0x556f7f723ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.438+0000 7fd236443740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7f35d4d0 0x556f7f724430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.439+0000 7fd22da70640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7f35d4d0 0x556f7f724430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.439+0000 7fd22da70640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7f35d4d0 0x556f7f724430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45446/0 (socket says 172.21.15.117:45446) 2024-09-21T05:59:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.439+0000 7fd22da70640 1 -- 172.21.15.117:0/4127483996 learned_addr learned my addr 172.21.15.117:0/4127483996 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.439+0000 7fd236443740 1 -- 172.21.15.117:0/4127483996 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556f7f6986f0 con 0x556f7f35d4d0 2024-09-21T05:59:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.439+0000 7fd236443740 1 -- 172.21.15.117:0/4127483996 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556f7f687b20 con 0x556f7f37e1b0 2024-09-21T05:59:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.439+0000 7fd236443740 1 -- 172.21.15.117:0/4127483996 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556f7f68b930 con 0x556f7f3552f0 2024-09-21T05:59:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.440+0000 7fd22ca6e640 1 --2- 172.21.15.117:0/4127483996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f7f3552f0 0x556f7f723ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.440+0000 7fd22d26f640 1 --2- 172.21.15.117:0/4127483996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f7f37e1b0 0x556f7f7239b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.442+0000 7fd22ca6e640 1 -- 172.21.15.117:0/4127483996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f7f37e1b0 msgr2=0x556f7f7239b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.442+0000 7fd22ca6e640 1 --2- 172.21.15.117:0/4127483996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f7f37e1b0 0x556f7f7239b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.442+0000 7fd22ca6e640 1 -- 172.21.15.117:0/4127483996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7f35d4d0 msgr2=0x556f7f724430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.442+0000 7fd22ca6e640 1 --2- 172.21.15.117:0/4127483996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7f35d4d0 0x556f7f724430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.442+0000 7fd22ca6e640 1 -- 172.21.15.117:0/4127483996 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f7f64d530 con 0x556f7f3552f0 2024-09-21T05:59:08.447 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.445+0000 7fd22ca6e640 1 --2- 172.21.15.117:0/4127483996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f7f3552f0 0x556f7f723ef0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd22806c5c0 tx=0x7fd220042820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95815e7d6ac1edad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:08.448 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.446+0000 7fd2137fe640 1 -- 172.21.15.117:0/4127483996 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22003e070 con 0x556f7f3552f0 2024-09-21T05:59:08.448 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.447+0000 7fd2137fe640 1 -- 172.21.15.117:0/4127483996 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd220050070 con 0x556f7f3552f0 2024-09-21T05:59:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.448+0000 7fd236443740 1 -- 172.21.15.117:0/4127483996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f7f3552f0 msgr2=0x556f7f723ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.448+0000 7fd236443740 1 --2- 172.21.15.117:0/4127483996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f7f3552f0 0x556f7f723ef0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd22806c5c0 tx=0x7fd220042820 comp rx=0 tx=0).stop 2024-09-21T05:59:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.448+0000 7fd236443740 1 -- 172.21.15.117:0/4127483996 shutdown_connections 2024-09-21T05:59:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.448+0000 7fd236443740 1 --2- 172.21.15.117:0/4127483996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7f35d4d0 0x556f7f724430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.448+0000 7fd236443740 1 --2- 172.21.15.117:0/4127483996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f7f3552f0 0x556f7f723ef0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:08.451 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.448+0000 7fd236443740 1 --2- 172.21.15.117:0/4127483996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f7f37e1b0 0x556f7f7239b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:08.451 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.448+0000 7fd236443740 1 -- 172.21.15.117:0/4127483996 >> 172.21.15.117:0/4127483996 conn(0x556f7f372300 msgr2=0x556f7f7218a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.452+0000 7fd236443740 1 -- 172.21.15.117:0/4127483996 shutdown_connections 2024-09-21T05:59:08.457 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.452+0000 7fd236443740 1 -- 172.21.15.117:0/4127483996 wait complete. 2024-09-21T05:59:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.452+0000 7fd236443740 1 -- start start 2024-09-21T05:59:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.453+0000 7fd236443740 1 Processor -- start 2024-09-21T05:59:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.453+0000 7fd236443740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7f3552f0 0x556f7f72ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.453+0000 7fd236443740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f7f35d4d0 0x556f7f72f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.453+0000 7fd236443740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f7f37e1b0 0x556f7f72f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.453+0000 7fd236443740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556f7f5bfab0 con 0x556f7f3552f0 2024-09-21T05:59:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.453+0000 7fd236443740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556f7f62d730 con 0x556f7f35d4d0 2024-09-21T05:59:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.453+0000 7fd236443740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556f7f732120 con 0x556f7f37e1b0 2024-09-21T05:59:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.453+0000 7fd236443740 10 client.0 _ll_register_callbacks cb 0x556f7f6816b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:59:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.454+0000 7fd22d26f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7f3552f0 0x556f7f72ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.454+0000 7fd22d26f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7f3552f0 0x556f7f72ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45448/0 (socket says 172.21.15.117:45448) 2024-09-21T05:59:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.454+0000 7fd22d26f640 1 -- 172.21.15.117:0/3476033146 learned_addr learned my addr 172.21.15.117:0/3476033146 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.455+0000 7fd22d26f640 1 -- 172.21.15.117:0/3476033146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f7f37e1b0 msgr2=0x556f7f72f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.455+0000 7fd22d26f640 1 --2- 172.21.15.117:0/3476033146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f7f37e1b0 0x556f7f72f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.455+0000 7fd22d26f640 1 -- 172.21.15.117:0/3476033146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f7f35d4d0 msgr2=0x556f7f72f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.455+0000 7fd22d26f640 1 --2- 172.21.15.117:0/3476033146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f7f35d4d0 0x556f7f72f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.455+0000 7fd22d26f640 1 -- 172.21.15.117:0/3476033146 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f7f3e9790 con 0x556f7f3552f0 2024-09-21T05:59:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.456+0000 7fd22d26f640 1 --2- 172.21.15.117:0/3476033146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7f3552f0 0x556f7f72ef70 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd2240117c0 tx=0x7fd224004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb51014b1927bc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:08.461 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.459+0000 7fd210ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:59:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.466+0000 7fd210ff9640 1 -- 172.21.15.117:0/3476033146 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22401e940 con 0x556f7f3552f0 2024-09-21T05:59:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.466+0000 7fd236443740 1 -- 172.21.15.117:0/3476033146 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd220050070 con 0x556f7f3552f0 2024-09-21T05:59:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.466+0000 7fd210ff9640 1 -- 172.21.15.117:0/3476033146 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fd22401eae0 con 0x556f7f3552f0 2024-09-21T05:59:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.471+0000 7fd210ff9640 1 -- 172.21.15.117:0/3476033146 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd22401ed20 con 0x556f7f3552f0 2024-09-21T05:59:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.471+0000 7fd236443740 10 client.15939 fetch_fsmap learned FSMap version 27 2024-09-21T05:59:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.471+0000 7fd236443740 1 -- 172.21.15.117:0/3476033146 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556f7f462620 con 0x556f7f3552f0 2024-09-21T05:59:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.473+0000 7fd210ff9640 1 -- 172.21.15.117:0/3476033146 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd220050070 con 0x556f7f3552f0 2024-09-21T05:59:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.473+0000 7fd210ff9640 1 -- 172.21.15.117:0/3476033146 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd224028430 con 0x556f7f3552f0 2024-09-21T05:59:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.473+0000 7fd236443740 10 client.15939 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:59:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.473+0000 7fd236443740 10 client.15939 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T05:59:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.473+0000 7fd236443740 20 client.15939 populate_metadata read hostname 'smithi117' 2024-09-21T05:59:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.473+0000 7fd236443740 1 --2- 172.21.15.117:0/3476033146 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x556f7f33cd80 0x556f7f763310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.473+0000 7fd236443740 4 client.15939 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:59:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.473+0000 7fd236443740 1 -- 172.21.15.117:0/3476033146 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f7f69a640 con 0x556f7f33cd80 2024-09-21T05:59:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.486+0000 7fd22ca6e640 1 --2- 172.21.15.117:0/3476033146 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x556f7f33cd80 0x556f7f763310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.499+0000 7fd210ff9640 10 client.15939 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:59:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.499+0000 7fd22ca6e640 1 --2- 172.21.15.117:0/3476033146 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x556f7f33cd80 0x556f7f763310 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c556570cb983ea05 server_cookie=31a8880e18573448 in_seq=0 out_seq=0 2024-09-21T05:59:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.506+0000 7fd210ff9640 1 -- 172.21.15.117:0/3476033146 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556f7f69a640 con 0x556f7f33cd80 2024-09-21T05:59:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.507+0000 7fd210ff9640 10 client.15939 handle_command_reply: tid=0 2024-09-21T05:59:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.507+0000 7fd236443740 1 client.15939 shutdown 2024-09-21T05:59:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.509+0000 7fd236443740 1 -- 172.21.15.117:0/3476033146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7f3552f0 msgr2=0x556f7f72ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.509+0000 7fd236443740 1 --2- 172.21.15.117:0/3476033146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7f3552f0 0x556f7f72ef70 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd2240117c0 tx=0x7fd224004730 comp rx=0 tx=0).stop 2024-09-21T05:59:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.509+0000 7fd236443740 1 -- 172.21.15.117:0/3476033146 shutdown_connections 2024-09-21T05:59:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.509+0000 7fd236443740 1 --2- 172.21.15.117:0/3476033146 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x556f7f33cd80 0x556f7f763310 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.510+0000 7fd236443740 1 --2- 172.21.15.117:0/3476033146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f7f37e1b0 0x556f7f72f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.510+0000 7fd236443740 1 --2- 172.21.15.117:0/3476033146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f7f35d4d0 0x556f7f72f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.510+0000 7fd236443740 1 --2- 172.21.15.117:0/3476033146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7f3552f0 0x556f7f72ef70 secure :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd2240117c0 tx=0x7fd224004730 comp rx=0 tx=0).stop 2024-09-21T05:59:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.510+0000 7fd236443740 1 -- 172.21.15.117:0/3476033146 >> 172.21.15.117:0/3476033146 conn(0x556f7f372300 msgr2=0x556f7f7249e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.510+0000 7fd210ff9640 0 client.15939 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:59:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.511+0000 7fd236443740 1 -- 172.21.15.117:0/3476033146 shutdown_connections 2024-09-21T05:59:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.514+0000 7fd236443740 1 -- 172.21.15.117:0/3476033146 wait complete. 2024-09-21T05:59:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.516+0000 7fd236443740 20 client.15939 trim_cache size 0 max 16384 2024-09-21T05:59:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.570+0000 7fd236443740 1 Processor -- start 2024-09-21T05:59:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.570+0000 7fd236443740 1 -- start start 2024-09-21T05:59:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.571+0000 7fd236443740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f7f35d4d0 0x556f7f72f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.571+0000 7fd236443740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7f37e1b0 0x556f7f730ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.572+0000 7fd236443740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f7f3121c0 0x556f7f72c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.572+0000 7fd236443740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd20c00e740 con 0x556f7f37e1b0 2024-09-21T05:59:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.572+0000 7fd236443740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd20c010280 con 0x556f7f3121c0 2024-09-21T05:59:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.572+0000 7fd236443740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd20c00ee10 con 0x556f7f35d4d0 2024-09-21T05:59:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.573+0000 7fd22ca6e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7f37e1b0 0x556f7f730ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.573+0000 7fd22ca6e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7f37e1b0 0x556f7f730ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45458/0 (socket says 172.21.15.117:45458) 2024-09-21T05:59:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.573+0000 7fd22ca6e640 1 -- 172.21.15.117:0/379419796 learned_addr learned my addr 172.21.15.117:0/379419796 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.574+0000 7fd22ca6e640 1 -- 172.21.15.117:0/379419796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f7f35d4d0 msgr2=0x556f7f72f7d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T05:59:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.574+0000 7fd22ca6e640 1 --2- 172.21.15.117:0/379419796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f7f35d4d0 0x556f7f72f7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.574+0000 7fd22ca6e640 1 -- 172.21.15.117:0/379419796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f7f3121c0 msgr2=0x556f7f72c140 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T05:59:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.574+0000 7fd22ca6e640 1 --2- 172.21.15.117:0/379419796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f7f3121c0 0x556f7f72c140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:08.578 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.574+0000 7fd22ca6e640 1 -- 172.21.15.117:0/379419796 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f7f725cb0 con 0x556f7f37e1b0 2024-09-21T05:59:08.578 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.575+0000 7fd22ca6e640 1 --2- 172.21.15.117:0/379419796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7f37e1b0 0x556f7f730ba0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd22003be00 tx=0x7fd22001a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1376108d0ad0eb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:08.578 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.576+0000 7fd212ffd640 1 -- 172.21.15.117:0/379419796 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd220003b70 con 0x556f7f37e1b0 2024-09-21T05:59:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.576+0000 7fd212ffd640 1 -- 172.21.15.117:0/379419796 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd220050040 con 0x556f7f37e1b0 2024-09-21T05:59:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.576+0000 7fd212ffd640 1 -- 172.21.15.117:0/379419796 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd220041840 con 0x556f7f37e1b0 2024-09-21T05:59:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.578+0000 7fd236443740 1 -- 172.21.15.117:0/379419796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7f37e1b0 msgr2=0x556f7f730ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.578+0000 7fd236443740 1 --2- 172.21.15.117:0/379419796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7f37e1b0 0x556f7f730ba0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd22003be00 tx=0x7fd22001a5d0 comp rx=0 tx=0).stop 2024-09-21T05:59:08.581 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.578+0000 7fd236443740 1 -- 172.21.15.117:0/379419796 shutdown_connections 2024-09-21T05:59:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.578+0000 7fd236443740 1 --2- 172.21.15.117:0/379419796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7f37e1b0 0x556f7f730ba0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.578+0000 7fd236443740 1 --2- 172.21.15.117:0/379419796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f7f3121c0 0x556f7f72c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.578+0000 7fd236443740 1 --2- 172.21.15.117:0/379419796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f7f35d4d0 0x556f7f72f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.578+0000 7fd236443740 1 -- 172.21.15.117:0/379419796 >> 172.21.15.117:0/379419796 conn(0x556f7f372300 msgr2=0x556f7f730490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.578+0000 7fd236443740 1 -- 172.21.15.117:0/379419796 shutdown_connections 2024-09-21T05:59:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.578+0000 7fd236443740 1 -- 172.21.15.117:0/379419796 wait complete. 2024-09-21T05:59:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.579+0000 7fd236443740 1 -- start start 2024-09-21T05:59:08.582 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.579+0000 7fd236443740 1 Processor -- start 2024-09-21T05:59:08.584 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.579+0000 7fd236443740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f7f3121c0 0x556f7f734e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:08.584 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.579+0000 7fd236443740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f7f35d4d0 0x556f7f7353e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:08.585 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.580+0000 7fd236443740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7f37e1b0 0x556f7f7c5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:08.585 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.580+0000 7fd236443740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd20c00fbb0 con 0x556f7f37e1b0 2024-09-21T05:59:08.585 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.580+0000 7fd236443740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd20c00f4e0 con 0x556f7f3121c0 2024-09-21T05:59:08.585 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.580+0000 7fd236443740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd20c00c720 con 0x556f7f35d4d0 2024-09-21T05:59:08.585 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.580+0000 7fd236443740 10 client.0 _ll_register_callbacks cb 0x7fd22400bcd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:59:08.585 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.580+0000 7fd22da70640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7f37e1b0 0x556f7f7c5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:08.585 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.580+0000 7fd22da70640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7f37e1b0 0x556f7f7c5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45468/0 (socket says 172.21.15.117:45468) 2024-09-21T05:59:08.585 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.580+0000 7fd22da70640 1 -- 172.21.15.117:0/1439075712 learned_addr learned my addr 172.21.15.117:0/1439075712 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:08.585 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.580+0000 7fd22da70640 1 -- 172.21.15.117:0/1439075712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f7f35d4d0 msgr2=0x556f7f7353e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:08.585 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.580+0000 7fd22da70640 1 --2- 172.21.15.117:0/1439075712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f7f35d4d0 0x556f7f7353e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:08.585 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.580+0000 7fd22da70640 1 -- 172.21.15.117:0/1439075712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f7f3121c0 msgr2=0x556f7f734e00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T05:59:08.585 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.580+0000 7fd22da70640 1 --2- 172.21.15.117:0/1439075712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f7f3121c0 0x556f7f734e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:08.585 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.580+0000 7fd22da70640 1 -- 172.21.15.117:0/1439075712 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f7f7c6380 con 0x556f7f37e1b0 2024-09-21T05:59:08.585 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.580+0000 7fd22da70640 1 --2- 172.21.15.117:0/1439075712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7f37e1b0 0x556f7f7c5ce0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd21c00d650 tx=0x7fd21c0ac380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69fefe761feda514 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:08.586 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.580+0000 7fd210ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:59:08.586 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.581+0000 7fd210ff9640 1 -- 172.21.15.117:0/1439075712 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21c020040 con 0x556f7f37e1b0 2024-09-21T05:59:08.586 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.581+0000 7fd210ff9640 1 -- 172.21.15.117:0/1439075712 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fd21c0abe00 con 0x556f7f37e1b0 2024-09-21T05:59:08.586 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.581+0000 7fd236443740 1 -- 172.21.15.117:0/1439075712 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd220050040 con 0x556f7f37e1b0 2024-09-21T05:59:08.586 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.581+0000 7fd210ff9640 1 -- 172.21.15.117:0/1439075712 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd21c0d4170 con 0x556f7f37e1b0 2024-09-21T05:59:08.586 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.581+0000 7fd236443740 10 client.15945 fetch_fsmap learned FSMap version 27 2024-09-21T05:59:08.586 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.581+0000 7fd236443740 1 -- 172.21.15.117:0/1439075712 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556f7f7cd7a0 con 0x556f7f37e1b0 2024-09-21T05:59:08.586 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.581+0000 7fd210ff9640 1 -- 172.21.15.117:0/1439075712 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd220050040 con 0x556f7f37e1b0 2024-09-21T05:59:08.586 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.581+0000 7fd210ff9640 1 -- 172.21.15.117:0/1439075712 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd21c0ab870 con 0x556f7f37e1b0 2024-09-21T05:59:08.586 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.581+0000 7fd236443740 10 client.15945 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:59:08.586 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.581+0000 7fd236443740 10 client.15945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T05:59:08.586 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.581+0000 7fd236443740 20 client.15945 populate_metadata read hostname 'smithi117' 2024-09-21T05:59:08.586 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.581+0000 7fd236443740 1 --2- 172.21.15.117:0/1439075712 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x556f7f7cdfa0 0x556f7f7d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:08.587 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.581+0000 7fd236443740 4 client.15945 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T05:59:08.587 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.581+0000 7fd236443740 1 -- 172.21.15.117:0/1439075712 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556f7f6960f0 con 0x556f7f7cdfa0 2024-09-21T05:59:08.587 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.583+0000 7fd22d26f640 1 --2- 172.21.15.117:0/1439075712 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x556f7f7cdfa0 0x556f7f7d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:08.590 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.588+0000 7fd210ff9640 10 client.15945 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:59:08.590 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:08.588+0000 7fd22d26f640 1 --2- 172.21.15.117:0/1439075712 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x556f7f7cdfa0 0x556f7f7d03c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e137e70835659977 server_cookie=9b8410bf08c0ff46 in_seq=0 out_seq=0 2024-09-21T05:59:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:09.429+0000 7fd210ff9640 1 -- 172.21.15.117:0/1439075712 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556f7f6960f0 con 0x556f7f7cdfa0 2024-09-21T05:59:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:09.429+0000 7fd210ff9640 10 client.15945 handle_command_reply: tid=0 2024-09-21T05:59:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:09.429+0000 7fd236443740 1 client.15945 shutdown 2024-09-21T05:59:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:09.429+0000 7fd236443740 1 -- 172.21.15.117:0/1439075712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7f37e1b0 msgr2=0x556f7f7c5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:09.429+0000 7fd236443740 1 --2- 172.21.15.117:0/1439075712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7f37e1b0 0x556f7f7c5ce0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd21c00d650 tx=0x7fd21c0ac380 comp rx=0 tx=0).stop 2024-09-21T05:59:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:09.429+0000 7fd236443740 1 -- 172.21.15.117:0/1439075712 shutdown_connections 2024-09-21T05:59:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:09.429+0000 7fd236443740 1 --2- 172.21.15.117:0/1439075712 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x556f7f7cdfa0 0x556f7f7d03c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:09.429+0000 7fd210ff9640 0 client.15945 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:59:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:09.429+0000 7fd236443740 1 --2- 172.21.15.117:0/1439075712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f7f37e1b0 0x556f7f7c5ce0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:09.429+0000 7fd236443740 1 --2- 172.21.15.117:0/1439075712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f7f35d4d0 0x556f7f7353e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:09.429+0000 7fd236443740 1 --2- 172.21.15.117:0/1439075712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f7f3121c0 0x556f7f734e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:09.429+0000 7fd236443740 1 -- 172.21.15.117:0/1439075712 >> 172.21.15.117:0/1439075712 conn(0x556f7f372300 msgr2=0x556f7f721880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:09.430+0000 7fd236443740 1 -- 172.21.15.117:0/1439075712 shutdown_connections 2024-09-21T05:59:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:09.430+0000 7fd236443740 1 -- 172.21.15.117:0/1439075712 wait complete. 2024-09-21T05:59:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:09.430+0000 7fd236443740 20 client.15945 trim_cache size 0 max 16384 2024-09-21T05:59:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T05:59:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T05:59:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T05:59:09.433 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T05:59:09.434 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:09.432+0000 7fd22f4d3640 1 -- 172.21.15.117:0/112446717 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd2140b6850 msgr2=0x7fd2140b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:09.432+0000 7fd22f4d3640 1 --2- 172.21.15.117:0/112446717 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd2140b6850 0x7fd2140b8d60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd22400bca0 tx=0x7fd22400b410 comp rx=0 tx=0).stop 2024-09-21T05:59:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:09.432+0000 7fd22f4d3640 1 -- 172.21.15.117:0/112446717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd228110350 msgr2=0x7fd2281a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:09.432+0000 7fd22f4d3640 1 --2- 172.21.15.117:0/112446717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd228110350 0x7fd2281a6270 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd21c00d680 tx=0x7fd21c00db40 comp rx=0 tx=0).stop 2024-09-21T05:59:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:09.432+0000 7fd22f4d3640 1 -- 172.21.15.117:0/112446717 shutdown_connections 2024-09-21T05:59:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:09.432+0000 7fd22f4d3640 1 --2- 172.21.15.117:0/112446717 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd2140b6850 0x7fd2140b8d60 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:09.432+0000 7fd22f4d3640 1 --2- 172.21.15.117:0/112446717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd228110350 0x7fd2281a6270 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:09.432+0000 7fd22f4d3640 1 --2- 172.21.15.117:0/112446717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd22810fa00 0x7fd2281a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:09.432+0000 7fd22f4d3640 1 --2- 172.21.15.117:0/112446717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd22810a660 0x7fd2281a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:09.432+0000 7fd22f4d3640 1 -- 172.21.15.117:0/112446717 >> 172.21.15.117:0/112446717 conn(0x7fd228070dd0 msgr2=0x7fd228073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:09.433+0000 7fd22f4d3640 1 -- 172.21.15.117:0/112446717 shutdown_connections 2024-09-21T05:59:09.436 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:09.433+0000 7fd22f4d3640 1 -- 172.21.15.117:0/112446717 wait complete. 2024-09-21T05:59:09.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:09 smithi192 ceph-mon[34700]: pgmap v382: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 996 MiB/s wr, 2.73k op/s 2024-09-21T05:59:10.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:09 smithi117 ceph-mon[30947]: pgmap v382: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 996 MiB/s wr, 2.73k op/s 2024-09-21T05:59:10.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:09 smithi138 ceph-mon[34391]: pgmap v382: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 996 MiB/s wr, 2.73k op/s 2024-09-21T05:59:11.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:11 smithi192 ceph-mon[34700]: pgmap v383: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 798 MiB/s wr, 2.59k op/s 2024-09-21T05:59:12.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:11 smithi117 ceph-mon[30947]: pgmap v383: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 798 MiB/s wr, 2.59k op/s 2024-09-21T05:59:12.055 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T05:59:12.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:11 smithi138 ceph-mon[34391]: pgmap v383: 97 pgs: 97 active+clean; 12 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 798 MiB/s wr, 2.59k op/s 2024-09-21T05:59:12.999 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:59:13.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:13 smithi192 ceph-mon[34700]: pgmap v384: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 897 MiB/s wr, 3.48k op/s 2024-09-21T05:59:14.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:13 smithi117 ceph-mon[30947]: pgmap v384: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 897 MiB/s wr, 3.48k op/s 2024-09-21T05:59:14.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:13 smithi138 ceph-mon[34391]: pgmap v384: 97 pgs: 97 active+clean; 12 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 897 MiB/s wr, 3.48k op/s 2024-09-21T05:59:15.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:15 smithi192 ceph-mon[34700]: pgmap v385: 97 pgs: 97 active+clean; 13 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 872 MiB/s wr, 3.91k op/s 2024-09-21T05:59:16.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:15 smithi117 ceph-mon[30947]: pgmap v385: 97 pgs: 97 active+clean; 13 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 872 MiB/s wr, 3.91k op/s 2024-09-21T05:59:16.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:15 smithi138 ceph-mon[34391]: pgmap v385: 97 pgs: 97 active+clean; 13 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 872 MiB/s wr, 3.91k op/s 2024-09-21T05:59:17.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:17 smithi192 ceph-mon[34700]: pgmap v386: 97 pgs: 97 active+clean; 14 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 819 MiB/s wr, 4.09k op/s 2024-09-21T05:59:18.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:17 smithi117 ceph-mon[30947]: pgmap v386: 97 pgs: 97 active+clean; 14 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 819 MiB/s wr, 4.09k op/s 2024-09-21T05:59:18.062 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T05:59:18.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.510+0000 7f874ad03640 1 -- 172.21.15.117:0/4097032684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8744111f40 msgr2=0x7f874410a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:18.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.510+0000 7f874ad03640 1 --2- 172.21.15.117:0/4097032684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8744111f40 0x7f874410a3c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8738009900 tx=0x7f873802a440 comp rx=0 tx=0).stop 2024-09-21T05:59:18.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.510+0000 7f874ad03640 1 -- 172.21.15.117:0/4097032684 shutdown_connections 2024-09-21T05:59:18.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.510+0000 7f874ad03640 1 --2- 172.21.15.117:0/4097032684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f874411cc30 0x7f874411f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:18.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.510+0000 7f874ad03640 1 --2- 172.21.15.117:0/4097032684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f874410a9e0 0x7f874410adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:18.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.510+0000 7f874ad03640 1 --2- 172.21.15.117:0/4097032684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8744111f40 0x7f874410a3c0 secure :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8738009900 tx=0x7f873802a440 comp rx=0 tx=0).stop 2024-09-21T05:59:18.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.510+0000 7f874ad03640 1 -- 172.21.15.117:0/4097032684 >> 172.21.15.117:0/4097032684 conn(0x7f8744070ec0 msgr2=0x7f87440712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:18.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.516+0000 7f874ad03640 1 -- 172.21.15.117:0/4097032684 shutdown_connections 2024-09-21T05:59:18.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.520+0000 7f874ad03640 1 -- 172.21.15.117:0/4097032684 wait complete. 2024-09-21T05:59:18.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.522+0000 7f874ad03640 1 Processor -- start 2024-09-21T05:59:18.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.522+0000 7f874ad03640 1 -- start start 2024-09-21T05:59:18.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.523+0000 7f874ad03640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f874410a9e0 0x7f87441a94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:18.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.523+0000 7f874ad03640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f874411cc30 0x7f87441a9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:18.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.523+0000 7f874ad03640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87441aec70 0x7f87441af080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:18.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.523+0000 7f874ad03640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8744122960 con 0x7f874410a9e0 2024-09-21T05:59:18.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.523+0000 7f874ad03640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f87441227e0 con 0x7f87441aec70 2024-09-21T05:59:18.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.523+0000 7f874ad03640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8744122ae0 con 0x7f874411cc30 2024-09-21T05:59:18.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.527+0000 7f87492a0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87441aec70 0x7f87441af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:18.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.527+0000 7f8743fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f874411cc30 0x7f87441a9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:18.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.531+0000 7f87492a0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87441aec70 0x7f87441af080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:48848/0 (socket says 172.21.15.117:48848) 2024-09-21T05:59:18.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.531+0000 7f87492a0640 1 -- 172.21.15.117:0/365368342 learned_addr learned my addr 172.21.15.117:0/365368342 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:18.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.535+0000 7f8748a9f640 1 --2- 172.21.15.117:0/365368342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f874410a9e0 0x7f87441a94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:18.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.538+0000 7f87492a0640 1 -- 172.21.15.117:0/365368342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f874411cc30 msgr2=0x7f87441a9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:18.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.538+0000 7f87492a0640 1 --2- 172.21.15.117:0/365368342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f874411cc30 0x7f87441a9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:18.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.538+0000 7f87492a0640 1 -- 172.21.15.117:0/365368342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f874410a9e0 msgr2=0x7f87441a94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:18.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.538+0000 7f87492a0640 1 --2- 172.21.15.117:0/365368342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f874410a9e0 0x7f87441a94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:18.064 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.538+0000 7f87492a0640 1 -- 172.21.15.117:0/365368342 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87441af850 con 0x7f87441aec70 2024-09-21T05:59:18.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.540+0000 7f87492a0640 1 --2- 172.21.15.117:0/365368342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87441aec70 0x7f87441af080 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f873400e970 tx=0x7f873400ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:18.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.541+0000 7f8741ffb640 1 -- 172.21.15.117:0/365368342 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873400ccc0 con 0x7f87441aec70 2024-09-21T05:59:18.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.541+0000 7f8741ffb640 1 -- 172.21.15.117:0/365368342 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f873400ce60 con 0x7f87441aec70 2024-09-21T05:59:18.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.541+0000 7f8741ffb640 1 -- 172.21.15.117:0/365368342 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8734010430 con 0x7f87441aec70 2024-09-21T05:59:18.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.544+0000 7f874ad03640 1 -- 172.21.15.117:0/365368342 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8744112380 con 0x7f87441aec70 2024-09-21T05:59:18.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.544+0000 7f874ad03640 1 -- 172.21.15.117:0/365368342 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8744112870 con 0x7f87441aec70 2024-09-21T05:59:18.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.550+0000 7f8741ffb640 1 -- 172.21.15.117:0/365368342 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f8734004ff0 con 0x7f87441aec70 2024-09-21T05:59:18.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.552+0000 7f8741ffb640 1 --2- 172.21.15.117:0/365368342 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f872c0b67b0 0x7f872c0b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:18.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.553+0000 7f8741ffb640 1 -- 172.21.15.117:0/365368342 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f87340e7b50 con 0x7f87441aec70 2024-09-21T05:59:18.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.554+0000 7f8748a9f640 1 --2- 172.21.15.117:0/365368342 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f872c0b67b0 0x7f872c0b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:18.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.556+0000 7f8751277740 1 Processor -- start 2024-09-21T05:59:18.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.556+0000 7f8748a9f640 1 --2- 172.21.15.117:0/365368342 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f872c0b67b0 0x7f872c0b8cc0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8738009850 tx=0x7f8738000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:18.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.556+0000 7f8751277740 1 -- start start 2024-09-21T05:59:18.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.557+0000 7f8751277740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c84da070 0x5601c88816a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:18.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.557+0000 7f8751277740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5601c84b11b0 0x5601c8881be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:18.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.557+0000 7f8751277740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c84b9390 0x5601c8882120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:18.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.557+0000 7f8751277740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5601c87f7ea0 con 0x5601c84b9390 2024-09-21T05:59:18.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.558+0000 7f8751277740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5601c87e72d0 con 0x5601c84da070 2024-09-21T05:59:18.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.558+0000 7f8751277740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5601c87eb0e0 con 0x5601c84b11b0 2024-09-21T05:59:18.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.558+0000 7f8748a9f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c84da070 0x5601c88816a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:18.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.558+0000 7f8748a9f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c84da070 0x5601c88816a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:48864/0 (socket says 172.21.15.117:48864) 2024-09-21T05:59:18.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.558+0000 7f8748a9f640 1 -- 172.21.15.117:0/156069677 learned_addr learned my addr 172.21.15.117:0/156069677 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:18.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.560+0000 7f8743fff640 1 --2- 172.21.15.117:0/156069677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5601c84b11b0 0x5601c8881be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:18.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.560+0000 7f87492a0640 1 --2- 172.21.15.117:0/156069677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c84b9390 0x5601c8882120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:18.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.560+0000 7f8743fff640 1 -- 172.21.15.117:0/156069677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c84da070 msgr2=0x5601c88816a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:18.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.560+0000 7f8743fff640 1 --2- 172.21.15.117:0/156069677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c84da070 0x5601c88816a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:18.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.560+0000 7f8743fff640 1 -- 172.21.15.117:0/156069677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c84b9390 msgr2=0x5601c8882120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:18.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.560+0000 7f8743fff640 1 --2- 172.21.15.117:0/156069677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c84b9390 0x5601c8882120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:18.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.560+0000 7f8743fff640 1 -- 172.21.15.117:0/156069677 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601c8545650 con 0x5601c84b11b0 2024-09-21T05:59:18.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.562+0000 7f8743fff640 1 --2- 172.21.15.117:0/156069677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5601c84b11b0 0x5601c8881be0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f874406c970 tx=0x7f873c0096d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3241930f953627d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:18.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.562+0000 7f872affd640 1 -- 172.21.15.117:0/156069677 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873c013790 con 0x5601c84b11b0 2024-09-21T05:59:18.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.562+0000 7f872affd640 1 -- 172.21.15.117:0/156069677 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f873c019070 con 0x5601c84b11b0 2024-09-21T05:59:18.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.564+0000 7f8751277740 1 -- 172.21.15.117:0/156069677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5601c84b11b0 msgr2=0x5601c8881be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:18.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.564+0000 7f8751277740 1 --2- 172.21.15.117:0/156069677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5601c84b11b0 0x5601c8881be0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f874406c970 tx=0x7f873c0096d0 comp rx=0 tx=0).stop 2024-09-21T05:59:18.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.564+0000 7f8751277740 1 -- 172.21.15.117:0/156069677 shutdown_connections 2024-09-21T05:59:18.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.564+0000 7f8751277740 1 --2- 172.21.15.117:0/156069677 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c84b9390 0x5601c8882120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:18.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.564+0000 7f8751277740 1 --2- 172.21.15.117:0/156069677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5601c84b11b0 0x5601c8881be0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:18.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.564+0000 7f8751277740 1 --2- 172.21.15.117:0/156069677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c84da070 0x5601c88816a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:18.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.564+0000 7f8751277740 1 -- 172.21.15.117:0/156069677 >> 172.21.15.117:0/156069677 conn(0x5601c84ce460 msgr2=0x5601c887f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:18.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.565+0000 7f8751277740 1 -- 172.21.15.117:0/156069677 shutdown_connections 2024-09-21T05:59:18.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.565+0000 7f8751277740 1 -- 172.21.15.117:0/156069677 wait complete. 2024-09-21T05:59:18.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.565+0000 7f8751277740 1 -- start start 2024-09-21T05:59:18.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.566+0000 7f8751277740 1 Processor -- start 2024-09-21T05:59:18.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.567+0000 7f8751277740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5601c84b11b0 0x5601c888cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:18.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.567+0000 7f8751277740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c84b9390 0x5601c888d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:18.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.567+0000 7f8751277740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c84da070 0x5601c888d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:18.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.567+0000 7f8751277740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5601c8648700 con 0x5601c84b9390 2024-09-21T05:59:18.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.567+0000 7f8751277740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5601c8782160 con 0x5601c84da070 2024-09-21T05:59:18.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.567+0000 7f8751277740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5601c888fed0 con 0x5601c84b11b0 2024-09-21T05:59:18.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.567+0000 7f8751277740 10 client.0 _ll_register_callbacks cb 0x5601c879b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:59:18.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.569+0000 7f8743fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c84b9390 0x5601c888d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:18.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.569+0000 7f8743fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c84b9390 0x5601c888d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40462/0 (socket says 172.21.15.117:40462) 2024-09-21T05:59:18.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.569+0000 7f8743fff640 1 -- 172.21.15.117:0/1361037056 learned_addr learned my addr 172.21.15.117:0/1361037056 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:18.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.569+0000 7f87492a0640 1 --2- 172.21.15.117:0/1361037056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c84da070 0x5601c888d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:18.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.569+0000 7f8748a9f640 1 --2- 172.21.15.117:0/1361037056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5601c84b11b0 0x5601c888cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:18.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.570+0000 7f8743fff640 1 -- 172.21.15.117:0/1361037056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5601c84b11b0 msgr2=0x5601c888cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:18.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.570+0000 7f8743fff640 1 --2- 172.21.15.117:0/1361037056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5601c84b11b0 0x5601c888cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:18.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.570+0000 7f8743fff640 1 -- 172.21.15.117:0/1361037056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c84da070 msgr2=0x5601c888d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:18.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.570+0000 7f8743fff640 1 --2- 172.21.15.117:0/1361037056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c84da070 0x5601c888d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:18.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.571+0000 7f8743fff640 1 -- 172.21.15.117:0/1361037056 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601c87afc90 con 0x5601c84b9390 2024-09-21T05:59:18.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.571+0000 7f8743fff640 1 --2- 172.21.15.117:0/1361037056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c84b9390 0x5601c888d1b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f873c0149d0 tx=0x7f873c014650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25dafb077fc210cc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:18.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.571+0000 7f8723fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:59:18.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.572+0000 7f8723fff640 1 -- 172.21.15.117:0/1361037056 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873c013790 con 0x5601c84b9390 2024-09-21T05:59:18.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.572+0000 7f8751277740 1 -- 172.21.15.117:0/1361037056 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f873c019070 con 0x5601c84b9390 2024-09-21T05:59:18.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.572+0000 7f8723fff640 1 -- 172.21.15.117:0/1361037056 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f873c013930 con 0x5601c84b9390 2024-09-21T05:59:18.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.573+0000 7f8723fff640 1 -- 172.21.15.117:0/1361037056 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f873c013b70 con 0x5601c84b9390 2024-09-21T05:59:18.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.573+0000 7f8751277740 10 client.15957 fetch_fsmap learned FSMap version 27 2024-09-21T05:59:18.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.573+0000 7f8751277740 1 -- 172.21.15.117:0/1361037056 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5601c889bb50 con 0x5601c84b9390 2024-09-21T05:59:18.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.573+0000 7f8723fff640 1 -- 172.21.15.117:0/1361037056 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f873c019070 con 0x5601c84b9390 2024-09-21T05:59:18.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.574+0000 7f8723fff640 1 -- 172.21.15.117:0/1361037056 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f873c01e400 con 0x5601c84b9390 2024-09-21T05:59:18.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.575+0000 7f8751277740 10 client.15957 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:59:18.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.575+0000 7f8751277740 10 client.15957 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T05:59:18.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.575+0000 7f8751277740 20 client.15957 populate_metadata read hostname 'smithi117' 2024-09-21T05:59:18.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.575+0000 7f8751277740 1 --2- 172.21.15.117:0/1361037056 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x5601c84989f0 0x5601c889f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:18.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.575+0000 7f8751277740 4 client.15957 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:59:18.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.575+0000 7f8751277740 1 -- 172.21.15.117:0/1361037056 --> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601c87f9df0 con 0x5601c84989f0 2024-09-21T05:59:18.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.579+0000 7f8748a9f640 1 --2- 172.21.15.117:0/1361037056 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x5601c84989f0 0x5601c889f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:18.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.587+0000 7f8748a9f640 1 --2- 172.21.15.117:0/1361037056 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x5601c84989f0 0x5601c889f2a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9856a4b15f54d79d server_cookie=fbf1ce021054a8cb in_seq=0 out_seq=0 2024-09-21T05:59:18.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.588+0000 7f8723fff640 10 client.15957 ms_handle_connect on v2:172.21.15.192:6838/1727623231 2024-09-21T05:59:18.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.591+0000 7f8723fff640 1 -- 172.21.15.117:0/1361037056 <== mds.1 v2:172.21.15.192:6838/1727623231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5601c87f9df0 con 0x5601c84989f0 2024-09-21T05:59:18.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.592+0000 7f8723fff640 10 client.15957 handle_command_reply: tid=0 2024-09-21T05:59:18.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.592+0000 7f8751277740 1 client.15957 shutdown 2024-09-21T05:59:18.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.594+0000 7f8751277740 1 -- 172.21.15.117:0/1361037056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c84b9390 msgr2=0x5601c888d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:18.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.594+0000 7f8751277740 1 --2- 172.21.15.117:0/1361037056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c84b9390 0x5601c888d1b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f873c0149d0 tx=0x7f873c014650 comp rx=0 tx=0).stop 2024-09-21T05:59:18.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.603+0000 7f8751277740 1 -- 172.21.15.117:0/1361037056 shutdown_connections 2024-09-21T05:59:18.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.603+0000 7f8751277740 1 --2- 172.21.15.117:0/1361037056 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x5601c84989f0 0x5601c889f2a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:18.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.603+0000 7f8751277740 1 --2- 172.21.15.117:0/1361037056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c84b9390 0x5601c888d1b0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:18.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.603+0000 7f8723fff640 0 client.15957 ms_handle_reset on v2:172.21.15.192:6838/1727623231 2024-09-21T05:59:18.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.603+0000 7f8751277740 1 --2- 172.21.15.117:0/1361037056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c84da070 0x5601c888d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:18.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.603+0000 7f8751277740 1 --2- 172.21.15.117:0/1361037056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5601c84b11b0 0x5601c888cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:18.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.603+0000 7f8751277740 1 -- 172.21.15.117:0/1361037056 >> 172.21.15.117:0/1361037056 conn(0x5601c84ce460 msgr2=0x5601c8884880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:18.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.603+0000 7f8751277740 1 -- 172.21.15.117:0/1361037056 shutdown_connections 2024-09-21T05:59:18.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.603+0000 7f8751277740 1 -- 172.21.15.117:0/1361037056 wait complete. 2024-09-21T05:59:18.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.607+0000 7f8751277740 20 client.15957 trim_cache size 0 max 16384 2024-09-21T05:59:18.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.651+0000 7f8751277740 1 Processor -- start 2024-09-21T05:59:18.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.651+0000 7f8751277740 1 -- start start 2024-09-21T05:59:18.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.651+0000 7f8751277740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c84989f0 0x5601c8883090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:18.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.651+0000 7f8751277740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5601c84b11b0 0x5601c888e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:18.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.651+0000 7f8751277740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c84b9390 0x5601c8889e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:18.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.651+0000 7f8751277740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f87441c7390 con 0x5601c84b9390 2024-09-21T05:59:18.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.651+0000 7f8751277740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f87441c8ed0 con 0x5601c84989f0 2024-09-21T05:59:18.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.651+0000 7f8751277740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f87441c7a60 con 0x5601c84b11b0 2024-09-21T05:59:18.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.652+0000 7f87492a0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c84b9390 0x5601c8889e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:18.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.652+0000 7f87492a0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c84b9390 0x5601c8889e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40464/0 (socket says 172.21.15.117:40464) 2024-09-21T05:59:18.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.652+0000 7f87492a0640 1 -- 172.21.15.117:0/4190693043 learned_addr learned my addr 172.21.15.117:0/4190693043 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:18.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.653+0000 7f87492a0640 1 -- 172.21.15.117:0/4190693043 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5601c84b11b0 msgr2=0x5601c888e8a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T05:59:18.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.653+0000 7f87492a0640 1 --2- 172.21.15.117:0/4190693043 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5601c84b11b0 0x5601c888e8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:18.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.653+0000 7f87492a0640 1 -- 172.21.15.117:0/4190693043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c84989f0 msgr2=0x5601c8883090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T05:59:18.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.653+0000 7f87492a0640 1 --2- 172.21.15.117:0/4190693043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c84989f0 0x5601c8883090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:18.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.653+0000 7f87492a0640 1 -- 172.21.15.117:0/4190693043 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601c8883990 con 0x5601c84b9390 2024-09-21T05:59:18.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.653+0000 7f87492a0640 1 --2- 172.21.15.117:0/4190693043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c84b9390 0x5601c8889e60 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f87340ad950 tx=0x7f87340ad1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ef17977a31bea8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:18.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.653+0000 7f872a7fc640 1 -- 172.21.15.117:0/4190693043 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87340055f0 con 0x5601c84b9390 2024-09-21T05:59:18.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.654+0000 7f872a7fc640 1 -- 172.21.15.117:0/4190693043 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87340ad4d0 con 0x5601c84b9390 2024-09-21T05:59:18.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.654+0000 7f872a7fc640 1 -- 172.21.15.117:0/4190693043 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87340b73e0 con 0x5601c84b9390 2024-09-21T05:59:18.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.655+0000 7f8751277740 1 -- 172.21.15.117:0/4190693043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c84b9390 msgr2=0x5601c8889e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:18.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.655+0000 7f8751277740 1 --2- 172.21.15.117:0/4190693043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c84b9390 0x5601c8889e60 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f87340ad950 tx=0x7f87340ad1f0 comp rx=0 tx=0).stop 2024-09-21T05:59:18.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.655+0000 7f8751277740 1 -- 172.21.15.117:0/4190693043 shutdown_connections 2024-09-21T05:59:18.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.655+0000 7f8751277740 1 --2- 172.21.15.117:0/4190693043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c84b9390 0x5601c8889e60 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:18.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.655+0000 7f8751277740 1 --2- 172.21.15.117:0/4190693043 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5601c84b11b0 0x5601c888e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:18.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.655+0000 7f8751277740 1 --2- 172.21.15.117:0/4190693043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c84989f0 0x5601c8883090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:18.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.655+0000 7f8751277740 1 -- 172.21.15.117:0/4190693043 >> 172.21.15.117:0/4190693043 conn(0x5601c84ce460 msgr2=0x5601c888e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:18.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.655+0000 7f8751277740 1 -- 172.21.15.117:0/4190693043 shutdown_connections 2024-09-21T05:59:18.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.655+0000 7f8751277740 1 -- 172.21.15.117:0/4190693043 wait complete. 2024-09-21T05:59:18.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.656+0000 7f8751277740 1 -- start start 2024-09-21T05:59:18.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.656+0000 7f8751277740 1 Processor -- start 2024-09-21T05:59:18.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.657+0000 7f8751277740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c84989f0 0x5601c889fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:18.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.657+0000 7f8751277740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c84b11b0 0x5601c88a0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:18.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.657+0000 7f8751277740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5601c84b9390 0x5601c8900ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:18.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.657+0000 7f8751277740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f87441c8800 con 0x5601c84b11b0 2024-09-21T05:59:18.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.657+0000 7f8751277740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f87441c8130 con 0x5601c84989f0 2024-09-21T05:59:18.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.657+0000 7f8751277740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f87441aa5a0 con 0x5601c84b9390 2024-09-21T05:59:18.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.658+0000 7f8751277740 10 client.0 _ll_register_callbacks cb 0x7f873c009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:59:18.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.658+0000 7f8743fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c84b11b0 0x5601c88a0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:18.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.658+0000 7f8743fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c84b11b0 0x5601c88a0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40478/0 (socket says 172.21.15.117:40478) 2024-09-21T05:59:18.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.658+0000 7f8743fff640 1 -- 172.21.15.117:0/3504985842 learned_addr learned my addr 172.21.15.117:0/3504985842 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:18.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.659+0000 7f87492a0640 1 --2- 172.21.15.117:0/3504985842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5601c84b9390 0x5601c8900ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:18.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.659+0000 7f8748a9f640 1 --2- 172.21.15.117:0/3504985842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c84989f0 0x5601c889fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:18.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.659+0000 7f8743fff640 1 -- 172.21.15.117:0/3504985842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5601c84b9390 msgr2=0x5601c8900ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:18.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.659+0000 7f8743fff640 1 --2- 172.21.15.117:0/3504985842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5601c84b9390 0x5601c8900ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:18.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.659+0000 7f8743fff640 1 -- 172.21.15.117:0/3504985842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c84989f0 msgr2=0x5601c889fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:18.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.659+0000 7f8743fff640 1 --2- 172.21.15.117:0/3504985842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c84989f0 0x5601c889fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:18.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.659+0000 7f8743fff640 1 -- 172.21.15.117:0/3504985842 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601c8901130 con 0x5601c84b11b0 2024-09-21T05:59:18.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.660+0000 7f8743fff640 1 --2- 172.21.15.117:0/3504985842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c84b11b0 0x5601c88a0370 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f873c0070b0 tx=0x7f873c013f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a55649807290b76e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:18.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.660+0000 7f8723fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:59:18.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.661+0000 7f8723fff640 1 -- 172.21.15.117:0/3504985842 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873c034070 con 0x5601c84b11b0 2024-09-21T05:59:18.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.661+0000 7f8751277740 1 -- 172.21.15.117:0/3504985842 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87340ad4d0 con 0x5601c84b11b0 2024-09-21T05:59:18.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.661+0000 7f8723fff640 1 -- 172.21.15.117:0/3504985842 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f873c013590 con 0x5601c84b11b0 2024-09-21T05:59:18.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.661+0000 7f8723fff640 1 -- 172.21.15.117:0/3504985842 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f873c0096d0 con 0x5601c84b11b0 2024-09-21T05:59:18.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.661+0000 7f8751277740 10 client.15963 fetch_fsmap learned FSMap version 27 2024-09-21T05:59:18.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.661+0000 7f8751277740 1 -- 172.21.15.117:0/3504985842 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5601c8909610 con 0x5601c84b11b0 2024-09-21T05:59:18.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.662+0000 7f8723fff640 1 -- 172.21.15.117:0/3504985842 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87340ad4d0 con 0x5601c84b11b0 2024-09-21T05:59:18.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.662+0000 7f8723fff640 1 -- 172.21.15.117:0/3504985842 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f873c037020 con 0x5601c84b11b0 2024-09-21T05:59:18.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.662+0000 7f8751277740 10 client.15963 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:59:18.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.662+0000 7f8751277740 10 client.15963 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T05:59:18.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.662+0000 7f8751277740 20 client.15963 populate_metadata read hostname 'smithi117' 2024-09-21T05:59:18.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.662+0000 7f8751277740 1 --2- 172.21.15.117:0/3504985842 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x5601c84da070 0x5601c890bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:18.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.662+0000 7f8751277740 4 client.15963 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T05:59:18.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.662+0000 7f8751277740 1 -- 172.21.15.117:0/3504985842 --> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5601c87f58a0 con 0x5601c84da070 2024-09-21T05:59:18.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.666+0000 7f8748a9f640 1 --2- 172.21.15.117:0/3504985842 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x5601c84da070 0x5601c890bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:18.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.675+0000 7f8723fff640 10 client.15963 ms_handle_connect on v2:172.21.15.192:6838/1727623231 2024-09-21T05:59:18.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.675+0000 7f8748a9f640 1 --2- 172.21.15.117:0/3504985842 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x5601c84da070 0x5601c890bab0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a899ce8671274083 server_cookie=4c7be7398324982c in_seq=0 out_seq=0 2024-09-21T05:59:18.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.678+0000 7f8723fff640 1 -- 172.21.15.117:0/3504985842 <== mds.1 v2:172.21.15.192:6838/1727623231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5601c87f58a0 con 0x5601c84da070 2024-09-21T05:59:18.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.678+0000 7f8723fff640 10 client.15963 handle_command_reply: tid=0 2024-09-21T05:59:18.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.678+0000 7f8751277740 1 client.15963 shutdown 2024-09-21T05:59:18.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.679+0000 7f8751277740 1 -- 172.21.15.117:0/3504985842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c84b11b0 msgr2=0x5601c88a0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:18.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.679+0000 7f8751277740 1 --2- 172.21.15.117:0/3504985842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c84b11b0 0x5601c88a0370 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f873c0070b0 tx=0x7f873c013f20 comp rx=0 tx=0).stop 2024-09-21T05:59:18.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.679+0000 7f8751277740 1 -- 172.21.15.117:0/3504985842 shutdown_connections 2024-09-21T05:59:18.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.679+0000 7f8751277740 1 --2- 172.21.15.117:0/3504985842 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x5601c84da070 0x5601c890bab0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:18.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.679+0000 7f8751277740 1 --2- 172.21.15.117:0/3504985842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5601c84b11b0 0x5601c88a0370 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:18.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.679+0000 7f8751277740 1 --2- 172.21.15.117:0/3504985842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5601c84b9390 0x5601c8900ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:18.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.679+0000 7f8751277740 1 --2- 172.21.15.117:0/3504985842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5601c84989f0 0x5601c889fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:18.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.679+0000 7f8751277740 1 -- 172.21.15.117:0/3504985842 >> 172.21.15.117:0/3504985842 conn(0x5601c84ce460 msgr2=0x5601c8898b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:18.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.679+0000 7f8723fff640 0 client.15963 ms_handle_reset on v2:172.21.15.192:6838/1727623231 2024-09-21T05:59:18.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.679+0000 7f8751277740 1 -- 172.21.15.117:0/3504985842 shutdown_connections 2024-09-21T05:59:18.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.679+0000 7f8751277740 1 -- 172.21.15.117:0/3504985842 wait complete. 2024-09-21T05:59:18.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.679+0000 7f8751277740 20 client.15963 trim_cache size 0 max 16384 2024-09-21T05:59:18.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.680+0000 7f874ad03640 1 -- 172.21.15.117:0/365368342 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f872c0b67b0 msgr2=0x7f872c0b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:18.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.680+0000 7f874ad03640 1 --2- 172.21.15.117:0/365368342 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f872c0b67b0 0x7f872c0b8cc0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8738009850 tx=0x7f8738000bc0 comp rx=0 tx=0).stop 2024-09-21T05:59:18.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.680+0000 7f874ad03640 1 -- 172.21.15.117:0/365368342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87441aec70 msgr2=0x7f87441af080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:18.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.680+0000 7f874ad03640 1 --2- 172.21.15.117:0/365368342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87441aec70 0x7f87441af080 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f873400e970 tx=0x7f873400ed70 comp rx=0 tx=0).stop 2024-09-21T05:59:18.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.680+0000 7f874ad03640 1 -- 172.21.15.117:0/365368342 shutdown_connections 2024-09-21T05:59:18.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.680+0000 7f874ad03640 1 --2- 172.21.15.117:0/365368342 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f872c0b67b0 0x7f872c0b8cc0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:18.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.680+0000 7f874ad03640 1 --2- 172.21.15.117:0/365368342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87441aec70 0x7f87441af080 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:18.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.680+0000 7f874ad03640 1 --2- 172.21.15.117:0/365368342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f874411cc30 0x7f87441a9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:18.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.680+0000 7f874ad03640 1 --2- 172.21.15.117:0/365368342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f874410a9e0 0x7f87441a94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:18.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.680+0000 7f874ad03640 1 -- 172.21.15.117:0/365368342 >> 172.21.15.117:0/365368342 conn(0x7f8744070ec0 msgr2=0x7f874411e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:18.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.681+0000 7f874ad03640 1 -- 172.21.15.117:0/365368342 shutdown_connections 2024-09-21T05:59:18.077 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:17.681+0000 7f874ad03640 1 -- 172.21.15.117:0/365368342 wait complete. 2024-09-21T05:59:18.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:17 smithi138 ceph-mon[34391]: pgmap v386: 97 pgs: 97 active+clean; 14 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 819 MiB/s wr, 4.09k op/s 2024-09-21T05:59:19.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:19 smithi192 ceph-mon[34700]: pgmap v387: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 847 MiB/s wr, 4.40k op/s 2024-09-21T05:59:20.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:19 smithi117 ceph-mon[30947]: pgmap v387: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 847 MiB/s wr, 4.40k op/s 2024-09-21T05:59:20.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:19 smithi138 ceph-mon[34391]: pgmap v387: 97 pgs: 97 active+clean; 16 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 847 MiB/s wr, 4.40k op/s 2024-09-21T05:59:22.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:21 smithi117 ceph-mon[30947]: pgmap v388: 97 pgs: 97 active+clean; 16 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 693 MiB/s wr, 3.49k op/s 2024-09-21T05:59:22.036 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T05:59:22.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:21 smithi138 ceph-mon[34391]: pgmap v388: 97 pgs: 97 active+clean; 16 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 693 MiB/s wr, 3.49k op/s 2024-09-21T05:59:22.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:21 smithi192 ceph-mon[34700]: pgmap v388: 97 pgs: 97 active+clean; 16 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 693 MiB/s wr, 3.49k op/s 2024-09-21T05:59:22.772 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:59:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.792+0000 7f7cb9e07640 1 -- 172.21.15.117:0/3621042798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cb4108310 msgr2=0x7f7cb4108720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.792+0000 7f7cb9e07640 1 --2- 172.21.15.117:0/3621042798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cb4108310 0x7f7cb4108720 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f7cac009cc0 tx=0x7f7cac02cb20 comp rx=0 tx=0).stop 2024-09-21T05:59:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.794+0000 7f7cb9e07640 1 -- 172.21.15.117:0/3621042798 shutdown_connections 2024-09-21T05:59:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.794+0000 7f7cb9e07640 1 --2- 172.21.15.117:0/3621042798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cb4108310 0x7f7cb4108720 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.794+0000 7f7cb9e07640 1 --2- 172.21.15.117:0/3621042798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cb4108c60 0x7f7cb410f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.794+0000 7f7cb9e07640 1 --2- 172.21.15.117:0/3621042798 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cb41029e0 0x7f7cb4102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.794+0000 7f7cb9e07640 1 -- 172.21.15.117:0/3621042798 >> 172.21.15.117:0/3621042798 conn(0x7f7cb407a6e0 msgr2=0x7f7cb407aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.810+0000 7f7cb9e07640 1 -- 172.21.15.117:0/3621042798 shutdown_connections 2024-09-21T05:59:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.810+0000 7f7cb9e07640 1 -- 172.21.15.117:0/3621042798 wait complete. 2024-09-21T05:59:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.812+0000 7f7cb9e07640 1 Processor -- start 2024-09-21T05:59:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.812+0000 7f7cb9e07640 1 -- start start 2024-09-21T05:59:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.813+0000 7f7cb9e07640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cb41029e0 0x7f7cb41980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.814+0000 7f7cb9e07640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cb4108310 0x7f7cb4198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.814+0000 7f7cb9e07640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cb4108c60 0x7f7cb419d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.814+0000 7f7cb9e07640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb41117c0 con 0x7f7cb4108c60 2024-09-21T05:59:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.814+0000 7f7cb9e07640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb4111640 con 0x7f7cb41029e0 2024-09-21T05:59:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.814+0000 7f7cb9e07640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb4111940 con 0x7f7cb4108310 2024-09-21T05:59:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.816+0000 7f7cb2ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cb4108310 0x7f7cb4198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.816+0000 7f7cb37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cb41029e0 0x7f7cb41980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.816+0000 7f7cb2ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cb4108310 0x7f7cb4198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:50504/0 (socket says 172.21.15.117:50504) 2024-09-21T05:59:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.816+0000 7f7cb2ffd640 1 -- 172.21.15.117:0/4241515285 learned_addr learned my addr 172.21.15.117:0/4241515285 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.821+0000 7f7cb3fff640 1 --2- 172.21.15.117:0/4241515285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cb4108c60 0x7f7cb419d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.823+0000 7f7cb3fff640 1 -- 172.21.15.117:0/4241515285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cb4108310 msgr2=0x7f7cb4198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.823+0000 7f7cb3fff640 1 --2- 172.21.15.117:0/4241515285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cb4108310 0x7f7cb4198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.823+0000 7f7cb3fff640 1 -- 172.21.15.117:0/4241515285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cb41029e0 msgr2=0x7f7cb41980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.823+0000 7f7cb3fff640 1 --2- 172.21.15.117:0/4241515285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cb41029e0 0x7f7cb41980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.823+0000 7f7cb3fff640 1 -- 172.21.15.117:0/4241515285 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb419e010 con 0x7f7cb4108c60 2024-09-21T05:59:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.834+0000 7f7cb3fff640 1 --2- 172.21.15.117:0/4241515285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cb4108c60 0x7f7cb419d910 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f7ca400d680 tx=0x7f7ca400db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.856+0000 7f7cb0ff9640 1 -- 172.21.15.117:0/4241515285 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4017070 con 0x7f7cb4108c60 2024-09-21T05:59:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.857+0000 7f7cb0ff9640 1 -- 172.21.15.117:0/4241515285 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ca4009820 con 0x7f7cb4108c60 2024-09-21T05:59:22.860 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.857+0000 7f7cb0ff9640 1 -- 172.21.15.117:0/4241515285 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4004ce0 con 0x7f7cb4108c60 2024-09-21T05:59:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.868+0000 7f7cb9e07640 1 -- 172.21.15.117:0/4241515285 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cb419e300 con 0x7f7cb4108c60 2024-09-21T05:59:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.872+0000 7f7cb9e07640 1 -- 172.21.15.117:0/4241515285 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cb419e710 con 0x7f7cb4108c60 2024-09-21T05:59:22.884 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.881+0000 7f7cc0e48740 1 Processor -- start 2024-09-21T05:59:22.884 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.881+0000 7f7cc0e48740 1 -- start start 2024-09-21T05:59:22.886 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.882+0000 7f7cc0e48740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e29561b0 0x5633e2d06e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.882+0000 7f7cc0e48740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633e292d2f0 0x5633e2d043b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.882+0000 7f7cc0e48740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633e29354d0 0x5633e2d04aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.882+0000 7f7cc0e48740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5633e2c706f0 con 0x5633e29354d0 2024-09-21T05:59:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.882+0000 7f7cc0e48740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5633e2c5fb20 con 0x5633e29561b0 2024-09-21T05:59:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.882+0000 7f7cc0e48740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5633e2c63930 con 0x5633e292d2f0 2024-09-21T05:59:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.883+0000 7f7cb2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633e29354d0 0x5633e2d04aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.883+0000 7f7cb2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633e29354d0 0x5633e2d04aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40502/0 (socket says 172.21.15.117:40502) 2024-09-21T05:59:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.883+0000 7f7cb2ffd640 1 -- 172.21.15.117:0/395003523 learned_addr learned my addr 172.21.15.117:0/395003523 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.884+0000 7f7cb2ffd640 1 --2- 172.21.15.117:0/395003523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e29561b0 0x5633e2d06e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.885+0000 7f7cb37fe640 1 --2- 172.21.15.117:0/395003523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633e292d2f0 0x5633e2d043b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.885+0000 7f7cb2ffd640 1 -- 172.21.15.117:0/395003523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633e292d2f0 msgr2=0x5633e2d043b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.885+0000 7f7cb2ffd640 1 --2- 172.21.15.117:0/395003523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633e292d2f0 0x5633e2d043b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.885+0000 7f7cb2ffd640 1 -- 172.21.15.117:0/395003523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e29561b0 msgr2=0x5633e2d06e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.885+0000 7f7cb2ffd640 1 --2- 172.21.15.117:0/395003523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e29561b0 0x5633e2d06e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.885+0000 7f7cb2ffd640 1 -- 172.21.15.117:0/395003523 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633e2c25530 con 0x5633e29354d0 2024-09-21T05:59:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.889+0000 7f7cb0ff9640 1 -- 172.21.15.117:0/4241515285 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f7ca40099d0 con 0x7f7cb4108c60 2024-09-21T05:59:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.893+0000 7f7cb2ffd640 1 --2- 172.21.15.117:0/395003523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633e29354d0 0x5633e2d04aa0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7cac02e860 tx=0x7f7cac02fe90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5459de5062a9c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.895+0000 7f7cb0ff9640 1 --2- 172.21.15.117:0/4241515285 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7c9c0b6850 0x7f7c9c0b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.896+0000 7f7cb0ff9640 1 -- 172.21.15.117:0/4241515285 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f7ca40e7220 con 0x7f7cb4108c60 2024-09-21T05:59:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.897+0000 7f7c997fa640 1 -- 172.21.15.117:0/395003523 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cac02ebc0 con 0x5633e29354d0 2024-09-21T05:59:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.897+0000 7f7c997fa640 1 -- 172.21.15.117:0/395003523 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cac03d030 con 0x5633e29354d0 2024-09-21T05:59:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.897+0000 7f7c997fa640 1 -- 172.21.15.117:0/395003523 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cac038660 con 0x5633e29354d0 2024-09-21T05:59:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.898+0000 7f7cb37fe640 1 --2- 172.21.15.117:0/4241515285 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7c9c0b6850 0x7f7c9c0b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.899+0000 7f7cc0e48740 1 -- 172.21.15.117:0/395003523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633e29354d0 msgr2=0x5633e2d04aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.899+0000 7f7cc0e48740 1 --2- 172.21.15.117:0/395003523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633e29354d0 0x5633e2d04aa0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7cac02e860 tx=0x7f7cac02fe90 comp rx=0 tx=0).stop 2024-09-21T05:59:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.902+0000 7f7cc0e48740 1 -- 172.21.15.117:0/395003523 shutdown_connections 2024-09-21T05:59:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.902+0000 7f7cc0e48740 1 --2- 172.21.15.117:0/395003523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633e29354d0 0x5633e2d04aa0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.902+0000 7f7cc0e48740 1 --2- 172.21.15.117:0/395003523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633e292d2f0 0x5633e2d043b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.902+0000 7f7cc0e48740 1 --2- 172.21.15.117:0/395003523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e29561b0 0x5633e2d06e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.902+0000 7f7cc0e48740 1 -- 172.21.15.117:0/395003523 >> 172.21.15.117:0/395003523 conn(0x5633e294a300 msgr2=0x5633e2d022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.903+0000 7f7cc0e48740 1 -- 172.21.15.117:0/395003523 shutdown_connections 2024-09-21T05:59:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.905+0000 7f7cc0e48740 1 -- 172.21.15.117:0/395003523 wait complete. 2024-09-21T05:59:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.906+0000 7f7cc0e48740 1 -- start start 2024-09-21T05:59:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.906+0000 7f7cc0e48740 1 Processor -- start 2024-09-21T05:59:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.907+0000 7f7cc0e48740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633e292d2f0 0x5633e2d0fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.907+0000 7f7cc0e48740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e29354d0 0x5633e2d10070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.907+0000 7f7cc0e48740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633e29561b0 0x5633e2d1bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.907+0000 7f7cc0e48740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5633e2b97ab0 con 0x5633e292d2f0 2024-09-21T05:59:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.907+0000 7f7cc0e48740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5633e2c05730 con 0x5633e29354d0 2024-09-21T05:59:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.907+0000 7f7cc0e48740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5633e2d12910 con 0x5633e29561b0 2024-09-21T05:59:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.907+0000 7f7cc0e48740 10 client.0 _ll_register_callbacks cb 0x5633e2c596b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:59:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.909+0000 7f7cb2ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e29354d0 0x5633e2d10070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.909+0000 7f7cb2ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e29354d0 0x5633e2d10070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:48916/0 (socket says 172.21.15.117:48916) 2024-09-21T05:59:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.909+0000 7f7cb2ffd640 1 -- 172.21.15.117:0/2298274674 learned_addr learned my addr 172.21.15.117:0/2298274674 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.909+0000 7f7cb37fe640 1 --2- 172.21.15.117:0/2298274674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633e292d2f0 0x5633e2d0fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.910+0000 7f7cb37fe640 1 --2- 172.21.15.117:0/4241515285 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7c9c0b6850 0x7f7c9c0b8d60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8005fd0 tx=0x7f7ca8005f20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.912+0000 7f7cb37fe640 1 -- 172.21.15.117:0/2298274674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633e29561b0 msgr2=0x5633e2d1bbc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T05:59:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.912+0000 7f7cb37fe640 1 --2- 172.21.15.117:0/2298274674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633e29561b0 0x5633e2d1bbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.912+0000 7f7cb37fe640 1 -- 172.21.15.117:0/2298274674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e29354d0 msgr2=0x5633e2d10070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.912+0000 7f7cb37fe640 1 --2- 172.21.15.117:0/2298274674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e29354d0 0x5633e2d10070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.912+0000 7f7cb37fe640 1 -- 172.21.15.117:0/2298274674 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633e29c1790 con 0x5633e292d2f0 2024-09-21T05:59:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.916+0000 7f7cb37fe640 1 --2- 172.21.15.117:0/2298274674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633e292d2f0 0x5633e2d0fb30 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f7ca80058c0 tx=0x7f7ca80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c85f73aac6681a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:22.922 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.918+0000 7f7c92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:59:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.919+0000 7f7c92ffd640 1 -- 172.21.15.117:0/2298274674 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8004020 con 0x5633e292d2f0 2024-09-21T05:59:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.919+0000 7f7cc0e48740 1 -- 172.21.15.117:0/2298274674 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cac03d030 con 0x5633e292d2f0 2024-09-21T05:59:22.923 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.919+0000 7f7c92ffd640 1 -- 172.21.15.117:0/2298274674 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f7ca80041c0 con 0x5633e292d2f0 2024-09-21T05:59:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.922+0000 7f7c92ffd640 1 -- 172.21.15.117:0/2298274674 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7ca802d050 con 0x5633e292d2f0 2024-09-21T05:59:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.922+0000 7f7cc0e48740 10 client.15975 fetch_fsmap learned FSMap version 27 2024-09-21T05:59:22.925 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.922+0000 7f7cc0e48740 1 -- 172.21.15.117:0/2298274674 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5633e2a3a620 con 0x5633e292d2f0 2024-09-21T05:59:22.926 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.923+0000 7f7c92ffd640 1 -- 172.21.15.117:0/2298274674 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cac03d030 con 0x5633e292d2f0 2024-09-21T05:59:22.926 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.923+0000 7f7c92ffd640 1 -- 172.21.15.117:0/2298274674 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7ca801e4f0 con 0x5633e292d2f0 2024-09-21T05:59:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.925+0000 7f7cc0e48740 10 client.15975 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:59:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.925+0000 7f7cc0e48740 10 client.15975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T05:59:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.925+0000 7f7cc0e48740 20 client.15975 populate_metadata read hostname 'smithi117' 2024-09-21T05:59:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.925+0000 7f7cc0e48740 1 --2- 172.21.15.117:0/2298274674 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5633e2914d80 0x5633e2cff060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.925+0000 7f7cc0e48740 4 client.15975 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:59:22.929 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.925+0000 7f7cc0e48740 1 -- 172.21.15.117:0/2298274674 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633e2c72640 con 0x5633e2914d80 2024-09-21T05:59:22.930 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.927+0000 7f7cb2ffd640 1 --2- 172.21.15.117:0/2298274674 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5633e2914d80 0x5633e2cff060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.940+0000 7f7cb2ffd640 1 --2- 172.21.15.117:0/2298274674 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5633e2914d80 0x5633e2cff060 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6c111a77fda8928 server_cookie=6ec0983f1195f90f in_seq=0 out_seq=0 2024-09-21T05:59:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.941+0000 7f7c92ffd640 10 client.15975 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:59:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.958+0000 7f7c92ffd640 1 -- 172.21.15.117:0/2298274674 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5633e2c72640 con 0x5633e2914d80 2024-09-21T05:59:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.958+0000 7f7c92ffd640 10 client.15975 handle_command_reply: tid=0 2024-09-21T05:59:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.958+0000 7f7cc0e48740 1 client.15975 shutdown 2024-09-21T05:59:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.958+0000 7f7cc0e48740 1 -- 172.21.15.117:0/2298274674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633e292d2f0 msgr2=0x5633e2d0fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.958+0000 7f7cc0e48740 1 --2- 172.21.15.117:0/2298274674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633e292d2f0 0x5633e2d0fb30 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f7ca80058c0 tx=0x7f7ca80046c0 comp rx=0 tx=0).stop 2024-09-21T05:59:22.962 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.958+0000 7f7cc0e48740 1 -- 172.21.15.117:0/2298274674 shutdown_connections 2024-09-21T05:59:22.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.958+0000 7f7cc0e48740 1 --2- 172.21.15.117:0/2298274674 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5633e2914d80 0x5633e2cff060 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:22.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.958+0000 7f7cc0e48740 1 --2- 172.21.15.117:0/2298274674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633e29561b0 0x5633e2d1bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:22.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.958+0000 7f7cc0e48740 1 --2- 172.21.15.117:0/2298274674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e29354d0 0x5633e2d10070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:22.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.958+0000 7f7cc0e48740 1 --2- 172.21.15.117:0/2298274674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633e292d2f0 0x5633e2d0fb30 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:22.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.958+0000 7f7cc0e48740 1 -- 172.21.15.117:0/2298274674 >> 172.21.15.117:0/2298274674 conn(0x5633e294a300 msgr2=0x5633e2d053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.960+0000 7f7c92ffd640 0 client.15975 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:59:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.960+0000 7f7cc0e48740 1 -- 172.21.15.117:0/2298274674 shutdown_connections 2024-09-21T05:59:22.967 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.961+0000 7f7cc0e48740 1 -- 172.21.15.117:0/2298274674 wait complete. 2024-09-21T05:59:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:22.969+0000 7f7cc0e48740 20 client.15975 trim_cache size 0 max 16384 2024-09-21T05:59:23.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:22 smithi117 ceph-mon[30947]: pgmap v389: 97 pgs: 97 active+clean; 17 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 789 MiB/s wr, 3.43k op/s 2024-09-21T05:59:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.029+0000 7f7cc0e48740 1 Processor -- start 2024-09-21T05:59:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.030+0000 7f7cc0e48740 1 -- start start 2024-09-21T05:59:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.030+0000 7f7cc0e48740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633e292d2f0 0x5633e2d1bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.030+0000 7f7cc0e48740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e29354d0 0x5633e2d10010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.030+0000 7f7cc0e48740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633e29561b0 0x5633e2d053e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.030+0000 7f7cc0e48740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb41ad360 con 0x5633e292d2f0 2024-09-21T05:59:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.030+0000 7f7cc0e48740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb41aeea0 con 0x5633e29354d0 2024-09-21T05:59:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.030+0000 7f7cc0e48740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb41ada30 con 0x5633e29561b0 2024-09-21T05:59:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.034+0000 7f7cb3fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633e29561b0 0x5633e2d053e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:23.038 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.035+0000 7f7cb3fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633e29561b0 0x5633e2d053e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:50528/0 (socket says 172.21.15.117:50528) 2024-09-21T05:59:23.039 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.035+0000 7f7cb3fff640 1 -- 172.21.15.117:0/587635262 learned_addr learned my addr 172.21.15.117:0/587635262 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:23.039 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.036+0000 7f7cb2ffd640 1 --2- 172.21.15.117:0/587635262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e29354d0 0x5633e2d10010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:23.040 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.037+0000 7f7cb3fff640 1 -- 172.21.15.117:0/587635262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e29354d0 msgr2=0x5633e2d10010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:23.040 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.037+0000 7f7cb3fff640 1 --2- 172.21.15.117:0/587635262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e29354d0 0x5633e2d10010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:23.040 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.037+0000 7f7cb3fff640 1 -- 172.21.15.117:0/587635262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633e292d2f0 msgr2=0x5633e2d1bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:23.040 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.037+0000 7f7cb3fff640 1 --2- 172.21.15.117:0/587635262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633e292d2f0 0x5633e2d1bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:23.040 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.037+0000 7f7cb3fff640 1 -- 172.21.15.117:0/587635262 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633e2cf5150 con 0x5633e29561b0 2024-09-21T05:59:23.041 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.038+0000 7f7cb3fff640 1 --2- 172.21.15.117:0/587635262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633e29561b0 0x5633e2d053e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7ca40a96e0 tx=0x7f7ca40a90e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb34e33a2d947239 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:23.042 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.039+0000 7f7c98ff9640 1 -- 172.21.15.117:0/587635262 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca40ad810 con 0x5633e29561b0 2024-09-21T05:59:23.042 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.039+0000 7f7c98ff9640 1 -- 172.21.15.117:0/587635262 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ca40b1bf0 con 0x5633e29561b0 2024-09-21T05:59:23.042 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.039+0000 7f7c98ff9640 1 -- 172.21.15.117:0/587635262 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4098420 con 0x5633e29561b0 2024-09-21T05:59:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.041+0000 7f7cc0e48740 1 -- 172.21.15.117:0/587635262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633e29561b0 msgr2=0x5633e2d053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.041+0000 7f7cc0e48740 1 --2- 172.21.15.117:0/587635262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633e29561b0 0x5633e2d053e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7ca40a96e0 tx=0x7f7ca40a90e0 comp rx=0 tx=0).stop 2024-09-21T05:59:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.041+0000 7f7cc0e48740 1 -- 172.21.15.117:0/587635262 shutdown_connections 2024-09-21T05:59:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.041+0000 7f7cc0e48740 1 --2- 172.21.15.117:0/587635262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633e29561b0 0x5633e2d053e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.041+0000 7f7cc0e48740 1 --2- 172.21.15.117:0/587635262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e29354d0 0x5633e2d10010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.042+0000 7f7cc0e48740 1 --2- 172.21.15.117:0/587635262 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633e292d2f0 0x5633e2d1bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.042+0000 7f7cc0e48740 1 -- 172.21.15.117:0/587635262 >> 172.21.15.117:0/587635262 conn(0x5633e294a300 msgr2=0x5633e2d0e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.045+0000 7f7cc0e48740 1 -- 172.21.15.117:0/587635262 shutdown_connections 2024-09-21T05:59:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.045+0000 7f7cc0e48740 1 -- 172.21.15.117:0/587635262 wait complete. 2024-09-21T05:59:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.046+0000 7f7cc0e48740 1 -- start start 2024-09-21T05:59:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.047+0000 7f7cc0e48740 1 Processor -- start 2024-09-21T05:59:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.048+0000 7f7cc0e48740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e292d2f0 0x5633e2d17b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.048+0000 7f7cc0e48740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633e29354d0 0x5633e2d18120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.049+0000 7f7cc0e48740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633e29561b0 0x5633e2d31760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.049+0000 7f7cc0e48740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb41ae7d0 con 0x5633e29561b0 2024-09-21T05:59:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.049+0000 7f7cc0e48740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb41ae100 con 0x5633e292d2f0 2024-09-21T05:59:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.049+0000 7f7cb37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e292d2f0 0x5633e2d17b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.049+0000 7f7cb37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e292d2f0 0x5633e2d17b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:48932/0 (socket says 172.21.15.117:48932) 2024-09-21T05:59:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.049+0000 7f7cb37fe640 1 -- 172.21.15.117:0/3895629599 learned_addr learned my addr 172.21.15.117:0/3895629599 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.049+0000 7f7cc0e48740 1 -- 172.21.15.117:0/3895629599 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb41ab340 con 0x5633e29354d0 2024-09-21T05:59:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.050+0000 7f7cc0e48740 10 client.0 _ll_register_callbacks cb 0x7f7ca8010ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:59:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.050+0000 7f7cb2ffd640 1 --2- 172.21.15.117:0/3895629599 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633e29354d0 0x5633e2d18120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.050+0000 7f7cb3fff640 1 --2- 172.21.15.117:0/3895629599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633e29561b0 0x5633e2d31760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.050+0000 7f7cb37fe640 1 -- 172.21.15.117:0/3895629599 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633e29354d0 msgr2=0x5633e2d18120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.051+0000 7f7cb37fe640 1 --2- 172.21.15.117:0/3895629599 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633e29354d0 0x5633e2d18120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:23.055 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.051+0000 7f7cb37fe640 1 -- 172.21.15.117:0/3895629599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633e29561b0 msgr2=0x5633e2d31760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:23.055 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.051+0000 7f7cb37fe640 1 --2- 172.21.15.117:0/3895629599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633e29561b0 0x5633e2d31760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:23.055 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.051+0000 7f7cb37fe640 1 -- 172.21.15.117:0/3895629599 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633e2d04720 con 0x5633e292d2f0 2024-09-21T05:59:23.055 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.051+0000 7f7cb37fe640 1 --2- 172.21.15.117:0/3895629599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e292d2f0 0x5633e2d17b60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7ca801ea60 tx=0x7f7ca8020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10b80048fd55c2e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:23.055 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.053+0000 7f7c927fc640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-21T05:59:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.053+0000 7f7c927fc640 1 -- 172.21.15.117:0/3895629599 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca801e620 con 0x5633e292d2f0 2024-09-21T05:59:23.057 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.053+0000 7f7c927fc640 1 -- 172.21.15.117:0/3895629599 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f7ca8005040 con 0x5633e292d2f0 2024-09-21T05:59:23.058 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.054+0000 7f7cc0e48740 1 -- 172.21.15.117:0/3895629599 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ca40b1bf0 con 0x5633e292d2f0 2024-09-21T05:59:23.059 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.055+0000 7f7c927fc640 1 -- 172.21.15.117:0/3895629599 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7ca8005240 con 0x5633e292d2f0 2024-09-21T05:59:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.055+0000 7f7cc0e48740 10 client.25403 fetch_fsmap learned FSMap version 27 2024-09-21T05:59:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.055+0000 7f7cc0e48740 1 -- 172.21.15.117:0/3895629599 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5633e2d94940 con 0x5633e292d2f0 2024-09-21T05:59:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.058+0000 7f7c927fc640 1 -- 172.21.15.117:0/3895629599 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ca8035030 con 0x5633e292d2f0 2024-09-21T05:59:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.058+0000 7f7c927fc640 1 -- 172.21.15.117:0/3895629599 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7ca8026430 con 0x5633e292d2f0 2024-09-21T05:59:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.058+0000 7f7cc0e48740 10 client.25403 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:59:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.058+0000 7f7cc0e48740 10 client.25403 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T05:59:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.058+0000 7f7cc0e48740 20 client.25403 populate_metadata read hostname 'smithi117' 2024-09-21T05:59:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.058+0000 7f7cc0e48740 1 --2- 172.21.15.117:0/3895629599 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5633e28ea1c0 0x5633e2d970e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.058+0000 7f7cc0e48740 4 client.25403 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T05:59:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.058+0000 7f7cc0e48740 1 -- 172.21.15.117:0/3895629599 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f7ca40b1bf0 con 0x5633e28ea1c0 2024-09-21T05:59:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.061+0000 7f7cb2ffd640 1 --2- 172.21.15.117:0/3895629599 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5633e28ea1c0 0x5633e2d970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.070+0000 7f7cb2ffd640 1 --2- 172.21.15.117:0/3895629599 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5633e28ea1c0 0x5633e2d970e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9cc29f06e7da0ea server_cookie=420ef09d3d732ebd in_seq=0 out_seq=0 2024-09-21T05:59:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:23.070+0000 7f7c927fc640 10 client.25403 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:59:23.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:22 smithi138 ceph-mon[34391]: pgmap v389: 97 pgs: 97 active+clean; 17 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 789 MiB/s wr, 3.43k op/s 2024-09-21T05:59:23.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:22 smithi192 ceph-mon[34700]: pgmap v389: 97 pgs: 97 active+clean; 17 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 789 MiB/s wr, 3.43k op/s 2024-09-21T05:59:24.554 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:24 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:59:24.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:24 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:59:24.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:24 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T05:59:24.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.550+0000 7f3f22022640 1 -- 172.21.15.117:0/784942073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f1c10ab80 msgr2=0x7f3f1c10af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:24.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.550+0000 7f3f22022640 1 --2- 172.21.15.117:0/784942073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f1c10ab80 0x7f3f1c10af90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3f10009860 tx=0x7f3f1002a360 comp rx=0 tx=0).stop 2024-09-21T05:59:24.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.553+0000 7f3f22022640 1 -- 172.21.15.117:0/784942073 shutdown_connections 2024-09-21T05:59:24.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.553+0000 7f3f22022640 1 --2- 172.21.15.117:0/784942073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f1c11cba0 0x7f3f1c11efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.553+0000 7f3f22022640 1 --2- 172.21.15.117:0/784942073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f1c10ab80 0x7f3f1c10af90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.553+0000 7f3f22022640 1 --2- 172.21.15.117:0/784942073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f1c111d40 0x7f3f1c10a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.553+0000 7f3f22022640 1 -- 172.21.15.117:0/784942073 >> 172.21.15.117:0/784942073 conn(0x7f3f1c06d950 msgr2=0x7f3f1c06dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:24.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.554+0000 7f3f22022640 1 -- 172.21.15.117:0/784942073 shutdown_connections 2024-09-21T05:59:24.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.555+0000 7f3f22022640 1 -- 172.21.15.117:0/784942073 wait complete. 2024-09-21T05:59:24.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.556+0000 7f3f22022640 1 Processor -- start 2024-09-21T05:59:24.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.557+0000 7f3f22022640 1 -- start start 2024-09-21T05:59:24.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.558+0000 7f3f22022640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f1c10ab80 0x7f3f1c1a9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:24.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.559+0000 7f3f22022640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f1c111d40 0x7f3f1c1a96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:24.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.559+0000 7f3f22022640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f1c11cba0 0x7f3f1c1ae980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:24.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.559+0000 7f3f22022640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1c122710 con 0x7f3f1c10ab80 2024-09-21T05:59:24.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.559+0000 7f3f22022640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1c122590 con 0x7f3f1c111d40 2024-09-21T05:59:24.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.559+0000 7f3f22022640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1c122890 con 0x7f3f1c11cba0 2024-09-21T05:59:24.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.561+0000 7f3f1bfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f1c11cba0 0x7f3f1c1ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:24.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.561+0000 7f3f1bfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f1c11cba0 0x7f3f1c1ae980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:51748/0 (socket says 172.21.15.117:51748) 2024-09-21T05:59:24.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.561+0000 7f3f1bfff640 1 -- 172.21.15.117:0/2371771116 learned_addr learned my addr 172.21.15.117:0/2371771116 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:24.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.563+0000 7f3f1b7fe640 1 --2- 172.21.15.117:0/2371771116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f1c10ab80 0x7f3f1c1a9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:24.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.563+0000 7f3f1affd640 1 --2- 172.21.15.117:0/2371771116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f1c111d40 0x7f3f1c1a96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:24.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.564+0000 7f3f1bfff640 1 -- 172.21.15.117:0/2371771116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f1c111d40 msgr2=0x7f3f1c1a96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:24.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.564+0000 7f3f1bfff640 1 --2- 172.21.15.117:0/2371771116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f1c111d40 0x7f3f1c1a96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.564+0000 7f3f1bfff640 1 -- 172.21.15.117:0/2371771116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f1c10ab80 msgr2=0x7f3f1c1a9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:24.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.564+0000 7f3f1bfff640 1 --2- 172.21.15.117:0/2371771116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f1c10ab80 0x7f3f1c1a9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.564+0000 7f3f1bfff640 1 -- 172.21.15.117:0/2371771116 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f1c1af0b0 con 0x7f3f1c11cba0 2024-09-21T05:59:24.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.570+0000 7f3f1bfff640 1 --2- 172.21.15.117:0/2371771116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f1c11cba0 0x7f3f1c1ae980 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3f1400eb10 tx=0x7f3f1400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:24.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.571+0000 7f3f18ff9640 1 -- 172.21.15.117:0/2371771116 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f14004b20 con 0x7f3f1c11cba0 2024-09-21T05:59:24.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.571+0000 7f3f18ff9640 1 -- 172.21.15.117:0/2371771116 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f14004cc0 con 0x7f3f1c11cba0 2024-09-21T05:59:24.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.571+0000 7f3f18ff9640 1 -- 172.21.15.117:0/2371771116 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f14010670 con 0x7f3f1c11cba0 2024-09-21T05:59:24.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.571+0000 7f3f22022640 1 -- 172.21.15.117:0/2371771116 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f1c1af3a0 con 0x7f3f1c11cba0 2024-09-21T05:59:24.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.571+0000 7f3f22022640 1 -- 172.21.15.117:0/2371771116 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f1c1b5b00 con 0x7f3f1c11cba0 2024-09-21T05:59:24.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.582+0000 7f3f28596740 1 Processor -- start 2024-09-21T05:59:24.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.584+0000 7f3f28596740 1 -- start start 2024-09-21T05:59:24.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.585+0000 7f3f18ff9640 1 -- 172.21.15.117:0/2371771116 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f3f14005020 con 0x7f3f1c11cba0 2024-09-21T05:59:24.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.585+0000 7f3f28596740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a203adc070 0x55a203e83860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:24.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.585+0000 7f3f28596740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a203ab31b0 0x55a203e83da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:24.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.585+0000 7f3f28596740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a203abb390 0x55a203e842e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:24.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.585+0000 7f3f28596740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a203df9ea0 con 0x55a203abb390 2024-09-21T05:59:24.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.590+0000 7f3f28596740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a203de92d0 con 0x55a203ab31b0 2024-09-21T05:59:24.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.590+0000 7f3f28596740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a203ded0e0 con 0x55a203adc070 2024-09-21T05:59:24.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.588+0000 7f3f1b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a203ab31b0 0x55a203e83da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:24.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.590+0000 7f3f1b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a203ab31b0 0x55a203e83da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:42304/0 (socket says 172.21.15.117:42304) 2024-09-21T05:59:24.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.590+0000 7f3f1b7fe640 1 -- 172.21.15.117:0/3777282232 learned_addr learned my addr 172.21.15.117:0/3777282232 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:24.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.588+0000 7f3f1affd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a203adc070 0x55a203e83860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:24.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.597+0000 7f3f1affd640 1 --2- 172.21.15.117:0/3777282232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a203abb390 0x55a203e842e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:24.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.598+0000 7f3f18ff9640 1 --2- 172.21.15.117:0/2371771116 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3f040b6880 0x7f3f040b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:24.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.598+0000 7f3f18ff9640 1 -- 172.21.15.117:0/2371771116 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f3f140e7bb0 con 0x7f3f1c11cba0 2024-09-21T05:59:24.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.599+0000 7f3f1affd640 1 -- 172.21.15.117:0/3777282232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a203ab31b0 msgr2=0x55a203e83da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:24.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.599+0000 7f3f1affd640 1 --2- 172.21.15.117:0/3777282232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a203ab31b0 0x55a203e83da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.599+0000 7f3f1affd640 1 -- 172.21.15.117:0/3777282232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a203abb390 msgr2=0x55a203e842e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:24.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.599+0000 7f3f1affd640 1 --2- 172.21.15.117:0/3777282232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a203abb390 0x55a203e842e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.599+0000 7f3f1affd640 1 -- 172.21.15.117:0/3777282232 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a203b47650 con 0x55a203adc070 2024-09-21T05:59:24.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.599+0000 7f3f1b7fe640 1 --2- 172.21.15.117:0/3777282232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a203ab31b0 0x55a203e83da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:59:24.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.599+0000 7f3f1bfff640 1 --2- 172.21.15.117:0/2371771116 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3f040b6880 0x7f3f040b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:24.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.600+0000 7f3f1affd640 1 --2- 172.21.15.117:0/3777282232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a203adc070 0x55a203e83860 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3f1002a950 tx=0x7f3f10005750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b31476b73cd01e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:24.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.601+0000 7f3f1bfff640 1 --2- 172.21.15.117:0/2371771116 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3f040b6880 0x7f3f040b8d90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3f14009270 tx=0x7f3f14009200 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:24.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.603+0000 7f3f01ffb640 1 -- 172.21.15.117:0/3777282232 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f10037070 con 0x55a203adc070 2024-09-21T05:59:24.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.603+0000 7f3f01ffb640 1 -- 172.21.15.117:0/3777282232 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f10038040 con 0x55a203adc070 2024-09-21T05:59:24.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.603+0000 7f3f01ffb640 1 -- 172.21.15.117:0/3777282232 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f100052b0 con 0x55a203adc070 2024-09-21T05:59:24.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.605+0000 7f3f28596740 1 -- 172.21.15.117:0/3777282232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a203adc070 msgr2=0x55a203e83860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:24.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.605+0000 7f3f28596740 1 --2- 172.21.15.117:0/3777282232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a203adc070 0x55a203e83860 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3f1002a950 tx=0x7f3f10005750 comp rx=0 tx=0).stop 2024-09-21T05:59:24.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.605+0000 7f3f28596740 1 -- 172.21.15.117:0/3777282232 shutdown_connections 2024-09-21T05:59:24.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.605+0000 7f3f28596740 1 --2- 172.21.15.117:0/3777282232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a203abb390 0x55a203e842e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.605+0000 7f3f28596740 1 --2- 172.21.15.117:0/3777282232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a203ab31b0 0x55a203e83da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.605+0000 7f3f28596740 1 --2- 172.21.15.117:0/3777282232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a203adc070 0x55a203e83860 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.605+0000 7f3f28596740 1 -- 172.21.15.117:0/3777282232 >> 172.21.15.117:0/3777282232 conn(0x55a203ad0460 msgr2=0x55a203e81750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:24.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.608+0000 7f3f28596740 1 -- 172.21.15.117:0/3777282232 shutdown_connections 2024-09-21T05:59:24.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.608+0000 7f3f28596740 1 -- 172.21.15.117:0/3777282232 wait complete. 2024-09-21T05:59:24.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.609+0000 7f3f28596740 1 -- start start 2024-09-21T05:59:24.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.610+0000 7f3f28596740 1 Processor -- start 2024-09-21T05:59:24.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.611+0000 7f3f28596740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a203ab31b0 0x55a203e8ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:24.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.611+0000 7f3f28596740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a203abb390 0x55a203e8f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:24.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.611+0000 7f3f28596740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a203adc070 0x55a203eac340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:24.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.611+0000 7f3f28596740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a203c4a700 con 0x55a203adc070 2024-09-21T05:59:24.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.611+0000 7f3f28596740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a203d84160 con 0x55a203abb390 2024-09-21T05:59:24.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.611+0000 7f3f28596740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a203ea2ff0 con 0x55a203ab31b0 2024-09-21T05:59:24.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.611+0000 7f3f28596740 10 client.0 _ll_register_callbacks cb 0x55a203d9d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:59:24.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.612+0000 7f3f1b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a203abb390 0x55a203e8f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:24.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.612+0000 7f3f1b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a203abb390 0x55a203e8f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:42320/0 (socket says 172.21.15.117:42320) 2024-09-21T05:59:24.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.612+0000 7f3f1b7fe640 1 -- 172.21.15.117:0/1332031937 learned_addr learned my addr 172.21.15.117:0/1332031937 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:24.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.613+0000 7f3f1affd640 1 --2- 172.21.15.117:0/1332031937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a203ab31b0 0x55a203e8ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:24.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.613+0000 7f3f1affd640 1 --2- 172.21.15.117:0/1332031937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a203adc070 0x55a203eac340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:24.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.614+0000 7f3f1affd640 1 -- 172.21.15.117:0/1332031937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a203abb390 msgr2=0x55a203e8f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:24.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.614+0000 7f3f1affd640 1 --2- 172.21.15.117:0/1332031937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a203abb390 0x55a203e8f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.614+0000 7f3f1affd640 1 -- 172.21.15.117:0/1332031937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a203adc070 msgr2=0x55a203eac340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:24.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.614+0000 7f3f1affd640 1 --2- 172.21.15.117:0/1332031937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a203adc070 0x55a203eac340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.614+0000 7f3f1affd640 1 -- 172.21.15.117:0/1332031937 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a203db1c90 con 0x55a203ab31b0 2024-09-21T05:59:24.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.617+0000 7f3f1affd640 1 --2- 172.21.15.117:0/1332031937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a203ab31b0 0x55a203e8ef80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3f10030490 tx=0x7f3f10005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a468908c0834224d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:24.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.617+0000 7f3efaffd640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-21T05:59:24.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.617+0000 7f3efaffd640 1 -- 172.21.15.117:0/1332031937 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f10037040 con 0x55a203ab31b0 2024-09-21T05:59:24.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.617+0000 7f3f28596740 1 -- 172.21.15.117:0/1332031937 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f10038040 con 0x55a203ab31b0 2024-09-21T05:59:24.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.618+0000 7f3efaffd640 1 -- 172.21.15.117:0/1332031937 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f3f10004030 con 0x55a203ab31b0 2024-09-21T05:59:24.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.618+0000 7f3efaffd640 1 -- 172.21.15.117:0/1332031937 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3f10054050 con 0x55a203ab31b0 2024-09-21T05:59:24.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.618+0000 7f3f28596740 10 client.25408 fetch_fsmap learned FSMap version 27 2024-09-21T05:59:24.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.618+0000 7f3f28596740 1 -- 172.21.15.117:0/1332031937 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a203eaed60 con 0x55a203ab31b0 2024-09-21T05:59:24.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.619+0000 7f3efaffd640 1 -- 172.21.15.117:0/1332031937 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f10038040 con 0x55a203ab31b0 2024-09-21T05:59:24.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.619+0000 7f3efaffd640 1 -- 172.21.15.117:0/1332031937 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3f10003c30 con 0x55a203ab31b0 2024-09-21T05:59:24.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.620+0000 7f3f28596740 10 client.25408 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:59:24.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.620+0000 7f3f28596740 10 client.25408 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T05:59:24.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.620+0000 7f3f28596740 20 client.25408 populate_metadata read hostname 'smithi117' 2024-09-21T05:59:24.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.620+0000 7f3f28596740 1 --2- 172.21.15.117:0/1332031937 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55a203a9a9f0 0x55a203eb20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:24.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.620+0000 7f3f28596740 4 client.25408 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:59:24.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.620+0000 7f3f28596740 1 -- 172.21.15.117:0/1332031937 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a203dfbdf0 con 0x55a203a9a9f0 2024-09-21T05:59:24.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.622+0000 7f3f1b7fe640 1 --2- 172.21.15.117:0/1332031937 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55a203a9a9f0 0x55a203eb20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:24.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.625+0000 7f3f1b7fe640 1 --2- 172.21.15.117:0/1332031937 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55a203a9a9f0 0x55a203eb20c0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62ab3cd0a59ff339 server_cookie=7f404f5fb136dc86 in_seq=0 out_seq=0 2024-09-21T05:59:24.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.625+0000 7f3efaffd640 10 client.25408 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:59:24.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.628+0000 7f3efaffd640 1 -- 172.21.15.117:0/1332031937 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a203dfbdf0 con 0x55a203a9a9f0 2024-09-21T05:59:24.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.628+0000 7f3efaffd640 10 client.25408 handle_command_reply: tid=0 2024-09-21T05:59:24.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.628+0000 7f3f28596740 1 client.25408 shutdown 2024-09-21T05:59:24.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.630+0000 7f3f28596740 1 -- 172.21.15.117:0/1332031937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a203ab31b0 msgr2=0x55a203e8ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:24.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.630+0000 7f3f28596740 1 --2- 172.21.15.117:0/1332031937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a203ab31b0 0x55a203e8ef80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3f10030490 tx=0x7f3f10005f90 comp rx=0 tx=0).stop 2024-09-21T05:59:24.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.630+0000 7f3f28596740 1 -- 172.21.15.117:0/1332031937 shutdown_connections 2024-09-21T05:59:24.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.630+0000 7f3f28596740 1 --2- 172.21.15.117:0/1332031937 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55a203a9a9f0 0x55a203eb20c0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.630+0000 7f3f28596740 1 --2- 172.21.15.117:0/1332031937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a203adc070 0x55a203eac340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.630+0000 7f3f28596740 1 --2- 172.21.15.117:0/1332031937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a203abb390 0x55a203e8f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.630+0000 7f3f28596740 1 --2- 172.21.15.117:0/1332031937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a203ab31b0 0x55a203e8ef80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.630+0000 7f3efaffd640 0 client.25408 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:59:24.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.630+0000 7f3f28596740 1 -- 172.21.15.117:0/1332031937 >> 172.21.15.117:0/1332031937 conn(0x55a203ad0460 msgr2=0x55a203e869f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:24.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.631+0000 7f3f28596740 1 -- 172.21.15.117:0/1332031937 shutdown_connections 2024-09-21T05:59:24.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.631+0000 7f3f28596740 1 -- 172.21.15.117:0/1332031937 wait complete. 2024-09-21T05:59:24.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.631+0000 7f3f28596740 20 client.25408 trim_cache size 0 max 16384 2024-09-21T05:59:24.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.663+0000 7f3f28596740 1 Processor -- start 2024-09-21T05:59:24.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.663+0000 7f3f28596740 1 -- start start 2024-09-21T05:59:24.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.664+0000 7f3f28596740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a203ab31b0 0x55a203e8f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:24.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.664+0000 7f3f28596740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a203abb390 0x55a203e86900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:24.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.664+0000 7f3f28596740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a203adc070 0x55a203e86e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:24.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.664+0000 7f3f28596740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0800e6f0 con 0x55a203abb390 2024-09-21T05:59:24.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.664+0000 7f3f28596740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3f08010230 con 0x55a203ab31b0 2024-09-21T05:59:24.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.664+0000 7f3f28596740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0800edc0 con 0x55a203adc070 2024-09-21T05:59:24.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.665+0000 7f3f1b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a203abb390 0x55a203e86900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:24.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.665+0000 7f3f1affd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a203adc070 0x55a203e86e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:24.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.665+0000 7f3f1affd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a203adc070 0x55a203e86e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:51772/0 (socket says 172.21.15.117:51772) 2024-09-21T05:59:24.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.665+0000 7f3f1affd640 1 -- 172.21.15.117:0/1378087387 learned_addr learned my addr 172.21.15.117:0/1378087387 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:24.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.666+0000 7f3f1affd640 1 --2- 172.21.15.117:0/1378087387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a203ab31b0 0x55a203e8f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:24.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.667+0000 7f3f1b7fe640 1 -- 172.21.15.117:0/1378087387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a203adc070 msgr2=0x55a203e86e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:24.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.667+0000 7f3f1b7fe640 1 --2- 172.21.15.117:0/1378087387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a203adc070 0x55a203e86e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.667+0000 7f3f1b7fe640 1 -- 172.21.15.117:0/1378087387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a203ab31b0 msgr2=0x55a203e8f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:24.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.667+0000 7f3f1b7fe640 1 --2- 172.21.15.117:0/1378087387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a203ab31b0 0x55a203e8f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.667+0000 7f3f1b7fe640 1 -- 172.21.15.117:0/1378087387 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a203e90240 con 0x55a203abb390 2024-09-21T05:59:24.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.667+0000 7f3f1affd640 1 --2- 172.21.15.117:0/1378087387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a203adc070 0x55a203e86e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:59:24.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.667+0000 7f3f1b7fe640 1 --2- 172.21.15.117:0/1378087387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a203abb390 0x55a203e86900 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x55a203c4a850 tx=0x7f3f0c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1ff0cb4d779421 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:24.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.668+0000 7f3f017fa640 1 -- 172.21.15.117:0/1378087387 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c024600 con 0x55a203abb390 2024-09-21T05:59:24.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.668+0000 7f3f017fa640 1 -- 172.21.15.117:0/1378087387 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f0c0118f0 con 0x55a203abb390 2024-09-21T05:59:24.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.668+0000 7f3f017fa640 1 -- 172.21.15.117:0/1378087387 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c01e8f0 con 0x55a203abb390 2024-09-21T05:59:24.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.670+0000 7f3f28596740 1 -- 172.21.15.117:0/1378087387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a203abb390 msgr2=0x55a203e86900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:24.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.670+0000 7f3f28596740 1 --2- 172.21.15.117:0/1378087387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a203abb390 0x55a203e86900 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x55a203c4a850 tx=0x7f3f0c009260 comp rx=0 tx=0).stop 2024-09-21T05:59:24.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.671+0000 7f3f28596740 1 -- 172.21.15.117:0/1378087387 shutdown_connections 2024-09-21T05:59:24.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.671+0000 7f3f28596740 1 --2- 172.21.15.117:0/1378087387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a203abb390 0x55a203e86900 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.671+0000 7f3f28596740 1 --2- 172.21.15.117:0/1378087387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a203adc070 0x55a203e86e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.671+0000 7f3f28596740 1 --2- 172.21.15.117:0/1378087387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a203ab31b0 0x55a203e8f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.671+0000 7f3f28596740 1 -- 172.21.15.117:0/1378087387 >> 172.21.15.117:0/1378087387 conn(0x55a203ad0460 msgr2=0x55a203e8ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:24.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.672+0000 7f3f28596740 1 -- 172.21.15.117:0/1378087387 shutdown_connections 2024-09-21T05:59:24.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.672+0000 7f3f28596740 1 -- 172.21.15.117:0/1378087387 wait complete. 2024-09-21T05:59:24.838 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T05:59:24.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.673+0000 7f3f28596740 1 -- start start 2024-09-21T05:59:24.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.673+0000 7f3f28596740 1 Processor -- start 2024-09-21T05:59:24.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.678+0000 7f3f28596740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a203ab31b0 0x55a203ea4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:24.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.678+0000 7f3f28596740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a203abb390 0x55a203ea4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:24.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.678+0000 7f3f28596740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a203adc070 0x55a203ea4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:24.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.678+0000 7f3f28596740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0800fb60 con 0x55a203adc070 2024-09-21T05:59:24.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.678+0000 7f3f28596740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0800f490 con 0x55a203ab31b0 2024-09-21T05:59:24.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.678+0000 7f3f28596740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0800c6d0 con 0x55a203abb390 2024-09-21T05:59:24.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.678+0000 7f3f28596740 10 client.0 _ll_register_callbacks cb 0x7f3f0c009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:59:24.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.681+0000 7f3f1affd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a203ab31b0 0x55a203ea4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:24.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.681+0000 7f3f1affd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a203adc070 0x55a203ea4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:24.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.681+0000 7f3f1affd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a203adc070 0x55a203ea4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:52994/0 (socket says 172.21.15.117:52994) 2024-09-21T05:59:24.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.681+0000 7f3f1affd640 1 -- 172.21.15.117:0/1193648301 learned_addr learned my addr 172.21.15.117:0/1193648301 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:24.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.682+0000 7f3f1b7fe640 1 --2- 172.21.15.117:0/1193648301 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a203abb390 0x55a203ea4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:24.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.683+0000 7f3f1affd640 1 -- 172.21.15.117:0/1193648301 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a203abb390 msgr2=0x55a203ea4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:24.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.683+0000 7f3f1affd640 1 --2- 172.21.15.117:0/1193648301 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a203abb390 0x55a203ea4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.683+0000 7f3f1affd640 1 -- 172.21.15.117:0/1193648301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a203ab31b0 msgr2=0x55a203ea4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:24.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.683+0000 7f3f1affd640 1 --2- 172.21.15.117:0/1193648301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a203ab31b0 0x55a203ea4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.683+0000 7f3f1affd640 1 -- 172.21.15.117:0/1193648301 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a203f1bb10 con 0x55a203adc070 2024-09-21T05:59:24.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.684+0000 7f3f1affd640 1 --2- 172.21.15.117:0/1193648301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a203adc070 0x55a203ea4dd0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3f10030cf0 tx=0x7f3f10000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7047524073b6838f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:24.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.684+0000 7f3efaffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:59:24.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.685+0000 7f3efaffd640 1 -- 172.21.15.117:0/1193648301 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f10037070 con 0x55a203adc070 2024-09-21T05:59:24.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.686+0000 7f3efaffd640 1 -- 172.21.15.117:0/1193648301 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f3f10041ca0 con 0x55a203adc070 2024-09-21T05:59:24.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.687+0000 7f3f28596740 1 -- 172.21.15.117:0/1193648301 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f0c0118f0 con 0x55a203adc070 2024-09-21T05:59:24.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.688+0000 7f3efaffd640 1 -- 172.21.15.117:0/1193648301 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3f1005f050 con 0x55a203adc070 2024-09-21T05:59:24.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.689+0000 7f3f28596740 10 client.15993 fetch_fsmap learned FSMap version 27 2024-09-21T05:59:24.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.689+0000 7f3f28596740 1 -- 172.21.15.117:0/1193648301 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a203f1cd90 con 0x55a203adc070 2024-09-21T05:59:24.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.690+0000 7f3efaffd640 1 -- 172.21.15.117:0/1193648301 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f0c0118f0 con 0x55a203adc070 2024-09-21T05:59:24.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.690+0000 7f3efaffd640 1 -- 172.21.15.117:0/1193648301 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3f10003990 con 0x55a203adc070 2024-09-21T05:59:24.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.690+0000 7f3f28596740 10 client.15993 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:59:24.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.690+0000 7f3f28596740 10 client.15993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T05:59:24.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.690+0000 7f3f28596740 20 client.15993 populate_metadata read hostname 'smithi117' 2024-09-21T05:59:24.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.690+0000 7f3f28596740 1 --2- 172.21.15.117:0/1193648301 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55a203a6fda0 0x55a203f1f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:24.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.690+0000 7f3f28596740 4 client.15993 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T05:59:24.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.690+0000 7f3f28596740 1 -- 172.21.15.117:0/1193648301 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a203df78a0 con 0x55a203a6fda0 2024-09-21T05:59:24.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.694+0000 7f3f1b7fe640 1 --2- 172.21.15.117:0/1193648301 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55a203a6fda0 0x55a203f1f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:24.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.705+0000 7f3f1b7fe640 1 --2- 172.21.15.117:0/1193648301 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55a203a6fda0 0x55a203f1f4a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c25d9fd617f523 server_cookie=e269cdc996e75320 in_seq=0 out_seq=0 2024-09-21T05:59:24.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.705+0000 7f3efaffd640 10 client.15993 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:59:24.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.710+0000 7f3efaffd640 1 -- 172.21.15.117:0/1193648301 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a203df78a0 con 0x55a203a6fda0 2024-09-21T05:59:24.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.710+0000 7f3efaffd640 10 client.15993 handle_command_reply: tid=0 2024-09-21T05:59:24.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.710+0000 7f3f28596740 1 client.15993 shutdown 2024-09-21T05:59:24.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.710+0000 7f3f28596740 1 -- 172.21.15.117:0/1193648301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a203adc070 msgr2=0x55a203ea4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:24.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.710+0000 7f3f28596740 1 --2- 172.21.15.117:0/1193648301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a203adc070 0x55a203ea4dd0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3f10030cf0 tx=0x7f3f10000bc0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.711+0000 7f3f28596740 1 -- 172.21.15.117:0/1193648301 shutdown_connections 2024-09-21T05:59:24.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.711+0000 7f3f28596740 1 --2- 172.21.15.117:0/1193648301 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55a203a6fda0 0x55a203f1f4a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.711+0000 7f3f28596740 1 --2- 172.21.15.117:0/1193648301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a203adc070 0x55a203ea4dd0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.711+0000 7f3f28596740 1 --2- 172.21.15.117:0/1193648301 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a203abb390 0x55a203ea4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.711+0000 7f3f28596740 1 --2- 172.21.15.117:0/1193648301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a203ab31b0 0x55a203ea4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.711+0000 7f3f28596740 1 -- 172.21.15.117:0/1193648301 >> 172.21.15.117:0/1193648301 conn(0x55a203ad0460 msgr2=0x55a203e90410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:24.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.711+0000 7f3efaffd640 0 client.15993 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:59:24.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.712+0000 7f3f28596740 1 -- 172.21.15.117:0/1193648301 shutdown_connections 2024-09-21T05:59:24.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.712+0000 7f3f28596740 1 -- 172.21.15.117:0/1193648301 wait complete. 2024-09-21T05:59:24.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.713+0000 7f3f28596740 20 client.15993 trim_cache size 0 max 16384 2024-09-21T05:59:24.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.714+0000 7f3f22022640 1 -- 172.21.15.117:0/2371771116 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3f040b6880 msgr2=0x7f3f040b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:24.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.714+0000 7f3f22022640 1 --2- 172.21.15.117:0/2371771116 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3f040b6880 0x7f3f040b8d90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3f14009270 tx=0x7f3f14009200 comp rx=0 tx=0).stop 2024-09-21T05:59:24.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.714+0000 7f3f22022640 1 -- 172.21.15.117:0/2371771116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f1c11cba0 msgr2=0x7f3f1c1ae980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:24.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.714+0000 7f3f22022640 1 --2- 172.21.15.117:0/2371771116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f1c11cba0 0x7f3f1c1ae980 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3f1400eb10 tx=0x7f3f1400c2d0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.714+0000 7f3f22022640 1 -- 172.21.15.117:0/2371771116 shutdown_connections 2024-09-21T05:59:24.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.714+0000 7f3f22022640 1 --2- 172.21.15.117:0/2371771116 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3f040b6880 0x7f3f040b8d90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.714+0000 7f3f22022640 1 --2- 172.21.15.117:0/2371771116 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f1c11cba0 0x7f3f1c1ae980 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.714+0000 7f3f22022640 1 --2- 172.21.15.117:0/2371771116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f1c111d40 0x7f3f1c1a96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.714+0000 7f3f22022640 1 --2- 172.21.15.117:0/2371771116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f1c10ab80 0x7f3f1c1a9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:24.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.714+0000 7f3f22022640 1 -- 172.21.15.117:0/2371771116 >> 172.21.15.117:0/2371771116 conn(0x7f3f1c06d950 msgr2=0x7f3f1c110c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:24.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.715+0000 7f3f22022640 1 -- 172.21.15.117:0/2371771116 shutdown_connections 2024-09-21T05:59:24.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:24.715+0000 7f3f22022640 1 -- 172.21.15.117:0/2371771116 wait complete. 2024-09-21T05:59:25.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:25 smithi138 ceph-mon[34391]: pgmap v390: 97 pgs: 97 active+clean; 18 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 865 MiB/s wr, 2.88k op/s 2024-09-21T05:59:25.730 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:25 smithi192 ceph-mon[34700]: pgmap v390: 97 pgs: 97 active+clean; 18 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 865 MiB/s wr, 2.88k op/s 2024-09-21T05:59:25.779 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:25 smithi117 ceph-mon[30947]: pgmap v390: 97 pgs: 97 active+clean; 18 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 865 MiB/s wr, 2.88k op/s 2024-09-21T05:59:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:26.441+0000 7f7c927fc640 1 -- 172.21.15.117:0/3895629599 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f7ca40b1bf0 con 0x5633e28ea1c0 2024-09-21T05:59:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:26.441+0000 7f7c927fc640 10 client.25403 handle_command_reply: tid=0 2024-09-21T05:59:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:26.441+0000 7f7cc0e48740 1 client.25403 shutdown 2024-09-21T05:59:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:26.441+0000 7f7cc0e48740 1 -- 172.21.15.117:0/3895629599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e292d2f0 msgr2=0x5633e2d17b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:26.441+0000 7f7cc0e48740 1 --2- 172.21.15.117:0/3895629599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e292d2f0 0x5633e2d17b60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7ca801ea60 tx=0x7f7ca8020000 comp rx=0 tx=0).stop 2024-09-21T05:59:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:26.442+0000 7f7cc0e48740 1 -- 172.21.15.117:0/3895629599 shutdown_connections 2024-09-21T05:59:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:26.442+0000 7f7cc0e48740 1 --2- 172.21.15.117:0/3895629599 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5633e28ea1c0 0x5633e2d970e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:26.442+0000 7f7cc0e48740 1 --2- 172.21.15.117:0/3895629599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5633e29561b0 0x5633e2d31760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:26.442+0000 7f7cc0e48740 1 --2- 172.21.15.117:0/3895629599 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633e29354d0 0x5633e2d18120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:26.442+0000 7f7cc0e48740 1 --2- 172.21.15.117:0/3895629599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633e292d2f0 0x5633e2d17b60 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:26.442+0000 7f7cc0e48740 1 -- 172.21.15.117:0/3895629599 >> 172.21.15.117:0/3895629599 conn(0x5633e294a300 msgr2=0x5633e2d01a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:26.442+0000 7f7c927fc640 0 client.25403 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:59:26.448 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T05:59:26.448 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T05:59:26.448 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T05:59:26.449 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T05:59:26.449 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:26.442+0000 7f7cc0e48740 1 -- 172.21.15.117:0/3895629599 shutdown_connections 2024-09-21T05:59:26.449 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:26.442+0000 7f7cc0e48740 1 -- 172.21.15.117:0/3895629599 wait complete. 2024-09-21T05:59:26.449 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:26.442+0000 7f7cc0e48740 20 client.25403 trim_cache size 0 max 16384 2024-09-21T05:59:26.452 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:26.446+0000 7f7c9a7fc640 1 -- 172.21.15.117:0/4241515285 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7c9c0b6850 msgr2=0x7f7c9c0b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:26.452 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:26.446+0000 7f7c9a7fc640 1 --2- 172.21.15.117:0/4241515285 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7c9c0b6850 0x7f7c9c0b8d60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8005fd0 tx=0x7f7ca8005f20 comp rx=0 tx=0).stop 2024-09-21T05:59:26.452 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:26.446+0000 7f7c9a7fc640 1 -- 172.21.15.117:0/4241515285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cb4108c60 msgr2=0x7f7cb419d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:26.452 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:26.446+0000 7f7c9a7fc640 1 --2- 172.21.15.117:0/4241515285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cb4108c60 0x7f7cb419d910 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f7ca400d680 tx=0x7f7ca400db40 comp rx=0 tx=0).stop 2024-09-21T05:59:26.452 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:26.447+0000 7f7c9a7fc640 1 -- 172.21.15.117:0/4241515285 shutdown_connections 2024-09-21T05:59:26.452 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:26.447+0000 7f7c9a7fc640 1 --2- 172.21.15.117:0/4241515285 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7c9c0b6850 0x7f7c9c0b8d60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:26.452 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:26.447+0000 7f7c9a7fc640 1 --2- 172.21.15.117:0/4241515285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cb4108c60 0x7f7cb419d910 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:26.452 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:26.447+0000 7f7c9a7fc640 1 --2- 172.21.15.117:0/4241515285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cb4108310 0x7f7cb4198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:26.453 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:26.447+0000 7f7c9a7fc640 1 --2- 172.21.15.117:0/4241515285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cb41029e0 0x7f7cb41980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:26.453 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:26.447+0000 7f7c9a7fc640 1 -- 172.21.15.117:0/4241515285 >> 172.21.15.117:0/4241515285 conn(0x7f7cb407a6e0 msgr2=0x7f7cb4106300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:26.453 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:26.449+0000 7f7c9a7fc640 1 -- 172.21.15.117:0/4241515285 shutdown_connections 2024-09-21T05:59:26.453 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:26.449+0000 7f7c9a7fc640 1 -- 172.21.15.117:0/4241515285 wait complete. 2024-09-21T05:59:27.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:27 smithi192 ceph-mon[34700]: pgmap v391: 97 pgs: 97 active+clean; 19 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 868 MiB/s wr, 2.38k op/s 2024-09-21T05:59:27.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:27 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:59:27.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:27 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:59:27.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:27 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:59:28.029 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:27 smithi117 ceph-mon[30947]: pgmap v391: 97 pgs: 97 active+clean; 19 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 868 MiB/s wr, 2.38k op/s 2024-09-21T05:59:28.029 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:27 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:59:28.029 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:27 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:59:28.029 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:27 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:59:28.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:27 smithi138 ceph-mon[34391]: pgmap v391: 97 pgs: 97 active+clean; 19 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 868 MiB/s wr, 2.38k op/s 2024-09-21T05:59:28.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:27 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T05:59:28.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:27 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T05:59:28.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:27 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T05:59:29.430 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T05:59:29.431 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T05:59:29.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:29 smithi192 ceph-mon[34700]: pgmap v392: 97 pgs: 97 active+clean; 19 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 933 MiB/s wr, 2.22k op/s 2024-09-21T05:59:30.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:29 smithi117 ceph-mon[30947]: pgmap v392: 97 pgs: 97 active+clean; 19 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 933 MiB/s wr, 2.22k op/s 2024-09-21T05:59:30.064 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:59:30.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:29 smithi138 ceph-mon[34391]: pgmap v392: 97 pgs: 97 active+clean; 19 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 933 MiB/s wr, 2.22k op/s 2024-09-21T05:59:31.697 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:31 smithi117 ceph-mon[30947]: pgmap v393: 97 pgs: 97 active+clean; 20 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 844 MiB/s wr, 1.72k op/s 2024-09-21T05:59:31.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.624+0000 7fe93c357640 1 -- 172.21.15.117:0/807754935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9341021f0 msgr2=0x7fe934102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:31.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.624+0000 7fe93c357640 1 --2- 172.21.15.117:0/807754935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9341021f0 0x7fe934102600 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe930009900 tx=0x7fe93002a410 comp rx=0 tx=0).stop 2024-09-21T05:59:31.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.624+0000 7fe93c357640 1 -- 172.21.15.117:0/807754935 shutdown_connections 2024-09-21T05:59:31.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.624+0000 7fe93c357640 1 --2- 172.21.15.117:0/807754935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe934108480 0x7fe93410ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.624+0000 7fe93c357640 1 --2- 172.21.15.117:0/807754935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe934107b30 0x7fe934107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.624+0000 7fe93c357640 1 --2- 172.21.15.117:0/807754935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9341021f0 0x7fe934102600 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.624+0000 7fe93c357640 1 -- 172.21.15.117:0/807754935 >> 172.21.15.117:0/807754935 conn(0x7fe934077c60 msgr2=0x7fe9340780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:31.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.624+0000 7fe93c357640 1 -- 172.21.15.117:0/807754935 shutdown_connections 2024-09-21T05:59:31.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.624+0000 7fe93c357640 1 -- 172.21.15.117:0/807754935 wait complete. 2024-09-21T05:59:31.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.625+0000 7fe93c357640 1 Processor -- start 2024-09-21T05:59:31.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.626+0000 7fe93c357640 1 -- start start 2024-09-21T05:59:31.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.626+0000 7fe93c357640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9341021f0 0x7fe9341982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:31.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.626+0000 7fe93c357640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe934107b30 0x7fe934198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:31.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.626+0000 7fe93c357640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe934108480 0x7fe93419dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:31.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.626+0000 7fe93c357640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe934111910 con 0x7fe934107b30 2024-09-21T05:59:31.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.626+0000 7fe93c357640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe934111790 con 0x7fe9341021f0 2024-09-21T05:59:31.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.626+0000 7fe93c357640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe934111a90 con 0x7fe934108480 2024-09-21T05:59:31.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.626+0000 7fe9398f2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe934107b30 0x7fe934198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:31.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.626+0000 7fe9398f2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe934107b30 0x7fe934198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53006/0 (socket says 172.21.15.117:53006) 2024-09-21T05:59:31.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.626+0000 7fe9398f2640 1 -- 172.21.15.117:0/1248167518 learned_addr learned my addr 172.21.15.117:0/1248167518 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:31.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.627+0000 7fe9398f2640 1 -- 172.21.15.117:0/1248167518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe934108480 msgr2=0x7fe93419dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:31.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.627+0000 7fe9398f2640 1 --2- 172.21.15.117:0/1248167518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe934108480 0x7fe93419dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.627+0000 7fe9398f2640 1 -- 172.21.15.117:0/1248167518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9341021f0 msgr2=0x7fe9341982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:31.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.627+0000 7fe9398f2640 1 --2- 172.21.15.117:0/1248167518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9341021f0 0x7fe9341982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.627+0000 7fe9398f2640 1 -- 172.21.15.117:0/1248167518 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe93419e210 con 0x7fe934107b30 2024-09-21T05:59:31.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.627+0000 7fe9398f2640 1 --2- 172.21.15.117:0/1248167518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe934107b30 0x7fe934198840 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe92400db90 tx=0x7fe92400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:31.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.627+0000 7fe92b7fe640 1 -- 172.21.15.117:0/1248167518 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92401b070 con 0x7fe934107b30 2024-09-21T05:59:31.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.627+0000 7fe92b7fe640 1 -- 172.21.15.117:0/1248167518 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe92400ae50 con 0x7fe934107b30 2024-09-21T05:59:31.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.628+0000 7fe92b7fe640 1 -- 172.21.15.117:0/1248167518 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924016a00 con 0x7fe934107b30 2024-09-21T05:59:31.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.628+0000 7fe93c357640 1 -- 172.21.15.117:0/1248167518 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe93419e500 con 0x7fe934107b30 2024-09-21T05:59:31.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.628+0000 7fe93c357640 1 -- 172.21.15.117:0/1248167518 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9341a4c60 con 0x7fe934107b30 2024-09-21T05:59:31.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.631+0000 7fe9428cb740 1 Processor -- start 2024-09-21T05:59:31.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.631+0000 7fe9428cb740 1 -- start start 2024-09-21T05:59:31.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.632+0000 7fe9428cb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565b0def350 0x5565b119a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:31.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.632+0000 7fe9428cb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565b0dc6490 0x5565b119ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:31.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.632+0000 7fe9428cb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565b0dce670 0x5565b119b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:31.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.632+0000 7fe9428cb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5565b110ce60 con 0x5565b0dc6490 2024-09-21T05:59:31.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.632+0000 7fe9428cb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5565b10fc290 con 0x5565b0dce670 2024-09-21T05:59:31.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.632+0000 7fe9428cb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5565b11000a0 con 0x5565b0def350 2024-09-21T05:59:31.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.632+0000 7fe93a0f3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565b0dc6490 0x5565b119ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:31.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.632+0000 7fe93a0f3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565b0dc6490 0x5565b119ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53014/0 (socket says 172.21.15.117:53014) 2024-09-21T05:59:31.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.632+0000 7fe93a0f3640 1 -- 172.21.15.117:0/3443732881 learned_addr learned my addr 172.21.15.117:0/3443732881 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:31.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.633+0000 7fe93a0f3640 1 -- 172.21.15.117:0/3443732881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565b0def350 msgr2=0x5565b119a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:31.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.633+0000 7fe93a0f3640 1 --2- 172.21.15.117:0/3443732881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565b0def350 0x5565b119a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.633+0000 7fe93a0f3640 1 -- 172.21.15.117:0/3443732881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565b0dce670 msgr2=0x5565b119b2c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T05:59:31.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.633+0000 7fe93a0f3640 1 --2- 172.21.15.117:0/3443732881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565b0dce670 0x5565b119b2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.633+0000 7fe93a0f3640 1 -- 172.21.15.117:0/3443732881 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565b0e5a930 con 0x5565b0dc6490 2024-09-21T05:59:31.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.633+0000 7fe93a0f3640 1 --2- 172.21.15.117:0/3443732881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565b0dc6490 0x5565b119ad80 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe930005f70 tx=0x7fe930000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14414617c1bce15e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:31.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.634+0000 7fe928d9a640 1 -- 172.21.15.117:0/3443732881 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93003e030 con 0x5565b0dc6490 2024-09-21T05:59:31.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.634+0000 7fe928d9a640 1 -- 172.21.15.117:0/3443732881 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9300054d0 con 0x5565b0dc6490 2024-09-21T05:59:31.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.634+0000 7fe928d9a640 1 -- 172.21.15.117:0/3443732881 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe930035040 con 0x5565b0dc6490 2024-09-21T05:59:31.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.635+0000 7fe92b7fe640 1 -- 172.21.15.117:0/1248167518 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fe92401e070 con 0x7fe934107b30 2024-09-21T05:59:31.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.635+0000 7fe9428cb740 1 -- 172.21.15.117:0/3443732881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565b0dc6490 msgr2=0x5565b119ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:31.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.635+0000 7fe9428cb740 1 --2- 172.21.15.117:0/3443732881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565b0dc6490 0x5565b119ad80 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe930005f70 tx=0x7fe930000bc0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.635+0000 7fe9428cb740 1 -- 172.21.15.117:0/3443732881 shutdown_connections 2024-09-21T05:59:31.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.635+0000 7fe9428cb740 1 --2- 172.21.15.117:0/3443732881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565b0dc6490 0x5565b119ad80 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.635+0000 7fe9428cb740 1 --2- 172.21.15.117:0/3443732881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565b0dce670 0x5565b119b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.635+0000 7fe9428cb740 1 --2- 172.21.15.117:0/3443732881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565b0def350 0x5565b119a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.635+0000 7fe9428cb740 1 -- 172.21.15.117:0/3443732881 >> 172.21.15.117:0/3443732881 conn(0x5565b0de3740 msgr2=0x5565b1198730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:31.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.635+0000 7fe9428cb740 1 -- 172.21.15.117:0/3443732881 shutdown_connections 2024-09-21T05:59:31.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.636+0000 7fe9428cb740 1 -- 172.21.15.117:0/3443732881 wait complete. 2024-09-21T05:59:31.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.637+0000 7fe9428cb740 1 -- start start 2024-09-21T05:59:31.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.638+0000 7fe9428cb740 1 Processor -- start 2024-09-21T05:59:31.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.638+0000 7fe92b7fe640 1 --2- 172.21.15.117:0/1248167518 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe9200b6610 0x7fe9200b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:31.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.638+0000 7fe93a8f4640 1 --2- 172.21.15.117:0/1248167518 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe9200b6610 0x7fe9200b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:31.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.639+0000 7fe9428cb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565b0dc6490 0x5565b11a6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:31.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.640+0000 7fe92b7fe640 1 -- 172.21.15.117:0/1248167518 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fe9240e7290 con 0x7fe934107b30 2024-09-21T05:59:31.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.640+0000 7fe93a8f4640 1 --2- 172.21.15.117:0/1248167518 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe9200b6610 0x7fe9200b8b20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe92c005c40 tx=0x7fe92c005bd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:31.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.640+0000 7fe9428cb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565b0dce670 0x5565b11a6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:31.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.640+0000 7fe9428cb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565b0def350 0x5565b11b2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:31.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.640+0000 7fe9428cb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5565b0f5d890 con 0x5565b0def350 2024-09-21T05:59:31.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.640+0000 7fe9428cb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5565b1096d40 con 0x5565b0dce670 2024-09-21T05:59:31.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.640+0000 7fe9428cb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5565b11a9010 con 0x5565b0dc6490 2024-09-21T05:59:31.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.640+0000 7fe9428cb740 10 client.0 _ll_register_callbacks cb 0x5565b10fac80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:59:31.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.642+0000 7fe9398f2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565b0dce670 0x5565b11a6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:31.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.642+0000 7fe93a8f4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565b0def350 0x5565b11b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:31.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.642+0000 7fe93a8f4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565b0def350 0x5565b11b2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53018/0 (socket says 172.21.15.117:53018) 2024-09-21T05:59:31.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.642+0000 7fe93a8f4640 1 -- 172.21.15.117:0/2173145792 learned_addr learned my addr 172.21.15.117:0/2173145792 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:31.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.643+0000 7fe93a0f3640 1 --2- 172.21.15.117:0/2173145792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565b0dc6490 0x5565b11a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:31.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.643+0000 7fe93a8f4640 1 -- 172.21.15.117:0/2173145792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565b0dc6490 msgr2=0x5565b11a6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:31.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.643+0000 7fe93a8f4640 1 --2- 172.21.15.117:0/2173145792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565b0dc6490 0x5565b11a6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.643+0000 7fe93a8f4640 1 -- 172.21.15.117:0/2173145792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565b0dce670 msgr2=0x5565b11a6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:31.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.643+0000 7fe93a8f4640 1 --2- 172.21.15.117:0/2173145792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565b0dce670 0x5565b11a6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.643+0000 7fe93a8f4640 1 -- 172.21.15.117:0/2173145792 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565b11b28f0 con 0x5565b0def350 2024-09-21T05:59:31.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.643+0000 7fe93a8f4640 1 --2- 172.21.15.117:0/2173145792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565b0def350 0x5565b11b2300 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fe92c011810 tx=0x7fe92c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82924a3e9a4cd6e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:31.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.643+0000 7fe91dffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:59:31.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.644+0000 7fe91dffb640 1 -- 172.21.15.117:0/2173145792 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c022020 con 0x5565b0def350 2024-09-21T05:59:31.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.644+0000 7fe91dffb640 1 -- 172.21.15.117:0/2173145792 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fe92c010950 con 0x5565b0def350 2024-09-21T05:59:31.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.644+0000 7fe9428cb740 1 -- 172.21.15.117:0/2173145792 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9300054d0 con 0x5565b0def350 2024-09-21T05:59:31.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.645+0000 7fe91dffb640 1 -- 172.21.15.117:0/2173145792 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe92c02d050 con 0x5565b0def350 2024-09-21T05:59:31.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.645+0000 7fe9428cb740 10 client.16005 fetch_fsmap learned FSMap version 27 2024-09-21T05:59:31.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.645+0000 7fe9428cb740 1 -- 172.21.15.117:0/2173145792 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5565b1193c20 con 0x5565b0def350 2024-09-21T05:59:31.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.645+0000 7fe91dffb640 1 -- 172.21.15.117:0/2173145792 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe92c017030 con 0x5565b0def350 2024-09-21T05:59:31.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.646+0000 7fe91dffb640 1 -- 172.21.15.117:0/2173145792 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe92c00f040 con 0x5565b0def350 2024-09-21T05:59:31.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.646+0000 7fe9428cb740 10 client.16005 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:59:31.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.647+0000 7fe9428cb740 10 client.16005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T05:59:31.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.647+0000 7fe9428cb740 20 client.16005 populate_metadata read hostname 'smithi117' 2024-09-21T05:59:31.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.647+0000 7fe9428cb740 1 --2- 172.21.15.117:0/2173145792 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5565b0dadcd0 0x5565b1197170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:31.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.647+0000 7fe9428cb740 4 client.16005 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:59:31.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.647+0000 7fe9428cb740 1 -- 172.21.15.117:0/2173145792 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe9300054d0 con 0x5565b0dadcd0 2024-09-21T05:59:31.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.647+0000 7fe93a0f3640 1 --2- 172.21.15.117:0/2173145792 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5565b0dadcd0 0x5565b1197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:31.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.650+0000 7fe93a0f3640 1 --2- 172.21.15.117:0/2173145792 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5565b0dadcd0 0x5565b1197170 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29bc2733d8735d7e server_cookie=dbe7c9c7ad451ddf in_seq=0 out_seq=0 2024-09-21T05:59:31.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.650+0000 7fe91dffb640 10 client.16005 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:59:31.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.653+0000 7fe91dffb640 1 -- 172.21.15.117:0/2173145792 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fe9300054d0 con 0x5565b0dadcd0 2024-09-21T05:59:31.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.653+0000 7fe91dffb640 10 client.16005 handle_command_reply: tid=0 2024-09-21T05:59:31.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.653+0000 7fe9428cb740 1 client.16005 shutdown 2024-09-21T05:59:31.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.654+0000 7fe9428cb740 1 -- 172.21.15.117:0/2173145792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565b0def350 msgr2=0x5565b11b2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:31.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.654+0000 7fe9428cb740 1 --2- 172.21.15.117:0/2173145792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565b0def350 0x5565b11b2300 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fe92c011810 tx=0x7fe92c004a40 comp rx=0 tx=0).stop 2024-09-21T05:59:31.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.654+0000 7fe9428cb740 1 -- 172.21.15.117:0/2173145792 shutdown_connections 2024-09-21T05:59:31.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.654+0000 7fe9428cb740 1 --2- 172.21.15.117:0/2173145792 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5565b0dadcd0 0x5565b1197170 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.654+0000 7fe9428cb740 1 --2- 172.21.15.117:0/2173145792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565b0def350 0x5565b11b2300 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.654+0000 7fe91dffb640 0 client.16005 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:59:31.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.654+0000 7fe9428cb740 1 --2- 172.21.15.117:0/2173145792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565b0dce670 0x5565b11a6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.654+0000 7fe9428cb740 1 --2- 172.21.15.117:0/2173145792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565b0dc6490 0x5565b11a6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.654+0000 7fe9428cb740 1 -- 172.21.15.117:0/2173145792 >> 172.21.15.117:0/2173145792 conn(0x5565b0de3740 msgr2=0x5565b119d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:31.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.654+0000 7fe9428cb740 1 -- 172.21.15.117:0/2173145792 shutdown_connections 2024-09-21T05:59:31.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.654+0000 7fe9428cb740 1 -- 172.21.15.117:0/2173145792 wait complete. 2024-09-21T05:59:31.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.655+0000 7fe9428cb740 20 client.16005 trim_cache size 0 max 16384 2024-09-21T05:59:31.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.695+0000 7fe9428cb740 1 Processor -- start 2024-09-21T05:59:31.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.695+0000 7fe9428cb740 1 -- start start 2024-09-21T05:59:31.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.695+0000 7fe9428cb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565b0dadcd0 0x5565b11a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:31.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.696+0000 7fe93a0f3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565b0dadcd0 0x5565b11a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:31.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.696+0000 7fe93a0f3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565b0dadcd0 0x5565b11a5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53024/0 (socket says 172.21.15.117:53024) 2024-09-21T05:59:31.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.696+0000 7fe93a0f3640 1 -- 172.21.15.117:0/111032219 learned_addr learned my addr 172.21.15.117:0/111032219 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:31.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.696+0000 7fe9428cb740 1 --2- 172.21.15.117:0/111032219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565b0dc6490 0x5565b11a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:31.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.696+0000 7fe9428cb740 1 --2- 172.21.15.117:0/111032219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565b0dce670 0x5565b119e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:31.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.696+0000 7fe9428cb740 1 -- 172.21.15.117:0/111032219 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341ad360 con 0x5565b0dadcd0 2024-09-21T05:59:31.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.697+0000 7fe9428cb740 1 -- 172.21.15.117:0/111032219 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341aeea0 con 0x5565b0dc6490 2024-09-21T05:59:31.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.697+0000 7fe9428cb740 1 -- 172.21.15.117:0/111032219 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341ada30 con 0x5565b0dce670 2024-09-21T05:59:31.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.697+0000 7fe93a0f3640 1 -- 172.21.15.117:0/111032219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565b0dce670 msgr2=0x5565b119e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:31.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.697+0000 7fe93a0f3640 1 --2- 172.21.15.117:0/111032219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565b0dce670 0x5565b119e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.697+0000 7fe93a0f3640 1 -- 172.21.15.117:0/111032219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565b0dc6490 msgr2=0x5565b11a1a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T05:59:31.771 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.697+0000 7fe93a0f3640 1 --2- 172.21.15.117:0/111032219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565b0dc6490 0x5565b11a1a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.697+0000 7fe93a0f3640 1 -- 172.21.15.117:0/111032219 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565b1193510 con 0x5565b0dadcd0 2024-09-21T05:59:31.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.698+0000 7fe93a0f3640 1 --2- 172.21.15.117:0/111032219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565b0dadcd0 0x5565b11a5ac0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe9300048b0 tx=0x7fe9300097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575b56cbfdd2db2c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:31.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.698+0000 7fe91ffff640 1 -- 172.21.15.117:0/111032219 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93003e040 con 0x5565b0dadcd0 2024-09-21T05:59:31.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.698+0000 7fe91ffff640 1 -- 172.21.15.117:0/111032219 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe930003aa0 con 0x5565b0dadcd0 2024-09-21T05:59:31.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.698+0000 7fe91ffff640 1 -- 172.21.15.117:0/111032219 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe930035040 con 0x5565b0dadcd0 2024-09-21T05:59:31.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.700+0000 7fe9428cb740 1 -- 172.21.15.117:0/111032219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565b0dadcd0 msgr2=0x5565b11a5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:31.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.700+0000 7fe9428cb740 1 --2- 172.21.15.117:0/111032219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565b0dadcd0 0x5565b11a5ac0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe9300048b0 tx=0x7fe9300097a0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.700+0000 7fe9428cb740 1 -- 172.21.15.117:0/111032219 shutdown_connections 2024-09-21T05:59:31.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.700+0000 7fe9428cb740 1 --2- 172.21.15.117:0/111032219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565b0dce670 0x5565b119e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.700+0000 7fe9428cb740 1 --2- 172.21.15.117:0/111032219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565b0dc6490 0x5565b11a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.700+0000 7fe9428cb740 1 --2- 172.21.15.117:0/111032219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565b0dadcd0 0x5565b11a5ac0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.700+0000 7fe9428cb740 1 -- 172.21.15.117:0/111032219 >> 172.21.15.117:0/111032219 conn(0x5565b0de3740 msgr2=0x5565b119d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:31.772 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.700+0000 7fe9428cb740 1 -- 172.21.15.117:0/111032219 shutdown_connections 2024-09-21T05:59:31.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.700+0000 7fe9428cb740 1 -- 172.21.15.117:0/111032219 wait complete. 2024-09-21T05:59:31.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.701+0000 7fe9428cb740 1 -- start start 2024-09-21T05:59:31.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.701+0000 7fe9428cb740 1 Processor -- start 2024-09-21T05:59:31.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.701+0000 7fe9428cb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565b0dadcd0 0x5565b11b89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:31.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.701+0000 7fe9428cb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565b0dc6490 0x5565b11b8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:31.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.702+0000 7fe9428cb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565b0dce670 0x5565b121e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:31.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.702+0000 7fe9428cb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341ae7d0 con 0x5565b0dc6490 2024-09-21T05:59:31.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.702+0000 7fe9398f2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565b0dc6490 0x5565b11b8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:31.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.702+0000 7fe9428cb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341ae100 con 0x5565b0dce670 2024-09-21T05:59:31.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.702+0000 7fe9428cb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341ab3f0 con 0x5565b0dadcd0 2024-09-21T05:59:31.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.702+0000 7fe9398f2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565b0dc6490 0x5565b11b8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53036/0 (socket says 172.21.15.117:53036) 2024-09-21T05:59:31.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.702+0000 7fe9398f2640 1 -- 172.21.15.117:0/3985556067 learned_addr learned my addr 172.21.15.117:0/3985556067 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:31.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.702+0000 7fe9428cb740 10 client.0 _ll_register_callbacks cb 0x7fe92c009540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:59:31.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.703+0000 7fe9398f2640 1 -- 172.21.15.117:0/3985556067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565b0dadcd0 msgr2=0x5565b11b89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:31.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.703+0000 7fe9398f2640 1 --2- 172.21.15.117:0/3985556067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565b0dadcd0 0x5565b11b89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.703+0000 7fe9398f2640 1 -- 172.21.15.117:0/3985556067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565b0dce670 msgr2=0x5565b121e440 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T05:59:31.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.703+0000 7fe9398f2640 1 --2- 172.21.15.117:0/3985556067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565b0dce670 0x5565b121e440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.703+0000 7fe9398f2640 1 -- 172.21.15.117:0/3985556067 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565b121e9d0 con 0x5565b0dc6490 2024-09-21T05:59:31.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.703+0000 7fe9398f2640 1 --2- 172.21.15.117:0/3985556067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565b0dc6490 0x5565b11b8f30 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe9240aa3d0 tx=0x7fe924000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c4ce2816635cb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:31.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.703+0000 7fe91dffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:59:31.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.704+0000 7fe91dffb640 1 -- 172.21.15.117:0/3985556067 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9240d0040 con 0x5565b0dc6490 2024-09-21T05:59:31.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.704+0000 7fe91dffb640 1 -- 172.21.15.117:0/3985556067 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fe9240ac3c0 con 0x5565b0dc6490 2024-09-21T05:59:31.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.704+0000 7fe9428cb740 1 -- 172.21.15.117:0/3985556067 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe930003aa0 con 0x5565b0dc6490 2024-09-21T05:59:31.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.704+0000 7fe91dffb640 1 -- 172.21.15.117:0/3985556067 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe9240ad860 con 0x5565b0dc6490 2024-09-21T05:59:31.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.704+0000 7fe9428cb740 10 client.16011 fetch_fsmap learned FSMap version 27 2024-09-21T05:59:31.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.705+0000 7fe9428cb740 1 -- 172.21.15.117:0/3985556067 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5565b121fcf0 con 0x5565b0dc6490 2024-09-21T05:59:31.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.705+0000 7fe91dffb640 1 -- 172.21.15.117:0/3985556067 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe930003aa0 con 0x5565b0dc6490 2024-09-21T05:59:31.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.705+0000 7fe91dffb640 1 -- 172.21.15.117:0/3985556067 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe92400b040 con 0x5565b0dc6490 2024-09-21T05:59:31.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.706+0000 7fe9428cb740 10 client.16011 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:59:31.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.706+0000 7fe9428cb740 10 client.16011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T05:59:31.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.706+0000 7fe9428cb740 20 client.16011 populate_metadata read hostname 'smithi117' 2024-09-21T05:59:31.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.706+0000 7fe9428cb740 1 --2- 172.21.15.117:0/3985556067 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5565b0def350 0x5565b1222500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:31.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.706+0000 7fe9428cb740 4 client.16011 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T05:59:31.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.706+0000 7fe9428cb740 1 -- 172.21.15.117:0/3985556067 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5565b110edb0 con 0x5565b0def350 2024-09-21T05:59:31.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.709+0000 7fe93a0f3640 1 --2- 172.21.15.117:0/3985556067 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5565b0def350 0x5565b1222500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:31.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.711+0000 7fe91dffb640 10 client.16011 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:59:31.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.711+0000 7fe93a0f3640 1 --2- 172.21.15.117:0/3985556067 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5565b0def350 0x5565b1222500 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa02d83e397e5027 server_cookie=eac4306fc182d1ad in_seq=0 out_seq=0 2024-09-21T05:59:31.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.951+0000 7fe91dffb640 1 -- 172.21.15.117:0/3985556067 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5565b110edb0 con 0x5565b0def350 2024-09-21T05:59:31.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.952+0000 7fe91dffb640 10 client.16011 handle_command_reply: tid=0 2024-09-21T05:59:31.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.952+0000 7fe9428cb740 1 client.16011 shutdown 2024-09-21T05:59:31.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.952+0000 7fe9428cb740 1 -- 172.21.15.117:0/3985556067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565b0dc6490 msgr2=0x5565b11b8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:31.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.952+0000 7fe9428cb740 1 --2- 172.21.15.117:0/3985556067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565b0dc6490 0x5565b11b8f30 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe9240aa3d0 tx=0x7fe924000bc0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.952+0000 7fe9428cb740 1 -- 172.21.15.117:0/3985556067 shutdown_connections 2024-09-21T05:59:31.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.952+0000 7fe9428cb740 1 --2- 172.21.15.117:0/3985556067 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5565b0def350 0x5565b1222500 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.952+0000 7fe9428cb740 1 --2- 172.21.15.117:0/3985556067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5565b0dc6490 0x5565b11b8f30 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.953+0000 7fe91dffb640 0 client.16011 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:59:31.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.953+0000 7fe9428cb740 1 --2- 172.21.15.117:0/3985556067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5565b0dce670 0x5565b121e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.953+0000 7fe9428cb740 1 --2- 172.21.15.117:0/3985556067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565b0dadcd0 0x5565b11b89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.953+0000 7fe9428cb740 1 -- 172.21.15.117:0/3985556067 >> 172.21.15.117:0/3985556067 conn(0x5565b0de3740 msgr2=0x5565b1198000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:31.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.954+0000 7fe9428cb740 1 -- 172.21.15.117:0/3985556067 shutdown_connections 2024-09-21T05:59:31.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.954+0000 7fe9428cb740 1 -- 172.21.15.117:0/3985556067 wait complete. 2024-09-21T05:59:31.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.954+0000 7fe9428cb740 20 client.16011 trim_cache size 0 max 16384 2024-09-21T05:59:31.958 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T05:59:31.959 INFO:teuthology.orchestra.run.smithi117.stdout: "return_code": 0, 2024-09-21T05:59:31.959 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_tag": "ad80d9f1-6aff-46c0-91ab-2c1ab0681039", 2024-09-21T05:59:31.959 INFO:teuthology.orchestra.run.smithi117.stdout: "mode": "asynchronous" 2024-09-21T05:59:31.959 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T05:59:31.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.956+0000 7fe93c357640 1 -- 172.21.15.117:0/1248167518 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe9200b6610 msgr2=0x7fe9200b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:31.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.956+0000 7fe93c357640 1 --2- 172.21.15.117:0/1248167518 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe9200b6610 0x7fe9200b8b20 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe92c005c40 tx=0x7fe92c005bd0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.956+0000 7fe93c357640 1 -- 172.21.15.117:0/1248167518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe934107b30 msgr2=0x7fe934198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:31.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.956+0000 7fe93c357640 1 --2- 172.21.15.117:0/1248167518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe934107b30 0x7fe934198840 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe92400db90 tx=0x7fe92400a620 comp rx=0 tx=0).stop 2024-09-21T05:59:31.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.956+0000 7fe93c357640 1 -- 172.21.15.117:0/1248167518 shutdown_connections 2024-09-21T05:59:31.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.956+0000 7fe93c357640 1 --2- 172.21.15.117:0/1248167518 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe9200b6610 0x7fe9200b8b20 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.956+0000 7fe93c357640 1 --2- 172.21.15.117:0/1248167518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe934107b30 0x7fe934198840 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.956+0000 7fe93c357640 1 --2- 172.21.15.117:0/1248167518 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe934108480 0x7fe93419dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.956+0000 7fe93c357640 1 --2- 172.21.15.117:0/1248167518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9341021f0 0x7fe9341982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:31.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.956+0000 7fe93c357640 1 -- 172.21.15.117:0/1248167518 >> 172.21.15.117:0/1248167518 conn(0x7fe934077c60 msgr2=0x7fe934105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:31.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.957+0000 7fe93c357640 1 -- 172.21.15.117:0/1248167518 shutdown_connections 2024-09-21T05:59:31.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:31.957+0000 7fe93c357640 1 -- 172.21.15.117:0/1248167518 wait complete. 2024-09-21T05:59:31.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:31 smithi192 ceph-mon[34700]: pgmap v393: 97 pgs: 97 active+clean; 20 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 844 MiB/s wr, 1.72k op/s 2024-09-21T05:59:32.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:31 smithi138 ceph-mon[34391]: pgmap v393: 97 pgs: 97 active+clean; 20 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 844 MiB/s wr, 1.72k op/s 2024-09-21T05:59:32.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:32 smithi192 ceph-mon[34700]: scrub queued for path: / 2024-09-21T05:59:32.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:32 smithi192 ceph-mon[34700]: scrub summary: idle+waiting paths [/] 2024-09-21T05:59:32.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:32 smithi192 ceph-mon[34700]: scrub summary: active paths [/] 2024-09-21T05:59:33.032 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:32 smithi117 ceph-mon[30947]: scrub queued for path: / 2024-09-21T05:59:33.032 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:32 smithi117 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-21T05:59:33.032 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:32 smithi117 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-21T05:59:33.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:32 smithi138 ceph-mon[34391]: scrub queued for path: / 2024-09-21T05:59:33.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:32 smithi138 ceph-mon[34391]: scrub summary: idle+waiting paths [/] 2024-09-21T05:59:33.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:32 smithi138 ceph-mon[34391]: scrub summary: active paths [/] 2024-09-21T05:59:33.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:33 smithi192 ceph-mon[34700]: pgmap v394: 97 pgs: 97 active+clean; 20 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 929 MiB/s wr, 1.85k op/s 2024-09-21T05:59:34.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:33 smithi117 ceph-mon[30947]: pgmap v394: 97 pgs: 97 active+clean; 20 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 929 MiB/s wr, 1.85k op/s 2024-09-21T05:59:34.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:33 smithi138 ceph-mon[34391]: pgmap v394: 97 pgs: 97 active+clean; 20 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 929 MiB/s wr, 1.85k op/s 2024-09-21T05:59:34.296 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T05:59:35.151 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T05:59:35.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:35 smithi192 ceph-mon[34700]: pgmap v395: 97 pgs: 97 active+clean; 20 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 986 MiB/s wr, 2.04k op/s 2024-09-21T05:59:36.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:35 smithi117 ceph-mon[30947]: pgmap v395: 97 pgs: 97 active+clean; 20 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 986 MiB/s wr, 2.04k op/s 2024-09-21T05:59:36.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:35 smithi138 ceph-mon[34391]: pgmap v395: 97 pgs: 97 active+clean; 20 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 986 MiB/s wr, 2.04k op/s 2024-09-21T05:59:37.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:37 smithi192 ceph-mon[34700]: pgmap v396: 97 pgs: 97 active+clean; 20 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 910 MiB/s wr, 1.89k op/s 2024-09-21T05:59:37.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:37 smithi192 ceph-mon[34700]: scrub summary: idle+waiting paths [/] 2024-09-21T05:59:37.990 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:37 smithi117 ceph-mon[30947]: pgmap v396: 97 pgs: 97 active+clean; 20 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 910 MiB/s wr, 1.89k op/s 2024-09-21T05:59:37.990 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:37 smithi117 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-21T05:59:38.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:37.986+0000 7f4ba1671640 1 -- 172.21.15.117:0/3550786072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b9c10a9e0 msgr2=0x7f4b9c10adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:38.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:37.986+0000 7f4ba1671640 1 --2- 172.21.15.117:0/3550786072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b9c10a9e0 0x7f4b9c10adf0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4b90009900 tx=0x7f4b9002a440 comp rx=0 tx=0).stop 2024-09-21T05:59:38.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:37.987+0000 7f4ba1671640 1 -- 172.21.15.117:0/3550786072 shutdown_connections 2024-09-21T05:59:38.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:37.987+0000 7f4ba1671640 1 --2- 172.21.15.117:0/3550786072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b9c10a9e0 0x7f4b9c10adf0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:37.987+0000 7f4ba1671640 1 --2- 172.21.15.117:0/3550786072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b9c11cc30 0x7f4b9c11f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:37.987+0000 7f4ba1671640 1 --2- 172.21.15.117:0/3550786072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b9c111f40 0x7f4b9c10a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:37.987+0000 7f4ba1671640 1 -- 172.21.15.117:0/3550786072 >> 172.21.15.117:0/3550786072 conn(0x7f4b9c070ec0 msgr2=0x7f4b9c0712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:38.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:37.991+0000 7f4ba1671640 1 -- 172.21.15.117:0/3550786072 shutdown_connections 2024-09-21T05:59:38.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:37.992+0000 7f4ba1671640 1 -- 172.21.15.117:0/3550786072 wait complete. 2024-09-21T05:59:38.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:37.993+0000 7f4ba1671640 1 Processor -- start 2024-09-21T05:59:38.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.002+0000 7f4ba1671640 1 -- start start 2024-09-21T05:59:38.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.003+0000 7f4ba1671640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b9c10a9e0 0x7f4b9c1a92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:38.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.003+0000 7f4ba1671640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b9c111f40 0x7f4b9c1a9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:38.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.003+0000 7f4ba1671640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b9c11cc30 0x7f4b9c1aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:38.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.003+0000 7f4ba1671640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c122900 con 0x7f4b9c111f40 2024-09-21T05:59:38.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.003+0000 7f4ba1671640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c122780 con 0x7f4b9c10a9e0 2024-09-21T05:59:38.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.003+0000 7f4ba1671640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c122a80 con 0x7f4b9c11cc30 2024-09-21T05:59:38.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.006+0000 7f4b9affd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b9c10a9e0 0x7f4b9c1a92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:38.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.007+0000 7f4b9affd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b9c10a9e0 0x7f4b9c1a92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:42346/0 (socket says 172.21.15.117:42346) 2024-09-21T05:59:38.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.007+0000 7f4b9affd640 1 -- 172.21.15.117:0/4057694623 learned_addr learned my addr 172.21.15.117:0/4057694623 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:38.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.010+0000 7f4b9b7fe640 1 --2- 172.21.15.117:0/4057694623 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b9c11cc30 0x7f4b9c1aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:38.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.011+0000 7f4b9a7fc640 1 --2- 172.21.15.117:0/4057694623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b9c111f40 0x7f4b9c1a9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:38.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.012+0000 7f4b9affd640 1 -- 172.21.15.117:0/4057694623 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b9c11cc30 msgr2=0x7f4b9c1aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:38.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.012+0000 7f4b9affd640 1 --2- 172.21.15.117:0/4057694623 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b9c11cc30 0x7f4b9c1aeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.012+0000 7f4b9affd640 1 -- 172.21.15.117:0/4057694623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b9c111f40 msgr2=0x7f4b9c1a9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:38.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.012+0000 7f4b9affd640 1 --2- 172.21.15.117:0/4057694623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b9c111f40 0x7f4b9c1a9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.012+0000 7f4b9affd640 1 -- 172.21.15.117:0/4057694623 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b9c1af220 con 0x7f4b9c10a9e0 2024-09-21T05:59:38.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.023+0000 7f4b9affd640 1 --2- 172.21.15.117:0/4057694623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b9c10a9e0 0x7f4b9c1a92b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4b9400eab0 tx=0x7f4b9400ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:38.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.026+0000 7f4b83fff640 1 -- 172.21.15.117:0/4057694623 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b9400ce40 con 0x7f4b9c10a9e0 2024-09-21T05:59:38.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.026+0000 7f4b83fff640 1 -- 172.21.15.117:0/4057694623 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b94011920 con 0x7f4b9c10a9e0 2024-09-21T05:59:38.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.026+0000 7f4b83fff640 1 -- 172.21.15.117:0/4057694623 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b94010a30 con 0x7f4b9c10a9e0 2024-09-21T05:59:38.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.029+0000 7f4ba1671640 1 -- 172.21.15.117:0/4057694623 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b9c1af510 con 0x7f4b9c10a9e0 2024-09-21T05:59:38.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.029+0000 7f4ba1671640 1 -- 172.21.15.117:0/4057694623 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b9c1b5c70 con 0x7f4b9c10a9e0 2024-09-21T05:59:38.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.033+0000 7f4ba7be5740 1 Processor -- start 2024-09-21T05:59:38.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.034+0000 7f4ba7be5740 1 -- start start 2024-09-21T05:59:38.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.035+0000 7f4ba7be5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1b59f1070 0x55f1b5d98270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:38.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.035+0000 7f4ba7be5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1b59c81b0 0x55f1b5d987b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:38.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.035+0000 7f4ba7be5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b59d0390 0x55f1b5d98cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:38.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.035+0000 7f4ba7be5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f1b5d0ee50 con 0x55f1b59d0390 2024-09-21T05:59:38.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.035+0000 7f4ba7be5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f1b5cfe280 con 0x55f1b59c81b0 2024-09-21T05:59:38.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.035+0000 7f4ba7be5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f1b5d02090 con 0x55f1b59f1070 2024-09-21T05:59:38.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.035+0000 7f4b9a7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b59d0390 0x55f1b5d98cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:38.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.036+0000 7f4b9a7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b59d0390 0x55f1b5d98cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33874/0 (socket says 172.21.15.117:33874) 2024-09-21T05:59:38.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.036+0000 7f4b9a7fc640 1 -- 172.21.15.117:0/1935809674 learned_addr learned my addr 172.21.15.117:0/1935809674 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:38.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.036+0000 7f4b9b7fe640 1 --2- 172.21.15.117:0/1935809674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1b59f1070 0x55f1b5d98270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:38.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.036+0000 7f4b9affd640 1 --2- 172.21.15.117:0/1935809674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1b59c81b0 0x55f1b5d987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:38.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.042+0000 7f4b83fff640 1 -- 172.21.15.117:0/4057694623 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f4b94010bd0 con 0x7f4b9c10a9e0 2024-09-21T05:59:38.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.043+0000 7f4b9b7fe640 1 -- 172.21.15.117:0/1935809674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1b59c81b0 msgr2=0x55f1b5d987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:38.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.043+0000 7f4b9b7fe640 1 --2- 172.21.15.117:0/1935809674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1b59c81b0 0x55f1b5d987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.043+0000 7f4b9b7fe640 1 -- 172.21.15.117:0/1935809674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b59d0390 msgr2=0x55f1b5d98cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:38.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.043+0000 7f4b9b7fe640 1 --2- 172.21.15.117:0/1935809674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b59d0390 0x55f1b5d98cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.043+0000 7f4b9b7fe640 1 -- 172.21.15.117:0/1935809674 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1b5a5c650 con 0x55f1b59f1070 2024-09-21T05:59:38.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.044+0000 7f4b83fff640 1 --2- 172.21.15.117:0/4057694623 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4b840b6730 0x7f4b840b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:38.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.045+0000 7f4b83fff640 1 -- 172.21.15.117:0/4057694623 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f4b940ecdf0 con 0x7f4b9c10a9e0 2024-09-21T05:59:38.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.047+0000 7f4b9b7fe640 1 --2- 172.21.15.117:0/1935809674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1b59f1070 0x55f1b5d98270 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c0045c0 tx=0x7f4b8c00ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=426a77472dd11a3f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:38.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.048+0000 7f4b9affd640 1 --2- 172.21.15.117:0/4057694623 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4b840b6730 0x7f4b840b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:38.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.049+0000 7f4b9affd640 1 --2- 172.21.15.117:0/4057694623 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4b840b6730 0x7f4b840b8c40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4b94010510 tx=0x7f4b940094a0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:38.096 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.051+0000 7f4b817fa640 1 -- 172.21.15.117:0/1935809674 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c003a70 con 0x55f1b59f1070 2024-09-21T05:59:38.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.051+0000 7f4b817fa640 1 -- 172.21.15.117:0/1935809674 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b8c015070 con 0x55f1b59f1070 2024-09-21T05:59:38.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.053+0000 7f4ba7be5740 1 -- 172.21.15.117:0/1935809674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1b59f1070 msgr2=0x55f1b5d98270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:38.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.053+0000 7f4ba7be5740 1 --2- 172.21.15.117:0/1935809674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1b59f1070 0x55f1b5d98270 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c0045c0 tx=0x7f4b8c00ba90 comp rx=0 tx=0).stop 2024-09-21T05:59:38.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.057+0000 7f4ba7be5740 1 -- 172.21.15.117:0/1935809674 shutdown_connections 2024-09-21T05:59:38.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.057+0000 7f4ba7be5740 1 --2- 172.21.15.117:0/1935809674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b59d0390 0x55f1b5d98cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.057+0000 7f4ba7be5740 1 --2- 172.21.15.117:0/1935809674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1b59c81b0 0x55f1b5d987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.057+0000 7f4ba7be5740 1 --2- 172.21.15.117:0/1935809674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1b59f1070 0x55f1b5d98270 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.057+0000 7f4ba7be5740 1 -- 172.21.15.117:0/1935809674 >> 172.21.15.117:0/1935809674 conn(0x55f1b59e5460 msgr2=0x55f1b5d96160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:38.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.057+0000 7f4ba7be5740 1 -- 172.21.15.117:0/1935809674 shutdown_connections 2024-09-21T05:59:38.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.058+0000 7f4ba7be5740 1 -- 172.21.15.117:0/1935809674 wait complete. 2024-09-21T05:59:38.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.058+0000 7f4ba7be5740 1 -- start start 2024-09-21T05:59:38.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.059+0000 7f4ba7be5740 1 Processor -- start 2024-09-21T05:59:38.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.060+0000 7f4ba7be5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1b59c81b0 0x55f1b5da37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:38.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.060+0000 7f4ba7be5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b59d0390 0x55f1b5da3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:38.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.060+0000 7f4ba7be5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1b59f1070 0x55f1b5da4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:38.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.060+0000 7f4ba7be5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f1b5b5f700 con 0x55f1b59d0390 2024-09-21T05:59:38.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.060+0000 7f4ba7be5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f1b5c99110 con 0x55f1b59c81b0 2024-09-21T05:59:38.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.060+0000 7f4ba7be5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f1b5da6af0 con 0x55f1b59f1070 2024-09-21T05:59:38.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.060+0000 7f4ba7be5740 10 client.0 _ll_register_callbacks cb 0x55f1b5cb2990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:59:38.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.064+0000 7f4b9b7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1b59f1070 0x55f1b5da4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:38.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.064+0000 7f4b9b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1b59c81b0 0x55f1b5da37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:38.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.064+0000 7f4b9b7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1b59f1070 0x55f1b5da4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:42886/0 (socket says 172.21.15.117:42886) 2024-09-21T05:59:38.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.064+0000 7f4b9b7fe640 1 -- 172.21.15.117:0/2500135403 learned_addr learned my addr 172.21.15.117:0/2500135403 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:38.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.066+0000 7f4b9a7fc640 1 --2- 172.21.15.117:0/2500135403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b59d0390 0x55f1b5da3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:38.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.071+0000 7f4b9a7fc640 1 -- 172.21.15.117:0/2500135403 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1b59f1070 msgr2=0x55f1b5da4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:38.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.071+0000 7f4b9a7fc640 1 --2- 172.21.15.117:0/2500135403 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1b59f1070 0x55f1b5da4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.071+0000 7f4b9a7fc640 1 -- 172.21.15.117:0/2500135403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1b59c81b0 msgr2=0x55f1b5da37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:38.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.071+0000 7f4b9a7fc640 1 --2- 172.21.15.117:0/2500135403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1b59c81b0 0x55f1b5da37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.071+0000 7f4b9a7fc640 1 -- 172.21.15.117:0/2500135403 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1b5cc6c40 con 0x55f1b59d0390 2024-09-21T05:59:38.098 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.078+0000 7f4b9a7fc640 1 --2- 172.21.15.117:0/2500135403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b59d0390 0x55f1b5da3d10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4b90000c00 tx=0x7f4b90031670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87487ec0b3d37b74 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:38.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.078+0000 7f4b7eff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:59:38.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.084+0000 7f4b7eff5640 1 -- 172.21.15.117:0/2500135403 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b90031bf0 con 0x55f1b59d0390 2024-09-21T05:59:38.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.084+0000 7f4ba7be5740 1 -- 172.21.15.117:0/2500135403 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b8c015070 con 0x55f1b59d0390 2024-09-21T05:59:38.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.084+0000 7f4b7eff5640 1 -- 172.21.15.117:0/2500135403 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f4b90031d90 con 0x55f1b59d0390 2024-09-21T05:59:38.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.088+0000 7f4b7eff5640 1 -- 172.21.15.117:0/2500135403 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4b90047440 con 0x55f1b59d0390 2024-09-21T05:59:38.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.088+0000 7f4ba7be5740 10 client.16023 fetch_fsmap learned FSMap version 27 2024-09-21T05:59:38.099 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.088+0000 7f4ba7be5740 1 -- 172.21.15.117:0/2500135403 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f1b5dc3750 con 0x55f1b59d0390 2024-09-21T05:59:38.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.099+0000 7f4b7eff5640 1 -- 172.21.15.117:0/2500135403 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b8c015070 con 0x55f1b59d0390 2024-09-21T05:59:38.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.100+0000 7f4b7eff5640 1 -- 172.21.15.117:0/2500135403 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4b90045070 con 0x55f1b59d0390 2024-09-21T05:59:38.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.101+0000 7f4ba7be5740 10 client.16023 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:59:38.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.101+0000 7f4ba7be5740 10 client.16023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T05:59:38.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.101+0000 7f4ba7be5740 20 client.16023 populate_metadata read hostname 'smithi117' 2024-09-21T05:59:38.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.101+0000 7f4ba7be5740 1 --2- 172.21.15.117:0/2500135403 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f1b59af9f0 0x55f1b5dc7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:38.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.101+0000 7f4ba7be5740 4 client.16023 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:59:38.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.101+0000 7f4ba7be5740 1 -- 172.21.15.117:0/2500135403 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1b5d10da0 con 0x55f1b59af9f0 2024-09-21T05:59:38.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.111+0000 7f4b9b7fe640 1 --2- 172.21.15.117:0/2500135403 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f1b59af9f0 0x55f1b5dc7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:38.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:37 smithi138 ceph-mon[34391]: pgmap v396: 97 pgs: 97 active+clean; 20 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 910 MiB/s wr, 1.89k op/s 2024-09-21T05:59:38.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:37 smithi138 ceph-mon[34391]: scrub summary: idle+waiting paths [/] 2024-09-21T05:59:38.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.118+0000 7f4b9b7fe640 1 --2- 172.21.15.117:0/2500135403 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f1b59af9f0 0x55f1b5dc7010 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b374b37897b7ee3 server_cookie=85e2b504b62bf5ad in_seq=0 out_seq=0 2024-09-21T05:59:38.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.118+0000 7f4b7eff5640 10 client.16023 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:59:38.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.121+0000 7f4b7eff5640 1 -- 172.21.15.117:0/2500135403 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f1b5d10da0 con 0x55f1b59af9f0 2024-09-21T05:59:38.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.121+0000 7f4b7eff5640 10 client.16023 handle_command_reply: tid=0 2024-09-21T05:59:38.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.121+0000 7f4ba7be5740 1 client.16023 shutdown 2024-09-21T05:59:38.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.122+0000 7f4ba7be5740 1 -- 172.21.15.117:0/2500135403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b59d0390 msgr2=0x55f1b5da3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:38.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.122+0000 7f4ba7be5740 1 --2- 172.21.15.117:0/2500135403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b59d0390 0x55f1b5da3d10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4b90000c00 tx=0x7f4b90031670 comp rx=0 tx=0).stop 2024-09-21T05:59:38.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.125+0000 7f4ba7be5740 1 -- 172.21.15.117:0/2500135403 shutdown_connections 2024-09-21T05:59:38.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.125+0000 7f4ba7be5740 1 --2- 172.21.15.117:0/2500135403 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f1b59af9f0 0x55f1b5dc7010 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.125+0000 7f4ba7be5740 1 --2- 172.21.15.117:0/2500135403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b59d0390 0x55f1b5da3d10 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.125+0000 7f4ba7be5740 1 --2- 172.21.15.117:0/2500135403 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1b59f1070 0x55f1b5da4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.125+0000 7f4ba7be5740 1 --2- 172.21.15.117:0/2500135403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1b59c81b0 0x55f1b5da37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.125+0000 7f4ba7be5740 1 -- 172.21.15.117:0/2500135403 >> 172.21.15.117:0/2500135403 conn(0x55f1b59e5460 msgr2=0x55f1b5d9b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:38.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.125+0000 7f4b7eff5640 0 client.16023 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:59:38.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.126+0000 7f4ba7be5740 1 -- 172.21.15.117:0/2500135403 shutdown_connections 2024-09-21T05:59:38.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.128+0000 7f4ba7be5740 1 -- 172.21.15.117:0/2500135403 wait complete. 2024-09-21T05:59:38.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.129+0000 7f4ba7be5740 20 client.16023 trim_cache size 0 max 16384 2024-09-21T05:59:38.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.175+0000 7f4ba7be5740 1 Processor -- start 2024-09-21T05:59:38.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.175+0000 7f4ba7be5740 1 -- start start 2024-09-21T05:59:38.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.176+0000 7f4ba7be5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b59c81b0 0x55f1b5daf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:38.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.176+0000 7f4ba7be5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1b59d0390 0x55f1b5d95710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:38.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.176+0000 7f4ba7be5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1b59f1070 0x55f1b5d95c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:38.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.176+0000 7f4ba7be5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c11a890 con 0x55f1b59c81b0 2024-09-21T05:59:38.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.176+0000 7f4ba7be5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c1b7690 con 0x55f1b59f1070 2024-09-21T05:59:38.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.176+0000 7f4ba7be5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c1b6220 con 0x55f1b59d0390 2024-09-21T05:59:38.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.178+0000 7f4b9a7fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1b59d0390 0x55f1b5d95710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:38.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.178+0000 7f4b9a7fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1b59d0390 0x55f1b5d95710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:42896/0 (socket says 172.21.15.117:42896) 2024-09-21T05:59:38.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.178+0000 7f4b9a7fc640 1 -- 172.21.15.117:0/3110003600 learned_addr learned my addr 172.21.15.117:0/3110003600 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:38.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.179+0000 7f4b9b7fe640 1 --2- 172.21.15.117:0/3110003600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b59c81b0 0x55f1b5daf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:38.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.179+0000 7f4b9b7fe640 1 --2- 172.21.15.117:0/3110003600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1b59f1070 0x55f1b5d95c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:38.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.182+0000 7f4b9b7fe640 1 -- 172.21.15.117:0/3110003600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1b59d0390 msgr2=0x55f1b5d95710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:38.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.182+0000 7f4b9b7fe640 1 --2- 172.21.15.117:0/3110003600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1b59d0390 0x55f1b5d95710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.182+0000 7f4b9b7fe640 1 -- 172.21.15.117:0/3110003600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1b59f1070 msgr2=0x55f1b5d95c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:38.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.182+0000 7f4b9b7fe640 1 --2- 172.21.15.117:0/3110003600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1b59f1070 0x55f1b5d95c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.182+0000 7f4b9b7fe640 1 -- 172.21.15.117:0/3110003600 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1b5da2730 con 0x55f1b59c81b0 2024-09-21T05:59:38.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.183+0000 7f4b9b7fe640 1 --2- 172.21.15.117:0/3110003600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b59c81b0 0x55f1b5daf430 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x55f1b5dc7750 tx=0x7f4b8c00b2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddba582ff30181cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:38.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.186+0000 7f4b80ff9640 1 -- 172.21.15.117:0/3110003600 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c026040 con 0x55f1b59c81b0 2024-09-21T05:59:38.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.186+0000 7f4b80ff9640 1 -- 172.21.15.117:0/3110003600 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b8c015040 con 0x55f1b59c81b0 2024-09-21T05:59:38.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.186+0000 7f4b80ff9640 1 -- 172.21.15.117:0/3110003600 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c005280 con 0x55f1b59c81b0 2024-09-21T05:59:38.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.188+0000 7f4ba7be5740 1 -- 172.21.15.117:0/3110003600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b59c81b0 msgr2=0x55f1b5daf430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:38.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.188+0000 7f4ba7be5740 1 --2- 172.21.15.117:0/3110003600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b59c81b0 0x55f1b5daf430 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x55f1b5dc7750 tx=0x7f4b8c00b2e0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.189+0000 7f4ba7be5740 1 -- 172.21.15.117:0/3110003600 shutdown_connections 2024-09-21T05:59:38.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.189+0000 7f4ba7be5740 1 --2- 172.21.15.117:0/3110003600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1b59f1070 0x55f1b5d95c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.189+0000 7f4ba7be5740 1 --2- 172.21.15.117:0/3110003600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1b59d0390 0x55f1b5d95710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.189+0000 7f4ba7be5740 1 --2- 172.21.15.117:0/3110003600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b59c81b0 0x55f1b5daf430 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.189+0000 7f4ba7be5740 1 -- 172.21.15.117:0/3110003600 >> 172.21.15.117:0/3110003600 conn(0x55f1b59e5460 msgr2=0x55f1b5daed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:38.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.191+0000 7f4ba7be5740 1 -- 172.21.15.117:0/3110003600 shutdown_connections 2024-09-21T05:59:38.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.191+0000 7f4ba7be5740 1 -- 172.21.15.117:0/3110003600 wait complete. 2024-09-21T05:59:38.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.192+0000 7f4ba7be5740 1 -- start start 2024-09-21T05:59:38.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.192+0000 7f4ba7be5740 1 Processor -- start 2024-09-21T05:59:38.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.193+0000 7f4ba7be5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1b59c81b0 0x55f1b5dac1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:38.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.193+0000 7f4ba7be5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b59d0390 0x55f1b5dac790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:38.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.193+0000 7f4ba7be5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1b59f1070 0x55f1b5da7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:38.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.193+0000 7f4ba7be5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c1b6fc0 con 0x55f1b59d0390 2024-09-21T05:59:38.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.193+0000 7f4ba7be5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c1b68f0 con 0x55f1b59c81b0 2024-09-21T05:59:38.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.193+0000 7f4ba7be5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c1aa3e0 con 0x55f1b59f1070 2024-09-21T05:59:38.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.194+0000 7f4ba7be5740 10 client.0 _ll_register_callbacks cb 0x7f4b90009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:59:38.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.196+0000 7f4b9b7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1b59f1070 0x55f1b5da7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:38.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.196+0000 7f4b9b7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1b59f1070 0x55f1b5da7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:42906/0 (socket says 172.21.15.117:42906) 2024-09-21T05:59:38.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.196+0000 7f4b9b7fe640 1 -- 172.21.15.117:0/4067451491 learned_addr learned my addr 172.21.15.117:0/4067451491 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:38.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.196+0000 7f4b9b7fe640 1 --2- 172.21.15.117:0/4067451491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1b59c81b0 0x55f1b5dac1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:38.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.197+0000 7f4b9b7fe640 1 -- 172.21.15.117:0/4067451491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1b59c81b0 msgr2=0x55f1b5dac1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:38.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.197+0000 7f4b9b7fe640 1 --2- 172.21.15.117:0/4067451491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1b59c81b0 0x55f1b5dac1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.197+0000 7f4b9b7fe640 1 -- 172.21.15.117:0/4067451491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b59d0390 msgr2=0x55f1b5dac790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T05:59:38.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.197+0000 7f4b9b7fe640 1 --2- 172.21.15.117:0/4067451491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b59d0390 0x55f1b5dac790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.197+0000 7f4b9b7fe640 1 -- 172.21.15.117:0/4067451491 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1b5da7fa0 con 0x55f1b59f1070 2024-09-21T05:59:38.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.204+0000 7f4b9b7fe640 1 --2- 172.21.15.117:0/4067451491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1b59f1070 0x55f1b5da7810 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c011b40 tx=0x7f4b8c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ea0b82378b1c29b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:38.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.205+0000 7f4b7eff5640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-21T05:59:38.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.210+0000 7f4b7eff5640 1 -- 172.21.15.117:0/4067451491 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c01aa80 con 0x55f1b59f1070 2024-09-21T05:59:38.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.210+0000 7f4ba7be5740 1 -- 172.21.15.117:0/4067451491 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b8c015040 con 0x55f1b59f1070 2024-09-21T05:59:38.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.210+0000 7f4b7eff5640 1 -- 172.21.15.117:0/4067451491 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f4b8c003870 con 0x55f1b59f1070 2024-09-21T05:59:38.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.215+0000 7f4b7eff5640 1 -- 172.21.15.117:0/4067451491 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4b8c003a70 con 0x55f1b59f1070 2024-09-21T05:59:38.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.218+0000 7f4ba7be5740 10 client.25438 fetch_fsmap learned FSMap version 27 2024-09-21T05:59:38.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.218+0000 7f4ba7be5740 1 -- 172.21.15.117:0/4067451491 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f1b5e315c0 con 0x55f1b59f1070 2024-09-21T05:59:38.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.222+0000 7f4b7eff5640 1 -- 172.21.15.117:0/4067451491 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b8c015040 con 0x55f1b59f1070 2024-09-21T05:59:38.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.222+0000 7f4b7eff5640 1 -- 172.21.15.117:0/4067451491 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4b8c0045a0 con 0x55f1b59f1070 2024-09-21T05:59:38.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.223+0000 7f4ba7be5740 10 client.25438 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:59:38.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.223+0000 7f4ba7be5740 10 client.25438 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T05:59:38.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.223+0000 7f4ba7be5740 20 client.25438 populate_metadata read hostname 'smithi117' 2024-09-21T05:59:38.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.223+0000 7f4ba7be5740 1 --2- 172.21.15.117:0/4067451491 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f1b5984da0 0x55f1b5e33a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:38.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.223+0000 7f4ba7be5740 4 client.25438 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T05:59:38.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.223+0000 7f4ba7be5740 1 -- 172.21.15.117:0/4067451491 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f1b5d0c850 con 0x55f1b5984da0 2024-09-21T05:59:38.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.224+0000 7f4b9a7fc640 1 --2- 172.21.15.117:0/4067451491 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f1b5984da0 0x55f1b5e33a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:38.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.237+0000 7f4b7eff5640 10 client.25438 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:59:38.239 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.237+0000 7f4b9a7fc640 1 --2- 172.21.15.117:0/4067451491 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f1b5984da0 0x55f1b5e33a10 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2cd0b769c25b859 server_cookie=5caefa7654c06f6c in_seq=0 out_seq=0 2024-09-21T05:59:38.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.240+0000 7f4b7eff5640 1 -- 172.21.15.117:0/4067451491 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1138 (crc 0 0 0) 0x55f1b5d0c850 con 0x55f1b5984da0 2024-09-21T05:59:38.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.240+0000 7f4b7eff5640 10 client.25438 handle_command_reply: tid=0 2024-09-21T05:59:38.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.241+0000 7f4ba7be5740 1 client.25438 shutdown 2024-09-21T05:59:38.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.242+0000 7f4ba7be5740 1 -- 172.21.15.117:0/4067451491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1b59f1070 msgr2=0x55f1b5da7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:38.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.242+0000 7f4ba7be5740 1 --2- 172.21.15.117:0/4067451491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1b59f1070 0x55f1b5da7810 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c011b40 tx=0x7f4b8c000bc0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.243+0000 7f4ba7be5740 1 -- 172.21.15.117:0/4067451491 shutdown_connections 2024-09-21T05:59:38.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.243+0000 7f4ba7be5740 1 --2- 172.21.15.117:0/4067451491 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f1b5984da0 0x55f1b5e33a10 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.243+0000 7f4ba7be5740 1 --2- 172.21.15.117:0/4067451491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1b59d0390 0x55f1b5dac790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.243+0000 7f4ba7be5740 1 --2- 172.21.15.117:0/4067451491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1b59f1070 0x55f1b5da7810 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.243+0000 7f4ba7be5740 1 --2- 172.21.15.117:0/4067451491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1b59c81b0 0x55f1b5dac1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.243+0000 7f4ba7be5740 1 -- 172.21.15.117:0/4067451491 >> 172.21.15.117:0/4067451491 conn(0x55f1b59e5460 msgr2=0x55f1b5d9b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:38.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.243+0000 7f4b7eff5640 0 client.25438 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:59:38.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.244+0000 7f4ba7be5740 1 -- 172.21.15.117:0/4067451491 shutdown_connections 2024-09-21T05:59:38.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.244+0000 7f4ba7be5740 1 -- 172.21.15.117:0/4067451491 wait complete. 2024-09-21T05:59:38.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.244+0000 7f4ba7be5740 20 client.25438 trim_cache size 0 max 16384 2024-09-21T05:59:38.248 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T05:59:38.248 INFO:teuthology.orchestra.run.smithi117.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T05:59:38.248 INFO:teuthology.orchestra.run.smithi117.stdout: "scrubs": { 2024-09-21T05:59:38.248 INFO:teuthology.orchestra.run.smithi117.stdout: "ad80d9f1-6aff-46c0-91ab-2c1ab0681039": { 2024-09-21T05:59:38.248 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T05:59:38.248 INFO:teuthology.orchestra.run.smithi117.stdout: "tag": "ad80d9f1-6aff-46c0-91ab-2c1ab0681039", 2024-09-21T05:59:38.248 INFO:teuthology.orchestra.run.smithi117.stdout: "options": "recursive,force" 2024-09-21T05:59:38.249 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T05:59:38.249 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T05:59:38.249 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_stats": { 2024-09-21T05:59:38.249 INFO:teuthology.orchestra.run.smithi117.stdout: "ad80d9f1-6aff-46c0-91ab-2c1ab0681039": { 2024-09-21T05:59:38.249 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:59:31.949108+0000", 2024-09-21T05:59:38.249 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T05:59:38.249 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T05:59:38.249 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T05:59:38.249 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T05:59:38.250 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T05:59:38.250 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T05:59:38.250 INFO:teuthology.orchestra.run.smithi117.stdout: "8b3fe95a-b76b-4e94-afd0-72fe35e2dd92": { 2024-09-21T05:59:38.250 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:58:01.246076+0000", 2024-09-21T05:59:38.250 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T05:59:38.250 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T05:59:38.250 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T05:59:38.251 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T05:59:38.251 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T05:59:38.251 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T05:59:38.251 INFO:teuthology.orchestra.run.smithi117.stdout: "b30bfff0-0f6b-4e30-8467-ff7db421eba6": { 2024-09-21T05:59:38.251 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:57:14.939082+0000", 2024-09-21T05:59:38.251 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T05:59:38.251 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T05:59:38.252 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T05:59:38.252 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T05:59:38.252 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T05:59:38.252 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T05:59:38.252 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T05:59:38.252 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T05:59:38.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.246+0000 7f4ba1671640 1 -- 172.21.15.117:0/4057694623 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4b840b6730 msgr2=0x7f4b840b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:38.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.246+0000 7f4ba1671640 1 --2- 172.21.15.117:0/4057694623 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4b840b6730 0x7f4b840b8c40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4b94010510 tx=0x7f4b940094a0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.247+0000 7f4ba1671640 1 -- 172.21.15.117:0/4057694623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b9c10a9e0 msgr2=0x7f4b9c1a92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:38.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.247+0000 7f4ba1671640 1 --2- 172.21.15.117:0/4057694623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b9c10a9e0 0x7f4b9c1a92b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4b9400eab0 tx=0x7f4b9400ef70 comp rx=0 tx=0).stop 2024-09-21T05:59:38.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.247+0000 7f4ba1671640 1 -- 172.21.15.117:0/4057694623 shutdown_connections 2024-09-21T05:59:38.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.247+0000 7f4ba1671640 1 --2- 172.21.15.117:0/4057694623 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4b840b6730 0x7f4b840b8c40 secure :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4b94010510 tx=0x7f4b940094a0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.247+0000 7f4ba1671640 1 --2- 172.21.15.117:0/4057694623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b9c111f40 0x7f4b9c1a9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.247+0000 7f4ba1671640 1 --2- 172.21.15.117:0/4057694623 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b9c11cc30 0x7f4b9c1aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:38.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.247+0000 7f4ba1671640 1 --2- 172.21.15.117:0/4057694623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b9c10a9e0 0x7f4b9c1a92b0 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4b9400eab0 tx=0x7f4b9400ef70 comp rx=0 tx=0).stop 2024-09-21T05:59:38.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.247+0000 7f4ba1671640 1 -- 172.21.15.117:0/4057694623 >> 172.21.15.117:0/4057694623 conn(0x7f4b9c070ec0 msgr2=0x7f4b9c071e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:38.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.249+0000 7f4ba1671640 1 -- 172.21.15.117:0/4057694623 shutdown_connections 2024-09-21T05:59:38.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T05:59:38.249+0000 7f4ba1671640 1 -- 172.21.15.117:0/4057694623 wait complete. 2024-09-21T05:59:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.797+0000 7f610e3c5640 1 -- 172.21.15.117:0/4258198688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f610810a3c0 msgr2=0x7f610810a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.797+0000 7f60fffff640 1 -- 172.21.15.117:0/4258198688 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6104035020 con 0x7f610810a3c0 2024-09-21T05:59:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.797+0000 7f610e3c5640 1 --2- 172.21.15.117:0/4258198688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f610810a3c0 0x7f610810a7d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6104009900 tx=0x7f610402a440 comp rx=0 tx=0).stop 2024-09-21T05:59:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.798+0000 7f610e3c5640 1 -- 172.21.15.117:0/4258198688 shutdown_connections 2024-09-21T05:59:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.798+0000 7f610e3c5640 1 --2- 172.21.15.117:0/4258198688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6108116510 0x7f6108118930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.798+0000 7f610e3c5640 1 --2- 172.21.15.117:0/4258198688 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f610810ad10 0x7f6108115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.798+0000 7f610e3c5640 1 --2- 172.21.15.117:0/4258198688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f610810a3c0 0x7f610810a7d0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.798+0000 7f610e3c5640 1 -- 172.21.15.117:0/4258198688 >> 172.21.15.117:0/4258198688 conn(0x7f6108070c00 msgr2=0x7f6108071010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:39.800 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.798+0000 7f610e3c5640 1 -- 172.21.15.117:0/4258198688 shutdown_connections 2024-09-21T05:59:39.801 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.798+0000 7f610e3c5640 1 -- 172.21.15.117:0/4258198688 wait complete. 2024-09-21T05:59:39.801 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.799+0000 7f610e3c5640 1 Processor -- start 2024-09-21T05:59:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.800+0000 7f610e3c5640 1 -- start start 2024-09-21T05:59:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.800+0000 7f610e3c5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f610810a3c0 0x7f61081a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.800+0000 7f610e3c5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f610810ad10 0x7f61081a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.800+0000 7f610e3c5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6108116510 0x7f61081a8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.800+0000 7f610e3c5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f610811c0d0 con 0x7f610810ad10 2024-09-21T05:59:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.800+0000 7f610e3c5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f610811bf50 con 0x7f610810a3c0 2024-09-21T05:59:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.800+0000 7f610e3c5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f610811c250 con 0x7f6108116510 2024-09-21T05:59:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.803+0000 7f610d3c3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f610810a3c0 0x7f61081a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.803+0000 7f610dbc4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6108116510 0x7f61081a8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.803+0000 7f610dbc4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6108116510 0x7f61081a8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:42926/0 (socket says 172.21.15.117:42926) 2024-09-21T05:59:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.803+0000 7f610d3c3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f610810a3c0 0x7f61081a15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:42394/0 (socket says 172.21.15.117:42394) 2024-09-21T05:59:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.803+0000 7f610d3c3640 1 -- 172.21.15.117:0/2754422528 learned_addr learned my addr 172.21.15.117:0/2754422528 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.805+0000 7f610cbc2640 1 --2- 172.21.15.117:0/2754422528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f610810ad10 0x7f61081a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.806+0000 7f610dbc4640 1 -- 172.21.15.117:0/2754422528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f610810a3c0 msgr2=0x7f61081a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.806+0000 7f610dbc4640 1 --2- 172.21.15.117:0/2754422528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f610810a3c0 0x7f61081a15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.806+0000 7f610dbc4640 1 -- 172.21.15.117:0/2754422528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f610810ad10 msgr2=0x7f61081a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.806+0000 7f610dbc4640 1 --2- 172.21.15.117:0/2754422528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f610810ad10 0x7f61081a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.806+0000 7f610dbc4640 1 -- 172.21.15.117:0/2754422528 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61081a8b40 con 0x7f6108116510 2024-09-21T05:59:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.810+0000 7f610dbc4640 1 --2- 172.21.15.117:0/2754422528 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6108116510 0x7f61081a8600 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f60f800c710 tx=0x7f60f800cbd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:39.814 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.812+0000 7f60fe7fc640 1 -- 172.21.15.117:0/2754422528 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f8004220 con 0x7f6108116510 2024-09-21T05:59:39.814 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.812+0000 7f610e3c5640 1 -- 172.21.15.117:0/2754422528 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61081a8e30 con 0x7f6108116510 2024-09-21T05:59:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.812+0000 7f610e3c5640 1 -- 172.21.15.117:0/2754422528 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61081a9380 con 0x7f6108116510 2024-09-21T05:59:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.813+0000 7f60fe7fc640 1 -- 172.21.15.117:0/2754422528 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60f800ce20 con 0x7f6108116510 2024-09-21T05:59:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.813+0000 7f60fe7fc640 1 -- 172.21.15.117:0/2754422528 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f8003d50 con 0x7f6108116510 2024-09-21T05:59:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.820+0000 7f611666d740 1 Processor -- start 2024-09-21T05:59:39.828 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.827+0000 7f611666d740 1 -- start start 2024-09-21T05:59:39.829 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.827+0000 7f611666d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e2581251b0 0x55e2584caba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.827+0000 7f611666d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2580fc2f0 0x55e2584cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.828+0000 7f611666d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2581044d0 0x55e2584cb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.828+0000 7f611666d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e25843f6f0 con 0x55e2581044d0 2024-09-21T05:59:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.828+0000 7f611666d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e25842eb20 con 0x55e2580fc2f0 2024-09-21T05:59:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.828+0000 7f611666d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55e258432930 con 0x55e2581251b0 2024-09-21T05:59:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.828+0000 7f60fe7fc640 1 -- 172.21.15.117:0/2754422528 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f60f800f470 con 0x7f6108116510 2024-09-21T05:59:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.829+0000 7f610cbc2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e2581251b0 0x55e2584caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.829+0000 7f610cbc2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e2581251b0 0x55e2584caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:42942/0 (socket says 172.21.15.117:42942) 2024-09-21T05:59:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.829+0000 7f610cbc2640 1 -- 172.21.15.117:0/1797984701 learned_addr learned my addr 172.21.15.117:0/1797984701 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.829+0000 7f60fe7fc640 1 --2- 172.21.15.117:0/2754422528 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f60f40b6880 0x7f60f40b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.830+0000 7f610d3c3640 1 --2- 172.21.15.117:0/1797984701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2580fc2f0 0x55e2584cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.830+0000 7f60fe7fc640 1 -- 172.21.15.117:0/2754422528 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f60f80e7460 con 0x7f6108116510 2024-09-21T05:59:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.831+0000 7f610dbc4640 1 --2- 172.21.15.117:0/2754422528 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f60f40b6880 0x7f60f40b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.832+0000 7f610cbc2640 1 -- 172.21.15.117:0/1797984701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2580fc2f0 msgr2=0x55e2584cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.832+0000 7f610cbc2640 1 --2- 172.21.15.117:0/1797984701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2580fc2f0 0x55e2584cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.832+0000 7f610cbc2640 1 -- 172.21.15.117:0/1797984701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2581044d0 msgr2=0x55e2584cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.832+0000 7f610cbc2640 1 --2- 172.21.15.117:0/1797984701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2581044d0 0x55e2584cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.832+0000 7f610cbc2640 1 -- 172.21.15.117:0/1797984701 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2583f4530 con 0x55e2581251b0 2024-09-21T05:59:39.841 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.838+0000 7f610cbc2640 1 --2- 172.21.15.117:0/1797984701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e2581251b0 0x55e2584caba0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6100009fa0 tx=0x7f610000b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d016566314ad078d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.842+0000 7f60f37fe640 1 -- 172.21.15.117:0/1797984701 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610001b070 con 0x55e2581251b0 2024-09-21T05:59:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.842+0000 7f60f37fe640 1 -- 172.21.15.117:0/1797984701 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6100008d20 con 0x55e2581251b0 2024-09-21T05:59:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.842+0000 7f60f37fe640 1 -- 172.21.15.117:0/1797984701 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6100016e50 con 0x55e2581251b0 2024-09-21T05:59:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.842+0000 7f610dbc4640 1 --2- 172.21.15.117:0/2754422528 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f60f40b6880 0x7f60f40b8d90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f60f8007590 tx=0x7f60f8003950 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.845+0000 7f611666d740 1 -- 172.21.15.117:0/1797984701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e2581251b0 msgr2=0x55e2584caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.845+0000 7f611666d740 1 --2- 172.21.15.117:0/1797984701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e2581251b0 0x55e2584caba0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6100009fa0 tx=0x7f610000b9a0 comp rx=0 tx=0).stop 2024-09-21T05:59:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.846+0000 7f611666d740 1 -- 172.21.15.117:0/1797984701 shutdown_connections 2024-09-21T05:59:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.846+0000 7f611666d740 1 --2- 172.21.15.117:0/1797984701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2581044d0 0x55e2584cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.846+0000 7f611666d740 1 --2- 172.21.15.117:0/1797984701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2580fc2f0 0x55e2584cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.846+0000 7f611666d740 1 --2- 172.21.15.117:0/1797984701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e2581251b0 0x55e2584caba0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.846+0000 7f611666d740 1 -- 172.21.15.117:0/1797984701 >> 172.21.15.117:0/1797984701 conn(0x55e258119300 msgr2=0x55e2584c8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:39.850 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.848+0000 7f611666d740 1 -- 172.21.15.117:0/1797984701 shutdown_connections 2024-09-21T05:59:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.848+0000 7f611666d740 1 -- 172.21.15.117:0/1797984701 wait complete. 2024-09-21T05:59:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.849+0000 7f611666d740 1 -- start start 2024-09-21T05:59:39.851 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.849+0000 7f611666d740 1 Processor -- start 2024-09-21T05:59:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.850+0000 7f611666d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2580fc2f0 0x55e2584d6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.850+0000 7f611666d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2581044d0 0x55e2584d6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.850+0000 7f611666d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e2581251b0 0x55e2584e2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.850+0000 7f611666d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e258366ab0 con 0x55e2580fc2f0 2024-09-21T05:59:39.854 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.850+0000 7f611666d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e2583d4730 con 0x55e2581044d0 2024-09-21T05:59:39.855 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.850+0000 7f611666d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55e2584d9320 con 0x55e2581251b0 2024-09-21T05:59:39.855 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.850+0000 7f611666d740 10 client.0 _ll_register_callbacks cb 0x55e2584286b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:59:39.855 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.852+0000 7f610cbc2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2580fc2f0 0x55e2584d6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:39.855 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.852+0000 7f610cbc2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2580fc2f0 0x55e2584d6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33932/0 (socket says 172.21.15.117:33932) 2024-09-21T05:59:39.855 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.852+0000 7f610cbc2640 1 -- 172.21.15.117:0/73077831 learned_addr learned my addr 172.21.15.117:0/73077831 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:39.855 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.852+0000 7f610cbc2640 1 --2- 172.21.15.117:0/73077831 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e2581251b0 0x55e2584e2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:39.855 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.852+0000 7f610d3c3640 1 --2- 172.21.15.117:0/73077831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2581044d0 0x55e2584d6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:39.856 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.854+0000 7f610d3c3640 1 -- 172.21.15.117:0/73077831 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e2581251b0 msgr2=0x55e2584e2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:39.856 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.854+0000 7f610d3c3640 1 --2- 172.21.15.117:0/73077831 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e2581251b0 0x55e2584e2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:39.856 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.854+0000 7f610d3c3640 1 -- 172.21.15.117:0/73077831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2580fc2f0 msgr2=0x55e2584d6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.854+0000 7f610d3c3640 1 --2- 172.21.15.117:0/73077831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2580fc2f0 0x55e2584d6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.854+0000 7f610d3c3640 1 -- 172.21.15.117:0/73077831 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e258190790 con 0x55e2581044d0 2024-09-21T05:59:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.855+0000 7f610d3c3640 1 --2- 172.21.15.117:0/73077831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2581044d0 0x55e2584d6880 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6108116160 tx=0x7f61040044f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14fe7e98cc593a6a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.855+0000 7f60f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-21T05:59:39.858 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.856+0000 7f60f0ff9640 1 -- 172.21.15.117:0/73077831 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6104035040 con 0x55e2581044d0 2024-09-21T05:59:39.861 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.858+0000 7f611666d740 1 -- 172.21.15.117:0/73077831 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6100008d20 con 0x55e2581044d0 2024-09-21T05:59:39.861 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.858+0000 7f60f0ff9640 1 -- 172.21.15.117:0/73077831 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f6104005220 con 0x55e2581044d0 2024-09-21T05:59:39.861 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.859+0000 7f60f0ff9640 1 -- 172.21.15.117:0/73077831 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6104005420 con 0x55e2581044d0 2024-09-21T05:59:39.861 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.859+0000 7f611666d740 10 client.25451 fetch_fsmap learned FSMap version 27 2024-09-21T05:59:39.861 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.859+0000 7f611666d740 1 -- 172.21.15.117:0/73077831 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e258209620 con 0x55e2581044d0 2024-09-21T05:59:39.862 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.859+0000 7f60f0ff9640 1 -- 172.21.15.117:0/73077831 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6100008d20 con 0x55e2581044d0 2024-09-21T05:59:39.863 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.860+0000 7f60f0ff9640 1 -- 172.21.15.117:0/73077831 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f61040039a0 con 0x55e2581044d0 2024-09-21T05:59:39.863 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.861+0000 7f611666d740 10 client.25451 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:59:39.863 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.861+0000 7f611666d740 10 client.25451 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T05:59:39.863 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.861+0000 7f611666d740 20 client.25451 populate_metadata read hostname 'smithi117' 2024-09-21T05:59:39.863 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.861+0000 7f611666d740 1 --2- 172.21.15.117:0/73077831 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55e2580e3d80 0x55e2584e89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:39.863 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.861+0000 7f611666d740 4 client.25451 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:59:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.861+0000 7f611666d740 1 -- 172.21.15.117:0/73077831 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e258441640 con 0x55e2580e3d80 2024-09-21T05:59:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.864+0000 7f610cbc2640 1 --2- 172.21.15.117:0/73077831 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55e2580e3d80 0x55e2584e89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:39.868 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.866+0000 7f610cbc2640 1 --2- 172.21.15.117:0/73077831 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55e2580e3d80 0x55e2584e89d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=877ebd6f85952742 server_cookie=a2c7014ce33339c5 in_seq=0 out_seq=0 2024-09-21T05:59:39.868 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.866+0000 7f60f0ff9640 10 client.25451 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:59:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.870+0000 7f60f0ff9640 1 -- 172.21.15.117:0/73077831 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e258441640 con 0x55e2580e3d80 2024-09-21T05:59:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.870+0000 7f60f0ff9640 10 client.25451 handle_command_reply: tid=0 2024-09-21T05:59:39.872 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.870+0000 7f611666d740 1 client.25451 shutdown 2024-09-21T05:59:39.876 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.873+0000 7f611666d740 1 -- 172.21.15.117:0/73077831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2581044d0 msgr2=0x55e2584d6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:39.877 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.873+0000 7f611666d740 1 --2- 172.21.15.117:0/73077831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2581044d0 0x55e2584d6880 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6108116160 tx=0x7f61040044f0 comp rx=0 tx=0).stop 2024-09-21T05:59:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.878+0000 7f611666d740 1 -- 172.21.15.117:0/73077831 shutdown_connections 2024-09-21T05:59:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.878+0000 7f611666d740 1 --2- 172.21.15.117:0/73077831 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55e2580e3d80 0x55e2584e89d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.878+0000 7f611666d740 1 --2- 172.21.15.117:0/73077831 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e2581251b0 0x55e2584e2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.878+0000 7f611666d740 1 --2- 172.21.15.117:0/73077831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2581044d0 0x55e2584d6880 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:39.881 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.878+0000 7f611666d740 1 --2- 172.21.15.117:0/73077831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2580fc2f0 0x55e2584d6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:39.881 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.878+0000 7f611666d740 1 -- 172.21.15.117:0/73077831 >> 172.21.15.117:0/73077831 conn(0x55e258119300 msgr2=0x55e2584cbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:39.881 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.879+0000 7f60f0ff9640 0 client.25451 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:59:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.885+0000 7f611666d740 1 -- 172.21.15.117:0/73077831 shutdown_connections 2024-09-21T05:59:39.888 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.886+0000 7f611666d740 1 -- 172.21.15.117:0/73077831 wait complete. 2024-09-21T05:59:39.891 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.887+0000 7f611666d740 20 client.25451 trim_cache size 0 max 16384 2024-09-21T05:59:39.939 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.937+0000 7f611666d740 1 Processor -- start 2024-09-21T05:59:39.939 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.937+0000 7f611666d740 1 -- start start 2024-09-21T05:59:39.940 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.937+0000 7f611666d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e2580fc2f0 0x55e2584d6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:39.940 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.937+0000 7f611666d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2581044d0 0x55e2584d8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:39.940 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.938+0000 7f611666d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2581251b0 0x55e2584cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:39.940 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.938+0000 7f611666d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f60ec00e490 con 0x55e2581251b0 2024-09-21T05:59:39.940 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.938+0000 7f611666d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f60ec00ffd0 con 0x55e2581044d0 2024-09-21T05:59:39.940 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.938+0000 7f611666d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f60ec00eb60 con 0x55e2580fc2f0 2024-09-21T05:59:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.942+0000 7f610d3c3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2581044d0 0x55e2584d8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.942+0000 7f610cbc2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e2580fc2f0 0x55e2584d6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.942+0000 7f610cbc2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e2580fc2f0 0x55e2584d6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:42966/0 (socket says 172.21.15.117:42966) 2024-09-21T05:59:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.942+0000 7f610cbc2640 1 -- 172.21.15.117:0/1083270769 learned_addr learned my addr 172.21.15.117:0/1083270769 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:39.944 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.942+0000 7f610cbc2640 1 --2- 172.21.15.117:0/1083270769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2581251b0 0x55e2584cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.944+0000 7f610cbc2640 1 -- 172.21.15.117:0/1083270769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2581044d0 msgr2=0x55e2584d8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.944+0000 7f610cbc2640 1 --2- 172.21.15.117:0/1083270769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2581044d0 0x55e2584d8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.944+0000 7f610cbc2640 1 -- 172.21.15.117:0/1083270769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2581251b0 msgr2=0x55e2584cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.944+0000 7f610cbc2640 1 --2- 172.21.15.117:0/1083270769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2581251b0 0x55e2584cc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.944+0000 7f610cbc2640 1 -- 172.21.15.117:0/1083270769 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2584c9430 con 0x55e2580fc2f0 2024-09-21T05:59:39.947 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.945+0000 7f610cbc2640 1 --2- 172.21.15.117:0/1083270769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e2580fc2f0 0x55e2584d6820 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f610000b4e0 tx=0x7f6100007000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7295341af8cc9e95 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.950+0000 7f60f2ffd640 1 -- 172.21.15.117:0/1083270769 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610001b040 con 0x55e2580fc2f0 2024-09-21T05:59:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.950+0000 7f60f2ffd640 1 -- 172.21.15.117:0/1083270769 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6100003c10 con 0x55e2580fc2f0 2024-09-21T05:59:39.952 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.950+0000 7f60f2ffd640 1 -- 172.21.15.117:0/1083270769 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61000043b0 con 0x55e2580fc2f0 2024-09-21T05:59:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.952+0000 7f611666d740 1 -- 172.21.15.117:0/1083270769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e2580fc2f0 msgr2=0x55e2584d6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:39.955 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.953+0000 7f611666d740 1 --2- 172.21.15.117:0/1083270769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e2580fc2f0 0x55e2584d6820 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f610000b4e0 tx=0x7f6100007000 comp rx=0 tx=0).stop 2024-09-21T05:59:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.955+0000 7f611666d740 1 -- 172.21.15.117:0/1083270769 shutdown_connections 2024-09-21T05:59:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.955+0000 7f611666d740 1 --2- 172.21.15.117:0/1083270769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2581251b0 0x55e2584cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.955+0000 7f611666d740 1 --2- 172.21.15.117:0/1083270769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2581044d0 0x55e2584d8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.955+0000 7f611666d740 1 --2- 172.21.15.117:0/1083270769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e2580fc2f0 0x55e2584d6820 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.955+0000 7f611666d740 1 -- 172.21.15.117:0/1083270769 >> 172.21.15.117:0/1083270769 conn(0x55e258119300 msgr2=0x55e2584cbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.956+0000 7f611666d740 1 -- 172.21.15.117:0/1083270769 shutdown_connections 2024-09-21T05:59:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.956+0000 7f611666d740 1 -- 172.21.15.117:0/1083270769 wait complete. 2024-09-21T05:59:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.956+0000 7f611666d740 1 -- start start 2024-09-21T05:59:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.957+0000 7f611666d740 1 Processor -- start 2024-09-21T05:59:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.958+0000 7f611666d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2580fc2f0 0x55e2584d9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.958+0000 7f611666d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e2581044d0 0x55e2584d9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.958+0000 7f611666d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2581251b0 0x55e2584da200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.958+0000 7f611666d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f60ec00f900 con 0x55e2581251b0 2024-09-21T05:59:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.958+0000 7f611666d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f60ec00f230 con 0x55e2580fc2f0 2024-09-21T05:59:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.958+0000 7f611666d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f60ec00c520 con 0x55e2581044d0 2024-09-21T05:59:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.958+0000 7f611666d740 10 client.0 _ll_register_callbacks cb 0x7f6104005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:59:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.959+0000 7f610cbc2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2581251b0 0x55e2584da200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.959+0000 7f610cbc2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2581251b0 0x55e2584da200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:33958/0 (socket says 172.21.15.117:33958) 2024-09-21T05:59:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.959+0000 7f610cbc2640 1 -- 172.21.15.117:0/2100190833 learned_addr learned my addr 172.21.15.117:0/2100190833 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.959+0000 7f610cbc2640 1 -- 172.21.15.117:0/2100190833 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e2581044d0 msgr2=0x55e2584d9cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T05:59:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.959+0000 7f610cbc2640 1 --2- 172.21.15.117:0/2100190833 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e2581044d0 0x55e2584d9cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.960+0000 7f610cbc2640 1 -- 172.21.15.117:0/2100190833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2580fc2f0 msgr2=0x55e2584d9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.960+0000 7f610cbc2640 1 --2- 172.21.15.117:0/2100190833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2580fc2f0 0x55e2584d9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.960+0000 7f610cbc2640 1 -- 172.21.15.117:0/2100190833 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e258551580 con 0x55e2581251b0 2024-09-21T05:59:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.964+0000 7f610cbc2640 1 --2- 172.21.15.117:0/2100190833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2581251b0 0x55e2584da200 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6100009010 tx=0x7f6100016400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb99c455ca12af2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.964+0000 7f60f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:59:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.965+0000 7f60f0ff9640 1 -- 172.21.15.117:0/2100190833 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610001b070 con 0x55e2581251b0 2024-09-21T05:59:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.965+0000 7f60f0ff9640 1 -- 172.21.15.117:0/2100190833 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f6100008730 con 0x55e2581251b0 2024-09-21T05:59:39.967 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.965+0000 7f611666d740 1 -- 172.21.15.117:0/2100190833 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6100003c10 con 0x55e2581251b0 2024-09-21T05:59:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.966+0000 7f60f0ff9640 1 -- 172.21.15.117:0/2100190833 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6100008930 con 0x55e2581251b0 2024-09-21T05:59:39.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.968+0000 7f611666d740 10 client.16041 fetch_fsmap learned FSMap version 27 2024-09-21T05:59:39.972 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.968+0000 7f611666d740 1 -- 172.21.15.117:0/2100190833 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e258552810 con 0x55e2581251b0 2024-09-21T05:59:39.973 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.970+0000 7f60f0ff9640 1 -- 172.21.15.117:0/2100190833 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f610002e440 con 0x55e2581251b0 2024-09-21T05:59:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.972+0000 7f60f0ff9640 1 -- 172.21.15.117:0/2100190833 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6100026410 con 0x55e2581251b0 2024-09-21T05:59:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.972+0000 7f611666d740 10 client.16041 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:59:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.972+0000 7f611666d740 10 client.16041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T05:59:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.972+0000 7f611666d740 20 client.16041 populate_metadata read hostname 'smithi117' 2024-09-21T05:59:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.973+0000 7f611666d740 1 --2- 172.21.15.117:0/2100190833 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55e2580b91c0 0x55e258554c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.973+0000 7f611666d740 4 client.16041 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T05:59:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.973+0000 7f611666d740 1 -- 172.21.15.117:0/2100190833 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f6100003c10 con 0x55e2580b91c0 2024-09-21T05:59:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.975+0000 7f610d3c3640 1 --2- 172.21.15.117:0/2100190833 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55e2580b91c0 0x55e258554c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.978+0000 7f610d3c3640 1 --2- 172.21.15.117:0/2100190833 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55e2580b91c0 0x55e258554c00 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4d36a68d492f1f7 server_cookie=b1c888cd01fab29 in_seq=0 out_seq=0 2024-09-21T05:59:39.987 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:39.985+0000 7f60f0ff9640 10 client.16041 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:59:40.036 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:39 smithi117 ceph-mon[30947]: pgmap v397: 97 pgs: 97 active+clean; 20 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 951 MiB/s wr, 2.11k op/s 2024-09-21T05:59:40.037 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:39 smithi117 ceph-mon[30947]: scrub summary: idle 2024-09-21T05:59:40.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:39 smithi138 ceph-mon[34391]: pgmap v397: 97 pgs: 97 active+clean; 20 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 951 MiB/s wr, 2.11k op/s 2024-09-21T05:59:40.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:39 smithi138 ceph-mon[34391]: scrub summary: idle 2024-09-21T05:59:40.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:39 smithi192 ceph-mon[34700]: pgmap v397: 97 pgs: 97 active+clean; 20 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 951 MiB/s wr, 2.11k op/s 2024-09-21T05:59:40.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:39 smithi192 ceph-mon[34700]: scrub summary: idle 2024-09-21T05:59:41.061 INFO:tasks.cephfs.filesystem:scrub status for tag:ad80d9f1-6aff-46c0-91ab-2c1ab0681039 - {'path': '/', 'tag': 'ad80d9f1-6aff-46c0-91ab-2c1ab0681039', 'options': 'recursive,force'} 2024-09-21T05:59:42.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:41 smithi117 ceph-mon[30947]: pgmap v398: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 833 MiB/s wr, 1.91k op/s 2024-09-21T05:59:42.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:41 smithi138 ceph-mon[34391]: pgmap v398: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 833 MiB/s wr, 1.91k op/s 2024-09-21T05:59:42.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:41 smithi192 ceph-mon[34700]: pgmap v398: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 833 MiB/s wr, 1.91k op/s 2024-09-21T05:59:43.354 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:43.351+0000 7f60f0ff9640 1 -- 172.21.15.117:0/2100190833 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f6100003c10 con 0x55e2580b91c0 2024-09-21T05:59:43.354 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:43.351+0000 7f60f0ff9640 10 client.16041 handle_command_reply: tid=0 2024-09-21T05:59:43.354 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:43.351+0000 7f611666d740 1 client.16041 shutdown 2024-09-21T05:59:43.354 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:43.351+0000 7f611666d740 1 -- 172.21.15.117:0/2100190833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2581251b0 msgr2=0x55e2584da200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:43.354 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:43.351+0000 7f611666d740 1 --2- 172.21.15.117:0/2100190833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2581251b0 0x55e2584da200 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6100009010 tx=0x7f6100016400 comp rx=0 tx=0).stop 2024-09-21T05:59:43.354 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:43.351+0000 7f611666d740 1 -- 172.21.15.117:0/2100190833 shutdown_connections 2024-09-21T05:59:43.354 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:43.351+0000 7f611666d740 1 --2- 172.21.15.117:0/2100190833 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55e2580b91c0 0x55e258554c00 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:43.354 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:43.351+0000 7f611666d740 1 --2- 172.21.15.117:0/2100190833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e2581251b0 0x55e2584da200 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:43.354 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:43.351+0000 7f611666d740 1 --2- 172.21.15.117:0/2100190833 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e2581044d0 0x55e2584d9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:43.351+0000 7f611666d740 1 --2- 172.21.15.117:0/2100190833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e2580fc2f0 0x55e2584d9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:43.351+0000 7f611666d740 1 -- 172.21.15.117:0/2100190833 >> 172.21.15.117:0/2100190833 conn(0x55e258119300 msgr2=0x55e2584d7fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:43.351+0000 7f60f0ff9640 0 client.16041 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:59:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:43.352+0000 7f611666d740 1 -- 172.21.15.117:0/2100190833 shutdown_connections 2024-09-21T05:59:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:43.352+0000 7f611666d740 1 -- 172.21.15.117:0/2100190833 wait complete. 2024-09-21T05:59:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:43.352+0000 7f611666d740 20 client.16041 trim_cache size 0 max 16384 2024-09-21T05:59:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T05:59:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T05:59:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T05:59:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T05:59:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:43.362+0000 7f610e3c5640 1 -- 172.21.15.117:0/2754422528 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f60f40b6880 msgr2=0x7f60f40b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:43.362+0000 7f610e3c5640 1 --2- 172.21.15.117:0/2754422528 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f60f40b6880 0x7f60f40b8d90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f60f8007590 tx=0x7f60f8003950 comp rx=0 tx=0).stop 2024-09-21T05:59:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:43.362+0000 7f610e3c5640 1 -- 172.21.15.117:0/2754422528 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6108116510 msgr2=0x7f61081a8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:43.362+0000 7f610e3c5640 1 --2- 172.21.15.117:0/2754422528 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6108116510 0x7f61081a8600 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f60f800c710 tx=0x7f60f800cbd0 comp rx=0 tx=0).stop 2024-09-21T05:59:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:43.371+0000 7f610e3c5640 1 -- 172.21.15.117:0/2754422528 shutdown_connections 2024-09-21T05:59:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:43.371+0000 7f610e3c5640 1 --2- 172.21.15.117:0/2754422528 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f60f40b6880 0x7f60f40b8d90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:43.371+0000 7f610e3c5640 1 --2- 172.21.15.117:0/2754422528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f610810ad10 0x7f61081a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:43.371+0000 7f610e3c5640 1 --2- 172.21.15.117:0/2754422528 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6108116510 0x7f61081a8600 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:43.371+0000 7f610e3c5640 1 --2- 172.21.15.117:0/2754422528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f610810a3c0 0x7f61081a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:43.371+0000 7f610e3c5640 1 -- 172.21.15.117:0/2754422528 >> 172.21.15.117:0/2754422528 conn(0x7f6108070c00 msgr2=0x7f6108071b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:43.376+0000 7f610e3c5640 1 -- 172.21.15.117:0/2754422528 shutdown_connections 2024-09-21T05:59:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:43.376+0000 7f610e3c5640 1 -- 172.21.15.117:0/2754422528 wait complete. 2024-09-21T05:59:44.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:43 smithi117 ceph-mon[30947]: pgmap v399: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 861 MiB/s wr, 2.03k op/s 2024-09-21T05:59:44.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:43 smithi138 ceph-mon[34391]: pgmap v399: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 861 MiB/s wr, 2.03k op/s 2024-09-21T05:59:44.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:43 smithi192 ceph-mon[34700]: pgmap v399: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 861 MiB/s wr, 2.03k op/s 2024-09-21T05:59:46.031 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:45 smithi192 ceph-mon[34700]: pgmap v400: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 924 MiB/s wr, 2.20k op/s 2024-09-21T05:59:46.034 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:45 smithi117 ceph-mon[30947]: pgmap v400: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 924 MiB/s wr, 2.20k op/s 2024-09-21T05:59:46.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:45 smithi138 ceph-mon[34391]: pgmap v400: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 924 MiB/s wr, 2.20k op/s 2024-09-21T05:59:47.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:46 smithi117 ceph-mon[30947]: pgmap v401: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 873 MiB/s wr, 2.06k op/s 2024-09-21T05:59:47.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:46 smithi138 ceph-mon[34391]: pgmap v401: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 873 MiB/s wr, 2.06k op/s 2024-09-21T05:59:47.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:46 smithi192 ceph-mon[34700]: pgmap v401: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 873 MiB/s wr, 2.06k op/s 2024-09-21T05:59:49.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:49 smithi192 ceph-mon[34700]: pgmap v402: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 920 MiB/s wr, 2.23k op/s 2024-09-21T05:59:50.030 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:49 smithi117 ceph-mon[30947]: pgmap v402: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 920 MiB/s wr, 2.23k op/s 2024-09-21T05:59:50.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:49 smithi138 ceph-mon[34391]: pgmap v402: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 920 MiB/s wr, 2.23k op/s 2024-09-21T05:59:51.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:51 smithi192 ceph-mon[34700]: pgmap v403: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 815 MiB/s wr, 1.96k op/s 2024-09-21T05:59:52.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:51 smithi117 ceph-mon[30947]: pgmap v403: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 815 MiB/s wr, 1.96k op/s 2024-09-21T05:59:52.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:51 smithi138 ceph-mon[34391]: pgmap v403: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 815 MiB/s wr, 1.96k op/s 2024-09-21T05:59:53.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:53 smithi192 ceph-mon[34700]: pgmap v404: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 886 MiB/s wr, 2.29k op/s 2024-09-21T05:59:54.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:53 smithi117 ceph-mon[30947]: pgmap v404: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 886 MiB/s wr, 2.29k op/s 2024-09-21T05:59:54.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:53 smithi138 ceph-mon[34391]: pgmap v404: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 886 MiB/s wr, 2.29k op/s 2024-09-21T05:59:55.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:55 smithi192 ceph-mon[34700]: pgmap v405: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 977 MiB/s wr, 2.71k op/s 2024-09-21T05:59:56.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:55 smithi117 ceph-mon[30947]: pgmap v405: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 977 MiB/s wr, 2.71k op/s 2024-09-21T05:59:56.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:55 smithi138 ceph-mon[34391]: pgmap v405: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 977 MiB/s wr, 2.71k op/s 2024-09-21T05:59:56.826 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.823+0000 7f1eca7fc640 1 -- 172.21.15.117:0/2169788111 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb400a0d0 con 0x7f1ecc110350 2024-09-21T05:59:56.827 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.824+0000 7f1ed1ca2640 1 -- 172.21.15.117:0/2169788111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ecc110350 msgr2=0x7f1ecc07b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:56.827 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.824+0000 7f1ed1ca2640 1 --2- 172.21.15.117:0/2169788111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ecc110350 0x7f1ecc07b690 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4008570 tx=0x7f1eb402e390 comp rx=0 tx=0).stop 2024-09-21T05:59:56.828 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.825+0000 7f1ed1ca2640 1 -- 172.21.15.117:0/2169788111 shutdown_connections 2024-09-21T05:59:56.828 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.825+0000 7f1ed1ca2640 1 --2- 172.21.15.117:0/2169788111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ecc110350 0x7f1ecc07b690 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.825+0000 7f1ed1ca2640 1 --2- 172.21.15.117:0/2169788111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ecc10fa00 0x7f1ecc10fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.825+0000 7f1ed1ca2640 1 --2- 172.21.15.117:0/2169788111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ecc10ba30 0x7f1ecc10be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.825+0000 7f1ed1ca2640 1 -- 172.21.15.117:0/2169788111 >> 172.21.15.117:0/2169788111 conn(0x7f1ecc070dd0 msgr2=0x7f1ecc0711e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:56.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.825+0000 7f1ed1ca2640 1 -- 172.21.15.117:0/2169788111 shutdown_connections 2024-09-21T05:59:56.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.825+0000 7f1ed1ca2640 1 -- 172.21.15.117:0/2169788111 wait complete. 2024-09-21T05:59:56.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.827+0000 7f1ed1ca2640 1 Processor -- start 2024-09-21T05:59:56.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.827+0000 7f1ed1ca2640 1 -- start start 2024-09-21T05:59:56.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.827+0000 7f1ed1ca2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ecc10ba30 0x7f1ecc0806a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:56.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.827+0000 7f1ed1ca2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ecc10fa00 0x7f1ecc080be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.827+0000 7f1ed1ca2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ecc110350 0x7f1ecc1bd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.827+0000 7f1ed1ca2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc07ebc0 con 0x7f1ecc10ba30 2024-09-21T05:59:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.827+0000 7f1ed1ca2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc07ea40 con 0x7f1ecc10fa00 2024-09-21T05:59:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.827+0000 7f1ed1ca2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc07ed40 con 0x7f1ecc110350 2024-09-21T05:59:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.828+0000 7f1ecb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ecc10ba30 0x7f1ecc0806a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.828+0000 7f1ecb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ecc10ba30 0x7f1ecc0806a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45470/0 (socket says 172.21.15.117:45470) 2024-09-21T05:59:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.828+0000 7f1ecb7fe640 1 -- 172.21.15.117:0/945518912 learned_addr learned my addr 172.21.15.117:0/945518912 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.828+0000 7f1ecbfff640 1 --2- 172.21.15.117:0/945518912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ecc110350 0x7f1ecc1bd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.829+0000 7f1ecaffd640 1 --2- 172.21.15.117:0/945518912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ecc10fa00 0x7f1ecc080be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.829+0000 7f1ecb7fe640 1 -- 172.21.15.117:0/945518912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ecc110350 msgr2=0x7f1ecc1bd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.829+0000 7f1ecb7fe640 1 --2- 172.21.15.117:0/945518912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ecc110350 0x7f1ecc1bd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.829+0000 7f1ecb7fe640 1 -- 172.21.15.117:0/945518912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ecc10fa00 msgr2=0x7f1ecc080be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.829+0000 7f1ecb7fe640 1 --2- 172.21.15.117:0/945518912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ecc10fa00 0x7f1ecc080be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.829+0000 7f1ecb7fe640 1 -- 172.21.15.117:0/945518912 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ecc081460 con 0x7f1ecc10ba30 2024-09-21T05:59:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.830+0000 7f1ecb7fe640 1 --2- 172.21.15.117:0/945518912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ecc10ba30 0x7f1ecc0806a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc00b450 tx=0x7f1ebc00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.831+0000 7f1ec8ff9640 1 -- 172.21.15.117:0/945518912 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc015020 con 0x7f1ecc10ba30 2024-09-21T05:59:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.831+0000 7f1ed1ca2640 1 -- 172.21.15.117:0/945518912 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ecc1bde50 con 0x7f1ecc10ba30 2024-09-21T05:59:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.831+0000 7f1ed1ca2640 1 -- 172.21.15.117:0/945518912 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ecc1be420 con 0x7f1ecc10ba30 2024-09-21T05:59:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.831+0000 7f1ec8ff9640 1 -- 172.21.15.117:0/945518912 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ebc011d40 con 0x7f1ecc10ba30 2024-09-21T05:59:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.832+0000 7f1ec8ff9640 1 -- 172.21.15.117:0/945518912 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc00f340 con 0x7f1ecc10ba30 2024-09-21T05:59:56.842 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.839+0000 7f1ec8ff9640 1 -- 172.21.15.117:0/945518912 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f1ebc018020 con 0x7f1ecc10ba30 2024-09-21T05:59:56.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.840+0000 7f1ed8c16740 1 Processor -- start 2024-09-21T05:59:56.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.841+0000 7f1ed8c16740 1 -- start start 2024-09-21T05:59:56.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.841+0000 7f1ec8ff9640 1 --2- 172.21.15.117:0/945518912 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1eb80b6770 0x7f1eb80b8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.841+0000 7f1ec8ff9640 1 -- 172.21.15.117:0/945518912 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f1ebc0e70c0 con 0x7f1ecc10ba30 2024-09-21T05:59:56.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.842+0000 7f1ed8c16740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eadb8e1b0 0x555eadf339b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.842+0000 7f1ecbfff640 1 --2- 172.21.15.117:0/945518912 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1eb80b6770 0x7f1eb80b8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.842+0000 7f1ed8c16740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555eadb652f0 0x555eadf33ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.842+0000 7f1ecaffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555eadb652f0 0x555eadf33ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.842+0000 7f1ecaffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555eadb652f0 0x555eadf33ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45480/0 (socket says 172.21.15.117:45480) 2024-09-21T05:59:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.842+0000 7f1ecaffd640 1 -- 172.21.15.117:0/1275651006 learned_addr learned my addr 172.21.15.117:0/1275651006 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.842+0000 7f1ed8c16740 1 --2- 172.21.15.117:0/1275651006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eadb6d4d0 0x555eadf34430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.842+0000 7f1ed8c16740 1 -- 172.21.15.117:0/1275651006 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555eadea86f0 con 0x555eadb652f0 2024-09-21T05:59:56.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.842+0000 7f1ed8c16740 1 -- 172.21.15.117:0/1275651006 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555eade97b20 con 0x555eadb8e1b0 2024-09-21T05:59:56.846 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.842+0000 7f1ed8c16740 1 -- 172.21.15.117:0/1275651006 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555eade9b930 con 0x555eadb6d4d0 2024-09-21T05:59:56.846 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.843+0000 7f1ecb7fe640 1 --2- 172.21.15.117:0/1275651006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eadb8e1b0 0x555eadf339b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:56.846 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.843+0000 7f1ecaffd640 1 -- 172.21.15.117:0/1275651006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eadb6d4d0 msgr2=0x555eadf34430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:56.846 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.843+0000 7f1ecaffd640 1 --2- 172.21.15.117:0/1275651006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eadb6d4d0 0x555eadf34430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:56.846 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.843+0000 7f1ecaffd640 1 -- 172.21.15.117:0/1275651006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eadb8e1b0 msgr2=0x555eadf339b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:56.847 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.843+0000 7f1ecaffd640 1 --2- 172.21.15.117:0/1275651006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eadb8e1b0 0x555eadf339b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:56.847 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.843+0000 7f1ecaffd640 1 -- 172.21.15.117:0/1275651006 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eade5d530 con 0x555eadb652f0 2024-09-21T05:59:56.847 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.843+0000 7f1ecbfff640 1 --2- 172.21.15.117:0/945518912 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1eb80b6770 0x7f1eb80b8c80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4005f30 tx=0x7f1eb4008230 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:56.847 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.844+0000 7f1ecaffd640 1 --2- 172.21.15.117:0/1275651006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555eadb652f0 0x555eadf33ef0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1ec400be40 tx=0x7f1ec4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff133c9f1398fc63 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:56.847 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.845+0000 7f1ec1ffb640 1 -- 172.21.15.117:0/1275651006 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4003970 con 0x555eadb652f0 2024-09-21T05:59:56.847 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.845+0000 7f1ec1ffb640 1 -- 172.21.15.117:0/1275651006 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ec4004050 con 0x555eadb652f0 2024-09-21T05:59:56.847 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.845+0000 7f1ec1ffb640 1 -- 172.21.15.117:0/1275651006 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec400a680 con 0x555eadb652f0 2024-09-21T05:59:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.847+0000 7f1ed8c16740 1 -- 172.21.15.117:0/1275651006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555eadb652f0 msgr2=0x555eadf33ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.847+0000 7f1ed8c16740 1 --2- 172.21.15.117:0/1275651006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555eadb652f0 0x555eadf33ef0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1ec400be40 tx=0x7f1ec4009260 comp rx=0 tx=0).stop 2024-09-21T05:59:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.848+0000 7f1ed8c16740 1 -- 172.21.15.117:0/1275651006 shutdown_connections 2024-09-21T05:59:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.848+0000 7f1ed8c16740 1 --2- 172.21.15.117:0/1275651006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555eadb652f0 0x555eadf33ef0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.848+0000 7f1ed8c16740 1 --2- 172.21.15.117:0/1275651006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eadb6d4d0 0x555eadf34430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.848+0000 7f1ed8c16740 1 --2- 172.21.15.117:0/1275651006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eadb8e1b0 0x555eadf339b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.848+0000 7f1ed8c16740 1 -- 172.21.15.117:0/1275651006 >> 172.21.15.117:0/1275651006 conn(0x555eadb82300 msgr2=0x555eadf318a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.848+0000 7f1ed8c16740 1 -- 172.21.15.117:0/1275651006 shutdown_connections 2024-09-21T05:59:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.849+0000 7f1ed8c16740 1 -- 172.21.15.117:0/1275651006 wait complete. 2024-09-21T05:59:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.850+0000 7f1ed8c16740 1 -- start start 2024-09-21T05:59:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.850+0000 7f1ed8c16740 1 Processor -- start 2024-09-21T05:59:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.851+0000 7f1ed8c16740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555eadb652f0 0x555eadf3f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.851+0000 7f1ed8c16740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eadb6d4d0 0x555eadf3f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.851+0000 7f1ed8c16740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eadb8e1b0 0x555eadf4b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.851+0000 7f1ed8c16740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555eaddcfab0 con 0x555eadb652f0 2024-09-21T05:59:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.851+0000 7f1ed8c16740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555eade3d730 con 0x555eadb6d4d0 2024-09-21T05:59:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.851+0000 7f1ed8c16740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555eadf42130 con 0x555eadb8e1b0 2024-09-21T05:59:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.851+0000 7f1ed8c16740 10 client.0 _ll_register_callbacks cb 0x555eade916b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:59:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.851+0000 7f1ecb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555eadb652f0 0x555eadf3f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.851+0000 7f1ecb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555eadb652f0 0x555eadf3f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45492/0 (socket says 172.21.15.117:45492) 2024-09-21T05:59:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.851+0000 7f1ecb7fe640 1 -- 172.21.15.117:0/1902321806 learned_addr learned my addr 172.21.15.117:0/1902321806 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.853+0000 7f1ecb7fe640 1 -- 172.21.15.117:0/1902321806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eadb8e1b0 msgr2=0x555eadf4b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.853+0000 7f1ecb7fe640 1 --2- 172.21.15.117:0/1902321806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eadb8e1b0 0x555eadf4b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.853+0000 7f1ecb7fe640 1 -- 172.21.15.117:0/1902321806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eadb6d4d0 msgr2=0x555eadf3f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.853+0000 7f1ecb7fe640 1 --2- 172.21.15.117:0/1902321806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eadb6d4d0 0x555eadf3f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.853+0000 7f1ecb7fe640 1 -- 172.21.15.117:0/1902321806 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eadbf9790 con 0x555eadb652f0 2024-09-21T05:59:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.854+0000 7f1ecb7fe640 1 --2- 172.21.15.117:0/1902321806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555eadb652f0 0x555eadf3f090 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0aa1a0 tx=0x7f1ebc0a8ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3723ffc2dc49733a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.854+0000 7f1eaaffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T05:59:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.854+0000 7f1eaaffd640 1 -- 172.21.15.117:0/1902321806 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc0ac6a0 con 0x555eadb652f0 2024-09-21T05:59:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.854+0000 7f1eaaffd640 1 -- 172.21.15.117:0/1902321806 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f1ebc098190 con 0x555eadb652f0 2024-09-21T05:59:56.857 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.855+0000 7f1ed8c16740 1 -- 172.21.15.117:0/1902321806 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ec4004050 con 0x555eadb652f0 2024-09-21T05:59:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.856+0000 7f1eaaffd640 1 -- 172.21.15.117:0/1902321806 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1ebc0aca40 con 0x555eadb652f0 2024-09-21T05:59:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.856+0000 7f1ed8c16740 10 client.16053 fetch_fsmap learned FSMap version 27 2024-09-21T05:59:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.856+0000 7f1ed8c16740 1 -- 172.21.15.117:0/1902321806 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555eadc72620 con 0x555eadb652f0 2024-09-21T05:59:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.857+0000 7f1eaaffd640 1 -- 172.21.15.117:0/1902321806 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ebc0c5020 con 0x555eadb652f0 2024-09-21T05:59:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.857+0000 7f1eaaffd640 1 -- 172.21.15.117:0/1902321806 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1ebc0a8880 con 0x555eadb652f0 2024-09-21T05:59:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.857+0000 7f1ed8c16740 10 client.16053 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:59:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.857+0000 7f1ed8c16740 10 client.16053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T05:59:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.858+0000 7f1ed8c16740 20 client.16053 populate_metadata read hostname 'smithi117' 2024-09-21T05:59:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.858+0000 7f1ed8c16740 1 --2- 172.21.15.117:0/1902321806 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555eadb4cd80 0x555eadf51390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.858+0000 7f1ed8c16740 4 client.16053 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:59:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.858+0000 7f1ed8c16740 1 -- 172.21.15.117:0/1902321806 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1ec4004050 con 0x555eadb4cd80 2024-09-21T05:59:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.861+0000 7f1ecaffd640 1 --2- 172.21.15.117:0/1902321806 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555eadb4cd80 0x555eadf51390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.866+0000 7f1ecaffd640 1 --2- 172.21.15.117:0/1902321806 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555eadb4cd80 0x555eadf51390 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ebcb748745982d3 server_cookie=9c1ed3b6674201f0 in_seq=0 out_seq=0 2024-09-21T05:59:56.869 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.867+0000 7f1eaaffd640 10 client.16053 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:59:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.869+0000 7f1eaaffd640 1 -- 172.21.15.117:0/1902321806 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f1ec4004050 con 0x555eadb4cd80 2024-09-21T05:59:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.869+0000 7f1eaaffd640 10 client.16053 handle_command_reply: tid=0 2024-09-21T05:59:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.869+0000 7f1ed8c16740 1 client.16053 shutdown 2024-09-21T05:59:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.870+0000 7f1ed8c16740 1 -- 172.21.15.117:0/1902321806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555eadb652f0 msgr2=0x555eadf3f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.870+0000 7f1ed8c16740 1 --2- 172.21.15.117:0/1902321806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555eadb652f0 0x555eadf3f090 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0aa1a0 tx=0x7f1ebc0a8ee0 comp rx=0 tx=0).stop 2024-09-21T05:59:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.870+0000 7f1ed8c16740 1 -- 172.21.15.117:0/1902321806 shutdown_connections 2024-09-21T05:59:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.870+0000 7f1ed8c16740 1 --2- 172.21.15.117:0/1902321806 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555eadb4cd80 0x555eadf51390 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.870+0000 7f1ed8c16740 1 --2- 172.21.15.117:0/1902321806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eadb8e1b0 0x555eadf4b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.870+0000 7f1ed8c16740 1 --2- 172.21.15.117:0/1902321806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eadb6d4d0 0x555eadf3f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.870+0000 7f1ed8c16740 1 --2- 172.21.15.117:0/1902321806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555eadb652f0 0x555eadf3f090 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.870+0000 7f1eaaffd640 0 client.16053 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T05:59:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.870+0000 7f1ed8c16740 1 -- 172.21.15.117:0/1902321806 >> 172.21.15.117:0/1902321806 conn(0x555eadb82300 msgr2=0x555eadf349e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.870+0000 7f1ed8c16740 1 -- 172.21.15.117:0/1902321806 shutdown_connections 2024-09-21T05:59:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.871+0000 7f1ed8c16740 1 -- 172.21.15.117:0/1902321806 wait complete. 2024-09-21T05:59:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.871+0000 7f1ed8c16740 20 client.16053 trim_cache size 0 max 16384 2024-09-21T05:59:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.908+0000 7f1ed8c16740 1 Processor -- start 2024-09-21T05:59:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.908+0000 7f1ed8c16740 1 -- start start 2024-09-21T05:59:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.909+0000 7f1ed8c16740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eadb652f0 0x555eadf353a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.909+0000 7f1ed8c16740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555eadb6d4d0 0x555eadf40cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.910+0000 7f1ed8c16740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eadb8e1b0 0x555eadf3c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.910+0000 7f1ed8c16740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac00e670 con 0x555eadb6d4d0 2024-09-21T05:59:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.910+0000 7f1ed8c16740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac0101b0 con 0x555eadb8e1b0 2024-09-21T05:59:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.910+0000 7f1ed8c16740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac00ed40 con 0x555eadb652f0 2024-09-21T05:59:56.912 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.910+0000 7f1ecb7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eadb652f0 0x555eadf353a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.910+0000 7f1ecb7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eadb652f0 0x555eadf353a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:35688/0 (socket says 172.21.15.117:35688) 2024-09-21T05:59:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.910+0000 7f1ecb7fe640 1 -- 172.21.15.117:0/1877870363 learned_addr learned my addr 172.21.15.117:0/1877870363 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.911+0000 7f1ecb7fe640 1 -- 172.21.15.117:0/1877870363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eadb8e1b0 msgr2=0x555eadf3c260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T05:59:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.911+0000 7f1ecb7fe640 1 --2- 172.21.15.117:0/1877870363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eadb8e1b0 0x555eadf3c260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.911+0000 7f1ecb7fe640 1 -- 172.21.15.117:0/1877870363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555eadb6d4d0 msgr2=0x555eadf40cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.911+0000 7f1ecb7fe640 1 --2- 172.21.15.117:0/1877870363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555eadb6d4d0 0x555eadf40cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.911+0000 7f1ecb7fe640 1 -- 172.21.15.117:0/1877870363 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eadf2cf50 con 0x555eadb652f0 2024-09-21T05:59:56.914 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.912+0000 7f1ecb7fe640 1 --2- 172.21.15.117:0/1877870363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eadb652f0 0x555eadf353a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0acc30 tx=0x7f1ebc0c83e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6978cfcd4f7009e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.915+0000 7f1ec17fa640 1 -- 172.21.15.117:0/1877870363 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc0cc030 con 0x555eadb652f0 2024-09-21T05:59:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.915+0000 7f1ec17fa640 1 -- 172.21.15.117:0/1877870363 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ebc0c5070 con 0x555eadb652f0 2024-09-21T05:59:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.915+0000 7f1ec17fa640 1 -- 172.21.15.117:0/1877870363 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc0b6970 con 0x555eadb652f0 2024-09-21T05:59:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.917+0000 7f1ed8c16740 1 -- 172.21.15.117:0/1877870363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eadb652f0 msgr2=0x555eadf353a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:59:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.917+0000 7f1ed8c16740 1 --2- 172.21.15.117:0/1877870363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eadb652f0 0x555eadf353a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0acc30 tx=0x7f1ebc0c83e0 comp rx=0 tx=0).stop 2024-09-21T05:59:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.917+0000 7f1ed8c16740 1 -- 172.21.15.117:0/1877870363 shutdown_connections 2024-09-21T05:59:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.917+0000 7f1ed8c16740 1 --2- 172.21.15.117:0/1877870363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555eadb6d4d0 0x555eadf40cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.917+0000 7f1ed8c16740 1 --2- 172.21.15.117:0/1877870363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eadb8e1b0 0x555eadf3c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.917+0000 7f1ed8c16740 1 --2- 172.21.15.117:0/1877870363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eadb652f0 0x555eadf353a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.917+0000 7f1ed8c16740 1 -- 172.21.15.117:0/1877870363 >> 172.21.15.117:0/1877870363 conn(0x555eadb82300 msgr2=0x555eadf405b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:59:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.925+0000 7f1ed8c16740 1 -- 172.21.15.117:0/1877870363 shutdown_connections 2024-09-21T05:59:56.928 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.925+0000 7f1ed8c16740 1 -- 172.21.15.117:0/1877870363 wait complete. 2024-09-21T05:59:56.931 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.926+0000 7f1ed8c16740 1 -- start start 2024-09-21T05:59:56.931 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.927+0000 7f1ed8c16740 1 Processor -- start 2024-09-21T05:59:56.931 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.927+0000 7f1ed8c16740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eadb652f0 0x555eadf44e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.927+0000 7f1ed8c16740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555eadb6d4d0 0x555eadf453a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.927+0000 7f1ed8c16740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eadb8e1b0 0x555eadfb3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.928+0000 7f1ed8c16740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac00fae0 con 0x555eadb6d4d0 2024-09-21T05:59:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.928+0000 7f1ed8c16740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac00f410 con 0x555eadb8e1b0 2024-09-21T05:59:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.928+0000 7f1ed8c16740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1eac00c650 con 0x555eadb652f0 2024-09-21T05:59:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.928+0000 7f1ed8c16740 10 client.0 _ll_register_callbacks cb 0x7f1ebc012f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:59:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.928+0000 7f1ecbfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eadb8e1b0 0x555eadfb3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.929+0000 7f1ecbfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eadb8e1b0 0x555eadfb3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:58962/0 (socket says 172.21.15.117:58962) 2024-09-21T05:59:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.929+0000 7f1ecbfff640 1 -- 172.21.15.117:0/255691828 learned_addr learned my addr 172.21.15.117:0/255691828 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T05:59:56.932 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.929+0000 7f1ecb7fe640 1 --2- 172.21.15.117:0/255691828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eadb652f0 0x555eadf44e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.930+0000 7f1ecbfff640 1 -- 172.21.15.117:0/255691828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eadb652f0 msgr2=0x555eadf44e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:59:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.930+0000 7f1ecbfff640 1 --2- 172.21.15.117:0/255691828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eadb652f0 0x555eadf44e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.930+0000 7f1ecbfff640 1 -- 172.21.15.117:0/255691828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555eadb6d4d0 msgr2=0x555eadf453a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T05:59:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.930+0000 7f1ecbfff640 1 --2- 172.21.15.117:0/255691828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555eadb6d4d0 0x555eadf453a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:59:56.933 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.930+0000 7f1ecbfff640 1 -- 172.21.15.117:0/255691828 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eadfb44b0 con 0x555eadb8e1b0 2024-09-21T05:59:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.940+0000 7f1ecbfff640 1 --2- 172.21.15.117:0/255691828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eadb8e1b0 0x555eadfb3e40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1eb400f400 tx=0x7f1eb4004f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50296642db4d3131 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:59:56.942 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.940+0000 7f1eaaffd640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-21T05:59:56.943 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.942+0000 7f1eaaffd640 1 -- 172.21.15.117:0/255691828 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb403c070 con 0x555eadb8e1b0 2024-09-21T05:59:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.942+0000 7f1ed8c16740 1 -- 172.21.15.117:0/255691828 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ebc0c5070 con 0x555eadb8e1b0 2024-09-21T05:59:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.942+0000 7f1eaaffd640 1 -- 172.21.15.117:0/255691828 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f1eb40054d0 con 0x555eadb8e1b0 2024-09-21T05:59:56.945 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.943+0000 7f1eaaffd640 1 -- 172.21.15.117:0/255691828 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1eb400a0d0 con 0x555eadb8e1b0 2024-09-21T05:59:56.947 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.946+0000 7f1ed8c16740 10 client.25463 fetch_fsmap learned FSMap version 27 2024-09-21T05:59:56.947 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.946+0000 7f1ed8c16740 1 -- 172.21.15.117:0/255691828 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555eadfbb870 con 0x555eadb8e1b0 2024-09-21T05:59:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.948+0000 7f1eaaffd640 1 -- 172.21.15.117:0/255691828 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ebc0c5070 con 0x555eadb8e1b0 2024-09-21T05:59:56.952 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.950+0000 7f1eaaffd640 1 -- 172.21.15.117:0/255691828 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f1eb404c760 con 0x555eadb8e1b0 2024-09-21T05:59:56.952 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.951+0000 7f1ed8c16740 10 client.25463 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T05:59:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.951+0000 7f1ed8c16740 10 client.25463 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T05:59:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.951+0000 7f1ed8c16740 20 client.25463 populate_metadata read hostname 'smithi117' 2024-09-21T05:59:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.951+0000 7f1ed8c16740 1 --2- 172.21.15.117:0/255691828 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555eadb221c0 0x555eadfbdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:59:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.951+0000 7f1ed8c16740 4 client.25463 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T05:59:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.951+0000 7f1ed8c16740 1 -- 172.21.15.117:0/255691828 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555eadeaa640 con 0x555eadb221c0 2024-09-21T05:59:56.954 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.952+0000 7f1ecb7fe640 1 --2- 172.21.15.117:0/255691828 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555eadb221c0 0x555eadfbdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:59:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.959+0000 7f1eaaffd640 10 client.25463 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T05:59:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T05:59:56.959+0000 7f1ecb7fe640 1 --2- 172.21.15.117:0/255691828 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555eadb221c0 0x555eadfbdfc0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c679f0cc457b5b0b server_cookie=dc4f965468f22466 in_seq=0 out_seq=0 2024-09-21T05:59:57.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:57 smithi192 ceph-mon[34700]: pgmap v406: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 897 MiB/s wr, 2.60k op/s 2024-09-21T05:59:58.032 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:57 smithi117 ceph-mon[30947]: pgmap v406: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 897 MiB/s wr, 2.60k op/s 2024-09-21T05:59:58.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:57 smithi138 ceph-mon[34391]: pgmap v406: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 897 MiB/s wr, 2.60k op/s 2024-09-21T05:59:59.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 05:59:59 smithi192 ceph-mon[34700]: pgmap v407: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 952 MiB/s wr, 2.90k op/s 2024-09-21T06:00:00.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 05:59:59 smithi117 ceph-mon[30947]: pgmap v407: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 952 MiB/s wr, 2.90k op/s 2024-09-21T06:00:00.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 05:59:59 smithi138 ceph-mon[34391]: pgmap v407: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 952 MiB/s wr, 2.90k op/s 2024-09-21T06:00:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:00.247+0000 7f1eaaffd640 1 -- 172.21.15.117:0/255691828 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555eadeaa640 con 0x555eadb221c0 2024-09-21T06:00:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:00.247+0000 7f1eaaffd640 10 client.25463 handle_command_reply: tid=0 2024-09-21T06:00:00.250 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:00.247+0000 7f1ed8c16740 1 client.25463 shutdown 2024-09-21T06:00:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:00.251+0000 7f1ed8c16740 1 -- 172.21.15.117:0/255691828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eadb8e1b0 msgr2=0x555eadfb3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:00.251+0000 7f1ed8c16740 1 --2- 172.21.15.117:0/255691828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eadb8e1b0 0x555eadfb3e40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1eb400f400 tx=0x7f1eb4004f60 comp rx=0 tx=0).stop 2024-09-21T06:00:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:00.253+0000 7f1ed8c16740 1 -- 172.21.15.117:0/255691828 shutdown_connections 2024-09-21T06:00:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:00.253+0000 7f1ed8c16740 1 --2- 172.21.15.117:0/255691828 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555eadb221c0 0x555eadfbdfc0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:00.253+0000 7f1ed8c16740 1 --2- 172.21.15.117:0/255691828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555eadb6d4d0 0x555eadf453a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:00.253+0000 7f1ed8c16740 1 --2- 172.21.15.117:0/255691828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555eadb8e1b0 0x555eadfb3e40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:00.253+0000 7f1ed8c16740 1 --2- 172.21.15.117:0/255691828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555eadb652f0 0x555eadf44e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:00.253+0000 7f1ed8c16740 1 -- 172.21.15.117:0/255691828 >> 172.21.15.117:0/255691828 conn(0x555eadb82300 msgr2=0x555eadf35080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:00.253+0000 7f1eaaffd640 0 client.25463 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:00:00.258 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:00.256+0000 7f1ed8c16740 1 -- 172.21.15.117:0/255691828 shutdown_connections 2024-09-21T06:00:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:00.256+0000 7f1ed8c16740 1 -- 172.21.15.117:0/255691828 wait complete. 2024-09-21T06:00:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:00.256+0000 7f1ed8c16740 20 client.25463 trim_cache size 0 max 16384 2024-09-21T06:00:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T06:00:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T06:00:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T06:00:00.262 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T06:00:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:00.265+0000 7f1ed1ca2640 1 -- 172.21.15.117:0/945518912 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1eb80b6770 msgr2=0x7f1eb80b8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:00.266+0000 7f1ed1ca2640 1 --2- 172.21.15.117:0/945518912 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1eb80b6770 0x7f1eb80b8c80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4005f30 tx=0x7f1eb4008230 comp rx=0 tx=0).stop 2024-09-21T06:00:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:00.266+0000 7f1ed1ca2640 1 -- 172.21.15.117:0/945518912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ecc10ba30 msgr2=0x7f1ecc0806a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:00.266+0000 7f1ed1ca2640 1 --2- 172.21.15.117:0/945518912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ecc10ba30 0x7f1ecc0806a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc00b450 tx=0x7f1ebc00b910 comp rx=0 tx=0).stop 2024-09-21T06:00:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:00.266+0000 7f1ed1ca2640 1 -- 172.21.15.117:0/945518912 shutdown_connections 2024-09-21T06:00:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:00.266+0000 7f1ed1ca2640 1 --2- 172.21.15.117:0/945518912 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1eb80b6770 0x7f1eb80b8c80 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:00.267+0000 7f1ed1ca2640 1 --2- 172.21.15.117:0/945518912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ecc110350 0x7f1ecc1bd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:00.267+0000 7f1ed1ca2640 1 --2- 172.21.15.117:0/945518912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ecc10fa00 0x7f1ecc080be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:00.267+0000 7f1ed1ca2640 1 --2- 172.21.15.117:0/945518912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ecc10ba30 0x7f1ecc0806a0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:00.267+0000 7f1ed1ca2640 1 -- 172.21.15.117:0/945518912 >> 172.21.15.117:0/945518912 conn(0x7f1ecc070dd0 msgr2=0x7f1ecc10da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:00.267+0000 7f1ed1ca2640 1 -- 172.21.15.117:0/945518912 shutdown_connections 2024-09-21T06:00:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:00.267+0000 7f1ed1ca2640 1 -- 172.21.15.117:0/945518912 wait complete. 2024-09-21T06:00:01.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:00 smithi117 ceph-mon[30947]: overall HEALTH_OK 2024-09-21T06:00:01.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:00 smithi138 ceph-mon[34391]: overall HEALTH_OK 2024-09-21T06:00:01.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:00 smithi192 ceph-mon[34700]: overall HEALTH_OK 2024-09-21T06:00:02.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:01 smithi117 ceph-mon[30947]: pgmap v408: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 847 MiB/s wr, 2.71k op/s 2024-09-21T06:00:02.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:01 smithi138 ceph-mon[34391]: pgmap v408: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 847 MiB/s wr, 2.71k op/s 2024-09-21T06:00:02.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:01 smithi192 ceph-mon[34700]: pgmap v408: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 847 MiB/s wr, 2.71k op/s 2024-09-21T06:00:03.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:02 smithi117 ceph-mon[30947]: pgmap v409: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 952 MiB/s wr, 3.05k op/s 2024-09-21T06:00:03.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:02 smithi138 ceph-mon[34391]: pgmap v409: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 952 MiB/s wr, 3.05k op/s 2024-09-21T06:00:03.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:02 smithi192 ceph-mon[34700]: pgmap v409: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 952 MiB/s wr, 3.05k op/s 2024-09-21T06:00:05.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:05 smithi192 ceph-mon[34700]: pgmap v410: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 992 MiB/s wr, 2.98k op/s 2024-09-21T06:00:06.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:05 smithi117 ceph-mon[30947]: pgmap v410: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 992 MiB/s wr, 2.98k op/s 2024-09-21T06:00:06.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:05 smithi138 ceph-mon[34391]: pgmap v410: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 992 MiB/s wr, 2.98k op/s 2024-09-21T06:00:07.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:07 smithi192 ceph-mon[34700]: pgmap v411: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 895 MiB/s wr, 2.43k op/s 2024-09-21T06:00:08.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:07 smithi117 ceph-mon[30947]: pgmap v411: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 895 MiB/s wr, 2.43k op/s 2024-09-21T06:00:08.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:07 smithi138 ceph-mon[34391]: pgmap v411: 97 pgs: 97 active+clean; 20 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 895 MiB/s wr, 2.43k op/s 2024-09-21T06:00:09.979 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:09 smithi192 ceph-mon[34700]: pgmap v412: 97 pgs: 97 active+clean; 20 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 938 MiB/s wr, 2.41k op/s 2024-09-21T06:00:10.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:09 smithi117 ceph-mon[30947]: pgmap v412: 97 pgs: 97 active+clean; 20 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 938 MiB/s wr, 2.41k op/s 2024-09-21T06:00:10.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:09 smithi138 ceph-mon[34391]: pgmap v412: 97 pgs: 97 active+clean; 20 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 938 MiB/s wr, 2.41k op/s 2024-09-21T06:00:11.062 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:00:11.630 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:00:12.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:11 smithi117 ceph-mon[30947]: pgmap v413: 97 pgs: 97 active+clean; 21 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 816 MiB/s wr, 2.13k op/s 2024-09-21T06:00:12.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:11 smithi138 ceph-mon[34391]: pgmap v413: 97 pgs: 97 active+clean; 21 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 816 MiB/s wr, 2.13k op/s 2024-09-21T06:00:12.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:11 smithi192 ceph-mon[34700]: pgmap v413: 97 pgs: 97 active+clean; 21 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 816 MiB/s wr, 2.13k op/s 2024-09-21T06:00:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.560+0000 7f2b02382640 1 -- 172.21.15.117:0/4226443401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2afc114be0 msgr2=0x7f2afc1170f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.560+0000 7f2b02382640 1 --2- 172.21.15.117:0/4226443401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2afc114be0 0x7f2afc1170f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4009900 tx=0x7f2ae402a410 comp rx=0 tx=0).stop 2024-09-21T06:00:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.562+0000 7f2b02382640 1 -- 172.21.15.117:0/4226443401 shutdown_connections 2024-09-21T06:00:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.562+0000 7f2b02382640 1 --2- 172.21.15.117:0/4226443401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2afc114be0 0x7f2afc1170f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.562+0000 7f2b02382640 1 --2- 172.21.15.117:0/4226443401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2afc10bdb0 0x7f2afc10c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.562+0000 7f2b02382640 1 --2- 172.21.15.117:0/4226443401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2afc10a3c0 0x7f2afc10b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.562+0000 7f2b02382640 1 -- 172.21.15.117:0/4226443401 >> 172.21.15.117:0/4226443401 conn(0x7f2afc070dd0 msgr2=0x7f2afc0711e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.565+0000 7f2b02382640 1 -- 172.21.15.117:0/4226443401 shutdown_connections 2024-09-21T06:00:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.566+0000 7f2b02382640 1 -- 172.21.15.117:0/4226443401 wait complete. 2024-09-21T06:00:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.568+0000 7f2b02382640 1 Processor -- start 2024-09-21T06:00:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.570+0000 7f2b02382640 1 -- start start 2024-09-21T06:00:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.570+0000 7f2b02382640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2afc10a3c0 0x7f2afc1a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.570+0000 7f2b02382640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2afc10bdb0 0x7f2afc1a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.570+0000 7f2b02382640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2afc114be0 0x7f2afc1a21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.570+0000 7f2b02382640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc11b0d0 con 0x7f2afc10a3c0 2024-09-21T06:00:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.570+0000 7f2b02382640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc11af50 con 0x7f2afc114be0 2024-09-21T06:00:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.570+0000 7f2b02382640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc11b250 con 0x7f2afc10bdb0 2024-09-21T06:00:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.571+0000 7f2afbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2afc10a3c0 0x7f2afc1a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.571+0000 7f2afbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2afc10a3c0 0x7f2afc1a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60952/0 (socket says 172.21.15.117:60952) 2024-09-21T06:00:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.571+0000 7f2afbfff640 1 -- 172.21.15.117:0/1041231194 learned_addr learned my addr 172.21.15.117:0/1041231194 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.571+0000 7f2b0091f640 1 --2- 172.21.15.117:0/1041231194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2afc114be0 0x7f2afc1a21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.572+0000 7f2afbfff640 1 -- 172.21.15.117:0/1041231194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2afc10bdb0 msgr2=0x7f2afc1a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.572+0000 7f2afbfff640 1 --2- 172.21.15.117:0/1041231194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2afc10bdb0 0x7f2afc1a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.572+0000 7f2afbfff640 1 -- 172.21.15.117:0/1041231194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2afc114be0 msgr2=0x7f2afc1a21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.572+0000 7f2afbfff640 1 --2- 172.21.15.117:0/1041231194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2afc114be0 0x7f2afc1a21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.572+0000 7f2afbfff640 1 -- 172.21.15.117:0/1041231194 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2afc1a6a30 con 0x7f2afc10a3c0 2024-09-21T06:00:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.573+0000 7f2afbfff640 1 --2- 172.21.15.117:0/1041231194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2afc10a3c0 0x7f2afc1a1750 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2aec00ea60 tx=0x7f2aec00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.575+0000 7f2af97fa640 1 -- 172.21.15.117:0/1041231194 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec00cde0 con 0x7f2afc10a3c0 2024-09-21T06:00:13.578 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.576+0000 7f2af97fa640 1 -- 172.21.15.117:0/1041231194 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2aec0035f0 con 0x7f2afc10a3c0 2024-09-21T06:00:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.576+0000 7f2b02382640 1 -- 172.21.15.117:0/1041231194 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2afc1a6d20 con 0x7f2afc10a3c0 2024-09-21T06:00:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.576+0000 7f2b02382640 1 -- 172.21.15.117:0/1041231194 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2afc1a7270 con 0x7f2afc10a3c0 2024-09-21T06:00:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.576+0000 7f2af97fa640 1 -- 172.21.15.117:0/1041231194 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec010690 con 0x7f2afc10a3c0 2024-09-21T06:00:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.585+0000 7f2af97fa640 1 -- 172.21.15.117:0/1041231194 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f2aec010830 con 0x7f2afc10a3c0 2024-09-21T06:00:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.587+0000 7f2af97fa640 1 --2- 172.21.15.117:0/1041231194 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2ae80b6880 0x7f2ae80b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.587+0000 7f2af97fa640 1 -- 172.21.15.117:0/1041231194 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f2aec0e7c70 con 0x7f2afc10a3c0 2024-09-21T06:00:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.587+0000 7f2afb7fe640 1 --2- 172.21.15.117:0/1041231194 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2ae80b6880 0x7f2ae80b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:13.591 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.588+0000 7f2afb7fe640 1 --2- 172.21.15.117:0/1041231194 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2ae80b6880 0x7f2ae80b8d90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2af4005c40 tx=0x7f2af4005bd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.591+0000 7f2b093cb740 1 Processor -- start 2024-09-21T06:00:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.591+0000 7f2b093cb740 1 -- start start 2024-09-21T06:00:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.592+0000 7f2b093cb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5607d92741b0 0x5607d96199b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.592+0000 7f2b093cb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d924b2f0 0x5607d9619ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.592+0000 7f2b093cb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d92534d0 0x5607d961a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.592+0000 7f2b093cb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5607d958e6f0 con 0x5607d924b2f0 2024-09-21T06:00:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.592+0000 7f2b093cb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5607d957db20 con 0x5607d92534d0 2024-09-21T06:00:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.593+0000 7f2b093cb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5607d9581930 con 0x5607d92741b0 2024-09-21T06:00:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.593+0000 7f2afbfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5607d92741b0 0x5607d96199b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.593+0000 7f2b0091f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d92534d0 0x5607d961a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.593+0000 7f2afbfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5607d92741b0 0x5607d96199b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:53842/0 (socket says 172.21.15.117:53842) 2024-09-21T06:00:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.593+0000 7f2afbfff640 1 -- 172.21.15.117:0/2669706738 learned_addr learned my addr 172.21.15.117:0/2669706738 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.595+0000 7f2afb7fe640 1 --2- 172.21.15.117:0/2669706738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d924b2f0 0x5607d9619ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.595+0000 7f2b0091f640 1 -- 172.21.15.117:0/2669706738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5607d92741b0 msgr2=0x5607d96199b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.595+0000 7f2b0091f640 1 --2- 172.21.15.117:0/2669706738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5607d92741b0 0x5607d96199b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.595+0000 7f2b0091f640 1 -- 172.21.15.117:0/2669706738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d924b2f0 msgr2=0x5607d9619ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.595+0000 7f2b0091f640 1 --2- 172.21.15.117:0/2669706738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d924b2f0 0x5607d9619ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:13.599 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.595+0000 7f2b0091f640 1 -- 172.21.15.117:0/2669706738 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607d9543530 con 0x5607d92534d0 2024-09-21T06:00:13.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.596+0000 7f2b0091f640 1 --2- 172.21.15.117:0/2669706738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d92534d0 0x5607d961a430 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4009230 tx=0x7f2ae40040f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d862290d476534d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:13.603 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.598+0000 7f2af27fc640 1 -- 172.21.15.117:0/2669706738 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae403f070 con 0x5607d92534d0 2024-09-21T06:00:13.603 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.599+0000 7f2af27fc640 1 -- 172.21.15.117:0/2669706738 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ae4004460 con 0x5607d92534d0 2024-09-21T06:00:13.603 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.599+0000 7f2af27fc640 1 -- 172.21.15.117:0/2669706738 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4035070 con 0x5607d92534d0 2024-09-21T06:00:13.603 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.601+0000 7f2b093cb740 1 -- 172.21.15.117:0/2669706738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d92534d0 msgr2=0x5607d961a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:13.604 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.601+0000 7f2b093cb740 1 --2- 172.21.15.117:0/2669706738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d92534d0 0x5607d961a430 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4009230 tx=0x7f2ae40040f0 comp rx=0 tx=0).stop 2024-09-21T06:00:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.601+0000 7f2b093cb740 1 -- 172.21.15.117:0/2669706738 shutdown_connections 2024-09-21T06:00:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.601+0000 7f2b093cb740 1 --2- 172.21.15.117:0/2669706738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d924b2f0 0x5607d9619ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.601+0000 7f2b093cb740 1 --2- 172.21.15.117:0/2669706738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d92534d0 0x5607d961a430 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.601+0000 7f2b093cb740 1 --2- 172.21.15.117:0/2669706738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5607d92741b0 0x5607d96199b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.601+0000 7f2b093cb740 1 -- 172.21.15.117:0/2669706738 >> 172.21.15.117:0/2669706738 conn(0x5607d9268300 msgr2=0x5607d96178a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:13.611 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.609+0000 7f2b093cb740 1 -- 172.21.15.117:0/2669706738 shutdown_connections 2024-09-21T06:00:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.610+0000 7f2b093cb740 1 -- 172.21.15.117:0/2669706738 wait complete. 2024-09-21T06:00:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.610+0000 7f2b093cb740 1 -- start start 2024-09-21T06:00:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.611+0000 7f2b093cb740 1 Processor -- start 2024-09-21T06:00:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.612+0000 7f2b093cb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5607d924b2f0 0x5607d9625150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.612+0000 7f2b093cb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d92534d0 0x5607d9625690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.612+0000 7f2b093cb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d92741b0 0x5607d9631420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.612+0000 7f2b093cb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5607d94b5ab0 con 0x5607d92534d0 2024-09-21T06:00:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.612+0000 7f2b093cb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5607d9523730 con 0x5607d92741b0 2024-09-21T06:00:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.612+0000 7f2b093cb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5607d96280d0 con 0x5607d924b2f0 2024-09-21T06:00:13.615 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.612+0000 7f2b093cb740 10 client.0 _ll_register_callbacks cb 0x5607d95776b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:00:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.613+0000 7f2b0091f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d92741b0 0x5607d9631420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.613+0000 7f2afbfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5607d924b2f0 0x5607d9625150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.613+0000 7f2b0091f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d92741b0 0x5607d9631420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:50610/0 (socket says 172.21.15.117:50610) 2024-09-21T06:00:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.613+0000 7f2b0091f640 1 -- 172.21.15.117:0/857080546 learned_addr learned my addr 172.21.15.117:0/857080546 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.614+0000 7f2b0091f640 1 -- 172.21.15.117:0/857080546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5607d924b2f0 msgr2=0x5607d9625150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.614+0000 7f2b0091f640 1 --2- 172.21.15.117:0/857080546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5607d924b2f0 0x5607d9625150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.614+0000 7f2b0091f640 1 -- 172.21.15.117:0/857080546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d92534d0 msgr2=0x5607d9625690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.614+0000 7f2b0091f640 1 --2- 172.21.15.117:0/857080546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d92534d0 0x5607d9625690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.614+0000 7f2b0091f640 1 -- 172.21.15.117:0/857080546 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607d92df790 con 0x5607d92741b0 2024-09-21T06:00:13.621 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.619+0000 7f2b0091f640 1 --2- 172.21.15.117:0/857080546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d92741b0 0x5607d9631420 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4009850 tx=0x7f2ae40048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f79e6e2da633825 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:13.623 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.620+0000 7f2adb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-21T06:00:13.626 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.623+0000 7f2adb7fe640 1 -- 172.21.15.117:0/857080546 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae403f070 con 0x5607d92741b0 2024-09-21T06:00:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.625+0000 7f2adb7fe640 1 -- 172.21.15.117:0/857080546 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f2ae4005260 con 0x5607d92741b0 2024-09-21T06:00:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.626+0000 7f2b093cb740 1 -- 172.21.15.117:0/857080546 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ae4004460 con 0x5607d92741b0 2024-09-21T06:00:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.631+0000 7f2adb7fe640 1 -- 172.21.15.117:0/857080546 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2ae4035040 con 0x5607d92741b0 2024-09-21T06:00:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.631+0000 7f2b093cb740 10 client.25475 fetch_fsmap learned FSMap version 27 2024-09-21T06:00:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.631+0000 7f2b093cb740 1 -- 172.21.15.117:0/857080546 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5607d9358620 con 0x5607d92741b0 2024-09-21T06:00:13.637 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.634+0000 7f2adb7fe640 1 -- 172.21.15.117:0/857080546 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ae4004460 con 0x5607d92741b0 2024-09-21T06:00:13.639 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.636+0000 7f2adb7fe640 1 -- 172.21.15.117:0/857080546 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2ae4003ce0 con 0x5607d92741b0 2024-09-21T06:00:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.637+0000 7f2b093cb740 10 client.25475 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:00:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.637+0000 7f2b093cb740 10 client.25475 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:00:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.637+0000 7f2b093cb740 20 client.25475 populate_metadata read hostname 'smithi117' 2024-09-21T06:00:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.637+0000 7f2b093cb740 1 --2- 172.21.15.117:0/857080546 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5607d9232d80 0x5607d96370c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.638+0000 7f2b093cb740 4 client.25475 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:00:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.638+0000 7f2b093cb740 1 -- 172.21.15.117:0/857080546 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607d9590640 con 0x5607d9232d80 2024-09-21T06:00:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.641+0000 7f2afbfff640 1 --2- 172.21.15.117:0/857080546 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5607d9232d80 0x5607d96370c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:13.657 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.653+0000 7f2afbfff640 1 --2- 172.21.15.117:0/857080546 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5607d9232d80 0x5607d96370c0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed4c34de9718e95c server_cookie=68ece7e80f2874ef in_seq=0 out_seq=0 2024-09-21T06:00:13.660 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.656+0000 7f2adb7fe640 10 client.25475 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:00:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.660+0000 7f2adb7fe640 1 -- 172.21.15.117:0/857080546 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5607d9590640 con 0x5607d9232d80 2024-09-21T06:00:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.660+0000 7f2adb7fe640 10 client.25475 handle_command_reply: tid=0 2024-09-21T06:00:13.663 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.660+0000 7f2b093cb740 1 client.25475 shutdown 2024-09-21T06:00:13.667 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.662+0000 7f2b093cb740 1 -- 172.21.15.117:0/857080546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d92741b0 msgr2=0x5607d9631420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:13.667 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.662+0000 7f2b093cb740 1 --2- 172.21.15.117:0/857080546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d92741b0 0x5607d9631420 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4009850 tx=0x7f2ae40048f0 comp rx=0 tx=0).stop 2024-09-21T06:00:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.668+0000 7f2b093cb740 1 -- 172.21.15.117:0/857080546 shutdown_connections 2024-09-21T06:00:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.668+0000 7f2b093cb740 1 --2- 172.21.15.117:0/857080546 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5607d9232d80 0x5607d96370c0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.668+0000 7f2b093cb740 1 --2- 172.21.15.117:0/857080546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d92534d0 0x5607d9625690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.668+0000 7f2b093cb740 1 --2- 172.21.15.117:0/857080546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d92741b0 0x5607d9631420 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.668+0000 7f2b093cb740 1 --2- 172.21.15.117:0/857080546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5607d924b2f0 0x5607d9625150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.668+0000 7f2b093cb740 1 -- 172.21.15.117:0/857080546 >> 172.21.15.117:0/857080546 conn(0x5607d9268300 msgr2=0x5607d961a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.669+0000 7f2adb7fe640 0 client.25475 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:00:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.676+0000 7f2b093cb740 1 -- 172.21.15.117:0/857080546 shutdown_connections 2024-09-21T06:00:13.680 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.677+0000 7f2b093cb740 1 -- 172.21.15.117:0/857080546 wait complete. 2024-09-21T06:00:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.684+0000 7f2b093cb740 20 client.25475 trim_cache size 0 max 16384 2024-09-21T06:00:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.725+0000 7f2b093cb740 1 Processor -- start 2024-09-21T06:00:13.731 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.728+0000 7f2b093cb740 1 -- start start 2024-09-21T06:00:13.731 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.728+0000 7f2b093cb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d924b2f0 0x5607d9625630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:13.731 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.728+0000 7f2b093cb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5607d92534d0 0x5607d961b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:13.731 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.728+0000 7f2b093cb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d92741b0 0x5607d9631390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:13.731 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.728+0000 7f2b093cb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc00e5b0 con 0x5607d92741b0 2024-09-21T06:00:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.728+0000 7f2b093cb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc0100f0 con 0x5607d924b2f0 2024-09-21T06:00:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.728+0000 7f2b093cb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc00ec80 con 0x5607d92534d0 2024-09-21T06:00:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.730+0000 7f2afb7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5607d92534d0 0x5607d961b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.730+0000 7f2afb7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5607d92534d0 0x5607d961b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:53872/0 (socket says 172.21.15.117:53872) 2024-09-21T06:00:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.730+0000 7f2afb7fe640 1 -- 172.21.15.117:0/3808229751 learned_addr learned my addr 172.21.15.117:0/3808229751 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:13.735 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.731+0000 7f2b0091f640 1 --2- 172.21.15.117:0/3808229751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d92741b0 0x5607d9631390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:13.735 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.732+0000 7f2afbfff640 1 --2- 172.21.15.117:0/3808229751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d924b2f0 0x5607d9625630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:13.735 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.732+0000 7f2b0091f640 1 -- 172.21.15.117:0/3808229751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5607d92534d0 msgr2=0x5607d961b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:13.735 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.732+0000 7f2b0091f640 1 --2- 172.21.15.117:0/3808229751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5607d92534d0 0x5607d961b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:13.735 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.732+0000 7f2b0091f640 1 -- 172.21.15.117:0/3808229751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d924b2f0 msgr2=0x5607d9625630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:13.735 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.732+0000 7f2b0091f640 1 --2- 172.21.15.117:0/3808229751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d924b2f0 0x5607d9625630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:13.735 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.732+0000 7f2b0091f640 1 -- 172.21.15.117:0/3808229751 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607d9626410 con 0x5607d92741b0 2024-09-21T06:00:13.736 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.732+0000 7f2afbfff640 1 --2- 172.21.15.117:0/3808229751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d924b2f0 0x5607d9625630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:00:13.736 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.732+0000 7f2b0091f640 1 --2- 172.21.15.117:0/3808229751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d92741b0 0x5607d9631390 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4004650 tx=0x7f2ae4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4929b60b7e75a0c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:13.737 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.733+0000 7f2af1ffb640 1 -- 172.21.15.117:0/3808229751 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae405a020 con 0x5607d92741b0 2024-09-21T06:00:13.737 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.733+0000 7f2af1ffb640 1 -- 172.21.15.117:0/3808229751 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ae404d070 con 0x5607d92741b0 2024-09-21T06:00:13.737 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.733+0000 7f2af1ffb640 1 -- 172.21.15.117:0/3808229751 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae403f040 con 0x5607d92741b0 2024-09-21T06:00:13.737 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.734+0000 7f2b093cb740 1 -- 172.21.15.117:0/3808229751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d92741b0 msgr2=0x5607d9631390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:13.738 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.734+0000 7f2b093cb740 1 --2- 172.21.15.117:0/3808229751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d92741b0 0x5607d9631390 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4004650 tx=0x7f2ae4000bc0 comp rx=0 tx=0).stop 2024-09-21T06:00:13.738 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.734+0000 7f2b093cb740 1 -- 172.21.15.117:0/3808229751 shutdown_connections 2024-09-21T06:00:13.738 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.734+0000 7f2b093cb740 1 --2- 172.21.15.117:0/3808229751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d92741b0 0x5607d9631390 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:13.738 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.734+0000 7f2b093cb740 1 --2- 172.21.15.117:0/3808229751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5607d92534d0 0x5607d961b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:13.738 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.734+0000 7f2b093cb740 1 --2- 172.21.15.117:0/3808229751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d924b2f0 0x5607d9625630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:13.738 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.734+0000 7f2b093cb740 1 -- 172.21.15.117:0/3808229751 >> 172.21.15.117:0/3808229751 conn(0x5607d9268300 msgr2=0x5607d961a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.745+0000 7f2b093cb740 1 -- 172.21.15.117:0/3808229751 shutdown_connections 2024-09-21T06:00:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.745+0000 7f2b093cb740 1 -- 172.21.15.117:0/3808229751 wait complete. 2024-09-21T06:00:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.746+0000 7f2b093cb740 1 -- start start 2024-09-21T06:00:13.749 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.746+0000 7f2b093cb740 1 Processor -- start 2024-09-21T06:00:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.747+0000 7f2b093cb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d924b2f0 0x5607d9637530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.747+0000 7f2b093cb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d92534d0 0x5607d9637b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.747+0000 7f2b093cb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5607d92741b0 0x5607d969f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.747+0000 7f2b093cb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc00fa20 con 0x5607d92534d0 2024-09-21T06:00:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.747+0000 7f2b093cb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc00f350 con 0x5607d924b2f0 2024-09-21T06:00:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.747+0000 7f2b093cb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2adc00c640 con 0x5607d92741b0 2024-09-21T06:00:13.750 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.747+0000 7f2b093cb740 10 client.0 _ll_register_callbacks cb 0x7f2af4014050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:00:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.747+0000 7f2afbfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d924b2f0 0x5607d9637530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.747+0000 7f2afb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d92534d0 0x5607d9637b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.747+0000 7f2afbfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d924b2f0 0x5607d9637530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:50624/0 (socket says 172.21.15.117:50624) 2024-09-21T06:00:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.747+0000 7f2afbfff640 1 -- 172.21.15.117:0/1471901658 learned_addr learned my addr 172.21.15.117:0/1471901658 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.747+0000 7f2b0091f640 1 --2- 172.21.15.117:0/1471901658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5607d92741b0 0x5607d969f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.748+0000 7f2afbfff640 1 -- 172.21.15.117:0/1471901658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5607d92741b0 msgr2=0x5607d969f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.748+0000 7f2afbfff640 1 --2- 172.21.15.117:0/1471901658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5607d92741b0 0x5607d969f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.748+0000 7f2afbfff640 1 -- 172.21.15.117:0/1471901658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d92534d0 msgr2=0x5607d9637b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.748+0000 7f2afbfff640 1 --2- 172.21.15.117:0/1471901658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d92534d0 0x5607d9637b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.748+0000 7f2afbfff640 1 -- 172.21.15.117:0/1471901658 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607d969fd70 con 0x5607d924b2f0 2024-09-21T06:00:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.748+0000 7f2afbfff640 1 --2- 172.21.15.117:0/1471901658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d924b2f0 0x5607d9637530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2aec0aad50 tx=0x7f2aec0b0040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9c635e2c0e21c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:13.751 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.748+0000 7f2adb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-21T06:00:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.748+0000 7f2adb7fe640 1 -- 172.21.15.117:0/1471901658 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec0ae1e0 con 0x5607d924b2f0 2024-09-21T06:00:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.748+0000 7f2adb7fe640 1 -- 172.21.15.117:0/1471901658 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f2aec0e5660 con 0x5607d924b2f0 2024-09-21T06:00:13.752 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.749+0000 7f2b093cb740 1 -- 172.21.15.117:0/1471901658 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ae404d070 con 0x5607d924b2f0 2024-09-21T06:00:13.754 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.750+0000 7f2adb7fe640 1 -- 172.21.15.117:0/1471901658 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2aec0e5860 con 0x5607d924b2f0 2024-09-21T06:00:13.754 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.751+0000 7f2b093cb740 10 client.25478 fetch_fsmap learned FSMap version 27 2024-09-21T06:00:13.754 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.751+0000 7f2b093cb740 1 -- 172.21.15.117:0/1471901658 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5607d96a0f00 con 0x5607d924b2f0 2024-09-21T06:00:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.752+0000 7f2adb7fe640 1 -- 172.21.15.117:0/1471901658 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ae404d070 con 0x5607d924b2f0 2024-09-21T06:00:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.752+0000 7f2adb7fe640 1 -- 172.21.15.117:0/1471901658 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f2aec0acd40 con 0x5607d924b2f0 2024-09-21T06:00:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.753+0000 7f2b093cb740 10 client.25478 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:00:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.753+0000 7f2b093cb740 10 client.25478 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:00:13.755 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.753+0000 7f2b093cb740 20 client.25478 populate_metadata read hostname 'smithi117' 2024-09-21T06:00:13.756 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.753+0000 7f2b093cb740 1 --2- 172.21.15.117:0/1471901658 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5607d92081c0 0x5607d96a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:13.756 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.753+0000 7f2b093cb740 4 client.25478 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:00:13.756 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.753+0000 7f2b093cb740 1 -- 172.21.15.117:0/1471901658 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5607d958c0f0 con 0x5607d92081c0 2024-09-21T06:00:13.757 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.754+0000 7f2afb7fe640 1 --2- 172.21.15.117:0/1471901658 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5607d92081c0 0x5607d96a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.756+0000 7f2afb7fe640 1 --2- 172.21.15.117:0/1471901658 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5607d92081c0 0x5607d96a36f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1f5c950b7142d18 server_cookie=248be219a52310a8 in_seq=0 out_seq=0 2024-09-21T06:00:13.759 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:13.757+0000 7f2adb7fe640 10 client.25478 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:00:14.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:13 smithi117 ceph-mon[30947]: pgmap v414: 97 pgs: 97 active+clean; 21 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 842 MiB/s wr, 2.66k op/s 2024-09-21T06:00:14.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:13 smithi138 ceph-mon[34391]: pgmap v414: 97 pgs: 97 active+clean; 21 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 842 MiB/s wr, 2.66k op/s 2024-09-21T06:00:14.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:13 smithi192 ceph-mon[34700]: pgmap v414: 97 pgs: 97 active+clean; 21 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 842 MiB/s wr, 2.66k op/s 2024-09-21T06:00:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:15.871+0000 7f2adb7fe640 1 -- 172.21.15.117:0/1471901658 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5607d958c0f0 con 0x5607d92081c0 2024-09-21T06:00:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:15.871+0000 7f2adb7fe640 10 client.25478 handle_command_reply: tid=0 2024-09-21T06:00:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:15.871+0000 7f2b093cb740 1 client.25478 shutdown 2024-09-21T06:00:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:15.871+0000 7f2b093cb740 1 -- 172.21.15.117:0/1471901658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d924b2f0 msgr2=0x5607d9637530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:15.871+0000 7f2b093cb740 1 --2- 172.21.15.117:0/1471901658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d924b2f0 0x5607d9637530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2aec0aad50 tx=0x7f2aec0b0040 comp rx=0 tx=0).stop 2024-09-21T06:00:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:15.871+0000 7f2b093cb740 1 -- 172.21.15.117:0/1471901658 shutdown_connections 2024-09-21T06:00:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:15.871+0000 7f2b093cb740 1 --2- 172.21.15.117:0/1471901658 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5607d92081c0 0x5607d96a36f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:15.871+0000 7f2b093cb740 1 --2- 172.21.15.117:0/1471901658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d92534d0 0x5607d9637b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:15.871+0000 7f2b093cb740 1 --2- 172.21.15.117:0/1471901658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5607d92741b0 0x5607d969f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:15.871+0000 7f2b093cb740 1 --2- 172.21.15.117:0/1471901658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607d924b2f0 0x5607d9637530 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:15.871+0000 7f2b093cb740 1 -- 172.21.15.117:0/1471901658 >> 172.21.15.117:0/1471901658 conn(0x5607d9268300 msgr2=0x5607d961b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:15.871+0000 7f2adb7fe640 0 client.25478 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:00:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:15.872+0000 7f2b093cb740 1 -- 172.21.15.117:0/1471901658 shutdown_connections 2024-09-21T06:00:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:15.872+0000 7f2b093cb740 1 -- 172.21.15.117:0/1471901658 wait complete. 2024-09-21T06:00:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:15.872+0000 7f2b093cb740 20 client.25478 trim_cache size 0 max 16384 2024-09-21T06:00:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T06:00:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T06:00:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T06:00:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T06:00:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:15.878+0000 7f2b02382640 1 -- 172.21.15.117:0/1041231194 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2ae80b6880 msgr2=0x7f2ae80b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:15.878+0000 7f2b02382640 1 --2- 172.21.15.117:0/1041231194 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2ae80b6880 0x7f2ae80b8d90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2af4005c40 tx=0x7f2af4005bd0 comp rx=0 tx=0).stop 2024-09-21T06:00:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:15.878+0000 7f2b02382640 1 -- 172.21.15.117:0/1041231194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2afc10a3c0 msgr2=0x7f2afc1a1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:15.878+0000 7f2b02382640 1 --2- 172.21.15.117:0/1041231194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2afc10a3c0 0x7f2afc1a1750 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f2aec00ea60 tx=0x7f2aec00ef20 comp rx=0 tx=0).stop 2024-09-21T06:00:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:15.879+0000 7f2b02382640 1 -- 172.21.15.117:0/1041231194 shutdown_connections 2024-09-21T06:00:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:15.879+0000 7f2b02382640 1 --2- 172.21.15.117:0/1041231194 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2ae80b6880 0x7f2ae80b8d90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:15.879+0000 7f2b02382640 1 --2- 172.21.15.117:0/1041231194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2afc114be0 0x7f2afc1a21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:15.879+0000 7f2b02382640 1 --2- 172.21.15.117:0/1041231194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2afc10bdb0 0x7f2afc1a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:15.879+0000 7f2b02382640 1 --2- 172.21.15.117:0/1041231194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2afc10a3c0 0x7f2afc1a1750 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:15.879+0000 7f2b02382640 1 -- 172.21.15.117:0/1041231194 >> 172.21.15.117:0/1041231194 conn(0x7f2afc070dd0 msgr2=0x7f2afc10c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:15.880+0000 7f2b02382640 1 -- 172.21.15.117:0/1041231194 shutdown_connections 2024-09-21T06:00:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:15.880+0000 7f2b02382640 1 -- 172.21.15.117:0/1041231194 wait complete. 2024-09-21T06:00:16.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:15 smithi117 ceph-mon[30947]: pgmap v415: 97 pgs: 97 active+clean; 22 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 867 MiB/s wr, 4.06k op/s 2024-09-21T06:00:16.031 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:15 smithi192 ceph-mon[34700]: pgmap v415: 97 pgs: 97 active+clean; 22 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 867 MiB/s wr, 4.06k op/s 2024-09-21T06:00:16.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:15 smithi138 ceph-mon[34391]: pgmap v415: 97 pgs: 97 active+clean; 22 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 867 MiB/s wr, 4.06k op/s 2024-09-21T06:00:17.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:16 smithi117 ceph-mon[30947]: pgmap v416: 97 pgs: 97 active+clean; 23 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 786 MiB/s wr, 4.29k op/s 2024-09-21T06:00:17.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:16 smithi138 ceph-mon[34391]: pgmap v416: 97 pgs: 97 active+clean; 23 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 786 MiB/s wr, 4.29k op/s 2024-09-21T06:00:17.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:16 smithi192 ceph-mon[34700]: pgmap v416: 97 pgs: 97 active+clean; 23 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 786 MiB/s wr, 4.29k op/s 2024-09-21T06:00:18.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.662+0000 7f584de42640 1 -- 172.21.15.117:0/3080389529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5848116510 msgr2=0x7f5848118990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:18.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.662+0000 7f584de42640 1 --2- 172.21.15.117:0/3080389529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5848116510 0x7f5848118990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5840009be0 tx=0x7f584002a6f0 comp rx=0 tx=0).stop 2024-09-21T06:00:18.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.662+0000 7f584de42640 1 -- 172.21.15.117:0/3080389529 shutdown_connections 2024-09-21T06:00:18.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.662+0000 7f584de42640 1 --2- 172.21.15.117:0/3080389529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f584810af90 0x7f5848115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:18.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.663+0000 7f584de42640 1 --2- 172.21.15.117:0/3080389529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5848116510 0x7f5848118990 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:18.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.663+0000 7f584de42640 1 --2- 172.21.15.117:0/3080389529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f584810a640 0x7f584810aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:18.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.663+0000 7f584de42640 1 -- 172.21.15.117:0/3080389529 >> 172.21.15.117:0/3080389529 conn(0x7f584806e9f0 msgr2=0x7f584806ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:18.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.664+0000 7f584de42640 1 -- 172.21.15.117:0/3080389529 shutdown_connections 2024-09-21T06:00:18.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.665+0000 7f584de42640 1 -- 172.21.15.117:0/3080389529 wait complete. 2024-09-21T06:00:18.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.667+0000 7f584de42640 1 Processor -- start 2024-09-21T06:00:18.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.667+0000 7f584de42640 1 -- start start 2024-09-21T06:00:18.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.667+0000 7f584de42640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f584810a640 0x7f5848115a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:18.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.667+0000 7f584de42640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f584810af90 0x7f5848115fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:18.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.667+0000 7f584de42640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5848116510 0x7f584810fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:18.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.668+0000 7f584de42640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5848119eb0 con 0x7f584810af90 2024-09-21T06:00:18.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.668+0000 7f584de42640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5848119d30 con 0x7f584810a640 2024-09-21T06:00:18.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.668+0000 7f584de42640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f584811a030 con 0x7f5848116510 2024-09-21T06:00:18.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.672+0000 7f584d641640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5848116510 0x7f584810fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:18.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.672+0000 7f5847fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f584810af90 0x7f5848115fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:18.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.672+0000 7f584ce40640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f584810a640 0x7f5848115a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:18.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.673+0000 7f5847fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f584810af90 0x7f5848115fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37670/0 (socket says 172.21.15.117:37670) 2024-09-21T06:00:18.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.673+0000 7f5847fff640 1 -- 172.21.15.117:0/2078285580 learned_addr learned my addr 172.21.15.117:0/2078285580 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:18.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.675+0000 7f5847fff640 1 -- 172.21.15.117:0/2078285580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5848116510 msgr2=0x7f584810fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:18.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.675+0000 7f5847fff640 1 --2- 172.21.15.117:0/2078285580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5848116510 0x7f584810fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:18.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.675+0000 7f5847fff640 1 -- 172.21.15.117:0/2078285580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f584810a640 msgr2=0x7f5848115a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:18.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.675+0000 7f5847fff640 1 --2- 172.21.15.117:0/2078285580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f584810a640 0x7f5848115a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:18.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.675+0000 7f5847fff640 1 -- 172.21.15.117:0/2078285580 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5848110710 con 0x7f584810af90 2024-09-21T06:00:18.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.677+0000 7f5847fff640 1 --2- 172.21.15.117:0/2078285580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f584810af90 0x7f5848115fa0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f583800e9c0 tx=0x7f583800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:18.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.680+0000 7f5845ffb640 1 -- 172.21.15.117:0/2078285580 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583800ccc0 con 0x7f584810af90 2024-09-21T06:00:18.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.680+0000 7f5845ffb640 1 -- 172.21.15.117:0/2078285580 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58380108e0 con 0x7f584810af90 2024-09-21T06:00:18.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.680+0000 7f5845ffb640 1 -- 172.21.15.117:0/2078285580 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58380187e0 con 0x7f584810af90 2024-09-21T06:00:18.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.683+0000 7f584de42640 1 -- 172.21.15.117:0/2078285580 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5848110a00 con 0x7f584810af90 2024-09-21T06:00:18.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.683+0000 7f584de42640 1 -- 172.21.15.117:0/2078285580 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58481bdf90 con 0x7f584810af90 2024-09-21T06:00:18.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.689+0000 7f5855618740 1 Processor -- start 2024-09-21T06:00:18.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.689+0000 7f5855618740 1 -- start start 2024-09-21T06:00:18.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.690+0000 7f5855618740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb894f2070 0x55fb89899b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:18.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.690+0000 7f5855618740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb894c91b0 0x55fb8989a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:18.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.690+0000 7f5855618740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb894d1390 0x55fb8989a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:18.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.691+0000 7f5855618740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fb8980fe50 con 0x55fb894c91b0 2024-09-21T06:00:18.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.691+0000 7f5855618740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fb897ff280 con 0x55fb894f2070 2024-09-21T06:00:18.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.691+0000 7f5855618740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55fb89803090 con 0x55fb894d1390 2024-09-21T06:00:18.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.691+0000 7f584d641640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb894f2070 0x55fb89899b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:18.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.691+0000 7f584d641640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb894f2070 0x55fb89899b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:59780/0 (socket says 172.21.15.117:59780) 2024-09-21T06:00:18.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.691+0000 7f584d641640 1 -- 172.21.15.117:0/3495120870 learned_addr learned my addr 172.21.15.117:0/3495120870 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:18.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.692+0000 7f5847fff640 1 --2- 172.21.15.117:0/3495120870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb894d1390 0x55fb8989a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:18.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.692+0000 7f584ce40640 1 --2- 172.21.15.117:0/3495120870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb894c91b0 0x55fb8989a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:18.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.693+0000 7f584ce40640 1 -- 172.21.15.117:0/3495120870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb894d1390 msgr2=0x55fb8989a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:18.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.693+0000 7f584ce40640 1 --2- 172.21.15.117:0/3495120870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb894d1390 0x55fb8989a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:18.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.693+0000 7f584ce40640 1 -- 172.21.15.117:0/3495120870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb894f2070 msgr2=0x55fb89899b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:18.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.693+0000 7f584ce40640 1 --2- 172.21.15.117:0/3495120870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb894f2070 0x55fb89899b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:18.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.693+0000 7f584ce40640 1 -- 172.21.15.117:0/3495120870 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb8955d650 con 0x55fb894c91b0 2024-09-21T06:00:18.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.694+0000 7f584ce40640 1 --2- 172.21.15.117:0/3495120870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb894c91b0 0x55fb8989a040 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f583c009bf0 tx=0x7f583c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688ecfa1cd895cad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:18.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.696+0000 7f582e7fc640 1 -- 172.21.15.117:0/3495120870 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c009da0 con 0x55fb894c91b0 2024-09-21T06:00:18.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.700+0000 7f582e7fc640 1 -- 172.21.15.117:0/3495120870 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f583c005020 con 0x55fb894c91b0 2024-09-21T06:00:18.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.700+0000 7f582e7fc640 1 -- 172.21.15.117:0/3495120870 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c004b00 con 0x55fb894c91b0 2024-09-21T06:00:18.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.700+0000 7f5845ffb640 1 -- 172.21.15.117:0/2078285580 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f5838010430 con 0x7f584810af90 2024-09-21T06:00:18.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.702+0000 7f5845ffb640 1 --2- 172.21.15.117:0/2078285580 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f58300b6890 0x7f58300b8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:18.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.702+0000 7f5855618740 1 -- 172.21.15.117:0/3495120870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb894c91b0 msgr2=0x55fb8989a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:18.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.702+0000 7f5855618740 1 --2- 172.21.15.117:0/3495120870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb894c91b0 0x55fb8989a040 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f583c009bf0 tx=0x7f583c011810 comp rx=0 tx=0).stop 2024-09-21T06:00:18.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.702+0000 7f5845ffb640 1 -- 172.21.15.117:0/2078285580 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f58380eea10 con 0x7f584810af90 2024-09-21T06:00:18.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.703+0000 7f5855618740 1 -- 172.21.15.117:0/3495120870 shutdown_connections 2024-09-21T06:00:18.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.703+0000 7f5855618740 1 --2- 172.21.15.117:0/3495120870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb894c91b0 0x55fb8989a040 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:18.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.703+0000 7f5855618740 1 --2- 172.21.15.117:0/3495120870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb894d1390 0x55fb8989a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:18.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.703+0000 7f5855618740 1 --2- 172.21.15.117:0/3495120870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb894f2070 0x55fb89899b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:18.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.703+0000 7f5855618740 1 -- 172.21.15.117:0/3495120870 >> 172.21.15.117:0/3495120870 conn(0x55fb894e6460 msgr2=0x55fb898979f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:18.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.706+0000 7f584d641640 1 --2- 172.21.15.117:0/2078285580 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f58300b6890 0x7f58300b8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:18.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.706+0000 7f5855618740 1 -- 172.21.15.117:0/3495120870 shutdown_connections 2024-09-21T06:00:18.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.709+0000 7f584d641640 1 --2- 172.21.15.117:0/2078285580 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f58300b6890 0x7f58300b8da0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f584000b230 tx=0x7f584000b040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:18.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.709+0000 7f5855618740 1 -- 172.21.15.117:0/3495120870 wait complete. 2024-09-21T06:00:18.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.710+0000 7f5855618740 1 -- start start 2024-09-21T06:00:18.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.710+0000 7f5855618740 1 Processor -- start 2024-09-21T06:00:18.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.711+0000 7f5855618740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb894c91b0 0x55fb898a5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:18.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.711+0000 7f5855618740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb894d1390 0x55fb898a5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:18.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.711+0000 7f5855618740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb894f2070 0x55fb898a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:18.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.711+0000 7f5855618740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fb89660700 con 0x55fb894d1390 2024-09-21T06:00:18.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.711+0000 7f5855618740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fb8979a110 con 0x55fb894f2070 2024-09-21T06:00:18.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.711+0000 7f5855618740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55fb898a81f0 con 0x55fb894c91b0 2024-09-21T06:00:18.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.711+0000 7f5855618740 10 client.0 _ll_register_callbacks cb 0x55fb897b3990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:00:18.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.712+0000 7f5847fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb894d1390 0x55fb898a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:18.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.712+0000 7f5847fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb894d1390 0x55fb898a5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37690/0 (socket says 172.21.15.117:37690) 2024-09-21T06:00:18.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.712+0000 7f5847fff640 1 -- 172.21.15.117:0/3440584005 learned_addr learned my addr 172.21.15.117:0/3440584005 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:18.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.713+0000 7f5847fff640 1 -- 172.21.15.117:0/3440584005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb894c91b0 msgr2=0x55fb898a5010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:00:18.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.713+0000 7f5847fff640 1 --2- 172.21.15.117:0/3440584005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb894c91b0 0x55fb898a5010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:18.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.713+0000 7f5847fff640 1 -- 172.21.15.117:0/3440584005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb894f2070 msgr2=0x55fb898a5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:18.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.713+0000 7f5847fff640 1 --2- 172.21.15.117:0/3440584005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb894f2070 0x55fb898a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:18.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.713+0000 7f5847fff640 1 -- 172.21.15.117:0/3440584005 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb897c7c40 con 0x55fb894d1390 2024-09-21T06:00:18.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.713+0000 7f5847fff640 1 --2- 172.21.15.117:0/3440584005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb894d1390 0x55fb898a5550 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f5838005c80 tx=0x7f58380ec700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18cdac156a8319cf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:18.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.714+0000 7f5827fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:00:18.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.714+0000 7f5827fff640 1 -- 172.21.15.117:0/3440584005 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58380b9020 con 0x55fb894d1390 2024-09-21T06:00:18.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.714+0000 7f5855618740 1 -- 172.21.15.117:0/3440584005 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f583c005020 con 0x55fb894d1390 2024-09-21T06:00:18.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.714+0000 7f5827fff640 1 -- 172.21.15.117:0/3440584005 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f58380b29e0 con 0x55fb894d1390 2024-09-21T06:00:18.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.715+0000 7f5827fff640 1 -- 172.21.15.117:0/3440584005 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f58380b4650 con 0x55fb894d1390 2024-09-21T06:00:18.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.716+0000 7f5855618740 10 client.16080 fetch_fsmap learned FSMap version 27 2024-09-21T06:00:18.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.716+0000 7f5855618740 1 -- 172.21.15.117:0/3440584005 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fb898b3d70 con 0x55fb894d1390 2024-09-21T06:00:18.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.717+0000 7f5827fff640 1 -- 172.21.15.117:0/3440584005 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f583c005020 con 0x55fb894d1390 2024-09-21T06:00:18.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.717+0000 7f5827fff640 1 -- 172.21.15.117:0/3440584005 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f58380b5bb0 con 0x55fb894d1390 2024-09-21T06:00:18.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.719+0000 7f5855618740 10 client.16080 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:00:18.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.719+0000 7f5855618740 10 client.16080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:00:18.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.719+0000 7f5855618740 20 client.16080 populate_metadata read hostname 'smithi117' 2024-09-21T06:00:18.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.719+0000 7f5855618740 1 --2- 172.21.15.117:0/3440584005 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55fb894b09f0 0x55fb898b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:18.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.719+0000 7f5855618740 4 client.16080 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:00:18.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.719+0000 7f5855618740 1 -- 172.21.15.117:0/3440584005 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb89811da0 con 0x55fb894b09f0 2024-09-21T06:00:18.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.719+0000 7f584ce40640 1 --2- 172.21.15.117:0/3440584005 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55fb894b09f0 0x55fb898b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:18.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.724+0000 7f584ce40640 1 --2- 172.21.15.117:0/3440584005 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55fb894b09f0 0x55fb898b77a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1cd5716b8ef5958 server_cookie=230b5160ebea8ea3 in_seq=0 out_seq=0 2024-09-21T06:00:18.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.725+0000 7f5827fff640 10 client.16080 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:00:18.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.728+0000 7f5827fff640 1 -- 172.21.15.117:0/3440584005 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fb89811da0 con 0x55fb894b09f0 2024-09-21T06:00:18.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.728+0000 7f5827fff640 10 client.16080 handle_command_reply: tid=0 2024-09-21T06:00:18.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.729+0000 7f5855618740 1 client.16080 shutdown 2024-09-21T06:00:18.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.730+0000 7f5855618740 1 -- 172.21.15.117:0/3440584005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb894d1390 msgr2=0x55fb898a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:18.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.730+0000 7f5855618740 1 --2- 172.21.15.117:0/3440584005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb894d1390 0x55fb898a5550 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f5838005c80 tx=0x7f58380ec700 comp rx=0 tx=0).stop 2024-09-21T06:00:18.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.733+0000 7f5855618740 1 -- 172.21.15.117:0/3440584005 shutdown_connections 2024-09-21T06:00:18.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.733+0000 7f5855618740 1 --2- 172.21.15.117:0/3440584005 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55fb894b09f0 0x55fb898b77a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:18.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.733+0000 7f5855618740 1 --2- 172.21.15.117:0/3440584005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb894d1390 0x55fb898a5550 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:18.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.733+0000 7f5855618740 1 --2- 172.21.15.117:0/3440584005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb894f2070 0x55fb898a5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:18.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.733+0000 7f5855618740 1 --2- 172.21.15.117:0/3440584005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb894c91b0 0x55fb898a5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:18.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.733+0000 7f5855618740 1 -- 172.21.15.117:0/3440584005 >> 172.21.15.117:0/3440584005 conn(0x55fb894e6460 msgr2=0x55fb8989cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:18.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.733+0000 7f5827fff640 0 client.16080 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:00:18.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.736+0000 7f5855618740 1 -- 172.21.15.117:0/3440584005 shutdown_connections 2024-09-21T06:00:18.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.736+0000 7f5855618740 1 -- 172.21.15.117:0/3440584005 wait complete. 2024-09-21T06:00:18.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.737+0000 7f5855618740 20 client.16080 trim_cache size 0 max 16384 2024-09-21T06:00:18.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.776+0000 7f5855618740 1 Processor -- start 2024-09-21T06:00:18.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.776+0000 7f5855618740 1 -- start start 2024-09-21T06:00:18.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.777+0000 7f5855618740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb894c91b0 0x55fb898a6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:18.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.777+0000 7f5855618740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb894d1390 0x55fb8989b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:18.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.777+0000 7f5855618740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb894f2070 0x55fb89896fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:18.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.777+0000 7f5855618740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f58481c6c20 con 0x55fb894c91b0 2024-09-21T06:00:18.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.777+0000 7f5855618740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f58481c8760 con 0x55fb894f2070 2024-09-21T06:00:18.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.777+0000 7f5855618740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f58481c72f0 con 0x55fb894d1390 2024-09-21T06:00:18.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.778+0000 7f584d641640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb894f2070 0x55fb89896fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:18.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.778+0000 7f584d641640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb894f2070 0x55fb89896fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:59800/0 (socket says 172.21.15.117:59800) 2024-09-21T06:00:18.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.778+0000 7f584d641640 1 -- 172.21.15.117:0/174052682 learned_addr learned my addr 172.21.15.117:0/174052682 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:18.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.778+0000 7f5847fff640 1 --2- 172.21.15.117:0/174052682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb894d1390 0x55fb8989b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:18.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.778+0000 7f584d641640 1 -- 172.21.15.117:0/174052682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb894d1390 msgr2=0x55fb8989b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:18.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.778+0000 7f584d641640 1 --2- 172.21.15.117:0/174052682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb894d1390 0x55fb8989b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:18.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.778+0000 7f584d641640 1 -- 172.21.15.117:0/174052682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb894c91b0 msgr2=0x55fb898a6ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:00:18.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.778+0000 7f584d641640 1 --2- 172.21.15.117:0/174052682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb894c91b0 0x55fb898a6ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:18.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.778+0000 7f584d641640 1 -- 172.21.15.117:0/174052682 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb897b5840 con 0x55fb894f2070 2024-09-21T06:00:18.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.782+0000 7f584d641640 1 --2- 172.21.15.117:0/174052682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb894f2070 0x55fb89896fc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5840009be0 tx=0x7f5840005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=202268e1e59e59f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:18.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.783+0000 7f582dffb640 1 -- 172.21.15.117:0/174052682 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5840050070 con 0x55fb894f2070 2024-09-21T06:00:18.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.783+0000 7f582dffb640 1 -- 172.21.15.117:0/174052682 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5840044030 con 0x55fb894f2070 2024-09-21T06:00:18.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.785+0000 7f5855618740 1 -- 172.21.15.117:0/174052682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb894f2070 msgr2=0x55fb89896fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:18.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.785+0000 7f5855618740 1 --2- 172.21.15.117:0/174052682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb894f2070 0x55fb89896fc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5840009be0 tx=0x7f5840005fd0 comp rx=0 tx=0).stop 2024-09-21T06:00:18.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.790+0000 7f5855618740 1 -- 172.21.15.117:0/174052682 shutdown_connections 2024-09-21T06:00:18.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.790+0000 7f5855618740 1 --2- 172.21.15.117:0/174052682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb894f2070 0x55fb89896fc0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:18.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.790+0000 7f5855618740 1 --2- 172.21.15.117:0/174052682 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb894d1390 0x55fb8989b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:18.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.790+0000 7f5855618740 1 --2- 172.21.15.117:0/174052682 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb894c91b0 0x55fb898a6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:18.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.790+0000 7f5855618740 1 -- 172.21.15.117:0/174052682 >> 172.21.15.117:0/174052682 conn(0x55fb894e6460 msgr2=0x55fb8989ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:18.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.791+0000 7f5855618740 1 -- 172.21.15.117:0/174052682 shutdown_connections 2024-09-21T06:00:18.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.794+0000 7f5855618740 1 -- 172.21.15.117:0/174052682 wait complete. 2024-09-21T06:00:18.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.795+0000 7f5855618740 1 -- start start 2024-09-21T06:00:18.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.795+0000 7f5855618740 1 Processor -- start 2024-09-21T06:00:18.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.797+0000 7f5855618740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb894c91b0 0x55fb898a8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:18.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.797+0000 7f5855618740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb894d1390 0x55fb898a9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:18.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.797+0000 7f5855618740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb894f2070 0x55fb8991a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:18.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.797+0000 7f5855618740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f58481c8090 con 0x55fb894c91b0 2024-09-21T06:00:18.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.797+0000 7f5855618740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f58481c79c0 con 0x55fb894f2070 2024-09-21T06:00:18.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.797+0000 7f5855618740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f58481c4c20 con 0x55fb894d1390 2024-09-21T06:00:18.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.797+0000 7f5855618740 10 client.0 _ll_register_callbacks cb 0x7f58380107a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:00:18.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.798+0000 7f584ce40640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb894c91b0 0x55fb898a8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:18.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.798+0000 7f584ce40640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb894c91b0 0x55fb898a8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37700/0 (socket says 172.21.15.117:37700) 2024-09-21T06:00:18.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.798+0000 7f584ce40640 1 -- 172.21.15.117:0/2230484905 learned_addr learned my addr 172.21.15.117:0/2230484905 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:18.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.799+0000 7f584ce40640 1 -- 172.21.15.117:0/2230484905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb894d1390 msgr2=0x55fb898a9250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:00:18.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.799+0000 7f584ce40640 1 --2- 172.21.15.117:0/2230484905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb894d1390 0x55fb898a9250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:18.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.799+0000 7f584ce40640 1 -- 172.21.15.117:0/2230484905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb894f2070 msgr2=0x55fb8991a160 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:00:18.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.799+0000 7f584ce40640 1 --2- 172.21.15.117:0/2230484905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb894f2070 0x55fb8991a160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:18.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.799+0000 7f584ce40640 1 -- 172.21.15.117:0/2230484905 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb8991a7d0 con 0x55fb894c91b0 2024-09-21T06:00:18.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.799+0000 7f584ce40640 1 --2- 172.21.15.117:0/2230484905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb894c91b0 0x55fb898a8c70 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f583c009b40 tx=0x7f583c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4445ca0bf291d819 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:18.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.799+0000 7f5827fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:00:18.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.800+0000 7f5827fff640 1 -- 172.21.15.117:0/2230484905 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c032060 con 0x55fb894c91b0 2024-09-21T06:00:18.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.800+0000 7f5855618740 1 -- 172.21.15.117:0/2230484905 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5840044030 con 0x55fb894c91b0 2024-09-21T06:00:18.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.800+0000 7f5827fff640 1 -- 172.21.15.117:0/2230484905 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f583c02d430 con 0x55fb894c91b0 2024-09-21T06:00:18.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.800+0000 7f5827fff640 1 -- 172.21.15.117:0/2230484905 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f583c02d630 con 0x55fb894c91b0 2024-09-21T06:00:18.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.800+0000 7f5855618740 10 client.16083 fetch_fsmap learned FSMap version 27 2024-09-21T06:00:18.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.800+0000 7f5855618740 1 -- 172.21.15.117:0/2230484905 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fb89921b90 con 0x55fb894c91b0 2024-09-21T06:00:18.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.801+0000 7f5827fff640 1 -- 172.21.15.117:0/2230484905 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5840044030 con 0x55fb894c91b0 2024-09-21T06:00:18.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.801+0000 7f5827fff640 1 -- 172.21.15.117:0/2230484905 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f583c025950 con 0x55fb894c91b0 2024-09-21T06:00:18.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.801+0000 7f5855618740 10 client.16083 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:00:18.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.801+0000 7f5855618740 10 client.16083 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:00:18.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.801+0000 7f5855618740 20 client.16083 populate_metadata read hostname 'smithi117' 2024-09-21T06:00:18.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.801+0000 7f5855618740 1 --2- 172.21.15.117:0/2230484905 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55fb89485da0 0x55fb89924330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:18.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.801+0000 7f5855618740 4 client.16083 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:00:18.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.801+0000 7f5855618740 1 -- 172.21.15.117:0/2230484905 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fb8980d850 con 0x55fb89485da0 2024-09-21T06:00:18.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.803+0000 7f5847fff640 1 --2- 172.21.15.117:0/2230484905 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55fb89485da0 0x55fb89924330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:18.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.807+0000 7f5847fff640 1 --2- 172.21.15.117:0/2230484905 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55fb89485da0 0x55fb89924330 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc6d7f3a9fc51897 server_cookie=e672c428d7912bb9 in_seq=0 out_seq=0 2024-09-21T06:00:18.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.807+0000 7f5827fff640 10 client.16083 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:00:18.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.811+0000 7f5827fff640 1 -- 172.21.15.117:0/2230484905 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+942 (crc 0 0 0) 0x55fb8980d850 con 0x55fb89485da0 2024-09-21T06:00:18.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.811+0000 7f5827fff640 10 client.16083 handle_command_reply: tid=0 2024-09-21T06:00:18.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.811+0000 7f5855618740 1 client.16083 shutdown 2024-09-21T06:00:18.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.815+0000 7f5855618740 1 -- 172.21.15.117:0/2230484905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb894c91b0 msgr2=0x55fb898a8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:18.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.815+0000 7f5855618740 1 --2- 172.21.15.117:0/2230484905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb894c91b0 0x55fb898a8c70 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f583c009b40 tx=0x7f583c009e30 comp rx=0 tx=0).stop 2024-09-21T06:00:18.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.815+0000 7f5855618740 1 -- 172.21.15.117:0/2230484905 shutdown_connections 2024-09-21T06:00:18.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.815+0000 7f5855618740 1 --2- 172.21.15.117:0/2230484905 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55fb89485da0 0x55fb89924330 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:18.726 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T06:00:18.726 INFO:teuthology.orchestra.run.smithi117.stdout: "status": "no active scrubs running", 2024-09-21T06:00:18.726 INFO:teuthology.orchestra.run.smithi117.stdout: "scrubs": {}, 2024-09-21T06:00:18.726 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_stats": { 2024-09-21T06:00:18.726 INFO:teuthology.orchestra.run.smithi117.stdout: "ad80d9f1-6aff-46c0-91ab-2c1ab0681039": { 2024-09-21T06:00:18.726 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:59:31.949108+0000", 2024-09-21T06:00:18.727 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:00:18.727 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:00:18.727 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:00:18.727 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:00:18.727 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:00:18.727 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:00:18.727 INFO:teuthology.orchestra.run.smithi117.stdout: "8b3fe95a-b76b-4e94-afd0-72fe35e2dd92": { 2024-09-21T06:00:18.727 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:58:01.246076+0000", 2024-09-21T06:00:18.727 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:00:18.727 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:00:18.727 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:00:18.727 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:00:18.727 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:00:18.727 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:00:18.728 INFO:teuthology.orchestra.run.smithi117.stdout: "b30bfff0-0f6b-4e30-8467-ff7db421eba6": { 2024-09-21T06:00:18.728 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:57:14.939082+0000", 2024-09-21T06:00:18.728 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:00:18.728 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:00:18.728 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:00:18.728 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:00:18.728 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:00:18.728 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:00:18.728 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:00:18.728 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T06:00:18.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.815+0000 7f5855618740 1 --2- 172.21.15.117:0/2230484905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fb894f2070 0x55fb8991a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:18.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.815+0000 7f5855618740 1 --2- 172.21.15.117:0/2230484905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fb894d1390 0x55fb898a9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:18.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.815+0000 7f5855618740 1 --2- 172.21.15.117:0/2230484905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fb894c91b0 0x55fb898a8c70 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:18.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.815+0000 7f5827fff640 0 client.16083 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:00:18.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.815+0000 7f5855618740 1 -- 172.21.15.117:0/2230484905 >> 172.21.15.117:0/2230484905 conn(0x55fb894e6460 msgr2=0x55fb898a6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:18.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.816+0000 7f5855618740 1 -- 172.21.15.117:0/2230484905 shutdown_connections 2024-09-21T06:00:18.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.817+0000 7f5855618740 1 -- 172.21.15.117:0/2230484905 wait complete. 2024-09-21T06:00:18.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.817+0000 7f5855618740 20 client.16083 trim_cache size 0 max 16384 2024-09-21T06:00:18.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.820+0000 7f582f7fe640 1 -- 172.21.15.117:0/2078285580 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f58300b6890 msgr2=0x7f58300b8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:18.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.820+0000 7f582f7fe640 1 --2- 172.21.15.117:0/2078285580 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f58300b6890 0x7f58300b8da0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f584000b230 tx=0x7f584000b040 comp rx=0 tx=0).stop 2024-09-21T06:00:18.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.820+0000 7f582f7fe640 1 -- 172.21.15.117:0/2078285580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f584810af90 msgr2=0x7f5848115fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:18.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.820+0000 7f582f7fe640 1 --2- 172.21.15.117:0/2078285580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f584810af90 0x7f5848115fa0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f583800e9c0 tx=0x7f583800ee80 comp rx=0 tx=0).stop 2024-09-21T06:00:18.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.821+0000 7f582f7fe640 1 -- 172.21.15.117:0/2078285580 shutdown_connections 2024-09-21T06:00:18.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.821+0000 7f582f7fe640 1 --2- 172.21.15.117:0/2078285580 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f58300b6890 0x7f58300b8da0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:18.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.821+0000 7f582f7fe640 1 --2- 172.21.15.117:0/2078285580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f584810af90 0x7f5848115fa0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:18.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.821+0000 7f582f7fe640 1 --2- 172.21.15.117:0/2078285580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5848116510 0x7f584810fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:18.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.821+0000 7f582f7fe640 1 --2- 172.21.15.117:0/2078285580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f584810a640 0x7f5848115a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:18.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.821+0000 7f582f7fe640 1 -- 172.21.15.117:0/2078285580 >> 172.21.15.117:0/2078285580 conn(0x7f584806e9f0 msgr2=0x7f584811a650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:18.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.822+0000 7f582f7fe640 1 -- 172.21.15.117:0/2078285580 shutdown_connections 2024-09-21T06:00:18.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:17.823+0000 7f582f7fe640 1 -- 172.21.15.117:0/2078285580 wait complete. 2024-09-21T06:00:19.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:19 smithi192 ceph-mon[34700]: pgmap v417: 97 pgs: 97 active+clean; 23 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 805 MiB/s wr, 4.82k op/s 2024-09-21T06:00:20.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:19 smithi117 ceph-mon[30947]: pgmap v417: 97 pgs: 97 active+clean; 23 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 805 MiB/s wr, 4.82k op/s 2024-09-21T06:00:20.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:19 smithi138 ceph-mon[34391]: pgmap v417: 97 pgs: 97 active+clean; 23 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 805 MiB/s wr, 4.82k op/s 2024-09-21T06:00:21.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:21 smithi192 ceph-mon[34700]: pgmap v418: 97 pgs: 97 active+clean; 24 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 720 MiB/s wr, 4.96k op/s 2024-09-21T06:00:22.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:21 smithi117 ceph-mon[30947]: pgmap v418: 97 pgs: 97 active+clean; 24 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 720 MiB/s wr, 4.96k op/s 2024-09-21T06:00:22.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:21 smithi138 ceph-mon[34391]: pgmap v418: 97 pgs: 97 active+clean; 24 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 720 MiB/s wr, 4.96k op/s 2024-09-21T06:00:23.826 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:00:23.827 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:00:24.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:23 smithi117 ceph-mon[30947]: pgmap v419: 97 pgs: 97 active+clean; 25 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 756 MiB/s wr, 5.09k op/s 2024-09-21T06:00:24.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:23 smithi138 ceph-mon[34391]: pgmap v419: 97 pgs: 97 active+clean; 25 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 756 MiB/s wr, 5.09k op/s 2024-09-21T06:00:24.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:23 smithi192 ceph-mon[34700]: pgmap v419: 97 pgs: 97 active+clean; 25 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 756 MiB/s wr, 5.09k op/s 2024-09-21T06:00:24.371 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:00:25.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.689+0000 7f00fd7d4640 1 -- 172.21.15.117:0/2934883179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00f8110640 msgr2=0x7f00f8116e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:25.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.689+0000 7f00fd7d4640 1 --2- 172.21.15.117:0/2934883179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00f8110640 0x7f00f8116e90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f00ec0074e0 tx=0x7f00ec02b080 comp rx=0 tx=0).stop 2024-09-21T06:00:25.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.689+0000 7f00fd7d4640 1 -- 172.21.15.117:0/2934883179 shutdown_connections 2024-09-21T06:00:25.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.689+0000 7f00fd7d4640 1 --2- 172.21.15.117:0/2934883179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00f810fcf0 0x7f00f8110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:25.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.689+0000 7f00fd7d4640 1 --2- 172.21.15.117:0/2934883179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00f8110640 0x7f00f8116e90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:25.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.689+0000 7f00fd7d4640 1 --2- 172.21.15.117:0/2934883179 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00f810a650 0x7f00f810aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:25.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.689+0000 7f00fd7d4640 1 -- 172.21.15.117:0/2934883179 >> 172.21.15.117:0/2934883179 conn(0x7f00f806ee50 msgr2=0x7f00f806f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:25.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.689+0000 7f00fd7d4640 1 -- 172.21.15.117:0/2934883179 shutdown_connections 2024-09-21T06:00:25.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.690+0000 7f00fd7d4640 1 -- 172.21.15.117:0/2934883179 wait complete. 2024-09-21T06:00:25.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.691+0000 7f00fd7d4640 1 Processor -- start 2024-09-21T06:00:25.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.696+0000 7f00fd7d4640 1 -- start start 2024-09-21T06:00:25.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.697+0000 7f00fd7d4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00f810a650 0x7f00f8120460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:25.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.698+0000 7f00fd7d4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00f810fcf0 0x7f00f811a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:25.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.698+0000 7f00fd7d4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00f8110640 0x7f00f811ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:25.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.698+0000 7f00fd7d4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f00f8118c00 con 0x7f00f8110640 2024-09-21T06:00:25.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.698+0000 7f00fd7d4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f00f8118a80 con 0x7f00f810fcf0 2024-09-21T06:00:25.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.698+0000 7f00fd7d4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f00f8118d80 con 0x7f00f810a650 2024-09-21T06:00:25.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.699+0000 7f00fcfd3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00f8110640 0x7f00f811ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:25.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.699+0000 7f00fcfd3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00f8110640 0x7f00f811ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50848/0 (socket says 172.21.15.117:50848) 2024-09-21T06:00:25.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.699+0000 7f00fcfd3640 1 -- 172.21.15.117:0/3734470935 learned_addr learned my addr 172.21.15.117:0/3734470935 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:25.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.701+0000 7f00fcfd3640 1 -- 172.21.15.117:0/3734470935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00f810a650 msgr2=0x7f00f8120460 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:00:25.801 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.701+0000 7f00fcfd3640 1 --2- 172.21.15.117:0/3734470935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00f810a650 0x7f00f8120460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:25.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.701+0000 7f00fcfd3640 1 -- 172.21.15.117:0/3734470935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00f810fcf0 msgr2=0x7f00f811a720 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:00:25.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.701+0000 7f00fcfd3640 1 --2- 172.21.15.117:0/3734470935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00f810fcf0 0x7f00f811a720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:25.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.702+0000 7f00fcfd3640 1 -- 172.21.15.117:0/3734470935 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f81bdb70 con 0x7f00f8110640 2024-09-21T06:00:25.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.702+0000 7f00fcfd3640 1 --2- 172.21.15.117:0/3734470935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00f8110640 0x7f00f811ada0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f00ec005ab0 tx=0x7f00ec007cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:25.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.705+0000 7f00f57fa640 1 -- 172.21.15.117:0/3734470935 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec02bbd0 con 0x7f00f8110640 2024-09-21T06:00:25.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.705+0000 7f00f57fa640 1 -- 172.21.15.117:0/3734470935 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00ec02bd70 con 0x7f00f8110640 2024-09-21T06:00:25.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.705+0000 7f00f57fa640 1 -- 172.21.15.117:0/3734470935 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec00a480 con 0x7f00f8110640 2024-09-21T06:00:25.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.706+0000 7f00fd7d4640 1 -- 172.21.15.117:0/3734470935 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00f81bdd40 con 0x7f00f8110640 2024-09-21T06:00:25.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.707+0000 7f00fd7d4640 1 -- 172.21.15.117:0/3734470935 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00f81be1c0 con 0x7f00f8110640 2024-09-21T06:00:25.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.710+0000 7f00fd7d4640 1 -- 172.21.15.117:0/3734470935 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00f810aa60 con 0x7f00f8110640 2024-09-21T06:00:25.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.724+0000 7f00f57fa640 1 -- 172.21.15.117:0/3734470935 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f00ec043070 con 0x7f00f8110640 2024-09-21T06:00:25.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.727+0000 7f00f57fa640 1 --2- 172.21.15.117:0/3734470935 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f00e00b67d0 0x7f00e00b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:25.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.728+0000 7f00f57fa640 1 -- 172.21.15.117:0/3734470935 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f00ec0c8990 con 0x7f00f8110640 2024-09-21T06:00:25.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.734+0000 7f00f57fa640 1 -- 172.21.15.117:0/3734470935 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f00ec049070 con 0x7f00f8110640 2024-09-21T06:00:25.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.736+0000 7f00f7fff640 1 --2- 172.21.15.117:0/3734470935 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f00e00b67d0 0x7f00e00b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:25.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:25.755+0000 7f00f7fff640 1 --2- 172.21.15.117:0/3734470935 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f00e00b67d0 0x7f00e00b8ce0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f00f81201b0 tx=0x7f00e800b370 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:26.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:25 smithi117 ceph-mon[30947]: pgmap v420: 97 pgs: 97 active+clean; 27 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 882 MiB/s wr, 4.63k op/s 2024-09-21T06:00:26.119 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:25 smithi138 ceph-mon[34391]: pgmap v420: 97 pgs: 97 active+clean; 27 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 882 MiB/s wr, 4.63k op/s 2024-09-21T06:00:26.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:26.184+0000 7f00fd7d4640 1 -- 172.21.15.117:0/3734470935 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00f811b5a0 con 0x7f00f8110640 2024-09-21T06:00:26.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:26.187+0000 7f00f57fa640 1 -- 172.21.15.117:0/3734470935 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+12001 (secure 0 0 0) 0x7f00ec0c6e20 con 0x7f00f8110640 2024-09-21T06:00:26.194 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T06:00:26.194 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":27,"btime":"2024-09-21T05:57:06:651813+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3227057504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3227057504},{"type":"v1","addr":"172.21.15.117:6835","nonce":3227057504}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3303829183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3303829183},{"type":"v1","addr":"172.21.15.117:6837","nonce":3303829183}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T05:55:14.555897+0000","modified":"2024-09-21T05:57:06.599449+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24638,"mds_1":24637,"mds_2":24628,"mds_3":24616,"mds_4":24602},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6839/2314572411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2314572411},{"type":"v1","addr":"172.21.15.117:6839","nonce":2314572411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6841/3378096561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":3378096561},{"type":"v1","addr":"172.21.15.117:6841","nonce":3378096561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/3725629862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3725629862},{"type":"v1","addr":"172.21.15.138:6833","nonce":3725629862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/4276801396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4276801396},{"type":"v1","addr":"172.21.15.138:6835","nonce":4276801396}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6833/2224910879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2224910879},{"type":"v1","addr":"172.21.15.192:6833","nonce":2224910879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.138:6837/2540335374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2540335374},{"type":"v1","addr":"172.21.15.138:6837","nonce":2540335374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24616":{"gid":24616,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.192:6835/3530688860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3530688860},{"type":"v1","addr":"172.21.15.192:6835","nonce":3530688860}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.192:6837/2683860842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2683860842},{"type":"v1","addr":"172.21.15.192:6837","nonce":2683860842}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.192:6839/1727623231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1727623231},{"type":"v1","addr":"172.21.15.192:6839","nonce":1727623231}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/3230372576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":3230372576},{"type":"v1","addr":"172.21.15.138:6839","nonce":3230372576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24602,24628,24616,24637,24638]},"id":1}]} 2024-09-21T06:00:26.194 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 27 2024-09-21T06:00:26.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:26.196+0000 7f00fd7d4640 1 -- 172.21.15.117:0/3734470935 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f00e00b67d0 msgr2=0x7f00e00b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:26.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:26.196+0000 7f00fd7d4640 1 --2- 172.21.15.117:0/3734470935 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f00e00b67d0 0x7f00e00b8ce0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f00f81201b0 tx=0x7f00e800b370 comp rx=0 tx=0).stop 2024-09-21T06:00:26.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:26.196+0000 7f00fd7d4640 1 -- 172.21.15.117:0/3734470935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00f8110640 msgr2=0x7f00f811ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:26.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:26.196+0000 7f00fd7d4640 1 --2- 172.21.15.117:0/3734470935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00f8110640 0x7f00f811ada0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f00ec005ab0 tx=0x7f00ec007cf0 comp rx=0 tx=0).stop 2024-09-21T06:00:26.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:26.196+0000 7f00fd7d4640 1 -- 172.21.15.117:0/3734470935 shutdown_connections 2024-09-21T06:00:26.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:26.196+0000 7f00fd7d4640 1 --2- 172.21.15.117:0/3734470935 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f00e00b67d0 0x7f00e00b8ce0 secure :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f00f81201b0 tx=0x7f00e800b370 comp rx=0 tx=0).stop 2024-09-21T06:00:26.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:26.196+0000 7f00fd7d4640 1 --2- 172.21.15.117:0/3734470935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00f8110640 0x7f00f811ada0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:26.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:26.196+0000 7f00fd7d4640 1 --2- 172.21.15.117:0/3734470935 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00f810fcf0 0x7f00f811a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:26.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:26.196+0000 7f00fd7d4640 1 --2- 172.21.15.117:0/3734470935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00f810a650 0x7f00f8120460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:26.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:26.196+0000 7f00fd7d4640 1 -- 172.21.15.117:0/3734470935 >> 172.21.15.117:0/3734470935 conn(0x7f00f806ee50 msgr2=0x7f00f810dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:26.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:26.199+0000 7f00fd7d4640 1 -- 172.21.15.117:0/3734470935 shutdown_connections 2024-09-21T06:00:26.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:26.199+0000 7f00fd7d4640 1 -- 172.21.15.117:0/3734470935 wait complete. 2024-09-21T06:00:26.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:25 smithi192 ceph-mon[34700]: pgmap v420: 97 pgs: 97 active+clean; 27 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 882 MiB/s wr, 4.63k op/s 2024-09-21T06:00:27.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:26 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3734470935' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:00:27.118 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:26 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3734470935' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:00:27.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:26 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3734470935' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:00:28.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:27 smithi117 ceph-mon[30947]: pgmap v421: 97 pgs: 97 active+clean; 28 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 781 MiB/s wr, 2.80k op/s 2024-09-21T06:00:28.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:27 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:00:28.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:27 smithi138 ceph-mon[34391]: pgmap v421: 97 pgs: 97 active+clean; 28 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 781 MiB/s wr, 2.80k op/s 2024-09-21T06:00:28.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:27 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:00:28.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:27 smithi192 ceph-mon[34700]: pgmap v421: 97 pgs: 97 active+clean; 28 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 781 MiB/s wr, 2.80k op/s 2024-09-21T06:00:28.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:27 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:00:29.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:28 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:00:29.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:28 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:00:29.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:28 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:00:29.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:28 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:00:29.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:28 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:00:29.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:28 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:00:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.242+0000 7f16abfff640 1 -- 172.21.15.117:0/2258858823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16ac10a3c0 msgr2=0x7f16ac10a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.242+0000 7f16abfff640 1 --2- 172.21.15.117:0/2258858823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16ac10a3c0 0x7f16ac10a7d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f169c009860 tx=0x7f169c02a360 comp rx=0 tx=0).stop 2024-09-21T06:00:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.243+0000 7f16abfff640 1 -- 172.21.15.117:0/2258858823 shutdown_connections 2024-09-21T06:00:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.243+0000 7f16abfff640 1 --2- 172.21.15.117:0/2258858823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16ac10ad10 0x7f16ac115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.243+0000 7f16abfff640 1 --2- 172.21.15.117:0/2258858823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16ac116510 0x7f16ac118930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.243+0000 7f16abfff640 1 --2- 172.21.15.117:0/2258858823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16ac10a3c0 0x7f16ac10a7d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.243+0000 7f16abfff640 1 -- 172.21.15.117:0/2258858823 >> 172.21.15.117:0/2258858823 conn(0x7f16ac070c00 msgr2=0x7f16ac071010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.243+0000 7f16abfff640 1 -- 172.21.15.117:0/2258858823 shutdown_connections 2024-09-21T06:00:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.243+0000 7f16abfff640 1 -- 172.21.15.117:0/2258858823 wait complete. 2024-09-21T06:00:29.247 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.245+0000 7f16abfff640 1 Processor -- start 2024-09-21T06:00:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.247+0000 7f16abfff640 1 -- start start 2024-09-21T06:00:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.248+0000 7f16abfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16ac10a3c0 0x7f16ac1a9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.248+0000 7f16abfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16ac10ad10 0x7f16ac1a96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.248+0000 7f16abfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16ac116510 0x7f16ac1ae960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.251+0000 7f16abfff640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac119e60 con 0x7f16ac116510 2024-09-21T06:00:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.251+0000 7f16abfff640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac119ce0 con 0x7f16ac10a3c0 2024-09-21T06:00:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.251+0000 7f16abfff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac119fe0 con 0x7f16ac10ad10 2024-09-21T06:00:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.253+0000 7f16aaffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16ac10a3c0 0x7f16ac1a9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.253+0000 7f16aaffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16ac10a3c0 0x7f16ac1a9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:48166/0 (socket says 172.21.15.117:48166) 2024-09-21T06:00:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.253+0000 7f16aaffd640 1 -- 172.21.15.117:0/359776586 learned_addr learned my addr 172.21.15.117:0/359776586 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.254+0000 7f16ab7fe640 1 --2- 172.21.15.117:0/359776586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16ac116510 0x7f16ac1ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.254+0000 7f16aa7fc640 1 --2- 172.21.15.117:0/359776586 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16ac10ad10 0x7f16ac1a96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.255+0000 7f16ab7fe640 1 -- 172.21.15.117:0/359776586 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16ac10ad10 msgr2=0x7f16ac1a96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.255+0000 7f16ab7fe640 1 --2- 172.21.15.117:0/359776586 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16ac10ad10 0x7f16ac1a96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.255+0000 7f16ab7fe640 1 -- 172.21.15.117:0/359776586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16ac10a3c0 msgr2=0x7f16ac1a9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.255+0000 7f16ab7fe640 1 --2- 172.21.15.117:0/359776586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16ac10a3c0 0x7f16ac1a9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:29.258 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.255+0000 7f16ab7fe640 1 -- 172.21.15.117:0/359776586 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16ac1af060 con 0x7f16ac116510 2024-09-21T06:00:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.256+0000 7f16ab7fe640 1 --2- 172.21.15.117:0/359776586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16ac116510 0x7f16ac1ae960 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f16a000e970 tx=0x7f16a000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:29.260 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.257+0000 7f1693fff640 1 -- 172.21.15.117:0/359776586 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a000ccc0 con 0x7f16ac116510 2024-09-21T06:00:29.261 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.257+0000 7f16abfff640 1 -- 172.21.15.117:0/359776586 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16ac1af350 con 0x7f16ac116510 2024-09-21T06:00:29.261 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.258+0000 7f16abfff640 1 -- 172.21.15.117:0/359776586 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16ac1b5ae0 con 0x7f16ac116510 2024-09-21T06:00:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.270+0000 7f16b820f740 1 Processor -- start 2024-09-21T06:00:29.272 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.270+0000 7f16b820f740 1 -- start start 2024-09-21T06:00:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.271+0000 7f16b820f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb84d091b0 0x55cb850aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.271+0000 7f16b820f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb84ce02f0 0x55cb850af0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.271+0000 7f1693fff640 1 -- 172.21.15.117:0/359776586 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16a00035f0 con 0x7f16ac116510 2024-09-21T06:00:29.273 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.271+0000 7f16b820f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb84ce84d0 0x55cb850af620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.272+0000 7f1693fff640 1 -- 172.21.15.117:0/359776586 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a000ce60 con 0x7f16ac116510 2024-09-21T06:00:29.274 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.272+0000 7f16b820f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cb850236f0 con 0x55cb84ce84d0 2024-09-21T06:00:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.272+0000 7f16b820f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cb85012b20 con 0x55cb84d091b0 2024-09-21T06:00:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.272+0000 7f16b820f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55cb85016930 con 0x55cb84ce02f0 2024-09-21T06:00:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.272+0000 7f1693fff640 1 -- 172.21.15.117:0/359776586 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f16a0010750 con 0x7f16ac116510 2024-09-21T06:00:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.272+0000 7f16aa7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb84d091b0 0x55cb850aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.272+0000 7f16aa7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb84d091b0 0x55cb850aeba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:48172/0 (socket says 172.21.15.117:48172) 2024-09-21T06:00:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.272+0000 7f16aa7fc640 1 -- 172.21.15.117:0/672655526 learned_addr learned my addr 172.21.15.117:0/672655526 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.274+0000 7f1693fff640 1 --2- 172.21.15.117:0/359776586 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f16940b68c0 0x7f16940b8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.274+0000 7f1693fff640 1 -- 172.21.15.117:0/359776586 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f16a00e88f0 con 0x7f16ac116510 2024-09-21T06:00:29.278 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.275+0000 7f16aa7fc640 1 --2- 172.21.15.117:0/672655526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb84ce84d0 0x55cb850af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.275+0000 7f16aa7fc640 1 -- 172.21.15.117:0/672655526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb84ce02f0 msgr2=0x55cb850af0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.275+0000 7f16aa7fc640 1 --2- 172.21.15.117:0/672655526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb84ce02f0 0x55cb850af0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.275+0000 7f16aa7fc640 1 -- 172.21.15.117:0/672655526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb84ce84d0 msgr2=0x55cb850af620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.275+0000 7f16aa7fc640 1 --2- 172.21.15.117:0/672655526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb84ce84d0 0x55cb850af620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.276+0000 7f16ab7fe640 1 --2- 172.21.15.117:0/359776586 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f16940b68c0 0x7f16940b8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.276+0000 7f16aa7fc640 1 -- 172.21.15.117:0/672655526 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb84fd8530 con 0x55cb84d091b0 2024-09-21T06:00:29.279 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:28 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:00:29.279 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:28 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:00:29.279 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:28 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:00:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.277+0000 7f16ab7fe640 1 --2- 172.21.15.117:0/359776586 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f16940b68c0 0x7f16940b8dd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f16a00aba50 tx=0x7f16a0009310 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.277+0000 7f16aa7fc640 1 --2- 172.21.15.117:0/672655526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb84d091b0 0x55cb850aeba0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f16a4009fd0 tx=0x7f16a4009af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7f7a7d9287e3f64 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.278+0000 7f16917fa640 1 -- 172.21.15.117:0/672655526 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a400c4b0 con 0x55cb84d091b0 2024-09-21T06:00:29.280 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.278+0000 7f16917fa640 1 -- 172.21.15.117:0/672655526 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16a400cb00 con 0x55cb84d091b0 2024-09-21T06:00:29.282 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.279+0000 7f16b820f740 1 -- 172.21.15.117:0/672655526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb84d091b0 msgr2=0x55cb850aeba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:29.282 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.279+0000 7f16b820f740 1 --2- 172.21.15.117:0/672655526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb84d091b0 0x55cb850aeba0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f16a4009fd0 tx=0x7f16a4009af0 comp rx=0 tx=0).stop 2024-09-21T06:00:29.283 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.280+0000 7f16b820f740 1 -- 172.21.15.117:0/672655526 shutdown_connections 2024-09-21T06:00:29.283 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.280+0000 7f16b820f740 1 --2- 172.21.15.117:0/672655526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb84ce84d0 0x55cb850af620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:29.283 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.280+0000 7f16b820f740 1 --2- 172.21.15.117:0/672655526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb84ce02f0 0x55cb850af0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:29.283 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.280+0000 7f16b820f740 1 --2- 172.21.15.117:0/672655526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb84d091b0 0x55cb850aeba0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:29.283 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.280+0000 7f16b820f740 1 -- 172.21.15.117:0/672655526 >> 172.21.15.117:0/672655526 conn(0x55cb84cfd300 msgr2=0x55cb850aca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:29.283 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.280+0000 7f16b820f740 1 -- 172.21.15.117:0/672655526 shutdown_connections 2024-09-21T06:00:29.284 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.280+0000 7f16b820f740 1 -- 172.21.15.117:0/672655526 wait complete. 2024-09-21T06:00:29.284 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.281+0000 7f16b820f740 1 -- start start 2024-09-21T06:00:29.284 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.282+0000 7f16b820f740 1 Processor -- start 2024-09-21T06:00:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.282+0000 7f16b820f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb84ce02f0 0x55cb850ba1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.283+0000 7f16b820f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb84ce84d0 0x55cb850ba710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.283+0000 7f16b820f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb84d091b0 0x55cb850bac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.283+0000 7f16b820f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cb84f4aab0 con 0x55cb84d091b0 2024-09-21T06:00:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.283+0000 7f16b820f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cb84fb8730 con 0x55cb84ce02f0 2024-09-21T06:00:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.283+0000 7f16b820f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55cb850bd3c0 con 0x55cb84ce84d0 2024-09-21T06:00:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.283+0000 7f16b820f740 10 client.0 _ll_register_callbacks cb 0x55cb8500c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:00:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.284+0000 7f16aa7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb84ce02f0 0x55cb850ba1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.284+0000 7f16aa7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb84d091b0 0x55cb850bac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.284+0000 7f16aa7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb84d091b0 0x55cb850bac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50874/0 (socket says 172.21.15.117:50874) 2024-09-21T06:00:29.287 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.284+0000 7f16aa7fc640 1 -- 172.21.15.117:0/3637610164 learned_addr learned my addr 172.21.15.117:0/3637610164 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:29.287 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.285+0000 7f16aa7fc640 1 -- 172.21.15.117:0/3637610164 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb84ce84d0 msgr2=0x55cb850ba710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:29.287 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.285+0000 7f16aa7fc640 1 --2- 172.21.15.117:0/3637610164 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb84ce84d0 0x55cb850ba710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:29.287 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.285+0000 7f16aa7fc640 1 -- 172.21.15.117:0/3637610164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb84ce02f0 msgr2=0x55cb850ba1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:29.287 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.285+0000 7f16aa7fc640 1 --2- 172.21.15.117:0/3637610164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb84ce02f0 0x55cb850ba1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:29.287 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.285+0000 7f16aa7fc640 1 -- 172.21.15.117:0/3637610164 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb84d74790 con 0x55cb84d091b0 2024-09-21T06:00:29.299 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.292+0000 7f16aa7fc640 1 --2- 172.21.15.117:0/3637610164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb84d091b0 0x55cb850bac90 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f16a4005fd0 tx=0x7f16a40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d29b66446658cb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:29.299 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.292+0000 7f168eff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:00:29.303 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.301+0000 7f168eff5640 1 -- 172.21.15.117:0/3637610164 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a4023070 con 0x55cb84d091b0 2024-09-21T06:00:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.302+0000 7f16b820f740 1 -- 172.21.15.117:0/3637610164 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16a400cb00 con 0x55cb84d091b0 2024-09-21T06:00:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.302+0000 7f168eff5640 1 -- 172.21.15.117:0/3637610164 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f16a4004c00 con 0x55cb84d091b0 2024-09-21T06:00:29.309 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.305+0000 7f168eff5640 1 -- 172.21.15.117:0/3637610164 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f16a4032050 con 0x55cb84d091b0 2024-09-21T06:00:29.313 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.311+0000 7f16b820f740 10 client.16098 fetch_fsmap learned FSMap version 27 2024-09-21T06:00:29.313 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.311+0000 7f16b820f740 1 -- 172.21.15.117:0/3637610164 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cb84ded620 con 0x55cb84d091b0 2024-09-21T06:00:29.317 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.315+0000 7f168eff5640 1 -- 172.21.15.117:0/3637610164 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16a400cb00 con 0x55cb84d091b0 2024-09-21T06:00:29.317 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.315+0000 7f168eff5640 1 -- 172.21.15.117:0/3637610164 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f16a400b040 con 0x55cb84d091b0 2024-09-21T06:00:29.317 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.315+0000 7f16b820f740 10 client.16098 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:00:29.317 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.315+0000 7f16b820f740 10 client.16098 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:00:29.317 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.315+0000 7f16b820f740 20 client.16098 populate_metadata read hostname 'smithi117' 2024-09-21T06:00:29.318 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.315+0000 7f16b820f740 1 --2- 172.21.15.117:0/3637610164 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55cb84cc7d80 0x55cb850cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:29.318 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.315+0000 7f16b820f740 4 client.16098 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:00:29.318 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.315+0000 7f16b820f740 1 -- 172.21.15.117:0/3637610164 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb85025640 con 0x55cb84cc7d80 2024-09-21T06:00:29.322 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.320+0000 7f16aaffd640 1 --2- 172.21.15.117:0/3637610164 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55cb84cc7d80 0x55cb850cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.329+0000 7f168eff5640 10 client.16098 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:00:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.329+0000 7f16aaffd640 1 --2- 172.21.15.117:0/3637610164 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55cb84cc7d80 0x55cb850cc7e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3aa60a83eb62df9 server_cookie=de19f5607c14c3b4 in_seq=0 out_seq=0 2024-09-21T06:00:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.334+0000 7f168eff5640 1 -- 172.21.15.117:0/3637610164 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cb85025640 con 0x55cb84cc7d80 2024-09-21T06:00:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.334+0000 7f168eff5640 10 client.16098 handle_command_reply: tid=0 2024-09-21T06:00:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.334+0000 7f16b820f740 1 client.16098 shutdown 2024-09-21T06:00:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.337+0000 7f16b820f740 1 -- 172.21.15.117:0/3637610164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb84d091b0 msgr2=0x55cb850bac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.337+0000 7f16b820f740 1 --2- 172.21.15.117:0/3637610164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb84d091b0 0x55cb850bac90 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f16a4005fd0 tx=0x7f16a40095c0 comp rx=0 tx=0).stop 2024-09-21T06:00:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.337+0000 7f16b820f740 1 -- 172.21.15.117:0/3637610164 shutdown_connections 2024-09-21T06:00:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.337+0000 7f16b820f740 1 --2- 172.21.15.117:0/3637610164 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55cb84cc7d80 0x55cb850cc7e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.337+0000 7f16b820f740 1 --2- 172.21.15.117:0/3637610164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb84d091b0 0x55cb850bac90 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.337+0000 7f16b820f740 1 --2- 172.21.15.117:0/3637610164 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb84ce84d0 0x55cb850ba710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.337+0000 7f16b820f740 1 --2- 172.21.15.117:0/3637610164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb84ce02f0 0x55cb850ba1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.337+0000 7f16b820f740 1 -- 172.21.15.117:0/3637610164 >> 172.21.15.117:0/3637610164 conn(0x55cb84cfd300 msgr2=0x55cb850afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.337+0000 7f168eff5640 0 client.16098 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:00:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.338+0000 7f16b820f740 1 -- 172.21.15.117:0/3637610164 shutdown_connections 2024-09-21T06:00:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.338+0000 7f16b820f740 1 -- 172.21.15.117:0/3637610164 wait complete. 2024-09-21T06:00:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.342+0000 7f16b820f740 20 client.16098 trim_cache size 0 max 16384 2024-09-21T06:00:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.381+0000 7f16b820f740 1 Processor -- start 2024-09-21T06:00:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.381+0000 7f16b820f740 1 -- start start 2024-09-21T06:00:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.381+0000 7f16b820f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb84ce02f0 0x55cb850b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.381+0000 7f16b820f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb84ce84d0 0x55cb850b0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.381+0000 7f16b820f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb84c9d1c0 0x55cb850ba6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.382+0000 7f16b820f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f169800e790 con 0x55cb84c9d1c0 2024-09-21T06:00:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.382+0000 7f16b820f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f16980102d0 con 0x55cb84ce84d0 2024-09-21T06:00:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.382+0000 7f16b820f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f169800ee60 con 0x55cb84ce02f0 2024-09-21T06:00:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.382+0000 7f16aa7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb84c9d1c0 0x55cb850ba6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.382+0000 7f16aa7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb84c9d1c0 0x55cb850ba6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50890/0 (socket says 172.21.15.117:50890) 2024-09-21T06:00:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.382+0000 7f16aa7fc640 1 -- 172.21.15.117:0/930063221 learned_addr learned my addr 172.21.15.117:0/930063221 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.383+0000 7f16aa7fc640 1 -- 172.21.15.117:0/930063221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb84ce02f0 msgr2=0x55cb850b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.383+0000 7f16aa7fc640 1 --2- 172.21.15.117:0/930063221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb84ce02f0 0x55cb850b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.383+0000 7f16aa7fc640 1 -- 172.21.15.117:0/930063221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb84ce84d0 msgr2=0x55cb850b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.383+0000 7f16aa7fc640 1 --2- 172.21.15.117:0/930063221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb84ce84d0 0x55cb850b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.383+0000 7f16aa7fc640 1 -- 172.21.15.117:0/930063221 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb850bd430 con 0x55cb84c9d1c0 2024-09-21T06:00:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.383+0000 7f16aa7fc640 1 --2- 172.21.15.117:0/930063221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb84c9d1c0 0x55cb850ba6b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f16a4002230 tx=0x7f16a400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a39ef6fda6a9f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.384+0000 7f1690ff9640 1 -- 172.21.15.117:0/930063221 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a403a070 con 0x55cb84c9d1c0 2024-09-21T06:00:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.384+0000 7f1690ff9640 1 -- 172.21.15.117:0/930063221 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16a400c7c0 con 0x55cb84c9d1c0 2024-09-21T06:00:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.384+0000 7f1690ff9640 1 -- 172.21.15.117:0/930063221 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a4023040 con 0x55cb84c9d1c0 2024-09-21T06:00:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.386+0000 7f16b820f740 1 -- 172.21.15.117:0/930063221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb84c9d1c0 msgr2=0x55cb850ba6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.386+0000 7f16b820f740 1 --2- 172.21.15.117:0/930063221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb84c9d1c0 0x55cb850ba6b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f16a4002230 tx=0x7f16a400f580 comp rx=0 tx=0).stop 2024-09-21T06:00:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.386+0000 7f16b820f740 1 -- 172.21.15.117:0/930063221 shutdown_connections 2024-09-21T06:00:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.386+0000 7f16b820f740 1 --2- 172.21.15.117:0/930063221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb84c9d1c0 0x55cb850ba6b0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.386+0000 7f16b820f740 1 --2- 172.21.15.117:0/930063221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb84ce84d0 0x55cb850b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.386+0000 7f16b820f740 1 --2- 172.21.15.117:0/930063221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb84ce02f0 0x55cb850b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.386+0000 7f16b820f740 1 -- 172.21.15.117:0/930063221 >> 172.21.15.117:0/930063221 conn(0x55cb84cfd300 msgr2=0x55cb850afa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:29.389 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.386+0000 7f16b820f740 1 -- 172.21.15.117:0/930063221 shutdown_connections 2024-09-21T06:00:29.389 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.386+0000 7f16b820f740 1 -- 172.21.15.117:0/930063221 wait complete. 2024-09-21T06:00:29.389 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.387+0000 7f16b820f740 1 -- start start 2024-09-21T06:00:29.389 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.387+0000 7f16b820f740 1 Processor -- start 2024-09-21T06:00:29.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.388+0000 7f16b820f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb84c9d1c0 0x55cb850c2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:29.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.388+0000 7f16b820f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb84ce02f0 0x55cb850bf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:29.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.388+0000 7f16b820f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb84ce84d0 0x55cb850bf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:29.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.388+0000 7f16b820f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f169800fc00 con 0x55cb84c9d1c0 2024-09-21T06:00:29.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.388+0000 7f16b820f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f169800f530 con 0x55cb84ce84d0 2024-09-21T06:00:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.388+0000 7f16b820f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f169800c770 con 0x55cb84ce02f0 2024-09-21T06:00:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.388+0000 7f16b820f740 10 client.0 _ll_register_callbacks cb 0x7f16a400f490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:00:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.388+0000 7f16aa7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb84c9d1c0 0x55cb850c2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.388+0000 7f16aa7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb84c9d1c0 0x55cb850c2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50896/0 (socket says 172.21.15.117:50896) 2024-09-21T06:00:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.388+0000 7f16aa7fc640 1 -- 172.21.15.117:0/3218303673 learned_addr learned my addr 172.21.15.117:0/3218303673 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.389+0000 7f16aa7fc640 1 -- 172.21.15.117:0/3218303673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb84ce02f0 msgr2=0x55cb850bf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.389+0000 7f16aa7fc640 1 --2- 172.21.15.117:0/3218303673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb84ce02f0 0x55cb850bf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.389+0000 7f16aa7fc640 1 -- 172.21.15.117:0/3218303673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb84ce84d0 msgr2=0x55cb850bf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.389+0000 7f16aa7fc640 1 --2- 172.21.15.117:0/3218303673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb84ce84d0 0x55cb850bf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.389+0000 7f16aa7fc640 1 -- 172.21.15.117:0/3218303673 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb850bfc50 con 0x55cb84c9d1c0 2024-09-21T06:00:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.389+0000 7f16aa7fc640 1 --2- 172.21.15.117:0/3218303673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb84c9d1c0 0x55cb850c2c80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f16a4009db0 tx=0x7f16a401c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21af4e7071d0a239 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.401+0000 7f168eff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:00:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.401+0000 7f168eff5640 1 -- 172.21.15.117:0/3218303673 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a403a040 con 0x55cb84c9d1c0 2024-09-21T06:00:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.401+0000 7f168eff5640 1 -- 172.21.15.117:0/3218303673 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f16a400cd30 con 0x55cb84c9d1c0 2024-09-21T06:00:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.401+0000 7f16b820f740 1 -- 172.21.15.117:0/3218303673 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16a400c7c0 con 0x55cb84c9d1c0 2024-09-21T06:00:29.411 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.409+0000 7f168eff5640 1 -- 172.21.15.117:0/3218303673 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f16a4023040 con 0x55cb84c9d1c0 2024-09-21T06:00:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.410+0000 7f16b820f740 10 client.16104 fetch_fsmap learned FSMap version 27 2024-09-21T06:00:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.410+0000 7f16b820f740 1 -- 172.21.15.117:0/3218303673 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cb85135ea0 con 0x55cb84c9d1c0 2024-09-21T06:00:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.411+0000 7f168eff5640 1 -- 172.21.15.117:0/3218303673 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16a400c7c0 con 0x55cb84c9d1c0 2024-09-21T06:00:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.411+0000 7f168eff5640 1 -- 172.21.15.117:0/3218303673 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f16a400b040 con 0x55cb84c9d1c0 2024-09-21T06:00:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.411+0000 7f16b820f740 10 client.16104 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:00:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.411+0000 7f16b820f740 10 client.16104 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:00:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.411+0000 7f16b820f740 20 client.16104 populate_metadata read hostname 'smithi117' 2024-09-21T06:00:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.411+0000 7f16b820f740 1 --2- 172.21.15.117:0/3218303673 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55cb851364c0 0x55cb851388e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.411+0000 7f16b820f740 4 client.16104 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:00:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.411+0000 7f16b820f740 1 -- 172.21.15.117:0/3218303673 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cb850210f0 con 0x55cb851364c0 2024-09-21T06:00:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.415+0000 7f16aaffd640 1 --2- 172.21.15.117:0/3218303673 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55cb851364c0 0x55cb851388e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.418+0000 7f16aaffd640 1 --2- 172.21.15.117:0/3218303673 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55cb851364c0 0x55cb851388e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f95afd73a428a08 server_cookie=5841e21c39c152de in_seq=0 out_seq=0 2024-09-21T06:00:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:29.419+0000 7f168eff5640 10 client.16104 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:00:30.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:29 smithi138 ceph-mon[34391]: pgmap v422: 97 pgs: 97 active+clean; 28 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 822 MiB/s wr, 2.41k op/s 2024-09-21T06:00:30.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:29 smithi192 ceph-mon[34700]: pgmap v422: 97 pgs: 97 active+clean; 28 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 822 MiB/s wr, 2.41k op/s 2024-09-21T06:00:30.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:29 smithi117 ceph-mon[30947]: pgmap v422: 97 pgs: 97 active+clean; 28 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 822 MiB/s wr, 2.41k op/s 2024-09-21T06:00:31.107 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:30 smithi117 ceph-mon[30947]: pgmap v423: 97 pgs: 97 active+clean; 29 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 819 MiB/s wr, 1.95k op/s 2024-09-21T06:00:31.109 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T06:00:31.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:30 smithi138 ceph-mon[34391]: pgmap v423: 97 pgs: 97 active+clean; 29 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 819 MiB/s wr, 1.95k op/s 2024-09-21T06:00:31.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:30 smithi192 ceph-mon[34700]: pgmap v423: 97 pgs: 97 active+clean; 29 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 819 MiB/s wr, 1.95k op/s 2024-09-21T06:00:31.749 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:00:33.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:33 smithi192 ceph-mon[34700]: pgmap v424: 97 pgs: 97 active+clean; 29 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 841 MiB/s wr, 1.60k op/s 2024-09-21T06:00:34.030 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:33 smithi117 ceph-mon[30947]: pgmap v424: 97 pgs: 97 active+clean; 29 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 841 MiB/s wr, 1.60k op/s 2024-09-21T06:00:34.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:33 smithi138 ceph-mon[34391]: pgmap v424: 97 pgs: 97 active+clean; 29 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 841 MiB/s wr, 1.60k op/s 2024-09-21T06:00:34.419 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:34.416+0000 7f168eff5640 1 -- 172.21.15.117:0/3218303673 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cb850210f0 con 0x55cb851364c0 2024-09-21T06:00:34.419 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:34.416+0000 7f168eff5640 10 client.16104 handle_command_reply: tid=0 2024-09-21T06:00:34.420 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:34.417+0000 7f16b820f740 1 client.16104 shutdown 2024-09-21T06:00:34.421 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:34.419+0000 7f16b820f740 1 -- 172.21.15.117:0/3218303673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb84c9d1c0 msgr2=0x55cb850c2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:34.421 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:34.419+0000 7f16b820f740 1 --2- 172.21.15.117:0/3218303673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb84c9d1c0 0x55cb850c2c80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f16a4009db0 tx=0x7f16a401c7b0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.422 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:34.419+0000 7f16b820f740 1 -- 172.21.15.117:0/3218303673 shutdown_connections 2024-09-21T06:00:34.422 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:34.419+0000 7f16b820f740 1 --2- 172.21.15.117:0/3218303673 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55cb851364c0 0x55cb851388e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.422 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:34.419+0000 7f16b820f740 1 --2- 172.21.15.117:0/3218303673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cb84ce84d0 0x55cb850bf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.422 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:34.419+0000 7f16b820f740 1 --2- 172.21.15.117:0/3218303673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cb84ce02f0 0x55cb850bf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.422 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:34.419+0000 7f16b820f740 1 --2- 172.21.15.117:0/3218303673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cb84c9d1c0 0x55cb850c2c80 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.423 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:34.419+0000 7f16b820f740 1 -- 172.21.15.117:0/3218303673 >> 172.21.15.117:0/3218303673 conn(0x55cb84cfd300 msgr2=0x55cb850b8600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:34.423 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:34.419+0000 7f168eff5640 0 client.16104 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:00:34.423 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:34.420+0000 7f16b820f740 1 -- 172.21.15.117:0/3218303673 shutdown_connections 2024-09-21T06:00:34.423 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:34.420+0000 7f16b820f740 1 -- 172.21.15.117:0/3218303673 wait complete. 2024-09-21T06:00:34.423 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:34.420+0000 7f16b820f740 20 client.16104 trim_cache size 0 max 16384 2024-09-21T06:00:34.423 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T06:00:34.424 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T06:00:34.424 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T06:00:34.424 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T06:00:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:34.424+0000 7f16abfff640 1 -- 172.21.15.117:0/359776586 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f16940b68c0 msgr2=0x7f16940b8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:34.424+0000 7f16abfff640 1 --2- 172.21.15.117:0/359776586 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f16940b68c0 0x7f16940b8dd0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f16a00aba50 tx=0x7f16a0009310 comp rx=0 tx=0).stop 2024-09-21T06:00:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:34.424+0000 7f16abfff640 1 -- 172.21.15.117:0/359776586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16ac116510 msgr2=0x7f16ac1ae960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:34.424+0000 7f16abfff640 1 --2- 172.21.15.117:0/359776586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16ac116510 0x7f16ac1ae960 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f16a000e970 tx=0x7f16a000ee30 comp rx=0 tx=0).stop 2024-09-21T06:00:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:34.424+0000 7f16abfff640 1 -- 172.21.15.117:0/359776586 shutdown_connections 2024-09-21T06:00:34.426 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:34.424+0000 7f16abfff640 1 --2- 172.21.15.117:0/359776586 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f16940b68c0 0x7f16940b8dd0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:34.424+0000 7f16abfff640 1 --2- 172.21.15.117:0/359776586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16ac116510 0x7f16ac1ae960 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:34.424+0000 7f16abfff640 1 --2- 172.21.15.117:0/359776586 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16ac10ad10 0x7f16ac1a96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:34.424+0000 7f16abfff640 1 --2- 172.21.15.117:0/359776586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16ac10a3c0 0x7f16ac1a9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:34.424+0000 7f16abfff640 1 -- 172.21.15.117:0/359776586 >> 172.21.15.117:0/359776586 conn(0x7f16ac070c00 msgr2=0x7f16ac1100f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:34.428 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:34.425+0000 7f16abfff640 1 -- 172.21.15.117:0/359776586 shutdown_connections 2024-09-21T06:00:34.428 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:34.425+0000 7f16abfff640 1 -- 172.21.15.117:0/359776586 wait complete. 2024-09-21T06:00:34.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.379+0000 7fd934e67640 1 -- 172.21.15.117:0/2631221869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd930114770 msgr2=0x7fd930116b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:34.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.379+0000 7fd934e67640 1 --2- 172.21.15.117:0/2631221869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd930114770 0x7fd930116b90 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd928009ca0 tx=0x7fd92802a8d0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.379+0000 7fd934e67640 1 -- 172.21.15.117:0/2631221869 shutdown_connections 2024-09-21T06:00:34.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.379+0000 7fd934e67640 1 --2- 172.21.15.117:0/2631221869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd930114770 0x7fd930116b90 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.379+0000 7fd934e67640 1 --2- 172.21.15.117:0/2631221869 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd930078290 0x7fd930114230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.379+0000 7fd934e67640 1 --2- 172.21.15.117:0/2631221869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd930077940 0x7fd930077d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.379+0000 7fd934e67640 1 -- 172.21.15.117:0/2631221869 >> 172.21.15.117:0/2631221869 conn(0x7fd9301017c0 msgr2=0x7fd930103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:34.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.379+0000 7fd934e67640 1 -- 172.21.15.117:0/2631221869 shutdown_connections 2024-09-21T06:00:34.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.379+0000 7fd934e67640 1 -- 172.21.15.117:0/2631221869 wait complete. 2024-09-21T06:00:34.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.380+0000 7fd934e67640 1 Processor -- start 2024-09-21T06:00:34.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.380+0000 7fd934e67640 1 -- start start 2024-09-21T06:00:34.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.381+0000 7fd934e67640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd930077940 0x7fd9301a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:34.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.381+0000 7fd934e67640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd930078290 0x7fd9301a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:34.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.381+0000 7fd934e67640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd930114770 0x7fd9301a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:34.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.381+0000 7fd934e67640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd93011a200 con 0x7fd930077940 2024-09-21T06:00:34.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.381+0000 7fd934e67640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd93011a080 con 0x7fd930078290 2024-09-21T06:00:34.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.381+0000 7fd934e67640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd93011a380 con 0x7fd930114770 2024-09-21T06:00:34.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.381+0000 7fd92e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd930077940 0x7fd9301a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:34.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.381+0000 7fd92e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd930077940 0x7fd9301a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50916/0 (socket says 172.21.15.117:50916) 2024-09-21T06:00:34.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.381+0000 7fd92e59c640 1 -- 172.21.15.117:0/1940201489 learned_addr learned my addr 172.21.15.117:0/1940201489 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:34.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.381+0000 7fd92ed9d640 1 --2- 172.21.15.117:0/1940201489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd930114770 0x7fd9301a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:34.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.382+0000 7fd92e59c640 1 -- 172.21.15.117:0/1940201489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd930114770 msgr2=0x7fd9301a63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:34.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.382+0000 7fd92e59c640 1 --2- 172.21.15.117:0/1940201489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd930114770 0x7fd9301a63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.382+0000 7fd92e59c640 1 -- 172.21.15.117:0/1940201489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd930078290 msgr2=0x7fd9301a1150 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:00:34.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.382+0000 7fd92e59c640 1 --2- 172.21.15.117:0/1940201489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd930078290 0x7fd9301a1150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.382+0000 7fd92e59c640 1 -- 172.21.15.117:0/1940201489 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9301a6b20 con 0x7fd930077940 2024-09-21T06:00:34.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.382+0000 7fd92ed9d640 1 --2- 172.21.15.117:0/1940201489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd930114770 0x7fd9301a63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:00:34.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.382+0000 7fd92e59c640 1 --2- 172.21.15.117:0/1940201489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd930077940 0x7fd9301a0bb0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd92000b4a0 tx=0x7fd92000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:34.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.382+0000 7fd91f7fe640 1 -- 172.21.15.117:0/1940201489 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd920014070 con 0x7fd930077940 2024-09-21T06:00:34.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.383+0000 7fd91f7fe640 1 -- 172.21.15.117:0/1940201489 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd920010ac0 con 0x7fd930077940 2024-09-21T06:00:34.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.383+0000 7fd934e67640 1 -- 172.21.15.117:0/1940201489 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9301a6e10 con 0x7fd930077940 2024-09-21T06:00:34.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.383+0000 7fd91f7fe640 1 -- 172.21.15.117:0/1940201489 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd920017c30 con 0x7fd930077940 2024-09-21T06:00:34.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.383+0000 7fd934e67640 1 -- 172.21.15.117:0/1940201489 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9301ad570 con 0x7fd930077940 2024-09-21T06:00:34.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.386+0000 7fd91f7fe640 1 -- 172.21.15.117:0/1940201489 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fd920017070 con 0x7fd930077940 2024-09-21T06:00:34.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.387+0000 7fd93b3db740 1 Processor -- start 2024-09-21T06:00:34.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.387+0000 7fd93b3db740 1 -- start start 2024-09-21T06:00:34.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.387+0000 7fd93b3db740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569fedef070 0x5569ff1966a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:34.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.388+0000 7fd93b3db740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fedc61b0 0x5569ff196be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:34.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.388+0000 7fd93b3db740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569fedce390 0x5569ff197120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:34.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.388+0000 7fd93b3db740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5569ff10cea0 con 0x5569fedef070 2024-09-21T06:00:34.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.388+0000 7fd93b3db740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5569ff0fc2d0 con 0x5569fedc61b0 2024-09-21T06:00:34.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.388+0000 7fd93b3db740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5569ff1000e0 con 0x5569fedce390 2024-09-21T06:00:34.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.388+0000 7fd92ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569fedef070 0x5569ff1966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:34.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.388+0000 7fd92ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569fedef070 0x5569ff1966a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50932/0 (socket says 172.21.15.117:50932) 2024-09-21T06:00:34.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.388+0000 7fd92ed9d640 1 -- 172.21.15.117:0/1918698538 learned_addr learned my addr 172.21.15.117:0/1918698538 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:34.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.388+0000 7fd91f7fe640 1 --2- 172.21.15.117:0/1940201489 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd9180b67d0 0x7fd9180b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:34.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.388+0000 7fd92e59c640 1 --2- 172.21.15.117:0/1918698538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fedc61b0 0x5569ff196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:34.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.388+0000 7fd92ed9d640 1 --2- 172.21.15.117:0/1940201489 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd9180b67d0 0x7fd9180b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:34.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.389+0000 7fd92ed9d640 1 -- 172.21.15.117:0/1918698538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569fedce390 msgr2=0x5569ff197120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:00:34.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.389+0000 7fd92ed9d640 1 --2- 172.21.15.117:0/1918698538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569fedce390 0x5569ff197120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.389+0000 7fd92ed9d640 1 -- 172.21.15.117:0/1918698538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fedc61b0 msgr2=0x5569ff196be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:34.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.389+0000 7fd92ed9d640 1 --2- 172.21.15.117:0/1918698538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fedc61b0 0x5569ff196be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.389+0000 7fd92ed9d640 1 -- 172.21.15.117:0/1918698538 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569fee5a650 con 0x5569fedef070 2024-09-21T06:00:34.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.389+0000 7fd91f7fe640 1 -- 172.21.15.117:0/1940201489 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fd9200e68f0 con 0x7fd930077940 2024-09-21T06:00:34.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.389+0000 7fd92ed9d640 1 --2- 172.21.15.117:0/1918698538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569fedef070 0x5569ff1966a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd928035000 tx=0x7fd928031f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ccf83c9a4a75de server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:34.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.390+0000 7fd91cff9640 1 -- 172.21.15.117:0/1918698538 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd928031760 con 0x5569fedef070 2024-09-21T06:00:34.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.390+0000 7fd91cff9640 1 -- 172.21.15.117:0/1918698538 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd92804e070 con 0x5569fedef070 2024-09-21T06:00:34.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.390+0000 7fd91cff9640 1 -- 172.21.15.117:0/1918698538 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9280046f0 con 0x5569fedef070 2024-09-21T06:00:34.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.391+0000 7fd93b3db740 1 -- 172.21.15.117:0/1918698538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569fedef070 msgr2=0x5569ff1966a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:34.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.391+0000 7fd93b3db740 1 --2- 172.21.15.117:0/1918698538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569fedef070 0x5569ff1966a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd928035000 tx=0x7fd928031f50 comp rx=0 tx=0).stop 2024-09-21T06:00:34.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.391+0000 7fd92ed9d640 1 --2- 172.21.15.117:0/1940201489 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd9180b67d0 0x7fd9180b8ce0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd92804d000 tx=0x7fd92804b000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:34.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.392+0000 7fd93b3db740 1 -- 172.21.15.117:0/1918698538 shutdown_connections 2024-09-21T06:00:34.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.392+0000 7fd93b3db740 1 --2- 172.21.15.117:0/1918698538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569fedce390 0x5569ff197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.392+0000 7fd93b3db740 1 --2- 172.21.15.117:0/1918698538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fedc61b0 0x5569ff196be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.392+0000 7fd93b3db740 1 --2- 172.21.15.117:0/1918698538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569fedef070 0x5569ff1966a0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.392+0000 7fd93b3db740 1 -- 172.21.15.117:0/1918698538 >> 172.21.15.117:0/1918698538 conn(0x5569fede3460 msgr2=0x5569ff194590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:34.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.392+0000 7fd93b3db740 1 -- 172.21.15.117:0/1918698538 shutdown_connections 2024-09-21T06:00:34.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.392+0000 7fd93b3db740 1 -- 172.21.15.117:0/1918698538 wait complete. 2024-09-21T06:00:34.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.393+0000 7fd93b3db740 1 -- start start 2024-09-21T06:00:34.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.393+0000 7fd93b3db740 1 Processor -- start 2024-09-21T06:00:34.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.394+0000 7fd93b3db740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fedc61b0 0x5569ff1a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:34.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.394+0000 7fd93b3db740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569fedce390 0x5569ff1a22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:34.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.394+0000 7fd93b3db740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569fedef070 0x5569ff1ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:34.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.394+0000 7fd93b3db740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5569fef5d700 con 0x5569fedef070 2024-09-21T06:00:34.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.394+0000 7fd93b3db740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5569ff097160 con 0x5569fedc61b0 2024-09-21T06:00:34.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.394+0000 7fd93b3db740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5569ff1a4e80 con 0x5569fedce390 2024-09-21T06:00:34.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.394+0000 7fd93b3db740 10 client.0 _ll_register_callbacks cb 0x5569ff0b09e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:00:34.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.394+0000 7fd92ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569fedef070 0x5569ff1ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:34.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.394+0000 7fd92ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569fedef070 0x5569ff1ae170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50948/0 (socket says 172.21.15.117:50948) 2024-09-21T06:00:34.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.394+0000 7fd92ed9d640 1 -- 172.21.15.117:0/906570306 learned_addr learned my addr 172.21.15.117:0/906570306 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:34.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.395+0000 7fd92ed9d640 1 -- 172.21.15.117:0/906570306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569fedce390 msgr2=0x5569ff1a22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:34.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.395+0000 7fd92ed9d640 1 --2- 172.21.15.117:0/906570306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569fedce390 0x5569ff1a22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.395+0000 7fd92ed9d640 1 -- 172.21.15.117:0/906570306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fedc61b0 msgr2=0x5569ff1a1db0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:00:34.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.395+0000 7fd92ed9d640 1 --2- 172.21.15.117:0/906570306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fedc61b0 0x5569ff1a1db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.395+0000 7fd92ed9d640 1 -- 172.21.15.117:0/906570306 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569ff0c4c90 con 0x5569fedef070 2024-09-21T06:00:34.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.395+0000 7fd92ed9d640 1 --2- 172.21.15.117:0/906570306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569fedef070 0x5569ff1ae170 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd928004450 tx=0x7fd928005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd156fdb2093cd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:34.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.395+0000 7fd915ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:00:34.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.395+0000 7fd915ffb640 1 -- 172.21.15.117:0/906570306 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd928060060 con 0x5569fedef070 2024-09-21T06:00:34.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.395+0000 7fd915ffb640 1 -- 172.21.15.117:0/906570306 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fd928003e00 con 0x5569fedef070 2024-09-21T06:00:34.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.396+0000 7fd93b3db740 1 -- 172.21.15.117:0/906570306 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd92804e070 con 0x5569fedef070 2024-09-21T06:00:34.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.396+0000 7fd915ffb640 1 -- 172.21.15.117:0/906570306 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd9280044f0 con 0x5569fedef070 2024-09-21T06:00:34.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.396+0000 7fd93b3db740 10 client.16116 fetch_fsmap learned FSMap version 27 2024-09-21T06:00:34.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.396+0000 7fd93b3db740 1 -- 172.21.15.117:0/906570306 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5569ff1b0b90 con 0x5569fedef070 2024-09-21T06:00:34.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.397+0000 7fd915ffb640 1 -- 172.21.15.117:0/906570306 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd92804e070 con 0x5569fedef070 2024-09-21T06:00:34.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.397+0000 7fd915ffb640 1 -- 172.21.15.117:0/906570306 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd92802fbf0 con 0x5569fedef070 2024-09-21T06:00:34.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.398+0000 7fd93b3db740 10 client.16116 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:00:34.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.398+0000 7fd93b3db740 10 client.16116 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T06:00:34.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.398+0000 7fd93b3db740 20 client.16116 populate_metadata read hostname 'smithi117' 2024-09-21T06:00:34.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.398+0000 7fd93b3db740 1 --2- 172.21.15.117:0/906570306 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x5569fedad9f0 0x5569ff1b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:34.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.398+0000 7fd93b3db740 4 client.16116 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:00:34.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.398+0000 7fd93b3db740 1 -- 172.21.15.117:0/906570306 --> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569ff10edf0 con 0x5569fedad9f0 2024-09-21T06:00:34.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.400+0000 7fd92e59c640 1 --2- 172.21.15.117:0/906570306 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x5569fedad9f0 0x5569ff1b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:34.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.404+0000 7fd92e59c640 1 --2- 172.21.15.117:0/906570306 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x5569fedad9f0 0x5569ff1b3ef0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=50b7666d2e6f3f4 server_cookie=3075ca5e489b8fd3 in_seq=0 out_seq=0 2024-09-21T06:00:34.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.404+0000 7fd915ffb640 10 client.16116 ms_handle_connect on v2:172.21.15.138:6836/2540335374 2024-09-21T06:00:34.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.407+0000 7fd915ffb640 1 -- 172.21.15.117:0/906570306 <== mds.4 v2:172.21.15.138:6836/2540335374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5569ff10edf0 con 0x5569fedad9f0 2024-09-21T06:00:34.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.407+0000 7fd915ffb640 10 client.16116 handle_command_reply: tid=0 2024-09-21T06:00:34.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.407+0000 7fd93b3db740 1 client.16116 shutdown 2024-09-21T06:00:34.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.407+0000 7fd93b3db740 1 -- 172.21.15.117:0/906570306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569fedef070 msgr2=0x5569ff1ae170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:34.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.407+0000 7fd93b3db740 1 --2- 172.21.15.117:0/906570306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569fedef070 0x5569ff1ae170 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd928004450 tx=0x7fd928005b50 comp rx=0 tx=0).stop 2024-09-21T06:00:34.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.407+0000 7fd93b3db740 1 -- 172.21.15.117:0/906570306 shutdown_connections 2024-09-21T06:00:34.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.408+0000 7fd93b3db740 1 --2- 172.21.15.117:0/906570306 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x5569fedad9f0 0x5569ff1b3ef0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.408+0000 7fd915ffb640 0 client.16116 ms_handle_reset on v2:172.21.15.138:6836/2540335374 2024-09-21T06:00:34.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.408+0000 7fd93b3db740 1 --2- 172.21.15.117:0/906570306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569fedef070 0x5569ff1ae170 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.408+0000 7fd93b3db740 1 --2- 172.21.15.117:0/906570306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569fedce390 0x5569ff1a22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.408+0000 7fd93b3db740 1 --2- 172.21.15.117:0/906570306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fedc61b0 0x5569ff1a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.408+0000 7fd93b3db740 1 -- 172.21.15.117:0/906570306 >> 172.21.15.117:0/906570306 conn(0x5569fede3460 msgr2=0x5569ff199880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:34.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.408+0000 7fd93b3db740 1 -- 172.21.15.117:0/906570306 shutdown_connections 2024-09-21T06:00:34.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.408+0000 7fd93b3db740 1 -- 172.21.15.117:0/906570306 wait complete. 2024-09-21T06:00:34.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.408+0000 7fd93b3db740 20 client.16116 trim_cache size 0 max 16384 2024-09-21T06:00:34.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.433+0000 7fd93b3db740 1 Processor -- start 2024-09-21T06:00:34.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.433+0000 7fd93b3db740 1 -- start start 2024-09-21T06:00:34.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.433+0000 7fd93b3db740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569fedad9f0 0x5569ff198090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:34.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.433+0000 7fd93b3db740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569fedc61b0 0x5569ff1a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:34.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.433+0000 7fd93b3db740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fedce390 0x5569ff1ae0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:34.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.433+0000 7fd93b3db740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd9301b5d10 con 0x5569fedc61b0 2024-09-21T06:00:34.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.433+0000 7fd93b3db740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd9301b7850 con 0x5569fedce390 2024-09-21T06:00:34.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.433+0000 7fd93b3db740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd9301b63e0 con 0x5569fedad9f0 2024-09-21T06:00:34.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.434+0000 7fd92dd9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569fedc61b0 0x5569ff1a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:34.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.434+0000 7fd92e59c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569fedad9f0 0x5569ff198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:34.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.434+0000 7fd92e59c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569fedad9f0 0x5569ff198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:46392/0 (socket says 172.21.15.117:46392) 2024-09-21T06:00:34.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.434+0000 7fd92e59c640 1 -- 172.21.15.117:0/3487401641 learned_addr learned my addr 172.21.15.117:0/3487401641 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:34.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.434+0000 7fd92e59c640 1 -- 172.21.15.117:0/3487401641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fedce390 msgr2=0x5569ff1ae0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:34.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.434+0000 7fd92e59c640 1 --2- 172.21.15.117:0/3487401641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fedce390 0x5569ff1ae0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.434+0000 7fd92e59c640 1 -- 172.21.15.117:0/3487401641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569fedc61b0 msgr2=0x5569ff1a39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:34.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.434+0000 7fd92e59c640 1 --2- 172.21.15.117:0/3487401641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569fedc61b0 0x5569ff1a39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.434+0000 7fd92e59c640 1 -- 172.21.15.117:0/3487401641 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569ff1a4980 con 0x5569fedad9f0 2024-09-21T06:00:34.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.435+0000 7fd92e59c640 1 --2- 172.21.15.117:0/3487401641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569fedad9f0 0x5569ff198090 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd92000b470 tx=0x7fd9200783f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f4cfd85b1fd9ef7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:34.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.435+0000 7fd917fff640 1 -- 172.21.15.117:0/3487401641 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92001e040 con 0x5569fedad9f0 2024-09-21T06:00:34.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.435+0000 7fd917fff640 1 -- 172.21.15.117:0/3487401641 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd920070d90 con 0x5569fedad9f0 2024-09-21T06:00:34.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.435+0000 7fd917fff640 1 -- 172.21.15.117:0/3487401641 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd920010c70 con 0x5569fedad9f0 2024-09-21T06:00:34.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.437+0000 7fd93b3db740 1 -- 172.21.15.117:0/3487401641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569fedad9f0 msgr2=0x5569ff198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:34.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.437+0000 7fd93b3db740 1 --2- 172.21.15.117:0/3487401641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569fedad9f0 0x5569ff198090 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd92000b470 tx=0x7fd9200783f0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.437+0000 7fd93b3db740 1 -- 172.21.15.117:0/3487401641 shutdown_connections 2024-09-21T06:00:34.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.437+0000 7fd93b3db740 1 --2- 172.21.15.117:0/3487401641 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569fedc61b0 0x5569ff1a39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.437+0000 7fd93b3db740 1 --2- 172.21.15.117:0/3487401641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fedce390 0x5569ff1ae0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.437+0000 7fd93b3db740 1 --2- 172.21.15.117:0/3487401641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569fedad9f0 0x5569ff198090 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.437+0000 7fd93b3db740 1 -- 172.21.15.117:0/3487401641 >> 172.21.15.117:0/3487401641 conn(0x5569fede3460 msgr2=0x5569ff1a32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:34.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.438+0000 7fd93b3db740 1 -- 172.21.15.117:0/3487401641 shutdown_connections 2024-09-21T06:00:34.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.438+0000 7fd93b3db740 1 -- 172.21.15.117:0/3487401641 wait complete. 2024-09-21T06:00:34.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.439+0000 7fd93b3db740 1 -- start start 2024-09-21T06:00:34.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.439+0000 7fd93b3db740 1 Processor -- start 2024-09-21T06:00:34.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.440+0000 7fd93b3db740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569fedad9f0 0x5569ff212600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:34.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.440+0000 7fd93b3db740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569fedc61b0 0x5569ff212b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:34.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.440+0000 7fd93b3db740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fedce390 0x5569ff21d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:34.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.440+0000 7fd93b3db740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd9301b7180 con 0x5569fedad9f0 2024-09-21T06:00:34.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.440+0000 7fd93b3db740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd9301b6ab0 con 0x5569fedce390 2024-09-21T06:00:34.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.440+0000 7fd93b3db740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd9301b3cf0 con 0x5569fedc61b0 2024-09-21T06:00:34.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.440+0000 7fd93b3db740 10 client.0 _ll_register_callbacks cb 0x7fd928031da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:00:34.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.441+0000 7fd92ed9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fedce390 0x5569ff21d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:34.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.441+0000 7fd92ed9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fedce390 0x5569ff21d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:48240/0 (socket says 172.21.15.117:48240) 2024-09-21T06:00:34.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.441+0000 7fd92ed9d640 1 -- 172.21.15.117:0/509410541 learned_addr learned my addr 172.21.15.117:0/509410541 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:34.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.443+0000 7fd92e59c640 1 --2- 172.21.15.117:0/509410541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569fedad9f0 0x5569ff212600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:34.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.443+0000 7fd92dd9b640 1 --2- 172.21.15.117:0/509410541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569fedc61b0 0x5569ff212b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:34.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.444+0000 7fd92dd9b640 1 -- 172.21.15.117:0/509410541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fedce390 msgr2=0x5569ff21d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:34.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.444+0000 7fd92dd9b640 1 --2- 172.21.15.117:0/509410541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fedce390 0x5569ff21d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.444+0000 7fd92dd9b640 1 -- 172.21.15.117:0/509410541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569fedad9f0 msgr2=0x5569ff212600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:34.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.444+0000 7fd92dd9b640 1 --2- 172.21.15.117:0/509410541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569fedad9f0 0x5569ff212600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.444+0000 7fd92dd9b640 1 -- 172.21.15.117:0/509410541 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569ff213350 con 0x5569fedc61b0 2024-09-21T06:00:34.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.445+0000 7fd92dd9b640 1 --2- 172.21.15.117:0/509410541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569fedc61b0 0x5569ff212b40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x5569ff18fe50 tx=0x7fd92400f5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10b9c31ee48b96c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:34.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.445+0000 7fd915ffb640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-21T06:00:34.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.445+0000 7fd915ffb640 1 -- 172.21.15.117:0/509410541 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd924019020 con 0x5569fedc61b0 2024-09-21T06:00:34.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.445+0000 7fd915ffb640 1 -- 172.21.15.117:0/509410541 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fd924014460 con 0x5569fedc61b0 2024-09-21T06:00:34.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.446+0000 7fd93b3db740 1 -- 172.21.15.117:0/509410541 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd920070d90 con 0x5569fedc61b0 2024-09-21T06:00:34.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.447+0000 7fd915ffb640 1 -- 172.21.15.117:0/509410541 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd92400de40 con 0x5569fedc61b0 2024-09-21T06:00:34.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.448+0000 7fd93b3db740 10 client.25495 fetch_fsmap learned FSMap version 27 2024-09-21T06:00:34.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.448+0000 7fd93b3db740 1 -- 172.21.15.117:0/509410541 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5569ff21ebe0 con 0x5569fedc61b0 2024-09-21T06:00:34.871 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.449+0000 7fd915ffb640 1 -- 172.21.15.117:0/509410541 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd920070d90 con 0x5569fedc61b0 2024-09-21T06:00:34.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.449+0000 7fd915ffb640 1 -- 172.21.15.117:0/509410541 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd9240158d0 con 0x5569fedc61b0 2024-09-21T06:00:34.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.449+0000 7fd93b3db740 10 client.25495 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:00:34.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.449+0000 7fd93b3db740 10 client.25495 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T06:00:34.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.449+0000 7fd93b3db740 20 client.25495 populate_metadata read hostname 'smithi117' 2024-09-21T06:00:34.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.449+0000 7fd93b3db740 1 --2- 172.21.15.117:0/509410541 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x5569fedef070 0x5569ff221370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:34.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.449+0000 7fd93b3db740 4 client.25495 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:00:34.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.449+0000 7fd93b3db740 1 -- 172.21.15.117:0/509410541 --> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5569ff10a8a0 con 0x5569fedef070 2024-09-21T06:00:34.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.449+0000 7fd92e59c640 1 --2- 172.21.15.117:0/509410541 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x5569fedef070 0x5569ff221370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:34.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.451+0000 7fd915ffb640 10 client.25495 ms_handle_connect on v2:172.21.15.138:6836/2540335374 2024-09-21T06:00:34.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.451+0000 7fd92e59c640 1 --2- 172.21.15.117:0/509410541 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x5569fedef070 0x5569ff221370 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=edef2f1aac41f35b server_cookie=393ef12bc19d82b in_seq=0 out_seq=0 2024-09-21T06:00:34.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.452+0000 7fd915ffb640 1 -- 172.21.15.117:0/509410541 <== mds.4 v2:172.21.15.138:6836/2540335374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5569ff10a8a0 con 0x5569fedef070 2024-09-21T06:00:34.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.452+0000 7fd915ffb640 10 client.25495 handle_command_reply: tid=0 2024-09-21T06:00:34.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.452+0000 7fd93b3db740 1 client.25495 shutdown 2024-09-21T06:00:34.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.453+0000 7fd93b3db740 1 -- 172.21.15.117:0/509410541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569fedc61b0 msgr2=0x5569ff212b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:34.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.453+0000 7fd93b3db740 1 --2- 172.21.15.117:0/509410541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569fedc61b0 0x5569ff212b40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x5569ff18fe50 tx=0x7fd92400f5b0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.453+0000 7fd93b3db740 1 -- 172.21.15.117:0/509410541 shutdown_connections 2024-09-21T06:00:34.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.454+0000 7fd93b3db740 1 --2- 172.21.15.117:0/509410541 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x5569fedef070 0x5569ff221370 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.454+0000 7fd93b3db740 1 --2- 172.21.15.117:0/509410541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fedce390 0x5569ff21d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.454+0000 7fd93b3db740 1 --2- 172.21.15.117:0/509410541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569fedc61b0 0x5569ff212b40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.454+0000 7fd93b3db740 1 --2- 172.21.15.117:0/509410541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5569fedad9f0 0x5569ff212600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.454+0000 7fd93b3db740 1 -- 172.21.15.117:0/509410541 >> 172.21.15.117:0/509410541 conn(0x5569fede3460 msgr2=0x5569ff1a3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:34.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.454+0000 7fd915ffb640 0 client.25495 ms_handle_reset on v2:172.21.15.138:6836/2540335374 2024-09-21T06:00:34.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.455+0000 7fd93b3db740 1 -- 172.21.15.117:0/509410541 shutdown_connections 2024-09-21T06:00:34.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.455+0000 7fd93b3db740 1 -- 172.21.15.117:0/509410541 wait complete. 2024-09-21T06:00:34.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.455+0000 7fd93b3db740 20 client.25495 trim_cache size 0 max 16384 2024-09-21T06:00:34.873 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:00:34.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.457+0000 7fd934e67640 1 -- 172.21.15.117:0/1940201489 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd9180b67d0 msgr2=0x7fd9180b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:34.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.457+0000 7fd934e67640 1 --2- 172.21.15.117:0/1940201489 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd9180b67d0 0x7fd9180b8ce0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd92804d000 tx=0x7fd92804b000 comp rx=0 tx=0).stop 2024-09-21T06:00:34.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.457+0000 7fd934e67640 1 -- 172.21.15.117:0/1940201489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd930077940 msgr2=0x7fd9301a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:34.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.457+0000 7fd934e67640 1 --2- 172.21.15.117:0/1940201489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd930077940 0x7fd9301a0bb0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd92000b4a0 tx=0x7fd92000b960 comp rx=0 tx=0).stop 2024-09-21T06:00:34.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.457+0000 7fd934e67640 1 -- 172.21.15.117:0/1940201489 shutdown_connections 2024-09-21T06:00:34.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.457+0000 7fd934e67640 1 --2- 172.21.15.117:0/1940201489 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd9180b67d0 0x7fd9180b8ce0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.457+0000 7fd934e67640 1 --2- 172.21.15.117:0/1940201489 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd930114770 0x7fd9301a63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.457+0000 7fd934e67640 1 --2- 172.21.15.117:0/1940201489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd930078290 0x7fd9301a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.457+0000 7fd934e67640 1 --2- 172.21.15.117:0/1940201489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd930077940 0x7fd9301a0bb0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:34.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.457+0000 7fd934e67640 1 -- 172.21.15.117:0/1940201489 >> 172.21.15.117:0/1940201489 conn(0x7fd9301017c0 msgr2=0x7fd930103be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:34.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.457+0000 7fd934e67640 1 -- 172.21.15.117:0/1940201489 shutdown_connections 2024-09-21T06:00:34.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:34.458+0000 7fd934e67640 1 -- 172.21.15.117:0/1940201489 wait complete. 2024-09-21T06:00:35.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:35 smithi192 ceph-mon[34700]: pgmap v425: 97 pgs: 97 active+clean; 30 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 973 MiB/s wr, 1.54k op/s 2024-09-21T06:00:36.029 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:35 smithi117 ceph-mon[30947]: pgmap v425: 97 pgs: 97 active+clean; 30 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 973 MiB/s wr, 1.54k op/s 2024-09-21T06:00:36.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:35 smithi138 ceph-mon[34391]: pgmap v425: 97 pgs: 97 active+clean; 30 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 973 MiB/s wr, 1.54k op/s 2024-09-21T06:00:38.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:37 smithi117 ceph-mon[30947]: pgmap v426: 97 pgs: 97 active+clean; 30 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 852 MiB/s wr, 1.37k op/s 2024-09-21T06:00:38.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:37 smithi138 ceph-mon[34391]: pgmap v426: 97 pgs: 97 active+clean; 30 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 852 MiB/s wr, 1.37k op/s 2024-09-21T06:00:38.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:37 smithi192 ceph-mon[34700]: pgmap v426: 97 pgs: 97 active+clean; 30 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 852 MiB/s wr, 1.37k op/s 2024-09-21T06:00:39.107 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T06:00:39.957 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:00:40.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:39 smithi117 ceph-mon[30947]: pgmap v427: 97 pgs: 97 active+clean; 30 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 894 MiB/s wr, 1.43k op/s 2024-09-21T06:00:40.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:39 smithi138 ceph-mon[34391]: pgmap v427: 97 pgs: 97 active+clean; 30 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 894 MiB/s wr, 1.43k op/s 2024-09-21T06:00:40.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:39 smithi192 ceph-mon[34700]: pgmap v427: 97 pgs: 97 active+clean; 30 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 894 MiB/s wr, 1.43k op/s 2024-09-21T06:00:42.031 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:41 smithi117 ceph-mon[30947]: pgmap v428: 97 pgs: 97 active+clean; 30 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 873 MiB/s wr, 1.32k op/s 2024-09-21T06:00:42.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:41 smithi138 ceph-mon[34391]: pgmap v428: 97 pgs: 97 active+clean; 30 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 873 MiB/s wr, 1.32k op/s 2024-09-21T06:00:42.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:41 smithi192 ceph-mon[34700]: pgmap v428: 97 pgs: 97 active+clean; 30 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 873 MiB/s wr, 1.32k op/s 2024-09-21T06:00:42.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.591+0000 7fec79895640 1 -- 172.21.15.117:0/2514715773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec74111d40 msgr2=0x7fec7410a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:42.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.591+0000 7fec79895640 1 --2- 172.21.15.117:0/2514715773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec74111d40 0x7fec7410a640 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fec6c009ca0 tx=0x7fec6c02a8f0 comp rx=0 tx=0).stop 2024-09-21T06:00:42.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.592+0000 7fec79895640 1 -- 172.21.15.117:0/2514715773 shutdown_connections 2024-09-21T06:00:42.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.592+0000 7fec79895640 1 --2- 172.21.15.117:0/2514715773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec7411cba0 0x7fec7411efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:42.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.592+0000 7fec79895640 1 --2- 172.21.15.117:0/2514715773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec7410ab80 0x7fec7410af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:42.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.592+0000 7fec79895640 1 --2- 172.21.15.117:0/2514715773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec74111d40 0x7fec7410a640 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:42.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.592+0000 7fec79895640 1 -- 172.21.15.117:0/2514715773 >> 172.21.15.117:0/2514715773 conn(0x7fec7406d950 msgr2=0x7fec7406dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:42.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.597+0000 7fec79895640 1 -- 172.21.15.117:0/2514715773 shutdown_connections 2024-09-21T06:00:42.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.597+0000 7fec79895640 1 -- 172.21.15.117:0/2514715773 wait complete. 2024-09-21T06:00:42.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.598+0000 7fec79895640 1 Processor -- start 2024-09-21T06:00:42.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.601+0000 7fec79895640 1 -- start start 2024-09-21T06:00:42.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.601+0000 7fec79895640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec7410ab80 0x7fec74112670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:42.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.601+0000 7fec79895640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec74111d40 0x7fec74112bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:42.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.601+0000 7fec79895640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec7411cba0 0x7fec74117eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:42.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.601+0000 7fec79895640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fec74122830 con 0x7fec7411cba0 2024-09-21T06:00:42.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.601+0000 7fec79895640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fec741226b0 con 0x7fec74111d40 2024-09-21T06:00:42.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.601+0000 7fec79895640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fec741229b0 con 0x7fec7410ab80 2024-09-21T06:00:42.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.602+0000 7fec72ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec7410ab80 0x7fec74112670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:42.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.602+0000 7fec72ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec7410ab80 0x7fec74112670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:35176/0 (socket says 172.21.15.117:35176) 2024-09-21T06:00:42.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.602+0000 7fec72ffd640 1 -- 172.21.15.117:0/1234671022 learned_addr learned my addr 172.21.15.117:0/1234671022 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:42.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.604+0000 7fec72ffd640 1 -- 172.21.15.117:0/1234671022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec74111d40 msgr2=0x7fec74112bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:42.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.604+0000 7fec72ffd640 1 --2- 172.21.15.117:0/1234671022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec74111d40 0x7fec74112bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:42.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.604+0000 7fec72ffd640 1 -- 172.21.15.117:0/1234671022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec7411cba0 msgr2=0x7fec74117eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:42.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.604+0000 7fec72ffd640 1 --2- 172.21.15.117:0/1234671022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec7411cba0 0x7fec74117eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:42.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.604+0000 7fec72ffd640 1 -- 172.21.15.117:0/1234671022 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec741185e0 con 0x7fec7410ab80 2024-09-21T06:00:42.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.605+0000 7fec72ffd640 1 --2- 172.21.15.117:0/1234671022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec7410ab80 0x7fec74112670 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fec6c031a20 tx=0x7fec6c02f480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:42.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.606+0000 7fec78893640 1 -- 172.21.15.117:0/1234671022 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c003bf0 con 0x7fec7410ab80 2024-09-21T06:00:42.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.606+0000 7fec79895640 1 -- 172.21.15.117:0/1234671022 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec74118810 con 0x7fec7410ab80 2024-09-21T06:00:42.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.606+0000 7fec79895640 1 -- 172.21.15.117:0/1234671022 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec74119020 con 0x7fec7410ab80 2024-09-21T06:00:42.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.607+0000 7fec78893640 1 -- 172.21.15.117:0/1234671022 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec6c02d070 con 0x7fec7410ab80 2024-09-21T06:00:42.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.607+0000 7fec78893640 1 -- 172.21.15.117:0/1234671022 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c02f9e0 con 0x7fec7410ab80 2024-09-21T06:00:42.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.615+0000 7fec78893640 1 -- 172.21.15.117:0/1234671022 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fec6c004040 con 0x7fec7410ab80 2024-09-21T06:00:42.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.616+0000 7fec78893640 1 --2- 172.21.15.117:0/1234671022 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fec5c0b6880 0x7fec5c0b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:42.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.618+0000 7fec78893640 1 -- 172.21.15.117:0/1234671022 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fec6c108c50 con 0x7fec7410ab80 2024-09-21T06:00:42.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.622+0000 7fec7fe09740 1 Processor -- start 2024-09-21T06:00:42.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.618+0000 7fec727fc640 1 --2- 172.21.15.117:0/1234671022 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fec5c0b6880 0x7fec5c0b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:42.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.623+0000 7fec7fe09740 1 -- start start 2024-09-21T06:00:42.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.624+0000 7fec7fe09740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bf3457070 0x557bf37fe860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:42.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.624+0000 7fec7fe09740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bf342e1b0 0x557bf37feda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:42.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.624+0000 7fec7fe09740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bf3436390 0x557bf37ff2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:42.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.624+0000 7fec7fe09740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557bf3774ea0 con 0x557bf3457070 2024-09-21T06:00:42.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.624+0000 7fec7fe09740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557bf37642d0 con 0x557bf342e1b0 2024-09-21T06:00:42.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.624+0000 7fec7fe09740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557bf37680e0 con 0x557bf3436390 2024-09-21T06:00:42.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.625+0000 7fec737fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bf3436390 0x557bf37ff2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:42.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.625+0000 7fec737fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bf3436390 0x557bf37ff2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:35188/0 (socket says 172.21.15.117:35188) 2024-09-21T06:00:42.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.625+0000 7fec737fe640 1 -- 172.21.15.117:0/3807636290 learned_addr learned my addr 172.21.15.117:0/3807636290 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:42.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.626+0000 7fec737fe640 1 -- 172.21.15.117:0/3807636290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bf342e1b0 msgr2=0x557bf37feda0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:00:42.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.626+0000 7fec737fe640 1 --2- 172.21.15.117:0/3807636290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bf342e1b0 0x557bf37feda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:42.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.626+0000 7fec737fe640 1 -- 172.21.15.117:0/3807636290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bf3457070 msgr2=0x557bf37fe860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:42.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.626+0000 7fec737fe640 1 --2- 172.21.15.117:0/3807636290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bf3457070 0x557bf37fe860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:42.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.626+0000 7fec737fe640 1 -- 172.21.15.117:0/3807636290 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bf34c2650 con 0x557bf3436390 2024-09-21T06:00:42.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.626+0000 7fec737fe640 1 --2- 172.21.15.117:0/3807636290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bf3436390 0x557bf37ff2e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fec68009760 tx=0x7fec68009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbec32683016a54 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:42.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.628+0000 7fec59ffb640 1 -- 172.21.15.117:0/3807636290 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec68015020 con 0x557bf3436390 2024-09-21T06:00:42.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.629+0000 7fec59ffb640 1 -- 172.21.15.117:0/3807636290 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec68011420 con 0x557bf3436390 2024-09-21T06:00:42.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.631+0000 7fec7fe09740 1 -- 172.21.15.117:0/3807636290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bf3436390 msgr2=0x557bf37ff2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:42.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.631+0000 7fec7fe09740 1 --2- 172.21.15.117:0/3807636290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bf3436390 0x557bf37ff2e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fec68009760 tx=0x7fec68009c20 comp rx=0 tx=0).stop 2024-09-21T06:00:42.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.631+0000 7fec7fe09740 1 -- 172.21.15.117:0/3807636290 shutdown_connections 2024-09-21T06:00:42.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.632+0000 7fec7fe09740 1 --2- 172.21.15.117:0/3807636290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bf3436390 0x557bf37ff2e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:42.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.632+0000 7fec7fe09740 1 --2- 172.21.15.117:0/3807636290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bf342e1b0 0x557bf37feda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:42.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.632+0000 7fec7fe09740 1 --2- 172.21.15.117:0/3807636290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bf3457070 0x557bf37fe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:42.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.632+0000 7fec7fe09740 1 -- 172.21.15.117:0/3807636290 >> 172.21.15.117:0/3807636290 conn(0x557bf344b460 msgr2=0x557bf37fc750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:42.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.632+0000 7fec7fe09740 1 -- 172.21.15.117:0/3807636290 shutdown_connections 2024-09-21T06:00:42.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.632+0000 7fec7fe09740 1 -- 172.21.15.117:0/3807636290 wait complete. 2024-09-21T06:00:42.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.633+0000 7fec7fe09740 1 -- start start 2024-09-21T06:00:42.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.634+0000 7fec7fe09740 1 Processor -- start 2024-09-21T06:00:42.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.636+0000 7fec7fe09740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bf342e1b0 0x557bf3809bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:42.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.637+0000 7fec7fe09740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bf3457070 0x557bf380a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:42.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.637+0000 7fec7fe09740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bf34159f0 0x557bf380a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:42.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.637+0000 7fec7fe09740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557bf35c5700 con 0x557bf3457070 2024-09-21T06:00:42.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.637+0000 7fec7fe09740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557bf36ff160 con 0x557bf34159f0 2024-09-21T06:00:42.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.637+0000 7fec7fe09740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557bf380d090 con 0x557bf342e1b0 2024-09-21T06:00:42.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.638+0000 7fec7fe09740 10 client.0 _ll_register_callbacks cb 0x557bf37189e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:00:42.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.640+0000 7fec727fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bf3457070 0x557bf380a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:42.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.640+0000 7fec727fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bf3457070 0x557bf380a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39570/0 (socket says 172.21.15.117:39570) 2024-09-21T06:00:42.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.640+0000 7fec727fc640 1 -- 172.21.15.117:0/1211718832 learned_addr learned my addr 172.21.15.117:0/1211718832 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:42.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.640+0000 7fec72ffd640 1 --2- 172.21.15.117:0/1211718832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bf342e1b0 0x557bf3809bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:42.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.640+0000 7fec737fe640 1 --2- 172.21.15.117:0/1211718832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bf34159f0 0x557bf380a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:42.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.643+0000 7fec727fc640 1 --2- 172.21.15.117:0/1234671022 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fec5c0b6880 0x7fec5c0b8d90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fec74110710 tx=0x7fec64006c90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:42.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.648+0000 7fec72ffd640 1 -- 172.21.15.117:0/1211718832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bf34159f0 msgr2=0x557bf380a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:42.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.648+0000 7fec72ffd640 1 --2- 172.21.15.117:0/1211718832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bf34159f0 0x557bf380a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:42.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.648+0000 7fec72ffd640 1 -- 172.21.15.117:0/1211718832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bf3457070 msgr2=0x557bf380a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:42.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.648+0000 7fec72ffd640 1 --2- 172.21.15.117:0/1211718832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bf3457070 0x557bf380a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:42.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.648+0000 7fec72ffd640 1 -- 172.21.15.117:0/1211718832 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bf372cc90 con 0x557bf342e1b0 2024-09-21T06:00:42.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.653+0000 7fec72ffd640 1 --2- 172.21.15.117:0/1211718832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bf342e1b0 0x557bf3809bd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fec6c1063c0 tx=0x7fec6c0ce4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d2c551d1e611fcb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:42.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.653+0000 7fec52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-21T06:00:42.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.654+0000 7fec52ffd640 1 -- 172.21.15.117:0/1211718832 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c0d3070 con 0x557bf342e1b0 2024-09-21T06:00:42.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.654+0000 7fec52ffd640 1 -- 172.21.15.117:0/1211718832 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fec6c0cad30 con 0x557bf342e1b0 2024-09-21T06:00:42.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.655+0000 7fec7fe09740 1 -- 172.21.15.117:0/1211718832 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557bf3776df0 con 0x557bf342e1b0 2024-09-21T06:00:42.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.658+0000 7fec52ffd640 1 -- 172.21.15.117:0/1211718832 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fec6c0d35a0 con 0x557bf342e1b0 2024-09-21T06:00:42.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.659+0000 7fec7fe09740 10 client.25504 fetch_fsmap learned FSMap version 27 2024-09-21T06:00:42.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.659+0000 7fec7fe09740 1 -- 172.21.15.117:0/1211718832 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557bf3818b90 con 0x557bf342e1b0 2024-09-21T06:00:42.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.665+0000 7fec52ffd640 1 -- 172.21.15.117:0/1211718832 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557bf3776df0 con 0x557bf342e1b0 2024-09-21T06:00:42.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.665+0000 7fec52ffd640 1 -- 172.21.15.117:0/1211718832 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fec6c106640 con 0x557bf342e1b0 2024-09-21T06:00:42.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.666+0000 7fec7fe09740 10 client.25504 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:00:42.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.666+0000 7fec7fe09740 10 client.25504 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T06:00:42.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.666+0000 7fec7fe09740 20 client.25504 populate_metadata read hostname 'smithi117' 2024-09-21T06:00:42.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.666+0000 7fec7fe09740 1 --2- 172.21.15.117:0/1211718832 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x557bf33eada0 0x557bf381c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:42.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.666+0000 7fec7fe09740 4 client.25504 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:00:42.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.666+0000 7fec7fe09740 1 -- 172.21.15.117:0/1211718832 --> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557bf37728a0 con 0x557bf33eada0 2024-09-21T06:00:42.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.667+0000 7fec727fc640 1 --2- 172.21.15.117:0/1211718832 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x557bf33eada0 0x557bf381c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:42.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.680+0000 7fec52ffd640 10 client.25504 ms_handle_connect on v2:172.21.15.192:6834/3530688860 2024-09-21T06:00:42.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.680+0000 7fec727fc640 1 --2- 172.21.15.117:0/1211718832 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x557bf33eada0 0x557bf381c160 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2b2c2cf095c4be9a server_cookie=8f7211f4562bacdc in_seq=0 out_seq=0 2024-09-21T06:00:42.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.686+0000 7fec52ffd640 1 -- 172.21.15.117:0/1211718832 <== mds.3 v2:172.21.15.192:6834/3530688860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557bf37728a0 con 0x557bf33eada0 2024-09-21T06:00:42.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.686+0000 7fec52ffd640 10 client.25504 handle_command_reply: tid=0 2024-09-21T06:00:42.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.686+0000 7fec7fe09740 1 client.25504 shutdown 2024-09-21T06:00:42.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.687+0000 7fec7fe09740 1 -- 172.21.15.117:0/1211718832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bf342e1b0 msgr2=0x557bf3809bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:42.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.687+0000 7fec7fe09740 1 --2- 172.21.15.117:0/1211718832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bf342e1b0 0x557bf3809bd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fec6c1063c0 tx=0x7fec6c0ce4d0 comp rx=0 tx=0).stop 2024-09-21T06:00:42.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.687+0000 7fec7fe09740 1 -- 172.21.15.117:0/1211718832 shutdown_connections 2024-09-21T06:00:42.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.687+0000 7fec7fe09740 1 --2- 172.21.15.117:0/1211718832 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x557bf33eada0 0x557bf381c160 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:42.817 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.687+0000 7fec7fe09740 1 --2- 172.21.15.117:0/1211718832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bf3457070 0x557bf380a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:42.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.687+0000 7fec7fe09740 1 --2- 172.21.15.117:0/1211718832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bf34159f0 0x557bf380a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:42.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.687+0000 7fec7fe09740 1 --2- 172.21.15.117:0/1211718832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bf342e1b0 0x557bf3809bd0 secure :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fec6c1063c0 tx=0x7fec6c0ce4d0 comp rx=0 tx=0).stop 2024-09-21T06:00:42.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.687+0000 7fec7fe09740 1 -- 172.21.15.117:0/1211718832 >> 172.21.15.117:0/1211718832 conn(0x557bf344b460 msgr2=0x557bf3801a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:42.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.687+0000 7fec52ffd640 0 client.25504 ms_handle_reset on v2:172.21.15.192:6834/3530688860 2024-09-21T06:00:42.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.688+0000 7fec7fe09740 1 -- 172.21.15.117:0/1211718832 shutdown_connections 2024-09-21T06:00:42.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.688+0000 7fec7fe09740 1 -- 172.21.15.117:0/1211718832 wait complete. 2024-09-21T06:00:42.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.689+0000 7fec7fe09740 20 client.25504 trim_cache size 0 max 16384 2024-09-21T06:00:42.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.721+0000 7fec7fe09740 1 Processor -- start 2024-09-21T06:00:42.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.722+0000 7fec7fe09740 1 -- start start 2024-09-21T06:00:42.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.723+0000 7fec7fe09740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bf34159f0 0x557bf3802320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:42.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.726+0000 7fec7fe09740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bf3457070 0x557bf3816360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:42.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.726+0000 7fec72ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bf34159f0 0x557bf3802320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:42.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.726+0000 7fec7fe09740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bf380a2e0 0x557bf380a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:42.818 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.727+0000 7fec72ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bf34159f0 0x557bf3802320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:43106/0 (socket says 172.21.15.117:43106) 2024-09-21T06:00:42.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.727+0000 7fec72ffd640 1 -- 172.21.15.117:0/421622715 learned_addr learned my addr 172.21.15.117:0/421622715 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:42.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.727+0000 7fec7fe09740 1 -- 172.21.15.117:0/421622715 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fec6000e5f0 con 0x557bf380a2e0 2024-09-21T06:00:42.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.727+0000 7fec7fe09740 1 -- 172.21.15.117:0/421622715 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fec60010130 con 0x557bf34159f0 2024-09-21T06:00:42.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.728+0000 7fec7fe09740 1 -- 172.21.15.117:0/421622715 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fec6000ecc0 con 0x557bf3457070 2024-09-21T06:00:42.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.729+0000 7fec727fc640 1 --2- 172.21.15.117:0/421622715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bf3457070 0x557bf3816360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:42.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.729+0000 7fec737fe640 1 --2- 172.21.15.117:0/421622715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bf380a2e0 0x557bf380a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:42.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.729+0000 7fec737fe640 1 -- 172.21.15.117:0/421622715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bf3457070 msgr2=0x557bf3816360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:42.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.729+0000 7fec737fe640 1 --2- 172.21.15.117:0/421622715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bf3457070 0x557bf3816360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:42.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.729+0000 7fec737fe640 1 -- 172.21.15.117:0/421622715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bf34159f0 msgr2=0x557bf3802320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:42.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.729+0000 7fec737fe640 1 --2- 172.21.15.117:0/421622715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bf34159f0 0x557bf3802320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:42.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.730+0000 7fec737fe640 1 -- 172.21.15.117:0/421622715 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bf3800b50 con 0x557bf380a2e0 2024-09-21T06:00:42.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.730+0000 7fec737fe640 1 --2- 172.21.15.117:0/421622715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bf380a2e0 0x557bf380a6f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fec6800b4b0 tx=0x7fec68013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c0b7cc85c48ab8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:42.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.732+0000 7fec597fa640 1 -- 172.21.15.117:0/421622715 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec68015070 con 0x557bf380a2e0 2024-09-21T06:00:42.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.732+0000 7fec597fa640 1 -- 172.21.15.117:0/421622715 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec68010c30 con 0x557bf380a2e0 2024-09-21T06:00:42.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.732+0000 7fec597fa640 1 -- 172.21.15.117:0/421622715 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6801a640 con 0x557bf380a2e0 2024-09-21T06:00:42.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.734+0000 7fec7fe09740 1 -- 172.21.15.117:0/421622715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bf380a2e0 msgr2=0x557bf380a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:42.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.734+0000 7fec7fe09740 1 --2- 172.21.15.117:0/421622715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bf380a2e0 0x557bf380a6f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fec6800b4b0 tx=0x7fec68013040 comp rx=0 tx=0).stop 2024-09-21T06:00:42.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.737+0000 7fec7fe09740 1 -- 172.21.15.117:0/421622715 shutdown_connections 2024-09-21T06:00:42.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.737+0000 7fec7fe09740 1 --2- 172.21.15.117:0/421622715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bf380a2e0 0x557bf380a6f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:42.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.737+0000 7fec7fe09740 1 --2- 172.21.15.117:0/421622715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bf3457070 0x557bf3816360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:42.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.737+0000 7fec7fe09740 1 --2- 172.21.15.117:0/421622715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bf34159f0 0x557bf3802320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:42.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.737+0000 7fec7fe09740 1 -- 172.21.15.117:0/421622715 >> 172.21.15.117:0/421622715 conn(0x557bf344b460 msgr2=0x557bf3809bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:42.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.739+0000 7fec7fe09740 1 -- 172.21.15.117:0/421622715 shutdown_connections 2024-09-21T06:00:42.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.741+0000 7fec7fe09740 1 -- 172.21.15.117:0/421622715 wait complete. 2024-09-21T06:00:42.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.742+0000 7fec7fe09740 1 -- start start 2024-09-21T06:00:42.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.742+0000 7fec7fe09740 1 Processor -- start 2024-09-21T06:00:42.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.743+0000 7fec7fe09740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bf34159f0 0x557bf3812a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:42.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.744+0000 7fec7fe09740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bf3457070 0x557bf380ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:42.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.744+0000 7fec7fe09740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bf380a2e0 0x557bf380f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:42.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.744+0000 7fec7fe09740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fec6000fa60 con 0x557bf380a2e0 2024-09-21T06:00:42.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.744+0000 7fec7fe09740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fec6000f390 con 0x557bf3457070 2024-09-21T06:00:42.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.744+0000 7fec7fe09740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fec6000c5d0 con 0x557bf34159f0 2024-09-21T06:00:42.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.744+0000 7fec7fe09740 10 client.0 _ll_register_callbacks cb 0x7fec6c004870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:00:42.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.745+0000 7fec727fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bf3457070 0x557bf380ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:42.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.745+0000 7fec737fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bf380a2e0 0x557bf380f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:42.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.745+0000 7fec737fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bf380a2e0 0x557bf380f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39580/0 (socket says 172.21.15.117:39580) 2024-09-21T06:00:42.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.745+0000 7fec737fe640 1 -- 172.21.15.117:0/3986301849 learned_addr learned my addr 172.21.15.117:0/3986301849 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:42.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.745+0000 7fec737fe640 1 -- 172.21.15.117:0/3986301849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bf34159f0 msgr2=0x557bf3812a90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:00:42.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.745+0000 7fec737fe640 1 --2- 172.21.15.117:0/3986301849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bf34159f0 0x557bf3812a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:42.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.746+0000 7fec737fe640 1 -- 172.21.15.117:0/3986301849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bf3457070 msgr2=0x557bf380ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:42.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.746+0000 7fec737fe640 1 --2- 172.21.15.117:0/3986301849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bf3457070 0x557bf380ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:42.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.746+0000 7fec737fe640 1 -- 172.21.15.117:0/3986301849 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bf380fb00 con 0x557bf380a2e0 2024-09-21T06:00:42.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.746+0000 7fec737fe640 1 --2- 172.21.15.117:0/3986301849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bf380a2e0 0x557bf380f490 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fec68006bc0 tx=0x7fec68009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f506b1cf4109838 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:42.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.746+0000 7fec52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:00:42.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.746+0000 7fec52ffd640 1 -- 172.21.15.117:0/3986301849 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec68015040 con 0x557bf380a2e0 2024-09-21T06:00:42.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.746+0000 7fec52ffd640 1 -- 172.21.15.117:0/3986301849 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fec68010980 con 0x557bf380a2e0 2024-09-21T06:00:42.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.746+0000 7fec7fe09740 1 -- 172.21.15.117:0/3986301849 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec68010c30 con 0x557bf380a2e0 2024-09-21T06:00:42.822 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:00:42.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.747+0000 7fec52ffd640 1 -- 172.21.15.117:0/3986301849 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fec68034050 con 0x557bf380a2e0 2024-09-21T06:00:42.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.747+0000 7fec7fe09740 10 client.16134 fetch_fsmap learned FSMap version 27 2024-09-21T06:00:42.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.747+0000 7fec7fe09740 1 -- 172.21.15.117:0/3986301849 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557bf3887920 con 0x557bf380a2e0 2024-09-21T06:00:42.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.747+0000 7fec52ffd640 1 -- 172.21.15.117:0/3986301849 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec680118d0 con 0x557bf380a2e0 2024-09-21T06:00:42.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.748+0000 7fec52ffd640 1 -- 172.21.15.117:0/3986301849 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fec6801a6c0 con 0x557bf380a2e0 2024-09-21T06:00:42.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.749+0000 7fec7fe09740 10 client.16134 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:00:42.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.750+0000 7fec7fe09740 10 client.16134 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T06:00:42.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.750+0000 7fec7fe09740 20 client.16134 populate_metadata read hostname 'smithi117' 2024-09-21T06:00:42.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.750+0000 7fec7fe09740 1 --2- 172.21.15.117:0/3986301849 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x557bf3887d40 0x557bf388a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:42.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.750+0000 7fec7fe09740 4 client.16134 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:00:42.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.750+0000 7fec7fe09740 1 -- 172.21.15.117:0/3986301849 --> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fec68010c30 con 0x557bf3887d40 2024-09-21T06:00:42.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.754+0000 7fec72ffd640 1 --2- 172.21.15.117:0/3986301849 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x557bf3887d40 0x557bf388a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:42.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.763+0000 7fec52ffd640 10 client.16134 ms_handle_connect on v2:172.21.15.192:6834/3530688860 2024-09-21T06:00:42.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.763+0000 7fec72ffd640 1 --2- 172.21.15.117:0/3986301849 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x557bf3887d40 0x557bf388a160 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8850b91c36112d76 server_cookie=3fa9b8cc40a0d537 in_seq=0 out_seq=0 2024-09-21T06:00:42.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.767+0000 7fec52ffd640 1 -- 172.21.15.117:0/3986301849 <== mds.3 v2:172.21.15.192:6834/3530688860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fec68010c30 con 0x557bf3887d40 2024-09-21T06:00:42.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.767+0000 7fec52ffd640 10 client.16134 handle_command_reply: tid=0 2024-09-21T06:00:42.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.767+0000 7fec7fe09740 1 client.16134 shutdown 2024-09-21T06:00:42.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.768+0000 7fec7fe09740 1 -- 172.21.15.117:0/3986301849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bf380a2e0 msgr2=0x557bf380f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:42.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.768+0000 7fec7fe09740 1 --2- 172.21.15.117:0/3986301849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bf380a2e0 0x557bf380f490 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fec68006bc0 tx=0x7fec68009260 comp rx=0 tx=0).stop 2024-09-21T06:00:42.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.770+0000 7fec7fe09740 1 -- 172.21.15.117:0/3986301849 shutdown_connections 2024-09-21T06:00:42.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.770+0000 7fec7fe09740 1 --2- 172.21.15.117:0/3986301849 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x557bf3887d40 0x557bf388a160 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:42.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.770+0000 7fec7fe09740 1 --2- 172.21.15.117:0/3986301849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557bf380a2e0 0x557bf380f490 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:42.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.770+0000 7fec7fe09740 1 --2- 172.21.15.117:0/3986301849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557bf3457070 0x557bf380ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:42.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.770+0000 7fec52ffd640 0 client.16134 ms_handle_reset on v2:172.21.15.192:6834/3530688860 2024-09-21T06:00:42.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.770+0000 7fec7fe09740 1 --2- 172.21.15.117:0/3986301849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557bf34159f0 0x557bf3812a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:42.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.770+0000 7fec7fe09740 1 -- 172.21.15.117:0/3986301849 >> 172.21.15.117:0/3986301849 conn(0x557bf344b460 msgr2=0x557bf3809bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:42.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.770+0000 7fec7fe09740 1 -- 172.21.15.117:0/3986301849 shutdown_connections 2024-09-21T06:00:42.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.772+0000 7fec7fe09740 1 -- 172.21.15.117:0/3986301849 wait complete. 2024-09-21T06:00:42.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.772+0000 7fec7fe09740 20 client.16134 trim_cache size 0 max 16384 2024-09-21T06:00:42.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.774+0000 7fec79895640 1 -- 172.21.15.117:0/1234671022 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fec5c0b6880 msgr2=0x7fec5c0b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:42.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.774+0000 7fec79895640 1 --2- 172.21.15.117:0/1234671022 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fec5c0b6880 0x7fec5c0b8d90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fec74110710 tx=0x7fec64006c90 comp rx=0 tx=0).stop 2024-09-21T06:00:42.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.774+0000 7fec79895640 1 -- 172.21.15.117:0/1234671022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec7410ab80 msgr2=0x7fec74112670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:42.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.774+0000 7fec79895640 1 --2- 172.21.15.117:0/1234671022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec7410ab80 0x7fec74112670 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fec6c031a20 tx=0x7fec6c02f480 comp rx=0 tx=0).stop 2024-09-21T06:00:42.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.774+0000 7fec79895640 1 -- 172.21.15.117:0/1234671022 shutdown_connections 2024-09-21T06:00:42.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.774+0000 7fec79895640 1 --2- 172.21.15.117:0/1234671022 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fec5c0b6880 0x7fec5c0b8d90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:42.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.774+0000 7fec79895640 1 --2- 172.21.15.117:0/1234671022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec7411cba0 0x7fec74117eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:42.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.774+0000 7fec79895640 1 --2- 172.21.15.117:0/1234671022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec74111d40 0x7fec74112bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:42.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.774+0000 7fec79895640 1 --2- 172.21.15.117:0/1234671022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec7410ab80 0x7fec74112670 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:42.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.774+0000 7fec79895640 1 -- 172.21.15.117:0/1234671022 >> 172.21.15.117:0/1234671022 conn(0x7fec7406d950 msgr2=0x7fec741109d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:42.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.775+0000 7fec79895640 1 -- 172.21.15.117:0/1234671022 shutdown_connections 2024-09-21T06:00:42.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:42.776+0000 7fec79895640 1 -- 172.21.15.117:0/1234671022 wait complete. 2024-09-21T06:00:44.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:43 smithi117 ceph-mon[30947]: pgmap v429: 97 pgs: 97 active+clean; 30 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 879 MiB/s wr, 1.27k op/s 2024-09-21T06:00:44.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:43 smithi138 ceph-mon[34391]: pgmap v429: 97 pgs: 97 active+clean; 30 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 879 MiB/s wr, 1.27k op/s 2024-09-21T06:00:44.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:43 smithi192 ceph-mon[34700]: pgmap v429: 97 pgs: 97 active+clean; 30 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 879 MiB/s wr, 1.27k op/s 2024-09-21T06:00:46.032 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:45 smithi192 ceph-mon[34700]: pgmap v430: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 995 MiB/s wr, 1.43k op/s 2024-09-21T06:00:46.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:45 smithi138 ceph-mon[34391]: pgmap v430: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 995 MiB/s wr, 1.43k op/s 2024-09-21T06:00:46.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:45 smithi117 ceph-mon[30947]: pgmap v430: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 995 MiB/s wr, 1.43k op/s 2024-09-21T06:00:46.963 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T06:00:47.634 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:00:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.671+0000 7f81f57da640 1 -- 172.21.15.117:0/649595422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81f010ab50 msgr2=0x7f81f0116910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.671+0000 7f81f57da640 1 --2- 172.21.15.117:0/649595422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81f010ab50 0x7f81f0116910 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f81e8009ca0 tx=0x7f81e802a8f0 comp rx=0 tx=0).stop 2024-09-21T06:00:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.673+0000 7f81f57da640 1 -- 172.21.15.117:0/649595422 shutdown_connections 2024-09-21T06:00:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.673+0000 7f81f57da640 1 --2- 172.21.15.117:0/649595422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81f010ab50 0x7f81f0116910 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.673+0000 7f81f57da640 1 --2- 172.21.15.117:0/649595422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81f010a200 0x7f81f010a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.673+0000 7f81f57da640 1 --2- 172.21.15.117:0/649595422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81f010eab0 0x7f81f010eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.673+0000 7f81f57da640 1 -- 172.21.15.117:0/649595422 >> 172.21.15.117:0/649595422 conn(0x7f81f0079400 msgr2=0x7f81f007b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.673+0000 7f81f57da640 1 -- 172.21.15.117:0/649595422 shutdown_connections 2024-09-21T06:00:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.673+0000 7f81f57da640 1 -- 172.21.15.117:0/649595422 wait complete. 2024-09-21T06:00:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.676+0000 7f81f57da640 1 Processor -- start 2024-09-21T06:00:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.677+0000 7f81f57da640 1 -- start start 2024-09-21T06:00:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.678+0000 7f81f57da640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81f010a200 0x7f81f01a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.678+0000 7f81f57da640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81f010ab50 0x7f81f01a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.678+0000 7f81f57da640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81f010eab0 0x7f81f01a6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.678+0000 7f81f57da640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f81f0119fa0 con 0x7f81f010eab0 2024-09-21T06:00:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.678+0000 7f81f57da640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81f0119e20 con 0x7f81f010a200 2024-09-21T06:00:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.678+0000 7f81f57da640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f81f011a120 con 0x7f81f010ab50 2024-09-21T06:00:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.678+0000 7f81ef7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81f010eab0 0x7f81f01a6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.678+0000 7f81ef7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81f010eab0 0x7f81f01a6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39212/0 (socket says 172.21.15.117:39212) 2024-09-21T06:00:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.678+0000 7f81ef7fe640 1 -- 172.21.15.117:0/3900325389 learned_addr learned my addr 172.21.15.117:0/3900325389 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.679+0000 7f81ef7fe640 1 -- 172.21.15.117:0/3900325389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81f010ab50 msgr2=0x7f81f01a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.679+0000 7f81ef7fe640 1 --2- 172.21.15.117:0/3900325389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81f010ab50 0x7f81f01a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.679+0000 7f81ef7fe640 1 -- 172.21.15.117:0/3900325389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81f010a200 msgr2=0x7f81f01a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.679+0000 7f81ef7fe640 1 --2- 172.21.15.117:0/3900325389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81f010a200 0x7f81f01a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.679+0000 7f81ef7fe640 1 -- 172.21.15.117:0/3900325389 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f01a6700 con 0x7f81f010eab0 2024-09-21T06:00:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.680+0000 7f81ef7fe640 1 --2- 172.21.15.117:0/3900325389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81f010eab0 0x7f81f01a6000 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f81e8009620 tx=0x7f81e8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.680+0000 7f81d7fff640 1 -- 172.21.15.117:0/3900325389 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e8004040 con 0x7f81f010eab0 2024-09-21T06:00:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.680+0000 7f81d7fff640 1 -- 172.21.15.117:0/3900325389 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81e802d070 con 0x7f81f010eab0 2024-09-21T06:00:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.680+0000 7f81f57da640 1 -- 172.21.15.117:0/3900325389 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81f01a6990 con 0x7f81f010eab0 2024-09-21T06:00:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.681+0000 7f81f57da640 1 -- 172.21.15.117:0/3900325389 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81f01ad180 con 0x7f81f010eab0 2024-09-21T06:00:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.681+0000 7f81d7fff640 1 -- 172.21.15.117:0/3900325389 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e802fd00 con 0x7f81f010eab0 2024-09-21T06:00:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.686+0000 7f81fc824740 1 Processor -- start 2024-09-21T06:00:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.687+0000 7f81d7fff640 1 -- 172.21.15.117:0/3900325389 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f81e80305f0 con 0x7f81f010eab0 2024-09-21T06:00:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.687+0000 7f81fc824740 1 -- start start 2024-09-21T06:00:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.688+0000 7f81fc824740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0a577e1b0 0x55b0a5b23ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.688+0000 7f81ee7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0a577e1b0 0x55b0a5b23ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.689+0000 7f81ee7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0a577e1b0 0x55b0a5b23ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39222/0 (socket says 172.21.15.117:39222) 2024-09-21T06:00:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.689+0000 7f81ee7fc640 1 -- 172.21.15.117:0/4249963346 learned_addr learned my addr 172.21.15.117:0/4249963346 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.689+0000 7f81fc824740 1 --2- 172.21.15.117:0/4249963346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b0a57552f0 0x55b0a5b240e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.689+0000 7f81fc824740 1 --2- 172.21.15.117:0/4249963346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0a575d4d0 0x55b0a5b24620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.689+0000 7f81fc824740 1 -- 172.21.15.117:0/4249963346 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b0a5a986f0 con 0x55b0a577e1b0 2024-09-21T06:00:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.689+0000 7f81fc824740 1 -- 172.21.15.117:0/4249963346 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b0a5a87b20 con 0x55b0a575d4d0 2024-09-21T06:00:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.689+0000 7f81fc824740 1 -- 172.21.15.117:0/4249963346 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b0a5a8b930 con 0x55b0a57552f0 2024-09-21T06:00:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.690+0000 7f81ee7fc640 1 -- 172.21.15.117:0/4249963346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b0a57552f0 msgr2=0x55b0a5b240e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.690+0000 7f81ee7fc640 1 --2- 172.21.15.117:0/4249963346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b0a57552f0 0x55b0a5b240e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.690+0000 7f81ee7fc640 1 -- 172.21.15.117:0/4249963346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0a575d4d0 msgr2=0x55b0a5b24620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:00:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.690+0000 7f81ee7fc640 1 --2- 172.21.15.117:0/4249963346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0a575d4d0 0x55b0a5b24620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.690+0000 7f81ee7fc640 1 -- 172.21.15.117:0/4249963346 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0a5a4d530 con 0x55b0a577e1b0 2024-09-21T06:00:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.692+0000 7f81ee7fc640 1 --2- 172.21.15.117:0/4249963346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0a577e1b0 0x55b0a5b23ba0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f81e4009940 tx=0x7f81e4009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20e78c65ffc76c89 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.693+0000 7f81d57fa640 1 -- 172.21.15.117:0/4249963346 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e4015020 con 0x55b0a577e1b0 2024-09-21T06:00:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.693+0000 7f81d57fa640 1 -- 172.21.15.117:0/4249963346 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81e400be30 con 0x55b0a577e1b0 2024-09-21T06:00:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.693+0000 7f81d57fa640 1 -- 172.21.15.117:0/4249963346 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e4010380 con 0x55b0a577e1b0 2024-09-21T06:00:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.695+0000 7f81d7fff640 1 --2- 172.21.15.117:0/3900325389 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f81d80b67d0 0x7f81d80b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.696+0000 7f81d7fff640 1 -- 172.21.15.117:0/3900325389 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f81e8108900 con 0x7f81f010eab0 2024-09-21T06:00:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.696+0000 7f81eeffd640 1 --2- 172.21.15.117:0/3900325389 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f81d80b67d0 0x7f81d80b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.700+0000 7f81fc824740 1 -- 172.21.15.117:0/4249963346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0a577e1b0 msgr2=0x55b0a5b23ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.700+0000 7f81fc824740 1 --2- 172.21.15.117:0/4249963346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0a577e1b0 0x55b0a5b23ba0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f81e4009940 tx=0x7f81e4009e00 comp rx=0 tx=0).stop 2024-09-21T06:00:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.701+0000 7f81fc824740 1 -- 172.21.15.117:0/4249963346 shutdown_connections 2024-09-21T06:00:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.701+0000 7f81fc824740 1 --2- 172.21.15.117:0/4249963346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0a575d4d0 0x55b0a5b24620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.701+0000 7f81fc824740 1 --2- 172.21.15.117:0/4249963346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b0a57552f0 0x55b0a5b240e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.701+0000 7f81fc824740 1 --2- 172.21.15.117:0/4249963346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0a577e1b0 0x55b0a5b23ba0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.701+0000 7f81fc824740 1 -- 172.21.15.117:0/4249963346 >> 172.21.15.117:0/4249963346 conn(0x55b0a5772300 msgr2=0x55b0a5b21a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.701+0000 7f81eeffd640 1 --2- 172.21.15.117:0/3900325389 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f81d80b67d0 0x7f81d80b8ce0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55b0a5b22820 tx=0x7f81e0006cb0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.701+0000 7f81fc824740 1 -- 172.21.15.117:0/4249963346 shutdown_connections 2024-09-21T06:00:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.701+0000 7f81fc824740 1 -- 172.21.15.117:0/4249963346 wait complete. 2024-09-21T06:00:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.702+0000 7f81fc824740 1 -- start start 2024-09-21T06:00:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.702+0000 7f81fc824740 1 Processor -- start 2024-09-21T06:00:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.702+0000 7f81fc824740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0a57552f0 0x55b0a5b2f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.703+0000 7f81fc824740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0a575d4d0 0x55b0a5b2f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.703+0000 7f81fc824740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b0a577e1b0 0x55b0a5b3b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.703+0000 7f81fc824740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b0a59bfab0 con 0x55b0a57552f0 2024-09-21T06:00:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.703+0000 7f81fc824740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b0a5a2d730 con 0x55b0a575d4d0 2024-09-21T06:00:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.703+0000 7f81fc824740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b0a5b32320 con 0x55b0a577e1b0 2024-09-21T06:00:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.703+0000 7f81fc824740 10 client.0 _ll_register_callbacks cb 0x55b0a5a816b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:00:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.703+0000 7f81eeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0a57552f0 0x55b0a5b2f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.703+0000 7f81eeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0a57552f0 0x55b0a5b2f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39224/0 (socket says 172.21.15.117:39224) 2024-09-21T06:00:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.703+0000 7f81eeffd640 1 -- 172.21.15.117:0/176467810 learned_addr learned my addr 172.21.15.117:0/176467810 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.703+0000 7f81ee7fc640 1 --2- 172.21.15.117:0/176467810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0a575d4d0 0x55b0a5b2f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.704+0000 7f81eeffd640 1 -- 172.21.15.117:0/176467810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b0a577e1b0 msgr2=0x55b0a5b3b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.704+0000 7f81eeffd640 1 --2- 172.21.15.117:0/176467810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b0a577e1b0 0x55b0a5b3b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.704+0000 7f81eeffd640 1 -- 172.21.15.117:0/176467810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0a575d4d0 msgr2=0x55b0a5b2f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.704+0000 7f81eeffd640 1 --2- 172.21.15.117:0/176467810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0a575d4d0 0x55b0a5b2f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.704+0000 7f81eeffd640 1 -- 172.21.15.117:0/176467810 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0a57e9790 con 0x55b0a57552f0 2024-09-21T06:00:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.704+0000 7f81eeffd640 1 --2- 172.21.15.117:0/176467810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0a57552f0 0x55b0a5b2f310 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f81e0013fb0 tx=0x7f81e00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da95516397dff7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.704+0000 7f81d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:00:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.704+0000 7f81d2ff5640 1 -- 172.21.15.117:0/176467810 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e001d5f0 con 0x55b0a57552f0 2024-09-21T06:00:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.704+0000 7f81d2ff5640 1 -- 172.21.15.117:0/176467810 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f81e001d790 con 0x55b0a57552f0 2024-09-21T06:00:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.704+0000 7f81fc824740 1 -- 172.21.15.117:0/176467810 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81e400be30 con 0x55b0a57552f0 2024-09-21T06:00:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.704+0000 7f81d2ff5640 1 -- 172.21.15.117:0/176467810 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f81e001d990 con 0x55b0a57552f0 2024-09-21T06:00:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.705+0000 7f81fc824740 10 client.16146 fetch_fsmap learned FSMap version 27 2024-09-21T06:00:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.705+0000 7f81fc824740 1 -- 172.21.15.117:0/176467810 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b0a5862620 con 0x55b0a57552f0 2024-09-21T06:00:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.705+0000 7f81d2ff5640 1 -- 172.21.15.117:0/176467810 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81e400be30 con 0x55b0a57552f0 2024-09-21T06:00:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.706+0000 7f81d2ff5640 1 -- 172.21.15.117:0/176467810 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f81e0012070 con 0x55b0a57552f0 2024-09-21T06:00:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.707+0000 7f81fc824740 10 client.16146 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:00:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.707+0000 7f81fc824740 10 client.16146 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:00:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.707+0000 7f81fc824740 20 client.16146 populate_metadata read hostname 'smithi117' 2024-09-21T06:00:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.707+0000 7f81fc824740 1 --2- 172.21.15.117:0/176467810 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b0a573cd80 0x55b0a5b416e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.707+0000 7f81fc824740 4 client.16146 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:00:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.707+0000 7f81fc824740 1 -- 172.21.15.117:0/176467810 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0a5a9a640 con 0x55b0a573cd80 2024-09-21T06:00:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.710+0000 7f81ee7fc640 1 --2- 172.21.15.117:0/176467810 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b0a573cd80 0x55b0a5b416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.712+0000 7f81ee7fc640 1 --2- 172.21.15.117:0/176467810 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b0a573cd80 0x55b0a5b416e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d191815a09445cd server_cookie=3af468dc1c4fe503 in_seq=0 out_seq=0 2024-09-21T06:00:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.712+0000 7f81d2ff5640 10 client.16146 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:00:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.716+0000 7f81d2ff5640 1 -- 172.21.15.117:0/176467810 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b0a5a9a640 con 0x55b0a573cd80 2024-09-21T06:00:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.716+0000 7f81d2ff5640 10 client.16146 handle_command_reply: tid=0 2024-09-21T06:00:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.716+0000 7f81fc824740 1 client.16146 shutdown 2024-09-21T06:00:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.716+0000 7f81fc824740 1 -- 172.21.15.117:0/176467810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0a57552f0 msgr2=0x55b0a5b2f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.716+0000 7f81fc824740 1 --2- 172.21.15.117:0/176467810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0a57552f0 0x55b0a5b2f310 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f81e0013fb0 tx=0x7f81e00047c0 comp rx=0 tx=0).stop 2024-09-21T06:00:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.718+0000 7f81fc824740 1 -- 172.21.15.117:0/176467810 shutdown_connections 2024-09-21T06:00:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.718+0000 7f81fc824740 1 --2- 172.21.15.117:0/176467810 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b0a573cd80 0x55b0a5b416e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.718+0000 7f81fc824740 1 --2- 172.21.15.117:0/176467810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b0a577e1b0 0x55b0a5b3b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.718+0000 7f81fc824740 1 --2- 172.21.15.117:0/176467810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0a575d4d0 0x55b0a5b2f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.718+0000 7f81fc824740 1 --2- 172.21.15.117:0/176467810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0a57552f0 0x55b0a5b2f310 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.718+0000 7f81fc824740 1 -- 172.21.15.117:0/176467810 >> 172.21.15.117:0/176467810 conn(0x55b0a5772300 msgr2=0x55b0a5b24bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.719+0000 7f81d2ff5640 0 client.16146 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:00:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.719+0000 7f81fc824740 1 -- 172.21.15.117:0/176467810 shutdown_connections 2024-09-21T06:00:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.719+0000 7f81fc824740 1 -- 172.21.15.117:0/176467810 wait complete. 2024-09-21T06:00:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.719+0000 7f81fc824740 20 client.16146 trim_cache size 0 max 16384 2024-09-21T06:00:47.775 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.773+0000 7f81fc824740 1 Processor -- start 2024-09-21T06:00:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.774+0000 7f81fc824740 1 -- start start 2024-09-21T06:00:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.775+0000 7f81fc824740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0a5772300 0x55b0a5b2f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.775+0000 7f81fc824740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0a57552f0 0x55b0a5b311f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:47.777 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.775+0000 7f81fc824740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b0a575d4d0 0x55b0a5b252e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.776+0000 7f81fc824740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc00e620 con 0x55b0a57552f0 2024-09-21T06:00:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.776+0000 7f81fc824740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc010160 con 0x55b0a5772300 2024-09-21T06:00:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.776+0000 7f81fc824740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc00ecf0 con 0x55b0a575d4d0 2024-09-21T06:00:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.777+0000 7f81ee7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0a57552f0 0x55b0a5b311f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:47.778 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.777+0000 7f81ee7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0a57552f0 0x55b0a5b311f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39240/0 (socket says 172.21.15.117:39240) 2024-09-21T06:00:47.779 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.777+0000 7f81ee7fc640 1 -- 172.21.15.117:0/1356466660 learned_addr learned my addr 172.21.15.117:0/1356466660 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:47.779 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.777+0000 7f81eeffd640 1 --2- 172.21.15.117:0/1356466660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0a5772300 0x55b0a5b2f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.779+0000 7f81ef7fe640 1 --2- 172.21.15.117:0/1356466660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b0a575d4d0 0x55b0a5b252e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.781+0000 7f81eeffd640 1 -- 172.21.15.117:0/1356466660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b0a575d4d0 msgr2=0x55b0a5b252e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.781+0000 7f81eeffd640 1 --2- 172.21.15.117:0/1356466660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b0a575d4d0 0x55b0a5b252e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.782+0000 7f81eeffd640 1 -- 172.21.15.117:0/1356466660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0a57552f0 msgr2=0x55b0a5b311f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.782+0000 7f81eeffd640 1 --2- 172.21.15.117:0/1356466660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0a57552f0 0x55b0a5b311f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.782+0000 7f81eeffd640 1 -- 172.21.15.117:0/1356466660 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0a5b26140 con 0x55b0a5772300 2024-09-21T06:00:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.787+0000 7f81eeffd640 1 --2- 172.21.15.117:0/1356466660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0a5772300 0x55b0a5b2f7f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f81e0000c00 tx=0x7f81e0013f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c03cf8b9c4ea27cc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.796+0000 7f81d4ff9640 1 -- 172.21.15.117:0/1356466660 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e0035030 con 0x55b0a5772300 2024-09-21T06:00:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.796+0000 7f81d4ff9640 1 -- 172.21.15.117:0/1356466660 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81e0022070 con 0x55b0a5772300 2024-09-21T06:00:47.800 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.796+0000 7f81d4ff9640 1 -- 172.21.15.117:0/1356466660 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e00268a0 con 0x55b0a5772300 2024-09-21T06:00:47.804 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.799+0000 7f81fc824740 1 -- 172.21.15.117:0/1356466660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0a5772300 msgr2=0x55b0a5b2f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:47.804 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.799+0000 7f81fc824740 1 --2- 172.21.15.117:0/1356466660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0a5772300 0x55b0a5b2f7f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f81e0000c00 tx=0x7f81e0013f40 comp rx=0 tx=0).stop 2024-09-21T06:00:47.808 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.801+0000 7f81fc824740 1 -- 172.21.15.117:0/1356466660 shutdown_connections 2024-09-21T06:00:47.808 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.802+0000 7f81fc824740 1 --2- 172.21.15.117:0/1356466660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0a57552f0 0x55b0a5b311f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:47.808 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.802+0000 7f81fc824740 1 --2- 172.21.15.117:0/1356466660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b0a575d4d0 0x55b0a5b252e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:47.808 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.802+0000 7f81fc824740 1 --2- 172.21.15.117:0/1356466660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0a5772300 0x55b0a5b2f7f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:47.808 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.802+0000 7f81fc824740 1 -- 172.21.15.117:0/1356466660 >> 172.21.15.117:0/1356466660 conn(0x55b0a5b29660 msgr2=0x55b0a5b24bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.808+0000 7f81fc824740 1 -- 172.21.15.117:0/1356466660 shutdown_connections 2024-09-21T06:00:47.810 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.808+0000 7f81fc824740 1 -- 172.21.15.117:0/1356466660 wait complete. 2024-09-21T06:00:47.810 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.809+0000 7f81fc824740 1 -- start start 2024-09-21T06:00:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.809+0000 7f81fc824740 1 Processor -- start 2024-09-21T06:00:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.813+0000 7f81fc824740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0a57552f0 0x55b0a5b37a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.813+0000 7f81fc824740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b0a575d4d0 0x55b0a5b38000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.814+0000 7f81fc824740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0a5772300 0x55b0a5b330c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.814+0000 7f81fc824740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc00fa90 con 0x55b0a57552f0 2024-09-21T06:00:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.814+0000 7f81fc824740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc00f3c0 con 0x55b0a5772300 2024-09-21T06:00:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.814+0000 7f81fc824740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc00c600 con 0x55b0a575d4d0 2024-09-21T06:00:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.814+0000 7f81ef7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0a5772300 0x55b0a5b330c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.814+0000 7f81ef7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0a5772300 0x55b0a5b330c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:39024/0 (socket says 172.21.15.117:39024) 2024-09-21T06:00:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.814+0000 7f81ef7fe640 1 -- 172.21.15.117:0/2405326368 learned_addr learned my addr 172.21.15.117:0/2405326368 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.815+0000 7f81ee7fc640 1 --2- 172.21.15.117:0/2405326368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b0a575d4d0 0x55b0a5b38000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.815+0000 7f81fc824740 10 client.0 _ll_register_callbacks cb 0x7f81e0013c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:00:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.815+0000 7f81ef7fe640 1 -- 172.21.15.117:0/2405326368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b0a575d4d0 msgr2=0x55b0a5b38000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.815+0000 7f81ef7fe640 1 --2- 172.21.15.117:0/2405326368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b0a575d4d0 0x55b0a5b38000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:47.818 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.815+0000 7f81ef7fe640 1 -- 172.21.15.117:0/2405326368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0a57552f0 msgr2=0x55b0a5b37a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:47.818 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.815+0000 7f81ef7fe640 1 --2- 172.21.15.117:0/2405326368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0a57552f0 0x55b0a5b37a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:47.818 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.815+0000 7f81ef7fe640 1 -- 172.21.15.117:0/2405326368 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0a5b33850 con 0x55b0a5772300 2024-09-21T06:00:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.816+0000 7f81ef7fe640 1 --2- 172.21.15.117:0/2405326368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0a5772300 0x55b0a5b330c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f81e8106180 tx=0x7f81e80ce000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9ba3a5cbbb5e5b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.817+0000 7f81d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-21T06:00:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.818+0000 7f81d2ff5640 1 -- 172.21.15.117:0/2405326368 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e80ceed0 con 0x55b0a5772300 2024-09-21T06:00:47.821 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.819+0000 7f81fc824740 1 -- 172.21.15.117:0/2405326368 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81e0022070 con 0x55b0a5772300 2024-09-21T06:00:47.821 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.818+0000 7f81d2ff5640 1 -- 172.21.15.117:0/2405326368 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f81e80d7d60 con 0x55b0a5772300 2024-09-21T06:00:47.822 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.820+0000 7f81d2ff5640 1 -- 172.21.15.117:0/2405326368 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f81e80d3070 con 0x55b0a5772300 2024-09-21T06:00:47.823 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.821+0000 7f81fc824740 10 client.25529 fetch_fsmap learned FSMap version 27 2024-09-21T06:00:47.823 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.821+0000 7f81fc824740 1 -- 172.21.15.117:0/2405326368 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b0a5babcf0 con 0x55b0a5772300 2024-09-21T06:00:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.822+0000 7f81d2ff5640 1 -- 172.21.15.117:0/2405326368 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81e8036030 con 0x55b0a5772300 2024-09-21T06:00:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.823+0000 7f81d2ff5640 1 -- 172.21.15.117:0/2405326368 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f81e80cda30 con 0x55b0a5772300 2024-09-21T06:00:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.830+0000 7f81fc824740 10 client.25529 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:00:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.830+0000 7f81fc824740 10 client.25529 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:00:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.830+0000 7f81fc824740 20 client.25529 populate_metadata read hostname 'smithi117' 2024-09-21T06:00:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.830+0000 7f81fc824740 1 --2- 172.21.15.117:0/2405326368 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b0a577e1b0 0x55b0a5bae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.830+0000 7f81fc824740 4 client.25529 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:00:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.830+0000 7f81fc824740 1 -- 172.21.15.117:0/2405326368 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f81e0022070 con 0x55b0a577e1b0 2024-09-21T06:00:47.838 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.836+0000 7f81eeffd640 1 --2- 172.21.15.117:0/2405326368 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b0a577e1b0 0x55b0a5bae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:47.865 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.859+0000 7f81eeffd640 1 --2- 172.21.15.117:0/2405326368 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b0a577e1b0 0x55b0a5bae410 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11a80d5419662a0b server_cookie=e2841d6b1541fc6a in_seq=0 out_seq=0 2024-09-21T06:00:47.865 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:47.859+0000 7f81d2ff5640 10 client.25529 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:00:48.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:47 smithi117 ceph-mon[30947]: pgmap v431: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 872 MiB/s wr, 1.24k op/s 2024-09-21T06:00:48.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:47 smithi138 ceph-mon[34391]: pgmap v431: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 872 MiB/s wr, 1.24k op/s 2024-09-21T06:00:48.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:47 smithi192 ceph-mon[34700]: pgmap v431: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 872 MiB/s wr, 1.24k op/s 2024-09-21T06:00:50.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:49 smithi138 ceph-mon[34391]: pgmap v432: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 909 MiB/s wr, 1.28k op/s 2024-09-21T06:00:50.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:49 smithi192 ceph-mon[34700]: pgmap v432: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 909 MiB/s wr, 1.28k op/s 2024-09-21T06:00:50.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:49 smithi117 ceph-mon[30947]: pgmap v432: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 909 MiB/s wr, 1.28k op/s 2024-09-21T06:00:50.632 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:00:50.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.284+0000 7fb9312c1640 1 -- 172.21.15.117:0/2438621083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb92c11cc30 msgr2=0x7fb92c11f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:50.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.284+0000 7fb9312c1640 1 --2- 172.21.15.117:0/2438621083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb92c11cc30 0x7fb92c11f140 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fb924009ca0 tx=0x7fb92402a8f0 comp rx=0 tx=0).stop 2024-09-21T06:00:50.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.285+0000 7fb9312c1640 1 -- 172.21.15.117:0/2438621083 shutdown_connections 2024-09-21T06:00:50.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.285+0000 7fb9312c1640 1 --2- 172.21.15.117:0/2438621083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb92c11cc30 0x7fb92c11f140 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:50.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.285+0000 7fb9312c1640 1 --2- 172.21.15.117:0/2438621083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb92c10a9e0 0x7fb92c10adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:50.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.285+0000 7fb9312c1640 1 --2- 172.21.15.117:0/2438621083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb92c111f40 0x7fb92c10a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:50.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.285+0000 7fb9312c1640 1 -- 172.21.15.117:0/2438621083 >> 172.21.15.117:0/2438621083 conn(0x7fb92c070ec0 msgr2=0x7fb92c0712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:50.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.286+0000 7fb9312c1640 1 -- 172.21.15.117:0/2438621083 shutdown_connections 2024-09-21T06:00:50.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.286+0000 7fb9312c1640 1 -- 172.21.15.117:0/2438621083 wait complete. 2024-09-21T06:00:50.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.287+0000 7fb9312c1640 1 Processor -- start 2024-09-21T06:00:50.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.287+0000 7fb9312c1640 1 -- start start 2024-09-21T06:00:50.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.288+0000 7fb9312c1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb92c10a9e0 0x7fb92c1a91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:50.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.288+0000 7fb9312c1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb92c111f40 0x7fb92c1a9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:50.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.288+0000 7fb9312c1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb92c11cc30 0x7fb92c1ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:50.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.288+0000 7fb9312c1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c122900 con 0x7fb92c111f40 2024-09-21T06:00:50.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.288+0000 7fb9312c1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c122780 con 0x7fb92c11cc30 2024-09-21T06:00:50.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.288+0000 7fb9312c1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c122a80 con 0x7fb92c10a9e0 2024-09-21T06:00:50.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.290+0000 7fb92affd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb92c10a9e0 0x7fb92c1a91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:50.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.290+0000 7fb92affd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb92c10a9e0 0x7fb92c1a91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:49574/0 (socket says 172.21.15.117:49574) 2024-09-21T06:00:50.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.290+0000 7fb92affd640 1 -- 172.21.15.117:0/1713125091 learned_addr learned my addr 172.21.15.117:0/1713125091 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:50.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.290+0000 7fb92b7fe640 1 --2- 172.21.15.117:0/1713125091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb92c11cc30 0x7fb92c1ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:50.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.293+0000 7fb92affd640 1 -- 172.21.15.117:0/1713125091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb92c11cc30 msgr2=0x7fb92c1ae9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:50.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.293+0000 7fb92affd640 1 --2- 172.21.15.117:0/1713125091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb92c11cc30 0x7fb92c1ae9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:50.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.293+0000 7fb92affd640 1 -- 172.21.15.117:0/1713125091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb92c111f40 msgr2=0x7fb92c1a9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:50.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.293+0000 7fb92affd640 1 --2- 172.21.15.117:0/1713125091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb92c111f40 0x7fb92c1a9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:50.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.293+0000 7fb92affd640 1 -- 172.21.15.117:0/1713125091 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92c1af0e0 con 0x7fb92c10a9e0 2024-09-21T06:00:50.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.294+0000 7fb92affd640 1 --2- 172.21.15.117:0/1713125091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb92c10a9e0 0x7fb92c1a91a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb91c00d980 tx=0x7fb91c00de40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:50.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.295+0000 7fb923fff640 1 -- 172.21.15.117:0/1713125091 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c017070 con 0x7fb92c10a9e0 2024-09-21T06:00:50.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.295+0000 7fb923fff640 1 -- 172.21.15.117:0/1713125091 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb91c009c30 con 0x7fb92c10a9e0 2024-09-21T06:00:50.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.295+0000 7fb923fff640 1 -- 172.21.15.117:0/1713125091 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c00f620 con 0x7fb92c10a9e0 2024-09-21T06:00:50.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.298+0000 7fb9312c1640 1 -- 172.21.15.117:0/1713125091 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb92c1af3d0 con 0x7fb92c10a9e0 2024-09-21T06:00:50.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.298+0000 7fb9312c1640 1 -- 172.21.15.117:0/1713125091 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb92c112380 con 0x7fb92c10a9e0 2024-09-21T06:00:50.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.307+0000 7fb937835740 1 Processor -- start 2024-09-21T06:00:50.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.307+0000 7fb937835740 1 -- start start 2024-09-21T06:00:50.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.307+0000 7fb937835740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357fea1070 0x5635802482c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:50.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.307+0000 7fb937835740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56357fe781b0 0x563580248800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:50.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.307+0000 7fb937835740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357fe80390 0x563580248d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:50.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.307+0000 7fb937835740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5635801beea0 con 0x56357fea1070 2024-09-21T06:00:50.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.307+0000 7fb937835740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5635801ae2d0 con 0x56357fe781b0 2024-09-21T06:00:50.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.307+0000 7fb937835740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5635801b20e0 con 0x56357fe80390 2024-09-21T06:00:50.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.313+0000 7fb92a7fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357fe80390 0x563580248d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:50.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.313+0000 7fb92b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357fea1070 0x5635802482c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:50.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.313+0000 7fb92b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357fea1070 0x5635802482c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39272/0 (socket says 172.21.15.117:39272) 2024-09-21T06:00:50.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.313+0000 7fb92b7fe640 1 -- 172.21.15.117:0/804137330 learned_addr learned my addr 172.21.15.117:0/804137330 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:50.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.314+0000 7fb923fff640 1 -- 172.21.15.117:0/1713125091 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fb91c010460 con 0x7fb92c10a9e0 2024-09-21T06:00:50.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.315+0000 7fb923fff640 1 --2- 172.21.15.117:0/1713125091 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb9180b6880 0x7fb9180b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:50.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.316+0000 7fb92affd640 1 --2- 172.21.15.117:0/804137330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56357fe781b0 0x563580248800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:50.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.316+0000 7fb92b7fe640 1 -- 172.21.15.117:0/804137330 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357fe80390 msgr2=0x563580248d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:50.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.316+0000 7fb92b7fe640 1 --2- 172.21.15.117:0/804137330 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357fe80390 0x563580248d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:50.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.316+0000 7fb92b7fe640 1 -- 172.21.15.117:0/804137330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56357fe781b0 msgr2=0x563580248800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:50.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.316+0000 7fb92b7fe640 1 --2- 172.21.15.117:0/804137330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56357fe781b0 0x563580248800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:50.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.316+0000 7fb92b7fe640 1 -- 172.21.15.117:0/804137330 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56357ff0c650 con 0x56357fea1070 2024-09-21T06:00:50.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.316+0000 7fb92b7fe640 1 --2- 172.21.15.117:0/1713125091 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb9180b6880 0x7fb9180b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:50.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.317+0000 7fb923fff640 1 -- 172.21.15.117:0/1713125091 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fb91c0e7060 con 0x7fb92c10a9e0 2024-09-21T06:00:50.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.318+0000 7fb92b7fe640 1 --2- 172.21.15.117:0/804137330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357fea1070 0x5635802482c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb92402ae60 tx=0x7fb924031480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca259fbbafa07e8a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:50.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.319+0000 7fb92b7fe640 1 --2- 172.21.15.117:0/1713125091 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb9180b6880 0x7fb9180b8d90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb924006010 tx=0x7fb924046000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:50.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.319+0000 7fb9217fa640 1 -- 172.21.15.117:0/804137330 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92404d070 con 0x56357fea1070 2024-09-21T06:00:50.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.319+0000 7fb9217fa640 1 -- 172.21.15.117:0/804137330 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb924036040 con 0x56357fea1070 2024-09-21T06:00:50.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.321+0000 7fb937835740 1 -- 172.21.15.117:0/804137330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357fea1070 msgr2=0x5635802482c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:50.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.321+0000 7fb937835740 1 --2- 172.21.15.117:0/804137330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357fea1070 0x5635802482c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb92402ae60 tx=0x7fb924031480 comp rx=0 tx=0).stop 2024-09-21T06:00:50.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.322+0000 7fb937835740 1 -- 172.21.15.117:0/804137330 shutdown_connections 2024-09-21T06:00:50.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.322+0000 7fb937835740 1 --2- 172.21.15.117:0/804137330 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357fe80390 0x563580248d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:50.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.322+0000 7fb937835740 1 --2- 172.21.15.117:0/804137330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56357fe781b0 0x563580248800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:50.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.322+0000 7fb937835740 1 --2- 172.21.15.117:0/804137330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357fea1070 0x5635802482c0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:50.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.322+0000 7fb937835740 1 -- 172.21.15.117:0/804137330 >> 172.21.15.117:0/804137330 conn(0x56357fe95460 msgr2=0x5635802461b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:50.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.322+0000 7fb937835740 1 -- 172.21.15.117:0/804137330 shutdown_connections 2024-09-21T06:00:50.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.323+0000 7fb937835740 1 -- 172.21.15.117:0/804137330 wait complete. 2024-09-21T06:00:50.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.323+0000 7fb937835740 1 -- start start 2024-09-21T06:00:50.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.323+0000 7fb937835740 1 Processor -- start 2024-09-21T06:00:50.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.324+0000 7fb937835740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357fe781b0 0x563580253950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:50.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.324+0000 7fb937835740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357fe80390 0x563580253e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:50.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.324+0000 7fb937835740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56357fea1070 0x56358025fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:50.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.324+0000 7fb937835740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56358000f700 con 0x56357fe80390 2024-09-21T06:00:50.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.324+0000 7fb937835740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563580149160 con 0x56357fea1070 2024-09-21T06:00:50.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.324+0000 7fb937835740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x563580256a90 con 0x56357fe781b0 2024-09-21T06:00:50.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.324+0000 7fb937835740 10 client.0 _ll_register_callbacks cb 0x5635801629e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:00:50.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.324+0000 7fb92a7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357fe80390 0x563580253e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:50.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.325+0000 7fb92a7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357fe80390 0x563580253e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39278/0 (socket says 172.21.15.117:39278) 2024-09-21T06:00:50.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.325+0000 7fb92a7fc640 1 -- 172.21.15.117:0/3747851285 learned_addr learned my addr 172.21.15.117:0/3747851285 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:50.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.325+0000 7fb92affd640 1 --2- 172.21.15.117:0/3747851285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357fe781b0 0x563580253950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:50.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.325+0000 7fb92a7fc640 1 -- 172.21.15.117:0/3747851285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357fe781b0 msgr2=0x563580253950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:50.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.325+0000 7fb92a7fc640 1 --2- 172.21.15.117:0/3747851285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357fe781b0 0x563580253950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:50.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.325+0000 7fb92a7fc640 1 -- 172.21.15.117:0/3747851285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56357fea1070 msgr2=0x56358025fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:50.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.325+0000 7fb92a7fc640 1 --2- 172.21.15.117:0/3747851285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56357fea1070 0x56358025fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:50.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.325+0000 7fb92a7fc640 1 -- 172.21.15.117:0/3747851285 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563580176c90 con 0x56357fe80390 2024-09-21T06:00:50.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.326+0000 7fb92a7fc640 1 --2- 172.21.15.117:0/3747851285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357fe80390 0x563580253e90 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb91400f5f0 tx=0x7fb91400fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94dd74f9dae22282 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:50.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.326+0000 7fb9127fc640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:00:50.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.326+0000 7fb9127fc640 1 -- 172.21.15.117:0/3747851285 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb914004220 con 0x56357fe80390 2024-09-21T06:00:50.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.326+0000 7fb9127fc640 1 -- 172.21.15.117:0/3747851285 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fb9140043c0 con 0x56357fe80390 2024-09-21T06:00:50.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.326+0000 7fb937835740 1 -- 172.21.15.117:0/3747851285 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb924036040 con 0x56357fe80390 2024-09-21T06:00:50.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.327+0000 7fb9127fc640 1 -- 172.21.15.117:0/3747851285 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb914005020 con 0x56357fe80390 2024-09-21T06:00:50.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.327+0000 7fb937835740 10 client.16158 fetch_fsmap learned FSMap version 27 2024-09-21T06:00:50.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.327+0000 7fb937835740 1 -- 172.21.15.117:0/3747851285 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563580262870 con 0x56357fe80390 2024-09-21T06:00:50.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.333+0000 7fb9127fc640 1 -- 172.21.15.117:0/3747851285 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb924036040 con 0x56357fe80390 2024-09-21T06:00:50.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.333+0000 7fb9127fc640 1 -- 172.21.15.117:0/3747851285 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb9140047a0 con 0x56357fe80390 2024-09-21T06:00:50.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.334+0000 7fb937835740 10 client.16158 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:00:50.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.334+0000 7fb937835740 10 client.16158 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:00:50.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.334+0000 7fb937835740 20 client.16158 populate_metadata read hostname 'smithi117' 2024-09-21T06:00:50.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.334+0000 7fb937835740 1 --2- 172.21.15.117:0/3747851285 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x56357fe5f9f0 0x563580265e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:50.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.334+0000 7fb937835740 4 client.16158 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:00:50.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.334+0000 7fb937835740 1 -- 172.21.15.117:0/3747851285 --> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635801c0df0 con 0x56357fe5f9f0 2024-09-21T06:00:50.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.334+0000 7fb92affd640 1 --2- 172.21.15.117:0/3747851285 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x56357fe5f9f0 0x563580265e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:50.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.335+0000 7fb9127fc640 10 client.16158 ms_handle_connect on v2:172.21.15.192:6836/2683860842 2024-09-21T06:00:50.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.336+0000 7fb92affd640 1 --2- 172.21.15.117:0/3747851285 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x56357fe5f9f0 0x563580265e40 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b11598a93d5fa055 server_cookie=b833b83fb63ecca3 in_seq=0 out_seq=0 2024-09-21T06:00:50.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.337+0000 7fb9127fc640 1 -- 172.21.15.117:0/3747851285 <== mds.2 v2:172.21.15.192:6836/2683860842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5635801c0df0 con 0x56357fe5f9f0 2024-09-21T06:00:50.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.337+0000 7fb9127fc640 10 client.16158 handle_command_reply: tid=0 2024-09-21T06:00:50.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.338+0000 7fb937835740 1 client.16158 shutdown 2024-09-21T06:00:50.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.345+0000 7fb937835740 1 -- 172.21.15.117:0/3747851285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357fe80390 msgr2=0x563580253e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:50.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.345+0000 7fb937835740 1 --2- 172.21.15.117:0/3747851285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357fe80390 0x563580253e90 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb91400f5f0 tx=0x7fb91400fab0 comp rx=0 tx=0).stop 2024-09-21T06:00:50.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.345+0000 7fb937835740 1 -- 172.21.15.117:0/3747851285 shutdown_connections 2024-09-21T06:00:50.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.345+0000 7fb937835740 1 --2- 172.21.15.117:0/3747851285 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x56357fe5f9f0 0x563580265e40 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:50.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.345+0000 7fb937835740 1 --2- 172.21.15.117:0/3747851285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357fe80390 0x563580253e90 secure :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb91400f5f0 tx=0x7fb91400fab0 comp rx=0 tx=0).stop 2024-09-21T06:00:50.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.345+0000 7fb937835740 1 --2- 172.21.15.117:0/3747851285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56357fea1070 0x56358025fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:50.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.345+0000 7fb937835740 1 --2- 172.21.15.117:0/3747851285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357fe781b0 0x563580253950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:50.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.345+0000 7fb937835740 1 -- 172.21.15.117:0/3747851285 >> 172.21.15.117:0/3747851285 conn(0x56357fe95460 msgr2=0x5635802492f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:50.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.345+0000 7fb9127fc640 0 client.16158 ms_handle_reset on v2:172.21.15.192:6836/2683860842 2024-09-21T06:00:50.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.346+0000 7fb937835740 1 -- 172.21.15.117:0/3747851285 shutdown_connections 2024-09-21T06:00:50.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.346+0000 7fb937835740 1 -- 172.21.15.117:0/3747851285 wait complete. 2024-09-21T06:00:50.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.346+0000 7fb937835740 20 client.16158 trim_cache size 0 max 16384 2024-09-21T06:00:50.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.387+0000 7fb937835740 1 Processor -- start 2024-09-21T06:00:50.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.388+0000 7fb937835740 1 -- start start 2024-09-21T06:00:50.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.389+0000 7fb937835740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357fe781b0 0x563580246200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:50.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.389+0000 7fb937835740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56357fea1070 0x56358025fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:50.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.390+0000 7fb937835740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357fe34da0 0x563580250b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:50.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.390+0000 7fb937835740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c1c6c60 con 0x56357fe781b0 2024-09-21T06:00:50.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.390+0000 7fb937835740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c1c87a0 con 0x56357fea1070 2024-09-21T06:00:50.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.390+0000 7fb937835740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c1c7330 con 0x56357fe34da0 2024-09-21T06:00:50.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.390+0000 7fb92a7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56357fea1070 0x56358025fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:50.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.390+0000 7fb92a7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56357fea1070 0x56358025fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:39066/0 (socket says 172.21.15.117:39066) 2024-09-21T06:00:50.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.390+0000 7fb92a7fc640 1 -- 172.21.15.117:0/874915910 learned_addr learned my addr 172.21.15.117:0/874915910 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:50.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.390+0000 7fb92b7fe640 1 --2- 172.21.15.117:0/874915910 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357fe34da0 0x563580250b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:50.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.391+0000 7fb92affd640 1 --2- 172.21.15.117:0/874915910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357fe781b0 0x563580246200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:50.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.392+0000 7fb92b7fe640 1 -- 172.21.15.117:0/874915910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56357fea1070 msgr2=0x56358025fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:50.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.392+0000 7fb92b7fe640 1 --2- 172.21.15.117:0/874915910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56357fea1070 0x56358025fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:50.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.392+0000 7fb92b7fe640 1 -- 172.21.15.117:0/874915910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357fe781b0 msgr2=0x563580246200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:50.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.392+0000 7fb92b7fe640 1 --2- 172.21.15.117:0/874915910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357fe781b0 0x563580246200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:50.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.392+0000 7fb92b7fe640 1 -- 172.21.15.117:0/874915910 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563580251e50 con 0x56357fe34da0 2024-09-21T06:00:50.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.392+0000 7fb92affd640 1 --2- 172.21.15.117:0/874915910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357fe781b0 0x563580246200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:00:50.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.393+0000 7fb92b7fe640 1 --2- 172.21.15.117:0/874915910 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357fe34da0 0x563580250b40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb9240094e0 tx=0x7fb92402a8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f034c996726e1188 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:50.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.396+0000 7fb920ff9640 1 -- 172.21.15.117:0/874915910 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb924055070 con 0x56357fe34da0 2024-09-21T06:00:50.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.396+0000 7fb920ff9640 1 -- 172.21.15.117:0/874915910 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb924036050 con 0x56357fe34da0 2024-09-21T06:00:50.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.398+0000 7fb937835740 1 -- 172.21.15.117:0/874915910 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357fe34da0 msgr2=0x563580250b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:50.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.398+0000 7fb937835740 1 --2- 172.21.15.117:0/874915910 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357fe34da0 0x563580250b40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb9240094e0 tx=0x7fb92402a8f0 comp rx=0 tx=0).stop 2024-09-21T06:00:50.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.402+0000 7fb937835740 1 -- 172.21.15.117:0/874915910 shutdown_connections 2024-09-21T06:00:50.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.402+0000 7fb937835740 1 --2- 172.21.15.117:0/874915910 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357fe34da0 0x563580250b40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:50.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.402+0000 7fb937835740 1 --2- 172.21.15.117:0/874915910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56357fea1070 0x56358025fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:50.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.402+0000 7fb937835740 1 --2- 172.21.15.117:0/874915910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357fe781b0 0x563580246200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:50.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.402+0000 7fb937835740 1 -- 172.21.15.117:0/874915910 >> 172.21.15.117:0/874915910 conn(0x56357fe95460 msgr2=0x563580245af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:50.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.403+0000 7fb937835740 1 -- 172.21.15.117:0/874915910 shutdown_connections 2024-09-21T06:00:50.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.403+0000 7fb937835740 1 -- 172.21.15.117:0/874915910 wait complete. 2024-09-21T06:00:50.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.404+0000 7fb937835740 1 -- start start 2024-09-21T06:00:50.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.404+0000 7fb937835740 1 Processor -- start 2024-09-21T06:00:50.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.405+0000 7fb937835740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357fe34da0 0x563580257cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:50.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.407+0000 7fb937835740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357fe781b0 0x563580258210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:50.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.407+0000 7fb937835740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56357fea1070 0x563580258750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:50.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.407+0000 7fb937835740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c1c80d0 con 0x56357fe781b0 2024-09-21T06:00:50.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.407+0000 7fb937835740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c1c7a00 con 0x56357fea1070 2024-09-21T06:00:50.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.407+0000 7fb937835740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c1aa2d0 con 0x56357fe34da0 2024-09-21T06:00:50.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.407+0000 7fb937835740 10 client.0 _ll_register_callbacks cb 0x7fb91400a4e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:00:50.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.409+0000 7fb92a7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357fe781b0 0x563580258210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:50.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.409+0000 7fb92a7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357fe781b0 0x563580258210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39286/0 (socket says 172.21.15.117:39286) 2024-09-21T06:00:50.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.409+0000 7fb92a7fc640 1 -- 172.21.15.117:0/3345564895 learned_addr learned my addr 172.21.15.117:0/3345564895 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:50.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.409+0000 7fb92b7fe640 1 --2- 172.21.15.117:0/3345564895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56357fea1070 0x563580258750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:50.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.410+0000 7fb92a7fc640 1 -- 172.21.15.117:0/3345564895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357fe34da0 msgr2=0x563580257cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:00:50.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.410+0000 7fb92a7fc640 1 --2- 172.21.15.117:0/3345564895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357fe34da0 0x563580257cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:50.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.410+0000 7fb92a7fc640 1 -- 172.21.15.117:0/3345564895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56357fea1070 msgr2=0x563580258750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:50.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.410+0000 7fb92a7fc640 1 --2- 172.21.15.117:0/3345564895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56357fea1070 0x563580258750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:50.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.410+0000 7fb92a7fc640 1 -- 172.21.15.117:0/3345564895 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635802cf840 con 0x56357fe781b0 2024-09-21T06:00:50.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.411+0000 7fb92a7fc640 1 --2- 172.21.15.117:0/3345564895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357fe781b0 0x563580258210 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb91400a2a0 tx=0x7fb9140053f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f93046cf3c347839 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:50.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.411+0000 7fb9127fc640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:00:50.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.412+0000 7fb9127fc640 1 -- 172.21.15.117:0/3345564895 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb914030030 con 0x56357fe781b0 2024-09-21T06:00:50.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.412+0000 7fb937835740 1 -- 172.21.15.117:0/3345564895 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb924036050 con 0x56357fe781b0 2024-09-21T06:00:50.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.412+0000 7fb9127fc640 1 -- 172.21.15.117:0/3345564895 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fb9140055e0 con 0x56357fe781b0 2024-09-21T06:00:50.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.413+0000 7fb9127fc640 1 -- 172.21.15.117:0/3345564895 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb91403a050 con 0x56357fe781b0 2024-09-21T06:00:50.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.416+0000 7fb937835740 10 client.16164 fetch_fsmap learned FSMap version 27 2024-09-21T06:00:50.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.416+0000 7fb937835740 1 -- 172.21.15.117:0/3345564895 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5635802d0a90 con 0x56357fe781b0 2024-09-21T06:00:50.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.419+0000 7fb9127fc640 1 -- 172.21.15.117:0/3345564895 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb924036050 con 0x56357fe781b0 2024-09-21T06:00:50.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.419+0000 7fb9127fc640 1 -- 172.21.15.117:0/3345564895 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb9140035f0 con 0x56357fe781b0 2024-09-21T06:00:50.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.420+0000 7fb937835740 10 client.16164 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:00:50.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.420+0000 7fb937835740 10 client.16164 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:00:50.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.420+0000 7fb937835740 20 client.16164 populate_metadata read hostname 'smithi117' 2024-09-21T06:00:50.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.420+0000 7fb937835740 1 --2- 172.21.15.117:0/3345564895 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x5635802d1270 0x5635802d3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:50.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.420+0000 7fb937835740 4 client.16164 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:00:50.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.420+0000 7fb937835740 1 -- 172.21.15.117:0/3345564895 --> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5635801bc8a0 con 0x5635802d1270 2024-09-21T06:00:50.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.422+0000 7fb92affd640 1 --2- 172.21.15.117:0/3345564895 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x5635802d1270 0x5635802d3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:50.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.427+0000 7fb92affd640 1 --2- 172.21.15.117:0/3345564895 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x5635802d1270 0x5635802d3690 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d7b7cb8f684eb3be server_cookie=276ffe86359ed07c in_seq=0 out_seq=0 2024-09-21T06:00:50.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.427+0000 7fb9127fc640 10 client.16164 ms_handle_connect on v2:172.21.15.192:6836/2683860842 2024-09-21T06:00:50.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.430+0000 7fb9127fc640 1 -- 172.21.15.117:0/3345564895 <== mds.2 v2:172.21.15.192:6836/2683860842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5635801bc8a0 con 0x5635802d1270 2024-09-21T06:00:50.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.430+0000 7fb9127fc640 10 client.16164 handle_command_reply: tid=0 2024-09-21T06:00:50.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.430+0000 7fb937835740 1 client.16164 shutdown 2024-09-21T06:00:50.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.431+0000 7fb937835740 1 -- 172.21.15.117:0/3345564895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357fe781b0 msgr2=0x563580258210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:50.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.431+0000 7fb937835740 1 --2- 172.21.15.117:0/3345564895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357fe781b0 0x563580258210 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb91400a2a0 tx=0x7fb9140053f0 comp rx=0 tx=0).stop 2024-09-21T06:00:50.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.431+0000 7fb937835740 1 -- 172.21.15.117:0/3345564895 shutdown_connections 2024-09-21T06:00:50.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.431+0000 7fb937835740 1 --2- 172.21.15.117:0/3345564895 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x5635802d1270 0x5635802d3690 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:50.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.431+0000 7fb937835740 1 --2- 172.21.15.117:0/3345564895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56357fe781b0 0x563580258210 secure :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb91400a2a0 tx=0x7fb9140053f0 comp rx=0 tx=0).stop 2024-09-21T06:00:50.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.431+0000 7fb937835740 1 --2- 172.21.15.117:0/3345564895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56357fea1070 0x563580258750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:50.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.431+0000 7fb937835740 1 --2- 172.21.15.117:0/3345564895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56357fe34da0 0x563580257cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:50.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.431+0000 7fb937835740 1 -- 172.21.15.117:0/3345564895 >> 172.21.15.117:0/3345564895 conn(0x56357fe95460 msgr2=0x563580254e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:50.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.431+0000 7fb9127fc640 0 client.16164 ms_handle_reset on v2:172.21.15.192:6836/2683860842 2024-09-21T06:00:50.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.433+0000 7fb937835740 1 -- 172.21.15.117:0/3345564895 shutdown_connections 2024-09-21T06:00:50.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.433+0000 7fb937835740 1 -- 172.21.15.117:0/3345564895 wait complete. 2024-09-21T06:00:50.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.433+0000 7fb937835740 20 client.16164 trim_cache size 0 max 16384 2024-09-21T06:00:50.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.435+0000 7fb9312c1640 1 -- 172.21.15.117:0/1713125091 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb9180b6880 msgr2=0x7fb9180b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:50.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.435+0000 7fb9312c1640 1 --2- 172.21.15.117:0/1713125091 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb9180b6880 0x7fb9180b8d90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb924006010 tx=0x7fb924046000 comp rx=0 tx=0).stop 2024-09-21T06:00:50.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.435+0000 7fb9312c1640 1 -- 172.21.15.117:0/1713125091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb92c10a9e0 msgr2=0x7fb92c1a91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:50.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.435+0000 7fb9312c1640 1 --2- 172.21.15.117:0/1713125091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb92c10a9e0 0x7fb92c1a91a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb91c00d980 tx=0x7fb91c00de40 comp rx=0 tx=0).stop 2024-09-21T06:00:50.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.436+0000 7fb9312c1640 1 -- 172.21.15.117:0/1713125091 shutdown_connections 2024-09-21T06:00:50.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.436+0000 7fb9312c1640 1 --2- 172.21.15.117:0/1713125091 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb9180b6880 0x7fb9180b8d90 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:50.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.436+0000 7fb9312c1640 1 --2- 172.21.15.117:0/1713125091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb92c111f40 0x7fb92c1a9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:50.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.436+0000 7fb9312c1640 1 --2- 172.21.15.117:0/1713125091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb92c11cc30 0x7fb92c1ae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:50.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.436+0000 7fb9312c1640 1 --2- 172.21.15.117:0/1713125091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb92c10a9e0 0x7fb92c1a91a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:50.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.436+0000 7fb9312c1640 1 -- 172.21.15.117:0/1713125091 >> 172.21.15.117:0/1713125091 conn(0x7fb92c070ec0 msgr2=0x7fb92c11d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:50.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.436+0000 7fb9312c1640 1 -- 172.21.15.117:0/1713125091 shutdown_connections 2024-09-21T06:00:50.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:50.436+0000 7fb9312c1640 1 -- 172.21.15.117:0/1713125091 wait complete. 2024-09-21T06:00:51.030 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:50 smithi117 ceph-mon[30947]: pgmap v433: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 875 MiB/s wr, 1.22k op/s 2024-09-21T06:00:51.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:50 smithi138 ceph-mon[34391]: pgmap v433: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 875 MiB/s wr, 1.22k op/s 2024-09-21T06:00:51.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:50 smithi192 ceph-mon[34700]: pgmap v433: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 875 MiB/s wr, 1.22k op/s 2024-09-21T06:00:51.579 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:51.575+0000 7f81d2ff5640 1 -- 172.21.15.117:0/2405326368 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f81e0022070 con 0x55b0a577e1b0 2024-09-21T06:00:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:51.575+0000 7f81d2ff5640 10 client.25529 handle_command_reply: tid=0 2024-09-21T06:00:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:51.576+0000 7f81fc824740 1 client.25529 shutdown 2024-09-21T06:00:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:51.576+0000 7f81fc824740 1 -- 172.21.15.117:0/2405326368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0a5772300 msgr2=0x55b0a5b330c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:51.576+0000 7f81fc824740 1 --2- 172.21.15.117:0/2405326368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0a5772300 0x55b0a5b330c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f81e8106180 tx=0x7f81e80ce000 comp rx=0 tx=0).stop 2024-09-21T06:00:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:51.577+0000 7f81fc824740 1 -- 172.21.15.117:0/2405326368 shutdown_connections 2024-09-21T06:00:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:51.577+0000 7f81fc824740 1 --2- 172.21.15.117:0/2405326368 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b0a577e1b0 0x55b0a5bae410 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:51.577+0000 7f81fc824740 1 --2- 172.21.15.117:0/2405326368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b0a5772300 0x55b0a5b330c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:51.577+0000 7f81fc824740 1 --2- 172.21.15.117:0/2405326368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b0a575d4d0 0x55b0a5b38000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:51.577+0000 7f81fc824740 1 --2- 172.21.15.117:0/2405326368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b0a57552f0 0x55b0a5b37a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:51.577+0000 7f81fc824740 1 -- 172.21.15.117:0/2405326368 >> 172.21.15.117:0/2405326368 conn(0x55b0a5b29660 msgr2=0x55b0a5b31010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:51.577+0000 7f81d2ff5640 0 client.25529 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:00:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:51.577+0000 7f81fc824740 1 -- 172.21.15.117:0/2405326368 shutdown_connections 2024-09-21T06:00:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:51.577+0000 7f81fc824740 1 -- 172.21.15.117:0/2405326368 wait complete. 2024-09-21T06:00:51.583 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:51.580+0000 7f81fc824740 20 client.25529 trim_cache size 0 max 16384 2024-09-21T06:00:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T06:00:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T06:00:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T06:00:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T06:00:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:51.588+0000 7f81f57da640 1 -- 172.21.15.117:0/3900325389 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f81d80b67d0 msgr2=0x7f81d80b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:51.588+0000 7f81f57da640 1 --2- 172.21.15.117:0/3900325389 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f81d80b67d0 0x7f81d80b8ce0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55b0a5b22820 tx=0x7f81e0006cb0 comp rx=0 tx=0).stop 2024-09-21T06:00:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:51.588+0000 7f81f57da640 1 -- 172.21.15.117:0/3900325389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81f010eab0 msgr2=0x7f81f01a6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:51.588+0000 7f81f57da640 1 --2- 172.21.15.117:0/3900325389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81f010eab0 0x7f81f01a6000 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f81e8009620 tx=0x7f81e8003ba0 comp rx=0 tx=0).stop 2024-09-21T06:00:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:51.588+0000 7f81f57da640 1 -- 172.21.15.117:0/3900325389 shutdown_connections 2024-09-21T06:00:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:51.588+0000 7f81f57da640 1 --2- 172.21.15.117:0/3900325389 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f81d80b67d0 0x7f81d80b8ce0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:51.588+0000 7f81f57da640 1 --2- 172.21.15.117:0/3900325389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81f010eab0 0x7f81f01a6000 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:51.588+0000 7f81f57da640 1 --2- 172.21.15.117:0/3900325389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81f010ab50 0x7f81f01a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:51.588+0000 7f81f57da640 1 --2- 172.21.15.117:0/3900325389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81f010a200 0x7f81f01a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:51.589+0000 7f81f57da640 1 -- 172.21.15.117:0/3900325389 >> 172.21.15.117:0/3900325389 conn(0x7f81f0079400 msgr2=0x7f81f010c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:51.589+0000 7f81f57da640 1 -- 172.21.15.117:0/3900325389 shutdown_connections 2024-09-21T06:00:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:00:51.589+0000 7f81f57da640 1 -- 172.21.15.117:0/3900325389 wait complete. 2024-09-21T06:00:54.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:53 smithi117 ceph-mon[30947]: pgmap v434: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 877 MiB/s wr, 1.22k op/s 2024-09-21T06:00:54.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:53 smithi138 ceph-mon[34391]: pgmap v434: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 877 MiB/s wr, 1.22k op/s 2024-09-21T06:00:54.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:53 smithi192 ceph-mon[34700]: pgmap v434: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 877 MiB/s wr, 1.22k op/s 2024-09-21T06:00:55.130 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T06:00:55.740 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:00:56.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:55 smithi117 ceph-mon[30947]: pgmap v435: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 988 MiB/s wr, 1.41k op/s 2024-09-21T06:00:56.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:55 smithi138 ceph-mon[34391]: pgmap v435: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 988 MiB/s wr, 1.41k op/s 2024-09-21T06:00:56.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:55 smithi192 ceph-mon[34700]: pgmap v435: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 988 MiB/s wr, 1.41k op/s 2024-09-21T06:00:57.935 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:57 smithi117 ceph-mon[30947]: pgmap v436: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 811 MiB/s wr, 1.15k op/s 2024-09-21T06:00:58.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:57 smithi138 ceph-mon[34391]: pgmap v436: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 811 MiB/s wr, 1.15k op/s 2024-09-21T06:00:58.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:57 smithi192 ceph-mon[34700]: pgmap v436: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 811 MiB/s wr, 1.15k op/s 2024-09-21T06:00:58.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.902+0000 7f062f0a0640 1 -- 172.21.15.117:0/3457327982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f062810a9e0 msgr2=0x7f062810adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:58.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.902+0000 7f062f0a0640 1 --2- 172.21.15.117:0/3457327982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f062810a9e0 0x7f062810adf0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f061c009900 tx=0x7f061c02a420 comp rx=0 tx=0).stop 2024-09-21T06:00:58.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.904+0000 7f062f0a0640 1 -- 172.21.15.117:0/3457327982 shutdown_connections 2024-09-21T06:00:58.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.904+0000 7f062f0a0640 1 --2- 172.21.15.117:0/3457327982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f062810a9e0 0x7f062810adf0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:58.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.904+0000 7f062f0a0640 1 --2- 172.21.15.117:0/3457327982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f062811cc30 0x7f062811f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:58.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.904+0000 7f062f0a0640 1 --2- 172.21.15.117:0/3457327982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0628111f40 0x7f062810a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:58.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.904+0000 7f062f0a0640 1 -- 172.21.15.117:0/3457327982 >> 172.21.15.117:0/3457327982 conn(0x7f0628070ec0 msgr2=0x7f06280712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:58.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.906+0000 7f062f0a0640 1 -- 172.21.15.117:0/3457327982 shutdown_connections 2024-09-21T06:00:58.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.908+0000 7f062f0a0640 1 -- 172.21.15.117:0/3457327982 wait complete. 2024-09-21T06:00:58.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.909+0000 7f062f0a0640 1 Processor -- start 2024-09-21T06:00:58.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.910+0000 7f062f0a0640 1 -- start start 2024-09-21T06:00:58.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.910+0000 7f062f0a0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f062810a9e0 0x7f062811a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:58.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.910+0000 7f062f0a0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0628111f40 0x7f0628113710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:58.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.910+0000 7f062f0a0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f062811cc30 0x7f0628113cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:58.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.910+0000 7f062f0a0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0628122900 con 0x7f062810a9e0 2024-09-21T06:00:58.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.910+0000 7f062f0a0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0628122780 con 0x7f0628111f40 2024-09-21T06:00:58.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.910+0000 7f062f0a0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0628122a80 con 0x7f062811cc30 2024-09-21T06:00:58.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.911+0000 7f062ce3c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f062810a9e0 0x7f062811a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:58.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.911+0000 7f062ce3c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f062810a9e0 0x7f062811a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51492/0 (socket says 172.21.15.117:51492) 2024-09-21T06:00:58.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.911+0000 7f062ce3c640 1 -- 172.21.15.117:0/719674501 learned_addr learned my addr 172.21.15.117:0/719674501 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:58.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.911+0000 7f0627fff640 1 --2- 172.21.15.117:0/719674501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0628111f40 0x7f0628113710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:58.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.912+0000 7f062d63d640 1 --2- 172.21.15.117:0/719674501 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f062811cc30 0x7f0628113cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:58.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.914+0000 7f0627fff640 1 -- 172.21.15.117:0/719674501 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f062811cc30 msgr2=0x7f0628113cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:58.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.914+0000 7f0627fff640 1 --2- 172.21.15.117:0/719674501 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f062811cc30 0x7f0628113cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:58.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.914+0000 7f0627fff640 1 -- 172.21.15.117:0/719674501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f062810a9e0 msgr2=0x7f062811a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:58.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.914+0000 7f0627fff640 1 --2- 172.21.15.117:0/719674501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f062810a9e0 0x7f062811a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:58.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.914+0000 7f0627fff640 1 -- 172.21.15.117:0/719674501 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06281144f0 con 0x7f0628111f40 2024-09-21T06:00:58.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.919+0000 7f0627fff640 1 --2- 172.21.15.117:0/719674501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0628111f40 0x7f0628113710 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f061c009230 tx=0x7f061c003ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:58.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.924+0000 7f0625ffb640 1 -- 172.21.15.117:0/719674501 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c040030 con 0x7f0628111f40 2024-09-21T06:00:58.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.924+0000 7f0625ffb640 1 -- 172.21.15.117:0/719674501 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f061c0040e0 con 0x7f0628111f40 2024-09-21T06:00:58.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.924+0000 7f0625ffb640 1 -- 172.21.15.117:0/719674501 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c0317c0 con 0x7f0628111f40 2024-09-21T06:00:58.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.925+0000 7f062f0a0640 1 -- 172.21.15.117:0/719674501 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0628119110 con 0x7f0628111f40 2024-09-21T06:00:58.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.925+0000 7f062f0a0640 1 -- 172.21.15.117:0/719674501 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0628119570 con 0x7f0628111f40 2024-09-21T06:00:58.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.932+0000 7f0635614740 1 Processor -- start 2024-09-21T06:00:58.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.934+0000 7f0635614740 1 -- start start 2024-09-21T06:00:58.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.935+0000 7f0635614740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ad4508070 0x564ad48af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:58.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.935+0000 7f0635614740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad44df1b0 0x564ad48af800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:58.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.935+0000 7f0635614740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ad44e7390 0x564ad48afd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:58.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.935+0000 7f0635614740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564ad4825ea0 con 0x564ad44e7390 2024-09-21T06:00:58.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.935+0000 7f0635614740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564ad48152d0 con 0x564ad4508070 2024-09-21T06:00:58.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.935+0000 7f0635614740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564ad48190e0 con 0x564ad44df1b0 2024-09-21T06:00:58.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.935+0000 7f0625ffb640 1 -- 172.21.15.117:0/719674501 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f061c043030 con 0x7f0628111f40 2024-09-21T06:00:58.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.937+0000 7f062d63d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ad4508070 0x564ad48af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:58.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.937+0000 7f062ce3c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad44df1b0 0x564ad48af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:58.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.937+0000 7f062d63d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ad4508070 0x564ad48af2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:47218/0 (socket says 172.21.15.117:47218) 2024-09-21T06:00:58.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.937+0000 7f062d63d640 1 -- 172.21.15.117:0/3947894328 learned_addr learned my addr 172.21.15.117:0/3947894328 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:58.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.940+0000 7f0625ffb640 1 --2- 172.21.15.117:0/719674501 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f06100b6850 0x7f06100b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:58.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.941+0000 7f0627fff640 1 --2- 172.21.15.117:0/3947894328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ad44e7390 0x564ad48afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:58.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.941+0000 7f062d63d640 1 --2- 172.21.15.117:0/719674501 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f06100b6850 0x7f06100b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:58.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.942+0000 7f062ce3c640 1 -- 172.21.15.117:0/3947894328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ad4508070 msgr2=0x564ad48af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:58.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.942+0000 7f062ce3c640 1 --2- 172.21.15.117:0/3947894328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ad4508070 0x564ad48af2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:58.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.942+0000 7f062ce3c640 1 -- 172.21.15.117:0/3947894328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ad44e7390 msgr2=0x564ad48afd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:58.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.942+0000 7f062ce3c640 1 --2- 172.21.15.117:0/3947894328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ad44e7390 0x564ad48afd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:58.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.942+0000 7f062ce3c640 1 -- 172.21.15.117:0/3947894328 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ad4573650 con 0x564ad44df1b0 2024-09-21T06:00:58.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.942+0000 7f0625ffb640 1 -- 172.21.15.117:0/719674501 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f061c107cc0 con 0x7f0628111f40 2024-09-21T06:00:58.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.945+0000 7f062d63d640 1 --2- 172.21.15.117:0/719674501 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f06100b6850 0x7f06100b8d60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0620005c40 tx=0x7f0620004ab0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:58.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.946+0000 7f062ce3c640 1 --2- 172.21.15.117:0/3947894328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad44df1b0 0x564ad48af800 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0618009c80 tx=0x7f0618011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a9b590daf7c522e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:58.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.949+0000 7f060effd640 1 -- 172.21.15.117:0/3947894328 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0618004230 con 0x564ad44df1b0 2024-09-21T06:00:58.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.949+0000 7f060effd640 1 -- 172.21.15.117:0/3947894328 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06180043d0 con 0x564ad44df1b0 2024-09-21T06:00:58.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.951+0000 7f0635614740 1 -- 172.21.15.117:0/3947894328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad44df1b0 msgr2=0x564ad48af800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:58.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.951+0000 7f0635614740 1 --2- 172.21.15.117:0/3947894328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad44df1b0 0x564ad48af800 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0618009c80 tx=0x7f0618011810 comp rx=0 tx=0).stop 2024-09-21T06:00:58.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.951+0000 7f0635614740 1 -- 172.21.15.117:0/3947894328 shutdown_connections 2024-09-21T06:00:58.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.951+0000 7f0635614740 1 --2- 172.21.15.117:0/3947894328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ad44e7390 0x564ad48afd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:58.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.951+0000 7f0635614740 1 --2- 172.21.15.117:0/3947894328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad44df1b0 0x564ad48af800 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:58.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.951+0000 7f0635614740 1 --2- 172.21.15.117:0/3947894328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ad4508070 0x564ad48af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:58.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.951+0000 7f0635614740 1 -- 172.21.15.117:0/3947894328 >> 172.21.15.117:0/3947894328 conn(0x564ad44fc460 msgr2=0x564ad48ad1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:58.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.951+0000 7f0635614740 1 -- 172.21.15.117:0/3947894328 shutdown_connections 2024-09-21T06:00:58.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.952+0000 7f0635614740 1 -- 172.21.15.117:0/3947894328 wait complete. 2024-09-21T06:00:58.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.952+0000 7f0635614740 1 -- start start 2024-09-21T06:00:58.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.952+0000 7f0635614740 1 Processor -- start 2024-09-21T06:00:58.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.953+0000 7f0635614740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ad44df1b0 0x564ad48ba840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:58.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.953+0000 7f0635614740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ad44e7390 0x564ad48bad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:58.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.953+0000 7f0635614740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad4508070 0x564ad48bb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:58.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.953+0000 7f0635614740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564ad4676700 con 0x564ad44e7390 2024-09-21T06:00:58.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.953+0000 7f0635614740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564ad47b0160 con 0x564ad44df1b0 2024-09-21T06:00:58.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.953+0000 7f0635614740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564ad48bdb40 con 0x564ad4508070 2024-09-21T06:00:58.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.954+0000 7f0635614740 10 client.0 _ll_register_callbacks cb 0x564ad47c99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:00:58.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.956+0000 7f062ce3c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ad44df1b0 0x564ad48ba840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:58.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.956+0000 7f0627fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ad44e7390 0x564ad48bad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:58.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.956+0000 7f0627fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ad44e7390 0x564ad48bad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51516/0 (socket says 172.21.15.117:51516) 2024-09-21T06:00:58.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.956+0000 7f0627fff640 1 -- 172.21.15.117:0/931080666 learned_addr learned my addr 172.21.15.117:0/931080666 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:58.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.956+0000 7f062d63d640 1 --2- 172.21.15.117:0/931080666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad4508070 0x564ad48bb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:58.677 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:00:58.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.958+0000 7f0627fff640 1 -- 172.21.15.117:0/931080666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad4508070 msgr2=0x564ad48bb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:58.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.958+0000 7f0627fff640 1 --2- 172.21.15.117:0/931080666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad4508070 0x564ad48bb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:58.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.958+0000 7f0627fff640 1 -- 172.21.15.117:0/931080666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ad44df1b0 msgr2=0x564ad48ba840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:58.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.958+0000 7f0627fff640 1 --2- 172.21.15.117:0/931080666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ad44df1b0 0x564ad48ba840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:58.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.958+0000 7f0627fff640 1 -- 172.21.15.117:0/931080666 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ad47ddc90 con 0x564ad44e7390 2024-09-21T06:00:58.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.967+0000 7f0627fff640 1 --2- 172.21.15.117:0/931080666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ad44e7390 0x564ad48bad80 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f061c0cd830 tx=0x7f061c004510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6265de8d92566a94 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:58.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.967+0000 7f0607fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:00:58.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.969+0000 7f0607fff640 1 -- 172.21.15.117:0/931080666 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c0cee60 con 0x564ad44e7390 2024-09-21T06:00:58.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.969+0000 7f0607fff640 1 -- 172.21.15.117:0/931080666 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f061c03caa0 con 0x564ad44e7390 2024-09-21T06:00:58.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.969+0000 7f0635614740 1 -- 172.21.15.117:0/931080666 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06180043d0 con 0x564ad44e7390 2024-09-21T06:00:58.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.969+0000 7f0607fff640 1 -- 172.21.15.117:0/931080666 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f061c04a070 con 0x564ad44e7390 2024-09-21T06:00:58.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.969+0000 7f0635614740 10 client.16173 fetch_fsmap learned FSMap version 27 2024-09-21T06:00:58.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.969+0000 7f0635614740 1 -- 172.21.15.117:0/931080666 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564ad48eb770 con 0x564ad44e7390 2024-09-21T06:00:58.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.970+0000 7f0607fff640 1 -- 172.21.15.117:0/931080666 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06180043d0 con 0x564ad44e7390 2024-09-21T06:00:58.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.972+0000 7f0607fff640 1 -- 172.21.15.117:0/931080666 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f061c0d21b0 con 0x564ad44e7390 2024-09-21T06:00:58.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.972+0000 7f0635614740 10 client.16173 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:00:58.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.972+0000 7f0635614740 10 client.16173 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:00:58.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.972+0000 7f0635614740 20 client.16173 populate_metadata read hostname 'smithi117' 2024-09-21T06:00:58.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.972+0000 7f0635614740 1 --2- 172.21.15.117:0/931080666 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x564ad44c69f0 0x564ad48eec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:58.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.972+0000 7f0635614740 4 client.16173 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:00:58.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.972+0000 7f0635614740 1 -- 172.21.15.117:0/931080666 --> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ad4827df0 con 0x564ad44c69f0 2024-09-21T06:00:58.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.974+0000 7f062ce3c640 1 --2- 172.21.15.117:0/931080666 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x564ad44c69f0 0x564ad48eec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:58.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.980+0000 7f062ce3c640 1 --2- 172.21.15.117:0/931080666 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x564ad44c69f0 0x564ad48eec90 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7579bfaec249b655 server_cookie=a9b6294445c874ed in_seq=0 out_seq=0 2024-09-21T06:00:58.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.980+0000 7f0607fff640 10 client.16173 ms_handle_connect on v2:172.21.15.192:6838/1727623231 2024-09-21T06:00:58.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.984+0000 7f0607fff640 1 -- 172.21.15.117:0/931080666 <== mds.1 v2:172.21.15.192:6838/1727623231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564ad4827df0 con 0x564ad44c69f0 2024-09-21T06:00:58.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.984+0000 7f0607fff640 10 client.16173 handle_command_reply: tid=0 2024-09-21T06:00:58.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.984+0000 7f0635614740 1 client.16173 shutdown 2024-09-21T06:00:58.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.985+0000 7f0635614740 1 -- 172.21.15.117:0/931080666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ad44e7390 msgr2=0x564ad48bad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:58.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.985+0000 7f0635614740 1 --2- 172.21.15.117:0/931080666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ad44e7390 0x564ad48bad80 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f061c0cd830 tx=0x7f061c004510 comp rx=0 tx=0).stop 2024-09-21T06:00:58.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.985+0000 7f0635614740 1 -- 172.21.15.117:0/931080666 shutdown_connections 2024-09-21T06:00:58.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.985+0000 7f0635614740 1 --2- 172.21.15.117:0/931080666 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x564ad44c69f0 0x564ad48eec90 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:58.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.985+0000 7f0635614740 1 --2- 172.21.15.117:0/931080666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ad44e7390 0x564ad48bad80 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:58.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.985+0000 7f0635614740 1 --2- 172.21.15.117:0/931080666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad4508070 0x564ad48bb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:58.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.985+0000 7f0635614740 1 --2- 172.21.15.117:0/931080666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ad44df1b0 0x564ad48ba840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:58.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.985+0000 7f0635614740 1 -- 172.21.15.117:0/931080666 >> 172.21.15.117:0/931080666 conn(0x564ad44fc460 msgr2=0x564ad48b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:58.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.986+0000 7f0607fff640 0 client.16173 ms_handle_reset on v2:172.21.15.192:6838/1727623231 2024-09-21T06:00:58.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.987+0000 7f0635614740 1 -- 172.21.15.117:0/931080666 shutdown_connections 2024-09-21T06:00:58.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.987+0000 7f0635614740 1 -- 172.21.15.117:0/931080666 wait complete. 2024-09-21T06:00:58.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:57.987+0000 7f0635614740 20 client.16173 trim_cache size 0 max 16384 2024-09-21T06:00:58.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.023+0000 7f0635614740 1 Processor -- start 2024-09-21T06:00:58.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.023+0000 7f0635614740 1 -- start start 2024-09-21T06:00:58.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.024+0000 7f0635614740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad44df1b0 0x564ad48bb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:58.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.024+0000 7f0635614740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ad44e7390 0x564ad48b0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:58.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.024+0000 7f0635614740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ad4508070 0x564ad48ac760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:58.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.024+0000 7f0635614740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f062807d340 con 0x564ad44e7390 2024-09-21T06:00:58.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.025+0000 7f0635614740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f062807ee80 con 0x564ad4508070 2024-09-21T06:00:58.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.025+0000 7f0635614740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f062807da10 con 0x564ad44df1b0 2024-09-21T06:00:58.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.025+0000 7f062ce3c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad44df1b0 0x564ad48bb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:58.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.025+0000 7f062ce3c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad44df1b0 0x564ad48bb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:46010/0 (socket says 172.21.15.117:46010) 2024-09-21T06:00:58.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.025+0000 7f062ce3c640 1 -- 172.21.15.117:0/358990903 learned_addr learned my addr 172.21.15.117:0/358990903 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:58.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.026+0000 7f062d63d640 1 --2- 172.21.15.117:0/358990903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ad4508070 0x564ad48ac760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:58.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.026+0000 7f0627fff640 1 --2- 172.21.15.117:0/358990903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ad44e7390 0x564ad48b0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:58.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.028+0000 7f062ce3c640 1 -- 172.21.15.117:0/358990903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ad4508070 msgr2=0x564ad48ac760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:58.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.028+0000 7f062ce3c640 1 --2- 172.21.15.117:0/358990903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ad4508070 0x564ad48ac760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:58.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.028+0000 7f062ce3c640 1 -- 172.21.15.117:0/358990903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ad44e7390 msgr2=0x564ad48b0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:58.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.028+0000 7f062ce3c640 1 --2- 172.21.15.117:0/358990903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ad44e7390 0x564ad48b0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:58.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.028+0000 7f062ce3c640 1 -- 172.21.15.117:0/358990903 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ad48bb8e0 con 0x564ad44df1b0 2024-09-21T06:00:58.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.031+0000 7f062ce3c640 1 --2- 172.21.15.117:0/358990903 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad44df1b0 0x564ad48bb170 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x564ad48adf40 tx=0x7f0618009ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=363cd6194f25aace server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:58.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.033+0000 7f060e7fc640 1 -- 172.21.15.117:0/358990903 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061801e040 con 0x564ad44df1b0 2024-09-21T06:00:58.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.033+0000 7f060e7fc640 1 -- 172.21.15.117:0/358990903 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f061801f040 con 0x564ad44df1b0 2024-09-21T06:00:58.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.033+0000 7f060e7fc640 1 -- 172.21.15.117:0/358990903 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06180266a0 con 0x564ad44df1b0 2024-09-21T06:00:58.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.039+0000 7f0635614740 1 -- 172.21.15.117:0/358990903 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad44df1b0 msgr2=0x564ad48bb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:58.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.039+0000 7f0635614740 1 --2- 172.21.15.117:0/358990903 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad44df1b0 0x564ad48bb170 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x564ad48adf40 tx=0x7f0618009ef0 comp rx=0 tx=0).stop 2024-09-21T06:00:58.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.039+0000 7f0635614740 1 -- 172.21.15.117:0/358990903 shutdown_connections 2024-09-21T06:00:58.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.039+0000 7f0635614740 1 --2- 172.21.15.117:0/358990903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ad44e7390 0x564ad48b0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:58.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.039+0000 7f0635614740 1 --2- 172.21.15.117:0/358990903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ad4508070 0x564ad48ac760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:58.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.039+0000 7f0635614740 1 --2- 172.21.15.117:0/358990903 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad44df1b0 0x564ad48bb170 secure :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0x564ad48adf40 tx=0x7f0618009ef0 comp rx=0 tx=0).stop 2024-09-21T06:00:58.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.039+0000 7f0635614740 1 -- 172.21.15.117:0/358990903 >> 172.21.15.117:0/358990903 conn(0x564ad44fc460 msgr2=0x564ad48b02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:58.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.040+0000 7f0635614740 1 -- 172.21.15.117:0/358990903 shutdown_connections 2024-09-21T06:00:58.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.041+0000 7f0635614740 1 -- 172.21.15.117:0/358990903 wait complete. 2024-09-21T06:00:58.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.042+0000 7f0635614740 1 -- start start 2024-09-21T06:00:58.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.042+0000 7f0635614740 1 Processor -- start 2024-09-21T06:00:58.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.043+0000 7f0635614740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ad44e7390 0x564ad48c36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:58.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.043+0000 7f0635614740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad4508070 0x564ad48beb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:58.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.043+0000 7f0635614740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ad449bda0 0x564ad48bf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:58.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.043+0000 7f0635614740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f062807e7b0 con 0x564ad44e7390 2024-09-21T06:00:58.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.043+0000 7f0635614740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f062807e0e0 con 0x564ad449bda0 2024-09-21T06:00:58.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.043+0000 7f0635614740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f062807b3d0 con 0x564ad4508070 2024-09-21T06:00:58.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.043+0000 7f0635614740 10 client.0 _ll_register_callbacks cb 0x7f061c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:00:58.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.044+0000 7f062ce3c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ad44e7390 0x564ad48c36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:58.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.044+0000 7f062ce3c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ad44e7390 0x564ad48c36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51532/0 (socket says 172.21.15.117:51532) 2024-09-21T06:00:58.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.044+0000 7f062ce3c640 1 -- 172.21.15.117:0/2894649933 learned_addr learned my addr 172.21.15.117:0/2894649933 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:00:58.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.046+0000 7f062d63d640 1 --2- 172.21.15.117:0/2894649933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ad449bda0 0x564ad48bf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:58.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.046+0000 7f0627fff640 1 --2- 172.21.15.117:0/2894649933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad4508070 0x564ad48beb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:58.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.046+0000 7f062ce3c640 1 -- 172.21.15.117:0/2894649933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad4508070 msgr2=0x564ad48beb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:58.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.046+0000 7f062ce3c640 1 --2- 172.21.15.117:0/2894649933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad4508070 0x564ad48beb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:58.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.046+0000 7f062ce3c640 1 -- 172.21.15.117:0/2894649933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ad449bda0 msgr2=0x564ad48bf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:00:58.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.046+0000 7f062ce3c640 1 --2- 172.21.15.117:0/2894649933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ad449bda0 0x564ad48bf1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:58.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.046+0000 7f062ce3c640 1 -- 172.21.15.117:0/2894649933 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ad48bf930 con 0x564ad44e7390 2024-09-21T06:00:58.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.048+0000 7f062ce3c640 1 --2- 172.21.15.117:0/2894649933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ad44e7390 0x564ad48c36c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0618009bd0 tx=0x7f06180048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4590c1112edec47 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:00:58.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.048+0000 7f0607fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:00:58.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.050+0000 7f0607fff640 1 -- 172.21.15.117:0/2894649933 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0618039020 con 0x564ad44e7390 2024-09-21T06:00:58.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.050+0000 7f0635614740 1 -- 172.21.15.117:0/2894649933 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f061801f040 con 0x564ad44e7390 2024-09-21T06:00:58.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.050+0000 7f0607fff640 1 -- 172.21.15.117:0/2894649933 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f0618032040 con 0x564ad44e7390 2024-09-21T06:00:58.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.051+0000 7f0607fff640 1 -- 172.21.15.117:0/2894649933 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0618005130 con 0x564ad44e7390 2024-09-21T06:00:58.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.051+0000 7f0635614740 10 client.16179 fetch_fsmap learned FSMap version 27 2024-09-21T06:00:58.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.051+0000 7f0635614740 1 -- 172.21.15.117:0/2894649933 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564ad4959690 con 0x564ad44e7390 2024-09-21T06:00:58.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.052+0000 7f0607fff640 1 -- 172.21.15.117:0/2894649933 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f061801f040 con 0x564ad44e7390 2024-09-21T06:00:58.684 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.052+0000 7f0607fff640 1 -- 172.21.15.117:0/2894649933 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0618024040 con 0x564ad44e7390 2024-09-21T06:00:58.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.052+0000 7f0635614740 10 client.16179 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:00:58.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.052+0000 7f0635614740 10 client.16179 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:00:58.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.052+0000 7f0635614740 20 client.16179 populate_metadata read hostname 'smithi117' 2024-09-21T06:00:58.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.052+0000 7f0635614740 1 --2- 172.21.15.117:0/2894649933 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x564ad4959c30 0x564ad495c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:00:58.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.053+0000 7f0635614740 4 client.16179 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:00:58.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.053+0000 7f0635614740 1 -- 172.21.15.117:0/2894649933 --> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564ad48238a0 con 0x564ad4959c30 2024-09-21T06:00:58.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.058+0000 7f0627fff640 1 --2- 172.21.15.117:0/2894649933 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x564ad4959c30 0x564ad495c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:00:58.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.070+0000 7f0627fff640 1 --2- 172.21.15.117:0/2894649933 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x564ad4959c30 0x564ad495c050 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6c080e21d99132e3 server_cookie=81d9849657bb6618 in_seq=0 out_seq=0 2024-09-21T06:00:58.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.071+0000 7f0607fff640 10 client.16179 ms_handle_connect on v2:172.21.15.192:6838/1727623231 2024-09-21T06:00:58.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.072+0000 7f0607fff640 1 -- 172.21.15.117:0/2894649933 <== mds.1 v2:172.21.15.192:6838/1727623231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564ad48238a0 con 0x564ad4959c30 2024-09-21T06:00:58.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.072+0000 7f0607fff640 10 client.16179 handle_command_reply: tid=0 2024-09-21T06:00:58.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.072+0000 7f0635614740 1 client.16179 shutdown 2024-09-21T06:00:58.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.074+0000 7f0635614740 1 -- 172.21.15.117:0/2894649933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ad44e7390 msgr2=0x564ad48c36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:58.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.074+0000 7f0635614740 1 --2- 172.21.15.117:0/2894649933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ad44e7390 0x564ad48c36c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0618009bd0 tx=0x7f06180048d0 comp rx=0 tx=0).stop 2024-09-21T06:00:58.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.080+0000 7f0635614740 1 -- 172.21.15.117:0/2894649933 shutdown_connections 2024-09-21T06:00:58.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.080+0000 7f0635614740 1 --2- 172.21.15.117:0/2894649933 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x564ad4959c30 0x564ad495c050 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:58.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.080+0000 7f0635614740 1 --2- 172.21.15.117:0/2894649933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564ad449bda0 0x564ad48bf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:58.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.080+0000 7f0635614740 1 --2- 172.21.15.117:0/2894649933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564ad4508070 0x564ad48beb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:58.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.080+0000 7f0635614740 1 --2- 172.21.15.117:0/2894649933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564ad44e7390 0x564ad48c36c0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:58.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.080+0000 7f0635614740 1 -- 172.21.15.117:0/2894649933 >> 172.21.15.117:0/2894649933 conn(0x564ad44fc460 msgr2=0x564ad48b8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:58.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.080+0000 7f0607fff640 0 client.16179 ms_handle_reset on v2:172.21.15.192:6838/1727623231 2024-09-21T06:00:58.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.080+0000 7f0635614740 1 -- 172.21.15.117:0/2894649933 shutdown_connections 2024-09-21T06:00:58.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.081+0000 7f0635614740 1 -- 172.21.15.117:0/2894649933 wait complete. 2024-09-21T06:00:58.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.082+0000 7f0635614740 20 client.16179 trim_cache size 0 max 16384 2024-09-21T06:00:58.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.083+0000 7f062f0a0640 1 -- 172.21.15.117:0/719674501 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f06100b6850 msgr2=0x7f06100b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:58.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.083+0000 7f062f0a0640 1 --2- 172.21.15.117:0/719674501 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f06100b6850 0x7f06100b8d60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0620005c40 tx=0x7f0620004ab0 comp rx=0 tx=0).stop 2024-09-21T06:00:58.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.083+0000 7f062f0a0640 1 -- 172.21.15.117:0/719674501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0628111f40 msgr2=0x7f0628113710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:00:58.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.083+0000 7f062f0a0640 1 --2- 172.21.15.117:0/719674501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0628111f40 0x7f0628113710 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f061c009230 tx=0x7f061c003ce0 comp rx=0 tx=0).stop 2024-09-21T06:00:58.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.086+0000 7f062f0a0640 1 -- 172.21.15.117:0/719674501 shutdown_connections 2024-09-21T06:00:58.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.086+0000 7f062f0a0640 1 --2- 172.21.15.117:0/719674501 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f06100b6850 0x7f06100b8d60 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:58.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.086+0000 7f062f0a0640 1 --2- 172.21.15.117:0/719674501 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f062811cc30 0x7f0628113cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:58.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.086+0000 7f062f0a0640 1 --2- 172.21.15.117:0/719674501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0628111f40 0x7f0628113710 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:58.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.086+0000 7f062f0a0640 1 --2- 172.21.15.117:0/719674501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f062810a9e0 0x7f062811a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:00:58.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.086+0000 7f062f0a0640 1 -- 172.21.15.117:0/719674501 >> 172.21.15.117:0/719674501 conn(0x7f0628070ec0 msgr2=0x7f062811e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:00:58.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.090+0000 7f062f0a0640 1 -- 172.21.15.117:0/719674501 shutdown_connections 2024-09-21T06:00:58.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:00:58.090+0000 7f062f0a0640 1 -- 172.21.15.117:0/719674501 wait complete. 2024-09-21T06:01:00.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:00:59 smithi138 ceph-mon[34391]: pgmap v437: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 891 MiB/s wr, 1.35k op/s 2024-09-21T06:01:00.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:00:59 smithi192 ceph-mon[34700]: pgmap v437: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 891 MiB/s wr, 1.35k op/s 2024-09-21T06:01:00.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:00:59 smithi117 ceph-mon[30947]: pgmap v437: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 891 MiB/s wr, 1.35k op/s 2024-09-21T06:01:02.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:01 smithi138 ceph-mon[34391]: pgmap v438: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 867 MiB/s wr, 1.47k op/s 2024-09-21T06:01:02.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:01 smithi192 ceph-mon[34700]: pgmap v438: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 867 MiB/s wr, 1.47k op/s 2024-09-21T06:01:02.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:01 smithi117 ceph-mon[30947]: pgmap v438: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 867 MiB/s wr, 1.47k op/s 2024-09-21T06:01:03.191 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:01:03.740 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:01:04.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:03 smithi138 ceph-mon[34391]: pgmap v439: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 868 MiB/s wr, 1.70k op/s 2024-09-21T06:01:04.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:03 smithi192 ceph-mon[34700]: pgmap v439: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 868 MiB/s wr, 1.70k op/s 2024-09-21T06:01:04.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:03 smithi117 ceph-mon[30947]: pgmap v439: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 868 MiB/s wr, 1.70k op/s 2024-09-21T06:01:04.767 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.763+0000 7fdf36e29640 1 -- 172.21.15.117:0/368322899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf3011cba0 msgr2=0x7fdf3011efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:04.767 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.763+0000 7fdf36e29640 1 --2- 172.21.15.117:0/368322899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf3011cba0 0x7fdf3011efc0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c009900 tx=0x7fdf2c02a420 comp rx=0 tx=0).stop 2024-09-21T06:01:04.767 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.763+0000 7fdf36e29640 1 -- 172.21.15.117:0/368322899 shutdown_connections 2024-09-21T06:01:04.767 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.763+0000 7fdf36e29640 1 --2- 172.21.15.117:0/368322899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf3011cba0 0x7fdf3011efc0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:04.767 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.763+0000 7fdf36e29640 1 --2- 172.21.15.117:0/368322899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf3010a900 0x7fdf3010ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:04.767 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.763+0000 7fdf36e29640 1 --2- 172.21.15.117:0/368322899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf30111ca0 0x7fdf3010a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:04.767 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.763+0000 7fdf36e29640 1 -- 172.21.15.117:0/368322899 >> 172.21.15.117:0/368322899 conn(0x7fdf30070c00 msgr2=0x7fdf30071010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:04.770 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.765+0000 7fdf36e29640 1 -- 172.21.15.117:0/368322899 shutdown_connections 2024-09-21T06:01:04.770 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.766+0000 7fdf36e29640 1 -- 172.21.15.117:0/368322899 wait complete. 2024-09-21T06:01:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.767+0000 7fdf36e29640 1 Processor -- start 2024-09-21T06:01:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.767+0000 7fdf36e29640 1 -- start start 2024-09-21T06:01:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.767+0000 7fdf36e29640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf3010a900 0x7fdf30112680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.768+0000 7fdf36e29640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf30111ca0 0x7fdf30112c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.768+0000 7fdf36e29640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf3011cba0 0x7fdf30117eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.768+0000 7fdf36e29640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdf301226e0 con 0x7fdf3010a900 2024-09-21T06:01:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.768+0000 7fdf36e29640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdf30122560 con 0x7fdf3011cba0 2024-09-21T06:01:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.768+0000 7fdf36e29640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdf30122860 con 0x7fdf30111ca0 2024-09-21T06:01:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.769+0000 7fdf35e27640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf3010a900 0x7fdf30112680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.769+0000 7fdf35e27640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf3010a900 0x7fdf30112680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54014/0 (socket says 172.21.15.117:54014) 2024-09-21T06:01:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.769+0000 7fdf35e27640 1 -- 172.21.15.117:0/2656591366 learned_addr learned my addr 172.21.15.117:0/2656591366 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:01:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.769+0000 7fdf35e27640 1 -- 172.21.15.117:0/2656591366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf30111ca0 msgr2=0x7fdf30112c00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:01:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.769+0000 7fdf35e27640 1 --2- 172.21.15.117:0/2656591366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf30111ca0 0x7fdf30112c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.769+0000 7fdf35e27640 1 -- 172.21.15.117:0/2656591366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf3011cba0 msgr2=0x7fdf30117eb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:01:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.769+0000 7fdf35e27640 1 --2- 172.21.15.117:0/2656591366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf3011cba0 0x7fdf30117eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.770+0000 7fdf35e27640 1 -- 172.21.15.117:0/2656591366 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf30118590 con 0x7fdf3010a900 2024-09-21T06:01:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.770+0000 7fdf35e27640 1 --2- 172.21.15.117:0/2656591366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf3010a900 0x7fdf30112680 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fdf28030a60 tx=0x7fdf28030f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.772+0000 7fdf26ffd640 1 -- 172.21.15.117:0/2656591366 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2802edf0 con 0x7fdf3010a900 2024-09-21T06:01:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.772+0000 7fdf26ffd640 1 -- 172.21.15.117:0/2656591366 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf28005020 con 0x7fdf3010a900 2024-09-21T06:01:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.772+0000 7fdf26ffd640 1 -- 172.21.15.117:0/2656591366 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf280336c0 con 0x7fdf3010a900 2024-09-21T06:01:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.773+0000 7fdf36e29640 1 -- 172.21.15.117:0/2656591366 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf30118880 con 0x7fdf3010a900 2024-09-21T06:01:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.773+0000 7fdf36e29640 1 -- 172.21.15.117:0/2656591366 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf30119020 con 0x7fdf3010a900 2024-09-21T06:01:04.781 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.779+0000 7fdf26ffd640 1 -- 172.21.15.117:0/2656591366 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fdf280054a0 con 0x7fdf3010a900 2024-09-21T06:01:04.782 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.780+0000 7fdf3effe740 1 Processor -- start 2024-09-21T06:01:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.780+0000 7fdf26ffd640 1 --2- 172.21.15.117:0/2656591366 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fdf1c0b6880 0x7fdf1c0b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.781+0000 7fdf26ffd640 1 -- 172.21.15.117:0/2656591366 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fdf28109920 con 0x7fdf3010a900 2024-09-21T06:01:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.784+0000 7fdf3effe740 1 -- start start 2024-09-21T06:01:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.784+0000 7fdf3effe740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be640e21b0 0x55be64487ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.784+0000 7fdf3effe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be640b92f0 0x55be644880e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.784+0000 7fdf3effe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be640c14d0 0x55be64488620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.784+0000 7fdf3effe740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55be643fc6f0 con 0x55be640c14d0 2024-09-21T06:01:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.784+0000 7fdf3effe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55be643ebb20 con 0x55be640b92f0 2024-09-21T06:01:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.784+0000 7fdf3effe740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55be643ef930 con 0x55be640e21b0 2024-09-21T06:01:04.792 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.788+0000 7fdf36628640 1 --2- 172.21.15.117:0/2656591366 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fdf1c0b6880 0x7fdf1c0b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.789+0000 7fdf35626640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be640b92f0 0x55be644880e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.789+0000 7fdf35626640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be640b92f0 0x55be644880e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:42840/0 (socket says 172.21.15.117:42840) 2024-09-21T06:01:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.789+0000 7fdf36628640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be640c14d0 0x55be64488620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.789+0000 7fdf35626640 1 -- 172.21.15.117:0/2002131502 learned_addr learned my addr 172.21.15.117:0/2002131502 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:01:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.789+0000 7fdf35e27640 1 --2- 172.21.15.117:0/2002131502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be640e21b0 0x55be64487ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.790+0000 7fdf36628640 1 -- 172.21.15.117:0/2002131502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be640e21b0 msgr2=0x55be64487ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.790+0000 7fdf36628640 1 --2- 172.21.15.117:0/2002131502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be640e21b0 0x55be64487ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.790+0000 7fdf36628640 1 -- 172.21.15.117:0/2002131502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be640b92f0 msgr2=0x55be644880e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.790+0000 7fdf36628640 1 --2- 172.21.15.117:0/2002131502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be640b92f0 0x55be644880e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.790+0000 7fdf36628640 1 -- 172.21.15.117:0/2002131502 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be643b1530 con 0x55be640c14d0 2024-09-21T06:01:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.790+0000 7fdf35626640 1 --2- 172.21.15.117:0/2002131502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be640b92f0 0x55be644880e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:01:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.791+0000 7fdf36628640 1 --2- 172.21.15.117:0/2002131502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be640c14d0 0x55be64488620 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c02aa10 tx=0x7fdf2c031760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c1f0c28043360a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.791+0000 7fdf36628640 1 --2- 172.21.15.117:0/2656591366 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fdf1c0b6880 0x7fdf1c0b8d90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c005f70 tx=0x7fdf2c039000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.792+0000 7fdf1bfff640 1 -- 172.21.15.117:0/2002131502 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c031d20 con 0x55be640c14d0 2024-09-21T06:01:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.792+0000 7fdf1bfff640 1 -- 172.21.15.117:0/2002131502 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf2c04e070 con 0x55be640c14d0 2024-09-21T06:01:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.792+0000 7fdf1bfff640 1 -- 172.21.15.117:0/2002131502 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c03b6b0 con 0x55be640c14d0 2024-09-21T06:01:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.793+0000 7fdf3effe740 1 -- 172.21.15.117:0/2002131502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be640c14d0 msgr2=0x55be64488620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.793+0000 7fdf3effe740 1 --2- 172.21.15.117:0/2002131502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be640c14d0 0x55be64488620 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c02aa10 tx=0x7fdf2c031760 comp rx=0 tx=0).stop 2024-09-21T06:01:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.794+0000 7fdf3effe740 1 -- 172.21.15.117:0/2002131502 shutdown_connections 2024-09-21T06:01:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.794+0000 7fdf3effe740 1 --2- 172.21.15.117:0/2002131502 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be640c14d0 0x55be64488620 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.794+0000 7fdf3effe740 1 --2- 172.21.15.117:0/2002131502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be640b92f0 0x55be644880e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.794+0000 7fdf3effe740 1 --2- 172.21.15.117:0/2002131502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be640e21b0 0x55be64487ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.794+0000 7fdf3effe740 1 -- 172.21.15.117:0/2002131502 >> 172.21.15.117:0/2002131502 conn(0x55be640d6300 msgr2=0x55be64485a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.794+0000 7fdf3effe740 1 -- 172.21.15.117:0/2002131502 shutdown_connections 2024-09-21T06:01:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.794+0000 7fdf3effe740 1 -- 172.21.15.117:0/2002131502 wait complete. 2024-09-21T06:01:04.798 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.795+0000 7fdf3effe740 1 -- start start 2024-09-21T06:01:04.798 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.795+0000 7fdf3effe740 1 Processor -- start 2024-09-21T06:01:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.795+0000 7fdf3effe740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be640b92f0 0x55be644933e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.796+0000 7fdf3effe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be640c14d0 0x55be644939a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.796+0000 7fdf3effe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be640e21b0 0x55be6449f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.796+0000 7fdf3effe740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55be64323ab0 con 0x55be640e21b0 2024-09-21T06:01:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.796+0000 7fdf3effe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55be64391730 con 0x55be640c14d0 2024-09-21T06:01:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.796+0000 7fdf3effe740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55be644962c0 con 0x55be640b92f0 2024-09-21T06:01:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.796+0000 7fdf3effe740 10 client.0 _ll_register_callbacks cb 0x55be643e56b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:01:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.797+0000 7fdf36628640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be640e21b0 0x55be6449f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.797+0000 7fdf36628640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be640e21b0 0x55be6449f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54026/0 (socket says 172.21.15.117:54026) 2024-09-21T06:01:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.797+0000 7fdf36628640 1 -- 172.21.15.117:0/2275932236 learned_addr learned my addr 172.21.15.117:0/2275932236 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:01:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.797+0000 7fdf35626640 1 --2- 172.21.15.117:0/2275932236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be640c14d0 0x55be644939a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.797+0000 7fdf36628640 1 -- 172.21.15.117:0/2275932236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be640b92f0 msgr2=0x55be644933e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.797+0000 7fdf36628640 1 --2- 172.21.15.117:0/2275932236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be640b92f0 0x55be644933e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.798+0000 7fdf36628640 1 -- 172.21.15.117:0/2275932236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be640c14d0 msgr2=0x55be644939a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.798+0000 7fdf36628640 1 --2- 172.21.15.117:0/2275932236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be640c14d0 0x55be644939a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.798+0000 7fdf36628640 1 -- 172.21.15.117:0/2275932236 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be6414d790 con 0x55be640e21b0 2024-09-21T06:01:04.802 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.799+0000 7fdf36628640 1 --2- 172.21.15.117:0/2275932236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be640e21b0 0x55be6449f570 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0092a0 tx=0x7fdf2c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc4fe5be91e0e90 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:04.802 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.799+0000 7fdf197fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:01:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.800+0000 7fdf197fa640 1 -- 172.21.15.117:0/2275932236 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c056070 con 0x55be640e21b0 2024-09-21T06:01:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.800+0000 7fdf197fa640 1 -- 172.21.15.117:0/2275932236 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fdf2c03e670 con 0x55be640e21b0 2024-09-21T06:01:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.800+0000 7fdf3effe740 1 -- 172.21.15.117:0/2275932236 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf2c04e070 con 0x55be640e21b0 2024-09-21T06:01:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.801+0000 7fdf197fa640 1 -- 172.21.15.117:0/2275932236 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdf2c03e870 con 0x55be640e21b0 2024-09-21T06:01:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.801+0000 7fdf3effe740 10 client.16191 fetch_fsmap learned FSMap version 27 2024-09-21T06:01:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.801+0000 7fdf3effe740 1 -- 172.21.15.117:0/2275932236 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55be641c6620 con 0x55be640e21b0 2024-09-21T06:01:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.802+0000 7fdf197fa640 1 -- 172.21.15.117:0/2275932236 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf2c04e070 con 0x55be640e21b0 2024-09-21T06:01:04.805 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.802+0000 7fdf197fa640 1 -- 172.21.15.117:0/2275932236 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdf2c05bce0 con 0x55be640e21b0 2024-09-21T06:01:04.805 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.802+0000 7fdf3effe740 10 client.16191 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:01:04.805 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.802+0000 7fdf3effe740 10 client.16191 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:01:04.805 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.802+0000 7fdf3effe740 20 client.16191 populate_metadata read hostname 'smithi117' 2024-09-21T06:01:04.805 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.802+0000 7fdf3effe740 1 --2- 172.21.15.117:0/2275932236 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55be640a0d80 0x55be644a5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:04.806 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.802+0000 7fdf3effe740 4 client.16191 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:01:04.806 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.802+0000 7fdf3effe740 1 -- 172.21.15.117:0/2275932236 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be643fe640 con 0x55be640a0d80 2024-09-21T06:01:04.806 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.803+0000 7fdf35e27640 1 --2- 172.21.15.117:0/2275932236 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55be640a0d80 0x55be644a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:04.806 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.803+0000 7fdf35e27640 1 --2- 172.21.15.117:0/2275932236 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55be640a0d80 0x55be644a5220 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4dfc63dcb24d53d server_cookie=ec582b2e6f990b82 in_seq=0 out_seq=0 2024-09-21T06:01:04.806 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.803+0000 7fdf197fa640 10 client.16191 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:01:04.810 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.807+0000 7fdf197fa640 1 -- 172.21.15.117:0/2275932236 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55be643fe640 con 0x55be640a0d80 2024-09-21T06:01:04.810 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.808+0000 7fdf197fa640 10 client.16191 handle_command_reply: tid=0 2024-09-21T06:01:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.808+0000 7fdf3effe740 1 client.16191 shutdown 2024-09-21T06:01:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.809+0000 7fdf3effe740 1 -- 172.21.15.117:0/2275932236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be640e21b0 msgr2=0x55be6449f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.809+0000 7fdf3effe740 1 --2- 172.21.15.117:0/2275932236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be640e21b0 0x55be6449f570 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0092a0 tx=0x7fdf2c005fd0 comp rx=0 tx=0).stop 2024-09-21T06:01:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.809+0000 7fdf3effe740 1 -- 172.21.15.117:0/2275932236 shutdown_connections 2024-09-21T06:01:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.809+0000 7fdf3effe740 1 --2- 172.21.15.117:0/2275932236 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55be640a0d80 0x55be644a5220 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.809+0000 7fdf3effe740 1 --2- 172.21.15.117:0/2275932236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be640e21b0 0x55be6449f570 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.809+0000 7fdf3effe740 1 --2- 172.21.15.117:0/2275932236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be640c14d0 0x55be644939a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.809+0000 7fdf3effe740 1 --2- 172.21.15.117:0/2275932236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be640b92f0 0x55be644933e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.809+0000 7fdf3effe740 1 -- 172.21.15.117:0/2275932236 >> 172.21.15.117:0/2275932236 conn(0x55be640d6300 msgr2=0x55be64488bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.809+0000 7fdf197fa640 0 client.16191 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:01:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.810+0000 7fdf3effe740 1 -- 172.21.15.117:0/2275932236 shutdown_connections 2024-09-21T06:01:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.810+0000 7fdf3effe740 1 -- 172.21.15.117:0/2275932236 wait complete. 2024-09-21T06:01:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.810+0000 7fdf3effe740 20 client.16191 trim_cache size 0 max 16384 2024-09-21T06:01:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.840+0000 7fdf3effe740 1 Processor -- start 2024-09-21T06:01:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.843+0000 7fdf3effe740 1 -- start start 2024-09-21T06:01:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.843+0000 7fdf3effe740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be640b92f0 0x55be644892e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.844+0000 7fdf3effe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be640c14d0 0x55be6449f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.844+0000 7fdf35e27640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be640b92f0 0x55be644892e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:04.847 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.844+0000 7fdf3effe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be640e21b0 0x55be6449fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:04.847 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.844+0000 7fdf3effe740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdf301af9c0 con 0x55be640c14d0 2024-09-21T06:01:04.847 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.844+0000 7fdf3effe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdf301b1500 con 0x55be640e21b0 2024-09-21T06:01:04.847 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.844+0000 7fdf3effe740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdf301b0090 con 0x55be640b92f0 2024-09-21T06:01:04.847 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.844+0000 7fdf35e27640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be640b92f0 0x55be644892e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:49978/0 (socket says 172.21.15.117:49978) 2024-09-21T06:01:04.847 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.844+0000 7fdf35e27640 1 -- 172.21.15.117:0/2502213284 learned_addr learned my addr 172.21.15.117:0/2502213284 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:01:04.847 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.844+0000 7fdf35626640 1 --2- 172.21.15.117:0/2502213284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be640c14d0 0x55be6449f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.845+0000 7fdf36628640 1 --2- 172.21.15.117:0/2502213284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be640e21b0 0x55be6449fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.845+0000 7fdf36628640 1 -- 172.21.15.117:0/2502213284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be640b92f0 msgr2=0x55be644892e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.845+0000 7fdf36628640 1 --2- 172.21.15.117:0/2502213284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be640b92f0 0x55be644892e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.845+0000 7fdf36628640 1 -- 172.21.15.117:0/2502213284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be640c14d0 msgr2=0x55be6449f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.845+0000 7fdf36628640 1 --2- 172.21.15.117:0/2502213284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be640c14d0 0x55be6449f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.845+0000 7fdf36628640 1 -- 172.21.15.117:0/2502213284 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be64495d50 con 0x55be640e21b0 2024-09-21T06:01:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.845+0000 7fdf36628640 1 --2- 172.21.15.117:0/2502213284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be640e21b0 0x55be6449fa60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c004b50 tx=0x7fdf2c03e750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efb7fbdff11636c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:04.850 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.847+0000 7fdf1b7fe640 1 -- 172.21.15.117:0/2502213284 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c066020 con 0x55be640e21b0 2024-09-21T06:01:04.850 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.847+0000 7fdf1b7fe640 1 -- 172.21.15.117:0/2502213284 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf2c04e040 con 0x55be640e21b0 2024-09-21T06:01:04.850 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.848+0000 7fdf1b7fe640 1 -- 172.21.15.117:0/2502213284 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c056070 con 0x55be640e21b0 2024-09-21T06:01:04.852 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.849+0000 7fdf3effe740 1 -- 172.21.15.117:0/2502213284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be640e21b0 msgr2=0x55be6449fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:04.852 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.849+0000 7fdf3effe740 1 --2- 172.21.15.117:0/2502213284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be640e21b0 0x55be6449fa60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c004b50 tx=0x7fdf2c03e750 comp rx=0 tx=0).stop 2024-09-21T06:01:04.857 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.854+0000 7fdf3effe740 1 -- 172.21.15.117:0/2502213284 shutdown_connections 2024-09-21T06:01:04.857 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.854+0000 7fdf3effe740 1 --2- 172.21.15.117:0/2502213284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be640c14d0 0x55be6449f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.854+0000 7fdf3effe740 1 --2- 172.21.15.117:0/2502213284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be640e21b0 0x55be6449fa60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.855+0000 7fdf3effe740 1 --2- 172.21.15.117:0/2502213284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be640b92f0 0x55be644892e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.855+0000 7fdf3effe740 1 -- 172.21.15.117:0/2502213284 >> 172.21.15.117:0/2502213284 conn(0x55be640d6300 msgr2=0x55be64488bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:04.860 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.857+0000 7fdf3effe740 1 -- 172.21.15.117:0/2502213284 shutdown_connections 2024-09-21T06:01:04.860 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.857+0000 7fdf3effe740 1 -- 172.21.15.117:0/2502213284 wait complete. 2024-09-21T06:01:04.861 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.858+0000 7fdf3effe740 1 -- start start 2024-09-21T06:01:04.861 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.858+0000 7fdf3effe740 1 Processor -- start 2024-09-21T06:01:04.862 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.859+0000 7fdf3effe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be640b92f0 0x55be644a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:04.863 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.860+0000 7fdf3effe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be640c14d0 0x55be644a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:04.863 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.860+0000 7fdf3effe740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be640e21b0 0x55be6450d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:04.863 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.860+0000 7fdf3effe740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdf301b0e30 con 0x55be640b92f0 2024-09-21T06:01:04.863 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.860+0000 7fdf3effe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdf301b0760 con 0x55be640c14d0 2024-09-21T06:01:04.863 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.860+0000 7fdf3effe740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdf301ad9c0 con 0x55be640e21b0 2024-09-21T06:01:04.863 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.860+0000 7fdf35626640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be640c14d0 0x55be644a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:04.863 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.860+0000 7fdf35626640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be640c14d0 0x55be644a5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:42850/0 (socket says 172.21.15.117:42850) 2024-09-21T06:01:04.863 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.860+0000 7fdf35626640 1 -- 172.21.15.117:0/2413514466 learned_addr learned my addr 172.21.15.117:0/2413514466 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:01:04.864 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.860+0000 7fdf3effe740 10 client.0 _ll_register_callbacks cb 0x7fdf2c031660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:01:04.864 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.861+0000 7fdf35e27640 1 --2- 172.21.15.117:0/2413514466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be640b92f0 0x55be644a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:04.865 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.862+0000 7fdf35626640 1 -- 172.21.15.117:0/2413514466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be640e21b0 msgr2=0x55be6450d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:04.865 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.862+0000 7fdf35626640 1 --2- 172.21.15.117:0/2413514466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be640e21b0 0x55be6450d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.862+0000 7fdf35626640 1 -- 172.21.15.117:0/2413514466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be640b92f0 msgr2=0x55be644a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.862+0000 7fdf35626640 1 --2- 172.21.15.117:0/2413514466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be640b92f0 0x55be644a54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.862+0000 7fdf35626640 1 -- 172.21.15.117:0/2413514466 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be644a6220 con 0x55be640c14d0 2024-09-21T06:01:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.864+0000 7fdf35626640 1 --2- 172.21.15.117:0/2413514466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be640c14d0 0x55be644a5a00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdf2000bf40 tx=0x7fdf20012760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dae6582d25b836ae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.866+0000 7fdf197fa640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-21T06:01:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.866+0000 7fdf197fa640 1 -- 172.21.15.117:0/2413514466 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf20019030 con 0x55be640c14d0 2024-09-21T06:01:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.867+0000 7fdf197fa640 1 -- 172.21.15.117:0/2413514466 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fdf2000f040 con 0x55be640c14d0 2024-09-21T06:01:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.867+0000 7fdf3effe740 1 -- 172.21.15.117:0/2413514466 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf2c04e040 con 0x55be640c14d0 2024-09-21T06:01:04.870 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.867+0000 7fdf197fa640 1 -- 172.21.15.117:0/2413514466 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdf200059c0 con 0x55be640c14d0 2024-09-21T06:01:04.870 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.867+0000 7fdf3effe740 10 client.25562 fetch_fsmap learned FSMap version 27 2024-09-21T06:01:04.870 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.867+0000 7fdf3effe740 1 -- 172.21.15.117:0/2413514466 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55be6450eef0 con 0x55be640c14d0 2024-09-21T06:01:04.871 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.868+0000 7fdf197fa640 1 -- 172.21.15.117:0/2413514466 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf20015b10 con 0x55be640c14d0 2024-09-21T06:01:04.871 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.868+0000 7fdf197fa640 1 -- 172.21.15.117:0/2413514466 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdf2001e9c0 con 0x55be640c14d0 2024-09-21T06:01:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.869+0000 7fdf3effe740 10 client.25562 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:01:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.869+0000 7fdf3effe740 10 client.25562 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:01:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.869+0000 7fdf3effe740 20 client.25562 populate_metadata read hostname 'smithi117' 2024-09-21T06:01:04.873 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.869+0000 7fdf3effe740 1 --2- 172.21.15.117:0/2413514466 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55be640761c0 0x55be64511710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:04.874 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.869+0000 7fdf3effe740 4 client.25562 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:01:04.874 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.869+0000 7fdf3effe740 1 -- 172.21.15.117:0/2413514466 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fdf2c04e040 con 0x55be640761c0 2024-09-21T06:01:04.874 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.871+0000 7fdf35e27640 1 --2- 172.21.15.117:0/2413514466 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55be640761c0 0x55be64511710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.876+0000 7fdf35e27640 1 --2- 172.21.15.117:0/2413514466 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55be640761c0 0x55be64511710 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef07818aef795a80 server_cookie=de86bfec472cf49c in_seq=0 out_seq=0 2024-09-21T06:01:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:04.876+0000 7fdf197fa640 10 client.25562 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:01:06.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:05 smithi138 ceph-mon[34391]: pgmap v440: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 947 MiB/s wr, 2.25k op/s 2024-09-21T06:01:06.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:05 smithi192 ceph-mon[34700]: pgmap v440: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 947 MiB/s wr, 2.25k op/s 2024-09-21T06:01:06.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:05 smithi117 ceph-mon[30947]: pgmap v440: 97 pgs: 97 active+clean; 30 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 947 MiB/s wr, 2.25k op/s 2024-09-21T06:01:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:06.703+0000 7fdf197fa640 1 -- 172.21.15.117:0/2413514466 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fdf2c04e040 con 0x55be640761c0 2024-09-21T06:01:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:06.703+0000 7fdf197fa640 10 client.25562 handle_command_reply: tid=0 2024-09-21T06:01:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:06.703+0000 7fdf3effe740 1 client.25562 shutdown 2024-09-21T06:01:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:06.706+0000 7fdf3effe740 1 -- 172.21.15.117:0/2413514466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be640c14d0 msgr2=0x55be644a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:06.706+0000 7fdf3effe740 1 --2- 172.21.15.117:0/2413514466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be640c14d0 0x55be644a5a00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdf2000bf40 tx=0x7fdf20012760 comp rx=0 tx=0).stop 2024-09-21T06:01:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:06.710+0000 7fdf3effe740 1 -- 172.21.15.117:0/2413514466 shutdown_connections 2024-09-21T06:01:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:06.710+0000 7fdf3effe740 1 --2- 172.21.15.117:0/2413514466 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55be640761c0 0x55be64511710 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:06.710+0000 7fdf3effe740 1 --2- 172.21.15.117:0/2413514466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be640e21b0 0x55be6450d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:06.710+0000 7fdf3effe740 1 --2- 172.21.15.117:0/2413514466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be640c14d0 0x55be644a5a00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:06.710+0000 7fdf3effe740 1 --2- 172.21.15.117:0/2413514466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be640b92f0 0x55be644a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:06.710+0000 7fdf3effe740 1 -- 172.21.15.117:0/2413514466 >> 172.21.15.117:0/2413514466 conn(0x55be640d6300 msgr2=0x55be6448d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:06.710+0000 7fdf197fa640 0 client.25562 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:01:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:06.712+0000 7fdf3effe740 1 -- 172.21.15.117:0/2413514466 shutdown_connections 2024-09-21T06:01:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:06.712+0000 7fdf3effe740 1 -- 172.21.15.117:0/2413514466 wait complete. 2024-09-21T06:01:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:06.713+0000 7fdf3effe740 20 client.25562 trim_cache size 0 max 16384 2024-09-21T06:01:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T06:01:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T06:01:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T06:01:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T06:01:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:06.719+0000 7fdf36e29640 1 -- 172.21.15.117:0/2656591366 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fdf1c0b6880 msgr2=0x7fdf1c0b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:06.719+0000 7fdf36e29640 1 --2- 172.21.15.117:0/2656591366 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fdf1c0b6880 0x7fdf1c0b8d90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c005f70 tx=0x7fdf2c039000 comp rx=0 tx=0).stop 2024-09-21T06:01:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:06.719+0000 7fdf36e29640 1 -- 172.21.15.117:0/2656591366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf3010a900 msgr2=0x7fdf30112680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:06.719+0000 7fdf36e29640 1 --2- 172.21.15.117:0/2656591366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf3010a900 0x7fdf30112680 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fdf28030a60 tx=0x7fdf28030f20 comp rx=0 tx=0).stop 2024-09-21T06:01:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:06.719+0000 7fdf36e29640 1 -- 172.21.15.117:0/2656591366 shutdown_connections 2024-09-21T06:01:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:06.719+0000 7fdf36e29640 1 --2- 172.21.15.117:0/2656591366 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fdf1c0b6880 0x7fdf1c0b8d90 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:06.719+0000 7fdf36e29640 1 --2- 172.21.15.117:0/2656591366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf3011cba0 0x7fdf30117eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:06.719+0000 7fdf36e29640 1 --2- 172.21.15.117:0/2656591366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf30111ca0 0x7fdf30112c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:06.719+0000 7fdf36e29640 1 --2- 172.21.15.117:0/2656591366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf3010a900 0x7fdf30112680 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:06.719+0000 7fdf36e29640 1 -- 172.21.15.117:0/2656591366 >> 172.21.15.117:0/2656591366 conn(0x7fdf30070c00 msgr2=0x7fdf3011ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:06.722+0000 7fdf36e29640 1 -- 172.21.15.117:0/2656591366 shutdown_connections 2024-09-21T06:01:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:06.724+0000 7fdf36e29640 1 -- 172.21.15.117:0/2656591366 wait complete. 2024-09-21T06:01:08.002 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:01:08.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.879+0000 7f8db1fd2640 1 -- 172.21.15.117:0/1598110758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8dac1079f0 msgr2=0x7f8dac107ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:08.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.879+0000 7f8db1fd2640 1 --2- 172.21.15.117:0/1598110758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8dac1079f0 0x7f8dac107ed0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8da4009cd0 tx=0x7f8da402a930 comp rx=0 tx=0).stop 2024-09-21T06:01:08.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.880+0000 7f8db1fd2640 1 -- 172.21.15.117:0/1598110758 shutdown_connections 2024-09-21T06:01:08.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.880+0000 7f8db1fd2640 1 --2- 172.21.15.117:0/1598110758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8dac1079f0 0x7f8dac107ed0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:08.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.880+0000 7f8db1fd2640 1 --2- 172.21.15.117:0/1598110758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8dac102f00 0x7f8dac107430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:08.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.880+0000 7f8db1fd2640 1 --2- 172.21.15.117:0/1598110758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dac1025b0 0x7f8dac1029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:08.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.880+0000 7f8db1fd2640 1 -- 172.21.15.117:0/1598110758 >> 172.21.15.117:0/1598110758 conn(0x7f8dac0fe2c0 msgr2=0x7f8dac100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:08.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.881+0000 7f8db1fd2640 1 -- 172.21.15.117:0/1598110758 shutdown_connections 2024-09-21T06:01:08.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.881+0000 7f8db1fd2640 1 -- 172.21.15.117:0/1598110758 wait complete. 2024-09-21T06:01:08.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.882+0000 7f8db1fd2640 1 Processor -- start 2024-09-21T06:01:08.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.882+0000 7f8db1fd2640 1 -- start start 2024-09-21T06:01:08.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.883+0000 7f8db1fd2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8dac1025b0 0x7f8dac116b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:08.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.883+0000 7f8db1fd2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dac102f00 0x7f8dac10fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:08.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.883+0000 7f8db1fd2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8dac1079f0 0x7f8dac110420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:08.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.883+0000 7f8db1fd2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8dac119f90 con 0x7f8dac1025b0 2024-09-21T06:01:08.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.883+0000 7f8db1fd2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8dac119e10 con 0x7f8dac1079f0 2024-09-21T06:01:08.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.883+0000 7f8db1fd2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8dac11a110 con 0x7f8dac102f00 2024-09-21T06:01:08.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.885+0000 7f8db17d1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8dac1079f0 0x7f8dac110420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:08.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.885+0000 7f8dabfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dac102f00 0x7f8dac10fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:08.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.885+0000 7f8dabfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dac102f00 0x7f8dac10fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:49994/0 (socket says 172.21.15.117:49994) 2024-09-21T06:01:08.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.885+0000 7f8dabfff640 1 -- 172.21.15.117:0/2543183896 learned_addr learned my addr 172.21.15.117:0/2543183896 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:01:08.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.886+0000 7f8dabfff640 1 -- 172.21.15.117:0/2543183896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8dac1079f0 msgr2=0x7f8dac110420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:08.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.886+0000 7f8dabfff640 1 --2- 172.21.15.117:0/2543183896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8dac1079f0 0x7f8dac110420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:08.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.886+0000 7f8dabfff640 1 -- 172.21.15.117:0/2543183896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8dac1025b0 msgr2=0x7f8dac116b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:08.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.886+0000 7f8dabfff640 1 --2- 172.21.15.117:0/2543183896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8dac1025b0 0x7f8dac116b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:08.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.886+0000 7f8dabfff640 1 -- 172.21.15.117:0/2543183896 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dac110c40 con 0x7f8dac102f00 2024-09-21T06:01:08.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.887+0000 7f8dabfff640 1 --2- 172.21.15.117:0/2543183896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dac102f00 0x7f8dac10fe10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8da0006fd0 tx=0x7f8da000b8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:08.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.888+0000 7f8da9ffb640 1 -- 172.21.15.117:0/2543183896 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da0011830 con 0x7f8dac102f00 2024-09-21T06:01:08.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.889+0000 7f8da9ffb640 1 -- 172.21.15.117:0/2543183896 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8da0016070 con 0x7f8dac102f00 2024-09-21T06:01:08.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.889+0000 7f8db1fd2640 1 -- 172.21.15.117:0/2543183896 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dac1158d0 con 0x7f8dac102f00 2024-09-21T06:01:08.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.889+0000 7f8db1fd2640 1 -- 172.21.15.117:0/2543183896 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dac115e90 con 0x7f8dac102f00 2024-09-21T06:01:08.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.891+0000 7f8da9ffb640 1 -- 172.21.15.117:0/2543183896 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da0010e00 con 0x7f8dac102f00 2024-09-21T06:01:08.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.897+0000 7f8db97a8740 1 Processor -- start 2024-09-21T06:01:08.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.899+0000 7f8db97a8740 1 -- start start 2024-09-21T06:01:08.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.899+0000 7f8db97a8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f19e785070 0x55f19eb2c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:08.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.899+0000 7f8db97a8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f19e75c1b0 0x55f19eb2cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:08.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.899+0000 7f8db97a8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f19e764390 0x55f19eb2d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:08.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.899+0000 7f8db97a8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f19eaa2ea0 con 0x55f19e75c1b0 2024-09-21T06:01:08.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.899+0000 7f8db97a8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f19ea922d0 con 0x55f19e785070 2024-09-21T06:01:08.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.899+0000 7f8db97a8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f19ea960e0 con 0x55f19e764390 2024-09-21T06:01:08.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.900+0000 7f8db0fd0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f19e75c1b0 0x55f19eb2cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:08.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.900+0000 7f8db0fd0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f19e75c1b0 0x55f19eb2cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54076/0 (socket says 172.21.15.117:54076) 2024-09-21T06:01:08.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.900+0000 7f8db0fd0640 1 -- 172.21.15.117:0/822284889 learned_addr learned my addr 172.21.15.117:0/822284889 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:01:08.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.900+0000 7f8db17d1640 1 --2- 172.21.15.117:0/822284889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f19e785070 0x55f19eb2c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:08.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.902+0000 7f8da9ffb640 1 -- 172.21.15.117:0/2543183896 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f8da0010380 con 0x7f8dac102f00 2024-09-21T06:01:08.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.903+0000 7f8db0fd0640 1 -- 172.21.15.117:0/822284889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f19e764390 msgr2=0x55f19eb2d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:08.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.905+0000 7f8da9ffb640 1 --2- 172.21.15.117:0/2543183896 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8d9c0b66b0 0x7f8d9c0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:08.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.906+0000 7f8dabfff640 1 --2- 172.21.15.117:0/822284889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f19e764390 0x55f19eb2d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:08.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.906+0000 7f8db0fd0640 1 --2- 172.21.15.117:0/822284889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f19e764390 0x55f19eb2d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:08.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.906+0000 7f8db0fd0640 1 -- 172.21.15.117:0/822284889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f19e785070 msgr2=0x55f19eb2c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:08.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.906+0000 7f8db0fd0640 1 --2- 172.21.15.117:0/822284889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f19e785070 0x55f19eb2c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:08.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.906+0000 7f8db0fd0640 1 -- 172.21.15.117:0/822284889 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f19e7f0650 con 0x55f19e75c1b0 2024-09-21T06:01:08.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.906+0000 7f8db17d1640 1 --2- 172.21.15.117:0/822284889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f19e785070 0x55f19eb2c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:01:08.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.907+0000 7f8da9ffb640 1 -- 172.21.15.117:0/2543183896 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f8da00e6b10 con 0x7f8dac102f00 2024-09-21T06:01:08.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.908+0000 7f8db17d1640 1 --2- 172.21.15.117:0/2543183896 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8d9c0b66b0 0x7f8d9c0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:08.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.909+0000 7f8db0fd0640 1 --2- 172.21.15.117:0/822284889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f19e75c1b0 0x55f19eb2cda0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f8d980097c0 tx=0x7f8d98009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5991d4d3843964f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:08.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.910+0000 7f8d96ffd640 1 -- 172.21.15.117:0/822284889 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d98015020 con 0x55f19e75c1b0 2024-09-21T06:01:08.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.910+0000 7f8d96ffd640 1 -- 172.21.15.117:0/822284889 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d98011420 con 0x55f19e75c1b0 2024-09-21T06:01:08.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.910+0000 7f8d96ffd640 1 -- 172.21.15.117:0/822284889 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d980105c0 con 0x55f19e75c1b0 2024-09-21T06:01:08.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.911+0000 7f8db17d1640 1 --2- 172.21.15.117:0/2543183896 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8d9c0b66b0 0x7f8d9c0b8bc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8da4031ca0 tx=0x7f8da40095c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:08.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.914+0000 7f8db97a8740 1 -- 172.21.15.117:0/822284889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f19e75c1b0 msgr2=0x55f19eb2cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:08.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.914+0000 7f8db97a8740 1 --2- 172.21.15.117:0/822284889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f19e75c1b0 0x55f19eb2cda0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f8d980097c0 tx=0x7f8d98009c80 comp rx=0 tx=0).stop 2024-09-21T06:01:08.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.915+0000 7f8db97a8740 1 -- 172.21.15.117:0/822284889 shutdown_connections 2024-09-21T06:01:08.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.915+0000 7f8db97a8740 1 --2- 172.21.15.117:0/822284889 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f19e75c1b0 0x55f19eb2cda0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:08.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.915+0000 7f8db97a8740 1 --2- 172.21.15.117:0/822284889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f19e764390 0x55f19eb2d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:08.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.915+0000 7f8db97a8740 1 --2- 172.21.15.117:0/822284889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f19e785070 0x55f19eb2c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:08.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.915+0000 7f8db97a8740 1 -- 172.21.15.117:0/822284889 >> 172.21.15.117:0/822284889 conn(0x55f19e779460 msgr2=0x55f19eb2a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:08.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.918+0000 7f8db97a8740 1 -- 172.21.15.117:0/822284889 shutdown_connections 2024-09-21T06:01:08.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.918+0000 7f8db97a8740 1 -- 172.21.15.117:0/822284889 wait complete. 2024-09-21T06:01:08.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.919+0000 7f8db97a8740 1 -- start start 2024-09-21T06:01:08.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.919+0000 7f8db97a8740 1 Processor -- start 2024-09-21T06:01:08.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.921+0000 7f8db97a8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f19e75c1b0 0x55f19eb37f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:08.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.921+0000 7f8db97a8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f19e764390 0x55f19eb38490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:08.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.921+0000 7f8db97a8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f19e785070 0x55f19eb44360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:08.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.921+0000 7f8db97a8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f19e8f3700 con 0x55f19e764390 2024-09-21T06:01:08.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.921+0000 7f8db97a8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f19ea2d160 con 0x55f19e75c1b0 2024-09-21T06:01:08.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.921+0000 7f8db97a8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f19eb3b040 con 0x55f19e785070 2024-09-21T06:01:08.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.921+0000 7f8dabfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f19e764390 0x55f19eb38490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:08.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.921+0000 7f8dabfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f19e764390 0x55f19eb38490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54086/0 (socket says 172.21.15.117:54086) 2024-09-21T06:01:08.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.921+0000 7f8dabfff640 1 -- 172.21.15.117:0/2624517443 learned_addr learned my addr 172.21.15.117:0/2624517443 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:01:08.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.921+0000 7f8db97a8740 10 client.0 _ll_register_callbacks cb 0x55f19ea469e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:01:08.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.921+0000 7f8dabfff640 1 -- 172.21.15.117:0/2624517443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f19e785070 msgr2=0x55f19eb44360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:08.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.921+0000 7f8dabfff640 1 --2- 172.21.15.117:0/2624517443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f19e785070 0x55f19eb44360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:08.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.922+0000 7f8dabfff640 1 -- 172.21.15.117:0/2624517443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f19e75c1b0 msgr2=0x55f19eb37f50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:01:08.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.922+0000 7f8dabfff640 1 --2- 172.21.15.117:0/2624517443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f19e75c1b0 0x55f19eb37f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:08.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.922+0000 7f8dabfff640 1 -- 172.21.15.117:0/2624517443 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f19ea5ac90 con 0x55f19e764390 2024-09-21T06:01:08.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.922+0000 7f8dabfff640 1 --2- 172.21.15.117:0/2624517443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f19e764390 0x55f19eb38490 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8da000bde0 tx=0x7f8da00ac7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90106067861333bc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:08.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.923+0000 7f8d8bfff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:01:08.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.923+0000 7f8d8bfff640 1 -- 172.21.15.117:0/2624517443 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da00eb040 con 0x55f19e764390 2024-09-21T06:01:08.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.923+0000 7f8d8bfff640 1 -- 172.21.15.117:0/2624517443 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f8da00a8530 con 0x55f19e764390 2024-09-21T06:01:08.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.923+0000 7f8db97a8740 1 -- 172.21.15.117:0/2624517443 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d98011420 con 0x55f19e764390 2024-09-21T06:01:08.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.924+0000 7f8d8bfff640 1 -- 172.21.15.117:0/2624517443 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8da00ac150 con 0x55f19e764390 2024-09-21T06:01:08.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.924+0000 7f8db97a8740 10 client.16206 fetch_fsmap learned FSMap version 27 2024-09-21T06:01:08.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.924+0000 7f8db97a8740 1 -- 172.21.15.117:0/2624517443 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f19eb46db0 con 0x55f19e764390 2024-09-21T06:01:08.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.925+0000 7f8d8bfff640 1 -- 172.21.15.117:0/2624517443 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d98011420 con 0x55f19e764390 2024-09-21T06:01:08.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.925+0000 7f8d8bfff640 1 -- 172.21.15.117:0/2624517443 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8da00c2b90 con 0x55f19e764390 2024-09-21T06:01:08.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.925+0000 7f8db97a8740 10 client.16206 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:01:08.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.925+0000 7f8db97a8740 10 client.16206 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:01:08.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.925+0000 7f8db97a8740 20 client.16206 populate_metadata read hostname 'smithi117' 2024-09-21T06:01:08.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.925+0000 7f8db97a8740 1 --2- 172.21.15.117:0/2624517443 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f19e7439f0 0x55f19eb4a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:08.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.925+0000 7f8db97a8740 4 client.16206 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:01:08.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.925+0000 7f8db97a8740 1 -- 172.21.15.117:0/2624517443 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f19eaa4df0 con 0x55f19e7439f0 2024-09-21T06:01:08.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.929+0000 7f8db0fd0640 1 --2- 172.21.15.117:0/2624517443 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f19e7439f0 0x55f19eb4a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:08.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.939+0000 7f8d8bfff640 10 client.16206 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:01:08.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.939+0000 7f8db0fd0640 1 --2- 172.21.15.117:0/2624517443 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f19e7439f0 0x55f19eb4a300 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7d30879005a1de8 server_cookie=d6f5d6c9effce160 in_seq=0 out_seq=0 2024-09-21T06:01:08.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.943+0000 7f8d8bfff640 1 -- 172.21.15.117:0/2624517443 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f19eaa4df0 con 0x55f19e7439f0 2024-09-21T06:01:08.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.943+0000 7f8d8bfff640 10 client.16206 handle_command_reply: tid=0 2024-09-21T06:01:08.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.943+0000 7f8db97a8740 1 client.16206 shutdown 2024-09-21T06:01:08.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.943+0000 7f8db97a8740 1 -- 172.21.15.117:0/2624517443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f19e764390 msgr2=0x55f19eb38490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:08.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.943+0000 7f8db97a8740 1 --2- 172.21.15.117:0/2624517443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f19e764390 0x55f19eb38490 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8da000bde0 tx=0x7f8da00ac7b0 comp rx=0 tx=0).stop 2024-09-21T06:01:08.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.944+0000 7f8db97a8740 1 -- 172.21.15.117:0/2624517443 shutdown_connections 2024-09-21T06:01:08.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.944+0000 7f8db97a8740 1 --2- 172.21.15.117:0/2624517443 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f19e7439f0 0x55f19eb4a300 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:08.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.944+0000 7f8db97a8740 1 --2- 172.21.15.117:0/2624517443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f19e764390 0x55f19eb38490 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:08.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.944+0000 7f8db97a8740 1 --2- 172.21.15.117:0/2624517443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f19e785070 0x55f19eb44360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:08.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.944+0000 7f8d8bfff640 0 client.16206 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:01:08.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.944+0000 7f8db97a8740 1 --2- 172.21.15.117:0/2624517443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f19e75c1b0 0x55f19eb37f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:08.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.944+0000 7f8db97a8740 1 -- 172.21.15.117:0/2624517443 >> 172.21.15.117:0/2624517443 conn(0x55f19e779460 msgr2=0x55f19eb2f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:08.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.944+0000 7f8db97a8740 1 -- 172.21.15.117:0/2624517443 shutdown_connections 2024-09-21T06:01:08.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.944+0000 7f8db97a8740 1 -- 172.21.15.117:0/2624517443 wait complete. 2024-09-21T06:01:08.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.944+0000 7f8db97a8740 20 client.16206 trim_cache size 0 max 16384 2024-09-21T06:01:08.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.958+0000 7f8db97a8740 1 Processor -- start 2024-09-21T06:01:08.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.958+0000 7f8db97a8740 1 -- start start 2024-09-21T06:01:08.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.958+0000 7f8db97a8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f19e7439f0 0x55f19eb2a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:08.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.958+0000 7f8db97a8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f19e75c1b0 0x55f19eb44260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:08.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.958+0000 7f8db97a8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f19e764390 0x55f19eb35140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:08.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.958+0000 7f8db97a8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8dac1a7190 con 0x55f19e764390 2024-09-21T06:01:08.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.958+0000 7f8db97a8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8dac1a8cd0 con 0x55f19e75c1b0 2024-09-21T06:01:08.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.958+0000 7f8db97a8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8dac1a7860 con 0x55f19e7439f0 2024-09-21T06:01:08.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.958+0000 7f8db17d1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f19e764390 0x55f19eb35140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:08.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.958+0000 7f8db17d1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f19e764390 0x55f19eb35140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54088/0 (socket says 172.21.15.117:54088) 2024-09-21T06:01:08.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.958+0000 7f8db17d1640 1 -- 172.21.15.117:0/3368213286 learned_addr learned my addr 172.21.15.117:0/3368213286 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:01:08.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.959+0000 7f8db17d1640 1 -- 172.21.15.117:0/3368213286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f19e7439f0 msgr2=0x55f19eb2a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:08.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.959+0000 7f8db17d1640 1 --2- 172.21.15.117:0/3368213286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f19e7439f0 0x55f19eb2a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:08.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.959+0000 7f8db17d1640 1 -- 172.21.15.117:0/3368213286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f19e75c1b0 msgr2=0x55f19eb44260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:01:08.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.959+0000 7f8db17d1640 1 --2- 172.21.15.117:0/3368213286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f19e75c1b0 0x55f19eb44260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:08.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.959+0000 7f8db17d1640 1 -- 172.21.15.117:0/3368213286 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f19eb2b510 con 0x55f19e764390 2024-09-21T06:01:08.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.959+0000 7f8db17d1640 1 --2- 172.21.15.117:0/3368213286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f19e764390 0x55f19eb35140 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8da4035000 tx=0x7f8da4040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31b2ab244b39dc54 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:08.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.959+0000 7f8d967fc640 1 -- 172.21.15.117:0/3368213286 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da40307f0 con 0x55f19e764390 2024-09-21T06:01:08.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.959+0000 7f8d967fc640 1 -- 172.21.15.117:0/3368213286 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8da404d030 con 0x55f19e764390 2024-09-21T06:01:08.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.959+0000 7f8d967fc640 1 -- 172.21.15.117:0/3368213286 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da4004240 con 0x55f19e764390 2024-09-21T06:01:08.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.961+0000 7f8db97a8740 1 -- 172.21.15.117:0/3368213286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f19e764390 msgr2=0x55f19eb35140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:08.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.961+0000 7f8db97a8740 1 --2- 172.21.15.117:0/3368213286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f19e764390 0x55f19eb35140 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8da4035000 tx=0x7f8da4040040 comp rx=0 tx=0).stop 2024-09-21T06:01:08.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.961+0000 7f8db97a8740 1 -- 172.21.15.117:0/3368213286 shutdown_connections 2024-09-21T06:01:08.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.961+0000 7f8db97a8740 1 --2- 172.21.15.117:0/3368213286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f19e764390 0x55f19eb35140 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:08.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.961+0000 7f8db97a8740 1 --2- 172.21.15.117:0/3368213286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f19e75c1b0 0x55f19eb44260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:08.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.961+0000 7f8db97a8740 1 --2- 172.21.15.117:0/3368213286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f19e7439f0 0x55f19eb2a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:08.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.961+0000 7f8db97a8740 1 -- 172.21.15.117:0/3368213286 >> 172.21.15.117:0/3368213286 conn(0x55f19e779460 msgr2=0x55f19eb2a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:08.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.962+0000 7f8db97a8740 1 -- 172.21.15.117:0/3368213286 shutdown_connections 2024-09-21T06:01:08.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.962+0000 7f8db97a8740 1 -- 172.21.15.117:0/3368213286 wait complete. 2024-09-21T06:01:08.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.962+0000 7f8db97a8740 1 -- start start 2024-09-21T06:01:08.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.963+0000 7f8db97a8740 1 Processor -- start 2024-09-21T06:01:08.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.963+0000 7f8db97a8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f19e7439f0 0x55f19eb3c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:08.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.963+0000 7f8db97a8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f19e75c1b0 0x55f19eb3c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:08.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.963+0000 7f8db97a8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f19e764390 0x55f19eb3cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:08.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.964+0000 7f8db0fd0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f19e7439f0 0x55f19eb3c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:08.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.964+0000 7f8db0fd0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f19e7439f0 0x55f19eb3c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:42900/0 (socket says 172.21.15.117:42900) 2024-09-21T06:01:08.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.964+0000 7f8dabfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f19e75c1b0 0x55f19eb3c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:08.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.964+0000 7f8dabfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f19e75c1b0 0x55f19eb3c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:50022/0 (socket says 172.21.15.117:50022) 2024-09-21T06:01:08.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.964+0000 7f8db17d1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f19e764390 0x55f19eb3cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:08.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.964+0000 7f8db17d1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f19e764390 0x55f19eb3cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54104/0 (socket says 172.21.15.117:54104) 2024-09-21T06:01:08.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.964+0000 7f8db0fd0640 1 -- 172.21.15.117:0/3508061391 learned_addr learned my addr 172.21.15.117:0/3508061391 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:01:08.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.964+0000 7f8db97a8740 1 -- 172.21.15.117:0/3508061391 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8dac1a8600 con 0x55f19e764390 2024-09-21T06:01:08.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.964+0000 7f8db97a8740 1 -- 172.21.15.117:0/3508061391 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8dac1a7f30 con 0x55f19e7439f0 2024-09-21T06:01:08.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.964+0000 7f8db97a8740 1 -- 172.21.15.117:0/3508061391 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8dac06b560 con 0x55f19e75c1b0 2024-09-21T06:01:08.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.964+0000 7f8db97a8740 10 client.0 _ll_register_callbacks cb 0x7f8da0010fa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:01:08.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.965+0000 7f8db17d1640 1 -- 172.21.15.117:0/3508061391 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f19e75c1b0 msgr2=0x55f19eb3c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:08.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.965+0000 7f8db17d1640 1 --2- 172.21.15.117:0/3508061391 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f19e75c1b0 0x55f19eb3c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:08.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.965+0000 7f8db17d1640 1 -- 172.21.15.117:0/3508061391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f19e7439f0 msgr2=0x55f19eb3c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:08.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.965+0000 7f8db17d1640 1 --2- 172.21.15.117:0/3508061391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f19e7439f0 0x55f19eb3c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:08.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.965+0000 7f8db17d1640 1 -- 172.21.15.117:0/3508061391 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f19eb32bb0 con 0x55f19e764390 2024-09-21T06:01:08.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.965+0000 7f8db17d1640 1 --2- 172.21.15.117:0/3508061391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f19e764390 0x55f19eb3cd70 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f8da4006010 tx=0x7f8da40098c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a929938d2c8e2cc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:08.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.965+0000 7f8d8bfff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:01:08.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.965+0000 7f8d8bfff640 1 -- 172.21.15.117:0/3508061391 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da4031e80 con 0x55f19e764390 2024-09-21T06:01:08.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.965+0000 7f8d8bfff640 1 -- 172.21.15.117:0/3508061391 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f8da4003b80 con 0x55f19e764390 2024-09-21T06:01:08.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.965+0000 7f8db97a8740 1 -- 172.21.15.117:0/3508061391 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8da404d030 con 0x55f19e764390 2024-09-21T06:01:08.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.965+0000 7f8d8bfff640 1 -- 172.21.15.117:0/3508061391 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8da4030b10 con 0x55f19e764390 2024-09-21T06:01:08.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.965+0000 7f8db97a8740 10 client.16212 fetch_fsmap learned FSMap version 27 2024-09-21T06:01:08.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.965+0000 7f8db97a8740 1 -- 172.21.15.117:0/3508061391 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f19ebb4d30 con 0x55f19e764390 2024-09-21T06:01:08.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.966+0000 7f8d8bfff640 1 -- 172.21.15.117:0/3508061391 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8da404d030 con 0x55f19e764390 2024-09-21T06:01:08.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.966+0000 7f8d8bfff640 1 -- 172.21.15.117:0/3508061391 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f8da4066610 con 0x55f19e764390 2024-09-21T06:01:08.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.966+0000 7f8db97a8740 10 client.16212 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:01:08.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.966+0000 7f8db97a8740 10 client.16212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:01:08.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.966+0000 7f8db97a8740 20 client.16212 populate_metadata read hostname 'smithi117' 2024-09-21T06:01:08.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.966+0000 7f8db97a8740 1 --2- 172.21.15.117:0/3508061391 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f19e785070 0x55f19ebb7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:08.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.966+0000 7f8db97a8740 4 client.16212 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:01:08.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.966+0000 7f8db97a8740 1 -- 172.21.15.117:0/3508061391 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f19eb38840 con 0x55f19e785070 2024-09-21T06:01:08.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.967+0000 7f8db0fd0640 1 --2- 172.21.15.117:0/3508061391 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f19e785070 0x55f19ebb7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:08.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.977+0000 7f8db0fd0640 1 --2- 172.21.15.117:0/3508061391 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f19e785070 0x55f19ebb7440 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd3efd172fbd6e5 server_cookie=f31c004c1a42c579 in_seq=0 out_seq=0 2024-09-21T06:01:08.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.977+0000 7f8d8bfff640 10 client.16212 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:01:08.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.982+0000 7f8d8bfff640 1 -- 172.21.15.117:0/3508061391 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f19eb38840 con 0x55f19e785070 2024-09-21T06:01:08.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.982+0000 7f8d8bfff640 10 client.16212 handle_command_reply: tid=0 2024-09-21T06:01:08.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.982+0000 7f8db97a8740 1 client.16212 shutdown 2024-09-21T06:01:08.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.982+0000 7f8db97a8740 1 -- 172.21.15.117:0/3508061391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f19e764390 msgr2=0x55f19eb3cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:08.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.982+0000 7f8db97a8740 1 --2- 172.21.15.117:0/3508061391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f19e764390 0x55f19eb3cd70 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f8da4006010 tx=0x7f8da40098c0 comp rx=0 tx=0).stop 2024-09-21T06:01:08.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.982+0000 7f8db97a8740 1 -- 172.21.15.117:0/3508061391 shutdown_connections 2024-09-21T06:01:08.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.982+0000 7f8db97a8740 1 --2- 172.21.15.117:0/3508061391 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f19e785070 0x55f19ebb7440 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:08.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.982+0000 7f8db97a8740 1 --2- 172.21.15.117:0/3508061391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f19e764390 0x55f19eb3cd70 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:08.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.982+0000 7f8d8bfff640 0 client.16212 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:01:08.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.982+0000 7f8db97a8740 1 --2- 172.21.15.117:0/3508061391 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f19e75c1b0 0x55f19eb3c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:08.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.982+0000 7f8db97a8740 1 --2- 172.21.15.117:0/3508061391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f19e7439f0 0x55f19eb3c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:08.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.982+0000 7f8db97a8740 1 -- 172.21.15.117:0/3508061391 >> 172.21.15.117:0/3508061391 conn(0x55f19e779460 msgr2=0x55f19eb39400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:08.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.983+0000 7f8db97a8740 1 -- 172.21.15.117:0/3508061391 shutdown_connections 2024-09-21T06:01:08.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.983+0000 7f8db97a8740 1 -- 172.21.15.117:0/3508061391 wait complete. 2024-09-21T06:01:08.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.983+0000 7f8db97a8740 20 client.16212 trim_cache size 0 max 16384 2024-09-21T06:01:08.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.984+0000 7f8db1fd2640 1 -- 172.21.15.117:0/2543183896 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8d9c0b66b0 msgr2=0x7f8d9c0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:08.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.984+0000 7f8db1fd2640 1 --2- 172.21.15.117:0/2543183896 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8d9c0b66b0 0x7f8d9c0b8bc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8da4031ca0 tx=0x7f8da40095c0 comp rx=0 tx=0).stop 2024-09-21T06:01:08.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.984+0000 7f8db1fd2640 1 -- 172.21.15.117:0/2543183896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dac102f00 msgr2=0x7f8dac10fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:08.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.984+0000 7f8db1fd2640 1 --2- 172.21.15.117:0/2543183896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dac102f00 0x7f8dac10fe10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8da0006fd0 tx=0x7f8da000b8e0 comp rx=0 tx=0).stop 2024-09-21T06:01:08.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.984+0000 7f8db1fd2640 1 -- 172.21.15.117:0/2543183896 shutdown_connections 2024-09-21T06:01:08.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.984+0000 7f8db1fd2640 1 --2- 172.21.15.117:0/2543183896 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8d9c0b66b0 0x7f8d9c0b8bc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:08.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.984+0000 7f8db1fd2640 1 --2- 172.21.15.117:0/2543183896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8dac1079f0 0x7f8dac110420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:08.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.984+0000 7f8db1fd2640 1 --2- 172.21.15.117:0/2543183896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dac102f00 0x7f8dac10fe10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:08.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.984+0000 7f8db1fd2640 1 --2- 172.21.15.117:0/2543183896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8dac1025b0 0x7f8dac116b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:08.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.984+0000 7f8db1fd2640 1 -- 172.21.15.117:0/2543183896 >> 172.21.15.117:0/2543183896 conn(0x7f8dac0fe2c0 msgr2=0x7f8dac105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:08.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.984+0000 7f8db1fd2640 1 -- 172.21.15.117:0/2543183896 shutdown_connections 2024-09-21T06:01:08.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:06.984+0000 7f8db1fd2640 1 -- 172.21.15.117:0/2543183896 wait complete. 2024-09-21T06:01:08.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:07 smithi138 ceph-mon[34391]: pgmap v441: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 805 MiB/s wr, 2.25k op/s 2024-09-21T06:01:08.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:07 smithi192 ceph-mon[34700]: pgmap v441: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 805 MiB/s wr, 2.25k op/s 2024-09-21T06:01:08.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:07 smithi117 ceph-mon[30947]: pgmap v441: 97 pgs: 97 active+clean; 30 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 805 MiB/s wr, 2.25k op/s 2024-09-21T06:01:10.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:09 smithi138 ceph-mon[34391]: pgmap v442: 97 pgs: 97 active+clean; 31 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 905 MiB/s wr, 2.90k op/s 2024-09-21T06:01:10.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:09 smithi192 ceph-mon[34700]: pgmap v442: 97 pgs: 97 active+clean; 31 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 905 MiB/s wr, 2.90k op/s 2024-09-21T06:01:10.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:09 smithi117 ceph-mon[30947]: pgmap v442: 97 pgs: 97 active+clean; 31 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 905 MiB/s wr, 2.90k op/s 2024-09-21T06:01:12.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:11 smithi138 ceph-mon[34391]: pgmap v443: 97 pgs: 97 active+clean; 31 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 824 MiB/s wr, 2.94k op/s 2024-09-21T06:01:12.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:11 smithi192 ceph-mon[34700]: pgmap v443: 97 pgs: 97 active+clean; 31 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 824 MiB/s wr, 2.94k op/s 2024-09-21T06:01:12.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:11 smithi117 ceph-mon[30947]: pgmap v443: 97 pgs: 97 active+clean; 31 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 824 MiB/s wr, 2.94k op/s 2024-09-21T06:01:13.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:12 smithi192 ceph-mon[34700]: pgmap v444: 97 pgs: 97 active+clean; 32 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 814 MiB/s wr, 3.12k op/s 2024-09-21T06:01:13.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:12 smithi117 ceph-mon[30947]: pgmap v444: 97 pgs: 97 active+clean; 32 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 814 MiB/s wr, 3.12k op/s 2024-09-21T06:01:13.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:12 smithi138 ceph-mon[34391]: pgmap v444: 97 pgs: 97 active+clean; 32 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 814 MiB/s wr, 3.12k op/s 2024-09-21T06:01:16.029 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:15 smithi117 ceph-mon[30947]: pgmap v445: 97 pgs: 97 active+clean; 34 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 877 MiB/s wr, 3.53k op/s 2024-09-21T06:01:16.066 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:15 smithi138 ceph-mon[34391]: pgmap v445: 97 pgs: 97 active+clean; 34 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 877 MiB/s wr, 3.53k op/s 2024-09-21T06:01:16.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:15 smithi192 ceph-mon[34700]: pgmap v445: 97 pgs: 97 active+clean; 34 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 877 MiB/s wr, 3.53k op/s 2024-09-21T06:01:17.459 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:01:17.460 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:01:18.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:17 smithi117 ceph-mon[30947]: pgmap v446: 97 pgs: 97 active+clean; 35 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 775 MiB/s wr, 3.44k op/s 2024-09-21T06:01:18.031 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:17 smithi192 ceph-mon[34700]: pgmap v446: 97 pgs: 97 active+clean; 35 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 775 MiB/s wr, 3.44k op/s 2024-09-21T06:01:18.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:17 smithi138 ceph-mon[34391]: pgmap v446: 97 pgs: 97 active+clean; 35 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 775 MiB/s wr, 3.44k op/s 2024-09-21T06:01:18.249 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:01:19.972 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:19 smithi117 ceph-mon[30947]: pgmap v447: 97 pgs: 97 active+clean; 36 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 836 MiB/s wr, 3.82k op/s 2024-09-21T06:01:20.059 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.057+0000 7f7fed24d640 1 -- 172.21.15.117:0/1447435325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fe810ab50 msgr2=0x7f7fe8116910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.057+0000 7f7fed24d640 1 --2- 172.21.15.117:0/1447435325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fe810ab50 0x7f7fe8116910 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0009ca0 tx=0x7f7fe002a8f0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.068 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.066+0000 7f7fed24d640 1 -- 172.21.15.117:0/1447435325 shutdown_connections 2024-09-21T06:01:20.068 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.066+0000 7f7fed24d640 1 --2- 172.21.15.117:0/1447435325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fe810ab50 0x7f7fe8116910 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.068 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.066+0000 7f7fed24d640 1 --2- 172.21.15.117:0/1447435325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fe810a200 0x7f7fe810a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.068 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.066+0000 7f7fed24d640 1 --2- 172.21.15.117:0/1447435325 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fe810eab0 0x7f7fe810eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.068 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.066+0000 7f7fed24d640 1 -- 172.21.15.117:0/1447435325 >> 172.21.15.117:0/1447435325 conn(0x7f7fe8079400 msgr2=0x7f7fe807b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.077+0000 7f7fed24d640 1 -- 172.21.15.117:0/1447435325 shutdown_connections 2024-09-21T06:01:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.078+0000 7f7fed24d640 1 -- 172.21.15.117:0/1447435325 wait complete. 2024-09-21T06:01:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.079+0000 7f7fed24d640 1 Processor -- start 2024-09-21T06:01:20.081 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.080+0000 7f7fed24d640 1 -- start start 2024-09-21T06:01:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.080+0000 7f7fed24d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fe810a200 0x7f7fe81a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.081+0000 7f7fed24d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fe810ab50 0x7f7fe81a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.081+0000 7f7fed24d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fe810eab0 0x7f7fe81a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.081+0000 7f7fed24d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe8119f30 con 0x7f7fe810ab50 2024-09-21T06:01:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.081+0000 7f7fed24d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe8119db0 con 0x7f7fe810a200 2024-09-21T06:01:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.081+0000 7f7fe6d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fe810a200 0x7f7fe81a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.081+0000 7f7fe6d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fe810a200 0x7f7fe81a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:38418/0 (socket says 172.21.15.117:38418) 2024-09-21T06:01:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.081+0000 7f7fe6d9d640 1 -- 172.21.15.117:0/1174768940 learned_addr learned my addr 172.21.15.117:0/1174768940 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:01:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.082+0000 7f7fe759e640 1 --2- 172.21.15.117:0/1174768940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fe810eab0 0x7f7fe81a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.081+0000 7f7fed24d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe811a0b0 con 0x7f7fe810eab0 2024-09-21T06:01:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.084+0000 7f7fe659c640 1 --2- 172.21.15.117:0/1174768940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fe810ab50 0x7f7fe81a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.085+0000 7f7fe759e640 1 -- 172.21.15.117:0/1174768940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fe810a200 msgr2=0x7f7fe81a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.085+0000 7f7fe759e640 1 --2- 172.21.15.117:0/1174768940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fe810a200 0x7f7fe81a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.085+0000 7f7fe759e640 1 -- 172.21.15.117:0/1174768940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fe810ab50 msgr2=0x7f7fe81a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.085+0000 7f7fe759e640 1 --2- 172.21.15.117:0/1174768940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fe810ab50 0x7f7fe81a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.085+0000 7f7fe759e640 1 -- 172.21.15.117:0/1174768940 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe81a66a0 con 0x7f7fe810eab0 2024-09-21T06:01:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.086+0000 7f7fe6d9d640 1 --2- 172.21.15.117:0/1174768940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fe810a200 0x7f7fe81a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:01:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.088+0000 7f7fe759e640 1 --2- 172.21.15.117:0/1174768940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fe810eab0 0x7f7fe81a5fa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7fe002ae10 tx=0x7f7fe0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.089+0000 7f7fcffff640 1 -- 172.21.15.117:0/1174768940 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe00307f0 con 0x7f7fe810eab0 2024-09-21T06:01:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.089+0000 7f7fed24d640 1 -- 172.21.15.117:0/1174768940 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fe81a6930 con 0x7f7fe810eab0 2024-09-21T06:01:20.091 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.089+0000 7f7fed24d640 1 -- 172.21.15.117:0/1174768940 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fe81ad120 con 0x7f7fe810eab0 2024-09-21T06:01:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.091+0000 7f7fcffff640 1 -- 172.21.15.117:0/1174768940 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fe002d070 con 0x7f7fe810eab0 2024-09-21T06:01:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.091+0000 7f7fcffff640 1 -- 172.21.15.117:0/1174768940 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe003b410 con 0x7f7fe810eab0 2024-09-21T06:01:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.098+0000 7f7ff41bb740 1 Processor -- start 2024-09-21T06:01:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.098+0000 7f7ff41bb740 1 -- start start 2024-09-21T06:01:20.101 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.099+0000 7f7ff41bb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56226f3081b0 0x56226f6adba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.101 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.099+0000 7f7ff41bb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56226f2df2f0 0x56226f6ae0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.101 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.099+0000 7f7fcffff640 1 -- 172.21.15.117:0/1174768940 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f7fe003b5b0 con 0x7f7fe810eab0 2024-09-21T06:01:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.099+0000 7f7ff41bb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56226f2e74d0 0x56226f6ae620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.099+0000 7f7ff41bb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56226f6226f0 con 0x56226f3081b0 2024-09-21T06:01:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.099+0000 7f7ff41bb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56226f611b20 con 0x56226f2df2f0 2024-09-21T06:01:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.099+0000 7f7ff41bb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56226f615930 con 0x56226f2e74d0 2024-09-21T06:01:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.100+0000 7f7fe659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56226f3081b0 0x56226f6adba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.100+0000 7f7fe659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56226f3081b0 0x56226f6adba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59582/0 (socket says 172.21.15.117:59582) 2024-09-21T06:01:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.100+0000 7f7fe659c640 1 -- 172.21.15.117:0/2907042982 learned_addr learned my addr 172.21.15.117:0/2907042982 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:01:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.100+0000 7f7fe6d9d640 1 --2- 172.21.15.117:0/2907042982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56226f2df2f0 0x56226f6ae0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.100+0000 7f7fcffff640 1 --2- 172.21.15.117:0/1174768940 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7fd00b6850 0x7f7fd00b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.101+0000 7f7fcffff640 1 -- 172.21.15.117:0/1174768940 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f7fe0109b70 con 0x7f7fe810eab0 2024-09-21T06:01:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.102+0000 7f7fe659c640 1 --2- 172.21.15.117:0/2907042982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56226f2e74d0 0x56226f6ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.103+0000 7f7fe659c640 1 -- 172.21.15.117:0/2907042982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56226f2e74d0 msgr2=0x56226f6ae620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.103+0000 7f7fe659c640 1 --2- 172.21.15.117:0/2907042982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56226f2e74d0 0x56226f6ae620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.103+0000 7f7fe659c640 1 -- 172.21.15.117:0/2907042982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56226f2df2f0 msgr2=0x56226f6ae0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.103+0000 7f7fe659c640 1 --2- 172.21.15.117:0/2907042982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56226f2df2f0 0x56226f6ae0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.103+0000 7f7fe659c640 1 -- 172.21.15.117:0/2907042982 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56226f5d7530 con 0x56226f3081b0 2024-09-21T06:01:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.103+0000 7f7fe759e640 1 --2- 172.21.15.117:0/1174768940 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7fd00b6850 0x7f7fd00b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.104+0000 7f7fe659c640 1 --2- 172.21.15.117:0/2907042982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56226f3081b0 0x56226f6adba0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc017700 tx=0x7f7fdc0094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e9e2e7fabc1c6b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.108+0000 7f7fe759e640 1 --2- 172.21.15.117:0/1174768940 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7fd00b6850 0x7f7fd00b8d60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0003fd0 tx=0x7f7fe0005fd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.111+0000 7f7fcd7fa640 1 -- 172.21.15.117:0/2907042982 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc015760 con 0x56226f3081b0 2024-09-21T06:01:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.111+0000 7f7fcd7fa640 1 -- 172.21.15.117:0/2907042982 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fdc01d070 con 0x56226f3081b0 2024-09-21T06:01:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.111+0000 7f7fcd7fa640 1 -- 172.21.15.117:0/2907042982 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc018910 con 0x56226f3081b0 2024-09-21T06:01:20.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:19 smithi138 ceph-mon[34391]: pgmap v447: 97 pgs: 97 active+clean; 36 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 836 MiB/s wr, 3.82k op/s 2024-09-21T06:01:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.115+0000 7f7ff41bb740 1 -- 172.21.15.117:0/2907042982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56226f3081b0 msgr2=0x56226f6adba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.115+0000 7f7ff41bb740 1 --2- 172.21.15.117:0/2907042982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56226f3081b0 0x56226f6adba0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc017700 tx=0x7f7fdc0094a0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.115+0000 7f7ff41bb740 1 -- 172.21.15.117:0/2907042982 shutdown_connections 2024-09-21T06:01:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.115+0000 7f7ff41bb740 1 --2- 172.21.15.117:0/2907042982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56226f2e74d0 0x56226f6ae620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.115+0000 7f7ff41bb740 1 --2- 172.21.15.117:0/2907042982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56226f2df2f0 0x56226f6ae0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.115+0000 7f7ff41bb740 1 --2- 172.21.15.117:0/2907042982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56226f3081b0 0x56226f6adba0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.115+0000 7f7ff41bb740 1 -- 172.21.15.117:0/2907042982 >> 172.21.15.117:0/2907042982 conn(0x56226f2fc300 msgr2=0x56226f6aba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.117+0000 7f7ff41bb740 1 -- 172.21.15.117:0/2907042982 shutdown_connections 2024-09-21T06:01:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.118+0000 7f7ff41bb740 1 -- 172.21.15.117:0/2907042982 wait complete. 2024-09-21T06:01:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.119+0000 7f7ff41bb740 1 -- start start 2024-09-21T06:01:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.119+0000 7f7ff41bb740 1 Processor -- start 2024-09-21T06:01:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.119+0000 7f7ff41bb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56226f2df2f0 0x56226f6b9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.119+0000 7f7ff41bb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56226f2e74d0 0x56226f6b9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.119+0000 7f7ff41bb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56226f3081b0 0x56226f6c5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.119+0000 7f7ff41bb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56226f549ab0 con 0x56226f3081b0 2024-09-21T06:01:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.119+0000 7f7ff41bb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56226f5b7730 con 0x56226f2e74d0 2024-09-21T06:01:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.119+0000 7f7ff41bb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56226f6bc2c0 con 0x56226f2df2f0 2024-09-21T06:01:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.120+0000 7f7ff41bb740 10 client.0 _ll_register_callbacks cb 0x56226f60b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:01:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.120+0000 7f7fe659c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56226f2df2f0 0x56226f6b9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.120+0000 7f7fe659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56226f3081b0 0x56226f6c5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.120+0000 7f7fe659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56226f3081b0 0x56226f6c5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59592/0 (socket says 172.21.15.117:59592) 2024-09-21T06:01:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.120+0000 7f7fe659c640 1 -- 172.21.15.117:0/73655640 learned_addr learned my addr 172.21.15.117:0/73655640 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:01:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.121+0000 7f7fe659c640 1 -- 172.21.15.117:0/73655640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56226f2df2f0 msgr2=0x56226f6b9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.121+0000 7f7fe659c640 1 --2- 172.21.15.117:0/73655640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56226f2df2f0 0x56226f6b9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.121+0000 7f7fe659c640 1 -- 172.21.15.117:0/73655640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56226f2e74d0 msgr2=0x56226f6b9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.121+0000 7f7fe659c640 1 --2- 172.21.15.117:0/73655640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56226f2e74d0 0x56226f6b9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.121+0000 7f7fe659c640 1 -- 172.21.15.117:0/73655640 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56226f373790 con 0x56226f3081b0 2024-09-21T06:01:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.122+0000 7f7fe659c640 1 --2- 172.21.15.117:0/73655640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56226f3081b0 0x56226f6c5640 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc006a90 tx=0x7f7fdc018450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c06325265c5686 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.122+0000 7f7fcaff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:01:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.122+0000 7f7fcaff5640 1 -- 172.21.15.117:0/73655640 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc018d40 con 0x56226f3081b0 2024-09-21T06:01:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.123+0000 7f7ff41bb740 1 -- 172.21.15.117:0/73655640 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fdc01d070 con 0x56226f3081b0 2024-09-21T06:01:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.123+0000 7f7fcaff5640 1 -- 172.21.15.117:0/73655640 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f7fdc015730 con 0x56226f3081b0 2024-09-21T06:01:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.123+0000 7f7fcaff5640 1 -- 172.21.15.117:0/73655640 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7fdc015930 con 0x56226f3081b0 2024-09-21T06:01:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.124+0000 7f7ff41bb740 10 client.16224 fetch_fsmap learned FSMap version 27 2024-09-21T06:01:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.124+0000 7f7ff41bb740 1 -- 172.21.15.117:0/73655640 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56226f3ec620 con 0x56226f3081b0 2024-09-21T06:01:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.125+0000 7f7fcaff5640 1 -- 172.21.15.117:0/73655640 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fdc01d070 con 0x56226f3081b0 2024-09-21T06:01:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.125+0000 7f7fcaff5640 1 -- 172.21.15.117:0/73655640 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7fdc02a9c0 con 0x56226f3081b0 2024-09-21T06:01:20.128 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.126+0000 7f7ff41bb740 10 client.16224 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:01:20.128 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.126+0000 7f7ff41bb740 10 client.16224 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:01:20.128 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.126+0000 7f7ff41bb740 20 client.16224 populate_metadata read hostname 'smithi117' 2024-09-21T06:01:20.128 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.126+0000 7f7ff41bb740 1 --2- 172.21.15.117:0/73655640 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56226f2c6d80 0x56226f6dc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.128 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.126+0000 7f7ff41bb740 4 client.16224 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:01:20.129 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.126+0000 7f7ff41bb740 1 -- 172.21.15.117:0/73655640 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56226f624640 con 0x56226f2c6d80 2024-09-21T06:01:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.129+0000 7f7fe6d9d640 1 --2- 172.21.15.117:0/73655640 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56226f2c6d80 0x56226f6dc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.134+0000 7f7fe6d9d640 1 --2- 172.21.15.117:0/73655640 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56226f2c6d80 0x56226f6dc6e0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c874830190d2571 server_cookie=e4e7619f11da51e5 in_seq=0 out_seq=0 2024-09-21T06:01:20.136 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.134+0000 7f7fcaff5640 10 client.16224 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:01:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.138+0000 7f7fcaff5640 1 -- 172.21.15.117:0/73655640 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56226f624640 con 0x56226f2c6d80 2024-09-21T06:01:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.138+0000 7f7fcaff5640 10 client.16224 handle_command_reply: tid=0 2024-09-21T06:01:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.138+0000 7f7ff41bb740 1 client.16224 shutdown 2024-09-21T06:01:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.140+0000 7f7ff41bb740 1 -- 172.21.15.117:0/73655640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56226f3081b0 msgr2=0x56226f6c5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:20.142 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.140+0000 7f7ff41bb740 1 --2- 172.21.15.117:0/73655640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56226f3081b0 0x56226f6c5640 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc006a90 tx=0x7f7fdc018450 comp rx=0 tx=0).stop 2024-09-21T06:01:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.140+0000 7f7ff41bb740 1 -- 172.21.15.117:0/73655640 shutdown_connections 2024-09-21T06:01:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.140+0000 7f7ff41bb740 1 --2- 172.21.15.117:0/73655640 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56226f2c6d80 0x56226f6dc6e0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.141+0000 7f7ff41bb740 1 --2- 172.21.15.117:0/73655640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56226f3081b0 0x56226f6c5640 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.141+0000 7f7ff41bb740 1 --2- 172.21.15.117:0/73655640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56226f2e74d0 0x56226f6b9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.141+0000 7f7ff41bb740 1 --2- 172.21.15.117:0/73655640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56226f2df2f0 0x56226f6b9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.141+0000 7f7fcaff5640 0 client.16224 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:01:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.141+0000 7f7ff41bb740 1 -- 172.21.15.117:0/73655640 >> 172.21.15.117:0/73655640 conn(0x56226f2fc300 msgr2=0x56226f6aebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.141+0000 7f7ff41bb740 1 -- 172.21.15.117:0/73655640 shutdown_connections 2024-09-21T06:01:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.141+0000 7f7ff41bb740 1 -- 172.21.15.117:0/73655640 wait complete. 2024-09-21T06:01:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.142+0000 7f7ff41bb740 20 client.16224 trim_cache size 0 max 16384 2024-09-21T06:01:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.209+0000 7f7ff41bb740 1 Processor -- start 2024-09-21T06:01:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.209+0000 7f7ff41bb740 1 -- start start 2024-09-21T06:01:20.212 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.209+0000 7f7ff41bb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56226f2df2f0 0x56226f6c5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.211+0000 7f7ff41bb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56226f2e74d0 0x56226f6af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.212+0000 7f7ff41bb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56226f3081b0 0x56226f6b63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.214 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.212+0000 7f7ff41bb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd400e750 con 0x56226f2df2f0 2024-09-21T06:01:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.212+0000 7f7ff41bb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd4010290 con 0x56226f2e74d0 2024-09-21T06:01:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.212+0000 7f7ff41bb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd400ee20 con 0x56226f3081b0 2024-09-21T06:01:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.212+0000 7f7fe659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56226f2df2f0 0x56226f6c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.212+0000 7f7fe659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56226f2df2f0 0x56226f6c5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59596/0 (socket says 172.21.15.117:59596) 2024-09-21T06:01:20.215 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.212+0000 7f7fe659c640 1 -- 172.21.15.117:0/2228289506 learned_addr learned my addr 172.21.15.117:0/2228289506 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:01:20.216 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.215+0000 7f7fe6d9d640 1 --2- 172.21.15.117:0/2228289506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56226f2e74d0 0x56226f6af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:20.217 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.215+0000 7f7fe659c640 1 --2- 172.21.15.117:0/2228289506 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56226f3081b0 0x56226f6b63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:20.219 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.216+0000 7f7fe6d9d640 1 -- 172.21.15.117:0/2228289506 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56226f3081b0 msgr2=0x56226f6b63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:20.219 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.216+0000 7f7fe6d9d640 1 --2- 172.21.15.117:0/2228289506 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56226f3081b0 0x56226f6b63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.219 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.216+0000 7f7fe6d9d640 1 -- 172.21.15.117:0/2228289506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56226f2df2f0 msgr2=0x56226f6c5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:20.219 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.216+0000 7f7fe6d9d640 1 --2- 172.21.15.117:0/2228289506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56226f2df2f0 0x56226f6c5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.219 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.216+0000 7f7fe6d9d640 1 -- 172.21.15.117:0/2228289506 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56226f6b09b0 con 0x56226f2e74d0 2024-09-21T06:01:20.219 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.217+0000 7f7fe6d9d640 1 --2- 172.21.15.117:0/2228289506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56226f2e74d0 0x56226f6af190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x56226f6dcc40 tx=0x7f7fd8009af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1dd0164219b670 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.219+0000 7f7fccff9640 1 -- 172.21.15.117:0/2228289506 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd8016de0 con 0x56226f2e74d0 2024-09-21T06:01:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.219+0000 7f7fccff9640 1 -- 172.21.15.117:0/2228289506 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fd80148c0 con 0x56226f2e74d0 2024-09-21T06:01:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.219+0000 7f7fccff9640 1 -- 172.21.15.117:0/2228289506 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd8013860 con 0x56226f2e74d0 2024-09-21T06:01:20.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:19 smithi192 ceph-mon[34700]: pgmap v447: 97 pgs: 97 active+clean; 36 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 836 MiB/s wr, 3.82k op/s 2024-09-21T06:01:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.225+0000 7f7ff41bb740 1 -- 172.21.15.117:0/2228289506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56226f2e74d0 msgr2=0x56226f6af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.230+0000 7f7ff41bb740 1 --2- 172.21.15.117:0/2228289506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56226f2e74d0 0x56226f6af190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x56226f6dcc40 tx=0x7f7fd8009af0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.241+0000 7f7ff41bb740 1 -- 172.21.15.117:0/2228289506 shutdown_connections 2024-09-21T06:01:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.241+0000 7f7ff41bb740 1 --2- 172.21.15.117:0/2228289506 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56226f3081b0 0x56226f6b63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.241+0000 7f7ff41bb740 1 --2- 172.21.15.117:0/2228289506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56226f2e74d0 0x56226f6af190 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.242+0000 7f7ff41bb740 1 --2- 172.21.15.117:0/2228289506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56226f2df2f0 0x56226f6c5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.242+0000 7f7ff41bb740 1 -- 172.21.15.117:0/2228289506 >> 172.21.15.117:0/2228289506 conn(0x56226f2fc300 msgr2=0x56226f6aea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.244+0000 7f7ff41bb740 1 -- 172.21.15.117:0/2228289506 shutdown_connections 2024-09-21T06:01:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.249+0000 7f7ff41bb740 1 -- 172.21.15.117:0/2228289506 wait complete. 2024-09-21T06:01:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.249+0000 7f7ff41bb740 1 -- start start 2024-09-21T06:01:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.250+0000 7f7ff41bb740 1 Processor -- start 2024-09-21T06:01:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.251+0000 7f7ff41bb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56226f2df2f0 0x56226f6be6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.251+0000 7f7ff41bb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56226f2e74d0 0x56226f6bebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.251+0000 7f7ff41bb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56226f3081b0 0x56226f6bf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.251+0000 7f7ff41bb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd400fbc0 con 0x56226f2e74d0 2024-09-21T06:01:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.251+0000 7f7ff41bb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd400f4f0 con 0x56226f2df2f0 2024-09-21T06:01:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.251+0000 7f7ff41bb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd400c730 con 0x56226f3081b0 2024-09-21T06:01:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.251+0000 7f7ff41bb740 10 client.0 _ll_register_callbacks cb 0x7f7fd8009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:01:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.254+0000 7f7fe6d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56226f2e74d0 0x56226f6bebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.254+0000 7f7fe6d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56226f2e74d0 0x56226f6bebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59612/0 (socket says 172.21.15.117:59612) 2024-09-21T06:01:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.254+0000 7f7fe6d9d640 1 -- 172.21.15.117:0/871250578 learned_addr learned my addr 172.21.15.117:0/871250578 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:01:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.258+0000 7f7fe659c640 1 --2- 172.21.15.117:0/871250578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56226f3081b0 0x56226f6bf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.258+0000 7f7fe659c640 1 --2- 172.21.15.117:0/871250578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56226f2df2f0 0x56226f6be6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.260+0000 7f7fe659c640 1 -- 172.21.15.117:0/871250578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56226f2df2f0 msgr2=0x56226f6be6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.260+0000 7f7fe659c640 1 --2- 172.21.15.117:0/871250578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56226f2df2f0 0x56226f6be6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.260+0000 7f7fe659c640 1 -- 172.21.15.117:0/871250578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56226f2e74d0 msgr2=0x56226f6bebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.260+0000 7f7fe659c640 1 --2- 172.21.15.117:0/871250578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56226f2e74d0 0x56226f6bebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.260+0000 7f7fe659c640 1 -- 172.21.15.117:0/871250578 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56226f745210 con 0x56226f3081b0 2024-09-21T06:01:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.266+0000 7f7fe659c640 1 --2- 172.21.15.117:0/871250578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56226f3081b0 0x56226f6bf120 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc02aed0 tx=0x7f7fdc000e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=529528d43303a496 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.269+0000 7f7fcaff5640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-21T06:01:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.269+0000 7f7fcaff5640 1 -- 172.21.15.117:0/871250578 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc015cd0 con 0x56226f3081b0 2024-09-21T06:01:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.269+0000 7f7ff41bb740 1 -- 172.21.15.117:0/871250578 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fd80148c0 con 0x56226f3081b0 2024-09-21T06:01:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.269+0000 7f7fcaff5640 1 -- 172.21.15.117:0/871250578 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f7fdc004040 con 0x56226f3081b0 2024-09-21T06:01:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.272+0000 7f7fcaff5640 1 -- 172.21.15.117:0/871250578 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7fdc045050 con 0x56226f3081b0 2024-09-21T06:01:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.275+0000 7f7ff41bb740 10 client.25567 fetch_fsmap learned FSMap version 27 2024-09-21T06:01:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.275+0000 7f7ff41bb740 1 -- 172.21.15.117:0/871250578 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56226f7464a0 con 0x56226f3081b0 2024-09-21T06:01:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.282+0000 7f7fcaff5640 1 -- 172.21.15.117:0/871250578 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fd80148c0 con 0x56226f3081b0 2024-09-21T06:01:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.282+0000 7f7fcaff5640 1 -- 172.21.15.117:0/871250578 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7fdc0226d0 con 0x56226f3081b0 2024-09-21T06:01:20.286 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.284+0000 7f7ff41bb740 10 client.25567 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:01:20.286 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.284+0000 7f7ff41bb740 10 client.25567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:01:20.286 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.284+0000 7f7ff41bb740 20 client.25567 populate_metadata read hostname 'smithi117' 2024-09-21T06:01:20.286 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.284+0000 7f7ff41bb740 1 --2- 172.21.15.117:0/871250578 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56226f29c1c0 0x56226f748ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.286 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.284+0000 7f7ff41bb740 4 client.25567 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:01:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.284+0000 7f7ff41bb740 1 -- 172.21.15.117:0/871250578 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56226f6200f0 con 0x56226f29c1c0 2024-09-21T06:01:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.285+0000 7f7fe6d9d640 1 --2- 172.21.15.117:0/871250578 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56226f29c1c0 0x56226f748ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.298+0000 7f7fe6d9d640 1 --2- 172.21.15.117:0/871250578 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56226f29c1c0 0x56226f748ba0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33abe632e67d5c0b server_cookie=e749cb8e9e828212 in_seq=0 out_seq=0 2024-09-21T06:01:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:20.298+0000 7f7fcaff5640 10 client.25567 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:01:20.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.267+0000 7fb2b1ffb640 1 -- 172.21.15.117:0/1426909472 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8030850 con 0x7fb2b411cc30 2024-09-21T06:01:20.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.268+0000 7fb2b968b640 1 -- 172.21.15.117:0/1426909472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2b411cc30 msgr2=0x7fb2b411f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:20.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.268+0000 7fb2b968b640 1 --2- 172.21.15.117:0/1426909472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2b411cc30 0x7fb2b411f140 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8009860 tx=0x7fb2a802a340 comp rx=0 tx=0).stop 2024-09-21T06:01:20.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.272+0000 7fb2b968b640 1 -- 172.21.15.117:0/1426909472 shutdown_connections 2024-09-21T06:01:20.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.272+0000 7fb2b968b640 1 --2- 172.21.15.117:0/1426909472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2b410a9e0 0x7fb2b410adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.272+0000 7fb2b968b640 1 --2- 172.21.15.117:0/1426909472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2b411cc30 0x7fb2b411f140 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.272+0000 7fb2b968b640 1 --2- 172.21.15.117:0/1426909472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2b4111f40 0x7fb2b410a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.272+0000 7fb2b968b640 1 -- 172.21.15.117:0/1426909472 >> 172.21.15.117:0/1426909472 conn(0x7fb2b4070ec0 msgr2=0x7fb2b40712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:20.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.277+0000 7fb2b968b640 1 -- 172.21.15.117:0/1426909472 shutdown_connections 2024-09-21T06:01:20.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.277+0000 7fb2b968b640 1 -- 172.21.15.117:0/1426909472 wait complete. 2024-09-21T06:01:20.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.278+0000 7fb2b968b640 1 Processor -- start 2024-09-21T06:01:20.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.284+0000 7fb2b968b640 1 -- start start 2024-09-21T06:01:20.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.285+0000 7fb2b968b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2b410a9e0 0x7fb2b41a92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.285+0000 7fb2b968b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2b4111f40 0x7fb2b41a9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.285+0000 7fb2b968b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2b411cc30 0x7fb2b41aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.285+0000 7fb2b968b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b4122900 con 0x7fb2b4111f40 2024-09-21T06:01:20.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.285+0000 7fb2b968b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b4122780 con 0x7fb2b410a9e0 2024-09-21T06:01:20.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.285+0000 7fb2b968b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b4122a80 con 0x7fb2b411cc30 2024-09-21T06:01:20.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.289+0000 7fb2b27fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2b4111f40 0x7fb2b41a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:20.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.289+0000 7fb2b37fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2b411cc30 0x7fb2b41aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:20.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.289+0000 7fb2b37fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2b411cc30 0x7fb2b41aeae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:45956/0 (socket says 172.21.15.117:45956) 2024-09-21T06:01:20.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.289+0000 7fb2b37fe640 1 -- 172.21.15.117:0/2291126212 learned_addr learned my addr 172.21.15.117:0/2291126212 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:01:20.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.289+0000 7fb2b2ffd640 1 --2- 172.21.15.117:0/2291126212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2b410a9e0 0x7fb2b41a92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:20.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.292+0000 7fb2b2ffd640 1 -- 172.21.15.117:0/2291126212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2b411cc30 msgr2=0x7fb2b41aeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:20.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.292+0000 7fb2b2ffd640 1 --2- 172.21.15.117:0/2291126212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2b411cc30 0x7fb2b41aeae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.292+0000 7fb2b2ffd640 1 -- 172.21.15.117:0/2291126212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2b4111f40 msgr2=0x7fb2b41a9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:20.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.292+0000 7fb2b2ffd640 1 --2- 172.21.15.117:0/2291126212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2b4111f40 0x7fb2b41a9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.292+0000 7fb2b2ffd640 1 -- 172.21.15.117:0/2291126212 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b41af210 con 0x7fb2b410a9e0 2024-09-21T06:01:20.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.293+0000 7fb2b2ffd640 1 --2- 172.21.15.117:0/2291126212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2b410a9e0 0x7fb2b41a92a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac00e9f0 tx=0x7fb2ac00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:20.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.294+0000 7fb29bfff640 1 -- 172.21.15.117:0/2291126212 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac00cd20 con 0x7fb2b410a9e0 2024-09-21T06:01:20.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.294+0000 7fb2b968b640 1 -- 172.21.15.117:0/2291126212 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2b41af500 con 0x7fb2b410a9e0 2024-09-21T06:01:20.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.294+0000 7fb2b968b640 1 -- 172.21.15.117:0/2291126212 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2b41b5c60 con 0x7fb2b410a9e0 2024-09-21T06:01:20.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.294+0000 7fb29bfff640 1 -- 172.21.15.117:0/2291126212 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2ac011920 con 0x7fb2b410a9e0 2024-09-21T06:01:20.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.294+0000 7fb29bfff640 1 -- 172.21.15.117:0/2291126212 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac01a830 con 0x7fb2b410a9e0 2024-09-21T06:01:20.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.303+0000 7fb2bfbff740 1 Processor -- start 2024-09-21T06:01:20.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.304+0000 7fb2bfbff740 1 -- start start 2024-09-21T06:01:20.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.304+0000 7fb2bfbff740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8182cc350 0x55e818673430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.305+0000 7fb2bfbff740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8182a3490 0x55e818673970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.305+0000 7fb2bfbff740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8182ab670 0x55e818673eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.306+0000 7fb2b37fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8182cc350 0x55e818673430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:20.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.307+0000 7fb2bfbff740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e8185e9e60 con 0x55e8182a3490 2024-09-21T06:01:20.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.307+0000 7fb2b37fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8182cc350 0x55e818673430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:45960/0 (socket says 172.21.15.117:45960) 2024-09-21T06:01:20.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.307+0000 7fb2b37fe640 1 -- 172.21.15.117:0/3509525098 learned_addr learned my addr 172.21.15.117:0/3509525098 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:01:20.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.307+0000 7fb2bfbff740 1 -- 172.21.15.117:0/3509525098 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e8185d9290 con 0x55e8182ab670 2024-09-21T06:01:20.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.307+0000 7fb2bfbff740 1 -- 172.21.15.117:0/3509525098 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55e8185dd0a0 con 0x55e8182cc350 2024-09-21T06:01:20.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.307+0000 7fb2b27fc640 1 --2- 172.21.15.117:0/3509525098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8182ab670 0x55e818673eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:20.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.308+0000 7fb2b27fc640 1 -- 172.21.15.117:0/3509525098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8182cc350 msgr2=0x55e818673430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:20.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.308+0000 7fb2b27fc640 1 --2- 172.21.15.117:0/3509525098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8182cc350 0x55e818673430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.308+0000 7fb2b27fc640 1 -- 172.21.15.117:0/3509525098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8182a3490 msgr2=0x55e818673970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:01:20.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.308+0000 7fb2b27fc640 1 --2- 172.21.15.117:0/3509525098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8182a3490 0x55e818673970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.308+0000 7fb2b27fc640 1 -- 172.21.15.117:0/3509525098 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e818337930 con 0x55e8182ab670 2024-09-21T06:01:20.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.309+0000 7fb2b27fc640 1 --2- 172.21.15.117:0/3509525098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8182ab670 0x55e818673eb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb2b406c970 tx=0x7fb2a4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed178eef96f92f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:20.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.309+0000 7fb2997fa640 1 -- 172.21.15.117:0/3509525098 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a4009db0 con 0x55e8182ab670 2024-09-21T06:01:20.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.310+0000 7fb2997fa640 1 -- 172.21.15.117:0/3509525098 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2a40035f0 con 0x55e8182ab670 2024-09-21T06:01:20.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.311+0000 7fb29bfff640 1 -- 172.21.15.117:0/2291126212 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fb2ac011470 con 0x7fb2b410a9e0 2024-09-21T06:01:20.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.311+0000 7fb2bfbff740 1 -- 172.21.15.117:0/3509525098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8182ab670 msgr2=0x55e818673eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:20.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.311+0000 7fb2bfbff740 1 --2- 172.21.15.117:0/3509525098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8182ab670 0x55e818673eb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb2b406c970 tx=0x7fb2a4011810 comp rx=0 tx=0).stop 2024-09-21T06:01:20.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.315+0000 7fb2bfbff740 1 -- 172.21.15.117:0/3509525098 shutdown_connections 2024-09-21T06:01:20.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.315+0000 7fb2bfbff740 1 --2- 172.21.15.117:0/3509525098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8182a3490 0x55e818673970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.315+0000 7fb2bfbff740 1 --2- 172.21.15.117:0/3509525098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8182ab670 0x55e818673eb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.315+0000 7fb2bfbff740 1 --2- 172.21.15.117:0/3509525098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8182cc350 0x55e818673430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.315+0000 7fb2bfbff740 1 -- 172.21.15.117:0/3509525098 >> 172.21.15.117:0/3509525098 conn(0x55e8182c0740 msgr2=0x55e818671320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:20.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.315+0000 7fb29bfff640 1 --2- 172.21.15.117:0/2291126212 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb29c0b6780 0x7fb29c0b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.315+0000 7fb2bfbff740 1 -- 172.21.15.117:0/3509525098 shutdown_connections 2024-09-21T06:01:20.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.315+0000 7fb2bfbff740 1 -- 172.21.15.117:0/3509525098 wait complete. 2024-09-21T06:01:20.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.316+0000 7fb29bfff640 1 -- 172.21.15.117:0/2291126212 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fb2ac0e7950 con 0x7fb2b410a9e0 2024-09-21T06:01:20.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.316+0000 7fb2bfbff740 1 -- start start 2024-09-21T06:01:20.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.316+0000 7fb2bfbff740 1 Processor -- start 2024-09-21T06:01:20.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.317+0000 7fb2bfbff740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8182a3490 0x55e81867ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.317+0000 7fb2bfbff740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8182ab670 0x55e81867f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.317+0000 7fb2bfbff740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8182cc350 0x55e81868b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.317+0000 7fb2bfbff740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e81843a890 con 0x55e8182ab670 2024-09-21T06:01:20.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.317+0000 7fb2bfbff740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e818573d40 con 0x55e8182cc350 2024-09-21T06:01:20.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.317+0000 7fb2bfbff740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55e818681c60 con 0x55e8182a3490 2024-09-21T06:01:20.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.317+0000 7fb2bfbff740 10 client.0 _ll_register_callbacks cb 0x55e8185d7c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:01:20.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.320+0000 7fb2b37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8182cc350 0x55e81868b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:20.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.320+0000 7fb2b37fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8182cc350 0x55e81868b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:38462/0 (socket says 172.21.15.117:38462) 2024-09-21T06:01:20.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.320+0000 7fb2b37fe640 1 -- 172.21.15.117:0/406696827 learned_addr learned my addr 172.21.15.117:0/406696827 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:01:20.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.320+0000 7fb2b2ffd640 1 --2- 172.21.15.117:0/406696827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8182a3490 0x55e81867ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:20.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.321+0000 7fb2b37fe640 1 -- 172.21.15.117:0/406696827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8182a3490 msgr2=0x55e81867ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:20.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.321+0000 7fb2b37fe640 1 --2- 172.21.15.117:0/406696827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8182a3490 0x55e81867ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.321+0000 7fb2b37fe640 1 -- 172.21.15.117:0/406696827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8182ab670 msgr2=0x55e81867f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:20.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.321+0000 7fb2b37fe640 1 --2- 172.21.15.117:0/406696827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8182ab670 0x55e81867f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.321+0000 7fb2b37fe640 1 -- 172.21.15.117:0/406696827 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e81869c580 con 0x55e8182cc350 2024-09-21T06:01:20.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.324+0000 7fb2b27fc640 1 --2- 172.21.15.117:0/2291126212 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb29c0b6780 0x7fb29c0b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:20.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.324+0000 7fb2b37fe640 1 --2- 172.21.15.117:0/406696827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8182cc350 0x55e81868b030 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8030f40 tx=0x7fb2a8030f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2181a9818938e99a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:20.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.325+0000 7fb296ff5640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-21T06:01:20.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.329+0000 7fb296ff5640 1 -- 172.21.15.117:0/406696827 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8042070 con 0x55e8182cc350 2024-09-21T06:01:20.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.329+0000 7fb2bfbff740 1 -- 172.21.15.117:0/406696827 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2a40035f0 con 0x55e8182cc350 2024-09-21T06:01:20.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.329+0000 7fb296ff5640 1 -- 172.21.15.117:0/406696827 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fb2a8004480 con 0x55e8182cc350 2024-09-21T06:01:20.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.333+0000 7fb296ff5640 1 -- 172.21.15.117:0/406696827 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb2a804e050 con 0x55e8182cc350 2024-09-21T06:01:20.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.333+0000 7fb2bfbff740 10 client.25595 fetch_fsmap learned FSMap version 27 2024-09-21T06:01:20.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.333+0000 7fb2bfbff740 1 -- 172.21.15.117:0/406696827 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e81869ee60 con 0x55e8182cc350 2024-09-21T06:01:20.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.334+0000 7fb296ff5640 1 -- 172.21.15.117:0/406696827 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2a40035f0 con 0x55e8182cc350 2024-09-21T06:01:20.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.334+0000 7fb296ff5640 1 -- 172.21.15.117:0/406696827 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb2a803e460 con 0x55e8182cc350 2024-09-21T06:01:20.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.334+0000 7fb2bfbff740 10 client.25595 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:01:20.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.334+0000 7fb2bfbff740 10 client.25595 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:01:20.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.334+0000 7fb2bfbff740 20 client.25595 populate_metadata read hostname 'smithi117' 2024-09-21T06:01:20.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.334+0000 7fb2bfbff740 1 --2- 172.21.15.117:0/406696827 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55e81828acd0 0x55e8186a2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.334+0000 7fb2bfbff740 4 client.25595 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:01:20.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.334+0000 7fb2bfbff740 1 -- 172.21.15.117:0/406696827 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8185ebdb0 con 0x55e81828acd0 2024-09-21T06:01:20.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.336+0000 7fb2b2ffd640 1 --2- 172.21.15.117:0/406696827 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55e81828acd0 0x55e8186a2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:20.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.340+0000 7fb2b2ffd640 1 --2- 172.21.15.117:0/406696827 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55e81828acd0 0x55e8186a2780 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa2e8c97a36a3eb3 server_cookie=f518a287744e1f36 in_seq=0 out_seq=0 2024-09-21T06:01:20.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.340+0000 7fb296ff5640 10 client.25595 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:01:20.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.340+0000 7fb2b27fc640 1 --2- 172.21.15.117:0/2291126212 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb29c0b6780 0x7fb29c0b8c90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb2a40099f0 tx=0x7fb2a4009600 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:20.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.345+0000 7fb296ff5640 1 -- 172.21.15.117:0/406696827 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e8185ebdb0 con 0x55e81828acd0 2024-09-21T06:01:20.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.345+0000 7fb296ff5640 10 client.25595 handle_command_reply: tid=0 2024-09-21T06:01:20.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.345+0000 7fb2bfbff740 1 client.25595 shutdown 2024-09-21T06:01:20.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.347+0000 7fb2bfbff740 1 -- 172.21.15.117:0/406696827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8182cc350 msgr2=0x55e81868b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:20.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.347+0000 7fb2bfbff740 1 --2- 172.21.15.117:0/406696827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8182cc350 0x55e81868b030 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8030f40 tx=0x7fb2a8030f70 comp rx=0 tx=0).stop 2024-09-21T06:01:20.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.347+0000 7fb2bfbff740 1 -- 172.21.15.117:0/406696827 shutdown_connections 2024-09-21T06:01:20.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.347+0000 7fb2bfbff740 1 --2- 172.21.15.117:0/406696827 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55e81828acd0 0x55e8186a2780 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.347+0000 7fb2bfbff740 1 --2- 172.21.15.117:0/406696827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8182ab670 0x55e81867f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.347+0000 7fb2bfbff740 1 --2- 172.21.15.117:0/406696827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8182cc350 0x55e81868b030 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.347+0000 7fb2bfbff740 1 --2- 172.21.15.117:0/406696827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8182a3490 0x55e81867ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.347+0000 7fb2bfbff740 1 -- 172.21.15.117:0/406696827 >> 172.21.15.117:0/406696827 conn(0x55e8182c0740 msgr2=0x55e8186765c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:20.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.347+0000 7fb296ff5640 0 client.25595 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:01:20.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.351+0000 7fb2bfbff740 1 -- 172.21.15.117:0/406696827 shutdown_connections 2024-09-21T06:01:20.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.351+0000 7fb2bfbff740 1 -- 172.21.15.117:0/406696827 wait complete. 2024-09-21T06:01:20.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.351+0000 7fb2bfbff740 20 client.25595 trim_cache size 0 max 16384 2024-09-21T06:01:20.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.398+0000 7fb2bfbff740 1 Processor -- start 2024-09-21T06:01:20.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.398+0000 7fb2bfbff740 1 -- start start 2024-09-21T06:01:20.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.399+0000 7fb2bfbff740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8182a3490 0x55e818679550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.399+0000 7fb2bfbff740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8182ab670 0x55e81868af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.399+0000 7fb2bfbff740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8182cc350 0x55e81867bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.399+0000 7fb2bfbff740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a000e6f0 con 0x55e8182a3490 2024-09-21T06:01:20.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.399+0000 7fb2bfbff740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a0010230 con 0x55e8182ab670 2024-09-21T06:01:20.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.399+0000 7fb2bfbff740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a000edc0 con 0x55e8182cc350 2024-09-21T06:01:20.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.400+0000 7fb2b37fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8182cc350 0x55e81867bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:20.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.400+0000 7fb2b37fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8182cc350 0x55e81867bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:45966/0 (socket says 172.21.15.117:45966) 2024-09-21T06:01:20.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.400+0000 7fb2b37fe640 1 -- 172.21.15.117:0/907436786 learned_addr learned my addr 172.21.15.117:0/907436786 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:01:20.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.402+0000 7fb2b27fc640 1 --2- 172.21.15.117:0/907436786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8182ab670 0x55e81868af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:20.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.403+0000 7fb2b2ffd640 1 --2- 172.21.15.117:0/907436786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8182a3490 0x55e818679550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:20.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.406+0000 7fb2b37fe640 1 -- 172.21.15.117:0/907436786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8182ab670 msgr2=0x55e81868af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:20.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.406+0000 7fb2b37fe640 1 --2- 172.21.15.117:0/907436786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8182ab670 0x55e81868af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.406+0000 7fb2b37fe640 1 -- 172.21.15.117:0/907436786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8182a3490 msgr2=0x55e818679550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:20.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.406+0000 7fb2b37fe640 1 --2- 172.21.15.117:0/907436786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8182a3490 0x55e818679550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.406+0000 7fb2b37fe640 1 -- 172.21.15.117:0/907436786 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e81828af00 con 0x55e8182cc350 2024-09-21T06:01:20.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.406+0000 7fb2b27fc640 1 --2- 172.21.15.117:0/907436786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8182ab670 0x55e81868af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:01:20.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.408+0000 7fb2b37fe640 1 --2- 172.21.15.117:0/907436786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8182cc350 0x55e81867bf70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8009780 tx=0x7fb2a802a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e0ab75caaf0058a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:20.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.412+0000 7fb298ff9640 1 -- 172.21.15.117:0/907436786 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8042070 con 0x55e8182cc350 2024-09-21T06:01:20.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.412+0000 7fb298ff9640 1 -- 172.21.15.117:0/907436786 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2a8030900 con 0x55e8182cc350 2024-09-21T06:01:20.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.413+0000 7fb2bfbff740 1 -- 172.21.15.117:0/907436786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8182cc350 msgr2=0x55e81867bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:20.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.413+0000 7fb2bfbff740 1 --2- 172.21.15.117:0/907436786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8182cc350 0x55e81867bf70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8009780 tx=0x7fb2a802a340 comp rx=0 tx=0).stop 2024-09-21T06:01:20.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.416+0000 7fb2bfbff740 1 -- 172.21.15.117:0/907436786 shutdown_connections 2024-09-21T06:01:20.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.416+0000 7fb2bfbff740 1 --2- 172.21.15.117:0/907436786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8182cc350 0x55e81867bf70 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.416+0000 7fb2bfbff740 1 --2- 172.21.15.117:0/907436786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8182ab670 0x55e81868af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.416+0000 7fb2bfbff740 1 --2- 172.21.15.117:0/907436786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8182a3490 0x55e818679550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.416+0000 7fb2bfbff740 1 -- 172.21.15.117:0/907436786 >> 172.21.15.117:0/907436786 conn(0x55e8182c0740 msgr2=0x55e81867d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:20.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.418+0000 7fb2bfbff740 1 -- 172.21.15.117:0/907436786 shutdown_connections 2024-09-21T06:01:20.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.422+0000 7fb2bfbff740 1 -- 172.21.15.117:0/907436786 wait complete. 2024-09-21T06:01:20.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.423+0000 7fb2bfbff740 1 -- start start 2024-09-21T06:01:20.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.423+0000 7fb2bfbff740 1 Processor -- start 2024-09-21T06:01:20.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.424+0000 7fb2bfbff740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8182a3490 0x55e818683310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.424+0000 7fb2bfbff740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8182ab670 0x55e818683850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.424+0000 7fb2bfbff740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8182cc350 0x55e81870c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.424+0000 7fb2bfbff740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a000fb60 con 0x55e8182a3490 2024-09-21T06:01:20.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.424+0000 7fb2bfbff740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a000f490 con 0x55e8182cc350 2024-09-21T06:01:20.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.424+0000 7fb2bfbff740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a000c6d0 con 0x55e8182ab670 2024-09-21T06:01:20.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.424+0000 7fb2bfbff740 10 client.0 _ll_register_callbacks cb 0x7fb2a80096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:01:20.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.427+0000 7fb2b27fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8182ab670 0x55e818683850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:20.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.428+0000 7fb2b2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8182a3490 0x55e818683310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:20.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.428+0000 7fb2b2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8182a3490 0x55e818683310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59672/0 (socket says 172.21.15.117:59672) 2024-09-21T06:01:20.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.428+0000 7fb2b2ffd640 1 -- 172.21.15.117:0/2698758845 learned_addr learned my addr 172.21.15.117:0/2698758845 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:01:20.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.429+0000 7fb2b37fe640 1 --2- 172.21.15.117:0/2698758845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8182cc350 0x55e81870c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:20.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.433+0000 7fb2b37fe640 1 -- 172.21.15.117:0/2698758845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8182ab670 msgr2=0x55e818683850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:20.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.433+0000 7fb2b37fe640 1 --2- 172.21.15.117:0/2698758845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8182ab670 0x55e818683850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.433+0000 7fb2b37fe640 1 -- 172.21.15.117:0/2698758845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8182a3490 msgr2=0x55e818683310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:20.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.433+0000 7fb2b37fe640 1 --2- 172.21.15.117:0/2698758845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8182a3490 0x55e818683310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.433+0000 7fb2b37fe640 1 -- 172.21.15.117:0/2698758845 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e81870c680 con 0x55e8182cc350 2024-09-21T06:01:20.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.437+0000 7fb2b37fe640 1 --2- 172.21.15.117:0/2698758845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8182cc350 0x55e81870c140 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb2a803e950 tx=0x7fb2a803e730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ceb0d4f0bc180385 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:20.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.437+0000 7fb296ff5640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-21T06:01:20.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.441+0000 7fb296ff5640 1 -- 172.21.15.117:0/2698758845 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8042040 con 0x55e8182cc350 2024-09-21T06:01:20.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.442+0000 7fb2bfbff740 1 -- 172.21.15.117:0/2698758845 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e8185e7860 con 0x55e8182cc350 2024-09-21T06:01:20.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.442+0000 7fb296ff5640 1 -- 172.21.15.117:0/2698758845 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fb2a8004480 con 0x55e8182cc350 2024-09-21T06:01:20.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.447+0000 7fb296ff5640 1 -- 172.21.15.117:0/2698758845 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb2a8060050 con 0x55e8182cc350 2024-09-21T06:01:20.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.447+0000 7fb2bfbff740 10 client.25601 fetch_fsmap learned FSMap version 27 2024-09-21T06:01:20.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.447+0000 7fb2bfbff740 1 -- 172.21.15.117:0/2698758845 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e81870d840 con 0x55e8182cc350 2024-09-21T06:01:20.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.448+0000 7fb296ff5640 1 -- 172.21.15.117:0/2698758845 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e8185e7860 con 0x55e8182cc350 2024-09-21T06:01:20.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.448+0000 7fb296ff5640 1 -- 172.21.15.117:0/2698758845 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb2a80047e0 con 0x55e8182cc350 2024-09-21T06:01:20.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.448+0000 7fb2bfbff740 10 client.25601 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:01:20.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.448+0000 7fb2bfbff740 10 client.25601 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:01:20.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.448+0000 7fb2bfbff740 20 client.25601 populate_metadata read hostname 'smithi117' 2024-09-21T06:01:20.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.448+0000 7fb2bfbff740 1 --2- 172.21.15.117:0/2698758845 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55e818260080 0x55e81870fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:20.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.449+0000 7fb2bfbff740 4 client.25601 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:01:20.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.449+0000 7fb2bfbff740 1 -- 172.21.15.117:0/2698758845 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e8185e4c90 con 0x55e818260080 2024-09-21T06:01:20.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.452+0000 7fb2b2ffd640 1 --2- 172.21.15.117:0/2698758845 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55e818260080 0x55e81870fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:20.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.461+0000 7fb2b2ffd640 1 --2- 172.21.15.117:0/2698758845 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55e818260080 0x55e81870fe90 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca1bfd933edce54b server_cookie=9b15b39c6f316b5c in_seq=0 out_seq=0 2024-09-21T06:01:20.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.462+0000 7fb296ff5640 10 client.25601 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:01:20.587 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T06:01:20.587 INFO:teuthology.orchestra.run.smithi117.stdout: "return_code": 0, 2024-09-21T06:01:20.588 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_tag": "d543b0ae-b710-4153-99da-fe5714839c1a", 2024-09-21T06:01:20.588 INFO:teuthology.orchestra.run.smithi117.stdout: "mode": "asynchronous" 2024-09-21T06:01:20.588 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T06:01:20.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.577+0000 7fb296ff5640 1 -- 172.21.15.117:0/2698758845 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e8185e4c90 con 0x55e818260080 2024-09-21T06:01:20.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.577+0000 7fb296ff5640 10 client.25601 handle_command_reply: tid=0 2024-09-21T06:01:20.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.577+0000 7fb2bfbff740 1 client.25601 shutdown 2024-09-21T06:01:20.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.577+0000 7fb2bfbff740 1 -- 172.21.15.117:0/2698758845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8182cc350 msgr2=0x55e81870c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:20.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.577+0000 7fb2bfbff740 1 --2- 172.21.15.117:0/2698758845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8182cc350 0x55e81870c140 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb2a803e950 tx=0x7fb2a803e730 comp rx=0 tx=0).stop 2024-09-21T06:01:20.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.578+0000 7fb2bfbff740 1 -- 172.21.15.117:0/2698758845 shutdown_connections 2024-09-21T06:01:20.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.578+0000 7fb2bfbff740 1 --2- 172.21.15.117:0/2698758845 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55e818260080 0x55e81870fe90 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.578+0000 7fb2bfbff740 1 --2- 172.21.15.117:0/2698758845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e8182cc350 0x55e81870c140 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.578+0000 7fb2bfbff740 1 --2- 172.21.15.117:0/2698758845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e8182ab670 0x55e818683850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.578+0000 7fb2bfbff740 1 --2- 172.21.15.117:0/2698758845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e8182a3490 0x55e818683310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.578+0000 7fb2bfbff740 1 -- 172.21.15.117:0/2698758845 >> 172.21.15.117:0/2698758845 conn(0x55e8182c0740 msgr2=0x55e818674460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:20.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.578+0000 7fb296ff5640 0 client.25601 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:01:20.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.578+0000 7fb2bfbff740 1 -- 172.21.15.117:0/2698758845 shutdown_connections 2024-09-21T06:01:20.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.578+0000 7fb2bfbff740 1 -- 172.21.15.117:0/2698758845 wait complete. 2024-09-21T06:01:20.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.579+0000 7fb2bfbff740 20 client.25601 trim_cache size 0 max 16384 2024-09-21T06:01:20.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.580+0000 7fb2b968b640 1 -- 172.21.15.117:0/2291126212 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb29c0b6780 msgr2=0x7fb29c0b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:20.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.580+0000 7fb2b968b640 1 --2- 172.21.15.117:0/2291126212 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb29c0b6780 0x7fb29c0b8c90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb2a40099f0 tx=0x7fb2a4009600 comp rx=0 tx=0).stop 2024-09-21T06:01:20.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.580+0000 7fb2b968b640 1 -- 172.21.15.117:0/2291126212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2b410a9e0 msgr2=0x7fb2b41a92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:20.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.580+0000 7fb2b968b640 1 --2- 172.21.15.117:0/2291126212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2b410a9e0 0x7fb2b41a92a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac00e9f0 tx=0x7fb2ac00edf0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.581+0000 7fb2b968b640 1 -- 172.21.15.117:0/2291126212 shutdown_connections 2024-09-21T06:01:20.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.581+0000 7fb2b968b640 1 --2- 172.21.15.117:0/2291126212 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb29c0b6780 0x7fb29c0b8c90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.581+0000 7fb2b968b640 1 --2- 172.21.15.117:0/2291126212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2b4111f40 0x7fb2b41a9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.581+0000 7fb2b968b640 1 --2- 172.21.15.117:0/2291126212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2b411cc30 0x7fb2b41aeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.581+0000 7fb2b968b640 1 --2- 172.21.15.117:0/2291126212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb2b410a9e0 0x7fb2b41a92a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:20.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.581+0000 7fb2b968b640 1 -- 172.21.15.117:0/2291126212 >> 172.21.15.117:0/2291126212 conn(0x7fb2b4070ec0 msgr2=0x7fb2b411e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:20.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.581+0000 7fb2b968b640 1 -- 172.21.15.117:0/2291126212 shutdown_connections 2024-09-21T06:01:20.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:20.581+0000 7fb2b968b640 1 -- 172.21.15.117:0/2291126212 wait complete. 2024-09-21T06:01:22.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:21 smithi117 ceph-mon[30947]: scrub queued for path: / 2024-09-21T06:01:22.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:21 smithi117 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-21T06:01:22.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:21 smithi117 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-21T06:01:22.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:21 smithi117 ceph-mon[30947]: pgmap v448: 97 pgs: 97 active+clean; 37 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 752 MiB/s wr, 3.27k op/s 2024-09-21T06:01:22.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:21 smithi138 ceph-mon[34391]: scrub queued for path: / 2024-09-21T06:01:22.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:21 smithi138 ceph-mon[34391]: scrub summary: idle+waiting paths [/] 2024-09-21T06:01:22.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:21 smithi138 ceph-mon[34391]: scrub summary: active paths [/] 2024-09-21T06:01:22.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:21 smithi138 ceph-mon[34391]: pgmap v448: 97 pgs: 97 active+clean; 37 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 752 MiB/s wr, 3.27k op/s 2024-09-21T06:01:22.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:21 smithi192 ceph-mon[34700]: scrub queued for path: / 2024-09-21T06:01:22.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:21 smithi192 ceph-mon[34700]: scrub summary: idle+waiting paths [/] 2024-09-21T06:01:22.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:21 smithi192 ceph-mon[34700]: scrub summary: active paths [/] 2024-09-21T06:01:22.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:21 smithi192 ceph-mon[34700]: pgmap v448: 97 pgs: 97 active+clean; 37 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 752 MiB/s wr, 3.27k op/s 2024-09-21T06:01:22.765 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:01:23.017 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:01:23.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.544+0000 7f9373fd2640 1 -- 172.21.15.117:0/861169835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f936c10b6d0 msgr2=0x7f936c10bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:23.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.544+0000 7f9373fd2640 1 --2- 172.21.15.117:0/861169835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f936c10b6d0 0x7f936c10bae0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9368009860 tx=0x7f936802a340 comp rx=0 tx=0).stop 2024-09-21T06:01:23.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.544+0000 7f9373fd2640 1 -- 172.21.15.117:0/861169835 shutdown_connections 2024-09-21T06:01:23.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.544+0000 7f9373fd2640 1 --2- 172.21.15.117:0/861169835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f936c10c020 0x7f936c112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:23.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.544+0000 7f9373fd2640 1 --2- 172.21.15.117:0/861169835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f936c10b6d0 0x7f936c10bae0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:23.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.544+0000 7f9373fd2640 1 --2- 172.21.15.117:0/861169835 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f936c105d90 0x7f936c1061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:23.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.544+0000 7f9373fd2640 1 -- 172.21.15.117:0/861169835 >> 172.21.15.117:0/861169835 conn(0x7f936c1017c0 msgr2=0x7f936c103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:23.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.545+0000 7f9373fd2640 1 -- 172.21.15.117:0/861169835 shutdown_connections 2024-09-21T06:01:23.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.545+0000 7f9373fd2640 1 -- 172.21.15.117:0/861169835 wait complete. 2024-09-21T06:01:23.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.546+0000 7f9373fd2640 1 Processor -- start 2024-09-21T06:01:23.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.546+0000 7f9373fd2640 1 -- start start 2024-09-21T06:01:23.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.546+0000 7f9373fd2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f936c105d90 0x7f936c10fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:23.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.546+0000 7f9373fd2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f936c10b6d0 0x7f936c110280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:23.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.546+0000 7f9373fd2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f936c10c020 0x7f936c1107c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:23.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.546+0000 7f9373fd2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f936c115ce0 con 0x7f936c10c020 2024-09-21T06:01:23.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.546+0000 7f9373fd2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f936c115b60 con 0x7f936c105d90 2024-09-21T06:01:23.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.546+0000 7f9373fd2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f936c115e60 con 0x7f936c10b6d0 2024-09-21T06:01:23.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.547+0000 7f937156d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f936c10b6d0 0x7f936c110280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:23.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.547+0000 7f937156d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f936c10b6d0 0x7f936c110280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:45976/0 (socket says 172.21.15.117:45976) 2024-09-21T06:01:23.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.547+0000 7f937156d640 1 -- 172.21.15.117:0/131735442 learned_addr learned my addr 172.21.15.117:0/131735442 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:01:23.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.547+0000 7f937256f640 1 --2- 172.21.15.117:0/131735442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f936c10c020 0x7f936c1107c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:23.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.547+0000 7f9371d6e640 1 --2- 172.21.15.117:0/131735442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f936c105d90 0x7f936c10fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:23.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.547+0000 7f937256f640 1 -- 172.21.15.117:0/131735442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f936c10b6d0 msgr2=0x7f936c110280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:23.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.547+0000 7f937256f640 1 --2- 172.21.15.117:0/131735442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f936c10b6d0 0x7f936c110280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:23.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.547+0000 7f937256f640 1 -- 172.21.15.117:0/131735442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f936c105d90 msgr2=0x7f936c10fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:23.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.547+0000 7f937256f640 1 --2- 172.21.15.117:0/131735442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f936c105d90 0x7f936c10fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:23.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.547+0000 7f937256f640 1 -- 172.21.15.117:0/131735442 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f936c1acb80 con 0x7f936c10c020 2024-09-21T06:01:23.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.547+0000 7f937256f640 1 --2- 172.21.15.117:0/131735442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f936c10c020 0x7f936c1107c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f936400efc0 tx=0x7f936400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:23.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.548+0000 7f9362ffd640 1 -- 172.21.15.117:0/131735442 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9364003970 con 0x7f936c10c020 2024-09-21T06:01:23.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.548+0000 7f9373fd2640 1 -- 172.21.15.117:0/131735442 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f936c1ace70 con 0x7f936c10c020 2024-09-21T06:01:23.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.548+0000 7f9373fd2640 1 -- 172.21.15.117:0/131735442 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f936c1ad2d0 con 0x7f936c10c020 2024-09-21T06:01:23.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.548+0000 7f9362ffd640 1 -- 172.21.15.117:0/131735442 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9364004020 con 0x7f936c10c020 2024-09-21T06:01:23.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.548+0000 7f9362ffd640 1 -- 172.21.15.117:0/131735442 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936400b940 con 0x7f936c10c020 2024-09-21T06:01:23.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.551+0000 7f937a546740 1 Processor -- start 2024-09-21T06:01:23.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.552+0000 7f9362ffd640 1 -- 172.21.15.117:0/131735442 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f936401d020 con 0x7f936c10c020 2024-09-21T06:01:23.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.552+0000 7f937a546740 1 -- start start 2024-09-21T06:01:23.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.553+0000 7f937a546740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b1a7dc8070 0x55b1a816f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:23.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.553+0000 7f937a546740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1a7d9f1b0 0x55b1a816fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:23.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.553+0000 7f937a546740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1a7da7390 0x55b1a81700d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:23.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.553+0000 7f937a546740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b1a80e5e50 con 0x55b1a7da7390 2024-09-21T06:01:23.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.553+0000 7f937a546740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b1a80d5280 con 0x55b1a7d9f1b0 2024-09-21T06:01:23.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.553+0000 7f937a546740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b1a80d9090 con 0x55b1a7dc8070 2024-09-21T06:01:23.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.553+0000 7f9371d6e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1a7d9f1b0 0x55b1a816fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:23.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.553+0000 7f9371d6e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1a7d9f1b0 0x55b1a816fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:38508/0 (socket says 172.21.15.117:38508) 2024-09-21T06:01:23.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.553+0000 7f9371d6e640 1 -- 172.21.15.117:0/4206770424 learned_addr learned my addr 172.21.15.117:0/4206770424 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:01:23.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.553+0000 7f937156d640 1 --2- 172.21.15.117:0/4206770424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b1a7dc8070 0x55b1a816f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:23.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.553+0000 7f937156d640 1 --2- 172.21.15.117:0/4206770424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1a7da7390 0x55b1a81700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:23.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.553+0000 7f9371d6e640 1 -- 172.21.15.117:0/4206770424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b1a7dc8070 msgr2=0x55b1a816f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:23.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.553+0000 7f9371d6e640 1 --2- 172.21.15.117:0/4206770424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b1a7dc8070 0x55b1a816f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:23.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.553+0000 7f9371d6e640 1 -- 172.21.15.117:0/4206770424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1a7da7390 msgr2=0x55b1a81700d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:23.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.553+0000 7f9371d6e640 1 --2- 172.21.15.117:0/4206770424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1a7da7390 0x55b1a81700d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:23.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.553+0000 7f9371d6e640 1 -- 172.21.15.117:0/4206770424 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1a7e33650 con 0x55b1a7d9f1b0 2024-09-21T06:01:23.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.554+0000 7f9362ffd640 1 --2- 172.21.15.117:0/131735442 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f93580b6700 0x7f93580b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:23.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.554+0000 7f9371d6e640 1 --2- 172.21.15.117:0/4206770424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1a7d9f1b0 0x55b1a816fb90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f935c009c70 tx=0x7f935c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27195894c5a843d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:23.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.554+0000 7f937156d640 1 --2- 172.21.15.117:0/131735442 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f93580b6700 0x7f93580b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:23.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.554+0000 7f9362ffd640 1 -- 172.21.15.117:0/131735442 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f93640e72c0 con 0x7f936c10c020 2024-09-21T06:01:23.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.554+0000 7f9357fff640 1 -- 172.21.15.117:0/4206770424 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c01e070 con 0x55b1a7d9f1b0 2024-09-21T06:01:23.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.555+0000 7f9357fff640 1 -- 172.21.15.117:0/4206770424 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f935c009e20 con 0x55b1a7d9f1b0 2024-09-21T06:01:23.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.555+0000 7f9357fff640 1 -- 172.21.15.117:0/4206770424 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c018730 con 0x55b1a7d9f1b0 2024-09-21T06:01:23.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.556+0000 7f937156d640 1 --2- 172.21.15.117:0/131735442 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f93580b6700 0x7f93580b8c10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f93680057c0 tx=0x7f936802a340 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:23.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.556+0000 7f937a546740 1 -- 172.21.15.117:0/4206770424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1a7d9f1b0 msgr2=0x55b1a816fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:23.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.556+0000 7f937a546740 1 --2- 172.21.15.117:0/4206770424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1a7d9f1b0 0x55b1a816fb90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f935c009c70 tx=0x7f935c011810 comp rx=0 tx=0).stop 2024-09-21T06:01:23.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.556+0000 7f937a546740 1 -- 172.21.15.117:0/4206770424 shutdown_connections 2024-09-21T06:01:23.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.556+0000 7f937a546740 1 --2- 172.21.15.117:0/4206770424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1a7da7390 0x55b1a81700d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:23.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.556+0000 7f937a546740 1 --2- 172.21.15.117:0/4206770424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1a7d9f1b0 0x55b1a816fb90 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:23.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.556+0000 7f937a546740 1 --2- 172.21.15.117:0/4206770424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b1a7dc8070 0x55b1a816f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:23.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.556+0000 7f937a546740 1 -- 172.21.15.117:0/4206770424 >> 172.21.15.117:0/4206770424 conn(0x55b1a7dbc460 msgr2=0x55b1a816d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:23.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.556+0000 7f937a546740 1 -- 172.21.15.117:0/4206770424 shutdown_connections 2024-09-21T06:01:23.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.556+0000 7f937a546740 1 -- 172.21.15.117:0/4206770424 wait complete. 2024-09-21T06:01:23.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.557+0000 7f937a546740 1 -- start start 2024-09-21T06:01:23.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.557+0000 7f937a546740 1 Processor -- start 2024-09-21T06:01:23.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.558+0000 7f937a546740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1a7d9f1b0 0x55b1a817ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:23.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.558+0000 7f937a546740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1a7da7390 0x55b1a817b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:23.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.558+0000 7f937a546740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b1a7dc8070 0x55b1a81871b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:23.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.558+0000 7f937a546740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b1a7f36700 con 0x55b1a7da7390 2024-09-21T06:01:23.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.558+0000 7f937a546740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b1a8070110 con 0x55b1a7d9f1b0 2024-09-21T06:01:23.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.558+0000 7f937a546740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b1a817de30 con 0x55b1a7dc8070 2024-09-21T06:01:23.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.558+0000 7f937a546740 10 client.0 _ll_register_callbacks cb 0x55b1a8089990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:01:23.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.558+0000 7f937256f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b1a7dc8070 0x55b1a81871b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:23.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.558+0000 7f937256f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b1a7dc8070 0x55b1a81871b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:45992/0 (socket says 172.21.15.117:45992) 2024-09-21T06:01:23.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.558+0000 7f937256f640 1 -- 172.21.15.117:0/2921094397 learned_addr learned my addr 172.21.15.117:0/2921094397 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:01:23.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.558+0000 7f937156d640 1 --2- 172.21.15.117:0/2921094397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1a7da7390 0x55b1a817b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:23.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.558+0000 7f937256f640 1 -- 172.21.15.117:0/2921094397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1a7d9f1b0 msgr2=0x55b1a817ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:23.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.558+0000 7f9371d6e640 1 --2- 172.21.15.117:0/2921094397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1a7d9f1b0 0x55b1a817ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:23.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.558+0000 7f937256f640 1 --2- 172.21.15.117:0/2921094397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1a7d9f1b0 0x55b1a817ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:23.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.558+0000 7f937256f640 1 -- 172.21.15.117:0/2921094397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1a7da7390 msgr2=0x55b1a817b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:23.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.558+0000 7f937256f640 1 --2- 172.21.15.117:0/2921094397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1a7da7390 0x55b1a817b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:23.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.558+0000 7f937256f640 1 -- 172.21.15.117:0/2921094397 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1a809dc40 con 0x55b1a7dc8070 2024-09-21T06:01:23.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.558+0000 7f937156d640 1 --2- 172.21.15.117:0/2921094397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1a7da7390 0x55b1a817b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:01:23.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.559+0000 7f937256f640 1 --2- 172.21.15.117:0/2921094397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b1a7dc8070 0x55b1a81871b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f93640ace90 tx=0x7f93640b6ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4f9009b3b4d8cde server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:23.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.559+0000 7f93557fa640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-21T06:01:23.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.559+0000 7f93557fa640 1 -- 172.21.15.117:0/2921094397 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93640ac5f0 con 0x55b1a7dc8070 2024-09-21T06:01:23.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.559+0000 7f93557fa640 1 -- 172.21.15.117:0/2921094397 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f93640d0b10 con 0x55b1a7dc8070 2024-09-21T06:01:23.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.559+0000 7f937a546740 1 -- 172.21.15.117:0/2921094397 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f935c009e20 con 0x55b1a7dc8070 2024-09-21T06:01:23.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.559+0000 7f93557fa640 1 -- 172.21.15.117:0/2921094397 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f93640ad890 con 0x55b1a7dc8070 2024-09-21T06:01:23.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.559+0000 7f937a546740 10 client.25594 fetch_fsmap learned FSMap version 27 2024-09-21T06:01:23.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.559+0000 7f937a546740 1 -- 172.21.15.117:0/2921094397 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b1a8189ba0 con 0x55b1a7dc8070 2024-09-21T06:01:23.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.560+0000 7f93557fa640 1 -- 172.21.15.117:0/2921094397 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f935c009e20 con 0x55b1a7dc8070 2024-09-21T06:01:23.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.560+0000 7f93557fa640 1 -- 172.21.15.117:0/2921094397 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f936400b040 con 0x55b1a7dc8070 2024-09-21T06:01:23.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.560+0000 7f937a546740 10 client.25594 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:01:23.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.560+0000 7f937a546740 10 client.25594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:01:23.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.560+0000 7f937a546740 20 client.25594 populate_metadata read hostname 'smithi117' 2024-09-21T06:01:23.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.560+0000 7f937a546740 1 --2- 172.21.15.117:0/2921094397 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b1a7d869f0 0x55b1a818d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:23.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.560+0000 7f937a546740 4 client.25594 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:01:23.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.560+0000 7f937a546740 1 -- 172.21.15.117:0/2921094397 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1a80e7da0 con 0x55b1a7d869f0 2024-09-21T06:01:23.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.560+0000 7f9371d6e640 1 --2- 172.21.15.117:0/2921094397 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b1a7d869f0 0x55b1a818d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:23.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.561+0000 7f9371d6e640 1 --2- 172.21.15.117:0/2921094397 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b1a7d869f0 0x55b1a818d0c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8a40134cc0a4980 server_cookie=a6a393c6dfea02ff in_seq=0 out_seq=0 2024-09-21T06:01:23.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.561+0000 7f93557fa640 10 client.25594 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:01:23.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.563+0000 7f93557fa640 1 -- 172.21.15.117:0/2921094397 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b1a80e7da0 con 0x55b1a7d869f0 2024-09-21T06:01:23.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.563+0000 7f93557fa640 10 client.25594 handle_command_reply: tid=0 2024-09-21T06:01:23.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.563+0000 7f937a546740 1 client.25594 shutdown 2024-09-21T06:01:23.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.563+0000 7f937a546740 1 -- 172.21.15.117:0/2921094397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b1a7dc8070 msgr2=0x55b1a81871b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:23.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.563+0000 7f937a546740 1 --2- 172.21.15.117:0/2921094397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b1a7dc8070 0x55b1a81871b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f93640ace90 tx=0x7f93640b6ae0 comp rx=0 tx=0).stop 2024-09-21T06:01:23.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.564+0000 7f937a546740 1 -- 172.21.15.117:0/2921094397 shutdown_connections 2024-09-21T06:01:23.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.564+0000 7f937a546740 1 --2- 172.21.15.117:0/2921094397 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b1a7d869f0 0x55b1a818d0c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:23.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.564+0000 7f937a546740 1 --2- 172.21.15.117:0/2921094397 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1a7da7390 0x55b1a817b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:23.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.564+0000 7f93557fa640 0 client.25594 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:01:23.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.564+0000 7f937a546740 1 --2- 172.21.15.117:0/2921094397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b1a7dc8070 0x55b1a81871b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:23.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.564+0000 7f937a546740 1 --2- 172.21.15.117:0/2921094397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1a7d9f1b0 0x55b1a817ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:23.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.564+0000 7f937a546740 1 -- 172.21.15.117:0/2921094397 >> 172.21.15.117:0/2921094397 conn(0x55b1a7dbc460 msgr2=0x55b1a81727e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:23.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.564+0000 7f937a546740 1 -- 172.21.15.117:0/2921094397 shutdown_connections 2024-09-21T06:01:23.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.564+0000 7f937a546740 1 -- 172.21.15.117:0/2921094397 wait complete. 2024-09-21T06:01:23.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.564+0000 7f937a546740 20 client.25594 trim_cache size 0 max 16384 2024-09-21T06:01:23.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.594+0000 7f937a546740 1 Processor -- start 2024-09-21T06:01:23.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.594+0000 7f937a546740 1 -- start start 2024-09-21T06:01:23.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.594+0000 7f937a546740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1a7d869f0 0x55b1a817b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:23.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.594+0000 7f937a546740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b1a7d9f1b0 0x55b1a8172750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:23.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.594+0000 7f937a546740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1a7da7390 0x55b1a8172c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:23.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.594+0000 7f937a546740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f935000e580 con 0x55b1a7da7390 2024-09-21T06:01:23.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.594+0000 7f937a546740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f93500100c0 con 0x55b1a7d869f0 2024-09-21T06:01:23.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.594+0000 7f937a546740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f935000ec50 con 0x55b1a7d9f1b0 2024-09-21T06:01:23.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.594+0000 7f9371d6e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1a7d869f0 0x55b1a817b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:23.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.594+0000 7f937156d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b1a7d9f1b0 0x55b1a8172750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:23.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.594+0000 7f9371d6e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1a7d869f0 0x55b1a817b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:38518/0 (socket says 172.21.15.117:38518) 2024-09-21T06:01:23.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.594+0000 7f937156d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b1a7d9f1b0 0x55b1a8172750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:46000/0 (socket says 172.21.15.117:46000) 2024-09-21T06:01:23.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.594+0000 7f9371d6e640 1 -- 172.21.15.117:0/3116510049 learned_addr learned my addr 172.21.15.117:0/3116510049 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:01:23.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.595+0000 7f937256f640 1 --2- 172.21.15.117:0/3116510049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1a7da7390 0x55b1a8172c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:23.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.595+0000 7f9371d6e640 1 -- 172.21.15.117:0/3116510049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b1a7d9f1b0 msgr2=0x55b1a8172750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:23.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.595+0000 7f9371d6e640 1 --2- 172.21.15.117:0/3116510049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b1a7d9f1b0 0x55b1a8172750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:23.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.595+0000 7f9371d6e640 1 -- 172.21.15.117:0/3116510049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1a7da7390 msgr2=0x55b1a8172c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:23.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.595+0000 7f9371d6e640 1 --2- 172.21.15.117:0/3116510049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1a7da7390 0x55b1a8172c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:23.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.595+0000 7f9371d6e640 1 -- 172.21.15.117:0/3116510049 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1a8171940 con 0x55b1a7d869f0 2024-09-21T06:01:23.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.595+0000 7f937156d640 1 --2- 172.21.15.117:0/3116510049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b1a7d9f1b0 0x55b1a8172750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:01:23.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.595+0000 7f937256f640 1 --2- 172.21.15.117:0/3116510049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1a7da7390 0x55b1a8172c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:01:23.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.596+0000 7f9371d6e640 1 --2- 172.21.15.117:0/3116510049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1a7d869f0 0x55b1a817b200 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f935c009bc0 tx=0x7f935c009770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfcf7a4b3d91206 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:23.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.596+0000 7f93577fe640 1 -- 172.21.15.117:0/3116510049 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c033070 con 0x55b1a7d869f0 2024-09-21T06:01:23.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.596+0000 7f93577fe640 1 -- 172.21.15.117:0/3116510049 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f935c02a030 con 0x55b1a7d869f0 2024-09-21T06:01:23.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.596+0000 7f93577fe640 1 -- 172.21.15.117:0/3116510049 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c01e040 con 0x55b1a7d869f0 2024-09-21T06:01:23.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.597+0000 7f937a546740 1 -- 172.21.15.117:0/3116510049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1a7d869f0 msgr2=0x55b1a817b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:23.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.597+0000 7f937a546740 1 --2- 172.21.15.117:0/3116510049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1a7d869f0 0x55b1a817b200 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f935c009bc0 tx=0x7f935c009770 comp rx=0 tx=0).stop 2024-09-21T06:01:23.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.597+0000 7f937a546740 1 -- 172.21.15.117:0/3116510049 shutdown_connections 2024-09-21T06:01:23.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.597+0000 7f937a546740 1 --2- 172.21.15.117:0/3116510049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1a7da7390 0x55b1a8172c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:23.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.597+0000 7f937a546740 1 --2- 172.21.15.117:0/3116510049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b1a7d9f1b0 0x55b1a8172750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:23.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.597+0000 7f937a546740 1 --2- 172.21.15.117:0/3116510049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1a7d869f0 0x55b1a817b200 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:23.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.597+0000 7f937a546740 1 -- 172.21.15.117:0/3116510049 >> 172.21.15.117:0/3116510049 conn(0x55b1a7dbc460 msgr2=0x55b1a817aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:23.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.597+0000 7f937a546740 1 -- 172.21.15.117:0/3116510049 shutdown_connections 2024-09-21T06:01:23.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.598+0000 7f937a546740 1 -- 172.21.15.117:0/3116510049 wait complete. 2024-09-21T06:01:23.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.598+0000 7f937a546740 1 -- start start 2024-09-21T06:01:23.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.598+0000 7f937a546740 1 Processor -- start 2024-09-21T06:01:23.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.599+0000 7f937a546740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b1a7d869f0 0x55b1a817f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:23.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.599+0000 7f937a546740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1a7d9f1b0 0x55b1a817f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:23.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.599+0000 7f937a546740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1a7da7390 0x55b1a817fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:23.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.599+0000 7f9371d6e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b1a7d869f0 0x55b1a817f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:23.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.599+0000 7f937a546740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f935000f9f0 con 0x55b1a7da7390 2024-09-21T06:01:23.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.599+0000 7f937a546740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f935000f320 con 0x55b1a7d9f1b0 2024-09-21T06:01:23.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.599+0000 7f937a546740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f935000c560 con 0x55b1a7d869f0 2024-09-21T06:01:23.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.599+0000 7f9371d6e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b1a7d869f0 0x55b1a817f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:46012/0 (socket says 172.21.15.117:46012) 2024-09-21T06:01:23.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.599+0000 7f937156d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1a7d9f1b0 0x55b1a817f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:23.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.599+0000 7f9371d6e640 1 -- 172.21.15.117:0/2501972765 learned_addr learned my addr 172.21.15.117:0/2501972765 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:01:23.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.599+0000 7f937256f640 1 --2- 172.21.15.117:0/2501972765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1a7da7390 0x55b1a817fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:23.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.599+0000 7f937a546740 10 client.0 _ll_register_callbacks cb 0x7f9364009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:01:23.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.599+0000 7f9371d6e640 1 -- 172.21.15.117:0/2501972765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1a7d9f1b0 msgr2=0x55b1a817f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:23.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.599+0000 7f9371d6e640 1 --2- 172.21.15.117:0/2501972765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1a7d9f1b0 0x55b1a817f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:23.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.600+0000 7f9371d6e640 1 -- 172.21.15.117:0/2501972765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1a7da7390 msgr2=0x55b1a817fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:23.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.600+0000 7f9371d6e640 1 --2- 172.21.15.117:0/2501972765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1a7da7390 0x55b1a817fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:23.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.600+0000 7f9371d6e640 1 -- 172.21.15.117:0/2501972765 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1a81f6c80 con 0x55b1a7d869f0 2024-09-21T06:01:23.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.600+0000 7f937156d640 1 --2- 172.21.15.117:0/2501972765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1a7d9f1b0 0x55b1a817f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:01:23.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.600+0000 7f937256f640 1 --2- 172.21.15.117:0/2501972765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1a7da7390 0x55b1a817fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:01:23.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.600+0000 7f9371d6e640 1 --2- 172.21.15.117:0/2501972765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b1a7d869f0 0x55b1a817f2e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f935c000c00 tx=0x7f935c0098e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed3a031bc25fbdeb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:23.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.600+0000 7f93557fa640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-21T06:01:23.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.600+0000 7f93557fa640 1 -- 172.21.15.117:0/2501972765 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c03c060 con 0x55b1a7d869f0 2024-09-21T06:01:23.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.600+0000 7f93557fa640 1 -- 172.21.15.117:0/2501972765 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f935c005380 con 0x55b1a7d869f0 2024-09-21T06:01:23.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.600+0000 7f937a546740 1 -- 172.21.15.117:0/2501972765 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f935c02a030 con 0x55b1a7d869f0 2024-09-21T06:01:23.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.601+0000 7f93557fa640 1 -- 172.21.15.117:0/2501972765 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f935c01e070 con 0x55b1a7d869f0 2024-09-21T06:01:23.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.601+0000 7f937a546740 10 client.25600 fetch_fsmap learned FSMap version 27 2024-09-21T06:01:23.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.601+0000 7f937a546740 1 -- 172.21.15.117:0/2501972765 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b1a81f7f10 con 0x55b1a7d869f0 2024-09-21T06:01:23.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.601+0000 7f93557fa640 1 -- 172.21.15.117:0/2501972765 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f935c02a030 con 0x55b1a7d869f0 2024-09-21T06:01:23.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.601+0000 7f93557fa640 1 -- 172.21.15.117:0/2501972765 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f935c004040 con 0x55b1a7d869f0 2024-09-21T06:01:23.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.601+0000 7f937a546740 10 client.25600 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:01:23.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.601+0000 7f937a546740 10 client.25600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:01:23.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.601+0000 7f937a546740 20 client.25600 populate_metadata read hostname 'smithi117' 2024-09-21T06:01:23.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.601+0000 7f937a546740 1 --2- 172.21.15.117:0/2501972765 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b1a7dc8070 0x55b1a81fa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:23.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.601+0000 7f937a546740 4 client.25600 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:01:23.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.601+0000 7f937a546740 1 -- 172.21.15.117:0/2501972765 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b1a80e3850 con 0x55b1a7dc8070 2024-09-21T06:01:23.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.602+0000 7f937156d640 1 --2- 172.21.15.117:0/2501972765 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b1a7dc8070 0x55b1a81fa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:23.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.602+0000 7f937156d640 1 --2- 172.21.15.117:0/2501972765 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b1a7dc8070 0x55b1a81fa3b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fee9650b1e3fd5b server_cookie=fa40ee5790bf7ea0 in_seq=0 out_seq=0 2024-09-21T06:01:23.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.602+0000 7f93557fa640 10 client.25600 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:01:23.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.604+0000 7f93557fa640 1 -- 172.21.15.117:0/2501972765 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1428 (crc 0 0 0) 0x55b1a80e3850 con 0x55b1a7dc8070 2024-09-21T06:01:23.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.604+0000 7f93557fa640 10 client.25600 handle_command_reply: tid=0 2024-09-21T06:01:23.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.604+0000 7f937a546740 1 client.25600 shutdown 2024-09-21T06:01:23.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.604+0000 7f937a546740 1 -- 172.21.15.117:0/2501972765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b1a7d869f0 msgr2=0x55b1a817f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:23.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.604+0000 7f937a546740 1 --2- 172.21.15.117:0/2501972765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b1a7d869f0 0x55b1a817f2e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f935c000c00 tx=0x7f935c0098e0 comp rx=0 tx=0).stop 2024-09-21T06:01:23.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.604+0000 7f937a546740 1 -- 172.21.15.117:0/2501972765 shutdown_connections 2024-09-21T06:01:23.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.604+0000 7f937a546740 1 --2- 172.21.15.117:0/2501972765 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b1a7dc8070 0x55b1a81fa3b0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:23.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.604+0000 7f937a546740 1 --2- 172.21.15.117:0/2501972765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b1a7da7390 0x55b1a817fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:23.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.604+0000 7f93557fa640 0 client.25600 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:01:23.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.604+0000 7f937a546740 1 --2- 172.21.15.117:0/2501972765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b1a7d9f1b0 0x55b1a817f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:23.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.604+0000 7f937a546740 1 --2- 172.21.15.117:0/2501972765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b1a7d869f0 0x55b1a817f2e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:23.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.604+0000 7f937a546740 1 -- 172.21.15.117:0/2501972765 >> 172.21.15.117:0/2501972765 conn(0x55b1a7dbc460 msgr2=0x55b1a8172750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:23.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.604+0000 7f937a546740 1 -- 172.21.15.117:0/2501972765 shutdown_connections 2024-09-21T06:01:23.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.604+0000 7f937a546740 1 -- 172.21.15.117:0/2501972765 wait complete. 2024-09-21T06:01:23.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.604+0000 7f937a546740 20 client.25600 trim_cache size 0 max 16384 2024-09-21T06:01:23.619 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T06:01:23.619 INFO:teuthology.orchestra.run.smithi117.stdout: "status": "scrub active (0 inodes in the stack)", 2024-09-21T06:01:23.619 INFO:teuthology.orchestra.run.smithi117.stdout: "scrubs": { 2024-09-21T06:01:23.619 INFO:teuthology.orchestra.run.smithi117.stdout: "d543b0ae-b710-4153-99da-fe5714839c1a": { 2024-09-21T06:01:23.619 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:01:23.619 INFO:teuthology.orchestra.run.smithi117.stdout: "tag": "d543b0ae-b710-4153-99da-fe5714839c1a", 2024-09-21T06:01:23.619 INFO:teuthology.orchestra.run.smithi117.stdout: "options": "recursive,force" 2024-09-21T06:01:23.620 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:01:23.620 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:01:23.620 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_stats": { 2024-09-21T06:01:23.620 INFO:teuthology.orchestra.run.smithi117.stdout: "d543b0ae-b710-4153-99da-fe5714839c1a": { 2024-09-21T06:01:23.620 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:01:20.574754+0000", 2024-09-21T06:01:23.620 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:01:23.620 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:01:23.620 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:01:23.620 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:01:23.620 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:01:23.620 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:01:23.620 INFO:teuthology.orchestra.run.smithi117.stdout: "ad80d9f1-6aff-46c0-91ab-2c1ab0681039": { 2024-09-21T06:01:23.620 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:59:31.949108+0000", 2024-09-21T06:01:23.620 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:01:23.621 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:01:23.621 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:01:23.621 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:01:23.621 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:01:23.621 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:01:23.621 INFO:teuthology.orchestra.run.smithi117.stdout: "8b3fe95a-b76b-4e94-afd0-72fe35e2dd92": { 2024-09-21T06:01:23.621 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:58:01.246076+0000", 2024-09-21T06:01:23.621 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:01:23.621 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:01:23.621 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:01:23.621 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:01:23.621 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:01:23.621 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:01:23.621 INFO:teuthology.orchestra.run.smithi117.stdout: "b30bfff0-0f6b-4e30-8467-ff7db421eba6": { 2024-09-21T06:01:23.621 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:57:14.939082+0000", 2024-09-21T06:01:23.622 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:01:23.622 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:01:23.622 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:01:23.622 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:01:23.622 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:01:23.622 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:01:23.622 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:01:23.622 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T06:01:23.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.605+0000 7f9373fd2640 1 -- 172.21.15.117:0/131735442 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f93580b6700 msgr2=0x7f93580b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:23.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.605+0000 7f9373fd2640 1 --2- 172.21.15.117:0/131735442 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f93580b6700 0x7f93580b8c10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f93680057c0 tx=0x7f936802a340 comp rx=0 tx=0).stop 2024-09-21T06:01:23.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.605+0000 7f9373fd2640 1 -- 172.21.15.117:0/131735442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f936c10c020 msgr2=0x7f936c1107c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:23.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.605+0000 7f9373fd2640 1 --2- 172.21.15.117:0/131735442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f936c10c020 0x7f936c1107c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f936400efc0 tx=0x7f936400c800 comp rx=0 tx=0).stop 2024-09-21T06:01:23.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.606+0000 7f9373fd2640 1 -- 172.21.15.117:0/131735442 shutdown_connections 2024-09-21T06:01:23.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.606+0000 7f9373fd2640 1 --2- 172.21.15.117:0/131735442 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f93580b6700 0x7f93580b8c10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:23.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.606+0000 7f9373fd2640 1 --2- 172.21.15.117:0/131735442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f936c10c020 0x7f936c1107c0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:23.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.606+0000 7f9373fd2640 1 --2- 172.21.15.117:0/131735442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f936c10b6d0 0x7f936c110280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:23.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.606+0000 7f9373fd2640 1 --2- 172.21.15.117:0/131735442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f936c105d90 0x7f936c10fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:23.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.606+0000 7f9373fd2640 1 -- 172.21.15.117:0/131735442 >> 172.21.15.117:0/131735442 conn(0x7f936c1017c0 msgr2=0x7f936c108d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:23.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.606+0000 7f9373fd2640 1 -- 172.21.15.117:0/131735442 shutdown_connections 2024-09-21T06:01:23.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:01:23.606+0000 7f9373fd2640 1 -- 172.21.15.117:0/131735442 wait complete. 2024-09-21T06:01:24.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:23 smithi117 ceph-mon[30947]: pgmap v449: 97 pgs: 97 active+clean; 37 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 809 MiB/s wr, 3.09k op/s 2024-09-21T06:01:24.097 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:24.095+0000 7f7fcaff5640 1 -- 172.21.15.117:0/871250578 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56226f6200f0 con 0x56226f29c1c0 2024-09-21T06:01:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:24.095+0000 7f7fcaff5640 10 client.25567 handle_command_reply: tid=0 2024-09-21T06:01:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:24.095+0000 7f7ff41bb740 1 client.25567 shutdown 2024-09-21T06:01:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:24.095+0000 7f7ff41bb740 1 -- 172.21.15.117:0/871250578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56226f3081b0 msgr2=0x56226f6bf120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:24.095+0000 7f7ff41bb740 1 --2- 172.21.15.117:0/871250578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56226f3081b0 0x56226f6bf120 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc02aed0 tx=0x7f7fdc000e40 comp rx=0 tx=0).stop 2024-09-21T06:01:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:24.095+0000 7f7ff41bb740 1 -- 172.21.15.117:0/871250578 shutdown_connections 2024-09-21T06:01:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:24.095+0000 7f7ff41bb740 1 --2- 172.21.15.117:0/871250578 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56226f29c1c0 0x56226f748ba0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:24.095+0000 7f7ff41bb740 1 --2- 172.21.15.117:0/871250578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56226f2e74d0 0x56226f6bebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:24.095+0000 7f7ff41bb740 1 --2- 172.21.15.117:0/871250578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56226f3081b0 0x56226f6bf120 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:24.095+0000 7f7ff41bb740 1 --2- 172.21.15.117:0/871250578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56226f2df2f0 0x56226f6be6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:24.095+0000 7f7ff41bb740 1 -- 172.21.15.117:0/871250578 >> 172.21.15.117:0/871250578 conn(0x56226f2fc300 msgr2=0x56226f6ba6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:24.095+0000 7f7fcaff5640 0 client.25567 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:01:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:24.096+0000 7f7ff41bb740 1 -- 172.21.15.117:0/871250578 shutdown_connections 2024-09-21T06:01:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:24.096+0000 7f7ff41bb740 1 -- 172.21.15.117:0/871250578 wait complete. 2024-09-21T06:01:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:24.096+0000 7f7ff41bb740 20 client.25567 trim_cache size 0 max 16384 2024-09-21T06:01:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T06:01:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T06:01:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T06:01:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T06:01:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:24.096+0000 7f7fed24d640 1 -- 172.21.15.117:0/1174768940 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7fd00b6850 msgr2=0x7f7fd00b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:24.096+0000 7f7fed24d640 1 --2- 172.21.15.117:0/1174768940 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7fd00b6850 0x7f7fd00b8d60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0003fd0 tx=0x7f7fe0005fd0 comp rx=0 tx=0).stop 2024-09-21T06:01:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:24.096+0000 7f7fed24d640 1 -- 172.21.15.117:0/1174768940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fe810eab0 msgr2=0x7f7fe81a5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:24.096+0000 7f7fed24d640 1 --2- 172.21.15.117:0/1174768940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fe810eab0 0x7f7fe81a5fa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7fe002ae10 tx=0x7f7fe0009780 comp rx=0 tx=0).stop 2024-09-21T06:01:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:24.097+0000 7f7fed24d640 1 -- 172.21.15.117:0/1174768940 shutdown_connections 2024-09-21T06:01:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:24.097+0000 7f7fed24d640 1 --2- 172.21.15.117:0/1174768940 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7fd00b6850 0x7f7fd00b8d60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:24.097+0000 7f7fed24d640 1 --2- 172.21.15.117:0/1174768940 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fe810ab50 0x7f7fe81a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:24.101 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:24.097+0000 7f7fed24d640 1 --2- 172.21.15.117:0/1174768940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fe810eab0 0x7f7fe81a5fa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:24.101 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:24.097+0000 7f7fed24d640 1 --2- 172.21.15.117:0/1174768940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fe810a200 0x7f7fe81a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:24.101 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:24.097+0000 7f7fed24d640 1 -- 172.21.15.117:0/1174768940 >> 172.21.15.117:0/1174768940 conn(0x7f7fe8079400 msgr2=0x7f7fe810b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:24.101 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:24.097+0000 7f7fed24d640 1 -- 172.21.15.117:0/1174768940 shutdown_connections 2024-09-21T06:01:24.101 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:24.097+0000 7f7fed24d640 1 -- 172.21.15.117:0/1174768940 wait complete. 2024-09-21T06:01:24.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:23 smithi138 ceph-mon[34391]: pgmap v449: 97 pgs: 97 active+clean; 37 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 809 MiB/s wr, 3.09k op/s 2024-09-21T06:01:24.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:23 smithi192 ceph-mon[34700]: pgmap v449: 97 pgs: 97 active+clean; 37 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 809 MiB/s wr, 3.09k op/s 2024-09-21T06:01:25.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:24 smithi138 ceph-mon[34391]: scrub summary: idle+waiting paths [/] 2024-09-21T06:01:25.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:24 smithi192 ceph-mon[34700]: scrub summary: idle+waiting paths [/] 2024-09-21T06:01:25.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:24 smithi117 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-21T06:01:26.097 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:25 smithi117 ceph-mon[30947]: scrub summary: idle 2024-09-21T06:01:26.097 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:25 smithi117 ceph-mon[30947]: pgmap v450: 97 pgs: 97 active+clean; 38 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 826 MiB/s wr, 2.90k op/s 2024-09-21T06:01:26.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:25 smithi138 ceph-mon[34391]: scrub summary: idle 2024-09-21T06:01:26.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:25 smithi138 ceph-mon[34391]: pgmap v450: 97 pgs: 97 active+clean; 38 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 826 MiB/s wr, 2.90k op/s 2024-09-21T06:01:26.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:25 smithi192 ceph-mon[34700]: scrub summary: idle 2024-09-21T06:01:26.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:25 smithi192 ceph-mon[34700]: pgmap v450: 97 pgs: 97 active+clean; 38 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 826 MiB/s wr, 2.90k op/s 2024-09-21T06:01:28.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:27 smithi138 ceph-mon[34391]: pgmap v451: 97 pgs: 97 active+clean; 38 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 730 MiB/s wr, 2.42k op/s 2024-09-21T06:01:28.190 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:27 smithi117 ceph-mon[30947]: pgmap v451: 97 pgs: 97 active+clean; 38 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 730 MiB/s wr, 2.42k op/s 2024-09-21T06:01:28.232 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:27 smithi192 ceph-mon[34700]: pgmap v451: 97 pgs: 97 active+clean; 38 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 730 MiB/s wr, 2.42k op/s 2024-09-21T06:01:28.269 INFO:tasks.cephfs.filesystem:scrub status for tag:d543b0ae-b710-4153-99da-fe5714839c1a - {'path': '/', 'tag': 'd543b0ae-b710-4153-99da-fe5714839c1a', 'options': 'recursive,force'} 2024-09-21T06:01:29.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:28 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:01:29.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:28 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:01:29.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:28 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:01:30.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:29 smithi192 ceph-mon[34700]: pgmap v452: 97 pgs: 97 active+clean; 38 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 738 MiB/s wr, 2.12k op/s 2024-09-21T06:01:30.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:29 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:01:30.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:29 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:01:30.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:29 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:01:30.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:29 smithi117 ceph-mon[30947]: pgmap v452: 97 pgs: 97 active+clean; 38 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 738 MiB/s wr, 2.12k op/s 2024-09-21T06:01:30.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:29 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:01:30.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:29 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:01:30.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:29 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:01:30.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:29 smithi138 ceph-mon[34391]: pgmap v452: 97 pgs: 97 active+clean; 38 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 738 MiB/s wr, 2.12k op/s 2024-09-21T06:01:30.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:29 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:01:30.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:29 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:01:30.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:29 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:01:31.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:30 smithi192 ceph-mon[34700]: pgmap v453: 97 pgs: 97 active+clean; 39 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 663 MiB/s wr, 1.54k op/s 2024-09-21T06:01:31.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:30 smithi117 ceph-mon[30947]: pgmap v453: 97 pgs: 97 active+clean; 39 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 663 MiB/s wr, 1.54k op/s 2024-09-21T06:01:31.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:30 smithi138 ceph-mon[34391]: pgmap v453: 97 pgs: 97 active+clean; 39 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 663 MiB/s wr, 1.54k op/s 2024-09-21T06:01:34.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:33 smithi117 ceph-mon[30947]: pgmap v454: 97 pgs: 97 active+clean; 39 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 740 MiB/s wr, 1.67k op/s 2024-09-21T06:01:34.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:33 smithi138 ceph-mon[34391]: pgmap v454: 97 pgs: 97 active+clean; 39 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 740 MiB/s wr, 1.67k op/s 2024-09-21T06:01:34.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:33 smithi192 ceph-mon[34700]: pgmap v454: 97 pgs: 97 active+clean; 39 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 740 MiB/s wr, 1.67k op/s 2024-09-21T06:01:36.031 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:35 smithi117 ceph-mon[30947]: pgmap v455: 97 pgs: 97 active+clean; 40 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 767 MiB/s wr, 1.93k op/s 2024-09-21T06:01:36.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:35 smithi138 ceph-mon[34391]: pgmap v455: 97 pgs: 97 active+clean; 40 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 767 MiB/s wr, 1.93k op/s 2024-09-21T06:01:36.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:35 smithi192 ceph-mon[34700]: pgmap v455: 97 pgs: 97 active+clean; 40 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 767 MiB/s wr, 1.93k op/s 2024-09-21T06:01:38.029 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:37 smithi117 ceph-mon[30947]: pgmap v456: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 759 MiB/s wr, 1.91k op/s 2024-09-21T06:01:38.097 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.094+0000 7f3e07052640 1 -- 172.21.15.117:0/2225324054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3df809b2f0 msgr2=0x7f3df80a1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:38.097 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.094+0000 7f3e07052640 1 --2- 172.21.15.117:0/2225324054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3df809b2f0 0x7f3df80a1b20 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3e00066480 tx=0x7f3e000980a0 comp rx=0 tx=0).stop 2024-09-21T06:01:38.097 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.094+0000 7f3e07052640 1 -- 172.21.15.117:0/2225324054 shutdown_connections 2024-09-21T06:01:38.097 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.094+0000 7f3e07052640 1 --2- 172.21.15.117:0/2225324054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3df809b2f0 0x7f3df80a1b20 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:38.097 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.094+0000 7f3e07052640 1 --2- 172.21.15.117:0/2225324054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3df809a9a0 0x7f3df809adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:38.097 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.094+0000 7f3e07052640 1 --2- 172.21.15.117:0/2225324054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3df8095070 0x7f3df8095480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:38.097 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.094+0000 7f3e07052640 1 -- 172.21.15.117:0/2225324054 >> 172.21.15.117:0/2225324054 conn(0x7f3df800cd80 msgr2=0x7f3df800d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:38.097 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.094+0000 7f3e07052640 1 -- 172.21.15.117:0/2225324054 shutdown_connections 2024-09-21T06:01:38.097 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.095+0000 7f3e07052640 1 -- 172.21.15.117:0/2225324054 wait complete. 2024-09-21T06:01:38.098 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.096+0000 7f3e07052640 1 Processor -- start 2024-09-21T06:01:38.098 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.096+0000 7f3e07052640 1 -- start start 2024-09-21T06:01:38.099 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.096+0000 7f3e07052640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3df8095070 0x7f3df809e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:38.099 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.096+0000 7f3e07052640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3df809a9a0 0x7f3df809eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:38.099 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.096+0000 7f3e07052640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3df809b2f0 0x7f3df813eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:38.099 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.096+0000 7f3e07052640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3df80a3ea0 con 0x7f3df809b2f0 2024-09-21T06:01:38.099 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.096+0000 7f3e07052640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3df80a3d20 con 0x7f3df8095070 2024-09-21T06:01:38.099 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.096+0000 7f3e07052640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3df80a4020 con 0x7f3df809a9a0 2024-09-21T06:01:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.097+0000 7f3e06851640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3df809b2f0 0x7f3df813eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.097+0000 7f3e06851640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3df809b2f0 0x7f3df813eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56536/0 (socket says 172.21.15.117:56536) 2024-09-21T06:01:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.097+0000 7f3e06851640 1 -- 172.21.15.117:0/3171721015 learned_addr learned my addr 172.21.15.117:0/3171721015 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:01:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.097+0000 7f3e0584f640 1 --2- 172.21.15.117:0/3171721015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3df809a9a0 0x7f3df809eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.098+0000 7f3e06050640 1 --2- 172.21.15.117:0/3171721015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3df8095070 0x7f3df809e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.098+0000 7f3e06851640 1 -- 172.21.15.117:0/3171721015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3df809a9a0 msgr2=0x7f3df809eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.098+0000 7f3e06851640 1 --2- 172.21.15.117:0/3171721015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3df809a9a0 0x7f3df809eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.098+0000 7f3e06851640 1 -- 172.21.15.117:0/3171721015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3df8095070 msgr2=0x7f3df809e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.098+0000 7f3e06851640 1 --2- 172.21.15.117:0/3171721015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3df8095070 0x7f3df809e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.098+0000 7f3e06851640 1 -- 172.21.15.117:0/3171721015 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df813f0d0 con 0x7f3df809b2f0 2024-09-21T06:01:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.098+0000 7f3e0584f640 1 --2- 172.21.15.117:0/3171721015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3df809a9a0 0x7f3df809eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:01:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.099+0000 7f3e06851640 1 --2- 172.21.15.117:0/3171721015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3df809b2f0 0x7f3df813eb40 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3e00000b60 tx=0x7f3e000a0ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.099+0000 7f3df77fe640 1 -- 172.21.15.117:0/3171721015 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e000af070 con 0x7f3df809b2f0 2024-09-21T06:01:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.099+0000 7f3df77fe640 1 -- 172.21.15.117:0/3171721015 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e000a1b30 con 0x7f3df809b2f0 2024-09-21T06:01:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.099+0000 7f3df77fe640 1 -- 172.21.15.117:0/3171721015 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e000aaa30 con 0x7f3df809b2f0 2024-09-21T06:01:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.099+0000 7f3e07052640 1 -- 172.21.15.117:0/3171721015 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3df813f360 con 0x7f3df809b2f0 2024-09-21T06:01:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.099+0000 7f3e07052640 1 -- 172.21.15.117:0/3171721015 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3df813f7f0 con 0x7f3df809b2f0 2024-09-21T06:01:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.104+0000 7f3df77fe640 1 -- 172.21.15.117:0/3171721015 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f3e000b2070 con 0x7f3df809b2f0 2024-09-21T06:01:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.105+0000 7f3df77fe640 1 --2- 172.21.15.117:0/3171721015 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3dec0b6730 0x7f3dec0b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.106+0000 7f3df77fe640 1 -- 172.21.15.117:0/3171721015 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f3e00174970 con 0x7f3df809b2f0 2024-09-21T06:01:38.108 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.106+0000 7f3e06050640 1 --2- 172.21.15.117:0/3171721015 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3dec0b6730 0x7f3dec0b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.108+0000 7f3e06050640 1 --2- 172.21.15.117:0/3171721015 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3dec0b6730 0x7f3dec0b8c40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3df80a1e10 tx=0x7f3dfc008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:38.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:37 smithi138 ceph-mon[34391]: pgmap v456: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 759 MiB/s wr, 1.91k op/s 2024-09-21T06:01:38.211 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.209+0000 7f3e0f227740 1 Processor -- start 2024-09-21T06:01:38.212 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.210+0000 7f3e0f227740 1 -- start start 2024-09-21T06:01:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.210+0000 7f3e0f227740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfcc0d21b0 0x55cfcc477ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.210+0000 7f3e0f227740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cfcc0a92f0 0x55cfcc4780e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.211+0000 7f3e0f227740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cfcc0b14d0 0x55cfcc478620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.211+0000 7f3e0f227740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cfcc3ec6f0 con 0x55cfcc0d21b0 2024-09-21T06:01:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.211+0000 7f3e0f227740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cfcc3dbb20 con 0x55cfcc0b14d0 2024-09-21T06:01:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.211+0000 7f3e0f227740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55cfcc3df930 con 0x55cfcc0a92f0 2024-09-21T06:01:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.211+0000 7f3e06050640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfcc0d21b0 0x55cfcc477ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.211+0000 7f3e06050640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfcc0d21b0 0x55cfcc477ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56538/0 (socket says 172.21.15.117:56538) 2024-09-21T06:01:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.211+0000 7f3e06050640 1 -- 172.21.15.117:0/2534898669 learned_addr learned my addr 172.21.15.117:0/2534898669 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:01:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.211+0000 7f3e06050640 1 -- 172.21.15.117:0/2534898669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cfcc0a92f0 msgr2=0x55cfcc4780e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.211+0000 7f3e06050640 1 --2- 172.21.15.117:0/2534898669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cfcc0a92f0 0x55cfcc4780e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.211+0000 7f3e06050640 1 -- 172.21.15.117:0/2534898669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cfcc0b14d0 msgr2=0x55cfcc478620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.211+0000 7f3e06050640 1 --2- 172.21.15.117:0/2534898669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cfcc0b14d0 0x55cfcc478620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.211+0000 7f3e06050640 1 -- 172.21.15.117:0/2534898669 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfcc3a1530 con 0x55cfcc0d21b0 2024-09-21T06:01:38.214 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.212+0000 7f3e06050640 1 --2- 172.21.15.117:0/2534898669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfcc0d21b0 0x55cfcc477ba0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc00ca60 tx=0x7f3dfc00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92170a87050e816b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.212+0000 7f3df4bfe640 1 -- 172.21.15.117:0/2534898669 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc023070 con 0x55cfcc0d21b0 2024-09-21T06:01:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.212+0000 7f3df4bfe640 1 -- 172.21.15.117:0/2534898669 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dfc013db0 con 0x55cfcc0d21b0 2024-09-21T06:01:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.212+0000 7f3df4bfe640 1 -- 172.21.15.117:0/2534898669 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc011660 con 0x55cfcc0d21b0 2024-09-21T06:01:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.214+0000 7f3e0f227740 1 -- 172.21.15.117:0/2534898669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfcc0d21b0 msgr2=0x55cfcc477ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:38.216 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.214+0000 7f3e0f227740 1 --2- 172.21.15.117:0/2534898669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfcc0d21b0 0x55cfcc477ba0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc00ca60 tx=0x7f3dfc00f000 comp rx=0 tx=0).stop 2024-09-21T06:01:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.214+0000 7f3e0f227740 1 -- 172.21.15.117:0/2534898669 shutdown_connections 2024-09-21T06:01:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.215+0000 7f3e0f227740 1 --2- 172.21.15.117:0/2534898669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cfcc0b14d0 0x55cfcc478620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.215+0000 7f3e0f227740 1 --2- 172.21.15.117:0/2534898669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cfcc0a92f0 0x55cfcc4780e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.215+0000 7f3e0f227740 1 --2- 172.21.15.117:0/2534898669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfcc0d21b0 0x55cfcc477ba0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.215+0000 7f3e0f227740 1 -- 172.21.15.117:0/2534898669 >> 172.21.15.117:0/2534898669 conn(0x55cfcc0c6300 msgr2=0x55cfcc475a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.215+0000 7f3e0f227740 1 -- 172.21.15.117:0/2534898669 shutdown_connections 2024-09-21T06:01:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.216+0000 7f3e0f227740 1 -- 172.21.15.117:0/2534898669 wait complete. 2024-09-21T06:01:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.217+0000 7f3e0f227740 1 -- start start 2024-09-21T06:01:38.219 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.217+0000 7f3e0f227740 1 Processor -- start 2024-09-21T06:01:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.218+0000 7f3e0f227740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cfcc0a92f0 0x55cfcc4832c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.219+0000 7f3e0f227740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfcc0b14d0 0x55cfcc483800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.219+0000 7f3e0f227740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cfcc0d21b0 0x55cfcc48f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.219+0000 7f3e0584f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfcc0b14d0 0x55cfcc483800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.219+0000 7f3e0584f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfcc0b14d0 0x55cfcc483800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56546/0 (socket says 172.21.15.117:56546) 2024-09-21T06:01:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.219+0000 7f3e0f227740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cfcc313ab0 con 0x55cfcc0b14d0 2024-09-21T06:01:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.219+0000 7f3e0f227740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cfcc381730 con 0x55cfcc0d21b0 2024-09-21T06:01:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.219+0000 7f3e0f227740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55cfcc486320 con 0x55cfcc0a92f0 2024-09-21T06:01:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.219+0000 7f3e0584f640 1 -- 172.21.15.117:0/2598127393 learned_addr learned my addr 172.21.15.117:0/2598127393 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:01:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.220+0000 7f3e0f227740 10 client.0 _ll_register_callbacks cb 0x55cfcc3d56b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:01:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.220+0000 7f3e0584f640 1 -- 172.21.15.117:0/2598127393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cfcc0a92f0 msgr2=0x55cfcc4832c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.220+0000 7f3e0584f640 1 --2- 172.21.15.117:0/2598127393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cfcc0a92f0 0x55cfcc4832c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:38.223 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.220+0000 7f3e0584f640 1 -- 172.21.15.117:0/2598127393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cfcc0d21b0 msgr2=0x55cfcc48f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:38.223 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.220+0000 7f3e0584f640 1 --2- 172.21.15.117:0/2598127393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cfcc0d21b0 0x55cfcc48f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:38.223 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.220+0000 7f3e0584f640 1 -- 172.21.15.117:0/2598127393 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfcc13d790 con 0x55cfcc0b14d0 2024-09-21T06:01:38.223 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.220+0000 7f3e0584f640 1 --2- 172.21.15.117:0/2598127393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfcc0b14d0 0x55cfcc483800 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f3df000e7c0 tx=0x7f3df000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73d5c573dffeee2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:38.223 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.220+0000 7f3de9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:01:38.224 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.221+0000 7f3de9ffb640 1 -- 172.21.15.117:0/2598127393 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df0017070 con 0x55cfcc0b14d0 2024-09-21T06:01:38.224 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.221+0000 7f3de9ffb640 1 -- 172.21.15.117:0/2598127393 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f3df000c840 con 0x55cfcc0b14d0 2024-09-21T06:01:38.224 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.221+0000 7f3e0f227740 1 -- 172.21.15.117:0/2598127393 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3dfc013db0 con 0x55cfcc0b14d0 2024-09-21T06:01:38.224 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.222+0000 7f3de9ffb640 1 -- 172.21.15.117:0/2598127393 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3df0021050 con 0x55cfcc0b14d0 2024-09-21T06:01:38.224 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.222+0000 7f3e0f227740 10 client.16272 fetch_fsmap learned FSMap version 27 2024-09-21T06:01:38.224 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.222+0000 7f3e0f227740 1 -- 172.21.15.117:0/2598127393 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cfcc1b6620 con 0x55cfcc0b14d0 2024-09-21T06:01:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.223+0000 7f3de9ffb640 1 -- 172.21.15.117:0/2598127393 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dfc013db0 con 0x55cfcc0b14d0 2024-09-21T06:01:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.223+0000 7f3de9ffb640 1 -- 172.21.15.117:0/2598127393 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3df0013b70 con 0x55cfcc0b14d0 2024-09-21T06:01:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.223+0000 7f3e0f227740 10 client.16272 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:01:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.223+0000 7f3e0f227740 10 client.16272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:01:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.223+0000 7f3e0f227740 20 client.16272 populate_metadata read hostname 'smithi117' 2024-09-21T06:01:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.223+0000 7f3e0f227740 1 --2- 172.21.15.117:0/2598127393 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55cfcc090d80 0x55cfcc495680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.223+0000 7f3e0f227740 4 client.16272 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:01:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.223+0000 7f3e0f227740 1 -- 172.21.15.117:0/2598127393 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfcc3ee640 con 0x55cfcc090d80 2024-09-21T06:01:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.228+0000 7f3e06050640 1 --2- 172.21.15.117:0/2598127393 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55cfcc090d80 0x55cfcc495680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:38.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:37 smithi192 ceph-mon[34700]: pgmap v456: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 759 MiB/s wr, 1.91k op/s 2024-09-21T06:01:38.231 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.229+0000 7f3de9ffb640 10 client.16272 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:01:38.231 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.229+0000 7f3e06050640 1 --2- 172.21.15.117:0/2598127393 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55cfcc090d80 0x55cfcc495680 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=380167856861f9d6 server_cookie=17e43cf7b32ce1d9 in_seq=0 out_seq=0 2024-09-21T06:01:38.236 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.234+0000 7f3de9ffb640 1 -- 172.21.15.117:0/2598127393 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cfcc3ee640 con 0x55cfcc090d80 2024-09-21T06:01:38.236 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.234+0000 7f3de9ffb640 10 client.16272 handle_command_reply: tid=0 2024-09-21T06:01:38.237 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.234+0000 7f3e0f227740 1 client.16272 shutdown 2024-09-21T06:01:38.237 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.235+0000 7f3e0f227740 1 -- 172.21.15.117:0/2598127393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfcc0b14d0 msgr2=0x55cfcc483800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:38.237 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.235+0000 7f3e0f227740 1 --2- 172.21.15.117:0/2598127393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfcc0b14d0 0x55cfcc483800 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f3df000e7c0 tx=0x7f3df000ec80 comp rx=0 tx=0).stop 2024-09-21T06:01:38.238 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.236+0000 7f3e0f227740 1 -- 172.21.15.117:0/2598127393 shutdown_connections 2024-09-21T06:01:38.238 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.236+0000 7f3e0f227740 1 --2- 172.21.15.117:0/2598127393 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55cfcc090d80 0x55cfcc495680 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:38.238 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.236+0000 7f3e0f227740 1 --2- 172.21.15.117:0/2598127393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfcc0b14d0 0x55cfcc483800 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:38.238 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.236+0000 7f3e0f227740 1 --2- 172.21.15.117:0/2598127393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cfcc0d21b0 0x55cfcc48f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:38.238 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.236+0000 7f3e0f227740 1 --2- 172.21.15.117:0/2598127393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cfcc0a92f0 0x55cfcc4832c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:38.238 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.236+0000 7f3e0f227740 1 -- 172.21.15.117:0/2598127393 >> 172.21.15.117:0/2598127393 conn(0x55cfcc0c6300 msgr2=0x55cfcc47adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:38.238 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.236+0000 7f3de9ffb640 0 client.16272 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:01:38.239 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.236+0000 7f3e0f227740 1 -- 172.21.15.117:0/2598127393 shutdown_connections 2024-09-21T06:01:38.239 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.236+0000 7f3e0f227740 1 -- 172.21.15.117:0/2598127393 wait complete. 2024-09-21T06:01:38.239 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.236+0000 7f3e0f227740 20 client.16272 trim_cache size 0 max 16384 2024-09-21T06:01:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.274+0000 7f3e0f227740 1 Processor -- start 2024-09-21T06:01:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.274+0000 7f3e0f227740 1 -- start start 2024-09-21T06:01:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.274+0000 7f3e0f227740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfcc0a92f0 0x55cfcc47ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.275+0000 7f3e0f227740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cfcc0b14d0 0x55cfcc470870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.275+0000 7f3e0f227740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cfcc0d21b0 0x55cfcc48f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.275+0000 7f3e0f227740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3df81482b0 con 0x55cfcc0a92f0 2024-09-21T06:01:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.275+0000 7f3e0f227740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3df8149df0 con 0x55cfcc0d21b0 2024-09-21T06:01:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.275+0000 7f3e0f227740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3df8148980 con 0x55cfcc0b14d0 2024-09-21T06:01:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.276+0000 7f3e06050640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfcc0a92f0 0x55cfcc47ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.276+0000 7f3e06050640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfcc0a92f0 0x55cfcc47ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56552/0 (socket says 172.21.15.117:56552) 2024-09-21T06:01:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.276+0000 7f3e06050640 1 -- 172.21.15.117:0/3764649802 learned_addr learned my addr 172.21.15.117:0/3764649802 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:01:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.278+0000 7f3e06050640 1 -- 172.21.15.117:0/3764649802 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cfcc0b14d0 msgr2=0x55cfcc470870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.278+0000 7f3e06050640 1 --2- 172.21.15.117:0/3764649802 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cfcc0b14d0 0x55cfcc470870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.278+0000 7f3e06050640 1 -- 172.21.15.117:0/3764649802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cfcc0d21b0 msgr2=0x55cfcc48f5a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:01:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.278+0000 7f3e06050640 1 --2- 172.21.15.117:0/3764649802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cfcc0d21b0 0x55cfcc48f5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.278+0000 7f3e06050640 1 -- 172.21.15.117:0/3764649802 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfcc483be0 con 0x55cfcc0a92f0 2024-09-21T06:01:38.285 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.283+0000 7f3e06050640 1 --2- 172.21.15.117:0/3764649802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfcc0a92f0 0x55cfcc47ed60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc01ef10 tx=0x7f3dfc01eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2abb87f650f05c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:38.293 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.290+0000 7f3debfff640 1 -- 172.21.15.117:0/3764649802 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc023040 con 0x55cfcc0a92f0 2024-09-21T06:01:38.294 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.290+0000 7f3debfff640 1 -- 172.21.15.117:0/3764649802 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dfc01e8b0 con 0x55cfcc0a92f0 2024-09-21T06:01:38.294 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.292+0000 7f3e0f227740 1 -- 172.21.15.117:0/3764649802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfcc0a92f0 msgr2=0x55cfcc47ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:38.294 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.292+0000 7f3e0f227740 1 --2- 172.21.15.117:0/3764649802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfcc0a92f0 0x55cfcc47ed60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc01ef10 tx=0x7f3dfc01eea0 comp rx=0 tx=0).stop 2024-09-21T06:01:38.295 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.293+0000 7f3e0f227740 1 -- 172.21.15.117:0/3764649802 shutdown_connections 2024-09-21T06:01:38.295 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.293+0000 7f3e0f227740 1 --2- 172.21.15.117:0/3764649802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cfcc0d21b0 0x55cfcc48f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:38.295 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.293+0000 7f3e0f227740 1 --2- 172.21.15.117:0/3764649802 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cfcc0b14d0 0x55cfcc470870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.293+0000 7f3e0f227740 1 --2- 172.21.15.117:0/3764649802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfcc0a92f0 0x55cfcc47ed60 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.293+0000 7f3e0f227740 1 -- 172.21.15.117:0/3764649802 >> 172.21.15.117:0/3764649802 conn(0x55cfcc0c6300 msgr2=0x55cfcc481b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.294+0000 7f3e0f227740 1 -- 172.21.15.117:0/3764649802 shutdown_connections 2024-09-21T06:01:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.294+0000 7f3e0f227740 1 -- 172.21.15.117:0/3764649802 wait complete. 2024-09-21T06:01:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.294+0000 7f3e0f227740 1 -- start start 2024-09-21T06:01:38.296 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.295+0000 7f3e0f227740 1 Processor -- start 2024-09-21T06:01:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.295+0000 7f3e0f227740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfcc0a92f0 0x55cfcc49ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.295+0000 7f3e0f227740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cfcc0b14d0 0x55cfcc49b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.295+0000 7f3e0f227740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cfcc0d21b0 0x55cfcc48b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:38.297 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.295+0000 7f3e0f227740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3df8149720 con 0x55cfcc0a92f0 2024-09-21T06:01:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.295+0000 7f3e0f227740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3df8149050 con 0x55cfcc0d21b0 2024-09-21T06:01:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.295+0000 7f3e0f227740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3df8146290 con 0x55cfcc0b14d0 2024-09-21T06:01:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.295+0000 7f3e0f227740 10 client.0 _ll_register_callbacks cb 0x7f3df0009750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:01:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.296+0000 7f3e06050640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfcc0a92f0 0x55cfcc49ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.296+0000 7f3e06050640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfcc0a92f0 0x55cfcc49ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56558/0 (socket says 172.21.15.117:56558) 2024-09-21T06:01:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.296+0000 7f3e06050640 1 -- 172.21.15.117:0/3369953226 learned_addr learned my addr 172.21.15.117:0/3369953226 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:01:38.298 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.296+0000 7f3e06851640 1 --2- 172.21.15.117:0/3369953226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cfcc0d21b0 0x55cfcc48b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.297+0000 7f3e0584f640 1 --2- 172.21.15.117:0/3369953226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cfcc0b14d0 0x55cfcc49b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.297+0000 7f3e06050640 1 -- 172.21.15.117:0/3369953226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cfcc0b14d0 msgr2=0x55cfcc49b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.297+0000 7f3e06050640 1 --2- 172.21.15.117:0/3369953226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cfcc0b14d0 0x55cfcc49b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.297+0000 7f3e06050640 1 -- 172.21.15.117:0/3369953226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cfcc0d21b0 msgr2=0x55cfcc48b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.297+0000 7f3e06050640 1 --2- 172.21.15.117:0/3369953226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cfcc0d21b0 0x55cfcc48b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.297+0000 7f3e06050640 1 -- 172.21.15.117:0/3369953226 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfcc49bad0 con 0x55cfcc0a92f0 2024-09-21T06:01:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.298+0000 7f3e06050640 1 --2- 172.21.15.117:0/3369953226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfcc0a92f0 0x55cfcc49ae80 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc021040 tx=0x7f3dfc00afd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf7fac97b04ffae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.298+0000 7f3de9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:01:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.299+0000 7f3de9ffb640 1 -- 172.21.15.117:0/3369953226 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc023040 con 0x55cfcc0a92f0 2024-09-21T06:01:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.299+0000 7f3e0f227740 1 -- 172.21.15.117:0/3369953226 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3dfc01e8b0 con 0x55cfcc0a92f0 2024-09-21T06:01:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.299+0000 7f3de9ffb640 1 -- 172.21.15.117:0/3369953226 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f3dfc01ea60 con 0x55cfcc0a92f0 2024-09-21T06:01:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.300+0000 7f3de9ffb640 1 -- 172.21.15.117:0/3369953226 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3dfc013c30 con 0x55cfcc0a92f0 2024-09-21T06:01:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.300+0000 7f3e0f227740 10 client.16278 fetch_fsmap learned FSMap version 27 2024-09-21T06:01:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.300+0000 7f3e0f227740 1 -- 172.21.15.117:0/3369953226 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cfcc47fc40 con 0x55cfcc0a92f0 2024-09-21T06:01:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.303+0000 7f3de9ffb640 1 -- 172.21.15.117:0/3369953226 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dfc01e8b0 con 0x55cfcc0a92f0 2024-09-21T06:01:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.305+0000 7f3de9ffb640 1 -- 172.21.15.117:0/3369953226 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3dfc03a070 con 0x55cfcc0a92f0 2024-09-21T06:01:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.305+0000 7f3e0f227740 10 client.16278 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:01:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.305+0000 7f3e0f227740 10 client.16278 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:01:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.305+0000 7f3e0f227740 20 client.16278 populate_metadata read hostname 'smithi117' 2024-09-21T06:01:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.306+0000 7f3e0f227740 1 --2- 172.21.15.117:0/3369953226 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55cfcc0661c0 0x55cfcc5240b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.306+0000 7f3e0f227740 4 client.16278 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:01:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.306+0000 7f3e0f227740 1 -- 172.21.15.117:0/3369953226 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cfcc3ea0f0 con 0x55cfcc0661c0 2024-09-21T06:01:38.321 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.319+0000 7f3e0584f640 1 --2- 172.21.15.117:0/3369953226 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55cfcc0661c0 0x55cfcc5240b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:38.322 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.321+0000 7f3e0584f640 1 --2- 172.21.15.117:0/3369953226 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55cfcc0661c0 0x55cfcc5240b0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=607b6df50cf3d308 server_cookie=43c7a1b2bf746138 in_seq=0 out_seq=0 2024-09-21T06:01:38.330 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:38.327+0000 7f3de9ffb640 10 client.16278 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:01:40.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:39 smithi138 ceph-mon[34391]: pgmap v457: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 857 MiB/s wr, 1.97k op/s 2024-09-21T06:01:40.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:39 smithi192 ceph-mon[34700]: pgmap v457: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 857 MiB/s wr, 1.97k op/s 2024-09-21T06:01:40.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:39 smithi117 ceph-mon[30947]: pgmap v457: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 857 MiB/s wr, 1.97k op/s 2024-09-21T06:01:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:40.537+0000 7f3de9ffb640 1 -- 172.21.15.117:0/3369953226 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cfcc3ea0f0 con 0x55cfcc0661c0 2024-09-21T06:01:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:40.537+0000 7f3de9ffb640 10 client.16278 handle_command_reply: tid=0 2024-09-21T06:01:40.550 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:40.547+0000 7f3e0f227740 1 client.16278 shutdown 2024-09-21T06:01:40.553 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:40.551+0000 7f3e0f227740 1 -- 172.21.15.117:0/3369953226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfcc0a92f0 msgr2=0x55cfcc49ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:40.553 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:40.551+0000 7f3e0f227740 1 --2- 172.21.15.117:0/3369953226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfcc0a92f0 0x55cfcc49ae80 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc021040 tx=0x7f3dfc00afd0 comp rx=0 tx=0).stop 2024-09-21T06:01:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:40.552+0000 7f3e0f227740 1 -- 172.21.15.117:0/3369953226 shutdown_connections 2024-09-21T06:01:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:40.552+0000 7f3e0f227740 1 --2- 172.21.15.117:0/3369953226 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55cfcc0661c0 0x55cfcc5240b0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:40.552+0000 7f3e0f227740 1 --2- 172.21.15.117:0/3369953226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cfcc0d21b0 0x55cfcc48b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:40.552+0000 7f3e0f227740 1 --2- 172.21.15.117:0/3369953226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cfcc0b14d0 0x55cfcc49b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:40.552+0000 7f3e0f227740 1 --2- 172.21.15.117:0/3369953226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cfcc0a92f0 0x55cfcc49ae80 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:40.552+0000 7f3e0f227740 1 -- 172.21.15.117:0/3369953226 >> 172.21.15.117:0/3369953226 conn(0x55cfcc0c6300 msgr2=0x55cfcc474150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:40.555 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:40.552+0000 7f3de9ffb640 0 client.16278 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:01:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:40.552+0000 7f3e0f227740 1 -- 172.21.15.117:0/3369953226 shutdown_connections 2024-09-21T06:01:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:40.552+0000 7f3e0f227740 1 -- 172.21.15.117:0/3369953226 wait complete. 2024-09-21T06:01:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:40.556+0000 7f3e0f227740 20 client.16278 trim_cache size 0 max 16384 2024-09-21T06:01:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T06:01:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T06:01:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T06:01:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T06:01:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:40.559+0000 7f3e07052640 1 -- 172.21.15.117:0/3171721015 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3dec0b6730 msgr2=0x7f3dec0b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:40.559+0000 7f3e07052640 1 --2- 172.21.15.117:0/3171721015 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3dec0b6730 0x7f3dec0b8c40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3df80a1e10 tx=0x7f3dfc008040 comp rx=0 tx=0).stop 2024-09-21T06:01:40.561 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:40.559+0000 7f3e07052640 1 -- 172.21.15.117:0/3171721015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3df809b2f0 msgr2=0x7f3df813eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:40.559+0000 7f3e07052640 1 --2- 172.21.15.117:0/3171721015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3df809b2f0 0x7f3df813eb40 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3e00000b60 tx=0x7f3e000a0ed0 comp rx=0 tx=0).stop 2024-09-21T06:01:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:40.559+0000 7f3e07052640 1 -- 172.21.15.117:0/3171721015 shutdown_connections 2024-09-21T06:01:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:40.559+0000 7f3e07052640 1 --2- 172.21.15.117:0/3171721015 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3dec0b6730 0x7f3dec0b8c40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:40.559+0000 7f3e07052640 1 --2- 172.21.15.117:0/3171721015 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3df809b2f0 0x7f3df813eb40 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:40.559+0000 7f3e07052640 1 --2- 172.21.15.117:0/3171721015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3df809a9a0 0x7f3df809eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:40.559+0000 7f3e07052640 1 --2- 172.21.15.117:0/3171721015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3df8095070 0x7f3df809e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:40.562 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:40.559+0000 7f3e07052640 1 -- 172.21.15.117:0/3171721015 >> 172.21.15.117:0/3171721015 conn(0x7f3df800cd80 msgr2=0x7f3df8099eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:40.564+0000 7f3e07052640 1 -- 172.21.15.117:0/3171721015 shutdown_connections 2024-09-21T06:01:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:40.564+0000 7f3e07052640 1 -- 172.21.15.117:0/3171721015 wait complete. 2024-09-21T06:01:42.117 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:41 smithi138 ceph-mon[34391]: pgmap v458: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 870 MiB/s wr, 1.97k op/s 2024-09-21T06:01:42.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:41 smithi192 ceph-mon[34700]: pgmap v458: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 870 MiB/s wr, 1.97k op/s 2024-09-21T06:01:42.279 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:41 smithi117 ceph-mon[30947]: pgmap v458: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 870 MiB/s wr, 1.97k op/s 2024-09-21T06:01:44.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:43 smithi138 ceph-mon[34391]: pgmap v459: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 902 MiB/s wr, 2.02k op/s 2024-09-21T06:01:44.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:43 smithi192 ceph-mon[34700]: pgmap v459: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 902 MiB/s wr, 2.02k op/s 2024-09-21T06:01:44.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:43 smithi117 ceph-mon[30947]: pgmap v459: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 902 MiB/s wr, 2.02k op/s 2024-09-21T06:01:46.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:45 smithi138 ceph-mon[34391]: pgmap v460: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 907 MiB/s wr, 1.94k op/s 2024-09-21T06:01:46.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:45 smithi192 ceph-mon[34700]: pgmap v460: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 907 MiB/s wr, 1.94k op/s 2024-09-21T06:01:46.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:45 smithi117 ceph-mon[30947]: pgmap v460: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 907 MiB/s wr, 1.94k op/s 2024-09-21T06:01:48.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:47 smithi138 ceph-mon[34391]: pgmap v461: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 872 MiB/s wr, 1.75k op/s 2024-09-21T06:01:48.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:47 smithi192 ceph-mon[34700]: pgmap v461: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 872 MiB/s wr, 1.75k op/s 2024-09-21T06:01:48.279 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:47 smithi117 ceph-mon[30947]: pgmap v461: 97 pgs: 97 active+clean; 40 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 872 MiB/s wr, 1.75k op/s 2024-09-21T06:01:50.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:49 smithi138 ceph-mon[34391]: pgmap v462: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 939 MiB/s wr, 1.86k op/s 2024-09-21T06:01:50.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:49 smithi192 ceph-mon[34700]: pgmap v462: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 939 MiB/s wr, 1.86k op/s 2024-09-21T06:01:50.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:49 smithi117 ceph-mon[30947]: pgmap v462: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 939 MiB/s wr, 1.86k op/s 2024-09-21T06:01:52.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:51 smithi138 ceph-mon[34391]: pgmap v463: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 885 MiB/s wr, 1.65k op/s 2024-09-21T06:01:52.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:51 smithi192 ceph-mon[34700]: pgmap v463: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 885 MiB/s wr, 1.65k op/s 2024-09-21T06:01:52.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:51 smithi117 ceph-mon[30947]: pgmap v463: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 885 MiB/s wr, 1.65k op/s 2024-09-21T06:01:53.898 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.893+0000 7f536f59e640 1 -- 172.21.15.117:0/3234537374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5370111ca0 msgr2=0x7f537010a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:53.898 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.893+0000 7f536f59e640 1 --2- 172.21.15.117:0/3234537374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5370111ca0 0x7f537010a3c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f5360009860 tx=0x7f536002a340 comp rx=0 tx=0).stop 2024-09-21T06:01:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.905+0000 7f536f59e640 1 -- 172.21.15.117:0/3234537374 shutdown_connections 2024-09-21T06:01:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.905+0000 7f536f59e640 1 --2- 172.21.15.117:0/3234537374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f537011cba0 0x7f537011efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.905+0000 7f536f59e640 1 --2- 172.21.15.117:0/3234537374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f537010a900 0x7f537010ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.905+0000 7f536f59e640 1 --2- 172.21.15.117:0/3234537374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5370111ca0 0x7f537010a3c0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.905+0000 7f536f59e640 1 -- 172.21.15.117:0/3234537374 >> 172.21.15.117:0/3234537374 conn(0x7f5370070c00 msgr2=0x7f5370071010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.910+0000 7f536f59e640 1 -- 172.21.15.117:0/3234537374 shutdown_connections 2024-09-21T06:01:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.914+0000 7f536f59e640 1 -- 172.21.15.117:0/3234537374 wait complete. 2024-09-21T06:01:53.920 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.916+0000 7f536f59e640 1 Processor -- start 2024-09-21T06:01:53.920 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.917+0000 7f536f59e640 1 -- start start 2024-09-21T06:01:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.918+0000 7f536f59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f537010a900 0x7f53701a9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.918+0000 7f536f59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5370111ca0 0x7f53701a9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.918+0000 7f536f59e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f537011cba0 0x7f53701ae990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.918+0000 7f536f59e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5370122680 con 0x7f5370111ca0 2024-09-21T06:01:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.918+0000 7f536f59e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5370122500 con 0x7f537011cba0 2024-09-21T06:01:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.918+0000 7f536f59e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5370122800 con 0x7f537010a900 2024-09-21T06:01:53.924 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.921+0000 7f536ed9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f537011cba0 0x7f53701ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:53.924 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.921+0000 7f536ed9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f537011cba0 0x7f53701ae990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:34796/0 (socket says 172.21.15.117:34796) 2024-09-21T06:01:53.924 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.921+0000 7f536ed9d640 1 -- 172.21.15.117:0/3362707766 learned_addr learned my addr 172.21.15.117:0/3362707766 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:01:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.923+0000 7f536dd9b640 1 --2- 172.21.15.117:0/3362707766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5370111ca0 0x7f53701a9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.923+0000 7f536e59c640 1 --2- 172.21.15.117:0/3362707766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f537010a900 0x7f53701a9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.923+0000 7f536ed9d640 1 -- 172.21.15.117:0/3362707766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f537010a900 msgr2=0x7f53701a9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.923+0000 7f536ed9d640 1 --2- 172.21.15.117:0/3362707766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f537010a900 0x7f53701a9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.923+0000 7f536ed9d640 1 -- 172.21.15.117:0/3362707766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5370111ca0 msgr2=0x7f53701a9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.923+0000 7f536ed9d640 1 --2- 172.21.15.117:0/3362707766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5370111ca0 0x7f53701a9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.923+0000 7f536ed9d640 1 -- 172.21.15.117:0/3362707766 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53701af090 con 0x7f537011cba0 2024-09-21T06:01:53.932 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.928+0000 7f536ed9d640 1 --2- 172.21.15.117:0/3362707766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f537011cba0 0x7f53701ae990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f536400c970 tx=0x7f536400ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:53.933 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.930+0000 7f535f7fe640 1 -- 172.21.15.117:0/3362707766 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5364007bb0 con 0x7f537011cba0 2024-09-21T06:01:53.933 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.930+0000 7f536f59e640 1 -- 172.21.15.117:0/3362707766 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53701af380 con 0x7f537011cba0 2024-09-21T06:01:53.933 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.930+0000 7f535f7fe640 1 -- 172.21.15.117:0/3362707766 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5364007d50 con 0x7f537011cba0 2024-09-21T06:01:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.930+0000 7f535f7fe640 1 -- 172.21.15.117:0/3362707766 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536401d060 con 0x7f537011cba0 2024-09-21T06:01:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.930+0000 7f536f59e640 1 -- 172.21.15.117:0/3362707766 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53701b5b10 con 0x7f537011cba0 2024-09-21T06:01:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.937+0000 7f537ba68740 1 Processor -- start 2024-09-21T06:01:53.940 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.937+0000 7f537ba68740 1 -- start start 2024-09-21T06:01:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.938+0000 7f537ba68740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de31221b0 0x561de34c7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.938+0000 7f537ba68740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de30f92f0 0x561de34c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.938+0000 7f537ba68740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de31014d0 0x561de34c8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.938+0000 7f537ba68740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561de343c6f0 con 0x561de31014d0 2024-09-21T06:01:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.938+0000 7f537ba68740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561de342bb20 con 0x561de31221b0 2024-09-21T06:01:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.938+0000 7f537ba68740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x561de342f930 con 0x561de30f92f0 2024-09-21T06:01:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.938+0000 7f536dd9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de31221b0 0x561de34c7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.938+0000 7f536dd9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de31221b0 0x561de34c7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:34810/0 (socket says 172.21.15.117:34810) 2024-09-21T06:01:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.938+0000 7f536dd9b640 1 -- 172.21.15.117:0/4207664215 learned_addr learned my addr 172.21.15.117:0/4207664215 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:01:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.939+0000 7f536dd9b640 1 -- 172.21.15.117:0/4207664215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de30f92f0 msgr2=0x561de34c80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:53.942 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.939+0000 7f536dd9b640 1 --2- 172.21.15.117:0/4207664215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de30f92f0 0x561de34c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.939+0000 7f536dd9b640 1 -- 172.21.15.117:0/4207664215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de31014d0 msgr2=0x561de34c8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.939+0000 7f536dd9b640 1 --2- 172.21.15.117:0/4207664215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de31014d0 0x561de34c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.939+0000 7f536dd9b640 1 -- 172.21.15.117:0/4207664215 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561de33f1530 con 0x561de31221b0 2024-09-21T06:01:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.940+0000 7f536dd9b640 1 --2- 172.21.15.117:0/4207664215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de31221b0 0x561de34c7ba0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5368010550 tx=0x7f5368010a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c869af178329015b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.947+0000 7f535f7fe640 1 -- 172.21.15.117:0/3362707766 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f5364004e40 con 0x7f537011cba0 2024-09-21T06:01:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.948+0000 7f535cff9640 1 -- 172.21.15.117:0/4207664215 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53680100d0 con 0x561de31221b0 2024-09-21T06:01:53.952 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.949+0000 7f535cff9640 1 -- 172.21.15.117:0/4207664215 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5368003e20 con 0x561de31221b0 2024-09-21T06:01:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.950+0000 7f535f7fe640 1 --2- 172.21.15.117:0/3362707766 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f53580b6730 0x7f53580b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.950+0000 7f537ba68740 1 -- 172.21.15.117:0/4207664215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de31221b0 msgr2=0x561de34c7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.950+0000 7f537ba68740 1 --2- 172.21.15.117:0/4207664215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de31221b0 0x561de34c7ba0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5368010550 tx=0x7f5368010a10 comp rx=0 tx=0).stop 2024-09-21T06:01:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.950+0000 7f537ba68740 1 -- 172.21.15.117:0/4207664215 shutdown_connections 2024-09-21T06:01:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.950+0000 7f537ba68740 1 --2- 172.21.15.117:0/4207664215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de31014d0 0x561de34c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.950+0000 7f537ba68740 1 --2- 172.21.15.117:0/4207664215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de30f92f0 0x561de34c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.950+0000 7f537ba68740 1 --2- 172.21.15.117:0/4207664215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de31221b0 0x561de34c7ba0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.950+0000 7f537ba68740 1 -- 172.21.15.117:0/4207664215 >> 172.21.15.117:0/4207664215 conn(0x561de3116300 msgr2=0x561de34c5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.950+0000 7f535f7fe640 1 -- 172.21.15.117:0/3362707766 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f53640e77d0 con 0x7f537011cba0 2024-09-21T06:01:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.950+0000 7f537ba68740 1 -- 172.21.15.117:0/4207664215 shutdown_connections 2024-09-21T06:01:53.955 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.951+0000 7f536e59c640 1 --2- 172.21.15.117:0/3362707766 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f53580b6730 0x7f53580b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:53.955 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.952+0000 7f537ba68740 1 -- 172.21.15.117:0/4207664215 wait complete. 2024-09-21T06:01:53.955 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.952+0000 7f537ba68740 1 -- start start 2024-09-21T06:01:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.953+0000 7f537ba68740 1 Processor -- start 2024-09-21T06:01:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.953+0000 7f537ba68740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de30f92f0 0x561de34d5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.953+0000 7f537ba68740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de31014d0 0x561de34d2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.953+0000 7f537ba68740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de30e0d80 0x561de34d3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.953+0000 7f537ba68740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561de3363ab0 con 0x561de31014d0 2024-09-21T06:01:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.953+0000 7f537ba68740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561de33d1730 con 0x561de30e0d80 2024-09-21T06:01:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.953+0000 7f537ba68740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x561de34d6370 con 0x561de30f92f0 2024-09-21T06:01:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.954+0000 7f537ba68740 10 client.0 _ll_register_callbacks cb 0x561de34256b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:01:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.954+0000 7f536e59c640 1 --2- 172.21.15.117:0/3362707766 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f53580b6730 0x7f53580b8c40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f536002a8b0 tx=0x7f536002a840 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.954+0000 7f536dd9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de31014d0 0x561de34d2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.954+0000 7f536dd9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de31014d0 0x561de34d2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40058/0 (socket says 172.21.15.117:40058) 2024-09-21T06:01:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.954+0000 7f536dd9b640 1 -- 172.21.15.117:0/3809424270 learned_addr learned my addr 172.21.15.117:0/3809424270 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:01:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.955+0000 7f536dd9b640 1 -- 172.21.15.117:0/3809424270 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de30f92f0 msgr2=0x561de34d5950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:01:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.955+0000 7f536dd9b640 1 --2- 172.21.15.117:0/3809424270 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de30f92f0 0x561de34d5950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.955+0000 7f536dd9b640 1 -- 172.21.15.117:0/3809424270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de30e0d80 msgr2=0x561de34d3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.955+0000 7f536dd9b640 1 --2- 172.21.15.117:0/3809424270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de30e0d80 0x561de34d3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.955+0000 7f536dd9b640 1 -- 172.21.15.117:0/3809424270 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561de318d790 con 0x561de31014d0 2024-09-21T06:01:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.956+0000 7f536dd9b640 1 --2- 172.21.15.117:0/3809424270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de31014d0 0x561de34d2e10 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f5368005b20 tx=0x7f53680046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=763bed1057a3b9cb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.956+0000 7f5355ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:01:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.956+0000 7f5355ffb640 1 -- 172.21.15.117:0/3809424270 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53680047f0 con 0x561de31014d0 2024-09-21T06:01:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.957+0000 7f537ba68740 1 -- 172.21.15.117:0/3809424270 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561de343e640 con 0x561de31014d0 2024-09-21T06:01:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.957+0000 7f5355ffb640 1 -- 172.21.15.117:0/3809424270 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f5368004cc0 con 0x561de31014d0 2024-09-21T06:01:53.961 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.958+0000 7f5355ffb640 1 -- 172.21.15.117:0/3809424270 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5368003970 con 0x561de31014d0 2024-09-21T06:01:53.961 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.958+0000 7f537ba68740 10 client.16287 fetch_fsmap learned FSMap version 27 2024-09-21T06:01:53.961 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.958+0000 7f537ba68740 1 -- 172.21.15.117:0/3809424270 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561de3206620 con 0x561de31014d0 2024-09-21T06:01:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.959+0000 7f5355ffb640 1 -- 172.21.15.117:0/3809424270 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561de343e640 con 0x561de31014d0 2024-09-21T06:01:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.959+0000 7f5355ffb640 1 -- 172.21.15.117:0/3809424270 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5368022020 con 0x561de31014d0 2024-09-21T06:01:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.959+0000 7f537ba68740 10 client.16287 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:01:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.959+0000 7f537ba68740 10 client.16287 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:01:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.959+0000 7f537ba68740 20 client.16287 populate_metadata read hostname 'smithi117' 2024-09-21T06:01:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.959+0000 7f537ba68740 1 --2- 172.21.15.117:0/3809424270 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x561de30b61c0 0x561de34e5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.959+0000 7f537ba68740 4 client.16287 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:01:53.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.959+0000 7f537ba68740 1 -- 172.21.15.117:0/3809424270 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561de343a0f0 con 0x561de30b61c0 2024-09-21T06:01:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.961+0000 7f536e59c640 1 --2- 172.21.15.117:0/3809424270 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x561de30b61c0 0x561de34e5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:53.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.972+0000 7f536e59c640 1 --2- 172.21.15.117:0/3809424270 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x561de30b61c0 0x561de34e5470 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b75025af008f5c6 server_cookie=101bd437e5a4525d in_seq=0 out_seq=0 2024-09-21T06:01:53.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.972+0000 7f5355ffb640 10 client.16287 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:01:53.981 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.978+0000 7f5355ffb640 1 -- 172.21.15.117:0/3809424270 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561de343a0f0 con 0x561de30b61c0 2024-09-21T06:01:53.982 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.978+0000 7f5355ffb640 10 client.16287 handle_command_reply: tid=0 2024-09-21T06:01:53.983 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.978+0000 7f537ba68740 1 client.16287 shutdown 2024-09-21T06:01:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.982+0000 7f537ba68740 1 -- 172.21.15.117:0/3809424270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de31014d0 msgr2=0x561de34d2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.982+0000 7f537ba68740 1 --2- 172.21.15.117:0/3809424270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de31014d0 0x561de34d2e10 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f5368005b20 tx=0x7f53680046d0 comp rx=0 tx=0).stop 2024-09-21T06:01:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.982+0000 7f537ba68740 1 -- 172.21.15.117:0/3809424270 shutdown_connections 2024-09-21T06:01:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.982+0000 7f537ba68740 1 --2- 172.21.15.117:0/3809424270 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x561de30b61c0 0x561de34e5470 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.982+0000 7f537ba68740 1 --2- 172.21.15.117:0/3809424270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de31014d0 0x561de34d2e10 secure :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f5368005b20 tx=0x7f53680046d0 comp rx=0 tx=0).stop 2024-09-21T06:01:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.982+0000 7f537ba68740 1 --2- 172.21.15.117:0/3809424270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de30e0d80 0x561de34d3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.982+0000 7f537ba68740 1 --2- 172.21.15.117:0/3809424270 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de30f92f0 0x561de34d5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.982+0000 7f537ba68740 1 -- 172.21.15.117:0/3809424270 >> 172.21.15.117:0/3809424270 conn(0x561de3116300 msgr2=0x561de34c8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:53.988 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.983+0000 7f5355ffb640 0 client.16287 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:01:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.987+0000 7f537ba68740 1 -- 172.21.15.117:0/3809424270 shutdown_connections 2024-09-21T06:01:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.990+0000 7f537ba68740 1 -- 172.21.15.117:0/3809424270 wait complete. 2024-09-21T06:01:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:53.992+0000 7f537ba68740 20 client.16287 trim_cache size 0 max 16384 2024-09-21T06:01:54.041 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:53 smithi117 ceph-mon[30947]: pgmap v464: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 877 MiB/s wr, 1.52k op/s 2024-09-21T06:01:54.079 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.074+0000 7f537ba68740 1 Processor -- start 2024-09-21T06:01:54.086 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.080+0000 7f537ba68740 1 -- start start 2024-09-21T06:01:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.080+0000 7f537ba68740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de30e0d80 0x561de34d3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.080+0000 7f537ba68740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de30f92f0 0x561de34c0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.080+0000 7f537ba68740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de34c5160 0x561de34c5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.080+0000 7f537ba68740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f53701be290 con 0x561de30e0d80 2024-09-21T06:01:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.080+0000 7f537ba68740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f53701bfdd0 con 0x561de34c5160 2024-09-21T06:01:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.080+0000 7f537ba68740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f53701be960 con 0x561de30f92f0 2024-09-21T06:01:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.084+0000 7f536ed9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de34c5160 0x561de34c5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.086+0000 7f536ed9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de34c5160 0x561de34c5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:34836/0 (socket says 172.21.15.117:34836) 2024-09-21T06:01:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.086+0000 7f536ed9d640 1 -- 172.21.15.117:0/762090479 learned_addr learned my addr 172.21.15.117:0/762090479 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:01:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.087+0000 7f536e59c640 1 --2- 172.21.15.117:0/762090479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de30e0d80 0x561de34d3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:54.093 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.089+0000 7f536dd9b640 1 --2- 172.21.15.117:0/762090479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de30f92f0 0x561de34c0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:54.095 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.092+0000 7f536dd9b640 1 -- 172.21.15.117:0/762090479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de34c5160 msgr2=0x561de34c5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:54.096 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.092+0000 7f536dd9b640 1 --2- 172.21.15.117:0/762090479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de34c5160 0x561de34c5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:54.096 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.092+0000 7f536dd9b640 1 -- 172.21.15.117:0/762090479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de30e0d80 msgr2=0x561de34d3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:54.096 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.092+0000 7f536dd9b640 1 --2- 172.21.15.117:0/762090479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de30e0d80 0x561de34d3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:54.096 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.092+0000 7f536dd9b640 1 -- 172.21.15.117:0/762090479 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561de34c9ea0 con 0x561de30f92f0 2024-09-21T06:01:54.096 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.093+0000 7f536dd9b640 1 --2- 172.21.15.117:0/762090479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de30f92f0 0x561de34c0870 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5368005000 tx=0x7f5368010a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2308d59925dbb076 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:54.097 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.094+0000 7f5357fff640 1 -- 172.21.15.117:0/762090479 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5368004cc0 con 0x561de30f92f0 2024-09-21T06:01:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.095+0000 7f5357fff640 1 -- 172.21.15.117:0/762090479 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5368009900 con 0x561de30f92f0 2024-09-21T06:01:54.098 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.095+0000 7f5357fff640 1 -- 172.21.15.117:0/762090479 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5368005240 con 0x561de30f92f0 2024-09-21T06:01:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.097+0000 7f537ba68740 1 -- 172.21.15.117:0/762090479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de30f92f0 msgr2=0x561de34c0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:54.101 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.097+0000 7f537ba68740 1 --2- 172.21.15.117:0/762090479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de30f92f0 0x561de34c0870 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5368005000 tx=0x7f5368010a10 comp rx=0 tx=0).stop 2024-09-21T06:01:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.107+0000 7f537ba68740 1 -- 172.21.15.117:0/762090479 shutdown_connections 2024-09-21T06:01:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.107+0000 7f537ba68740 1 --2- 172.21.15.117:0/762090479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de34c5160 0x561de34c5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.107+0000 7f537ba68740 1 --2- 172.21.15.117:0/762090479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de30f92f0 0x561de34c0870 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.107+0000 7f537ba68740 1 --2- 172.21.15.117:0/762090479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de30e0d80 0x561de34d3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.107+0000 7f537ba68740 1 -- 172.21.15.117:0/762090479 >> 172.21.15.117:0/762090479 conn(0x561de3116300 msgr2=0x561de34e53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.117+0000 7f537ba68740 1 -- 172.21.15.117:0/762090479 shutdown_connections 2024-09-21T06:01:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.117+0000 7f537ba68740 1 -- 172.21.15.117:0/762090479 wait complete. 2024-09-21T06:01:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.119+0000 7f537ba68740 1 -- start start 2024-09-21T06:01:54.122 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.120+0000 7f537ba68740 1 Processor -- start 2024-09-21T06:01:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.120+0000 7f537ba68740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de30e0d80 0x561de34d0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.121+0000 7f537ba68740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de30f92f0 0x561de34db650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.121+0000 7f537ba68740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de34c5160 0x561de34dbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.121+0000 7f537ba68740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f53701bf700 con 0x561de30e0d80 2024-09-21T06:01:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.121+0000 7f537ba68740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f53701bf030 con 0x561de30f92f0 2024-09-21T06:01:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.121+0000 7f537ba68740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f53701bc270 con 0x561de34c5160 2024-09-21T06:01:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.121+0000 7f537ba68740 10 client.0 _ll_register_callbacks cb 0x7f5368010470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:01:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.121+0000 7f536ed9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de34c5160 0x561de34dbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.121+0000 7f536ed9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de34c5160 0x561de34dbb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:32960/0 (socket says 172.21.15.117:32960) 2024-09-21T06:01:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.121+0000 7f536ed9d640 1 -- 172.21.15.117:0/3642573448 learned_addr learned my addr 172.21.15.117:0/3642573448 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:01:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.124+0000 7f536dd9b640 1 --2- 172.21.15.117:0/3642573448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de30f92f0 0x561de34db650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.125+0000 7f536e59c640 1 --2- 172.21.15.117:0/3642573448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de30e0d80 0x561de34d0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.125+0000 7f536ed9d640 1 -- 172.21.15.117:0/3642573448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de30f92f0 msgr2=0x561de34db650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:54.130 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.125+0000 7f536ed9d640 1 --2- 172.21.15.117:0/3642573448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de30f92f0 0x561de34db650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.125+0000 7f536ed9d640 1 -- 172.21.15.117:0/3642573448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de30e0d80 msgr2=0x561de34d0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:01:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.125+0000 7f536ed9d640 1 --2- 172.21.15.117:0/3642573448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de30e0d80 0x561de34d0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:54.131 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.125+0000 7f536ed9d640 1 -- 172.21.15.117:0/3642573448 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561de34dc0d0 con 0x561de34c5160 2024-09-21T06:01:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.131+0000 7f536ed9d640 1 --2- 172.21.15.117:0/3642573448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de34c5160 0x561de34dbb90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5364007590 tx=0x7f5364053440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbed0ad4df1392a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:01:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.132+0000 7f5355ffb640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-21T06:01:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.132+0000 7f5355ffb640 1 -- 172.21.15.117:0/3642573448 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53640e53d0 con 0x561de34c5160 2024-09-21T06:01:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.132+0000 7f537ba68740 1 -- 172.21.15.117:0/3642573448 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5368009900 con 0x561de34c5160 2024-09-21T06:01:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.132+0000 7f5355ffb640 1 -- 172.21.15.117:0/3642573448 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f53640acaa0 con 0x561de34c5160 2024-09-21T06:01:54.137 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.134+0000 7f5355ffb640 1 -- 172.21.15.117:0/3642573448 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f53640b2120 con 0x561de34c5160 2024-09-21T06:01:54.137 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.134+0000 7f537ba68740 10 client.25612 fetch_fsmap learned FSMap version 27 2024-09-21T06:01:54.137 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.134+0000 7f537ba68740 1 -- 172.21.15.117:0/3642573448 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561de354bb40 con 0x561de34c5160 2024-09-21T06:01:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.136+0000 7f5355ffb640 1 -- 172.21.15.117:0/3642573448 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5368009900 con 0x561de34c5160 2024-09-21T06:01:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.136+0000 7f5355ffb640 1 -- 172.21.15.117:0/3642573448 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f53640ac310 con 0x561de34c5160 2024-09-21T06:01:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.136+0000 7f537ba68740 10 client.25612 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:01:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.136+0000 7f537ba68740 10 client.25612 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:01:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.136+0000 7f537ba68740 20 client.25612 populate_metadata read hostname 'smithi117' 2024-09-21T06:01:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.136+0000 7f537ba68740 1 --2- 172.21.15.117:0/3642573448 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x561de34cf4d0 0x561de34cf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:01:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.136+0000 7f537ba68740 4 client.25612 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:01:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.136+0000 7f537ba68740 1 -- 172.21.15.117:0/3642573448 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561de3437520 con 0x561de34cf4d0 2024-09-21T06:01:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.138+0000 7f536e59c640 1 --2- 172.21.15.117:0/3642573448 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x561de34cf4d0 0x561de34cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:01:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.141+0000 7f536e59c640 1 --2- 172.21.15.117:0/3642573448 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x561de34cf4d0 0x561de34cf910 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ac407936c8bfe6d server_cookie=cabc56927ac15e4b in_seq=0 out_seq=0 2024-09-21T06:01:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:54.141+0000 7f5355ffb640 10 client.25612 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:01:54.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:53 smithi192 ceph-mon[34700]: pgmap v464: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 877 MiB/s wr, 1.52k op/s 2024-09-21T06:01:54.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:53 smithi138 ceph-mon[34391]: pgmap v464: 97 pgs: 97 active+clean; 40 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 877 MiB/s wr, 1.52k op/s 2024-09-21T06:01:55.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:54 smithi192 ceph-mon[34700]: pgmap v465: 97 pgs: 97 active+clean; 40 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 919 MiB/s wr, 1.56k op/s 2024-09-21T06:01:55.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:54 smithi117 ceph-mon[30947]: pgmap v465: 97 pgs: 97 active+clean; 40 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 919 MiB/s wr, 1.56k op/s 2024-09-21T06:01:55.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:54 smithi138 ceph-mon[34391]: pgmap v465: 97 pgs: 97 active+clean; 40 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 919 MiB/s wr, 1.56k op/s 2024-09-21T06:01:58.031 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:57 smithi117 ceph-mon[30947]: pgmap v466: 97 pgs: 97 active+clean; 40 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 924 MiB/s wr, 1.73k op/s 2024-09-21T06:01:58.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:57 smithi138 ceph-mon[34391]: pgmap v466: 97 pgs: 97 active+clean; 40 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 924 MiB/s wr, 1.73k op/s 2024-09-21T06:01:58.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:57 smithi192 ceph-mon[34700]: pgmap v466: 97 pgs: 97 active+clean; 40 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 924 MiB/s wr, 1.73k op/s 2024-09-21T06:01:58.269 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:01:58.367 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:58.364+0000 7f5355ffb640 1 -- 172.21.15.117:0/3642573448 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561de3437520 con 0x561de34cf4d0 2024-09-21T06:01:58.367 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:58.364+0000 7f5355ffb640 10 client.25612 handle_command_reply: tid=0 2024-09-21T06:01:58.367 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:58.364+0000 7f537ba68740 1 client.25612 shutdown 2024-09-21T06:01:58.367 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:58.364+0000 7f537ba68740 1 -- 172.21.15.117:0/3642573448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de34c5160 msgr2=0x561de34dbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:58.367 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:58.364+0000 7f537ba68740 1 --2- 172.21.15.117:0/3642573448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de34c5160 0x561de34dbb90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5364007590 tx=0x7f5364053440 comp rx=0 tx=0).stop 2024-09-21T06:01:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:58.365+0000 7f537ba68740 1 -- 172.21.15.117:0/3642573448 shutdown_connections 2024-09-21T06:01:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:58.365+0000 7f537ba68740 1 --2- 172.21.15.117:0/3642573448 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x561de34cf4d0 0x561de34cf910 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:58.365+0000 7f537ba68740 1 --2- 172.21.15.117:0/3642573448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de34c5160 0x561de34dbb90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:58.365+0000 7f537ba68740 1 --2- 172.21.15.117:0/3642573448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de30f92f0 0x561de34db650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:58.365+0000 7f537ba68740 1 --2- 172.21.15.117:0/3642573448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de30e0d80 0x561de34d0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:58.365+0000 7f537ba68740 1 -- 172.21.15.117:0/3642573448 >> 172.21.15.117:0/3642573448 conn(0x561de3116300 msgr2=0x561de34c5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:58.365+0000 7f5355ffb640 0 client.25612 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:01:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:58.366+0000 7f537ba68740 1 -- 172.21.15.117:0/3642573448 shutdown_connections 2024-09-21T06:01:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:58.366+0000 7f537ba68740 1 -- 172.21.15.117:0/3642573448 wait complete. 2024-09-21T06:01:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:58.366+0000 7f537ba68740 20 client.25612 trim_cache size 0 max 16384 2024-09-21T06:01:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T06:01:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T06:01:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T06:01:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T06:01:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:58.375+0000 7f536f59e640 1 -- 172.21.15.117:0/3362707766 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f53580b6730 msgr2=0x7f53580b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:58.375+0000 7f536f59e640 1 --2- 172.21.15.117:0/3362707766 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f53580b6730 0x7f53580b8c40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f536002a8b0 tx=0x7f536002a840 comp rx=0 tx=0).stop 2024-09-21T06:01:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:58.375+0000 7f536f59e640 1 -- 172.21.15.117:0/3362707766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f537011cba0 msgr2=0x7f53701ae990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:01:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:58.375+0000 7f536f59e640 1 --2- 172.21.15.117:0/3362707766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f537011cba0 0x7f53701ae990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f536400c970 tx=0x7f536400ce30 comp rx=0 tx=0).stop 2024-09-21T06:01:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:58.375+0000 7f536f59e640 1 -- 172.21.15.117:0/3362707766 shutdown_connections 2024-09-21T06:01:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:58.376+0000 7f536f59e640 1 --2- 172.21.15.117:0/3362707766 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f53580b6730 0x7f53580b8c40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:58.376+0000 7f536f59e640 1 --2- 172.21.15.117:0/3362707766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5370111ca0 0x7f53701a9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:58.376+0000 7f536f59e640 1 --2- 172.21.15.117:0/3362707766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f537011cba0 0x7f53701ae990 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:58.376+0000 7f536f59e640 1 --2- 172.21.15.117:0/3362707766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f537010a900 0x7f53701a9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:01:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:58.376+0000 7f536f59e640 1 -- 172.21.15.117:0/3362707766 >> 172.21.15.117:0/3362707766 conn(0x7f5370070c00 msgr2=0x7f5370110920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:01:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:58.380+0000 7f536f59e640 1 -- 172.21.15.117:0/3362707766 shutdown_connections 2024-09-21T06:01:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:01:58.380+0000 7f536f59e640 1 -- 172.21.15.117:0/3362707766 wait complete. 2024-09-21T06:01:59.269 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:01:59.980 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:01:59 smithi117 ceph-mon[30947]: pgmap v467: 97 pgs: 97 active+clean; 40 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 948 MiB/s wr, 1.88k op/s 2024-09-21T06:02:00.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:01:59 smithi138 ceph-mon[34391]: pgmap v467: 97 pgs: 97 active+clean; 40 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 948 MiB/s wr, 1.88k op/s 2024-09-21T06:02:00.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:01:59 smithi192 ceph-mon[34700]: pgmap v467: 97 pgs: 97 active+clean; 40 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 948 MiB/s wr, 1.88k op/s 2024-09-21T06:02:01.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.486+0000 7fef2f8ed640 1 -- 172.21.15.117:0/3380881727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef2810ad10 msgr2=0x7fef28116fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:01.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.486+0000 7fef2f8ed640 1 --2- 172.21.15.117:0/3380881727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef2810ad10 0x7fef28116fc0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fef20009900 tx=0x7fef2002b6d0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.487+0000 7fef2f8ed640 1 -- 172.21.15.117:0/3380881727 shutdown_connections 2024-09-21T06:02:01.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.487+0000 7fef2f8ed640 1 --2- 172.21.15.117:0/3380881727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef2810ad10 0x7fef28116fc0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.487+0000 7fef2f8ed640 1 --2- 172.21.15.117:0/3380881727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef2810a3c0 0x7fef2810a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.487+0000 7fef2f8ed640 1 --2- 172.21.15.117:0/3380881727 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef2810f150 0x7fef2810f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.487+0000 7fef2f8ed640 1 -- 172.21.15.117:0/3380881727 >> 172.21.15.117:0/3380881727 conn(0x7fef280712f0 msgr2=0x7fef28073e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:01.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.493+0000 7fef2f8ed640 1 -- 172.21.15.117:0/3380881727 shutdown_connections 2024-09-21T06:02:01.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.493+0000 7fef2f8ed640 1 -- 172.21.15.117:0/3380881727 wait complete. 2024-09-21T06:02:01.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.494+0000 7fef2f8ed640 1 Processor -- start 2024-09-21T06:02:01.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.494+0000 7fef2f8ed640 1 -- start start 2024-09-21T06:02:01.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.494+0000 7fef2f8ed640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef2810a3c0 0x7fef281a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:01.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.495+0000 7fef2f8ed640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef2810f150 0x7fef281a2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:01.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.495+0000 7fef2f8ed640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef281a8160 0x7fef281a8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:01.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.495+0000 7fef2d689640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef2810a3c0 0x7fef281a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:01.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.495+0000 7fef2d689640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef2810a3c0 0x7fef281a29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41078/0 (socket says 172.21.15.117:41078) 2024-09-21T06:02:01.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.495+0000 7fef2d689640 1 -- 172.21.15.117:0/1311498509 learned_addr learned my addr 172.21.15.117:0/1311498509 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:01.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.495+0000 7fef2f8ed640 1 -- 172.21.15.117:0/1311498509 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fef2811a200 con 0x7fef2810a3c0 2024-09-21T06:02:01.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.495+0000 7fef2f8ed640 1 -- 172.21.15.117:0/1311498509 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fef2811a080 con 0x7fef2810f150 2024-09-21T06:02:01.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.495+0000 7fef2f8ed640 1 -- 172.21.15.117:0/1311498509 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fef2811a380 con 0x7fef281a8160 2024-09-21T06:02:01.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.496+0000 7fef2ce88640 1 --2- 172.21.15.117:0/1311498509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef2810f150 0x7fef281a2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:01.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.497+0000 7fef2d689640 1 -- 172.21.15.117:0/1311498509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef281a8160 msgr2=0x7fef281a8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:01.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.497+0000 7fef2d689640 1 --2- 172.21.15.117:0/1311498509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef281a8160 0x7fef281a8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.497+0000 7fef2d689640 1 -- 172.21.15.117:0/1311498509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef2810f150 msgr2=0x7fef281a2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:01.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.497+0000 7fef2d689640 1 --2- 172.21.15.117:0/1311498509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef2810f150 0x7fef281a2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.497+0000 7fef2d689640 1 -- 172.21.15.117:0/1311498509 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef281a8db0 con 0x7fef2810a3c0 2024-09-21T06:02:01.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.497+0000 7fef2d689640 1 --2- 172.21.15.117:0/1311498509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef2810a3c0 0x7fef281a29e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fef1c00b450 tx=0x7fef1c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:01.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.501+0000 7fef267fc640 1 -- 172.21.15.117:0/1311498509 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1c036070 con 0x7fef2810a3c0 2024-09-21T06:02:01.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.501+0000 7fef2f8ed640 1 -- 172.21.15.117:0/1311498509 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef280755f0 con 0x7fef2810a3c0 2024-09-21T06:02:01.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.501+0000 7fef2f8ed640 1 -- 172.21.15.117:0/1311498509 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef28075bb0 con 0x7fef2810a3c0 2024-09-21T06:02:01.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.502+0000 7fef267fc640 1 -- 172.21.15.117:0/1311498509 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef1c032cb0 con 0x7fef2810a3c0 2024-09-21T06:02:01.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.502+0000 7fef267fc640 1 -- 172.21.15.117:0/1311498509 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1c039d20 con 0x7fef2810a3c0 2024-09-21T06:02:01.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.512+0000 7fef267fc640 1 -- 172.21.15.117:0/1311498509 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fef1c039070 con 0x7fef2810a3c0 2024-09-21T06:02:01.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.528+0000 7fef35e61740 1 Processor -- start 2024-09-21T06:02:01.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.537+0000 7fef35e61740 1 -- start start 2024-09-21T06:02:01.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.537+0000 7fef35e61740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f49032070 0x558f493d9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:01.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.538+0000 7fef35e61740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f490091b0 0x558f493d9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:01.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.538+0000 7fef35e61740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f49011390 0x558f493da0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:01.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.538+0000 7fef267fc640 1 --2- 172.21.15.117:0/1311498509 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fef100b6880 0x7fef100b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:01.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.538+0000 7fef35e61740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558f4934fe50 con 0x558f49011390 2024-09-21T06:02:01.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.538+0000 7fef35e61740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558f4933f280 con 0x558f490091b0 2024-09-21T06:02:01.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.538+0000 7fef2de8a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f49032070 0x558f493d9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:01.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.538+0000 7fef2de8a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f49032070 0x558f493d9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:55160/0 (socket says 172.21.15.117:55160) 2024-09-21T06:02:01.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.538+0000 7fef2de8a640 1 -- 172.21.15.117:0/159957142 learned_addr learned my addr 172.21.15.117:0/159957142 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:01.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.538+0000 7fef35e61740 1 -- 172.21.15.117:0/159957142 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558f49343090 con 0x558f49032070 2024-09-21T06:02:01.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.538+0000 7fef2ce88640 1 --2- 172.21.15.117:0/159957142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f49011390 0x558f493da0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:01.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.539+0000 7fef267fc640 1 -- 172.21.15.117:0/1311498509 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fef1c109990 con 0x7fef2810a3c0 2024-09-21T06:02:01.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.540+0000 7fef2d689640 1 --2- 172.21.15.117:0/159957142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f490091b0 0x558f493d9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:01.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.540+0000 7fef2de8a640 1 --2- 172.21.15.117:0/1311498509 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fef100b6880 0x7fef100b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:01.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.542+0000 7fef2ce88640 1 -- 172.21.15.117:0/159957142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f49032070 msgr2=0x558f493d9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:01.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.542+0000 7fef2ce88640 1 --2- 172.21.15.117:0/159957142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f49032070 0x558f493d9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.542+0000 7fef2ce88640 1 -- 172.21.15.117:0/159957142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f490091b0 msgr2=0x558f493d9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:01.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.542+0000 7fef2ce88640 1 --2- 172.21.15.117:0/159957142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f490091b0 0x558f493d9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.542+0000 7fef2ce88640 1 -- 172.21.15.117:0/159957142 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f4909d650 con 0x558f49011390 2024-09-21T06:02:01.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.543+0000 7fef2ce88640 1 --2- 172.21.15.117:0/159957142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f49011390 0x558f493da0d0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fef2806c970 tx=0x7fef18009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf5b6899b6fc140 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:01.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.546+0000 7fef2de8a640 1 --2- 172.21.15.117:0/1311498509 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fef100b6880 0x7fef100b8d90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fef20003e20 tx=0x7fef20005fd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:01.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.546+0000 7fef0f7fe640 1 -- 172.21.15.117:0/159957142 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef18017070 con 0x558f49011390 2024-09-21T06:02:01.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.546+0000 7fef0f7fe640 1 -- 172.21.15.117:0/159957142 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef18013470 con 0x558f49011390 2024-09-21T06:02:01.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.546+0000 7fef0f7fe640 1 -- 172.21.15.117:0/159957142 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef18012610 con 0x558f49011390 2024-09-21T06:02:01.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.547+0000 7fef35e61740 1 -- 172.21.15.117:0/159957142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f49011390 msgr2=0x558f493da0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:01.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.548+0000 7fef35e61740 1 --2- 172.21.15.117:0/159957142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f49011390 0x558f493da0d0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fef2806c970 tx=0x7fef18009de0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.548+0000 7fef35e61740 1 -- 172.21.15.117:0/159957142 shutdown_connections 2024-09-21T06:02:01.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.548+0000 7fef35e61740 1 --2- 172.21.15.117:0/159957142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f49011390 0x558f493da0d0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.548+0000 7fef35e61740 1 --2- 172.21.15.117:0/159957142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f490091b0 0x558f493d9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.548+0000 7fef35e61740 1 --2- 172.21.15.117:0/159957142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f49032070 0x558f493d9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.548+0000 7fef35e61740 1 -- 172.21.15.117:0/159957142 >> 172.21.15.117:0/159957142 conn(0x558f49026460 msgr2=0x558f493d7540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:01.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.548+0000 7fef35e61740 1 -- 172.21.15.117:0/159957142 shutdown_connections 2024-09-21T06:02:01.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.549+0000 7fef35e61740 1 -- 172.21.15.117:0/159957142 wait complete. 2024-09-21T06:02:01.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.549+0000 7fef35e61740 1 -- start start 2024-09-21T06:02:01.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.550+0000 7fef35e61740 1 Processor -- start 2024-09-21T06:02:01.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.551+0000 7fef35e61740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f490091b0 0x558f493e4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:01.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.551+0000 7fef35e61740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f49011390 0x558f493e5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:01.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.551+0000 7fef35e61740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f49032070 0x558f493f1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:01.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.551+0000 7fef35e61740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558f491a0700 con 0x558f49032070 2024-09-21T06:02:01.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.551+0000 7fef35e61740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558f492da110 con 0x558f49011390 2024-09-21T06:02:01.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.551+0000 7fef35e61740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558f493e7e30 con 0x558f490091b0 2024-09-21T06:02:01.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.551+0000 7fef35e61740 10 client.0 _ll_register_callbacks cb 0x558f492f3990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:02:01.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.553+0000 7fef2d689640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f490091b0 0x558f493e4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:01.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.553+0000 7fef2d689640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f490091b0 0x558f493e4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:55168/0 (socket says 172.21.15.117:55168) 2024-09-21T06:02:01.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.553+0000 7fef2d689640 1 -- 172.21.15.117:0/611334380 learned_addr learned my addr 172.21.15.117:0/611334380 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:01.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.555+0000 7fef2ce88640 1 --2- 172.21.15.117:0/611334380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f49011390 0x558f493e5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:01.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.557+0000 7fef2de8a640 1 --2- 172.21.15.117:0/611334380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f49032070 0x558f493f1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:01.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.557+0000 7fef2d689640 1 -- 172.21.15.117:0/611334380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f49011390 msgr2=0x558f493e5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:01.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.557+0000 7fef2d689640 1 --2- 172.21.15.117:0/611334380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f49011390 0x558f493e5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.557+0000 7fef2d689640 1 -- 172.21.15.117:0/611334380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f49032070 msgr2=0x558f493f1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:01.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.557+0000 7fef2d689640 1 --2- 172.21.15.117:0/611334380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f49032070 0x558f493f1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.557+0000 7fef2d689640 1 -- 172.21.15.117:0/611334380 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f49307c40 con 0x558f490091b0 2024-09-21T06:02:01.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.560+0000 7fef2d689640 1 --2- 172.21.15.117:0/611334380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f490091b0 0x558f493e4d40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fef1c0cbdd0 tx=0x7fef1c0cf3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c170ef96b566c46 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:01.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.561+0000 7fef0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-21T06:02:01.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.561+0000 7fef0cff9640 1 -- 172.21.15.117:0/611334380 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1c032e60 con 0x558f490091b0 2024-09-21T06:02:01.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.562+0000 7fef0cff9640 1 -- 172.21.15.117:0/611334380 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fef1c0d0b30 con 0x558f490091b0 2024-09-21T06:02:01.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.562+0000 7fef35e61740 1 -- 172.21.15.117:0/611334380 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef18013470 con 0x558f490091b0 2024-09-21T06:02:01.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.564+0000 7fef0cff9640 1 -- 172.21.15.117:0/611334380 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fef1c040040 con 0x558f490091b0 2024-09-21T06:02:01.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.568+0000 7fef35e61740 10 client.25618 fetch_fsmap learned FSMap version 27 2024-09-21T06:02:01.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.568+0000 7fef35e61740 1 -- 172.21.15.117:0/611334380 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558f493f3bd0 con 0x558f490091b0 2024-09-21T06:02:01.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.575+0000 7fef0cff9640 1 -- 172.21.15.117:0/611334380 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef1c0eae50 con 0x558f490091b0 2024-09-21T06:02:01.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.575+0000 7fef0cff9640 1 -- 172.21.15.117:0/611334380 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fef1c0cb2f0 con 0x558f490091b0 2024-09-21T06:02:01.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.577+0000 7fef35e61740 10 client.25618 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:02:01.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.577+0000 7fef35e61740 10 client.25618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:02:01.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.577+0000 7fef35e61740 20 client.25618 populate_metadata read hostname 'smithi117' 2024-09-21T06:02:01.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.578+0000 7fef35e61740 1 --2- 172.21.15.117:0/611334380 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x558f48ff09f0 0x558f493f7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:01.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.578+0000 7fef35e61740 4 client.25618 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:02:01.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.578+0000 7fef35e61740 1 -- 172.21.15.117:0/611334380 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fef18013470 con 0x558f48ff09f0 2024-09-21T06:02:01.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.579+0000 7fef2ce88640 1 --2- 172.21.15.117:0/611334380 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x558f48ff09f0 0x558f493f7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:01.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.582+0000 7fef2ce88640 1 --2- 172.21.15.117:0/611334380 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x558f48ff09f0 0x558f493f7190 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90feda21d34cd41a server_cookie=128c922fcb71268e in_seq=0 out_seq=0 2024-09-21T06:02:01.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.582+0000 7fef0cff9640 10 client.25618 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:02:01.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.585+0000 7fef0cff9640 1 -- 172.21.15.117:0/611334380 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fef18013470 con 0x558f48ff09f0 2024-09-21T06:02:01.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.585+0000 7fef0cff9640 10 client.25618 handle_command_reply: tid=0 2024-09-21T06:02:01.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.585+0000 7fef35e61740 1 client.25618 shutdown 2024-09-21T06:02:01.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.586+0000 7fef35e61740 1 -- 172.21.15.117:0/611334380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f490091b0 msgr2=0x558f493e4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:01.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.586+0000 7fef35e61740 1 --2- 172.21.15.117:0/611334380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f490091b0 0x558f493e4d40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fef1c0cbdd0 tx=0x7fef1c0cf3d0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.588+0000 7fef35e61740 1 -- 172.21.15.117:0/611334380 shutdown_connections 2024-09-21T06:02:01.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.588+0000 7fef35e61740 1 --2- 172.21.15.117:0/611334380 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x558f48ff09f0 0x558f493f7190 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.588+0000 7fef0cff9640 0 client.25618 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:02:01.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.589+0000 7fef35e61740 1 --2- 172.21.15.117:0/611334380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f49032070 0x558f493f1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.589+0000 7fef35e61740 1 --2- 172.21.15.117:0/611334380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f49011390 0x558f493e5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.589+0000 7fef35e61740 1 --2- 172.21.15.117:0/611334380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f490091b0 0x558f493e4d40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.589+0000 7fef35e61740 1 -- 172.21.15.117:0/611334380 >> 172.21.15.117:0/611334380 conn(0x558f49026460 msgr2=0x558f493dc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:01.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.590+0000 7fef35e61740 1 -- 172.21.15.117:0/611334380 shutdown_connections 2024-09-21T06:02:01.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.590+0000 7fef35e61740 1 -- 172.21.15.117:0/611334380 wait complete. 2024-09-21T06:02:01.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.591+0000 7fef35e61740 20 client.25618 trim_cache size 0 max 16384 2024-09-21T06:02:01.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.636+0000 7fef35e61740 1 Processor -- start 2024-09-21T06:02:01.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.644+0000 7fef35e61740 1 -- start start 2024-09-21T06:02:01.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.645+0000 7fef35e61740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f48ff09f0 0x558f493e5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:01.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.645+0000 7fef35e61740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f490091b0 0x558f493e6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:01.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.646+0000 7fef35e61740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f49011390 0x558f493dad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:01.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.646+0000 7fef2d689640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f48ff09f0 0x558f493e5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:01.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.646+0000 7fef2d689640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f48ff09f0 0x558f493e5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41108/0 (socket says 172.21.15.117:41108) 2024-09-21T06:02:01.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.646+0000 7fef2d689640 1 -- 172.21.15.117:0/742882654 learned_addr learned my addr 172.21.15.117:0/742882654 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:01.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.646+0000 7fef35e61740 1 -- 172.21.15.117:0/742882654 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fef1400e5c0 con 0x558f48ff09f0 2024-09-21T06:02:01.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.646+0000 7fef35e61740 1 -- 172.21.15.117:0/742882654 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fef14010100 con 0x558f490091b0 2024-09-21T06:02:01.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.646+0000 7fef35e61740 1 -- 172.21.15.117:0/742882654 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fef1400ec90 con 0x558f49011390 2024-09-21T06:02:01.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.647+0000 7fef2d689640 1 -- 172.21.15.117:0/742882654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f49011390 msgr2=0x558f493dad90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:02:01.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.647+0000 7fef2d689640 1 --2- 172.21.15.117:0/742882654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f49011390 0x558f493dad90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.647+0000 7fef2d689640 1 -- 172.21.15.117:0/742882654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f490091b0 msgr2=0x558f493e6c20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:02:01.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.647+0000 7fef2d689640 1 --2- 172.21.15.117:0/742882654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f490091b0 0x558f493e6c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.647+0000 7fef2d689640 1 -- 172.21.15.117:0/742882654 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f493da460 con 0x558f48ff09f0 2024-09-21T06:02:01.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.649+0000 7fef2d689640 1 --2- 172.21.15.117:0/742882654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f48ff09f0 0x558f493e5160 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fef1c0cf790 tx=0x7fef1c0cef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce28eca08917354 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:01.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.654+0000 7fef0effd640 1 -- 172.21.15.117:0/742882654 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1c032e60 con 0x558f48ff09f0 2024-09-21T06:02:01.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.654+0000 7fef0effd640 1 -- 172.21.15.117:0/742882654 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef1c0eae50 con 0x558f48ff09f0 2024-09-21T06:02:01.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.655+0000 7fef35e61740 1 -- 172.21.15.117:0/742882654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f48ff09f0 msgr2=0x558f493e5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:01.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.655+0000 7fef35e61740 1 --2- 172.21.15.117:0/742882654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f48ff09f0 0x558f493e5160 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fef1c0cf790 tx=0x7fef1c0cef70 comp rx=0 tx=0).stop 2024-09-21T06:02:01.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.656+0000 7fef35e61740 1 -- 172.21.15.117:0/742882654 shutdown_connections 2024-09-21T06:02:01.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.656+0000 7fef35e61740 1 --2- 172.21.15.117:0/742882654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f49011390 0x558f493dad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.656+0000 7fef35e61740 1 --2- 172.21.15.117:0/742882654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f490091b0 0x558f493e6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.656+0000 7fef35e61740 1 --2- 172.21.15.117:0/742882654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f48ff09f0 0x558f493e5160 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.656+0000 7fef35e61740 1 -- 172.21.15.117:0/742882654 >> 172.21.15.117:0/742882654 conn(0x558f49026460 msgr2=0x558f493da680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:01.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.660+0000 7fef35e61740 1 -- 172.21.15.117:0/742882654 shutdown_connections 2024-09-21T06:02:01.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.660+0000 7fef35e61740 1 -- 172.21.15.117:0/742882654 wait complete. 2024-09-21T06:02:01.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.660+0000 7fef35e61740 1 -- start start 2024-09-21T06:02:01.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.661+0000 7fef35e61740 1 Processor -- start 2024-09-21T06:02:01.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.661+0000 7fef35e61740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f48ff09f0 0x558f493ed850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:01.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.662+0000 7fef35e61740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f490091b0 0x558f493e9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:01.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.662+0000 7fef35e61740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f49011390 0x558f493ea250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:01.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.662+0000 7fef35e61740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fef1400fa30 con 0x558f48ff09f0 2024-09-21T06:02:01.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.662+0000 7fef35e61740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fef1400f360 con 0x558f490091b0 2024-09-21T06:02:01.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.662+0000 7fef35e61740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fef1400c5a0 con 0x558f49011390 2024-09-21T06:02:01.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.662+0000 7fef35e61740 10 client.0 _ll_register_callbacks cb 0x7fef1c0cbe50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:02:01.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.662+0000 7fef2d689640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f48ff09f0 0x558f493ed850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:01.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.662+0000 7fef2d689640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f48ff09f0 0x558f493ed850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41122/0 (socket says 172.21.15.117:41122) 2024-09-21T06:02:01.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.662+0000 7fef2d689640 1 -- 172.21.15.117:0/1229902905 learned_addr learned my addr 172.21.15.117:0/1229902905 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:01.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.663+0000 7fef2de8a640 1 --2- 172.21.15.117:0/1229902905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f49011390 0x558f493ea250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:01.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.668+0000 7fef2ce88640 1 --2- 172.21.15.117:0/1229902905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f490091b0 0x558f493e9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:01.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.669+0000 7fef2d689640 1 -- 172.21.15.117:0/1229902905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f49011390 msgr2=0x558f493ea250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:01.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.669+0000 7fef2d689640 1 --2- 172.21.15.117:0/1229902905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f49011390 0x558f493ea250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.669+0000 7fef2d689640 1 -- 172.21.15.117:0/1229902905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f490091b0 msgr2=0x558f493e9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:01.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.669+0000 7fef2d689640 1 --2- 172.21.15.117:0/1229902905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f490091b0 0x558f493e9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.669+0000 7fef2d689640 1 -- 172.21.15.117:0/1229902905 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f493ea960 con 0x558f48ff09f0 2024-09-21T06:02:01.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.692+0000 7fef2d689640 1 --2- 172.21.15.117:0/1229902905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f48ff09f0 0x558f493ed850 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fef1c0ce6e0 tx=0x7fef1c0d5470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d24f2262c97c01d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:01.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.692+0000 7fef0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:02:01.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.693+0000 7fef0cff9640 1 -- 172.21.15.117:0/1229902905 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1c032e60 con 0x558f48ff09f0 2024-09-21T06:02:01.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.693+0000 7fef0cff9640 1 -- 172.21.15.117:0/1229902905 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fef1c0cea60 con 0x558f48ff09f0 2024-09-21T06:02:01.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.696+0000 7fef35e61740 1 -- 172.21.15.117:0/1229902905 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef1c0eae50 con 0x558f48ff09f0 2024-09-21T06:02:01.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.704+0000 7fef0cff9640 1 -- 172.21.15.117:0/1229902905 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fef1c0d9220 con 0x558f48ff09f0 2024-09-21T06:02:01.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.704+0000 7fef35e61740 10 client.16311 fetch_fsmap learned FSMap version 27 2024-09-21T06:02:01.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.704+0000 7fef35e61740 1 -- 172.21.15.117:0/1229902905 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558f493ea960 con 0x558f48ff09f0 2024-09-21T06:02:01.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.712+0000 7fef0cff9640 1 -- 172.21.15.117:0/1229902905 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef1c0eae50 con 0x558f48ff09f0 2024-09-21T06:02:01.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.714+0000 7fef0cff9640 1 -- 172.21.15.117:0/1229902905 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fef1c0d8960 con 0x558f48ff09f0 2024-09-21T06:02:01.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.714+0000 7fef35e61740 10 client.16311 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:02:01.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.714+0000 7fef35e61740 10 client.16311 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:02:01.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.714+0000 7fef35e61740 20 client.16311 populate_metadata read hostname 'smithi117' 2024-09-21T06:02:01.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.714+0000 7fef35e61740 1 --2- 172.21.15.117:0/1229902905 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x558f49032070 0x558f49464d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:01.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.714+0000 7fef35e61740 4 client.16311 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:02:01.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.714+0000 7fef35e61740 1 -- 172.21.15.117:0/1229902905 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558f49351da0 con 0x558f49032070 2024-09-21T06:02:01.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.724+0000 7fef2ce88640 1 --2- 172.21.15.117:0/1229902905 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x558f49032070 0x558f49464d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:01.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.725+0000 7fef0cff9640 10 client.16311 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:02:01.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.725+0000 7fef2ce88640 1 --2- 172.21.15.117:0/1229902905 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x558f49032070 0x558f49464d50 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10f5cf8e3043fc18 server_cookie=6a100cc2df2d2e68 in_seq=0 out_seq=0 2024-09-21T06:02:01.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.729+0000 7fef0cff9640 1 -- 172.21.15.117:0/1229902905 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1226 (crc 0 0 0) 0x558f49351da0 con 0x558f49032070 2024-09-21T06:02:01.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.729+0000 7fef0cff9640 10 client.16311 handle_command_reply: tid=0 2024-09-21T06:02:01.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.730+0000 7fef35e61740 1 client.16311 shutdown 2024-09-21T06:02:01.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.730+0000 7fef35e61740 1 -- 172.21.15.117:0/1229902905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f48ff09f0 msgr2=0x558f493ed850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:01.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.730+0000 7fef35e61740 1 --2- 172.21.15.117:0/1229902905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f48ff09f0 0x558f493ed850 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fef1c0ce6e0 tx=0x7fef1c0d5470 comp rx=0 tx=0).stop 2024-09-21T06:02:01.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.732+0000 7fef35e61740 1 -- 172.21.15.117:0/1229902905 shutdown_connections 2024-09-21T06:02:01.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.732+0000 7fef35e61740 1 --2- 172.21.15.117:0/1229902905 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x558f49032070 0x558f49464d50 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.732+0000 7fef35e61740 1 --2- 172.21.15.117:0/1229902905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558f49011390 0x558f493ea250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.732+0000 7fef35e61740 1 --2- 172.21.15.117:0/1229902905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558f490091b0 0x558f493e9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.732+0000 7fef35e61740 1 --2- 172.21.15.117:0/1229902905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f48ff09f0 0x558f493ed850 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.732+0000 7fef35e61740 1 -- 172.21.15.117:0/1229902905 >> 172.21.15.117:0/1229902905 conn(0x558f49026460 msgr2=0x558f493e6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:01.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.732+0000 7fef0cff9640 0 client.16311 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:02:01.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.733+0000 7fef35e61740 1 -- 172.21.15.117:0/1229902905 shutdown_connections 2024-09-21T06:02:01.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.733+0000 7fef35e61740 1 -- 172.21.15.117:0/1229902905 wait complete. 2024-09-21T06:02:01.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.733+0000 7fef35e61740 20 client.16311 trim_cache size 0 max 16384 2024-09-21T06:02:01.735 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T06:02:01.736 INFO:teuthology.orchestra.run.smithi117.stdout: "status": "no active scrubs running", 2024-09-21T06:02:01.736 INFO:teuthology.orchestra.run.smithi117.stdout: "scrubs": {}, 2024-09-21T06:02:01.736 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_stats": { 2024-09-21T06:02:01.736 INFO:teuthology.orchestra.run.smithi117.stdout: "d543b0ae-b710-4153-99da-fe5714839c1a": { 2024-09-21T06:02:01.736 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:01:20.574754+0000", 2024-09-21T06:02:01.736 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:02:01.736 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:02:01.736 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:02:01.736 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:02:01.736 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:02:01.736 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:02:01.736 INFO:teuthology.orchestra.run.smithi117.stdout: "ad80d9f1-6aff-46c0-91ab-2c1ab0681039": { 2024-09-21T06:02:01.736 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:59:31.949108+0000", 2024-09-21T06:02:01.736 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:02:01.737 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:02:01.737 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:02:01.737 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:02:01.737 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:02:01.737 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:02:01.737 INFO:teuthology.orchestra.run.smithi117.stdout: "8b3fe95a-b76b-4e94-afd0-72fe35e2dd92": { 2024-09-21T06:02:01.737 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:58:01.246076+0000", 2024-09-21T06:02:01.737 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:02:01.737 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:02:01.737 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:02:01.737 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:02:01.737 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:02:01.737 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:02:01.737 INFO:teuthology.orchestra.run.smithi117.stdout: "b30bfff0-0f6b-4e30-8467-ff7db421eba6": { 2024-09-21T06:02:01.737 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:57:14.939082+0000", 2024-09-21T06:02:01.738 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:02:01.738 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:02:01.738 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:02:01.738 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:02:01.738 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:02:01.738 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:02:01.738 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:02:01.738 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T06:02:01.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.739+0000 7fef2f8ed640 1 -- 172.21.15.117:0/1311498509 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fef100b6880 msgr2=0x7fef100b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:01.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.739+0000 7fef2f8ed640 1 --2- 172.21.15.117:0/1311498509 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fef100b6880 0x7fef100b8d90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fef20003e20 tx=0x7fef20005fd0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.739+0000 7fef2f8ed640 1 -- 172.21.15.117:0/1311498509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef2810a3c0 msgr2=0x7fef281a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:01.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.739+0000 7fef2f8ed640 1 --2- 172.21.15.117:0/1311498509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef2810a3c0 0x7fef281a29e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fef1c00b450 tx=0x7fef1c00b910 comp rx=0 tx=0).stop 2024-09-21T06:02:01.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.741+0000 7fef2f8ed640 1 -- 172.21.15.117:0/1311498509 shutdown_connections 2024-09-21T06:02:01.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.741+0000 7fef2f8ed640 1 --2- 172.21.15.117:0/1311498509 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fef100b6880 0x7fef100b8d90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.741+0000 7fef2f8ed640 1 --2- 172.21.15.117:0/1311498509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef281a8160 0x7fef281a8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.741+0000 7fef2f8ed640 1 --2- 172.21.15.117:0/1311498509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef2810f150 0x7fef281a2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.741+0000 7fef2f8ed640 1 --2- 172.21.15.117:0/1311498509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef2810a3c0 0x7fef281a29e0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:01.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.741+0000 7fef2f8ed640 1 -- 172.21.15.117:0/1311498509 >> 172.21.15.117:0/1311498509 conn(0x7fef280712f0 msgr2=0x7fef2810dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:01.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.742+0000 7fef2f8ed640 1 -- 172.21.15.117:0/1311498509 shutdown_connections 2024-09-21T06:02:01.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:01.742+0000 7fef2f8ed640 1 -- 172.21.15.117:0/1311498509 wait complete. 2024-09-21T06:02:02.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:02:01 smithi117 ceph-mon[30947]: pgmap v468: 97 pgs: 97 active+clean; 40 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 888 MiB/s wr, 1.73k op/s 2024-09-21T06:02:02.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:02:01 smithi138 ceph-mon[34391]: pgmap v468: 97 pgs: 97 active+clean; 40 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 888 MiB/s wr, 1.73k op/s 2024-09-21T06:02:02.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:02:01 smithi192 ceph-mon[34700]: pgmap v468: 97 pgs: 97 active+clean; 40 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 888 MiB/s wr, 1.73k op/s 2024-09-21T06:02:04.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:02:03 smithi138 ceph-mon[34391]: pgmap v469: 97 pgs: 97 active+clean; 40 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 869 MiB/s wr, 1.69k op/s 2024-09-21T06:02:04.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:02:03 smithi192 ceph-mon[34700]: pgmap v469: 97 pgs: 97 active+clean; 40 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 869 MiB/s wr, 1.69k op/s 2024-09-21T06:02:04.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:02:03 smithi117 ceph-mon[30947]: pgmap v469: 97 pgs: 97 active+clean; 40 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 869 MiB/s wr, 1.69k op/s 2024-09-21T06:02:04.821 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:02:04.821 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:02:05.320 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:02:06.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:02:05 smithi138 ceph-mon[34391]: pgmap v470: 97 pgs: 97 active+clean; 40 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 911 MiB/s wr, 1.74k op/s 2024-09-21T06:02:06.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:02:05 smithi192 ceph-mon[34700]: pgmap v470: 97 pgs: 97 active+clean; 40 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 911 MiB/s wr, 1.74k op/s 2024-09-21T06:02:06.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:02:05 smithi117 ceph-mon[30947]: pgmap v470: 97 pgs: 97 active+clean; 40 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 911 MiB/s wr, 1.74k op/s 2024-09-21T06:02:07.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.950+0000 7fb6c8286640 1 -- 172.21.15.117:0/739563374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6c006c6d0 msgr2=0x7fb6c010fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:07.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.950+0000 7fb6c8286640 1 --2- 172.21.15.117:0/739563374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6c006c6d0 0x7fb6c010fa70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8009be0 tx=0x7fb6b802a6d0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.950+0000 7fb6c8286640 1 -- 172.21.15.117:0/739563374 shutdown_connections 2024-09-21T06:02:07.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.950+0000 7fb6c8286640 1 --2- 172.21.15.117:0/739563374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6c006bc70 0x7fb6c006c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.950+0000 7fb6c8286640 1 --2- 172.21.15.117:0/739563374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6c006c6d0 0x7fb6c010fa70 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.950+0000 7fb6c8286640 1 --2- 172.21.15.117:0/739563374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6c0110ea0 0x7fb6c01112b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.950+0000 7fb6c8286640 1 -- 172.21.15.117:0/739563374 >> 172.21.15.117:0/739563374 conn(0x7fb6c006e930 msgr2=0x7fb6c0070d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:07.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.954+0000 7fb6c8286640 1 -- 172.21.15.117:0/739563374 shutdown_connections 2024-09-21T06:02:07.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.954+0000 7fb6c8286640 1 -- 172.21.15.117:0/739563374 wait complete. 2024-09-21T06:02:07.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.955+0000 7fb6c8286640 1 Processor -- start 2024-09-21T06:02:07.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.955+0000 7fb6c8286640 1 -- start start 2024-09-21T06:02:07.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.956+0000 7fb6c8286640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6c006bc70 0x7fb6c011f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:07.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.956+0000 7fb6c8286640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6c006c6d0 0x7fb6c0118b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:07.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.956+0000 7fb6c8286640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6c0110ea0 0x7fb6c01190d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:07.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.956+0000 7fb6c8286640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c01228a0 con 0x7fb6c006c6d0 2024-09-21T06:02:07.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.956+0000 7fb6c8286640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c0122720 con 0x7fb6c0110ea0 2024-09-21T06:02:07.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.956+0000 7fb6c6823640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6c0110ea0 0x7fb6c01190d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:07.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.956+0000 7fb6c6823640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6c0110ea0 0x7fb6c01190d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:52948/0 (socket says 172.21.15.117:52948) 2024-09-21T06:02:07.468 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T06:02:07.468 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":27,"btime":"2024-09-21T05:57:06:651813+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3227057504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3227057504},{"type":"v1","addr":"172.21.15.117:6835","nonce":3227057504}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3303829183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3303829183},{"type":"v1","addr":"172.21.15.117:6837","nonce":3303829183}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T05:55:14.555897+0000","modified":"2024-09-21T05:57:06.599449+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24638,"mds_1":24637,"mds_2":24628,"mds_3":24616,"mds_4":24602},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6839/2314572411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2314572411},{"type":"v1","addr":"172.21.15.117:6839","nonce":2314572411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6841/3378096561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":3378096561},{"type":"v1","addr":"172.21.15.117:6841","nonce":3378096561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/3725629862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3725629862},{"type":"v1","addr":"172.21.15.138:6833","nonce":3725629862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/4276801396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4276801396},{"type":"v1","addr":"172.21.15.138:6835","nonce":4276801396}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6833/2224910879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2224910879},{"type":"v1","addr":"172.21.15.192:6833","nonce":2224910879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.138:6837/2540335374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2540335374},{"type":"v1","addr":"172.21.15.138:6837","nonce":2540335374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24616":{"gid":24616,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.192:6835/3530688860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3530688860},{"type":"v1","addr":"172.21.15.192:6835","nonce":3530688860}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.192:6837/2683860842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2683860842},{"type":"v1","addr":"172.21.15.192:6837","nonce":2683860842}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.192:6839/1727623231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1727623231},{"type":"v1","addr":"172.21.15.192:6839","nonce":1727623231}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/3230372576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":3230372576},{"type":"v1","addr":"172.21.15.138:6839","nonce":3230372576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24602,24628,24616,24637,24638]},"id":1}]} 2024-09-21T06:02:07.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.956+0000 7fb6c6823640 1 -- 172.21.15.117:0/3735140939 learned_addr learned my addr 172.21.15.117:0/3735140939 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:07.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.956+0000 7fb6c8286640 1 -- 172.21.15.117:0/3735140939 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c0122a20 con 0x7fb6c006bc70 2024-09-21T06:02:07.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.959+0000 7fb6c6022640 1 --2- 172.21.15.117:0/3735140939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6c006bc70 0x7fb6c011f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:07.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.959+0000 7fb6c6823640 1 -- 172.21.15.117:0/3735140939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6c006bc70 msgr2=0x7fb6c011f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:07.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.959+0000 7fb6c6823640 1 --2- 172.21.15.117:0/3735140939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6c006bc70 0x7fb6c011f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.959+0000 7fb6c6823640 1 -- 172.21.15.117:0/3735140939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6c006c6d0 msgr2=0x7fb6c0118b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:07.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.959+0000 7fb6c6823640 1 --2- 172.21.15.117:0/3735140939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6c006c6d0 0x7fb6c0118b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.959+0000 7fb6c6823640 1 -- 172.21.15.117:0/3735140939 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c0119770 con 0x7fb6c0110ea0 2024-09-21T06:02:07.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.960+0000 7fb6c6823640 1 --2- 172.21.15.117:0/3735140939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6c0110ea0 0x7fb6c01190d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8009560 tx=0x7fb6b8003ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:07.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.960+0000 7fb6b77fe640 1 -- 172.21.15.117:0/3735140939 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8041020 con 0x7fb6c0110ea0 2024-09-21T06:02:07.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.961+0000 7fb6c8286640 1 -- 172.21.15.117:0/3735140939 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6c011e530 con 0x7fb6c0110ea0 2024-09-21T06:02:07.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.961+0000 7fb6c8286640 1 -- 172.21.15.117:0/3735140939 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6c011e950 con 0x7fb6c0110ea0 2024-09-21T06:02:07.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.962+0000 7fb6b77fe640 1 -- 172.21.15.117:0/3735140939 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6b8030e20 con 0x7fb6c0110ea0 2024-09-21T06:02:07.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.962+0000 7fb6b77fe640 1 -- 172.21.15.117:0/3735140939 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8036040 con 0x7fb6c0110ea0 2024-09-21T06:02:07.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.966+0000 7fb6b77fe640 1 -- 172.21.15.117:0/3735140939 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fb6b8044020 con 0x7fb6c0110ea0 2024-09-21T06:02:07.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.967+0000 7fb6b77fe640 1 --2- 172.21.15.117:0/3735140939 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6ac0b67b0 0x7fb6ac0b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:07.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.967+0000 7fb6b57fa640 1 -- 172.21.15.117:0/3735140939 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6a40052c0 con 0x7fb6c0110ea0 2024-09-21T06:02:07.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.968+0000 7fb6b77fe640 1 -- 172.21.15.117:0/3735140939 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fb6b8108380 con 0x7fb6c0110ea0 2024-09-21T06:02:07.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.968+0000 7fb6c6022640 1 --2- 172.21.15.117:0/3735140939 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6ac0b67b0 0x7fb6ac0b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:07.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.969+0000 7fb6c6022640 1 --2- 172.21.15.117:0/3735140939 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6ac0b67b0 0x7fb6ac0b8cc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc0058c0 tx=0x7fb6bc005be0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:07.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:06.978+0000 7fb6b77fe640 1 -- 172.21.15.117:0/3735140939 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb6b804b020 con 0x7fb6c0110ea0 2024-09-21T06:02:07.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:07.335+0000 7fb6b57fa640 1 -- 172.21.15.117:0/3735140939 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6a4005880 con 0x7fb6c0110ea0 2024-09-21T06:02:07.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:07.337+0000 7fb6b77fe640 1 -- 172.21.15.117:0/3735140939 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+12001 (secure 0 0 0) 0x7fb6b80ce8f0 con 0x7fb6c0110ea0 2024-09-21T06:02:07.470 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 27 2024-09-21T06:02:07.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:07.342+0000 7fb6b57fa640 1 -- 172.21.15.117:0/3735140939 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6ac0b67b0 msgr2=0x7fb6ac0b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:07.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:07.342+0000 7fb6b57fa640 1 --2- 172.21.15.117:0/3735140939 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6ac0b67b0 0x7fb6ac0b8cc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc0058c0 tx=0x7fb6bc005be0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:07.342+0000 7fb6b57fa640 1 -- 172.21.15.117:0/3735140939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6c0110ea0 msgr2=0x7fb6c01190d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:07.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:07.342+0000 7fb6b57fa640 1 --2- 172.21.15.117:0/3735140939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6c0110ea0 0x7fb6c01190d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8009560 tx=0x7fb6b8003ea0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:07.342+0000 7fb6b57fa640 1 -- 172.21.15.117:0/3735140939 shutdown_connections 2024-09-21T06:02:07.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:07.342+0000 7fb6b57fa640 1 --2- 172.21.15.117:0/3735140939 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6ac0b67b0 0x7fb6ac0b8cc0 secure :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc0058c0 tx=0x7fb6bc005be0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:07.342+0000 7fb6b57fa640 1 --2- 172.21.15.117:0/3735140939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6c006c6d0 0x7fb6c0118b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:07.342+0000 7fb6b57fa640 1 --2- 172.21.15.117:0/3735140939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6c0110ea0 0x7fb6c01190d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:07.342+0000 7fb6b57fa640 1 --2- 172.21.15.117:0/3735140939 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6c006bc70 0x7fb6c011f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:07.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:07.342+0000 7fb6b57fa640 1 -- 172.21.15.117:0/3735140939 >> 172.21.15.117:0/3735140939 conn(0x7fb6c006e930 msgr2=0x7fb6c010da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:07.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:07.342+0000 7fb6b57fa640 1 -- 172.21.15.117:0/3735140939 shutdown_connections 2024-09-21T06:02:07.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:07.342+0000 7fb6b57fa640 1 -- 172.21.15.117:0/3735140939 wait complete. 2024-09-21T06:02:08.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:02:07 smithi117 ceph-mon[30947]: pgmap v471: 97 pgs: 97 active+clean; 40 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 921 MiB/s wr, 1.81k op/s 2024-09-21T06:02:08.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:02:07 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3735140939' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:02:08.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:02:07 smithi138 ceph-mon[34391]: pgmap v471: 97 pgs: 97 active+clean; 40 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 921 MiB/s wr, 1.81k op/s 2024-09-21T06:02:08.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:02:07 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3735140939' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:02:08.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:02:07 smithi192 ceph-mon[34700]: pgmap v471: 97 pgs: 97 active+clean; 40 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 921 MiB/s wr, 1.81k op/s 2024-09-21T06:02:08.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:02:07 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3735140939' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:02:10.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:02:09 smithi138 ceph-mon[34391]: pgmap v472: 97 pgs: 97 active+clean; 40 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 925 MiB/s wr, 1.97k op/s 2024-09-21T06:02:10.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:02:09 smithi192 ceph-mon[34700]: pgmap v472: 97 pgs: 97 active+clean; 40 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 925 MiB/s wr, 1.97k op/s 2024-09-21T06:02:10.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:02:09 smithi117 ceph-mon[30947]: pgmap v472: 97 pgs: 97 active+clean; 40 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 925 MiB/s wr, 1.97k op/s 2024-09-21T06:02:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.968+0000 7fca2134b640 1 -- 172.21.15.117:0/1802704563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca140aacf0 msgr2=0x7fca140b2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.968+0000 7fca2134b640 1 --2- 172.21.15.117:0/1802704563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca140aacf0 0x7fca140b2f80 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fca1c066480 tx=0x7fca1c08f840 comp rx=0 tx=0).stop 2024-09-21T06:02:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.968+0000 7fca2134b640 1 -- 172.21.15.117:0/1802704563 shutdown_connections 2024-09-21T06:02:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.968+0000 7fca2134b640 1 --2- 172.21.15.117:0/1802704563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca140aacf0 0x7fca140b2f80 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.968+0000 7fca2134b640 1 --2- 172.21.15.117:0/1802704563 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca14016c50 0x7fca140ab580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.968+0000 7fca2134b640 1 --2- 172.21.15.117:0/1802704563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca140a53e0 0x7fca140a57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.968+0000 7fca2134b640 1 -- 172.21.15.117:0/1802704563 >> 172.21.15.117:0/1802704563 conn(0x7fca1401d0c0 msgr2=0x7fca140a39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.969+0000 7fca2134b640 1 -- 172.21.15.117:0/1802704563 shutdown_connections 2024-09-21T06:02:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.969+0000 7fca2134b640 1 -- 172.21.15.117:0/1802704563 wait complete. 2024-09-21T06:02:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.970+0000 7fca2134b640 1 Processor -- start 2024-09-21T06:02:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.971+0000 7fca2134b640 1 -- start start 2024-09-21T06:02:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.971+0000 7fca2134b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca14016c50 0x7fca140b9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.971+0000 7fca2134b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca140a53e0 0x7fca140b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.972+0000 7fca2134b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca140aacf0 0x7fca140b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.972+0000 7fca2134b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fca140b4dd0 con 0x7fca140aacf0 2024-09-21T06:02:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.972+0000 7fca2134b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fca140b4c50 con 0x7fca140a53e0 2024-09-21T06:02:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.972+0000 7fca2134b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fca140b4f50 con 0x7fca14016c50 2024-09-21T06:02:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.972+0000 7fca20b4a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca140aacf0 0x7fca140b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.972+0000 7fca20b4a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca140aacf0 0x7fca140b5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41944/0 (socket says 172.21.15.117:41944) 2024-09-21T06:02:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.973+0000 7fca20b4a640 1 -- 172.21.15.117:0/1950587764 learned_addr learned my addr 172.21.15.117:0/1950587764 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.973+0000 7fca20b4a640 1 -- 172.21.15.117:0/1950587764 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca14016c50 msgr2=0x7fca140b9520 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:02:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.973+0000 7fca20b4a640 1 --2- 172.21.15.117:0/1950587764 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca14016c50 0x7fca140b9520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.973+0000 7fca20b4a640 1 -- 172.21.15.117:0/1950587764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca140a53e0 msgr2=0x7fca140b57f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:02:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.973+0000 7fca20b4a640 1 --2- 172.21.15.117:0/1950587764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca140a53e0 0x7fca140b57f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.973+0000 7fca20b4a640 1 -- 172.21.15.117:0/1950587764 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca1413f880 con 0x7fca140aacf0 2024-09-21T06:02:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.974+0000 7fca20b4a640 1 --2- 172.21.15.117:0/1950587764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca140aacf0 0x7fca140b5f20 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fca1c000b60 tx=0x7fca1c099660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.975+0000 7fca197fa640 1 -- 172.21.15.117:0/1950587764 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c09d040 con 0x7fca140aacf0 2024-09-21T06:02:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.975+0000 7fca2134b640 1 -- 172.21.15.117:0/1950587764 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca1413fb10 con 0x7fca140aacf0 2024-09-21T06:02:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.975+0000 7fca197fa640 1 -- 172.21.15.117:0/1950587764 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca1c063cb0 con 0x7fca140aacf0 2024-09-21T06:02:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.975+0000 7fca197fa640 1 -- 172.21.15.117:0/1950587764 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c099a30 con 0x7fca140aacf0 2024-09-21T06:02:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.975+0000 7fca2134b640 1 -- 172.21.15.117:0/1950587764 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca1413ffd0 con 0x7fca140aacf0 2024-09-21T06:02:11.981 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.979+0000 7fca295fe740 1 Processor -- start 2024-09-21T06:02:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.979+0000 7fca295fe740 1 -- start start 2024-09-21T06:02:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.980+0000 7fca295fe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56403b3e91b0 0x56403b78eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.980+0000 7fca295fe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56403b3c02f0 0x56403b78f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.980+0000 7fca295fe740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403b3c84d0 0x56403b78f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.980+0000 7fca295fe740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56403b7036f0 con 0x56403b3c02f0 2024-09-21T06:02:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.980+0000 7fca295fe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56403b6f2b20 con 0x56403b3e91b0 2024-09-21T06:02:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.980+0000 7fca295fe740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56403b6f6930 con 0x56403b3c84d0 2024-09-21T06:02:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.982+0000 7fca1b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56403b3e91b0 0x56403b78eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.982+0000 7fca1b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56403b3e91b0 0x56403b78eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:52980/0 (socket says 172.21.15.117:52980) 2024-09-21T06:02:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.982+0000 7fca1b7fe640 1 -- 172.21.15.117:0/1230833147 learned_addr learned my addr 172.21.15.117:0/1230833147 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:11.984 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.982+0000 7fca1b7fe640 1 --2- 172.21.15.117:0/1230833147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403b3c84d0 0x56403b78f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.987+0000 7fca1bfff640 1 --2- 172.21.15.117:0/1230833147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56403b3c02f0 0x56403b78f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.988+0000 7fca1b7fe640 1 -- 172.21.15.117:0/1230833147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403b3c84d0 msgr2=0x56403b78f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.988+0000 7fca1b7fe640 1 --2- 172.21.15.117:0/1230833147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403b3c84d0 0x56403b78f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.988+0000 7fca1b7fe640 1 -- 172.21.15.117:0/1230833147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56403b3c02f0 msgr2=0x56403b78f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.988+0000 7fca1b7fe640 1 --2- 172.21.15.117:0/1230833147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56403b3c02f0 0x56403b78f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.988+0000 7fca1b7fe640 1 -- 172.21.15.117:0/1230833147 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56403b6b8530 con 0x56403b3e91b0 2024-09-21T06:02:11.990 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.989+0000 7fca1b7fe640 1 --2- 172.21.15.117:0/1230833147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56403b3e91b0 0x56403b78eba0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fca08013770 tx=0x7fca0800c270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f15f622394d2f03 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:11.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.990+0000 7fca197fa640 1 -- 172.21.15.117:0/1950587764 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fca1c0a9030 con 0x7fca140aacf0 2024-09-21T06:02:11.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.990+0000 7fca067fc640 1 -- 172.21.15.117:0/1230833147 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca08004220 con 0x56403b3e91b0 2024-09-21T06:02:11.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.991+0000 7fca067fc640 1 -- 172.21.15.117:0/1230833147 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca080043c0 con 0x56403b3e91b0 2024-09-21T06:02:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.991+0000 7fca197fa640 1 --2- 172.21.15.117:0/1950587764 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fca0c0b6800 0x7fca0c0b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.992+0000 7fca295fe740 1 -- 172.21.15.117:0/1230833147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56403b3e91b0 msgr2=0x56403b78eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.992+0000 7fca295fe740 1 --2- 172.21.15.117:0/1230833147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56403b3e91b0 0x56403b78eba0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fca08013770 tx=0x7fca0800c270 comp rx=0 tx=0).stop 2024-09-21T06:02:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.992+0000 7fca197fa640 1 -- 172.21.15.117:0/1950587764 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fca1c16c120 con 0x7fca140aacf0 2024-09-21T06:02:11.995 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.993+0000 7fca1bfff640 1 --2- 172.21.15.117:0/1950587764 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fca0c0b6800 0x7fca0c0b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:11.994+0000 7fca1bfff640 1 --2- 172.21.15.117:0/1950587764 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fca0c0b6800 0x7fca0c0b8d10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fca10005fd0 tx=0x7fca10005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.004+0000 7fca295fe740 1 -- 172.21.15.117:0/1230833147 shutdown_connections 2024-09-21T06:02:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.004+0000 7fca295fe740 1 --2- 172.21.15.117:0/1230833147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56403b3c02f0 0x56403b78f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.004+0000 7fca295fe740 1 --2- 172.21.15.117:0/1230833147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403b3c84d0 0x56403b78f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.004+0000 7fca295fe740 1 --2- 172.21.15.117:0/1230833147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56403b3e91b0 0x56403b78eba0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.004+0000 7fca295fe740 1 -- 172.21.15.117:0/1230833147 >> 172.21.15.117:0/1230833147 conn(0x56403b3dd300 msgr2=0x56403b78ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:12.008 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.006+0000 7fca295fe740 1 -- 172.21.15.117:0/1230833147 shutdown_connections 2024-09-21T06:02:12.009 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.007+0000 7fca295fe740 1 -- 172.21.15.117:0/1230833147 wait complete. 2024-09-21T06:02:12.009 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.007+0000 7fca295fe740 1 -- start start 2024-09-21T06:02:12.009 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.008+0000 7fca295fe740 1 Processor -- start 2024-09-21T06:02:12.010 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.008+0000 7fca295fe740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403b3c02f0 0x56403b79a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:12.010 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.008+0000 7fca295fe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56403b3c84d0 0x56403b79a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:12.010 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.008+0000 7fca295fe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56403b3e91b0 0x56403b79abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:12.010 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.008+0000 7fca295fe740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56403b62aab0 con 0x56403b3e91b0 2024-09-21T06:02:12.010 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.008+0000 7fca295fe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56403b698730 con 0x56403b3c84d0 2024-09-21T06:02:12.011 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.008+0000 7fca295fe740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56403b79d3c0 con 0x56403b3c02f0 2024-09-21T06:02:12.011 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.008+0000 7fca295fe740 10 client.0 _ll_register_callbacks cb 0x56403b6ec6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:02:12.011 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.008+0000 7fca1b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56403b3c84d0 0x56403b79a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:12.011 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.009+0000 7fca1b7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56403b3c84d0 0x56403b79a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:52982/0 (socket says 172.21.15.117:52982) 2024-09-21T06:02:12.011 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.009+0000 7fca1b7fe640 1 -- 172.21.15.117:0/109969372 learned_addr learned my addr 172.21.15.117:0/109969372 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.009+0000 7fca1b7fe640 1 -- 172.21.15.117:0/109969372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403b3c02f0 msgr2=0x56403b79a150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:02:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.009+0000 7fca1b7fe640 1 --2- 172.21.15.117:0/109969372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403b3c02f0 0x56403b79a150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.009+0000 7fca1b7fe640 1 -- 172.21.15.117:0/109969372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56403b3e91b0 msgr2=0x56403b79abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.009+0000 7fca1b7fe640 1 --2- 172.21.15.117:0/109969372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56403b3e91b0 0x56403b79abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.009+0000 7fca1b7fe640 1 -- 172.21.15.117:0/109969372 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56403b454790 con 0x56403b3c84d0 2024-09-21T06:02:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.009+0000 7fca1b7fe640 1 --2- 172.21.15.117:0/109969372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56403b3c84d0 0x56403b79a690 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fca080042d0 tx=0x7fca0800cad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21dad2d043220626 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.009+0000 7fc9fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-21T06:02:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.010+0000 7fc9fb7fe640 1 -- 172.21.15.117:0/109969372 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca08013070 con 0x56403b3c84d0 2024-09-21T06:02:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.010+0000 7fca295fe740 1 -- 172.21.15.117:0/109969372 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca080043c0 con 0x56403b3c84d0 2024-09-21T06:02:12.012 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.010+0000 7fc9fb7fe640 1 -- 172.21.15.117:0/109969372 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fca08013210 con 0x56403b3c84d0 2024-09-21T06:02:12.013 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.011+0000 7fc9fb7fe640 1 -- 172.21.15.117:0/109969372 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fca0802d050 con 0x56403b3c84d0 2024-09-21T06:02:12.013 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.011+0000 7fca295fe740 10 client.25655 fetch_fsmap learned FSMap version 27 2024-09-21T06:02:12.013 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.011+0000 7fca295fe740 1 -- 172.21.15.117:0/109969372 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56403b4cd620 con 0x56403b3c84d0 2024-09-21T06:02:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.012+0000 7fc9fb7fe640 1 -- 172.21.15.117:0/109969372 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca08021020 con 0x56403b3c84d0 2024-09-21T06:02:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.012+0000 7fc9fb7fe640 1 -- 172.21.15.117:0/109969372 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fca0800f040 con 0x56403b3c84d0 2024-09-21T06:02:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.012+0000 7fca295fe740 10 client.25655 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:02:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.012+0000 7fca295fe740 10 client.25655 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:02:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.012+0000 7fca295fe740 20 client.25655 populate_metadata read hostname 'smithi117' 2024-09-21T06:02:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.012+0000 7fca295fe740 1 --2- 172.21.15.117:0/109969372 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56403b3a7d80 0x56403b7ac7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.012+0000 7fca295fe740 4 client.25655 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:02:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.012+0000 7fca295fe740 1 -- 172.21.15.117:0/109969372 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56403b705640 con 0x56403b3a7d80 2024-09-21T06:02:12.015 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.013+0000 7fca1bfff640 1 --2- 172.21.15.117:0/109969372 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56403b3a7d80 0x56403b7ac7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:12.023 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.021+0000 7fca1bfff640 1 --2- 172.21.15.117:0/109969372 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56403b3a7d80 0x56403b7ac7d0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79250365f571f1fe server_cookie=7182412676ff454e in_seq=0 out_seq=0 2024-09-21T06:02:12.023 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.022+0000 7fc9fb7fe640 10 client.25655 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:02:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.026+0000 7fc9fb7fe640 1 -- 172.21.15.117:0/109969372 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56403b705640 con 0x56403b3a7d80 2024-09-21T06:02:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.026+0000 7fc9fb7fe640 10 client.25655 handle_command_reply: tid=0 2024-09-21T06:02:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.026+0000 7fca295fe740 1 client.25655 shutdown 2024-09-21T06:02:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.027+0000 7fca295fe740 1 -- 172.21.15.117:0/109969372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56403b3c84d0 msgr2=0x56403b79a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.027+0000 7fca295fe740 1 --2- 172.21.15.117:0/109969372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56403b3c84d0 0x56403b79a690 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fca080042d0 tx=0x7fca0800cad0 comp rx=0 tx=0).stop 2024-09-21T06:02:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.027+0000 7fca295fe740 1 -- 172.21.15.117:0/109969372 shutdown_connections 2024-09-21T06:02:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.027+0000 7fca295fe740 1 --2- 172.21.15.117:0/109969372 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56403b3a7d80 0x56403b7ac7d0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.027+0000 7fca295fe740 1 --2- 172.21.15.117:0/109969372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56403b3e91b0 0x56403b79abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.027+0000 7fca295fe740 1 --2- 172.21.15.117:0/109969372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56403b3c84d0 0x56403b79a690 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.027+0000 7fca295fe740 1 --2- 172.21.15.117:0/109969372 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403b3c02f0 0x56403b79a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.027+0000 7fca295fe740 1 -- 172.21.15.117:0/109969372 >> 172.21.15.117:0/109969372 conn(0x56403b3dd300 msgr2=0x56403b78fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:12.029 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.027+0000 7fc9fb7fe640 0 client.25655 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:02:12.030 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.028+0000 7fca295fe740 1 -- 172.21.15.117:0/109969372 shutdown_connections 2024-09-21T06:02:12.030 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.028+0000 7fca295fe740 1 -- 172.21.15.117:0/109969372 wait complete. 2024-09-21T06:02:12.030 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.028+0000 7fca295fe740 20 client.25655 trim_cache size 0 max 16384 2024-09-21T06:02:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.096+0000 7fca295fe740 1 Processor -- start 2024-09-21T06:02:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.097+0000 7fca295fe740 1 -- start start 2024-09-21T06:02:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.098+0000 7fca295fe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56403b3c02f0 0x56403b7902e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:12.100 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.099+0000 7fca295fe740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403b3c84d0 0x56403b78c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.099+0000 7fca295fe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56403b3e91b0 0x56403b78c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.099+0000 7fca1bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56403b3c02f0 0x56403b7902e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.099+0000 7fca1bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56403b3c02f0 0x56403b7902e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:52988/0 (socket says 172.21.15.117:52988) 2024-09-21T06:02:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.099+0000 7fca1bfff640 1 -- 172.21.15.117:0/2540840279 learned_addr learned my addr 172.21.15.117:0/2540840279 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.099+0000 7fca1b7fe640 1 --2- 172.21.15.117:0/2540840279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403b3c84d0 0x56403b78c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.099+0000 7fca295fe740 1 -- 172.21.15.117:0/2540840279 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fca0000e740 con 0x56403b3e91b0 2024-09-21T06:02:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.100+0000 7fca295fe740 1 -- 172.21.15.117:0/2540840279 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fca00010280 con 0x56403b3c02f0 2024-09-21T06:02:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.100+0000 7fca295fe740 1 -- 172.21.15.117:0/2540840279 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fca0000ee10 con 0x56403b3c84d0 2024-09-21T06:02:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.100+0000 7fca20b4a640 1 --2- 172.21.15.117:0/2540840279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56403b3e91b0 0x56403b78c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.101+0000 7fca1bfff640 1 -- 172.21.15.117:0/2540840279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403b3c84d0 msgr2=0x56403b78c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.101+0000 7fca1bfff640 1 --2- 172.21.15.117:0/2540840279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403b3c84d0 0x56403b78c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.101+0000 7fca1bfff640 1 -- 172.21.15.117:0/2540840279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56403b3e91b0 msgr2=0x56403b78c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.101+0000 7fca1bfff640 1 --2- 172.21.15.117:0/2540840279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56403b3e91b0 0x56403b78c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.101+0000 7fca1bfff640 1 -- 172.21.15.117:0/2540840279 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56403b79d430 con 0x56403b3c02f0 2024-09-21T06:02:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.105+0000 7fca1bfff640 1 --2- 172.21.15.117:0/2540840279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56403b3c02f0 0x56403b7902e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fca100058c0 tx=0x7fca10015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86cb3c2af68bbb3d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.107+0000 7fca05ffb640 1 -- 172.21.15.117:0/2540840279 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1002b070 con 0x56403b3c02f0 2024-09-21T06:02:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.107+0000 7fca05ffb640 1 -- 172.21.15.117:0/2540840279 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca10011c60 con 0x56403b3c02f0 2024-09-21T06:02:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.108+0000 7fca05ffb640 1 -- 172.21.15.117:0/2540840279 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca10010630 con 0x56403b3c02f0 2024-09-21T06:02:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.109+0000 7fca295fe740 1 -- 172.21.15.117:0/2540840279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56403b3c02f0 msgr2=0x56403b7902e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.109+0000 7fca295fe740 1 --2- 172.21.15.117:0/2540840279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56403b3c02f0 0x56403b7902e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fca100058c0 tx=0x7fca10015000 comp rx=0 tx=0).stop 2024-09-21T06:02:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.109+0000 7fca295fe740 1 -- 172.21.15.117:0/2540840279 shutdown_connections 2024-09-21T06:02:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.109+0000 7fca295fe740 1 --2- 172.21.15.117:0/2540840279 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56403b3e91b0 0x56403b78c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.109+0000 7fca295fe740 1 --2- 172.21.15.117:0/2540840279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403b3c84d0 0x56403b78c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.109+0000 7fca295fe740 1 --2- 172.21.15.117:0/2540840279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56403b3c02f0 0x56403b7902e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.109+0000 7fca295fe740 1 -- 172.21.15.117:0/2540840279 >> 172.21.15.117:0/2540840279 conn(0x56403b3dd300 msgr2=0x56403b78fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.109+0000 7fca295fe740 1 -- 172.21.15.117:0/2540840279 shutdown_connections 2024-09-21T06:02:12.112 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.109+0000 7fca295fe740 1 -- 172.21.15.117:0/2540840279 wait complete. 2024-09-21T06:02:12.112 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.110+0000 7fca295fe740 1 -- start start 2024-09-21T06:02:12.112 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.110+0000 7fca295fe740 1 Processor -- start 2024-09-21T06:02:12.113 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.111+0000 7fca295fe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56403b3c02f0 0x56403b7a2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:12.113 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.111+0000 7fca295fe740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403b3c84d0 0x56403b79f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:12.113 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.111+0000 7fca295fe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56403b3e91b0 0x56403b79f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:12.113 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.111+0000 7fca295fe740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fca0000fbb0 con 0x56403b3c02f0 2024-09-21T06:02:12.113 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.111+0000 7fca295fe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fca0000f4e0 con 0x56403b3e91b0 2024-09-21T06:02:12.113 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.111+0000 7fca295fe740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fca0000c720 con 0x56403b3c84d0 2024-09-21T06:02:12.113 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.111+0000 7fca295fe740 10 client.0 _ll_register_callbacks cb 0x7fca0800cc90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:02:12.114 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.112+0000 7fca1b7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403b3c84d0 0x56403b79f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:12.114 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.112+0000 7fca1b7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403b3c84d0 0x56403b79f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:36254/0 (socket says 172.21.15.117:36254) 2024-09-21T06:02:12.114 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.112+0000 7fca1b7fe640 1 -- 172.21.15.117:0/2466819787 learned_addr learned my addr 172.21.15.117:0/2466819787 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:12.114 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.112+0000 7fca20b4a640 1 --2- 172.21.15.117:0/2466819787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56403b3e91b0 0x56403b79f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:12.114 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.112+0000 7fca1bfff640 1 --2- 172.21.15.117:0/2466819787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56403b3c02f0 0x56403b7a2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:12.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:02:11 smithi138 ceph-mon[34391]: pgmap v473: 97 pgs: 97 active+clean; 40 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 853 MiB/s wr, 2.18k op/s 2024-09-21T06:02:12.115 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.113+0000 7fca1b7fe640 1 -- 172.21.15.117:0/2466819787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56403b3e91b0 msgr2=0x56403b79f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:12.115 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.113+0000 7fca1b7fe640 1 --2- 172.21.15.117:0/2466819787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56403b3e91b0 0x56403b79f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:12.115 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.113+0000 7fca1b7fe640 1 -- 172.21.15.117:0/2466819787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56403b3c02f0 msgr2=0x56403b7a2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:12.115 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.113+0000 7fca1b7fe640 1 --2- 172.21.15.117:0/2466819787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56403b3c02f0 0x56403b7a2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:12.115 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.113+0000 7fca1b7fe640 1 -- 172.21.15.117:0/2466819787 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56403b7a0000 con 0x56403b3c84d0 2024-09-21T06:02:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.118+0000 7fca1b7fe640 1 --2- 172.21.15.117:0/2466819787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403b3c84d0 0x56403b79f350 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fca080096b0 tx=0x7fca08005ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f25be33c93db6fb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.118+0000 7fc9fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-21T06:02:12.121 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.119+0000 7fc9fb7fe640 1 -- 172.21.15.117:0/2466819787 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca08036060 con 0x56403b3c84d0 2024-09-21T06:02:12.122 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.119+0000 7fc9fb7fe640 1 -- 172.21.15.117:0/2466819787 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fca08003b60 con 0x56403b3c84d0 2024-09-21T06:02:12.122 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.120+0000 7fca295fe740 1 -- 172.21.15.117:0/2466819787 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca10011c60 con 0x56403b3c84d0 2024-09-21T06:02:12.123 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.121+0000 7fc9fb7fe640 1 -- 172.21.15.117:0/2466819787 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fca08009d30 con 0x56403b3c84d0 2024-09-21T06:02:12.123 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.121+0000 7fca295fe740 10 client.25630 fetch_fsmap learned FSMap version 27 2024-09-21T06:02:12.123 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.121+0000 7fca295fe740 1 -- 172.21.15.117:0/2466819787 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56403b8160e0 con 0x56403b3c84d0 2024-09-21T06:02:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.123+0000 7fc9fb7fe640 1 -- 172.21.15.117:0/2466819787 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca10011c60 con 0x56403b3c84d0 2024-09-21T06:02:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.125+0000 7fc9fb7fe640 1 -- 172.21.15.117:0/2466819787 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fca08030030 con 0x56403b3c84d0 2024-09-21T06:02:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.125+0000 7fca295fe740 10 client.25630 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:02:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.125+0000 7fca295fe740 10 client.25630 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:02:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.125+0000 7fca295fe740 20 client.25630 populate_metadata read hostname 'smithi117' 2024-09-21T06:02:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.125+0000 7fca295fe740 1 --2- 172.21.15.117:0/2466819787 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56403b37d1c0 0x56403b8187f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.125+0000 7fca295fe740 4 client.25630 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:02:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.125+0000 7fca295fe740 1 -- 172.21.15.117:0/2466819787 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56403b7010f0 con 0x56403b37d1c0 2024-09-21T06:02:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.127+0000 7fca1bfff640 1 --2- 172.21.15.117:0/2466819787 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56403b37d1c0 0x56403b8187f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.133+0000 7fca1bfff640 1 --2- 172.21.15.117:0/2466819787 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56403b37d1c0 0x56403b8187f0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e2de57375b0378f server_cookie=1a0baf0801f20cae in_seq=0 out_seq=0 2024-09-21T06:02:12.136 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:12.134+0000 7fc9fb7fe640 10 client.25630 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:02:12.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:02:11 smithi192 ceph-mon[34700]: pgmap v473: 97 pgs: 97 active+clean; 40 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 853 MiB/s wr, 2.18k op/s 2024-09-21T06:02:12.280 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:02:11 smithi117 ceph-mon[30947]: pgmap v473: 97 pgs: 97 active+clean; 40 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 853 MiB/s wr, 2.18k op/s 2024-09-21T06:02:14.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:02:13 smithi138 ceph-mon[34391]: pgmap v474: 97 pgs: 97 active+clean; 40 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 784 MiB/s wr, 2.32k op/s 2024-09-21T06:02:14.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:02:13 smithi192 ceph-mon[34700]: pgmap v474: 97 pgs: 97 active+clean; 40 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 784 MiB/s wr, 2.32k op/s 2024-09-21T06:02:14.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:02:13 smithi117 ceph-mon[30947]: pgmap v474: 97 pgs: 97 active+clean; 40 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 784 MiB/s wr, 2.32k op/s 2024-09-21T06:02:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:14.834+0000 7fc9fb7fe640 1 -- 172.21.15.117:0/2466819787 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56403b7010f0 con 0x56403b37d1c0 2024-09-21T06:02:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:14.834+0000 7fc9fb7fe640 10 client.25630 handle_command_reply: tid=0 2024-09-21T06:02:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:14.834+0000 7fca295fe740 1 client.25630 shutdown 2024-09-21T06:02:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:14.836+0000 7fca295fe740 1 -- 172.21.15.117:0/2466819787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403b3c84d0 msgr2=0x56403b79f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:14.836+0000 7fca295fe740 1 --2- 172.21.15.117:0/2466819787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403b3c84d0 0x56403b79f350 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fca080096b0 tx=0x7fca08005ba0 comp rx=0 tx=0).stop 2024-09-21T06:02:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:14.836+0000 7fca295fe740 1 -- 172.21.15.117:0/2466819787 shutdown_connections 2024-09-21T06:02:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:14.836+0000 7fca295fe740 1 --2- 172.21.15.117:0/2466819787 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56403b37d1c0 0x56403b8187f0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:14.837+0000 7fca295fe740 1 --2- 172.21.15.117:0/2466819787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56403b3e91b0 0x56403b79f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:14.837+0000 7fca295fe740 1 --2- 172.21.15.117:0/2466819787 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56403b3c84d0 0x56403b79f350 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:14.837+0000 7fca295fe740 1 --2- 172.21.15.117:0/2466819787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56403b3c02f0 0x56403b7a2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:14.837+0000 7fca295fe740 1 -- 172.21.15.117:0/2466819787 >> 172.21.15.117:0/2466819787 conn(0x56403b3dd300 msgr2=0x56403b79b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:14.837+0000 7fc9fb7fe640 0 client.25630 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:02:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:14.837+0000 7fca295fe740 1 -- 172.21.15.117:0/2466819787 shutdown_connections 2024-09-21T06:02:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:14.837+0000 7fca295fe740 1 -- 172.21.15.117:0/2466819787 wait complete. 2024-09-21T06:02:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:14.837+0000 7fca295fe740 20 client.25630 trim_cache size 0 max 16384 2024-09-21T06:02:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T06:02:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T06:02:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T06:02:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T06:02:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:14.838+0000 7fca2134b640 1 -- 172.21.15.117:0/1950587764 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fca0c0b6800 msgr2=0x7fca0c0b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:14.838+0000 7fca2134b640 1 --2- 172.21.15.117:0/1950587764 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fca0c0b6800 0x7fca0c0b8d10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fca10005fd0 tx=0x7fca10005850 comp rx=0 tx=0).stop 2024-09-21T06:02:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:14.838+0000 7fca2134b640 1 -- 172.21.15.117:0/1950587764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca140aacf0 msgr2=0x7fca140b5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:14.838+0000 7fca2134b640 1 --2- 172.21.15.117:0/1950587764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca140aacf0 0x7fca140b5f20 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fca1c000b60 tx=0x7fca1c099660 comp rx=0 tx=0).stop 2024-09-21T06:02:14.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:14.841+0000 7fca2134b640 1 -- 172.21.15.117:0/1950587764 shutdown_connections 2024-09-21T06:02:14.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:14.841+0000 7fca2134b640 1 --2- 172.21.15.117:0/1950587764 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fca0c0b6800 0x7fca0c0b8d10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:14.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:14.841+0000 7fca2134b640 1 --2- 172.21.15.117:0/1950587764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca140aacf0 0x7fca140b5f20 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:14.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:14.841+0000 7fca2134b640 1 --2- 172.21.15.117:0/1950587764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca140a53e0 0x7fca140b57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:14.841+0000 7fca2134b640 1 --2- 172.21.15.117:0/1950587764 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fca14016c50 0x7fca140b9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:14.841+0000 7fca2134b640 1 -- 172.21.15.117:0/1950587764 >> 172.21.15.117:0/1950587764 conn(0x7fca1401d0c0 msgr2=0x7fca140a8160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:14.841+0000 7fca2134b640 1 -- 172.21.15.117:0/1950587764 shutdown_connections 2024-09-21T06:02:14.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:14.841+0000 7fca2134b640 1 -- 172.21.15.117:0/1950587764 wait complete. 2024-09-21T06:02:16.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:02:15 smithi138 ceph-mon[34391]: pgmap v475: 97 pgs: 97 active+clean; 41 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 830 MiB/s wr, 2.89k op/s 2024-09-21T06:02:16.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:02:15 smithi192 ceph-mon[34700]: pgmap v475: 97 pgs: 97 active+clean; 41 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 830 MiB/s wr, 2.89k op/s 2024-09-21T06:02:16.286 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:02:15 smithi117 ceph-mon[30947]: pgmap v475: 97 pgs: 97 active+clean; 41 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 830 MiB/s wr, 2.89k op/s 2024-09-21T06:02:16.452 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T06:02:17.590 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:02:18.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:02:17 smithi138 ceph-mon[34391]: pgmap v476: 97 pgs: 97 active+clean; 43 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 811 MiB/s wr, 3.13k op/s 2024-09-21T06:02:18.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:02:17 smithi192 ceph-mon[34700]: pgmap v476: 97 pgs: 97 active+clean; 43 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 811 MiB/s wr, 3.13k op/s 2024-09-21T06:02:18.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:02:17 smithi117 ceph-mon[30947]: pgmap v476: 97 pgs: 97 active+clean; 43 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 811 MiB/s wr, 3.13k op/s 2024-09-21T06:02:20.028 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:02:19 smithi117 ceph-mon[30947]: pgmap v477: 97 pgs: 97 active+clean; 45 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 800 MiB/s wr, 3.44k op/s 2024-09-21T06:02:20.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:02:19 smithi138 ceph-mon[34391]: pgmap v477: 97 pgs: 97 active+clean; 45 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 800 MiB/s wr, 3.44k op/s 2024-09-21T06:02:20.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:02:19 smithi192 ceph-mon[34700]: pgmap v477: 97 pgs: 97 active+clean; 45 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 800 MiB/s wr, 3.44k op/s 2024-09-21T06:02:21.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.460+0000 7ff3b3dc6640 1 -- 172.21.15.117:0/4008230290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3ac107be0 msgr2=0x7ff3ac1080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:21.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.460+0000 7ff3b3dc6640 1 --2- 172.21.15.117:0/4008230290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3ac107be0 0x7ff3ac1080c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009930 tx=0x7ff39c02a480 comp rx=0 tx=0).stop 2024-09-21T06:02:21.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.460+0000 7ff3b3dc6640 1 -- 172.21.15.117:0/4008230290 shutdown_connections 2024-09-21T06:02:21.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.460+0000 7ff3b3dc6640 1 --2- 172.21.15.117:0/4008230290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3ac107be0 0x7ff3ac1080c0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.460+0000 7ff3b3dc6640 1 --2- 172.21.15.117:0/4008230290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3ac1030f0 0x7ff3ac107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.460+0000 7ff3b3dc6640 1 --2- 172.21.15.117:0/4008230290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3ac1027a0 0x7ff3ac102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.460+0000 7ff3b3dc6640 1 -- 172.21.15.117:0/4008230290 >> 172.21.15.117:0/4008230290 conn(0x7ff3ac079730 msgr2=0x7ff3ac079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:21.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.460+0000 7ff3b3dc6640 1 -- 172.21.15.117:0/4008230290 shutdown_connections 2024-09-21T06:02:21.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.460+0000 7ff3b3dc6640 1 -- 172.21.15.117:0/4008230290 wait complete. 2024-09-21T06:02:21.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.462+0000 7ff3b3dc6640 1 Processor -- start 2024-09-21T06:02:21.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.462+0000 7ff3b3dc6640 1 -- start start 2024-09-21T06:02:21.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.463+0000 7ff3b3dc6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3ac1027a0 0x7ff3ac116cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.463+0000 7ff3b3dc6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3ac1030f0 0x7ff3ac10ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.463+0000 7ff3b3dc6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3ac107be0 0x7ff3ac110520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.463+0000 7ff3b3dc6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac11a080 con 0x7ff3ac107be0 2024-09-21T06:02:21.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.463+0000 7ff3b3dc6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac119f00 con 0x7ff3ac1030f0 2024-09-21T06:02:21.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.463+0000 7ff3b3dc6640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac11a200 con 0x7ff3ac1027a0 2024-09-21T06:02:21.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.463+0000 7ff3b2363640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3ac107be0 0x7ff3ac110520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.463+0000 7ff3b2363640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3ac107be0 0x7ff3ac110520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34394/0 (socket says 172.21.15.117:34394) 2024-09-21T06:02:21.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.463+0000 7ff3b2363640 1 -- 172.21.15.117:0/801573269 learned_addr learned my addr 172.21.15.117:0/801573269 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:21.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.464+0000 7ff3b1361640 1 --2- 172.21.15.117:0/801573269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3ac1030f0 0x7ff3ac10ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.464+0000 7ff3b1b62640 1 --2- 172.21.15.117:0/801573269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3ac1027a0 0x7ff3ac116cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.465+0000 7ff3b2363640 1 -- 172.21.15.117:0/801573269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3ac1027a0 msgr2=0x7ff3ac116cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:21.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.465+0000 7ff3b2363640 1 --2- 172.21.15.117:0/801573269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3ac1027a0 0x7ff3ac116cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.465+0000 7ff3b2363640 1 -- 172.21.15.117:0/801573269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3ac1030f0 msgr2=0x7ff3ac10ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:21.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.465+0000 7ff3b2363640 1 --2- 172.21.15.117:0/801573269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3ac1030f0 0x7ff3ac10ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.465+0000 7ff3b2363640 1 -- 172.21.15.117:0/801573269 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3ac110d40 con 0x7ff3ac107be0 2024-09-21T06:02:21.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.466+0000 7ff3b2363640 1 --2- 172.21.15.117:0/801573269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3ac107be0 0x7ff3ac110520 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff39c005ab0 tx=0x7ff39c003f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:21.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.466+0000 7ff3a2ffd640 1 -- 172.21.15.117:0/801573269 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c040030 con 0x7ff3ac107be0 2024-09-21T06:02:21.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.467+0000 7ff3a2ffd640 1 -- 172.21.15.117:0/801573269 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff39c03c8f0 con 0x7ff3ac107be0 2024-09-21T06:02:21.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.468+0000 7ff3a2ffd640 1 -- 172.21.15.117:0/801573269 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c0318a0 con 0x7ff3ac107be0 2024-09-21T06:02:21.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.470+0000 7ff3b3dc6640 1 -- 172.21.15.117:0/801573269 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3ac115980 con 0x7ff3ac107be0 2024-09-21T06:02:21.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.470+0000 7ff3b3dc6640 1 -- 172.21.15.117:0/801573269 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3ac115de0 con 0x7ff3ac107be0 2024-09-21T06:02:21.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.474+0000 7ff3ba33a740 1 Processor -- start 2024-09-21T06:02:21.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.478+0000 7ff3ba33a740 1 -- start start 2024-09-21T06:02:21.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.478+0000 7ff3ba33a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc09cd2070 0x55fc0a0796a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.478+0000 7ff3ba33a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09ca91b0 0x55fc0a079be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.478+0000 7ff3ba33a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc09cb1390 0x55fc0a07a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.478+0000 7ff3ba33a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fc09fefea0 con 0x55fc09cd2070 2024-09-21T06:02:21.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.478+0000 7ff3ba33a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fc09fdf2d0 con 0x55fc09ca91b0 2024-09-21T06:02:21.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.478+0000 7ff3ba33a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55fc09fe30e0 con 0x55fc09cb1390 2024-09-21T06:02:21.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.478+0000 7ff3a2ffd640 1 -- 172.21.15.117:0/801573269 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7ff39c043030 con 0x7ff3ac107be0 2024-09-21T06:02:21.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.479+0000 7ff3b1b62640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09ca91b0 0x55fc0a079be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.479+0000 7ff3b1b62640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09ca91b0 0x55fc0a079be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:57228/0 (socket says 172.21.15.117:57228) 2024-09-21T06:02:21.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.479+0000 7ff3b1b62640 1 -- 172.21.15.117:0/925409259 learned_addr learned my addr 172.21.15.117:0/925409259 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:21.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.480+0000 7ff3b1361640 1 --2- 172.21.15.117:0/925409259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc09cd2070 0x55fc0a0796a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.480+0000 7ff3b1b62640 1 -- 172.21.15.117:0/925409259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc09cb1390 msgr2=0x55fc0a07a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:21.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.480+0000 7ff3b1361640 1 --2- 172.21.15.117:0/925409259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc09cb1390 0x55fc0a07a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.480+0000 7ff3b1b62640 1 --2- 172.21.15.117:0/925409259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc09cb1390 0x55fc0a07a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.480+0000 7ff3b1b62640 1 -- 172.21.15.117:0/925409259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc09cd2070 msgr2=0x55fc0a0796a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:21.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.480+0000 7ff3b1b62640 1 --2- 172.21.15.117:0/925409259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc09cd2070 0x55fc0a0796a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.480+0000 7ff3b1b62640 1 -- 172.21.15.117:0/925409259 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc09d3d650 con 0x55fc09ca91b0 2024-09-21T06:02:21.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.481+0000 7ff3a2ffd640 1 --2- 172.21.15.117:0/801573269 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff3980b66f0 0x7ff3980b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.482+0000 7ff3a2ffd640 1 -- 172.21.15.117:0/801573269 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7ff39c108020 con 0x7ff3ac107be0 2024-09-21T06:02:21.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.484+0000 7ff3b1b62640 1 --2- 172.21.15.117:0/925409259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09ca91b0 0x55fc0a079be0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4010420 tx=0x7ff3a40108e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6287f5278dc1c13b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:21.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.484+0000 7ff3b1361640 1 --2- 172.21.15.117:0/801573269 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff3980b66f0 0x7ff3980b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.485+0000 7ff3b1361640 1 --2- 172.21.15.117:0/801573269 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff3980b66f0 0x7ff3980b8c00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8005d00 tx=0x7ff3a8014040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:21.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.489+0000 7ff397fff640 1 -- 172.21.15.117:0/925409259 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4016070 con 0x55fc09ca91b0 2024-09-21T06:02:21.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.489+0000 7ff397fff640 1 -- 172.21.15.117:0/925409259 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3a4012bc0 con 0x55fc09ca91b0 2024-09-21T06:02:21.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.490+0000 7ff397fff640 1 -- 172.21.15.117:0/925409259 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a401bcd0 con 0x55fc09ca91b0 2024-09-21T06:02:21.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.491+0000 7ff3ba33a740 1 -- 172.21.15.117:0/925409259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09ca91b0 msgr2=0x55fc0a079be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:21.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.491+0000 7ff3ba33a740 1 --2- 172.21.15.117:0/925409259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09ca91b0 0x55fc0a079be0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4010420 tx=0x7ff3a40108e0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.491+0000 7ff3ba33a740 1 -- 172.21.15.117:0/925409259 shutdown_connections 2024-09-21T06:02:21.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.491+0000 7ff3ba33a740 1 --2- 172.21.15.117:0/925409259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc09cb1390 0x55fc0a07a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.491+0000 7ff3ba33a740 1 --2- 172.21.15.117:0/925409259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09ca91b0 0x55fc0a079be0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.491+0000 7ff3ba33a740 1 --2- 172.21.15.117:0/925409259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc09cd2070 0x55fc0a0796a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.491+0000 7ff3ba33a740 1 -- 172.21.15.117:0/925409259 >> 172.21.15.117:0/925409259 conn(0x55fc09cc6460 msgr2=0x55fc0a077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:21.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.491+0000 7ff3ba33a740 1 -- 172.21.15.117:0/925409259 shutdown_connections 2024-09-21T06:02:21.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.491+0000 7ff3ba33a740 1 -- 172.21.15.117:0/925409259 wait complete. 2024-09-21T06:02:21.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.492+0000 7ff3ba33a740 1 -- start start 2024-09-21T06:02:21.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.492+0000 7ff3ba33a740 1 Processor -- start 2024-09-21T06:02:21.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.493+0000 7ff3ba33a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc09ca91b0 0x55fc0a084d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.493+0000 7ff3ba33a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09cb1390 0x55fc0a085260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.493+0000 7ff3ba33a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc09cd2070 0x55fc0a0857c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.493+0000 7ff3ba33a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fc09e40700 con 0x55fc09ca91b0 2024-09-21T06:02:21.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.493+0000 7ff3ba33a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fc09f7a160 con 0x55fc09cb1390 2024-09-21T06:02:21.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.493+0000 7ff3ba33a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55fc0a087f20 con 0x55fc09cd2070 2024-09-21T06:02:21.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.493+0000 7ff3ba33a740 10 client.0 _ll_register_callbacks cb 0x55fc09f939e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:02:21.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.493+0000 7ff3b2363640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc09cd2070 0x55fc0a0857c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.493+0000 7ff3b2363640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc09cd2070 0x55fc0a0857c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:50210/0 (socket says 172.21.15.117:50210) 2024-09-21T06:02:21.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.493+0000 7ff3b2363640 1 -- 172.21.15.117:0/2006461143 learned_addr learned my addr 172.21.15.117:0/2006461143 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:21.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.493+0000 7ff3b1b62640 1 --2- 172.21.15.117:0/2006461143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc09ca91b0 0x55fc0a084d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.494+0000 7ff3b1b62640 1 -- 172.21.15.117:0/2006461143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc09cd2070 msgr2=0x55fc0a0857c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:21.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.494+0000 7ff3b1b62640 1 --2- 172.21.15.117:0/2006461143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc09cd2070 0x55fc0a0857c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.494+0000 7ff3b1b62640 1 -- 172.21.15.117:0/2006461143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09cb1390 msgr2=0x55fc0a085260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:02:21.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.494+0000 7ff3b1b62640 1 --2- 172.21.15.117:0/2006461143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09cb1390 0x55fc0a085260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.494+0000 7ff3b1b62640 1 -- 172.21.15.117:0/2006461143 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc09fa7c90 con 0x55fc09ca91b0 2024-09-21T06:02:21.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.495+0000 7ff3b1b62640 1 --2- 172.21.15.117:0/2006461143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc09ca91b0 0x55fc0a084d20 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4000f80 tx=0x7ff3a401b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d8c5138f820577f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:21.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.495+0000 7ff3957fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:02:21.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.496+0000 7ff3957fa640 1 -- 172.21.15.117:0/2006461143 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4016040 con 0x55fc09ca91b0 2024-09-21T06:02:21.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.496+0000 7ff3957fa640 1 -- 172.21.15.117:0/2006461143 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7ff3a4004a40 con 0x55fc09ca91b0 2024-09-21T06:02:21.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.496+0000 7ff3ba33a740 1 -- 172.21.15.117:0/2006461143 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3a4012bc0 con 0x55fc09ca91b0 2024-09-21T06:02:21.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.496+0000 7ff3957fa640 1 -- 172.21.15.117:0/2006461143 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff3a401be60 con 0x55fc09ca91b0 2024-09-21T06:02:21.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.496+0000 7ff3ba33a740 10 client.16341 fetch_fsmap learned FSMap version 27 2024-09-21T06:02:21.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.496+0000 7ff3ba33a740 1 -- 172.21.15.117:0/2006461143 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fc0a093bf0 con 0x55fc09ca91b0 2024-09-21T06:02:21.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.496+0000 7ff3957fa640 1 -- 172.21.15.117:0/2006461143 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3a4012bc0 con 0x55fc09ca91b0 2024-09-21T06:02:21.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.497+0000 7ff3957fa640 1 -- 172.21.15.117:0/2006461143 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff3a4023070 con 0x55fc09ca91b0 2024-09-21T06:02:21.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.497+0000 7ff3ba33a740 10 client.16341 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:02:21.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.497+0000 7ff3ba33a740 10 client.16341 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T06:02:21.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.497+0000 7ff3ba33a740 20 client.16341 populate_metadata read hostname 'smithi117' 2024-09-21T06:02:21.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.497+0000 7ff3ba33a740 1 --2- 172.21.15.117:0/2006461143 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x55fc09c909f0 0x55fc0a096f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.497+0000 7ff3ba33a740 4 client.16341 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:02:21.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.497+0000 7ff3ba33a740 1 -- 172.21.15.117:0/2006461143 --> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc09ff1df0 con 0x55fc09c909f0 2024-09-21T06:02:21.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.499+0000 7ff3b1361640 1 --2- 172.21.15.117:0/2006461143 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x55fc09c909f0 0x55fc0a096f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.502+0000 7ff3957fa640 10 client.16341 ms_handle_connect on v2:172.21.15.138:6836/2540335374 2024-09-21T06:02:21.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.502+0000 7ff3b1361640 1 --2- 172.21.15.117:0/2006461143 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x55fc09c909f0 0x55fc0a096f50 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=87669c9f25d0f964 server_cookie=f64f80b661dc670b in_seq=0 out_seq=0 2024-09-21T06:02:21.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.504+0000 7ff3957fa640 1 -- 172.21.15.117:0/2006461143 <== mds.4 v2:172.21.15.138:6836/2540335374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fc09ff1df0 con 0x55fc09c909f0 2024-09-21T06:02:21.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.504+0000 7ff3957fa640 10 client.16341 handle_command_reply: tid=0 2024-09-21T06:02:21.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.504+0000 7ff3ba33a740 1 client.16341 shutdown 2024-09-21T06:02:21.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.505+0000 7ff3ba33a740 1 -- 172.21.15.117:0/2006461143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc09ca91b0 msgr2=0x55fc0a084d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:21.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.505+0000 7ff3ba33a740 1 --2- 172.21.15.117:0/2006461143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc09ca91b0 0x55fc0a084d20 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4000f80 tx=0x7ff3a401b690 comp rx=0 tx=0).stop 2024-09-21T06:02:21.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.505+0000 7ff3ba33a740 1 -- 172.21.15.117:0/2006461143 shutdown_connections 2024-09-21T06:02:21.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.505+0000 7ff3ba33a740 1 --2- 172.21.15.117:0/2006461143 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x55fc09c909f0 0x55fc0a096f50 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.505+0000 7ff3ba33a740 1 --2- 172.21.15.117:0/2006461143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc09cd2070 0x55fc0a0857c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.505+0000 7ff3ba33a740 1 --2- 172.21.15.117:0/2006461143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09cb1390 0x55fc0a085260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.505+0000 7ff3ba33a740 1 --2- 172.21.15.117:0/2006461143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc09ca91b0 0x55fc0a084d20 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.505+0000 7ff3ba33a740 1 -- 172.21.15.117:0/2006461143 >> 172.21.15.117:0/2006461143 conn(0x55fc09cc6460 msgr2=0x55fc0a07c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:21.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.505+0000 7ff3957fa640 0 client.16341 ms_handle_reset on v2:172.21.15.138:6836/2540335374 2024-09-21T06:02:21.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.506+0000 7ff3ba33a740 1 -- 172.21.15.117:0/2006461143 shutdown_connections 2024-09-21T06:02:21.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.506+0000 7ff3ba33a740 1 -- 172.21.15.117:0/2006461143 wait complete. 2024-09-21T06:02:21.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.506+0000 7ff3ba33a740 20 client.16341 trim_cache size 0 max 16384 2024-09-21T06:02:21.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.528+0000 7ff3ba33a740 1 Processor -- start 2024-09-21T06:02:21.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.528+0000 7ff3ba33a740 1 -- start start 2024-09-21T06:02:21.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.529+0000 7ff3ba33a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09ca91b0 0x55fc0a083c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.529+0000 7ff3ba33a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc09cb1390 0x55fc0a07ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.529+0000 7ff3ba33a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc09cd2070 0x55fc0a076bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.530+0000 7ff3b1b62640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09ca91b0 0x55fc0a083c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.530+0000 7ff3b1b62640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09ca91b0 0x55fc0a083c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:57246/0 (socket says 172.21.15.117:57246) 2024-09-21T06:02:21.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.530+0000 7ff3b1361640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc09cb1390 0x55fc0a07ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.530+0000 7ff3b1361640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc09cb1390 0x55fc0a07ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:50214/0 (socket says 172.21.15.117:50214) 2024-09-21T06:02:21.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.530+0000 7ff3b2363640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc09cd2070 0x55fc0a076bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.530+0000 7ff3b2363640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc09cd2070 0x55fc0a076bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34436/0 (socket says 172.21.15.117:34436) 2024-09-21T06:02:21.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.530+0000 7ff3ba33a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1ad1b0 con 0x55fc09cd2070 2024-09-21T06:02:21.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.530+0000 7ff3ba33a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1aecf0 con 0x55fc09ca91b0 2024-09-21T06:02:21.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.530+0000 7ff3ba33a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1ad880 con 0x55fc09cb1390 2024-09-21T06:02:21.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.530+0000 7ff3b1b62640 1 -- 172.21.15.117:0/2349883573 learned_addr learned my addr 172.21.15.117:0/2349883573 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:21.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.531+0000 7ff3b2363640 1 -- 172.21.15.117:0/2349883573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc09cb1390 msgr2=0x55fc0a07ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:21.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.531+0000 7ff3b2363640 1 --2- 172.21.15.117:0/2349883573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc09cb1390 0x55fc0a07ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.531+0000 7ff3b2363640 1 -- 172.21.15.117:0/2349883573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09ca91b0 msgr2=0x55fc0a083c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:21.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.531+0000 7ff3b2363640 1 --2- 172.21.15.117:0/2349883573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09ca91b0 0x55fc0a083c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.531+0000 7ff3b2363640 1 -- 172.21.15.117:0/2349883573 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc0a085fe0 con 0x55fc09cd2070 2024-09-21T06:02:21.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.531+0000 7ff3b2363640 1 --2- 172.21.15.117:0/2349883573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc09cd2070 0x55fc0a076bc0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff39c003820 tx=0x7ff39c0cd6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4898c67329b9114 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:21.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.532+0000 7ff3977fe640 1 -- 172.21.15.117:0/2349883573 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c0d2970 con 0x55fc09cd2070 2024-09-21T06:02:21.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.532+0000 7ff3977fe640 1 -- 172.21.15.117:0/2349883573 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff39c0d78b0 con 0x55fc09cd2070 2024-09-21T06:02:21.535 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.532+0000 7ff3977fe640 1 -- 172.21.15.117:0/2349883573 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c0d7680 con 0x55fc09cd2070 2024-09-21T06:02:21.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.533+0000 7ff3ba33a740 1 -- 172.21.15.117:0/2349883573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc09cd2070 msgr2=0x55fc0a076bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:21.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.533+0000 7ff3ba33a740 1 --2- 172.21.15.117:0/2349883573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc09cd2070 0x55fc0a076bc0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff39c003820 tx=0x7ff39c0cd6c0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.533+0000 7ff3ba33a740 1 -- 172.21.15.117:0/2349883573 shutdown_connections 2024-09-21T06:02:21.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.533+0000 7ff3ba33a740 1 --2- 172.21.15.117:0/2349883573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc09cd2070 0x55fc0a076bc0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.533+0000 7ff3ba33a740 1 --2- 172.21.15.117:0/2349883573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc09cb1390 0x55fc0a07ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.533+0000 7ff3ba33a740 1 --2- 172.21.15.117:0/2349883573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09ca91b0 0x55fc0a083c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.533+0000 7ff3ba33a740 1 -- 172.21.15.117:0/2349883573 >> 172.21.15.117:0/2349883573 conn(0x55fc09cc6460 msgr2=0x55fc0a07a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:21.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.533+0000 7ff3ba33a740 1 -- 172.21.15.117:0/2349883573 shutdown_connections 2024-09-21T06:02:21.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.533+0000 7ff3ba33a740 1 -- 172.21.15.117:0/2349883573 wait complete. 2024-09-21T06:02:21.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.534+0000 7ff3ba33a740 1 -- start start 2024-09-21T06:02:21.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.534+0000 7ff3ba33a740 1 Processor -- start 2024-09-21T06:02:21.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.535+0000 7ff3ba33a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc09ca91b0 0x55fc0a098120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.535+0000 7ff3b1b62640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc09ca91b0 0x55fc0a098120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.535+0000 7ff3b1b62640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc09ca91b0 0x55fc0a098120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34452/0 (socket says 172.21.15.117:34452) 2024-09-21T06:02:21.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.535+0000 7ff3ba33a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc09cb1390 0x55fc0a098660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.535+0000 7ff3b1b62640 1 -- 172.21.15.117:0/1217640132 learned_addr learned my addr 172.21.15.117:0/1217640132 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:21.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.536+0000 7ff3ba33a740 1 --2- 172.21.15.117:0/1217640132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09cd2070 0x55fc0a098ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.536+0000 7ff3ba33a740 1 -- 172.21.15.117:0/1217640132 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1ae620 con 0x55fc09ca91b0 2024-09-21T06:02:21.538 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.536+0000 7ff3ba33a740 1 -- 172.21.15.117:0/1217640132 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1adf50 con 0x55fc09cd2070 2024-09-21T06:02:21.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.536+0000 7ff3ba33a740 1 -- 172.21.15.117:0/1217640132 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ac1ab190 con 0x55fc09cb1390 2024-09-21T06:02:21.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.536+0000 7ff3b1361640 1 --2- 172.21.15.117:0/1217640132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc09cb1390 0x55fc0a098660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.536+0000 7ff3b2363640 1 --2- 172.21.15.117:0/1217640132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09cd2070 0x55fc0a098ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.536+0000 7ff3ba33a740 10 client.0 _ll_register_callbacks cb 0x7ff3a4009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:02:21.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.537+0000 7ff3b1b62640 1 -- 172.21.15.117:0/1217640132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc09cb1390 msgr2=0x55fc0a098660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:21.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.537+0000 7ff3b1b62640 1 --2- 172.21.15.117:0/1217640132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc09cb1390 0x55fc0a098660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.537+0000 7ff3b1b62640 1 -- 172.21.15.117:0/1217640132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09cd2070 msgr2=0x55fc0a098ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:21.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.537+0000 7ff3b1b62640 1 --2- 172.21.15.117:0/1217640132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09cd2070 0x55fc0a098ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.537+0000 7ff3b1b62640 1 -- 172.21.15.117:0/1217640132 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc0a100880 con 0x55fc09ca91b0 2024-09-21T06:02:21.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.537+0000 7ff3b1b62640 1 --2- 172.21.15.117:0/1217640132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc09ca91b0 0x55fc0a098120 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff3a40054f0 tx=0x7ff3a4005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b25dd18391579ea9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:21.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.537+0000 7ff3957fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:02:21.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.537+0000 7ff3ba33a740 1 -- 172.21.15.117:0/1217640132 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff39c0d78b0 con 0x55fc09ca91b0 2024-09-21T06:02:21.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.537+0000 7ff3957fa640 1 -- 172.21.15.117:0/1217640132 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4016040 con 0x55fc09ca91b0 2024-09-21T06:02:21.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.537+0000 7ff3957fa640 1 -- 172.21.15.117:0/1217640132 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7ff3a4010f90 con 0x55fc09ca91b0 2024-09-21T06:02:21.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.537+0000 7ff3957fa640 1 -- 172.21.15.117:0/1217640132 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff3a4011190 con 0x55fc09ca91b0 2024-09-21T06:02:21.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.538+0000 7ff3ba33a740 10 client.16347 fetch_fsmap learned FSMap version 27 2024-09-21T06:02:21.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.538+0000 7ff3ba33a740 1 -- 172.21.15.117:0/1217640132 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55fc0a1019d0 con 0x55fc09ca91b0 2024-09-21T06:02:21.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.538+0000 7ff3957fa640 1 -- 172.21.15.117:0/1217640132 <== mon.0 v2:172.21.15.117:3300/0 4 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff3a4023070 con 0x55fc09ca91b0 2024-09-21T06:02:21.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.539+0000 7ff3ba33a740 10 client.16347 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:02:21.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.539+0000 7ff3ba33a740 10 client.16347 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T06:02:21.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.539+0000 7ff3ba33a740 20 client.16347 populate_metadata read hostname 'smithi117' 2024-09-21T06:02:21.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.539+0000 7ff3ba33a740 1 --2- 172.21.15.117:0/1217640132 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x55fc09c65da0 0x55fc0a104090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:21.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.539+0000 7ff3ba33a740 4 client.16347 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:02:21.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.539+0000 7ff3ba33a740 1 -- 172.21.15.117:0/1217640132 --> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fc09fed8a0 con 0x55fc09c65da0 2024-09-21T06:02:21.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.539+0000 7ff3b1361640 1 --2- 172.21.15.117:0/1217640132 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x55fc09c65da0 0x55fc0a104090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:21.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.541+0000 7ff3b1361640 1 --2- 172.21.15.117:0/1217640132 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x55fc09c65da0 0x55fc0a104090 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=27d68410499708c9 server_cookie=b463822b9062a94c in_seq=0 out_seq=0 2024-09-21T06:02:21.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.541+0000 7ff3957fa640 10 client.16347 ms_handle_connect on v2:172.21.15.138:6836/2540335374 2024-09-21T06:02:21.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.542+0000 7ff3957fa640 1 -- 172.21.15.117:0/1217640132 <== mds.4 v2:172.21.15.138:6836/2540335374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fc09fed8a0 con 0x55fc09c65da0 2024-09-21T06:02:21.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.542+0000 7ff3957fa640 10 client.16347 handle_command_reply: tid=0 2024-09-21T06:02:21.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.542+0000 7ff3ba33a740 1 client.16347 shutdown 2024-09-21T06:02:21.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.542+0000 7ff3ba33a740 1 -- 172.21.15.117:0/1217640132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc09ca91b0 msgr2=0x55fc0a098120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:21.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.542+0000 7ff3ba33a740 1 --2- 172.21.15.117:0/1217640132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc09ca91b0 0x55fc0a098120 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff3a40054f0 tx=0x7ff3a4005b00 comp rx=0 tx=0).stop 2024-09-21T06:02:21.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.543+0000 7ff3ba33a740 1 -- 172.21.15.117:0/1217640132 shutdown_connections 2024-09-21T06:02:21.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.543+0000 7ff3ba33a740 1 --2- 172.21.15.117:0/1217640132 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x55fc09c65da0 0x55fc0a104090 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.543+0000 7ff3ba33a740 1 --2- 172.21.15.117:0/1217640132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc09cd2070 0x55fc0a098ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.543+0000 7ff3ba33a740 1 --2- 172.21.15.117:0/1217640132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc09cb1390 0x55fc0a098660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.547 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:02:21.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.543+0000 7ff3ba33a740 1 --2- 172.21.15.117:0/1217640132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc09ca91b0 0x55fc0a098120 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.543+0000 7ff3957fa640 0 client.16347 ms_handle_reset on v2:172.21.15.138:6836/2540335374 2024-09-21T06:02:21.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.543+0000 7ff3ba33a740 1 -- 172.21.15.117:0/1217640132 >> 172.21.15.117:0/1217640132 conn(0x55fc09cc6460 msgr2=0x55fc0a0861b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:21.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.543+0000 7ff3ba33a740 1 -- 172.21.15.117:0/1217640132 shutdown_connections 2024-09-21T06:02:21.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.543+0000 7ff3ba33a740 1 -- 172.21.15.117:0/1217640132 wait complete. 2024-09-21T06:02:21.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.544+0000 7ff3ba33a740 20 client.16347 trim_cache size 0 max 16384 2024-09-21T06:02:21.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.546+0000 7ff3b3dc6640 1 -- 172.21.15.117:0/801573269 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff3980b66f0 msgr2=0x7ff3980b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:21.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.546+0000 7ff3b3dc6640 1 --2- 172.21.15.117:0/801573269 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff3980b66f0 0x7ff3980b8c00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8005d00 tx=0x7ff3a8014040 comp rx=0 tx=0).stop 2024-09-21T06:02:21.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.546+0000 7ff3b3dc6640 1 -- 172.21.15.117:0/801573269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3ac107be0 msgr2=0x7ff3ac110520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:21.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.546+0000 7ff3b3dc6640 1 --2- 172.21.15.117:0/801573269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3ac107be0 0x7ff3ac110520 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff39c005ab0 tx=0x7ff39c003f00 comp rx=0 tx=0).stop 2024-09-21T06:02:21.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.546+0000 7ff3b3dc6640 1 -- 172.21.15.117:0/801573269 shutdown_connections 2024-09-21T06:02:21.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.546+0000 7ff3b3dc6640 1 --2- 172.21.15.117:0/801573269 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff3980b66f0 0x7ff3980b8c00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.546+0000 7ff3b3dc6640 1 --2- 172.21.15.117:0/801573269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3ac107be0 0x7ff3ac110520 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.546+0000 7ff3b3dc6640 1 --2- 172.21.15.117:0/801573269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3ac1030f0 0x7ff3ac10ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.546+0000 7ff3b3dc6640 1 --2- 172.21.15.117:0/801573269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3ac1027a0 0x7ff3ac116cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:21.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.546+0000 7ff3b3dc6640 1 -- 172.21.15.117:0/801573269 >> 172.21.15.117:0/801573269 conn(0x7ff3ac079730 msgr2=0x7ff3ac1055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:21.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.547+0000 7ff3b3dc6640 1 -- 172.21.15.117:0/801573269 shutdown_connections 2024-09-21T06:02:21.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:21.547+0000 7ff3b3dc6640 1 -- 172.21.15.117:0/801573269 wait complete. 2024-09-21T06:02:22.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:02:21 smithi192 ceph-mon[34700]: pgmap v478: 97 pgs: 97 active+clean; 46 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 705 MiB/s wr, 3.02k op/s 2024-09-21T06:02:22.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:02:21 smithi117 ceph-mon[30947]: pgmap v478: 97 pgs: 97 active+clean; 46 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 705 MiB/s wr, 3.02k op/s 2024-09-21T06:02:22.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:02:21 smithi138 ceph-mon[34391]: pgmap v478: 97 pgs: 97 active+clean; 46 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 705 MiB/s wr, 3.02k op/s 2024-09-21T06:02:24.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:02:23 smithi192 ceph-mon[34700]: pgmap v479: 97 pgs: 97 active+clean; 47 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 718 MiB/s wr, 2.54k op/s 2024-09-21T06:02:24.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:02:23 smithi117 ceph-mon[30947]: pgmap v479: 97 pgs: 97 active+clean; 47 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 718 MiB/s wr, 2.54k op/s 2024-09-21T06:02:24.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:02:23 smithi138 ceph-mon[34391]: pgmap v479: 97 pgs: 97 active+clean; 47 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 718 MiB/s wr, 2.54k op/s 2024-09-21T06:02:26.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:02:25 smithi192 ceph-mon[34700]: pgmap v480: 97 pgs: 97 active+clean; 48 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 846 MiB/s wr, 2.39k op/s 2024-09-21T06:02:26.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:02:25 smithi117 ceph-mon[30947]: pgmap v480: 97 pgs: 97 active+clean; 48 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 846 MiB/s wr, 2.39k op/s 2024-09-21T06:02:26.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:02:25 smithi138 ceph-mon[34391]: pgmap v480: 97 pgs: 97 active+clean; 48 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 846 MiB/s wr, 2.39k op/s 2024-09-21T06:02:27.139 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:02:26 smithi117 ceph-mon[30947]: pgmap v481: 97 pgs: 97 active+clean; 49 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 806 MiB/s wr, 1.97k op/s 2024-09-21T06:02:27.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:02:26 smithi192 ceph-mon[34700]: pgmap v481: 97 pgs: 97 active+clean; 49 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 806 MiB/s wr, 1.97k op/s 2024-09-21T06:02:27.282 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T06:02:27.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:02:26 smithi138 ceph-mon[34391]: pgmap v481: 97 pgs: 97 active+clean; 49 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 806 MiB/s wr, 1.97k op/s 2024-09-21T06:02:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.153+0000 7fedefef9640 1 -- 172.21.15.117:0/2237199657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fede8111ca0 msgr2=0x7fede810a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.153+0000 7fedefef9640 1 --2- 172.21.15.117:0/2237199657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fede8111ca0 0x7fede810a3c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fede4009900 tx=0x7fede402a400 comp rx=0 tx=0).stop 2024-09-21T06:02:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.153+0000 7fedefef9640 1 -- 172.21.15.117:0/2237199657 shutdown_connections 2024-09-21T06:02:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.153+0000 7fedefef9640 1 --2- 172.21.15.117:0/2237199657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fede811cba0 0x7fede811efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.153+0000 7fedefef9640 1 --2- 172.21.15.117:0/2237199657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fede810a900 0x7fede810ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.153+0000 7fedefef9640 1 --2- 172.21.15.117:0/2237199657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fede8111ca0 0x7fede810a3c0 secure :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fede4009900 tx=0x7fede402a400 comp rx=0 tx=0).stop 2024-09-21T06:02:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.153+0000 7fedefef9640 1 -- 172.21.15.117:0/2237199657 >> 172.21.15.117:0/2237199657 conn(0x7fede8070c30 msgr2=0x7fede8071040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.157+0000 7fedefef9640 1 -- 172.21.15.117:0/2237199657 shutdown_connections 2024-09-21T06:02:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.159+0000 7fedefef9640 1 -- 172.21.15.117:0/2237199657 wait complete. 2024-09-21T06:02:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.160+0000 7fedefef9640 1 Processor -- start 2024-09-21T06:02:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.160+0000 7fedefef9640 1 -- start start 2024-09-21T06:02:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.160+0000 7fedefef9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fede810a900 0x7fede811c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.160+0000 7fedefef9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fede811cba0 0x7fede81157f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.160+0000 7fedefef9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fede8115e70 0x7fede811b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.160+0000 7fedefef9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fede8122670 con 0x7fede810a900 2024-09-21T06:02:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.160+0000 7fedefef9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fede81224f0 con 0x7fede811cba0 2024-09-21T06:02:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.160+0000 7fedefef9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fede81227f0 con 0x7fede8115e70 2024-09-21T06:02:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.161+0000 7fedee496640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fede8115e70 0x7fede811b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.162+0000 7fedee496640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fede8115e70 0x7fede811b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:48774/0 (socket says 172.21.15.117:48774) 2024-09-21T06:02:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.162+0000 7fedee496640 1 -- 172.21.15.117:0/757773046 learned_addr learned my addr 172.21.15.117:0/757773046 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.163+0000 7fedee496640 1 -- 172.21.15.117:0/757773046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fede811cba0 msgr2=0x7fede81157f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:02:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.163+0000 7fedee496640 1 --2- 172.21.15.117:0/757773046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fede811cba0 0x7fede81157f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.163+0000 7fedee496640 1 -- 172.21.15.117:0/757773046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fede810a900 msgr2=0x7fede811c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:28.165 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.163+0000 7fedee496640 1 --2- 172.21.15.117:0/757773046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fede810a900 0x7fede811c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:28.166 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.163+0000 7fedee496640 1 -- 172.21.15.117:0/757773046 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede811b6d0 con 0x7fede8115e70 2024-09-21T06:02:28.167 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.165+0000 7fedee496640 1 --2- 172.21.15.117:0/757773046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fede8115e70 0x7fede811b190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fede000eaa0 tx=0x7fede000ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:28.169 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.167+0000 7feddeffd640 1 -- 172.21.15.117:0/757773046 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede000cdb0 con 0x7fede8115e70 2024-09-21T06:02:28.169 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.167+0000 7feddeffd640 1 -- 172.21.15.117:0/757773046 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fede0005020 con 0x7fede8115e70 2024-09-21T06:02:28.169 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.167+0000 7feddeffd640 1 -- 172.21.15.117:0/757773046 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede00116d0 con 0x7fede8115e70 2024-09-21T06:02:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.169+0000 7fedefef9640 1 -- 172.21.15.117:0/757773046 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede811b9c0 con 0x7fede8115e70 2024-09-21T06:02:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.169+0000 7fedefef9640 1 -- 172.21.15.117:0/757773046 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede811bec0 con 0x7fede8115e70 2024-09-21T06:02:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.174+0000 7fedf6e6b740 1 Processor -- start 2024-09-21T06:02:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.180+0000 7feddeffd640 1 -- 172.21.15.117:0/757773046 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fede0004b20 con 0x7fede8115e70 2024-09-21T06:02:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.182+0000 7feddeffd640 1 --2- 172.21.15.117:0/757773046 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fedd40b6850 0x7fedd40b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.182+0000 7fedf6e6b740 1 -- start start 2024-09-21T06:02:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.183+0000 7fedf6e6b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a8da141b0 0x555a8ddb9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.183+0000 7fedf6e6b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a8d9eb2f0 0x555a8ddba0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.183+0000 7fedf6e6b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a8d9f34d0 0x555a8ddba620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.183+0000 7fedf6e6b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555a8dd2e6f0 con 0x555a8d9eb2f0 2024-09-21T06:02:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.183+0000 7fedf6e6b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555a8dd1db20 con 0x555a8d9f34d0 2024-09-21T06:02:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.183+0000 7fedf6e6b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555a8dd21930 con 0x555a8da141b0 2024-09-21T06:02:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.185+0000 7feded494640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a8d9eb2f0 0x555a8ddba0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.185+0000 7feded494640 1 --2- 172.21.15.117:0/757773046 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fedd40b6850 0x7fedd40b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.185+0000 7feddeffd640 1 -- 172.21.15.117:0/757773046 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fede00e75c0 con 0x7fede8115e70 2024-09-21T06:02:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.187+0000 7feded494640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a8d9eb2f0 0x555a8ddba0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49132/0 (socket says 172.21.15.117:49132) 2024-09-21T06:02:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.187+0000 7feded494640 1 -- 172.21.15.117:0/2804204069 learned_addr learned my addr 172.21.15.117:0/2804204069 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.187+0000 7fededc95640 1 --2- 172.21.15.117:0/2804204069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a8da141b0 0x555a8ddb9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.189+0000 7fedee496640 1 --2- 172.21.15.117:0/2804204069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a8d9f34d0 0x555a8ddba620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.191+0000 7feded494640 1 -- 172.21.15.117:0/2804204069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a8da141b0 msgr2=0x555a8ddb9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.191+0000 7feded494640 1 --2- 172.21.15.117:0/2804204069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a8da141b0 0x555a8ddb9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.191+0000 7feded494640 1 -- 172.21.15.117:0/2804204069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a8d9f34d0 msgr2=0x555a8ddba620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.191+0000 7feded494640 1 --2- 172.21.15.117:0/2804204069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a8d9f34d0 0x555a8ddba620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.191+0000 7feded494640 1 -- 172.21.15.117:0/2804204069 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a8dce3530 con 0x555a8d9eb2f0 2024-09-21T06:02:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.194+0000 7feded494640 1 --2- 172.21.15.117:0/2804204069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a8d9eb2f0 0x555a8ddba0e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fedd8012fa0 tx=0x7fedd8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=635782057ed6143c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.194+0000 7feded494640 1 --2- 172.21.15.117:0/757773046 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fedd40b6850 0x7fedd40b8d60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fede806c460 tx=0x7fedd800c2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.195+0000 7fedd3fff640 1 -- 172.21.15.117:0/2804204069 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd8012270 con 0x555a8d9eb2f0 2024-09-21T06:02:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.195+0000 7fedd3fff640 1 -- 172.21.15.117:0/2804204069 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedd8024070 con 0x555a8d9eb2f0 2024-09-21T06:02:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.195+0000 7fedd3fff640 1 -- 172.21.15.117:0/2804204069 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd801f6e0 con 0x555a8d9eb2f0 2024-09-21T06:02:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.196+0000 7fedf6e6b740 1 -- 172.21.15.117:0/2804204069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a8d9eb2f0 msgr2=0x555a8ddba0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.196+0000 7fedf6e6b740 1 --2- 172.21.15.117:0/2804204069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a8d9eb2f0 0x555a8ddba0e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fedd8012fa0 tx=0x7fedd8019000 comp rx=0 tx=0).stop 2024-09-21T06:02:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.197+0000 7fedf6e6b740 1 -- 172.21.15.117:0/2804204069 shutdown_connections 2024-09-21T06:02:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.197+0000 7fedf6e6b740 1 --2- 172.21.15.117:0/2804204069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a8d9eb2f0 0x555a8ddba0e0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.197+0000 7fedf6e6b740 1 --2- 172.21.15.117:0/2804204069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a8d9f34d0 0x555a8ddba620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.197+0000 7fedf6e6b740 1 --2- 172.21.15.117:0/2804204069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a8da141b0 0x555a8ddb9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.197+0000 7fedf6e6b740 1 -- 172.21.15.117:0/2804204069 >> 172.21.15.117:0/2804204069 conn(0x555a8da08300 msgr2=0x555a8ddb7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:28.204 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.201+0000 7fedf6e6b740 1 -- 172.21.15.117:0/2804204069 shutdown_connections 2024-09-21T06:02:28.206 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.203+0000 7fedf6e6b740 1 -- 172.21.15.117:0/2804204069 wait complete. 2024-09-21T06:02:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.203+0000 7fedf6e6b740 1 -- start start 2024-09-21T06:02:28.208 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.203+0000 7fedf6e6b740 1 Processor -- start 2024-09-21T06:02:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.205+0000 7fedf6e6b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a8d9eb2f0 0x555a8ddc5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.206+0000 7fedf6e6b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a8d9f34d0 0x555a8ddc5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.206+0000 7fedf6e6b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a8da141b0 0x555a8ddd1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.206+0000 7fedf6e6b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555a8dc55ab0 con 0x555a8d9f34d0 2024-09-21T06:02:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.206+0000 7fedf6e6b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555a8dcc3730 con 0x555a8da141b0 2024-09-21T06:02:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.206+0000 7fedf6e6b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555a8ddc8310 con 0x555a8d9eb2f0 2024-09-21T06:02:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.206+0000 7fedf6e6b740 10 client.0 _ll_register_callbacks cb 0x555a8dd176b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:02:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.207+0000 7fedee496640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a8da141b0 0x555a8ddd1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.207+0000 7fedee496640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a8da141b0 0x555a8ddd1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:48444/0 (socket says 172.21.15.117:48444) 2024-09-21T06:02:28.211 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.207+0000 7fedee496640 1 -- 172.21.15.117:0/1553249927 learned_addr learned my addr 172.21.15.117:0/1553249927 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.209+0000 7fedee496640 1 -- 172.21.15.117:0/1553249927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a8d9eb2f0 msgr2=0x555a8ddc5250 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:02:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.209+0000 7fedee496640 1 --2- 172.21.15.117:0/1553249927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a8d9eb2f0 0x555a8ddc5250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.209+0000 7fedee496640 1 -- 172.21.15.117:0/1553249927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a8d9f34d0 msgr2=0x555a8ddc5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.209+0000 7fedee496640 1 --2- 172.21.15.117:0/1553249927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a8d9f34d0 0x555a8ddc5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.209+0000 7fedee496640 1 -- 172.21.15.117:0/1553249927 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a8da7f790 con 0x555a8da141b0 2024-09-21T06:02:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.212+0000 7fedee496640 1 --2- 172.21.15.117:0/1553249927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a8da141b0 0x555a8ddd1650 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fede00094f0 tx=0x7fede00ad070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7168db11709e4de server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:28.214 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.212+0000 7fedd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-21T06:02:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.213+0000 7fedd17fa640 1 -- 172.21.15.117:0/1553249927 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede001f040 con 0x555a8da141b0 2024-09-21T06:02:28.217 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.213+0000 7fedd17fa640 1 -- 172.21.15.117:0/1553249927 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fede0094b70 con 0x555a8da141b0 2024-09-21T06:02:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.216+0000 7fedf6e6b740 1 -- 172.21.15.117:0/1553249927 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedd8024070 con 0x555a8da141b0 2024-09-21T06:02:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.216+0000 7fedd17fa640 1 -- 172.21.15.117:0/1553249927 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fede00b1f10 con 0x555a8da141b0 2024-09-21T06:02:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.217+0000 7fedf6e6b740 10 client.25682 fetch_fsmap learned FSMap version 27 2024-09-21T06:02:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.217+0000 7fedf6e6b740 1 -- 172.21.15.117:0/1553249927 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555a8daf8620 con 0x555a8da141b0 2024-09-21T06:02:28.222 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.217+0000 7fedd17fa640 1 -- 172.21.15.117:0/1553249927 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedd8024070 con 0x555a8da141b0 2024-09-21T06:02:28.222 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.218+0000 7fedd17fa640 1 -- 172.21.15.117:0/1553249927 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fede000f040 con 0x555a8da141b0 2024-09-21T06:02:28.222 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.220+0000 7fedf6e6b740 10 client.25682 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:02:28.222 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.220+0000 7fedf6e6b740 10 client.25682 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:02:28.223 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.220+0000 7fedf6e6b740 20 client.25682 populate_metadata read hostname 'smithi117' 2024-09-21T06:02:28.223 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.220+0000 7fedf6e6b740 1 --2- 172.21.15.117:0/1553249927 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555a8d9d2d80 0x555a8ddd7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:28.223 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.220+0000 7fedf6e6b740 4 client.25682 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:02:28.223 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.220+0000 7fedf6e6b740 1 -- 172.21.15.117:0/1553249927 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a8dd30640 con 0x555a8d9d2d80 2024-09-21T06:02:28.224 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.222+0000 7fededc95640 1 --2- 172.21.15.117:0/1553249927 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555a8d9d2d80 0x555a8ddd7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.235+0000 7fededc95640 1 --2- 172.21.15.117:0/1553249927 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555a8d9d2d80 0x555a8ddd7480 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9763c7c1d250383 server_cookie=1042b75a571ca1fe in_seq=0 out_seq=0 2024-09-21T06:02:28.238 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.236+0000 7fedd17fa640 10 client.25682 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:02:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.241+0000 7fedd17fa640 1 -- 172.21.15.117:0/1553249927 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555a8dd30640 con 0x555a8d9d2d80 2024-09-21T06:02:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.241+0000 7fedd17fa640 10 client.25682 handle_command_reply: tid=0 2024-09-21T06:02:28.244 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.242+0000 7fedf6e6b740 1 client.25682 shutdown 2024-09-21T06:02:28.250 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.248+0000 7fedf6e6b740 1 -- 172.21.15.117:0/1553249927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a8da141b0 msgr2=0x555a8ddd1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:28.250 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.248+0000 7fedf6e6b740 1 --2- 172.21.15.117:0/1553249927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a8da141b0 0x555a8ddd1650 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fede00094f0 tx=0x7fede00ad070 comp rx=0 tx=0).stop 2024-09-21T06:02:28.250 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.248+0000 7fedf6e6b740 1 -- 172.21.15.117:0/1553249927 shutdown_connections 2024-09-21T06:02:28.250 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.248+0000 7fedf6e6b740 1 --2- 172.21.15.117:0/1553249927 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555a8d9d2d80 0x555a8ddd7480 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:28.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.248+0000 7fedf6e6b740 1 --2- 172.21.15.117:0/1553249927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a8d9f34d0 0x555a8ddc5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:28.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.248+0000 7fedf6e6b740 1 --2- 172.21.15.117:0/1553249927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a8da141b0 0x555a8ddd1650 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:28.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.248+0000 7fedf6e6b740 1 --2- 172.21.15.117:0/1553249927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a8d9eb2f0 0x555a8ddc5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:28.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.248+0000 7fedf6e6b740 1 -- 172.21.15.117:0/1553249927 >> 172.21.15.117:0/1553249927 conn(0x555a8da08300 msgr2=0x555a8ddbabd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:28.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.248+0000 7fedd17fa640 0 client.25682 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:02:28.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.248+0000 7fedf6e6b740 1 -- 172.21.15.117:0/1553249927 shutdown_connections 2024-09-21T06:02:28.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.248+0000 7fedf6e6b740 1 -- 172.21.15.117:0/1553249927 wait complete. 2024-09-21T06:02:28.255 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.252+0000 7fedf6e6b740 20 client.25682 trim_cache size 0 max 16384 2024-09-21T06:02:28.335 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.333+0000 7fedf6e6b740 1 Processor -- start 2024-09-21T06:02:28.336 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.335+0000 7fedf6e6b740 1 -- start start 2024-09-21T06:02:28.338 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.336+0000 7fedf6e6b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a8d9eb2f0 0x555a8ddc5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:28.338 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.336+0000 7fedf6e6b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a8d9f34d0 0x555a8ddbb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:28.338 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.336+0000 7fedf6e6b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a8da141b0 0x555a8ddc6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:28.339 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.336+0000 7fedf6e6b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fedcc00e700 con 0x555a8d9eb2f0 2024-09-21T06:02:28.339 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.336+0000 7fedf6e6b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fedcc010240 con 0x555a8da141b0 2024-09-21T06:02:28.339 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.336+0000 7fedf6e6b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fedcc00edd0 con 0x555a8d9f34d0 2024-09-21T06:02:28.340 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.338+0000 7feded494640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a8d9f34d0 0x555a8ddbb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:28.340 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.338+0000 7feded494640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a8d9f34d0 0x555a8ddbb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:48814/0 (socket says 172.21.15.117:48814) 2024-09-21T06:02:28.340 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.338+0000 7feded494640 1 -- 172.21.15.117:0/4212873999 learned_addr learned my addr 172.21.15.117:0/4212873999 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:28.340 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.338+0000 7fededc95640 1 --2- 172.21.15.117:0/4212873999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a8d9eb2f0 0x555a8ddc5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.339+0000 7feded494640 1 -- 172.21.15.117:0/4212873999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a8da141b0 msgr2=0x555a8ddc6e80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:02:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.339+0000 7feded494640 1 --2- 172.21.15.117:0/4212873999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a8da141b0 0x555a8ddc6e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.339+0000 7feded494640 1 -- 172.21.15.117:0/4212873999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a8d9eb2f0 msgr2=0x555a8ddc5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.339+0000 7feded494640 1 --2- 172.21.15.117:0/4212873999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a8d9eb2f0 0x555a8ddc5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.339+0000 7feded494640 1 -- 172.21.15.117:0/4212873999 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a8ddbbea0 con 0x555a8d9f34d0 2024-09-21T06:02:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.343+0000 7feded494640 1 --2- 172.21.15.117:0/4212873999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a8d9f34d0 0x555a8ddbb590 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fedd8000c00 tx=0x7fedd8012280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8fdd998bfbcaa1d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:28.354 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.351+0000 7fedd37fe640 1 -- 172.21.15.117:0/4212873999 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd802c020 con 0x555a8d9f34d0 2024-09-21T06:02:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.351+0000 7fedd37fe640 1 -- 172.21.15.117:0/4212873999 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedd8024070 con 0x555a8d9f34d0 2024-09-21T06:02:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.351+0000 7fedd37fe640 1 -- 172.21.15.117:0/4212873999 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd801cbf0 con 0x555a8d9f34d0 2024-09-21T06:02:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.354+0000 7fedf6e6b740 1 -- 172.21.15.117:0/4212873999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a8d9f34d0 msgr2=0x555a8ddbb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.354+0000 7fedf6e6b740 1 --2- 172.21.15.117:0/4212873999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a8d9f34d0 0x555a8ddbb590 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fedd8000c00 tx=0x7fedd8012280 comp rx=0 tx=0).stop 2024-09-21T06:02:28.361 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.359+0000 7fedf6e6b740 1 -- 172.21.15.117:0/4212873999 shutdown_connections 2024-09-21T06:02:28.361 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.359+0000 7fedf6e6b740 1 --2- 172.21.15.117:0/4212873999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a8da141b0 0x555a8ddc6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:28.361 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.359+0000 7fedf6e6b740 1 --2- 172.21.15.117:0/4212873999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a8d9f34d0 0x555a8ddbb590 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:28.361 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.359+0000 7fedf6e6b740 1 --2- 172.21.15.117:0/4212873999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a8d9eb2f0 0x555a8ddc5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:28.361 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.359+0000 7fedf6e6b740 1 -- 172.21.15.117:0/4212873999 >> 172.21.15.117:0/4212873999 conn(0x555a8da08300 msgr2=0x555a8ddc6770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:28.372 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.370+0000 7fedf6e6b740 1 -- 172.21.15.117:0/4212873999 shutdown_connections 2024-09-21T06:02:28.372 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.370+0000 7fedf6e6b740 1 -- 172.21.15.117:0/4212873999 wait complete. 2024-09-21T06:02:28.372 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.371+0000 7fedf6e6b740 1 -- start start 2024-09-21T06:02:28.373 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.371+0000 7fedf6e6b740 1 Processor -- start 2024-09-21T06:02:28.374 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.372+0000 7fedf6e6b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a8d9eb2f0 0x555a8ddcda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:28.374 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.372+0000 7fedf6e6b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a8d9f34d0 0x555a8ddc8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.373+0000 7fedf6e6b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a8da141b0 0x555a8ddc9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.373+0000 7fedf6e6b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fedcc00fb70 con 0x555a8d9f34d0 2024-09-21T06:02:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.373+0000 7fedf6e6b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fedcc00f4a0 con 0x555a8d9eb2f0 2024-09-21T06:02:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.373+0000 7fedf6e6b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fedcc00c6e0 con 0x555a8da141b0 2024-09-21T06:02:28.375 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.373+0000 7fedf6e6b740 10 client.0 _ll_register_callbacks cb 0x7fede00acc30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:02:28.377 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.374+0000 7fededc95640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a8d9eb2f0 0x555a8ddcda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.374+0000 7fededc95640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a8d9eb2f0 0x555a8ddcda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:48458/0 (socket says 172.21.15.117:48458) 2024-09-21T06:02:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.374+0000 7fededc95640 1 -- 172.21.15.117:0/2393707954 learned_addr learned my addr 172.21.15.117:0/2393707954 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.374+0000 7feded494640 1 --2- 172.21.15.117:0/2393707954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a8d9f34d0 0x555a8ddc8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:28.378 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.376+0000 7fedee496640 1 --2- 172.21.15.117:0/2393707954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a8da141b0 0x555a8ddc9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.377+0000 7fededc95640 1 -- 172.21.15.117:0/2393707954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a8da141b0 msgr2=0x555a8ddc9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.377+0000 7fededc95640 1 --2- 172.21.15.117:0/2393707954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a8da141b0 0x555a8ddc9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:28.379 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.377+0000 7fededc95640 1 -- 172.21.15.117:0/2393707954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a8d9f34d0 msgr2=0x555a8ddc8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.377+0000 7fededc95640 1 --2- 172.21.15.117:0/2393707954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a8d9f34d0 0x555a8ddc8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.377+0000 7fededc95640 1 -- 172.21.15.117:0/2393707954 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a8ddc99b0 con 0x555a8d9eb2f0 2024-09-21T06:02:28.381 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.379+0000 7fededc95640 1 --2- 172.21.15.117:0/2393707954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a8d9eb2f0 0x555a8ddcda90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fede4006010 tx=0x7fede4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b6f893e017ad0e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:28.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.382+0000 7fedd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-21T06:02:28.388 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.382+0000 7fedd17fa640 1 -- 172.21.15.117:0/2393707954 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede4035020 con 0x555a8d9eb2f0 2024-09-21T06:02:28.388 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.386+0000 7fedd17fa640 1 -- 172.21.15.117:0/2393707954 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fede40054b0 con 0x555a8d9eb2f0 2024-09-21T06:02:28.389 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.387+0000 7fedf6e6b740 1 -- 172.21.15.117:0/2393707954 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedd8024070 con 0x555a8d9eb2f0 2024-09-21T06:02:28.393 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.390+0000 7fedd17fa640 1 -- 172.21.15.117:0/2393707954 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fede40316d0 con 0x555a8d9eb2f0 2024-09-21T06:02:28.393 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.390+0000 7fedf6e6b740 10 client.25685 fetch_fsmap learned FSMap version 27 2024-09-21T06:02:28.393 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.390+0000 7fedf6e6b740 1 -- 172.21.15.117:0/2393707954 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555a8de41a90 con 0x555a8d9eb2f0 2024-09-21T06:02:28.396 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.393+0000 7fedd17fa640 1 -- 172.21.15.117:0/2393707954 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedd8024070 con 0x555a8d9eb2f0 2024-09-21T06:02:28.397 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.395+0000 7fedd17fa640 1 -- 172.21.15.117:0/2393707954 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fede403a040 con 0x555a8d9eb2f0 2024-09-21T06:02:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.396+0000 7fedf6e6b740 10 client.25685 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:02:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.396+0000 7fedf6e6b740 10 client.25685 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:02:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.396+0000 7fedf6e6b740 20 client.25685 populate_metadata read hostname 'smithi117' 2024-09-21T06:02:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.396+0000 7fedf6e6b740 1 --2- 172.21.15.117:0/2393707954 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555a8d9a81c0 0x555a8de44250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.396+0000 7fedf6e6b740 4 client.25685 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:02:28.399 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.396+0000 7fedf6e6b740 1 -- 172.21.15.117:0/2393707954 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555a8dd2c0f0 con 0x555a8d9a81c0 2024-09-21T06:02:28.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.402+0000 7feded494640 1 --2- 172.21.15.117:0/2393707954 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555a8d9a81c0 0x555a8de44250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.412+0000 7feded494640 1 --2- 172.21.15.117:0/2393707954 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555a8d9a81c0 0x555a8de44250 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e4cdb987698e117 server_cookie=9a14eef9c0596bdc in_seq=0 out_seq=0 2024-09-21T06:02:28.417 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:02:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:28.416+0000 7fedd17fa640 10 client.25685 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:02:30.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:02:29 smithi138 ceph-mon[34391]: pgmap v482: 97 pgs: 97 active+clean; 49 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 893 MiB/s wr, 1.99k op/s 2024-09-21T06:02:30.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:02:29 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:02:30.219 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:02:29 smithi117 ceph-mon[30947]: pgmap v482: 97 pgs: 97 active+clean; 49 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 893 MiB/s wr, 1.99k op/s 2024-09-21T06:02:30.219 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:02:29 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:02:30.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:02:29 smithi192 ceph-mon[34700]: pgmap v482: 97 pgs: 97 active+clean; 49 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 893 MiB/s wr, 1.99k op/s 2024-09-21T06:02:30.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:02:29 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:02:31.931 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:02:31 smithi117 ceph-mon[30947]: pgmap v483: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 818 MiB/s wr, 1.45k op/s 2024-09-21T06:02:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:32.075+0000 7fedd17fa640 1 -- 172.21.15.117:0/2393707954 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555a8dd2c0f0 con 0x555a8d9a81c0 2024-09-21T06:02:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:32.075+0000 7fedd17fa640 10 client.25685 handle_command_reply: tid=0 2024-09-21T06:02:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:32.075+0000 7fedf6e6b740 1 client.25685 shutdown 2024-09-21T06:02:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:32.077+0000 7fedf6e6b740 1 -- 172.21.15.117:0/2393707954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a8d9eb2f0 msgr2=0x555a8ddcda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:32.077+0000 7fedf6e6b740 1 --2- 172.21.15.117:0/2393707954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a8d9eb2f0 0x555a8ddcda90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fede4006010 tx=0x7fede4000bc0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:32.077+0000 7fedf6e6b740 1 -- 172.21.15.117:0/2393707954 shutdown_connections 2024-09-21T06:02:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:32.077+0000 7fedf6e6b740 1 --2- 172.21.15.117:0/2393707954 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x555a8d9a81c0 0x555a8de44250 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:32.077+0000 7fedf6e6b740 1 --2- 172.21.15.117:0/2393707954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555a8d9f34d0 0x555a8ddc8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:32.077+0000 7fedf6e6b740 1 --2- 172.21.15.117:0/2393707954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a8da141b0 0x555a8ddc9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:32.077+0000 7fedf6e6b740 1 --2- 172.21.15.117:0/2393707954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555a8d9eb2f0 0x555a8ddcda90 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:32.077+0000 7fedf6e6b740 1 -- 172.21.15.117:0/2393707954 >> 172.21.15.117:0/2393707954 conn(0x555a8da08300 msgr2=0x555a8ddc7060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:32.077+0000 7fedd17fa640 0 client.25685 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:02:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:32.077+0000 7fedf6e6b740 1 -- 172.21.15.117:0/2393707954 shutdown_connections 2024-09-21T06:02:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:32.078+0000 7fedf6e6b740 1 -- 172.21.15.117:0/2393707954 wait complete. 2024-09-21T06:02:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:32.078+0000 7fedf6e6b740 20 client.25685 trim_cache size 0 max 16384 2024-09-21T06:02:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T06:02:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T06:02:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T06:02:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T06:02:32.095 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:32.093+0000 7fedefef9640 1 -- 172.21.15.117:0/757773046 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fedd40b6850 msgr2=0x7fedd40b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:32.095 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:32.093+0000 7fedefef9640 1 --2- 172.21.15.117:0/757773046 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fedd40b6850 0x7fedd40b8d60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fede806c460 tx=0x7fedd800c2d0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.095 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:32.093+0000 7fedefef9640 1 -- 172.21.15.117:0/757773046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fede8115e70 msgr2=0x7fede811b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:32.095 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:32.093+0000 7fedefef9640 1 --2- 172.21.15.117:0/757773046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fede8115e70 0x7fede811b190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fede000eaa0 tx=0x7fede000ef60 comp rx=0 tx=0).stop 2024-09-21T06:02:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:32.093+0000 7fedefef9640 1 -- 172.21.15.117:0/757773046 shutdown_connections 2024-09-21T06:02:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:32.093+0000 7fedefef9640 1 --2- 172.21.15.117:0/757773046 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fedd40b6850 0x7fedd40b8d60 secure :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fede806c460 tx=0x7fedd800c2d0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:32.093+0000 7fedefef9640 1 --2- 172.21.15.117:0/757773046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fede8115e70 0x7fede811b190 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:32.093+0000 7fedefef9640 1 --2- 172.21.15.117:0/757773046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fede811cba0 0x7fede81157f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:32.093+0000 7fedefef9640 1 --2- 172.21.15.117:0/757773046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fede810a900 0x7fede811c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:32.093+0000 7fedefef9640 1 -- 172.21.15.117:0/757773046 >> 172.21.15.117:0/757773046 conn(0x7fede8070c30 msgr2=0x7fede810b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:32.093+0000 7fedefef9640 1 -- 172.21.15.117:0/757773046 shutdown_connections 2024-09-21T06:02:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:32.093+0000 7fedefef9640 1 -- 172.21.15.117:0/757773046 wait complete. 2024-09-21T06:02:32.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:02:31 smithi138 ceph-mon[34391]: pgmap v483: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 818 MiB/s wr, 1.45k op/s 2024-09-21T06:02:32.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:02:31 smithi192 ceph-mon[34700]: pgmap v483: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 818 MiB/s wr, 1.45k op/s 2024-09-21T06:02:32.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.477+0000 7f59dad72640 1 -- 172.21.15.117:0/3072702336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59d4109fc0 msgr2=0x7f59d410a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:32.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.478+0000 7f59dad72640 1 --2- 172.21.15.117:0/3072702336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59d4109fc0 0x7f59d410a3d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f59d0009860 tx=0x7f59d002a360 comp rx=0 tx=0).stop 2024-09-21T06:02:32.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.480+0000 7f59dad72640 1 -- 172.21.15.117:0/3072702336 shutdown_connections 2024-09-21T06:02:32.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.480+0000 7f59dad72640 1 --2- 172.21.15.117:0/3072702336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59d4110250 0x7f59d4116a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.480+0000 7f59dad72640 1 --2- 172.21.15.117:0/3072702336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59d410f900 0x7f59d410fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.480+0000 7f59dad72640 1 --2- 172.21.15.117:0/3072702336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59d4109fc0 0x7f59d410a3d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.480+0000 7f59dad72640 1 -- 172.21.15.117:0/3072702336 >> 172.21.15.117:0/3072702336 conn(0x7f59d40795c0 msgr2=0x7f59d41081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:32.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.480+0000 7f59dad72640 1 -- 172.21.15.117:0/3072702336 shutdown_connections 2024-09-21T06:02:32.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.480+0000 7f59dad72640 1 -- 172.21.15.117:0/3072702336 wait complete. 2024-09-21T06:02:32.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.483+0000 7f59dad72640 1 Processor -- start 2024-09-21T06:02:32.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.483+0000 7f59dad72640 1 -- start start 2024-09-21T06:02:32.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.484+0000 7f59dad72640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59d4109fc0 0x7f59d41a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:32.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.485+0000 7f59dad72640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59d410f900 0x7f59d41a0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:32.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.485+0000 7f59dad72640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59d4110250 0x7f59d41a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:32.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.485+0000 7f59dad72640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f59d4119fc0 con 0x7f59d4109fc0 2024-09-21T06:02:32.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.485+0000 7f59dad72640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f59d4119e40 con 0x7f59d410f900 2024-09-21T06:02:32.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.485+0000 7f59dad72640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f59d411a140 con 0x7f59d4110250 2024-09-21T06:02:32.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.489+0000 7f59d9d70640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59d4109fc0 0x7f59d41a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:32.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.489+0000 7f59d9d70640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59d4109fc0 0x7f59d41a0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49188/0 (socket says 172.21.15.117:49188) 2024-09-21T06:02:32.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.489+0000 7f59d9d70640 1 -- 172.21.15.117:0/1149864496 learned_addr learned my addr 172.21.15.117:0/1149864496 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:32.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.490+0000 7f59d956f640 1 --2- 172.21.15.117:0/1149864496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59d410f900 0x7f59d41a0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:32.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.493+0000 7f59da571640 1 --2- 172.21.15.117:0/1149864496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59d4110250 0x7f59d41a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:32.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.493+0000 7f59d956f640 1 -- 172.21.15.117:0/1149864496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59d4110250 msgr2=0x7f59d41a61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:32.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.493+0000 7f59d956f640 1 --2- 172.21.15.117:0/1149864496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59d4110250 0x7f59d41a61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.493+0000 7f59d956f640 1 -- 172.21.15.117:0/1149864496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59d4109fc0 msgr2=0x7f59d41a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:32.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.493+0000 7f59d956f640 1 --2- 172.21.15.117:0/1149864496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59d4109fc0 0x7f59d41a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.493+0000 7f59d956f640 1 -- 172.21.15.117:0/1149864496 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d41a6900 con 0x7f59d410f900 2024-09-21T06:02:32.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.495+0000 7f59d956f640 1 --2- 172.21.15.117:0/1149864496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59d410f900 0x7f59d41a0f30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f59d406b920 tx=0x7f59c400ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:32.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.496+0000 7f59caffd640 1 -- 172.21.15.117:0/1149864496 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c400cc90 con 0x7f59d410f900 2024-09-21T06:02:32.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.497+0000 7f59caffd640 1 -- 172.21.15.117:0/1149864496 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59c400ee70 con 0x7f59d410f900 2024-09-21T06:02:32.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.497+0000 7f59caffd640 1 -- 172.21.15.117:0/1149864496 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c4018680 con 0x7f59d410f900 2024-09-21T06:02:32.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.497+0000 7f59dad72640 1 -- 172.21.15.117:0/1149864496 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59d41a6bf0 con 0x7f59d410f900 2024-09-21T06:02:32.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.498+0000 7f59dad72640 1 -- 172.21.15.117:0/1149864496 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59d41ad350 con 0x7f59d410f900 2024-09-21T06:02:32.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.506+0000 7f59caffd640 1 -- 172.21.15.117:0/1149864496 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f59c401f020 con 0x7f59d410f900 2024-09-21T06:02:32.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.508+0000 7f59caffd640 1 --2- 172.21.15.117:0/1149864496 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f59c00b6880 0x7f59c00b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:32.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.508+0000 7f59caffd640 1 -- 172.21.15.117:0/1149864496 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f59c40e73d0 con 0x7f59d410f900 2024-09-21T06:02:32.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.509+0000 7f59d9d70640 1 --2- 172.21.15.117:0/1149864496 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f59c00b6880 0x7f59c00b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:32.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.511+0000 7f59d9d70640 1 --2- 172.21.15.117:0/1149864496 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f59c00b6880 0x7f59c00b8d90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f59d0005eb0 tx=0x7f59d002a860 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:32.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.521+0000 7f59e2548740 1 Processor -- start 2024-09-21T06:02:32.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.522+0000 7f59e2548740 1 -- start start 2024-09-21T06:02:32.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.522+0000 7f59e2548740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556895bed070 0x556895f94460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:32.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.522+0000 7f59e2548740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556895bc41b0 0x556895f949a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:32.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.522+0000 7f59e2548740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556895bcc390 0x556895f94ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:32.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.522+0000 7f59e2548740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556895f0aea0 con 0x556895bed070 2024-09-21T06:02:32.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.522+0000 7f59e2548740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556895efa2d0 con 0x556895bcc390 2024-09-21T06:02:32.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.523+0000 7f59e2548740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556895efe0e0 con 0x556895bc41b0 2024-09-21T06:02:32.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.526+0000 7f59da571640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556895bcc390 0x556895f94ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:32.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.526+0000 7f59da571640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556895bcc390 0x556895f94ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:48484/0 (socket says 172.21.15.117:48484) 2024-09-21T06:02:32.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.526+0000 7f59da571640 1 -- 172.21.15.117:0/3786305948 learned_addr learned my addr 172.21.15.117:0/3786305948 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:32.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.527+0000 7f59d956f640 1 --2- 172.21.15.117:0/3786305948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556895bc41b0 0x556895f949a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:32.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.528+0000 7f59da571640 1 -- 172.21.15.117:0/3786305948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556895bc41b0 msgr2=0x556895f949a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:32.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.528+0000 7f59da571640 1 --2- 172.21.15.117:0/3786305948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556895bc41b0 0x556895f949a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.528+0000 7f59da571640 1 -- 172.21.15.117:0/3786305948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556895bed070 msgr2=0x556895f94460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:32.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.528+0000 7f59da571640 1 --2- 172.21.15.117:0/3786305948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556895bed070 0x556895f94460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.528+0000 7f59da571640 1 -- 172.21.15.117:0/3786305948 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556895c58650 con 0x556895bcc390 2024-09-21T06:02:32.534 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.531+0000 7f59da571640 1 --2- 172.21.15.117:0/3786305948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556895bcc390 0x556895f94ee0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f59cc009f20 tx=0x7f59cc00c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b030b4d1daaacbc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:32.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.537+0000 7f59bffff640 1 -- 172.21.15.117:0/3786305948 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59cc01a070 con 0x556895bcc390 2024-09-21T06:02:32.539 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.537+0000 7f59bffff640 1 -- 172.21.15.117:0/3786305948 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59cc0158c0 con 0x556895bcc390 2024-09-21T06:02:32.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.537+0000 7f59bffff640 1 -- 172.21.15.117:0/3786305948 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59cc00a960 con 0x556895bcc390 2024-09-21T06:02:32.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.541+0000 7f59e2548740 1 -- 172.21.15.117:0/3786305948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556895bcc390 msgr2=0x556895f94ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:32.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.541+0000 7f59e2548740 1 --2- 172.21.15.117:0/3786305948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556895bcc390 0x556895f94ee0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f59cc009f20 tx=0x7f59cc00c8b0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.544+0000 7f59e2548740 1 -- 172.21.15.117:0/3786305948 shutdown_connections 2024-09-21T06:02:32.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.544+0000 7f59e2548740 1 --2- 172.21.15.117:0/3786305948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556895bcc390 0x556895f94ee0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.544+0000 7f59e2548740 1 --2- 172.21.15.117:0/3786305948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556895bc41b0 0x556895f949a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.544+0000 7f59e2548740 1 --2- 172.21.15.117:0/3786305948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556895bed070 0x556895f94460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.544+0000 7f59e2548740 1 -- 172.21.15.117:0/3786305948 >> 172.21.15.117:0/3786305948 conn(0x556895be1460 msgr2=0x556895f92350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:32.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.544+0000 7f59e2548740 1 -- 172.21.15.117:0/3786305948 shutdown_connections 2024-09-21T06:02:32.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.544+0000 7f59e2548740 1 -- 172.21.15.117:0/3786305948 wait complete. 2024-09-21T06:02:32.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.545+0000 7f59e2548740 1 -- start start 2024-09-21T06:02:32.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.546+0000 7f59e2548740 1 Processor -- start 2024-09-21T06:02:32.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.547+0000 7f59e2548740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556895bc41b0 0x556895f9fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:32.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.547+0000 7f59e2548740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556895bcc390 0x556895fa0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:32.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.547+0000 7f59e2548740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556895bed070 0x556895fbcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:32.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.547+0000 7f59e2548740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556895d5b700 con 0x556895bed070 2024-09-21T06:02:32.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.547+0000 7f59e2548740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556895e95160 con 0x556895bcc390 2024-09-21T06:02:32.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.547+0000 7f59e2548740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556895fb3c50 con 0x556895bc41b0 2024-09-21T06:02:32.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.547+0000 7f59e2548740 10 client.0 _ll_register_callbacks cb 0x556895eae9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:02:32.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.549+0000 7f59d9d70640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556895bc41b0 0x556895f9fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:32.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.549+0000 7f59d9d70640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556895bc41b0 0x556895f9fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:48842/0 (socket says 172.21.15.117:48842) 2024-09-21T06:02:32.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.549+0000 7f59d9d70640 1 -- 172.21.15.117:0/3545681369 learned_addr learned my addr 172.21.15.117:0/3545681369 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:32.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.550+0000 7f59da571640 1 --2- 172.21.15.117:0/3545681369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556895bed070 0x556895fbcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:32.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.551+0000 7f59d9d70640 1 -- 172.21.15.117:0/3545681369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556895bcc390 msgr2=0x556895fa0120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:02:32.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.551+0000 7f59d9d70640 1 --2- 172.21.15.117:0/3545681369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556895bcc390 0x556895fa0120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.551+0000 7f59d9d70640 1 -- 172.21.15.117:0/3545681369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556895bed070 msgr2=0x556895fbcfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:32.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.551+0000 7f59d9d70640 1 --2- 172.21.15.117:0/3545681369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556895bed070 0x556895fbcfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.551+0000 7f59d9d70640 1 -- 172.21.15.117:0/3545681369 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556895ec2c90 con 0x556895bc41b0 2024-09-21T06:02:32.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.555+0000 7f59d9d70640 1 --2- 172.21.15.117:0/3545681369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556895bc41b0 0x556895f9fbe0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f59d0035040 tx=0x7f59d0005e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ead2bead38816803 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:32.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.557+0000 7f59bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-21T06:02:32.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.557+0000 7f59bd7fa640 1 -- 172.21.15.117:0/3545681369 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d0037040 con 0x556895bc41b0 2024-09-21T06:02:32.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.557+0000 7f59bd7fa640 1 -- 172.21.15.117:0/3545681369 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f59d0030860 con 0x556895bc41b0 2024-09-21T06:02:32.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.561+0000 7f59e2548740 1 -- 172.21.15.117:0/3545681369 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59cc0158c0 con 0x556895bc41b0 2024-09-21T06:02:32.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.562+0000 7f59bd7fa640 1 -- 172.21.15.117:0/3545681369 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f59d00035f0 con 0x556895bc41b0 2024-09-21T06:02:32.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.562+0000 7f59e2548740 10 client.25669 fetch_fsmap learned FSMap version 27 2024-09-21T06:02:32.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.562+0000 7f59e2548740 1 -- 172.21.15.117:0/3545681369 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556895fbfa50 con 0x556895bc41b0 2024-09-21T06:02:32.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.567+0000 7f59bd7fa640 1 -- 172.21.15.117:0/3545681369 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59cc0158c0 con 0x556895bc41b0 2024-09-21T06:02:32.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.567+0000 7f59bd7fa640 1 -- 172.21.15.117:0/3545681369 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f59d0032940 con 0x556895bc41b0 2024-09-21T06:02:32.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.567+0000 7f59e2548740 10 client.25669 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:02:32.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.567+0000 7f59e2548740 10 client.25669 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T06:02:32.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.567+0000 7f59e2548740 20 client.25669 populate_metadata read hostname 'smithi117' 2024-09-21T06:02:32.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.567+0000 7f59e2548740 1 --2- 172.21.15.117:0/3545681369 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x556895bab9f0 0x556895fc3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:32.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.567+0000 7f59e2548740 4 client.25669 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:02:32.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.567+0000 7f59e2548740 1 -- 172.21.15.117:0/3545681369 --> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556895f0cdf0 con 0x556895bab9f0 2024-09-21T06:02:32.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.573+0000 7f59d956f640 1 --2- 172.21.15.117:0/3545681369 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x556895bab9f0 0x556895fc3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:32.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.580+0000 7f59bd7fa640 10 client.25669 ms_handle_connect on v2:172.21.15.192:6834/3530688860 2024-09-21T06:02:32.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.580+0000 7f59d956f640 1 --2- 172.21.15.117:0/3545681369 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x556895bab9f0 0x556895fc3420 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5d7db4fc59f4ecfc server_cookie=8bb822b61cff8578 in_seq=0 out_seq=0 2024-09-21T06:02:32.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.583+0000 7f59bd7fa640 1 -- 172.21.15.117:0/3545681369 <== mds.3 v2:172.21.15.192:6834/3530688860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556895f0cdf0 con 0x556895bab9f0 2024-09-21T06:02:32.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.583+0000 7f59bd7fa640 10 client.25669 handle_command_reply: tid=0 2024-09-21T06:02:32.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.583+0000 7f59e2548740 1 client.25669 shutdown 2024-09-21T06:02:32.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.584+0000 7f59e2548740 1 -- 172.21.15.117:0/3545681369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556895bc41b0 msgr2=0x556895f9fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:32.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.584+0000 7f59e2548740 1 --2- 172.21.15.117:0/3545681369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556895bc41b0 0x556895f9fbe0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f59d0035040 tx=0x7f59d0005e40 comp rx=0 tx=0).stop 2024-09-21T06:02:32.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.584+0000 7f59e2548740 1 -- 172.21.15.117:0/3545681369 shutdown_connections 2024-09-21T06:02:32.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.584+0000 7f59e2548740 1 --2- 172.21.15.117:0/3545681369 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x556895bab9f0 0x556895fc3420 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.585+0000 7f59e2548740 1 --2- 172.21.15.117:0/3545681369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556895bed070 0x556895fbcfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.585+0000 7f59e2548740 1 --2- 172.21.15.117:0/3545681369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556895bcc390 0x556895fa0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.585+0000 7f59e2548740 1 --2- 172.21.15.117:0/3545681369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556895bc41b0 0x556895f9fbe0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.585+0000 7f59e2548740 1 -- 172.21.15.117:0/3545681369 >> 172.21.15.117:0/3545681369 conn(0x556895be1460 msgr2=0x556895f975f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:32.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.585+0000 7f59bd7fa640 0 client.25669 ms_handle_reset on v2:172.21.15.192:6834/3530688860 2024-09-21T06:02:32.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.585+0000 7f59e2548740 1 -- 172.21.15.117:0/3545681369 shutdown_connections 2024-09-21T06:02:32.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.585+0000 7f59e2548740 1 -- 172.21.15.117:0/3545681369 wait complete. 2024-09-21T06:02:32.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.585+0000 7f59e2548740 20 client.25669 trim_cache size 0 max 16384 2024-09-21T06:02:32.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.615+0000 7f59e2548740 1 Processor -- start 2024-09-21T06:02:32.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.615+0000 7f59e2548740 1 -- start start 2024-09-21T06:02:32.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.615+0000 7f59e2548740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556895bc41b0 0x556895fa00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:32.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.615+0000 7f59e2548740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556895bcc390 0x556895f97560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:32.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.615+0000 7f59e2548740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556895bed070 0x556895f97aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:32.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.615+0000 7f59e2548740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f59b800e6f0 con 0x556895bed070 2024-09-21T06:02:32.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.615+0000 7f59e2548740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f59b8010230 con 0x556895bcc390 2024-09-21T06:02:32.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.615+0000 7f59e2548740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f59b800edc0 con 0x556895bc41b0 2024-09-21T06:02:32.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.616+0000 7f59da571640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556895bed070 0x556895f97aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:32.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.616+0000 7f59da571640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556895bed070 0x556895f97aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49226/0 (socket says 172.21.15.117:49226) 2024-09-21T06:02:32.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.616+0000 7f59da571640 1 -- 172.21.15.117:0/4155239656 learned_addr learned my addr 172.21.15.117:0/4155239656 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:32.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.616+0000 7f59d9d70640 1 --2- 172.21.15.117:0/4155239656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556895bc41b0 0x556895fa00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:32.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.616+0000 7f59da571640 1 -- 172.21.15.117:0/4155239656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556895bc41b0 msgr2=0x556895fa00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:32.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.617+0000 7f59da571640 1 --2- 172.21.15.117:0/4155239656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556895bc41b0 0x556895fa00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.617+0000 7f59da571640 1 -- 172.21.15.117:0/4155239656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556895bcc390 msgr2=0x556895f97560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:02:32.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.617+0000 7f59da571640 1 --2- 172.21.15.117:0/4155239656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556895bcc390 0x556895f97560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.617+0000 7f59da571640 1 -- 172.21.15.117:0/4155239656 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556895f9a810 con 0x556895bed070 2024-09-21T06:02:32.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.617+0000 7f59da571640 1 --2- 172.21.15.117:0/4155239656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556895bed070 0x556895f97aa0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f59cc00a5b0 tx=0x7f59cc00a540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3790835f030052d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:32.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.619+0000 7f59bf7fe640 1 -- 172.21.15.117:0/4155239656 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59cc025020 con 0x556895bed070 2024-09-21T06:02:32.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.619+0000 7f59bf7fe640 1 -- 172.21.15.117:0/4155239656 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59cc009590 con 0x556895bed070 2024-09-21T06:02:32.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.620+0000 7f59bf7fe640 1 -- 172.21.15.117:0/4155239656 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59cc01a040 con 0x556895bed070 2024-09-21T06:02:32.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.621+0000 7f59e2548740 1 -- 172.21.15.117:0/4155239656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556895bed070 msgr2=0x556895f97aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:32.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.621+0000 7f59e2548740 1 --2- 172.21.15.117:0/4155239656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556895bed070 0x556895f97aa0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f59cc00a5b0 tx=0x7f59cc00a540 comp rx=0 tx=0).stop 2024-09-21T06:02:32.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.621+0000 7f59e2548740 1 -- 172.21.15.117:0/4155239656 shutdown_connections 2024-09-21T06:02:32.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.621+0000 7f59e2548740 1 --2- 172.21.15.117:0/4155239656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556895bed070 0x556895f97aa0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.621+0000 7f59e2548740 1 --2- 172.21.15.117:0/4155239656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556895bcc390 0x556895f97560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.621+0000 7f59e2548740 1 --2- 172.21.15.117:0/4155239656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556895bc41b0 0x556895fa00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.624 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.621+0000 7f59e2548740 1 -- 172.21.15.117:0/4155239656 >> 172.21.15.117:0/4155239656 conn(0x556895be1460 msgr2=0x556895f9f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:32.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.622+0000 7f59e2548740 1 -- 172.21.15.117:0/4155239656 shutdown_connections 2024-09-21T06:02:32.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.622+0000 7f59e2548740 1 -- 172.21.15.117:0/4155239656 wait complete. 2024-09-21T06:02:32.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.623+0000 7f59e2548740 1 -- start start 2024-09-21T06:02:32.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.623+0000 7f59e2548740 1 Processor -- start 2024-09-21T06:02:32.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.623+0000 7f59e2548740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556895bc41b0 0x556895fb4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:32.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.623+0000 7f59e2548740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556895bcc390 0x556895fb53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:32.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.623+0000 7f59e2548740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556895bed070 0x556895fb5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:32.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.623+0000 7f59e2548740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f59b800fb60 con 0x556895bc41b0 2024-09-21T06:02:32.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.624+0000 7f59e2548740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f59b800f490 con 0x556895bcc390 2024-09-21T06:02:32.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.624+0000 7f59e2548740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f59b800c6d0 con 0x556895bed070 2024-09-21T06:02:32.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.624+0000 7f59e2548740 10 client.0 _ll_register_callbacks cb 0x7f59d0035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:02:32.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.624+0000 7f59d9d70640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556895bc41b0 0x556895fb4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:32.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.624+0000 7f59d9d70640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556895bc41b0 0x556895fb4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49230/0 (socket says 172.21.15.117:49230) 2024-09-21T06:02:32.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.624+0000 7f59d9d70640 1 -- 172.21.15.117:0/2152197731 learned_addr learned my addr 172.21.15.117:0/2152197731 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:32.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.625+0000 7f59d9d70640 1 -- 172.21.15.117:0/2152197731 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556895bed070 msgr2=0x556895fb5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:32.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.625+0000 7f59da571640 1 --2- 172.21.15.117:0/2152197731 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556895bed070 0x556895fb5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:32.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.625+0000 7f59d9d70640 1 --2- 172.21.15.117:0/2152197731 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556895bed070 0x556895fb5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.625+0000 7f59d9d70640 1 -- 172.21.15.117:0/2152197731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556895bcc390 msgr2=0x556895fb53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:32.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.625+0000 7f59d9d70640 1 --2- 172.21.15.117:0/2152197731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556895bcc390 0x556895fb53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.625+0000 7f59d9d70640 1 -- 172.21.15.117:0/2152197731 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55689603de30 con 0x556895bc41b0 2024-09-21T06:02:32.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.625+0000 7f59da571640 1 --2- 172.21.15.117:0/2152197731 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556895bed070 0x556895fb5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:02:32.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.626+0000 7f59d9d70640 1 --2- 172.21.15.117:0/2152197731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556895bc41b0 0x556895fb4e90 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f59d002a8d0 tx=0x7f59d00326f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb4f8e33daccabf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:32.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.626+0000 7f59bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:02:32.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.626+0000 7f59bd7fa640 1 -- 172.21.15.117:0/2152197731 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d0062070 con 0x556895bc41b0 2024-09-21T06:02:32.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.626+0000 7f59e2548740 1 -- 172.21.15.117:0/2152197731 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59cc009590 con 0x556895bc41b0 2024-09-21T06:02:32.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.626+0000 7f59bd7fa640 1 -- 172.21.15.117:0/2152197731 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f59d0003990 con 0x556895bc41b0 2024-09-21T06:02:32.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.626+0000 7f59bd7fa640 1 -- 172.21.15.117:0/2152197731 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f59d0032af0 con 0x556895bc41b0 2024-09-21T06:02:32.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.626+0000 7f59e2548740 10 client.16371 fetch_fsmap learned FSMap version 27 2024-09-21T06:02:32.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.626+0000 7f59e2548740 1 -- 172.21.15.117:0/2152197731 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55689603f060 con 0x556895bc41b0 2024-09-21T06:02:32.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.628+0000 7f59bd7fa640 1 -- 172.21.15.117:0/2152197731 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59cc009590 con 0x556895bc41b0 2024-09-21T06:02:32.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.628+0000 7f59bd7fa640 1 -- 172.21.15.117:0/2152197731 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f59d0065030 con 0x556895bc41b0 2024-09-21T06:02:32.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.628+0000 7f59e2548740 10 client.16371 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:02:32.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.628+0000 7f59e2548740 10 client.16371 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T06:02:32.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.628+0000 7f59e2548740 20 client.16371 populate_metadata read hostname 'smithi117' 2024-09-21T06:02:32.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.628+0000 7f59e2548740 1 --2- 172.21.15.117:0/2152197731 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x556895b80da0 0x556896041840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:32.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.628+0000 7f59e2548740 4 client.16371 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:02:32.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.628+0000 7f59e2548740 1 -- 172.21.15.117:0/2152197731 --> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556895f088a0 con 0x556895b80da0 2024-09-21T06:02:32.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.630+0000 7f59d956f640 1 --2- 172.21.15.117:0/2152197731 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x556895b80da0 0x556896041840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:32.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.632+0000 7f59d956f640 1 --2- 172.21.15.117:0/2152197731 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x556895b80da0 0x556896041840 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6bd6f0c62958608 server_cookie=f297292ff9c942d9 in_seq=0 out_seq=0 2024-09-21T06:02:32.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.633+0000 7f59bd7fa640 10 client.16371 ms_handle_connect on v2:172.21.15.192:6834/3530688860 2024-09-21T06:02:32.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.634+0000 7f59bd7fa640 1 -- 172.21.15.117:0/2152197731 <== mds.3 v2:172.21.15.192:6834/3530688860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556895f088a0 con 0x556895b80da0 2024-09-21T06:02:32.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.634+0000 7f59bd7fa640 10 client.16371 handle_command_reply: tid=0 2024-09-21T06:02:32.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.634+0000 7f59e2548740 1 client.16371 shutdown 2024-09-21T06:02:32.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.635+0000 7f59e2548740 1 -- 172.21.15.117:0/2152197731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556895bc41b0 msgr2=0x556895fb4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:32.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.635+0000 7f59e2548740 1 --2- 172.21.15.117:0/2152197731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556895bc41b0 0x556895fb4e90 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f59d002a8d0 tx=0x7f59d00326f0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.635+0000 7f59e2548740 1 -- 172.21.15.117:0/2152197731 shutdown_connections 2024-09-21T06:02:32.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.635+0000 7f59e2548740 1 --2- 172.21.15.117:0/2152197731 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x556895b80da0 0x556896041840 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.635+0000 7f59e2548740 1 --2- 172.21.15.117:0/2152197731 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556895bed070 0x556895fb5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.635+0000 7f59e2548740 1 --2- 172.21.15.117:0/2152197731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556895bcc390 0x556895fb53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.635+0000 7f59e2548740 1 --2- 172.21.15.117:0/2152197731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556895bc41b0 0x556895fb4e90 secure :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f59d002a8d0 tx=0x7f59d00326f0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.635+0000 7f59e2548740 1 -- 172.21.15.117:0/2152197731 >> 172.21.15.117:0/2152197731 conn(0x556895be1460 msgr2=0x556895f97560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:32.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.635+0000 7f59bd7fa640 0 client.16371 ms_handle_reset on v2:172.21.15.192:6834/3530688860 2024-09-21T06:02:32.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.636+0000 7f59e2548740 1 -- 172.21.15.117:0/2152197731 shutdown_connections 2024-09-21T06:02:32.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.636+0000 7f59e2548740 1 -- 172.21.15.117:0/2152197731 wait complete. 2024-09-21T06:02:32.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.637+0000 7f59e2548740 20 client.16371 trim_cache size 0 max 16384 2024-09-21T06:02:32.640 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:02:32.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.638+0000 7f59dad72640 1 -- 172.21.15.117:0/1149864496 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f59c00b6880 msgr2=0x7f59c00b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:32.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.638+0000 7f59dad72640 1 --2- 172.21.15.117:0/1149864496 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f59c00b6880 0x7f59c00b8d90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f59d0005eb0 tx=0x7f59d002a860 comp rx=0 tx=0).stop 2024-09-21T06:02:32.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.638+0000 7f59dad72640 1 -- 172.21.15.117:0/1149864496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59d410f900 msgr2=0x7f59d41a0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:32.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.638+0000 7f59dad72640 1 --2- 172.21.15.117:0/1149864496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59d410f900 0x7f59d41a0f30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f59d406b920 tx=0x7f59c400ec20 comp rx=0 tx=0).stop 2024-09-21T06:02:32.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.639+0000 7f59dad72640 1 -- 172.21.15.117:0/1149864496 shutdown_connections 2024-09-21T06:02:32.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.639+0000 7f59dad72640 1 --2- 172.21.15.117:0/1149864496 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f59c00b6880 0x7f59c00b8d90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.639+0000 7f59dad72640 1 --2- 172.21.15.117:0/1149864496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59d4110250 0x7f59d41a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.639+0000 7f59dad72640 1 --2- 172.21.15.117:0/1149864496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59d410f900 0x7f59d41a0f30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.639+0000 7f59dad72640 1 --2- 172.21.15.117:0/1149864496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59d4109fc0 0x7f59d41a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:32.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.639+0000 7f59dad72640 1 -- 172.21.15.117:0/1149864496 >> 172.21.15.117:0/1149864496 conn(0x7f59d40795c0 msgr2=0x7f59d4108510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:32.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.639+0000 7f59dad72640 1 -- 172.21.15.117:0/1149864496 shutdown_connections 2024-09-21T06:02:32.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:32.639+0000 7f59dad72640 1 -- 172.21.15.117:0/1149864496 wait complete. 2024-09-21T06:02:34.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:02:33 smithi138 ceph-mon[34391]: pgmap v484: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 865 MiB/s wr, 1.49k op/s 2024-09-21T06:02:34.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:02:33 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:02:34.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:02:33 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:02:34.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:02:33 smithi192 ceph-mon[34700]: pgmap v484: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 865 MiB/s wr, 1.49k op/s 2024-09-21T06:02:34.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:02:33 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:02:34.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:02:33 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:02:34.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:02:33 smithi117 ceph-mon[30947]: pgmap v484: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 865 MiB/s wr, 1.49k op/s 2024-09-21T06:02:34.279 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:02:33 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:02:34.279 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:02:33 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:02:35.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:02:34 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:02:35.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:02:34 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:02:35.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:02:34 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:02:36.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:02:35 smithi138 ceph-mon[34391]: pgmap v485: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 944 MiB/s wr, 1.70k op/s 2024-09-21T06:02:36.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:02:35 smithi192 ceph-mon[34700]: pgmap v485: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 944 MiB/s wr, 1.70k op/s 2024-09-21T06:02:36.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:02:35 smithi117 ceph-mon[30947]: pgmap v485: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 944 MiB/s wr, 1.70k op/s 2024-09-21T06:02:36.535 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T06:02:37.262 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:02:38.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:02:37 smithi138 ceph-mon[34391]: pgmap v486: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 883 MiB/s wr, 1.68k op/s 2024-09-21T06:02:38.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:02:37 smithi192 ceph-mon[34700]: pgmap v486: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 883 MiB/s wr, 1.68k op/s 2024-09-21T06:02:38.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:02:37 smithi117 ceph-mon[30947]: pgmap v486: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 883 MiB/s wr, 1.68k op/s 2024-09-21T06:02:40.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:02:39 smithi138 ceph-mon[34391]: pgmap v487: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 981 MiB/s wr, 1.72k op/s 2024-09-21T06:02:40.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:02:39 smithi192 ceph-mon[34700]: pgmap v487: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 981 MiB/s wr, 1.72k op/s 2024-09-21T06:02:40.280 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:02:39 smithi117 ceph-mon[30947]: pgmap v487: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 981 MiB/s wr, 1.72k op/s 2024-09-21T06:02:41.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.064+0000 7fe1c9055640 1 -- 172.21.15.117:0/3159025202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1c411cc30 msgr2=0x7fe1c411f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:41.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.064+0000 7fe1c9055640 1 --2- 172.21.15.117:0/3159025202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1c411cc30 0x7fe1c411f140 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc009ca0 tx=0x7fe1bc02a8f0 comp rx=0 tx=0).stop 2024-09-21T06:02:41.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.066+0000 7fe1c9055640 1 -- 172.21.15.117:0/3159025202 shutdown_connections 2024-09-21T06:02:41.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.066+0000 7fe1c9055640 1 --2- 172.21.15.117:0/3159025202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1c411cc30 0x7fe1c411f140 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:41.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.066+0000 7fe1c9055640 1 --2- 172.21.15.117:0/3159025202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1c410a9e0 0x7fe1c410adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:41.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.066+0000 7fe1c9055640 1 --2- 172.21.15.117:0/3159025202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1c4111f40 0x7fe1c410a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:41.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.066+0000 7fe1c9055640 1 -- 172.21.15.117:0/3159025202 >> 172.21.15.117:0/3159025202 conn(0x7fe1c4070ec0 msgr2=0x7fe1c40712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:41.075 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.070+0000 7fe1c9055640 1 -- 172.21.15.117:0/3159025202 shutdown_connections 2024-09-21T06:02:41.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.071+0000 7fe1c9055640 1 -- 172.21.15.117:0/3159025202 wait complete. 2024-09-21T06:02:41.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.073+0000 7fe1c9055640 1 Processor -- start 2024-09-21T06:02:41.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.073+0000 7fe1c9055640 1 -- start start 2024-09-21T06:02:41.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.074+0000 7fe1c9055640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1c410a9e0 0x7fe1c411a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:41.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.074+0000 7fe1c9055640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1c4111f40 0x7fe1c4113650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:41.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.074+0000 7fe1c9055640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1c411cc30 0x7fe1c4113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:41.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.074+0000 7fe1c9055640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c41228b0 con 0x7fe1c4111f40 2024-09-21T06:02:41.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.074+0000 7fe1c9055640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c4122730 con 0x7fe1c410a9e0 2024-09-21T06:02:41.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.074+0000 7fe1c9055640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c4122a30 con 0x7fe1c411cc30 2024-09-21T06:02:41.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.075+0000 7fe1c359e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1c411cc30 0x7fe1c4113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:41.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.075+0000 7fe1c359e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1c411cc30 0x7fe1c4113b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:39190/0 (socket says 172.21.15.117:39190) 2024-09-21T06:02:41.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.075+0000 7fe1c359e640 1 -- 172.21.15.117:0/3129500982 learned_addr learned my addr 172.21.15.117:0/3129500982 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:41.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.076+0000 7fe1c359e640 1 -- 172.21.15.117:0/3129500982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1c410a9e0 msgr2=0x7fe1c411a380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:02:41.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.076+0000 7fe1c359e640 1 --2- 172.21.15.117:0/3129500982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1c410a9e0 0x7fe1c411a380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:41.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.076+0000 7fe1c359e640 1 -- 172.21.15.117:0/3129500982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1c4111f40 msgr2=0x7fe1c4113650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T06:02:41.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.076+0000 7fe1c259c640 1 --2- 172.21.15.117:0/3129500982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1c4111f40 0x7fe1c4113650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:41.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.076+0000 7fe1c359e640 1 --2- 172.21.15.117:0/3129500982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1c4111f40 0x7fe1c4113650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:41.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.076+0000 7fe1c359e640 1 -- 172.21.15.117:0/3129500982 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c4114390 con 0x7fe1c411cc30 2024-09-21T06:02:41.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.076+0000 7fe1c259c640 1 --2- 172.21.15.117:0/3129500982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1c4111f40 0x7fe1c4113650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:02:41.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.077+0000 7fe1c359e640 1 --2- 172.21.15.117:0/3129500982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1c411cc30 0x7fe1c4113b90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc030820 tx=0x7fe1bc030850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:41.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.079+0000 7fe1abfff640 1 -- 172.21.15.117:0/3129500982 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc030af0 con 0x7fe1c411cc30 2024-09-21T06:02:41.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.079+0000 7fe1c9055640 1 -- 172.21.15.117:0/3129500982 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1c4118ff0 con 0x7fe1c411cc30 2024-09-21T06:02:41.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.079+0000 7fe1c9055640 1 -- 172.21.15.117:0/3129500982 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1c4119530 con 0x7fe1c411cc30 2024-09-21T06:02:41.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.080+0000 7fe1abfff640 1 -- 172.21.15.117:0/3129500982 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1bc02d070 con 0x7fe1c411cc30 2024-09-21T06:02:41.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.080+0000 7fe1abfff640 1 -- 172.21.15.117:0/3129500982 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc03a680 con 0x7fe1c411cc30 2024-09-21T06:02:41.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.086+0000 7fe1abfff640 1 -- 172.21.15.117:0/3129500982 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fe1bc03a820 con 0x7fe1c411cc30 2024-09-21T06:02:41.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.087+0000 7fe1abfff640 1 --2- 172.21.15.117:0/3129500982 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe1ac0b6880 0x7fe1ac0b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:41.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.088+0000 7fe1abfff640 1 -- 172.21.15.117:0/3129500982 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fe1bc109950 con 0x7fe1c411cc30 2024-09-21T06:02:41.097 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.093+0000 7fe1c2d9d640 1 --2- 172.21.15.117:0/3129500982 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe1ac0b6880 0x7fe1ac0b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:41.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.110+0000 7fe1c2d9d640 1 --2- 172.21.15.117:0/3129500982 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe1ac0b6880 0x7fe1ac0b8d90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe1b400b990 tx=0x7fe1b400b370 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:41.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.111+0000 7fe1cf5c9740 1 Processor -- start 2024-09-21T06:02:41.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.111+0000 7fe1cf5c9740 1 -- start start 2024-09-21T06:02:41.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.111+0000 7fe1cf5c9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cd107f070 0x559cd14266a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:41.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.111+0000 7fe1cf5c9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559cd10561b0 0x559cd1426be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:41.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.111+0000 7fe1cf5c9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559cd105e390 0x559cd1427120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:41.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.111+0000 7fe1cf5c9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559cd139cea0 con 0x559cd10561b0 2024-09-21T06:02:41.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.111+0000 7fe1cf5c9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559cd138c2d0 con 0x559cd107f070 2024-09-21T06:02:41.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.111+0000 7fe1cf5c9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x559cd13900e0 con 0x559cd105e390 2024-09-21T06:02:41.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.113+0000 7fe1c2d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cd107f070 0x559cd14266a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:41.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.113+0000 7fe1c359e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559cd105e390 0x559cd1427120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:41.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.113+0000 7fe1c259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559cd10561b0 0x559cd1426be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:41.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.113+0000 7fe1c259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559cd10561b0 0x559cd1426be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34090/0 (socket says 172.21.15.117:34090) 2024-09-21T06:02:41.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.113+0000 7fe1c259c640 1 -- 172.21.15.117:0/4017205493 learned_addr learned my addr 172.21.15.117:0/4017205493 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:41.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.118+0000 7fe1c259c640 1 -- 172.21.15.117:0/4017205493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559cd105e390 msgr2=0x559cd1427120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:41.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.118+0000 7fe1c259c640 1 --2- 172.21.15.117:0/4017205493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559cd105e390 0x559cd1427120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:41.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.118+0000 7fe1c259c640 1 -- 172.21.15.117:0/4017205493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cd107f070 msgr2=0x559cd14266a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:41.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.118+0000 7fe1c259c640 1 --2- 172.21.15.117:0/4017205493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cd107f070 0x559cd14266a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:41.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.118+0000 7fe1c259c640 1 -- 172.21.15.117:0/4017205493 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cd10ea650 con 0x559cd10561b0 2024-09-21T06:02:41.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.119+0000 7fe1c259c640 1 --2- 172.21.15.117:0/4017205493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559cd10561b0 0x559cd1426be0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8009be0 tx=0x7fe1b800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a198ac9bd9c399 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:41.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.120+0000 7fe1a97fa640 1 -- 172.21.15.117:0/4017205493 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8016070 con 0x559cd10561b0 2024-09-21T06:02:41.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.120+0000 7fe1a97fa640 1 -- 172.21.15.117:0/4017205493 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1b80054d0 con 0x559cd10561b0 2024-09-21T06:02:41.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.125+0000 7fe1a97fa640 1 -- 172.21.15.117:0/4017205493 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8011470 con 0x559cd10561b0 2024-09-21T06:02:41.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.126+0000 7fe1cf5c9740 1 -- 172.21.15.117:0/4017205493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559cd10561b0 msgr2=0x559cd1426be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:41.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.126+0000 7fe1cf5c9740 1 --2- 172.21.15.117:0/4017205493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559cd10561b0 0x559cd1426be0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8009be0 tx=0x7fe1b800e6c0 comp rx=0 tx=0).stop 2024-09-21T06:02:41.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.131+0000 7fe1cf5c9740 1 -- 172.21.15.117:0/4017205493 shutdown_connections 2024-09-21T06:02:41.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.131+0000 7fe1cf5c9740 1 --2- 172.21.15.117:0/4017205493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559cd10561b0 0x559cd1426be0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:41.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.131+0000 7fe1cf5c9740 1 --2- 172.21.15.117:0/4017205493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559cd105e390 0x559cd1427120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:41.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.132+0000 7fe1cf5c9740 1 --2- 172.21.15.117:0/4017205493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cd107f070 0x559cd14266a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:41.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.132+0000 7fe1cf5c9740 1 -- 172.21.15.117:0/4017205493 >> 172.21.15.117:0/4017205493 conn(0x559cd1073460 msgr2=0x559cd1424590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:41.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.132+0000 7fe1cf5c9740 1 -- 172.21.15.117:0/4017205493 shutdown_connections 2024-09-21T06:02:41.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.132+0000 7fe1cf5c9740 1 -- 172.21.15.117:0/4017205493 wait complete. 2024-09-21T06:02:41.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.133+0000 7fe1cf5c9740 1 -- start start 2024-09-21T06:02:41.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.133+0000 7fe1cf5c9740 1 Processor -- start 2024-09-21T06:02:41.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.134+0000 7fe1cf5c9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cd10561b0 0x559cd1431ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:41.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.135+0000 7fe1cf5c9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559cd105e390 0x559cd14321e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:41.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.135+0000 7fe1cf5c9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559cd107f070 0x559cd1432720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:41.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.135+0000 7fe1cf5c9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559cd11ed700 con 0x559cd107f070 2024-09-21T06:02:41.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.135+0000 7fe1cf5c9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559cd1327160 con 0x559cd10561b0 2024-09-21T06:02:41.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.135+0000 7fe1cf5c9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x559cd1434ed0 con 0x559cd105e390 2024-09-21T06:02:41.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.136+0000 7fe1cf5c9740 10 client.0 _ll_register_callbacks cb 0x559cd13409e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:02:41.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.139+0000 7fe1c2d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cd10561b0 0x559cd1431ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:41.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.140+0000 7fe1c2d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cd10561b0 0x559cd1431ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:39840/0 (socket says 172.21.15.117:39840) 2024-09-21T06:02:41.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.140+0000 7fe1c2d9d640 1 -- 172.21.15.117:0/1181682425 learned_addr learned my addr 172.21.15.117:0/1181682425 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:41.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.140+0000 7fe1c359e640 1 --2- 172.21.15.117:0/1181682425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559cd107f070 0x559cd1432720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:41.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.140+0000 7fe1c259c640 1 --2- 172.21.15.117:0/1181682425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559cd105e390 0x559cd14321e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:41.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.145+0000 7fe1c2d9d640 1 -- 172.21.15.117:0/1181682425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559cd105e390 msgr2=0x559cd14321e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:41.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.145+0000 7fe1c2d9d640 1 --2- 172.21.15.117:0/1181682425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559cd105e390 0x559cd14321e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:41.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.145+0000 7fe1c2d9d640 1 -- 172.21.15.117:0/1181682425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559cd107f070 msgr2=0x559cd1432720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:41.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.145+0000 7fe1c2d9d640 1 --2- 172.21.15.117:0/1181682425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559cd107f070 0x559cd1432720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:41.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.145+0000 7fe1c2d9d640 1 -- 172.21.15.117:0/1181682425 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cd1354c90 con 0x559cd10561b0 2024-09-21T06:02:41.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.152+0000 7fe1c2d9d640 1 --2- 172.21.15.117:0/1181682425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cd10561b0 0x559cd1431ca0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40104e0 tx=0x7fe1b4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4ced0107844531b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:41.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.153+0000 7fe1a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-21T06:02:41.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.153+0000 7fe1a6ff5640 1 -- 172.21.15.117:0/1181682425 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4026600 con 0x559cd10561b0 2024-09-21T06:02:41.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.154+0000 7fe1cf5c9740 1 -- 172.21.15.117:0/1181682425 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1b80054d0 con 0x559cd10561b0 2024-09-21T06:02:41.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.154+0000 7fe1a6ff5640 1 -- 172.21.15.117:0/1181682425 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fe1b40267a0 con 0x559cd10561b0 2024-09-21T06:02:41.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.157+0000 7fe1a6ff5640 1 -- 172.21.15.117:0/1181682425 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe1b40269a0 con 0x559cd10561b0 2024-09-21T06:02:41.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.161+0000 7fe1cf5c9740 10 client.25703 fetch_fsmap learned FSMap version 27 2024-09-21T06:02:41.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.161+0000 7fe1cf5c9740 1 -- 172.21.15.117:0/1181682425 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559cd1440c40 con 0x559cd10561b0 2024-09-21T06:02:41.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.163+0000 7fe1a6ff5640 1 -- 172.21.15.117:0/1181682425 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1b4016030 con 0x559cd10561b0 2024-09-21T06:02:41.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.163+0000 7fe1a6ff5640 1 -- 172.21.15.117:0/1181682425 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe1b401e510 con 0x559cd10561b0 2024-09-21T06:02:41.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.166+0000 7fe1cf5c9740 10 client.25703 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:02:41.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.166+0000 7fe1cf5c9740 10 client.25703 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:02:41.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.166+0000 7fe1cf5c9740 20 client.25703 populate_metadata read hostname 'smithi117' 2024-09-21T06:02:41.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.167+0000 7fe1cf5c9740 1 --2- 172.21.15.117:0/1181682425 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x559cd103d9f0 0x559cd1444190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:41.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.167+0000 7fe1cf5c9740 4 client.25703 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:02:41.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.167+0000 7fe1cf5c9740 1 -- 172.21.15.117:0/1181682425 --> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe1b80054d0 con 0x559cd103d9f0 2024-09-21T06:02:41.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.171+0000 7fe1c259c640 1 --2- 172.21.15.117:0/1181682425 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x559cd103d9f0 0x559cd1444190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:41.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.186+0000 7fe1c259c640 1 --2- 172.21.15.117:0/1181682425 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x559cd103d9f0 0x559cd1444190 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f3ed6e208200433d server_cookie=7eab922be9668ee5 in_seq=0 out_seq=0 2024-09-21T06:02:41.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.187+0000 7fe1a6ff5640 10 client.25703 ms_handle_connect on v2:172.21.15.192:6836/2683860842 2024-09-21T06:02:41.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.192+0000 7fe1a6ff5640 1 -- 172.21.15.117:0/1181682425 <== mds.2 v2:172.21.15.192:6836/2683860842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fe1b80054d0 con 0x559cd103d9f0 2024-09-21T06:02:41.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.192+0000 7fe1a6ff5640 10 client.25703 handle_command_reply: tid=0 2024-09-21T06:02:41.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.193+0000 7fe1cf5c9740 1 client.25703 shutdown 2024-09-21T06:02:41.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.193+0000 7fe1cf5c9740 1 -- 172.21.15.117:0/1181682425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cd10561b0 msgr2=0x559cd1431ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:41.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.193+0000 7fe1cf5c9740 1 --2- 172.21.15.117:0/1181682425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cd10561b0 0x559cd1431ca0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40104e0 tx=0x7fe1b4004730 comp rx=0 tx=0).stop 2024-09-21T06:02:41.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.193+0000 7fe1cf5c9740 1 -- 172.21.15.117:0/1181682425 shutdown_connections 2024-09-21T06:02:41.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.193+0000 7fe1cf5c9740 1 --2- 172.21.15.117:0/1181682425 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x559cd103d9f0 0x559cd1444190 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:41.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.193+0000 7fe1cf5c9740 1 --2- 172.21.15.117:0/1181682425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559cd107f070 0x559cd1432720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:41.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.193+0000 7fe1cf5c9740 1 --2- 172.21.15.117:0/1181682425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559cd105e390 0x559cd14321e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:41.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.193+0000 7fe1cf5c9740 1 --2- 172.21.15.117:0/1181682425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cd10561b0 0x559cd1431ca0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:41.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.193+0000 7fe1cf5c9740 1 -- 172.21.15.117:0/1181682425 >> 172.21.15.117:0/1181682425 conn(0x559cd1073460 msgr2=0x559cd1429830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:41.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.194+0000 7fe1a6ff5640 0 client.25703 ms_handle_reset on v2:172.21.15.192:6836/2683860842 2024-09-21T06:02:41.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.196+0000 7fe1cf5c9740 1 -- 172.21.15.117:0/1181682425 shutdown_connections 2024-09-21T06:02:41.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.196+0000 7fe1cf5c9740 1 -- 172.21.15.117:0/1181682425 wait complete. 2024-09-21T06:02:41.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.196+0000 7fe1cf5c9740 20 client.25703 trim_cache size 0 max 16384 2024-09-21T06:02:41.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.227+0000 7fe1cf5c9740 1 Processor -- start 2024-09-21T06:02:41.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.228+0000 7fe1cf5c9740 1 -- start start 2024-09-21T06:02:41.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.228+0000 7fe1cf5c9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559cd10561b0 0x559cd14338d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:41.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.229+0000 7fe1cf5c9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559cd105e390 0x559cd142ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:41.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.229+0000 7fe1cf5c9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cd107f070 0x559cd142f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:41.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.229+0000 7fe1cf5c9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b000e6f0 con 0x559cd105e390 2024-09-21T06:02:41.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.229+0000 7fe1cf5c9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b0010230 con 0x559cd107f070 2024-09-21T06:02:41.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.229+0000 7fe1cf5c9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b000edc0 con 0x559cd10561b0 2024-09-21T06:02:41.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.229+0000 7fe1c2d9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559cd10561b0 0x559cd14338d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:41.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.229+0000 7fe1c2d9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559cd10561b0 0x559cd14338d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:39224/0 (socket says 172.21.15.117:39224) 2024-09-21T06:02:41.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.229+0000 7fe1c2d9d640 1 -- 172.21.15.117:0/3546491934 learned_addr learned my addr 172.21.15.117:0/3546491934 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:41.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.230+0000 7fe1c259c640 1 --2- 172.21.15.117:0/3546491934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559cd105e390 0x559cd142ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:41.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.233+0000 7fe1c359e640 1 --2- 172.21.15.117:0/3546491934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cd107f070 0x559cd142f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:41.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.233+0000 7fe1c2d9d640 1 -- 172.21.15.117:0/3546491934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cd107f070 msgr2=0x559cd142f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:41.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.233+0000 7fe1c2d9d640 1 --2- 172.21.15.117:0/3546491934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cd107f070 0x559cd142f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:41.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.233+0000 7fe1c2d9d640 1 -- 172.21.15.117:0/3546491934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559cd105e390 msgr2=0x559cd142ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:41.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.233+0000 7fe1c2d9d640 1 --2- 172.21.15.117:0/3546491934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559cd105e390 0x559cd142ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:41.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.233+0000 7fe1c2d9d640 1 -- 172.21.15.117:0/3546491934 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cd14246a0 con 0x559cd10561b0 2024-09-21T06:02:41.241 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.237+0000 7fe1c2d9d640 1 --2- 172.21.15.117:0/3546491934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559cd10561b0 0x559cd14338d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe1b401e930 tx=0x7fe1b401ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d962cb75e4afa195 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:41.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.239+0000 7fe1a8ff9640 1 -- 172.21.15.117:0/3546491934 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b403d070 con 0x559cd10561b0 2024-09-21T06:02:41.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.239+0000 7fe1a8ff9640 1 -- 172.21.15.117:0/3546491934 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1b4016040 con 0x559cd10561b0 2024-09-21T06:02:41.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.239+0000 7fe1a8ff9640 1 -- 172.21.15.117:0/3546491934 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4038440 con 0x559cd10561b0 2024-09-21T06:02:41.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.243+0000 7fe1cf5c9740 1 -- 172.21.15.117:0/3546491934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559cd10561b0 msgr2=0x559cd14338d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:41.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.243+0000 7fe1cf5c9740 1 --2- 172.21.15.117:0/3546491934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559cd10561b0 0x559cd14338d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe1b401e930 tx=0x7fe1b401ecb0 comp rx=0 tx=0).stop 2024-09-21T06:02:41.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.244+0000 7fe1cf5c9740 1 -- 172.21.15.117:0/3546491934 shutdown_connections 2024-09-21T06:02:41.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.244+0000 7fe1cf5c9740 1 --2- 172.21.15.117:0/3546491934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559cd105e390 0x559cd142ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:41.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.244+0000 7fe1cf5c9740 1 --2- 172.21.15.117:0/3546491934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cd107f070 0x559cd142f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:41.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.244+0000 7fe1cf5c9740 1 --2- 172.21.15.117:0/3546491934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559cd10561b0 0x559cd14338d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:41.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.244+0000 7fe1cf5c9740 1 -- 172.21.15.117:0/3546491934 >> 172.21.15.117:0/3546491934 conn(0x559cd1073460 msgr2=0x559cd14331c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:41.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.246+0000 7fe1cf5c9740 1 -- 172.21.15.117:0/3546491934 shutdown_connections 2024-09-21T06:02:41.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.246+0000 7fe1cf5c9740 1 -- 172.21.15.117:0/3546491934 wait complete. 2024-09-21T06:02:41.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.247+0000 7fe1cf5c9740 1 -- start start 2024-09-21T06:02:41.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.247+0000 7fe1cf5c9740 1 Processor -- start 2024-09-21T06:02:41.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.248+0000 7fe1cf5c9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559cd10561b0 0x559cd1436200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:41.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.248+0000 7fe1cf5c9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559cd105e390 0x559cd1436740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:41.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.248+0000 7fe1cf5c9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cd107f070 0x559cd1436c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:41.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.248+0000 7fe1cf5c9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b000fb60 con 0x559cd105e390 2024-09-21T06:02:41.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.248+0000 7fe1cf5c9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b000f490 con 0x559cd107f070 2024-09-21T06:02:41.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.249+0000 7fe1cf5c9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b000c6d0 con 0x559cd10561b0 2024-09-21T06:02:41.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.249+0000 7fe1cf5c9740 10 client.0 _ll_register_callbacks cb 0x7fe1b400dd00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:02:41.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.250+0000 7fe1c259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559cd105e390 0x559cd1436740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:41.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.250+0000 7fe1c259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559cd105e390 0x559cd1436740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34116/0 (socket says 172.21.15.117:34116) 2024-09-21T06:02:41.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.250+0000 7fe1c259c640 1 -- 172.21.15.117:0/3070333033 learned_addr learned my addr 172.21.15.117:0/3070333033 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:41.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.252+0000 7fe1c259c640 1 -- 172.21.15.117:0/3070333033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559cd10561b0 msgr2=0x559cd1436200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:02:41.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.252+0000 7fe1c259c640 1 --2- 172.21.15.117:0/3070333033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559cd10561b0 0x559cd1436200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:41.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.252+0000 7fe1c259c640 1 -- 172.21.15.117:0/3070333033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cd107f070 msgr2=0x559cd1436c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:02:41.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.252+0000 7fe1c259c640 1 --2- 172.21.15.117:0/3070333033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cd107f070 0x559cd1436c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:41.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.252+0000 7fe1c259c640 1 -- 172.21.15.117:0/3070333033 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cd14adb80 con 0x559cd105e390 2024-09-21T06:02:41.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.253+0000 7fe1c259c640 1 --2- 172.21.15.117:0/3070333033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559cd105e390 0x559cd1436740 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80057c0 tx=0x7fe1b80094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6621c2c1cc6d0170 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:41.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.254+0000 7fe1a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:02:41.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.255+0000 7fe1a6ff5640 1 -- 172.21.15.117:0/3070333033 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8033060 con 0x559cd105e390 2024-09-21T06:02:41.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.255+0000 7fe1cf5c9740 1 -- 172.21.15.117:0/3070333033 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1b4016040 con 0x559cd105e390 2024-09-21T06:02:41.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.256+0000 7fe1a6ff5640 1 -- 172.21.15.117:0/3070333033 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fe1b8004c60 con 0x559cd105e390 2024-09-21T06:02:41.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.256+0000 7fe1a6ff5640 1 -- 172.21.15.117:0/3070333033 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe1b8016040 con 0x559cd105e390 2024-09-21T06:02:41.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.261+0000 7fe1cf5c9740 10 client.16386 fetch_fsmap learned FSMap version 27 2024-09-21T06:02:41.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.261+0000 7fe1cf5c9740 1 -- 172.21.15.117:0/3070333033 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559cd14aee30 con 0x559cd105e390 2024-09-21T06:02:41.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.262+0000 7fe1a6ff5640 1 -- 172.21.15.117:0/3070333033 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1b4016040 con 0x559cd105e390 2024-09-21T06:02:41.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.262+0000 7fe1a6ff5640 1 -- 172.21.15.117:0/3070333033 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe1b800f040 con 0x559cd105e390 2024-09-21T06:02:41.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.262+0000 7fe1cf5c9740 10 client.16386 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:02:41.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.262+0000 7fe1cf5c9740 10 client.16386 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:02:41.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.262+0000 7fe1cf5c9740 20 client.16386 populate_metadata read hostname 'smithi117' 2024-09-21T06:02:41.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.262+0000 7fe1cf5c9740 1 --2- 172.21.15.117:0/3070333033 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x559cd1012da0 0x559cd14b1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:41.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.262+0000 7fe1cf5c9740 4 client.16386 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:02:41.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.262+0000 7fe1cf5c9740 1 -- 172.21.15.117:0/3070333033 --> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559cd139edf0 con 0x559cd1012da0 2024-09-21T06:02:41.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.263+0000 7fe1c2d9d640 1 --2- 172.21.15.117:0/3070333033 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x559cd1012da0 0x559cd14b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:41.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.265+0000 7fe1a6ff5640 10 client.16386 ms_handle_connect on v2:172.21.15.192:6836/2683860842 2024-09-21T06:02:41.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.265+0000 7fe1c2d9d640 1 --2- 172.21.15.117:0/3070333033 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x559cd1012da0 0x559cd14b1590 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dea428a5dc4426d9 server_cookie=349a19650f8abf73 in_seq=0 out_seq=0 2024-09-21T06:02:41.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.267+0000 7fe1a6ff5640 1 -- 172.21.15.117:0/3070333033 <== mds.2 v2:172.21.15.192:6836/2683860842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559cd139edf0 con 0x559cd1012da0 2024-09-21T06:02:41.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.267+0000 7fe1a6ff5640 10 client.16386 handle_command_reply: tid=0 2024-09-21T06:02:41.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.267+0000 7fe1cf5c9740 1 client.16386 shutdown 2024-09-21T06:02:41.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.270+0000 7fe1cf5c9740 1 -- 172.21.15.117:0/3070333033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559cd105e390 msgr2=0x559cd1436740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:41.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.270+0000 7fe1cf5c9740 1 --2- 172.21.15.117:0/3070333033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559cd105e390 0x559cd1436740 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80057c0 tx=0x7fe1b80094f0 comp rx=0 tx=0).stop 2024-09-21T06:02:41.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.271+0000 7fe1cf5c9740 1 -- 172.21.15.117:0/3070333033 shutdown_connections 2024-09-21T06:02:41.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.271+0000 7fe1cf5c9740 1 --2- 172.21.15.117:0/3070333033 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x559cd1012da0 0x559cd14b1590 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:41.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.271+0000 7fe1cf5c9740 1 --2- 172.21.15.117:0/3070333033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559cd105e390 0x559cd1436740 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:41.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.271+0000 7fe1cf5c9740 1 --2- 172.21.15.117:0/3070333033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559cd107f070 0x559cd1436c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:41.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.271+0000 7fe1cf5c9740 1 --2- 172.21.15.117:0/3070333033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559cd10561b0 0x559cd1436200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:41.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.271+0000 7fe1a6ff5640 0 client.16386 ms_handle_reset on v2:172.21.15.192:6836/2683860842 2024-09-21T06:02:41.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.271+0000 7fe1cf5c9740 1 -- 172.21.15.117:0/3070333033 >> 172.21.15.117:0/3070333033 conn(0x559cd1073460 msgr2=0x559cd14276d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:41.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.271+0000 7fe1cf5c9740 1 -- 172.21.15.117:0/3070333033 shutdown_connections 2024-09-21T06:02:41.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.271+0000 7fe1cf5c9740 1 -- 172.21.15.117:0/3070333033 wait complete. 2024-09-21T06:02:41.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.273+0000 7fe1cf5c9740 20 client.16386 trim_cache size 0 max 16384 2024-09-21T06:02:41.278 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:02:41.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.277+0000 7fe1c9055640 1 -- 172.21.15.117:0/3129500982 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe1ac0b6880 msgr2=0x7fe1ac0b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:41.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.277+0000 7fe1c9055640 1 --2- 172.21.15.117:0/3129500982 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe1ac0b6880 0x7fe1ac0b8d90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe1b400b990 tx=0x7fe1b400b370 comp rx=0 tx=0).stop 2024-09-21T06:02:41.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.277+0000 7fe1c9055640 1 -- 172.21.15.117:0/3129500982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1c411cc30 msgr2=0x7fe1c4113b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:41.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.277+0000 7fe1c9055640 1 --2- 172.21.15.117:0/3129500982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1c411cc30 0x7fe1c4113b90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc030820 tx=0x7fe1bc030850 comp rx=0 tx=0).stop 2024-09-21T06:02:41.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.277+0000 7fe1c9055640 1 -- 172.21.15.117:0/3129500982 shutdown_connections 2024-09-21T06:02:41.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.277+0000 7fe1c9055640 1 --2- 172.21.15.117:0/3129500982 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe1ac0b6880 0x7fe1ac0b8d90 secure :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe1b400b990 tx=0x7fe1b400b370 comp rx=0 tx=0).stop 2024-09-21T06:02:41.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.277+0000 7fe1c9055640 1 --2- 172.21.15.117:0/3129500982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1c4111f40 0x7fe1c4113650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:41.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.277+0000 7fe1c9055640 1 --2- 172.21.15.117:0/3129500982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1c411cc30 0x7fe1c4113b90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:41.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.277+0000 7fe1c9055640 1 --2- 172.21.15.117:0/3129500982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1c410a9e0 0x7fe1c411a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:41.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.277+0000 7fe1c9055640 1 -- 172.21.15.117:0/3129500982 >> 172.21.15.117:0/3129500982 conn(0x7fe1c4070ec0 msgr2=0x7fe1c4071e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:41.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.277+0000 7fe1c9055640 1 -- 172.21.15.117:0/3129500982 shutdown_connections 2024-09-21T06:02:41.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:41.278+0000 7fe1c9055640 1 -- 172.21.15.117:0/3129500982 wait complete. 2024-09-21T06:02:42.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:02:41 smithi117 ceph-mon[30947]: pgmap v488: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 840 MiB/s wr, 1.45k op/s 2024-09-21T06:02:42.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:02:41 smithi192 ceph-mon[34700]: pgmap v488: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 840 MiB/s wr, 1.45k op/s 2024-09-21T06:02:42.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:02:41 smithi138 ceph-mon[34391]: pgmap v488: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 840 MiB/s wr, 1.45k op/s 2024-09-21T06:02:44.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:02:43 smithi192 ceph-mon[34700]: pgmap v489: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 882 MiB/s wr, 1.68k op/s 2024-09-21T06:02:44.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:02:43 smithi117 ceph-mon[30947]: pgmap v489: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 882 MiB/s wr, 1.68k op/s 2024-09-21T06:02:44.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:02:43 smithi138 ceph-mon[34391]: pgmap v489: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 882 MiB/s wr, 1.68k op/s 2024-09-21T06:02:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.458+0000 7fbf4836e640 1 -- 172.21.15.117:0/3857470233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf401165a0 msgr2=0x7fbf40118a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.458+0000 7fbf4836e640 1 --2- 172.21.15.117:0/3857470233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf401165a0 0x7fbf40118a80 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c009900 tx=0x7fbf3c02a410 comp rx=0 tx=0).stop 2024-09-21T06:02:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.459+0000 7fbf4836e640 1 -- 172.21.15.117:0/3857470233 shutdown_connections 2024-09-21T06:02:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.459+0000 7fbf4836e640 1 --2- 172.21.15.117:0/3857470233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf401165a0 0x7fbf40118a80 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.459+0000 7fbf4836e640 1 --2- 172.21.15.117:0/3857470233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf4010aef0 0x7fbf40115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.459+0000 7fbf4836e640 1 --2- 172.21.15.117:0/3857470233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf4010a5a0 0x7fbf4010a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.459+0000 7fbf4836e640 1 -- 172.21.15.117:0/3857470233 >> 172.21.15.117:0/3857470233 conn(0x7fbf40070dd0 msgr2=0x7fbf400711e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.459+0000 7fbf4836e640 1 -- 172.21.15.117:0/3857470233 shutdown_connections 2024-09-21T06:02:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.459+0000 7fbf4836e640 1 -- 172.21.15.117:0/3857470233 wait complete. 2024-09-21T06:02:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.460+0000 7fbf4836e640 1 Processor -- start 2024-09-21T06:02:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.460+0000 7fbf4836e640 1 -- start start 2024-09-21T06:02:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.460+0000 7fbf4836e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf4010a5a0 0x7fbf40115fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.460+0000 7fbf4836e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf4010aef0 0x7fbf401132e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.460+0000 7fbf4836e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf401165a0 0x7fbf40113a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.460+0000 7fbf4836e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4010d570 con 0x7fbf4010a5a0 2024-09-21T06:02:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.460+0000 7fbf4836e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4010d3f0 con 0x7fbf401165a0 2024-09-21T06:02:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.460+0000 7fbf4836e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4010d6f0 con 0x7fbf4010aef0 2024-09-21T06:02:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.461+0000 7fbf4610a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf4010a5a0 0x7fbf40115fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.461+0000 7fbf4610a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf4010a5a0 0x7fbf40115fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51102/0 (socket says 172.21.15.117:51102) 2024-09-21T06:02:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.461+0000 7fbf4610a640 1 -- 172.21.15.117:0/4288307807 learned_addr learned my addr 172.21.15.117:0/4288307807 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.461+0000 7fbf4610a640 1 -- 172.21.15.117:0/4288307807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf4010aef0 msgr2=0x7fbf401132e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:02:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.461+0000 7fbf4610a640 1 --2- 172.21.15.117:0/4288307807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf4010aef0 0x7fbf401132e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.461+0000 7fbf4610a640 1 -- 172.21.15.117:0/4288307807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf401165a0 msgr2=0x7fbf40113a50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:02:45.466 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.461+0000 7fbf4610a640 1 --2- 172.21.15.117:0/4288307807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf401165a0 0x7fbf40113a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.461+0000 7fbf4610a640 1 -- 172.21.15.117:0/4288307807 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf401b53d0 con 0x7fbf4010a5a0 2024-09-21T06:02:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.462+0000 7fbf4610a640 1 --2- 172.21.15.117:0/4288307807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf4010a5a0 0x7fbf40115fc0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fbf3400d950 tx=0x7fbf3400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.463+0000 7fbf337fe640 1 -- 172.21.15.117:0/4288307807 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf34017070 con 0x7fbf4010a5a0 2024-09-21T06:02:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.463+0000 7fbf337fe640 1 -- 172.21.15.117:0/4288307807 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf34009bb0 con 0x7fbf4010a5a0 2024-09-21T06:02:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.463+0000 7fbf337fe640 1 -- 172.21.15.117:0/4288307807 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3400f620 con 0x7fbf4010a5a0 2024-09-21T06:02:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.463+0000 7fbf4836e640 1 -- 172.21.15.117:0/4288307807 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf401b56c0 con 0x7fbf4010a5a0 2024-09-21T06:02:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.463+0000 7fbf4836e640 1 -- 172.21.15.117:0/4288307807 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf401b5ad0 con 0x7fbf4010a5a0 2024-09-21T06:02:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.468+0000 7fbf337fe640 1 -- 172.21.15.117:0/4288307807 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fbf34010460 con 0x7fbf4010a5a0 2024-09-21T06:02:45.471 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.469+0000 7fbf4f3bd740 1 Processor -- start 2024-09-21T06:02:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.469+0000 7fbf4f3bd740 1 -- start start 2024-09-21T06:02:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.470+0000 7fbf4f3bd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5dab621b0 0x55b5daf079b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.470+0000 7fbf4f3bd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5dab392f0 0x55b5daf07ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.470+0000 7fbf4f3bd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5dab414d0 0x55b5daf08430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.470+0000 7fbf4f3bd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b5dae7c6f0 con 0x55b5dab392f0 2024-09-21T06:02:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.470+0000 7fbf4f3bd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b5dae6bb20 con 0x55b5dab414d0 2024-09-21T06:02:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.470+0000 7fbf4f3bd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b5dae6f930 con 0x55b5dab621b0 2024-09-21T06:02:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.470+0000 7fbf4610a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5dab392f0 0x55b5daf07ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.470+0000 7fbf4610a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5dab392f0 0x55b5daf07ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51108/0 (socket says 172.21.15.117:51108) 2024-09-21T06:02:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.470+0000 7fbf4610a640 1 -- 172.21.15.117:0/855860891 learned_addr learned my addr 172.21.15.117:0/855860891 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.471+0000 7fbf337fe640 1 --2- 172.21.15.117:0/4288307807 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbf2c0b6730 0x7fbf2c0b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.471+0000 7fbf4690b640 1 --2- 172.21.15.117:0/4288307807 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbf2c0b6730 0x7fbf2c0b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.471+0000 7fbf4610a640 1 -- 172.21.15.117:0/855860891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5dab621b0 msgr2=0x55b5daf079b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.471+0000 7fbf4610a640 1 --2- 172.21.15.117:0/855860891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5dab621b0 0x55b5daf079b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.471+0000 7fbf4610a640 1 -- 172.21.15.117:0/855860891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5dab414d0 msgr2=0x55b5daf08430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:02:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.471+0000 7fbf4610a640 1 --2- 172.21.15.117:0/855860891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5dab414d0 0x55b5daf08430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.471+0000 7fbf4610a640 1 -- 172.21.15.117:0/855860891 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5dae31530 con 0x55b5dab392f0 2024-09-21T06:02:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.471+0000 7fbf337fe640 1 -- 172.21.15.117:0/4288307807 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fbf340e72c0 con 0x7fbf4010a5a0 2024-09-21T06:02:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.472+0000 7fbf4610a640 1 --2- 172.21.15.117:0/855860891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5dab392f0 0x55b5daf07ef0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fbf340a9700 tx=0x7fbf340acbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61fb2088d43eedb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.472+0000 7fbf30bfe640 1 -- 172.21.15.117:0/855860891 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf34020030 con 0x55b5dab392f0 2024-09-21T06:02:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.472+0000 7fbf30bfe640 1 -- 172.21.15.117:0/855860891 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf340acc40 con 0x55b5dab392f0 2024-09-21T06:02:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.472+0000 7fbf30bfe640 1 -- 172.21.15.117:0/855860891 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf340b1c10 con 0x55b5dab392f0 2024-09-21T06:02:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.473+0000 7fbf4f3bd740 1 -- 172.21.15.117:0/855860891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5dab392f0 msgr2=0x55b5daf07ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.473+0000 7fbf4f3bd740 1 --2- 172.21.15.117:0/855860891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5dab392f0 0x55b5daf07ef0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fbf340a9700 tx=0x7fbf340acbb0 comp rx=0 tx=0).stop 2024-09-21T06:02:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.473+0000 7fbf4690b640 1 --2- 172.21.15.117:0/4288307807 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbf2c0b6730 0x7fbf2c0b8c40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c0057c0 tx=0x7fbf3c031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.474+0000 7fbf4f3bd740 1 -- 172.21.15.117:0/855860891 shutdown_connections 2024-09-21T06:02:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.474+0000 7fbf4f3bd740 1 --2- 172.21.15.117:0/855860891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5dab392f0 0x55b5daf07ef0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.474+0000 7fbf4f3bd740 1 --2- 172.21.15.117:0/855860891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5dab414d0 0x55b5daf08430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.474+0000 7fbf4f3bd740 1 --2- 172.21.15.117:0/855860891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5dab621b0 0x55b5daf079b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.474+0000 7fbf4f3bd740 1 -- 172.21.15.117:0/855860891 >> 172.21.15.117:0/855860891 conn(0x55b5dab56300 msgr2=0x55b5daf058a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.474+0000 7fbf4f3bd740 1 -- 172.21.15.117:0/855860891 shutdown_connections 2024-09-21T06:02:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.474+0000 7fbf4f3bd740 1 -- 172.21.15.117:0/855860891 wait complete. 2024-09-21T06:02:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.477+0000 7fbf4f3bd740 1 -- start start 2024-09-21T06:02:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.480+0000 7fbf4f3bd740 1 Processor -- start 2024-09-21T06:02:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.482+0000 7fbf4f3bd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5dab392f0 0x55b5daf13030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.482+0000 7fbf4f3bd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5dab414d0 0x55b5daf13570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.482+0000 7fbf4f3bd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5dab621b0 0x55b5daf1f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.482+0000 7fbf4f3bd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b5dada3ab0 con 0x55b5dab392f0 2024-09-21T06:02:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.482+0000 7fbf4f3bd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b5dae11730 con 0x55b5dab621b0 2024-09-21T06:02:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.482+0000 7fbf4f3bd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b5daf16070 con 0x55b5dab414d0 2024-09-21T06:02:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.482+0000 7fbf4f3bd740 10 client.0 _ll_register_callbacks cb 0x55b5dae656b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:02:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.482+0000 7fbf45909640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5dab414d0 0x55b5daf13570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.482+0000 7fbf4610a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5dab392f0 0x55b5daf13030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.482+0000 7fbf45909640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5dab414d0 0x55b5daf13570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:58442/0 (socket says 172.21.15.117:58442) 2024-09-21T06:02:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.482+0000 7fbf45909640 1 -- 172.21.15.117:0/2814458270 learned_addr learned my addr 172.21.15.117:0/2814458270 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.483+0000 7fbf4610a640 1 -- 172.21.15.117:0/2814458270 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5dab414d0 msgr2=0x55b5daf13570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.483+0000 7fbf4690b640 1 --2- 172.21.15.117:0/2814458270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5dab621b0 0x55b5daf1f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.483+0000 7fbf4610a640 1 --2- 172.21.15.117:0/2814458270 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5dab414d0 0x55b5daf13570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.483+0000 7fbf4610a640 1 -- 172.21.15.117:0/2814458270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5dab621b0 msgr2=0x55b5daf1f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.483+0000 7fbf4610a640 1 --2- 172.21.15.117:0/2814458270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5dab621b0 0x55b5daf1f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.483+0000 7fbf4610a640 1 -- 172.21.15.117:0/2814458270 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5dabcd790 con 0x55b5dab392f0 2024-09-21T06:02:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.483+0000 7fbf4690b640 1 --2- 172.21.15.117:0/2814458270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5dab621b0 0x55b5daf1f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:02:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.484+0000 7fbf4610a640 1 --2- 172.21.15.117:0/2814458270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5dab392f0 0x55b5daf13030 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x55b5daf06630 tx=0x7fbf340e4f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b29df8ef780f4823 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.484+0000 7fbf29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:02:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.484+0000 7fbf29ffb640 1 -- 172.21.15.117:0/2814458270 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf34020030 con 0x55b5dab392f0 2024-09-21T06:02:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.484+0000 7fbf4f3bd740 1 -- 172.21.15.117:0/2814458270 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf340acc40 con 0x55b5dab392f0 2024-09-21T06:02:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.484+0000 7fbf29ffb640 1 -- 172.21.15.117:0/2814458270 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fbf340ac390 con 0x55b5dab392f0 2024-09-21T06:02:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.485+0000 7fbf29ffb640 1 -- 172.21.15.117:0/2814458270 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbf340b6b50 con 0x55b5dab392f0 2024-09-21T06:02:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.485+0000 7fbf4f3bd740 10 client.16398 fetch_fsmap learned FSMap version 27 2024-09-21T06:02:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.485+0000 7fbf4f3bd740 1 -- 172.21.15.117:0/2814458270 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b5dac46620 con 0x55b5dab392f0 2024-09-21T06:02:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.486+0000 7fbf29ffb640 1 -- 172.21.15.117:0/2814458270 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf340acc40 con 0x55b5dab392f0 2024-09-21T06:02:45.489 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.487+0000 7fbf29ffb640 1 -- 172.21.15.117:0/2814458270 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbf340d0ab0 con 0x55b5dab392f0 2024-09-21T06:02:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.487+0000 7fbf4f3bd740 10 client.16398 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:02:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.487+0000 7fbf4f3bd740 10 client.16398 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:02:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.487+0000 7fbf4f3bd740 20 client.16398 populate_metadata read hostname 'smithi117' 2024-09-21T06:02:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.487+0000 7fbf4f3bd740 1 --2- 172.21.15.117:0/2814458270 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b5dab20d80 0x55b5daf361b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.487+0000 7fbf4f3bd740 4 client.16398 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:02:45.490 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.487+0000 7fbf4f3bd740 1 -- 172.21.15.117:0/2814458270 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5dae7e640 con 0x55b5dab20d80 2024-09-21T06:02:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.487+0000 7fbf45909640 1 --2- 172.21.15.117:0/2814458270 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b5dab20d80 0x55b5daf361b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.489+0000 7fbf45909640 1 --2- 172.21.15.117:0/2814458270 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b5dab20d80 0x55b5daf361b0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d4c259d0f43eebd server_cookie=7f4341f434337c5 in_seq=0 out_seq=0 2024-09-21T06:02:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.489+0000 7fbf29ffb640 10 client.16398 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:02:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.492+0000 7fbf29ffb640 1 -- 172.21.15.117:0/2814458270 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b5dae7e640 con 0x55b5dab20d80 2024-09-21T06:02:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.492+0000 7fbf29ffb640 10 client.16398 handle_command_reply: tid=0 2024-09-21T06:02:45.496 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.493+0000 7fbf4f3bd740 1 client.16398 shutdown 2024-09-21T06:02:45.497 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.494+0000 7fbf4f3bd740 1 -- 172.21.15.117:0/2814458270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5dab392f0 msgr2=0x55b5daf13030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:45.497 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.494+0000 7fbf4f3bd740 1 --2- 172.21.15.117:0/2814458270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5dab392f0 0x55b5daf13030 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x55b5daf06630 tx=0x7fbf340e4f60 comp rx=0 tx=0).stop 2024-09-21T06:02:45.497 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.494+0000 7fbf4f3bd740 1 -- 172.21.15.117:0/2814458270 shutdown_connections 2024-09-21T06:02:45.497 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.494+0000 7fbf4f3bd740 1 --2- 172.21.15.117:0/2814458270 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b5dab20d80 0x55b5daf361b0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:45.497 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.494+0000 7fbf4f3bd740 1 --2- 172.21.15.117:0/2814458270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5dab621b0 0x55b5daf1f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:45.497 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.494+0000 7fbf4f3bd740 1 --2- 172.21.15.117:0/2814458270 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5dab414d0 0x55b5daf13570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.494+0000 7fbf4f3bd740 1 --2- 172.21.15.117:0/2814458270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5dab392f0 0x55b5daf13030 secure :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0x55b5daf06630 tx=0x7fbf340e4f60 comp rx=0 tx=0).stop 2024-09-21T06:02:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.494+0000 7fbf4f3bd740 1 -- 172.21.15.117:0/2814458270 >> 172.21.15.117:0/2814458270 conn(0x55b5dab56300 msgr2=0x55b5daf089e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.494+0000 7fbf29ffb640 0 client.16398 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:02:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.495+0000 7fbf4f3bd740 1 -- 172.21.15.117:0/2814458270 shutdown_connections 2024-09-21T06:02:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.495+0000 7fbf4f3bd740 1 -- 172.21.15.117:0/2814458270 wait complete. 2024-09-21T06:02:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.495+0000 7fbf4f3bd740 20 client.16398 trim_cache size 0 max 16384 2024-09-21T06:02:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.520+0000 7fbf4f3bd740 1 Processor -- start 2024-09-21T06:02:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.520+0000 7fbf4f3bd740 1 -- start start 2024-09-21T06:02:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.521+0000 7fbf4f3bd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5dab414d0 0x55b5daf1f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.521+0000 7fbf4f3bd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5dab621b0 0x55b5daf090f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.521+0000 7fbf4f3bd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5daaf61c0 0x55b5daf10200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.521+0000 7fbf4f3bd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbf401b5d90 con 0x55b5daaf61c0 2024-09-21T06:02:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.521+0000 7fbf4f3bd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbf401b7880 con 0x55b5dab414d0 2024-09-21T06:02:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.521+0000 7fbf4f3bd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbf401b6410 con 0x55b5dab621b0 2024-09-21T06:02:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.521+0000 7fbf45909640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5dab621b0 0x55b5daf090f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.522+0000 7fbf4690b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5daaf61c0 0x55b5daf10200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.522+0000 7fbf45909640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5dab621b0 0x55b5daf090f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:58456/0 (socket says 172.21.15.117:58456) 2024-09-21T06:02:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.522+0000 7fbf45909640 1 -- 172.21.15.117:0/2951058912 learned_addr learned my addr 172.21.15.117:0/2951058912 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.522+0000 7fbf4610a640 1 --2- 172.21.15.117:0/2951058912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5dab414d0 0x55b5daf1f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.523+0000 7fbf4690b640 1 -- 172.21.15.117:0/2951058912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5dab621b0 msgr2=0x55b5daf090f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.523+0000 7fbf4690b640 1 --2- 172.21.15.117:0/2951058912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5dab621b0 0x55b5daf090f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.523+0000 7fbf4690b640 1 -- 172.21.15.117:0/2951058912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5dab414d0 msgr2=0x55b5daf1f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.523+0000 7fbf4690b640 1 --2- 172.21.15.117:0/2951058912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5dab414d0 0x55b5daf1f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.523+0000 7fbf4690b640 1 -- 172.21.15.117:0/2951058912 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5daf0a180 con 0x55b5daaf61c0 2024-09-21T06:02:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.523+0000 7fbf4690b640 1 --2- 172.21.15.117:0/2951058912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5daaf61c0 0x55b5daf10200 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c02aa00 tx=0x7fbf3c03c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e73331a4073704b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:45.527 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.524+0000 7fbf2bfff640 1 -- 172.21.15.117:0/2951058912 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c005560 con 0x55b5daaf61c0 2024-09-21T06:02:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.524+0000 7fbf2bfff640 1 -- 172.21.15.117:0/2951058912 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf3c004b70 con 0x55b5daaf61c0 2024-09-21T06:02:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.524+0000 7fbf2bfff640 1 -- 172.21.15.117:0/2951058912 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c031b70 con 0x55b5daaf61c0 2024-09-21T06:02:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.525+0000 7fbf4f3bd740 1 -- 172.21.15.117:0/2951058912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5daaf61c0 msgr2=0x55b5daf10200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.525+0000 7fbf4f3bd740 1 --2- 172.21.15.117:0/2951058912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5daaf61c0 0x55b5daf10200 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c02aa00 tx=0x7fbf3c03c000 comp rx=0 tx=0).stop 2024-09-21T06:02:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.525+0000 7fbf4f3bd740 1 -- 172.21.15.117:0/2951058912 shutdown_connections 2024-09-21T06:02:45.528 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.525+0000 7fbf4f3bd740 1 --2- 172.21.15.117:0/2951058912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5daaf61c0 0x55b5daf10200 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.525+0000 7fbf4f3bd740 1 --2- 172.21.15.117:0/2951058912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5dab621b0 0x55b5daf090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.525+0000 7fbf4f3bd740 1 --2- 172.21.15.117:0/2951058912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5dab414d0 0x55b5daf1f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.525+0000 7fbf4f3bd740 1 -- 172.21.15.117:0/2951058912 >> 172.21.15.117:0/2951058912 conn(0x55b5dab56300 msgr2=0x55b5daf089e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.526+0000 7fbf4f3bd740 1 -- 172.21.15.117:0/2951058912 shutdown_connections 2024-09-21T06:02:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.526+0000 7fbf4f3bd740 1 -- 172.21.15.117:0/2951058912 wait complete. 2024-09-21T06:02:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.526+0000 7fbf4f3bd740 1 -- start start 2024-09-21T06:02:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.527+0000 7fbf4f3bd740 1 Processor -- start 2024-09-21T06:02:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.527+0000 7fbf4f3bd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5daaf61c0 0x55b5daf186e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.527+0000 7fbf4f3bd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5dab414d0 0x55b5daf18c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:45.530 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.527+0000 7fbf4f3bd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5dab621b0 0x55b5daf9ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.527+0000 7fbf4f3bd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbf401b71b0 con 0x55b5daaf61c0 2024-09-21T06:02:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.527+0000 7fbf4f3bd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbf401b6ae0 con 0x55b5dab621b0 2024-09-21T06:02:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.527+0000 7fbf4f3bd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbf401145a0 con 0x55b5dab414d0 2024-09-21T06:02:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.527+0000 7fbf4f3bd740 10 client.0 _ll_register_callbacks cb 0x7fbf340aca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:02:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.528+0000 7fbf4610a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5daaf61c0 0x55b5daf186e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.528+0000 7fbf4610a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5daaf61c0 0x55b5daf186e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51130/0 (socket says 172.21.15.117:51130) 2024-09-21T06:02:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.528+0000 7fbf4610a640 1 -- 172.21.15.117:0/3091967790 learned_addr learned my addr 172.21.15.117:0/3091967790 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.528+0000 7fbf45909640 1 --2- 172.21.15.117:0/3091967790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5dab414d0 0x55b5daf18c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.528+0000 7fbf4690b640 1 --2- 172.21.15.117:0/3091967790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5dab621b0 0x55b5daf9ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.528+0000 7fbf4610a640 1 -- 172.21.15.117:0/3091967790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5dab414d0 msgr2=0x55b5daf18c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.528+0000 7fbf4610a640 1 --2- 172.21.15.117:0/3091967790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5dab414d0 0x55b5daf18c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.528+0000 7fbf4610a640 1 -- 172.21.15.117:0/3091967790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5dab621b0 msgr2=0x55b5daf9ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.528+0000 7fbf4610a640 1 --2- 172.21.15.117:0/3091967790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5dab621b0 0x55b5daf9ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.528+0000 7fbf4610a640 1 -- 172.21.15.117:0/3091967790 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5daf9f0f0 con 0x55b5daaf61c0 2024-09-21T06:02:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.529+0000 7fbf4610a640 1 --2- 172.21.15.117:0/3091967790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5daaf61c0 0x55b5daf186e0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fbf340ace30 tx=0x7fbf34010b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=703af8621e27adb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.529+0000 7fbf29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:02:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.529+0000 7fbf29ffb640 1 -- 172.21.15.117:0/3091967790 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf340a8fd0 con 0x55b5daaf61c0 2024-09-21T06:02:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.529+0000 7fbf4f3bd740 1 -- 172.21.15.117:0/3091967790 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf3c004b70 con 0x55b5daaf61c0 2024-09-21T06:02:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.529+0000 7fbf29ffb640 1 -- 172.21.15.117:0/3091967790 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fbf340ac590 con 0x55b5daaf61c0 2024-09-21T06:02:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.529+0000 7fbf29ffb640 1 -- 172.21.15.117:0/3091967790 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbf340acc40 con 0x55b5daaf61c0 2024-09-21T06:02:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.530+0000 7fbf4f3bd740 10 client.16404 fetch_fsmap learned FSMap version 27 2024-09-21T06:02:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.530+0000 7fbf4f3bd740 1 -- 172.21.15.117:0/3091967790 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b5dafa0250 con 0x55b5daaf61c0 2024-09-21T06:02:45.534 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.530+0000 7fbf29ffb640 1 -- 172.21.15.117:0/3091967790 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf3c004b70 con 0x55b5daaf61c0 2024-09-21T06:02:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.532+0000 7fbf29ffb640 1 -- 172.21.15.117:0/3091967790 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbf340a8a80 con 0x55b5daaf61c0 2024-09-21T06:02:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.532+0000 7fbf4f3bd740 10 client.16404 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:02:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.532+0000 7fbf4f3bd740 10 client.16404 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:02:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.532+0000 7fbf4f3bd740 20 client.16404 populate_metadata read hostname 'smithi117' 2024-09-21T06:02:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.532+0000 7fbf4f3bd740 1 --2- 172.21.15.117:0/3091967790 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b5dafa0680 0x55b5dafa2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:45.535 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.532+0000 7fbf4f3bd740 4 client.16404 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:02:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.532+0000 7fbf4f3bd740 1 -- 172.21.15.117:0/3091967790 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b5dae7a0f0 con 0x55b5dafa0680 2024-09-21T06:02:45.536 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.533+0000 7fbf45909640 1 --2- 172.21.15.117:0/3091967790 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b5dafa0680 0x55b5dafa2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.535+0000 7fbf45909640 1 --2- 172.21.15.117:0/3091967790 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b5dafa0680 0x55b5dafa2aa0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f554a7293c065c48 server_cookie=a98c69a5dca93717 in_seq=0 out_seq=0 2024-09-21T06:02:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:45.535+0000 7fbf29ffb640 10 client.16404 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:02:45.589 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T06:02:46.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:02:45 smithi192 ceph-mon[34700]: pgmap v490: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 961 MiB/s wr, 2.01k op/s 2024-09-21T06:02:46.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:02:45 smithi117 ceph-mon[30947]: pgmap v490: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 961 MiB/s wr, 2.01k op/s 2024-09-21T06:02:46.288 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:02:46.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:02:45 smithi138 ceph-mon[34391]: pgmap v490: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 961 MiB/s wr, 2.01k op/s 2024-09-21T06:02:47.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:02:46 smithi192 ceph-mon[34700]: pgmap v491: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 872 MiB/s wr, 1.91k op/s 2024-09-21T06:02:47.266 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:02:46 smithi138 ceph-mon[34391]: pgmap v491: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 872 MiB/s wr, 1.91k op/s 2024-09-21T06:02:47.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:02:46 smithi117 ceph-mon[30947]: pgmap v491: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 872 MiB/s wr, 1.91k op/s 2024-09-21T06:02:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:47.971+0000 7fbf29ffb640 1 -- 172.21.15.117:0/3091967790 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b5dae7a0f0 con 0x55b5dafa0680 2024-09-21T06:02:47.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:47.971+0000 7fbf29ffb640 10 client.16404 handle_command_reply: tid=0 2024-09-21T06:02:47.976 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:47.971+0000 7fbf4f3bd740 1 client.16404 shutdown 2024-09-21T06:02:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:47.973+0000 7fbf4f3bd740 1 -- 172.21.15.117:0/3091967790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5daaf61c0 msgr2=0x55b5daf186e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:47.973+0000 7fbf4f3bd740 1 --2- 172.21.15.117:0/3091967790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5daaf61c0 0x55b5daf186e0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fbf340ace30 tx=0x7fbf34010b90 comp rx=0 tx=0).stop 2024-09-21T06:02:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:47.973+0000 7fbf4f3bd740 1 -- 172.21.15.117:0/3091967790 shutdown_connections 2024-09-21T06:02:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:47.973+0000 7fbf4f3bd740 1 --2- 172.21.15.117:0/3091967790 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b5dafa0680 0x55b5dafa2aa0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:47.973+0000 7fbf4f3bd740 1 --2- 172.21.15.117:0/3091967790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5dab621b0 0x55b5daf9ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:47.973+0000 7fbf4f3bd740 1 --2- 172.21.15.117:0/3091967790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5dab414d0 0x55b5daf18c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:47.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:47.973+0000 7fbf4f3bd740 1 --2- 172.21.15.117:0/3091967790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b5daaf61c0 0x55b5daf186e0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:47.973+0000 7fbf4f3bd740 1 -- 172.21.15.117:0/3091967790 >> 172.21.15.117:0/3091967790 conn(0x55b5dab56300 msgr2=0x55b5daf144e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:47.974+0000 7fbf29ffb640 0 client.16404 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:02:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:47.974+0000 7fbf4f3bd740 1 -- 172.21.15.117:0/3091967790 shutdown_connections 2024-09-21T06:02:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:47.974+0000 7fbf4f3bd740 1 -- 172.21.15.117:0/3091967790 wait complete. 2024-09-21T06:02:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:47.974+0000 7fbf4f3bd740 20 client.16404 trim_cache size 0 max 16384 2024-09-21T06:02:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T06:02:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T06:02:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T06:02:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T06:02:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:47.989+0000 7fbf4836e640 1 -- 172.21.15.117:0/4288307807 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbf2c0b6730 msgr2=0x7fbf2c0b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:47.989+0000 7fbf4836e640 1 --2- 172.21.15.117:0/4288307807 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbf2c0b6730 0x7fbf2c0b8c40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c0057c0 tx=0x7fbf3c031400 comp rx=0 tx=0).stop 2024-09-21T06:02:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:47.989+0000 7fbf4836e640 1 -- 172.21.15.117:0/4288307807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf4010a5a0 msgr2=0x7fbf40115fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:47.989+0000 7fbf4836e640 1 --2- 172.21.15.117:0/4288307807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf4010a5a0 0x7fbf40115fc0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fbf3400d950 tx=0x7fbf3400de10 comp rx=0 tx=0).stop 2024-09-21T06:02:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:47.993+0000 7fbf4836e640 1 -- 172.21.15.117:0/4288307807 shutdown_connections 2024-09-21T06:02:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:47.993+0000 7fbf4836e640 1 --2- 172.21.15.117:0/4288307807 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbf2c0b6730 0x7fbf2c0b8c40 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:47.993+0000 7fbf4836e640 1 --2- 172.21.15.117:0/4288307807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf401165a0 0x7fbf40113a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:47.993+0000 7fbf4836e640 1 --2- 172.21.15.117:0/4288307807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf4010aef0 0x7fbf401132e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:47.993+0000 7fbf4836e640 1 --2- 172.21.15.117:0/4288307807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf4010a5a0 0x7fbf40115fc0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:47.993+0000 7fbf4836e640 1 -- 172.21.15.117:0/4288307807 >> 172.21.15.117:0/4288307807 conn(0x7fbf40070dd0 msgr2=0x7fbf4011dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:47.998+0000 7fbf4836e640 1 -- 172.21.15.117:0/4288307807 shutdown_connections 2024-09-21T06:02:48.001 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:02:47.998+0000 7fbf4836e640 1 -- 172.21.15.117:0/4288307807 wait complete. 2024-09-21T06:02:49.450 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:02:49.450 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.159+0000 7f946ebfc640 1 -- 172.21.15.117:0/2901522649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94681083c0 msgr2=0x7f94681087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:49.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.159+0000 7f946ebfc640 1 --2- 172.21.15.117:0/2901522649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94681083c0 0x7f94681087d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f9454009900 tx=0x7f945402a410 comp rx=0 tx=0).stop 2024-09-21T06:02:49.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.159+0000 7f946ebfc640 1 -- 172.21.15.117:0/2901522649 shutdown_connections 2024-09-21T06:02:49.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.159+0000 7f946ebfc640 1 --2- 172.21.15.117:0/2901522649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94681083c0 0x7f94681087d0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:49.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.159+0000 7f946ebfc640 1 --2- 172.21.15.117:0/2901522649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9468108d10 0x7f946810f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:49.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.159+0000 7f946ebfc640 1 --2- 172.21.15.117:0/2901522649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9468102a80 0x7f9468102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:49.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.159+0000 7f946ebfc640 1 -- 172.21.15.117:0/2901522649 >> 172.21.15.117:0/2901522649 conn(0x7f9468079740 msgr2=0x7f9468079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:49.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.159+0000 7f946ebfc640 1 -- 172.21.15.117:0/2901522649 shutdown_connections 2024-09-21T06:02:49.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.159+0000 7f946ebfc640 1 -- 172.21.15.117:0/2901522649 wait complete. 2024-09-21T06:02:49.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.160+0000 7f946ebfc640 1 Processor -- start 2024-09-21T06:02:49.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.160+0000 7f946ebfc640 1 -- start start 2024-09-21T06:02:49.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.160+0000 7f946ebfc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9468102a80 0x7f9468199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:49.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.160+0000 7f946ebfc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94681083c0 0x7f9468199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:49.451 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.160+0000 7f946ebfc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9468108d10 0x7f9468199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:49.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.160+0000 7f946ebfc640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9468112a40 con 0x7f94681083c0 2024-09-21T06:02:49.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.160+0000 7f946ebfc640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f94681128c0 con 0x7f9468108d10 2024-09-21T06:02:49.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.160+0000 7f946ebfc640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9468112bc0 con 0x7f9468102a80 2024-09-21T06:02:49.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.161+0000 7f9467fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94681083c0 0x7f9468199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:49.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.161+0000 7f9467fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94681083c0 0x7f9468199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51140/0 (socket says 172.21.15.117:51140) 2024-09-21T06:02:49.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.161+0000 7f9467fff640 1 -- 172.21.15.117:0/3693138177 learned_addr learned my addr 172.21.15.117:0/3693138177 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:49.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.161+0000 7f946c998640 1 --2- 172.21.15.117:0/3693138177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9468102a80 0x7f9468199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:49.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.161+0000 7f946d199640 1 --2- 172.21.15.117:0/3693138177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9468108d10 0x7f9468199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:49.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.161+0000 7f9467fff640 1 -- 172.21.15.117:0/3693138177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9468102a80 msgr2=0x7f9468199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:49.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.161+0000 7f9467fff640 1 --2- 172.21.15.117:0/3693138177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9468102a80 0x7f9468199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:49.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.161+0000 7f9467fff640 1 -- 172.21.15.117:0/3693138177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9468108d10 msgr2=0x7f9468199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:49.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.161+0000 7f9467fff640 1 --2- 172.21.15.117:0/3693138177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9468108d10 0x7f9468199ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:49.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.161+0000 7f9467fff640 1 -- 172.21.15.117:0/3693138177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94681a88d0 con 0x7f94681083c0 2024-09-21T06:02:49.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.162+0000 7f9467fff640 1 --2- 172.21.15.117:0/3693138177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94681083c0 0x7f9468199760 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f9454031a90 tx=0x7f9454004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:49.452 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.162+0000 7f9465ffb640 1 -- 172.21.15.117:0/3693138177 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945403f030 con 0x7f94681083c0 2024-09-21T06:02:49.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.162+0000 7f9465ffb640 1 -- 172.21.15.117:0/3693138177 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9454003cb0 con 0x7f94681083c0 2024-09-21T06:02:49.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.162+0000 7f946ebfc640 1 -- 172.21.15.117:0/3693138177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94681a8b60 con 0x7f94681083c0 2024-09-21T06:02:49.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.162+0000 7f9465ffb640 1 -- 172.21.15.117:0/3693138177 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9454035070 con 0x7f94681083c0 2024-09-21T06:02:49.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.163+0000 7f946ebfc640 1 -- 172.21.15.117:0/3693138177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94681a9090 con 0x7f94681083c0 2024-09-21T06:02:49.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.168+0000 7f9475170740 1 Processor -- start 2024-09-21T06:02:49.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.168+0000 7f9465ffb640 1 -- 172.21.15.117:0/3693138177 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f9454042030 con 0x7f94681083c0 2024-09-21T06:02:49.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.169+0000 7f9475170740 1 -- start start 2024-09-21T06:02:49.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.170+0000 7f9465ffb640 1 --2- 172.21.15.117:0/3693138177 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f94580b6780 0x7f94580b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:49.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.170+0000 7f9465ffb640 1 -- 172.21.15.117:0/3693138177 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f9454107a20 con 0x7f94681083c0 2024-09-21T06:02:49.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.170+0000 7f9475170740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70a927070 0x55a70acce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:49.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.170+0000 7f9475170740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70a8fe1b0 0x55a70acce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:49.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.170+0000 7f9475170740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70a906390 0x55a70acced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:49.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.170+0000 7f9475170740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a70ac44ea0 con 0x55a70a927070 2024-09-21T06:02:49.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.170+0000 7f9475170740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a70ac342d0 con 0x55a70a8fe1b0 2024-09-21T06:02:49.453 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.170+0000 7f9475170740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a70ac380e0 con 0x55a70a906390 2024-09-21T06:02:49.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.171+0000 7f946d199640 1 --2- 172.21.15.117:0/3693138177 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f94580b6780 0x7f94580b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:49.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.171+0000 7f946c998640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70a927070 0x55a70acce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:49.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.171+0000 7f946c998640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70a927070 0x55a70acce2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51146/0 (socket says 172.21.15.117:51146) 2024-09-21T06:02:49.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.171+0000 7f946c998640 1 -- 172.21.15.117:0/2319488325 learned_addr learned my addr 172.21.15.117:0/2319488325 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:49.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.171+0000 7f946c998640 1 -- 172.21.15.117:0/2319488325 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70a906390 msgr2=0x55a70acced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:49.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.171+0000 7f946c998640 1 --2- 172.21.15.117:0/2319488325 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70a906390 0x55a70acced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:49.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.171+0000 7f946c998640 1 -- 172.21.15.117:0/2319488325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70a8fe1b0 msgr2=0x55a70acce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:49.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.171+0000 7f946c998640 1 --2- 172.21.15.117:0/2319488325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70a8fe1b0 0x55a70acce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:49.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.171+0000 7f946d199640 1 --2- 172.21.15.117:0/3693138177 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f94580b6780 0x7f94580b8c90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9460008da0 tx=0x7f94600087e0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:49.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.171+0000 7f946c998640 1 -- 172.21.15.117:0/2319488325 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a70a992650 con 0x55a70a927070 2024-09-21T06:02:49.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.172+0000 7f946c998640 1 --2- 172.21.15.117:0/2319488325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70a927070 0x55a70acce2c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f945c00b7f0 tx=0x7f945c00bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ace6b6fe0d1d71 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:49.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.172+0000 7f9452ffd640 1 -- 172.21.15.117:0/2319488325 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c00d950 con 0x55a70a927070 2024-09-21T06:02:49.454 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.172+0000 7f9452ffd640 1 -- 172.21.15.117:0/2319488325 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f945c018070 con 0x55a70a927070 2024-09-21T06:02:49.455 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.172+0000 7f9452ffd640 1 -- 172.21.15.117:0/2319488325 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c0123e0 con 0x55a70a927070 2024-09-21T06:02:49.455 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.173+0000 7f9475170740 1 -- 172.21.15.117:0/2319488325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70a927070 msgr2=0x55a70acce2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:49.455 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.173+0000 7f9475170740 1 --2- 172.21.15.117:0/2319488325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70a927070 0x55a70acce2c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f945c00b7f0 tx=0x7f945c00bcb0 comp rx=0 tx=0).stop 2024-09-21T06:02:49.455 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.174+0000 7f9475170740 1 -- 172.21.15.117:0/2319488325 shutdown_connections 2024-09-21T06:02:49.455 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.174+0000 7f9475170740 1 --2- 172.21.15.117:0/2319488325 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70a906390 0x55a70acced40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:49.455 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.174+0000 7f9475170740 1 --2- 172.21.15.117:0/2319488325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70a8fe1b0 0x55a70acce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:49.455 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.174+0000 7f9475170740 1 --2- 172.21.15.117:0/2319488325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70a927070 0x55a70acce2c0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:49.455 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.174+0000 7f9475170740 1 -- 172.21.15.117:0/2319488325 >> 172.21.15.117:0/2319488325 conn(0x55a70a91b460 msgr2=0x55a70accc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:49.455 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.174+0000 7f9475170740 1 -- 172.21.15.117:0/2319488325 shutdown_connections 2024-09-21T06:02:49.455 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.174+0000 7f9475170740 1 -- 172.21.15.117:0/2319488325 wait complete. 2024-09-21T06:02:49.455 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.174+0000 7f9475170740 1 -- start start 2024-09-21T06:02:49.455 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.174+0000 7f9475170740 1 Processor -- start 2024-09-21T06:02:49.455 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.175+0000 7f9475170740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70a8fe1b0 0x55a70acd9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:49.455 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.175+0000 7f9475170740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70a906390 0x55a70acd9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:49.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.175+0000 7f9475170740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70a927070 0x55a70ace5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:49.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.175+0000 7f9475170740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a70aa95700 con 0x55a70a927070 2024-09-21T06:02:49.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.175+0000 7f9475170740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a70abcf160 con 0x55a70a8fe1b0 2024-09-21T06:02:49.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.175+0000 7f9475170740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a70acdcaa0 con 0x55a70a906390 2024-09-21T06:02:49.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.175+0000 7f9475170740 10 client.0 _ll_register_callbacks cb 0x55a70abe89e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:02:49.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.175+0000 7f9467fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70a906390 0x55a70acd9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:49.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.175+0000 7f9467fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70a906390 0x55a70acd9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:58492/0 (socket says 172.21.15.117:58492) 2024-09-21T06:02:49.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.175+0000 7f9467fff640 1 -- 172.21.15.117:0/3876558275 learned_addr learned my addr 172.21.15.117:0/3876558275 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:49.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.176+0000 7f946d199640 1 --2- 172.21.15.117:0/3876558275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70a927070 0x55a70ace5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:49.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.176+0000 7f946c998640 1 --2- 172.21.15.117:0/3876558275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70a8fe1b0 0x55a70acd9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:49.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.176+0000 7f9467fff640 1 -- 172.21.15.117:0/3876558275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70a8fe1b0 msgr2=0x55a70acd9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:49.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.176+0000 7f9467fff640 1 --2- 172.21.15.117:0/3876558275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70a8fe1b0 0x55a70acd9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:49.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.176+0000 7f9467fff640 1 -- 172.21.15.117:0/3876558275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70a927070 msgr2=0x55a70ace5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:49.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.176+0000 7f9467fff640 1 --2- 172.21.15.117:0/3876558275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70a927070 0x55a70ace5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:49.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.176+0000 7f9467fff640 1 -- 172.21.15.117:0/3876558275 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a70abfcc90 con 0x55a70a906390 2024-09-21T06:02:49.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.177+0000 7f9467fff640 1 --2- 172.21.15.117:0/3876558275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70a906390 0x55a70acd9fc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f94541051f0 tx=0x7f9454003f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b9d8e3c3b5587f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:49.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.177+0000 7f9447fff640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-21T06:02:49.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.177+0000 7f9447fff640 1 -- 172.21.15.117:0/3876558275 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9454049070 con 0x55a70a906390 2024-09-21T06:02:49.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.177+0000 7f9447fff640 1 -- 172.21.15.117:0/3876558275 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f94540d6e20 con 0x55a70a906390 2024-09-21T06:02:49.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.177+0000 7f9475170740 1 -- 172.21.15.117:0/3876558275 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f945c018070 con 0x55a70a906390 2024-09-21T06:02:49.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.178+0000 7f9447fff640 1 -- 172.21.15.117:0/3876558275 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f94540cd1e0 con 0x55a70a906390 2024-09-21T06:02:49.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.178+0000 7f9475170740 10 client.25699 fetch_fsmap learned FSMap version 27 2024-09-21T06:02:49.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.178+0000 7f9475170740 1 -- 172.21.15.117:0/3876558275 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a70ace8880 con 0x55a70a906390 2024-09-21T06:02:49.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.178+0000 7f9447fff640 1 -- 172.21.15.117:0/3876558275 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9454105610 con 0x55a70a906390 2024-09-21T06:02:49.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.178+0000 7f9447fff640 1 -- 172.21.15.117:0/3876558275 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f945404c060 con 0x55a70a906390 2024-09-21T06:02:49.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.178+0000 7f9475170740 10 client.25699 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:02:49.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.178+0000 7f9475170740 10 client.25699 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:02:49.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.178+0000 7f9475170740 20 client.25699 populate_metadata read hostname 'smithi117' 2024-09-21T06:02:49.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.178+0000 7f9475170740 1 --2- 172.21.15.117:0/3876558275 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55a70a8e59f0 0x55a70acebcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:49.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.179+0000 7f9475170740 4 client.25699 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:02:49.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.179+0000 7f9475170740 1 -- 172.21.15.117:0/3876558275 --> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f945c018070 con 0x55a70a8e59f0 2024-09-21T06:02:49.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.179+0000 7f946c998640 1 --2- 172.21.15.117:0/3876558275 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55a70a8e59f0 0x55a70acebcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:49.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.180+0000 7f946c998640 1 --2- 172.21.15.117:0/3876558275 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55a70a8e59f0 0x55a70acebcf0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6fb90d2efd3e976e server_cookie=32507fe3bcb75ad1 in_seq=0 out_seq=0 2024-09-21T06:02:49.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.180+0000 7f9447fff640 10 client.25699 ms_handle_connect on v2:172.21.15.192:6838/1727623231 2024-09-21T06:02:49.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.181+0000 7f9447fff640 1 -- 172.21.15.117:0/3876558275 <== mds.1 v2:172.21.15.192:6838/1727623231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f945c018070 con 0x55a70a8e59f0 2024-09-21T06:02:49.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.181+0000 7f9447fff640 10 client.25699 handle_command_reply: tid=0 2024-09-21T06:02:49.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.181+0000 7f9475170740 1 client.25699 shutdown 2024-09-21T06:02:49.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.182+0000 7f9475170740 1 -- 172.21.15.117:0/3876558275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70a906390 msgr2=0x55a70acd9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:49.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.182+0000 7f9475170740 1 --2- 172.21.15.117:0/3876558275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70a906390 0x55a70acd9fc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f94541051f0 tx=0x7f9454003f30 comp rx=0 tx=0).stop 2024-09-21T06:02:49.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.182+0000 7f9475170740 1 -- 172.21.15.117:0/3876558275 shutdown_connections 2024-09-21T06:02:49.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.182+0000 7f9475170740 1 --2- 172.21.15.117:0/3876558275 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55a70a8e59f0 0x55a70acebcf0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:49.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.182+0000 7f9475170740 1 --2- 172.21.15.117:0/3876558275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70a927070 0x55a70ace5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:49.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.182+0000 7f9447fff640 0 client.25699 ms_handle_reset on v2:172.21.15.192:6838/1727623231 2024-09-21T06:02:49.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.182+0000 7f9475170740 1 --2- 172.21.15.117:0/3876558275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70a906390 0x55a70acd9fc0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:49.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.182+0000 7f9475170740 1 --2- 172.21.15.117:0/3876558275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70a8fe1b0 0x55a70acd9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:49.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.182+0000 7f9475170740 1 -- 172.21.15.117:0/3876558275 >> 172.21.15.117:0/3876558275 conn(0x55a70a91b460 msgr2=0x55a70acd1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:49.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.182+0000 7f9475170740 1 -- 172.21.15.117:0/3876558275 shutdown_connections 2024-09-21T06:02:49.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.182+0000 7f9475170740 1 -- 172.21.15.117:0/3876558275 wait complete. 2024-09-21T06:02:49.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.182+0000 7f9475170740 20 client.25699 trim_cache size 0 max 16384 2024-09-21T06:02:49.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.201+0000 7f9475170740 1 Processor -- start 2024-09-21T06:02:49.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.201+0000 7f9475170740 1 -- start start 2024-09-21T06:02:49.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.201+0000 7f9475170740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70a8e59f0 0x55a70acd9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:49.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.201+0000 7f9475170740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70a8fe1b0 0x55a70acd13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:49.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.201+0000 7f9475170740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70a906390 0x55a70acd1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:49.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.201+0000 7f9475170740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f94681b1830 con 0x55a70a906390 2024-09-21T06:02:49.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.201+0000 7f9475170740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f94681b3370 con 0x55a70a8e59f0 2024-09-21T06:02:49.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.201+0000 7f9475170740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f94681b1f00 con 0x55a70a8fe1b0 2024-09-21T06:02:49.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.202+0000 7f946c998640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70a8e59f0 0x55a70acd9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:49.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.202+0000 7f946d199640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70a906390 0x55a70acd1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:49.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.202+0000 7f946c998640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70a8e59f0 0x55a70acd9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:49360/0 (socket says 172.21.15.117:49360) 2024-09-21T06:02:49.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.202+0000 7f946c998640 1 -- 172.21.15.117:0/2265847346 learned_addr learned my addr 172.21.15.117:0/2265847346 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:49.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.202+0000 7f9467fff640 1 --2- 172.21.15.117:0/2265847346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70a8fe1b0 0x55a70acd13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:49.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.202+0000 7f946d199640 1 -- 172.21.15.117:0/2265847346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70a8fe1b0 msgr2=0x55a70acd13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:49.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.202+0000 7f946d199640 1 --2- 172.21.15.117:0/2265847346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70a8fe1b0 0x55a70acd13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:49.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.202+0000 7f946d199640 1 -- 172.21.15.117:0/2265847346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70a8e59f0 msgr2=0x55a70acd9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:49.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.202+0000 7f946d199640 1 --2- 172.21.15.117:0/2265847346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70a8e59f0 0x55a70acd9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:49.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.202+0000 7f946d199640 1 -- 172.21.15.117:0/2265847346 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a70acd05b0 con 0x55a70a906390 2024-09-21T06:02:49.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.203+0000 7f946d199640 1 --2- 172.21.15.117:0/2265847346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70a906390 0x55a70acd1900 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f946000ffc0 tx=0x7f9460012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e022809671c9a566 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:49.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.203+0000 7f94527fc640 1 -- 172.21.15.117:0/2265847346 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9460014e10 con 0x55a70a906390 2024-09-21T06:02:49.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.203+0000 7f94527fc640 1 -- 172.21.15.117:0/2265847346 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9460026070 con 0x55a70a906390 2024-09-21T06:02:49.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.203+0000 7f94527fc640 1 -- 172.21.15.117:0/2265847346 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9460015830 con 0x55a70a906390 2024-09-21T06:02:49.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.204+0000 7f9475170740 1 -- 172.21.15.117:0/2265847346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70a906390 msgr2=0x55a70acd1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:49.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.204+0000 7f9475170740 1 --2- 172.21.15.117:0/2265847346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70a906390 0x55a70acd1900 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f946000ffc0 tx=0x7f9460012000 comp rx=0 tx=0).stop 2024-09-21T06:02:49.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.204+0000 7f9475170740 1 -- 172.21.15.117:0/2265847346 shutdown_connections 2024-09-21T06:02:49.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.204+0000 7f9475170740 1 --2- 172.21.15.117:0/2265847346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70a906390 0x55a70acd1900 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:49.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.204+0000 7f9475170740 1 --2- 172.21.15.117:0/2265847346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70a8fe1b0 0x55a70acd13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:49.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.204+0000 7f9475170740 1 --2- 172.21.15.117:0/2265847346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70a8e59f0 0x55a70acd9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:49.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.204+0000 7f9475170740 1 -- 172.21.15.117:0/2265847346 >> 172.21.15.117:0/2265847346 conn(0x55a70a91b460 msgr2=0x55a70acd9830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:49.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.204+0000 7f9475170740 1 -- 172.21.15.117:0/2265847346 shutdown_connections 2024-09-21T06:02:49.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.204+0000 7f9475170740 1 -- 172.21.15.117:0/2265847346 wait complete. 2024-09-21T06:02:49.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.205+0000 7f9475170740 1 -- start start 2024-09-21T06:02:49.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.205+0000 7f9475170740 1 Processor -- start 2024-09-21T06:02:49.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.206+0000 7f9475170740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70a8e59f0 0x55a70acde170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:49.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.206+0000 7f9475170740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70a8fe1b0 0x55a70acde710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:49.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.206+0000 7f9475170740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70a906390 0x55a70ad55620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:49.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.206+0000 7f9475170740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f94681b2ca0 con 0x55a70a8e59f0 2024-09-21T06:02:49.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.206+0000 7f9475170740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f94681b25d0 con 0x55a70a8fe1b0 2024-09-21T06:02:49.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.206+0000 7f9475170740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f94681af810 con 0x55a70a906390 2024-09-21T06:02:49.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.206+0000 7f9475170740 10 client.0 _ll_register_callbacks cb 0x7f9454005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:02:49.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.206+0000 7f946c998640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70a8e59f0 0x55a70acde170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:49.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.206+0000 7f946c998640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70a8e59f0 0x55a70acde170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51174/0 (socket says 172.21.15.117:51174) 2024-09-21T06:02:49.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.206+0000 7f946c998640 1 -- 172.21.15.117:0/2927101207 learned_addr learned my addr 172.21.15.117:0/2927101207 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:49.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.206+0000 7f9467fff640 1 --2- 172.21.15.117:0/2927101207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70a8fe1b0 0x55a70acde710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:49.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.206+0000 7f946c998640 1 -- 172.21.15.117:0/2927101207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70a906390 msgr2=0x55a70ad55620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:49.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.206+0000 7f946c998640 1 --2- 172.21.15.117:0/2927101207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70a906390 0x55a70ad55620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:49.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.206+0000 7f946c998640 1 -- 172.21.15.117:0/2927101207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70a8fe1b0 msgr2=0x55a70acde710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:49.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.206+0000 7f946c998640 1 --2- 172.21.15.117:0/2927101207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70a8fe1b0 0x55a70acde710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:49.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.206+0000 7f946c998640 1 -- 172.21.15.117:0/2927101207 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a70ad55b60 con 0x55a70a8e59f0 2024-09-21T06:02:49.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.206+0000 7f9467fff640 1 --2- 172.21.15.117:0/2927101207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70a8fe1b0 0x55a70acde710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:02:49.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.206+0000 7f946c998640 1 --2- 172.21.15.117:0/2927101207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70a8e59f0 0x55a70acde170 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f945c008010 tx=0x7f945c013a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69bba1de94135a84 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:49.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.206+0000 7f9447fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:02:49.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.207+0000 7f9447fff640 1 -- 172.21.15.117:0/2927101207 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945c00db50 con 0x55a70a8e59f0 2024-09-21T06:02:49.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.207+0000 7f9447fff640 1 -- 172.21.15.117:0/2927101207 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f945c00dcf0 con 0x55a70a8e59f0 2024-09-21T06:02:49.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.207+0000 7f9475170740 1 -- 172.21.15.117:0/2927101207 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9460026070 con 0x55a70a8e59f0 2024-09-21T06:02:49.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.207+0000 7f9447fff640 1 -- 172.21.15.117:0/2927101207 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f945c02f480 con 0x55a70a8e59f0 2024-09-21T06:02:49.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.207+0000 7f9475170740 10 client.16422 fetch_fsmap learned FSMap version 27 2024-09-21T06:02:49.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.207+0000 7f9475170740 1 -- 172.21.15.117:0/2927101207 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a70ad56ce0 con 0x55a70a8e59f0 2024-09-21T06:02:49.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.207+0000 7f9447fff640 1 -- 172.21.15.117:0/2927101207 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f945c018040 con 0x55a70a8e59f0 2024-09-21T06:02:49.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.207+0000 7f9447fff640 1 -- 172.21.15.117:0/2927101207 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f945c013ce0 con 0x55a70a8e59f0 2024-09-21T06:02:49.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.207+0000 7f9475170740 10 client.16422 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:02:49.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.207+0000 7f9475170740 10 client.16422 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:02:49.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.207+0000 7f9475170740 20 client.16422 populate_metadata read hostname 'smithi117' 2024-09-21T06:02:49.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.207+0000 7f9475170740 1 --2- 172.21.15.117:0/2927101207 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55a70a927070 0x55a70ad59220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:49.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.207+0000 7f9475170740 4 client.16422 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:02:49.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.207+0000 7f9475170740 1 -- 172.21.15.117:0/2927101207 --> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f9460026070 con 0x55a70a927070 2024-09-21T06:02:49.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.209+0000 7f9467fff640 1 --2- 172.21.15.117:0/2927101207 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55a70a927070 0x55a70ad59220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:49.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.210+0000 7f9447fff640 10 client.16422 ms_handle_connect on v2:172.21.15.192:6838/1727623231 2024-09-21T06:02:49.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.210+0000 7f9467fff640 1 --2- 172.21.15.117:0/2927101207 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55a70a927070 0x55a70ad59220 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=413f9442e5402c3f server_cookie=aee1d80c1ed9580b in_seq=0 out_seq=0 2024-09-21T06:02:49.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.210+0000 7f9447fff640 1 -- 172.21.15.117:0/2927101207 <== mds.1 v2:172.21.15.192:6838/1727623231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f9460026070 con 0x55a70a927070 2024-09-21T06:02:49.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.210+0000 7f9447fff640 10 client.16422 handle_command_reply: tid=0 2024-09-21T06:02:49.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.210+0000 7f9475170740 1 client.16422 shutdown 2024-09-21T06:02:49.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.211+0000 7f9475170740 1 -- 172.21.15.117:0/2927101207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70a8e59f0 msgr2=0x55a70acde170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:49.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.211+0000 7f9475170740 1 --2- 172.21.15.117:0/2927101207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70a8e59f0 0x55a70acde170 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f945c008010 tx=0x7f945c013a70 comp rx=0 tx=0).stop 2024-09-21T06:02:49.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.211+0000 7f9475170740 1 -- 172.21.15.117:0/2927101207 shutdown_connections 2024-09-21T06:02:49.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.211+0000 7f9475170740 1 --2- 172.21.15.117:0/2927101207 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55a70a927070 0x55a70ad59220 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:49.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.211+0000 7f9475170740 1 --2- 172.21.15.117:0/2927101207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70a906390 0x55a70ad55620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:49.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.211+0000 7f9447fff640 0 client.16422 ms_handle_reset on v2:172.21.15.192:6838/1727623231 2024-09-21T06:02:49.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.211+0000 7f9475170740 1 --2- 172.21.15.117:0/2927101207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70a8fe1b0 0x55a70acde710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:49.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.211+0000 7f9475170740 1 --2- 172.21.15.117:0/2927101207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70a8e59f0 0x55a70acde170 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:49.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.211+0000 7f9475170740 1 -- 172.21.15.117:0/2927101207 >> 172.21.15.117:0/2927101207 conn(0x55a70a91b460 msgr2=0x55a70acd13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:49.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.211+0000 7f9475170740 1 -- 172.21.15.117:0/2927101207 shutdown_connections 2024-09-21T06:02:49.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.211+0000 7f9475170740 1 -- 172.21.15.117:0/2927101207 wait complete. 2024-09-21T06:02:49.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.211+0000 7f9475170740 20 client.16422 trim_cache size 0 max 16384 2024-09-21T06:02:49.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.212+0000 7f946ebfc640 1 -- 172.21.15.117:0/3693138177 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f94580b6780 msgr2=0x7f94580b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:49.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.212+0000 7f946ebfc640 1 --2- 172.21.15.117:0/3693138177 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f94580b6780 0x7f94580b8c90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9460008da0 tx=0x7f94600087e0 comp rx=0 tx=0).stop 2024-09-21T06:02:49.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.212+0000 7f946ebfc640 1 -- 172.21.15.117:0/3693138177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94681083c0 msgr2=0x7f9468199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:49.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.212+0000 7f946ebfc640 1 --2- 172.21.15.117:0/3693138177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94681083c0 0x7f9468199760 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f9454031a90 tx=0x7f9454004270 comp rx=0 tx=0).stop 2024-09-21T06:02:49.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.212+0000 7f946ebfc640 1 -- 172.21.15.117:0/3693138177 shutdown_connections 2024-09-21T06:02:49.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.212+0000 7f946ebfc640 1 --2- 172.21.15.117:0/3693138177 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f94580b6780 0x7f94580b8c90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:49.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.212+0000 7f946ebfc640 1 --2- 172.21.15.117:0/3693138177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94681083c0 0x7f9468199760 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:49.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.212+0000 7f946ebfc640 1 --2- 172.21.15.117:0/3693138177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9468108d10 0x7f9468199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:49.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.212+0000 7f946ebfc640 1 --2- 172.21.15.117:0/3693138177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9468102a80 0x7f9468199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:49.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.212+0000 7f946ebfc640 1 -- 172.21.15.117:0/3693138177 >> 172.21.15.117:0/3693138177 conn(0x7f9468079740 msgr2=0x7f9468105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:49.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.212+0000 7f946ebfc640 1 -- 172.21.15.117:0/3693138177 shutdown_connections 2024-09-21T06:02:49.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:49.212+0000 7f946ebfc640 1 -- 172.21.15.117:0/3693138177 wait complete. 2024-09-21T06:02:50.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:02:49 smithi117 ceph-mon[30947]: pgmap v492: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 955 MiB/s wr, 2.23k op/s 2024-09-21T06:02:50.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:02:49 smithi138 ceph-mon[34391]: pgmap v492: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 955 MiB/s wr, 2.23k op/s 2024-09-21T06:02:50.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:02:49 smithi192 ceph-mon[34700]: pgmap v492: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 955 MiB/s wr, 2.23k op/s 2024-09-21T06:02:52.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:02:51 smithi138 ceph-mon[34391]: pgmap v493: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 829 MiB/s wr, 2.02k op/s 2024-09-21T06:02:52.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:02:51 smithi192 ceph-mon[34700]: pgmap v493: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 829 MiB/s wr, 2.02k op/s 2024-09-21T06:02:52.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:02:51 smithi117 ceph-mon[30947]: pgmap v493: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 829 MiB/s wr, 2.02k op/s 2024-09-21T06:02:53.435 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:02:54.073 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:02:54.116 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:02:53 smithi138 ceph-mon[34391]: pgmap v494: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 873 MiB/s wr, 1.99k op/s 2024-09-21T06:02:54.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:02:53 smithi192 ceph-mon[34700]: pgmap v494: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 873 MiB/s wr, 1.99k op/s 2024-09-21T06:02:54.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:02:53 smithi117 ceph-mon[30947]: pgmap v494: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 873 MiB/s wr, 1.99k op/s 2024-09-21T06:02:55.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.853+0000 7f387e9ee640 1 -- 172.21.15.117:0/78761191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f387811cc30 msgr2=0x7f387811f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:55.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.853+0000 7f387e9ee640 1 --2- 172.21.15.117:0/78761191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f387811cc30 0x7f387811f140 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3868009900 tx=0x7f386802a440 comp rx=0 tx=0).stop 2024-09-21T06:02:55.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.854+0000 7f387e9ee640 1 -- 172.21.15.117:0/78761191 shutdown_connections 2024-09-21T06:02:55.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.854+0000 7f387e9ee640 1 --2- 172.21.15.117:0/78761191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f387811cc30 0x7f387811f140 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:55.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.854+0000 7f387e9ee640 1 --2- 172.21.15.117:0/78761191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f387810a9e0 0x7f387810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:55.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.854+0000 7f387e9ee640 1 --2- 172.21.15.117:0/78761191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3878111f40 0x7f387810a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:55.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.854+0000 7f387e9ee640 1 -- 172.21.15.117:0/78761191 >> 172.21.15.117:0/78761191 conn(0x7f3878070ec0 msgr2=0x7f38780712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:55.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.861+0000 7f387e9ee640 1 -- 172.21.15.117:0/78761191 shutdown_connections 2024-09-21T06:02:55.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.863+0000 7f387e9ee640 1 -- 172.21.15.117:0/78761191 wait complete. 2024-09-21T06:02:55.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.864+0000 7f387e9ee640 1 Processor -- start 2024-09-21T06:02:55.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.873+0000 7f387e9ee640 1 -- start start 2024-09-21T06:02:55.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.874+0000 7f387e9ee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f387810a9e0 0x7f387811c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:55.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.874+0000 7f387e9ee640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3878111f40 0x7f38781154f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:55.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.874+0000 7f387e9ee640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f387811cc30 0x7f3878115a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:55.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.874+0000 7f387e9ee640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3878122930 con 0x7f387810a9e0 2024-09-21T06:02:55.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.875+0000 7f387e9ee640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f38781227b0 con 0x7f3878111f40 2024-09-21T06:02:55.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.875+0000 7f387e9ee640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3878122ab0 con 0x7f387811cc30 2024-09-21T06:02:55.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.878+0000 7f387cf8b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f387811cc30 0x7f3878115a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:55.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.879+0000 7f3877fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f387810a9e0 0x7f387811c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:55.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.879+0000 7f387cf8b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f387811cc30 0x7f3878115a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:54056/0 (socket says 172.21.15.117:54056) 2024-09-21T06:02:55.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.879+0000 7f387cf8b640 1 -- 172.21.15.117:0/3271788075 learned_addr learned my addr 172.21.15.117:0/3271788075 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:55.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.880+0000 7f387cf8b640 1 -- 172.21.15.117:0/3271788075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3878111f40 msgr2=0x7f38781154f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:02:55.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.880+0000 7f387cf8b640 1 --2- 172.21.15.117:0/3271788075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3878111f40 0x7f38781154f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:55.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.880+0000 7f387cf8b640 1 -- 172.21.15.117:0/3271788075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f387810a9e0 msgr2=0x7f387811c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:55.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.880+0000 7f387cf8b640 1 --2- 172.21.15.117:0/3271788075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f387810a9e0 0x7f387811c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:55.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.880+0000 7f387cf8b640 1 -- 172.21.15.117:0/3271788075 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38781160a0 con 0x7f387811cc30 2024-09-21T06:02:55.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.881+0000 7f387cf8b640 1 --2- 172.21.15.117:0/3271788075 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f387811cc30 0x7f3878115a30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3868009850 tx=0x7f3868003e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:55.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.885+0000 7f38757fa640 1 -- 172.21.15.117:0/3271788075 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3868041020 con 0x7f387811cc30 2024-09-21T06:02:55.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.885+0000 7f38757fa640 1 -- 172.21.15.117:0/3271788075 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f386803db30 con 0x7f387811cc30 2024-09-21T06:02:55.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.885+0000 7f38757fa640 1 -- 172.21.15.117:0/3271788075 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38680319a0 con 0x7f387811cc30 2024-09-21T06:02:55.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.887+0000 7f387e9ee640 1 -- 172.21.15.117:0/3271788075 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3878116330 con 0x7f387811cc30 2024-09-21T06:02:55.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.887+0000 7f387e9ee640 1 -- 172.21.15.117:0/3271788075 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f387811b180 con 0x7f387811cc30 2024-09-21T06:02:55.892 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:02:55 smithi117 ceph-mon[30947]: pgmap v495: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 945 MiB/s wr, 2.00k op/s 2024-09-21T06:02:55.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.893+0000 7f3884f62740 1 Processor -- start 2024-09-21T06:02:55.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.894+0000 7f3884f62740 1 -- start start 2024-09-21T06:02:55.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.903+0000 7f3884f62740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f6fc03070 0x559f6ffaa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:55.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.903+0000 7f3884f62740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f6fbda1b0 0x559f6ffaabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:55.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.903+0000 7f3884f62740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f6fbe2390 0x559f6ffab120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:55.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.903+0000 7f3884f62740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559f6ff20ea0 con 0x559f6fbe2390 2024-09-21T06:02:55.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.903+0000 7f3884f62740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559f6ff102d0 con 0x559f6fc03070 2024-09-21T06:02:55.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.903+0000 7f3884f62740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x559f6ff140e0 con 0x559f6fbda1b0 2024-09-21T06:02:55.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.905+0000 7f38757fa640 1 -- 172.21.15.117:0/3271788075 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f3868044020 con 0x7f387811cc30 2024-09-21T06:02:55.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.906+0000 7f3877fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f6fbda1b0 0x559f6ffaabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:55.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.907+0000 7f3877fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f6fbda1b0 0x559f6ffaabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:54060/0 (socket says 172.21.15.117:54060) 2024-09-21T06:02:55.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.907+0000 7f3877fff640 1 -- 172.21.15.117:0/3325193224 learned_addr learned my addr 172.21.15.117:0/3325193224 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:55.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.907+0000 7f38757fa640 1 --2- 172.21.15.117:0/3271788075 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f38600b6880 0x7f38600b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:55.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.908+0000 7f38757fa640 1 -- 172.21.15.117:0/3271788075 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f3868108730 con 0x7f387811cc30 2024-09-21T06:02:55.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.909+0000 7f3877fff640 1 -- 172.21.15.117:0/3325193224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f6fc03070 msgr2=0x559f6ffaa6a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:02:55.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.909+0000 7f3877fff640 1 --2- 172.21.15.117:0/3325193224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f6fc03070 0x559f6ffaa6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:55.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.909+0000 7f3877fff640 1 -- 172.21.15.117:0/3325193224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f6fbe2390 msgr2=0x559f6ffab120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:55.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.910+0000 7f3877fff640 1 --2- 172.21.15.117:0/3325193224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f6fbe2390 0x559f6ffab120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:55.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.910+0000 7f3877fff640 1 -- 172.21.15.117:0/3325193224 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f6fc6e650 con 0x559f6fbda1b0 2024-09-21T06:02:55.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.912+0000 7f3877fff640 1 --2- 172.21.15.117:0/3325193224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f6fbda1b0 0x559f6ffaabe0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3870009fd0 tx=0x7f387000c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=605e31a70ea47a5e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:55.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.916+0000 7f385e7fc640 1 -- 172.21.15.117:0/3325193224 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387001a070 con 0x559f6fbda1b0 2024-09-21T06:02:55.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.917+0000 7f385e7fc640 1 -- 172.21.15.117:0/3325193224 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f387000aaa0 con 0x559f6fbda1b0 2024-09-21T06:02:55.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.918+0000 7f3884f62740 1 -- 172.21.15.117:0/3325193224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f6fbda1b0 msgr2=0x559f6ffaabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:55.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.918+0000 7f387cf8b640 1 --2- 172.21.15.117:0/3271788075 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f38600b6880 0x7f38600b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:55.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.918+0000 7f385e7fc640 1 -- 172.21.15.117:0/3325193224 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387001a070 con 0x559f6fbda1b0 2024-09-21T06:02:55.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.918+0000 7f3884f62740 1 --2- 172.21.15.117:0/3325193224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f6fbda1b0 0x559f6ffaabe0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3870009fd0 tx=0x7f387000c890 comp rx=0 tx=0).stop 2024-09-21T06:02:55.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.919+0000 7f3884f62740 1 -- 172.21.15.117:0/3325193224 shutdown_connections 2024-09-21T06:02:55.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.919+0000 7f3884f62740 1 --2- 172.21.15.117:0/3325193224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f6fbe2390 0x559f6ffab120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:55.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.919+0000 7f3884f62740 1 --2- 172.21.15.117:0/3325193224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f6fbda1b0 0x559f6ffaabe0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:55.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.919+0000 7f3884f62740 1 --2- 172.21.15.117:0/3325193224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f6fc03070 0x559f6ffaa6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:55.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.919+0000 7f3884f62740 1 -- 172.21.15.117:0/3325193224 >> 172.21.15.117:0/3325193224 conn(0x559f6fbf7460 msgr2=0x559f6ffa8590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:55.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.925+0000 7f387cf8b640 1 --2- 172.21.15.117:0/3271788075 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f38600b6880 0x7f38600b8d90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f386802aa00 tx=0x7f386800b040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:55.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.927+0000 7f3884f62740 1 -- 172.21.15.117:0/3325193224 shutdown_connections 2024-09-21T06:02:55.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.929+0000 7f3884f62740 1 -- 172.21.15.117:0/3325193224 wait complete. 2024-09-21T06:02:55.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.930+0000 7f3884f62740 1 -- start start 2024-09-21T06:02:55.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.930+0000 7f3884f62740 1 Processor -- start 2024-09-21T06:02:55.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.931+0000 7f3884f62740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f6fbda1b0 0x559f6ffb5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:55.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.932+0000 7f3884f62740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f6fbe2390 0x559f6ffb61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:55.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.932+0000 7f3884f62740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f6fc03070 0x559f6ffb66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:55.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.932+0000 7f3884f62740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559f6fd71700 con 0x559f6fbe2390 2024-09-21T06:02:55.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.932+0000 7f3884f62740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559f6feab160 con 0x559f6fbda1b0 2024-09-21T06:02:55.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.932+0000 7f3884f62740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x559f6ffb8ed0 con 0x559f6fc03070 2024-09-21T06:02:55.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.932+0000 7f3884f62740 10 client.0 _ll_register_callbacks cb 0x559f6fec49e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:02:55.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.936+0000 7f3877fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f6fbe2390 0x559f6ffb61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:55.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.936+0000 7f3877fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f6fbe2390 0x559f6ffb61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50526/0 (socket says 172.21.15.117:50526) 2024-09-21T06:02:55.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.936+0000 7f3877fff640 1 -- 172.21.15.117:0/3187879381 learned_addr learned my addr 172.21.15.117:0/3187879381 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:55.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.941+0000 7f38777fe640 1 --2- 172.21.15.117:0/3187879381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f6fc03070 0x559f6ffb66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:55.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.941+0000 7f38777fe640 1 --2- 172.21.15.117:0/3187879381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f6fbda1b0 0x559f6ffb5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:55.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.941+0000 7f3877fff640 1 -- 172.21.15.117:0/3187879381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f6fc03070 msgr2=0x559f6ffb66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:55.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.941+0000 7f3877fff640 1 --2- 172.21.15.117:0/3187879381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f6fc03070 0x559f6ffb66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:55.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.941+0000 7f3877fff640 1 -- 172.21.15.117:0/3187879381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f6fbda1b0 msgr2=0x559f6ffb5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:55.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.941+0000 7f3877fff640 1 --2- 172.21.15.117:0/3187879381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f6fbda1b0 0x559f6ffb5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:55.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.941+0000 7f3877fff640 1 -- 172.21.15.117:0/3187879381 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f6fed8c90 con 0x559f6fbe2390 2024-09-21T06:02:55.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.941+0000 7f3877fff640 1 --2- 172.21.15.117:0/3187879381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f6fbe2390 0x559f6ffb61b0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3870015620 tx=0x7f3870009d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f7515834226ee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:55.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.942+0000 7f38577fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:02:55.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.942+0000 7f38577fe640 1 -- 172.21.15.117:0/3187879381 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3870023020 con 0x559f6fbe2390 2024-09-21T06:02:55.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.942+0000 7f38577fe640 1 -- 172.21.15.117:0/3187879381 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f38700158d0 con 0x559f6fbe2390 2024-09-21T06:02:55.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.946+0000 7f3884f62740 1 -- 172.21.15.117:0/3187879381 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f387000aaa0 con 0x559f6fbe2390 2024-09-21T06:02:55.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.946+0000 7f38577fe640 1 -- 172.21.15.117:0/3187879381 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f387001a050 con 0x559f6fbe2390 2024-09-21T06:02:55.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.948+0000 7f3884f62740 10 client.16428 fetch_fsmap learned FSMap version 27 2024-09-21T06:02:55.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.948+0000 7f3884f62740 1 -- 172.21.15.117:0/3187879381 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559f6ffc4b50 con 0x559f6fbe2390 2024-09-21T06:02:55.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.950+0000 7f38577fe640 1 -- 172.21.15.117:0/3187879381 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f387000aaa0 con 0x559f6fbe2390 2024-09-21T06:02:55.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.950+0000 7f38577fe640 1 -- 172.21.15.117:0/3187879381 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f387001fc50 con 0x559f6fbe2390 2024-09-21T06:02:55.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.951+0000 7f3884f62740 10 client.16428 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:02:55.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.951+0000 7f3884f62740 10 client.16428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:02:55.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.952+0000 7f3884f62740 20 client.16428 populate_metadata read hostname 'smithi117' 2024-09-21T06:02:55.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.952+0000 7f3884f62740 1 --2- 172.21.15.117:0/3187879381 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x559f6fbc19f0 0x559f6ffc7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:55.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.952+0000 7f3884f62740 4 client.16428 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:02:55.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.952+0000 7f3884f62740 1 -- 172.21.15.117:0/3187879381 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f6ff22df0 con 0x559f6fbc19f0 2024-09-21T06:02:55.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.965+0000 7f38777fe640 1 --2- 172.21.15.117:0/3187879381 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x559f6fbc19f0 0x559f6ffc7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:55.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.976+0000 7f38577fe640 10 client.16428 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:02:55.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.976+0000 7f38777fe640 1 --2- 172.21.15.117:0/3187879381 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x559f6fbc19f0 0x559f6ffc7fc0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1d2996dab672c56 server_cookie=f4b4a1432abfb382 in_seq=0 out_seq=0 2024-09-21T06:02:55.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.980+0000 7f38577fe640 1 -- 172.21.15.117:0/3187879381 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559f6ff22df0 con 0x559f6fbc19f0 2024-09-21T06:02:55.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.980+0000 7f38577fe640 10 client.16428 handle_command_reply: tid=0 2024-09-21T06:02:55.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.980+0000 7f3884f62740 1 client.16428 shutdown 2024-09-21T06:02:55.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.985+0000 7f3884f62740 1 -- 172.21.15.117:0/3187879381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f6fbe2390 msgr2=0x559f6ffb61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:55.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.985+0000 7f3884f62740 1 --2- 172.21.15.117:0/3187879381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f6fbe2390 0x559f6ffb61b0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3870015620 tx=0x7f3870009d80 comp rx=0 tx=0).stop 2024-09-21T06:02:55.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.985+0000 7f3884f62740 1 -- 172.21.15.117:0/3187879381 shutdown_connections 2024-09-21T06:02:55.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.985+0000 7f3884f62740 1 --2- 172.21.15.117:0/3187879381 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x559f6fbc19f0 0x559f6ffc7fc0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:55.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.985+0000 7f3884f62740 1 --2- 172.21.15.117:0/3187879381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f6fbe2390 0x559f6ffb61b0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:55.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.985+0000 7f3884f62740 1 --2- 172.21.15.117:0/3187879381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f6fc03070 0x559f6ffb66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:55.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.985+0000 7f3884f62740 1 --2- 172.21.15.117:0/3187879381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f6fbda1b0 0x559f6ffb5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:55.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.985+0000 7f3884f62740 1 -- 172.21.15.117:0/3187879381 >> 172.21.15.117:0/3187879381 conn(0x559f6fbf7460 msgr2=0x559f6ffad880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:55.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.985+0000 7f38577fe640 0 client.16428 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:02:55.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.986+0000 7f3884f62740 1 -- 172.21.15.117:0/3187879381 shutdown_connections 2024-09-21T06:02:55.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.986+0000 7f3884f62740 1 -- 172.21.15.117:0/3187879381 wait complete. 2024-09-21T06:02:55.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:55.986+0000 7f3884f62740 20 client.16428 trim_cache size 0 max 16384 2024-09-21T06:02:56.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.023+0000 7f3884f62740 1 Processor -- start 2024-09-21T06:02:56.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.024+0000 7f3884f62740 1 -- start start 2024-09-21T06:02:56.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.025+0000 7f3884f62740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f6fbda1b0 0x559f6ffb7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:56.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.025+0000 7f3884f62740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f6fbe2390 0x559f6ffabde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:56.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.025+0000 7f3884f62740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f6fc03070 0x559f6ffb2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:56.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.025+0000 7f3884f62740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f38781c6cb0 con 0x559f6fbda1b0 2024-09-21T06:02:56.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.025+0000 7f3884f62740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f38781c87f0 con 0x559f6fbe2390 2024-09-21T06:02:56.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.025+0000 7f3884f62740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f38781c7380 con 0x559f6fc03070 2024-09-21T06:02:56.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.027+0000 7f38777fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f6fbda1b0 0x559f6ffb7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:56.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.027+0000 7f38777fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f6fbda1b0 0x559f6ffb7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50532/0 (socket says 172.21.15.117:50532) 2024-09-21T06:02:56.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.027+0000 7f38777fe640 1 -- 172.21.15.117:0/1620417104 learned_addr learned my addr 172.21.15.117:0/1620417104 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:56.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.028+0000 7f3877fff640 1 --2- 172.21.15.117:0/1620417104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f6fbe2390 0x559f6ffabde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:56.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.029+0000 7f38777fe640 1 -- 172.21.15.117:0/1620417104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f6fc03070 msgr2=0x559f6ffb2e60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:02:56.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.029+0000 7f38777fe640 1 --2- 172.21.15.117:0/1620417104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f6fc03070 0x559f6ffb2e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:56.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.029+0000 7f38777fe640 1 -- 172.21.15.117:0/1620417104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f6fbe2390 msgr2=0x559f6ffabde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:56.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.029+0000 7f38777fe640 1 --2- 172.21.15.117:0/1620417104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f6fbe2390 0x559f6ffabde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:56.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.029+0000 7f38777fe640 1 -- 172.21.15.117:0/1620417104 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f6ffad500 con 0x559f6fbda1b0 2024-09-21T06:02:56.034 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.031+0000 7f38777fe640 1 --2- 172.21.15.117:0/1620417104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f6fbda1b0 0x559f6ffb7b50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f386c000c00 tx=0x7f386c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60627fd8c3e66931 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:56.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.031+0000 7f385dffb640 1 -- 172.21.15.117:0/1620417104 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386c01a480 con 0x559f6fbda1b0 2024-09-21T06:02:56.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.032+0000 7f385dffb640 1 -- 172.21.15.117:0/1620417104 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f386c01aad0 con 0x559f6fbda1b0 2024-09-21T06:02:56.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.032+0000 7f385dffb640 1 -- 172.21.15.117:0/1620417104 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386c010a50 con 0x559f6fbda1b0 2024-09-21T06:02:56.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.034+0000 7f3884f62740 1 -- 172.21.15.117:0/1620417104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f6fbda1b0 msgr2=0x559f6ffb7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:56.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.034+0000 7f3884f62740 1 --2- 172.21.15.117:0/1620417104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f6fbda1b0 0x559f6ffb7b50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f386c000c00 tx=0x7f386c005f90 comp rx=0 tx=0).stop 2024-09-21T06:02:56.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.035+0000 7f3884f62740 1 -- 172.21.15.117:0/1620417104 shutdown_connections 2024-09-21T06:02:56.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.035+0000 7f3884f62740 1 --2- 172.21.15.117:0/1620417104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f6fc03070 0x559f6ffb2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:56.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.035+0000 7f3884f62740 1 --2- 172.21.15.117:0/1620417104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f6fbe2390 0x559f6ffabde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:56.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.035+0000 7f3884f62740 1 --2- 172.21.15.117:0/1620417104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f6fbda1b0 0x559f6ffb7b50 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:56.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.035+0000 7f3884f62740 1 -- 172.21.15.117:0/1620417104 >> 172.21.15.117:0/1620417104 conn(0x559f6fbf7460 msgr2=0x559f6ffab6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:56.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.036+0000 7f3884f62740 1 -- 172.21.15.117:0/1620417104 shutdown_connections 2024-09-21T06:02:56.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.036+0000 7f3884f62740 1 -- 172.21.15.117:0/1620417104 wait complete. 2024-09-21T06:02:56.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.037+0000 7f3884f62740 1 -- start start 2024-09-21T06:02:56.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.038+0000 7f3884f62740 1 Processor -- start 2024-09-21T06:02:56.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.039+0000 7f3884f62740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f6fbda1b0 0x559f6ffcb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:56.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.039+0000 7f3884f62740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f6fbe2390 0x559f6ffc8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:56.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.039+0000 7f3884f62740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f6fc03070 0x559f6ffc94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:56.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.039+0000 7f3884f62740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f38781c8120 con 0x559f6fc03070 2024-09-21T06:02:56.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.039+0000 7f3884f62740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f38781c7a50 con 0x559f6fbda1b0 2024-09-21T06:02:56.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.039+0000 7f3884f62740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f38781c4c90 con 0x559f6fbe2390 2024-09-21T06:02:56.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.039+0000 7f3884f62740 10 client.0 _ll_register_callbacks cb 0x7f3870018070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:02:56.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.041+0000 7f38777fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f6fc03070 0x559f6ffc94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:56.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.041+0000 7f38777fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f6fc03070 0x559f6ffc94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50548/0 (socket says 172.21.15.117:50548) 2024-09-21T06:02:56.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.041+0000 7f38777fe640 1 -- 172.21.15.117:0/3487906083 learned_addr learned my addr 172.21.15.117:0/3487906083 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:02:56.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.041+0000 7f3877fff640 1 --2- 172.21.15.117:0/3487906083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f6fbe2390 0x559f6ffc8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:56.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.042+0000 7f38777fe640 1 -- 172.21.15.117:0/3487906083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f6fbe2390 msgr2=0x559f6ffc8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:56.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.042+0000 7f38777fe640 1 --2- 172.21.15.117:0/3487906083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f6fbe2390 0x559f6ffc8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:56.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.042+0000 7f38777fe640 1 -- 172.21.15.117:0/3487906083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f6fbda1b0 msgr2=0x559f6ffcb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:02:56.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.042+0000 7f38777fe640 1 --2- 172.21.15.117:0/3487906083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f6fbda1b0 0x559f6ffcb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:56.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.042+0000 7f38777fe640 1 -- 172.21.15.117:0/3487906083 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f6ffc9c30 con 0x559f6fc03070 2024-09-21T06:02:56.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.042+0000 7f38777fe640 1 --2- 172.21.15.117:0/3487906083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f6fc03070 0x559f6ffc94c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f386c001eb0 tx=0x7f386c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a8f33ec54f91fe3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:02:56.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.042+0000 7f38577fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:02:56.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.043+0000 7f38577fe640 1 -- 172.21.15.117:0/3487906083 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386c01a680 con 0x559f6fc03070 2024-09-21T06:02:56.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.043+0000 7f38577fe640 1 -- 172.21.15.117:0/3487906083 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f386c01a820 con 0x559f6fc03070 2024-09-21T06:02:56.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.043+0000 7f3884f62740 1 -- 172.21.15.117:0/3487906083 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f386c01aad0 con 0x559f6fc03070 2024-09-21T06:02:56.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.045+0000 7f38577fe640 1 -- 172.21.15.117:0/3487906083 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f386c036050 con 0x559f6fc03070 2024-09-21T06:02:56.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.046+0000 7f3884f62740 10 client.16434 fetch_fsmap learned FSMap version 27 2024-09-21T06:02:56.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.046+0000 7f3884f62740 1 -- 172.21.15.117:0/3487906083 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559f700327a0 con 0x559f6fc03070 2024-09-21T06:02:56.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.048+0000 7f38577fe640 1 -- 172.21.15.117:0/3487906083 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f386c02c070 con 0x559f6fc03070 2024-09-21T06:02:56.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.048+0000 7f38577fe640 1 -- 172.21.15.117:0/3487906083 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f386c024450 con 0x559f6fc03070 2024-09-21T06:02:56.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.049+0000 7f3884f62740 10 client.16434 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:02:56.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.049+0000 7f3884f62740 10 client.16434 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:02:56.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.049+0000 7f3884f62740 20 client.16434 populate_metadata read hostname 'smithi117' 2024-09-21T06:02:56.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.049+0000 7f3884f62740 1 --2- 172.21.15.117:0/3487906083 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x559f6fb96da0 0x559f70034e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:02:56.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.049+0000 7f3884f62740 4 client.16434 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:02:56.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.049+0000 7f3884f62740 1 -- 172.21.15.117:0/3487906083 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f386c01aad0 con 0x559f6fb96da0 2024-09-21T06:02:56.054 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.051+0000 7f3877fff640 1 --2- 172.21.15.117:0/3487906083 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x559f6fb96da0 0x559f70034e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:02:56.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.055+0000 7f3877fff640 1 --2- 172.21.15.117:0/3487906083 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x559f6fb96da0 0x559f70034e60 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c07032b38a63d845 server_cookie=3d338a1a90927ee2 in_seq=0 out_seq=0 2024-09-21T06:02:56.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.055+0000 7f38577fe640 10 client.16434 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:02:56.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.063+0000 7f38577fe640 1 -- 172.21.15.117:0/3487906083 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f386c01aad0 con 0x559f6fb96da0 2024-09-21T06:02:56.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.063+0000 7f38577fe640 10 client.16434 handle_command_reply: tid=0 2024-09-21T06:02:56.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.063+0000 7f3884f62740 1 client.16434 shutdown 2024-09-21T06:02:56.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.065+0000 7f3884f62740 1 -- 172.21.15.117:0/3487906083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f6fc03070 msgr2=0x559f6ffc94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:56.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.065+0000 7f3884f62740 1 --2- 172.21.15.117:0/3487906083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f6fc03070 0x559f6ffc94c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f386c001eb0 tx=0x7f386c005850 comp rx=0 tx=0).stop 2024-09-21T06:02:56.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.066+0000 7f3884f62740 1 -- 172.21.15.117:0/3487906083 shutdown_connections 2024-09-21T06:02:56.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.066+0000 7f3884f62740 1 --2- 172.21.15.117:0/3487906083 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x559f6fb96da0 0x559f70034e60 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:56.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.066+0000 7f3884f62740 1 --2- 172.21.15.117:0/3487906083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559f6fc03070 0x559f6ffc94c0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:56.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.066+0000 7f38577fe640 0 client.16434 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:02:56.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.067+0000 7f3884f62740 1 --2- 172.21.15.117:0/3487906083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f6fbe2390 0x559f6ffc8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:56.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.067+0000 7f3884f62740 1 --2- 172.21.15.117:0/3487906083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f6fbda1b0 0x559f6ffcb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:56.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.067+0000 7f3884f62740 1 -- 172.21.15.117:0/3487906083 >> 172.21.15.117:0/3487906083 conn(0x559f6fbf7460 msgr2=0x559f6ffabe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:56.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.068+0000 7f3884f62740 1 -- 172.21.15.117:0/3487906083 shutdown_connections 2024-09-21T06:02:56.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.068+0000 7f3884f62740 1 -- 172.21.15.117:0/3487906083 wait complete. 2024-09-21T06:02:56.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.069+0000 7f3884f62740 20 client.16434 trim_cache size 0 max 16384 2024-09-21T06:02:56.073 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:02:56.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.070+0000 7f387e9ee640 1 -- 172.21.15.117:0/3271788075 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f38600b6880 msgr2=0x7f38600b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:56.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.070+0000 7f387e9ee640 1 --2- 172.21.15.117:0/3271788075 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f38600b6880 0x7f38600b8d90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f386802aa00 tx=0x7f386800b040 comp rx=0 tx=0).stop 2024-09-21T06:02:56.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.070+0000 7f387e9ee640 1 -- 172.21.15.117:0/3271788075 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f387811cc30 msgr2=0x7f3878115a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:02:56.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.070+0000 7f387e9ee640 1 --2- 172.21.15.117:0/3271788075 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f387811cc30 0x7f3878115a30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3868009850 tx=0x7f3868003e90 comp rx=0 tx=0).stop 2024-09-21T06:02:56.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.070+0000 7f387e9ee640 1 -- 172.21.15.117:0/3271788075 shutdown_connections 2024-09-21T06:02:56.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.070+0000 7f387e9ee640 1 --2- 172.21.15.117:0/3271788075 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f38600b6880 0x7f38600b8d90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:56.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.070+0000 7f387e9ee640 1 --2- 172.21.15.117:0/3271788075 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f387811cc30 0x7f3878115a30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:56.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.070+0000 7f387e9ee640 1 --2- 172.21.15.117:0/3271788075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3878111f40 0x7f38781154f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:56.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.070+0000 7f387e9ee640 1 --2- 172.21.15.117:0/3271788075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f387810a9e0 0x7f387811c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:02:56.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.070+0000 7f387e9ee640 1 -- 172.21.15.117:0/3271788075 >> 172.21.15.117:0/3271788075 conn(0x7f3878070ec0 msgr2=0x7f3878073e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:02:56.074 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.071+0000 7f387e9ee640 1 -- 172.21.15.117:0/3271788075 shutdown_connections 2024-09-21T06:02:56.076 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:02:56.072+0000 7f387e9ee640 1 -- 172.21.15.117:0/3271788075 wait complete. 2024-09-21T06:02:56.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:02:55 smithi138 ceph-mon[34391]: pgmap v495: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 945 MiB/s wr, 2.00k op/s 2024-09-21T06:02:56.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:02:55 smithi192 ceph-mon[34700]: pgmap v495: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 945 MiB/s wr, 2.00k op/s 2024-09-21T06:02:58.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:02:57 smithi138 ceph-mon[34391]: pgmap v496: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 852 MiB/s wr, 1.69k op/s 2024-09-21T06:02:58.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:02:57 smithi192 ceph-mon[34700]: pgmap v496: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 852 MiB/s wr, 1.69k op/s 2024-09-21T06:02:58.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:02:57 smithi117 ceph-mon[30947]: pgmap v496: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 852 MiB/s wr, 1.69k op/s 2024-09-21T06:03:00.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:02:59 smithi138 ceph-mon[34391]: pgmap v497: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 958 MiB/s wr, 1.76k op/s 2024-09-21T06:03:00.151 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:03:00.152 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:03:00.224 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:02:59 smithi117 ceph-mon[30947]: pgmap v497: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 958 MiB/s wr, 1.76k op/s 2024-09-21T06:03:00.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:02:59 smithi192 ceph-mon[34700]: pgmap v497: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 958 MiB/s wr, 1.76k op/s 2024-09-21T06:03:00.810 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:01.269 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.264+0000 7fc8c1e7e640 1 -- 172.21.15.117:0/4262763543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8bc110350 msgr2=0x7fc8bc110b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:01.269 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.264+0000 7fc8c1e7e640 1 --2- 172.21.15.117:0/4262763543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8bc110350 0x7fc8bc110b70 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4009ca0 tx=0x7fc8b402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:03:01.270 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.265+0000 7fc8c1e7e640 1 -- 172.21.15.117:0/4262763543 shutdown_connections 2024-09-21T06:03:01.270 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.265+0000 7fc8c1e7e640 1 --2- 172.21.15.117:0/4262763543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8bc110350 0x7fc8bc110b70 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:01.270 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.265+0000 7fc8c1e7e640 1 --2- 172.21.15.117:0/4262763543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8bc10fa00 0x7fc8bc10fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:01.270 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.265+0000 7fc8c1e7e640 1 --2- 172.21.15.117:0/4262763543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8bc10a660 0x7fc8bc10aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:01.270 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.265+0000 7fc8c1e7e640 1 -- 172.21.15.117:0/4262763543 >> 172.21.15.117:0/4262763543 conn(0x7fc8bc070dd0 msgr2=0x7fc8bc0711e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:01.270 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.265+0000 7fc8c1e7e640 1 -- 172.21.15.117:0/4262763543 shutdown_connections 2024-09-21T06:03:01.271 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.267+0000 7fc8c1e7e640 1 -- 172.21.15.117:0/4262763543 wait complete. 2024-09-21T06:03:01.273 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.269+0000 7fc8c1e7e640 1 Processor -- start 2024-09-21T06:03:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.269+0000 7fc8c1e7e640 1 -- start start 2024-09-21T06:03:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.270+0000 7fc8c1e7e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8bc10a660 0x7fc8bc1a9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.270+0000 7fc8c1e7e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8bc10fa00 0x7fc8bc1a9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.270+0000 7fc8c1e7e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8bc110350 0x7fc8bc1aea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.270+0000 7fc8c1e7e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc8bc122840 con 0x7fc8bc110350 2024-09-21T06:03:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.270+0000 7fc8c1e7e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc8bc1226c0 con 0x7fc8bc10a660 2024-09-21T06:03:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.270+0000 7fc8c1e7e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc8bc1229c0 con 0x7fc8bc10fa00 2024-09-21T06:03:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.270+0000 7fc8bbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8bc110350 0x7fc8bc1aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.270+0000 7fc8bbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8bc110350 0x7fc8bc1aea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50570/0 (socket says 172.21.15.117:50570) 2024-09-21T06:03:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.270+0000 7fc8bbfff640 1 -- 172.21.15.117:0/702946754 learned_addr learned my addr 172.21.15.117:0/702946754 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.271+0000 7fc8bbfff640 1 -- 172.21.15.117:0/702946754 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8bc10fa00 msgr2=0x7fc8bc1a9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.271+0000 7fc8bbfff640 1 --2- 172.21.15.117:0/702946754 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8bc10fa00 0x7fc8bc1a9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.271+0000 7fc8bbfff640 1 -- 172.21.15.117:0/702946754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8bc10a660 msgr2=0x7fc8bc1a9260 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:03:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.271+0000 7fc8bbfff640 1 --2- 172.21.15.117:0/702946754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8bc10a660 0x7fc8bc1a9260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.271+0000 7fc8bbfff640 1 -- 172.21.15.117:0/702946754 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8bc1af260 con 0x7fc8bc110350 2024-09-21T06:03:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.271+0000 7fc8bbfff640 1 --2- 172.21.15.117:0/702946754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8bc110350 0x7fc8bc1aea90 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc8b402aeb0 tx=0x7fc8b40319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.272+0000 7fc8b97fa640 1 -- 172.21.15.117:0/702946754 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b4004450 con 0x7fc8bc110350 2024-09-21T06:03:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.272+0000 7fc8b97fa640 1 -- 172.21.15.117:0/702946754 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8b403f070 con 0x7fc8bc110350 2024-09-21T06:03:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.272+0000 7fc8b97fa640 1 -- 172.21.15.117:0/702946754 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b402f570 con 0x7fc8bc110350 2024-09-21T06:03:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.272+0000 7fc8c1e7e640 1 -- 172.21.15.117:0/702946754 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8bc1af4f0 con 0x7fc8bc110350 2024-09-21T06:03:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.273+0000 7fc8c1e7e640 1 -- 172.21.15.117:0/702946754 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8bc1b5c10 con 0x7fc8bc110350 2024-09-21T06:03:01.280 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.279+0000 7fc8c8dea740 1 Processor -- start 2024-09-21T06:03:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.279+0000 7fc8c8dea740 1 -- start start 2024-09-21T06:03:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.279+0000 7fc8c8dea740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7ef0271b0 0x55a7ef3cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.280+0000 7fc8c8dea740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a7eeffe2f0 0x55a7ef3ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:01.283 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.280+0000 7fc8c8dea740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7ef0064d0 0x55a7ef3cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.280+0000 7fc8c8dea740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a7ef3416f0 con 0x55a7ef0271b0 2024-09-21T06:03:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.280+0000 7fc8c8dea740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a7ef330b20 con 0x55a7ef0064d0 2024-09-21T06:03:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.280+0000 7fc8c8dea740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a7ef334930 con 0x55a7eeffe2f0 2024-09-21T06:03:01.291 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.282+0000 7fc8b3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7ef0271b0 0x55a7ef3cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:01.291 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.282+0000 7fc8b3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7ef0271b0 0x55a7ef3cc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50586/0 (socket says 172.21.15.117:50586) 2024-09-21T06:03:01.291 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.282+0000 7fc8b3fff640 1 -- 172.21.15.117:0/3209738955 learned_addr learned my addr 172.21.15.117:0/3209738955 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:01.291 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.285+0000 7fc8b3fff640 1 --2- 172.21.15.117:0/3209738955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7ef0064d0 0x55a7ef3cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:01.291 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.286+0000 7fc8bb7fe640 1 --2- 172.21.15.117:0/3209738955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a7eeffe2f0 0x55a7ef3ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:01.293 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.291+0000 7fc8b3fff640 1 -- 172.21.15.117:0/3209738955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a7eeffe2f0 msgr2=0x55a7ef3ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:01.293 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.291+0000 7fc8b3fff640 1 --2- 172.21.15.117:0/3209738955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a7eeffe2f0 0x55a7ef3ccef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:01.293 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.291+0000 7fc8b3fff640 1 -- 172.21.15.117:0/3209738955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7ef0064d0 msgr2=0x55a7ef3cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:01.293 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.291+0000 7fc8b3fff640 1 --2- 172.21.15.117:0/3209738955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7ef0064d0 0x55a7ef3cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:01.293 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.291+0000 7fc8b3fff640 1 -- 172.21.15.117:0/3209738955 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7ef2f6530 con 0x55a7ef0271b0 2024-09-21T06:03:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.293+0000 7fc8b3fff640 1 --2- 172.21.15.117:0/3209738955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7ef0271b0 0x55a7ef3cc9b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4013720 tx=0x7fc8a4005ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b7142c363e7b4d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.293+0000 7fc8b1ffb640 1 -- 172.21.15.117:0/3209738955 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4004220 con 0x55a7ef0271b0 2024-09-21T06:03:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.294+0000 7fc8b97fa640 1 -- 172.21.15.117:0/702946754 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fc8b4031ae0 con 0x7fc8bc110350 2024-09-21T06:03:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.294+0000 7fc8b1ffb640 1 -- 172.21.15.117:0/3209738955 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8a40043c0 con 0x55a7ef0271b0 2024-09-21T06:03:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.294+0000 7fc8b1ffb640 1 -- 172.21.15.117:0/3209738955 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4003c10 con 0x55a7ef0271b0 2024-09-21T06:03:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.295+0000 7fc8b97fa640 1 --2- 172.21.15.117:0/702946754 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc8a80b6880 0x7fc8a80b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.296+0000 7fc8b97fa640 1 -- 172.21.15.117:0/702946754 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fc8b4108820 con 0x7fc8bc110350 2024-09-21T06:03:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.296+0000 7fc8c8dea740 1 -- 172.21.15.117:0/3209738955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7ef0271b0 msgr2=0x55a7ef3cc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.296+0000 7fc8c8dea740 1 --2- 172.21.15.117:0/3209738955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7ef0271b0 0x55a7ef3cc9b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4013720 tx=0x7fc8a4005ec0 comp rx=0 tx=0).stop 2024-09-21T06:03:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.296+0000 7fc8bb7fe640 1 --2- 172.21.15.117:0/702946754 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc8a80b6880 0x7fc8a80b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.296+0000 7fc8c8dea740 1 -- 172.21.15.117:0/3209738955 shutdown_connections 2024-09-21T06:03:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.296+0000 7fc8c8dea740 1 --2- 172.21.15.117:0/3209738955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7ef0064d0 0x55a7ef3cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.296+0000 7fc8c8dea740 1 --2- 172.21.15.117:0/3209738955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a7eeffe2f0 0x55a7ef3ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.296+0000 7fc8c8dea740 1 --2- 172.21.15.117:0/3209738955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7ef0271b0 0x55a7ef3cc9b0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.296+0000 7fc8c8dea740 1 -- 172.21.15.117:0/3209738955 >> 172.21.15.117:0/3209738955 conn(0x55a7ef01b300 msgr2=0x55a7ef3ca8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.297+0000 7fc8c8dea740 1 -- 172.21.15.117:0/3209738955 shutdown_connections 2024-09-21T06:03:01.304 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.297+0000 7fc8bb7fe640 1 --2- 172.21.15.117:0/702946754 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc8a80b6880 0x7fc8a80b8d90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55a7ef3cb630 tx=0x7fc8ac00b2e0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:01.304 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.297+0000 7fc8c8dea740 1 -- 172.21.15.117:0/3209738955 wait complete. 2024-09-21T06:03:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.297+0000 7fc8c8dea740 1 -- start start 2024-09-21T06:03:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.298+0000 7fc8c8dea740 1 Processor -- start 2024-09-21T06:03:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.298+0000 7fc8c8dea740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7eeffe2f0 0x55a7ef3d80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.298+0000 7fc8c8dea740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7ef0064d0 0x55a7ef3d8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.298+0000 7fc8c8dea740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a7ef0271b0 0x55a7ef3e44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.298+0000 7fc8c8dea740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a7ef268ab0 con 0x55a7eeffe2f0 2024-09-21T06:03:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.298+0000 7fc8c8dea740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a7ef2d6730 con 0x55a7ef0064d0 2024-09-21T06:03:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.298+0000 7fc8c8dea740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a7ef3db180 con 0x55a7ef0271b0 2024-09-21T06:03:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.298+0000 7fc8bb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7eeffe2f0 0x55a7ef3d80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.298+0000 7fc8c8dea740 10 client.0 _ll_register_callbacks cb 0x55a7ef32a6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:03:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.299+0000 7fc8bb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7eeffe2f0 0x55a7ef3d80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50594/0 (socket says 172.21.15.117:50594) 2024-09-21T06:03:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.299+0000 7fc8bb7fe640 1 -- 172.21.15.117:0/4269000549 learned_addr learned my addr 172.21.15.117:0/4269000549 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.299+0000 7fc8bbfff640 1 --2- 172.21.15.117:0/4269000549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a7ef0271b0 0x55a7ef3e44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.299+0000 7fc8bb7fe640 1 -- 172.21.15.117:0/4269000549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a7ef0271b0 msgr2=0x55a7ef3e44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.299+0000 7fc8bb7fe640 1 --2- 172.21.15.117:0/4269000549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a7ef0271b0 0x55a7ef3e44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.299+0000 7fc8bb7fe640 1 -- 172.21.15.117:0/4269000549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7ef0064d0 msgr2=0x55a7ef3d8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.299+0000 7fc8bb7fe640 1 --2- 172.21.15.117:0/4269000549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7ef0064d0 0x55a7ef3d8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.299+0000 7fc8bb7fe640 1 -- 172.21.15.117:0/4269000549 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7ef092790 con 0x55a7eeffe2f0 2024-09-21T06:03:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.299+0000 7fc8bb7fe640 1 --2- 172.21.15.117:0/4269000549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7eeffe2f0 0x55a7ef3d80d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac005ce0 tx=0x7fc8ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f78c3c92532dc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.299+0000 7fc89affd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:03:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.300+0000 7fc89affd640 1 -- 172.21.15.117:0/4269000549 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac005220 con 0x55a7eeffe2f0 2024-09-21T06:03:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.300+0000 7fc89affd640 1 -- 172.21.15.117:0/4269000549 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fc8ac0053c0 con 0x55a7eeffe2f0 2024-09-21T06:03:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.301+0000 7fc8c8dea740 1 -- 172.21.15.117:0/4269000549 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8a40043c0 con 0x55a7eeffe2f0 2024-09-21T06:03:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.302+0000 7fc89affd640 1 -- 172.21.15.117:0/4269000549 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc8ac0055c0 con 0x55a7eeffe2f0 2024-09-21T06:03:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.302+0000 7fc8c8dea740 10 client.16446 fetch_fsmap learned FSMap version 27 2024-09-21T06:03:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.302+0000 7fc8c8dea740 1 -- 172.21.15.117:0/4269000549 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a7ef10b620 con 0x55a7eeffe2f0 2024-09-21T06:03:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.303+0000 7fc89affd640 1 -- 172.21.15.117:0/4269000549 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8ac004c50 con 0x55a7eeffe2f0 2024-09-21T06:03:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.303+0000 7fc89affd640 1 -- 172.21.15.117:0/4269000549 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc8ac01ea50 con 0x55a7eeffe2f0 2024-09-21T06:03:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.303+0000 7fc8c8dea740 10 client.16446 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:03:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.304+0000 7fc8c8dea740 10 client.16446 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:03:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.304+0000 7fc8c8dea740 20 client.16446 populate_metadata read hostname 'smithi117' 2024-09-21T06:03:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.304+0000 7fc8c8dea740 1 --2- 172.21.15.117:0/4269000549 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55a7eefe5d80 0x55a7ef3ea800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.304+0000 7fc8c8dea740 4 client.16446 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:03:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.304+0000 7fc8c8dea740 1 -- 172.21.15.117:0/4269000549 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc8a40043c0 con 0x55a7eefe5d80 2024-09-21T06:03:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.305+0000 7fc8b3fff640 1 --2- 172.21.15.117:0/4269000549 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55a7eefe5d80 0x55a7ef3ea800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.307+0000 7fc8b3fff640 1 --2- 172.21.15.117:0/4269000549 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55a7eefe5d80 0x55a7ef3ea800 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ab59b3046ea2193 server_cookie=29bd5ba476a2592a in_seq=0 out_seq=0 2024-09-21T06:03:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.307+0000 7fc89affd640 10 client.16446 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:03:01.319 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.311+0000 7fc89affd640 1 -- 172.21.15.117:0/4269000549 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fc8a40043c0 con 0x55a7eefe5d80 2024-09-21T06:03:01.319 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.311+0000 7fc89affd640 10 client.16446 handle_command_reply: tid=0 2024-09-21T06:03:01.319 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.311+0000 7fc8c8dea740 1 client.16446 shutdown 2024-09-21T06:03:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.311+0000 7fc8c8dea740 1 -- 172.21.15.117:0/4269000549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7eeffe2f0 msgr2=0x55a7ef3d80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.311+0000 7fc8c8dea740 1 --2- 172.21.15.117:0/4269000549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7eeffe2f0 0x55a7ef3d80d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac005ce0 tx=0x7fc8ac004730 comp rx=0 tx=0).stop 2024-09-21T06:03:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.311+0000 7fc8c8dea740 1 -- 172.21.15.117:0/4269000549 shutdown_connections 2024-09-21T06:03:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.311+0000 7fc8c8dea740 1 --2- 172.21.15.117:0/4269000549 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55a7eefe5d80 0x55a7ef3ea800 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.311+0000 7fc8c8dea740 1 --2- 172.21.15.117:0/4269000549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a7ef0271b0 0x55a7ef3e44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.311+0000 7fc89affd640 0 client.16446 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:03:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.311+0000 7fc8c8dea740 1 --2- 172.21.15.117:0/4269000549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7ef0064d0 0x55a7ef3d8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.311+0000 7fc8c8dea740 1 --2- 172.21.15.117:0/4269000549 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7eeffe2f0 0x55a7ef3d80d0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:01.322 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.311+0000 7fc8c8dea740 1 -- 172.21.15.117:0/4269000549 >> 172.21.15.117:0/4269000549 conn(0x55a7ef01b300 msgr2=0x55a7ef3cd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:01.322 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.312+0000 7fc8c8dea740 1 -- 172.21.15.117:0/4269000549 shutdown_connections 2024-09-21T06:03:01.322 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.312+0000 7fc8c8dea740 1 -- 172.21.15.117:0/4269000549 wait complete. 2024-09-21T06:03:01.322 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.312+0000 7fc8c8dea740 20 client.16446 trim_cache size 0 max 16384 2024-09-21T06:03:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.335+0000 7fc8c8dea740 1 Processor -- start 2024-09-21T06:03:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.335+0000 7fc8c8dea740 1 -- start start 2024-09-21T06:03:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.335+0000 7fc8c8dea740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a7eeffe2f0 0x55a7ef3d9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:01.340 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.336+0000 7fc8c8dea740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7ef0064d0 0x55a7ef3e43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:01.340 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.336+0000 7fc8c8dea740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7ef0271b0 0x55a7ef3c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:01.340 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.336+0000 7fc8bb7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a7eeffe2f0 0x55a7ef3d9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:01.340 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.336+0000 7fc8bb7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a7eeffe2f0 0x55a7ef3d9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:54144/0 (socket says 172.21.15.117:54144) 2024-09-21T06:03:01.341 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.336+0000 7fc8bb7fe640 1 -- 172.21.15.117:0/2696160236 learned_addr learned my addr 172.21.15.117:0/2696160236 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:01.341 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.337+0000 7fc8c8dea740 1 -- 172.21.15.117:0/2696160236 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc8bc1be3e0 con 0x55a7ef0271b0 2024-09-21T06:03:01.341 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.337+0000 7fc8c8dea740 1 -- 172.21.15.117:0/2696160236 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc8bc1bff20 con 0x55a7ef0064d0 2024-09-21T06:03:01.341 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.337+0000 7fc8c8dea740 1 -- 172.21.15.117:0/2696160236 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc8bc1beab0 con 0x55a7eeffe2f0 2024-09-21T06:03:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.339+0000 7fc8bb7fe640 1 -- 172.21.15.117:0/2696160236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7ef0064d0 msgr2=0x55a7ef3e43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.339+0000 7fc8bb7fe640 1 --2- 172.21.15.117:0/2696160236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7ef0064d0 0x55a7ef3e43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.340+0000 7fc8bb7fe640 1 -- 172.21.15.117:0/2696160236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7ef0271b0 msgr2=0x55a7ef3c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.340+0000 7fc8bb7fe640 1 --2- 172.21.15.117:0/2696160236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7ef0271b0 0x55a7ef3c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.340+0000 7fc8bb7fe640 1 -- 172.21.15.117:0/2696160236 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7ef3d65b0 con 0x55a7eeffe2f0 2024-09-21T06:03:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.340+0000 7fc8bb7fe640 1 --2- 172.21.15.117:0/2696160236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a7eeffe2f0 0x55a7ef3d9d00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac01eea0 tx=0x7fc8ac003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a30b85d206d9075 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.341+0000 7fc8b17fa640 1 -- 172.21.15.117:0/2696160236 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac005220 con 0x55a7eeffe2f0 2024-09-21T06:03:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.342+0000 7fc8b17fa640 1 -- 172.21.15.117:0/2696160236 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8ac036040 con 0x55a7eeffe2f0 2024-09-21T06:03:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.342+0000 7fc8b17fa640 1 -- 172.21.15.117:0/2696160236 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac004d10 con 0x55a7eeffe2f0 2024-09-21T06:03:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.344+0000 7fc8c8dea740 1 -- 172.21.15.117:0/2696160236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a7eeffe2f0 msgr2=0x55a7ef3d9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.344+0000 7fc8c8dea740 1 --2- 172.21.15.117:0/2696160236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a7eeffe2f0 0x55a7ef3d9d00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac01eea0 tx=0x7fc8ac003cd0 comp rx=0 tx=0).stop 2024-09-21T06:03:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.345+0000 7fc8c8dea740 1 -- 172.21.15.117:0/2696160236 shutdown_connections 2024-09-21T06:03:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.345+0000 7fc8c8dea740 1 --2- 172.21.15.117:0/2696160236 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7ef0271b0 0x55a7ef3c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.345+0000 7fc8c8dea740 1 --2- 172.21.15.117:0/2696160236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7ef0064d0 0x55a7ef3e43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.345+0000 7fc8c8dea740 1 --2- 172.21.15.117:0/2696160236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a7eeffe2f0 0x55a7ef3d9d00 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.345+0000 7fc8c8dea740 1 -- 172.21.15.117:0/2696160236 >> 172.21.15.117:0/2696160236 conn(0x55a7ef01b300 msgr2=0x55a7ef3d95f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.345+0000 7fc8c8dea740 1 -- 172.21.15.117:0/2696160236 shutdown_connections 2024-09-21T06:03:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.345+0000 7fc8c8dea740 1 -- 172.21.15.117:0/2696160236 wait complete. 2024-09-21T06:03:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.346+0000 7fc8c8dea740 1 -- start start 2024-09-21T06:03:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.347+0000 7fc8c8dea740 1 Processor -- start 2024-09-21T06:03:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.347+0000 7fc8c8dea740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7eeffe2f0 0x55a7ef3db5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.347+0000 7fc8c8dea740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7ef0064d0 0x55a7ef3dbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.347+0000 7fc8c8dea740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a7ef0271b0 0x55a7ef3dc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.347+0000 7fc8c8dea740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc8bc1bf850 con 0x55a7ef0064d0 2024-09-21T06:03:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.347+0000 7fc8c8dea740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc8bc1bf180 con 0x55a7eeffe2f0 2024-09-21T06:03:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.347+0000 7fc8c8dea740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc8bc1bc3c0 con 0x55a7ef0271b0 2024-09-21T06:03:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.347+0000 7fc8c8dea740 10 client.0 _ll_register_callbacks cb 0x7fc8ac00b4e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:03:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.349+0000 7fc8b3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7ef0064d0 0x55a7ef3dbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.349+0000 7fc8b3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7ef0064d0 0x55a7ef3dbb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50608/0 (socket says 172.21.15.117:50608) 2024-09-21T06:03:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.349+0000 7fc8b3fff640 1 -- 172.21.15.117:0/1647923652 learned_addr learned my addr 172.21.15.117:0/1647923652 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.349+0000 7fc8bbfff640 1 --2- 172.21.15.117:0/1647923652 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a7ef0271b0 0x55a7ef3dc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.350+0000 7fc8b3fff640 1 -- 172.21.15.117:0/1647923652 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a7ef0271b0 msgr2=0x55a7ef3dc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.350+0000 7fc8b3fff640 1 --2- 172.21.15.117:0/1647923652 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a7ef0271b0 0x55a7ef3dc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.350+0000 7fc8b3fff640 1 -- 172.21.15.117:0/1647923652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7eeffe2f0 msgr2=0x55a7ef3db5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.350+0000 7fc8b3fff640 1 --2- 172.21.15.117:0/1647923652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7eeffe2f0 0x55a7ef3db5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.350+0000 7fc8b3fff640 1 -- 172.21.15.117:0/1647923652 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7ef4533b0 con 0x55a7ef0064d0 2024-09-21T06:03:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.351+0000 7fc8b3fff640 1 --2- 172.21.15.117:0/1647923652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7ef0064d0 0x55a7ef3dbb20 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc8a400ea90 tx=0x7fc8a400e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8496b149f04a70f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:01.358 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.352+0000 7fc89affd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:03:01.358 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.352+0000 7fc89affd640 1 -- 172.21.15.117:0/1647923652 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4013230 con 0x55a7ef0064d0 2024-09-21T06:03:01.358 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.352+0000 7fc8c8dea740 1 -- 172.21.15.117:0/1647923652 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8ac036040 con 0x55a7ef0064d0 2024-09-21T06:03:01.358 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.352+0000 7fc89affd640 1 -- 172.21.15.117:0/1647923652 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fc8a400a860 con 0x55a7ef0064d0 2024-09-21T06:03:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.353+0000 7fc89affd640 1 -- 172.21.15.117:0/1647923652 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc8a4005bd0 con 0x55a7ef0064d0 2024-09-21T06:03:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.353+0000 7fc8c8dea740 10 client.16449 fetch_fsmap learned FSMap version 27 2024-09-21T06:03:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.353+0000 7fc8c8dea740 1 -- 172.21.15.117:0/1647923652 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a7ef454620 con 0x55a7ef0064d0 2024-09-21T06:03:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.354+0000 7fc89affd640 1 -- 172.21.15.117:0/1647923652 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8ac036040 con 0x55a7ef0064d0 2024-09-21T06:03:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.355+0000 7fc89affd640 1 -- 172.21.15.117:0/1647923652 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc8a400f040 con 0x55a7ef0064d0 2024-09-21T06:03:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.355+0000 7fc8c8dea740 10 client.16449 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:03:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.355+0000 7fc8c8dea740 10 client.16449 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:03:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.355+0000 7fc8c8dea740 20 client.16449 populate_metadata read hostname 'smithi117' 2024-09-21T06:03:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.355+0000 7fc8c8dea740 1 --2- 172.21.15.117:0/1647923652 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55a7eefbb1c0 0x55a7ef456a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.355+0000 7fc8c8dea740 4 client.16449 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:03:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.355+0000 7fc8c8dea740 1 -- 172.21.15.117:0/1647923652 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a7ef343640 con 0x55a7eefbb1c0 2024-09-21T06:03:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.357+0000 7fc8bb7fe640 1 --2- 172.21.15.117:0/1647923652 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55a7eefbb1c0 0x55a7ef456a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.359+0000 7fc89affd640 10 client.16449 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:03:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:01.359+0000 7fc8bb7fe640 1 --2- 172.21.15.117:0/1647923652 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55a7eefbb1c0 0x55a7ef456a00 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a3e849433eabd69 server_cookie=cdee4b1577511870 in_seq=0 out_seq=0 2024-09-21T06:03:02.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:01 smithi138 ceph-mon[34391]: pgmap v498: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 832 MiB/s wr, 1.36k op/s 2024-09-21T06:03:02.211 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:01 smithi117 ceph-mon[30947]: pgmap v498: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 832 MiB/s wr, 1.36k op/s 2024-09-21T06:03:02.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:01 smithi192 ceph-mon[34700]: pgmap v498: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 832 MiB/s wr, 1.36k op/s 2024-09-21T06:03:03.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.037+0000 7f57979f5640 1 -- 172.21.15.117:0/348047085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5790102a60 msgr2=0x7f5790102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:03.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.037+0000 7f57979f5640 1 --2- 172.21.15.117:0/348047085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5790102a60 0x7f5790102e70 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f578c009900 tx=0x7f578c02a410 comp rx=0 tx=0).stop 2024-09-21T06:03:03.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.038+0000 7f57979f5640 1 -- 172.21.15.117:0/348047085 shutdown_connections 2024-09-21T06:03:03.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.038+0000 7f57979f5640 1 --2- 172.21.15.117:0/348047085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5790108cf0 0x7f579010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:03.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.038+0000 7f57979f5640 1 --2- 172.21.15.117:0/348047085 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f57901083a0 0x7f57901087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:03.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.038+0000 7f57979f5640 1 --2- 172.21.15.117:0/348047085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5790102a60 0x7f5790102e70 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:03.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.038+0000 7f57979f5640 1 -- 172.21.15.117:0/348047085 >> 172.21.15.117:0/348047085 conn(0x7f5790079730 msgr2=0x7f5790079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:03.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.038+0000 7f57979f5640 1 -- 172.21.15.117:0/348047085 shutdown_connections 2024-09-21T06:03:03.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.038+0000 7f57979f5640 1 -- 172.21.15.117:0/348047085 wait complete. 2024-09-21T06:03:03.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.039+0000 7f57979f5640 1 Processor -- start 2024-09-21T06:03:03.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.039+0000 7f57979f5640 1 -- start start 2024-09-21T06:03:03.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.039+0000 7f57979f5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5790102a60 0x7f57901135d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:03.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.039+0000 7f57979f5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57901083a0 0x7f5790113b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:03.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.039+0000 7f57979f5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5790108cf0 0x7f579011dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:03.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.039+0000 7f57979f5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f57901118a0 con 0x7f5790102a60 2024-09-21T06:03:03.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.039+0000 7f57979f5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5790111720 con 0x7f57901083a0 2024-09-21T06:03:03.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.039+0000 7f57979f5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5790111a20 con 0x7f5790108cf0 2024-09-21T06:03:03.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.039+0000 7f5795791640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5790102a60 0x7f57901135d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:03.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.040+0000 7f5795791640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5790102a60 0x7f57901135d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50626/0 (socket says 172.21.15.117:50626) 2024-09-21T06:03:03.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.040+0000 7f5795791640 1 -- 172.21.15.117:0/476725491 learned_addr learned my addr 172.21.15.117:0/476725491 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:03.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.040+0000 7f5795791640 1 -- 172.21.15.117:0/476725491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5790108cf0 msgr2=0x7f579011dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:03.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.040+0000 7f5795791640 1 --2- 172.21.15.117:0/476725491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5790108cf0 0x7f579011dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:03.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.040+0000 7f5795791640 1 -- 172.21.15.117:0/476725491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57901083a0 msgr2=0x7f5790113b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:03.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.040+0000 7f5795791640 1 --2- 172.21.15.117:0/476725491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57901083a0 0x7f5790113b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:03.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.040+0000 7f5795791640 1 -- 172.21.15.117:0/476725491 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f579011e6d0 con 0x7f5790102a60 2024-09-21T06:03:03.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.041+0000 7f5795791640 1 --2- 172.21.15.117:0/476725491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5790102a60 0x7f57901135d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f578c005ab0 tx=0x7f578c003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:03.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.041+0000 7f57867fc640 1 -- 172.21.15.117:0/476725491 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578c03f070 con 0x7f5790102a60 2024-09-21T06:03:03.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.041+0000 7f57867fc640 1 -- 172.21.15.117:0/476725491 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f578c031d60 con 0x7f5790102a60 2024-09-21T06:03:03.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.041+0000 7f57867fc640 1 -- 172.21.15.117:0/476725491 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578c035030 con 0x7f5790102a60 2024-09-21T06:03:03.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.042+0000 7f57979f5640 1 -- 172.21.15.117:0/476725491 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f579011e900 con 0x7f5790102a60 2024-09-21T06:03:03.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.042+0000 7f57979f5640 1 -- 172.21.15.117:0/476725491 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5790118fe0 con 0x7f5790102a60 2024-09-21T06:03:03.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.047+0000 7f579df69740 1 Processor -- start 2024-09-21T06:03:03.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.047+0000 7f579df69740 1 -- start start 2024-09-21T06:03:03.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.051+0000 7f579df69740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5281350 0x5618d5628430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:03.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.051+0000 7f579df69740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d5258490 0x5618d5628970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:03.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.051+0000 7f579df69740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d5260670 0x5618d5628eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:03.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.051+0000 7f579df69740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5618d559ee60 con 0x5618d5260670 2024-09-21T06:03:03.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.051+0000 7f579df69740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5618d558e290 con 0x5618d5258490 2024-09-21T06:03:03.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.051+0000 7f579df69740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5618d55920a0 con 0x5618d5281350 2024-09-21T06:03:03.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.051+0000 7f5794f90640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d5260670 0x5618d5628eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:03.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.051+0000 7f5794f90640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d5260670 0x5618d5628eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50636/0 (socket says 172.21.15.117:50636) 2024-09-21T06:03:03.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.051+0000 7f5794f90640 1 -- 172.21.15.117:0/1181476139 learned_addr learned my addr 172.21.15.117:0/1181476139 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:03.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.051+0000 7f57867fc640 1 -- 172.21.15.117:0/476725491 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f578c042070 con 0x7f5790102a60 2024-09-21T06:03:03.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.052+0000 7f5795f92640 1 --2- 172.21.15.117:0/1181476139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5281350 0x5618d5628430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:03.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.052+0000 7f5794f90640 1 -- 172.21.15.117:0/1181476139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5281350 msgr2=0x5618d5628430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:03.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.052+0000 7f5794f90640 1 --2- 172.21.15.117:0/1181476139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5281350 0x5618d5628430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:03.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.052+0000 7f5794f90640 1 -- 172.21.15.117:0/1181476139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d5258490 msgr2=0x5618d5628970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:03:03.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.052+0000 7f5794f90640 1 --2- 172.21.15.117:0/1181476139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d5258490 0x5618d5628970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:03.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.052+0000 7f5794f90640 1 -- 172.21.15.117:0/1181476139 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618d52ec930 con 0x5618d5260670 2024-09-21T06:03:03.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.053+0000 7f57867fc640 1 --2- 172.21.15.117:0/476725491 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f577c0b67d0 0x7f577c0b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:03.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.054+0000 7f57867fc640 1 -- 172.21.15.117:0/476725491 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f578c108130 con 0x7f5790102a60 2024-09-21T06:03:03.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.055+0000 7f5794f90640 1 --2- 172.21.15.117:0/1181476139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d5260670 0x5618d5628eb0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f578000baa0 tx=0x7f578000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bec48853b6a07db server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:03.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.056+0000 7f577b7fe640 1 -- 172.21.15.117:0/1181476139 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5780016070 con 0x5618d5260670 2024-09-21T06:03:03.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.056+0000 7f577b7fe640 1 -- 172.21.15.117:0/1181476139 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57800054d0 con 0x5618d5260670 2024-09-21T06:03:03.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.057+0000 7f579df69740 1 -- 172.21.15.117:0/1181476139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d5260670 msgr2=0x5618d5628eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:03.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.057+0000 7f577b7fe640 1 -- 172.21.15.117:0/1181476139 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5780016070 con 0x5618d5260670 2024-09-21T06:03:03.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.057+0000 7f579df69740 1 --2- 172.21.15.117:0/1181476139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d5260670 0x5618d5628eb0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f578000baa0 tx=0x7f578000bf60 comp rx=0 tx=0).stop 2024-09-21T06:03:03.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.058+0000 7f5795f92640 1 --2- 172.21.15.117:0/476725491 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f577c0b67d0 0x7f577c0b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:03.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.060+0000 7f579df69740 1 -- 172.21.15.117:0/1181476139 shutdown_connections 2024-09-21T06:03:03.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.060+0000 7f579df69740 1 --2- 172.21.15.117:0/1181476139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d5260670 0x5618d5628eb0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:03.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.060+0000 7f579df69740 1 --2- 172.21.15.117:0/1181476139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d5258490 0x5618d5628970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:03.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.060+0000 7f579df69740 1 --2- 172.21.15.117:0/1181476139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5281350 0x5618d5628430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:03.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.060+0000 7f579df69740 1 -- 172.21.15.117:0/1181476139 >> 172.21.15.117:0/1181476139 conn(0x5618d5275740 msgr2=0x5618d5626320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:03.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.060+0000 7f579df69740 1 -- 172.21.15.117:0/1181476139 shutdown_connections 2024-09-21T06:03:03.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.061+0000 7f5795f92640 1 --2- 172.21.15.117:0/476725491 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f577c0b67d0 0x7f577c0b8ce0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f57880092f0 tx=0x7f5788009280 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:03.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.061+0000 7f579df69740 1 -- 172.21.15.117:0/1181476139 wait complete. 2024-09-21T06:03:03.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.062+0000 7f579df69740 1 -- start start 2024-09-21T06:03:03.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.062+0000 7f579df69740 1 Processor -- start 2024-09-21T06:03:03.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.063+0000 7f579df69740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d5258490 0x5618d5633f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:03.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.063+0000 7f579df69740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5260670 0x5618d5634470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:03.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.063+0000 7f579df69740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d5281350 0x5618d563ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:03.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.063+0000 7f579df69740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5618d53ef890 con 0x5618d5281350 2024-09-21T06:03:03.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.063+0000 7f579df69740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5618d5528d40 con 0x5618d5258490 2024-09-21T06:03:03.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.064+0000 7f579df69740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5618d5636c60 con 0x5618d5260670 2024-09-21T06:03:03.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.064+0000 7f579df69740 10 client.0 _ll_register_callbacks cb 0x5618d558cc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:03:03.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.064+0000 7f5795f92640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d5281350 0x5618d563ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:03.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.064+0000 7f5795f92640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d5281350 0x5618d563ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50638/0 (socket says 172.21.15.117:50638) 2024-09-21T06:03:03.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.064+0000 7f5795f92640 1 -- 172.21.15.117:0/876222509 learned_addr learned my addr 172.21.15.117:0/876222509 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:03.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.064+0000 7f5794f90640 1 --2- 172.21.15.117:0/876222509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5260670 0x5618d5634470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:03.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.065+0000 7f5795f92640 1 -- 172.21.15.117:0/876222509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5260670 msgr2=0x5618d5634470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:03.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.065+0000 7f5795f92640 1 --2- 172.21.15.117:0/876222509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5260670 0x5618d5634470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:03.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.065+0000 7f5795f92640 1 -- 172.21.15.117:0/876222509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d5258490 msgr2=0x5618d5633f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:03.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.065+0000 7f5795f92640 1 --2- 172.21.15.117:0/876222509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d5258490 0x5618d5633f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:03.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.065+0000 7f5795f92640 1 -- 172.21.15.117:0/876222509 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618d56404e0 con 0x5618d5281350 2024-09-21T06:03:03.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.065+0000 7f5795f92640 1 --2- 172.21.15.117:0/876222509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d5281350 0x5618d563ffa0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f57880116f0 tx=0x7f5788005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2af4279d5b069be server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:03.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.065+0000 7f5778ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:03:03.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.070+0000 7f5778ff9640 1 -- 172.21.15.117:0/876222509 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5788022070 con 0x5618d5281350 2024-09-21T06:03:03.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.070+0000 7f5778ff9640 1 -- 172.21.15.117:0/876222509 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f5788004d20 con 0x5618d5281350 2024-09-21T06:03:03.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.071+0000 7f579df69740 1 -- 172.21.15.117:0/876222509 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57800054d0 con 0x5618d5281350 2024-09-21T06:03:03.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.071+0000 7f5778ff9640 1 -- 172.21.15.117:0/876222509 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f578802b050 con 0x5618d5281350 2024-09-21T06:03:03.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.071+0000 7f579df69740 10 client.16461 fetch_fsmap learned FSMap version 27 2024-09-21T06:03:03.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.071+0000 7f579df69740 1 -- 172.21.15.117:0/876222509 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5618d5642e90 con 0x5618d5281350 2024-09-21T06:03:03.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.072+0000 7f5778ff9640 1 -- 172.21.15.117:0/876222509 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57800054d0 con 0x5618d5281350 2024-09-21T06:03:03.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.072+0000 7f5778ff9640 1 -- 172.21.15.117:0/876222509 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f578801e960 con 0x5618d5281350 2024-09-21T06:03:03.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.075+0000 7f579df69740 10 client.16461 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:03:03.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.075+0000 7f579df69740 10 client.16461 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:03:03.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.075+0000 7f579df69740 20 client.16461 populate_metadata read hostname 'smithi117' 2024-09-21T06:03:03.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.075+0000 7f579df69740 1 --2- 172.21.15.117:0/876222509 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5618d523fcd0 0x5618d5646410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:03.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.075+0000 7f579df69740 4 client.16461 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:03:03.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.075+0000 7f579df69740 1 -- 172.21.15.117:0/876222509 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618d55a0db0 con 0x5618d523fcd0 2024-09-21T06:03:03.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.079+0000 7f5795791640 1 --2- 172.21.15.117:0/876222509 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5618d523fcd0 0x5618d5646410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:03.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.086+0000 7f5795791640 1 --2- 172.21.15.117:0/876222509 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5618d523fcd0 0x5618d5646410 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c187c877ecd8700b server_cookie=987e3879ab65293b in_seq=0 out_seq=0 2024-09-21T06:03:03.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.087+0000 7f5778ff9640 10 client.16461 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:03:03.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.099+0000 7f5778ff9640 1 -- 172.21.15.117:0/876222509 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5618d55a0db0 con 0x5618d523fcd0 2024-09-21T06:03:03.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.099+0000 7f5778ff9640 10 client.16461 handle_command_reply: tid=0 2024-09-21T06:03:03.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.099+0000 7f579df69740 1 client.16461 shutdown 2024-09-21T06:03:03.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.099+0000 7f579df69740 1 -- 172.21.15.117:0/876222509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d5281350 msgr2=0x5618d563ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:03.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.099+0000 7f579df69740 1 --2- 172.21.15.117:0/876222509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d5281350 0x5618d563ffa0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f57880116f0 tx=0x7f5788005bd0 comp rx=0 tx=0).stop 2024-09-21T06:03:03.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.099+0000 7f579df69740 1 -- 172.21.15.117:0/876222509 shutdown_connections 2024-09-21T06:03:03.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.099+0000 7f579df69740 1 --2- 172.21.15.117:0/876222509 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5618d523fcd0 0x5618d5646410 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:03.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.099+0000 7f5778ff9640 0 client.16461 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:03:03.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.099+0000 7f579df69740 1 --2- 172.21.15.117:0/876222509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d5281350 0x5618d563ffa0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:03.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.099+0000 7f579df69740 1 --2- 172.21.15.117:0/876222509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5260670 0x5618d5634470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:03.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.099+0000 7f579df69740 1 --2- 172.21.15.117:0/876222509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d5258490 0x5618d5633f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:03.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.099+0000 7f579df69740 1 -- 172.21.15.117:0/876222509 >> 172.21.15.117:0/876222509 conn(0x5618d5275740 msgr2=0x5618d562b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:03.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.101+0000 7f579df69740 1 -- 172.21.15.117:0/876222509 shutdown_connections 2024-09-21T06:03:03.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.101+0000 7f579df69740 1 -- 172.21.15.117:0/876222509 wait complete. 2024-09-21T06:03:03.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.102+0000 7f579df69740 20 client.16461 trim_cache size 0 max 16384 2024-09-21T06:03:03.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.145+0000 7f579df69740 1 Processor -- start 2024-09-21T06:03:03.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.145+0000 7f579df69740 1 -- start start 2024-09-21T06:03:03.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.146+0000 7f579df69740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d523fcd0 0x5618d563fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:03.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.147+0000 7f579df69740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5258490 0x5618d5630f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:03.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.147+0000 7f579df69740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d5260670 0x5618d5631460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:03.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.147+0000 7f579df69740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f57901be460 con 0x5618d5260670 2024-09-21T06:03:03.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.147+0000 7f579df69740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f57901bffa0 con 0x5618d523fcd0 2024-09-21T06:03:03.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.147+0000 7f579df69740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f57901beb30 con 0x5618d5258490 2024-09-21T06:03:03.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.147+0000 7f5795f92640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d5260670 0x5618d5631460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:03.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.147+0000 7f5795f92640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d5260670 0x5618d5631460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50644/0 (socket says 172.21.15.117:50644) 2024-09-21T06:03:03.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.148+0000 7f5795f92640 1 -- 172.21.15.117:0/3016390355 learned_addr learned my addr 172.21.15.117:0/3016390355 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:03.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.149+0000 7f5795f92640 1 -- 172.21.15.117:0/3016390355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5258490 msgr2=0x5618d5630f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:03:03.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.149+0000 7f5795f92640 1 --2- 172.21.15.117:0/3016390355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5258490 0x5618d5630f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:03.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.149+0000 7f5795f92640 1 -- 172.21.15.117:0/3016390355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d523fcd0 msgr2=0x5618d563fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:03.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.149+0000 7f5795f92640 1 --2- 172.21.15.117:0/3016390355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d523fcd0 0x5618d563fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:03.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.149+0000 7f5795f92640 1 -- 172.21.15.117:0/3016390355 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618d562f5c0 con 0x5618d5260670 2024-09-21T06:03:03.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.151+0000 7f5795f92640 1 --2- 172.21.15.117:0/3016390355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d5260670 0x5618d5631460 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f5788011f80 tx=0x7f5788009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b3de32fcb25fbf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:03.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.152+0000 7f577affd640 1 -- 172.21.15.117:0/3016390355 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5788022070 con 0x5618d5260670 2024-09-21T06:03:03.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.152+0000 7f577affd640 1 -- 172.21.15.117:0/3016390355 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5788026cc0 con 0x5618d5260670 2024-09-21T06:03:03.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.152+0000 7f577affd640 1 -- 172.21.15.117:0/3016390355 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578801eaa0 con 0x5618d5260670 2024-09-21T06:03:03.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.153+0000 7f579df69740 1 -- 172.21.15.117:0/3016390355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d5260670 msgr2=0x5618d5631460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:03.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.153+0000 7f579df69740 1 --2- 172.21.15.117:0/3016390355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d5260670 0x5618d5631460 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f5788011f80 tx=0x7f5788009f80 comp rx=0 tx=0).stop 2024-09-21T06:03:03.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.154+0000 7f579df69740 1 -- 172.21.15.117:0/3016390355 shutdown_connections 2024-09-21T06:03:03.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.154+0000 7f579df69740 1 --2- 172.21.15.117:0/3016390355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d5260670 0x5618d5631460 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:03.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.154+0000 7f579df69740 1 --2- 172.21.15.117:0/3016390355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5258490 0x5618d5630f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:03.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.154+0000 7f579df69740 1 --2- 172.21.15.117:0/3016390355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d523fcd0 0x5618d563fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:03.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.154+0000 7f579df69740 1 -- 172.21.15.117:0/3016390355 >> 172.21.15.117:0/3016390355 conn(0x5618d5275740 msgr2=0x5618d56325b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:03.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.154+0000 7f579df69740 1 -- 172.21.15.117:0/3016390355 shutdown_connections 2024-09-21T06:03:03.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.154+0000 7f579df69740 1 -- 172.21.15.117:0/3016390355 wait complete. 2024-09-21T06:03:03.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.155+0000 7f579df69740 1 -- start start 2024-09-21T06:03:03.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.155+0000 7f579df69740 1 Processor -- start 2024-09-21T06:03:03.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.156+0000 7f579df69740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d523fcd0 0x5618d563c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:03.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.156+0000 7f579df69740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5258490 0x5618d5638ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:03.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.156+0000 7f579df69740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d5260670 0x5618d56392d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:03.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.157+0000 7f579df69740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f57901bf8d0 con 0x5618d5260670 2024-09-21T06:03:03.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.157+0000 7f579df69740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f57901bf200 con 0x5618d523fcd0 2024-09-21T06:03:03.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.157+0000 7f579df69740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f57901bc440 con 0x5618d5258490 2024-09-21T06:03:03.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.157+0000 7f579df69740 10 client.0 _ll_register_callbacks cb 0x7f5788015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:03:03.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.158+0000 7f5795f92640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d5260670 0x5618d56392d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:03.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.158+0000 7f5795f92640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d5260670 0x5618d56392d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50654/0 (socket says 172.21.15.117:50654) 2024-09-21T06:03:03.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.158+0000 7f5795f92640 1 -- 172.21.15.117:0/1646107215 learned_addr learned my addr 172.21.15.117:0/1646107215 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:03.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.158+0000 7f5795f92640 1 -- 172.21.15.117:0/1646107215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5258490 msgr2=0x5618d5638ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:03:03.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.158+0000 7f5795f92640 1 --2- 172.21.15.117:0/1646107215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5258490 0x5618d5638ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:03.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.158+0000 7f5795f92640 1 -- 172.21.15.117:0/1646107215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d523fcd0 msgr2=0x5618d563c7e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:03:03.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.158+0000 7f5795f92640 1 --2- 172.21.15.117:0/1646107215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d523fcd0 0x5618d563c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:03.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.158+0000 7f5795f92640 1 -- 172.21.15.117:0/1646107215 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618d5639a60 con 0x5618d5260670 2024-09-21T06:03:03.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.159+0000 7f5795f92640 1 --2- 172.21.15.117:0/1646107215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d5260670 0x5618d56392d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f578800a010 tx=0x7f578801e850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7efee1bf236f88d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:03.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.159+0000 7f5778ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:03:03.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.160+0000 7f5778ff9640 1 -- 172.21.15.117:0/1646107215 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5788022040 con 0x5618d5260670 2024-09-21T06:03:03.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.160+0000 7f579df69740 1 -- 172.21.15.117:0/1646107215 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5788026cc0 con 0x5618d5260670 2024-09-21T06:03:03.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.160+0000 7f5778ff9640 1 -- 172.21.15.117:0/1646107215 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f57880108a0 con 0x5618d5260670 2024-09-21T06:03:03.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.161+0000 7f5778ff9640 1 -- 172.21.15.117:0/1646107215 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f578801e450 con 0x5618d5260670 2024-09-21T06:03:03.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.162+0000 7f579df69740 10 client.16467 fetch_fsmap learned FSMap version 27 2024-09-21T06:03:03.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.162+0000 7f579df69740 1 -- 172.21.15.117:0/1646107215 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5618d56b1e60 con 0x5618d5260670 2024-09-21T06:03:03.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.163+0000 7f5778ff9640 1 -- 172.21.15.117:0/1646107215 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5788039440 con 0x5618d5260670 2024-09-21T06:03:03.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.163+0000 7f5778ff9640 1 -- 172.21.15.117:0/1646107215 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5788040020 con 0x5618d5260670 2024-09-21T06:03:03.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.164+0000 7f579df69740 10 client.16467 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:03:03.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.164+0000 7f579df69740 10 client.16467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:03:03.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.164+0000 7f579df69740 20 client.16467 populate_metadata read hostname 'smithi117' 2024-09-21T06:03:03.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.164+0000 7f579df69740 1 --2- 172.21.15.117:0/1646107215 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5618d5281350 0x5618d56b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:03.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.164+0000 7f579df69740 4 client.16467 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:03:03.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.164+0000 7f579df69740 1 -- 172.21.15.117:0/1646107215 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f5788026cc0 con 0x5618d5281350 2024-09-21T06:03:03.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.165+0000 7f5795791640 1 --2- 172.21.15.117:0/1646107215 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5618d5281350 0x5618d56b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:03.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.171+0000 7f5778ff9640 10 client.16467 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:03:03.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.171+0000 7f5795791640 1 --2- 172.21.15.117:0/1646107215 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5618d5281350 0x5618d56b4430 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da07a3576ece6ded server_cookie=35e2311fd2864dd2 in_seq=0 out_seq=0 2024-09-21T06:03:03.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.779+0000 7f5778ff9640 1 -- 172.21.15.117:0/1646107215 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f5788026cc0 con 0x5618d5281350 2024-09-21T06:03:03.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.779+0000 7f5778ff9640 10 client.16467 handle_command_reply: tid=0 2024-09-21T06:03:03.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.779+0000 7f579df69740 1 client.16467 shutdown 2024-09-21T06:03:03.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.780+0000 7f579df69740 1 -- 172.21.15.117:0/1646107215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d5260670 msgr2=0x5618d56392d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:03.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.780+0000 7f579df69740 1 --2- 172.21.15.117:0/1646107215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d5260670 0x5618d56392d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f578800a010 tx=0x7f578801e850 comp rx=0 tx=0).stop 2024-09-21T06:03:03.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.780+0000 7f579df69740 1 -- 172.21.15.117:0/1646107215 shutdown_connections 2024-09-21T06:03:03.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.780+0000 7f579df69740 1 --2- 172.21.15.117:0/1646107215 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5618d5281350 0x5618d56b4430 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:03.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.780+0000 7f579df69740 1 --2- 172.21.15.117:0/1646107215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d5260670 0x5618d56392d0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:03.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.780+0000 7f579df69740 1 --2- 172.21.15.117:0/1646107215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d5258490 0x5618d5638ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:03.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.780+0000 7f579df69740 1 --2- 172.21.15.117:0/1646107215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d523fcd0 0x5618d563c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:03.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.780+0000 7f579df69740 1 -- 172.21.15.117:0/1646107215 >> 172.21.15.117:0/1646107215 conn(0x5618d5275740 msgr2=0x5618d5635180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:03.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.780+0000 7f5778ff9640 0 client.16467 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:03:03.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.780+0000 7f579df69740 1 -- 172.21.15.117:0/1646107215 shutdown_connections 2024-09-21T06:03:03.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.780+0000 7f579df69740 1 -- 172.21.15.117:0/1646107215 wait complete. 2024-09-21T06:03:03.784 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T06:03:03.784 INFO:teuthology.orchestra.run.smithi117.stdout: "return_code": 0, 2024-09-21T06:03:03.784 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_tag": "29ff9b7c-49d9-4910-85be-bace7e940cd5", 2024-09-21T06:03:03.784 INFO:teuthology.orchestra.run.smithi117.stdout: "mode": "asynchronous" 2024-09-21T06:03:03.784 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T06:03:03.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.780+0000 7f579df69740 20 client.16467 trim_cache size 0 max 16384 2024-09-21T06:03:03.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.782+0000 7f57979f5640 1 -- 172.21.15.117:0/476725491 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f577c0b67d0 msgr2=0x7f577c0b8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:03.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.782+0000 7f57979f5640 1 --2- 172.21.15.117:0/476725491 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f577c0b67d0 0x7f577c0b8ce0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f57880092f0 tx=0x7f5788009280 comp rx=0 tx=0).stop 2024-09-21T06:03:03.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.782+0000 7f57979f5640 1 -- 172.21.15.117:0/476725491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5790102a60 msgr2=0x7f57901135d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:03.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.782+0000 7f57979f5640 1 --2- 172.21.15.117:0/476725491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5790102a60 0x7f57901135d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f578c005ab0 tx=0x7f578c003cd0 comp rx=0 tx=0).stop 2024-09-21T06:03:03.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.782+0000 7f57979f5640 1 -- 172.21.15.117:0/476725491 shutdown_connections 2024-09-21T06:03:03.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.782+0000 7f57979f5640 1 --2- 172.21.15.117:0/476725491 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f577c0b67d0 0x7f577c0b8ce0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:03.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.782+0000 7f57979f5640 1 --2- 172.21.15.117:0/476725491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5790108cf0 0x7f579011dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:03.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.782+0000 7f57979f5640 1 --2- 172.21.15.117:0/476725491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57901083a0 0x7f5790113b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:03.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.782+0000 7f57979f5640 1 --2- 172.21.15.117:0/476725491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5790102a60 0x7f57901135d0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:03.786 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.782+0000 7f57979f5640 1 -- 172.21.15.117:0/476725491 >> 172.21.15.117:0/476725491 conn(0x7f5790079730 msgr2=0x7f57901013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:03.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.784+0000 7f57979f5640 1 -- 172.21.15.117:0/476725491 shutdown_connections 2024-09-21T06:03:03.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:03.785+0000 7f57979f5640 1 -- 172.21.15.117:0/476725491 wait complete. 2024-09-21T06:03:04.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:03 smithi192 ceph-mon[34700]: pgmap v499: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 916 MiB/s wr, 1.57k op/s 2024-09-21T06:03:04.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:03 smithi117 ceph-mon[30947]: pgmap v499: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 916 MiB/s wr, 1.57k op/s 2024-09-21T06:03:04.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:03 smithi138 ceph-mon[34391]: pgmap v499: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 916 MiB/s wr, 1.57k op/s 2024-09-21T06:03:05.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:04 smithi192 ceph-mon[34700]: scrub queued for path: / 2024-09-21T06:03:05.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:04 smithi192 ceph-mon[34700]: scrub summary: idle+waiting paths [/] 2024-09-21T06:03:05.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:04 smithi192 ceph-mon[34700]: scrub summary: active paths [/] 2024-09-21T06:03:05.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:04 smithi117 ceph-mon[30947]: scrub queued for path: / 2024-09-21T06:03:05.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:04 smithi117 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-21T06:03:05.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:04 smithi117 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-21T06:03:05.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:04 smithi138 ceph-mon[34391]: scrub queued for path: / 2024-09-21T06:03:05.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:04 smithi138 ceph-mon[34391]: scrub summary: idle+waiting paths [/] 2024-09-21T06:03:05.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:04 smithi138 ceph-mon[34391]: scrub summary: active paths [/] 2024-09-21T06:03:05.599 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:05.597+0000 7fc89affd640 1 -- 172.21.15.117:0/1647923652 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a7ef343640 con 0x55a7eefbb1c0 2024-09-21T06:03:05.599 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:05.597+0000 7fc89affd640 10 client.16449 handle_command_reply: tid=0 2024-09-21T06:03:05.599 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:05.597+0000 7fc8c8dea740 1 client.16449 shutdown 2024-09-21T06:03:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:05.597+0000 7fc8c8dea740 1 -- 172.21.15.117:0/1647923652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7ef0064d0 msgr2=0x55a7ef3dbb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:05.597+0000 7fc8c8dea740 1 --2- 172.21.15.117:0/1647923652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7ef0064d0 0x55a7ef3dbb20 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc8a400ea90 tx=0x7fc8a400e8a0 comp rx=0 tx=0).stop 2024-09-21T06:03:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:05.597+0000 7fc8c8dea740 1 -- 172.21.15.117:0/1647923652 shutdown_connections 2024-09-21T06:03:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:05.597+0000 7fc8c8dea740 1 --2- 172.21.15.117:0/1647923652 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55a7eefbb1c0 0x55a7ef456a00 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:05.597+0000 7fc8c8dea740 1 --2- 172.21.15.117:0/1647923652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7ef0064d0 0x55a7ef3dbb20 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:05.597+0000 7fc8c8dea740 1 --2- 172.21.15.117:0/1647923652 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a7ef0271b0 0x55a7ef3dc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:05.597+0000 7fc89affd640 0 client.16449 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:03:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:05.597+0000 7fc8c8dea740 1 --2- 172.21.15.117:0/1647923652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7eeffe2f0 0x55a7ef3db5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:05.597+0000 7fc8c8dea740 1 -- 172.21.15.117:0/1647923652 >> 172.21.15.117:0/1647923652 conn(0x55a7ef01b300 msgr2=0x55a7ef3d94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:05.597+0000 7fc8c8dea740 1 -- 172.21.15.117:0/1647923652 shutdown_connections 2024-09-21T06:03:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:05.597+0000 7fc8c8dea740 1 -- 172.21.15.117:0/1647923652 wait complete. 2024-09-21T06:03:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:05.597+0000 7fc8c8dea740 20 client.16449 trim_cache size 0 max 16384 2024-09-21T06:03:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T06:03:05.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T06:03:05.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T06:03:05.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T06:03:05.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:05.598+0000 7fc8c1e7e640 1 -- 172.21.15.117:0/702946754 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc8a80b6880 msgr2=0x7fc8a80b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:05.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:05.598+0000 7fc8c1e7e640 1 --2- 172.21.15.117:0/702946754 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc8a80b6880 0x7fc8a80b8d90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55a7ef3cb630 tx=0x7fc8ac00b2e0 comp rx=0 tx=0).stop 2024-09-21T06:03:05.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:05.598+0000 7fc8c1e7e640 1 -- 172.21.15.117:0/702946754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8bc110350 msgr2=0x7fc8bc1aea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:05.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:05.598+0000 7fc8c1e7e640 1 --2- 172.21.15.117:0/702946754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8bc110350 0x7fc8bc1aea90 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc8b402aeb0 tx=0x7fc8b40319f0 comp rx=0 tx=0).stop 2024-09-21T06:03:05.602 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:05.598+0000 7fc8c1e7e640 1 -- 172.21.15.117:0/702946754 shutdown_connections 2024-09-21T06:03:05.602 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:05.598+0000 7fc8c1e7e640 1 --2- 172.21.15.117:0/702946754 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc8a80b6880 0x7fc8a80b8d90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:05.602 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:05.598+0000 7fc8c1e7e640 1 --2- 172.21.15.117:0/702946754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8bc110350 0x7fc8bc1aea90 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:05.602 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:05.598+0000 7fc8c1e7e640 1 --2- 172.21.15.117:0/702946754 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8bc10fa00 0x7fc8bc1a9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:05.602 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:05.598+0000 7fc8c1e7e640 1 --2- 172.21.15.117:0/702946754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8bc10a660 0x7fc8bc1a9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:05.602 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:05.598+0000 7fc8c1e7e640 1 -- 172.21.15.117:0/702946754 >> 172.21.15.117:0/702946754 conn(0x7fc8bc070dd0 msgr2=0x7fc8bc073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:05.602 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:05.598+0000 7fc8c1e7e640 1 -- 172.21.15.117:0/702946754 shutdown_connections 2024-09-21T06:03:05.602 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:05.598+0000 7fc8c1e7e640 1 -- 172.21.15.117:0/702946754 wait complete. 2024-09-21T06:03:05.856 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:03:06.170 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:06.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:05 smithi192 ceph-mon[34700]: pgmap v500: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 939 MiB/s wr, 1.67k op/s 2024-09-21T06:03:06.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:05 smithi117 ceph-mon[30947]: pgmap v500: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 939 MiB/s wr, 1.67k op/s 2024-09-21T06:03:06.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:05 smithi138 ceph-mon[34391]: pgmap v500: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 939 MiB/s wr, 1.67k op/s 2024-09-21T06:03:06.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.826+0000 7ff4239fa640 1 -- 172.21.15.117:0/1222541531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff41c108cf0 msgr2=0x7ff41c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:06.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.826+0000 7ff4239fa640 1 --2- 172.21.15.117:0/1222541531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff41c108cf0 0x7ff41c10f520 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff414009ca0 tx=0x7ff41402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:03:06.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.826+0000 7ff4239fa640 1 -- 172.21.15.117:0/1222541531 shutdown_connections 2024-09-21T06:03:06.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.826+0000 7ff4239fa640 1 --2- 172.21.15.117:0/1222541531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff41c108cf0 0x7ff41c10f520 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:06.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.826+0000 7ff4239fa640 1 --2- 172.21.15.117:0/1222541531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff41c1083a0 0x7ff41c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:06.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.826+0000 7ff4239fa640 1 --2- 172.21.15.117:0/1222541531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff41c102ab0 0x7ff41c102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:06.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.826+0000 7ff4239fa640 1 -- 172.21.15.117:0/1222541531 >> 172.21.15.117:0/1222541531 conn(0x7ff41c079680 msgr2=0x7ff41c079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:07.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.826+0000 7ff4239fa640 1 -- 172.21.15.117:0/1222541531 shutdown_connections 2024-09-21T06:03:07.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.826+0000 7ff4239fa640 1 -- 172.21.15.117:0/1222541531 wait complete. 2024-09-21T06:03:07.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.827+0000 7ff4239fa640 1 Processor -- start 2024-09-21T06:03:07.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.828+0000 7ff4239fa640 1 -- start start 2024-09-21T06:03:07.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.828+0000 7ff4239fa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff41c102ab0 0x7ff41c19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:07.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.828+0000 7ff4239fa640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff41c1083a0 0x7ff41c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:07.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.828+0000 7ff4239fa640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff41c108cf0 0x7ff41c1a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:07.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.828+0000 7ff4239fa640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c1129b0 con 0x7ff41c1083a0 2024-09-21T06:03:07.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.828+0000 7ff4239fa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c112830 con 0x7ff41c102ab0 2024-09-21T06:03:07.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.828+0000 7ff4239fa640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c112b30 con 0x7ff41c108cf0 2024-09-21T06:03:07.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.828+0000 7ff420f95640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff41c1083a0 0x7ff41c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:07.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.828+0000 7ff421f97640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff41c108cf0 0x7ff41c1a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:07.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.828+0000 7ff420f95640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff41c1083a0 0x7ff41c19cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59826/0 (socket says 172.21.15.117:59826) 2024-09-21T06:03:07.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.828+0000 7ff420f95640 1 -- 172.21.15.117:0/1920778681 learned_addr learned my addr 172.21.15.117:0/1920778681 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:07.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.828+0000 7ff421796640 1 --2- 172.21.15.117:0/1920778681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff41c102ab0 0x7ff41c19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:07.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.829+0000 7ff420f95640 1 -- 172.21.15.117:0/1920778681 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff41c108cf0 msgr2=0x7ff41c1a1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:07.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.829+0000 7ff420f95640 1 --2- 172.21.15.117:0/1920778681 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff41c108cf0 0x7ff41c1a1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:07.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.829+0000 7ff420f95640 1 -- 172.21.15.117:0/1920778681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff41c102ab0 msgr2=0x7ff41c19c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:07.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.829+0000 7ff420f95640 1 --2- 172.21.15.117:0/1920778681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff41c102ab0 0x7ff41c19c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:07.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.829+0000 7ff420f95640 1 -- 172.21.15.117:0/1920778681 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff41c1a25d0 con 0x7ff41c1083a0 2024-09-21T06:03:07.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.829+0000 7ff421f97640 1 --2- 172.21.15.117:0/1920778681 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff41c108cf0 0x7ff41c1a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:03:07.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.830+0000 7ff420f95640 1 --2- 172.21.15.117:0/1920778681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff41c1083a0 0x7ff41c19cc00 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff40c00e9f0 tx=0x7ff40c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:07.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.830+0000 7ff4127fc640 1 -- 172.21.15.117:0/1920778681 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c00ccc0 con 0x7ff41c1083a0 2024-09-21T06:03:07.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.830+0000 7ff4239fa640 1 -- 172.21.15.117:0/1920778681 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff41c1a28c0 con 0x7ff41c1083a0 2024-09-21T06:03:07.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.830+0000 7ff4127fc640 1 -- 172.21.15.117:0/1920778681 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff40c0108e0 con 0x7ff41c1083a0 2024-09-21T06:03:07.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.830+0000 7ff4127fc640 1 -- 172.21.15.117:0/1920778681 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c018860 con 0x7ff41c1083a0 2024-09-21T06:03:07.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.830+0000 7ff4239fa640 1 -- 172.21.15.117:0/1920778681 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff41c1a9020 con 0x7ff41c1083a0 2024-09-21T06:03:07.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.833+0000 7ff429f6e740 1 Processor -- start 2024-09-21T06:03:07.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.834+0000 7ff429f6e740 1 -- start start 2024-09-21T06:03:07.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.834+0000 7ff429f6e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd1663a070 0x55cd169e1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:07.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.834+0000 7ff429f6e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd166111b0 0x55cd169e1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:07.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.834+0000 7ff429f6e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd16619390 0x55cd169e20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:07.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.834+0000 7ff429f6e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cd16957e50 con 0x55cd166111b0 2024-09-21T06:03:07.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.834+0000 7ff429f6e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cd16947280 con 0x55cd1663a070 2024-09-21T06:03:07.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.834+0000 7ff429f6e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55cd1694b090 con 0x55cd16619390 2024-09-21T06:03:07.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.834+0000 7ff421f97640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd1663a070 0x55cd169e1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:07.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.834+0000 7ff421f97640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd1663a070 0x55cd169e1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:44768/0 (socket says 172.21.15.117:44768) 2024-09-21T06:03:07.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.834+0000 7ff421796640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd166111b0 0x55cd169e1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:07.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.834+0000 7ff421f97640 1 -- 172.21.15.117:0/1874320716 learned_addr learned my addr 172.21.15.117:0/1874320716 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:07.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.834+0000 7ff4127fc640 1 -- 172.21.15.117:0/1920778681 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7ff40c010430 con 0x7ff41c1083a0 2024-09-21T06:03:07.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.835+0000 7ff420f95640 1 --2- 172.21.15.117:0/1874320716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd16619390 0x55cd169e20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:07.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.835+0000 7ff421796640 1 -- 172.21.15.117:0/1874320716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd16619390 msgr2=0x55cd169e20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:07.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.835+0000 7ff421796640 1 --2- 172.21.15.117:0/1874320716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd16619390 0x55cd169e20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:07.002 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.835+0000 7ff421796640 1 -- 172.21.15.117:0/1874320716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd1663a070 msgr2=0x55cd169e1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:07.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.835+0000 7ff421796640 1 --2- 172.21.15.117:0/1874320716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd1663a070 0x55cd169e1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:07.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.835+0000 7ff421796640 1 -- 172.21.15.117:0/1874320716 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd166a5650 con 0x55cd166111b0 2024-09-21T06:03:07.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.835+0000 7ff421f97640 1 --2- 172.21.15.117:0/1874320716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd1663a070 0x55cd169e1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:03:07.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.835+0000 7ff4127fc640 1 --2- 172.21.15.117:0/1920778681 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff4080b6680 0x7ff4080b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:07.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.835+0000 7ff421796640 1 --2- 172.21.15.117:0/1874320716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd166111b0 0x55cd169e1b90 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff41800bbf0 tx=0x7ff4180107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f222a3d5b7dda85e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:07.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.836+0000 7ff4127fc640 1 -- 172.21.15.117:0/1920778681 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7ff40c0e7920 con 0x7ff41c1083a0 2024-09-21T06:03:07.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.836+0000 7ff421f97640 1 --2- 172.21.15.117:0/1920778681 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff4080b6680 0x7ff4080b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:07.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.836+0000 7ff4077fe640 1 -- 172.21.15.117:0/1874320716 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff418004030 con 0x55cd166111b0 2024-09-21T06:03:07.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.836+0000 7ff4077fe640 1 -- 172.21.15.117:0/1874320716 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4180035f0 con 0x55cd166111b0 2024-09-21T06:03:07.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.836+0000 7ff4077fe640 1 -- 172.21.15.117:0/1874320716 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff418011680 con 0x55cd166111b0 2024-09-21T06:03:07.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.837+0000 7ff429f6e740 1 -- 172.21.15.117:0/1874320716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd166111b0 msgr2=0x55cd169e1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:07.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.837+0000 7ff421f97640 1 --2- 172.21.15.117:0/1920778681 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff4080b6680 0x7ff4080b8b90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff414033040 tx=0x7ff414032eb0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:07.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.837+0000 7ff429f6e740 1 --2- 172.21.15.117:0/1874320716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd166111b0 0x55cd169e1b90 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff41800bbf0 tx=0x7ff4180107a0 comp rx=0 tx=0).stop 2024-09-21T06:03:07.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.837+0000 7ff429f6e740 1 -- 172.21.15.117:0/1874320716 shutdown_connections 2024-09-21T06:03:07.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.837+0000 7ff429f6e740 1 --2- 172.21.15.117:0/1874320716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd166111b0 0x55cd169e1b90 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:07.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.837+0000 7ff429f6e740 1 --2- 172.21.15.117:0/1874320716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd16619390 0x55cd169e20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:07.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.837+0000 7ff429f6e740 1 --2- 172.21.15.117:0/1874320716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd1663a070 0x55cd169e1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:07.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.837+0000 7ff429f6e740 1 -- 172.21.15.117:0/1874320716 >> 172.21.15.117:0/1874320716 conn(0x55cd1662e460 msgr2=0x55cd169df540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:07.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.837+0000 7ff429f6e740 1 -- 172.21.15.117:0/1874320716 shutdown_connections 2024-09-21T06:03:07.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.837+0000 7ff429f6e740 1 -- 172.21.15.117:0/1874320716 wait complete. 2024-09-21T06:03:07.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.838+0000 7ff429f6e740 1 -- start start 2024-09-21T06:03:07.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.838+0000 7ff429f6e740 1 Processor -- start 2024-09-21T06:03:07.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.838+0000 7ff429f6e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd166111b0 0x55cd169f54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:07.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.838+0000 7ff429f6e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd16619390 0x55cd169f5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:07.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.838+0000 7ff429f6e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd1663a070 0x55cd16a018c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:07.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.838+0000 7ff429f6e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cd167a8700 con 0x55cd166111b0 2024-09-21T06:03:07.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.838+0000 7ff429f6e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cd168e2110 con 0x55cd16619390 2024-09-21T06:03:07.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.838+0000 7ff429f6e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55cd169f8560 con 0x55cd1663a070 2024-09-21T06:03:07.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.838+0000 7ff429f6e740 10 client.0 _ll_register_callbacks cb 0x55cd168fb990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:03:07.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.839+0000 7ff421796640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd166111b0 0x55cd169f54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:07.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.839+0000 7ff421f97640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd1663a070 0x55cd16a018c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:07.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.839+0000 7ff421f97640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd1663a070 0x55cd16a018c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:49548/0 (socket says 172.21.15.117:49548) 2024-09-21T06:03:07.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.839+0000 7ff421f97640 1 -- 172.21.15.117:0/2096745991 learned_addr learned my addr 172.21.15.117:0/2096745991 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:07.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.839+0000 7ff420f95640 1 --2- 172.21.15.117:0/2096745991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd16619390 0x55cd169f5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:07.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.839+0000 7ff421796640 1 -- 172.21.15.117:0/2096745991 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd1663a070 msgr2=0x55cd16a018c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:07.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.839+0000 7ff421796640 1 --2- 172.21.15.117:0/2096745991 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd1663a070 0x55cd16a018c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:07.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.839+0000 7ff421796640 1 -- 172.21.15.117:0/2096745991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd16619390 msgr2=0x55cd169f5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:07.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.839+0000 7ff421796640 1 --2- 172.21.15.117:0/2096745991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd16619390 0x55cd169f5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:07.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.839+0000 7ff421796640 1 -- 172.21.15.117:0/2096745991 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd1690fc40 con 0x55cd166111b0 2024-09-21T06:03:07.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.839+0000 7ff421796640 1 --2- 172.21.15.117:0/2096745991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd166111b0 0x55cd169f54d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff418011010 tx=0x7ff41800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8c279cbec8d63ad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:07.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.839+0000 7ff404ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:03:07.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.839+0000 7ff404ff9640 1 -- 172.21.15.117:0/2096745991 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff418003fe0 con 0x55cd166111b0 2024-09-21T06:03:07.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.839+0000 7ff404ff9640 1 -- 172.21.15.117:0/2096745991 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7ff418004180 con 0x55cd166111b0 2024-09-21T06:03:07.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.839+0000 7ff429f6e740 1 -- 172.21.15.117:0/2096745991 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4180035f0 con 0x55cd166111b0 2024-09-21T06:03:07.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.840+0000 7ff404ff9640 1 -- 172.21.15.117:0/2096745991 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff418004380 con 0x55cd166111b0 2024-09-21T06:03:07.006 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:06 smithi117 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-21T06:03:07.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.840+0000 7ff429f6e740 10 client.16479 fetch_fsmap learned FSMap version 27 2024-09-21T06:03:07.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.840+0000 7ff429f6e740 1 -- 172.21.15.117:0/2096745991 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cd16a042a0 con 0x55cd166111b0 2024-09-21T06:03:07.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.840+0000 7ff404ff9640 1 -- 172.21.15.117:0/2096745991 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4180035f0 con 0x55cd166111b0 2024-09-21T06:03:07.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.840+0000 7ff404ff9640 1 -- 172.21.15.117:0/2096745991 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff418004b60 con 0x55cd166111b0 2024-09-21T06:03:07.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.840+0000 7ff429f6e740 10 client.16479 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:03:07.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.840+0000 7ff429f6e740 10 client.16479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:03:07.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.840+0000 7ff429f6e740 20 client.16479 populate_metadata read hostname 'smithi117' 2024-09-21T06:03:07.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.840+0000 7ff429f6e740 1 --2- 172.21.15.117:0/2096745991 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55cd165f89f0 0x55cd16a07600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:07.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.840+0000 7ff429f6e740 4 client.16479 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:03:07.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.840+0000 7ff429f6e740 1 -- 172.21.15.117:0/2096745991 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd16959da0 con 0x55cd165f89f0 2024-09-21T06:03:07.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.840+0000 7ff420f95640 1 --2- 172.21.15.117:0/2096745991 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55cd165f89f0 0x55cd16a07600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:07.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.841+0000 7ff404ff9640 10 client.16479 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:03:07.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.841+0000 7ff420f95640 1 --2- 172.21.15.117:0/2096745991 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55cd165f89f0 0x55cd16a07600 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4181f918723ee31 server_cookie=a4ff308b479ddf9d in_seq=0 out_seq=0 2024-09-21T06:03:07.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.844+0000 7ff404ff9640 1 -- 172.21.15.117:0/2096745991 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cd16959da0 con 0x55cd165f89f0 2024-09-21T06:03:07.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.844+0000 7ff404ff9640 10 client.16479 handle_command_reply: tid=0 2024-09-21T06:03:07.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.844+0000 7ff429f6e740 1 client.16479 shutdown 2024-09-21T06:03:07.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.844+0000 7ff429f6e740 1 -- 172.21.15.117:0/2096745991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd166111b0 msgr2=0x55cd169f54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:07.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.844+0000 7ff429f6e740 1 --2- 172.21.15.117:0/2096745991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd166111b0 0x55cd169f54d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff418011010 tx=0x7ff41800bfd0 comp rx=0 tx=0).stop 2024-09-21T06:03:07.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.844+0000 7ff429f6e740 1 -- 172.21.15.117:0/2096745991 shutdown_connections 2024-09-21T06:03:07.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.844+0000 7ff429f6e740 1 --2- 172.21.15.117:0/2096745991 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55cd165f89f0 0x55cd16a07600 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:07.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.844+0000 7ff429f6e740 1 --2- 172.21.15.117:0/2096745991 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd1663a070 0x55cd16a018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:07.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.844+0000 7ff429f6e740 1 --2- 172.21.15.117:0/2096745991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd16619390 0x55cd169f5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:07.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.844+0000 7ff429f6e740 1 --2- 172.21.15.117:0/2096745991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd166111b0 0x55cd169f54d0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:07.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.845+0000 7ff429f6e740 1 -- 172.21.15.117:0/2096745991 >> 172.21.15.117:0/2096745991 conn(0x55cd1662e460 msgr2=0x55cd169ddb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:07.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.845+0000 7ff404ff9640 0 client.16479 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:03:07.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.845+0000 7ff429f6e740 1 -- 172.21.15.117:0/2096745991 shutdown_connections 2024-09-21T06:03:07.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.845+0000 7ff429f6e740 1 -- 172.21.15.117:0/2096745991 wait complete. 2024-09-21T06:03:07.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.845+0000 7ff429f6e740 20 client.16479 trim_cache size 0 max 16384 2024-09-21T06:03:07.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.862+0000 7ff429f6e740 1 Processor -- start 2024-09-21T06:03:07.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.863+0000 7ff429f6e740 1 -- start start 2024-09-21T06:03:07.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.863+0000 7ff429f6e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd166111b0 0x55cd169f5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:07.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.863+0000 7ff429f6e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd16619390 0x55cd16a01830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:07.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.863+0000 7ff429f6e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd1663a070 0x55cd169dee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:07.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.863+0000 7ff429f6e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c1ad450 con 0x55cd166111b0 2024-09-21T06:03:07.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.863+0000 7ff429f6e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c1aef90 con 0x55cd1663a070 2024-09-21T06:03:07.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.863+0000 7ff429f6e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c1adb20 con 0x55cd16619390 2024-09-21T06:03:07.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.863+0000 7ff421796640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd166111b0 0x55cd169f5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:07.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.863+0000 7ff421796640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd166111b0 0x55cd169f5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59856/0 (socket says 172.21.15.117:59856) 2024-09-21T06:03:07.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.863+0000 7ff421796640 1 -- 172.21.15.117:0/1244442295 learned_addr learned my addr 172.21.15.117:0/1244442295 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:07.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.863+0000 7ff420f95640 1 --2- 172.21.15.117:0/1244442295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd16619390 0x55cd16a01830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:07.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.863+0000 7ff421f97640 1 --2- 172.21.15.117:0/1244442295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd1663a070 0x55cd169dee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:07.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.863+0000 7ff421796640 1 -- 172.21.15.117:0/1244442295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd16619390 msgr2=0x55cd16a01830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:07.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.863+0000 7ff421796640 1 --2- 172.21.15.117:0/1244442295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd16619390 0x55cd16a01830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:07.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.863+0000 7ff421796640 1 -- 172.21.15.117:0/1244442295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd1663a070 msgr2=0x55cd169dee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:07.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.863+0000 7ff421796640 1 --2- 172.21.15.117:0/1244442295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd1663a070 0x55cd169dee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:07.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.863+0000 7ff421796640 1 -- 172.21.15.117:0/1244442295 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd169f6790 con 0x55cd166111b0 2024-09-21T06:03:07.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.863+0000 7ff421796640 1 --2- 172.21.15.117:0/1244442295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd166111b0 0x55cd169f5990 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff418000c00 tx=0x7ff418010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36feba0dbed7adc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:07.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.864+0000 7ff406ffd640 1 -- 172.21.15.117:0/1244442295 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41803b070 con 0x55cd166111b0 2024-09-21T06:03:07.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.864+0000 7ff406ffd640 1 -- 172.21.15.117:0/1244442295 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4180035f0 con 0x55cd166111b0 2024-09-21T06:03:07.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.864+0000 7ff406ffd640 1 -- 172.21.15.117:0/1244442295 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff418033070 con 0x55cd166111b0 2024-09-21T06:03:07.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.864+0000 7ff429f6e740 1 -- 172.21.15.117:0/1244442295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd166111b0 msgr2=0x55cd169f5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:07.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.864+0000 7ff429f6e740 1 --2- 172.21.15.117:0/1244442295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd166111b0 0x55cd169f5990 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff418000c00 tx=0x7ff418010ca0 comp rx=0 tx=0).stop 2024-09-21T06:03:07.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.865+0000 7ff429f6e740 1 -- 172.21.15.117:0/1244442295 shutdown_connections 2024-09-21T06:03:07.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.865+0000 7ff429f6e740 1 --2- 172.21.15.117:0/1244442295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd1663a070 0x55cd169dee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:07.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.865+0000 7ff429f6e740 1 --2- 172.21.15.117:0/1244442295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd16619390 0x55cd16a01830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:07.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.865+0000 7ff429f6e740 1 --2- 172.21.15.117:0/1244442295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd166111b0 0x55cd169f5990 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:07.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.865+0000 7ff429f6e740 1 -- 172.21.15.117:0/1244442295 >> 172.21.15.117:0/1244442295 conn(0x55cd1662e460 msgr2=0x55cd169f5280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:07.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.865+0000 7ff429f6e740 1 -- 172.21.15.117:0/1244442295 shutdown_connections 2024-09-21T06:03:07.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.865+0000 7ff429f6e740 1 -- 172.21.15.117:0/1244442295 wait complete. 2024-09-21T06:03:07.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.865+0000 7ff429f6e740 1 -- start start 2024-09-21T06:03:07.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.865+0000 7ff429f6e740 1 Processor -- start 2024-09-21T06:03:07.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.865+0000 7ff429f6e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd166111b0 0x55cd16a075d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:07.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.865+0000 7ff429f6e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd16619390 0x55cd16a07b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:07.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.866+0000 7ff429f6e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd1663a070 0x55cd16a08050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:07.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.866+0000 7ff429f6e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c1ae8c0 con 0x55cd166111b0 2024-09-21T06:03:07.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.866+0000 7ff429f6e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c1ae1f0 con 0x55cd1663a070 2024-09-21T06:03:07.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.866+0000 7ff429f6e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c1ab470 con 0x55cd16619390 2024-09-21T06:03:07.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.866+0000 7ff421796640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd166111b0 0x55cd16a075d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:07.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.866+0000 7ff421796640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd166111b0 0x55cd16a075d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59868/0 (socket says 172.21.15.117:59868) 2024-09-21T06:03:07.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.866+0000 7ff421796640 1 -- 172.21.15.117:0/3625225301 learned_addr learned my addr 172.21.15.117:0/3625225301 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:07.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.866+0000 7ff429f6e740 10 client.0 _ll_register_callbacks cb 0x7ff41800ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:03:07.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.866+0000 7ff420f95640 1 --2- 172.21.15.117:0/3625225301 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd16619390 0x55cd16a07b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:07.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.866+0000 7ff421f97640 1 --2- 172.21.15.117:0/3625225301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd1663a070 0x55cd16a08050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:07.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.866+0000 7ff421796640 1 -- 172.21.15.117:0/3625225301 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd16619390 msgr2=0x55cd16a07b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:07.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.866+0000 7ff421796640 1 --2- 172.21.15.117:0/3625225301 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd16619390 0x55cd16a07b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:07.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.866+0000 7ff421796640 1 -- 172.21.15.117:0/3625225301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd1663a070 msgr2=0x55cd16a08050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:07.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.866+0000 7ff421796640 1 --2- 172.21.15.117:0/3625225301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd1663a070 0x55cd16a08050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:07.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.866+0000 7ff421796640 1 -- 172.21.15.117:0/3625225301 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd16a70d70 con 0x55cd166111b0 2024-09-21T06:03:07.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.866+0000 7ff421f97640 1 --2- 172.21.15.117:0/3625225301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd1663a070 0x55cd16a08050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:03:07.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.866+0000 7ff421796640 1 --2- 172.21.15.117:0/3625225301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd166111b0 0x55cd16a075d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff418039040 tx=0x7ff41802da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e407aee158b96c13 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:07.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.866+0000 7ff404ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:03:07.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.866+0000 7ff404ff9640 1 -- 172.21.15.117:0/3625225301 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41803b070 con 0x55cd166111b0 2024-09-21T06:03:07.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.866+0000 7ff404ff9640 1 -- 172.21.15.117:0/3625225301 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7ff418004720 con 0x55cd166111b0 2024-09-21T06:03:07.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.866+0000 7ff429f6e740 1 -- 172.21.15.117:0/3625225301 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4180035f0 con 0x55cd166111b0 2024-09-21T06:03:07.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.866+0000 7ff404ff9640 1 -- 172.21.15.117:0/3625225301 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff418033050 con 0x55cd166111b0 2024-09-21T06:03:07.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.866+0000 7ff429f6e740 10 client.16485 fetch_fsmap learned FSMap version 27 2024-09-21T06:03:07.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.866+0000 7ff429f6e740 1 -- 172.21.15.117:0/3625225301 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cd16a71f40 con 0x55cd166111b0 2024-09-21T06:03:07.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.867+0000 7ff404ff9640 1 -- 172.21.15.117:0/3625225301 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4180035f0 con 0x55cd166111b0 2024-09-21T06:03:07.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.867+0000 7ff404ff9640 1 -- 172.21.15.117:0/3625225301 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff418005410 con 0x55cd166111b0 2024-09-21T06:03:07.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.867+0000 7ff429f6e740 10 client.16485 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:03:07.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.867+0000 7ff429f6e740 10 client.16485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:03:07.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.867+0000 7ff429f6e740 20 client.16485 populate_metadata read hostname 'smithi117' 2024-09-21T06:03:07.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.867+0000 7ff429f6e740 1 --2- 172.21.15.117:0/3625225301 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55cd165cdda0 0x55cd16a74610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:07.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.867+0000 7ff429f6e740 4 client.16485 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:03:07.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.867+0000 7ff429f6e740 1 -- 172.21.15.117:0/3625225301 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cd16955850 con 0x55cd165cdda0 2024-09-21T06:03:07.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.867+0000 7ff420f95640 1 --2- 172.21.15.117:0/3625225301 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55cd165cdda0 0x55cd16a74610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:07.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.868+0000 7ff404ff9640 10 client.16485 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:03:07.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.868+0000 7ff420f95640 1 --2- 172.21.15.117:0/3625225301 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55cd165cdda0 0x55cd16a74610 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11b6f75e3f7930a4 server_cookie=23d565480e83c6de in_seq=0 out_seq=0 2024-09-21T06:03:07.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.870+0000 7ff404ff9640 1 -- 172.21.15.117:0/3625225301 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1706 (crc 0 0 0) 0x55cd16955850 con 0x55cd165cdda0 2024-09-21T06:03:07.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.870+0000 7ff404ff9640 10 client.16485 handle_command_reply: tid=0 2024-09-21T06:03:07.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.870+0000 7ff429f6e740 1 client.16485 shutdown 2024-09-21T06:03:07.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.870+0000 7ff429f6e740 1 -- 172.21.15.117:0/3625225301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd166111b0 msgr2=0x55cd16a075d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:07.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.870+0000 7ff429f6e740 1 --2- 172.21.15.117:0/3625225301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd166111b0 0x55cd16a075d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff418039040 tx=0x7ff41802da50 comp rx=0 tx=0).stop 2024-09-21T06:03:07.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.870+0000 7ff429f6e740 1 -- 172.21.15.117:0/3625225301 shutdown_connections 2024-09-21T06:03:07.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.870+0000 7ff429f6e740 1 --2- 172.21.15.117:0/3625225301 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55cd165cdda0 0x55cd16a74610 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:07.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.870+0000 7ff429f6e740 1 --2- 172.21.15.117:0/3625225301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd1663a070 0x55cd16a08050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:07.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.870+0000 7ff429f6e740 1 --2- 172.21.15.117:0/3625225301 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cd16619390 0x55cd16a07b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:07.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.870+0000 7ff429f6e740 1 --2- 172.21.15.117:0/3625225301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd166111b0 0x55cd16a075d0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:07.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.870+0000 7ff404ff9640 0 client.16485 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:03:07.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.870+0000 7ff429f6e740 1 -- 172.21.15.117:0/3625225301 >> 172.21.15.117:0/3625225301 conn(0x55cd1662e460 msgr2=0x55cd169dee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:07.016 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T06:03:07.016 INFO:teuthology.orchestra.run.smithi117.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:03:07.016 INFO:teuthology.orchestra.run.smithi117.stdout: "scrubs": { 2024-09-21T06:03:07.016 INFO:teuthology.orchestra.run.smithi117.stdout: "29ff9b7c-49d9-4910-85be-bace7e940cd5": { 2024-09-21T06:03:07.016 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:03:07.016 INFO:teuthology.orchestra.run.smithi117.stdout: "tag": "29ff9b7c-49d9-4910-85be-bace7e940cd5", 2024-09-21T06:03:07.016 INFO:teuthology.orchestra.run.smithi117.stdout: "options": "recursive,force" 2024-09-21T06:03:07.016 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:03:07.017 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:03:07.017 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_stats": { 2024-09-21T06:03:07.017 INFO:teuthology.orchestra.run.smithi117.stdout: "29ff9b7c-49d9-4910-85be-bace7e940cd5": { 2024-09-21T06:03:07.017 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:03:03.775527+0000", 2024-09-21T06:03:07.017 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:03:07.017 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:03:07.017 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:03:07.017 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:03:07.017 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:03:07.017 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:03:07.017 INFO:teuthology.orchestra.run.smithi117.stdout: "d543b0ae-b710-4153-99da-fe5714839c1a": { 2024-09-21T06:03:07.017 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:01:20.574754+0000", 2024-09-21T06:03:07.017 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:03:07.017 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:03:07.017 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:03:07.018 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:03:07.018 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:03:07.018 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:03:07.018 INFO:teuthology.orchestra.run.smithi117.stdout: "ad80d9f1-6aff-46c0-91ab-2c1ab0681039": { 2024-09-21T06:03:07.018 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:59:31.949108+0000", 2024-09-21T06:03:07.018 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:03:07.018 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:03:07.018 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:03:07.018 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:03:07.018 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:03:07.018 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:03:07.018 INFO:teuthology.orchestra.run.smithi117.stdout: "8b3fe95a-b76b-4e94-afd0-72fe35e2dd92": { 2024-09-21T06:03:07.018 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:58:01.246076+0000", 2024-09-21T06:03:07.019 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:03:07.019 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:03:07.019 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:03:07.019 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:03:07.019 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:03:07.019 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:03:07.019 INFO:teuthology.orchestra.run.smithi117.stdout: "b30bfff0-0f6b-4e30-8467-ff7db421eba6": { 2024-09-21T06:03:07.019 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:57:14.939082+0000", 2024-09-21T06:03:07.019 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:03:07.019 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:03:07.019 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:03:07.019 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:03:07.019 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:03:07.019 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:03:07.020 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:03:07.020 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T06:03:07.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.871+0000 7ff429f6e740 1 -- 172.21.15.117:0/3625225301 shutdown_connections 2024-09-21T06:03:07.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.871+0000 7ff429f6e740 1 -- 172.21.15.117:0/3625225301 wait complete. 2024-09-21T06:03:07.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.871+0000 7ff429f6e740 20 client.16485 trim_cache size 0 max 16384 2024-09-21T06:03:07.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.871+0000 7ff4239fa640 1 -- 172.21.15.117:0/1920778681 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff4080b6680 msgr2=0x7ff4080b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:07.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.871+0000 7ff4239fa640 1 --2- 172.21.15.117:0/1920778681 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff4080b6680 0x7ff4080b8b90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff414033040 tx=0x7ff414032eb0 comp rx=0 tx=0).stop 2024-09-21T06:03:07.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.871+0000 7ff4239fa640 1 -- 172.21.15.117:0/1920778681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff41c1083a0 msgr2=0x7ff41c19cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:07.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.871+0000 7ff4239fa640 1 --2- 172.21.15.117:0/1920778681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff41c1083a0 0x7ff41c19cc00 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff40c00e9f0 tx=0x7ff40c00eeb0 comp rx=0 tx=0).stop 2024-09-21T06:03:07.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.871+0000 7ff4239fa640 1 -- 172.21.15.117:0/1920778681 shutdown_connections 2024-09-21T06:03:07.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.871+0000 7ff4239fa640 1 --2- 172.21.15.117:0/1920778681 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff4080b6680 0x7ff4080b8b90 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:07.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.871+0000 7ff4239fa640 1 --2- 172.21.15.117:0/1920778681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff41c1083a0 0x7ff41c19cc00 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:07.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.871+0000 7ff4239fa640 1 --2- 172.21.15.117:0/1920778681 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff41c108cf0 0x7ff41c1a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:07.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.871+0000 7ff4239fa640 1 --2- 172.21.15.117:0/1920778681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff41c102ab0 0x7ff41c19c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:07.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.871+0000 7ff4239fa640 1 -- 172.21.15.117:0/1920778681 >> 172.21.15.117:0/1920778681 conn(0x7ff41c079680 msgr2=0x7ff41c1063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:07.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.871+0000 7ff4239fa640 1 -- 172.21.15.117:0/1920778681 shutdown_connections 2024-09-21T06:03:07.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:06.871+0000 7ff4239fa640 1 -- 172.21.15.117:0/1920778681 wait complete. 2024-09-21T06:03:07.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:06 smithi192 ceph-mon[34700]: scrub summary: idle+waiting paths [/] 2024-09-21T06:03:07.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:06 smithi138 ceph-mon[34391]: scrub summary: idle+waiting paths [/] 2024-09-21T06:03:08.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:07 smithi192 ceph-mon[34700]: pgmap v501: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 843 MiB/s wr, 1.71k op/s 2024-09-21T06:03:08.239 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:07 smithi117 ceph-mon[30947]: pgmap v501: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 843 MiB/s wr, 1.71k op/s 2024-09-21T06:03:08.267 INFO:tasks.cephfs.filesystem:scrub status for tag:29ff9b7c-49d9-4910-85be-bace7e940cd5 - {'path': '/', 'tag': '29ff9b7c-49d9-4910-85be-bace7e940cd5', 'options': 'recursive,force'} 2024-09-21T06:03:08.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:07 smithi138 ceph-mon[34391]: pgmap v501: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 843 MiB/s wr, 1.71k op/s 2024-09-21T06:03:10.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:09 smithi192 ceph-mon[34700]: pgmap v502: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 823 MiB/s wr, 1.92k op/s 2024-09-21T06:03:10.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:09 smithi117 ceph-mon[30947]: pgmap v502: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 823 MiB/s wr, 1.92k op/s 2024-09-21T06:03:10.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:09 smithi138 ceph-mon[34391]: pgmap v502: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 823 MiB/s wr, 1.92k op/s 2024-09-21T06:03:11.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:10 smithi192 ceph-mon[34700]: scrub summary: idle 2024-09-21T06:03:11.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:10 smithi117 ceph-mon[30947]: scrub summary: idle 2024-09-21T06:03:11.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:10 smithi138 ceph-mon[34391]: scrub summary: idle 2024-09-21T06:03:12.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:11 smithi192 ceph-mon[34700]: pgmap v503: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 644 MiB/s wr, 1.95k op/s 2024-09-21T06:03:12.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:11 smithi117 ceph-mon[30947]: pgmap v503: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 644 MiB/s wr, 1.95k op/s 2024-09-21T06:03:12.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:11 smithi138 ceph-mon[34391]: pgmap v503: 97 pgs: 97 active+clean; 50 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 644 MiB/s wr, 1.95k op/s 2024-09-21T06:03:14.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:13 smithi192 ceph-mon[34700]: pgmap v504: 97 pgs: 97 active+clean; 51 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 626 MiB/s wr, 2.33k op/s 2024-09-21T06:03:14.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:13 smithi117 ceph-mon[30947]: pgmap v504: 97 pgs: 97 active+clean; 51 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 626 MiB/s wr, 2.33k op/s 2024-09-21T06:03:14.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:13 smithi138 ceph-mon[34391]: pgmap v504: 97 pgs: 97 active+clean; 51 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 626 MiB/s wr, 2.33k op/s 2024-09-21T06:03:16.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:15 smithi192 ceph-mon[34700]: pgmap v505: 97 pgs: 97 active+clean; 52 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 586 MiB/s wr, 2.90k op/s 2024-09-21T06:03:16.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:15 smithi117 ceph-mon[30947]: pgmap v505: 97 pgs: 97 active+clean; 52 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 586 MiB/s wr, 2.90k op/s 2024-09-21T06:03:16.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:15 smithi138 ceph-mon[34391]: pgmap v505: 97 pgs: 97 active+clean; 52 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 586 MiB/s wr, 2.90k op/s 2024-09-21T06:03:18.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:17 smithi192 ceph-mon[34700]: pgmap v506: 97 pgs: 97 active+clean; 53 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 570 MiB/s wr, 3.43k op/s 2024-09-21T06:03:18.279 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:17 smithi117 ceph-mon[30947]: pgmap v506: 97 pgs: 97 active+clean; 53 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 570 MiB/s wr, 3.43k op/s 2024-09-21T06:03:18.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:17 smithi138 ceph-mon[34391]: pgmap v506: 97 pgs: 97 active+clean; 53 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 570 MiB/s wr, 3.43k op/s 2024-09-21T06:03:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.827+0000 7f1723fff640 1 -- 172.21.15.117:0/2830680205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1724101f90 msgr2=0x7f17241023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.827+0000 7f1723fff640 1 --2- 172.21.15.117:0/2830680205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1724101f90 0x7f17241023a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f1714009900 tx=0x7f171402a410 comp rx=0 tx=0).stop 2024-09-21T06:03:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.828+0000 7f1723fff640 1 -- 172.21.15.117:0/2830680205 shutdown_connections 2024-09-21T06:03:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.828+0000 7f1723fff640 1 --2- 172.21.15.117:0/2830680205 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1724108210 0x7f172410ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.828+0000 7f1723fff640 1 --2- 172.21.15.117:0/2830680205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17241078c0 0x7f1724107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.828+0000 7f1723fff640 1 --2- 172.21.15.117:0/2830680205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1724101f90 0x7f17241023a0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.828+0000 7f1723fff640 1 -- 172.21.15.117:0/2830680205 >> 172.21.15.117:0/2830680205 conn(0x7f1724079b00 msgr2=0x7f1724079f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.828+0000 7f1723fff640 1 -- 172.21.15.117:0/2830680205 shutdown_connections 2024-09-21T06:03:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.828+0000 7f1723fff640 1 -- 172.21.15.117:0/2830680205 wait complete. 2024-09-21T06:03:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.829+0000 7f1723fff640 1 Processor -- start 2024-09-21T06:03:18.834 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.830+0000 7f1723fff640 1 -- start start 2024-09-21T06:03:18.834 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.830+0000 7f1723fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1724101f90 0x7f172419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.830+0000 7f1723fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17241078c0 0x7f172419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.831+0000 7f1723fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1724108210 0x7f172419fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.831+0000 7f1723fff640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f17241138b0 con 0x7f1724101f90 2024-09-21T06:03:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.831+0000 7f1723fff640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1724113730 con 0x7f1724108210 2024-09-21T06:03:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.831+0000 7f1723fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1724113a30 con 0x7f17241078c0 2024-09-21T06:03:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.831+0000 7f1722ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1724101f90 0x7f172419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.831+0000 7f1722ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1724101f90 0x7f172419a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49902/0 (socket says 172.21.15.117:49902) 2024-09-21T06:03:18.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.831+0000 7f1722ffd640 1 -- 172.21.15.117:0/2283444582 learned_addr learned my addr 172.21.15.117:0/2283444582 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.831+0000 7f17237fe640 1 --2- 172.21.15.117:0/2283444582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1724108210 0x7f172419fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.832+0000 7f1722ffd640 1 -- 172.21.15.117:0/2283444582 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17241078c0 msgr2=0x7f172419a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.832+0000 7f1722ffd640 1 --2- 172.21.15.117:0/2283444582 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17241078c0 0x7f172419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.832+0000 7f1722ffd640 1 -- 172.21.15.117:0/2283444582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1724108210 msgr2=0x7f172419fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.832+0000 7f1722ffd640 1 --2- 172.21.15.117:0/2283444582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1724108210 0x7f172419fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.832+0000 7f1722ffd640 1 -- 172.21.15.117:0/2283444582 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17241a01e0 con 0x7f1724101f90 2024-09-21T06:03:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.832+0000 7f1722ffd640 1 --2- 172.21.15.117:0/2283444582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1724101f90 0x7f172419a2b0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f1714006010 tx=0x7f17140045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.834+0000 7f17288d3640 1 -- 172.21.15.117:0/2283444582 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1714040070 con 0x7f1724101f90 2024-09-21T06:03:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.834+0000 7f17288d3640 1 -- 172.21.15.117:0/2283444582 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1714004c60 con 0x7f1724101f90 2024-09-21T06:03:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.834+0000 7f17288d3640 1 -- 172.21.15.117:0/2283444582 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1714035020 con 0x7f1724101f90 2024-09-21T06:03:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.834+0000 7f1723fff640 1 -- 172.21.15.117:0/2283444582 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17241a0470 con 0x7f1724101f90 2024-09-21T06:03:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.834+0000 7f1723fff640 1 -- 172.21.15.117:0/2283444582 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1724110090 con 0x7f1724101f90 2024-09-21T06:03:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.838+0000 7f173083e740 1 Processor -- start 2024-09-21T06:03:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.839+0000 7f17288d3640 1 -- 172.21.15.117:0/2283444582 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f1714043070 con 0x7f1724101f90 2024-09-21T06:03:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.839+0000 7f173083e740 1 -- start start 2024-09-21T06:03:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.840+0000 7f173083e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ae4db1b0 0x5566ae880ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.840+0000 7f173083e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566ae4b22f0 0x5566ae8810e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.840+0000 7f173083e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566ae4ba4d0 0x5566ae881620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.840+0000 7f173083e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5566ae7f56f0 con 0x5566ae4ba4d0 2024-09-21T06:03:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.840+0000 7f173083e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5566ae7e4b20 con 0x5566ae4db1b0 2024-09-21T06:03:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.840+0000 7f173083e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5566ae7e8930 con 0x5566ae4b22f0 2024-09-21T06:03:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.840+0000 7f17227fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566ae4ba4d0 0x5566ae881620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.840+0000 7f17227fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566ae4ba4d0 0x5566ae881620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49918/0 (socket says 172.21.15.117:49918) 2024-09-21T06:03:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.840+0000 7f17227fc640 1 -- 172.21.15.117:0/1404122509 learned_addr learned my addr 172.21.15.117:0/1404122509 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.840+0000 7f17237fe640 1 --2- 172.21.15.117:0/1404122509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ae4db1b0 0x5566ae880ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.841+0000 7f17227fc640 1 -- 172.21.15.117:0/1404122509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566ae4b22f0 msgr2=0x5566ae8810e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.841+0000 7f17227fc640 1 --2- 172.21.15.117:0/1404122509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566ae4b22f0 0x5566ae8810e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.841+0000 7f17227fc640 1 -- 172.21.15.117:0/1404122509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ae4db1b0 msgr2=0x5566ae880ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.841+0000 7f17227fc640 1 --2- 172.21.15.117:0/1404122509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ae4db1b0 0x5566ae880ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.841+0000 7f17227fc640 1 -- 172.21.15.117:0/1404122509 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566ae7aa530 con 0x5566ae4ba4d0 2024-09-21T06:03:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.841+0000 7f17227fc640 1 --2- 172.21.15.117:0/1404122509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566ae4ba4d0 0x5566ae881620 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f1718009760 tx=0x7f1718009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d089177a6eb88a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.841+0000 7f17288d3640 1 --2- 172.21.15.117:0/2283444582 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f170c0b6700 0x7f170c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.841+0000 7f1709ffb640 1 -- 172.21.15.117:0/1404122509 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718015020 con 0x5566ae4ba4d0 2024-09-21T06:03:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.842+0000 7f17288d3640 1 -- 172.21.15.117:0/2283444582 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f1714108170 con 0x7f1724101f90 2024-09-21T06:03:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.842+0000 7f1709ffb640 1 -- 172.21.15.117:0/1404122509 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1718011420 con 0x5566ae4ba4d0 2024-09-21T06:03:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.842+0000 7f1709ffb640 1 -- 172.21.15.117:0/1404122509 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17180105e0 con 0x5566ae4ba4d0 2024-09-21T06:03:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.842+0000 7f17237fe640 1 --2- 172.21.15.117:0/2283444582 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f170c0b6700 0x7f170c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.844+0000 7f17237fe640 1 --2- 172.21.15.117:0/2283444582 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f170c0b6700 0x7f170c0b8c10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f171c005fd0 tx=0x7f171c009600 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.844+0000 7f173083e740 1 -- 172.21.15.117:0/1404122509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566ae4ba4d0 msgr2=0x5566ae881620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.844+0000 7f173083e740 1 --2- 172.21.15.117:0/1404122509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566ae4ba4d0 0x5566ae881620 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f1718009760 tx=0x7f1718009c20 comp rx=0 tx=0).stop 2024-09-21T06:03:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.844+0000 7f173083e740 1 -- 172.21.15.117:0/1404122509 shutdown_connections 2024-09-21T06:03:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.844+0000 7f173083e740 1 --2- 172.21.15.117:0/1404122509 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566ae4ba4d0 0x5566ae881620 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.844+0000 7f173083e740 1 --2- 172.21.15.117:0/1404122509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566ae4b22f0 0x5566ae8810e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.844+0000 7f173083e740 1 --2- 172.21.15.117:0/1404122509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ae4db1b0 0x5566ae880ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.844+0000 7f173083e740 1 -- 172.21.15.117:0/1404122509 >> 172.21.15.117:0/1404122509 conn(0x5566ae4cf300 msgr2=0x5566ae87ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.844+0000 7f173083e740 1 -- 172.21.15.117:0/1404122509 shutdown_connections 2024-09-21T06:03:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.844+0000 7f173083e740 1 -- 172.21.15.117:0/1404122509 wait complete. 2024-09-21T06:03:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.845+0000 7f173083e740 1 -- start start 2024-09-21T06:03:18.847 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.845+0000 7f173083e740 1 Processor -- start 2024-09-21T06:03:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.846+0000 7f173083e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566ae4b22f0 0x5566ae88c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.846+0000 7f173083e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ae4ba4d0 0x5566ae88c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.846+0000 7f173083e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566ae4db1b0 0x5566ae898610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.846+0000 7f173083e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5566ae71cab0 con 0x5566ae4db1b0 2024-09-21T06:03:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.846+0000 7f173083e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5566ae78a730 con 0x5566ae4ba4d0 2024-09-21T06:03:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.846+0000 7f173083e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5566ae88f320 con 0x5566ae4b22f0 2024-09-21T06:03:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.846+0000 7f173083e740 10 client.0 _ll_register_callbacks cb 0x5566ae7de6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:03:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.846+0000 7f17237fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566ae4db1b0 0x5566ae898610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.846+0000 7f17237fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566ae4db1b0 0x5566ae898610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49932/0 (socket says 172.21.15.117:49932) 2024-09-21T06:03:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.846+0000 7f17237fe640 1 -- 172.21.15.117:0/2239050724 learned_addr learned my addr 172.21.15.117:0/2239050724 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.847+0000 7f17227fc640 1 --2- 172.21.15.117:0/2239050724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ae4ba4d0 0x5566ae88c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.847+0000 7f17237fe640 1 -- 172.21.15.117:0/2239050724 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566ae4b22f0 msgr2=0x5566ae88c300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:03:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.847+0000 7f17237fe640 1 --2- 172.21.15.117:0/2239050724 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566ae4b22f0 0x5566ae88c300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.847+0000 7f17237fe640 1 -- 172.21.15.117:0/2239050724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ae4ba4d0 msgr2=0x5566ae88c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.847+0000 7f17237fe640 1 --2- 172.21.15.117:0/2239050724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ae4ba4d0 0x5566ae88c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.847+0000 7f17237fe640 1 -- 172.21.15.117:0/2239050724 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566ae546790 con 0x5566ae4db1b0 2024-09-21T06:03:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.848+0000 7f17237fe640 1 --2- 172.21.15.117:0/2239050724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566ae4db1b0 0x5566ae898610 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f171c01d6f0 tx=0x7f171c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d76a29d3983007f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.848+0000 7f1702ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:03:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.848+0000 7f1702ffd640 1 -- 172.21.15.117:0/2239050724 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171c003b70 con 0x5566ae4db1b0 2024-09-21T06:03:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.849+0000 7f173083e740 1 -- 172.21.15.117:0/2239050724 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1718011420 con 0x5566ae4db1b0 2024-09-21T06:03:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.849+0000 7f1702ffd640 1 -- 172.21.15.117:0/2239050724 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f171c003d10 con 0x5566ae4db1b0 2024-09-21T06:03:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.849+0000 7f1702ffd640 1 -- 172.21.15.117:0/2239050724 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f171c004030 con 0x5566ae4db1b0 2024-09-21T06:03:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.849+0000 7f173083e740 10 client.16497 fetch_fsmap learned FSMap version 27 2024-09-21T06:03:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.849+0000 7f173083e740 1 -- 172.21.15.117:0/2239050724 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5566ae5bf620 con 0x5566ae4db1b0 2024-09-21T06:03:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.851+0000 7f1702ffd640 1 -- 172.21.15.117:0/2239050724 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1718011420 con 0x5566ae4db1b0 2024-09-21T06:03:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.851+0000 7f1702ffd640 1 -- 172.21.15.117:0/2239050724 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f171c00f040 con 0x5566ae4db1b0 2024-09-21T06:03:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.852+0000 7f173083e740 10 client.16497 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:03:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.852+0000 7f173083e740 10 client.16497 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:03:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.852+0000 7f173083e740 20 client.16497 populate_metadata read hostname 'smithi117' 2024-09-21T06:03:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.852+0000 7f173083e740 1 --2- 172.21.15.117:0/2239050724 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5566ae499d80 0x5566ae89e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.852+0000 7f173083e740 4 client.16497 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:03:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.852+0000 7f173083e740 1 -- 172.21.15.117:0/2239050724 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566ae7f7640 con 0x5566ae499d80 2024-09-21T06:03:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.855+0000 7f1722ffd640 1 --2- 172.21.15.117:0/2239050724 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5566ae499d80 0x5566ae89e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.860+0000 7f1722ffd640 1 --2- 172.21.15.117:0/2239050724 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5566ae499d80 0x5566ae89e4e0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee7f238cfff53488 server_cookie=62325e4f28d35cce in_seq=0 out_seq=0 2024-09-21T06:03:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.860+0000 7f1702ffd640 10 client.16497 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:03:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.867+0000 7f1702ffd640 1 -- 172.21.15.117:0/2239050724 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5566ae7f7640 con 0x5566ae499d80 2024-09-21T06:03:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.867+0000 7f1702ffd640 10 client.16497 handle_command_reply: tid=0 2024-09-21T06:03:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.867+0000 7f173083e740 1 client.16497 shutdown 2024-09-21T06:03:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.867+0000 7f173083e740 1 -- 172.21.15.117:0/2239050724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566ae4db1b0 msgr2=0x5566ae898610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.867+0000 7f173083e740 1 --2- 172.21.15.117:0/2239050724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566ae4db1b0 0x5566ae898610 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f171c01d6f0 tx=0x7f171c004a40 comp rx=0 tx=0).stop 2024-09-21T06:03:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.867+0000 7f173083e740 1 -- 172.21.15.117:0/2239050724 shutdown_connections 2024-09-21T06:03:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.867+0000 7f173083e740 1 --2- 172.21.15.117:0/2239050724 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5566ae499d80 0x5566ae89e4e0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.867+0000 7f173083e740 1 --2- 172.21.15.117:0/2239050724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566ae4db1b0 0x5566ae898610 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.867+0000 7f173083e740 1 --2- 172.21.15.117:0/2239050724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ae4ba4d0 0x5566ae88c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.867+0000 7f173083e740 1 --2- 172.21.15.117:0/2239050724 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566ae4b22f0 0x5566ae88c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.867+0000 7f173083e740 1 -- 172.21.15.117:0/2239050724 >> 172.21.15.117:0/2239050724 conn(0x5566ae4cf300 msgr2=0x5566ae881bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.867+0000 7f1702ffd640 0 client.16497 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:03:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.867+0000 7f173083e740 1 -- 172.21.15.117:0/2239050724 shutdown_connections 2024-09-21T06:03:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.867+0000 7f173083e740 1 -- 172.21.15.117:0/2239050724 wait complete. 2024-09-21T06:03:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.868+0000 7f173083e740 20 client.16497 trim_cache size 0 max 16384 2024-09-21T06:03:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.893+0000 7f173083e740 1 Processor -- start 2024-09-21T06:03:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.893+0000 7f173083e740 1 -- start start 2024-09-21T06:03:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.894+0000 7f173083e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566ae4b22f0 0x5566ae88c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.894+0000 7f173083e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566ae4ba4d0 0x5566ae886370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.894+0000 7f173083e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ae4db1b0 0x5566ae898510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.894+0000 7f173083e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f17241b1620 con 0x5566ae4ba4d0 2024-09-21T06:03:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.894+0000 7f173083e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f17241b3160 con 0x5566ae4db1b0 2024-09-21T06:03:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.894+0000 7f173083e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f17241b1cf0 con 0x5566ae4b22f0 2024-09-21T06:03:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.894+0000 7f17227fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566ae4ba4d0 0x5566ae886370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.894+0000 7f17227fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566ae4ba4d0 0x5566ae886370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49938/0 (socket says 172.21.15.117:49938) 2024-09-21T06:03:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.894+0000 7f17227fc640 1 -- 172.21.15.117:0/1127216611 learned_addr learned my addr 172.21.15.117:0/1127216611 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.894+0000 7f17227fc640 1 -- 172.21.15.117:0/1127216611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566ae4b22f0 msgr2=0x5566ae88c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.894+0000 7f17227fc640 1 --2- 172.21.15.117:0/1127216611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566ae4b22f0 0x5566ae88c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.894+0000 7f17227fc640 1 -- 172.21.15.117:0/1127216611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ae4db1b0 msgr2=0x5566ae898510 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:03:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.894+0000 7f17227fc640 1 --2- 172.21.15.117:0/1127216611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ae4db1b0 0x5566ae898510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.895+0000 7f17227fc640 1 -- 172.21.15.117:0/1127216611 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566ae8826a0 con 0x5566ae4ba4d0 2024-09-21T06:03:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.895+0000 7f17227fc640 1 --2- 172.21.15.117:0/1127216611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566ae4ba4d0 0x5566ae886370 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1718006f90 tx=0x7f1718009320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42305459dd2378dc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.895+0000 7f17097fa640 1 -- 172.21.15.117:0/1127216611 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718004220 con 0x5566ae4ba4d0 2024-09-21T06:03:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.897+0000 7f17097fa640 1 -- 172.21.15.117:0/1127216611 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f171800bd80 con 0x5566ae4ba4d0 2024-09-21T06:03:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.897+0000 7f17097fa640 1 -- 172.21.15.117:0/1127216611 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718003d40 con 0x5566ae4ba4d0 2024-09-21T06:03:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.898+0000 7f173083e740 1 -- 172.21.15.117:0/1127216611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566ae4ba4d0 msgr2=0x5566ae886370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.898+0000 7f173083e740 1 --2- 172.21.15.117:0/1127216611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566ae4ba4d0 0x5566ae886370 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f1718006f90 tx=0x7f1718009320 comp rx=0 tx=0).stop 2024-09-21T06:03:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.899+0000 7f173083e740 1 -- 172.21.15.117:0/1127216611 shutdown_connections 2024-09-21T06:03:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.899+0000 7f173083e740 1 --2- 172.21.15.117:0/1127216611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566ae4ba4d0 0x5566ae886370 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.899+0000 7f173083e740 1 --2- 172.21.15.117:0/1127216611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ae4db1b0 0x5566ae898510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.899+0000 7f173083e740 1 --2- 172.21.15.117:0/1127216611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566ae4b22f0 0x5566ae88c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.899+0000 7f173083e740 1 -- 172.21.15.117:0/1127216611 >> 172.21.15.117:0/1127216611 conn(0x5566ae4cf300 msgr2=0x5566ae885c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.899+0000 7f173083e740 1 -- 172.21.15.117:0/1127216611 shutdown_connections 2024-09-21T06:03:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.899+0000 7f173083e740 1 -- 172.21.15.117:0/1127216611 wait complete. 2024-09-21T06:03:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.900+0000 7f173083e740 1 -- start start 2024-09-21T06:03:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.900+0000 7f173083e740 1 Processor -- start 2024-09-21T06:03:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.901+0000 7f173083e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566ae4b22f0 0x5566ae88f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:18.904 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.901+0000 7f173083e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566ae4ba4d0 0x5566ae88feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.901+0000 7f173083e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ae4db1b0 0x5566ae906ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.901+0000 7f173083e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f17241b2a90 con 0x5566ae4ba4d0 2024-09-21T06:03:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.901+0000 7f173083e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f17241b23c0 con 0x5566ae4db1b0 2024-09-21T06:03:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.901+0000 7f173083e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f17241af600 con 0x5566ae4b22f0 2024-09-21T06:03:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.901+0000 7f17227fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566ae4ba4d0 0x5566ae88feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.901+0000 7f173083e740 10 client.0 _ll_register_callbacks cb 0x7f171c009c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:03:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.901+0000 7f17227fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566ae4ba4d0 0x5566ae88feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49944/0 (socket says 172.21.15.117:49944) 2024-09-21T06:03:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.901+0000 7f17227fc640 1 -- 172.21.15.117:0/58063216 learned_addr learned my addr 172.21.15.117:0/58063216 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.902+0000 7f17227fc640 1 -- 172.21.15.117:0/58063216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566ae4b22f0 msgr2=0x5566ae88f970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:03:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.902+0000 7f17227fc640 1 --2- 172.21.15.117:0/58063216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566ae4b22f0 0x5566ae88f970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.902+0000 7f17227fc640 1 -- 172.21.15.117:0/58063216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ae4db1b0 msgr2=0x5566ae906ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:03:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.902+0000 7f17227fc640 1 --2- 172.21.15.117:0/58063216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ae4db1b0 0x5566ae906ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.902+0000 7f17227fc640 1 -- 172.21.15.117:0/58063216 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566ae890750 con 0x5566ae4ba4d0 2024-09-21T06:03:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.902+0000 7f17227fc640 1 --2- 172.21.15.117:0/58063216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566ae4ba4d0 0x5566ae88feb0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f17180038f0 tx=0x7f171800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad68c10cbb50359 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.902+0000 7f1702ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:03:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.903+0000 7f1702ffd640 1 -- 172.21.15.117:0/58063216 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718015050 con 0x5566ae4ba4d0 2024-09-21T06:03:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.903+0000 7f173083e740 1 -- 172.21.15.117:0/58063216 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f171800bd80 con 0x5566ae4ba4d0 2024-09-21T06:03:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.904+0000 7f1702ffd640 1 -- 172.21.15.117:0/58063216 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f1718003b80 con 0x5566ae4ba4d0 2024-09-21T06:03:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.904+0000 7f1702ffd640 1 -- 172.21.15.117:0/58063216 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1718003de0 con 0x5566ae4ba4d0 2024-09-21T06:03:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.904+0000 7f173083e740 10 client.16503 fetch_fsmap learned FSMap version 27 2024-09-21T06:03:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.904+0000 7f173083e740 1 -- 172.21.15.117:0/58063216 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5566ae9083f0 con 0x5566ae4ba4d0 2024-09-21T06:03:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.906+0000 7f1702ffd640 1 -- 172.21.15.117:0/58063216 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f171800bd80 con 0x5566ae4ba4d0 2024-09-21T06:03:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.906+0000 7f1702ffd640 1 -- 172.21.15.117:0/58063216 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f17180249b0 con 0x5566ae4ba4d0 2024-09-21T06:03:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.906+0000 7f173083e740 10 client.16503 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:03:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.906+0000 7f173083e740 10 client.16503 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:03:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.906+0000 7f173083e740 20 client.16503 populate_metadata read hostname 'smithi117' 2024-09-21T06:03:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.906+0000 7f173083e740 1 --2- 172.21.15.117:0/58063216 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5566ae46f1c0 0x5566ae90a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.906+0000 7f173083e740 4 client.16503 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:03:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.906+0000 7f173083e740 1 -- 172.21.15.117:0/58063216 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5566ae7f30f0 con 0x5566ae46f1c0 2024-09-21T06:03:18.909 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.906+0000 7f1722ffd640 1 --2- 172.21.15.117:0/58063216 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5566ae46f1c0 0x5566ae90a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:18.910 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.908+0000 7f1702ffd640 10 client.16503 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:03:18.910 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:18.908+0000 7f1722ffd640 1 --2- 172.21.15.117:0/58063216 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5566ae46f1c0 0x5566ae90a820 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=679aabae98919e75 server_cookie=5a85e46d2e99739b in_seq=0 out_seq=0 2024-09-21T06:03:20.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:19 smithi192 ceph-mon[34700]: pgmap v507: 97 pgs: 97 active+clean; 54 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 632 MiB/s wr, 4.05k op/s 2024-09-21T06:03:20.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:19 smithi117 ceph-mon[30947]: pgmap v507: 97 pgs: 97 active+clean; 54 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 632 MiB/s wr, 4.05k op/s 2024-09-21T06:03:20.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:19 smithi138 ceph-mon[34391]: pgmap v507: 97 pgs: 97 active+clean; 54 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 632 MiB/s wr, 4.05k op/s 2024-09-21T06:03:21.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:20 smithi192 ceph-mon[34700]: pgmap v508: 97 pgs: 97 active+clean; 55 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 649 MiB/s wr, 4.04k op/s 2024-09-21T06:03:21.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:20 smithi117 ceph-mon[30947]: pgmap v508: 97 pgs: 97 active+clean; 55 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 649 MiB/s wr, 4.04k op/s 2024-09-21T06:03:21.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:20 smithi138 ceph-mon[34391]: pgmap v508: 97 pgs: 97 active+clean; 55 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 649 MiB/s wr, 4.04k op/s 2024-09-21T06:03:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:22.380+0000 7f1702ffd640 1 -- 172.21.15.117:0/58063216 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5566ae7f30f0 con 0x5566ae46f1c0 2024-09-21T06:03:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:22.380+0000 7f1702ffd640 10 client.16503 handle_command_reply: tid=0 2024-09-21T06:03:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:22.380+0000 7f173083e740 1 client.16503 shutdown 2024-09-21T06:03:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:22.385+0000 7f173083e740 1 -- 172.21.15.117:0/58063216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566ae4ba4d0 msgr2=0x5566ae88feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:22.385+0000 7f173083e740 1 --2- 172.21.15.117:0/58063216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566ae4ba4d0 0x5566ae88feb0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f17180038f0 tx=0x7f171800b8d0 comp rx=0 tx=0).stop 2024-09-21T06:03:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:22.389+0000 7f173083e740 1 -- 172.21.15.117:0/58063216 shutdown_connections 2024-09-21T06:03:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:22.389+0000 7f173083e740 1 --2- 172.21.15.117:0/58063216 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5566ae46f1c0 0x5566ae90a820 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:22.389+0000 7f173083e740 1 --2- 172.21.15.117:0/58063216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5566ae4ba4d0 0x5566ae88feb0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:22.389+0000 7f173083e740 1 --2- 172.21.15.117:0/58063216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5566ae4db1b0 0x5566ae906ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:22.389+0000 7f173083e740 1 --2- 172.21.15.117:0/58063216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566ae4b22f0 0x5566ae88f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:22.389+0000 7f173083e740 1 -- 172.21.15.117:0/58063216 >> 172.21.15.117:0/58063216 conn(0x5566ae4cf300 msgr2=0x5566ae88b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:22.389+0000 7f1702ffd640 0 client.16503 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:03:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:22.389+0000 7f173083e740 1 -- 172.21.15.117:0/58063216 shutdown_connections 2024-09-21T06:03:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:22.389+0000 7f173083e740 1 -- 172.21.15.117:0/58063216 wait complete. 2024-09-21T06:03:22.394 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:22.392+0000 7f173083e740 20 client.16503 trim_cache size 0 max 16384 2024-09-21T06:03:22.394 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T06:03:22.395 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T06:03:22.395 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T06:03:22.395 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T06:03:22.400 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:22.398+0000 7f1723fff640 1 -- 172.21.15.117:0/2283444582 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f170c0b6700 msgr2=0x7f170c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:22.400 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:22.398+0000 7f1723fff640 1 --2- 172.21.15.117:0/2283444582 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f170c0b6700 0x7f170c0b8c10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f171c005fd0 tx=0x7f171c009600 comp rx=0 tx=0).stop 2024-09-21T06:03:22.400 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:22.398+0000 7f1723fff640 1 -- 172.21.15.117:0/2283444582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1724101f90 msgr2=0x7f172419a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:22.400 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:22.398+0000 7f1723fff640 1 --2- 172.21.15.117:0/2283444582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1724101f90 0x7f172419a2b0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f1714006010 tx=0x7f17140045e0 comp rx=0 tx=0).stop 2024-09-21T06:03:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:22.402+0000 7f1723fff640 1 -- 172.21.15.117:0/2283444582 shutdown_connections 2024-09-21T06:03:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:22.402+0000 7f1723fff640 1 --2- 172.21.15.117:0/2283444582 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f170c0b6700 0x7f170c0b8c10 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:22.402+0000 7f1723fff640 1 --2- 172.21.15.117:0/2283444582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1724108210 0x7f172419fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:22.402+0000 7f1723fff640 1 --2- 172.21.15.117:0/2283444582 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17241078c0 0x7f172419a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:22.402+0000 7f1723fff640 1 --2- 172.21.15.117:0/2283444582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1724101f90 0x7f172419a2b0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:22.402+0000 7f1723fff640 1 -- 172.21.15.117:0/2283444582 >> 172.21.15.117:0/2283444582 conn(0x7f1724079b00 msgr2=0x7f1724105910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:22.405+0000 7f1723fff640 1 -- 172.21.15.117:0/2283444582 shutdown_connections 2024-09-21T06:03:22.408 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:22.405+0000 7f1723fff640 1 -- 172.21.15.117:0/2283444582 wait complete. 2024-09-21T06:03:24.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:23 smithi138 ceph-mon[34391]: pgmap v509: 97 pgs: 97 active+clean; 56 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 727 MiB/s wr, 4.03k op/s 2024-09-21T06:03:24.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:23 smithi192 ceph-mon[34700]: pgmap v509: 97 pgs: 97 active+clean; 56 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 727 MiB/s wr, 4.03k op/s 2024-09-21T06:03:24.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:23 smithi117 ceph-mon[30947]: pgmap v509: 97 pgs: 97 active+clean; 56 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 727 MiB/s wr, 4.03k op/s 2024-09-21T06:03:26.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:25 smithi138 ceph-mon[34391]: pgmap v510: 97 pgs: 97 active+clean; 57 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 820 MiB/s wr, 3.94k op/s 2024-09-21T06:03:26.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:25 smithi192 ceph-mon[34700]: pgmap v510: 97 pgs: 97 active+clean; 57 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 820 MiB/s wr, 3.94k op/s 2024-09-21T06:03:26.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:25 smithi117 ceph-mon[30947]: pgmap v510: 97 pgs: 97 active+clean; 57 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 820 MiB/s wr, 3.94k op/s 2024-09-21T06:03:28.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:27 smithi138 ceph-mon[34391]: pgmap v511: 97 pgs: 97 active+clean; 57 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 841 MiB/s wr, 3.45k op/s 2024-09-21T06:03:28.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:27 smithi192 ceph-mon[34700]: pgmap v511: 97 pgs: 97 active+clean; 57 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 841 MiB/s wr, 3.45k op/s 2024-09-21T06:03:28.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:27 smithi117 ceph-mon[30947]: pgmap v511: 97 pgs: 97 active+clean; 57 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 841 MiB/s wr, 3.45k op/s 2024-09-21T06:03:30.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:29 smithi192 ceph-mon[34700]: pgmap v512: 97 pgs: 97 active+clean; 58 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 887 MiB/s wr, 3.03k op/s 2024-09-21T06:03:30.280 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:29 smithi117 ceph-mon[30947]: pgmap v512: 97 pgs: 97 active+clean; 58 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 887 MiB/s wr, 3.03k op/s 2024-09-21T06:03:30.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:29 smithi138 ceph-mon[34391]: pgmap v512: 97 pgs: 97 active+clean; 58 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 887 MiB/s wr, 3.03k op/s 2024-09-21T06:03:32.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:31 smithi192 ceph-mon[34700]: pgmap v513: 97 pgs: 97 active+clean; 59 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 840 MiB/s wr, 2.16k op/s 2024-09-21T06:03:32.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:31 smithi117 ceph-mon[30947]: pgmap v513: 97 pgs: 97 active+clean; 59 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 840 MiB/s wr, 2.16k op/s 2024-09-21T06:03:32.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:31 smithi138 ceph-mon[34391]: pgmap v513: 97 pgs: 97 active+clean; 59 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 840 MiB/s wr, 2.16k op/s 2024-09-21T06:03:34.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:33 smithi192 ceph-mon[34700]: pgmap v514: 97 pgs: 97 active+clean; 59 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 861 MiB/s wr, 1.99k op/s 2024-09-21T06:03:34.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:33 smithi117 ceph-mon[30947]: pgmap v514: 97 pgs: 97 active+clean; 59 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 861 MiB/s wr, 1.99k op/s 2024-09-21T06:03:34.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:33 smithi138 ceph-mon[34391]: pgmap v514: 97 pgs: 97 active+clean; 59 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 861 MiB/s wr, 1.99k op/s 2024-09-21T06:03:35.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:34 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:03:35.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:34 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:03:35.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:34 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:03:35.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:34 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:03:35.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:34 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T06:03:35.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:34 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:35.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:34 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:03:35.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:34 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:03:35.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:34 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:03:35.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:34 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:03:35.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:34 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:03:35.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:34 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T06:03:35.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:34 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:35.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:34 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:03:35.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:34 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:03:35.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:34 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T06:03:35.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:34 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T06:03:35.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:34 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T06:03:35.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:34 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T06:03:35.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:34 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:03:35.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:34 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:03:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.781+0000 7f0e82361640 1 -- 172.21.15.117:0/482374454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e7c11cba0 msgr2=0x7f0e7c11efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.781+0000 7f0e82361640 1 --2- 172.21.15.117:0/482374454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e7c11cba0 0x7f0e7c11efc0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009860 tx=0x7f0e6c02a340 comp rx=0 tx=0).stop 2024-09-21T06:03:35.792 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.790+0000 7f0e82361640 1 -- 172.21.15.117:0/482374454 shutdown_connections 2024-09-21T06:03:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.790+0000 7f0e82361640 1 --2- 172.21.15.117:0/482374454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e7c11cba0 0x7f0e7c11efc0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.790+0000 7f0e82361640 1 --2- 172.21.15.117:0/482374454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e7c10a900 0x7f0e7c10ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.790+0000 7f0e82361640 1 --2- 172.21.15.117:0/482374454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e7c111ca0 0x7f0e7c10a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.790+0000 7f0e82361640 1 -- 172.21.15.117:0/482374454 >> 172.21.15.117:0/482374454 conn(0x7f0e7c070c00 msgr2=0x7f0e7c071010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:35.798 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.794+0000 7f0e82361640 1 -- 172.21.15.117:0/482374454 shutdown_connections 2024-09-21T06:03:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.794+0000 7f0e82361640 1 -- 172.21.15.117:0/482374454 wait complete. 2024-09-21T06:03:35.799 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.796+0000 7f0e82361640 1 Processor -- start 2024-09-21T06:03:35.801 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.796+0000 7f0e82361640 1 -- start start 2024-09-21T06:03:35.801 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.797+0000 7f0e82361640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e7c10a900 0x7f0e7c1a90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:35.802 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.797+0000 7f0e82361640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e7c111ca0 0x7f0e7c1a9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:35.802 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.797+0000 7f0e82361640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e7c11cba0 0x7f0e7c1ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:35.802 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.797+0000 7f0e82361640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c122660 con 0x7f0e7c11cba0 2024-09-21T06:03:35.802 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.797+0000 7f0e82361640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c1224e0 con 0x7f0e7c111ca0 2024-09-21T06:03:35.802 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.797+0000 7f0e82361640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c1227e0 con 0x7f0e7c10a900 2024-09-21T06:03:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.798+0000 7f0e81b60640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e7c11cba0 0x7f0e7c1ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.798+0000 7f0e81b60640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e7c11cba0 0x7f0e7c1ae8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50268/0 (socket says 172.21.15.117:50268) 2024-09-21T06:03:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.798+0000 7f0e81b60640 1 -- 172.21.15.117:0/1955185177 learned_addr learned my addr 172.21.15.117:0/1955185177 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.798+0000 7f0e80b5e640 1 --2- 172.21.15.117:0/1955185177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e7c111ca0 0x7f0e7c1a9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.799+0000 7f0e81b60640 1 -- 172.21.15.117:0/1955185177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e7c10a900 msgr2=0x7f0e7c1a90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.798+0000 7f0e8135f640 1 --2- 172.21.15.117:0/1955185177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e7c10a900 0x7f0e7c1a90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.799+0000 7f0e81b60640 1 --2- 172.21.15.117:0/1955185177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e7c10a900 0x7f0e7c1a90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.799+0000 7f0e81b60640 1 -- 172.21.15.117:0/1955185177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e7c111ca0 msgr2=0x7f0e7c1a9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.799+0000 7f0e81b60640 1 --2- 172.21.15.117:0/1955185177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e7c111ca0 0x7f0e7c1a9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.799+0000 7f0e81b60640 1 -- 172.21.15.117:0/1955185177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e7c1aeff0 con 0x7f0e7c11cba0 2024-09-21T06:03:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.799+0000 7f0e80b5e640 1 --2- 172.21.15.117:0/1955185177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e7c111ca0 0x7f0e7c1a9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:03:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.799+0000 7f0e8135f640 1 --2- 172.21.15.117:0/1955185177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e7c10a900 0x7f0e7c1a90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:03:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.800+0000 7f0e81b60640 1 --2- 172.21.15.117:0/1955185177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e7c11cba0 0x7f0e7c1ae8f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c02a930 tx=0x7f0e6c032d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.803+0000 7f0e727fc640 1 -- 172.21.15.117:0/1955185177 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c037040 con 0x7f0e7c11cba0 2024-09-21T06:03:35.807 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.803+0000 7f0e82361640 1 -- 172.21.15.117:0/1955185177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e7c1af280 con 0x7f0e7c11cba0 2024-09-21T06:03:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.803+0000 7f0e82361640 1 -- 172.21.15.117:0/1955185177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e7c1af710 con 0x7f0e7c11cba0 2024-09-21T06:03:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.804+0000 7f0e727fc640 1 -- 172.21.15.117:0/1955185177 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e6c004050 con 0x7f0e7c11cba0 2024-09-21T06:03:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.804+0000 7f0e727fc640 1 -- 172.21.15.117:0/1955185177 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c005020 con 0x7f0e7c11cba0 2024-09-21T06:03:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.815+0000 7f0e8a60b740 1 Processor -- start 2024-09-21T06:03:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.817+0000 7f0e8a60b740 1 -- start start 2024-09-21T06:03:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.817+0000 7f0e727fc640 1 -- 172.21.15.117:0/1955185177 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f0e6c043030 con 0x7f0e7c11cba0 2024-09-21T06:03:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.817+0000 7f0e8a60b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a881dc1b0 0x562a88581ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.817+0000 7f0e8a60b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a881b32f0 0x562a885820e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.817+0000 7f0e8a60b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562a881bb4d0 0x562a88582620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.817+0000 7f0e8a60b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562a884f66f0 con 0x562a881b32f0 2024-09-21T06:03:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.817+0000 7f0e8a60b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562a884e5b20 con 0x562a881dc1b0 2024-09-21T06:03:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.817+0000 7f0e8a60b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x562a884e9930 con 0x562a881bb4d0 2024-09-21T06:03:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.819+0000 7f0e727fc640 1 --2- 172.21.15.117:0/1955185177 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0e680b6850 0x7f0e680b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.820+0000 7f0e8135f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a881b32f0 0x562a885820e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.820+0000 7f0e8135f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a881b32f0 0x562a885820e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50282/0 (socket says 172.21.15.117:50282) 2024-09-21T06:03:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.820+0000 7f0e8135f640 1 -- 172.21.15.117:0/3516898275 learned_addr learned my addr 172.21.15.117:0/3516898275 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.820+0000 7f0e80b5e640 1 --2- 172.21.15.117:0/3516898275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562a881bb4d0 0x562a88582620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.820+0000 7f0e727fc640 1 -- 172.21.15.117:0/1955185177 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f0e6c106c10 con 0x7f0e7c11cba0 2024-09-21T06:03:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.820+0000 7f0e8135f640 1 -- 172.21.15.117:0/3516898275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562a881bb4d0 msgr2=0x562a88582620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.820+0000 7f0e8135f640 1 --2- 172.21.15.117:0/3516898275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562a881bb4d0 0x562a88582620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.820+0000 7f0e8135f640 1 -- 172.21.15.117:0/3516898275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a881dc1b0 msgr2=0x562a88581ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.820+0000 7f0e8135f640 1 --2- 172.21.15.117:0/3516898275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a881dc1b0 0x562a88581ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.820+0000 7f0e8135f640 1 -- 172.21.15.117:0/3516898275 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a884ab530 con 0x562a881b32f0 2024-09-21T06:03:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.821+0000 7f0e81b60640 1 --2- 172.21.15.117:0/1955185177 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0e680b6850 0x7f0e680b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.822+0000 7f0e81b60640 1 --2- 172.21.15.117:0/1955185177 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0e680b6850 0x7f0e680b8d60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c0cda00 tx=0x7f0e6c004500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:35.827 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.823+0000 7f0e8135f640 1 --2- 172.21.15.117:0/3516898275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a881b32f0 0x562a885820e0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009c50 tx=0x7f0e78010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b1fbf76d62d600 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.829+0000 7f0e677fe640 1 -- 172.21.15.117:0/3516898275 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78004020 con 0x562a881b32f0 2024-09-21T06:03:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.830+0000 7f0e677fe640 1 -- 172.21.15.117:0/3516898275 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e780041c0 con 0x562a881b32f0 2024-09-21T06:03:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.832+0000 7f0e8a60b740 1 -- 172.21.15.117:0/3516898275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a881b32f0 msgr2=0x562a885820e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.832+0000 7f0e8a60b740 1 --2- 172.21.15.117:0/3516898275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a881b32f0 0x562a885820e0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009c50 tx=0x7f0e78010740 comp rx=0 tx=0).stop 2024-09-21T06:03:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.832+0000 7f0e8a60b740 1 -- 172.21.15.117:0/3516898275 shutdown_connections 2024-09-21T06:03:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.832+0000 7f0e8a60b740 1 --2- 172.21.15.117:0/3516898275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a881b32f0 0x562a885820e0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.832+0000 7f0e8a60b740 1 --2- 172.21.15.117:0/3516898275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562a881bb4d0 0x562a88582620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.832+0000 7f0e8a60b740 1 --2- 172.21.15.117:0/3516898275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a881dc1b0 0x562a88581ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.832+0000 7f0e8a60b740 1 -- 172.21.15.117:0/3516898275 >> 172.21.15.117:0/3516898275 conn(0x562a881d0300 msgr2=0x562a8857fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.832+0000 7f0e8a60b740 1 -- 172.21.15.117:0/3516898275 shutdown_connections 2024-09-21T06:03:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.833+0000 7f0e8a60b740 1 -- 172.21.15.117:0/3516898275 wait complete. 2024-09-21T06:03:35.838 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.834+0000 7f0e8a60b740 1 -- start start 2024-09-21T06:03:35.838 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.835+0000 7f0e8a60b740 1 Processor -- start 2024-09-21T06:03:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.836+0000 7f0e8a60b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562a881b32f0 0x562a8858d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.836+0000 7f0e8a60b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a881bb4d0 0x562a8858d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.836+0000 7f0e8a60b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a881dc1b0 0x562a8858dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.836+0000 7f0e8a60b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562a8841dab0 con 0x562a881dc1b0 2024-09-21T06:03:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.836+0000 7f0e8a60b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562a8848b730 con 0x562a881bb4d0 2024-09-21T06:03:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.836+0000 7f0e8a60b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x562a88590370 con 0x562a881b32f0 2024-09-21T06:03:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.836+0000 7f0e8a60b740 10 client.0 _ll_register_callbacks cb 0x562a884df6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:03:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.837+0000 7f0e80b5e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a881dc1b0 0x562a8858dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.837+0000 7f0e80b5e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a881dc1b0 0x562a8858dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50284/0 (socket says 172.21.15.117:50284) 2024-09-21T06:03:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.837+0000 7f0e80b5e640 1 -- 172.21.15.117:0/4073503807 learned_addr learned my addr 172.21.15.117:0/4073503807 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.837+0000 7f0e80b5e640 1 --2- 172.21.15.117:0/4073503807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562a881b32f0 0x562a8858d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.837+0000 7f0e80b5e640 1 -- 172.21.15.117:0/4073503807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562a881b32f0 msgr2=0x562a8858d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.837+0000 7f0e80b5e640 1 --2- 172.21.15.117:0/4073503807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562a881b32f0 0x562a8858d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.837+0000 7f0e80b5e640 1 -- 172.21.15.117:0/4073503807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a881bb4d0 msgr2=0x562a8858d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.837+0000 7f0e80b5e640 1 --2- 172.21.15.117:0/4073503807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a881bb4d0 0x562a8858d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.837+0000 7f0e80b5e640 1 -- 172.21.15.117:0/4073503807 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a88247790 con 0x562a881dc1b0 2024-09-21T06:03:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.838+0000 7f0e80b5e640 1 --2- 172.21.15.117:0/4073503807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a881dc1b0 0x562a8858dc50 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0e74010f80 tx=0x7f0e74009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddeb56a71362370b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.838+0000 7f0e64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:03:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.838+0000 7f0e64ff9640 1 -- 172.21.15.117:0/4073503807 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e740100d0 con 0x562a881dc1b0 2024-09-21T06:03:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.838+0000 7f0e8a60b740 1 -- 172.21.15.117:0/4073503807 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e780041c0 con 0x562a881dc1b0 2024-09-21T06:03:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.838+0000 7f0e64ff9640 1 -- 172.21.15.117:0/4073503807 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f0e74010270 con 0x562a881dc1b0 2024-09-21T06:03:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.839+0000 7f0e64ff9640 1 -- 172.21.15.117:0/4073503807 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0e74023050 con 0x562a881dc1b0 2024-09-21T06:03:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.839+0000 7f0e8a60b740 10 client.16515 fetch_fsmap learned FSMap version 27 2024-09-21T06:03:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.839+0000 7f0e8a60b740 1 -- 172.21.15.117:0/4073503807 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562a882c0620 con 0x562a881dc1b0 2024-09-21T06:03:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.839+0000 7f0e64ff9640 1 -- 172.21.15.117:0/4073503807 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e780041c0 con 0x562a881dc1b0 2024-09-21T06:03:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.840+0000 7f0e64ff9640 1 -- 172.21.15.117:0/4073503807 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0e7400b040 con 0x562a881dc1b0 2024-09-21T06:03:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.840+0000 7f0e8a60b740 10 client.16515 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:03:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.840+0000 7f0e8a60b740 10 client.16515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:03:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.840+0000 7f0e8a60b740 20 client.16515 populate_metadata read hostname 'smithi117' 2024-09-21T06:03:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.840+0000 7f0e8a60b740 1 --2- 172.21.15.117:0/4073503807 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x562a8819ad80 0x562a8859f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.840+0000 7f0e8a60b740 4 client.16515 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:03:35.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.840+0000 7f0e8a60b740 1 -- 172.21.15.117:0/4073503807 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a884f8640 con 0x562a8819ad80 2024-09-21T06:03:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.843+0000 7f0e8135f640 1 --2- 172.21.15.117:0/4073503807 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x562a8819ad80 0x562a8859f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.850+0000 7f0e8135f640 1 --2- 172.21.15.117:0/4073503807 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x562a8819ad80 0x562a8859f510 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b58b2a6cb0f76fa server_cookie=11145d80dbbc5d76 in_seq=0 out_seq=0 2024-09-21T06:03:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.850+0000 7f0e64ff9640 10 client.16515 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:03:35.857 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.854+0000 7f0e64ff9640 1 -- 172.21.15.117:0/4073503807 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562a884f8640 con 0x562a8819ad80 2024-09-21T06:03:35.857 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.854+0000 7f0e64ff9640 10 client.16515 handle_command_reply: tid=0 2024-09-21T06:03:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.854+0000 7f0e8a60b740 1 client.16515 shutdown 2024-09-21T06:03:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.854+0000 7f0e8a60b740 1 -- 172.21.15.117:0/4073503807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a881dc1b0 msgr2=0x562a8858dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:35.858 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.854+0000 7f0e8a60b740 1 --2- 172.21.15.117:0/4073503807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a881dc1b0 0x562a8858dc50 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0e74010f80 tx=0x7f0e74009a20 comp rx=0 tx=0).stop 2024-09-21T06:03:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.854+0000 7f0e8a60b740 1 -- 172.21.15.117:0/4073503807 shutdown_connections 2024-09-21T06:03:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.854+0000 7f0e8a60b740 1 --2- 172.21.15.117:0/4073503807 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x562a8819ad80 0x562a8859f510 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.855+0000 7f0e8a60b740 1 --2- 172.21.15.117:0/4073503807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a881dc1b0 0x562a8858dc50 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.855+0000 7f0e8a60b740 1 --2- 172.21.15.117:0/4073503807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a881bb4d0 0x562a8858d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.855+0000 7f0e8a60b740 1 --2- 172.21.15.117:0/4073503807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562a881b32f0 0x562a8858d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.855+0000 7f0e64ff9640 0 client.16515 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:03:35.859 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.855+0000 7f0e8a60b740 1 -- 172.21.15.117:0/4073503807 >> 172.21.15.117:0/4073503807 conn(0x562a881d0300 msgr2=0x562a88582bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:35.860 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.855+0000 7f0e8a60b740 1 -- 172.21.15.117:0/4073503807 shutdown_connections 2024-09-21T06:03:35.860 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.855+0000 7f0e8a60b740 1 -- 172.21.15.117:0/4073503807 wait complete. 2024-09-21T06:03:35.860 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.855+0000 7f0e8a60b740 20 client.16515 trim_cache size 0 max 16384 2024-09-21T06:03:35.893 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.890+0000 7f0e8a60b740 1 Processor -- start 2024-09-21T06:03:35.893 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.890+0000 7f0e8a60b740 1 -- start start 2024-09-21T06:03:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.891+0000 7f0e8a60b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a881b32f0 0x562a8858ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.891+0000 7f0e8a60b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a881bb4d0 0x562a885a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.891+0000 7f0e8a60b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562a881dc1b0 0x562a885a5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.891+0000 7f0e8a60b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6000e670 con 0x562a881b32f0 2024-09-21T06:03:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.891+0000 7f0e8a60b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0e600101b0 con 0x562a881bb4d0 2024-09-21T06:03:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.891+0000 7f0e8a60b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6000ed40 con 0x562a881dc1b0 2024-09-21T06:03:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.892+0000 7f0e80b5e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a881b32f0 0x562a8858ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.892+0000 7f0e80b5e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a881b32f0 0x562a8858ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50286/0 (socket says 172.21.15.117:50286) 2024-09-21T06:03:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.892+0000 7f0e80b5e640 1 -- 172.21.15.117:0/610129069 learned_addr learned my addr 172.21.15.117:0/610129069 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.892+0000 7f0e80b5e640 1 --2- 172.21.15.117:0/610129069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562a881dc1b0 0x562a885a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.892+0000 7f0e8135f640 1 --2- 172.21.15.117:0/610129069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a881bb4d0 0x562a885a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:35.896 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.892+0000 7f0e80b5e640 1 -- 172.21.15.117:0/610129069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562a881dc1b0 msgr2=0x562a885a5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:35.897 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.893+0000 7f0e80b5e640 1 --2- 172.21.15.117:0/610129069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562a881dc1b0 0x562a885a5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:35.897 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.893+0000 7f0e80b5e640 1 -- 172.21.15.117:0/610129069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a881bb4d0 msgr2=0x562a885a4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:35.897 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.893+0000 7f0e80b5e640 1 --2- 172.21.15.117:0/610129069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a881bb4d0 0x562a885a4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:35.897 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.893+0000 7f0e80b5e640 1 -- 172.21.15.117:0/610129069 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a8858e470 con 0x562a881b32f0 2024-09-21T06:03:35.897 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.893+0000 7f0e80b5e640 1 --2- 172.21.15.117:0/610129069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a881b32f0 0x562a8858ede0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0e74009600 tx=0x7f0e74010870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e4cf910a5b412a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.894+0000 7f0e66ffd640 1 -- 172.21.15.117:0/610129069 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7400c580 con 0x562a881b32f0 2024-09-21T06:03:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.894+0000 7f0e66ffd640 1 -- 172.21.15.117:0/610129069 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e74004e50 con 0x562a881b32f0 2024-09-21T06:03:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.894+0000 7f0e66ffd640 1 -- 172.21.15.117:0/610129069 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e74004750 con 0x562a881b32f0 2024-09-21T06:03:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.895+0000 7f0e8a60b740 1 -- 172.21.15.117:0/610129069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a881b32f0 msgr2=0x562a8858ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.895+0000 7f0e8a60b740 1 --2- 172.21.15.117:0/610129069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a881b32f0 0x562a8858ede0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0e74009600 tx=0x7f0e74010870 comp rx=0 tx=0).stop 2024-09-21T06:03:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.896+0000 7f0e8a60b740 1 -- 172.21.15.117:0/610129069 shutdown_connections 2024-09-21T06:03:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.896+0000 7f0e8a60b740 1 --2- 172.21.15.117:0/610129069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562a881dc1b0 0x562a885a5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.896+0000 7f0e8a60b740 1 --2- 172.21.15.117:0/610129069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a881bb4d0 0x562a885a4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.896+0000 7f0e8a60b740 1 --2- 172.21.15.117:0/610129069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a881b32f0 0x562a8858ede0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.896+0000 7f0e8a60b740 1 -- 172.21.15.117:0/610129069 >> 172.21.15.117:0/610129069 conn(0x562a881d0300 msgr2=0x562a8858e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.896+0000 7f0e8a60b740 1 -- 172.21.15.117:0/610129069 shutdown_connections 2024-09-21T06:03:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.896+0000 7f0e8a60b740 1 -- 172.21.15.117:0/610129069 wait complete. 2024-09-21T06:03:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.897+0000 7f0e8a60b740 1 -- start start 2024-09-21T06:03:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.897+0000 7f0e8a60b740 1 Processor -- start 2024-09-21T06:03:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.899+0000 7f0e8a60b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a881b32f0 0x562a88595ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.899+0000 7f0e8a60b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562a881bb4d0 0x562a88593400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.899+0000 7f0e8a60b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a881dc1b0 0x562a885939a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.899+0000 7f0e8a60b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6000fae0 con 0x562a881dc1b0 2024-09-21T06:03:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.899+0000 7f0e8a60b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6000f410 con 0x562a881b32f0 2024-09-21T06:03:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.899+0000 7f0e8a60b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0e6000c650 con 0x562a881bb4d0 2024-09-21T06:03:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.899+0000 7f0e8a60b740 10 client.0 _ll_register_callbacks cb 0x7f0e74009d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:03:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.900+0000 7f0e80b5e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a881dc1b0 0x562a885939a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.901+0000 7f0e80b5e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a881dc1b0 0x562a885939a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50294/0 (socket says 172.21.15.117:50294) 2024-09-21T06:03:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.901+0000 7f0e80b5e640 1 -- 172.21.15.117:0/3067080877 learned_addr learned my addr 172.21.15.117:0/3067080877 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.901+0000 7f0e80b5e640 1 -- 172.21.15.117:0/3067080877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562a881bb4d0 msgr2=0x562a88593400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.901+0000 7f0e80b5e640 1 --2- 172.21.15.117:0/3067080877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562a881bb4d0 0x562a88593400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.901+0000 7f0e80b5e640 1 -- 172.21.15.117:0/3067080877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a881b32f0 msgr2=0x562a88595ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.901+0000 7f0e80b5e640 1 --2- 172.21.15.117:0/3067080877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a881b32f0 0x562a88595ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.901+0000 7f0e80b5e640 1 -- 172.21.15.117:0/3067080877 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a88594130 con 0x562a881dc1b0 2024-09-21T06:03:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.902+0000 7f0e80b5e640 1 --2- 172.21.15.117:0/3067080877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a881dc1b0 0x562a885939a0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0e74005d50 tx=0x7f0e74004870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f46f28c18a0c652 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.902+0000 7f0e64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:03:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.965+0000 7f0e64ff9640 1 -- 172.21.15.117:0/3067080877 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e74035020 con 0x562a881dc1b0 2024-09-21T06:03:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.965+0000 7f0e64ff9640 1 -- 172.21.15.117:0/3067080877 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f0e74004980 con 0x562a881dc1b0 2024-09-21T06:03:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.965+0000 7f0e8a60b740 1 -- 172.21.15.117:0/3067080877 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e74004e50 con 0x562a881dc1b0 2024-09-21T06:03:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.966+0000 7f0e64ff9640 1 -- 172.21.15.117:0/3067080877 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0e740102e0 con 0x562a881dc1b0 2024-09-21T06:03:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.966+0000 7f0e8a60b740 10 client.16521 fetch_fsmap learned FSMap version 27 2024-09-21T06:03:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.966+0000 7f0e8a60b740 1 -- 172.21.15.117:0/3067080877 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562a88609ef0 con 0x562a881dc1b0 2024-09-21T06:03:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.967+0000 7f0e64ff9640 1 -- 172.21.15.117:0/3067080877 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e74004e50 con 0x562a881dc1b0 2024-09-21T06:03:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.968+0000 7f0e64ff9640 1 -- 172.21.15.117:0/3067080877 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0e7400b070 con 0x562a881dc1b0 2024-09-21T06:03:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.968+0000 7f0e8a60b740 10 client.16521 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:03:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.968+0000 7f0e8a60b740 10 client.16521 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:03:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.968+0000 7f0e8a60b740 20 client.16521 populate_metadata read hostname 'smithi117' 2024-09-21T06:03:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.968+0000 7f0e8a60b740 1 --2- 172.21.15.117:0/3067080877 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x562a881701c0 0x562a8860c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.968+0000 7f0e8a60b740 4 client.16521 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:03:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.968+0000 7f0e8a60b740 1 -- 172.21.15.117:0/3067080877 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562a884f40f0 con 0x562a881701c0 2024-09-21T06:03:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.969+0000 7f0e8135f640 1 --2- 172.21.15.117:0/3067080877 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x562a881701c0 0x562a8860c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.980+0000 7f0e8135f640 1 --2- 172.21.15.117:0/3067080877 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x562a881701c0 0x562a8860c530 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e09e3daf67521a1f server_cookie=5261ec7e6c54732b in_seq=0 out_seq=0 2024-09-21T06:03:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:35.981+0000 7f0e64ff9640 10 client.16521 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:03:36.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:35 smithi192 ceph-mon[34700]: pgmap v515: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 905 MiB/s wr, 1.97k op/s 2024-09-21T06:03:36.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:35 smithi192 ceph-mon[34700]: Adjusting osd_memory_target on smithi117 to 176.0M 2024-09-21T06:03:36.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:35 smithi192 ceph-mon[34700]: Unable to set osd_memory_target on smithi117 to 184576000: error parsing value: Value '184576000' is below minimum 939524096 2024-09-21T06:03:36.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:03:36.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:35 smithi117 ceph-mon[30947]: pgmap v515: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 905 MiB/s wr, 1.97k op/s 2024-09-21T06:03:36.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:35 smithi117 ceph-mon[30947]: Adjusting osd_memory_target on smithi117 to 176.0M 2024-09-21T06:03:36.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:35 smithi117 ceph-mon[30947]: Unable to set osd_memory_target on smithi117 to 184576000: error parsing value: Value '184576000' is below minimum 939524096 2024-09-21T06:03:36.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:03:36.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:35 smithi138 ceph-mon[34391]: pgmap v515: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 905 MiB/s wr, 1.97k op/s 2024-09-21T06:03:36.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:35 smithi138 ceph-mon[34391]: Adjusting osd_memory_target on smithi117 to 176.0M 2024-09-21T06:03:36.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:36 smithi138 ceph-mon[34391]: Unable to set osd_memory_target on smithi117 to 184576000: error parsing value: Value '184576000' is below minimum 939524096 2024-09-21T06:03:36.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:36 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:03:38.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:37 smithi192 ceph-mon[34700]: pgmap v516: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 892 MiB/s wr, 1.82k op/s 2024-09-21T06:03:38.268 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:03:38.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:37 smithi117 ceph-mon[30947]: pgmap v516: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 892 MiB/s wr, 1.82k op/s 2024-09-21T06:03:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:38.336+0000 7f0e64ff9640 1 -- 172.21.15.117:0/3067080877 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562a884f40f0 con 0x562a881701c0 2024-09-21T06:03:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:38.336+0000 7f0e64ff9640 10 client.16521 handle_command_reply: tid=0 2024-09-21T06:03:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:38.336+0000 7f0e8a60b740 1 client.16521 shutdown 2024-09-21T06:03:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:38.336+0000 7f0e8a60b740 1 -- 172.21.15.117:0/3067080877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a881dc1b0 msgr2=0x562a885939a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:38.336+0000 7f0e8a60b740 1 --2- 172.21.15.117:0/3067080877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a881dc1b0 0x562a885939a0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0e74005d50 tx=0x7f0e74004870 comp rx=0 tx=0).stop 2024-09-21T06:03:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:38.336+0000 7f0e8a60b740 1 -- 172.21.15.117:0/3067080877 shutdown_connections 2024-09-21T06:03:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:38.336+0000 7f0e8a60b740 1 --2- 172.21.15.117:0/3067080877 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x562a881701c0 0x562a8860c530 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:38.336+0000 7f0e8a60b740 1 --2- 172.21.15.117:0/3067080877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562a881dc1b0 0x562a885939a0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:38.336+0000 7f0e8a60b740 1 --2- 172.21.15.117:0/3067080877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562a881bb4d0 0x562a88593400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:38.336+0000 7f0e8a60b740 1 --2- 172.21.15.117:0/3067080877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562a881b32f0 0x562a88595ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:38.336+0000 7f0e8a60b740 1 -- 172.21.15.117:0/3067080877 >> 172.21.15.117:0/3067080877 conn(0x562a881d0300 msgr2=0x562a8858e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:38.336+0000 7f0e64ff9640 0 client.16521 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:03:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:38.341+0000 7f0e8a60b740 1 -- 172.21.15.117:0/3067080877 shutdown_connections 2024-09-21T06:03:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:38.341+0000 7f0e8a60b740 1 -- 172.21.15.117:0/3067080877 wait complete. 2024-09-21T06:03:38.345 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:38.342+0000 7f0e8a60b740 20 client.16521 trim_cache size 0 max 16384 2024-09-21T06:03:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T06:03:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T06:03:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T06:03:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T06:03:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:38.343+0000 7f0e82361640 1 -- 172.21.15.117:0/1955185177 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0e680b6850 msgr2=0x7f0e680b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:38.344+0000 7f0e82361640 1 --2- 172.21.15.117:0/1955185177 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0e680b6850 0x7f0e680b8d60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c0cda00 tx=0x7f0e6c004500 comp rx=0 tx=0).stop 2024-09-21T06:03:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:38.344+0000 7f0e82361640 1 -- 172.21.15.117:0/1955185177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e7c11cba0 msgr2=0x7f0e7c1ae8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:38.344+0000 7f0e82361640 1 --2- 172.21.15.117:0/1955185177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e7c11cba0 0x7f0e7c1ae8f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c02a930 tx=0x7f0e6c032d40 comp rx=0 tx=0).stop 2024-09-21T06:03:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:38.344+0000 7f0e82361640 1 -- 172.21.15.117:0/1955185177 shutdown_connections 2024-09-21T06:03:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:38.344+0000 7f0e82361640 1 --2- 172.21.15.117:0/1955185177 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0e680b6850 0x7f0e680b8d60 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:38.344+0000 7f0e82361640 1 --2- 172.21.15.117:0/1955185177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e7c11cba0 0x7f0e7c1ae8f0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:38.344+0000 7f0e82361640 1 --2- 172.21.15.117:0/1955185177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e7c111ca0 0x7f0e7c1a9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:38.344+0000 7f0e82361640 1 --2- 172.21.15.117:0/1955185177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e7c10a900 0x7f0e7c1a90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:38.344+0000 7f0e82361640 1 -- 172.21.15.117:0/1955185177 >> 172.21.15.117:0/1955185177 conn(0x7f0e7c070c00 msgr2=0x7f0e7c11acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:38.352 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:38.349+0000 7f0e82361640 1 -- 172.21.15.117:0/1955185177 shutdown_connections 2024-09-21T06:03:38.353 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:38.350+0000 7f0e82361640 1 -- 172.21.15.117:0/1955185177 wait complete. 2024-09-21T06:03:38.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:37 smithi138 ceph-mon[34391]: pgmap v516: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 892 MiB/s wr, 1.82k op/s 2024-09-21T06:03:39.056 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:40.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:39 smithi192 ceph-mon[34700]: pgmap v517: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 927 MiB/s wr, 1.78k op/s 2024-09-21T06:03:40.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:39 smithi117 ceph-mon[30947]: pgmap v517: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 927 MiB/s wr, 1.78k op/s 2024-09-21T06:03:40.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:39 smithi138 ceph-mon[34391]: pgmap v517: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 927 MiB/s wr, 1.78k op/s 2024-09-21T06:03:40.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.561+0000 7f35eb141640 1 -- 172.21.15.117:0/1600931989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35e410fad0 msgr2=0x7f35e410fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:40.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.561+0000 7f35eb141640 1 --2- 172.21.15.117:0/1600931989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35e410fad0 0x7f35e410fee0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009900 tx=0x7f35d002a420 comp rx=0 tx=0).stop 2024-09-21T06:03:40.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.561+0000 7f35eb141640 1 -- 172.21.15.117:0/1600931989 shutdown_connections 2024-09-21T06:03:40.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.561+0000 7f35eb141640 1 --2- 172.21.15.117:0/1600931989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35e410fad0 0x7f35e410fee0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.561+0000 7f35eb141640 1 --2- 172.21.15.117:0/1600931989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35e4110420 0x7f35e4116c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.561+0000 7f35eb141640 1 --2- 172.21.15.117:0/1600931989 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35e410a190 0x7f35e410a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.561+0000 7f35eb141640 1 -- 172.21.15.117:0/1600931989 >> 172.21.15.117:0/1600931989 conn(0x7f35e40780f0 msgr2=0x7f35e41081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:40.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.561+0000 7f35eb141640 1 -- 172.21.15.117:0/1600931989 shutdown_connections 2024-09-21T06:03:40.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.561+0000 7f35eb141640 1 -- 172.21.15.117:0/1600931989 wait complete. 2024-09-21T06:03:40.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.562+0000 7f35eb141640 1 Processor -- start 2024-09-21T06:03:40.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.562+0000 7f35eb141640 1 -- start start 2024-09-21T06:03:40.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.562+0000 7f35eb141640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35e410a190 0x7f35e4083d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:40.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.562+0000 7f35eb141640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35e410fad0 0x7f35e407cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:40.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.562+0000 7f35eb141640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35e4110420 0x7f35e407d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:40.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.562+0000 7f35eb141640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f35e411a1a0 con 0x7f35e410a190 2024-09-21T06:03:40.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.562+0000 7f35eb141640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f35e411a020 con 0x7f35e410fad0 2024-09-21T06:03:40.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.562+0000 7f35eb141640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f35e411a320 con 0x7f35e4110420 2024-09-21T06:03:40.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.563+0000 7f35e8edd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35e410a190 0x7f35e4083d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:40.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.563+0000 7f35e8edd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35e410a190 0x7f35e4083d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50316/0 (socket says 172.21.15.117:50316) 2024-09-21T06:03:40.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.563+0000 7f35e8edd640 1 -- 172.21.15.117:0/3827257984 learned_addr learned my addr 172.21.15.117:0/3827257984 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:40.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.564+0000 7f35e8edd640 1 -- 172.21.15.117:0/3827257984 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35e4110420 msgr2=0x7f35e407d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:40.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.564+0000 7f35e8edd640 1 --2- 172.21.15.117:0/3827257984 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35e4110420 0x7f35e407d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.564+0000 7f35e8edd640 1 -- 172.21.15.117:0/3827257984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35e410fad0 msgr2=0x7f35e407cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:40.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.564+0000 7f35e8edd640 1 --2- 172.21.15.117:0/3827257984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35e410fad0 0x7f35e407cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.564+0000 7f35e8edd640 1 -- 172.21.15.117:0/3827257984 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e407de00 con 0x7f35e410a190 2024-09-21T06:03:40.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.564+0000 7f35e8edd640 1 --2- 172.21.15.117:0/3827257984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35e410a190 0x7f35e4083d10 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f35d800b450 tx=0x7f35d800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:40.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.565+0000 7f35e1ffb640 1 -- 172.21.15.117:0/3827257984 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d8015020 con 0x7f35e410a190 2024-09-21T06:03:40.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.565+0000 7f35eb141640 1 -- 172.21.15.117:0/3827257984 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e4082aa0 con 0x7f35e410a190 2024-09-21T06:03:40.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.565+0000 7f35e1ffb640 1 -- 172.21.15.117:0/3827257984 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35d8011d40 con 0x7f35e410a190 2024-09-21T06:03:40.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.565+0000 7f35eb141640 1 -- 172.21.15.117:0/3827257984 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e40830b0 con 0x7f35e410a190 2024-09-21T06:03:40.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.568+0000 7f35e1ffb640 1 -- 172.21.15.117:0/3827257984 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d800f340 con 0x7f35e410a190 2024-09-21T06:03:40.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.571+0000 7f35f16b5740 1 Processor -- start 2024-09-21T06:03:40.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.571+0000 7f35f16b5740 1 -- start start 2024-09-21T06:03:40.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.571+0000 7f35f16b5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597ef268070 0x5597ef60f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:40.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.571+0000 7f35f16b5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597ef23f1b0 0x5597ef60f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:40.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.571+0000 7f35f16b5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597ef247390 0x5597ef60fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:40.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.572+0000 7f35f16b5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5597ef585e50 con 0x5597ef268070 2024-09-21T06:03:40.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.572+0000 7f35f16b5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5597ef575280 con 0x5597ef23f1b0 2024-09-21T06:03:40.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.572+0000 7f35f16b5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5597ef579090 con 0x5597ef247390 2024-09-21T06:03:40.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.572+0000 7f35e96de640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597ef268070 0x5597ef60f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:40.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.572+0000 7f35e96de640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597ef268070 0x5597ef60f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50324/0 (socket says 172.21.15.117:50324) 2024-09-21T06:03:40.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.572+0000 7f35e96de640 1 -- 172.21.15.117:0/4190224272 learned_addr learned my addr 172.21.15.117:0/4190224272 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:40.846 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T06:03:40.846 INFO:teuthology.orchestra.run.smithi117.stdout: "status": "no active scrubs running", 2024-09-21T06:03:40.846 INFO:teuthology.orchestra.run.smithi117.stdout: "scrubs": {}, 2024-09-21T06:03:40.847 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_stats": { 2024-09-21T06:03:40.847 INFO:teuthology.orchestra.run.smithi117.stdout: "29ff9b7c-49d9-4910-85be-bace7e940cd5": { 2024-09-21T06:03:40.847 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:03:03.775527+0000", 2024-09-21T06:03:40.847 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:03:40.847 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:03:40.847 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:03:40.847 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:03:40.847 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:03:40.847 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:03:40.847 INFO:teuthology.orchestra.run.smithi117.stdout: "d543b0ae-b710-4153-99da-fe5714839c1a": { 2024-09-21T06:03:40.847 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:01:20.574754+0000", 2024-09-21T06:03:40.847 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:03:40.847 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:03:40.847 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:03:40.848 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:03:40.848 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:03:40.848 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:03:40.848 INFO:teuthology.orchestra.run.smithi117.stdout: "ad80d9f1-6aff-46c0-91ab-2c1ab0681039": { 2024-09-21T06:03:40.848 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:59:31.949108+0000", 2024-09-21T06:03:40.848 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:03:40.848 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:03:40.848 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:03:40.848 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:03:40.848 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:03:40.848 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:03:40.848 INFO:teuthology.orchestra.run.smithi117.stdout: "8b3fe95a-b76b-4e94-afd0-72fe35e2dd92": { 2024-09-21T06:03:40.848 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:58:01.246076+0000", 2024-09-21T06:03:40.848 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:03:40.848 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:03:40.849 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:03:40.849 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:03:40.849 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:03:40.849 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:03:40.849 INFO:teuthology.orchestra.run.smithi117.stdout: "b30bfff0-0f6b-4e30-8467-ff7db421eba6": { 2024-09-21T06:03:40.849 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:57:14.939082+0000", 2024-09-21T06:03:40.849 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:03:40.849 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:03:40.849 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:03:40.849 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:03:40.849 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:03:40.849 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:03:40.849 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:03:40.849 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T06:03:40.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.572+0000 7f35e1ffb640 1 -- 172.21.15.117:0/3827257984 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f35d8018020 con 0x7f35e410a190 2024-09-21T06:03:40.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.573+0000 7f35e96de640 1 -- 172.21.15.117:0/4190224272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597ef247390 msgr2=0x5597ef60fcf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:03:40.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.573+0000 7f35e96de640 1 --2- 172.21.15.117:0/4190224272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597ef247390 0x5597ef60fcf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.573+0000 7f35e96de640 1 -- 172.21.15.117:0/4190224272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597ef23f1b0 msgr2=0x5597ef60f7b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:03:40.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.573+0000 7f35e96de640 1 --2- 172.21.15.117:0/4190224272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597ef23f1b0 0x5597ef60f7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.573+0000 7f35e96de640 1 -- 172.21.15.117:0/4190224272 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597ef2d3650 con 0x5597ef268070 2024-09-21T06:03:40.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.573+0000 7f35e96de640 1 --2- 172.21.15.117:0/4190224272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597ef268070 0x5597ef60f270 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f35dc009d00 tx=0x7f35dc00b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0611b648d3efe4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:40.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.574+0000 7f35ceffd640 1 -- 172.21.15.117:0/4190224272 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc00bd40 con 0x5597ef268070 2024-09-21T06:03:40.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.574+0000 7f35ceffd640 1 -- 172.21.15.117:0/4190224272 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35dc017070 con 0x5597ef268070 2024-09-21T06:03:40.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.575+0000 7f35e1ffb640 1 --2- 172.21.15.117:0/3827257984 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f35d40b6660 0x7f35d40b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:40.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.575+0000 7f35e1ffb640 1 -- 172.21.15.117:0/3827257984 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f35d80e70b0 con 0x7f35e410a190 2024-09-21T06:03:40.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.575+0000 7f35e8edd640 1 --2- 172.21.15.117:0/3827257984 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f35d40b6660 0x7f35d40b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:40.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.576+0000 7f35ceffd640 1 -- 172.21.15.117:0/4190224272 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc0115f0 con 0x5597ef268070 2024-09-21T06:03:40.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.577+0000 7f35e8edd640 1 --2- 172.21.15.117:0/3827257984 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f35d40b6660 0x7f35d40b8b70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x5597ef60def0 tx=0x7f35d80103e0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:40.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.577+0000 7f35f16b5740 1 -- 172.21.15.117:0/4190224272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597ef268070 msgr2=0x5597ef60f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:40.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.577+0000 7f35f16b5740 1 --2- 172.21.15.117:0/4190224272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597ef268070 0x5597ef60f270 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f35dc009d00 tx=0x7f35dc00b420 comp rx=0 tx=0).stop 2024-09-21T06:03:40.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.578+0000 7f35f16b5740 1 -- 172.21.15.117:0/4190224272 shutdown_connections 2024-09-21T06:03:40.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.578+0000 7f35f16b5740 1 --2- 172.21.15.117:0/4190224272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597ef247390 0x5597ef60fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.578+0000 7f35f16b5740 1 --2- 172.21.15.117:0/4190224272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597ef23f1b0 0x5597ef60f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.578+0000 7f35f16b5740 1 --2- 172.21.15.117:0/4190224272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597ef268070 0x5597ef60f270 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.578+0000 7f35f16b5740 1 -- 172.21.15.117:0/4190224272 >> 172.21.15.117:0/4190224272 conn(0x5597ef25c460 msgr2=0x5597ef60d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:40.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.578+0000 7f35f16b5740 1 -- 172.21.15.117:0/4190224272 shutdown_connections 2024-09-21T06:03:40.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.578+0000 7f35f16b5740 1 -- 172.21.15.117:0/4190224272 wait complete. 2024-09-21T06:03:40.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.580+0000 7f35f16b5740 1 -- start start 2024-09-21T06:03:40.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.580+0000 7f35f16b5740 1 Processor -- start 2024-09-21T06:03:40.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.581+0000 7f35f16b5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597ef23f1b0 0x5597ef61a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:40.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.582+0000 7f35f16b5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597ef247390 0x5597ef61ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:40.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.582+0000 7f35f16b5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597ef268070 0x5597ef61b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:40.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.582+0000 7f35f16b5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5597ef3d6700 con 0x5597ef268070 2024-09-21T06:03:40.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.582+0000 7f35f16b5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5597ef510110 con 0x5597ef23f1b0 2024-09-21T06:03:40.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.582+0000 7f35f16b5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5597ef61daa0 con 0x5597ef247390 2024-09-21T06:03:40.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.582+0000 7f35f16b5740 10 client.0 _ll_register_callbacks cb 0x5597ef529990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:03:40.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.582+0000 7f35e96de640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597ef268070 0x5597ef61b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:40.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.582+0000 7f35e96de640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597ef268070 0x5597ef61b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50336/0 (socket says 172.21.15.117:50336) 2024-09-21T06:03:40.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.582+0000 7f35e96de640 1 -- 172.21.15.117:0/3822367821 learned_addr learned my addr 172.21.15.117:0/3822367821 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:40.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.583+0000 7f35e3fff640 1 --2- 172.21.15.117:0/3822367821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597ef247390 0x5597ef61ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:40.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.583+0000 7f35e96de640 1 -- 172.21.15.117:0/3822367821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597ef247390 msgr2=0x5597ef61ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:40.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.583+0000 7f35e96de640 1 --2- 172.21.15.117:0/3822367821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597ef247390 0x5597ef61ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.583+0000 7f35e96de640 1 -- 172.21.15.117:0/3822367821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597ef23f1b0 msgr2=0x5597ef61a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:40.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.583+0000 7f35e96de640 1 --2- 172.21.15.117:0/3822367821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597ef23f1b0 0x5597ef61a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.583+0000 7f35e96de640 1 -- 172.21.15.117:0/3822367821 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597ef53dc40 con 0x5597ef268070 2024-09-21T06:03:40.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.584+0000 7f35e96de640 1 --2- 172.21.15.117:0/3822367821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597ef268070 0x5597ef61b3c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f35dc0070b0 tx=0x7f35dc0099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34df7a15a44505c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:40.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.584+0000 7f35c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:03:40.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.585+0000 7f35c3fff640 1 -- 172.21.15.117:0/3822367821 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc021070 con 0x5597ef268070 2024-09-21T06:03:40.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.585+0000 7f35c3fff640 1 -- 172.21.15.117:0/3822367821 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f35dc012640 con 0x5597ef268070 2024-09-21T06:03:40.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.585+0000 7f35f16b5740 1 -- 172.21.15.117:0/3822367821 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35dc017070 con 0x5597ef268070 2024-09-21T06:03:40.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.585+0000 7f35c3fff640 1 -- 172.21.15.117:0/3822367821 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f35dc012840 con 0x5597ef268070 2024-09-21T06:03:40.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.585+0000 7f35f16b5740 10 client.16533 fetch_fsmap learned FSMap version 27 2024-09-21T06:03:40.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.585+0000 7f35f16b5740 1 -- 172.21.15.117:0/3822367821 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5597ef629820 con 0x5597ef268070 2024-09-21T06:03:40.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.586+0000 7f35c3fff640 1 -- 172.21.15.117:0/3822367821 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35dc01f030 con 0x5597ef268070 2024-09-21T06:03:40.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.587+0000 7f35c3fff640 1 -- 172.21.15.117:0/3822367821 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f35dc011070 con 0x5597ef268070 2024-09-21T06:03:40.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.587+0000 7f35f16b5740 10 client.16533 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:03:40.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.587+0000 7f35f16b5740 10 client.16533 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:03:40.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.587+0000 7f35f16b5740 20 client.16533 populate_metadata read hostname 'smithi117' 2024-09-21T06:03:40.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.587+0000 7f35f16b5740 1 --2- 172.21.15.117:0/3822367821 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5597ef2269f0 0x5597ef62cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:40.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.587+0000 7f35f16b5740 4 client.16533 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:03:40.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.587+0000 7f35f16b5740 1 -- 172.21.15.117:0/3822367821 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f35dc017070 con 0x5597ef2269f0 2024-09-21T06:03:40.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.588+0000 7f35e3fff640 1 --2- 172.21.15.117:0/3822367821 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5597ef2269f0 0x5597ef62cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:40.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.595+0000 7f35e3fff640 1 --2- 172.21.15.117:0/3822367821 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5597ef2269f0 0x5597ef62cbe0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8995fd5d8630067 server_cookie=1e02f78bb18892e8 in_seq=0 out_seq=0 2024-09-21T06:03:40.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.595+0000 7f35c3fff640 10 client.16533 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:03:40.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.600+0000 7f35c3fff640 1 -- 172.21.15.117:0/3822367821 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f35dc017070 con 0x5597ef2269f0 2024-09-21T06:03:40.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.600+0000 7f35c3fff640 10 client.16533 handle_command_reply: tid=0 2024-09-21T06:03:40.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.600+0000 7f35f16b5740 1 client.16533 shutdown 2024-09-21T06:03:40.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.601+0000 7f35f16b5740 1 -- 172.21.15.117:0/3822367821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597ef268070 msgr2=0x5597ef61b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:40.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.601+0000 7f35f16b5740 1 --2- 172.21.15.117:0/3822367821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597ef268070 0x5597ef61b3c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f35dc0070b0 tx=0x7f35dc0099b0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.601+0000 7f35f16b5740 1 -- 172.21.15.117:0/3822367821 shutdown_connections 2024-09-21T06:03:40.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.602+0000 7f35f16b5740 1 --2- 172.21.15.117:0/3822367821 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5597ef2269f0 0x5597ef62cbe0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.602+0000 7f35f16b5740 1 --2- 172.21.15.117:0/3822367821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597ef268070 0x5597ef61b3c0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.602+0000 7f35c3fff640 0 client.16533 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:03:40.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.602+0000 7f35f16b5740 1 --2- 172.21.15.117:0/3822367821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597ef247390 0x5597ef61ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.602+0000 7f35f16b5740 1 --2- 172.21.15.117:0/3822367821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597ef23f1b0 0x5597ef61a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.602+0000 7f35f16b5740 1 -- 172.21.15.117:0/3822367821 >> 172.21.15.117:0/3822367821 conn(0x5597ef25c460 msgr2=0x5597ef612400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:40.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.602+0000 7f35f16b5740 1 -- 172.21.15.117:0/3822367821 shutdown_connections 2024-09-21T06:03:40.855 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.602+0000 7f35f16b5740 1 -- 172.21.15.117:0/3822367821 wait complete. 2024-09-21T06:03:40.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.603+0000 7f35f16b5740 20 client.16533 trim_cache size 0 max 16384 2024-09-21T06:03:40.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.642+0000 7f35f16b5740 1 Processor -- start 2024-09-21T06:03:40.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.643+0000 7f35f16b5740 1 -- start start 2024-09-21T06:03:40.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.643+0000 7f35f16b5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597ef2269f0 0x5597ef6109b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:40.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.643+0000 7f35f16b5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597ef23f1b0 0x5597ef617af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:40.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.643+0000 7f35f16b5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597ef247390 0x5597ef618030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:40.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.643+0000 7f35f16b5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f35c800e620 con 0x5597ef23f1b0 2024-09-21T06:03:40.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.643+0000 7f35f16b5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f35c8010160 con 0x5597ef247390 2024-09-21T06:03:40.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.643+0000 7f35f16b5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f35c800ecf0 con 0x5597ef2269f0 2024-09-21T06:03:40.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.644+0000 7f35e3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597ef23f1b0 0x5597ef617af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:40.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.644+0000 7f35e96de640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597ef2269f0 0x5597ef6109b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:40.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.644+0000 7f35e3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597ef23f1b0 0x5597ef617af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50350/0 (socket says 172.21.15.117:50350) 2024-09-21T06:03:40.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.644+0000 7f35e3fff640 1 -- 172.21.15.117:0/1814509736 learned_addr learned my addr 172.21.15.117:0/1814509736 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:40.856 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.645+0000 7f35e96de640 1 --2- 172.21.15.117:0/1814509736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597ef247390 0x5597ef618030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:40.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.647+0000 7f35e96de640 1 -- 172.21.15.117:0/1814509736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597ef247390 msgr2=0x5597ef618030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:40.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.647+0000 7f35e96de640 1 --2- 172.21.15.117:0/1814509736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597ef247390 0x5597ef618030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.647+0000 7f35e96de640 1 -- 172.21.15.117:0/1814509736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597ef23f1b0 msgr2=0x5597ef617af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:40.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.647+0000 7f35e96de640 1 --2- 172.21.15.117:0/1814509736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597ef23f1b0 0x5597ef617af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.647+0000 7f35e96de640 1 -- 172.21.15.117:0/1814509736 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597ef619860 con 0x5597ef2269f0 2024-09-21T06:03:40.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.651+0000 7f35e3fff640 1 --2- 172.21.15.117:0/1814509736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597ef23f1b0 0x5597ef617af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:03:40.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.651+0000 7f35e96de640 1 --2- 172.21.15.117:0/1814509736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597ef2269f0 0x5597ef6109b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f35dc02feb0 tx=0x7f35dc00b420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=491cbaf555fabb69 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:40.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.653+0000 7f35ce7fc640 1 -- 172.21.15.117:0/1814509736 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc021040 con 0x5597ef2269f0 2024-09-21T06:03:40.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.653+0000 7f35ce7fc640 1 -- 172.21.15.117:0/1814509736 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35dc01f070 con 0x5597ef2269f0 2024-09-21T06:03:40.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.653+0000 7f35ce7fc640 1 -- 172.21.15.117:0/1814509736 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc011e60 con 0x5597ef2269f0 2024-09-21T06:03:40.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.656+0000 7f35f16b5740 1 -- 172.21.15.117:0/1814509736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597ef2269f0 msgr2=0x5597ef6109b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:40.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.656+0000 7f35f16b5740 1 --2- 172.21.15.117:0/1814509736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597ef2269f0 0x5597ef6109b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f35dc02feb0 tx=0x7f35dc00b420 comp rx=0 tx=0).stop 2024-09-21T06:03:40.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.658+0000 7f35f16b5740 1 -- 172.21.15.117:0/1814509736 shutdown_connections 2024-09-21T06:03:40.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.658+0000 7f35f16b5740 1 --2- 172.21.15.117:0/1814509736 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597ef23f1b0 0x5597ef617af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.659+0000 7f35f16b5740 1 --2- 172.21.15.117:0/1814509736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597ef247390 0x5597ef618030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.659+0000 7f35f16b5740 1 --2- 172.21.15.117:0/1814509736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597ef2269f0 0x5597ef6109b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.659+0000 7f35f16b5740 1 -- 172.21.15.117:0/1814509736 >> 172.21.15.117:0/1814509736 conn(0x5597ef25c460 msgr2=0x5597ef6102a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:40.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.667+0000 7f35f16b5740 1 -- 172.21.15.117:0/1814509736 shutdown_connections 2024-09-21T06:03:40.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.667+0000 7f35f16b5740 1 -- 172.21.15.117:0/1814509736 wait complete. 2024-09-21T06:03:40.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.667+0000 7f35f16b5740 1 -- start start 2024-09-21T06:03:40.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.668+0000 7f35f16b5740 1 Processor -- start 2024-09-21T06:03:40.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.669+0000 7f35f16b5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597ef2269f0 0x5597ef62e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:40.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.669+0000 7f35f16b5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597ef23f1b0 0x5597ef62ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:40.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.669+0000 7f35f16b5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597ef247390 0x5597ef690330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:40.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.669+0000 7f35f16b5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f35c800fa90 con 0x5597ef23f1b0 2024-09-21T06:03:40.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.669+0000 7f35f16b5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f35c800f3c0 con 0x5597ef2269f0 2024-09-21T06:03:40.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.669+0000 7f35f16b5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f35c800c600 con 0x5597ef247390 2024-09-21T06:03:40.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.669+0000 7f35f16b5740 10 client.0 _ll_register_callbacks cb 0x7f35dc009bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:03:40.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.670+0000 7f35e3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597ef23f1b0 0x5597ef62ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:40.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.670+0000 7f35e3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597ef23f1b0 0x5597ef62ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50360/0 (socket says 172.21.15.117:50360) 2024-09-21T06:03:40.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.670+0000 7f35e3fff640 1 -- 172.21.15.117:0/3069803651 learned_addr learned my addr 172.21.15.117:0/3069803651 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:40.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.671+0000 7f35e3fff640 1 -- 172.21.15.117:0/3069803651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597ef247390 msgr2=0x5597ef690330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:40.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.671+0000 7f35e3fff640 1 --2- 172.21.15.117:0/3069803651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597ef247390 0x5597ef690330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.671+0000 7f35e3fff640 1 -- 172.21.15.117:0/3069803651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597ef2269f0 msgr2=0x5597ef62e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:40.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.671+0000 7f35e3fff640 1 --2- 172.21.15.117:0/3069803651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597ef2269f0 0x5597ef62e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.671+0000 7f35e3fff640 1 -- 172.21.15.117:0/3069803651 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597ef6909a0 con 0x5597ef23f1b0 2024-09-21T06:03:40.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.672+0000 7f35e3fff640 1 --2- 172.21.15.117:0/3069803651 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597ef23f1b0 0x5597ef62ead0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f35e406bb30 tx=0x7f35d0032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f9f5fd652228e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:40.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.672+0000 7f35c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:03:40.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.672+0000 7f35c3fff640 1 -- 172.21.15.117:0/3069803651 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d0046020 con 0x5597ef23f1b0 2024-09-21T06:03:40.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.673+0000 7f35f16b5740 1 -- 172.21.15.117:0/3069803651 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35dc01f070 con 0x5597ef23f1b0 2024-09-21T06:03:40.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.673+0000 7f35c3fff640 1 -- 172.21.15.117:0/3069803651 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f35d0005040 con 0x5597ef23f1b0 2024-09-21T06:03:40.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.674+0000 7f35c3fff640 1 -- 172.21.15.117:0/3069803651 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f35d0051050 con 0x5597ef23f1b0 2024-09-21T06:03:40.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.678+0000 7f35f16b5740 10 client.16539 fetch_fsmap learned FSMap version 27 2024-09-21T06:03:40.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.678+0000 7f35f16b5740 1 -- 172.21.15.117:0/3069803651 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5597ef697d00 con 0x5597ef23f1b0 2024-09-21T06:03:40.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.680+0000 7f35c3fff640 1 -- 172.21.15.117:0/3069803651 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35dc01f070 con 0x5597ef23f1b0 2024-09-21T06:03:40.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.680+0000 7f35c3fff640 1 -- 172.21.15.117:0/3069803651 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f35d00035f0 con 0x5597ef23f1b0 2024-09-21T06:03:40.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.681+0000 7f35f16b5740 10 client.16539 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:03:40.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.681+0000 7f35f16b5740 10 client.16539 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:03:40.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.681+0000 7f35f16b5740 20 client.16539 populate_metadata read hostname 'smithi117' 2024-09-21T06:03:40.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.681+0000 7f35f16b5740 1 --2- 172.21.15.117:0/3069803651 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5597ef268070 0x5597ef69a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:40.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.681+0000 7f35f16b5740 4 client.16539 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:03:40.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.681+0000 7f35f16b5740 1 -- 172.21.15.117:0/3069803651 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5597ef587da0 con 0x5597ef268070 2024-09-21T06:03:40.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.683+0000 7f35e96de640 1 --2- 172.21.15.117:0/3069803651 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5597ef268070 0x5597ef69a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:40.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.702+0000 7f35e96de640 1 --2- 172.21.15.117:0/3069803651 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5597ef268070 0x5597ef69a4f0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84c7e5622c2bc812 server_cookie=eaf3067fc696e27d in_seq=0 out_seq=0 2024-09-21T06:03:40.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.703+0000 7f35c3fff640 10 client.16539 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:03:40.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.709+0000 7f35c3fff640 1 -- 172.21.15.117:0/3069803651 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1510 (crc 0 0 0) 0x5597ef587da0 con 0x5597ef268070 2024-09-21T06:03:40.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.709+0000 7f35c3fff640 10 client.16539 handle_command_reply: tid=0 2024-09-21T06:03:40.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.711+0000 7f35f16b5740 1 client.16539 shutdown 2024-09-21T06:03:40.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.712+0000 7f35f16b5740 1 -- 172.21.15.117:0/3069803651 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597ef23f1b0 msgr2=0x5597ef62ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:40.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.712+0000 7f35f16b5740 1 --2- 172.21.15.117:0/3069803651 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597ef23f1b0 0x5597ef62ead0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f35e406bb30 tx=0x7f35d0032f90 comp rx=0 tx=0).stop 2024-09-21T06:03:40.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.712+0000 7f35f16b5740 1 -- 172.21.15.117:0/3069803651 shutdown_connections 2024-09-21T06:03:40.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.712+0000 7f35f16b5740 1 --2- 172.21.15.117:0/3069803651 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5597ef268070 0x5597ef69a4f0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.712+0000 7f35f16b5740 1 --2- 172.21.15.117:0/3069803651 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5597ef23f1b0 0x5597ef62ead0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.712+0000 7f35f16b5740 1 --2- 172.21.15.117:0/3069803651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597ef247390 0x5597ef690330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.712+0000 7f35f16b5740 1 --2- 172.21.15.117:0/3069803651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5597ef2269f0 0x5597ef62e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.712+0000 7f35f16b5740 1 -- 172.21.15.117:0/3069803651 >> 172.21.15.117:0/3069803651 conn(0x5597ef25c460 msgr2=0x5597ef61bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:40.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.713+0000 7f35c3fff640 0 client.16539 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:03:40.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.713+0000 7f35f16b5740 1 -- 172.21.15.117:0/3069803651 shutdown_connections 2024-09-21T06:03:40.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.713+0000 7f35f16b5740 1 -- 172.21.15.117:0/3069803651 wait complete. 2024-09-21T06:03:40.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.715+0000 7f35f16b5740 20 client.16539 trim_cache size 0 max 16384 2024-09-21T06:03:40.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.717+0000 7f35eb141640 1 -- 172.21.15.117:0/3827257984 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f35d40b6660 msgr2=0x7f35d40b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:40.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.717+0000 7f35eb141640 1 --2- 172.21.15.117:0/3827257984 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f35d40b6660 0x7f35d40b8b70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x5597ef60def0 tx=0x7f35d80103e0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.717+0000 7f35eb141640 1 -- 172.21.15.117:0/3827257984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35e410a190 msgr2=0x7f35e4083d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:40.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.717+0000 7f35eb141640 1 --2- 172.21.15.117:0/3827257984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35e410a190 0x7f35e4083d10 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f35d800b450 tx=0x7f35d800b910 comp rx=0 tx=0).stop 2024-09-21T06:03:40.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.718+0000 7f35eb141640 1 -- 172.21.15.117:0/3827257984 shutdown_connections 2024-09-21T06:03:40.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.718+0000 7f35eb141640 1 --2- 172.21.15.117:0/3827257984 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f35d40b6660 0x7f35d40b8b70 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.718+0000 7f35eb141640 1 --2- 172.21.15.117:0/3827257984 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f35e4110420 0x7f35e407d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.718+0000 7f35eb141640 1 --2- 172.21.15.117:0/3827257984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35e410fad0 0x7f35e407cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.718+0000 7f35eb141640 1 --2- 172.21.15.117:0/3827257984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35e410a190 0x7f35e4083d10 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:40.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.718+0000 7f35eb141640 1 -- 172.21.15.117:0/3827257984 >> 172.21.15.117:0/3827257984 conn(0x7f35e40780f0 msgr2=0x7f35e41081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:40.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.722+0000 7f35eb141640 1 -- 172.21.15.117:0/3827257984 shutdown_connections 2024-09-21T06:03:40.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:40.724+0000 7f35eb141640 1 -- 172.21.15.117:0/3827257984 wait complete. 2024-09-21T06:03:42.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:41 smithi192 ceph-mon[34700]: pgmap v518: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 869 MiB/s wr, 1.61k op/s 2024-09-21T06:03:42.234 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:41 smithi117 ceph-mon[30947]: pgmap v518: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 869 MiB/s wr, 1.61k op/s 2024-09-21T06:03:42.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:41 smithi138 ceph-mon[34391]: pgmap v518: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 869 MiB/s wr, 1.61k op/s 2024-09-21T06:03:43.903 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:03:43.903 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:03:44.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:43 smithi192 ceph-mon[34700]: pgmap v519: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 868 MiB/s wr, 1.74k op/s 2024-09-21T06:03:44.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:43 smithi117 ceph-mon[30947]: pgmap v519: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 868 MiB/s wr, 1.74k op/s 2024-09-21T06:03:44.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:43 smithi138 ceph-mon[34391]: pgmap v519: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 868 MiB/s wr, 1.74k op/s 2024-09-21T06:03:44.650 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:46.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:45 smithi192 ceph-mon[34700]: pgmap v520: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 918 MiB/s wr, 2.00k op/s 2024-09-21T06:03:46.268 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:45 smithi117 ceph-mon[30947]: pgmap v520: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 918 MiB/s wr, 2.00k op/s 2024-09-21T06:03:46.314 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T06:03:46.315 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":27,"btime":"2024-09-21T05:57:06:651813+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3227057504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3227057504},{"type":"v1","addr":"172.21.15.117:6835","nonce":3227057504}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3303829183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3303829183},{"type":"v1","addr":"172.21.15.117:6837","nonce":3303829183}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T05:55:14.555897+0000","modified":"2024-09-21T05:57:06.599449+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24638,"mds_1":24637,"mds_2":24628,"mds_3":24616,"mds_4":24602},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6839/2314572411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2314572411},{"type":"v1","addr":"172.21.15.117:6839","nonce":2314572411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6841/3378096561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":3378096561},{"type":"v1","addr":"172.21.15.117:6841","nonce":3378096561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/3725629862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3725629862},{"type":"v1","addr":"172.21.15.138:6833","nonce":3725629862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/4276801396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4276801396},{"type":"v1","addr":"172.21.15.138:6835","nonce":4276801396}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24586":{"gid":24586,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6833/2224910879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":2224910879},{"type":"v1","addr":"172.21.15.192:6833","nonce":2224910879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.138:6837/2540335374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2540335374},{"type":"v1","addr":"172.21.15.138:6837","nonce":2540335374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24616":{"gid":24616,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.192:6835/3530688860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3530688860},{"type":"v1","addr":"172.21.15.192:6835","nonce":3530688860}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.192:6837/2683860842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2683860842},{"type":"v1","addr":"172.21.15.192:6837","nonce":2683860842}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.192:6839/1727623231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1727623231},{"type":"v1","addr":"172.21.15.192:6839","nonce":1727623231}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/3230372576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":3230372576},{"type":"v1","addr":"172.21.15.138:6839","nonce":3230372576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24602,24628,24616,24637,24638]},"id":1}]} 2024-09-21T06:03:46.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.703+0000 7fe3e6bc1640 1 -- 172.21.15.117:0/630832037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3e0115360 msgr2=0x7fe3e0117850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:46.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.703+0000 7fe3e6bc1640 1 --2- 172.21.15.117:0/630832037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3e0115360 0x7fe3e0117850 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc009900 tx=0x7fe3dc02a3f0 comp rx=0 tx=0).stop 2024-09-21T06:03:46.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.705+0000 7fe3e6bc1640 1 -- 172.21.15.117:0/630832037 shutdown_connections 2024-09-21T06:03:46.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.705+0000 7fe3e6bc1640 1 --2- 172.21.15.117:0/630832037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3e0115360 0x7fe3e0117850 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:46.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.705+0000 7fe3e6bc1640 1 --2- 172.21.15.117:0/630832037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3e010c880 0x7fe3e0114dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:46.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.705+0000 7fe3e6bc1640 1 --2- 172.21.15.117:0/630832037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3e010bf30 0x7fe3e010c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:46.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.705+0000 7fe3e6bc1640 1 -- 172.21.15.117:0/630832037 >> 172.21.15.117:0/630832037 conn(0x7fe3e006ee50 msgr2=0x7fe3e006f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:46.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.707+0000 7fe3e6bc1640 1 -- 172.21.15.117:0/630832037 shutdown_connections 2024-09-21T06:03:46.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.711+0000 7fe3e6bc1640 1 -- 172.21.15.117:0/630832037 wait complete. 2024-09-21T06:03:46.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.712+0000 7fe3e6bc1640 1 Processor -- start 2024-09-21T06:03:46.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.712+0000 7fe3e6bc1640 1 -- start start 2024-09-21T06:03:46.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.713+0000 7fe3e6bc1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3e010bf30 0x7fe3e01123e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:46.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.713+0000 7fe3e6bc1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3e010c880 0x7fe3e0112920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:46.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.713+0000 7fe3e6bc1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3e0115360 0x7fe3e01b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:46.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.713+0000 7fe3e6bc1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e0119e60 con 0x7fe3e010c880 2024-09-21T06:03:46.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.713+0000 7fe3e5bbf640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3e010bf30 0x7fe3e01123e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:46.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.713+0000 7fe3e5bbf640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3e010bf30 0x7fe3e01123e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:46816/0 (socket says 172.21.15.117:46816) 2024-09-21T06:03:46.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.713+0000 7fe3e5bbf640 1 -- 172.21.15.117:0/3844656118 learned_addr learned my addr 172.21.15.117:0/3844656118 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:46.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.713+0000 7fe3e6bc1640 1 -- 172.21.15.117:0/3844656118 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e0119ce0 con 0x7fe3e0115360 2024-09-21T06:03:46.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.714+0000 7fe3e63c0640 1 --2- 172.21.15.117:0/3844656118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3e0115360 0x7fe3e01b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:46.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.717+0000 7fe3e53be640 1 --2- 172.21.15.117:0/3844656118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3e010c880 0x7fe3e0112920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:46.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.718+0000 7fe3e6bc1640 1 -- 172.21.15.117:0/3844656118 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e0119fe0 con 0x7fe3e010bf30 2024-09-21T06:03:46.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.721+0000 7fe3e5bbf640 1 -- 172.21.15.117:0/3844656118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3e0115360 msgr2=0x7fe3e01b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:46.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.721+0000 7fe3e5bbf640 1 --2- 172.21.15.117:0/3844656118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3e0115360 0x7fe3e01b4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:46.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.721+0000 7fe3e5bbf640 1 -- 172.21.15.117:0/3844656118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3e010c880 msgr2=0x7fe3e0112920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:46.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.721+0000 7fe3e5bbf640 1 --2- 172.21.15.117:0/3844656118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3e010c880 0x7fe3e0112920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:46.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.721+0000 7fe3e5bbf640 1 -- 172.21.15.117:0/3844656118 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e0113110 con 0x7fe3e010bf30 2024-09-21T06:03:46.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.721+0000 7fe3e53be640 1 --2- 172.21.15.117:0/3844656118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3e010c880 0x7fe3e0112920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:03:46.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.725+0000 7fe3e5bbf640 1 --2- 172.21.15.117:0/3844656118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3e010bf30 0x7fe3e01123e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe3d800cce0 tx=0x7fe3d800a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:46.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.726+0000 7fe3d6ffd640 1 -- 172.21.15.117:0/3844656118 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8004b20 con 0x7fe3e010bf30 2024-09-21T06:03:46.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.726+0000 7fe3d6ffd640 1 -- 172.21.15.117:0/3844656118 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3d8004cc0 con 0x7fe3e010bf30 2024-09-21T06:03:46.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.726+0000 7fe3d6ffd640 1 -- 172.21.15.117:0/3844656118 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d800f6b0 con 0x7fe3e010bf30 2024-09-21T06:03:46.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.727+0000 7fe3e6bc1640 1 -- 172.21.15.117:0/3844656118 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3e01b5490 con 0x7fe3e010bf30 2024-09-21T06:03:46.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.727+0000 7fe3e6bc1640 1 -- 172.21.15.117:0/3844656118 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3e01b58f0 con 0x7fe3e010bf30 2024-09-21T06:03:46.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.731+0000 7fe3e6bc1640 1 -- 172.21.15.117:0/3844656118 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3e010c340 con 0x7fe3e010bf30 2024-09-21T06:03:46.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.732+0000 7fe3d6ffd640 1 -- 172.21.15.117:0/3844656118 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fe3d800f850 con 0x7fe3e010bf30 2024-09-21T06:03:46.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.733+0000 7fe3d6ffd640 1 --2- 172.21.15.117:0/3844656118 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe3cc0b6880 0x7fe3cc0b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:46.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.734+0000 7fe3d6ffd640 1 -- 172.21.15.117:0/3844656118 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fe3d80e76b0 con 0x7fe3e010bf30 2024-09-21T06:03:46.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.737+0000 7fe3e53be640 1 --2- 172.21.15.117:0/3844656118 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe3cc0b6880 0x7fe3cc0b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:46.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.737+0000 7fe3d6ffd640 1 -- 172.21.15.117:0/3844656118 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe3d8014030 con 0x7fe3e010bf30 2024-09-21T06:03:46.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:45.740+0000 7fe3e53be640 1 --2- 172.21.15.117:0/3844656118 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe3cc0b6880 0x7fe3cc0b8d90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0116fd0 tx=0x7fe3d0005d20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:46.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:46.265+0000 7fe3e6bc1640 1 -- 172.21.15.117:0/3844656118 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3e01b5ce0 con 0x7fe3e010bf30 2024-09-21T06:03:46.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:46.278+0000 7fe3d6ffd640 1 -- 172.21.15.117:0/3844656118 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+12001 (secure 0 0 0) 0x7fe3d80adc20 con 0x7fe3e010bf30 2024-09-21T06:03:46.318 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 27 2024-09-21T06:03:46.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:46.283+0000 7fe3e6bc1640 1 -- 172.21.15.117:0/3844656118 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe3cc0b6880 msgr2=0x7fe3cc0b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:46.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:46.283+0000 7fe3e6bc1640 1 --2- 172.21.15.117:0/3844656118 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe3cc0b6880 0x7fe3cc0b8d90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0116fd0 tx=0x7fe3d0005d20 comp rx=0 tx=0).stop 2024-09-21T06:03:46.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:46.283+0000 7fe3e6bc1640 1 -- 172.21.15.117:0/3844656118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3e010bf30 msgr2=0x7fe3e01123e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:46.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:46.283+0000 7fe3e6bc1640 1 --2- 172.21.15.117:0/3844656118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3e010bf30 0x7fe3e01123e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe3d800cce0 tx=0x7fe3d800a620 comp rx=0 tx=0).stop 2024-09-21T06:03:46.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:46.284+0000 7fe3e6bc1640 1 -- 172.21.15.117:0/3844656118 shutdown_connections 2024-09-21T06:03:46.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:46.284+0000 7fe3e6bc1640 1 --2- 172.21.15.117:0/3844656118 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe3cc0b6880 0x7fe3cc0b8d90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:46.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:46.284+0000 7fe3e6bc1640 1 --2- 172.21.15.117:0/3844656118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3e010c880 0x7fe3e0112920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:46.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:46.284+0000 7fe3e6bc1640 1 --2- 172.21.15.117:0/3844656118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3e0115360 0x7fe3e01b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:46.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:46.284+0000 7fe3e6bc1640 1 --2- 172.21.15.117:0/3844656118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3e010bf30 0x7fe3e01123e0 secure :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe3d800cce0 tx=0x7fe3d800a620 comp rx=0 tx=0).stop 2024-09-21T06:03:46.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:46.284+0000 7fe3e6bc1640 1 -- 172.21.15.117:0/3844656118 >> 172.21.15.117:0/3844656118 conn(0x7fe3e006ee50 msgr2=0x7fe3e011b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:46.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:46.285+0000 7fe3e6bc1640 1 -- 172.21.15.117:0/3844656118 shutdown_connections 2024-09-21T06:03:46.319 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:46.285+0000 7fe3e6bc1640 1 -- 172.21.15.117:0/3844656118 wait complete. 2024-09-21T06:03:46.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:45 smithi138 ceph-mon[34391]: pgmap v520: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 918 MiB/s wr, 2.00k op/s 2024-09-21T06:03:46.996 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:46 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3844656118' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:03:47.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:46 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3844656118' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:03:47.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:46 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3844656118' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:03:48.266 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:47 smithi138 ceph-mon[34391]: pgmap v521: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 897 MiB/s wr, 2.19k op/s 2024-09-21T06:03:48.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:47 smithi117 ceph-mon[30947]: pgmap v521: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 897 MiB/s wr, 2.19k op/s 2024-09-21T06:03:48.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:47 smithi192 ceph-mon[34700]: pgmap v521: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 897 MiB/s wr, 2.19k op/s 2024-09-21T06:03:49.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:49 smithi117 ceph-mon[30947]: pgmap v522: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 914 MiB/s wr, 2.37k op/s 2024-09-21T06:03:49.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:49 smithi138 ceph-mon[34391]: pgmap v522: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 914 MiB/s wr, 2.37k op/s 2024-09-21T06:03:49.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:49 smithi192 ceph-mon[34700]: pgmap v522: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 914 MiB/s wr, 2.37k op/s 2024-09-21T06:03:49.879 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T06:03:50.716 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.565+0000 7fe77d88b640 1 -- 172.21.15.117:0/353168547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7781029b0 msgr2=0x7fe778102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.565+0000 7fe77d88b640 1 --2- 172.21.15.117:0/353168547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7781029b0 0x7fe778102dc0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe768009900 tx=0x7fe76802a410 comp rx=0 tx=0).stop 2024-09-21T06:03:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.565+0000 7fe77d88b640 1 -- 172.21.15.117:0/353168547 shutdown_connections 2024-09-21T06:03:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.565+0000 7fe77d88b640 1 --2- 172.21.15.117:0/353168547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe778108c30 0x7fe77810f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.565+0000 7fe77d88b640 1 --2- 172.21.15.117:0/353168547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7781082e0 0x7fe7781086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.565+0000 7fe77d88b640 1 --2- 172.21.15.117:0/353168547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7781029b0 0x7fe778102dc0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.565+0000 7fe77d88b640 1 -- 172.21.15.117:0/353168547 >> 172.21.15.117:0/353168547 conn(0x7fe77807a740 msgr2=0x7fe77807ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.565+0000 7fe77d88b640 1 -- 172.21.15.117:0/353168547 shutdown_connections 2024-09-21T06:03:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.565+0000 7fe77d88b640 1 -- 172.21.15.117:0/353168547 wait complete. 2024-09-21T06:03:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.566+0000 7fe77d88b640 1 Processor -- start 2024-09-21T06:03:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.567+0000 7fe77d88b640 1 -- start start 2024-09-21T06:03:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.567+0000 7fe77d88b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7781029b0 0x7fe778199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.567+0000 7fe77d88b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7781082e0 0x7fe778199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.567+0000 7fe77d88b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe778108c30 0x7fe778199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.567+0000 7fe77d88b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe778112a00 con 0x7fe7781029b0 2024-09-21T06:03:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.567+0000 7fe77d88b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe778112880 con 0x7fe778108c30 2024-09-21T06:03:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.567+0000 7fe77d88b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe778112b80 con 0x7fe7781082e0 2024-09-21T06:03:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.567+0000 7fe776ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7781029b0 0x7fe778199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.567+0000 7fe776ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7781029b0 0x7fe778199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44972/0 (socket says 172.21.15.117:44972) 2024-09-21T06:03:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.567+0000 7fe776ffd640 1 -- 172.21.15.117:0/2553430036 learned_addr learned my addr 172.21.15.117:0/2553430036 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.568+0000 7fe776ffd640 1 -- 172.21.15.117:0/2553430036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7781082e0 msgr2=0x7fe778199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.568+0000 7fe776ffd640 1 --2- 172.21.15.117:0/2553430036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7781082e0 0x7fe778199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.568+0000 7fe776ffd640 1 -- 172.21.15.117:0/2553430036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe778108c30 msgr2=0x7fe778199c80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T06:03:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.568+0000 7fe776ffd640 1 --2- 172.21.15.117:0/2553430036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe778108c30 0x7fe778199c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.568+0000 7fe776ffd640 1 -- 172.21.15.117:0/2553430036 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7781a88d0 con 0x7fe7781029b0 2024-09-21T06:03:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.568+0000 7fe776ffd640 1 --2- 172.21.15.117:0/2553430036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7781029b0 0x7fe778199200 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe76802aa00 tx=0x7fe768003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.568+0000 7fe77c889640 1 -- 172.21.15.117:0/2553430036 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76803f070 con 0x7fe7781029b0 2024-09-21T06:03:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.569+0000 7fe77c889640 1 -- 172.21.15.117:0/2553430036 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe768031d60 con 0x7fe7781029b0 2024-09-21T06:03:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.569+0000 7fe77c889640 1 -- 172.21.15.117:0/2553430036 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe768035030 con 0x7fe7781029b0 2024-09-21T06:03:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.569+0000 7fe77d88b640 1 -- 172.21.15.117:0/2553430036 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7781a8b60 con 0x7fe7781029b0 2024-09-21T06:03:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.569+0000 7fe77d88b640 1 -- 172.21.15.117:0/2553430036 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7781a9050 con 0x7fe7781029b0 2024-09-21T06:03:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.576+0000 7fe77c889640 1 -- 172.21.15.117:0/2553430036 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fe768042070 con 0x7fe7781029b0 2024-09-21T06:03:51.579 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.577+0000 7fe77c889640 1 --2- 172.21.15.117:0/2553430036 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe7600b6680 0x7fe7600b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:51.579 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.578+0000 7fe77c889640 1 -- 172.21.15.117:0/2553430036 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fe768108160 con 0x7fe7781029b0 2024-09-21T06:03:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.578+0000 7fe7767fc640 1 --2- 172.21.15.117:0/2553430036 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe7600b6680 0x7fe7600b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.584+0000 7fe7767fc640 1 --2- 172.21.15.117:0/2553430036 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe7600b6680 0x7fe7600b8b90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe77807b1c0 tx=0x7fe76c006c40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.587+0000 7fe7847f6740 1 Processor -- start 2024-09-21T06:03:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.588+0000 7fe7847f6740 1 -- start start 2024-09-21T06:03:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.589+0000 7fe7847f6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2bfe951b0 0x55f2c023a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.589+0000 7fe776ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2bfe951b0 0x55f2c023a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.589+0000 7fe776ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2bfe951b0 0x55f2c023a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44988/0 (socket says 172.21.15.117:44988) 2024-09-21T06:03:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.589+0000 7fe776ffd640 1 -- 172.21.15.117:0/3008845918 learned_addr learned my addr 172.21.15.117:0/3008845918 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.589+0000 7fe7847f6740 1 --2- 172.21.15.117:0/3008845918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2bfe6c2f0 0x55f2c023aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.589+0000 7fe7847f6740 1 --2- 172.21.15.117:0/3008845918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2bfe744d0 0x55f2c023b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.590+0000 7fe7847f6740 1 -- 172.21.15.117:0/3008845918 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f2c01af6f0 con 0x55f2bfe951b0 2024-09-21T06:03:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.590+0000 7fe7847f6740 1 -- 172.21.15.117:0/3008845918 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f2c019eb20 con 0x55f2bfe744d0 2024-09-21T06:03:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.590+0000 7fe7847f6740 1 -- 172.21.15.117:0/3008845918 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f2c01a2930 con 0x55f2bfe6c2f0 2024-09-21T06:03:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.590+0000 7fe776ffd640 1 -- 172.21.15.117:0/3008845918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2bfe6c2f0 msgr2=0x55f2c023aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.590+0000 7fe776ffd640 1 --2- 172.21.15.117:0/3008845918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2bfe6c2f0 0x55f2c023aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.590+0000 7fe776ffd640 1 -- 172.21.15.117:0/3008845918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2bfe744d0 msgr2=0x55f2c023b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.590+0000 7fe776ffd640 1 --2- 172.21.15.117:0/3008845918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2bfe744d0 0x55f2c023b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.590+0000 7fe776ffd640 1 -- 172.21.15.117:0/3008845918 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2c0164530 con 0x55f2bfe951b0 2024-09-21T06:03:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.591+0000 7fe776ffd640 1 --2- 172.21.15.117:0/3008845918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2bfe951b0 0x55f2c023a9b0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fe7680ca600 tx=0x7fe76803b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ce3236674d72c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:51.600 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.591+0000 7fe75dffb640 1 -- 172.21.15.117:0/3008845918 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7680ce730 con 0x55f2bfe951b0 2024-09-21T06:03:51.600 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.593+0000 7fe75dffb640 1 -- 172.21.15.117:0/3008845918 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7680d2b10 con 0x55f2bfe951b0 2024-09-21T06:03:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.593+0000 7fe75dffb640 1 -- 172.21.15.117:0/3008845918 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7680d75c0 con 0x55f2bfe951b0 2024-09-21T06:03:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.595+0000 7fe7847f6740 1 -- 172.21.15.117:0/3008845918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2bfe951b0 msgr2=0x55f2c023a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.595+0000 7fe7847f6740 1 --2- 172.21.15.117:0/3008845918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2bfe951b0 0x55f2c023a9b0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fe7680ca600 tx=0x7fe76803b920 comp rx=0 tx=0).stop 2024-09-21T06:03:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.596+0000 7fe7847f6740 1 -- 172.21.15.117:0/3008845918 shutdown_connections 2024-09-21T06:03:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.596+0000 7fe7847f6740 1 --2- 172.21.15.117:0/3008845918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2bfe744d0 0x55f2c023b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.596+0000 7fe7847f6740 1 --2- 172.21.15.117:0/3008845918 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2bfe6c2f0 0x55f2c023aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.596+0000 7fe7847f6740 1 --2- 172.21.15.117:0/3008845918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2bfe951b0 0x55f2c023a9b0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.596+0000 7fe7847f6740 1 -- 172.21.15.117:0/3008845918 >> 172.21.15.117:0/3008845918 conn(0x55f2bfe89300 msgr2=0x55f2c02388a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.597+0000 7fe7847f6740 1 -- 172.21.15.117:0/3008845918 shutdown_connections 2024-09-21T06:03:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.597+0000 7fe7847f6740 1 -- 172.21.15.117:0/3008845918 wait complete. 2024-09-21T06:03:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.598+0000 7fe7847f6740 1 -- start start 2024-09-21T06:03:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.599+0000 7fe7847f6740 1 Processor -- start 2024-09-21T06:03:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.600+0000 7fe7847f6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2bfe6c2f0 0x55f2c02461d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.600+0000 7fe7847f6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2bfe744d0 0x55f2c0246770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.601+0000 7fe776ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2bfe6c2f0 0x55f2c02461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.601+0000 7fe7767fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2bfe744d0 0x55f2c0246770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.601+0000 7fe776ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2bfe6c2f0 0x55f2c02461d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:46852/0 (socket says 172.21.15.117:46852) 2024-09-21T06:03:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.601+0000 7fe776ffd640 1 -- 172.21.15.117:0/4281957805 learned_addr learned my addr 172.21.15.117:0/4281957805 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.601+0000 7fe7847f6740 1 --2- 172.21.15.117:0/4281957805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2bfe951b0 0x55f2c025c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.602+0000 7fe7847f6740 1 -- 172.21.15.117:0/4281957805 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f2c00d6ab0 con 0x55f2bfe744d0 2024-09-21T06:03:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.602+0000 7fe7847f6740 1 -- 172.21.15.117:0/4281957805 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f2c0144730 con 0x55f2bfe951b0 2024-09-21T06:03:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.602+0000 7fe7847f6740 1 -- 172.21.15.117:0/4281957805 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f2c0248ef0 con 0x55f2bfe6c2f0 2024-09-21T06:03:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.602+0000 7fe7847f6740 10 client.0 _ll_register_callbacks cb 0x55f2c01986b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:03:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.603+0000 7fe7767fc640 1 -- 172.21.15.117:0/4281957805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2bfe6c2f0 msgr2=0x55f2c02461d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.603+0000 7fe7767fc640 1 --2- 172.21.15.117:0/4281957805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2bfe6c2f0 0x55f2c02461d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.603+0000 7fe7767fc640 1 -- 172.21.15.117:0/4281957805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2bfe951b0 msgr2=0x55f2c025c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.603+0000 7fe7767fc640 1 --2- 172.21.15.117:0/4281957805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2bfe951b0 0x55f2c025c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.603+0000 7fe7767fc640 1 -- 172.21.15.117:0/4281957805 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2bff00790 con 0x55f2bfe744d0 2024-09-21T06:03:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.605+0000 7fe7767fc640 1 --2- 172.21.15.117:0/4281957805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2bfe744d0 0x55f2c0246770 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x55f2c0239630 tx=0x7fe76c010770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c80752c2917ad164 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.605+0000 7fe756ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:03:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.606+0000 7fe756ffd640 1 -- 172.21.15.117:0/4281957805 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76c045060 con 0x55f2bfe744d0 2024-09-21T06:03:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.606+0000 7fe756ffd640 1 -- 172.21.15.117:0/4281957805 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fe76c0150d0 con 0x55f2bfe744d0 2024-09-21T06:03:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.606+0000 7fe7847f6740 1 -- 172.21.15.117:0/4281957805 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7680d2b10 con 0x55f2bfe744d0 2024-09-21T06:03:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.610+0000 7fe756ffd640 1 -- 172.21.15.117:0/4281957805 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe76c040420 con 0x55f2bfe744d0 2024-09-21T06:03:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.610+0000 7fe7847f6740 10 client.16554 fetch_fsmap learned FSMap version 27 2024-09-21T06:03:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.610+0000 7fe7847f6740 1 -- 172.21.15.117:0/4281957805 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f2bff79620 con 0x55f2bfe744d0 2024-09-21T06:03:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.611+0000 7fe756ffd640 1 -- 172.21.15.117:0/4281957805 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe76c040a70 con 0x55f2bfe744d0 2024-09-21T06:03:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.611+0000 7fe756ffd640 1 -- 172.21.15.117:0/4281957805 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe76c012070 con 0x55f2bfe744d0 2024-09-21T06:03:51.618 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.612+0000 7fe7847f6740 10 client.16554 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:03:51.618 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.612+0000 7fe7847f6740 10 client.16554 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:03:51.618 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.612+0000 7fe7847f6740 20 client.16554 populate_metadata read hostname 'smithi117' 2024-09-21T06:03:51.618 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.612+0000 7fe7847f6740 1 --2- 172.21.15.117:0/4281957805 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f2bfe53d80 0x55f2c02620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:51.618 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.612+0000 7fe7847f6740 4 client.16554 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:03:51.618 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.612+0000 7fe7847f6740 1 -- 172.21.15.117:0/4281957805 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe7680d2b10 con 0x55f2bfe53d80 2024-09-21T06:03:51.618 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.613+0000 7fe776ffd640 1 --2- 172.21.15.117:0/4281957805 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f2bfe53d80 0x55f2c02620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.620+0000 7fe776ffd640 1 --2- 172.21.15.117:0/4281957805 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f2bfe53d80 0x55f2c02620a0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1efdd807daecebc1 server_cookie=aafc4c5508004c26 in_seq=0 out_seq=0 2024-09-21T06:03:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.620+0000 7fe756ffd640 10 client.16554 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:03:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.627+0000 7fe756ffd640 1 -- 172.21.15.117:0/4281957805 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fe7680d2b10 con 0x55f2bfe53d80 2024-09-21T06:03:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.627+0000 7fe756ffd640 10 client.16554 handle_command_reply: tid=0 2024-09-21T06:03:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.627+0000 7fe7847f6740 1 client.16554 shutdown 2024-09-21T06:03:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.634+0000 7fe7847f6740 1 -- 172.21.15.117:0/4281957805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2bfe744d0 msgr2=0x55f2c0246770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.634+0000 7fe7847f6740 1 --2- 172.21.15.117:0/4281957805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2bfe744d0 0x55f2c0246770 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x55f2c0239630 tx=0x7fe76c010770 comp rx=0 tx=0).stop 2024-09-21T06:03:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.635+0000 7fe7847f6740 1 -- 172.21.15.117:0/4281957805 shutdown_connections 2024-09-21T06:03:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.635+0000 7fe7847f6740 1 --2- 172.21.15.117:0/4281957805 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f2bfe53d80 0x55f2c02620a0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.635+0000 7fe7847f6740 1 --2- 172.21.15.117:0/4281957805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2bfe744d0 0x55f2c0246770 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.635+0000 7fe7847f6740 1 --2- 172.21.15.117:0/4281957805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2bfe951b0 0x55f2c025c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.635+0000 7fe7847f6740 1 --2- 172.21.15.117:0/4281957805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2bfe6c2f0 0x55f2c02461d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.635+0000 7fe7847f6740 1 -- 172.21.15.117:0/4281957805 >> 172.21.15.117:0/4281957805 conn(0x55f2bfe89300 msgr2=0x55f2c023b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.635+0000 7fe756ffd640 0 client.16554 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:03:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.635+0000 7fe7847f6740 1 -- 172.21.15.117:0/4281957805 shutdown_connections 2024-09-21T06:03:51.638 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.635+0000 7fe7847f6740 1 -- 172.21.15.117:0/4281957805 wait complete. 2024-09-21T06:03:51.639 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.636+0000 7fe7847f6740 20 client.16554 trim_cache size 0 max 16384 2024-09-21T06:03:51.700 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.697+0000 7fe7847f6740 1 Processor -- start 2024-09-21T06:03:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.698+0000 7fe7847f6740 1 -- start start 2024-09-21T06:03:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.699+0000 7fe7847f6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2bfe6c2f0 0x55f2c02466b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.699+0000 7fe7847f6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2bfe744d0 0x55f2c0241890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.699+0000 7fe7847f6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2bfe951b0 0x55f2c0240180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.699+0000 7fe7847f6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe7780710e0 con 0x55f2bfe744d0 2024-09-21T06:03:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.699+0000 7fe7847f6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe778072c20 con 0x55f2bfe6c2f0 2024-09-21T06:03:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.699+0000 7fe7847f6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe7780717b0 con 0x55f2bfe951b0 2024-09-21T06:03:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.701+0000 7fe7767fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2bfe744d0 0x55f2c0241890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.701+0000 7fe7767fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2bfe744d0 0x55f2c0241890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45016/0 (socket says 172.21.15.117:45016) 2024-09-21T06:03:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.701+0000 7fe7767fc640 1 -- 172.21.15.117:0/3400164566 learned_addr learned my addr 172.21.15.117:0/3400164566 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.703+0000 7fe7777fe640 1 --2- 172.21.15.117:0/3400164566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2bfe951b0 0x55f2c0240180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.704+0000 7fe776ffd640 1 --2- 172.21.15.117:0/3400164566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2bfe6c2f0 0x55f2c02466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.707+0000 7fe7767fc640 1 -- 172.21.15.117:0/3400164566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2bfe951b0 msgr2=0x55f2c0240180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:51.709 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.707+0000 7fe7767fc640 1 --2- 172.21.15.117:0/3400164566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2bfe951b0 0x55f2c0240180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:51.710 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.707+0000 7fe7767fc640 1 -- 172.21.15.117:0/3400164566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2bfe6c2f0 msgr2=0x55f2c02466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:51.710 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.707+0000 7fe7767fc640 1 --2- 172.21.15.117:0/3400164566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2bfe6c2f0 0x55f2c02466b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:51.710 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.707+0000 7fe7767fc640 1 -- 172.21.15.117:0/3400164566 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2c023d180 con 0x55f2bfe744d0 2024-09-21T06:03:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.718+0000 7fe7767fc640 1 --2- 172.21.15.117:0/3400164566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2bfe744d0 0x55f2c0241890 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe76c0094d0 tx=0x7fe76c015350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c42310d2a4e374 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.720+0000 7fe75d7fa640 1 -- 172.21.15.117:0/3400164566 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76c058070 con 0x55f2bfe744d0 2024-09-21T06:03:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.721+0000 7fe75d7fa640 1 -- 172.21.15.117:0/3400164566 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe76c010c70 con 0x55f2bfe744d0 2024-09-21T06:03:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.721+0000 7fe75d7fa640 1 -- 172.21.15.117:0/3400164566 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76c04f080 con 0x55f2bfe744d0 2024-09-21T06:03:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.722+0000 7fe7847f6740 1 -- 172.21.15.117:0/3400164566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2bfe744d0 msgr2=0x55f2c0241890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.722+0000 7fe7847f6740 1 --2- 172.21.15.117:0/3400164566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2bfe744d0 0x55f2c0241890 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe76c0094d0 tx=0x7fe76c015350 comp rx=0 tx=0).stop 2024-09-21T06:03:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.723+0000 7fe7847f6740 1 -- 172.21.15.117:0/3400164566 shutdown_connections 2024-09-21T06:03:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.723+0000 7fe7847f6740 1 --2- 172.21.15.117:0/3400164566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2bfe744d0 0x55f2c0241890 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.723+0000 7fe7847f6740 1 --2- 172.21.15.117:0/3400164566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2bfe951b0 0x55f2c0240180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.723+0000 7fe7847f6740 1 --2- 172.21.15.117:0/3400164566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2bfe6c2f0 0x55f2c02466b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:51.727 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.723+0000 7fe7847f6740 1 -- 172.21.15.117:0/3400164566 >> 172.21.15.117:0/3400164566 conn(0x55f2bfe89300 msgr2=0x55f2c023fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.726+0000 7fe7847f6740 1 -- 172.21.15.117:0/3400164566 shutdown_connections 2024-09-21T06:03:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.726+0000 7fe7847f6740 1 -- 172.21.15.117:0/3400164566 wait complete. 2024-09-21T06:03:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.727+0000 7fe7847f6740 1 -- start start 2024-09-21T06:03:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.727+0000 7fe7847f6740 1 Processor -- start 2024-09-21T06:03:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.728+0000 7fe7847f6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2bfe6c2f0 0x55f2c02653e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.729+0000 7fe7847f6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2bfe744d0 0x55f2c02659c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.729+0000 7fe7847f6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2bfe951b0 0x55f2c02cac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.729+0000 7fe7847f6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe778072550 con 0x55f2bfe744d0 2024-09-21T06:03:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.729+0000 7fe7847f6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe778071e80 con 0x55f2bfe6c2f0 2024-09-21T06:03:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.729+0000 7fe7847f6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe77806f0e0 con 0x55f2bfe951b0 2024-09-21T06:03:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.729+0000 7fe7847f6740 10 client.0 _ll_register_callbacks cb 0x7fe76c015420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:03:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.729+0000 7fe7767fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2bfe744d0 0x55f2c02659c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.729+0000 7fe7767fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2bfe744d0 0x55f2c02659c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45026/0 (socket says 172.21.15.117:45026) 2024-09-21T06:03:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.729+0000 7fe7767fc640 1 -- 172.21.15.117:0/2880422949 learned_addr learned my addr 172.21.15.117:0/2880422949 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.730+0000 7fe7767fc640 1 -- 172.21.15.117:0/2880422949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2bfe951b0 msgr2=0x55f2c02cac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.730+0000 7fe7767fc640 1 --2- 172.21.15.117:0/2880422949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2bfe951b0 0x55f2c02cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.730+0000 7fe7767fc640 1 -- 172.21.15.117:0/2880422949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2bfe6c2f0 msgr2=0x55f2c02653e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:03:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.730+0000 7fe7767fc640 1 --2- 172.21.15.117:0/2880422949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2bfe6c2f0 0x55f2c02653e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.730+0000 7fe7767fc640 1 -- 172.21.15.117:0/2880422949 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2c02cb2b0 con 0x55f2bfe744d0 2024-09-21T06:03:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.731+0000 7fe7767fc640 1 --2- 172.21.15.117:0/2880422949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2bfe744d0 0x55f2c02659c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fe76c0138b0 tx=0x7fe76c00b150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f706d93ee929d8bc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.731+0000 7fe756ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:03:51.734 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.731+0000 7fe756ffd640 1 -- 172.21.15.117:0/2880422949 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76c058070 con 0x55f2bfe744d0 2024-09-21T06:03:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.732+0000 7fe756ffd640 1 -- 172.21.15.117:0/2880422949 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fe76c004220 con 0x55f2bfe744d0 2024-09-21T06:03:51.735 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.732+0000 7fe7847f6740 1 -- 172.21.15.117:0/2880422949 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe76c010c70 con 0x55f2bfe744d0 2024-09-21T06:03:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.733+0000 7fe756ffd640 1 -- 172.21.15.117:0/2880422949 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe76c015070 con 0x55f2bfe744d0 2024-09-21T06:03:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.734+0000 7fe7847f6740 10 client.16560 fetch_fsmap learned FSMap version 27 2024-09-21T06:03:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.734+0000 7fe7847f6740 1 -- 172.21.15.117:0/2880422949 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f2c02d3770 con 0x55f2bfe744d0 2024-09-21T06:03:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.735+0000 7fe756ffd640 1 -- 172.21.15.117:0/2880422949 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe76c004420 con 0x55f2bfe744d0 2024-09-21T06:03:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.735+0000 7fe756ffd640 1 -- 172.21.15.117:0/2880422949 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe76c012050 con 0x55f2bfe744d0 2024-09-21T06:03:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.735+0000 7fe7847f6740 10 client.16560 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:03:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.735+0000 7fe7847f6740 10 client.16560 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:03:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.735+0000 7fe7847f6740 20 client.16560 populate_metadata read hostname 'smithi117' 2024-09-21T06:03:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.736+0000 7fe7847f6740 1 --2- 172.21.15.117:0/2880422949 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f2bfe291c0 0x55f2c02d5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.736+0000 7fe7847f6740 4 client.16560 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:03:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.736+0000 7fe7847f6740 1 -- 172.21.15.117:0/2880422949 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fe76c010c70 con 0x55f2bfe291c0 2024-09-21T06:03:51.747 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.745+0000 7fe776ffd640 1 --2- 172.21.15.117:0/2880422949 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f2bfe291c0 0x55f2c02d5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.756+0000 7fe776ffd640 1 --2- 172.21.15.117:0/2880422949 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f2bfe291c0 0x55f2c02d5dd0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47d02ffd4abda664 server_cookie=a7fa184988bb5b3d in_seq=0 out_seq=0 2024-09-21T06:03:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:51.756+0000 7fe756ffd640 10 client.16560 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:03:52.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:51 smithi117 ceph-mon[30947]: pgmap v523: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 839 MiB/s wr, 2.26k op/s 2024-09-21T06:03:52.182 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:51 smithi192 ceph-mon[34700]: pgmap v523: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 839 MiB/s wr, 2.26k op/s 2024-09-21T06:03:52.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:51 smithi138 ceph-mon[34391]: pgmap v523: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 839 MiB/s wr, 2.26k op/s 2024-09-21T06:03:52.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.480+0000 7f417e730640 1 -- 172.21.15.117:0/2281340318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41781021f0 msgr2=0x7f4178102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:52.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.480+0000 7f417e730640 1 --2- 172.21.15.117:0/2281340318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41781021f0 0x7f4178102600 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f4168009900 tx=0x7f416802a440 comp rx=0 tx=0).stop 2024-09-21T06:03:52.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.480+0000 7f417e730640 1 -- 172.21.15.117:0/2281340318 shutdown_connections 2024-09-21T06:03:52.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.480+0000 7f417e730640 1 --2- 172.21.15.117:0/2281340318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4178108480 0x7f417810ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:52.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.480+0000 7f417e730640 1 --2- 172.21.15.117:0/2281340318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4178107b30 0x7f4178107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:52.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.480+0000 7f417e730640 1 --2- 172.21.15.117:0/2281340318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41781021f0 0x7f4178102600 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:52.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.480+0000 7f417e730640 1 -- 172.21.15.117:0/2281340318 >> 172.21.15.117:0/2281340318 conn(0x7f4178077c60 msgr2=0x7f41780780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:52.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.481+0000 7f417e730640 1 -- 172.21.15.117:0/2281340318 shutdown_connections 2024-09-21T06:03:52.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.481+0000 7f417e730640 1 -- 172.21.15.117:0/2281340318 wait complete. 2024-09-21T06:03:52.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.482+0000 7f417e730640 1 Processor -- start 2024-09-21T06:03:52.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.482+0000 7f417e730640 1 -- start start 2024-09-21T06:03:52.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.482+0000 7f417e730640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41781021f0 0x7f417819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:52.570 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.483+0000 7f417e730640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4178107b30 0x7f417819acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:52.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.483+0000 7f417e730640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4178108480 0x7f417819ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:52.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.483+0000 7f417e730640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4178112220 con 0x7f41781021f0 2024-09-21T06:03:52.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.483+0000 7f417e730640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f41781120a0 con 0x7f4178107b30 2024-09-21T06:03:52.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.483+0000 7f417e730640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f41781123a0 con 0x7f4178108480 2024-09-21T06:03:52.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.483+0000 7f4177fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41781021f0 0x7f417819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:52.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.483+0000 7f4177fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41781021f0 0x7f417819a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45046/0 (socket says 172.21.15.117:45046) 2024-09-21T06:03:52.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.483+0000 7f4177fff640 1 -- 172.21.15.117:0/1615605975 learned_addr learned my addr 172.21.15.117:0/1615605975 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:52.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.484+0000 7f4177fff640 1 -- 172.21.15.117:0/1615605975 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4178108480 msgr2=0x7f417819ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:52.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.484+0000 7f4177fff640 1 --2- 172.21.15.117:0/1615605975 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4178108480 0x7f417819ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:52.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.484+0000 7f4177fff640 1 -- 172.21.15.117:0/1615605975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4178107b30 msgr2=0x7f417819acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:52.571 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.484+0000 7f4177fff640 1 --2- 172.21.15.117:0/1615605975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4178107b30 0x7f417819acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:52.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.484+0000 7f4177fff640 1 -- 172.21.15.117:0/1615605975 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41781a0680 con 0x7f41781021f0 2024-09-21T06:03:52.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.484+0000 7f4177fff640 1 --2- 172.21.15.117:0/1615605975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41781021f0 0x7f417819a710 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f4168005940 tx=0x7f4168004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:52.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.484+0000 7f41757fa640 1 -- 172.21.15.117:0/1615605975 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416803f030 con 0x7f41781021f0 2024-09-21T06:03:52.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.484+0000 7f41757fa640 1 -- 172.21.15.117:0/1615605975 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4168003aa0 con 0x7f41781021f0 2024-09-21T06:03:52.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.484+0000 7f417e730640 1 -- 172.21.15.117:0/1615605975 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41781a0910 con 0x7f41781021f0 2024-09-21T06:03:52.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.485+0000 7f41757fa640 1 -- 172.21.15.117:0/1615605975 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4168035070 con 0x7f41781021f0 2024-09-21T06:03:52.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.485+0000 7f417e730640 1 -- 172.21.15.117:0/1615605975 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41781a70d0 con 0x7f41781021f0 2024-09-21T06:03:52.572 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.489+0000 7f4184ca4740 1 Processor -- start 2024-09-21T06:03:52.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.489+0000 7f4184ca4740 1 -- start start 2024-09-21T06:03:52.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.490+0000 7f41757fa640 1 -- 172.21.15.117:0/1615605975 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f4168042030 con 0x7f41781021f0 2024-09-21T06:03:52.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.491+0000 7f41757fa640 1 --2- 172.21.15.117:0/1615605975 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f41600b6700 0x7f41600b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:52.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.491+0000 7f4184ca4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56038a951070 0x56038acf86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:52.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.491+0000 7f4184ca4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56038a9281b0 0x56038acf8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:52.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.492+0000 7f4184ca4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56038a930390 0x56038acf9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:52.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.492+0000 7f4184ca4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56038ac6eea0 con 0x56038a951070 2024-09-21T06:03:52.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.492+0000 7f4184ca4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56038ac5e2d0 con 0x56038a9281b0 2024-09-21T06:03:52.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.492+0000 7f4184ca4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56038ac620e0 con 0x56038a930390 2024-09-21T06:03:52.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.492+0000 7f417cccd640 1 --2- 172.21.15.117:0/1615605975 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f41600b6700 0x7f41600b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:52.573 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.492+0000 7f4177fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56038a951070 0x56038acf86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:52.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.492+0000 7f4177fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56038a951070 0x56038acf86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45060/0 (socket says 172.21.15.117:45060) 2024-09-21T06:03:52.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.492+0000 7f4177fff640 1 -- 172.21.15.117:0/454717847 learned_addr learned my addr 172.21.15.117:0/454717847 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:52.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.493+0000 7f417cccd640 1 --2- 172.21.15.117:0/454717847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56038a930390 0x56038acf9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:52.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.493+0000 7f41757fa640 1 -- 172.21.15.117:0/1615605975 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f4168107bd0 con 0x7f41781021f0 2024-09-21T06:03:52.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.493+0000 7f417cccd640 1 --2- 172.21.15.117:0/1615605975 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f41600b6700 0x7f41600b8c10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f417000ab30 tx=0x7f417000af30 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:52.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.493+0000 7f4177fff640 1 -- 172.21.15.117:0/454717847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56038a930390 msgr2=0x56038acf9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:52.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.493+0000 7f4177fff640 1 --2- 172.21.15.117:0/454717847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56038a930390 0x56038acf9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:52.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.493+0000 7f4177fff640 1 -- 172.21.15.117:0/454717847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56038a9281b0 msgr2=0x56038acf8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:52.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.493+0000 7f4177fff640 1 --2- 172.21.15.117:0/454717847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56038a9281b0 0x56038acf8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:52.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.493+0000 7f4177fff640 1 -- 172.21.15.117:0/454717847 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56038a9bc650 con 0x56038a951070 2024-09-21T06:03:52.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.494+0000 7f4177fff640 1 --2- 172.21.15.117:0/454717847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56038a951070 0x56038acf86a0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f4168105310 tx=0x7f416809ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5bdd65fed96837c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:52.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.494+0000 7f415e7fc640 1 -- 172.21.15.117:0/454717847 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4168049070 con 0x56038a951070 2024-09-21T06:03:52.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.495+0000 7f415e7fc640 1 -- 172.21.15.117:0/454717847 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41680cd5b0 con 0x56038a951070 2024-09-21T06:03:52.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.495+0000 7f415e7fc640 1 -- 172.21.15.117:0/454717847 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4168049070 con 0x56038a951070 2024-09-21T06:03:52.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.496+0000 7f4184ca4740 1 -- 172.21.15.117:0/454717847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56038a951070 msgr2=0x56038acf86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:52.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.496+0000 7f4184ca4740 1 --2- 172.21.15.117:0/454717847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56038a951070 0x56038acf86a0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f4168105310 tx=0x7f416809ad80 comp rx=0 tx=0).stop 2024-09-21T06:03:52.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.496+0000 7f4184ca4740 1 -- 172.21.15.117:0/454717847 shutdown_connections 2024-09-21T06:03:52.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.496+0000 7f4184ca4740 1 --2- 172.21.15.117:0/454717847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56038a930390 0x56038acf9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:52.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.496+0000 7f4184ca4740 1 --2- 172.21.15.117:0/454717847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56038a9281b0 0x56038acf8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:52.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.496+0000 7f4184ca4740 1 --2- 172.21.15.117:0/454717847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56038a951070 0x56038acf86a0 secure :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f4168105310 tx=0x7f416809ad80 comp rx=0 tx=0).stop 2024-09-21T06:03:52.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.496+0000 7f4184ca4740 1 -- 172.21.15.117:0/454717847 >> 172.21.15.117:0/454717847 conn(0x56038a945460 msgr2=0x56038acf6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:52.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.497+0000 7f4184ca4740 1 -- 172.21.15.117:0/454717847 shutdown_connections 2024-09-21T06:03:52.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.497+0000 7f4184ca4740 1 -- 172.21.15.117:0/454717847 wait complete. 2024-09-21T06:03:52.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.498+0000 7f4184ca4740 1 -- start start 2024-09-21T06:03:52.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.498+0000 7f4184ca4740 1 Processor -- start 2024-09-21T06:03:52.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.499+0000 7f4184ca4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56038a9281b0 0x56038ad064e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:52.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.499+0000 7f4184ca4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56038a930390 0x56038ad03de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:52.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.499+0000 7f4184ca4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56038a90f9f0 0x56038ad044f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:52.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.499+0000 7f4184ca4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56038aabf700 con 0x56038a9281b0 2024-09-21T06:03:52.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.499+0000 7f4184ca4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56038abf9160 con 0x56038a930390 2024-09-21T06:03:52.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.499+0000 7f4184ca4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56038ad06c40 con 0x56038a90f9f0 2024-09-21T06:03:52.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.499+0000 7f4184ca4740 10 client.0 _ll_register_callbacks cb 0x56038ac129e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:03:52.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.501+0000 7f4177fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56038a9281b0 0x56038ad064e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:52.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.501+0000 7f4177fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56038a9281b0 0x56038ad064e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45068/0 (socket says 172.21.15.117:45068) 2024-09-21T06:03:52.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.501+0000 7f4177fff640 1 -- 172.21.15.117:0/2325005313 learned_addr learned my addr 172.21.15.117:0/2325005313 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:52.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.502+0000 7f417cccd640 1 --2- 172.21.15.117:0/2325005313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56038a90f9f0 0x56038ad044f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:52.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.504+0000 7f4177fff640 1 -- 172.21.15.117:0/2325005313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56038a90f9f0 msgr2=0x56038ad044f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:52.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.504+0000 7f4177fff640 1 --2- 172.21.15.117:0/2325005313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56038a90f9f0 0x56038ad044f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:52.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.504+0000 7f4177fff640 1 -- 172.21.15.117:0/2325005313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56038a930390 msgr2=0x56038ad03de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:52.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.504+0000 7f4177fff640 1 --2- 172.21.15.117:0/2325005313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56038a930390 0x56038ad03de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:52.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.504+0000 7f4177fff640 1 -- 172.21.15.117:0/2325005313 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56038ac26c90 con 0x56038a9281b0 2024-09-21T06:03:52.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.504+0000 7f4177fff640 1 --2- 172.21.15.117:0/2325005313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56038a9281b0 0x56038ad064e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f41680d7880 tx=0x7f4168005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660c7a06e20e4a4c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:52.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.504+0000 7f41577fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:03:52.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.505+0000 7f41577fe640 1 -- 172.21.15.117:0/2325005313 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4168049070 con 0x56038a9281b0 2024-09-21T06:03:52.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.505+0000 7f4184ca4740 1 -- 172.21.15.117:0/2325005313 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41680cd5b0 con 0x56038a9281b0 2024-09-21T06:03:52.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.505+0000 7f41577fe640 1 -- 172.21.15.117:0/2325005313 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f41680c9d10 con 0x56038a9281b0 2024-09-21T06:03:52.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.506+0000 7f41577fe640 1 -- 172.21.15.117:0/2325005313 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f41680c9210 con 0x56038a9281b0 2024-09-21T06:03:52.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.506+0000 7f4184ca4740 10 client.16572 fetch_fsmap learned FSMap version 27 2024-09-21T06:03:52.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.506+0000 7f4184ca4740 1 -- 172.21.15.117:0/2325005313 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56038ad0c8a0 con 0x56038a9281b0 2024-09-21T06:03:52.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.507+0000 7f41577fe640 1 -- 172.21.15.117:0/2325005313 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41680cd5b0 con 0x56038a9281b0 2024-09-21T06:03:52.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.508+0000 7f41577fe640 1 -- 172.21.15.117:0/2325005313 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f41680b8020 con 0x56038a9281b0 2024-09-21T06:03:52.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.508+0000 7f4184ca4740 10 client.16572 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:03:52.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.508+0000 7f4184ca4740 10 client.16572 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T06:03:52.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.509+0000 7f4184ca4740 20 client.16572 populate_metadata read hostname 'smithi117' 2024-09-21T06:03:52.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.509+0000 7f4184ca4740 1 --2- 172.21.15.117:0/2325005313 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x56038a8e4da0 0x56038ad0fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:52.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.509+0000 7f4184ca4740 4 client.16572 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:03:52.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.509+0000 7f4184ca4740 1 -- 172.21.15.117:0/2325005313 --> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56038ac70df0 con 0x56038a8e4da0 2024-09-21T06:03:52.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.510+0000 7f41777fe640 1 --2- 172.21.15.117:0/2325005313 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x56038a8e4da0 0x56038ad0fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:52.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.512+0000 7f41577fe640 10 client.16572 ms_handle_connect on v2:172.21.15.138:6836/2540335374 2024-09-21T06:03:52.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.512+0000 7f41777fe640 1 --2- 172.21.15.117:0/2325005313 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x56038a8e4da0 0x56038ad0fd10 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=577e6105762f3c6f server_cookie=2c133fc29731a63b in_seq=0 out_seq=0 2024-09-21T06:03:52.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.516+0000 7f41577fe640 1 -- 172.21.15.117:0/2325005313 <== mds.4 v2:172.21.15.138:6836/2540335374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56038ac70df0 con 0x56038a8e4da0 2024-09-21T06:03:52.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.516+0000 7f41577fe640 10 client.16572 handle_command_reply: tid=0 2024-09-21T06:03:52.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.516+0000 7f4184ca4740 1 client.16572 shutdown 2024-09-21T06:03:52.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.516+0000 7f4184ca4740 1 -- 172.21.15.117:0/2325005313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56038a9281b0 msgr2=0x56038ad064e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:52.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.516+0000 7f4184ca4740 1 --2- 172.21.15.117:0/2325005313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56038a9281b0 0x56038ad064e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f41680d7880 tx=0x7f4168005e60 comp rx=0 tx=0).stop 2024-09-21T06:03:52.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.516+0000 7f4184ca4740 1 -- 172.21.15.117:0/2325005313 shutdown_connections 2024-09-21T06:03:52.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.517+0000 7f4184ca4740 1 --2- 172.21.15.117:0/2325005313 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x56038a8e4da0 0x56038ad0fd10 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:52.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.517+0000 7f4184ca4740 1 --2- 172.21.15.117:0/2325005313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56038a90f9f0 0x56038ad044f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:52.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.517+0000 7f4184ca4740 1 --2- 172.21.15.117:0/2325005313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56038a930390 0x56038ad03de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:52.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.517+0000 7f4184ca4740 1 --2- 172.21.15.117:0/2325005313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56038a9281b0 0x56038ad064e0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:52.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.517+0000 7f4184ca4740 1 -- 172.21.15.117:0/2325005313 >> 172.21.15.117:0/2325005313 conn(0x56038a945460 msgr2=0x56038acfb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:52.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.517+0000 7f41577fe640 0 client.16572 ms_handle_reset on v2:172.21.15.138:6836/2540335374 2024-09-21T06:03:52.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.518+0000 7f4184ca4740 1 -- 172.21.15.117:0/2325005313 shutdown_connections 2024-09-21T06:03:52.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.518+0000 7f4184ca4740 1 -- 172.21.15.117:0/2325005313 wait complete. 2024-09-21T06:03:52.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.519+0000 7f4184ca4740 20 client.16572 trim_cache size 0 max 16384 2024-09-21T06:03:52.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.547+0000 7f4184ca4740 1 Processor -- start 2024-09-21T06:03:52.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.547+0000 7f4184ca4740 1 -- start start 2024-09-21T06:03:52.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.548+0000 7f4184ca4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56038a90f9f0 0x56038ad09de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:52.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.549+0000 7f4184ca4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56038a9281b0 0x56038ad05c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:52.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.549+0000 7f4184ca4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56038a930390 0x56038ad19290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:52.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.549+0000 7f4184ca4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f41781ad470 con 0x56038a90f9f0 2024-09-21T06:03:52.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.549+0000 7f4184ca4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f41781aefb0 con 0x56038a9281b0 2024-09-21T06:03:52.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.549+0000 7f4184ca4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f41781adb40 con 0x56038a930390 2024-09-21T06:03:52.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.549+0000 7f4177fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56038a90f9f0 0x56038ad09de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:52.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.549+0000 7f4177fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56038a90f9f0 0x56038ad09de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45084/0 (socket says 172.21.15.117:45084) 2024-09-21T06:03:52.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.549+0000 7f41777fe640 1 --2- 172.21.15.117:0/2364158621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56038a9281b0 0x56038ad05c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:52.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.549+0000 7f4177fff640 1 -- 172.21.15.117:0/2364158621 learned_addr learned my addr 172.21.15.117:0/2364158621 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:52.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.550+0000 7f417cccd640 1 --2- 172.21.15.117:0/2364158621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56038a930390 0x56038ad19290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:52.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.550+0000 7f41777fe640 1 -- 172.21.15.117:0/2364158621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56038a930390 msgr2=0x56038ad19290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:52.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.550+0000 7f41777fe640 1 --2- 172.21.15.117:0/2364158621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56038a930390 0x56038ad19290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:52.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.550+0000 7f41777fe640 1 -- 172.21.15.117:0/2364158621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56038a90f9f0 msgr2=0x56038ad09de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:52.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.550+0000 7f41777fe640 1 --2- 172.21.15.117:0/2364158621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56038a90f9f0 0x56038ad09de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:52.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.550+0000 7f41777fe640 1 -- 172.21.15.117:0/2364158621 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56038ad0c8a0 con 0x56038a9281b0 2024-09-21T06:03:52.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.550+0000 7f4177fff640 1 --2- 172.21.15.117:0/2364158621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56038a90f9f0 0x56038ad09de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:03:52.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.551+0000 7f41777fe640 1 --2- 172.21.15.117:0/2364158621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56038a9281b0 0x56038ad05c60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x56038ad06a20 tx=0x7f416c009d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24b1858cf2a86ab2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:52.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.551+0000 7f415dffb640 1 -- 172.21.15.117:0/2364158621 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c01d020 con 0x56038a9281b0 2024-09-21T06:03:52.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.551+0000 7f415dffb640 1 -- 172.21.15.117:0/2364158621 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f416c019410 con 0x56038a9281b0 2024-09-21T06:03:52.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.551+0000 7f415dffb640 1 -- 172.21.15.117:0/2364158621 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c011ea0 con 0x56038a9281b0 2024-09-21T06:03:52.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.554+0000 7f4184ca4740 1 -- 172.21.15.117:0/2364158621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56038a9281b0 msgr2=0x56038ad05c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:52.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.554+0000 7f4184ca4740 1 --2- 172.21.15.117:0/2364158621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56038a9281b0 0x56038ad05c60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x56038ad06a20 tx=0x7f416c009d40 comp rx=0 tx=0).stop 2024-09-21T06:03:52.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.554+0000 7f4184ca4740 1 -- 172.21.15.117:0/2364158621 shutdown_connections 2024-09-21T06:03:52.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.554+0000 7f4184ca4740 1 --2- 172.21.15.117:0/2364158621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56038a930390 0x56038ad19290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:52.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.554+0000 7f4184ca4740 1 --2- 172.21.15.117:0/2364158621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56038a9281b0 0x56038ad05c60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:52.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.555+0000 7f4184ca4740 1 --2- 172.21.15.117:0/2364158621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56038a90f9f0 0x56038ad09de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:52.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.555+0000 7f4184ca4740 1 -- 172.21.15.117:0/2364158621 >> 172.21.15.117:0/2364158621 conn(0x56038a945460 msgr2=0x56038acf96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:52.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.555+0000 7f4184ca4740 1 -- 172.21.15.117:0/2364158621 shutdown_connections 2024-09-21T06:03:52.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.555+0000 7f4184ca4740 1 -- 172.21.15.117:0/2364158621 wait complete. 2024-09-21T06:03:52.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.556+0000 7f4184ca4740 1 -- start start 2024-09-21T06:03:52.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.557+0000 7f4184ca4740 1 Processor -- start 2024-09-21T06:03:52.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.559+0000 7f4184ca4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56038a90f9f0 0x56038ad15500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:52.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.559+0000 7f4184ca4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56038a9281b0 0x56038ad15a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:52.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.559+0000 7f4184ca4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56038a930390 0x56038ad102b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:52.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.560+0000 7f41777fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56038a9281b0 0x56038ad15a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:52.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.560+0000 7f41777fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56038a9281b0 0x56038ad15a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45090/0 (socket says 172.21.15.117:45090) 2024-09-21T06:03:52.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.560+0000 7f41777fe640 1 -- 172.21.15.117:0/2639712476 learned_addr learned my addr 172.21.15.117:0/2639712476 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:52.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.560+0000 7f4184ca4740 1 -- 172.21.15.117:0/2639712476 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f41781ae8e0 con 0x56038a9281b0 2024-09-21T06:03:52.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.560+0000 7f417cccd640 1 --2- 172.21.15.117:0/2639712476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56038a930390 0x56038ad102b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:52.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.560+0000 7f4184ca4740 1 -- 172.21.15.117:0/2639712476 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f41781ae210 con 0x56038a90f9f0 2024-09-21T06:03:52.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.560+0000 7f4184ca4740 1 -- 172.21.15.117:0/2639712476 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f41781ab450 con 0x56038a930390 2024-09-21T06:03:52.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.561+0000 7f4184ca4740 10 client.0 _ll_register_callbacks cb 0x7f417000a800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:03:52.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.561+0000 7f417cccd640 1 -- 172.21.15.117:0/2639712476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56038a90f9f0 msgr2=0x56038ad15500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:52.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.561+0000 7f417cccd640 1 --2- 172.21.15.117:0/2639712476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56038a90f9f0 0x56038ad15500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:52.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.561+0000 7f417cccd640 1 -- 172.21.15.117:0/2639712476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56038a9281b0 msgr2=0x56038ad15a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:52.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.561+0000 7f417cccd640 1 --2- 172.21.15.117:0/2639712476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56038a9281b0 0x56038ad15a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:52.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.561+0000 7f417cccd640 1 -- 172.21.15.117:0/2639712476 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56038ad10920 con 0x56038a930390 2024-09-21T06:03:52.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.566+0000 7f417cccd640 1 --2- 172.21.15.117:0/2639712476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56038a930390 0x56038ad102b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4170012000 tx=0x7f4170004bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2edd1c4931643b78 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:52.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.567+0000 7f41577fe640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-21T06:03:52.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.567+0000 7f41577fe640 1 -- 172.21.15.117:0/2639712476 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4170012030 con 0x56038a930390 2024-09-21T06:03:52.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.567+0000 7f41577fe640 1 -- 172.21.15.117:0/2639712476 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f4170013070 con 0x56038a930390 2024-09-21T06:03:52.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.569+0000 7f4184ca4740 1 -- 172.21.15.117:0/2639712476 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f416c019410 con 0x56038a930390 2024-09-21T06:03:52.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.571+0000 7f41577fe640 1 -- 172.21.15.117:0/2639712476 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4170006e60 con 0x56038a930390 2024-09-21T06:03:52.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.581+0000 7f4184ca4740 10 client.25783 fetch_fsmap learned FSMap version 27 2024-09-21T06:03:52.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.581+0000 7f4184ca4740 1 -- 172.21.15.117:0/2639712476 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56038ad91550 con 0x56038a930390 2024-09-21T06:03:52.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.584+0000 7f41577fe640 1 -- 172.21.15.117:0/2639712476 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4170014070 con 0x56038a930390 2024-09-21T06:03:52.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.584+0000 7f41577fe640 1 -- 172.21.15.117:0/2639712476 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4170004470 con 0x56038a930390 2024-09-21T06:03:52.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.584+0000 7f4184ca4740 10 client.25783 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:03:52.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.584+0000 7f4184ca4740 10 client.25783 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T06:03:52.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.584+0000 7f4184ca4740 20 client.25783 populate_metadata read hostname 'smithi117' 2024-09-21T06:03:52.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.584+0000 7f4184ca4740 1 --2- 172.21.15.117:0/2639712476 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x56038ad91d30 0x56038ad94150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:52.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.584+0000 7f4184ca4740 4 client.25783 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:03:52.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.584+0000 7f4184ca4740 1 -- 172.21.15.117:0/2639712476 --> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f416c019410 con 0x56038ad91d30 2024-09-21T06:03:52.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.588+0000 7f4177fff640 1 --2- 172.21.15.117:0/2639712476 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x56038ad91d30 0x56038ad94150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:52.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.593+0000 7f4177fff640 1 --2- 172.21.15.117:0/2639712476 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x56038ad91d30 0x56038ad94150 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a5ff62312b142ca0 server_cookie=9136958f68520a7a in_seq=0 out_seq=0 2024-09-21T06:03:52.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.593+0000 7f41577fe640 10 client.25783 ms_handle_connect on v2:172.21.15.138:6836/2540335374 2024-09-21T06:03:52.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.593+0000 7f41577fe640 1 -- 172.21.15.117:0/2639712476 <== mds.4 v2:172.21.15.138:6836/2540335374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f416c019410 con 0x56038ad91d30 2024-09-21T06:03:52.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.593+0000 7f41577fe640 10 client.25783 handle_command_reply: tid=0 2024-09-21T06:03:52.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.593+0000 7f4184ca4740 1 client.25783 shutdown 2024-09-21T06:03:52.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.595+0000 7f4184ca4740 1 -- 172.21.15.117:0/2639712476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56038a930390 msgr2=0x56038ad102b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:52.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.595+0000 7f4184ca4740 1 --2- 172.21.15.117:0/2639712476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56038a930390 0x56038ad102b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4170012000 tx=0x7f4170004bd0 comp rx=0 tx=0).stop 2024-09-21T06:03:52.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.596+0000 7f4184ca4740 1 -- 172.21.15.117:0/2639712476 shutdown_connections 2024-09-21T06:03:52.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.596+0000 7f4184ca4740 1 --2- 172.21.15.117:0/2639712476 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x56038ad91d30 0x56038ad94150 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:52.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.596+0000 7f41577fe640 0 client.25783 ms_handle_reset on v2:172.21.15.138:6836/2540335374 2024-09-21T06:03:52.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.596+0000 7f4184ca4740 1 --2- 172.21.15.117:0/2639712476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56038a9281b0 0x56038ad15a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:52.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.596+0000 7f4184ca4740 1 --2- 172.21.15.117:0/2639712476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56038a930390 0x56038ad102b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:52.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.596+0000 7f4184ca4740 1 --2- 172.21.15.117:0/2639712476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56038a90f9f0 0x56038ad15500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:52.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.597+0000 7f4184ca4740 1 -- 172.21.15.117:0/2639712476 >> 172.21.15.117:0/2639712476 conn(0x56038a945460 msgr2=0x56038acfa090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:52.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.597+0000 7f4184ca4740 1 -- 172.21.15.117:0/2639712476 shutdown_connections 2024-09-21T06:03:52.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.597+0000 7f4184ca4740 1 -- 172.21.15.117:0/2639712476 wait complete. 2024-09-21T06:03:52.600 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.597+0000 7f4184ca4740 20 client.25783 trim_cache size 0 max 16384 2024-09-21T06:03:52.600 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:03:52.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.598+0000 7f417e730640 1 -- 172.21.15.117:0/1615605975 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f41600b6700 msgr2=0x7f41600b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:52.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.598+0000 7f417e730640 1 --2- 172.21.15.117:0/1615605975 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f41600b6700 0x7f41600b8c10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f417000ab30 tx=0x7f417000af30 comp rx=0 tx=0).stop 2024-09-21T06:03:52.601 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.598+0000 7f417e730640 1 -- 172.21.15.117:0/1615605975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41781021f0 msgr2=0x7f417819a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:52.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.598+0000 7f417e730640 1 --2- 172.21.15.117:0/1615605975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41781021f0 0x7f417819a710 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f4168005940 tx=0x7f4168004270 comp rx=0 tx=0).stop 2024-09-21T06:03:52.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.598+0000 7f417e730640 1 -- 172.21.15.117:0/1615605975 shutdown_connections 2024-09-21T06:03:52.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.598+0000 7f417e730640 1 --2- 172.21.15.117:0/1615605975 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f41600b6700 0x7f41600b8c10 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:52.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.599+0000 7f417e730640 1 --2- 172.21.15.117:0/1615605975 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4178108480 0x7f417819ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:52.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.599+0000 7f417e730640 1 --2- 172.21.15.117:0/1615605975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4178107b30 0x7f417819acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:52.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.599+0000 7f417e730640 1 --2- 172.21.15.117:0/1615605975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41781021f0 0x7f417819a710 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:52.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.599+0000 7f417e730640 1 -- 172.21.15.117:0/1615605975 >> 172.21.15.117:0/1615605975 conn(0x7f4178077c60 msgr2=0x7f4178108af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:52.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.599+0000 7f417e730640 1 -- 172.21.15.117:0/1615605975 shutdown_connections 2024-09-21T06:03:52.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:52.599+0000 7f417e730640 1 -- 172.21.15.117:0/1615605975 wait complete. 2024-09-21T06:03:54.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:53 smithi117 ceph-mon[30947]: pgmap v524: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 822 MiB/s wr, 2.12k op/s 2024-09-21T06:03:54.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:53 smithi192 ceph-mon[34700]: pgmap v524: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 822 MiB/s wr, 2.12k op/s 2024-09-21T06:03:54.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:53 smithi138 ceph-mon[34391]: pgmap v524: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 822 MiB/s wr, 2.12k op/s 2024-09-21T06:03:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:54.442+0000 7fe756ffd640 1 -- 172.21.15.117:0/2880422949 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fe76c010c70 con 0x55f2bfe291c0 2024-09-21T06:03:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:54.442+0000 7fe756ffd640 10 client.16560 handle_command_reply: tid=0 2024-09-21T06:03:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:54.443+0000 7fe7847f6740 1 client.16560 shutdown 2024-09-21T06:03:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:54.443+0000 7fe7847f6740 1 -- 172.21.15.117:0/2880422949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2bfe744d0 msgr2=0x55f2c02659c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:54.443+0000 7fe7847f6740 1 --2- 172.21.15.117:0/2880422949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2bfe744d0 0x55f2c02659c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fe76c0138b0 tx=0x7fe76c00b150 comp rx=0 tx=0).stop 2024-09-21T06:03:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:54.444+0000 7fe7847f6740 1 -- 172.21.15.117:0/2880422949 shutdown_connections 2024-09-21T06:03:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:54.444+0000 7fe7847f6740 1 --2- 172.21.15.117:0/2880422949 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f2bfe291c0 0x55f2c02d5dd0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:54.444+0000 7fe7847f6740 1 --2- 172.21.15.117:0/2880422949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f2bfe744d0 0x55f2c02659c0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:54.444+0000 7fe7847f6740 1 --2- 172.21.15.117:0/2880422949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2bfe951b0 0x55f2c02cac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:54.444+0000 7fe7847f6740 1 --2- 172.21.15.117:0/2880422949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f2bfe6c2f0 0x55f2c02653e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:54.444+0000 7fe7847f6740 1 -- 172.21.15.117:0/2880422949 >> 172.21.15.117:0/2880422949 conn(0x55f2bfe89300 msgr2=0x55f2c0238200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:54.444+0000 7fe756ffd640 0 client.16560 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:03:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:54.445+0000 7fe7847f6740 1 -- 172.21.15.117:0/2880422949 shutdown_connections 2024-09-21T06:03:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:54.445+0000 7fe7847f6740 1 -- 172.21.15.117:0/2880422949 wait complete. 2024-09-21T06:03:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:54.445+0000 7fe7847f6740 20 client.16560 trim_cache size 0 max 16384 2024-09-21T06:03:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T06:03:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T06:03:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T06:03:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T06:03:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:54.447+0000 7fe77d88b640 1 -- 172.21.15.117:0/2553430036 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe7600b6680 msgr2=0x7fe7600b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:54.447+0000 7fe77d88b640 1 --2- 172.21.15.117:0/2553430036 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe7600b6680 0x7fe7600b8b90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe77807b1c0 tx=0x7fe76c006c40 comp rx=0 tx=0).stop 2024-09-21T06:03:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:54.447+0000 7fe77d88b640 1 -- 172.21.15.117:0/2553430036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7781029b0 msgr2=0x7fe778199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:54.447+0000 7fe77d88b640 1 --2- 172.21.15.117:0/2553430036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7781029b0 0x7fe778199200 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe76802aa00 tx=0x7fe768003d50 comp rx=0 tx=0).stop 2024-09-21T06:03:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:54.447+0000 7fe77d88b640 1 -- 172.21.15.117:0/2553430036 shutdown_connections 2024-09-21T06:03:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:54.447+0000 7fe77d88b640 1 --2- 172.21.15.117:0/2553430036 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe7600b6680 0x7fe7600b8b90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:54.447+0000 7fe77d88b640 1 --2- 172.21.15.117:0/2553430036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe778108c30 0x7fe778199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:54.447+0000 7fe77d88b640 1 --2- 172.21.15.117:0/2553430036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7781082e0 0x7fe778199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:54.447+0000 7fe77d88b640 1 --2- 172.21.15.117:0/2553430036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7781029b0 0x7fe778199200 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:54.448+0000 7fe77d88b640 1 -- 172.21.15.117:0/2553430036 >> 172.21.15.117:0/2553430036 conn(0x7fe77807a740 msgr2=0x7fe778106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:54.449+0000 7fe77d88b640 1 -- 172.21.15.117:0/2553430036 shutdown_connections 2024-09-21T06:03:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:03:54.449+0000 7fe77d88b640 1 -- 172.21.15.117:0/2553430036 wait complete. 2024-09-21T06:03:55.505 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T06:03:56.030 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:55 smithi117 ceph-mon[30947]: pgmap v525: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 895 MiB/s wr, 2.13k op/s 2024-09-21T06:03:56.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:55 smithi192 ceph-mon[34700]: pgmap v525: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 895 MiB/s wr, 2.13k op/s 2024-09-21T06:03:56.297 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:03:56.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:55 smithi138 ceph-mon[34391]: pgmap v525: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 895 MiB/s wr, 2.13k op/s 2024-09-21T06:03:58.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:57 smithi192 ceph-mon[34700]: pgmap v526: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 864 MiB/s wr, 1.99k op/s 2024-09-21T06:03:58.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:57 smithi117 ceph-mon[30947]: pgmap v526: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 864 MiB/s wr, 1.99k op/s 2024-09-21T06:03:58.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:57 smithi138 ceph-mon[34391]: pgmap v526: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 864 MiB/s wr, 1.99k op/s 2024-09-21T06:03:59.009 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:03:59.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.764+0000 7f109d3e0640 1 -- 172.21.15.117:0/3893354907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f109810a650 msgr2=0x7f109810aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:59.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.764+0000 7f109d3e0640 1 --2- 172.21.15.117:0/3893354907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f109810a650 0x7f109810aa60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f108800b880 tx=0x7f1088005f90 comp rx=0 tx=0).stop 2024-09-21T06:03:59.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.766+0000 7f109d3e0640 1 -- 172.21.15.117:0/3893354907 shutdown_connections 2024-09-21T06:03:59.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.766+0000 7f109d3e0640 1 --2- 172.21.15.117:0/3893354907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f109810fcf0 0x7f1098110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:59.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.766+0000 7f109d3e0640 1 --2- 172.21.15.117:0/3893354907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1098110640 0x7f1098116e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:59.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.766+0000 7f109d3e0640 1 --2- 172.21.15.117:0/3893354907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f109810a650 0x7f109810aa60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:59.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.766+0000 7f109d3e0640 1 -- 172.21.15.117:0/3893354907 >> 172.21.15.117:0/3893354907 conn(0x7f109806ee50 msgr2=0x7f109806f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:59.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.766+0000 7f109d3e0640 1 -- 172.21.15.117:0/3893354907 shutdown_connections 2024-09-21T06:03:59.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.766+0000 7f109d3e0640 1 -- 172.21.15.117:0/3893354907 wait complete. 2024-09-21T06:03:59.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.769+0000 7f109d3e0640 1 Processor -- start 2024-09-21T06:03:59.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.769+0000 7f109d3e0640 1 -- start start 2024-09-21T06:03:59.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.770+0000 7f109d3e0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f109810a650 0x7f10981a9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:59.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.770+0000 7f109d3e0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f109810fcf0 0x7f10981a9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:59.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.770+0000 7f109d3e0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1098110640 0x7f10981ae9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:59.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.770+0000 7f109d3e0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1098122730 con 0x7f1098110640 2024-09-21T06:03:59.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.770+0000 7f109d3e0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f10981225b0 con 0x7f109810a650 2024-09-21T06:03:59.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.770+0000 7f109d3e0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f10981228b0 con 0x7f109810fcf0 2024-09-21T06:03:59.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.778+0000 7f10977fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f109810fcf0 0x7f10981a9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:59.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.778+0000 7f109cbdf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1098110640 0x7f10981ae9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:59.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.778+0000 7f109cbdf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1098110640 0x7f10981ae9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43794/0 (socket says 172.21.15.117:43794) 2024-09-21T06:03:59.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.778+0000 7f109cbdf640 1 -- 172.21.15.117:0/3509227458 learned_addr learned my addr 172.21.15.117:0/3509227458 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:59.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.779+0000 7f1097fff640 1 --2- 172.21.15.117:0/3509227458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f109810a650 0x7f10981a9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:59.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.781+0000 7f109cbdf640 1 -- 172.21.15.117:0/3509227458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f109810fcf0 msgr2=0x7f10981a9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:59.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.781+0000 7f109cbdf640 1 --2- 172.21.15.117:0/3509227458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f109810fcf0 0x7f10981a9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:59.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.781+0000 7f109cbdf640 1 -- 172.21.15.117:0/3509227458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f109810a650 msgr2=0x7f10981a9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:59.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.781+0000 7f109cbdf640 1 --2- 172.21.15.117:0/3509227458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f109810a650 0x7f10981a9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:59.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.781+0000 7f109cbdf640 1 -- 172.21.15.117:0/3509227458 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10981af0d0 con 0x7f1098110640 2024-09-21T06:03:59.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.786+0000 7f109cbdf640 1 --2- 172.21.15.117:0/3509227458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1098110640 0x7f10981ae9a0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f109000be80 tx=0x7f10900095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:59.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.786+0000 7f10957fa640 1 -- 172.21.15.117:0/3509227458 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109001a020 con 0x7f1098110640 2024-09-21T06:03:59.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.787+0000 7f10957fa640 1 -- 172.21.15.117:0/3509227458 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1090009c40 con 0x7f1098110640 2024-09-21T06:03:59.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.787+0000 7f109d3e0640 1 -- 172.21.15.117:0/3509227458 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10981af3c0 con 0x7f1098110640 2024-09-21T06:03:59.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.787+0000 7f109d3e0640 1 -- 172.21.15.117:0/3509227458 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10981b5b20 con 0x7f1098110640 2024-09-21T06:03:59.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.787+0000 7f10957fa640 1 -- 172.21.15.117:0/3509227458 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10900057c0 con 0x7f1098110640 2024-09-21T06:03:59.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.802+0000 7f10a4bb6740 1 Processor -- start 2024-09-21T06:03:59.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.814+0000 7f10957fa640 1 -- 172.21.15.117:0/3509227458 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f1090004a40 con 0x7f1098110640 2024-09-21T06:03:59.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.815+0000 7f10a4bb6740 1 -- start start 2024-09-21T06:03:59.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.815+0000 7f10a4bb6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ac06fa070 0x558ac0aa1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:59.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.815+0000 7f10a4bb6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ac06d11b0 0x558ac0aa19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:59.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.815+0000 7f10a4bb6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ac06d9390 0x558ac0aa1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:59.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.815+0000 7f10a4bb6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558ac0a17ea0 con 0x558ac06d9390 2024-09-21T06:03:59.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.815+0000 7f10a4bb6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558ac0a072d0 con 0x558ac06d11b0 2024-09-21T06:03:59.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.815+0000 7f10a4bb6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558ac0a0b0e0 con 0x558ac06fa070 2024-09-21T06:03:59.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.816+0000 7f10977fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ac06d9390 0x558ac0aa1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:59.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.816+0000 7f10977fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ac06d9390 0x558ac0aa1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43800/0 (socket says 172.21.15.117:43800) 2024-09-21T06:03:59.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.816+0000 7f10977fe640 1 -- 172.21.15.117:0/3833559854 learned_addr learned my addr 172.21.15.117:0/3833559854 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:59.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.816+0000 7f10977fe640 1 --2- 172.21.15.117:0/3833559854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ac06fa070 0x558ac0aa1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:59.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.817+0000 7f10957fa640 1 --2- 172.21.15.117:0/3509227458 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f10800b6880 0x7f10800b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:59.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.818+0000 7f10977fe640 1 -- 172.21.15.117:0/3833559854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ac06d11b0 msgr2=0x558ac0aa19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:59.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.818+0000 7f10977fe640 1 --2- 172.21.15.117:0/3833559854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ac06d11b0 0x558ac0aa19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:59.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.818+0000 7f10977fe640 1 -- 172.21.15.117:0/3833559854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ac06d9390 msgr2=0x558ac0aa1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:59.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.818+0000 7f10977fe640 1 --2- 172.21.15.117:0/3833559854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ac06d9390 0x558ac0aa1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:59.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.818+0000 7f10977fe640 1 -- 172.21.15.117:0/3833559854 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ac0765650 con 0x558ac06fa070 2024-09-21T06:03:59.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.818+0000 7f109cbdf640 1 --2- 172.21.15.117:0/3509227458 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f10800b6880 0x7f10800b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:59.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.818+0000 7f10977fe640 1 --2- 172.21.15.117:0/3833559854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ac06fa070 0x558ac0aa1460 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f108c018830 tx=0x7f108c00c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56115816411858e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:59.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.821+0000 7f107e7fc640 1 -- 172.21.15.117:0/3833559854 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108c004230 con 0x558ac06fa070 2024-09-21T06:03:59.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.821+0000 7f107e7fc640 1 -- 172.21.15.117:0/3833559854 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f108c01ae90 con 0x558ac06fa070 2024-09-21T06:03:59.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.821+0000 7f107e7fc640 1 -- 172.21.15.117:0/3833559854 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108c005580 con 0x558ac06fa070 2024-09-21T06:03:59.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.826+0000 7f10a4bb6740 1 -- 172.21.15.117:0/3833559854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ac06fa070 msgr2=0x558ac0aa1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:59.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.826+0000 7f10a4bb6740 1 --2- 172.21.15.117:0/3833559854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ac06fa070 0x558ac0aa1460 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f108c018830 tx=0x7f108c00c040 comp rx=0 tx=0).stop 2024-09-21T06:03:59.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.821+0000 7f10957fa640 1 -- 172.21.15.117:0/3509227458 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f10900e7240 con 0x7f1098110640 2024-09-21T06:03:59.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.833+0000 7f10a4bb6740 1 -- 172.21.15.117:0/3833559854 shutdown_connections 2024-09-21T06:03:59.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.833+0000 7f10a4bb6740 1 --2- 172.21.15.117:0/3833559854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ac06d9390 0x558ac0aa1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:59.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.833+0000 7f10a4bb6740 1 --2- 172.21.15.117:0/3833559854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ac06d11b0 0x558ac0aa19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:59.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.833+0000 7f10a4bb6740 1 --2- 172.21.15.117:0/3833559854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ac06fa070 0x558ac0aa1460 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:59.014 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.833+0000 7f10a4bb6740 1 -- 172.21.15.117:0/3833559854 >> 172.21.15.117:0/3833559854 conn(0x558ac06ee460 msgr2=0x558ac0a9f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:59.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.834+0000 7f109cbdf640 1 --2- 172.21.15.117:0/3509227458 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f10800b6880 0x7f10800b8d90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f10900ad0d0 tx=0x7f109000dad0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:59.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.837+0000 7f10a4bb6740 1 -- 172.21.15.117:0/3833559854 shutdown_connections 2024-09-21T06:03:59.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.839+0000 7f10a4bb6740 1 -- 172.21.15.117:0/3833559854 wait complete. 2024-09-21T06:03:59.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.840+0000 7f10a4bb6740 1 -- start start 2024-09-21T06:03:59.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.840+0000 7f10a4bb6740 1 Processor -- start 2024-09-21T06:03:59.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.841+0000 7f10a4bb6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ac06d11b0 0x558ac0aacbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:59.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.841+0000 7f10a4bb6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ac06d9390 0x558ac0aad120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:59.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.841+0000 7f10a4bb6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ac06fa070 0x558ac0ac9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:59.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.841+0000 7f10a4bb6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558ac0868700 con 0x558ac06d11b0 2024-09-21T06:03:59.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.841+0000 7f10a4bb6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558ac09a2160 con 0x558ac06d9390 2024-09-21T06:03:59.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.841+0000 7f10a4bb6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558ac0ac0bf0 con 0x558ac06fa070 2024-09-21T06:03:59.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.841+0000 7f10a4bb6740 10 client.0 _ll_register_callbacks cb 0x558ac09bb9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:03:59.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.849+0000 7f10977fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ac06fa070 0x558ac0ac9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:59.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.849+0000 7f10977fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ac06d11b0 0x558ac0aacbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:59.015 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.849+0000 7f10977fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ac06d11b0 0x558ac0aacbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43816/0 (socket says 172.21.15.117:43816) 2024-09-21T06:03:59.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.849+0000 7f10977fe640 1 -- 172.21.15.117:0/743839556 learned_addr learned my addr 172.21.15.117:0/743839556 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:59.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.849+0000 7f1097fff640 1 --2- 172.21.15.117:0/743839556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ac06d9390 0x558ac0aad120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:59.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.854+0000 7f10977fe640 1 -- 172.21.15.117:0/743839556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ac06d9390 msgr2=0x558ac0aad120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:59.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.854+0000 7f10977fe640 1 --2- 172.21.15.117:0/743839556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ac06d9390 0x558ac0aad120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:59.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.854+0000 7f10977fe640 1 -- 172.21.15.117:0/743839556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ac06d11b0 msgr2=0x558ac0aacbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:59.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.855+0000 7f10977fe640 1 --2- 172.21.15.117:0/743839556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ac06d11b0 0x558ac0aacbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:59.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.855+0000 7f10977fe640 1 -- 172.21.15.117:0/743839556 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ac09cfc90 con 0x558ac06fa070 2024-09-21T06:03:59.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.858+0000 7f10977fe640 1 --2- 172.21.15.117:0/743839556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ac06fa070 0x558ac0ac9f40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f108c00a8f0 tx=0x7f108c005160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6f39ce6ebbad8c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:59.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.859+0000 7f10777fe640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-21T06:03:59.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.860+0000 7f10777fe640 1 -- 172.21.15.117:0/743839556 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108c004020 con 0x558ac06fa070 2024-09-21T06:03:59.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.860+0000 7f10777fe640 1 -- 172.21.15.117:0/743839556 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f108c0041c0 con 0x558ac06fa070 2024-09-21T06:03:59.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.860+0000 7f10a4bb6740 1 -- 172.21.15.117:0/743839556 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f108c01ae90 con 0x558ac06fa070 2024-09-21T06:03:59.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.864+0000 7f10777fe640 1 -- 172.21.15.117:0/743839556 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f108c0043c0 con 0x558ac06fa070 2024-09-21T06:03:59.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.869+0000 7f10a4bb6740 10 client.25795 fetch_fsmap learned FSMap version 27 2024-09-21T06:03:59.016 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.869+0000 7f10a4bb6740 1 -- 172.21.15.117:0/743839556 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558ac0acc9f0 con 0x558ac06fa070 2024-09-21T06:03:59.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.879+0000 7f10777fe640 1 -- 172.21.15.117:0/743839556 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f108c01ae90 con 0x558ac06fa070 2024-09-21T06:03:59.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.879+0000 7f10777fe640 1 -- 172.21.15.117:0/743839556 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f108c034070 con 0x558ac06fa070 2024-09-21T06:03:59.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.879+0000 7f10a4bb6740 10 client.25795 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:03:59.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.879+0000 7f10a4bb6740 10 client.25795 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T06:03:59.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.879+0000 7f10a4bb6740 20 client.25795 populate_metadata read hostname 'smithi117' 2024-09-21T06:03:59.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.879+0000 7f10a4bb6740 1 --2- 172.21.15.117:0/743839556 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x558ac06b89f0 0x558ac0ad0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:59.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.879+0000 7f10a4bb6740 4 client.25795 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:03:59.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.879+0000 7f10a4bb6740 1 -- 172.21.15.117:0/743839556 --> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ac0a19df0 con 0x558ac06b89f0 2024-09-21T06:03:59.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.881+0000 7f1097fff640 1 --2- 172.21.15.117:0/743839556 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x558ac06b89f0 0x558ac0ad0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:59.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.885+0000 7f10777fe640 10 client.25795 ms_handle_connect on v2:172.21.15.192:6834/3530688860 2024-09-21T06:03:59.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.887+0000 7f1097fff640 1 --2- 172.21.15.117:0/743839556 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x558ac06b89f0 0x558ac0ad0390 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7692d7276a8d29ca server_cookie=2e5ddaf4ee2d5cae in_seq=0 out_seq=0 2024-09-21T06:03:59.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.888+0000 7f10777fe640 1 -- 172.21.15.117:0/743839556 <== mds.3 v2:172.21.15.192:6834/3530688860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558ac0a19df0 con 0x558ac06b89f0 2024-09-21T06:03:59.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.889+0000 7f10777fe640 10 client.25795 handle_command_reply: tid=0 2024-09-21T06:03:59.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.889+0000 7f10a4bb6740 1 client.25795 shutdown 2024-09-21T06:03:59.017 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.889+0000 7f10a4bb6740 1 -- 172.21.15.117:0/743839556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ac06fa070 msgr2=0x558ac0ac9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:59.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.889+0000 7f10a4bb6740 1 --2- 172.21.15.117:0/743839556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ac06fa070 0x558ac0ac9f40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f108c00a8f0 tx=0x7f108c005160 comp rx=0 tx=0).stop 2024-09-21T06:03:59.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.889+0000 7f10a4bb6740 1 -- 172.21.15.117:0/743839556 shutdown_connections 2024-09-21T06:03:59.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.889+0000 7f10a4bb6740 1 --2- 172.21.15.117:0/743839556 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x558ac06b89f0 0x558ac0ad0390 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:59.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.889+0000 7f10a4bb6740 1 --2- 172.21.15.117:0/743839556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ac06fa070 0x558ac0ac9f40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:59.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.889+0000 7f10a4bb6740 1 --2- 172.21.15.117:0/743839556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ac06d9390 0x558ac0aad120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:59.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.889+0000 7f10a4bb6740 1 --2- 172.21.15.117:0/743839556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ac06d11b0 0x558ac0aacbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:59.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.889+0000 7f10a4bb6740 1 -- 172.21.15.117:0/743839556 >> 172.21.15.117:0/743839556 conn(0x558ac06ee460 msgr2=0x558ac0aa45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:59.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.889+0000 7f10777fe640 0 client.25795 ms_handle_reset on v2:172.21.15.192:6834/3530688860 2024-09-21T06:03:59.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.889+0000 7f10a4bb6740 1 -- 172.21.15.117:0/743839556 shutdown_connections 2024-09-21T06:03:59.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.889+0000 7f10a4bb6740 1 -- 172.21.15.117:0/743839556 wait complete. 2024-09-21T06:03:59.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.890+0000 7f10a4bb6740 20 client.25795 trim_cache size 0 max 16384 2024-09-21T06:03:59.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.937+0000 7f10a4bb6740 1 Processor -- start 2024-09-21T06:03:59.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.938+0000 7f10a4bb6740 1 -- start start 2024-09-21T06:03:59.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.939+0000 7f10a4bb6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ac06d11b0 0x558ac0ac9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:59.018 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.939+0000 7f10a4bb6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ac06d9390 0x558ac0a9e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:59.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.939+0000 7f10a4bb6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ac06fa070 0x558ac0a9eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:59.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.939+0000 7f1097fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ac06d9390 0x558ac0a9e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:59.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.939+0000 7f1097fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ac06d9390 0x558ac0a9e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43830/0 (socket says 172.21.15.117:43830) 2024-09-21T06:03:59.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.939+0000 7f1097fff640 1 -- 172.21.15.117:0/3949562866 learned_addr learned my addr 172.21.15.117:0/3949562866 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:59.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.940+0000 7f10a4bb6740 1 -- 172.21.15.117:0/3949562866 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f108400e700 con 0x558ac06d9390 2024-09-21T06:03:59.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.940+0000 7f10a4bb6740 1 -- 172.21.15.117:0/3949562866 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1084010240 con 0x558ac06d11b0 2024-09-21T06:03:59.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.940+0000 7f10a4bb6740 1 -- 172.21.15.117:0/3949562866 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f108400edd0 con 0x558ac06fa070 2024-09-21T06:03:59.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.941+0000 7f10977fe640 1 --2- 172.21.15.117:0/3949562866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ac06d11b0 0x558ac0ac9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:59.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.941+0000 7f10977fe640 1 --2- 172.21.15.117:0/3949562866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ac06fa070 0x558ac0a9eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:59.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.942+0000 7f1097fff640 1 -- 172.21.15.117:0/3949562866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ac06fa070 msgr2=0x558ac0a9eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:59.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.942+0000 7f1097fff640 1 --2- 172.21.15.117:0/3949562866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ac06fa070 0x558ac0a9eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:59.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.942+0000 7f1097fff640 1 -- 172.21.15.117:0/3949562866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ac06d11b0 msgr2=0x558ac0ac9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:59.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.942+0000 7f1097fff640 1 --2- 172.21.15.117:0/3949562866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ac06d11b0 0x558ac0ac9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:59.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.942+0000 7f1097fff640 1 -- 172.21.15.117:0/3949562866 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ac0aaf320 con 0x558ac06d9390 2024-09-21T06:03:59.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.944+0000 7f1097fff640 1 --2- 172.21.15.117:0/3949562866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ac06d9390 0x558ac0a9e980 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f108802a490 tx=0x7f1088032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae4556dbdb86972 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:59.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.947+0000 7f107dffb640 1 -- 172.21.15.117:0/3949562866 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1088004040 con 0x558ac06d9390 2024-09-21T06:03:59.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.947+0000 7f107dffb640 1 -- 172.21.15.117:0/3949562866 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f108802ac90 con 0x558ac06d9390 2024-09-21T06:03:59.020 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.947+0000 7f107dffb640 1 -- 172.21.15.117:0/3949562866 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1088032550 con 0x558ac06d9390 2024-09-21T06:03:59.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.949+0000 7f10a4bb6740 1 -- 172.21.15.117:0/3949562866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ac06d9390 msgr2=0x558ac0a9e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:59.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.949+0000 7f10a4bb6740 1 --2- 172.21.15.117:0/3949562866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ac06d9390 0x558ac0a9e980 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f108802a490 tx=0x7f1088032da0 comp rx=0 tx=0).stop 2024-09-21T06:03:59.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.950+0000 7f10a4bb6740 1 -- 172.21.15.117:0/3949562866 shutdown_connections 2024-09-21T06:03:59.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.950+0000 7f10a4bb6740 1 --2- 172.21.15.117:0/3949562866 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ac06d9390 0x558ac0a9e980 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:59.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.950+0000 7f10a4bb6740 1 --2- 172.21.15.117:0/3949562866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ac06fa070 0x558ac0a9eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:59.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.951+0000 7f10a4bb6740 1 --2- 172.21.15.117:0/3949562866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ac06d11b0 0x558ac0ac9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:59.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.951+0000 7f10a4bb6740 1 -- 172.21.15.117:0/3949562866 >> 172.21.15.117:0/3949562866 conn(0x558ac06ee460 msgr2=0x558ac0aab460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:59.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.953+0000 7f10a4bb6740 1 -- 172.21.15.117:0/3949562866 shutdown_connections 2024-09-21T06:03:59.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.954+0000 7f10a4bb6740 1 -- 172.21.15.117:0/3949562866 wait complete. 2024-09-21T06:03:59.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.955+0000 7f10a4bb6740 1 -- start start 2024-09-21T06:03:59.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.956+0000 7f10a4bb6740 1 Processor -- start 2024-09-21T06:03:59.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.957+0000 7f10a4bb6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ac06d11b0 0x558ac0ac1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:59.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.957+0000 7f10a4bb6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ac06d9390 0x558ac0ac17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:59.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.957+0000 7f10a4bb6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ac06fa070 0x558ac0b38d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:59.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.957+0000 7f10a4bb6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f108400fb70 con 0x558ac06fa070 2024-09-21T06:03:59.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.957+0000 7f10a4bb6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f108400f4a0 con 0x558ac06d11b0 2024-09-21T06:03:59.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.957+0000 7f10a4bb6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f108400c6e0 con 0x558ac06d9390 2024-09-21T06:03:59.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.957+0000 7f10a4bb6740 10 client.0 _ll_register_callbacks cb 0x7f108c00a540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:03:59.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.961+0000 7f1097fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ac06d9390 0x558ac0ac17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:59.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.961+0000 7f10977fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ac06d11b0 0x558ac0ac1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:59.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.962+0000 7f10977fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ac06fa070 0x558ac0b38d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:59.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.962+0000 7f10977fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ac06fa070 0x558ac0b38d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43844/0 (socket says 172.21.15.117:43844) 2024-09-21T06:03:59.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.962+0000 7f10977fe640 1 -- 172.21.15.117:0/1345726808 learned_addr learned my addr 172.21.15.117:0/1345726808 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:03:59.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.964+0000 7f10977fe640 1 -- 172.21.15.117:0/1345726808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ac06d9390 msgr2=0x558ac0ac17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:59.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.964+0000 7f10977fe640 1 --2- 172.21.15.117:0/1345726808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ac06d9390 0x558ac0ac17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:59.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.964+0000 7f10977fe640 1 -- 172.21.15.117:0/1345726808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ac06d11b0 msgr2=0x558ac0ac1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:03:59.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.964+0000 7f10977fe640 1 --2- 172.21.15.117:0/1345726808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ac06d11b0 0x558ac0ac1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:59.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.964+0000 7f10977fe640 1 -- 172.21.15.117:0/1345726808 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ac0b392c0 con 0x558ac06fa070 2024-09-21T06:03:59.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.975+0000 7f10977fe640 1 --2- 172.21.15.117:0/1345726808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ac06fa070 0x558ac0b38d80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f108c003a00 tx=0x7f108c005160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=174cb0057a3e0ba4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:03:59.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.975+0000 7f10777fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:03:59.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.977+0000 7f10777fe640 1 -- 172.21.15.117:0/1345726808 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108c039040 con 0x558ac06fa070 2024-09-21T06:03:59.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.977+0000 7f10777fe640 1 -- 172.21.15.117:0/1345726808 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f108c004c90 con 0x558ac06fa070 2024-09-21T06:03:59.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.977+0000 7f10a4bb6740 1 -- 172.21.15.117:0/1345726808 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f108802ac90 con 0x558ac06fa070 2024-09-21T06:03:59.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.977+0000 7f10777fe640 1 -- 172.21.15.117:0/1345726808 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f108c047050 con 0x558ac06fa070 2024-09-21T06:03:59.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.978+0000 7f10a4bb6740 10 client.16596 fetch_fsmap learned FSMap version 27 2024-09-21T06:03:59.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.978+0000 7f10a4bb6740 1 -- 172.21.15.117:0/1345726808 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558ac0b3a390 con 0x558ac06fa070 2024-09-21T06:03:59.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.978+0000 7f10777fe640 1 -- 172.21.15.117:0/1345726808 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f108802ac90 con 0x558ac06fa070 2024-09-21T06:03:59.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.978+0000 7f10777fe640 1 -- 172.21.15.117:0/1345726808 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f108c03f070 con 0x558ac06fa070 2024-09-21T06:03:59.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.978+0000 7f10a4bb6740 10 client.16596 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:03:59.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.978+0000 7f10a4bb6740 10 client.16596 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T06:03:59.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.978+0000 7f10a4bb6740 20 client.16596 populate_metadata read hostname 'smithi117' 2024-09-21T06:03:59.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.978+0000 7f10a4bb6740 1 --2- 172.21.15.117:0/1345726808 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x558ac068dda0 0x558ac0b3ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:03:59.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.978+0000 7f10a4bb6740 4 client.16596 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:03:59.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.978+0000 7f10a4bb6740 1 -- 172.21.15.117:0/1345726808 --> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558ac0a158a0 con 0x558ac068dda0 2024-09-21T06:03:59.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.979+0000 7f1097fff640 1 --2- 172.21.15.117:0/1345726808 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x558ac068dda0 0x558ac0b3ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:03:59.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.980+0000 7f1097fff640 1 --2- 172.21.15.117:0/1345726808 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x558ac068dda0 0x558ac0b3ca30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c05d5e5f624d478 server_cookie=858ecf862f7be346 in_seq=0 out_seq=0 2024-09-21T06:03:59.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.980+0000 7f10777fe640 10 client.16596 ms_handle_connect on v2:172.21.15.192:6834/3530688860 2024-09-21T06:03:59.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.981+0000 7f10777fe640 1 -- 172.21.15.117:0/1345726808 <== mds.3 v2:172.21.15.192:6834/3530688860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ac0a158a0 con 0x558ac068dda0 2024-09-21T06:03:59.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.981+0000 7f10777fe640 10 client.16596 handle_command_reply: tid=0 2024-09-21T06:03:59.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.981+0000 7f10a4bb6740 1 client.16596 shutdown 2024-09-21T06:03:59.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.981+0000 7f10a4bb6740 1 -- 172.21.15.117:0/1345726808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ac06fa070 msgr2=0x558ac0b38d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:59.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.981+0000 7f10a4bb6740 1 --2- 172.21.15.117:0/1345726808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ac06fa070 0x558ac0b38d80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f108c003a00 tx=0x7f108c005160 comp rx=0 tx=0).stop 2024-09-21T06:03:59.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.981+0000 7f10a4bb6740 1 -- 172.21.15.117:0/1345726808 shutdown_connections 2024-09-21T06:03:59.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.981+0000 7f10a4bb6740 1 --2- 172.21.15.117:0/1345726808 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x558ac068dda0 0x558ac0b3ca30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:59.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.981+0000 7f10a4bb6740 1 --2- 172.21.15.117:0/1345726808 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558ac06fa070 0x558ac0b38d80 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:59.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.981+0000 7f10a4bb6740 1 --2- 172.21.15.117:0/1345726808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558ac06d9390 0x558ac0ac17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:59.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.981+0000 7f10a4bb6740 1 --2- 172.21.15.117:0/1345726808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558ac06d11b0 0x558ac0ac1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:59.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.981+0000 7f10a4bb6740 1 -- 172.21.15.117:0/1345726808 >> 172.21.15.117:0/1345726808 conn(0x558ac06ee460 msgr2=0x558ac0aa2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:59.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.982+0000 7f10777fe640 0 client.16596 ms_handle_reset on v2:172.21.15.192:6834/3530688860 2024-09-21T06:03:59.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.982+0000 7f10a4bb6740 1 -- 172.21.15.117:0/1345726808 shutdown_connections 2024-09-21T06:03:59.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.982+0000 7f10a4bb6740 1 -- 172.21.15.117:0/1345726808 wait complete. 2024-09-21T06:03:59.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.982+0000 7f10a4bb6740 20 client.16596 trim_cache size 0 max 16384 2024-09-21T06:03:59.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.983+0000 7f109d3e0640 1 -- 172.21.15.117:0/3509227458 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f10800b6880 msgr2=0x7f10800b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:59.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.983+0000 7f109d3e0640 1 --2- 172.21.15.117:0/3509227458 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f10800b6880 0x7f10800b8d90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f10900ad0d0 tx=0x7f109000dad0 comp rx=0 tx=0).stop 2024-09-21T06:03:59.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.983+0000 7f109d3e0640 1 -- 172.21.15.117:0/3509227458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1098110640 msgr2=0x7f10981ae9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:03:59.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.983+0000 7f109d3e0640 1 --2- 172.21.15.117:0/3509227458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1098110640 0x7f10981ae9a0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f109000be80 tx=0x7f10900095b0 comp rx=0 tx=0).stop 2024-09-21T06:03:59.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.983+0000 7f109d3e0640 1 -- 172.21.15.117:0/3509227458 shutdown_connections 2024-09-21T06:03:59.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.983+0000 7f109d3e0640 1 --2- 172.21.15.117:0/3509227458 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f10800b6880 0x7f10800b8d90 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:59.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.983+0000 7f109d3e0640 1 --2- 172.21.15.117:0/3509227458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1098110640 0x7f10981ae9a0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:59.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.983+0000 7f109d3e0640 1 --2- 172.21.15.117:0/3509227458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f109810fcf0 0x7f10981a9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:59.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.983+0000 7f109d3e0640 1 --2- 172.21.15.117:0/3509227458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f109810a650 0x7f10981a9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:03:59.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.983+0000 7f109d3e0640 1 -- 172.21.15.117:0/3509227458 >> 172.21.15.117:0/3509227458 conn(0x7f109806ee50 msgr2=0x7f109810dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:03:59.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.984+0000 7f109d3e0640 1 -- 172.21.15.117:0/3509227458 shutdown_connections 2024-09-21T06:03:59.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:03:58.984+0000 7f109d3e0640 1 -- 172.21.15.117:0/3509227458 wait complete. 2024-09-21T06:04:00.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:03:59 smithi192 ceph-mon[34700]: pgmap v527: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 908 MiB/s wr, 2.09k op/s 2024-09-21T06:04:00.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:03:59 smithi117 ceph-mon[30947]: pgmap v527: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 908 MiB/s wr, 2.09k op/s 2024-09-21T06:04:00.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:03:59 smithi138 ceph-mon[34391]: pgmap v527: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 908 MiB/s wr, 2.09k op/s 2024-09-21T06:04:02.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:01 smithi192 ceph-mon[34700]: pgmap v528: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 863 MiB/s wr, 1.95k op/s 2024-09-21T06:04:02.280 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:01 smithi117 ceph-mon[30947]: pgmap v528: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 863 MiB/s wr, 1.95k op/s 2024-09-21T06:04:02.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:01 smithi138 ceph-mon[34391]: pgmap v528: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 863 MiB/s wr, 1.95k op/s 2024-09-21T06:04:03.397 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T06:04:03.793 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:04:04.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:03 smithi192 ceph-mon[34700]: pgmap v529: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 857 MiB/s wr, 2.02k op/s 2024-09-21T06:04:04.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:03 smithi117 ceph-mon[30947]: pgmap v529: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 857 MiB/s wr, 2.02k op/s 2024-09-21T06:04:04.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:03 smithi138 ceph-mon[34391]: pgmap v529: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 857 MiB/s wr, 2.02k op/s 2024-09-21T06:04:04.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.901+0000 7f943498a640 1 -- 172.21.15.117:0/1581785561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94301083a0 msgr2=0x7f94301087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:04.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.901+0000 7f943498a640 1 --2- 172.21.15.117:0/1581785561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94301083a0 0x7f94301087b0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f9424009900 tx=0x7f942402a440 comp rx=0 tx=0).stop 2024-09-21T06:04:04.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.901+0000 7f943498a640 1 -- 172.21.15.117:0/1581785561 shutdown_connections 2024-09-21T06:04:04.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.901+0000 7f943498a640 1 --2- 172.21.15.117:0/1581785561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94301083a0 0x7f94301087b0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.901+0000 7f943498a640 1 --2- 172.21.15.117:0/1581785561 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9430108cf0 0x7f943010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.901+0000 7f943498a640 1 --2- 172.21.15.117:0/1581785561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9430102a60 0x7f9430102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.901+0000 7f943498a640 1 -- 172.21.15.117:0/1581785561 >> 172.21.15.117:0/1581785561 conn(0x7f9430079730 msgr2=0x7f9430079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:04.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.903+0000 7f943498a640 1 -- 172.21.15.117:0/1581785561 shutdown_connections 2024-09-21T06:04:04.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.903+0000 7f943498a640 1 -- 172.21.15.117:0/1581785561 wait complete. 2024-09-21T06:04:04.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.904+0000 7f943498a640 1 Processor -- start 2024-09-21T06:04:04.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.904+0000 7f943498a640 1 -- start start 2024-09-21T06:04:04.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.904+0000 7f943498a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9430102a60 0x7f9430109610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:04.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.904+0000 7f943498a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94301083a0 0x7f9430109b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:04.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.904+0000 7f943498a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9430108cf0 0x7f943010a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:04.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.904+0000 7f943498a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9430112ab0 con 0x7f9430102a60 2024-09-21T06:04:04.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.904+0000 7f943498a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9430112930 con 0x7f94301083a0 2024-09-21T06:04:04.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.904+0000 7f943498a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9430112c30 con 0x7f9430108cf0 2024-09-21T06:04:04.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.905+0000 7f942e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9430102a60 0x7f9430109610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:04.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.905+0000 7f942e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9430102a60 0x7f9430109610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38226/0 (socket says 172.21.15.117:38226) 2024-09-21T06:04:04.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.905+0000 7f942e59c640 1 -- 172.21.15.117:0/2547163006 learned_addr learned my addr 172.21.15.117:0/2547163006 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:04.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.905+0000 7f942ed9d640 1 --2- 172.21.15.117:0/2547163006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9430108cf0 0x7f943010a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:04.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.905+0000 7f942e59c640 1 -- 172.21.15.117:0/2547163006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9430108cf0 msgr2=0x7f943010a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:04.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.905+0000 7f942e59c640 1 --2- 172.21.15.117:0/2547163006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9430108cf0 0x7f943010a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.905+0000 7f942e59c640 1 -- 172.21.15.117:0/2547163006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94301083a0 msgr2=0x7f9430109b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:04.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.905+0000 7f942e59c640 1 --2- 172.21.15.117:0/2547163006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94301083a0 0x7f9430109b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.905+0000 7f942e59c640 1 -- 172.21.15.117:0/2547163006 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94301a87b0 con 0x7f9430102a60 2024-09-21T06:04:04.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.905+0000 7f942e59c640 1 --2- 172.21.15.117:0/2547163006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9430102a60 0x7f9430109610 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f942000b450 tx=0x7f942000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:04.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.906+0000 7f941f7fe640 1 -- 172.21.15.117:0/2547163006 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420015020 con 0x7f9430102a60 2024-09-21T06:04:04.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.906+0000 7f943498a640 1 -- 172.21.15.117:0/2547163006 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94301a8aa0 con 0x7f9430102a60 2024-09-21T06:04:04.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.906+0000 7f941f7fe640 1 -- 172.21.15.117:0/2547163006 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9420011d40 con 0x7f9430102a60 2024-09-21T06:04:04.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.906+0000 7f941f7fe640 1 -- 172.21.15.117:0/2547163006 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942000f340 con 0x7f9430102a60 2024-09-21T06:04:04.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.906+0000 7f943498a640 1 -- 172.21.15.117:0/2547163006 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94301a8fe0 con 0x7f9430102a60 2024-09-21T06:04:04.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.908+0000 7f943aefe740 1 Processor -- start 2024-09-21T06:04:04.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.909+0000 7f943aefe740 1 -- start start 2024-09-21T06:04:04.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.909+0000 7f943aefe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555561894070 0x555561c3b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:04.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.909+0000 7f943aefe740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55556186b1b0 0x555561c3bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:04.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.909+0000 7f943aefe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555561873390 0x555561c3c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:04.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.909+0000 7f943aefe740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555561bb1ea0 con 0x555561894070 2024-09-21T06:04:04.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.909+0000 7f943aefe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555561ba12d0 con 0x555561873390 2024-09-21T06:04:04.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.909+0000 7f943aefe740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555561ba50e0 con 0x55556186b1b0 2024-09-21T06:04:04.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.909+0000 7f942ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555561894070 0x555561c3b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:04.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.909+0000 7f942ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555561894070 0x555561c3b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38238/0 (socket says 172.21.15.117:38238) 2024-09-21T06:04:04.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.909+0000 7f942ed9d640 1 -- 172.21.15.117:0/3385495510 learned_addr learned my addr 172.21.15.117:0/3385495510 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:04.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.910+0000 7f942ed9d640 1 -- 172.21.15.117:0/3385495510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55556186b1b0 msgr2=0x555561c3bbe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T06:04:04.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.910+0000 7f942ed9d640 1 --2- 172.21.15.117:0/3385495510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55556186b1b0 0x555561c3bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.910+0000 7f942ed9d640 1 -- 172.21.15.117:0/3385495510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555561873390 msgr2=0x555561c3c120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:04:04.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.910+0000 7f942ed9d640 1 --2- 172.21.15.117:0/3385495510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555561873390 0x555561c3c120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.910+0000 7f942ed9d640 1 -- 172.21.15.117:0/3385495510 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555618ff650 con 0x555561894070 2024-09-21T06:04:04.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.910+0000 7f942ed9d640 1 --2- 172.21.15.117:0/3385495510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555561894070 0x555561c3b6a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f9428009f60 tx=0x7f9428009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36aac68cb0f84722 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:04.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.910+0000 7f941cff9640 1 -- 172.21.15.117:0/3385495510 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942801a070 con 0x555561894070 2024-09-21T06:04:04.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.910+0000 7f941cff9640 1 -- 172.21.15.117:0/3385495510 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94280158c0 con 0x555561894070 2024-09-21T06:04:04.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.910+0000 7f941cff9640 1 -- 172.21.15.117:0/3385495510 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942800a880 con 0x555561894070 2024-09-21T06:04:04.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.911+0000 7f941f7fe640 1 -- 172.21.15.117:0/2547163006 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f9420018020 con 0x7f9430102a60 2024-09-21T06:04:04.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.911+0000 7f943aefe740 1 -- 172.21.15.117:0/3385495510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555561894070 msgr2=0x555561c3b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:04.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.911+0000 7f943aefe740 1 --2- 172.21.15.117:0/3385495510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555561894070 0x555561c3b6a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f9428009f60 tx=0x7f9428009f90 comp rx=0 tx=0).stop 2024-09-21T06:04:04.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.911+0000 7f941f7fe640 1 --2- 172.21.15.117:0/2547163006 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f94180b6680 0x7f94180b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:04.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.912+0000 7f941f7fe640 1 -- 172.21.15.117:0/2547163006 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f94200e7140 con 0x7f9430102a60 2024-09-21T06:04:04.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.912+0000 7f943aefe740 1 -- 172.21.15.117:0/3385495510 shutdown_connections 2024-09-21T06:04:04.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.912+0000 7f942ed9d640 1 --2- 172.21.15.117:0/2547163006 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f94180b6680 0x7f94180b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:04.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.912+0000 7f943aefe740 1 --2- 172.21.15.117:0/3385495510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555561873390 0x555561c3c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.912+0000 7f943aefe740 1 --2- 172.21.15.117:0/3385495510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55556186b1b0 0x555561c3bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.912+0000 7f943aefe740 1 --2- 172.21.15.117:0/3385495510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555561894070 0x555561c3b6a0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.912+0000 7f943aefe740 1 -- 172.21.15.117:0/3385495510 >> 172.21.15.117:0/3385495510 conn(0x555561888460 msgr2=0x555561c39590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:04.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.912+0000 7f943aefe740 1 -- 172.21.15.117:0/3385495510 shutdown_connections 2024-09-21T06:04:04.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.912+0000 7f943aefe740 1 -- 172.21.15.117:0/3385495510 wait complete. 2024-09-21T06:04:04.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.913+0000 7f942ed9d640 1 --2- 172.21.15.117:0/2547163006 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f94180b6680 0x7f94180b8b90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9428009d70 tx=0x7f9428000f40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:04.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.913+0000 7f943aefe740 1 -- start start 2024-09-21T06:04:04.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.913+0000 7f943aefe740 1 Processor -- start 2024-09-21T06:04:04.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.914+0000 7f943aefe740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55556186b1b0 0x555561c4f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:04.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.914+0000 7f943aefe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555561873390 0x555561c4f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:04.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.914+0000 7f943aefe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555561894070 0x555561c4ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:04.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.914+0000 7f943aefe740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555561a02700 con 0x555561894070 2024-09-21T06:04:04.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.914+0000 7f942ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555561894070 0x555561c4ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:04.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.914+0000 7f942ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555561894070 0x555561c4ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38248/0 (socket says 172.21.15.117:38248) 2024-09-21T06:04:04.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.914+0000 7f942ed9d640 1 -- 172.21.15.117:0/3389901208 learned_addr learned my addr 172.21.15.117:0/3389901208 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:04.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.914+0000 7f943aefe740 1 -- 172.21.15.117:0/3389901208 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555561b3c160 con 0x555561873390 2024-09-21T06:04:04.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.914+0000 7f943aefe740 1 -- 172.21.15.117:0/3389901208 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555561c52650 con 0x55556186b1b0 2024-09-21T06:04:04.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.914+0000 7f943aefe740 10 client.0 _ll_register_callbacks cb 0x555561b559e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:04.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.914+0000 7f942ed9d640 1 -- 172.21.15.117:0/3389901208 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55556186b1b0 msgr2=0x555561c4f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:04.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.915+0000 7f942ed9d640 1 --2- 172.21.15.117:0/3389901208 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55556186b1b0 0x555561c4f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.915+0000 7f942ed9d640 1 -- 172.21.15.117:0/3389901208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555561873390 msgr2=0x555561c4f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:04.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.915+0000 7f942ed9d640 1 --2- 172.21.15.117:0/3389901208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555561873390 0x555561c4f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.915+0000 7f942ed9d640 1 -- 172.21.15.117:0/3389901208 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555561b69c90 con 0x555561894070 2024-09-21T06:04:04.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.915+0000 7f942ed9d640 1 --2- 172.21.15.117:0/3389901208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555561894070 0x555561c4ff10 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f9428015aa0 tx=0x7f942800c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddcc53c9abf40a64 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:04.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.915+0000 7f9415ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:04:04.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.915+0000 7f9415ffb640 1 -- 172.21.15.117:0/3389901208 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942801a040 con 0x555561894070 2024-09-21T06:04:04.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.915+0000 7f9415ffb640 1 -- 172.21.15.117:0/3389901208 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f942800ad50 con 0x555561894070 2024-09-21T06:04:04.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.915+0000 7f943aefe740 1 -- 172.21.15.117:0/3389901208 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94280158c0 con 0x555561894070 2024-09-21T06:04:04.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.916+0000 7f9415ffb640 1 -- 172.21.15.117:0/3389901208 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9428020450 con 0x555561894070 2024-09-21T06:04:04.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.916+0000 7f943aefe740 10 client.16608 fetch_fsmap learned FSMap version 27 2024-09-21T06:04:04.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.916+0000 7f943aefe740 1 -- 172.21.15.117:0/3389901208 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555561c5e2f0 con 0x555561894070 2024-09-21T06:04:04.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.916+0000 7f9415ffb640 1 -- 172.21.15.117:0/3389901208 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94280158c0 con 0x555561894070 2024-09-21T06:04:04.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.916+0000 7f9415ffb640 1 -- 172.21.15.117:0/3389901208 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f94280336a0 con 0x555561894070 2024-09-21T06:04:04.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.916+0000 7f943aefe740 10 client.16608 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:04:04.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.917+0000 7f943aefe740 10 client.16608 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:04:04.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.917+0000 7f943aefe740 20 client.16608 populate_metadata read hostname 'smithi117' 2024-09-21T06:04:04.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.917+0000 7f943aefe740 1 --2- 172.21.15.117:0/3389901208 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x5555618529f0 0x555561c61c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:04.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.917+0000 7f943aefe740 4 client.16608 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:04:04.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.917+0000 7f943aefe740 1 -- 172.21.15.117:0/3389901208 --> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555561bb3df0 con 0x5555618529f0 2024-09-21T06:04:04.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.918+0000 7f942e59c640 1 --2- 172.21.15.117:0/3389901208 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x5555618529f0 0x555561c61c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:04.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.920+0000 7f9415ffb640 10 client.16608 ms_handle_connect on v2:172.21.15.192:6836/2683860842 2024-09-21T06:04:04.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.920+0000 7f942e59c640 1 --2- 172.21.15.117:0/3389901208 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x5555618529f0 0x555561c61c80 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2d91bedbc0cc2ffd server_cookie=6f591dd7ca09e2e6 in_seq=0 out_seq=0 2024-09-21T06:04:04.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.923+0000 7f9415ffb640 1 -- 172.21.15.117:0/3389901208 <== mds.2 v2:172.21.15.192:6836/2683860842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555561bb3df0 con 0x5555618529f0 2024-09-21T06:04:04.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.923+0000 7f9415ffb640 10 client.16608 handle_command_reply: tid=0 2024-09-21T06:04:04.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.923+0000 7f943aefe740 1 client.16608 shutdown 2024-09-21T06:04:04.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.923+0000 7f943aefe740 1 -- 172.21.15.117:0/3389901208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555561894070 msgr2=0x555561c4ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:04.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.923+0000 7f943aefe740 1 --2- 172.21.15.117:0/3389901208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555561894070 0x555561c4ff10 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f9428015aa0 tx=0x7f942800c380 comp rx=0 tx=0).stop 2024-09-21T06:04:04.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.923+0000 7f943aefe740 1 -- 172.21.15.117:0/3389901208 shutdown_connections 2024-09-21T06:04:04.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.923+0000 7f943aefe740 1 --2- 172.21.15.117:0/3389901208 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x5555618529f0 0x555561c61c80 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.923+0000 7f943aefe740 1 --2- 172.21.15.117:0/3389901208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555561894070 0x555561c4ff10 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.923+0000 7f943aefe740 1 --2- 172.21.15.117:0/3389901208 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555561873390 0x555561c4f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.923+0000 7f943aefe740 1 --2- 172.21.15.117:0/3389901208 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55556186b1b0 0x555561c4f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.923+0000 7f943aefe740 1 -- 172.21.15.117:0/3389901208 >> 172.21.15.117:0/3389901208 conn(0x555561888460 msgr2=0x555561c3c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:04.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.924+0000 7f9415ffb640 0 client.16608 ms_handle_reset on v2:172.21.15.192:6836/2683860842 2024-09-21T06:04:04.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.924+0000 7f943aefe740 1 -- 172.21.15.117:0/3389901208 shutdown_connections 2024-09-21T06:04:04.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.924+0000 7f943aefe740 1 -- 172.21.15.117:0/3389901208 wait complete. 2024-09-21T06:04:04.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.924+0000 7f943aefe740 20 client.16608 trim_cache size 0 max 16384 2024-09-21T06:04:04.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.941+0000 7f943aefe740 1 Processor -- start 2024-09-21T06:04:04.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.942+0000 7f943aefe740 1 -- start start 2024-09-21T06:04:04.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.942+0000 7f943aefe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556186b1b0 0x555561c3acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:04.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.942+0000 7f943aefe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555561873390 0x555561c509b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:04.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.942+0000 7f943aefe740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555561894070 0x555561c67bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:04.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.942+0000 7f943aefe740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f94301ad5d0 con 0x55556186b1b0 2024-09-21T06:04:04.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.942+0000 7f943aefe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f94301af110 con 0x555561873390 2024-09-21T06:04:04.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.942+0000 7f943aefe740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f94301adca0 con 0x555561894070 2024-09-21T06:04:04.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.942+0000 7f942e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556186b1b0 0x555561c3acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:04.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.943+0000 7f942e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556186b1b0 0x555561c3acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38264/0 (socket says 172.21.15.117:38264) 2024-09-21T06:04:04.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.943+0000 7f942e59c640 1 -- 172.21.15.117:0/934863577 learned_addr learned my addr 172.21.15.117:0/934863577 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:04.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.943+0000 7f942e59c640 1 -- 172.21.15.117:0/934863577 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555561894070 msgr2=0x555561c67bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:04:04.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.943+0000 7f942e59c640 1 --2- 172.21.15.117:0/934863577 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555561894070 0x555561c67bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.943+0000 7f942e59c640 1 -- 172.21.15.117:0/934863577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555561873390 msgr2=0x555561c509b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:04:04.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.943+0000 7f942e59c640 1 --2- 172.21.15.117:0/934863577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555561873390 0x555561c509b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.943+0000 7f942e59c640 1 -- 172.21.15.117:0/934863577 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555561c3e4b0 con 0x55556186b1b0 2024-09-21T06:04:04.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.943+0000 7f942e59c640 1 --2- 172.21.15.117:0/934863577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556186b1b0 0x555561c3acd0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f94200e4970 tx=0x7f9420094040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=178a33bc066c8224 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:04.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.943+0000 7f9417fff640 1 -- 172.21.15.117:0/934863577 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94200acde0 con 0x55556186b1b0 2024-09-21T06:04:04.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.944+0000 7f9417fff640 1 -- 172.21.15.117:0/934863577 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94200acb20 con 0x55556186b1b0 2024-09-21T06:04:04.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.944+0000 7f9417fff640 1 -- 172.21.15.117:0/934863577 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94200981d0 con 0x55556186b1b0 2024-09-21T06:04:04.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.945+0000 7f943aefe740 1 -- 172.21.15.117:0/934863577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556186b1b0 msgr2=0x555561c3acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:04.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.945+0000 7f943aefe740 1 --2- 172.21.15.117:0/934863577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556186b1b0 0x555561c3acd0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f94200e4970 tx=0x7f9420094040 comp rx=0 tx=0).stop 2024-09-21T06:04:04.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.949+0000 7f943aefe740 1 -- 172.21.15.117:0/934863577 shutdown_connections 2024-09-21T06:04:04.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.949+0000 7f943aefe740 1 --2- 172.21.15.117:0/934863577 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555561894070 0x555561c67bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.949+0000 7f943aefe740 1 --2- 172.21.15.117:0/934863577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555561873390 0x555561c509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.949+0000 7f943aefe740 1 --2- 172.21.15.117:0/934863577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556186b1b0 0x555561c3acd0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.949+0000 7f943aefe740 1 -- 172.21.15.117:0/934863577 >> 172.21.15.117:0/934863577 conn(0x555561888460 msgr2=0x555561c3a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:04.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.949+0000 7f943aefe740 1 -- 172.21.15.117:0/934863577 shutdown_connections 2024-09-21T06:04:04.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.949+0000 7f943aefe740 1 -- 172.21.15.117:0/934863577 wait complete. 2024-09-21T06:04:04.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.950+0000 7f943aefe740 1 -- start start 2024-09-21T06:04:04.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.950+0000 7f943aefe740 1 Processor -- start 2024-09-21T06:04:04.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.951+0000 7f943aefe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556186b1b0 0x555561c62c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:04.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.951+0000 7f943aefe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555561873390 0x555561c631a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:04.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.951+0000 7f943aefe740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555561894070 0x555561cc4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:04.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.951+0000 7f943aefe740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f94301aea40 con 0x55556186b1b0 2024-09-21T06:04:04.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.951+0000 7f943aefe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f94301ae370 con 0x555561873390 2024-09-21T06:04:04.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.951+0000 7f943aefe740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f94301ab610 con 0x555561894070 2024-09-21T06:04:04.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.951+0000 7f943aefe740 10 client.0 _ll_register_callbacks cb 0x7f9428018070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:04.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.952+0000 7f942e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556186b1b0 0x555561c62c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:04.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.952+0000 7f942e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556186b1b0 0x555561c62c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38266/0 (socket says 172.21.15.117:38266) 2024-09-21T06:04:04.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.952+0000 7f942e59c640 1 -- 172.21.15.117:0/1040707231 learned_addr learned my addr 172.21.15.117:0/1040707231 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:04.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.952+0000 7f942ed9d640 1 --2- 172.21.15.117:0/1040707231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555561894070 0x555561cc4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:04.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.953+0000 7f942dd9b640 1 --2- 172.21.15.117:0/1040707231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555561873390 0x555561c631a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:04.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.953+0000 7f942ed9d640 1 -- 172.21.15.117:0/1040707231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555561873390 msgr2=0x555561c631a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:04.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.953+0000 7f942ed9d640 1 --2- 172.21.15.117:0/1040707231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555561873390 0x555561c631a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.953+0000 7f942ed9d640 1 -- 172.21.15.117:0/1040707231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556186b1b0 msgr2=0x555561c62c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:04.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.953+0000 7f942ed9d640 1 --2- 172.21.15.117:0/1040707231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556186b1b0 0x555561c62c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.953+0000 7f942ed9d640 1 -- 172.21.15.117:0/1040707231 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555561cc4b90 con 0x555561894070 2024-09-21T06:04:04.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.955+0000 7f942ed9d640 1 --2- 172.21.15.117:0/1040707231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555561894070 0x555561cc4520 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f942800cfd0 tx=0x7f9428009750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=186a97901a5eda52 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:04.961 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.958+0000 7f9415ffb640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-21T06:04:04.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.959+0000 7f9415ffb640 1 -- 172.21.15.117:0/1040707231 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942801a070 con 0x555561894070 2024-09-21T06:04:04.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.960+0000 7f9415ffb640 1 -- 172.21.15.117:0/1040707231 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f942800ad50 con 0x555561894070 2024-09-21T06:04:04.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.960+0000 7f943aefe740 1 -- 172.21.15.117:0/1040707231 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94200acb20 con 0x555561894070 2024-09-21T06:04:04.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.961+0000 7f9415ffb640 1 -- 172.21.15.117:0/1040707231 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9428033400 con 0x555561894070 2024-09-21T06:04:04.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.961+0000 7f943aefe740 10 client.25801 fetch_fsmap learned FSMap version 27 2024-09-21T06:04:04.963 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.961+0000 7f943aefe740 1 -- 172.21.15.117:0/1040707231 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555561ccbeb0 con 0x555561894070 2024-09-21T06:04:04.964 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.962+0000 7f9415ffb640 1 -- 172.21.15.117:0/1040707231 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f942802f070 con 0x555561894070 2024-09-21T06:04:04.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.964+0000 7f9415ffb640 1 -- 172.21.15.117:0/1040707231 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9428020b70 con 0x555561894070 2024-09-21T06:04:04.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.964+0000 7f943aefe740 10 client.25801 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:04:04.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.964+0000 7f943aefe740 10 client.25801 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:04:04.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.964+0000 7f943aefe740 20 client.25801 populate_metadata read hostname 'smithi117' 2024-09-21T06:04:04.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.964+0000 7f943aefe740 1 --2- 172.21.15.117:0/1040707231 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x555561827da0 0x555561cce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:04.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.964+0000 7f943aefe740 4 client.25801 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:04:04.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.964+0000 7f943aefe740 1 -- 172.21.15.117:0/1040707231 --> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f94200acb20 con 0x555561827da0 2024-09-21T06:04:04.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.965+0000 7f942e59c640 1 --2- 172.21.15.117:0/1040707231 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x555561827da0 0x555561cce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:04.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.967+0000 7f942e59c640 1 --2- 172.21.15.117:0/1040707231 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x555561827da0 0x555561cce520 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2d996e4ab1cab31b server_cookie=dded3a590b721e81 in_seq=0 out_seq=0 2024-09-21T06:04:04.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.967+0000 7f9415ffb640 10 client.25801 ms_handle_connect on v2:172.21.15.192:6836/2683860842 2024-09-21T06:04:04.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.968+0000 7f9415ffb640 1 -- 172.21.15.117:0/1040707231 <== mds.2 v2:172.21.15.192:6836/2683860842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f94200acb20 con 0x555561827da0 2024-09-21T06:04:04.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.968+0000 7f9415ffb640 10 client.25801 handle_command_reply: tid=0 2024-09-21T06:04:04.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.968+0000 7f943aefe740 1 client.25801 shutdown 2024-09-21T06:04:04.971 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.969+0000 7f943aefe740 1 -- 172.21.15.117:0/1040707231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555561894070 msgr2=0x555561cc4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:04.971 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.969+0000 7f943aefe740 1 --2- 172.21.15.117:0/1040707231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555561894070 0x555561cc4520 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f942800cfd0 tx=0x7f9428009750 comp rx=0 tx=0).stop 2024-09-21T06:04:04.971 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.969+0000 7f943aefe740 1 -- 172.21.15.117:0/1040707231 shutdown_connections 2024-09-21T06:04:04.971 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.969+0000 7f943aefe740 1 --2- 172.21.15.117:0/1040707231 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x555561827da0 0x555561cce520 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.971 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.969+0000 7f943aefe740 1 --2- 172.21.15.117:0/1040707231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555561894070 0x555561cc4520 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.971 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.969+0000 7f943aefe740 1 --2- 172.21.15.117:0/1040707231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555561873390 0x555561c631a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.971 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.969+0000 7f943aefe740 1 --2- 172.21.15.117:0/1040707231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55556186b1b0 0x555561c62c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.969+0000 7f943aefe740 1 -- 172.21.15.117:0/1040707231 >> 172.21.15.117:0/1040707231 conn(0x555561888460 msgr2=0x555561c394d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:04.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.969+0000 7f9415ffb640 0 client.25801 ms_handle_reset on v2:172.21.15.192:6836/2683860842 2024-09-21T06:04:04.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.970+0000 7f943aefe740 1 -- 172.21.15.117:0/1040707231 shutdown_connections 2024-09-21T06:04:04.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.970+0000 7f943aefe740 1 -- 172.21.15.117:0/1040707231 wait complete. 2024-09-21T06:04:04.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.970+0000 7f943aefe740 20 client.25801 trim_cache size 0 max 16384 2024-09-21T06:04:04.973 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:04:04.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.971+0000 7f943498a640 1 -- 172.21.15.117:0/2547163006 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f94180b6680 msgr2=0x7f94180b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:04.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.971+0000 7f943498a640 1 --2- 172.21.15.117:0/2547163006 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f94180b6680 0x7f94180b8b90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9428009d70 tx=0x7f9428000f40 comp rx=0 tx=0).stop 2024-09-21T06:04:04.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.972+0000 7f943498a640 1 -- 172.21.15.117:0/2547163006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9430102a60 msgr2=0x7f9430109610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:04.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.972+0000 7f943498a640 1 --2- 172.21.15.117:0/2547163006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9430102a60 0x7f9430109610 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f942000b450 tx=0x7f942000b910 comp rx=0 tx=0).stop 2024-09-21T06:04:04.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.972+0000 7f943498a640 1 -- 172.21.15.117:0/2547163006 shutdown_connections 2024-09-21T06:04:04.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.972+0000 7f943498a640 1 --2- 172.21.15.117:0/2547163006 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f94180b6680 0x7f94180b8b90 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.972+0000 7f943498a640 1 --2- 172.21.15.117:0/2547163006 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9430108cf0 0x7f943010a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.972+0000 7f943498a640 1 --2- 172.21.15.117:0/2547163006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94301083a0 0x7f9430109b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.972+0000 7f943498a640 1 --2- 172.21.15.117:0/2547163006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9430102a60 0x7f9430109610 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:04.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.972+0000 7f943498a640 1 -- 172.21.15.117:0/2547163006 >> 172.21.15.117:0/2547163006 conn(0x7f9430079730 msgr2=0x7f9430101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:04.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.972+0000 7f943498a640 1 -- 172.21.15.117:0/2547163006 shutdown_connections 2024-09-21T06:04:04.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:04.972+0000 7f943498a640 1 -- 172.21.15.117:0/2547163006 wait complete. 2024-09-21T06:04:05.698 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T06:04:06.026 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:04:06.189 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:05 smithi117 ceph-mon[30947]: pgmap v530: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 928 MiB/s wr, 2.37k op/s 2024-09-21T06:04:06.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:05 smithi192 ceph-mon[34700]: pgmap v530: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 928 MiB/s wr, 2.37k op/s 2024-09-21T06:04:06.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:05 smithi138 ceph-mon[34391]: pgmap v530: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 928 MiB/s wr, 2.37k op/s 2024-09-21T06:04:06.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.608+0000 7fe575b77640 1 -- 172.21.15.117:0/1620177131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe57010aee0 msgr2=0x7fe57010b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:06.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.608+0000 7fe575b77640 1 --2- 172.21.15.117:0/1620177131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe57010aee0 0x7fe57010b2f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe560009900 tx=0x7fe56002a3f0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.609+0000 7fe575b77640 1 -- 172.21.15.117:0/1620177131 shutdown_connections 2024-09-21T06:04:06.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.609+0000 7fe575b77640 1 --2- 172.21.15.117:0/1620177131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe57010ead0 0x7fe5701174d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.609+0000 7fe575b77640 1 --2- 172.21.15.117:0/1620177131 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe57010e180 0x7fe57010e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.609+0000 7fe575b77640 1 --2- 172.21.15.117:0/1620177131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe57010aee0 0x7fe57010b2f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.609+0000 7fe575b77640 1 -- 172.21.15.117:0/1620177131 >> 172.21.15.117:0/1620177131 conn(0x7fe5700712f0 msgr2=0x7fe570073e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:06.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.609+0000 7fe575b77640 1 -- 172.21.15.117:0/1620177131 shutdown_connections 2024-09-21T06:04:06.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.609+0000 7fe575b77640 1 -- 172.21.15.117:0/1620177131 wait complete. 2024-09-21T06:04:06.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.610+0000 7fe575b77640 1 Processor -- start 2024-09-21T06:04:06.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.610+0000 7fe575b77640 1 -- start start 2024-09-21T06:04:06.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.610+0000 7fe575b77640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe57010aee0 0x7fe5701a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:06.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.610+0000 7fe575b77640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe57010e180 0x7fe5701a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:06.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.610+0000 7fe575b77640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe57010ead0 0x7fe5701aa500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:06.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.610+0000 7fe575b77640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe57011c2d0 con 0x7fe57010aee0 2024-09-21T06:04:06.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.610+0000 7fe575b77640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe57011c150 con 0x7fe57010ead0 2024-09-21T06:04:06.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.610+0000 7fe575b77640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe57011c450 con 0x7fe57010e180 2024-09-21T06:04:06.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.611+0000 7fe56effd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe57010e180 0x7fe5701a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:06.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.611+0000 7fe56effd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe57010e180 0x7fe5701a5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:59728/0 (socket says 172.21.15.117:59728) 2024-09-21T06:04:06.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.611+0000 7fe56effd640 1 -- 172.21.15.117:0/1594512770 learned_addr learned my addr 172.21.15.117:0/1594512770 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:06.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.611+0000 7fe56ffff640 1 --2- 172.21.15.117:0/1594512770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe57010ead0 0x7fe5701aa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:06.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.611+0000 7fe56f7fe640 1 --2- 172.21.15.117:0/1594512770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe57010aee0 0x7fe5701a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:06.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.611+0000 7fe56effd640 1 -- 172.21.15.117:0/1594512770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe57010ead0 msgr2=0x7fe5701aa500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:06.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.611+0000 7fe56effd640 1 --2- 172.21.15.117:0/1594512770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe57010ead0 0x7fe5701aa500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.611+0000 7fe56effd640 1 -- 172.21.15.117:0/1594512770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe57010aee0 msgr2=0x7fe5701a4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:06.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.611+0000 7fe56effd640 1 --2- 172.21.15.117:0/1594512770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe57010aee0 0x7fe5701a4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.611+0000 7fe56effd640 1 -- 172.21.15.117:0/1594512770 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5701aac30 con 0x7fe57010e180 2024-09-21T06:04:06.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.611+0000 7fe56effd640 1 --2- 172.21.15.117:0/1594512770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe57010e180 0x7fe5701a5260 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe56400e9a0 tx=0x7fe56400ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:06.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.612+0000 7fe56cff9640 1 -- 172.21.15.117:0/1594512770 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56400cd80 con 0x7fe57010e180 2024-09-21T06:04:06.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.612+0000 7fe56cff9640 1 -- 172.21.15.117:0/1594512770 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5640035f0 con 0x7fe57010e180 2024-09-21T06:04:06.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.612+0000 7fe56cff9640 1 -- 172.21.15.117:0/1594512770 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe564010690 con 0x7fe57010e180 2024-09-21T06:04:06.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.612+0000 7fe575b77640 1 -- 172.21.15.117:0/1594512770 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5701aaf20 con 0x7fe57010e180 2024-09-21T06:04:06.616 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.612+0000 7fe575b77640 1 -- 172.21.15.117:0/1594512770 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5701b1680 con 0x7fe57010e180 2024-09-21T06:04:06.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.615+0000 7fe57c0eb740 1 Processor -- start 2024-09-21T06:04:06.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.615+0000 7fe57c0eb740 1 -- start start 2024-09-21T06:04:06.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.615+0000 7fe57c0eb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562109861070 0x562109c082c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:06.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.615+0000 7fe57c0eb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5621098381b0 0x562109c08800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:06.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.615+0000 7fe56cff9640 1 -- 172.21.15.117:0/1594512770 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fe564004000 con 0x7fe57010e180 2024-09-21T06:04:06.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.615+0000 7fe57c0eb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562109840390 0x562109c08d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:06.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.615+0000 7fe57c0eb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562109b7eea0 con 0x562109861070 2024-09-21T06:04:06.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.616+0000 7fe57c0eb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562109b6e2d0 con 0x562109840390 2024-09-21T06:04:06.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.616+0000 7fe57c0eb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x562109b720e0 con 0x5621098381b0 2024-09-21T06:04:06.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.616+0000 7fe56ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562109861070 0x562109c082c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:06.618 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.616+0000 7fe56ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562109861070 0x562109c082c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38306/0 (socket says 172.21.15.117:38306) 2024-09-21T06:04:06.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.616+0000 7fe56ffff640 1 -- 172.21.15.117:0/2307624190 learned_addr learned my addr 172.21.15.117:0/2307624190 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:06.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.616+0000 7fe56f7fe640 1 --2- 172.21.15.117:0/2307624190 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5621098381b0 0x562109c08800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:06.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.616+0000 7fe56ffff640 1 -- 172.21.15.117:0/2307624190 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5621098381b0 msgr2=0x562109c08800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:06.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.616+0000 7fe56ffff640 1 --2- 172.21.15.117:0/2307624190 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5621098381b0 0x562109c08800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.616+0000 7fe56ffff640 1 -- 172.21.15.117:0/2307624190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562109840390 msgr2=0x562109c08d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:06.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.617+0000 7fe56ffff640 1 --2- 172.21.15.117:0/2307624190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562109840390 0x562109c08d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.617+0000 7fe56ffff640 1 -- 172.21.15.117:0/2307624190 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621098cc650 con 0x562109861070 2024-09-21T06:04:06.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.617+0000 7fe56cff9640 1 --2- 172.21.15.117:0/1594512770 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe5580b6850 0x7fe5580b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:06.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.617+0000 7fe56ffff640 1 --2- 172.21.15.117:0/2307624190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562109861070 0x562109c082c0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe568009fd0 tx=0x7fe5680118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c47317dfaedfca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:06.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.617+0000 7fe56cff9640 1 -- 172.21.15.117:0/1594512770 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fe5640e7890 con 0x7fe57010e180 2024-09-21T06:04:06.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.617+0000 7fe555ffb640 1 -- 172.21.15.117:0/2307624190 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe568004d20 con 0x562109861070 2024-09-21T06:04:06.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.617+0000 7fe555ffb640 1 -- 172.21.15.117:0/2307624190 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe568004050 con 0x562109861070 2024-09-21T06:04:06.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.617+0000 7fe555ffb640 1 -- 172.21.15.117:0/2307624190 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe568005260 con 0x562109861070 2024-09-21T06:04:06.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.618+0000 7fe56f7fe640 1 --2- 172.21.15.117:0/1594512770 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe5580b6850 0x7fe5580b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:06.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.618+0000 7fe57c0eb740 1 -- 172.21.15.117:0/2307624190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562109861070 msgr2=0x562109c082c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:06.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.618+0000 7fe57c0eb740 1 --2- 172.21.15.117:0/2307624190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562109861070 0x562109c082c0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe568009fd0 tx=0x7fe5680118d0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.618+0000 7fe56f7fe640 1 --2- 172.21.15.117:0/1594512770 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe5580b6850 0x7fe5580b8d60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe56000b230 tx=0x7fe56000b040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:06.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.619+0000 7fe57c0eb740 1 -- 172.21.15.117:0/2307624190 shutdown_connections 2024-09-21T06:04:06.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.619+0000 7fe57c0eb740 1 --2- 172.21.15.117:0/2307624190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562109840390 0x562109c08d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.619+0000 7fe57c0eb740 1 --2- 172.21.15.117:0/2307624190 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5621098381b0 0x562109c08800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.619+0000 7fe57c0eb740 1 --2- 172.21.15.117:0/2307624190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562109861070 0x562109c082c0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.619+0000 7fe57c0eb740 1 -- 172.21.15.117:0/2307624190 >> 172.21.15.117:0/2307624190 conn(0x562109855460 msgr2=0x562109c061b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:06.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.619+0000 7fe57c0eb740 1 -- 172.21.15.117:0/2307624190 shutdown_connections 2024-09-21T06:04:06.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.619+0000 7fe57c0eb740 1 -- 172.21.15.117:0/2307624190 wait complete. 2024-09-21T06:04:06.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.619+0000 7fe57c0eb740 1 -- start start 2024-09-21T06:04:06.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.620+0000 7fe57c0eb740 1 Processor -- start 2024-09-21T06:04:06.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.620+0000 7fe57c0eb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5621098381b0 0x562109c13920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:06.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.620+0000 7fe57c0eb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562109840390 0x562109c13e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:06.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.620+0000 7fe57c0eb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562109861070 0x562109c285a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:06.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.620+0000 7fe57c0eb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5621099cf700 con 0x562109861070 2024-09-21T06:04:06.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.620+0000 7fe57c0eb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562109b09160 con 0x562109840390 2024-09-21T06:04:06.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.620+0000 7fe57c0eb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x562109c16aa0 con 0x5621098381b0 2024-09-21T06:04:06.625 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.620+0000 7fe56effd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562109840390 0x562109c13e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:06.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.620+0000 7fe57c0eb740 10 client.0 _ll_register_callbacks cb 0x562109b229e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:06.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.621+0000 7fe56f7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5621098381b0 0x562109c13920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:06.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.621+0000 7fe56f7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5621098381b0 0x562109c13920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:59758/0 (socket says 172.21.15.117:59758) 2024-09-21T06:04:06.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.621+0000 7fe56f7fe640 1 -- 172.21.15.117:0/2841521607 learned_addr learned my addr 172.21.15.117:0/2841521607 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:06.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.621+0000 7fe56ffff640 1 --2- 172.21.15.117:0/2841521607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562109861070 0x562109c285a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:06.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.621+0000 7fe56f7fe640 1 -- 172.21.15.117:0/2841521607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562109840390 msgr2=0x562109c13e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:06.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.621+0000 7fe56f7fe640 1 --2- 172.21.15.117:0/2841521607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562109840390 0x562109c13e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.621+0000 7fe56f7fe640 1 -- 172.21.15.117:0/2841521607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562109861070 msgr2=0x562109c285a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:06.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.621+0000 7fe56f7fe640 1 --2- 172.21.15.117:0/2841521607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562109861070 0x562109c285a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.621+0000 7fe56f7fe640 1 -- 172.21.15.117:0/2841521607 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562109b36c90 con 0x5621098381b0 2024-09-21T06:04:06.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.622+0000 7fe56f7fe640 1 --2- 172.21.15.117:0/2841521607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5621098381b0 0x562109c13920 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe560033000 tx=0x7fe560031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=329655478d18f45b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:06.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.622+0000 7fe54effd640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-21T06:04:06.626 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.622+0000 7fe54effd640 1 -- 172.21.15.117:0/2841521607 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe560035040 con 0x5621098381b0 2024-09-21T06:04:06.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.622+0000 7fe57c0eb740 1 -- 172.21.15.117:0/2841521607 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe568004050 con 0x5621098381b0 2024-09-21T06:04:06.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.622+0000 7fe54effd640 1 -- 172.21.15.117:0/2841521607 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fe560005020 con 0x5621098381b0 2024-09-21T06:04:06.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.622+0000 7fe54effd640 1 -- 172.21.15.117:0/2841521607 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe560005220 con 0x5621098381b0 2024-09-21T06:04:06.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.622+0000 7fe57c0eb740 10 client.25813 fetch_fsmap learned FSMap version 27 2024-09-21T06:04:06.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.622+0000 7fe57c0eb740 1 -- 172.21.15.117:0/2841521607 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562109c2afb0 con 0x5621098381b0 2024-09-21T06:04:06.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.622+0000 7fe54effd640 1 -- 172.21.15.117:0/2841521607 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe568004050 con 0x5621098381b0 2024-09-21T06:04:06.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.623+0000 7fe54effd640 1 -- 172.21.15.117:0/2841521607 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe560044030 con 0x5621098381b0 2024-09-21T06:04:06.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.623+0000 7fe57c0eb740 10 client.25813 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:04:06.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.623+0000 7fe57c0eb740 10 client.25813 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:04:06.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.623+0000 7fe57c0eb740 20 client.25813 populate_metadata read hostname 'smithi117' 2024-09-21T06:04:06.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.623+0000 7fe57c0eb740 1 --2- 172.21.15.117:0/2841521607 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x56210981f9f0 0x562109c2e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:06.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.623+0000 7fe57c0eb740 4 client.25813 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:04:06.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.623+0000 7fe57c0eb740 1 -- 172.21.15.117:0/2841521607 --> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562109b80df0 con 0x56210981f9f0 2024-09-21T06:04:06.627 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.623+0000 7fe56effd640 1 --2- 172.21.15.117:0/2841521607 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x56210981f9f0 0x562109c2e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:06.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.624+0000 7fe54effd640 10 client.25813 ms_handle_connect on v2:172.21.15.192:6838/1727623231 2024-09-21T06:04:06.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.624+0000 7fe56effd640 1 --2- 172.21.15.117:0/2841521607 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x56210981f9f0 0x562109c2e820 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=73aa79882b797833 server_cookie=322f3aa6acaade67 in_seq=0 out_seq=0 2024-09-21T06:04:06.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.625+0000 7fe54effd640 1 -- 172.21.15.117:0/2841521607 <== mds.1 v2:172.21.15.192:6838/1727623231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562109b80df0 con 0x56210981f9f0 2024-09-21T06:04:06.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.625+0000 7fe54effd640 10 client.25813 handle_command_reply: tid=0 2024-09-21T06:04:06.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.625+0000 7fe57c0eb740 1 client.25813 shutdown 2024-09-21T06:04:06.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.625+0000 7fe57c0eb740 1 -- 172.21.15.117:0/2841521607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5621098381b0 msgr2=0x562109c13920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:06.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.625+0000 7fe57c0eb740 1 --2- 172.21.15.117:0/2841521607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5621098381b0 0x562109c13920 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe560033000 tx=0x7fe560031400 comp rx=0 tx=0).stop 2024-09-21T06:04:06.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.625+0000 7fe57c0eb740 1 -- 172.21.15.117:0/2841521607 shutdown_connections 2024-09-21T06:04:06.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.625+0000 7fe57c0eb740 1 --2- 172.21.15.117:0/2841521607 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x56210981f9f0 0x562109c2e820 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.625+0000 7fe57c0eb740 1 --2- 172.21.15.117:0/2841521607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562109861070 0x562109c285a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.625+0000 7fe57c0eb740 1 --2- 172.21.15.117:0/2841521607 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562109840390 0x562109c13e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.625+0000 7fe57c0eb740 1 --2- 172.21.15.117:0/2841521607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5621098381b0 0x562109c13920 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.625+0000 7fe57c0eb740 1 -- 172.21.15.117:0/2841521607 >> 172.21.15.117:0/2841521607 conn(0x562109855460 msgr2=0x562109c0b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:06.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.625+0000 7fe54effd640 0 client.25813 ms_handle_reset on v2:172.21.15.192:6838/1727623231 2024-09-21T06:04:06.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.625+0000 7fe57c0eb740 1 -- 172.21.15.117:0/2841521607 shutdown_connections 2024-09-21T06:04:06.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.625+0000 7fe57c0eb740 1 -- 172.21.15.117:0/2841521607 wait complete. 2024-09-21T06:04:06.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.626+0000 7fe57c0eb740 20 client.25813 trim_cache size 0 max 16384 2024-09-21T06:04:06.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.643+0000 7fe57c0eb740 1 Processor -- start 2024-09-21T06:04:06.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.643+0000 7fe57c0eb740 1 -- start start 2024-09-21T06:04:06.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.643+0000 7fe57c0eb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621098381b0 0x562109c12810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:06.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.643+0000 7fe57c0eb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562109840390 0x562109c061e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:06.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.643+0000 7fe57c0eb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562109861070 0x562109c28590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:06.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.643+0000 7fe57c0eb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe5701b5b80 con 0x562109840390 2024-09-21T06:04:06.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.643+0000 7fe57c0eb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe5701b76c0 con 0x5621098381b0 2024-09-21T06:04:06.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.643+0000 7fe57c0eb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe5701b6250 con 0x562109861070 2024-09-21T06:04:06.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.644+0000 7fe56f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621098381b0 0x562109c12810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:06.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.644+0000 7fe56effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562109840390 0x562109c061e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:06.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.644+0000 7fe56f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621098381b0 0x562109c12810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:58742/0 (socket says 172.21.15.117:58742) 2024-09-21T06:04:06.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.644+0000 7fe56f7fe640 1 -- 172.21.15.117:0/2891027761 learned_addr learned my addr 172.21.15.117:0/2891027761 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:06.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.644+0000 7fe56ffff640 1 --2- 172.21.15.117:0/2891027761 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562109861070 0x562109c28590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:06.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.644+0000 7fe56f7fe640 1 -- 172.21.15.117:0/2891027761 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562109861070 msgr2=0x562109c28590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:06.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.644+0000 7fe56f7fe640 1 --2- 172.21.15.117:0/2891027761 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562109861070 0x562109c28590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.644+0000 7fe56f7fe640 1 -- 172.21.15.117:0/2891027761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562109840390 msgr2=0x562109c061e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:06.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.644+0000 7fe56f7fe640 1 --2- 172.21.15.117:0/2891027761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562109840390 0x562109c061e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.644+0000 7fe56f7fe640 1 -- 172.21.15.117:0/2891027761 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562109c14be0 con 0x5621098381b0 2024-09-21T06:04:06.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.645+0000 7fe56f7fe640 1 --2- 172.21.15.117:0/2891027761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621098381b0 0x562109c12810 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe560005f70 tx=0x7fe560005600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42d8f5ccd87ffdc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:06.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.645+0000 7fe5557fa640 1 -- 172.21.15.117:0/2891027761 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe560050860 con 0x5621098381b0 2024-09-21T06:04:06.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.645+0000 7fe5557fa640 1 -- 172.21.15.117:0/2891027761 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe56005e070 con 0x5621098381b0 2024-09-21T06:04:06.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.645+0000 7fe5557fa640 1 -- 172.21.15.117:0/2891027761 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe560004ee0 con 0x5621098381b0 2024-09-21T06:04:06.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.647+0000 7fe57c0eb740 1 -- 172.21.15.117:0/2891027761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621098381b0 msgr2=0x562109c12810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:06.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.647+0000 7fe57c0eb740 1 --2- 172.21.15.117:0/2891027761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621098381b0 0x562109c12810 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe560005f70 tx=0x7fe560005600 comp rx=0 tx=0).stop 2024-09-21T06:04:06.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.647+0000 7fe57c0eb740 1 -- 172.21.15.117:0/2891027761 shutdown_connections 2024-09-21T06:04:06.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.647+0000 7fe57c0eb740 1 --2- 172.21.15.117:0/2891027761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562109840390 0x562109c061e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.647+0000 7fe57c0eb740 1 --2- 172.21.15.117:0/2891027761 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562109861070 0x562109c28590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.647+0000 7fe57c0eb740 1 --2- 172.21.15.117:0/2891027761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5621098381b0 0x562109c12810 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.647+0000 7fe57c0eb740 1 -- 172.21.15.117:0/2891027761 >> 172.21.15.117:0/2891027761 conn(0x562109855460 msgr2=0x562109c05ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:06.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.647+0000 7fe57c0eb740 1 -- 172.21.15.117:0/2891027761 shutdown_connections 2024-09-21T06:04:06.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.647+0000 7fe57c0eb740 1 -- 172.21.15.117:0/2891027761 wait complete. 2024-09-21T06:04:06.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.648+0000 7fe57c0eb740 1 -- start start 2024-09-21T06:04:06.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.648+0000 7fe57c0eb740 1 Processor -- start 2024-09-21T06:04:06.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.649+0000 7fe57c0eb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621098381b0 0x562109c2f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:06.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.649+0000 7fe57c0eb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562109840390 0x562109c2f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:06.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.649+0000 7fe57c0eb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562109861070 0x562109c900c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:06.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.649+0000 7fe56f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621098381b0 0x562109c2f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:06.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.649+0000 7fe56f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621098381b0 0x562109c2f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38320/0 (socket says 172.21.15.117:38320) 2024-09-21T06:04:06.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.649+0000 7fe56f7fe640 1 -- 172.21.15.117:0/2679460709 learned_addr learned my addr 172.21.15.117:0/2679460709 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:06.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.649+0000 7fe56effd640 1 --2- 172.21.15.117:0/2679460709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562109840390 0x562109c2f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:06.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.649+0000 7fe57c0eb740 1 -- 172.21.15.117:0/2679460709 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe5701b6ff0 con 0x5621098381b0 2024-09-21T06:04:06.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.649+0000 7fe57c0eb740 1 -- 172.21.15.117:0/2679460709 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe5701b6920 con 0x562109861070 2024-09-21T06:04:06.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.649+0000 7fe57c0eb740 1 -- 172.21.15.117:0/2679460709 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe5701b3be0 con 0x562109840390 2024-09-21T06:04:06.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.649+0000 7fe56ffff640 1 --2- 172.21.15.117:0/2679460709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562109861070 0x562109c900c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:06.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.649+0000 7fe57c0eb740 10 client.0 _ll_register_callbacks cb 0x7fe560009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:06.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.650+0000 7fe56ffff640 1 -- 172.21.15.117:0/2679460709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562109840390 msgr2=0x562109c2f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:06.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.650+0000 7fe56ffff640 1 --2- 172.21.15.117:0/2679460709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562109840390 0x562109c2f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.650+0000 7fe56ffff640 1 -- 172.21.15.117:0/2679460709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621098381b0 msgr2=0x562109c2f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:06.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.650+0000 7fe56ffff640 1 --2- 172.21.15.117:0/2679460709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621098381b0 0x562109c2f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.650+0000 7fe56ffff640 1 -- 172.21.15.117:0/2679460709 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562109c90730 con 0x562109861070 2024-09-21T06:04:06.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.650+0000 7fe57c0eb740 1 -- 172.21.15.117:0/2679460709 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe56005e070 con 0x562109861070 2024-09-21T06:04:06.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.650+0000 7fe56ffff640 1 --2- 172.21.15.117:0/2679460709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562109861070 0x562109c900c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe568004920 tx=0x7fe568004b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8366563159dc1c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:06.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.650+0000 7fe54effd640 10 client.25805 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-21T06:04:06.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.650+0000 7fe54effd640 1 -- 172.21.15.117:0/2679460709 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe568004250 con 0x562109861070 2024-09-21T06:04:06.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.650+0000 7fe54effd640 1 -- 172.21.15.117:0/2679460709 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7fe5680043f0 con 0x562109861070 2024-09-21T06:04:06.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.650+0000 7fe54effd640 1 -- 172.21.15.117:0/2679460709 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe568004650 con 0x562109861070 2024-09-21T06:04:06.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.650+0000 7fe57c0eb740 10 client.25805 fetch_fsmap learned FSMap version 27 2024-09-21T06:04:06.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.651+0000 7fe57c0eb740 1 -- 172.21.15.117:0/2679460709 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({fsmap=27}) -- 0x562109cbaa90 con 0x562109861070 2024-09-21T06:04:06.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.651+0000 7fe54effd640 1 -- 172.21.15.117:0/2679460709 <== mon.2 v2:172.21.15.138:3300/0 4 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe56800f4e0 con 0x562109861070 2024-09-21T06:04:06.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.651+0000 7fe57c0eb740 10 client.25805 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:04:06.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.651+0000 7fe57c0eb740 10 client.25805 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:04:06.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.651+0000 7fe57c0eb740 20 client.25805 populate_metadata read hostname 'smithi117' 2024-09-21T06:04:06.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.651+0000 7fe57c0eb740 1 --2- 172.21.15.117:0/2679460709 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x5621097f4da0 0x562109cbcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:06.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.651+0000 7fe57c0eb740 4 client.25805 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:04:06.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.651+0000 7fe57c0eb740 1 -- 172.21.15.117:0/2679460709 --> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fe56005e070 con 0x5621097f4da0 2024-09-21T06:04:06.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.652+0000 7fe56f7fe640 1 --2- 172.21.15.117:0/2679460709 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x5621097f4da0 0x562109cbcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:06.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.652+0000 7fe56f7fe640 1 --2- 172.21.15.117:0/2679460709 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x5621097f4da0 0x562109cbcfd0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=624bec83c3fb5595 server_cookie=df044ae629950f61 in_seq=0 out_seq=0 2024-09-21T06:04:06.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.652+0000 7fe54effd640 10 client.25805 ms_handle_connect on v2:172.21.15.192:6838/1727623231 2024-09-21T06:04:06.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.653+0000 7fe54effd640 1 -- 172.21.15.117:0/2679460709 <== mds.1 v2:172.21.15.192:6838/1727623231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fe56005e070 con 0x5621097f4da0 2024-09-21T06:04:06.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.653+0000 7fe54effd640 10 client.25805 handle_command_reply: tid=0 2024-09-21T06:04:06.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.653+0000 7fe57c0eb740 1 client.25805 shutdown 2024-09-21T06:04:06.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.653+0000 7fe57c0eb740 1 -- 172.21.15.117:0/2679460709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562109861070 msgr2=0x562109c900c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:06.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.653+0000 7fe57c0eb740 1 --2- 172.21.15.117:0/2679460709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562109861070 0x562109c900c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe568004920 tx=0x7fe568004b80 comp rx=0 tx=0).stop 2024-09-21T06:04:06.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.653+0000 7fe57c0eb740 1 -- 172.21.15.117:0/2679460709 shutdown_connections 2024-09-21T06:04:06.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.653+0000 7fe57c0eb740 1 --2- 172.21.15.117:0/2679460709 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x5621097f4da0 0x562109cbcfd0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.653+0000 7fe57c0eb740 1 --2- 172.21.15.117:0/2679460709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562109861070 0x562109c900c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.653+0000 7fe57c0eb740 1 --2- 172.21.15.117:0/2679460709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562109840390 0x562109c2f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.653+0000 7fe57c0eb740 1 --2- 172.21.15.117:0/2679460709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5621098381b0 0x562109c2f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.653+0000 7fe57c0eb740 1 -- 172.21.15.117:0/2679460709 >> 172.21.15.117:0/2679460709 conn(0x562109855460 msgr2=0x562109c092f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:06.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.654+0000 7fe54effd640 0 client.25805 ms_handle_reset on v2:172.21.15.192:6838/1727623231 2024-09-21T06:04:06.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.654+0000 7fe57c0eb740 1 -- 172.21.15.117:0/2679460709 shutdown_connections 2024-09-21T06:04:06.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.654+0000 7fe57c0eb740 1 -- 172.21.15.117:0/2679460709 wait complete. 2024-09-21T06:04:06.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.654+0000 7fe57c0eb740 20 client.25805 trim_cache size 0 max 16384 2024-09-21T06:04:06.658 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:04:06.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.655+0000 7fe575b77640 1 -- 172.21.15.117:0/1594512770 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe5580b6850 msgr2=0x7fe5580b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:06.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.655+0000 7fe575b77640 1 --2- 172.21.15.117:0/1594512770 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe5580b6850 0x7fe5580b8d60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe56000b230 tx=0x7fe56000b040 comp rx=0 tx=0).stop 2024-09-21T06:04:06.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.655+0000 7fe575b77640 1 -- 172.21.15.117:0/1594512770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe57010e180 msgr2=0x7fe5701a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:06.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.655+0000 7fe575b77640 1 --2- 172.21.15.117:0/1594512770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe57010e180 0x7fe5701a5260 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe56400e9a0 tx=0x7fe56400ee60 comp rx=0 tx=0).stop 2024-09-21T06:04:06.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.655+0000 7fe575b77640 1 -- 172.21.15.117:0/1594512770 shutdown_connections 2024-09-21T06:04:06.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.655+0000 7fe575b77640 1 --2- 172.21.15.117:0/1594512770 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe5580b6850 0x7fe5580b8d60 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.655+0000 7fe575b77640 1 --2- 172.21.15.117:0/1594512770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe57010ead0 0x7fe5701aa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.655+0000 7fe575b77640 1 --2- 172.21.15.117:0/1594512770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe57010e180 0x7fe5701a5260 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.655+0000 7fe575b77640 1 --2- 172.21.15.117:0/1594512770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe57010aee0 0x7fe5701a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:06.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.655+0000 7fe575b77640 1 -- 172.21.15.117:0/1594512770 >> 172.21.15.117:0/1594512770 conn(0x7fe5700712f0 msgr2=0x7fe570119690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:06.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.655+0000 7fe575b77640 1 -- 172.21.15.117:0/1594512770 shutdown_connections 2024-09-21T06:04:06.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:06.655+0000 7fe575b77640 1 -- 172.21.15.117:0/1594512770 wait complete. 2024-09-21T06:04:07.529 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:04:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.669+0000 7f382569b640 1 -- 172.21.15.117:0/3510015151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38201026f0 msgr2=0x7f3820102b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.669+0000 7f382569b640 1 --2- 172.21.15.117:0/3510015151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38201026f0 0x7f3820102b00 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f3814009900 tx=0x7f381402a440 comp rx=0 tx=0).stop 2024-09-21T06:04:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.670+0000 7f382569b640 1 -- 172.21.15.117:0/3510015151 shutdown_connections 2024-09-21T06:04:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.670+0000 7f382569b640 1 --2- 172.21.15.117:0/3510015151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3820107bb0 0x7f3820108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.670+0000 7f382569b640 1 --2- 172.21.15.117:0/3510015151 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3820103040 0x7f3820107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.670+0000 7f382569b640 1 --2- 172.21.15.117:0/3510015151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38201026f0 0x7f3820102b00 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.670+0000 7f382569b640 1 -- 172.21.15.117:0/3510015151 >> 172.21.15.117:0/3510015151 conn(0x7f382007a740 msgr2=0x7f382007ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.670+0000 7f382569b640 1 -- 172.21.15.117:0/3510015151 shutdown_connections 2024-09-21T06:04:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.670+0000 7f382569b640 1 -- 172.21.15.117:0/3510015151 wait complete. 2024-09-21T06:04:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.671+0000 7f382569b640 1 Processor -- start 2024-09-21T06:04:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.671+0000 7f382569b640 1 -- start start 2024-09-21T06:04:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.672+0000 7f382569b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38201026f0 0x7f3820116ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.672+0000 7f382569b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3820103040 0x7f3820110f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.672+0000 7f382569b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3820107bb0 0x7f38201115f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.672+0000 7f382569b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3820109430 con 0x7f38201026f0 2024-09-21T06:04:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.672+0000 7f382569b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f38201092b0 con 0x7f3820107bb0 2024-09-21T06:04:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.672+0000 7f382569b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f38201095b0 con 0x7f3820103040 2024-09-21T06:04:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.672+0000 7f381e7fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3820103040 0x7f3820110f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.672+0000 7f381e7fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3820103040 0x7f3820110f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:59794/0 (socket says 172.21.15.117:59794) 2024-09-21T06:04:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.672+0000 7f381e7fc640 1 -- 172.21.15.117:0/3895507932 learned_addr learned my addr 172.21.15.117:0/3895507932 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.672+0000 7f381f7fe640 1 --2- 172.21.15.117:0/3895507932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3820107bb0 0x7f38201115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.673+0000 7f381effd640 1 --2- 172.21.15.117:0/3895507932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38201026f0 0x7f3820116ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.673+0000 7f381e7fc640 1 -- 172.21.15.117:0/3895507932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3820107bb0 msgr2=0x7f38201115f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.673+0000 7f381e7fc640 1 --2- 172.21.15.117:0/3895507932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3820107bb0 0x7f38201115f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.673+0000 7f381e7fc640 1 -- 172.21.15.117:0/3895507932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38201026f0 msgr2=0x7f3820116ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.673+0000 7f381e7fc640 1 --2- 172.21.15.117:0/3895507932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38201026f0 0x7f3820116ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.673+0000 7f381e7fc640 1 -- 172.21.15.117:0/3895507932 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38201b5500 con 0x7f3820103040 2024-09-21T06:04:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.673+0000 7f381e7fc640 1 --2- 172.21.15.117:0/3895507932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3820103040 0x7f3820110f90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f381000b9b0 tx=0x7f381000be70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:07.678 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.677+0000 7f3807fff640 1 -- 172.21.15.117:0/3895507932 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810019070 con 0x7f3820103040 2024-09-21T06:04:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.677+0000 7f3807fff640 1 -- 172.21.15.117:0/3895507932 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f381000cba0 con 0x7f3820103040 2024-09-21T06:04:07.679 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.677+0000 7f3807fff640 1 -- 172.21.15.117:0/3895507932 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38100125a0 con 0x7f3820103040 2024-09-21T06:04:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.678+0000 7f382569b640 1 -- 172.21.15.117:0/3895507932 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38201b56d0 con 0x7f3820103040 2024-09-21T06:04:07.680 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.678+0000 7f382569b640 1 -- 172.21.15.117:0/3895507932 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38201b5b70 con 0x7f3820103040 2024-09-21T06:04:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.682+0000 7f382c610740 1 Processor -- start 2024-09-21T06:04:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.683+0000 7f382c610740 1 -- start start 2024-09-21T06:04:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.683+0000 7f3807fff640 1 -- 172.21.15.117:0/3895507932 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f381001c070 con 0x7f3820103040 2024-09-21T06:04:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.683+0000 7f382c610740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b737d261b0 0x55b7380cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.683+0000 7f382c610740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b737cfd2f0 0x55b7380cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.683+0000 7f382c610740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b737d054d0 0x55b7380cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.683+0000 7f382c610740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b7380406f0 con 0x55b737d054d0 2024-09-21T06:04:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.683+0000 7f382c610740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b73802fb20 con 0x55b737d261b0 2024-09-21T06:04:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.683+0000 7f382c610740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b738033930 con 0x55b737cfd2f0 2024-09-21T06:04:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.684+0000 7f381f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b737d261b0 0x55b7380cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.684+0000 7f381f7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b737d261b0 0x55b7380cb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:58770/0 (socket says 172.21.15.117:58770) 2024-09-21T06:04:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.684+0000 7f381f7fe640 1 -- 172.21.15.117:0/324933648 learned_addr learned my addr 172.21.15.117:0/324933648 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.684+0000 7f381effd640 1 --2- 172.21.15.117:0/324933648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b737cfd2f0 0x55b7380cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.684+0000 7f381e7fc640 1 --2- 172.21.15.117:0/324933648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b737d054d0 0x55b7380cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.684+0000 7f381f7fe640 1 -- 172.21.15.117:0/324933648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b737cfd2f0 msgr2=0x55b7380cbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.684+0000 7f381f7fe640 1 --2- 172.21.15.117:0/324933648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b737cfd2f0 0x55b7380cbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.684+0000 7f381f7fe640 1 -- 172.21.15.117:0/324933648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b737d054d0 msgr2=0x55b7380cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.684+0000 7f381f7fe640 1 --2- 172.21.15.117:0/324933648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b737d054d0 0x55b7380cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.684+0000 7f381f7fe640 1 -- 172.21.15.117:0/324933648 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b737ff5530 con 0x55b737d261b0 2024-09-21T06:04:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.684+0000 7f381effd640 1 --2- 172.21.15.117:0/324933648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b737cfd2f0 0x55b7380cbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:04:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.685+0000 7f381f7fe640 1 --2- 172.21.15.117:0/324933648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b737d261b0 0x55b7380cb9b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3818009be0 tx=0x7f38180117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d2b6b9f46a1312f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.685+0000 7f3807fff640 1 --2- 172.21.15.117:0/3895507932 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f38080b6880 0x7f38080b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.685+0000 7f38057fa640 1 -- 172.21.15.117:0/324933648 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3818012450 con 0x55b737d261b0 2024-09-21T06:04:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.685+0000 7f38057fa640 1 -- 172.21.15.117:0/324933648 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3818012aa0 con 0x55b737d261b0 2024-09-21T06:04:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.685+0000 7f38057fa640 1 -- 172.21.15.117:0/324933648 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381800fa80 con 0x55b737d261b0 2024-09-21T06:04:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.685+0000 7f3807fff640 1 -- 172.21.15.117:0/3895507932 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f38100e6a60 con 0x7f3820103040 2024-09-21T06:04:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.685+0000 7f381effd640 1 --2- 172.21.15.117:0/3895507932 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f38080b6880 0x7f38080b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.686+0000 7f382c610740 1 -- 172.21.15.117:0/324933648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b737d261b0 msgr2=0x55b7380cb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.686+0000 7f381effd640 1 --2- 172.21.15.117:0/3895507932 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f38080b6880 0x7f38080b8d90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3814006010 tx=0x7f3814005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.686+0000 7f382c610740 1 --2- 172.21.15.117:0/324933648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b737d261b0 0x55b7380cb9b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3818009be0 tx=0x7f38180117b0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.686+0000 7f382c610740 1 -- 172.21.15.117:0/324933648 shutdown_connections 2024-09-21T06:04:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.686+0000 7f382c610740 1 --2- 172.21.15.117:0/324933648 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b737d054d0 0x55b7380cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.686+0000 7f382c610740 1 --2- 172.21.15.117:0/324933648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b737cfd2f0 0x55b7380cbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.686+0000 7f382c610740 1 --2- 172.21.15.117:0/324933648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b737d261b0 0x55b7380cb9b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.686+0000 7f382c610740 1 -- 172.21.15.117:0/324933648 >> 172.21.15.117:0/324933648 conn(0x55b737d1a300 msgr2=0x55b7380c98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.686+0000 7f382c610740 1 -- 172.21.15.117:0/324933648 shutdown_connections 2024-09-21T06:04:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.686+0000 7f382c610740 1 -- 172.21.15.117:0/324933648 wait complete. 2024-09-21T06:04:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.687+0000 7f382c610740 1 -- start start 2024-09-21T06:04:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.687+0000 7f382c610740 1 Processor -- start 2024-09-21T06:04:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.687+0000 7f382c610740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b737cfd2f0 0x55b7380d7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.687+0000 7f382c610740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b737d054d0 0x55b7380d7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.687+0000 7f382c610740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b737d261b0 0x55b7380e3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.687+0000 7f382c610740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b737f67ab0 con 0x55b737d261b0 2024-09-21T06:04:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.687+0000 7f382c610740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b737fd5730 con 0x55b737d054d0 2024-09-21T06:04:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.687+0000 7f382c610740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b7380da130 con 0x55b737cfd2f0 2024-09-21T06:04:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.687+0000 7f382c610740 10 client.0 _ll_register_callbacks cb 0x55b7380296b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.687+0000 7f381e7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b737d054d0 0x55b7380d7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.687+0000 7f381e7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b737d054d0 0x55b7380d7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:58776/0 (socket says 172.21.15.117:58776) 2024-09-21T06:04:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.687+0000 7f381e7fc640 1 -- 172.21.15.117:0/2647835081 learned_addr learned my addr 172.21.15.117:0/2647835081 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.687+0000 7f381f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b737d261b0 0x55b7380e3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.687+0000 7f381effd640 1 --2- 172.21.15.117:0/2647835081 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b737cfd2f0 0x55b7380d7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.688+0000 7f381f7fe640 1 -- 172.21.15.117:0/2647835081 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b737cfd2f0 msgr2=0x55b7380d7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.688+0000 7f381f7fe640 1 --2- 172.21.15.117:0/2647835081 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b737cfd2f0 0x55b7380d7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.688+0000 7f381f7fe640 1 -- 172.21.15.117:0/2647835081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b737d054d0 msgr2=0x55b7380d7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.688+0000 7f381f7fe640 1 --2- 172.21.15.117:0/2647835081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b737d054d0 0x55b7380d7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.688+0000 7f381f7fe640 1 -- 172.21.15.117:0/2647835081 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b737d91790 con 0x55b737d261b0 2024-09-21T06:04:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.688+0000 7f381e7fc640 1 --2- 172.21.15.117:0/2647835081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b737d054d0 0x55b7380d7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:04:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.688+0000 7f381f7fe640 1 --2- 172.21.15.117:0/2647835081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b737d261b0 0x55b7380e3420 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f3818009be0 tx=0x7f3818005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa717c3cd763dca8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.688+0000 7f3802ff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:04:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.688+0000 7f3802ff5640 1 -- 172.21.15.117:0/2647835081 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3818029030 con 0x55b737d261b0 2024-09-21T06:04:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.688+0000 7f382c610740 1 -- 172.21.15.117:0/2647835081 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3818012aa0 con 0x55b737d261b0 2024-09-21T06:04:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.688+0000 7f3802ff5640 1 -- 172.21.15.117:0/2647835081 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f3818020040 con 0x55b737d261b0 2024-09-21T06:04:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.689+0000 7f3802ff5640 1 -- 172.21.15.117:0/2647835081 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f38180128c0 con 0x55b737d261b0 2024-09-21T06:04:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.689+0000 7f382c610740 10 client.16644 fetch_fsmap learned FSMap version 27 2024-09-21T06:04:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.689+0000 7f382c610740 1 -- 172.21.15.117:0/2647835081 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b737e0a620 con 0x55b737d261b0 2024-09-21T06:04:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.689+0000 7f3802ff5640 1 -- 172.21.15.117:0/2647835081 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3818012aa0 con 0x55b737d261b0 2024-09-21T06:04:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.689+0000 7f3802ff5640 1 -- 172.21.15.117:0/2647835081 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f38180055c0 con 0x55b737d261b0 2024-09-21T06:04:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.689+0000 7f382c610740 10 client.16644 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:04:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.689+0000 7f382c610740 10 client.16644 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:04:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.689+0000 7f382c610740 20 client.16644 populate_metadata read hostname 'smithi117' 2024-09-21T06:04:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.689+0000 7f382c610740 1 --2- 172.21.15.117:0/2647835081 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b737ce4d80 0x55b7380e9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.689+0000 7f382c610740 4 client.16644 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:04:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.689+0000 7f382c610740 1 -- 172.21.15.117:0/2647835081 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b738042640 con 0x55b737ce4d80 2024-09-21T06:04:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.690+0000 7f381effd640 1 --2- 172.21.15.117:0/2647835081 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b737ce4d80 0x55b7380e9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.691+0000 7f3802ff5640 10 client.16644 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.691+0000 7f381effd640 1 --2- 172.21.15.117:0/2647835081 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b737ce4d80 0x55b7380e9230 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ec68bf97f0583d7 server_cookie=19a5c84f3f6cf171 in_seq=0 out_seq=0 2024-09-21T06:04:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.693+0000 7f3802ff5640 1 -- 172.21.15.117:0/2647835081 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b738042640 con 0x55b737ce4d80 2024-09-21T06:04:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.693+0000 7f3802ff5640 10 client.16644 handle_command_reply: tid=0 2024-09-21T06:04:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.693+0000 7f382c610740 1 client.16644 shutdown 2024-09-21T06:04:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.694+0000 7f382c610740 1 -- 172.21.15.117:0/2647835081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b737d261b0 msgr2=0x55b7380e3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.694+0000 7f382c610740 1 --2- 172.21.15.117:0/2647835081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b737d261b0 0x55b7380e3420 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f3818009be0 tx=0x7f3818005d30 comp rx=0 tx=0).stop 2024-09-21T06:04:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.694+0000 7f382c610740 1 -- 172.21.15.117:0/2647835081 shutdown_connections 2024-09-21T06:04:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.694+0000 7f382c610740 1 --2- 172.21.15.117:0/2647835081 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b737ce4d80 0x55b7380e9230 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.694+0000 7f382c610740 1 --2- 172.21.15.117:0/2647835081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b737d261b0 0x55b7380e3420 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.694+0000 7f382c610740 1 --2- 172.21.15.117:0/2647835081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b737d054d0 0x55b7380d7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.694+0000 7f382c610740 1 --2- 172.21.15.117:0/2647835081 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b737cfd2f0 0x55b7380d7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.694+0000 7f382c610740 1 -- 172.21.15.117:0/2647835081 >> 172.21.15.117:0/2647835081 conn(0x55b737d1a300 msgr2=0x55b7380cc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.694+0000 7f3802ff5640 0 client.16644 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:07.700 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.696+0000 7f382c610740 1 -- 172.21.15.117:0/2647835081 shutdown_connections 2024-09-21T06:04:07.701 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.700+0000 7f382c610740 1 -- 172.21.15.117:0/2647835081 wait complete. 2024-09-21T06:04:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.704+0000 7f382c610740 20 client.16644 trim_cache size 0 max 16384 2024-09-21T06:04:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.753+0000 7f382c610740 1 Processor -- start 2024-09-21T06:04:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.755+0000 7f382c610740 1 -- start start 2024-09-21T06:04:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.756+0000 7f382c610740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b737cfd2f0 0x55b7380d2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.756+0000 7f382c610740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b737d054d0 0x55b7380d9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.756+0000 7f381effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b737cfd2f0 0x55b7380d2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.757+0000 7f381effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b737cfd2f0 0x55b7380d2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38384/0 (socket says 172.21.15.117:38384) 2024-09-21T06:04:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.757+0000 7f381effd640 1 -- 172.21.15.117:0/2495081468 learned_addr learned my addr 172.21.15.117:0/2495081468 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.757+0000 7f382c610740 1 --2- 172.21.15.117:0/2495081468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b737d261b0 0x55b7380cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.757+0000 7f382c610740 1 -- 172.21.15.117:0/2495081468 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f38201be370 con 0x55b737cfd2f0 2024-09-21T06:04:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.757+0000 7f382c610740 1 -- 172.21.15.117:0/2495081468 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f38201bfeb0 con 0x55b737d054d0 2024-09-21T06:04:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.757+0000 7f382c610740 1 -- 172.21.15.117:0/2495081468 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f38201bea40 con 0x55b737d261b0 2024-09-21T06:04:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.757+0000 7f381f7fe640 1 --2- 172.21.15.117:0/2495081468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b737d261b0 0x55b7380cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.758+0000 7f381effd640 1 -- 172.21.15.117:0/2495081468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b737d261b0 msgr2=0x55b7380cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.758+0000 7f381effd640 1 --2- 172.21.15.117:0/2495081468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b737d261b0 0x55b7380cd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.758+0000 7f381effd640 1 -- 172.21.15.117:0/2495081468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b737d054d0 msgr2=0x55b7380d9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.758+0000 7f381effd640 1 --2- 172.21.15.117:0/2495081468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b737d054d0 0x55b7380d9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.758+0000 7f381effd640 1 -- 172.21.15.117:0/2495081468 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7380cdcb0 con 0x55b737cfd2f0 2024-09-21T06:04:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.758+0000 7f381effd640 1 --2- 172.21.15.117:0/2495081468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b737cfd2f0 0x55b7380d2890 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f3814009230 tx=0x7f381403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e953d5515987da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.760+0000 7f3804ff9640 1 -- 172.21.15.117:0/2495081468 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3814035070 con 0x55b737cfd2f0 2024-09-21T06:04:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.760+0000 7f3804ff9640 1 -- 172.21.15.117:0/2495081468 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38140054b0 con 0x55b737cfd2f0 2024-09-21T06:04:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.760+0000 7f3804ff9640 1 -- 172.21.15.117:0/2495081468 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381403c440 con 0x55b737cfd2f0 2024-09-21T06:04:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.761+0000 7f382c610740 1 -- 172.21.15.117:0/2495081468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b737cfd2f0 msgr2=0x55b7380d2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.761+0000 7f382c610740 1 --2- 172.21.15.117:0/2495081468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b737cfd2f0 0x55b7380d2890 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f3814009230 tx=0x7f381403f000 comp rx=0 tx=0).stop 2024-09-21T06:04:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.769+0000 7f382c610740 1 -- 172.21.15.117:0/2495081468 shutdown_connections 2024-09-21T06:04:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.769+0000 7f382c610740 1 --2- 172.21.15.117:0/2495081468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b737d261b0 0x55b7380cd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.769+0000 7f382c610740 1 --2- 172.21.15.117:0/2495081468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b737d054d0 0x55b7380d9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.769+0000 7f382c610740 1 --2- 172.21.15.117:0/2495081468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b737cfd2f0 0x55b7380d2890 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.771 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.769+0000 7f382c610740 1 -- 172.21.15.117:0/2495081468 >> 172.21.15.117:0/2495081468 conn(0x55b737d1a300 msgr2=0x55b7380cc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.772+0000 7f382c610740 1 -- 172.21.15.117:0/2495081468 shutdown_connections 2024-09-21T06:04:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.775+0000 7f382c610740 1 -- 172.21.15.117:0/2495081468 wait complete. 2024-09-21T06:04:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.775+0000 7f382c610740 1 -- start start 2024-09-21T06:04:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.776+0000 7f382c610740 1 Processor -- start 2024-09-21T06:04:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.780+0000 7f382c610740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b737cfd2f0 0x55b7380ebd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.780+0000 7f382c610740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b737d054d0 0x55b7380ec300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.780+0000 7f382c610740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b737d261b0 0x55b73814a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.780+0000 7f382c610740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f38201bf7e0 con 0x55b737d261b0 2024-09-21T06:04:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.780+0000 7f382c610740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f38201bf110 con 0x55b737d054d0 2024-09-21T06:04:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.780+0000 7f382c610740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f38201bc350 con 0x55b737cfd2f0 2024-09-21T06:04:07.782 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.780+0000 7f382c610740 10 client.0 _ll_register_callbacks cb 0x7f3818005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.781+0000 7f381e7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b737d054d0 0x55b7380ec300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.782+0000 7f381effd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b737cfd2f0 0x55b7380ebd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.782+0000 7f381effd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b737cfd2f0 0x55b7380ebd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:59840/0 (socket says 172.21.15.117:59840) 2024-09-21T06:04:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.782+0000 7f381effd640 1 -- 172.21.15.117:0/471024659 learned_addr learned my addr 172.21.15.117:0/471024659 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.783+0000 7f381f7fe640 1 --2- 172.21.15.117:0/471024659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b737d261b0 0x55b73814a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.783+0000 7f381f7fe640 1 -- 172.21.15.117:0/471024659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b737cfd2f0 msgr2=0x55b7380ebd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.783+0000 7f381f7fe640 1 --2- 172.21.15.117:0/471024659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b737cfd2f0 0x55b7380ebd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.783+0000 7f381f7fe640 1 -- 172.21.15.117:0/471024659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b737d054d0 msgr2=0x55b7380ec300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.783+0000 7f381f7fe640 1 --2- 172.21.15.117:0/471024659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b737d054d0 0x55b7380ec300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.783+0000 7f381f7fe640 1 -- 172.21.15.117:0/471024659 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b73814afe0 con 0x55b737d261b0 2024-09-21T06:04:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.789+0000 7f381f7fe640 1 --2- 172.21.15.117:0/471024659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b737d261b0 0x55b73814a970 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f381800a010 tx=0x7f3818013d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=775060989eabb836 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.789+0000 7f3802ff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:04:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.791+0000 7f3802ff5640 1 -- 172.21.15.117:0/471024659 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3818029040 con 0x55b737d261b0 2024-09-21T06:04:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.792+0000 7f3802ff5640 1 -- 172.21.15.117:0/471024659 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f381800fc10 con 0x55b737d261b0 2024-09-21T06:04:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.792+0000 7f382c610740 1 -- 172.21.15.117:0/471024659 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38140054b0 con 0x55b737d261b0 2024-09-21T06:04:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.797+0000 7f3802ff5640 1 -- 172.21.15.117:0/471024659 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f381800f660 con 0x55b737d261b0 2024-09-21T06:04:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.798+0000 7f382c610740 10 client.16650 fetch_fsmap learned FSMap version 27 2024-09-21T06:04:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.798+0000 7f382c610740 1 -- 172.21.15.117:0/471024659 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b738153510 con 0x55b737d261b0 2024-09-21T06:04:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.801+0000 7f3802ff5640 1 -- 172.21.15.117:0/471024659 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38140054b0 con 0x55b737d261b0 2024-09-21T06:04:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.801+0000 7f3802ff5640 1 -- 172.21.15.117:0/471024659 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f38180054b0 con 0x55b737d261b0 2024-09-21T06:04:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.802+0000 7f382c610740 10 client.16650 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:04:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.802+0000 7f382c610740 10 client.16650 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:04:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.802+0000 7f382c610740 20 client.16650 populate_metadata read hostname 'smithi117' 2024-09-21T06:04:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.802+0000 7f382c610740 1 --2- 172.21.15.117:0/471024659 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b737cba1c0 0x55b738155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.802+0000 7f382c610740 4 client.16650 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:04:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.802+0000 7f382c610740 1 -- 172.21.15.117:0/471024659 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b73803e0f0 con 0x55b737cba1c0 2024-09-21T06:04:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.808+0000 7f381effd640 1 --2- 172.21.15.117:0/471024659 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b737cba1c0 0x55b738155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:07.820 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.818+0000 7f381effd640 1 --2- 172.21.15.117:0/471024659 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b737cba1c0 0x55b738155c10 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51cf51053c64969b server_cookie=e4adafab7b1d46c7 in_seq=0 out_seq=0 2024-09-21T06:04:07.820 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.819+0000 7f3802ff5640 10 client.16650 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.956+0000 7f3802ff5640 1 -- 172.21.15.117:0/471024659 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b73803e0f0 con 0x55b737cba1c0 2024-09-21T06:04:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.956+0000 7f3802ff5640 10 client.16650 handle_command_reply: tid=0 2024-09-21T06:04:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.956+0000 7f382c610740 1 client.16650 shutdown 2024-09-21T06:04:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.956+0000 7f382c610740 1 -- 172.21.15.117:0/471024659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b737d261b0 msgr2=0x55b73814a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.956+0000 7f382c610740 1 --2- 172.21.15.117:0/471024659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b737d261b0 0x55b73814a970 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f381800a010 tx=0x7f3818013d20 comp rx=0 tx=0).stop 2024-09-21T06:04:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.956+0000 7f382c610740 1 -- 172.21.15.117:0/471024659 shutdown_connections 2024-09-21T06:04:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.956+0000 7f382c610740 1 --2- 172.21.15.117:0/471024659 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b737cba1c0 0x55b738155c10 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.956+0000 7f382c610740 1 --2- 172.21.15.117:0/471024659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b737d261b0 0x55b73814a970 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.956+0000 7f382c610740 1 --2- 172.21.15.117:0/471024659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b737d054d0 0x55b7380ec300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.956+0000 7f382c610740 1 --2- 172.21.15.117:0/471024659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b737cfd2f0 0x55b7380ebd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.956+0000 7f382c610740 1 -- 172.21.15.117:0/471024659 >> 172.21.15.117:0/471024659 conn(0x55b737d1a300 msgr2=0x55b7380cd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.956+0000 7f3802ff5640 0 client.16650 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.959+0000 7f382c610740 1 -- 172.21.15.117:0/471024659 shutdown_connections 2024-09-21T06:04:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.959+0000 7f382c610740 1 -- 172.21.15.117:0/471024659 wait complete. 2024-09-21T06:04:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T06:04:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T06:04:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T06:04:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T06:04:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.961+0000 7f382c610740 20 client.16650 trim_cache size 0 max 16384 2024-09-21T06:04:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.962+0000 7f382569b640 1 -- 172.21.15.117:0/3895507932 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f38080b6880 msgr2=0x7f38080b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.962+0000 7f382569b640 1 --2- 172.21.15.117:0/3895507932 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f38080b6880 0x7f38080b8d90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3814006010 tx=0x7f3814005a20 comp rx=0 tx=0).stop 2024-09-21T06:04:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.962+0000 7f382569b640 1 -- 172.21.15.117:0/3895507932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3820103040 msgr2=0x7f3820110f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.962+0000 7f382569b640 1 --2- 172.21.15.117:0/3895507932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3820103040 0x7f3820110f90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f381000b9b0 tx=0x7f381000be70 comp rx=0 tx=0).stop 2024-09-21T06:04:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.962+0000 7f382569b640 1 -- 172.21.15.117:0/3895507932 shutdown_connections 2024-09-21T06:04:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.962+0000 7f382569b640 1 --2- 172.21.15.117:0/3895507932 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f38080b6880 0x7f38080b8d90 secure :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3814006010 tx=0x7f3814005a20 comp rx=0 tx=0).stop 2024-09-21T06:04:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.962+0000 7f382569b640 1 --2- 172.21.15.117:0/3895507932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3820107bb0 0x7f38201115f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.962+0000 7f382569b640 1 --2- 172.21.15.117:0/3895507932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3820103040 0x7f3820110f90 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.962+0000 7f382569b640 1 --2- 172.21.15.117:0/3895507932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38201026f0 0x7f3820116ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.962+0000 7f382569b640 1 -- 172.21.15.117:0/3895507932 >> 172.21.15.117:0/3895507932 conn(0x7f382007a740 msgr2=0x7f3820105530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.962+0000 7f382569b640 1 -- 172.21.15.117:0/3895507932 shutdown_connections 2024-09-21T06:04:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:07.962+0000 7f382569b640 1 -- 172.21.15.117:0/3895507932 wait complete. 2024-09-21T06:04:08.016 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:04:08.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:07 smithi192 ceph-mon[34700]: pgmap v531: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 861 MiB/s wr, 2.51k op/s 2024-09-21T06:04:08.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:07 smithi117 ceph-mon[30947]: pgmap v531: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 861 MiB/s wr, 2.51k op/s 2024-09-21T06:04:08.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:07 smithi138 ceph-mon[34391]: pgmap v531: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 861 MiB/s wr, 2.51k op/s 2024-09-21T06:04:08.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.539+0000 7f991816a640 1 -- 172.21.15.117:0/3161154240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9910108cf0 msgr2=0x7f991010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:08.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.539+0000 7f991816a640 1 --2- 172.21.15.117:0/3161154240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9910108cf0 0x7f991010f520 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f9908009ca0 tx=0x7f990802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.539+0000 7f991816a640 1 -- 172.21.15.117:0/3161154240 shutdown_connections 2024-09-21T06:04:08.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.539+0000 7f991816a640 1 --2- 172.21.15.117:0/3161154240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9910108cf0 0x7f991010f520 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.539+0000 7f991816a640 1 --2- 172.21.15.117:0/3161154240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99101083a0 0x7f99101087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.539+0000 7f991816a640 1 --2- 172.21.15.117:0/3161154240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9910102a60 0x7f9910102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.539+0000 7f991816a640 1 -- 172.21.15.117:0/3161154240 >> 172.21.15.117:0/3161154240 conn(0x7f9910079730 msgr2=0x7f9910079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:08.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.539+0000 7f991816a640 1 -- 172.21.15.117:0/3161154240 shutdown_connections 2024-09-21T06:04:08.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.539+0000 7f991816a640 1 -- 172.21.15.117:0/3161154240 wait complete. 2024-09-21T06:04:08.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.540+0000 7f991816a640 1 Processor -- start 2024-09-21T06:04:08.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.540+0000 7f991816a640 1 -- start start 2024-09-21T06:04:08.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.540+0000 7f991816a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9910102a60 0x7f991019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:08.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.540+0000 7f991816a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99101083a0 0x7f991019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:08.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.540+0000 7f991816a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9910108cf0 0x7f991019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:08.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.540+0000 7f991816a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f991006b720 con 0x7f99101083a0 2024-09-21T06:04:08.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.540+0000 7f991816a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f991006b5a0 con 0x7f9910102a60 2024-09-21T06:04:08.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.540+0000 7f991816a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f991006b8a0 con 0x7f9910108cf0 2024-09-21T06:04:08.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.540+0000 7f9915705640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99101083a0 0x7f991019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:08.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.540+0000 7f9915f06640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9910102a60 0x7f991019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:08.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.540+0000 7f9915705640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99101083a0 0x7f991019a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38416/0 (socket says 172.21.15.117:38416) 2024-09-21T06:04:08.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.540+0000 7f9915705640 1 -- 172.21.15.117:0/1099908374 learned_addr learned my addr 172.21.15.117:0/1099908374 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:08.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.540+0000 7f9916707640 1 --2- 172.21.15.117:0/1099908374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9910108cf0 0x7f991019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:08.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.541+0000 7f9915705640 1 -- 172.21.15.117:0/1099908374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9910108cf0 msgr2=0x7f991019fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:08.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.541+0000 7f9915705640 1 --2- 172.21.15.117:0/1099908374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9910108cf0 0x7f991019fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.541+0000 7f9915705640 1 -- 172.21.15.117:0/1099908374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9910102a60 msgr2=0x7f991019a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:08.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.541+0000 7f9915705640 1 --2- 172.21.15.117:0/1099908374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9910102a60 0x7f991019a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.541+0000 7f9915705640 1 -- 172.21.15.117:0/1099908374 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99101a0390 con 0x7f99101083a0 2024-09-21T06:04:08.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.541+0000 7f9915705640 1 --2- 172.21.15.117:0/1099908374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99101083a0 0x7f991019a9c0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f990000e9c0 tx=0x7f990000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:08.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.541+0000 7f9906ffd640 1 -- 172.21.15.117:0/1099908374 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990000cd80 con 0x7f99101083a0 2024-09-21T06:04:08.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.541+0000 7f991816a640 1 -- 172.21.15.117:0/1099908374 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99101a0680 con 0x7f99101083a0 2024-09-21T06:04:08.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.541+0000 7f9906ffd640 1 -- 172.21.15.117:0/1099908374 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99000054d0 con 0x7f99101083a0 2024-09-21T06:04:08.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.541+0000 7f9906ffd640 1 -- 172.21.15.117:0/1099908374 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9900010430 con 0x7f99101083a0 2024-09-21T06:04:08.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.542+0000 7f991816a640 1 -- 172.21.15.117:0/1099908374 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99101120a0 con 0x7f99101083a0 2024-09-21T06:04:08.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.544+0000 7f9906ffd640 1 -- 172.21.15.117:0/1099908374 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f9900004fd0 con 0x7f99101083a0 2024-09-21T06:04:08.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.545+0000 7f991e6de740 1 Processor -- start 2024-09-21T06:04:08.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.545+0000 7f991e6de740 1 -- start start 2024-09-21T06:04:08.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.545+0000 7f991e6de740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588475db070 0x5588479822c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:08.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.545+0000 7f991e6de740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588475b21b0 0x558847982800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:08.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.545+0000 7f991e6de740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588475ba390 0x558847982d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:08.548 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.545+0000 7f991e6de740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5588478f8ea0 con 0x5588475b21b0 2024-09-21T06:04:08.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.545+0000 7f991e6de740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5588478e82d0 con 0x5588475ba390 2024-09-21T06:04:08.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.545+0000 7f991e6de740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5588478ec0e0 con 0x5588475db070 2024-09-21T06:04:08.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.546+0000 7f9916707640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588475db070 0x5588479822c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:08.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.546+0000 7f9915f06640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588475b21b0 0x558847982800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:08.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.546+0000 7f9916707640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588475db070 0x5588479822c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:59874/0 (socket says 172.21.15.117:59874) 2024-09-21T06:04:08.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.546+0000 7f9916707640 1 -- 172.21.15.117:0/2037589589 learned_addr learned my addr 172.21.15.117:0/2037589589 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:08.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.546+0000 7f9906ffd640 1 --2- 172.21.15.117:0/1099908374 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f98fc0b6700 0x7f98fc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:08.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.546+0000 7f9915705640 1 --2- 172.21.15.117:0/2037589589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588475ba390 0x558847982d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:08.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.546+0000 7f9916707640 1 -- 172.21.15.117:0/2037589589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588475ba390 msgr2=0x558847982d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:08.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.546+0000 7f9916707640 1 --2- 172.21.15.117:0/2037589589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588475ba390 0x558847982d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.546+0000 7f9916707640 1 -- 172.21.15.117:0/2037589589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588475b21b0 msgr2=0x558847982800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:08.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.546+0000 7f9916707640 1 --2- 172.21.15.117:0/2037589589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588475b21b0 0x558847982800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.546+0000 7f9916707640 1 -- 172.21.15.117:0/2037589589 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558847646650 con 0x5588475db070 2024-09-21T06:04:08.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.546+0000 7f9915705640 1 --2- 172.21.15.117:0/2037589589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588475ba390 0x558847982d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:04:08.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.546+0000 7f9916707640 1 --2- 172.21.15.117:0/1099908374 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f98fc0b6700 0x7f98fc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:08.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.546+0000 7f9906ffd640 1 -- 172.21.15.117:0/1099908374 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f99000e7a60 con 0x7f99101083a0 2024-09-21T06:04:08.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.546+0000 7f9916707640 1 --2- 172.21.15.117:0/2037589589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588475db070 0x5588479822c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9908009480 tx=0x7f9908009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c20accfd9e2cfd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:08.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.547+0000 7f98fbfff640 1 -- 172.21.15.117:0/2037589589 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990804c070 con 0x5588475db070 2024-09-21T06:04:08.550 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.547+0000 7f98fbfff640 1 -- 172.21.15.117:0/2037589589 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9908036070 con 0x5588475db070 2024-09-21T06:04:08.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.548+0000 7f98fbfff640 1 -- 172.21.15.117:0/2037589589 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9908042040 con 0x5588475db070 2024-09-21T06:04:08.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.548+0000 7f991e6de740 1 -- 172.21.15.117:0/2037589589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588475db070 msgr2=0x5588479822c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:08.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.548+0000 7f9916707640 1 --2- 172.21.15.117:0/1099908374 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f98fc0b6700 0x7f98fc0b8c10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9908003fd0 tx=0x7f9908003f20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:08.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.548+0000 7f991e6de740 1 --2- 172.21.15.117:0/2037589589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588475db070 0x5588479822c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9908009480 tx=0x7f9908009800 comp rx=0 tx=0).stop 2024-09-21T06:04:08.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.548+0000 7f991e6de740 1 -- 172.21.15.117:0/2037589589 shutdown_connections 2024-09-21T06:04:08.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.548+0000 7f991e6de740 1 --2- 172.21.15.117:0/2037589589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588475b21b0 0x558847982800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.548+0000 7f991e6de740 1 --2- 172.21.15.117:0/2037589589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588475ba390 0x558847982d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.548+0000 7f991e6de740 1 --2- 172.21.15.117:0/2037589589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588475db070 0x5588479822c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.548+0000 7f991e6de740 1 -- 172.21.15.117:0/2037589589 >> 172.21.15.117:0/2037589589 conn(0x5588475cf460 msgr2=0x5588479801b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:08.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.548+0000 7f991e6de740 1 -- 172.21.15.117:0/2037589589 shutdown_connections 2024-09-21T06:04:08.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.548+0000 7f991e6de740 1 -- 172.21.15.117:0/2037589589 wait complete. 2024-09-21T06:04:08.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.549+0000 7f991e6de740 1 -- start start 2024-09-21T06:04:08.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.549+0000 7f991e6de740 1 Processor -- start 2024-09-21T06:04:08.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.550+0000 7f991e6de740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588475b21b0 0x55884798d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:08.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.550+0000 7f991e6de740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588475ba390 0x55884798dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:08.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.550+0000 7f9915f06640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588475b21b0 0x55884798d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:08.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.550+0000 7f9915f06640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588475b21b0 0x55884798d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:59884/0 (socket says 172.21.15.117:59884) 2024-09-21T06:04:08.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.550+0000 7f9915f06640 1 -- 172.21.15.117:0/3453911936 learned_addr learned my addr 172.21.15.117:0/3453911936 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:08.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.550+0000 7f991e6de740 1 --2- 172.21.15.117:0/3453911936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588475db070 0x55884798e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:08.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.551+0000 7f991e6de740 1 -- 172.21.15.117:0/3453911936 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558847749700 con 0x5588475ba390 2024-09-21T06:04:08.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.551+0000 7f991e6de740 1 -- 172.21.15.117:0/3453911936 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558847883160 con 0x5588475db070 2024-09-21T06:04:08.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.551+0000 7f991e6de740 1 -- 172.21.15.117:0/3453911936 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558847990b40 con 0x5588475b21b0 2024-09-21T06:04:08.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.551+0000 7f991e6de740 10 client.0 _ll_register_callbacks cb 0x55884789c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:08.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.551+0000 7f9915f06640 1 -- 172.21.15.117:0/3453911936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588475db070 msgr2=0x55884798e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:08.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.551+0000 7f9915f06640 1 --2- 172.21.15.117:0/3453911936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588475db070 0x55884798e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.551+0000 7f9915f06640 1 -- 172.21.15.117:0/3453911936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588475ba390 msgr2=0x55884798dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:08.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.551+0000 7f9915f06640 1 --2- 172.21.15.117:0/3453911936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588475ba390 0x55884798dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.551+0000 7f9915f06640 1 -- 172.21.15.117:0/3453911936 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588478b0c90 con 0x5588475b21b0 2024-09-21T06:04:08.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.551+0000 7f991e6de740 1 -- 172.21.15.117:0/3453911936 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9908036070 con 0x5588475b21b0 2024-09-21T06:04:08.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.551+0000 7f9915f06640 1 --2- 172.21.15.117:0/3453911936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588475b21b0 0x55884798d820 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f990c009980 tx=0x7f990c009e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0aa8c6297266280 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:08.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.551+0000 7f98f97fa640 10 client.25852 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-21T06:04:08.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.552+0000 7f98f97fa640 1 -- 172.21.15.117:0/3453911936 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c029050 con 0x5588475b21b0 2024-09-21T06:04:08.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.552+0000 7f98f97fa640 1 -- 172.21.15.117:0/3453911936 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f990c00bdb0 con 0x5588475b21b0 2024-09-21T06:04:08.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.552+0000 7f98f97fa640 1 -- 172.21.15.117:0/3453911936 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f990c01f070 con 0x5588475b21b0 2024-09-21T06:04:08.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.552+0000 7f991e6de740 10 client.25852 fetch_fsmap learned FSMap version 27 2024-09-21T06:04:08.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.552+0000 7f991e6de740 1 -- 172.21.15.117:0/3453911936 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55884799c730 con 0x5588475b21b0 2024-09-21T06:04:08.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.552+0000 7f98f97fa640 1 -- 172.21.15.117:0/3453911936 <== mon.1 v2:172.21.15.192:3300/0 4 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f990c024440 con 0x5588475b21b0 2024-09-21T06:04:08.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.552+0000 7f991e6de740 10 client.25852 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:04:08.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.552+0000 7f991e6de740 10 client.25852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:04:08.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.553+0000 7f991e6de740 20 client.25852 populate_metadata read hostname 'smithi117' 2024-09-21T06:04:08.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.553+0000 7f991e6de740 1 --2- 172.21.15.117:0/3453911936 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5588475999f0 0x55884799fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:08.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.553+0000 7f991e6de740 4 client.25852 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:04:08.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.553+0000 7f991e6de740 1 -- 172.21.15.117:0/3453911936 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588478fadf0 con 0x5588475999f0 2024-09-21T06:04:08.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.553+0000 7f9915705640 1 --2- 172.21.15.117:0/3453911936 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5588475999f0 0x55884799fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:08.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.553+0000 7f9915705640 1 --2- 172.21.15.117:0/3453911936 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5588475999f0 0x55884799fb80 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbd4143011a61980 server_cookie=ed9641e6112b8453 in_seq=0 out_seq=0 2024-09-21T06:04:08.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.553+0000 7f98f97fa640 10 client.25852 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:08.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.554+0000 7f98f97fa640 1 -- 172.21.15.117:0/3453911936 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5588478fadf0 con 0x5588475999f0 2024-09-21T06:04:08.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.554+0000 7f98f97fa640 10 client.25852 handle_command_reply: tid=0 2024-09-21T06:04:08.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.554+0000 7f991e6de740 1 client.25852 shutdown 2024-09-21T06:04:08.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.554+0000 7f991e6de740 1 -- 172.21.15.117:0/3453911936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588475b21b0 msgr2=0x55884798d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:08.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.554+0000 7f991e6de740 1 --2- 172.21.15.117:0/3453911936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588475b21b0 0x55884798d820 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f990c009980 tx=0x7f990c009e40 comp rx=0 tx=0).stop 2024-09-21T06:04:08.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.555+0000 7f991e6de740 1 -- 172.21.15.117:0/3453911936 shutdown_connections 2024-09-21T06:04:08.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.555+0000 7f991e6de740 1 --2- 172.21.15.117:0/3453911936 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5588475999f0 0x55884799fb80 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.555+0000 7f991e6de740 1 --2- 172.21.15.117:0/3453911936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588475ba390 0x55884798dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.555+0000 7f98f97fa640 0 client.25852 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:08.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.555+0000 7f991e6de740 1 --2- 172.21.15.117:0/3453911936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588475db070 0x55884798e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.555+0000 7f991e6de740 1 --2- 172.21.15.117:0/3453911936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588475b21b0 0x55884798d820 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.555+0000 7f991e6de740 1 -- 172.21.15.117:0/3453911936 >> 172.21.15.117:0/3453911936 conn(0x5588475cf460 msgr2=0x558847985450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:08.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.555+0000 7f991e6de740 1 -- 172.21.15.117:0/3453911936 shutdown_connections 2024-09-21T06:04:08.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.555+0000 7f991e6de740 1 -- 172.21.15.117:0/3453911936 wait complete. 2024-09-21T06:04:08.559 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.555+0000 7f991e6de740 20 client.25852 trim_cache size 0 max 16384 2024-09-21T06:04:08.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.572+0000 7f991e6de740 1 Processor -- start 2024-09-21T06:04:08.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.572+0000 7f991e6de740 1 -- start start 2024-09-21T06:04:08.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.572+0000 7f991e6de740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588475999f0 0x55884798f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:08.574 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.572+0000 7f991e6de740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588475b21b0 0x558847999480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:08.575 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.573+0000 7f991e6de740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588475ba390 0x55884798aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:08.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.573+0000 7f991e6de740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f99101ad3a0 con 0x5588475ba390 2024-09-21T06:04:08.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.573+0000 7f991e6de740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f99101aeee0 con 0x5588475b21b0 2024-09-21T06:04:08.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.573+0000 7f991e6de740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f99101ada70 con 0x5588475999f0 2024-09-21T06:04:08.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.573+0000 7f9916707640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588475ba390 0x55884798aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:08.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.573+0000 7f9915f06640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588475999f0 0x55884798f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:08.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.573+0000 7f9915705640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588475b21b0 0x558847999480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:08.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.573+0000 7f9915f06640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588475999f0 0x55884798f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:59892/0 (socket says 172.21.15.117:59892) 2024-09-21T06:04:08.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.573+0000 7f9915f06640 1 -- 172.21.15.117:0/2030045786 learned_addr learned my addr 172.21.15.117:0/2030045786 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:08.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.573+0000 7f9915f06640 1 -- 172.21.15.117:0/2030045786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588475b21b0 msgr2=0x558847999480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:08.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.573+0000 7f9915f06640 1 --2- 172.21.15.117:0/2030045786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588475b21b0 0x558847999480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.573+0000 7f9915f06640 1 -- 172.21.15.117:0/2030045786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588475ba390 msgr2=0x55884798aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:08.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.573+0000 7f9915f06640 1 --2- 172.21.15.117:0/2030045786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588475ba390 0x55884798aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.573+0000 7f9915f06640 1 -- 172.21.15.117:0/2030045786 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55884798e5e0 con 0x5588475999f0 2024-09-21T06:04:08.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.573+0000 7f9915f06640 1 --2- 172.21.15.117:0/2030045786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588475999f0 0x55884798f700 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f990c00a010 tx=0x7f990c01c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caeb4f33918fa005 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:08.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.574+0000 7f98fb7fe640 1 -- 172.21.15.117:0/2030045786 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c029030 con 0x5588475999f0 2024-09-21T06:04:08.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.574+0000 7f98fb7fe640 1 -- 172.21.15.117:0/2030045786 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9908036070 con 0x5588475999f0 2024-09-21T06:04:08.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.574+0000 7f98fb7fe640 1 -- 172.21.15.117:0/2030045786 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c004a00 con 0x5588475999f0 2024-09-21T06:04:08.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.575+0000 7f991e6de740 1 -- 172.21.15.117:0/2030045786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588475999f0 msgr2=0x55884798f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:08.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.575+0000 7f991e6de740 1 --2- 172.21.15.117:0/2030045786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588475999f0 0x55884798f700 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f990c00a010 tx=0x7f990c01c000 comp rx=0 tx=0).stop 2024-09-21T06:04:08.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.575+0000 7f991e6de740 1 -- 172.21.15.117:0/2030045786 shutdown_connections 2024-09-21T06:04:08.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.575+0000 7f991e6de740 1 --2- 172.21.15.117:0/2030045786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588475ba390 0x55884798aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.575+0000 7f991e6de740 1 --2- 172.21.15.117:0/2030045786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588475b21b0 0x558847999480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.575+0000 7f991e6de740 1 --2- 172.21.15.117:0/2030045786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588475999f0 0x55884798f700 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.575+0000 7f991e6de740 1 -- 172.21.15.117:0/2030045786 >> 172.21.15.117:0/2030045786 conn(0x5588475cf460 msgr2=0x558847998d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:08.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.575+0000 7f991e6de740 1 -- 172.21.15.117:0/2030045786 shutdown_connections 2024-09-21T06:04:08.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.575+0000 7f991e6de740 1 -- 172.21.15.117:0/2030045786 wait complete. 2024-09-21T06:04:08.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.576+0000 7f991e6de740 1 -- start start 2024-09-21T06:04:08.578 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.576+0000 7f991e6de740 1 Processor -- start 2024-09-21T06:04:08.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.576+0000 7f991e6de740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588475999f0 0x558847991ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:08.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.576+0000 7f991e6de740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588475b21b0 0x558847992430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:08.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.576+0000 7f991e6de740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588475ba390 0x558847992970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:08.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.576+0000 7f991e6de740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f99101ae810 con 0x5588475b21b0 2024-09-21T06:04:08.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.576+0000 7f991e6de740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f99101ae140 con 0x5588475999f0 2024-09-21T06:04:08.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.576+0000 7f991e6de740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f99101ab3a0 con 0x5588475ba390 2024-09-21T06:04:08.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.576+0000 7f991e6de740 10 client.0 _ll_register_callbacks cb 0x7f990c010ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:08.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.577+0000 7f9915705640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588475b21b0 0x558847992430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:08.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.577+0000 7f9915705640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588475b21b0 0x558847992430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38444/0 (socket says 172.21.15.117:38444) 2024-09-21T06:04:08.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.577+0000 7f9915705640 1 -- 172.21.15.117:0/1650758551 learned_addr learned my addr 172.21.15.117:0/1650758551 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:08.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.577+0000 7f9915f06640 1 --2- 172.21.15.117:0/1650758551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588475999f0 0x558847991ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:08.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.577+0000 7f9916707640 1 --2- 172.21.15.117:0/1650758551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588475ba390 0x558847992970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:08.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.577+0000 7f9915705640 1 -- 172.21.15.117:0/1650758551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588475ba390 msgr2=0x558847992970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:08.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.577+0000 7f9915705640 1 --2- 172.21.15.117:0/1650758551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588475ba390 0x558847992970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.577+0000 7f9915705640 1 -- 172.21.15.117:0/1650758551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588475999f0 msgr2=0x558847991ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:08.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.577+0000 7f9915705640 1 --2- 172.21.15.117:0/1650758551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588475999f0 0x558847991ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.577+0000 7f9915705640 1 -- 172.21.15.117:0/1650758551 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558847a09670 con 0x5588475b21b0 2024-09-21T06:04:08.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.577+0000 7f9915705640 1 --2- 172.21.15.117:0/1650758551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588475b21b0 0x558847992430 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f99000e52e0 tx=0x7f99000eb0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4882e4b3f161cf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:08.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.577+0000 7f98f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:04:08.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.577+0000 7f98f97fa640 1 -- 172.21.15.117:0/1650758551 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99000ad700 con 0x5588475b21b0 2024-09-21T06:04:08.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.578+0000 7f98f97fa640 1 -- 172.21.15.117:0/1650758551 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(76..76 src has 1..76) ==== 8397+0+0 (secure 0 0 0) 0x7f99000a9ba0 con 0x5588475b21b0 2024-09-21T06:04:08.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.578+0000 7f991e6de740 1 -- 172.21.15.117:0/1650758551 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9908036070 con 0x5588475b21b0 2024-09-21T06:04:08.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.578+0000 7f98f97fa640 1 -- 172.21.15.117:0/1650758551 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f99000a90a0 con 0x5588475b21b0 2024-09-21T06:04:08.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.578+0000 7f991e6de740 10 client.16671 fetch_fsmap learned FSMap version 27 2024-09-21T06:04:08.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.578+0000 7f991e6de740 1 -- 172.21.15.117:0/1650758551 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558847a0a950 con 0x5588475b21b0 2024-09-21T06:04:08.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.578+0000 7f98f97fa640 1 -- 172.21.15.117:0/1650758551 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9908036070 con 0x5588475b21b0 2024-09-21T06:04:08.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.578+0000 7f98f97fa640 1 -- 172.21.15.117:0/1650758551 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f99000a9280 con 0x5588475b21b0 2024-09-21T06:04:08.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.578+0000 7f991e6de740 10 client.16671 fetch_fsmap finished waiting for FSMap version 27 2024-09-21T06:04:08.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.578+0000 7f991e6de740 10 client.16671 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:04:08.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.578+0000 7f991e6de740 20 client.16671 populate_metadata read hostname 'smithi117' 2024-09-21T06:04:08.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.578+0000 7f991e6de740 1 --2- 172.21.15.117:0/1650758551 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5588475db070 0x558847a0d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:08.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.578+0000 7f991e6de740 4 client.16671 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:04:08.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.578+0000 7f991e6de740 1 -- 172.21.15.117:0/1650758551 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5588478f68a0 con 0x5588475db070 2024-09-21T06:04:08.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.579+0000 7f9915f06640 1 --2- 172.21.15.117:0/1650758551 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5588475db070 0x558847a0d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:08.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.579+0000 7f98f97fa640 10 client.16671 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:08.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.579+0000 7f9915f06640 1 --2- 172.21.15.117:0/1650758551 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5588475db070 0x558847a0d140 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37607df3e25900a2 server_cookie=fb8639b0e456144d in_seq=0 out_seq=0 2024-09-21T06:04:08.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.580+0000 7f98f97fa640 1 -- 172.21.15.117:0/1650758551 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5588478f68a0 con 0x5588475db070 2024-09-21T06:04:08.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.580+0000 7f98f97fa640 10 client.16671 handle_command_reply: tid=0 2024-09-21T06:04:08.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.580+0000 7f991e6de740 1 client.16671 shutdown 2024-09-21T06:04:08.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.580+0000 7f991e6de740 1 -- 172.21.15.117:0/1650758551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588475b21b0 msgr2=0x558847992430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:08.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.580+0000 7f991e6de740 1 --2- 172.21.15.117:0/1650758551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588475b21b0 0x558847992430 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f99000e52e0 tx=0x7f99000eb0c0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.580+0000 7f991e6de740 1 -- 172.21.15.117:0/1650758551 shutdown_connections 2024-09-21T06:04:08.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.580+0000 7f991e6de740 1 --2- 172.21.15.117:0/1650758551 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5588475db070 0x558847a0d140 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.580+0000 7f991e6de740 1 --2- 172.21.15.117:0/1650758551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588475b21b0 0x558847992430 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.580+0000 7f991e6de740 1 --2- 172.21.15.117:0/1650758551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588475ba390 0x558847992970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.580+0000 7f991e6de740 1 --2- 172.21.15.117:0/1650758551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5588475999f0 0x558847991ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.580+0000 7f991e6de740 1 -- 172.21.15.117:0/1650758551 >> 172.21.15.117:0/1650758551 conn(0x5588475cf460 msgr2=0x55884798f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:08.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.580+0000 7f98f97fa640 0 client.16671 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:08.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.580+0000 7f991e6de740 1 -- 172.21.15.117:0/1650758551 shutdown_connections 2024-09-21T06:04:08.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.580+0000 7f991e6de740 1 -- 172.21.15.117:0/1650758551 wait complete. 2024-09-21T06:04:08.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.581+0000 7f991e6de740 20 client.16671 trim_cache size 0 max 16384 2024-09-21T06:04:08.585 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:04:08.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.581+0000 7f991816a640 1 -- 172.21.15.117:0/1099908374 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f98fc0b6700 msgr2=0x7f98fc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:08.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.581+0000 7f991816a640 1 --2- 172.21.15.117:0/1099908374 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f98fc0b6700 0x7f98fc0b8c10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9908003fd0 tx=0x7f9908003f20 comp rx=0 tx=0).stop 2024-09-21T06:04:08.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.581+0000 7f991816a640 1 -- 172.21.15.117:0/1099908374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99101083a0 msgr2=0x7f991019a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:08.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.581+0000 7f991816a640 1 --2- 172.21.15.117:0/1099908374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99101083a0 0x7f991019a9c0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f990000e9c0 tx=0x7f990000ee80 comp rx=0 tx=0).stop 2024-09-21T06:04:08.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.581+0000 7f991816a640 1 -- 172.21.15.117:0/1099908374 shutdown_connections 2024-09-21T06:04:08.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.581+0000 7f991816a640 1 --2- 172.21.15.117:0/1099908374 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f98fc0b6700 0x7f98fc0b8c10 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.581+0000 7f991816a640 1 --2- 172.21.15.117:0/1099908374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99101083a0 0x7f991019a9c0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.581+0000 7f991816a640 1 --2- 172.21.15.117:0/1099908374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9910108cf0 0x7f991019fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.581+0000 7f991816a640 1 --2- 172.21.15.117:0/1099908374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9910102a60 0x7f991019a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:08.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.581+0000 7f991816a640 1 -- 172.21.15.117:0/1099908374 >> 172.21.15.117:0/1099908374 conn(0x7f9910079730 msgr2=0x7f9910104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:08.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.582+0000 7f991816a640 1 -- 172.21.15.117:0/1099908374 shutdown_connections 2024-09-21T06:04:08.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:08.582+0000 7f991816a640 1 -- 172.21.15.117:0/1099908374 wait complete. 2024-09-21T06:04:08.730 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 21 06:04:08 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-l[63468]: ignoring --setuser ceph since I am not root 2024-09-21T06:04:08.730 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 21 06:04:08 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-l[63468]: ignoring --setgroup ceph since I am not root 2024-09-21T06:04:08.730 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 21 06:04:08 smithi192 ceph-mds[63472]: -- 172.21.15.192:0/1557750108 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e539b5f860 con 0x55e53dd07a80 2024-09-21T06:04:08.730 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 21 06:04:08 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-l[63468]: starting mds.l at 2024-09-21T06:04:09.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:08 smithi192 ceph-mon[34700]: daemon mds.l restarted 2024-09-21T06:04:09.250 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:08 smithi117 ceph-mon[30947]: daemon mds.l restarted 2024-09-21T06:04:09.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:08 smithi138 ceph-mon[34391]: daemon mds.l restarted 2024-09-21T06:04:10.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:09 smithi192 ceph-mon[34700]: pgmap v532: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 798 MiB/s wr, 2.54k op/s 2024-09-21T06:04:10.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:09 smithi192 ceph-mon[34700]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T06:04:10.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:09 smithi192 ceph-mon[34700]: mds.? [v2:172.21.15.192:6832/1596605058,v1:172.21.15.192:6833/1596605058] up:boot 2024-09-21T06:04:10.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:09 smithi192 ceph-mon[34700]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-21T06:04:10.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:09 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T06:04:10.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:09 smithi192 ceph-mon[34700]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:04:10.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:09 smithi117 ceph-mon[30947]: pgmap v532: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 798 MiB/s wr, 2.54k op/s 2024-09-21T06:04:10.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:09 smithi117 ceph-mon[30947]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T06:04:10.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:09 smithi117 ceph-mon[30947]: mds.? [v2:172.21.15.192:6832/1596605058,v1:172.21.15.192:6833/1596605058] up:boot 2024-09-21T06:04:10.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:09 smithi117 ceph-mon[30947]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-21T06:04:10.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:09 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T06:04:10.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:09 smithi117 ceph-mon[30947]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:04:10.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:09 smithi138 ceph-mon[34391]: pgmap v532: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 798 MiB/s wr, 2.54k op/s 2024-09-21T06:04:10.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:09 smithi138 ceph-mon[34391]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T06:04:10.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:09 smithi138 ceph-mon[34391]: mds.? [v2:172.21.15.192:6832/1596605058,v1:172.21.15.192:6833/1596605058] up:boot 2024-09-21T06:04:10.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:09 smithi138 ceph-mon[34391]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-21T06:04:10.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:09 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T06:04:10.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:09 smithi138 ceph-mon[34391]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-21T06:04:10.533 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:04:10.533 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:04:10.799 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:04:11.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:10 smithi192 ceph-mon[34700]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T06:04:11.239 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:10 smithi117 ceph-mon[30947]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T06:04:11.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.331+0000 7fbea678e640 1 -- 172.21.15.117:0/3349025053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbea0108d70 msgr2=0x7fbea010f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:11.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.331+0000 7fbea678e640 1 --2- 172.21.15.117:0/3349025053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbea0108d70 0x7fbea010f5a0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fbe98009ca0 tx=0x7fbe9802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.331+0000 7fbea678e640 1 -- 172.21.15.117:0/3349025053 shutdown_connections 2024-09-21T06:04:11.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.331+0000 7fbea678e640 1 --2- 172.21.15.117:0/3349025053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbea0108d70 0x7fbea010f5a0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.331+0000 7fbea678e640 1 --2- 172.21.15.117:0/3349025053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbea0108420 0x7fbea0108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.331+0000 7fbea678e640 1 --2- 172.21.15.117:0/3349025053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbea0102ae0 0x7fbea0102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.331+0000 7fbea678e640 1 -- 172.21.15.117:0/3349025053 >> 172.21.15.117:0/3349025053 conn(0x7fbea0079770 msgr2=0x7fbea0079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:11.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.331+0000 7fbea678e640 1 -- 172.21.15.117:0/3349025053 shutdown_connections 2024-09-21T06:04:11.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.331+0000 7fbea678e640 1 -- 172.21.15.117:0/3349025053 wait complete. 2024-09-21T06:04:11.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.331+0000 7fbea678e640 1 Processor -- start 2024-09-21T06:04:11.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.332+0000 7fbea678e640 1 -- start start 2024-09-21T06:04:11.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.332+0000 7fbea678e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbea0102ae0 0x7fbea019b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:11.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.332+0000 7fbea678e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbea0108420 0x7fbea019b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:11.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.332+0000 7fbea678e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbea0108d70 0x7fbea019bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:11.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.332+0000 7fbea678e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbea0112a30 con 0x7fbea0102ae0 2024-09-21T06:04:11.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.332+0000 7fbea678e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbea01128b0 con 0x7fbea0108420 2024-09-21T06:04:11.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.332+0000 7fbea678e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbea0112bb0 con 0x7fbea0108d70 2024-09-21T06:04:11.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.332+0000 7fbe9ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbea0102ae0 0x7fbea019b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:11.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.332+0000 7fbe9ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbea0102ae0 0x7fbea019b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38456/0 (socket says 172.21.15.117:38456) 2024-09-21T06:04:11.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.332+0000 7fbe9ffff640 1 -- 172.21.15.117:0/261172832 learned_addr learned my addr 172.21.15.117:0/261172832 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:11.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.332+0000 7fbea4d2b640 1 --2- 172.21.15.117:0/261172832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbea0108d70 0x7fbea019bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:11.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.332+0000 7fbe97fff640 1 --2- 172.21.15.117:0/261172832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbea0108420 0x7fbea019b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:11.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.332+0000 7fbe9ffff640 1 -- 172.21.15.117:0/261172832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbea0108d70 msgr2=0x7fbea019bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:11.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.332+0000 7fbe9ffff640 1 --2- 172.21.15.117:0/261172832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbea0108d70 0x7fbea019bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.332+0000 7fbe9ffff640 1 -- 172.21.15.117:0/261172832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbea0108420 msgr2=0x7fbea019b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:11.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.332+0000 7fbe9ffff640 1 --2- 172.21.15.117:0/261172832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbea0108420 0x7fbea019b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.332+0000 7fbe9ffff640 1 -- 172.21.15.117:0/261172832 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea01a6580 con 0x7fbea0102ae0 2024-09-21T06:04:11.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.332+0000 7fbe9ffff640 1 --2- 172.21.15.117:0/261172832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbea0102ae0 0x7fbea019b2c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fbe9000d950 tx=0x7fbe9000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:11.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.333+0000 7fbe9dffb640 1 -- 172.21.15.117:0/261172832 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe90017070 con 0x7fbea0102ae0 2024-09-21T06:04:11.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.333+0000 7fbe9dffb640 1 -- 172.21.15.117:0/261172832 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe90009c30 con 0x7fbea0102ae0 2024-09-21T06:04:11.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.333+0000 7fbea678e640 1 -- 172.21.15.117:0/261172832 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbea01a6870 con 0x7fbea0102ae0 2024-09-21T06:04:11.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.333+0000 7fbe9dffb640 1 -- 172.21.15.117:0/261172832 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9000f620 con 0x7fbea0102ae0 2024-09-21T06:04:11.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.333+0000 7fbea678e640 1 -- 172.21.15.117:0/261172832 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbea01a6db0 con 0x7fbea0102ae0 2024-09-21T06:04:11.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.336+0000 7fbe9dffb640 1 -- 172.21.15.117:0/261172832 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fbe90010460 con 0x7fbea0102ae0 2024-09-21T06:04:11.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.336+0000 7fbeacd02740 1 Processor -- start 2024-09-21T06:04:11.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.336+0000 7fbeacd02740 1 -- start start 2024-09-21T06:04:11.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.336+0000 7fbeacd02740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6371bb350 0x55f637562430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:11.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.336+0000 7fbeacd02740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f637192490 0x55f637562970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:11.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.336+0000 7fbeacd02740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f63719a670 0x55f637562eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:11.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.336+0000 7fbeacd02740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f6374d8e60 con 0x55f6371bb350 2024-09-21T06:04:11.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.336+0000 7fbeacd02740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f6374c8290 con 0x55f637192490 2024-09-21T06:04:11.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.336+0000 7fbeacd02740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f6374cc0a0 con 0x55f63719a670 2024-09-21T06:04:11.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.336+0000 7fbea4d2b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6371bb350 0x55f637562430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:11.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.336+0000 7fbea4d2b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6371bb350 0x55f637562430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38464/0 (socket says 172.21.15.117:38464) 2024-09-21T06:04:11.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.336+0000 7fbea4d2b640 1 -- 172.21.15.117:0/3393247146 learned_addr learned my addr 172.21.15.117:0/3393247146 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:11.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.337+0000 7fbe9ffff640 1 --2- 172.21.15.117:0/3393247146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f637192490 0x55f637562970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:11.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.337+0000 7fbe9dffb640 1 --2- 172.21.15.117:0/261172832 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbe8c0b6680 0x7fbe8c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:11.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.337+0000 7fbe97fff640 1 --2- 172.21.15.117:0/3393247146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f63719a670 0x55f637562eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:11.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.337+0000 7fbea4d2b640 1 -- 172.21.15.117:0/3393247146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f63719a670 msgr2=0x55f637562eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:11.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.337+0000 7fbea4d2b640 1 --2- 172.21.15.117:0/3393247146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f63719a670 0x55f637562eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.337+0000 7fbea4d2b640 1 -- 172.21.15.117:0/3393247146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f637192490 msgr2=0x55f637562970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:11.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.337+0000 7fbea4d2b640 1 --2- 172.21.15.117:0/3393247146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f637192490 0x55f637562970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.337+0000 7fbea4d2b640 1 -- 172.21.15.117:0/3393247146 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f637226930 con 0x55f6371bb350 2024-09-21T06:04:11.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.337+0000 7fbe9ffff640 1 --2- 172.21.15.117:0/3393247146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f637192490 0x55f637562970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:04:11.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.337+0000 7fbe97fff640 1 --2- 172.21.15.117:0/3393247146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f63719a670 0x55f637562eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:11.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.337+0000 7fbea4d2b640 1 --2- 172.21.15.117:0/261172832 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbe8c0b6680 0x7fbe8c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:11.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.337+0000 7fbe9dffb640 1 -- 172.21.15.117:0/261172832 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8485+0+0 (secure 0 0 0) 0x7fbe900e72c0 con 0x7fbea0102ae0 2024-09-21T06:04:11.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.337+0000 7fbea4d2b640 1 --2- 172.21.15.117:0/3393247146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6371bb350 0x55f637562430 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fbe980098d0 tx=0x7fbe98009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70a23d6f9492046 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:11.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.337+0000 7fbe967fc640 1 -- 172.21.15.117:0/3393247146 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe98042040 con 0x55f6371bb350 2024-09-21T06:04:11.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.337+0000 7fbea4d2b640 1 --2- 172.21.15.117:0/261172832 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbe8c0b6680 0x7fbe8c0b8b90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbe9804d000 tx=0x7fbe9804e000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:11.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.338+0000 7fbe967fc640 1 -- 172.21.15.117:0/3393247146 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe98036070 con 0x55f6371bb350 2024-09-21T06:04:11.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.338+0000 7fbe967fc640 1 -- 172.21.15.117:0/3393247146 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9802f660 con 0x55f6371bb350 2024-09-21T06:04:11.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.338+0000 7fbeacd02740 1 -- 172.21.15.117:0/3393247146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6371bb350 msgr2=0x55f637562430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:11.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.338+0000 7fbeacd02740 1 --2- 172.21.15.117:0/3393247146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6371bb350 0x55f637562430 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fbe980098d0 tx=0x7fbe98009900 comp rx=0 tx=0).stop 2024-09-21T06:04:11.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.338+0000 7fbeacd02740 1 -- 172.21.15.117:0/3393247146 shutdown_connections 2024-09-21T06:04:11.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.338+0000 7fbeacd02740 1 --2- 172.21.15.117:0/3393247146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f63719a670 0x55f637562eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.338+0000 7fbeacd02740 1 --2- 172.21.15.117:0/3393247146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f637192490 0x55f637562970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.338+0000 7fbeacd02740 1 --2- 172.21.15.117:0/3393247146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6371bb350 0x55f637562430 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.338+0000 7fbeacd02740 1 -- 172.21.15.117:0/3393247146 >> 172.21.15.117:0/3393247146 conn(0x55f6371af740 msgr2=0x55f637560320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:11.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.339+0000 7fbeacd02740 1 -- 172.21.15.117:0/3393247146 shutdown_connections 2024-09-21T06:04:11.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.339+0000 7fbeacd02740 1 -- 172.21.15.117:0/3393247146 wait complete. 2024-09-21T06:04:11.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.339+0000 7fbeacd02740 1 -- start start 2024-09-21T06:04:11.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.339+0000 7fbeacd02740 1 Processor -- start 2024-09-21T06:04:11.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.339+0000 7fbeacd02740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f637192490 0x55f63756e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:11.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.339+0000 7fbeacd02740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f63719a670 0x55f63756e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:11.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.339+0000 7fbeacd02740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6371bb350 0x55f637579f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:11.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.339+0000 7fbeacd02740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f637329890 con 0x55f63719a670 2024-09-21T06:04:11.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.339+0000 7fbeacd02740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f637462d40 con 0x55f6371bb350 2024-09-21T06:04:11.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.339+0000 7fbeacd02740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f637570c10 con 0x55f637192490 2024-09-21T06:04:11.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.340+0000 7fbeacd02740 10 client.0 _ll_register_callbacks cb 0x55f6374c6c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:11.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.340+0000 7fbe97fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f63719a670 0x55f63756e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:11.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.340+0000 7fbe97fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f63719a670 0x55f63756e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38466/0 (socket says 172.21.15.117:38466) 2024-09-21T06:04:11.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.340+0000 7fbe97fff640 1 -- 172.21.15.117:0/3664953836 learned_addr learned my addr 172.21.15.117:0/3664953836 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:11.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.340+0000 7fbe9ffff640 1 --2- 172.21.15.117:0/3664953836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f637192490 0x55f63756e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:11.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.340+0000 7fbea4d2b640 1 --2- 172.21.15.117:0/3664953836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6371bb350 0x55f637579f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:11.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.340+0000 7fbe97fff640 1 -- 172.21.15.117:0/3664953836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f637192490 msgr2=0x55f63756e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:11.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.340+0000 7fbe97fff640 1 --2- 172.21.15.117:0/3664953836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f637192490 0x55f63756e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.340+0000 7fbe97fff640 1 -- 172.21.15.117:0/3664953836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6371bb350 msgr2=0x55f637579f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:11.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.340+0000 7fbe97fff640 1 --2- 172.21.15.117:0/3664953836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6371bb350 0x55f637579f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.340+0000 7fbe97fff640 1 -- 172.21.15.117:0/3664953836 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f63757a540 con 0x55f63719a670 2024-09-21T06:04:11.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.340+0000 7fbe97fff640 1 --2- 172.21.15.117:0/3664953836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f63719a670 0x55f63756e620 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fbe8800b920 tx=0x7fbe88004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29105e703f0f49cb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:11.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.340+0000 7fbe7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:04:11.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.340+0000 7fbe7f7fe640 1 -- 172.21.15.117:0/3664953836 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8801b470 con 0x55f63719a670 2024-09-21T06:04:11.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.340+0000 7fbe7f7fe640 1 -- 172.21.15.117:0/3664953836 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8485+0+0 (secure 0 0 0) 0x7fbe8801b610 con 0x55f63719a670 2024-09-21T06:04:11.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.340+0000 7fbeacd02740 1 -- 172.21.15.117:0/3664953836 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe98036070 con 0x55f63719a670 2024-09-21T06:04:11.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.341+0000 7fbe7f7fe640 1 -- 172.21.15.117:0/3664953836 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbe8801b810 con 0x55f63719a670 2024-09-21T06:04:11.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.341+0000 7fbeacd02740 10 client.16683 fetch_fsmap learned FSMap version 29 2024-09-21T06:04:11.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.341+0000 7fbeacd02740 1 -- 172.21.15.117:0/3664953836 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f63757ce40 con 0x55f63719a670 2024-09-21T06:04:11.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.341+0000 7fbe7f7fe640 1 -- 172.21.15.117:0/3664953836 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe98036070 con 0x55f63719a670 2024-09-21T06:04:11.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.341+0000 7fbe7f7fe640 1 -- 172.21.15.117:0/3664953836 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbe88022020 con 0x55f63719a670 2024-09-21T06:04:11.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.341+0000 7fbeacd02740 10 client.16683 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:04:11.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.341+0000 7fbeacd02740 10 client.16683 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:04:11.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.341+0000 7fbeacd02740 20 client.16683 populate_metadata read hostname 'smithi117' 2024-09-21T06:04:11.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.341+0000 7fbeacd02740 1 --2- 172.21.15.117:0/3664953836 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f637179cd0 0x55f637580360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:11.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.341+0000 7fbeacd02740 4 client.16683 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:04:11.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.341+0000 7fbeacd02740 1 -- 172.21.15.117:0/3664953836 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6374dadb0 con 0x55f637179cd0 2024-09-21T06:04:11.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.341+0000 7fbe9ffff640 1 --2- 172.21.15.117:0/3664953836 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f637179cd0 0x55f637580360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:11.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.342+0000 7fbe7f7fe640 10 client.16683 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:11.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.342+0000 7fbe9ffff640 1 --2- 172.21.15.117:0/3664953836 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f637179cd0 0x55f637580360 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74f40d5ab16475d server_cookie=f95a44cac54f9597 in_seq=0 out_seq=0 2024-09-21T06:04:11.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.343+0000 7fbe7f7fe640 1 -- 172.21.15.117:0/3664953836 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f6374dadb0 con 0x55f637179cd0 2024-09-21T06:04:11.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.343+0000 7fbe7f7fe640 10 client.16683 handle_command_reply: tid=0 2024-09-21T06:04:11.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.343+0000 7fbeacd02740 1 client.16683 shutdown 2024-09-21T06:04:11.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.343+0000 7fbeacd02740 1 -- 172.21.15.117:0/3664953836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f63719a670 msgr2=0x55f63756e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:11.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.343+0000 7fbeacd02740 1 --2- 172.21.15.117:0/3664953836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f63719a670 0x55f63756e620 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fbe8800b920 tx=0x7fbe88004610 comp rx=0 tx=0).stop 2024-09-21T06:04:11.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.343+0000 7fbeacd02740 1 -- 172.21.15.117:0/3664953836 shutdown_connections 2024-09-21T06:04:11.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.343+0000 7fbeacd02740 1 --2- 172.21.15.117:0/3664953836 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f637179cd0 0x55f637580360 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.343+0000 7fbeacd02740 1 --2- 172.21.15.117:0/3664953836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f63719a670 0x55f63756e620 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.343+0000 7fbeacd02740 1 --2- 172.21.15.117:0/3664953836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f6371bb350 0x55f637579f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.343+0000 7fbeacd02740 1 --2- 172.21.15.117:0/3664953836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f637192490 0x55f63756e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.343+0000 7fbeacd02740 1 -- 172.21.15.117:0/3664953836 >> 172.21.15.117:0/3664953836 conn(0x55f6371af740 msgr2=0x55f637565610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:11.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.343+0000 7fbe7f7fe640 0 client.16683 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:11.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.343+0000 7fbeacd02740 1 -- 172.21.15.117:0/3664953836 shutdown_connections 2024-09-21T06:04:11.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.343+0000 7fbeacd02740 1 -- 172.21.15.117:0/3664953836 wait complete. 2024-09-21T06:04:11.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.343+0000 7fbeacd02740 20 client.16683 trim_cache size 0 max 16384 2024-09-21T06:04:11.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.360+0000 7fbeacd02740 1 Processor -- start 2024-09-21T06:04:11.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.360+0000 7fbeacd02740 1 -- start start 2024-09-21T06:04:11.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.361+0000 7fbeacd02740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f637192490 0x55f637579e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:11.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.361+0000 7fbeacd02740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f63719a670 0x55f637567510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:11.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.361+0000 7fbeacd02740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6371bb350 0x55f637567a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:11.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.361+0000 7fbeacd02740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbea01af640 con 0x55f6371bb350 2024-09-21T06:04:11.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.361+0000 7fbeacd02740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbea01b1180 con 0x55f637192490 2024-09-21T06:04:11.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.361+0000 7fbeacd02740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbea01afd10 con 0x55f63719a670 2024-09-21T06:04:11.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.361+0000 7fbea4d2b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6371bb350 0x55f637567a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:11.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.361+0000 7fbea4d2b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6371bb350 0x55f637567a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38478/0 (socket says 172.21.15.117:38478) 2024-09-21T06:04:11.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.361+0000 7fbea4d2b640 1 -- 172.21.15.117:0/1747809024 learned_addr learned my addr 172.21.15.117:0/1747809024 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:11.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.361+0000 7fbe97fff640 1 --2- 172.21.15.117:0/1747809024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f63719a670 0x55f637567510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:11.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.361+0000 7fbea4d2b640 1 -- 172.21.15.117:0/1747809024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f63719a670 msgr2=0x55f637567510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:11.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.361+0000 7fbea4d2b640 1 --2- 172.21.15.117:0/1747809024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f63719a670 0x55f637567510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.361+0000 7fbea4d2b640 1 -- 172.21.15.117:0/1747809024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f637192490 msgr2=0x55f637579e30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:04:11.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.361+0000 7fbea4d2b640 1 --2- 172.21.15.117:0/1747809024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f637192490 0x55f637579e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.361+0000 7fbea4d2b640 1 -- 172.21.15.117:0/1747809024 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f637560c90 con 0x55f6371bb350 2024-09-21T06:04:11.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.361+0000 7fbea4d2b640 1 --2- 172.21.15.117:0/1747809024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6371bb350 0x55f637567a50 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fbe98009970 tx=0x7fbe98031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=242802216ca828ec server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:11.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.361+0000 7fbe95ffb640 1 -- 172.21.15.117:0/1747809024 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe98042040 con 0x55f6371bb350 2024-09-21T06:04:11.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.361+0000 7fbe95ffb640 1 -- 172.21.15.117:0/1747809024 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe98036040 con 0x55f6371bb350 2024-09-21T06:04:11.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.361+0000 7fbe95ffb640 1 -- 172.21.15.117:0/1747809024 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9802f660 con 0x55f6371bb350 2024-09-21T06:04:11.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.362+0000 7fbeacd02740 1 -- 172.21.15.117:0/1747809024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6371bb350 msgr2=0x55f637567a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:11.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.362+0000 7fbeacd02740 1 --2- 172.21.15.117:0/1747809024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6371bb350 0x55f637567a50 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fbe98009970 tx=0x7fbe98031a20 comp rx=0 tx=0).stop 2024-09-21T06:04:11.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.362+0000 7fbeacd02740 1 -- 172.21.15.117:0/1747809024 shutdown_connections 2024-09-21T06:04:11.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.362+0000 7fbeacd02740 1 --2- 172.21.15.117:0/1747809024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f6371bb350 0x55f637567a50 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.362+0000 7fbeacd02740 1 --2- 172.21.15.117:0/1747809024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f63719a670 0x55f637567510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.362+0000 7fbeacd02740 1 --2- 172.21.15.117:0/1747809024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f637192490 0x55f637579e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.362+0000 7fbeacd02740 1 -- 172.21.15.117:0/1747809024 >> 172.21.15.117:0/1747809024 conn(0x55f6371af740 msgr2=0x55f63756c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:11.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.362+0000 7fbeacd02740 1 -- 172.21.15.117:0/1747809024 shutdown_connections 2024-09-21T06:04:11.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.362+0000 7fbeacd02740 1 -- 172.21.15.117:0/1747809024 wait complete. 2024-09-21T06:04:11.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.363+0000 7fbeacd02740 1 -- start start 2024-09-21T06:04:11.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.363+0000 7fbeacd02740 1 Processor -- start 2024-09-21T06:04:11.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:10 smithi138 ceph-mon[34391]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T06:04:11.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.363+0000 7fbeacd02740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f637192490 0x55f637575ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:11.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.363+0000 7fbeacd02740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63719a670 0x55f6375763e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:11.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.363+0000 7fbeacd02740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6371bb350 0x55f637570e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:11.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.363+0000 7fbeacd02740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbea01b0ab0 con 0x55f637192490 2024-09-21T06:04:11.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.363+0000 7fbeacd02740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbea01b03e0 con 0x55f63719a670 2024-09-21T06:04:11.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.363+0000 7fbeacd02740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbea01ad640 con 0x55f6371bb350 2024-09-21T06:04:11.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.363+0000 7fbeacd02740 10 client.0 _ll_register_callbacks cb 0x7fbe88004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:11.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.363+0000 7fbe9ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f637192490 0x55f637575ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:11.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.363+0000 7fbe9ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f637192490 0x55f637575ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38492/0 (socket says 172.21.15.117:38492) 2024-09-21T06:04:11.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.363+0000 7fbe9ffff640 1 -- 172.21.15.117:0/437433654 learned_addr learned my addr 172.21.15.117:0/437433654 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:11.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.363+0000 7fbe97fff640 1 --2- 172.21.15.117:0/437433654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63719a670 0x55f6375763e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:11.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.363+0000 7fbea4d2b640 1 --2- 172.21.15.117:0/437433654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6371bb350 0x55f637570e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:11.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.364+0000 7fbe9ffff640 1 -- 172.21.15.117:0/437433654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6371bb350 msgr2=0x55f637570e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:11.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.364+0000 7fbe9ffff640 1 --2- 172.21.15.117:0/437433654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6371bb350 0x55f637570e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.364+0000 7fbe9ffff640 1 -- 172.21.15.117:0/437433654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63719a670 msgr2=0x55f6375763e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:11.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.364+0000 7fbe9ffff640 1 --2- 172.21.15.117:0/437433654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63719a670 0x55f6375763e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.364+0000 7fbe9ffff640 1 -- 172.21.15.117:0/437433654 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f637576920 con 0x55f637192490 2024-09-21T06:04:11.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.364+0000 7fbe9ffff640 1 --2- 172.21.15.117:0/437433654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f637192490 0x55f637575ea0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fbe900a97b0 tx=0x7fbe900e97d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1be9d8f4bbc6b6f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:11.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.364+0000 7fbe7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:04:11.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.364+0000 7fbe7f7fe640 1 -- 172.21.15.117:0/437433654 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe90020040 con 0x55f637192490 2024-09-21T06:04:11.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.364+0000 7fbe7f7fe640 1 -- 172.21.15.117:0/437433654 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8485+0+0 (secure 0 0 0) 0x7fbe900a8d30 con 0x55f637192490 2024-09-21T06:04:11.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.364+0000 7fbeacd02740 1 -- 172.21.15.117:0/437433654 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe98036040 con 0x55f637192490 2024-09-21T06:04:11.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.364+0000 7fbe7f7fe640 1 -- 172.21.15.117:0/437433654 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbe900ac950 con 0x55f637192490 2024-09-21T06:04:11.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.364+0000 7fbeacd02740 10 client.16689 fetch_fsmap learned FSMap version 29 2024-09-21T06:04:11.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.364+0000 7fbeacd02740 1 -- 172.21.15.117:0/437433654 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f6375eb0c0 con 0x55f637192490 2024-09-21T06:04:11.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.364+0000 7fbe7f7fe640 1 -- 172.21.15.117:0/437433654 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe98036040 con 0x55f637192490 2024-09-21T06:04:11.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.364+0000 7fbe7f7fe640 1 -- 172.21.15.117:0/437433654 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbe900abeb0 con 0x55f637192490 2024-09-21T06:04:11.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.365+0000 7fbeacd02740 10 client.16689 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:04:11.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.365+0000 7fbeacd02740 10 client.16689 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:04:11.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.365+0000 7fbeacd02740 20 client.16689 populate_metadata read hostname 'smithi117' 2024-09-21T06:04:11.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.365+0000 7fbeacd02740 1 --2- 172.21.15.117:0/437433654 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f63714f080 0x55f6375ed7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:11.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.365+0000 7fbeacd02740 4 client.16689 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:04:11.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.365+0000 7fbeacd02740 1 -- 172.21.15.117:0/437433654 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f6374d6860 con 0x55f63714f080 2024-09-21T06:04:11.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.365+0000 7fbe97fff640 1 --2- 172.21.15.117:0/437433654 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f63714f080 0x55f6375ed7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:11.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.365+0000 7fbe97fff640 1 --2- 172.21.15.117:0/437433654 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f63714f080 0x55f6375ed7d0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eea2236672ff760b server_cookie=a1356e1bf0cc98ff in_seq=0 out_seq=0 2024-09-21T06:04:11.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.365+0000 7fbe7f7fe640 10 client.16689 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:11.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.367+0000 7fbe7f7fe640 1 -- 172.21.15.117:0/437433654 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f6374d6860 con 0x55f63714f080 2024-09-21T06:04:11.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.367+0000 7fbe7f7fe640 10 client.16689 handle_command_reply: tid=0 2024-09-21T06:04:11.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.367+0000 7fbeacd02740 1 client.16689 shutdown 2024-09-21T06:04:11.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.367+0000 7fbeacd02740 1 -- 172.21.15.117:0/437433654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f637192490 msgr2=0x55f637575ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:11.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.367+0000 7fbeacd02740 1 --2- 172.21.15.117:0/437433654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f637192490 0x55f637575ea0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fbe900a97b0 tx=0x7fbe900e97d0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.367+0000 7fbeacd02740 1 -- 172.21.15.117:0/437433654 shutdown_connections 2024-09-21T06:04:11.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.367+0000 7fbeacd02740 1 --2- 172.21.15.117:0/437433654 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f63714f080 0x55f6375ed7d0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.367+0000 7fbeacd02740 1 --2- 172.21.15.117:0/437433654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f6371bb350 0x55f637570e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.367+0000 7fbeacd02740 1 --2- 172.21.15.117:0/437433654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f63719a670 0x55f6375763e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.367+0000 7fbeacd02740 1 --2- 172.21.15.117:0/437433654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f637192490 0x55f637575ea0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.367+0000 7fbe7f7fe640 0 client.16689 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:11.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.367+0000 7fbeacd02740 1 -- 172.21.15.117:0/437433654 >> 172.21.15.117:0/437433654 conn(0x55f6371af740 msgr2=0x55f637565530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:11.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.367+0000 7fbeacd02740 1 -- 172.21.15.117:0/437433654 shutdown_connections 2024-09-21T06:04:11.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.367+0000 7fbeacd02740 1 -- 172.21.15.117:0/437433654 wait complete. 2024-09-21T06:04:11.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.368+0000 7fbeacd02740 20 client.16689 trim_cache size 0 max 16384 2024-09-21T06:04:11.373 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T06:04:11.373 INFO:teuthology.orchestra.run.smithi117.stdout: "return_code": 0, 2024-09-21T06:04:11.373 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_tag": "0902e33a-ce33-45fb-92d8-2e6fcb8ec8a0", 2024-09-21T06:04:11.373 INFO:teuthology.orchestra.run.smithi117.stdout: "mode": "asynchronous" 2024-09-21T06:04:11.373 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T06:04:11.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.368+0000 7fbea678e640 1 -- 172.21.15.117:0/261172832 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbe8c0b6680 msgr2=0x7fbe8c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:11.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.368+0000 7fbea678e640 1 --2- 172.21.15.117:0/261172832 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbe8c0b6680 0x7fbe8c0b8b90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbe9804d000 tx=0x7fbe9804e000 comp rx=0 tx=0).stop 2024-09-21T06:04:11.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.368+0000 7fbea678e640 1 -- 172.21.15.117:0/261172832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbea0102ae0 msgr2=0x7fbea019b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:11.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.368+0000 7fbea678e640 1 --2- 172.21.15.117:0/261172832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbea0102ae0 0x7fbea019b2c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fbe9000d950 tx=0x7fbe9000de10 comp rx=0 tx=0).stop 2024-09-21T06:04:11.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.368+0000 7fbea678e640 1 -- 172.21.15.117:0/261172832 shutdown_connections 2024-09-21T06:04:11.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.368+0000 7fbea678e640 1 --2- 172.21.15.117:0/261172832 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbe8c0b6680 0x7fbe8c0b8b90 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.368+0000 7fbea678e640 1 --2- 172.21.15.117:0/261172832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbea0108d70 0x7fbea019bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.368+0000 7fbea678e640 1 --2- 172.21.15.117:0/261172832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbea0108420 0x7fbea019b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.368+0000 7fbea678e640 1 --2- 172.21.15.117:0/261172832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbea0102ae0 0x7fbea019b2c0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:11.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.368+0000 7fbea678e640 1 -- 172.21.15.117:0/261172832 >> 172.21.15.117:0/261172832 conn(0x7fbea0079770 msgr2=0x7fbea0106430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:11.374 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.368+0000 7fbea678e640 1 -- 172.21.15.117:0/261172832 shutdown_connections 2024-09-21T06:04:11.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:11.368+0000 7fbea678e640 1 -- 172.21.15.117:0/261172832 wait complete. 2024-09-21T06:04:12.056 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:04:12.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:11 smithi192 ceph-mon[34700]: pgmap v535: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 592 MiB/s wr, 2.21k op/s 2024-09-21T06:04:12.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:11 smithi192 ceph-mon[34700]: scrub queued for path: / 2024-09-21T06:04:12.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:11 smithi192 ceph-mon[34700]: scrub summary: idle+waiting paths [/] 2024-09-21T06:04:12.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:11 smithi192 ceph-mon[34700]: scrub summary: active paths [/] 2024-09-21T06:04:12.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:11 smithi192 ceph-mon[34700]: scrub summary: idle+waiting paths [/] 2024-09-21T06:04:12.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:11 smithi117 ceph-mon[30947]: pgmap v535: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 592 MiB/s wr, 2.21k op/s 2024-09-21T06:04:12.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:11 smithi117 ceph-mon[30947]: scrub queued for path: / 2024-09-21T06:04:12.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:11 smithi117 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-21T06:04:12.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:11 smithi117 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-21T06:04:12.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:11 smithi117 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-21T06:04:12.304 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:04:12.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:11 smithi138 ceph-mon[34391]: pgmap v535: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 592 MiB/s wr, 2.21k op/s 2024-09-21T06:04:12.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:11 smithi138 ceph-mon[34391]: scrub queued for path: / 2024-09-21T06:04:12.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:11 smithi138 ceph-mon[34391]: scrub summary: idle+waiting paths [/] 2024-09-21T06:04:12.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:11 smithi138 ceph-mon[34391]: scrub summary: active paths [/] 2024-09-21T06:04:12.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:11 smithi138 ceph-mon[34391]: scrub summary: idle+waiting paths [/] 2024-09-21T06:04:12.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.789+0000 7f85b1c5d640 1 -- 172.21.15.117:0/2217181267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85ac1094a0 msgr2=0x7f85ac1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:12.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.789+0000 7f85b1c5d640 1 --2- 172.21.15.117:0/2217181267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85ac1094a0 0x7f85ac1098b0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009900 tx=0x7f85a002a410 comp rx=0 tx=0).stop 2024-09-21T06:04:12.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.789+0000 7f85b1c5d640 1 -- 172.21.15.117:0/2217181267 shutdown_connections 2024-09-21T06:04:12.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.789+0000 7f85b1c5d640 1 --2- 172.21.15.117:0/2217181267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85ac1094a0 0x7f85ac1098b0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.789+0000 7f85b1c5d640 1 --2- 172.21.15.117:0/2217181267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85ac109df0 0x7f85ac110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.789+0000 7f85b1c5d640 1 --2- 172.21.15.117:0/2217181267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85ac103b60 0x7f85ac103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.789+0000 7f85b1c5d640 1 -- 172.21.15.117:0/2217181267 >> 172.21.15.117:0/2217181267 conn(0x7f85ac07b850 msgr2=0x7f85ac07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:12.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.789+0000 7f85b1c5d640 1 -- 172.21.15.117:0/2217181267 shutdown_connections 2024-09-21T06:04:12.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.789+0000 7f85b1c5d640 1 -- 172.21.15.117:0/2217181267 wait complete. 2024-09-21T06:04:12.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.790+0000 7f85b1c5d640 1 Processor -- start 2024-09-21T06:04:12.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.790+0000 7f85b1c5d640 1 -- start start 2024-09-21T06:04:12.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.790+0000 7f85b1c5d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85ac103b60 0x7f85ac19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:12.808 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.790+0000 7f85b1c5d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85ac1094a0 0x7f85ac19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:12.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.790+0000 7f85b1c5d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85ac109df0 0x7f85ac19fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:12.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.790+0000 7f85b1c5d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac113ad0 con 0x7f85ac109df0 2024-09-21T06:04:12.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.790+0000 7f85b1c5d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac113950 con 0x7f85ac103b60 2024-09-21T06:04:12.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.790+0000 7f85b1c5d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac113c50 con 0x7f85ac1094a0 2024-09-21T06:04:12.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.790+0000 7f85abfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85ac109df0 0x7f85ac19fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:12.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.790+0000 7f85abfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85ac109df0 0x7f85ac19fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38500/0 (socket says 172.21.15.117:38500) 2024-09-21T06:04:12.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.790+0000 7f85abfff640 1 -- 172.21.15.117:0/129436945 learned_addr learned my addr 172.21.15.117:0/129436945 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:12.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.790+0000 7f85ab7fe640 1 --2- 172.21.15.117:0/129436945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85ac103b60 0x7f85ac19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:12.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.790+0000 7f85aaffd640 1 --2- 172.21.15.117:0/129436945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85ac1094a0 0x7f85ac19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:12.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.791+0000 7f85abfff640 1 -- 172.21.15.117:0/129436945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85ac1094a0 msgr2=0x7f85ac19aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:12.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.791+0000 7f85abfff640 1 --2- 172.21.15.117:0/129436945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85ac1094a0 0x7f85ac19aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.791+0000 7f85abfff640 1 -- 172.21.15.117:0/129436945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85ac103b60 msgr2=0x7f85ac19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:12.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.791+0000 7f85abfff640 1 --2- 172.21.15.117:0/129436945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85ac103b60 0x7f85ac19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.791+0000 7f85abfff640 1 -- 172.21.15.117:0/129436945 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85ac1a04b0 con 0x7f85ac109df0 2024-09-21T06:04:12.809 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.791+0000 7f85aaffd640 1 --2- 172.21.15.117:0/129436945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85ac1094a0 0x7f85ac19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:12.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.791+0000 7f85abfff640 1 --2- 172.21.15.117:0/129436945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85ac109df0 0x7f85ac19fe40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f85a4009fd0 tx=0x7f85a400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:12.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.791+0000 7f85a8ff9640 1 -- 172.21.15.117:0/129436945 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a4011830 con 0x7f85ac109df0 2024-09-21T06:04:12.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.791+0000 7f85b1c5d640 1 -- 172.21.15.117:0/129436945 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85ac1a07a0 con 0x7f85ac109df0 2024-09-21T06:04:12.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.791+0000 7f85a8ff9640 1 -- 172.21.15.117:0/129436945 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85a4016070 con 0x7f85ac109df0 2024-09-21T06:04:12.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.791+0000 7f85a8ff9640 1 -- 172.21.15.117:0/129436945 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a4010de0 con 0x7f85ac109df0 2024-09-21T06:04:12.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.791+0000 7f85b1c5d640 1 -- 172.21.15.117:0/129436945 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85ac1a6fc0 con 0x7f85ac109df0 2024-09-21T06:04:12.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.793+0000 7f85b81d1740 1 Processor -- start 2024-09-21T06:04:12.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.793+0000 7f85a8ff9640 1 -- 172.21.15.117:0/129436945 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f85a40119d0 con 0x7f85ac109df0 2024-09-21T06:04:12.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.794+0000 7f85b81d1740 1 -- start start 2024-09-21T06:04:12.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.794+0000 7f85b81d1740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be22938070 0x55be22cdf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:12.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.794+0000 7f85b81d1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be2290f1b0 0x55be22cdfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:12.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.794+0000 7f85b81d1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be22917390 0x55be22ce00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:12.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.794+0000 7f85b81d1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55be22c55e50 con 0x55be2290f1b0 2024-09-21T06:04:12.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.794+0000 7f85b81d1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55be22c45280 con 0x55be22917390 2024-09-21T06:04:12.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.794+0000 7f85b81d1740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55be22c49090 con 0x55be22938070 2024-09-21T06:04:12.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.794+0000 7f85ab7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be2290f1b0 0x55be22cdfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:12.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.794+0000 7f85ab7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be2290f1b0 0x55be22cdfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38512/0 (socket says 172.21.15.117:38512) 2024-09-21T06:04:12.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.794+0000 7f85ab7fe640 1 -- 172.21.15.117:0/1547214855 learned_addr learned my addr 172.21.15.117:0/1547214855 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:12.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.794+0000 7f85aaffd640 1 --2- 172.21.15.117:0/1547214855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be22917390 0x55be22ce00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:12.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.794+0000 7f85ab7fe640 1 -- 172.21.15.117:0/1547214855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be22938070 msgr2=0x55be22cdf650 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:04:12.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.794+0000 7f85ab7fe640 1 --2- 172.21.15.117:0/1547214855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be22938070 0x55be22cdf650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.794+0000 7f85a8ff9640 1 --2- 172.21.15.117:0/129436945 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f85940b66b0 0x7f85940b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:12.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.794+0000 7f85ab7fe640 1 -- 172.21.15.117:0/1547214855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be22917390 msgr2=0x55be22ce00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:12.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.794+0000 7f85ab7fe640 1 --2- 172.21.15.117:0/1547214855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be22917390 0x55be22ce00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.794+0000 7f85ab7fe640 1 -- 172.21.15.117:0/1547214855 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be229a3650 con 0x55be2290f1b0 2024-09-21T06:04:12.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.794+0000 7f85aaffd640 1 --2- 172.21.15.117:0/1547214855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be22917390 0x55be22ce00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:12.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.794+0000 7f85abfff640 1 --2- 172.21.15.117:0/129436945 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f85940b66b0 0x7f85940b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:12.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.795+0000 7f85ab7fe640 1 --2- 172.21.15.117:0/1547214855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be2290f1b0 0x55be22cdfb90 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f859c007f90 tx=0x7f859c00f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26690b585b2bfa22 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:12.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.795+0000 7f85a8ff9640 1 -- 172.21.15.117:0/129436945 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8485+0+0 (secure 0 0 0) 0x7f85a40e6c40 con 0x7f85ac109df0 2024-09-21T06:04:12.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.795+0000 7f8591ffb640 1 -- 172.21.15.117:0/1547214855 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c004d20 con 0x55be2290f1b0 2024-09-21T06:04:12.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.795+0000 7f8591ffb640 1 -- 172.21.15.117:0/1547214855 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f859c010bb0 con 0x55be2290f1b0 2024-09-21T06:04:12.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.795+0000 7f8591ffb640 1 -- 172.21.15.117:0/1547214855 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c0056d0 con 0x55be2290f1b0 2024-09-21T06:04:12.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.796+0000 7f85b81d1740 1 -- 172.21.15.117:0/1547214855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be2290f1b0 msgr2=0x55be22cdfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:12.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.796+0000 7f85abfff640 1 --2- 172.21.15.117:0/129436945 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f85940b66b0 0x7f85940b8bc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f85a40e4460 tx=0x7f85a40a2040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:12.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.796+0000 7f85b81d1740 1 --2- 172.21.15.117:0/1547214855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be2290f1b0 0x55be22cdfb90 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f859c007f90 tx=0x7f859c00f850 comp rx=0 tx=0).stop 2024-09-21T06:04:12.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.796+0000 7f85b81d1740 1 -- 172.21.15.117:0/1547214855 shutdown_connections 2024-09-21T06:04:12.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.796+0000 7f85b81d1740 1 --2- 172.21.15.117:0/1547214855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be2290f1b0 0x55be22cdfb90 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.796+0000 7f85b81d1740 1 --2- 172.21.15.117:0/1547214855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be22917390 0x55be22ce00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.796+0000 7f85b81d1740 1 --2- 172.21.15.117:0/1547214855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be22938070 0x55be22cdf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.796+0000 7f85b81d1740 1 -- 172.21.15.117:0/1547214855 >> 172.21.15.117:0/1547214855 conn(0x55be2292c460 msgr2=0x55be22cdd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:12.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.796+0000 7f85b81d1740 1 -- 172.21.15.117:0/1547214855 shutdown_connections 2024-09-21T06:04:12.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.796+0000 7f85b81d1740 1 -- 172.21.15.117:0/1547214855 wait complete. 2024-09-21T06:04:12.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.796+0000 7f85b81d1740 1 -- start start 2024-09-21T06:04:12.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.796+0000 7f85b81d1740 1 Processor -- start 2024-09-21T06:04:12.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.797+0000 7f85b81d1740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be2290f1b0 0x55be22ceadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:12.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.797+0000 7f85b81d1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be22917390 0x55be22ceb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:12.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.797+0000 7f85b81d1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be22938070 0x55be22cf7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:12.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.797+0000 7f85b81d1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55be22aa6700 con 0x55be22917390 2024-09-21T06:04:12.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.797+0000 7f85b81d1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55be22be0110 con 0x55be22938070 2024-09-21T06:04:12.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.797+0000 7f85b81d1740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55be22cede30 con 0x55be2290f1b0 2024-09-21T06:04:12.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.797+0000 7f85b81d1740 10 client.0 _ll_register_callbacks cb 0x55be22bf9990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:12.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.797+0000 7f85ab7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be22917390 0x55be22ceb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:12.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.797+0000 7f85ab7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be22917390 0x55be22ceb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38518/0 (socket says 172.21.15.117:38518) 2024-09-21T06:04:12.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.797+0000 7f85ab7fe640 1 -- 172.21.15.117:0/942191508 learned_addr learned my addr 172.21.15.117:0/942191508 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:12.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.797+0000 7f85aaffd640 1 --2- 172.21.15.117:0/942191508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be2290f1b0 0x55be22ceadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:12.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.797+0000 7f85aaffd640 1 --2- 172.21.15.117:0/942191508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be22938070 0x55be22cf7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:12.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.797+0000 7f85ab7fe640 1 -- 172.21.15.117:0/942191508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be2290f1b0 msgr2=0x55be22ceadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:12.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.797+0000 7f85ab7fe640 1 --2- 172.21.15.117:0/942191508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be2290f1b0 0x55be22ceadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.797+0000 7f85ab7fe640 1 -- 172.21.15.117:0/942191508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be22938070 msgr2=0x55be22cf7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:12.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.797+0000 7f85ab7fe640 1 --2- 172.21.15.117:0/942191508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be22938070 0x55be22cf7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.797+0000 7f85ab7fe640 1 -- 172.21.15.117:0/942191508 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be22c0dc40 con 0x55be22917390 2024-09-21T06:04:12.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.797+0000 7f85aaffd640 1 --2- 172.21.15.117:0/942191508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be2290f1b0 0x55be22ceadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:12.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.797+0000 7f85ab7fe640 1 --2- 172.21.15.117:0/942191508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be22917390 0x55be22ceb2f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f859c000f80 tx=0x7f859c010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b0ae2d96366d50 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:12.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.797+0000 7f858affd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:04:12.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.798+0000 7f858affd640 1 -- 172.21.15.117:0/942191508 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c026020 con 0x55be22917390 2024-09-21T06:04:12.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.798+0000 7f858affd640 1 -- 172.21.15.117:0/942191508 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8485+0+0 (secure 0 0 0) 0x7f859c005810 con 0x55be22917390 2024-09-21T06:04:12.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.798+0000 7f85b81d1740 1 -- 172.21.15.117:0/942191508 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f859c010bb0 con 0x55be22917390 2024-09-21T06:04:12.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.798+0000 7f858affd640 1 -- 172.21.15.117:0/942191508 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f859c031050 con 0x55be22917390 2024-09-21T06:04:12.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.798+0000 7f85b81d1740 10 client.16701 fetch_fsmap learned FSMap version 29 2024-09-21T06:04:12.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.798+0000 7f85b81d1740 1 -- 172.21.15.117:0/942191508 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55be22cf9bd0 con 0x55be22917390 2024-09-21T06:04:12.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.798+0000 7f858affd640 1 -- 172.21.15.117:0/942191508 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f859c010bb0 con 0x55be22917390 2024-09-21T06:04:12.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.798+0000 7f858affd640 1 -- 172.21.15.117:0/942191508 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f859c022820 con 0x55be22917390 2024-09-21T06:04:12.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.798+0000 7f85b81d1740 10 client.16701 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:04:12.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.798+0000 7f85b81d1740 10 client.16701 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:04:12.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.798+0000 7f85b81d1740 20 client.16701 populate_metadata read hostname 'smithi117' 2024-09-21T06:04:12.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.798+0000 7f85b81d1740 1 --2- 172.21.15.117:0/942191508 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55be228f69f0 0x55be22cfcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:12.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.798+0000 7f85b81d1740 4 client.16701 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:04:12.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.798+0000 7f85b81d1740 1 -- 172.21.15.117:0/942191508 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be22c57da0 con 0x55be228f69f0 2024-09-21T06:04:12.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.799+0000 7f85aaffd640 1 --2- 172.21.15.117:0/942191508 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55be228f69f0 0x55be22cfcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:12.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.799+0000 7f85aaffd640 1 --2- 172.21.15.117:0/942191508 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55be228f69f0 0x55be22cfcf30 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a8ea06ca9549097 server_cookie=935d64b6638ae2f5 in_seq=0 out_seq=0 2024-09-21T06:04:12.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.799+0000 7f858affd640 10 client.16701 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:12.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.800+0000 7f858affd640 1 -- 172.21.15.117:0/942191508 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55be22c57da0 con 0x55be228f69f0 2024-09-21T06:04:12.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.800+0000 7f858affd640 10 client.16701 handle_command_reply: tid=0 2024-09-21T06:04:12.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.800+0000 7f85b81d1740 1 client.16701 shutdown 2024-09-21T06:04:12.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.800+0000 7f85b81d1740 1 -- 172.21.15.117:0/942191508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be22917390 msgr2=0x55be22ceb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:12.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.800+0000 7f85b81d1740 1 --2- 172.21.15.117:0/942191508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be22917390 0x55be22ceb2f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f859c000f80 tx=0x7f859c010f90 comp rx=0 tx=0).stop 2024-09-21T06:04:12.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.800+0000 7f85b81d1740 1 -- 172.21.15.117:0/942191508 shutdown_connections 2024-09-21T06:04:12.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.800+0000 7f85b81d1740 1 --2- 172.21.15.117:0/942191508 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55be228f69f0 0x55be22cfcf30 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.800+0000 7f85b81d1740 1 --2- 172.21.15.117:0/942191508 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be22917390 0x55be22ceb2f0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.800+0000 7f85b81d1740 1 --2- 172.21.15.117:0/942191508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be22938070 0x55be22cf7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.800+0000 7f858affd640 0 client.16701 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:12.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.800+0000 7f85b81d1740 1 --2- 172.21.15.117:0/942191508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be2290f1b0 0x55be22ceadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.800+0000 7f85b81d1740 1 -- 172.21.15.117:0/942191508 >> 172.21.15.117:0/942191508 conn(0x55be2292c460 msgr2=0x55be22ce27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:12.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.800+0000 7f85b81d1740 1 -- 172.21.15.117:0/942191508 shutdown_connections 2024-09-21T06:04:12.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.800+0000 7f85b81d1740 1 -- 172.21.15.117:0/942191508 wait complete. 2024-09-21T06:04:12.816 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.800+0000 7f85b81d1740 20 client.16701 trim_cache size 0 max 16384 2024-09-21T06:04:12.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.818+0000 7f85b81d1740 1 Processor -- start 2024-09-21T06:04:12.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.818+0000 7f85b81d1740 1 -- start start 2024-09-21T06:04:12.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.818+0000 7f85b81d1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be2290f1b0 0x55be22ceb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:12.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.818+0000 7f85b81d1740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be22917390 0x55be22ce2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:12.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.818+0000 7f85b81d1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be22938070 0x55be22cf70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:12.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.818+0000 7f85b81d1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac1af850 con 0x55be22938070 2024-09-21T06:04:12.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.818+0000 7f85b81d1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac1b1390 con 0x55be2290f1b0 2024-09-21T06:04:12.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.818+0000 7f85b81d1740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac1aff20 con 0x55be22917390 2024-09-21T06:04:12.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.818+0000 7f85aaffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be22938070 0x55be22cf70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:12.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.818+0000 7f85aaffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be22938070 0x55be22cf70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38534/0 (socket says 172.21.15.117:38534) 2024-09-21T06:04:12.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.818+0000 7f85aaffd640 1 -- 172.21.15.117:0/1417566160 learned_addr learned my addr 172.21.15.117:0/1417566160 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:12.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.818+0000 7f85aaffd640 1 -- 172.21.15.117:0/1417566160 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be22917390 msgr2=0x55be22ce2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:12.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.818+0000 7f85aaffd640 1 --2- 172.21.15.117:0/1417566160 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be22917390 0x55be22ce2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.818+0000 7f85aaffd640 1 -- 172.21.15.117:0/1417566160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be2290f1b0 msgr2=0x55be22ceb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:12.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.818+0000 7f85aaffd640 1 --2- 172.21.15.117:0/1417566160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be2290f1b0 0x55be22ceb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.818+0000 7f85aaffd640 1 -- 172.21.15.117:0/1417566160 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be22ce24b0 con 0x55be22938070 2024-09-21T06:04:12.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.818+0000 7f85aaffd640 1 --2- 172.21.15.117:0/1417566160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be22938070 0x55be22cf70c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f85ac06c9f0 tx=0x7f85a0032660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6879ed3897cdba5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:12.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.819+0000 7f85917fa640 1 -- 172.21.15.117:0/1417566160 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0035040 con 0x55be22938070 2024-09-21T06:04:12.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.819+0000 7f85917fa640 1 -- 172.21.15.117:0/1417566160 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85a0044070 con 0x55be22938070 2024-09-21T06:04:12.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.819+0000 7f85917fa640 1 -- 172.21.15.117:0/1417566160 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a00046c0 con 0x55be22938070 2024-09-21T06:04:12.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.820+0000 7f85b81d1740 1 -- 172.21.15.117:0/1417566160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be22938070 msgr2=0x55be22cf70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:12.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.820+0000 7f85b81d1740 1 --2- 172.21.15.117:0/1417566160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be22938070 0x55be22cf70c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f85ac06c9f0 tx=0x7f85a0032660 comp rx=0 tx=0).stop 2024-09-21T06:04:12.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.820+0000 7f85b81d1740 1 -- 172.21.15.117:0/1417566160 shutdown_connections 2024-09-21T06:04:12.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.820+0000 7f85b81d1740 1 --2- 172.21.15.117:0/1417566160 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be22938070 0x55be22cf70c0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.820+0000 7f85b81d1740 1 --2- 172.21.15.117:0/1417566160 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be22917390 0x55be22ce2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.820+0000 7f85b81d1740 1 --2- 172.21.15.117:0/1417566160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be2290f1b0 0x55be22ceb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.820+0000 7f85b81d1740 1 -- 172.21.15.117:0/1417566160 >> 172.21.15.117:0/1417566160 conn(0x55be2292c460 msgr2=0x55be22ce2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:12.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.820+0000 7f85b81d1740 1 -- 172.21.15.117:0/1417566160 shutdown_connections 2024-09-21T06:04:12.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.820+0000 7f85b81d1740 1 -- 172.21.15.117:0/1417566160 wait complete. 2024-09-21T06:04:12.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.820+0000 7f85b81d1740 1 -- start start 2024-09-21T06:04:12.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.820+0000 7f85b81d1740 1 Processor -- start 2024-09-21T06:04:12.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.821+0000 7f85b81d1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be2290f1b0 0x55be22d00650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:12.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.821+0000 7f85b81d1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be22917390 0x55be22d60200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:12.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.821+0000 7f85b81d1740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be22938070 0x55be22d60740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:12.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.821+0000 7f85b81d1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac1b0cc0 con 0x55be2290f1b0 2024-09-21T06:04:12.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.821+0000 7f85b81d1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac1b05f0 con 0x55be22917390 2024-09-21T06:04:12.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.821+0000 7f85b81d1740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac1ad850 con 0x55be22938070 2024-09-21T06:04:12.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.821+0000 7f85b81d1740 10 client.0 _ll_register_callbacks cb 0x7f859c007de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:12.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.821+0000 7f85aaffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be2290f1b0 0x55be22d00650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:12.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.821+0000 7f85aaffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be2290f1b0 0x55be22d00650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38550/0 (socket says 172.21.15.117:38550) 2024-09-21T06:04:12.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.821+0000 7f85aaffd640 1 -- 172.21.15.117:0/1818151115 learned_addr learned my addr 172.21.15.117:0/1818151115 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:12.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.821+0000 7f85ab7fe640 1 --2- 172.21.15.117:0/1818151115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be22917390 0x55be22d60200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:12.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.821+0000 7f85aaffd640 1 --2- 172.21.15.117:0/1818151115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be22938070 0x55be22d60740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:12.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.821+0000 7f85aaffd640 1 -- 172.21.15.117:0/1818151115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be22938070 msgr2=0x55be22d60740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:12.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.821+0000 7f85aaffd640 1 --2- 172.21.15.117:0/1818151115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be22938070 0x55be22d60740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.821+0000 7f85aaffd640 1 -- 172.21.15.117:0/1818151115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be22917390 msgr2=0x55be22d60200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:12.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.821+0000 7f85aaffd640 1 --2- 172.21.15.117:0/1818151115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be22917390 0x55be22d60200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.821+0000 7f85aaffd640 1 -- 172.21.15.117:0/1818151115 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be22d60db0 con 0x55be2290f1b0 2024-09-21T06:04:12.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.821+0000 7f85ab7fe640 1 --2- 172.21.15.117:0/1818151115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be22917390 0x55be22d60200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:04:12.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.821+0000 7f85aaffd640 1 --2- 172.21.15.117:0/1818151115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be2290f1b0 0x55be22d00650 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f85a00040f0 tx=0x7f85a0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb3119bc3c866d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:12.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.821+0000 7f858affd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:04:12.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.821+0000 7f858affd640 1 -- 172.21.15.117:0/1818151115 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0059060 con 0x55be2290f1b0 2024-09-21T06:04:12.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.821+0000 7f858affd640 1 -- 172.21.15.117:0/1818151115 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 8485+0+0 (secure 0 0 0) 0x7f85a0005050 con 0x55be2290f1b0 2024-09-21T06:04:12.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.821+0000 7f85b81d1740 1 -- 172.21.15.117:0/1818151115 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85a0044070 con 0x55be2290f1b0 2024-09-21T06:04:12.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.822+0000 7f858affd640 1 -- 172.21.15.117:0/1818151115 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f85a0035040 con 0x55be2290f1b0 2024-09-21T06:04:12.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.822+0000 7f85b81d1740 10 client.16707 fetch_fsmap learned FSMap version 29 2024-09-21T06:04:12.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.822+0000 7f85b81d1740 1 -- 172.21.15.117:0/1818151115 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55be22cfcec0 con 0x55be2290f1b0 2024-09-21T06:04:12.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.822+0000 7f858affd640 1 -- 172.21.15.117:0/1818151115 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85a0044070 con 0x55be2290f1b0 2024-09-21T06:04:12.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.822+0000 7f858affd640 1 -- 172.21.15.117:0/1818151115 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f85a00035f0 con 0x55be2290f1b0 2024-09-21T06:04:12.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.822+0000 7f85b81d1740 10 client.16707 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:04:12.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.822+0000 7f85b81d1740 10 client.16707 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:04:12.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.822+0000 7f85b81d1740 20 client.16707 populate_metadata read hostname 'smithi117' 2024-09-21T06:04:12.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.822+0000 7f85b81d1740 1 --2- 172.21.15.117:0/1818151115 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55be228cbda0 0x55be22d6dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:12.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.822+0000 7f85b81d1740 4 client.16707 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:04:12.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.822+0000 7f85b81d1740 1 -- 172.21.15.117:0/1818151115 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55be22c53850 con 0x55be228cbda0 2024-09-21T06:04:12.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.822+0000 7f85ab7fe640 1 --2- 172.21.15.117:0/1818151115 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55be228cbda0 0x55be22d6dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:12.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.823+0000 7f85ab7fe640 1 --2- 172.21.15.117:0/1818151115 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55be228cbda0 0x55be22d6dff0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd7e61ef72a8ad4b server_cookie=1a26dbbaf1ac9188 in_seq=0 out_seq=0 2024-09-21T06:04:12.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.823+0000 7f858affd640 10 client.16707 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:12.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.823+0000 7f858affd640 1 -- 172.21.15.117:0/1818151115 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1990 (crc 0 0 0) 0x55be22c53850 con 0x55be228cbda0 2024-09-21T06:04:12.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.823+0000 7f858affd640 10 client.16707 handle_command_reply: tid=0 2024-09-21T06:04:12.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.823+0000 7f85b81d1740 1 client.16707 shutdown 2024-09-21T06:04:12.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.823+0000 7f85b81d1740 1 -- 172.21.15.117:0/1818151115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be2290f1b0 msgr2=0x55be22d00650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:12.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.823+0000 7f85b81d1740 1 --2- 172.21.15.117:0/1818151115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be2290f1b0 0x55be22d00650 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f85a00040f0 tx=0x7f85a0005fd0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.823+0000 7f85b81d1740 1 -- 172.21.15.117:0/1818151115 shutdown_connections 2024-09-21T06:04:12.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.823+0000 7f85b81d1740 1 --2- 172.21.15.117:0/1818151115 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55be228cbda0 0x55be22d6dff0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.823+0000 7f85b81d1740 1 --2- 172.21.15.117:0/1818151115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55be22938070 0x55be22d60740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.823+0000 7f85b81d1740 1 --2- 172.21.15.117:0/1818151115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55be22917390 0x55be22d60200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.823+0000 7f858affd640 0 client.16707 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:12.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.823+0000 7f85b81d1740 1 --2- 172.21.15.117:0/1818151115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55be2290f1b0 0x55be22d00650 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.823+0000 7f85b81d1740 1 -- 172.21.15.117:0/1818151115 >> 172.21.15.117:0/1818151115 conn(0x55be2292c460 msgr2=0x55be22ce0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:12.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.823+0000 7f85b81d1740 1 -- 172.21.15.117:0/1818151115 shutdown_connections 2024-09-21T06:04:12.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.823+0000 7f85b81d1740 1 -- 172.21.15.117:0/1818151115 wait complete. 2024-09-21T06:04:12.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.824+0000 7f85b81d1740 20 client.16707 trim_cache size 0 max 16384 2024-09-21T06:04:12.830 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T06:04:12.831 INFO:teuthology.orchestra.run.smithi117.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:04:12.831 INFO:teuthology.orchestra.run.smithi117.stdout: "scrubs": { 2024-09-21T06:04:12.831 INFO:teuthology.orchestra.run.smithi117.stdout: "0902e33a-ce33-45fb-92d8-2e6fcb8ec8a0": { 2024-09-21T06:04:12.831 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:04:12.831 INFO:teuthology.orchestra.run.smithi117.stdout: "tag": "0902e33a-ce33-45fb-92d8-2e6fcb8ec8a0", 2024-09-21T06:04:12.831 INFO:teuthology.orchestra.run.smithi117.stdout: "options": "recursive,force" 2024-09-21T06:04:12.831 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:04:12.831 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:04:12.831 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_stats": { 2024-09-21T06:04:12.831 INFO:teuthology.orchestra.run.smithi117.stdout: "29ff9b7c-49d9-4910-85be-bace7e940cd5": { 2024-09-21T06:04:12.831 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:03:03.775527+0000", 2024-09-21T06:04:12.831 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:04:12.831 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:04:12.832 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:04:12.832 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:04:12.832 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:04:12.832 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:04:12.832 INFO:teuthology.orchestra.run.smithi117.stdout: "0902e33a-ce33-45fb-92d8-2e6fcb8ec8a0": { 2024-09-21T06:04:12.832 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:04:11.365073+0000", 2024-09-21T06:04:12.832 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:04:12.832 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:04:12.832 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:04:12.832 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:04:12.832 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:04:12.832 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:04:12.832 INFO:teuthology.orchestra.run.smithi117.stdout: "d543b0ae-b710-4153-99da-fe5714839c1a": { 2024-09-21T06:04:12.832 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:01:20.574754+0000", 2024-09-21T06:04:12.833 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:04:12.833 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:04:12.833 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:04:12.833 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:04:12.833 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:04:12.833 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:04:12.833 INFO:teuthology.orchestra.run.smithi117.stdout: "ad80d9f1-6aff-46c0-91ab-2c1ab0681039": { 2024-09-21T06:04:12.833 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:59:31.949108+0000", 2024-09-21T06:04:12.833 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:04:12.833 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:04:12.833 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:04:12.833 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:04:12.833 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:04:12.833 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:04:12.833 INFO:teuthology.orchestra.run.smithi117.stdout: "8b3fe95a-b76b-4e94-afd0-72fe35e2dd92": { 2024-09-21T06:04:12.834 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:58:01.246076+0000", 2024-09-21T06:04:12.834 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:04:12.834 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:04:12.834 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:04:12.834 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:04:12.834 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:04:12.834 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:04:12.834 INFO:teuthology.orchestra.run.smithi117.stdout: "b30bfff0-0f6b-4e30-8467-ff7db421eba6": { 2024-09-21T06:04:12.834 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:57:14.939082+0000", 2024-09-21T06:04:12.834 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:04:12.834 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:04:12.834 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:04:12.834 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:04:12.834 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:04:12.835 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:04:12.835 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:04:12.835 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T06:04:12.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.824+0000 7f85b1c5d640 1 -- 172.21.15.117:0/129436945 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f85940b66b0 msgr2=0x7f85940b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:12.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.824+0000 7f85b1c5d640 1 --2- 172.21.15.117:0/129436945 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f85940b66b0 0x7f85940b8bc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f85a40e4460 tx=0x7f85a40a2040 comp rx=0 tx=0).stop 2024-09-21T06:04:12.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.824+0000 7f85b1c5d640 1 -- 172.21.15.117:0/129436945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85ac109df0 msgr2=0x7f85ac19fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:12.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.824+0000 7f85b1c5d640 1 --2- 172.21.15.117:0/129436945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85ac109df0 0x7f85ac19fe40 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f85a4009fd0 tx=0x7f85a400b8e0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.824+0000 7f85b1c5d640 1 -- 172.21.15.117:0/129436945 shutdown_connections 2024-09-21T06:04:12.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.824+0000 7f85b1c5d640 1 --2- 172.21.15.117:0/129436945 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f85940b66b0 0x7f85940b8bc0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.824+0000 7f85b1c5d640 1 --2- 172.21.15.117:0/129436945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85ac109df0 0x7f85ac19fe40 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.824+0000 7f85b1c5d640 1 --2- 172.21.15.117:0/129436945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85ac1094a0 0x7f85ac19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.824+0000 7f85b1c5d640 1 --2- 172.21.15.117:0/129436945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85ac103b60 0x7f85ac19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:12.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.824+0000 7f85b1c5d640 1 -- 172.21.15.117:0/129436945 >> 172.21.15.117:0/129436945 conn(0x7f85ac07b850 msgr2=0x7f85ac107600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:12.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.824+0000 7f85b1c5d640 1 -- 172.21.15.117:0/129436945 shutdown_connections 2024-09-21T06:04:12.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:12.824+0000 7f85b1c5d640 1 -- 172.21.15.117:0/129436945 wait complete. 2024-09-21T06:04:13.549 INFO:tasks.cephfs.filesystem:scrub status for tag:0902e33a-ce33-45fb-92d8-2e6fcb8ec8a0 - {'path': '/', 'tag': '0902e33a-ce33-45fb-92d8-2e6fcb8ec8a0', 'options': 'recursive,force'} 2024-09-21T06:04:14.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:13 smithi192 ceph-mon[34700]: pgmap v536: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 330 MiB/s wr, 1.64k op/s 2024-09-21T06:04:14.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:13 smithi117 ceph-mon[30947]: pgmap v536: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 330 MiB/s wr, 1.64k op/s 2024-09-21T06:04:14.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:13 smithi138 ceph-mon[34391]: pgmap v536: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 330 MiB/s wr, 1.64k op/s 2024-09-21T06:04:16.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:15 smithi192 ceph-mon[34700]: scrub summary: idle 2024-09-21T06:04:16.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:15 smithi192 ceph-mon[34700]: pgmap v537: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 153 MiB/s wr, 1.11k op/s 2024-09-21T06:04:16.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:15 smithi117 ceph-mon[30947]: scrub summary: idle 2024-09-21T06:04:16.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:15 smithi117 ceph-mon[30947]: pgmap v537: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 153 MiB/s wr, 1.11k op/s 2024-09-21T06:04:16.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:15 smithi138 ceph-mon[34391]: scrub summary: idle 2024-09-21T06:04:16.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:15 smithi138 ceph-mon[34391]: pgmap v537: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 153 MiB/s wr, 1.11k op/s 2024-09-21T06:04:18.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:17 smithi192 ceph-mon[34700]: pgmap v538: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 24 MiB/s wr, 505 op/s 2024-09-21T06:04:18.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:17 smithi192 ceph-mon[34700]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T06:04:18.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:17 smithi117 ceph-mon[30947]: pgmap v538: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 24 MiB/s wr, 505 op/s 2024-09-21T06:04:18.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:17 smithi117 ceph-mon[30947]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T06:04:18.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:17 smithi138 ceph-mon[34391]: pgmap v538: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 24 MiB/s wr, 505 op/s 2024-09-21T06:04:18.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:17 smithi138 ceph-mon[34391]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T06:04:20.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:19 smithi192 ceph-mon[34700]: pgmap v540: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 12 MiB/s wr, 345 op/s 2024-09-21T06:04:20.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:19 smithi117 ceph-mon[30947]: pgmap v540: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 12 MiB/s wr, 345 op/s 2024-09-21T06:04:20.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:19 smithi138 ceph-mon[34391]: pgmap v540: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 12 MiB/s wr, 345 op/s 2024-09-21T06:04:21.180 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.175+0000 7efda836e640 1 -- 172.21.15.117:0/999510820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efda010e4e0 msgr2=0x7efda010e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:21.180 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.175+0000 7efda836e640 1 --2- 172.21.15.117:0/999510820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efda010e4e0 0x7efda010e8f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7efd90009900 tx=0x7efd9002a3f0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.180 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.176+0000 7efda836e640 1 -- 172.21.15.117:0/999510820 shutdown_connections 2024-09-21T06:04:21.180 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.176+0000 7efda836e640 1 --2- 172.21.15.117:0/999510820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efda0114760 0x7efda011af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.180 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.176+0000 7efda836e640 1 --2- 172.21.15.117:0/999510820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efda0113e10 0x7efda0114220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.180 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.176+0000 7efda836e640 1 --2- 172.21.15.117:0/999510820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efda010e4e0 0x7efda010e8f0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.180 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.176+0000 7efda836e640 1 -- 172.21.15.117:0/999510820 >> 172.21.15.117:0/999510820 conn(0x7efda010a3c0 msgr2=0x7efda010c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:21.181 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.176+0000 7efda836e640 1 -- 172.21.15.117:0/999510820 shutdown_connections 2024-09-21T06:04:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.177+0000 7efda836e640 1 -- 172.21.15.117:0/999510820 wait complete. 2024-09-21T06:04:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.179+0000 7efda836e640 1 Processor -- start 2024-09-21T06:04:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.179+0000 7efda836e640 1 -- start start 2024-09-21T06:04:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.180+0000 7efda836e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efda010e4e0 0x7efda01a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.180+0000 7efda836e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efda0113e10 0x7efda01a5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.180+0000 7efda836e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efda0114760 0x7efda01aa7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.180+0000 7efda836e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efda011e510 con 0x7efda0113e10 2024-09-21T06:04:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.180+0000 7efda836e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efda011e390 con 0x7efda010e4e0 2024-09-21T06:04:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.180+0000 7efda836e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efda011e690 con 0x7efda0114760 2024-09-21T06:04:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.180+0000 7efda5909640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efda0113e10 0x7efda01a5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.180+0000 7efda5909640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efda0113e10 0x7efda01a5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39420/0 (socket says 172.21.15.117:39420) 2024-09-21T06:04:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.180+0000 7efda5909640 1 -- 172.21.15.117:0/4029174560 learned_addr learned my addr 172.21.15.117:0/4029174560 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.180+0000 7efda610a640 1 --2- 172.21.15.117:0/4029174560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efda010e4e0 0x7efda01a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.180+0000 7efda5909640 1 -- 172.21.15.117:0/4029174560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efda0114760 msgr2=0x7efda01aa7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.181+0000 7efda5909640 1 --2- 172.21.15.117:0/4029174560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efda0114760 0x7efda01aa7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.181+0000 7efda5909640 1 -- 172.21.15.117:0/4029174560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efda010e4e0 msgr2=0x7efda01a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.181+0000 7efda5909640 1 --2- 172.21.15.117:0/4029174560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efda010e4e0 0x7efda01a4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.181+0000 7efda5909640 1 -- 172.21.15.117:0/4029174560 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda01aaee0 con 0x7efda0113e10 2024-09-21T06:04:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.181+0000 7efda610a640 1 --2- 172.21.15.117:0/4029174560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efda010e4e0 0x7efda01a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.181+0000 7efda5909640 1 --2- 172.21.15.117:0/4029174560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efda0113e10 0x7efda01a5550 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00b450 tx=0x7efd9c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.181+0000 7efd977fe640 1 -- 172.21.15.117:0/4029174560 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c015020 con 0x7efda0113e10 2024-09-21T06:04:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.181+0000 7efda836e640 1 -- 172.21.15.117:0/4029174560 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda01ab1d0 con 0x7efda0113e10 2024-09-21T06:04:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.181+0000 7efd977fe640 1 -- 172.21.15.117:0/4029174560 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd9c011d40 con 0x7efda0113e10 2024-09-21T06:04:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.181+0000 7efd977fe640 1 -- 172.21.15.117:0/4029174560 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c00f340 con 0x7efda0113e10 2024-09-21T06:04:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.182+0000 7efda836e640 1 -- 172.21.15.117:0/4029174560 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efda01b1960 con 0x7efda0113e10 2024-09-21T06:04:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.185+0000 7efd977fe640 1 -- 172.21.15.117:0/4029174560 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7efd9c018020 con 0x7efda0113e10 2024-09-21T06:04:21.188 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.186+0000 7efdaf3bb740 1 Processor -- start 2024-09-21T06:04:21.189 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.187+0000 7efdaf3bb740 1 -- start start 2024-09-21T06:04:21.189 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.187+0000 7efdaf3bb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556fd9d161b0 0x556fda0bb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:21.189 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.187+0000 7efdaf3bb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fd9ced2f0 0x556fda0bbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.187+0000 7efdaf3bb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556fd9cf54d0 0x556fda0bc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.187+0000 7efdaf3bb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556fda0306f0 con 0x556fd9d161b0 2024-09-21T06:04:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.187+0000 7efdaf3bb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556fda01fb20 con 0x556fd9ced2f0 2024-09-21T06:04:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.187+0000 7efdaf3bb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556fda023930 con 0x556fd9cf54d0 2024-09-21T06:04:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.187+0000 7efd977fe640 1 --2- 172.21.15.117:0/4029174560 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7efd8c0b6730 0x7efd8c0b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.187+0000 7efda690b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556fd9d161b0 0x556fda0bb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.187+0000 7efda690b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556fd9d161b0 0x556fda0bb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39426/0 (socket says 172.21.15.117:39426) 2024-09-21T06:04:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.187+0000 7efda690b640 1 -- 172.21.15.117:0/114484189 learned_addr learned my addr 172.21.15.117:0/114484189 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.188+0000 7efd977fe640 1 -- 172.21.15.117:0/4029174560 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7efd9c0e71c0 con 0x7efda0113e10 2024-09-21T06:04:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.188+0000 7efda690b640 1 --2- 172.21.15.117:0/4029174560 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7efd8c0b6730 0x7efd8c0b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.188+0000 7efda690b640 1 -- 172.21.15.117:0/114484189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556fd9cf54d0 msgr2=0x556fda0bc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.188+0000 7efda690b640 1 --2- 172.21.15.117:0/114484189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556fd9cf54d0 0x556fda0bc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.188+0000 7efda690b640 1 -- 172.21.15.117:0/114484189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fd9ced2f0 msgr2=0x556fda0bbef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:04:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.188+0000 7efda690b640 1 --2- 172.21.15.117:0/114484189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fd9ced2f0 0x556fda0bbef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.188+0000 7efda690b640 1 -- 172.21.15.117:0/114484189 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fd9fe5530 con 0x556fd9d161b0 2024-09-21T06:04:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.189+0000 7efda690b640 1 --2- 172.21.15.117:0/114484189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556fd9d161b0 0x556fda0bb9b0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7efd980122e0 tx=0x7efd980127a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4dacdc19ea92f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:21.193 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.189+0000 7efd94bfe640 1 -- 172.21.15.117:0/114484189 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd98017e60 con 0x556fd9d161b0 2024-09-21T06:04:21.193 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.189+0000 7efd94bfe640 1 -- 172.21.15.117:0/114484189 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd9800bcc0 con 0x556fd9d161b0 2024-09-21T06:04:21.193 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.189+0000 7efd94bfe640 1 -- 172.21.15.117:0/114484189 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd980208b0 con 0x556fd9d161b0 2024-09-21T06:04:21.195 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.191+0000 7efdaf3bb740 1 -- 172.21.15.117:0/114484189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556fd9d161b0 msgr2=0x556fda0bb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:21.195 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.191+0000 7efdaf3bb740 1 --2- 172.21.15.117:0/114484189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556fd9d161b0 0x556fda0bb9b0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7efd980122e0 tx=0x7efd980127a0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.195 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.191+0000 7efda690b640 1 --2- 172.21.15.117:0/4029174560 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7efd8c0b6730 0x7efd8c0b8c40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efd9801d000 tx=0x7efd98018000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:21.195 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.191+0000 7efdaf3bb740 1 -- 172.21.15.117:0/114484189 shutdown_connections 2024-09-21T06:04:21.195 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.191+0000 7efdaf3bb740 1 --2- 172.21.15.117:0/114484189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556fd9cf54d0 0x556fda0bc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.195 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.191+0000 7efdaf3bb740 1 --2- 172.21.15.117:0/114484189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fd9ced2f0 0x556fda0bbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.195 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.191+0000 7efdaf3bb740 1 --2- 172.21.15.117:0/114484189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556fd9d161b0 0x556fda0bb9b0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.195 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.191+0000 7efdaf3bb740 1 -- 172.21.15.117:0/114484189 >> 172.21.15.117:0/114484189 conn(0x556fd9d0a300 msgr2=0x556fda0b98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:21.196 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.192+0000 7efdaf3bb740 1 -- 172.21.15.117:0/114484189 shutdown_connections 2024-09-21T06:04:21.196 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.192+0000 7efdaf3bb740 1 -- 172.21.15.117:0/114484189 wait complete. 2024-09-21T06:04:21.196 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.193+0000 7efdaf3bb740 1 -- start start 2024-09-21T06:04:21.198 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.193+0000 7efdaf3bb740 1 Processor -- start 2024-09-21T06:04:21.198 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.194+0000 7efdaf3bb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556fd9ced2f0 0x556fda0cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:21.198 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.194+0000 7efdaf3bb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fd9cf54d0 0x556fda0cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:21.198 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.194+0000 7efdaf3bb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556fd9d161b0 0x556fda0dbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:21.198 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.194+0000 7efdaf3bb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556fd9f57ab0 con 0x556fd9ced2f0 2024-09-21T06:04:21.198 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.194+0000 7efdaf3bb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556fd9fc5730 con 0x556fd9cf54d0 2024-09-21T06:04:21.198 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.194+0000 7efdaf3bb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556fda0d28f0 con 0x556fd9d161b0 2024-09-21T06:04:21.198 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.194+0000 7efdaf3bb740 10 client.0 _ll_register_callbacks cb 0x556fda0196b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.195+0000 7efda690b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556fd9d161b0 0x556fda0dbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.195+0000 7efda690b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556fd9d161b0 0x556fda0dbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:42460/0 (socket says 172.21.15.117:42460) 2024-09-21T06:04:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.195+0000 7efda690b640 1 -- 172.21.15.117:0/3683222617 learned_addr learned my addr 172.21.15.117:0/3683222617 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.195+0000 7efda610a640 1 --2- 172.21.15.117:0/3683222617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556fd9ced2f0 0x556fda0cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:21.199 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.195+0000 7efda5909640 1 --2- 172.21.15.117:0/3683222617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fd9cf54d0 0x556fda0cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.196+0000 7efda610a640 1 -- 172.21.15.117:0/3683222617 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556fd9d161b0 msgr2=0x556fda0dbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.196+0000 7efda610a640 1 --2- 172.21.15.117:0/3683222617 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556fd9d161b0 0x556fda0dbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.196+0000 7efda610a640 1 -- 172.21.15.117:0/3683222617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fd9cf54d0 msgr2=0x556fda0cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.196+0000 7efda610a640 1 --2- 172.21.15.117:0/3683222617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fd9cf54d0 0x556fda0cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.196+0000 7efda610a640 1 -- 172.21.15.117:0/3683222617 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fd9d81790 con 0x556fd9ced2f0 2024-09-21T06:04:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.197+0000 7efda610a640 1 --2- 172.21.15.117:0/3683222617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556fd9ced2f0 0x556fda0cf880 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7efd90005f70 tx=0x7efd90004510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba67b718f4b10e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.197+0000 7efd89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:04:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.198+0000 7efd89ffb640 1 -- 172.21.15.117:0/3683222617 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9003f070 con 0x556fd9ced2f0 2024-09-21T06:04:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.198+0000 7efd89ffb640 1 -- 172.21.15.117:0/3683222617 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7efd90005450 con 0x556fd9ced2f0 2024-09-21T06:04:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.199+0000 7efdaf3bb740 1 -- 172.21.15.117:0/3683222617 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd9800bcc0 con 0x556fd9ced2f0 2024-09-21T06:04:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.199+0000 7efd89ffb640 1 -- 172.21.15.117:0/3683222617 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efd90049050 con 0x556fd9ced2f0 2024-09-21T06:04:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.200+0000 7efdaf3bb740 10 client.16719 fetch_fsmap learned FSMap version 29 2024-09-21T06:04:21.204 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.200+0000 7efdaf3bb740 1 -- 172.21.15.117:0/3683222617 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556fd9dfa620 con 0x556fd9ced2f0 2024-09-21T06:04:21.205 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.202+0000 7efd89ffb640 1 -- 172.21.15.117:0/3683222617 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd9800bcc0 con 0x556fd9ced2f0 2024-09-21T06:04:21.205 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.202+0000 7efd89ffb640 1 -- 172.21.15.117:0/3683222617 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7efd90003990 con 0x556fd9ced2f0 2024-09-21T06:04:21.205 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.203+0000 7efdaf3bb740 10 client.16719 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:04:21.205 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.203+0000 7efdaf3bb740 10 client.16719 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:04:21.205 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.203+0000 7efdaf3bb740 20 client.16719 populate_metadata read hostname 'smithi117' 2024-09-21T06:04:21.205 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.203+0000 7efdaf3bb740 1 --2- 172.21.15.117:0/3683222617 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x556fd9cd4d80 0x556fda0e1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:21.205 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.203+0000 7efdaf3bb740 4 client.16719 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:04:21.205 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.203+0000 7efdaf3bb740 1 -- 172.21.15.117:0/3683222617 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fda032640 con 0x556fd9cd4d80 2024-09-21T06:04:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.205+0000 7efda5909640 1 --2- 172.21.15.117:0/3683222617 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x556fd9cd4d80 0x556fda0e1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.208+0000 7efd89ffb640 10 client.16719 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.208+0000 7efda5909640 1 --2- 172.21.15.117:0/3683222617 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x556fd9cd4d80 0x556fda0e1950 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9433da1c5f9c054d server_cookie=7d5d679789044121 in_seq=0 out_seq=0 2024-09-21T06:04:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.211+0000 7efd89ffb640 1 -- 172.21.15.117:0/3683222617 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556fda032640 con 0x556fd9cd4d80 2024-09-21T06:04:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.211+0000 7efd89ffb640 10 client.16719 handle_command_reply: tid=0 2024-09-21T06:04:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.211+0000 7efdaf3bb740 1 client.16719 shutdown 2024-09-21T06:04:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.211+0000 7efdaf3bb740 1 -- 172.21.15.117:0/3683222617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556fd9ced2f0 msgr2=0x556fda0cf880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.211+0000 7efdaf3bb740 1 --2- 172.21.15.117:0/3683222617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556fd9ced2f0 0x556fda0cf880 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7efd90005f70 tx=0x7efd90004510 comp rx=0 tx=0).stop 2024-09-21T06:04:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.211+0000 7efdaf3bb740 1 -- 172.21.15.117:0/3683222617 shutdown_connections 2024-09-21T06:04:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.211+0000 7efdaf3bb740 1 --2- 172.21.15.117:0/3683222617 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x556fd9cd4d80 0x556fda0e1950 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.211+0000 7efdaf3bb740 1 --2- 172.21.15.117:0/3683222617 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556fd9d161b0 0x556fda0dbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.211+0000 7efdaf3bb740 1 --2- 172.21.15.117:0/3683222617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fd9cf54d0 0x556fda0cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.211+0000 7efdaf3bb740 1 --2- 172.21.15.117:0/3683222617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556fd9ced2f0 0x556fda0cf880 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.211+0000 7efdaf3bb740 1 -- 172.21.15.117:0/3683222617 >> 172.21.15.117:0/3683222617 conn(0x556fd9d0a300 msgr2=0x556fda0b7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.211+0000 7efd89ffb640 0 client.16719 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.211+0000 7efdaf3bb740 1 -- 172.21.15.117:0/3683222617 shutdown_connections 2024-09-21T06:04:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.212+0000 7efdaf3bb740 1 -- 172.21.15.117:0/3683222617 wait complete. 2024-09-21T06:04:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.212+0000 7efdaf3bb740 20 client.16719 trim_cache size 0 max 16384 2024-09-21T06:04:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.233+0000 7efdaf3bb740 1 Processor -- start 2024-09-21T06:04:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.234+0000 7efdaf3bb740 1 -- start start 2024-09-21T06:04:21.238 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.234+0000 7efdaf3bb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fd9ced2f0 0x556fda0cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:21.238 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.234+0000 7efdaf3bb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556fd9cf54d0 0x556fda0b99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:21.238 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.234+0000 7efdaf3bb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556fd9d161b0 0x556fda0dbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:21.238 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.234+0000 7efdaf3bb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efda01b5f00 con 0x556fd9cf54d0 2024-09-21T06:04:21.238 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.234+0000 7efdaf3bb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efda01b7a40 con 0x556fd9ced2f0 2024-09-21T06:04:21.238 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.234+0000 7efdaf3bb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efda01b65d0 con 0x556fd9d161b0 2024-09-21T06:04:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.234+0000 7efda5909640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556fd9cf54d0 0x556fda0b99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.234+0000 7efda5909640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556fd9cf54d0 0x556fda0b99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39440/0 (socket says 172.21.15.117:39440) 2024-09-21T06:04:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.234+0000 7efda5909640 1 -- 172.21.15.117:0/948332359 learned_addr learned my addr 172.21.15.117:0/948332359 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.235+0000 7efda5909640 1 -- 172.21.15.117:0/948332359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556fd9d161b0 msgr2=0x556fda0dbb50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:04:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.235+0000 7efda5909640 1 --2- 172.21.15.117:0/948332359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556fd9d161b0 0x556fda0dbb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.235+0000 7efda5909640 1 -- 172.21.15.117:0/948332359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fd9ced2f0 msgr2=0x556fda0cfd80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:04:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.235+0000 7efda5909640 1 --2- 172.21.15.117:0/948332359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fd9ced2f0 0x556fda0cfd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.235+0000 7efda5909640 1 -- 172.21.15.117:0/948332359 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fda0e1eb0 con 0x556fd9cf54d0 2024-09-21T06:04:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.235+0000 7efda5909640 1 --2- 172.21.15.117:0/948332359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556fd9cf54d0 0x556fda0b99a0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0ab160 tx=0x7efd9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad74f06ffad98f7d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.235+0000 7efd8bfff640 1 -- 172.21.15.117:0/948332359 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c0aa160 con 0x556fd9cf54d0 2024-09-21T06:04:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.235+0000 7efd8bfff640 1 -- 172.21.15.117:0/948332359 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd9c0950b0 con 0x556fd9cf54d0 2024-09-21T06:04:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.235+0000 7efd8bfff640 1 -- 172.21.15.117:0/948332359 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c0acbc0 con 0x556fd9cf54d0 2024-09-21T06:04:21.241 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.237+0000 7efdaf3bb740 1 -- 172.21.15.117:0/948332359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556fd9cf54d0 msgr2=0x556fda0b99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:21.241 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.237+0000 7efdaf3bb740 1 --2- 172.21.15.117:0/948332359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556fd9cf54d0 0x556fda0b99a0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0ab160 tx=0x7efd9c0046c0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.241 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.237+0000 7efdaf3bb740 1 -- 172.21.15.117:0/948332359 shutdown_connections 2024-09-21T06:04:21.241 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.237+0000 7efdaf3bb740 1 --2- 172.21.15.117:0/948332359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556fd9cf54d0 0x556fda0b99a0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.241 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.237+0000 7efdaf3bb740 1 --2- 172.21.15.117:0/948332359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556fd9d161b0 0x556fda0dbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.241 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.237+0000 7efdaf3bb740 1 --2- 172.21.15.117:0/948332359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fd9ced2f0 0x556fda0cfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.241 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.237+0000 7efdaf3bb740 1 -- 172.21.15.117:0/948332359 >> 172.21.15.117:0/948332359 conn(0x556fd9d0a300 msgr2=0x556fda0b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:21.242 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.237+0000 7efdaf3bb740 1 -- 172.21.15.117:0/948332359 shutdown_connections 2024-09-21T06:04:21.242 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.237+0000 7efdaf3bb740 1 -- 172.21.15.117:0/948332359 wait complete. 2024-09-21T06:04:21.242 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.238+0000 7efdaf3bb740 1 -- start start 2024-09-21T06:04:21.242 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.238+0000 7efdaf3bb740 1 Processor -- start 2024-09-21T06:04:21.242 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.238+0000 7efdaf3bb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fd9ced2f0 0x556fda0d3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:21.243 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.238+0000 7efdaf3bb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556fd9cf54d0 0x556fda0d3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:21.243 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.238+0000 7efdaf3bb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556fd9d161b0 0x556fda0d79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:21.243 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.238+0000 7efdaf3bb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efda01b7370 con 0x556fd9d161b0 2024-09-21T06:04:21.243 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.238+0000 7efdaf3bb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efda01b6ca0 con 0x556fd9ced2f0 2024-09-21T06:04:21.243 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.238+0000 7efdaf3bb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efda01b3f00 con 0x556fd9cf54d0 2024-09-21T06:04:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.238+0000 7efdaf3bb740 10 client.0 _ll_register_callbacks cb 0x7efd90009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.238+0000 7efda690b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556fd9d161b0 0x556fda0d79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.238+0000 7efda690b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556fd9d161b0 0x556fda0d79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39452/0 (socket says 172.21.15.117:39452) 2024-09-21T06:04:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.238+0000 7efda690b640 1 -- 172.21.15.117:0/965362046 learned_addr learned my addr 172.21.15.117:0/965362046 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.239+0000 7efda690b640 1 -- 172.21.15.117:0/965362046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556fd9cf54d0 msgr2=0x556fda0d3ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:04:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.239+0000 7efda690b640 1 --2- 172.21.15.117:0/965362046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556fd9cf54d0 0x556fda0d3ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.239+0000 7efda690b640 1 -- 172.21.15.117:0/965362046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fd9ced2f0 msgr2=0x556fda0d3510 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:04:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.239+0000 7efda690b640 1 --2- 172.21.15.117:0/965362046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fd9ced2f0 0x556fda0d3510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.239+0000 7efda690b640 1 -- 172.21.15.117:0/965362046 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fda0d8020 con 0x556fd9d161b0 2024-09-21T06:04:21.245 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.239+0000 7efda690b640 1 --2- 172.21.15.117:0/965362046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556fd9d161b0 0x556fda0d79b0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7efd980122b0 tx=0x7efd98017e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a764e46107e4a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.239+0000 7efd89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:04:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.239+0000 7efd89ffb640 1 -- 172.21.15.117:0/965362046 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9800bcc0 con 0x556fd9d161b0 2024-09-21T06:04:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.239+0000 7efd89ffb640 1 -- 172.21.15.117:0/965362046 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7efd98020420 con 0x556fd9d161b0 2024-09-21T06:04:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.239+0000 7efdaf3bb740 1 -- 172.21.15.117:0/965362046 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd9c0950b0 con 0x556fd9d161b0 2024-09-21T06:04:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.240+0000 7efd89ffb640 1 -- 172.21.15.117:0/965362046 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efd98020640 con 0x556fd9d161b0 2024-09-21T06:04:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.240+0000 7efdaf3bb740 10 client.16725 fetch_fsmap learned FSMap version 29 2024-09-21T06:04:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.240+0000 7efdaf3bb740 1 -- 172.21.15.117:0/965362046 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556fda14bd40 con 0x556fd9d161b0 2024-09-21T06:04:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.240+0000 7efd89ffb640 1 -- 172.21.15.117:0/965362046 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd9c0950b0 con 0x556fd9d161b0 2024-09-21T06:04:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.240+0000 7efd89ffb640 1 -- 172.21.15.117:0/965362046 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7efd98032030 con 0x556fd9d161b0 2024-09-21T06:04:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.240+0000 7efdaf3bb740 10 client.16725 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:04:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.240+0000 7efdaf3bb740 10 client.16725 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:04:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.240+0000 7efdaf3bb740 20 client.16725 populate_metadata read hostname 'smithi117' 2024-09-21T06:04:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.240+0000 7efdaf3bb740 1 --2- 172.21.15.117:0/965362046 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x556fd9caa1c0 0x556fda14e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:21.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.240+0000 7efdaf3bb740 4 client.16725 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:04:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.240+0000 7efdaf3bb740 1 -- 172.21.15.117:0/965362046 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556fda02e0f0 con 0x556fd9caa1c0 2024-09-21T06:04:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.241+0000 7efda610a640 1 --2- 172.21.15.117:0/965362046 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x556fd9caa1c0 0x556fda14e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.243+0000 7efda610a640 1 --2- 172.21.15.117:0/965362046 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x556fd9caa1c0 0x556fda14e400 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25acff726abc677 server_cookie=6dc202b7817a685f in_seq=0 out_seq=0 2024-09-21T06:04:21.247 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.243+0000 7efd89ffb640 10 client.16725 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.380+0000 7efd89ffb640 1 -- 172.21.15.117:0/965362046 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556fda02e0f0 con 0x556fd9caa1c0 2024-09-21T06:04:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.380+0000 7efd89ffb640 10 client.16725 handle_command_reply: tid=0 2024-09-21T06:04:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.380+0000 7efdaf3bb740 1 client.16725 shutdown 2024-09-21T06:04:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.380+0000 7efdaf3bb740 1 -- 172.21.15.117:0/965362046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556fd9d161b0 msgr2=0x556fda0d79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.380+0000 7efdaf3bb740 1 --2- 172.21.15.117:0/965362046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556fd9d161b0 0x556fda0d79b0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7efd980122b0 tx=0x7efd98017e60 comp rx=0 tx=0).stop 2024-09-21T06:04:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.380+0000 7efdaf3bb740 1 -- 172.21.15.117:0/965362046 shutdown_connections 2024-09-21T06:04:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.380+0000 7efdaf3bb740 1 --2- 172.21.15.117:0/965362046 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x556fd9caa1c0 0x556fda14e400 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.380+0000 7efdaf3bb740 1 --2- 172.21.15.117:0/965362046 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556fd9d161b0 0x556fda0d79b0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.380+0000 7efd89ffb640 0 client.16725 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.380+0000 7efdaf3bb740 1 --2- 172.21.15.117:0/965362046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556fd9cf54d0 0x556fda0d3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.380+0000 7efdaf3bb740 1 --2- 172.21.15.117:0/965362046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556fd9ced2f0 0x556fda0d3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.381+0000 7efdaf3bb740 1 -- 172.21.15.117:0/965362046 >> 172.21.15.117:0/965362046 conn(0x556fd9d0a300 msgr2=0x556fda0b7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.381+0000 7efdaf3bb740 1 -- 172.21.15.117:0/965362046 shutdown_connections 2024-09-21T06:04:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.381+0000 7efdaf3bb740 1 -- 172.21.15.117:0/965362046 wait complete. 2024-09-21T06:04:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.381+0000 7efdaf3bb740 20 client.16725 trim_cache size 0 max 16384 2024-09-21T06:04:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T06:04:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T06:04:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T06:04:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T06:04:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.382+0000 7efda836e640 1 -- 172.21.15.117:0/4029174560 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7efd8c0b6730 msgr2=0x7efd8c0b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.382+0000 7efda836e640 1 --2- 172.21.15.117:0/4029174560 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7efd8c0b6730 0x7efd8c0b8c40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efd9801d000 tx=0x7efd98018000 comp rx=0 tx=0).stop 2024-09-21T06:04:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.382+0000 7efda836e640 1 -- 172.21.15.117:0/4029174560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efda0113e10 msgr2=0x7efda01a5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.382+0000 7efda836e640 1 --2- 172.21.15.117:0/4029174560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efda0113e10 0x7efda01a5550 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00b450 tx=0x7efd9c00b910 comp rx=0 tx=0).stop 2024-09-21T06:04:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.382+0000 7efda836e640 1 -- 172.21.15.117:0/4029174560 shutdown_connections 2024-09-21T06:04:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.382+0000 7efda836e640 1 --2- 172.21.15.117:0/4029174560 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7efd8c0b6730 0x7efd8c0b8c40 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.382+0000 7efda836e640 1 --2- 172.21.15.117:0/4029174560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efda0113e10 0x7efda01a5550 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.382+0000 7efda836e640 1 --2- 172.21.15.117:0/4029174560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efda0114760 0x7efda01aa7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.382+0000 7efda836e640 1 --2- 172.21.15.117:0/4029174560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efda010e4e0 0x7efda01a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.382+0000 7efda836e640 1 -- 172.21.15.117:0/4029174560 >> 172.21.15.117:0/4029174560 conn(0x7efda010a3c0 msgr2=0x7efda010cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.382+0000 7efda836e640 1 -- 172.21.15.117:0/4029174560 shutdown_connections 2024-09-21T06:04:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:21.382+0000 7efda836e640 1 -- 172.21.15.117:0/4029174560 wait complete. 2024-09-21T06:04:22.231 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:21 smithi117 ceph-mon[30947]: pgmap v541: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 16 MiB/s wr, 243 op/s 2024-09-21T06:04:22.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:21 smithi138 ceph-mon[34391]: pgmap v541: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 16 MiB/s wr, 243 op/s 2024-09-21T06:04:22.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:21 smithi192 ceph-mon[34700]: pgmap v541: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 16 MiB/s wr, 243 op/s 2024-09-21T06:04:23.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:23 smithi138 ceph-mon[34391]: pgmap v542: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 50 MiB/s wr, 367 op/s 2024-09-21T06:04:23.479 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:23 smithi192 ceph-mon[34700]: pgmap v542: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 50 MiB/s wr, 367 op/s 2024-09-21T06:04:23.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:23 smithi117 ceph-mon[30947]: pgmap v542: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 50 MiB/s wr, 367 op/s 2024-09-21T06:04:26.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:25 smithi138 ceph-mon[34391]: pgmap v543: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 185 MiB/s wr, 1.08k op/s 2024-09-21T06:04:26.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:25 smithi192 ceph-mon[34700]: pgmap v543: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 185 MiB/s wr, 1.08k op/s 2024-09-21T06:04:26.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:25 smithi117 ceph-mon[30947]: pgmap v543: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 185 MiB/s wr, 1.08k op/s 2024-09-21T06:04:28.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:27 smithi192 ceph-mon[34700]: pgmap v544: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 226 MiB/s wr, 1.33k op/s 2024-09-21T06:04:28.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:27 smithi117 ceph-mon[30947]: pgmap v544: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 226 MiB/s wr, 1.33k op/s 2024-09-21T06:04:28.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:27 smithi138 ceph-mon[34391]: pgmap v544: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 226 MiB/s wr, 1.33k op/s 2024-09-21T06:04:30.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:29 smithi192 ceph-mon[34700]: pgmap v545: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 199 MiB/s wr, 1.18k op/s 2024-09-21T06:04:30.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:29 smithi117 ceph-mon[30947]: pgmap v545: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 199 MiB/s wr, 1.18k op/s 2024-09-21T06:04:30.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:29 smithi138 ceph-mon[34391]: pgmap v545: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 199 MiB/s wr, 1.18k op/s 2024-09-21T06:04:32.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:31 smithi192 ceph-mon[34700]: pgmap v546: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 188 MiB/s wr, 1.11k op/s 2024-09-21T06:04:32.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:31 smithi117 ceph-mon[30947]: pgmap v546: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 188 MiB/s wr, 1.11k op/s 2024-09-21T06:04:32.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:31 smithi138 ceph-mon[34391]: pgmap v546: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 188 MiB/s wr, 1.11k op/s 2024-09-21T06:04:34.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:33 smithi192 ceph-mon[34700]: pgmap v547: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 179 MiB/s wr, 1.05k op/s 2024-09-21T06:04:34.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:33 smithi117 ceph-mon[30947]: pgmap v547: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 179 MiB/s wr, 1.05k op/s 2024-09-21T06:04:34.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:33 smithi138 ceph-mon[34391]: pgmap v547: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 179 MiB/s wr, 1.05k op/s 2024-09-21T06:04:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.504+0000 7f195e5ba640 1 -- 172.21.15.117:0/348590029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19581026b0 msgr2=0x7f19581104e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.504+0000 7f195e5ba640 1 --2- 172.21.15.117:0/348590029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19581026b0 0x7f19581104e0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f1950009ca0 tx=0x7f195002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.504+0000 7f195e5ba640 1 -- 172.21.15.117:0/348590029 shutdown_connections 2024-09-21T06:04:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.504+0000 7f195e5ba640 1 --2- 172.21.15.117:0/348590029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f19581026b0 0x7f19581104e0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.504+0000 7f195e5ba640 1 --2- 172.21.15.117:0/348590029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1958101d60 0x7f1958102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.504+0000 7f195e5ba640 1 --2- 172.21.15.117:0/348590029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1958108680 0x7f1958108a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.504+0000 7f195e5ba640 1 -- 172.21.15.117:0/348590029 >> 172.21.15.117:0/348590029 conn(0x7f1958077ad0 msgr2=0x7f1958077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.504+0000 7f195e5ba640 1 -- 172.21.15.117:0/348590029 shutdown_connections 2024-09-21T06:04:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.504+0000 7f195e5ba640 1 -- 172.21.15.117:0/348590029 wait complete. 2024-09-21T06:04:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.504+0000 7f195e5ba640 1 Processor -- start 2024-09-21T06:04:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.505+0000 7f195e5ba640 1 -- start start 2024-09-21T06:04:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.505+0000 7f195e5ba640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1958101d60 0x7f1958102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.505+0000 7f195e5ba640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19581026b0 0x7f19581034d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.505+0000 7f195e5ba640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1958108680 0x7f1958103a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.505+0000 7f195e5ba640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1958113a20 con 0x7f1958101d60 2024-09-21T06:04:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.505+0000 7f195e5ba640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f19581138a0 con 0x7f1958108680 2024-09-21T06:04:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.505+0000 7f195e5ba640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1958113ba0 con 0x7f19581026b0 2024-09-21T06:04:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.505+0000 7f1957fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1958101d60 0x7f1958102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.505+0000 7f1957fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1958101d60 0x7f1958102f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35858/0 (socket says 172.21.15.117:35858) 2024-09-21T06:04:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.505+0000 7f1957fff640 1 -- 172.21.15.117:0/3431778663 learned_addr learned my addr 172.21.15.117:0/3431778663 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.505+0000 7f195cb57640 1 --2- 172.21.15.117:0/3431778663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1958108680 0x7f1958103a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.505+0000 7f19577fe640 1 --2- 172.21.15.117:0/3431778663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19581026b0 0x7f19581034d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.505+0000 7f1957fff640 1 -- 172.21.15.117:0/3431778663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19581026b0 msgr2=0x7f19581034d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.505+0000 7f1957fff640 1 --2- 172.21.15.117:0/3431778663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19581026b0 0x7f19581034d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.505+0000 7f1957fff640 1 -- 172.21.15.117:0/3431778663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1958108680 msgr2=0x7f1958103a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.505+0000 7f1957fff640 1 --2- 172.21.15.117:0/3431778663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1958108680 0x7f1958103a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.505+0000 7f1957fff640 1 -- 172.21.15.117:0/3431778663 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19581a86b0 con 0x7f1958101d60 2024-09-21T06:04:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.505+0000 7f195cb57640 1 --2- 172.21.15.117:0/3431778663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1958108680 0x7f1958103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:04:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.505+0000 7f19577fe640 1 --2- 172.21.15.117:0/3431778663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19581026b0 0x7f19581034d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.506+0000 7f1957fff640 1 --2- 172.21.15.117:0/3431778663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1958101d60 0x7f1958102f90 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f194800e9f0 tx=0x7f194800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:34.511 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.506+0000 7f19557fa640 1 -- 172.21.15.117:0/3431778663 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194800cd80 con 0x7f1958101d60 2024-09-21T06:04:34.511 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.506+0000 7f195e5ba640 1 -- 172.21.15.117:0/3431778663 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19581a89a0 con 0x7f1958101d60 2024-09-21T06:04:34.511 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.506+0000 7f19557fa640 1 -- 172.21.15.117:0/3431778663 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19480108e0 con 0x7f1958101d60 2024-09-21T06:04:34.511 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.506+0000 7f19557fa640 1 -- 172.21.15.117:0/3431778663 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1948018980 con 0x7f1958101d60 2024-09-21T06:04:34.511 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.506+0000 7f195e5ba640 1 -- 172.21.15.117:0/3431778663 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19581a8ef0 con 0x7f1958101d60 2024-09-21T06:04:34.517 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.509+0000 7f1965603740 1 Processor -- start 2024-09-21T06:04:34.517 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.509+0000 7f1965603740 1 -- start start 2024-09-21T06:04:34.517 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.509+0000 7f1965603740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5bcf931b0 0x55d5bd3389b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:34.517 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.509+0000 7f1965603740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5bcf6a2f0 0x55d5bd338ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:34.517 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.509+0000 7f1965603740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d5bcf724d0 0x55d5bd339430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:34.517 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.509+0000 7f1965603740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d5bd2ad6f0 con 0x55d5bcf931b0 2024-09-21T06:04:34.517 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.509+0000 7f1965603740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d5bd29cb20 con 0x55d5bcf6a2f0 2024-09-21T06:04:34.517 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.509+0000 7f1965603740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55d5bd2a0930 con 0x55d5bcf724d0 2024-09-21T06:04:34.517 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.509+0000 7f195cb57640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5bcf931b0 0x55d5bd3389b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:34.517 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.509+0000 7f19557fa640 1 -- 172.21.15.117:0/3431778663 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f1948010430 con 0x7f1958101d60 2024-09-21T06:04:34.517 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.509+0000 7f195cb57640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5bcf931b0 0x55d5bd3389b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35860/0 (socket says 172.21.15.117:35860) 2024-09-21T06:04:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.509+0000 7f195cb57640 1 -- 172.21.15.117:0/723871833 learned_addr learned my addr 172.21.15.117:0/723871833 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.509+0000 7f19577fe640 1 --2- 172.21.15.117:0/723871833 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d5bcf724d0 0x55d5bd339430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.509+0000 7f1957fff640 1 --2- 172.21.15.117:0/723871833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5bcf6a2f0 0x55d5bd338ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.509+0000 7f195cb57640 1 -- 172.21.15.117:0/723871833 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d5bcf724d0 msgr2=0x55d5bd339430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.509+0000 7f195cb57640 1 --2- 172.21.15.117:0/723871833 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d5bcf724d0 0x55d5bd339430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.509+0000 7f195cb57640 1 -- 172.21.15.117:0/723871833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5bcf6a2f0 msgr2=0x55d5bd338ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.509+0000 7f195cb57640 1 --2- 172.21.15.117:0/723871833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5bcf6a2f0 0x55d5bd338ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.509+0000 7f195cb57640 1 -- 172.21.15.117:0/723871833 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5bd262530 con 0x55d5bcf931b0 2024-09-21T06:04:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.510+0000 7f1957fff640 1 --2- 172.21.15.117:0/723871833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5bcf6a2f0 0x55d5bd338ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.510+0000 7f195cb57640 1 --2- 172.21.15.117:0/723871833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5bcf931b0 0x55d5bd3389b0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f1950009470 tx=0x7f19500094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=455201ff00bb5e6a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.510+0000 7f193e7fc640 1 -- 172.21.15.117:0/723871833 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950031cf0 con 0x55d5bcf931b0 2024-09-21T06:04:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.510+0000 7f193e7fc640 1 -- 172.21.15.117:0/723871833 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1950036040 con 0x55d5bcf931b0 2024-09-21T06:04:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.510+0000 7f193e7fc640 1 -- 172.21.15.117:0/723871833 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19500042a0 con 0x55d5bcf931b0 2024-09-21T06:04:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.510+0000 7f19557fa640 1 --2- 172.21.15.117:0/3431778663 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f19400b66b0 0x7f19400b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.510+0000 7f1957fff640 1 --2- 172.21.15.117:0/3431778663 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f19400b66b0 0x7f19400b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.510+0000 7f19557fa640 1 -- 172.21.15.117:0/3431778663 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f19480e7b50 con 0x7f1958101d60 2024-09-21T06:04:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.511+0000 7f1965603740 1 -- 172.21.15.117:0/723871833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5bcf931b0 msgr2=0x55d5bd3389b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.511+0000 7f1957fff640 1 --2- 172.21.15.117:0/3431778663 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f19400b66b0 0x7f19400b8bc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f19480e53a0 tx=0x7f194800cf20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.511+0000 7f1965603740 1 --2- 172.21.15.117:0/723871833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5bcf931b0 0x55d5bd3389b0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f1950009470 tx=0x7f19500094a0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.511+0000 7f1965603740 1 -- 172.21.15.117:0/723871833 shutdown_connections 2024-09-21T06:04:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.511+0000 7f1965603740 1 --2- 172.21.15.117:0/723871833 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d5bcf724d0 0x55d5bd339430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.511+0000 7f1965603740 1 --2- 172.21.15.117:0/723871833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5bcf6a2f0 0x55d5bd338ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.511+0000 7f1965603740 1 --2- 172.21.15.117:0/723871833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5bcf931b0 0x55d5bd3389b0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.511+0000 7f1965603740 1 -- 172.21.15.117:0/723871833 >> 172.21.15.117:0/723871833 conn(0x55d5bcf87300 msgr2=0x55d5bd3368a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.511+0000 7f1965603740 1 -- 172.21.15.117:0/723871833 shutdown_connections 2024-09-21T06:04:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.511+0000 7f1965603740 1 -- 172.21.15.117:0/723871833 wait complete. 2024-09-21T06:04:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.511+0000 7f1965603740 1 -- start start 2024-09-21T06:04:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.512+0000 7f1965603740 1 Processor -- start 2024-09-21T06:04:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.512+0000 7f1965603740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5bcf6a2f0 0x55d5bd344000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.512+0000 7f1965603740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d5bcf724d0 0x55d5bd344540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.512+0000 7f1965603740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5bcf931b0 0x55d5bd3503c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.512+0000 7f1965603740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d5bd1d4ab0 con 0x55d5bcf931b0 2024-09-21T06:04:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.512+0000 7f1965603740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d5bd242730 con 0x55d5bcf6a2f0 2024-09-21T06:04:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.512+0000 7f1965603740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55d5bd3470d0 con 0x55d5bcf724d0 2024-09-21T06:04:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.512+0000 7f1965603740 10 client.0 _ll_register_callbacks cb 0x55d5bd2966b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.512+0000 7f195cb57640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5bcf931b0 0x55d5bd3503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.512+0000 7f195cb57640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5bcf931b0 0x55d5bd3503c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35870/0 (socket says 172.21.15.117:35870) 2024-09-21T06:04:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.512+0000 7f195cb57640 1 -- 172.21.15.117:0/98399998 learned_addr learned my addr 172.21.15.117:0/98399998 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.512+0000 7f195cb57640 1 --2- 172.21.15.117:0/98399998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5bcf6a2f0 0x55d5bd344000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.512+0000 7f19577fe640 1 --2- 172.21.15.117:0/98399998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d5bcf724d0 0x55d5bd344540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.513+0000 7f195cb57640 1 -- 172.21.15.117:0/98399998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d5bcf724d0 msgr2=0x55d5bd344540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.513+0000 7f195cb57640 1 --2- 172.21.15.117:0/98399998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d5bcf724d0 0x55d5bd344540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.513+0000 7f195cb57640 1 -- 172.21.15.117:0/98399998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5bcf6a2f0 msgr2=0x55d5bd344000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.513+0000 7f195cb57640 1 --2- 172.21.15.117:0/98399998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5bcf6a2f0 0x55d5bd344000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.513+0000 7f195cb57640 1 -- 172.21.15.117:0/98399998 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5bcffe790 con 0x55d5bcf931b0 2024-09-21T06:04:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.513+0000 7f195cb57640 1 --2- 172.21.15.117:0/98399998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5bcf931b0 0x55d5bd3503c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x55d5bd3375c0 tx=0x7f1950003df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4fa1186c743ecda server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.513+0000 7f19377fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:04:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.513+0000 7f19377fe640 1 -- 172.21.15.117:0/98399998 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950031af0 con 0x55d5bcf931b0 2024-09-21T06:04:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.513+0000 7f19377fe640 1 -- 172.21.15.117:0/98399998 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f1950031c90 con 0x55d5bcf931b0 2024-09-21T06:04:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.513+0000 7f1965603740 1 -- 172.21.15.117:0/98399998 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1950036040 con 0x55d5bcf931b0 2024-09-21T06:04:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.513+0000 7f19377fe640 1 -- 172.21.15.117:0/98399998 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f19500044c0 con 0x55d5bcf931b0 2024-09-21T06:04:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.513+0000 7f1965603740 10 client.16737 fetch_fsmap learned FSMap version 29 2024-09-21T06:04:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.513+0000 7f1965603740 1 -- 172.21.15.117:0/98399998 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d5bd077620 con 0x55d5bcf931b0 2024-09-21T06:04:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.513+0000 7f19377fe640 1 -- 172.21.15.117:0/98399998 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1950036040 con 0x55d5bcf931b0 2024-09-21T06:04:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.513+0000 7f19377fe640 1 -- 172.21.15.117:0/98399998 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1950050070 con 0x55d5bcf931b0 2024-09-21T06:04:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.513+0000 7f1965603740 10 client.16737 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:04:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.514+0000 7f1965603740 10 client.16737 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:04:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.514+0000 7f1965603740 20 client.16737 populate_metadata read hostname 'smithi117' 2024-09-21T06:04:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.514+0000 7f1965603740 1 --2- 172.21.15.117:0/98399998 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55d5bcf51d80 0x55d5bd355e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.514+0000 7f1965603740 4 client.16737 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:04:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.514+0000 7f1965603740 1 -- 172.21.15.117:0/98399998 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5bd2af640 con 0x55d5bcf51d80 2024-09-21T06:04:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.514+0000 7f19577fe640 1 --2- 172.21.15.117:0/98399998 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55d5bcf51d80 0x55d5bd355e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.514+0000 7f19577fe640 1 --2- 172.21.15.117:0/98399998 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55d5bcf51d80 0x55d5bd355e80 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3a4a25e6cb0b466 server_cookie=948c0b829a617f23 in_seq=0 out_seq=0 2024-09-21T06:04:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.514+0000 7f19377fe640 10 client.16737 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.515+0000 7f19377fe640 1 -- 172.21.15.117:0/98399998 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d5bd2af640 con 0x55d5bcf51d80 2024-09-21T06:04:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.515+0000 7f19377fe640 10 client.16737 handle_command_reply: tid=0 2024-09-21T06:04:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.515+0000 7f1965603740 1 client.16737 shutdown 2024-09-21T06:04:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.515+0000 7f1965603740 1 -- 172.21.15.117:0/98399998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5bcf931b0 msgr2=0x55d5bd3503c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.515+0000 7f1965603740 1 --2- 172.21.15.117:0/98399998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5bcf931b0 0x55d5bd3503c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x55d5bd3375c0 tx=0x7f1950003df0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.515+0000 7f1965603740 1 -- 172.21.15.117:0/98399998 shutdown_connections 2024-09-21T06:04:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.515+0000 7f1965603740 1 --2- 172.21.15.117:0/98399998 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55d5bcf51d80 0x55d5bd355e80 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.515+0000 7f1965603740 1 --2- 172.21.15.117:0/98399998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5bcf931b0 0x55d5bd3503c0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.515+0000 7f1965603740 1 --2- 172.21.15.117:0/98399998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d5bcf724d0 0x55d5bd344540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.515+0000 7f19377fe640 0 client.16737 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.515+0000 7f1965603740 1 --2- 172.21.15.117:0/98399998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5bcf6a2f0 0x55d5bd344000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.515+0000 7f1965603740 1 -- 172.21.15.117:0/98399998 >> 172.21.15.117:0/98399998 conn(0x55d5bcf87300 msgr2=0x55d5bd3399e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.515+0000 7f1965603740 1 -- 172.21.15.117:0/98399998 shutdown_connections 2024-09-21T06:04:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.515+0000 7f1965603740 1 -- 172.21.15.117:0/98399998 wait complete. 2024-09-21T06:04:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.516+0000 7f1965603740 20 client.16737 trim_cache size 0 max 16384 2024-09-21T06:04:34.533 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.530+0000 7f1965603740 1 Processor -- start 2024-09-21T06:04:34.533 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.530+0000 7f1965603740 1 -- start start 2024-09-21T06:04:34.533 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.530+0000 7f1965603740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5bcf6a2f0 0x55d5bd33a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:34.533 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.530+0000 7f1965603740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d5bcf724d0 0x55d5bd350330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:34.533 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.530+0000 7f1965603740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5bcf931b0 0x55d5bd345c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:34.533 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.530+0000 7f1965603740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f19581b1820 con 0x55d5bcf6a2f0 2024-09-21T06:04:34.533 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.530+0000 7f1965603740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f19581b3360 con 0x55d5bcf931b0 2024-09-21T06:04:34.533 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.530+0000 7f1965603740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f19581b1ef0 con 0x55d5bcf724d0 2024-09-21T06:04:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.530+0000 7f195cb57640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5bcf6a2f0 0x55d5bd33a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.530+0000 7f195cb57640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5bcf6a2f0 0x55d5bd33a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35886/0 (socket says 172.21.15.117:35886) 2024-09-21T06:04:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.530+0000 7f195cb57640 1 -- 172.21.15.117:0/794378771 learned_addr learned my addr 172.21.15.117:0/794378771 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.531+0000 7f19577fe640 1 --2- 172.21.15.117:0/794378771 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d5bcf724d0 0x55d5bd350330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.531+0000 7f195cb57640 1 --2- 172.21.15.117:0/794378771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5bcf931b0 0x55d5bd345c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.531+0000 7f195cb57640 1 -- 172.21.15.117:0/794378771 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d5bcf724d0 msgr2=0x55d5bd350330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.531+0000 7f195cb57640 1 --2- 172.21.15.117:0/794378771 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d5bcf724d0 0x55d5bd350330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.531+0000 7f195cb57640 1 -- 172.21.15.117:0/794378771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5bcf931b0 msgr2=0x55d5bd345c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.531+0000 7f195cb57640 1 --2- 172.21.15.117:0/794378771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5bcf931b0 0x55d5bd345c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.531+0000 7f195cb57640 1 -- 172.21.15.117:0/794378771 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5bd33acb0 con 0x55d5bcf6a2f0 2024-09-21T06:04:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.531+0000 7f19577fe640 1 --2- 172.21.15.117:0/794378771 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d5bcf724d0 0x55d5bd350330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:04:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.531+0000 7f195cb57640 1 --2- 172.21.15.117:0/794378771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5bcf6a2f0 0x55d5bd33a3a0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f1950002230 tx=0x7f195002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ee2436d15f5a98 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.531+0000 7f193dffb640 1 -- 172.21.15.117:0/794378771 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950061040 con 0x55d5bcf6a2f0 2024-09-21T06:04:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.531+0000 7f193dffb640 1 -- 172.21.15.117:0/794378771 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1950036040 con 0x55d5bcf6a2f0 2024-09-21T06:04:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.531+0000 7f193dffb640 1 -- 172.21.15.117:0/794378771 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950003b10 con 0x55d5bcf6a2f0 2024-09-21T06:04:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.532+0000 7f1965603740 1 -- 172.21.15.117:0/794378771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5bcf6a2f0 msgr2=0x55d5bd33a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.532+0000 7f1965603740 1 --2- 172.21.15.117:0/794378771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5bcf6a2f0 0x55d5bd33a3a0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f1950002230 tx=0x7f195002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.532+0000 7f1965603740 1 -- 172.21.15.117:0/794378771 shutdown_connections 2024-09-21T06:04:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.532+0000 7f1965603740 1 --2- 172.21.15.117:0/794378771 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5bcf931b0 0x55d5bd345c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.532+0000 7f1965603740 1 --2- 172.21.15.117:0/794378771 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d5bcf724d0 0x55d5bd350330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.532+0000 7f1965603740 1 --2- 172.21.15.117:0/794378771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5bcf6a2f0 0x55d5bd33a3a0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.532+0000 7f1965603740 1 -- 172.21.15.117:0/794378771 >> 172.21.15.117:0/794378771 conn(0x55d5bcf87300 msgr2=0x55d5bd345520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.532+0000 7f1965603740 1 -- 172.21.15.117:0/794378771 shutdown_connections 2024-09-21T06:04:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.532+0000 7f1965603740 1 -- 172.21.15.117:0/794378771 wait complete. 2024-09-21T06:04:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.532+0000 7f1965603740 1 -- start start 2024-09-21T06:04:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.532+0000 7f1965603740 1 Processor -- start 2024-09-21T06:04:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.533+0000 7f1965603740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d5bcf6a2f0 0x55d5bd358310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.533+0000 7f1965603740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5bcf724d0 0x55d5bd3588f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.533+0000 7f1965603740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5bcf931b0 0x55d5bd3be600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.533+0000 7f1965603740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f19581b2c90 con 0x55d5bcf931b0 2024-09-21T06:04:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.533+0000 7f1965603740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f19581b25c0 con 0x55d5bcf724d0 2024-09-21T06:04:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.533+0000 7f1965603740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f19581af820 con 0x55d5bcf6a2f0 2024-09-21T06:04:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.533+0000 7f1965603740 10 client.0 _ll_register_callbacks cb 0x7f194c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.533+0000 7f195cb57640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5bcf931b0 0x55d5bd3be600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.533+0000 7f195cb57640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5bcf931b0 0x55d5bd3be600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35890/0 (socket says 172.21.15.117:35890) 2024-09-21T06:04:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.533+0000 7f195cb57640 1 -- 172.21.15.117:0/1396500666 learned_addr learned my addr 172.21.15.117:0/1396500666 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.533+0000 7f195cb57640 1 --2- 172.21.15.117:0/1396500666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d5bcf6a2f0 0x55d5bd358310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.533+0000 7f19577fe640 1 --2- 172.21.15.117:0/1396500666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5bcf724d0 0x55d5bd3588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.533+0000 7f195cb57640 1 -- 172.21.15.117:0/1396500666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d5bcf6a2f0 msgr2=0x55d5bd358310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.533+0000 7f195cb57640 1 --2- 172.21.15.117:0/1396500666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d5bcf6a2f0 0x55d5bd358310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.533+0000 7f195cb57640 1 -- 172.21.15.117:0/1396500666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5bcf724d0 msgr2=0x55d5bd3588f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.533+0000 7f195cb57640 1 --2- 172.21.15.117:0/1396500666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5bcf724d0 0x55d5bd3588f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.533+0000 7f195cb57640 1 -- 172.21.15.117:0/1396500666 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5bd3beb40 con 0x55d5bcf931b0 2024-09-21T06:04:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.533+0000 7f195cb57640 1 --2- 172.21.15.117:0/1396500666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5bcf931b0 0x55d5bd3be600 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f19500094e0 tx=0x7f1950003a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8bf7e92a5285f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.533+0000 7f19377fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:04:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.534+0000 7f19377fe640 1 -- 172.21.15.117:0/1396500666 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195002fe80 con 0x55d5bcf931b0 2024-09-21T06:04:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.534+0000 7f19377fe640 1 -- 172.21.15.117:0/1396500666 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f1950031af0 con 0x55d5bcf931b0 2024-09-21T06:04:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.534+0000 7f1965603740 1 -- 172.21.15.117:0/1396500666 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1950036040 con 0x55d5bcf931b0 2024-09-21T06:04:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.534+0000 7f19377fe640 1 -- 172.21.15.117:0/1396500666 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1950031cf0 con 0x55d5bcf931b0 2024-09-21T06:04:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.534+0000 7f1965603740 10 client.16743 fetch_fsmap learned FSMap version 29 2024-09-21T06:04:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.534+0000 7f1965603740 1 -- 172.21.15.117:0/1396500666 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d5bd3bfd00 con 0x55d5bcf931b0 2024-09-21T06:04:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.534+0000 7f19377fe640 1 -- 172.21.15.117:0/1396500666 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1950036040 con 0x55d5bcf931b0 2024-09-21T06:04:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.534+0000 7f19377fe640 1 -- 172.21.15.117:0/1396500666 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1950064070 con 0x55d5bcf931b0 2024-09-21T06:04:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.534+0000 7f1965603740 10 client.16743 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:04:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.534+0000 7f1965603740 10 client.16743 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:04:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.534+0000 7f1965603740 20 client.16743 populate_metadata read hostname 'smithi117' 2024-09-21T06:04:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.534+0000 7f1965603740 1 --2- 172.21.15.117:0/1396500666 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55d5bcf271c0 0x55d5bd3c23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.534+0000 7f1965603740 4 client.16743 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:04:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.534+0000 7f1965603740 1 -- 172.21.15.117:0/1396500666 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d5bd2ab0f0 con 0x55d5bcf271c0 2024-09-21T06:04:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.535+0000 7f19577fe640 1 --2- 172.21.15.117:0/1396500666 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55d5bcf271c0 0x55d5bd3c23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.535+0000 7f19377fe640 10 client.16743 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.535+0000 7f19577fe640 1 --2- 172.21.15.117:0/1396500666 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55d5bcf271c0 0x55d5bd3c23c0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a993f1b3b9757385 server_cookie=ef7bfe04acd78296 in_seq=0 out_seq=0 2024-09-21T06:04:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.549+0000 7f19377fe640 1 -- 172.21.15.117:0/1396500666 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d5bd2ab0f0 con 0x55d5bcf271c0 2024-09-21T06:04:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.549+0000 7f19377fe640 10 client.16743 handle_command_reply: tid=0 2024-09-21T06:04:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.549+0000 7f1965603740 1 client.16743 shutdown 2024-09-21T06:04:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.549+0000 7f1965603740 1 -- 172.21.15.117:0/1396500666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5bcf931b0 msgr2=0x55d5bd3be600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.549+0000 7f1965603740 1 --2- 172.21.15.117:0/1396500666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5bcf931b0 0x55d5bd3be600 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f19500094e0 tx=0x7f1950003a10 comp rx=0 tx=0).stop 2024-09-21T06:04:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.549+0000 7f1965603740 1 -- 172.21.15.117:0/1396500666 shutdown_connections 2024-09-21T06:04:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.549+0000 7f1965603740 1 --2- 172.21.15.117:0/1396500666 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55d5bcf271c0 0x55d5bd3c23c0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.549+0000 7f1965603740 1 --2- 172.21.15.117:0/1396500666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d5bcf931b0 0x55d5bd3be600 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.549+0000 7f1965603740 1 --2- 172.21.15.117:0/1396500666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5bcf724d0 0x55d5bd3588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.549+0000 7f19377fe640 0 client.16743 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.549+0000 7f1965603740 1 --2- 172.21.15.117:0/1396500666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d5bcf6a2f0 0x55d5bd358310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.549+0000 7f1965603740 1 -- 172.21.15.117:0/1396500666 >> 172.21.15.117:0/1396500666 conn(0x55d5bcf87300 msgr2=0x55d5bd33a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.549+0000 7f1965603740 1 -- 172.21.15.117:0/1396500666 shutdown_connections 2024-09-21T06:04:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.549+0000 7f1965603740 1 -- 172.21.15.117:0/1396500666 wait complete. 2024-09-21T06:04:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.549+0000 7f1965603740 20 client.16743 trim_cache size 0 max 16384 2024-09-21T06:04:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T06:04:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T06:04:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T06:04:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T06:04:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.550+0000 7f195e5ba640 1 -- 172.21.15.117:0/3431778663 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f19400b66b0 msgr2=0x7f19400b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.550+0000 7f195e5ba640 1 --2- 172.21.15.117:0/3431778663 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f19400b66b0 0x7f19400b8bc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f19480e53a0 tx=0x7f194800cf20 comp rx=0 tx=0).stop 2024-09-21T06:04:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.550+0000 7f195e5ba640 1 -- 172.21.15.117:0/3431778663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1958101d60 msgr2=0x7f1958102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.550+0000 7f195e5ba640 1 --2- 172.21.15.117:0/3431778663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1958101d60 0x7f1958102f90 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f194800e9f0 tx=0x7f194800eeb0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.550+0000 7f195e5ba640 1 -- 172.21.15.117:0/3431778663 shutdown_connections 2024-09-21T06:04:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.550+0000 7f195e5ba640 1 --2- 172.21.15.117:0/3431778663 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f19400b66b0 0x7f19400b8bc0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.550+0000 7f195e5ba640 1 --2- 172.21.15.117:0/3431778663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1958108680 0x7f1958103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.550+0000 7f195e5ba640 1 --2- 172.21.15.117:0/3431778663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f19581026b0 0x7f19581034d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.550+0000 7f195e5ba640 1 --2- 172.21.15.117:0/3431778663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1958101d60 0x7f1958102f90 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.550+0000 7f195e5ba640 1 -- 172.21.15.117:0/3431778663 >> 172.21.15.117:0/3431778663 conn(0x7f1958077ad0 msgr2=0x7f19581063e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.550+0000 7f195e5ba640 1 -- 172.21.15.117:0/3431778663 shutdown_connections 2024-09-21T06:04:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:34.550+0000 7f195e5ba640 1 -- 172.21.15.117:0/3431778663 wait complete. 2024-09-21T06:04:36.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:35 smithi192 ceph-mon[34700]: pgmap v548: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 147 MiB/s wr, 875 op/s 2024-09-21T06:04:36.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:04:36.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:04:36.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:04:36.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:04:36.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:35 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T06:04:36.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:35 smithi117 ceph-mon[30947]: pgmap v548: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 147 MiB/s wr, 875 op/s 2024-09-21T06:04:36.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:04:36.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:04:36.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:04:36.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:04:36.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:35 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T06:04:36.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:35 smithi138 ceph-mon[34391]: pgmap v548: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 147 MiB/s wr, 875 op/s 2024-09-21T06:04:36.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:04:36.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T06:04:36.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T06:04:36.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T06:04:36.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:35 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T06:04:37.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:36 smithi192 ceph-mon[34700]: Adjusting osd_memory_target on smithi138 to 176.0M 2024-09-21T06:04:37.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:36 smithi192 ceph-mon[34700]: Unable to set osd_memory_target on smithi138 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-21T06:04:37.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:36 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:04:37.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:36 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:04:37.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:36 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:04:37.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:36 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T06:04:37.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:36 smithi192 ceph-mon[34700]: Adjusting osd_memory_target on smithi192 to 176.0M 2024-09-21T06:04:37.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:36 smithi192 ceph-mon[34700]: Unable to set osd_memory_target on smithi192 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-21T06:04:37.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:36 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:04:37.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:36 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:04:37.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:36 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:04:37.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:36 smithi117 ceph-mon[30947]: Adjusting osd_memory_target on smithi138 to 176.0M 2024-09-21T06:04:37.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:36 smithi117 ceph-mon[30947]: Unable to set osd_memory_target on smithi138 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-21T06:04:37.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:36 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:04:37.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:36 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:04:37.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:36 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:04:37.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:36 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T06:04:37.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:36 smithi117 ceph-mon[30947]: Adjusting osd_memory_target on smithi192 to 176.0M 2024-09-21T06:04:37.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:36 smithi117 ceph-mon[30947]: Unable to set osd_memory_target on smithi192 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-21T06:04:37.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:36 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:04:37.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:36 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:04:37.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:36 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:04:37.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:36 smithi138 ceph-mon[34391]: Adjusting osd_memory_target on smithi138 to 176.0M 2024-09-21T06:04:37.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:36 smithi138 ceph-mon[34391]: Unable to set osd_memory_target on smithi138 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-21T06:04:37.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:36 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T06:04:37.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:36 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T06:04:37.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:36 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T06:04:37.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:36 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T06:04:37.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:36 smithi138 ceph-mon[34391]: Adjusting osd_memory_target on smithi192 to 176.0M 2024-09-21T06:04:37.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:36 smithi138 ceph-mon[34391]: Unable to set osd_memory_target on smithi192 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-21T06:04:37.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:36 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:04:37.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:36 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:04:37.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:36 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:04:38.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:37 smithi192 ceph-mon[34700]: pgmap v549: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 34 MiB/s wr, 218 op/s 2024-09-21T06:04:38.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:37 smithi117 ceph-mon[30947]: pgmap v549: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 34 MiB/s wr, 218 op/s 2024-09-21T06:04:38.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:37 smithi138 ceph-mon[34391]: pgmap v549: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 34 MiB/s wr, 218 op/s 2024-09-21T06:04:40.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:39 smithi192 ceph-mon[34700]: pgmap v550: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-21T06:04:40.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:39 smithi117 ceph-mon[30947]: pgmap v550: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-21T06:04:40.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:39 smithi138 ceph-mon[34391]: pgmap v550: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-09-21T06:04:42.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:41 smithi192 ceph-mon[34700]: pgmap v551: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.4 KiB/s wr, 13 op/s 2024-09-21T06:04:42.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:41 smithi117 ceph-mon[30947]: pgmap v551: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.4 KiB/s wr, 13 op/s 2024-09-21T06:04:42.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:41 smithi138 ceph-mon[34391]: pgmap v551: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 8.4 KiB/s wr, 13 op/s 2024-09-21T06:04:43.550 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:04:43.813 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:04:44.207 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:43 smithi117 ceph-mon[30947]: pgmap v552: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.4 KiB/s wr, 11 op/s 2024-09-21T06:04:44.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:43 smithi192 ceph-mon[34700]: pgmap v552: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.4 KiB/s wr, 11 op/s 2024-09-21T06:04:44.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.300+0000 7fc64d377640 1 -- 172.21.15.117:0/1743859150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc648102aa0 msgr2=0x7fc648102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:44.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.300+0000 7fc64d377640 1 --2- 172.21.15.117:0/1743859150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc648102aa0 0x7fc648102eb0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fc638009900 tx=0x7fc63802a410 comp rx=0 tx=0).stop 2024-09-21T06:04:44.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.300+0000 7fc64d377640 1 -- 172.21.15.117:0/1743859150 shutdown_connections 2024-09-21T06:04:44.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.300+0000 7fc64d377640 1 --2- 172.21.15.117:0/1743859150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc648108d30 0x7fc64810f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.300+0000 7fc64d377640 1 --2- 172.21.15.117:0/1743859150 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6481083e0 0x7fc6481087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.300+0000 7fc64d377640 1 --2- 172.21.15.117:0/1743859150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc648102aa0 0x7fc648102eb0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.300+0000 7fc64d377640 1 -- 172.21.15.117:0/1743859150 >> 172.21.15.117:0/1743859150 conn(0x7fc6480797e0 msgr2=0x7fc648079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:44.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.300+0000 7fc64d377640 1 -- 172.21.15.117:0/1743859150 shutdown_connections 2024-09-21T06:04:44.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.300+0000 7fc64d377640 1 -- 172.21.15.117:0/1743859150 wait complete. 2024-09-21T06:04:44.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.301+0000 7fc64d377640 1 Processor -- start 2024-09-21T06:04:44.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.301+0000 7fc64d377640 1 -- start start 2024-09-21T06:04:44.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.301+0000 7fc64d377640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc648102aa0 0x7fc648199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:44.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.301+0000 7fc64d377640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6481083e0 0x7fc648199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:44.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.301+0000 7fc64d377640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc648108d30 0x7fc64819ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:44.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.301+0000 7fc64d377640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc6481129a0 con 0x7fc648108d30 2024-09-21T06:04:44.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.301+0000 7fc64d377640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc648112820 con 0x7fc6481083e0 2024-09-21T06:04:44.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.301+0000 7fc64d377640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc648112b20 con 0x7fc648102aa0 2024-09-21T06:04:44.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.301+0000 7fc6477fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc648108d30 0x7fc64819ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:44.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.301+0000 7fc6477fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc648108d30 0x7fc64819ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35914/0 (socket says 172.21.15.117:35914) 2024-09-21T06:04:44.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.301+0000 7fc6477fe640 1 -- 172.21.15.117:0/3407730646 learned_addr learned my addr 172.21.15.117:0/3407730646 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:44.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.301+0000 7fc6467fc640 1 --2- 172.21.15.117:0/3407730646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6481083e0 0x7fc648199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:44.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.302+0000 7fc646ffd640 1 --2- 172.21.15.117:0/3407730646 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc648102aa0 0x7fc648199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:44.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.302+0000 7fc6477fe640 1 -- 172.21.15.117:0/3407730646 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc648102aa0 msgr2=0x7fc648199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:44.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.302+0000 7fc6477fe640 1 --2- 172.21.15.117:0/3407730646 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc648102aa0 0x7fc648199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.302+0000 7fc6477fe640 1 -- 172.21.15.117:0/3407730646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6481083e0 msgr2=0x7fc648199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:44.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.302+0000 7fc6477fe640 1 --2- 172.21.15.117:0/3407730646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6481083e0 0x7fc648199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.302+0000 7fc6477fe640 1 -- 172.21.15.117:0/3407730646 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc64819f300 con 0x7fc648108d30 2024-09-21T06:04:44.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.302+0000 7fc6467fc640 1 --2- 172.21.15.117:0/3407730646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6481083e0 0x7fc648199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:04:44.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.302+0000 7fc6477fe640 1 --2- 172.21.15.117:0/3407730646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc648108d30 0x7fc64819ebd0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fc64000eb10 tx=0x7fc64000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:44.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.302+0000 7fc62ffff640 1 -- 172.21.15.117:0/3407730646 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc640004b20 con 0x7fc648108d30 2024-09-21T06:04:44.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.302+0000 7fc64d377640 1 -- 172.21.15.117:0/3407730646 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc64819f5f0 con 0x7fc648108d30 2024-09-21T06:04:44.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.302+0000 7fc62ffff640 1 -- 172.21.15.117:0/3407730646 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc640004cc0 con 0x7fc648108d30 2024-09-21T06:04:44.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.302+0000 7fc62ffff640 1 -- 172.21.15.117:0/3407730646 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc640010670 con 0x7fc648108d30 2024-09-21T06:04:44.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.302+0000 7fc64d377640 1 -- 172.21.15.117:0/3407730646 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc64806b820 con 0x7fc648108d30 2024-09-21T06:04:44.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.305+0000 7fc62ffff640 1 -- 172.21.15.117:0/3407730646 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fc640005020 con 0x7fc648108d30 2024-09-21T06:04:44.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.306+0000 7fc6538eb740 1 Processor -- start 2024-09-21T06:04:44.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.306+0000 7fc6538eb740 1 -- start start 2024-09-21T06:04:44.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.306+0000 7fc62ffff640 1 --2- 172.21.15.117:0/3407730646 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc6300b6680 0x7fc6300b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:44.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.306+0000 7fc6538eb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55754ea75070 0x55754ee1c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:44.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.306+0000 7fc62ffff640 1 -- 172.21.15.117:0/3407730646 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7fc6400e7e00 con 0x7fc648108d30 2024-09-21T06:04:44.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.306+0000 7fc6467fc640 1 --2- 172.21.15.117:0/3407730646 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc6300b6680 0x7fc6300b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:44.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.306+0000 7fc6538eb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55754ea4c1b0 0x55754ee1cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:44.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.306+0000 7fc6538eb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55754ea54390 0x55754ee1d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:44.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.306+0000 7fc6538eb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55754ed92e50 con 0x55754ea4c1b0 2024-09-21T06:04:44.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.306+0000 7fc6538eb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55754ed82280 con 0x55754ea54390 2024-09-21T06:04:44.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.306+0000 7fc6538eb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55754ed86090 con 0x55754ea75070 2024-09-21T06:04:44.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.306+0000 7fc6467fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55754ea4c1b0 0x55754ee1cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:44.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.306+0000 7fc6467fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55754ea4c1b0 0x55754ee1cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35926/0 (socket says 172.21.15.117:35926) 2024-09-21T06:04:44.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.306+0000 7fc6467fc640 1 -- 172.21.15.117:0/116155650 learned_addr learned my addr 172.21.15.117:0/116155650 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:44.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.306+0000 7fc646ffd640 1 --2- 172.21.15.117:0/116155650 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55754ea75070 0x55754ee1c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:44.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.306+0000 7fc6477fe640 1 --2- 172.21.15.117:0/116155650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55754ea54390 0x55754ee1d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:44.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.306+0000 7fc6467fc640 1 -- 172.21.15.117:0/116155650 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55754ea75070 msgr2=0x55754ee1c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:44.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.306+0000 7fc6467fc640 1 --2- 172.21.15.117:0/116155650 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55754ea75070 0x55754ee1c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.306+0000 7fc6467fc640 1 -- 172.21.15.117:0/116155650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55754ea54390 msgr2=0x55754ee1d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:44.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.306+0000 7fc6467fc640 1 --2- 172.21.15.117:0/116155650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55754ea54390 0x55754ee1d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.306+0000 7fc6467fc640 1 -- 172.21.15.117:0/116155650 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55754eae0650 con 0x55754ea4c1b0 2024-09-21T06:04:44.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.306+0000 7fc6477fe640 1 --2- 172.21.15.117:0/116155650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55754ea54390 0x55754ee1d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:44.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.307+0000 7fc6467fc640 1 --2- 172.21.15.117:0/116155650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55754ea4c1b0 0x55754ee1cb90 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fc63c013aa0 tx=0x7fc63c013f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45318e4cdc25952f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:44.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.307+0000 7fc6467fc640 1 --2- 172.21.15.117:0/3407730646 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc6300b6680 0x7fc6300b8b90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc63c013fd0 tx=0x7fc63c015000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:44.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.307+0000 7fc62d7fa640 1 -- 172.21.15.117:0/116155650 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c023070 con 0x55754ea4c1b0 2024-09-21T06:04:44.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.307+0000 7fc62d7fa640 1 -- 172.21.15.117:0/116155650 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc63c0135b0 con 0x55754ea4c1b0 2024-09-21T06:04:44.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.307+0000 7fc62d7fa640 1 -- 172.21.15.117:0/116155650 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c01f420 con 0x55754ea4c1b0 2024-09-21T06:04:44.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.308+0000 7fc6538eb740 1 -- 172.21.15.117:0/116155650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55754ea4c1b0 msgr2=0x55754ee1cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:44.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.308+0000 7fc6538eb740 1 --2- 172.21.15.117:0/116155650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55754ea4c1b0 0x55754ee1cb90 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fc63c013aa0 tx=0x7fc63c013f60 comp rx=0 tx=0).stop 2024-09-21T06:04:44.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.308+0000 7fc6538eb740 1 -- 172.21.15.117:0/116155650 shutdown_connections 2024-09-21T06:04:44.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.308+0000 7fc6538eb740 1 --2- 172.21.15.117:0/116155650 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55754ea4c1b0 0x55754ee1cb90 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.308+0000 7fc6538eb740 1 --2- 172.21.15.117:0/116155650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55754ea54390 0x55754ee1d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.308+0000 7fc6538eb740 1 --2- 172.21.15.117:0/116155650 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55754ea75070 0x55754ee1c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.308+0000 7fc6538eb740 1 -- 172.21.15.117:0/116155650 >> 172.21.15.117:0/116155650 conn(0x55754ea69460 msgr2=0x55754ee1a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:44.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.308+0000 7fc6538eb740 1 -- 172.21.15.117:0/116155650 shutdown_connections 2024-09-21T06:04:44.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.308+0000 7fc6538eb740 1 -- 172.21.15.117:0/116155650 wait complete. 2024-09-21T06:04:44.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.308+0000 7fc6538eb740 1 -- start start 2024-09-21T06:04:44.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.308+0000 7fc6538eb740 1 Processor -- start 2024-09-21T06:04:44.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.309+0000 7fc6538eb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55754ea4c1b0 0x55754ee27df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:44.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.309+0000 7fc6538eb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55754ea54390 0x55754ee28330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:44.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.309+0000 7fc6538eb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55754ea75070 0x55754ee340c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:44.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.309+0000 7fc6538eb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55754ebe3700 con 0x55754ea75070 2024-09-21T06:04:44.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.309+0000 7fc6538eb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55754ed1d110 con 0x55754ea54390 2024-09-21T06:04:44.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.309+0000 7fc6538eb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55754ee2add0 con 0x55754ea4c1b0 2024-09-21T06:04:44.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.309+0000 7fc6538eb740 10 client.0 _ll_register_callbacks cb 0x55754ed36990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:44.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.309+0000 7fc6477fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55754ea75070 0x55754ee340c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:44.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.309+0000 7fc6477fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55754ea75070 0x55754ee340c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35930/0 (socket says 172.21.15.117:35930) 2024-09-21T06:04:44.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.309+0000 7fc6477fe640 1 -- 172.21.15.117:0/1786304536 learned_addr learned my addr 172.21.15.117:0/1786304536 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:44.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.309+0000 7fc646ffd640 1 --2- 172.21.15.117:0/1786304536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55754ea4c1b0 0x55754ee27df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:44.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.309+0000 7fc6467fc640 1 --2- 172.21.15.117:0/1786304536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55754ea54390 0x55754ee28330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:44.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.309+0000 7fc6477fe640 1 -- 172.21.15.117:0/1786304536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55754ea4c1b0 msgr2=0x55754ee27df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:44.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.309+0000 7fc6477fe640 1 --2- 172.21.15.117:0/1786304536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55754ea4c1b0 0x55754ee27df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.309+0000 7fc6477fe640 1 -- 172.21.15.117:0/1786304536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55754ea54390 msgr2=0x55754ee28330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:44.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.309+0000 7fc6477fe640 1 --2- 172.21.15.117:0/1786304536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55754ea54390 0x55754ee28330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.309+0000 7fc6477fe640 1 -- 172.21.15.117:0/1786304536 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55754ed4ac40 con 0x55754ea75070 2024-09-21T06:04:44.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.309+0000 7fc6477fe640 1 --2- 172.21.15.117:0/1786304536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55754ea75070 0x55754ee340c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fc6400aa270 tx=0x7fc6400ad5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cef0e59317251b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:44.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.309+0000 7fc62aff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:04:44.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.310+0000 7fc62aff5640 1 -- 172.21.15.117:0/1786304536 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6400ad7b0 con 0x55754ea75070 2024-09-21T06:04:44.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.310+0000 7fc62aff5640 1 -- 172.21.15.117:0/1786304536 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7fc6400d1110 con 0x55754ea75070 2024-09-21T06:04:44.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.310+0000 7fc6538eb740 1 -- 172.21.15.117:0/1786304536 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc63c0135b0 con 0x55754ea75070 2024-09-21T06:04:44.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.310+0000 7fc62aff5640 1 -- 172.21.15.117:0/1786304536 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc6400c8050 con 0x55754ea75070 2024-09-21T06:04:44.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.310+0000 7fc6538eb740 10 client.16755 fetch_fsmap learned FSMap version 29 2024-09-21T06:04:44.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.310+0000 7fc6538eb740 1 -- 172.21.15.117:0/1786304536 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55754ee36ba0 con 0x55754ea75070 2024-09-21T06:04:44.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.310+0000 7fc62aff5640 1 -- 172.21.15.117:0/1786304536 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc63c0135b0 con 0x55754ea75070 2024-09-21T06:04:44.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.310+0000 7fc62aff5640 1 -- 172.21.15.117:0/1786304536 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc6400b7230 con 0x55754ea75070 2024-09-21T06:04:44.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.310+0000 7fc6538eb740 10 client.16755 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:04:44.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.310+0000 7fc6538eb740 10 client.16755 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:04:44.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.310+0000 7fc6538eb740 20 client.16755 populate_metadata read hostname 'smithi117' 2024-09-21T06:04:44.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.310+0000 7fc6538eb740 1 --2- 172.21.15.117:0/1786304536 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55754ea339f0 0x55754ee3a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:44.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.310+0000 7fc6538eb740 4 client.16755 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:04:44.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.310+0000 7fc6538eb740 1 -- 172.21.15.117:0/1786304536 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55754ed94da0 con 0x55754ea339f0 2024-09-21T06:04:44.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.311+0000 7fc646ffd640 1 --2- 172.21.15.117:0/1786304536 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55754ea339f0 0x55754ee3a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:44.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.311+0000 7fc62aff5640 10 client.16755 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:44.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.311+0000 7fc646ffd640 1 --2- 172.21.15.117:0/1786304536 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55754ea339f0 0x55754ee3a070 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6934cabeed040c5f server_cookie=dce3b3a5dec3530d in_seq=0 out_seq=0 2024-09-21T06:04:44.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.312+0000 7fc62aff5640 1 -- 172.21.15.117:0/1786304536 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55754ed94da0 con 0x55754ea339f0 2024-09-21T06:04:44.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.312+0000 7fc62aff5640 10 client.16755 handle_command_reply: tid=0 2024-09-21T06:04:44.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.312+0000 7fc6538eb740 1 client.16755 shutdown 2024-09-21T06:04:44.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.312+0000 7fc6538eb740 1 -- 172.21.15.117:0/1786304536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55754ea75070 msgr2=0x55754ee340c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:44.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.312+0000 7fc6538eb740 1 --2- 172.21.15.117:0/1786304536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55754ea75070 0x55754ee340c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fc6400aa270 tx=0x7fc6400ad5f0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.312+0000 7fc6538eb740 1 -- 172.21.15.117:0/1786304536 shutdown_connections 2024-09-21T06:04:44.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.312+0000 7fc6538eb740 1 --2- 172.21.15.117:0/1786304536 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55754ea339f0 0x55754ee3a070 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.312+0000 7fc6538eb740 1 --2- 172.21.15.117:0/1786304536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55754ea75070 0x55754ee340c0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.312+0000 7fc6538eb740 1 --2- 172.21.15.117:0/1786304536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55754ea54390 0x55754ee28330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.312+0000 7fc62aff5640 0 client.16755 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:44.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.312+0000 7fc6538eb740 1 --2- 172.21.15.117:0/1786304536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55754ea4c1b0 0x55754ee27df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.312+0000 7fc6538eb740 1 -- 172.21.15.117:0/1786304536 >> 172.21.15.117:0/1786304536 conn(0x55754ea69460 msgr2=0x55754ee1f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:44.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.312+0000 7fc6538eb740 1 -- 172.21.15.117:0/1786304536 shutdown_connections 2024-09-21T06:04:44.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.312+0000 7fc6538eb740 1 -- 172.21.15.117:0/1786304536 wait complete. 2024-09-21T06:04:44.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.312+0000 7fc6538eb740 20 client.16755 trim_cache size 0 max 16384 2024-09-21T06:04:44.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.329+0000 7fc6538eb740 1 Processor -- start 2024-09-21T06:04:44.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.329+0000 7fc6538eb740 1 -- start start 2024-09-21T06:04:44.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.330+0000 7fc6538eb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55754ea339f0 0x55754ee1dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:44.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.330+0000 7fc6538eb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55754ea4c1b0 0x55754ee33fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:44.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.330+0000 7fc6538eb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55754ea54390 0x55754ee34500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:44.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.330+0000 7fc6538eb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc6481b1810 con 0x55754ea4c1b0 2024-09-21T06:04:44.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.330+0000 7fc6538eb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc6481b3350 con 0x55754ea54390 2024-09-21T06:04:44.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.330+0000 7fc6538eb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc6481b1ee0 con 0x55754ea339f0 2024-09-21T06:04:44.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.330+0000 7fc6467fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55754ea4c1b0 0x55754ee33fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:44.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.330+0000 7fc6467fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55754ea4c1b0 0x55754ee33fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35936/0 (socket says 172.21.15.117:35936) 2024-09-21T06:04:44.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.330+0000 7fc6467fc640 1 -- 172.21.15.117:0/372571407 learned_addr learned my addr 172.21.15.117:0/372571407 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:44.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.330+0000 7fc646ffd640 1 --2- 172.21.15.117:0/372571407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55754ea339f0 0x55754ee1dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:44.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.330+0000 7fc6477fe640 1 --2- 172.21.15.117:0/372571407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55754ea54390 0x55754ee34500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:44.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.330+0000 7fc6467fc640 1 -- 172.21.15.117:0/372571407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55754ea339f0 msgr2=0x55754ee1dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:44.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.330+0000 7fc6467fc640 1 --2- 172.21.15.117:0/372571407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55754ea339f0 0x55754ee1dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.330+0000 7fc6467fc640 1 -- 172.21.15.117:0/372571407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55754ea54390 msgr2=0x55754ee34500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:44.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.330+0000 7fc6467fc640 1 --2- 172.21.15.117:0/372571407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55754ea54390 0x55754ee34500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.330+0000 7fc6467fc640 1 -- 172.21.15.117:0/372571407 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55754ee1aa80 con 0x55754ea4c1b0 2024-09-21T06:04:44.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.331+0000 7fc6477fe640 1 --2- 172.21.15.117:0/372571407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55754ea54390 0x55754ee34500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:04:44.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.331+0000 7fc6467fc640 1 --2- 172.21.15.117:0/372571407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55754ea4c1b0 0x55754ee33fc0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x55754ee1b2d0 tx=0x7fc63c011fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801f31cb7f8194a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:44.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.331+0000 7fc62cff9640 1 -- 172.21.15.117:0/372571407 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c023050 con 0x55754ea4c1b0 2024-09-21T06:04:44.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.331+0000 7fc62cff9640 1 -- 172.21.15.117:0/372571407 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc63c017070 con 0x55754ea4c1b0 2024-09-21T06:04:44.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.331+0000 7fc62cff9640 1 -- 172.21.15.117:0/372571407 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63c00c900 con 0x55754ea4c1b0 2024-09-21T06:04:44.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.332+0000 7fc6538eb740 1 -- 172.21.15.117:0/372571407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55754ea4c1b0 msgr2=0x55754ee33fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:44.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.332+0000 7fc6538eb740 1 --2- 172.21.15.117:0/372571407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55754ea4c1b0 0x55754ee33fc0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x55754ee1b2d0 tx=0x7fc63c011fa0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.332+0000 7fc6538eb740 1 -- 172.21.15.117:0/372571407 shutdown_connections 2024-09-21T06:04:44.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.332+0000 7fc6538eb740 1 --2- 172.21.15.117:0/372571407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55754ea4c1b0 0x55754ee33fc0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.332+0000 7fc6538eb740 1 --2- 172.21.15.117:0/372571407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55754ea54390 0x55754ee34500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.332+0000 7fc6538eb740 1 --2- 172.21.15.117:0/372571407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55754ea339f0 0x55754ee1dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.332+0000 7fc6538eb740 1 -- 172.21.15.117:0/372571407 >> 172.21.15.117:0/372571407 conn(0x55754ea69460 msgr2=0x55754ee1d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:44.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.332+0000 7fc6538eb740 1 -- 172.21.15.117:0/372571407 shutdown_connections 2024-09-21T06:04:44.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.332+0000 7fc6538eb740 1 -- 172.21.15.117:0/372571407 wait complete. 2024-09-21T06:04:44.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.332+0000 7fc6538eb740 1 -- start start 2024-09-21T06:04:44.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.332+0000 7fc6538eb740 1 Processor -- start 2024-09-21T06:04:44.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.333+0000 7fc6538eb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55754ea339f0 0x55754ee2c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:44.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.333+0000 7fc6538eb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55754ea4c1b0 0x55754ee2c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:44.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.333+0000 7fc6538eb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55754ea54390 0x55754ee2cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:44.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.333+0000 7fc6538eb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc6481b2c80 con 0x55754ea339f0 2024-09-21T06:04:44.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.333+0000 7fc6538eb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc6481b25b0 con 0x55754ea54390 2024-09-21T06:04:44.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.333+0000 7fc6538eb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc6481af810 con 0x55754ea4c1b0 2024-09-21T06:04:44.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.333+0000 7fc6538eb740 10 client.0 _ll_register_callbacks cb 0x7fc64000c8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:44.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.333+0000 7fc646ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55754ea339f0 0x55754ee2c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:44.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.333+0000 7fc646ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55754ea339f0 0x55754ee2c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35950/0 (socket says 172.21.15.117:35950) 2024-09-21T06:04:44.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.333+0000 7fc646ffd640 1 -- 172.21.15.117:0/2585619000 learned_addr learned my addr 172.21.15.117:0/2585619000 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:44.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.333+0000 7fc6467fc640 1 --2- 172.21.15.117:0/2585619000 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55754ea4c1b0 0x55754ee2c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:44.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.333+0000 7fc6477fe640 1 --2- 172.21.15.117:0/2585619000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55754ea54390 0x55754ee2cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:44.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.333+0000 7fc646ffd640 1 -- 172.21.15.117:0/2585619000 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55754ea4c1b0 msgr2=0x55754ee2c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:44.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.333+0000 7fc646ffd640 1 --2- 172.21.15.117:0/2585619000 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55754ea4c1b0 0x55754ee2c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.333+0000 7fc646ffd640 1 -- 172.21.15.117:0/2585619000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55754ea54390 msgr2=0x55754ee2cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:44.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.333+0000 7fc646ffd640 1 --2- 172.21.15.117:0/2585619000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55754ea54390 0x55754ee2cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.333+0000 7fc646ffd640 1 -- 172.21.15.117:0/2585619000 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55754eea3bf0 con 0x55754ea339f0 2024-09-21T06:04:44.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.333+0000 7fc646ffd640 1 --2- 172.21.15.117:0/2585619000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55754ea339f0 0x55754ee2c1d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fc638031440 tx=0x7fc638004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b82e706b00f60a0b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:44.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.333+0000 7fc62aff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:04:44.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.334+0000 7fc62aff5640 1 -- 172.21.15.117:0/2585619000 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc638035040 con 0x55754ea339f0 2024-09-21T06:04:44.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.334+0000 7fc62aff5640 1 -- 172.21.15.117:0/2585619000 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7fc638005240 con 0x55754ea339f0 2024-09-21T06:04:44.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.334+0000 7fc6538eb740 1 -- 172.21.15.117:0/2585619000 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc63c017070 con 0x55754ea339f0 2024-09-21T06:04:44.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.334+0000 7fc62aff5640 1 -- 172.21.15.117:0/2585619000 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc638051020 con 0x55754ea339f0 2024-09-21T06:04:44.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.334+0000 7fc6538eb740 10 client.16761 fetch_fsmap learned FSMap version 29 2024-09-21T06:04:44.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.334+0000 7fc6538eb740 1 -- 172.21.15.117:0/2585619000 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55754eea4ec0 con 0x55754ea339f0 2024-09-21T06:04:44.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.334+0000 7fc62aff5640 1 -- 172.21.15.117:0/2585619000 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc63c017070 con 0x55754ea339f0 2024-09-21T06:04:44.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.334+0000 7fc62aff5640 1 -- 172.21.15.117:0/2585619000 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc638003680 con 0x55754ea339f0 2024-09-21T06:04:44.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.334+0000 7fc6538eb740 10 client.16761 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:04:44.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.334+0000 7fc6538eb740 10 client.16761 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:04:44.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.334+0000 7fc6538eb740 20 client.16761 populate_metadata read hostname 'smithi117' 2024-09-21T06:04:44.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.334+0000 7fc6538eb740 1 --2- 172.21.15.117:0/2585619000 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55754ea75070 0x55754eea7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:44.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.334+0000 7fc6538eb740 4 client.16761 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:04:44.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.334+0000 7fc6538eb740 1 -- 172.21.15.117:0/2585619000 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55754ed90850 con 0x55754ea75070 2024-09-21T06:04:44.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.335+0000 7fc6467fc640 1 --2- 172.21.15.117:0/2585619000 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55754ea75070 0x55754eea7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:44.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.335+0000 7fc62aff5640 10 client.16761 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:44.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.335+0000 7fc6467fc640 1 --2- 172.21.15.117:0/2585619000 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55754ea75070 0x55754eea7620 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ad71ace54e68d6 server_cookie=2fdeee866a807ac8 in_seq=0 out_seq=0 2024-09-21T06:04:44.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.335+0000 7fc62aff5640 1 -- 172.21.15.117:0/2585619000 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1794 (crc 0 0 0) 0x55754ed90850 con 0x55754ea75070 2024-09-21T06:04:44.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.335+0000 7fc62aff5640 10 client.16761 handle_command_reply: tid=0 2024-09-21T06:04:44.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.335+0000 7fc6538eb740 1 client.16761 shutdown 2024-09-21T06:04:44.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.336+0000 7fc6538eb740 1 -- 172.21.15.117:0/2585619000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55754ea339f0 msgr2=0x55754ee2c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:44.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.336+0000 7fc6538eb740 1 --2- 172.21.15.117:0/2585619000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55754ea339f0 0x55754ee2c1d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fc638031440 tx=0x7fc638004180 comp rx=0 tx=0).stop 2024-09-21T06:04:44.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.336+0000 7fc6538eb740 1 -- 172.21.15.117:0/2585619000 shutdown_connections 2024-09-21T06:04:44.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.336+0000 7fc6538eb740 1 --2- 172.21.15.117:0/2585619000 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55754ea75070 0x55754eea7620 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.336+0000 7fc62aff5640 0 client.16761 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:44.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.336+0000 7fc6538eb740 1 --2- 172.21.15.117:0/2585619000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55754ea54390 0x55754ee2cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.336+0000 7fc6538eb740 1 --2- 172.21.15.117:0/2585619000 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55754ea4c1b0 0x55754ee2c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.336+0000 7fc6538eb740 1 --2- 172.21.15.117:0/2585619000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55754ea339f0 0x55754ee2c1d0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.336+0000 7fc6538eb740 1 -- 172.21.15.117:0/2585619000 >> 172.21.15.117:0/2585619000 conn(0x55754ea69460 msgr2=0x55754ee1dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:44.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.336+0000 7fc6538eb740 1 -- 172.21.15.117:0/2585619000 shutdown_connections 2024-09-21T06:04:44.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.336+0000 7fc6538eb740 1 -- 172.21.15.117:0/2585619000 wait complete. 2024-09-21T06:04:44.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.336+0000 7fc6538eb740 20 client.16761 trim_cache size 0 max 16384 2024-09-21T06:04:44.352 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T06:04:44.352 INFO:teuthology.orchestra.run.smithi117.stdout: "status": "no active scrubs running", 2024-09-21T06:04:44.352 INFO:teuthology.orchestra.run.smithi117.stdout: "scrubs": {}, 2024-09-21T06:04:44.352 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_stats": { 2024-09-21T06:04:44.352 INFO:teuthology.orchestra.run.smithi117.stdout: "29ff9b7c-49d9-4910-85be-bace7e940cd5": { 2024-09-21T06:04:44.352 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:03:03.775527+0000", 2024-09-21T06:04:44.352 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:04:44.352 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:04:44.352 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:04:44.352 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:04:44.352 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:04:44.353 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:04:44.353 INFO:teuthology.orchestra.run.smithi117.stdout: "0902e33a-ce33-45fb-92d8-2e6fcb8ec8a0": { 2024-09-21T06:04:44.353 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:04:11.365073+0000", 2024-09-21T06:04:44.353 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:04:44.353 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:04:44.353 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:04:44.353 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:04:44.353 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:04:44.353 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:04:44.353 INFO:teuthology.orchestra.run.smithi117.stdout: "d543b0ae-b710-4153-99da-fe5714839c1a": { 2024-09-21T06:04:44.353 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:01:20.574754+0000", 2024-09-21T06:04:44.353 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:04:44.353 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:04:44.353 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:04:44.354 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:04:44.354 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:04:44.354 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:04:44.354 INFO:teuthology.orchestra.run.smithi117.stdout: "ad80d9f1-6aff-46c0-91ab-2c1ab0681039": { 2024-09-21T06:04:44.354 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:59:31.949108+0000", 2024-09-21T06:04:44.354 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:04:44.354 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:04:44.354 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:04:44.354 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:04:44.354 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:04:44.354 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:04:44.354 INFO:teuthology.orchestra.run.smithi117.stdout: "8b3fe95a-b76b-4e94-afd0-72fe35e2dd92": { 2024-09-21T06:04:44.354 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:58:01.246076+0000", 2024-09-21T06:04:44.354 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:04:44.354 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:04:44.355 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:04:44.355 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:04:44.355 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:04:44.355 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:04:44.355 INFO:teuthology.orchestra.run.smithi117.stdout: "b30bfff0-0f6b-4e30-8467-ff7db421eba6": { 2024-09-21T06:04:44.355 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:57:14.939082+0000", 2024-09-21T06:04:44.355 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:04:44.355 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:04:44.355 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:04:44.355 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:04:44.355 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:04:44.355 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:04:44.355 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:04:44.355 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T06:04:44.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.336+0000 7fc64d377640 1 -- 172.21.15.117:0/3407730646 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc6300b6680 msgr2=0x7fc6300b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:44.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.336+0000 7fc64d377640 1 --2- 172.21.15.117:0/3407730646 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc6300b6680 0x7fc6300b8b90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc63c013fd0 tx=0x7fc63c015000 comp rx=0 tx=0).stop 2024-09-21T06:04:44.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.336+0000 7fc64d377640 1 -- 172.21.15.117:0/3407730646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc648108d30 msgr2=0x7fc64819ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:44.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.336+0000 7fc64d377640 1 --2- 172.21.15.117:0/3407730646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc648108d30 0x7fc64819ebd0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fc64000eb10 tx=0x7fc64000c2d0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.337+0000 7fc64d377640 1 -- 172.21.15.117:0/3407730646 shutdown_connections 2024-09-21T06:04:44.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.337+0000 7fc64d377640 1 --2- 172.21.15.117:0/3407730646 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc6300b6680 0x7fc6300b8b90 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.337+0000 7fc64d377640 1 --2- 172.21.15.117:0/3407730646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc648108d30 0x7fc64819ebd0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.356 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.337+0000 7fc64d377640 1 --2- 172.21.15.117:0/3407730646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6481083e0 0x7fc648199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.337+0000 7fc64d377640 1 --2- 172.21.15.117:0/3407730646 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc648102aa0 0x7fc648199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:44.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.337+0000 7fc64d377640 1 -- 172.21.15.117:0/3407730646 >> 172.21.15.117:0/3407730646 conn(0x7fc6480797e0 msgr2=0x7fc6481013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:44.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.337+0000 7fc64d377640 1 -- 172.21.15.117:0/3407730646 shutdown_connections 2024-09-21T06:04:44.357 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:44.337+0000 7fc64d377640 1 -- 172.21.15.117:0/3407730646 wait complete. 2024-09-21T06:04:44.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:43 smithi138 ceph-mon[34391]: pgmap v552: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 8.4 KiB/s wr, 11 op/s 2024-09-21T06:04:45.125 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:04:45.126 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:04:45.380 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:04:45.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.889+0000 7f1a0789f640 1 -- 172.21.15.117:0/3130655713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a001094a0 msgr2=0x7f1a001098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:45.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.889+0000 7f1a0789f640 1 --2- 172.21.15.117:0/3130655713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a001094a0 0x7f1a001098b0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f19f4009900 tx=0x7f19f402a410 comp rx=0 tx=0).stop 2024-09-21T06:04:45.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.889+0000 7f1a0789f640 1 -- 172.21.15.117:0/3130655713 shutdown_connections 2024-09-21T06:04:45.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.889+0000 7f1a0789f640 1 --2- 172.21.15.117:0/3130655713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a001094a0 0x7f1a001098b0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:45.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.889+0000 7f1a0789f640 1 --2- 172.21.15.117:0/3130655713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a00109df0 0x7f1a00110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:45.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.889+0000 7f1a0789f640 1 --2- 172.21.15.117:0/3130655713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a00103b60 0x7f1a00103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:45.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.889+0000 7f1a0789f640 1 -- 172.21.15.117:0/3130655713 >> 172.21.15.117:0/3130655713 conn(0x7f1a0007b850 msgr2=0x7f1a0007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:45.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.889+0000 7f1a0789f640 1 -- 172.21.15.117:0/3130655713 shutdown_connections 2024-09-21T06:04:45.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.889+0000 7f1a0789f640 1 -- 172.21.15.117:0/3130655713 wait complete. 2024-09-21T06:04:45.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.890+0000 7f1a0789f640 1 Processor -- start 2024-09-21T06:04:45.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.890+0000 7f1a0789f640 1 -- start start 2024-09-21T06:04:45.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.890+0000 7f1a0789f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a00103b60 0x7f1a0019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:45.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.890+0000 7f1a0789f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a001094a0 0x7f1a0019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:45.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.890+0000 7f1a0789f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a00109df0 0x7f1a0019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:45.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.890+0000 7f1a0789f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1a00113ab0 con 0x7f1a00103b60 2024-09-21T06:04:45.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.890+0000 7f1a0789f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1a00113930 con 0x7f1a001094a0 2024-09-21T06:04:45.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.890+0000 7f1a0789f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1a00113c30 con 0x7f1a00109df0 2024-09-21T06:04:45.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.890+0000 7f1a0563b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a00103b60 0x7f1a0019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:45.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.890+0000 7f1a0563b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a00103b60 0x7f1a0019a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43404/0 (socket says 172.21.15.117:43404) 2024-09-21T06:04:45.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.890+0000 7f1a0563b640 1 -- 172.21.15.117:0/1365418657 learned_addr learned my addr 172.21.15.117:0/1365418657 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:45.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.891+0000 7f1a05e3c640 1 --2- 172.21.15.117:0/1365418657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a00109df0 0x7f1a0019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:45.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.891+0000 7f1a04e3a640 1 --2- 172.21.15.117:0/1365418657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a001094a0 0x7f1a0019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:45.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.891+0000 7f1a0563b640 1 -- 172.21.15.117:0/1365418657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a00109df0 msgr2=0x7f1a0019fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:45.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.891+0000 7f1a0563b640 1 --2- 172.21.15.117:0/1365418657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a00109df0 0x7f1a0019fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:45.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.891+0000 7f1a0563b640 1 -- 172.21.15.117:0/1365418657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a001094a0 msgr2=0x7f1a0019aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:45.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.891+0000 7f1a0563b640 1 --2- 172.21.15.117:0/1365418657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a001094a0 0x7f1a0019aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:45.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.891+0000 7f1a0563b640 1 -- 172.21.15.117:0/1365418657 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a001a0410 con 0x7f1a00103b60 2024-09-21T06:04:45.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.891+0000 7f1a0563b640 1 --2- 172.21.15.117:0/1365418657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a00103b60 0x7f1a0019a4a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f19fc00d950 tx=0x7f19fc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:45.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.891+0000 7f19f27fc640 1 -- 172.21.15.117:0/1365418657 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19fc017030 con 0x7f1a00103b60 2024-09-21T06:04:45.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.891+0000 7f19f27fc640 1 -- 172.21.15.117:0/1365418657 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19fc004020 con 0x7f1a00103b60 2024-09-21T06:04:45.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.891+0000 7f1a0789f640 1 -- 172.21.15.117:0/1365418657 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a001a0700 con 0x7f1a00103b60 2024-09-21T06:04:45.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.891+0000 7f19f27fc640 1 -- 172.21.15.117:0/1365418657 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19fc0047a0 con 0x7f1a00103b60 2024-09-21T06:04:45.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.891+0000 7f1a0789f640 1 -- 172.21.15.117:0/1365418657 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a001a6e60 con 0x7f1a00103b60 2024-09-21T06:04:45.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.892+0000 7f1a0789f640 1 -- 172.21.15.117:0/1365418657 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a00103f70 con 0x7f1a00103b60 2024-09-21T06:04:45.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.894+0000 7f19f27fc640 1 -- 172.21.15.117:0/1365418657 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f19fc009a30 con 0x7f1a00103b60 2024-09-21T06:04:45.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.894+0000 7f19f27fc640 1 --2- 172.21.15.117:0/1365418657 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f19ec0b6700 0x7f19ec0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:45.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.894+0000 7f1a04e3a640 1 --2- 172.21.15.117:0/1365418657 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f19ec0b6700 0x7f19ec0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:45.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.895+0000 7f19f27fc640 1 -- 172.21.15.117:0/1365418657 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f19fc0e7220 con 0x7f1a00103b60 2024-09-21T06:04:45.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.895+0000 7f1a04e3a640 1 --2- 172.21.15.117:0/1365418657 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f19ec0b6700 0x7f19ec0b8c10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f19f4009230 tx=0x7f19f4031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:45.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:45.899+0000 7f19f27fc640 1 -- 172.21.15.117:0/1365418657 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f19fc0ad7c0 con 0x7f1a00103b60 2024-09-21T06:04:46.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:46.115+0000 7f1a0789f640 1 -- 172.21.15.117:0/1365418657 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a0002e640 con 0x7f1a00103b60 2024-09-21T06:04:46.117 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:45 smithi117 ceph-mon[30947]: pgmap v553: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-09-21T06:04:46.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:46.116+0000 7f19f27fc640 1 -- 172.21.15.117:0/1365418657 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+12001 (secure 0 0 0) 0x7f19fc0b1ba0 con 0x7f1a00103b60 2024-09-21T06:04:46.118 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T06:04:46.118 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":29,"btime":"2024-09-21T06:04:08:952483+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3227057504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3227057504},{"type":"v1","addr":"172.21.15.117:6835","nonce":3227057504}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3303829183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3303829183},{"type":"v1","addr":"172.21.15.117:6837","nonce":3303829183}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T05:55:14.555897+0000","modified":"2024-09-21T06:04:08.952466+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24638,"mds_1":24637,"mds_2":24628,"mds_3":24616,"mds_4":24602},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6839/2314572411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2314572411},{"type":"v1","addr":"172.21.15.117:6839","nonce":2314572411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6841/3378096561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":3378096561},{"type":"v1","addr":"172.21.15.117:6841","nonce":3378096561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/3725629862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3725629862},{"type":"v1","addr":"172.21.15.138:6833","nonce":3725629862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/4276801396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4276801396},{"type":"v1","addr":"172.21.15.138:6835","nonce":4276801396}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.138:6837/2540335374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2540335374},{"type":"v1","addr":"172.21.15.138:6837","nonce":2540335374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24616":{"gid":24616,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.192:6835/3530688860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3530688860},{"type":"v1","addr":"172.21.15.192:6835","nonce":3530688860}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.192:6837/2683860842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2683860842},{"type":"v1","addr":"172.21.15.192:6837","nonce":2683860842}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.192:6839/1727623231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1727623231},{"type":"v1","addr":"172.21.15.192:6839","nonce":1727623231}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/3230372576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":3230372576},{"type":"v1","addr":"172.21.15.138:6839","nonce":3230372576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25840":{"gid":25840,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6833/1596605058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1596605058},{"type":"v1","addr":"172.21.15.192:6833","nonce":1596605058}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24602,24628,24616,24637,24638]},"id":1}]} 2024-09-21T06:04:46.119 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 29 2024-09-21T06:04:46.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:46.118+0000 7f1a0789f640 1 -- 172.21.15.117:0/1365418657 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f19ec0b6700 msgr2=0x7f19ec0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:46.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:46.118+0000 7f1a0789f640 1 --2- 172.21.15.117:0/1365418657 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f19ec0b6700 0x7f19ec0b8c10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f19f4009230 tx=0x7f19f4031400 comp rx=0 tx=0).stop 2024-09-21T06:04:46.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:46.118+0000 7f1a0789f640 1 -- 172.21.15.117:0/1365418657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a00103b60 msgr2=0x7f1a0019a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:46.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:46.118+0000 7f1a0789f640 1 --2- 172.21.15.117:0/1365418657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a00103b60 0x7f1a0019a4a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f19fc00d950 tx=0x7f19fc00de10 comp rx=0 tx=0).stop 2024-09-21T06:04:46.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:46.118+0000 7f1a0789f640 1 -- 172.21.15.117:0/1365418657 shutdown_connections 2024-09-21T06:04:46.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:46.118+0000 7f1a0789f640 1 --2- 172.21.15.117:0/1365418657 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f19ec0b6700 0x7f19ec0b8c10 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:46.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:46.118+0000 7f1a0789f640 1 --2- 172.21.15.117:0/1365418657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a00109df0 0x7f1a0019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:46.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:46.118+0000 7f1a0789f640 1 --2- 172.21.15.117:0/1365418657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a001094a0 0x7f1a0019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:46.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:46.118+0000 7f1a0789f640 1 --2- 172.21.15.117:0/1365418657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a00103b60 0x7f1a0019a4a0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:46.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:46.118+0000 7f1a0789f640 1 -- 172.21.15.117:0/1365418657 >> 172.21.15.117:0/1365418657 conn(0x7f1a0007b850 msgr2=0x7f1a00105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:46.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:46.118+0000 7f1a0789f640 1 -- 172.21.15.117:0/1365418657 shutdown_connections 2024-09-21T06:04:46.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:46.118+0000 7f1a0789f640 1 -- 172.21.15.117:0/1365418657 wait complete. 2024-09-21T06:04:46.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:45 smithi192 ceph-mon[34700]: pgmap v553: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-09-21T06:04:46.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:45 smithi138 ceph-mon[34391]: pgmap v553: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-09-21T06:04:46.818 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T06:04:47.068 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:04:47.189 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:46 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/1365418657' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:04:47.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:46 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/1365418657' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:04:47.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:46 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/1365418657' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:04:47.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.591+0000 7f006d5d6640 1 -- 172.21.15.117:0/26068248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00681094c0 msgr2=0x7f00681098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:47.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.591+0000 7f006d5d6640 1 --2- 172.21.15.117:0/26068248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00681094c0 0x7f00681098d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f0050009900 tx=0x7f005002a410 comp rx=0 tx=0).stop 2024-09-21T06:04:47.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.591+0000 7f006d5d6640 1 -- 172.21.15.117:0/26068248 shutdown_connections 2024-09-21T06:04:47.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.591+0000 7f006d5d6640 1 --2- 172.21.15.117:0/26068248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00681094c0 0x7f00681098d0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.591+0000 7f006d5d6640 1 --2- 172.21.15.117:0/26068248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0068109e10 0x7f0068110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.591+0000 7f006d5d6640 1 --2- 172.21.15.117:0/26068248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0068103b80 0x7f0068103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.591+0000 7f006d5d6640 1 -- 172.21.15.117:0/26068248 >> 172.21.15.117:0/26068248 conn(0x7f00680ff5d0 msgr2=0x7f0068101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:47.604 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.591+0000 7f006d5d6640 1 -- 172.21.15.117:0/26068248 shutdown_connections 2024-09-21T06:04:47.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.591+0000 7f006d5d6640 1 -- 172.21.15.117:0/26068248 wait complete. 2024-09-21T06:04:47.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.591+0000 7f006d5d6640 1 Processor -- start 2024-09-21T06:04:47.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.592+0000 7f006d5d6640 1 -- start start 2024-09-21T06:04:47.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.592+0000 7f006d5d6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0068103b80 0x7f006819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.592+0000 7f006d5d6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00681094c0 0x7f006819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.592+0000 7f006d5d6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0068109e10 0x7f006819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.592+0000 7f006d5d6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0068113ad0 con 0x7f0068103b80 2024-09-21T06:04:47.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.592+0000 7f006d5d6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0068113950 con 0x7f00681094c0 2024-09-21T06:04:47.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.592+0000 7f006d5d6640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0068113c50 con 0x7f0068109e10 2024-09-21T06:04:47.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.592+0000 7f0066ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0068103b80 0x7f006819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.592+0000 7f0066ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0068103b80 0x7f006819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43428/0 (socket says 172.21.15.117:43428) 2024-09-21T06:04:47.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.592+0000 7f0066ffd640 1 -- 172.21.15.117:0/2375612654 learned_addr learned my addr 172.21.15.117:0/2375612654 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:47.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.592+0000 7f00667fc640 1 --2- 172.21.15.117:0/2375612654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00681094c0 0x7f006819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.592+0000 7f0066ffd640 1 -- 172.21.15.117:0/2375612654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0068109e10 msgr2=0x7f006819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:47.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.592+0000 7f00677fe640 1 --2- 172.21.15.117:0/2375612654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0068109e10 0x7f006819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.605 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.592+0000 7f0066ffd640 1 --2- 172.21.15.117:0/2375612654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0068109e10 0x7f006819fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.592+0000 7f0066ffd640 1 -- 172.21.15.117:0/2375612654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00681094c0 msgr2=0x7f006819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:47.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.592+0000 7f0066ffd640 1 --2- 172.21.15.117:0/2375612654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00681094c0 0x7f006819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.592+0000 7f0066ffd640 1 -- 172.21.15.117:0/2375612654 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00681a0410 con 0x7f0068103b80 2024-09-21T06:04:47.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.592+0000 7f00667fc640 1 --2- 172.21.15.117:0/2375612654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00681094c0 0x7f006819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:47.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.593+0000 7f0066ffd640 1 --2- 172.21.15.117:0/2375612654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0068103b80 0x7f006819a4a0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f005800d770 tx=0x7f005800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:47.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.593+0000 7f005ffff640 1 -- 172.21.15.117:0/2375612654 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0058017070 con 0x7f0068103b80 2024-09-21T06:04:47.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.593+0000 7f006d5d6640 1 -- 172.21.15.117:0/2375612654 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00681a0700 con 0x7f0068103b80 2024-09-21T06:04:47.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.593+0000 7f005ffff640 1 -- 172.21.15.117:0/2375612654 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00580054d0 con 0x7f0068103b80 2024-09-21T06:04:47.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.593+0000 7f005ffff640 1 -- 172.21.15.117:0/2375612654 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0058004c30 con 0x7f0068103b80 2024-09-21T06:04:47.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.593+0000 7f006d5d6640 1 -- 172.21.15.117:0/2375612654 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00681a6e60 con 0x7f0068103b80 2024-09-21T06:04:47.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.596+0000 7f0073b4a740 1 Processor -- start 2024-09-21T06:04:47.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.596+0000 7f005ffff640 1 -- 172.21.15.117:0/2375612654 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f0058004e10 con 0x7f0068103b80 2024-09-21T06:04:47.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.596+0000 7f0073b4a740 1 -- start start 2024-09-21T06:04:47.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.596+0000 7f0073b4a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f309d2070 0x556f30d792c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.596+0000 7f0073b4a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f309a91b0 0x556f30d79800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.596+0000 7f0073b4a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309b1390 0x556f30d79d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.596+0000 7f0073b4a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556f30cefea0 con 0x556f309d2070 2024-09-21T06:04:47.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.596+0000 7f0073b4a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556f30cdf2d0 con 0x556f309a91b0 2024-09-21T06:04:47.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.596+0000 7f0073b4a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556f30ce30e0 con 0x556f309b1390 2024-09-21T06:04:47.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.596+0000 7f00677fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f309d2070 0x556f30d792c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.596+0000 7f00677fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f309d2070 0x556f30d792c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43430/0 (socket says 172.21.15.117:43430) 2024-09-21T06:04:47.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.596+0000 7f00677fe640 1 -- 172.21.15.117:0/3261950117 learned_addr learned my addr 172.21.15.117:0/3261950117 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:47.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.597+0000 7f00667fc640 1 --2- 172.21.15.117:0/3261950117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309b1390 0x556f30d79d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.597+0000 7f0066ffd640 1 --2- 172.21.15.117:0/3261950117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f309a91b0 0x556f30d79800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.597+0000 7f00677fe640 1 -- 172.21.15.117:0/3261950117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309b1390 msgr2=0x556f30d79d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:47.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.597+0000 7f00677fe640 1 --2- 172.21.15.117:0/3261950117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309b1390 0x556f30d79d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.607 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.597+0000 7f00677fe640 1 -- 172.21.15.117:0/3261950117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f309a91b0 msgr2=0x556f30d79800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:47.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.597+0000 7f00677fe640 1 --2- 172.21.15.117:0/3261950117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f309a91b0 0x556f30d79800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.597+0000 7f00677fe640 1 -- 172.21.15.117:0/3261950117 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f30a3d650 con 0x556f309d2070 2024-09-21T06:04:47.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.597+0000 7f0066ffd640 1 --2- 172.21.15.117:0/3261950117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f309a91b0 0x556f30d79800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:47.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.597+0000 7f005ffff640 1 --2- 172.21.15.117:0/2375612654 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f00540b6680 0x7f00540b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.597+0000 7f0066ffd640 1 --2- 172.21.15.117:0/2375612654 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f00540b6680 0x7f00540b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.597+0000 7f00677fe640 1 --2- 172.21.15.117:0/3261950117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f309d2070 0x556f30d792c0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f0060009df0 tx=0x7f006000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d540e58cda3cecfe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:47.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.597+0000 7f005ffff640 1 -- 172.21.15.117:0/2375612654 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f00580e7290 con 0x7f0068103b80 2024-09-21T06:04:47.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.597+0000 7f005d7fa640 1 -- 172.21.15.117:0/3261950117 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0060003970 con 0x556f309d2070 2024-09-21T06:04:47.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.597+0000 7f005d7fa640 1 -- 172.21.15.117:0/3261950117 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0060004030 con 0x556f309d2070 2024-09-21T06:04:47.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.597+0000 7f005d7fa640 1 -- 172.21.15.117:0/3261950117 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0060011770 con 0x556f309d2070 2024-09-21T06:04:47.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.598+0000 7f0073b4a740 1 -- 172.21.15.117:0/3261950117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f309d2070 msgr2=0x556f30d792c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:47.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.598+0000 7f0066ffd640 1 --2- 172.21.15.117:0/2375612654 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f00540b6680 0x7f00540b8b90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f00580e4ae0 tx=0x7f0058009f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:47.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.598+0000 7f0073b4a740 1 --2- 172.21.15.117:0/3261950117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f309d2070 0x556f30d792c0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f0060009df0 tx=0x7f006000f730 comp rx=0 tx=0).stop 2024-09-21T06:04:47.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.598+0000 7f0073b4a740 1 -- 172.21.15.117:0/3261950117 shutdown_connections 2024-09-21T06:04:47.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.598+0000 7f0073b4a740 1 --2- 172.21.15.117:0/3261950117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309b1390 0x556f30d79d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.598+0000 7f0073b4a740 1 --2- 172.21.15.117:0/3261950117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f309a91b0 0x556f30d79800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.598+0000 7f0073b4a740 1 --2- 172.21.15.117:0/3261950117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f309d2070 0x556f30d792c0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.598+0000 7f0073b4a740 1 -- 172.21.15.117:0/3261950117 >> 172.21.15.117:0/3261950117 conn(0x556f309c6460 msgr2=0x556f30d771b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:47.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.598+0000 7f0073b4a740 1 -- 172.21.15.117:0/3261950117 shutdown_connections 2024-09-21T06:04:47.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.599+0000 7f0073b4a740 1 -- 172.21.15.117:0/3261950117 wait complete. 2024-09-21T06:04:47.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.599+0000 7f0073b4a740 1 -- start start 2024-09-21T06:04:47.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.599+0000 7f0073b4a740 1 Processor -- start 2024-09-21T06:04:47.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.599+0000 7f0073b4a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f309a91b0 0x556f30d84970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.599+0000 7f0073b4a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f309b1390 0x556f30d84eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.599+0000 7f0073b4a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309d2070 0x556f30d90d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.599+0000 7f0073b4a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556f30b40700 con 0x556f309b1390 2024-09-21T06:04:47.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.599+0000 7f0073b4a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556f30c7a160 con 0x556f309a91b0 2024-09-21T06:04:47.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.599+0000 7f0073b4a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556f30d87aa0 con 0x556f309d2070 2024-09-21T06:04:47.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.599+0000 7f0073b4a740 10 client.0 _ll_register_callbacks cb 0x556f30c939e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:47.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.599+0000 7f00667fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f309b1390 0x556f30d84eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.600+0000 7f00667fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f309b1390 0x556f30d84eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43444/0 (socket says 172.21.15.117:43444) 2024-09-21T06:04:47.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.600+0000 7f00667fc640 1 -- 172.21.15.117:0/3974292330 learned_addr learned my addr 172.21.15.117:0/3974292330 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:47.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.600+0000 7f00677fe640 1 --2- 172.21.15.117:0/3974292330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f309a91b0 0x556f30d84970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.600+0000 7f00677fe640 1 --2- 172.21.15.117:0/3974292330 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309d2070 0x556f30d90d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.600+0000 7f00667fc640 1 -- 172.21.15.117:0/3974292330 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309d2070 msgr2=0x556f30d90d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:47.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.600+0000 7f00667fc640 1 --2- 172.21.15.117:0/3974292330 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309d2070 0x556f30d90d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.600+0000 7f00667fc640 1 -- 172.21.15.117:0/3974292330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f309a91b0 msgr2=0x556f30d84970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:47.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.600+0000 7f00667fc640 1 --2- 172.21.15.117:0/3974292330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f309a91b0 0x556f30d84970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.600+0000 7f00667fc640 1 -- 172.21.15.117:0/3974292330 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f30ca7c90 con 0x556f309b1390 2024-09-21T06:04:47.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.600+0000 7f00677fe640 1 --2- 172.21.15.117:0/3974292330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f309a91b0 0x556f30d84970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:04:47.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.600+0000 7f00667fc640 1 --2- 172.21.15.117:0/3974292330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f309b1390 0x556f30d84eb0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f0050005ab0 tx=0x7f0050009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef27851e5a90eb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:47.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.600+0000 7f004e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:04:47.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.600+0000 7f004e7fc640 1 -- 172.21.15.117:0/3974292330 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0050040070 con 0x556f309b1390 2024-09-21T06:04:47.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.600+0000 7f004e7fc640 1 -- 172.21.15.117:0/3974292330 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f0050004240 con 0x556f309b1390 2024-09-21T06:04:47.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.600+0000 7f0073b4a740 1 -- 172.21.15.117:0/3974292330 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0060004030 con 0x556f309b1390 2024-09-21T06:04:47.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.601+0000 7f004e7fc640 1 -- 172.21.15.117:0/3974292330 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0050035040 con 0x556f309b1390 2024-09-21T06:04:47.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.601+0000 7f0073b4a740 10 client.16779 fetch_fsmap learned FSMap version 29 2024-09-21T06:04:47.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.601+0000 7f0073b4a740 1 -- 172.21.15.117:0/3974292330 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556f30d936c0 con 0x556f309b1390 2024-09-21T06:04:47.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.601+0000 7f004e7fc640 1 -- 172.21.15.117:0/3974292330 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0060004030 con 0x556f309b1390 2024-09-21T06:04:47.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.601+0000 7f004e7fc640 1 -- 172.21.15.117:0/3974292330 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f00500049e0 con 0x556f309b1390 2024-09-21T06:04:47.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.601+0000 7f0073b4a740 10 client.16779 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:04:47.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.601+0000 7f0073b4a740 10 client.16779 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T06:04:47.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.601+0000 7f0073b4a740 20 client.16779 populate_metadata read hostname 'smithi117' 2024-09-21T06:04:47.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.601+0000 7f0073b4a740 1 --2- 172.21.15.117:0/3974292330 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x556f309909f0 0x556f30d96b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.601+0000 7f0073b4a740 4 client.16779 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:04:47.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.601+0000 7f0073b4a740 1 -- 172.21.15.117:0/3974292330 --> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f30cf1df0 con 0x556f309909f0 2024-09-21T06:04:47.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.602+0000 7f00677fe640 1 --2- 172.21.15.117:0/3974292330 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x556f309909f0 0x556f30d96b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.602+0000 7f004e7fc640 10 client.16779 ms_handle_connect on v2:172.21.15.138:6836/2540335374 2024-09-21T06:04:47.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.602+0000 7f00677fe640 1 --2- 172.21.15.117:0/3974292330 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x556f309909f0 0x556f30d96b90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9ac15facafdd994b server_cookie=bd91dde5f0b9a620 in_seq=0 out_seq=0 2024-09-21T06:04:47.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.603+0000 7f004e7fc640 1 -- 172.21.15.117:0/3974292330 <== mds.4 v2:172.21.15.138:6836/2540335374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556f30cf1df0 con 0x556f309909f0 2024-09-21T06:04:47.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.603+0000 7f004e7fc640 10 client.16779 handle_command_reply: tid=0 2024-09-21T06:04:47.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.603+0000 7f0073b4a740 1 client.16779 shutdown 2024-09-21T06:04:47.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.603+0000 7f0073b4a740 1 -- 172.21.15.117:0/3974292330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f309b1390 msgr2=0x556f30d84eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:47.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.603+0000 7f0073b4a740 1 --2- 172.21.15.117:0/3974292330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f309b1390 0x556f30d84eb0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f0050005ab0 tx=0x7f0050009880 comp rx=0 tx=0).stop 2024-09-21T06:04:47.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.603+0000 7f0073b4a740 1 -- 172.21.15.117:0/3974292330 shutdown_connections 2024-09-21T06:04:47.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.603+0000 7f0073b4a740 1 --2- 172.21.15.117:0/3974292330 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x556f309909f0 0x556f30d96b90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.603+0000 7f0073b4a740 1 --2- 172.21.15.117:0/3974292330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f309b1390 0x556f30d84eb0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.603+0000 7f0073b4a740 1 --2- 172.21.15.117:0/3974292330 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309d2070 0x556f30d90d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.603+0000 7f0073b4a740 1 --2- 172.21.15.117:0/3974292330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f309a91b0 0x556f30d84970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.603+0000 7f0073b4a740 1 -- 172.21.15.117:0/3974292330 >> 172.21.15.117:0/3974292330 conn(0x556f309c6460 msgr2=0x556f30d7c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:47.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.603+0000 7f004e7fc640 0 client.16779 ms_handle_reset on v2:172.21.15.138:6836/2540335374 2024-09-21T06:04:47.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.603+0000 7f0073b4a740 1 -- 172.21.15.117:0/3974292330 shutdown_connections 2024-09-21T06:04:47.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.603+0000 7f0073b4a740 1 -- 172.21.15.117:0/3974292330 wait complete. 2024-09-21T06:04:47.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.603+0000 7f0073b4a740 20 client.16779 trim_cache size 0 max 16384 2024-09-21T06:04:47.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.617+0000 7f0073b4a740 1 Processor -- start 2024-09-21T06:04:47.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.617+0000 7f0073b4a740 1 -- start start 2024-09-21T06:04:47.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.617+0000 7f0073b4a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f309a91b0 0x556f30d86850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.617+0000 7f0073b4a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f309b1390 0x556f30d7aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.617+0000 7f0073b4a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309d2070 0x556f30d90c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.617+0000 7f0073b4a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f00681af700 con 0x556f309a91b0 2024-09-21T06:04:47.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.617+0000 7f0073b4a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f00681b1240 con 0x556f309b1390 2024-09-21T06:04:47.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.617+0000 7f0073b4a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f00681afdd0 con 0x556f309d2070 2024-09-21T06:04:47.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.617+0000 7f00677fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f309a91b0 0x556f30d86850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.618+0000 7f00677fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f309a91b0 0x556f30d86850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43456/0 (socket says 172.21.15.117:43456) 2024-09-21T06:04:47.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.618+0000 7f00677fe640 1 -- 172.21.15.117:0/4141260744 learned_addr learned my addr 172.21.15.117:0/4141260744 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:47.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.618+0000 7f00667fc640 1 --2- 172.21.15.117:0/4141260744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f309b1390 0x556f30d7aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.618+0000 7f00677fe640 1 --2- 172.21.15.117:0/4141260744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309d2070 0x556f30d90c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.618+0000 7f00677fe640 1 -- 172.21.15.117:0/4141260744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309d2070 msgr2=0x556f30d90c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:47.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.618+0000 7f00677fe640 1 --2- 172.21.15.117:0/4141260744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309d2070 0x556f30d90c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.618+0000 7f00677fe640 1 -- 172.21.15.117:0/4141260744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f309b1390 msgr2=0x556f30d7aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:47.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.618+0000 7f00677fe640 1 --2- 172.21.15.117:0/4141260744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f309b1390 0x556f30d7aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.618+0000 7f00677fe640 1 -- 172.21.15.117:0/4141260744 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f30d7b8a0 con 0x556f309a91b0 2024-09-21T06:04:47.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.618+0000 7f00677fe640 1 --2- 172.21.15.117:0/4141260744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f309a91b0 0x556f30d86850 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f0060000f80 tx=0x7f00600096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4094a5643b9b43bf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:47.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.618+0000 7f005cff9640 1 -- 172.21.15.117:0/4141260744 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0060021030 con 0x556f309a91b0 2024-09-21T06:04:47.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.618+0000 7f005cff9640 1 -- 172.21.15.117:0/4141260744 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00600044e0 con 0x556f309a91b0 2024-09-21T06:04:47.623 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.618+0000 7f005cff9640 1 -- 172.21.15.117:0/4141260744 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0060011770 con 0x556f309a91b0 2024-09-21T06:04:47.628 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.619+0000 7f0073b4a740 1 -- 172.21.15.117:0/4141260744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f309a91b0 msgr2=0x556f30d86850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:47.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.619+0000 7f0073b4a740 1 --2- 172.21.15.117:0/4141260744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f309a91b0 0x556f30d86850 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f0060000f80 tx=0x7f00600096c0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.619+0000 7f0073b4a740 1 -- 172.21.15.117:0/4141260744 shutdown_connections 2024-09-21T06:04:47.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.619+0000 7f0073b4a740 1 --2- 172.21.15.117:0/4141260744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309d2070 0x556f30d90c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.619+0000 7f0073b4a740 1 --2- 172.21.15.117:0/4141260744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f309b1390 0x556f30d7aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.619+0000 7f0073b4a740 1 --2- 172.21.15.117:0/4141260744 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f309a91b0 0x556f30d86850 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.619+0000 7f0073b4a740 1 -- 172.21.15.117:0/4141260744 >> 172.21.15.117:0/4141260744 conn(0x556f309c6460 msgr2=0x556f30d7a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:47.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.619+0000 7f0073b4a740 1 -- 172.21.15.117:0/4141260744 shutdown_connections 2024-09-21T06:04:47.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.619+0000 7f0073b4a740 1 -- 172.21.15.117:0/4141260744 wait complete. 2024-09-21T06:04:47.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.620+0000 7f0073b4a740 1 -- start start 2024-09-21T06:04:47.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.620+0000 7f0073b4a740 1 Processor -- start 2024-09-21T06:04:47.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.620+0000 7f0073b4a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f309a91b0 0x556f30d88d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.620+0000 7f0073b4a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309b1390 0x556f30d89280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.620+0000 7f0073b4a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f309d2070 0x556f30d897c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.620+0000 7f0073b4a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f00681b0b70 con 0x556f309d2070 2024-09-21T06:04:47.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.620+0000 7f0073b4a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f00681b04a0 con 0x556f309a91b0 2024-09-21T06:04:47.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.620+0000 7f0073b4a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f00681ad700 con 0x556f309b1390 2024-09-21T06:04:47.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.620+0000 7f0073b4a740 10 client.0 _ll_register_callbacks cb 0x7f0050005db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:47.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.620+0000 7f00677fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f309d2070 0x556f30d897c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.620+0000 7f00677fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f309d2070 0x556f30d897c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43458/0 (socket says 172.21.15.117:43458) 2024-09-21T06:04:47.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.620+0000 7f00677fe640 1 -- 172.21.15.117:0/1592019237 learned_addr learned my addr 172.21.15.117:0/1592019237 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:47.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.621+0000 7f00677fe640 1 --2- 172.21.15.117:0/1592019237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f309a91b0 0x556f30d88d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.621+0000 7f00667fc640 1 --2- 172.21.15.117:0/1592019237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309b1390 0x556f30d89280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.621+0000 7f00677fe640 1 -- 172.21.15.117:0/1592019237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309b1390 msgr2=0x556f30d89280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:47.632 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.621+0000 7f00677fe640 1 --2- 172.21.15.117:0/1592019237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309b1390 0x556f30d89280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.621+0000 7f00677fe640 1 -- 172.21.15.117:0/1592019237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f309a91b0 msgr2=0x556f30d88d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:47.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.621+0000 7f00677fe640 1 --2- 172.21.15.117:0/1592019237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f309a91b0 0x556f30d88d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.621+0000 7f00677fe640 1 -- 172.21.15.117:0/1592019237 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f30e005d0 con 0x556f309d2070 2024-09-21T06:04:47.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.621+0000 7f00667fc640 1 --2- 172.21.15.117:0/1592019237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309b1390 0x556f30d89280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:47.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.621+0000 7f00677fe640 1 --2- 172.21.15.117:0/1592019237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f309d2070 0x556f30d897c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f006000f770 tx=0x7f0060003dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df820d31285b9a90 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:47.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.621+0000 7f004e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:04:47.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.621+0000 7f004e7fc640 1 -- 172.21.15.117:0/1592019237 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006003a060 con 0x556f309d2070 2024-09-21T06:04:47.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.621+0000 7f004e7fc640 1 -- 172.21.15.117:0/1592019237 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f0060004280 con 0x556f309d2070 2024-09-21T06:04:47.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.621+0000 7f0073b4a740 1 -- 172.21.15.117:0/1592019237 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00600044e0 con 0x556f309d2070 2024-09-21T06:04:47.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.621+0000 7f004e7fc640 1 -- 172.21.15.117:0/1592019237 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0060021040 con 0x556f309d2070 2024-09-21T06:04:47.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.621+0000 7f0073b4a740 10 client.16785 fetch_fsmap learned FSMap version 29 2024-09-21T06:04:47.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.621+0000 7f0073b4a740 1 -- 172.21.15.117:0/1592019237 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556f30e01850 con 0x556f309d2070 2024-09-21T06:04:47.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.622+0000 7f004e7fc640 1 -- 172.21.15.117:0/1592019237 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00600044e0 con 0x556f309d2070 2024-09-21T06:04:47.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.622+0000 7f004e7fc640 1 -- 172.21.15.117:0/1592019237 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f00600326b0 con 0x556f309d2070 2024-09-21T06:04:47.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.622+0000 7f0073b4a740 10 client.16785 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:04:47.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.622+0000 7f0073b4a740 10 client.16785 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T06:04:47.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.622+0000 7f0073b4a740 20 client.16785 populate_metadata read hostname 'smithi117' 2024-09-21T06:04:47.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.622+0000 7f0073b4a740 1 --2- 172.21.15.117:0/1592019237 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x556f30965da0 0x556f30e03d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.622+0000 7f0073b4a740 4 client.16785 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:04:47.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.622+0000 7f0073b4a740 1 -- 172.21.15.117:0/1592019237 --> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556f30ced8a0 con 0x556f30965da0 2024-09-21T06:04:47.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.622+0000 7f00667fc640 1 --2- 172.21.15.117:0/1592019237 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x556f30965da0 0x556f30e03d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.622+0000 7f004e7fc640 10 client.16785 ms_handle_connect on v2:172.21.15.138:6836/2540335374 2024-09-21T06:04:47.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.623+0000 7f00667fc640 1 --2- 172.21.15.117:0/1592019237 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x556f30965da0 0x556f30e03d30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=12a43902eca558a4 server_cookie=d07e165049e10b1f in_seq=0 out_seq=0 2024-09-21T06:04:47.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.623+0000 7f004e7fc640 1 -- 172.21.15.117:0/1592019237 <== mds.4 v2:172.21.15.138:6836/2540335374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556f30ced8a0 con 0x556f30965da0 2024-09-21T06:04:47.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.623+0000 7f004e7fc640 10 client.16785 handle_command_reply: tid=0 2024-09-21T06:04:47.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.623+0000 7f0073b4a740 1 client.16785 shutdown 2024-09-21T06:04:47.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.623+0000 7f0073b4a740 1 -- 172.21.15.117:0/1592019237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f309d2070 msgr2=0x556f30d897c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:47.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.623+0000 7f0073b4a740 1 --2- 172.21.15.117:0/1592019237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f309d2070 0x556f30d897c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f006000f770 tx=0x7f0060003dd0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.623+0000 7f0073b4a740 1 -- 172.21.15.117:0/1592019237 shutdown_connections 2024-09-21T06:04:47.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.623+0000 7f0073b4a740 1 --2- 172.21.15.117:0/1592019237 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x556f30965da0 0x556f30e03d30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.623+0000 7f0073b4a740 1 --2- 172.21.15.117:0/1592019237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556f309d2070 0x556f30d897c0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.623+0000 7f004e7fc640 0 client.16785 ms_handle_reset on v2:172.21.15.138:6836/2540335374 2024-09-21T06:04:47.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.623+0000 7f0073b4a740 1 --2- 172.21.15.117:0/1592019237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309b1390 0x556f30d89280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.623+0000 7f0073b4a740 1 --2- 172.21.15.117:0/1592019237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556f309a91b0 0x556f30d88d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.623+0000 7f0073b4a740 1 -- 172.21.15.117:0/1592019237 >> 172.21.15.117:0/1592019237 conn(0x556f309c6460 msgr2=0x556f30d865f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:47.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.623+0000 7f0073b4a740 1 -- 172.21.15.117:0/1592019237 shutdown_connections 2024-09-21T06:04:47.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.623+0000 7f0073b4a740 1 -- 172.21.15.117:0/1592019237 wait complete. 2024-09-21T06:04:47.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.623+0000 7f0073b4a740 20 client.16785 trim_cache size 0 max 16384 2024-09-21T06:04:47.637 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:04:47.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.624+0000 7f006d5d6640 1 -- 172.21.15.117:0/2375612654 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f00540b6680 msgr2=0x7f00540b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:47.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.624+0000 7f006d5d6640 1 --2- 172.21.15.117:0/2375612654 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f00540b6680 0x7f00540b8b90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f00580e4ae0 tx=0x7f0058009f90 comp rx=0 tx=0).stop 2024-09-21T06:04:47.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.624+0000 7f006d5d6640 1 -- 172.21.15.117:0/2375612654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0068103b80 msgr2=0x7f006819a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:47.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.624+0000 7f006d5d6640 1 --2- 172.21.15.117:0/2375612654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0068103b80 0x7f006819a4a0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f005800d770 tx=0x7f005800dc30 comp rx=0 tx=0).stop 2024-09-21T06:04:47.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.624+0000 7f006d5d6640 1 -- 172.21.15.117:0/2375612654 shutdown_connections 2024-09-21T06:04:47.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.624+0000 7f006d5d6640 1 --2- 172.21.15.117:0/2375612654 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f00540b6680 0x7f00540b8b90 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.624+0000 7f006d5d6640 1 --2- 172.21.15.117:0/2375612654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0068109e10 0x7f006819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.624+0000 7f006d5d6640 1 --2- 172.21.15.117:0/2375612654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00681094c0 0x7f006819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.624+0000 7f006d5d6640 1 --2- 172.21.15.117:0/2375612654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0068103b80 0x7f006819a4a0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.624+0000 7f006d5d6640 1 -- 172.21.15.117:0/2375612654 >> 172.21.15.117:0/2375612654 conn(0x7f00680ff5d0 msgr2=0x7f00681019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:47.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.624+0000 7f006d5d6640 1 -- 172.21.15.117:0/2375612654 shutdown_connections 2024-09-21T06:04:47.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:47.624+0000 7f006d5d6640 1 -- 172.21.15.117:0/2375612654 wait complete. 2024-09-21T06:04:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.660+0000 7f9f6d112640 1 -- 172.21.15.117:0/2501693526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f68108310 msgr2=0x7f9f68108720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.660+0000 7f9f6d112640 1 --2- 172.21.15.117:0/2501693526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f68108310 0x7f9f68108720 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c009900 tx=0x7f9f5c02a410 comp rx=0 tx=0).stop 2024-09-21T06:04:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.660+0000 7f9f6d112640 1 -- 172.21.15.117:0/2501693526 shutdown_connections 2024-09-21T06:04:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.660+0000 7f9f6d112640 1 --2- 172.21.15.117:0/2501693526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f68108310 0x7f9f68108720 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.660+0000 7f9f6d112640 1 --2- 172.21.15.117:0/2501693526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f68108c60 0x7f9f6810f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.660+0000 7f9f6d112640 1 --2- 172.21.15.117:0/2501693526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f681029e0 0x7f9f68102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.660+0000 7f9f6d112640 1 -- 172.21.15.117:0/2501693526 >> 172.21.15.117:0/2501693526 conn(0x7f9f680fe3e0 msgr2=0x7f9f68100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.660+0000 7f9f6d112640 1 -- 172.21.15.117:0/2501693526 shutdown_connections 2024-09-21T06:04:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.660+0000 7f9f6d112640 1 -- 172.21.15.117:0/2501693526 wait complete. 2024-09-21T06:04:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.661+0000 7f9f6d112640 1 Processor -- start 2024-09-21T06:04:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.661+0000 7f9f6d112640 1 -- start start 2024-09-21T06:04:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.661+0000 7f9f6d112640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f681029e0 0x7f9f68199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.661+0000 7f9f6d112640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f68108310 0x7f9f68199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.661+0000 7f9f6d112640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f68108c60 0x7f9f6819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.661+0000 7f9f6d112640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9f68112920 con 0x7f9f681029e0 2024-09-21T06:04:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.661+0000 7f9f6d112640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9f681127a0 con 0x7f9f68108c60 2024-09-21T06:04:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.661+0000 7f9f6d112640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9f68112aa0 con 0x7f9f68108310 2024-09-21T06:04:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.661+0000 7f9f66d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f681029e0 0x7f9f68199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.661+0000 7f9f66d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f681029e0 0x7f9f68199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43478/0 (socket says 172.21.15.117:43478) 2024-09-21T06:04:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.661+0000 7f9f66d9d640 1 -- 172.21.15.117:0/360743386 learned_addr learned my addr 172.21.15.117:0/360743386 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.662+0000 7f9f6759e640 1 --2- 172.21.15.117:0/360743386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f68108c60 0x7f9f6819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.662+0000 7f9f6659c640 1 --2- 172.21.15.117:0/360743386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f68108310 0x7f9f68199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.662+0000 7f9f66d9d640 1 -- 172.21.15.117:0/360743386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f68108310 msgr2=0x7f9f68199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.662+0000 7f9f66d9d640 1 --2- 172.21.15.117:0/360743386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f68108310 0x7f9f68199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.662+0000 7f9f66d9d640 1 -- 172.21.15.117:0/360743386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f68108c60 msgr2=0x7f9f6819eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.662+0000 7f9f66d9d640 1 --2- 172.21.15.117:0/360743386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f68108c60 0x7f9f6819eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.662+0000 7f9f66d9d640 1 -- 172.21.15.117:0/360743386 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f6819f290 con 0x7f9f681029e0 2024-09-21T06:04:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.662+0000 7f9f6759e640 1 --2- 172.21.15.117:0/360743386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f68108c60 0x7f9f6819eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:04:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.662+0000 7f9f66d9d640 1 --2- 172.21.15.117:0/360743386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f681029e0 0x7f9f68199360 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f9f5800e9f0 tx=0x7f9f5800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.662+0000 7f9f4ffff640 1 -- 172.21.15.117:0/360743386 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5800cd80 con 0x7f9f681029e0 2024-09-21T06:04:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.662+0000 7f9f4ffff640 1 -- 172.21.15.117:0/360743386 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f580108e0 con 0x7f9f681029e0 2024-09-21T06:04:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.662+0000 7f9f6d112640 1 -- 172.21.15.117:0/360743386 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f6819f580 con 0x7f9f681029e0 2024-09-21T06:04:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.662+0000 7f9f4ffff640 1 -- 172.21.15.117:0/360743386 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f58018980 con 0x7f9f681029e0 2024-09-21T06:04:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.662+0000 7f9f6d112640 1 -- 172.21.15.117:0/360743386 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f681a5d10 con 0x7f9f681029e0 2024-09-21T06:04:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.665+0000 7f9f74079740 1 Processor -- start 2024-09-21T06:04:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.665+0000 7f9f74079740 1 -- start start 2024-09-21T06:04:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.665+0000 7f9f74079740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56075f0fb1b0 0x56075f4a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.665+0000 7f9f74079740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56075f0d22f0 0x56075f4a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.665+0000 7f9f74079740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075f0da4d0 0x56075f4a1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.665+0000 7f9f74079740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56075f4156f0 con 0x56075f0da4d0 2024-09-21T06:04:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.665+0000 7f9f74079740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56075f404b20 con 0x56075f0fb1b0 2024-09-21T06:04:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.665+0000 7f9f74079740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56075f408930 con 0x56075f0d22f0 2024-09-21T06:04:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.665+0000 7f9f4ffff640 1 -- 172.21.15.117:0/360743386 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f9f58018b20 con 0x7f9f681029e0 2024-09-21T06:04:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.665+0000 7f9f6659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075f0da4d0 0x56075f4a1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.665+0000 7f9f6659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075f0da4d0 0x56075f4a1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43492/0 (socket says 172.21.15.117:43492) 2024-09-21T06:04:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.665+0000 7f9f6659c640 1 -- 172.21.15.117:0/4254706966 learned_addr learned my addr 172.21.15.117:0/4254706966 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.665+0000 7f9f6759e640 1 --2- 172.21.15.117:0/4254706966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56075f0fb1b0 0x56075f4a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.665+0000 7f9f6659c640 1 -- 172.21.15.117:0/4254706966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56075f0d22f0 msgr2=0x56075f4a0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.665+0000 7f9f6659c640 1 --2- 172.21.15.117:0/4254706966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56075f0d22f0 0x56075f4a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.665+0000 7f9f6659c640 1 -- 172.21.15.117:0/4254706966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56075f0fb1b0 msgr2=0x56075f4a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.665+0000 7f9f6659c640 1 --2- 172.21.15.117:0/4254706966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56075f0fb1b0 0x56075f4a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.665+0000 7f9f6659c640 1 -- 172.21.15.117:0/4254706966 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56075f3ca530 con 0x56075f0da4d0 2024-09-21T06:04:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.666+0000 7f9f6659c640 1 --2- 172.21.15.117:0/4254706966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075f0da4d0 0x56075f4a1430 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c000c00 tx=0x7f9f5c032ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7b57824d727783 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.666+0000 7f9f4d7fa640 1 -- 172.21.15.117:0/4254706966 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c03f070 con 0x56075f0da4d0 2024-09-21T06:04:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.666+0000 7f9f4ffff640 1 --2- 172.21.15.117:0/360743386 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9f500b6680 0x7f9f500b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.666+0000 7f9f4d7fa640 1 -- 172.21.15.117:0/4254706966 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f5c004030 con 0x56075f0da4d0 2024-09-21T06:04:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.666+0000 7f9f4d7fa640 1 -- 172.21.15.117:0/4254706966 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c035070 con 0x56075f0da4d0 2024-09-21T06:04:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.666+0000 7f9f6759e640 1 --2- 172.21.15.117:0/360743386 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9f500b6680 0x7f9f500b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.666+0000 7f9f4ffff640 1 -- 172.21.15.117:0/360743386 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f9f580e7b50 con 0x7f9f681029e0 2024-09-21T06:04:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.667+0000 7f9f74079740 1 -- 172.21.15.117:0/4254706966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075f0da4d0 msgr2=0x56075f4a1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.667+0000 7f9f6759e640 1 --2- 172.21.15.117:0/360743386 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9f500b6680 0x7f9f500b8b90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9f60005fd0 tx=0x7f9f60009800 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.667+0000 7f9f74079740 1 --2- 172.21.15.117:0/4254706966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075f0da4d0 0x56075f4a1430 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c000c00 tx=0x7f9f5c032ba0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.667+0000 7f9f74079740 1 -- 172.21.15.117:0/4254706966 shutdown_connections 2024-09-21T06:04:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.667+0000 7f9f74079740 1 --2- 172.21.15.117:0/4254706966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075f0da4d0 0x56075f4a1430 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.667+0000 7f9f74079740 1 --2- 172.21.15.117:0/4254706966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56075f0d22f0 0x56075f4a0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.667+0000 7f9f74079740 1 --2- 172.21.15.117:0/4254706966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56075f0fb1b0 0x56075f4a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.667+0000 7f9f74079740 1 -- 172.21.15.117:0/4254706966 >> 172.21.15.117:0/4254706966 conn(0x56075f0ef300 msgr2=0x56075f49e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.667+0000 7f9f74079740 1 -- 172.21.15.117:0/4254706966 shutdown_connections 2024-09-21T06:04:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.667+0000 7f9f74079740 1 -- 172.21.15.117:0/4254706966 wait complete. 2024-09-21T06:04:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.667+0000 7f9f74079740 1 -- start start 2024-09-21T06:04:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.667+0000 7f9f74079740 1 Processor -- start 2024-09-21T06:04:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.668+0000 7f9f74079740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56075f0d22f0 0x56075f4ac0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.668+0000 7f9f74079740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075f0da4d0 0x56075f4ac620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.668+0000 7f9f74079740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56075f0fb1b0 0x56075f4b83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.668+0000 7f9f74079740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56075f33cab0 con 0x56075f0da4d0 2024-09-21T06:04:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.668+0000 7f9f74079740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56075f3aa730 con 0x56075f0d22f0 2024-09-21T06:04:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.668+0000 7f9f74079740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56075f4af0d0 con 0x56075f0fb1b0 2024-09-21T06:04:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.668+0000 7f9f74079740 10 client.0 _ll_register_callbacks cb 0x56075f3fe6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.668+0000 7f9f6659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075f0da4d0 0x56075f4ac620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.668+0000 7f9f6659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075f0da4d0 0x56075f4ac620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43496/0 (socket says 172.21.15.117:43496) 2024-09-21T06:04:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.668+0000 7f9f6659c640 1 -- 172.21.15.117:0/1814708958 learned_addr learned my addr 172.21.15.117:0/1814708958 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.668+0000 7f9f66d9d640 1 --2- 172.21.15.117:0/1814708958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56075f0d22f0 0x56075f4ac0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.668+0000 7f9f6759e640 1 --2- 172.21.15.117:0/1814708958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56075f0fb1b0 0x56075f4b83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.668+0000 7f9f6659c640 1 -- 172.21.15.117:0/1814708958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56075f0fb1b0 msgr2=0x56075f4b83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.668+0000 7f9f6659c640 1 --2- 172.21.15.117:0/1814708958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56075f0fb1b0 0x56075f4b83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.668+0000 7f9f6659c640 1 -- 172.21.15.117:0/1814708958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56075f0d22f0 msgr2=0x56075f4ac0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.668+0000 7f9f6659c640 1 --2- 172.21.15.117:0/1814708958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56075f0d22f0 0x56075f4ac0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.668+0000 7f9f6659c640 1 -- 172.21.15.117:0/1814708958 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56075f166790 con 0x56075f0da4d0 2024-09-21T06:04:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.668+0000 7f9f6659c640 1 --2- 172.21.15.117:0/1814708958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075f0da4d0 0x56075f4ac620 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c02a930 tx=0x7f9f5c005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e548ceaf88d34a8e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.668+0000 7f9f4aff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:04:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.668+0000 7f9f4aff5640 1 -- 172.21.15.117:0/1814708958 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c035070 con 0x56075f0da4d0 2024-09-21T06:04:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.668+0000 7f9f4aff5640 1 -- 172.21.15.117:0/1814708958 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f9f5c0316a0 con 0x56075f0da4d0 2024-09-21T06:04:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.668+0000 7f9f74079740 1 -- 172.21.15.117:0/1814708958 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f5c004030 con 0x56075f0da4d0 2024-09-21T06:04:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.668+0000 7f9f4aff5640 1 -- 172.21.15.117:0/1814708958 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9f5c055050 con 0x56075f0da4d0 2024-09-21T06:04:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.668+0000 7f9f74079740 10 client.16797 fetch_fsmap learned FSMap version 29 2024-09-21T06:04:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.669+0000 7f9f74079740 1 -- 172.21.15.117:0/1814708958 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56075f1df620 con 0x56075f0da4d0 2024-09-21T06:04:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.669+0000 7f9f4aff5640 1 -- 172.21.15.117:0/1814708958 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f5c004030 con 0x56075f0da4d0 2024-09-21T06:04:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.669+0000 7f9f4aff5640 1 -- 172.21.15.117:0/1814708958 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9f5c0046c0 con 0x56075f0da4d0 2024-09-21T06:04:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.669+0000 7f9f74079740 10 client.16797 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:04:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.669+0000 7f9f74079740 10 client.16797 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:04:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.669+0000 7f9f74079740 20 client.16797 populate_metadata read hostname 'smithi117' 2024-09-21T06:04:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.669+0000 7f9f74079740 1 --2- 172.21.15.117:0/1814708958 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56075f0b9d80 0x56075f4be020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.669+0000 7f9f74079740 4 client.16797 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:04:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.669+0000 7f9f74079740 1 -- 172.21.15.117:0/1814708958 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56075f417640 con 0x56075f0b9d80 2024-09-21T06:04:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.669+0000 7f9f66d9d640 1 --2- 172.21.15.117:0/1814708958 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56075f0b9d80 0x56075f4be020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.670+0000 7f9f4aff5640 10 client.16797 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.670+0000 7f9f66d9d640 1 --2- 172.21.15.117:0/1814708958 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56075f0b9d80 0x56075f4be020 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96393e5bb777803b server_cookie=e44e729ca356f30f in_seq=0 out_seq=0 2024-09-21T06:04:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.670+0000 7f9f4aff5640 1 -- 172.21.15.117:0/1814708958 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56075f417640 con 0x56075f0b9d80 2024-09-21T06:04:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.670+0000 7f9f4aff5640 10 client.16797 handle_command_reply: tid=0 2024-09-21T06:04:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.670+0000 7f9f74079740 1 client.16797 shutdown 2024-09-21T06:04:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.670+0000 7f9f74079740 1 -- 172.21.15.117:0/1814708958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075f0da4d0 msgr2=0x56075f4ac620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.670+0000 7f9f74079740 1 --2- 172.21.15.117:0/1814708958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075f0da4d0 0x56075f4ac620 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c02a930 tx=0x7f9f5c005e50 comp rx=0 tx=0).stop 2024-09-21T06:04:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.670+0000 7f9f74079740 1 -- 172.21.15.117:0/1814708958 shutdown_connections 2024-09-21T06:04:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.670+0000 7f9f74079740 1 --2- 172.21.15.117:0/1814708958 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56075f0b9d80 0x56075f4be020 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.670+0000 7f9f74079740 1 --2- 172.21.15.117:0/1814708958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075f0da4d0 0x56075f4ac620 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.670+0000 7f9f74079740 1 --2- 172.21.15.117:0/1814708958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56075f0fb1b0 0x56075f4b83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.670+0000 7f9f4aff5640 0 client.16797 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.670+0000 7f9f74079740 1 --2- 172.21.15.117:0/1814708958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56075f0d22f0 0x56075f4ac0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.671+0000 7f9f74079740 1 -- 172.21.15.117:0/1814708958 >> 172.21.15.117:0/1814708958 conn(0x56075f0ef300 msgr2=0x56075f4a19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.671+0000 7f9f74079740 1 -- 172.21.15.117:0/1814708958 shutdown_connections 2024-09-21T06:04:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.671+0000 7f9f74079740 1 -- 172.21.15.117:0/1814708958 wait complete. 2024-09-21T06:04:47.679 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.671+0000 7f9f74079740 20 client.16797 trim_cache size 0 max 16384 2024-09-21T06:04:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.685+0000 7f9f74079740 1 Processor -- start 2024-09-21T06:04:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.685+0000 7f9f74079740 1 -- start start 2024-09-21T06:04:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.685+0000 7f9f74079740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075f0b9d80 0x56075f4aafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.685+0000 7f9f74079740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56075f0d22f0 0x56075f4ac5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.685+0000 7f9f74079740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56075f0da4d0 0x56075f4a20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.685+0000 7f9f74079740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9f681ae4a0 con 0x56075f0b9d80 2024-09-21T06:04:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.685+0000 7f9f74079740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9f681affe0 con 0x56075f0da4d0 2024-09-21T06:04:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.685+0000 7f9f74079740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9f681aeb70 con 0x56075f0d22f0 2024-09-21T06:04:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.685+0000 7f9f66d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075f0b9d80 0x56075f4aafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.685+0000 7f9f66d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075f0b9d80 0x56075f4aafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43512/0 (socket says 172.21.15.117:43512) 2024-09-21T06:04:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.685+0000 7f9f66d9d640 1 -- 172.21.15.117:0/3410514307 learned_addr learned my addr 172.21.15.117:0/3410514307 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.686+0000 7f9f6759e640 1 --2- 172.21.15.117:0/3410514307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56075f0da4d0 0x56075f4a20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.686+0000 7f9f66d9d640 1 -- 172.21.15.117:0/3410514307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56075f0d22f0 msgr2=0x56075f4ac5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.686+0000 7f9f6659c640 1 --2- 172.21.15.117:0/3410514307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56075f0d22f0 0x56075f4ac5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.686+0000 7f9f66d9d640 1 --2- 172.21.15.117:0/3410514307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56075f0d22f0 0x56075f4ac5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.686+0000 7f9f66d9d640 1 -- 172.21.15.117:0/3410514307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56075f0da4d0 msgr2=0x56075f4a20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.686+0000 7f9f66d9d640 1 --2- 172.21.15.117:0/3410514307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56075f0da4d0 0x56075f4a20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.686+0000 7f9f66d9d640 1 -- 172.21.15.117:0/3410514307 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56075f4a2ee0 con 0x56075f0b9d80 2024-09-21T06:04:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.686+0000 7f9f6759e640 1 --2- 172.21.15.117:0/3410514307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56075f0da4d0 0x56075f4a20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.686+0000 7f9f6659c640 1 --2- 172.21.15.117:0/3410514307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56075f0d22f0 0x56075f4ac5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.686+0000 7f9f66d9d640 1 --2- 172.21.15.117:0/3410514307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075f0b9d80 0x56075f4aafb0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f9f580e53a0 tx=0x7f9f58000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f79eb165ca21f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.686+0000 7f9f4cff9640 1 -- 172.21.15.117:0/3410514307 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f580ad500 con 0x56075f0b9d80 2024-09-21T06:04:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.686+0000 7f9f4cff9640 1 -- 172.21.15.117:0/3410514307 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f580ae0f0 con 0x56075f0b9d80 2024-09-21T06:04:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.686+0000 7f9f4cff9640 1 -- 172.21.15.117:0/3410514307 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f580a9340 con 0x56075f0b9d80 2024-09-21T06:04:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.687+0000 7f9f74079740 1 -- 172.21.15.117:0/3410514307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075f0b9d80 msgr2=0x56075f4aafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.687+0000 7f9f74079740 1 --2- 172.21.15.117:0/3410514307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075f0b9d80 0x56075f4aafb0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f9f580e53a0 tx=0x7f9f58000bc0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.687+0000 7f9f74079740 1 -- 172.21.15.117:0/3410514307 shutdown_connections 2024-09-21T06:04:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.687+0000 7f9f74079740 1 --2- 172.21.15.117:0/3410514307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56075f0da4d0 0x56075f4a20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.687+0000 7f9f74079740 1 --2- 172.21.15.117:0/3410514307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56075f0d22f0 0x56075f4ac5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.687+0000 7f9f74079740 1 --2- 172.21.15.117:0/3410514307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075f0b9d80 0x56075f4aafb0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.687+0000 7f9f74079740 1 -- 172.21.15.117:0/3410514307 >> 172.21.15.117:0/3410514307 conn(0x56075f0ef300 msgr2=0x56075f4a19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.687+0000 7f9f74079740 1 -- 172.21.15.117:0/3410514307 shutdown_connections 2024-09-21T06:04:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.687+0000 7f9f74079740 1 -- 172.21.15.117:0/3410514307 wait complete. 2024-09-21T06:04:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.687+0000 7f9f74079740 1 -- start start 2024-09-21T06:04:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.687+0000 7f9f74079740 1 Processor -- start 2024-09-21T06:04:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.688+0000 7f9f74079740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56075f0b9d80 0x56075f4a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.688+0000 7f9f74079740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56075f0d22f0 0x56075f4a6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.688+0000 7f9f74079740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075f0da4d0 0x56075f5287b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.688+0000 7f9f74079740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9f681af910 con 0x56075f0da4d0 2024-09-21T06:04:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.688+0000 7f9f74079740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9f681af240 con 0x56075f0d22f0 2024-09-21T06:04:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.688+0000 7f9f74079740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9f681ac4a0 con 0x56075f0b9d80 2024-09-21T06:04:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.688+0000 7f9f74079740 10 client.0 _ll_register_callbacks cb 0x7f9f600096a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.688+0000 7f9f6759e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075f0da4d0 0x56075f5287b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.688+0000 7f9f6759e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075f0da4d0 0x56075f5287b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43514/0 (socket says 172.21.15.117:43514) 2024-09-21T06:04:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.688+0000 7f9f6759e640 1 -- 172.21.15.117:0/982314068 learned_addr learned my addr 172.21.15.117:0/982314068 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.688+0000 7f9f6659c640 1 --2- 172.21.15.117:0/982314068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56075f0d22f0 0x56075f4a6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.688+0000 7f9f66d9d640 1 --2- 172.21.15.117:0/982314068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56075f0b9d80 0x56075f4a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.688+0000 7f9f6759e640 1 -- 172.21.15.117:0/982314068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56075f0b9d80 msgr2=0x56075f4a61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.688+0000 7f9f6759e640 1 --2- 172.21.15.117:0/982314068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56075f0b9d80 0x56075f4a61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.688+0000 7f9f6759e640 1 -- 172.21.15.117:0/982314068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56075f0d22f0 msgr2=0x56075f4a6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.688+0000 7f9f6759e640 1 --2- 172.21.15.117:0/982314068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56075f0d22f0 0x56075f4a6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.688+0000 7f9f6759e640 1 -- 172.21.15.117:0/982314068 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56075f4a6f30 con 0x56075f0da4d0 2024-09-21T06:04:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.688+0000 7f9f66d9d640 1 --2- 172.21.15.117:0/982314068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56075f0b9d80 0x56075f4a61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.688+0000 7f9f6759e640 1 --2- 172.21.15.117:0/982314068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075f0da4d0 0x56075f5287b0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f9f600095b0 tx=0x7f9f60004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d21f2719ae06c1e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.688+0000 7f9f4aff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:04:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.688+0000 7f9f4aff5640 1 -- 172.21.15.117:0/982314068 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6002e060 con 0x56075f0da4d0 2024-09-21T06:04:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.688+0000 7f9f4aff5640 1 -- 172.21.15.117:0/982314068 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f9f60020b30 con 0x56075f0da4d0 2024-09-21T06:04:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.688+0000 7f9f74079740 1 -- 172.21.15.117:0/982314068 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f580ae0f0 con 0x56075f0da4d0 2024-09-21T06:04:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.689+0000 7f9f4aff5640 1 -- 172.21.15.117:0/982314068 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9f60020d30 con 0x56075f0da4d0 2024-09-21T06:04:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.689+0000 7f9f74079740 10 client.16803 fetch_fsmap learned FSMap version 29 2024-09-21T06:04:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.689+0000 7f9f74079740 1 -- 172.21.15.117:0/982314068 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56075f529c20 con 0x56075f0da4d0 2024-09-21T06:04:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.689+0000 7f9f4aff5640 1 -- 172.21.15.117:0/982314068 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f580ae0f0 con 0x56075f0da4d0 2024-09-21T06:04:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.689+0000 7f9f4aff5640 1 -- 172.21.15.117:0/982314068 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9f60028070 con 0x56075f0da4d0 2024-09-21T06:04:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.689+0000 7f9f74079740 10 client.16803 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:04:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.689+0000 7f9f74079740 10 client.16803 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:04:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.689+0000 7f9f74079740 20 client.16803 populate_metadata read hostname 'smithi117' 2024-09-21T06:04:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.689+0000 7f9f74079740 1 --2- 172.21.15.117:0/982314068 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56075f0fb1b0 0x56075f52c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.689+0000 7f9f74079740 4 client.16803 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:04:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.689+0000 7f9f74079740 1 -- 172.21.15.117:0/982314068 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56075f4130f0 con 0x56075f0fb1b0 2024-09-21T06:04:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.689+0000 7f9f66d9d640 1 --2- 172.21.15.117:0/982314068 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56075f0fb1b0 0x56075f52c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.690+0000 7f9f4aff5640 10 client.16803 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:47.697 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.690+0000 7f9f66d9d640 1 --2- 172.21.15.117:0/982314068 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56075f0fb1b0 0x56075f52c410 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43213989680d080e server_cookie=19d475c02fabe74b in_seq=0 out_seq=0 2024-09-21T06:04:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.699+0000 7f9f4aff5640 1 -- 172.21.15.117:0/982314068 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56075f4130f0 con 0x56075f0fb1b0 2024-09-21T06:04:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.699+0000 7f9f4aff5640 10 client.16803 handle_command_reply: tid=0 2024-09-21T06:04:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.699+0000 7f9f74079740 1 client.16803 shutdown 2024-09-21T06:04:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.699+0000 7f9f74079740 1 -- 172.21.15.117:0/982314068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075f0da4d0 msgr2=0x56075f5287b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.699+0000 7f9f74079740 1 --2- 172.21.15.117:0/982314068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075f0da4d0 0x56075f5287b0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f9f600095b0 tx=0x7f9f60004a40 comp rx=0 tx=0).stop 2024-09-21T06:04:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.699+0000 7f9f74079740 1 -- 172.21.15.117:0/982314068 shutdown_connections 2024-09-21T06:04:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.699+0000 7f9f74079740 1 --2- 172.21.15.117:0/982314068 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56075f0fb1b0 0x56075f52c410 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.699+0000 7f9f74079740 1 --2- 172.21.15.117:0/982314068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56075f0da4d0 0x56075f5287b0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.699+0000 7f9f4aff5640 0 client.16803 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.699+0000 7f9f74079740 1 --2- 172.21.15.117:0/982314068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56075f0d22f0 0x56075f4a6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.699+0000 7f9f74079740 1 --2- 172.21.15.117:0/982314068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56075f0b9d80 0x56075f4a61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.702 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.699+0000 7f9f74079740 1 -- 172.21.15.117:0/982314068 >> 172.21.15.117:0/982314068 conn(0x56075f0ef300 msgr2=0x56075f4a2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.700+0000 7f9f74079740 1 -- 172.21.15.117:0/982314068 shutdown_connections 2024-09-21T06:04:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.700+0000 7f9f74079740 1 -- 172.21.15.117:0/982314068 wait complete. 2024-09-21T06:04:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.700+0000 7f9f74079740 20 client.16803 trim_cache size 0 max 16384 2024-09-21T06:04:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T06:04:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T06:04:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T06:04:47.703 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T06:04:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.700+0000 7f9f6d112640 1 -- 172.21.15.117:0/360743386 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9f500b6680 msgr2=0x7f9f500b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.700+0000 7f9f6d112640 1 --2- 172.21.15.117:0/360743386 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9f500b6680 0x7f9f500b8b90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9f60005fd0 tx=0x7f9f60009800 comp rx=0 tx=0).stop 2024-09-21T06:04:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.700+0000 7f9f6d112640 1 -- 172.21.15.117:0/360743386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f681029e0 msgr2=0x7f9f68199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.700+0000 7f9f6d112640 1 --2- 172.21.15.117:0/360743386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f681029e0 0x7f9f68199360 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f9f5800e9f0 tx=0x7f9f5800eeb0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.700+0000 7f9f6d112640 1 -- 172.21.15.117:0/360743386 shutdown_connections 2024-09-21T06:04:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.700+0000 7f9f6d112640 1 --2- 172.21.15.117:0/360743386 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9f500b6680 0x7f9f500b8b90 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.700+0000 7f9f6d112640 1 --2- 172.21.15.117:0/360743386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f68108c60 0x7f9f6819eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.700+0000 7f9f6d112640 1 --2- 172.21.15.117:0/360743386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f68108310 0x7f9f68199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.700+0000 7f9f6d112640 1 --2- 172.21.15.117:0/360743386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f681029e0 0x7f9f68199360 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.700+0000 7f9f6d112640 1 -- 172.21.15.117:0/360743386 >> 172.21.15.117:0/360743386 conn(0x7f9f680fe3e0 msgr2=0x7f9f681079b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:47.704 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.700+0000 7f9f6d112640 1 -- 172.21.15.117:0/360743386 shutdown_connections 2024-09-21T06:04:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:04:47.700+0000 7f9f6d112640 1 -- 172.21.15.117:0/360743386 wait complete. 2024-09-21T06:04:48.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:47 smithi192 ceph-mon[34700]: pgmap v554: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 7.2 KiB/s wr, 12 op/s 2024-09-21T06:04:48.266 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:47 smithi138 ceph-mon[34391]: pgmap v554: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 7.2 KiB/s wr, 12 op/s 2024-09-21T06:04:48.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:47 smithi117 ceph-mon[30947]: pgmap v554: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 7.2 KiB/s wr, 12 op/s 2024-09-21T06:04:48.343 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T06:04:48.589 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:04:49.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.117+0000 7ff2d3951640 1 -- 172.21.15.117:0/1968000436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2cc105310 msgr2=0x7ff2cc10d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:49.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.117+0000 7ff2d3951640 1 --2- 172.21.15.117:0/1968000436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2cc105310 0x7ff2cc10d860 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0009ca0 tx=0x7ff2c002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:04:49.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.117+0000 7ff2d3951640 1 -- 172.21.15.117:0/1968000436 shutdown_connections 2024-09-21T06:04:49.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.117+0000 7ff2d3951640 1 --2- 172.21.15.117:0/1968000436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2cc105310 0x7ff2cc10d860 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:49.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.117+0000 7ff2d3951640 1 --2- 172.21.15.117:0/1968000436 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2cc10de20 0x7ff2cc110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:49.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.117+0000 7ff2d3951640 1 --2- 172.21.15.117:0/1968000436 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2cc1049c0 0x7ff2cc104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:49.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.117+0000 7ff2d3951640 1 -- 172.21.15.117:0/1968000436 >> 172.21.15.117:0/1968000436 conn(0x7ff2cc079730 msgr2=0x7ff2cc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:49.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.117+0000 7ff2d3951640 1 -- 172.21.15.117:0/1968000436 shutdown_connections 2024-09-21T06:04:49.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.117+0000 7ff2d3951640 1 -- 172.21.15.117:0/1968000436 wait complete. 2024-09-21T06:04:49.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.118+0000 7ff2d3951640 1 Processor -- start 2024-09-21T06:04:49.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.118+0000 7ff2d3951640 1 -- start start 2024-09-21T06:04:49.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.118+0000 7ff2d3951640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2cc1049c0 0x7ff2cc102700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:49.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.118+0000 7ff2d3951640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2cc105310 0x7ff2cc102c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:49.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.118+0000 7ff2d3951640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2cc10de20 0x7ff2cc103180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:49.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.118+0000 7ff2d3951640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff2cc113a80 con 0x7ff2cc10de20 2024-09-21T06:04:49.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.118+0000 7ff2d3951640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff2cc113900 con 0x7ff2cc1049c0 2024-09-21T06:04:49.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.118+0000 7ff2d3951640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff2cc113c00 con 0x7ff2cc105310 2024-09-21T06:04:49.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.118+0000 7ff2d1eee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2cc10de20 0x7ff2cc103180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:49.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.118+0000 7ff2d1eee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2cc10de20 0x7ff2cc103180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43532/0 (socket says 172.21.15.117:43532) 2024-09-21T06:04:49.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.118+0000 7ff2d1eee640 1 -- 172.21.15.117:0/2554788366 learned_addr learned my addr 172.21.15.117:0/2554788366 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:49.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.118+0000 7ff2d16ed640 1 --2- 172.21.15.117:0/2554788366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2cc1049c0 0x7ff2cc102700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:49.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.118+0000 7ff2d0eec640 1 --2- 172.21.15.117:0/2554788366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2cc105310 0x7ff2cc102c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:49.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.118+0000 7ff2d1eee640 1 -- 172.21.15.117:0/2554788366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2cc105310 msgr2=0x7ff2cc102c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:49.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.118+0000 7ff2d1eee640 1 --2- 172.21.15.117:0/2554788366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2cc105310 0x7ff2cc102c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:49.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.118+0000 7ff2d1eee640 1 -- 172.21.15.117:0/2554788366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2cc1049c0 msgr2=0x7ff2cc102700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:49.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.118+0000 7ff2d1eee640 1 --2- 172.21.15.117:0/2554788366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2cc1049c0 0x7ff2cc102700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:49.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.119+0000 7ff2d1eee640 1 -- 172.21.15.117:0/2554788366 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2cc1a8770 con 0x7ff2cc10de20 2024-09-21T06:04:49.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.119+0000 7ff2d0eec640 1 --2- 172.21.15.117:0/2554788366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2cc105310 0x7ff2cc102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:04:49.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.119+0000 7ff2d1eee640 1 --2- 172.21.15.117:0/2554788366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2cc10de20 0x7ff2cc103180 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7ff2c400d950 tx=0x7ff2c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:49.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.119+0000 7ff2ca7fc640 1 -- 172.21.15.117:0/2554788366 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c4017030 con 0x7ff2cc10de20 2024-09-21T06:04:49.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.119+0000 7ff2d3951640 1 -- 172.21.15.117:0/2554788366 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2cc1a8a60 con 0x7ff2cc10de20 2024-09-21T06:04:49.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.119+0000 7ff2ca7fc640 1 -- 172.21.15.117:0/2554788366 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2c4004020 con 0x7ff2cc10de20 2024-09-21T06:04:49.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.119+0000 7ff2ca7fc640 1 -- 172.21.15.117:0/2554788366 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c4003b50 con 0x7ff2cc10de20 2024-09-21T06:04:49.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.119+0000 7ff2d3951640 1 -- 172.21.15.117:0/2554788366 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2cc1a8f20 con 0x7ff2cc10de20 2024-09-21T06:04:49.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.122+0000 7ff2d9ec5740 1 Processor -- start 2024-09-21T06:04:49.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.122+0000 7ff2ca7fc640 1 -- 172.21.15.117:0/2554788366 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7ff2c401a070 con 0x7ff2cc10de20 2024-09-21T06:04:49.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.122+0000 7ff2d9ec5740 1 -- start start 2024-09-21T06:04:49.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.122+0000 7ff2d9ec5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7a7612070 0x55f7a79b92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:49.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.122+0000 7ff2d9ec5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7a75e91b0 0x55f7a79b9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:49.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.122+0000 7ff2d9ec5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7a75f1390 0x55f7a79b9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:49.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.122+0000 7ff2d9ec5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f7a792fea0 con 0x55f7a7612070 2024-09-21T06:04:49.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.122+0000 7ff2d9ec5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f7a791f2d0 con 0x55f7a75f1390 2024-09-21T06:04:49.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.122+0000 7ff2d9ec5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f7a79230e0 con 0x55f7a75e91b0 2024-09-21T06:04:49.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.122+0000 7ff2d0eec640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7a7612070 0x55f7a79b92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:49.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.122+0000 7ff2d0eec640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7a7612070 0x55f7a79b92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43542/0 (socket says 172.21.15.117:43542) 2024-09-21T06:04:49.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.122+0000 7ff2d0eec640 1 -- 172.21.15.117:0/4283363756 learned_addr learned my addr 172.21.15.117:0/4283363756 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:49.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.123+0000 7ff2d16ed640 1 --2- 172.21.15.117:0/4283363756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7a75e91b0 0x55f7a79b9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:49.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.123+0000 7ff2ca7fc640 1 --2- 172.21.15.117:0/2554788366 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff2b40b6680 0x7ff2b40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:49.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.123+0000 7ff2d0eec640 1 --2- 172.21.15.117:0/4283363756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7a75f1390 0x55f7a79b9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:49.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.123+0000 7ff2d0eec640 1 -- 172.21.15.117:0/4283363756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7a75e91b0 msgr2=0x55f7a79b9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:49.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.123+0000 7ff2d0eec640 1 --2- 172.21.15.117:0/4283363756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7a75e91b0 0x55f7a79b9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:49.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.123+0000 7ff2d0eec640 1 -- 172.21.15.117:0/4283363756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7a75f1390 msgr2=0x55f7a79b9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:49.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.123+0000 7ff2d0eec640 1 --2- 172.21.15.117:0/4283363756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7a75f1390 0x55f7a79b9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:49.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.123+0000 7ff2d0eec640 1 -- 172.21.15.117:0/4283363756 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7a767d650 con 0x55f7a7612070 2024-09-21T06:04:49.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.123+0000 7ff2d16ed640 1 --2- 172.21.15.117:0/4283363756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7a75e91b0 0x55f7a79b9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:04:49.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.123+0000 7ff2d1eee640 1 --2- 172.21.15.117:0/2554788366 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff2b40b6680 0x7ff2b40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:49.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.123+0000 7ff2ca7fc640 1 -- 172.21.15.117:0/2554788366 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7ff2c40e6f50 con 0x7ff2cc10de20 2024-09-21T06:04:49.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.123+0000 7ff2d0eec640 1 --2- 172.21.15.117:0/4283363756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7a7612070 0x55f7a79b92c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0004af0 tx=0x7ff2c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c1437cc76a4340 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:49.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.123+0000 7ff2b37fe640 1 -- 172.21.15.117:0/4283363756 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0031650 con 0x55f7a7612070 2024-09-21T06:04:49.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.123+0000 7ff2b37fe640 1 -- 172.21.15.117:0/4283363756 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2c0046070 con 0x55f7a7612070 2024-09-21T06:04:49.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.123+0000 7ff2b37fe640 1 -- 172.21.15.117:0/4283363756 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c00044f0 con 0x55f7a7612070 2024-09-21T06:04:49.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.124+0000 7ff2d9ec5740 1 -- 172.21.15.117:0/4283363756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7a7612070 msgr2=0x55f7a79b92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:49.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.124+0000 7ff2d1eee640 1 --2- 172.21.15.117:0/2554788366 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff2b40b6680 0x7ff2b40b8b90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff2c40e47a0 tx=0x7ff2c40035f0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:49.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.124+0000 7ff2d9ec5740 1 --2- 172.21.15.117:0/4283363756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7a7612070 0x55f7a79b92c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0004af0 tx=0x7ff2c0004a40 comp rx=0 tx=0).stop 2024-09-21T06:04:49.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.124+0000 7ff2d9ec5740 1 -- 172.21.15.117:0/4283363756 shutdown_connections 2024-09-21T06:04:49.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.124+0000 7ff2d9ec5740 1 --2- 172.21.15.117:0/4283363756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7a75f1390 0x55f7a79b9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:49.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.124+0000 7ff2d9ec5740 1 --2- 172.21.15.117:0/4283363756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7a75e91b0 0x55f7a79b9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:49.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.124+0000 7ff2d9ec5740 1 --2- 172.21.15.117:0/4283363756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7a7612070 0x55f7a79b92c0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:49.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.124+0000 7ff2d9ec5740 1 -- 172.21.15.117:0/4283363756 >> 172.21.15.117:0/4283363756 conn(0x55f7a7606460 msgr2=0x55f7a79b71b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:49.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.124+0000 7ff2d9ec5740 1 -- 172.21.15.117:0/4283363756 shutdown_connections 2024-09-21T06:04:49.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.124+0000 7ff2d9ec5740 1 -- 172.21.15.117:0/4283363756 wait complete. 2024-09-21T06:04:49.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.125+0000 7ff2d9ec5740 1 -- start start 2024-09-21T06:04:49.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.125+0000 7ff2d9ec5740 1 Processor -- start 2024-09-21T06:04:49.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.125+0000 7ff2d9ec5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7a75e91b0 0x55f7a79c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:49.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.125+0000 7ff2d9ec5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7a75f1390 0x55f7a79c4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:49.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.125+0000 7ff2d9ec5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7a7612070 0x55f7a79d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:49.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.125+0000 7ff2d9ec5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f7a7780700 con 0x55f7a75e91b0 2024-09-21T06:04:49.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.125+0000 7ff2d9ec5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f7a78ba160 con 0x55f7a7612070 2024-09-21T06:04:49.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.125+0000 7ff2d9ec5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f7a79c7a40 con 0x55f7a75f1390 2024-09-21T06:04:49.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.126+0000 7ff2d9ec5740 10 client.0 _ll_register_callbacks cb 0x55f7a78d39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:49.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.126+0000 7ff2d0eec640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7a75e91b0 0x55f7a79c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:49.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.126+0000 7ff2d0eec640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7a75e91b0 0x55f7a79c49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43546/0 (socket says 172.21.15.117:43546) 2024-09-21T06:04:49.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.126+0000 7ff2d0eec640 1 -- 172.21.15.117:0/1426126114 learned_addr learned my addr 172.21.15.117:0/1426126114 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:49.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.126+0000 7ff2d16ed640 1 --2- 172.21.15.117:0/1426126114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7a75f1390 0x55f7a79c4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:49.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.126+0000 7ff2d0eec640 1 -- 172.21.15.117:0/1426126114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7a75f1390 msgr2=0x55f7a79c4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:49.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.126+0000 7ff2d0eec640 1 --2- 172.21.15.117:0/1426126114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7a75f1390 0x55f7a79c4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:49.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.126+0000 7ff2d0eec640 1 -- 172.21.15.117:0/1426126114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7a7612070 msgr2=0x55f7a79d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:49.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.126+0000 7ff2d0eec640 1 --2- 172.21.15.117:0/1426126114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7a7612070 0x55f7a79d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:49.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.126+0000 7ff2d0eec640 1 -- 172.21.15.117:0/1426126114 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7a78e7c90 con 0x55f7a75e91b0 2024-09-21T06:04:49.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.126+0000 7ff2d16ed640 1 --2- 172.21.15.117:0/1426126114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7a75f1390 0x55f7a79c4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:49.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.126+0000 7ff2d0eec640 1 --2- 172.21.15.117:0/1426126114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7a75e91b0 0x55f7a79c49f0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0031a00 tx=0x7ff2c0005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27817feaafe6f40 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:49.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.126+0000 7ff2b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:04:49.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.126+0000 7ff2b0ff9640 1 -- 172.21.15.117:0/1426126114 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0050070 con 0x55f7a75e91b0 2024-09-21T06:04:49.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.126+0000 7ff2b0ff9640 1 -- 172.21.15.117:0/1426126114 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7ff2c0003d80 con 0x55f7a75e91b0 2024-09-21T06:04:49.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.126+0000 7ff2d9ec5740 1 -- 172.21.15.117:0/1426126114 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2c0046070 con 0x55f7a75e91b0 2024-09-21T06:04:49.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.126+0000 7ff2b0ff9640 1 -- 172.21.15.117:0/1426126114 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff2c0004270 con 0x55f7a75e91b0 2024-09-21T06:04:49.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.126+0000 7ff2d9ec5740 10 client.16815 fetch_fsmap learned FSMap version 29 2024-09-21T06:04:49.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.126+0000 7ff2d9ec5740 1 -- 172.21.15.117:0/1426126114 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f7a79d37c0 con 0x55f7a75e91b0 2024-09-21T06:04:49.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.127+0000 7ff2b0ff9640 1 -- 172.21.15.117:0/1426126114 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2c0046070 con 0x55f7a75e91b0 2024-09-21T06:04:49.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.127+0000 7ff2b0ff9640 1 -- 172.21.15.117:0/1426126114 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff2c002fb80 con 0x55f7a75e91b0 2024-09-21T06:04:49.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.127+0000 7ff2d9ec5740 10 client.16815 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:04:49.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.127+0000 7ff2d9ec5740 10 client.16815 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T06:04:49.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.127+0000 7ff2d9ec5740 20 client.16815 populate_metadata read hostname 'smithi117' 2024-09-21T06:04:49.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.127+0000 7ff2d9ec5740 1 --2- 172.21.15.117:0/1426126114 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55f7a75d09f0 0x55f7a79d6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:49.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.127+0000 7ff2d9ec5740 4 client.16815 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:04:49.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.127+0000 7ff2d9ec5740 1 -- 172.21.15.117:0/1426126114 --> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7a7931df0 con 0x55f7a75d09f0 2024-09-21T06:04:49.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.127+0000 7ff2d16ed640 1 --2- 172.21.15.117:0/1426126114 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55f7a75d09f0 0x55f7a79d6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:49.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.128+0000 7ff2d16ed640 1 --2- 172.21.15.117:0/1426126114 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55f7a75d09f0 0x55f7a79d6b20 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=240e2aee2c10db14 server_cookie=328e1cd955d85a87 in_seq=0 out_seq=0 2024-09-21T06:04:49.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.128+0000 7ff2b0ff9640 10 client.16815 ms_handle_connect on v2:172.21.15.192:6834/3530688860 2024-09-21T06:04:49.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.129+0000 7ff2b0ff9640 1 -- 172.21.15.117:0/1426126114 <== mds.3 v2:172.21.15.192:6834/3530688860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f7a7931df0 con 0x55f7a75d09f0 2024-09-21T06:04:49.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.129+0000 7ff2b0ff9640 10 client.16815 handle_command_reply: tid=0 2024-09-21T06:04:49.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.129+0000 7ff2d9ec5740 1 client.16815 shutdown 2024-09-21T06:04:49.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.129+0000 7ff2d9ec5740 1 -- 172.21.15.117:0/1426126114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7a75e91b0 msgr2=0x55f7a79c49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:49.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.129+0000 7ff2d9ec5740 1 --2- 172.21.15.117:0/1426126114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7a75e91b0 0x55f7a79c49f0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0031a00 tx=0x7ff2c0005d80 comp rx=0 tx=0).stop 2024-09-21T06:04:49.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.129+0000 7ff2d9ec5740 1 -- 172.21.15.117:0/1426126114 shutdown_connections 2024-09-21T06:04:49.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.129+0000 7ff2d9ec5740 1 --2- 172.21.15.117:0/1426126114 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55f7a75d09f0 0x55f7a79d6b20 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:49.142 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.129+0000 7ff2d9ec5740 1 --2- 172.21.15.117:0/1426126114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7a7612070 0x55f7a79d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:49.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.129+0000 7ff2d9ec5740 1 --2- 172.21.15.117:0/1426126114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7a75f1390 0x55f7a79c4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:49.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.129+0000 7ff2d9ec5740 1 --2- 172.21.15.117:0/1426126114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7a75e91b0 0x55f7a79c49f0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:49.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.129+0000 7ff2b0ff9640 0 client.16815 ms_handle_reset on v2:172.21.15.192:6834/3530688860 2024-09-21T06:04:49.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.129+0000 7ff2d9ec5740 1 -- 172.21.15.117:0/1426126114 >> 172.21.15.117:0/1426126114 conn(0x55f7a7606460 msgr2=0x55f7a79bc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:49.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.129+0000 7ff2d9ec5740 1 -- 172.21.15.117:0/1426126114 shutdown_connections 2024-09-21T06:04:49.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.129+0000 7ff2d9ec5740 1 -- 172.21.15.117:0/1426126114 wait complete. 2024-09-21T06:04:49.143 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.129+0000 7ff2d9ec5740 20 client.16815 trim_cache size 0 max 16384 2024-09-21T06:04:49.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.142+0000 7ff2d9ec5740 1 Processor -- start 2024-09-21T06:04:49.144 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.142+0000 7ff2d9ec5740 1 -- start start 2024-09-21T06:04:49.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.142+0000 7ff2d9ec5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7a7606460 0x55f7a79baa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:49.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.142+0000 7ff2d9ec5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7a75e91b0 0x55f7a79c1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:49.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.142+0000 7ff2d9ec5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7a75f1390 0x55f7a79c2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:49.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.142+0000 7ff2d9ec5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff2cc1b16d0 con 0x55f7a75f1390 2024-09-21T06:04:49.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.142+0000 7ff2d9ec5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff2cc1b3210 con 0x55f7a75e91b0 2024-09-21T06:04:49.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.142+0000 7ff2d9ec5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff2cc1b1da0 con 0x55f7a7606460 2024-09-21T06:04:49.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.142+0000 7ff2d0eec640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7a75f1390 0x55f7a79c2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:49.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.142+0000 7ff2d0eec640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7a75f1390 0x55f7a79c2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43562/0 (socket says 172.21.15.117:43562) 2024-09-21T06:04:49.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.142+0000 7ff2d0eec640 1 -- 172.21.15.117:0/2371069695 learned_addr learned my addr 172.21.15.117:0/2371069695 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:49.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.143+0000 7ff2d16ed640 1 --2- 172.21.15.117:0/2371069695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7a75e91b0 0x55f7a79c1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:49.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.143+0000 7ff2d0eec640 1 -- 172.21.15.117:0/2371069695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7a7606460 msgr2=0x55f7a79baa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:49.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.143+0000 7ff2d0eec640 1 --2- 172.21.15.117:0/2371069695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7a7606460 0x55f7a79baa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:49.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.143+0000 7ff2d0eec640 1 -- 172.21.15.117:0/2371069695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7a75e91b0 msgr2=0x55f7a79c1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:49.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.143+0000 7ff2d0eec640 1 --2- 172.21.15.117:0/2371069695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7a75e91b0 0x55f7a79c1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:49.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.143+0000 7ff2d0eec640 1 -- 172.21.15.117:0/2371069695 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7a79b25d0 con 0x55f7a75f1390 2024-09-21T06:04:49.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.143+0000 7ff2d16ed640 1 --2- 172.21.15.117:0/2371069695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7a75e91b0 0x55f7a79c1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:49.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.143+0000 7ff2d0eec640 1 --2- 172.21.15.117:0/2371069695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7a75f1390 0x55f7a79c2120 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0002230 tx=0x7ff2c0004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c64c4a2e5faa3e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:49.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.143+0000 7ff2b2ffd640 1 -- 172.21.15.117:0/2371069695 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0050070 con 0x55f7a75f1390 2024-09-21T06:04:49.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.143+0000 7ff2b2ffd640 1 -- 172.21.15.117:0/2371069695 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2c005f020 con 0x55f7a75f1390 2024-09-21T06:04:49.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.143+0000 7ff2b2ffd640 1 -- 172.21.15.117:0/2371069695 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0030aa0 con 0x55f7a75f1390 2024-09-21T06:04:49.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.144+0000 7ff2d9ec5740 1 -- 172.21.15.117:0/2371069695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7a75f1390 msgr2=0x55f7a79c2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:49.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.144+0000 7ff2d9ec5740 1 --2- 172.21.15.117:0/2371069695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7a75f1390 0x55f7a79c2120 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0002230 tx=0x7ff2c0004200 comp rx=0 tx=0).stop 2024-09-21T06:04:49.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.144+0000 7ff2d9ec5740 1 -- 172.21.15.117:0/2371069695 shutdown_connections 2024-09-21T06:04:49.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.144+0000 7ff2d9ec5740 1 --2- 172.21.15.117:0/2371069695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7a75f1390 0x55f7a79c2120 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:49.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.144+0000 7ff2d9ec5740 1 --2- 172.21.15.117:0/2371069695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7a75e91b0 0x55f7a79c1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:49.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.144+0000 7ff2d9ec5740 1 --2- 172.21.15.117:0/2371069695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7a7606460 0x55f7a79baa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:49.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.144+0000 7ff2d9ec5740 1 -- 172.21.15.117:0/2371069695 >> 172.21.15.117:0/2371069695 conn(0x55f7a79c50b0 msgr2=0x55f7a79ba2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:49.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.144+0000 7ff2d9ec5740 1 -- 172.21.15.117:0/2371069695 shutdown_connections 2024-09-21T06:04:49.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.144+0000 7ff2d9ec5740 1 -- 172.21.15.117:0/2371069695 wait complete. 2024-09-21T06:04:49.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.144+0000 7ff2d9ec5740 1 -- start start 2024-09-21T06:04:49.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.144+0000 7ff2d9ec5740 1 Processor -- start 2024-09-21T06:04:49.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.145+0000 7ff2d9ec5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7a75e91b0 0x55f7a79d9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:49.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.145+0000 7ff2d9ec5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7a75f1390 0x55f7a79d9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:49.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.145+0000 7ff2d9ec5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7a7606460 0x55f7a7a392e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:49.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.145+0000 7ff2d9ec5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff2cc1b2b40 con 0x55f7a7606460 2024-09-21T06:04:49.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.145+0000 7ff2d9ec5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff2cc1b2470 con 0x55f7a75f1390 2024-09-21T06:04:49.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.145+0000 7ff2d9ec5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff2cc1af6d0 con 0x55f7a75e91b0 2024-09-21T06:04:49.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.145+0000 7ff2d9ec5740 10 client.0 _ll_register_callbacks cb 0x7ff2c0031b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:49.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.145+0000 7ff2d0eec640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7a7606460 0x55f7a7a392e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:49.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.145+0000 7ff2d0eec640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7a7606460 0x55f7a7a392e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43566/0 (socket says 172.21.15.117:43566) 2024-09-21T06:04:49.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.145+0000 7ff2d0eec640 1 -- 172.21.15.117:0/4123729624 learned_addr learned my addr 172.21.15.117:0/4123729624 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:49.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.145+0000 7ff2d0eec640 1 --2- 172.21.15.117:0/4123729624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7a75e91b0 0x55f7a79d9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:49.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.145+0000 7ff2d16ed640 1 --2- 172.21.15.117:0/4123729624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7a75f1390 0x55f7a79d9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:49.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.145+0000 7ff2d0eec640 1 -- 172.21.15.117:0/4123729624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7a75e91b0 msgr2=0x55f7a79d9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:49.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.145+0000 7ff2d0eec640 1 --2- 172.21.15.117:0/4123729624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7a75e91b0 0x55f7a79d9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:49.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.145+0000 7ff2d0eec640 1 -- 172.21.15.117:0/4123729624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7a75f1390 msgr2=0x55f7a79d9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:49.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.145+0000 7ff2d0eec640 1 --2- 172.21.15.117:0/4123729624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7a75f1390 0x55f7a79d9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:49.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.145+0000 7ff2d0eec640 1 -- 172.21.15.117:0/4123729624 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7a7a39950 con 0x55f7a7606460 2024-09-21T06:04:49.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.145+0000 7ff2d0eec640 1 --2- 172.21.15.117:0/4123729624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7a7606460 0x55f7a7a392e0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0030a20 tx=0x7ff2c0000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f22be3f27658c63 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:49.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.145+0000 7ff2b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:04:49.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.146+0000 7ff2b0ff9640 1 -- 172.21.15.117:0/4123729624 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0050070 con 0x55f7a7606460 2024-09-21T06:04:49.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.146+0000 7ff2b0ff9640 1 -- 172.21.15.117:0/4123729624 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7ff2c0004780 con 0x55f7a7606460 2024-09-21T06:04:49.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.146+0000 7ff2d9ec5740 1 -- 172.21.15.117:0/4123729624 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2c005f020 con 0x55f7a7606460 2024-09-21T06:04:49.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.146+0000 7ff2b0ff9640 1 -- 172.21.15.117:0/4123729624 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff2c006d070 con 0x55f7a7606460 2024-09-21T06:04:49.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.146+0000 7ff2d9ec5740 10 client.16821 fetch_fsmap learned FSMap version 29 2024-09-21T06:04:49.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.146+0000 7ff2d9ec5740 1 -- 172.21.15.117:0/4123729624 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f7a7a40cb0 con 0x55f7a7606460 2024-09-21T06:04:49.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.146+0000 7ff2b0ff9640 1 -- 172.21.15.117:0/4123729624 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2c005f020 con 0x55f7a7606460 2024-09-21T06:04:49.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.146+0000 7ff2b0ff9640 1 -- 172.21.15.117:0/4123729624 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff2c0066040 con 0x55f7a7606460 2024-09-21T06:04:49.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.146+0000 7ff2d9ec5740 10 client.16821 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:04:49.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.146+0000 7ff2d9ec5740 10 client.16821 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T06:04:49.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.146+0000 7ff2d9ec5740 20 client.16821 populate_metadata read hostname 'smithi117' 2024-09-21T06:04:49.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.146+0000 7ff2d9ec5740 1 --2- 172.21.15.117:0/4123729624 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55f7a7612070 0x55f7a7a43360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:49.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.146+0000 7ff2d9ec5740 4 client.16821 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:04:49.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.146+0000 7ff2d9ec5740 1 -- 172.21.15.117:0/4123729624 --> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f7a792d8a0 con 0x55f7a7612070 2024-09-21T06:04:49.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.147+0000 7ff2d16ed640 1 --2- 172.21.15.117:0/4123729624 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55f7a7612070 0x55f7a7a43360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:49.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.147+0000 7ff2b0ff9640 10 client.16821 ms_handle_connect on v2:172.21.15.192:6834/3530688860 2024-09-21T06:04:49.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.147+0000 7ff2d16ed640 1 --2- 172.21.15.117:0/4123729624 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55f7a7612070 0x55f7a7a43360 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ccbe0523eb924ae9 server_cookie=4c3de44a35b73031 in_seq=0 out_seq=0 2024-09-21T06:04:49.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.147+0000 7ff2b0ff9640 1 -- 172.21.15.117:0/4123729624 <== mds.3 v2:172.21.15.192:6834/3530688860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f7a792d8a0 con 0x55f7a7612070 2024-09-21T06:04:49.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.147+0000 7ff2b0ff9640 10 client.16821 handle_command_reply: tid=0 2024-09-21T06:04:49.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.147+0000 7ff2d9ec5740 1 client.16821 shutdown 2024-09-21T06:04:49.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.147+0000 7ff2d9ec5740 1 -- 172.21.15.117:0/4123729624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7a7606460 msgr2=0x55f7a7a392e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:49.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.147+0000 7ff2d9ec5740 1 --2- 172.21.15.117:0/4123729624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7a7606460 0x55f7a7a392e0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0030a20 tx=0x7ff2c0000f40 comp rx=0 tx=0).stop 2024-09-21T06:04:49.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.147+0000 7ff2d9ec5740 1 -- 172.21.15.117:0/4123729624 shutdown_connections 2024-09-21T06:04:49.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.147+0000 7ff2d9ec5740 1 --2- 172.21.15.117:0/4123729624 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55f7a7612070 0x55f7a7a43360 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:49.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.147+0000 7ff2d9ec5740 1 --2- 172.21.15.117:0/4123729624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f7a7606460 0x55f7a7a392e0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:49.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.147+0000 7ff2b0ff9640 0 client.16821 ms_handle_reset on v2:172.21.15.192:6834/3530688860 2024-09-21T06:04:49.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.147+0000 7ff2d9ec5740 1 --2- 172.21.15.117:0/4123729624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f7a75f1390 0x55f7a79d9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:49.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.147+0000 7ff2d9ec5740 1 --2- 172.21.15.117:0/4123729624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7a75e91b0 0x55f7a79d9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:49.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.147+0000 7ff2d9ec5740 1 -- 172.21.15.117:0/4123729624 >> 172.21.15.117:0/4123729624 conn(0x55f7a79c50b0 msgr2=0x55f7a79c5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:49.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.148+0000 7ff2d9ec5740 1 -- 172.21.15.117:0/4123729624 shutdown_connections 2024-09-21T06:04:49.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.148+0000 7ff2d9ec5740 1 -- 172.21.15.117:0/4123729624 wait complete. 2024-09-21T06:04:49.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.148+0000 7ff2d9ec5740 20 client.16821 trim_cache size 0 max 16384 2024-09-21T06:04:49.161 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:04:49.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.148+0000 7ff2d3951640 1 -- 172.21.15.117:0/2554788366 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff2b40b6680 msgr2=0x7ff2b40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:49.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.148+0000 7ff2d3951640 1 --2- 172.21.15.117:0/2554788366 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff2b40b6680 0x7ff2b40b8b90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff2c40e47a0 tx=0x7ff2c40035f0 comp rx=0 tx=0).stop 2024-09-21T06:04:49.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.148+0000 7ff2d3951640 1 -- 172.21.15.117:0/2554788366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2cc10de20 msgr2=0x7ff2cc103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:49.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.148+0000 7ff2d3951640 1 --2- 172.21.15.117:0/2554788366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2cc10de20 0x7ff2cc103180 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7ff2c400d950 tx=0x7ff2c400de10 comp rx=0 tx=0).stop 2024-09-21T06:04:49.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.148+0000 7ff2d3951640 1 -- 172.21.15.117:0/2554788366 shutdown_connections 2024-09-21T06:04:49.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.148+0000 7ff2d3951640 1 --2- 172.21.15.117:0/2554788366 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff2b40b6680 0x7ff2b40b8b90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:49.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.148+0000 7ff2d3951640 1 --2- 172.21.15.117:0/2554788366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2cc10de20 0x7ff2cc103180 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:49.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.148+0000 7ff2d3951640 1 --2- 172.21.15.117:0/2554788366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2cc105310 0x7ff2cc102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:49.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.148+0000 7ff2d3951640 1 --2- 172.21.15.117:0/2554788366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2cc1049c0 0x7ff2cc102700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:49.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.148+0000 7ff2d3951640 1 -- 172.21.15.117:0/2554788366 >> 172.21.15.117:0/2554788366 conn(0x7ff2cc079730 msgr2=0x7ff2cc1008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:49.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.148+0000 7ff2d3951640 1 -- 172.21.15.117:0/2554788366 shutdown_connections 2024-09-21T06:04:49.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:49.148+0000 7ff2d3951640 1 -- 172.21.15.117:0/2554788366 wait complete. 2024-09-21T06:04:50.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:49 smithi117 ceph-mon[30947]: pgmap v555: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.5 KiB/s wr, 14 op/s 2024-09-21T06:04:50.087 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T06:04:50.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:49 smithi192 ceph-mon[34700]: pgmap v555: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.5 KiB/s wr, 14 op/s 2024-09-21T06:04:50.347 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:04:50.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:49 smithi138 ceph-mon[34391]: pgmap v555: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.5 KiB/s wr, 14 op/s 2024-09-21T06:04:50.889 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:04:50.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.852+0000 7f5360b54640 1 -- 172.21.15.117:0/3417217966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f535c108d10 msgr2=0x7f535c10f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:50.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.852+0000 7f5360b54640 1 --2- 172.21.15.117:0/3417217966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f535c108d10 0x7f535c10f540 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5354009ca0 tx=0x7f535402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.852+0000 7f5360b54640 1 -- 172.21.15.117:0/3417217966 shutdown_connections 2024-09-21T06:04:50.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.852+0000 7f5360b54640 1 --2- 172.21.15.117:0/3417217966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f535c108d10 0x7f535c10f540 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.852+0000 7f5360b54640 1 --2- 172.21.15.117:0/3417217966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f535c1083c0 0x7f535c1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.852+0000 7f5360b54640 1 --2- 172.21.15.117:0/3417217966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f535c102a80 0x7f535c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.852+0000 7f5360b54640 1 -- 172.21.15.117:0/3417217966 >> 172.21.15.117:0/3417217966 conn(0x7f535c079740 msgr2=0x7f535c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:50.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.852+0000 7f5360b54640 1 -- 172.21.15.117:0/3417217966 shutdown_connections 2024-09-21T06:04:50.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.852+0000 7f5360b54640 1 -- 172.21.15.117:0/3417217966 wait complete. 2024-09-21T06:04:50.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.853+0000 7f5360b54640 1 Processor -- start 2024-09-21T06:04:50.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.853+0000 7f5360b54640 1 -- start start 2024-09-21T06:04:50.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.853+0000 7f5360b54640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f535c102a80 0x7f535c199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:50.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.853+0000 7f5360b54640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f535c1083c0 0x7f535c199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:50.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.853+0000 7f5360b54640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f535c108d10 0x7f535c19ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:50.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.853+0000 7f5360b54640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f535c1129e0 con 0x7f535c102a80 2024-09-21T06:04:50.890 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.853+0000 7f5360b54640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f535c112860 con 0x7f535c1083c0 2024-09-21T06:04:50.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.853+0000 7f5360b54640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f535c112b60 con 0x7f535c108d10 2024-09-21T06:04:50.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.853+0000 7f535a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f535c102a80 0x7f535c199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:50.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.853+0000 7f535a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f535c102a80 0x7f535c199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43586/0 (socket says 172.21.15.117:43586) 2024-09-21T06:04:50.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.853+0000 7f535a59c640 1 -- 172.21.15.117:0/1680943469 learned_addr learned my addr 172.21.15.117:0/1680943469 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:50.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.853+0000 7f5353fff640 1 --2- 172.21.15.117:0/1680943469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f535c1083c0 0x7f535c199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:50.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.854+0000 7f535ad9d640 1 --2- 172.21.15.117:0/1680943469 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f535c108d10 0x7f535c19ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:50.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.854+0000 7f535a59c640 1 -- 172.21.15.117:0/1680943469 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f535c108d10 msgr2=0x7f535c19ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:50.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.854+0000 7f535a59c640 1 --2- 172.21.15.117:0/1680943469 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f535c108d10 0x7f535c19ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.854+0000 7f535a59c640 1 -- 172.21.15.117:0/1680943469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f535c1083c0 msgr2=0x7f535c199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:50.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.854+0000 7f535a59c640 1 --2- 172.21.15.117:0/1680943469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f535c1083c0 0x7f535c199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.854+0000 7f535a59c640 1 -- 172.21.15.117:0/1680943469 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f535c19f300 con 0x7f535c102a80 2024-09-21T06:04:50.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.854+0000 7f535a59c640 1 --2- 172.21.15.117:0/1680943469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f535c102a80 0x7f535c199390 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f534c00d950 tx=0x7f534c00dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:50.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.854+0000 7f53537fe640 1 -- 172.21.15.117:0/1680943469 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c017070 con 0x7f535c102a80 2024-09-21T06:04:50.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.854+0000 7f53537fe640 1 -- 172.21.15.117:0/1680943469 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f534c00de80 con 0x7f535c102a80 2024-09-21T06:04:50.891 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.854+0000 7f5360b54640 1 -- 172.21.15.117:0/1680943469 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f535c19f5f0 con 0x7f535c102a80 2024-09-21T06:04:50.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.854+0000 7f53537fe640 1 -- 172.21.15.117:0/1680943469 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c00f620 con 0x7f535c102a80 2024-09-21T06:04:50.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.854+0000 7f5360b54640 1 -- 172.21.15.117:0/1680943469 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f535c06bb40 con 0x7f535c102a80 2024-09-21T06:04:50.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.857+0000 7f53670c8740 1 Processor -- start 2024-09-21T06:04:50.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.857+0000 7f53537fe640 1 -- 172.21.15.117:0/1680943469 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f534c009c30 con 0x7f535c102a80 2024-09-21T06:04:50.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.857+0000 7f53670c8740 1 -- start start 2024-09-21T06:04:50.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.857+0000 7f53670c8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd71f21070 0x55bd722c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:50.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.857+0000 7f53670c8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd71ef81b0 0x55bd722c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:50.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.857+0000 7f53670c8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd71f00390 0x55bd722c9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:50.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.857+0000 7f53670c8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bd7223eea0 con 0x55bd71f00390 2024-09-21T06:04:50.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.857+0000 7f53670c8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bd7222e2d0 con 0x55bd71ef81b0 2024-09-21T06:04:50.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.857+0000 7f53670c8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55bd722320e0 con 0x55bd71f21070 2024-09-21T06:04:50.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.857+0000 7f5353fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd71f00390 0x55bd722c9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:50.892 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.857+0000 7f5353fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd71f00390 0x55bd722c9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43600/0 (socket says 172.21.15.117:43600) 2024-09-21T06:04:50.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.857+0000 7f5353fff640 1 -- 172.21.15.117:0/2936958374 learned_addr learned my addr 172.21.15.117:0/2936958374 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:50.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.857+0000 7f535ad9d640 1 --2- 172.21.15.117:0/2936958374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd71f21070 0x55bd722c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:50.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.857+0000 7f535a59c640 1 --2- 172.21.15.117:0/2936958374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd71ef81b0 0x55bd722c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:50.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.857+0000 7f5353fff640 1 -- 172.21.15.117:0/2936958374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd71f21070 msgr2=0x55bd722c86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:50.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.858+0000 7f5353fff640 1 --2- 172.21.15.117:0/2936958374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd71f21070 0x55bd722c86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.858+0000 7f5353fff640 1 -- 172.21.15.117:0/2936958374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd71ef81b0 msgr2=0x55bd722c8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:50.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.858+0000 7f5353fff640 1 --2- 172.21.15.117:0/2936958374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd71ef81b0 0x55bd722c8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.858+0000 7f5353fff640 1 -- 172.21.15.117:0/2936958374 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd71f8c650 con 0x55bd71f00390 2024-09-21T06:04:50.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.858+0000 7f535ad9d640 1 --2- 172.21.15.117:0/2936958374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd71f21070 0x55bd722c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:04:50.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.858+0000 7f53537fe640 1 --2- 172.21.15.117:0/1680943469 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f53480b8910 0x7f53480bae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:50.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.858+0000 7f5353fff640 1 --2- 172.21.15.117:0/2936958374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd71f00390 0x55bd722c9120 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f5344009a00 tx=0x7f5344009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a865cdc5c5bc4583 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:50.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.858+0000 7f535ad9d640 1 --2- 172.21.15.117:0/1680943469 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f53480b8910 0x7f53480bae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:50.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.858+0000 7f5350ff9640 1 -- 172.21.15.117:0/2936958374 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5344017070 con 0x55bd71f00390 2024-09-21T06:04:50.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.858+0000 7f53537fe640 1 -- 172.21.15.117:0/1680943469 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f534c0e72c0 con 0x7f535c102a80 2024-09-21T06:04:50.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.858+0000 7f5350ff9640 1 -- 172.21.15.117:0/2936958374 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5344013470 con 0x55bd71f00390 2024-09-21T06:04:50.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.858+0000 7f5350ff9640 1 -- 172.21.15.117:0/2936958374 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5344012630 con 0x55bd71f00390 2024-09-21T06:04:50.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.859+0000 7f53670c8740 1 -- 172.21.15.117:0/2936958374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd71f00390 msgr2=0x55bd722c9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:50.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.859+0000 7f53670c8740 1 --2- 172.21.15.117:0/2936958374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd71f00390 0x55bd722c9120 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f5344009a00 tx=0x7f5344009ec0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.859+0000 7f535ad9d640 1 --2- 172.21.15.117:0/1680943469 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f53480b8910 0x7f53480bae20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f53540094e0 tx=0x7f53540093f0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:50.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.859+0000 7f53670c8740 1 -- 172.21.15.117:0/2936958374 shutdown_connections 2024-09-21T06:04:50.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.859+0000 7f53670c8740 1 --2- 172.21.15.117:0/2936958374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd71f00390 0x55bd722c9120 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.859+0000 7f53670c8740 1 --2- 172.21.15.117:0/2936958374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd71ef81b0 0x55bd722c8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.859+0000 7f53670c8740 1 --2- 172.21.15.117:0/2936958374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd71f21070 0x55bd722c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.859+0000 7f53670c8740 1 -- 172.21.15.117:0/2936958374 >> 172.21.15.117:0/2936958374 conn(0x55bd71f15460 msgr2=0x55bd722c6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:50.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.859+0000 7f53670c8740 1 -- 172.21.15.117:0/2936958374 shutdown_connections 2024-09-21T06:04:50.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.859+0000 7f53670c8740 1 -- 172.21.15.117:0/2936958374 wait complete. 2024-09-21T06:04:50.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.859+0000 7f53670c8740 1 -- start start 2024-09-21T06:04:50.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.859+0000 7f53670c8740 1 Processor -- start 2024-09-21T06:04:50.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.860+0000 7f53670c8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd71ef81b0 0x55bd722d3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:50.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.860+0000 7f53670c8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd71f00390 0x55bd722d4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:50.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.860+0000 7f53670c8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd71f21070 0x55bd722e0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:50.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.860+0000 7f53670c8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bd7208f700 con 0x55bd71f21070 2024-09-21T06:04:50.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.860+0000 7f53670c8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bd721c9160 con 0x55bd71ef81b0 2024-09-21T06:04:50.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.860+0000 7f53670c8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55bd722d6e80 con 0x55bd71f00390 2024-09-21T06:04:50.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.860+0000 7f53670c8740 10 client.0 _ll_register_callbacks cb 0x55bd721e29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:50.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.860+0000 7f535ad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd71f21070 0x55bd722e0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:50.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.860+0000 7f535ad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd71f21070 0x55bd722e0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43616/0 (socket says 172.21.15.117:43616) 2024-09-21T06:04:50.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.860+0000 7f535ad9d640 1 -- 172.21.15.117:0/2417188408 learned_addr learned my addr 172.21.15.117:0/2417188408 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:50.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.860+0000 7f535a59c640 1 --2- 172.21.15.117:0/2417188408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd71ef81b0 0x55bd722d3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:50.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.860+0000 7f5353fff640 1 --2- 172.21.15.117:0/2417188408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd71f00390 0x55bd722d4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:50.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.860+0000 7f535ad9d640 1 -- 172.21.15.117:0/2417188408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd71f00390 msgr2=0x55bd722d4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:50.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.860+0000 7f535ad9d640 1 --2- 172.21.15.117:0/2417188408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd71f00390 0x55bd722d4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.860+0000 7f535ad9d640 1 -- 172.21.15.117:0/2417188408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd71ef81b0 msgr2=0x55bd722d3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:50.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.860+0000 7f535ad9d640 1 --2- 172.21.15.117:0/2417188408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd71ef81b0 0x55bd722d3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.860+0000 7f535ad9d640 1 -- 172.21.15.117:0/2417188408 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd721f6c90 con 0x55bd71f21070 2024-09-21T06:04:50.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.860+0000 7f535a59c640 1 --2- 172.21.15.117:0/2417188408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd71ef81b0 0x55bd722d3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:04:50.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.860+0000 7f535ad9d640 1 --2- 172.21.15.117:0/2417188408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd71f21070 0x55bd722e0170 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f53540316d0 tx=0x7f53540095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=371f7c7c0bce2baf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:50.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.860+0000 7f5341ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:04:50.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.861+0000 7f5341ffb640 1 -- 172.21.15.117:0/2417188408 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535404e070 con 0x55bd71f21070 2024-09-21T06:04:50.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.861+0000 7f5341ffb640 1 -- 172.21.15.117:0/2417188408 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f53540045c0 con 0x55bd71f21070 2024-09-21T06:04:50.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.861+0000 7f53670c8740 1 -- 172.21.15.117:0/2417188408 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5344013470 con 0x55bd71f21070 2024-09-21T06:04:50.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.861+0000 7f5341ffb640 1 -- 172.21.15.117:0/2417188408 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5354031900 con 0x55bd71f21070 2024-09-21T06:04:50.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.861+0000 7f53670c8740 10 client.16833 fetch_fsmap learned FSMap version 29 2024-09-21T06:04:50.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.861+0000 7f53670c8740 1 -- 172.21.15.117:0/2417188408 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bd722e2b60 con 0x55bd71f21070 2024-09-21T06:04:50.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.861+0000 7f5341ffb640 1 -- 172.21.15.117:0/2417188408 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5344013470 con 0x55bd71f21070 2024-09-21T06:04:50.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.861+0000 7f5341ffb640 1 -- 172.21.15.117:0/2417188408 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5354043040 con 0x55bd71f21070 2024-09-21T06:04:50.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.861+0000 7f53670c8740 10 client.16833 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:04:50.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.861+0000 7f53670c8740 10 client.16833 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:04:50.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.861+0000 7f53670c8740 20 client.16833 populate_metadata read hostname 'smithi117' 2024-09-21T06:04:50.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.861+0000 7f53670c8740 1 --2- 172.21.15.117:0/2417188408 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55bd71edf9f0 0x55bd722e6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:50.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.861+0000 7f53670c8740 4 client.16833 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:04:50.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.861+0000 7f53670c8740 1 -- 172.21.15.117:0/2417188408 --> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd72240df0 con 0x55bd71edf9f0 2024-09-21T06:04:50.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.862+0000 7f535a59c640 1 --2- 172.21.15.117:0/2417188408 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55bd71edf9f0 0x55bd722e6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:50.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.862+0000 7f535a59c640 1 --2- 172.21.15.117:0/2417188408 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55bd71edf9f0 0x55bd722e6030 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c4f87faf192ed01e server_cookie=8383eb7685a20633 in_seq=0 out_seq=0 2024-09-21T06:04:50.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.862+0000 7f5341ffb640 10 client.16833 ms_handle_connect on v2:172.21.15.192:6836/2683860842 2024-09-21T06:04:50.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.863+0000 7f5341ffb640 1 -- 172.21.15.117:0/2417188408 <== mds.2 v2:172.21.15.192:6836/2683860842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bd72240df0 con 0x55bd71edf9f0 2024-09-21T06:04:50.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.863+0000 7f5341ffb640 10 client.16833 handle_command_reply: tid=0 2024-09-21T06:04:50.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.863+0000 7f53670c8740 1 client.16833 shutdown 2024-09-21T06:04:50.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.863+0000 7f53670c8740 1 -- 172.21.15.117:0/2417188408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd71f21070 msgr2=0x55bd722e0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:50.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.863+0000 7f53670c8740 1 --2- 172.21.15.117:0/2417188408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd71f21070 0x55bd722e0170 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f53540316d0 tx=0x7f53540095c0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.863+0000 7f53670c8740 1 -- 172.21.15.117:0/2417188408 shutdown_connections 2024-09-21T06:04:50.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.863+0000 7f53670c8740 1 --2- 172.21.15.117:0/2417188408 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55bd71edf9f0 0x55bd722e6030 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.863+0000 7f53670c8740 1 --2- 172.21.15.117:0/2417188408 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd71f21070 0x55bd722e0170 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.863+0000 7f53670c8740 1 --2- 172.21.15.117:0/2417188408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd71f00390 0x55bd722d4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.863+0000 7f53670c8740 1 --2- 172.21.15.117:0/2417188408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd71ef81b0 0x55bd722d3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.863+0000 7f5341ffb640 0 client.16833 ms_handle_reset on v2:172.21.15.192:6836/2683860842 2024-09-21T06:04:50.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.863+0000 7f53670c8740 1 -- 172.21.15.117:0/2417188408 >> 172.21.15.117:0/2417188408 conn(0x55bd71f15460 msgr2=0x55bd722cb830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:50.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.863+0000 7f53670c8740 1 -- 172.21.15.117:0/2417188408 shutdown_connections 2024-09-21T06:04:50.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.863+0000 7f53670c8740 1 -- 172.21.15.117:0/2417188408 wait complete. 2024-09-21T06:04:50.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.863+0000 7f53670c8740 20 client.16833 trim_cache size 0 max 16384 2024-09-21T06:04:50.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.876+0000 7f53670c8740 1 Processor -- start 2024-09-21T06:04:50.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.876+0000 7f53670c8740 1 -- start start 2024-09-21T06:04:50.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.876+0000 7f53670c8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd71ef81b0 0x55bd722ca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:50.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.876+0000 7f53670c8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd71f00390 0x55bd722d5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:50.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.876+0000 7f53670c8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd71f21070 0x55bd722e0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:50.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.876+0000 7f53670c8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f535c1ad3d0 con 0x55bd71f21070 2024-09-21T06:04:50.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.876+0000 7f53670c8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f535c1aef10 con 0x55bd71f00390 2024-09-21T06:04:50.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.876+0000 7f53670c8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f535c1adaa0 con 0x55bd71ef81b0 2024-09-21T06:04:50.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.876+0000 7f535ad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd71f21070 0x55bd722e0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:50.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.877+0000 7f535ad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd71f21070 0x55bd722e0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43626/0 (socket says 172.21.15.117:43626) 2024-09-21T06:04:50.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.877+0000 7f535ad9d640 1 -- 172.21.15.117:0/1215740891 learned_addr learned my addr 172.21.15.117:0/1215740891 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:50.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.877+0000 7f5353fff640 1 --2- 172.21.15.117:0/1215740891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd71f00390 0x55bd722d5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:50.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.877+0000 7f535ad9d640 1 -- 172.21.15.117:0/1215740891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd71ef81b0 msgr2=0x55bd722ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:50.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.877+0000 7f535a59c640 1 --2- 172.21.15.117:0/1215740891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd71ef81b0 0x55bd722ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:50.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.877+0000 7f535ad9d640 1 --2- 172.21.15.117:0/1215740891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd71ef81b0 0x55bd722ca090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.877+0000 7f535ad9d640 1 -- 172.21.15.117:0/1215740891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd71f00390 msgr2=0x55bd722d5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:50.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.877+0000 7f535ad9d640 1 --2- 172.21.15.117:0/1215740891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd71f00390 0x55bd722d5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.877+0000 7f535ad9d640 1 -- 172.21.15.117:0/1215740891 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd722ce9d0 con 0x55bd71f21070 2024-09-21T06:04:50.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.877+0000 7f5353fff640 1 --2- 172.21.15.117:0/1215740891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd71f00390 0x55bd722d5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:50.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.877+0000 7f535a59c640 1 --2- 172.21.15.117:0/1215740891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd71ef81b0 0x55bd722ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:50.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.877+0000 7f535ad9d640 1 --2- 172.21.15.117:0/1215740891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd71f21070 0x55bd722e0070 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f535402ade0 tx=0x7f535402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=839b8a90a2e051cf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:50.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.877+0000 7f5343fff640 1 -- 172.21.15.117:0/1215740891 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535404e070 con 0x55bd71f21070 2024-09-21T06:04:50.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.877+0000 7f5343fff640 1 -- 172.21.15.117:0/1215740891 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5354036040 con 0x55bd71f21070 2024-09-21T06:04:50.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.877+0000 7f5343fff640 1 -- 172.21.15.117:0/1215740891 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535402fb20 con 0x55bd71f21070 2024-09-21T06:04:50.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.878+0000 7f53670c8740 1 -- 172.21.15.117:0/1215740891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd71f21070 msgr2=0x55bd722e0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:50.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.878+0000 7f53670c8740 1 --2- 172.21.15.117:0/1215740891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd71f21070 0x55bd722e0070 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f535402ade0 tx=0x7f535402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.878+0000 7f53670c8740 1 -- 172.21.15.117:0/1215740891 shutdown_connections 2024-09-21T06:04:50.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.878+0000 7f53670c8740 1 --2- 172.21.15.117:0/1215740891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd71f21070 0x55bd722e0070 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.878+0000 7f53670c8740 1 --2- 172.21.15.117:0/1215740891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd71f00390 0x55bd722d5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.878+0000 7f53670c8740 1 --2- 172.21.15.117:0/1215740891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd71ef81b0 0x55bd722ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.878+0000 7f53670c8740 1 -- 172.21.15.117:0/1215740891 >> 172.21.15.117:0/1215740891 conn(0x55bd71f15460 msgr2=0x55bd722d5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:50.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.878+0000 7f53670c8740 1 -- 172.21.15.117:0/1215740891 shutdown_connections 2024-09-21T06:04:50.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.878+0000 7f53670c8740 1 -- 172.21.15.117:0/1215740891 wait complete. 2024-09-21T06:04:50.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.878+0000 7f53670c8740 1 -- start start 2024-09-21T06:04:50.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.878+0000 7f53670c8740 1 Processor -- start 2024-09-21T06:04:50.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.879+0000 7f53670c8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd71ef81b0 0x55bd722d8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:50.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.879+0000 7f53670c8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd71f00390 0x55bd722d8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:50.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.879+0000 7f53670c8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd71f21070 0x55bd7234f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:50.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.879+0000 7f53670c8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f535c1ae840 con 0x55bd71f00390 2024-09-21T06:04:50.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.879+0000 7f53670c8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f535c1ae170 con 0x55bd71ef81b0 2024-09-21T06:04:50.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.879+0000 7f53670c8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f535c1ab3d0 con 0x55bd71f21070 2024-09-21T06:04:50.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.879+0000 7f53670c8740 10 client.0 _ll_register_callbacks cb 0x7f5354009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:50.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.879+0000 7f5353fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd71f00390 0x55bd722d8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:50.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.879+0000 7f5353fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd71f00390 0x55bd722d8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43634/0 (socket says 172.21.15.117:43634) 2024-09-21T06:04:50.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.879+0000 7f5353fff640 1 -- 172.21.15.117:0/774156079 learned_addr learned my addr 172.21.15.117:0/774156079 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:50.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.879+0000 7f535a59c640 1 --2- 172.21.15.117:0/774156079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd71ef81b0 0x55bd722d8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:50.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.879+0000 7f535ad9d640 1 --2- 172.21.15.117:0/774156079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd71f21070 0x55bd7234f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:50.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.879+0000 7f5353fff640 1 -- 172.21.15.117:0/774156079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd71f21070 msgr2=0x55bd7234f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:50.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.879+0000 7f5353fff640 1 --2- 172.21.15.117:0/774156079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd71f21070 0x55bd7234f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.879+0000 7f5353fff640 1 -- 172.21.15.117:0/774156079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd71ef81b0 msgr2=0x55bd722d8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:50.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.879+0000 7f5353fff640 1 --2- 172.21.15.117:0/774156079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd71ef81b0 0x55bd722d8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.879+0000 7f5353fff640 1 -- 172.21.15.117:0/774156079 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd7234ff30 con 0x55bd71f00390 2024-09-21T06:04:50.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.879+0000 7f5353fff640 1 --2- 172.21.15.117:0/774156079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd71f00390 0x55bd722d8980 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f5344019000 tx=0x7f53440093a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=425ea6081e5360fd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:50.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.879+0000 7f5341ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:04:50.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.879+0000 7f5341ffb640 1 -- 172.21.15.117:0/774156079 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5344017040 con 0x55bd71f00390 2024-09-21T06:04:50.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.879+0000 7f5341ffb640 1 -- 172.21.15.117:0/774156079 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f5344013a80 con 0x55bd71f00390 2024-09-21T06:04:50.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.879+0000 7f53670c8740 1 -- 172.21.15.117:0/774156079 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5354036040 con 0x55bd71f00390 2024-09-21T06:04:50.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.880+0000 7f5341ffb640 1 -- 172.21.15.117:0/774156079 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5344013c80 con 0x55bd71f00390 2024-09-21T06:04:50.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.880+0000 7f53670c8740 10 client.16839 fetch_fsmap learned FSMap version 29 2024-09-21T06:04:50.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.880+0000 7f53670c8740 1 -- 172.21.15.117:0/774156079 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bd72350ee0 con 0x55bd71f00390 2024-09-21T06:04:50.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.880+0000 7f5341ffb640 1 -- 172.21.15.117:0/774156079 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5354036040 con 0x55bd71f00390 2024-09-21T06:04:50.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.880+0000 7f5341ffb640 1 -- 172.21.15.117:0/774156079 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5344023070 con 0x55bd71f00390 2024-09-21T06:04:50.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.880+0000 7f53670c8740 10 client.16839 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:04:50.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.880+0000 7f53670c8740 10 client.16839 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:04:50.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.880+0000 7f53670c8740 20 client.16839 populate_metadata read hostname 'smithi117' 2024-09-21T06:04:50.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.880+0000 7f53670c8740 1 --2- 172.21.15.117:0/774156079 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55bd71eb4da0 0x55bd723533c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:50.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.880+0000 7f53670c8740 4 client.16839 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:04:50.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.880+0000 7f53670c8740 1 -- 172.21.15.117:0/774156079 --> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bd7223c8a0 con 0x55bd71eb4da0 2024-09-21T06:04:50.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.880+0000 7f535a59c640 1 --2- 172.21.15.117:0/774156079 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55bd71eb4da0 0x55bd723533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:50.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.881+0000 7f5341ffb640 10 client.16839 ms_handle_connect on v2:172.21.15.192:6836/2683860842 2024-09-21T06:04:50.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.881+0000 7f535a59c640 1 --2- 172.21.15.117:0/774156079 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55bd71eb4da0 0x55bd723533c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=583bf4ddafa3c347 server_cookie=96bcf414c387b33c in_seq=0 out_seq=0 2024-09-21T06:04:50.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.881+0000 7f5341ffb640 1 -- 172.21.15.117:0/774156079 <== mds.2 v2:172.21.15.192:6836/2683860842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bd7223c8a0 con 0x55bd71eb4da0 2024-09-21T06:04:50.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.881+0000 7f5341ffb640 10 client.16839 handle_command_reply: tid=0 2024-09-21T06:04:50.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.881+0000 7f53670c8740 1 client.16839 shutdown 2024-09-21T06:04:50.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.881+0000 7f53670c8740 1 -- 172.21.15.117:0/774156079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd71f00390 msgr2=0x55bd722d8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:50.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.881+0000 7f53670c8740 1 --2- 172.21.15.117:0/774156079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd71f00390 0x55bd722d8980 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f5344019000 tx=0x7f53440093a0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.881+0000 7f53670c8740 1 -- 172.21.15.117:0/774156079 shutdown_connections 2024-09-21T06:04:50.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.881+0000 7f53670c8740 1 --2- 172.21.15.117:0/774156079 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55bd71eb4da0 0x55bd723533c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.881+0000 7f53670c8740 1 --2- 172.21.15.117:0/774156079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bd71f00390 0x55bd722d8980 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.881+0000 7f53670c8740 1 --2- 172.21.15.117:0/774156079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bd71f21070 0x55bd7234f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.881+0000 7f5341ffb640 0 client.16839 ms_handle_reset on v2:172.21.15.192:6836/2683860842 2024-09-21T06:04:50.903 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.881+0000 7f53670c8740 1 --2- 172.21.15.117:0/774156079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bd71ef81b0 0x55bd722d8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.881+0000 7f53670c8740 1 -- 172.21.15.117:0/774156079 >> 172.21.15.117:0/774156079 conn(0x55bd71f15460 msgr2=0x55bd722d5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:50.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.881+0000 7f53670c8740 1 -- 172.21.15.117:0/774156079 shutdown_connections 2024-09-21T06:04:50.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.881+0000 7f53670c8740 1 -- 172.21.15.117:0/774156079 wait complete. 2024-09-21T06:04:50.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.881+0000 7f53670c8740 20 client.16839 trim_cache size 0 max 16384 2024-09-21T06:04:50.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.882+0000 7f5360b54640 1 -- 172.21.15.117:0/1680943469 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f53480b8910 msgr2=0x7f53480bae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:50.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.882+0000 7f5360b54640 1 --2- 172.21.15.117:0/1680943469 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f53480b8910 0x7f53480bae20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f53540094e0 tx=0x7f53540093f0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.882+0000 7f5360b54640 1 -- 172.21.15.117:0/1680943469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f535c102a80 msgr2=0x7f535c199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:50.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.882+0000 7f5360b54640 1 --2- 172.21.15.117:0/1680943469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f535c102a80 0x7f535c199390 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f534c00d950 tx=0x7f534c00dd50 comp rx=0 tx=0).stop 2024-09-21T06:04:50.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.882+0000 7f5360b54640 1 -- 172.21.15.117:0/1680943469 shutdown_connections 2024-09-21T06:04:50.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.882+0000 7f5360b54640 1 --2- 172.21.15.117:0/1680943469 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f53480b8910 0x7f53480bae20 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.882+0000 7f5360b54640 1 --2- 172.21.15.117:0/1680943469 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f535c108d10 0x7f535c19ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.882+0000 7f5360b54640 1 --2- 172.21.15.117:0/1680943469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f535c1083c0 0x7f535c199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.882+0000 7f5360b54640 1 --2- 172.21.15.117:0/1680943469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f535c102a80 0x7f535c199390 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:50.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.882+0000 7f5360b54640 1 -- 172.21.15.117:0/1680943469 >> 172.21.15.117:0/1680943469 conn(0x7f535c079740 msgr2=0x7f535c105260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:50.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.882+0000 7f5360b54640 1 -- 172.21.15.117:0/1680943469 shutdown_connections 2024-09-21T06:04:50.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:50.882+0000 7f5360b54640 1 -- 172.21.15.117:0/1680943469 wait complete. 2024-09-21T06:04:51.555 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T06:04:51.804 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:04:52.180 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:51 smithi117 ceph-mon[30947]: pgmap v556: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-09-21T06:04:52.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:51 smithi192 ceph-mon[34700]: pgmap v556: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-09-21T06:04:52.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.270+0000 7f88aa424640 1 -- 172.21.15.117:0/401517229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88a4102a60 msgr2=0x7f88a4102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:52.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.270+0000 7f88aa424640 1 --2- 172.21.15.117:0/401517229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88a4102a60 0x7f88a4102e70 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f8894009900 tx=0x7f889402a410 comp rx=0 tx=0).stop 2024-09-21T06:04:52.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.270+0000 7f88aa424640 1 -- 172.21.15.117:0/401517229 shutdown_connections 2024-09-21T06:04:52.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.270+0000 7f88aa424640 1 --2- 172.21.15.117:0/401517229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88a4108cf0 0x7f88a410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:52.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.270+0000 7f88aa424640 1 --2- 172.21.15.117:0/401517229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88a41083a0 0x7f88a41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:52.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.270+0000 7f88aa424640 1 --2- 172.21.15.117:0/401517229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88a4102a60 0x7f88a4102e70 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:52.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.270+0000 7f88aa424640 1 -- 172.21.15.117:0/401517229 >> 172.21.15.117:0/401517229 conn(0x7f88a4079730 msgr2=0x7f88a4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:52.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.270+0000 7f88aa424640 1 -- 172.21.15.117:0/401517229 shutdown_connections 2024-09-21T06:04:52.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.270+0000 7f88aa424640 1 -- 172.21.15.117:0/401517229 wait complete. 2024-09-21T06:04:52.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.270+0000 7f88aa424640 1 Processor -- start 2024-09-21T06:04:52.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.270+0000 7f88aa424640 1 -- start start 2024-09-21T06:04:52.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.271+0000 7f88aa424640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88a4102a60 0x7f88a419b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:52.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.271+0000 7f88aa424640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88a41083a0 0x7f88a419b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:52.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.271+0000 7f88aa424640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88a4108cf0 0x7f88a419bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:52.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.271+0000 7f88aa424640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f88a4112960 con 0x7f88a41083a0 2024-09-21T06:04:52.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.271+0000 7f88aa424640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88a41127e0 con 0x7f88a4108cf0 2024-09-21T06:04:52.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.271+0000 7f88aa424640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f88a4112ae0 con 0x7f88a4102a60 2024-09-21T06:04:52.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.271+0000 7f88a37fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88a41083a0 0x7f88a419b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:52.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.271+0000 7f88a37fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88a41083a0 0x7f88a419b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43650/0 (socket says 172.21.15.117:43650) 2024-09-21T06:04:52.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.271+0000 7f88a37fe640 1 -- 172.21.15.117:0/3975683963 learned_addr learned my addr 172.21.15.117:0/3975683963 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:52.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.271+0000 7f88a3fff640 1 --2- 172.21.15.117:0/3975683963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88a4102a60 0x7f88a419b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:52.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.271+0000 7f88a89c1640 1 --2- 172.21.15.117:0/3975683963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88a4108cf0 0x7f88a419bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:52.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.271+0000 7f88a37fe640 1 -- 172.21.15.117:0/3975683963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88a4102a60 msgr2=0x7f88a419b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:52.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.271+0000 7f88a37fe640 1 --2- 172.21.15.117:0/3975683963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88a4102a60 0x7f88a419b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:52.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.271+0000 7f88a37fe640 1 -- 172.21.15.117:0/3975683963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88a4108cf0 msgr2=0x7f88a419bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:52.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.271+0000 7f88a37fe640 1 --2- 172.21.15.117:0/3975683963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88a4108cf0 0x7f88a419bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:52.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.271+0000 7f88a37fe640 1 -- 172.21.15.117:0/3975683963 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a41a6500 con 0x7f88a41083a0 2024-09-21T06:04:52.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.271+0000 7f88a37fe640 1 --2- 172.21.15.117:0/3975683963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88a41083a0 0x7f88a419b730 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f8898005fd0 tx=0x7f889800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:52.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.272+0000 7f88a17fa640 1 -- 172.21.15.117:0/3975683963 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8898017070 con 0x7f88a41083a0 2024-09-21T06:04:52.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.272+0000 7f88a17fa640 1 -- 172.21.15.117:0/3975683963 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8898009c30 con 0x7f88a41083a0 2024-09-21T06:04:52.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.272+0000 7f88aa424640 1 -- 172.21.15.117:0/3975683963 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88a41a67f0 con 0x7f88a41083a0 2024-09-21T06:04:52.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.272+0000 7f88a17fa640 1 -- 172.21.15.117:0/3975683963 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8898003d30 con 0x7f88a41083a0 2024-09-21T06:04:52.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.272+0000 7f88aa424640 1 -- 172.21.15.117:0/3975683963 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88a41a6d30 con 0x7f88a41083a0 2024-09-21T06:04:52.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.274+0000 7f88a17fa640 1 -- 172.21.15.117:0/3975683963 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f8898003ed0 con 0x7f88a41083a0 2024-09-21T06:04:52.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.274+0000 7f88b0998740 1 Processor -- start 2024-09-21T06:04:52.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.274+0000 7f88b0998740 1 -- start start 2024-09-21T06:04:52.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.275+0000 7f88b0998740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55617a567070 0x55617a90e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:52.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.275+0000 7f88b0998740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55617a53e1b0 0x55617a90ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:52.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.275+0000 7f88b0998740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55617a546390 0x55617a90f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:52.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.275+0000 7f88b0998740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55617a884ea0 con 0x55617a567070 2024-09-21T06:04:52.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.275+0000 7f88b0998740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55617a8742d0 con 0x55617a53e1b0 2024-09-21T06:04:52.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.275+0000 7f88b0998740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55617a8780e0 con 0x55617a546390 2024-09-21T06:04:52.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.275+0000 7f88a89c1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55617a567070 0x55617a90e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:52.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.275+0000 7f88a89c1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55617a567070 0x55617a90e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43652/0 (socket says 172.21.15.117:43652) 2024-09-21T06:04:52.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.275+0000 7f88a89c1640 1 -- 172.21.15.117:0/3201491131 learned_addr learned my addr 172.21.15.117:0/3201491131 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:52.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.275+0000 7f88a3fff640 1 --2- 172.21.15.117:0/3201491131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55617a53e1b0 0x55617a90ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:52.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.275+0000 7f88a37fe640 1 --2- 172.21.15.117:0/3201491131 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55617a546390 0x55617a90f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:52.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.275+0000 7f88a89c1640 1 -- 172.21.15.117:0/3201491131 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55617a546390 msgr2=0x55617a90f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:52.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.275+0000 7f88a17fa640 1 --2- 172.21.15.117:0/3975683963 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f888c0b6680 0x7f888c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:52.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.275+0000 7f88a89c1640 1 --2- 172.21.15.117:0/3201491131 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55617a546390 0x55617a90f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:52.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.275+0000 7f88a89c1640 1 -- 172.21.15.117:0/3201491131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55617a53e1b0 msgr2=0x55617a90ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:52.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.275+0000 7f88a89c1640 1 --2- 172.21.15.117:0/3201491131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55617a53e1b0 0x55617a90ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:52.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.275+0000 7f88a89c1640 1 -- 172.21.15.117:0/3201491131 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55617a5d2650 con 0x55617a567070 2024-09-21T06:04:52.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.275+0000 7f88a17fa640 1 -- 172.21.15.117:0/3975683963 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f88980e73e0 con 0x7f88a41083a0 2024-09-21T06:04:52.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.275+0000 7f88a89c1640 1 --2- 172.21.15.117:0/3201491131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55617a567070 0x55617a90e6a0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f889c008d10 tx=0x7f889c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16ba7079f9a85f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:52.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.275+0000 7f88a89c1640 1 --2- 172.21.15.117:0/3975683963 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f888c0b6680 0x7f888c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:52.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.276+0000 7f888a7fc640 1 -- 172.21.15.117:0/3201491131 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c020070 con 0x55617a567070 2024-09-21T06:04:52.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.276+0000 7f888a7fc640 1 -- 172.21.15.117:0/3201491131 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f889c01ade0 con 0x55617a567070 2024-09-21T06:04:52.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.276+0000 7f888a7fc640 1 -- 172.21.15.117:0/3201491131 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c014070 con 0x55617a567070 2024-09-21T06:04:52.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.277+0000 7f88b0998740 1 -- 172.21.15.117:0/3201491131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55617a567070 msgr2=0x55617a90e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:52.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.277+0000 7f88b0998740 1 --2- 172.21.15.117:0/3201491131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55617a567070 0x55617a90e6a0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f889c008d10 tx=0x7f889c00e720 comp rx=0 tx=0).stop 2024-09-21T06:04:52.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.277+0000 7f88a89c1640 1 --2- 172.21.15.117:0/3975683963 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f888c0b6680 0x7f888c0b8b90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f889c01d000 tx=0x7f889c020000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:52.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.277+0000 7f88b0998740 1 -- 172.21.15.117:0/3201491131 shutdown_connections 2024-09-21T06:04:52.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.277+0000 7f88b0998740 1 --2- 172.21.15.117:0/3201491131 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55617a546390 0x55617a90f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:52.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.277+0000 7f88b0998740 1 --2- 172.21.15.117:0/3201491131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55617a53e1b0 0x55617a90ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:52.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.277+0000 7f88b0998740 1 --2- 172.21.15.117:0/3201491131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55617a567070 0x55617a90e6a0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:52.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.277+0000 7f88b0998740 1 -- 172.21.15.117:0/3201491131 >> 172.21.15.117:0/3201491131 conn(0x55617a55b460 msgr2=0x55617a90c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:52.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.277+0000 7f88b0998740 1 -- 172.21.15.117:0/3201491131 shutdown_connections 2024-09-21T06:04:52.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.277+0000 7f88b0998740 1 -- 172.21.15.117:0/3201491131 wait complete. 2024-09-21T06:04:52.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.277+0000 7f88b0998740 1 -- start start 2024-09-21T06:04:52.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.278+0000 7f88b0998740 1 Processor -- start 2024-09-21T06:04:52.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.278+0000 7f88b0998740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55617a53e1b0 0x55617a91e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:52.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.278+0000 7f88b0998740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55617a546390 0x55617a91e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:52.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.278+0000 7f88b0998740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55617a567070 0x55617a91eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:52.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.278+0000 7f88b0998740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55617a6d5700 con 0x55617a567070 2024-09-21T06:04:52.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.278+0000 7f88b0998740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55617a80f160 con 0x55617a546390 2024-09-21T06:04:52.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.278+0000 7f88b0998740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55617a9211e0 con 0x55617a53e1b0 2024-09-21T06:04:52.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.278+0000 7f88b0998740 10 client.0 _ll_register_callbacks cb 0x55617a8289e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:52.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.278+0000 7f88a89c1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55617a567070 0x55617a91eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:52.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.278+0000 7f88a89c1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55617a567070 0x55617a91eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43656/0 (socket says 172.21.15.117:43656) 2024-09-21T06:04:52.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.278+0000 7f88a89c1640 1 -- 172.21.15.117:0/2742294452 learned_addr learned my addr 172.21.15.117:0/2742294452 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:52.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.278+0000 7f88a3fff640 1 --2- 172.21.15.117:0/2742294452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55617a53e1b0 0x55617a91e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:52.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.278+0000 7f88a37fe640 1 --2- 172.21.15.117:0/2742294452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55617a546390 0x55617a91e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:52.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.278+0000 7f88a89c1640 1 -- 172.21.15.117:0/2742294452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55617a53e1b0 msgr2=0x55617a91e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:52.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.278+0000 7f88a89c1640 1 --2- 172.21.15.117:0/2742294452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55617a53e1b0 0x55617a91e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:52.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.278+0000 7f88a89c1640 1 -- 172.21.15.117:0/2742294452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55617a546390 msgr2=0x55617a91e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:52.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.278+0000 7f88a89c1640 1 --2- 172.21.15.117:0/2742294452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55617a546390 0x55617a91e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:52.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.278+0000 7f88a89c1640 1 -- 172.21.15.117:0/2742294452 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55617a83cc90 con 0x55617a567070 2024-09-21T06:04:52.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.278+0000 7f88a37fe640 1 --2- 172.21.15.117:0/2742294452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55617a546390 0x55617a91e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:52.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.278+0000 7f88a89c1640 1 --2- 172.21.15.117:0/2742294452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55617a567070 0x55617a91eb60 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f889c008490 tx=0x7f889c010a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e80ea4b89296162 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:52.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.278+0000 7f88837fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:04:52.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.279+0000 7f88837fe640 1 -- 172.21.15.117:0/2742294452 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c032060 con 0x55617a567070 2024-09-21T06:04:52.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.279+0000 7f88837fe640 1 -- 172.21.15.117:0/2742294452 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f889c01a8b0 con 0x55617a567070 2024-09-21T06:04:52.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.279+0000 7f88b0998740 1 -- 172.21.15.117:0/2742294452 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f889c01ade0 con 0x55617a567070 2024-09-21T06:04:52.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.279+0000 7f88837fe640 1 -- 172.21.15.117:0/2742294452 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f889c014050 con 0x55617a567070 2024-09-21T06:04:52.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.279+0000 7f88b0998740 10 client.16851 fetch_fsmap learned FSMap version 29 2024-09-21T06:04:52.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.279+0000 7f88b0998740 1 -- 172.21.15.117:0/2742294452 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55617a92ce70 con 0x55617a567070 2024-09-21T06:04:52.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.279+0000 7f88837fe640 1 -- 172.21.15.117:0/2742294452 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f889c01ade0 con 0x55617a567070 2024-09-21T06:04:52.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.279+0000 7f88837fe640 1 -- 172.21.15.117:0/2742294452 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f889c02c030 con 0x55617a567070 2024-09-21T06:04:52.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.279+0000 7f88b0998740 10 client.16851 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:04:52.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.279+0000 7f88b0998740 10 client.16851 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:04:52.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.279+0000 7f88b0998740 20 client.16851 populate_metadata read hostname 'smithi117' 2024-09-21T06:04:52.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.279+0000 7f88b0998740 1 --2- 172.21.15.117:0/2742294452 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55617a5259f0 0x55617a9302e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:52.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.279+0000 7f88b0998740 4 client.16851 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:04:52.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.279+0000 7f88b0998740 1 -- 172.21.15.117:0/2742294452 --> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55617a886df0 con 0x55617a5259f0 2024-09-21T06:04:52.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.280+0000 7f88a3fff640 1 --2- 172.21.15.117:0/2742294452 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55617a5259f0 0x55617a9302e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:52.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.280+0000 7f88837fe640 10 client.16851 ms_handle_connect on v2:172.21.15.192:6838/1727623231 2024-09-21T06:04:52.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.280+0000 7f88a3fff640 1 --2- 172.21.15.117:0/2742294452 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55617a5259f0 0x55617a9302e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2ebf32ee5a5577b1 server_cookie=b3d57916b8ae8b4e in_seq=0 out_seq=0 2024-09-21T06:04:52.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.281+0000 7f88837fe640 1 -- 172.21.15.117:0/2742294452 <== mds.1 v2:172.21.15.192:6838/1727623231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55617a886df0 con 0x55617a5259f0 2024-09-21T06:04:52.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.281+0000 7f88837fe640 10 client.16851 handle_command_reply: tid=0 2024-09-21T06:04:52.296 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.281+0000 7f88b0998740 1 client.16851 shutdown 2024-09-21T06:04:52.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.281+0000 7f88b0998740 1 -- 172.21.15.117:0/2742294452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55617a567070 msgr2=0x55617a91eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:52.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.281+0000 7f88b0998740 1 --2- 172.21.15.117:0/2742294452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55617a567070 0x55617a91eb60 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f889c008490 tx=0x7f889c010a20 comp rx=0 tx=0).stop 2024-09-21T06:04:52.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.281+0000 7f88b0998740 1 -- 172.21.15.117:0/2742294452 shutdown_connections 2024-09-21T06:04:52.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.281+0000 7f88b0998740 1 --2- 172.21.15.117:0/2742294452 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55617a5259f0 0x55617a9302e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:52.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.281+0000 7f88b0998740 1 --2- 172.21.15.117:0/2742294452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55617a567070 0x55617a91eb60 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:52.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.281+0000 7f88b0998740 1 --2- 172.21.15.117:0/2742294452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55617a546390 0x55617a91e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:52.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.281+0000 7f88837fe640 0 client.16851 ms_handle_reset on v2:172.21.15.192:6838/1727623231 2024-09-21T06:04:52.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.281+0000 7f88b0998740 1 --2- 172.21.15.117:0/2742294452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55617a53e1b0 0x55617a91e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:52.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.281+0000 7f88b0998740 1 -- 172.21.15.117:0/2742294452 >> 172.21.15.117:0/2742294452 conn(0x55617a55b460 msgr2=0x55617a910090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:52.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.281+0000 7f88b0998740 1 -- 172.21.15.117:0/2742294452 shutdown_connections 2024-09-21T06:04:52.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.281+0000 7f88b0998740 1 -- 172.21.15.117:0/2742294452 wait complete. 2024-09-21T06:04:52.297 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.281+0000 7f88b0998740 20 client.16851 trim_cache size 0 max 16384 2024-09-21T06:04:52.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.295+0000 7f88b0998740 1 Processor -- start 2024-09-21T06:04:52.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.295+0000 7f88b0998740 1 -- start start 2024-09-21T06:04:52.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.295+0000 7f88b0998740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55617a5259f0 0x55617a90c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:52.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.295+0000 7f88b0998740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55617a53e1b0 0x55617a91b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:52.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.295+0000 7f88b0998740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55617a546390 0x55617a91b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:52.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.295+0000 7f88b0998740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f88a41ad350 con 0x55617a53e1b0 2024-09-21T06:04:52.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.295+0000 7f88b0998740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88a41aee90 con 0x55617a5259f0 2024-09-21T06:04:52.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.295+0000 7f88b0998740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f88a41ada20 con 0x55617a546390 2024-09-21T06:04:52.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.295+0000 7f88a37fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55617a53e1b0 0x55617a91b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:52.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.295+0000 7f88a37fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55617a53e1b0 0x55617a91b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43664/0 (socket says 172.21.15.117:43664) 2024-09-21T06:04:52.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.295+0000 7f88a37fe640 1 -- 172.21.15.117:0/3056900341 learned_addr learned my addr 172.21.15.117:0/3056900341 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:52.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.295+0000 7f88a89c1640 1 --2- 172.21.15.117:0/3056900341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55617a546390 0x55617a91b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:52.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.295+0000 7f88a3fff640 1 --2- 172.21.15.117:0/3056900341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55617a5259f0 0x55617a90c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:52.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.295+0000 7f88a37fe640 1 -- 172.21.15.117:0/3056900341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55617a546390 msgr2=0x55617a91b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:52.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.295+0000 7f88a37fe640 1 --2- 172.21.15.117:0/3056900341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55617a546390 0x55617a91b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:52.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.295+0000 7f88a37fe640 1 -- 172.21.15.117:0/3056900341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55617a5259f0 msgr2=0x55617a90c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:52.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.295+0000 7f88a37fe640 1 --2- 172.21.15.117:0/3056900341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55617a5259f0 0x55617a90c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:52.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.295+0000 7f88a37fe640 1 -- 172.21.15.117:0/3056900341 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55617a91eea0 con 0x55617a53e1b0 2024-09-21T06:04:52.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.296+0000 7f88a37fe640 1 --2- 172.21.15.117:0/3056900341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55617a53e1b0 0x55617a91b2d0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f8898009610 tx=0x7f88980acbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd9bde2031d51a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:52.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.296+0000 7f8889ffb640 1 -- 172.21.15.117:0/3056900341 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8898009de0 con 0x55617a53e1b0 2024-09-21T06:04:52.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.296+0000 7f8889ffb640 1 -- 172.21.15.117:0/3056900341 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88980ad980 con 0x55617a53e1b0 2024-09-21T06:04:52.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.296+0000 7f8889ffb640 1 -- 172.21.15.117:0/3056900341 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88980a9680 con 0x55617a53e1b0 2024-09-21T06:04:52.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.297+0000 7f88b0998740 1 -- 172.21.15.117:0/3056900341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55617a53e1b0 msgr2=0x55617a91b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:52.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.297+0000 7f88b0998740 1 --2- 172.21.15.117:0/3056900341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55617a53e1b0 0x55617a91b2d0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f8898009610 tx=0x7f88980acbd0 comp rx=0 tx=0).stop 2024-09-21T06:04:52.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.297+0000 7f88b0998740 1 -- 172.21.15.117:0/3056900341 shutdown_connections 2024-09-21T06:04:52.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.297+0000 7f88b0998740 1 --2- 172.21.15.117:0/3056900341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55617a53e1b0 0x55617a91b2d0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:52.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.297+0000 7f88b0998740 1 --2- 172.21.15.117:0/3056900341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55617a546390 0x55617a91b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:52.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.297+0000 7f88b0998740 1 --2- 172.21.15.117:0/3056900341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55617a5259f0 0x55617a90c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:52.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.297+0000 7f88b0998740 1 -- 172.21.15.117:0/3056900341 >> 172.21.15.117:0/3056900341 conn(0x55617a55b460 msgr2=0x55617a90beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:52.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.297+0000 7f88b0998740 1 -- 172.21.15.117:0/3056900341 shutdown_connections 2024-09-21T06:04:52.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.297+0000 7f88b0998740 1 -- 172.21.15.117:0/3056900341 wait complete. 2024-09-21T06:04:52.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.297+0000 7f88b0998740 1 -- start start 2024-09-21T06:04:52.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.297+0000 7f88b0998740 1 Processor -- start 2024-09-21T06:04:52.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.298+0000 7f88b0998740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55617a5259f0 0x55617a9224e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:52.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.298+0000 7f88b0998740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55617a53e1b0 0x55617a922a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:52.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.298+0000 7f88b0998740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55617a546390 0x55617a922f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:52.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.298+0000 7f88b0998740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f88a41ae7c0 con 0x55617a546390 2024-09-21T06:04:52.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.298+0000 7f88b0998740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88a41ae0f0 con 0x55617a53e1b0 2024-09-21T06:04:52.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.298+0000 7f88b0998740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f88a41ab370 con 0x55617a5259f0 2024-09-21T06:04:52.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.298+0000 7f88b0998740 10 client.0 _ll_register_callbacks cb 0x7f889c01e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:52.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.298+0000 7f88a89c1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55617a546390 0x55617a922f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:52.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.298+0000 7f88a89c1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55617a546390 0x55617a922f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43674/0 (socket says 172.21.15.117:43674) 2024-09-21T06:04:52.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.298+0000 7f88a89c1640 1 -- 172.21.15.117:0/1220203007 learned_addr learned my addr 172.21.15.117:0/1220203007 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:52.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.298+0000 7f88a37fe640 1 --2- 172.21.15.117:0/1220203007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55617a53e1b0 0x55617a922a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:52.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.298+0000 7f88a3fff640 1 --2- 172.21.15.117:0/1220203007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55617a5259f0 0x55617a9224e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:52.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.298+0000 7f88a89c1640 1 -- 172.21.15.117:0/1220203007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55617a5259f0 msgr2=0x55617a9224e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:52.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.298+0000 7f88a89c1640 1 --2- 172.21.15.117:0/1220203007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55617a5259f0 0x55617a9224e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:52.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.298+0000 7f88a89c1640 1 -- 172.21.15.117:0/1220203007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55617a53e1b0 msgr2=0x55617a922a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:52.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.298+0000 7f88a89c1640 1 --2- 172.21.15.117:0/1220203007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55617a53e1b0 0x55617a922a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:52.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.298+0000 7f88a89c1640 1 -- 172.21.15.117:0/1220203007 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55617a999c20 con 0x55617a546390 2024-09-21T06:04:52.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.298+0000 7f88a3fff640 1 --2- 172.21.15.117:0/1220203007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55617a5259f0 0x55617a9224e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:04:52.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.298+0000 7f88a89c1640 1 --2- 172.21.15.117:0/1220203007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55617a546390 0x55617a922f60 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f889c01e040 tx=0x7f889c008c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f64dd25c8527f305 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:52.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.298+0000 7f88837fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:04:52.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.298+0000 7f88837fe640 1 -- 172.21.15.117:0/1220203007 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889c032040 con 0x55617a546390 2024-09-21T06:04:52.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.299+0000 7f88b0998740 1 -- 172.21.15.117:0/1220203007 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88980ad980 con 0x55617a546390 2024-09-21T06:04:52.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.299+0000 7f88837fe640 1 -- 172.21.15.117:0/1220203007 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f889c005940 con 0x55617a546390 2024-09-21T06:04:52.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.299+0000 7f88837fe640 1 -- 172.21.15.117:0/1220203007 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f889c044020 con 0x55617a546390 2024-09-21T06:04:52.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.299+0000 7f88b0998740 10 client.16857 fetch_fsmap learned FSMap version 29 2024-09-21T06:04:52.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.299+0000 7f88b0998740 1 -- 172.21.15.117:0/1220203007 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55617a99ae80 con 0x55617a546390 2024-09-21T06:04:52.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.299+0000 7f88837fe640 1 -- 172.21.15.117:0/1220203007 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88980ad980 con 0x55617a546390 2024-09-21T06:04:52.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.299+0000 7f88837fe640 1 -- 172.21.15.117:0/1220203007 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f889c02c040 con 0x55617a546390 2024-09-21T06:04:52.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.299+0000 7f88b0998740 10 client.16857 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:04:52.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.299+0000 7f88b0998740 10 client.16857 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:04:52.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.299+0000 7f88b0998740 20 client.16857 populate_metadata read hostname 'smithi117' 2024-09-21T06:04:52.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.299+0000 7f88b0998740 1 --2- 172.21.15.117:0/1220203007 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55617a567070 0x55617a99d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:52.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.299+0000 7f88b0998740 4 client.16857 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:04:52.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.299+0000 7f88b0998740 1 -- 172.21.15.117:0/1220203007 --> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55617a8828a0 con 0x55617a567070 2024-09-21T06:04:52.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.299+0000 7f88a3fff640 1 --2- 172.21.15.117:0/1220203007 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55617a567070 0x55617a99d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:52.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.300+0000 7f88837fe640 10 client.16857 ms_handle_connect on v2:172.21.15.192:6838/1727623231 2024-09-21T06:04:52.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.300+0000 7f88a3fff640 1 --2- 172.21.15.117:0/1220203007 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55617a567070 0x55617a99d580 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b2d211ad2fc7679f server_cookie=d780f4bc4886a188 in_seq=0 out_seq=0 2024-09-21T06:04:52.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.300+0000 7f88837fe640 1 -- 172.21.15.117:0/1220203007 <== mds.1 v2:172.21.15.192:6838/1727623231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55617a8828a0 con 0x55617a567070 2024-09-21T06:04:52.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.300+0000 7f88837fe640 10 client.16857 handle_command_reply: tid=0 2024-09-21T06:04:52.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.300+0000 7f88b0998740 1 client.16857 shutdown 2024-09-21T06:04:52.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.300+0000 7f88b0998740 1 -- 172.21.15.117:0/1220203007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55617a546390 msgr2=0x55617a922f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:52.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.300+0000 7f88b0998740 1 --2- 172.21.15.117:0/1220203007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55617a546390 0x55617a922f60 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f889c01e040 tx=0x7f889c008c10 comp rx=0 tx=0).stop 2024-09-21T06:04:52.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.300+0000 7f88b0998740 1 -- 172.21.15.117:0/1220203007 shutdown_connections 2024-09-21T06:04:52.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.300+0000 7f88b0998740 1 --2- 172.21.15.117:0/1220203007 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55617a567070 0x55617a99d580 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:52.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.300+0000 7f88b0998740 1 --2- 172.21.15.117:0/1220203007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55617a546390 0x55617a922f60 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:52.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.300+0000 7f88b0998740 1 --2- 172.21.15.117:0/1220203007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55617a53e1b0 0x55617a922a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:52.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.300+0000 7f88b0998740 1 --2- 172.21.15.117:0/1220203007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55617a5259f0 0x55617a9224e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:52.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.300+0000 7f88b0998740 1 -- 172.21.15.117:0/1220203007 >> 172.21.15.117:0/1220203007 conn(0x55617a55b460 msgr2=0x55617a91f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:52.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.300+0000 7f88837fe640 0 client.16857 ms_handle_reset on v2:172.21.15.192:6838/1727623231 2024-09-21T06:04:52.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.301+0000 7f88b0998740 1 -- 172.21.15.117:0/1220203007 shutdown_connections 2024-09-21T06:04:52.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.301+0000 7f88b0998740 1 -- 172.21.15.117:0/1220203007 wait complete. 2024-09-21T06:04:52.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.301+0000 7f88b0998740 20 client.16857 trim_cache size 0 max 16384 2024-09-21T06:04:52.310 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:04:52.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.301+0000 7f88aa424640 1 -- 172.21.15.117:0/3975683963 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f888c0b6680 msgr2=0x7f888c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:52.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.301+0000 7f88aa424640 1 --2- 172.21.15.117:0/3975683963 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f888c0b6680 0x7f888c0b8b90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f889c01d000 tx=0x7f889c020000 comp rx=0 tx=0).stop 2024-09-21T06:04:52.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.301+0000 7f88aa424640 1 -- 172.21.15.117:0/3975683963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88a41083a0 msgr2=0x7f88a419b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:52.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.301+0000 7f88aa424640 1 --2- 172.21.15.117:0/3975683963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88a41083a0 0x7f88a419b730 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f8898005fd0 tx=0x7f889800dde0 comp rx=0 tx=0).stop 2024-09-21T06:04:52.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.301+0000 7f88aa424640 1 -- 172.21.15.117:0/3975683963 shutdown_connections 2024-09-21T06:04:52.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.301+0000 7f88aa424640 1 --2- 172.21.15.117:0/3975683963 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f888c0b6680 0x7f888c0b8b90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:52.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.301+0000 7f88aa424640 1 --2- 172.21.15.117:0/3975683963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88a41083a0 0x7f88a419b730 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:52.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.301+0000 7f88aa424640 1 --2- 172.21.15.117:0/3975683963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88a4108cf0 0x7f88a419bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:52.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.301+0000 7f88aa424640 1 --2- 172.21.15.117:0/3975683963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88a4102a60 0x7f88a419b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:52.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.301+0000 7f88aa424640 1 -- 172.21.15.117:0/3975683963 >> 172.21.15.117:0/3975683963 conn(0x7f88a4079730 msgr2=0x7f88a4101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:52.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.301+0000 7f88aa424640 1 -- 172.21.15.117:0/3975683963 shutdown_connections 2024-09-21T06:04:52.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:52.301+0000 7f88aa424640 1 -- 172.21.15.117:0/3975683963 wait complete. 2024-09-21T06:04:52.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:51 smithi138 ceph-mon[34391]: pgmap v556: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-09-21T06:04:53.066 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:04:53.313 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:04:53.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.813+0000 7f8cd2c2c640 1 -- 172.21.15.117:0/1659034915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ccc102ae0 msgr2=0x7f8ccc102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:53.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.813+0000 7f8cd2c2c640 1 --2- 172.21.15.117:0/1659034915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ccc102ae0 0x7f8ccc102ef0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0009900 tx=0x7f8cc002a410 comp rx=0 tx=0).stop 2024-09-21T06:04:53.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.813+0000 7f8cd2c2c640 1 -- 172.21.15.117:0/1659034915 shutdown_connections 2024-09-21T06:04:53.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.813+0000 7f8cd2c2c640 1 --2- 172.21.15.117:0/1659034915 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ccc108d70 0x7f8ccc10f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:53.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.813+0000 7f8cd2c2c640 1 --2- 172.21.15.117:0/1659034915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ccc108420 0x7f8ccc108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:53.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.813+0000 7f8cd2c2c640 1 --2- 172.21.15.117:0/1659034915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ccc102ae0 0x7f8ccc102ef0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:53.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.813+0000 7f8cd2c2c640 1 -- 172.21.15.117:0/1659034915 >> 172.21.15.117:0/1659034915 conn(0x7f8ccc079770 msgr2=0x7f8ccc079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:53.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.814+0000 7f8cd2c2c640 1 -- 172.21.15.117:0/1659034915 shutdown_connections 2024-09-21T06:04:53.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.814+0000 7f8cd2c2c640 1 -- 172.21.15.117:0/1659034915 wait complete. 2024-09-21T06:04:53.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.814+0000 7f8cd2c2c640 1 Processor -- start 2024-09-21T06:04:53.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.814+0000 7f8cd2c2c640 1 -- start start 2024-09-21T06:04:53.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.815+0000 7f8cd2c2c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ccc102ae0 0x7f8ccc199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:53.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.815+0000 7f8cd2c2c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ccc108420 0x7f8ccc199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:53.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.815+0000 7f8cd2c2c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ccc108d70 0x7f8ccc19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:53.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.815+0000 7f8cd2c2c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc112a90 con 0x7f8ccc108d70 2024-09-21T06:04:53.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.815+0000 7f8cd2c2c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc112910 con 0x7f8ccc102ae0 2024-09-21T06:04:53.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.815+0000 7f8cd2c2c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc112c10 con 0x7f8ccc108420 2024-09-21T06:04:53.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.815+0000 7f8cd11c9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ccc108d70 0x7f8ccc19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:53.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.815+0000 7f8cd11c9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ccc108d70 0x7f8ccc19ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43694/0 (socket says 172.21.15.117:43694) 2024-09-21T06:04:53.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.815+0000 7f8cd11c9640 1 -- 172.21.15.117:0/1938984588 learned_addr learned my addr 172.21.15.117:0/1938984588 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:53.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.815+0000 7f8cd09c8640 1 --2- 172.21.15.117:0/1938984588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ccc102ae0 0x7f8ccc199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:53.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.815+0000 7f8ccbfff640 1 --2- 172.21.15.117:0/1938984588 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ccc108420 0x7f8ccc199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:53.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.815+0000 7f8cd11c9640 1 -- 172.21.15.117:0/1938984588 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ccc108420 msgr2=0x7f8ccc199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:53.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.815+0000 7f8cd11c9640 1 --2- 172.21.15.117:0/1938984588 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ccc108420 0x7f8ccc199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:53.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.815+0000 7f8cd11c9640 1 -- 172.21.15.117:0/1938984588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ccc102ae0 msgr2=0x7f8ccc199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:53.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.815+0000 7f8cd11c9640 1 --2- 172.21.15.117:0/1938984588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ccc102ae0 0x7f8ccc199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:53.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.815+0000 7f8cd11c9640 1 -- 172.21.15.117:0/1938984588 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ccc19f3f0 con 0x7f8ccc108d70 2024-09-21T06:04:53.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.815+0000 7f8cd11c9640 1 --2- 172.21.15.117:0/1938984588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ccc108d70 0x7f8ccc19ecc0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400b970 tx=0x7f8cc400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:53.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.816+0000 7f8cc9ffb640 1 -- 172.21.15.117:0/1938984588 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4019070 con 0x7f8ccc108d70 2024-09-21T06:04:53.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.816+0000 7f8cc9ffb640 1 -- 172.21.15.117:0/1938984588 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cc4014440 con 0x7f8ccc108d70 2024-09-21T06:04:53.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.816+0000 7f8cd2c2c640 1 -- 172.21.15.117:0/1938984588 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ccc19f6e0 con 0x7f8ccc108d70 2024-09-21T06:04:53.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.816+0000 7f8cc9ffb640 1 -- 172.21.15.117:0/1938984588 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc40075d0 con 0x7f8ccc108d70 2024-09-21T06:04:53.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.816+0000 7f8cd2c2c640 1 -- 172.21.15.117:0/1938984588 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ccc1a5e40 con 0x7f8ccc108d70 2024-09-21T06:04:53.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.818+0000 7f8cd91a0740 1 Processor -- start 2024-09-21T06:04:53.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.818+0000 7f8cc9ffb640 1 -- 172.21.15.117:0/1938984588 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f8cc401c070 con 0x7f8ccc108d70 2024-09-21T06:04:53.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.818+0000 7f8cd91a0740 1 -- start start 2024-09-21T06:04:53.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.819+0000 7f8cd91a0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300d902070 0x56300dca96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:53.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.819+0000 7f8cd91a0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56300d8d91b0 0x56300dca9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:53.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.819+0000 7f8cd91a0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56300d8e1390 0x56300dcaa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:53.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.819+0000 7f8cd91a0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56300dc1fea0 con 0x56300d902070 2024-09-21T06:04:53.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.819+0000 7f8cd91a0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56300dc0f2d0 con 0x56300d8d91b0 2024-09-21T06:04:53.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.819+0000 7f8cd91a0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56300dc130e0 con 0x56300d8e1390 2024-09-21T06:04:53.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.819+0000 7f8ccbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300d902070 0x56300dca96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:53.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.819+0000 7f8ccbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300d902070 0x56300dca96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43698/0 (socket says 172.21.15.117:43698) 2024-09-21T06:04:53.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.819+0000 7f8ccbfff640 1 -- 172.21.15.117:0/2569027750 learned_addr learned my addr 172.21.15.117:0/2569027750 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:53.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.819+0000 7f8cd09c8640 1 --2- 172.21.15.117:0/2569027750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56300d8d91b0 0x56300dca9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:53.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.819+0000 7f8cc9ffb640 1 --2- 172.21.15.117:0/1938984588 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8cbc0b6700 0x7f8cbc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:53.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.819+0000 7f8ccbfff640 1 --2- 172.21.15.117:0/2569027750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56300d8e1390 0x56300dcaa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:53.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.819+0000 7f8ccbfff640 1 -- 172.21.15.117:0/2569027750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56300d8e1390 msgr2=0x56300dcaa120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:53.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.819+0000 7f8ccbfff640 1 --2- 172.21.15.117:0/2569027750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56300d8e1390 0x56300dcaa120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:53.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.819+0000 7f8ccbfff640 1 -- 172.21.15.117:0/2569027750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56300d8d91b0 msgr2=0x56300dca9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:53.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.819+0000 7f8ccbfff640 1 --2- 172.21.15.117:0/2569027750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56300d8d91b0 0x56300dca9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:53.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.819+0000 7f8cd11c9640 1 --2- 172.21.15.117:0/1938984588 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8cbc0b6700 0x7f8cbc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:53.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.819+0000 7f8ccbfff640 1 -- 172.21.15.117:0/2569027750 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56300d96d650 con 0x56300d902070 2024-09-21T06:04:53.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.819+0000 7f8cd09c8640 1 --2- 172.21.15.117:0/2569027750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56300d8d91b0 0x56300dca9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:04:53.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.819+0000 7f8cc9ffb640 1 -- 172.21.15.117:0/1938984588 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f8cc40e6b10 con 0x7f8ccc108d70 2024-09-21T06:04:53.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.820+0000 7f8ccbfff640 1 --2- 172.21.15.117:0/2569027750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300d902070 0x56300dca96a0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800bfa0 tx=0x7f8cb800b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eed90b201b74ad4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:53.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.820+0000 7f8cb6ffd640 1 -- 172.21.15.117:0/2569027750 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8019030 con 0x56300d902070 2024-09-21T06:04:53.828 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.820+0000 7f8cd11c9640 1 --2- 172.21.15.117:0/1938984588 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8cbc0b6700 0x7f8cbc0b8c10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40e4360 tx=0x7f8cc400e040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:53.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.820+0000 7f8cb6ffd640 1 -- 172.21.15.117:0/2569027750 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cb80118f0 con 0x56300d902070 2024-09-21T06:04:53.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.820+0000 7f8cb6ffd640 1 -- 172.21.15.117:0/2569027750 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb800f6d0 con 0x56300d902070 2024-09-21T06:04:53.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.821+0000 7f8cd91a0740 1 -- 172.21.15.117:0/2569027750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300d902070 msgr2=0x56300dca96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:53.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.821+0000 7f8cd91a0740 1 --2- 172.21.15.117:0/2569027750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300d902070 0x56300dca96a0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800bfa0 tx=0x7f8cb800b650 comp rx=0 tx=0).stop 2024-09-21T06:04:53.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.821+0000 7f8cd91a0740 1 -- 172.21.15.117:0/2569027750 shutdown_connections 2024-09-21T06:04:53.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.821+0000 7f8cd91a0740 1 --2- 172.21.15.117:0/2569027750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56300d8e1390 0x56300dcaa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:53.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.821+0000 7f8cd91a0740 1 --2- 172.21.15.117:0/2569027750 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56300d8d91b0 0x56300dca9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:53.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.821+0000 7f8cd91a0740 1 --2- 172.21.15.117:0/2569027750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300d902070 0x56300dca96a0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:53.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.821+0000 7f8cd91a0740 1 -- 172.21.15.117:0/2569027750 >> 172.21.15.117:0/2569027750 conn(0x56300d8f6460 msgr2=0x56300dca7590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:53.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.821+0000 7f8cd91a0740 1 -- 172.21.15.117:0/2569027750 shutdown_connections 2024-09-21T06:04:53.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.821+0000 7f8cd91a0740 1 -- 172.21.15.117:0/2569027750 wait complete. 2024-09-21T06:04:53.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.821+0000 7f8cd91a0740 1 -- start start 2024-09-21T06:04:53.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.821+0000 7f8cd91a0740 1 Processor -- start 2024-09-21T06:04:53.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.822+0000 7f8cd91a0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300d8d91b0 0x56300dcb4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:53.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.822+0000 7f8cd91a0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56300d8e1390 0x56300dcb5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:53.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.822+0000 7f8cd91a0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56300d902070 0x56300dcc32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:53.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.822+0000 7f8cd91a0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56300da70700 con 0x56300d8d91b0 2024-09-21T06:04:53.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.822+0000 7f8cd91a0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56300dbaa160 con 0x56300d8e1390 2024-09-21T06:04:53.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.822+0000 7f8cd91a0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56300dcba030 con 0x56300d902070 2024-09-21T06:04:53.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.822+0000 7f8cd91a0740 10 client.0 _ll_register_callbacks cb 0x56300dbc39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:53.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.823+0000 7f8ccbfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56300d902070 0x56300dcc32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:53.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.823+0000 7f8cd09c8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56300d8e1390 0x56300dcb5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:53.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.823+0000 7f8ccbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300d8d91b0 0x56300dcb4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:53.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.823+0000 7f8ccbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300d8d91b0 0x56300dcb4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43710/0 (socket says 172.21.15.117:43710) 2024-09-21T06:04:53.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.823+0000 7f8ccbfff640 1 -- 172.21.15.117:0/1502094826 learned_addr learned my addr 172.21.15.117:0/1502094826 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:53.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.823+0000 7f8ccbfff640 1 -- 172.21.15.117:0/1502094826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56300d902070 msgr2=0x56300dcc32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:53.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.823+0000 7f8ccbfff640 1 --2- 172.21.15.117:0/1502094826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56300d902070 0x56300dcc32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:53.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.823+0000 7f8ccbfff640 1 -- 172.21.15.117:0/1502094826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56300d8e1390 msgr2=0x56300dcb5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:53.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.823+0000 7f8ccbfff640 1 --2- 172.21.15.117:0/1502094826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56300d8e1390 0x56300dcb5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:53.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.823+0000 7f8ccbfff640 1 -- 172.21.15.117:0/1502094826 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56300dbd7c90 con 0x56300d8d91b0 2024-09-21T06:04:53.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.823+0000 7f8ccbfff640 1 --2- 172.21.15.117:0/1502094826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300d8d91b0 0x56300dcb4dd0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800b7a0 tx=0x7f8cb8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ab4d1379f594e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:53.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.823+0000 7f8cabfff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:04:53.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.823+0000 7f8cabfff640 1 -- 172.21.15.117:0/1502094826 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8019040 con 0x56300d8d91b0 2024-09-21T06:04:53.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.824+0000 7f8cabfff640 1 -- 172.21.15.117:0/1502094826 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f8cb8011cb0 con 0x56300d8d91b0 2024-09-21T06:04:53.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.824+0000 7f8cd91a0740 1 -- 172.21.15.117:0/1502094826 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cb80118f0 con 0x56300d8d91b0 2024-09-21T06:04:53.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.824+0000 7f8cabfff640 1 -- 172.21.15.117:0/1502094826 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8cb8032050 con 0x56300d8d91b0 2024-09-21T06:04:53.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.824+0000 7f8cd91a0740 10 client.16869 fetch_fsmap learned FSMap version 29 2024-09-21T06:04:53.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.824+0000 7f8cd91a0740 1 -- 172.21.15.117:0/1502094826 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56300dcb8fb0 con 0x56300d8d91b0 2024-09-21T06:04:53.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.824+0000 7f8cabfff640 1 -- 172.21.15.117:0/1502094826 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cb80118f0 con 0x56300d8d91b0 2024-09-21T06:04:53.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.824+0000 7f8cabfff640 1 -- 172.21.15.117:0/1502094826 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8cb8005440 con 0x56300d8d91b0 2024-09-21T06:04:53.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.824+0000 7f8cd91a0740 10 client.16869 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:04:53.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.824+0000 7f8cd91a0740 10 client.16869 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:04:53.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.824+0000 7f8cd91a0740 20 client.16869 populate_metadata read hostname 'smithi117' 2024-09-21T06:04:53.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.824+0000 7f8cd91a0740 1 --2- 172.21.15.117:0/1502094826 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56300d8c09f0 0x56300dcb92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:53.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.824+0000 7f8cd91a0740 4 client.16869 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:04:53.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.824+0000 7f8cd91a0740 1 -- 172.21.15.117:0/1502094826 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56300dc21df0 con 0x56300d8c09f0 2024-09-21T06:04:53.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.825+0000 7f8cd09c8640 1 --2- 172.21.15.117:0/1502094826 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56300d8c09f0 0x56300dcb92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:53.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.825+0000 7f8cabfff640 10 client.16869 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:53.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.825+0000 7f8cd09c8640 1 --2- 172.21.15.117:0/1502094826 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56300d8c09f0 0x56300dcb92f0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb7a78873d17afb3 server_cookie=71f8127e632dd137 in_seq=0 out_seq=0 2024-09-21T06:04:53.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.826+0000 7f8cabfff640 1 -- 172.21.15.117:0/1502094826 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56300dc21df0 con 0x56300d8c09f0 2024-09-21T06:04:53.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.826+0000 7f8cabfff640 10 client.16869 handle_command_reply: tid=0 2024-09-21T06:04:53.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.826+0000 7f8cd91a0740 1 client.16869 shutdown 2024-09-21T06:04:53.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.826+0000 7f8cd91a0740 1 -- 172.21.15.117:0/1502094826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300d8d91b0 msgr2=0x56300dcb4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:53.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.826+0000 7f8cd91a0740 1 --2- 172.21.15.117:0/1502094826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300d8d91b0 0x56300dcb4dd0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800b7a0 tx=0x7f8cb8005fd0 comp rx=0 tx=0).stop 2024-09-21T06:04:53.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.826+0000 7f8cd91a0740 1 -- 172.21.15.117:0/1502094826 shutdown_connections 2024-09-21T06:04:53.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.826+0000 7f8cd91a0740 1 --2- 172.21.15.117:0/1502094826 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56300d8c09f0 0x56300dcb92f0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:53.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.826+0000 7f8cd91a0740 1 --2- 172.21.15.117:0/1502094826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56300d902070 0x56300dcc32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:53.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.826+0000 7f8cd91a0740 1 --2- 172.21.15.117:0/1502094826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56300d8e1390 0x56300dcb5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:53.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.826+0000 7f8cabfff640 0 client.16869 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:53.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.826+0000 7f8cd91a0740 1 --2- 172.21.15.117:0/1502094826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300d8d91b0 0x56300dcb4dd0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:53.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.826+0000 7f8cd91a0740 1 -- 172.21.15.117:0/1502094826 >> 172.21.15.117:0/1502094826 conn(0x56300d8f6460 msgr2=0x56300dcac880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:53.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.827+0000 7f8cd91a0740 1 -- 172.21.15.117:0/1502094826 shutdown_connections 2024-09-21T06:04:53.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.827+0000 7f8cd91a0740 1 -- 172.21.15.117:0/1502094826 wait complete. 2024-09-21T06:04:53.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.827+0000 7f8cd91a0740 20 client.16869 trim_cache size 0 max 16384 2024-09-21T06:04:53.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.840+0000 7f8cd91a0740 1 Processor -- start 2024-09-21T06:04:53.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.840+0000 7f8cd91a0740 1 -- start start 2024-09-21T06:04:53.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.840+0000 7f8cd91a0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56300d8d91b0 0x56300dcb6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:53.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.840+0000 7f8cd91a0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300d8e1390 0x56300dcd0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:53.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.840+0000 7f8cd91a0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56300d902070 0x56300dcd14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:53.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.840+0000 7f8cd91a0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1ad440 con 0x56300d8e1390 2024-09-21T06:04:53.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.840+0000 7f8cd91a0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1aef80 con 0x56300d8d91b0 2024-09-21T06:04:53.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.840+0000 7f8cd91a0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1adb10 con 0x56300d902070 2024-09-21T06:04:53.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.840+0000 7f8cd09c8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300d8e1390 0x56300dcd0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:53.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.840+0000 7f8cd09c8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300d8e1390 0x56300dcd0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43722/0 (socket says 172.21.15.117:43722) 2024-09-21T06:04:53.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.840+0000 7f8cd09c8640 1 -- 172.21.15.117:0/4261096930 learned_addr learned my addr 172.21.15.117:0/4261096930 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:53.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.840+0000 7f8ccbfff640 1 --2- 172.21.15.117:0/4261096930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56300d8d91b0 0x56300dcb6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:53.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.840+0000 7f8cd09c8640 1 -- 172.21.15.117:0/4261096930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56300d902070 msgr2=0x56300dcd14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:53.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.840+0000 7f8cd09c8640 1 --2- 172.21.15.117:0/4261096930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56300d902070 0x56300dcd14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:53.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.840+0000 7f8cd09c8640 1 -- 172.21.15.117:0/4261096930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56300d8d91b0 msgr2=0x56300dcb6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:53.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.840+0000 7f8cd09c8640 1 --2- 172.21.15.117:0/4261096930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56300d8d91b0 0x56300dcb6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:53.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.840+0000 7f8cd09c8640 1 -- 172.21.15.117:0/4261096930 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56300dcb9700 con 0x56300d8e1390 2024-09-21T06:04:53.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.840+0000 7f8cd09c8640 1 --2- 172.21.15.117:0/4261096930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300d8e1390 0x56300dcd0f80 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8cc002a930 tx=0x7f8cc0032ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2730395c014b5417 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:53.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.841+0000 7f8cb67fc640 1 -- 172.21.15.117:0/4261096930 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc004a020 con 0x56300d8e1390 2024-09-21T06:04:53.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.841+0000 7f8cb67fc640 1 -- 172.21.15.117:0/4261096930 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cc0005040 con 0x56300d8e1390 2024-09-21T06:04:53.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.841+0000 7f8cb67fc640 1 -- 172.21.15.117:0/4261096930 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0035040 con 0x56300d8e1390 2024-09-21T06:04:53.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.841+0000 7f8cd91a0740 1 -- 172.21.15.117:0/4261096930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300d8e1390 msgr2=0x56300dcd0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:53.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.841+0000 7f8cd91a0740 1 --2- 172.21.15.117:0/4261096930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300d8e1390 0x56300dcd0f80 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8cc002a930 tx=0x7f8cc0032ba0 comp rx=0 tx=0).stop 2024-09-21T06:04:53.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.842+0000 7f8cd91a0740 1 -- 172.21.15.117:0/4261096930 shutdown_connections 2024-09-21T06:04:53.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.842+0000 7f8cd91a0740 1 --2- 172.21.15.117:0/4261096930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300d8e1390 0x56300dcd0f80 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:53.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.842+0000 7f8cd91a0740 1 --2- 172.21.15.117:0/4261096930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56300d902070 0x56300dcd14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:53.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.842+0000 7f8cd91a0740 1 --2- 172.21.15.117:0/4261096930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56300d8d91b0 0x56300dcb6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:53.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.842+0000 7f8cd91a0740 1 -- 172.21.15.117:0/4261096930 >> 172.21.15.117:0/4261096930 conn(0x56300d8f6460 msgr2=0x56300dcb62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:53.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.842+0000 7f8cd91a0740 1 -- 172.21.15.117:0/4261096930 shutdown_connections 2024-09-21T06:04:53.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.842+0000 7f8cd91a0740 1 -- 172.21.15.117:0/4261096930 wait complete. 2024-09-21T06:04:53.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.842+0000 7f8cd91a0740 1 -- start start 2024-09-21T06:04:53.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.842+0000 7f8cd91a0740 1 Processor -- start 2024-09-21T06:04:53.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.842+0000 7f8cd91a0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300d8d91b0 0x56300dcd1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:53.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.842+0000 7f8cd91a0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56300d8e1390 0x56300dccea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:53.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.843+0000 7f8cd91a0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56300d902070 0x56300dccef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:53.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.843+0000 7f8cd91a0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1ae8b0 con 0x56300d8d91b0 2024-09-21T06:04:53.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.843+0000 7f8cd91a0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1ae1e0 con 0x56300d902070 2024-09-21T06:04:53.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.843+0000 7f8cd91a0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1ab440 con 0x56300d8e1390 2024-09-21T06:04:53.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.843+0000 7f8cd91a0740 10 client.0 _ll_register_callbacks cb 0x7f8cc0009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:53.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.843+0000 7f8ccbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300d8d91b0 0x56300dcd1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:53.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.843+0000 7f8ccbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300d8d91b0 0x56300dcd1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:43736/0 (socket says 172.21.15.117:43736) 2024-09-21T06:04:53.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.843+0000 7f8ccbfff640 1 -- 172.21.15.117:0/3547802704 learned_addr learned my addr 172.21.15.117:0/3547802704 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:53.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.843+0000 7f8ccbfff640 1 --2- 172.21.15.117:0/3547802704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56300d902070 0x56300dccef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:53.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.843+0000 7f8cd09c8640 1 --2- 172.21.15.117:0/3547802704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56300d8e1390 0x56300dccea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:53.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.843+0000 7f8ccbfff640 1 -- 172.21.15.117:0/3547802704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56300d8e1390 msgr2=0x56300dccea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:53.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.843+0000 7f8ccbfff640 1 --2- 172.21.15.117:0/3547802704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56300d8e1390 0x56300dccea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:53.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.843+0000 7f8ccbfff640 1 -- 172.21.15.117:0/3547802704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56300d902070 msgr2=0x56300dccef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:53.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.843+0000 7f8ccbfff640 1 --2- 172.21.15.117:0/3547802704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56300d902070 0x56300dccef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:53.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.843+0000 7f8ccbfff640 1 -- 172.21.15.117:0/3547802704 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56300dccf520 con 0x56300d8d91b0 2024-09-21T06:04:53.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.843+0000 7f8ccbfff640 1 --2- 172.21.15.117:0/3547802704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300d8d91b0 0x56300dcd1550 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8005bf0 tx=0x7f8cb8005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27959d0080974f8c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:53.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.843+0000 7f8cabfff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:04:53.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.843+0000 7f8cabfff640 1 -- 172.21.15.117:0/3547802704 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb803d060 con 0x56300d8d91b0 2024-09-21T06:04:53.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.843+0000 7f8cabfff640 1 -- 172.21.15.117:0/3547802704 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f8cb80116a0 con 0x56300d8d91b0 2024-09-21T06:04:53.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.843+0000 7f8cd91a0740 1 -- 172.21.15.117:0/3547802704 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cc0005040 con 0x56300d8d91b0 2024-09-21T06:04:53.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.844+0000 7f8cabfff640 1 -- 172.21.15.117:0/3547802704 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8cb8019070 con 0x56300d8d91b0 2024-09-21T06:04:53.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.844+0000 7f8cd91a0740 10 client.16875 fetch_fsmap learned FSMap version 29 2024-09-21T06:04:53.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.844+0000 7f8cd91a0740 1 -- 172.21.15.117:0/3547802704 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56300dd35ed0 con 0x56300d8d91b0 2024-09-21T06:04:53.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.844+0000 7f8cabfff640 1 -- 172.21.15.117:0/3547802704 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cc0005040 con 0x56300d8d91b0 2024-09-21T06:04:53.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.844+0000 7f8cabfff640 1 -- 172.21.15.117:0/3547802704 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8cb8003960 con 0x56300d8d91b0 2024-09-21T06:04:53.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.844+0000 7f8cd91a0740 10 client.16875 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:04:53.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.844+0000 7f8cd91a0740 10 client.16875 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:04:53.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.844+0000 7f8cd91a0740 20 client.16875 populate_metadata read hostname 'smithi117' 2024-09-21T06:04:53.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.844+0000 7f8cd91a0740 1 --2- 172.21.15.117:0/3547802704 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56300d895da0 0x56300dd385d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:53.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.844+0000 7f8cd91a0740 4 client.16875 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:04:53.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.844+0000 7f8cd91a0740 1 -- 172.21.15.117:0/3547802704 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56300dc1d8a0 con 0x56300d895da0 2024-09-21T06:04:53.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.844+0000 7f8cd09c8640 1 --2- 172.21.15.117:0/3547802704 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56300d895da0 0x56300dd385d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:53.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.845+0000 7f8cd09c8640 1 --2- 172.21.15.117:0/3547802704 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56300d895da0 0x56300dd385d0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d8282b3509e0f7f server_cookie=780af3c7cdeb9331 in_seq=0 out_seq=0 2024-09-21T06:04:53.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.845+0000 7f8cabfff640 10 client.16875 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:53.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.845+0000 7f8cabfff640 1 -- 172.21.15.117:0/3547802704 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56300dc1d8a0 con 0x56300d895da0 2024-09-21T06:04:53.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.845+0000 7f8cabfff640 10 client.16875 handle_command_reply: tid=0 2024-09-21T06:04:53.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.845+0000 7f8cd91a0740 1 client.16875 shutdown 2024-09-21T06:04:53.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.845+0000 7f8cd91a0740 1 -- 172.21.15.117:0/3547802704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300d8d91b0 msgr2=0x56300dcd1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:53.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.845+0000 7f8cd91a0740 1 --2- 172.21.15.117:0/3547802704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300d8d91b0 0x56300dcd1550 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8005bf0 tx=0x7f8cb8005970 comp rx=0 tx=0).stop 2024-09-21T06:04:53.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.845+0000 7f8cd91a0740 1 -- 172.21.15.117:0/3547802704 shutdown_connections 2024-09-21T06:04:53.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.845+0000 7f8cd91a0740 1 --2- 172.21.15.117:0/3547802704 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56300d895da0 0x56300dd385d0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:53.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.845+0000 7f8cabfff640 0 client.16875 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:53.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.845+0000 7f8cd91a0740 1 --2- 172.21.15.117:0/3547802704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56300d902070 0x56300dccef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:53.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.845+0000 7f8cd91a0740 1 --2- 172.21.15.117:0/3547802704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56300d8e1390 0x56300dccea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:53.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.845+0000 7f8cd91a0740 1 --2- 172.21.15.117:0/3547802704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56300d8d91b0 0x56300dcd1550 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:53.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.845+0000 7f8cd91a0740 1 -- 172.21.15.117:0/3547802704 >> 172.21.15.117:0/3547802704 conn(0x56300d8f6460 msgr2=0x56300dcb6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:53.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.845+0000 7f8cd91a0740 1 -- 172.21.15.117:0/3547802704 shutdown_connections 2024-09-21T06:04:53.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.845+0000 7f8cd91a0740 1 -- 172.21.15.117:0/3547802704 wait complete. 2024-09-21T06:04:53.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.845+0000 7f8cd91a0740 20 client.16875 trim_cache size 0 max 16384 2024-09-21T06:04:53.852 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:04:53.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.846+0000 7f8cd2c2c640 1 -- 172.21.15.117:0/1938984588 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8cbc0b6700 msgr2=0x7f8cbc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:53.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.846+0000 7f8cd2c2c640 1 --2- 172.21.15.117:0/1938984588 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8cbc0b6700 0x7f8cbc0b8c10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40e4360 tx=0x7f8cc400e040 comp rx=0 tx=0).stop 2024-09-21T06:04:53.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.846+0000 7f8cd2c2c640 1 -- 172.21.15.117:0/1938984588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ccc108d70 msgr2=0x7f8ccc19ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:53.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.846+0000 7f8cd2c2c640 1 --2- 172.21.15.117:0/1938984588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ccc108d70 0x7f8ccc19ecc0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400b970 tx=0x7f8cc400be30 comp rx=0 tx=0).stop 2024-09-21T06:04:53.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.846+0000 7f8cd2c2c640 1 -- 172.21.15.117:0/1938984588 shutdown_connections 2024-09-21T06:04:53.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.846+0000 7f8cd2c2c640 1 --2- 172.21.15.117:0/1938984588 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8cbc0b6700 0x7f8cbc0b8c10 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:53.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.846+0000 7f8cd2c2c640 1 --2- 172.21.15.117:0/1938984588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ccc108d70 0x7f8ccc19ecc0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:53.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.846+0000 7f8cd2c2c640 1 --2- 172.21.15.117:0/1938984588 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ccc108420 0x7f8ccc199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:53.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.846+0000 7f8cd2c2c640 1 --2- 172.21.15.117:0/1938984588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ccc102ae0 0x7f8ccc199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:53.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.846+0000 7f8cd2c2c640 1 -- 172.21.15.117:0/1938984588 >> 172.21.15.117:0/1938984588 conn(0x7f8ccc079770 msgr2=0x7f8ccc106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:53.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.846+0000 7f8cd2c2c640 1 -- 172.21.15.117:0/1938984588 shutdown_connections 2024-09-21T06:04:53.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:53.846+0000 7f8cd2c2c640 1 -- 172.21.15.117:0/1938984588 wait complete. 2024-09-21T06:04:54.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:53 smithi117 ceph-mon[30947]: pgmap v557: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-09-21T06:04:54.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:53 smithi192 ceph-mon[34700]: pgmap v557: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-09-21T06:04:54.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:53 smithi138 ceph-mon[34391]: pgmap v557: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-09-21T06:04:55.517 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:04:55.518 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:04:55.779 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:04:56.183 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:55 smithi117 ceph-mon[30947]: pgmap v558: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.6 KiB/s wr, 13 op/s 2024-09-21T06:04:56.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:55 smithi192 ceph-mon[34700]: pgmap v558: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.6 KiB/s wr, 13 op/s 2024-09-21T06:04:56.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.274+0000 7fdfaffee640 1 -- 172.21.15.117:0/2816751769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfa8109df0 msgr2=0x7fdfa8110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:56.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.274+0000 7fdfaffee640 1 --2- 172.21.15.117:0/2816751769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfa8109df0 0x7fdfa8110620 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0009ca0 tx=0x7fdfa002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.274+0000 7fdfaffee640 1 -- 172.21.15.117:0/2816751769 shutdown_connections 2024-09-21T06:04:56.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.274+0000 7fdfaffee640 1 --2- 172.21.15.117:0/2816751769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfa8109df0 0x7fdfa8110620 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.274+0000 7fdfaffee640 1 --2- 172.21.15.117:0/2816751769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfa81094a0 0x7fdfa81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.274+0000 7fdfaffee640 1 --2- 172.21.15.117:0/2816751769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfa8103b60 0x7fdfa8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.274+0000 7fdfaffee640 1 -- 172.21.15.117:0/2816751769 >> 172.21.15.117:0/2816751769 conn(0x7fdfa807b850 msgr2=0x7fdfa807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:56.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.274+0000 7fdfaffee640 1 -- 172.21.15.117:0/2816751769 shutdown_connections 2024-09-21T06:04:56.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.274+0000 7fdfaffee640 1 -- 172.21.15.117:0/2816751769 wait complete. 2024-09-21T06:04:56.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.275+0000 7fdfaffee640 1 Processor -- start 2024-09-21T06:04:56.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.275+0000 7fdfaffee640 1 -- start start 2024-09-21T06:04:56.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.275+0000 7fdfaffee640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfa8103b60 0x7fdfa8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:56.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.275+0000 7fdfaffee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfa81094a0 0x7fdfa8075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:56.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.275+0000 7fdfaffee640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfa8109df0 0x7fdfa8076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:56.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.275+0000 7fdfaffee640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa8113ab0 con 0x7fdfa81094a0 2024-09-21T06:04:56.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.275+0000 7fdfaffee640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa8113930 con 0x7fdfa8109df0 2024-09-21T06:04:56.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.275+0000 7fdfaffee640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa8113c30 con 0x7fdfa8103b60 2024-09-21T06:04:56.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.275+0000 7fdfad589640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfa81094a0 0x7fdfa8075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:56.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.275+0000 7fdfad589640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfa81094a0 0x7fdfa8075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:46584/0 (socket says 172.21.15.117:46584) 2024-09-21T06:04:56.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.275+0000 7fdfad589640 1 -- 172.21.15.117:0/3123433520 learned_addr learned my addr 172.21.15.117:0/3123433520 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:56.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.275+0000 7fdfadd8a640 1 --2- 172.21.15.117:0/3123433520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfa8103b60 0x7fdfa8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:56.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.275+0000 7fdfae58b640 1 --2- 172.21.15.117:0/3123433520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfa8109df0 0x7fdfa8076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:56.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.275+0000 7fdfad589640 1 -- 172.21.15.117:0/3123433520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfa8103b60 msgr2=0x7fdfa8075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:56.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.275+0000 7fdfad589640 1 --2- 172.21.15.117:0/3123433520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfa8103b60 0x7fdfa8075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.275+0000 7fdfad589640 1 -- 172.21.15.117:0/3123433520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfa8109df0 msgr2=0x7fdfa8076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:56.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.275+0000 7fdfad589640 1 --2- 172.21.15.117:0/3123433520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfa8109df0 0x7fdfa8076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.275+0000 7fdfad589640 1 -- 172.21.15.117:0/3123433520 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa81a8700 con 0x7fdfa81094a0 2024-09-21T06:04:56.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.275+0000 7fdfadd8a640 1 --2- 172.21.15.117:0/3123433520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfa8103b60 0x7fdfa8075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:04:56.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.276+0000 7fdfad589640 1 --2- 172.21.15.117:0/3123433520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfa81094a0 0x7fdfa8075c50 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fdf9800d740 tx=0x7fdf9800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:56.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.276+0000 7fdf9effd640 1 -- 172.21.15.117:0/3123433520 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98017070 con 0x7fdfa81094a0 2024-09-21T06:04:56.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.276+0000 7fdfaffee640 1 -- 172.21.15.117:0/3123433520 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfa81a89f0 con 0x7fdfa81094a0 2024-09-21T06:04:56.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.276+0000 7fdfaffee640 1 -- 172.21.15.117:0/3123433520 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfa81a8f30 con 0x7fdfa81094a0 2024-09-21T06:04:56.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.276+0000 7fdf9effd640 1 -- 172.21.15.117:0/3123433520 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf9800de50 con 0x7fdfa81094a0 2024-09-21T06:04:56.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.276+0000 7fdf9effd640 1 -- 172.21.15.117:0/3123433520 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9800f620 con 0x7fdfa81094a0 2024-09-21T06:04:56.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.279+0000 7fdfb6562740 1 Processor -- start 2024-09-21T06:04:56.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.279+0000 7fdfb6562740 1 -- start start 2024-09-21T06:04:56.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.279+0000 7fdf9effd640 1 -- 172.21.15.117:0/3123433520 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fdf9801a070 con 0x7fdfa81094a0 2024-09-21T06:04:56.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.279+0000 7fdfb6562740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0199b2350 0x55c019d59430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:56.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.279+0000 7fdfb6562740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c019989490 0x55c019d59970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:56.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.279+0000 7fdfb6562740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c019991670 0x55c019d59eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:56.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.279+0000 7fdfb6562740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c019ccfe60 con 0x55c019989490 2024-09-21T06:04:56.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.279+0000 7fdfb6562740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c019cbf290 con 0x55c0199b2350 2024-09-21T06:04:56.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.279+0000 7fdfb6562740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c019cc30a0 con 0x55c019991670 2024-09-21T06:04:56.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.280+0000 7fdfadd8a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c019989490 0x55c019d59970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:56.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.280+0000 7fdfadd8a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c019989490 0x55c019d59970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:46598/0 (socket says 172.21.15.117:46598) 2024-09-21T06:04:56.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.280+0000 7fdfadd8a640 1 -- 172.21.15.117:0/4048846162 learned_addr learned my addr 172.21.15.117:0/4048846162 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:56.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.280+0000 7fdfae58b640 1 --2- 172.21.15.117:0/4048846162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0199b2350 0x55c019d59430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:56.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.280+0000 7fdfad589640 1 --2- 172.21.15.117:0/4048846162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c019991670 0x55c019d59eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:56.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.280+0000 7fdfadd8a640 1 -- 172.21.15.117:0/4048846162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c019991670 msgr2=0x55c019d59eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:56.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.280+0000 7fdfadd8a640 1 --2- 172.21.15.117:0/4048846162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c019991670 0x55c019d59eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.280+0000 7fdfadd8a640 1 -- 172.21.15.117:0/4048846162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0199b2350 msgr2=0x55c019d59430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:56.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.280+0000 7fdfadd8a640 1 --2- 172.21.15.117:0/4048846162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0199b2350 0x55c019d59430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.280+0000 7fdfadd8a640 1 -- 172.21.15.117:0/4048846162 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c019a1d930 con 0x55c019989490 2024-09-21T06:04:56.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.280+0000 7fdfae58b640 1 --2- 172.21.15.117:0/4048846162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0199b2350 0x55c019d59430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:04:56.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.280+0000 7fdf9effd640 1 --2- 172.21.15.117:0/3123433520 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fdf940b6680 0x7fdf940b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:56.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.280+0000 7fdfadd8a640 1 --2- 172.21.15.117:0/4048846162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c019989490 0x55c019d59970 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fdfa400bd30 tx=0x7fdfa400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d7d937588e97d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:56.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.280+0000 7fdfae58b640 1 --2- 172.21.15.117:0/3123433520 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fdf940b6680 0x7fdf940b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:56.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.280+0000 7fdf9effd640 1 -- 172.21.15.117:0/3123433520 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7fdf980e6f70 con 0x7fdfa81094a0 2024-09-21T06:04:56.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.280+0000 7fdf93fff640 1 -- 172.21.15.117:0/4048846162 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4015070 con 0x55c019989490 2024-09-21T06:04:56.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.280+0000 7fdf93fff640 1 -- 172.21.15.117:0/4048846162 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfa400be20 con 0x55c019989490 2024-09-21T06:04:56.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.280+0000 7fdf93fff640 1 -- 172.21.15.117:0/4048846162 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa40116a0 con 0x55c019989490 2024-09-21T06:04:56.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.281+0000 7fdfb6562740 1 -- 172.21.15.117:0/4048846162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c019989490 msgr2=0x55c019d59970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:56.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.281+0000 7fdfae58b640 1 --2- 172.21.15.117:0/3123433520 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fdf940b6680 0x7fdf940b8b90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdfa00094e0 tx=0x7fdfa00093f0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:56.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.281+0000 7fdfb6562740 1 --2- 172.21.15.117:0/4048846162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c019989490 0x55c019d59970 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fdfa400bd30 tx=0x7fdfa400f730 comp rx=0 tx=0).stop 2024-09-21T06:04:56.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.281+0000 7fdfb6562740 1 -- 172.21.15.117:0/4048846162 shutdown_connections 2024-09-21T06:04:56.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.281+0000 7fdfb6562740 1 --2- 172.21.15.117:0/4048846162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c019989490 0x55c019d59970 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.290 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.281+0000 7fdfb6562740 1 --2- 172.21.15.117:0/4048846162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c019991670 0x55c019d59eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.281+0000 7fdfb6562740 1 --2- 172.21.15.117:0/4048846162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0199b2350 0x55c019d59430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.281+0000 7fdfb6562740 1 -- 172.21.15.117:0/4048846162 >> 172.21.15.117:0/4048846162 conn(0x55c0199a6740 msgr2=0x55c019d57320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:56.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.281+0000 7fdfb6562740 1 -- 172.21.15.117:0/4048846162 shutdown_connections 2024-09-21T06:04:56.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.281+0000 7fdfb6562740 1 -- 172.21.15.117:0/4048846162 wait complete. 2024-09-21T06:04:56.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.282+0000 7fdfb6562740 1 -- start start 2024-09-21T06:04:56.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.282+0000 7fdfb6562740 1 Processor -- start 2024-09-21T06:04:56.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.282+0000 7fdfb6562740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c019989490 0x55c019d65050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:56.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.282+0000 7fdfb6562740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c019991670 0x55c019d65590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:56.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.282+0000 7fdfb6562740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0199b2350 0x55c019d70f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:56.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.282+0000 7fdfb6562740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c019b20890 con 0x55c0199b2350 2024-09-21T06:04:56.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.282+0000 7fdfb6562740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c019c59d40 con 0x55c019989490 2024-09-21T06:04:56.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.282+0000 7fdfb6562740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c019d67c10 con 0x55c019991670 2024-09-21T06:04:56.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.283+0000 7fdfb6562740 10 client.0 _ll_register_callbacks cb 0x55c019cbdc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:56.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.283+0000 7fdfae58b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0199b2350 0x55c019d70f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:56.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.283+0000 7fdfae58b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0199b2350 0x55c019d70f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:46600/0 (socket says 172.21.15.117:46600) 2024-09-21T06:04:56.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.283+0000 7fdfadd8a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c019989490 0x55c019d65050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:56.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.283+0000 7fdfae58b640 1 -- 172.21.15.117:0/1479336698 learned_addr learned my addr 172.21.15.117:0/1479336698 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:56.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.283+0000 7fdfad589640 1 --2- 172.21.15.117:0/1479336698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c019991670 0x55c019d65590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:56.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.283+0000 7fdfae58b640 1 -- 172.21.15.117:0/1479336698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c019991670 msgr2=0x55c019d65590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:56.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.283+0000 7fdfae58b640 1 --2- 172.21.15.117:0/1479336698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c019991670 0x55c019d65590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.283+0000 7fdfae58b640 1 -- 172.21.15.117:0/1479336698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c019989490 msgr2=0x55c019d65050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:56.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.283+0000 7fdfae58b640 1 --2- 172.21.15.117:0/1479336698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c019989490 0x55c019d65050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.283+0000 7fdfae58b640 1 -- 172.21.15.117:0/1479336698 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c019d71580 con 0x55c0199b2350 2024-09-21T06:04:56.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.283+0000 7fdfae58b640 1 --2- 172.21.15.117:0/1479336698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0199b2350 0x55c019d70f90 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0031700 tx=0x7fdfa00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e73567432b2aae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:56.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.283+0000 7fdf917fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:04:56.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.283+0000 7fdf917fa640 1 -- 172.21.15.117:0/1479336698 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa004f070 con 0x55c0199b2350 2024-09-21T06:04:56.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.283+0000 7fdfb6562740 1 -- 172.21.15.117:0/1479336698 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfa400be20 con 0x55c0199b2350 2024-09-21T06:04:56.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.283+0000 7fdf917fa640 1 -- 172.21.15.117:0/1479336698 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7fdfa0004240 con 0x55c0199b2350 2024-09-21T06:04:56.292 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.284+0000 7fdf917fa640 1 -- 172.21.15.117:0/1479336698 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdfa0004440 con 0x55c0199b2350 2024-09-21T06:04:56.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.284+0000 7fdfb6562740 10 client.16887 fetch_fsmap learned FSMap version 29 2024-09-21T06:04:56.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.284+0000 7fdfb6562740 1 -- 172.21.15.117:0/1479336698 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c019d73e90 con 0x55c0199b2350 2024-09-21T06:04:56.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.284+0000 7fdf917fa640 1 -- 172.21.15.117:0/1479336698 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfa400be20 con 0x55c0199b2350 2024-09-21T06:04:56.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.284+0000 7fdf917fa640 1 -- 172.21.15.117:0/1479336698 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdfa0044040 con 0x55c0199b2350 2024-09-21T06:04:56.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.284+0000 7fdfb6562740 10 client.16887 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:04:56.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.284+0000 7fdfb6562740 10 client.16887 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:04:56.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.284+0000 7fdfb6562740 20 client.16887 populate_metadata read hostname 'smithi117' 2024-09-21T06:04:56.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.284+0000 7fdfb6562740 1 --2- 172.21.15.117:0/1479336698 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c019970cd0 0x55c019d77450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:56.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.284+0000 7fdfb6562740 4 client.16887 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:04:56.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.284+0000 7fdfb6562740 1 -- 172.21.15.117:0/1479336698 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c019cd1db0 con 0x55c019970cd0 2024-09-21T06:04:56.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.284+0000 7fdfadd8a640 1 --2- 172.21.15.117:0/1479336698 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c019970cd0 0x55c019d77450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:56.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.285+0000 7fdfadd8a640 1 --2- 172.21.15.117:0/1479336698 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c019970cd0 0x55c019d77450 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=300a805bc2b8e32c server_cookie=2d8aad40a0bec828 in_seq=0 out_seq=0 2024-09-21T06:04:56.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.285+0000 7fdf917fa640 10 client.16887 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:56.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.285+0000 7fdf917fa640 1 -- 172.21.15.117:0/1479336698 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c019cd1db0 con 0x55c019970cd0 2024-09-21T06:04:56.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.285+0000 7fdf917fa640 10 client.16887 handle_command_reply: tid=0 2024-09-21T06:04:56.293 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.285+0000 7fdfb6562740 1 client.16887 shutdown 2024-09-21T06:04:56.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.286+0000 7fdfb6562740 1 -- 172.21.15.117:0/1479336698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0199b2350 msgr2=0x55c019d70f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:56.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.286+0000 7fdfb6562740 1 --2- 172.21.15.117:0/1479336698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0199b2350 0x55c019d70f90 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0031700 tx=0x7fdfa00095c0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.286+0000 7fdfb6562740 1 -- 172.21.15.117:0/1479336698 shutdown_connections 2024-09-21T06:04:56.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.286+0000 7fdfb6562740 1 --2- 172.21.15.117:0/1479336698 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c019970cd0 0x55c019d77450 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.286+0000 7fdfb6562740 1 --2- 172.21.15.117:0/1479336698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0199b2350 0x55c019d70f90 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.286+0000 7fdfb6562740 1 --2- 172.21.15.117:0/1479336698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c019991670 0x55c019d65590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.286+0000 7fdfb6562740 1 --2- 172.21.15.117:0/1479336698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c019989490 0x55c019d65050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.286+0000 7fdf917fa640 0 client.16887 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:56.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.286+0000 7fdfb6562740 1 -- 172.21.15.117:0/1479336698 >> 172.21.15.117:0/1479336698 conn(0x55c0199a6740 msgr2=0x55c019d5c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:56.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.286+0000 7fdfb6562740 1 -- 172.21.15.117:0/1479336698 shutdown_connections 2024-09-21T06:04:56.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.286+0000 7fdfb6562740 1 -- 172.21.15.117:0/1479336698 wait complete. 2024-09-21T06:04:56.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.286+0000 7fdfb6562740 20 client.16887 trim_cache size 0 max 16384 2024-09-21T06:04:56.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.303+0000 7fdfb6562740 1 Processor -- start 2024-09-21T06:04:56.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.303+0000 7fdfb6562740 1 -- start start 2024-09-21T06:04:56.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.303+0000 7fdfb6562740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c019989490 0x55c019d5ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:56.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.303+0000 7fdfb6562740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c019991670 0x55c019d5ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:56.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.303+0000 7fdfb6562740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0199b2350 0x55c019d56860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:56.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.303+0000 7fdfb6562740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8c00e680 con 0x55c019989490 2024-09-21T06:04:56.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.303+0000 7fdfb6562740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8c0101c0 con 0x55c019991670 2024-09-21T06:04:56.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.303+0000 7fdfb6562740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8c00ed50 con 0x55c0199b2350 2024-09-21T06:04:56.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.304+0000 7fdfadd8a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c019989490 0x55c019d5ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:56.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.304+0000 7fdfadd8a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c019989490 0x55c019d5ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:46610/0 (socket says 172.21.15.117:46610) 2024-09-21T06:04:56.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.304+0000 7fdfadd8a640 1 -- 172.21.15.117:0/3375030936 learned_addr learned my addr 172.21.15.117:0/3375030936 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:56.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.304+0000 7fdfadd8a640 1 -- 172.21.15.117:0/3375030936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0199b2350 msgr2=0x55c019d56860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:56.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.304+0000 7fdfadd8a640 1 --2- 172.21.15.117:0/3375030936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0199b2350 0x55c019d56860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.304+0000 7fdfadd8a640 1 -- 172.21.15.117:0/3375030936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c019991670 msgr2=0x55c019d5ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:56.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.304+0000 7fdfad589640 1 --2- 172.21.15.117:0/3375030936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c019991670 0x55c019d5ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:56.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.304+0000 7fdfadd8a640 1 --2- 172.21.15.117:0/3375030936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c019991670 0x55c019d5ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.304+0000 7fdfadd8a640 1 -- 172.21.15.117:0/3375030936 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c019d63350 con 0x55c019989490 2024-09-21T06:04:56.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.304+0000 7fdfadd8a640 1 --2- 172.21.15.117:0/3375030936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c019989490 0x55c019d5ebb0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x55c019d580b0 tx=0x7fdfa400b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7abca60b03816652 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:56.309 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.304+0000 7fdf937fe640 1 -- 172.21.15.117:0/3375030936 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4015040 con 0x55c019989490 2024-09-21T06:04:56.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.304+0000 7fdf937fe640 1 -- 172.21.15.117:0/3375030936 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfa4031070 con 0x55c019989490 2024-09-21T06:04:56.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.304+0000 7fdf937fe640 1 -- 172.21.15.117:0/3375030936 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa402c6b0 con 0x55c019989490 2024-09-21T06:04:56.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.305+0000 7fdfb6562740 1 -- 172.21.15.117:0/3375030936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c019989490 msgr2=0x55c019d5ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:56.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.305+0000 7fdfb6562740 1 --2- 172.21.15.117:0/3375030936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c019989490 0x55c019d5ebb0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x55c019d580b0 tx=0x7fdfa400b5c0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.305+0000 7fdfb6562740 1 -- 172.21.15.117:0/3375030936 shutdown_connections 2024-09-21T06:04:56.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.305+0000 7fdfb6562740 1 --2- 172.21.15.117:0/3375030936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0199b2350 0x55c019d56860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.305+0000 7fdfb6562740 1 --2- 172.21.15.117:0/3375030936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c019991670 0x55c019d5ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.305+0000 7fdfb6562740 1 --2- 172.21.15.117:0/3375030936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c019989490 0x55c019d5ebb0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.305+0000 7fdfb6562740 1 -- 172.21.15.117:0/3375030936 >> 172.21.15.117:0/3375030936 conn(0x55c0199a6740 msgr2=0x55c019d5a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:56.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.305+0000 7fdfb6562740 1 -- 172.21.15.117:0/3375030936 shutdown_connections 2024-09-21T06:04:56.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.305+0000 7fdfb6562740 1 -- 172.21.15.117:0/3375030936 wait complete. 2024-09-21T06:04:56.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.305+0000 7fdfb6562740 1 -- start start 2024-09-21T06:04:56.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.305+0000 7fdfb6562740 1 Processor -- start 2024-09-21T06:04:56.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.306+0000 7fdfb6562740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c019989490 0x55c019d69770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:56.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.306+0000 7fdfb6562740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c019991670 0x55c019d69d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:56.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.306+0000 7fdfb6562740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0199b2350 0x55c019ddae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:56.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.306+0000 7fdfb6562740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8c00faf0 con 0x55c019991670 2024-09-21T06:04:56.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.306+0000 7fdfb6562740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8c00f420 con 0x55c019989490 2024-09-21T06:04:56.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.306+0000 7fdfb6562740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8c00c680 con 0x55c0199b2350 2024-09-21T06:04:56.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.306+0000 7fdfb6562740 10 client.0 _ll_register_callbacks cb 0x7fdfa0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:56.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.306+0000 7fdfad589640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c019991670 0x55c019d69d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:56.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.306+0000 7fdfad589640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c019991670 0x55c019d69d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:46626/0 (socket says 172.21.15.117:46626) 2024-09-21T06:04:56.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.306+0000 7fdfad589640 1 -- 172.21.15.117:0/1031661321 learned_addr learned my addr 172.21.15.117:0/1031661321 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:56.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.306+0000 7fdfae58b640 1 --2- 172.21.15.117:0/1031661321 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0199b2350 0x55c019ddae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:56.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.306+0000 7fdfadd8a640 1 --2- 172.21.15.117:0/1031661321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c019989490 0x55c019d69770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:56.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.306+0000 7fdfad589640 1 -- 172.21.15.117:0/1031661321 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0199b2350 msgr2=0x55c019ddae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:56.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.306+0000 7fdfad589640 1 --2- 172.21.15.117:0/1031661321 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0199b2350 0x55c019ddae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.306+0000 7fdfad589640 1 -- 172.21.15.117:0/1031661321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c019989490 msgr2=0x55c019d69770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:56.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.306+0000 7fdfad589640 1 --2- 172.21.15.117:0/1031661321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c019989490 0x55c019d69770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.306+0000 7fdfad589640 1 -- 172.21.15.117:0/1031661321 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c019ddb490 con 0x55c019991670 2024-09-21T06:04:56.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.306+0000 7fdfadd8a640 1 --2- 172.21.15.117:0/1031661321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c019989490 0x55c019d69770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:04:56.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.306+0000 7fdfad589640 1 --2- 172.21.15.117:0/1031661321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c019991670 0x55c019d69d50 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fdf98009610 tx=0x7fdf980acbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f93989e27ea545 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:56.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.306+0000 7fdf917fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:04:56.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.307+0000 7fdf917fa640 1 -- 172.21.15.117:0/1031661321 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf980e4e60 con 0x55c019991670 2024-09-21T06:04:56.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.307+0000 7fdf917fa640 1 -- 172.21.15.117:0/1031661321 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7fdf98010a00 con 0x55c019991670 2024-09-21T06:04:56.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.307+0000 7fdfb6562740 1 -- 172.21.15.117:0/1031661321 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfa4031070 con 0x55c019991670 2024-09-21T06:04:56.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.307+0000 7fdf917fa640 1 -- 172.21.15.117:0/1031661321 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdf98021070 con 0x55c019991670 2024-09-21T06:04:56.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.307+0000 7fdfb6562740 10 client.16893 fetch_fsmap learned FSMap version 29 2024-09-21T06:04:56.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.307+0000 7fdfb6562740 1 -- 172.21.15.117:0/1031661321 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c019de27a0 con 0x55c019991670 2024-09-21T06:04:56.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.307+0000 7fdf917fa640 1 -- 172.21.15.117:0/1031661321 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfa4031070 con 0x55c019991670 2024-09-21T06:04:56.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.307+0000 7fdf917fa640 1 -- 172.21.15.117:0/1031661321 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdf980b63a0 con 0x55c019991670 2024-09-21T06:04:56.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.307+0000 7fdfb6562740 10 client.16893 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:04:56.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.307+0000 7fdfb6562740 10 client.16893 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:04:56.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.307+0000 7fdfb6562740 20 client.16893 populate_metadata read hostname 'smithi117' 2024-09-21T06:04:56.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.307+0000 7fdfb6562740 1 --2- 172.21.15.117:0/1031661321 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c019946080 0x55c019de4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:56.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.307+0000 7fdfb6562740 4 client.16893 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:04:56.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.307+0000 7fdfb6562740 1 -- 172.21.15.117:0/1031661321 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c019ccd860 con 0x55c019946080 2024-09-21T06:04:56.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.308+0000 7fdfadd8a640 1 --2- 172.21.15.117:0/1031661321 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c019946080 0x55c019de4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:56.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.308+0000 7fdfadd8a640 1 --2- 172.21.15.117:0/1031661321 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c019946080 0x55c019de4f40 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29ddef6e2bad2e0a server_cookie=aa12aa29b67b2823 in_seq=0 out_seq=0 2024-09-21T06:04:56.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.308+0000 7fdf917fa640 10 client.16893 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:56.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.310+0000 7fdf917fa640 1 -- 172.21.15.117:0/1031661321 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c019ccd860 con 0x55c019946080 2024-09-21T06:04:56.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.310+0000 7fdf917fa640 10 client.16893 handle_command_reply: tid=0 2024-09-21T06:04:56.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.310+0000 7fdfb6562740 1 client.16893 shutdown 2024-09-21T06:04:56.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.310+0000 7fdfb6562740 1 -- 172.21.15.117:0/1031661321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c019991670 msgr2=0x55c019d69d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:56.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.311+0000 7fdfb6562740 1 --2- 172.21.15.117:0/1031661321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c019991670 0x55c019d69d50 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fdf98009610 tx=0x7fdf980acbe0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.311+0000 7fdfb6562740 1 -- 172.21.15.117:0/1031661321 shutdown_connections 2024-09-21T06:04:56.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.311+0000 7fdfb6562740 1 --2- 172.21.15.117:0/1031661321 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c019946080 0x55c019de4f40 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.311+0000 7fdfb6562740 1 --2- 172.21.15.117:0/1031661321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c019991670 0x55c019d69d50 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.311+0000 7fdfb6562740 1 --2- 172.21.15.117:0/1031661321 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0199b2350 0x55c019ddae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.311+0000 7fdfb6562740 1 --2- 172.21.15.117:0/1031661321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c019989490 0x55c019d69770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.311+0000 7fdfb6562740 1 -- 172.21.15.117:0/1031661321 >> 172.21.15.117:0/1031661321 conn(0x55c0199a6740 msgr2=0x55c019d5a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:56.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.311+0000 7fdf917fa640 0 client.16893 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:56.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.311+0000 7fdfb6562740 1 -- 172.21.15.117:0/1031661321 shutdown_connections 2024-09-21T06:04:56.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.311+0000 7fdfb6562740 1 -- 172.21.15.117:0/1031661321 wait complete. 2024-09-21T06:04:56.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.311+0000 7fdfb6562740 20 client.16893 trim_cache size 0 max 16384 2024-09-21T06:04:56.315 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T06:04:56.315 INFO:teuthology.orchestra.run.smithi117.stdout: "return_code": 0, 2024-09-21T06:04:56.316 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_tag": "fb2f217d-463b-4c8f-ab19-920b04bb7306", 2024-09-21T06:04:56.316 INFO:teuthology.orchestra.run.smithi117.stdout: "mode": "asynchronous" 2024-09-21T06:04:56.316 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T06:04:56.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.311+0000 7fdfaffee640 1 -- 172.21.15.117:0/3123433520 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fdf940b6680 msgr2=0x7fdf940b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:56.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.311+0000 7fdfaffee640 1 --2- 172.21.15.117:0/3123433520 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fdf940b6680 0x7fdf940b8b90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdfa00094e0 tx=0x7fdfa00093f0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.311+0000 7fdfaffee640 1 -- 172.21.15.117:0/3123433520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfa81094a0 msgr2=0x7fdfa8075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:56.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.311+0000 7fdfaffee640 1 --2- 172.21.15.117:0/3123433520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfa81094a0 0x7fdfa8075c50 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fdf9800d740 tx=0x7fdf9800dc00 comp rx=0 tx=0).stop 2024-09-21T06:04:56.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.311+0000 7fdfaffee640 1 -- 172.21.15.117:0/3123433520 shutdown_connections 2024-09-21T06:04:56.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.311+0000 7fdfaffee640 1 --2- 172.21.15.117:0/3123433520 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fdf940b6680 0x7fdf940b8b90 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.311+0000 7fdfaffee640 1 --2- 172.21.15.117:0/3123433520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfa81094a0 0x7fdfa8075c50 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.312+0000 7fdfaffee640 1 --2- 172.21.15.117:0/3123433520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfa8109df0 0x7fdfa8076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.312+0000 7fdfaffee640 1 --2- 172.21.15.117:0/3123433520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfa8103b60 0x7fdfa8075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:56.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.312+0000 7fdfaffee640 1 -- 172.21.15.117:0/3123433520 >> 172.21.15.117:0/3123433520 conn(0x7fdfa807b850 msgr2=0x7fdfa8105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:56.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.312+0000 7fdfaffee640 1 -- 172.21.15.117:0/3123433520 shutdown_connections 2024-09-21T06:04:56.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:56.312+0000 7fdfaffee640 1 -- 172.21.15.117:0/3123433520 wait complete. 2024-09-21T06:04:56.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:55 smithi138 ceph-mon[34391]: pgmap v558: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.6 KiB/s wr, 13 op/s 2024-09-21T06:04:57.043 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:04:57.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:56 smithi192 ceph-mon[34700]: scrub queued for path: / 2024-09-21T06:04:57.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:56 smithi192 ceph-mon[34700]: scrub summary: idle+waiting paths [/] 2024-09-21T06:04:57.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:56 smithi192 ceph-mon[34700]: scrub summary: active paths [/] 2024-09-21T06:04:57.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:56 smithi192 ceph-mon[34700]: scrub summary: idle+waiting paths [/] 2024-09-21T06:04:57.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:56 smithi117 ceph-mon[30947]: scrub queued for path: / 2024-09-21T06:04:57.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:56 smithi117 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-21T06:04:57.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:56 smithi117 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-21T06:04:57.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:56 smithi117 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-21T06:04:57.290 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:04:57.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:56 smithi138 ceph-mon[34391]: scrub queued for path: / 2024-09-21T06:04:57.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:56 smithi138 ceph-mon[34391]: scrub summary: idle+waiting paths [/] 2024-09-21T06:04:57.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:56 smithi138 ceph-mon[34391]: scrub summary: active paths [/] 2024-09-21T06:04:57.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:56 smithi138 ceph-mon[34391]: scrub summary: idle+waiting paths [/] 2024-09-21T06:04:57.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.798+0000 7f387ba42640 1 -- 172.21.15.117:0/3991724315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38741083d0 msgr2=0x7f38741087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:57.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.798+0000 7f387ba42640 1 --2- 172.21.15.117:0/3991724315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38741083d0 0x7f38741087e0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f3864009900 tx=0x7f386402a410 comp rx=0 tx=0).stop 2024-09-21T06:04:57.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.798+0000 7f387ba42640 1 -- 172.21.15.117:0/3991724315 shutdown_connections 2024-09-21T06:04:57.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.798+0000 7f387ba42640 1 --2- 172.21.15.117:0/3991724315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38741083d0 0x7f38741087e0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.798+0000 7f387ba42640 1 --2- 172.21.15.117:0/3991724315 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3874108d20 0x7f387410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.798+0000 7f387ba42640 1 --2- 172.21.15.117:0/3991724315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3874102ae0 0x7f3874102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.798+0000 7f387ba42640 1 -- 172.21.15.117:0/3991724315 >> 172.21.15.117:0/3991724315 conn(0x7f3874079760 msgr2=0x7f3874079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:57.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.798+0000 7f387ba42640 1 -- 172.21.15.117:0/3991724315 shutdown_connections 2024-09-21T06:04:57.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.798+0000 7f387ba42640 1 -- 172.21.15.117:0/3991724315 wait complete. 2024-09-21T06:04:57.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.799+0000 7f387ba42640 1 Processor -- start 2024-09-21T06:04:57.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.799+0000 7f387ba42640 1 -- start start 2024-09-21T06:04:57.819 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.799+0000 7f387ba42640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3874102ae0 0x7f387419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:57.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.799+0000 7f387ba42640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38741083d0 0x7f387419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:57.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.799+0000 7f387ba42640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3874108d20 0x7f387419fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:57.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.799+0000 7f387ba42640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f387406c6b0 con 0x7f38741083d0 2024-09-21T06:04:57.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.799+0000 7f387ba42640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f387406c530 con 0x7f3874102ae0 2024-09-21T06:04:57.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.799+0000 7f387ba42640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f387406c830 con 0x7f3874108d20 2024-09-21T06:04:57.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.799+0000 7f3878fdd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38741083d0 0x7f387419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:57.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.799+0000 7f3878fdd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38741083d0 0x7f387419aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:46648/0 (socket says 172.21.15.117:46648) 2024-09-21T06:04:57.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.799+0000 7f3878fdd640 1 -- 172.21.15.117:0/1750722032 learned_addr learned my addr 172.21.15.117:0/1750722032 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:57.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.799+0000 7f38797de640 1 --2- 172.21.15.117:0/1750722032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3874102ae0 0x7f387419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:57.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.799+0000 7f3878fdd640 1 -- 172.21.15.117:0/1750722032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3874108d20 msgr2=0x7f387419fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:57.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.799+0000 7f3879fdf640 1 --2- 172.21.15.117:0/1750722032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3874108d20 0x7f387419fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:57.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.799+0000 7f3878fdd640 1 --2- 172.21.15.117:0/1750722032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3874108d20 0x7f387419fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.799+0000 7f3878fdd640 1 -- 172.21.15.117:0/1750722032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3874102ae0 msgr2=0x7f387419a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:57.820 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.799+0000 7f3878fdd640 1 --2- 172.21.15.117:0/1750722032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3874102ae0 0x7f387419a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.799+0000 7f3878fdd640 1 -- 172.21.15.117:0/1750722032 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38741a0490 con 0x7f38741083d0 2024-09-21T06:04:57.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.800+0000 7f3878fdd640 1 --2- 172.21.15.117:0/1750722032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38741083d0 0x7f387419aac0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f3864032fd0 tx=0x7f3864004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:57.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.800+0000 7f386a7fc640 1 -- 172.21.15.117:0/1750722032 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386403f030 con 0x7f38741083d0 2024-09-21T06:04:57.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.800+0000 7f387ba42640 1 -- 172.21.15.117:0/1750722032 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38741a0720 con 0x7f38741083d0 2024-09-21T06:04:57.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.800+0000 7f386a7fc640 1 -- 172.21.15.117:0/1750722032 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3864004490 con 0x7f38741083d0 2024-09-21T06:04:57.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.800+0000 7f386a7fc640 1 -- 172.21.15.117:0/1750722032 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3864035070 con 0x7f38741083d0 2024-09-21T06:04:57.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.800+0000 7f387ba42640 1 -- 172.21.15.117:0/1750722032 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38741a6ee0 con 0x7f38741083d0 2024-09-21T06:04:57.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.803+0000 7f3881fb6740 1 Processor -- start 2024-09-21T06:04:57.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.803+0000 7f386a7fc640 1 -- 172.21.15.117:0/1750722032 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f3864042030 con 0x7f38741083d0 2024-09-21T06:04:57.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.803+0000 7f3881fb6740 1 -- start start 2024-09-21T06:04:57.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.803+0000 7f3881fb6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558417c19070 0x558417fc0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:57.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.803+0000 7f3881fb6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558417bf01b0 0x558417fc0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:57.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.803+0000 7f3881fb6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558417bf8390 0x558417fc10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:57.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.803+0000 7f3881fb6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558417f36e50 con 0x558417bf8390 2024-09-21T06:04:57.821 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.803+0000 7f3881fb6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558417f26280 con 0x558417c19070 2024-09-21T06:04:57.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.803+0000 7f3881fb6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558417f2a090 con 0x558417bf01b0 2024-09-21T06:04:57.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.803+0000 7f3878fdd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558417bf8390 0x558417fc10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:57.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.803+0000 7f3878fdd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558417bf8390 0x558417fc10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:46654/0 (socket says 172.21.15.117:46654) 2024-09-21T06:04:57.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.803+0000 7f3878fdd640 1 -- 172.21.15.117:0/203983054 learned_addr learned my addr 172.21.15.117:0/203983054 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:57.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.803+0000 7f3879fdf640 1 --2- 172.21.15.117:0/203983054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558417c19070 0x558417fc0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:57.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.803+0000 7f38797de640 1 --2- 172.21.15.117:0/203983054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558417bf01b0 0x558417fc0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:57.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.804+0000 7f3878fdd640 1 -- 172.21.15.117:0/203983054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558417bf01b0 msgr2=0x558417fc0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:57.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.804+0000 7f3878fdd640 1 --2- 172.21.15.117:0/203983054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558417bf01b0 0x558417fc0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.804+0000 7f3878fdd640 1 -- 172.21.15.117:0/203983054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558417c19070 msgr2=0x558417fc0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:57.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.804+0000 7f3878fdd640 1 --2- 172.21.15.117:0/203983054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558417c19070 0x558417fc0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.804+0000 7f3878fdd640 1 -- 172.21.15.117:0/203983054 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558417c84650 con 0x558417bf8390 2024-09-21T06:04:57.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.804+0000 7f38797de640 1 --2- 172.21.15.117:0/203983054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558417bf01b0 0x558417fc0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:04:57.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.804+0000 7f386a7fc640 1 --2- 172.21.15.117:0/1750722032 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f38600b6680 0x7f38600b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:57.822 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.804+0000 7f3878fdd640 1 --2- 172.21.15.117:0/203983054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558417bf8390 0x558417fc10d0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f3864104d30 tx=0x7f386410dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=709b61480fb3b290 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:57.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.804+0000 7f3879fdf640 1 --2- 172.21.15.117:0/1750722032 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f38600b6680 0x7f38600b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:57.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.804+0000 7f385f7fe640 1 -- 172.21.15.117:0/203983054 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38640c9b10 con 0x558417bf8390 2024-09-21T06:04:57.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.804+0000 7f386a7fc640 1 -- 172.21.15.117:0/1750722032 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f3864107620 con 0x7f38741083d0 2024-09-21T06:04:57.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.804+0000 7f385f7fe640 1 -- 172.21.15.117:0/203983054 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38640ccfd0 con 0x558417bf8390 2024-09-21T06:04:57.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.804+0000 7f385f7fe640 1 -- 172.21.15.117:0/203983054 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38640f6130 con 0x558417bf8390 2024-09-21T06:04:57.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.805+0000 7f3881fb6740 1 -- 172.21.15.117:0/203983054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558417bf8390 msgr2=0x558417fc10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:57.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.805+0000 7f3879fdf640 1 --2- 172.21.15.117:0/1750722032 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f38600b6680 0x7f38600b8b90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f386c007ed0 tx=0x7f386c0075c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:57.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.805+0000 7f3881fb6740 1 --2- 172.21.15.117:0/203983054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558417bf8390 0x558417fc10d0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f3864104d30 tx=0x7f386410dcd0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.805+0000 7f3881fb6740 1 -- 172.21.15.117:0/203983054 shutdown_connections 2024-09-21T06:04:57.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.805+0000 7f3881fb6740 1 --2- 172.21.15.117:0/203983054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558417bf8390 0x558417fc10d0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.805+0000 7f3881fb6740 1 --2- 172.21.15.117:0/203983054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558417bf01b0 0x558417fc0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.805+0000 7f3881fb6740 1 --2- 172.21.15.117:0/203983054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558417c19070 0x558417fc0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.805+0000 7f3881fb6740 1 -- 172.21.15.117:0/203983054 >> 172.21.15.117:0/203983054 conn(0x558417c0d460 msgr2=0x558417fbe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:57.823 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.805+0000 7f3881fb6740 1 -- 172.21.15.117:0/203983054 shutdown_connections 2024-09-21T06:04:57.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.805+0000 7f3881fb6740 1 -- 172.21.15.117:0/203983054 wait complete. 2024-09-21T06:04:57.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.805+0000 7f3881fb6740 1 -- start start 2024-09-21T06:04:57.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.806+0000 7f3881fb6740 1 Processor -- start 2024-09-21T06:04:57.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.806+0000 7f3881fb6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558417bf01b0 0x558417fcbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:57.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.806+0000 7f3881fb6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558417bf8390 0x558417fcc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:57.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.806+0000 7f3881fb6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558417c19070 0x558417fcc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:57.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.806+0000 7f3881fb6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558417d87700 con 0x558417bf8390 2024-09-21T06:04:57.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.806+0000 7f3881fb6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558417ec1110 con 0x558417bf01b0 2024-09-21T06:04:57.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.806+0000 7f3881fb6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558417fcebf0 con 0x558417c19070 2024-09-21T06:04:57.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.806+0000 7f3881fb6740 10 client.0 _ll_register_callbacks cb 0x558417eda990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:57.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.806+0000 7f3878fdd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558417bf8390 0x558417fcc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:57.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.806+0000 7f3878fdd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558417bf8390 0x558417fcc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:46668/0 (socket says 172.21.15.117:46668) 2024-09-21T06:04:57.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.806+0000 7f3878fdd640 1 -- 172.21.15.117:0/1040222911 learned_addr learned my addr 172.21.15.117:0/1040222911 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:57.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.806+0000 7f38797de640 1 --2- 172.21.15.117:0/1040222911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558417bf01b0 0x558417fcbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:57.824 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.806+0000 7f3879fdf640 1 --2- 172.21.15.117:0/1040222911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558417c19070 0x558417fcc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:57.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.806+0000 7f3878fdd640 1 -- 172.21.15.117:0/1040222911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558417c19070 msgr2=0x558417fcc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:57.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.806+0000 7f3878fdd640 1 --2- 172.21.15.117:0/1040222911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558417c19070 0x558417fcc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.806+0000 7f3878fdd640 1 -- 172.21.15.117:0/1040222911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558417bf01b0 msgr2=0x558417fcbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:57.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.806+0000 7f3878fdd640 1 --2- 172.21.15.117:0/1040222911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558417bf01b0 0x558417fcbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.806+0000 7f3878fdd640 1 -- 172.21.15.117:0/1040222911 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558417eeec40 con 0x558417bf8390 2024-09-21T06:04:57.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.807+0000 7f3879fdf640 1 --2- 172.21.15.117:0/1040222911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558417c19070 0x558417fcc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:57.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.807+0000 7f3878fdd640 1 --2- 172.21.15.117:0/1040222911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558417bf8390 0x558417fcc1d0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f3864104e70 tx=0x7f3864000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a47836f9124b67 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:57.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.807+0000 7f385cff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:04:57.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.807+0000 7f385cff9640 1 -- 172.21.15.117:0/1040222911 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3864049070 con 0x558417bf8390 2024-09-21T06:04:57.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.807+0000 7f385cff9640 1 -- 172.21.15.117:0/1040222911 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f386403bc20 con 0x558417bf8390 2024-09-21T06:04:57.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.807+0000 7f3881fb6740 1 -- 172.21.15.117:0/1040222911 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38640ccfd0 con 0x558417bf8390 2024-09-21T06:04:57.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.807+0000 7f385cff9640 1 -- 172.21.15.117:0/1040222911 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f38640b8020 con 0x558417bf8390 2024-09-21T06:04:57.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.807+0000 7f3881fb6740 10 client.16905 fetch_fsmap learned FSMap version 29 2024-09-21T06:04:57.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.807+0000 7f3881fb6740 1 -- 172.21.15.117:0/1040222911 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558417fda800 con 0x558417bf8390 2024-09-21T06:04:57.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.807+0000 7f385cff9640 1 -- 172.21.15.117:0/1040222911 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38640ccfd0 con 0x558417bf8390 2024-09-21T06:04:57.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.807+0000 7f385cff9640 1 -- 172.21.15.117:0/1040222911 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f38640d6a50 con 0x558417bf8390 2024-09-21T06:04:57.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.807+0000 7f3881fb6740 10 client.16905 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:04:57.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.807+0000 7f3881fb6740 10 client.16905 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:04:57.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.807+0000 7f3881fb6740 20 client.16905 populate_metadata read hostname 'smithi117' 2024-09-21T06:04:57.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.807+0000 7f3881fb6740 1 --2- 172.21.15.117:0/1040222911 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x558417bd79f0 0x558417fddb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:57.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.807+0000 7f3881fb6740 4 client.16905 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:04:57.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.807+0000 7f3881fb6740 1 -- 172.21.15.117:0/1040222911 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558417f38da0 con 0x558417bd79f0 2024-09-21T06:04:57.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.808+0000 7f38797de640 1 --2- 172.21.15.117:0/1040222911 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x558417bd79f0 0x558417fddb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:57.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.808+0000 7f38797de640 1 --2- 172.21.15.117:0/1040222911 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x558417bd79f0 0x558417fddb60 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a18fe065d36d80c0 server_cookie=2b315e8c3fa7a812 in_seq=0 out_seq=0 2024-09-21T06:04:57.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.808+0000 7f385cff9640 10 client.16905 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:57.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.809+0000 7f385cff9640 1 -- 172.21.15.117:0/1040222911 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558417f38da0 con 0x558417bd79f0 2024-09-21T06:04:57.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.809+0000 7f385cff9640 10 client.16905 handle_command_reply: tid=0 2024-09-21T06:04:57.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.809+0000 7f3881fb6740 1 client.16905 shutdown 2024-09-21T06:04:57.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.809+0000 7f3881fb6740 1 -- 172.21.15.117:0/1040222911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558417bf8390 msgr2=0x558417fcc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:57.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.809+0000 7f3881fb6740 1 --2- 172.21.15.117:0/1040222911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558417bf8390 0x558417fcc1d0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f3864104e70 tx=0x7f3864000bc0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.809+0000 7f3881fb6740 1 -- 172.21.15.117:0/1040222911 shutdown_connections 2024-09-21T06:04:57.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.809+0000 7f3881fb6740 1 --2- 172.21.15.117:0/1040222911 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x558417bd79f0 0x558417fddb60 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.809+0000 7f3881fb6740 1 --2- 172.21.15.117:0/1040222911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558417bf8390 0x558417fcc1d0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.809+0000 7f3881fb6740 1 --2- 172.21.15.117:0/1040222911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558417c19070 0x558417fcc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.809+0000 7f385cff9640 0 client.16905 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:57.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.809+0000 7f3881fb6740 1 --2- 172.21.15.117:0/1040222911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558417bf01b0 0x558417fcbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.809+0000 7f3881fb6740 1 -- 172.21.15.117:0/1040222911 >> 172.21.15.117:0/1040222911 conn(0x558417c0d460 msgr2=0x558417fc3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:57.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.809+0000 7f3881fb6740 1 -- 172.21.15.117:0/1040222911 shutdown_connections 2024-09-21T06:04:57.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.809+0000 7f3881fb6740 1 -- 172.21.15.117:0/1040222911 wait complete. 2024-09-21T06:04:57.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.809+0000 7f3881fb6740 20 client.16905 trim_cache size 0 max 16384 2024-09-21T06:04:57.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.827+0000 7f3881fb6740 1 Processor -- start 2024-09-21T06:04:57.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.827+0000 7f3881fb6740 1 -- start start 2024-09-21T06:04:57.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.827+0000 7f3881fb6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558417bf01b0 0x558417fc1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:57.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.827+0000 7f3881fb6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558417bf8390 0x558417fc8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:57.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.827+0000 7f3881fb6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558417c19070 0x558417fc93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:57.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.827+0000 7f3881fb6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f38741af680 con 0x558417bf01b0 2024-09-21T06:04:57.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.827+0000 7f3881fb6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f38741b11c0 con 0x558417c19070 2024-09-21T06:04:57.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.827+0000 7f3881fb6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f38741afd50 con 0x558417bf8390 2024-09-21T06:04:57.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.827+0000 7f38797de640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558417bf01b0 0x558417fc1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:57.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.827+0000 7f38797de640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558417bf01b0 0x558417fc1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:46680/0 (socket says 172.21.15.117:46680) 2024-09-21T06:04:57.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.827+0000 7f38797de640 1 -- 172.21.15.117:0/1118367177 learned_addr learned my addr 172.21.15.117:0/1118367177 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:57.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.828+0000 7f3879fdf640 1 --2- 172.21.15.117:0/1118367177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558417c19070 0x558417fc93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:57.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.828+0000 7f3878fdd640 1 --2- 172.21.15.117:0/1118367177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558417bf8390 0x558417fc8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:57.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.828+0000 7f38797de640 1 -- 172.21.15.117:0/1118367177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558417bf8390 msgr2=0x558417fc8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:57.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.828+0000 7f38797de640 1 --2- 172.21.15.117:0/1118367177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558417bf8390 0x558417fc8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.828+0000 7f38797de640 1 -- 172.21.15.117:0/1118367177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558417c19070 msgr2=0x558417fc93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:57.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.828+0000 7f38797de640 1 --2- 172.21.15.117:0/1118367177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558417c19070 0x558417fc93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.828+0000 7f38797de640 1 -- 172.21.15.117:0/1118367177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558417fcabf0 con 0x558417bf01b0 2024-09-21T06:04:57.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.828+0000 7f3879fdf640 1 --2- 172.21.15.117:0/1118367177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558417c19070 0x558417fc93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:57.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.828+0000 7f38797de640 1 --2- 172.21.15.117:0/1118367177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558417bf01b0 0x558417fc1d90 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x558417fde190 tx=0x7f387000bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b1a0f748b7eb7f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:57.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.828+0000 7f385effd640 1 -- 172.21.15.117:0/1118367177 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3870003ac0 con 0x558417bf01b0 2024-09-21T06:04:57.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.828+0000 7f385effd640 1 -- 172.21.15.117:0/1118367177 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38700054d0 con 0x558417bf01b0 2024-09-21T06:04:57.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.828+0000 7f385effd640 1 -- 172.21.15.117:0/1118367177 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3870004cd0 con 0x558417bf01b0 2024-09-21T06:04:57.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.829+0000 7f3881fb6740 1 -- 172.21.15.117:0/1118367177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558417bf01b0 msgr2=0x558417fc1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:57.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.829+0000 7f3881fb6740 1 --2- 172.21.15.117:0/1118367177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558417bf01b0 0x558417fc1d90 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x558417fde190 tx=0x7f387000bdb0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.829+0000 7f3881fb6740 1 -- 172.21.15.117:0/1118367177 shutdown_connections 2024-09-21T06:04:57.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.829+0000 7f3881fb6740 1 --2- 172.21.15.117:0/1118367177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558417c19070 0x558417fc93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.829+0000 7f3881fb6740 1 --2- 172.21.15.117:0/1118367177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558417bf8390 0x558417fc8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.829+0000 7f3881fb6740 1 --2- 172.21.15.117:0/1118367177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558417bf01b0 0x558417fc1d90 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.829+0000 7f3881fb6740 1 -- 172.21.15.117:0/1118367177 >> 172.21.15.117:0/1118367177 conn(0x558417c0d460 msgr2=0x558417fc1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:57.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.829+0000 7f3881fb6740 1 -- 172.21.15.117:0/1118367177 shutdown_connections 2024-09-21T06:04:57.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.829+0000 7f3881fb6740 1 -- 172.21.15.117:0/1118367177 wait complete. 2024-09-21T06:04:57.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.829+0000 7f3881fb6740 1 -- start start 2024-09-21T06:04:57.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.829+0000 7f3881fb6740 1 Processor -- start 2024-09-21T06:04:57.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.830+0000 7f3881fb6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558417bf01b0 0x558417fcfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:57.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.830+0000 7f3881fb6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558417bf8390 0x558417fd0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:57.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.830+0000 7f3881fb6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558417c19070 0x558417fd0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:57.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.830+0000 7f3881fb6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f38741b0af0 con 0x558417bf8390 2024-09-21T06:04:57.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.830+0000 7f3881fb6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f38741b0420 con 0x558417c19070 2024-09-21T06:04:57.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.830+0000 7f3881fb6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f38741ad680 con 0x558417bf01b0 2024-09-21T06:04:57.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.830+0000 7f3881fb6740 10 client.0 _ll_register_callbacks cb 0x7f38640f3070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:04:57.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.830+0000 7f3878fdd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558417bf8390 0x558417fd0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:57.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.830+0000 7f3878fdd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558417bf8390 0x558417fd0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:46684/0 (socket says 172.21.15.117:46684) 2024-09-21T06:04:57.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.830+0000 7f3878fdd640 1 -- 172.21.15.117:0/3432160145 learned_addr learned my addr 172.21.15.117:0/3432160145 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:04:57.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.830+0000 7f38797de640 1 --2- 172.21.15.117:0/3432160145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558417bf01b0 0x558417fcfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:57.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.830+0000 7f3879fdf640 1 --2- 172.21.15.117:0/3432160145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558417c19070 0x558417fd0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:57.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.830+0000 7f3878fdd640 1 -- 172.21.15.117:0/3432160145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558417bf01b0 msgr2=0x558417fcfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:57.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.830+0000 7f3878fdd640 1 --2- 172.21.15.117:0/3432160145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558417bf01b0 0x558417fcfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.830+0000 7f3878fdd640 1 -- 172.21.15.117:0/3432160145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558417c19070 msgr2=0x558417fd0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:04:57.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.830+0000 7f3878fdd640 1 --2- 172.21.15.117:0/3432160145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558417c19070 0x558417fd0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.830+0000 7f3878fdd640 1 -- 172.21.15.117:0/3432160145 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558418047520 con 0x558417bf8390 2024-09-21T06:04:57.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.830+0000 7f38797de640 1 --2- 172.21.15.117:0/3432160145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558417bf01b0 0x558417fcfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:57.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.830+0000 7f3879fdf640 1 --2- 172.21.15.117:0/3432160145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558417c19070 0x558417fd0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:04:57.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.831+0000 7f3878fdd640 1 --2- 172.21.15.117:0/3432160145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558417bf8390 0x558417fd0400 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f38640ca730 tx=0x7f386410dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf82d7a29064916 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:04:57.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.831+0000 7f385cff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:04:57.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.831+0000 7f385cff9640 1 -- 172.21.15.117:0/3432160145 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3864049070 con 0x558417bf8390 2024-09-21T06:04:57.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.831+0000 7f385cff9640 1 -- 172.21.15.117:0/3432160145 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f38640c7dc0 con 0x558417bf8390 2024-09-21T06:04:57.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.831+0000 7f3881fb6740 1 -- 172.21.15.117:0/3432160145 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38700054d0 con 0x558417bf8390 2024-09-21T06:04:57.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.831+0000 7f385cff9640 1 -- 172.21.15.117:0/3432160145 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f38640f6db0 con 0x558417bf8390 2024-09-21T06:04:57.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.831+0000 7f3881fb6740 10 client.16911 fetch_fsmap learned FSMap version 29 2024-09-21T06:04:57.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.831+0000 7f3881fb6740 1 -- 172.21.15.117:0/3432160145 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558418047520 con 0x558417bf8390 2024-09-21T06:04:57.837 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.831+0000 7f385cff9640 1 -- 172.21.15.117:0/3432160145 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38700054d0 con 0x558417bf8390 2024-09-21T06:04:57.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.831+0000 7f385cff9640 1 -- 172.21.15.117:0/3432160145 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f38640e8b10 con 0x558417bf8390 2024-09-21T06:04:57.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.831+0000 7f3881fb6740 10 client.16911 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:04:57.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.831+0000 7f3881fb6740 10 client.16911 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:04:57.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.831+0000 7f3881fb6740 20 client.16911 populate_metadata read hostname 'smithi117' 2024-09-21T06:04:57.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.831+0000 7f3881fb6740 1 --2- 172.21.15.117:0/3432160145 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x558417bacda0 0x55841804aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:04:57.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.831+0000 7f3881fb6740 4 client.16911 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:04:57.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.831+0000 7f3881fb6740 1 -- 172.21.15.117:0/3432160145 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558417f34850 con 0x558417bacda0 2024-09-21T06:04:57.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.832+0000 7f38797de640 1 --2- 172.21.15.117:0/3432160145 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x558417bacda0 0x55841804aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:04:57.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.832+0000 7f38797de640 1 --2- 172.21.15.117:0/3432160145 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x558417bacda0 0x55841804aad0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=740a4478cc037540 server_cookie=1da8cfa6f50edb59 in_seq=0 out_seq=0 2024-09-21T06:04:57.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.832+0000 7f385cff9640 10 client.16911 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:57.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.832+0000 7f385cff9640 1 -- 172.21.15.117:0/3432160145 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2274 (crc 0 0 0) 0x558417f34850 con 0x558417bacda0 2024-09-21T06:04:57.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.832+0000 7f385cff9640 10 client.16911 handle_command_reply: tid=0 2024-09-21T06:04:57.838 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.832+0000 7f3881fb6740 1 client.16911 shutdown 2024-09-21T06:04:57.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.832+0000 7f3881fb6740 1 -- 172.21.15.117:0/3432160145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558417bf8390 msgr2=0x558417fd0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:57.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.832+0000 7f3881fb6740 1 --2- 172.21.15.117:0/3432160145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558417bf8390 0x558417fd0400 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f38640ca730 tx=0x7f386410dcd0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.833+0000 7f3881fb6740 1 -- 172.21.15.117:0/3432160145 shutdown_connections 2024-09-21T06:04:57.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.833+0000 7f3881fb6740 1 --2- 172.21.15.117:0/3432160145 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x558417bacda0 0x55841804aad0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.833+0000 7f3881fb6740 1 --2- 172.21.15.117:0/3432160145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558417bf8390 0x558417fd0400 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.833+0000 7f3881fb6740 1 --2- 172.21.15.117:0/3432160145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558417c19070 0x558417fd0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.833+0000 7f3881fb6740 1 --2- 172.21.15.117:0/3432160145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558417bf01b0 0x558417fcfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.833+0000 7f3881fb6740 1 -- 172.21.15.117:0/3432160145 >> 172.21.15.117:0/3432160145 conn(0x558417c0d460 msgr2=0x558417fd6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:57.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.833+0000 7f385cff9640 0 client.16911 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:04:57.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.833+0000 7f3881fb6740 1 -- 172.21.15.117:0/3432160145 shutdown_connections 2024-09-21T06:04:57.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.833+0000 7f3881fb6740 1 -- 172.21.15.117:0/3432160145 wait complete. 2024-09-21T06:04:57.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.833+0000 7f3881fb6740 20 client.16911 trim_cache size 0 max 16384 2024-09-21T06:04:57.840 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T06:04:57.840 INFO:teuthology.orchestra.run.smithi117.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:04:57.840 INFO:teuthology.orchestra.run.smithi117.stdout: "scrubs": { 2024-09-21T06:04:57.840 INFO:teuthology.orchestra.run.smithi117.stdout: "fb2f217d-463b-4c8f-ab19-920b04bb7306": { 2024-09-21T06:04:57.840 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:04:57.840 INFO:teuthology.orchestra.run.smithi117.stdout: "tag": "fb2f217d-463b-4c8f-ab19-920b04bb7306", 2024-09-21T06:04:57.840 INFO:teuthology.orchestra.run.smithi117.stdout: "options": "recursive,force" 2024-09-21T06:04:57.840 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:04:57.840 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:04:57.840 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_stats": { 2024-09-21T06:04:57.840 INFO:teuthology.orchestra.run.smithi117.stdout: "29ff9b7c-49d9-4910-85be-bace7e940cd5": { 2024-09-21T06:04:57.840 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:03:03.775527+0000", 2024-09-21T06:04:57.841 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:04:57.841 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:04:57.841 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:04:57.841 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:04:57.841 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:04:57.841 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:04:57.841 INFO:teuthology.orchestra.run.smithi117.stdout: "0902e33a-ce33-45fb-92d8-2e6fcb8ec8a0": { 2024-09-21T06:04:57.841 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:04:11.365073+0000", 2024-09-21T06:04:57.841 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:04:57.841 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:04:57.841 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:04:57.841 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:04:57.841 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:04:57.841 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:04:57.841 INFO:teuthology.orchestra.run.smithi117.stdout: "d543b0ae-b710-4153-99da-fe5714839c1a": { 2024-09-21T06:04:57.842 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:01:20.574754+0000", 2024-09-21T06:04:57.842 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:04:57.842 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:04:57.842 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:04:57.842 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:04:57.842 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:04:57.842 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:04:57.842 INFO:teuthology.orchestra.run.smithi117.stdout: "ad80d9f1-6aff-46c0-91ab-2c1ab0681039": { 2024-09-21T06:04:57.842 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:59:31.949108+0000", 2024-09-21T06:04:57.842 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:04:57.842 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:04:57.842 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:04:57.842 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:04:57.842 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:04:57.843 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:04:57.843 INFO:teuthology.orchestra.run.smithi117.stdout: "fb2f217d-463b-4c8f-ab19-920b04bb7306": { 2024-09-21T06:04:57.843 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:04:56.308107+0000", 2024-09-21T06:04:57.843 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:04:57.843 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:04:57.843 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:04:57.843 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:04:57.843 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:04:57.843 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:04:57.843 INFO:teuthology.orchestra.run.smithi117.stdout: "8b3fe95a-b76b-4e94-afd0-72fe35e2dd92": { 2024-09-21T06:04:57.843 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:58:01.246076+0000", 2024-09-21T06:04:57.843 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:04:57.843 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:04:57.843 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:04:57.843 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:04:57.844 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:04:57.844 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:04:57.844 INFO:teuthology.orchestra.run.smithi117.stdout: "b30bfff0-0f6b-4e30-8467-ff7db421eba6": { 2024-09-21T06:04:57.844 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:57:14.939082+0000", 2024-09-21T06:04:57.844 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:04:57.844 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:04:57.844 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:04:57.844 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:04:57.844 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:04:57.844 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:04:57.844 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:04:57.844 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T06:04:57.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.833+0000 7f387ba42640 1 -- 172.21.15.117:0/1750722032 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f38600b6680 msgr2=0x7f38600b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:57.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.833+0000 7f387ba42640 1 --2- 172.21.15.117:0/1750722032 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f38600b6680 0x7f38600b8b90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f386c007ed0 tx=0x7f386c0075c0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.833+0000 7f387ba42640 1 -- 172.21.15.117:0/1750722032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38741083d0 msgr2=0x7f387419aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:04:57.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.833+0000 7f387ba42640 1 --2- 172.21.15.117:0/1750722032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38741083d0 0x7f387419aac0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f3864032fd0 tx=0x7f3864004270 comp rx=0 tx=0).stop 2024-09-21T06:04:57.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.833+0000 7f387ba42640 1 -- 172.21.15.117:0/1750722032 shutdown_connections 2024-09-21T06:04:57.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.833+0000 7f387ba42640 1 --2- 172.21.15.117:0/1750722032 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f38600b6680 0x7f38600b8b90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.833+0000 7f387ba42640 1 --2- 172.21.15.117:0/1750722032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38741083d0 0x7f387419aac0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.833+0000 7f387ba42640 1 --2- 172.21.15.117:0/1750722032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3874108d20 0x7f387419fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.833+0000 7f387ba42640 1 --2- 172.21.15.117:0/1750722032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3874102ae0 0x7f387419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:04:57.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.833+0000 7f387ba42640 1 -- 172.21.15.117:0/1750722032 >> 172.21.15.117:0/1750722032 conn(0x7f3874079760 msgr2=0x7f3874104400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:04:57.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.834+0000 7f387ba42640 1 -- 172.21.15.117:0/1750722032 shutdown_connections 2024-09-21T06:04:57.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:04:57.834+0000 7f387ba42640 1 -- 172.21.15.117:0/1750722032 wait complete. 2024-09-21T06:04:58.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:57 smithi117 ceph-mon[30947]: pgmap v559: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-09-21T06:04:58.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:57 smithi192 ceph-mon[34700]: pgmap v559: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-09-21T06:04:58.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:57 smithi138 ceph-mon[34391]: pgmap v559: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-09-21T06:04:58.554 INFO:tasks.cephfs.filesystem:scrub status for tag:fb2f217d-463b-4c8f-ab19-920b04bb7306 - {'path': '/', 'tag': 'fb2f217d-463b-4c8f-ab19-920b04bb7306', 'options': 'recursive,force'} 2024-09-21T06:05:00.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:04:59 smithi192 ceph-mon[34700]: pgmap v560: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-09-21T06:05:00.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:04:59 smithi117 ceph-mon[30947]: pgmap v560: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-09-21T06:05:00.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:04:59 smithi138 ceph-mon[34391]: pgmap v560: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-09-21T06:05:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.808+0000 7f4c91638640 1 -- 172.21.15.117:0/1440192275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c8c1082e0 msgr2=0x7f4c8c1086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.808+0000 7f4c91638640 1 --2- 172.21.15.117:0/1440192275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c8c1082e0 0x7f4c8c1086f0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f4c74009900 tx=0x7f4c7402a410 comp rx=0 tx=0).stop 2024-09-21T06:05:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.808+0000 7f4c91638640 1 -- 172.21.15.117:0/1440192275 shutdown_connections 2024-09-21T06:05:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.808+0000 7f4c91638640 1 --2- 172.21.15.117:0/1440192275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c8c1082e0 0x7f4c8c1086f0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.808+0000 7f4c91638640 1 --2- 172.21.15.117:0/1440192275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c8c108c30 0x7f4c8c10f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.808+0000 7f4c91638640 1 --2- 172.21.15.117:0/1440192275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c8c1029b0 0x7f4c8c102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.808+0000 7f4c91638640 1 -- 172.21.15.117:0/1440192275 >> 172.21.15.117:0/1440192275 conn(0x7f4c8c07a740 msgr2=0x7f4c8c07ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.808+0000 7f4c91638640 1 -- 172.21.15.117:0/1440192275 shutdown_connections 2024-09-21T06:05:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.808+0000 7f4c91638640 1 -- 172.21.15.117:0/1440192275 wait complete. 2024-09-21T06:05:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.809+0000 7f4c91638640 1 Processor -- start 2024-09-21T06:05:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.809+0000 7f4c91638640 1 -- start start 2024-09-21T06:05:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.809+0000 7f4c91638640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c8c1029b0 0x7f4c8c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.809+0000 7f4c91638640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c8c1082e0 0x7f4c8c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.809+0000 7f4c91638640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c8c108c30 0x7f4c8c19fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.810+0000 7f4c91638640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c113a10 con 0x7f4c8c1029b0 2024-09-21T06:05:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.810+0000 7f4c91638640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c113890 con 0x7f4c8c108c30 2024-09-21T06:05:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.810+0000 7f4c91638640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c113b90 con 0x7f4c8c1082e0 2024-09-21T06:05:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.810+0000 7f4c8affd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c8c1029b0 0x7f4c8c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.810+0000 7f4c8affd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c8c1029b0 0x7f4c8c19a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:46694/0 (socket says 172.21.15.117:46694) 2024-09-21T06:05:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.810+0000 7f4c8affd640 1 -- 172.21.15.117:0/469370295 learned_addr learned my addr 172.21.15.117:0/469370295 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.810+0000 7f4c8b7fe640 1 --2- 172.21.15.117:0/469370295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c8c108c30 0x7f4c8c19fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.810+0000 7f4c8a7fc640 1 --2- 172.21.15.117:0/469370295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c8c1082e0 0x7f4c8c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.810+0000 7f4c8affd640 1 -- 172.21.15.117:0/469370295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c8c1082e0 msgr2=0x7f4c8c19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.810+0000 7f4c8affd640 1 --2- 172.21.15.117:0/469370295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c8c1082e0 0x7f4c8c19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.810+0000 7f4c8affd640 1 -- 172.21.15.117:0/469370295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c8c108c30 msgr2=0x7f4c8c19fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.810+0000 7f4c8affd640 1 --2- 172.21.15.117:0/469370295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c8c108c30 0x7f4c8c19fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.810+0000 7f4c8affd640 1 -- 172.21.15.117:0/469370295 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c8c1a03c0 con 0x7f4c8c1029b0 2024-09-21T06:05:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.810+0000 7f4c8a7fc640 1 --2- 172.21.15.117:0/469370295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c8c1082e0 0x7f4c8c19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:05:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.810+0000 7f4c8affd640 1 --2- 172.21.15.117:0/469370295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c8c1029b0 0x7f4c8c19a490 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00ba50 tx=0x7f4c7c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.810+0000 7f4c83fff640 1 -- 172.21.15.117:0/469370295 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c019070 con 0x7f4c8c1029b0 2024-09-21T06:05:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.810+0000 7f4c91638640 1 -- 172.21.15.117:0/469370295 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c8c1a06b0 con 0x7f4c8c1029b0 2024-09-21T06:05:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.810+0000 7f4c83fff640 1 -- 172.21.15.117:0/469370295 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c7c00cbe0 con 0x7f4c8c1029b0 2024-09-21T06:05:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.810+0000 7f4c83fff640 1 -- 172.21.15.117:0/469370295 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c0075d0 con 0x7f4c8c1029b0 2024-09-21T06:05:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.811+0000 7f4c91638640 1 -- 172.21.15.117:0/469370295 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c8c1a6e40 con 0x7f4c8c1029b0 2024-09-21T06:05:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.813+0000 7f4c98686740 1 Processor -- start 2024-09-21T06:05:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.813+0000 7f4c98686740 1 -- start start 2024-09-21T06:05:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.814+0000 7f4c98686740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615921b21b0 0x5615925579b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.814+0000 7f4c98686740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615921892f0 0x561592557ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.814+0000 7f4c98686740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615921914d0 0x561592558430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.814+0000 7f4c98686740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5615924cc6f0 con 0x5615921892f0 2024-09-21T06:05:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.814+0000 7f4c98686740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5615924bbb20 con 0x5615921b21b0 2024-09-21T06:05:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.814+0000 7f4c98686740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5615924bf930 con 0x5615921914d0 2024-09-21T06:05:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.814+0000 7f4c83fff640 1 -- 172.21.15.117:0/469370295 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f4c7c01c070 con 0x7f4c8c1029b0 2024-09-21T06:05:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.814+0000 7f4c8affd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615921892f0 0x561592557ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.814+0000 7f4c8a7fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615921914d0 0x561592558430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.814+0000 7f4c8affd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615921892f0 0x561592557ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:46706/0 (socket says 172.21.15.117:46706) 2024-09-21T06:05:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.814+0000 7f4c8affd640 1 -- 172.21.15.117:0/515605088 learned_addr learned my addr 172.21.15.117:0/515605088 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.814+0000 7f4c8b7fe640 1 --2- 172.21.15.117:0/515605088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615921b21b0 0x5615925579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.814+0000 7f4c8affd640 1 -- 172.21.15.117:0/515605088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615921914d0 msgr2=0x561592558430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.814+0000 7f4c8affd640 1 --2- 172.21.15.117:0/515605088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615921914d0 0x561592558430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.814+0000 7f4c8affd640 1 -- 172.21.15.117:0/515605088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615921b21b0 msgr2=0x5615925579b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.814+0000 7f4c8affd640 1 --2- 172.21.15.117:0/515605088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615921b21b0 0x5615925579b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.814+0000 7f4c8affd640 1 -- 172.21.15.117:0/515605088 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561592481530 con 0x5615921892f0 2024-09-21T06:05:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.815+0000 7f4c8affd640 1 --2- 172.21.15.117:0/515605088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615921892f0 0x561592557ef0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c0e42a0 tx=0x7f4c7c0eacd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f589b7c672f0fe3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.815+0000 7f4c83fff640 1 --2- 172.21.15.117:0/469370295 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4c780b6680 0x7f4c780b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.815+0000 7f4c8b7fe640 1 --2- 172.21.15.117:0/469370295 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4c780b6680 0x7f4c780b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.815+0000 7f4c817fa640 1 -- 172.21.15.117:0/515605088 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c0f6070 con 0x5615921892f0 2024-09-21T06:05:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.815+0000 7f4c817fa640 1 -- 172.21.15.117:0/515605088 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c7c07e230 con 0x5615921892f0 2024-09-21T06:05:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.815+0000 7f4c817fa640 1 -- 172.21.15.117:0/515605088 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c0ac8b0 con 0x5615921892f0 2024-09-21T06:05:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.815+0000 7f4c83fff640 1 -- 172.21.15.117:0/469370295 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f4c7c0e6b90 con 0x7f4c8c1029b0 2024-09-21T06:05:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.816+0000 7f4c98686740 1 -- 172.21.15.117:0/515605088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615921892f0 msgr2=0x561592557ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.816+0000 7f4c8b7fe640 1 --2- 172.21.15.117:0/469370295 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4c780b6680 0x7f4c780b8b90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4c84009da0 tx=0x7f4c84005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.816+0000 7f4c98686740 1 --2- 172.21.15.117:0/515605088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615921892f0 0x561592557ef0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c0e42a0 tx=0x7f4c7c0eacd0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.816+0000 7f4c98686740 1 -- 172.21.15.117:0/515605088 shutdown_connections 2024-09-21T06:05:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.816+0000 7f4c98686740 1 --2- 172.21.15.117:0/515605088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615921892f0 0x561592557ef0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.816+0000 7f4c98686740 1 --2- 172.21.15.117:0/515605088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615921914d0 0x561592558430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.816+0000 7f4c98686740 1 --2- 172.21.15.117:0/515605088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615921b21b0 0x5615925579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.816+0000 7f4c98686740 1 -- 172.21.15.117:0/515605088 >> 172.21.15.117:0/515605088 conn(0x5615921a6300 msgr2=0x5615925558a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.816+0000 7f4c98686740 1 -- 172.21.15.117:0/515605088 shutdown_connections 2024-09-21T06:05:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.816+0000 7f4c98686740 1 -- 172.21.15.117:0/515605088 wait complete. 2024-09-21T06:05:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.816+0000 7f4c98686740 1 -- start start 2024-09-21T06:05:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.816+0000 7f4c98686740 1 Processor -- start 2024-09-21T06:05:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.817+0000 7f4c98686740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615921892f0 0x561592562fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.817+0000 7f4c98686740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615921914d0 0x561592563520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.817+0000 7f4c98686740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615921b21b0 0x561592563aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.817+0000 7f4c98686740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5615923f3ab0 con 0x5615921914d0 2024-09-21T06:05:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.817+0000 7f4c98686740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561592461730 con 0x5615921892f0 2024-09-21T06:05:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.817+0000 7f4c98686740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x561592566130 con 0x5615921b21b0 2024-09-21T06:05:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.817+0000 7f4c98686740 10 client.0 _ll_register_callbacks cb 0x5615924b56b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:05:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.817+0000 7f4c8a7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615921914d0 0x561592563520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.817+0000 7f4c8a7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615921914d0 0x561592563520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:46720/0 (socket says 172.21.15.117:46720) 2024-09-21T06:05:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.817+0000 7f4c8a7fc640 1 -- 172.21.15.117:0/236237224 learned_addr learned my addr 172.21.15.117:0/236237224 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.817+0000 7f4c8affd640 1 --2- 172.21.15.117:0/236237224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615921892f0 0x561592562fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.817+0000 7f4c8b7fe640 1 --2- 172.21.15.117:0/236237224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615921b21b0 0x561592563aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.817+0000 7f4c8a7fc640 1 -- 172.21.15.117:0/236237224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615921b21b0 msgr2=0x561592563aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.817+0000 7f4c8a7fc640 1 --2- 172.21.15.117:0/236237224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615921b21b0 0x561592563aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.817+0000 7f4c8a7fc640 1 -- 172.21.15.117:0/236237224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615921892f0 msgr2=0x561592562fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.817+0000 7f4c8a7fc640 1 --2- 172.21.15.117:0/236237224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615921892f0 0x561592562fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.817+0000 7f4c8a7fc640 1 -- 172.21.15.117:0/236237224 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56159221d790 con 0x5615921914d0 2024-09-21T06:05:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.817+0000 7f4c8b7fe640 1 --2- 172.21.15.117:0/236237224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615921b21b0 0x561592563aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:05:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.817+0000 7f4c8a7fc640 1 --2- 172.21.15.117:0/236237224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615921914d0 0x561592563520 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f4c74005ab0 tx=0x7f4c740317a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47522aeba9b83a54 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.817+0000 7f4c727fc640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:05:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.818+0000 7f4c727fc640 1 -- 172.21.15.117:0/236237224 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c74035040 con 0x5615921914d0 2024-09-21T06:05:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.818+0000 7f4c727fc640 1 -- 172.21.15.117:0/236237224 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f4c74004240 con 0x5615921914d0 2024-09-21T06:05:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.818+0000 7f4c98686740 1 -- 172.21.15.117:0/236237224 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c7c07e230 con 0x5615921914d0 2024-09-21T06:05:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.818+0000 7f4c727fc640 1 -- 172.21.15.117:0/236237224 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4c74004440 con 0x5615921914d0 2024-09-21T06:05:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.818+0000 7f4c98686740 10 client.16923 fetch_fsmap learned FSMap version 29 2024-09-21T06:05:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.818+0000 7f4c98686740 1 -- 172.21.15.117:0/236237224 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561592296620 con 0x5615921914d0 2024-09-21T06:05:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.818+0000 7f4c727fc640 1 -- 172.21.15.117:0/236237224 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c7c07e230 con 0x5615921914d0 2024-09-21T06:05:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.818+0000 7f4c727fc640 1 -- 172.21.15.117:0/236237224 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4c74004b50 con 0x5615921914d0 2024-09-21T06:05:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.818+0000 7f4c98686740 10 client.16923 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:05:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.818+0000 7f4c98686740 10 client.16923 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:05:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.818+0000 7f4c98686740 20 client.16923 populate_metadata read hostname 'smithi117' 2024-09-21T06:05:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.818+0000 7f4c98686740 1 --2- 172.21.15.117:0/236237224 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x561592170d80 0x561592575100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.818+0000 7f4c98686740 4 client.16923 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:05:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.818+0000 7f4c98686740 1 -- 172.21.15.117:0/236237224 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615924ce640 con 0x561592170d80 2024-09-21T06:05:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.819+0000 7f4c8affd640 1 --2- 172.21.15.117:0/236237224 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x561592170d80 0x561592575100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.819+0000 7f4c727fc640 10 client.16923 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.819+0000 7f4c8affd640 1 --2- 172.21.15.117:0/236237224 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x561592170d80 0x561592575100 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=417ab0f4b6058df0 server_cookie=4906e302a22937ba in_seq=0 out_seq=0 2024-09-21T06:05:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.820+0000 7f4c727fc640 1 -- 172.21.15.117:0/236237224 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5615924ce640 con 0x561592170d80 2024-09-21T06:05:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.820+0000 7f4c727fc640 10 client.16923 handle_command_reply: tid=0 2024-09-21T06:05:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.820+0000 7f4c98686740 1 client.16923 shutdown 2024-09-21T06:05:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.820+0000 7f4c98686740 1 -- 172.21.15.117:0/236237224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615921914d0 msgr2=0x561592563520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:00.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.820+0000 7f4c98686740 1 --2- 172.21.15.117:0/236237224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615921914d0 0x561592563520 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f4c74005ab0 tx=0x7f4c740317a0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.820+0000 7f4c98686740 1 -- 172.21.15.117:0/236237224 shutdown_connections 2024-09-21T06:05:00.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.820+0000 7f4c98686740 1 --2- 172.21.15.117:0/236237224 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x561592170d80 0x561592575100 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.820+0000 7f4c98686740 1 --2- 172.21.15.117:0/236237224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615921914d0 0x561592563520 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.820+0000 7f4c98686740 1 --2- 172.21.15.117:0/236237224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615921b21b0 0x561592563aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.820+0000 7f4c727fc640 0 client.16923 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:00.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.820+0000 7f4c98686740 1 --2- 172.21.15.117:0/236237224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615921892f0 0x561592562fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.820+0000 7f4c98686740 1 -- 172.21.15.117:0/236237224 >> 172.21.15.117:0/236237224 conn(0x5615921a6300 msgr2=0x5615925589e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:00.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.820+0000 7f4c98686740 1 -- 172.21.15.117:0/236237224 shutdown_connections 2024-09-21T06:05:00.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.820+0000 7f4c98686740 1 -- 172.21.15.117:0/236237224 wait complete. 2024-09-21T06:05:00.831 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.820+0000 7f4c98686740 20 client.16923 trim_cache size 0 max 16384 2024-09-21T06:05:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.834+0000 7f4c98686740 1 Processor -- start 2024-09-21T06:05:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.835+0000 7f4c98686740 1 -- start start 2024-09-21T06:05:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.835+0000 7f4c98686740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615921892f0 0x561592564c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.835+0000 7f4c98686740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615921914d0 0x5615925601b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.835+0000 7f4c98686740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615921b21b0 0x5615925606f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.835+0000 7f4c98686740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c1af640 con 0x5615921b21b0 2024-09-21T06:05:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.835+0000 7f4c98686740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c1b1180 con 0x5615921914d0 2024-09-21T06:05:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.835+0000 7f4c98686740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c1afd10 con 0x5615921892f0 2024-09-21T06:05:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.835+0000 7f4c8b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615921b21b0 0x5615925606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.835+0000 7f4c8b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615921b21b0 0x5615925606f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:46726/0 (socket says 172.21.15.117:46726) 2024-09-21T06:05:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.835+0000 7f4c8b7fe640 1 -- 172.21.15.117:0/475341862 learned_addr learned my addr 172.21.15.117:0/475341862 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.835+0000 7f4c8a7fc640 1 --2- 172.21.15.117:0/475341862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615921914d0 0x5615925601b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.835+0000 7f4c8affd640 1 --2- 172.21.15.117:0/475341862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615921892f0 0x561592564c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.835+0000 7f4c8b7fe640 1 -- 172.21.15.117:0/475341862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615921892f0 msgr2=0x561592564c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.835+0000 7f4c8b7fe640 1 --2- 172.21.15.117:0/475341862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615921892f0 0x561592564c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.835+0000 7f4c8b7fe640 1 -- 172.21.15.117:0/475341862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615921914d0 msgr2=0x5615925601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.835+0000 7f4c8b7fe640 1 --2- 172.21.15.117:0/475341862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615921914d0 0x5615925601b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.835+0000 7f4c8b7fe640 1 -- 172.21.15.117:0/475341862 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615925614c0 con 0x5615921b21b0 2024-09-21T06:05:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.835+0000 7f4c8a7fc640 1 --2- 172.21.15.117:0/475341862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615921914d0 0x5615925601b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:05:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.835+0000 7f4c8affd640 1 --2- 172.21.15.117:0/475341862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615921892f0 0x561592564c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:05:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.836+0000 7f4c8b7fe640 1 --2- 172.21.15.117:0/475341862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615921b21b0 0x5615925606f0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f4c84004390 tx=0x7f4c84017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60dd0ff69f136c5b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.836+0000 7f4c80ff9640 1 -- 172.21.15.117:0/475341862 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8402d070 con 0x5615921b21b0 2024-09-21T06:05:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.836+0000 7f4c80ff9640 1 -- 172.21.15.117:0/475341862 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c84003b70 con 0x5615921b21b0 2024-09-21T06:05:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.836+0000 7f4c80ff9640 1 -- 172.21.15.117:0/475341862 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c840283f0 con 0x5615921b21b0 2024-09-21T06:05:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.837+0000 7f4c98686740 1 -- 172.21.15.117:0/475341862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615921b21b0 msgr2=0x5615925606f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.837+0000 7f4c98686740 1 --2- 172.21.15.117:0/475341862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615921b21b0 0x5615925606f0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f4c84004390 tx=0x7f4c84017000 comp rx=0 tx=0).stop 2024-09-21T06:05:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.837+0000 7f4c98686740 1 -- 172.21.15.117:0/475341862 shutdown_connections 2024-09-21T06:05:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.837+0000 7f4c98686740 1 --2- 172.21.15.117:0/475341862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615921b21b0 0x5615925606f0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.837+0000 7f4c98686740 1 --2- 172.21.15.117:0/475341862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615921914d0 0x5615925601b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.837+0000 7f4c98686740 1 --2- 172.21.15.117:0/475341862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615921892f0 0x561592564c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.837+0000 7f4c98686740 1 -- 172.21.15.117:0/475341862 >> 172.21.15.117:0/475341862 conn(0x5615921a6300 msgr2=0x561592564500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.837+0000 7f4c98686740 1 -- 172.21.15.117:0/475341862 shutdown_connections 2024-09-21T06:05:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.837+0000 7f4c98686740 1 -- 172.21.15.117:0/475341862 wait complete. 2024-09-21T06:05:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.837+0000 7f4c98686740 1 -- start start 2024-09-21T06:05:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.837+0000 7f4c98686740 1 Processor -- start 2024-09-21T06:05:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.837+0000 7f4c98686740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615921892f0 0x56159256bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.837+0000 7f4c98686740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615921914d0 0x5615925692e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.837+0000 7f4c98686740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615921b21b0 0x5615925699f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.837+0000 7f4c98686740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c1b0ab0 con 0x5615921914d0 2024-09-21T06:05:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.837+0000 7f4c98686740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c1b03e0 con 0x5615921b21b0 2024-09-21T06:05:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.837+0000 7f4c98686740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c1ad640 con 0x5615921892f0 2024-09-21T06:05:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.837+0000 7f4c98686740 10 client.0 _ll_register_callbacks cb 0x7f4c74009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:05:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.838+0000 7f4c8a7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615921914d0 0x5615925692e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.838+0000 7f4c8a7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615921914d0 0x5615925692e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:46736/0 (socket says 172.21.15.117:46736) 2024-09-21T06:05:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.838+0000 7f4c8a7fc640 1 -- 172.21.15.117:0/2219836026 learned_addr learned my addr 172.21.15.117:0/2219836026 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.838+0000 7f4c8b7fe640 1 --2- 172.21.15.117:0/2219836026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615921b21b0 0x5615925699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:00.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.838+0000 7f4c8affd640 1 --2- 172.21.15.117:0/2219836026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615921892f0 0x56159256bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.838+0000 7f4c8a7fc640 1 -- 172.21.15.117:0/2219836026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615921892f0 msgr2=0x56159256bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.838+0000 7f4c8a7fc640 1 --2- 172.21.15.117:0/2219836026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615921892f0 0x56159256bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.838+0000 7f4c8a7fc640 1 -- 172.21.15.117:0/2219836026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615921b21b0 msgr2=0x5615925699f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.838+0000 7f4c8a7fc640 1 --2- 172.21.15.117:0/2219836026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615921b21b0 0x5615925699f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.838+0000 7f4c8a7fc640 1 -- 172.21.15.117:0/2219836026 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615925e2ed0 con 0x5615921914d0 2024-09-21T06:05:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.838+0000 7f4c8a7fc640 1 --2- 172.21.15.117:0/2219836026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615921914d0 0x5615925692e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4c74006010 tx=0x7f4c74000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbee93a9e88a81b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.838+0000 7f4c727fc640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:05:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.838+0000 7f4c727fc640 1 -- 172.21.15.117:0/2219836026 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c74053030 con 0x5615921914d0 2024-09-21T06:05:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.838+0000 7f4c727fc640 1 -- 172.21.15.117:0/2219836026 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f4c740046c0 con 0x5615921914d0 2024-09-21T06:05:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.838+0000 7f4c98686740 1 -- 172.21.15.117:0/2219836026 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c84003b70 con 0x5615921914d0 2024-09-21T06:05:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.838+0000 7f4c727fc640 1 -- 172.21.15.117:0/2219836026 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4c74035070 con 0x5615921914d0 2024-09-21T06:05:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.838+0000 7f4c98686740 10 client.16929 fetch_fsmap learned FSMap version 29 2024-09-21T06:05:00.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.838+0000 7f4c98686740 1 -- 172.21.15.117:0/2219836026 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5615925e3f40 con 0x5615921914d0 2024-09-21T06:05:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.838+0000 7f4c727fc640 1 -- 172.21.15.117:0/2219836026 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c84003b70 con 0x5615921914d0 2024-09-21T06:05:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.839+0000 7f4c727fc640 1 -- 172.21.15.117:0/2219836026 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4c74003bb0 con 0x5615921914d0 2024-09-21T06:05:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.839+0000 7f4c98686740 10 client.16929 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:05:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.839+0000 7f4c98686740 10 client.16929 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:05:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.839+0000 7f4c98686740 20 client.16929 populate_metadata read hostname 'smithi117' 2024-09-21T06:05:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.839+0000 7f4c98686740 1 --2- 172.21.15.117:0/2219836026 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5615921461c0 0x5615925e6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.839+0000 7f4c98686740 4 client.16929 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:05:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.839+0000 7f4c98686740 1 -- 172.21.15.117:0/2219836026 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5615924ca0f0 con 0x5615921461c0 2024-09-21T06:05:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.839+0000 7f4c8affd640 1 --2- 172.21.15.117:0/2219836026 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5615921461c0 0x5615925e6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.839+0000 7f4c727fc640 10 client.16929 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.839+0000 7f4c8affd640 1 --2- 172.21.15.117:0/2219836026 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5615921461c0 0x5615925e6780 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5195197de12632b4 server_cookie=b6c2e98491d91675 in_seq=0 out_seq=0 2024-09-21T06:05:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.848+0000 7f4c727fc640 1 -- 172.21.15.117:0/2219836026 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5615924ca0f0 con 0x5615921461c0 2024-09-21T06:05:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.848+0000 7f4c727fc640 10 client.16929 handle_command_reply: tid=0 2024-09-21T06:05:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.848+0000 7f4c98686740 1 client.16929 shutdown 2024-09-21T06:05:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.848+0000 7f4c98686740 1 -- 172.21.15.117:0/2219836026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615921914d0 msgr2=0x5615925692e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.848+0000 7f4c98686740 1 --2- 172.21.15.117:0/2219836026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615921914d0 0x5615925692e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4c74006010 tx=0x7f4c74000bc0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.848+0000 7f4c98686740 1 -- 172.21.15.117:0/2219836026 shutdown_connections 2024-09-21T06:05:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.848+0000 7f4c98686740 1 --2- 172.21.15.117:0/2219836026 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5615921461c0 0x5615925e6780 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.848+0000 7f4c98686740 1 --2- 172.21.15.117:0/2219836026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615921914d0 0x5615925692e0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.848+0000 7f4c727fc640 0 client.16929 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.848+0000 7f4c98686740 1 --2- 172.21.15.117:0/2219836026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5615921b21b0 0x5615925699f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.848+0000 7f4c98686740 1 --2- 172.21.15.117:0/2219836026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5615921892f0 0x56159256bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.848+0000 7f4c98686740 1 -- 172.21.15.117:0/2219836026 >> 172.21.15.117:0/2219836026 conn(0x5615921a6300 msgr2=0x56159256e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.848+0000 7f4c98686740 1 -- 172.21.15.117:0/2219836026 shutdown_connections 2024-09-21T06:05:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.848+0000 7f4c98686740 1 -- 172.21.15.117:0/2219836026 wait complete. 2024-09-21T06:05:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.849+0000 7f4c98686740 20 client.16929 trim_cache size 0 max 16384 2024-09-21T06:05:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T06:05:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T06:05:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T06:05:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T06:05:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.849+0000 7f4c91638640 1 -- 172.21.15.117:0/469370295 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4c780b6680 msgr2=0x7f4c780b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.849+0000 7f4c91638640 1 --2- 172.21.15.117:0/469370295 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4c780b6680 0x7f4c780b8b90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4c84009da0 tx=0x7f4c84005f90 comp rx=0 tx=0).stop 2024-09-21T06:05:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.849+0000 7f4c91638640 1 -- 172.21.15.117:0/469370295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c8c1029b0 msgr2=0x7f4c8c19a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.849+0000 7f4c91638640 1 --2- 172.21.15.117:0/469370295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c8c1029b0 0x7f4c8c19a490 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00ba50 tx=0x7f4c7c00bf10 comp rx=0 tx=0).stop 2024-09-21T06:05:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.849+0000 7f4c91638640 1 -- 172.21.15.117:0/469370295 shutdown_connections 2024-09-21T06:05:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.849+0000 7f4c91638640 1 --2- 172.21.15.117:0/469370295 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4c780b6680 0x7f4c780b8b90 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.849+0000 7f4c91638640 1 --2- 172.21.15.117:0/469370295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c8c108c30 0x7f4c8c19fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.849+0000 7f4c91638640 1 --2- 172.21.15.117:0/469370295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c8c1082e0 0x7f4c8c19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.849+0000 7f4c91638640 1 --2- 172.21.15.117:0/469370295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c8c1029b0 0x7f4c8c19a490 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.849+0000 7f4c91638640 1 -- 172.21.15.117:0/469370295 >> 172.21.15.117:0/469370295 conn(0x7f4c8c07a740 msgr2=0x7f4c8c1063d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.849+0000 7f4c91638640 1 -- 172.21.15.117:0/469370295 shutdown_connections 2024-09-21T06:05:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:00.849+0000 7f4c91638640 1 -- 172.21.15.117:0/469370295 wait complete. 2024-09-21T06:05:01.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:00 smithi192 ceph-mon[34700]: scrub summary: idle 2024-09-21T06:05:01.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:00 smithi117 ceph-mon[30947]: scrub summary: idle 2024-09-21T06:05:01.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:00 smithi138 ceph-mon[34391]: scrub summary: idle 2024-09-21T06:05:02.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:01 smithi192 ceph-mon[34700]: pgmap v561: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.0 KiB/s wr, 13 op/s 2024-09-21T06:05:02.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:01 smithi117 ceph-mon[30947]: pgmap v561: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.0 KiB/s wr, 13 op/s 2024-09-21T06:05:02.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:01 smithi138 ceph-mon[34391]: pgmap v561: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.0 KiB/s wr, 13 op/s 2024-09-21T06:05:04.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:03 smithi192 ceph-mon[34700]: pgmap v562: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.3 KiB/s wr, 12 op/s 2024-09-21T06:05:04.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:03 smithi117 ceph-mon[30947]: pgmap v562: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.3 KiB/s wr, 12 op/s 2024-09-21T06:05:04.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:03 smithi138 ceph-mon[34391]: pgmap v562: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.3 KiB/s wr, 12 op/s 2024-09-21T06:05:06.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:05 smithi192 ceph-mon[34700]: pgmap v563: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 29 op/s 2024-09-21T06:05:06.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:05 smithi117 ceph-mon[30947]: pgmap v563: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 29 op/s 2024-09-21T06:05:06.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:05 smithi138 ceph-mon[34391]: pgmap v563: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 29 op/s 2024-09-21T06:05:08.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:07 smithi192 ceph-mon[34700]: pgmap v564: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 29 KiB/s wr, 30 op/s 2024-09-21T06:05:08.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:07 smithi117 ceph-mon[30947]: pgmap v564: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 29 KiB/s wr, 30 op/s 2024-09-21T06:05:08.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:07 smithi138 ceph-mon[34391]: pgmap v564: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 29 KiB/s wr, 30 op/s 2024-09-21T06:05:10.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:09 smithi192 ceph-mon[34700]: pgmap v565: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 29 KiB/s wr, 31 op/s 2024-09-21T06:05:10.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:09 smithi117 ceph-mon[30947]: pgmap v565: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 29 KiB/s wr, 31 op/s 2024-09-21T06:05:10.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:09 smithi138 ceph-mon[34391]: pgmap v565: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 29 KiB/s wr, 31 op/s 2024-09-21T06:05:12.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:11 smithi192 ceph-mon[34700]: pgmap v566: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-09-21T06:05:12.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:11 smithi117 ceph-mon[30947]: pgmap v566: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-09-21T06:05:12.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:11 smithi138 ceph-mon[34391]: pgmap v566: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-09-21T06:05:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.956+0000 7f7b08fcc640 1 -- 172.21.15.117:0/4113905078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b04104930 msgr2=0x7f7b04104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.956+0000 7f7b08fcc640 1 --2- 172.21.15.117:0/4113905078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b04104930 0x7f7b04104d40 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009ca0 tx=0x7f7af402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:05:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.956+0000 7f7b08fcc640 1 -- 172.21.15.117:0/4113905078 shutdown_connections 2024-09-21T06:05:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.956+0000 7f7b08fcc640 1 --2- 172.21.15.117:0/4113905078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b0410dd90 0x7f7b04110280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.956+0000 7f7b08fcc640 1 --2- 172.21.15.117:0/4113905078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b04105280 0x7f7b0410d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.956+0000 7f7b08fcc640 1 --2- 172.21.15.117:0/4113905078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b04104930 0x7f7b04104d40 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.956+0000 7f7b08fcc640 1 -- 172.21.15.117:0/4113905078 >> 172.21.15.117:0/4113905078 conn(0x7f7b040fe3c0 msgr2=0x7f7b04100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.956+0000 7f7b08fcc640 1 -- 172.21.15.117:0/4113905078 shutdown_connections 2024-09-21T06:05:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.956+0000 7f7b08fcc640 1 -- 172.21.15.117:0/4113905078 wait complete. 2024-09-21T06:05:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.957+0000 7f7b08fcc640 1 Processor -- start 2024-09-21T06:05:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.957+0000 7f7b08fcc640 1 -- start start 2024-09-21T06:05:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.957+0000 7f7b08fcc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b04104930 0x7f7b0419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.957+0000 7f7b08fcc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b04105280 0x7f7b0419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.957+0000 7f7b08fcc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b0410dd90 0x7f7b0419fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.957+0000 7f7b08fcc640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041139f0 con 0x7f7b0410dd90 2024-09-21T06:05:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.957+0000 7f7b08fcc640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7b04113870 con 0x7f7b04104930 2024-09-21T06:05:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.957+0000 7f7b08fcc640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7b04113b70 con 0x7f7b04105280 2024-09-21T06:05:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.957+0000 7f7b02d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b0410dd90 0x7f7b0419fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.957+0000 7f7b02d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b0410dd90 0x7f7b0419fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44790/0 (socket says 172.21.15.117:44790) 2024-09-21T06:05:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.957+0000 7f7b02d9d640 1 -- 172.21.15.117:0/1401739982 learned_addr learned my addr 172.21.15.117:0/1401739982 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.957+0000 7f7b0259c640 1 --2- 172.21.15.117:0/1401739982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b04104930 0x7f7b0419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.958+0000 7f7b01d9b640 1 --2- 172.21.15.117:0/1401739982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b04105280 0x7f7b0419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.958+0000 7f7b02d9d640 1 -- 172.21.15.117:0/1401739982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b04105280 msgr2=0x7f7b0419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.958+0000 7f7b02d9d640 1 --2- 172.21.15.117:0/1401739982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b04105280 0x7f7b0419a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.958+0000 7f7b02d9d640 1 -- 172.21.15.117:0/1401739982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b04104930 msgr2=0x7f7b0419a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.958+0000 7f7b02d9d640 1 --2- 172.21.15.117:0/1401739982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b04104930 0x7f7b0419a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.958+0000 7f7b02d9d640 1 -- 172.21.15.117:0/1401739982 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b041a02b0 con 0x7f7b0410dd90 2024-09-21T06:05:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.958+0000 7f7b02d9d640 1 --2- 172.21.15.117:0/1401739982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b0410dd90 0x7f7b0419fc40 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00d750 tx=0x7f7afc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.958+0000 7f7af37fe640 1 -- 172.21.15.117:0/1401739982 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc017070 con 0x7f7b0410dd90 2024-09-21T06:05:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.958+0000 7f7af37fe640 1 -- 172.21.15.117:0/1401739982 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7afc0054d0 con 0x7f7b0410dd90 2024-09-21T06:05:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.958+0000 7f7b08fcc640 1 -- 172.21.15.117:0/1401739982 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b041a05a0 con 0x7f7b0410dd90 2024-09-21T06:05:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.958+0000 7f7af37fe640 1 -- 172.21.15.117:0/1401739982 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc004c60 con 0x7f7b0410dd90 2024-09-21T06:05:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.958+0000 7f7b08fcc640 1 -- 172.21.15.117:0/1401739982 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b041a6dc0 con 0x7f7b0410dd90 2024-09-21T06:05:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.961+0000 7f7b10014740 1 Processor -- start 2024-09-21T06:05:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.962+0000 7f7af37fe640 1 -- 172.21.15.117:0/1401739982 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f7afc010460 con 0x7f7b0410dd90 2024-09-21T06:05:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.962+0000 7f7b10014740 1 -- start start 2024-09-21T06:05:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.962+0000 7f7b10014740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5591ee12a1b0 0x5591ee4cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.962+0000 7f7b10014740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591ee1012f0 0x5591ee4cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.962+0000 7f7b10014740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591ee1094d0 0x5591ee4d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.962+0000 7f7b10014740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5591ee4446f0 con 0x5591ee1094d0 2024-09-21T06:05:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.962+0000 7f7b10014740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5591ee433b20 con 0x5591ee1012f0 2024-09-21T06:05:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.962+0000 7f7b10014740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5591ee437930 con 0x5591ee12a1b0 2024-09-21T06:05:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.962+0000 7f7b01d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591ee1094d0 0x5591ee4d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.962+0000 7f7b01d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591ee1094d0 0x5591ee4d0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44800/0 (socket says 172.21.15.117:44800) 2024-09-21T06:05:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.962+0000 7f7b01d9b640 1 -- 172.21.15.117:0/4247738433 learned_addr learned my addr 172.21.15.117:0/4247738433 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.962+0000 7f7b01d9b640 1 --2- 172.21.15.117:0/4247738433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5591ee12a1b0 0x5591ee4cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.962+0000 7f7b0259c640 1 --2- 172.21.15.117:0/4247738433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591ee1012f0 0x5591ee4cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.963+0000 7f7af37fe640 1 --2- 172.21.15.117:0/1401739982 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7aec0b6680 0x7f7aec0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.963+0000 7f7b01d9b640 1 -- 172.21.15.117:0/4247738433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5591ee12a1b0 msgr2=0x5591ee4cf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.963+0000 7f7b01d9b640 1 --2- 172.21.15.117:0/4247738433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5591ee12a1b0 0x5591ee4cf9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.963+0000 7f7b01d9b640 1 -- 172.21.15.117:0/4247738433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591ee1012f0 msgr2=0x5591ee4cfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:13.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.963+0000 7f7b01d9b640 1 --2- 172.21.15.117:0/4247738433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591ee1012f0 0x5591ee4cfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.963+0000 7f7b01d9b640 1 -- 172.21.15.117:0/4247738433 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591ee3f9530 con 0x5591ee1094d0 2024-09-21T06:05:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.963+0000 7f7b0259c640 1 --2- 172.21.15.117:0/4247738433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591ee1012f0 0x5591ee4cfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:05:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.963+0000 7f7b02d9d640 1 --2- 172.21.15.117:0/1401739982 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7aec0b6680 0x7f7aec0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.963+0000 7f7af37fe640 1 -- 172.21.15.117:0/1401739982 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f7afc0e70c0 con 0x7f7b0410dd90 2024-09-21T06:05:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.963+0000 7f7b01d9b640 1 --2- 172.21.15.117:0/4247738433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591ee1094d0 0x5591ee4d0430 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f7af8005fd0 tx=0x7f7af800b440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7174308740702e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.963+0000 7f7af0ff9640 1 -- 172.21.15.117:0/4247738433 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af800e920 con 0x5591ee1094d0 2024-09-21T06:05:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.963+0000 7f7af0ff9640 1 -- 172.21.15.117:0/4247738433 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7af8010430 con 0x5591ee1094d0 2024-09-21T06:05:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.963+0000 7f7af0ff9640 1 -- 172.21.15.117:0/4247738433 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af800f620 con 0x5591ee1094d0 2024-09-21T06:05:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.964+0000 7f7b10014740 1 -- 172.21.15.117:0/4247738433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591ee1094d0 msgr2=0x5591ee4d0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.964+0000 7f7b02d9d640 1 --2- 172.21.15.117:0/1401739982 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7aec0b6680 0x7f7aec0b8b90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7afc0a9530 tx=0x7f7afc094040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.964+0000 7f7b10014740 1 --2- 172.21.15.117:0/4247738433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591ee1094d0 0x5591ee4d0430 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f7af8005fd0 tx=0x7f7af800b440 comp rx=0 tx=0).stop 2024-09-21T06:05:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.964+0000 7f7b10014740 1 -- 172.21.15.117:0/4247738433 shutdown_connections 2024-09-21T06:05:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.964+0000 7f7b10014740 1 --2- 172.21.15.117:0/4247738433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591ee1094d0 0x5591ee4d0430 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:13.972 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.964+0000 7f7b10014740 1 --2- 172.21.15.117:0/4247738433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591ee1012f0 0x5591ee4cfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.964+0000 7f7b10014740 1 --2- 172.21.15.117:0/4247738433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5591ee12a1b0 0x5591ee4cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.964+0000 7f7b10014740 1 -- 172.21.15.117:0/4247738433 >> 172.21.15.117:0/4247738433 conn(0x5591ee11e300 msgr2=0x5591ee4cd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.964+0000 7f7b10014740 1 -- 172.21.15.117:0/4247738433 shutdown_connections 2024-09-21T06:05:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.964+0000 7f7b10014740 1 -- 172.21.15.117:0/4247738433 wait complete. 2024-09-21T06:05:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.965+0000 7f7b10014740 1 -- start start 2024-09-21T06:05:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.965+0000 7f7b10014740 1 Processor -- start 2024-09-21T06:05:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.965+0000 7f7b10014740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591ee1012f0 0x5591ee4db050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.965+0000 7f7b10014740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5591ee1094d0 0x5591ee4db590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.965+0000 7f7b10014740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591ee12a1b0 0x5591ee4e7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.965+0000 7f7b10014740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5591ee36bab0 con 0x5591ee1012f0 2024-09-21T06:05:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.965+0000 7f7b10014740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5591ee3d9730 con 0x5591ee12a1b0 2024-09-21T06:05:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.965+0000 7f7b10014740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5591ee4de130 con 0x5591ee1094d0 2024-09-21T06:05:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.965+0000 7f7b10014740 10 client.0 _ll_register_callbacks cb 0x5591ee42d6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:05:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.965+0000 7f7b01d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591ee1012f0 0x5591ee4db050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.965+0000 7f7b01d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591ee1012f0 0x5591ee4db050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44814/0 (socket says 172.21.15.117:44814) 2024-09-21T06:05:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.965+0000 7f7b01d9b640 1 -- 172.21.15.117:0/2051989394 learned_addr learned my addr 172.21.15.117:0/2051989394 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.965+0000 7f7b0259c640 1 --2- 172.21.15.117:0/2051989394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5591ee1094d0 0x5591ee4db590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.966+0000 7f7b01d9b640 1 -- 172.21.15.117:0/2051989394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5591ee1094d0 msgr2=0x5591ee4db590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.966+0000 7f7b01d9b640 1 --2- 172.21.15.117:0/2051989394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5591ee1094d0 0x5591ee4db590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.966+0000 7f7b01d9b640 1 -- 172.21.15.117:0/2051989394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591ee12a1b0 msgr2=0x5591ee4e7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.966+0000 7f7b01d9b640 1 --2- 172.21.15.117:0/2051989394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591ee12a1b0 0x5591ee4e7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.966+0000 7f7b01d9b640 1 -- 172.21.15.117:0/2051989394 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591ee195790 con 0x5591ee1012f0 2024-09-21T06:05:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.966+0000 7f7b01d9b640 1 --2- 172.21.15.117:0/2051989394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591ee1012f0 0x5591ee4db050 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f7af8010e20 tx=0x7f7af800bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4d251ca84e03d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.966+0000 7f7ae9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:05:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.966+0000 7f7ae9ffb640 1 -- 172.21.15.117:0/2051989394 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af8004030 con 0x5591ee1012f0 2024-09-21T06:05:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.966+0000 7f7ae9ffb640 1 -- 172.21.15.117:0/2051989394 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f7af80041d0 con 0x5591ee1012f0 2024-09-21T06:05:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.966+0000 7f7b10014740 1 -- 172.21.15.117:0/2051989394 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7af8010430 con 0x5591ee1012f0 2024-09-21T06:05:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.966+0000 7f7ae9ffb640 1 -- 172.21.15.117:0/2051989394 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7af8004410 con 0x5591ee1012f0 2024-09-21T06:05:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.966+0000 7f7b10014740 10 client.16941 fetch_fsmap learned FSMap version 29 2024-09-21T06:05:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.966+0000 7f7b10014740 1 -- 172.21.15.117:0/2051989394 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5591ee20e620 con 0x5591ee1012f0 2024-09-21T06:05:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.966+0000 7f7ae9ffb640 1 -- 172.21.15.117:0/2051989394 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7af8010430 con 0x5591ee1012f0 2024-09-21T06:05:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.967+0000 7f7ae9ffb640 1 -- 172.21.15.117:0/2051989394 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7af80035f0 con 0x5591ee1012f0 2024-09-21T06:05:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.967+0000 7f7b10014740 10 client.16941 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:05:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.967+0000 7f7b10014740 10 client.16941 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:05:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.967+0000 7f7b10014740 20 client.16941 populate_metadata read hostname 'smithi117' 2024-09-21T06:05:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.967+0000 7f7b10014740 1 --2- 172.21.15.117:0/2051989394 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5591ee0e8d80 0x5591ee4ed0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.967+0000 7f7b10014740 4 client.16941 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:05:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.967+0000 7f7b10014740 1 -- 172.21.15.117:0/2051989394 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591ee446640 con 0x5591ee0e8d80 2024-09-21T06:05:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.967+0000 7f7b0259c640 1 --2- 172.21.15.117:0/2051989394 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5591ee0e8d80 0x5591ee4ed0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.967+0000 7f7ae9ffb640 10 client.16941 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.967+0000 7f7b0259c640 1 --2- 172.21.15.117:0/2051989394 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5591ee0e8d80 0x5591ee4ed0e0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11ec0dce794cfd3a server_cookie=4acb9b4f4f55a527 in_seq=0 out_seq=0 2024-09-21T06:05:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.968+0000 7f7ae9ffb640 1 -- 172.21.15.117:0/2051989394 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5591ee446640 con 0x5591ee0e8d80 2024-09-21T06:05:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.968+0000 7f7ae9ffb640 10 client.16941 handle_command_reply: tid=0 2024-09-21T06:05:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.968+0000 7f7b10014740 1 client.16941 shutdown 2024-09-21T06:05:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.968+0000 7f7b10014740 1 -- 172.21.15.117:0/2051989394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591ee1012f0 msgr2=0x5591ee4db050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.968+0000 7f7b10014740 1 --2- 172.21.15.117:0/2051989394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591ee1012f0 0x5591ee4db050 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f7af8010e20 tx=0x7f7af800bfc0 comp rx=0 tx=0).stop 2024-09-21T06:05:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.968+0000 7f7b10014740 1 -- 172.21.15.117:0/2051989394 shutdown_connections 2024-09-21T06:05:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.968+0000 7f7b10014740 1 --2- 172.21.15.117:0/2051989394 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5591ee0e8d80 0x5591ee4ed0e0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.968+0000 7f7b10014740 1 --2- 172.21.15.117:0/2051989394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591ee12a1b0 0x5591ee4e7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.968+0000 7f7b10014740 1 --2- 172.21.15.117:0/2051989394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5591ee1094d0 0x5591ee4db590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.968+0000 7f7ae9ffb640 0 client.16941 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.968+0000 7f7b10014740 1 --2- 172.21.15.117:0/2051989394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591ee1012f0 0x5591ee4db050 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.968+0000 7f7b10014740 1 -- 172.21.15.117:0/2051989394 >> 172.21.15.117:0/2051989394 conn(0x5591ee11e300 msgr2=0x5591ee4d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.968+0000 7f7b10014740 1 -- 172.21.15.117:0/2051989394 shutdown_connections 2024-09-21T06:05:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.968+0000 7f7b10014740 1 -- 172.21.15.117:0/2051989394 wait complete. 2024-09-21T06:05:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.969+0000 7f7b10014740 20 client.16941 trim_cache size 0 max 16384 2024-09-21T06:05:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.983+0000 7f7b10014740 1 Processor -- start 2024-09-21T06:05:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.983+0000 7f7b10014740 1 -- start start 2024-09-21T06:05:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.983+0000 7f7b10014740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591ee1012f0 0x5591ee4cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.983+0000 7f7b10014740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5591ee1094d0 0x5591ee4cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.983+0000 7f7b10014740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591ee12a1b0 0x5591ee4f26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.983+0000 7f7b10014740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041af4b0 con 0x5591ee1012f0 2024-09-21T06:05:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.983+0000 7f7b10014740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041b0ff0 con 0x5591ee12a1b0 2024-09-21T06:05:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.983+0000 7f7b10014740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041afb80 con 0x5591ee1094d0 2024-09-21T06:05:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.983+0000 7f7b01d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591ee1012f0 0x5591ee4cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.983+0000 7f7b01d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591ee1012f0 0x5591ee4cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44816/0 (socket says 172.21.15.117:44816) 2024-09-21T06:05:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.983+0000 7f7b01d9b640 1 -- 172.21.15.117:0/1322176630 learned_addr learned my addr 172.21.15.117:0/1322176630 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.984+0000 7f7b0259c640 1 --2- 172.21.15.117:0/1322176630 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5591ee1094d0 0x5591ee4cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.984+0000 7f7b01d9b640 1 --2- 172.21.15.117:0/1322176630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591ee12a1b0 0x5591ee4f26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.984+0000 7f7b01d9b640 1 -- 172.21.15.117:0/1322176630 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5591ee1094d0 msgr2=0x5591ee4cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.984+0000 7f7b01d9b640 1 --2- 172.21.15.117:0/1322176630 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5591ee1094d0 0x5591ee4cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.984+0000 7f7b01d9b640 1 -- 172.21.15.117:0/1322176630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591ee12a1b0 msgr2=0x5591ee4f26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.984+0000 7f7b01d9b640 1 --2- 172.21.15.117:0/1322176630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591ee12a1b0 0x5591ee4f26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.984+0000 7f7b01d9b640 1 -- 172.21.15.117:0/1322176630 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591ee4d1ee0 con 0x5591ee1012f0 2024-09-21T06:05:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.984+0000 7f7b01d9b640 1 --2- 172.21.15.117:0/1322176630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591ee1012f0 0x5591ee4cceb0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f7af800b480 tx=0x7f7af800bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94eca2d4a3bc333a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.984+0000 7f7aebfff640 1 -- 172.21.15.117:0/1322176630 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af8035040 con 0x5591ee1012f0 2024-09-21T06:05:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.984+0000 7f7aebfff640 1 -- 172.21.15.117:0/1322176630 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7af80035f0 con 0x5591ee1012f0 2024-09-21T06:05:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.984+0000 7f7aebfff640 1 -- 172.21.15.117:0/1322176630 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af8005040 con 0x5591ee1012f0 2024-09-21T06:05:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.985+0000 7f7b10014740 1 -- 172.21.15.117:0/1322176630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591ee1012f0 msgr2=0x5591ee4cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.985+0000 7f7b10014740 1 --2- 172.21.15.117:0/1322176630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591ee1012f0 0x5591ee4cceb0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f7af800b480 tx=0x7f7af800bb80 comp rx=0 tx=0).stop 2024-09-21T06:05:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.985+0000 7f7b10014740 1 -- 172.21.15.117:0/1322176630 shutdown_connections 2024-09-21T06:05:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.985+0000 7f7b10014740 1 --2- 172.21.15.117:0/1322176630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591ee12a1b0 0x5591ee4f26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.985+0000 7f7b10014740 1 --2- 172.21.15.117:0/1322176630 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5591ee1094d0 0x5591ee4cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.985+0000 7f7b10014740 1 --2- 172.21.15.117:0/1322176630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591ee1012f0 0x5591ee4cceb0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.985+0000 7f7b10014740 1 -- 172.21.15.117:0/1322176630 >> 172.21.15.117:0/1322176630 conn(0x5591ee11e300 msgr2=0x5591ee4d1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.985+0000 7f7b10014740 1 -- 172.21.15.117:0/1322176630 shutdown_connections 2024-09-21T06:05:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.985+0000 7f7b10014740 1 -- 172.21.15.117:0/1322176630 wait complete. 2024-09-21T06:05:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.985+0000 7f7b10014740 1 -- start start 2024-09-21T06:05:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.985+0000 7f7b10014740 1 Processor -- start 2024-09-21T06:05:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.986+0000 7f7b10014740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591ee1012f0 0x5591ee4e2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.986+0000 7f7b10014740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591ee1094d0 0x5591ee4e2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.986+0000 7f7b10014740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5591ee12a1b0 0x5591ee5558f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.986+0000 7f7b10014740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041b0920 con 0x5591ee1094d0 2024-09-21T06:05:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.986+0000 7f7b10014740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041b0250 con 0x5591ee1012f0 2024-09-21T06:05:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.986+0000 7f7b10014740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041ad4b0 con 0x5591ee12a1b0 2024-09-21T06:05:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.986+0000 7f7b10014740 10 client.0 _ll_register_callbacks cb 0x7f7af800bea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:05:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.986+0000 7f7b0259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591ee1094d0 0x5591ee4e2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.986+0000 7f7b0259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591ee1094d0 0x5591ee4e2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44828/0 (socket says 172.21.15.117:44828) 2024-09-21T06:05:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.986+0000 7f7b0259c640 1 -- 172.21.15.117:0/2144654542 learned_addr learned my addr 172.21.15.117:0/2144654542 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.986+0000 7f7b01d9b640 1 --2- 172.21.15.117:0/2144654542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5591ee12a1b0 0x5591ee5558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.986+0000 7f7b01d9b640 1 --2- 172.21.15.117:0/2144654542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591ee1012f0 0x5591ee4e2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.986+0000 7f7b0259c640 1 -- 172.21.15.117:0/2144654542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5591ee12a1b0 msgr2=0x5591ee5558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.986+0000 7f7b0259c640 1 --2- 172.21.15.117:0/2144654542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5591ee12a1b0 0x5591ee5558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.986+0000 7f7b0259c640 1 -- 172.21.15.117:0/2144654542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591ee1012f0 msgr2=0x5591ee4e2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.986+0000 7f7b0259c640 1 --2- 172.21.15.117:0/2144654542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591ee1012f0 0x5591ee4e2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.986+0000 7f7b0259c640 1 -- 172.21.15.117:0/2144654542 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591ee555e30 con 0x5591ee1094d0 2024-09-21T06:05:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.986+0000 7f7b01d9b640 1 --2- 172.21.15.117:0/2144654542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591ee1012f0 0x5591ee4e2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:05:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.986+0000 7f7b0259c640 1 --2- 172.21.15.117:0/2144654542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591ee1094d0 0x5591ee4e2d00 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009ca0 tx=0x7f7af4031680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38fa4433093e8d7c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.986+0000 7f7ae9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:05:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.986+0000 7f7ae9ffb640 1 -- 172.21.15.117:0/2144654542 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af4003b70 con 0x5591ee1094d0 2024-09-21T06:05:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.986+0000 7f7ae9ffb640 1 -- 172.21.15.117:0/2144654542 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f7af4003d10 con 0x5591ee1094d0 2024-09-21T06:05:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.986+0000 7f7b10014740 1 -- 172.21.15.117:0/2144654542 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7af80035f0 con 0x5591ee1094d0 2024-09-21T06:05:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.987+0000 7f7ae9ffb640 1 -- 172.21.15.117:0/2144654542 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7af4004040 con 0x5591ee1094d0 2024-09-21T06:05:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.987+0000 7f7b10014740 10 client.16947 fetch_fsmap learned FSMap version 29 2024-09-21T06:05:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.987+0000 7f7b10014740 1 -- 172.21.15.117:0/2144654542 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5591ee556ff0 con 0x5591ee1094d0 2024-09-21T06:05:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.987+0000 7f7ae9ffb640 1 -- 172.21.15.117:0/2144654542 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7af80035f0 con 0x5591ee1094d0 2024-09-21T06:05:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.987+0000 7f7ae9ffb640 1 -- 172.21.15.117:0/2144654542 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7af4057030 con 0x5591ee1094d0 2024-09-21T06:05:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.987+0000 7f7b10014740 10 client.16947 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:05:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.987+0000 7f7b10014740 10 client.16947 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:05:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.987+0000 7f7b10014740 20 client.16947 populate_metadata read hostname 'smithi117' 2024-09-21T06:05:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.987+0000 7f7b10014740 1 --2- 172.21.15.117:0/2144654542 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5591ee0be1c0 0x5591ee559860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.987+0000 7f7b10014740 4 client.16947 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:05:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.987+0000 7f7b10014740 1 -- 172.21.15.117:0/2144654542 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5591ee4420f0 con 0x5591ee0be1c0 2024-09-21T06:05:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.987+0000 7f7b01d9b640 1 --2- 172.21.15.117:0/2144654542 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5591ee0be1c0 0x5591ee559860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.988+0000 7f7ae9ffb640 10 client.16947 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.988+0000 7f7b01d9b640 1 --2- 172.21.15.117:0/2144654542 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5591ee0be1c0 0x5591ee559860 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad36e5fda991f2a4 server_cookie=adc5fdcc8c1dc20d in_seq=0 out_seq=0 2024-09-21T06:05:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.995+0000 7f7ae9ffb640 1 -- 172.21.15.117:0/2144654542 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5591ee4420f0 con 0x5591ee0be1c0 2024-09-21T06:05:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.995+0000 7f7ae9ffb640 10 client.16947 handle_command_reply: tid=0 2024-09-21T06:05:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.995+0000 7f7b10014740 1 client.16947 shutdown 2024-09-21T06:05:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.995+0000 7f7b10014740 1 -- 172.21.15.117:0/2144654542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591ee1094d0 msgr2=0x5591ee4e2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.995+0000 7f7b10014740 1 --2- 172.21.15.117:0/2144654542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591ee1094d0 0x5591ee4e2d00 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009ca0 tx=0x7f7af4031680 comp rx=0 tx=0).stop 2024-09-21T06:05:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.995+0000 7f7b10014740 1 -- 172.21.15.117:0/2144654542 shutdown_connections 2024-09-21T06:05:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.995+0000 7f7b10014740 1 --2- 172.21.15.117:0/2144654542 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5591ee0be1c0 0x5591ee559860 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.995+0000 7f7b10014740 1 --2- 172.21.15.117:0/2144654542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5591ee1094d0 0x5591ee4e2d00 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.995+0000 7f7ae9ffb640 0 client.16947 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.995+0000 7f7b10014740 1 --2- 172.21.15.117:0/2144654542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5591ee12a1b0 0x5591ee5558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.995+0000 7f7b10014740 1 --2- 172.21.15.117:0/2144654542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5591ee1012f0 0x5591ee4e2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.995+0000 7f7b10014740 1 -- 172.21.15.117:0/2144654542 >> 172.21.15.117:0/2144654542 conn(0x5591ee11e300 msgr2=0x5591ee4cbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.995+0000 7f7b10014740 1 -- 172.21.15.117:0/2144654542 shutdown_connections 2024-09-21T06:05:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.995+0000 7f7b10014740 1 -- 172.21.15.117:0/2144654542 wait complete. 2024-09-21T06:05:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.996+0000 7f7b10014740 20 client.16947 trim_cache size 0 max 16384 2024-09-21T06:05:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T06:05:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T06:05:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T06:05:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T06:05:14.001 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.996+0000 7f7b08fcc640 1 -- 172.21.15.117:0/1401739982 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7aec0b6680 msgr2=0x7f7aec0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:14.001 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.996+0000 7f7b08fcc640 1 --2- 172.21.15.117:0/1401739982 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7aec0b6680 0x7f7aec0b8b90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7afc0a9530 tx=0x7f7afc094040 comp rx=0 tx=0).stop 2024-09-21T06:05:14.001 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.996+0000 7f7b08fcc640 1 -- 172.21.15.117:0/1401739982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b0410dd90 msgr2=0x7f7b0419fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:14.001 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.996+0000 7f7b08fcc640 1 --2- 172.21.15.117:0/1401739982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b0410dd90 0x7f7b0419fc40 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00d750 tx=0x7f7afc00dc10 comp rx=0 tx=0).stop 2024-09-21T06:05:14.001 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.996+0000 7f7b08fcc640 1 -- 172.21.15.117:0/1401739982 shutdown_connections 2024-09-21T06:05:14.001 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.996+0000 7f7b08fcc640 1 --2- 172.21.15.117:0/1401739982 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7aec0b6680 0x7f7aec0b8b90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:14.001 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.996+0000 7f7b08fcc640 1 --2- 172.21.15.117:0/1401739982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b0410dd90 0x7f7b0419fc40 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:14.001 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.996+0000 7f7b08fcc640 1 --2- 172.21.15.117:0/1401739982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b04105280 0x7f7b0419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.996+0000 7f7b08fcc640 1 --2- 172.21.15.117:0/1401739982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b04104930 0x7f7b0419a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.996+0000 7f7b08fcc640 1 -- 172.21.15.117:0/1401739982 >> 172.21.15.117:0/1401739982 conn(0x7f7b040fe3c0 msgr2=0x7f7b041078a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.996+0000 7f7b08fcc640 1 -- 172.21.15.117:0/1401739982 shutdown_connections 2024-09-21T06:05:14.002 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:13.996+0000 7f7b08fcc640 1 -- 172.21.15.117:0/1401739982 wait complete. 2024-09-21T06:05:14.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:13 smithi192 ceph-mon[34700]: pgmap v567: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-09-21T06:05:14.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:13 smithi117 ceph-mon[30947]: pgmap v567: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-09-21T06:05:14.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:13 smithi138 ceph-mon[34391]: pgmap v567: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-09-21T06:05:16.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:15 smithi192 ceph-mon[34700]: pgmap v568: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-09-21T06:05:16.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:15 smithi117 ceph-mon[30947]: pgmap v568: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-09-21T06:05:16.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:15 smithi138 ceph-mon[34391]: pgmap v568: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-09-21T06:05:18.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:17 smithi117 ceph-mon[30947]: pgmap v569: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-21T06:05:18.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:17 smithi138 ceph-mon[34391]: pgmap v569: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-21T06:05:18.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:17 smithi192 ceph-mon[34700]: pgmap v569: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-09-21T06:05:20.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:19 smithi117 ceph-mon[30947]: pgmap v570: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-21T06:05:20.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:19 smithi138 ceph-mon[34391]: pgmap v570: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-21T06:05:20.482 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:19 smithi192 ceph-mon[34700]: pgmap v570: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-21T06:05:22.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:21 smithi117 ceph-mon[30947]: pgmap v571: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T06:05:22.282 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:21 smithi192 ceph-mon[34700]: pgmap v571: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T06:05:22.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:21 smithi138 ceph-mon[34391]: pgmap v571: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T06:05:24.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:23 smithi117 ceph-mon[30947]: pgmap v572: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-21T06:05:24.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:23 smithi138 ceph-mon[34391]: pgmap v572: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-21T06:05:24.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:23 smithi192 ceph-mon[34700]: pgmap v572: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-21T06:05:25.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:25 smithi138 ceph-mon[34391]: pgmap v573: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-21T06:05:25.479 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:25 smithi192 ceph-mon[34700]: pgmap v573: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-21T06:05:25.526 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:25 smithi117 ceph-mon[30947]: pgmap v573: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-21T06:05:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.107+0000 7fe602994640 1 -- 172.21.15.117:0/4247193945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5fc10dd80 msgr2=0x7fe5fc110260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.107+0000 7fe602994640 1 --2- 172.21.15.117:0/4247193945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5fc10dd80 0x7fe5fc110260 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4009ca0 tx=0x7fe5f402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.107+0000 7fe602994640 1 -- 172.21.15.117:0/4247193945 shutdown_connections 2024-09-21T06:05:27.109 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.107+0000 7fe602994640 1 --2- 172.21.15.117:0/4247193945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5fc10dd80 0x7fe5fc110260 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.107+0000 7fe602994640 1 --2- 172.21.15.117:0/4247193945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5fc1052b0 0x7fe5fc10d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.107+0000 7fe602994640 1 --2- 172.21.15.117:0/4247193945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5fc104960 0x7fe5fc104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.107+0000 7fe602994640 1 -- 172.21.15.117:0/4247193945 >> 172.21.15.117:0/4247193945 conn(0x7fe5fc07a740 msgr2=0x7fe5fc07ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.107+0000 7fe602994640 1 -- 172.21.15.117:0/4247193945 shutdown_connections 2024-09-21T06:05:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.107+0000 7fe602994640 1 -- 172.21.15.117:0/4247193945 wait complete. 2024-09-21T06:05:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.108+0000 7fe602994640 1 Processor -- start 2024-09-21T06:05:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.108+0000 7fe602994640 1 -- start start 2024-09-21T06:05:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.108+0000 7fe602994640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5fc104960 0x7fe5fc19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.108+0000 7fe602994640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5fc1052b0 0x7fe5fc19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.108+0000 7fe602994640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5fc10dd80 0x7fe5fc19fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.108+0000 7fe602994640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe5fc1139f0 con 0x7fe5fc1052b0 2024-09-21T06:05:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.108+0000 7fe602994640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe5fc113870 con 0x7fe5fc104960 2024-09-21T06:05:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.108+0000 7fe602994640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe5fc113b70 con 0x7fe5fc10dd80 2024-09-21T06:05:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.108+0000 7fe5fb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5fc1052b0 0x7fe5fc19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.109+0000 7fe5fb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5fc1052b0 0x7fe5fc19a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41746/0 (socket says 172.21.15.117:41746) 2024-09-21T06:05:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.109+0000 7fe5fb7fe640 1 -- 172.21.15.117:0/4181756214 learned_addr learned my addr 172.21.15.117:0/4181756214 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.109+0000 7fe600f31640 1 --2- 172.21.15.117:0/4181756214 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5fc10dd80 0x7fe5fc19fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.109+0000 7fe5fbfff640 1 --2- 172.21.15.117:0/4181756214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5fc104960 0x7fe5fc19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.109+0000 7fe5fb7fe640 1 -- 172.21.15.117:0/4181756214 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5fc10dd80 msgr2=0x7fe5fc19fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.109+0000 7fe5fb7fe640 1 --2- 172.21.15.117:0/4181756214 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5fc10dd80 0x7fe5fc19fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.109+0000 7fe5fb7fe640 1 -- 172.21.15.117:0/4181756214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5fc104960 msgr2=0x7fe5fc19a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.109+0000 7fe5fb7fe640 1 --2- 172.21.15.117:0/4181756214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5fc104960 0x7fe5fc19a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.109+0000 7fe5fb7fe640 1 -- 172.21.15.117:0/4181756214 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5fc1a0360 con 0x7fe5fc1052b0 2024-09-21T06:05:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.109+0000 7fe600f31640 1 --2- 172.21.15.117:0/4181756214 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5fc10dd80 0x7fe5fc19fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:05:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.109+0000 7fe5fb7fe640 1 --2- 172.21.15.117:0/4181756214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5fc1052b0 0x7fe5fc19a9d0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fe5f000d750 tx=0x7fe5f000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.109+0000 7fe5f97fa640 1 -- 172.21.15.117:0/4181756214 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f0017070 con 0x7fe5fc1052b0 2024-09-21T06:05:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.109+0000 7fe602994640 1 -- 172.21.15.117:0/4181756214 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5fc1a0650 con 0x7fe5fc1052b0 2024-09-21T06:05:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.109+0000 7fe5f97fa640 1 -- 172.21.15.117:0/4181756214 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5f00054d0 con 0x7fe5fc1052b0 2024-09-21T06:05:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.109+0000 7fe5f97fa640 1 -- 172.21.15.117:0/4181756214 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f0004c30 con 0x7fe5fc1052b0 2024-09-21T06:05:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.109+0000 7fe602994640 1 -- 172.21.15.117:0/4181756214 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5fc1a6de0 con 0x7fe5fc1052b0 2024-09-21T06:05:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.112+0000 7fe6099da740 1 Processor -- start 2024-09-21T06:05:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.113+0000 7fe6099da740 1 -- start start 2024-09-21T06:05:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.113+0000 7fe6099da740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0cfec21b0 0x55c0d02679b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.113+0000 7fe6099da740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0cfe992f0 0x55c0d0267ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.113+0000 7fe6099da740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0cfea14d0 0x55c0d0268430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.113+0000 7fe6099da740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c0d01dc6f0 con 0x55c0cfea14d0 2024-09-21T06:05:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.113+0000 7fe6099da740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c0d01cbb20 con 0x55c0cfec21b0 2024-09-21T06:05:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.113+0000 7fe6099da740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c0d01cf930 con 0x55c0cfe992f0 2024-09-21T06:05:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.113+0000 7fe5f97fa640 1 -- 172.21.15.117:0/4181756214 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fe5f0004dd0 con 0x7fe5fc1052b0 2024-09-21T06:05:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.113+0000 7fe5fb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0cfea14d0 0x55c0d0268430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.113+0000 7fe5fbfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0cfe992f0 0x55c0d0267ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.113+0000 7fe600f31640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0cfec21b0 0x55c0d02679b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.113+0000 7fe5fb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0cfea14d0 0x55c0d0268430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41748/0 (socket says 172.21.15.117:41748) 2024-09-21T06:05:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.113+0000 7fe5fb7fe640 1 -- 172.21.15.117:0/3728990948 learned_addr learned my addr 172.21.15.117:0/3728990948 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.113+0000 7fe5fb7fe640 1 -- 172.21.15.117:0/3728990948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0cfe992f0 msgr2=0x55c0d0267ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.113+0000 7fe5fb7fe640 1 --2- 172.21.15.117:0/3728990948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0cfe992f0 0x55c0d0267ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.113+0000 7fe5fb7fe640 1 -- 172.21.15.117:0/3728990948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0cfec21b0 msgr2=0x55c0d02679b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.113+0000 7fe5fb7fe640 1 --2- 172.21.15.117:0/3728990948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0cfec21b0 0x55c0d02679b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.114+0000 7fe5fb7fe640 1 -- 172.21.15.117:0/3728990948 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0d0191530 con 0x55c0cfea14d0 2024-09-21T06:05:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.114+0000 7fe5fb7fe640 1 --2- 172.21.15.117:0/3728990948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0cfea14d0 0x55c0d0268430 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00e48a0 tx=0x7fe5f00ef880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e9bcd0db30f793 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.114+0000 7fe5f97fa640 1 --2- 172.21.15.117:0/4181756214 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe5e40b6680 0x7fe5e40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.114+0000 7fe5e27fc640 1 -- 172.21.15.117:0/3728990948 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f00fa070 con 0x55c0cfea14d0 2024-09-21T06:05:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.114+0000 7fe600f31640 1 --2- 172.21.15.117:0/4181756214 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe5e40b6680 0x7fe5e40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.114+0000 7fe5e27fc640 1 -- 172.21.15.117:0/3728990948 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5f00acb40 con 0x55c0cfea14d0 2024-09-21T06:05:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.114+0000 7fe5e27fc640 1 -- 172.21.15.117:0/3728990948 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f00ac7a0 con 0x55c0cfea14d0 2024-09-21T06:05:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.114+0000 7fe5f97fa640 1 -- 172.21.15.117:0/4181756214 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7fe5f00e7190 con 0x7fe5fc1052b0 2024-09-21T06:05:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.115+0000 7fe6099da740 1 -- 172.21.15.117:0/3728990948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0cfea14d0 msgr2=0x55c0d0268430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.115+0000 7fe600f31640 1 --2- 172.21.15.117:0/4181756214 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe5e40b6680 0x7fe5e40b8b90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4004af0 tx=0x7fe5f4005c30 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.115+0000 7fe6099da740 1 --2- 172.21.15.117:0/3728990948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0cfea14d0 0x55c0d0268430 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00e48a0 tx=0x7fe5f00ef880 comp rx=0 tx=0).stop 2024-09-21T06:05:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.115+0000 7fe6099da740 1 -- 172.21.15.117:0/3728990948 shutdown_connections 2024-09-21T06:05:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.115+0000 7fe6099da740 1 --2- 172.21.15.117:0/3728990948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0cfea14d0 0x55c0d0268430 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.115+0000 7fe6099da740 1 --2- 172.21.15.117:0/3728990948 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0cfe992f0 0x55c0d0267ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.115+0000 7fe6099da740 1 --2- 172.21.15.117:0/3728990948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0cfec21b0 0x55c0d02679b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.115+0000 7fe6099da740 1 -- 172.21.15.117:0/3728990948 >> 172.21.15.117:0/3728990948 conn(0x55c0cfeb6300 msgr2=0x55c0d02658a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.115+0000 7fe6099da740 1 -- 172.21.15.117:0/3728990948 shutdown_connections 2024-09-21T06:05:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.115+0000 7fe6099da740 1 -- 172.21.15.117:0/3728990948 wait complete. 2024-09-21T06:05:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.115+0000 7fe6099da740 1 -- start start 2024-09-21T06:05:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.116+0000 7fe6099da740 1 Processor -- start 2024-09-21T06:05:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.116+0000 7fe6099da740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0cfe992f0 0x55c0d0272fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.116+0000 7fe6099da740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0cfea14d0 0x55c0d0273500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.116+0000 7fe6099da740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0cfec21b0 0x55c0d0273a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.116+0000 7fe6099da740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c0d0103ab0 con 0x55c0cfea14d0 2024-09-21T06:05:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.116+0000 7fe6099da740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c0d0171730 con 0x55c0cfe992f0 2024-09-21T06:05:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.116+0000 7fe6099da740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c0d0276130 con 0x55c0cfec21b0 2024-09-21T06:05:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.116+0000 7fe6099da740 10 client.0 _ll_register_callbacks cb 0x55c0d01c56b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:05:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.116+0000 7fe5fb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0cfea14d0 0x55c0d0273500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.116+0000 7fe5fb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0cfea14d0 0x55c0d0273500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41750/0 (socket says 172.21.15.117:41750) 2024-09-21T06:05:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.116+0000 7fe5fb7fe640 1 -- 172.21.15.117:0/2526879962 learned_addr learned my addr 172.21.15.117:0/2526879962 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.116+0000 7fe600f31640 1 --2- 172.21.15.117:0/2526879962 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0cfec21b0 0x55c0d0273a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.116+0000 7fe5fbfff640 1 --2- 172.21.15.117:0/2526879962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0cfe992f0 0x55c0d0272fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.116+0000 7fe5fb7fe640 1 -- 172.21.15.117:0/2526879962 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0cfec21b0 msgr2=0x55c0d0273a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.116+0000 7fe5fb7fe640 1 --2- 172.21.15.117:0/2526879962 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0cfec21b0 0x55c0d0273a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.116+0000 7fe5fb7fe640 1 -- 172.21.15.117:0/2526879962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0cfe992f0 msgr2=0x55c0d0272fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.116+0000 7fe5fb7fe640 1 --2- 172.21.15.117:0/2526879962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0cfe992f0 0x55c0d0272fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.117+0000 7fe5fb7fe640 1 -- 172.21.15.117:0/2526879962 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0cff2d790 con 0x55c0cfea14d0 2024-09-21T06:05:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.117+0000 7fe5fb7fe640 1 --2- 172.21.15.117:0/2526879962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0cfea14d0 0x55c0d0273500 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00acfc0 tx=0x7fe5f00ac9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a428f567e96d9bfe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.117+0000 7fe5db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:05:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.117+0000 7fe5db7fe640 1 -- 172.21.15.117:0/2526879962 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f009cb00 con 0x55c0cfea14d0 2024-09-21T06:05:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.117+0000 7fe5db7fe640 1 -- 172.21.15.117:0/2526879962 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7fe5f009c110 con 0x55c0cfea14d0 2024-09-21T06:05:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.117+0000 7fe6099da740 1 -- 172.21.15.117:0/2526879962 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5f00acb40 con 0x55c0cfea14d0 2024-09-21T06:05:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.117+0000 7fe5db7fe640 1 -- 172.21.15.117:0/2526879962 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe5f0020020 con 0x55c0cfea14d0 2024-09-21T06:05:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.117+0000 7fe6099da740 10 client.16959 fetch_fsmap learned FSMap version 29 2024-09-21T06:05:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.117+0000 7fe6099da740 1 -- 172.21.15.117:0/2526879962 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c0cffa6620 con 0x55c0cfea14d0 2024-09-21T06:05:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.117+0000 7fe5db7fe640 1 -- 172.21.15.117:0/2526879962 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5f00acb40 con 0x55c0cfea14d0 2024-09-21T06:05:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.117+0000 7fe5db7fe640 1 -- 172.21.15.117:0/2526879962 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe5f00e4bb0 con 0x55c0cfea14d0 2024-09-21T06:05:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.117+0000 7fe6099da740 10 client.16959 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:05:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.117+0000 7fe6099da740 10 client.16959 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:05:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.117+0000 7fe6099da740 20 client.16959 populate_metadata read hostname 'smithi117' 2024-09-21T06:05:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.117+0000 7fe6099da740 1 --2- 172.21.15.117:0/2526879962 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c0cfe80d80 0x55c0d0284f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.117+0000 7fe6099da740 4 client.16959 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:05:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.117+0000 7fe6099da740 1 -- 172.21.15.117:0/2526879962 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0d01de640 con 0x55c0cfe80d80 2024-09-21T06:05:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.118+0000 7fe5fbfff640 1 --2- 172.21.15.117:0/2526879962 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c0cfe80d80 0x55c0d0284f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.118+0000 7fe5fbfff640 1 --2- 172.21.15.117:0/2526879962 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c0cfe80d80 0x55c0d0284f70 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fede47486abe8c06 server_cookie=1a781d241da276db in_seq=0 out_seq=0 2024-09-21T06:05:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.118+0000 7fe5db7fe640 10 client.16959 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.119+0000 7fe5db7fe640 1 -- 172.21.15.117:0/2526879962 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c0d01de640 con 0x55c0cfe80d80 2024-09-21T06:05:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.119+0000 7fe5db7fe640 10 client.16959 handle_command_reply: tid=0 2024-09-21T06:05:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.119+0000 7fe6099da740 1 client.16959 shutdown 2024-09-21T06:05:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.119+0000 7fe6099da740 1 -- 172.21.15.117:0/2526879962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0cfea14d0 msgr2=0x55c0d0273500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.119+0000 7fe6099da740 1 --2- 172.21.15.117:0/2526879962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0cfea14d0 0x55c0d0273500 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fe5f00acfc0 tx=0x7fe5f00ac9f0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.119+0000 7fe6099da740 1 -- 172.21.15.117:0/2526879962 shutdown_connections 2024-09-21T06:05:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.119+0000 7fe6099da740 1 --2- 172.21.15.117:0/2526879962 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c0cfe80d80 0x55c0d0284f70 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.119+0000 7fe6099da740 1 --2- 172.21.15.117:0/2526879962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0cfea14d0 0x55c0d0273500 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.119+0000 7fe5db7fe640 0 client.16959 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.119+0000 7fe6099da740 1 --2- 172.21.15.117:0/2526879962 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0cfec21b0 0x55c0d0273a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.119+0000 7fe6099da740 1 --2- 172.21.15.117:0/2526879962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0cfe992f0 0x55c0d0272fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.119+0000 7fe6099da740 1 -- 172.21.15.117:0/2526879962 >> 172.21.15.117:0/2526879962 conn(0x55c0cfeb6300 msgr2=0x55c0d02689e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.119+0000 7fe6099da740 1 -- 172.21.15.117:0/2526879962 shutdown_connections 2024-09-21T06:05:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.119+0000 7fe6099da740 1 -- 172.21.15.117:0/2526879962 wait complete. 2024-09-21T06:05:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.119+0000 7fe6099da740 20 client.16959 trim_cache size 0 max 16384 2024-09-21T06:05:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.134+0000 7fe6099da740 1 Processor -- start 2024-09-21T06:05:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.134+0000 7fe6099da740 1 -- start start 2024-09-21T06:05:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.134+0000 7fe6099da740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0cfe992f0 0x55c0d02690f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.134+0000 7fe6099da740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0cfea14d0 0x55c0d0270190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.134+0000 7fe6099da740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0cfec21b0 0x55c0d02706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.134+0000 7fe6099da740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe5fc1af5b0 con 0x55c0cfe992f0 2024-09-21T06:05:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.134+0000 7fe6099da740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe5fc1b10f0 con 0x55c0cfea14d0 2024-09-21T06:05:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.134+0000 7fe6099da740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe5fc1afc80 con 0x55c0cfec21b0 2024-09-21T06:05:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.134+0000 7fe5fbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0cfe992f0 0x55c0d02690f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.134+0000 7fe5fbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0cfe992f0 0x55c0d02690f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41766/0 (socket says 172.21.15.117:41766) 2024-09-21T06:05:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.134+0000 7fe5fbfff640 1 -- 172.21.15.117:0/2698761619 learned_addr learned my addr 172.21.15.117:0/2698761619 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.135+0000 7fe600f31640 1 --2- 172.21.15.117:0/2698761619 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0cfec21b0 0x55c0d02706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.135+0000 7fe5fbfff640 1 -- 172.21.15.117:0/2698761619 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0cfec21b0 msgr2=0x55c0d02706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.135+0000 7fe5fbfff640 1 --2- 172.21.15.117:0/2698761619 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0cfec21b0 0x55c0d02706d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.135+0000 7fe5fb7fe640 1 --2- 172.21.15.117:0/2698761619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0cfea14d0 0x55c0d0270190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.135+0000 7fe5fbfff640 1 -- 172.21.15.117:0/2698761619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0cfea14d0 msgr2=0x55c0d0270190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.135+0000 7fe5fbfff640 1 --2- 172.21.15.117:0/2698761619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0cfea14d0 0x55c0d0270190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.135+0000 7fe5fbfff640 1 -- 172.21.15.117:0/2698761619 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0d0269cb0 con 0x55c0cfe992f0 2024-09-21T06:05:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.135+0000 7fe5fbfff640 1 --2- 172.21.15.117:0/2698761619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0cfe992f0 0x55c0d02690f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec000c00 tx=0x7fe5ec011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad7e2d9bb7f8b64 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.135+0000 7fe5e1ffb640 1 -- 172.21.15.117:0/2698761619 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec01d620 con 0x55c0cfe992f0 2024-09-21T06:05:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.135+0000 7fe5e1ffb640 1 -- 172.21.15.117:0/2698761619 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5ec01dc70 con 0x55c0cfe992f0 2024-09-21T06:05:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.135+0000 7fe5e1ffb640 1 -- 172.21.15.117:0/2698761619 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec01ccb0 con 0x55c0cfe992f0 2024-09-21T06:05:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.136+0000 7fe6099da740 1 -- 172.21.15.117:0/2698761619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0cfe992f0 msgr2=0x55c0d02690f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.136+0000 7fe6099da740 1 --2- 172.21.15.117:0/2698761619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0cfe992f0 0x55c0d02690f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec000c00 tx=0x7fe5ec011810 comp rx=0 tx=0).stop 2024-09-21T06:05:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.136+0000 7fe6099da740 1 -- 172.21.15.117:0/2698761619 shutdown_connections 2024-09-21T06:05:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.136+0000 7fe6099da740 1 --2- 172.21.15.117:0/2698761619 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0cfec21b0 0x55c0d02706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.136+0000 7fe6099da740 1 --2- 172.21.15.117:0/2698761619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0cfea14d0 0x55c0d0270190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.136+0000 7fe6099da740 1 --2- 172.21.15.117:0/2698761619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0cfe992f0 0x55c0d02690f0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.136+0000 7fe6099da740 1 -- 172.21.15.117:0/2698761619 >> 172.21.15.117:0/2698761619 conn(0x55c0cfeb6300 msgr2=0x55c0d02689e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.136+0000 7fe6099da740 1 -- 172.21.15.117:0/2698761619 shutdown_connections 2024-09-21T06:05:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.136+0000 7fe6099da740 1 -- 172.21.15.117:0/2698761619 wait complete. 2024-09-21T06:05:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.136+0000 7fe6099da740 1 -- start start 2024-09-21T06:05:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.136+0000 7fe6099da740 1 Processor -- start 2024-09-21T06:05:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.137+0000 7fe6099da740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0cfe992f0 0x55c0d0287aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.137+0000 7fe6099da740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0cfea14d0 0x55c0d0288080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.137+0000 7fe6099da740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0cfec21b0 0x55c0d02e6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.137+0000 7fe6099da740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe5fc1b0a20 con 0x55c0cfe992f0 2024-09-21T06:05:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.137+0000 7fe6099da740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe5fc1b0350 con 0x55c0cfec21b0 2024-09-21T06:05:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.137+0000 7fe6099da740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe5fc1ad5b0 con 0x55c0cfea14d0 2024-09-21T06:05:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.137+0000 7fe6099da740 10 client.0 _ll_register_callbacks cb 0x7fe5f00f8070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:05:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.137+0000 7fe5fbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0cfe992f0 0x55c0d0287aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.137+0000 7fe5fbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0cfe992f0 0x55c0d0287aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41772/0 (socket says 172.21.15.117:41772) 2024-09-21T06:05:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.137+0000 7fe5fbfff640 1 -- 172.21.15.117:0/3133720914 learned_addr learned my addr 172.21.15.117:0/3133720914 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.137+0000 7fe5fb7fe640 1 --2- 172.21.15.117:0/3133720914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0cfea14d0 0x55c0d0288080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.137+0000 7fe600f31640 1 --2- 172.21.15.117:0/3133720914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0cfec21b0 0x55c0d02e6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.137+0000 7fe5fbfff640 1 -- 172.21.15.117:0/3133720914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0cfea14d0 msgr2=0x55c0d0288080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.137+0000 7fe5fbfff640 1 --2- 172.21.15.117:0/3133720914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0cfea14d0 0x55c0d0288080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.137+0000 7fe5fbfff640 1 -- 172.21.15.117:0/3133720914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0cfec21b0 msgr2=0x55c0d02e6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.137+0000 7fe5fbfff640 1 --2- 172.21.15.117:0/3133720914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0cfec21b0 0x55c0d02e6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.137+0000 7fe5fbfff640 1 -- 172.21.15.117:0/3133720914 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0d02e6dc0 con 0x55c0cfe992f0 2024-09-21T06:05:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.137+0000 7fe600f31640 1 --2- 172.21.15.117:0/3133720914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0cfec21b0 0x55c0d02e6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:05:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.137+0000 7fe5fbfff640 1 --2- 172.21.15.117:0/3133720914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0cfe992f0 0x55c0d0287aa0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00bca0 tx=0x7fe5ec01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf26073256bbdb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.137+0000 7fe5db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:05:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.137+0000 7fe5db7fe640 1 -- 172.21.15.117:0/3133720914 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec01cdc0 con 0x55c0cfe992f0 2024-09-21T06:05:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.137+0000 7fe5db7fe640 1 -- 172.21.15.117:0/3133720914 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7fe5ec004020 con 0x55c0cfe992f0 2024-09-21T06:05:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.137+0000 7fe6099da740 1 -- 172.21.15.117:0/3133720914 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5ec01dc70 con 0x55c0cfe992f0 2024-09-21T06:05:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.138+0000 7fe5db7fe640 1 -- 172.21.15.117:0/3133720914 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe5ec004220 con 0x55c0cfe992f0 2024-09-21T06:05:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.138+0000 7fe6099da740 10 client.16965 fetch_fsmap learned FSMap version 29 2024-09-21T06:05:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.138+0000 7fe6099da740 1 -- 172.21.15.117:0/3133720914 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c0d02ef2b0 con 0x55c0cfe992f0 2024-09-21T06:05:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.138+0000 7fe5db7fe640 1 -- 172.21.15.117:0/3133720914 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5ec01dc70 con 0x55c0cfe992f0 2024-09-21T06:05:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.138+0000 7fe5db7fe640 1 -- 172.21.15.117:0/3133720914 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe5ec01d720 con 0x55c0cfe992f0 2024-09-21T06:05:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.138+0000 7fe6099da740 10 client.16965 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:05:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.138+0000 7fe6099da740 10 client.16965 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:05:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.138+0000 7fe6099da740 20 client.16965 populate_metadata read hostname 'smithi117' 2024-09-21T06:05:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.138+0000 7fe6099da740 1 --2- 172.21.15.117:0/3133720914 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c0cfe561c0 0x55c0d02f1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.138+0000 7fe6099da740 4 client.16965 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:05:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.138+0000 7fe6099da740 1 -- 172.21.15.117:0/3133720914 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c0d01da0f0 con 0x55c0cfe561c0 2024-09-21T06:05:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.138+0000 7fe5fb7fe640 1 --2- 172.21.15.117:0/3133720914 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c0cfe561c0 0x55c0d02f1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.139+0000 7fe5db7fe640 10 client.16965 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.139+0000 7fe5fb7fe640 1 --2- 172.21.15.117:0/3133720914 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c0cfe561c0 0x55c0d02f1a90 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=774072d0736634e server_cookie=6cd8a1ad515c3c8c in_seq=0 out_seq=0 2024-09-21T06:05:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.146+0000 7fe5db7fe640 1 -- 172.21.15.117:0/3133720914 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c0d01da0f0 con 0x55c0cfe561c0 2024-09-21T06:05:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.146+0000 7fe5db7fe640 10 client.16965 handle_command_reply: tid=0 2024-09-21T06:05:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.146+0000 7fe6099da740 1 client.16965 shutdown 2024-09-21T06:05:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.146+0000 7fe6099da740 1 -- 172.21.15.117:0/3133720914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0cfe992f0 msgr2=0x55c0d0287aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.146+0000 7fe6099da740 1 --2- 172.21.15.117:0/3133720914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0cfe992f0 0x55c0d0287aa0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00bca0 tx=0x7fe5ec01c500 comp rx=0 tx=0).stop 2024-09-21T06:05:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.146+0000 7fe6099da740 1 -- 172.21.15.117:0/3133720914 shutdown_connections 2024-09-21T06:05:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.146+0000 7fe6099da740 1 --2- 172.21.15.117:0/3133720914 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c0cfe561c0 0x55c0d02f1a90 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.146+0000 7fe6099da740 1 --2- 172.21.15.117:0/3133720914 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c0cfec21b0 0x55c0d02e6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.146+0000 7fe5db7fe640 0 client.16965 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.146+0000 7fe6099da740 1 --2- 172.21.15.117:0/3133720914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0cfea14d0 0x55c0d0288080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.146+0000 7fe6099da740 1 --2- 172.21.15.117:0/3133720914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c0cfe992f0 0x55c0d0287aa0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.146+0000 7fe6099da740 1 -- 172.21.15.117:0/3133720914 >> 172.21.15.117:0/3133720914 conn(0x55c0cfeb6300 msgr2=0x55c0d0265880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.146+0000 7fe6099da740 1 -- 172.21.15.117:0/3133720914 shutdown_connections 2024-09-21T06:05:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.146+0000 7fe6099da740 1 -- 172.21.15.117:0/3133720914 wait complete. 2024-09-21T06:05:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.146+0000 7fe6099da740 20 client.16965 trim_cache size 0 max 16384 2024-09-21T06:05:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T06:05:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T06:05:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T06:05:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T06:05:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.147+0000 7fe602994640 1 -- 172.21.15.117:0/4181756214 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe5e40b6680 msgr2=0x7fe5e40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.147+0000 7fe602994640 1 --2- 172.21.15.117:0/4181756214 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe5e40b6680 0x7fe5e40b8b90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4004af0 tx=0x7fe5f4005c30 comp rx=0 tx=0).stop 2024-09-21T06:05:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.147+0000 7fe602994640 1 -- 172.21.15.117:0/4181756214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5fc1052b0 msgr2=0x7fe5fc19a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.147+0000 7fe602994640 1 --2- 172.21.15.117:0/4181756214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5fc1052b0 0x7fe5fc19a9d0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fe5f000d750 tx=0x7fe5f000dc10 comp rx=0 tx=0).stop 2024-09-21T06:05:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.147+0000 7fe602994640 1 -- 172.21.15.117:0/4181756214 shutdown_connections 2024-09-21T06:05:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.147+0000 7fe602994640 1 --2- 172.21.15.117:0/4181756214 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe5e40b6680 0x7fe5e40b8b90 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.147+0000 7fe602994640 1 --2- 172.21.15.117:0/4181756214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5fc1052b0 0x7fe5fc19a9d0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.147+0000 7fe602994640 1 --2- 172.21.15.117:0/4181756214 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5fc10dd80 0x7fe5fc19fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.147+0000 7fe602994640 1 --2- 172.21.15.117:0/4181756214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5fc104960 0x7fe5fc19a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.147+0000 7fe602994640 1 -- 172.21.15.117:0/4181756214 >> 172.21.15.117:0/4181756214 conn(0x7fe5fc07a740 msgr2=0x7fe5fc107840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.147+0000 7fe602994640 1 -- 172.21.15.117:0/4181756214 shutdown_connections 2024-09-21T06:05:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:27.147+0000 7fe602994640 1 -- 172.21.15.117:0/4181756214 wait complete. 2024-09-21T06:05:28.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:27 smithi192 ceph-mon[34700]: pgmap v574: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-21T06:05:28.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:27 smithi117 ceph-mon[30947]: pgmap v574: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-21T06:05:28.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:27 smithi138 ceph-mon[34391]: pgmap v574: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-21T06:05:28.555 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:05:28.820 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:05:29.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.349+0000 7f677c517640 1 -- 172.21.15.117:0/1843394439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67741094a0 msgr2=0x7f67741098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:29.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.349+0000 7f677c517640 1 --2- 172.21.15.117:0/1843394439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67741094a0 0x7f67741098b0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f6764009900 tx=0x7f676402a410 comp rx=0 tx=0).stop 2024-09-21T06:05:29.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.349+0000 7f677c517640 1 -- 172.21.15.117:0/1843394439 shutdown_connections 2024-09-21T06:05:29.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.349+0000 7f677c517640 1 --2- 172.21.15.117:0/1843394439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67741094a0 0x7f67741098b0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.349+0000 7f677c517640 1 --2- 172.21.15.117:0/1843394439 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6774109df0 0x7f6774110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.349+0000 7f677c517640 1 --2- 172.21.15.117:0/1843394439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6774103b60 0x7f6774103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.349+0000 7f677c517640 1 -- 172.21.15.117:0/1843394439 >> 172.21.15.117:0/1843394439 conn(0x7f677407b850 msgr2=0x7f677407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:29.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.349+0000 7f677c517640 1 -- 172.21.15.117:0/1843394439 shutdown_connections 2024-09-21T06:05:29.359 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.349+0000 7f677c517640 1 -- 172.21.15.117:0/1843394439 wait complete. 2024-09-21T06:05:29.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.350+0000 7f677c517640 1 Processor -- start 2024-09-21T06:05:29.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.350+0000 7f677c517640 1 -- start start 2024-09-21T06:05:29.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.350+0000 7f677c517640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6774103b60 0x7f677419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:29.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.350+0000 7f677c517640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67741094a0 0x7f677419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:29.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.350+0000 7f677c517640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6774109df0 0x7f677419fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:29.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.350+0000 7f677c517640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6774113ad0 con 0x7f67741094a0 2024-09-21T06:05:29.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.350+0000 7f677c517640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6774113950 con 0x7f6774103b60 2024-09-21T06:05:29.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.350+0000 7f677c517640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6774113c50 con 0x7f6774109df0 2024-09-21T06:05:29.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.350+0000 7f6779ab2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67741094a0 0x7f677419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:29.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.350+0000 7f6779ab2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67741094a0 0x7f677419ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41782/0 (socket says 172.21.15.117:41782) 2024-09-21T06:05:29.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.350+0000 7f6779ab2640 1 -- 172.21.15.117:0/1506393177 learned_addr learned my addr 172.21.15.117:0/1506393177 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:29.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.350+0000 7f677a2b3640 1 --2- 172.21.15.117:0/1506393177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6774103b60 0x7f677419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:29.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.351+0000 7f677aab4640 1 --2- 172.21.15.117:0/1506393177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6774109df0 0x7f677419fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:29.360 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.351+0000 7f6779ab2640 1 -- 172.21.15.117:0/1506393177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6774109df0 msgr2=0x7f677419fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:29.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.351+0000 7f6779ab2640 1 --2- 172.21.15.117:0/1506393177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6774109df0 0x7f677419fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.351+0000 7f6779ab2640 1 -- 172.21.15.117:0/1506393177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6774103b60 msgr2=0x7f677419a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:29.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.351+0000 7f6779ab2640 1 --2- 172.21.15.117:0/1506393177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6774103b60 0x7f677419a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.351+0000 7f6779ab2640 1 -- 172.21.15.117:0/1506393177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67741a04d0 con 0x7f67741094a0 2024-09-21T06:05:29.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.351+0000 7f6779ab2640 1 --2- 172.21.15.117:0/1506393177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67741094a0 0x7f677419ab00 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f676402a930 tx=0x7f6764004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:29.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.351+0000 7f676b7fe640 1 -- 172.21.15.117:0/1506393177 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676403f030 con 0x7f67741094a0 2024-09-21T06:05:29.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.351+0000 7f676b7fe640 1 -- 172.21.15.117:0/1506393177 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6764003aa0 con 0x7f67741094a0 2024-09-21T06:05:29.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.351+0000 7f677c517640 1 -- 172.21.15.117:0/1506393177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67741a0760 con 0x7f67741094a0 2024-09-21T06:05:29.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.351+0000 7f676b7fe640 1 -- 172.21.15.117:0/1506393177 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764035070 con 0x7f67741094a0 2024-09-21T06:05:29.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.351+0000 7f677c517640 1 -- 172.21.15.117:0/1506393177 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67741a6f20 con 0x7f67741094a0 2024-09-21T06:05:29.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.354+0000 7f6782a8b740 1 Processor -- start 2024-09-21T06:05:29.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.354+0000 7f6782a8b740 1 -- start start 2024-09-21T06:05:29.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.354+0000 7f676b7fe640 1 -- 172.21.15.117:0/1506393177 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f6764042030 con 0x7f67741094a0 2024-09-21T06:05:29.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.354+0000 7f6782a8b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0cd17070 0x556e0d0be650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:29.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.354+0000 7f6782a8b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e0ccee1b0 0x556e0d0beb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:29.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.354+0000 7f6782a8b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e0ccf6390 0x556e0d0bf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:29.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.354+0000 7f6782a8b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556e0d034e50 con 0x556e0cd17070 2024-09-21T06:05:29.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.354+0000 7f6782a8b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556e0d024280 con 0x556e0ccf6390 2024-09-21T06:05:29.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.354+0000 7f6782a8b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556e0d028090 con 0x556e0ccee1b0 2024-09-21T06:05:29.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.354+0000 7f677aab4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0cd17070 0x556e0d0be650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:29.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.355+0000 7f677aab4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0cd17070 0x556e0d0be650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41788/0 (socket says 172.21.15.117:41788) 2024-09-21T06:05:29.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.355+0000 7f677aab4640 1 -- 172.21.15.117:0/2370618062 learned_addr learned my addr 172.21.15.117:0/2370618062 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:29.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.355+0000 7f677a2b3640 1 --2- 172.21.15.117:0/2370618062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e0ccee1b0 0x556e0d0beb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:29.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.355+0000 7f6779ab2640 1 --2- 172.21.15.117:0/2370618062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e0ccf6390 0x556e0d0bf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:29.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.355+0000 7f677aab4640 1 -- 172.21.15.117:0/2370618062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e0ccee1b0 msgr2=0x556e0d0beb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:29.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.355+0000 7f677aab4640 1 --2- 172.21.15.117:0/2370618062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e0ccee1b0 0x556e0d0beb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.355+0000 7f677aab4640 1 -- 172.21.15.117:0/2370618062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e0ccf6390 msgr2=0x556e0d0bf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:29.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.355+0000 7f677aab4640 1 --2- 172.21.15.117:0/2370618062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e0ccf6390 0x556e0d0bf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.355+0000 7f677aab4640 1 -- 172.21.15.117:0/2370618062 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e0cd82650 con 0x556e0cd17070 2024-09-21T06:05:29.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.355+0000 7f6779ab2640 1 --2- 172.21.15.117:0/2370618062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e0ccf6390 0x556e0d0bf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:05:29.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.355+0000 7f677aab4640 1 --2- 172.21.15.117:0/2370618062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0cd17070 0x556e0d0be650 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f676c009ee0 tx=0x7f676c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c963c2044327248 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:29.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.355+0000 7f676b7fe640 1 --2- 172.21.15.117:0/1506393177 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f67600b6680 0x7f67600b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:29.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.355+0000 7f677a2b3640 1 --2- 172.21.15.117:0/1506393177 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f67600b6680 0x7f67600b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:29.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.355+0000 7f6768ff9640 1 -- 172.21.15.117:0/2370618062 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c016070 con 0x556e0cd17070 2024-09-21T06:05:29.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.355+0000 7f676b7fe640 1 -- 172.21.15.117:0/1506393177 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f6764107dc0 con 0x7f67741094a0 2024-09-21T06:05:29.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.355+0000 7f6768ff9640 1 -- 172.21.15.117:0/2370618062 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f676c004d20 con 0x556e0cd17070 2024-09-21T06:05:29.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.355+0000 7f6768ff9640 1 -- 172.21.15.117:0/2370618062 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c012740 con 0x556e0cd17070 2024-09-21T06:05:29.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.356+0000 7f6782a8b740 1 -- 172.21.15.117:0/2370618062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0cd17070 msgr2=0x556e0d0be650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:29.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.356+0000 7f6782a8b740 1 --2- 172.21.15.117:0/2370618062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0cd17070 0x556e0d0be650 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f676c009ee0 tx=0x7f676c010740 comp rx=0 tx=0).stop 2024-09-21T06:05:29.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.357+0000 7f677a2b3640 1 --2- 172.21.15.117:0/1506393177 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f67600b6680 0x7f67600b8b90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x556e0d0bd2d0 tx=0x7f6770009110 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:29.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.357+0000 7f6782a8b740 1 -- 172.21.15.117:0/2370618062 shutdown_connections 2024-09-21T06:05:29.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.357+0000 7f6782a8b740 1 --2- 172.21.15.117:0/2370618062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e0ccf6390 0x556e0d0bf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.357+0000 7f6782a8b740 1 --2- 172.21.15.117:0/2370618062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e0ccee1b0 0x556e0d0beb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.357+0000 7f6782a8b740 1 --2- 172.21.15.117:0/2370618062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0cd17070 0x556e0d0be650 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.357+0000 7f6782a8b740 1 -- 172.21.15.117:0/2370618062 >> 172.21.15.117:0/2370618062 conn(0x556e0cd0b460 msgr2=0x556e0d0bc540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:29.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.357+0000 7f6782a8b740 1 -- 172.21.15.117:0/2370618062 shutdown_connections 2024-09-21T06:05:29.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.357+0000 7f6782a8b740 1 -- 172.21.15.117:0/2370618062 wait complete. 2024-09-21T06:05:29.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.358+0000 7f6782a8b740 1 -- start start 2024-09-21T06:05:29.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.358+0000 7f6782a8b740 1 Processor -- start 2024-09-21T06:05:29.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.358+0000 7f6782a8b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e0ccee1b0 0x556e0d0c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:29.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.358+0000 7f6782a8b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0ccf6390 0x556e0d0ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:29.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.358+0000 7f6782a8b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e0cd17070 0x556e0d0d6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:29.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.358+0000 7f6782a8b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556e0ce85700 con 0x556e0ccf6390 2024-09-21T06:05:29.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.358+0000 7f6782a8b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556e0cfbf110 con 0x556e0cd17070 2024-09-21T06:05:29.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.358+0000 7f6782a8b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556e0d0cce30 con 0x556e0ccee1b0 2024-09-21T06:05:29.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.358+0000 7f6782a8b740 10 client.0 _ll_register_callbacks cb 0x556e0cfd8990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:05:29.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.358+0000 7f6779ab2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0ccf6390 0x556e0d0ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:29.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.358+0000 7f6779ab2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0ccf6390 0x556e0d0ca2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41792/0 (socket says 172.21.15.117:41792) 2024-09-21T06:05:29.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.358+0000 7f6779ab2640 1 -- 172.21.15.117:0/1365050523 learned_addr learned my addr 172.21.15.117:0/1365050523 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:29.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.358+0000 7f677a2b3640 1 --2- 172.21.15.117:0/1365050523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e0ccee1b0 0x556e0d0c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:29.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.358+0000 7f677aab4640 1 --2- 172.21.15.117:0/1365050523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e0cd17070 0x556e0d0d6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:29.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.359+0000 7f6779ab2640 1 -- 172.21.15.117:0/1365050523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e0ccee1b0 msgr2=0x556e0d0c9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:29.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.359+0000 7f6779ab2640 1 --2- 172.21.15.117:0/1365050523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e0ccee1b0 0x556e0d0c9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.359+0000 7f6779ab2640 1 -- 172.21.15.117:0/1365050523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e0cd17070 msgr2=0x556e0d0d6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:29.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.359+0000 7f6779ab2640 1 --2- 172.21.15.117:0/1365050523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e0cd17070 0x556e0d0d6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.359+0000 7f6779ab2640 1 -- 172.21.15.117:0/1365050523 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e0cfecc40 con 0x556e0ccf6390 2024-09-21T06:05:29.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.359+0000 7f6779ab2640 1 --2- 172.21.15.117:0/1365050523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0ccf6390 0x556e0d0ca2b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f6764031440 tx=0x7f6764009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b7dab7e0115ccbe server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:29.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.359+0000 7f675dffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:05:29.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.359+0000 7f675dffb640 1 -- 172.21.15.117:0/1365050523 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764049070 con 0x556e0ccf6390 2024-09-21T06:05:29.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.359+0000 7f675dffb640 1 -- 172.21.15.117:0/1365050523 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f67640f10b0 con 0x556e0ccf6390 2024-09-21T06:05:29.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.359+0000 7f6782a8b740 1 -- 172.21.15.117:0/1365050523 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f676c004d20 con 0x556e0ccf6390 2024-09-21T06:05:29.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.359+0000 7f675dffb640 1 -- 172.21.15.117:0/1365050523 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f67640c9b00 con 0x556e0ccf6390 2024-09-21T06:05:29.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.359+0000 7f6782a8b740 10 client.16977 fetch_fsmap learned FSMap version 29 2024-09-21T06:05:29.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.359+0000 7f6782a8b740 1 -- 172.21.15.117:0/1365050523 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556e0d0d8b70 con 0x556e0ccf6390 2024-09-21T06:05:29.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.359+0000 7f675dffb640 1 -- 172.21.15.117:0/1365050523 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f676c004d20 con 0x556e0ccf6390 2024-09-21T06:05:29.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.359+0000 7f675dffb640 1 -- 172.21.15.117:0/1365050523 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f67640d71f0 con 0x556e0ccf6390 2024-09-21T06:05:29.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.360+0000 7f6782a8b740 10 client.16977 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:05:29.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.360+0000 7f6782a8b740 10 client.16977 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:05:29.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.360+0000 7f6782a8b740 20 client.16977 populate_metadata read hostname 'smithi117' 2024-09-21T06:05:29.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.360+0000 7f6782a8b740 1 --2- 172.21.15.117:0/1365050523 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x556e0ccd59f0 0x556e0d0dc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:29.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.360+0000 7f6782a8b740 4 client.16977 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:05:29.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.360+0000 7f6782a8b740 1 -- 172.21.15.117:0/1365050523 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e0d036da0 con 0x556e0ccd59f0 2024-09-21T06:05:29.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.360+0000 7f677a2b3640 1 --2- 172.21.15.117:0/1365050523 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x556e0ccd59f0 0x556e0d0dc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:29.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.360+0000 7f677a2b3640 1 --2- 172.21.15.117:0/1365050523 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x556e0ccd59f0 0x556e0d0dc0b0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d4858cfbc5cde49 server_cookie=3751f825efade907 in_seq=0 out_seq=0 2024-09-21T06:05:29.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.360+0000 7f675dffb640 10 client.16977 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:29.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.361+0000 7f675dffb640 1 -- 172.21.15.117:0/1365050523 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556e0d036da0 con 0x556e0ccd59f0 2024-09-21T06:05:29.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.361+0000 7f675dffb640 10 client.16977 handle_command_reply: tid=0 2024-09-21T06:05:29.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.361+0000 7f6782a8b740 1 client.16977 shutdown 2024-09-21T06:05:29.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.361+0000 7f6782a8b740 1 -- 172.21.15.117:0/1365050523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0ccf6390 msgr2=0x556e0d0ca2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:29.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.361+0000 7f6782a8b740 1 --2- 172.21.15.117:0/1365050523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0ccf6390 0x556e0d0ca2b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f6764031440 tx=0x7f6764009260 comp rx=0 tx=0).stop 2024-09-21T06:05:29.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.361+0000 7f6782a8b740 1 -- 172.21.15.117:0/1365050523 shutdown_connections 2024-09-21T06:05:29.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.361+0000 7f6782a8b740 1 --2- 172.21.15.117:0/1365050523 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x556e0ccd59f0 0x556e0d0dc0b0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.361+0000 7f6782a8b740 1 --2- 172.21.15.117:0/1365050523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0ccf6390 0x556e0d0ca2b0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.361+0000 7f6782a8b740 1 --2- 172.21.15.117:0/1365050523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e0cd17070 0x556e0d0d6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.361+0000 7f6782a8b740 1 --2- 172.21.15.117:0/1365050523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e0ccee1b0 0x556e0d0c9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.361+0000 7f675dffb640 0 client.16977 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:29.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.361+0000 7f6782a8b740 1 -- 172.21.15.117:0/1365050523 >> 172.21.15.117:0/1365050523 conn(0x556e0cd0b460 msgr2=0x556e0d0c1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:29.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.362+0000 7f6782a8b740 1 -- 172.21.15.117:0/1365050523 shutdown_connections 2024-09-21T06:05:29.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.362+0000 7f6782a8b740 1 -- 172.21.15.117:0/1365050523 wait complete. 2024-09-21T06:05:29.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.362+0000 7f6782a8b740 20 client.16977 trim_cache size 0 max 16384 2024-09-21T06:05:29.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.379+0000 7f6782a8b740 1 Processor -- start 2024-09-21T06:05:29.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.379+0000 7f6782a8b740 1 -- start start 2024-09-21T06:05:29.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.379+0000 7f6782a8b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0ccee1b0 0x556e0d0bfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:29.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.379+0000 7f6782a8b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e0ccf6390 0x556e0d0d6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:29.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.379+0000 7f6782a8b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e0cd17070 0x556e0d0c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:29.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.379+0000 7f6782a8b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f67741af650 con 0x556e0ccee1b0 2024-09-21T06:05:29.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.379+0000 7f6782a8b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f67741b1190 con 0x556e0ccf6390 2024-09-21T06:05:29.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.379+0000 7f6782a8b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f67741afd20 con 0x556e0cd17070 2024-09-21T06:05:29.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.379+0000 7f677a2b3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0ccee1b0 0x556e0d0bfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:29.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.379+0000 7f677a2b3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0ccee1b0 0x556e0d0bfd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41808/0 (socket says 172.21.15.117:41808) 2024-09-21T06:05:29.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.379+0000 7f677a2b3640 1 -- 172.21.15.117:0/2590524883 learned_addr learned my addr 172.21.15.117:0/2590524883 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:29.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.380+0000 7f6779ab2640 1 --2- 172.21.15.117:0/2590524883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e0ccf6390 0x556e0d0d6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:29.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.380+0000 7f677a2b3640 1 -- 172.21.15.117:0/2590524883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e0cd17070 msgr2=0x556e0d0c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:29.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.380+0000 7f677a2b3640 1 --2- 172.21.15.117:0/2590524883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e0cd17070 0x556e0d0c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.380+0000 7f677a2b3640 1 -- 172.21.15.117:0/2590524883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e0ccf6390 msgr2=0x556e0d0d6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:29.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.380+0000 7f677a2b3640 1 --2- 172.21.15.117:0/2590524883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e0ccf6390 0x556e0d0d6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.380+0000 7f677a2b3640 1 -- 172.21.15.117:0/2590524883 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e0d0c0940 con 0x556e0ccee1b0 2024-09-21T06:05:29.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.380+0000 7f6779ab2640 1 --2- 172.21.15.117:0/2590524883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e0ccf6390 0x556e0d0d6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:05:29.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.380+0000 7f677a2b3640 1 --2- 172.21.15.117:0/2590524883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0ccee1b0 0x556e0d0bfd90 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x556e0d0dc830 tx=0x7f6770009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1651c42fcb079e6c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:29.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.380+0000 7f675ffff640 1 -- 172.21.15.117:0/2590524883 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6770023ae0 con 0x556e0ccee1b0 2024-09-21T06:05:29.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.380+0000 7f675ffff640 1 -- 172.21.15.117:0/2590524883 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f677002f070 con 0x556e0ccee1b0 2024-09-21T06:05:29.386 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.380+0000 7f675ffff640 1 -- 172.21.15.117:0/2590524883 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6770014de0 con 0x556e0ccee1b0 2024-09-21T06:05:29.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.381+0000 7f6782a8b740 1 -- 172.21.15.117:0/2590524883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0ccee1b0 msgr2=0x556e0d0bfd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:29.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.381+0000 7f6782a8b740 1 --2- 172.21.15.117:0/2590524883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0ccee1b0 0x556e0d0bfd90 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x556e0d0dc830 tx=0x7f6770009f90 comp rx=0 tx=0).stop 2024-09-21T06:05:29.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.381+0000 7f6782a8b740 1 -- 172.21.15.117:0/2590524883 shutdown_connections 2024-09-21T06:05:29.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.381+0000 7f6782a8b740 1 --2- 172.21.15.117:0/2590524883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e0cd17070 0x556e0d0c6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.381+0000 7f6782a8b740 1 --2- 172.21.15.117:0/2590524883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e0ccf6390 0x556e0d0d6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.381+0000 7f6782a8b740 1 --2- 172.21.15.117:0/2590524883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0ccee1b0 0x556e0d0bfd90 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.381+0000 7f6782a8b740 1 -- 172.21.15.117:0/2590524883 >> 172.21.15.117:0/2590524883 conn(0x556e0cd0b460 msgr2=0x556e0d0bf680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:29.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.381+0000 7f6782a8b740 1 -- 172.21.15.117:0/2590524883 shutdown_connections 2024-09-21T06:05:29.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.381+0000 7f6782a8b740 1 -- 172.21.15.117:0/2590524883 wait complete. 2024-09-21T06:05:29.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.381+0000 7f6782a8b740 1 -- start start 2024-09-21T06:05:29.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.381+0000 7f6782a8b740 1 Processor -- start 2024-09-21T06:05:29.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.382+0000 7f6782a8b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0ccee1b0 0x556e0d0ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:29.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.382+0000 7f6782a8b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e0ccf6390 0x556e0d0cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:29.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.382+0000 7f6782a8b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e0cd17070 0x556e0d145a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:29.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.382+0000 7f6782a8b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f67741b0ac0 con 0x556e0ccee1b0 2024-09-21T06:05:29.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.382+0000 7f6782a8b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f67741b03f0 con 0x556e0cd17070 2024-09-21T06:05:29.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.382+0000 7f6782a8b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f67741ad650 con 0x556e0ccf6390 2024-09-21T06:05:29.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.382+0000 7f677a2b3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0ccee1b0 0x556e0d0ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:29.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.382+0000 7f6782a8b740 10 client.0 _ll_register_callbacks cb 0x7f6764005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:05:29.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.382+0000 7f677a2b3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0ccee1b0 0x556e0d0ce770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41820/0 (socket says 172.21.15.117:41820) 2024-09-21T06:05:29.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.382+0000 7f677a2b3640 1 -- 172.21.15.117:0/1358377642 learned_addr learned my addr 172.21.15.117:0/1358377642 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:29.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.382+0000 7f6779ab2640 1 --2- 172.21.15.117:0/1358377642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e0ccf6390 0x556e0d0cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:29.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.382+0000 7f677aab4640 1 --2- 172.21.15.117:0/1358377642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e0cd17070 0x556e0d145a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:29.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.382+0000 7f677a2b3640 1 -- 172.21.15.117:0/1358377642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e0ccf6390 msgr2=0x556e0d0cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:29.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.382+0000 7f677a2b3640 1 --2- 172.21.15.117:0/1358377642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e0ccf6390 0x556e0d0cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.382+0000 7f677a2b3640 1 -- 172.21.15.117:0/1358377642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e0cd17070 msgr2=0x556e0d145a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:29.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.382+0000 7f677a2b3640 1 --2- 172.21.15.117:0/1358377642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e0cd17070 0x556e0d145a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.382+0000 7f677a2b3640 1 -- 172.21.15.117:0/1358377642 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e0d145fd0 con 0x556e0ccee1b0 2024-09-21T06:05:29.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.382+0000 7f6779ab2640 1 --2- 172.21.15.117:0/1358377642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e0ccf6390 0x556e0d0cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:05:29.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.382+0000 7f677a2b3640 1 --2- 172.21.15.117:0/1358377642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0ccee1b0 0x556e0d0ce770 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f6770012010 tx=0x7f6770023b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf8a7c85dbcdb4c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:29.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.382+0000 7f675dffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:05:29.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.383+0000 7f675dffb640 1 -- 172.21.15.117:0/1358377642 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6770037070 con 0x556e0ccee1b0 2024-09-21T06:05:29.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.383+0000 7f675dffb640 1 -- 172.21.15.117:0/1358377642 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f6770005dd0 con 0x556e0ccee1b0 2024-09-21T06:05:29.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.383+0000 7f6782a8b740 1 -- 172.21.15.117:0/1358377642 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f677002f070 con 0x556e0ccee1b0 2024-09-21T06:05:29.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.383+0000 7f675dffb640 1 -- 172.21.15.117:0/1358377642 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6770023d90 con 0x556e0ccee1b0 2024-09-21T06:05:29.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.383+0000 7f6782a8b740 10 client.16983 fetch_fsmap learned FSMap version 29 2024-09-21T06:05:29.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.383+0000 7f6782a8b740 1 -- 172.21.15.117:0/1358377642 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556e0d147210 con 0x556e0ccee1b0 2024-09-21T06:05:29.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.383+0000 7f675dffb640 1 -- 172.21.15.117:0/1358377642 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f677002f070 con 0x556e0ccee1b0 2024-09-21T06:05:29.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.383+0000 7f675dffb640 1 -- 172.21.15.117:0/1358377642 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f677002b030 con 0x556e0ccee1b0 2024-09-21T06:05:29.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.383+0000 7f6782a8b740 10 client.16983 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:05:29.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.383+0000 7f6782a8b740 10 client.16983 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:05:29.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.383+0000 7f6782a8b740 20 client.16983 populate_metadata read hostname 'smithi117' 2024-09-21T06:05:29.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.383+0000 7f6782a8b740 1 --2- 172.21.15.117:0/1358377642 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x556e0ccaada0 0x556e0d1498e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:29.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.383+0000 7f6782a8b740 4 client.16983 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:05:29.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.383+0000 7f6782a8b740 1 -- 172.21.15.117:0/1358377642 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556e0d0bcaa0 con 0x556e0ccaada0 2024-09-21T06:05:29.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.383+0000 7f6779ab2640 1 --2- 172.21.15.117:0/1358377642 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x556e0ccaada0 0x556e0d1498e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:29.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.384+0000 7f6779ab2640 1 --2- 172.21.15.117:0/1358377642 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x556e0ccaada0 0x556e0d1498e0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=417c51450b8716b7 server_cookie=64e3690e088037c7 in_seq=0 out_seq=0 2024-09-21T06:05:29.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.384+0000 7f675dffb640 10 client.16983 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:29.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.384+0000 7f675dffb640 1 -- 172.21.15.117:0/1358377642 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2078 (crc 0 0 0) 0x556e0d0bcaa0 con 0x556e0ccaada0 2024-09-21T06:05:29.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.384+0000 7f675dffb640 10 client.16983 handle_command_reply: tid=0 2024-09-21T06:05:29.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.384+0000 7f6782a8b740 1 client.16983 shutdown 2024-09-21T06:05:29.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.384+0000 7f6782a8b740 1 -- 172.21.15.117:0/1358377642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0ccee1b0 msgr2=0x556e0d0ce770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:29.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.384+0000 7f6782a8b740 1 --2- 172.21.15.117:0/1358377642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0ccee1b0 0x556e0d0ce770 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f6770012010 tx=0x7f6770023b60 comp rx=0 tx=0).stop 2024-09-21T06:05:29.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.384+0000 7f6782a8b740 1 -- 172.21.15.117:0/1358377642 shutdown_connections 2024-09-21T06:05:29.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.384+0000 7f6782a8b740 1 --2- 172.21.15.117:0/1358377642 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x556e0ccaada0 0x556e0d1498e0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.384+0000 7f6782a8b740 1 --2- 172.21.15.117:0/1358377642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556e0cd17070 0x556e0d145a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.384+0000 7f6782a8b740 1 --2- 172.21.15.117:0/1358377642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556e0ccf6390 0x556e0d0cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.384+0000 7f675dffb640 0 client.16983 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:29.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.384+0000 7f6782a8b740 1 --2- 172.21.15.117:0/1358377642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556e0ccee1b0 0x556e0d0ce770 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.384+0000 7f6782a8b740 1 -- 172.21.15.117:0/1358377642 >> 172.21.15.117:0/1358377642 conn(0x556e0cd0b460 msgr2=0x556e0d0cb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:29.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.384+0000 7f6782a8b740 1 -- 172.21.15.117:0/1358377642 shutdown_connections 2024-09-21T06:05:29.390 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.384+0000 7f6782a8b740 1 -- 172.21.15.117:0/1358377642 wait complete. 2024-09-21T06:05:29.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.384+0000 7f6782a8b740 20 client.16983 trim_cache size 0 max 16384 2024-09-21T06:05:29.391 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T06:05:29.391 INFO:teuthology.orchestra.run.smithi117.stdout: "status": "no active scrubs running", 2024-09-21T06:05:29.391 INFO:teuthology.orchestra.run.smithi117.stdout: "scrubs": {}, 2024-09-21T06:05:29.391 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_stats": { 2024-09-21T06:05:29.391 INFO:teuthology.orchestra.run.smithi117.stdout: "29ff9b7c-49d9-4910-85be-bace7e940cd5": { 2024-09-21T06:05:29.391 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:03:03.775527+0000", 2024-09-21T06:05:29.391 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:05:29.391 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:05:29.391 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:05:29.391 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:05:29.392 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:05:29.392 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:05:29.392 INFO:teuthology.orchestra.run.smithi117.stdout: "0902e33a-ce33-45fb-92d8-2e6fcb8ec8a0": { 2024-09-21T06:05:29.392 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:04:11.365073+0000", 2024-09-21T06:05:29.392 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:05:29.392 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:05:29.392 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:05:29.392 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:05:29.392 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:05:29.392 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:05:29.392 INFO:teuthology.orchestra.run.smithi117.stdout: "d543b0ae-b710-4153-99da-fe5714839c1a": { 2024-09-21T06:05:29.392 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:01:20.574754+0000", 2024-09-21T06:05:29.392 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:05:29.392 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:05:29.392 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:05:29.392 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:05:29.393 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:05:29.393 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:05:29.393 INFO:teuthology.orchestra.run.smithi117.stdout: "ad80d9f1-6aff-46c0-91ab-2c1ab0681039": { 2024-09-21T06:05:29.393 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:59:31.949108+0000", 2024-09-21T06:05:29.393 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:05:29.393 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:05:29.393 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:05:29.393 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:05:29.393 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:05:29.393 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:05:29.393 INFO:teuthology.orchestra.run.smithi117.stdout: "fb2f217d-463b-4c8f-ab19-920b04bb7306": { 2024-09-21T06:05:29.393 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:04:56.308107+0000", 2024-09-21T06:05:29.393 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:05:29.393 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:05:29.393 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:05:29.393 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:05:29.394 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:05:29.394 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:05:29.394 INFO:teuthology.orchestra.run.smithi117.stdout: "8b3fe95a-b76b-4e94-afd0-72fe35e2dd92": { 2024-09-21T06:05:29.394 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:58:01.246076+0000", 2024-09-21T06:05:29.394 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:05:29.394 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:05:29.394 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:05:29.394 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:05:29.394 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:05:29.394 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:05:29.394 INFO:teuthology.orchestra.run.smithi117.stdout: "b30bfff0-0f6b-4e30-8467-ff7db421eba6": { 2024-09-21T06:05:29.394 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:57:14.939082+0000", 2024-09-21T06:05:29.394 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:05:29.394 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:05:29.394 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:05:29.395 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:05:29.395 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:05:29.395 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:05:29.395 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:05:29.395 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T06:05:29.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.385+0000 7f677c517640 1 -- 172.21.15.117:0/1506393177 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f67600b6680 msgr2=0x7f67600b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:29.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.385+0000 7f677c517640 1 --2- 172.21.15.117:0/1506393177 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f67600b6680 0x7f67600b8b90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x556e0d0bd2d0 tx=0x7f6770009110 comp rx=0 tx=0).stop 2024-09-21T06:05:29.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.385+0000 7f677c517640 1 -- 172.21.15.117:0/1506393177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67741094a0 msgr2=0x7f677419ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:29.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.385+0000 7f677c517640 1 --2- 172.21.15.117:0/1506393177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67741094a0 0x7f677419ab00 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f676402a930 tx=0x7f6764004270 comp rx=0 tx=0).stop 2024-09-21T06:05:29.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.385+0000 7f677c517640 1 -- 172.21.15.117:0/1506393177 shutdown_connections 2024-09-21T06:05:29.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.385+0000 7f677c517640 1 --2- 172.21.15.117:0/1506393177 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f67600b6680 0x7f67600b8b90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.385+0000 7f677c517640 1 --2- 172.21.15.117:0/1506393177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67741094a0 0x7f677419ab00 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.385+0000 7f677c517640 1 --2- 172.21.15.117:0/1506393177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6774109df0 0x7f677419fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.385+0000 7f677c517640 1 --2- 172.21.15.117:0/1506393177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6774103b60 0x7f677419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:29.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.385+0000 7f677c517640 1 -- 172.21.15.117:0/1506393177 >> 172.21.15.117:0/1506393177 conn(0x7f677407b850 msgr2=0x7f67741075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:29.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.385+0000 7f677c517640 1 -- 172.21.15.117:0/1506393177 shutdown_connections 2024-09-21T06:05:29.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:29.385+0000 7f677c517640 1 -- 172.21.15.117:0/1506393177 wait complete. 2024-09-21T06:05:30.164 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:29 smithi117 ceph-mon[30947]: pgmap v575: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-21T06:05:30.194 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:05:30.194 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:05:30.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:29 smithi192 ceph-mon[34700]: pgmap v575: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-21T06:05:30.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:29 smithi138 ceph-mon[34391]: pgmap v575: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-21T06:05:30.443 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:05:30.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.923+0000 7fbc6abf4640 1 -- 172.21.15.117:0/1889888093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc641094c0 msgr2=0x7fbc641098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:30.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.923+0000 7fbc6abf4640 1 --2- 172.21.15.117:0/1889888093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc641094c0 0x7fbc641098d0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fbc50009900 tx=0x7fbc5002a410 comp rx=0 tx=0).stop 2024-09-21T06:05:30.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.923+0000 7fbc6abf4640 1 -- 172.21.15.117:0/1889888093 shutdown_connections 2024-09-21T06:05:30.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.924+0000 7fbc6abf4640 1 --2- 172.21.15.117:0/1889888093 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc641094c0 0x7fbc641098d0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:30.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.924+0000 7fbc6abf4640 1 --2- 172.21.15.117:0/1889888093 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc64109e10 0x7fbc64110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:30.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.924+0000 7fbc6abf4640 1 --2- 172.21.15.117:0/1889888093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc64103b80 0x7fbc64103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:30.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.924+0000 7fbc6abf4640 1 -- 172.21.15.117:0/1889888093 >> 172.21.15.117:0/1889888093 conn(0x7fbc640ff5b0 msgr2=0x7fbc64101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:30.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.924+0000 7fbc6abf4640 1 -- 172.21.15.117:0/1889888093 shutdown_connections 2024-09-21T06:05:30.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.924+0000 7fbc6abf4640 1 -- 172.21.15.117:0/1889888093 wait complete. 2024-09-21T06:05:30.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.924+0000 7fbc6abf4640 1 Processor -- start 2024-09-21T06:05:30.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.924+0000 7fbc6abf4640 1 -- start start 2024-09-21T06:05:30.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.925+0000 7fbc6abf4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc64103b80 0x7fbc6419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:30.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.925+0000 7fbc6abf4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc641094c0 0x7fbc6419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:30.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.925+0000 7fbc6abf4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc64109e10 0x7fbc6419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:30.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.925+0000 7fbc6abf4640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbc64113ad0 con 0x7fbc641094c0 2024-09-21T06:05:30.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.925+0000 7fbc6abf4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbc64113950 con 0x7fbc64109e10 2024-09-21T06:05:30.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.925+0000 7fbc6abf4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbc64113c50 con 0x7fbc64103b80 2024-09-21T06:05:30.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.925+0000 7fbc63fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc641094c0 0x7fbc6419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:30.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.925+0000 7fbc63fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc641094c0 0x7fbc6419aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41838/0 (socket says 172.21.15.117:41838) 2024-09-21T06:05:30.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.925+0000 7fbc63fff640 1 -- 172.21.15.117:0/4011002738 learned_addr learned my addr 172.21.15.117:0/4011002738 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:30.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.925+0000 7fbc68990640 1 --2- 172.21.15.117:0/4011002738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc64103b80 0x7fbc6419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:30.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.925+0000 7fbc69191640 1 --2- 172.21.15.117:0/4011002738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc64109e10 0x7fbc6419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:30.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.925+0000 7fbc63fff640 1 -- 172.21.15.117:0/4011002738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc64103b80 msgr2=0x7fbc6419a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:30.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.925+0000 7fbc63fff640 1 --2- 172.21.15.117:0/4011002738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc64103b80 0x7fbc6419a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:30.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.925+0000 7fbc63fff640 1 -- 172.21.15.117:0/4011002738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc64109e10 msgr2=0x7fbc6419fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:30.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.925+0000 7fbc63fff640 1 --2- 172.21.15.117:0/4011002738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc64109e10 0x7fbc6419fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:30.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.925+0000 7fbc63fff640 1 -- 172.21.15.117:0/4011002738 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc641a03b0 con 0x7fbc641094c0 2024-09-21T06:05:30.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.925+0000 7fbc69191640 1 --2- 172.21.15.117:0/4011002738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc64109e10 0x7fbc6419fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:05:30.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.925+0000 7fbc63fff640 1 --2- 172.21.15.117:0/4011002738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc641094c0 0x7fbc6419aaa0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fbc5002a930 tx=0x7fbc50004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:30.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.926+0000 7fbc61ffb640 1 -- 172.21.15.117:0/4011002738 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc5003f030 con 0x7fbc641094c0 2024-09-21T06:05:30.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.926+0000 7fbc6abf4640 1 -- 172.21.15.117:0/4011002738 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc641a0640 con 0x7fbc641094c0 2024-09-21T06:05:30.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.926+0000 7fbc61ffb640 1 -- 172.21.15.117:0/4011002738 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc50003cb0 con 0x7fbc641094c0 2024-09-21T06:05:30.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.926+0000 7fbc61ffb640 1 -- 172.21.15.117:0/4011002738 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc50035070 con 0x7fbc641094c0 2024-09-21T06:05:30.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.926+0000 7fbc6abf4640 1 -- 172.21.15.117:0/4011002738 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc641a0b70 con 0x7fbc641094c0 2024-09-21T06:05:30.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.927+0000 7fbc6abf4640 1 -- 172.21.15.117:0/4011002738 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc64104e00 con 0x7fbc641094c0 2024-09-21T06:05:30.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.929+0000 7fbc61ffb640 1 -- 172.21.15.117:0/4011002738 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fbc50042030 con 0x7fbc641094c0 2024-09-21T06:05:30.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.929+0000 7fbc61ffb640 1 --2- 172.21.15.117:0/4011002738 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbc540b66b0 0x7fbc540b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:30.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.929+0000 7fbc68990640 1 --2- 172.21.15.117:0/4011002738 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbc540b66b0 0x7fbc540b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:30.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.929+0000 7fbc61ffb640 1 -- 172.21.15.117:0/4011002738 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7fbc50108100 con 0x7fbc641094c0 2024-09-21T06:05:30.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.930+0000 7fbc68990640 1 --2- 172.21.15.117:0/4011002738 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbc540b66b0 0x7fbc540b8bc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbc580058c0 tx=0x7fbc5800a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:30.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:30.933+0000 7fbc61ffb640 1 -- 172.21.15.117:0/4011002738 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbc641a0b70 con 0x7fbc641094c0 2024-09-21T06:05:31.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:31.150+0000 7fbc6abf4640 1 -- 172.21.15.117:0/4011002738 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc6419b630 con 0x7fbc641094c0 2024-09-21T06:05:31.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:31.150+0000 7fbc61ffb640 1 -- 172.21.15.117:0/4011002738 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+12001 (secure 0 0 0) 0x7fbc6419b630 con 0x7fbc641094c0 2024-09-21T06:05:31.153 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T06:05:31.153 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":29,"btime":"2024-09-21T06:04:08:952483+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3227057504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3227057504},{"type":"v1","addr":"172.21.15.117:6835","nonce":3227057504}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3303829183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3303829183},{"type":"v1","addr":"172.21.15.117:6837","nonce":3303829183}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T05:55:14.555897+0000","modified":"2024-09-21T06:04:08.952466+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24638,"mds_1":24637,"mds_2":24628,"mds_3":24616,"mds_4":24602},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6839/2314572411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2314572411},{"type":"v1","addr":"172.21.15.117:6839","nonce":2314572411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6841/3378096561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":3378096561},{"type":"v1","addr":"172.21.15.117:6841","nonce":3378096561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/3725629862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3725629862},{"type":"v1","addr":"172.21.15.138:6833","nonce":3725629862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/4276801396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4276801396},{"type":"v1","addr":"172.21.15.138:6835","nonce":4276801396}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.138:6837/2540335374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2540335374},{"type":"v1","addr":"172.21.15.138:6837","nonce":2540335374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24616":{"gid":24616,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.192:6835/3530688860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3530688860},{"type":"v1","addr":"172.21.15.192:6835","nonce":3530688860}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.192:6837/2683860842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2683860842},{"type":"v1","addr":"172.21.15.192:6837","nonce":2683860842}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.192:6839/1727623231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1727623231},{"type":"v1","addr":"172.21.15.192:6839","nonce":1727623231}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/3230372576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":3230372576},{"type":"v1","addr":"172.21.15.138:6839","nonce":3230372576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25840":{"gid":25840,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6833/1596605058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1596605058},{"type":"v1","addr":"172.21.15.192:6833","nonce":1596605058}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24602,24628,24616,24637,24638]},"id":1}]} 2024-09-21T06:05:31.153 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 29 2024-09-21T06:05:31.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:31.153+0000 7fbc6abf4640 1 -- 172.21.15.117:0/4011002738 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbc540b66b0 msgr2=0x7fbc540b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:31.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:31.153+0000 7fbc6abf4640 1 --2- 172.21.15.117:0/4011002738 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbc540b66b0 0x7fbc540b8bc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbc580058c0 tx=0x7fbc5800a2d0 comp rx=0 tx=0).stop 2024-09-21T06:05:31.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:31.153+0000 7fbc6abf4640 1 -- 172.21.15.117:0/4011002738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc641094c0 msgr2=0x7fbc6419aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:31.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:31.153+0000 7fbc6abf4640 1 --2- 172.21.15.117:0/4011002738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc641094c0 0x7fbc6419aaa0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fbc5002a930 tx=0x7fbc50004270 comp rx=0 tx=0).stop 2024-09-21T06:05:31.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:31.153+0000 7fbc6abf4640 1 -- 172.21.15.117:0/4011002738 shutdown_connections 2024-09-21T06:05:31.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:31.153+0000 7fbc6abf4640 1 --2- 172.21.15.117:0/4011002738 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbc540b66b0 0x7fbc540b8bc0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:31.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:31.153+0000 7fbc6abf4640 1 --2- 172.21.15.117:0/4011002738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc641094c0 0x7fbc6419aaa0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:31.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:31.153+0000 7fbc6abf4640 1 --2- 172.21.15.117:0/4011002738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc64109e10 0x7fbc6419fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:31.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:31.153+0000 7fbc6abf4640 1 --2- 172.21.15.117:0/4011002738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbc64103b80 0x7fbc6419a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:31.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:31.153+0000 7fbc6abf4640 1 -- 172.21.15.117:0/4011002738 >> 172.21.15.117:0/4011002738 conn(0x7fbc640ff5b0 msgr2=0x7fbc64106b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:31.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:31.153+0000 7fbc6abf4640 1 -- 172.21.15.117:0/4011002738 shutdown_connections 2024-09-21T06:05:31.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:31.153+0000 7fbc6abf4640 1 -- 172.21.15.117:0/4011002738 wait complete. 2024-09-21T06:05:31.820 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T06:05:32.071 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:05:32.191 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:31 smithi117 ceph-mon[30947]: pgmap v576: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-21T06:05:32.192 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:31 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/4011002738' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:05:32.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:31 smithi192 ceph-mon[34700]: pgmap v576: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-21T06:05:32.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:31 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/4011002738' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:05:32.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:31 smithi138 ceph-mon[34391]: pgmap v576: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-21T06:05:32.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:31 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/4011002738' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:05:32.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.551+0000 7fd6f5e39640 1 -- 172.21.15.117:0/4277287406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f0102a60 msgr2=0x7fd6f0102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:32.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.551+0000 7fd6f5e39640 1 --2- 172.21.15.117:0/4277287406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f0102a60 0x7fd6f0102e70 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0009900 tx=0x7fd6e002a410 comp rx=0 tx=0).stop 2024-09-21T06:05:32.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.551+0000 7fd6f5e39640 1 -- 172.21.15.117:0/4277287406 shutdown_connections 2024-09-21T06:05:32.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.551+0000 7fd6f5e39640 1 --2- 172.21.15.117:0/4277287406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6f0108cf0 0x7fd6f010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:32.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.551+0000 7fd6f5e39640 1 --2- 172.21.15.117:0/4277287406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6f01083a0 0x7fd6f01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:32.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.551+0000 7fd6f5e39640 1 --2- 172.21.15.117:0/4277287406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f0102a60 0x7fd6f0102e70 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:32.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.551+0000 7fd6f5e39640 1 -- 172.21.15.117:0/4277287406 >> 172.21.15.117:0/4277287406 conn(0x7fd6f0079730 msgr2=0x7fd6f0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:32.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.551+0000 7fd6f5e39640 1 -- 172.21.15.117:0/4277287406 shutdown_connections 2024-09-21T06:05:32.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.551+0000 7fd6f5e39640 1 -- 172.21.15.117:0/4277287406 wait complete. 2024-09-21T06:05:32.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.552+0000 7fd6f5e39640 1 Processor -- start 2024-09-21T06:05:32.555 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.552+0000 7fd6f5e39640 1 -- start start 2024-09-21T06:05:32.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.552+0000 7fd6f5e39640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f0102a60 0x7fd6f0104250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:32.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.552+0000 7fd6f5e39640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6f01083a0 0x7fd6f0104790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:32.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.553+0000 7fd6f5e39640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6f0108cf0 0x7fd6f0104cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:32.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.553+0000 7fd6f5e39640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f01129b0 con 0x7fd6f0102a60 2024-09-21T06:05:32.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.553+0000 7fd6f5e39640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f0112830 con 0x7fd6f0108cf0 2024-09-21T06:05:32.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.553+0000 7fd6f5e39640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f0112b30 con 0x7fd6f01083a0 2024-09-21T06:05:32.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.553+0000 7fd6ef7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f0102a60 0x7fd6f0104250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:32.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.553+0000 7fd6ef7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f0102a60 0x7fd6f0104250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41858/0 (socket says 172.21.15.117:41858) 2024-09-21T06:05:32.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.553+0000 7fd6ef7fe640 1 -- 172.21.15.117:0/996698770 learned_addr learned my addr 172.21.15.117:0/996698770 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:32.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.553+0000 7fd6effff640 1 --2- 172.21.15.117:0/996698770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6f0108cf0 0x7fd6f0104cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:32.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.553+0000 7fd6eeffd640 1 --2- 172.21.15.117:0/996698770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6f01083a0 0x7fd6f0104790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:32.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.553+0000 7fd6ef7fe640 1 -- 172.21.15.117:0/996698770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6f01083a0 msgr2=0x7fd6f0104790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:32.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.553+0000 7fd6ef7fe640 1 --2- 172.21.15.117:0/996698770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6f01083a0 0x7fd6f0104790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:32.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.553+0000 7fd6ef7fe640 1 -- 172.21.15.117:0/996698770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6f0108cf0 msgr2=0x7fd6f0104cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:32.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.553+0000 7fd6ef7fe640 1 --2- 172.21.15.117:0/996698770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6f0108cf0 0x7fd6f0104cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:32.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.553+0000 7fd6ef7fe640 1 -- 172.21.15.117:0/996698770 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f01a8710 con 0x7fd6f0102a60 2024-09-21T06:05:32.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.553+0000 7fd6eeffd640 1 --2- 172.21.15.117:0/996698770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6f01083a0 0x7fd6f0104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:05:32.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.553+0000 7fd6ef7fe640 1 --2- 172.21.15.117:0/996698770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f0102a60 0x7fd6f0104250 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0005ab0 tx=0x7fd6e0003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:32.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.553+0000 7fd6ecff9640 1 -- 172.21.15.117:0/996698770 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e003f070 con 0x7fd6f0102a60 2024-09-21T06:05:32.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.553+0000 7fd6f5e39640 1 -- 172.21.15.117:0/996698770 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f01a89a0 con 0x7fd6f0102a60 2024-09-21T06:05:32.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.553+0000 7fd6ecff9640 1 -- 172.21.15.117:0/996698770 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6e0031d60 con 0x7fd6f0102a60 2024-09-21T06:05:32.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.553+0000 7fd6ecff9640 1 -- 172.21.15.117:0/996698770 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0035030 con 0x7fd6f0102a60 2024-09-21T06:05:32.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.553+0000 7fd6f5e39640 1 -- 172.21.15.117:0/996698770 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f01a8ed0 con 0x7fd6f0102a60 2024-09-21T06:05:32.560 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.556+0000 7fd6fc3ad740 1 Processor -- start 2024-09-21T06:05:32.560 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.556+0000 7fd6fc3ad740 1 -- start start 2024-09-21T06:05:32.560 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.556+0000 7fd6fc3ad740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563397ab070 0x556339b526a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:32.560 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.556+0000 7fd6ecff9640 1 -- 172.21.15.117:0/996698770 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fd6e0042070 con 0x7fd6f0102a60 2024-09-21T06:05:32.560 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.556+0000 7fd6fc3ad740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563397821b0 0x556339b52be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:32.560 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.556+0000 7fd6fc3ad740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55633978a390 0x556339b53120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:32.560 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.556+0000 7fd6fc3ad740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556339ac8ea0 con 0x5563397821b0 2024-09-21T06:05:32.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.556+0000 7fd6fc3ad740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556339ab82d0 con 0x5563397ab070 2024-09-21T06:05:32.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.556+0000 7fd6fc3ad740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556339abc0e0 con 0x55633978a390 2024-09-21T06:05:32.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.556+0000 7fd6ef7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563397821b0 0x556339b52be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:32.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.556+0000 7fd6ef7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563397821b0 0x556339b52be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41866/0 (socket says 172.21.15.117:41866) 2024-09-21T06:05:32.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.556+0000 7fd6ef7fe640 1 -- 172.21.15.117:0/4112467706 learned_addr learned my addr 172.21.15.117:0/4112467706 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:32.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.557+0000 7fd6effff640 1 --2- 172.21.15.117:0/4112467706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563397ab070 0x556339b526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:32.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.557+0000 7fd6eeffd640 1 --2- 172.21.15.117:0/4112467706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55633978a390 0x556339b53120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:32.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.557+0000 7fd6ef7fe640 1 -- 172.21.15.117:0/4112467706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55633978a390 msgr2=0x556339b53120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:32.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.557+0000 7fd6ef7fe640 1 --2- 172.21.15.117:0/4112467706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55633978a390 0x556339b53120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:32.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.557+0000 7fd6ef7fe640 1 -- 172.21.15.117:0/4112467706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563397ab070 msgr2=0x556339b526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:32.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.557+0000 7fd6ef7fe640 1 --2- 172.21.15.117:0/4112467706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563397ab070 0x556339b526a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:32.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.557+0000 7fd6ef7fe640 1 -- 172.21.15.117:0/4112467706 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556339816650 con 0x5563397821b0 2024-09-21T06:05:32.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.557+0000 7fd6eeffd640 1 --2- 172.21.15.117:0/4112467706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55633978a390 0x556339b53120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:05:32.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.557+0000 7fd6effff640 1 --2- 172.21.15.117:0/4112467706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563397ab070 0x556339b526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:05:32.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.557+0000 7fd6ef7fe640 1 --2- 172.21.15.117:0/4112467706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563397821b0 0x556339b52be0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fd6e01057c0 tx=0x7fd6e0110540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da7e0593a5568ea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:32.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.557+0000 7fd6e5ffb640 1 -- 172.21.15.117:0/4112467706 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0110850 con 0x5563397821b0 2024-09-21T06:05:32.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.557+0000 7fd6ecff9640 1 --2- 172.21.15.117:0/996698770 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd6dc0b6680 0x7fd6dc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:32.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.557+0000 7fd6e5ffb640 1 -- 172.21.15.117:0/4112467706 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6e0115030 con 0x5563397821b0 2024-09-21T06:05:32.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.557+0000 7fd6e5ffb640 1 -- 172.21.15.117:0/4112467706 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e00cd6c0 con 0x5563397821b0 2024-09-21T06:05:32.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.557+0000 7fd6effff640 1 --2- 172.21.15.117:0/996698770 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd6dc0b6680 0x7fd6dc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:32.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.558+0000 7fd6ecff9640 1 -- 172.21.15.117:0/996698770 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7fd6e01080b0 con 0x7fd6f0102a60 2024-09-21T06:05:32.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.558+0000 7fd6fc3ad740 1 -- 172.21.15.117:0/4112467706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563397821b0 msgr2=0x556339b52be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:32.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.558+0000 7fd6effff640 1 --2- 172.21.15.117:0/996698770 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd6dc0b6680 0x7fd6dc0b8b90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8005fd0 tx=0x7fd6e8009580 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:32.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.558+0000 7fd6fc3ad740 1 --2- 172.21.15.117:0/4112467706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563397821b0 0x556339b52be0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fd6e01057c0 tx=0x7fd6e0110540 comp rx=0 tx=0).stop 2024-09-21T06:05:32.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.558+0000 7fd6fc3ad740 1 -- 172.21.15.117:0/4112467706 shutdown_connections 2024-09-21T06:05:32.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.558+0000 7fd6fc3ad740 1 --2- 172.21.15.117:0/4112467706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563397821b0 0x556339b52be0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:32.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.558+0000 7fd6fc3ad740 1 --2- 172.21.15.117:0/4112467706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55633978a390 0x556339b53120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:32.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.558+0000 7fd6fc3ad740 1 --2- 172.21.15.117:0/4112467706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563397ab070 0x556339b526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:32.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.558+0000 7fd6fc3ad740 1 -- 172.21.15.117:0/4112467706 >> 172.21.15.117:0/4112467706 conn(0x55633979f460 msgr2=0x556339b50590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:32.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.558+0000 7fd6fc3ad740 1 -- 172.21.15.117:0/4112467706 shutdown_connections 2024-09-21T06:05:32.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.558+0000 7fd6fc3ad740 1 -- 172.21.15.117:0/4112467706 wait complete. 2024-09-21T06:05:32.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.559+0000 7fd6fc3ad740 1 -- start start 2024-09-21T06:05:32.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.559+0000 7fd6fc3ad740 1 Processor -- start 2024-09-21T06:05:32.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.559+0000 7fd6fc3ad740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563397821b0 0x556339b5dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:32.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.559+0000 7fd6fc3ad740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55633978a390 0x556339b5e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:32.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.559+0000 7fd6fc3ad740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563397ab070 0x556339b5e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:32.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.559+0000 7fd6fc3ad740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556339919700 con 0x5563397ab070 2024-09-21T06:05:32.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.559+0000 7fd6fc3ad740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556339a53160 con 0x55633978a390 2024-09-21T06:05:32.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.559+0000 7fd6fc3ad740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556339b60c40 con 0x5563397821b0 2024-09-21T06:05:32.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.559+0000 7fd6fc3ad740 10 client.0 _ll_register_callbacks cb 0x556339a6c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:05:32.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.559+0000 7fd6effff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563397ab070 0x556339b5e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:32.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.559+0000 7fd6effff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563397ab070 0x556339b5e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41876/0 (socket says 172.21.15.117:41876) 2024-09-21T06:05:32.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.560+0000 7fd6effff640 1 -- 172.21.15.117:0/1497098119 learned_addr learned my addr 172.21.15.117:0/1497098119 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:32.566 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.560+0000 7fd6ef7fe640 1 --2- 172.21.15.117:0/1497098119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563397821b0 0x556339b5dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:32.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.560+0000 7fd6eeffd640 1 --2- 172.21.15.117:0/1497098119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55633978a390 0x556339b5e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:32.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.560+0000 7fd6effff640 1 -- 172.21.15.117:0/1497098119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563397821b0 msgr2=0x556339b5dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:32.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.560+0000 7fd6effff640 1 --2- 172.21.15.117:0/1497098119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563397821b0 0x556339b5dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:32.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.560+0000 7fd6effff640 1 -- 172.21.15.117:0/1497098119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55633978a390 msgr2=0x556339b5e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:32.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.560+0000 7fd6effff640 1 --2- 172.21.15.117:0/1497098119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55633978a390 0x556339b5e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:32.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.560+0000 7fd6effff640 1 -- 172.21.15.117:0/1497098119 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556339a80c90 con 0x5563397ab070 2024-09-21T06:05:32.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.560+0000 7fd6effff640 1 --2- 172.21.15.117:0/1497098119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563397ab070 0x556339b5e7c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8012930 tx=0x7fd6e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95fbf4cf4946e2f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:32.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.560+0000 7fd6ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:05:32.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.560+0000 7fd6ceffd640 1 -- 172.21.15.117:0/1497098119 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e800ebe0 con 0x5563397ab070 2024-09-21T06:05:32.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.560+0000 7fd6ceffd640 1 -- 172.21.15.117:0/1497098119 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7fd6e800ed80 con 0x5563397ab070 2024-09-21T06:05:32.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.560+0000 7fd6fc3ad740 1 -- 172.21.15.117:0/1497098119 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6e0115030 con 0x5563397ab070 2024-09-21T06:05:32.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.560+0000 7fd6ceffd640 1 -- 172.21.15.117:0/1497098119 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd6e801f420 con 0x5563397ab070 2024-09-21T06:05:32.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.560+0000 7fd6fc3ad740 10 client.17001 fetch_fsmap learned FSMap version 29 2024-09-21T06:05:32.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.560+0000 7fd6fc3ad740 1 -- 172.21.15.117:0/1497098119 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556339b6c890 con 0x5563397ab070 2024-09-21T06:05:32.567 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.561+0000 7fd6ceffd640 1 -- 172.21.15.117:0/1497098119 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6e0115030 con 0x5563397ab070 2024-09-21T06:05:32.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.561+0000 7fd6ceffd640 1 -- 172.21.15.117:0/1497098119 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd6e800f040 con 0x5563397ab070 2024-09-21T06:05:32.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.561+0000 7fd6fc3ad740 10 client.17001 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:05:32.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.561+0000 7fd6fc3ad740 10 client.17001 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T06:05:32.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.561+0000 7fd6fc3ad740 20 client.17001 populate_metadata read hostname 'smithi117' 2024-09-21T06:05:32.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.561+0000 7fd6fc3ad740 1 --2- 172.21.15.117:0/1497098119 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x5563397699f0 0x556339b6fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:32.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.561+0000 7fd6fc3ad740 4 client.17001 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:05:32.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.561+0000 7fd6fc3ad740 1 -- 172.21.15.117:0/1497098119 --> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556339acadf0 con 0x5563397699f0 2024-09-21T06:05:32.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.561+0000 7fd6ef7fe640 1 --2- 172.21.15.117:0/1497098119 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x5563397699f0 0x556339b6fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:32.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.562+0000 7fd6ceffd640 10 client.17001 ms_handle_connect on v2:172.21.15.138:6836/2540335374 2024-09-21T06:05:32.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.562+0000 7fd6ef7fe640 1 --2- 172.21.15.117:0/1497098119 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x5563397699f0 0x556339b6fd00 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d5d35e01f8379e66 server_cookie=ed6fecee689c4e21 in_seq=0 out_seq=0 2024-09-21T06:05:32.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.562+0000 7fd6ceffd640 1 -- 172.21.15.117:0/1497098119 <== mds.4 v2:172.21.15.138:6836/2540335374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556339acadf0 con 0x5563397699f0 2024-09-21T06:05:32.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.562+0000 7fd6ceffd640 10 client.17001 handle_command_reply: tid=0 2024-09-21T06:05:32.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.562+0000 7fd6fc3ad740 1 client.17001 shutdown 2024-09-21T06:05:32.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.563+0000 7fd6fc3ad740 1 -- 172.21.15.117:0/1497098119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563397ab070 msgr2=0x556339b5e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:32.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.563+0000 7fd6fc3ad740 1 --2- 172.21.15.117:0/1497098119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563397ab070 0x556339b5e7c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8012930 tx=0x7fd6e8004a40 comp rx=0 tx=0).stop 2024-09-21T06:05:32.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.563+0000 7fd6fc3ad740 1 -- 172.21.15.117:0/1497098119 shutdown_connections 2024-09-21T06:05:32.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.563+0000 7fd6fc3ad740 1 --2- 172.21.15.117:0/1497098119 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x5563397699f0 0x556339b6fd00 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:32.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.563+0000 7fd6fc3ad740 1 --2- 172.21.15.117:0/1497098119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563397ab070 0x556339b5e7c0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:32.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.563+0000 7fd6fc3ad740 1 --2- 172.21.15.117:0/1497098119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55633978a390 0x556339b5e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:32.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.563+0000 7fd6fc3ad740 1 --2- 172.21.15.117:0/1497098119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563397821b0 0x556339b5dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:32.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.563+0000 7fd6fc3ad740 1 -- 172.21.15.117:0/1497098119 >> 172.21.15.117:0/1497098119 conn(0x55633979f460 msgr2=0x556339b55830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:32.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.563+0000 7fd6ceffd640 0 client.17001 ms_handle_reset on v2:172.21.15.138:6836/2540335374 2024-09-21T06:05:32.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.563+0000 7fd6fc3ad740 1 -- 172.21.15.117:0/1497098119 shutdown_connections 2024-09-21T06:05:32.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.563+0000 7fd6fc3ad740 1 -- 172.21.15.117:0/1497098119 wait complete. 2024-09-21T06:05:32.569 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.563+0000 7fd6fc3ad740 20 client.17001 trim_cache size 0 max 16384 2024-09-21T06:05:32.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.576+0000 7fd6fc3ad740 1 Processor -- start 2024-09-21T06:05:32.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.576+0000 7fd6fc3ad740 1 -- start start 2024-09-21T06:05:32.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.576+0000 7fd6fc3ad740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563397821b0 0x556339b5fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:32.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.576+0000 7fd6fc3ad740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55633978a390 0x556339b53d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:32.579 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.576+0000 7fd6fc3ad740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563397ab070 0x556339b69d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:32.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.576+0000 7fd6fc3ad740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f01b1820 con 0x55633978a390 2024-09-21T06:05:32.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.576+0000 7fd6fc3ad740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f01b3360 con 0x5563397821b0 2024-09-21T06:05:32.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.576+0000 7fd6fc3ad740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f01b1ef0 con 0x5563397ab070 2024-09-21T06:05:32.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.576+0000 7fd6eeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55633978a390 0x556339b53d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:32.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.576+0000 7fd6eeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55633978a390 0x556339b53d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41890/0 (socket says 172.21.15.117:41890) 2024-09-21T06:05:32.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.576+0000 7fd6eeffd640 1 -- 172.21.15.117:0/2786912434 learned_addr learned my addr 172.21.15.117:0/2786912434 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:32.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.576+0000 7fd6effff640 1 --2- 172.21.15.117:0/2786912434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563397ab070 0x556339b69d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:32.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.576+0000 7fd6ef7fe640 1 --2- 172.21.15.117:0/2786912434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563397821b0 0x556339b5fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:32.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.576+0000 7fd6eeffd640 1 -- 172.21.15.117:0/2786912434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563397ab070 msgr2=0x556339b69d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:32.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.576+0000 7fd6eeffd640 1 --2- 172.21.15.117:0/2786912434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563397ab070 0x556339b69d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:32.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.576+0000 7fd6eeffd640 1 -- 172.21.15.117:0/2786912434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563397821b0 msgr2=0x556339b5fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:32.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.576+0000 7fd6eeffd640 1 --2- 172.21.15.117:0/2786912434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563397821b0 0x556339b5fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:32.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.576+0000 7fd6eeffd640 1 -- 172.21.15.117:0/2786912434 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556339b55500 con 0x55633978a390 2024-09-21T06:05:32.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.577+0000 7fd6eeffd640 1 --2- 172.21.15.117:0/2786912434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55633978a390 0x556339b53d90 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fd6d800dd00 tx=0x7fd6d8012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3486a5c015bde2ad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:32.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.577+0000 7fd6e57fa640 1 -- 172.21.15.117:0/2786912434 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d801f070 con 0x55633978a390 2024-09-21T06:05:32.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.577+0000 7fd6e57fa640 1 -- 172.21.15.117:0/2786912434 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6d801a8c0 con 0x55633978a390 2024-09-21T06:05:32.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.577+0000 7fd6e57fa640 1 -- 172.21.15.117:0/2786912434 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d80109b0 con 0x55633978a390 2024-09-21T06:05:32.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.578+0000 7fd6fc3ad740 1 -- 172.21.15.117:0/2786912434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55633978a390 msgr2=0x556339b53d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:32.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.578+0000 7fd6fc3ad740 1 --2- 172.21.15.117:0/2786912434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55633978a390 0x556339b53d90 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fd6d800dd00 tx=0x7fd6d8012820 comp rx=0 tx=0).stop 2024-09-21T06:05:32.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.578+0000 7fd6fc3ad740 1 -- 172.21.15.117:0/2786912434 shutdown_connections 2024-09-21T06:05:32.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.578+0000 7fd6fc3ad740 1 --2- 172.21.15.117:0/2786912434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55633978a390 0x556339b53d90 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:32.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.578+0000 7fd6fc3ad740 1 --2- 172.21.15.117:0/2786912434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563397ab070 0x556339b69d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:32.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.578+0000 7fd6fc3ad740 1 --2- 172.21.15.117:0/2786912434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563397821b0 0x556339b5fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:32.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.578+0000 7fd6fc3ad740 1 -- 172.21.15.117:0/2786912434 >> 172.21.15.117:0/2786912434 conn(0x55633979f460 msgr2=0x556339b53680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:32.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.578+0000 7fd6fc3ad740 1 -- 172.21.15.117:0/2786912434 shutdown_connections 2024-09-21T06:05:32.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.578+0000 7fd6fc3ad740 1 -- 172.21.15.117:0/2786912434 wait complete. 2024-09-21T06:05:32.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.578+0000 7fd6fc3ad740 1 -- start start 2024-09-21T06:05:32.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.578+0000 7fd6fc3ad740 1 Processor -- start 2024-09-21T06:05:32.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.579+0000 7fd6fc3ad740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563397821b0 0x556339b62180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:32.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.579+0000 7fd6fc3ad740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55633978a390 0x556339b62720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:32.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.579+0000 7fd6fc3ad740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563397ab070 0x556339bd9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:32.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.579+0000 7fd6fc3ad740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f01b2c90 con 0x55633978a390 2024-09-21T06:05:32.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.579+0000 7fd6fc3ad740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f01b25c0 con 0x5563397821b0 2024-09-21T06:05:32.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.579+0000 7fd6fc3ad740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f01af820 con 0x5563397ab070 2024-09-21T06:05:32.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.579+0000 7fd6fc3ad740 10 client.0 _ll_register_callbacks cb 0x7fd6e8009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:05:32.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.579+0000 7fd6eeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55633978a390 0x556339b62720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:32.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.579+0000 7fd6eeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55633978a390 0x556339b62720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41898/0 (socket says 172.21.15.117:41898) 2024-09-21T06:05:32.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.579+0000 7fd6eeffd640 1 -- 172.21.15.117:0/714418467 learned_addr learned my addr 172.21.15.117:0/714418467 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:32.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.579+0000 7fd6ef7fe640 1 --2- 172.21.15.117:0/714418467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563397821b0 0x556339b62180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:32.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.579+0000 7fd6effff640 1 --2- 172.21.15.117:0/714418467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563397ab070 0x556339bd9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:32.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.579+0000 7fd6eeffd640 1 -- 172.21.15.117:0/714418467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563397ab070 msgr2=0x556339bd9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:32.587 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.579+0000 7fd6eeffd640 1 --2- 172.21.15.117:0/714418467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563397ab070 0x556339bd9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:32.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.579+0000 7fd6eeffd640 1 -- 172.21.15.117:0/714418467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563397821b0 msgr2=0x556339b62180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:32.588 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.579+0000 7fd6eeffd640 1 --2- 172.21.15.117:0/714418467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563397821b0 0x556339b62180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:32.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.579+0000 7fd6eeffd640 1 -- 172.21.15.117:0/714418467 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556339bd9bb0 con 0x55633978a390 2024-09-21T06:05:32.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.579+0000 7fd6ef7fe640 1 --2- 172.21.15.117:0/714418467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563397821b0 0x556339b62180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:05:32.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.579+0000 7fd6effff640 1 --2- 172.21.15.117:0/714418467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563397ab070 0x556339bd9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:05:32.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.579+0000 7fd6eeffd640 1 --2- 172.21.15.117:0/714418467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55633978a390 0x556339b62720 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8010610 tx=0x7fd6d801a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c5e1d1f9c1c8c79 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:32.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.579+0000 7fd6ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:05:32.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.580+0000 7fd6ceffd640 1 -- 172.21.15.117:0/714418467 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8027020 con 0x55633978a390 2024-09-21T06:05:32.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.580+0000 7fd6ceffd640 1 -- 172.21.15.117:0/714418467 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7fd6d801ab30 con 0x55633978a390 2024-09-21T06:05:32.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.580+0000 7fd6fc3ad740 1 -- 172.21.15.117:0/714418467 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6d801a8c0 con 0x55633978a390 2024-09-21T06:05:32.589 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.580+0000 7fd6ceffd640 1 -- 172.21.15.117:0/714418467 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd6d8032050 con 0x55633978a390 2024-09-21T06:05:32.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.580+0000 7fd6fc3ad740 10 client.17007 fetch_fsmap learned FSMap version 29 2024-09-21T06:05:32.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.580+0000 7fd6fc3ad740 1 -- 172.21.15.117:0/714418467 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556339bdad30 con 0x55633978a390 2024-09-21T06:05:32.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.580+0000 7fd6ceffd640 1 -- 172.21.15.117:0/714418467 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6d801a8c0 con 0x55633978a390 2024-09-21T06:05:32.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.580+0000 7fd6ceffd640 1 -- 172.21.15.117:0/714418467 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd6d8022030 con 0x55633978a390 2024-09-21T06:05:32.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.580+0000 7fd6fc3ad740 10 client.17007 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:05:32.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.580+0000 7fd6fc3ad740 10 client.17007 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T06:05:32.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.580+0000 7fd6fc3ad740 20 client.17007 populate_metadata read hostname 'smithi117' 2024-09-21T06:05:32.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.580+0000 7fd6fc3ad740 1 --2- 172.21.15.117:0/714418467 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x55633973eda0 0x556339bdd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:32.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.580+0000 7fd6fc3ad740 4 client.17007 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:05:32.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.580+0000 7fd6fc3ad740 1 -- 172.21.15.117:0/714418467 --> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556339ac68a0 con 0x55633973eda0 2024-09-21T06:05:32.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.580+0000 7fd6ef7fe640 1 --2- 172.21.15.117:0/714418467 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x55633973eda0 0x556339bdd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:32.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.581+0000 7fd6ef7fe640 1 --2- 172.21.15.117:0/714418467 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x55633973eda0 0x556339bdd3b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7be9c360f08ee463 server_cookie=4ce8732fae23efb7 in_seq=0 out_seq=0 2024-09-21T06:05:32.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.581+0000 7fd6ceffd640 10 client.17007 ms_handle_connect on v2:172.21.15.138:6836/2540335374 2024-09-21T06:05:32.590 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.581+0000 7fd6ceffd640 1 -- 172.21.15.117:0/714418467 <== mds.4 v2:172.21.15.138:6836/2540335374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556339ac68a0 con 0x55633973eda0 2024-09-21T06:05:32.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.581+0000 7fd6ceffd640 10 client.17007 handle_command_reply: tid=0 2024-09-21T06:05:32.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.581+0000 7fd6fc3ad740 1 client.17007 shutdown 2024-09-21T06:05:32.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.581+0000 7fd6fc3ad740 1 -- 172.21.15.117:0/714418467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55633978a390 msgr2=0x556339b62720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:32.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.581+0000 7fd6fc3ad740 1 --2- 172.21.15.117:0/714418467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55633978a390 0x556339b62720 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8010610 tx=0x7fd6d801a6c0 comp rx=0 tx=0).stop 2024-09-21T06:05:32.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.581+0000 7fd6fc3ad740 1 -- 172.21.15.117:0/714418467 shutdown_connections 2024-09-21T06:05:32.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.581+0000 7fd6fc3ad740 1 --2- 172.21.15.117:0/714418467 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x55633973eda0 0x556339bdd3b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:32.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.581+0000 7fd6fc3ad740 1 --2- 172.21.15.117:0/714418467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55633978a390 0x556339b62720 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:32.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.581+0000 7fd6fc3ad740 1 --2- 172.21.15.117:0/714418467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563397ab070 0x556339bd9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:32.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.581+0000 7fd6fc3ad740 1 --2- 172.21.15.117:0/714418467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563397821b0 0x556339b62180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:32.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.581+0000 7fd6fc3ad740 1 -- 172.21.15.117:0/714418467 >> 172.21.15.117:0/714418467 conn(0x55633979f460 msgr2=0x556339b53f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:32.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.581+0000 7fd6ceffd640 0 client.17007 ms_handle_reset on v2:172.21.15.138:6836/2540335374 2024-09-21T06:05:32.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.581+0000 7fd6fc3ad740 1 -- 172.21.15.117:0/714418467 shutdown_connections 2024-09-21T06:05:32.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.581+0000 7fd6fc3ad740 1 -- 172.21.15.117:0/714418467 wait complete. 2024-09-21T06:05:32.591 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.582+0000 7fd6fc3ad740 20 client.17007 trim_cache size 0 max 16384 2024-09-21T06:05:32.592 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:05:32.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.582+0000 7fd6f5e39640 1 -- 172.21.15.117:0/996698770 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd6dc0b6680 msgr2=0x7fd6dc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:32.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.582+0000 7fd6f5e39640 1 --2- 172.21.15.117:0/996698770 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd6dc0b6680 0x7fd6dc0b8b90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8005fd0 tx=0x7fd6e8009580 comp rx=0 tx=0).stop 2024-09-21T06:05:32.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.582+0000 7fd6f5e39640 1 -- 172.21.15.117:0/996698770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f0102a60 msgr2=0x7fd6f0104250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:32.592 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.582+0000 7fd6f5e39640 1 --2- 172.21.15.117:0/996698770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f0102a60 0x7fd6f0104250 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0005ab0 tx=0x7fd6e0003cd0 comp rx=0 tx=0).stop 2024-09-21T06:05:32.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.582+0000 7fd6f5e39640 1 -- 172.21.15.117:0/996698770 shutdown_connections 2024-09-21T06:05:32.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.582+0000 7fd6f5e39640 1 --2- 172.21.15.117:0/996698770 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd6dc0b6680 0x7fd6dc0b8b90 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:32.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.582+0000 7fd6f5e39640 1 --2- 172.21.15.117:0/996698770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6f0108cf0 0x7fd6f0104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:32.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.582+0000 7fd6f5e39640 1 --2- 172.21.15.117:0/996698770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6f01083a0 0x7fd6f0104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:32.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.582+0000 7fd6f5e39640 1 --2- 172.21.15.117:0/996698770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f0102a60 0x7fd6f0104250 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:32.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.582+0000 7fd6f5e39640 1 -- 172.21.15.117:0/996698770 >> 172.21.15.117:0/996698770 conn(0x7fd6f0079730 msgr2=0x7fd6f0106670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:32.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.582+0000 7fd6f5e39640 1 -- 172.21.15.117:0/996698770 shutdown_connections 2024-09-21T06:05:32.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:32.582+0000 7fd6f5e39640 1 -- 172.21.15.117:0/996698770 wait complete. 2024-09-21T06:05:33.277 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T06:05:33.522 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:05:34.013 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:33 smithi117 ceph-mon[30947]: pgmap v577: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-21T06:05:34.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.103+0000 7faed2eb8640 1 -- 172.21.15.117:0/3310727204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faecc1094a0 msgr2=0x7faecc1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:34.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.103+0000 7faed2eb8640 1 --2- 172.21.15.117:0/3310727204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faecc1094a0 0x7faecc1098b0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7faebc009ca0 tx=0x7faebc02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.103+0000 7faed2eb8640 1 -- 172.21.15.117:0/3310727204 shutdown_connections 2024-09-21T06:05:34.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.103+0000 7faed2eb8640 1 --2- 172.21.15.117:0/3310727204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faecc1094a0 0x7faecc1098b0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.103+0000 7faed2eb8640 1 --2- 172.21.15.117:0/3310727204 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faecc109df0 0x7faecc110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.103+0000 7faed2eb8640 1 --2- 172.21.15.117:0/3310727204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faecc103b60 0x7faecc103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.103+0000 7faed2eb8640 1 -- 172.21.15.117:0/3310727204 >> 172.21.15.117:0/3310727204 conn(0x7faecc07b850 msgr2=0x7faecc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:34.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.103+0000 7faed2eb8640 1 -- 172.21.15.117:0/3310727204 shutdown_connections 2024-09-21T06:05:34.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.103+0000 7faed2eb8640 1 -- 172.21.15.117:0/3310727204 wait complete. 2024-09-21T06:05:34.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.104+0000 7faed2eb8640 1 Processor -- start 2024-09-21T06:05:34.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.104+0000 7faed2eb8640 1 -- start start 2024-09-21T06:05:34.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.104+0000 7faed2eb8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faecc103b60 0x7faecc19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:34.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.104+0000 7faed2eb8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faecc1094a0 0x7faecc19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:34.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.104+0000 7faed2eb8640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faecc109df0 0x7faecc19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:34.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.104+0000 7faed2eb8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faecc113ab0 con 0x7faecc103b60 2024-09-21T06:05:34.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.104+0000 7faed2eb8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faecc113930 con 0x7faecc1094a0 2024-09-21T06:05:34.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.104+0000 7faed2eb8640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7faecc113c30 con 0x7faecc109df0 2024-09-21T06:05:34.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.104+0000 7faed0c54640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faecc103b60 0x7faecc19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:34.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.104+0000 7faed0c54640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faecc103b60 0x7faecc19a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41918/0 (socket says 172.21.15.117:41918) 2024-09-21T06:05:34.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.104+0000 7faed0c54640 1 -- 172.21.15.117:0/3031099164 learned_addr learned my addr 172.21.15.117:0/3031099164 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:34.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.105+0000 7faecbfff640 1 --2- 172.21.15.117:0/3031099164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faecc1094a0 0x7faecc19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:34.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.105+0000 7faed1455640 1 --2- 172.21.15.117:0/3031099164 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faecc109df0 0x7faecc19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:34.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.105+0000 7faed0c54640 1 -- 172.21.15.117:0/3031099164 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faecc109df0 msgr2=0x7faecc19fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:34.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.105+0000 7faed0c54640 1 --2- 172.21.15.117:0/3031099164 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faecc109df0 0x7faecc19fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.133 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.105+0000 7faed0c54640 1 -- 172.21.15.117:0/3031099164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faecc1094a0 msgr2=0x7faecc19aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:34.133 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.105+0000 7faed0c54640 1 --2- 172.21.15.117:0/3031099164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faecc1094a0 0x7faecc19aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.133 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.105+0000 7faed0c54640 1 -- 172.21.15.117:0/3031099164 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faecc1a0450 con 0x7faecc103b60 2024-09-21T06:05:34.133 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.105+0000 7faecbfff640 1 --2- 172.21.15.117:0/3031099164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faecc1094a0 0x7faecc19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:05:34.133 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.105+0000 7faed1455640 1 --2- 172.21.15.117:0/3031099164 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faecc109df0 0x7faecc19fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:05:34.133 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.105+0000 7faed0c54640 1 --2- 172.21.15.117:0/3031099164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faecc103b60 0x7faecc19a4e0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7faec400d950 tx=0x7faec400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:34.133 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.105+0000 7faec9ffb640 1 -- 172.21.15.117:0/3031099164 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec4017070 con 0x7faecc103b60 2024-09-21T06:05:34.133 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.105+0000 7faec9ffb640 1 -- 172.21.15.117:0/3031099164 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faec4009c30 con 0x7faecc103b60 2024-09-21T06:05:34.133 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.105+0000 7faed2eb8640 1 -- 172.21.15.117:0/3031099164 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faecc1a0740 con 0x7faecc103b60 2024-09-21T06:05:34.133 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.105+0000 7faec9ffb640 1 -- 172.21.15.117:0/3031099164 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec400f620 con 0x7faecc103b60 2024-09-21T06:05:34.133 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.105+0000 7faed2eb8640 1 -- 172.21.15.117:0/3031099164 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faecc1a6ea0 con 0x7faecc103b60 2024-09-21T06:05:34.133 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.108+0000 7faec9ffb640 1 -- 172.21.15.117:0/3031099164 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7faec4010460 con 0x7faecc103b60 2024-09-21T06:05:34.133 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.108+0000 7faed942c740 1 Processor -- start 2024-09-21T06:05:34.133 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.108+0000 7faed942c740 1 -- start start 2024-09-21T06:05:34.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.108+0000 7faed942c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c92513d070 0x55c9254e46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:34.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.108+0000 7faed942c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9251141b0 0x55c9254e4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:34.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.108+0000 7faed942c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c92511c390 0x55c9254e5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:34.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.108+0000 7faed942c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c92545aea0 con 0x55c92513d070 2024-09-21T06:05:34.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.108+0000 7faed942c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c92544a2d0 con 0x55c92511c390 2024-09-21T06:05:34.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.108+0000 7faed942c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c92544e0e0 con 0x55c9251141b0 2024-09-21T06:05:34.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.108+0000 7faed1455640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c92513d070 0x55c9254e46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:34.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.108+0000 7faec9ffb640 1 --2- 172.21.15.117:0/3031099164 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7faeb80b6680 0x7faeb80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:34.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.108+0000 7faed1455640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c92513d070 0x55c9254e46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41932/0 (socket says 172.21.15.117:41932) 2024-09-21T06:05:34.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.108+0000 7faed1455640 1 -- 172.21.15.117:0/2337723805 learned_addr learned my addr 172.21.15.117:0/2337723805 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:34.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.109+0000 7faecbfff640 1 --2- 172.21.15.117:0/2337723805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c92511c390 0x55c9254e5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:34.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.109+0000 7faed1455640 1 --2- 172.21.15.117:0/3031099164 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7faeb80b6680 0x7faeb80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:34.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.109+0000 7faed0c54640 1 --2- 172.21.15.117:0/2337723805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9251141b0 0x55c9254e4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:34.134 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.109+0000 7faed1455640 1 -- 172.21.15.117:0/2337723805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9251141b0 msgr2=0x55c9254e4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:34.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.109+0000 7faed1455640 1 --2- 172.21.15.117:0/2337723805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9251141b0 0x55c9254e4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.109+0000 7faec9ffb640 1 -- 172.21.15.117:0/3031099164 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7faec40e7450 con 0x7faecc103b60 2024-09-21T06:05:34.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.109+0000 7faed1455640 1 -- 172.21.15.117:0/2337723805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c92511c390 msgr2=0x55c9254e5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:34.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.109+0000 7faed1455640 1 --2- 172.21.15.117:0/2337723805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c92511c390 0x55c9254e5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.109+0000 7faed1455640 1 -- 172.21.15.117:0/2337723805 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9251a8650 con 0x55c92513d070 2024-09-21T06:05:34.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.109+0000 7faecbfff640 1 --2- 172.21.15.117:0/2337723805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c92511c390 0x55c9254e5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:05:34.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.109+0000 7faed0c54640 1 --2- 172.21.15.117:0/2337723805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9251141b0 0x55c9254e4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:05:34.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.109+0000 7faed1455640 1 --2- 172.21.15.117:0/2337723805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c92513d070 0x55c9254e46a0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7faeb4015870 tx=0x7faeb4015d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97191a7e85d2ca99 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:34.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.109+0000 7faec2ffd640 1 -- 172.21.15.117:0/2337723805 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb4022070 con 0x55c92513d070 2024-09-21T06:05:34.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.109+0000 7faec2ffd640 1 -- 172.21.15.117:0/2337723805 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faeb401dcb0 con 0x55c92513d070 2024-09-21T06:05:34.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.109+0000 7faec2ffd640 1 -- 172.21.15.117:0/2337723805 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb400fca0 con 0x55c92513d070 2024-09-21T06:05:34.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.110+0000 7faed942c740 1 -- 172.21.15.117:0/2337723805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c92513d070 msgr2=0x55c9254e46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:34.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.110+0000 7faed1455640 1 --2- 172.21.15.117:0/3031099164 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7faeb80b6680 0x7faeb80b8b90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faeb401f000 tx=0x7faeb401b000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:34.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.110+0000 7faed942c740 1 --2- 172.21.15.117:0/2337723805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c92513d070 0x55c9254e46a0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7faeb4015870 tx=0x7faeb4015d30 comp rx=0 tx=0).stop 2024-09-21T06:05:34.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.110+0000 7faed942c740 1 -- 172.21.15.117:0/2337723805 shutdown_connections 2024-09-21T06:05:34.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.110+0000 7faed942c740 1 --2- 172.21.15.117:0/2337723805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c92511c390 0x55c9254e5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.110+0000 7faed942c740 1 --2- 172.21.15.117:0/2337723805 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9251141b0 0x55c9254e4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.110+0000 7faed942c740 1 --2- 172.21.15.117:0/2337723805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c92513d070 0x55c9254e46a0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.110+0000 7faed942c740 1 -- 172.21.15.117:0/2337723805 >> 172.21.15.117:0/2337723805 conn(0x55c925131460 msgr2=0x55c9254e2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:34.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.110+0000 7faed942c740 1 -- 172.21.15.117:0/2337723805 shutdown_connections 2024-09-21T06:05:34.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.110+0000 7faed942c740 1 -- 172.21.15.117:0/2337723805 wait complete. 2024-09-21T06:05:34.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.111+0000 7faed942c740 1 -- start start 2024-09-21T06:05:34.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.111+0000 7faed942c740 1 Processor -- start 2024-09-21T06:05:34.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.111+0000 7faed942c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9251141b0 0x55c9254efe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:34.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.111+0000 7faed942c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c92511c390 0x55c9254f0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:34.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.111+0000 7faed942c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c92513d070 0x55c9254fc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:34.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.111+0000 7faed942c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c9252ab700 con 0x55c92511c390 2024-09-21T06:05:34.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.111+0000 7faed942c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c9253e5160 con 0x55c9251141b0 2024-09-21T06:05:34.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.111+0000 7faed942c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c9254f2e80 con 0x55c92513d070 2024-09-21T06:05:34.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.111+0000 7faed942c740 10 client.0 _ll_register_callbacks cb 0x55c9253fe9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:05:34.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.111+0000 7faecbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c92511c390 0x55c9254f0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:34.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.111+0000 7faecbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c92511c390 0x55c9254f0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41936/0 (socket says 172.21.15.117:41936) 2024-09-21T06:05:34.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.111+0000 7faecbfff640 1 -- 172.21.15.117:0/1268704943 learned_addr learned my addr 172.21.15.117:0/1268704943 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:34.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.111+0000 7faed0c54640 1 --2- 172.21.15.117:0/1268704943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9251141b0 0x55c9254efe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:34.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.112+0000 7faed1455640 1 --2- 172.21.15.117:0/1268704943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c92513d070 0x55c9254fc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:34.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.112+0000 7faecbfff640 1 -- 172.21.15.117:0/1268704943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c92513d070 msgr2=0x55c9254fc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:34.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.112+0000 7faecbfff640 1 --2- 172.21.15.117:0/1268704943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c92513d070 0x55c9254fc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.112+0000 7faecbfff640 1 -- 172.21.15.117:0/1268704943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9251141b0 msgr2=0x55c9254efe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:34.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.112+0000 7faecbfff640 1 --2- 172.21.15.117:0/1268704943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9251141b0 0x55c9254efe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.112+0000 7faecbfff640 1 -- 172.21.15.117:0/1268704943 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c925412c90 con 0x55c92511c390 2024-09-21T06:05:34.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.112+0000 7faed1455640 1 --2- 172.21.15.117:0/1268704943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c92513d070 0x55c9254fc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:05:34.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.112+0000 7faecbfff640 1 --2- 172.21.15.117:0/1268704943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c92511c390 0x55c9254f0350 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7faebc005e30 tx=0x7faebc02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1257a3e2263259cb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:34.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.112+0000 7faeabfff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:05:34.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.112+0000 7faeabfff640 1 -- 172.21.15.117:0/1268704943 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc0318e0 con 0x55c92511c390 2024-09-21T06:05:34.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.112+0000 7faeabfff640 1 -- 172.21.15.117:0/1268704943 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7faebc003970 con 0x55c92511c390 2024-09-21T06:05:34.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.112+0000 7faed942c740 1 -- 172.21.15.117:0/1268704943 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faeb401dcb0 con 0x55c92511c390 2024-09-21T06:05:34.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.112+0000 7faeabfff640 1 -- 172.21.15.117:0/1268704943 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faebc003b70 con 0x55c92511c390 2024-09-21T06:05:34.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.112+0000 7faed942c740 10 client.17019 fetch_fsmap learned FSMap version 29 2024-09-21T06:05:34.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.112+0000 7faed942c740 1 -- 172.21.15.117:0/1268704943 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c9254feb90 con 0x55c92511c390 2024-09-21T06:05:34.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.112+0000 7faeabfff640 1 -- 172.21.15.117:0/1268704943 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faeb401dcb0 con 0x55c92511c390 2024-09-21T06:05:34.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.113+0000 7faeabfff640 1 -- 172.21.15.117:0/1268704943 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7faebc02f740 con 0x55c92511c390 2024-09-21T06:05:34.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.113+0000 7faed942c740 10 client.17019 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:05:34.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.113+0000 7faed942c740 10 client.17019 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T06:05:34.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.113+0000 7faed942c740 20 client.17019 populate_metadata read hostname 'smithi117' 2024-09-21T06:05:34.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.113+0000 7faed942c740 1 --2- 172.21.15.117:0/1268704943 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55c9250fb9f0 0x55c925502030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:34.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.113+0000 7faed942c740 4 client.17019 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:05:34.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.113+0000 7faed942c740 1 -- 172.21.15.117:0/1268704943 --> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c92545cdf0 con 0x55c9250fb9f0 2024-09-21T06:05:34.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.113+0000 7faed0c54640 1 --2- 172.21.15.117:0/1268704943 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55c9250fb9f0 0x55c925502030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:34.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.114+0000 7faeabfff640 10 client.17019 ms_handle_connect on v2:172.21.15.192:6834/3530688860 2024-09-21T06:05:34.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.114+0000 7faed0c54640 1 --2- 172.21.15.117:0/1268704943 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55c9250fb9f0 0x55c925502030 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2660ed81318e54f4 server_cookie=e1c9670f53f83112 in_seq=0 out_seq=0 2024-09-21T06:05:34.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.114+0000 7faeabfff640 1 -- 172.21.15.117:0/1268704943 <== mds.3 v2:172.21.15.192:6834/3530688860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c92545cdf0 con 0x55c9250fb9f0 2024-09-21T06:05:34.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.114+0000 7faeabfff640 10 client.17019 handle_command_reply: tid=0 2024-09-21T06:05:34.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.114+0000 7faed942c740 1 client.17019 shutdown 2024-09-21T06:05:34.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.114+0000 7faed942c740 1 -- 172.21.15.117:0/1268704943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c92511c390 msgr2=0x55c9254f0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:34.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.114+0000 7faed942c740 1 --2- 172.21.15.117:0/1268704943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c92511c390 0x55c9254f0350 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7faebc005e30 tx=0x7faebc02f380 comp rx=0 tx=0).stop 2024-09-21T06:05:34.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.115+0000 7faed942c740 1 -- 172.21.15.117:0/1268704943 shutdown_connections 2024-09-21T06:05:34.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.115+0000 7faed942c740 1 --2- 172.21.15.117:0/1268704943 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55c9250fb9f0 0x55c925502030 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.115+0000 7faed942c740 1 --2- 172.21.15.117:0/1268704943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c92511c390 0x55c9254f0350 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.115+0000 7faeabfff640 0 client.17019 ms_handle_reset on v2:172.21.15.192:6834/3530688860 2024-09-21T06:05:34.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.115+0000 7faed942c740 1 --2- 172.21.15.117:0/1268704943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c92513d070 0x55c9254fc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.115+0000 7faed942c740 1 --2- 172.21.15.117:0/1268704943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9251141b0 0x55c9254efe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.115+0000 7faed942c740 1 -- 172.21.15.117:0/1268704943 >> 172.21.15.117:0/1268704943 conn(0x55c925131460 msgr2=0x55c9254e7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:34.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.115+0000 7faed942c740 1 -- 172.21.15.117:0/1268704943 shutdown_connections 2024-09-21T06:05:34.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.115+0000 7faed942c740 1 -- 172.21.15.117:0/1268704943 wait complete. 2024-09-21T06:05:34.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.115+0000 7faed942c740 20 client.17019 trim_cache size 0 max 16384 2024-09-21T06:05:34.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.128+0000 7faed942c740 1 Processor -- start 2024-09-21T06:05:34.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.128+0000 7faed942c740 1 -- start start 2024-09-21T06:05:34.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.128+0000 7faed942c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9251141b0 0x55c9254f02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:34.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.128+0000 7faed942c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c92511c390 0x55c9254fc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:34.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.128+0000 7faed942c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c92513d070 0x55c9254e1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:34.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.128+0000 7faed942c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faecc1af6d0 con 0x55c9251141b0 2024-09-21T06:05:34.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.128+0000 7faed942c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faecc1b1210 con 0x55c92513d070 2024-09-21T06:05:34.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.128+0000 7faed942c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7faecc1afda0 con 0x55c92511c390 2024-09-21T06:05:34.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.128+0000 7faed0c54640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9251141b0 0x55c9254f02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:34.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.128+0000 7faed0c54640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9251141b0 0x55c9254f02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41944/0 (socket says 172.21.15.117:41944) 2024-09-21T06:05:34.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.128+0000 7faed0c54640 1 -- 172.21.15.117:0/3586068019 learned_addr learned my addr 172.21.15.117:0/3586068019 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:34.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.128+0000 7faecbfff640 1 --2- 172.21.15.117:0/3586068019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c92511c390 0x55c9254fc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:34.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.128+0000 7faed1455640 1 --2- 172.21.15.117:0/3586068019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c92513d070 0x55c9254e1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:34.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.128+0000 7faed0c54640 1 -- 172.21.15.117:0/3586068019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c92511c390 msgr2=0x55c9254fc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:34.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.128+0000 7faed0c54640 1 --2- 172.21.15.117:0/3586068019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c92511c390 0x55c9254fc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.128+0000 7faed0c54640 1 -- 172.21.15.117:0/3586068019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c92513d070 msgr2=0x55c9254e1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:34.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.128+0000 7faed0c54640 1 --2- 172.21.15.117:0/3586068019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c92513d070 0x55c9254e1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.128+0000 7faed0c54640 1 -- 172.21.15.117:0/3586068019 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9254ee310 con 0x55c9251141b0 2024-09-21T06:05:34.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.129+0000 7faecbfff640 1 --2- 172.21.15.117:0/3586068019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c92511c390 0x55c9254fc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:05:34.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.129+0000 7faed0c54640 1 --2- 172.21.15.117:0/3586068019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9251141b0 0x55c9254f02d0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7faec40a9860 tx=0x7faec4015300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=576d83a5d0076043 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:34.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.129+0000 7faec27fc640 1 -- 172.21.15.117:0/3586068019 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec4020040 con 0x55c9251141b0 2024-09-21T06:05:34.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.129+0000 7faec27fc640 1 -- 172.21.15.117:0/3586068019 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faec40ad990 con 0x55c9251141b0 2024-09-21T06:05:34.141 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.129+0000 7faec27fc640 1 -- 172.21.15.117:0/3586068019 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec40b6cb0 con 0x55c9251141b0 2024-09-21T06:05:34.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.130+0000 7faed942c740 1 -- 172.21.15.117:0/3586068019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9251141b0 msgr2=0x55c9254f02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:34.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.130+0000 7faed942c740 1 --2- 172.21.15.117:0/3586068019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9251141b0 0x55c9254f02d0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7faec40a9860 tx=0x7faec4015300 comp rx=0 tx=0).stop 2024-09-21T06:05:34.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.130+0000 7faed942c740 1 -- 172.21.15.117:0/3586068019 shutdown_connections 2024-09-21T06:05:34.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.130+0000 7faed942c740 1 --2- 172.21.15.117:0/3586068019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c92513d070 0x55c9254e1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.130+0000 7faed942c740 1 --2- 172.21.15.117:0/3586068019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c92511c390 0x55c9254fc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.130+0000 7faed942c740 1 --2- 172.21.15.117:0/3586068019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9251141b0 0x55c9254f02d0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.130+0000 7faed942c740 1 -- 172.21.15.117:0/3586068019 >> 172.21.15.117:0/3586068019 conn(0x55c925131460 msgr2=0x55c9254efbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:34.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.130+0000 7faed942c740 1 -- 172.21.15.117:0/3586068019 shutdown_connections 2024-09-21T06:05:34.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.130+0000 7faed942c740 1 -- 172.21.15.117:0/3586068019 wait complete. 2024-09-21T06:05:34.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.130+0000 7faed942c740 1 -- start start 2024-09-21T06:05:34.145 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.130+0000 7faed942c740 1 Processor -- start 2024-09-21T06:05:34.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.131+0000 7faed942c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9251141b0 0x55c9254f4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:34.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.131+0000 7faed942c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c92511c390 0x55c9254f4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:34.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.131+0000 7faed942c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c92513d070 0x55c9254f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:34.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.131+0000 7faed942c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faecc1b0b40 con 0x55c92513d070 2024-09-21T06:05:34.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.131+0000 7faed942c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faecc1b0470 con 0x55c9251141b0 2024-09-21T06:05:34.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.131+0000 7faed942c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7faecc1ad6d0 con 0x55c92511c390 2024-09-21T06:05:34.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.131+0000 7faed942c740 10 client.0 _ll_register_callbacks cb 0x7faebc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:05:34.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.131+0000 7faed1455640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c92513d070 0x55c9254f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:34.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.131+0000 7faed1455640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c92513d070 0x55c9254f4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41950/0 (socket says 172.21.15.117:41950) 2024-09-21T06:05:34.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.131+0000 7faed1455640 1 -- 172.21.15.117:0/1761182225 learned_addr learned my addr 172.21.15.117:0/1761182225 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:34.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.131+0000 7faed0c54640 1 --2- 172.21.15.117:0/1761182225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9251141b0 0x55c9254f4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:34.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.131+0000 7faecbfff640 1 --2- 172.21.15.117:0/1761182225 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c92511c390 0x55c9254f4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:34.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.131+0000 7faed1455640 1 -- 172.21.15.117:0/1761182225 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c92511c390 msgr2=0x55c9254f4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:34.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.131+0000 7faed1455640 1 --2- 172.21.15.117:0/1761182225 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c92511c390 0x55c9254f4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.146 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.131+0000 7faed1455640 1 -- 172.21.15.117:0/1761182225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9251141b0 msgr2=0x55c9254f4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:34.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.131+0000 7faed1455640 1 --2- 172.21.15.117:0/1761182225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9251141b0 0x55c9254f4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.131+0000 7faed1455640 1 -- 172.21.15.117:0/1761182225 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c92556ba20 con 0x55c92513d070 2024-09-21T06:05:34.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.131+0000 7faed1455640 1 --2- 172.21.15.117:0/1761182225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c92513d070 0x55c9254f4bb0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7faeb400ffd0 tx=0x7faeb40093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6450fec971cc71e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:34.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.131+0000 7faeabfff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:05:34.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.131+0000 7faeabfff640 1 -- 172.21.15.117:0/1761182225 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb4035060 con 0x55c92513d070 2024-09-21T06:05:34.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.131+0000 7faeabfff640 1 -- 172.21.15.117:0/1761182225 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7faeb4005040 con 0x55c92513d070 2024-09-21T06:05:34.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.131+0000 7faed942c740 1 -- 172.21.15.117:0/1761182225 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faec40ad990 con 0x55c92513d070 2024-09-21T06:05:34.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.132+0000 7faeabfff640 1 -- 172.21.15.117:0/1761182225 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faeb4022040 con 0x55c92513d070 2024-09-21T06:05:34.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.132+0000 7faed942c740 10 client.17025 fetch_fsmap learned FSMap version 29 2024-09-21T06:05:34.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.132+0000 7faed942c740 1 -- 172.21.15.117:0/1761182225 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c92556ccd0 con 0x55c92513d070 2024-09-21T06:05:34.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.132+0000 7faeabfff640 1 -- 172.21.15.117:0/1761182225 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faec40ad990 con 0x55c92513d070 2024-09-21T06:05:34.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.132+0000 7faeabfff640 1 -- 172.21.15.117:0/1761182225 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7faeb401db50 con 0x55c92513d070 2024-09-21T06:05:34.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.132+0000 7faed942c740 10 client.17025 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:05:34.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.132+0000 7faed942c740 10 client.17025 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T06:05:34.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.132+0000 7faed942c740 20 client.17025 populate_metadata read hostname 'smithi117' 2024-09-21T06:05:34.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.132+0000 7faed942c740 1 --2- 172.21.15.117:0/1761182225 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55c9250d0da0 0x55c92556f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:34.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.132+0000 7faed942c740 4 client.17025 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:05:34.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.132+0000 7faed942c740 1 -- 172.21.15.117:0/1761182225 --> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c9254588a0 con 0x55c9250d0da0 2024-09-21T06:05:34.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.132+0000 7faed0c54640 1 --2- 172.21.15.117:0/1761182225 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55c9250d0da0 0x55c92556f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:34.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.133+0000 7faed0c54640 1 --2- 172.21.15.117:0/1761182225 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55c9250d0da0 0x55c92556f3b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8eb8de59db3bfa4 server_cookie=3f3cb28656e7328b in_seq=0 out_seq=0 2024-09-21T06:05:34.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.133+0000 7faeabfff640 10 client.17025 ms_handle_connect on v2:172.21.15.192:6834/3530688860 2024-09-21T06:05:34.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.133+0000 7faeabfff640 1 -- 172.21.15.117:0/1761182225 <== mds.3 v2:172.21.15.192:6834/3530688860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c9254588a0 con 0x55c9250d0da0 2024-09-21T06:05:34.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.133+0000 7faeabfff640 10 client.17025 handle_command_reply: tid=0 2024-09-21T06:05:34.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.133+0000 7faed942c740 1 client.17025 shutdown 2024-09-21T06:05:34.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.133+0000 7faed942c740 1 -- 172.21.15.117:0/1761182225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c92513d070 msgr2=0x55c9254f4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:34.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.133+0000 7faed942c740 1 --2- 172.21.15.117:0/1761182225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c92513d070 0x55c9254f4bb0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7faeb400ffd0 tx=0x7faeb40093d0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.133+0000 7faed942c740 1 -- 172.21.15.117:0/1761182225 shutdown_connections 2024-09-21T06:05:34.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.133+0000 7faed942c740 1 --2- 172.21.15.117:0/1761182225 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55c9250d0da0 0x55c92556f3b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.133+0000 7faed942c740 1 --2- 172.21.15.117:0/1761182225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c92513d070 0x55c9254f4bb0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.133+0000 7faed942c740 1 --2- 172.21.15.117:0/1761182225 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c92511c390 0x55c9254f4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.133+0000 7faed942c740 1 --2- 172.21.15.117:0/1761182225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9251141b0 0x55c9254f4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.134+0000 7faed942c740 1 -- 172.21.15.117:0/1761182225 >> 172.21.15.117:0/1761182225 conn(0x55c925131460 msgr2=0x55c9254e1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:34.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.134+0000 7faeabfff640 0 client.17025 ms_handle_reset on v2:172.21.15.192:6834/3530688860 2024-09-21T06:05:34.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.134+0000 7faed942c740 1 -- 172.21.15.117:0/1761182225 shutdown_connections 2024-09-21T06:05:34.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.134+0000 7faed942c740 1 -- 172.21.15.117:0/1761182225 wait complete. 2024-09-21T06:05:34.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.134+0000 7faed942c740 20 client.17025 trim_cache size 0 max 16384 2024-09-21T06:05:34.149 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:05:34.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.134+0000 7faed2eb8640 1 -- 172.21.15.117:0/3031099164 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7faeb80b6680 msgr2=0x7faeb80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:34.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.134+0000 7faed2eb8640 1 --2- 172.21.15.117:0/3031099164 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7faeb80b6680 0x7faeb80b8b90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faeb401f000 tx=0x7faeb401b000 comp rx=0 tx=0).stop 2024-09-21T06:05:34.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.134+0000 7faed2eb8640 1 -- 172.21.15.117:0/3031099164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faecc103b60 msgr2=0x7faecc19a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:34.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.134+0000 7faed2eb8640 1 --2- 172.21.15.117:0/3031099164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faecc103b60 0x7faecc19a4e0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7faec400d950 tx=0x7faec400de10 comp rx=0 tx=0).stop 2024-09-21T06:05:34.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.134+0000 7faed2eb8640 1 -- 172.21.15.117:0/3031099164 shutdown_connections 2024-09-21T06:05:34.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.134+0000 7faed2eb8640 1 --2- 172.21.15.117:0/3031099164 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7faeb80b6680 0x7faeb80b8b90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.134+0000 7faed2eb8640 1 --2- 172.21.15.117:0/3031099164 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faecc109df0 0x7faecc19fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.134+0000 7faed2eb8640 1 --2- 172.21.15.117:0/3031099164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faecc1094a0 0x7faecc19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.134+0000 7faed2eb8640 1 --2- 172.21.15.117:0/3031099164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faecc103b60 0x7faecc19a4e0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:34.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.134+0000 7faed2eb8640 1 -- 172.21.15.117:0/3031099164 >> 172.21.15.117:0/3031099164 conn(0x7faecc07b850 msgr2=0x7faecc107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:34.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.135+0000 7faed2eb8640 1 -- 172.21.15.117:0/3031099164 shutdown_connections 2024-09-21T06:05:34.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:34.135+0000 7faed2eb8640 1 -- 172.21.15.117:0/3031099164 wait complete. 2024-09-21T06:05:34.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:33 smithi192 ceph-mon[34700]: pgmap v577: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-21T06:05:34.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:33 smithi138 ceph-mon[34391]: pgmap v577: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-21T06:05:35.040 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T06:05:35.286 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:05:35.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.797+0000 7f7fc1790640 1 -- 172.21.15.117:0/533865152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fbc1094a0 msgr2=0x7f7fbc1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:35.799 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.797+0000 7f7fc1790640 1 --2- 172.21.15.117:0/533865152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fbc1094a0 0x7f7fbc1098b0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0009900 tx=0x7f7fb002a3f0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.797+0000 7f7fc1790640 1 -- 172.21.15.117:0/533865152 shutdown_connections 2024-09-21T06:05:35.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.797+0000 7f7fc1790640 1 --2- 172.21.15.117:0/533865152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fbc1094a0 0x7f7fbc1098b0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.797+0000 7f7fc1790640 1 --2- 172.21.15.117:0/533865152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fbc109df0 0x7f7fbc110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.797+0000 7f7fc1790640 1 --2- 172.21.15.117:0/533865152 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fbc103b60 0x7f7fbc103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.797+0000 7f7fc1790640 1 -- 172.21.15.117:0/533865152 >> 172.21.15.117:0/533865152 conn(0x7f7fbc07b850 msgr2=0x7f7fbc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:35.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.797+0000 7f7fc1790640 1 -- 172.21.15.117:0/533865152 shutdown_connections 2024-09-21T06:05:35.800 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.797+0000 7f7fc1790640 1 -- 172.21.15.117:0/533865152 wait complete. 2024-09-21T06:05:35.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.798+0000 7f7fc1790640 1 Processor -- start 2024-09-21T06:05:35.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.798+0000 7f7fc1790640 1 -- start start 2024-09-21T06:05:35.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.799+0000 7f7fc1790640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fbc103b60 0x7f7fbc19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:35.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.799+0000 7f7fc1790640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fbc1094a0 0x7f7fbc19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:35.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.799+0000 7f7fc1790640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fbc109df0 0x7f7fbc19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:35.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.799+0000 7f7fc1790640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc113b00 con 0x7f7fbc1094a0 2024-09-21T06:05:35.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.799+0000 7f7fc1790640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc113980 con 0x7f7fbc103b60 2024-09-21T06:05:35.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.799+0000 7f7fc1790640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc113c80 con 0x7f7fbc109df0 2024-09-21T06:05:35.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.799+0000 7f7fba7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fbc1094a0 0x7f7fbc19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:35.802 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.799+0000 7f7fba7fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fbc1094a0 0x7f7fbc19a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53800/0 (socket says 172.21.15.117:53800) 2024-09-21T06:05:35.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.799+0000 7f7fba7fc640 1 -- 172.21.15.117:0/1303921748 learned_addr learned my addr 172.21.15.117:0/1303921748 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:35.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.799+0000 7f7fbaffd640 1 --2- 172.21.15.117:0/1303921748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fbc103b60 0x7f7fbc19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:35.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.799+0000 7f7fbb7fe640 1 --2- 172.21.15.117:0/1303921748 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fbc109df0 0x7f7fbc19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:35.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.799+0000 7f7fba7fc640 1 -- 172.21.15.117:0/1303921748 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fbc109df0 msgr2=0x7f7fbc19fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:35.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.799+0000 7f7fba7fc640 1 --2- 172.21.15.117:0/1303921748 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fbc109df0 0x7f7fbc19fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.799+0000 7f7fba7fc640 1 -- 172.21.15.117:0/1303921748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fbc103b60 msgr2=0x7f7fbc19a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:35.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.799+0000 7f7fba7fc640 1 --2- 172.21.15.117:0/1303921748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fbc103b60 0x7f7fbc19a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.799+0000 7f7fba7fc640 1 -- 172.21.15.117:0/1303921748 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fbc1a0300 con 0x7f7fbc1094a0 2024-09-21T06:05:35.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.799+0000 7f7fba7fc640 1 --2- 172.21.15.117:0/1303921748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fbc1094a0 0x7f7fbc19a9f0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0031440 tx=0x7f7fb0003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:35.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.799+0000 7f7fa3fff640 1 -- 172.21.15.117:0/1303921748 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb003f070 con 0x7f7fbc1094a0 2024-09-21T06:05:35.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.800+0000 7f7fa3fff640 1 -- 172.21.15.117:0/1303921748 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fb0031d60 con 0x7f7fbc1094a0 2024-09-21T06:05:35.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.800+0000 7f7fc1790640 1 -- 172.21.15.117:0/1303921748 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fbc1a0590 con 0x7f7fbc1094a0 2024-09-21T06:05:35.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.800+0000 7f7fa3fff640 1 -- 172.21.15.117:0/1303921748 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb0035030 con 0x7f7fbc1094a0 2024-09-21T06:05:35.803 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.800+0000 7f7fc1790640 1 -- 172.21.15.117:0/1303921748 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fbc1a0a70 con 0x7f7fbc1094a0 2024-09-21T06:05:35.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.802+0000 7f7fc7d04740 1 Processor -- start 2024-09-21T06:05:35.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.802+0000 7f7fa3fff640 1 -- 172.21.15.117:0/1303921748 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f7fb0042070 con 0x7f7fbc1094a0 2024-09-21T06:05:35.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.802+0000 7f7fc7d04740 1 -- start start 2024-09-21T06:05:35.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.802+0000 7f7fc7d04740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c9b551070 0x555c9b8f86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:35.805 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.802+0000 7f7fc7d04740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c9b5281b0 0x555c9b8f8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:35.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.802+0000 7f7fc7d04740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c9b530390 0x555c9b8f9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:35.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.802+0000 7f7fc7d04740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555c9b86eea0 con 0x555c9b5281b0 2024-09-21T06:05:35.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.802+0000 7f7fc7d04740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555c9b85e2d0 con 0x555c9b551070 2024-09-21T06:05:35.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.802+0000 7f7fc7d04740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555c9b8620e0 con 0x555c9b530390 2024-09-21T06:05:35.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.803+0000 7f7fbaffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c9b5281b0 0x555c9b8f8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:35.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.803+0000 7f7fbaffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c9b5281b0 0x555c9b8f8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53804/0 (socket says 172.21.15.117:53804) 2024-09-21T06:05:35.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.803+0000 7f7fbaffd640 1 -- 172.21.15.117:0/4149080224 learned_addr learned my addr 172.21.15.117:0/4149080224 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:35.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.803+0000 7f7fbb7fe640 1 --2- 172.21.15.117:0/4149080224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c9b551070 0x555c9b8f86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:35.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.803+0000 7f7fba7fc640 1 --2- 172.21.15.117:0/4149080224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c9b530390 0x555c9b8f9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:35.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.803+0000 7f7fbaffd640 1 -- 172.21.15.117:0/4149080224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c9b530390 msgr2=0x555c9b8f9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:35.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.803+0000 7f7fbaffd640 1 --2- 172.21.15.117:0/4149080224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c9b530390 0x555c9b8f9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.803+0000 7f7fbaffd640 1 -- 172.21.15.117:0/4149080224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c9b551070 msgr2=0x555c9b8f86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:35.806 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.803+0000 7f7fbaffd640 1 --2- 172.21.15.117:0/4149080224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c9b551070 0x555c9b8f86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.803+0000 7f7fbaffd640 1 -- 172.21.15.117:0/4149080224 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c9b5bc650 con 0x555c9b5281b0 2024-09-21T06:05:35.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.803+0000 7f7fa3fff640 1 --2- 172.21.15.117:0/1303921748 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7fa40b66b0 0x7f7fa40b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:35.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.803+0000 7f7fbaffd640 1 --2- 172.21.15.117:0/4149080224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c9b5281b0 0x555c9b8f8be0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f7fac00ad30 tx=0x7f7fac00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a21872c6ddcf7456 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:35.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.803+0000 7f7fbb7fe640 1 --2- 172.21.15.117:0/1303921748 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7fa40b66b0 0x7f7fa40b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:35.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.803+0000 7f7fa3fff640 1 -- 172.21.15.117:0/1303921748 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f7fb0107f00 con 0x7f7fbc1094a0 2024-09-21T06:05:35.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.803+0000 7f7fa17fa640 1 -- 172.21.15.117:0/4149080224 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fac010840 con 0x555c9b5281b0 2024-09-21T06:05:35.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.803+0000 7f7fa17fa640 1 -- 172.21.15.117:0/4149080224 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fac014070 con 0x555c9b5281b0 2024-09-21T06:05:35.807 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.803+0000 7f7fa17fa640 1 -- 172.21.15.117:0/4149080224 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fac018a30 con 0x555c9b5281b0 2024-09-21T06:05:35.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.804+0000 7f7fc7d04740 1 -- 172.21.15.117:0/4149080224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c9b5281b0 msgr2=0x555c9b8f8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:35.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.804+0000 7f7fbb7fe640 1 --2- 172.21.15.117:0/1303921748 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7fa40b66b0 0x7f7fa40b8bc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4007f80 tx=0x7f7fb4007520 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:35.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.804+0000 7f7fc7d04740 1 --2- 172.21.15.117:0/4149080224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c9b5281b0 0x555c9b8f8be0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f7fac00ad30 tx=0x7f7fac00f790 comp rx=0 tx=0).stop 2024-09-21T06:05:35.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.804+0000 7f7fc7d04740 1 -- 172.21.15.117:0/4149080224 shutdown_connections 2024-09-21T06:05:35.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.804+0000 7f7fc7d04740 1 --2- 172.21.15.117:0/4149080224 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c9b5281b0 0x555c9b8f8be0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.804+0000 7f7fc7d04740 1 --2- 172.21.15.117:0/4149080224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c9b530390 0x555c9b8f9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.804+0000 7f7fc7d04740 1 --2- 172.21.15.117:0/4149080224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c9b551070 0x555c9b8f86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.804+0000 7f7fc7d04740 1 -- 172.21.15.117:0/4149080224 >> 172.21.15.117:0/4149080224 conn(0x555c9b545460 msgr2=0x555c9b8f6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:35.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.805+0000 7f7fc7d04740 1 -- 172.21.15.117:0/4149080224 shutdown_connections 2024-09-21T06:05:35.810 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.805+0000 7f7fc7d04740 1 -- 172.21.15.117:0/4149080224 wait complete. 2024-09-21T06:05:35.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.805+0000 7f7fc7d04740 1 -- start start 2024-09-21T06:05:35.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.805+0000 7f7fc7d04740 1 Processor -- start 2024-09-21T06:05:35.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.805+0000 7f7fc7d04740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c9b5281b0 0x555c9b9080c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:35.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.805+0000 7f7fc7d04740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c9b530390 0x555c9b908600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:35.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.805+0000 7f7fc7d04740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c9b551070 0x555c9b908b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:35.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.805+0000 7f7fc7d04740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555c9b6bf700 con 0x555c9b551070 2024-09-21T06:05:35.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.805+0000 7f7fc7d04740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555c9b7f9160 con 0x555c9b530390 2024-09-21T06:05:35.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.805+0000 7f7fc7d04740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555c9b90b1b0 con 0x555c9b5281b0 2024-09-21T06:05:35.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.805+0000 7f7fc7d04740 10 client.0 _ll_register_callbacks cb 0x555c9b8129e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:05:35.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.805+0000 7f7fbb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c9b551070 0x555c9b908b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:35.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.805+0000 7f7fbb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c9b551070 0x555c9b908b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53820/0 (socket says 172.21.15.117:53820) 2024-09-21T06:05:35.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.805+0000 7f7fbb7fe640 1 -- 172.21.15.117:0/813628905 learned_addr learned my addr 172.21.15.117:0/813628905 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:35.811 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.806+0000 7f7fba7fc640 1 --2- 172.21.15.117:0/813628905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c9b530390 0x555c9b908600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:35.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.806+0000 7f7fbaffd640 1 --2- 172.21.15.117:0/813628905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c9b5281b0 0x555c9b9080c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:35.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.806+0000 7f7fba7fc640 1 -- 172.21.15.117:0/813628905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c9b5281b0 msgr2=0x555c9b9080c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:35.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.806+0000 7f7fba7fc640 1 --2- 172.21.15.117:0/813628905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c9b5281b0 0x555c9b9080c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.806+0000 7f7fba7fc640 1 -- 172.21.15.117:0/813628905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c9b551070 msgr2=0x555c9b908b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:35.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.806+0000 7f7fba7fc640 1 --2- 172.21.15.117:0/813628905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c9b551070 0x555c9b908b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.806+0000 7f7fba7fc640 1 -- 172.21.15.117:0/813628905 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c9b826c90 con 0x555c9b530390 2024-09-21T06:05:35.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.806+0000 7f7fbb7fe640 1 --2- 172.21.15.117:0/813628905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c9b551070 0x555c9b908b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:05:35.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.806+0000 7f7fba7fc640 1 --2- 172.21.15.117:0/813628905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c9b530390 0x555c9b908600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7fb01056a0 tx=0x7f7fb003b920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4979cd539148669a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:35.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.806+0000 7f7f9eff5640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-21T06:05:35.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.806+0000 7f7f9eff5640 1 -- 172.21.15.117:0/813628905 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb0049030 con 0x555c9b530390 2024-09-21T06:05:35.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.806+0000 7f7fc7d04740 1 -- 172.21.15.117:0/813628905 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fac014070 con 0x555c9b530390 2024-09-21T06:05:35.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.806+0000 7f7f9eff5640 1 -- 172.21.15.117:0/813628905 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f7fb003b580 con 0x555c9b530390 2024-09-21T06:05:35.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.806+0000 7f7f9eff5640 1 -- 172.21.15.117:0/813628905 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7fb00ca3f0 con 0x555c9b530390 2024-09-21T06:05:35.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.806+0000 7f7fc7d04740 10 client.26123 fetch_fsmap learned FSMap version 29 2024-09-21T06:05:35.812 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.806+0000 7f7fc7d04740 1 -- 172.21.15.117:0/813628905 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555c9b916ea0 con 0x555c9b530390 2024-09-21T06:05:35.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.807+0000 7f7f9eff5640 1 -- 172.21.15.117:0/813628905 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fac014070 con 0x555c9b530390 2024-09-21T06:05:35.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.807+0000 7f7f9eff5640 1 -- 172.21.15.117:0/813628905 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7fb00b3b70 con 0x555c9b530390 2024-09-21T06:05:35.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.807+0000 7f7fc7d04740 10 client.26123 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:05:35.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.807+0000 7f7fc7d04740 10 client.26123 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:05:35.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.807+0000 7f7fc7d04740 20 client.26123 populate_metadata read hostname 'smithi117' 2024-09-21T06:05:35.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.807+0000 7f7fc7d04740 1 --2- 172.21.15.117:0/813628905 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x555c9b50f9f0 0x555c9b91a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:35.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.807+0000 7f7fc7d04740 4 client.26123 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:05:35.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.807+0000 7f7fc7d04740 1 -- 172.21.15.117:0/813628905 --> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c9b870df0 con 0x555c9b50f9f0 2024-09-21T06:05:35.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.807+0000 7f7fbaffd640 1 --2- 172.21.15.117:0/813628905 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x555c9b50f9f0 0x555c9b91a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:35.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.808+0000 7f7f9eff5640 10 client.26123 ms_handle_connect on v2:172.21.15.192:6836/2683860842 2024-09-21T06:05:35.813 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.808+0000 7f7fbaffd640 1 --2- 172.21.15.117:0/813628905 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x555c9b50f9f0 0x555c9b91a310 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=af17267f75672025 server_cookie=9994e38595797e2b in_seq=0 out_seq=0 2024-09-21T06:05:35.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.808+0000 7f7f9eff5640 1 -- 172.21.15.117:0/813628905 <== mds.2 v2:172.21.15.192:6836/2683860842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555c9b870df0 con 0x555c9b50f9f0 2024-09-21T06:05:35.814 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.808+0000 7f7f9eff5640 10 client.26123 handle_command_reply: tid=0 2024-09-21T06:05:35.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.808+0000 7f7fc7d04740 1 client.26123 shutdown 2024-09-21T06:05:35.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.809+0000 7f7fc7d04740 1 -- 172.21.15.117:0/813628905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c9b530390 msgr2=0x555c9b908600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:35.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.809+0000 7f7fc7d04740 1 --2- 172.21.15.117:0/813628905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c9b530390 0x555c9b908600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7fb01056a0 tx=0x7f7fb003b920 comp rx=0 tx=0).stop 2024-09-21T06:05:35.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.809+0000 7f7fc7d04740 1 -- 172.21.15.117:0/813628905 shutdown_connections 2024-09-21T06:05:35.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.809+0000 7f7fc7d04740 1 --2- 172.21.15.117:0/813628905 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x555c9b50f9f0 0x555c9b91a310 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.809+0000 7f7fc7d04740 1 --2- 172.21.15.117:0/813628905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c9b551070 0x555c9b908b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.809+0000 7f7fc7d04740 1 --2- 172.21.15.117:0/813628905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c9b530390 0x555c9b908600 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.809+0000 7f7f9eff5640 0 client.26123 ms_handle_reset on v2:172.21.15.192:6836/2683860842 2024-09-21T06:05:35.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.809+0000 7f7fc7d04740 1 --2- 172.21.15.117:0/813628905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c9b5281b0 0x555c9b9080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.809+0000 7f7fc7d04740 1 -- 172.21.15.117:0/813628905 >> 172.21.15.117:0/813628905 conn(0x555c9b545460 msgr2=0x555c9b8fb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:35.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.809+0000 7f7fc7d04740 1 -- 172.21.15.117:0/813628905 shutdown_connections 2024-09-21T06:05:35.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.809+0000 7f7fc7d04740 1 -- 172.21.15.117:0/813628905 wait complete. 2024-09-21T06:05:35.815 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.809+0000 7f7fc7d04740 20 client.26123 trim_cache size 0 max 16384 2024-09-21T06:05:35.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.822+0000 7f7fc7d04740 1 Processor -- start 2024-09-21T06:05:35.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.822+0000 7f7fc7d04740 1 -- start start 2024-09-21T06:05:35.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.822+0000 7f7fc7d04740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c9b50f9f0 0x555c9b9052b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:35.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.822+0000 7f7fc7d04740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c9b5281b0 0x555c9b9057f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:35.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.822+0000 7f7fc7d04740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c9b530390 0x555c9b9095e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:35.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.822+0000 7f7fc7d04740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc1ad320 con 0x555c9b50f9f0 2024-09-21T06:05:35.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.822+0000 7f7fc7d04740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc1aee60 con 0x555c9b5281b0 2024-09-21T06:05:35.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.822+0000 7f7fc7d04740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc1ad9f0 con 0x555c9b530390 2024-09-21T06:05:35.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.822+0000 7f7fbaffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c9b50f9f0 0x555c9b9052b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:35.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.822+0000 7f7fbaffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c9b50f9f0 0x555c9b9052b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53824/0 (socket says 172.21.15.117:53824) 2024-09-21T06:05:35.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.822+0000 7f7fbaffd640 1 -- 172.21.15.117:0/759487363 learned_addr learned my addr 172.21.15.117:0/759487363 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:35.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.823+0000 7f7fba7fc640 1 --2- 172.21.15.117:0/759487363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c9b5281b0 0x555c9b9057f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:35.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.823+0000 7f7fbb7fe640 1 --2- 172.21.15.117:0/759487363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c9b530390 0x555c9b9095e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:35.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.823+0000 7f7fbaffd640 1 -- 172.21.15.117:0/759487363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c9b530390 msgr2=0x555c9b9095e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:35.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.823+0000 7f7fbaffd640 1 --2- 172.21.15.117:0/759487363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c9b530390 0x555c9b9095e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.823+0000 7f7fbaffd640 1 -- 172.21.15.117:0/759487363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c9b5281b0 msgr2=0x555c9b9057f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:35.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.823+0000 7f7fbaffd640 1 --2- 172.21.15.117:0/759487363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c9b5281b0 0x555c9b9057f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.823+0000 7f7fbaffd640 1 -- 172.21.15.117:0/759487363 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c9b914410 con 0x555c9b50f9f0 2024-09-21T06:05:35.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.823+0000 7f7fba7fc640 1 --2- 172.21.15.117:0/759487363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c9b5281b0 0x555c9b9057f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:05:35.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.823+0000 7f7fbaffd640 1 --2- 172.21.15.117:0/759487363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c9b50f9f0 0x555c9b9052b0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f7fac00ac50 tx=0x7f7fac0186d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04d827ea0313619 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:35.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.823+0000 7f7fa0ff9640 1 -- 172.21.15.117:0/759487363 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fac005040 con 0x555c9b50f9f0 2024-09-21T06:05:35.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.823+0000 7f7fa0ff9640 1 -- 172.21.15.117:0/759487363 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fac014040 con 0x555c9b50f9f0 2024-09-21T06:05:35.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.823+0000 7f7fa0ff9640 1 -- 172.21.15.117:0/759487363 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fac018bb0 con 0x555c9b50f9f0 2024-09-21T06:05:35.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.824+0000 7f7fc7d04740 1 -- 172.21.15.117:0/759487363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c9b50f9f0 msgr2=0x555c9b9052b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:35.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.824+0000 7f7fc7d04740 1 --2- 172.21.15.117:0/759487363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c9b50f9f0 0x555c9b9052b0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f7fac00ac50 tx=0x7f7fac0186d0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.824+0000 7f7fc7d04740 1 -- 172.21.15.117:0/759487363 shutdown_connections 2024-09-21T06:05:35.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.824+0000 7f7fc7d04740 1 --2- 172.21.15.117:0/759487363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c9b530390 0x555c9b9095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.824+0000 7f7fc7d04740 1 --2- 172.21.15.117:0/759487363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c9b5281b0 0x555c9b9057f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.824+0000 7f7fc7d04740 1 --2- 172.21.15.117:0/759487363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c9b50f9f0 0x555c9b9052b0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.824+0000 7f7fc7d04740 1 -- 172.21.15.117:0/759487363 >> 172.21.15.117:0/759487363 conn(0x555c9b545460 msgr2=0x555c9b906940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:35.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.824+0000 7f7fc7d04740 1 -- 172.21.15.117:0/759487363 shutdown_connections 2024-09-21T06:05:35.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.824+0000 7f7fc7d04740 1 -- 172.21.15.117:0/759487363 wait complete. 2024-09-21T06:05:35.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.824+0000 7f7fc7d04740 1 -- start start 2024-09-21T06:05:35.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.824+0000 7f7fc7d04740 1 Processor -- start 2024-09-21T06:05:35.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.825+0000 7f7fc7d04740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c9b50f9f0 0x555c9b910e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:35.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.825+0000 7f7fc7d04740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c9b5281b0 0x555c9b90c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:35.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.825+0000 7f7fc7d04740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c9b530390 0x555c9b90ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:35.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.825+0000 7f7fc7d04740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc1ae790 con 0x555c9b530390 2024-09-21T06:05:35.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.825+0000 7f7fc7d04740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc1ae0c0 con 0x555c9b50f9f0 2024-09-21T06:05:35.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.825+0000 7f7fc7d04740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc1ab320 con 0x555c9b5281b0 2024-09-21T06:05:35.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.825+0000 7f7fc7d04740 10 client.0 _ll_register_callbacks cb 0x7f7fb0009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:05:35.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.825+0000 7f7fbb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c9b530390 0x555c9b90ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:35.831 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.825+0000 7f7fbb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c9b530390 0x555c9b90ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53830/0 (socket says 172.21.15.117:53830) 2024-09-21T06:05:35.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.825+0000 7f7fbb7fe640 1 -- 172.21.15.117:0/3913194654 learned_addr learned my addr 172.21.15.117:0/3913194654 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:35.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.825+0000 7f7fbaffd640 1 --2- 172.21.15.117:0/3913194654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c9b50f9f0 0x555c9b910e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:35.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.825+0000 7f7fba7fc640 1 --2- 172.21.15.117:0/3913194654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c9b5281b0 0x555c9b90c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:35.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.825+0000 7f7fbb7fe640 1 -- 172.21.15.117:0/3913194654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c9b5281b0 msgr2=0x555c9b90c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:35.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.825+0000 7f7fbb7fe640 1 --2- 172.21.15.117:0/3913194654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c9b5281b0 0x555c9b90c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.825+0000 7f7fbb7fe640 1 -- 172.21.15.117:0/3913194654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c9b50f9f0 msgr2=0x555c9b910e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:35.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.825+0000 7f7fbb7fe640 1 --2- 172.21.15.117:0/3913194654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c9b50f9f0 0x555c9b910e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.825+0000 7f7fbb7fe640 1 -- 172.21.15.117:0/3913194654 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c9b983cd0 con 0x555c9b530390 2024-09-21T06:05:35.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.825+0000 7f7fbaffd640 1 --2- 172.21.15.117:0/3913194654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c9b50f9f0 0x555c9b910e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:05:35.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.825+0000 7f7fbb7fe640 1 --2- 172.21.15.117:0/3913194654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c9b530390 0x555c9b90ca60 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4007d60 tx=0x7f7fb4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89138eb2a1aeef3f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:35.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.825+0000 7f7f9eff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:05:35.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.826+0000 7f7f9eff5640 1 -- 172.21.15.117:0/3913194654 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4004750 con 0x555c9b530390 2024-09-21T06:05:35.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.826+0000 7f7f9eff5640 1 -- 172.21.15.117:0/3913194654 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f7fb4016030 con 0x555c9b530390 2024-09-21T06:05:35.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.826+0000 7f7fc7d04740 1 -- 172.21.15.117:0/3913194654 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fac014040 con 0x555c9b530390 2024-09-21T06:05:35.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.826+0000 7f7f9eff5640 1 -- 172.21.15.117:0/3913194654 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7fb4004b20 con 0x555c9b530390 2024-09-21T06:05:35.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.826+0000 7f7fc7d04740 10 client.17043 fetch_fsmap learned FSMap version 29 2024-09-21T06:05:35.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.826+0000 7f7fc7d04740 1 -- 172.21.15.117:0/3913194654 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555c9b984d90 con 0x555c9b530390 2024-09-21T06:05:35.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.826+0000 7f7f9eff5640 1 -- 172.21.15.117:0/3913194654 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fac014040 con 0x555c9b530390 2024-09-21T06:05:35.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.826+0000 7f7f9eff5640 1 -- 172.21.15.117:0/3913194654 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7fb401b400 con 0x555c9b530390 2024-09-21T06:05:35.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.826+0000 7f7fc7d04740 10 client.17043 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:05:35.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.826+0000 7f7fc7d04740 10 client.17043 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:05:35.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.826+0000 7f7fc7d04740 20 client.17043 populate_metadata read hostname 'smithi117' 2024-09-21T06:05:35.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.826+0000 7f7fc7d04740 1 --2- 172.21.15.117:0/3913194654 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x555c9b551070 0x555c9b987600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:35.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.826+0000 7f7fc7d04740 4 client.17043 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:05:35.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.826+0000 7f7fc7d04740 1 -- 172.21.15.117:0/3913194654 --> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555c9b86c8a0 con 0x555c9b551070 2024-09-21T06:05:35.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.827+0000 7f7fbaffd640 1 --2- 172.21.15.117:0/3913194654 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x555c9b551070 0x555c9b987600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:35.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.827+0000 7f7f9eff5640 10 client.17043 ms_handle_connect on v2:172.21.15.192:6836/2683860842 2024-09-21T06:05:35.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.827+0000 7f7fbaffd640 1 --2- 172.21.15.117:0/3913194654 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x555c9b551070 0x555c9b987600 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f95a7c72e1daa427 server_cookie=f30ff802ee8b0553 in_seq=0 out_seq=0 2024-09-21T06:05:35.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.827+0000 7f7f9eff5640 1 -- 172.21.15.117:0/3913194654 <== mds.2 v2:172.21.15.192:6836/2683860842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555c9b86c8a0 con 0x555c9b551070 2024-09-21T06:05:35.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.827+0000 7f7f9eff5640 10 client.17043 handle_command_reply: tid=0 2024-09-21T06:05:35.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.827+0000 7f7fc7d04740 1 client.17043 shutdown 2024-09-21T06:05:35.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.827+0000 7f7fc7d04740 1 -- 172.21.15.117:0/3913194654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c9b530390 msgr2=0x555c9b90ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:35.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.827+0000 7f7fc7d04740 1 --2- 172.21.15.117:0/3913194654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c9b530390 0x555c9b90ca60 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4007d60 tx=0x7f7fb4004ab0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.827+0000 7f7fc7d04740 1 -- 172.21.15.117:0/3913194654 shutdown_connections 2024-09-21T06:05:35.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.827+0000 7f7fc7d04740 1 --2- 172.21.15.117:0/3913194654 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x555c9b551070 0x555c9b987600 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.828+0000 7f7fc7d04740 1 --2- 172.21.15.117:0/3913194654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c9b530390 0x555c9b90ca60 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.828+0000 7f7f9eff5640 0 client.17043 ms_handle_reset on v2:172.21.15.192:6836/2683860842 2024-09-21T06:05:35.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.828+0000 7f7fc7d04740 1 --2- 172.21.15.117:0/3913194654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c9b5281b0 0x555c9b90c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.828+0000 7f7fc7d04740 1 --2- 172.21.15.117:0/3913194654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555c9b50f9f0 0x555c9b910e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.828+0000 7f7fc7d04740 1 -- 172.21.15.117:0/3913194654 >> 172.21.15.117:0/3913194654 conn(0x555c9b545460 msgr2=0x555c9b8f96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:35.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.828+0000 7f7fc7d04740 1 -- 172.21.15.117:0/3913194654 shutdown_connections 2024-09-21T06:05:35.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.828+0000 7f7fc7d04740 1 -- 172.21.15.117:0/3913194654 wait complete. 2024-09-21T06:05:35.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.828+0000 7f7fc7d04740 20 client.17043 trim_cache size 0 max 16384 2024-09-21T06:05:35.835 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:05:35.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.828+0000 7f7fc1790640 1 -- 172.21.15.117:0/1303921748 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7fa40b66b0 msgr2=0x7f7fa40b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:35.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.828+0000 7f7fc1790640 1 --2- 172.21.15.117:0/1303921748 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7fa40b66b0 0x7f7fa40b8bc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4007f80 tx=0x7f7fb4007520 comp rx=0 tx=0).stop 2024-09-21T06:05:35.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.828+0000 7f7fc1790640 1 -- 172.21.15.117:0/1303921748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fbc1094a0 msgr2=0x7f7fbc19a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:35.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.828+0000 7f7fc1790640 1 --2- 172.21.15.117:0/1303921748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fbc1094a0 0x7f7fbc19a9f0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0031440 tx=0x7f7fb0003d50 comp rx=0 tx=0).stop 2024-09-21T06:05:35.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.828+0000 7f7fc1790640 1 -- 172.21.15.117:0/1303921748 shutdown_connections 2024-09-21T06:05:35.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.828+0000 7f7fc1790640 1 --2- 172.21.15.117:0/1303921748 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f7fa40b66b0 0x7f7fa40b8bc0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.828+0000 7f7fc1790640 1 --2- 172.21.15.117:0/1303921748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fbc1094a0 0x7f7fbc19a9f0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.828+0000 7f7fc1790640 1 --2- 172.21.15.117:0/1303921748 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fbc109df0 0x7f7fbc19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.828+0000 7f7fc1790640 1 --2- 172.21.15.117:0/1303921748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fbc103b60 0x7f7fbc19a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:35.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.828+0000 7f7fc1790640 1 -- 172.21.15.117:0/1303921748 >> 172.21.15.117:0/1303921748 conn(0x7f7fbc07b850 msgr2=0x7f7fbc1074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:35.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.828+0000 7f7fc1790640 1 -- 172.21.15.117:0/1303921748 shutdown_connections 2024-09-21T06:05:35.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:35.828+0000 7f7fc1790640 1 -- 172.21.15.117:0/1303921748 wait complete. 2024-09-21T06:05:36.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:35 smithi117 ceph-mon[30947]: pgmap v578: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T06:05:36.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:35 smithi192 ceph-mon[34700]: pgmap v578: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T06:05:36.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:35 smithi138 ceph-mon[34391]: pgmap v578: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T06:05:36.555 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T06:05:36.813 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:05:37.207 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:36 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:05:37.208 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:36 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:37.208 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:36 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:05:37.208 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:36 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:05:37.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:36 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:05:37.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:36 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:37.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:36 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:05:37.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:36 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:05:37.299 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.297+0000 7fbb9d160640 1 -- 172.21.15.117:0/1113147024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb981052f0 msgr2=0x7fbb9810d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:37.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.297+0000 7fbb9d160640 1 --2- 172.21.15.117:0/1113147024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb981052f0 0x7fbb9810d930 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fbb80009900 tx=0x7fbb8002a410 comp rx=0 tx=0).stop 2024-09-21T06:05:37.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.298+0000 7fbb9d160640 1 -- 172.21.15.117:0/1113147024 shutdown_connections 2024-09-21T06:05:37.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.298+0000 7fbb9d160640 1 --2- 172.21.15.117:0/1113147024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb981052f0 0x7fbb9810d930 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:37.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.298+0000 7fbb9d160640 1 --2- 172.21.15.117:0/1113147024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb9810de70 0x7fbb98110380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:37.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.298+0000 7fbb9d160640 1 --2- 172.21.15.117:0/1113147024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb981048c0 0x7fbb98104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:37.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.298+0000 7fbb9d160640 1 -- 172.21.15.117:0/1113147024 >> 172.21.15.117:0/1113147024 conn(0x7fbb98079770 msgr2=0x7fbb98079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:37.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.298+0000 7fbb9d160640 1 -- 172.21.15.117:0/1113147024 shutdown_connections 2024-09-21T06:05:37.300 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.298+0000 7fbb9d160640 1 -- 172.21.15.117:0/1113147024 wait complete. 2024-09-21T06:05:37.302 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.299+0000 7fbb9d160640 1 Processor -- start 2024-09-21T06:05:37.302 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.299+0000 7fbb9d160640 1 -- start start 2024-09-21T06:05:37.302 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.299+0000 7fbb9d160640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb981048c0 0x7fbb9819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:37.302 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.299+0000 7fbb9d160640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb981052f0 0x7fbb9819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:37.302 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.299+0000 7fbb9d160640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb9810de70 0x7fbb9819fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:37.302 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.299+0000 7fbb9d160640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbb98113af0 con 0x7fbb981048c0 2024-09-21T06:05:37.302 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.299+0000 7fbb9d160640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbb98113970 con 0x7fbb981052f0 2024-09-21T06:05:37.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.299+0000 7fbb9d160640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbb98113c70 con 0x7fbb9810de70 2024-09-21T06:05:37.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.299+0000 7fbb96d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb981048c0 0x7fbb9819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:37.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.299+0000 7fbb96d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb981048c0 0x7fbb9819a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53860/0 (socket says 172.21.15.117:53860) 2024-09-21T06:05:37.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.299+0000 7fbb96d9d640 1 -- 172.21.15.117:0/2688885071 learned_addr learned my addr 172.21.15.117:0/2688885071 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:37.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.300+0000 7fbb9659c640 1 --2- 172.21.15.117:0/2688885071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb981052f0 0x7fbb9819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:37.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.300+0000 7fbb9759e640 1 --2- 172.21.15.117:0/2688885071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb9810de70 0x7fbb9819fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:37.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.300+0000 7fbb96d9d640 1 -- 172.21.15.117:0/2688885071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb9810de70 msgr2=0x7fbb9819fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:37.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.300+0000 7fbb96d9d640 1 --2- 172.21.15.117:0/2688885071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb9810de70 0x7fbb9819fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:37.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.300+0000 7fbb96d9d640 1 -- 172.21.15.117:0/2688885071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb981052f0 msgr2=0x7fbb9819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:37.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.300+0000 7fbb96d9d640 1 --2- 172.21.15.117:0/2688885071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb981052f0 0x7fbb9819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:37.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.300+0000 7fbb96d9d640 1 -- 172.21.15.117:0/2688885071 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb981a0510 con 0x7fbb981048c0 2024-09-21T06:05:37.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.300+0000 7fbb96d9d640 1 --2- 172.21.15.117:0/2688885071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb981048c0 0x7fbb9819a5a0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fbb9000cce0 tx=0x7fbb9000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:37.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.300+0000 7fbb8ffff640 1 -- 172.21.15.117:0/2688885071 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb90004b20 con 0x7fbb981048c0 2024-09-21T06:05:37.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.300+0000 7fbb9d160640 1 -- 172.21.15.117:0/2688885071 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb981a0800 con 0x7fbb981048c0 2024-09-21T06:05:37.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.300+0000 7fbb8ffff640 1 -- 172.21.15.117:0/2688885071 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb90004cc0 con 0x7fbb981048c0 2024-09-21T06:05:37.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.300+0000 7fbb8ffff640 1 -- 172.21.15.117:0/2688885071 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9000f6b0 con 0x7fbb981048c0 2024-09-21T06:05:37.304 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.300+0000 7fbb9d160640 1 -- 172.21.15.117:0/2688885071 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb981a6f60 con 0x7fbb981048c0 2024-09-21T06:05:37.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.303+0000 7fbba36d4740 1 Processor -- start 2024-09-21T06:05:37.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.303+0000 7fbba36d4740 1 -- start start 2024-09-21T06:05:37.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.303+0000 7fbb8ffff640 1 -- 172.21.15.117:0/2688885071 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fbb9000f910 con 0x7fbb981048c0 2024-09-21T06:05:37.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.303+0000 7fbba36d4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ad92aa070 0x556ad96512c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:37.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.303+0000 7fbba36d4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556ad92811b0 0x556ad9651800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:37.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.304+0000 7fbba36d4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ad9289390 0x556ad9651d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:37.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.304+0000 7fbba36d4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556ad95c7ea0 con 0x556ad9289390 2024-09-21T06:05:37.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.304+0000 7fbba36d4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556ad95b72d0 con 0x556ad92811b0 2024-09-21T06:05:37.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.304+0000 7fbba36d4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556ad95bb0e0 con 0x556ad92aa070 2024-09-21T06:05:37.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.304+0000 7fbb9659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ad9289390 0x556ad9651d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:37.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.304+0000 7fbb9659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ad9289390 0x556ad9651d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53866/0 (socket says 172.21.15.117:53866) 2024-09-21T06:05:37.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.304+0000 7fbb9659c640 1 -- 172.21.15.117:0/1940545062 learned_addr learned my addr 172.21.15.117:0/1940545062 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:37.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.304+0000 7fbb9759e640 1 --2- 172.21.15.117:0/1940545062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ad92aa070 0x556ad96512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:37.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.304+0000 7fbb96d9d640 1 --2- 172.21.15.117:0/1940545062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556ad92811b0 0x556ad9651800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:37.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.304+0000 7fbb9659c640 1 -- 172.21.15.117:0/1940545062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ad92aa070 msgr2=0x556ad96512c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:37.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.304+0000 7fbb9659c640 1 --2- 172.21.15.117:0/1940545062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ad92aa070 0x556ad96512c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:37.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.304+0000 7fbb9659c640 1 -- 172.21.15.117:0/1940545062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556ad92811b0 msgr2=0x556ad9651800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:37.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.304+0000 7fbb9659c640 1 --2- 172.21.15.117:0/1940545062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556ad92811b0 0x556ad9651800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:37.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.304+0000 7fbb9659c640 1 -- 172.21.15.117:0/1940545062 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ad9315650 con 0x556ad9289390 2024-09-21T06:05:37.307 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.304+0000 7fbb8ffff640 1 --2- 172.21.15.117:0/2688885071 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbb840b6680 0x7fbb840b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:37.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.304+0000 7fbb9759e640 1 --2- 172.21.15.117:0/2688885071 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbb840b6680 0x7fbb840b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:37.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.304+0000 7fbb9659c640 1 --2- 172.21.15.117:0/1940545062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ad9289390 0x556ad9651d40 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fbb80009900 tx=0x7fbb80000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8153b6c138e3b3b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:37.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.304+0000 7fbb8ffff640 1 -- 172.21.15.117:0/2688885071 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7fbb900e7760 con 0x7fbb981048c0 2024-09-21T06:05:37.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.305+0000 7fbb8d7fa640 1 -- 172.21.15.117:0/1940545062 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb80040070 con 0x556ad9289390 2024-09-21T06:05:37.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.305+0000 7fbb8d7fa640 1 -- 172.21.15.117:0/1940545062 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb80005db0 con 0x556ad9289390 2024-09-21T06:05:37.308 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.305+0000 7fbb8d7fa640 1 -- 172.21.15.117:0/1940545062 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb80035040 con 0x556ad9289390 2024-09-21T06:05:37.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.306+0000 7fbba36d4740 1 -- 172.21.15.117:0/1940545062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ad9289390 msgr2=0x556ad9651d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:37.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.306+0000 7fbb9759e640 1 --2- 172.21.15.117:0/2688885071 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbb840b6680 0x7fbb840b8b90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbb88005d80 tx=0x7fbb8800b420 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:37.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.306+0000 7fbba36d4740 1 --2- 172.21.15.117:0/1940545062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ad9289390 0x556ad9651d40 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fbb80009900 tx=0x7fbb80000bc0 comp rx=0 tx=0).stop 2024-09-21T06:05:37.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.306+0000 7fbba36d4740 1 -- 172.21.15.117:0/1940545062 shutdown_connections 2024-09-21T06:05:37.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.306+0000 7fbba36d4740 1 --2- 172.21.15.117:0/1940545062 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ad9289390 0x556ad9651d40 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:37.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.306+0000 7fbba36d4740 1 --2- 172.21.15.117:0/1940545062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556ad92811b0 0x556ad9651800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:37.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.306+0000 7fbba36d4740 1 --2- 172.21.15.117:0/1940545062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ad92aa070 0x556ad96512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:37.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.306+0000 7fbba36d4740 1 -- 172.21.15.117:0/1940545062 >> 172.21.15.117:0/1940545062 conn(0x556ad929e460 msgr2=0x556ad964f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:37.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.306+0000 7fbba36d4740 1 -- 172.21.15.117:0/1940545062 shutdown_connections 2024-09-21T06:05:37.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.306+0000 7fbba36d4740 1 -- 172.21.15.117:0/1940545062 wait complete. 2024-09-21T06:05:37.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.306+0000 7fbba36d4740 1 -- start start 2024-09-21T06:05:37.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.306+0000 7fbba36d4740 1 Processor -- start 2024-09-21T06:05:37.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.306+0000 7fbba36d4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ad92811b0 0x556ad9660da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:37.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.307+0000 7fbba36d4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556ad9289390 0x556ad96612e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:37.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.307+0000 7fbba36d4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ad92aa070 0x556ad966d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:37.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.307+0000 7fbba36d4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556ad9418700 con 0x556ad92811b0 2024-09-21T06:05:37.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.307+0000 7fbba36d4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556ad9552160 con 0x556ad9289390 2024-09-21T06:05:37.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.307+0000 7fbba36d4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556ad9663da0 con 0x556ad92aa070 2024-09-21T06:05:37.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.307+0000 7fbba36d4740 10 client.0 _ll_register_callbacks cb 0x556ad956b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:05:37.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.307+0000 7fbb96d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ad92811b0 0x556ad9660da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:37.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.307+0000 7fbb96d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ad92811b0 0x556ad9660da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53882/0 (socket says 172.21.15.117:53882) 2024-09-21T06:05:37.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.307+0000 7fbb96d9d640 1 -- 172.21.15.117:0/2685294102 learned_addr learned my addr 172.21.15.117:0/2685294102 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:37.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.307+0000 7fbb9759e640 1 --2- 172.21.15.117:0/2685294102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ad92aa070 0x556ad966d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:37.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.307+0000 7fbb9659c640 1 --2- 172.21.15.117:0/2685294102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556ad9289390 0x556ad96612e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:37.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.307+0000 7fbb96d9d640 1 -- 172.21.15.117:0/2685294102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ad92aa070 msgr2=0x556ad966d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:37.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.307+0000 7fbb96d9d640 1 --2- 172.21.15.117:0/2685294102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ad92aa070 0x556ad966d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:37.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.307+0000 7fbb96d9d640 1 -- 172.21.15.117:0/2685294102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556ad9289390 msgr2=0x556ad96612e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:37.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.307+0000 7fbb96d9d640 1 --2- 172.21.15.117:0/2685294102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556ad9289390 0x556ad96612e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:37.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.307+0000 7fbb96d9d640 1 -- 172.21.15.117:0/2685294102 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ad957fc90 con 0x556ad92811b0 2024-09-21T06:05:37.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.307+0000 7fbb96d9d640 1 --2- 172.21.15.117:0/2685294102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ad92811b0 0x556ad9660da0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fbb90005c40 tx=0x7fbb9000ff40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2300a853745409d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:37.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.307+0000 7fbb7e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:05:37.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.307+0000 7fbb7e7fc640 1 -- 172.21.15.117:0/2685294102 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9001d040 con 0x556ad92811b0 2024-09-21T06:05:37.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.307+0000 7fbb7e7fc640 1 -- 172.21.15.117:0/2685294102 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7fbb900a8f50 con 0x556ad92811b0 2024-09-21T06:05:37.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.307+0000 7fbba36d4740 1 -- 172.21.15.117:0/2685294102 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb80005db0 con 0x556ad92811b0 2024-09-21T06:05:37.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.308+0000 7fbb7e7fc640 1 -- 172.21.15.117:0/2685294102 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbb900a9c50 con 0x556ad92811b0 2024-09-21T06:05:37.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.308+0000 7fbba36d4740 10 client.17055 fetch_fsmap learned FSMap version 29 2024-09-21T06:05:37.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.308+0000 7fbba36d4740 1 -- 172.21.15.117:0/2685294102 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556ad966fad0 con 0x556ad92811b0 2024-09-21T06:05:37.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.308+0000 7fbb7e7fc640 1 -- 172.21.15.117:0/2685294102 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb80005db0 con 0x556ad92811b0 2024-09-21T06:05:37.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.308+0000 7fbb7e7fc640 1 -- 172.21.15.117:0/2685294102 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbb900a9150 con 0x556ad92811b0 2024-09-21T06:05:37.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.308+0000 7fbba36d4740 10 client.17055 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:05:37.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.308+0000 7fbba36d4740 10 client.17055 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:05:37.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.308+0000 7fbba36d4740 20 client.17055 populate_metadata read hostname 'smithi117' 2024-09-21T06:05:37.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.308+0000 7fbba36d4740 1 --2- 172.21.15.117:0/2685294102 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x556ad92689f0 0x556ad9672f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:37.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.308+0000 7fbba36d4740 4 client.17055 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:05:37.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.308+0000 7fbba36d4740 1 -- 172.21.15.117:0/2685294102 --> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ad95c9df0 con 0x556ad92689f0 2024-09-21T06:05:37.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.308+0000 7fbb9659c640 1 --2- 172.21.15.117:0/2685294102 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x556ad92689f0 0x556ad9672f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:37.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.309+0000 7fbb7e7fc640 10 client.17055 ms_handle_connect on v2:172.21.15.192:6838/1727623231 2024-09-21T06:05:37.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.309+0000 7fbb9659c640 1 --2- 172.21.15.117:0/2685294102 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x556ad92689f0 0x556ad9672f80 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9616b570016b27df server_cookie=b9e9eef2f2e83722 in_seq=0 out_seq=0 2024-09-21T06:05:37.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.309+0000 7fbb7e7fc640 1 -- 172.21.15.117:0/2685294102 <== mds.1 v2:172.21.15.192:6838/1727623231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556ad95c9df0 con 0x556ad92689f0 2024-09-21T06:05:37.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.309+0000 7fbb7e7fc640 10 client.17055 handle_command_reply: tid=0 2024-09-21T06:05:37.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.309+0000 7fbba36d4740 1 client.17055 shutdown 2024-09-21T06:05:37.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.310+0000 7fbba36d4740 1 -- 172.21.15.117:0/2685294102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ad92811b0 msgr2=0x556ad9660da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:37.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.310+0000 7fbba36d4740 1 --2- 172.21.15.117:0/2685294102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ad92811b0 0x556ad9660da0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fbb90005c40 tx=0x7fbb9000ff40 comp rx=0 tx=0).stop 2024-09-21T06:05:37.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.310+0000 7fbba36d4740 1 -- 172.21.15.117:0/2685294102 shutdown_connections 2024-09-21T06:05:37.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.310+0000 7fbba36d4740 1 --2- 172.21.15.117:0/2685294102 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x556ad92689f0 0x556ad9672f80 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:37.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.310+0000 7fbba36d4740 1 --2- 172.21.15.117:0/2685294102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ad92aa070 0x556ad966d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:37.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.310+0000 7fbba36d4740 1 --2- 172.21.15.117:0/2685294102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556ad9289390 0x556ad96612e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:37.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.310+0000 7fbb7e7fc640 0 client.17055 ms_handle_reset on v2:172.21.15.192:6838/1727623231 2024-09-21T06:05:37.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.310+0000 7fbba36d4740 1 --2- 172.21.15.117:0/2685294102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ad92811b0 0x556ad9660da0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:37.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.310+0000 7fbba36d4740 1 -- 172.21.15.117:0/2685294102 >> 172.21.15.117:0/2685294102 conn(0x556ad929e460 msgr2=0x556ad964d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:37.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.310+0000 7fbba36d4740 1 -- 172.21.15.117:0/2685294102 shutdown_connections 2024-09-21T06:05:37.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.310+0000 7fbba36d4740 1 -- 172.21.15.117:0/2685294102 wait complete. 2024-09-21T06:05:37.316 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.310+0000 7fbba36d4740 20 client.17055 trim_cache size 0 max 16384 2024-09-21T06:05:37.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.323+0000 7fbba36d4740 1 Processor -- start 2024-09-21T06:05:37.324 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.323+0000 7fbba36d4740 1 -- start start 2024-09-21T06:05:37.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.323+0000 7fbba36d4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556ad92811b0 0x556ad965c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:37.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.323+0000 7fbba36d4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ad9289390 0x556ad964f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:37.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.323+0000 7fbba36d4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ad92aa070 0x556ad966cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:37.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.323+0000 7fbba36d4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbb981ad430 con 0x556ad9289390 2024-09-21T06:05:37.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.323+0000 7fbba36d4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbb981aef70 con 0x556ad92811b0 2024-09-21T06:05:37.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.323+0000 7fbba36d4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbb981adb00 con 0x556ad92aa070 2024-09-21T06:05:37.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.323+0000 7fbb9659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ad9289390 0x556ad964f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:37.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.323+0000 7fbb9659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ad9289390 0x556ad964f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53888/0 (socket says 172.21.15.117:53888) 2024-09-21T06:05:37.326 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.323+0000 7fbb9659c640 1 -- 172.21.15.117:0/2915260447 learned_addr learned my addr 172.21.15.117:0/2915260447 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:37.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.324+0000 7fbb9759e640 1 --2- 172.21.15.117:0/2915260447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ad92aa070 0x556ad966cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:37.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.324+0000 7fbb96d9d640 1 --2- 172.21.15.117:0/2915260447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556ad92811b0 0x556ad965c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:37.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.324+0000 7fbb9659c640 1 -- 172.21.15.117:0/2915260447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ad92aa070 msgr2=0x556ad966cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:37.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.324+0000 7fbb9659c640 1 --2- 172.21.15.117:0/2915260447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ad92aa070 0x556ad966cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:37.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.324+0000 7fbb9659c640 1 -- 172.21.15.117:0/2915260447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556ad92811b0 msgr2=0x556ad965c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:37.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.324+0000 7fbb9659c640 1 --2- 172.21.15.117:0/2915260447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556ad92811b0 0x556ad965c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:37.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.324+0000 7fbb9659c640 1 -- 172.21.15.117:0/2915260447 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ad965f2a0 con 0x556ad9289390 2024-09-21T06:05:37.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.324+0000 7fbb96d9d640 1 --2- 172.21.15.117:0/2915260447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556ad92811b0 0x556ad965c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:05:37.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.324+0000 7fbb9659c640 1 --2- 172.21.15.117:0/2915260447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ad9289390 0x556ad964f1e0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fbb8002a930 tx=0x7fbb800051f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd41e8016f004a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:37.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.324+0000 7fbb8cff9640 1 -- 172.21.15.117:0/2915260447 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb80055070 con 0x556ad9289390 2024-09-21T06:05:37.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.324+0000 7fbb8cff9640 1 -- 172.21.15.117:0/2915260447 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb800054b0 con 0x556ad9289390 2024-09-21T06:05:37.327 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.324+0000 7fbb8cff9640 1 -- 172.21.15.117:0/2915260447 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb80035040 con 0x556ad9289390 2024-09-21T06:05:37.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.325+0000 7fbba36d4740 1 -- 172.21.15.117:0/2915260447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ad9289390 msgr2=0x556ad964f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:37.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.325+0000 7fbba36d4740 1 --2- 172.21.15.117:0/2915260447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ad9289390 0x556ad964f1e0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fbb8002a930 tx=0x7fbb800051f0 comp rx=0 tx=0).stop 2024-09-21T06:05:37.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.325+0000 7fbba36d4740 1 -- 172.21.15.117:0/2915260447 shutdown_connections 2024-09-21T06:05:37.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.325+0000 7fbba36d4740 1 --2- 172.21.15.117:0/2915260447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ad9289390 0x556ad964f1e0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:37.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.325+0000 7fbba36d4740 1 --2- 172.21.15.117:0/2915260447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ad92aa070 0x556ad966cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:37.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.325+0000 7fbba36d4740 1 --2- 172.21.15.117:0/2915260447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556ad92811b0 0x556ad965c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:37.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.325+0000 7fbba36d4740 1 -- 172.21.15.117:0/2915260447 >> 172.21.15.117:0/2915260447 conn(0x556ad929e460 msgr2=0x556ad964ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:37.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.325+0000 7fbba36d4740 1 -- 172.21.15.117:0/2915260447 shutdown_connections 2024-09-21T06:05:37.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.325+0000 7fbba36d4740 1 -- 172.21.15.117:0/2915260447 wait complete. 2024-09-21T06:05:37.331 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.325+0000 7fbba36d4740 1 -- start start 2024-09-21T06:05:37.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.325+0000 7fbba36d4740 1 Processor -- start 2024-09-21T06:05:37.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.326+0000 7fbba36d4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ad92811b0 0x556ad9665120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:37.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.326+0000 7fbba36d4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556ad9289390 0x556ad9665660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:37.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.326+0000 7fbba36d4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ad92aa070 0x556ad9665ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:37.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.326+0000 7fbba36d4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbb981ae8a0 con 0x556ad92aa070 2024-09-21T06:05:37.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.326+0000 7fbba36d4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbb981ae1d0 con 0x556ad9289390 2024-09-21T06:05:37.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.326+0000 7fbba36d4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbb981ab430 con 0x556ad92811b0 2024-09-21T06:05:37.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.326+0000 7fbba36d4740 10 client.0 _ll_register_callbacks cb 0x7fbb9000cc30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:05:37.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.326+0000 7fbb9759e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ad92aa070 0x556ad9665ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:37.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.326+0000 7fbb9759e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ad92aa070 0x556ad9665ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53894/0 (socket says 172.21.15.117:53894) 2024-09-21T06:05:37.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.326+0000 7fbb9759e640 1 -- 172.21.15.117:0/2915172969 learned_addr learned my addr 172.21.15.117:0/2915172969 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:37.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.326+0000 7fbb96d9d640 1 --2- 172.21.15.117:0/2915172969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ad92811b0 0x556ad9665120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:37.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.326+0000 7fbb9659c640 1 --2- 172.21.15.117:0/2915172969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556ad9289390 0x556ad9665660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:37.332 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.326+0000 7fbb9759e640 1 -- 172.21.15.117:0/2915172969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ad92811b0 msgr2=0x556ad9665120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:37.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.326+0000 7fbb9759e640 1 --2- 172.21.15.117:0/2915172969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ad92811b0 0x556ad9665120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:37.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.326+0000 7fbb9759e640 1 -- 172.21.15.117:0/2915172969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556ad9289390 msgr2=0x556ad9665660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:37.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.326+0000 7fbb9759e640 1 --2- 172.21.15.117:0/2915172969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556ad9289390 0x556ad9665660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:37.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.326+0000 7fbb9759e640 1 -- 172.21.15.117:0/2915172969 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ad96dca70 con 0x556ad92aa070 2024-09-21T06:05:37.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.327+0000 7fbb9759e640 1 --2- 172.21.15.117:0/2915172969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ad92aa070 0x556ad9665ba0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fbb8801f7a0 tx=0x7fbb880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8288f8a652d7236c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:37.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.327+0000 7fbb7e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:05:37.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.327+0000 7fbb7e7fc640 1 -- 172.21.15.117:0/2915172969 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8802d060 con 0x556ad92aa070 2024-09-21T06:05:37.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.327+0000 7fbb7e7fc640 1 -- 172.21.15.117:0/2915172969 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7fbb8801d6d0 con 0x556ad92aa070 2024-09-21T06:05:37.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.327+0000 7fbba36d4740 1 -- 172.21.15.117:0/2915172969 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556ad95c58a0 con 0x556ad92aa070 2024-09-21T06:05:37.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.327+0000 7fbb7e7fc640 1 -- 172.21.15.117:0/2915172969 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbb8801d910 con 0x556ad92aa070 2024-09-21T06:05:37.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.327+0000 7fbba36d4740 10 client.17061 fetch_fsmap learned FSMap version 29 2024-09-21T06:05:37.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.327+0000 7fbba36d4740 1 -- 172.21.15.117:0/2915172969 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556ad96ddca0 con 0x556ad92aa070 2024-09-21T06:05:37.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.327+0000 7fbb7e7fc640 1 -- 172.21.15.117:0/2915172969 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556ad95c58a0 con 0x556ad92aa070 2024-09-21T06:05:37.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.327+0000 7fbb7e7fc640 1 -- 172.21.15.117:0/2915172969 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbb88027070 con 0x556ad92aa070 2024-09-21T06:05:37.333 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.327+0000 7fbba36d4740 10 client.17061 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:05:37.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.327+0000 7fbba36d4740 10 client.17061 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:05:37.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.327+0000 7fbba36d4740 20 client.17061 populate_metadata read hostname 'smithi117' 2024-09-21T06:05:37.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.327+0000 7fbba36d4740 1 --2- 172.21.15.117:0/2915172969 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x556ad923dda0 0x556ad96e04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:37.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.327+0000 7fbba36d4740 4 client.17061 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:05:37.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.327+0000 7fbba36d4740 1 -- 172.21.15.117:0/2915172969 --> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556ad95c2cd0 con 0x556ad923dda0 2024-09-21T06:05:37.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.328+0000 7fbb96d9d640 1 --2- 172.21.15.117:0/2915172969 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x556ad923dda0 0x556ad96e04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:37.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.328+0000 7fbb7e7fc640 10 client.17061 ms_handle_connect on v2:172.21.15.192:6838/1727623231 2024-09-21T06:05:37.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.328+0000 7fbb96d9d640 1 --2- 172.21.15.117:0/2915172969 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x556ad923dda0 0x556ad96e04c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ceeb8f51da71c871 server_cookie=42e4be1863668626 in_seq=0 out_seq=0 2024-09-21T06:05:37.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.328+0000 7fbb7e7fc640 1 -- 172.21.15.117:0/2915172969 <== mds.1 v2:172.21.15.192:6838/1727623231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556ad95c2cd0 con 0x556ad923dda0 2024-09-21T06:05:37.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.328+0000 7fbb7e7fc640 10 client.17061 handle_command_reply: tid=0 2024-09-21T06:05:37.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.328+0000 7fbba36d4740 1 client.17061 shutdown 2024-09-21T06:05:37.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.328+0000 7fbba36d4740 1 -- 172.21.15.117:0/2915172969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ad92aa070 msgr2=0x556ad9665ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:37.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.328+0000 7fbba36d4740 1 --2- 172.21.15.117:0/2915172969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ad92aa070 0x556ad9665ba0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fbb8801f7a0 tx=0x7fbb880046c0 comp rx=0 tx=0).stop 2024-09-21T06:05:37.334 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.328+0000 7fbba36d4740 1 -- 172.21.15.117:0/2915172969 shutdown_connections 2024-09-21T06:05:37.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.328+0000 7fbba36d4740 1 --2- 172.21.15.117:0/2915172969 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x556ad923dda0 0x556ad96e04c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:37.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.328+0000 7fbba36d4740 1 --2- 172.21.15.117:0/2915172969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556ad92aa070 0x556ad9665ba0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:37.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.328+0000 7fbba36d4740 1 --2- 172.21.15.117:0/2915172969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556ad9289390 0x556ad9665660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:37.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.329+0000 7fbba36d4740 1 --2- 172.21.15.117:0/2915172969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556ad92811b0 0x556ad9665120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:37.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.329+0000 7fbb7e7fc640 0 client.17061 ms_handle_reset on v2:172.21.15.192:6838/1727623231 2024-09-21T06:05:37.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.329+0000 7fbba36d4740 1 -- 172.21.15.117:0/2915172969 >> 172.21.15.117:0/2915172969 conn(0x556ad929e460 msgr2=0x556ad96522d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:37.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.329+0000 7fbba36d4740 1 -- 172.21.15.117:0/2915172969 shutdown_connections 2024-09-21T06:05:37.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.329+0000 7fbba36d4740 1 -- 172.21.15.117:0/2915172969 wait complete. 2024-09-21T06:05:37.335 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.329+0000 7fbba36d4740 20 client.17061 trim_cache size 0 max 16384 2024-09-21T06:05:37.335 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:05:37.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.329+0000 7fbb9d160640 1 -- 172.21.15.117:0/2688885071 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbb840b6680 msgr2=0x7fbb840b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:37.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.329+0000 7fbb9d160640 1 --2- 172.21.15.117:0/2688885071 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbb840b6680 0x7fbb840b8b90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbb88005d80 tx=0x7fbb8800b420 comp rx=0 tx=0).stop 2024-09-21T06:05:37.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.329+0000 7fbb9d160640 1 -- 172.21.15.117:0/2688885071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb981048c0 msgr2=0x7fbb9819a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:37.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.329+0000 7fbb9d160640 1 --2- 172.21.15.117:0/2688885071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb981048c0 0x7fbb9819a5a0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fbb9000cce0 tx=0x7fbb9000a620 comp rx=0 tx=0).stop 2024-09-21T06:05:37.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.329+0000 7fbb9d160640 1 -- 172.21.15.117:0/2688885071 shutdown_connections 2024-09-21T06:05:37.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.329+0000 7fbb9d160640 1 --2- 172.21.15.117:0/2688885071 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbb840b6680 0x7fbb840b8b90 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:37.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.329+0000 7fbb9d160640 1 --2- 172.21.15.117:0/2688885071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb9810de70 0x7fbb9819fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:37.336 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.329+0000 7fbb9d160640 1 --2- 172.21.15.117:0/2688885071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb981052f0 0x7fbb9819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:37.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.329+0000 7fbb9d160640 1 --2- 172.21.15.117:0/2688885071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb981048c0 0x7fbb9819a5a0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:37.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.329+0000 7fbb9d160640 1 -- 172.21.15.117:0/2688885071 >> 172.21.15.117:0/2688885071 conn(0x7fbb98079770 msgr2=0x7fbb98100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:37.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.329+0000 7fbb9d160640 1 -- 172.21.15.117:0/2688885071 shutdown_connections 2024-09-21T06:05:37.337 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:37.329+0000 7fbb9d160640 1 -- 172.21.15.117:0/2688885071 wait complete. 2024-09-21T06:05:37.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:36 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:05:37.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:36 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:05:37.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:36 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:05:37.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:36 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:05:37.999 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:05:38.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:37 smithi192 ceph-mon[34700]: pgmap v579: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T06:05:38.247 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:05:38.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:37 smithi117 ceph-mon[30947]: pgmap v579: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T06:05:38.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:37 smithi138 ceph-mon[34391]: pgmap v579: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T06:05:38.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.705+0000 7fd22f741640 1 -- 172.21.15.117:0/2074701041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd228104a30 msgr2=0x7fd228104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:38.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.705+0000 7fd22f741640 1 --2- 172.21.15.117:0/2074701041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd228104a30 0x7fd228104e40 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fd224009900 tx=0x7fd22402a410 comp rx=0 tx=0).stop 2024-09-21T06:05:38.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.705+0000 7fd22f741640 1 -- 172.21.15.117:0/2074701041 shutdown_connections 2024-09-21T06:05:38.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.705+0000 7fd22f741640 1 --2- 172.21.15.117:0/2074701041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd22810de90 0x7fd228110380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.705+0000 7fd22f741640 1 --2- 172.21.15.117:0/2074701041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd228105380 0x7fd22810d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.705+0000 7fd22f741640 1 --2- 172.21.15.117:0/2074701041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd228104a30 0x7fd228104e40 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.705+0000 7fd22f741640 1 -- 172.21.15.117:0/2074701041 >> 172.21.15.117:0/2074701041 conn(0x7fd228079770 msgr2=0x7fd228079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:38.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.705+0000 7fd22f741640 1 -- 172.21.15.117:0/2074701041 shutdown_connections 2024-09-21T06:05:38.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.705+0000 7fd22f741640 1 -- 172.21.15.117:0/2074701041 wait complete. 2024-09-21T06:05:38.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.705+0000 7fd22f741640 1 Processor -- start 2024-09-21T06:05:38.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.705+0000 7fd22f741640 1 -- start start 2024-09-21T06:05:38.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.706+0000 7fd22f741640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd228104a30 0x7fd22819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:38.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.706+0000 7fd22f741640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd228105380 0x7fd22819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:38.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.706+0000 7fd22f741640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd22810de90 0x7fd22819fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:38.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.706+0000 7fd22f741640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd228113b10 con 0x7fd228105380 2024-09-21T06:05:38.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.706+0000 7fd22f741640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd228113990 con 0x7fd22810de90 2024-09-21T06:05:38.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.706+0000 7fd22f741640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd228113c90 con 0x7fd228104a30 2024-09-21T06:05:38.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.706+0000 7fd22ccdc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd228105380 0x7fd22819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:38.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.706+0000 7fd22ccdc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd228105380 0x7fd22819aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53926/0 (socket says 172.21.15.117:53926) 2024-09-21T06:05:38.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.706+0000 7fd22ccdc640 1 -- 172.21.15.117:0/3694330413 learned_addr learned my addr 172.21.15.117:0/3694330413 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:38.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.706+0000 7fd22dcde640 1 --2- 172.21.15.117:0/3694330413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd22810de90 0x7fd22819fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:38.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.706+0000 7fd22d4dd640 1 --2- 172.21.15.117:0/3694330413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd228104a30 0x7fd22819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:38.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.706+0000 7fd22ccdc640 1 -- 172.21.15.117:0/3694330413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd228104a30 msgr2=0x7fd22819a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:38.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.706+0000 7fd22ccdc640 1 --2- 172.21.15.117:0/3694330413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd228104a30 0x7fd22819a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.706+0000 7fd22ccdc640 1 -- 172.21.15.117:0/3694330413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd22810de90 msgr2=0x7fd22819fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:38.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.706+0000 7fd22ccdc640 1 --2- 172.21.15.117:0/3694330413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd22810de90 0x7fd22819fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.706+0000 7fd22ccdc640 1 -- 172.21.15.117:0/3694330413 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2281a04b0 con 0x7fd228105380 2024-09-21T06:05:38.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.706+0000 7fd22ccdc640 1 --2- 172.21.15.117:0/3694330413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd228105380 0x7fd22819aae0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fd21800e9c0 tx=0x7fd21800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:38.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.707+0000 7fd21e7fc640 1 -- 172.21.15.117:0/3694330413 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21800ccc0 con 0x7fd228105380 2024-09-21T06:05:38.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.707+0000 7fd21e7fc640 1 -- 172.21.15.117:0/3694330413 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2180054d0 con 0x7fd228105380 2024-09-21T06:05:38.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.707+0000 7fd21e7fc640 1 -- 172.21.15.117:0/3694330413 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd218010430 con 0x7fd228105380 2024-09-21T06:05:38.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.707+0000 7fd22f741640 1 -- 172.21.15.117:0/3694330413 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2281a07a0 con 0x7fd228105380 2024-09-21T06:05:38.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.707+0000 7fd22f741640 1 -- 172.21.15.117:0/3694330413 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2281a6f00 con 0x7fd228105380 2024-09-21T06:05:38.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.709+0000 7fd235cb5740 1 Processor -- start 2024-09-21T06:05:38.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.709+0000 7fd21e7fc640 1 -- 172.21.15.117:0/3694330413 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fd218005020 con 0x7fd228105380 2024-09-21T06:05:38.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.709+0000 7fd235cb5740 1 -- start start 2024-09-21T06:05:38.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.710+0000 7fd235cb5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5941da070 0x55f5945812c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:38.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.710+0000 7fd235cb5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5941b11b0 0x55f594581800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:38.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.710+0000 7fd235cb5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5941b9390 0x55f594581d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:38.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.710+0000 7fd235cb5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f5944f7ea0 con 0x55f5941b11b0 2024-09-21T06:05:38.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.710+0000 7fd235cb5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f5944e72d0 con 0x55f5941da070 2024-09-21T06:05:38.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.710+0000 7fd235cb5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f5944eb0e0 con 0x55f5941b9390 2024-09-21T06:05:38.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.710+0000 7fd22d4dd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5941b11b0 0x55f594581800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:38.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.710+0000 7fd22d4dd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5941b11b0 0x55f594581800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53930/0 (socket says 172.21.15.117:53930) 2024-09-21T06:05:38.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.710+0000 7fd22d4dd640 1 -- 172.21.15.117:0/627855735 learned_addr learned my addr 172.21.15.117:0/627855735 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:38.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.710+0000 7fd22dcde640 1 --2- 172.21.15.117:0/627855735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5941da070 0x55f5945812c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:38.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.710+0000 7fd22ccdc640 1 --2- 172.21.15.117:0/627855735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5941b9390 0x55f594581d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:38.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.710+0000 7fd22d4dd640 1 -- 172.21.15.117:0/627855735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5941b9390 msgr2=0x55f594581d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:38.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.710+0000 7fd22d4dd640 1 --2- 172.21.15.117:0/627855735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5941b9390 0x55f594581d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.710+0000 7fd22d4dd640 1 -- 172.21.15.117:0/627855735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5941da070 msgr2=0x55f5945812c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:38.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.710+0000 7fd22d4dd640 1 --2- 172.21.15.117:0/627855735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5941da070 0x55f5945812c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.710+0000 7fd22d4dd640 1 -- 172.21.15.117:0/627855735 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f594245650 con 0x55f5941b11b0 2024-09-21T06:05:38.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.710+0000 7fd22d4dd640 1 --2- 172.21.15.117:0/627855735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5941b11b0 0x55f594581800 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fd224009900 tx=0x7fd224032c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4cdc3b86ea4b312 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:38.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.710+0000 7fd21e7fc640 1 --2- 172.21.15.117:0/3694330413 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd2140b6680 0x7fd2140b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:38.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.710+0000 7fd2137fe640 1 -- 172.21.15.117:0/627855735 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd224040070 con 0x55f5941b11b0 2024-09-21T06:05:38.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.711+0000 7fd22dcde640 1 --2- 172.21.15.117:0/3694330413 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd2140b6680 0x7fd2140b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:38.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.711+0000 7fd2137fe640 1 -- 172.21.15.117:0/627855735 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd224004030 con 0x55f5941b11b0 2024-09-21T06:05:38.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.711+0000 7fd2137fe640 1 -- 172.21.15.117:0/627855735 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd224035040 con 0x55f5941b11b0 2024-09-21T06:05:38.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.711+0000 7fd21e7fc640 1 -- 172.21.15.117:0/3694330413 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7fd2180e7a50 con 0x7fd228105380 2024-09-21T06:05:38.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.711+0000 7fd235cb5740 1 -- 172.21.15.117:0/627855735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5941b11b0 msgr2=0x55f594581800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:38.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.711+0000 7fd22dcde640 1 --2- 172.21.15.117:0/3694330413 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd2140b6680 0x7fd2140b8b90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd220005fd0 tx=0x7fd220009600 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:38.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.711+0000 7fd235cb5740 1 --2- 172.21.15.117:0/627855735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5941b11b0 0x55f594581800 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fd224009900 tx=0x7fd224032c40 comp rx=0 tx=0).stop 2024-09-21T06:05:38.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.711+0000 7fd235cb5740 1 -- 172.21.15.117:0/627855735 shutdown_connections 2024-09-21T06:05:38.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.711+0000 7fd235cb5740 1 --2- 172.21.15.117:0/627855735 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5941b11b0 0x55f594581800 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.711+0000 7fd235cb5740 1 --2- 172.21.15.117:0/627855735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5941b9390 0x55f594581d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.711+0000 7fd235cb5740 1 --2- 172.21.15.117:0/627855735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5941da070 0x55f5945812c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.711+0000 7fd235cb5740 1 -- 172.21.15.117:0/627855735 >> 172.21.15.117:0/627855735 conn(0x55f5941ce460 msgr2=0x55f59457f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:38.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.712+0000 7fd235cb5740 1 -- 172.21.15.117:0/627855735 shutdown_connections 2024-09-21T06:05:38.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.712+0000 7fd235cb5740 1 -- 172.21.15.117:0/627855735 wait complete. 2024-09-21T06:05:38.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.712+0000 7fd235cb5740 1 -- start start 2024-09-21T06:05:38.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.712+0000 7fd235cb5740 1 Processor -- start 2024-09-21T06:05:38.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.712+0000 7fd235cb5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5941b11b0 0x55f59458c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:38.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.712+0000 7fd235cb5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5941b9390 0x55f59458ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:38.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.712+0000 7fd235cb5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5941da070 0x55f59458d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:38.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.712+0000 7fd235cb5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f594348700 con 0x55f5941da070 2024-09-21T06:05:38.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.712+0000 7fd235cb5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f594482160 con 0x55f5941b9390 2024-09-21T06:05:38.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.712+0000 7fd235cb5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f59458fa40 con 0x55f5941b11b0 2024-09-21T06:05:38.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.713+0000 7fd235cb5740 10 client.0 _ll_register_callbacks cb 0x55f59449b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:05:38.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.713+0000 7fd22dcde640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5941da070 0x55f59458d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:38.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.713+0000 7fd22dcde640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5941da070 0x55f59458d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53934/0 (socket says 172.21.15.117:53934) 2024-09-21T06:05:38.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.713+0000 7fd22dcde640 1 -- 172.21.15.117:0/3650649931 learned_addr learned my addr 172.21.15.117:0/3650649931 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:38.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.713+0000 7fd22ccdc640 1 --2- 172.21.15.117:0/3650649931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5941b9390 0x55f59458ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:38.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.713+0000 7fd22d4dd640 1 --2- 172.21.15.117:0/3650649931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5941b11b0 0x55f59458c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:38.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.713+0000 7fd22dcde640 1 -- 172.21.15.117:0/3650649931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5941b11b0 msgr2=0x55f59458c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:38.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.713+0000 7fd22dcde640 1 --2- 172.21.15.117:0/3650649931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5941b11b0 0x55f59458c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.713+0000 7fd22dcde640 1 -- 172.21.15.117:0/3650649931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5941b9390 msgr2=0x55f59458ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:38.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.713+0000 7fd22dcde640 1 --2- 172.21.15.117:0/3650649931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5941b9390 0x55f59458ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.713+0000 7fd22dcde640 1 -- 172.21.15.117:0/3650649931 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5944afc90 con 0x55f5941da070 2024-09-21T06:05:38.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.713+0000 7fd22d4dd640 1 --2- 172.21.15.117:0/3650649931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5941b11b0 0x55f59458c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:05:38.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.713+0000 7fd22dcde640 1 --2- 172.21.15.117:0/3650649931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5941da070 0x55f59458d3e0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fd22001d6f0 tx=0x7fd220004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec21b861833e4b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:38.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.713+0000 7fd210ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:05:38.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.713+0000 7fd210ff9640 1 -- 172.21.15.117:0/3650649931 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd220003b70 con 0x55f5941da070 2024-09-21T06:05:38.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.713+0000 7fd235cb5740 1 -- 172.21.15.117:0/3650649931 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd224004030 con 0x55f5941da070 2024-09-21T06:05:38.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.713+0000 7fd210ff9640 1 -- 172.21.15.117:0/3650649931 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7fd220003d10 con 0x55f5941da070 2024-09-21T06:05:38.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.713+0000 7fd210ff9640 1 -- 172.21.15.117:0/3650649931 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd220004030 con 0x55f5941da070 2024-09-21T06:05:38.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.713+0000 7fd235cb5740 10 client.17073 fetch_fsmap learned FSMap version 29 2024-09-21T06:05:38.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.714+0000 7fd235cb5740 1 -- 172.21.15.117:0/3650649931 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f59459b780 con 0x55f5941da070 2024-09-21T06:05:38.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.714+0000 7fd210ff9640 1 -- 172.21.15.117:0/3650649931 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd224004030 con 0x55f5941da070 2024-09-21T06:05:38.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.714+0000 7fd210ff9640 1 -- 172.21.15.117:0/3650649931 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd22000f040 con 0x55f5941da070 2024-09-21T06:05:38.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.714+0000 7fd235cb5740 10 client.17073 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:05:38.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.714+0000 7fd235cb5740 10 client.17073 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:05:38.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.714+0000 7fd235cb5740 20 client.17073 populate_metadata read hostname 'smithi117' 2024-09-21T06:05:38.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.714+0000 7fd235cb5740 1 --2- 172.21.15.117:0/3650649931 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f5941989f0 0x55f59459eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:38.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.714+0000 7fd235cb5740 4 client.17073 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:05:38.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.714+0000 7fd235cb5740 1 -- 172.21.15.117:0/3650649931 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5944f9df0 con 0x55f5941989f0 2024-09-21T06:05:38.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.714+0000 7fd22d4dd640 1 --2- 172.21.15.117:0/3650649931 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f5941989f0 0x55f59459eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:38.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.715+0000 7fd210ff9640 10 client.17073 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:38.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.715+0000 7fd22d4dd640 1 --2- 172.21.15.117:0/3650649931 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f5941989f0 0x55f59459eda0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a33e04ffec8133e server_cookie=e661f91f3ad09e69 in_seq=0 out_seq=0 2024-09-21T06:05:38.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.715+0000 7fd210ff9640 1 -- 172.21.15.117:0/3650649931 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f5944f9df0 con 0x55f5941989f0 2024-09-21T06:05:38.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.716+0000 7fd210ff9640 10 client.17073 handle_command_reply: tid=0 2024-09-21T06:05:38.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.716+0000 7fd235cb5740 1 client.17073 shutdown 2024-09-21T06:05:38.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.716+0000 7fd235cb5740 1 -- 172.21.15.117:0/3650649931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5941da070 msgr2=0x55f59458d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:38.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.716+0000 7fd235cb5740 1 --2- 172.21.15.117:0/3650649931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5941da070 0x55f59458d3e0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fd22001d6f0 tx=0x7fd220004a40 comp rx=0 tx=0).stop 2024-09-21T06:05:38.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.716+0000 7fd235cb5740 1 -- 172.21.15.117:0/3650649931 shutdown_connections 2024-09-21T06:05:38.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.716+0000 7fd235cb5740 1 --2- 172.21.15.117:0/3650649931 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f5941989f0 0x55f59459eda0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.716+0000 7fd235cb5740 1 --2- 172.21.15.117:0/3650649931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5941da070 0x55f59458d3e0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.716+0000 7fd210ff9640 0 client.17073 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:38.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.716+0000 7fd235cb5740 1 --2- 172.21.15.117:0/3650649931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5941b9390 0x55f59458ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.716+0000 7fd235cb5740 1 --2- 172.21.15.117:0/3650649931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5941b11b0 0x55f59458c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.716+0000 7fd235cb5740 1 -- 172.21.15.117:0/3650649931 >> 172.21.15.117:0/3650649931 conn(0x55f5941ce460 msgr2=0x55f594584450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:38.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.716+0000 7fd235cb5740 1 -- 172.21.15.117:0/3650649931 shutdown_connections 2024-09-21T06:05:38.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.716+0000 7fd235cb5740 1 -- 172.21.15.117:0/3650649931 wait complete. 2024-09-21T06:05:38.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.716+0000 7fd235cb5740 20 client.17073 trim_cache size 0 max 16384 2024-09-21T06:05:38.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.729+0000 7fd235cb5740 1 Processor -- start 2024-09-21T06:05:38.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.729+0000 7fd235cb5740 1 -- start start 2024-09-21T06:05:38.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.729+0000 7fd235cb5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5941989f0 0x55f594582cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:38.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.729+0000 7fd235cb5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5941b11b0 0x55f59458e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:38.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.729+0000 7fd235cb5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5941b9390 0x55f594598c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:38.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.729+0000 7fd235cb5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd2281af5d0 con 0x55f5941b9390 2024-09-21T06:05:38.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.729+0000 7fd235cb5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd2281b1110 con 0x55f5941989f0 2024-09-21T06:05:38.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.729+0000 7fd235cb5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd2281afca0 con 0x55f5941b11b0 2024-09-21T06:05:38.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.729+0000 7fd22dcde640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5941b9390 0x55f594598c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:38.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.729+0000 7fd22dcde640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5941b9390 0x55f594598c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53946/0 (socket says 172.21.15.117:53946) 2024-09-21T06:05:38.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.729+0000 7fd22dcde640 1 -- 172.21.15.117:0/2196601490 learned_addr learned my addr 172.21.15.117:0/2196601490 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:38.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.730+0000 7fd22d4dd640 1 --2- 172.21.15.117:0/2196601490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5941989f0 0x55f594582cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:38.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.730+0000 7fd22dcde640 1 -- 172.21.15.117:0/2196601490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5941b11b0 msgr2=0x55f59458e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:38.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.730+0000 7fd22dcde640 1 --2- 172.21.15.117:0/2196601490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5941b11b0 0x55f59458e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.730+0000 7fd22dcde640 1 -- 172.21.15.117:0/2196601490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5941989f0 msgr2=0x55f594582cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:38.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.730+0000 7fd22dcde640 1 --2- 172.21.15.117:0/2196601490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5941989f0 0x55f594582cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.730+0000 7fd22dcde640 1 -- 172.21.15.117:0/2196601490 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5945835b0 con 0x55f5941b9390 2024-09-21T06:05:38.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.730+0000 7fd22dcde640 1 --2- 172.21.15.117:0/2196601490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5941b9390 0x55f594598c90 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fd220012d40 tx=0x7fd22001d4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6193b58a4a99c439 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:38.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.730+0000 7fd212ffd640 1 -- 172.21.15.117:0/2196601490 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd220034070 con 0x55f5941b9390 2024-09-21T06:05:38.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.730+0000 7fd212ffd640 1 -- 172.21.15.117:0/2196601490 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd220011120 con 0x55f5941b9390 2024-09-21T06:05:38.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.730+0000 7fd212ffd640 1 -- 172.21.15.117:0/2196601490 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd220003bd0 con 0x55f5941b9390 2024-09-21T06:05:38.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.731+0000 7fd235cb5740 1 -- 172.21.15.117:0/2196601490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5941b9390 msgr2=0x55f594598c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:38.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.731+0000 7fd235cb5740 1 --2- 172.21.15.117:0/2196601490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5941b9390 0x55f594598c90 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fd220012d40 tx=0x7fd22001d4c0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.731+0000 7fd235cb5740 1 -- 172.21.15.117:0/2196601490 shutdown_connections 2024-09-21T06:05:38.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.731+0000 7fd235cb5740 1 --2- 172.21.15.117:0/2196601490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5941b9390 0x55f594598c90 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.731+0000 7fd235cb5740 1 --2- 172.21.15.117:0/2196601490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5941b11b0 0x55f59458e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.731+0000 7fd235cb5740 1 --2- 172.21.15.117:0/2196601490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5941989f0 0x55f594582cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.731+0000 7fd235cb5740 1 -- 172.21.15.117:0/2196601490 >> 172.21.15.117:0/2196601490 conn(0x55f5941ce460 msgr2=0x55f59458de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:38.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.731+0000 7fd235cb5740 1 -- 172.21.15.117:0/2196601490 shutdown_connections 2024-09-21T06:05:38.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.731+0000 7fd235cb5740 1 -- 172.21.15.117:0/2196601490 wait complete. 2024-09-21T06:05:38.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.731+0000 7fd235cb5740 1 -- start start 2024-09-21T06:05:38.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.731+0000 7fd235cb5740 1 Processor -- start 2024-09-21T06:05:38.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.732+0000 7fd235cb5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5941989f0 0x55f594591400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:38.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.732+0000 7fd235cb5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5941b11b0 0x55f594591940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:38.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.732+0000 7fd235cb5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5941b9390 0x55f5946087a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:38.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.732+0000 7fd235cb5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd2281b0a40 con 0x55f5941b11b0 2024-09-21T06:05:38.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.732+0000 7fd235cb5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd2281b0370 con 0x55f5941989f0 2024-09-21T06:05:38.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.732+0000 7fd235cb5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd2281ad5d0 con 0x55f5941b9390 2024-09-21T06:05:38.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.732+0000 7fd22ccdc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5941b11b0 0x55f594591940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:38.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.732+0000 7fd235cb5740 10 client.0 _ll_register_callbacks cb 0x7fd220009c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:05:38.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.732+0000 7fd22ccdc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5941b11b0 0x55f594591940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53958/0 (socket says 172.21.15.117:53958) 2024-09-21T06:05:38.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.732+0000 7fd22ccdc640 1 -- 172.21.15.117:0/2258955506 learned_addr learned my addr 172.21.15.117:0/2258955506 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:38.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.732+0000 7fd22d4dd640 1 --2- 172.21.15.117:0/2258955506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5941989f0 0x55f594591400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:38.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.732+0000 7fd22dcde640 1 --2- 172.21.15.117:0/2258955506 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5941b9390 0x55f5946087a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:38.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.732+0000 7fd22ccdc640 1 -- 172.21.15.117:0/2258955506 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5941b9390 msgr2=0x55f5946087a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:38.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.732+0000 7fd22ccdc640 1 --2- 172.21.15.117:0/2258955506 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5941b9390 0x55f5946087a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.732+0000 7fd22ccdc640 1 -- 172.21.15.117:0/2258955506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5941989f0 msgr2=0x55f594591400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:38.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.732+0000 7fd22ccdc640 1 --2- 172.21.15.117:0/2258955506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5941989f0 0x55f594591400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.732+0000 7fd22ccdc640 1 -- 172.21.15.117:0/2258955506 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f594608ce0 con 0x55f5941b11b0 2024-09-21T06:05:38.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.732+0000 7fd22ccdc640 1 --2- 172.21.15.117:0/2258955506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5941b11b0 0x55f594591940 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fd2180e7db0 tx=0x7fd2180a98c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39154aca65a7aed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:38.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.732+0000 7fd210ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:05:38.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.733+0000 7fd210ff9640 1 -- 172.21.15.117:0/2258955506 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21800ce60 con 0x55f5941b11b0 2024-09-21T06:05:38.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.733+0000 7fd210ff9640 1 -- 172.21.15.117:0/2258955506 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7fd2180a9240 con 0x55f5941b11b0 2024-09-21T06:05:38.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.733+0000 7fd235cb5740 1 -- 172.21.15.117:0/2258955506 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd220011120 con 0x55f5941b11b0 2024-09-21T06:05:38.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.733+0000 7fd210ff9640 1 -- 172.21.15.117:0/2258955506 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd2180a9060 con 0x55f5941b11b0 2024-09-21T06:05:38.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.733+0000 7fd235cb5740 10 client.17079 fetch_fsmap learned FSMap version 29 2024-09-21T06:05:38.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.733+0000 7fd235cb5740 1 -- 172.21.15.117:0/2258955506 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f594609f60 con 0x55f5941b11b0 2024-09-21T06:05:38.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.733+0000 7fd210ff9640 1 -- 172.21.15.117:0/2258955506 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd220011120 con 0x55f5941b11b0 2024-09-21T06:05:38.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.733+0000 7fd210ff9640 1 -- 172.21.15.117:0/2258955506 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd2180a9440 con 0x55f5941b11b0 2024-09-21T06:05:38.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.733+0000 7fd235cb5740 10 client.17079 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:05:38.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.733+0000 7fd235cb5740 10 client.17079 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:05:38.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.733+0000 7fd235cb5740 20 client.17079 populate_metadata read hostname 'smithi117' 2024-09-21T06:05:38.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.733+0000 7fd235cb5740 1 --2- 172.21.15.117:0/2258955506 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f5941da070 0x55f59460c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:38.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.733+0000 7fd235cb5740 4 client.17079 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:05:38.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.733+0000 7fd235cb5740 1 -- 172.21.15.117:0/2258955506 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f5944f58a0 con 0x55f5941da070 2024-09-21T06:05:38.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.734+0000 7fd22d4dd640 1 --2- 172.21.15.117:0/2258955506 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f5941da070 0x55f59460c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:38.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.734+0000 7fd22d4dd640 1 --2- 172.21.15.117:0/2258955506 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f5941da070 0x55f59460c710 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7aab0ceefd3d26c server_cookie=fe698e5f0e61dd35 in_seq=0 out_seq=0 2024-09-21T06:05:38.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.734+0000 7fd210ff9640 10 client.17079 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:38.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.734+0000 7fd210ff9640 1 -- 172.21.15.117:0/2258955506 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f5944f58a0 con 0x55f5941da070 2024-09-21T06:05:38.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.734+0000 7fd210ff9640 10 client.17079 handle_command_reply: tid=0 2024-09-21T06:05:38.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.734+0000 7fd235cb5740 1 client.17079 shutdown 2024-09-21T06:05:38.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.734+0000 7fd235cb5740 1 -- 172.21.15.117:0/2258955506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5941b11b0 msgr2=0x55f594591940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:38.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.734+0000 7fd235cb5740 1 --2- 172.21.15.117:0/2258955506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5941b11b0 0x55f594591940 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fd2180e7db0 tx=0x7fd2180a98c0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.734+0000 7fd235cb5740 1 -- 172.21.15.117:0/2258955506 shutdown_connections 2024-09-21T06:05:38.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.734+0000 7fd235cb5740 1 --2- 172.21.15.117:0/2258955506 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f5941da070 0x55f59460c710 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.734+0000 7fd235cb5740 1 --2- 172.21.15.117:0/2258955506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5941b11b0 0x55f594591940 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.734+0000 7fd235cb5740 1 --2- 172.21.15.117:0/2258955506 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5941b9390 0x55f5946087a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.734+0000 7fd235cb5740 1 --2- 172.21.15.117:0/2258955506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5941989f0 0x55f594591400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.734+0000 7fd235cb5740 1 -- 172.21.15.117:0/2258955506 >> 172.21.15.117:0/2258955506 conn(0x55f5941ce460 msgr2=0x55f59458e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:38.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.734+0000 7fd210ff9640 0 client.17079 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:38.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.735+0000 7fd235cb5740 1 -- 172.21.15.117:0/2258955506 shutdown_connections 2024-09-21T06:05:38.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.735+0000 7fd235cb5740 1 -- 172.21.15.117:0/2258955506 wait complete. 2024-09-21T06:05:38.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.735+0000 7fd235cb5740 20 client.17079 trim_cache size 0 max 16384 2024-09-21T06:05:38.742 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:05:38.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.735+0000 7fd22f741640 1 -- 172.21.15.117:0/3694330413 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd2140b6680 msgr2=0x7fd2140b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:38.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.735+0000 7fd22f741640 1 --2- 172.21.15.117:0/3694330413 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd2140b6680 0x7fd2140b8b90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd220005fd0 tx=0x7fd220009600 comp rx=0 tx=0).stop 2024-09-21T06:05:38.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.735+0000 7fd22f741640 1 -- 172.21.15.117:0/3694330413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd228105380 msgr2=0x7fd22819aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:38.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.735+0000 7fd22f741640 1 --2- 172.21.15.117:0/3694330413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd228105380 0x7fd22819aae0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fd21800e9c0 tx=0x7fd21800ee80 comp rx=0 tx=0).stop 2024-09-21T06:05:38.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.735+0000 7fd22f741640 1 -- 172.21.15.117:0/3694330413 shutdown_connections 2024-09-21T06:05:38.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.735+0000 7fd22f741640 1 --2- 172.21.15.117:0/3694330413 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd2140b6680 0x7fd2140b8b90 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.735+0000 7fd22f741640 1 --2- 172.21.15.117:0/3694330413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd228105380 0x7fd22819aae0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.735+0000 7fd22f741640 1 --2- 172.21.15.117:0/3694330413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd22810de90 0x7fd22819fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.735+0000 7fd22f741640 1 --2- 172.21.15.117:0/3694330413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd228104a30 0x7fd22819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:38.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.735+0000 7fd22f741640 1 -- 172.21.15.117:0/3694330413 >> 172.21.15.117:0/3694330413 conn(0x7fd228079770 msgr2=0x7fd228100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:38.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.735+0000 7fd22f741640 1 -- 172.21.15.117:0/3694330413 shutdown_connections 2024-09-21T06:05:38.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:38.735+0000 7fd22f741640 1 -- 172.21.15.117:0/3694330413 wait complete. 2024-09-21T06:05:40.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:39 smithi192 ceph-mon[34700]: pgmap v580: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T06:05:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.243+0000 7f5c6e425640 1 -- 172.21.15.117:0/1370809877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c681082e0 msgr2=0x7f5c681086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.243+0000 7f5c6e425640 1 --2- 172.21.15.117:0/1370809877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c681082e0 0x7f5c681086f0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f5c50009900 tx=0x7f5c5002a410 comp rx=0 tx=0).stop 2024-09-21T06:05:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.243+0000 7f5c6e425640 1 -- 172.21.15.117:0/1370809877 shutdown_connections 2024-09-21T06:05:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.243+0000 7f5c6e425640 1 --2- 172.21.15.117:0/1370809877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c681082e0 0x7f5c681086f0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.243+0000 7f5c6e425640 1 --2- 172.21.15.117:0/1370809877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c68108c30 0x7f5c6810f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.243+0000 7f5c6e425640 1 --2- 172.21.15.117:0/1370809877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c681029b0 0x7f5c68102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.243+0000 7f5c6e425640 1 -- 172.21.15.117:0/1370809877 >> 172.21.15.117:0/1370809877 conn(0x7f5c6807a740 msgr2=0x7f5c6807ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.243+0000 7f5c6e425640 1 -- 172.21.15.117:0/1370809877 shutdown_connections 2024-09-21T06:05:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.243+0000 7f5c6e425640 1 -- 172.21.15.117:0/1370809877 wait complete. 2024-09-21T06:05:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.244+0000 7f5c6e425640 1 Processor -- start 2024-09-21T06:05:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.244+0000 7f5c6e425640 1 -- start start 2024-09-21T06:05:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.245+0000 7f5c6e425640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c681029b0 0x7f5c6819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.245+0000 7f5c6e425640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c681082e0 0x7f5c6819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.245+0000 7f5c6e425640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c68108c30 0x7f5c6819fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.245+0000 7f5c6e425640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5c681128f0 con 0x7f5c68108c30 2024-09-21T06:05:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.245+0000 7f5c6e425640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5c68112770 con 0x7f5c681082e0 2024-09-21T06:05:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.245+0000 7f5c6e425640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5c68112a70 con 0x7f5c681029b0 2024-09-21T06:05:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.245+0000 7f5c6c9c2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c68108c30 0x7f5c6819fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.245+0000 7f5c6c9c2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c68108c30 0x7f5c6819fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53972/0 (socket says 172.21.15.117:53972) 2024-09-21T06:05:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.245+0000 7f5c6c9c2640 1 -- 172.21.15.117:0/478170835 learned_addr learned my addr 172.21.15.117:0/478170835 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.245+0000 7f5c677fe640 1 --2- 172.21.15.117:0/478170835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c681082e0 0x7f5c6819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.245+0000 7f5c67fff640 1 --2- 172.21.15.117:0/478170835 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c681029b0 0x7f5c6819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.245+0000 7f5c6c9c2640 1 -- 172.21.15.117:0/478170835 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c681029b0 msgr2=0x7f5c6819a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.245+0000 7f5c6c9c2640 1 --2- 172.21.15.117:0/478170835 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c681029b0 0x7f5c6819a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.245+0000 7f5c6c9c2640 1 -- 172.21.15.117:0/478170835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c681082e0 msgr2=0x7f5c6819aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.245+0000 7f5c6c9c2640 1 --2- 172.21.15.117:0/478170835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c681082e0 0x7f5c6819aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.245+0000 7f5c6c9c2640 1 -- 172.21.15.117:0/478170835 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c681a0450 con 0x7f5c68108c30 2024-09-21T06:05:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.245+0000 7f5c677fe640 1 --2- 172.21.15.117:0/478170835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c681082e0 0x7f5c6819aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:05:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.245+0000 7f5c67fff640 1 --2- 172.21.15.117:0/478170835 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c681029b0 0x7f5c6819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:05:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.245+0000 7f5c6c9c2640 1 --2- 172.21.15.117:0/478170835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c68108c30 0x7f5c6819fd50 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f5c60009fd0 tx=0x7f5c6000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.245+0000 7f5c657fa640 1 -- 172.21.15.117:0/478170835 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c60017070 con 0x7f5c68108c30 2024-09-21T06:05:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.245+0000 7f5c6e425640 1 -- 172.21.15.117:0/478170835 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c681a0740 con 0x7f5c68108c30 2024-09-21T06:05:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.245+0000 7f5c657fa640 1 -- 172.21.15.117:0/478170835 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c60003e20 con 0x7f5c68108c30 2024-09-21T06:05:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.245+0000 7f5c657fa640 1 -- 172.21.15.117:0/478170835 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c60010460 con 0x7f5c68108c30 2024-09-21T06:05:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.245+0000 7f5c6e425640 1 -- 172.21.15.117:0/478170835 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c681a6ed0 con 0x7f5c68108c30 2024-09-21T06:05:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.248+0000 7f5c75474740 1 Processor -- start 2024-09-21T06:05:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.248+0000 7f5c657fa640 1 -- 172.21.15.117:0/478170835 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f5c60003970 con 0x7f5c68108c30 2024-09-21T06:05:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.248+0000 7f5c75474740 1 -- start start 2024-09-21T06:05:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.248+0000 7f5c75474740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641762b01b0 0x5641766559b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.248+0000 7f5c75474740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641762872f0 0x564176655ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.248+0000 7f5c75474740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56417628f4d0 0x564176656430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.248+0000 7f5c75474740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5641765ca6f0 con 0x5641762b01b0 2024-09-21T06:05:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.248+0000 7f5c75474740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5641765b9b20 con 0x5641762872f0 2024-09-21T06:05:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.248+0000 7f5c75474740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5641765bd930 con 0x56417628f4d0 2024-09-21T06:05:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.248+0000 7f5c677fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641762b01b0 0x5641766559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.248+0000 7f5c677fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641762b01b0 0x5641766559b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53980/0 (socket says 172.21.15.117:53980) 2024-09-21T06:05:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.248+0000 7f5c677fe640 1 -- 172.21.15.117:0/3364487074 learned_addr learned my addr 172.21.15.117:0/3364487074 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.249+0000 7f5c67fff640 1 --2- 172.21.15.117:0/3364487074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641762872f0 0x564176655ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.249+0000 7f5c657fa640 1 --2- 172.21.15.117:0/478170835 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5c540b6680 0x7f5c540b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.249+0000 7f5c677fe640 1 --2- 172.21.15.117:0/3364487074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56417628f4d0 0x564176656430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.249+0000 7f5c677fe640 1 -- 172.21.15.117:0/3364487074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56417628f4d0 msgr2=0x564176656430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.249+0000 7f5c677fe640 1 --2- 172.21.15.117:0/3364487074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56417628f4d0 0x564176656430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.249+0000 7f5c677fe640 1 -- 172.21.15.117:0/3364487074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641762872f0 msgr2=0x564176655ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.249+0000 7f5c677fe640 1 --2- 172.21.15.117:0/3364487074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641762872f0 0x564176655ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.249+0000 7f5c677fe640 1 -- 172.21.15.117:0/3364487074 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56417657f530 con 0x5641762b01b0 2024-09-21T06:05:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.249+0000 7f5c6c9c2640 1 --2- 172.21.15.117:0/478170835 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5c540b6680 0x7f5c540b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.249+0000 7f5c657fa640 1 -- 172.21.15.117:0/478170835 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f5c600e74b0 con 0x7f5c68108c30 2024-09-21T06:05:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.249+0000 7f5c677fe640 1 --2- 172.21.15.117:0/3364487074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641762b01b0 0x5641766559b0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f5c50005ab0 tx=0x7f5c50009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c21c0b1e75d27f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.249+0000 7f5c5e7fc640 1 -- 172.21.15.117:0/3364487074 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50035050 con 0x5641762b01b0 2024-09-21T06:05:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.249+0000 7f5c5e7fc640 1 -- 172.21.15.117:0/3364487074 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c50031620 con 0x5641762b01b0 2024-09-21T06:05:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.249+0000 7f5c5e7fc640 1 -- 172.21.15.117:0/3364487074 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50005640 con 0x5641762b01b0 2024-09-21T06:05:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.249+0000 7f5c6c9c2640 1 --2- 172.21.15.117:0/478170835 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5c540b6680 0x7f5c540b8b90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5c600e4d00 tx=0x7f5c60004020 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.250+0000 7f5c75474740 1 -- 172.21.15.117:0/3364487074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641762b01b0 msgr2=0x5641766559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.250+0000 7f5c75474740 1 --2- 172.21.15.117:0/3364487074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641762b01b0 0x5641766559b0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f5c50005ab0 tx=0x7f5c50009560 comp rx=0 tx=0).stop 2024-09-21T06:05:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.250+0000 7f5c75474740 1 -- 172.21.15.117:0/3364487074 shutdown_connections 2024-09-21T06:05:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.250+0000 7f5c75474740 1 --2- 172.21.15.117:0/3364487074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56417628f4d0 0x564176656430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.250+0000 7f5c75474740 1 --2- 172.21.15.117:0/3364487074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641762872f0 0x564176655ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.250+0000 7f5c75474740 1 --2- 172.21.15.117:0/3364487074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5641762b01b0 0x5641766559b0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.250+0000 7f5c75474740 1 -- 172.21.15.117:0/3364487074 >> 172.21.15.117:0/3364487074 conn(0x5641762a4300 msgr2=0x5641766538a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.250+0000 7f5c75474740 1 -- 172.21.15.117:0/3364487074 shutdown_connections 2024-09-21T06:05:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.250+0000 7f5c75474740 1 -- 172.21.15.117:0/3364487074 wait complete. 2024-09-21T06:05:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.251+0000 7f5c75474740 1 -- start start 2024-09-21T06:05:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.251+0000 7f5c75474740 1 Processor -- start 2024-09-21T06:05:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.251+0000 7f5c75474740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641762872f0 0x564176665470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.251+0000 7f5c75474740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56417628f4d0 0x5641766659b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.251+0000 7f5c75474740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5641762b01b0 0x5641766717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.251+0000 7f5c75474740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5641764f1ab0 con 0x56417628f4d0 2024-09-21T06:05:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.251+0000 7f5c75474740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56417655f730 con 0x5641762872f0 2024-09-21T06:05:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.251+0000 7f5c75474740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564176668490 con 0x5641762b01b0 2024-09-21T06:05:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.251+0000 7f5c75474740 10 client.0 _ll_register_callbacks cb 0x5641765b36b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:05:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.251+0000 7f5c67fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56417628f4d0 0x5641766659b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.251+0000 7f5c67fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56417628f4d0 0x5641766659b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53992/0 (socket says 172.21.15.117:53992) 2024-09-21T06:05:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.251+0000 7f5c67fff640 1 -- 172.21.15.117:0/652789146 learned_addr learned my addr 172.21.15.117:0/652789146 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.251+0000 7f5c677fe640 1 --2- 172.21.15.117:0/652789146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641762872f0 0x564176665470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.251+0000 7f5c677fe640 1 --2- 172.21.15.117:0/652789146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5641762b01b0 0x5641766717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.252+0000 7f5c67fff640 1 -- 172.21.15.117:0/652789146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5641762b01b0 msgr2=0x5641766717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.252+0000 7f5c67fff640 1 --2- 172.21.15.117:0/652789146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5641762b01b0 0x5641766717f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.252+0000 7f5c67fff640 1 -- 172.21.15.117:0/652789146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641762872f0 msgr2=0x564176665470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.252+0000 7f5c67fff640 1 --2- 172.21.15.117:0/652789146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641762872f0 0x564176665470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.252+0000 7f5c67fff640 1 -- 172.21.15.117:0/652789146 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56417631b790 con 0x56417628f4d0 2024-09-21T06:05:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.252+0000 7f5c67fff640 1 --2- 172.21.15.117:0/652789146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56417628f4d0 0x5641766659b0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f5c58009d40 tx=0x7f5c580039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afe497fbd1dca475 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.252+0000 7f5c477fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:05:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.252+0000 7f5c477fe640 1 -- 172.21.15.117:0/652789146 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c58013930 con 0x56417628f4d0 2024-09-21T06:05:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.252+0000 7f5c477fe640 1 -- 172.21.15.117:0/652789146 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f5c58013ad0 con 0x56417628f4d0 2024-09-21T06:05:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.252+0000 7f5c75474740 1 -- 172.21.15.117:0/652789146 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c50031620 con 0x56417628f4d0 2024-09-21T06:05:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.252+0000 7f5c477fe640 1 -- 172.21.15.117:0/652789146 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5c58013cd0 con 0x56417628f4d0 2024-09-21T06:05:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.252+0000 7f5c75474740 10 client.17091 fetch_fsmap learned FSMap version 29 2024-09-21T06:05:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.252+0000 7f5c75474740 1 -- 172.21.15.117:0/652789146 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564176394620 con 0x56417628f4d0 2024-09-21T06:05:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.252+0000 7f5c477fe640 1 -- 172.21.15.117:0/652789146 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c50031620 con 0x56417628f4d0 2024-09-21T06:05:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.252+0000 7f5c477fe640 1 -- 172.21.15.117:0/652789146 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5c5801a070 con 0x56417628f4d0 2024-09-21T06:05:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.253+0000 7f5c75474740 10 client.17091 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:05:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.253+0000 7f5c75474740 10 client.17091 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:05:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.253+0000 7f5c75474740 20 client.17091 populate_metadata read hostname 'smithi117' 2024-09-21T06:05:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.253+0000 7f5c75474740 1 --2- 172.21.15.117:0/652789146 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56417626ed80 0x564176677610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.253+0000 7f5c75474740 4 client.17091 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:05:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.253+0000 7f5c75474740 1 -- 172.21.15.117:0/652789146 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641765cc640 con 0x56417626ed80 2024-09-21T06:05:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.253+0000 7f5c677fe640 1 --2- 172.21.15.117:0/652789146 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56417626ed80 0x564176677610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.253+0000 7f5c477fe640 10 client.17091 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.253+0000 7f5c677fe640 1 --2- 172.21.15.117:0/652789146 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56417626ed80 0x564176677610 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c1d425fb970977 server_cookie=bd94ccdb71da599a in_seq=0 out_seq=0 2024-09-21T06:05:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.254+0000 7f5c477fe640 1 -- 172.21.15.117:0/652789146 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5641765cc640 con 0x56417626ed80 2024-09-21T06:05:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.254+0000 7f5c477fe640 10 client.17091 handle_command_reply: tid=0 2024-09-21T06:05:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.254+0000 7f5c75474740 1 client.17091 shutdown 2024-09-21T06:05:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.254+0000 7f5c75474740 1 -- 172.21.15.117:0/652789146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56417628f4d0 msgr2=0x5641766659b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.254+0000 7f5c75474740 1 --2- 172.21.15.117:0/652789146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56417628f4d0 0x5641766659b0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f5c58009d40 tx=0x7f5c580039c0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.254+0000 7f5c75474740 1 -- 172.21.15.117:0/652789146 shutdown_connections 2024-09-21T06:05:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.254+0000 7f5c75474740 1 --2- 172.21.15.117:0/652789146 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56417626ed80 0x564176677610 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.254+0000 7f5c75474740 1 --2- 172.21.15.117:0/652789146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56417628f4d0 0x5641766659b0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.254+0000 7f5c477fe640 0 client.17091 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.254+0000 7f5c75474740 1 --2- 172.21.15.117:0/652789146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5641762b01b0 0x5641766717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.254+0000 7f5c75474740 1 --2- 172.21.15.117:0/652789146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641762872f0 0x564176665470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.254+0000 7f5c75474740 1 -- 172.21.15.117:0/652789146 >> 172.21.15.117:0/652789146 conn(0x5641762a4300 msgr2=0x564176651ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.254+0000 7f5c75474740 1 -- 172.21.15.117:0/652789146 shutdown_connections 2024-09-21T06:05:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.254+0000 7f5c75474740 1 -- 172.21.15.117:0/652789146 wait complete. 2024-09-21T06:05:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.255+0000 7f5c75474740 20 client.17091 trim_cache size 0 max 16384 2024-09-21T06:05:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.269+0000 7f5c75474740 1 Processor -- start 2024-09-21T06:05:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.269+0000 7f5c75474740 1 -- start start 2024-09-21T06:05:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.269+0000 7f5c75474740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56417626ed80 0x5641766525b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.269+0000 7f5c75474740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641762872f0 0x564176665950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.269+0000 7f5c75474740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56417628f4d0 0x5641766716f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.269+0000 7f5c75474740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5c681ad2c0 con 0x56417628f4d0 2024-09-21T06:05:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.269+0000 7f5c75474740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5c681aee00 con 0x5641762872f0 2024-09-21T06:05:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.269+0000 7f5c75474740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5c681ad990 con 0x56417626ed80 2024-09-21T06:05:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.269+0000 7f5c677fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56417628f4d0 0x5641766716f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:40.272 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.269+0000 7f5c677fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56417628f4d0 0x5641766716f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54006/0 (socket says 172.21.15.117:54006) 2024-09-21T06:05:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.269+0000 7f5c677fe640 1 -- 172.21.15.117:0/1850131738 learned_addr learned my addr 172.21.15.117:0/1850131738 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.269+0000 7f5c677fe640 1 --2- 172.21.15.117:0/1850131738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56417626ed80 0x5641766525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.270+0000 7f5c67fff640 1 --2- 172.21.15.117:0/1850131738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641762872f0 0x564176665950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.270+0000 7f5c677fe640 1 -- 172.21.15.117:0/1850131738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56417626ed80 msgr2=0x5641766525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.270+0000 7f5c677fe640 1 --2- 172.21.15.117:0/1850131738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56417626ed80 0x5641766525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.270+0000 7f5c677fe640 1 -- 172.21.15.117:0/1850131738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641762872f0 msgr2=0x564176665950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.270+0000 7f5c677fe640 1 --2- 172.21.15.117:0/1850131738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641762872f0 0x564176665950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.270+0000 7f5c677fe640 1 -- 172.21.15.117:0/1850131738 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564176671c30 con 0x56417628f4d0 2024-09-21T06:05:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.270+0000 7f5c67fff640 1 --2- 172.21.15.117:0/1850131738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641762872f0 0x564176665950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:05:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.270+0000 7f5c677fe640 1 --2- 172.21.15.117:0/1850131738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56417628f4d0 0x5641766716f0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f5c50006010 tx=0x7f5c50009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25981347afd3d307 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.270+0000 7f5c5dffb640 1 -- 172.21.15.117:0/1850131738 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50035040 con 0x56417628f4d0 2024-09-21T06:05:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.270+0000 7f5c5dffb640 1 -- 172.21.15.117:0/1850131738 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c50031620 con 0x56417628f4d0 2024-09-21T06:05:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.270+0000 7f5c5dffb640 1 -- 172.21.15.117:0/1850131738 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50003860 con 0x56417628f4d0 2024-09-21T06:05:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.271+0000 7f5c75474740 1 -- 172.21.15.117:0/1850131738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56417628f4d0 msgr2=0x5641766716f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.271+0000 7f5c75474740 1 --2- 172.21.15.117:0/1850131738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56417628f4d0 0x5641766716f0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f5c50006010 tx=0x7f5c50009560 comp rx=0 tx=0).stop 2024-09-21T06:05:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.271+0000 7f5c75474740 1 -- 172.21.15.117:0/1850131738 shutdown_connections 2024-09-21T06:05:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.271+0000 7f5c75474740 1 --2- 172.21.15.117:0/1850131738 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56417628f4d0 0x5641766716f0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.271+0000 7f5c75474740 1 --2- 172.21.15.117:0/1850131738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641762872f0 0x564176665950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.271+0000 7f5c75474740 1 --2- 172.21.15.117:0/1850131738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56417626ed80 0x5641766525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.271+0000 7f5c75474740 1 -- 172.21.15.117:0/1850131738 >> 172.21.15.117:0/1850131738 conn(0x5641762a4300 msgr2=0x564176651ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.271+0000 7f5c75474740 1 -- 172.21.15.117:0/1850131738 shutdown_connections 2024-09-21T06:05:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.271+0000 7f5c75474740 1 -- 172.21.15.117:0/1850131738 wait complete. 2024-09-21T06:05:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.271+0000 7f5c75474740 1 -- start start 2024-09-21T06:05:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.271+0000 7f5c75474740 1 Processor -- start 2024-09-21T06:05:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.272+0000 7f5c75474740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56417626ed80 0x56417666e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.272+0000 7f5c75474740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641762872f0 0x564176669600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.272+0000 7f5c75474740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56417628f4d0 0x564176669d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.272+0000 7f5c75474740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5c681ae730 con 0x56417626ed80 2024-09-21T06:05:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.272+0000 7f5c75474740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5c681ae060 con 0x5641762872f0 2024-09-21T06:05:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.272+0000 7f5c75474740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5c681ab2e0 con 0x56417628f4d0 2024-09-21T06:05:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.272+0000 7f5c75474740 10 client.0 _ll_register_callbacks cb 0x7f5c58004160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:05:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.272+0000 7f5c677fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56417626ed80 0x56417666e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.272+0000 7f5c677fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56417626ed80 0x56417666e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54014/0 (socket says 172.21.15.117:54014) 2024-09-21T06:05:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.272+0000 7f5c677fe640 1 -- 172.21.15.117:0/3041421591 learned_addr learned my addr 172.21.15.117:0/3041421591 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.272+0000 7f5c677fe640 1 --2- 172.21.15.117:0/3041421591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56417628f4d0 0x564176669d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.272+0000 7f5c67fff640 1 --2- 172.21.15.117:0/3041421591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641762872f0 0x564176669600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.272+0000 7f5c677fe640 1 -- 172.21.15.117:0/3041421591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56417628f4d0 msgr2=0x564176669d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.272+0000 7f5c677fe640 1 --2- 172.21.15.117:0/3041421591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56417628f4d0 0x564176669d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.272+0000 7f5c677fe640 1 -- 172.21.15.117:0/3041421591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641762872f0 msgr2=0x564176669600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.272+0000 7f5c677fe640 1 --2- 172.21.15.117:0/3041421591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641762872f0 0x564176669600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.272+0000 7f5c677fe640 1 -- 172.21.15.117:0/3041421591 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641766e1020 con 0x56417626ed80 2024-09-21T06:05:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.272+0000 7f5c67fff640 1 --2- 172.21.15.117:0/3041421591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641762872f0 0x564176669600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:05:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.272+0000 7f5c677fe640 1 --2- 172.21.15.117:0/3041421591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56417626ed80 0x56417666e110 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f5c500059d0 tx=0x7f5c50003700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4929153c2bbf28fc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.272+0000 7f5c477fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:05:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.272+0000 7f5c477fe640 1 -- 172.21.15.117:0/3041421591 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c50035070 con 0x56417626ed80 2024-09-21T06:05:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.272+0000 7f5c477fe640 1 -- 172.21.15.117:0/3041421591 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f5c50004030 con 0x56417626ed80 2024-09-21T06:05:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.272+0000 7f5c75474740 1 -- 172.21.15.117:0/3041421591 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c50031620 con 0x56417626ed80 2024-09-21T06:05:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.272+0000 7f5c477fe640 1 -- 172.21.15.117:0/3041421591 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5c50004230 con 0x56417626ed80 2024-09-21T06:05:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.272+0000 7f5c75474740 10 client.17097 fetch_fsmap learned FSMap version 29 2024-09-21T06:05:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.272+0000 7f5c75474740 1 -- 172.21.15.117:0/3041421591 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5641766e20c0 con 0x56417626ed80 2024-09-21T06:05:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.273+0000 7f5c477fe640 1 -- 172.21.15.117:0/3041421591 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c50031620 con 0x56417626ed80 2024-09-21T06:05:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.273+0000 7f5c477fe640 1 -- 172.21.15.117:0/3041421591 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5c50004880 con 0x56417626ed80 2024-09-21T06:05:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.273+0000 7f5c75474740 10 client.17097 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:05:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.273+0000 7f5c75474740 10 client.17097 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:05:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.273+0000 7f5c75474740 20 client.17097 populate_metadata read hostname 'smithi117' 2024-09-21T06:05:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.273+0000 7f5c75474740 1 --2- 172.21.15.117:0/3041421591 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5641762b01b0 0x5641766e42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.273+0000 7f5c75474740 4 client.17097 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:05:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.273+0000 7f5c75474740 1 -- 172.21.15.117:0/3041421591 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5641765c80f0 con 0x5641762b01b0 2024-09-21T06:05:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.273+0000 7f5c67fff640 1 --2- 172.21.15.117:0/3041421591 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5641762b01b0 0x5641766e42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.274+0000 7f5c477fe640 10 client.17097 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.274+0000 7f5c67fff640 1 --2- 172.21.15.117:0/3041421591 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5641762b01b0 0x5641766e42f0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bccb056c767dc51 server_cookie=3fcbf7d8d0ca7477 in_seq=0 out_seq=0 2024-09-21T06:05:40.280 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:39 smithi117 ceph-mon[30947]: pgmap v580: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T06:05:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.280+0000 7f5c477fe640 1 -- 172.21.15.117:0/3041421591 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5641765c80f0 con 0x5641762b01b0 2024-09-21T06:05:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.280+0000 7f5c477fe640 10 client.17097 handle_command_reply: tid=0 2024-09-21T06:05:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.280+0000 7f5c75474740 1 client.17097 shutdown 2024-09-21T06:05:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.280+0000 7f5c75474740 1 -- 172.21.15.117:0/3041421591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56417626ed80 msgr2=0x56417666e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.280+0000 7f5c75474740 1 --2- 172.21.15.117:0/3041421591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56417626ed80 0x56417666e110 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f5c500059d0 tx=0x7f5c50003700 comp rx=0 tx=0).stop 2024-09-21T06:05:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.280+0000 7f5c75474740 1 -- 172.21.15.117:0/3041421591 shutdown_connections 2024-09-21T06:05:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.280+0000 7f5c75474740 1 --2- 172.21.15.117:0/3041421591 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5641762b01b0 0x5641766e42f0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.280+0000 7f5c75474740 1 --2- 172.21.15.117:0/3041421591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56417628f4d0 0x564176669d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.280+0000 7f5c75474740 1 --2- 172.21.15.117:0/3041421591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641762872f0 0x564176669600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.280+0000 7f5c477fe640 0 client.17097 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.280+0000 7f5c75474740 1 --2- 172.21.15.117:0/3041421591 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56417626ed80 0x56417666e110 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.280+0000 7f5c75474740 1 -- 172.21.15.117:0/3041421591 >> 172.21.15.117:0/3041421591 conn(0x5641762a4300 msgr2=0x5641766600f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.280+0000 7f5c75474740 1 -- 172.21.15.117:0/3041421591 shutdown_connections 2024-09-21T06:05:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.280+0000 7f5c75474740 1 -- 172.21.15.117:0/3041421591 wait complete. 2024-09-21T06:05:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.280+0000 7f5c75474740 20 client.17097 trim_cache size 0 max 16384 2024-09-21T06:05:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T06:05:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T06:05:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T06:05:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T06:05:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.281+0000 7f5c6e425640 1 -- 172.21.15.117:0/478170835 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5c540b6680 msgr2=0x7f5c540b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.281+0000 7f5c6e425640 1 --2- 172.21.15.117:0/478170835 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5c540b6680 0x7f5c540b8b90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5c600e4d00 tx=0x7f5c60004020 comp rx=0 tx=0).stop 2024-09-21T06:05:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.281+0000 7f5c6e425640 1 -- 172.21.15.117:0/478170835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c68108c30 msgr2=0x7f5c6819fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.281+0000 7f5c6e425640 1 --2- 172.21.15.117:0/478170835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c68108c30 0x7f5c6819fd50 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f5c60009fd0 tx=0x7f5c6000b260 comp rx=0 tx=0).stop 2024-09-21T06:05:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.281+0000 7f5c6e425640 1 -- 172.21.15.117:0/478170835 shutdown_connections 2024-09-21T06:05:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.281+0000 7f5c6e425640 1 --2- 172.21.15.117:0/478170835 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5c540b6680 0x7f5c540b8b90 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.281+0000 7f5c6e425640 1 --2- 172.21.15.117:0/478170835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c68108c30 0x7f5c6819fd50 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.281+0000 7f5c6e425640 1 --2- 172.21.15.117:0/478170835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c681082e0 0x7f5c6819aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.281+0000 7f5c6e425640 1 --2- 172.21.15.117:0/478170835 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c681029b0 0x7f5c6819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.281+0000 7f5c6e425640 1 -- 172.21.15.117:0/478170835 >> 172.21.15.117:0/478170835 conn(0x7f5c6807a740 msgr2=0x7f5c68101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.281+0000 7f5c6e425640 1 -- 172.21.15.117:0/478170835 shutdown_connections 2024-09-21T06:05:40.286 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:40.281+0000 7f5c6e425640 1 -- 172.21.15.117:0/478170835 wait complete. 2024-09-21T06:05:40.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:39 smithi138 ceph-mon[34391]: pgmap v580: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T06:05:40.653 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:05:40.653 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:05:40.905 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:05:41.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.462+0000 7f27c912f640 1 -- 172.21.15.117:0/305996450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27c4108d70 msgr2=0x7f27c410f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:41.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.463+0000 7f27c912f640 1 --2- 172.21.15.117:0/305996450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27c4108d70 0x7f27c410f5a0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009900 tx=0x7f27bc02a410 comp rx=0 tx=0).stop 2024-09-21T06:05:41.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.463+0000 7f27c912f640 1 -- 172.21.15.117:0/305996450 shutdown_connections 2024-09-21T06:05:41.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.463+0000 7f27c912f640 1 --2- 172.21.15.117:0/305996450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27c4108d70 0x7f27c410f5a0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:41.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.463+0000 7f27c912f640 1 --2- 172.21.15.117:0/305996450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27c4108420 0x7f27c4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:41.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.463+0000 7f27c912f640 1 --2- 172.21.15.117:0/305996450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27c4102ae0 0x7f27c4102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:41.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.463+0000 7f27c912f640 1 -- 172.21.15.117:0/305996450 >> 172.21.15.117:0/305996450 conn(0x7f27c40796b0 msgr2=0x7f27c4079ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:41.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.463+0000 7f27c912f640 1 -- 172.21.15.117:0/305996450 shutdown_connections 2024-09-21T06:05:41.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.463+0000 7f27c912f640 1 -- 172.21.15.117:0/305996450 wait complete. 2024-09-21T06:05:41.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.463+0000 7f27c912f640 1 Processor -- start 2024-09-21T06:05:41.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.464+0000 7f27c912f640 1 -- start start 2024-09-21T06:05:41.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.464+0000 7f27c912f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27c4102ae0 0x7f27c419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:41.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.464+0000 7f27c912f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27c4108420 0x7f27c419b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:41.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.464+0000 7f27c912f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27c4108d70 0x7f27c419bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:41.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.464+0000 7f27c912f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f27c4112a30 con 0x7f27c4102ae0 2024-09-21T06:05:41.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.464+0000 7f27c912f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f27c41128b0 con 0x7f27c4108d70 2024-09-21T06:05:41.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.464+0000 7f27c912f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f27c4112bb0 con 0x7f27c4108420 2024-09-21T06:05:41.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.464+0000 7f27c2d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27c4102ae0 0x7f27c419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:41.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.464+0000 7f27c2d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27c4102ae0 0x7f27c419b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54032/0 (socket says 172.21.15.117:54032) 2024-09-21T06:05:41.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.464+0000 7f27c2d9d640 1 -- 172.21.15.117:0/1367374045 learned_addr learned my addr 172.21.15.117:0/1367374045 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:41.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.464+0000 7f27c259c640 1 --2- 172.21.15.117:0/1367374045 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27c4108420 0x7f27c419b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:41.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.464+0000 7f27c359e640 1 --2- 172.21.15.117:0/1367374045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27c4108d70 0x7f27c419bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:41.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.464+0000 7f27c2d9d640 1 -- 172.21.15.117:0/1367374045 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27c4108420 msgr2=0x7f27c419b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:41.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.464+0000 7f27c2d9d640 1 --2- 172.21.15.117:0/1367374045 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27c4108420 0x7f27c419b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:41.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.464+0000 7f27c2d9d640 1 -- 172.21.15.117:0/1367374045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27c4108d70 msgr2=0x7f27c419bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:41.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.464+0000 7f27c2d9d640 1 --2- 172.21.15.117:0/1367374045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27c4108d70 0x7f27c419bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:41.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.464+0000 7f27c2d9d640 1 -- 172.21.15.117:0/1367374045 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c41a65d0 con 0x7f27c4102ae0 2024-09-21T06:05:41.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.464+0000 7f27c359e640 1 --2- 172.21.15.117:0/1367374045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27c4108d70 0x7f27c419bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:05:41.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.465+0000 7f27c2d9d640 1 --2- 172.21.15.117:0/1367374045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27c4102ae0 0x7f27c419b340 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f27b400eb90 tx=0x7f27b400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:41.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.465+0000 7f27abfff640 1 -- 172.21.15.117:0/1367374045 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b4004d20 con 0x7f27c4102ae0 2024-09-21T06:05:41.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.465+0000 7f27c912f640 1 -- 172.21.15.117:0/1367374045 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27c41a68c0 con 0x7f27c4102ae0 2024-09-21T06:05:41.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.465+0000 7f27abfff640 1 -- 172.21.15.117:0/1367374045 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27b400ee90 con 0x7f27c4102ae0 2024-09-21T06:05:41.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.465+0000 7f27abfff640 1 -- 172.21.15.117:0/1367374045 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b4010670 con 0x7f27c4102ae0 2024-09-21T06:05:41.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.465+0000 7f27c912f640 1 -- 172.21.15.117:0/1367374045 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27c41a6e00 con 0x7f27c4102ae0 2024-09-21T06:05:41.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.467+0000 7f27abfff640 1 -- 172.21.15.117:0/1367374045 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f27b4010830 con 0x7f27c4102ae0 2024-09-21T06:05:41.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.468+0000 7f27cf6a3740 1 Processor -- start 2024-09-21T06:05:41.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.468+0000 7f27cf6a3740 1 -- start start 2024-09-21T06:05:41.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.468+0000 7f27cf6a3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db683c1350 0x55db68768430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:41.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.468+0000 7f27cf6a3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db68398490 0x55db68768970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:41.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.468+0000 7f27cf6a3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db683a0670 0x55db68768eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:41.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.468+0000 7f27cf6a3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55db686dee60 con 0x55db683a0670 2024-09-21T06:05:41.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.468+0000 7f27cf6a3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55db686ce290 con 0x55db683c1350 2024-09-21T06:05:41.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.468+0000 7f27cf6a3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55db686d20a0 con 0x55db68398490 2024-09-21T06:05:41.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.468+0000 7f27c259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db683a0670 0x55db68768eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:41.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.468+0000 7f27c259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db683a0670 0x55db68768eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54044/0 (socket says 172.21.15.117:54044) 2024-09-21T06:05:41.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.468+0000 7f27c259c640 1 -- 172.21.15.117:0/934784519 learned_addr learned my addr 172.21.15.117:0/934784519 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:41.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.468+0000 7f27c359e640 1 --2- 172.21.15.117:0/934784519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db683c1350 0x55db68768430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:41.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.468+0000 7f27abfff640 1 --2- 172.21.15.117:0/1367374045 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f27ac0b66b0 0x7f27ac0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:41.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.468+0000 7f27c2d9d640 1 --2- 172.21.15.117:0/934784519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db68398490 0x55db68768970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:41.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.468+0000 7f27c359e640 1 --2- 172.21.15.117:0/1367374045 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f27ac0b66b0 0x7f27ac0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:41.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.468+0000 7f27c259c640 1 -- 172.21.15.117:0/934784519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db68398490 msgr2=0x55db68768970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:41.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.468+0000 7f27c259c640 1 --2- 172.21.15.117:0/934784519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db68398490 0x55db68768970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:41.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.468+0000 7f27c259c640 1 -- 172.21.15.117:0/934784519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db683c1350 msgr2=0x55db68768430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:41.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.468+0000 7f27c259c640 1 --2- 172.21.15.117:0/934784519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db683c1350 0x55db68768430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:41.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.468+0000 7f27c259c640 1 -- 172.21.15.117:0/934784519 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db6842c930 con 0x55db683a0670 2024-09-21T06:05:41.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.469+0000 7f27abfff640 1 -- 172.21.15.117:0/1367374045 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f27b40e7dc0 con 0x7f27c4102ae0 2024-09-21T06:05:41.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.469+0000 7f27c259c640 1 --2- 172.21.15.117:0/934784519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db683a0670 0x55db68768eb0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f27b800bbb0 tx=0x7f27b80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ee8bff7499e51b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:41.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.469+0000 7f27a97fa640 1 -- 172.21.15.117:0/934784519 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b8004030 con 0x55db683a0670 2024-09-21T06:05:41.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.469+0000 7f27c359e640 1 --2- 172.21.15.117:0/1367374045 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f27ac0b66b0 0x7f27ac0b8bc0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f27bc005940 tx=0x7f27bc02a410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:41.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.469+0000 7f27a97fa640 1 -- 172.21.15.117:0/934784519 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27b8015850 con 0x55db683a0670 2024-09-21T06:05:41.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.469+0000 7f27a97fa640 1 -- 172.21.15.117:0/934784519 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b8005420 con 0x55db683a0670 2024-09-21T06:05:41.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.470+0000 7f27cf6a3740 1 -- 172.21.15.117:0/934784519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db683a0670 msgr2=0x55db68768eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:41.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.470+0000 7f27cf6a3740 1 --2- 172.21.15.117:0/934784519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db683a0670 0x55db68768eb0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f27b800bbb0 tx=0x7f27b80107a0 comp rx=0 tx=0).stop 2024-09-21T06:05:41.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.470+0000 7f27cf6a3740 1 -- 172.21.15.117:0/934784519 shutdown_connections 2024-09-21T06:05:41.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.470+0000 7f27cf6a3740 1 --2- 172.21.15.117:0/934784519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db683a0670 0x55db68768eb0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:41.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.470+0000 7f27cf6a3740 1 --2- 172.21.15.117:0/934784519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db68398490 0x55db68768970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:41.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.470+0000 7f27cf6a3740 1 --2- 172.21.15.117:0/934784519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db683c1350 0x55db68768430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:41.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.470+0000 7f27cf6a3740 1 -- 172.21.15.117:0/934784519 >> 172.21.15.117:0/934784519 conn(0x55db683b5740 msgr2=0x55db68766320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:41.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.470+0000 7f27cf6a3740 1 -- 172.21.15.117:0/934784519 shutdown_connections 2024-09-21T06:05:41.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.470+0000 7f27cf6a3740 1 -- 172.21.15.117:0/934784519 wait complete. 2024-09-21T06:05:41.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.471+0000 7f27cf6a3740 1 -- start start 2024-09-21T06:05:41.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.471+0000 7f27cf6a3740 1 Processor -- start 2024-09-21T06:05:41.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.471+0000 7f27cf6a3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db68398490 0x55db68773fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:41.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.471+0000 7f27cf6a3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db683a0670 0x55db687744f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:41.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.471+0000 7f27cf6a3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db683c1350 0x55db6877ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:41.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.471+0000 7f27cf6a3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55db6852f890 con 0x55db68398490 2024-09-21T06:05:41.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.471+0000 7f27cf6a3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55db68668d40 con 0x55db683a0670 2024-09-21T06:05:41.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.471+0000 7f27cf6a3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55db68776c10 con 0x55db683c1350 2024-09-21T06:05:41.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.471+0000 7f27cf6a3740 10 client.0 _ll_register_callbacks cb 0x55db686ccc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:05:41.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.471+0000 7f27c2d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db68398490 0x55db68773fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:41.481 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.471+0000 7f27c2d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db68398490 0x55db68773fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54046/0 (socket says 172.21.15.117:54046) 2024-09-21T06:05:41.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.471+0000 7f27c2d9d640 1 -- 172.21.15.117:0/2471138467 learned_addr learned my addr 172.21.15.117:0/2471138467 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:41.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.471+0000 7f27c359e640 1 --2- 172.21.15.117:0/2471138467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db683c1350 0x55db6877ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:41.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.471+0000 7f27c259c640 1 --2- 172.21.15.117:0/2471138467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db683a0670 0x55db687744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:41.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.472+0000 7f27c2d9d640 1 -- 172.21.15.117:0/2471138467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db683c1350 msgr2=0x55db6877ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:41.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.472+0000 7f27c2d9d640 1 --2- 172.21.15.117:0/2471138467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db683c1350 0x55db6877ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:41.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.472+0000 7f27c2d9d640 1 -- 172.21.15.117:0/2471138467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db683a0670 msgr2=0x55db687744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:41.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.472+0000 7f27c2d9d640 1 --2- 172.21.15.117:0/2471138467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db683a0670 0x55db687744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:41.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.472+0000 7f27c2d9d640 1 -- 172.21.15.117:0/2471138467 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db68780490 con 0x55db68398490 2024-09-21T06:05:41.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.472+0000 7f27c359e640 1 --2- 172.21.15.117:0/2471138467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db683c1350 0x55db6877ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:05:41.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.472+0000 7f27c2d9d640 1 --2- 172.21.15.117:0/2471138467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db68398490 0x55db68773fb0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f27b40e5560 tx=0x7f27b4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e940328947c0b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:41.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.472+0000 7f27a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:05:41.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.472+0000 7f27a6ff5640 1 -- 172.21.15.117:0/2471138467 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b40ad550 con 0x55db68398490 2024-09-21T06:05:41.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.472+0000 7f27a6ff5640 1 -- 172.21.15.117:0/2471138467 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f27b40b21d0 con 0x55db68398490 2024-09-21T06:05:41.482 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.472+0000 7f27cf6a3740 1 -- 172.21.15.117:0/2471138467 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27b8015850 con 0x55db68398490 2024-09-21T06:05:41.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.472+0000 7f27a6ff5640 1 -- 172.21.15.117:0/2471138467 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f27b40ad370 con 0x55db68398490 2024-09-21T06:05:41.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.472+0000 7f27cf6a3740 10 client.17109 fetch_fsmap learned FSMap version 29 2024-09-21T06:05:41.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.472+0000 7f27cf6a3740 1 -- 172.21.15.117:0/2471138467 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55db68782dc0 con 0x55db68398490 2024-09-21T06:05:41.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.472+0000 7f27a6ff5640 1 -- 172.21.15.117:0/2471138467 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27b40ad710 con 0x55db68398490 2024-09-21T06:05:41.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.472+0000 7f27a6ff5640 1 -- 172.21.15.117:0/2471138467 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f27b40a9550 con 0x55db68398490 2024-09-21T06:05:41.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.473+0000 7f27cf6a3740 10 client.17109 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:05:41.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.473+0000 7f27cf6a3740 10 client.17109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:05:41.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.473+0000 7f27cf6a3740 20 client.17109 populate_metadata read hostname 'smithi117' 2024-09-21T06:05:41.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.473+0000 7f27cf6a3740 1 --2- 172.21.15.117:0/2471138467 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55db6837fcd0 0x55db687862d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:41.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.473+0000 7f27cf6a3740 4 client.17109 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:05:41.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.473+0000 7f27cf6a3740 1 -- 172.21.15.117:0/2471138467 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f27b8015850 con 0x55db6837fcd0 2024-09-21T06:05:41.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.473+0000 7f27c259c640 1 --2- 172.21.15.117:0/2471138467 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55db6837fcd0 0x55db687862d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:41.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.473+0000 7f27c259c640 1 --2- 172.21.15.117:0/2471138467 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55db6837fcd0 0x55db687862d0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c3654af750bae6a server_cookie=bb76156a9d1abf69 in_seq=0 out_seq=0 2024-09-21T06:05:41.483 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.473+0000 7f27a6ff5640 10 client.17109 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:41.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.474+0000 7f27a6ff5640 1 -- 172.21.15.117:0/2471138467 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f27b8015850 con 0x55db6837fcd0 2024-09-21T06:05:41.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.474+0000 7f27a6ff5640 10 client.17109 handle_command_reply: tid=0 2024-09-21T06:05:41.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.474+0000 7f27cf6a3740 1 client.17109 shutdown 2024-09-21T06:05:41.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.474+0000 7f27cf6a3740 1 -- 172.21.15.117:0/2471138467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db68398490 msgr2=0x55db68773fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:41.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.474+0000 7f27cf6a3740 1 --2- 172.21.15.117:0/2471138467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db68398490 0x55db68773fb0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f27b40e5560 tx=0x7f27b4017000 comp rx=0 tx=0).stop 2024-09-21T06:05:41.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.475+0000 7f27cf6a3740 1 -- 172.21.15.117:0/2471138467 shutdown_connections 2024-09-21T06:05:41.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.475+0000 7f27cf6a3740 1 --2- 172.21.15.117:0/2471138467 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55db6837fcd0 0x55db687862d0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:41.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.475+0000 7f27cf6a3740 1 --2- 172.21.15.117:0/2471138467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db683c1350 0x55db6877ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:41.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.475+0000 7f27a6ff5640 0 client.17109 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:41.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.475+0000 7f27cf6a3740 1 --2- 172.21.15.117:0/2471138467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db683a0670 0x55db687744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:41.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.475+0000 7f27cf6a3740 1 --2- 172.21.15.117:0/2471138467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db68398490 0x55db68773fb0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:41.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.475+0000 7f27cf6a3740 1 -- 172.21.15.117:0/2471138467 >> 172.21.15.117:0/2471138467 conn(0x55db683b5740 msgr2=0x55db6876b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:41.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.475+0000 7f27cf6a3740 1 -- 172.21.15.117:0/2471138467 shutdown_connections 2024-09-21T06:05:41.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.475+0000 7f27cf6a3740 1 -- 172.21.15.117:0/2471138467 wait complete. 2024-09-21T06:05:41.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.475+0000 7f27cf6a3740 20 client.17109 trim_cache size 0 max 16384 2024-09-21T06:05:41.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.492+0000 7f27cf6a3740 1 Processor -- start 2024-09-21T06:05:41.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.492+0000 7f27cf6a3740 1 -- start start 2024-09-21T06:05:41.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.492+0000 7f27cf6a3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db68398490 0x55db6877fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:41.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.492+0000 7f27cf6a3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db683a0670 0x55db68770fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:41.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.492+0000 7f27cf6a3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db683c1350 0x55db687714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:41.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.492+0000 7f27cf6a3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f27c41ad3d0 con 0x55db683a0670 2024-09-21T06:05:41.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.492+0000 7f27cf6a3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f27c41aef10 con 0x55db683c1350 2024-09-21T06:05:41.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.492+0000 7f27cf6a3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f27c41adaa0 con 0x55db68398490 2024-09-21T06:05:41.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.492+0000 7f27c259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db683a0670 0x55db68770fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:41.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.492+0000 7f27c259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db683a0670 0x55db68770fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54050/0 (socket says 172.21.15.117:54050) 2024-09-21T06:05:41.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.492+0000 7f27c259c640 1 -- 172.21.15.117:0/3148411358 learned_addr learned my addr 172.21.15.117:0/3148411358 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:41.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.493+0000 7f27c359e640 1 --2- 172.21.15.117:0/3148411358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db683c1350 0x55db687714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:41.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.493+0000 7f27c2d9d640 1 --2- 172.21.15.117:0/3148411358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db68398490 0x55db6877fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:41.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.493+0000 7f27c259c640 1 -- 172.21.15.117:0/3148411358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db68398490 msgr2=0x55db6877fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:41.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.493+0000 7f27c259c640 1 --2- 172.21.15.117:0/3148411358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db68398490 0x55db6877fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:41.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.493+0000 7f27c259c640 1 -- 172.21.15.117:0/3148411358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db683c1350 msgr2=0x55db687714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:41.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.493+0000 7f27c259c640 1 --2- 172.21.15.117:0/3148411358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db683c1350 0x55db687714e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:41.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.493+0000 7f27c259c640 1 -- 172.21.15.117:0/3148411358 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db6876f640 con 0x55db683a0670 2024-09-21T06:05:41.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.493+0000 7f27c359e640 1 --2- 172.21.15.117:0/3148411358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db683c1350 0x55db687714e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:05:41.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.493+0000 7f27c2d9d640 1 --2- 172.21.15.117:0/3148411358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db68398490 0x55db6877fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:05:41.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.493+0000 7f27c259c640 1 --2- 172.21.15.117:0/3148411358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db683a0670 0x55db68770fa0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f27b800bb80 tx=0x7f27b80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94346ea9abcca70 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:41.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.493+0000 7f27a8ff9640 1 -- 172.21.15.117:0/3148411358 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b8015c80 con 0x55db683a0670 2024-09-21T06:05:41.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.493+0000 7f27a8ff9640 1 -- 172.21.15.117:0/3148411358 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27b800b7b0 con 0x55db683a0670 2024-09-21T06:05:41.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.493+0000 7f27a8ff9640 1 -- 172.21.15.117:0/3148411358 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b8003af0 con 0x55db683a0670 2024-09-21T06:05:41.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.494+0000 7f27cf6a3740 1 -- 172.21.15.117:0/3148411358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db683a0670 msgr2=0x55db68770fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:41.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.494+0000 7f27cf6a3740 1 --2- 172.21.15.117:0/3148411358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db683a0670 0x55db68770fa0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f27b800bb80 tx=0x7f27b80107a0 comp rx=0 tx=0).stop 2024-09-21T06:05:41.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.494+0000 7f27cf6a3740 1 -- 172.21.15.117:0/3148411358 shutdown_connections 2024-09-21T06:05:41.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.494+0000 7f27cf6a3740 1 --2- 172.21.15.117:0/3148411358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db683a0670 0x55db68770fa0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:41.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.494+0000 7f27cf6a3740 1 --2- 172.21.15.117:0/3148411358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db683c1350 0x55db687714e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:41.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.494+0000 7f27cf6a3740 1 --2- 172.21.15.117:0/3148411358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db68398490 0x55db6877fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:41.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.494+0000 7f27cf6a3740 1 -- 172.21.15.117:0/3148411358 >> 172.21.15.117:0/3148411358 conn(0x55db683b5740 msgr2=0x55db68772630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:41.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.494+0000 7f27cf6a3740 1 -- 172.21.15.117:0/3148411358 shutdown_connections 2024-09-21T06:05:41.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.494+0000 7f27cf6a3740 1 -- 172.21.15.117:0/3148411358 wait complete. 2024-09-21T06:05:41.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.494+0000 7f27cf6a3740 1 -- start start 2024-09-21T06:05:41.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.494+0000 7f27cf6a3740 1 Processor -- start 2024-09-21T06:05:41.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.495+0000 7f27cf6a3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db68398490 0x55db6877c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:41.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.495+0000 7f27cf6a3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db683a0670 0x55db68778cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:41.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.495+0000 7f27cf6a3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db683c1350 0x55db687793a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:41.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.495+0000 7f27cf6a3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f27c41ae840 con 0x55db683c1350 2024-09-21T06:05:41.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.495+0000 7f27cf6a3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f27c41ae170 con 0x55db68398490 2024-09-21T06:05:41.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.495+0000 7f27cf6a3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f27c41ab3d0 con 0x55db683a0670 2024-09-21T06:05:41.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.495+0000 7f27cf6a3740 10 client.0 _ll_register_callbacks cb 0x7f27b40e59b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:05:41.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.495+0000 7f27c359e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db683c1350 0x55db687793a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:41.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.495+0000 7f27c359e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db683c1350 0x55db687793a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54058/0 (socket says 172.21.15.117:54058) 2024-09-21T06:05:41.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.495+0000 7f27c359e640 1 -- 172.21.15.117:0/1523393644 learned_addr learned my addr 172.21.15.117:0/1523393644 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:41.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.495+0000 7f27c2d9d640 1 --2- 172.21.15.117:0/1523393644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db68398490 0x55db6877c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:41.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.495+0000 7f27c259c640 1 --2- 172.21.15.117:0/1523393644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db683a0670 0x55db68778cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:41.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.495+0000 7f27c359e640 1 -- 172.21.15.117:0/1523393644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db683a0670 msgr2=0x55db68778cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:41.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.495+0000 7f27c359e640 1 --2- 172.21.15.117:0/1523393644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db683a0670 0x55db68778cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:41.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.495+0000 7f27c359e640 1 -- 172.21.15.117:0/1523393644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db68398490 msgr2=0x55db6877c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:41.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.495+0000 7f27c359e640 1 --2- 172.21.15.117:0/1523393644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db68398490 0x55db6877c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:41.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.495+0000 7f27c359e640 1 -- 172.21.15.117:0/1523393644 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db687f0c50 con 0x55db683c1350 2024-09-21T06:05:41.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.495+0000 7f27c2d9d640 1 --2- 172.21.15.117:0/1523393644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db68398490 0x55db6877c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:05:41.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.495+0000 7f27c359e640 1 --2- 172.21.15.117:0/1523393644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db683c1350 0x55db687793a0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009900 tx=0x7f27bc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21917e7327c40bf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:41.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.495+0000 7f27a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:05:41.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.496+0000 7f27a6ff5640 1 -- 172.21.15.117:0/1523393644 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27bc04f020 con 0x55db683c1350 2024-09-21T06:05:41.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.496+0000 7f27a6ff5640 1 -- 172.21.15.117:0/1523393644 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f27bc005040 con 0x55db683c1350 2024-09-21T06:05:41.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.496+0000 7f27cf6a3740 1 -- 172.21.15.117:0/1523393644 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55db686e0db0 con 0x55db683c1350 2024-09-21T06:05:41.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.496+0000 7f27a6ff5640 1 -- 172.21.15.117:0/1523393644 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f27bc05a050 con 0x55db683c1350 2024-09-21T06:05:41.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.496+0000 7f27cf6a3740 10 client.17115 fetch_fsmap learned FSMap version 29 2024-09-21T06:05:41.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.496+0000 7f27cf6a3740 1 -- 172.21.15.117:0/1523393644 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55db687f1cf0 con 0x55db683c1350 2024-09-21T06:05:41.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.496+0000 7f27a6ff5640 1 -- 172.21.15.117:0/1523393644 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55db686e0db0 con 0x55db683c1350 2024-09-21T06:05:41.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.496+0000 7f27a6ff5640 1 -- 172.21.15.117:0/1523393644 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f27bc004c60 con 0x55db683c1350 2024-09-21T06:05:41.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.496+0000 7f27cf6a3740 10 client.17115 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:05:41.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.496+0000 7f27cf6a3740 10 client.17115 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:05:41.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.496+0000 7f27cf6a3740 20 client.17115 populate_metadata read hostname 'smithi117' 2024-09-21T06:05:41.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.496+0000 7f27cf6a3740 1 --2- 172.21.15.117:0/1523393644 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55db68355080 0x55db687f4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:41.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.496+0000 7f27cf6a3740 4 client.17115 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:05:41.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.496+0000 7f27cf6a3740 1 -- 172.21.15.117:0/1523393644 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55db686dc860 con 0x55db68355080 2024-09-21T06:05:41.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.497+0000 7f27c2d9d640 1 --2- 172.21.15.117:0/1523393644 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55db68355080 0x55db687f4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:41.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.497+0000 7f27c2d9d640 1 --2- 172.21.15.117:0/1523393644 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55db68355080 0x55db687f4530 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ddaf113aaa852a7 server_cookie=fd7d4f1ae5b784f5 in_seq=0 out_seq=0 2024-09-21T06:05:41.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.497+0000 7f27a6ff5640 10 client.17115 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:41.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.499+0000 7f27a6ff5640 1 -- 172.21.15.117:0/1523393644 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55db686dc860 con 0x55db68355080 2024-09-21T06:05:41.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.499+0000 7f27a6ff5640 10 client.17115 handle_command_reply: tid=0 2024-09-21T06:05:41.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.499+0000 7f27cf6a3740 1 client.17115 shutdown 2024-09-21T06:05:41.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.499+0000 7f27cf6a3740 1 -- 172.21.15.117:0/1523393644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db683c1350 msgr2=0x55db687793a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:41.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.499+0000 7f27cf6a3740 1 --2- 172.21.15.117:0/1523393644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db683c1350 0x55db687793a0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009900 tx=0x7f27bc009260 comp rx=0 tx=0).stop 2024-09-21T06:05:41.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.499+0000 7f27cf6a3740 1 -- 172.21.15.117:0/1523393644 shutdown_connections 2024-09-21T06:05:41.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.499+0000 7f27cf6a3740 1 --2- 172.21.15.117:0/1523393644 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55db68355080 0x55db687f4530 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:41.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.499+0000 7f27cf6a3740 1 --2- 172.21.15.117:0/1523393644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55db683c1350 0x55db687793a0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:41.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.499+0000 7f27cf6a3740 1 --2- 172.21.15.117:0/1523393644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55db683a0670 0x55db68778cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:41.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.499+0000 7f27cf6a3740 1 --2- 172.21.15.117:0/1523393644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55db68398490 0x55db6877c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:41.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.499+0000 7f27a6ff5640 0 client.17115 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:41.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.499+0000 7f27cf6a3740 1 -- 172.21.15.117:0/1523393644 >> 172.21.15.117:0/1523393644 conn(0x55db683b5740 msgr2=0x55db68769460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:41.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.499+0000 7f27cf6a3740 1 -- 172.21.15.117:0/1523393644 shutdown_connections 2024-09-21T06:05:41.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.499+0000 7f27cf6a3740 1 -- 172.21.15.117:0/1523393644 wait complete. 2024-09-21T06:05:41.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.499+0000 7f27cf6a3740 20 client.17115 trim_cache size 0 max 16384 2024-09-21T06:05:41.505 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T06:05:41.505 INFO:teuthology.orchestra.run.smithi117.stdout: "return_code": 0, 2024-09-21T06:05:41.505 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_tag": "39eeca8a-6b4f-465f-af34-b54f77320d70", 2024-09-21T06:05:41.505 INFO:teuthology.orchestra.run.smithi117.stdout: "mode": "asynchronous" 2024-09-21T06:05:41.505 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T06:05:41.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.500+0000 7f27c912f640 1 -- 172.21.15.117:0/1367374045 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f27ac0b66b0 msgr2=0x7f27ac0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:41.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.500+0000 7f27c912f640 1 --2- 172.21.15.117:0/1367374045 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f27ac0b66b0 0x7f27ac0b8bc0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f27bc005940 tx=0x7f27bc02a410 comp rx=0 tx=0).stop 2024-09-21T06:05:41.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.500+0000 7f27c912f640 1 -- 172.21.15.117:0/1367374045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27c4102ae0 msgr2=0x7f27c419b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:41.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.500+0000 7f27c912f640 1 --2- 172.21.15.117:0/1367374045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27c4102ae0 0x7f27c419b340 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f27b400eb90 tx=0x7f27b400c6a0 comp rx=0 tx=0).stop 2024-09-21T06:05:41.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.500+0000 7f27c912f640 1 -- 172.21.15.117:0/1367374045 shutdown_connections 2024-09-21T06:05:41.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.500+0000 7f27c912f640 1 --2- 172.21.15.117:0/1367374045 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f27ac0b66b0 0x7f27ac0b8bc0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:41.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.500+0000 7f27c912f640 1 --2- 172.21.15.117:0/1367374045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27c4108d70 0x7f27c419bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:41.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.500+0000 7f27c912f640 1 --2- 172.21.15.117:0/1367374045 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27c4108420 0x7f27c419b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:41.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.500+0000 7f27c912f640 1 --2- 172.21.15.117:0/1367374045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27c4102ae0 0x7f27c419b340 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:41.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.500+0000 7f27c912f640 1 -- 172.21.15.117:0/1367374045 >> 172.21.15.117:0/1367374045 conn(0x7f27c40796b0 msgr2=0x7f27c41043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:41.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.500+0000 7f27c912f640 1 -- 172.21.15.117:0/1367374045 shutdown_connections 2024-09-21T06:05:41.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:41.500+0000 7f27c912f640 1 -- 172.21.15.117:0/1367374045 wait complete. 2024-09-21T06:05:42.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:41 smithi192 ceph-mon[34700]: pgmap v581: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T06:05:42.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:41 smithi192 ceph-mon[34700]: scrub queued for path: / 2024-09-21T06:05:42.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:41 smithi192 ceph-mon[34700]: scrub summary: idle+waiting paths [/] 2024-09-21T06:05:42.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:41 smithi192 ceph-mon[34700]: scrub summary: active paths [/] 2024-09-21T06:05:42.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:41 smithi192 ceph-mon[34700]: scrub summary: idle+waiting paths [/] 2024-09-21T06:05:42.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:41 smithi117 ceph-mon[30947]: pgmap v581: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T06:05:42.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:41 smithi117 ceph-mon[30947]: scrub queued for path: / 2024-09-21T06:05:42.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:41 smithi117 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-21T06:05:42.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:41 smithi117 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-21T06:05:42.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:41 smithi117 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-21T06:05:42.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:41 smithi138 ceph-mon[34391]: pgmap v581: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-21T06:05:42.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:41 smithi138 ceph-mon[34391]: scrub queued for path: / 2024-09-21T06:05:42.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:41 smithi138 ceph-mon[34391]: scrub summary: idle+waiting paths [/] 2024-09-21T06:05:42.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:41 smithi138 ceph-mon[34391]: scrub summary: active paths [/] 2024-09-21T06:05:42.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:41 smithi138 ceph-mon[34391]: scrub summary: idle+waiting paths [/] 2024-09-21T06:05:42.409 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:05:42.655 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:05:43.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.156+0000 7f6db6083640 1 -- 172.21.15.117:0/828985612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6db0103b60 msgr2=0x7f6db0103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:43.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.156+0000 7f6db6083640 1 --2- 172.21.15.117:0/828985612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6db0103b60 0x7f6db0103f70 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009900 tx=0x7f6da002a410 comp rx=0 tx=0).stop 2024-09-21T06:05:43.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.156+0000 7f6db6083640 1 -- 172.21.15.117:0/828985612 shutdown_connections 2024-09-21T06:05:43.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.156+0000 7f6db6083640 1 --2- 172.21.15.117:0/828985612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6db0109df0 0x7f6db0110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.156+0000 7f6db6083640 1 --2- 172.21.15.117:0/828985612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6db01094a0 0x7f6db01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.156+0000 7f6db6083640 1 --2- 172.21.15.117:0/828985612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6db0103b60 0x7f6db0103f70 secure :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009900 tx=0x7f6da002a410 comp rx=0 tx=0).stop 2024-09-21T06:05:43.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.156+0000 7f6db6083640 1 -- 172.21.15.117:0/828985612 >> 172.21.15.117:0/828985612 conn(0x7f6db007b850 msgr2=0x7f6db007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:43.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.156+0000 7f6db6083640 1 -- 172.21.15.117:0/828985612 shutdown_connections 2024-09-21T06:05:43.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.156+0000 7f6db6083640 1 -- 172.21.15.117:0/828985612 wait complete. 2024-09-21T06:05:43.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.157+0000 7f6db6083640 1 Processor -- start 2024-09-21T06:05:43.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.157+0000 7f6db6083640 1 -- start start 2024-09-21T06:05:43.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.157+0000 7f6db6083640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6db01094a0 0x7f6db0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:43.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.157+0000 7f6db6083640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6db0109df0 0x7f6db0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:43.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.157+0000 7f6db6083640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6db0076190 0x7f6db01a8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:43.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.157+0000 7f6db6083640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6db0113ab0 con 0x7f6db0076190 2024-09-21T06:05:43.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.157+0000 7f6db6083640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6db0113930 con 0x7f6db01094a0 2024-09-21T06:05:43.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.157+0000 7f6db6083640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6db0113c30 con 0x7f6db0109df0 2024-09-21T06:05:43.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.158+0000 7f6daffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6db0076190 0x7f6db01a8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:43.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.158+0000 7f6daffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6db0076190 0x7f6db01a8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54062/0 (socket says 172.21.15.117:54062) 2024-09-21T06:05:43.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.158+0000 7f6daf7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6db01094a0 0x7f6db0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:43.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.158+0000 7f6daffff640 1 -- 172.21.15.117:0/1968544307 learned_addr learned my addr 172.21.15.117:0/1968544307 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:43.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.158+0000 7f6daeffd640 1 --2- 172.21.15.117:0/1968544307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6db0109df0 0x7f6db0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:43.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.158+0000 7f6daffff640 1 -- 172.21.15.117:0/1968544307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6db0109df0 msgr2=0x7f6db0075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:43.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.158+0000 7f6daffff640 1 --2- 172.21.15.117:0/1968544307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6db0109df0 0x7f6db0075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.158+0000 7f6daffff640 1 -- 172.21.15.117:0/1968544307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6db01094a0 msgr2=0x7f6db0075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:43.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.158+0000 7f6daffff640 1 --2- 172.21.15.117:0/1968544307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6db01094a0 0x7f6db0075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.158+0000 7f6daffff640 1 -- 172.21.15.117:0/1968544307 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db01a8ce0 con 0x7f6db0076190 2024-09-21T06:05:43.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.158+0000 7f6daffff640 1 --2- 172.21.15.117:0/1968544307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6db0076190 0x7f6db01a8750 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f6da800eaa0 tx=0x7f6da800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:43.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.158+0000 7f6dacff9640 1 -- 172.21.15.117:0/1968544307 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da8004d20 con 0x7f6db0076190 2024-09-21T06:05:43.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.159+0000 7f6dacff9640 1 -- 172.21.15.117:0/1968544307 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6da8005020 con 0x7f6db0076190 2024-09-21T06:05:43.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.159+0000 7f6db6083640 1 -- 172.21.15.117:0/1968544307 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6db01a8fd0 con 0x7f6db0076190 2024-09-21T06:05:43.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.159+0000 7f6dacff9640 1 -- 172.21.15.117:0/1968544307 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da80116b0 con 0x7f6db0076190 2024-09-21T06:05:43.162 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.159+0000 7f6db6083640 1 -- 172.21.15.117:0/1968544307 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6db01a9430 con 0x7f6db0076190 2024-09-21T06:05:43.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.161+0000 7f6dbc5f7740 1 Processor -- start 2024-09-21T06:05:43.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.161+0000 7f6dacff9640 1 -- 172.21.15.117:0/1968544307 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f6da8011850 con 0x7f6db0076190 2024-09-21T06:05:43.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.161+0000 7f6dbc5f7740 1 -- start start 2024-09-21T06:05:43.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.162+0000 7f6dbc5f7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b393807070 0x55b393bae650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:43.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.162+0000 7f6dbc5f7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3937de1b0 0x55b393baeb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:43.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.162+0000 7f6dbc5f7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3937e6390 0x55b393baf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:43.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.162+0000 7f6dbc5f7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b393b24e50 con 0x55b393807070 2024-09-21T06:05:43.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.162+0000 7f6dbc5f7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b393b14280 con 0x55b3937e6390 2024-09-21T06:05:43.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.162+0000 7f6dbc5f7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b393b18090 con 0x55b3937de1b0 2024-09-21T06:05:43.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.162+0000 7f6daeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b393807070 0x55b393bae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:43.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.162+0000 7f6daeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b393807070 0x55b393bae650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54068/0 (socket says 172.21.15.117:54068) 2024-09-21T06:05:43.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.162+0000 7f6daeffd640 1 -- 172.21.15.117:0/3416410746 learned_addr learned my addr 172.21.15.117:0/3416410746 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:43.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.162+0000 7f6daf7fe640 1 --2- 172.21.15.117:0/3416410746 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3937de1b0 0x55b393baeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:43.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.162+0000 7f6daeffd640 1 --2- 172.21.15.117:0/3416410746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3937e6390 0x55b393baf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:43.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.162+0000 7f6dacff9640 1 --2- 172.21.15.117:0/1968544307 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6d980b6680 0x7f6d980b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:43.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.162+0000 7f6daeffd640 1 -- 172.21.15.117:0/3416410746 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3937de1b0 msgr2=0x55b393baeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:43.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.162+0000 7f6daeffd640 1 --2- 172.21.15.117:0/3416410746 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3937de1b0 0x55b393baeb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.162+0000 7f6daeffd640 1 -- 172.21.15.117:0/3416410746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3937e6390 msgr2=0x55b393baf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:43.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.162+0000 7f6daeffd640 1 --2- 172.21.15.117:0/3416410746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3937e6390 0x55b393baf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.162+0000 7f6daeffd640 1 -- 172.21.15.117:0/3416410746 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b393872650 con 0x55b393807070 2024-09-21T06:05:43.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.162+0000 7f6daffff640 1 --2- 172.21.15.117:0/1968544307 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6d980b6680 0x7f6d980b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:43.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.162+0000 7f6daeffd640 1 --2- 172.21.15.117:0/3416410746 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b393807070 0x55b393bae650 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f6da400af90 tx=0x7f6da400a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=553e8c7ca5f9af86 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:43.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.162+0000 7f6dacff9640 1 -- 172.21.15.117:0/1968544307 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f6da80e7880 con 0x7f6db0076190 2024-09-21T06:05:43.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.162+0000 7f6d95ffb640 1 -- 172.21.15.117:0/3416410746 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da4019030 con 0x55b393807070 2024-09-21T06:05:43.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.163+0000 7f6d95ffb640 1 -- 172.21.15.117:0/3416410746 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6da40110d0 con 0x55b393807070 2024-09-21T06:05:43.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.163+0000 7f6d95ffb640 1 -- 172.21.15.117:0/3416410746 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da4004cd0 con 0x55b393807070 2024-09-21T06:05:43.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.163+0000 7f6dbc5f7740 1 -- 172.21.15.117:0/3416410746 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b393807070 msgr2=0x55b393bae650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:43.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.163+0000 7f6daffff640 1 --2- 172.21.15.117:0/1968544307 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6d980b6680 0x7f6d980b8b90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6da80ad6b0 tx=0x7f6da807aa00 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:43.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.163+0000 7f6dbc5f7740 1 --2- 172.21.15.117:0/3416410746 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b393807070 0x55b393bae650 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f6da400af90 tx=0x7f6da400a780 comp rx=0 tx=0).stop 2024-09-21T06:05:43.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.164+0000 7f6dbc5f7740 1 -- 172.21.15.117:0/3416410746 shutdown_connections 2024-09-21T06:05:43.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.164+0000 7f6dbc5f7740 1 --2- 172.21.15.117:0/3416410746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3937e6390 0x55b393baf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.164+0000 7f6dbc5f7740 1 --2- 172.21.15.117:0/3416410746 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3937de1b0 0x55b393baeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.164+0000 7f6dbc5f7740 1 --2- 172.21.15.117:0/3416410746 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b393807070 0x55b393bae650 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.164+0000 7f6dbc5f7740 1 -- 172.21.15.117:0/3416410746 >> 172.21.15.117:0/3416410746 conn(0x55b3937fb460 msgr2=0x55b393bac540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:43.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.164+0000 7f6dbc5f7740 1 -- 172.21.15.117:0/3416410746 shutdown_connections 2024-09-21T06:05:43.170 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.164+0000 7f6dbc5f7740 1 -- 172.21.15.117:0/3416410746 wait complete. 2024-09-21T06:05:43.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.164+0000 7f6dbc5f7740 1 -- start start 2024-09-21T06:05:43.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.164+0000 7f6dbc5f7740 1 Processor -- start 2024-09-21T06:05:43.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.165+0000 7f6dbc5f7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3937de1b0 0x55b393bb9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:43.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.165+0000 7f6dbc5f7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3937e6390 0x55b393bba290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:43.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.165+0000 7f6dbc5f7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b393807070 0x55b393bc6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:43.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.165+0000 7f6dbc5f7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b393975700 con 0x55b393807070 2024-09-21T06:05:43.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.165+0000 7f6dbc5f7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b393aaf110 con 0x55b3937de1b0 2024-09-21T06:05:43.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.165+0000 7f6dbc5f7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b393bbce30 con 0x55b3937e6390 2024-09-21T06:05:43.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.165+0000 7f6dbc5f7740 10 client.0 _ll_register_callbacks cb 0x55b393ac8990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:05:43.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.165+0000 7f6daeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b393807070 0x55b393bc6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:43.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.165+0000 7f6daeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b393807070 0x55b393bc6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54070/0 (socket says 172.21.15.117:54070) 2024-09-21T06:05:43.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.165+0000 7f6daeffd640 1 -- 172.21.15.117:0/2464933259 learned_addr learned my addr 172.21.15.117:0/2464933259 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:43.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.165+0000 7f6daf7fe640 1 --2- 172.21.15.117:0/2464933259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3937e6390 0x55b393bba290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:43.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.165+0000 7f6daeffd640 1 --2- 172.21.15.117:0/2464933259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3937de1b0 0x55b393bb9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:43.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.165+0000 7f6daeffd640 1 -- 172.21.15.117:0/2464933259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3937e6390 msgr2=0x55b393bba290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:43.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.165+0000 7f6daeffd640 1 --2- 172.21.15.117:0/2464933259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3937e6390 0x55b393bba290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.165+0000 7f6daeffd640 1 -- 172.21.15.117:0/2464933259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3937de1b0 msgr2=0x55b393bb9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:43.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.165+0000 7f6daeffd640 1 --2- 172.21.15.117:0/2464933259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3937de1b0 0x55b393bb9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.165+0000 7f6daeffd640 1 -- 172.21.15.117:0/2464933259 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b393adcc40 con 0x55b393807070 2024-09-21T06:05:43.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.165+0000 7f6daeffd640 1 --2- 172.21.15.117:0/2464933259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b393807070 0x55b393bc6120 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f6da4004fd0 tx=0x7f6da4004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e025793277dd5681 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:43.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.165+0000 7f6d8effd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:05:43.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.165+0000 7f6d8effd640 1 -- 172.21.15.117:0/2464933259 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da4019040 con 0x55b393807070 2024-09-21T06:05:43.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.166+0000 7f6d8effd640 1 -- 172.21.15.117:0/2464933259 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f6da4005000 con 0x55b393807070 2024-09-21T06:05:43.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.166+0000 7f6dbc5f7740 1 -- 172.21.15.117:0/2464933259 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6da40110d0 con 0x55b393807070 2024-09-21T06:05:43.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.166+0000 7f6d8effd640 1 -- 172.21.15.117:0/2464933259 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6da400ad30 con 0x55b393807070 2024-09-21T06:05:43.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.166+0000 7f6dbc5f7740 10 client.17127 fetch_fsmap learned FSMap version 29 2024-09-21T06:05:43.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.166+0000 7f6dbc5f7740 1 -- 172.21.15.117:0/2464933259 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b393bc8b80 con 0x55b393807070 2024-09-21T06:05:43.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.166+0000 7f6d8effd640 1 -- 172.21.15.117:0/2464933259 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6da40110d0 con 0x55b393807070 2024-09-21T06:05:43.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.166+0000 7f6d8effd640 1 -- 172.21.15.117:0/2464933259 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6da400f420 con 0x55b393807070 2024-09-21T06:05:43.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.166+0000 7f6dbc5f7740 10 client.17127 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:05:43.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.166+0000 7f6dbc5f7740 10 client.17127 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:05:43.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.166+0000 7f6dbc5f7740 20 client.17127 populate_metadata read hostname 'smithi117' 2024-09-21T06:05:43.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.166+0000 7f6dbc5f7740 1 --2- 172.21.15.117:0/2464933259 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b3937c59f0 0x55b393bcbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:43.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.166+0000 7f6dbc5f7740 4 client.17127 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:05:43.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.166+0000 7f6dbc5f7740 1 -- 172.21.15.117:0/2464933259 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b393b26da0 con 0x55b3937c59f0 2024-09-21T06:05:43.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.166+0000 7f6daf7fe640 1 --2- 172.21.15.117:0/2464933259 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b3937c59f0 0x55b393bcbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:43.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.167+0000 7f6daf7fe640 1 --2- 172.21.15.117:0/2464933259 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b3937c59f0 0x55b393bcbff0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=357c88932fe3af4d server_cookie=5027235932681b8a in_seq=0 out_seq=0 2024-09-21T06:05:43.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.167+0000 7f6d8effd640 10 client.17127 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:43.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.168+0000 7f6d8effd640 1 -- 172.21.15.117:0/2464933259 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b393b26da0 con 0x55b3937c59f0 2024-09-21T06:05:43.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.168+0000 7f6d8effd640 10 client.17127 handle_command_reply: tid=0 2024-09-21T06:05:43.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.168+0000 7f6dbc5f7740 1 client.17127 shutdown 2024-09-21T06:05:43.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.168+0000 7f6dbc5f7740 1 -- 172.21.15.117:0/2464933259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b393807070 msgr2=0x55b393bc6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:43.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.168+0000 7f6dbc5f7740 1 --2- 172.21.15.117:0/2464933259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b393807070 0x55b393bc6120 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f6da4004fd0 tx=0x7f6da4004970 comp rx=0 tx=0).stop 2024-09-21T06:05:43.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.168+0000 7f6dbc5f7740 1 -- 172.21.15.117:0/2464933259 shutdown_connections 2024-09-21T06:05:43.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.168+0000 7f6dbc5f7740 1 --2- 172.21.15.117:0/2464933259 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b3937c59f0 0x55b393bcbff0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.168+0000 7f6dbc5f7740 1 --2- 172.21.15.117:0/2464933259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b393807070 0x55b393bc6120 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.168+0000 7f6dbc5f7740 1 --2- 172.21.15.117:0/2464933259 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3937e6390 0x55b393bba290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.168+0000 7f6d8effd640 0 client.17127 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:43.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.168+0000 7f6dbc5f7740 1 --2- 172.21.15.117:0/2464933259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3937de1b0 0x55b393bb9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.168+0000 7f6dbc5f7740 1 -- 172.21.15.117:0/2464933259 >> 172.21.15.117:0/2464933259 conn(0x55b3937fb460 msgr2=0x55b393bb1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:43.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.168+0000 7f6dbc5f7740 1 -- 172.21.15.117:0/2464933259 shutdown_connections 2024-09-21T06:05:43.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.168+0000 7f6dbc5f7740 1 -- 172.21.15.117:0/2464933259 wait complete. 2024-09-21T06:05:43.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.168+0000 7f6dbc5f7740 20 client.17127 trim_cache size 0 max 16384 2024-09-21T06:05:43.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.185+0000 7f6dbc5f7740 1 Processor -- start 2024-09-21T06:05:43.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.185+0000 7f6dbc5f7740 1 -- start start 2024-09-21T06:05:43.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.186+0000 7f6dbc5f7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3937fb460 0x55b393bbbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:43.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.186+0000 7f6dbc5f7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3937de1b0 0x55b393baf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:43.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.186+0000 7f6dbc5f7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3937e6390 0x55b393bc6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:43.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.186+0000 7f6dbc5f7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6db01b1d90 con 0x55b3937fb460 2024-09-21T06:05:43.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.186+0000 7f6dbc5f7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6db01b38d0 con 0x55b3937de1b0 2024-09-21T06:05:43.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.186+0000 7f6dbc5f7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6db01b2460 con 0x55b3937e6390 2024-09-21T06:05:43.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.186+0000 7f6daeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3937fb460 0x55b393bbbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:43.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.186+0000 7f6daeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3937fb460 0x55b393bbbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54076/0 (socket says 172.21.15.117:54076) 2024-09-21T06:05:43.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.186+0000 7f6daeffd640 1 -- 172.21.15.117:0/1765910928 learned_addr learned my addr 172.21.15.117:0/1765910928 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:43.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.186+0000 7f6daf7fe640 1 --2- 172.21.15.117:0/1765910928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3937de1b0 0x55b393baf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:43.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.186+0000 7f6daeffd640 1 -- 172.21.15.117:0/1765910928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3937e6390 msgr2=0x55b393bc6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:43.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.186+0000 7f6daeffd640 1 --2- 172.21.15.117:0/1765910928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3937e6390 0x55b393bc6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.186+0000 7f6daeffd640 1 -- 172.21.15.117:0/1765910928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3937de1b0 msgr2=0x55b393baf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:43.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.186+0000 7f6daeffd640 1 --2- 172.21.15.117:0/1765910928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3937de1b0 0x55b393baf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.186+0000 7f6daeffd640 1 -- 172.21.15.117:0/1765910928 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b393bb14b0 con 0x55b3937fb460 2024-09-21T06:05:43.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.186+0000 7f6daf7fe640 1 --2- 172.21.15.117:0/1765910928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3937de1b0 0x55b393baf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:05:43.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.186+0000 7f6daeffd640 1 --2- 172.21.15.117:0/1765910928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3937fb460 0x55b393bbbc30 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f6da400f840 tx=0x7f6da4004880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20e129bf3973733c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:43.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.186+0000 7f6d957fa640 1 -- 172.21.15.117:0/1765910928 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da4019030 con 0x55b3937fb460 2024-09-21T06:05:43.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.186+0000 7f6d957fa640 1 -- 172.21.15.117:0/1765910928 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6da400a400 con 0x55b3937fb460 2024-09-21T06:05:43.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.186+0000 7f6d957fa640 1 -- 172.21.15.117:0/1765910928 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da400f420 con 0x55b3937fb460 2024-09-21T06:05:43.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.187+0000 7f6dbc5f7740 1 -- 172.21.15.117:0/1765910928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3937fb460 msgr2=0x55b393bbbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:43.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.187+0000 7f6dbc5f7740 1 --2- 172.21.15.117:0/1765910928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3937fb460 0x55b393bbbc30 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f6da400f840 tx=0x7f6da4004880 comp rx=0 tx=0).stop 2024-09-21T06:05:43.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.187+0000 7f6dbc5f7740 1 -- 172.21.15.117:0/1765910928 shutdown_connections 2024-09-21T06:05:43.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.187+0000 7f6dbc5f7740 1 --2- 172.21.15.117:0/1765910928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3937e6390 0x55b393bc6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.187+0000 7f6dbc5f7740 1 --2- 172.21.15.117:0/1765910928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3937de1b0 0x55b393baf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.187+0000 7f6dbc5f7740 1 --2- 172.21.15.117:0/1765910928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3937fb460 0x55b393bbbc30 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.187+0000 7f6dbc5f7740 1 -- 172.21.15.117:0/1765910928 >> 172.21.15.117:0/1765910928 conn(0x55b393bba410 msgr2=0x55b393bb3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:43.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.187+0000 7f6dbc5f7740 1 -- 172.21.15.117:0/1765910928 shutdown_connections 2024-09-21T06:05:43.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.187+0000 7f6dbc5f7740 1 -- 172.21.15.117:0/1765910928 wait complete. 2024-09-21T06:05:43.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.187+0000 7f6dbc5f7740 1 -- start start 2024-09-21T06:05:43.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.188+0000 7f6dbc5f7740 1 Processor -- start 2024-09-21T06:05:43.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.188+0000 7f6dbc5f7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3937de1b0 0x55b393bbe140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:43.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.188+0000 7f6dbc5f7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3937e6390 0x55b393bbe680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:43.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.188+0000 7f6dbc5f7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3937fb460 0x55b393bbebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:43.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.188+0000 7f6dbc5f7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6db01b3200 con 0x55b3937e6390 2024-09-21T06:05:43.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.188+0000 7f6dbc5f7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6db01b2b30 con 0x55b3937de1b0 2024-09-21T06:05:43.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.188+0000 7f6dbc5f7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6db01afd90 con 0x55b3937fb460 2024-09-21T06:05:43.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.188+0000 7f6dbc5f7740 10 client.0 _ll_register_callbacks cb 0x7f6da0009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:05:43.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.188+0000 7f6daf7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3937e6390 0x55b393bbe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:43.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.188+0000 7f6daf7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3937e6390 0x55b393bbe680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54084/0 (socket says 172.21.15.117:54084) 2024-09-21T06:05:43.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.188+0000 7f6daf7fe640 1 -- 172.21.15.117:0/923273813 learned_addr learned my addr 172.21.15.117:0/923273813 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:43.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.188+0000 7f6daeffd640 1 --2- 172.21.15.117:0/923273813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3937de1b0 0x55b393bbe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:43.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.188+0000 7f6daf7fe640 1 -- 172.21.15.117:0/923273813 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3937fb460 msgr2=0x55b393bbebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:43.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.188+0000 7f6daeffd640 1 --2- 172.21.15.117:0/923273813 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3937fb460 0x55b393bbebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:43.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.189+0000 7f6daf7fe640 1 --2- 172.21.15.117:0/923273813 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3937fb460 0x55b393bbebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.189+0000 7f6daf7fe640 1 -- 172.21.15.117:0/923273813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3937de1b0 msgr2=0x55b393bbe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:43.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.189+0000 7f6daf7fe640 1 --2- 172.21.15.117:0/923273813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3937de1b0 0x55b393bbe140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.189+0000 7f6daf7fe640 1 -- 172.21.15.117:0/923273813 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b393c35890 con 0x55b3937e6390 2024-09-21T06:05:43.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.189+0000 7f6daeffd640 1 --2- 172.21.15.117:0/923273813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3937de1b0 0x55b393bbe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:05:43.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.189+0000 7f6daf7fe640 1 --2- 172.21.15.117:0/923273813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3937e6390 0x55b393bbe680 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009230 tx=0x7f6da0031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ca4421ef5779f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:43.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.189+0000 7f6d8effd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:05:43.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.189+0000 7f6d8effd640 1 -- 172.21.15.117:0/923273813 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da004a070 con 0x55b3937e6390 2024-09-21T06:05:43.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.189+0000 7f6d8effd640 1 -- 172.21.15.117:0/923273813 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7f6da0004240 con 0x55b3937e6390 2024-09-21T06:05:43.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.189+0000 7f6dbc5f7740 1 -- 172.21.15.117:0/923273813 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6da400a400 con 0x55b3937e6390 2024-09-21T06:05:43.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.189+0000 7f6d8effd640 1 -- 172.21.15.117:0/923273813 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6da0056050 con 0x55b3937e6390 2024-09-21T06:05:43.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.189+0000 7f6dbc5f7740 10 client.17133 fetch_fsmap learned FSMap version 29 2024-09-21T06:05:43.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.189+0000 7f6dbc5f7740 1 -- 172.21.15.117:0/923273813 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b393c36aa0 con 0x55b3937e6390 2024-09-21T06:05:43.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.189+0000 7f6d8effd640 1 -- 172.21.15.117:0/923273813 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6da400a400 con 0x55b3937e6390 2024-09-21T06:05:43.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.189+0000 7f6d8effd640 1 -- 172.21.15.117:0/923273813 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6da00048e0 con 0x55b3937e6390 2024-09-21T06:05:43.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.190+0000 7f6dbc5f7740 10 client.17133 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:05:43.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.190+0000 7f6dbc5f7740 10 client.17133 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:05:43.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.190+0000 7f6dbc5f7740 20 client.17133 populate_metadata read hostname 'smithi117' 2024-09-21T06:05:43.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.190+0000 7f6dbc5f7740 1 --2- 172.21.15.117:0/923273813 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b393807070 0x55b393c39290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:43.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.190+0000 7f6dbc5f7740 4 client.17133 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:05:43.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.190+0000 7f6dbc5f7740 1 -- 172.21.15.117:0/923273813 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b393b22850 con 0x55b393807070 2024-09-21T06:05:43.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.190+0000 7f6daeffd640 1 --2- 172.21.15.117:0/923273813 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b393807070 0x55b393c39290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:43.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.190+0000 7f6d8effd640 10 client.17133 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:43.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.190+0000 7f6daeffd640 1 --2- 172.21.15.117:0/923273813 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b393807070 0x55b393c39290 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9801715b2d6efe6a server_cookie=9f8b7ad3fe2e6c7a in_seq=0 out_seq=0 2024-09-21T06:05:43.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.191+0000 7f6d8effd640 1 -- 172.21.15.117:0/923273813 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2558 (crc 0 0 0) 0x55b393b22850 con 0x55b393807070 2024-09-21T06:05:43.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.191+0000 7f6d8effd640 10 client.17133 handle_command_reply: tid=0 2024-09-21T06:05:43.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.191+0000 7f6dbc5f7740 1 client.17133 shutdown 2024-09-21T06:05:43.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.191+0000 7f6dbc5f7740 1 -- 172.21.15.117:0/923273813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3937e6390 msgr2=0x55b393bbe680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:43.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.191+0000 7f6dbc5f7740 1 --2- 172.21.15.117:0/923273813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3937e6390 0x55b393bbe680 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009230 tx=0x7f6da0031400 comp rx=0 tx=0).stop 2024-09-21T06:05:43.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.191+0000 7f6dbc5f7740 1 -- 172.21.15.117:0/923273813 shutdown_connections 2024-09-21T06:05:43.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.191+0000 7f6dbc5f7740 1 --2- 172.21.15.117:0/923273813 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b393807070 0x55b393c39290 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.191+0000 7f6dbc5f7740 1 --2- 172.21.15.117:0/923273813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3937e6390 0x55b393bbe680 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.191+0000 7f6dbc5f7740 1 --2- 172.21.15.117:0/923273813 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3937fb460 0x55b393bbebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.191+0000 7f6dbc5f7740 1 --2- 172.21.15.117:0/923273813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3937de1b0 0x55b393bbe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.191+0000 7f6d8effd640 0 client.17133 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:43.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.191+0000 7f6dbc5f7740 1 -- 172.21.15.117:0/923273813 >> 172.21.15.117:0/923273813 conn(0x55b393bba410 msgr2=0x55b393bb3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:43.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.191+0000 7f6dbc5f7740 1 -- 172.21.15.117:0/923273813 shutdown_connections 2024-09-21T06:05:43.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.191+0000 7f6dbc5f7740 1 -- 172.21.15.117:0/923273813 wait complete. 2024-09-21T06:05:43.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.191+0000 7f6dbc5f7740 20 client.17133 trim_cache size 0 max 16384 2024-09-21T06:05:43.199 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T06:05:43.199 INFO:teuthology.orchestra.run.smithi117.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:05:43.199 INFO:teuthology.orchestra.run.smithi117.stdout: "scrubs": { 2024-09-21T06:05:43.199 INFO:teuthology.orchestra.run.smithi117.stdout: "39eeca8a-6b4f-465f-af34-b54f77320d70": { 2024-09-21T06:05:43.199 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:05:43.199 INFO:teuthology.orchestra.run.smithi117.stdout: "tag": "39eeca8a-6b4f-465f-af34-b54f77320d70", 2024-09-21T06:05:43.200 INFO:teuthology.orchestra.run.smithi117.stdout: "options": "recursive,force" 2024-09-21T06:05:43.200 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:05:43.200 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:05:43.200 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_stats": { 2024-09-21T06:05:43.200 INFO:teuthology.orchestra.run.smithi117.stdout: "39eeca8a-6b4f-465f-af34-b54f77320d70": { 2024-09-21T06:05:43.200 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:05:41.497141+0000", 2024-09-21T06:05:43.200 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:05:43.200 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:05:43.200 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:05:43.200 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:05:43.200 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:05:43.200 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:05:43.200 INFO:teuthology.orchestra.run.smithi117.stdout: "29ff9b7c-49d9-4910-85be-bace7e940cd5": { 2024-09-21T06:05:43.200 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:03:03.775527+0000", 2024-09-21T06:05:43.201 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:05:43.201 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:05:43.201 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:05:43.201 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:05:43.201 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:05:43.201 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:05:43.201 INFO:teuthology.orchestra.run.smithi117.stdout: "0902e33a-ce33-45fb-92d8-2e6fcb8ec8a0": { 2024-09-21T06:05:43.201 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:04:11.365073+0000", 2024-09-21T06:05:43.201 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:05:43.201 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:05:43.201 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:05:43.201 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:05:43.201 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:05:43.201 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:05:43.202 INFO:teuthology.orchestra.run.smithi117.stdout: "d543b0ae-b710-4153-99da-fe5714839c1a": { 2024-09-21T06:05:43.202 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:01:20.574754+0000", 2024-09-21T06:05:43.202 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:05:43.202 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:05:43.202 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:05:43.202 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:05:43.202 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:05:43.202 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:05:43.202 INFO:teuthology.orchestra.run.smithi117.stdout: "ad80d9f1-6aff-46c0-91ab-2c1ab0681039": { 2024-09-21T06:05:43.202 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:59:31.949108+0000", 2024-09-21T06:05:43.202 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:05:43.202 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:05:43.202 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:05:43.202 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:05:43.202 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:05:43.203 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:05:43.203 INFO:teuthology.orchestra.run.smithi117.stdout: "fb2f217d-463b-4c8f-ab19-920b04bb7306": { 2024-09-21T06:05:43.203 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:04:56.308107+0000", 2024-09-21T06:05:43.203 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:05:43.203 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:05:43.203 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:05:43.203 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:05:43.203 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:05:43.203 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:05:43.203 INFO:teuthology.orchestra.run.smithi117.stdout: "8b3fe95a-b76b-4e94-afd0-72fe35e2dd92": { 2024-09-21T06:05:43.203 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:58:01.246076+0000", 2024-09-21T06:05:43.203 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:05:43.203 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:05:43.203 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:05:43.203 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:05:43.204 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:05:43.204 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:05:43.204 INFO:teuthology.orchestra.run.smithi117.stdout: "b30bfff0-0f6b-4e30-8467-ff7db421eba6": { 2024-09-21T06:05:43.204 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:57:14.939082+0000", 2024-09-21T06:05:43.204 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:05:43.204 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:05:43.204 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:05:43.204 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:05:43.204 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:05:43.204 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:05:43.204 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:05:43.204 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T06:05:43.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.192+0000 7f6db6083640 1 -- 172.21.15.117:0/1968544307 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6d980b6680 msgr2=0x7f6d980b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:43.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.192+0000 7f6db6083640 1 --2- 172.21.15.117:0/1968544307 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6d980b6680 0x7f6d980b8b90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6da80ad6b0 tx=0x7f6da807aa00 comp rx=0 tx=0).stop 2024-09-21T06:05:43.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.192+0000 7f6db6083640 1 -- 172.21.15.117:0/1968544307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6db0076190 msgr2=0x7f6db01a8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:43.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.192+0000 7f6db6083640 1 --2- 172.21.15.117:0/1968544307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6db0076190 0x7f6db01a8750 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f6da800eaa0 tx=0x7f6da800eea0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.192+0000 7f6db6083640 1 -- 172.21.15.117:0/1968544307 shutdown_connections 2024-09-21T06:05:43.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.192+0000 7f6db6083640 1 --2- 172.21.15.117:0/1968544307 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6d980b6680 0x7f6d980b8b90 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.192+0000 7f6db6083640 1 --2- 172.21.15.117:0/1968544307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6db0076190 0x7f6db01a8750 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.192+0000 7f6db6083640 1 --2- 172.21.15.117:0/1968544307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6db0109df0 0x7f6db0075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.192+0000 7f6db6083640 1 --2- 172.21.15.117:0/1968544307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6db01094a0 0x7f6db0075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:43.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.192+0000 7f6db6083640 1 -- 172.21.15.117:0/1968544307 >> 172.21.15.117:0/1968544307 conn(0x7f6db007b850 msgr2=0x7f6db0107690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:43.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.192+0000 7f6db6083640 1 -- 172.21.15.117:0/1968544307 shutdown_connections 2024-09-21T06:05:43.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:05:43.192+0000 7f6db6083640 1 -- 172.21.15.117:0/1968544307 wait complete. 2024-09-21T06:05:43.968 INFO:tasks.cephfs.filesystem:scrub status for tag:39eeca8a-6b4f-465f-af34-b54f77320d70 - {'path': '/', 'tag': '39eeca8a-6b4f-465f-af34-b54f77320d70', 'options': 'recursive,force'} 2024-09-21T06:05:44.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:43 smithi192 ceph-mon[34700]: pgmap v582: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-21T06:05:44.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:43 smithi117 ceph-mon[30947]: pgmap v582: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-21T06:05:44.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:43 smithi138 ceph-mon[34391]: pgmap v582: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-21T06:05:46.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:45 smithi192 ceph-mon[34700]: scrub summary: idle 2024-09-21T06:05:46.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:45 smithi192 ceph-mon[34700]: pgmap v583: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:05:46.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:45 smithi117 ceph-mon[30947]: scrub summary: idle 2024-09-21T06:05:46.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:45 smithi117 ceph-mon[30947]: pgmap v583: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:05:46.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:45 smithi138 ceph-mon[34391]: scrub summary: idle 2024-09-21T06:05:46.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:45 smithi138 ceph-mon[34391]: pgmap v583: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:05:48.167 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:47 smithi138 ceph-mon[34391]: pgmap v584: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-21T06:05:48.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:47 smithi192 ceph-mon[34700]: pgmap v584: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-21T06:05:48.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:47 smithi117 ceph-mon[30947]: pgmap v584: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-21T06:05:50.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:49 smithi192 ceph-mon[34700]: pgmap v585: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:05:50.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:49 smithi117 ceph-mon[30947]: pgmap v585: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:05:50.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:49 smithi138 ceph-mon[34391]: pgmap v585: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:05:52.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:51 smithi192 ceph-mon[34700]: pgmap v586: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:05:52.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:51 smithi117 ceph-mon[30947]: pgmap v586: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:05:52.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:51 smithi138 ceph-mon[34391]: pgmap v586: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:05:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.392+0000 7ff31a01f640 1 -- 172.21.15.117:0/932643741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff314109d50 msgr2=0x7ff314110580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.392+0000 7ff31a01f640 1 --2- 172.21.15.117:0/932643741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff314109d50 0x7ff314110580 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7ff30c009ca0 tx=0x7ff30c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.392+0000 7ff31a01f640 1 -- 172.21.15.117:0/932643741 shutdown_connections 2024-09-21T06:05:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.392+0000 7ff31a01f640 1 --2- 172.21.15.117:0/932643741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff314109d50 0x7ff314110580 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.392+0000 7ff31a01f640 1 --2- 172.21.15.117:0/932643741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff314109400 0x7ff314109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.392+0000 7ff31a01f640 1 --2- 172.21.15.117:0/932643741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff314103ad0 0x7ff314103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.392+0000 7ff31a01f640 1 -- 172.21.15.117:0/932643741 >> 172.21.15.117:0/932643741 conn(0x7ff3140ff510 msgr2=0x7ff314101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.392+0000 7ff31a01f640 1 -- 172.21.15.117:0/932643741 shutdown_connections 2024-09-21T06:05:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.392+0000 7ff31a01f640 1 -- 172.21.15.117:0/932643741 wait complete. 2024-09-21T06:05:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.393+0000 7ff31a01f640 1 Processor -- start 2024-09-21T06:05:53.395 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.393+0000 7ff31a01f640 1 -- start start 2024-09-21T06:05:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.393+0000 7ff31a01f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff314103ad0 0x7ff31419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.393+0000 7ff31a01f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff314109400 0x7ff31419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.393+0000 7ff31a01f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff314109d50 0x7ff31419fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.393+0000 7ff31a01f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff314113a10 con 0x7ff314103ad0 2024-09-21T06:05:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.393+0000 7ff31a01f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff314113890 con 0x7ff314109d50 2024-09-21T06:05:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.393+0000 7ff31a01f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff314113b90 con 0x7ff314109400 2024-09-21T06:05:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.393+0000 7ff3137fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff314103ad0 0x7ff31419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:53.397 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.393+0000 7ff3137fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff314103ad0 0x7ff31419a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37926/0 (socket says 172.21.15.117:37926) 2024-09-21T06:05:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.393+0000 7ff3137fe640 1 -- 172.21.15.117:0/1264045246 learned_addr learned my addr 172.21.15.117:0/1264045246 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.393+0000 7ff313fff640 1 --2- 172.21.15.117:0/1264045246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff314109d50 0x7ff31419fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.393+0000 7ff30bfff640 1 --2- 172.21.15.117:0/1264045246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff314109400 0x7ff31419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.393+0000 7ff3137fe640 1 -- 172.21.15.117:0/1264045246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff314109400 msgr2=0x7ff31419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.393+0000 7ff3137fe640 1 --2- 172.21.15.117:0/1264045246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff314109400 0x7ff31419a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.393+0000 7ff3137fe640 1 -- 172.21.15.117:0/1264045246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff314109d50 msgr2=0x7ff31419fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.393+0000 7ff3137fe640 1 --2- 172.21.15.117:0/1264045246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff314109d50 0x7ff31419fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.393+0000 7ff3137fe640 1 -- 172.21.15.117:0/1264045246 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3141a02b0 con 0x7ff314103ad0 2024-09-21T06:05:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.393+0000 7ff30bfff640 1 --2- 172.21.15.117:0/1264045246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff314109400 0x7ff31419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:05:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.393+0000 7ff313fff640 1 --2- 172.21.15.117:0/1264045246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff314109d50 0x7ff31419fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:05:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.393+0000 7ff3137fe640 1 --2- 172.21.15.117:0/1264045246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff314103ad0 0x7ff31419a410 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff30400c920 tx=0x7ff30400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.394+0000 7ff3117fa640 1 -- 172.21.15.117:0/1264045246 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304007c70 con 0x7ff314103ad0 2024-09-21T06:05:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.394+0000 7ff3117fa640 1 -- 172.21.15.117:0/1264045246 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff304007e10 con 0x7ff314103ad0 2024-09-21T06:05:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.394+0000 7ff3117fa640 1 -- 172.21.15.117:0/1264045246 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304018820 con 0x7ff314103ad0 2024-09-21T06:05:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.394+0000 7ff31a01f640 1 -- 172.21.15.117:0/1264045246 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3141a05a0 con 0x7ff314103ad0 2024-09-21T06:05:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.394+0000 7ff31a01f640 1 -- 172.21.15.117:0/1264045246 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3141a6dc0 con 0x7ff314103ad0 2024-09-21T06:05:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.396+0000 7ff321065740 1 Processor -- start 2024-09-21T06:05:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.396+0000 7ff3117fa640 1 -- 172.21.15.117:0/1264045246 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7ff30400f450 con 0x7ff314103ad0 2024-09-21T06:05:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.396+0000 7ff321065740 1 -- start start 2024-09-21T06:05:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.396+0000 7ff321065740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f50854e1b0 0x55f5088f39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.396+0000 7ff321065740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5085252f0 0x55f5088f3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.396+0000 7ff321065740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50852d4d0 0x55f5088f4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.397+0000 7ff321065740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f5088686f0 con 0x55f50854e1b0 2024-09-21T06:05:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.397+0000 7ff321065740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f508857b20 con 0x55f50852d4d0 2024-09-21T06:05:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.397+0000 7ff321065740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f50885b930 con 0x55f5085252f0 2024-09-21T06:05:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.397+0000 7ff313fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f50854e1b0 0x55f5088f39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.397+0000 7ff313fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f50854e1b0 0x55f5088f39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37936/0 (socket says 172.21.15.117:37936) 2024-09-21T06:05:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.397+0000 7ff313fff640 1 -- 172.21.15.117:0/552045820 learned_addr learned my addr 172.21.15.117:0/552045820 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.397+0000 7ff3137fe640 1 --2- 172.21.15.117:0/552045820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5085252f0 0x55f5088f3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.397+0000 7ff30bfff640 1 --2- 172.21.15.117:0/552045820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50852d4d0 0x55f5088f4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.397+0000 7ff313fff640 1 -- 172.21.15.117:0/552045820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5085252f0 msgr2=0x55f5088f3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.397+0000 7ff313fff640 1 --2- 172.21.15.117:0/552045820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5085252f0 0x55f5088f3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.397+0000 7ff313fff640 1 -- 172.21.15.117:0/552045820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50852d4d0 msgr2=0x55f5088f4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.397+0000 7ff313fff640 1 --2- 172.21.15.117:0/552045820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50852d4d0 0x55f5088f4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.397+0000 7ff313fff640 1 -- 172.21.15.117:0/552045820 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f50881d530 con 0x55f50854e1b0 2024-09-21T06:05:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.397+0000 7ff313fff640 1 --2- 172.21.15.117:0/552045820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f50854e1b0 0x55f5088f39b0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff30c006010 tx=0x7ff30c009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e610c45715550446 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.397+0000 7ff3117fa640 1 --2- 172.21.15.117:0/1264045246 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff3000b6680 0x7ff3000b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.397+0000 7ff3137fe640 1 --2- 172.21.15.117:0/1264045246 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff3000b6680 0x7ff3000b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.397+0000 7ff309ffb640 1 -- 172.21.15.117:0/552045820 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c031d70 con 0x55f50854e1b0 2024-09-21T06:05:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.397+0000 7ff309ffb640 1 -- 172.21.15.117:0/552045820 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff30c036070 con 0x55f50854e1b0 2024-09-21T06:05:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.397+0000 7ff309ffb640 1 -- 172.21.15.117:0/552045820 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c004330 con 0x55f50854e1b0 2024-09-21T06:05:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.398+0000 7ff3117fa640 1 -- 172.21.15.117:0/1264045246 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7ff3040e78f0 con 0x7ff314103ad0 2024-09-21T06:05:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.398+0000 7ff321065740 1 -- 172.21.15.117:0/552045820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f50854e1b0 msgr2=0x55f5088f39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.398+0000 7ff3137fe640 1 --2- 172.21.15.117:0/1264045246 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff3000b6680 0x7ff3000b8b90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff3040e5140 tx=0x7ff3040e9040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.398+0000 7ff321065740 1 --2- 172.21.15.117:0/552045820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f50854e1b0 0x55f5088f39b0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff30c006010 tx=0x7ff30c009880 comp rx=0 tx=0).stop 2024-09-21T06:05:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.398+0000 7ff321065740 1 -- 172.21.15.117:0/552045820 shutdown_connections 2024-09-21T06:05:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.398+0000 7ff321065740 1 --2- 172.21.15.117:0/552045820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50852d4d0 0x55f5088f4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.398+0000 7ff321065740 1 --2- 172.21.15.117:0/552045820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5085252f0 0x55f5088f3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.398+0000 7ff321065740 1 --2- 172.21.15.117:0/552045820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f50854e1b0 0x55f5088f39b0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.398+0000 7ff321065740 1 -- 172.21.15.117:0/552045820 >> 172.21.15.117:0/552045820 conn(0x55f508542300 msgr2=0x55f5088f18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.398+0000 7ff321065740 1 -- 172.21.15.117:0/552045820 shutdown_connections 2024-09-21T06:05:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.398+0000 7ff321065740 1 -- 172.21.15.117:0/552045820 wait complete. 2024-09-21T06:05:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.399+0000 7ff321065740 1 -- start start 2024-09-21T06:05:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.399+0000 7ff321065740 1 Processor -- start 2024-09-21T06:05:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.399+0000 7ff321065740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5085252f0 0x55f5088ff010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.399+0000 7ff321065740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f50852d4d0 0x55f5088ff550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.399+0000 7ff321065740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50854e1b0 0x55f50890b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.399+0000 7ff321065740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f50878fab0 con 0x55f50852d4d0 2024-09-21T06:05:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.399+0000 7ff321065740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f5087fd730 con 0x55f50854e1b0 2024-09-21T06:05:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.399+0000 7ff321065740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f5089020d0 con 0x55f5085252f0 2024-09-21T06:05:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.399+0000 7ff321065740 10 client.0 _ll_register_callbacks cb 0x55f5088516b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:05:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.399+0000 7ff30bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f50852d4d0 0x55f5088ff550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.400+0000 7ff30bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f50852d4d0 0x55f5088ff550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37944/0 (socket says 172.21.15.117:37944) 2024-09-21T06:05:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.400+0000 7ff30bfff640 1 -- 172.21.15.117:0/234176244 learned_addr learned my addr 172.21.15.117:0/234176244 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.400+0000 7ff313fff640 1 --2- 172.21.15.117:0/234176244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5085252f0 0x55f5088ff010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.400+0000 7ff313fff640 1 --2- 172.21.15.117:0/234176244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50854e1b0 0x55f50890b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.400+0000 7ff30bfff640 1 -- 172.21.15.117:0/234176244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5085252f0 msgr2=0x55f5088ff010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.400+0000 7ff30bfff640 1 --2- 172.21.15.117:0/234176244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5085252f0 0x55f5088ff010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.400+0000 7ff30bfff640 1 -- 172.21.15.117:0/234176244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50854e1b0 msgr2=0x55f50890b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.400+0000 7ff30bfff640 1 --2- 172.21.15.117:0/234176244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50854e1b0 0x55f50890b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.400+0000 7ff30bfff640 1 -- 172.21.15.117:0/234176244 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5085b9790 con 0x55f50852d4d0 2024-09-21T06:05:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.400+0000 7ff313fff640 1 --2- 172.21.15.117:0/234176244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5085252f0 0x55f5088ff010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:05:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.400+0000 7ff30bfff640 1 --2- 172.21.15.117:0/234176244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f50852d4d0 0x55f5088ff550 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc010670 tx=0x7ff2fc010b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b08e9a280225fea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.400+0000 7ff2f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:05:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.400+0000 7ff2f2ffd640 1 -- 172.21.15.117:0/234176244 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc00ead0 con 0x55f50852d4d0 2024-09-21T06:05:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.400+0000 7ff2f2ffd640 1 -- 172.21.15.117:0/234176244 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7ff2fc00ec70 con 0x55f50852d4d0 2024-09-21T06:05:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.400+0000 7ff321065740 1 -- 172.21.15.117:0/234176244 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff30c036070 con 0x55f50852d4d0 2024-09-21T06:05:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.400+0000 7ff2f2ffd640 1 -- 172.21.15.117:0/234176244 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff2fc020020 con 0x55f50852d4d0 2024-09-21T06:05:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.400+0000 7ff321065740 10 client.17145 fetch_fsmap learned FSMap version 29 2024-09-21T06:05:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.400+0000 7ff321065740 1 -- 172.21.15.117:0/234176244 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f508632620 con 0x55f50852d4d0 2024-09-21T06:05:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.400+0000 7ff2f2ffd640 1 -- 172.21.15.117:0/234176244 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff30c036070 con 0x55f50852d4d0 2024-09-21T06:05:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.401+0000 7ff2f2ffd640 1 -- 172.21.15.117:0/234176244 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff2fc00f040 con 0x55f50852d4d0 2024-09-21T06:05:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.401+0000 7ff321065740 10 client.17145 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:05:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.401+0000 7ff321065740 10 client.17145 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:05:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.401+0000 7ff321065740 20 client.17145 populate_metadata read hostname 'smithi117' 2024-09-21T06:05:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.401+0000 7ff321065740 1 --2- 172.21.15.117:0/234176244 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f50850cd80 0x55f508911170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.401+0000 7ff321065740 4 client.17145 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:05:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.401+0000 7ff321065740 1 -- 172.21.15.117:0/234176244 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f50886a640 con 0x55f50850cd80 2024-09-21T06:05:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.401+0000 7ff313fff640 1 --2- 172.21.15.117:0/234176244 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f50850cd80 0x55f508911170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.402+0000 7ff2f2ffd640 10 client.17145 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:53.409 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.402+0000 7ff313fff640 1 --2- 172.21.15.117:0/234176244 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f50850cd80 0x55f508911170 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c41ace4e6754e630 server_cookie=c5df7a928e56030 in_seq=0 out_seq=0 2024-09-21T06:05:53.409 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.402+0000 7ff2f2ffd640 1 -- 172.21.15.117:0/234176244 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f50886a640 con 0x55f50850cd80 2024-09-21T06:05:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.402+0000 7ff2f2ffd640 10 client.17145 handle_command_reply: tid=0 2024-09-21T06:05:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.402+0000 7ff321065740 1 client.17145 shutdown 2024-09-21T06:05:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.402+0000 7ff321065740 1 -- 172.21.15.117:0/234176244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f50852d4d0 msgr2=0x55f5088ff550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.403+0000 7ff321065740 1 --2- 172.21.15.117:0/234176244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f50852d4d0 0x55f5088ff550 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc010670 tx=0x7ff2fc010b30 comp rx=0 tx=0).stop 2024-09-21T06:05:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.403+0000 7ff321065740 1 -- 172.21.15.117:0/234176244 shutdown_connections 2024-09-21T06:05:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.403+0000 7ff321065740 1 --2- 172.21.15.117:0/234176244 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f50850cd80 0x55f508911170 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.403+0000 7ff321065740 1 --2- 172.21.15.117:0/234176244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f50852d4d0 0x55f5088ff550 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.403+0000 7ff321065740 1 --2- 172.21.15.117:0/234176244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50854e1b0 0x55f50890b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.403+0000 7ff321065740 1 --2- 172.21.15.117:0/234176244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5085252f0 0x55f5088ff010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.403+0000 7ff2f2ffd640 0 client.17145 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.403+0000 7ff321065740 1 -- 172.21.15.117:0/234176244 >> 172.21.15.117:0/234176244 conn(0x55f508542300 msgr2=0x55f5088f49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.403+0000 7ff321065740 1 -- 172.21.15.117:0/234176244 shutdown_connections 2024-09-21T06:05:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.403+0000 7ff321065740 1 -- 172.21.15.117:0/234176244 wait complete. 2024-09-21T06:05:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.403+0000 7ff321065740 20 client.17145 trim_cache size 0 max 16384 2024-09-21T06:05:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.417+0000 7ff321065740 1 Processor -- start 2024-09-21T06:05:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.417+0000 7ff321065740 1 -- start start 2024-09-21T06:05:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.417+0000 7ff321065740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5085252f0 0x55f508900c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.417+0000 7ff321065740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f50852d4d0 0x55f50890b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.417+0000 7ff321065740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f50854e1b0 0x55f5088fc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.417+0000 7ff321065740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff3141af520 con 0x55f50852d4d0 2024-09-21T06:05:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.417+0000 7ff321065740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff3141b1060 con 0x55f5085252f0 2024-09-21T06:05:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.418+0000 7ff321065740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff3141afbf0 con 0x55f50854e1b0 2024-09-21T06:05:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.418+0000 7ff30bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f50852d4d0 0x55f50890b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.418+0000 7ff30bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f50852d4d0 0x55f50890b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37960/0 (socket says 172.21.15.117:37960) 2024-09-21T06:05:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.418+0000 7ff30bfff640 1 -- 172.21.15.117:0/1693361589 learned_addr learned my addr 172.21.15.117:0/1693361589 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.418+0000 7ff313fff640 1 --2- 172.21.15.117:0/1693361589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5085252f0 0x55f508900c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.418+0000 7ff30bfff640 1 -- 172.21.15.117:0/1693361589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f50854e1b0 msgr2=0x55f5088fc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.418+0000 7ff30bfff640 1 --2- 172.21.15.117:0/1693361589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f50854e1b0 0x55f5088fc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.418+0000 7ff30bfff640 1 -- 172.21.15.117:0/1693361589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5085252f0 msgr2=0x55f508900c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.418+0000 7ff30bfff640 1 --2- 172.21.15.117:0/1693361589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5085252f0 0x55f508900c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.418+0000 7ff30bfff640 1 -- 172.21.15.117:0/1693361589 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5088fd520 con 0x55f50852d4d0 2024-09-21T06:05:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.418+0000 7ff30bfff640 1 --2- 172.21.15.117:0/1693361589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f50852d4d0 0x55f50890b2c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc000c00 tx=0x7ff2fc009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc870dfabb349c31 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.418+0000 7ff3097fa640 1 -- 172.21.15.117:0/1693361589 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc025620 con 0x55f50852d4d0 2024-09-21T06:05:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.418+0000 7ff3097fa640 1 -- 172.21.15.117:0/1693361589 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2fc02a070 con 0x55f50852d4d0 2024-09-21T06:05:53.421 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.418+0000 7ff3097fa640 1 -- 172.21.15.117:0/1693361589 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc00edf0 con 0x55f50852d4d0 2024-09-21T06:05:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.419+0000 7ff321065740 1 -- 172.21.15.117:0/1693361589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f50852d4d0 msgr2=0x55f50890b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.419+0000 7ff321065740 1 --2- 172.21.15.117:0/1693361589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f50852d4d0 0x55f50890b2c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc000c00 tx=0x7ff2fc009ce0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.419+0000 7ff321065740 1 -- 172.21.15.117:0/1693361589 shutdown_connections 2024-09-21T06:05:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.419+0000 7ff321065740 1 --2- 172.21.15.117:0/1693361589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f50852d4d0 0x55f50890b2c0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.419+0000 7ff321065740 1 --2- 172.21.15.117:0/1693361589 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f50854e1b0 0x55f5088fc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.419+0000 7ff321065740 1 --2- 172.21.15.117:0/1693361589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f5085252f0 0x55f508900c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.419+0000 7ff321065740 1 -- 172.21.15.117:0/1693361589 >> 172.21.15.117:0/1693361589 conn(0x55f508542300 msgr2=0x55f508900530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.419+0000 7ff321065740 1 -- 172.21.15.117:0/1693361589 shutdown_connections 2024-09-21T06:05:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.419+0000 7ff321065740 1 -- 172.21.15.117:0/1693361589 wait complete. 2024-09-21T06:05:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.419+0000 7ff321065740 1 -- start start 2024-09-21T06:05:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.420+0000 7ff321065740 1 Processor -- start 2024-09-21T06:05:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.420+0000 7ff321065740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5085252f0 0x55f508904dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.420+0000 7ff321065740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50852d4d0 0x55f508905360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.420+0000 7ff321065740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f50854e1b0 0x55f508972af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.420+0000 7ff321065740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff3141b0990 con 0x55f5085252f0 2024-09-21T06:05:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.420+0000 7ff321065740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff3141b02c0 con 0x55f50852d4d0 2024-09-21T06:05:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.420+0000 7ff321065740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff3141ad520 con 0x55f50854e1b0 2024-09-21T06:05:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.420+0000 7ff321065740 10 client.0 _ll_register_callbacks cb 0x7ff2fc009380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:05:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.420+0000 7ff313fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5085252f0 0x55f508904dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.420+0000 7ff313fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5085252f0 0x55f508904dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37970/0 (socket says 172.21.15.117:37970) 2024-09-21T06:05:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.420+0000 7ff313fff640 1 -- 172.21.15.117:0/1010528343 learned_addr learned my addr 172.21.15.117:0/1010528343 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:05:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.420+0000 7ff313fff640 1 --2- 172.21.15.117:0/1010528343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f50854e1b0 0x55f508972af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.420+0000 7ff30bfff640 1 --2- 172.21.15.117:0/1010528343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50852d4d0 0x55f508905360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.420+0000 7ff313fff640 1 -- 172.21.15.117:0/1010528343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f50854e1b0 msgr2=0x55f508972af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.420+0000 7ff313fff640 1 --2- 172.21.15.117:0/1010528343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f50854e1b0 0x55f508972af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.420+0000 7ff313fff640 1 -- 172.21.15.117:0/1010528343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50852d4d0 msgr2=0x55f508905360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:05:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.420+0000 7ff313fff640 1 --2- 172.21.15.117:0/1010528343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50852d4d0 0x55f508905360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.420+0000 7ff313fff640 1 -- 172.21.15.117:0/1010528343 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f508973160 con 0x55f5085252f0 2024-09-21T06:05:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.420+0000 7ff30bfff640 1 --2- 172.21.15.117:0/1010528343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50852d4d0 0x55f508905360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:05:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.420+0000 7ff313fff640 1 --2- 172.21.15.117:0/1010528343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5085252f0 0x55f508904dc0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff30c003c10 tx=0x7ff30c003de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd863c6539b8d98f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:05:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.420+0000 7ff2f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:05:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.421+0000 7ff2f2ffd640 1 -- 172.21.15.117:0/1010528343 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30c031650 con 0x55f5085252f0 2024-09-21T06:05:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.421+0000 7ff2f2ffd640 1 -- 172.21.15.117:0/1010528343 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7ff30c0317f0 con 0x55f5085252f0 2024-09-21T06:05:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.421+0000 7ff321065740 1 -- 172.21.15.117:0/1010528343 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2fc02a070 con 0x55f5085252f0 2024-09-21T06:05:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.421+0000 7ff2f2ffd640 1 -- 172.21.15.117:0/1010528343 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff30c004040 con 0x55f5085252f0 2024-09-21T06:05:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.421+0000 7ff321065740 10 client.17151 fetch_fsmap learned FSMap version 29 2024-09-21T06:05:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.421+0000 7ff321065740 1 -- 172.21.15.117:0/1010528343 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f50897b670 con 0x55f5085252f0 2024-09-21T06:05:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.421+0000 7ff2f2ffd640 1 -- 172.21.15.117:0/1010528343 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2fc02a070 con 0x55f5085252f0 2024-09-21T06:05:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.421+0000 7ff2f2ffd640 1 -- 172.21.15.117:0/1010528343 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff30c02fa20 con 0x55f5085252f0 2024-09-21T06:05:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.421+0000 7ff321065740 10 client.17151 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:05:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.421+0000 7ff321065740 10 client.17151 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:05:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.421+0000 7ff321065740 20 client.17151 populate_metadata read hostname 'smithi117' 2024-09-21T06:05:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.421+0000 7ff321065740 1 --2- 172.21.15.117:0/1010528343 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f5084e21c0 0x55f50897d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:05:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.421+0000 7ff321065740 4 client.17151 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:05:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.421+0000 7ff321065740 1 -- 172.21.15.117:0/1010528343 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f5088660f0 con 0x55f5084e21c0 2024-09-21T06:05:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.422+0000 7ff30bfff640 1 --2- 172.21.15.117:0/1010528343 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f5084e21c0 0x55f50897d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:05:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.422+0000 7ff2f2ffd640 10 client.17151 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.422+0000 7ff30bfff640 1 --2- 172.21.15.117:0/1010528343 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f5084e21c0 0x55f50897d9c0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c811cf53f24147fe server_cookie=418f8975e6d6264f in_seq=0 out_seq=0 2024-09-21T06:05:53.431 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.429+0000 7ff2f2ffd640 1 -- 172.21.15.117:0/1010528343 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f5088660f0 con 0x55f5084e21c0 2024-09-21T06:05:53.431 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.429+0000 7ff2f2ffd640 10 client.17151 handle_command_reply: tid=0 2024-09-21T06:05:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.429+0000 7ff321065740 1 client.17151 shutdown 2024-09-21T06:05:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.429+0000 7ff321065740 1 -- 172.21.15.117:0/1010528343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5085252f0 msgr2=0x55f508904dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.429+0000 7ff321065740 1 --2- 172.21.15.117:0/1010528343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5085252f0 0x55f508904dc0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff30c003c10 tx=0x7ff30c003de0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.429+0000 7ff321065740 1 -- 172.21.15.117:0/1010528343 shutdown_connections 2024-09-21T06:05:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.429+0000 7ff321065740 1 --2- 172.21.15.117:0/1010528343 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f5084e21c0 0x55f50897d9c0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.429+0000 7ff321065740 1 --2- 172.21.15.117:0/1010528343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f50854e1b0 0x55f508972af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.429+0000 7ff321065740 1 --2- 172.21.15.117:0/1010528343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f50852d4d0 0x55f508905360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.429+0000 7ff321065740 1 --2- 172.21.15.117:0/1010528343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f5085252f0 0x55f508904dc0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.429+0000 7ff2f2ffd640 0 client.17151 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:05:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.429+0000 7ff321065740 1 -- 172.21.15.117:0/1010528343 >> 172.21.15.117:0/1010528343 conn(0x55f508542300 msgr2=0x55f5088f1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.429+0000 7ff321065740 1 -- 172.21.15.117:0/1010528343 shutdown_connections 2024-09-21T06:05:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.429+0000 7ff321065740 1 -- 172.21.15.117:0/1010528343 wait complete. 2024-09-21T06:05:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.429+0000 7ff321065740 20 client.17151 trim_cache size 0 max 16384 2024-09-21T06:05:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T06:05:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T06:05:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T06:05:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T06:05:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.430+0000 7ff31a01f640 1 -- 172.21.15.117:0/1264045246 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff3000b6680 msgr2=0x7ff3000b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.430+0000 7ff31a01f640 1 --2- 172.21.15.117:0/1264045246 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff3000b6680 0x7ff3000b8b90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff3040e5140 tx=0x7ff3040e9040 comp rx=0 tx=0).stop 2024-09-21T06:05:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.430+0000 7ff31a01f640 1 -- 172.21.15.117:0/1264045246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff314103ad0 msgr2=0x7ff31419a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:05:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.430+0000 7ff31a01f640 1 --2- 172.21.15.117:0/1264045246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff314103ad0 0x7ff31419a410 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7ff30400c920 tx=0x7ff30400cde0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.430+0000 7ff31a01f640 1 -- 172.21.15.117:0/1264045246 shutdown_connections 2024-09-21T06:05:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.430+0000 7ff31a01f640 1 --2- 172.21.15.117:0/1264045246 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff3000b6680 0x7ff3000b8b90 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.430+0000 7ff31a01f640 1 --2- 172.21.15.117:0/1264045246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff314109d50 0x7ff31419fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.430+0000 7ff31a01f640 1 --2- 172.21.15.117:0/1264045246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff314109400 0x7ff31419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.430+0000 7ff31a01f640 1 --2- 172.21.15.117:0/1264045246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff314103ad0 0x7ff31419a410 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:05:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.430+0000 7ff31a01f640 1 -- 172.21.15.117:0/1264045246 >> 172.21.15.117:0/1264045246 conn(0x7ff3140ff510 msgr2=0x7ff314101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:05:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.430+0000 7ff31a01f640 1 -- 172.21.15.117:0/1264045246 shutdown_connections 2024-09-21T06:05:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:05:53.430+0000 7ff31a01f640 1 -- 172.21.15.117:0/1264045246 wait complete. 2024-09-21T06:05:54.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:53 smithi192 ceph-mon[34700]: pgmap v587: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T06:05:54.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:53 smithi117 ceph-mon[30947]: pgmap v587: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T06:05:54.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:53 smithi138 ceph-mon[34391]: pgmap v587: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T06:05:56.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:55 smithi192 ceph-mon[34700]: pgmap v588: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:05:56.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:55 smithi117 ceph-mon[30947]: pgmap v588: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:05:56.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:55 smithi138 ceph-mon[34391]: pgmap v588: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:05:58.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:57 smithi192 ceph-mon[34700]: pgmap v589: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-09-21T06:05:58.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:57 smithi117 ceph-mon[30947]: pgmap v589: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-09-21T06:05:58.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:57 smithi138 ceph-mon[34391]: pgmap v589: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-09-21T06:06:00.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:05:59 smithi192 ceph-mon[34700]: pgmap v590: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T06:06:00.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:05:59 smithi117 ceph-mon[30947]: pgmap v590: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T06:06:00.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:05:59 smithi138 ceph-mon[34391]: pgmap v590: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T06:06:02.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:01 smithi192 ceph-mon[34700]: pgmap v591: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-21T06:06:02.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:01 smithi117 ceph-mon[30947]: pgmap v591: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-21T06:06:02.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:01 smithi138 ceph-mon[34391]: pgmap v591: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-21T06:06:04.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:03 smithi192 ceph-mon[34700]: pgmap v592: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T06:06:04.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:03 smithi117 ceph-mon[30947]: pgmap v592: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T06:06:04.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:03 smithi138 ceph-mon[34391]: pgmap v592: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-21T06:06:06.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:05 smithi192 ceph-mon[34700]: pgmap v593: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T06:06:06.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:05 smithi117 ceph-mon[30947]: pgmap v593: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T06:06:06.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:05 smithi138 ceph-mon[34391]: pgmap v593: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-09-21T06:06:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.536+0000 7ff20a339640 1 -- 172.21.15.117:0/2775223300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff204106b50 msgr2=0x7ff204110590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.536+0000 7ff20a339640 1 --2- 172.21.15.117:0/2775223300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff204106b50 0x7ff204110590 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc009ca0 tx=0x7ff1fc02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.539 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.536+0000 7ff20a339640 1 -- 172.21.15.117:0/2775223300 shutdown_connections 2024-09-21T06:06:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.536+0000 7ff20a339640 1 --2- 172.21.15.117:0/2775223300 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff204106b50 0x7ff204110590 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.536+0000 7ff20a339640 1 --2- 172.21.15.117:0/2775223300 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff204106200 0x7ff204106610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.536+0000 7ff20a339640 1 --2- 172.21.15.117:0/2775223300 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff204102090 0x7ff2041024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.536+0000 7ff20a339640 1 -- 172.21.15.117:0/2775223300 >> 172.21.15.117:0/2775223300 conn(0x7ff204079ec0 msgr2=0x7ff20407a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.536+0000 7ff20a339640 1 -- 172.21.15.117:0/2775223300 shutdown_connections 2024-09-21T06:06:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.536+0000 7ff20a339640 1 -- 172.21.15.117:0/2775223300 wait complete. 2024-09-21T06:06:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.536+0000 7ff20a339640 1 Processor -- start 2024-09-21T06:06:06.540 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.537+0000 7ff20a339640 1 -- start start 2024-09-21T06:06:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.537+0000 7ff20a339640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff204102090 0x7ff20419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.537+0000 7ff20a339640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff204106200 0x7ff20419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.537+0000 7ff20a339640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff204106b50 0x7ff20419fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.537+0000 7ff20a339640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff204113a20 con 0x7ff204106b50 2024-09-21T06:06:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.537+0000 7ff20a339640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff2041138a0 con 0x7ff204102090 2024-09-21T06:06:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.537+0000 7ff20a339640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff204113ba0 con 0x7ff204106200 2024-09-21T06:06:06.542 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.537+0000 7ff2088d6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff204106b50 0x7ff20419fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.537+0000 7ff2088d6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff204106b50 0x7ff20419fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59194/0 (socket says 172.21.15.117:59194) 2024-09-21T06:06:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.537+0000 7ff2088d6640 1 -- 172.21.15.117:0/610191161 learned_addr learned my addr 172.21.15.117:0/610191161 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.537+0000 7ff203fff640 1 --2- 172.21.15.117:0/610191161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff204102090 0x7ff20419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.537+0000 7ff2037fe640 1 --2- 172.21.15.117:0/610191161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff204106200 0x7ff20419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.538+0000 7ff2088d6640 1 -- 172.21.15.117:0/610191161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff204106200 msgr2=0x7ff20419aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.538+0000 7ff2088d6640 1 --2- 172.21.15.117:0/610191161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff204106200 0x7ff20419aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.538+0000 7ff2088d6640 1 -- 172.21.15.117:0/610191161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff204102090 msgr2=0x7ff20419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.538+0000 7ff2088d6640 1 --2- 172.21.15.117:0/610191161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff204102090 0x7ff20419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.538+0000 7ff2088d6640 1 -- 172.21.15.117:0/610191161 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2041a03b0 con 0x7ff204106b50 2024-09-21T06:06:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.538+0000 7ff203fff640 1 --2- 172.21.15.117:0/610191161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff204102090 0x7ff20419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:06:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.538+0000 7ff2088d6640 1 --2- 172.21.15.117:0/610191161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff204106b50 0x7ff20419fcb0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc005cc0 tx=0x7ff1fc02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.538+0000 7ff2017fa640 1 -- 172.21.15.117:0/610191161 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc003bf0 con 0x7ff204106b50 2024-09-21T06:06:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.538+0000 7ff2017fa640 1 -- 172.21.15.117:0/610191161 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1fc003d90 con 0x7ff204106b50 2024-09-21T06:06:06.543 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.538+0000 7ff20a339640 1 -- 172.21.15.117:0/610191161 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2041a0640 con 0x7ff204106b50 2024-09-21T06:06:06.544 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.538+0000 7ff2017fa640 1 -- 172.21.15.117:0/610191161 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc02fbc0 con 0x7ff204106b50 2024-09-21T06:06:06.544 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.538+0000 7ff20a339640 1 -- 172.21.15.117:0/610191161 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2041a6e30 con 0x7ff204106b50 2024-09-21T06:06:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.541+0000 7ff2112a8740 1 Processor -- start 2024-09-21T06:06:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.541+0000 7ff2017fa640 1 -- 172.21.15.117:0/610191161 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7ff1fc02fd60 con 0x7ff204106b50 2024-09-21T06:06:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.541+0000 7ff2112a8740 1 -- start start 2024-09-21T06:06:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.541+0000 7ff2112a8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5e7681b0 0x55fc5eb0d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.541+0000 7ff2112a8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc5e73f2f0 0x55fc5eb0def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.541+0000 7ff2112a8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc5e7474d0 0x55fc5eb0e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.541+0000 7ff2112a8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fc5ea826f0 con 0x55fc5e7681b0 2024-09-21T06:06:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.541+0000 7ff2112a8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fc5ea71b20 con 0x55fc5e7474d0 2024-09-21T06:06:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.541+0000 7ff2112a8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55fc5ea75930 con 0x55fc5e73f2f0 2024-09-21T06:06:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.541+0000 7ff2037fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5e7681b0 0x55fc5eb0d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.541+0000 7ff2037fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5e7681b0 0x55fc5eb0d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59198/0 (socket says 172.21.15.117:59198) 2024-09-21T06:06:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.541+0000 7ff2037fe640 1 -- 172.21.15.117:0/3463304119 learned_addr learned my addr 172.21.15.117:0/3463304119 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.542+0000 7ff2037fe640 1 --2- 172.21.15.117:0/3463304119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc5e7474d0 0x55fc5eb0e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.542+0000 7ff203fff640 1 --2- 172.21.15.117:0/3463304119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc5e73f2f0 0x55fc5eb0def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.542+0000 7ff2017fa640 1 --2- 172.21.15.117:0/610191161 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff1ec0b6680 0x7ff1ec0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.542+0000 7ff2037fe640 1 -- 172.21.15.117:0/3463304119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc5e73f2f0 msgr2=0x55fc5eb0def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.542+0000 7ff2037fe640 1 --2- 172.21.15.117:0/3463304119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc5e73f2f0 0x55fc5eb0def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.542+0000 7ff2037fe640 1 -- 172.21.15.117:0/3463304119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc5e7474d0 msgr2=0x55fc5eb0e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.542+0000 7ff2037fe640 1 --2- 172.21.15.117:0/3463304119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc5e7474d0 0x55fc5eb0e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.542+0000 7ff2037fe640 1 -- 172.21.15.117:0/3463304119 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc5ea37530 con 0x55fc5e7681b0 2024-09-21T06:06:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.542+0000 7ff2088d6640 1 --2- 172.21.15.117:0/610191161 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff1ec0b6680 0x7ff1ec0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.542+0000 7ff2037fe640 1 --2- 172.21.15.117:0/3463304119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5e7681b0 0x55fc5eb0d9b0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff204075b40 tx=0x7ff1f800b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57cd526d72b59a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.542+0000 7ff2017fa640 1 -- 172.21.15.117:0/610191161 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7ff1fc108c10 con 0x7ff204106b50 2024-09-21T06:06:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.542+0000 7ff1ea7fc640 1 -- 172.21.15.117:0/3463304119 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8004020 con 0x55fc5e7681b0 2024-09-21T06:06:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.542+0000 7ff1ea7fc640 1 -- 172.21.15.117:0/3463304119 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1f8004c50 con 0x55fc5e7681b0 2024-09-21T06:06:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.542+0000 7ff1ea7fc640 1 -- 172.21.15.117:0/3463304119 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8005380 con 0x55fc5e7681b0 2024-09-21T06:06:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.543+0000 7ff2112a8740 1 -- 172.21.15.117:0/3463304119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5e7681b0 msgr2=0x55fc5eb0d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.543+0000 7ff2112a8740 1 --2- 172.21.15.117:0/3463304119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5e7681b0 0x55fc5eb0d9b0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff204075b40 tx=0x7ff1f800b650 comp rx=0 tx=0).stop 2024-09-21T06:06:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.543+0000 7ff2088d6640 1 --2- 172.21.15.117:0/610191161 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff1ec0b6680 0x7ff1ec0b8b90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc106460 tx=0x7ff1fc0b6040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.543+0000 7ff2112a8740 1 -- 172.21.15.117:0/3463304119 shutdown_connections 2024-09-21T06:06:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.543+0000 7ff2112a8740 1 --2- 172.21.15.117:0/3463304119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc5e7474d0 0x55fc5eb0e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.543+0000 7ff2112a8740 1 --2- 172.21.15.117:0/3463304119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc5e73f2f0 0x55fc5eb0def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.543+0000 7ff2112a8740 1 --2- 172.21.15.117:0/3463304119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5e7681b0 0x55fc5eb0d9b0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.543+0000 7ff2112a8740 1 -- 172.21.15.117:0/3463304119 >> 172.21.15.117:0/3463304119 conn(0x55fc5e75c300 msgr2=0x55fc5eb0b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.543+0000 7ff2112a8740 1 -- 172.21.15.117:0/3463304119 shutdown_connections 2024-09-21T06:06:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.543+0000 7ff2112a8740 1 -- 172.21.15.117:0/3463304119 wait complete. 2024-09-21T06:06:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.543+0000 7ff2112a8740 1 -- start start 2024-09-21T06:06:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.544+0000 7ff2112a8740 1 Processor -- start 2024-09-21T06:06:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.544+0000 7ff2112a8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5e73f2f0 0x55fc5eb19050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.544+0000 7ff2112a8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc5e7474d0 0x55fc5eb19590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.544+0000 7ff2112a8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc5e7681b0 0x55fc5eb253f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.544+0000 7ff2112a8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55fc5e9a9ab0 con 0x55fc5e73f2f0 2024-09-21T06:06:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.544+0000 7ff2112a8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55fc5ea17730 con 0x55fc5e7681b0 2024-09-21T06:06:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.544+0000 7ff2112a8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55fc5eb1c0d0 con 0x55fc5e7474d0 2024-09-21T06:06:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.544+0000 7ff2112a8740 10 client.0 _ll_register_callbacks cb 0x55fc5ea6b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.544+0000 7ff2037fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5e73f2f0 0x55fc5eb19050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.544+0000 7ff2037fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5e73f2f0 0x55fc5eb19050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59202/0 (socket says 172.21.15.117:59202) 2024-09-21T06:06:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.544+0000 7ff2037fe640 1 -- 172.21.15.117:0/3450352406 learned_addr learned my addr 172.21.15.117:0/3450352406 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.544+0000 7ff2037fe640 1 --2- 172.21.15.117:0/3450352406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc5e7681b0 0x55fc5eb253f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.544+0000 7ff203fff640 1 --2- 172.21.15.117:0/3450352406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc5e7474d0 0x55fc5eb19590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.544+0000 7ff2037fe640 1 -- 172.21.15.117:0/3450352406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc5e7474d0 msgr2=0x55fc5eb19590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.544+0000 7ff2037fe640 1 --2- 172.21.15.117:0/3450352406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc5e7474d0 0x55fc5eb19590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.544+0000 7ff2037fe640 1 -- 172.21.15.117:0/3450352406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc5e7681b0 msgr2=0x55fc5eb253f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.544+0000 7ff2037fe640 1 --2- 172.21.15.117:0/3450352406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc5e7681b0 0x55fc5eb253f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.544+0000 7ff2037fe640 1 -- 172.21.15.117:0/3450352406 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc5e7d3790 con 0x55fc5e73f2f0 2024-09-21T06:06:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.544+0000 7ff203fff640 1 --2- 172.21.15.117:0/3450352406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc5e7474d0 0x55fc5eb19590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:06:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.545+0000 7ff2037fe640 1 --2- 172.21.15.117:0/3450352406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5e73f2f0 0x55fc5eb19050 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7ff1f800b470 tx=0x7ff1f80045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5205a26197ba2f6a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.545+0000 7ff1e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.545+0000 7ff1e37fe640 1 -- 172.21.15.117:0/3450352406 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8005100 con 0x55fc5e73f2f0 2024-09-21T06:06:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.545+0000 7ff1e37fe640 1 -- 172.21.15.117:0/3450352406 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7ff1f80052a0 con 0x55fc5e73f2f0 2024-09-21T06:06:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.545+0000 7ff2112a8740 1 -- 172.21.15.117:0/3450352406 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1f8004c50 con 0x55fc5e73f2f0 2024-09-21T06:06:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.545+0000 7ff1e37fe640 1 -- 172.21.15.117:0/3450352406 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff1f8035050 con 0x55fc5e73f2f0 2024-09-21T06:06:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.545+0000 7ff2112a8740 10 client.17163 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.545+0000 7ff2112a8740 1 -- 172.21.15.117:0/3450352406 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fc5e84c620 con 0x55fc5e73f2f0 2024-09-21T06:06:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.545+0000 7ff1e37fe640 1 -- 172.21.15.117:0/3450352406 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1f8023020 con 0x55fc5e73f2f0 2024-09-21T06:06:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.545+0000 7ff1e37fe640 1 -- 172.21.15.117:0/3450352406 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff1f801c070 con 0x55fc5e73f2f0 2024-09-21T06:06:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.545+0000 7ff2112a8740 10 client.17163 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.545+0000 7ff2112a8740 10 client.17163 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:06:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.545+0000 7ff2112a8740 20 client.17163 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.545+0000 7ff2112a8740 1 --2- 172.21.15.117:0/3450352406 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55fc5e726d80 0x55fc5eb2b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.545+0000 7ff2112a8740 4 client.17163 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:06:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.545+0000 7ff2112a8740 1 -- 172.21.15.117:0/3450352406 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc5ea84640 con 0x55fc5e726d80 2024-09-21T06:06:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.546+0000 7ff203fff640 1 --2- 172.21.15.117:0/3450352406 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55fc5e726d80 0x55fc5eb2b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.546+0000 7ff203fff640 1 --2- 172.21.15.117:0/3450352406 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55fc5e726d80 0x55fc5eb2b0b0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51d803a83251611c server_cookie=c8fe0b1d9922adaf in_seq=0 out_seq=0 2024-09-21T06:06:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.546+0000 7ff1e37fe640 10 client.17163 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.547+0000 7ff1e37fe640 1 -- 172.21.15.117:0/3450352406 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fc5ea84640 con 0x55fc5e726d80 2024-09-21T06:06:06.555 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.547+0000 7ff1e37fe640 10 client.17163 handle_command_reply: tid=0 2024-09-21T06:06:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.547+0000 7ff2112a8740 1 client.17163 shutdown 2024-09-21T06:06:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.547+0000 7ff2112a8740 1 -- 172.21.15.117:0/3450352406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5e73f2f0 msgr2=0x55fc5eb19050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.547+0000 7ff2112a8740 1 --2- 172.21.15.117:0/3450352406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5e73f2f0 0x55fc5eb19050 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7ff1f800b470 tx=0x7ff1f80045b0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.547+0000 7ff2112a8740 1 -- 172.21.15.117:0/3450352406 shutdown_connections 2024-09-21T06:06:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.547+0000 7ff2112a8740 1 --2- 172.21.15.117:0/3450352406 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55fc5e726d80 0x55fc5eb2b0b0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.547+0000 7ff2112a8740 1 --2- 172.21.15.117:0/3450352406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc5e7681b0 0x55fc5eb253f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.547+0000 7ff2112a8740 1 --2- 172.21.15.117:0/3450352406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc5e7474d0 0x55fc5eb19590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.547+0000 7ff2112a8740 1 --2- 172.21.15.117:0/3450352406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5e73f2f0 0x55fc5eb19050 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.547+0000 7ff1e37fe640 0 client.17163 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.547+0000 7ff2112a8740 1 -- 172.21.15.117:0/3450352406 >> 172.21.15.117:0/3450352406 conn(0x55fc5e75c300 msgr2=0x55fc5eb0e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.547+0000 7ff2112a8740 1 -- 172.21.15.117:0/3450352406 shutdown_connections 2024-09-21T06:06:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.547+0000 7ff2112a8740 1 -- 172.21.15.117:0/3450352406 wait complete. 2024-09-21T06:06:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.547+0000 7ff2112a8740 20 client.17163 trim_cache size 0 max 16384 2024-09-21T06:06:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.562+0000 7ff2112a8740 1 Processor -- start 2024-09-21T06:06:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.562+0000 7ff2112a8740 1 -- start start 2024-09-21T06:06:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.562+0000 7ff2112a8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc5e73f2f0 0x55fc5eb0a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.562+0000 7ff2112a8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc5e7474d0 0x55fc5eb252f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.562+0000 7ff2112a8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5e7681b0 0x55fc5eb16220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.562+0000 7ff2112a8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff2041af5e0 con 0x55fc5e7681b0 2024-09-21T06:06:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.562+0000 7ff2112a8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff2041b1120 con 0x55fc5e73f2f0 2024-09-21T06:06:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.562+0000 7ff2112a8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff2041afcb0 con 0x55fc5e7474d0 2024-09-21T06:06:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.562+0000 7ff2037fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5e7681b0 0x55fc5eb16220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.562+0000 7ff2037fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5e7681b0 0x55fc5eb16220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59210/0 (socket says 172.21.15.117:59210) 2024-09-21T06:06:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.562+0000 7ff2037fe640 1 -- 172.21.15.117:0/669976966 learned_addr learned my addr 172.21.15.117:0/669976966 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.562+0000 7ff2037fe640 1 --2- 172.21.15.117:0/669976966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc5e73f2f0 0x55fc5eb0a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.562+0000 7ff203fff640 1 --2- 172.21.15.117:0/669976966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc5e7474d0 0x55fc5eb252f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.562+0000 7ff2037fe640 1 -- 172.21.15.117:0/669976966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc5e7474d0 msgr2=0x55fc5eb252f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.562+0000 7ff2037fe640 1 --2- 172.21.15.117:0/669976966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc5e7474d0 0x55fc5eb252f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.562+0000 7ff2037fe640 1 -- 172.21.15.117:0/669976966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc5e73f2f0 msgr2=0x55fc5eb0a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.562+0000 7ff2037fe640 1 --2- 172.21.15.117:0/669976966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc5e73f2f0 0x55fc5eb0a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.562+0000 7ff2037fe640 1 -- 172.21.15.117:0/669976966 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc5eb1b630 con 0x55fc5e7681b0 2024-09-21T06:06:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.563+0000 7ff2037fe640 1 --2- 172.21.15.117:0/669976966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5e7681b0 0x55fc5eb16220 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7ff1f800bfb0 tx=0x7ff1f8021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b4521fae260f9d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.563+0000 7ff1e9ffb640 1 -- 172.21.15.117:0/669976966 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f800b4a0 con 0x55fc5e7681b0 2024-09-21T06:06:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.563+0000 7ff1e9ffb640 1 -- 172.21.15.117:0/669976966 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1f8023070 con 0x55fc5e7681b0 2024-09-21T06:06:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.563+0000 7ff1e9ffb640 1 -- 172.21.15.117:0/669976966 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f800fbf0 con 0x55fc5e7681b0 2024-09-21T06:06:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.563+0000 7ff2112a8740 1 -- 172.21.15.117:0/669976966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5e7681b0 msgr2=0x55fc5eb16220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.563+0000 7ff2112a8740 1 --2- 172.21.15.117:0/669976966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5e7681b0 0x55fc5eb16220 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7ff1f800bfb0 tx=0x7ff1f8021040 comp rx=0 tx=0).stop 2024-09-21T06:06:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.563+0000 7ff2112a8740 1 -- 172.21.15.117:0/669976966 shutdown_connections 2024-09-21T06:06:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.563+0000 7ff2112a8740 1 --2- 172.21.15.117:0/669976966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5e7681b0 0x55fc5eb16220 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.563+0000 7ff2112a8740 1 --2- 172.21.15.117:0/669976966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc5e7474d0 0x55fc5eb252f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.563+0000 7ff2112a8740 1 --2- 172.21.15.117:0/669976966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc5e73f2f0 0x55fc5eb0a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.563+0000 7ff2112a8740 1 -- 172.21.15.117:0/669976966 >> 172.21.15.117:0/669976966 conn(0x55fc5e75c300 msgr2=0x55fc5eb09ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.563+0000 7ff2112a8740 1 -- 172.21.15.117:0/669976966 shutdown_connections 2024-09-21T06:06:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.564+0000 7ff2112a8740 1 -- 172.21.15.117:0/669976966 wait complete. 2024-09-21T06:06:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.564+0000 7ff2112a8740 1 -- start start 2024-09-21T06:06:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.564+0000 7ff2112a8740 1 Processor -- start 2024-09-21T06:06:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.564+0000 7ff2112a8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc5e73f2f0 0x55fc5eb1e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.564+0000 7ff2112a8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5e7474d0 0x55fc5eb1ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.564+0000 7ff2112a8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc5e7681b0 0x55fc5eb938e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.564+0000 7ff2112a8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff2041b0a50 con 0x55fc5e7474d0 2024-09-21T06:06:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.564+0000 7ff2112a8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff2041b0380 con 0x55fc5e73f2f0 2024-09-21T06:06:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.564+0000 7ff2112a8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff2041ad5e0 con 0x55fc5e7681b0 2024-09-21T06:06:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.564+0000 7ff2112a8740 10 client.0 _ll_register_callbacks cb 0x7ff1f80123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.564+0000 7ff203fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5e7474d0 0x55fc5eb1ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.564+0000 7ff203fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5e7474d0 0x55fc5eb1ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:59212/0 (socket says 172.21.15.117:59212) 2024-09-21T06:06:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.564+0000 7ff203fff640 1 -- 172.21.15.117:0/3177533235 learned_addr learned my addr 172.21.15.117:0/3177533235 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.565+0000 7ff2037fe640 1 --2- 172.21.15.117:0/3177533235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc5e73f2f0 0x55fc5eb1e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:06.572 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.565+0000 7ff2037fe640 1 --2- 172.21.15.117:0/3177533235 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc5e7681b0 0x55fc5eb938e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.565+0000 7ff203fff640 1 -- 172.21.15.117:0/3177533235 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc5e7681b0 msgr2=0x55fc5eb938e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.565+0000 7ff203fff640 1 --2- 172.21.15.117:0/3177533235 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc5e7681b0 0x55fc5eb938e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.565+0000 7ff203fff640 1 -- 172.21.15.117:0/3177533235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc5e73f2f0 msgr2=0x55fc5eb1e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.565+0000 7ff203fff640 1 --2- 172.21.15.117:0/3177533235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc5e73f2f0 0x55fc5eb1e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.565+0000 7ff203fff640 1 -- 172.21.15.117:0/3177533235 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc5eb93e20 con 0x55fc5e7474d0 2024-09-21T06:06:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.565+0000 7ff2037fe640 1 --2- 172.21.15.117:0/3177533235 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc5e7681b0 0x55fc5eb938e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:06:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.565+0000 7ff203fff640 1 --2- 172.21.15.117:0/3177533235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5e7474d0 0x55fc5eb1ebf0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x55fc5eb2b620 tx=0x7ff1f40128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e71631f5a7fed0a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.565+0000 7ff1e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.565+0000 7ff1e37fe640 1 -- 172.21.15.117:0/3177533235 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4011850 con 0x55fc5e7474d0 2024-09-21T06:06:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.565+0000 7ff1e37fe640 1 -- 172.21.15.117:0/3177533235 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8441+0+0 (secure 0 0 0) 0x7ff1f40119f0 con 0x55fc5e7474d0 2024-09-21T06:06:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.565+0000 7ff2112a8740 1 -- 172.21.15.117:0/3177533235 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1f8023070 con 0x55fc5e7474d0 2024-09-21T06:06:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.565+0000 7ff1e37fe640 1 -- 172.21.15.117:0/3177533235 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff1f4011bf0 con 0x55fc5e7474d0 2024-09-21T06:06:06.573 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.565+0000 7ff2112a8740 10 client.17169 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.565+0000 7ff2112a8740 1 -- 172.21.15.117:0/3177533235 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fc5eb94f90 con 0x55fc5e7474d0 2024-09-21T06:06:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.566+0000 7ff1e37fe640 1 -- 172.21.15.117:0/3177533235 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1f8023070 con 0x55fc5e7474d0 2024-09-21T06:06:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.566+0000 7ff1e37fe640 1 -- 172.21.15.117:0/3177533235 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff1f4021070 con 0x55fc5e7474d0 2024-09-21T06:06:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.566+0000 7ff2112a8740 10 client.17169 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.566+0000 7ff2112a8740 10 client.17169 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:06:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.566+0000 7ff2112a8740 20 client.17169 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.566+0000 7ff2112a8740 1 --2- 172.21.15.117:0/3177533235 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55fc5e6fc1c0 0x55fc5eb97780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.566+0000 7ff2112a8740 4 client.17169 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:06:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.566+0000 7ff2112a8740 1 -- 172.21.15.117:0/3177533235 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fc5ea800f0 con 0x55fc5e6fc1c0 2024-09-21T06:06:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.566+0000 7ff2037fe640 1 --2- 172.21.15.117:0/3177533235 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55fc5e6fc1c0 0x55fc5eb97780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.566+0000 7ff1e37fe640 10 client.17169 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.566+0000 7ff2037fe640 1 --2- 172.21.15.117:0/3177533235 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55fc5e6fc1c0 0x55fc5eb97780 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b6d763871247a36 server_cookie=4c5b11352c226d3d in_seq=0 out_seq=0 2024-09-21T06:06:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.574+0000 7ff1e37fe640 1 -- 172.21.15.117:0/3177533235 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fc5ea800f0 con 0x55fc5e6fc1c0 2024-09-21T06:06:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.574+0000 7ff1e37fe640 10 client.17169 handle_command_reply: tid=0 2024-09-21T06:06:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.574+0000 7ff2112a8740 1 client.17169 shutdown 2024-09-21T06:06:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.574+0000 7ff2112a8740 1 -- 172.21.15.117:0/3177533235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5e7474d0 msgr2=0x55fc5eb1ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.574+0000 7ff2112a8740 1 --2- 172.21.15.117:0/3177533235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5e7474d0 0x55fc5eb1ebf0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x55fc5eb2b620 tx=0x7ff1f40128e0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.574+0000 7ff2112a8740 1 -- 172.21.15.117:0/3177533235 shutdown_connections 2024-09-21T06:06:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.574+0000 7ff2112a8740 1 --2- 172.21.15.117:0/3177533235 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55fc5e6fc1c0 0x55fc5eb97780 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.574+0000 7ff2112a8740 1 --2- 172.21.15.117:0/3177533235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55fc5e7474d0 0x55fc5eb1ebf0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.574+0000 7ff2112a8740 1 --2- 172.21.15.117:0/3177533235 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fc5e7681b0 0x55fc5eb938e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.574+0000 7ff1e37fe640 0 client.17169 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.574+0000 7ff2112a8740 1 --2- 172.21.15.117:0/3177533235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55fc5e73f2f0 0x55fc5eb1e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.574+0000 7ff2112a8740 1 -- 172.21.15.117:0/3177533235 >> 172.21.15.117:0/3177533235 conn(0x55fc5e75c300 msgr2=0x55fc5eb0b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.574+0000 7ff2112a8740 1 -- 172.21.15.117:0/3177533235 shutdown_connections 2024-09-21T06:06:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.574+0000 7ff2112a8740 1 -- 172.21.15.117:0/3177533235 wait complete. 2024-09-21T06:06:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.574+0000 7ff2112a8740 20 client.17169 trim_cache size 0 max 16384 2024-09-21T06:06:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T06:06:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T06:06:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T06:06:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T06:06:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.575+0000 7ff20a339640 1 -- 172.21.15.117:0/610191161 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff1ec0b6680 msgr2=0x7ff1ec0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.575+0000 7ff20a339640 1 --2- 172.21.15.117:0/610191161 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff1ec0b6680 0x7ff1ec0b8b90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc106460 tx=0x7ff1fc0b6040 comp rx=0 tx=0).stop 2024-09-21T06:06:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.575+0000 7ff20a339640 1 -- 172.21.15.117:0/610191161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff204106b50 msgr2=0x7ff20419fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.575+0000 7ff20a339640 1 --2- 172.21.15.117:0/610191161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff204106b50 0x7ff20419fcb0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc005cc0 tx=0x7ff1fc02f380 comp rx=0 tx=0).stop 2024-09-21T06:06:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.575+0000 7ff20a339640 1 -- 172.21.15.117:0/610191161 shutdown_connections 2024-09-21T06:06:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.575+0000 7ff20a339640 1 --2- 172.21.15.117:0/610191161 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff1ec0b6680 0x7ff1ec0b8b90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.575+0000 7ff20a339640 1 --2- 172.21.15.117:0/610191161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff204106b50 0x7ff20419fcb0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.575+0000 7ff20a339640 1 --2- 172.21.15.117:0/610191161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff204106200 0x7ff20419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.575+0000 7ff20a339640 1 --2- 172.21.15.117:0/610191161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff204102090 0x7ff20419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.575+0000 7ff20a339640 1 -- 172.21.15.117:0/610191161 >> 172.21.15.117:0/610191161 conn(0x7ff204079ec0 msgr2=0x7ff204108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.575+0000 7ff20a339640 1 -- 172.21.15.117:0/610191161 shutdown_connections 2024-09-21T06:06:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:06.575+0000 7ff20a339640 1 -- 172.21.15.117:0/610191161 wait complete. 2024-09-21T06:06:08.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:07 smithi192 ceph-mon[34700]: pgmap v594: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-21T06:06:08.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:07 smithi117 ceph-mon[30947]: pgmap v594: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-21T06:06:08.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:07 smithi138 ceph-mon[34391]: pgmap v594: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-21T06:06:10.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:09 smithi192 ceph-mon[34700]: pgmap v595: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 28 op/s 2024-09-21T06:06:10.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:09 smithi117 ceph-mon[30947]: pgmap v595: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 28 op/s 2024-09-21T06:06:10.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:09 smithi138 ceph-mon[34391]: pgmap v595: 97 pgs: 97 active+clean; 60 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 28 op/s 2024-09-21T06:06:11.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:10 smithi192 ceph-mon[34700]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T06:06:11.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:10 smithi117 ceph-mon[30947]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T06:06:11.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:10 smithi138 ceph-mon[34391]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T06:06:12.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:11 smithi192 ceph-mon[34700]: pgmap v597: 97 pgs: 6 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 82 active+clean; 59 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-21T06:06:12.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:11 smithi117 ceph-mon[30947]: pgmap v597: 97 pgs: 6 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 82 active+clean; 59 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-21T06:06:12.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:11 smithi138 ceph-mon[34391]: pgmap v597: 97 pgs: 6 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 82 active+clean; 59 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-21T06:06:13.968 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:06:14.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:13 smithi192 ceph-mon[34700]: pgmap v598: 97 pgs: 6 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 82 active+clean; 56 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-21T06:06:14.232 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:14.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:13 smithi117 ceph-mon[30947]: pgmap v598: 97 pgs: 6 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 82 active+clean; 56 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-21T06:06:14.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:13 smithi138 ceph-mon[34391]: pgmap v598: 97 pgs: 6 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 82 active+clean; 56 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-21T06:06:14.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.750+0000 7f64d959e640 1 -- 172.21.15.117:0/573645140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64d4108450 msgr2=0x7f64d4108860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:14.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.750+0000 7f64d959e640 1 --2- 172.21.15.117:0/573645140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64d4108450 0x7f64d4108860 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f64bc009900 tx=0x7f64bc02a410 comp rx=0 tx=0).stop 2024-09-21T06:06:14.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.751+0000 7f64d959e640 1 -- 172.21.15.117:0/573645140 shutdown_connections 2024-09-21T06:06:14.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.751+0000 7f64d959e640 1 --2- 172.21.15.117:0/573645140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64d4108450 0x7f64d4108860 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.751+0000 7f64d959e640 1 --2- 172.21.15.117:0/573645140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64d4108da0 0x7f64d410f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.751+0000 7f64d959e640 1 --2- 172.21.15.117:0/573645140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64d4102b10 0x7f64d4102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.751+0000 7f64d959e640 1 -- 172.21.15.117:0/573645140 >> 172.21.15.117:0/573645140 conn(0x7f64d40796d0 msgr2=0x7f64d4079ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:14.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.751+0000 7f64d959e640 1 -- 172.21.15.117:0/573645140 shutdown_connections 2024-09-21T06:06:14.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.751+0000 7f64d959e640 1 -- 172.21.15.117:0/573645140 wait complete. 2024-09-21T06:06:14.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.751+0000 7f64d959e640 1 Processor -- start 2024-09-21T06:06:14.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.751+0000 7f64d959e640 1 -- start start 2024-09-21T06:06:14.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.752+0000 7f64d959e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64d4102b10 0x7f64d4199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:14.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.752+0000 7f64d959e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64d4108450 0x7f64d41999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:14.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.752+0000 7f64d959e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64d4108da0 0x7f64d419ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:14.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.752+0000 7f64d959e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f64d4112a60 con 0x7f64d4108da0 2024-09-21T06:06:14.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.752+0000 7f64d959e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f64d41128e0 con 0x7f64d4102b10 2024-09-21T06:06:14.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.752+0000 7f64d959e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f64d4112be0 con 0x7f64d4108450 2024-09-21T06:06:14.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.752+0000 7f64d37fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64d4108da0 0x7f64d419ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:14.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.752+0000 7f64d37fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64d4108da0 0x7f64d419ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45352/0 (socket says 172.21.15.117:45352) 2024-09-21T06:06:14.927 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.752+0000 7f64d27fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64d4108450 0x7f64d41999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:14.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.752+0000 7f64d37fe640 1 -- 172.21.15.117:0/4068225961 learned_addr learned my addr 172.21.15.117:0/4068225961 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:14.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.752+0000 7f64d2ffd640 1 --2- 172.21.15.117:0/4068225961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64d4102b10 0x7f64d4199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:14.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.752+0000 7f64d37fe640 1 -- 172.21.15.117:0/4068225961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64d4108450 msgr2=0x7f64d41999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:14.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.752+0000 7f64d37fe640 1 --2- 172.21.15.117:0/4068225961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64d4108450 0x7f64d41999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.752+0000 7f64d37fe640 1 -- 172.21.15.117:0/4068225961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64d4102b10 msgr2=0x7f64d4199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:14.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.752+0000 7f64d37fe640 1 --2- 172.21.15.117:0/4068225961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64d4102b10 0x7f64d4199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.752+0000 7f64d37fe640 1 -- 172.21.15.117:0/4068225961 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64d419f380 con 0x7f64d4108da0 2024-09-21T06:06:14.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.752+0000 7f64d37fe640 1 --2- 172.21.15.117:0/4068225961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64d4108da0 0x7f64d419ec50 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f64cc009fd0 tx=0x7f64cc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:14.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.753+0000 7f64cbfff640 1 -- 172.21.15.117:0/4068225961 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64cc017070 con 0x7f64d4108da0 2024-09-21T06:06:14.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.753+0000 7f64d959e640 1 -- 172.21.15.117:0/4068225961 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64d419f670 con 0x7f64d4108da0 2024-09-21T06:06:14.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.753+0000 7f64cbfff640 1 -- 172.21.15.117:0/4068225961 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64cc003e20 con 0x7f64d4108da0 2024-09-21T06:06:14.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.753+0000 7f64cbfff640 1 -- 172.21.15.117:0/4068225961 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64cc010460 con 0x7f64d4108da0 2024-09-21T06:06:14.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.753+0000 7f64d959e640 1 -- 172.21.15.117:0/4068225961 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64d41a5dd0 con 0x7f64d4108da0 2024-09-21T06:06:14.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.756+0000 7f64dfb12740 1 Processor -- start 2024-09-21T06:06:14.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.756+0000 7f64cbfff640 1 -- 172.21.15.117:0/4068225961 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f64cc003970 con 0x7f64d4108da0 2024-09-21T06:06:14.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.756+0000 7f64dfb12740 1 -- start start 2024-09-21T06:06:14.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.757+0000 7f64dfb12740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e46dc0a070 0x55e46dfb1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:14.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.757+0000 7f64dfb12740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e46dbe11b0 0x55e46dfb17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:14.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.757+0000 7f64dfb12740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e46dbe9390 0x55e46dfb1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:14.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.757+0000 7f64dfb12740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e46df27e50 con 0x55e46dc0a070 2024-09-21T06:06:14.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.757+0000 7f64dfb12740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e46df17280 con 0x55e46dbe9390 2024-09-21T06:06:14.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.757+0000 7f64dfb12740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55e46df1b090 con 0x55e46dbe11b0 2024-09-21T06:06:14.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.757+0000 7f64d27fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e46dc0a070 0x55e46dfb1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:14.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.757+0000 7f64d27fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e46dc0a070 0x55e46dfb1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45364/0 (socket says 172.21.15.117:45364) 2024-09-21T06:06:14.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.757+0000 7f64d27fc640 1 -- 172.21.15.117:0/3899504585 learned_addr learned my addr 172.21.15.117:0/3899504585 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:14.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.757+0000 7f64d2ffd640 1 --2- 172.21.15.117:0/3899504585 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e46dbe11b0 0x55e46dfb17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:14.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.757+0000 7f64d27fc640 1 --2- 172.21.15.117:0/3899504585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e46dbe9390 0x55e46dfb1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:14.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.757+0000 7f64cbfff640 1 --2- 172.21.15.117:0/4068225961 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f64c00b6680 0x7f64c00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:14.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.757+0000 7f64d27fc640 1 -- 172.21.15.117:0/3899504585 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e46dbe11b0 msgr2=0x55e46dfb17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:14.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.757+0000 7f64d27fc640 1 --2- 172.21.15.117:0/3899504585 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e46dbe11b0 0x55e46dfb17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.757+0000 7f64d27fc640 1 -- 172.21.15.117:0/3899504585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e46dbe9390 msgr2=0x55e46dfb1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:14.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.757+0000 7f64d27fc640 1 --2- 172.21.15.117:0/3899504585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e46dbe9390 0x55e46dfb1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.757+0000 7f64d27fc640 1 -- 172.21.15.117:0/3899504585 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e46dc75650 con 0x55e46dc0a070 2024-09-21T06:06:14.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.758+0000 7f64d37fe640 1 --2- 172.21.15.117:0/4068225961 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f64c00b6680 0x7f64c00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:14.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.758+0000 7f64cbfff640 1 -- 172.21.15.117:0/4068225961 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8485+0+0 (secure 0 0 0) 0x7f64cc0e7400 con 0x7f64d4108da0 2024-09-21T06:06:14.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.758+0000 7f64d27fc640 1 --2- 172.21.15.117:0/3899504585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e46dc0a070 0x55e46dfb1270 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f64d406b9f0 tx=0x7f64bc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f170b5a67794b43 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:14.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.758+0000 7f64d37fe640 1 --2- 172.21.15.117:0/4068225961 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f64c00b6680 0x7f64c00b8b90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f64cc0aa590 tx=0x7f64cc07a600 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:14.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.758+0000 7f64c97fa640 1 -- 172.21.15.117:0/3899504585 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64bc044020 con 0x55e46dc0a070 2024-09-21T06:06:14.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.758+0000 7f64c97fa640 1 -- 172.21.15.117:0/3899504585 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64bc031d10 con 0x55e46dc0a070 2024-09-21T06:06:14.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.758+0000 7f64c97fa640 1 -- 172.21.15.117:0/3899504585 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64bc035040 con 0x55e46dc0a070 2024-09-21T06:06:14.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.759+0000 7f64dfb12740 1 -- 172.21.15.117:0/3899504585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e46dc0a070 msgr2=0x55e46dfb1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:14.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.760+0000 7f64dfb12740 1 --2- 172.21.15.117:0/3899504585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e46dc0a070 0x55e46dfb1270 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f64d406b9f0 tx=0x7f64bc000bc0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.760+0000 7f64dfb12740 1 -- 172.21.15.117:0/3899504585 shutdown_connections 2024-09-21T06:06:14.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.760+0000 7f64dfb12740 1 --2- 172.21.15.117:0/3899504585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e46dbe9390 0x55e46dfb1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.760+0000 7f64dfb12740 1 --2- 172.21.15.117:0/3899504585 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e46dbe11b0 0x55e46dfb17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.760+0000 7f64dfb12740 1 --2- 172.21.15.117:0/3899504585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e46dc0a070 0x55e46dfb1270 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.760+0000 7f64dfb12740 1 -- 172.21.15.117:0/3899504585 >> 172.21.15.117:0/3899504585 conn(0x55e46dbfe460 msgr2=0x55e46dfaf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:14.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.760+0000 7f64dfb12740 1 -- 172.21.15.117:0/3899504585 shutdown_connections 2024-09-21T06:06:14.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.760+0000 7f64dfb12740 1 -- 172.21.15.117:0/3899504585 wait complete. 2024-09-21T06:06:14.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.760+0000 7f64dfb12740 1 -- start start 2024-09-21T06:06:14.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.760+0000 7f64dfb12740 1 Processor -- start 2024-09-21T06:06:14.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.761+0000 7f64dfb12740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e46dbe11b0 0x55e46dfbeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:14.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.761+0000 7f64dfb12740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e46dbe9390 0x55e46dfbf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:14.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.761+0000 7f64dfb12740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e46dc0a070 0x55e46dfbf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:14.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.761+0000 7f64dfb12740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e46dd78700 con 0x55e46dbe11b0 2024-09-21T06:06:14.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.761+0000 7f64dfb12740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e46deb2110 con 0x55e46dbe9390 2024-09-21T06:06:14.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.761+0000 7f64dfb12740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55e46dfc1b50 con 0x55e46dc0a070 2024-09-21T06:06:14.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.761+0000 7f64dfb12740 10 client.0 _ll_register_callbacks cb 0x55e46decb990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:14.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.761+0000 7f64d27fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e46dbe11b0 0x55e46dfbeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:14.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.761+0000 7f64d2ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e46dbe9390 0x55e46dfbf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:14.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.761+0000 7f64d27fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e46dbe11b0 0x55e46dfbeaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45366/0 (socket says 172.21.15.117:45366) 2024-09-21T06:06:14.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.761+0000 7f64d27fc640 1 -- 172.21.15.117:0/2754513856 learned_addr learned my addr 172.21.15.117:0/2754513856 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:14.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.761+0000 7f64d27fc640 1 --2- 172.21.15.117:0/2754513856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e46dc0a070 0x55e46dfbf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:14.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.762+0000 7f64d2ffd640 1 -- 172.21.15.117:0/2754513856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e46dc0a070 msgr2=0x55e46dfbf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:14.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.762+0000 7f64d2ffd640 1 --2- 172.21.15.117:0/2754513856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e46dc0a070 0x55e46dfbf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.762+0000 7f64d2ffd640 1 -- 172.21.15.117:0/2754513856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e46dbe11b0 msgr2=0x55e46dfbeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:14.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.762+0000 7f64d2ffd640 1 --2- 172.21.15.117:0/2754513856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e46dbe11b0 0x55e46dfbeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.762+0000 7f64d2ffd640 1 -- 172.21.15.117:0/2754513856 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e46dedfc40 con 0x55e46dbe9390 2024-09-21T06:06:14.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.762+0000 7f64d27fc640 1 --2- 172.21.15.117:0/2754513856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e46dbe11b0 0x55e46dfbeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:06:14.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.762+0000 7f64d2ffd640 1 --2- 172.21.15.117:0/2754513856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e46dbe9390 0x55e46dfbf030 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f64c40126b0 tx=0x7f64c4012b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37f8a7c8ebbd17d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:14.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.762+0000 7f64ba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-21T06:06:14.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.762+0000 7f64ba7fc640 1 -- 172.21.15.117:0/2754513856 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c4004020 con 0x55e46dbe9390 2024-09-21T06:06:14.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.762+0000 7f64ba7fc640 1 -- 172.21.15.117:0/2754513856 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8485+0+0 (secure 0 0 0) 0x7f64c40041c0 con 0x55e46dbe9390 2024-09-21T06:06:14.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.762+0000 7f64dfb12740 1 -- 172.21.15.117:0/2754513856 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64bc031d10 con 0x55e46dbe9390 2024-09-21T06:06:14.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.762+0000 7f64ba7fc640 1 -- 172.21.15.117:0/2754513856 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f64c40043c0 con 0x55e46dbe9390 2024-09-21T06:06:14.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.762+0000 7f64dfb12740 10 client.26237 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:14.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.762+0000 7f64dfb12740 1 -- 172.21.15.117:0/2754513856 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e46dfcd910 con 0x55e46dbe9390 2024-09-21T06:06:14.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.763+0000 7f64ba7fc640 1 -- 172.21.15.117:0/2754513856 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64bc031d10 con 0x55e46dbe9390 2024-09-21T06:06:14.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.763+0000 7f64ba7fc640 1 -- 172.21.15.117:0/2754513856 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f64c4003b90 con 0x55e46dbe9390 2024-09-21T06:06:14.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.763+0000 7f64dfb12740 10 client.26237 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:14.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.763+0000 7f64dfb12740 10 client.26237 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:06:14.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.763+0000 7f64dfb12740 20 client.26237 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:14.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.763+0000 7f64dfb12740 1 --2- 172.21.15.117:0/2754513856 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55e46dbc89f0 0x55e46dfd0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:14.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.763+0000 7f64dfb12740 4 client.26237 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:06:14.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.763+0000 7f64dfb12740 1 -- 172.21.15.117:0/2754513856 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e46df29da0 con 0x55e46dbc89f0 2024-09-21T06:06:14.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.763+0000 7f64d27fc640 1 --2- 172.21.15.117:0/2754513856 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55e46dbc89f0 0x55e46dfd0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:14.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.764+0000 7f64ba7fc640 10 client.26237 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:14.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.764+0000 7f64d27fc640 1 --2- 172.21.15.117:0/2754513856 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55e46dbc89f0 0x55e46dfd0e80 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e14bacdad97852f server_cookie=90c569d25e39ae64 in_seq=0 out_seq=0 2024-09-21T06:06:14.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.764+0000 7f64ba7fc640 1 -- 172.21.15.117:0/2754513856 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e46df29da0 con 0x55e46dbc89f0 2024-09-21T06:06:14.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.764+0000 7f64ba7fc640 10 client.26237 handle_command_reply: tid=0 2024-09-21T06:06:14.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.765+0000 7f64dfb12740 1 client.26237 shutdown 2024-09-21T06:06:14.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.765+0000 7f64dfb12740 1 -- 172.21.15.117:0/2754513856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e46dbe9390 msgr2=0x55e46dfbf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:14.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.765+0000 7f64dfb12740 1 --2- 172.21.15.117:0/2754513856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e46dbe9390 0x55e46dfbf030 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f64c40126b0 tx=0x7f64c4012b70 comp rx=0 tx=0).stop 2024-09-21T06:06:14.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.765+0000 7f64dfb12740 1 -- 172.21.15.117:0/2754513856 shutdown_connections 2024-09-21T06:06:14.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.765+0000 7f64dfb12740 1 --2- 172.21.15.117:0/2754513856 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55e46dbc89f0 0x55e46dfd0e80 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.765+0000 7f64dfb12740 1 --2- 172.21.15.117:0/2754513856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e46dc0a070 0x55e46dfbf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.765+0000 7f64ba7fc640 0 client.26237 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:14.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.765+0000 7f64dfb12740 1 --2- 172.21.15.117:0/2754513856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e46dbe9390 0x55e46dfbf030 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.765+0000 7f64dfb12740 1 --2- 172.21.15.117:0/2754513856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e46dbe11b0 0x55e46dfbeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.765+0000 7f64dfb12740 1 -- 172.21.15.117:0/2754513856 >> 172.21.15.117:0/2754513856 conn(0x55e46dbfe460 msgr2=0x55e46dfb2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:14.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.765+0000 7f64dfb12740 1 -- 172.21.15.117:0/2754513856 shutdown_connections 2024-09-21T06:06:14.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.765+0000 7f64dfb12740 1 -- 172.21.15.117:0/2754513856 wait complete. 2024-09-21T06:06:14.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.765+0000 7f64dfb12740 20 client.26237 trim_cache size 0 max 16384 2024-09-21T06:06:14.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.782+0000 7f64dfb12740 1 Processor -- start 2024-09-21T06:06:14.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.782+0000 7f64dfb12740 1 -- start start 2024-09-21T06:06:14.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.783+0000 7f64dfb12740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e46dbfe460 0x55e46dfc09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:14.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.783+0000 7f64dfb12740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e46dbc89f0 0x55e46dfb29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:14.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.783+0000 7f64dfb12740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e46dbe11b0 0x55e46dfbbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:14.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.783+0000 7f64dfb12740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f64d41ad2f0 con 0x55e46dbfe460 2024-09-21T06:06:14.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.783+0000 7f64dfb12740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f64d41aee30 con 0x55e46dbe11b0 2024-09-21T06:06:14.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.783+0000 7f64dfb12740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f64d41ad9c0 con 0x55e46dbc89f0 2024-09-21T06:06:14.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.783+0000 7f64d27fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e46dbfe460 0x55e46dfc09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:14.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.783+0000 7f64d27fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e46dbfe460 0x55e46dfc09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45380/0 (socket says 172.21.15.117:45380) 2024-09-21T06:06:14.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.783+0000 7f64d27fc640 1 -- 172.21.15.117:0/2978017358 learned_addr learned my addr 172.21.15.117:0/2978017358 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:14.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.783+0000 7f64d27fc640 1 --2- 172.21.15.117:0/2978017358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e46dbe11b0 0x55e46dfbbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:14.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.783+0000 7f64d2ffd640 1 --2- 172.21.15.117:0/2978017358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e46dbc89f0 0x55e46dfb29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:14.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.783+0000 7f64d27fc640 1 -- 172.21.15.117:0/2978017358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e46dbc89f0 msgr2=0x55e46dfb29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:14.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.783+0000 7f64d27fc640 1 --2- 172.21.15.117:0/2978017358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e46dbc89f0 0x55e46dfb29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.783+0000 7f64d27fc640 1 -- 172.21.15.117:0/2978017358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e46dbe11b0 msgr2=0x55e46dfbbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:14.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.783+0000 7f64d27fc640 1 --2- 172.21.15.117:0/2978017358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e46dbe11b0 0x55e46dfbbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.783+0000 7f64d27fc640 1 -- 172.21.15.117:0/2978017358 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e46dfcadb0 con 0x55e46dbfe460 2024-09-21T06:06:14.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.783+0000 7f64d27fc640 1 --2- 172.21.15.117:0/2978017358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e46dbfe460 0x55e46dfc09d0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f64bc033000 tx=0x7f64bc03f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a02101af310faf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:14.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.784+0000 7f64c8ff9640 1 -- 172.21.15.117:0/2978017358 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64bc044040 con 0x55e46dbfe460 2024-09-21T06:06:14.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.784+0000 7f64c8ff9640 1 -- 172.21.15.117:0/2978017358 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64bc005500 con 0x55e46dbfe460 2024-09-21T06:06:14.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.784+0000 7f64c8ff9640 1 -- 172.21.15.117:0/2978017358 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64bc035040 con 0x55e46dbfe460 2024-09-21T06:06:14.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.784+0000 7f64dfb12740 1 -- 172.21.15.117:0/2978017358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e46dbfe460 msgr2=0x55e46dfc09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:14.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.784+0000 7f64dfb12740 1 --2- 172.21.15.117:0/2978017358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e46dbfe460 0x55e46dfc09d0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f64bc033000 tx=0x7f64bc03f040 comp rx=0 tx=0).stop 2024-09-21T06:06:14.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.785+0000 7f64dfb12740 1 -- 172.21.15.117:0/2978017358 shutdown_connections 2024-09-21T06:06:14.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.785+0000 7f64dfb12740 1 --2- 172.21.15.117:0/2978017358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e46dbe11b0 0x55e46dfbbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.785+0000 7f64dfb12740 1 --2- 172.21.15.117:0/2978017358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e46dbc89f0 0x55e46dfb29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.785+0000 7f64dfb12740 1 --2- 172.21.15.117:0/2978017358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e46dbfe460 0x55e46dfc09d0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.785+0000 7f64dfb12740 1 -- 172.21.15.117:0/2978017358 >> 172.21.15.117:0/2978017358 conn(0x55e46dfaa340 msgr2=0x55e46dfb22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:14.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.785+0000 7f64dfb12740 1 -- 172.21.15.117:0/2978017358 shutdown_connections 2024-09-21T06:06:14.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.785+0000 7f64dfb12740 1 -- 172.21.15.117:0/2978017358 wait complete. 2024-09-21T06:06:14.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.785+0000 7f64dfb12740 1 -- start start 2024-09-21T06:06:14.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.785+0000 7f64dfb12740 1 Processor -- start 2024-09-21T06:06:14.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.786+0000 7f64dfb12740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e46dbc89f0 0x55e46dfc4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:14.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.786+0000 7f64dfb12740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e46dbe11b0 0x55e46dfc5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:14.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.786+0000 7f64dfb12740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e46dbfe460 0x55e46dfc5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:14.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.786+0000 7f64dfb12740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f64d41ae760 con 0x55e46dbe11b0 2024-09-21T06:06:14.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.786+0000 7f64dfb12740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f64d41ae090 con 0x55e46dbc89f0 2024-09-21T06:06:14.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.786+0000 7f64dfb12740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f64d41ab2f0 con 0x55e46dbfe460 2024-09-21T06:06:14.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.786+0000 7f64dfb12740 10 client.0 _ll_register_callbacks cb 0x7f64c4009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:14.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.786+0000 7f64d2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e46dbe11b0 0x55e46dfc5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:14.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.786+0000 7f64d27fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e46dbc89f0 0x55e46dfc4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:14.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.786+0000 7f64d2ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e46dbe11b0 0x55e46dfc5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45396/0 (socket says 172.21.15.117:45396) 2024-09-21T06:06:14.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.786+0000 7f64d2ffd640 1 -- 172.21.15.117:0/4137572482 learned_addr learned my addr 172.21.15.117:0/4137572482 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:14.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.786+0000 7f64d27fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e46dbc89f0 0x55e46dfc4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:50872/0 (socket says 172.21.15.117:50872) 2024-09-21T06:06:14.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.786+0000 7f64d27fc640 1 --2- 172.21.15.117:0/4137572482 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e46dbfe460 0x55e46dfc5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:14.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.786+0000 7f64d2ffd640 1 -- 172.21.15.117:0/4137572482 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e46dbfe460 msgr2=0x55e46dfc5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:14.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.786+0000 7f64d2ffd640 1 --2- 172.21.15.117:0/4137572482 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e46dbfe460 0x55e46dfc5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.786+0000 7f64d2ffd640 1 -- 172.21.15.117:0/4137572482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e46dbc89f0 msgr2=0x55e46dfc4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:14.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.786+0000 7f64d2ffd640 1 --2- 172.21.15.117:0/4137572482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e46dbc89f0 0x55e46dfc4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.786+0000 7f64d2ffd640 1 -- 172.21.15.117:0/4137572482 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e46e03a890 con 0x55e46dbe11b0 2024-09-21T06:06:14.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.787+0000 7f64d2ffd640 1 --2- 172.21.15.117:0/4137572482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e46dbe11b0 0x55e46dfc5320 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f64c4001eb0 tx=0x7f64c40042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a5674e51af6b2f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:14.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.787+0000 7f64ba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:14.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.787+0000 7f64ba7fc640 1 -- 172.21.15.117:0/4137572482 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c4029040 con 0x55e46dbe11b0 2024-09-21T06:06:14.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.787+0000 7f64ba7fc640 1 -- 172.21.15.117:0/4137572482 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8485+0+0 (secure 0 0 0) 0x7f64c400f9a0 con 0x55e46dbe11b0 2024-09-21T06:06:14.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.787+0000 7f64dfb12740 1 -- 172.21.15.117:0/4137572482 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64bc005500 con 0x55e46dbe11b0 2024-09-21T06:06:14.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.787+0000 7f64ba7fc640 1 -- 172.21.15.117:0/4137572482 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f64c400fba0 con 0x55e46dbe11b0 2024-09-21T06:06:14.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.787+0000 7f64dfb12740 10 client.17187 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:14.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.787+0000 7f64dfb12740 1 -- 172.21.15.117:0/4137572482 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e46e03baf0 con 0x55e46dbe11b0 2024-09-21T06:06:14.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.787+0000 7f64ba7fc640 1 -- 172.21.15.117:0/4137572482 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64bc005500 con 0x55e46dbe11b0 2024-09-21T06:06:14.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.787+0000 7f64ba7fc640 1 -- 172.21.15.117:0/4137572482 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f64c4035020 con 0x55e46dbe11b0 2024-09-21T06:06:14.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.787+0000 7f64dfb12740 10 client.17187 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:14.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.787+0000 7f64dfb12740 10 client.17187 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:06:14.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.787+0000 7f64dfb12740 20 client.17187 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:14.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.788+0000 7f64dfb12740 1 --2- 172.21.15.117:0/4137572482 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55e46dbe9390 0x55e46e03e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:14.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.788+0000 7f64dfb12740 4 client.17187 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:06:14.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.788+0000 7f64dfb12740 1 -- 172.21.15.117:0/4137572482 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e46df25850 con 0x55e46dbe9390 2024-09-21T06:06:14.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.788+0000 7f64d27fc640 1 --2- 172.21.15.117:0/4137572482 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55e46dbe9390 0x55e46e03e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:14.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.788+0000 7f64ba7fc640 10 client.17187 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:14.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.788+0000 7f64d27fc640 1 --2- 172.21.15.117:0/4137572482 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55e46dbe9390 0x55e46e03e230 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20bd4f3b5644bcef server_cookie=efd04b7be8da0df8 in_seq=0 out_seq=0 2024-09-21T06:06:14.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.788+0000 7f64ba7fc640 1 -- 172.21.15.117:0/4137572482 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2362 (crc 0 0 0) 0x55e46df25850 con 0x55e46dbe9390 2024-09-21T06:06:14.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.788+0000 7f64ba7fc640 10 client.17187 handle_command_reply: tid=0 2024-09-21T06:06:14.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.788+0000 7f64dfb12740 1 client.17187 shutdown 2024-09-21T06:06:14.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.789+0000 7f64dfb12740 1 -- 172.21.15.117:0/4137572482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e46dbe11b0 msgr2=0x55e46dfc5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:14.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.789+0000 7f64dfb12740 1 --2- 172.21.15.117:0/4137572482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e46dbe11b0 0x55e46dfc5320 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f64c4001eb0 tx=0x7f64c40042a0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.789+0000 7f64dfb12740 1 -- 172.21.15.117:0/4137572482 shutdown_connections 2024-09-21T06:06:14.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.789+0000 7f64dfb12740 1 --2- 172.21.15.117:0/4137572482 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55e46dbe9390 0x55e46e03e230 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.789+0000 7f64dfb12740 1 --2- 172.21.15.117:0/4137572482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e46dbe11b0 0x55e46dfc5320 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.789+0000 7f64dfb12740 1 --2- 172.21.15.117:0/4137572482 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e46dbfe460 0x55e46dfc5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.789+0000 7f64dfb12740 1 --2- 172.21.15.117:0/4137572482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e46dbc89f0 0x55e46dfc4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.789+0000 7f64dfb12740 1 -- 172.21.15.117:0/4137572482 >> 172.21.15.117:0/4137572482 conn(0x55e46dfaa340 msgr2=0x55e46dfc0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:14.945 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T06:06:14.945 INFO:teuthology.orchestra.run.smithi117.stdout: "status": "no active scrubs running", 2024-09-21T06:06:14.945 INFO:teuthology.orchestra.run.smithi117.stdout: "scrubs": {}, 2024-09-21T06:06:14.945 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_stats": { 2024-09-21T06:06:14.945 INFO:teuthology.orchestra.run.smithi117.stdout: "39eeca8a-6b4f-465f-af34-b54f77320d70": { 2024-09-21T06:06:14.946 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:05:41.497141+0000", 2024-09-21T06:06:14.946 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:06:14.946 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:06:14.946 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:06:14.946 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:06:14.946 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:06:14.946 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:06:14.946 INFO:teuthology.orchestra.run.smithi117.stdout: "29ff9b7c-49d9-4910-85be-bace7e940cd5": { 2024-09-21T06:06:14.946 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:03:03.775527+0000", 2024-09-21T06:06:14.946 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:06:14.946 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:06:14.946 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:06:14.946 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:06:14.946 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:06:14.947 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:06:14.947 INFO:teuthology.orchestra.run.smithi117.stdout: "0902e33a-ce33-45fb-92d8-2e6fcb8ec8a0": { 2024-09-21T06:06:14.947 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:04:11.365073+0000", 2024-09-21T06:06:14.947 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:06:14.947 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:06:14.947 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:06:14.947 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:06:14.947 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:06:14.947 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:06:14.947 INFO:teuthology.orchestra.run.smithi117.stdout: "d543b0ae-b710-4153-99da-fe5714839c1a": { 2024-09-21T06:06:14.947 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:01:20.574754+0000", 2024-09-21T06:06:14.947 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:06:14.947 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:06:14.947 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:06:14.948 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:06:14.948 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:06:14.948 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:06:14.948 INFO:teuthology.orchestra.run.smithi117.stdout: "ad80d9f1-6aff-46c0-91ab-2c1ab0681039": { 2024-09-21T06:06:14.948 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:59:31.949108+0000", 2024-09-21T06:06:14.948 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:06:14.948 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:06:14.948 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:06:14.948 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:06:14.948 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:06:14.948 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:06:14.948 INFO:teuthology.orchestra.run.smithi117.stdout: "fb2f217d-463b-4c8f-ab19-920b04bb7306": { 2024-09-21T06:06:14.948 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:04:56.308107+0000", 2024-09-21T06:06:14.948 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:06:14.949 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:06:14.949 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:06:14.949 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:06:14.949 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:06:14.949 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:06:14.949 INFO:teuthology.orchestra.run.smithi117.stdout: "8b3fe95a-b76b-4e94-afd0-72fe35e2dd92": { 2024-09-21T06:06:14.949 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:58:01.246076+0000", 2024-09-21T06:06:14.949 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:06:14.949 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:06:14.949 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:06:14.949 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:06:14.949 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:06:14.949 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:06:14.949 INFO:teuthology.orchestra.run.smithi117.stdout: "b30bfff0-0f6b-4e30-8467-ff7db421eba6": { 2024-09-21T06:06:14.949 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:57:14.939082+0000", 2024-09-21T06:06:14.949 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:06:14.950 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:06:14.950 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:06:14.950 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:06:14.950 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:06:14.950 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:06:14.950 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:06:14.950 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T06:06:14.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.789+0000 7f64ba7fc640 0 client.17187 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:14.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.789+0000 7f64dfb12740 1 -- 172.21.15.117:0/4137572482 shutdown_connections 2024-09-21T06:06:14.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.789+0000 7f64dfb12740 1 -- 172.21.15.117:0/4137572482 wait complete. 2024-09-21T06:06:14.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.789+0000 7f64dfb12740 20 client.17187 trim_cache size 0 max 16384 2024-09-21T06:06:14.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.790+0000 7f64d959e640 1 -- 172.21.15.117:0/4068225961 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f64c00b6680 msgr2=0x7f64c00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:14.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.790+0000 7f64d959e640 1 --2- 172.21.15.117:0/4068225961 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f64c00b6680 0x7f64c00b8b90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f64cc0aa590 tx=0x7f64cc07a600 comp rx=0 tx=0).stop 2024-09-21T06:06:14.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.790+0000 7f64d959e640 1 -- 172.21.15.117:0/4068225961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64d4108da0 msgr2=0x7f64d419ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:14.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.790+0000 7f64d959e640 1 --2- 172.21.15.117:0/4068225961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64d4108da0 0x7f64d419ec50 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f64cc009fd0 tx=0x7f64cc00b260 comp rx=0 tx=0).stop 2024-09-21T06:06:14.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.790+0000 7f64d959e640 1 -- 172.21.15.117:0/4068225961 shutdown_connections 2024-09-21T06:06:14.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.790+0000 7f64d959e640 1 --2- 172.21.15.117:0/4068225961 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f64c00b6680 0x7f64c00b8b90 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.790+0000 7f64d959e640 1 --2- 172.21.15.117:0/4068225961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64d4108da0 0x7f64d419ec50 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.790+0000 7f64d959e640 1 --2- 172.21.15.117:0/4068225961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64d4108450 0x7f64d41999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.790+0000 7f64d959e640 1 --2- 172.21.15.117:0/4068225961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64d4102b10 0x7f64d4199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:14.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.790+0000 7f64d959e640 1 -- 172.21.15.117:0/4068225961 >> 172.21.15.117:0/4068225961 conn(0x7f64d40796d0 msgr2=0x7f64d41044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:14.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.790+0000 7f64d959e640 1 -- 172.21.15.117:0/4068225961 shutdown_connections 2024-09-21T06:06:14.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:14.790+0000 7f64d959e640 1 -- 172.21.15.117:0/4068225961 wait complete. 2024-09-21T06:06:16.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:15 smithi117 ceph-mon[30947]: pgmap v599: 97 pgs: 6 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 82 active+clean; 51 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-09-21T06:06:16.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:15 smithi138 ceph-mon[34391]: pgmap v599: 97 pgs: 6 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 82 active+clean; 51 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-09-21T06:06:16.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:15 smithi192 ceph-mon[34700]: pgmap v599: 97 pgs: 6 active+clean+snaptrim, 9 active+clean+snaptrim_wait, 82 active+clean; 51 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-09-21T06:06:16.485 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:06:16.486 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:06:16.732 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:17.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.233+0000 7f0e0090f640 1 -- 172.21.15.117:0/2380802829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dfc108d10 msgr2=0x7f0dfc10f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:17.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.233+0000 7f0e0090f640 1 --2- 172.21.15.117:0/2380802829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dfc108d10 0x7f0dfc10f540 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f0df4009ca0 tx=0x7f0df402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:06:17.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.233+0000 7f0e0090f640 1 -- 172.21.15.117:0/2380802829 shutdown_connections 2024-09-21T06:06:17.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.233+0000 7f0e0090f640 1 --2- 172.21.15.117:0/2380802829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dfc108d10 0x7f0dfc10f540 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:17.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.233+0000 7f0e0090f640 1 --2- 172.21.15.117:0/2380802829 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0dfc1083c0 0x7f0dfc1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:17.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.233+0000 7f0e0090f640 1 --2- 172.21.15.117:0/2380802829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dfc102a80 0x7f0dfc102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:17.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.233+0000 7f0e0090f640 1 -- 172.21.15.117:0/2380802829 >> 172.21.15.117:0/2380802829 conn(0x7f0dfc079740 msgr2=0x7f0dfc079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:17.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.233+0000 7f0e0090f640 1 -- 172.21.15.117:0/2380802829 shutdown_connections 2024-09-21T06:06:17.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.233+0000 7f0e0090f640 1 -- 172.21.15.117:0/2380802829 wait complete. 2024-09-21T06:06:17.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.233+0000 7f0e0090f640 1 Processor -- start 2024-09-21T06:06:17.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.234+0000 7f0e0090f640 1 -- start start 2024-09-21T06:06:17.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.234+0000 7f0e0090f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0dfc102a80 0x7f0dfc0ff690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:17.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.234+0000 7f0e0090f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dfc1083c0 0x7f0dfc0ffbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:17.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.234+0000 7f0e0090f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dfc108d10 0x7f0dfc100110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:17.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.234+0000 7f0e0090f640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0dfc112980 con 0x7f0dfc1083c0 2024-09-21T06:06:17.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.234+0000 7f0e0090f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0dfc112800 con 0x7f0dfc108d10 2024-09-21T06:06:17.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.234+0000 7f0e0090f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0dfc112b00 con 0x7f0dfc102a80 2024-09-21T06:06:17.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.234+0000 7f0df3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dfc1083c0 0x7f0dfc0ffbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:17.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.234+0000 7f0df3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dfc1083c0 0x7f0dfc0ffbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45414/0 (socket says 172.21.15.117:45414) 2024-09-21T06:06:17.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.234+0000 7f0df3fff640 1 -- 172.21.15.117:0/3073275467 learned_addr learned my addr 172.21.15.117:0/3073275467 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:17.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.234+0000 7f0dfa59c640 1 --2- 172.21.15.117:0/3073275467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0dfc102a80 0x7f0dfc0ff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:17.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.234+0000 7f0dfad9d640 1 --2- 172.21.15.117:0/3073275467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dfc108d10 0x7f0dfc100110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:17.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.234+0000 7f0df3fff640 1 -- 172.21.15.117:0/3073275467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0dfc102a80 msgr2=0x7f0dfc0ff690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:17.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.234+0000 7f0df3fff640 1 --2- 172.21.15.117:0/3073275467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0dfc102a80 0x7f0dfc0ff690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:17.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.234+0000 7f0df3fff640 1 -- 172.21.15.117:0/3073275467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dfc108d10 msgr2=0x7f0dfc100110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:17.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.234+0000 7f0df3fff640 1 --2- 172.21.15.117:0/3073275467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dfc108d10 0x7f0dfc100110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:17.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.234+0000 7f0df3fff640 1 -- 172.21.15.117:0/3073275467 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dfc1a8700 con 0x7f0dfc1083c0 2024-09-21T06:06:17.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.234+0000 7f0df3fff640 1 --2- 172.21.15.117:0/3073275467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dfc1083c0 0x7f0dfc0ffbd0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f0de400d770 tx=0x7f0de400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:17.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.235+0000 7f0df37fe640 1 -- 172.21.15.117:0/3073275467 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de4017070 con 0x7f0dfc1083c0 2024-09-21T06:06:17.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.235+0000 7f0e0090f640 1 -- 172.21.15.117:0/3073275467 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dfc1a89f0 con 0x7f0dfc1083c0 2024-09-21T06:06:17.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.235+0000 7f0df37fe640 1 -- 172.21.15.117:0/3073275467 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0de4009c30 con 0x7f0dfc1083c0 2024-09-21T06:06:17.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.235+0000 7f0df37fe640 1 -- 172.21.15.117:0/3073275467 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0de400f620 con 0x7f0dfc1083c0 2024-09-21T06:06:17.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.235+0000 7f0e0090f640 1 -- 172.21.15.117:0/3073275467 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dfc1a8f30 con 0x7f0dfc1083c0 2024-09-21T06:06:17.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.236+0000 7f0e0090f640 1 -- 172.21.15.117:0/3073275467 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dfc103d00 con 0x7f0dfc1083c0 2024-09-21T06:06:17.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.238+0000 7f0df37fe640 1 -- 172.21.15.117:0/3073275467 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f0de401a070 con 0x7f0dfc1083c0 2024-09-21T06:06:17.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.239+0000 7f0df37fe640 1 --2- 172.21.15.117:0/3073275467 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0de80b6680 0x7f0de80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:17.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.239+0000 7f0dfa59c640 1 --2- 172.21.15.117:0/3073275467 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0de80b6680 0x7f0de80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:17.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.239+0000 7f0df37fe640 1 -- 172.21.15.117:0/3073275467 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8485+0+0 (secure 0 0 0) 0x7f0de40e71e0 con 0x7f0dfc1083c0 2024-09-21T06:06:17.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.239+0000 7f0dfa59c640 1 --2- 172.21.15.117:0/3073275467 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0de80b6680 0x7f0de80b8b90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc1010e0 tx=0x7f0dec00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:17.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.242+0000 7f0df37fe640 1 -- 172.21.15.117:0/3073275467 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0de4009de0 con 0x7f0dfc1083c0 2024-09-21T06:06:17.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.460+0000 7f0e0090f640 1 -- 172.21.15.117:0/3073275467 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0dfc100990 con 0x7f0dfc1083c0 2024-09-21T06:06:17.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.460+0000 7f0df37fe640 1 -- 172.21.15.117:0/3073275467 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+12001 (secure 0 0 0) 0x7f0de40ad750 con 0x7f0dfc1083c0 2024-09-21T06:06:17.463 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T06:06:17.463 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":29,"btime":"2024-09-21T06:04:08:952483+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3227057504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3227057504},{"type":"v1","addr":"172.21.15.117:6835","nonce":3227057504}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3303829183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3303829183},{"type":"v1","addr":"172.21.15.117:6837","nonce":3303829183}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T05:55:14.555897+0000","modified":"2024-09-21T06:04:08.952466+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24638,"mds_1":24637,"mds_2":24628,"mds_3":24616,"mds_4":24602},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6839/2314572411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2314572411},{"type":"v1","addr":"172.21.15.117:6839","nonce":2314572411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6841/3378096561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":3378096561},{"type":"v1","addr":"172.21.15.117:6841","nonce":3378096561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/3725629862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3725629862},{"type":"v1","addr":"172.21.15.138:6833","nonce":3725629862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/4276801396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4276801396},{"type":"v1","addr":"172.21.15.138:6835","nonce":4276801396}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.138:6837/2540335374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2540335374},{"type":"v1","addr":"172.21.15.138:6837","nonce":2540335374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24616":{"gid":24616,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.192:6835/3530688860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3530688860},{"type":"v1","addr":"172.21.15.192:6835","nonce":3530688860}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.192:6837/2683860842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2683860842},{"type":"v1","addr":"172.21.15.192:6837","nonce":2683860842}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.192:6839/1727623231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1727623231},{"type":"v1","addr":"172.21.15.192:6839","nonce":1727623231}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/3230372576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":3230372576},{"type":"v1","addr":"172.21.15.138:6839","nonce":3230372576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25840":{"gid":25840,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6833/1596605058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1596605058},{"type":"v1","addr":"172.21.15.192:6833","nonce":1596605058}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24602,24628,24616,24637,24638]},"id":1}]} 2024-09-21T06:06:17.463 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 29 2024-09-21T06:06:17.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.463+0000 7f0e0090f640 1 -- 172.21.15.117:0/3073275467 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0de80b6680 msgr2=0x7f0de80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:17.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.463+0000 7f0e0090f640 1 --2- 172.21.15.117:0/3073275467 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0de80b6680 0x7f0de80b8b90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc1010e0 tx=0x7f0dec00b390 comp rx=0 tx=0).stop 2024-09-21T06:06:17.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.463+0000 7f0e0090f640 1 -- 172.21.15.117:0/3073275467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dfc1083c0 msgr2=0x7f0dfc0ffbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:17.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.463+0000 7f0e0090f640 1 --2- 172.21.15.117:0/3073275467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dfc1083c0 0x7f0dfc0ffbd0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f0de400d770 tx=0x7f0de400dc30 comp rx=0 tx=0).stop 2024-09-21T06:06:17.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.463+0000 7f0e0090f640 1 -- 172.21.15.117:0/3073275467 shutdown_connections 2024-09-21T06:06:17.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.463+0000 7f0e0090f640 1 --2- 172.21.15.117:0/3073275467 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0de80b6680 0x7f0de80b8b90 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:17.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.463+0000 7f0e0090f640 1 --2- 172.21.15.117:0/3073275467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dfc1083c0 0x7f0dfc0ffbd0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:17.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.463+0000 7f0e0090f640 1 --2- 172.21.15.117:0/3073275467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0dfc108d10 0x7f0dfc100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:17.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.463+0000 7f0e0090f640 1 --2- 172.21.15.117:0/3073275467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0dfc102a80 0x7f0dfc0ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:17.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.463+0000 7f0e0090f640 1 -- 172.21.15.117:0/3073275467 >> 172.21.15.117:0/3073275467 conn(0x7f0dfc079740 msgr2=0x7f0dfc06bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:17.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.463+0000 7f0e0090f640 1 -- 172.21.15.117:0/3073275467 shutdown_connections 2024-09-21T06:06:17.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:17.463+0000 7f0e0090f640 1 -- 172.21.15.117:0/3073275467 wait complete. 2024-09-21T06:06:18.146 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:17 smithi117 ceph-mon[30947]: pgmap v600: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 921 B/s wr, 10 op/s 2024-09-21T06:06:18.146 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:17 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3073275467' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:06:18.146 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:17 smithi117 ceph-mon[30947]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T06:06:18.171 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T06:06:18.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:17 smithi138 ceph-mon[34391]: pgmap v600: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 921 B/s wr, 10 op/s 2024-09-21T06:06:18.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:17 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3073275467' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:06:18.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:17 smithi138 ceph-mon[34391]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T06:06:18.418 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:18.479 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:17 smithi192 ceph-mon[34700]: pgmap v600: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 921 B/s wr, 10 op/s 2024-09-21T06:06:18.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:17 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3073275467' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:06:18.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:17 smithi192 ceph-mon[34700]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T06:06:19.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.010+0000 7ff9b4356640 1 -- 172.21.15.117:0/1482643517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9ac109df0 msgr2=0x7ff9ac110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:19.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.010+0000 7ff9b4356640 1 --2- 172.21.15.117:0/1482643517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9ac109df0 0x7ff9ac110620 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4009ca0 tx=0x7ff9a402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.010+0000 7ff9b4356640 1 -- 172.21.15.117:0/1482643517 shutdown_connections 2024-09-21T06:06:19.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.010+0000 7ff9b4356640 1 --2- 172.21.15.117:0/1482643517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9ac109df0 0x7ff9ac110620 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.010+0000 7ff9b4356640 1 --2- 172.21.15.117:0/1482643517 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9ac1094a0 0x7ff9ac1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.010+0000 7ff9b4356640 1 --2- 172.21.15.117:0/1482643517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9ac103b60 0x7ff9ac103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.010+0000 7ff9b4356640 1 -- 172.21.15.117:0/1482643517 >> 172.21.15.117:0/1482643517 conn(0x7ff9ac07b850 msgr2=0x7ff9ac07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:19.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.010+0000 7ff9b4356640 1 -- 172.21.15.117:0/1482643517 shutdown_connections 2024-09-21T06:06:19.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.010+0000 7ff9b4356640 1 -- 172.21.15.117:0/1482643517 wait complete. 2024-09-21T06:06:19.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.011+0000 7ff9b4356640 1 Processor -- start 2024-09-21T06:06:19.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.011+0000 7ff9b4356640 1 -- start start 2024-09-21T06:06:19.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.011+0000 7ff9b4356640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9ac103b60 0x7ff9ac19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.011+0000 7ff9b4356640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9ac1094a0 0x7ff9ac19a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.011+0000 7ff9b4356640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9ac109df0 0x7ff9ac19fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.011+0000 7ff9b4356640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac113ab0 con 0x7ff9ac109df0 2024-09-21T06:06:19.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.011+0000 7ff9b4356640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac113930 con 0x7ff9ac103b60 2024-09-21T06:06:19.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.011+0000 7ff9b4356640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac113c30 con 0x7ff9ac1094a0 2024-09-21T06:06:19.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.012+0000 7ff9b28f3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9ac109df0 0x7ff9ac19fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.012+0000 7ff9b28f3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9ac109df0 0x7ff9ac19fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45428/0 (socket says 172.21.15.117:45428) 2024-09-21T06:06:19.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.012+0000 7ff9b28f3640 1 -- 172.21.15.117:0/1208419815 learned_addr learned my addr 172.21.15.117:0/1208419815 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:19.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.012+0000 7ff9b18f1640 1 --2- 172.21.15.117:0/1208419815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9ac1094a0 0x7ff9ac19a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.012+0000 7ff9b20f2640 1 --2- 172.21.15.117:0/1208419815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9ac103b60 0x7ff9ac19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.012+0000 7ff9b28f3640 1 -- 172.21.15.117:0/1208419815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9ac1094a0 msgr2=0x7ff9ac19a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:19.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.012+0000 7ff9b28f3640 1 --2- 172.21.15.117:0/1208419815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9ac1094a0 0x7ff9ac19a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.012+0000 7ff9b28f3640 1 -- 172.21.15.117:0/1208419815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9ac103b60 msgr2=0x7ff9ac19a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:19.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.012+0000 7ff9b28f3640 1 --2- 172.21.15.117:0/1208419815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9ac103b60 0x7ff9ac19a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.012+0000 7ff9b28f3640 1 -- 172.21.15.117:0/1208419815 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9ac1a0290 con 0x7ff9ac109df0 2024-09-21T06:06:19.251 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.012+0000 7ff9b28f3640 1 --2- 172.21.15.117:0/1208419815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9ac109df0 0x7ff9ac19fc20 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4005cc0 tx=0x7ff9a402f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:19.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.012+0000 7ff99f7fe640 1 -- 172.21.15.117:0/1208419815 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4003cf0 con 0x7ff9ac109df0 2024-09-21T06:06:19.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.012+0000 7ff99f7fe640 1 -- 172.21.15.117:0/1208419815 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9a40044f0 con 0x7ff9ac109df0 2024-09-21T06:06:19.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.012+0000 7ff9b4356640 1 -- 172.21.15.117:0/1208419815 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9ac1a0520 con 0x7ff9ac109df0 2024-09-21T06:06:19.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.012+0000 7ff99f7fe640 1 -- 172.21.15.117:0/1208419815 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a402fc30 con 0x7ff9ac109df0 2024-09-21T06:06:19.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.012+0000 7ff9b4356640 1 -- 172.21.15.117:0/1208419815 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9ac1a0a60 con 0x7ff9ac109df0 2024-09-21T06:06:19.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.015+0000 7ff9ba8ca740 1 Processor -- start 2024-09-21T06:06:19.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.015+0000 7ff99f7fe640 1 -- 172.21.15.117:0/1208419815 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7ff9a40046a0 con 0x7ff9ac109df0 2024-09-21T06:06:19.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.015+0000 7ff9ba8ca740 1 -- start start 2024-09-21T06:06:19.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.015+0000 7ff9ba8ca740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580619c9070 0x558061d706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.015+0000 7ff9ba8ca740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5580619a01b0 0x558061d70be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.015+0000 7ff9ba8ca740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580619a8390 0x558061d71120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.015+0000 7ff9ba8ca740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558061ce6ea0 con 0x5580619a8390 2024-09-21T06:06:19.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.015+0000 7ff9ba8ca740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558061cd62d0 con 0x5580619c9070 2024-09-21T06:06:19.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.015+0000 7ff9ba8ca740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558061cda0e0 con 0x5580619a01b0 2024-09-21T06:06:19.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.016+0000 7ff9b18f1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580619a8390 0x558061d71120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.016+0000 7ff9b18f1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580619a8390 0x558061d71120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45438/0 (socket says 172.21.15.117:45438) 2024-09-21T06:06:19.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.016+0000 7ff9b18f1640 1 -- 172.21.15.117:0/899490366 learned_addr learned my addr 172.21.15.117:0/899490366 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:19.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.016+0000 7ff9b20f2640 1 --2- 172.21.15.117:0/899490366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5580619a01b0 0x558061d70be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.016+0000 7ff9b18f1640 1 --2- 172.21.15.117:0/899490366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580619c9070 0x558061d706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.016+0000 7ff99f7fe640 1 --2- 172.21.15.117:0/1208419815 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff9980b66b0 0x7ff9980b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.016+0000 7ff9b18f1640 1 -- 172.21.15.117:0/899490366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5580619a01b0 msgr2=0x558061d70be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:19.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.016+0000 7ff9b18f1640 1 --2- 172.21.15.117:0/899490366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5580619a01b0 0x558061d70be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.016+0000 7ff9b18f1640 1 -- 172.21.15.117:0/899490366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580619c9070 msgr2=0x558061d706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:19.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.016+0000 7ff9b18f1640 1 --2- 172.21.15.117:0/899490366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580619c9070 0x558061d706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.016+0000 7ff9b18f1640 1 -- 172.21.15.117:0/899490366 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558061a34650 con 0x5580619a8390 2024-09-21T06:06:19.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.016+0000 7ff9b20f2640 1 --2- 172.21.15.117:0/899490366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5580619a01b0 0x558061d70be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:06:19.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.016+0000 7ff9b28f3640 1 --2- 172.21.15.117:0/1208419815 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff9980b66b0 0x7ff9980b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.016+0000 7ff99f7fe640 1 -- 172.21.15.117:0/1208419815 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7ff9a4108d30 con 0x7ff9ac109df0 2024-09-21T06:06:19.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.016+0000 7ff9b18f1640 1 --2- 172.21.15.117:0/899490366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580619a8390 0x558061d71120 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0011780 tx=0x7ff9a000a700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e751d6b1c125f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:19.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.016+0000 7ff99cd9a640 1 -- 172.21.15.117:0/899490366 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0005020 con 0x5580619a8390 2024-09-21T06:06:19.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.017+0000 7ff99cd9a640 1 -- 172.21.15.117:0/899490366 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9a00047a0 con 0x5580619a8390 2024-09-21T06:06:19.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.017+0000 7ff99cd9a640 1 -- 172.21.15.117:0/899490366 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a000fa20 con 0x5580619a8390 2024-09-21T06:06:19.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.017+0000 7ff9ba8ca740 1 -- 172.21.15.117:0/899490366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580619a8390 msgr2=0x558061d71120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:19.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.017+0000 7ff9ba8ca740 1 --2- 172.21.15.117:0/899490366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580619a8390 0x558061d71120 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0011780 tx=0x7ff9a000a700 comp rx=0 tx=0).stop 2024-09-21T06:06:19.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.017+0000 7ff9b28f3640 1 --2- 172.21.15.117:0/1208419815 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff9980b66b0 0x7ff9980b8bc0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4106580 tx=0x7ff9a409b990 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:19.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.017+0000 7ff9ba8ca740 1 -- 172.21.15.117:0/899490366 shutdown_connections 2024-09-21T06:06:19.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.017+0000 7ff9ba8ca740 1 --2- 172.21.15.117:0/899490366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580619a8390 0x558061d71120 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.017+0000 7ff9ba8ca740 1 --2- 172.21.15.117:0/899490366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5580619a01b0 0x558061d70be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.017+0000 7ff9ba8ca740 1 --2- 172.21.15.117:0/899490366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580619c9070 0x558061d706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.017+0000 7ff9ba8ca740 1 -- 172.21.15.117:0/899490366 >> 172.21.15.117:0/899490366 conn(0x5580619bd460 msgr2=0x558061d6e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:19.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.018+0000 7ff9ba8ca740 1 -- 172.21.15.117:0/899490366 shutdown_connections 2024-09-21T06:06:19.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.018+0000 7ff9ba8ca740 1 -- 172.21.15.117:0/899490366 wait complete. 2024-09-21T06:06:19.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.018+0000 7ff9ba8ca740 1 -- start start 2024-09-21T06:06:19.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.018+0000 7ff9ba8ca740 1 Processor -- start 2024-09-21T06:06:19.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.018+0000 7ff9ba8ca740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5580619a01b0 0x558061d7bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.018+0000 7ff9ba8ca740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580619a8390 0x558061d7c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.018+0000 7ff9ba8ca740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580619c9070 0x558061d88170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.018+0000 7ff9ba8ca740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558061b37700 con 0x5580619a8390 2024-09-21T06:06:19.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.018+0000 7ff9ba8ca740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558061c71160 con 0x5580619c9070 2024-09-21T06:06:19.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.018+0000 7ff9ba8ca740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558061d7ee80 con 0x5580619a01b0 2024-09-21T06:06:19.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.019+0000 7ff9ba8ca740 10 client.0 _ll_register_callbacks cb 0x558061c8a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:19.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.019+0000 7ff9b20f2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580619a8390 0x558061d7c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.019+0000 7ff9b20f2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580619a8390 0x558061d7c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45452/0 (socket says 172.21.15.117:45452) 2024-09-21T06:06:19.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.019+0000 7ff9b20f2640 1 -- 172.21.15.117:0/503120747 learned_addr learned my addr 172.21.15.117:0/503120747 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:19.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.019+0000 7ff9b18f1640 1 --2- 172.21.15.117:0/503120747 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5580619a01b0 0x558061d7bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.019+0000 7ff9b18f1640 1 --2- 172.21.15.117:0/503120747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580619c9070 0x558061d88170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.019+0000 7ff9b20f2640 1 -- 172.21.15.117:0/503120747 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5580619a01b0 msgr2=0x558061d7bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:19.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.019+0000 7ff9b20f2640 1 --2- 172.21.15.117:0/503120747 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5580619a01b0 0x558061d7bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.019+0000 7ff9b20f2640 1 -- 172.21.15.117:0/503120747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580619c9070 msgr2=0x558061d88170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:19.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.019+0000 7ff9b20f2640 1 --2- 172.21.15.117:0/503120747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580619c9070 0x558061d88170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.019+0000 7ff9b20f2640 1 -- 172.21.15.117:0/503120747 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558061c9ec90 con 0x5580619a8390 2024-09-21T06:06:19.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.019+0000 7ff9b20f2640 1 --2- 172.21.15.117:0/503120747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580619a8390 0x558061d7c300 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7ff9a800ce70 tx=0x7ff9a80039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f376a55296e8cd37 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:19.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.019+0000 7ff995ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:19.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.019+0000 7ff995ffb640 1 -- 172.21.15.117:0/503120747 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8011970 con 0x5580619a8390 2024-09-21T06:06:19.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.019+0000 7ff995ffb640 1 -- 172.21.15.117:0/503120747 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7ff9a8011b10 con 0x5580619a8390 2024-09-21T06:06:19.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.019+0000 7ff9ba8ca740 1 -- 172.21.15.117:0/503120747 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9a00047a0 con 0x5580619a8390 2024-09-21T06:06:19.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.020+0000 7ff995ffb640 1 -- 172.21.15.117:0/503120747 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff9a8011d10 con 0x5580619a8390 2024-09-21T06:06:19.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.020+0000 7ff9ba8ca740 10 client.17205 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:19.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.020+0000 7ff9ba8ca740 1 -- 172.21.15.117:0/503120747 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558061d8ac50 con 0x5580619a8390 2024-09-21T06:06:19.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.020+0000 7ff995ffb640 1 -- 172.21.15.117:0/503120747 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9a00047a0 con 0x5580619a8390 2024-09-21T06:06:19.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.020+0000 7ff995ffb640 1 -- 172.21.15.117:0/503120747 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff9a8018070 con 0x5580619a8390 2024-09-21T06:06:19.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.020+0000 7ff9ba8ca740 10 client.17205 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:19.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.020+0000 7ff9ba8ca740 10 client.17205 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T06:06:19.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.020+0000 7ff9ba8ca740 20 client.17205 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:19.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.020+0000 7ff9ba8ca740 1 --2- 172.21.15.117:0/503120747 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x5580619879f0 0x558061d8e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.020+0000 7ff9ba8ca740 4 client.17205 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:06:19.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.020+0000 7ff9ba8ca740 1 -- 172.21.15.117:0/503120747 --> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558061ce8df0 con 0x5580619879f0 2024-09-21T06:06:19.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.020+0000 7ff9b18f1640 1 --2- 172.21.15.117:0/503120747 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x5580619879f0 0x558061d8e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.021+0000 7ff9b18f1640 1 --2- 172.21.15.117:0/503120747 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x5580619879f0 0x558061d8e660 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e686a91e8b7b3cea server_cookie=53d5be119c0d4226 in_seq=0 out_seq=0 2024-09-21T06:06:19.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.021+0000 7ff995ffb640 10 client.17205 ms_handle_connect on v2:172.21.15.138:6836/2540335374 2024-09-21T06:06:19.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.021+0000 7ff995ffb640 1 -- 172.21.15.117:0/503120747 <== mds.4 v2:172.21.15.138:6836/2540335374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558061ce8df0 con 0x5580619879f0 2024-09-21T06:06:19.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.021+0000 7ff995ffb640 10 client.17205 handle_command_reply: tid=0 2024-09-21T06:06:19.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.021+0000 7ff9ba8ca740 1 client.17205 shutdown 2024-09-21T06:06:19.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.022+0000 7ff9ba8ca740 1 -- 172.21.15.117:0/503120747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580619a8390 msgr2=0x558061d7c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:19.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.022+0000 7ff9ba8ca740 1 --2- 172.21.15.117:0/503120747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580619a8390 0x558061d7c300 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7ff9a800ce70 tx=0x7ff9a80039c0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.022+0000 7ff9ba8ca740 1 -- 172.21.15.117:0/503120747 shutdown_connections 2024-09-21T06:06:19.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.022+0000 7ff9ba8ca740 1 --2- 172.21.15.117:0/503120747 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x5580619879f0 0x558061d8e660 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.022+0000 7ff9ba8ca740 1 --2- 172.21.15.117:0/503120747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580619a8390 0x558061d7c300 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.022+0000 7ff9ba8ca740 1 --2- 172.21.15.117:0/503120747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580619c9070 0x558061d88170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.022+0000 7ff9ba8ca740 1 --2- 172.21.15.117:0/503120747 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5580619a01b0 0x558061d7bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.022+0000 7ff995ffb640 0 client.17205 ms_handle_reset on v2:172.21.15.138:6836/2540335374 2024-09-21T06:06:19.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.022+0000 7ff9ba8ca740 1 -- 172.21.15.117:0/503120747 >> 172.21.15.117:0/503120747 conn(0x5580619bd460 msgr2=0x558061d73880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:19.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.022+0000 7ff9ba8ca740 1 -- 172.21.15.117:0/503120747 shutdown_connections 2024-09-21T06:06:19.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.022+0000 7ff9ba8ca740 1 -- 172.21.15.117:0/503120747 wait complete. 2024-09-21T06:06:19.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.022+0000 7ff9ba8ca740 20 client.17205 trim_cache size 0 max 16384 2024-09-21T06:06:19.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.035+0000 7ff9ba8ca740 1 Processor -- start 2024-09-21T06:06:19.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.035+0000 7ff9ba8ca740 1 -- start start 2024-09-21T06:06:19.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.035+0000 7ff9ba8ca740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580619a01b0 0x558061d7c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.035+0000 7ff9ba8ca740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580619a8390 0x558061d88070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.035+0000 7ff9ba8ca740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5580619c9070 0x558061d93c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.035+0000 7ff9ba8ca740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac1af4f0 con 0x5580619a01b0 2024-09-21T06:06:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.035+0000 7ff9ba8ca740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac1b1030 con 0x5580619a8390 2024-09-21T06:06:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.035+0000 7ff9ba8ca740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac1afbc0 con 0x5580619c9070 2024-09-21T06:06:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.035+0000 7ff9b18f1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580619a01b0 0x558061d7c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.035+0000 7ff9b18f1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580619a01b0 0x558061d7c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45454/0 (socket says 172.21.15.117:45454) 2024-09-21T06:06:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.035+0000 7ff9b18f1640 1 -- 172.21.15.117:0/3401372668 learned_addr learned my addr 172.21.15.117:0/3401372668 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.035+0000 7ff9b20f2640 1 --2- 172.21.15.117:0/3401372668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580619a8390 0x558061d88070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.035+0000 7ff9b18f1640 1 -- 172.21.15.117:0/3401372668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5580619c9070 msgr2=0x558061d93c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.035+0000 7ff9b18f1640 1 --2- 172.21.15.117:0/3401372668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5580619c9070 0x558061d93c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.035+0000 7ff9b18f1640 1 -- 172.21.15.117:0/3401372668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580619a8390 msgr2=0x558061d88070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:19.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.036+0000 7ff9b18f1640 1 --2- 172.21.15.117:0/3401372668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580619a8390 0x558061d88070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.036+0000 7ff9b18f1640 1 -- 172.21.15.117:0/3401372668 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558061d7ce60 con 0x5580619a01b0 2024-09-21T06:06:19.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.036+0000 7ff9b20f2640 1 --2- 172.21.15.117:0/3401372668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580619a8390 0x558061d88070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:06:19.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.036+0000 7ff9b18f1640 1 --2- 172.21.15.117:0/3401372668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580619a01b0 0x558061d7c280 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x558061d72f50 tx=0x7ff9a000a700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7410e41431c6c99 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:19.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.036+0000 7ff997fff640 1 -- 172.21.15.117:0/3401372668 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0004780 con 0x5580619a01b0 2024-09-21T06:06:19.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.036+0000 7ff997fff640 1 -- 172.21.15.117:0/3401372668 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9a0005430 con 0x5580619a01b0 2024-09-21T06:06:19.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.036+0000 7ff997fff640 1 -- 172.21.15.117:0/3401372668 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0004a50 con 0x5580619a01b0 2024-09-21T06:06:19.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.037+0000 7ff9ba8ca740 1 -- 172.21.15.117:0/3401372668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580619a01b0 msgr2=0x558061d7c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:19.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.037+0000 7ff9ba8ca740 1 --2- 172.21.15.117:0/3401372668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580619a01b0 0x558061d7c280 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x558061d72f50 tx=0x7ff9a000a700 comp rx=0 tx=0).stop 2024-09-21T06:06:19.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.037+0000 7ff9ba8ca740 1 -- 172.21.15.117:0/3401372668 shutdown_connections 2024-09-21T06:06:19.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.037+0000 7ff9ba8ca740 1 --2- 172.21.15.117:0/3401372668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5580619c9070 0x558061d93c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.037+0000 7ff9ba8ca740 1 --2- 172.21.15.117:0/3401372668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580619a8390 0x558061d88070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.037+0000 7ff9ba8ca740 1 --2- 172.21.15.117:0/3401372668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580619a01b0 0x558061d7c280 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.037+0000 7ff9ba8ca740 1 -- 172.21.15.117:0/3401372668 >> 172.21.15.117:0/3401372668 conn(0x5580619bd460 msgr2=0x558061d7bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:19.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.037+0000 7ff9ba8ca740 1 -- 172.21.15.117:0/3401372668 shutdown_connections 2024-09-21T06:06:19.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.037+0000 7ff9ba8ca740 1 -- 172.21.15.117:0/3401372668 wait complete. 2024-09-21T06:06:19.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.037+0000 7ff9ba8ca740 1 -- start start 2024-09-21T06:06:19.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.037+0000 7ff9ba8ca740 1 Processor -- start 2024-09-21T06:06:19.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.038+0000 7ff9ba8ca740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580619a01b0 0x558061d84830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.038+0000 7ff9ba8ca740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580619a8390 0x558061d7fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.038+0000 7ff9ba8ca740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5580619c9070 0x558061d80220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.038+0000 7ff9ba8ca740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac1b0960 con 0x5580619a8390 2024-09-21T06:06:19.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.038+0000 7ff9ba8ca740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac1b0290 con 0x5580619a01b0 2024-09-21T06:06:19.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.038+0000 7ff9ba8ca740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac1ad4f0 con 0x5580619c9070 2024-09-21T06:06:19.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.038+0000 7ff9ba8ca740 10 client.0 _ll_register_callbacks cb 0x7ff9a8004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:19.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.038+0000 7ff9b20f2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580619a8390 0x558061d7fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.038+0000 7ff9b20f2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580619a8390 0x558061d7fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45464/0 (socket says 172.21.15.117:45464) 2024-09-21T06:06:19.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.038+0000 7ff9b20f2640 1 -- 172.21.15.117:0/2573349289 learned_addr learned my addr 172.21.15.117:0/2573349289 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:19.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.038+0000 7ff9b18f1640 1 --2- 172.21.15.117:0/2573349289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580619a01b0 0x558061d84830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.038+0000 7ff9b18f1640 1 --2- 172.21.15.117:0/2573349289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5580619c9070 0x558061d80220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.038+0000 7ff9b20f2640 1 -- 172.21.15.117:0/2573349289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5580619c9070 msgr2=0x558061d80220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:19.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.038+0000 7ff9b20f2640 1 --2- 172.21.15.117:0/2573349289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5580619c9070 0x558061d80220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.038+0000 7ff9b20f2640 1 -- 172.21.15.117:0/2573349289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580619a01b0 msgr2=0x558061d84830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:19.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.038+0000 7ff9b20f2640 1 --2- 172.21.15.117:0/2573349289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580619a01b0 0x558061d84830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.038+0000 7ff9b20f2640 1 -- 172.21.15.117:0/2573349289 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558061d80960 con 0x5580619a8390 2024-09-21T06:06:19.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.038+0000 7ff9b20f2640 1 --2- 172.21.15.117:0/2573349289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580619a8390 0x558061d7fce0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7ff9a80058c0 tx=0x7ff9a8005d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=588781aaee875dd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:19.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.038+0000 7ff995ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:19.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.038+0000 7ff995ffb640 1 -- 172.21.15.117:0/2573349289 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a802f060 con 0x5580619a8390 2024-09-21T06:06:19.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.038+0000 7ff995ffb640 1 -- 172.21.15.117:0/2573349289 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7ff9a8005030 con 0x5580619a8390 2024-09-21T06:06:19.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.038+0000 7ff9ba8ca740 1 -- 172.21.15.117:0/2573349289 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9a0005430 con 0x5580619a8390 2024-09-21T06:06:19.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.038+0000 7ff995ffb640 1 -- 172.21.15.117:0/2573349289 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff9a8027070 con 0x5580619a8390 2024-09-21T06:06:19.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.039+0000 7ff9ba8ca740 10 client.17211 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:19.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.039+0000 7ff9ba8ca740 1 -- 172.21.15.117:0/2573349289 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558061df8c70 con 0x5580619a8390 2024-09-21T06:06:19.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.039+0000 7ff995ffb640 1 -- 172.21.15.117:0/2573349289 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9a0005430 con 0x5580619a8390 2024-09-21T06:06:19.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.039+0000 7ff995ffb640 1 -- 172.21.15.117:0/2573349289 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff9a8018070 con 0x5580619a8390 2024-09-21T06:06:19.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.039+0000 7ff9ba8ca740 10 client.17211 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:19.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.039+0000 7ff9ba8ca740 10 client.17211 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T06:06:19.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.039+0000 7ff9ba8ca740 20 client.17211 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:19.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.039+0000 7ff9ba8ca740 1 --2- 172.21.15.117:0/2573349289 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x55806195cda0 0x558061dfb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.039+0000 7ff9ba8ca740 4 client.17211 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:06:19.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.039+0000 7ff9ba8ca740 1 -- 172.21.15.117:0/2573349289 --> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558061ce48a0 con 0x55806195cda0 2024-09-21T06:06:19.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.039+0000 7ff9b18f1640 1 --2- 172.21.15.117:0/2573349289 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x55806195cda0 0x558061dfb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.040+0000 7ff9b18f1640 1 --2- 172.21.15.117:0/2573349289 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x55806195cda0 0x558061dfb400 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=99e20c1227875e3d server_cookie=1b2c53e109bcbf90 in_seq=0 out_seq=0 2024-09-21T06:06:19.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.040+0000 7ff995ffb640 10 client.17211 ms_handle_connect on v2:172.21.15.138:6836/2540335374 2024-09-21T06:06:19.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.040+0000 7ff995ffb640 1 -- 172.21.15.117:0/2573349289 <== mds.4 v2:172.21.15.138:6836/2540335374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558061ce48a0 con 0x55806195cda0 2024-09-21T06:06:19.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.040+0000 7ff995ffb640 10 client.17211 handle_command_reply: tid=0 2024-09-21T06:06:19.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.040+0000 7ff9ba8ca740 1 client.17211 shutdown 2024-09-21T06:06:19.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.040+0000 7ff9ba8ca740 1 -- 172.21.15.117:0/2573349289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580619a8390 msgr2=0x558061d7fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:19.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.040+0000 7ff9ba8ca740 1 --2- 172.21.15.117:0/2573349289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580619a8390 0x558061d7fce0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7ff9a80058c0 tx=0x7ff9a8005d70 comp rx=0 tx=0).stop 2024-09-21T06:06:19.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.040+0000 7ff9ba8ca740 1 -- 172.21.15.117:0/2573349289 shutdown_connections 2024-09-21T06:06:19.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.040+0000 7ff9ba8ca740 1 --2- 172.21.15.117:0/2573349289 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x55806195cda0 0x558061dfb400 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.040+0000 7ff9ba8ca740 1 --2- 172.21.15.117:0/2573349289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5580619a8390 0x558061d7fce0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.040+0000 7ff9ba8ca740 1 --2- 172.21.15.117:0/2573349289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5580619c9070 0x558061d80220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.040+0000 7ff995ffb640 0 client.17211 ms_handle_reset on v2:172.21.15.138:6836/2540335374 2024-09-21T06:06:19.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.040+0000 7ff9ba8ca740 1 --2- 172.21.15.117:0/2573349289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5580619a01b0 0x558061d84830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.040+0000 7ff9ba8ca740 1 -- 172.21.15.117:0/2573349289 >> 172.21.15.117:0/2573349289 conn(0x5580619bd460 msgr2=0x558061d7a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:19.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.040+0000 7ff9ba8ca740 1 -- 172.21.15.117:0/2573349289 shutdown_connections 2024-09-21T06:06:19.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.040+0000 7ff9ba8ca740 1 -- 172.21.15.117:0/2573349289 wait complete. 2024-09-21T06:06:19.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.040+0000 7ff9ba8ca740 20 client.17211 trim_cache size 0 max 16384 2024-09-21T06:06:19.268 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:06:19.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.041+0000 7ff9b4356640 1 -- 172.21.15.117:0/1208419815 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff9980b66b0 msgr2=0x7ff9980b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:19.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.041+0000 7ff9b4356640 1 --2- 172.21.15.117:0/1208419815 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff9980b66b0 0x7ff9980b8bc0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4106580 tx=0x7ff9a409b990 comp rx=0 tx=0).stop 2024-09-21T06:06:19.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.041+0000 7ff9b4356640 1 -- 172.21.15.117:0/1208419815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9ac109df0 msgr2=0x7ff9ac19fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:19.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.041+0000 7ff9b4356640 1 --2- 172.21.15.117:0/1208419815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9ac109df0 0x7ff9ac19fc20 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4005cc0 tx=0x7ff9a402f470 comp rx=0 tx=0).stop 2024-09-21T06:06:19.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.041+0000 7ff9b4356640 1 -- 172.21.15.117:0/1208419815 shutdown_connections 2024-09-21T06:06:19.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.041+0000 7ff9b4356640 1 --2- 172.21.15.117:0/1208419815 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff9980b66b0 0x7ff9980b8bc0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.041+0000 7ff9b4356640 1 --2- 172.21.15.117:0/1208419815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9ac109df0 0x7ff9ac19fc20 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.041+0000 7ff9b4356640 1 --2- 172.21.15.117:0/1208419815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9ac1094a0 0x7ff9ac19a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.041+0000 7ff9b4356640 1 --2- 172.21.15.117:0/1208419815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9ac103b60 0x7ff9ac19a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.041+0000 7ff9b4356640 1 -- 172.21.15.117:0/1208419815 >> 172.21.15.117:0/1208419815 conn(0x7ff9ac07b850 msgr2=0x7ff9ac105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:19.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.041+0000 7ff9b4356640 1 -- 172.21.15.117:0/1208419815 shutdown_connections 2024-09-21T06:06:19.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:19.041+0000 7ff9b4356640 1 -- 172.21.15.117:0/1208419815 wait complete. 2024-09-21T06:06:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.675+0000 7f856109b640 1 -- 172.21.15.117:0/4199262902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f855c108c80 msgr2=0x7f855c10f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.675+0000 7f856109b640 1 --2- 172.21.15.117:0/4199262902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f855c108c80 0x7f855c10f4b0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f8554009ca0 tx=0x7f855402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.675+0000 7f856109b640 1 -- 172.21.15.117:0/4199262902 shutdown_connections 2024-09-21T06:06:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.675+0000 7f856109b640 1 --2- 172.21.15.117:0/4199262902 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f855c108c80 0x7f855c10f4b0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.675+0000 7f856109b640 1 --2- 172.21.15.117:0/4199262902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f855c108330 0x7f855c108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.675+0000 7f856109b640 1 --2- 172.21.15.117:0/4199262902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c103170 0x7f855c103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.675+0000 7f856109b640 1 -- 172.21.15.117:0/4199262902 >> 172.21.15.117:0/4199262902 conn(0x7f855c079e90 msgr2=0x7f855c07a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.675+0000 7f856109b640 1 -- 172.21.15.117:0/4199262902 shutdown_connections 2024-09-21T06:06:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.675+0000 7f856109b640 1 -- 172.21.15.117:0/4199262902 wait complete. 2024-09-21T06:06:19.680 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.676+0000 7f856109b640 1 Processor -- start 2024-09-21T06:06:19.680 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.676+0000 7f856109b640 1 -- start start 2024-09-21T06:06:19.680 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.676+0000 7f856109b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f855c103170 0x7f855c19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.680 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.676+0000 7f856109b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f855c108330 0x7f855c19b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.680 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.676+0000 7f856109b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c108c80 0x7f855c19bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.680 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.676+0000 7f856109b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f855c112990 con 0x7f855c108330 2024-09-21T06:06:19.680 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.676+0000 7f856109b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f855c112810 con 0x7f855c108c80 2024-09-21T06:06:19.680 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.676+0000 7f856109b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f855c112b10 con 0x7f855c103170 2024-09-21T06:06:19.680 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.676+0000 7f855259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f855c108330 0x7f855c19b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.680 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.677+0000 7f855259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f855c108330 0x7f855c19b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45490/0 (socket says 172.21.15.117:45490) 2024-09-21T06:06:19.680 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.677+0000 7f855259c640 1 -- 172.21.15.117:0/1531758020 learned_addr learned my addr 172.21.15.117:0/1531758020 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:19.680 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.677+0000 7f855ad9d640 1 --2- 172.21.15.117:0/1531758020 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f855c103170 0x7f855c19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.677+0000 7f855b59e640 1 --2- 172.21.15.117:0/1531758020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c108c80 0x7f855c19bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.677+0000 7f855259c640 1 -- 172.21.15.117:0/1531758020 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f855c103170 msgr2=0x7f855c19b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.677+0000 7f855259c640 1 --2- 172.21.15.117:0/1531758020 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f855c103170 0x7f855c19b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.677+0000 7f855259c640 1 -- 172.21.15.117:0/1531758020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c108c80 msgr2=0x7f855c19bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.677+0000 7f855259c640 1 --2- 172.21.15.117:0/1531758020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c108c80 0x7f855c19bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.677+0000 7f855259c640 1 -- 172.21.15.117:0/1531758020 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f855c1a65d0 con 0x7f855c108330 2024-09-21T06:06:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.677+0000 7f855259c640 1 --2- 172.21.15.117:0/1531758020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f855c108330 0x7f855c19b880 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f854400e9c0 tx=0x7f854400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.677+0000 7f8558d99640 1 -- 172.21.15.117:0/1531758020 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854400ccc0 con 0x7f855c108330 2024-09-21T06:06:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.677+0000 7f856109b640 1 -- 172.21.15.117:0/1531758020 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f855c1a68c0 con 0x7f855c108330 2024-09-21T06:06:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.677+0000 7f8558d99640 1 -- 172.21.15.117:0/1531758020 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85440054d0 con 0x7f855c108330 2024-09-21T06:06:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.677+0000 7f8558d99640 1 -- 172.21.15.117:0/1531758020 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8544010430 con 0x7f855c108330 2024-09-21T06:06:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.677+0000 7f856109b640 1 -- 172.21.15.117:0/1531758020 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f855c1a6dc0 con 0x7f855c108330 2024-09-21T06:06:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.680+0000 7f8568013740 1 Processor -- start 2024-09-21T06:06:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.680+0000 7f8568013740 1 -- start start 2024-09-21T06:06:19.684 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.681+0000 7f8568013740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf64f7e1b0 0x55cf653239b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.684 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.681+0000 7f8568013740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf64f552f0 0x55cf65323ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.684 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.681+0000 7f8568013740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf64f5d4d0 0x55cf65324430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.684 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.681+0000 7f8568013740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cf652986f0 con 0x55cf64f5d4d0 2024-09-21T06:06:19.684 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.681+0000 7f8568013740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cf65287b20 con 0x55cf64f7e1b0 2024-09-21T06:06:19.684 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.681+0000 7f8568013740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55cf6528b930 con 0x55cf64f552f0 2024-09-21T06:06:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.681+0000 7f8558d99640 1 -- 172.21.15.117:0/1531758020 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f85440105d0 con 0x7f855c108330 2024-09-21T06:06:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.681+0000 7f855b59e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf64f7e1b0 0x55cf653239b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.681+0000 7f855b59e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf64f7e1b0 0x55cf653239b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:50970/0 (socket says 172.21.15.117:50970) 2024-09-21T06:06:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.681+0000 7f855b59e640 1 -- 172.21.15.117:0/1007268336 learned_addr learned my addr 172.21.15.117:0/1007268336 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.681+0000 7f855259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf64f5d4d0 0x55cf65324430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.681+0000 7f855ad9d640 1 --2- 172.21.15.117:0/1007268336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf64f552f0 0x55cf65323ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.681+0000 7f855b59e640 1 -- 172.21.15.117:0/1007268336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf64f552f0 msgr2=0x55cf65323ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.681+0000 7f855b59e640 1 --2- 172.21.15.117:0/1007268336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf64f552f0 0x55cf65323ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.681+0000 7f855b59e640 1 -- 172.21.15.117:0/1007268336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf64f5d4d0 msgr2=0x55cf65324430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.681+0000 7f855b59e640 1 --2- 172.21.15.117:0/1007268336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf64f5d4d0 0x55cf65324430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.681+0000 7f855b59e640 1 -- 172.21.15.117:0/1007268336 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf6524d530 con 0x55cf64f7e1b0 2024-09-21T06:06:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.681+0000 7f855259c640 1 --2- 172.21.15.117:0/1007268336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf64f5d4d0 0x55cf65324430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:06:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.682+0000 7f855b59e640 1 --2- 172.21.15.117:0/1007268336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf64f7e1b0 0x55cf653239b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8554035000 tx=0x7f8554009460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=800971e1d805288 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.682+0000 7f855159a640 1 -- 172.21.15.117:0/1007268336 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8554031cf0 con 0x55cf64f7e1b0 2024-09-21T06:06:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.682+0000 7f855159a640 1 -- 172.21.15.117:0/1007268336 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8554040030 con 0x55cf64f7e1b0 2024-09-21T06:06:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.682+0000 7f855159a640 1 -- 172.21.15.117:0/1007268336 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85540042a0 con 0x55cf64f7e1b0 2024-09-21T06:06:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.682+0000 7f8558d99640 1 --2- 172.21.15.117:0/1531758020 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f85480b6680 0x7f85480b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.682+0000 7f855ad9d640 1 --2- 172.21.15.117:0/1531758020 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f85480b6680 0x7f85480b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.682+0000 7f8558d99640 1 -- 172.21.15.117:0/1531758020 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7f85440e7ac0 con 0x7f855c108330 2024-09-21T06:06:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.683+0000 7f8568013740 1 -- 172.21.15.117:0/1007268336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf64f7e1b0 msgr2=0x55cf653239b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.683+0000 7f855ad9d640 1 --2- 172.21.15.117:0/1531758020 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f85480b6680 0x7f85480b8b90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x55cf65322630 tx=0x7f854c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.683+0000 7f8568013740 1 --2- 172.21.15.117:0/1007268336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf64f7e1b0 0x55cf653239b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8554035000 tx=0x7f8554009460 comp rx=0 tx=0).stop 2024-09-21T06:06:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.683+0000 7f8568013740 1 -- 172.21.15.117:0/1007268336 shutdown_connections 2024-09-21T06:06:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.683+0000 7f8568013740 1 --2- 172.21.15.117:0/1007268336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf64f5d4d0 0x55cf65324430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.683+0000 7f8568013740 1 --2- 172.21.15.117:0/1007268336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf64f552f0 0x55cf65323ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.683+0000 7f8568013740 1 --2- 172.21.15.117:0/1007268336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf64f7e1b0 0x55cf653239b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.683+0000 7f8568013740 1 -- 172.21.15.117:0/1007268336 >> 172.21.15.117:0/1007268336 conn(0x55cf64f72300 msgr2=0x55cf653218a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.683+0000 7f8568013740 1 -- 172.21.15.117:0/1007268336 shutdown_connections 2024-09-21T06:06:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.683+0000 7f8568013740 1 -- 172.21.15.117:0/1007268336 wait complete. 2024-09-21T06:06:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.683+0000 7f8568013740 1 -- start start 2024-09-21T06:06:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.683+0000 7f8568013740 1 Processor -- start 2024-09-21T06:06:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.684+0000 7f8568013740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf64f552f0 0x55cf6532f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.684+0000 7f8568013740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf64f5d4d0 0x55cf6532f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.684+0000 7f8568013740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf64f7e1b0 0x55cf6533b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.684+0000 7f8568013740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cf651bfab0 con 0x55cf64f7e1b0 2024-09-21T06:06:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.684+0000 7f8568013740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cf6522d730 con 0x55cf64f552f0 2024-09-21T06:06:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.684+0000 7f8568013740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55cf653320d0 con 0x55cf64f5d4d0 2024-09-21T06:06:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.684+0000 7f8568013740 10 client.0 _ll_register_callbacks cb 0x55cf652816b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.684+0000 7f855b59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf64f7e1b0 0x55cf6533b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.684+0000 7f855b59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf64f7e1b0 0x55cf6533b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45514/0 (socket says 172.21.15.117:45514) 2024-09-21T06:06:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.684+0000 7f855259c640 1 --2- 172.21.15.117:0/105064229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf64f5d4d0 0x55cf6532f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.684+0000 7f855ad9d640 1 --2- 172.21.15.117:0/105064229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf64f552f0 0x55cf6532f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.684+0000 7f855b59e640 1 -- 172.21.15.117:0/105064229 learned_addr learned my addr 172.21.15.117:0/105064229 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.684+0000 7f855b59e640 1 -- 172.21.15.117:0/105064229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf64f5d4d0 msgr2=0x55cf6532f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.684+0000 7f855b59e640 1 --2- 172.21.15.117:0/105064229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf64f5d4d0 0x55cf6532f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.684+0000 7f855b59e640 1 -- 172.21.15.117:0/105064229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf64f552f0 msgr2=0x55cf6532f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.684+0000 7f855b59e640 1 --2- 172.21.15.117:0/105064229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf64f552f0 0x55cf6532f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.684+0000 7f855b59e640 1 -- 172.21.15.117:0/105064229 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf64fe9790 con 0x55cf64f7e1b0 2024-09-21T06:06:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.684+0000 7f855b59e640 1 --2- 172.21.15.117:0/105064229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf64f7e1b0 0x55cf6533b3c0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f8554004af0 tx=0x7f85540095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=239d979670203b30 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.684+0000 7f85427fc640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.684+0000 7f85427fc640 1 -- 172.21.15.117:0/105064229 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8554053060 con 0x55cf64f7e1b0 2024-09-21T06:06:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.684+0000 7f85427fc640 1 -- 172.21.15.117:0/105064229 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7f8554031660 con 0x55cf64f7e1b0 2024-09-21T06:06:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.685+0000 7f8568013740 1 -- 172.21.15.117:0/105064229 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8554040030 con 0x55cf64f7e1b0 2024-09-21T06:06:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.685+0000 7f85427fc640 1 -- 172.21.15.117:0/105064229 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8554031860 con 0x55cf64f7e1b0 2024-09-21T06:06:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.685+0000 7f8568013740 10 client.17223 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.685+0000 7f8568013740 1 -- 172.21.15.117:0/105064229 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cf65062620 con 0x55cf64f7e1b0 2024-09-21T06:06:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.685+0000 7f85427fc640 1 -- 172.21.15.117:0/105064229 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8554040030 con 0x55cf64f7e1b0 2024-09-21T06:06:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.685+0000 7f85427fc640 1 -- 172.21.15.117:0/105064229 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f85540305c0 con 0x55cf64f7e1b0 2024-09-21T06:06:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.685+0000 7f8568013740 10 client.17223 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.685+0000 7f8568013740 10 client.17223 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:06:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.685+0000 7f8568013740 20 client.17223 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.685+0000 7f8568013740 1 --2- 172.21.15.117:0/105064229 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55cf64f3cd80 0x55cf65341010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.685+0000 7f8568013740 4 client.17223 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:06:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.685+0000 7f8568013740 1 -- 172.21.15.117:0/105064229 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf6529a640 con 0x55cf64f3cd80 2024-09-21T06:06:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.685+0000 7f855ad9d640 1 --2- 172.21.15.117:0/105064229 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55cf64f3cd80 0x55cf65341010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.686+0000 7f85427fc640 10 client.17223 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.686+0000 7f855ad9d640 1 --2- 172.21.15.117:0/105064229 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55cf64f3cd80 0x55cf65341010 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dff8f75b248ec92e server_cookie=3911db08a23fa971 in_seq=0 out_seq=0 2024-09-21T06:06:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.686+0000 7f85427fc640 1 -- 172.21.15.117:0/105064229 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cf6529a640 con 0x55cf64f3cd80 2024-09-21T06:06:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.686+0000 7f85427fc640 10 client.17223 handle_command_reply: tid=0 2024-09-21T06:06:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.687+0000 7f8568013740 1 client.17223 shutdown 2024-09-21T06:06:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.687+0000 7f8568013740 1 -- 172.21.15.117:0/105064229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf64f7e1b0 msgr2=0x55cf6533b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.687+0000 7f8568013740 1 --2- 172.21.15.117:0/105064229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf64f7e1b0 0x55cf6533b3c0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f8554004af0 tx=0x7f85540095c0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.687+0000 7f8568013740 1 -- 172.21.15.117:0/105064229 shutdown_connections 2024-09-21T06:06:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.687+0000 7f8568013740 1 --2- 172.21.15.117:0/105064229 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55cf64f3cd80 0x55cf65341010 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.687+0000 7f8568013740 1 --2- 172.21.15.117:0/105064229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf64f7e1b0 0x55cf6533b3c0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.687+0000 7f8568013740 1 --2- 172.21.15.117:0/105064229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf64f5d4d0 0x55cf6532f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.687+0000 7f85427fc640 0 client.17223 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.687+0000 7f8568013740 1 --2- 172.21.15.117:0/105064229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf64f552f0 0x55cf6532f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.687+0000 7f8568013740 1 -- 172.21.15.117:0/105064229 >> 172.21.15.117:0/105064229 conn(0x55cf64f72300 msgr2=0x55cf653249e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.687+0000 7f8568013740 1 -- 172.21.15.117:0/105064229 shutdown_connections 2024-09-21T06:06:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.687+0000 7f8568013740 1 -- 172.21.15.117:0/105064229 wait complete. 2024-09-21T06:06:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.687+0000 7f8568013740 20 client.17223 trim_cache size 0 max 16384 2024-09-21T06:06:19.705 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.702+0000 7f8568013740 1 Processor -- start 2024-09-21T06:06:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.702+0000 7f8568013740 1 -- start start 2024-09-21T06:06:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.703+0000 7f8568013740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf64f552f0 0x55cf653253a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.703+0000 7f8568013740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf64f5d4d0 0x55cf6532f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.703+0000 7f8568013740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf64f7e1b0 0x55cf65330da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.703+0000 7f8568013740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f855c1af690 con 0x55cf64f7e1b0 2024-09-21T06:06:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.703+0000 7f8568013740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f855c1b11d0 con 0x55cf64f552f0 2024-09-21T06:06:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.703+0000 7f8568013740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f855c1afd60 con 0x55cf64f5d4d0 2024-09-21T06:06:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.703+0000 7f855b59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf64f7e1b0 0x55cf65330da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.703+0000 7f855b59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf64f7e1b0 0x55cf65330da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45528/0 (socket says 172.21.15.117:45528) 2024-09-21T06:06:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.703+0000 7f855b59e640 1 -- 172.21.15.117:0/1775305716 learned_addr learned my addr 172.21.15.117:0/1775305716 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.703+0000 7f855259c640 1 --2- 172.21.15.117:0/1775305716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf64f5d4d0 0x55cf6532f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.703+0000 7f855ad9d640 1 --2- 172.21.15.117:0/1775305716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf64f552f0 0x55cf653253a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.703+0000 7f855b59e640 1 -- 172.21.15.117:0/1775305716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf64f5d4d0 msgr2=0x55cf6532f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.703+0000 7f855b59e640 1 --2- 172.21.15.117:0/1775305716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf64f5d4d0 0x55cf6532f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.703+0000 7f855b59e640 1 -- 172.21.15.117:0/1775305716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf64f552f0 msgr2=0x55cf653253a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.703+0000 7f855b59e640 1 --2- 172.21.15.117:0/1775305716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf64f552f0 0x55cf653253a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.703+0000 7f855b59e640 1 -- 172.21.15.117:0/1775305716 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf6531bbd0 con 0x55cf64f7e1b0 2024-09-21T06:06:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.703+0000 7f855ad9d640 1 --2- 172.21.15.117:0/1775305716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf64f552f0 0x55cf653253a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:06:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.703+0000 7f855b59e640 1 --2- 172.21.15.117:0/1775305716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf64f7e1b0 0x55cf65330da0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f85540094e0 tx=0x7f8554030fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df8b655d732d9b51 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.704+0000 7f8550d99640 1 -- 172.21.15.117:0/1775305716 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8554053070 con 0x55cf64f7e1b0 2024-09-21T06:06:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.704+0000 7f8550d99640 1 -- 172.21.15.117:0/1775305716 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8554040040 con 0x55cf64f7e1b0 2024-09-21T06:06:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.704+0000 7f8550d99640 1 -- 172.21.15.117:0/1775305716 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8554030c70 con 0x55cf64f7e1b0 2024-09-21T06:06:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.705+0000 7f8568013740 1 -- 172.21.15.117:0/1775305716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf64f7e1b0 msgr2=0x55cf65330da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.705+0000 7f8568013740 1 --2- 172.21.15.117:0/1775305716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf64f7e1b0 0x55cf65330da0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f85540094e0 tx=0x7f8554030fd0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.705+0000 7f8568013740 1 -- 172.21.15.117:0/1775305716 shutdown_connections 2024-09-21T06:06:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.705+0000 7f8568013740 1 --2- 172.21.15.117:0/1775305716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf64f7e1b0 0x55cf65330da0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.705+0000 7f8568013740 1 --2- 172.21.15.117:0/1775305716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf64f5d4d0 0x55cf6532f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.705+0000 7f8568013740 1 --2- 172.21.15.117:0/1775305716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf64f552f0 0x55cf653253a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.705+0000 7f8568013740 1 -- 172.21.15.117:0/1775305716 >> 172.21.15.117:0/1775305716 conn(0x55cf64f72300 msgr2=0x55cf65330690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.705+0000 7f8568013740 1 -- 172.21.15.117:0/1775305716 shutdown_connections 2024-09-21T06:06:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.705+0000 7f8568013740 1 -- 172.21.15.117:0/1775305716 wait complete. 2024-09-21T06:06:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.705+0000 7f8568013740 1 -- start start 2024-09-21T06:06:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.705+0000 7f8568013740 1 Processor -- start 2024-09-21T06:06:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.705+0000 7f8568013740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf64f552f0 0x55cf65341890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.705+0000 7f8568013740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf64f5d4d0 0x55cf65341e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.705+0000 7f8568013740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf64f7e1b0 0x55cf653a9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.706+0000 7f8568013740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f855c1b0b00 con 0x55cf64f5d4d0 2024-09-21T06:06:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.706+0000 7f8568013740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f855c1b0430 con 0x55cf64f7e1b0 2024-09-21T06:06:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.706+0000 7f8568013740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f855c1ad690 con 0x55cf64f552f0 2024-09-21T06:06:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.706+0000 7f8568013740 10 client.0 _ll_register_callbacks cb 0x7f85440eb4e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.706+0000 7f855259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf64f5d4d0 0x55cf65341e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.706+0000 7f855259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf64f5d4d0 0x55cf65341e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45544/0 (socket says 172.21.15.117:45544) 2024-09-21T06:06:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.706+0000 7f855259c640 1 -- 172.21.15.117:0/2145834871 learned_addr learned my addr 172.21.15.117:0/2145834871 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.706+0000 7f855b59e640 1 --2- 172.21.15.117:0/2145834871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf64f7e1b0 0x55cf653a9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.706+0000 7f855ad9d640 1 --2- 172.21.15.117:0/2145834871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf64f552f0 0x55cf65341890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.706+0000 7f855b59e640 1 -- 172.21.15.117:0/2145834871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf64f552f0 msgr2=0x55cf65341890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.706+0000 7f855b59e640 1 --2- 172.21.15.117:0/2145834871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf64f552f0 0x55cf65341890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.706+0000 7f855b59e640 1 -- 172.21.15.117:0/2145834871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf64f5d4d0 msgr2=0x55cf65341e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.706+0000 7f855b59e640 1 --2- 172.21.15.117:0/2145834871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf64f5d4d0 0x55cf65341e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.706+0000 7f855b59e640 1 -- 172.21.15.117:0/2145834871 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf653ae200 con 0x55cf64f7e1b0 2024-09-21T06:06:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.706+0000 7f855259c640 1 --2- 172.21.15.117:0/2145834871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf64f5d4d0 0x55cf65341e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:06:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.706+0000 7f855b59e640 1 --2- 172.21.15.117:0/2145834871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf64f7e1b0 0x55cf653a9780 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8554000f80 tx=0x7f855402f380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12bd37303df01a65 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.706+0000 7f85427fc640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-21T06:06:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.707+0000 7f85427fc640 1 -- 172.21.15.117:0/2145834871 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8554053070 con 0x55cf64f7e1b0 2024-09-21T06:06:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.707+0000 7f8568013740 1 -- 172.21.15.117:0/2145834871 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8554040040 con 0x55cf64f7e1b0 2024-09-21T06:06:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.707+0000 7f85427fc640 1 -- 172.21.15.117:0/2145834871 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7f855402f610 con 0x55cf64f7e1b0 2024-09-21T06:06:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.707+0000 7f85427fc640 1 -- 172.21.15.117:0/2145834871 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8554064070 con 0x55cf64f7e1b0 2024-09-21T06:06:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.707+0000 7f8568013740 10 client.26279 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.707+0000 7f8568013740 1 -- 172.21.15.117:0/2145834871 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cf653af490 con 0x55cf64f7e1b0 2024-09-21T06:06:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.707+0000 7f85427fc640 1 -- 172.21.15.117:0/2145834871 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8554040040 con 0x55cf64f7e1b0 2024-09-21T06:06:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.707+0000 7f85427fc640 1 -- 172.21.15.117:0/2145834871 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f85540457c0 con 0x55cf64f7e1b0 2024-09-21T06:06:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.707+0000 7f8568013740 10 client.26279 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.707+0000 7f8568013740 10 client.26279 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:06:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.707+0000 7f8568013740 20 client.26279 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.707+0000 7f8568013740 1 --2- 172.21.15.117:0/2145834871 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55cf64f121c0 0x55cf653b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.707+0000 7f8568013740 4 client.26279 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:06:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.707+0000 7f8568013740 1 -- 172.21.15.117:0/2145834871 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cf652960f0 con 0x55cf64f121c0 2024-09-21T06:06:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.708+0000 7f855ad9d640 1 --2- 172.21.15.117:0/2145834871 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55cf64f121c0 0x55cf653b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.708+0000 7f85427fc640 10 client.26279 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.708+0000 7f855ad9d640 1 --2- 172.21.15.117:0/2145834871 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55cf64f121c0 0x55cf653b1a60 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c876e4f8c98cb772 server_cookie=1e92ac58e497bdbd in_seq=0 out_seq=0 2024-09-21T06:06:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.715+0000 7f85427fc640 1 -- 172.21.15.117:0/2145834871 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cf652960f0 con 0x55cf64f121c0 2024-09-21T06:06:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.715+0000 7f85427fc640 10 client.26279 handle_command_reply: tid=0 2024-09-21T06:06:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.715+0000 7f8568013740 1 client.26279 shutdown 2024-09-21T06:06:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.715+0000 7f8568013740 1 -- 172.21.15.117:0/2145834871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf64f7e1b0 msgr2=0x55cf653a9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.715+0000 7f8568013740 1 --2- 172.21.15.117:0/2145834871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf64f7e1b0 0x55cf653a9780 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8554000f80 tx=0x7f855402f380 comp rx=0 tx=0).stop 2024-09-21T06:06:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.715+0000 7f8568013740 1 -- 172.21.15.117:0/2145834871 shutdown_connections 2024-09-21T06:06:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.715+0000 7f8568013740 1 --2- 172.21.15.117:0/2145834871 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55cf64f121c0 0x55cf653b1a60 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.715+0000 7f8568013740 1 --2- 172.21.15.117:0/2145834871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cf64f5d4d0 0x55cf65341e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.715+0000 7f8568013740 1 --2- 172.21.15.117:0/2145834871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf64f7e1b0 0x55cf653a9780 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.715+0000 7f85427fc640 0 client.26279 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.715+0000 7f8568013740 1 --2- 172.21.15.117:0/2145834871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55cf64f552f0 0x55cf65341890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.715+0000 7f8568013740 1 -- 172.21.15.117:0/2145834871 >> 172.21.15.117:0/2145834871 conn(0x55cf64f72300 msgr2=0x55cf65325170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.715+0000 7f8568013740 1 -- 172.21.15.117:0/2145834871 shutdown_connections 2024-09-21T06:06:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.715+0000 7f8568013740 1 -- 172.21.15.117:0/2145834871 wait complete. 2024-09-21T06:06:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.715+0000 7f8568013740 20 client.26279 trim_cache size 0 max 16384 2024-09-21T06:06:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T06:06:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T06:06:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T06:06:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T06:06:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.715+0000 7f856109b640 1 -- 172.21.15.117:0/1531758020 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f85480b6680 msgr2=0x7f85480b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.715+0000 7f856109b640 1 --2- 172.21.15.117:0/1531758020 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f85480b6680 0x7f85480b8b90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x55cf65322630 tx=0x7f854c005f90 comp rx=0 tx=0).stop 2024-09-21T06:06:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.716+0000 7f856109b640 1 -- 172.21.15.117:0/1531758020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f855c108330 msgr2=0x7f855c19b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.716+0000 7f856109b640 1 --2- 172.21.15.117:0/1531758020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f855c108330 0x7f855c19b880 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f854400e9c0 tx=0x7f854400ee80 comp rx=0 tx=0).stop 2024-09-21T06:06:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.716+0000 7f856109b640 1 -- 172.21.15.117:0/1531758020 shutdown_connections 2024-09-21T06:06:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.716+0000 7f856109b640 1 --2- 172.21.15.117:0/1531758020 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f85480b6680 0x7f85480b8b90 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.716+0000 7f856109b640 1 --2- 172.21.15.117:0/1531758020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f855c108330 0x7f855c19b880 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.716+0000 7f856109b640 1 --2- 172.21.15.117:0/1531758020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c108c80 0x7f855c19bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.716+0000 7f856109b640 1 --2- 172.21.15.117:0/1531758020 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f855c103170 0x7f855c19b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.716+0000 7f856109b640 1 -- 172.21.15.117:0/1531758020 >> 172.21.15.117:0/1531758020 conn(0x7f855c079e90 msgr2=0x7f855c106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.716+0000 7f856109b640 1 -- 172.21.15.117:0/1531758020 shutdown_connections 2024-09-21T06:06:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:19.716+0000 7f856109b640 1 -- 172.21.15.117:0/1531758020 wait complete. 2024-09-21T06:06:20.129 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:19 smithi117 ceph-mon[30947]: pgmap v602: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 230 B/s wr, 11 op/s 2024-09-21T06:06:20.156 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T06:06:20.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:19 smithi138 ceph-mon[34391]: pgmap v602: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 230 B/s wr, 11 op/s 2024-09-21T06:06:20.401 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:20.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:19 smithi192 ceph-mon[34700]: pgmap v602: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 230 B/s wr, 11 op/s 2024-09-21T06:06:21.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.720+0000 7fb142112640 1 -- 172.21.15.117:0/1416929409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb13c1083b0 msgr2=0x7fb13c1087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:21.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.720+0000 7fb142112640 1 --2- 172.21.15.117:0/1416929409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb13c1083b0 0x7fb13c1087c0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fb130009900 tx=0x7fb13002a410 comp rx=0 tx=0).stop 2024-09-21T06:06:21.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.720+0000 7fb142112640 1 -- 172.21.15.117:0/1416929409 shutdown_connections 2024-09-21T06:06:21.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.720+0000 7fb142112640 1 --2- 172.21.15.117:0/1416929409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb13c1083b0 0x7fb13c1087c0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:21.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.720+0000 7fb142112640 1 --2- 172.21.15.117:0/1416929409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb13c108d00 0x7fb13c10f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:21.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.720+0000 7fb142112640 1 --2- 172.21.15.117:0/1416929409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb13c102a70 0x7fb13c102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:21.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.720+0000 7fb142112640 1 -- 172.21.15.117:0/1416929409 >> 172.21.15.117:0/1416929409 conn(0x7fb13c079710 msgr2=0x7fb13c079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:21.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.721+0000 7fb142112640 1 -- 172.21.15.117:0/1416929409 shutdown_connections 2024-09-21T06:06:21.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.721+0000 7fb142112640 1 -- 172.21.15.117:0/1416929409 wait complete. 2024-09-21T06:06:21.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.721+0000 7fb142112640 1 Processor -- start 2024-09-21T06:06:21.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.721+0000 7fb142112640 1 -- start start 2024-09-21T06:06:21.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.722+0000 7fb142112640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb13c102a70 0x7fb13c1993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:21.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.722+0000 7fb142112640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb13c1083b0 0x7fb13c199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:21.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.722+0000 7fb142112640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb13c108d00 0x7fb13c19ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:21.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.722+0000 7fb142112640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c1129c0 con 0x7fb13c1083b0 2024-09-21T06:06:21.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.722+0000 7fb142112640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c112840 con 0x7fb13c102a70 2024-09-21T06:06:21.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.722+0000 7fb142112640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c112b40 con 0x7fb13c108d00 2024-09-21T06:06:21.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.722+0000 7fb13affd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb13c1083b0 0x7fb13c199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:21.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.722+0000 7fb13affd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb13c1083b0 0x7fb13c199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45554/0 (socket says 172.21.15.117:45554) 2024-09-21T06:06:21.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.722+0000 7fb13affd640 1 -- 172.21.15.117:0/2370504921 learned_addr learned my addr 172.21.15.117:0/2370504921 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:21.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.722+0000 7fb13b7fe640 1 --2- 172.21.15.117:0/2370504921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb13c102a70 0x7fb13c1993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:21.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.722+0000 7fb13bfff640 1 --2- 172.21.15.117:0/2370504921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb13c108d00 0x7fb13c19ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:21.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.722+0000 7fb13affd640 1 -- 172.21.15.117:0/2370504921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb13c108d00 msgr2=0x7fb13c19ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:21.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.722+0000 7fb13affd640 1 --2- 172.21.15.117:0/2370504921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb13c108d00 0x7fb13c19ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:21.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.722+0000 7fb13affd640 1 -- 172.21.15.117:0/2370504921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb13c102a70 msgr2=0x7fb13c1993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:21.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.722+0000 7fb13affd640 1 --2- 172.21.15.117:0/2370504921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb13c102a70 0x7fb13c1993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:21.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.722+0000 7fb13affd640 1 -- 172.21.15.117:0/2370504921 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb13c19f360 con 0x7fb13c1083b0 2024-09-21T06:06:21.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.722+0000 7fb13affd640 1 --2- 172.21.15.117:0/2370504921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb13c1083b0 0x7fb13c199990 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fb13002a930 tx=0x7fb130004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:21.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.723+0000 7fb138ff9640 1 -- 172.21.15.117:0/2370504921 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13003f030 con 0x7fb13c1083b0 2024-09-21T06:06:21.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.723+0000 7fb142112640 1 -- 172.21.15.117:0/2370504921 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb13c19f5f0 con 0x7fb13c1083b0 2024-09-21T06:06:21.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.723+0000 7fb138ff9640 1 -- 172.21.15.117:0/2370504921 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb130003aa0 con 0x7fb13c1083b0 2024-09-21T06:06:21.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.723+0000 7fb138ff9640 1 -- 172.21.15.117:0/2370504921 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb130035070 con 0x7fb13c1083b0 2024-09-21T06:06:21.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.723+0000 7fb142112640 1 -- 172.21.15.117:0/2370504921 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb13c1a5db0 con 0x7fb13c1083b0 2024-09-21T06:06:21.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.725+0000 7fb148686740 1 Processor -- start 2024-09-21T06:06:21.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.725+0000 7fb148686740 1 -- start start 2024-09-21T06:06:21.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.726+0000 7fb148686740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc32f0d070 0x55bc332c0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:21.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.726+0000 7fb148686740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc32ee41b0 0x55bc332c08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:21.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.726+0000 7fb148686740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc32eec390 0x55bc332bf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:21.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.726+0000 7fb148686740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bc3322aea0 con 0x55bc32eec390 2024-09-21T06:06:21.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.726+0000 7fb148686740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bc3321a2d0 con 0x55bc32f0d070 2024-09-21T06:06:21.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.726+0000 7fb148686740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55bc3321e0e0 con 0x55bc32ee41b0 2024-09-21T06:06:21.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.726+0000 7fb138ff9640 1 -- 172.21.15.117:0/2370504921 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fb130042030 con 0x7fb13c1083b0 2024-09-21T06:06:21.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.726+0000 7fb13affd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc32eec390 0x55bc332bf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:21.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.726+0000 7fb13affd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc32eec390 0x55bc332bf0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45568/0 (socket says 172.21.15.117:45568) 2024-09-21T06:06:21.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.726+0000 7fb13b7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc32ee41b0 0x55bc332c08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:21.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.726+0000 7fb13affd640 1 -- 172.21.15.117:0/1075067478 learned_addr learned my addr 172.21.15.117:0/1075067478 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:21.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.726+0000 7fb13bfff640 1 --2- 172.21.15.117:0/1075067478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc32f0d070 0x55bc332c0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:21.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.726+0000 7fb13b7fe640 1 -- 172.21.15.117:0/1075067478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc32f0d070 msgr2=0x55bc332c0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:21.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.726+0000 7fb13b7fe640 1 --2- 172.21.15.117:0/1075067478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc32f0d070 0x55bc332c0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:21.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.726+0000 7fb13b7fe640 1 -- 172.21.15.117:0/1075067478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc32eec390 msgr2=0x55bc332bf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:21.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.726+0000 7fb13b7fe640 1 --2- 172.21.15.117:0/1075067478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc32eec390 0x55bc332bf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:21.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.726+0000 7fb13b7fe640 1 -- 172.21.15.117:0/1075067478 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc32f78650 con 0x55bc32ee41b0 2024-09-21T06:06:21.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.727+0000 7fb13b7fe640 1 --2- 172.21.15.117:0/1075067478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc32ee41b0 0x55bc332c08e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb12c004630 tx=0x7fb12c00bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8440c2615e32263d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:21.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.727+0000 7fb138ff9640 1 --2- 172.21.15.117:0/2370504921 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb1240b6700 0x7fb1240b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:21.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.727+0000 7fb121ffb640 1 -- 172.21.15.117:0/1075067478 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12c003bd0 con 0x55bc32ee41b0 2024-09-21T06:06:21.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.727+0000 7fb13bfff640 1 --2- 172.21.15.117:0/2370504921 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb1240b6700 0x7fb1240b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:21.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.727+0000 7fb121ffb640 1 -- 172.21.15.117:0/1075067478 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb12c015070 con 0x55bc32ee41b0 2024-09-21T06:06:21.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.727+0000 7fb138ff9640 1 -- 172.21.15.117:0/2370504921 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fb130107da0 con 0x7fb13c1083b0 2024-09-21T06:06:21.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.727+0000 7fb121ffb640 1 -- 172.21.15.117:0/1075067478 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12c004cd0 con 0x55bc32ee41b0 2024-09-21T06:06:21.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.728+0000 7fb148686740 1 -- 172.21.15.117:0/1075067478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc32ee41b0 msgr2=0x55bc332c08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:21.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.728+0000 7fb13bfff640 1 --2- 172.21.15.117:0/2370504921 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb1240b6700 0x7fb1240b8c10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb13400af10 tx=0x7fb13400a4d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:21.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.728+0000 7fb148686740 1 --2- 172.21.15.117:0/1075067478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc32ee41b0 0x55bc332c08e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb12c004630 tx=0x7fb12c00bb20 comp rx=0 tx=0).stop 2024-09-21T06:06:21.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.728+0000 7fb148686740 1 -- 172.21.15.117:0/1075067478 shutdown_connections 2024-09-21T06:06:21.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.728+0000 7fb148686740 1 --2- 172.21.15.117:0/1075067478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc32eec390 0x55bc332bf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:21.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.728+0000 7fb148686740 1 --2- 172.21.15.117:0/1075067478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc32ee41b0 0x55bc332c08e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:21.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.728+0000 7fb148686740 1 --2- 172.21.15.117:0/1075067478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc32f0d070 0x55bc332c0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:21.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.728+0000 7fb148686740 1 -- 172.21.15.117:0/1075067478 >> 172.21.15.117:0/1075067478 conn(0x55bc32f01460 msgr2=0x55bc332b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:21.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.728+0000 7fb148686740 1 -- 172.21.15.117:0/1075067478 shutdown_connections 2024-09-21T06:06:21.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.728+0000 7fb148686740 1 -- 172.21.15.117:0/1075067478 wait complete. 2024-09-21T06:06:21.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.729+0000 7fb148686740 1 -- start start 2024-09-21T06:06:21.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.729+0000 7fb148686740 1 Processor -- start 2024-09-21T06:06:21.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.729+0000 7fb148686740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc32ee41b0 0x55bc332c45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:21.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.729+0000 7fb148686740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc32eec390 0x55bc332c4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:21.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.729+0000 7fb148686740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc32f0d070 0x55bc332c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:21.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.729+0000 7fb148686740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bc3307b700 con 0x55bc32f0d070 2024-09-21T06:06:21.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.729+0000 7fb148686740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bc331b5160 con 0x55bc32eec390 2024-09-21T06:06:21.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.729+0000 7fb148686740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55bc332bfb70 con 0x55bc32ee41b0 2024-09-21T06:06:21.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.729+0000 7fb148686740 10 client.0 _ll_register_callbacks cb 0x55bc331ce9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:21.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.729+0000 7fb13bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc32f0d070 0x55bc332c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:21.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.729+0000 7fb13bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc32f0d070 0x55bc332c9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45582/0 (socket says 172.21.15.117:45582) 2024-09-21T06:06:21.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.729+0000 7fb13bfff640 1 -- 172.21.15.117:0/3623720384 learned_addr learned my addr 172.21.15.117:0/3623720384 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:21.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.729+0000 7fb13affd640 1 --2- 172.21.15.117:0/3623720384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc32eec390 0x55bc332c4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:21.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.730+0000 7fb13b7fe640 1 --2- 172.21.15.117:0/3623720384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc32ee41b0 0x55bc332c45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:21.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.730+0000 7fb13bfff640 1 -- 172.21.15.117:0/3623720384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc32ee41b0 msgr2=0x55bc332c45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:21.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.730+0000 7fb13bfff640 1 --2- 172.21.15.117:0/3623720384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc32ee41b0 0x55bc332c45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:21.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.730+0000 7fb13bfff640 1 -- 172.21.15.117:0/3623720384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc32eec390 msgr2=0x55bc332c4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:21.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.730+0000 7fb13bfff640 1 --2- 172.21.15.117:0/3623720384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc32eec390 0x55bc332c4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:21.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.730+0000 7fb13bfff640 1 -- 172.21.15.117:0/3623720384 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc331e2c90 con 0x55bc32f0d070 2024-09-21T06:06:21.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.730+0000 7fb13affd640 1 --2- 172.21.15.117:0/3623720384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc32eec390 0x55bc332c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:06:21.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.730+0000 7fb13bfff640 1 --2- 172.21.15.117:0/3623720384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc32f0d070 0x55bc332c9be0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fb13401cbe0 tx=0x7fb134004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751e475459429c4e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:21.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.730+0000 7fb11affd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:21.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.730+0000 7fb11affd640 1 -- 172.21.15.117:0/3623720384 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13401cd90 con 0x55bc32f0d070 2024-09-21T06:06:21.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.730+0000 7fb11affd640 1 -- 172.21.15.117:0/3623720384 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fb134016030 con 0x55bc32f0d070 2024-09-21T06:06:21.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.730+0000 7fb148686740 1 -- 172.21.15.117:0/3623720384 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb12c015070 con 0x55bc32f0d070 2024-09-21T06:06:21.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.730+0000 7fb11affd640 1 -- 172.21.15.117:0/3623720384 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb13400f8a0 con 0x55bc32f0d070 2024-09-21T06:06:21.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.730+0000 7fb148686740 10 client.17241 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:21.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.730+0000 7fb148686740 1 -- 172.21.15.117:0/3623720384 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bc332d2b90 con 0x55bc32f0d070 2024-09-21T06:06:21.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.731+0000 7fb11affd640 1 -- 172.21.15.117:0/3623720384 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb12c015070 con 0x55bc32f0d070 2024-09-21T06:06:21.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.731+0000 7fb11affd640 1 -- 172.21.15.117:0/3623720384 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb134027bb0 con 0x55bc32f0d070 2024-09-21T06:06:21.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.731+0000 7fb148686740 10 client.17241 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:21.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.731+0000 7fb148686740 10 client.17241 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T06:06:21.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.731+0000 7fb148686740 20 client.17241 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:21.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.731+0000 7fb148686740 1 --2- 172.21.15.117:0/3623720384 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55bc32ecb9f0 0x55bc332d6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:21.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.731+0000 7fb148686740 4 client.17241 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:06:21.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.731+0000 7fb148686740 1 -- 172.21.15.117:0/3623720384 --> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc3322cdf0 con 0x55bc32ecb9f0 2024-09-21T06:06:21.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.731+0000 7fb13b7fe640 1 --2- 172.21.15.117:0/3623720384 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55bc32ecb9f0 0x55bc332d6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:21.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.732+0000 7fb13b7fe640 1 --2- 172.21.15.117:0/3623720384 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55bc32ecb9f0 0x55bc332d6170 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a171b31b59874998 server_cookie=10b22ed0017a23b in_seq=0 out_seq=0 2024-09-21T06:06:21.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.732+0000 7fb11affd640 10 client.17241 ms_handle_connect on v2:172.21.15.192:6834/3530688860 2024-09-21T06:06:21.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.732+0000 7fb11affd640 1 -- 172.21.15.117:0/3623720384 <== mds.3 v2:172.21.15.192:6834/3530688860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bc3322cdf0 con 0x55bc32ecb9f0 2024-09-21T06:06:21.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.732+0000 7fb11affd640 10 client.17241 handle_command_reply: tid=0 2024-09-21T06:06:21.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.732+0000 7fb148686740 1 client.17241 shutdown 2024-09-21T06:06:21.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.733+0000 7fb148686740 1 -- 172.21.15.117:0/3623720384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc32f0d070 msgr2=0x55bc332c9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:21.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.733+0000 7fb148686740 1 --2- 172.21.15.117:0/3623720384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc32f0d070 0x55bc332c9be0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fb13401cbe0 tx=0x7fb134004ab0 comp rx=0 tx=0).stop 2024-09-21T06:06:21.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.733+0000 7fb148686740 1 -- 172.21.15.117:0/3623720384 shutdown_connections 2024-09-21T06:06:21.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.733+0000 7fb148686740 1 --2- 172.21.15.117:0/3623720384 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55bc32ecb9f0 0x55bc332d6170 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:21.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.733+0000 7fb11affd640 0 client.17241 ms_handle_reset on v2:172.21.15.192:6834/3530688860 2024-09-21T06:06:21.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.733+0000 7fb148686740 1 --2- 172.21.15.117:0/3623720384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc32f0d070 0x55bc332c9be0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:21.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.733+0000 7fb148686740 1 --2- 172.21.15.117:0/3623720384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc32eec390 0x55bc332c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:21.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.733+0000 7fb148686740 1 --2- 172.21.15.117:0/3623720384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc32ee41b0 0x55bc332c45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:21.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.733+0000 7fb148686740 1 -- 172.21.15.117:0/3623720384 >> 172.21.15.117:0/3623720384 conn(0x55bc32f01460 msgr2=0x55bc332b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:21.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.733+0000 7fb148686740 1 -- 172.21.15.117:0/3623720384 shutdown_connections 2024-09-21T06:06:21.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.733+0000 7fb148686740 1 -- 172.21.15.117:0/3623720384 wait complete. 2024-09-21T06:06:21.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.733+0000 7fb148686740 20 client.17241 trim_cache size 0 max 16384 2024-09-21T06:06:21.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.746+0000 7fb148686740 1 Processor -- start 2024-09-21T06:06:21.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.746+0000 7fb148686740 1 -- start start 2024-09-21T06:06:21.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.746+0000 7fb148686740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc32ecb9f0 0x55bc332d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:21.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.746+0000 7fb148686740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc32ee41b0 0x55bc332bdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:21.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.746+0000 7fb148686740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc32eec390 0x55bc332bdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:21.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.746+0000 7fb148686740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c1ad3b0 con 0x55bc32ecb9f0 2024-09-21T06:06:21.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.746+0000 7fb148686740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c1aeef0 con 0x55bc32eec390 2024-09-21T06:06:21.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.746+0000 7fb148686740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c1ada80 con 0x55bc32ee41b0 2024-09-21T06:06:21.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.746+0000 7fb13b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc32ecb9f0 0x55bc332d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:21.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.746+0000 7fb13b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc32ecb9f0 0x55bc332d09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45588/0 (socket says 172.21.15.117:45588) 2024-09-21T06:06:21.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.746+0000 7fb13b7fe640 1 -- 172.21.15.117:0/3306144072 learned_addr learned my addr 172.21.15.117:0/3306144072 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:21.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.746+0000 7fb13bfff640 1 --2- 172.21.15.117:0/3306144072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc32eec390 0x55bc332bdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:21.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.746+0000 7fb13b7fe640 1 -- 172.21.15.117:0/3306144072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc32ee41b0 msgr2=0x55bc332bdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:21.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.746+0000 7fb13affd640 1 --2- 172.21.15.117:0/3306144072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc32ee41b0 0x55bc332bdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:21.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.746+0000 7fb13b7fe640 1 --2- 172.21.15.117:0/3306144072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc32ee41b0 0x55bc332bdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:21.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.746+0000 7fb13b7fe640 1 -- 172.21.15.117:0/3306144072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc32eec390 msgr2=0x55bc332bdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:21.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.746+0000 7fb13b7fe640 1 --2- 172.21.15.117:0/3306144072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc32eec390 0x55bc332bdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:21.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.746+0000 7fb13b7fe640 1 -- 172.21.15.117:0/3306144072 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc332b3a60 con 0x55bc32ecb9f0 2024-09-21T06:06:21.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.747+0000 7fb13b7fe640 1 --2- 172.21.15.117:0/3306144072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc32ecb9f0 0x55bc332d09f0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fb12c0039c0 tx=0x7fb12c00bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7016207bb90759 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:21.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.747+0000 7fb1217fa640 1 -- 172.21.15.117:0/3306144072 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12c032070 con 0x55bc32ecb9f0 2024-09-21T06:06:21.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.747+0000 7fb1217fa640 1 -- 172.21.15.117:0/3306144072 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb12c015030 con 0x55bc32ecb9f0 2024-09-21T06:06:21.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.747+0000 7fb1217fa640 1 -- 172.21.15.117:0/3306144072 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12c011ab0 con 0x55bc32ecb9f0 2024-09-21T06:06:21.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.748+0000 7fb148686740 1 -- 172.21.15.117:0/3306144072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc32ecb9f0 msgr2=0x55bc332d09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:21.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.748+0000 7fb148686740 1 --2- 172.21.15.117:0/3306144072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc32ecb9f0 0x55bc332d09f0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fb12c0039c0 tx=0x7fb12c00bfb0 comp rx=0 tx=0).stop 2024-09-21T06:06:21.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.748+0000 7fb148686740 1 -- 172.21.15.117:0/3306144072 shutdown_connections 2024-09-21T06:06:21.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.748+0000 7fb148686740 1 --2- 172.21.15.117:0/3306144072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc32eec390 0x55bc332bdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:21.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.748+0000 7fb148686740 1 --2- 172.21.15.117:0/3306144072 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc32ee41b0 0x55bc332bdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:21.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.748+0000 7fb148686740 1 --2- 172.21.15.117:0/3306144072 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc32ecb9f0 0x55bc332d09f0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:21.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.748+0000 7fb148686740 1 -- 172.21.15.117:0/3306144072 >> 172.21.15.117:0/3306144072 conn(0x55bc32f01460 msgr2=0x55bc332c0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:21.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.748+0000 7fb148686740 1 -- 172.21.15.117:0/3306144072 shutdown_connections 2024-09-21T06:06:21.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.748+0000 7fb148686740 1 -- 172.21.15.117:0/3306144072 wait complete. 2024-09-21T06:06:21.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.748+0000 7fb148686740 1 -- start start 2024-09-21T06:06:21.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.748+0000 7fb148686740 1 Processor -- start 2024-09-21T06:06:21.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.749+0000 7fb148686740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc32ecb9f0 0x55bc332cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:21.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.749+0000 7fb148686740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc32ee41b0 0x55bc332ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:21.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.749+0000 7fb148686740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc32eec390 0x55bc332ce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:21.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.749+0000 7fb148686740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c1ae820 con 0x55bc32ecb9f0 2024-09-21T06:06:21.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.749+0000 7fb148686740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c1ae150 con 0x55bc32eec390 2024-09-21T06:06:21.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.749+0000 7fb148686740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb13c1ab3b0 con 0x55bc32ee41b0 2024-09-21T06:06:21.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.749+0000 7fb148686740 10 client.0 _ll_register_callbacks cb 0x7fb13401c890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:21.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.749+0000 7fb13b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc32ecb9f0 0x55bc332cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:21.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.749+0000 7fb13b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc32ecb9f0 0x55bc332cdf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45600/0 (socket says 172.21.15.117:45600) 2024-09-21T06:06:21.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.749+0000 7fb13b7fe640 1 -- 172.21.15.117:0/3130004406 learned_addr learned my addr 172.21.15.117:0/3130004406 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:21.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.749+0000 7fb13bfff640 1 --2- 172.21.15.117:0/3130004406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc32eec390 0x55bc332ce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:21.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.749+0000 7fb13affd640 1 --2- 172.21.15.117:0/3130004406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc32ee41b0 0x55bc332ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:21.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.749+0000 7fb13b7fe640 1 -- 172.21.15.117:0/3130004406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc32ee41b0 msgr2=0x55bc332ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:21.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.749+0000 7fb13b7fe640 1 --2- 172.21.15.117:0/3130004406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc32ee41b0 0x55bc332ce460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:21.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.749+0000 7fb13b7fe640 1 -- 172.21.15.117:0/3130004406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc32eec390 msgr2=0x55bc332ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:21.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.749+0000 7fb13b7fe640 1 --2- 172.21.15.117:0/3130004406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc32eec390 0x55bc332ce9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:21.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.749+0000 7fb13b7fe640 1 -- 172.21.15.117:0/3130004406 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc3333fb40 con 0x55bc32ecb9f0 2024-09-21T06:06:21.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.749+0000 7fb13affd640 1 --2- 172.21.15.117:0/3130004406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc32ee41b0 0x55bc332ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:06:21.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.749+0000 7fb13b7fe640 1 --2- 172.21.15.117:0/3130004406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc32ecb9f0 0x55bc332cdf20 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fb12c030040 tx=0x7fb12c00b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac9edf3b3251f43 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:21.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.749+0000 7fb11affd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:21.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.749+0000 7fb11affd640 1 -- 172.21.15.117:0/3130004406 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12c032040 con 0x55bc32ecb9f0 2024-09-21T06:06:21.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.749+0000 7fb11affd640 1 -- 172.21.15.117:0/3130004406 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fb12c003a70 con 0x55bc32ecb9f0 2024-09-21T06:06:21.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.749+0000 7fb148686740 1 -- 172.21.15.117:0/3130004406 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb12c015030 con 0x55bc32ecb9f0 2024-09-21T06:06:21.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.750+0000 7fb11affd640 1 -- 172.21.15.117:0/3130004406 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb12c003c70 con 0x55bc32ecb9f0 2024-09-21T06:06:21.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.750+0000 7fb148686740 10 client.17247 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:21.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.750+0000 7fb148686740 1 -- 172.21.15.117:0/3130004406 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bc33340ce0 con 0x55bc32ecb9f0 2024-09-21T06:06:21.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.750+0000 7fb11affd640 1 -- 172.21.15.117:0/3130004406 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb12c015030 con 0x55bc32ecb9f0 2024-09-21T06:06:21.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.750+0000 7fb11affd640 1 -- 172.21.15.117:0/3130004406 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb12c02c030 con 0x55bc32ecb9f0 2024-09-21T06:06:21.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.750+0000 7fb148686740 10 client.17247 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:21.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.750+0000 7fb148686740 10 client.17247 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T06:06:21.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.750+0000 7fb148686740 20 client.17247 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:21.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.750+0000 7fb148686740 1 --2- 172.21.15.117:0/3130004406 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55bc32f0d070 0x55bc333431f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:21.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.750+0000 7fb148686740 4 client.17247 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:06:21.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.750+0000 7fb148686740 1 -- 172.21.15.117:0/3130004406 --> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bc332288a0 con 0x55bc32f0d070 2024-09-21T06:06:21.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.750+0000 7fb13affd640 1 --2- 172.21.15.117:0/3130004406 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55bc32f0d070 0x55bc333431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:21.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.751+0000 7fb11affd640 10 client.17247 ms_handle_connect on v2:172.21.15.192:6834/3530688860 2024-09-21T06:06:21.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.751+0000 7fb13affd640 1 --2- 172.21.15.117:0/3130004406 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55bc32f0d070 0x55bc333431f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f430d2e3920b2ce0 server_cookie=267886d74bd7a9e8 in_seq=0 out_seq=0 2024-09-21T06:06:21.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.751+0000 7fb11affd640 1 -- 172.21.15.117:0/3130004406 <== mds.3 v2:172.21.15.192:6834/3530688860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bc332288a0 con 0x55bc32f0d070 2024-09-21T06:06:21.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.751+0000 7fb11affd640 10 client.17247 handle_command_reply: tid=0 2024-09-21T06:06:21.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.751+0000 7fb148686740 1 client.17247 shutdown 2024-09-21T06:06:21.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.751+0000 7fb148686740 1 -- 172.21.15.117:0/3130004406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc32ecb9f0 msgr2=0x55bc332cdf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:21.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.751+0000 7fb148686740 1 --2- 172.21.15.117:0/3130004406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc32ecb9f0 0x55bc332cdf20 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fb12c030040 tx=0x7fb12c00b460 comp rx=0 tx=0).stop 2024-09-21T06:06:21.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.751+0000 7fb148686740 1 -- 172.21.15.117:0/3130004406 shutdown_connections 2024-09-21T06:06:21.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.751+0000 7fb148686740 1 --2- 172.21.15.117:0/3130004406 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55bc32f0d070 0x55bc333431f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:21.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.751+0000 7fb148686740 1 --2- 172.21.15.117:0/3130004406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc32eec390 0x55bc332ce9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:21.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.751+0000 7fb148686740 1 --2- 172.21.15.117:0/3130004406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bc32ee41b0 0x55bc332ce460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:21.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.751+0000 7fb148686740 1 --2- 172.21.15.117:0/3130004406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bc32ecb9f0 0x55bc332cdf20 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:21.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.751+0000 7fb148686740 1 -- 172.21.15.117:0/3130004406 >> 172.21.15.117:0/3130004406 conn(0x55bc32f01460 msgr2=0x55bc332bdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:21.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.751+0000 7fb11affd640 0 client.17247 ms_handle_reset on v2:172.21.15.192:6834/3530688860 2024-09-21T06:06:21.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.751+0000 7fb148686740 1 -- 172.21.15.117:0/3130004406 shutdown_connections 2024-09-21T06:06:21.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.751+0000 7fb148686740 1 -- 172.21.15.117:0/3130004406 wait complete. 2024-09-21T06:06:21.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.751+0000 7fb148686740 20 client.17247 trim_cache size 0 max 16384 2024-09-21T06:06:21.758 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:06:21.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.752+0000 7fb142112640 1 -- 172.21.15.117:0/2370504921 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb1240b6700 msgr2=0x7fb1240b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:21.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.752+0000 7fb142112640 1 --2- 172.21.15.117:0/2370504921 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb1240b6700 0x7fb1240b8c10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb13400af10 tx=0x7fb13400a4d0 comp rx=0 tx=0).stop 2024-09-21T06:06:21.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.752+0000 7fb142112640 1 -- 172.21.15.117:0/2370504921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb13c1083b0 msgr2=0x7fb13c199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:21.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.752+0000 7fb142112640 1 --2- 172.21.15.117:0/2370504921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb13c1083b0 0x7fb13c199990 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fb13002a930 tx=0x7fb130004270 comp rx=0 tx=0).stop 2024-09-21T06:06:21.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.752+0000 7fb142112640 1 -- 172.21.15.117:0/2370504921 shutdown_connections 2024-09-21T06:06:21.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.752+0000 7fb142112640 1 --2- 172.21.15.117:0/2370504921 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb1240b6700 0x7fb1240b8c10 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:21.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.752+0000 7fb142112640 1 --2- 172.21.15.117:0/2370504921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb13c1083b0 0x7fb13c199990 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:21.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.752+0000 7fb142112640 1 --2- 172.21.15.117:0/2370504921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb13c108d00 0x7fb13c19ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:21.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.752+0000 7fb142112640 1 --2- 172.21.15.117:0/2370504921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb13c102a70 0x7fb13c1993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:21.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.752+0000 7fb142112640 1 -- 172.21.15.117:0/2370504921 >> 172.21.15.117:0/2370504921 conn(0x7fb13c079710 msgr2=0x7fb13c104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:21.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.752+0000 7fb142112640 1 -- 172.21.15.117:0/2370504921 shutdown_connections 2024-09-21T06:06:21.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:21.752+0000 7fb142112640 1 -- 172.21.15.117:0/2370504921 wait complete. 2024-09-21T06:06:22.280 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:21 smithi117 ceph-mon[30947]: pgmap v603: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T06:06:22.291 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:21 smithi192 ceph-mon[34700]: pgmap v603: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T06:06:22.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:21 smithi138 ceph-mon[34391]: pgmap v603: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T06:06:22.571 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T06:06:22.820 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:23.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.333+0000 7fb6344c1640 1 -- 172.21.15.117:0/1708598363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb62c1094d0 msgr2=0x7fb62c1098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:23.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.333+0000 7fb6344c1640 1 --2- 172.21.15.117:0/1708598363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb62c1094d0 0x7fb62c1098e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fb61c009900 tx=0x7fb61c02a410 comp rx=0 tx=0).stop 2024-09-21T06:06:23.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.334+0000 7fb6344c1640 1 -- 172.21.15.117:0/1708598363 shutdown_connections 2024-09-21T06:06:23.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.334+0000 7fb6344c1640 1 --2- 172.21.15.117:0/1708598363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb62c1094d0 0x7fb62c1098e0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:23.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.334+0000 7fb6344c1640 1 --2- 172.21.15.117:0/1708598363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb62c109e20 0x7fb62c110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:23.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.334+0000 7fb6344c1640 1 --2- 172.21.15.117:0/1708598363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb62c103b90 0x7fb62c103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:23.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.334+0000 7fb6344c1640 1 -- 172.21.15.117:0/1708598363 >> 172.21.15.117:0/1708598363 conn(0x7fb62c0ff600 msgr2=0x7fb62c101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:23.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.334+0000 7fb6344c1640 1 -- 172.21.15.117:0/1708598363 shutdown_connections 2024-09-21T06:06:23.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.334+0000 7fb6344c1640 1 -- 172.21.15.117:0/1708598363 wait complete. 2024-09-21T06:06:23.338 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.334+0000 7fb6344c1640 1 Processor -- start 2024-09-21T06:06:23.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.334+0000 7fb6344c1640 1 -- start start 2024-09-21T06:06:23.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.335+0000 7fb6344c1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb62c103b90 0x7fb62c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:23.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.335+0000 7fb6344c1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb62c1094d0 0x7fb62c19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:23.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.335+0000 7fb6344c1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb62c109e20 0x7fb62c19fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:23.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.335+0000 7fb6344c1640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c113ae0 con 0x7fb62c1094d0 2024-09-21T06:06:23.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.335+0000 7fb6344c1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c113960 con 0x7fb62c109e20 2024-09-21T06:06:23.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.335+0000 7fb6344c1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c113c60 con 0x7fb62c103b90 2024-09-21T06:06:23.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.335+0000 7fb631a5c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb62c1094d0 0x7fb62c19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:23.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.335+0000 7fb631a5c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb62c1094d0 0x7fb62c19aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45610/0 (socket says 172.21.15.117:45610) 2024-09-21T06:06:23.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.335+0000 7fb631a5c640 1 -- 172.21.15.117:0/2668122822 learned_addr learned my addr 172.21.15.117:0/2668122822 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:23.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.335+0000 7fb63225d640 1 --2- 172.21.15.117:0/2668122822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb62c103b90 0x7fb62c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:23.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.335+0000 7fb632a5e640 1 --2- 172.21.15.117:0/2668122822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb62c109e20 0x7fb62c19fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:23.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.336+0000 7fb631a5c640 1 -- 172.21.15.117:0/2668122822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb62c103b90 msgr2=0x7fb62c19a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:23.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.336+0000 7fb631a5c640 1 --2- 172.21.15.117:0/2668122822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb62c103b90 0x7fb62c19a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:23.339 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.336+0000 7fb631a5c640 1 -- 172.21.15.117:0/2668122822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb62c109e20 msgr2=0x7fb62c19fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:23.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.336+0000 7fb631a5c640 1 --2- 172.21.15.117:0/2668122822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb62c109e20 0x7fb62c19fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:23.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.336+0000 7fb631a5c640 1 -- 172.21.15.117:0/2668122822 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb62c1a0420 con 0x7fb62c1094d0 2024-09-21T06:06:23.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.336+0000 7fb631a5c640 1 --2- 172.21.15.117:0/2668122822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb62c1094d0 0x7fb62c19aa50 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fb61c02a930 tx=0x7fb61c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:23.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.336+0000 7fb6237fe640 1 -- 172.21.15.117:0/2668122822 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61c03f030 con 0x7fb62c1094d0 2024-09-21T06:06:23.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.336+0000 7fb6344c1640 1 -- 172.21.15.117:0/2668122822 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb62c1a06b0 con 0x7fb62c1094d0 2024-09-21T06:06:23.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.336+0000 7fb6237fe640 1 -- 172.21.15.117:0/2668122822 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb61c003aa0 con 0x7fb62c1094d0 2024-09-21T06:06:23.340 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.336+0000 7fb6237fe640 1 -- 172.21.15.117:0/2668122822 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61c035070 con 0x7fb62c1094d0 2024-09-21T06:06:23.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.336+0000 7fb6344c1640 1 -- 172.21.15.117:0/2668122822 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb62c1a6e70 con 0x7fb62c1094d0 2024-09-21T06:06:23.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.339+0000 7fb63aa35740 1 Processor -- start 2024-09-21T06:06:23.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.339+0000 7fb6237fe640 1 -- 172.21.15.117:0/2668122822 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fb61c042030 con 0x7fb62c1094d0 2024-09-21T06:06:23.342 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.339+0000 7fb63aa35740 1 -- start start 2024-09-21T06:06:23.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.339+0000 7fb63aa35740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0e18a2070 0x55f0e1c492c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:23.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.339+0000 7fb63aa35740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0e18791b0 0x55f0e1c49800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:23.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.339+0000 7fb63aa35740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0e1881390 0x55f0e1c49d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:23.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.339+0000 7fb63aa35740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f0e1bbfea0 con 0x55f0e18791b0 2024-09-21T06:06:23.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.339+0000 7fb63aa35740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f0e1baf2d0 con 0x55f0e1881390 2024-09-21T06:06:23.343 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.339+0000 7fb63aa35740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f0e1bb30e0 con 0x55f0e18a2070 2024-09-21T06:06:23.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.340+0000 7fb63225d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0e18791b0 0x55f0e1c49800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:23.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.340+0000 7fb63225d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0e18791b0 0x55f0e1c49800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45616/0 (socket says 172.21.15.117:45616) 2024-09-21T06:06:23.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.340+0000 7fb63225d640 1 -- 172.21.15.117:0/2534136530 learned_addr learned my addr 172.21.15.117:0/2534136530 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:23.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.340+0000 7fb631a5c640 1 --2- 172.21.15.117:0/2534136530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0e1881390 0x55f0e1c49d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:23.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.340+0000 7fb632a5e640 1 --2- 172.21.15.117:0/2534136530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0e18a2070 0x55f0e1c492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:23.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.340+0000 7fb63225d640 1 -- 172.21.15.117:0/2534136530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0e18a2070 msgr2=0x55f0e1c492c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:23.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.340+0000 7fb63225d640 1 --2- 172.21.15.117:0/2534136530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0e18a2070 0x55f0e1c492c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:23.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.340+0000 7fb63225d640 1 -- 172.21.15.117:0/2534136530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0e1881390 msgr2=0x55f0e1c49d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:23.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.340+0000 7fb63225d640 1 --2- 172.21.15.117:0/2534136530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0e1881390 0x55f0e1c49d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:23.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.340+0000 7fb63225d640 1 -- 172.21.15.117:0/2534136530 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0e190d650 con 0x55f0e18791b0 2024-09-21T06:06:23.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.340+0000 7fb6237fe640 1 --2- 172.21.15.117:0/2668122822 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6180b6680 0x7fb6180b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:23.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.340+0000 7fb632a5e640 1 --2- 172.21.15.117:0/2668122822 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6180b6680 0x7fb6180b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:23.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.340+0000 7fb63225d640 1 --2- 172.21.15.117:0/2534136530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0e18791b0 0x55f0e1c49800 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fb62800bcf0 tx=0x7fb62800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a348c27b6eac129 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:23.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.340+0000 7fb6237fe640 1 -- 172.21.15.117:0/2668122822 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fb61c107ec0 con 0x7fb62c1094d0 2024-09-21T06:06:23.344 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.340+0000 7fb620d9a640 1 -- 172.21.15.117:0/2534136530 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb628015070 con 0x55f0e18791b0 2024-09-21T06:06:23.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.341+0000 7fb620d9a640 1 -- 172.21.15.117:0/2534136530 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb62801a910 con 0x55f0e18791b0 2024-09-21T06:06:23.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.341+0000 7fb620d9a640 1 -- 172.21.15.117:0/2534136530 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6280119a0 con 0x55f0e18791b0 2024-09-21T06:06:23.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.341+0000 7fb632a5e640 1 --2- 172.21.15.117:0/2668122822 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6180b6680 0x7fb6180b8b90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb624005fd0 tx=0x7fb6240097e0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:23.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.341+0000 7fb63aa35740 1 -- 172.21.15.117:0/2534136530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0e18791b0 msgr2=0x55f0e1c49800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:23.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.341+0000 7fb63aa35740 1 --2- 172.21.15.117:0/2534136530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0e18791b0 0x55f0e1c49800 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fb62800bcf0 tx=0x7fb62800f730 comp rx=0 tx=0).stop 2024-09-21T06:06:23.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.341+0000 7fb63aa35740 1 -- 172.21.15.117:0/2534136530 shutdown_connections 2024-09-21T06:06:23.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.342+0000 7fb63aa35740 1 --2- 172.21.15.117:0/2534136530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0e18791b0 0x55f0e1c49800 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:23.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.342+0000 7fb63aa35740 1 --2- 172.21.15.117:0/2534136530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0e1881390 0x55f0e1c49d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:23.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.342+0000 7fb63aa35740 1 --2- 172.21.15.117:0/2534136530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0e18a2070 0x55f0e1c492c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:23.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.342+0000 7fb63aa35740 1 -- 172.21.15.117:0/2534136530 >> 172.21.15.117:0/2534136530 conn(0x55f0e1896460 msgr2=0x55f0e1c471b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:23.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.342+0000 7fb63aa35740 1 -- 172.21.15.117:0/2534136530 shutdown_connections 2024-09-21T06:06:23.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.342+0000 7fb63aa35740 1 -- 172.21.15.117:0/2534136530 wait complete. 2024-09-21T06:06:23.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.342+0000 7fb63aa35740 1 -- start start 2024-09-21T06:06:23.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.342+0000 7fb63aa35740 1 Processor -- start 2024-09-21T06:06:23.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.342+0000 7fb63aa35740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0e18791b0 0x55f0e1c54980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:23.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.343+0000 7fb63aa35740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0e1881390 0x55f0e1c54ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:23.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.343+0000 7fb63aa35740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0e18a2070 0x55f0e1c60d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:23.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.343+0000 7fb63aa35740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f0e1a10700 con 0x55f0e18a2070 2024-09-21T06:06:23.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.343+0000 7fb63aa35740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f0e1b4a160 con 0x55f0e18791b0 2024-09-21T06:06:23.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.343+0000 7fb63aa35740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f0e1c57aa0 con 0x55f0e1881390 2024-09-21T06:06:23.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.343+0000 7fb63aa35740 10 client.0 _ll_register_callbacks cb 0x55f0e1b639e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:23.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.343+0000 7fb632a5e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0e18a2070 0x55f0e1c60d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:23.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.343+0000 7fb632a5e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0e18a2070 0x55f0e1c60d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45618/0 (socket says 172.21.15.117:45618) 2024-09-21T06:06:23.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.343+0000 7fb632a5e640 1 -- 172.21.15.117:0/3705789794 learned_addr learned my addr 172.21.15.117:0/3705789794 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:23.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.343+0000 7fb63225d640 1 --2- 172.21.15.117:0/3705789794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0e18791b0 0x55f0e1c54980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:23.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.343+0000 7fb631a5c640 1 --2- 172.21.15.117:0/3705789794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0e1881390 0x55f0e1c54ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:23.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.343+0000 7fb632a5e640 1 -- 172.21.15.117:0/3705789794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0e1881390 msgr2=0x55f0e1c54ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:23.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.343+0000 7fb632a5e640 1 --2- 172.21.15.117:0/3705789794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0e1881390 0x55f0e1c54ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:23.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.343+0000 7fb632a5e640 1 -- 172.21.15.117:0/3705789794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0e18791b0 msgr2=0x55f0e1c54980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:23.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.343+0000 7fb632a5e640 1 --2- 172.21.15.117:0/3705789794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0e18791b0 0x55f0e1c54980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:23.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.343+0000 7fb632a5e640 1 -- 172.21.15.117:0/3705789794 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0e1b77c90 con 0x55f0e18a2070 2024-09-21T06:06:23.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.343+0000 7fb632a5e640 1 --2- 172.21.15.117:0/3705789794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0e18a2070 0x55f0e1c60d90 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fb624003ee0 tx=0x7fb624004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e97b70d5a88299e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:23.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.343+0000 7fb615ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:23.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.343+0000 7fb615ffb640 1 -- 172.21.15.117:0/3705789794 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6240109c0 con 0x55f0e18a2070 2024-09-21T06:06:23.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.343+0000 7fb615ffb640 1 -- 172.21.15.117:0/3705789794 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fb624010b60 con 0x55f0e18a2070 2024-09-21T06:06:23.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.343+0000 7fb63aa35740 1 -- 172.21.15.117:0/3705789794 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb62801a910 con 0x55f0e18a2070 2024-09-21T06:06:23.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.344+0000 7fb615ffb640 1 -- 172.21.15.117:0/3705789794 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb624010d60 con 0x55f0e18a2070 2024-09-21T06:06:23.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.344+0000 7fb63aa35740 10 client.17259 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:23.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.344+0000 7fb63aa35740 1 -- 172.21.15.117:0/3705789794 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f0e1c63810 con 0x55f0e18a2070 2024-09-21T06:06:23.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.344+0000 7fb615ffb640 1 -- 172.21.15.117:0/3705789794 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb62801a910 con 0x55f0e18a2070 2024-09-21T06:06:23.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.344+0000 7fb615ffb640 1 -- 172.21.15.117:0/3705789794 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb62401f540 con 0x55f0e18a2070 2024-09-21T06:06:23.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.344+0000 7fb63aa35740 10 client.17259 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:23.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.344+0000 7fb63aa35740 10 client.17259 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:06:23.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.344+0000 7fb63aa35740 20 client.17259 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:23.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.344+0000 7fb63aa35740 1 --2- 172.21.15.117:0/3705789794 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55f0e18609f0 0x55f0e1c66d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:23.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.344+0000 7fb63aa35740 4 client.17259 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:06:23.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.344+0000 7fb63aa35740 1 -- 172.21.15.117:0/3705789794 --> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0e1bc1df0 con 0x55f0e18609f0 2024-09-21T06:06:23.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.344+0000 7fb63225d640 1 --2- 172.21.15.117:0/3705789794 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55f0e18609f0 0x55f0e1c66d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:23.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.345+0000 7fb615ffb640 10 client.17259 ms_handle_connect on v2:172.21.15.192:6836/2683860842 2024-09-21T06:06:23.351 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.345+0000 7fb63225d640 1 --2- 172.21.15.117:0/3705789794 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55f0e18609f0 0x55f0e1c66d30 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=150b0a643b82c5ca server_cookie=f8eb3842d1cc5dd5 in_seq=0 out_seq=0 2024-09-21T06:06:23.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.346+0000 7fb615ffb640 1 -- 172.21.15.117:0/3705789794 <== mds.2 v2:172.21.15.192:6836/2683860842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f0e1bc1df0 con 0x55f0e18609f0 2024-09-21T06:06:23.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.346+0000 7fb615ffb640 10 client.17259 handle_command_reply: tid=0 2024-09-21T06:06:23.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.346+0000 7fb63aa35740 1 client.17259 shutdown 2024-09-21T06:06:23.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.346+0000 7fb63aa35740 1 -- 172.21.15.117:0/3705789794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0e18a2070 msgr2=0x55f0e1c60d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:23.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.346+0000 7fb63aa35740 1 --2- 172.21.15.117:0/3705789794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0e18a2070 0x55f0e1c60d90 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fb624003ee0 tx=0x7fb624004a40 comp rx=0 tx=0).stop 2024-09-21T06:06:23.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.346+0000 7fb63aa35740 1 -- 172.21.15.117:0/3705789794 shutdown_connections 2024-09-21T06:06:23.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.346+0000 7fb63aa35740 1 --2- 172.21.15.117:0/3705789794 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55f0e18609f0 0x55f0e1c66d30 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:23.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.346+0000 7fb63aa35740 1 --2- 172.21.15.117:0/3705789794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0e18a2070 0x55f0e1c60d90 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:23.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.346+0000 7fb63aa35740 1 --2- 172.21.15.117:0/3705789794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0e1881390 0x55f0e1c54ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:23.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.346+0000 7fb615ffb640 0 client.17259 ms_handle_reset on v2:172.21.15.192:6836/2683860842 2024-09-21T06:06:23.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.346+0000 7fb63aa35740 1 --2- 172.21.15.117:0/3705789794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0e18791b0 0x55f0e1c54980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:23.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.346+0000 7fb63aa35740 1 -- 172.21.15.117:0/3705789794 >> 172.21.15.117:0/3705789794 conn(0x55f0e1896460 msgr2=0x55f0e1c4c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:23.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.346+0000 7fb63aa35740 1 -- 172.21.15.117:0/3705789794 shutdown_connections 2024-09-21T06:06:23.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.346+0000 7fb63aa35740 1 -- 172.21.15.117:0/3705789794 wait complete. 2024-09-21T06:06:23.352 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.346+0000 7fb63aa35740 20 client.17259 trim_cache size 0 max 16384 2024-09-21T06:06:23.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.359+0000 7fb63aa35740 1 Processor -- start 2024-09-21T06:06:23.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.359+0000 7fb63aa35740 1 -- start start 2024-09-21T06:06:23.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.359+0000 7fb63aa35740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0e18791b0 0x55f0e1c53870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:23.361 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.359+0000 7fb63aa35740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0e1881390 0x55f0e1c4cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:23.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.359+0000 7fb63aa35740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0e18a2070 0x55f0e1c51b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:23.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.359+0000 7fb63aa35740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c1af5c0 con 0x55f0e18a2070 2024-09-21T06:06:23.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.359+0000 7fb63aa35740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c1b1100 con 0x55f0e1881390 2024-09-21T06:06:23.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.359+0000 7fb63aa35740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c1afc90 con 0x55f0e18791b0 2024-09-21T06:06:23.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.359+0000 7fb632a5e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0e18a2070 0x55f0e1c51b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:23.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.359+0000 7fb632a5e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0e18a2070 0x55f0e1c51b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45622/0 (socket says 172.21.15.117:45622) 2024-09-21T06:06:23.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.359+0000 7fb632a5e640 1 -- 172.21.15.117:0/462078246 learned_addr learned my addr 172.21.15.117:0/462078246 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:23.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.359+0000 7fb632a5e640 1 -- 172.21.15.117:0/462078246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0e18791b0 msgr2=0x55f0e1c53870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:23.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.359+0000 7fb63225d640 1 --2- 172.21.15.117:0/462078246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0e18791b0 0x55f0e1c53870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:23.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.360+0000 7fb631a5c640 1 --2- 172.21.15.117:0/462078246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0e1881390 0x55f0e1c4cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:23.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.360+0000 7fb632a5e640 1 --2- 172.21.15.117:0/462078246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0e18791b0 0x55f0e1c53870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:23.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.360+0000 7fb632a5e640 1 -- 172.21.15.117:0/462078246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0e1881390 msgr2=0x55f0e1c4cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:23.362 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.360+0000 7fb632a5e640 1 --2- 172.21.15.117:0/462078246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0e1881390 0x55f0e1c4cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:23.363 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.360+0000 7fb632a5e640 1 -- 172.21.15.117:0/462078246 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0e1c4c120 con 0x55f0e18a2070 2024-09-21T06:06:23.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.360+0000 7fb63225d640 1 --2- 172.21.15.117:0/462078246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0e18791b0 0x55f0e1c53870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:06:23.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.360+0000 7fb631a5c640 1 --2- 172.21.15.117:0/462078246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0e1881390 0x55f0e1c4cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:06:23.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.360+0000 7fb632a5e640 1 --2- 172.21.15.117:0/462078246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0e18a2070 0x55f0e1c51b70 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fb624012d00 tx=0x7fb624003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5fc9c923f27b63f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:23.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.360+0000 7fb617fff640 1 -- 172.21.15.117:0/462078246 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62403e070 con 0x55f0e18a2070 2024-09-21T06:06:23.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.360+0000 7fb617fff640 1 -- 172.21.15.117:0/462078246 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb62401f420 con 0x55f0e18a2070 2024-09-21T06:06:23.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.360+0000 7fb617fff640 1 -- 172.21.15.117:0/462078246 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6240107c0 con 0x55f0e18a2070 2024-09-21T06:06:23.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.361+0000 7fb63aa35740 1 -- 172.21.15.117:0/462078246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0e18a2070 msgr2=0x55f0e1c51b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:23.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.361+0000 7fb63aa35740 1 --2- 172.21.15.117:0/462078246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0e18a2070 0x55f0e1c51b70 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fb624012d00 tx=0x7fb624003f10 comp rx=0 tx=0).stop 2024-09-21T06:06:23.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.361+0000 7fb63aa35740 1 -- 172.21.15.117:0/462078246 shutdown_connections 2024-09-21T06:06:23.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.361+0000 7fb63aa35740 1 --2- 172.21.15.117:0/462078246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0e18a2070 0x55f0e1c51b70 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:23.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.361+0000 7fb63aa35740 1 --2- 172.21.15.117:0/462078246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0e1881390 0x55f0e1c4cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:23.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.361+0000 7fb63aa35740 1 --2- 172.21.15.117:0/462078246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0e18791b0 0x55f0e1c53870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:23.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.361+0000 7fb63aa35740 1 -- 172.21.15.117:0/462078246 >> 172.21.15.117:0/462078246 conn(0x55f0e1896460 msgr2=0x55f0e1c4c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:23.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.361+0000 7fb63aa35740 1 -- 172.21.15.117:0/462078246 shutdown_connections 2024-09-21T06:06:23.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.361+0000 7fb63aa35740 1 -- 172.21.15.117:0/462078246 wait complete. 2024-09-21T06:06:23.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.361+0000 7fb63aa35740 1 -- start start 2024-09-21T06:06:23.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.361+0000 7fb63aa35740 1 Processor -- start 2024-09-21T06:06:23.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.362+0000 7fb63aa35740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0e18791b0 0x55f0e1c5ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:23.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.362+0000 7fb63aa35740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0e1881390 0x55f0e1c5d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:23.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.362+0000 7fb63aa35740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0e18a2070 0x55f0e1ccaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:23.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.362+0000 7fb63aa35740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c1b0a30 con 0x55f0e18791b0 2024-09-21T06:06:23.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.362+0000 7fb63aa35740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c1b0360 con 0x55f0e1881390 2024-09-21T06:06:23.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.362+0000 7fb63aa35740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c1ad5c0 con 0x55f0e18a2070 2024-09-21T06:06:23.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.362+0000 7fb63aa35740 10 client.0 _ll_register_callbacks cb 0x7fb624009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:23.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.362+0000 7fb63225d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0e18791b0 0x55f0e1c5ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:23.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.362+0000 7fb63225d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0e18791b0 0x55f0e1c5ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45630/0 (socket says 172.21.15.117:45630) 2024-09-21T06:06:23.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.362+0000 7fb63225d640 1 -- 172.21.15.117:0/281190249 learned_addr learned my addr 172.21.15.117:0/281190249 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:23.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.362+0000 7fb632a5e640 1 --2- 172.21.15.117:0/281190249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0e18a2070 0x55f0e1ccaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:23.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.362+0000 7fb631a5c640 1 --2- 172.21.15.117:0/281190249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0e1881390 0x55f0e1c5d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:23.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.362+0000 7fb63225d640 1 -- 172.21.15.117:0/281190249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0e18a2070 msgr2=0x55f0e1ccaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:23.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.362+0000 7fb63225d640 1 --2- 172.21.15.117:0/281190249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0e18a2070 0x55f0e1ccaa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:23.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.362+0000 7fb63225d640 1 -- 172.21.15.117:0/281190249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0e1881390 msgr2=0x55f0e1c5d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:23.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.362+0000 7fb63225d640 1 --2- 172.21.15.117:0/281190249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0e1881390 0x55f0e1c5d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:23.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.362+0000 7fb63225d640 1 -- 172.21.15.117:0/281190249 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0e1c5da60 con 0x55f0e18791b0 2024-09-21T06:06:23.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.362+0000 7fb632a5e640 1 --2- 172.21.15.117:0/281190249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0e18a2070 0x55f0e1ccaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:06:23.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.362+0000 7fb63225d640 1 --2- 172.21.15.117:0/281190249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0e18791b0 0x55f0e1c5ccf0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x55f0e1c47f40 tx=0x7fb628005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e673aba34e47c4a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:23.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.362+0000 7fb615ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:23.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.362+0000 7fb615ffb640 1 -- 172.21.15.117:0/281190249 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb628015040 con 0x55f0e18791b0 2024-09-21T06:06:23.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.362+0000 7fb615ffb640 1 -- 172.21.15.117:0/281190249 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fb628005450 con 0x55f0e18791b0 2024-09-21T06:06:23.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.362+0000 7fb63aa35740 1 -- 172.21.15.117:0/281190249 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb62401f420 con 0x55f0e18791b0 2024-09-21T06:06:23.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.363+0000 7fb615ffb640 1 -- 172.21.15.117:0/281190249 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb62800b900 con 0x55f0e18791b0 2024-09-21T06:06:23.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.363+0000 7fb63aa35740 10 client.17265 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:23.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.363+0000 7fb63aa35740 1 -- 172.21.15.117:0/281190249 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f0e1cd6070 con 0x55f0e18791b0 2024-09-21T06:06:23.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.363+0000 7fb615ffb640 1 -- 172.21.15.117:0/281190249 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb62401f420 con 0x55f0e18791b0 2024-09-21T06:06:23.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.363+0000 7fb615ffb640 1 -- 172.21.15.117:0/281190249 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb6280115f0 con 0x55f0e18791b0 2024-09-21T06:06:23.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.363+0000 7fb63aa35740 10 client.17265 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:23.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.363+0000 7fb63aa35740 10 client.17265 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:06:23.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.363+0000 7fb63aa35740 20 client.17265 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:23.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.363+0000 7fb63aa35740 1 --2- 172.21.15.117:0/281190249 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55f0e1835da0 0x55f0e1cd8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:23.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.363+0000 7fb63aa35740 4 client.17265 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:06:23.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.363+0000 7fb63aa35740 1 -- 172.21.15.117:0/281190249 --> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f0e1c50530 con 0x55f0e1835da0 2024-09-21T06:06:23.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.363+0000 7fb631a5c640 1 --2- 172.21.15.117:0/281190249 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55f0e1835da0 0x55f0e1cd8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:23.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.364+0000 7fb631a5c640 1 --2- 172.21.15.117:0/281190249 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55f0e1835da0 0x55f0e1cd8520 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b97c5ab8406339e server_cookie=711c86154add7efc in_seq=0 out_seq=0 2024-09-21T06:06:23.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.364+0000 7fb615ffb640 10 client.17265 ms_handle_connect on v2:172.21.15.192:6836/2683860842 2024-09-21T06:06:23.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.364+0000 7fb615ffb640 1 -- 172.21.15.117:0/281190249 <== mds.2 v2:172.21.15.192:6836/2683860842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f0e1c50530 con 0x55f0e1835da0 2024-09-21T06:06:23.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.364+0000 7fb615ffb640 10 client.17265 handle_command_reply: tid=0 2024-09-21T06:06:23.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.364+0000 7fb63aa35740 1 client.17265 shutdown 2024-09-21T06:06:23.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.364+0000 7fb63aa35740 1 -- 172.21.15.117:0/281190249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0e18791b0 msgr2=0x55f0e1c5ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:23.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.364+0000 7fb63aa35740 1 --2- 172.21.15.117:0/281190249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0e18791b0 0x55f0e1c5ccf0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x55f0e1c47f40 tx=0x7fb628005c50 comp rx=0 tx=0).stop 2024-09-21T06:06:23.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.364+0000 7fb63aa35740 1 -- 172.21.15.117:0/281190249 shutdown_connections 2024-09-21T06:06:23.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.364+0000 7fb63aa35740 1 --2- 172.21.15.117:0/281190249 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55f0e1835da0 0x55f0e1cd8520 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:23.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.364+0000 7fb615ffb640 0 client.17265 ms_handle_reset on v2:172.21.15.192:6836/2683860842 2024-09-21T06:06:23.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.364+0000 7fb63aa35740 1 --2- 172.21.15.117:0/281190249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0e18a2070 0x55f0e1ccaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:23.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.364+0000 7fb63aa35740 1 --2- 172.21.15.117:0/281190249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0e1881390 0x55f0e1c5d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:23.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.364+0000 7fb63aa35740 1 --2- 172.21.15.117:0/281190249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0e18791b0 0x55f0e1c5ccf0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:23.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.364+0000 7fb63aa35740 1 -- 172.21.15.117:0/281190249 >> 172.21.15.117:0/281190249 conn(0x55f0e1896460 msgr2=0x55f0e1c4a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:23.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.364+0000 7fb63aa35740 1 -- 172.21.15.117:0/281190249 shutdown_connections 2024-09-21T06:06:23.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.364+0000 7fb63aa35740 1 -- 172.21.15.117:0/281190249 wait complete. 2024-09-21T06:06:23.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.365+0000 7fb63aa35740 20 client.17265 trim_cache size 0 max 16384 2024-09-21T06:06:23.371 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:06:23.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.365+0000 7fb6344c1640 1 -- 172.21.15.117:0/2668122822 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6180b6680 msgr2=0x7fb6180b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:23.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.365+0000 7fb6344c1640 1 --2- 172.21.15.117:0/2668122822 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6180b6680 0x7fb6180b8b90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb624005fd0 tx=0x7fb6240097e0 comp rx=0 tx=0).stop 2024-09-21T06:06:23.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.365+0000 7fb6344c1640 1 -- 172.21.15.117:0/2668122822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb62c1094d0 msgr2=0x7fb62c19aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:23.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.365+0000 7fb6344c1640 1 --2- 172.21.15.117:0/2668122822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb62c1094d0 0x7fb62c19aa50 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fb61c02a930 tx=0x7fb61c004270 comp rx=0 tx=0).stop 2024-09-21T06:06:23.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.365+0000 7fb6344c1640 1 -- 172.21.15.117:0/2668122822 shutdown_connections 2024-09-21T06:06:23.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.365+0000 7fb6344c1640 1 --2- 172.21.15.117:0/2668122822 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6180b6680 0x7fb6180b8b90 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:23.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.365+0000 7fb6344c1640 1 --2- 172.21.15.117:0/2668122822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb62c1094d0 0x7fb62c19aa50 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:23.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.365+0000 7fb6344c1640 1 --2- 172.21.15.117:0/2668122822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb62c109e20 0x7fb62c19fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:23.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.365+0000 7fb6344c1640 1 --2- 172.21.15.117:0/2668122822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb62c103b90 0x7fb62c19a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:23.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.365+0000 7fb6344c1640 1 -- 172.21.15.117:0/2668122822 >> 172.21.15.117:0/2668122822 conn(0x7fb62c0ff600 msgr2=0x7fb62c101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:23.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.365+0000 7fb6344c1640 1 -- 172.21.15.117:0/2668122822 shutdown_connections 2024-09-21T06:06:23.373 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:23.365+0000 7fb6344c1640 1 -- 172.21.15.117:0/2668122822 wait complete. 2024-09-21T06:06:24.163 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T06:06:24.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:23 smithi117 ceph-mon[30947]: pgmap v604: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T06:06:24.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:23 smithi138 ceph-mon[34391]: pgmap v604: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T06:06:24.411 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:24.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:23 smithi192 ceph-mon[34700]: pgmap v604: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 10 op/s 2024-09-21T06:06:24.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.902+0000 7f2095aa2640 1 -- 172.21.15.117:0/295268973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2090105380 msgr2=0x7f209010d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:24.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.902+0000 7f2095aa2640 1 --2- 172.21.15.117:0/295268973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2090105380 0x7f209010d8d0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f2084009900 tx=0x7f208402a410 comp rx=0 tx=0).stop 2024-09-21T06:06:24.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.902+0000 7f2095aa2640 1 -- 172.21.15.117:0/295268973 shutdown_connections 2024-09-21T06:06:24.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.902+0000 7f2095aa2640 1 --2- 172.21.15.117:0/295268973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2090105380 0x7f209010d8d0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:24.915 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.902+0000 7f2095aa2640 1 --2- 172.21.15.117:0/295268973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f209010de90 0x7f2090110380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:24.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.902+0000 7f2095aa2640 1 --2- 172.21.15.117:0/295268973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2090104a30 0x7f2090104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:24.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.902+0000 7f2095aa2640 1 -- 172.21.15.117:0/295268973 >> 172.21.15.117:0/295268973 conn(0x7f2090079770 msgr2=0x7f2090079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:24.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.902+0000 7f2095aa2640 1 -- 172.21.15.117:0/295268973 shutdown_connections 2024-09-21T06:06:24.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.902+0000 7f2095aa2640 1 -- 172.21.15.117:0/295268973 wait complete. 2024-09-21T06:06:24.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.902+0000 7f2095aa2640 1 Processor -- start 2024-09-21T06:06:24.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.903+0000 7f2095aa2640 1 -- start start 2024-09-21T06:06:24.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.903+0000 7f2095aa2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2090104a30 0x7f209019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:24.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.903+0000 7f2095aa2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2090105380 0x7f209019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:24.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.903+0000 7f2095aa2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f209010de90 0x7f209019fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:24.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.903+0000 7f2095aa2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2090113b30 con 0x7f2090104a30 2024-09-21T06:06:24.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.903+0000 7f2095aa2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f20901139b0 con 0x7f209010de90 2024-09-21T06:06:24.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.903+0000 7f2095aa2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2090113cb0 con 0x7f2090105380 2024-09-21T06:06:24.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.903+0000 7f208f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2090104a30 0x7f209019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:24.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.903+0000 7f208f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2090104a30 0x7f209019a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55036/0 (socket says 172.21.15.117:55036) 2024-09-21T06:06:24.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.903+0000 7f208f7fe640 1 -- 172.21.15.117:0/1808471064 learned_addr learned my addr 172.21.15.117:0/1808471064 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:24.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.903+0000 7f208ffff640 1 --2- 172.21.15.117:0/1808471064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f209010de90 0x7f209019fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:24.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.903+0000 7f208effd640 1 --2- 172.21.15.117:0/1808471064 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2090105380 0x7f209019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:24.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.903+0000 7f208f7fe640 1 -- 172.21.15.117:0/1808471064 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2090105380 msgr2=0x7f209019ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:24.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.903+0000 7f208f7fe640 1 --2- 172.21.15.117:0/1808471064 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2090105380 0x7f209019ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:24.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.903+0000 7f208f7fe640 1 -- 172.21.15.117:0/1808471064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f209010de90 msgr2=0x7f209019fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:24.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.903+0000 7f208f7fe640 1 --2- 172.21.15.117:0/1808471064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f209010de90 0x7f209019fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:24.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.903+0000 7f208f7fe640 1 -- 172.21.15.117:0/1808471064 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20901a0510 con 0x7f2090104a30 2024-09-21T06:06:24.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.903+0000 7f208ffff640 1 --2- 172.21.15.117:0/1808471064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f209010de90 0x7f209019fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:06:24.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.904+0000 7f208f7fe640 1 --2- 172.21.15.117:0/1808471064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2090104a30 0x7f209019a5a0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f208000b4a0 tx=0x7f208000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:24.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.904+0000 7f208cff9640 1 -- 172.21.15.117:0/1808471064 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2080014070 con 0x7f2090104a30 2024-09-21T06:06:24.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.904+0000 7f208cff9640 1 -- 172.21.15.117:0/1808471064 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2080010ac0 con 0x7f2090104a30 2024-09-21T06:06:24.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.904+0000 7f208cff9640 1 -- 172.21.15.117:0/1808471064 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2080017c30 con 0x7f2090104a30 2024-09-21T06:06:24.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.904+0000 7f2095aa2640 1 -- 172.21.15.117:0/1808471064 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20901a0800 con 0x7f2090104a30 2024-09-21T06:06:24.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.904+0000 7f2095aa2640 1 -- 172.21.15.117:0/1808471064 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20901a6f60 con 0x7f2090104a30 2024-09-21T06:06:24.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.906+0000 7f209c016740 1 Processor -- start 2024-09-21T06:06:24.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.906+0000 7f208cff9640 1 -- 172.21.15.117:0/1808471064 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f2080017070 con 0x7f2090104a30 2024-09-21T06:06:24.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.906+0000 7f209c016740 1 -- start start 2024-09-21T06:06:24.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.907+0000 7f209c016740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55973a70b070 0x55973aab26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:24.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.907+0000 7f209c016740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55973a6e21b0 0x55973aab2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:24.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.907+0000 7f209c016740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55973a6ea390 0x55973aab3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:24.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.907+0000 7f209c016740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55973aa28ea0 con 0x55973a6ea390 2024-09-21T06:06:24.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.907+0000 7f209c016740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55973aa182d0 con 0x55973a6e21b0 2024-09-21T06:06:24.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.907+0000 7f209c016740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55973aa1c0e0 con 0x55973a70b070 2024-09-21T06:06:24.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.907+0000 7f208effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55973a6ea390 0x55973aab3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:24.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.907+0000 7f208effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55973a6ea390 0x55973aab3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55052/0 (socket says 172.21.15.117:55052) 2024-09-21T06:06:24.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.907+0000 7f208effd640 1 -- 172.21.15.117:0/3571192519 learned_addr learned my addr 172.21.15.117:0/3571192519 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:24.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.907+0000 7f208ffff640 1 --2- 172.21.15.117:0/3571192519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55973a70b070 0x55973aab26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:24.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.907+0000 7f208f7fe640 1 --2- 172.21.15.117:0/3571192519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55973a6e21b0 0x55973aab2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:24.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.907+0000 7f208effd640 1 -- 172.21.15.117:0/3571192519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55973a70b070 msgr2=0x55973aab26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:24.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.907+0000 7f208effd640 1 --2- 172.21.15.117:0/3571192519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55973a70b070 0x55973aab26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:24.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.907+0000 7f208effd640 1 -- 172.21.15.117:0/3571192519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55973a6e21b0 msgr2=0x55973aab2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:24.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.907+0000 7f208effd640 1 --2- 172.21.15.117:0/3571192519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55973a6e21b0 0x55973aab2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:24.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.907+0000 7f208effd640 1 -- 172.21.15.117:0/3571192519 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55973a776650 con 0x55973a6ea390 2024-09-21T06:06:24.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.907+0000 7f208f7fe640 1 --2- 172.21.15.117:0/3571192519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55973a6e21b0 0x55973aab2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:06:24.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.907+0000 7f208cff9640 1 --2- 172.21.15.117:0/1808471064 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f20780b6680 0x7f20780b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:24.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.907+0000 7f208effd640 1 --2- 172.21.15.117:0/3571192519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55973a6ea390 0x55973aab3120 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f2084006010 tx=0x7f2084032c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1869133e5338e642 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:24.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.907+0000 7f208ffff640 1 --2- 172.21.15.117:0/1808471064 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f20780b6680 0x7f20780b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:24.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.908+0000 7f208cff9640 1 -- 172.21.15.117:0/1808471064 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7f20800e6a40 con 0x7f2090104a30 2024-09-21T06:06:24.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.908+0000 7f2075ffb640 1 -- 172.21.15.117:0/3571192519 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208403f070 con 0x55973a6ea390 2024-09-21T06:06:24.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.908+0000 7f2075ffb640 1 -- 172.21.15.117:0/3571192519 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2084032e30 con 0x55973a6ea390 2024-09-21T06:06:24.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.908+0000 7f2075ffb640 1 -- 172.21.15.117:0/3571192519 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2084035070 con 0x55973a6ea390 2024-09-21T06:06:24.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.909+0000 7f209c016740 1 -- 172.21.15.117:0/3571192519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55973a6ea390 msgr2=0x55973aab3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:24.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.909+0000 7f209c016740 1 --2- 172.21.15.117:0/3571192519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55973a6ea390 0x55973aab3120 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f2084006010 tx=0x7f2084032c40 comp rx=0 tx=0).stop 2024-09-21T06:06:24.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.909+0000 7f208ffff640 1 --2- 172.21.15.117:0/1808471064 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f20780b6680 0x7f20780b8b90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2088005fd0 tx=0x7f2088009700 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:24.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.909+0000 7f209c016740 1 -- 172.21.15.117:0/3571192519 shutdown_connections 2024-09-21T06:06:24.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.909+0000 7f209c016740 1 --2- 172.21.15.117:0/3571192519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55973a6ea390 0x55973aab3120 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:24.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.909+0000 7f209c016740 1 --2- 172.21.15.117:0/3571192519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55973a6e21b0 0x55973aab2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:24.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.909+0000 7f209c016740 1 --2- 172.21.15.117:0/3571192519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55973a70b070 0x55973aab26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:24.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.909+0000 7f209c016740 1 -- 172.21.15.117:0/3571192519 >> 172.21.15.117:0/3571192519 conn(0x55973a6ff460 msgr2=0x55973aab0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:24.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.909+0000 7f209c016740 1 -- 172.21.15.117:0/3571192519 shutdown_connections 2024-09-21T06:06:24.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.909+0000 7f209c016740 1 -- 172.21.15.117:0/3571192519 wait complete. 2024-09-21T06:06:24.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.909+0000 7f209c016740 1 -- start start 2024-09-21T06:06:24.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.909+0000 7f209c016740 1 Processor -- start 2024-09-21T06:06:24.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.910+0000 7f209c016740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55973a6e21b0 0x55973aabdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:24.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.910+0000 7f209c016740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55973a6ea390 0x55973aabe280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:24.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.910+0000 7f209c016740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55973a70b070 0x55973aaca110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:24.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.910+0000 7f209c016740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55973a879700 con 0x55973a70b070 2024-09-21T06:06:24.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.910+0000 7f209c016740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55973a9b3160 con 0x55973a6e21b0 2024-09-21T06:06:24.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.910+0000 7f209c016740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55973aac0e20 con 0x55973a6ea390 2024-09-21T06:06:24.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.910+0000 7f209c016740 10 client.0 _ll_register_callbacks cb 0x55973a9cc9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:24.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.910+0000 7f208ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55973a70b070 0x55973aaca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:24.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.910+0000 7f208ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55973a70b070 0x55973aaca110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55062/0 (socket says 172.21.15.117:55062) 2024-09-21T06:06:24.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.910+0000 7f208ffff640 1 -- 172.21.15.117:0/775269110 learned_addr learned my addr 172.21.15.117:0/775269110 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:24.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.910+0000 7f208f7fe640 1 --2- 172.21.15.117:0/775269110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55973a6e21b0 0x55973aabdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:24.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.910+0000 7f208effd640 1 --2- 172.21.15.117:0/775269110 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55973a6ea390 0x55973aabe280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:24.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.910+0000 7f208ffff640 1 -- 172.21.15.117:0/775269110 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55973a6ea390 msgr2=0x55973aabe280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:24.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.910+0000 7f208ffff640 1 --2- 172.21.15.117:0/775269110 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55973a6ea390 0x55973aabe280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:24.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.910+0000 7f208ffff640 1 -- 172.21.15.117:0/775269110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55973a6e21b0 msgr2=0x55973aabdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:24.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.910+0000 7f208ffff640 1 --2- 172.21.15.117:0/775269110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55973a6e21b0 0x55973aabdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:24.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.910+0000 7f208ffff640 1 -- 172.21.15.117:0/775269110 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55973a9e0c90 con 0x55973a70b070 2024-09-21T06:06:24.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.910+0000 7f208effd640 1 --2- 172.21.15.117:0/775269110 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55973a6ea390 0x55973aabe280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:06:24.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.910+0000 7f208ffff640 1 --2- 172.21.15.117:0/775269110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55973a70b070 0x55973aaca110 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f2088013fd0 tx=0x7f2088004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3104126cc09bd016 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:24.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.910+0000 7f206effd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:24.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.910+0000 7f206effd640 1 -- 172.21.15.117:0/775269110 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208801e6d0 con 0x55973a70b070 2024-09-21T06:06:24.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.910+0000 7f206effd640 1 -- 172.21.15.117:0/775269110 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7f208801e870 con 0x55973a70b070 2024-09-21T06:06:24.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.910+0000 7f209c016740 1 -- 172.21.15.117:0/775269110 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2084032e30 con 0x55973a70b070 2024-09-21T06:06:24.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.911+0000 7f206effd640 1 -- 172.21.15.117:0/775269110 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f208801eab0 con 0x55973a70b070 2024-09-21T06:06:24.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.911+0000 7f209c016740 10 client.17277 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:24.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.911+0000 7f209c016740 1 -- 172.21.15.117:0/775269110 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55973aaccb70 con 0x55973a70b070 2024-09-21T06:06:24.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.911+0000 7f206effd640 1 -- 172.21.15.117:0/775269110 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2084032e30 con 0x55973a70b070 2024-09-21T06:06:24.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.911+0000 7f206effd640 1 -- 172.21.15.117:0/775269110 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f208801d410 con 0x55973a70b070 2024-09-21T06:06:24.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.911+0000 7f209c016740 10 client.17277 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:24.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.911+0000 7f209c016740 10 client.17277 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:06:24.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.911+0000 7f209c016740 20 client.17277 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:24.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.911+0000 7f209c016740 1 --2- 172.21.15.117:0/775269110 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55973a6c99f0 0x55973aad00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:24.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.911+0000 7f209c016740 4 client.17277 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:06:24.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.911+0000 7f209c016740 1 -- 172.21.15.117:0/775269110 --> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55973aa2adf0 con 0x55973a6c99f0 2024-09-21T06:06:24.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.911+0000 7f208f7fe640 1 --2- 172.21.15.117:0/775269110 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55973a6c99f0 0x55973aad00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:24.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.912+0000 7f208f7fe640 1 --2- 172.21.15.117:0/775269110 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55973a6c99f0 0x55973aad00e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=56fed2ae0b795021 server_cookie=bac027f4d000b892 in_seq=0 out_seq=0 2024-09-21T06:06:24.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.912+0000 7f206effd640 10 client.17277 ms_handle_connect on v2:172.21.15.192:6838/1727623231 2024-09-21T06:06:24.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.913+0000 7f206effd640 1 -- 172.21.15.117:0/775269110 <== mds.1 v2:172.21.15.192:6838/1727623231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55973aa2adf0 con 0x55973a6c99f0 2024-09-21T06:06:24.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.913+0000 7f206effd640 10 client.17277 handle_command_reply: tid=0 2024-09-21T06:06:24.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.913+0000 7f209c016740 1 client.17277 shutdown 2024-09-21T06:06:24.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.913+0000 7f209c016740 1 -- 172.21.15.117:0/775269110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55973a70b070 msgr2=0x55973aaca110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:24.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.913+0000 7f209c016740 1 --2- 172.21.15.117:0/775269110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55973a70b070 0x55973aaca110 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f2088013fd0 tx=0x7f2088004a40 comp rx=0 tx=0).stop 2024-09-21T06:06:24.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.913+0000 7f209c016740 1 -- 172.21.15.117:0/775269110 shutdown_connections 2024-09-21T06:06:24.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.913+0000 7f209c016740 1 --2- 172.21.15.117:0/775269110 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55973a6c99f0 0x55973aad00e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:24.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.913+0000 7f206effd640 0 client.17277 ms_handle_reset on v2:172.21.15.192:6838/1727623231 2024-09-21T06:06:24.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.913+0000 7f209c016740 1 --2- 172.21.15.117:0/775269110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55973a70b070 0x55973aaca110 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:24.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.913+0000 7f209c016740 1 --2- 172.21.15.117:0/775269110 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55973a6ea390 0x55973aabe280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:24.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.913+0000 7f209c016740 1 --2- 172.21.15.117:0/775269110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55973a6e21b0 0x55973aabdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:24.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.913+0000 7f209c016740 1 -- 172.21.15.117:0/775269110 >> 172.21.15.117:0/775269110 conn(0x55973a6ff460 msgr2=0x55973aab5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:24.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.913+0000 7f209c016740 1 -- 172.21.15.117:0/775269110 shutdown_connections 2024-09-21T06:06:24.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.913+0000 7f209c016740 1 -- 172.21.15.117:0/775269110 wait complete. 2024-09-21T06:06:24.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.913+0000 7f209c016740 20 client.17277 trim_cache size 0 max 16384 2024-09-21T06:06:24.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.926+0000 7f209c016740 1 Processor -- start 2024-09-21T06:06:24.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.926+0000 7f209c016740 1 -- start start 2024-09-21T06:06:24.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.926+0000 7f209c016740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55973a6c99f0 0x55973aabf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:24.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.926+0000 7f209c016740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55973a6e21b0 0x55973aaca010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:24.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.926+0000 7f209c016740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55973a6ea390 0x55973aabaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:24.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.926+0000 7f209c016740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f20901b18c0 con 0x55973a6ea390 2024-09-21T06:06:24.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.926+0000 7f209c016740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f20901b3400 con 0x55973a6c99f0 2024-09-21T06:06:24.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.926+0000 7f209c016740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f20901b1f90 con 0x55973a6e21b0 2024-09-21T06:06:24.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.926+0000 7f208ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55973a6ea390 0x55973aabaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:24.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.926+0000 7f208ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55973a6ea390 0x55973aabaf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55076/0 (socket says 172.21.15.117:55076) 2024-09-21T06:06:24.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.926+0000 7f208ffff640 1 -- 172.21.15.117:0/2396107975 learned_addr learned my addr 172.21.15.117:0/2396107975 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:24.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.926+0000 7f208f7fe640 1 --2- 172.21.15.117:0/2396107975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55973a6c99f0 0x55973aabf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:24.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.926+0000 7f208effd640 1 --2- 172.21.15.117:0/2396107975 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55973a6e21b0 0x55973aaca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:24.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.927+0000 7f208ffff640 1 -- 172.21.15.117:0/2396107975 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55973a6e21b0 msgr2=0x55973aaca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:24.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.927+0000 7f208ffff640 1 --2- 172.21.15.117:0/2396107975 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55973a6e21b0 0x55973aaca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:24.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.927+0000 7f208ffff640 1 -- 172.21.15.117:0/2396107975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55973a6c99f0 msgr2=0x55973aabf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:24.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.927+0000 7f208ffff640 1 --2- 172.21.15.117:0/2396107975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55973a6c99f0 0x55973aabf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:24.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.927+0000 7f208ffff640 1 -- 172.21.15.117:0/2396107975 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55973aab07c0 con 0x55973a6ea390 2024-09-21T06:06:24.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.927+0000 7f208effd640 1 --2- 172.21.15.117:0/2396107975 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55973a6e21b0 0x55973aaca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:06:24.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.927+0000 7f208ffff640 1 --2- 172.21.15.117:0/2396107975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55973a6ea390 0x55973aabaf30 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f2088009de0 tx=0x7f2088010540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c42f95ea0f7b6d5e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:24.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.927+0000 7f20757fa640 1 -- 172.21.15.117:0/2396107975 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2088034040 con 0x55973a6ea390 2024-09-21T06:06:24.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.927+0000 7f20757fa640 1 -- 172.21.15.117:0/2396107975 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2088010e90 con 0x55973a6ea390 2024-09-21T06:06:24.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.927+0000 7f20757fa640 1 -- 172.21.15.117:0/2396107975 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2088005040 con 0x55973a6ea390 2024-09-21T06:06:24.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.928+0000 7f209c016740 1 -- 172.21.15.117:0/2396107975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55973a6ea390 msgr2=0x55973aabaf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:24.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.928+0000 7f209c016740 1 --2- 172.21.15.117:0/2396107975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55973a6ea390 0x55973aabaf30 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f2088009de0 tx=0x7f2088010540 comp rx=0 tx=0).stop 2024-09-21T06:06:24.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.928+0000 7f209c016740 1 -- 172.21.15.117:0/2396107975 shutdown_connections 2024-09-21T06:06:24.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.928+0000 7f209c016740 1 --2- 172.21.15.117:0/2396107975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55973a6ea390 0x55973aabaf30 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:24.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.928+0000 7f209c016740 1 --2- 172.21.15.117:0/2396107975 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55973a6e21b0 0x55973aaca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:24.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.928+0000 7f209c016740 1 --2- 172.21.15.117:0/2396107975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55973a6c99f0 0x55973aabf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:24.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.928+0000 7f209c016740 1 -- 172.21.15.117:0/2396107975 >> 172.21.15.117:0/2396107975 conn(0x55973a6ff460 msgr2=0x55973aabf260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:24.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.928+0000 7f209c016740 1 -- 172.21.15.117:0/2396107975 shutdown_connections 2024-09-21T06:06:24.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.928+0000 7f209c016740 1 -- 172.21.15.117:0/2396107975 wait complete. 2024-09-21T06:06:24.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.928+0000 7f209c016740 1 -- start start 2024-09-21T06:06:24.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.929+0000 7f209c016740 1 Processor -- start 2024-09-21T06:06:24.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.929+0000 7f209c016740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55973a6c99f0 0x55973aac24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:24.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.929+0000 7f209c016740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55973a6e21b0 0x55973aac2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:24.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.929+0000 7f209c016740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55973a6ea390 0x55973ab39ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:24.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.929+0000 7f209c016740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f20901b2d30 con 0x55973a6ea390 2024-09-21T06:06:24.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.929+0000 7f209c016740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f20901b2660 con 0x55973a6c99f0 2024-09-21T06:06:24.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.929+0000 7f209c016740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f20901af8c0 con 0x55973a6e21b0 2024-09-21T06:06:24.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.929+0000 7f209c016740 10 client.0 _ll_register_callbacks cb 0x7f2088009c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:24.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.929+0000 7f208ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55973a6ea390 0x55973ab39ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:24.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.929+0000 7f208ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55973a6ea390 0x55973ab39ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55084/0 (socket says 172.21.15.117:55084) 2024-09-21T06:06:24.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.929+0000 7f208ffff640 1 -- 172.21.15.117:0/423371369 learned_addr learned my addr 172.21.15.117:0/423371369 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:24.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.929+0000 7f208effd640 1 --2- 172.21.15.117:0/423371369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55973a6e21b0 0x55973aac2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:24.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.929+0000 7f208f7fe640 1 --2- 172.21.15.117:0/423371369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55973a6c99f0 0x55973aac24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:24.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.929+0000 7f208ffff640 1 -- 172.21.15.117:0/423371369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55973a6e21b0 msgr2=0x55973aac2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:24.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.929+0000 7f208ffff640 1 --2- 172.21.15.117:0/423371369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55973a6e21b0 0x55973aac2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:24.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.929+0000 7f208ffff640 1 -- 172.21.15.117:0/423371369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55973a6c99f0 msgr2=0x55973aac24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:24.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.929+0000 7f208ffff640 1 --2- 172.21.15.117:0/423371369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55973a6c99f0 0x55973aac24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:24.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.929+0000 7f208ffff640 1 -- 172.21.15.117:0/423371369 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55973ab3a000 con 0x55973a6ea390 2024-09-21T06:06:24.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.930+0000 7f208ffff640 1 --2- 172.21.15.117:0/423371369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55973a6ea390 0x55973ab39ac0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f2088013fd0 tx=0x7f208801d610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee9bc035ac64b6da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:24.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.930+0000 7f206effd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:24.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.930+0000 7f206effd640 1 -- 172.21.15.117:0/423371369 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2088034040 con 0x55973a6ea390 2024-09-21T06:06:24.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.930+0000 7f206effd640 1 -- 172.21.15.117:0/423371369 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7f208801dc70 con 0x55973a6ea390 2024-09-21T06:06:24.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.930+0000 7f209c016740 1 -- 172.21.15.117:0/423371369 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2088010e90 con 0x55973a6ea390 2024-09-21T06:06:24.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.930+0000 7f206effd640 1 -- 172.21.15.117:0/423371369 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2088052050 con 0x55973a6ea390 2024-09-21T06:06:24.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.930+0000 7f209c016740 10 client.17283 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:24.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.930+0000 7f209c016740 1 -- 172.21.15.117:0/423371369 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55973ab3b120 con 0x55973a6ea390 2024-09-21T06:06:24.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.930+0000 7f206effd640 1 -- 172.21.15.117:0/423371369 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2088010e90 con 0x55973a6ea390 2024-09-21T06:06:24.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.930+0000 7f206effd640 1 -- 172.21.15.117:0/423371369 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2088043070 con 0x55973a6ea390 2024-09-21T06:06:24.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.930+0000 7f209c016740 10 client.17283 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:24.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.930+0000 7f209c016740 10 client.17283 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:06:24.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.930+0000 7f209c016740 20 client.17283 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:24.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.930+0000 7f209c016740 1 --2- 172.21.15.117:0/423371369 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55973a70b070 0x55973ab3d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:24.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.930+0000 7f209c016740 4 client.17283 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:06:24.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.930+0000 7f209c016740 1 -- 172.21.15.117:0/423371369 --> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55973aa268a0 con 0x55973a70b070 2024-09-21T06:06:24.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.931+0000 7f208f7fe640 1 --2- 172.21.15.117:0/423371369 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55973a70b070 0x55973ab3d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:24.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.931+0000 7f208f7fe640 1 --2- 172.21.15.117:0/423371369 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55973a70b070 0x55973ab3d780 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=79aae0369c22b9f2 server_cookie=e7d846edb0d3c91a in_seq=0 out_seq=0 2024-09-21T06:06:24.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.931+0000 7f206effd640 10 client.17283 ms_handle_connect on v2:172.21.15.192:6838/1727623231 2024-09-21T06:06:24.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.931+0000 7f206effd640 1 -- 172.21.15.117:0/423371369 <== mds.1 v2:172.21.15.192:6838/1727623231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55973aa268a0 con 0x55973a70b070 2024-09-21T06:06:24.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.931+0000 7f206effd640 10 client.17283 handle_command_reply: tid=0 2024-09-21T06:06:24.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.931+0000 7f209c016740 1 client.17283 shutdown 2024-09-21T06:06:24.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.931+0000 7f209c016740 1 -- 172.21.15.117:0/423371369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55973a6ea390 msgr2=0x55973ab39ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:24.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.931+0000 7f209c016740 1 --2- 172.21.15.117:0/423371369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55973a6ea390 0x55973ab39ac0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f2088013fd0 tx=0x7f208801d610 comp rx=0 tx=0).stop 2024-09-21T06:06:24.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.931+0000 7f209c016740 1 -- 172.21.15.117:0/423371369 shutdown_connections 2024-09-21T06:06:24.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.931+0000 7f209c016740 1 --2- 172.21.15.117:0/423371369 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55973a70b070 0x55973ab3d780 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:24.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.931+0000 7f209c016740 1 --2- 172.21.15.117:0/423371369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55973a6ea390 0x55973ab39ac0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:24.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.931+0000 7f206effd640 0 client.17283 ms_handle_reset on v2:172.21.15.192:6838/1727623231 2024-09-21T06:06:24.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.931+0000 7f209c016740 1 --2- 172.21.15.117:0/423371369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55973a6e21b0 0x55973aac2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:24.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.932+0000 7f209c016740 1 --2- 172.21.15.117:0/423371369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55973a6c99f0 0x55973aac24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:24.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.932+0000 7f209c016740 1 -- 172.21.15.117:0/423371369 >> 172.21.15.117:0/423371369 conn(0x55973a6ff460 msgr2=0x55973aab3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:24.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.932+0000 7f209c016740 1 -- 172.21.15.117:0/423371369 shutdown_connections 2024-09-21T06:06:24.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.932+0000 7f209c016740 1 -- 172.21.15.117:0/423371369 wait complete. 2024-09-21T06:06:24.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.932+0000 7f209c016740 20 client.17283 trim_cache size 0 max 16384 2024-09-21T06:06:24.939 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:06:24.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.932+0000 7f2095aa2640 1 -- 172.21.15.117:0/1808471064 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f20780b6680 msgr2=0x7f20780b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:24.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.932+0000 7f2095aa2640 1 --2- 172.21.15.117:0/1808471064 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f20780b6680 0x7f20780b8b90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2088005fd0 tx=0x7f2088009700 comp rx=0 tx=0).stop 2024-09-21T06:06:24.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.932+0000 7f2095aa2640 1 -- 172.21.15.117:0/1808471064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2090104a30 msgr2=0x7f209019a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:24.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.932+0000 7f2095aa2640 1 --2- 172.21.15.117:0/1808471064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2090104a30 0x7f209019a5a0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f208000b4a0 tx=0x7f208000b960 comp rx=0 tx=0).stop 2024-09-21T06:06:24.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.932+0000 7f2095aa2640 1 -- 172.21.15.117:0/1808471064 shutdown_connections 2024-09-21T06:06:24.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.932+0000 7f2095aa2640 1 --2- 172.21.15.117:0/1808471064 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f20780b6680 0x7f20780b8b90 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:24.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.932+0000 7f2095aa2640 1 --2- 172.21.15.117:0/1808471064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f209010de90 0x7f209019fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:24.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.932+0000 7f2095aa2640 1 --2- 172.21.15.117:0/1808471064 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2090105380 0x7f209019ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:24.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.932+0000 7f2095aa2640 1 --2- 172.21.15.117:0/1808471064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2090104a30 0x7f209019a5a0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:24.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.932+0000 7f2095aa2640 1 -- 172.21.15.117:0/1808471064 >> 172.21.15.117:0/1808471064 conn(0x7f2090079770 msgr2=0x7f20900ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:24.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.932+0000 7f2095aa2640 1 -- 172.21.15.117:0/1808471064 shutdown_connections 2024-09-21T06:06:24.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:24.932+0000 7f2095aa2640 1 -- 172.21.15.117:0/1808471064 wait complete. 2024-09-21T06:06:25.835 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:06:26.086 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:26.207 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:25 smithi117 ceph-mon[30947]: pgmap v605: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 409 B/s wr, 11 op/s 2024-09-21T06:06:26.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:25 smithi138 ceph-mon[34391]: pgmap v605: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 409 B/s wr, 11 op/s 2024-09-21T06:06:26.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:25 smithi192 ceph-mon[34700]: pgmap v605: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 409 B/s wr, 11 op/s 2024-09-21T06:06:26.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.631+0000 7f80bae61640 1 -- 172.21.15.117:0/31680504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b4109e20 msgr2=0x7f80b4110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:26.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.631+0000 7f80bae61640 1 --2- 172.21.15.117:0/31680504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b4109e20 0x7f80b4110650 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f80ac009ca0 tx=0x7f80ac02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:06:26.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.631+0000 7f80bae61640 1 -- 172.21.15.117:0/31680504 shutdown_connections 2024-09-21T06:06:26.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.631+0000 7f80bae61640 1 --2- 172.21.15.117:0/31680504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b4109e20 0x7f80b4110650 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:26.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.631+0000 7f80bae61640 1 --2- 172.21.15.117:0/31680504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80b41094d0 0x7f80b41098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:26.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.631+0000 7f80bae61640 1 --2- 172.21.15.117:0/31680504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80b4103b90 0x7f80b4103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:26.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.631+0000 7f80bae61640 1 -- 172.21.15.117:0/31680504 >> 172.21.15.117:0/31680504 conn(0x7f80b40ff620 msgr2=0x7f80b4101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:26.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.631+0000 7f80bae61640 1 -- 172.21.15.117:0/31680504 shutdown_connections 2024-09-21T06:06:26.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.631+0000 7f80bae61640 1 -- 172.21.15.117:0/31680504 wait complete. 2024-09-21T06:06:26.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.632+0000 7f80bae61640 1 Processor -- start 2024-09-21T06:06:26.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.632+0000 7f80bae61640 1 -- start start 2024-09-21T06:06:26.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.632+0000 7f80bae61640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80b4103b90 0x7f80b419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:26.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.632+0000 7f80bae61640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80b41094d0 0x7f80b419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:26.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.632+0000 7f80bae61640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b4109e20 0x7f80b419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:26.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.632+0000 7f80bae61640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f80b4113ae0 con 0x7f80b4109e20 2024-09-21T06:06:26.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.632+0000 7f80bae61640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f80b4113960 con 0x7f80b4103b90 2024-09-21T06:06:26.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.632+0000 7f80bae61640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f80b4113c60 con 0x7f80b41094d0 2024-09-21T06:06:26.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.632+0000 7f80b93fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b4109e20 0x7f80b419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:26.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.633+0000 7f80b93fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b4109e20 0x7f80b419fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55108/0 (socket says 172.21.15.117:55108) 2024-09-21T06:06:26.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.633+0000 7f80b93fe640 1 -- 172.21.15.117:0/3552355495 learned_addr learned my addr 172.21.15.117:0/3552355495 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:26.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.633+0000 7f80b3fff640 1 --2- 172.21.15.117:0/3552355495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80b41094d0 0x7f80b419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:26.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.633+0000 7f80b8bfd640 1 --2- 172.21.15.117:0/3552355495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80b4103b90 0x7f80b419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:26.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.633+0000 7f80b93fe640 1 -- 172.21.15.117:0/3552355495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80b41094d0 msgr2=0x7f80b419a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:26.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.633+0000 7f80b93fe640 1 --2- 172.21.15.117:0/3552355495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80b41094d0 0x7f80b419a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:26.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.633+0000 7f80b93fe640 1 -- 172.21.15.117:0/3552355495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80b4103b90 msgr2=0x7f80b419a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:26.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.633+0000 7f80b93fe640 1 --2- 172.21.15.117:0/3552355495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80b4103b90 0x7f80b419a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:26.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.633+0000 7f80b93fe640 1 -- 172.21.15.117:0/3552355495 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b41a03c0 con 0x7f80b4109e20 2024-09-21T06:06:26.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.633+0000 7f80b93fe640 1 --2- 172.21.15.117:0/3552355495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b4109e20 0x7f80b419fc90 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f80ac005cc0 tx=0x7f80ac02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:26.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.633+0000 7f80b1ffb640 1 -- 172.21.15.117:0/3552355495 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac003bf0 con 0x7f80b4109e20 2024-09-21T06:06:26.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.633+0000 7f80bae61640 1 -- 172.21.15.117:0/3552355495 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80b41a0650 con 0x7f80b4109e20 2024-09-21T06:06:26.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.633+0000 7f80b1ffb640 1 -- 172.21.15.117:0/3552355495 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80ac003d90 con 0x7f80b4109e20 2024-09-21T06:06:26.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.633+0000 7f80b1ffb640 1 -- 172.21.15.117:0/3552355495 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac02fbc0 con 0x7f80b4109e20 2024-09-21T06:06:26.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.633+0000 7f80bae61640 1 -- 172.21.15.117:0/3552355495 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80b41a6e10 con 0x7f80b4109e20 2024-09-21T06:06:26.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.636+0000 7f80c13d5740 1 Processor -- start 2024-09-21T06:06:26.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.636+0000 7f80b1ffb640 1 -- 172.21.15.117:0/3552355495 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f80ac02fd60 con 0x7f80b4109e20 2024-09-21T06:06:26.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.636+0000 7f80c13d5740 1 -- start start 2024-09-21T06:06:26.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.637+0000 7f80c13d5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564326941070 0x564326ce86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:26.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.637+0000 7f80c13d5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5643269181b0 0x564326ce8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:26.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.637+0000 7f80c13d5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564326920390 0x564326ce9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:26.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.637+0000 7f80c13d5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564326c5eea0 con 0x5643269181b0 2024-09-21T06:06:26.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.637+0000 7f80c13d5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564326c4e2d0 con 0x564326941070 2024-09-21T06:06:26.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.637+0000 7f80c13d5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564326c520e0 con 0x564326920390 2024-09-21T06:06:26.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.637+0000 7f80b8bfd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5643269181b0 0x564326ce8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:26.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.637+0000 7f80b8bfd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5643269181b0 0x564326ce8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55118/0 (socket says 172.21.15.117:55118) 2024-09-21T06:06:26.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.637+0000 7f80b8bfd640 1 -- 172.21.15.117:0/3006302289 learned_addr learned my addr 172.21.15.117:0/3006302289 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:26.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.637+0000 7f80b3fff640 1 --2- 172.21.15.117:0/3006302289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564326920390 0x564326ce9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:26.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.637+0000 7f80b3fff640 1 --2- 172.21.15.117:0/3006302289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564326941070 0x564326ce86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:26.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.637+0000 7f80b8bfd640 1 -- 172.21.15.117:0/3006302289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564326920390 msgr2=0x564326ce9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:26.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.637+0000 7f80b8bfd640 1 --2- 172.21.15.117:0/3006302289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564326920390 0x564326ce9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:26.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.637+0000 7f80b8bfd640 1 -- 172.21.15.117:0/3006302289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564326941070 msgr2=0x564326ce86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:26.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.637+0000 7f80b8bfd640 1 --2- 172.21.15.117:0/3006302289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564326941070 0x564326ce86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:26.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.637+0000 7f80b8bfd640 1 -- 172.21.15.117:0/3006302289 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643269ac650 con 0x5643269181b0 2024-09-21T06:06:26.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.637+0000 7f80b1ffb640 1 --2- 172.21.15.117:0/3552355495 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f80a40b6680 0x7f80a40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:26.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.637+0000 7f80b8bfd640 1 --2- 172.21.15.117:0/3006302289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5643269181b0 0x564326ce8be0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f80a800bcf0 tx=0x7f80a800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a77b4960803a438 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:26.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.637+0000 7f80b3fff640 1 --2- 172.21.15.117:0/3552355495 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f80a40b6680 0x7f80a40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:26.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.638+0000 7f80b1ffb640 1 -- 172.21.15.117:0/3552355495 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7f80ac108c60 con 0x7f80b4109e20 2024-09-21T06:06:26.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.638+0000 7f809effd640 1 -- 172.21.15.117:0/3006302289 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8015070 con 0x5643269181b0 2024-09-21T06:06:26.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.638+0000 7f809effd640 1 -- 172.21.15.117:0/3006302289 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80a80118f0 con 0x5643269181b0 2024-09-21T06:06:26.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.638+0000 7f809effd640 1 -- 172.21.15.117:0/3006302289 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8010850 con 0x5643269181b0 2024-09-21T06:06:26.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.639+0000 7f80c13d5740 1 -- 172.21.15.117:0/3006302289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5643269181b0 msgr2=0x564326ce8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:26.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.639+0000 7f80b3fff640 1 --2- 172.21.15.117:0/3552355495 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f80a40b6680 0x7f80a40b8b90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f80b4102e70 tx=0x7f80a00046c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:26.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.639+0000 7f80c13d5740 1 --2- 172.21.15.117:0/3006302289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5643269181b0 0x564326ce8be0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f80a800bcf0 tx=0x7f80a800f730 comp rx=0 tx=0).stop 2024-09-21T06:06:26.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.639+0000 7f80c13d5740 1 -- 172.21.15.117:0/3006302289 shutdown_connections 2024-09-21T06:06:26.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.639+0000 7f80c13d5740 1 --2- 172.21.15.117:0/3006302289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5643269181b0 0x564326ce8be0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:26.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.639+0000 7f80c13d5740 1 --2- 172.21.15.117:0/3006302289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564326920390 0x564326ce9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:26.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.639+0000 7f80c13d5740 1 --2- 172.21.15.117:0/3006302289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564326941070 0x564326ce86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:26.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.639+0000 7f80c13d5740 1 -- 172.21.15.117:0/3006302289 >> 172.21.15.117:0/3006302289 conn(0x564326935460 msgr2=0x564326ce6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:26.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.639+0000 7f80c13d5740 1 -- 172.21.15.117:0/3006302289 shutdown_connections 2024-09-21T06:06:26.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.639+0000 7f80c13d5740 1 -- 172.21.15.117:0/3006302289 wait complete. 2024-09-21T06:06:26.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.639+0000 7f80c13d5740 1 -- start start 2024-09-21T06:06:26.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.639+0000 7f80c13d5740 1 Processor -- start 2024-09-21T06:06:26.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.640+0000 7f80c13d5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5643269181b0 0x564326cf3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:26.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.640+0000 7f80c13d5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564326920390 0x564326cf4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:26.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.640+0000 7f80c13d5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564326941070 0x564326d001a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:26.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.640+0000 7f80c13d5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564326aaf700 con 0x564326941070 2024-09-21T06:06:26.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.640+0000 7f80c13d5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564326be9160 con 0x564326920390 2024-09-21T06:06:26.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.640+0000 7f80c13d5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564326cf6e80 con 0x5643269181b0 2024-09-21T06:06:26.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.640+0000 7f80c13d5740 10 client.0 _ll_register_callbacks cb 0x564326c029e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:26.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.640+0000 7f80b93fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564326941070 0x564326d001a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:26.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.640+0000 7f80b8bfd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5643269181b0 0x564326cf3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:26.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.640+0000 7f80b93fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564326941070 0x564326d001a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55124/0 (socket says 172.21.15.117:55124) 2024-09-21T06:06:26.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.640+0000 7f80b93fe640 1 -- 172.21.15.117:0/2446966730 learned_addr learned my addr 172.21.15.117:0/2446966730 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:26.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.640+0000 7f80b3fff640 1 --2- 172.21.15.117:0/2446966730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564326920390 0x564326cf4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:26.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.640+0000 7f80b93fe640 1 -- 172.21.15.117:0/2446966730 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5643269181b0 msgr2=0x564326cf3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:26.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.640+0000 7f80b93fe640 1 --2- 172.21.15.117:0/2446966730 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5643269181b0 0x564326cf3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:26.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.640+0000 7f80b93fe640 1 -- 172.21.15.117:0/2446966730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564326920390 msgr2=0x564326cf4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:26.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.640+0000 7f80b93fe640 1 --2- 172.21.15.117:0/2446966730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564326920390 0x564326cf4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:26.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.640+0000 7f80b93fe640 1 -- 172.21.15.117:0/2446966730 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564326c16c90 con 0x564326941070 2024-09-21T06:06:26.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.640+0000 7f80b93fe640 1 --2- 172.21.15.117:0/2446966730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564326941070 0x564326d001a0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f80ac0ce800 tx=0x7f80ac0cea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb2e5ae4b410ddc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:26.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.640+0000 7f8093fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:26.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.640+0000 7f8093fff640 1 -- 172.21.15.117:0/2446966730 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac0098d0 con 0x564326941070 2024-09-21T06:06:26.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.641+0000 7f8093fff640 1 -- 172.21.15.117:0/2446966730 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7f80ac0d31d0 con 0x564326941070 2024-09-21T06:06:26.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.641+0000 7f80c13d5740 1 -- 172.21.15.117:0/2446966730 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80a80118f0 con 0x564326941070 2024-09-21T06:06:26.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.641+0000 7f8093fff640 1 -- 172.21.15.117:0/2446966730 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f80ac0f6020 con 0x564326941070 2024-09-21T06:06:26.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.641+0000 7f80c13d5740 10 client.17295 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:26.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.641+0000 7f80c13d5740 1 -- 172.21.15.117:0/2446966730 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564326d02bc0 con 0x564326941070 2024-09-21T06:06:26.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.641+0000 7f8093fff640 1 -- 172.21.15.117:0/2446966730 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80a80118f0 con 0x564326941070 2024-09-21T06:06:26.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.641+0000 7f8093fff640 1 -- 172.21.15.117:0/2446966730 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f80ac0d8090 con 0x564326941070 2024-09-21T06:06:26.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.641+0000 7f80c13d5740 10 client.17295 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:26.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.641+0000 7f80c13d5740 10 client.17295 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:06:26.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.641+0000 7f80c13d5740 20 client.17295 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:26.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.641+0000 7f80c13d5740 1 --2- 172.21.15.117:0/2446966730 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5643268ff9f0 0x564326d06190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:26.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.641+0000 7f80c13d5740 4 client.17295 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:06:26.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.641+0000 7f80c13d5740 1 -- 172.21.15.117:0/2446966730 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564326c60df0 con 0x5643268ff9f0 2024-09-21T06:06:26.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.641+0000 7f80b8bfd640 1 --2- 172.21.15.117:0/2446966730 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5643268ff9f0 0x564326d06190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:26.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.642+0000 7f8093fff640 10 client.17295 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:26.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.642+0000 7f80b8bfd640 1 --2- 172.21.15.117:0/2446966730 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5643268ff9f0 0x564326d06190 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e62421a483aca541 server_cookie=d1349a156410f1cc in_seq=0 out_seq=0 2024-09-21T06:06:26.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.643+0000 7f8093fff640 1 -- 172.21.15.117:0/2446966730 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564326c60df0 con 0x5643268ff9f0 2024-09-21T06:06:26.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.643+0000 7f8093fff640 10 client.17295 handle_command_reply: tid=0 2024-09-21T06:06:26.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.643+0000 7f80c13d5740 1 client.17295 shutdown 2024-09-21T06:06:26.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.643+0000 7f80c13d5740 1 -- 172.21.15.117:0/2446966730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564326941070 msgr2=0x564326d001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:26.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.643+0000 7f80c13d5740 1 --2- 172.21.15.117:0/2446966730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564326941070 0x564326d001a0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f80ac0ce800 tx=0x7f80ac0cea00 comp rx=0 tx=0).stop 2024-09-21T06:06:26.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.643+0000 7f80c13d5740 1 -- 172.21.15.117:0/2446966730 shutdown_connections 2024-09-21T06:06:26.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.643+0000 7f80c13d5740 1 --2- 172.21.15.117:0/2446966730 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5643268ff9f0 0x564326d06190 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:26.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.643+0000 7f8093fff640 0 client.17295 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:26.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.643+0000 7f80c13d5740 1 --2- 172.21.15.117:0/2446966730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564326941070 0x564326d001a0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:26.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.643+0000 7f80c13d5740 1 --2- 172.21.15.117:0/2446966730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564326920390 0x564326cf4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:26.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.643+0000 7f80c13d5740 1 --2- 172.21.15.117:0/2446966730 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5643269181b0 0x564326cf3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:26.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.643+0000 7f80c13d5740 1 -- 172.21.15.117:0/2446966730 >> 172.21.15.117:0/2446966730 conn(0x564326935460 msgr2=0x564326ceb830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:26.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.643+0000 7f80c13d5740 1 -- 172.21.15.117:0/2446966730 shutdown_connections 2024-09-21T06:06:26.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.643+0000 7f80c13d5740 1 -- 172.21.15.117:0/2446966730 wait complete. 2024-09-21T06:06:26.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.643+0000 7f80c13d5740 20 client.17295 trim_cache size 0 max 16384 2024-09-21T06:06:26.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.656+0000 7f80c13d5740 1 Processor -- start 2024-09-21T06:06:26.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.656+0000 7f80c13d5740 1 -- start start 2024-09-21T06:06:26.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.656+0000 7f80c13d5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5643268ff9f0 0x564326cf2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:26.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.656+0000 7f80c13d5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5643269181b0 0x564326ce6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:26.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.656+0000 7f80c13d5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564326920390 0x564326d000a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:26.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.656+0000 7f80c13d5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f80b41af4f0 con 0x5643268ff9f0 2024-09-21T06:06:26.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.656+0000 7f80c13d5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f80b41b1030 con 0x564326920390 2024-09-21T06:06:26.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.656+0000 7f80c13d5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f80b41afbc0 con 0x5643269181b0 2024-09-21T06:06:26.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.656+0000 7f80b8bfd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5643268ff9f0 0x564326cf2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:26.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.656+0000 7f80b8bfd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5643268ff9f0 0x564326cf2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55128/0 (socket says 172.21.15.117:55128) 2024-09-21T06:06:26.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.656+0000 7f80b8bfd640 1 -- 172.21.15.117:0/974779361 learned_addr learned my addr 172.21.15.117:0/974779361 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:26.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.656+0000 7f80b8bfd640 1 -- 172.21.15.117:0/974779361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5643269181b0 msgr2=0x564326ce6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:26.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.656+0000 7f80b8bfd640 1 --2- 172.21.15.117:0/974779361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5643269181b0 0x564326ce6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:26.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.656+0000 7f80b8bfd640 1 -- 172.21.15.117:0/974779361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564326920390 msgr2=0x564326d000a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:26.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.656+0000 7f80b8bfd640 1 --2- 172.21.15.117:0/974779361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564326920390 0x564326d000a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:26.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.656+0000 7f80b8bfd640 1 -- 172.21.15.117:0/974779361 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564326cf6500 con 0x5643268ff9f0 2024-09-21T06:06:26.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.657+0000 7f80b8bfd640 1 --2- 172.21.15.117:0/974779361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5643268ff9f0 0x564326cf2cb0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f80a800bc10 tx=0x7f80a8010510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f25675df4b533bd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:26.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.657+0000 7f809e7fc640 1 -- 172.21.15.117:0/974779361 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8031070 con 0x5643268ff9f0 2024-09-21T06:06:26.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.657+0000 7f809e7fc640 1 -- 172.21.15.117:0/974779361 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80a8004640 con 0x5643268ff9f0 2024-09-21T06:06:26.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.657+0000 7f809e7fc640 1 -- 172.21.15.117:0/974779361 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a8015040 con 0x5643268ff9f0 2024-09-21T06:06:26.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.658+0000 7f80c13d5740 1 -- 172.21.15.117:0/974779361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5643268ff9f0 msgr2=0x564326cf2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:26.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.658+0000 7f80c13d5740 1 --2- 172.21.15.117:0/974779361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5643268ff9f0 0x564326cf2cb0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f80a800bc10 tx=0x7f80a8010510 comp rx=0 tx=0).stop 2024-09-21T06:06:26.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.658+0000 7f80c13d5740 1 -- 172.21.15.117:0/974779361 shutdown_connections 2024-09-21T06:06:26.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.658+0000 7f80c13d5740 1 --2- 172.21.15.117:0/974779361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564326920390 0x564326d000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:26.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.658+0000 7f80c13d5740 1 --2- 172.21.15.117:0/974779361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5643269181b0 0x564326ce6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:26.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.658+0000 7f80c13d5740 1 --2- 172.21.15.117:0/974779361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5643268ff9f0 0x564326cf2cb0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:26.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.658+0000 7f80c13d5740 1 -- 172.21.15.117:0/974779361 >> 172.21.15.117:0/974779361 conn(0x564326935460 msgr2=0x564326ce5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:26.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.658+0000 7f80c13d5740 1 -- 172.21.15.117:0/974779361 shutdown_connections 2024-09-21T06:06:26.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.658+0000 7f80c13d5740 1 -- 172.21.15.117:0/974779361 wait complete. 2024-09-21T06:06:26.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.658+0000 7f80c13d5740 1 -- start start 2024-09-21T06:06:26.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.658+0000 7f80c13d5740 1 Processor -- start 2024-09-21T06:06:26.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.659+0000 7f80c13d5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5643268ff9f0 0x564326cf8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:26.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.659+0000 7f80c13d5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643269181b0 0x564326cf8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:26.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.659+0000 7f80c13d5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564326920390 0x564326cf8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:26.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.659+0000 7f80c13d5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f80b41b0960 con 0x564326920390 2024-09-21T06:06:26.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.659+0000 7f80c13d5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f80b41b0290 con 0x5643269181b0 2024-09-21T06:06:26.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.659+0000 7f80c13d5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f80b41ad4f0 con 0x5643268ff9f0 2024-09-21T06:06:26.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.659+0000 7f80c13d5740 10 client.0 _ll_register_callbacks cb 0x7f80ac0319f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:26.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.659+0000 7f80b93fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564326920390 0x564326cf8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:26.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.659+0000 7f80b93fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564326920390 0x564326cf8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55136/0 (socket says 172.21.15.117:55136) 2024-09-21T06:06:26.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.659+0000 7f80b93fe640 1 -- 172.21.15.117:0/3347925079 learned_addr learned my addr 172.21.15.117:0/3347925079 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:26.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.659+0000 7f80b8bfd640 1 --2- 172.21.15.117:0/3347925079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5643268ff9f0 0x564326cf8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:26.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.659+0000 7f80b3fff640 1 --2- 172.21.15.117:0/3347925079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643269181b0 0x564326cf8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:26.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.659+0000 7f80b93fe640 1 -- 172.21.15.117:0/3347925079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5643268ff9f0 msgr2=0x564326cf8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:26.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.659+0000 7f80b93fe640 1 --2- 172.21.15.117:0/3347925079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5643268ff9f0 0x564326cf8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:26.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.659+0000 7f80b93fe640 1 -- 172.21.15.117:0/3347925079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643269181b0 msgr2=0x564326cf8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:26.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.659+0000 7f80b93fe640 1 --2- 172.21.15.117:0/3347925079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643269181b0 0x564326cf8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:26.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.659+0000 7f80b93fe640 1 -- 172.21.15.117:0/3347925079 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564326d6fc80 con 0x564326920390 2024-09-21T06:06:26.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.659+0000 7f80b93fe640 1 --2- 172.21.15.117:0/3347925079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564326920390 0x564326cf8c90 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f80ac0cea90 tx=0x7f80ac004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc7cd94eff70805 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:26.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.659+0000 7f8093fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:26.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.659+0000 7f8093fff640 1 -- 172.21.15.117:0/3347925079 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ac0ce270 con 0x564326920390 2024-09-21T06:06:26.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.659+0000 7f8093fff640 1 -- 172.21.15.117:0/3347925079 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7f80ac106d50 con 0x564326920390 2024-09-21T06:06:26.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.659+0000 7f80c13d5740 1 -- 172.21.15.117:0/3347925079 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80a8004640 con 0x564326920390 2024-09-21T06:06:26.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.660+0000 7f8093fff640 1 -- 172.21.15.117:0/3347925079 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f80ac0d33e0 con 0x564326920390 2024-09-21T06:06:26.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.660+0000 7f80c13d5740 10 client.17301 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:26.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.660+0000 7f80c13d5740 1 -- 172.21.15.117:0/3347925079 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564326d6fc80 con 0x564326920390 2024-09-21T06:06:26.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.660+0000 7f8093fff640 1 -- 172.21.15.117:0/3347925079 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80a8004640 con 0x564326920390 2024-09-21T06:06:26.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.660+0000 7f8093fff640 1 -- 172.21.15.117:0/3347925079 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f80ac0a8b80 con 0x564326920390 2024-09-21T06:06:26.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.660+0000 7f80c13d5740 10 client.17301 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:26.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.660+0000 7f80c13d5740 10 client.17301 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:06:26.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.660+0000 7f80c13d5740 20 client.17301 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:26.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.660+0000 7f80c13d5740 1 --2- 172.21.15.117:0/3347925079 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x564326941070 0x564326d73310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:26.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.660+0000 7f80c13d5740 4 client.17301 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:06:26.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.660+0000 7f80c13d5740 1 -- 172.21.15.117:0/3347925079 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564326ce19e0 con 0x564326941070 2024-09-21T06:06:26.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.660+0000 7f80b8bfd640 1 --2- 172.21.15.117:0/3347925079 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x564326941070 0x564326d73310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:26.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.661+0000 7f8093fff640 10 client.17301 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:26.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.661+0000 7f80b8bfd640 1 --2- 172.21.15.117:0/3347925079 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x564326941070 0x564326d73310 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d229bcca1bc5664 server_cookie=e42595ba2e7f31ec in_seq=0 out_seq=0 2024-09-21T06:06:26.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.661+0000 7f8093fff640 1 -- 172.21.15.117:0/3347925079 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564326ce19e0 con 0x564326941070 2024-09-21T06:06:26.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.661+0000 7f8093fff640 10 client.17301 handle_command_reply: tid=0 2024-09-21T06:06:26.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.661+0000 7f80c13d5740 1 client.17301 shutdown 2024-09-21T06:06:26.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.661+0000 7f80c13d5740 1 -- 172.21.15.117:0/3347925079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564326920390 msgr2=0x564326cf8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:26.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.661+0000 7f80c13d5740 1 --2- 172.21.15.117:0/3347925079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564326920390 0x564326cf8c90 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f80ac0cea90 tx=0x7f80ac004840 comp rx=0 tx=0).stop 2024-09-21T06:06:26.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.661+0000 7f80c13d5740 1 -- 172.21.15.117:0/3347925079 shutdown_connections 2024-09-21T06:06:26.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.661+0000 7f80c13d5740 1 --2- 172.21.15.117:0/3347925079 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x564326941070 0x564326d73310 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:26.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.661+0000 7f80c13d5740 1 --2- 172.21.15.117:0/3347925079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564326920390 0x564326cf8c90 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:26.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.661+0000 7f80c13d5740 1 --2- 172.21.15.117:0/3347925079 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5643269181b0 0x564326cf8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:26.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.661+0000 7f80c13d5740 1 --2- 172.21.15.117:0/3347925079 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5643268ff9f0 0x564326cf8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:26.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.661+0000 7f80c13d5740 1 -- 172.21.15.117:0/3347925079 >> 172.21.15.117:0/3347925079 conn(0x564326935460 msgr2=0x564326ce5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:26.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.661+0000 7f8093fff640 0 client.17301 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:26.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.661+0000 7f80c13d5740 1 -- 172.21.15.117:0/3347925079 shutdown_connections 2024-09-21T06:06:26.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.661+0000 7f80c13d5740 1 -- 172.21.15.117:0/3347925079 wait complete. 2024-09-21T06:06:26.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.662+0000 7f80c13d5740 20 client.17301 trim_cache size 0 max 16384 2024-09-21T06:06:26.670 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:06:26.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.662+0000 7f80bae61640 1 -- 172.21.15.117:0/3552355495 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f80a40b6680 msgr2=0x7f80a40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:26.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.662+0000 7f80bae61640 1 --2- 172.21.15.117:0/3552355495 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f80a40b6680 0x7f80a40b8b90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f80b4102e70 tx=0x7f80a00046c0 comp rx=0 tx=0).stop 2024-09-21T06:06:26.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.662+0000 7f80bae61640 1 -- 172.21.15.117:0/3552355495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b4109e20 msgr2=0x7f80b419fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:26.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.662+0000 7f80bae61640 1 --2- 172.21.15.117:0/3552355495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b4109e20 0x7f80b419fc90 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f80ac005cc0 tx=0x7f80ac02f380 comp rx=0 tx=0).stop 2024-09-21T06:06:26.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.662+0000 7f80bae61640 1 -- 172.21.15.117:0/3552355495 shutdown_connections 2024-09-21T06:06:26.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.662+0000 7f80bae61640 1 --2- 172.21.15.117:0/3552355495 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f80a40b6680 0x7f80a40b8b90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:26.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.662+0000 7f80bae61640 1 --2- 172.21.15.117:0/3552355495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80b4109e20 0x7f80b419fc90 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:26.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.662+0000 7f80bae61640 1 --2- 172.21.15.117:0/3552355495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80b41094d0 0x7f80b419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:26.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.662+0000 7f80bae61640 1 --2- 172.21.15.117:0/3552355495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80b4103b90 0x7f80b419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:26.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.662+0000 7f80bae61640 1 -- 172.21.15.117:0/3552355495 >> 172.21.15.117:0/3552355495 conn(0x7f80b40ff620 msgr2=0x7f80b4106a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:26.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.662+0000 7f80bae61640 1 -- 172.21.15.117:0/3552355495 shutdown_connections 2024-09-21T06:06:26.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:26.662+0000 7f80bae61640 1 -- 172.21.15.117:0/3552355495 wait complete. 2024-09-21T06:06:28.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:27 smithi117 ceph-mon[30947]: pgmap v606: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 409 B/s wr, 10 op/s 2024-09-21T06:06:28.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:27 smithi138 ceph-mon[34391]: pgmap v606: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 409 B/s wr, 10 op/s 2024-09-21T06:06:28.425 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:06:28.425 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:06:28.479 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:27 smithi192 ceph-mon[34700]: pgmap v606: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 409 B/s wr, 10 op/s 2024-09-21T06:06:28.684 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:29.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.184+0000 7fa9d2e2c640 1 -- 172.21.15.117:0/1933804188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9cc108530 msgr2=0x7fa9cc10ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:29.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.184+0000 7fa9d2e2c640 1 --2- 172.21.15.117:0/1933804188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9cc108530 0x7fa9cc10ed60 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4009ca0 tx=0x7fa9c402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.184+0000 7fa9d2e2c640 1 -- 172.21.15.117:0/1933804188 shutdown_connections 2024-09-21T06:06:29.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.184+0000 7fa9d2e2c640 1 --2- 172.21.15.117:0/1933804188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9cc108530 0x7fa9cc10ed60 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.184+0000 7fa9d2e2c640 1 --2- 172.21.15.117:0/1933804188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9cc107be0 0x7fa9cc107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.184+0000 7fa9d2e2c640 1 --2- 172.21.15.117:0/1933804188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9cc1022a0 0x7fa9cc1026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.184+0000 7fa9d2e2c640 1 -- 172.21.15.117:0/1933804188 >> 172.21.15.117:0/1933804188 conn(0x7fa9cc079ec0 msgr2=0x7fa9cc07a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:29.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.185+0000 7fa9d2e2c640 1 -- 172.21.15.117:0/1933804188 shutdown_connections 2024-09-21T06:06:29.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.185+0000 7fa9d2e2c640 1 -- 172.21.15.117:0/1933804188 wait complete. 2024-09-21T06:06:29.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.185+0000 7fa9d2e2c640 1 Processor -- start 2024-09-21T06:06:29.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.185+0000 7fa9d2e2c640 1 -- start start 2024-09-21T06:06:29.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.186+0000 7fa9d2e2c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9cc1022a0 0x7fa9cc1993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:29.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.186+0000 7fa9d2e2c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9cc107be0 0x7fa9cc199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:29.194 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.186+0000 7fa9d2e2c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9cc108530 0x7fa9cc19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:29.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.186+0000 7fa9d2e2c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa9cc112a70 con 0x7fa9cc1022a0 2024-09-21T06:06:29.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.186+0000 7fa9d2e2c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa9cc1128f0 con 0x7fa9cc108530 2024-09-21T06:06:29.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.186+0000 7fa9d2e2c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa9cc112bf0 con 0x7fa9cc107be0 2024-09-21T06:06:29.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.186+0000 7fa9d0bc8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9cc1022a0 0x7fa9cc1993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:29.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.186+0000 7fa9d0bc8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9cc1022a0 0x7fa9cc1993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55156/0 (socket says 172.21.15.117:55156) 2024-09-21T06:06:29.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.186+0000 7fa9d0bc8640 1 -- 172.21.15.117:0/433286569 learned_addr learned my addr 172.21.15.117:0/433286569 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:29.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.186+0000 7fa9cbfff640 1 --2- 172.21.15.117:0/433286569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9cc107be0 0x7fa9cc199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:29.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.186+0000 7fa9d13c9640 1 --2- 172.21.15.117:0/433286569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9cc108530 0x7fa9cc19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:29.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.186+0000 7fa9d0bc8640 1 -- 172.21.15.117:0/433286569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9cc107be0 msgr2=0x7fa9cc199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:29.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.186+0000 7fa9d0bc8640 1 --2- 172.21.15.117:0/433286569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9cc107be0 0x7fa9cc199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.186+0000 7fa9d0bc8640 1 -- 172.21.15.117:0/433286569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9cc108530 msgr2=0x7fa9cc19ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:29.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.186+0000 7fa9d0bc8640 1 --2- 172.21.15.117:0/433286569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9cc108530 0x7fa9cc19ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.186+0000 7fa9d0bc8640 1 -- 172.21.15.117:0/433286569 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9cc19f310 con 0x7fa9cc1022a0 2024-09-21T06:06:29.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.186+0000 7fa9cbfff640 1 --2- 172.21.15.117:0/433286569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9cc107be0 0x7fa9cc199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:06:29.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.186+0000 7fa9d0bc8640 1 --2- 172.21.15.117:0/433286569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9cc1022a0 0x7fa9cc1993a0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fa9c000d950 tx=0x7fa9c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:29.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.186+0000 7fa9c9ffb640 1 -- 172.21.15.117:0/433286569 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c0017070 con 0x7fa9cc1022a0 2024-09-21T06:06:29.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.186+0000 7fa9c9ffb640 1 -- 172.21.15.117:0/433286569 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9c0009c30 con 0x7fa9cc1022a0 2024-09-21T06:06:29.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.186+0000 7fa9d2e2c640 1 -- 172.21.15.117:0/433286569 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9cc19f600 con 0x7fa9cc1022a0 2024-09-21T06:06:29.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.186+0000 7fa9c9ffb640 1 -- 172.21.15.117:0/433286569 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c000f620 con 0x7fa9cc1022a0 2024-09-21T06:06:29.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.187+0000 7fa9d2e2c640 1 -- 172.21.15.117:0/433286569 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9cc1a5d60 con 0x7fa9cc1022a0 2024-09-21T06:06:29.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.189+0000 7fa9c9ffb640 1 -- 172.21.15.117:0/433286569 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fa9c0010460 con 0x7fa9cc1022a0 2024-09-21T06:06:29.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.189+0000 7fa9d93a0740 1 Processor -- start 2024-09-21T06:06:29.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.189+0000 7fa9d93a0740 1 -- start start 2024-09-21T06:06:29.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.190+0000 7fa9d93a0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae15dc5350 0x55ae1616c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:29.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.190+0000 7fa9d93a0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae15d9c490 0x55ae1616c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:29.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.190+0000 7fa9d93a0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae15da4670 0x55ae1616ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:29.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.190+0000 7fa9d93a0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ae160e2e60 con 0x55ae15d9c490 2024-09-21T06:06:29.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.190+0000 7fa9d93a0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ae160d2290 con 0x55ae15da4670 2024-09-21T06:06:29.197 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.190+0000 7fa9d93a0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55ae160d60a0 con 0x55ae15dc5350 2024-09-21T06:06:29.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.190+0000 7fa9d0bc8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae15d9c490 0x55ae1616c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:29.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.190+0000 7fa9d0bc8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae15d9c490 0x55ae1616c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55170/0 (socket says 172.21.15.117:55170) 2024-09-21T06:06:29.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.190+0000 7fa9d0bc8640 1 -- 172.21.15.117:0/365141676 learned_addr learned my addr 172.21.15.117:0/365141676 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:29.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.190+0000 7fa9c9ffb640 1 --2- 172.21.15.117:0/433286569 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa9bc0b6680 0x7fa9bc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:29.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.190+0000 7fa9d13c9640 1 --2- 172.21.15.117:0/365141676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae15dc5350 0x55ae1616c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:29.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.190+0000 7fa9cbfff640 1 --2- 172.21.15.117:0/365141676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae15da4670 0x55ae1616ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:29.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.190+0000 7fa9d13c9640 1 --2- 172.21.15.117:0/433286569 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa9bc0b6680 0x7fa9bc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:29.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.190+0000 7fa9c9ffb640 1 -- 172.21.15.117:0/433286569 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fa9c00e73f0 con 0x7fa9cc1022a0 2024-09-21T06:06:29.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.190+0000 7fa9d0bc8640 1 -- 172.21.15.117:0/365141676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae15dc5350 msgr2=0x55ae1616c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:29.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.190+0000 7fa9d0bc8640 1 --2- 172.21.15.117:0/365141676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae15dc5350 0x55ae1616c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.190+0000 7fa9d0bc8640 1 -- 172.21.15.117:0/365141676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae15da4670 msgr2=0x55ae1616ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:29.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.190+0000 7fa9d0bc8640 1 --2- 172.21.15.117:0/365141676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae15da4670 0x55ae1616ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.190+0000 7fa9d0bc8640 1 -- 172.21.15.117:0/365141676 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae15e30930 con 0x55ae15d9c490 2024-09-21T06:06:29.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.191+0000 7fa9d0bc8640 1 --2- 172.21.15.117:0/365141676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae15d9c490 0x55ae1616c970 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fa9c00aa500 tx=0x7fa9c0010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba6b91d46ebab156 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:29.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.191+0000 7fa9d13c9640 1 --2- 172.21.15.117:0/433286569 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa9bc0b6680 0x7fa9bc0b8b90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4004af0 tx=0x7fa9c40095c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:29.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.191+0000 7fa9b6ffd640 1 -- 172.21.15.117:0/365141676 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c0020070 con 0x55ae15d9c490 2024-09-21T06:06:29.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.191+0000 7fa9b6ffd640 1 -- 172.21.15.117:0/365141676 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9c00acda0 con 0x55ae15d9c490 2024-09-21T06:06:29.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.191+0000 7fa9b6ffd640 1 -- 172.21.15.117:0/365141676 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c00b1d70 con 0x55ae15d9c490 2024-09-21T06:06:29.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.192+0000 7fa9d93a0740 1 -- 172.21.15.117:0/365141676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae15d9c490 msgr2=0x55ae1616c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:29.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.192+0000 7fa9d93a0740 1 --2- 172.21.15.117:0/365141676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae15d9c490 0x55ae1616c970 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fa9c00aa500 tx=0x7fa9c0010cb0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.192+0000 7fa9d93a0740 1 -- 172.21.15.117:0/365141676 shutdown_connections 2024-09-21T06:06:29.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.192+0000 7fa9d93a0740 1 --2- 172.21.15.117:0/365141676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae15d9c490 0x55ae1616c970 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.192+0000 7fa9d93a0740 1 --2- 172.21.15.117:0/365141676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae15da4670 0x55ae1616ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.192+0000 7fa9d93a0740 1 --2- 172.21.15.117:0/365141676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae15dc5350 0x55ae1616c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.192+0000 7fa9d93a0740 1 -- 172.21.15.117:0/365141676 >> 172.21.15.117:0/365141676 conn(0x55ae15db9740 msgr2=0x55ae1616a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:29.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.192+0000 7fa9d93a0740 1 -- 172.21.15.117:0/365141676 shutdown_connections 2024-09-21T06:06:29.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.192+0000 7fa9d93a0740 1 -- 172.21.15.117:0/365141676 wait complete. 2024-09-21T06:06:29.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.192+0000 7fa9d93a0740 1 -- start start 2024-09-21T06:06:29.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.192+0000 7fa9d93a0740 1 Processor -- start 2024-09-21T06:06:29.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.193+0000 7fa9d93a0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae15d9c490 0x55ae161780d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:29.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.193+0000 7fa9d93a0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae15da4670 0x55ae161786b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:29.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.193+0000 7fa9d93a0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae15dc5350 0x55ae16183ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:29.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.193+0000 7fa9d93a0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ae15f33890 con 0x55ae15da4670 2024-09-21T06:06:29.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.193+0000 7fa9d93a0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ae1606cd40 con 0x55ae15d9c490 2024-09-21T06:06:29.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.193+0000 7fa9d93a0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55ae1617ab50 con 0x55ae15dc5350 2024-09-21T06:06:29.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.193+0000 7fa9d93a0740 10 client.0 _ll_register_callbacks cb 0x55ae160d0c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:29.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.193+0000 7fa9cbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae15da4670 0x55ae161786b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:29.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.193+0000 7fa9cbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae15da4670 0x55ae161786b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55180/0 (socket says 172.21.15.117:55180) 2024-09-21T06:06:29.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.193+0000 7fa9cbfff640 1 -- 172.21.15.117:0/2496105983 learned_addr learned my addr 172.21.15.117:0/2496105983 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:29.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.193+0000 7fa9d0bc8640 1 --2- 172.21.15.117:0/2496105983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae15d9c490 0x55ae161780d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:29.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.193+0000 7fa9d13c9640 1 --2- 172.21.15.117:0/2496105983 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae15dc5350 0x55ae16183ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:29.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.193+0000 7fa9cbfff640 1 -- 172.21.15.117:0/2496105983 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae15dc5350 msgr2=0x55ae16183ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:29.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.193+0000 7fa9cbfff640 1 --2- 172.21.15.117:0/2496105983 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae15dc5350 0x55ae16183ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.193+0000 7fa9cbfff640 1 -- 172.21.15.117:0/2496105983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae15d9c490 msgr2=0x55ae161780d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:29.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.193+0000 7fa9cbfff640 1 --2- 172.21.15.117:0/2496105983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae15d9c490 0x55ae161780d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.193+0000 7fa9cbfff640 1 -- 172.21.15.117:0/2496105983 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae16184510 con 0x55ae15da4670 2024-09-21T06:06:29.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.193+0000 7fa9d0bc8640 1 --2- 172.21.15.117:0/2496105983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae15d9c490 0x55ae161780d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:06:29.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.194+0000 7fa9cbfff640 1 --2- 172.21.15.117:0/2496105983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae15da4670 0x55ae161786b0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fa9b800b900 tx=0x7fa9b800bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a9fc142922cb55 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:29.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.194+0000 7fa9abfff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:29.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.194+0000 7fa9abfff640 1 -- 172.21.15.117:0/2496105983 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b801b640 con 0x55ae15da4670 2024-09-21T06:06:29.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.194+0000 7fa9abfff640 1 -- 172.21.15.117:0/2496105983 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fa9b801b7e0 con 0x55ae15da4670 2024-09-21T06:06:29.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.194+0000 7fa9d93a0740 1 -- 172.21.15.117:0/2496105983 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9c00acda0 con 0x55ae15da4670 2024-09-21T06:06:29.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.194+0000 7fa9abfff640 1 -- 172.21.15.117:0/2496105983 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa9b801b9e0 con 0x55ae15da4670 2024-09-21T06:06:29.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.194+0000 7fa9d93a0740 10 client.17313 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:29.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.194+0000 7fa9d93a0740 1 -- 172.21.15.117:0/2496105983 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ae16186ef0 con 0x55ae15da4670 2024-09-21T06:06:29.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.194+0000 7fa9abfff640 1 -- 172.21.15.117:0/2496105983 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9c00acda0 con 0x55ae15da4670 2024-09-21T06:06:29.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.194+0000 7fa9abfff640 1 -- 172.21.15.117:0/2496105983 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa9b8019020 con 0x55ae15da4670 2024-09-21T06:06:29.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.194+0000 7fa9d93a0740 10 client.17313 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:29.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.195+0000 7fa9d93a0740 10 client.17313 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:06:29.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.195+0000 7fa9d93a0740 20 client.17313 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:29.205 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.195+0000 7fa9d93a0740 1 --2- 172.21.15.117:0/2496105983 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ae15d83cd0 0x55ae1618a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:29.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.195+0000 7fa9d93a0740 4 client.17313 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:06:29.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.195+0000 7fa9d93a0740 1 -- 172.21.15.117:0/2496105983 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae160e4db0 con 0x55ae15d83cd0 2024-09-21T06:06:29.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.195+0000 7fa9d0bc8640 1 --2- 172.21.15.117:0/2496105983 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ae15d83cd0 0x55ae1618a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:29.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.195+0000 7fa9abfff640 10 client.17313 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:29.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.195+0000 7fa9d0bc8640 1 --2- 172.21.15.117:0/2496105983 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ae15d83cd0 0x55ae1618a430 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8020d1b4d4ce4aa5 server_cookie=6cc0ab72876ede65 in_seq=0 out_seq=0 2024-09-21T06:06:29.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.196+0000 7fa9abfff640 1 -- 172.21.15.117:0/2496105983 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ae160e4db0 con 0x55ae15d83cd0 2024-09-21T06:06:29.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.196+0000 7fa9abfff640 10 client.17313 handle_command_reply: tid=0 2024-09-21T06:06:29.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.196+0000 7fa9d93a0740 1 client.17313 shutdown 2024-09-21T06:06:29.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.196+0000 7fa9d93a0740 1 -- 172.21.15.117:0/2496105983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae15da4670 msgr2=0x55ae161786b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:29.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.196+0000 7fa9d93a0740 1 --2- 172.21.15.117:0/2496105983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae15da4670 0x55ae161786b0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fa9b800b900 tx=0x7fa9b800bdc0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.196+0000 7fa9d93a0740 1 -- 172.21.15.117:0/2496105983 shutdown_connections 2024-09-21T06:06:29.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.196+0000 7fa9d93a0740 1 --2- 172.21.15.117:0/2496105983 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ae15d83cd0 0x55ae1618a430 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.196+0000 7fa9d93a0740 1 --2- 172.21.15.117:0/2496105983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae15da4670 0x55ae161786b0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.196+0000 7fa9d93a0740 1 --2- 172.21.15.117:0/2496105983 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae15dc5350 0x55ae16183ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.196+0000 7fa9d93a0740 1 --2- 172.21.15.117:0/2496105983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae15d9c490 0x55ae161780d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.196+0000 7fa9d93a0740 1 -- 172.21.15.117:0/2496105983 >> 172.21.15.117:0/2496105983 conn(0x55ae15db9740 msgr2=0x55ae1616d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:29.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.196+0000 7fa9abfff640 0 client.17313 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:29.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.196+0000 7fa9d93a0740 1 -- 172.21.15.117:0/2496105983 shutdown_connections 2024-09-21T06:06:29.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.197+0000 7fa9d93a0740 1 -- 172.21.15.117:0/2496105983 wait complete. 2024-09-21T06:06:29.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.197+0000 7fa9d93a0740 20 client.17313 trim_cache size 0 max 16384 2024-09-21T06:06:29.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.214+0000 7fa9d93a0740 1 Processor -- start 2024-09-21T06:06:29.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.214+0000 7fa9d93a0740 1 -- start start 2024-09-21T06:06:29.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.214+0000 7fa9d93a0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae15d9c490 0x55ae16176750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:29.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.214+0000 7fa9d93a0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae15da4670 0x55ae16176c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:29.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.214+0000 7fa9d93a0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae15dc5350 0x55ae1616d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:29.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.214+0000 7fa9d93a0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa9cc1ae4d0 con 0x55ae15d9c490 2024-09-21T06:06:29.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.214+0000 7fa9d93a0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa9cc1b0010 con 0x55ae15dc5350 2024-09-21T06:06:29.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.214+0000 7fa9d93a0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa9cc1aeba0 con 0x55ae15da4670 2024-09-21T06:06:29.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.214+0000 7fa9d0bc8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae15d9c490 0x55ae16176750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:29.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.214+0000 7fa9d0bc8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae15d9c490 0x55ae16176750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55190/0 (socket says 172.21.15.117:55190) 2024-09-21T06:06:29.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.214+0000 7fa9d0bc8640 1 -- 172.21.15.117:0/2589879438 learned_addr learned my addr 172.21.15.117:0/2589879438 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:29.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.214+0000 7fa9cbfff640 1 --2- 172.21.15.117:0/2589879438 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae15da4670 0x55ae16176c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:29.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.214+0000 7fa9d13c9640 1 --2- 172.21.15.117:0/2589879438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae15dc5350 0x55ae1616d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:29.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.214+0000 7fa9d0bc8640 1 -- 172.21.15.117:0/2589879438 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae15da4670 msgr2=0x55ae16176c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:29.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.214+0000 7fa9d0bc8640 1 --2- 172.21.15.117:0/2589879438 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae15da4670 0x55ae16176c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.214+0000 7fa9d0bc8640 1 -- 172.21.15.117:0/2589879438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae15dc5350 msgr2=0x55ae1616d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:29.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.214+0000 7fa9d0bc8640 1 --2- 172.21.15.117:0/2589879438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae15dc5350 0x55ae1616d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.214+0000 7fa9d0bc8640 1 -- 172.21.15.117:0/2589879438 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae1616f290 con 0x55ae15d9c490 2024-09-21T06:06:29.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.214+0000 7fa9d13c9640 1 --2- 172.21.15.117:0/2589879438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae15dc5350 0x55ae1616d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:06:29.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.215+0000 7fa9d0bc8640 1 --2- 172.21.15.117:0/2589879438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae15d9c490 0x55ae16176750 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fa9c00ad220 tx=0x7fa9c000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7298922732e5f210 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:29.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.215+0000 7fa9b67fc640 1 -- 172.21.15.117:0/2589879438 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c0020040 con 0x55ae15d9c490 2024-09-21T06:06:29.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.215+0000 7fa9b67fc640 1 -- 172.21.15.117:0/2589879438 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9c00b6cb0 con 0x55ae15d9c490 2024-09-21T06:06:29.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.215+0000 7fa9b67fc640 1 -- 172.21.15.117:0/2589879438 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c0010750 con 0x55ae15d9c490 2024-09-21T06:06:29.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.215+0000 7fa9d93a0740 1 -- 172.21.15.117:0/2589879438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae15d9c490 msgr2=0x55ae16176750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:29.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.215+0000 7fa9d93a0740 1 --2- 172.21.15.117:0/2589879438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae15d9c490 0x55ae16176750 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fa9c00ad220 tx=0x7fa9c000bf60 comp rx=0 tx=0).stop 2024-09-21T06:06:29.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.216+0000 7fa9d93a0740 1 -- 172.21.15.117:0/2589879438 shutdown_connections 2024-09-21T06:06:29.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.216+0000 7fa9d93a0740 1 --2- 172.21.15.117:0/2589879438 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae15dc5350 0x55ae1616d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.216+0000 7fa9d93a0740 1 --2- 172.21.15.117:0/2589879438 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae15da4670 0x55ae16176c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.216+0000 7fa9d93a0740 1 --2- 172.21.15.117:0/2589879438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae15d9c490 0x55ae16176750 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.216+0000 7fa9d93a0740 1 -- 172.21.15.117:0/2589879438 >> 172.21.15.117:0/2589879438 conn(0x55ae15db9740 msgr2=0x55ae16173010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:29.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.216+0000 7fa9d93a0740 1 -- 172.21.15.117:0/2589879438 shutdown_connections 2024-09-21T06:06:29.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.216+0000 7fa9d93a0740 1 -- 172.21.15.117:0/2589879438 wait complete. 2024-09-21T06:06:29.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.216+0000 7fa9d93a0740 1 -- start start 2024-09-21T06:06:29.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.216+0000 7fa9d93a0740 1 Processor -- start 2024-09-21T06:06:29.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.216+0000 7fa9d93a0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae15d9c490 0x55ae16180380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:29.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.216+0000 7fa9d93a0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae15da4670 0x55ae1617b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:29.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.216+0000 7fa9d93a0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae15dc5350 0x55ae1617bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:29.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.217+0000 7fa9d93a0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa9cc1af940 con 0x55ae15d9c490 2024-09-21T06:06:29.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.217+0000 7fa9d93a0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa9cc1af270 con 0x55ae15da4670 2024-09-21T06:06:29.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.217+0000 7fa9d93a0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa9cc1ac4d0 con 0x55ae15dc5350 2024-09-21T06:06:29.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.217+0000 7fa9d93a0740 10 client.0 _ll_register_callbacks cb 0x7fa9b8004330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:29.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.217+0000 7fa9d0bc8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae15d9c490 0x55ae16180380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:29.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.217+0000 7fa9d0bc8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae15d9c490 0x55ae16180380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55202/0 (socket says 172.21.15.117:55202) 2024-09-21T06:06:29.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.217+0000 7fa9d0bc8640 1 -- 172.21.15.117:0/3474214234 learned_addr learned my addr 172.21.15.117:0/3474214234 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:29.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.217+0000 7fa9d13c9640 1 --2- 172.21.15.117:0/3474214234 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae15dc5350 0x55ae1617bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:29.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.217+0000 7fa9cbfff640 1 --2- 172.21.15.117:0/3474214234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae15da4670 0x55ae1617b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:29.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.217+0000 7fa9d0bc8640 1 -- 172.21.15.117:0/3474214234 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae15dc5350 msgr2=0x55ae1617bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:29.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.217+0000 7fa9d0bc8640 1 --2- 172.21.15.117:0/3474214234 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae15dc5350 0x55ae1617bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.217+0000 7fa9d0bc8640 1 -- 172.21.15.117:0/3474214234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae15da4670 msgr2=0x55ae1617b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:29.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.217+0000 7fa9d0bc8640 1 --2- 172.21.15.117:0/3474214234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae15da4670 0x55ae1617b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.217+0000 7fa9d0bc8640 1 -- 172.21.15.117:0/3474214234 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae1616a860 con 0x55ae15d9c490 2024-09-21T06:06:29.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.217+0000 7fa9d0bc8640 1 --2- 172.21.15.117:0/3474214234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae15d9c490 0x55ae16180380 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fa9c00058c0 tx=0x7fa9c00ac8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6f8ac2c04307f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:29.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.217+0000 7fa9abfff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:29.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.217+0000 7fa9abfff640 1 -- 172.21.15.117:0/3474214234 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c00c8060 con 0x55ae15d9c490 2024-09-21T06:06:29.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.217+0000 7fa9abfff640 1 -- 172.21.15.117:0/3474214234 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fa9c00a8be0 con 0x55ae15d9c490 2024-09-21T06:06:29.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.217+0000 7fa9d93a0740 1 -- 172.21.15.117:0/3474214234 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9c00b6cb0 con 0x55ae15d9c490 2024-09-21T06:06:29.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.218+0000 7fa9abfff640 1 -- 172.21.15.117:0/3474214234 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa9c0020040 con 0x55ae15d9c490 2024-09-21T06:06:29.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.218+0000 7fa9d93a0740 10 client.17319 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:29.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.218+0000 7fa9d93a0740 1 -- 172.21.15.117:0/3474214234 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ae161f48a0 con 0x55ae15d9c490 2024-09-21T06:06:29.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.218+0000 7fa9abfff640 1 -- 172.21.15.117:0/3474214234 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9c00b6cb0 con 0x55ae15d9c490 2024-09-21T06:06:29.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.218+0000 7fa9abfff640 1 -- 172.21.15.117:0/3474214234 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa9c00abf60 con 0x55ae15d9c490 2024-09-21T06:06:29.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.218+0000 7fa9d93a0740 10 client.17319 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:29.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.218+0000 7fa9d93a0740 10 client.17319 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:06:29.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.218+0000 7fa9d93a0740 20 client.17319 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:29.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.218+0000 7fa9d93a0740 1 --2- 172.21.15.117:0/3474214234 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ae15d59080 0x55ae161f6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:29.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.218+0000 7fa9d93a0740 4 client.17319 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:06:29.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.218+0000 7fa9d93a0740 1 -- 172.21.15.117:0/3474214234 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ae160e0860 con 0x55ae15d59080 2024-09-21T06:06:29.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.218+0000 7fa9cbfff640 1 --2- 172.21.15.117:0/3474214234 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ae15d59080 0x55ae161f6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:29.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.219+0000 7fa9abfff640 10 client.17319 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:29.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.219+0000 7fa9cbfff640 1 --2- 172.21.15.117:0/3474214234 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ae15d59080 0x55ae161f6ad0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a79c92799251161 server_cookie=53bd4d299335e6fd in_seq=0 out_seq=0 2024-09-21T06:06:29.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.221+0000 7fa9abfff640 1 -- 172.21.15.117:0/3474214234 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ae160e0860 con 0x55ae15d59080 2024-09-21T06:06:29.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.221+0000 7fa9abfff640 10 client.17319 handle_command_reply: tid=0 2024-09-21T06:06:29.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.221+0000 7fa9d93a0740 1 client.17319 shutdown 2024-09-21T06:06:29.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.221+0000 7fa9d93a0740 1 -- 172.21.15.117:0/3474214234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae15d9c490 msgr2=0x55ae16180380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:29.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.221+0000 7fa9d93a0740 1 --2- 172.21.15.117:0/3474214234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae15d9c490 0x55ae16180380 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fa9c00058c0 tx=0x7fa9c00ac8c0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.221+0000 7fa9d93a0740 1 -- 172.21.15.117:0/3474214234 shutdown_connections 2024-09-21T06:06:29.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.221+0000 7fa9d93a0740 1 --2- 172.21.15.117:0/3474214234 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ae15d59080 0x55ae161f6ad0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.221+0000 7fa9d93a0740 1 --2- 172.21.15.117:0/3474214234 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ae15dc5350 0x55ae1617bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.221+0000 7fa9d93a0740 1 --2- 172.21.15.117:0/3474214234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ae15da4670 0x55ae1617b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.221+0000 7fa9abfff640 0 client.17319 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:29.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.221+0000 7fa9d93a0740 1 --2- 172.21.15.117:0/3474214234 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ae15d9c490 0x55ae16180380 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.221+0000 7fa9d93a0740 1 -- 172.21.15.117:0/3474214234 >> 172.21.15.117:0/3474214234 conn(0x55ae15db9740 msgr2=0x55ae16173010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:29.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.221+0000 7fa9d93a0740 1 -- 172.21.15.117:0/3474214234 shutdown_connections 2024-09-21T06:06:29.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.221+0000 7fa9d93a0740 1 -- 172.21.15.117:0/3474214234 wait complete. 2024-09-21T06:06:29.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.222+0000 7fa9d93a0740 20 client.17319 trim_cache size 0 max 16384 2024-09-21T06:06:29.227 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T06:06:29.227 INFO:teuthology.orchestra.run.smithi117.stdout: "return_code": 0, 2024-09-21T06:06:29.227 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_tag": "26e3d90e-072a-4a4e-9661-4d004051865b", 2024-09-21T06:06:29.227 INFO:teuthology.orchestra.run.smithi117.stdout: "mode": "asynchronous" 2024-09-21T06:06:29.227 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T06:06:29.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.222+0000 7fa9d2e2c640 1 -- 172.21.15.117:0/433286569 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa9bc0b6680 msgr2=0x7fa9bc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:29.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.222+0000 7fa9d2e2c640 1 --2- 172.21.15.117:0/433286569 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa9bc0b6680 0x7fa9bc0b8b90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4004af0 tx=0x7fa9c40095c0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.222+0000 7fa9d2e2c640 1 -- 172.21.15.117:0/433286569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9cc1022a0 msgr2=0x7fa9cc1993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:29.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.222+0000 7fa9d2e2c640 1 --2- 172.21.15.117:0/433286569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9cc1022a0 0x7fa9cc1993a0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fa9c000d950 tx=0x7fa9c000de10 comp rx=0 tx=0).stop 2024-09-21T06:06:29.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.222+0000 7fa9d2e2c640 1 -- 172.21.15.117:0/433286569 shutdown_connections 2024-09-21T06:06:29.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.222+0000 7fa9d2e2c640 1 --2- 172.21.15.117:0/433286569 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa9bc0b6680 0x7fa9bc0b8b90 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.222+0000 7fa9d2e2c640 1 --2- 172.21.15.117:0/433286569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9cc108530 0x7fa9cc19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.222+0000 7fa9d2e2c640 1 --2- 172.21.15.117:0/433286569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9cc107be0 0x7fa9cc199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.222+0000 7fa9d2e2c640 1 --2- 172.21.15.117:0/433286569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9cc1022a0 0x7fa9cc1993a0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:29.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.222+0000 7fa9d2e2c640 1 -- 172.21.15.117:0/433286569 >> 172.21.15.117:0/433286569 conn(0x7fa9cc079ec0 msgr2=0x7fa9cc103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:29.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.222+0000 7fa9d2e2c640 1 -- 172.21.15.117:0/433286569 shutdown_connections 2024-09-21T06:06:29.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:29.222+0000 7fa9d2e2c640 1 -- 172.21.15.117:0/433286569 wait complete. 2024-09-21T06:06:29.963 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:06:30.213 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:30.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:30 smithi117 ceph-mon[30947]: pgmap v607: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 360 B/s wr, 11 op/s 2024-09-21T06:06:30.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:30 smithi117 ceph-mon[30947]: scrub queued for path: / 2024-09-21T06:06:30.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:30 smithi117 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-21T06:06:30.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:30 smithi117 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-21T06:06:30.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:30 smithi117 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-21T06:06:30.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:30 smithi138 ceph-mon[34391]: pgmap v607: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 360 B/s wr, 11 op/s 2024-09-21T06:06:30.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:30 smithi138 ceph-mon[34391]: scrub queued for path: / 2024-09-21T06:06:30.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:30 smithi138 ceph-mon[34391]: scrub summary: idle+waiting paths [/] 2024-09-21T06:06:30.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:30 smithi138 ceph-mon[34391]: scrub summary: active paths [/] 2024-09-21T06:06:30.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:30 smithi138 ceph-mon[34391]: scrub summary: idle+waiting paths [/] 2024-09-21T06:06:30.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:30 smithi192 ceph-mon[34700]: pgmap v607: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 360 B/s wr, 11 op/s 2024-09-21T06:06:30.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:30 smithi192 ceph-mon[34700]: scrub queued for path: / 2024-09-21T06:06:30.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:30 smithi192 ceph-mon[34700]: scrub summary: idle+waiting paths [/] 2024-09-21T06:06:30.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:30 smithi192 ceph-mon[34700]: scrub summary: active paths [/] 2024-09-21T06:06:30.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:30 smithi192 ceph-mon[34700]: scrub summary: idle+waiting paths [/] 2024-09-21T06:06:30.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.699+0000 7f8f73087640 1 -- 172.21.15.117:0/4240636088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f6c109ec0 msgr2=0x7f8f6c1106f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:30.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.699+0000 7f8f73087640 1 --2- 172.21.15.117:0/4240636088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f6c109ec0 0x7f8f6c1106f0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f8f64009900 tx=0x7f8f6402a410 comp rx=0 tx=0).stop 2024-09-21T06:06:30.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.699+0000 7f8f73087640 1 -- 172.21.15.117:0/4240636088 shutdown_connections 2024-09-21T06:06:30.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.699+0000 7f8f73087640 1 --2- 172.21.15.117:0/4240636088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f6c109ec0 0x7f8f6c1106f0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:30.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.699+0000 7f8f73087640 1 --2- 172.21.15.117:0/4240636088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f6c109570 0x7f8f6c109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:30.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.699+0000 7f8f73087640 1 --2- 172.21.15.117:0/4240636088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f6c103c30 0x7f8f6c104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:30.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.699+0000 7f8f73087640 1 -- 172.21.15.117:0/4240636088 >> 172.21.15.117:0/4240636088 conn(0x7f8f6c07b8d0 msgr2=0x7f8f6c07bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:30.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.699+0000 7f8f73087640 1 -- 172.21.15.117:0/4240636088 shutdown_connections 2024-09-21T06:06:30.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.699+0000 7f8f73087640 1 -- 172.21.15.117:0/4240636088 wait complete. 2024-09-21T06:06:30.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.700+0000 7f8f73087640 1 Processor -- start 2024-09-21T06:06:30.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.700+0000 7f8f73087640 1 -- start start 2024-09-21T06:06:30.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.700+0000 7f8f73087640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f6c103c30 0x7f8f6c19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:30.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.700+0000 7f8f73087640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f6c109570 0x7f8f6c19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:30.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.700+0000 7f8f73087640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f6c109ec0 0x7f8f6c19fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:30.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.700+0000 7f8f73087640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c113ae0 con 0x7f8f6c109ec0 2024-09-21T06:06:30.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.700+0000 7f8f73087640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c113960 con 0x7f8f6c109570 2024-09-21T06:06:30.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.700+0000 7f8f73087640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c113c60 con 0x7f8f6c103c30 2024-09-21T06:06:30.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.700+0000 7f8f71624640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f6c109ec0 0x7f8f6c19fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:30.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.700+0000 7f8f71624640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f6c109ec0 0x7f8f6c19fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55222/0 (socket says 172.21.15.117:55222) 2024-09-21T06:06:30.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.700+0000 7f8f71624640 1 -- 172.21.15.117:0/587265352 learned_addr learned my addr 172.21.15.117:0/587265352 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:30.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.700+0000 7f8f70e23640 1 --2- 172.21.15.117:0/587265352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f6c103c30 0x7f8f6c19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:30.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.700+0000 7f8f6bfff640 1 --2- 172.21.15.117:0/587265352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f6c109570 0x7f8f6c19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:30.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.700+0000 7f8f71624640 1 -- 172.21.15.117:0/587265352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f6c103c30 msgr2=0x7f8f6c19a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:30.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.700+0000 7f8f71624640 1 --2- 172.21.15.117:0/587265352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f6c103c30 0x7f8f6c19a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:30.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.700+0000 7f8f71624640 1 -- 172.21.15.117:0/587265352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f6c109570 msgr2=0x7f8f6c19aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:30.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.700+0000 7f8f71624640 1 --2- 172.21.15.117:0/587265352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f6c109570 0x7f8f6c19aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:30.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.700+0000 7f8f71624640 1 -- 172.21.15.117:0/587265352 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6c1a0460 con 0x7f8f6c109ec0 2024-09-21T06:06:30.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.701+0000 7f8f71624640 1 --2- 172.21.15.117:0/587265352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f6c109ec0 0x7f8f6c19fd30 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f8f64004770 tx=0x7f8f64004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:30.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.701+0000 7f8f69ffb640 1 -- 172.21.15.117:0/587265352 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6403f030 con 0x7f8f6c109ec0 2024-09-21T06:06:30.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.701+0000 7f8f69ffb640 1 -- 172.21.15.117:0/587265352 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f64003aa0 con 0x7f8f6c109ec0 2024-09-21T06:06:30.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.701+0000 7f8f69ffb640 1 -- 172.21.15.117:0/587265352 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f64035070 con 0x7f8f6c109ec0 2024-09-21T06:06:30.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.701+0000 7f8f73087640 1 -- 172.21.15.117:0/587265352 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f6c1a06f0 con 0x7f8f6c109ec0 2024-09-21T06:06:30.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.701+0000 7f8f73087640 1 -- 172.21.15.117:0/587265352 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f6c1a0b80 con 0x7f8f6c109ec0 2024-09-21T06:06:30.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.704+0000 7f8f795fb740 1 Processor -- start 2024-09-21T06:06:30.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.704+0000 7f8f69ffb640 1 -- 172.21.15.117:0/587265352 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f8f64042030 con 0x7f8f6c109ec0 2024-09-21T06:06:30.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.704+0000 7f8f795fb740 1 -- start start 2024-09-21T06:06:30.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.705+0000 7f8f795fb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573b3eee070 0x5573b4295650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:30.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.705+0000 7f8f795fb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573b3ec51b0 0x5573b4295b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:30.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.705+0000 7f8f795fb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b3ecd390 0x5573b42960d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:30.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.705+0000 7f8f795fb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5573b420be50 con 0x5573b3ecd390 2024-09-21T06:06:30.848 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.705+0000 7f8f795fb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5573b41fb280 con 0x5573b3ec51b0 2024-09-21T06:06:30.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.705+0000 7f8f795fb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5573b41ff090 con 0x5573b3eee070 2024-09-21T06:06:30.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.705+0000 7f8f6bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b3ecd390 0x5573b42960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:30.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.705+0000 7f8f6bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b3ecd390 0x5573b42960d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55226/0 (socket says 172.21.15.117:55226) 2024-09-21T06:06:30.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.705+0000 7f8f6bfff640 1 -- 172.21.15.117:0/3921799799 learned_addr learned my addr 172.21.15.117:0/3921799799 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:30.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.705+0000 7f8f70e23640 1 --2- 172.21.15.117:0/3921799799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573b3ec51b0 0x5573b4295b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:30.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.705+0000 7f8f6bfff640 1 --2- 172.21.15.117:0/3921799799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573b3eee070 0x5573b4295650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:30.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.705+0000 7f8f69ffb640 1 --2- 172.21.15.117:0/587265352 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8f580b6700 0x7f8f580b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:30.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.705+0000 7f8f71624640 1 --2- 172.21.15.117:0/587265352 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8f580b6700 0x7f8f580b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:30.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.705+0000 7f8f70e23640 1 -- 172.21.15.117:0/3921799799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573b3eee070 msgr2=0x5573b4295650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:30.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.705+0000 7f8f70e23640 1 --2- 172.21.15.117:0/3921799799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573b3eee070 0x5573b4295650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:30.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.705+0000 7f8f70e23640 1 -- 172.21.15.117:0/3921799799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b3ecd390 msgr2=0x5573b42960d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:30.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.705+0000 7f8f70e23640 1 --2- 172.21.15.117:0/3921799799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b3ecd390 0x5573b42960d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:30.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.705+0000 7f8f70e23640 1 -- 172.21.15.117:0/3921799799 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573b3f59650 con 0x5573b3ec51b0 2024-09-21T06:06:30.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.705+0000 7f8f6bfff640 1 --2- 172.21.15.117:0/3921799799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b3ecd390 0x5573b42960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:06:30.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.705+0000 7f8f69ffb640 1 -- 172.21.15.117:0/587265352 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7f8f64107c30 con 0x7f8f6c109ec0 2024-09-21T06:06:30.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.706+0000 7f8f70e23640 1 --2- 172.21.15.117:0/3921799799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573b3ec51b0 0x5573b4295b90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c009fd0 tx=0x7f8f5c003e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4286f1e748cc6c9c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:30.850 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.706+0000 7f8f71624640 1 --2- 172.21.15.117:0/587265352 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8f580b6700 0x7f8f580b8c10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8f64105480 tx=0x7f8f6409ae60 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:30.850 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T06:06:30.850 INFO:teuthology.orchestra.run.smithi117.stdout: "status": "no active scrubs running", 2024-09-21T06:06:30.850 INFO:teuthology.orchestra.run.smithi117.stdout: "scrubs": {}, 2024-09-21T06:06:30.851 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_stats": { 2024-09-21T06:06:30.851 INFO:teuthology.orchestra.run.smithi117.stdout: "26e3d90e-072a-4a4e-9661-4d004051865b": { 2024-09-21T06:06:30.851 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:06:29.219142+0000", 2024-09-21T06:06:30.851 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:06:30.851 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:06:30.851 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:06:30.851 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:06:30.851 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:06:30.851 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:06:30.851 INFO:teuthology.orchestra.run.smithi117.stdout: "39eeca8a-6b4f-465f-af34-b54f77320d70": { 2024-09-21T06:06:30.851 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:05:41.497141+0000", 2024-09-21T06:06:30.851 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:06:30.852 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:06:30.852 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:06:30.852 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:06:30.852 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:06:30.852 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:06:30.852 INFO:teuthology.orchestra.run.smithi117.stdout: "29ff9b7c-49d9-4910-85be-bace7e940cd5": { 2024-09-21T06:06:30.852 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:03:03.775527+0000", 2024-09-21T06:06:30.852 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:06:30.852 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:06:30.852 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:06:30.852 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:06:30.852 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:06:30.853 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:06:30.853 INFO:teuthology.orchestra.run.smithi117.stdout: "0902e33a-ce33-45fb-92d8-2e6fcb8ec8a0": { 2024-09-21T06:06:30.853 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:04:11.365073+0000", 2024-09-21T06:06:30.853 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:06:30.853 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:06:30.853 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:06:30.853 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:06:30.853 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:06:30.853 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:06:30.853 INFO:teuthology.orchestra.run.smithi117.stdout: "d543b0ae-b710-4153-99da-fe5714839c1a": { 2024-09-21T06:06:30.853 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:01:20.574754+0000", 2024-09-21T06:06:30.853 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:06:30.853 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:06:30.854 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:06:30.854 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:06:30.854 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:06:30.854 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:06:30.854 INFO:teuthology.orchestra.run.smithi117.stdout: "ad80d9f1-6aff-46c0-91ab-2c1ab0681039": { 2024-09-21T06:06:30.854 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:59:31.949108+0000", 2024-09-21T06:06:30.854 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:06:30.854 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:06:30.854 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:06:30.854 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:06:30.854 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:06:30.854 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:06:30.855 INFO:teuthology.orchestra.run.smithi117.stdout: "fb2f217d-463b-4c8f-ab19-920b04bb7306": { 2024-09-21T06:06:30.855 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:04:56.308107+0000", 2024-09-21T06:06:30.855 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:06:30.855 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:06:30.855 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:06:30.855 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:06:30.855 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:06:30.855 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:06:30.855 INFO:teuthology.orchestra.run.smithi117.stdout: "8b3fe95a-b76b-4e94-afd0-72fe35e2dd92": { 2024-09-21T06:06:30.855 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:58:01.246076+0000", 2024-09-21T06:06:30.855 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:06:30.855 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:06:30.856 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:06:30.856 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:06:30.856 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:06:30.856 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:06:30.856 INFO:teuthology.orchestra.run.smithi117.stdout: "b30bfff0-0f6b-4e30-8467-ff7db421eba6": { 2024-09-21T06:06:30.856 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:57:14.939082+0000", 2024-09-21T06:06:30.856 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:06:30.856 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:06:30.856 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:06:30.856 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:06:30.856 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:06:30.856 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:06:30.857 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:06:30.857 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T06:06:30.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.706+0000 7f8f62ffd640 1 -- 172.21.15.117:0/3921799799 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c011440 con 0x5573b3ec51b0 2024-09-21T06:06:30.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.706+0000 7f8f62ffd640 1 -- 172.21.15.117:0/3921799799 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f5c015030 con 0x5573b3ec51b0 2024-09-21T06:06:30.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.706+0000 7f8f62ffd640 1 -- 172.21.15.117:0/3921799799 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c010940 con 0x5573b3ec51b0 2024-09-21T06:06:30.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.707+0000 7f8f795fb740 1 -- 172.21.15.117:0/3921799799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573b3ec51b0 msgr2=0x5573b4295b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:30.857 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.707+0000 7f8f795fb740 1 --2- 172.21.15.117:0/3921799799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573b3ec51b0 0x5573b4295b90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c009fd0 tx=0x7f8f5c003e00 comp rx=0 tx=0).stop 2024-09-21T06:06:30.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.707+0000 7f8f795fb740 1 -- 172.21.15.117:0/3921799799 shutdown_connections 2024-09-21T06:06:30.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.707+0000 7f8f795fb740 1 --2- 172.21.15.117:0/3921799799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b3ecd390 0x5573b42960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:30.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.707+0000 7f8f795fb740 1 --2- 172.21.15.117:0/3921799799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573b3ec51b0 0x5573b4295b90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:30.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.707+0000 7f8f795fb740 1 --2- 172.21.15.117:0/3921799799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573b3eee070 0x5573b4295650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:30.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.707+0000 7f8f795fb740 1 -- 172.21.15.117:0/3921799799 >> 172.21.15.117:0/3921799799 conn(0x5573b3ee2460 msgr2=0x5573b4293540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:30.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.707+0000 7f8f795fb740 1 -- 172.21.15.117:0/3921799799 shutdown_connections 2024-09-21T06:06:30.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.707+0000 7f8f795fb740 1 -- 172.21.15.117:0/3921799799 wait complete. 2024-09-21T06:06:30.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.707+0000 7f8f795fb740 1 -- start start 2024-09-21T06:06:30.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.707+0000 7f8f795fb740 1 Processor -- start 2024-09-21T06:06:30.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.708+0000 7f8f795fb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573b3ec51b0 0x5573b42a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:30.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.708+0000 7f8f795fb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b3ecd390 0x5573b42a1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:30.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.708+0000 7f8f795fb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573b3eee070 0x5573b42a1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:30.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.708+0000 7f8f795fb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5573b405c700 con 0x5573b3ecd390 2024-09-21T06:06:30.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.708+0000 7f8f795fb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5573b4196110 con 0x5573b3eee070 2024-09-21T06:06:30.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.708+0000 7f8f795fb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5573b42a3e30 con 0x5573b3ec51b0 2024-09-21T06:06:30.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.708+0000 7f8f795fb740 10 client.0 _ll_register_callbacks cb 0x5573b41af990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:30.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.708+0000 7f8f70e23640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b3ecd390 0x5573b42a1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:30.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.708+0000 7f8f70e23640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b3ecd390 0x5573b42a1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55230/0 (socket says 172.21.15.117:55230) 2024-09-21T06:06:30.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.708+0000 7f8f70e23640 1 -- 172.21.15.117:0/2950066406 learned_addr learned my addr 172.21.15.117:0/2950066406 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:30.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.708+0000 7f8f6bfff640 1 --2- 172.21.15.117:0/2950066406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573b3eee070 0x5573b42a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:30.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.708+0000 7f8f6bfff640 1 --2- 172.21.15.117:0/2950066406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573b3ec51b0 0x5573b42a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:30.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.708+0000 7f8f70e23640 1 -- 172.21.15.117:0/2950066406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573b3ec51b0 msgr2=0x5573b42a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:30.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.708+0000 7f8f70e23640 1 --2- 172.21.15.117:0/2950066406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573b3ec51b0 0x5573b42a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:30.859 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.708+0000 7f8f70e23640 1 -- 172.21.15.117:0/2950066406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573b3eee070 msgr2=0x5573b42a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:30.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.708+0000 7f8f70e23640 1 --2- 172.21.15.117:0/2950066406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573b3eee070 0x5573b42a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:30.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.708+0000 7f8f70e23640 1 -- 172.21.15.117:0/2950066406 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573b41c3c40 con 0x5573b3ecd390 2024-09-21T06:06:30.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.708+0000 7f8f70e23640 1 --2- 172.21.15.117:0/2950066406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b3ecd390 0x5573b42a1200 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c010520 tx=0x7f8f5c005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59747cf977a12efa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:30.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.708+0000 7f8f4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:30.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.709+0000 7f8f4bfff640 1 -- 172.21.15.117:0/2950066406 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c029060 con 0x5573b3ecd390 2024-09-21T06:06:30.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.709+0000 7f8f4bfff640 1 -- 172.21.15.117:0/2950066406 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7f8f5c010a90 con 0x5573b3ecd390 2024-09-21T06:06:30.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.709+0000 7f8f795fb740 1 -- 172.21.15.117:0/2950066406 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f5c015030 con 0x5573b3ecd390 2024-09-21T06:06:30.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.709+0000 7f8f4bfff640 1 -- 172.21.15.117:0/2950066406 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8f5c004860 con 0x5573b3ecd390 2024-09-21T06:06:30.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.709+0000 7f8f795fb740 10 client.17331 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:30.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.709+0000 7f8f795fb740 1 -- 172.21.15.117:0/2950066406 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5573b42afb00 con 0x5573b3ecd390 2024-09-21T06:06:30.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.709+0000 7f8f4bfff640 1 -- 172.21.15.117:0/2950066406 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f5c015030 con 0x5573b3ecd390 2024-09-21T06:06:30.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.709+0000 7f8f4bfff640 1 -- 172.21.15.117:0/2950066406 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8f5c023070 con 0x5573b3ecd390 2024-09-21T06:06:30.860 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.709+0000 7f8f795fb740 10 client.17331 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:30.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.709+0000 7f8f795fb740 10 client.17331 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:06:30.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.709+0000 7f8f795fb740 20 client.17331 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:30.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.709+0000 7f8f795fb740 1 --2- 172.21.15.117:0/2950066406 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5573b3eac9f0 0x5573b42b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:30.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.709+0000 7f8f795fb740 4 client.17331 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:06:30.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.709+0000 7f8f795fb740 1 -- 172.21.15.117:0/2950066406 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573b420dda0 con 0x5573b3eac9f0 2024-09-21T06:06:30.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.710+0000 7f8f6bfff640 1 --2- 172.21.15.117:0/2950066406 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5573b3eac9f0 0x5573b42b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:30.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.710+0000 7f8f4bfff640 10 client.17331 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:30.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.710+0000 7f8f6bfff640 1 --2- 172.21.15.117:0/2950066406 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5573b3eac9f0 0x5573b42b2e60 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fecf1eb3a498d38a server_cookie=d83be014a4fcf35d in_seq=0 out_seq=0 2024-09-21T06:06:30.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.711+0000 7f8f4bfff640 1 -- 172.21.15.117:0/2950066406 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5573b420dda0 con 0x5573b3eac9f0 2024-09-21T06:06:30.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.711+0000 7f8f4bfff640 10 client.17331 handle_command_reply: tid=0 2024-09-21T06:06:30.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.711+0000 7f8f795fb740 1 client.17331 shutdown 2024-09-21T06:06:30.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.711+0000 7f8f795fb740 1 -- 172.21.15.117:0/2950066406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b3ecd390 msgr2=0x5573b42a1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:30.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.711+0000 7f8f795fb740 1 --2- 172.21.15.117:0/2950066406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b3ecd390 0x5573b42a1200 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c010520 tx=0x7f8f5c005b00 comp rx=0 tx=0).stop 2024-09-21T06:06:30.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.711+0000 7f8f795fb740 1 -- 172.21.15.117:0/2950066406 shutdown_connections 2024-09-21T06:06:30.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.711+0000 7f8f795fb740 1 --2- 172.21.15.117:0/2950066406 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5573b3eac9f0 0x5573b42b2e60 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:30.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.711+0000 7f8f795fb740 1 --2- 172.21.15.117:0/2950066406 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b3ecd390 0x5573b42a1200 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:30.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.711+0000 7f8f795fb740 1 --2- 172.21.15.117:0/2950066406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573b3eee070 0x5573b42a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:30.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.711+0000 7f8f795fb740 1 --2- 172.21.15.117:0/2950066406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573b3ec51b0 0x5573b42a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:30.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.711+0000 7f8f4bfff640 0 client.17331 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:30.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.711+0000 7f8f795fb740 1 -- 172.21.15.117:0/2950066406 >> 172.21.15.117:0/2950066406 conn(0x5573b3ee2460 msgr2=0x5573b42987e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:30.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.711+0000 7f8f795fb740 1 -- 172.21.15.117:0/2950066406 shutdown_connections 2024-09-21T06:06:30.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.711+0000 7f8f795fb740 1 -- 172.21.15.117:0/2950066406 wait complete. 2024-09-21T06:06:30.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.711+0000 7f8f795fb740 20 client.17331 trim_cache size 0 max 16384 2024-09-21T06:06:30.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.728+0000 7f8f795fb740 1 Processor -- start 2024-09-21T06:06:30.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.728+0000 7f8f795fb740 1 -- start start 2024-09-21T06:06:30.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.728+0000 7f8f795fb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573b3ec51b0 0x5573b4297040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:30.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.728+0000 7f8f795fb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573b3ecd390 0x5573b42a28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:30.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.728+0000 7f8f795fb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b3eee070 0x5573b42b6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:30.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.728+0000 7f8f795fb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c1af640 con 0x5573b3eee070 2024-09-21T06:06:30.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.728+0000 7f8f795fb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c1b1180 con 0x5573b3ec51b0 2024-09-21T06:06:30.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.728+0000 7f8f795fb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c1afd10 con 0x5573b3ecd390 2024-09-21T06:06:30.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.729+0000 7f8f6bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b3eee070 0x5573b42b6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:30.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.729+0000 7f8f6bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b3eee070 0x5573b42b6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55232/0 (socket says 172.21.15.117:55232) 2024-09-21T06:06:30.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.729+0000 7f8f6bfff640 1 -- 172.21.15.117:0/3535446316 learned_addr learned my addr 172.21.15.117:0/3535446316 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:30.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.729+0000 7f8f6bfff640 1 -- 172.21.15.117:0/3535446316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573b3ecd390 msgr2=0x5573b42a28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:30.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.729+0000 7f8f6bfff640 1 --2- 172.21.15.117:0/3535446316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573b3ecd390 0x5573b42a28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:30.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.729+0000 7f8f6bfff640 1 -- 172.21.15.117:0/3535446316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573b3ec51b0 msgr2=0x5573b4297040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:30.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.729+0000 7f8f6bfff640 1 --2- 172.21.15.117:0/3535446316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573b3ec51b0 0x5573b4297040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:30.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.729+0000 7f8f6bfff640 1 -- 172.21.15.117:0/3535446316 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573b4297940 con 0x5573b3eee070 2024-09-21T06:06:30.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.729+0000 7f8f6bfff640 1 --2- 172.21.15.117:0/3535446316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b3eee070 0x5573b42b6270 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f8f54000c00 tx=0x7f8f54009aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b848344737a27d63 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:30.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.729+0000 7f8f627fc640 1 -- 172.21.15.117:0/3535446316 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5402b070 con 0x5573b3eee070 2024-09-21T06:06:30.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.729+0000 7f8f627fc640 1 -- 172.21.15.117:0/3535446316 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f540047a0 con 0x5573b3eee070 2024-09-21T06:06:30.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.729+0000 7f8f627fc640 1 -- 172.21.15.117:0/3535446316 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5401a040 con 0x5573b3eee070 2024-09-21T06:06:30.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.730+0000 7f8f795fb740 1 -- 172.21.15.117:0/3535446316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b3eee070 msgr2=0x5573b42b6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:30.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.730+0000 7f8f795fb740 1 --2- 172.21.15.117:0/3535446316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b3eee070 0x5573b42b6270 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f8f54000c00 tx=0x7f8f54009aa0 comp rx=0 tx=0).stop 2024-09-21T06:06:30.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.730+0000 7f8f795fb740 1 -- 172.21.15.117:0/3535446316 shutdown_connections 2024-09-21T06:06:30.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.730+0000 7f8f795fb740 1 --2- 172.21.15.117:0/3535446316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b3eee070 0x5573b42b6270 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:30.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.730+0000 7f8f795fb740 1 --2- 172.21.15.117:0/3535446316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573b3ecd390 0x5573b42a28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:30.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.730+0000 7f8f795fb740 1 --2- 172.21.15.117:0/3535446316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573b3ec51b0 0x5573b4297040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:30.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.730+0000 7f8f795fb740 1 -- 172.21.15.117:0/3535446316 >> 172.21.15.117:0/3535446316 conn(0x5573b3ee2460 msgr2=0x5573b42a21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:30.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.730+0000 7f8f795fb740 1 -- 172.21.15.117:0/3535446316 shutdown_connections 2024-09-21T06:06:30.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.730+0000 7f8f795fb740 1 -- 172.21.15.117:0/3535446316 wait complete. 2024-09-21T06:06:30.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.730+0000 7f8f795fb740 1 -- start start 2024-09-21T06:06:30.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.730+0000 7f8f795fb740 1 Processor -- start 2024-09-21T06:06:30.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.731+0000 7f8f795fb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b3ec51b0 0x5573b4315160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:30.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.731+0000 7f8f795fb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573b3ecd390 0x5573b4315740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:30.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.731+0000 7f8f795fb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573b3eee070 0x5573b431a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:30.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.731+0000 7f8f795fb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c1b0ab0 con 0x5573b3ec51b0 2024-09-21T06:06:30.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.731+0000 7f8f795fb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c1b03e0 con 0x5573b3ecd390 2024-09-21T06:06:30.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.731+0000 7f8f795fb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c1ad640 con 0x5573b3eee070 2024-09-21T06:06:30.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.731+0000 7f8f795fb740 10 client.0 _ll_register_callbacks cb 0x7f8f540099f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:30.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.731+0000 7f8f6bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b3ec51b0 0x5573b4315160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:30.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.731+0000 7f8f6bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b3ec51b0 0x5573b4315160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55248/0 (socket says 172.21.15.117:55248) 2024-09-21T06:06:30.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.731+0000 7f8f6bfff640 1 -- 172.21.15.117:0/3612071722 learned_addr learned my addr 172.21.15.117:0/3612071722 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:30.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.731+0000 7f8f70e23640 1 --2- 172.21.15.117:0/3612071722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573b3ecd390 0x5573b4315740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:30.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.731+0000 7f8f6bfff640 1 --2- 172.21.15.117:0/3612071722 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573b3eee070 0x5573b431a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:30.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.731+0000 7f8f6bfff640 1 -- 172.21.15.117:0/3612071722 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573b3eee070 msgr2=0x5573b431a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:30.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.731+0000 7f8f6bfff640 1 --2- 172.21.15.117:0/3612071722 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573b3eee070 0x5573b431a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:30.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.731+0000 7f8f6bfff640 1 -- 172.21.15.117:0/3612071722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573b3ecd390 msgr2=0x5573b4315740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:30.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.731+0000 7f8f6bfff640 1 --2- 172.21.15.117:0/3612071722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573b3ecd390 0x5573b4315740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:30.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.731+0000 7f8f6bfff640 1 -- 172.21.15.117:0/3612071722 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573b431add0 con 0x5573b3ec51b0 2024-09-21T06:06:30.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.731+0000 7f8f6bfff640 1 --2- 172.21.15.117:0/3612071722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b3ec51b0 0x5573b4315160 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f8f54004fb0 tx=0x7f8f54009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=990fcfc555e68823 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:30.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.731+0000 7f8f4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:30.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.732+0000 7f8f4bfff640 1 -- 172.21.15.117:0/3612071722 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5402b070 con 0x5573b3ec51b0 2024-09-21T06:06:30.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.732+0000 7f8f4bfff640 1 -- 172.21.15.117:0/3612071722 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7f8f54012230 con 0x5573b3ec51b0 2024-09-21T06:06:30.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.732+0000 7f8f795fb740 1 -- 172.21.15.117:0/3612071722 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f540047a0 con 0x5573b3ec51b0 2024-09-21T06:06:30.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.732+0000 7f8f4bfff640 1 -- 172.21.15.117:0/3612071722 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8f54005260 con 0x5573b3ec51b0 2024-09-21T06:06:30.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.732+0000 7f8f795fb740 10 client.17337 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:30.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.732+0000 7f8f795fb740 1 -- 172.21.15.117:0/3612071722 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5573b43221d0 con 0x5573b3ec51b0 2024-09-21T06:06:30.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.732+0000 7f8f4bfff640 1 -- 172.21.15.117:0/3612071722 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f540047a0 con 0x5573b3ec51b0 2024-09-21T06:06:30.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.732+0000 7f8f4bfff640 1 -- 172.21.15.117:0/3612071722 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8f54003dc0 con 0x5573b3ec51b0 2024-09-21T06:06:30.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.732+0000 7f8f795fb740 10 client.17337 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:30.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.732+0000 7f8f795fb740 10 client.17337 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:06:30.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.732+0000 7f8f795fb740 20 client.17337 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:30.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.732+0000 7f8f795fb740 1 --2- 172.21.15.117:0/3612071722 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5573b3e81da0 0x5573b4324910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:30.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.732+0000 7f8f795fb740 4 client.17337 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:06:30.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.732+0000 7f8f795fb740 1 -- 172.21.15.117:0/3612071722 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5573b429c870 con 0x5573b3e81da0 2024-09-21T06:06:30.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.732+0000 7f8f70e23640 1 --2- 172.21.15.117:0/3612071722 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5573b3e81da0 0x5573b4324910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:30.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.733+0000 7f8f4bfff640 10 client.17337 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:30.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.733+0000 7f8f70e23640 1 --2- 172.21.15.117:0/3612071722 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5573b3e81da0 0x5573b4324910 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98eb152a7fcb462f server_cookie=1bc82f2ef8030ebc in_seq=0 out_seq=0 2024-09-21T06:06:30.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.733+0000 7f8f4bfff640 1 -- 172.21.15.117:0/3612071722 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2646 (crc 0 0 0) 0x5573b429c870 con 0x5573b3e81da0 2024-09-21T06:06:30.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.733+0000 7f8f4bfff640 10 client.17337 handle_command_reply: tid=0 2024-09-21T06:06:30.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.733+0000 7f8f795fb740 1 client.17337 shutdown 2024-09-21T06:06:30.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.733+0000 7f8f795fb740 1 -- 172.21.15.117:0/3612071722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b3ec51b0 msgr2=0x5573b4315160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:30.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.733+0000 7f8f795fb740 1 --2- 172.21.15.117:0/3612071722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b3ec51b0 0x5573b4315160 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f8f54004fb0 tx=0x7f8f54009e80 comp rx=0 tx=0).stop 2024-09-21T06:06:30.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.733+0000 7f8f795fb740 1 -- 172.21.15.117:0/3612071722 shutdown_connections 2024-09-21T06:06:30.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.733+0000 7f8f795fb740 1 --2- 172.21.15.117:0/3612071722 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5573b3e81da0 0x5573b4324910 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:30.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.733+0000 7f8f795fb740 1 --2- 172.21.15.117:0/3612071722 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573b3eee070 0x5573b431a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:30.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.733+0000 7f8f4bfff640 0 client.17337 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:30.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.733+0000 7f8f795fb740 1 --2- 172.21.15.117:0/3612071722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5573b3ecd390 0x5573b4315740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:30.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.733+0000 7f8f795fb740 1 --2- 172.21.15.117:0/3612071722 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5573b3ec51b0 0x5573b4315160 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:30.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.733+0000 7f8f795fb740 1 -- 172.21.15.117:0/3612071722 >> 172.21.15.117:0/3612071722 conn(0x5573b3ee2460 msgr2=0x5573b4296e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:30.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.733+0000 7f8f795fb740 1 -- 172.21.15.117:0/3612071722 shutdown_connections 2024-09-21T06:06:30.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.733+0000 7f8f795fb740 1 -- 172.21.15.117:0/3612071722 wait complete. 2024-09-21T06:06:30.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.734+0000 7f8f795fb740 20 client.17337 trim_cache size 0 max 16384 2024-09-21T06:06:30.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.734+0000 7f8f73087640 1 -- 172.21.15.117:0/587265352 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8f580b6700 msgr2=0x7f8f580b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:30.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.734+0000 7f8f73087640 1 --2- 172.21.15.117:0/587265352 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8f580b6700 0x7f8f580b8c10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8f64105480 tx=0x7f8f6409ae60 comp rx=0 tx=0).stop 2024-09-21T06:06:30.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.734+0000 7f8f73087640 1 -- 172.21.15.117:0/587265352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f6c109ec0 msgr2=0x7f8f6c19fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:30.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.734+0000 7f8f73087640 1 --2- 172.21.15.117:0/587265352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f6c109ec0 0x7f8f6c19fd30 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f8f64004770 tx=0x7f8f64004270 comp rx=0 tx=0).stop 2024-09-21T06:06:30.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.734+0000 7f8f73087640 1 -- 172.21.15.117:0/587265352 shutdown_connections 2024-09-21T06:06:30.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.734+0000 7f8f73087640 1 --2- 172.21.15.117:0/587265352 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8f580b6700 0x7f8f580b8c10 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:30.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.734+0000 7f8f73087640 1 --2- 172.21.15.117:0/587265352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f6c109ec0 0x7f8f6c19fd30 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:30.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.734+0000 7f8f73087640 1 --2- 172.21.15.117:0/587265352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f6c109570 0x7f8f6c19aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:30.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.734+0000 7f8f73087640 1 --2- 172.21.15.117:0/587265352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f6c103c30 0x7f8f6c19a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:30.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.734+0000 7f8f73087640 1 -- 172.21.15.117:0/587265352 >> 172.21.15.117:0/587265352 conn(0x7f8f6c07b8d0 msgr2=0x7f8f6c107600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:30.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.734+0000 7f8f73087640 1 -- 172.21.15.117:0/587265352 shutdown_connections 2024-09-21T06:06:30.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:30.734+0000 7f8f73087640 1 -- 172.21.15.117:0/587265352 wait complete. 2024-09-21T06:06:31.259 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:31 smithi117 ceph-mon[30947]: scrub summary: idle 2024-09-21T06:06:31.259 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:31 smithi117 ceph-mon[30947]: pgmap v608: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-21T06:06:31.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:31 smithi138 ceph-mon[34391]: scrub summary: idle 2024-09-21T06:06:31.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:31 smithi138 ceph-mon[34391]: pgmap v608: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-21T06:06:31.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:31 smithi192 ceph-mon[34700]: scrub summary: idle 2024-09-21T06:06:31.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:31 smithi192 ceph-mon[34700]: pgmap v608: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-21T06:06:31.543 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:06:31.544 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:06:31.791 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:32.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.262+0000 7fb674e94640 1 -- 172.21.15.117:0/2250348897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb670102a60 msgr2=0x7fb670102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:32.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.262+0000 7fb674e94640 1 --2- 172.21.15.117:0/2250348897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb670102a60 0x7fb670102e70 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fb660009ca0 tx=0x7fb66002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.262+0000 7fb674e94640 1 -- 172.21.15.117:0/2250348897 shutdown_connections 2024-09-21T06:06:32.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.262+0000 7fb674e94640 1 --2- 172.21.15.117:0/2250348897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb670108cf0 0x7fb67010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.262+0000 7fb674e94640 1 --2- 172.21.15.117:0/2250348897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6701083a0 0x7fb6701087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.262+0000 7fb674e94640 1 --2- 172.21.15.117:0/2250348897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb670102a60 0x7fb670102e70 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.262+0000 7fb674e94640 1 -- 172.21.15.117:0/2250348897 >> 172.21.15.117:0/2250348897 conn(0x7fb670079730 msgr2=0x7fb670079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:32.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.262+0000 7fb674e94640 1 -- 172.21.15.117:0/2250348897 shutdown_connections 2024-09-21T06:06:32.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.262+0000 7fb674e94640 1 -- 172.21.15.117:0/2250348897 wait complete. 2024-09-21T06:06:32.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.263+0000 7fb674e94640 1 Processor -- start 2024-09-21T06:06:32.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.263+0000 7fb674e94640 1 -- start start 2024-09-21T06:06:32.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.263+0000 7fb674e94640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb670102a60 0x7fb67019c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:32.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.263+0000 7fb674e94640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6701083a0 0x7fb67019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:32.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.263+0000 7fb674e94640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb670108cf0 0x7fb6701a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:32.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.263+0000 7fb674e94640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb670112960 con 0x7fb670102a60 2024-09-21T06:06:32.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.263+0000 7fb674e94640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb6701127e0 con 0x7fb6701083a0 2024-09-21T06:06:32.272 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.263+0000 7fb674e94640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb670112ae0 con 0x7fb670108cf0 2024-09-21T06:06:32.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.263+0000 7fb66e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb670102a60 0x7fb67019c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:32.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.263+0000 7fb66e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb670102a60 0x7fb67019c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55264/0 (socket says 172.21.15.117:55264) 2024-09-21T06:06:32.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.263+0000 7fb66e59c640 1 -- 172.21.15.117:0/3893107448 learned_addr learned my addr 172.21.15.117:0/3893107448 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:32.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.263+0000 7fb66dd9b640 1 --2- 172.21.15.117:0/3893107448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6701083a0 0x7fb67019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:32.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.263+0000 7fb66e59c640 1 -- 172.21.15.117:0/3893107448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb670108cf0 msgr2=0x7fb6701a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:32.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.263+0000 7fb66ed9d640 1 --2- 172.21.15.117:0/3893107448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb670108cf0 0x7fb6701a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:32.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.263+0000 7fb66e59c640 1 --2- 172.21.15.117:0/3893107448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb670108cf0 0x7fb6701a1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.263+0000 7fb66e59c640 1 -- 172.21.15.117:0/3893107448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6701083a0 msgr2=0x7fb67019cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:32.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.263+0000 7fb66e59c640 1 --2- 172.21.15.117:0/3893107448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6701083a0 0x7fb67019cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.263+0000 7fb66e59c640 1 -- 172.21.15.117:0/3893107448 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6701a2520 con 0x7fb670102a60 2024-09-21T06:06:32.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.263+0000 7fb66dd9b640 1 --2- 172.21.15.117:0/3893107448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6701083a0 0x7fb67019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:06:32.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.264+0000 7fb66e59c640 1 --2- 172.21.15.117:0/3893107448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb670102a60 0x7fb67019c670 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fb66002ade0 tx=0x7fb660003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:32.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.264+0000 7fb65f7fe640 1 -- 172.21.15.117:0/3893107448 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb660003d60 con 0x7fb670102a60 2024-09-21T06:06:32.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.264+0000 7fb674e94640 1 -- 172.21.15.117:0/3893107448 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6701a27b0 con 0x7fb670102a60 2024-09-21T06:06:32.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.264+0000 7fb65f7fe640 1 -- 172.21.15.117:0/3893107448 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6600044f0 con 0x7fb670102a60 2024-09-21T06:06:32.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.264+0000 7fb65f7fe640 1 -- 172.21.15.117:0/3893107448 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66002fd10 con 0x7fb670102a60 2024-09-21T06:06:32.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.264+0000 7fb674e94640 1 -- 172.21.15.117:0/3893107448 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6701a9030 con 0x7fb670102a60 2024-09-21T06:06:32.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.265+0000 7fb674e94640 1 -- 172.21.15.117:0/3893107448 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb670102e70 con 0x7fb670102a60 2024-09-21T06:06:32.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.266+0000 7fb65f7fe640 1 -- 172.21.15.117:0/3893107448 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fb6600046a0 con 0x7fb670102a60 2024-09-21T06:06:32.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.267+0000 7fb65f7fe640 1 --2- 172.21.15.117:0/3893107448 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6580b66b0 0x7fb6580b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:32.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.267+0000 7fb66dd9b640 1 --2- 172.21.15.117:0/3893107448 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6580b66b0 0x7fb6580b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:32.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.267+0000 7fb65f7fe640 1 -- 172.21.15.117:0/3893107448 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fb660108de0 con 0x7fb670102a60 2024-09-21T06:06:32.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.268+0000 7fb66dd9b640 1 --2- 172.21.15.117:0/3893107448 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6580b66b0 0x7fb6580b8bc0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb670100ee0 tx=0x7fb664005ec0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:32.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.271+0000 7fb65f7fe640 1 -- 172.21.15.117:0/3893107448 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb6600cf410 con 0x7fb670102a60 2024-09-21T06:06:32.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.485+0000 7fb674e94640 1 -- 172.21.15.117:0/3893107448 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb67019d7a0 con 0x7fb670102a60 2024-09-21T06:06:32.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.487+0000 7fb65f7fe640 1 -- 172.21.15.117:0/3893107448 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+12001 (secure 0 0 0) 0x7fb6600d37f0 con 0x7fb670102a60 2024-09-21T06:06:32.489 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T06:06:32.489 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":29,"btime":"2024-09-21T06:04:08:952483+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3227057504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3227057504},{"type":"v1","addr":"172.21.15.117:6835","nonce":3227057504}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3303829183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3303829183},{"type":"v1","addr":"172.21.15.117:6837","nonce":3303829183}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T05:55:14.555897+0000","modified":"2024-09-21T06:04:08.952466+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24638,"mds_1":24637,"mds_2":24628,"mds_3":24616,"mds_4":24602},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6839/2314572411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2314572411},{"type":"v1","addr":"172.21.15.117:6839","nonce":2314572411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6841/3378096561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":3378096561},{"type":"v1","addr":"172.21.15.117:6841","nonce":3378096561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/3725629862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3725629862},{"type":"v1","addr":"172.21.15.138:6833","nonce":3725629862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/4276801396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4276801396},{"type":"v1","addr":"172.21.15.138:6835","nonce":4276801396}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.138:6837/2540335374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2540335374},{"type":"v1","addr":"172.21.15.138:6837","nonce":2540335374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24616":{"gid":24616,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.192:6835/3530688860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3530688860},{"type":"v1","addr":"172.21.15.192:6835","nonce":3530688860}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.192:6837/2683860842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2683860842},{"type":"v1","addr":"172.21.15.192:6837","nonce":2683860842}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.192:6839/1727623231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1727623231},{"type":"v1","addr":"172.21.15.192:6839","nonce":1727623231}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/3230372576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":3230372576},{"type":"v1","addr":"172.21.15.138:6839","nonce":3230372576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25840":{"gid":25840,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6833/1596605058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1596605058},{"type":"v1","addr":"172.21.15.192:6833","nonce":1596605058}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24602,24628,24616,24637,24638]},"id":1}]} 2024-09-21T06:06:32.490 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 29 2024-09-21T06:06:32.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.489+0000 7fb674e94640 1 -- 172.21.15.117:0/3893107448 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6580b66b0 msgr2=0x7fb6580b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:32.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.489+0000 7fb674e94640 1 --2- 172.21.15.117:0/3893107448 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6580b66b0 0x7fb6580b8bc0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb670100ee0 tx=0x7fb664005ec0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.489+0000 7fb674e94640 1 -- 172.21.15.117:0/3893107448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb670102a60 msgr2=0x7fb67019c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:32.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.489+0000 7fb674e94640 1 --2- 172.21.15.117:0/3893107448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb670102a60 0x7fb67019c670 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fb66002ade0 tx=0x7fb660003970 comp rx=0 tx=0).stop 2024-09-21T06:06:32.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.489+0000 7fb674e94640 1 -- 172.21.15.117:0/3893107448 shutdown_connections 2024-09-21T06:06:32.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.489+0000 7fb674e94640 1 --2- 172.21.15.117:0/3893107448 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6580b66b0 0x7fb6580b8bc0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.489+0000 7fb674e94640 1 --2- 172.21.15.117:0/3893107448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb670108cf0 0x7fb6701a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.489+0000 7fb674e94640 1 --2- 172.21.15.117:0/3893107448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6701083a0 0x7fb67019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.489+0000 7fb674e94640 1 --2- 172.21.15.117:0/3893107448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb670102a60 0x7fb67019c670 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.489+0000 7fb674e94640 1 -- 172.21.15.117:0/3893107448 >> 172.21.15.117:0/3893107448 conn(0x7fb670079730 msgr2=0x7fb67006aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:32.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.489+0000 7fb674e94640 1 -- 172.21.15.117:0/3893107448 shutdown_connections 2024-09-21T06:06:32.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:32.489+0000 7fb674e94640 1 -- 172.21.15.117:0/3893107448 wait complete. 2024-09-21T06:06:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.814+0000 7fa14ee94640 1 -- 172.21.15.117:0/749502454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa148103170 msgr2=0x7fa148103580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.814+0000 7fa14ee94640 1 --2- 172.21.15.117:0/749502454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa148103170 0x7fa148103580 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fa13c009900 tx=0x7fa13c02a410 comp rx=0 tx=0).stop 2024-09-21T06:06:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.814+0000 7fa14ee94640 1 -- 172.21.15.117:0/749502454 shutdown_connections 2024-09-21T06:06:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.814+0000 7fa14ee94640 1 --2- 172.21.15.117:0/749502454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa148108c80 0x7fa14810f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.814+0000 7fa14ee94640 1 --2- 172.21.15.117:0/749502454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa148108330 0x7fa148108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.814+0000 7fa14ee94640 1 --2- 172.21.15.117:0/749502454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa148103170 0x7fa148103580 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.814+0000 7fa14ee94640 1 -- 172.21.15.117:0/749502454 >> 172.21.15.117:0/749502454 conn(0x7fa148079e90 msgr2=0x7fa14807a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.814+0000 7fa14ee94640 1 -- 172.21.15.117:0/749502454 shutdown_connections 2024-09-21T06:06:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.814+0000 7fa14ee94640 1 -- 172.21.15.117:0/749502454 wait complete. 2024-09-21T06:06:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.815+0000 7fa14ee94640 1 Processor -- start 2024-09-21T06:06:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.815+0000 7fa14ee94640 1 -- start start 2024-09-21T06:06:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.815+0000 7fa14ee94640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa148103170 0x7fa14819b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.815+0000 7fa14ee94640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa148108330 0x7fa14819b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.815+0000 7fa14ee94640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa148108c80 0x7fa14819bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.815+0000 7fa14ee94640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa148112940 con 0x7fa148103170 2024-09-21T06:06:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.815+0000 7fa14ee94640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa1481127c0 con 0x7fa148108c80 2024-09-21T06:06:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.815+0000 7fa14ee94640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa148112ac0 con 0x7fa148108330 2024-09-21T06:06:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.815+0000 7fa14cc30640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa148103170 0x7fa14819b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.815+0000 7fa14cc30640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa148103170 0x7fa14819b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55284/0 (socket says 172.21.15.117:55284) 2024-09-21T06:06:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.815+0000 7fa14cc30640 1 -- 172.21.15.117:0/1041599897 learned_addr learned my addr 172.21.15.117:0/1041599897 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.815+0000 7fa14d431640 1 --2- 172.21.15.117:0/1041599897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa148108c80 0x7fa14819bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.815+0000 7fa147fff640 1 --2- 172.21.15.117:0/1041599897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa148108330 0x7fa14819b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.816+0000 7fa14cc30640 1 -- 172.21.15.117:0/1041599897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa148108330 msgr2=0x7fa14819b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.816+0000 7fa14cc30640 1 --2- 172.21.15.117:0/1041599897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa148108330 0x7fa14819b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.816+0000 7fa14cc30640 1 -- 172.21.15.117:0/1041599897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa148108c80 msgr2=0x7fa14819bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.816+0000 7fa14cc30640 1 --2- 172.21.15.117:0/1041599897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa148108c80 0x7fa14819bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.816+0000 7fa14cc30640 1 -- 172.21.15.117:0/1041599897 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1481a6550 con 0x7fa148103170 2024-09-21T06:06:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.816+0000 7fa147fff640 1 --2- 172.21.15.117:0/1041599897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa148108330 0x7fa14819b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:06:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.816+0000 7fa14cc30640 1 --2- 172.21.15.117:0/1041599897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa148103170 0x7fa14819b290 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fa13c005940 tx=0x7fa13c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.816+0000 7fa145ffb640 1 -- 172.21.15.117:0/1041599897 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c03f030 con 0x7fa148103170 2024-09-21T06:06:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.816+0000 7fa14ee94640 1 -- 172.21.15.117:0/1041599897 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1481a67e0 con 0x7fa148103170 2024-09-21T06:06:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.816+0000 7fa145ffb640 1 -- 172.21.15.117:0/1041599897 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa13c003cb0 con 0x7fa148103170 2024-09-21T06:06:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.816+0000 7fa145ffb640 1 -- 172.21.15.117:0/1041599897 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c035070 con 0x7fa148103170 2024-09-21T06:06:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.816+0000 7fa14ee94640 1 -- 172.21.15.117:0/1041599897 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1481a6d20 con 0x7fa148103170 2024-09-21T06:06:32.823 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.819+0000 7fa155e02740 1 Processor -- start 2024-09-21T06:06:32.823 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.819+0000 7fa155e02740 1 -- start start 2024-09-21T06:06:32.823 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.819+0000 7fa155e02740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55762f3511b0 0x55762f6f69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.819+0000 7fa145ffb640 1 -- 172.21.15.117:0/1041599897 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fa13c042030 con 0x7fa148103170 2024-09-21T06:06:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.819+0000 7fa155e02740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55762f3282f0 0x55762f6f6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.819+0000 7fa155e02740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55762f3304d0 0x55762f6f7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.819+0000 7fa155e02740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55762f66b6f0 con 0x55762f3282f0 2024-09-21T06:06:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.819+0000 7fa155e02740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55762f65ab20 con 0x55762f3511b0 2024-09-21T06:06:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.819+0000 7fa155e02740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55762f65e930 con 0x55762f3304d0 2024-09-21T06:06:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.819+0000 7fa14cc30640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55762f3282f0 0x55762f6f6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.819+0000 7fa14cc30640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55762f3282f0 0x55762f6f6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55300/0 (socket says 172.21.15.117:55300) 2024-09-21T06:06:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.819+0000 7fa14cc30640 1 -- 172.21.15.117:0/3241838327 learned_addr learned my addr 172.21.15.117:0/3241838327 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.819+0000 7fa14d431640 1 --2- 172.21.15.117:0/3241838327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55762f3511b0 0x55762f6f69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.819+0000 7fa147fff640 1 --2- 172.21.15.117:0/3241838327 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55762f3304d0 0x55762f6f7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.820+0000 7fa14cc30640 1 -- 172.21.15.117:0/3241838327 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55762f3304d0 msgr2=0x55762f6f7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.820+0000 7fa14cc30640 1 --2- 172.21.15.117:0/3241838327 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55762f3304d0 0x55762f6f7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.820+0000 7fa14cc30640 1 -- 172.21.15.117:0/3241838327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55762f3511b0 msgr2=0x55762f6f69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.820+0000 7fa14cc30640 1 --2- 172.21.15.117:0/3241838327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55762f3511b0 0x55762f6f69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.820+0000 7fa14cc30640 1 -- 172.21.15.117:0/3241838327 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55762f620530 con 0x55762f3282f0 2024-09-21T06:06:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.820+0000 7fa14cc30640 1 --2- 172.21.15.117:0/3241838327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55762f3282f0 0x55762f6f6ef0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fa13c1056d0 tx=0x7fa13c110320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b49da5d6e18be3c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.820+0000 7fa145ffb640 1 --2- 172.21.15.117:0/1041599897 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa1380b6680 0x7fa1380b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.820+0000 7fa14d431640 1 --2- 172.21.15.117:0/1041599897 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa1380b6680 0x7fa1380b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.820+0000 7fa132ffd640 1 -- 172.21.15.117:0/3241838327 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c0ca4b0 con 0x55762f3282f0 2024-09-21T06:06:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.820+0000 7fa132ffd640 1 -- 172.21.15.117:0/3241838327 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa13c115030 con 0x55762f3282f0 2024-09-21T06:06:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.820+0000 7fa145ffb640 1 -- 172.21.15.117:0/1041599897 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fa13c107fc0 con 0x7fa148103170 2024-09-21T06:06:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.820+0000 7fa132ffd640 1 -- 172.21.15.117:0/3241838327 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c0cd5d0 con 0x55762f3282f0 2024-09-21T06:06:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.821+0000 7fa155e02740 1 -- 172.21.15.117:0/3241838327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55762f3282f0 msgr2=0x55762f6f6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.821+0000 7fa14d431640 1 --2- 172.21.15.117:0/1041599897 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa1380b6680 0x7fa1380b8b90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa140005fd0 tx=0x7fa1400096a0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.821+0000 7fa155e02740 1 --2- 172.21.15.117:0/3241838327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55762f3282f0 0x55762f6f6ef0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fa13c1056d0 tx=0x7fa13c110320 comp rx=0 tx=0).stop 2024-09-21T06:06:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.821+0000 7fa155e02740 1 -- 172.21.15.117:0/3241838327 shutdown_connections 2024-09-21T06:06:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.821+0000 7fa155e02740 1 --2- 172.21.15.117:0/3241838327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55762f3282f0 0x55762f6f6ef0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.825 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.821+0000 7fa155e02740 1 --2- 172.21.15.117:0/3241838327 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55762f3304d0 0x55762f6f7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.821+0000 7fa155e02740 1 --2- 172.21.15.117:0/3241838327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55762f3511b0 0x55762f6f69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.821+0000 7fa155e02740 1 -- 172.21.15.117:0/3241838327 >> 172.21.15.117:0/3241838327 conn(0x55762f345300 msgr2=0x55762f6f48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.821+0000 7fa155e02740 1 -- 172.21.15.117:0/3241838327 shutdown_connections 2024-09-21T06:06:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.821+0000 7fa155e02740 1 -- 172.21.15.117:0/3241838327 wait complete. 2024-09-21T06:06:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.822+0000 7fa155e02740 1 -- start start 2024-09-21T06:06:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.822+0000 7fa155e02740 1 Processor -- start 2024-09-21T06:06:32.829 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.822+0000 7fa155e02740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55762f3282f0 0x55762f702020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.822+0000 7fa155e02740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55762f3304d0 0x55762f702560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.822+0000 7fa155e02740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55762f3511b0 0x55762f70e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.822+0000 7fa155e02740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55762f592ab0 con 0x55762f3511b0 2024-09-21T06:06:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.822+0000 7fa155e02740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55762f600730 con 0x55762f3304d0 2024-09-21T06:06:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.822+0000 7fa155e02740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55762f704ef0 con 0x55762f3282f0 2024-09-21T06:06:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.822+0000 7fa155e02740 10 client.0 _ll_register_callbacks cb 0x55762f6546b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.822+0000 7fa14d431640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55762f3511b0 0x55762f70e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.822+0000 7fa14d431640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55762f3511b0 0x55762f70e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55306/0 (socket says 172.21.15.117:55306) 2024-09-21T06:06:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.822+0000 7fa14d431640 1 -- 172.21.15.117:0/3290374255 learned_addr learned my addr 172.21.15.117:0/3290374255 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.822+0000 7fa147fff640 1 --2- 172.21.15.117:0/3290374255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55762f3304d0 0x55762f702560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.822+0000 7fa14cc30640 1 --2- 172.21.15.117:0/3290374255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55762f3282f0 0x55762f702020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.823+0000 7fa14d431640 1 -- 172.21.15.117:0/3290374255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55762f3282f0 msgr2=0x55762f702020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:32.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.823+0000 7fa14d431640 1 --2- 172.21.15.117:0/3290374255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55762f3282f0 0x55762f702020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.823+0000 7fa14d431640 1 -- 172.21.15.117:0/3290374255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55762f3304d0 msgr2=0x55762f702560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.823+0000 7fa14d431640 1 --2- 172.21.15.117:0/3290374255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55762f3304d0 0x55762f702560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.823+0000 7fa14d431640 1 -- 172.21.15.117:0/3290374255 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55762f3bc790 con 0x55762f3511b0 2024-09-21T06:06:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.823+0000 7fa14cc30640 1 --2- 172.21.15.117:0/3290374255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55762f3282f0 0x55762f702020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:06:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.823+0000 7fa14d431640 1 --2- 172.21.15.117:0/3290374255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55762f3511b0 0x55762f70e150 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa14001f480 tx=0x7fa140004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34513549adbebcb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.823+0000 7fa127fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.823+0000 7fa127fff640 1 -- 172.21.15.117:0/3290374255 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14002c060 con 0x55762f3511b0 2024-09-21T06:06:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.823+0000 7fa127fff640 1 -- 172.21.15.117:0/3290374255 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fa140017030 con 0x55762f3511b0 2024-09-21T06:06:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.823+0000 7fa155e02740 1 -- 172.21.15.117:0/3290374255 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa13c115030 con 0x55762f3511b0 2024-09-21T06:06:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.823+0000 7fa127fff640 1 -- 172.21.15.117:0/3290374255 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa140011710 con 0x55762f3511b0 2024-09-21T06:06:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.823+0000 7fa155e02740 10 client.17355 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.823+0000 7fa155e02740 1 -- 172.21.15.117:0/3290374255 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55762f435620 con 0x55762f3511b0 2024-09-21T06:06:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.823+0000 7fa127fff640 1 -- 172.21.15.117:0/3290374255 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa13c115030 con 0x55762f3511b0 2024-09-21T06:06:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.824+0000 7fa127fff640 1 -- 172.21.15.117:0/3290374255 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa140027930 con 0x55762f3511b0 2024-09-21T06:06:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.824+0000 7fa155e02740 10 client.17355 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.824+0000 7fa155e02740 10 client.17355 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:06:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.824+0000 7fa155e02740 20 client.17355 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.824+0000 7fa155e02740 1 --2- 172.21.15.117:0/3290374255 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55762f30fd80 0x55762f713eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.824+0000 7fa155e02740 4 client.17355 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:06:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.824+0000 7fa155e02740 1 -- 172.21.15.117:0/3290374255 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55762f66d640 con 0x55762f30fd80 2024-09-21T06:06:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.824+0000 7fa14cc30640 1 --2- 172.21.15.117:0/3290374255 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55762f30fd80 0x55762f713eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.824+0000 7fa127fff640 10 client.17355 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.824+0000 7fa14cc30640 1 --2- 172.21.15.117:0/3290374255 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55762f30fd80 0x55762f713eb0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d81ae9343d20b4d6 server_cookie=ec566a578969485a in_seq=0 out_seq=0 2024-09-21T06:06:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.825+0000 7fa127fff640 1 -- 172.21.15.117:0/3290374255 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55762f66d640 con 0x55762f30fd80 2024-09-21T06:06:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.825+0000 7fa127fff640 10 client.17355 handle_command_reply: tid=0 2024-09-21T06:06:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.825+0000 7fa155e02740 1 client.17355 shutdown 2024-09-21T06:06:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.825+0000 7fa155e02740 1 -- 172.21.15.117:0/3290374255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55762f3511b0 msgr2=0x55762f70e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.825+0000 7fa155e02740 1 --2- 172.21.15.117:0/3290374255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55762f3511b0 0x55762f70e150 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa14001f480 tx=0x7fa140004ab0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.825+0000 7fa155e02740 1 -- 172.21.15.117:0/3290374255 shutdown_connections 2024-09-21T06:06:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.825+0000 7fa155e02740 1 --2- 172.21.15.117:0/3290374255 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55762f30fd80 0x55762f713eb0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.825+0000 7fa155e02740 1 --2- 172.21.15.117:0/3290374255 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55762f3511b0 0x55762f70e150 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.825+0000 7fa127fff640 0 client.17355 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.825+0000 7fa155e02740 1 --2- 172.21.15.117:0/3290374255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55762f3304d0 0x55762f702560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.825+0000 7fa155e02740 1 --2- 172.21.15.117:0/3290374255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55762f3282f0 0x55762f702020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.825+0000 7fa155e02740 1 -- 172.21.15.117:0/3290374255 >> 172.21.15.117:0/3290374255 conn(0x55762f345300 msgr2=0x55762f6f79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.825+0000 7fa155e02740 1 -- 172.21.15.117:0/3290374255 shutdown_connections 2024-09-21T06:06:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.825+0000 7fa155e02740 1 -- 172.21.15.117:0/3290374255 wait complete. 2024-09-21T06:06:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.826+0000 7fa155e02740 20 client.17355 trim_cache size 0 max 16384 2024-09-21T06:06:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.840+0000 7fa155e02740 1 Processor -- start 2024-09-21T06:06:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.840+0000 7fa155e02740 1 -- start start 2024-09-21T06:06:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.840+0000 7fa155e02740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55762f30fd80 0x55762f6f80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.841+0000 7fa155e02740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55762f3282f0 0x55762f70dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.841+0000 7fa155e02740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55762f3304d0 0x55762f6ff1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.841+0000 7fa155e02740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa1481af6a0 con 0x55762f3304d0 2024-09-21T06:06:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.841+0000 7fa155e02740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa1481b11e0 con 0x55762f3282f0 2024-09-21T06:06:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.841+0000 7fa155e02740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa1481afd70 con 0x55762f30fd80 2024-09-21T06:06:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.841+0000 7fa14d431640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55762f3304d0 0x55762f6ff1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.841+0000 7fa14d431640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55762f3304d0 0x55762f6ff1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55316/0 (socket says 172.21.15.117:55316) 2024-09-21T06:06:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.841+0000 7fa14d431640 1 -- 172.21.15.117:0/1204313241 learned_addr learned my addr 172.21.15.117:0/1204313241 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.841+0000 7fa147fff640 1 --2- 172.21.15.117:0/1204313241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55762f3282f0 0x55762f70dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.841+0000 7fa14cc30640 1 --2- 172.21.15.117:0/1204313241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55762f30fd80 0x55762f6f80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.841+0000 7fa14d431640 1 -- 172.21.15.117:0/1204313241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55762f30fd80 msgr2=0x55762f6f80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.841+0000 7fa14d431640 1 --2- 172.21.15.117:0/1204313241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55762f30fd80 0x55762f6f80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.841+0000 7fa14d431640 1 -- 172.21.15.117:0/1204313241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55762f3282f0 msgr2=0x55762f70dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.841+0000 7fa14d431640 1 --2- 172.21.15.117:0/1204313241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55762f3282f0 0x55762f70dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.841+0000 7fa14d431640 1 -- 172.21.15.117:0/1204313241 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55762f700500 con 0x55762f3304d0 2024-09-21T06:06:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.841+0000 7fa14d431640 1 --2- 172.21.15.117:0/1204313241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55762f3304d0 0x55762f6ff1f0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fa140018040 tx=0x7fa140005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b762babd081b7aee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.841+0000 7fa1327fc640 1 -- 172.21.15.117:0/1204313241 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa140011ab0 con 0x55762f3304d0 2024-09-21T06:06:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.841+0000 7fa1327fc640 1 -- 172.21.15.117:0/1204313241 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa140011c50 con 0x55762f3304d0 2024-09-21T06:06:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.841+0000 7fa1327fc640 1 -- 172.21.15.117:0/1204313241 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa140031770 con 0x55762f3304d0 2024-09-21T06:06:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.842+0000 7fa155e02740 1 -- 172.21.15.117:0/1204313241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55762f3304d0 msgr2=0x55762f6ff1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.842+0000 7fa155e02740 1 --2- 172.21.15.117:0/1204313241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55762f3304d0 0x55762f6ff1f0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fa140018040 tx=0x7fa140005ce0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.842+0000 7fa155e02740 1 -- 172.21.15.117:0/1204313241 shutdown_connections 2024-09-21T06:06:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.842+0000 7fa155e02740 1 --2- 172.21.15.117:0/1204313241 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55762f3304d0 0x55762f6ff1f0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.842+0000 7fa155e02740 1 --2- 172.21.15.117:0/1204313241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55762f3282f0 0x55762f70dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.842+0000 7fa155e02740 1 --2- 172.21.15.117:0/1204313241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55762f30fd80 0x55762f6f80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.842+0000 7fa155e02740 1 -- 172.21.15.117:0/1204313241 >> 172.21.15.117:0/1204313241 conn(0x55762f345300 msgr2=0x55762f6f79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.842+0000 7fa155e02740 1 -- 172.21.15.117:0/1204313241 shutdown_connections 2024-09-21T06:06:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.842+0000 7fa155e02740 1 -- 172.21.15.117:0/1204313241 wait complete. 2024-09-21T06:06:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.842+0000 7fa155e02740 1 -- start start 2024-09-21T06:06:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.843+0000 7fa155e02740 1 Processor -- start 2024-09-21T06:06:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.843+0000 7fa155e02740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55762f30fd80 0x55762f776b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.843+0000 7fa155e02740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55762f3282f0 0x55762f777150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.843+0000 7fa155e02740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55762f3304d0 0x55762f707700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.843+0000 7fa155e02740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa1481b0b10 con 0x55762f3304d0 2024-09-21T06:06:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.843+0000 7fa155e02740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa1481b0440 con 0x55762f3282f0 2024-09-21T06:06:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.843+0000 7fa155e02740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa1481ad6a0 con 0x55762f30fd80 2024-09-21T06:06:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.843+0000 7fa155e02740 10 client.0 _ll_register_callbacks cb 0x7fa140003f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.843+0000 7fa14d431640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55762f3304d0 0x55762f707700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.843+0000 7fa14d431640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55762f3304d0 0x55762f707700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55332/0 (socket says 172.21.15.117:55332) 2024-09-21T06:06:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.843+0000 7fa14d431640 1 -- 172.21.15.117:0/478317063 learned_addr learned my addr 172.21.15.117:0/478317063 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.843+0000 7fa147fff640 1 --2- 172.21.15.117:0/478317063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55762f3282f0 0x55762f777150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.843+0000 7fa14cc30640 1 --2- 172.21.15.117:0/478317063 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55762f30fd80 0x55762f776b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.843+0000 7fa14d431640 1 -- 172.21.15.117:0/478317063 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55762f30fd80 msgr2=0x55762f776b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.843+0000 7fa14d431640 1 --2- 172.21.15.117:0/478317063 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55762f30fd80 0x55762f776b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.843+0000 7fa14d431640 1 -- 172.21.15.117:0/478317063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55762f3282f0 msgr2=0x55762f777150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.843+0000 7fa14d431640 1 --2- 172.21.15.117:0/478317063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55762f3282f0 0x55762f777150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.843+0000 7fa14d431640 1 -- 172.21.15.117:0/478317063 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55762f707e70 con 0x55762f3304d0 2024-09-21T06:06:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.843+0000 7fa14cc30640 1 --2- 172.21.15.117:0/478317063 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55762f30fd80 0x55762f776b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:06:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.844+0000 7fa14d431640 1 --2- 172.21.15.117:0/478317063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55762f3304d0 0x55762f707700 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fa140027df0 tx=0x7fa140004930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd7abb27335c166 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.844+0000 7fa127fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.844+0000 7fa127fff640 1 -- 172.21.15.117:0/478317063 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14003e030 con 0x55762f3304d0 2024-09-21T06:06:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.844+0000 7fa127fff640 1 -- 172.21.15.117:0/478317063 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fa140004ab0 con 0x55762f3304d0 2024-09-21T06:06:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.844+0000 7fa155e02740 1 -- 172.21.15.117:0/478317063 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa140011c50 con 0x55762f3304d0 2024-09-21T06:06:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.844+0000 7fa127fff640 1 -- 172.21.15.117:0/478317063 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa140048050 con 0x55762f3304d0 2024-09-21T06:06:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.844+0000 7fa155e02740 10 client.17361 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.844+0000 7fa155e02740 1 -- 172.21.15.117:0/478317063 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55762f77e2d0 con 0x55762f3304d0 2024-09-21T06:06:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.844+0000 7fa127fff640 1 -- 172.21.15.117:0/478317063 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa140011c50 con 0x55762f3304d0 2024-09-21T06:06:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.844+0000 7fa127fff640 1 -- 172.21.15.117:0/478317063 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa140039070 con 0x55762f3304d0 2024-09-21T06:06:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.844+0000 7fa155e02740 10 client.17361 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.844+0000 7fa155e02740 10 client.17361 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:06:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.844+0000 7fa155e02740 20 client.17361 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.844+0000 7fa155e02740 1 --2- 172.21.15.117:0/478317063 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55762f3511b0 0x55762f7809f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.844+0000 7fa155e02740 4 client.17361 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:06:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.844+0000 7fa155e02740 1 -- 172.21.15.117:0/478317063 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55762f6690f0 con 0x55762f3511b0 2024-09-21T06:06:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.845+0000 7fa14cc30640 1 --2- 172.21.15.117:0/478317063 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55762f3511b0 0x55762f7809f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.845+0000 7fa127fff640 10 client.17361 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.845+0000 7fa14cc30640 1 --2- 172.21.15.117:0/478317063 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55762f3511b0 0x55762f7809f0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=178645b9027e2c42 server_cookie=ffc597450b8a2316 in_seq=0 out_seq=0 2024-09-21T06:06:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.852+0000 7fa127fff640 1 -- 172.21.15.117:0/478317063 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55762f6690f0 con 0x55762f3511b0 2024-09-21T06:06:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.852+0000 7fa127fff640 10 client.17361 handle_command_reply: tid=0 2024-09-21T06:06:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.852+0000 7fa155e02740 1 client.17361 shutdown 2024-09-21T06:06:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.852+0000 7fa155e02740 1 -- 172.21.15.117:0/478317063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55762f3304d0 msgr2=0x55762f707700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.852+0000 7fa155e02740 1 --2- 172.21.15.117:0/478317063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55762f3304d0 0x55762f707700 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fa140027df0 tx=0x7fa140004930 comp rx=0 tx=0).stop 2024-09-21T06:06:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.852+0000 7fa155e02740 1 -- 172.21.15.117:0/478317063 shutdown_connections 2024-09-21T06:06:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.852+0000 7fa155e02740 1 --2- 172.21.15.117:0/478317063 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55762f3511b0 0x55762f7809f0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.852+0000 7fa155e02740 1 --2- 172.21.15.117:0/478317063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55762f3304d0 0x55762f707700 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.852+0000 7fa155e02740 1 --2- 172.21.15.117:0/478317063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55762f3282f0 0x55762f777150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.852+0000 7fa127fff640 0 client.17361 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.852+0000 7fa155e02740 1 --2- 172.21.15.117:0/478317063 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55762f30fd80 0x55762f776b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.852+0000 7fa155e02740 1 -- 172.21.15.117:0/478317063 >> 172.21.15.117:0/478317063 conn(0x55762f345300 msgr2=0x55762f703460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.852+0000 7fa155e02740 1 -- 172.21.15.117:0/478317063 shutdown_connections 2024-09-21T06:06:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.853+0000 7fa155e02740 1 -- 172.21.15.117:0/478317063 wait complete. 2024-09-21T06:06:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.853+0000 7fa155e02740 20 client.17361 trim_cache size 0 max 16384 2024-09-21T06:06:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T06:06:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T06:06:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T06:06:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T06:06:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.853+0000 7fa14ee94640 1 -- 172.21.15.117:0/1041599897 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa1380b6680 msgr2=0x7fa1380b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.853+0000 7fa14ee94640 1 --2- 172.21.15.117:0/1041599897 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa1380b6680 0x7fa1380b8b90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa140005fd0 tx=0x7fa1400096a0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.853+0000 7fa14ee94640 1 -- 172.21.15.117:0/1041599897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa148103170 msgr2=0x7fa14819b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.853+0000 7fa14ee94640 1 --2- 172.21.15.117:0/1041599897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa148103170 0x7fa14819b290 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fa13c005940 tx=0x7fa13c0042f0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.853+0000 7fa14ee94640 1 -- 172.21.15.117:0/1041599897 shutdown_connections 2024-09-21T06:06:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.853+0000 7fa14ee94640 1 --2- 172.21.15.117:0/1041599897 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa1380b6680 0x7fa1380b8b90 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.853+0000 7fa14ee94640 1 --2- 172.21.15.117:0/1041599897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa148108c80 0x7fa14819bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.853+0000 7fa14ee94640 1 --2- 172.21.15.117:0/1041599897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa148108330 0x7fa14819b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.853+0000 7fa14ee94640 1 --2- 172.21.15.117:0/1041599897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa148103170 0x7fa14819b290 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.853+0000 7fa14ee94640 1 -- 172.21.15.117:0/1041599897 >> 172.21.15.117:0/1041599897 conn(0x7fa148079e90 msgr2=0x7fa148106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.853+0000 7fa14ee94640 1 -- 172.21.15.117:0/1041599897 shutdown_connections 2024-09-21T06:06:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:32.853+0000 7fa14ee94640 1 -- 172.21.15.117:0/1041599897 wait complete. 2024-09-21T06:06:33.162 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T06:06:33.406 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:33.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:33 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3893107448' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:06:33.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:33 smithi117 ceph-mon[30947]: pgmap v609: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-21T06:06:33.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:33 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3893107448' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:06:33.865 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:33 smithi138 ceph-mon[34391]: pgmap v609: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-21T06:06:33.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.904+0000 7fab52cdb640 1 -- 172.21.15.117:0/2462664340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab4c103b60 msgr2=0x7fab4c103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:33.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.904+0000 7fab52cdb640 1 --2- 172.21.15.117:0/2462664340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab4c103b60 0x7fab4c103f70 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fab3c009900 tx=0x7fab3c02a410 comp rx=0 tx=0).stop 2024-09-21T06:06:33.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.904+0000 7fab52cdb640 1 -- 172.21.15.117:0/2462664340 shutdown_connections 2024-09-21T06:06:33.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.904+0000 7fab52cdb640 1 --2- 172.21.15.117:0/2462664340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab4c109df0 0x7fab4c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.904+0000 7fab52cdb640 1 --2- 172.21.15.117:0/2462664340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab4c1094a0 0x7fab4c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.904+0000 7fab52cdb640 1 --2- 172.21.15.117:0/2462664340 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab4c103b60 0x7fab4c103f70 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.904+0000 7fab52cdb640 1 -- 172.21.15.117:0/2462664340 >> 172.21.15.117:0/2462664340 conn(0x7fab4c07b850 msgr2=0x7fab4c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:33.916 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.904+0000 7fab52cdb640 1 -- 172.21.15.117:0/2462664340 shutdown_connections 2024-09-21T06:06:33.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.904+0000 7fab52cdb640 1 -- 172.21.15.117:0/2462664340 wait complete. 2024-09-21T06:06:33.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.905+0000 7fab52cdb640 1 Processor -- start 2024-09-21T06:06:33.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.905+0000 7fab52cdb640 1 -- start start 2024-09-21T06:06:33.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.905+0000 7fab52cdb640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab4c103b60 0x7fab4c19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:33.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.905+0000 7fab52cdb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab4c1094a0 0x7fab4c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:33.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.905+0000 7fab52cdb640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab4c109df0 0x7fab4c19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:33.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.905+0000 7fab52cdb640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c113af0 con 0x7fab4c103b60 2024-09-21T06:06:33.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.905+0000 7fab52cdb640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c113970 con 0x7fab4c1094a0 2024-09-21T06:06:33.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.905+0000 7fab52cdb640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c113c70 con 0x7fab4c109df0 2024-09-21T06:06:33.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.905+0000 7fab50a77640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab4c103b60 0x7fab4c19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:33.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.905+0000 7fab50a77640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab4c103b60 0x7fab4c19a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55350/0 (socket says 172.21.15.117:55350) 2024-09-21T06:06:33.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.905+0000 7fab50a77640 1 -- 172.21.15.117:0/1133582813 learned_addr learned my addr 172.21.15.117:0/1133582813 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:33.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.905+0000 7fab51278640 1 --2- 172.21.15.117:0/1133582813 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab4c109df0 0x7fab4c19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:33.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.905+0000 7fab4bfff640 1 --2- 172.21.15.117:0/1133582813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab4c1094a0 0x7fab4c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:33.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.906+0000 7fab50a77640 1 -- 172.21.15.117:0/1133582813 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab4c109df0 msgr2=0x7fab4c19fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:33.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.906+0000 7fab50a77640 1 --2- 172.21.15.117:0/1133582813 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab4c109df0 0x7fab4c19fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.906+0000 7fab50a77640 1 -- 172.21.15.117:0/1133582813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab4c1094a0 msgr2=0x7fab4c19aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:33.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.906+0000 7fab50a77640 1 --2- 172.21.15.117:0/1133582813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab4c1094a0 0x7fab4c19aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.906+0000 7fab50a77640 1 -- 172.21.15.117:0/1133582813 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab4c1a0470 con 0x7fab4c103b60 2024-09-21T06:06:33.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.906+0000 7fab50a77640 1 --2- 172.21.15.117:0/1133582813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab4c103b60 0x7fab4c19a500 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fab3c005940 tx=0x7fab3c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:33.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.906+0000 7fab49ffb640 1 -- 172.21.15.117:0/1133582813 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3c03f030 con 0x7fab4c103b60 2024-09-21T06:06:33.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.906+0000 7fab52cdb640 1 -- 172.21.15.117:0/1133582813 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab4c1a0700 con 0x7fab4c103b60 2024-09-21T06:06:33.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.906+0000 7fab49ffb640 1 -- 172.21.15.117:0/1133582813 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab3c003cb0 con 0x7fab4c103b60 2024-09-21T06:06:33.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.906+0000 7fab49ffb640 1 -- 172.21.15.117:0/1133582813 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3c035070 con 0x7fab4c103b60 2024-09-21T06:06:33.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.906+0000 7fab52cdb640 1 -- 172.21.15.117:0/1133582813 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab4c1a6ec0 con 0x7fab4c103b60 2024-09-21T06:06:33.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.909+0000 7fab5924f740 1 Processor -- start 2024-09-21T06:06:33.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.909+0000 7fab5924f740 1 -- start start 2024-09-21T06:06:33.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.910+0000 7fab5924f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b3ddd7070 0x555b3e17e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:33.918 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.910+0000 7fab5924f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3ddae1b0 0x555b3e17ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:33.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.910+0000 7fab5924f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b3ddb6390 0x555b3e17f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:33.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.910+0000 7fab5924f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555b3e0f4ea0 con 0x555b3ddd7070 2024-09-21T06:06:33.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.910+0000 7fab5924f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555b3e0e42d0 con 0x555b3ddae1b0 2024-09-21T06:06:33.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.910+0000 7fab5924f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555b3e0e80e0 con 0x555b3ddb6390 2024-09-21T06:06:33.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.910+0000 7fab51278640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b3ddd7070 0x555b3e17e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:33.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.910+0000 7fab51278640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b3ddd7070 0x555b3e17e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55360/0 (socket says 172.21.15.117:55360) 2024-09-21T06:06:33.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.910+0000 7fab51278640 1 -- 172.21.15.117:0/872390690 learned_addr learned my addr 172.21.15.117:0/872390690 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:33.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.910+0000 7fab4bfff640 1 --2- 172.21.15.117:0/872390690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b3ddb6390 0x555b3e17f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:33.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.910+0000 7fab49ffb640 1 -- 172.21.15.117:0/1133582813 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fab3c042030 con 0x7fab4c103b60 2024-09-21T06:06:33.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.910+0000 7fab51278640 1 -- 172.21.15.117:0/872390690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b3ddb6390 msgr2=0x555b3e17f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:33.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.910+0000 7fab51278640 1 --2- 172.21.15.117:0/872390690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b3ddb6390 0x555b3e17f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.910+0000 7fab51278640 1 -- 172.21.15.117:0/872390690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3ddae1b0 msgr2=0x555b3e17ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:33.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.910+0000 7fab51278640 1 --2- 172.21.15.117:0/872390690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3ddae1b0 0x555b3e17ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.919 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.910+0000 7fab51278640 1 -- 172.21.15.117:0/872390690 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b3de42650 con 0x555b3ddd7070 2024-09-21T06:06:33.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.910+0000 7fab4bfff640 1 --2- 172.21.15.117:0/872390690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b3ddb6390 0x555b3e17f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:06:33.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.910+0000 7fab51278640 1 --2- 172.21.15.117:0/872390690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b3ddd7070 0x555b3e17e6a0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fab44003a40 tx=0x7fab44003f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d139daff19882d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:33.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.910+0000 7fab32ffd640 1 -- 172.21.15.117:0/872390690 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4401a470 con 0x555b3ddd7070 2024-09-21T06:06:33.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.910+0000 7fab32ffd640 1 -- 172.21.15.117:0/872390690 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab44015070 con 0x555b3ddd7070 2024-09-21T06:06:33.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.910+0000 7fab32ffd640 1 -- 172.21.15.117:0/872390690 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab440119b0 con 0x555b3ddd7070 2024-09-21T06:06:33.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.911+0000 7fab49ffb640 1 --2- 172.21.15.117:0/1133582813 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fab340b6680 0x7fab340b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:33.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.911+0000 7fab50a77640 1 --2- 172.21.15.117:0/1133582813 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fab340b6680 0x7fab340b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:33.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.911+0000 7fab49ffb640 1 -- 172.21.15.117:0/1133582813 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fab3c107fc0 con 0x7fab4c103b60 2024-09-21T06:06:33.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.912+0000 7fab5924f740 1 -- 172.21.15.117:0/872390690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b3ddd7070 msgr2=0x555b3e17e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:33.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.912+0000 7fab50a77640 1 --2- 172.21.15.117:0/1133582813 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fab340b6680 0x7fab340b8b90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x555b3e17d320 tx=0x7fab3c09b140 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:33.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.912+0000 7fab5924f740 1 --2- 172.21.15.117:0/872390690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b3ddd7070 0x555b3e17e6a0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fab44003a40 tx=0x7fab44003f00 comp rx=0 tx=0).stop 2024-09-21T06:06:33.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.912+0000 7fab5924f740 1 -- 172.21.15.117:0/872390690 shutdown_connections 2024-09-21T06:06:33.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.912+0000 7fab5924f740 1 --2- 172.21.15.117:0/872390690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b3ddb6390 0x555b3e17f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.912+0000 7fab5924f740 1 --2- 172.21.15.117:0/872390690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3ddae1b0 0x555b3e17ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.912+0000 7fab5924f740 1 --2- 172.21.15.117:0/872390690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b3ddd7070 0x555b3e17e6a0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.912+0000 7fab5924f740 1 -- 172.21.15.117:0/872390690 >> 172.21.15.117:0/872390690 conn(0x555b3ddcb460 msgr2=0x555b3e17c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:33.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.912+0000 7fab5924f740 1 -- 172.21.15.117:0/872390690 shutdown_connections 2024-09-21T06:06:33.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.912+0000 7fab5924f740 1 -- 172.21.15.117:0/872390690 wait complete. 2024-09-21T06:06:33.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.912+0000 7fab5924f740 1 -- start start 2024-09-21T06:06:33.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.912+0000 7fab5924f740 1 Processor -- start 2024-09-21T06:06:33.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.913+0000 7fab5924f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b3ddae1b0 0x555b3e189dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:33.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.913+0000 7fab5924f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3ddb6390 0x555b3e18a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:33.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.913+0000 7fab5924f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b3ddd7070 0x555b3e19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:33.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.913+0000 7fab5924f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555b3df45700 con 0x555b3ddd7070 2024-09-21T06:06:33.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.913+0000 7fab5924f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555b3e07f160 con 0x555b3ddb6390 2024-09-21T06:06:33.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.913+0000 7fab5924f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555b3e18ce80 con 0x555b3ddae1b0 2024-09-21T06:06:33.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.913+0000 7fab5924f740 10 client.0 _ll_register_callbacks cb 0x555b3e0989e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:33.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.913+0000 7fab51278640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b3ddd7070 0x555b3e19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:33.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.913+0000 7fab51278640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b3ddd7070 0x555b3e19a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55370/0 (socket says 172.21.15.117:55370) 2024-09-21T06:06:33.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.913+0000 7fab51278640 1 -- 172.21.15.117:0/184122701 learned_addr learned my addr 172.21.15.117:0/184122701 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:33.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.913+0000 7fab4bfff640 1 --2- 172.21.15.117:0/184122701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3ddb6390 0x555b3e18a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:33.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.914+0000 7fab51278640 1 --2- 172.21.15.117:0/184122701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b3ddae1b0 0x555b3e189dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:33.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.914+0000 7fab51278640 1 -- 172.21.15.117:0/184122701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b3ddae1b0 msgr2=0x555b3e189dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:33.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.914+0000 7fab51278640 1 --2- 172.21.15.117:0/184122701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b3ddae1b0 0x555b3e189dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.914+0000 7fab51278640 1 -- 172.21.15.117:0/184122701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3ddb6390 msgr2=0x555b3e18a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:33.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.914+0000 7fab51278640 1 --2- 172.21.15.117:0/184122701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3ddb6390 0x555b3e18a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.914+0000 7fab51278640 1 -- 172.21.15.117:0/184122701 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b3e0acc90 con 0x555b3ddd7070 2024-09-21T06:06:33.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.914+0000 7fab51278640 1 --2- 172.21.15.117:0/184122701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b3ddd7070 0x555b3e19a5b0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fab44011590 tx=0x7fab440115f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad769e5fd757a6df server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:33.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.914+0000 7fab2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:33.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.914+0000 7fab2bfff640 1 -- 172.21.15.117:0/184122701 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4401a470 con 0x555b3ddd7070 2024-09-21T06:06:33.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.914+0000 7fab2bfff640 1 -- 172.21.15.117:0/184122701 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fab4401a610 con 0x555b3ddd7070 2024-09-21T06:06:33.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.914+0000 7fab5924f740 1 -- 172.21.15.117:0/184122701 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab44015070 con 0x555b3ddd7070 2024-09-21T06:06:33.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.914+0000 7fab2bfff640 1 -- 172.21.15.117:0/184122701 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fab44031050 con 0x555b3ddd7070 2024-09-21T06:06:33.922 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.914+0000 7fab5924f740 10 client.17373 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:33.923 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.914+0000 7fab5924f740 1 -- 172.21.15.117:0/184122701 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555b3e196680 con 0x555b3ddd7070 2024-09-21T06:06:33.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.915+0000 7fab2bfff640 1 -- 172.21.15.117:0/184122701 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab44015070 con 0x555b3ddd7070 2024-09-21T06:06:33.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.915+0000 7fab2bfff640 1 -- 172.21.15.117:0/184122701 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fab440108f0 con 0x555b3ddd7070 2024-09-21T06:06:33.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.915+0000 7fab5924f740 10 client.17373 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:33.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.915+0000 7fab5924f740 10 client.17373 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T06:06:33.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.915+0000 7fab5924f740 20 client.17373 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:33.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.915+0000 7fab5924f740 1 --2- 172.21.15.117:0/184122701 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x555b3dd959f0 0x555b3e197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:33.924 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.915+0000 7fab5924f740 4 client.17373 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:06:33.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.915+0000 7fab5924f740 1 -- 172.21.15.117:0/184122701 --> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b3e0f6df0 con 0x555b3dd959f0 2024-09-21T06:06:33.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.915+0000 7fab4bfff640 1 --2- 172.21.15.117:0/184122701 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x555b3dd959f0 0x555b3e197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:33.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.916+0000 7fab2bfff640 10 client.17373 ms_handle_connect on v2:172.21.15.138:6836/2540335374 2024-09-21T06:06:33.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.916+0000 7fab4bfff640 1 --2- 172.21.15.117:0/184122701 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x555b3dd959f0 0x555b3e197ae0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=16e5666481b55fb0 server_cookie=aba9591b478f5013 in_seq=0 out_seq=0 2024-09-21T06:06:33.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.916+0000 7fab2bfff640 1 -- 172.21.15.117:0/184122701 <== mds.4 v2:172.21.15.138:6836/2540335374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555b3e0f6df0 con 0x555b3dd959f0 2024-09-21T06:06:33.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.916+0000 7fab2bfff640 10 client.17373 handle_command_reply: tid=0 2024-09-21T06:06:33.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.917+0000 7fab5924f740 1 client.17373 shutdown 2024-09-21T06:06:33.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.917+0000 7fab5924f740 1 -- 172.21.15.117:0/184122701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b3ddd7070 msgr2=0x555b3e19a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:33.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.917+0000 7fab5924f740 1 --2- 172.21.15.117:0/184122701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b3ddd7070 0x555b3e19a5b0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fab44011590 tx=0x7fab440115f0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.917+0000 7fab5924f740 1 -- 172.21.15.117:0/184122701 shutdown_connections 2024-09-21T06:06:33.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.917+0000 7fab5924f740 1 --2- 172.21.15.117:0/184122701 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x555b3dd959f0 0x555b3e197ae0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.917+0000 7fab5924f740 1 --2- 172.21.15.117:0/184122701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b3ddd7070 0x555b3e19a5b0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.917+0000 7fab5924f740 1 --2- 172.21.15.117:0/184122701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3ddb6390 0x555b3e18a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.917+0000 7fab5924f740 1 --2- 172.21.15.117:0/184122701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b3ddae1b0 0x555b3e189dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.917+0000 7fab5924f740 1 -- 172.21.15.117:0/184122701 >> 172.21.15.117:0/184122701 conn(0x555b3ddcb460 msgr2=0x555b3e181880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:33.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.917+0000 7fab2bfff640 0 client.17373 ms_handle_reset on v2:172.21.15.138:6836/2540335374 2024-09-21T06:06:33.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.917+0000 7fab5924f740 1 -- 172.21.15.117:0/184122701 shutdown_connections 2024-09-21T06:06:33.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.917+0000 7fab5924f740 1 -- 172.21.15.117:0/184122701 wait complete. 2024-09-21T06:06:33.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.917+0000 7fab5924f740 20 client.17373 trim_cache size 0 max 16384 2024-09-21T06:06:33.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.930+0000 7fab5924f740 1 Processor -- start 2024-09-21T06:06:33.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.930+0000 7fab5924f740 1 -- start start 2024-09-21T06:06:33.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.930+0000 7fab5924f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b3ddae1b0 0x555b3e18bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:33.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.930+0000 7fab5924f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b3ddb6390 0x555b3e17fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:33.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.930+0000 7fab5924f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3ddd7070 0x555b3e186fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:33.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.930+0000 7fab5924f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c1ad4d0 con 0x555b3ddb6390 2024-09-21T06:06:33.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.930+0000 7fab5924f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c1af010 con 0x555b3ddd7070 2024-09-21T06:06:33.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.930+0000 7fab5924f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c1adba0 con 0x555b3ddae1b0 2024-09-21T06:06:33.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.930+0000 7fab4bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b3ddb6390 0x555b3e17fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:33.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.930+0000 7fab4bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b3ddb6390 0x555b3e17fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55380/0 (socket says 172.21.15.117:55380) 2024-09-21T06:06:33.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.930+0000 7fab4bfff640 1 -- 172.21.15.117:0/502619103 learned_addr learned my addr 172.21.15.117:0/502619103 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:33.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.931+0000 7fab51278640 1 --2- 172.21.15.117:0/502619103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b3ddae1b0 0x555b3e18bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:33.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.931+0000 7fab4bfff640 1 -- 172.21.15.117:0/502619103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b3ddae1b0 msgr2=0x555b3e18bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:33.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.931+0000 7fab51278640 1 --2- 172.21.15.117:0/502619103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3ddd7070 0x555b3e186fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:33.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.931+0000 7fab4bfff640 1 --2- 172.21.15.117:0/502619103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b3ddae1b0 0x555b3e18bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.931+0000 7fab4bfff640 1 -- 172.21.15.117:0/502619103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3ddd7070 msgr2=0x555b3e186fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:33.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.931+0000 7fab4bfff640 1 --2- 172.21.15.117:0/502619103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3ddd7070 0x555b3e186fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.931+0000 7fab4bfff640 1 -- 172.21.15.117:0/502619103 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b3dd95c20 con 0x555b3ddb6390 2024-09-21T06:06:33.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.931+0000 7fab51278640 1 --2- 172.21.15.117:0/502619103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3ddd7070 0x555b3e186fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:06:33.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.931+0000 7fab4bfff640 1 --2- 172.21.15.117:0/502619103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b3ddb6390 0x555b3e17fd90 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fab4c06c9f0 tx=0x7fab40008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73cb692e0fa0b051 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:33.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.931+0000 7fab327fc640 1 -- 172.21.15.117:0/502619103 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab40020030 con 0x555b3ddb6390 2024-09-21T06:06:33.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.931+0000 7fab327fc640 1 -- 172.21.15.117:0/502619103 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab400243c0 con 0x555b3ddb6390 2024-09-21T06:06:33.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.931+0000 7fab327fc640 1 -- 172.21.15.117:0/502619103 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab400143b0 con 0x555b3ddb6390 2024-09-21T06:06:33.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.932+0000 7fab5924f740 1 -- 172.21.15.117:0/502619103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b3ddb6390 msgr2=0x555b3e17fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:33.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.932+0000 7fab5924f740 1 --2- 172.21.15.117:0/502619103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b3ddb6390 0x555b3e17fd90 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fab4c06c9f0 tx=0x7fab40008040 comp rx=0 tx=0).stop 2024-09-21T06:06:33.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.932+0000 7fab5924f740 1 -- 172.21.15.117:0/502619103 shutdown_connections 2024-09-21T06:06:33.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.932+0000 7fab5924f740 1 --2- 172.21.15.117:0/502619103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b3ddb6390 0x555b3e17fd90 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.932+0000 7fab5924f740 1 --2- 172.21.15.117:0/502619103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3ddd7070 0x555b3e186fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.932+0000 7fab5924f740 1 --2- 172.21.15.117:0/502619103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b3ddae1b0 0x555b3e18bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.932+0000 7fab5924f740 1 -- 172.21.15.117:0/502619103 >> 172.21.15.117:0/502619103 conn(0x555b3ddcb460 msgr2=0x555b3e17f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:33.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.932+0000 7fab5924f740 1 -- 172.21.15.117:0/502619103 shutdown_connections 2024-09-21T06:06:33.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.932+0000 7fab5924f740 1 -- 172.21.15.117:0/502619103 wait complete. 2024-09-21T06:06:33.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.932+0000 7fab5924f740 1 -- start start 2024-09-21T06:06:33.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.933+0000 7fab5924f740 1 Processor -- start 2024-09-21T06:06:33.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.933+0000 7fab5924f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3ddae1b0 0x555b3e18eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:33.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.933+0000 7fab5924f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b3ddb6390 0x555b3e18f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:33.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.933+0000 7fab5924f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b3ddd7070 0x555b3e20d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:33.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.933+0000 7fab5924f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c1ae940 con 0x555b3ddb6390 2024-09-21T06:06:33.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.933+0000 7fab5924f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c1ae270 con 0x555b3ddae1b0 2024-09-21T06:06:33.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.933+0000 7fab5924f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c1ab4d0 con 0x555b3ddd7070 2024-09-21T06:06:33.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.933+0000 7fab5924f740 10 client.0 _ll_register_callbacks cb 0x7fab44009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:33.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.933+0000 7fab4bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b3ddb6390 0x555b3e18f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:33.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.933+0000 7fab4bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b3ddb6390 0x555b3e18f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:55384/0 (socket says 172.21.15.117:55384) 2024-09-21T06:06:33.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.933+0000 7fab4bfff640 1 -- 172.21.15.117:0/1567795480 learned_addr learned my addr 172.21.15.117:0/1567795480 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:33.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.933+0000 7fab51278640 1 --2- 172.21.15.117:0/1567795480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3ddae1b0 0x555b3e18eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:33.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.933+0000 7fab51278640 1 --2- 172.21.15.117:0/1567795480 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b3ddd7070 0x555b3e20d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:33.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.933+0000 7fab4bfff640 1 -- 172.21.15.117:0/1567795480 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b3ddd7070 msgr2=0x555b3e20d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:33.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.933+0000 7fab4bfff640 1 --2- 172.21.15.117:0/1567795480 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b3ddd7070 0x555b3e20d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.933+0000 7fab4bfff640 1 -- 172.21.15.117:0/1567795480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3ddae1b0 msgr2=0x555b3e18eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:33.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.933+0000 7fab4bfff640 1 --2- 172.21.15.117:0/1567795480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3ddae1b0 0x555b3e18eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.933+0000 7fab4bfff640 1 -- 172.21.15.117:0/1567795480 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b3e20dba0 con 0x555b3ddb6390 2024-09-21T06:06:33.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.933+0000 7fab51278640 1 --2- 172.21.15.117:0/1567795480 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b3ddd7070 0x555b3e20d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:06:33.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.934+0000 7fab4bfff640 1 --2- 172.21.15.117:0/1567795480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b3ddb6390 0x555b3e18f5b0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fab40004810 tx=0x7fab40024600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca0e1e21a199896 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:33.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.934+0000 7fab2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:33.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.934+0000 7fab2bfff640 1 -- 172.21.15.117:0/1567795480 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab40029020 con 0x555b3ddb6390 2024-09-21T06:06:33.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.934+0000 7fab5924f740 1 -- 172.21.15.117:0/1567795480 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab400243c0 con 0x555b3ddb6390 2024-09-21T06:06:33.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.934+0000 7fab2bfff640 1 -- 172.21.15.117:0/1567795480 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fab40015b40 con 0x555b3ddb6390 2024-09-21T06:06:33.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.934+0000 7fab2bfff640 1 -- 172.21.15.117:0/1567795480 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fab40034050 con 0x555b3ddb6390 2024-09-21T06:06:33.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.934+0000 7fab5924f740 10 client.17379 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:33.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.934+0000 7fab5924f740 1 -- 172.21.15.117:0/1567795480 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555b3e20ed70 con 0x555b3ddb6390 2024-09-21T06:06:33.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.934+0000 7fab2bfff640 1 -- 172.21.15.117:0/1567795480 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab400243c0 con 0x555b3ddb6390 2024-09-21T06:06:33.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.934+0000 7fab2bfff640 1 -- 172.21.15.117:0/1567795480 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fab4002d3a0 con 0x555b3ddb6390 2024-09-21T06:06:33.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.934+0000 7fab5924f740 10 client.17379 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:33.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.934+0000 7fab5924f740 10 client.17379 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T06:06:33.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.934+0000 7fab5924f740 20 client.17379 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:33.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.935+0000 7fab5924f740 1 --2- 172.21.15.117:0/1567795480 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x555b3dd6ada0 0x555b3e211440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:33.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.935+0000 7fab5924f740 4 client.17379 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:06:33.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.935+0000 7fab5924f740 1 -- 172.21.15.117:0/1567795480 --> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555b3e0f28a0 con 0x555b3dd6ada0 2024-09-21T06:06:33.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.935+0000 7fab51278640 1 --2- 172.21.15.117:0/1567795480 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x555b3dd6ada0 0x555b3e211440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:33.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.935+0000 7fab51278640 1 --2- 172.21.15.117:0/1567795480 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x555b3dd6ada0 0x555b3e211440 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=93e022cdf09bdcc6 server_cookie=e1180efee0a44a17 in_seq=0 out_seq=0 2024-09-21T06:06:33.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.935+0000 7fab2bfff640 10 client.17379 ms_handle_connect on v2:172.21.15.138:6836/2540335374 2024-09-21T06:06:33.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.935+0000 7fab2bfff640 1 -- 172.21.15.117:0/1567795480 <== mds.4 v2:172.21.15.138:6836/2540335374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555b3e0f28a0 con 0x555b3dd6ada0 2024-09-21T06:06:33.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.935+0000 7fab2bfff640 10 client.17379 handle_command_reply: tid=0 2024-09-21T06:06:33.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.935+0000 7fab5924f740 1 client.17379 shutdown 2024-09-21T06:06:33.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.936+0000 7fab5924f740 1 -- 172.21.15.117:0/1567795480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b3ddb6390 msgr2=0x555b3e18f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:33.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.936+0000 7fab5924f740 1 --2- 172.21.15.117:0/1567795480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b3ddb6390 0x555b3e18f5b0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fab40004810 tx=0x7fab40024600 comp rx=0 tx=0).stop 2024-09-21T06:06:33.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.936+0000 7fab5924f740 1 -- 172.21.15.117:0/1567795480 shutdown_connections 2024-09-21T06:06:33.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.936+0000 7fab5924f740 1 --2- 172.21.15.117:0/1567795480 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x555b3dd6ada0 0x555b3e211440 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.936+0000 7fab5924f740 1 --2- 172.21.15.117:0/1567795480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555b3ddb6390 0x555b3e18f5b0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.936+0000 7fab2bfff640 0 client.17379 ms_handle_reset on v2:172.21.15.138:6836/2540335374 2024-09-21T06:06:33.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.936+0000 7fab5924f740 1 --2- 172.21.15.117:0/1567795480 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b3ddd7070 0x555b3e20d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.936+0000 7fab5924f740 1 --2- 172.21.15.117:0/1567795480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3ddae1b0 0x555b3e18eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.936+0000 7fab5924f740 1 -- 172.21.15.117:0/1567795480 >> 172.21.15.117:0/1567795480 conn(0x555b3ddcb460 msgr2=0x555b3e18ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:33.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.936+0000 7fab5924f740 1 -- 172.21.15.117:0/1567795480 shutdown_connections 2024-09-21T06:06:33.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.936+0000 7fab5924f740 1 -- 172.21.15.117:0/1567795480 wait complete. 2024-09-21T06:06:33.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.936+0000 7fab5924f740 20 client.17379 trim_cache size 0 max 16384 2024-09-21T06:06:33.942 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:06:33.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.937+0000 7fab52cdb640 1 -- 172.21.15.117:0/1133582813 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fab340b6680 msgr2=0x7fab340b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:33.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.937+0000 7fab52cdb640 1 --2- 172.21.15.117:0/1133582813 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fab340b6680 0x7fab340b8b90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x555b3e17d320 tx=0x7fab3c09b140 comp rx=0 tx=0).stop 2024-09-21T06:06:33.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.937+0000 7fab52cdb640 1 -- 172.21.15.117:0/1133582813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab4c103b60 msgr2=0x7fab4c19a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:33.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.937+0000 7fab52cdb640 1 --2- 172.21.15.117:0/1133582813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab4c103b60 0x7fab4c19a500 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fab3c005940 tx=0x7fab3c0042f0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.937+0000 7fab52cdb640 1 -- 172.21.15.117:0/1133582813 shutdown_connections 2024-09-21T06:06:33.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.937+0000 7fab52cdb640 1 --2- 172.21.15.117:0/1133582813 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fab340b6680 0x7fab340b8b90 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.937+0000 7fab52cdb640 1 --2- 172.21.15.117:0/1133582813 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab4c109df0 0x7fab4c19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.937+0000 7fab52cdb640 1 --2- 172.21.15.117:0/1133582813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab4c1094a0 0x7fab4c19aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.937+0000 7fab52cdb640 1 --2- 172.21.15.117:0/1133582813 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab4c103b60 0x7fab4c19a500 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:33.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.937+0000 7fab52cdb640 1 -- 172.21.15.117:0/1133582813 >> 172.21.15.117:0/1133582813 conn(0x7fab4c07b850 msgr2=0x7fab4c107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:33.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.937+0000 7fab52cdb640 1 -- 172.21.15.117:0/1133582813 shutdown_connections 2024-09-21T06:06:33.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:33.937+0000 7fab52cdb640 1 -- 172.21.15.117:0/1133582813 wait complete. 2024-09-21T06:06:33.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:33 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3893107448' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:06:33.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:33 smithi192 ceph-mon[34700]: pgmap v609: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-21T06:06:34.714 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T06:06:34.963 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:35.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.464+0000 7fef3338a640 1 -- 172.21.15.117:0/3575725491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef2c1062d0 msgr2=0x7fef2c1066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:35.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.464+0000 7fef3338a640 1 --2- 172.21.15.117:0/3575725491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef2c1062d0 0x7fef2c1066e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fef20009900 tx=0x7fef2002a410 comp rx=0 tx=0).stop 2024-09-21T06:06:35.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.465+0000 7fef3338a640 1 -- 172.21.15.117:0/3575725491 shutdown_connections 2024-09-21T06:06:35.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.465+0000 7fef3338a640 1 --2- 172.21.15.117:0/3575725491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef2c1062d0 0x7fef2c1066e0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.465+0000 7fef3338a640 1 --2- 172.21.15.117:0/3575725491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef2c106c20 0x7fef2c110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.465+0000 7fef3338a640 1 --2- 172.21.15.117:0/3575725491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef2c101ff0 0x7fef2c102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.485 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.465+0000 7fef3338a640 1 -- 172.21.15.117:0/3575725491 >> 172.21.15.117:0/3575725491 conn(0x7fef2c079e30 msgr2=0x7fef2c07a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:35.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.465+0000 7fef3338a640 1 -- 172.21.15.117:0/3575725491 shutdown_connections 2024-09-21T06:06:35.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.465+0000 7fef3338a640 1 -- 172.21.15.117:0/3575725491 wait complete. 2024-09-21T06:06:35.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.465+0000 7fef3338a640 1 Processor -- start 2024-09-21T06:06:35.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.465+0000 7fef3338a640 1 -- start start 2024-09-21T06:06:35.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.466+0000 7fef3338a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef2c101ff0 0x7fef2c102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:35.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.466+0000 7fef3338a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef2c1062d0 0x7fef2c1034d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:35.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.466+0000 7fef3338a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef2c106c20 0x7fef2c103a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:35.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.466+0000 7fef3338a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fef2c113a80 con 0x7fef2c1062d0 2024-09-21T06:06:35.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.466+0000 7fef3338a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fef2c113900 con 0x7fef2c106c20 2024-09-21T06:06:35.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.466+0000 7fef3338a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fef2c113c00 con 0x7fef2c101ff0 2024-09-21T06:06:35.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.466+0000 7fef30925640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef2c1062d0 0x7fef2c1034d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:35.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.466+0000 7fef30925640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef2c1062d0 0x7fef2c1034d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53378/0 (socket says 172.21.15.117:53378) 2024-09-21T06:06:35.486 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.466+0000 7fef30925640 1 -- 172.21.15.117:0/1273100886 learned_addr learned my addr 172.21.15.117:0/1273100886 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:35.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.466+0000 7fef31927640 1 --2- 172.21.15.117:0/1273100886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef2c106c20 0x7fef2c103a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:35.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.466+0000 7fef31126640 1 --2- 172.21.15.117:0/1273100886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef2c101ff0 0x7fef2c102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:35.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.466+0000 7fef30925640 1 -- 172.21.15.117:0/1273100886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef2c101ff0 msgr2=0x7fef2c102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:35.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.466+0000 7fef30925640 1 --2- 172.21.15.117:0/1273100886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef2c101ff0 0x7fef2c102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.466+0000 7fef30925640 1 -- 172.21.15.117:0/1273100886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef2c106c20 msgr2=0x7fef2c103a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:35.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.466+0000 7fef30925640 1 --2- 172.21.15.117:0/1273100886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef2c106c20 0x7fef2c103a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.466+0000 7fef30925640 1 -- 172.21.15.117:0/1273100886 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef2c1a8730 con 0x7fef2c1062d0 2024-09-21T06:06:35.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.466+0000 7fef30925640 1 --2- 172.21.15.117:0/1273100886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef2c1062d0 0x7fef2c1034d0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fef20032fd0 tx=0x7fef20004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:35.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.466+0000 7fef1e7fc640 1 -- 172.21.15.117:0/1273100886 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2003f030 con 0x7fef2c1062d0 2024-09-21T06:06:35.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.466+0000 7fef3338a640 1 -- 172.21.15.117:0/1273100886 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef2c1a89c0 con 0x7fef2c1062d0 2024-09-21T06:06:35.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.466+0000 7fef1e7fc640 1 -- 172.21.15.117:0/1273100886 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef20004490 con 0x7fef2c1062d0 2024-09-21T06:06:35.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.467+0000 7fef1e7fc640 1 -- 172.21.15.117:0/1273100886 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef20035070 con 0x7fef2c1062d0 2024-09-21T06:06:35.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.467+0000 7fef3338a640 1 -- 172.21.15.117:0/1273100886 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef2c1a8ef0 con 0x7fef2c1062d0 2024-09-21T06:06:35.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.470+0000 7fef398fe740 1 Processor -- start 2024-09-21T06:06:35.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.470+0000 7fef398fe740 1 -- start start 2024-09-21T06:06:35.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.470+0000 7fef398fe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce6391d070 0x55ce63cc46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:35.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.470+0000 7fef398fe740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce638f41b0 0x55ce63cc4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:35.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.470+0000 7fef398fe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce638fc390 0x55ce63cc5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:35.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.470+0000 7fef398fe740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ce63c3aea0 con 0x55ce638fc390 2024-09-21T06:06:35.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.470+0000 7fef398fe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ce63c2a2d0 con 0x55ce6391d070 2024-09-21T06:06:35.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.470+0000 7fef398fe740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55ce63c2e0e0 con 0x55ce638f41b0 2024-09-21T06:06:35.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.470+0000 7fef1e7fc640 1 -- 172.21.15.117:0/1273100886 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fef20042030 con 0x7fef2c1062d0 2024-09-21T06:06:35.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.470+0000 7fef31927640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce6391d070 0x55ce63cc46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:35.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.470+0000 7fef31927640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce6391d070 0x55ce63cc46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:58348/0 (socket says 172.21.15.117:58348) 2024-09-21T06:06:35.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.470+0000 7fef31927640 1 -- 172.21.15.117:0/3993420041 learned_addr learned my addr 172.21.15.117:0/3993420041 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:35.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.470+0000 7fef30925640 1 --2- 172.21.15.117:0/3993420041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce638fc390 0x55ce63cc5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:35.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.470+0000 7fef31126640 1 --2- 172.21.15.117:0/3993420041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce638f41b0 0x55ce63cc4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:35.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.471+0000 7fef31927640 1 -- 172.21.15.117:0/3993420041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce638f41b0 msgr2=0x55ce63cc4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:35.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.471+0000 7fef31927640 1 --2- 172.21.15.117:0/3993420041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce638f41b0 0x55ce63cc4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.471+0000 7fef31927640 1 -- 172.21.15.117:0/3993420041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce638fc390 msgr2=0x55ce63cc5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:35.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.471+0000 7fef31927640 1 --2- 172.21.15.117:0/3993420041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce638fc390 0x55ce63cc5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.471+0000 7fef31927640 1 -- 172.21.15.117:0/3993420041 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce63988650 con 0x55ce6391d070 2024-09-21T06:06:35.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.471+0000 7fef31927640 1 --2- 172.21.15.117:0/3993420041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce6391d070 0x55ce63cc46a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fef240039f0 tx=0x7fef24003eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b0fbd9d8095dc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:35.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.471+0000 7fef177fe640 1 -- 172.21.15.117:0/3993420041 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2401a470 con 0x55ce6391d070 2024-09-21T06:06:35.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.471+0000 7fef177fe640 1 -- 172.21.15.117:0/3993420041 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef24015070 con 0x55ce6391d070 2024-09-21T06:06:35.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.471+0000 7fef177fe640 1 -- 172.21.15.117:0/3993420041 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef240107e0 con 0x55ce6391d070 2024-09-21T06:06:35.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.471+0000 7fef1e7fc640 1 --2- 172.21.15.117:0/1273100886 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fef180b6680 0x7fef180b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:35.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.471+0000 7fef31126640 1 --2- 172.21.15.117:0/1273100886 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fef180b6680 0x7fef180b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:35.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.472+0000 7fef1e7fc640 1 -- 172.21.15.117:0/1273100886 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fef20107c10 con 0x7fef2c1062d0 2024-09-21T06:06:35.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.472+0000 7fef398fe740 1 -- 172.21.15.117:0/3993420041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce6391d070 msgr2=0x55ce63cc46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:35.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.472+0000 7fef31126640 1 --2- 172.21.15.117:0/1273100886 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fef180b6680 0x7fef180b8b90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x55ce63cc3320 tx=0x7fef2800b500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:35.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.472+0000 7fef398fe740 1 --2- 172.21.15.117:0/3993420041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce6391d070 0x55ce63cc46a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fef240039f0 tx=0x7fef24003eb0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.472+0000 7fef398fe740 1 -- 172.21.15.117:0/3993420041 shutdown_connections 2024-09-21T06:06:35.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.472+0000 7fef398fe740 1 --2- 172.21.15.117:0/3993420041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce638fc390 0x55ce63cc5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.472+0000 7fef398fe740 1 --2- 172.21.15.117:0/3993420041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce638f41b0 0x55ce63cc4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.472+0000 7fef398fe740 1 --2- 172.21.15.117:0/3993420041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce6391d070 0x55ce63cc46a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.472+0000 7fef398fe740 1 -- 172.21.15.117:0/3993420041 >> 172.21.15.117:0/3993420041 conn(0x55ce63911460 msgr2=0x55ce63cc2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:35.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.472+0000 7fef398fe740 1 -- 172.21.15.117:0/3993420041 shutdown_connections 2024-09-21T06:06:35.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.472+0000 7fef398fe740 1 -- 172.21.15.117:0/3993420041 wait complete. 2024-09-21T06:06:35.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.473+0000 7fef398fe740 1 -- start start 2024-09-21T06:06:35.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.473+0000 7fef398fe740 1 Processor -- start 2024-09-21T06:06:35.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.473+0000 7fef398fe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce638f41b0 0x55ce63ccfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:35.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.473+0000 7fef398fe740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce638fc390 0x55ce63cd0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:35.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.473+0000 7fef398fe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce6391d070 0x55ce63cdc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:35.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.473+0000 7fef398fe740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ce63a8b700 con 0x55ce638f41b0 2024-09-21T06:06:35.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.473+0000 7fef398fe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ce63bc5160 con 0x55ce6391d070 2024-09-21T06:06:35.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.473+0000 7fef398fe740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55ce63cd2e80 con 0x55ce638fc390 2024-09-21T06:06:35.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.473+0000 7fef398fe740 10 client.0 _ll_register_callbacks cb 0x55ce63bde9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:35.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.473+0000 7fef31126640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce638f41b0 0x55ce63ccfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:35.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.473+0000 7fef31126640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce638f41b0 0x55ce63ccfd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53398/0 (socket says 172.21.15.117:53398) 2024-09-21T06:06:35.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.473+0000 7fef31126640 1 -- 172.21.15.117:0/1015956987 learned_addr learned my addr 172.21.15.117:0/1015956987 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:35.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.473+0000 7fef31927640 1 --2- 172.21.15.117:0/1015956987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce6391d070 0x55ce63cdc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:35.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.473+0000 7fef30925640 1 --2- 172.21.15.117:0/1015956987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce638fc390 0x55ce63cd0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:35.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.474+0000 7fef31126640 1 -- 172.21.15.117:0/1015956987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce638fc390 msgr2=0x55ce63cd0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:35.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.474+0000 7fef31126640 1 --2- 172.21.15.117:0/1015956987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce638fc390 0x55ce63cd0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.474+0000 7fef31126640 1 -- 172.21.15.117:0/1015956987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce6391d070 msgr2=0x55ce63cdc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:35.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.474+0000 7fef31126640 1 --2- 172.21.15.117:0/1015956987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce6391d070 0x55ce63cdc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.474+0000 7fef31126640 1 -- 172.21.15.117:0/1015956987 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce63bf2c90 con 0x55ce638f41b0 2024-09-21T06:06:35.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.474+0000 7fef30925640 1 --2- 172.21.15.117:0/1015956987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce638fc390 0x55ce63cd0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:06:35.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.474+0000 7fef31126640 1 --2- 172.21.15.117:0/1015956987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce638f41b0 0x55ce63ccfd40 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fef2800b4d0 tx=0x7fef280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=981fd0a9c0fbb6ba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:35.491 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.474+0000 7fef14ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:35.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.474+0000 7fef14ff9640 1 -- 172.21.15.117:0/1015956987 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef28023070 con 0x55ce638f41b0 2024-09-21T06:06:35.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.474+0000 7fef14ff9640 1 -- 172.21.15.117:0/1015956987 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fef280035f0 con 0x55ce638f41b0 2024-09-21T06:06:35.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.474+0000 7fef398fe740 1 -- 172.21.15.117:0/1015956987 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef24015070 con 0x55ce638f41b0 2024-09-21T06:06:35.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.474+0000 7fef14ff9640 1 -- 172.21.15.117:0/1015956987 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fef2802f050 con 0x55ce638f41b0 2024-09-21T06:06:35.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.474+0000 7fef398fe740 10 client.17391 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:35.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.474+0000 7fef398fe740 1 -- 172.21.15.117:0/1015956987 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ce63cdebd0 con 0x55ce638f41b0 2024-09-21T06:06:35.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.474+0000 7fef14ff9640 1 -- 172.21.15.117:0/1015956987 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef24015070 con 0x55ce638f41b0 2024-09-21T06:06:35.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.475+0000 7fef14ff9640 1 -- 172.21.15.117:0/1015956987 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fef2800f8e0 con 0x55ce638f41b0 2024-09-21T06:06:35.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.475+0000 7fef398fe740 10 client.17391 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:35.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.475+0000 7fef398fe740 10 client.17391 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T06:06:35.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.475+0000 7fef398fe740 20 client.17391 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:35.492 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.475+0000 7fef398fe740 1 --2- 172.21.15.117:0/1015956987 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55ce638db9f0 0x55ce63ce20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:35.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.475+0000 7fef398fe740 4 client.17391 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:06:35.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.475+0000 7fef398fe740 1 -- 172.21.15.117:0/1015956987 --> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce63c3cdf0 con 0x55ce638db9f0 2024-09-21T06:06:35.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.475+0000 7fef30925640 1 --2- 172.21.15.117:0/1015956987 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55ce638db9f0 0x55ce63ce20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:35.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.475+0000 7fef30925640 1 --2- 172.21.15.117:0/1015956987 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55ce638db9f0 0x55ce63ce20c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=18069af2d3337d8a server_cookie=183d067d3bf6734b in_seq=0 out_seq=0 2024-09-21T06:06:35.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.475+0000 7fef14ff9640 10 client.17391 ms_handle_connect on v2:172.21.15.192:6834/3530688860 2024-09-21T06:06:35.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.476+0000 7fef14ff9640 1 -- 172.21.15.117:0/1015956987 <== mds.3 v2:172.21.15.192:6834/3530688860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ce63c3cdf0 con 0x55ce638db9f0 2024-09-21T06:06:35.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.476+0000 7fef14ff9640 10 client.17391 handle_command_reply: tid=0 2024-09-21T06:06:35.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.476+0000 7fef398fe740 1 client.17391 shutdown 2024-09-21T06:06:35.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.476+0000 7fef398fe740 1 -- 172.21.15.117:0/1015956987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce638f41b0 msgr2=0x55ce63ccfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:35.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.476+0000 7fef398fe740 1 --2- 172.21.15.117:0/1015956987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce638f41b0 0x55ce63ccfd40 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fef2800b4d0 tx=0x7fef280046c0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.476+0000 7fef398fe740 1 -- 172.21.15.117:0/1015956987 shutdown_connections 2024-09-21T06:06:35.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.476+0000 7fef398fe740 1 --2- 172.21.15.117:0/1015956987 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55ce638db9f0 0x55ce63ce20c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.476+0000 7fef398fe740 1 --2- 172.21.15.117:0/1015956987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce6391d070 0x55ce63cdc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.476+0000 7fef398fe740 1 --2- 172.21.15.117:0/1015956987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce638fc390 0x55ce63cd0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.476+0000 7fef398fe740 1 --2- 172.21.15.117:0/1015956987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce638f41b0 0x55ce63ccfd40 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.476+0000 7fef14ff9640 0 client.17391 ms_handle_reset on v2:172.21.15.192:6834/3530688860 2024-09-21T06:06:35.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.476+0000 7fef398fe740 1 -- 172.21.15.117:0/1015956987 >> 172.21.15.117:0/1015956987 conn(0x55ce63911460 msgr2=0x55ce63cc7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:35.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.477+0000 7fef398fe740 1 -- 172.21.15.117:0/1015956987 shutdown_connections 2024-09-21T06:06:35.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.477+0000 7fef398fe740 1 -- 172.21.15.117:0/1015956987 wait complete. 2024-09-21T06:06:35.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.477+0000 7fef398fe740 20 client.17391 trim_cache size 0 max 16384 2024-09-21T06:06:35.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.490+0000 7fef398fe740 1 Processor -- start 2024-09-21T06:06:35.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.490+0000 7fef398fe740 1 -- start start 2024-09-21T06:06:35.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.491+0000 7fef398fe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce638f41b0 0x55ce63cc6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:35.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.491+0000 7fef398fe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce638fc390 0x55ce63cd1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:35.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.491+0000 7fef398fe740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce6391d070 0x55ce63cc1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:35.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.491+0000 7fef398fe740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fef2c1b1840 con 0x55ce638fc390 2024-09-21T06:06:35.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.491+0000 7fef398fe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fef2c1b3380 con 0x55ce638f41b0 2024-09-21T06:06:35.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.491+0000 7fef398fe740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fef2c1b1f10 con 0x55ce6391d070 2024-09-21T06:06:35.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.491+0000 7fef30925640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce638fc390 0x55ce63cd1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:35.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.491+0000 7fef30925640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce638fc390 0x55ce63cd1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53404/0 (socket says 172.21.15.117:53404) 2024-09-21T06:06:35.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.491+0000 7fef30925640 1 -- 172.21.15.117:0/401089056 learned_addr learned my addr 172.21.15.117:0/401089056 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:35.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.491+0000 7fef31126640 1 --2- 172.21.15.117:0/401089056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce638f41b0 0x55ce63cc6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:35.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.491+0000 7fef31927640 1 --2- 172.21.15.117:0/401089056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce6391d070 0x55ce63cc1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:35.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.491+0000 7fef30925640 1 -- 172.21.15.117:0/401089056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce6391d070 msgr2=0x55ce63cc1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:35.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.491+0000 7fef30925640 1 --2- 172.21.15.117:0/401089056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce6391d070 0x55ce63cc1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.491+0000 7fef30925640 1 -- 172.21.15.117:0/401089056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce638f41b0 msgr2=0x55ce63cc6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:35.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.491+0000 7fef30925640 1 --2- 172.21.15.117:0/401089056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce638f41b0 0x55ce63cc6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.491+0000 7fef30925640 1 -- 172.21.15.117:0/401089056 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce63cc7500 con 0x55ce638fc390 2024-09-21T06:06:35.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.491+0000 7fef31126640 1 --2- 172.21.15.117:0/401089056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce638f41b0 0x55ce63cc6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:06:35.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.491+0000 7fef30925640 1 --2- 172.21.15.117:0/401089056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce638fc390 0x55ce63cd1970 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fef200cda40 tx=0x7fef2010b750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e14247e2080bf94 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:35.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.491+0000 7fef16ffd640 1 -- 172.21.15.117:0/401089056 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef20049040 con 0x55ce638fc390 2024-09-21T06:06:35.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.492+0000 7fef16ffd640 1 -- 172.21.15.117:0/401089056 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef200cd5c0 con 0x55ce638fc390 2024-09-21T06:06:35.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.492+0000 7fef16ffd640 1 -- 172.21.15.117:0/401089056 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef200c9220 con 0x55ce638fc390 2024-09-21T06:06:35.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.492+0000 7fef398fe740 1 -- 172.21.15.117:0/401089056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce638fc390 msgr2=0x55ce63cd1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:35.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.492+0000 7fef398fe740 1 --2- 172.21.15.117:0/401089056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce638fc390 0x55ce63cd1970 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fef200cda40 tx=0x7fef2010b750 comp rx=0 tx=0).stop 2024-09-21T06:06:35.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.492+0000 7fef398fe740 1 -- 172.21.15.117:0/401089056 shutdown_connections 2024-09-21T06:06:35.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.492+0000 7fef398fe740 1 --2- 172.21.15.117:0/401089056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce638fc390 0x55ce63cd1970 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.492+0000 7fef398fe740 1 --2- 172.21.15.117:0/401089056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce6391d070 0x55ce63cc1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.492+0000 7fef398fe740 1 --2- 172.21.15.117:0/401089056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce638f41b0 0x55ce63cc6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.492+0000 7fef398fe740 1 -- 172.21.15.117:0/401089056 >> 172.21.15.117:0/401089056 conn(0x55ce63911460 msgr2=0x55ce63cd1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:35.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.493+0000 7fef398fe740 1 -- 172.21.15.117:0/401089056 shutdown_connections 2024-09-21T06:06:35.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.493+0000 7fef398fe740 1 -- 172.21.15.117:0/401089056 wait complete. 2024-09-21T06:06:35.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.493+0000 7fef398fe740 1 -- start start 2024-09-21T06:06:35.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.493+0000 7fef398fe740 1 Processor -- start 2024-09-21T06:06:35.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.494+0000 7fef398fe740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce638f41b0 0x55ce63cd40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:35.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.494+0000 7fef398fe740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce638fc390 0x55ce63cd4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:35.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.494+0000 7fef398fe740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce6391d070 0x55ce63cd4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:35.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.494+0000 7fef398fe740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fef2c1b2cb0 con 0x55ce638fc390 2024-09-21T06:06:35.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.494+0000 7fef398fe740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fef2c1b25e0 con 0x55ce638f41b0 2024-09-21T06:06:35.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.494+0000 7fef398fe740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fef2c1af840 con 0x55ce6391d070 2024-09-21T06:06:35.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.494+0000 7fef398fe740 10 client.0 _ll_register_callbacks cb 0x7fef2800b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:35.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.494+0000 7fef30925640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce638fc390 0x55ce63cd4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:35.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.494+0000 7fef30925640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce638fc390 0x55ce63cd4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53406/0 (socket says 172.21.15.117:53406) 2024-09-21T06:06:35.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.494+0000 7fef30925640 1 -- 172.21.15.117:0/1852491369 learned_addr learned my addr 172.21.15.117:0/1852491369 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:35.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.494+0000 7fef31126640 1 --2- 172.21.15.117:0/1852491369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce638f41b0 0x55ce63cd40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:35.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.494+0000 7fef31927640 1 --2- 172.21.15.117:0/1852491369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce6391d070 0x55ce63cd4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:35.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.494+0000 7fef30925640 1 -- 172.21.15.117:0/1852491369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce6391d070 msgr2=0x55ce63cd4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:35.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.494+0000 7fef30925640 1 --2- 172.21.15.117:0/1852491369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce6391d070 0x55ce63cd4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.494+0000 7fef30925640 1 -- 172.21.15.117:0/1852491369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce638f41b0 msgr2=0x55ce63cd40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:35.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.494+0000 7fef30925640 1 --2- 172.21.15.117:0/1852491369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce638f41b0 0x55ce63cd40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.494+0000 7fef30925640 1 -- 172.21.15.117:0/1852491369 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce63d4bad0 con 0x55ce638fc390 2024-09-21T06:06:35.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.494+0000 7fef31927640 1 --2- 172.21.15.117:0/1852491369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce6391d070 0x55ce63cd4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:06:35.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.494+0000 7fef30925640 1 --2- 172.21.15.117:0/1852491369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce638fc390 0x55ce63cd4630 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fef20105460 tx=0x7fef20040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ef3e72f81f9fb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:35.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.494+0000 7fef14ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:35.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.494+0000 7fef14ff9640 1 -- 172.21.15.117:0/1852491369 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef200ca100 con 0x55ce638fc390 2024-09-21T06:06:35.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.494+0000 7fef14ff9640 1 -- 172.21.15.117:0/1852491369 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fef200c9d20 con 0x55ce638fc390 2024-09-21T06:06:35.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.494+0000 7fef398fe740 1 -- 172.21.15.117:0/1852491369 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef200cd5c0 con 0x55ce638fc390 2024-09-21T06:06:35.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.495+0000 7fef14ff9640 1 -- 172.21.15.117:0/1852491369 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fef200cd400 con 0x55ce638fc390 2024-09-21T06:06:35.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.495+0000 7fef398fe740 10 client.17397 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:35.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.495+0000 7fef398fe740 1 -- 172.21.15.117:0/1852491369 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ce63d4cd50 con 0x55ce638fc390 2024-09-21T06:06:35.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.495+0000 7fef14ff9640 1 -- 172.21.15.117:0/1852491369 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef200cd5c0 con 0x55ce638fc390 2024-09-21T06:06:35.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.495+0000 7fef14ff9640 1 -- 172.21.15.117:0/1852491369 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fef2010f040 con 0x55ce638fc390 2024-09-21T06:06:35.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.495+0000 7fef398fe740 10 client.17397 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:35.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.495+0000 7fef398fe740 10 client.17397 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T06:06:35.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.495+0000 7fef398fe740 20 client.17397 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:35.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.495+0000 7fef398fe740 1 --2- 172.21.15.117:0/1852491369 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55ce638b0da0 0x55ce63d4f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:35.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.495+0000 7fef398fe740 4 client.17397 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:06:35.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.495+0000 7fef398fe740 1 -- 172.21.15.117:0/1852491369 --> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ce63cd0630 con 0x55ce638b0da0 2024-09-21T06:06:35.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.495+0000 7fef31126640 1 --2- 172.21.15.117:0/1852491369 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55ce638b0da0 0x55ce63d4f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:35.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.496+0000 7fef31126640 1 --2- 172.21.15.117:0/1852491369 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55ce638b0da0 0x55ce63d4f4a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6feee1b0988c4994 server_cookie=31a93389f0493a21 in_seq=0 out_seq=0 2024-09-21T06:06:35.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.496+0000 7fef14ff9640 10 client.17397 ms_handle_connect on v2:172.21.15.192:6834/3530688860 2024-09-21T06:06:35.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.496+0000 7fef14ff9640 1 -- 172.21.15.117:0/1852491369 <== mds.3 v2:172.21.15.192:6834/3530688860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ce63cd0630 con 0x55ce638b0da0 2024-09-21T06:06:35.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.496+0000 7fef14ff9640 10 client.17397 handle_command_reply: tid=0 2024-09-21T06:06:35.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.496+0000 7fef398fe740 1 client.17397 shutdown 2024-09-21T06:06:35.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.496+0000 7fef398fe740 1 -- 172.21.15.117:0/1852491369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce638fc390 msgr2=0x55ce63cd4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:35.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.496+0000 7fef398fe740 1 --2- 172.21.15.117:0/1852491369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce638fc390 0x55ce63cd4630 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fef20105460 tx=0x7fef20040000 comp rx=0 tx=0).stop 2024-09-21T06:06:35.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.496+0000 7fef398fe740 1 -- 172.21.15.117:0/1852491369 shutdown_connections 2024-09-21T06:06:35.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.496+0000 7fef398fe740 1 --2- 172.21.15.117:0/1852491369 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55ce638b0da0 0x55ce63d4f4a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.496+0000 7fef398fe740 1 --2- 172.21.15.117:0/1852491369 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ce638fc390 0x55ce63cd4630 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.496+0000 7fef398fe740 1 --2- 172.21.15.117:0/1852491369 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ce6391d070 0x55ce63cd4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.496+0000 7fef14ff9640 0 client.17397 ms_handle_reset on v2:172.21.15.192:6834/3530688860 2024-09-21T06:06:35.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.496+0000 7fef398fe740 1 --2- 172.21.15.117:0/1852491369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce638f41b0 0x55ce63cd40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.496+0000 7fef398fe740 1 -- 172.21.15.117:0/1852491369 >> 172.21.15.117:0/1852491369 conn(0x55ce63911460 msgr2=0x55ce63cc5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:35.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.496+0000 7fef398fe740 1 -- 172.21.15.117:0/1852491369 shutdown_connections 2024-09-21T06:06:35.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.496+0000 7fef398fe740 1 -- 172.21.15.117:0/1852491369 wait complete. 2024-09-21T06:06:35.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.496+0000 7fef398fe740 20 client.17397 trim_cache size 0 max 16384 2024-09-21T06:06:35.506 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:06:35.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.497+0000 7fef3338a640 1 -- 172.21.15.117:0/1273100886 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fef180b6680 msgr2=0x7fef180b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:35.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.497+0000 7fef3338a640 1 --2- 172.21.15.117:0/1273100886 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fef180b6680 0x7fef180b8b90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x55ce63cc3320 tx=0x7fef2800b500 comp rx=0 tx=0).stop 2024-09-21T06:06:35.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.497+0000 7fef3338a640 1 -- 172.21.15.117:0/1273100886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef2c1062d0 msgr2=0x7fef2c1034d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:35.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.497+0000 7fef3338a640 1 --2- 172.21.15.117:0/1273100886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef2c1062d0 0x7fef2c1034d0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fef20032fd0 tx=0x7fef20004270 comp rx=0 tx=0).stop 2024-09-21T06:06:35.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.497+0000 7fef3338a640 1 -- 172.21.15.117:0/1273100886 shutdown_connections 2024-09-21T06:06:35.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.497+0000 7fef3338a640 1 --2- 172.21.15.117:0/1273100886 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fef180b6680 0x7fef180b8b90 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.497+0000 7fef3338a640 1 --2- 172.21.15.117:0/1273100886 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef2c1062d0 0x7fef2c1034d0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.497+0000 7fef3338a640 1 --2- 172.21.15.117:0/1273100886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef2c106c20 0x7fef2c103a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.497+0000 7fef3338a640 1 --2- 172.21.15.117:0/1273100886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef2c101ff0 0x7fef2c102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:35.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.497+0000 7fef3338a640 1 -- 172.21.15.117:0/1273100886 >> 172.21.15.117:0/1273100886 conn(0x7fef2c079e30 msgr2=0x7fef2c1075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:35.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.497+0000 7fef3338a640 1 -- 172.21.15.117:0/1273100886 shutdown_connections 2024-09-21T06:06:35.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:35.497+0000 7fef3338a640 1 -- 172.21.15.117:0/1273100886 wait complete. 2024-09-21T06:06:36.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:35 smithi192 ceph-mon[34700]: pgmap v610: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 341 B/s wr, 14 op/s 2024-09-21T06:06:36.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:35 smithi117 ceph-mon[30947]: pgmap v610: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 341 B/s wr, 14 op/s 2024-09-21T06:06:36.333 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T06:06:36.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:35 smithi138 ceph-mon[34391]: pgmap v610: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 341 B/s wr, 14 op/s 2024-09-21T06:06:36.578 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:36.985 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:36 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:06:37.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.076+0000 7f2e62fa3640 1 -- 172.21.15.117:0/210560268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e5c103b60 msgr2=0x7f2e5c103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:37.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.076+0000 7f2e62fa3640 1 --2- 172.21.15.117:0/210560268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e5c103b60 0x7f2e5c103f70 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c009900 tx=0x7f2e4c02a410 comp rx=0 tx=0).stop 2024-09-21T06:06:37.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.076+0000 7f2e62fa3640 1 -- 172.21.15.117:0/210560268 shutdown_connections 2024-09-21T06:06:37.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.076+0000 7f2e62fa3640 1 --2- 172.21.15.117:0/210560268 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e5c109df0 0x7f2e5c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.076+0000 7f2e62fa3640 1 --2- 172.21.15.117:0/210560268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e5c1094a0 0x7f2e5c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.078 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.076+0000 7f2e62fa3640 1 --2- 172.21.15.117:0/210560268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e5c103b60 0x7f2e5c103f70 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.076+0000 7f2e62fa3640 1 -- 172.21.15.117:0/210560268 >> 172.21.15.117:0/210560268 conn(0x7f2e5c07b850 msgr2=0x7f2e5c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:37.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.076+0000 7f2e62fa3640 1 -- 172.21.15.117:0/210560268 shutdown_connections 2024-09-21T06:06:37.079 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.076+0000 7f2e62fa3640 1 -- 172.21.15.117:0/210560268 wait complete. 2024-09-21T06:06:37.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.077+0000 7f2e62fa3640 1 Processor -- start 2024-09-21T06:06:37.080 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.077+0000 7f2e62fa3640 1 -- start start 2024-09-21T06:06:37.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.077+0000 7f2e62fa3640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e5c103b60 0x7f2e5c19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:37.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.077+0000 7f2e62fa3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e5c1094a0 0x7f2e5c19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:37.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.077+0000 7f2e62fa3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e5c109df0 0x7f2e5c19fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:37.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.077+0000 7f2e62fa3640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c113b00 con 0x7f2e5c109df0 2024-09-21T06:06:37.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.077+0000 7f2e62fa3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c113980 con 0x7f2e5c1094a0 2024-09-21T06:06:37.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.077+0000 7f2e62fa3640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c113c80 con 0x7f2e5c103b60 2024-09-21T06:06:37.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.077+0000 7f2e61540640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e5c109df0 0x7f2e5c19fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:37.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.077+0000 7f2e5bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e5c1094a0 0x7f2e5c19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:37.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.077+0000 7f2e61540640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e5c109df0 0x7f2e5c19fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53412/0 (socket says 172.21.15.117:53412) 2024-09-21T06:06:37.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.077+0000 7f2e61540640 1 -- 172.21.15.117:0/833239176 learned_addr learned my addr 172.21.15.117:0/833239176 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:37.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.077+0000 7f2e60d3f640 1 --2- 172.21.15.117:0/833239176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e5c103b60 0x7f2e5c19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:37.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.078+0000 7f2e61540640 1 -- 172.21.15.117:0/833239176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e5c103b60 msgr2=0x7f2e5c19a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:37.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.078+0000 7f2e61540640 1 --2- 172.21.15.117:0/833239176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e5c103b60 0x7f2e5c19a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.078+0000 7f2e61540640 1 -- 172.21.15.117:0/833239176 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e5c1094a0 msgr2=0x7f2e5c19aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:37.081 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.078+0000 7f2e61540640 1 --2- 172.21.15.117:0/833239176 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e5c1094a0 0x7f2e5c19aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.078+0000 7f2e61540640 1 -- 172.21.15.117:0/833239176 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e5c1a0480 con 0x7f2e5c109df0 2024-09-21T06:06:37.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.078+0000 7f2e5bfff640 1 --2- 172.21.15.117:0/833239176 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e5c1094a0 0x7f2e5c19aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:06:37.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.078+0000 7f2e61540640 1 --2- 172.21.15.117:0/833239176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e5c109df0 0x7f2e5c19fd50 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f2e54009fd0 tx=0x7f2e5400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:37.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.078+0000 7f2e59ffb640 1 -- 172.21.15.117:0/833239176 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e54017070 con 0x7f2e5c109df0 2024-09-21T06:06:37.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.078+0000 7f2e59ffb640 1 -- 172.21.15.117:0/833239176 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e54003e20 con 0x7f2e5c109df0 2024-09-21T06:06:37.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.078+0000 7f2e62fa3640 1 -- 172.21.15.117:0/833239176 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e5c1a0770 con 0x7f2e5c109df0 2024-09-21T06:06:37.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.078+0000 7f2e59ffb640 1 -- 172.21.15.117:0/833239176 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e54010460 con 0x7f2e5c109df0 2024-09-21T06:06:37.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.078+0000 7f2e62fa3640 1 -- 172.21.15.117:0/833239176 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e5c1a6ed0 con 0x7f2e5c109df0 2024-09-21T06:06:37.082 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.081+0000 7f2e69517740 1 Processor -- start 2024-09-21T06:06:37.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.081+0000 7f2e59ffb640 1 -- 172.21.15.117:0/833239176 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f2e54003970 con 0x7f2e5c109df0 2024-09-21T06:06:37.083 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.081+0000 7f2e69517740 1 -- start start 2024-09-21T06:06:37.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.081+0000 7f2e69517740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a7030b6070 0x55a70345d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:37.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.081+0000 7f2e69517740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70308d1b0 0x55a70345dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:37.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.081+0000 7f2e69517740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a703095390 0x55a70345e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:37.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.081+0000 7f2e69517740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a7033d3ea0 con 0x55a70308d1b0 2024-09-21T06:06:37.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.081+0000 7f2e69517740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a7033c32d0 con 0x55a703095390 2024-09-21T06:06:37.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.081+0000 7f2e69517740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a7033c70e0 con 0x55a7030b6070 2024-09-21T06:06:37.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.081+0000 7f2e60d3f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70308d1b0 0x55a70345dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:37.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.081+0000 7f2e60d3f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70308d1b0 0x55a70345dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53416/0 (socket says 172.21.15.117:53416) 2024-09-21T06:06:37.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.081+0000 7f2e60d3f640 1 -- 172.21.15.117:0/1257388191 learned_addr learned my addr 172.21.15.117:0/1257388191 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:37.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.081+0000 7f2e5bfff640 1 --2- 172.21.15.117:0/1257388191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a703095390 0x55a70345e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:37.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.081+0000 7f2e5bfff640 1 --2- 172.21.15.117:0/1257388191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a7030b6070 0x55a70345d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:37.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.082+0000 7f2e59ffb640 1 --2- 172.21.15.117:0/833239176 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2e440b6700 0x7f2e440b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:37.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.082+0000 7f2e60d3f640 1 -- 172.21.15.117:0/1257388191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a7030b6070 msgr2=0x55a70345d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:37.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.082+0000 7f2e60d3f640 1 --2- 172.21.15.117:0/1257388191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a7030b6070 0x55a70345d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.082+0000 7f2e60d3f640 1 -- 172.21.15.117:0/1257388191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a703095390 msgr2=0x55a70345e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:37.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.082+0000 7f2e60d3f640 1 --2- 172.21.15.117:0/1257388191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a703095390 0x55a70345e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.082+0000 7f2e60d3f640 1 -- 172.21.15.117:0/1257388191 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a703121650 con 0x55a70308d1b0 2024-09-21T06:06:37.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.082+0000 7f2e61540640 1 --2- 172.21.15.117:0/833239176 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2e440b6700 0x7f2e440b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:37.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.082+0000 7f2e59ffb640 1 -- 172.21.15.117:0/833239176 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7f2e540e7280 con 0x7f2e5c109df0 2024-09-21T06:06:37.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.082+0000 7f2e60d3f640 1 --2- 172.21.15.117:0/1257388191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70308d1b0 0x55a70345dbe0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c009230 tx=0x7f2e4c004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b672d4bc00107aae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:37.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.082+0000 7f2e42ffd640 1 -- 172.21.15.117:0/1257388191 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c035040 con 0x55a70308d1b0 2024-09-21T06:06:37.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.082+0000 7f2e42ffd640 1 -- 172.21.15.117:0/1257388191 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e4c0054f0 con 0x55a70308d1b0 2024-09-21T06:06:37.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.082+0000 7f2e42ffd640 1 -- 172.21.15.117:0/1257388191 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c0035f0 con 0x55a70308d1b0 2024-09-21T06:06:37.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.083+0000 7f2e69517740 1 -- 172.21.15.117:0/1257388191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70308d1b0 msgr2=0x55a70345dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:37.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.083+0000 7f2e69517740 1 --2- 172.21.15.117:0/1257388191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70308d1b0 0x55a70345dbe0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c009230 tx=0x7f2e4c004180 comp rx=0 tx=0).stop 2024-09-21T06:06:37.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.083+0000 7f2e61540640 1 --2- 172.21.15.117:0/833239176 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2e440b6700 0x7f2e440b8c10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2e540e4ad0 tx=0x7f2e54079e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:37.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.083+0000 7f2e69517740 1 -- 172.21.15.117:0/1257388191 shutdown_connections 2024-09-21T06:06:37.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.083+0000 7f2e69517740 1 --2- 172.21.15.117:0/1257388191 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70308d1b0 0x55a70345dbe0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.083+0000 7f2e69517740 1 --2- 172.21.15.117:0/1257388191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a703095390 0x55a70345e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.083+0000 7f2e69517740 1 --2- 172.21.15.117:0/1257388191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a7030b6070 0x55a70345d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.083+0000 7f2e69517740 1 -- 172.21.15.117:0/1257388191 >> 172.21.15.117:0/1257388191 conn(0x55a7030aa460 msgr2=0x55a70345b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:37.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.083+0000 7f2e69517740 1 -- 172.21.15.117:0/1257388191 shutdown_connections 2024-09-21T06:06:37.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.083+0000 7f2e69517740 1 -- 172.21.15.117:0/1257388191 wait complete. 2024-09-21T06:06:37.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.084+0000 7f2e69517740 1 -- start start 2024-09-21T06:06:37.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.084+0000 7f2e69517740 1 Processor -- start 2024-09-21T06:06:37.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.084+0000 7f2e69517740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70308d1b0 0x55a703468ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:37.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.084+0000 7f2e69517740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a703095390 0x55a703469440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:37.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.084+0000 7f2e69517740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7030b6070 0x55a703479500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:37.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.084+0000 7f2e69517740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a703224700 con 0x55a703095390 2024-09-21T06:06:37.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.084+0000 7f2e69517740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a70335e160 con 0x55a7030b6070 2024-09-21T06:06:37.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.084+0000 7f2e69517740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a703470140 con 0x55a70308d1b0 2024-09-21T06:06:37.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.084+0000 7f2e69517740 10 client.0 _ll_register_callbacks cb 0x55a7033779e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:37.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.084+0000 7f2e60d3f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a703095390 0x55a703469440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:37.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.084+0000 7f2e60d3f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a703095390 0x55a703469440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53422/0 (socket says 172.21.15.117:53422) 2024-09-21T06:06:37.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.084+0000 7f2e60d3f640 1 -- 172.21.15.117:0/1446489870 learned_addr learned my addr 172.21.15.117:0/1446489870 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:37.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.084+0000 7f2e5bfff640 1 --2- 172.21.15.117:0/1446489870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7030b6070 0x55a703479500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:37.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.085+0000 7f2e5bfff640 1 --2- 172.21.15.117:0/1446489870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70308d1b0 0x55a703468ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:37.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.085+0000 7f2e60d3f640 1 -- 172.21.15.117:0/1446489870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70308d1b0 msgr2=0x55a703468ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:37.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.085+0000 7f2e60d3f640 1 --2- 172.21.15.117:0/1446489870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70308d1b0 0x55a703468ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.085+0000 7f2e60d3f640 1 -- 172.21.15.117:0/1446489870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7030b6070 msgr2=0x55a703479500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:37.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.085+0000 7f2e60d3f640 1 --2- 172.21.15.117:0/1446489870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7030b6070 0x55a703479500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.085+0000 7f2e60d3f640 1 -- 172.21.15.117:0/1446489870 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a70338bc90 con 0x55a703095390 2024-09-21T06:06:37.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.085+0000 7f2e5bfff640 1 --2- 172.21.15.117:0/1446489870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70308d1b0 0x55a703468ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:06:37.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.085+0000 7f2e60d3f640 1 --2- 172.21.15.117:0/1446489870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a703095390 0x55a703469440 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c02a930 tx=0x7f2e4c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b05751d9a8398a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:37.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.085+0000 7f2e3bfff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:37.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.085+0000 7f2e3bfff640 1 -- 172.21.15.117:0/1446489870 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c035040 con 0x55a703095390 2024-09-21T06:06:37.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.085+0000 7f2e3bfff640 1 -- 172.21.15.117:0/1446489870 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7f2e4c004ca0 con 0x55a703095390 2024-09-21T06:06:37.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.085+0000 7f2e69517740 1 -- 172.21.15.117:0/1446489870 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e4c0054f0 con 0x55a703095390 2024-09-21T06:06:37.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.085+0000 7f2e3bfff640 1 -- 172.21.15.117:0/1446489870 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2e4c004ea0 con 0x55a703095390 2024-09-21T06:06:37.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.085+0000 7f2e69517740 10 client.17409 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:37.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.085+0000 7f2e69517740 1 -- 172.21.15.117:0/1446489870 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a70347c050 con 0x55a703095390 2024-09-21T06:06:37.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.085+0000 7f2e3bfff640 1 -- 172.21.15.117:0/1446489870 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e4c0054f0 con 0x55a703095390 2024-09-21T06:06:37.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.086+0000 7f2e3bfff640 1 -- 172.21.15.117:0/1446489870 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2e4c04c070 con 0x55a703095390 2024-09-21T06:06:37.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.086+0000 7f2e69517740 10 client.17409 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:37.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.086+0000 7f2e69517740 10 client.17409 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:06:37.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.086+0000 7f2e69517740 20 client.17409 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:37.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.086+0000 7f2e69517740 1 --2- 172.21.15.117:0/1446489870 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55a7030749f0 0x55a70347f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:37.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.086+0000 7f2e69517740 4 client.17409 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:06:37.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.086+0000 7f2e69517740 1 -- 172.21.15.117:0/1446489870 --> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7033d5df0 con 0x55a7030749f0 2024-09-21T06:06:37.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.086+0000 7f2e5bfff640 1 --2- 172.21.15.117:0/1446489870 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55a7030749f0 0x55a70347f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:37.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.086+0000 7f2e5bfff640 1 --2- 172.21.15.117:0/1446489870 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55a7030749f0 0x55a70347f3b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f8366f8e1ad5b9c9 server_cookie=7be57e6cc00a19d7 in_seq=0 out_seq=0 2024-09-21T06:06:37.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.086+0000 7f2e3bfff640 10 client.17409 ms_handle_connect on v2:172.21.15.192:6836/2683860842 2024-09-21T06:06:37.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.087+0000 7f2e3bfff640 1 -- 172.21.15.117:0/1446489870 <== mds.2 v2:172.21.15.192:6836/2683860842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a7033d5df0 con 0x55a7030749f0 2024-09-21T06:06:37.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.087+0000 7f2e3bfff640 10 client.17409 handle_command_reply: tid=0 2024-09-21T06:06:37.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.087+0000 7f2e69517740 1 client.17409 shutdown 2024-09-21T06:06:37.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.087+0000 7f2e69517740 1 -- 172.21.15.117:0/1446489870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a703095390 msgr2=0x55a703469440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:37.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.087+0000 7f2e69517740 1 --2- 172.21.15.117:0/1446489870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a703095390 0x55a703469440 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c02a930 tx=0x7f2e4c02a410 comp rx=0 tx=0).stop 2024-09-21T06:06:37.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.087+0000 7f2e69517740 1 -- 172.21.15.117:0/1446489870 shutdown_connections 2024-09-21T06:06:37.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.087+0000 7f2e69517740 1 --2- 172.21.15.117:0/1446489870 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55a7030749f0 0x55a70347f3b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.087+0000 7f2e69517740 1 --2- 172.21.15.117:0/1446489870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a703095390 0x55a703469440 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.087+0000 7f2e69517740 1 --2- 172.21.15.117:0/1446489870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a7030b6070 0x55a703479500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.087+0000 7f2e3bfff640 0 client.17409 ms_handle_reset on v2:172.21.15.192:6836/2683860842 2024-09-21T06:06:37.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.087+0000 7f2e69517740 1 --2- 172.21.15.117:0/1446489870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70308d1b0 0x55a703468ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.087+0000 7f2e69517740 1 -- 172.21.15.117:0/1446489870 >> 172.21.15.117:0/1446489870 conn(0x55a7030aa460 msgr2=0x55a703460830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:37.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.087+0000 7f2e69517740 1 -- 172.21.15.117:0/1446489870 shutdown_connections 2024-09-21T06:06:37.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.087+0000 7f2e69517740 1 -- 172.21.15.117:0/1446489870 wait complete. 2024-09-21T06:06:37.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.087+0000 7f2e69517740 20 client.17409 trim_cache size 0 max 16384 2024-09-21T06:06:37.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.101+0000 7f2e69517740 1 Processor -- start 2024-09-21T06:06:37.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.101+0000 7f2e69517740 1 -- start start 2024-09-21T06:06:37.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.101+0000 7f2e69517740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70308d1b0 0x55a7034693a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:37.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.101+0000 7f2e69517740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a703095390 0x55a7034794b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:37.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.101+0000 7f2e69517740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7030b6070 0x55a7034607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:37.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.101+0000 7f2e69517740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c1ad3b0 con 0x55a7030b6070 2024-09-21T06:06:37.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.101+0000 7f2e69517740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c1aeef0 con 0x55a703095390 2024-09-21T06:06:37.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.101+0000 7f2e69517740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c1ada80 con 0x55a70308d1b0 2024-09-21T06:06:37.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.102+0000 7f2e5bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7030b6070 0x55a7034607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:37.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.102+0000 7f2e5bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7030b6070 0x55a7034607a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53432/0 (socket says 172.21.15.117:53432) 2024-09-21T06:06:37.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.102+0000 7f2e5bfff640 1 -- 172.21.15.117:0/3783847049 learned_addr learned my addr 172.21.15.117:0/3783847049 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:37.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.102+0000 7f2e60d3f640 1 --2- 172.21.15.117:0/3783847049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a703095390 0x55a7034794b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:37.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.102+0000 7f2e5bfff640 1 --2- 172.21.15.117:0/3783847049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70308d1b0 0x55a7034693a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:37.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.102+0000 7f2e5bfff640 1 -- 172.21.15.117:0/3783847049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70308d1b0 msgr2=0x55a7034693a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:37.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.102+0000 7f2e5bfff640 1 --2- 172.21.15.117:0/3783847049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70308d1b0 0x55a7034693a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.102+0000 7f2e5bfff640 1 -- 172.21.15.117:0/3783847049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a703095390 msgr2=0x55a7034794b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:37.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.102+0000 7f2e5bfff640 1 --2- 172.21.15.117:0/3783847049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a703095390 0x55a7034794b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.102+0000 7f2e5bfff640 1 -- 172.21.15.117:0/3783847049 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a70346a1a0 con 0x55a7030b6070 2024-09-21T06:06:37.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.102+0000 7f2e5bfff640 1 --2- 172.21.15.117:0/3783847049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7030b6070 0x55a7034607a0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f2e50000c00 tx=0x7f2e5000b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b58f970af5dadd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:37.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.102+0000 7f2e427fc640 1 -- 172.21.15.117:0/3783847049 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e50004020 con 0x55a7030b6070 2024-09-21T06:06:37.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.102+0000 7f2e427fc640 1 -- 172.21.15.117:0/3783847049 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e500054d0 con 0x55a7030b6070 2024-09-21T06:06:37.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.102+0000 7f2e427fc640 1 -- 172.21.15.117:0/3783847049 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e50004ba0 con 0x55a7030b6070 2024-09-21T06:06:37.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.103+0000 7f2e69517740 1 -- 172.21.15.117:0/3783847049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7030b6070 msgr2=0x55a7034607a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:37.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.103+0000 7f2e69517740 1 --2- 172.21.15.117:0/3783847049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7030b6070 0x55a7034607a0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f2e50000c00 tx=0x7f2e5000b410 comp rx=0 tx=0).stop 2024-09-21T06:06:37.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.103+0000 7f2e69517740 1 -- 172.21.15.117:0/3783847049 shutdown_connections 2024-09-21T06:06:37.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.103+0000 7f2e69517740 1 --2- 172.21.15.117:0/3783847049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a7030b6070 0x55a7034607a0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.103+0000 7f2e69517740 1 --2- 172.21.15.117:0/3783847049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a703095390 0x55a7034794b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.103+0000 7f2e69517740 1 --2- 172.21.15.117:0/3783847049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70308d1b0 0x55a7034693a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.103+0000 7f2e69517740 1 -- 172.21.15.117:0/3783847049 >> 172.21.15.117:0/3783847049 conn(0x55a7030aa460 msgr2=0x55a703468c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:37.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.103+0000 7f2e69517740 1 -- 172.21.15.117:0/3783847049 shutdown_connections 2024-09-21T06:06:37.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.103+0000 7f2e69517740 1 -- 172.21.15.117:0/3783847049 wait complete. 2024-09-21T06:06:37.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.104+0000 7f2e69517740 1 -- start start 2024-09-21T06:06:37.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.104+0000 7f2e69517740 1 Processor -- start 2024-09-21T06:06:37.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.104+0000 7f2e69517740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70308d1b0 0x55a703482ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:37.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.104+0000 7f2e69517740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a703095390 0x55a703480070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:37.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.104+0000 7f2e69517740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a7030b6070 0x55a7034805b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:37.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.104+0000 7f2e69517740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c1ae820 con 0x55a703095390 2024-09-21T06:06:37.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.104+0000 7f2e69517740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c1ae150 con 0x55a70308d1b0 2024-09-21T06:06:37.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.104+0000 7f2e69517740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5c1ab390 con 0x55a7030b6070 2024-09-21T06:06:37.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.104+0000 7f2e69517740 10 client.0 _ll_register_callbacks cb 0x7f2e5000be40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:37.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.104+0000 7f2e60d3f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a703095390 0x55a703480070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:37.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.105+0000 7f2e60d3f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a703095390 0x55a703480070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53446/0 (socket says 172.21.15.117:53446) 2024-09-21T06:06:37.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.105+0000 7f2e60d3f640 1 -- 172.21.15.117:0/3283731882 learned_addr learned my addr 172.21.15.117:0/3283731882 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:37.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.105+0000 7f2e5bfff640 1 --2- 172.21.15.117:0/3283731882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70308d1b0 0x55a703482ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:37.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.105+0000 7f2e5bfff640 1 --2- 172.21.15.117:0/3283731882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a7030b6070 0x55a7034805b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:37.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.105+0000 7f2e60d3f640 1 -- 172.21.15.117:0/3283731882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a7030b6070 msgr2=0x55a7034805b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:37.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.105+0000 7f2e60d3f640 1 --2- 172.21.15.117:0/3283731882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a7030b6070 0x55a7034805b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.105+0000 7f2e60d3f640 1 -- 172.21.15.117:0/3283731882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70308d1b0 msgr2=0x55a703482ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:37.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.105+0000 7f2e60d3f640 1 --2- 172.21.15.117:0/3283731882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70308d1b0 0x55a703482ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.105+0000 7f2e60d3f640 1 -- 172.21.15.117:0/3283731882 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a703480d20 con 0x55a703095390 2024-09-21T06:06:37.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.105+0000 7f2e60d3f640 1 --2- 172.21.15.117:0/3283731882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a703095390 0x55a703480070 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c001eb0 tx=0x7f2e4c005eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c17b01a05f66af33 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:37.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.105+0000 7f2e3bfff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:37.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.105+0000 7f2e3bfff640 1 -- 172.21.15.117:0/3283731882 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c064060 con 0x55a703095390 2024-09-21T06:06:37.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.105+0000 7f2e3bfff640 1 -- 172.21.15.117:0/3283731882 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7f2e4c005450 con 0x55a703095390 2024-09-21T06:06:37.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.105+0000 7f2e69517740 1 -- 172.21.15.117:0/3283731882 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e500054d0 con 0x55a703095390 2024-09-21T06:06:37.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.105+0000 7f2e3bfff640 1 -- 172.21.15.117:0/3283731882 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2e4c035070 con 0x55a703095390 2024-09-21T06:06:37.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.105+0000 7f2e69517740 10 client.17415 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:37.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.105+0000 7f2e69517740 1 -- 172.21.15.117:0/3283731882 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a7034e9a50 con 0x55a703095390 2024-09-21T06:06:37.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.106+0000 7f2e3bfff640 1 -- 172.21.15.117:0/3283731882 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e500054d0 con 0x55a703095390 2024-09-21T06:06:37.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.106+0000 7f2e3bfff640 1 -- 172.21.15.117:0/3283731882 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2e4c04c040 con 0x55a703095390 2024-09-21T06:06:37.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.106+0000 7f2e69517740 10 client.17415 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:37.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.106+0000 7f2e69517740 10 client.17415 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:06:37.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.106+0000 7f2e69517740 20 client.17415 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:37.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.106+0000 7f2e69517740 1 --2- 172.21.15.117:0/3283731882 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55a703049da0 0x55a7034ec120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:37.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.106+0000 7f2e69517740 4 client.17415 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:06:37.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.106+0000 7f2e69517740 1 -- 172.21.15.117:0/3283731882 --> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a703464a90 con 0x55a703049da0 2024-09-21T06:06:37.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.106+0000 7f2e5bfff640 1 --2- 172.21.15.117:0/3283731882 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55a703049da0 0x55a7034ec120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:37.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.106+0000 7f2e3bfff640 10 client.17415 ms_handle_connect on v2:172.21.15.192:6836/2683860842 2024-09-21T06:06:37.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.106+0000 7f2e5bfff640 1 --2- 172.21.15.117:0/3283731882 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55a703049da0 0x55a7034ec120 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1bb31077a33088c1 server_cookie=85cd7ebcd5436cc7 in_seq=0 out_seq=0 2024-09-21T06:06:37.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.107+0000 7f2e3bfff640 1 -- 172.21.15.117:0/3283731882 <== mds.2 v2:172.21.15.192:6836/2683860842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a703464a90 con 0x55a703049da0 2024-09-21T06:06:37.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.107+0000 7f2e3bfff640 10 client.17415 handle_command_reply: tid=0 2024-09-21T06:06:37.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.107+0000 7f2e69517740 1 client.17415 shutdown 2024-09-21T06:06:37.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.107+0000 7f2e69517740 1 -- 172.21.15.117:0/3283731882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a703095390 msgr2=0x55a703480070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:37.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.107+0000 7f2e69517740 1 --2- 172.21.15.117:0/3283731882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a703095390 0x55a703480070 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c001eb0 tx=0x7f2e4c005eb0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.107+0000 7f2e69517740 1 -- 172.21.15.117:0/3283731882 shutdown_connections 2024-09-21T06:06:37.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.107+0000 7f2e69517740 1 --2- 172.21.15.117:0/3283731882 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55a703049da0 0x55a7034ec120 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.107+0000 7f2e69517740 1 --2- 172.21.15.117:0/3283731882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a703095390 0x55a703480070 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.107+0000 7f2e69517740 1 --2- 172.21.15.117:0/3283731882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a7030b6070 0x55a7034805b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.107+0000 7f2e69517740 1 --2- 172.21.15.117:0/3283731882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70308d1b0 0x55a703482ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.107+0000 7f2e3bfff640 0 client.17415 ms_handle_reset on v2:172.21.15.192:6836/2683860842 2024-09-21T06:06:37.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.107+0000 7f2e69517740 1 -- 172.21.15.117:0/3283731882 >> 172.21.15.117:0/3283731882 conn(0x55a7030aa460 msgr2=0x55a7034607a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:37.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.107+0000 7f2e69517740 1 -- 172.21.15.117:0/3283731882 shutdown_connections 2024-09-21T06:06:37.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.107+0000 7f2e69517740 1 -- 172.21.15.117:0/3283731882 wait complete. 2024-09-21T06:06:37.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.107+0000 7f2e69517740 20 client.17415 trim_cache size 0 max 16384 2024-09-21T06:06:37.113 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:06:37.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.108+0000 7f2e62fa3640 1 -- 172.21.15.117:0/833239176 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2e440b6700 msgr2=0x7f2e440b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:37.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.108+0000 7f2e62fa3640 1 --2- 172.21.15.117:0/833239176 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2e440b6700 0x7f2e440b8c10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2e540e4ad0 tx=0x7f2e54079e80 comp rx=0 tx=0).stop 2024-09-21T06:06:37.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.108+0000 7f2e62fa3640 1 -- 172.21.15.117:0/833239176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e5c109df0 msgr2=0x7f2e5c19fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:37.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.108+0000 7f2e62fa3640 1 --2- 172.21.15.117:0/833239176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e5c109df0 0x7f2e5c19fd50 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f2e54009fd0 tx=0x7f2e5400b260 comp rx=0 tx=0).stop 2024-09-21T06:06:37.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.108+0000 7f2e62fa3640 1 -- 172.21.15.117:0/833239176 shutdown_connections 2024-09-21T06:06:37.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.108+0000 7f2e62fa3640 1 --2- 172.21.15.117:0/833239176 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2e440b6700 0x7f2e440b8c10 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.108+0000 7f2e62fa3640 1 --2- 172.21.15.117:0/833239176 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e5c109df0 0x7f2e5c19fd50 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.108+0000 7f2e62fa3640 1 --2- 172.21.15.117:0/833239176 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e5c1094a0 0x7f2e5c19aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.108+0000 7f2e62fa3640 1 --2- 172.21.15.117:0/833239176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e5c103b60 0x7f2e5c19a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:37.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.108+0000 7f2e62fa3640 1 -- 172.21.15.117:0/833239176 >> 172.21.15.117:0/833239176 conn(0x7f2e5c07b850 msgr2=0x7f2e5c107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:37.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.108+0000 7f2e62fa3640 1 -- 172.21.15.117:0/833239176 shutdown_connections 2024-09-21T06:06:37.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:37.108+0000 7f2e62fa3640 1 -- 172.21.15.117:0/833239176 wait complete. 2024-09-21T06:06:37.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:36 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:06:37.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:36 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:06:37.911 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T06:06:38.159 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:38.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:37 smithi192 ceph-mon[34700]: pgmap v611: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-21T06:06:38.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:37 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:38.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:37 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:06:38.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:37 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:06:38.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:37 smithi117 ceph-mon[30947]: pgmap v611: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-21T06:06:38.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:37 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:38.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:37 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:06:38.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:37 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:06:38.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:37 smithi138 ceph-mon[34391]: pgmap v611: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-21T06:06:38.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:37 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:06:38.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:37 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:06:38.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:37 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:06:38.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.644+0000 7f38f1726640 1 -- 172.21.15.117:0/845008829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38ec1094a0 msgr2=0x7f38ec1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:38.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.644+0000 7f38f1726640 1 --2- 172.21.15.117:0/845008829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38ec1094a0 0x7f38ec1098b0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f38e0009900 tx=0x7f38e002a410 comp rx=0 tx=0).stop 2024-09-21T06:06:38.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.644+0000 7f38f1726640 1 -- 172.21.15.117:0/845008829 shutdown_connections 2024-09-21T06:06:38.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.644+0000 7f38f1726640 1 --2- 172.21.15.117:0/845008829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38ec1094a0 0x7f38ec1098b0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.644+0000 7f38f1726640 1 --2- 172.21.15.117:0/845008829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38ec109df0 0x7f38ec110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.644+0000 7f38f1726640 1 --2- 172.21.15.117:0/845008829 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38ec103b60 0x7f38ec103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.644+0000 7f38f1726640 1 -- 172.21.15.117:0/845008829 >> 172.21.15.117:0/845008829 conn(0x7f38ec07b850 msgr2=0x7f38ec07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:38.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.644+0000 7f38f1726640 1 -- 172.21.15.117:0/845008829 shutdown_connections 2024-09-21T06:06:38.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.644+0000 7f38f1726640 1 -- 172.21.15.117:0/845008829 wait complete. 2024-09-21T06:06:38.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.644+0000 7f38f1726640 1 Processor -- start 2024-09-21T06:06:38.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.645+0000 7f38f1726640 1 -- start start 2024-09-21T06:06:38.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.645+0000 7f38f1726640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38ec103b60 0x7f38ec19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:38.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.645+0000 7f38f1726640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38ec1094a0 0x7f38ec19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:38.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.645+0000 7f38f1726640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38ec109df0 0x7f38ec19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:38.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.645+0000 7f38f1726640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec113ab0 con 0x7f38ec109df0 2024-09-21T06:06:38.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.645+0000 7f38f1726640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec113930 con 0x7f38ec103b60 2024-09-21T06:06:38.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.645+0000 7f38f1726640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec113c30 con 0x7f38ec1094a0 2024-09-21T06:06:38.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.645+0000 7f38eb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38ec109df0 0x7f38ec19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:38.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.645+0000 7f38eb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38ec109df0 0x7f38ec19fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53472/0 (socket says 172.21.15.117:53472) 2024-09-21T06:06:38.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.645+0000 7f38eb7fe640 1 -- 172.21.15.117:0/1895057409 learned_addr learned my addr 172.21.15.117:0/1895057409 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:38.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.645+0000 7f38ea7fc640 1 --2- 172.21.15.117:0/1895057409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38ec1094a0 0x7f38ec19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:38.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.645+0000 7f38eaffd640 1 --2- 172.21.15.117:0/1895057409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38ec103b60 0x7f38ec19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:38.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.645+0000 7f38eb7fe640 1 -- 172.21.15.117:0/1895057409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38ec1094a0 msgr2=0x7f38ec19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:38.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.645+0000 7f38eb7fe640 1 --2- 172.21.15.117:0/1895057409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38ec1094a0 0x7f38ec19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.645+0000 7f38eb7fe640 1 -- 172.21.15.117:0/1895057409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38ec103b60 msgr2=0x7f38ec19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:38.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.645+0000 7f38eb7fe640 1 --2- 172.21.15.117:0/1895057409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38ec103b60 0x7f38ec19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.645+0000 7f38eb7fe640 1 -- 172.21.15.117:0/1895057409 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38ec1a0370 con 0x7f38ec109df0 2024-09-21T06:06:38.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.645+0000 7f38ea7fc640 1 --2- 172.21.15.117:0/1895057409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38ec1094a0 0x7f38ec19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:06:38.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.646+0000 7f38eb7fe640 1 --2- 172.21.15.117:0/1895057409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38ec109df0 0x7f38ec19fd00 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f38e400cce0 tx=0x7f38e400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:38.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.646+0000 7f38d3fff640 1 -- 172.21.15.117:0/1895057409 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e400f450 con 0x7f38ec109df0 2024-09-21T06:06:38.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.646+0000 7f38d3fff640 1 -- 172.21.15.117:0/1895057409 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38e400faa0 con 0x7f38ec109df0 2024-09-21T06:06:38.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.646+0000 7f38f1726640 1 -- 172.21.15.117:0/1895057409 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38ec1a0660 con 0x7f38ec109df0 2024-09-21T06:06:38.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.646+0000 7f38d3fff640 1 -- 172.21.15.117:0/1895057409 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e4018b70 con 0x7f38ec109df0 2024-09-21T06:06:38.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.646+0000 7f38f1726640 1 -- 172.21.15.117:0/1895057409 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38ec1a0b20 con 0x7f38ec109df0 2024-09-21T06:06:38.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.649+0000 7f38f7c9a740 1 Processor -- start 2024-09-21T06:06:38.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.649+0000 7f38d3fff640 1 -- 172.21.15.117:0/1895057409 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f38e4018d10 con 0x7f38ec109df0 2024-09-21T06:06:38.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.649+0000 7f38f7c9a740 1 -- start start 2024-09-21T06:06:38.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.649+0000 7f38f7c9a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0fd0ac070 0x55f0fd4536a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:38.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.649+0000 7f38f7c9a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0fd0831b0 0x55f0fd453be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:38.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.649+0000 7f38f7c9a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0fd08b390 0x55f0fd454120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:38.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.649+0000 7f38f7c9a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f0fd3c9ea0 con 0x55f0fd0831b0 2024-09-21T06:06:38.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.649+0000 7f38f7c9a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f0fd3b92d0 con 0x55f0fd0ac070 2024-09-21T06:06:38.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.649+0000 7f38f7c9a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f0fd3bd0e0 con 0x55f0fd08b390 2024-09-21T06:06:38.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.650+0000 7f38eaffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0fd0831b0 0x55f0fd453be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:38.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.650+0000 7f38eaffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0fd0831b0 0x55f0fd453be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53476/0 (socket says 172.21.15.117:53476) 2024-09-21T06:06:38.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.650+0000 7f38eaffd640 1 -- 172.21.15.117:0/2846029504 learned_addr learned my addr 172.21.15.117:0/2846029504 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:38.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.650+0000 7f38ea7fc640 1 --2- 172.21.15.117:0/2846029504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0fd0ac070 0x55f0fd4536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:38.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.650+0000 7f38ea7fc640 1 --2- 172.21.15.117:0/2846029504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0fd08b390 0x55f0fd454120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:38.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.650+0000 7f38eaffd640 1 -- 172.21.15.117:0/2846029504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0fd08b390 msgr2=0x55f0fd454120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:38.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.650+0000 7f38eaffd640 1 --2- 172.21.15.117:0/2846029504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0fd08b390 0x55f0fd454120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.650+0000 7f38eaffd640 1 -- 172.21.15.117:0/2846029504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0fd0ac070 msgr2=0x55f0fd4536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:38.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.650+0000 7f38eaffd640 1 --2- 172.21.15.117:0/2846029504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0fd0ac070 0x55f0fd4536a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.650+0000 7f38eaffd640 1 -- 172.21.15.117:0/2846029504 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0fd117650 con 0x55f0fd0831b0 2024-09-21T06:06:38.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.650+0000 7f38d3fff640 1 --2- 172.21.15.117:0/1895057409 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f38d40b66b0 0x7f38d40b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:38.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.650+0000 7f38ea7fc640 1 --2- 172.21.15.117:0/1895057409 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f38d40b66b0 0x7f38d40b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:38.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.650+0000 7f38eaffd640 1 --2- 172.21.15.117:0/2846029504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0fd0831b0 0x55f0fd453be0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00bcb0 tx=0x7f38dc00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f9eeebad82dbd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:38.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.650+0000 7f38d3fff640 1 -- 172.21.15.117:0/1895057409 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7f38e40e7bd0 con 0x7f38ec109df0 2024-09-21T06:06:38.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.650+0000 7f38d17fa640 1 -- 172.21.15.117:0/2846029504 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc015070 con 0x55f0fd0831b0 2024-09-21T06:06:38.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.650+0000 7f38d17fa640 1 -- 172.21.15.117:0/2846029504 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38dc01a910 con 0x55f0fd0831b0 2024-09-21T06:06:38.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.650+0000 7f38d17fa640 1 -- 172.21.15.117:0/2846029504 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc011820 con 0x55f0fd0831b0 2024-09-21T06:06:38.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.651+0000 7f38f7c9a740 1 -- 172.21.15.117:0/2846029504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0fd0831b0 msgr2=0x55f0fd453be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:38.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.651+0000 7f38f7c9a740 1 --2- 172.21.15.117:0/2846029504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0fd0831b0 0x55f0fd453be0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00bcb0 tx=0x7f38dc00f730 comp rx=0 tx=0).stop 2024-09-21T06:06:38.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.651+0000 7f38ea7fc640 1 --2- 172.21.15.117:0/1895057409 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f38d40b66b0 0x7f38d40b8bc0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f38e0009230 tx=0x7f38e0000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:38.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.651+0000 7f38f7c9a740 1 -- 172.21.15.117:0/2846029504 shutdown_connections 2024-09-21T06:06:38.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.651+0000 7f38f7c9a740 1 --2- 172.21.15.117:0/2846029504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0fd0831b0 0x55f0fd453be0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.651+0000 7f38f7c9a740 1 --2- 172.21.15.117:0/2846029504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0fd08b390 0x55f0fd454120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.651+0000 7f38f7c9a740 1 --2- 172.21.15.117:0/2846029504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0fd0ac070 0x55f0fd4536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.651+0000 7f38f7c9a740 1 -- 172.21.15.117:0/2846029504 >> 172.21.15.117:0/2846029504 conn(0x55f0fd0a0460 msgr2=0x55f0fd451590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:38.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.651+0000 7f38f7c9a740 1 -- 172.21.15.117:0/2846029504 shutdown_connections 2024-09-21T06:06:38.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.651+0000 7f38f7c9a740 1 -- 172.21.15.117:0/2846029504 wait complete. 2024-09-21T06:06:38.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.652+0000 7f38f7c9a740 1 -- start start 2024-09-21T06:06:38.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.652+0000 7f38f7c9a740 1 Processor -- start 2024-09-21T06:06:38.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.652+0000 7f38f7c9a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0fd0831b0 0x55f0fd45edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:38.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.652+0000 7f38f7c9a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0fd08b390 0x55f0fd45f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:38.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.652+0000 7f38f7c9a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0fd0ac070 0x55f0fd46b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:38.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.652+0000 7f38f7c9a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f0fd21a700 con 0x55f0fd0ac070 2024-09-21T06:06:38.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.652+0000 7f38f7c9a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f0fd354160 con 0x55f0fd08b390 2024-09-21T06:06:38.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.652+0000 7f38f7c9a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f0fd461e80 con 0x55f0fd0831b0 2024-09-21T06:06:38.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.652+0000 7f38f7c9a740 10 client.0 _ll_register_callbacks cb 0x55f0fd36d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:38.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.653+0000 7f38eb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0fd0ac070 0x55f0fd46b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:38.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.653+0000 7f38eb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0fd0ac070 0x55f0fd46b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53478/0 (socket says 172.21.15.117:53478) 2024-09-21T06:06:38.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.653+0000 7f38eb7fe640 1 -- 172.21.15.117:0/171099989 learned_addr learned my addr 172.21.15.117:0/171099989 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:38.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.653+0000 7f38ea7fc640 1 --2- 172.21.15.117:0/171099989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0fd08b390 0x55f0fd45f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:38.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.653+0000 7f38eaffd640 1 --2- 172.21.15.117:0/171099989 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0fd0831b0 0x55f0fd45edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:38.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.653+0000 7f38eb7fe640 1 -- 172.21.15.117:0/171099989 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0fd0831b0 msgr2=0x55f0fd45edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:38.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.653+0000 7f38eb7fe640 1 --2- 172.21.15.117:0/171099989 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0fd0831b0 0x55f0fd45edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.653+0000 7f38eb7fe640 1 -- 172.21.15.117:0/171099989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0fd08b390 msgr2=0x55f0fd45f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:38.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.653+0000 7f38eb7fe640 1 --2- 172.21.15.117:0/171099989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0fd08b390 0x55f0fd45f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.653+0000 7f38eb7fe640 1 -- 172.21.15.117:0/171099989 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0fd381c90 con 0x55f0fd0ac070 2024-09-21T06:06:38.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.653+0000 7f38ea7fc640 1 --2- 172.21.15.117:0/171099989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0fd08b390 0x55f0fd45f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:06:38.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.653+0000 7f38eb7fe640 1 --2- 172.21.15.117:0/171099989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0fd0ac070 0x55f0fd46b170 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f38e40aa040 tx=0x7f38e40ad480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac239363f6de7f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:38.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.653+0000 7f38ceff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:38.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.653+0000 7f38ceff5640 1 -- 172.21.15.117:0/171099989 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e40b2020 con 0x55f0fd0ac070 2024-09-21T06:06:38.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.653+0000 7f38ceff5640 1 -- 172.21.15.117:0/171099989 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7f38e40a9ce0 con 0x55f0fd0ac070 2024-09-21T06:06:38.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.653+0000 7f38f7c9a740 1 -- 172.21.15.117:0/171099989 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38dc01a910 con 0x55f0fd0ac070 2024-09-21T06:06:38.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.653+0000 7f38ceff5640 1 -- 172.21.15.117:0/171099989 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f38e40d5020 con 0x55f0fd0ac070 2024-09-21T06:06:38.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.653+0000 7f38f7c9a740 10 client.17427 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:38.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.653+0000 7f38f7c9a740 1 -- 172.21.15.117:0/171099989 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f0fd46db90 con 0x55f0fd0ac070 2024-09-21T06:06:38.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.654+0000 7f38ceff5640 1 -- 172.21.15.117:0/171099989 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38dc01a910 con 0x55f0fd0ac070 2024-09-21T06:06:38.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.654+0000 7f38ceff5640 1 -- 172.21.15.117:0/171099989 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f38e40a93c0 con 0x55f0fd0ac070 2024-09-21T06:06:38.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.654+0000 7f38f7c9a740 10 client.17427 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:38.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.654+0000 7f38f7c9a740 10 client.17427 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:06:38.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.654+0000 7f38f7c9a740 20 client.17427 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:38.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.654+0000 7f38f7c9a740 1 --2- 172.21.15.117:0/171099989 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55f0fd06a9f0 0x55f0fd471100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:38.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.654+0000 7f38f7c9a740 4 client.17427 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:06:38.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.654+0000 7f38f7c9a740 1 -- 172.21.15.117:0/171099989 --> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0fd3cbdf0 con 0x55f0fd06a9f0 2024-09-21T06:06:38.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.654+0000 7f38eaffd640 1 --2- 172.21.15.117:0/171099989 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55f0fd06a9f0 0x55f0fd471100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:38.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.655+0000 7f38ceff5640 10 client.17427 ms_handle_connect on v2:172.21.15.192:6838/1727623231 2024-09-21T06:06:38.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.655+0000 7f38eaffd640 1 --2- 172.21.15.117:0/171099989 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55f0fd06a9f0 0x55f0fd471100 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cd42498f40d0d35e server_cookie=9ce34d76aa5ab5df in_seq=0 out_seq=0 2024-09-21T06:06:38.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.656+0000 7f38ceff5640 1 -- 172.21.15.117:0/171099989 <== mds.1 v2:172.21.15.192:6838/1727623231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f0fd3cbdf0 con 0x55f0fd06a9f0 2024-09-21T06:06:38.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.656+0000 7f38ceff5640 10 client.17427 handle_command_reply: tid=0 2024-09-21T06:06:38.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.656+0000 7f38f7c9a740 1 client.17427 shutdown 2024-09-21T06:06:38.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.656+0000 7f38f7c9a740 1 -- 172.21.15.117:0/171099989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0fd0ac070 msgr2=0x55f0fd46b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:38.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.656+0000 7f38f7c9a740 1 --2- 172.21.15.117:0/171099989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0fd0ac070 0x55f0fd46b170 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f38e40aa040 tx=0x7f38e40ad480 comp rx=0 tx=0).stop 2024-09-21T06:06:38.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.656+0000 7f38f7c9a740 1 -- 172.21.15.117:0/171099989 shutdown_connections 2024-09-21T06:06:38.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.656+0000 7f38f7c9a740 1 --2- 172.21.15.117:0/171099989 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55f0fd06a9f0 0x55f0fd471100 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.656+0000 7f38f7c9a740 1 --2- 172.21.15.117:0/171099989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0fd0ac070 0x55f0fd46b170 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.656+0000 7f38ceff5640 0 client.17427 ms_handle_reset on v2:172.21.15.192:6838/1727623231 2024-09-21T06:06:38.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.656+0000 7f38f7c9a740 1 --2- 172.21.15.117:0/171099989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0fd08b390 0x55f0fd45f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.656+0000 7f38f7c9a740 1 --2- 172.21.15.117:0/171099989 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0fd0831b0 0x55f0fd45edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.656+0000 7f38f7c9a740 1 -- 172.21.15.117:0/171099989 >> 172.21.15.117:0/171099989 conn(0x55f0fd0a0460 msgr2=0x55f0fd456880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:38.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.656+0000 7f38f7c9a740 1 -- 172.21.15.117:0/171099989 shutdown_connections 2024-09-21T06:06:38.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.656+0000 7f38f7c9a740 1 -- 172.21.15.117:0/171099989 wait complete. 2024-09-21T06:06:38.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.656+0000 7f38f7c9a740 20 client.17427 trim_cache size 0 max 16384 2024-09-21T06:06:38.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.669+0000 7f38f7c9a740 1 Processor -- start 2024-09-21T06:06:38.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.669+0000 7f38f7c9a740 1 -- start start 2024-09-21T06:06:38.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.669+0000 7f38f7c9a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0fd06a9f0 0x55f0fd454de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:38.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.669+0000 7f38f7c9a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0fd0831b0 0x55f0fd46b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:38.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.669+0000 7f38f7c9a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0fd08b390 0x55f0fd45bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:38.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.669+0000 7f38f7c9a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec1af600 con 0x55f0fd06a9f0 2024-09-21T06:06:38.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.669+0000 7f38f7c9a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec1b1140 con 0x55f0fd08b390 2024-09-21T06:06:38.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.669+0000 7f38f7c9a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec1afcd0 con 0x55f0fd0831b0 2024-09-21T06:06:38.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.669+0000 7f38eaffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0fd06a9f0 0x55f0fd454de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:38.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.669+0000 7f38eaffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0fd06a9f0 0x55f0fd454de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53488/0 (socket says 172.21.15.117:53488) 2024-09-21T06:06:38.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.670+0000 7f38eaffd640 1 -- 172.21.15.117:0/478284487 learned_addr learned my addr 172.21.15.117:0/478284487 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:38.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.670+0000 7f38eb7fe640 1 --2- 172.21.15.117:0/478284487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0fd08b390 0x55f0fd45bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:38.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.670+0000 7f38ea7fc640 1 --2- 172.21.15.117:0/478284487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0fd0831b0 0x55f0fd46b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:38.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.670+0000 7f38eaffd640 1 -- 172.21.15.117:0/478284487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0fd0831b0 msgr2=0x55f0fd46b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:38.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.670+0000 7f38eaffd640 1 --2- 172.21.15.117:0/478284487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0fd0831b0 0x55f0fd46b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.670+0000 7f38eaffd640 1 -- 172.21.15.117:0/478284487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0fd08b390 msgr2=0x55f0fd45bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:38.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.670+0000 7f38eaffd640 1 --2- 172.21.15.117:0/478284487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0fd08b390 0x55f0fd45bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.670+0000 7f38eaffd640 1 -- 172.21.15.117:0/478284487 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0fd455990 con 0x55f0fd06a9f0 2024-09-21T06:06:38.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.670+0000 7f38ea7fc640 1 --2- 172.21.15.117:0/478284487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0fd0831b0 0x55f0fd46b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:06:38.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.670+0000 7f38eaffd640 1 --2- 172.21.15.117:0/478284487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0fd06a9f0 0x55f0fd454de0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00bcb0 tx=0x7f38dc00b3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4db01a2b9285d5b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:38.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.670+0000 7f38d0ff9640 1 -- 172.21.15.117:0/478284487 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc015070 con 0x55f0fd06a9f0 2024-09-21T06:06:38.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.670+0000 7f38d0ff9640 1 -- 172.21.15.117:0/478284487 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38dc0054f0 con 0x55f0fd06a9f0 2024-09-21T06:06:38.673 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.670+0000 7f38d0ff9640 1 -- 172.21.15.117:0/478284487 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc004b80 con 0x55f0fd06a9f0 2024-09-21T06:06:38.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.671+0000 7f38f7c9a740 1 -- 172.21.15.117:0/478284487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0fd06a9f0 msgr2=0x55f0fd454de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:38.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.671+0000 7f38f7c9a740 1 --2- 172.21.15.117:0/478284487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0fd06a9f0 0x55f0fd454de0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00bcb0 tx=0x7f38dc00b3a0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.671+0000 7f38f7c9a740 1 -- 172.21.15.117:0/478284487 shutdown_connections 2024-09-21T06:06:38.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.671+0000 7f38f7c9a740 1 --2- 172.21.15.117:0/478284487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0fd08b390 0x55f0fd45bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.671+0000 7f38f7c9a740 1 --2- 172.21.15.117:0/478284487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0fd0831b0 0x55f0fd46b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.671+0000 7f38f7c9a740 1 --2- 172.21.15.117:0/478284487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0fd06a9f0 0x55f0fd454de0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.671+0000 7f38f7c9a740 1 -- 172.21.15.117:0/478284487 >> 172.21.15.117:0/478284487 conn(0x55f0fd0a0460 msgr2=0x55f0fd4546d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:38.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.671+0000 7f38f7c9a740 1 -- 172.21.15.117:0/478284487 shutdown_connections 2024-09-21T06:06:38.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.671+0000 7f38f7c9a740 1 -- 172.21.15.117:0/478284487 wait complete. 2024-09-21T06:06:38.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.671+0000 7f38f7c9a740 1 -- start start 2024-09-21T06:06:38.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.672+0000 7f38f7c9a740 1 Processor -- start 2024-09-21T06:06:38.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.672+0000 7f38f7c9a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0fd06a9f0 0x55f0fd463a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:38.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.672+0000 7f38f7c9a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0fd0831b0 0x55f0fd464050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:38.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.672+0000 7f38f7c9a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0fd08b390 0x55f0fd4d4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:38.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.672+0000 7f38f7c9a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec1b0a70 con 0x55f0fd08b390 2024-09-21T06:06:38.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.672+0000 7f38f7c9a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec1b03a0 con 0x55f0fd0831b0 2024-09-21T06:06:38.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.672+0000 7f38f7c9a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec1ad600 con 0x55f0fd06a9f0 2024-09-21T06:06:38.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.672+0000 7f38f7c9a740 10 client.0 _ll_register_callbacks cb 0x7f38e400af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:38.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.672+0000 7f38eb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0fd08b390 0x55f0fd4d4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:38.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.672+0000 7f38eb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0fd08b390 0x55f0fd4d4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53496/0 (socket says 172.21.15.117:53496) 2024-09-21T06:06:38.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.672+0000 7f38eb7fe640 1 -- 172.21.15.117:0/1091811159 learned_addr learned my addr 172.21.15.117:0/1091811159 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:38.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.672+0000 7f38eaffd640 1 --2- 172.21.15.117:0/1091811159 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0fd06a9f0 0x55f0fd463a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:38.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.672+0000 7f38ea7fc640 1 --2- 172.21.15.117:0/1091811159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0fd0831b0 0x55f0fd464050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:38.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.672+0000 7f38eb7fe640 1 -- 172.21.15.117:0/1091811159 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0fd06a9f0 msgr2=0x55f0fd463a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:38.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.672+0000 7f38eb7fe640 1 --2- 172.21.15.117:0/1091811159 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0fd06a9f0 0x55f0fd463a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.672+0000 7f38eb7fe640 1 -- 172.21.15.117:0/1091811159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0fd0831b0 msgr2=0x55f0fd464050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:38.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.672+0000 7f38eb7fe640 1 --2- 172.21.15.117:0/1091811159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0fd0831b0 0x55f0fd464050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.672+0000 7f38eb7fe640 1 -- 172.21.15.117:0/1091811159 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0fd4d5170 con 0x55f0fd08b390 2024-09-21T06:06:38.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.672+0000 7f38eaffd640 1 --2- 172.21.15.117:0/1091811159 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0fd06a9f0 0x55f0fd463a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:06:38.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.672+0000 7f38ea7fc640 1 --2- 172.21.15.117:0/1091811159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0fd0831b0 0x55f0fd464050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:06:38.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.673+0000 7f38eb7fe640 1 --2- 172.21.15.117:0/1091811159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0fd08b390 0x55f0fd4d4b00 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f38e40ad770 tx=0x7f38e400ff50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2137bacc1eb30857 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:38.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.673+0000 7f38ceff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:38.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.673+0000 7f38ceff5640 1 -- 172.21.15.117:0/1091811159 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e40b2040 con 0x55f0fd08b390 2024-09-21T06:06:38.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.673+0000 7f38ceff5640 1 -- 172.21.15.117:0/1091811159 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7f38e40accd0 con 0x55f0fd08b390 2024-09-21T06:06:38.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.673+0000 7f38f7c9a740 1 -- 172.21.15.117:0/1091811159 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38dc0054f0 con 0x55f0fd08b390 2024-09-21T06:06:38.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.673+0000 7f38ceff5640 1 -- 172.21.15.117:0/1091811159 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f38e406f030 con 0x55f0fd08b390 2024-09-21T06:06:38.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.673+0000 7f38f7c9a740 10 client.17433 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:38.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.673+0000 7f38f7c9a740 1 -- 172.21.15.117:0/1091811159 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f0fd4dc450 con 0x55f0fd08b390 2024-09-21T06:06:38.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.673+0000 7f38ceff5640 1 -- 172.21.15.117:0/1091811159 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38dc0054f0 con 0x55f0fd08b390 2024-09-21T06:06:38.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.673+0000 7f38ceff5640 1 -- 172.21.15.117:0/1091811159 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f38e40ac740 con 0x55f0fd08b390 2024-09-21T06:06:38.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.673+0000 7f38f7c9a740 10 client.17433 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:38.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.673+0000 7f38f7c9a740 10 client.17433 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:06:38.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.673+0000 7f38f7c9a740 20 client.17433 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:38.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.673+0000 7f38f7c9a740 1 --2- 172.21.15.117:0/1091811159 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55f0fd0ac070 0x55f0fd4debf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:38.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.673+0000 7f38f7c9a740 4 client.17433 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:06:38.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.673+0000 7f38f7c9a740 1 -- 172.21.15.117:0/1091811159 --> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f0fd3c78a0 con 0x55f0fd0ac070 2024-09-21T06:06:38.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.674+0000 7f38eaffd640 1 --2- 172.21.15.117:0/1091811159 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55f0fd0ac070 0x55f0fd4debf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:38.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.674+0000 7f38ceff5640 10 client.17433 ms_handle_connect on v2:172.21.15.192:6838/1727623231 2024-09-21T06:06:38.680 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.674+0000 7f38eaffd640 1 --2- 172.21.15.117:0/1091811159 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55f0fd0ac070 0x55f0fd4debf0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7ec5eeca6e6a7051 server_cookie=fdd815bff39b6004 in_seq=0 out_seq=0 2024-09-21T06:06:38.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.674+0000 7f38ceff5640 1 -- 172.21.15.117:0/1091811159 <== mds.1 v2:172.21.15.192:6838/1727623231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f0fd3c78a0 con 0x55f0fd0ac070 2024-09-21T06:06:38.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.674+0000 7f38ceff5640 10 client.17433 handle_command_reply: tid=0 2024-09-21T06:06:38.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.674+0000 7f38f7c9a740 1 client.17433 shutdown 2024-09-21T06:06:38.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.674+0000 7f38f7c9a740 1 -- 172.21.15.117:0/1091811159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0fd08b390 msgr2=0x55f0fd4d4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:38.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.674+0000 7f38f7c9a740 1 --2- 172.21.15.117:0/1091811159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0fd08b390 0x55f0fd4d4b00 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f38e40ad770 tx=0x7f38e400ff50 comp rx=0 tx=0).stop 2024-09-21T06:06:38.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.675+0000 7f38f7c9a740 1 -- 172.21.15.117:0/1091811159 shutdown_connections 2024-09-21T06:06:38.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.675+0000 7f38f7c9a740 1 --2- 172.21.15.117:0/1091811159 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55f0fd0ac070 0x55f0fd4debf0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.675+0000 7f38f7c9a740 1 --2- 172.21.15.117:0/1091811159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0fd08b390 0x55f0fd4d4b00 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.675+0000 7f38ceff5640 0 client.17433 ms_handle_reset on v2:172.21.15.192:6838/1727623231 2024-09-21T06:06:38.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.675+0000 7f38f7c9a740 1 --2- 172.21.15.117:0/1091811159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0fd0831b0 0x55f0fd464050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.675+0000 7f38f7c9a740 1 --2- 172.21.15.117:0/1091811159 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f0fd06a9f0 0x55f0fd463a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.675+0000 7f38f7c9a740 1 -- 172.21.15.117:0/1091811159 >> 172.21.15.117:0/1091811159 conn(0x55f0fd0a0460 msgr2=0x55f0fd460260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:38.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.675+0000 7f38f7c9a740 1 -- 172.21.15.117:0/1091811159 shutdown_connections 2024-09-21T06:06:38.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.675+0000 7f38f7c9a740 1 -- 172.21.15.117:0/1091811159 wait complete. 2024-09-21T06:06:38.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.675+0000 7f38f7c9a740 20 client.17433 trim_cache size 0 max 16384 2024-09-21T06:06:38.682 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:06:38.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.675+0000 7f38f1726640 1 -- 172.21.15.117:0/1895057409 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f38d40b66b0 msgr2=0x7f38d40b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:38.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.675+0000 7f38f1726640 1 --2- 172.21.15.117:0/1895057409 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f38d40b66b0 0x7f38d40b8bc0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f38e0009230 tx=0x7f38e0000bc0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.675+0000 7f38f1726640 1 -- 172.21.15.117:0/1895057409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38ec109df0 msgr2=0x7f38ec19fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:38.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.675+0000 7f38f1726640 1 --2- 172.21.15.117:0/1895057409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38ec109df0 0x7f38ec19fd00 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f38e400cce0 tx=0x7f38e400a620 comp rx=0 tx=0).stop 2024-09-21T06:06:38.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.675+0000 7f38f1726640 1 -- 172.21.15.117:0/1895057409 shutdown_connections 2024-09-21T06:06:38.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.675+0000 7f38f1726640 1 --2- 172.21.15.117:0/1895057409 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f38d40b66b0 0x7f38d40b8bc0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.675+0000 7f38f1726640 1 --2- 172.21.15.117:0/1895057409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38ec109df0 0x7f38ec19fd00 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.675+0000 7f38f1726640 1 --2- 172.21.15.117:0/1895057409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38ec1094a0 0x7f38ec19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.675+0000 7f38f1726640 1 --2- 172.21.15.117:0/1895057409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38ec103b60 0x7f38ec19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:38.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.675+0000 7f38f1726640 1 -- 172.21.15.117:0/1895057409 >> 172.21.15.117:0/1895057409 conn(0x7f38ec07b850 msgr2=0x7f38ec105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:38.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.676+0000 7f38f1726640 1 -- 172.21.15.117:0/1895057409 shutdown_connections 2024-09-21T06:06:38.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:38.676+0000 7f38f1726640 1 -- 172.21.15.117:0/1895057409 wait complete. 2024-09-21T06:06:39.422 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:06:39.669 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:40.088 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:39 smithi117 ceph-mon[30947]: pgmap v612: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:06:40.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.178+0000 7fe50a7e9640 1 -- 172.21.15.117:0/3101085159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5041083d0 msgr2=0x7fe5041087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:40.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.178+0000 7fe50a7e9640 1 --2- 172.21.15.117:0/3101085159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5041083d0 0x7fe5041087e0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8009900 tx=0x7fe4f802a410 comp rx=0 tx=0).stop 2024-09-21T06:06:40.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.178+0000 7fe50a7e9640 1 -- 172.21.15.117:0/3101085159 shutdown_connections 2024-09-21T06:06:40.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.178+0000 7fe50a7e9640 1 --2- 172.21.15.117:0/3101085159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5041083d0 0x7fe5041087e0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.178+0000 7fe50a7e9640 1 --2- 172.21.15.117:0/3101085159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe504108d20 0x7fe50410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.178+0000 7fe50a7e9640 1 --2- 172.21.15.117:0/3101085159 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe504102ae0 0x7fe504102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.178+0000 7fe50a7e9640 1 -- 172.21.15.117:0/3101085159 >> 172.21.15.117:0/3101085159 conn(0x7fe504079790 msgr2=0x7fe504079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:40.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.178+0000 7fe50a7e9640 1 -- 172.21.15.117:0/3101085159 shutdown_connections 2024-09-21T06:06:40.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.178+0000 7fe50a7e9640 1 -- 172.21.15.117:0/3101085159 wait complete. 2024-09-21T06:06:40.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.178+0000 7fe50a7e9640 1 Processor -- start 2024-09-21T06:06:40.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.178+0000 7fe50a7e9640 1 -- start start 2024-09-21T06:06:40.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.179+0000 7fe50a7e9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe504102ae0 0x7fe50419b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:40.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.179+0000 7fe50a7e9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5041083d0 0x7fe50419b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:40.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.179+0000 7fe50a7e9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe504108d20 0x7fe50419bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:40.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.179+0000 7fe50a7e9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe5041129e0 con 0x7fe504108d20 2024-09-21T06:06:40.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.179+0000 7fe50a7e9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe504112860 con 0x7fe504102ae0 2024-09-21T06:06:40.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.179+0000 7fe50a7e9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe504112b60 con 0x7fe5041083d0 2024-09-21T06:06:40.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.179+0000 7fe508d86640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe504108d20 0x7fe50419bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:40.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.179+0000 7fe508d86640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe504108d20 0x7fe50419bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53514/0 (socket says 172.21.15.117:53514) 2024-09-21T06:06:40.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.179+0000 7fe508d86640 1 -- 172.21.15.117:0/974130534 learned_addr learned my addr 172.21.15.117:0/974130534 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:40.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.179+0000 7fe503fff640 1 --2- 172.21.15.117:0/974130534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe504102ae0 0x7fe50419b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:40.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.179+0000 7fe5037fe640 1 --2- 172.21.15.117:0/974130534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5041083d0 0x7fe50419b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:40.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.179+0000 7fe508d86640 1 -- 172.21.15.117:0/974130534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5041083d0 msgr2=0x7fe50419b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:40.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.179+0000 7fe508d86640 1 --2- 172.21.15.117:0/974130534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5041083d0 0x7fe50419b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.179+0000 7fe508d86640 1 -- 172.21.15.117:0/974130534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe504102ae0 msgr2=0x7fe50419b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:40.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.179+0000 7fe508d86640 1 --2- 172.21.15.117:0/974130534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe504102ae0 0x7fe50419b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.179+0000 7fe508d86640 1 -- 172.21.15.117:0/974130534 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5041a6560 con 0x7fe504108d20 2024-09-21T06:06:40.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.179+0000 7fe508d86640 1 --2- 172.21.15.117:0/974130534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe504108d20 0x7fe50419bcd0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00eb10 tx=0x7fe4fc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:40.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.180+0000 7fe5017fa640 1 -- 172.21.15.117:0/974130534 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc004b20 con 0x7fe504108d20 2024-09-21T06:06:40.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.180+0000 7fe5017fa640 1 -- 172.21.15.117:0/974130534 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4fc004cc0 con 0x7fe504108d20 2024-09-21T06:06:40.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.180+0000 7fe5017fa640 1 -- 172.21.15.117:0/974130534 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc010670 con 0x7fe504108d20 2024-09-21T06:06:40.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.180+0000 7fe50a7e9640 1 -- 172.21.15.117:0/974130534 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5041a6850 con 0x7fe504108d20 2024-09-21T06:06:40.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.180+0000 7fe50a7e9640 1 -- 172.21.15.117:0/974130534 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5041a6d10 con 0x7fe504108d20 2024-09-21T06:06:40.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.182+0000 7fe510d5d740 1 Processor -- start 2024-09-21T06:06:40.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.183+0000 7fe5017fa640 1 -- 172.21.15.117:0/974130534 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fe4fc005020 con 0x7fe504108d20 2024-09-21T06:06:40.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.183+0000 7fe510d5d740 1 -- start start 2024-09-21T06:06:40.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.183+0000 7fe510d5d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563a262c3070 0x563a2666a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:40.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.183+0000 7fe510d5d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a2629a1b0 0x563a2666a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:40.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.183+0000 7fe510d5d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a262a2390 0x563a2666ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:40.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.183+0000 7fe510d5d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563a265e0ea0 con 0x563a262a2390 2024-09-21T06:06:40.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.183+0000 7fe510d5d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563a265d02d0 con 0x563a2629a1b0 2024-09-21T06:06:40.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.183+0000 7fe510d5d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x563a265d40e0 con 0x563a262c3070 2024-09-21T06:06:40.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.183+0000 7fe5037fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a262a2390 0x563a2666ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:40.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.183+0000 7fe5037fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a262a2390 0x563a2666ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53516/0 (socket says 172.21.15.117:53516) 2024-09-21T06:06:40.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.183+0000 7fe5037fe640 1 -- 172.21.15.117:0/829423620 learned_addr learned my addr 172.21.15.117:0/829423620 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:40.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.183+0000 7fe503fff640 1 --2- 172.21.15.117:0/829423620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a2629a1b0 0x563a2666a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:40.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.183+0000 7fe5037fe640 1 --2- 172.21.15.117:0/829423620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563a262c3070 0x563a2666a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:40.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.183+0000 7fe503fff640 1 -- 172.21.15.117:0/829423620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563a262c3070 msgr2=0x563a2666a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:40.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.183+0000 7fe503fff640 1 --2- 172.21.15.117:0/829423620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563a262c3070 0x563a2666a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.183+0000 7fe5017fa640 1 --2- 172.21.15.117:0/974130534 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe4ec0b6680 0x7fe4ec0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:40.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.183+0000 7fe503fff640 1 -- 172.21.15.117:0/829423620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a262a2390 msgr2=0x563a2666ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:40.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.183+0000 7fe503fff640 1 --2- 172.21.15.117:0/829423620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a262a2390 0x563a2666ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.183+0000 7fe503fff640 1 -- 172.21.15.117:0/829423620 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a2632e650 con 0x563a2629a1b0 2024-09-21T06:06:40.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.184+0000 7fe508d86640 1 --2- 172.21.15.117:0/974130534 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe4ec0b6680 0x7fe4ec0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:40.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.184+0000 7fe503fff640 1 --2- 172.21.15.117:0/829423620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a2629a1b0 0x563a2666a800 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe50406c6e0 tx=0x7fe4f400f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e8fdb5032d3f16 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:40.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.184+0000 7fe5017fa640 1 -- 172.21.15.117:0/974130534 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fe4fc0e7d70 con 0x7fe504108d20 2024-09-21T06:06:40.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.184+0000 7fe4ea7fc640 1 -- 172.21.15.117:0/829423620 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4005020 con 0x563a2629a1b0 2024-09-21T06:06:40.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.184+0000 7fe4ea7fc640 1 -- 172.21.15.117:0/829423620 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4f4005670 con 0x563a2629a1b0 2024-09-21T06:06:40.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.184+0000 7fe4ea7fc640 1 -- 172.21.15.117:0/829423620 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f40186a0 con 0x563a2629a1b0 2024-09-21T06:06:40.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.185+0000 7fe510d5d740 1 -- 172.21.15.117:0/829423620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a2629a1b0 msgr2=0x563a2666a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:40.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.185+0000 7fe508d86640 1 --2- 172.21.15.117:0/974130534 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe4ec0b6680 0x7fe4ec0b8b90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc005f70 tx=0x7fe4fc07aef0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:40.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.185+0000 7fe510d5d740 1 --2- 172.21.15.117:0/829423620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a2629a1b0 0x563a2666a800 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe50406c6e0 tx=0x7fe4f400f3c0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.185+0000 7fe510d5d740 1 -- 172.21.15.117:0/829423620 shutdown_connections 2024-09-21T06:06:40.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.185+0000 7fe510d5d740 1 --2- 172.21.15.117:0/829423620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a262a2390 0x563a2666ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.185+0000 7fe510d5d740 1 --2- 172.21.15.117:0/829423620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a2629a1b0 0x563a2666a800 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.185+0000 7fe510d5d740 1 --2- 172.21.15.117:0/829423620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563a262c3070 0x563a2666a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.185+0000 7fe510d5d740 1 -- 172.21.15.117:0/829423620 >> 172.21.15.117:0/829423620 conn(0x563a262b7460 msgr2=0x563a266681b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:40.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.185+0000 7fe510d5d740 1 -- 172.21.15.117:0/829423620 shutdown_connections 2024-09-21T06:06:40.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.185+0000 7fe510d5d740 1 -- 172.21.15.117:0/829423620 wait complete. 2024-09-21T06:06:40.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.185+0000 7fe510d5d740 1 -- start start 2024-09-21T06:06:40.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.186+0000 7fe510d5d740 1 Processor -- start 2024-09-21T06:06:40.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.186+0000 7fe510d5d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563a2629a1b0 0x563a26675930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:40.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.186+0000 7fe510d5d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a262a2390 0x563a26675e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:40.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.186+0000 7fe510d5d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a262c3070 0x563a26681d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:40.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.186+0000 7fe510d5d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563a26431700 con 0x563a262c3070 2024-09-21T06:06:40.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.186+0000 7fe510d5d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563a2656b160 con 0x563a262a2390 2024-09-21T06:06:40.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.186+0000 7fe510d5d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x563a26678aa0 con 0x563a2629a1b0 2024-09-21T06:06:40.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.186+0000 7fe510d5d740 10 client.0 _ll_register_callbacks cb 0x563a265849e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:40.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.186+0000 7fe5037fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a262c3070 0x563a26681d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:40.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.186+0000 7fe5037fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a262c3070 0x563a26681d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53520/0 (socket says 172.21.15.117:53520) 2024-09-21T06:06:40.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.186+0000 7fe5037fe640 1 -- 172.21.15.117:0/102641763 learned_addr learned my addr 172.21.15.117:0/102641763 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:40.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.186+0000 7fe503fff640 1 --2- 172.21.15.117:0/102641763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a262a2390 0x563a26675e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:40.218 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.186+0000 7fe5037fe640 1 --2- 172.21.15.117:0/102641763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563a2629a1b0 0x563a26675930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:40.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.186+0000 7fe5037fe640 1 -- 172.21.15.117:0/102641763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563a2629a1b0 msgr2=0x563a26675930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:40.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.186+0000 7fe5037fe640 1 --2- 172.21.15.117:0/102641763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563a2629a1b0 0x563a26675930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.186+0000 7fe5037fe640 1 -- 172.21.15.117:0/102641763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a262a2390 msgr2=0x563a26675e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:40.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.186+0000 7fe5037fe640 1 --2- 172.21.15.117:0/102641763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a262a2390 0x563a26675e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.186+0000 7fe5037fe640 1 -- 172.21.15.117:0/102641763 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a26598c90 con 0x563a262c3070 2024-09-21T06:06:40.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.186+0000 7fe503fff640 1 --2- 172.21.15.117:0/102641763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a262a2390 0x563a26675e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:06:40.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.186+0000 7fe5037fe640 1 --2- 172.21.15.117:0/102641763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a262c3070 0x563a26681d90 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8005ea0 tx=0x7fe4f8032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c1159ebfe0f908 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:40.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.186+0000 7fe4e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:40.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.187+0000 7fe4e37fe640 1 -- 172.21.15.117:0/102641763 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f8035040 con 0x563a262c3070 2024-09-21T06:06:40.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.187+0000 7fe4e37fe640 1 -- 172.21.15.117:0/102641763 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fe4f8004480 con 0x563a262c3070 2024-09-21T06:06:40.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.187+0000 7fe510d5d740 1 -- 172.21.15.117:0/102641763 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4f4005670 con 0x563a262c3070 2024-09-21T06:06:40.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.187+0000 7fe4e37fe640 1 -- 172.21.15.117:0/102641763 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe4f804f030 con 0x563a262c3070 2024-09-21T06:06:40.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.187+0000 7fe510d5d740 10 client.17445 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:40.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.187+0000 7fe510d5d740 1 -- 172.21.15.117:0/102641763 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563a26684770 con 0x563a262c3070 2024-09-21T06:06:40.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.187+0000 7fe4e37fe640 1 -- 172.21.15.117:0/102641763 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4f4005670 con 0x563a262c3070 2024-09-21T06:06:40.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.187+0000 7fe4e37fe640 1 -- 172.21.15.117:0/102641763 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe4f803a040 con 0x563a262c3070 2024-09-21T06:06:40.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.187+0000 7fe510d5d740 10 client.17445 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:40.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.187+0000 7fe510d5d740 10 client.17445 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:06:40.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.187+0000 7fe510d5d740 20 client.17445 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:40.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.187+0000 7fe510d5d740 1 --2- 172.21.15.117:0/102641763 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x563a262819f0 0x563a26687c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:40.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.187+0000 7fe510d5d740 4 client.17445 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:06:40.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.187+0000 7fe510d5d740 1 -- 172.21.15.117:0/102641763 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a265e2df0 con 0x563a262819f0 2024-09-21T06:06:40.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.188+0000 7fe503fff640 1 --2- 172.21.15.117:0/102641763 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x563a262819f0 0x563a26687c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:40.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.188+0000 7fe4e37fe640 10 client.17445 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:40.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.188+0000 7fe503fff640 1 --2- 172.21.15.117:0/102641763 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x563a262819f0 0x563a26687c20 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a40f28b7ae3a6104 server_cookie=5803d9eb69a1e174 in_seq=0 out_seq=0 2024-09-21T06:06:40.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.189+0000 7fe4e37fe640 1 -- 172.21.15.117:0/102641763 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563a265e2df0 con 0x563a262819f0 2024-09-21T06:06:40.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.189+0000 7fe4e37fe640 10 client.17445 handle_command_reply: tid=0 2024-09-21T06:06:40.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.189+0000 7fe510d5d740 1 client.17445 shutdown 2024-09-21T06:06:40.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.189+0000 7fe510d5d740 1 -- 172.21.15.117:0/102641763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a262c3070 msgr2=0x563a26681d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:40.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.189+0000 7fe510d5d740 1 --2- 172.21.15.117:0/102641763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a262c3070 0x563a26681d90 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8005ea0 tx=0x7fe4f8032b20 comp rx=0 tx=0).stop 2024-09-21T06:06:40.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.189+0000 7fe510d5d740 1 -- 172.21.15.117:0/102641763 shutdown_connections 2024-09-21T06:06:40.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.189+0000 7fe510d5d740 1 --2- 172.21.15.117:0/102641763 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x563a262819f0 0x563a26687c20 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.189+0000 7fe510d5d740 1 --2- 172.21.15.117:0/102641763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a262c3070 0x563a26681d90 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.189+0000 7fe510d5d740 1 --2- 172.21.15.117:0/102641763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a262a2390 0x563a26675e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.189+0000 7fe4e37fe640 0 client.17445 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:40.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.189+0000 7fe510d5d740 1 --2- 172.21.15.117:0/102641763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563a2629a1b0 0x563a26675930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.189+0000 7fe510d5d740 1 -- 172.21.15.117:0/102641763 >> 172.21.15.117:0/102641763 conn(0x563a262b7460 msgr2=0x563a2666d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:40.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.189+0000 7fe510d5d740 1 -- 172.21.15.117:0/102641763 shutdown_connections 2024-09-21T06:06:40.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.189+0000 7fe510d5d740 1 -- 172.21.15.117:0/102641763 wait complete. 2024-09-21T06:06:40.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.189+0000 7fe510d5d740 20 client.17445 trim_cache size 0 max 16384 2024-09-21T06:06:40.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.202+0000 7fe510d5d740 1 Processor -- start 2024-09-21T06:06:40.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.202+0000 7fe510d5d740 1 -- start start 2024-09-21T06:06:40.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.202+0000 7fe510d5d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563a2629a1b0 0x563a2666bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:40.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.202+0000 7fe510d5d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a262a2390 0x563a26677560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:40.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.202+0000 7fe510d5d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a262c3070 0x563a26681c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:40.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.202+0000 7fe510d5d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe5041af550 con 0x563a262c3070 2024-09-21T06:06:40.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.202+0000 7fe510d5d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe5041b1090 con 0x563a262a2390 2024-09-21T06:06:40.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.202+0000 7fe510d5d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe5041afc20 con 0x563a2629a1b0 2024-09-21T06:06:40.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.203+0000 7fe5037fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a262c3070 0x563a26681c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:40.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.203+0000 7fe5037fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a262c3070 0x563a26681c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53532/0 (socket says 172.21.15.117:53532) 2024-09-21T06:06:40.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.203+0000 7fe5037fe640 1 -- 172.21.15.117:0/1913559536 learned_addr learned my addr 172.21.15.117:0/1913559536 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:40.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.203+0000 7fe503fff640 1 --2- 172.21.15.117:0/1913559536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a262a2390 0x563a26677560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:40.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.203+0000 7fe5037fe640 1 --2- 172.21.15.117:0/1913559536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563a2629a1b0 0x563a2666bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:40.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.203+0000 7fe5037fe640 1 -- 172.21.15.117:0/1913559536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563a2629a1b0 msgr2=0x563a2666bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:40.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.203+0000 7fe5037fe640 1 --2- 172.21.15.117:0/1913559536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563a2629a1b0 0x563a2666bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.203+0000 7fe5037fe640 1 -- 172.21.15.117:0/1913559536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a262a2390 msgr2=0x563a26677560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:40.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.203+0000 7fe5037fe640 1 --2- 172.21.15.117:0/1913559536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a262a2390 0x563a26677560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.203+0000 7fe5037fe640 1 -- 172.21.15.117:0/1913559536 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a2666c7e0 con 0x563a262c3070 2024-09-21T06:06:40.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.203+0000 7fe5037fe640 1 --2- 172.21.15.117:0/1913559536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a262c3070 0x563a26681c90 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8009850 tx=0x7fe4f8031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a35562ebeb7029 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:40.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.203+0000 7fe4e9ffb640 1 -- 172.21.15.117:0/1913559536 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f8035040 con 0x563a262c3070 2024-09-21T06:06:40.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.203+0000 7fe4e9ffb640 1 -- 172.21.15.117:0/1913559536 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4f8004480 con 0x563a262c3070 2024-09-21T06:06:40.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.203+0000 7fe4e9ffb640 1 -- 172.21.15.117:0/1913559536 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f8003a50 con 0x563a262c3070 2024-09-21T06:06:40.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.204+0000 7fe510d5d740 1 -- 172.21.15.117:0/1913559536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a262c3070 msgr2=0x563a26681c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:40.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.204+0000 7fe510d5d740 1 --2- 172.21.15.117:0/1913559536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a262c3070 0x563a26681c90 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8009850 tx=0x7fe4f8031400 comp rx=0 tx=0).stop 2024-09-21T06:06:40.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.204+0000 7fe510d5d740 1 -- 172.21.15.117:0/1913559536 shutdown_connections 2024-09-21T06:06:40.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.204+0000 7fe510d5d740 1 --2- 172.21.15.117:0/1913559536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a262c3070 0x563a26681c90 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.204+0000 7fe510d5d740 1 --2- 172.21.15.117:0/1913559536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a262a2390 0x563a26677560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.204+0000 7fe510d5d740 1 --2- 172.21.15.117:0/1913559536 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563a2629a1b0 0x563a2666bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.204+0000 7fe510d5d740 1 -- 172.21.15.117:0/1913559536 >> 172.21.15.117:0/1913559536 conn(0x563a262b7460 msgr2=0x563a26676e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:40.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.204+0000 7fe510d5d740 1 -- 172.21.15.117:0/1913559536 shutdown_connections 2024-09-21T06:06:40.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.204+0000 7fe510d5d740 1 -- 172.21.15.117:0/1913559536 wait complete. 2024-09-21T06:06:40.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.204+0000 7fe510d5d740 1 -- start start 2024-09-21T06:06:40.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.205+0000 7fe510d5d740 1 Processor -- start 2024-09-21T06:06:40.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.205+0000 7fe510d5d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563a2629a1b0 0x563a266796e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:40.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.205+0000 7fe510d5d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a262a2390 0x563a26679cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:40.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.205+0000 7fe510d5d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a262c3070 0x563a266ea610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:40.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.205+0000 7fe510d5d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe5041b09c0 con 0x563a262c3070 2024-09-21T06:06:40.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.205+0000 7fe510d5d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe5041b02f0 con 0x563a262a2390 2024-09-21T06:06:40.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.205+0000 7fe510d5d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe5041ad550 con 0x563a2629a1b0 2024-09-21T06:06:40.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.205+0000 7fe510d5d740 10 client.0 _ll_register_callbacks cb 0x7fe4f8031930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:40.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.205+0000 7fe503fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a262a2390 0x563a26679cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:40.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.205+0000 7fe503fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a262a2390 0x563a26679cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:58562/0 (socket says 172.21.15.117:58562) 2024-09-21T06:06:40.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.205+0000 7fe503fff640 1 -- 172.21.15.117:0/3160757313 learned_addr learned my addr 172.21.15.117:0/3160757313 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:40.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.205+0000 7fe5037fe640 1 --2- 172.21.15.117:0/3160757313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a262c3070 0x563a266ea610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:40.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.205+0000 7fe5037fe640 1 --2- 172.21.15.117:0/3160757313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563a2629a1b0 0x563a266796e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:40.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.206+0000 7fe503fff640 1 -- 172.21.15.117:0/3160757313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563a2629a1b0 msgr2=0x563a266796e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:40.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.206+0000 7fe503fff640 1 --2- 172.21.15.117:0/3160757313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563a2629a1b0 0x563a266796e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.206+0000 7fe503fff640 1 -- 172.21.15.117:0/3160757313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a262c3070 msgr2=0x563a266ea610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:40.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.206+0000 7fe503fff640 1 --2- 172.21.15.117:0/3160757313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a262c3070 0x563a266ea610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.206+0000 7fe503fff640 1 -- 172.21.15.117:0/3160757313 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a266eac80 con 0x563a262a2390 2024-09-21T06:06:40.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.206+0000 7fe5037fe640 1 --2- 172.21.15.117:0/3160757313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a262c3070 0x563a266ea610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:06:40.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.206+0000 7fe503fff640 1 --2- 172.21.15.117:0/3160757313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a262a2390 0x563a26679cc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x563a26668f40 tx=0x7fe4f400a450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f287f30dacc916e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:40.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.206+0000 7fe4e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-21T06:06:40.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.206+0000 7fe4e37fe640 1 -- 172.21.15.117:0/3160757313 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f402f030 con 0x563a262a2390 2024-09-21T06:06:40.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.206+0000 7fe4e37fe640 1 -- 172.21.15.117:0/3160757313 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fe4f401e040 con 0x563a262a2390 2024-09-21T06:06:40.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.206+0000 7fe510d5d740 1 -- 172.21.15.117:0/3160757313 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4f8004480 con 0x563a262a2390 2024-09-21T06:06:40.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.206+0000 7fe4e37fe640 1 -- 172.21.15.117:0/3160757313 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe4f4038020 con 0x563a262a2390 2024-09-21T06:06:40.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.206+0000 7fe510d5d740 10 client.26471 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:40.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.206+0000 7fe510d5d740 1 -- 172.21.15.117:0/3160757313 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563a266f2070 con 0x563a262a2390 2024-09-21T06:06:40.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.206+0000 7fe4e37fe640 1 -- 172.21.15.117:0/3160757313 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4f8004480 con 0x563a262a2390 2024-09-21T06:06:40.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.207+0000 7fe4e37fe640 1 -- 172.21.15.117:0/3160757313 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe4f4018770 con 0x563a262a2390 2024-09-21T06:06:40.225 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.207+0000 7fe510d5d740 10 client.26471 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:40.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.207+0000 7fe510d5d740 10 client.26471 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:06:40.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.207+0000 7fe510d5d740 20 client.26471 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:40.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.207+0000 7fe510d5d740 1 --2- 172.21.15.117:0/3160757313 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x563a26256da0 0x563a266f44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:40.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.207+0000 7fe510d5d740 4 client.26471 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:06:40.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.207+0000 7fe510d5d740 1 -- 172.21.15.117:0/3160757313 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563a265de8a0 con 0x563a26256da0 2024-09-21T06:06:40.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.207+0000 7fe5037fe640 1 --2- 172.21.15.117:0/3160757313 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x563a26256da0 0x563a266f44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:40.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.207+0000 7fe4e37fe640 10 client.26471 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:40.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.207+0000 7fe5037fe640 1 --2- 172.21.15.117:0/3160757313 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x563a26256da0 0x563a266f44c0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d6619d43ebc9b54 server_cookie=cf6fc1fa2694e8e in_seq=0 out_seq=0 2024-09-21T06:06:40.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.207+0000 7fe4e37fe640 1 -- 172.21.15.117:0/3160757313 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563a265de8a0 con 0x563a26256da0 2024-09-21T06:06:40.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.207+0000 7fe4e37fe640 10 client.26471 handle_command_reply: tid=0 2024-09-21T06:06:40.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.207+0000 7fe510d5d740 1 client.26471 shutdown 2024-09-21T06:06:40.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.208+0000 7fe510d5d740 1 -- 172.21.15.117:0/3160757313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a262a2390 msgr2=0x563a26679cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:40.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.208+0000 7fe510d5d740 1 --2- 172.21.15.117:0/3160757313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a262a2390 0x563a26679cc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x563a26668f40 tx=0x7fe4f400a450 comp rx=0 tx=0).stop 2024-09-21T06:06:40.226 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.208+0000 7fe510d5d740 1 -- 172.21.15.117:0/3160757313 shutdown_connections 2024-09-21T06:06:40.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.208+0000 7fe510d5d740 1 --2- 172.21.15.117:0/3160757313 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x563a26256da0 0x563a266f44c0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.208+0000 7fe510d5d740 1 --2- 172.21.15.117:0/3160757313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563a262c3070 0x563a266ea610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.208+0000 7fe510d5d740 1 --2- 172.21.15.117:0/3160757313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563a262a2390 0x563a26679cc0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.208+0000 7fe510d5d740 1 --2- 172.21.15.117:0/3160757313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563a2629a1b0 0x563a266796e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.208+0000 7fe510d5d740 1 -- 172.21.15.117:0/3160757313 >> 172.21.15.117:0/3160757313 conn(0x563a262b7460 msgr2=0x563a2666ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:40.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.208+0000 7fe4e37fe640 0 client.26471 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:40.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.208+0000 7fe510d5d740 1 -- 172.21.15.117:0/3160757313 shutdown_connections 2024-09-21T06:06:40.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.208+0000 7fe510d5d740 1 -- 172.21.15.117:0/3160757313 wait complete. 2024-09-21T06:06:40.227 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.208+0000 7fe510d5d740 20 client.26471 trim_cache size 0 max 16384 2024-09-21T06:06:40.227 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:06:40.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.208+0000 7fe50a7e9640 1 -- 172.21.15.117:0/974130534 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe4ec0b6680 msgr2=0x7fe4ec0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:40.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.208+0000 7fe50a7e9640 1 --2- 172.21.15.117:0/974130534 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe4ec0b6680 0x7fe4ec0b8b90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc005f70 tx=0x7fe4fc07aef0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.208+0000 7fe50a7e9640 1 -- 172.21.15.117:0/974130534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe504108d20 msgr2=0x7fe50419bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:40.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.208+0000 7fe50a7e9640 1 --2- 172.21.15.117:0/974130534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe504108d20 0x7fe50419bcd0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00eb10 tx=0x7fe4fc00c2d0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.209+0000 7fe50a7e9640 1 -- 172.21.15.117:0/974130534 shutdown_connections 2024-09-21T06:06:40.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.209+0000 7fe50a7e9640 1 --2- 172.21.15.117:0/974130534 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe4ec0b6680 0x7fe4ec0b8b90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.209+0000 7fe50a7e9640 1 --2- 172.21.15.117:0/974130534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe504108d20 0x7fe50419bcd0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.209+0000 7fe50a7e9640 1 --2- 172.21.15.117:0/974130534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5041083d0 0x7fe50419b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.209+0000 7fe50a7e9640 1 --2- 172.21.15.117:0/974130534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe504102ae0 0x7fe50419b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:40.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.209+0000 7fe50a7e9640 1 -- 172.21.15.117:0/974130534 >> 172.21.15.117:0/974130534 conn(0x7fe504079790 msgr2=0x7fe504106440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:40.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.209+0000 7fe50a7e9640 1 -- 172.21.15.117:0/974130534 shutdown_connections 2024-09-21T06:06:40.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:40.209+0000 7fe50a7e9640 1 -- 172.21.15.117:0/974130534 wait complete. 2024-09-21T06:06:40.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:39 smithi192 ceph-mon[34700]: pgmap v612: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:06:40.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:39 smithi138 ceph-mon[34391]: pgmap v612: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:06:42.083 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:06:42.083 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:06:42.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:41 smithi192 ceph-mon[34700]: pgmap v613: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:06:42.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:41 smithi117 ceph-mon[30947]: pgmap v613: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:06:42.346 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:42.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:41 smithi138 ceph-mon[34391]: pgmap v613: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:06:42.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.867+0000 7f0f1a28c640 1 -- 172.21.15.117:0/1911939593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f14103b60 msgr2=0x7f0f14103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:42.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.867+0000 7f0f1a28c640 1 --2- 172.21.15.117:0/1911939593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f14103b60 0x7f0f14103f70 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f0f04009900 tx=0x7f0f0402a410 comp rx=0 tx=0).stop 2024-09-21T06:06:42.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.867+0000 7f0f1a28c640 1 -- 172.21.15.117:0/1911939593 shutdown_connections 2024-09-21T06:06:42.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.867+0000 7f0f1a28c640 1 --2- 172.21.15.117:0/1911939593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f14109df0 0x7f0f14110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:42.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.867+0000 7f0f1a28c640 1 --2- 172.21.15.117:0/1911939593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f141094a0 0x7f0f141098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:42.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.867+0000 7f0f1a28c640 1 --2- 172.21.15.117:0/1911939593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f14103b60 0x7f0f14103f70 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:42.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.867+0000 7f0f1a28c640 1 -- 172.21.15.117:0/1911939593 >> 172.21.15.117:0/1911939593 conn(0x7f0f1407b850 msgr2=0x7f0f1407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:42.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.867+0000 7f0f1a28c640 1 -- 172.21.15.117:0/1911939593 shutdown_connections 2024-09-21T06:06:42.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.867+0000 7f0f1a28c640 1 -- 172.21.15.117:0/1911939593 wait complete. 2024-09-21T06:06:42.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.868+0000 7f0f1a28c640 1 Processor -- start 2024-09-21T06:06:42.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.868+0000 7f0f1a28c640 1 -- start start 2024-09-21T06:06:42.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.868+0000 7f0f1a28c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f14103b60 0x7f0f1419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:42.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.868+0000 7f0f1a28c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f141094a0 0x7f0f1419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:42.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.868+0000 7f0f1a28c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f14109df0 0x7f0f1419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:42.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.868+0000 7f0f1a28c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0f14113ab0 con 0x7f0f14103b60 2024-09-21T06:06:42.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.868+0000 7f0f1a28c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0f14113930 con 0x7f0f141094a0 2024-09-21T06:06:42.872 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.868+0000 7f0f1a28c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0f14113c30 con 0x7f0f14109df0 2024-09-21T06:06:42.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.868+0000 7f0f13fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f14103b60 0x7f0f1419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:42.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.868+0000 7f0f13fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f14103b60 0x7f0f1419a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53552/0 (socket says 172.21.15.117:53552) 2024-09-21T06:06:42.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.868+0000 7f0f13fff640 1 -- 172.21.15.117:0/2251966238 learned_addr learned my addr 172.21.15.117:0/2251966238 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:42.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.869+0000 7f0f137fe640 1 --2- 172.21.15.117:0/2251966238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f141094a0 0x7f0f1419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:42.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.869+0000 7f0f18829640 1 --2- 172.21.15.117:0/2251966238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f14109df0 0x7f0f1419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:42.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.869+0000 7f0f13fff640 1 -- 172.21.15.117:0/2251966238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f14109df0 msgr2=0x7f0f1419fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:42.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.869+0000 7f0f13fff640 1 --2- 172.21.15.117:0/2251966238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f14109df0 0x7f0f1419fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:42.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.869+0000 7f0f13fff640 1 -- 172.21.15.117:0/2251966238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f141094a0 msgr2=0x7f0f1419aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:42.873 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.869+0000 7f0f13fff640 1 --2- 172.21.15.117:0/2251966238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f141094a0 0x7f0f1419aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:42.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.869+0000 7f0f13fff640 1 -- 172.21.15.117:0/2251966238 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f141a03f0 con 0x7f0f14103b60 2024-09-21T06:06:42.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.869+0000 7f0f13fff640 1 --2- 172.21.15.117:0/2251966238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f14103b60 0x7f0f1419a480 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f0f04005940 tx=0x7f0f040040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:42.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.869+0000 7f0f117fa640 1 -- 172.21.15.117:0/2251966238 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0403f030 con 0x7f0f14103b60 2024-09-21T06:06:42.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.869+0000 7f0f1a28c640 1 -- 172.21.15.117:0/2251966238 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f141a0680 con 0x7f0f14103b60 2024-09-21T06:06:42.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.869+0000 7f0f117fa640 1 -- 172.21.15.117:0/2251966238 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f040044c0 con 0x7f0f14103b60 2024-09-21T06:06:42.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.869+0000 7f0f117fa640 1 -- 172.21.15.117:0/2251966238 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f04035070 con 0x7f0f14103b60 2024-09-21T06:06:42.874 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.869+0000 7f0f1a28c640 1 -- 172.21.15.117:0/2251966238 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f141a6e40 con 0x7f0f14103b60 2024-09-21T06:06:42.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.872+0000 7f0f117fa640 1 -- 172.21.15.117:0/2251966238 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f0f04042030 con 0x7f0f14103b60 2024-09-21T06:06:42.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.872+0000 7f0f20800740 1 Processor -- start 2024-09-21T06:06:42.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.872+0000 7f0f20800740 1 -- start start 2024-09-21T06:06:42.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.872+0000 7f0f20800740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e85c350 0x56541ec03430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:42.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.872+0000 7f0f20800740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56541e833490 0x56541ec03970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:42.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.872+0000 7f0f20800740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56541e83b670 0x56541ec03eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:42.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.872+0000 7f0f20800740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56541eb79e60 con 0x56541e83b670 2024-09-21T06:06:42.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.872+0000 7f0f20800740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56541eb69290 con 0x56541e85c350 2024-09-21T06:06:42.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.872+0000 7f0f20800740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56541eb6d0a0 con 0x56541e833490 2024-09-21T06:06:42.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.873+0000 7f0f137fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56541e83b670 0x56541ec03eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:42.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.873+0000 7f0f137fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56541e83b670 0x56541ec03eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53554/0 (socket says 172.21.15.117:53554) 2024-09-21T06:06:42.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.873+0000 7f0f137fe640 1 -- 172.21.15.117:0/3894306875 learned_addr learned my addr 172.21.15.117:0/3894306875 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:42.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.873+0000 7f0f13fff640 1 --2- 172.21.15.117:0/3894306875 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56541e833490 0x56541ec03970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:42.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.873+0000 7f0f117fa640 1 --2- 172.21.15.117:0/2251966238 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0efc0b6680 0x7f0efc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:42.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.873+0000 7f0f18829640 1 --2- 172.21.15.117:0/3894306875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e85c350 0x56541ec03430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:42.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.873+0000 7f0f137fe640 1 -- 172.21.15.117:0/3894306875 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56541e833490 msgr2=0x56541ec03970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:42.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.873+0000 7f0f137fe640 1 --2- 172.21.15.117:0/3894306875 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56541e833490 0x56541ec03970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:42.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.873+0000 7f0f137fe640 1 -- 172.21.15.117:0/3894306875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e85c350 msgr2=0x56541ec03430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:42.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.873+0000 7f0f137fe640 1 --2- 172.21.15.117:0/3894306875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e85c350 0x56541ec03430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:42.878 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.873+0000 7f0f18829640 1 --2- 172.21.15.117:0/2251966238 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0efc0b6680 0x7f0efc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:42.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.873+0000 7f0f137fe640 1 -- 172.21.15.117:0/3894306875 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56541e8c7930 con 0x56541e83b670 2024-09-21T06:06:42.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.873+0000 7f0f117fa640 1 -- 172.21.15.117:0/2251966238 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7f0f04107b10 con 0x7f0f14103b60 2024-09-21T06:06:42.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.873+0000 7f0f137fe640 1 --2- 172.21.15.117:0/3894306875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56541e83b670 0x56541ec03eb0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f0f08009c20 tx=0x7f0f08011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a232c5ccde6570 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:42.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.873+0000 7f0f18829640 1 --2- 172.21.15.117:0/2251966238 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0efc0b6680 0x7f0efc0b8b90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c005fd0 tx=0x7f0f0c004ab0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:42.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.873+0000 7f0efa7fc640 1 -- 172.21.15.117:0/3894306875 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08004230 con 0x56541e83b670 2024-09-21T06:06:42.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.874+0000 7f0efa7fc640 1 -- 172.21.15.117:0/3894306875 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f080043d0 con 0x56541e83b670 2024-09-21T06:06:42.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.874+0000 7f0efa7fc640 1 -- 172.21.15.117:0/3894306875 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08004d50 con 0x56541e83b670 2024-09-21T06:06:42.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.874+0000 7f0f20800740 1 -- 172.21.15.117:0/3894306875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56541e83b670 msgr2=0x56541ec03eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:42.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.874+0000 7f0f20800740 1 --2- 172.21.15.117:0/3894306875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56541e83b670 0x56541ec03eb0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f0f08009c20 tx=0x7f0f08011810 comp rx=0 tx=0).stop 2024-09-21T06:06:42.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.874+0000 7f0f20800740 1 -- 172.21.15.117:0/3894306875 shutdown_connections 2024-09-21T06:06:42.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.874+0000 7f0f20800740 1 --2- 172.21.15.117:0/3894306875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56541e83b670 0x56541ec03eb0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:42.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.874+0000 7f0f20800740 1 --2- 172.21.15.117:0/3894306875 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56541e833490 0x56541ec03970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:42.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.874+0000 7f0f20800740 1 --2- 172.21.15.117:0/3894306875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e85c350 0x56541ec03430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:42.879 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.874+0000 7f0f20800740 1 -- 172.21.15.117:0/3894306875 >> 172.21.15.117:0/3894306875 conn(0x56541e850740 msgr2=0x56541ec01320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:42.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.874+0000 7f0f20800740 1 -- 172.21.15.117:0/3894306875 shutdown_connections 2024-09-21T06:06:42.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.875+0000 7f0f20800740 1 -- 172.21.15.117:0/3894306875 wait complete. 2024-09-21T06:06:42.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.875+0000 7f0f20800740 1 -- start start 2024-09-21T06:06:42.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.875+0000 7f0f20800740 1 Processor -- start 2024-09-21T06:06:42.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.876+0000 7f0f20800740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56541e833490 0x56541ec0f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:42.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.876+0000 7f0f20800740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e83b670 0x56541ec0f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:42.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.876+0000 7f0f20800740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56541e85c350 0x56541ec1af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:42.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.876+0000 7f0f20800740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56541e9ca890 con 0x56541e833490 2024-09-21T06:06:42.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.876+0000 7f0f20800740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56541eb03d40 con 0x56541e83b670 2024-09-21T06:06:42.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.876+0000 7f0f20800740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56541ec11c10 con 0x56541e85c350 2024-09-21T06:06:42.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.876+0000 7f0f20800740 10 client.0 _ll_register_callbacks cb 0x56541eb67c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:42.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.876+0000 7f0f13fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56541e833490 0x56541ec0f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:42.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.876+0000 7f0f13fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56541e833490 0x56541ec0f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53568/0 (socket says 172.21.15.117:53568) 2024-09-21T06:06:42.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.876+0000 7f0f13fff640 1 -- 172.21.15.117:0/1072662379 learned_addr learned my addr 172.21.15.117:0/1072662379 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:42.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.876+0000 7f0f18829640 1 --2- 172.21.15.117:0/1072662379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56541e85c350 0x56541ec1af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:42.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.876+0000 7f0f137fe640 1 --2- 172.21.15.117:0/1072662379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e83b670 0x56541ec0f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:42.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.876+0000 7f0f13fff640 1 -- 172.21.15.117:0/1072662379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56541e85c350 msgr2=0x56541ec1af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:42.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.876+0000 7f0f13fff640 1 --2- 172.21.15.117:0/1072662379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56541e85c350 0x56541ec1af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:42.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.876+0000 7f0f13fff640 1 -- 172.21.15.117:0/1072662379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e83b670 msgr2=0x56541ec0f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:42.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.876+0000 7f0f13fff640 1 --2- 172.21.15.117:0/1072662379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e83b670 0x56541ec0f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:42.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.876+0000 7f0f13fff640 1 -- 172.21.15.117:0/1072662379 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56541ec1b4f0 con 0x56541e833490 2024-09-21T06:06:42.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.876+0000 7f0f13fff640 1 --2- 172.21.15.117:0/1072662379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56541e833490 0x56541ec0f010 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f0f04006010 tx=0x7f0f04005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd65451e31bab1e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:42.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.876+0000 7f0ef37fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:42.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.876+0000 7f0ef37fe640 1 -- 172.21.15.117:0/1072662379 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f04049070 con 0x56541e833490 2024-09-21T06:06:42.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.876+0000 7f0ef37fe640 1 -- 172.21.15.117:0/1072662379 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7f0f040035f0 con 0x56541e833490 2024-09-21T06:06:42.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.876+0000 7f0f20800740 1 -- 172.21.15.117:0/1072662379 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f080043d0 con 0x56541e833490 2024-09-21T06:06:42.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.877+0000 7f0ef37fe640 1 -- 172.21.15.117:0/1072662379 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0f0410c040 con 0x56541e833490 2024-09-21T06:06:42.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.877+0000 7f0f20800740 10 client.17463 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:42.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.877+0000 7f0f20800740 1 -- 172.21.15.117:0/1072662379 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56541ec1de20 con 0x56541e833490 2024-09-21T06:06:42.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.877+0000 7f0ef37fe640 1 -- 172.21.15.117:0/1072662379 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f080043d0 con 0x56541e833490 2024-09-21T06:06:42.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.877+0000 7f0ef37fe640 1 -- 172.21.15.117:0/1072662379 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0f040cece0 con 0x56541e833490 2024-09-21T06:06:42.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.877+0000 7f0f20800740 10 client.17463 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:42.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.877+0000 7f0f20800740 10 client.17463 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:06:42.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.877+0000 7f0f20800740 20 client.17463 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:42.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.877+0000 7f0f20800740 1 --2- 172.21.15.117:0/1072662379 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56541e81acd0 0x56541ec21210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:42.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.877+0000 7f0f20800740 4 client.17463 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:06:42.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.877+0000 7f0f20800740 1 -- 172.21.15.117:0/1072662379 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56541eb7bdb0 con 0x56541e81acd0 2024-09-21T06:06:42.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.877+0000 7f0f137fe640 1 --2- 172.21.15.117:0/1072662379 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56541e81acd0 0x56541ec21210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:42.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.878+0000 7f0f137fe640 1 --2- 172.21.15.117:0/1072662379 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56541e81acd0 0x56541ec21210 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c56cb0542fa94f47 server_cookie=f36d0ecf2c1e00aa in_seq=0 out_seq=0 2024-09-21T06:06:42.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.878+0000 7f0ef37fe640 10 client.17463 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:42.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.879+0000 7f0ef37fe640 1 -- 172.21.15.117:0/1072662379 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56541eb7bdb0 con 0x56541e81acd0 2024-09-21T06:06:42.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.879+0000 7f0ef37fe640 10 client.17463 handle_command_reply: tid=0 2024-09-21T06:06:42.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.879+0000 7f0f20800740 1 client.17463 shutdown 2024-09-21T06:06:42.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.879+0000 7f0f20800740 1 -- 172.21.15.117:0/1072662379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56541e833490 msgr2=0x56541ec0f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:42.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.879+0000 7f0f20800740 1 --2- 172.21.15.117:0/1072662379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56541e833490 0x56541ec0f010 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f0f04006010 tx=0x7f0f04005c90 comp rx=0 tx=0).stop 2024-09-21T06:06:42.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.879+0000 7f0f20800740 1 -- 172.21.15.117:0/1072662379 shutdown_connections 2024-09-21T06:06:42.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.879+0000 7f0f20800740 1 --2- 172.21.15.117:0/1072662379 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56541e81acd0 0x56541ec21210 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:42.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.879+0000 7f0f20800740 1 --2- 172.21.15.117:0/1072662379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56541e85c350 0x56541ec1af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:42.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.879+0000 7f0f20800740 1 --2- 172.21.15.117:0/1072662379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e83b670 0x56541ec0f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:42.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.879+0000 7f0f20800740 1 --2- 172.21.15.117:0/1072662379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56541e833490 0x56541ec0f010 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:42.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.879+0000 7f0f20800740 1 -- 172.21.15.117:0/1072662379 >> 172.21.15.117:0/1072662379 conn(0x56541e850740 msgr2=0x56541ec06610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:42.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.879+0000 7f0ef37fe640 0 client.17463 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:42.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.879+0000 7f0f20800740 1 -- 172.21.15.117:0/1072662379 shutdown_connections 2024-09-21T06:06:42.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.879+0000 7f0f20800740 1 -- 172.21.15.117:0/1072662379 wait complete. 2024-09-21T06:06:42.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.879+0000 7f0f20800740 20 client.17463 trim_cache size 0 max 16384 2024-09-21T06:06:42.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.897+0000 7f0f20800740 1 Processor -- start 2024-09-21T06:06:42.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.897+0000 7f0f20800740 1 -- start start 2024-09-21T06:06:42.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.897+0000 7f0f20800740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56541e833490 0x56541ec0e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:42.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.897+0000 7f0f20800740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56541e83b670 0x56541ec04b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:42.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.897+0000 7f0f20800740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e85c350 0x56541ec0c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:42.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.897+0000 7f0f20800740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0f141ad3d0 con 0x56541e833490 2024-09-21T06:06:42.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.897+0000 7f0f20800740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0f141aef10 con 0x56541e85c350 2024-09-21T06:06:42.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.897+0000 7f0f20800740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0f141adaa0 con 0x56541e83b670 2024-09-21T06:06:42.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.897+0000 7f0f13fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56541e833490 0x56541ec0e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:42.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.897+0000 7f0f13fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56541e833490 0x56541ec0e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53584/0 (socket says 172.21.15.117:53584) 2024-09-21T06:06:42.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.897+0000 7f0f13fff640 1 -- 172.21.15.117:0/2628455303 learned_addr learned my addr 172.21.15.117:0/2628455303 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:42.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.897+0000 7f0f13fff640 1 -- 172.21.15.117:0/2628455303 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56541e83b670 msgr2=0x56541ec04b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:42.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.897+0000 7f0f137fe640 1 --2- 172.21.15.117:0/2628455303 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56541e83b670 0x56541ec04b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:42.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.897+0000 7f0f13fff640 1 --2- 172.21.15.117:0/2628455303 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56541e83b670 0x56541ec04b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:42.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.897+0000 7f0f13fff640 1 -- 172.21.15.117:0/2628455303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e85c350 msgr2=0x56541ec0c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:42.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.897+0000 7f0f18829640 1 --2- 172.21.15.117:0/2628455303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e85c350 0x56541ec0c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:42.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.897+0000 7f0f13fff640 1 --2- 172.21.15.117:0/2628455303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e85c350 0x56541ec0c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:42.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.897+0000 7f0f13fff640 1 -- 172.21.15.117:0/2628455303 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56541ec0d310 con 0x56541e833490 2024-09-21T06:06:42.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.897+0000 7f0f13fff640 1 --2- 172.21.15.117:0/2628455303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56541e833490 0x56541ec0e570 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f0f040cd6b0 tx=0x7f0f0402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460c6971198c9c1d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:42.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.898+0000 7f0ef9ffb640 1 -- 172.21.15.117:0/2628455303 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f04049070 con 0x56541e833490 2024-09-21T06:06:42.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.898+0000 7f0ef9ffb640 1 -- 172.21.15.117:0/2628455303 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f040d2490 con 0x56541e833490 2024-09-21T06:06:42.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.898+0000 7f0ef9ffb640 1 -- 172.21.15.117:0/2628455303 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f040d73d0 con 0x56541e833490 2024-09-21T06:06:42.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.898+0000 7f0f20800740 1 -- 172.21.15.117:0/2628455303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56541e833490 msgr2=0x56541ec0e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:42.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.898+0000 7f0f20800740 1 --2- 172.21.15.117:0/2628455303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56541e833490 0x56541ec0e570 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f0f040cd6b0 tx=0x7f0f0402a410 comp rx=0 tx=0).stop 2024-09-21T06:06:42.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.899+0000 7f0f20800740 1 -- 172.21.15.117:0/2628455303 shutdown_connections 2024-09-21T06:06:42.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.899+0000 7f0f20800740 1 --2- 172.21.15.117:0/2628455303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e85c350 0x56541ec0c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:42.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.899+0000 7f0f20800740 1 --2- 172.21.15.117:0/2628455303 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56541e83b670 0x56541ec04b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:42.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.899+0000 7f0f20800740 1 --2- 172.21.15.117:0/2628455303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56541e833490 0x56541ec0e570 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:42.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.899+0000 7f0f20800740 1 -- 172.21.15.117:0/2628455303 >> 172.21.15.117:0/2628455303 conn(0x56541e850740 msgr2=0x56541ec04460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:42.904 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.899+0000 7f0f20800740 1 -- 172.21.15.117:0/2628455303 shutdown_connections 2024-09-21T06:06:42.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.899+0000 7f0f20800740 1 -- 172.21.15.117:0/2628455303 wait complete. 2024-09-21T06:06:42.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.899+0000 7f0f20800740 1 -- start start 2024-09-21T06:06:42.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.899+0000 7f0f20800740 1 Processor -- start 2024-09-21T06:06:42.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.900+0000 7f0f20800740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e833490 0x56541ec17880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:42.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.900+0000 7f0f20800740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56541e83b670 0x56541ec13da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:42.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.900+0000 7f0f20800740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56541e85c350 0x56541ec144b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:42.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.900+0000 7f0f20800740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0f141ae840 con 0x56541e83b670 2024-09-21T06:06:42.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.900+0000 7f0f20800740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0f141ae170 con 0x56541e833490 2024-09-21T06:06:42.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.900+0000 7f0f20800740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0f141ab3f0 con 0x56541e85c350 2024-09-21T06:06:42.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.900+0000 7f0f20800740 10 client.0 _ll_register_callbacks cb 0x7f0f04005ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:42.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.900+0000 7f0f137fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56541e83b670 0x56541ec13da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:42.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.900+0000 7f0f137fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56541e83b670 0x56541ec13da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53596/0 (socket says 172.21.15.117:53596) 2024-09-21T06:06:42.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.900+0000 7f0f137fe640 1 -- 172.21.15.117:0/2262838127 learned_addr learned my addr 172.21.15.117:0/2262838127 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:42.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.900+0000 7f0f13fff640 1 --2- 172.21.15.117:0/2262838127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e833490 0x56541ec17880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:42.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.900+0000 7f0f18829640 1 --2- 172.21.15.117:0/2262838127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56541e85c350 0x56541ec144b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:42.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.900+0000 7f0f137fe640 1 -- 172.21.15.117:0/2262838127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56541e85c350 msgr2=0x56541ec144b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:42.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.900+0000 7f0f137fe640 1 --2- 172.21.15.117:0/2262838127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56541e85c350 0x56541ec144b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:42.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.900+0000 7f0f137fe640 1 -- 172.21.15.117:0/2262838127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e833490 msgr2=0x56541ec17880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:42.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.900+0000 7f0f137fe640 1 --2- 172.21.15.117:0/2262838127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e833490 0x56541ec17880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:42.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.900+0000 7f0f137fe640 1 -- 172.21.15.117:0/2262838127 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56541ec8bbb0 con 0x56541e83b670 2024-09-21T06:06:42.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.900+0000 7f0f137fe640 1 --2- 172.21.15.117:0/2262838127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56541e83b670 0x56541ec13da0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f0f08009b40 tx=0x7f0f08009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1465e8a50b820545 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:42.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.900+0000 7f0ef37fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:42.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.900+0000 7f0ef37fe640 1 -- 172.21.15.117:0/2262838127 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f08032060 con 0x56541e83b670 2024-09-21T06:06:42.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.901+0000 7f0ef37fe640 1 -- 172.21.15.117:0/2262838127 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7f0f08004250 con 0x56541e83b670 2024-09-21T06:06:42.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.901+0000 7f0f20800740 1 -- 172.21.15.117:0/2262838127 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f040d2490 con 0x56541e83b670 2024-09-21T06:06:42.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.901+0000 7f0ef37fe640 1 -- 172.21.15.117:0/2262838127 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0f08005c50 con 0x56541e83b670 2024-09-21T06:06:42.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.901+0000 7f0f20800740 10 client.17469 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:42.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.901+0000 7f0f20800740 1 -- 172.21.15.117:0/2262838127 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56541ec8ce00 con 0x56541e83b670 2024-09-21T06:06:42.906 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.901+0000 7f0ef37fe640 1 -- 172.21.15.117:0/2262838127 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f040d2490 con 0x56541e83b670 2024-09-21T06:06:42.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.901+0000 7f0ef37fe640 1 -- 172.21.15.117:0/2262838127 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0f08024040 con 0x56541e83b670 2024-09-21T06:06:42.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.901+0000 7f0f20800740 10 client.17469 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:42.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.901+0000 7f0f20800740 10 client.17469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:06:42.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.901+0000 7f0f20800740 20 client.17469 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:42.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.901+0000 7f0f20800740 1 --2- 172.21.15.117:0/2262838127 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56541e7f0080 0x56541ec8f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:42.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.901+0000 7f0f20800740 4 client.17469 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:06:42.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.901+0000 7f0f20800740 1 -- 172.21.15.117:0/2262838127 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56541eb77860 con 0x56541e7f0080 2024-09-21T06:06:42.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.902+0000 7f0f13fff640 1 --2- 172.21.15.117:0/2262838127 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56541e7f0080 0x56541ec8f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:42.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.902+0000 7f0f13fff640 1 --2- 172.21.15.117:0/2262838127 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56541e7f0080 0x56541ec8f430 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6786c73564e7a867 server_cookie=df3ef1dcbdf9765e in_seq=0 out_seq=0 2024-09-21T06:06:42.907 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.902+0000 7f0ef37fe640 10 client.17469 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:42.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.904+0000 7f0ef37fe640 1 -- 172.21.15.117:0/2262838127 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56541eb77860 con 0x56541e7f0080 2024-09-21T06:06:42.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.904+0000 7f0ef37fe640 10 client.17469 handle_command_reply: tid=0 2024-09-21T06:06:42.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.904+0000 7f0f20800740 1 client.17469 shutdown 2024-09-21T06:06:42.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.904+0000 7f0f20800740 1 -- 172.21.15.117:0/2262838127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56541e83b670 msgr2=0x56541ec13da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:42.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.904+0000 7f0f20800740 1 --2- 172.21.15.117:0/2262838127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56541e83b670 0x56541ec13da0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f0f08009b40 tx=0x7f0f08009f20 comp rx=0 tx=0).stop 2024-09-21T06:06:42.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.904+0000 7f0f20800740 1 -- 172.21.15.117:0/2262838127 shutdown_connections 2024-09-21T06:06:42.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.904+0000 7f0f20800740 1 --2- 172.21.15.117:0/2262838127 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x56541e7f0080 0x56541ec8f430 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:42.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.904+0000 7f0f20800740 1 --2- 172.21.15.117:0/2262838127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56541e83b670 0x56541ec13da0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:42.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.904+0000 7f0f20800740 1 --2- 172.21.15.117:0/2262838127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56541e85c350 0x56541ec144b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:42.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.904+0000 7f0f20800740 1 --2- 172.21.15.117:0/2262838127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56541e833490 0x56541ec17880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:42.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.904+0000 7f0f20800740 1 -- 172.21.15.117:0/2262838127 >> 172.21.15.117:0/2262838127 conn(0x56541e850740 msgr2=0x56541ec100d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:42.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.904+0000 7f0ef37fe640 0 client.17469 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:42.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.904+0000 7f0f20800740 1 -- 172.21.15.117:0/2262838127 shutdown_connections 2024-09-21T06:06:42.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.904+0000 7f0f20800740 1 -- 172.21.15.117:0/2262838127 wait complete. 2024-09-21T06:06:42.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.904+0000 7f0f20800740 20 client.17469 trim_cache size 0 max 16384 2024-09-21T06:06:42.909 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T06:06:42.909 INFO:teuthology.orchestra.run.smithi117.stdout: "return_code": 0, 2024-09-21T06:06:42.909 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_tag": "cf43a1b2-838e-4d28-9ed6-ff0c726fd51a", 2024-09-21T06:06:42.910 INFO:teuthology.orchestra.run.smithi117.stdout: "mode": "asynchronous" 2024-09-21T06:06:42.910 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T06:06:42.910 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.905+0000 7f0f1a28c640 1 -- 172.21.15.117:0/2251966238 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0efc0b6680 msgr2=0x7f0efc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:42.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.905+0000 7f0f1a28c640 1 --2- 172.21.15.117:0/2251966238 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0efc0b6680 0x7f0efc0b8b90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c005fd0 tx=0x7f0f0c004ab0 comp rx=0 tx=0).stop 2024-09-21T06:06:42.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.905+0000 7f0f1a28c640 1 -- 172.21.15.117:0/2251966238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f14103b60 msgr2=0x7f0f1419a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:42.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.905+0000 7f0f1a28c640 1 --2- 172.21.15.117:0/2251966238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f14103b60 0x7f0f1419a480 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f0f04005940 tx=0x7f0f040040c0 comp rx=0 tx=0).stop 2024-09-21T06:06:42.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.905+0000 7f0f1a28c640 1 -- 172.21.15.117:0/2251966238 shutdown_connections 2024-09-21T06:06:42.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.905+0000 7f0f1a28c640 1 --2- 172.21.15.117:0/2251966238 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0efc0b6680 0x7f0efc0b8b90 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:42.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.905+0000 7f0f1a28c640 1 --2- 172.21.15.117:0/2251966238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f14109df0 0x7f0f1419fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:42.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.905+0000 7f0f1a28c640 1 --2- 172.21.15.117:0/2251966238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f141094a0 0x7f0f1419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:42.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.905+0000 7f0f1a28c640 1 --2- 172.21.15.117:0/2251966238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f14103b60 0x7f0f1419a480 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:42.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.905+0000 7f0f1a28c640 1 -- 172.21.15.117:0/2251966238 >> 172.21.15.117:0/2251966238 conn(0x7f0f1407b850 msgr2=0x7f0f14107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:42.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.905+0000 7f0f1a28c640 1 -- 172.21.15.117:0/2251966238 shutdown_connections 2024-09-21T06:06:42.911 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:42.905+0000 7f0f1a28c640 1 -- 172.21.15.117:0/2251966238 wait complete. 2024-09-21T06:06:43.599 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:06:43.846 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:06:44.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:43 smithi192 ceph-mon[34700]: pgmap v614: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T06:06:44.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:43 smithi192 ceph-mon[34700]: scrub queued for path: / 2024-09-21T06:06:44.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:43 smithi192 ceph-mon[34700]: scrub summary: idle+waiting paths [/] 2024-09-21T06:06:44.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:43 smithi192 ceph-mon[34700]: scrub summary: active paths [/] 2024-09-21T06:06:44.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:43 smithi192 ceph-mon[34700]: scrub summary: idle+waiting paths [/] 2024-09-21T06:06:44.268 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:43 smithi117 ceph-mon[30947]: pgmap v614: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T06:06:44.268 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:43 smithi117 ceph-mon[30947]: scrub queued for path: / 2024-09-21T06:06:44.268 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:43 smithi117 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-21T06:06:44.268 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:43 smithi117 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-21T06:06:44.268 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:43 smithi117 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-21T06:06:44.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:43 smithi138 ceph-mon[34391]: pgmap v614: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T06:06:44.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:43 smithi138 ceph-mon[34391]: scrub queued for path: / 2024-09-21T06:06:44.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:43 smithi138 ceph-mon[34391]: scrub summary: idle+waiting paths [/] 2024-09-21T06:06:44.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:43 smithi138 ceph-mon[34391]: scrub summary: active paths [/] 2024-09-21T06:06:44.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:43 smithi138 ceph-mon[34391]: scrub summary: idle+waiting paths [/] 2024-09-21T06:06:44.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.360+0000 7fd8f5d9c640 1 -- 172.21.15.117:0/3880137245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8f0104a00 msgr2=0x7fd8f0104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:44.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.360+0000 7fd8f5d9c640 1 --2- 172.21.15.117:0/3880137245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8f0104a00 0x7fd8f0104e10 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0009900 tx=0x7fd8e002a410 comp rx=0 tx=0).stop 2024-09-21T06:06:44.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.360+0000 7fd8f5d9c640 1 -- 172.21.15.117:0/3880137245 shutdown_connections 2024-09-21T06:06:44.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.360+0000 7fd8f5d9c640 1 --2- 172.21.15.117:0/3880137245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8f010de60 0x7fd8f0110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.360+0000 7fd8f5d9c640 1 --2- 172.21.15.117:0/3880137245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8f0105350 0x7fd8f010d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.360+0000 7fd8f5d9c640 1 --2- 172.21.15.117:0/3880137245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8f0104a00 0x7fd8f0104e10 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.360+0000 7fd8f5d9c640 1 -- 172.21.15.117:0/3880137245 >> 172.21.15.117:0/3880137245 conn(0x7fd8f00796b0 msgr2=0x7fd8f0079ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:44.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.360+0000 7fd8f5d9c640 1 -- 172.21.15.117:0/3880137245 shutdown_connections 2024-09-21T06:06:44.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.360+0000 7fd8f5d9c640 1 -- 172.21.15.117:0/3880137245 wait complete. 2024-09-21T06:06:44.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.361+0000 7fd8f5d9c640 1 Processor -- start 2024-09-21T06:06:44.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.361+0000 7fd8f5d9c640 1 -- start start 2024-09-21T06:06:44.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.361+0000 7fd8f5d9c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8f0104a00 0x7fd8f019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:44.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.361+0000 7fd8f5d9c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8f0105350 0x7fd8f019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:44.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.361+0000 7fd8f5d9c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8f010de60 0x7fd8f019fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:44.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.361+0000 7fd8f5d9c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f0113ae0 con 0x7fd8f010de60 2024-09-21T06:06:44.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.361+0000 7fd8f5d9c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f0113960 con 0x7fd8f0105350 2024-09-21T06:06:44.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.361+0000 7fd8f5d9c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f0113c60 con 0x7fd8f0104a00 2024-09-21T06:06:44.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.361+0000 7fd8effff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8f010de60 0x7fd8f019fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:44.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.361+0000 7fd8effff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8f010de60 0x7fd8f019fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53620/0 (socket says 172.21.15.117:53620) 2024-09-21T06:06:44.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.361+0000 7fd8effff640 1 -- 172.21.15.117:0/2577868356 learned_addr learned my addr 172.21.15.117:0/2577868356 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:44.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.362+0000 7fd8e7fff640 1 --2- 172.21.15.117:0/2577868356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8f0105350 0x7fd8f019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:44.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.362+0000 7fd8ef7fe640 1 --2- 172.21.15.117:0/2577868356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8f0104a00 0x7fd8f019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:44.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.362+0000 7fd8effff640 1 -- 172.21.15.117:0/2577868356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8f0104a00 msgr2=0x7fd8f019a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:44.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.362+0000 7fd8effff640 1 --2- 172.21.15.117:0/2577868356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8f0104a00 0x7fd8f019a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.362+0000 7fd8effff640 1 -- 172.21.15.117:0/2577868356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8f0105350 msgr2=0x7fd8f019aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:44.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.362+0000 7fd8effff640 1 --2- 172.21.15.117:0/2577868356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8f0105350 0x7fd8f019aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.362+0000 7fd8effff640 1 -- 172.21.15.117:0/2577868356 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f01a0480 con 0x7fd8f010de60 2024-09-21T06:06:44.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.362+0000 7fd8ef7fe640 1 --2- 172.21.15.117:0/2577868356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8f0104a00 0x7fd8f019a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:06:44.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.362+0000 7fd8effff640 1 --2- 172.21.15.117:0/2577868356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8f010de60 0x7fd8f019fd50 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fd8e800eaa0 tx=0x7fd8e800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:44.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.362+0000 7fd8ed7fa640 1 -- 172.21.15.117:0/2577868356 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e800cdb0 con 0x7fd8f010de60 2024-09-21T06:06:44.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.362+0000 7fd8ed7fa640 1 -- 172.21.15.117:0/2577868356 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8e8005020 con 0x7fd8f010de60 2024-09-21T06:06:44.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.362+0000 7fd8f5d9c640 1 -- 172.21.15.117:0/2577868356 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8f01a0770 con 0x7fd8f010de60 2024-09-21T06:06:44.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.362+0000 7fd8ed7fa640 1 -- 172.21.15.117:0/2577868356 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e80116d0 con 0x7fd8f010de60 2024-09-21T06:06:44.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.362+0000 7fd8f5d9c640 1 -- 172.21.15.117:0/2577868356 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8f01a6ed0 con 0x7fd8f010de60 2024-09-21T06:06:44.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.365+0000 7fd8ed7fa640 1 -- 172.21.15.117:0/2577868356 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fd8e8011870 con 0x7fd8f010de60 2024-09-21T06:06:44.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.365+0000 7fd8fc310740 1 Processor -- start 2024-09-21T06:06:44.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.365+0000 7fd8fc310740 1 -- start start 2024-09-21T06:06:44.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.365+0000 7fd8fc310740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3a6209070 0x55b3a65b0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:44.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.365+0000 7fd8fc310740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3a61e01b0 0x55b3a65b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:44.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.365+0000 7fd8fc310740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3a61e8390 0x55b3a65b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:44.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.365+0000 7fd8fc310740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b3a6526e50 con 0x55b3a6209070 2024-09-21T06:06:44.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.365+0000 7fd8fc310740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b3a6516280 con 0x55b3a61e01b0 2024-09-21T06:06:44.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.365+0000 7fd8fc310740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b3a651a090 con 0x55b3a61e8390 2024-09-21T06:06:44.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.365+0000 7fd8e7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3a6209070 0x55b3a65b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:44.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.365+0000 7fd8e7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3a6209070 0x55b3a65b0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53630/0 (socket says 172.21.15.117:53630) 2024-09-21T06:06:44.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.365+0000 7fd8e7fff640 1 -- 172.21.15.117:0/1408987285 learned_addr learned my addr 172.21.15.117:0/1408987285 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:44.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.365+0000 7fd8ed7fa640 1 --2- 172.21.15.117:0/2577868356 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd8dc0b6680 0x7fd8dc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:44.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.366+0000 7fd8effff640 1 --2- 172.21.15.117:0/2577868356 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd8dc0b6680 0x7fd8dc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:44.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.366+0000 7fd8e7fff640 1 --2- 172.21.15.117:0/1408987285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3a61e8390 0x55b3a65b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:44.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.366+0000 7fd8ef7fe640 1 --2- 172.21.15.117:0/1408987285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3a61e01b0 0x55b3a65b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:44.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.366+0000 7fd8e7fff640 1 -- 172.21.15.117:0/1408987285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3a61e8390 msgr2=0x55b3a65b10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:44.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.366+0000 7fd8e7fff640 1 --2- 172.21.15.117:0/1408987285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3a61e8390 0x55b3a65b10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.366+0000 7fd8e7fff640 1 -- 172.21.15.117:0/1408987285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3a61e01b0 msgr2=0x55b3a65b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:44.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.366+0000 7fd8e7fff640 1 --2- 172.21.15.117:0/1408987285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3a61e01b0 0x55b3a65b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.366+0000 7fd8ed7fa640 1 -- 172.21.15.117:0/2577868356 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fd8e80e79e0 con 0x7fd8f010de60 2024-09-21T06:06:44.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.366+0000 7fd8e7fff640 1 -- 172.21.15.117:0/1408987285 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3a6274650 con 0x55b3a6209070 2024-09-21T06:06:44.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.366+0000 7fd8e7fff640 1 --2- 172.21.15.117:0/1408987285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3a6209070 0x55b3a65b0650 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fd8d800bfb0 tx=0x7fd8d800b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf8643abb19f915 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:44.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.366+0000 7fd8effff640 1 --2- 172.21.15.117:0/2577868356 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd8dc0b6680 0x7fd8dc0b8b90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8009270 tx=0x7fd8e80a9040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:44.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.366+0000 7fd8e5ffb640 1 -- 172.21.15.117:0/1408987285 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d8019030 con 0x55b3a6209070 2024-09-21T06:06:44.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.366+0000 7fd8e5ffb640 1 -- 172.21.15.117:0/1408987285 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8d8011970 con 0x55b3a6209070 2024-09-21T06:06:44.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.366+0000 7fd8e5ffb640 1 -- 172.21.15.117:0/1408987285 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d801d7a0 con 0x55b3a6209070 2024-09-21T06:06:44.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.367+0000 7fd8fc310740 1 -- 172.21.15.117:0/1408987285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3a6209070 msgr2=0x55b3a65b0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:44.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.367+0000 7fd8fc310740 1 --2- 172.21.15.117:0/1408987285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3a6209070 0x55b3a65b0650 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fd8d800bfb0 tx=0x7fd8d800b630 comp rx=0 tx=0).stop 2024-09-21T06:06:44.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.367+0000 7fd8fc310740 1 -- 172.21.15.117:0/1408987285 shutdown_connections 2024-09-21T06:06:44.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.367+0000 7fd8fc310740 1 --2- 172.21.15.117:0/1408987285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3a61e8390 0x55b3a65b10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.367+0000 7fd8fc310740 1 --2- 172.21.15.117:0/1408987285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3a61e01b0 0x55b3a65b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.367+0000 7fd8fc310740 1 --2- 172.21.15.117:0/1408987285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3a6209070 0x55b3a65b0650 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.367+0000 7fd8fc310740 1 -- 172.21.15.117:0/1408987285 >> 172.21.15.117:0/1408987285 conn(0x55b3a61fd460 msgr2=0x55b3a65ae540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:44.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.367+0000 7fd8fc310740 1 -- 172.21.15.117:0/1408987285 shutdown_connections 2024-09-21T06:06:44.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.367+0000 7fd8fc310740 1 -- 172.21.15.117:0/1408987285 wait complete. 2024-09-21T06:06:44.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.368+0000 7fd8fc310740 1 -- start start 2024-09-21T06:06:44.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.368+0000 7fd8fc310740 1 Processor -- start 2024-09-21T06:06:44.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.369+0000 7fd8fc310740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3a61e01b0 0x55b3a65c0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:44.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.369+0000 7fd8fc310740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3a61e8390 0x55b3a65c05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:44.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.369+0000 7fd8fc310740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3a6209070 0x55b3a65c0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:44.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.369+0000 7fd8fc310740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b3a6377700 con 0x55b3a61e01b0 2024-09-21T06:06:44.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.369+0000 7fd8fc310740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b3a64b1110 con 0x55b3a61e8390 2024-09-21T06:06:44.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.369+0000 7fd8fc310740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b3a65c3130 con 0x55b3a6209070 2024-09-21T06:06:44.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.369+0000 7fd8fc310740 10 client.0 _ll_register_callbacks cb 0x55b3a64ca990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:44.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.369+0000 7fd8e7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3a61e01b0 0x55b3a65c0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:44.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.369+0000 7fd8e7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3a61e01b0 0x55b3a65c0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53634/0 (socket says 172.21.15.117:53634) 2024-09-21T06:06:44.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.369+0000 7fd8e7fff640 1 -- 172.21.15.117:0/1815816612 learned_addr learned my addr 172.21.15.117:0/1815816612 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:44.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.369+0000 7fd8ef7fe640 1 --2- 172.21.15.117:0/1815816612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3a61e8390 0x55b3a65c05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:44.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.369+0000 7fd8e7fff640 1 -- 172.21.15.117:0/1815816612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3a6209070 msgr2=0x55b3a65c0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:44.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.369+0000 7fd8e7fff640 1 --2- 172.21.15.117:0/1815816612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3a6209070 0x55b3a65c0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.369+0000 7fd8e7fff640 1 -- 172.21.15.117:0/1815816612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3a61e8390 msgr2=0x55b3a65c05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:44.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.369+0000 7fd8e7fff640 1 --2- 172.21.15.117:0/1815816612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3a61e8390 0x55b3a65c05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.369+0000 7fd8e7fff640 1 -- 172.21.15.117:0/1815816612 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3a64dec40 con 0x55b3a61e01b0 2024-09-21T06:06:44.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.369+0000 7fd8e7fff640 1 --2- 172.21.15.117:0/1815816612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3a61e01b0 0x55b3a65c0090 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fd8d800b600 tx=0x7fd8d80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e466bc6a98311c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:44.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.369+0000 7fd8ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:44.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.369+0000 7fd8ceffd640 1 -- 172.21.15.117:0/1815816612 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d8019040 con 0x55b3a61e01b0 2024-09-21T06:06:44.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.370+0000 7fd8ceffd640 1 -- 172.21.15.117:0/1815816612 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fd8d8011da0 con 0x55b3a61e01b0 2024-09-21T06:06:44.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.370+0000 7fd8fc310740 1 -- 172.21.15.117:0/1815816612 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8d8011970 con 0x55b3a61e01b0 2024-09-21T06:06:44.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.370+0000 7fd8ceffd640 1 -- 172.21.15.117:0/1815816612 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd8d802d030 con 0x55b3a61e01b0 2024-09-21T06:06:44.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.370+0000 7fd8fc310740 10 client.17481 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:44.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.370+0000 7fd8fc310740 1 -- 172.21.15.117:0/1815816612 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b3a65cedc0 con 0x55b3a61e01b0 2024-09-21T06:06:44.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.370+0000 7fd8ceffd640 1 -- 172.21.15.117:0/1815816612 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8d8011970 con 0x55b3a61e01b0 2024-09-21T06:06:44.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.370+0000 7fd8ceffd640 1 -- 172.21.15.117:0/1815816612 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd8d8004af0 con 0x55b3a61e01b0 2024-09-21T06:06:44.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.370+0000 7fd8fc310740 10 client.17481 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:44.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.370+0000 7fd8fc310740 10 client.17481 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:06:44.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.370+0000 7fd8fc310740 20 client.17481 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:44.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.370+0000 7fd8fc310740 1 --2- 172.21.15.117:0/1815816612 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b3a61c79f0 0x55b3a65d2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:44.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.370+0000 7fd8fc310740 4 client.17481 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:06:44.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.370+0000 7fd8fc310740 1 -- 172.21.15.117:0/1815816612 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3a6528da0 con 0x55b3a61c79f0 2024-09-21T06:06:44.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.371+0000 7fd8ef7fe640 1 --2- 172.21.15.117:0/1815816612 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b3a61c79f0 0x55b3a65d2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:44.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.371+0000 7fd8ceffd640 10 client.17481 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:44.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.371+0000 7fd8ef7fe640 1 --2- 172.21.15.117:0/1815816612 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b3a61c79f0 0x55b3a65d2180 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9487af5cddf7abc8 server_cookie=981c6f6d053f6414 in_seq=0 out_seq=0 2024-09-21T06:06:44.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.372+0000 7fd8ceffd640 1 -- 172.21.15.117:0/1815816612 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b3a6528da0 con 0x55b3a61c79f0 2024-09-21T06:06:44.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.372+0000 7fd8ceffd640 10 client.17481 handle_command_reply: tid=0 2024-09-21T06:06:44.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.372+0000 7fd8fc310740 1 client.17481 shutdown 2024-09-21T06:06:44.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.372+0000 7fd8fc310740 1 -- 172.21.15.117:0/1815816612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3a61e01b0 msgr2=0x55b3a65c0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:44.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.372+0000 7fd8fc310740 1 --2- 172.21.15.117:0/1815816612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3a61e01b0 0x55b3a65c0090 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fd8d800b600 tx=0x7fd8d80117b0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.372+0000 7fd8fc310740 1 -- 172.21.15.117:0/1815816612 shutdown_connections 2024-09-21T06:06:44.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.372+0000 7fd8fc310740 1 --2- 172.21.15.117:0/1815816612 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b3a61c79f0 0x55b3a65d2180 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.372+0000 7fd8fc310740 1 --2- 172.21.15.117:0/1815816612 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3a6209070 0x55b3a65c0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.372+0000 7fd8ceffd640 0 client.17481 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:44.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.372+0000 7fd8fc310740 1 --2- 172.21.15.117:0/1815816612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3a61e8390 0x55b3a65c05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.372+0000 7fd8fc310740 1 --2- 172.21.15.117:0/1815816612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3a61e01b0 0x55b3a65c0090 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.372+0000 7fd8fc310740 1 -- 172.21.15.117:0/1815816612 >> 172.21.15.117:0/1815816612 conn(0x55b3a61fd460 msgr2=0x55b3a65acb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:44.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.372+0000 7fd8fc310740 1 -- 172.21.15.117:0/1815816612 shutdown_connections 2024-09-21T06:06:44.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.372+0000 7fd8fc310740 1 -- 172.21.15.117:0/1815816612 wait complete. 2024-09-21T06:06:44.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.373+0000 7fd8fc310740 20 client.17481 trim_cache size 0 max 16384 2024-09-21T06:06:44.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.405+0000 7fd8fc310740 1 Processor -- start 2024-09-21T06:06:44.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.405+0000 7fd8fc310740 1 -- start start 2024-09-21T06:06:44.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.405+0000 7fd8fc310740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3a61c79f0 0x55b3a65c1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:44.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.405+0000 7fd8fc310740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3a61e01b0 0x55b3a65cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:44.408 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.405+0000 7fd8fc310740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3a61e8390 0x55b3a65adb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:44.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.405+0000 7fd8fc310740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f01ad4b0 con 0x55b3a61e01b0 2024-09-21T06:06:44.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.405+0000 7fd8fc310740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f01aeff0 con 0x55b3a61c79f0 2024-09-21T06:06:44.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.405+0000 7fd8fc310740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f01adb80 con 0x55b3a61e8390 2024-09-21T06:06:44.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.405+0000 7fd8ef7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3a61e01b0 0x55b3a65cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:44.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.406+0000 7fd8ef7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3a61e01b0 0x55b3a65cbad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53640/0 (socket says 172.21.15.117:53640) 2024-09-21T06:06:44.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.406+0000 7fd8ef7fe640 1 -- 172.21.15.117:0/1339432170 learned_addr learned my addr 172.21.15.117:0/1339432170 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:44.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.406+0000 7fd8e7fff640 1 --2- 172.21.15.117:0/1339432170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3a61c79f0 0x55b3a65c1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:44.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.406+0000 7fd8e7fff640 1 --2- 172.21.15.117:0/1339432170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3a61e8390 0x55b3a65adb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:44.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.406+0000 7fd8ef7fe640 1 -- 172.21.15.117:0/1339432170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3a61e8390 msgr2=0x55b3a65adb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:44.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.406+0000 7fd8ef7fe640 1 --2- 172.21.15.117:0/1339432170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3a61e8390 0x55b3a65adb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.406+0000 7fd8ef7fe640 1 -- 172.21.15.117:0/1339432170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3a61c79f0 msgr2=0x55b3a65c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:44.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.406+0000 7fd8ef7fe640 1 --2- 172.21.15.117:0/1339432170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3a61c79f0 0x55b3a65c1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.406+0000 7fd8ef7fe640 1 -- 172.21.15.117:0/1339432170 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3a65c2670 con 0x55b3a61e01b0 2024-09-21T06:06:44.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.406+0000 7fd8e7fff640 1 --2- 172.21.15.117:0/1339432170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3a61c79f0 0x55b3a65c1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:06:44.409 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.406+0000 7fd8ef7fe640 1 --2- 172.21.15.117:0/1339432170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3a61e01b0 0x55b3a65cbad0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fd8f006c960 tx=0x7fd8e0005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e021d2f598be5381 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:44.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.406+0000 7fd8e57fa640 1 -- 172.21.15.117:0/1339432170 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e0035040 con 0x55b3a61e01b0 2024-09-21T06:06:44.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.406+0000 7fd8e57fa640 1 -- 172.21.15.117:0/1339432170 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8e0048070 con 0x55b3a61e01b0 2024-09-21T06:06:44.410 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.406+0000 7fd8e57fa640 1 -- 172.21.15.117:0/1339432170 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e0004bc0 con 0x55b3a61e01b0 2024-09-21T06:06:44.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.407+0000 7fd8fc310740 1 -- 172.21.15.117:0/1339432170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3a61e01b0 msgr2=0x55b3a65cbad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:44.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.407+0000 7fd8fc310740 1 --2- 172.21.15.117:0/1339432170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3a61e01b0 0x55b3a65cbad0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fd8f006c960 tx=0x7fd8e0005a20 comp rx=0 tx=0).stop 2024-09-21T06:06:44.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.407+0000 7fd8fc310740 1 -- 172.21.15.117:0/1339432170 shutdown_connections 2024-09-21T06:06:44.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.407+0000 7fd8fc310740 1 --2- 172.21.15.117:0/1339432170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3a61e01b0 0x55b3a65cbad0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.407+0000 7fd8fc310740 1 --2- 172.21.15.117:0/1339432170 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3a61e8390 0x55b3a65adb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.407+0000 7fd8fc310740 1 --2- 172.21.15.117:0/1339432170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3a61c79f0 0x55b3a65c1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.407+0000 7fd8fc310740 1 -- 172.21.15.117:0/1339432170 >> 172.21.15.117:0/1339432170 conn(0x55b3a61fd460 msgr2=0x55b3a65cb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:44.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.407+0000 7fd8fc310740 1 -- 172.21.15.117:0/1339432170 shutdown_connections 2024-09-21T06:06:44.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.407+0000 7fd8fc310740 1 -- 172.21.15.117:0/1339432170 wait complete. 2024-09-21T06:06:44.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.408+0000 7fd8fc310740 1 -- start start 2024-09-21T06:06:44.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.408+0000 7fd8fc310740 1 Processor -- start 2024-09-21T06:06:44.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.408+0000 7fd8fc310740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3a61c79f0 0x55b3a65d3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:44.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.408+0000 7fd8fc310740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3a61e01b0 0x55b3a65d37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:44.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.408+0000 7fd8fc310740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3a61e8390 0x55b3a65d3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:44.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.408+0000 7fd8fc310740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f01ae920 con 0x55b3a61c79f0 2024-09-21T06:06:44.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.408+0000 7fd8fc310740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f01ae250 con 0x55b3a61e8390 2024-09-21T06:06:44.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.408+0000 7fd8fc310740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f01ab4b0 con 0x55b3a61e01b0 2024-09-21T06:06:44.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.408+0000 7fd8fc310740 10 client.0 _ll_register_callbacks cb 0x7fd8d8013400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:44.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.408+0000 7fd8e7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3a61c79f0 0x55b3a65d3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:44.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.408+0000 7fd8e7fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3a61c79f0 0x55b3a65d3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53652/0 (socket says 172.21.15.117:53652) 2024-09-21T06:06:44.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.408+0000 7fd8e7fff640 1 -- 172.21.15.117:0/2398700303 learned_addr learned my addr 172.21.15.117:0/2398700303 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:44.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.409+0000 7fd8ef7fe640 1 --2- 172.21.15.117:0/2398700303 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3a61e01b0 0x55b3a65d37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:44.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.409+0000 7fd8e7fff640 1 --2- 172.21.15.117:0/2398700303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3a61e8390 0x55b3a65d3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:44.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.409+0000 7fd8e7fff640 1 -- 172.21.15.117:0/2398700303 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3a61e01b0 msgr2=0x55b3a65d37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:44.415 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.409+0000 7fd8e7fff640 1 --2- 172.21.15.117:0/2398700303 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3a61e01b0 0x55b3a65d37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.409+0000 7fd8e7fff640 1 -- 172.21.15.117:0/2398700303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3a61e8390 msgr2=0x55b3a65d3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:44.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.409+0000 7fd8e7fff640 1 --2- 172.21.15.117:0/2398700303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3a61e8390 0x55b3a65d3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.409+0000 7fd8e7fff640 1 -- 172.21.15.117:0/2398700303 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3a663b9d0 con 0x55b3a61c79f0 2024-09-21T06:06:44.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.409+0000 7fd8e7fff640 1 --2- 172.21.15.117:0/2398700303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3a61c79f0 0x55b3a65d3280 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fd8d80134f0 tx=0x7fd8d800b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a38a4a470c8d75 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:44.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.409+0000 7fd8ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:44.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.409+0000 7fd8ceffd640 1 -- 172.21.15.117:0/2398700303 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d8019040 con 0x55b3a61c79f0 2024-09-21T06:06:44.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.409+0000 7fd8ceffd640 1 -- 172.21.15.117:0/2398700303 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fd8d8004680 con 0x55b3a61c79f0 2024-09-21T06:06:44.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.409+0000 7fd8fc310740 1 -- 172.21.15.117:0/2398700303 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8e0048070 con 0x55b3a61c79f0 2024-09-21T06:06:44.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.410+0000 7fd8ceffd640 1 -- 172.21.15.117:0/2398700303 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd8d801d3c0 con 0x55b3a61c79f0 2024-09-21T06:06:44.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.410+0000 7fd8fc310740 10 client.17487 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:44.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.410+0000 7fd8fc310740 1 -- 172.21.15.117:0/2398700303 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b3a663cbd0 con 0x55b3a61c79f0 2024-09-21T06:06:44.416 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.410+0000 7fd8ceffd640 1 -- 172.21.15.117:0/2398700303 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8e0048070 con 0x55b3a61c79f0 2024-09-21T06:06:44.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.410+0000 7fd8ceffd640 1 -- 172.21.15.117:0/2398700303 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd8d801d5e0 con 0x55b3a61c79f0 2024-09-21T06:06:44.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.410+0000 7fd8fc310740 10 client.17487 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:44.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.410+0000 7fd8fc310740 10 client.17487 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:06:44.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.410+0000 7fd8fc310740 20 client.17487 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:44.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.410+0000 7fd8fc310740 1 --2- 172.21.15.117:0/2398700303 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b3a6209070 0x55b3a663f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:44.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.410+0000 7fd8fc310740 4 client.17487 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:06:44.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.410+0000 7fd8fc310740 1 -- 172.21.15.117:0/2398700303 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b3a6524850 con 0x55b3a6209070 2024-09-21T06:06:44.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.410+0000 7fd8ef7fe640 1 --2- 172.21.15.117:0/2398700303 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b3a6209070 0x55b3a663f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:44.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.411+0000 7fd8ef7fe640 1 --2- 172.21.15.117:0/2398700303 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b3a6209070 0x55b3a663f2d0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f0bf8eb59d73a8b server_cookie=d75525fa510f9747 in_seq=0 out_seq=0 2024-09-21T06:06:44.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.411+0000 7fd8ceffd640 10 client.17487 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:44.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.411+0000 7fd8ceffd640 1 -- 172.21.15.117:0/2398700303 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3126 (crc 0 0 0) 0x55b3a6524850 con 0x55b3a6209070 2024-09-21T06:06:44.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.411+0000 7fd8ceffd640 10 client.17487 handle_command_reply: tid=0 2024-09-21T06:06:44.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.411+0000 7fd8fc310740 1 client.17487 shutdown 2024-09-21T06:06:44.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.411+0000 7fd8fc310740 1 -- 172.21.15.117:0/2398700303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3a61c79f0 msgr2=0x55b3a65d3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:44.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.411+0000 7fd8fc310740 1 --2- 172.21.15.117:0/2398700303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3a61c79f0 0x55b3a65d3280 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fd8d80134f0 tx=0x7fd8d800b7a0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.418 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.411+0000 7fd8fc310740 1 -- 172.21.15.117:0/2398700303 shutdown_connections 2024-09-21T06:06:44.418 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.411+0000 7fd8fc310740 1 --2- 172.21.15.117:0/2398700303 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55b3a6209070 0x55b3a663f2d0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.418 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.411+0000 7fd8fc310740 1 --2- 172.21.15.117:0/2398700303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3a61e8390 0x55b3a65d3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.418 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.411+0000 7fd8ceffd640 0 client.17487 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:44.418 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.411+0000 7fd8fc310740 1 --2- 172.21.15.117:0/2398700303 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b3a61e01b0 0x55b3a65d37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.418 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.411+0000 7fd8fc310740 1 --2- 172.21.15.117:0/2398700303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b3a61c79f0 0x55b3a65d3280 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.418 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.411+0000 7fd8fc310740 1 -- 172.21.15.117:0/2398700303 >> 172.21.15.117:0/2398700303 conn(0x55b3a61fd460 msgr2=0x55b3a65c1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:44.418 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.412+0000 7fd8fc310740 1 -- 172.21.15.117:0/2398700303 shutdown_connections 2024-09-21T06:06:44.418 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.412+0000 7fd8fc310740 1 -- 172.21.15.117:0/2398700303 wait complete. 2024-09-21T06:06:44.418 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.412+0000 7fd8fc310740 20 client.17487 trim_cache size 0 max 16384 2024-09-21T06:06:44.418 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T06:06:44.419 INFO:teuthology.orchestra.run.smithi117.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:06:44.419 INFO:teuthology.orchestra.run.smithi117.stdout: "scrubs": { 2024-09-21T06:06:44.419 INFO:teuthology.orchestra.run.smithi117.stdout: "cf43a1b2-838e-4d28-9ed6-ff0c726fd51a": { 2024-09-21T06:06:44.419 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:06:44.419 INFO:teuthology.orchestra.run.smithi117.stdout: "tag": "cf43a1b2-838e-4d28-9ed6-ff0c726fd51a", 2024-09-21T06:06:44.419 INFO:teuthology.orchestra.run.smithi117.stdout: "options": "recursive,force" 2024-09-21T06:06:44.419 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:06:44.419 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:06:44.419 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_stats": { 2024-09-21T06:06:44.419 INFO:teuthology.orchestra.run.smithi117.stdout: "26e3d90e-072a-4a4e-9661-4d004051865b": { 2024-09-21T06:06:44.419 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:06:29.219142+0000", 2024-09-21T06:06:44.419 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:06:44.419 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:06:44.420 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:06:44.420 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:06:44.420 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:06:44.420 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:06:44.420 INFO:teuthology.orchestra.run.smithi117.stdout: "cf43a1b2-838e-4d28-9ed6-ff0c726fd51a": { 2024-09-21T06:06:44.420 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:06:42.901863+0000", 2024-09-21T06:06:44.420 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:06:44.420 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:06:44.420 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:06:44.420 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:06:44.420 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:06:44.420 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:06:44.420 INFO:teuthology.orchestra.run.smithi117.stdout: "39eeca8a-6b4f-465f-af34-b54f77320d70": { 2024-09-21T06:06:44.420 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:05:41.497141+0000", 2024-09-21T06:06:44.421 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:06:44.421 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:06:44.421 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:06:44.421 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:06:44.421 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:06:44.421 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:06:44.421 INFO:teuthology.orchestra.run.smithi117.stdout: "29ff9b7c-49d9-4910-85be-bace7e940cd5": { 2024-09-21T06:06:44.421 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:03:03.775527+0000", 2024-09-21T06:06:44.421 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:06:44.421 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:06:44.421 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:06:44.421 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:06:44.421 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:06:44.422 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:06:44.422 INFO:teuthology.orchestra.run.smithi117.stdout: "0902e33a-ce33-45fb-92d8-2e6fcb8ec8a0": { 2024-09-21T06:06:44.422 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:04:11.365073+0000", 2024-09-21T06:06:44.422 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:06:44.422 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:06:44.422 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:06:44.422 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:06:44.422 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:06:44.422 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:06:44.422 INFO:teuthology.orchestra.run.smithi117.stdout: "d543b0ae-b710-4153-99da-fe5714839c1a": { 2024-09-21T06:06:44.422 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:01:20.574754+0000", 2024-09-21T06:06:44.422 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:06:44.423 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:06:44.423 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:06:44.423 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:06:44.423 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:06:44.423 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:06:44.423 INFO:teuthology.orchestra.run.smithi117.stdout: "ad80d9f1-6aff-46c0-91ab-2c1ab0681039": { 2024-09-21T06:06:44.423 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:59:31.949108+0000", 2024-09-21T06:06:44.423 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:06:44.423 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:06:44.423 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:06:44.423 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:06:44.423 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:06:44.423 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:06:44.424 INFO:teuthology.orchestra.run.smithi117.stdout: "fb2f217d-463b-4c8f-ab19-920b04bb7306": { 2024-09-21T06:06:44.424 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:04:56.308107+0000", 2024-09-21T06:06:44.424 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:06:44.424 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:06:44.424 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:06:44.424 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:06:44.424 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:06:44.424 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:06:44.424 INFO:teuthology.orchestra.run.smithi117.stdout: "8b3fe95a-b76b-4e94-afd0-72fe35e2dd92": { 2024-09-21T06:06:44.424 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:58:01.246076+0000", 2024-09-21T06:06:44.424 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:06:44.424 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:06:44.424 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:06:44.424 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:06:44.425 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:06:44.425 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:06:44.425 INFO:teuthology.orchestra.run.smithi117.stdout: "b30bfff0-0f6b-4e30-8467-ff7db421eba6": { 2024-09-21T06:06:44.425 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:57:14.939082+0000", 2024-09-21T06:06:44.425 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:06:44.425 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:06:44.425 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:06:44.425 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:06:44.425 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:06:44.425 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:06:44.425 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:06:44.425 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T06:06:44.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.412+0000 7fd8f5d9c640 1 -- 172.21.15.117:0/2577868356 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd8dc0b6680 msgr2=0x7fd8dc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:44.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.412+0000 7fd8f5d9c640 1 --2- 172.21.15.117:0/2577868356 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd8dc0b6680 0x7fd8dc0b8b90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8009270 tx=0x7fd8e80a9040 comp rx=0 tx=0).stop 2024-09-21T06:06:44.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.412+0000 7fd8f5d9c640 1 -- 172.21.15.117:0/2577868356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8f010de60 msgr2=0x7fd8f019fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:44.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.412+0000 7fd8f5d9c640 1 --2- 172.21.15.117:0/2577868356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8f010de60 0x7fd8f019fd50 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fd8e800eaa0 tx=0x7fd8e800ef60 comp rx=0 tx=0).stop 2024-09-21T06:06:44.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.412+0000 7fd8f5d9c640 1 -- 172.21.15.117:0/2577868356 shutdown_connections 2024-09-21T06:06:44.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.412+0000 7fd8f5d9c640 1 --2- 172.21.15.117:0/2577868356 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fd8dc0b6680 0x7fd8dc0b8b90 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.426 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.412+0000 7fd8f5d9c640 1 --2- 172.21.15.117:0/2577868356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8f010de60 0x7fd8f019fd50 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.413+0000 7fd8f5d9c640 1 --2- 172.21.15.117:0/2577868356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8f0105350 0x7fd8f019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.413+0000 7fd8f5d9c640 1 --2- 172.21.15.117:0/2577868356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8f0104a00 0x7fd8f019a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:44.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.413+0000 7fd8f5d9c640 1 -- 172.21.15.117:0/2577868356 >> 172.21.15.117:0/2577868356 conn(0x7fd8f00796b0 msgr2=0x7fd8f0107980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:44.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.413+0000 7fd8f5d9c640 1 -- 172.21.15.117:0/2577868356 shutdown_connections 2024-09-21T06:06:44.427 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:06:44.413+0000 7fd8f5d9c640 1 -- 172.21.15.117:0/2577868356 wait complete. 2024-09-21T06:06:45.169 INFO:tasks.cephfs.filesystem:scrub status for tag:cf43a1b2-838e-4d28-9ed6-ff0c726fd51a - {'path': '/', 'tag': 'cf43a1b2-838e-4d28-9ed6-ff0c726fd51a', 'options': 'recursive,force'} 2024-09-21T06:06:45.956 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.952+0000 7fb2a1b52640 1 -- 172.21.15.117:0/489238034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb29c1030d0 msgr2=0x7fb29c1034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:45.956 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.952+0000 7fb2a1b52640 1 --2- 172.21.15.117:0/489238034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb29c1030d0 0x7fb29c1034e0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fb28c009900 tx=0x7fb28c02a410 comp rx=0 tx=0).stop 2024-09-21T06:06:45.956 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.952+0000 7fb2a1b52640 1 -- 172.21.15.117:0/489238034 shutdown_connections 2024-09-21T06:06:45.956 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.952+0000 7fb2a1b52640 1 --2- 172.21.15.117:0/489238034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb29c108c80 0x7fb29c10f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.956 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.952+0000 7fb2a1b52640 1 --2- 172.21.15.117:0/489238034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb29c108330 0x7fb29c108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.956 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.952+0000 7fb2a1b52640 1 --2- 172.21.15.117:0/489238034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb29c1030d0 0x7fb29c1034e0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.957 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.952+0000 7fb2a1b52640 1 -- 172.21.15.117:0/489238034 >> 172.21.15.117:0/489238034 conn(0x7fb29c077ad0 msgr2=0x7fb29c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:45.957 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.952+0000 7fb2a1b52640 1 -- 172.21.15.117:0/489238034 shutdown_connections 2024-09-21T06:06:45.957 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.952+0000 7fb2a1b52640 1 -- 172.21.15.117:0/489238034 wait complete. 2024-09-21T06:06:45.957 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.953+0000 7fb2a1b52640 1 Processor -- start 2024-09-21T06:06:45.957 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.953+0000 7fb2a1b52640 1 -- start start 2024-09-21T06:06:45.957 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.953+0000 7fb2a1b52640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb29c1030d0 0x7fb29c19b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:45.957 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.953+0000 7fb2a1b52640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb29c108330 0x7fb29c19b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:45.957 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.953+0000 7fb2a1b52640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb29c108c80 0x7fb29c19bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:45.957 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.953+0000 7fb2a1b52640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c112990 con 0x7fb29c108330 2024-09-21T06:06:45.957 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.953+0000 7fb2a1b52640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c112810 con 0x7fb29c108c80 2024-09-21T06:06:45.957 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.953+0000 7fb2a1b52640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c112b10 con 0x7fb29c1030d0 2024-09-21T06:06:45.957 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.953+0000 7fb293fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb29c108330 0x7fb29c19b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:45.957 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.953+0000 7fb293fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb29c108330 0x7fb29c19b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57612/0 (socket says 172.21.15.117:57612) 2024-09-21T06:06:45.957 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.953+0000 7fb293fff640 1 -- 172.21.15.117:0/3816084227 learned_addr learned my addr 172.21.15.117:0/3816084227 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:45.958 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.953+0000 7fb293fff640 1 -- 172.21.15.117:0/3816084227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb29c1030d0 msgr2=0x7fb29c19b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:45.958 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.953+0000 7fb293fff640 1 --2- 172.21.15.117:0/3816084227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb29c1030d0 0x7fb29c19b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.958 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.953+0000 7fb29bfff640 1 --2- 172.21.15.117:0/3816084227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb29c108c80 0x7fb29c19bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:45.958 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.953+0000 7fb293fff640 1 -- 172.21.15.117:0/3816084227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb29c108c80 msgr2=0x7fb29c19bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:45.958 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.953+0000 7fb293fff640 1 --2- 172.21.15.117:0/3816084227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb29c108c80 0x7fb29c19bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.958 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.953+0000 7fb293fff640 1 -- 172.21.15.117:0/3816084227 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb29c1a64e0 con 0x7fb29c108330 2024-09-21T06:06:45.958 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.954+0000 7fb293fff640 1 --2- 172.21.15.117:0/3816084227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb29c108330 0x7fb29c19b7a0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fb28400c970 tx=0x7fb28400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:45.958 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.954+0000 7fb2997fa640 1 -- 172.21.15.117:0/3816084227 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb284004220 con 0x7fb29c108330 2024-09-21T06:06:45.958 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.954+0000 7fb2997fa640 1 -- 172.21.15.117:0/3816084227 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2840043c0 con 0x7fb29c108330 2024-09-21T06:06:45.958 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.954+0000 7fb2a1b52640 1 -- 172.21.15.117:0/3816084227 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb29c1a67d0 con 0x7fb29c108330 2024-09-21T06:06:45.958 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.954+0000 7fb2997fa640 1 -- 172.21.15.117:0/3816084227 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb284004a00 con 0x7fb29c108330 2024-09-21T06:06:45.958 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.954+0000 7fb2a1b52640 1 -- 172.21.15.117:0/3816084227 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb29c1a6cd0 con 0x7fb29c108330 2024-09-21T06:06:45.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.956+0000 7fb2a8ab7740 1 Processor -- start 2024-09-21T06:06:45.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.956+0000 7fb2a8ab7740 1 -- start start 2024-09-21T06:06:45.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.957+0000 7fb2a8ab7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ec699c1b0 0x557ec6d419b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:45.964 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.957+0000 7fb2a8ab7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ec69732f0 0x557ec6d41ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:45.964 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.957+0000 7fb2a8ab7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ec697b4d0 0x557ec6d42430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:45.964 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.957+0000 7fb2a8ab7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557ec6cb66f0 con 0x557ec69732f0 2024-09-21T06:06:45.964 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.957+0000 7fb2a8ab7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557ec6ca5b20 con 0x557ec697b4d0 2024-09-21T06:06:45.964 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.957+0000 7fb2a8ab7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557ec6ca9930 con 0x557ec699c1b0 2024-09-21T06:06:45.964 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.957+0000 7fb2997fa640 1 -- 172.21.15.117:0/3816084227 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fb284004ba0 con 0x7fb29c108330 2024-09-21T06:06:45.964 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.957+0000 7fb29b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ec69732f0 0x557ec6d41ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:45.964 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.957+0000 7fb29b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ec69732f0 0x557ec6d41ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57624/0 (socket says 172.21.15.117:57624) 2024-09-21T06:06:45.964 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.957+0000 7fb29b7fe640 1 -- 172.21.15.117:0/3316717175 learned_addr learned my addr 172.21.15.117:0/3316717175 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.957+0000 7fb29bfff640 1 --2- 172.21.15.117:0/3316717175 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ec699c1b0 0x557ec6d419b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.957+0000 7fb293fff640 1 --2- 172.21.15.117:0/3316717175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ec697b4d0 0x557ec6d42430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.957+0000 7fb29b7fe640 1 -- 172.21.15.117:0/3316717175 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ec699c1b0 msgr2=0x557ec6d419b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.957+0000 7fb29b7fe640 1 --2- 172.21.15.117:0/3316717175 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ec699c1b0 0x557ec6d419b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.957+0000 7fb29b7fe640 1 -- 172.21.15.117:0/3316717175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ec697b4d0 msgr2=0x557ec6d42430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.957+0000 7fb29b7fe640 1 --2- 172.21.15.117:0/3316717175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ec697b4d0 0x557ec6d42430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.957+0000 7fb29b7fe640 1 -- 172.21.15.117:0/3316717175 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ec6c6b530 con 0x557ec69732f0 2024-09-21T06:06:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.957+0000 7fb293fff640 1 --2- 172.21.15.117:0/3316717175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ec697b4d0 0x557ec6d42430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:06:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.957+0000 7fb29b7fe640 1 --2- 172.21.15.117:0/3316717175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ec69732f0 0x557ec6d41ef0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fb28c02a930 tx=0x7fb28c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=793848d6578c9617 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.958+0000 7fb291ffb640 1 -- 172.21.15.117:0/3316717175 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c03e030 con 0x557ec69732f0 2024-09-21T06:06:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.958+0000 7fb2997fa640 1 --2- 172.21.15.117:0/3816084227 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb2880b6680 0x7fb2880b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.958+0000 7fb291ffb640 1 -- 172.21.15.117:0/3316717175 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb28c0043f0 con 0x557ec69732f0 2024-09-21T06:06:45.965 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.958+0000 7fb291ffb640 1 -- 172.21.15.117:0/3316717175 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c035040 con 0x557ec69732f0 2024-09-21T06:06:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.958+0000 7fb29bfff640 1 --2- 172.21.15.117:0/3816084227 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb2880b6680 0x7fb2880b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.958+0000 7fb2997fa640 1 -- 172.21.15.117:0/3816084227 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fb2840e7530 con 0x7fb29c108330 2024-09-21T06:06:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.958+0000 7fb2a8ab7740 1 -- 172.21.15.117:0/3316717175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ec69732f0 msgr2=0x557ec6d41ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.958+0000 7fb29bfff640 1 --2- 172.21.15.117:0/3816084227 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb2880b6680 0x7fb2880b8b90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb294008e80 tx=0x7fb2940085d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.958+0000 7fb2a8ab7740 1 --2- 172.21.15.117:0/3316717175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ec69732f0 0x557ec6d41ef0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fb28c02a930 tx=0x7fb28c000bc0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.958+0000 7fb2a8ab7740 1 -- 172.21.15.117:0/3316717175 shutdown_connections 2024-09-21T06:06:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.958+0000 7fb2a8ab7740 1 --2- 172.21.15.117:0/3316717175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ec69732f0 0x557ec6d41ef0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.958+0000 7fb2a8ab7740 1 --2- 172.21.15.117:0/3316717175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ec697b4d0 0x557ec6d42430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.958+0000 7fb2a8ab7740 1 --2- 172.21.15.117:0/3316717175 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ec699c1b0 0x557ec6d419b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.958+0000 7fb2a8ab7740 1 -- 172.21.15.117:0/3316717175 >> 172.21.15.117:0/3316717175 conn(0x557ec6990300 msgr2=0x557ec6d3f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.958+0000 7fb2a8ab7740 1 -- 172.21.15.117:0/3316717175 shutdown_connections 2024-09-21T06:06:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.959+0000 7fb2a8ab7740 1 -- 172.21.15.117:0/3316717175 wait complete. 2024-09-21T06:06:45.966 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.959+0000 7fb2a8ab7740 1 -- start start 2024-09-21T06:06:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.959+0000 7fb2a8ab7740 1 Processor -- start 2024-09-21T06:06:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.959+0000 7fb2a8ab7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ec69732f0 0x557ec6d4cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.959+0000 7fb2a8ab7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ec697b4d0 0x557ec6d4d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.959+0000 7fb2a8ab7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ec699c1b0 0x557ec6d4daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.959+0000 7fb2a8ab7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557ec6bddab0 con 0x557ec69732f0 2024-09-21T06:06:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.959+0000 7fb2a8ab7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557ec6c4b730 con 0x557ec699c1b0 2024-09-21T06:06:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.959+0000 7fb2a8ab7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557ec6d500d0 con 0x557ec697b4d0 2024-09-21T06:06:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.960+0000 7fb2a8ab7740 10 client.0 _ll_register_callbacks cb 0x557ec6c9f6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.960+0000 7fb29b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ec69732f0 0x557ec6d4cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.960+0000 7fb29b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ec69732f0 0x557ec6d4cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57628/0 (socket says 172.21.15.117:57628) 2024-09-21T06:06:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.960+0000 7fb29b7fe640 1 -- 172.21.15.117:0/8466447 learned_addr learned my addr 172.21.15.117:0/8466447 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.960+0000 7fb29bfff640 1 --2- 172.21.15.117:0/8466447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ec699c1b0 0x557ec6d4daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:45.967 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.960+0000 7fb29b7fe640 1 -- 172.21.15.117:0/8466447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ec697b4d0 msgr2=0x557ec6d4d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.960+0000 7fb293fff640 1 --2- 172.21.15.117:0/8466447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ec697b4d0 0x557ec6d4d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.960+0000 7fb29b7fe640 1 --2- 172.21.15.117:0/8466447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ec697b4d0 0x557ec6d4d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.960+0000 7fb29b7fe640 1 -- 172.21.15.117:0/8466447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ec699c1b0 msgr2=0x557ec6d4daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.960+0000 7fb29b7fe640 1 --2- 172.21.15.117:0/8466447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ec699c1b0 0x557ec6d4daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.960+0000 7fb29b7fe640 1 -- 172.21.15.117:0/8466447 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ec6a07790 con 0x557ec69732f0 2024-09-21T06:06:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.960+0000 7fb29b7fe640 1 --2- 172.21.15.117:0/8466447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ec69732f0 0x557ec6d4cfe0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fb28c005850 tx=0x7fb28c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada28146d92372d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.960+0000 7fb27affd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.960+0000 7fb27affd640 1 -- 172.21.15.117:0/8466447 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c035050 con 0x557ec69732f0 2024-09-21T06:06:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.960+0000 7fb27affd640 1 -- 172.21.15.117:0/8466447 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fb28c031ab0 con 0x557ec69732f0 2024-09-21T06:06:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.960+0000 7fb2a8ab7740 1 -- 172.21.15.117:0/8466447 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb28c0043f0 con 0x557ec69732f0 2024-09-21T06:06:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.960+0000 7fb27affd640 1 -- 172.21.15.117:0/8466447 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb28c03e040 con 0x557ec69732f0 2024-09-21T06:06:45.968 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.960+0000 7fb2a8ab7740 10 client.17499 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.960+0000 7fb2a8ab7740 1 -- 172.21.15.117:0/8466447 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557ec6a80620 con 0x557ec69732f0 2024-09-21T06:06:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.961+0000 7fb27affd640 1 -- 172.21.15.117:0/8466447 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb28c0043f0 con 0x557ec69732f0 2024-09-21T06:06:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.961+0000 7fb27affd640 1 -- 172.21.15.117:0/8466447 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb28c054550 con 0x557ec69732f0 2024-09-21T06:06:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.961+0000 7fb2a8ab7740 10 client.17499 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.961+0000 7fb2a8ab7740 10 client.17499 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:06:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.961+0000 7fb2a8ab7740 20 client.17499 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.961+0000 7fb2a8ab7740 1 --2- 172.21.15.117:0/8466447 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x557ec695ad80 0x557ec6d5eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.961+0000 7fb2a8ab7740 4 client.17499 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:06:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.961+0000 7fb2a8ab7740 1 -- 172.21.15.117:0/8466447 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ec6cb8640 con 0x557ec695ad80 2024-09-21T06:06:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.961+0000 7fb293fff640 1 --2- 172.21.15.117:0/8466447 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x557ec695ad80 0x557ec6d5eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.962+0000 7fb27affd640 10 client.17499 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:45.969 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.962+0000 7fb293fff640 1 --2- 172.21.15.117:0/8466447 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x557ec695ad80 0x557ec6d5eed0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2efe139399aeffb server_cookie=990e3a938618d1ad in_seq=0 out_seq=0 2024-09-21T06:06:45.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.962+0000 7fb27affd640 1 -- 172.21.15.117:0/8466447 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557ec6cb8640 con 0x557ec695ad80 2024-09-21T06:06:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.962+0000 7fb27affd640 10 client.17499 handle_command_reply: tid=0 2024-09-21T06:06:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.962+0000 7fb2a8ab7740 1 client.17499 shutdown 2024-09-21T06:06:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.963+0000 7fb2a8ab7740 1 -- 172.21.15.117:0/8466447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ec69732f0 msgr2=0x557ec6d4cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.963+0000 7fb2a8ab7740 1 --2- 172.21.15.117:0/8466447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ec69732f0 0x557ec6d4cfe0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fb28c005850 tx=0x7fb28c005880 comp rx=0 tx=0).stop 2024-09-21T06:06:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.963+0000 7fb2a8ab7740 1 -- 172.21.15.117:0/8466447 shutdown_connections 2024-09-21T06:06:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.963+0000 7fb2a8ab7740 1 --2- 172.21.15.117:0/8466447 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x557ec695ad80 0x557ec6d5eed0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.963+0000 7fb2a8ab7740 1 --2- 172.21.15.117:0/8466447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ec699c1b0 0x557ec6d4daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.963+0000 7fb2a8ab7740 1 --2- 172.21.15.117:0/8466447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ec697b4d0 0x557ec6d4d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.963+0000 7fb2a8ab7740 1 --2- 172.21.15.117:0/8466447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ec69732f0 0x557ec6d4cfe0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.963+0000 7fb27affd640 0 client.17499 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.963+0000 7fb2a8ab7740 1 -- 172.21.15.117:0/8466447 >> 172.21.15.117:0/8466447 conn(0x557ec6990300 msgr2=0x557ec6d429e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:45.972 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.963+0000 7fb2a8ab7740 1 -- 172.21.15.117:0/8466447 shutdown_connections 2024-09-21T06:06:45.973 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.963+0000 7fb2a8ab7740 1 -- 172.21.15.117:0/8466447 wait complete. 2024-09-21T06:06:45.973 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.963+0000 7fb2a8ab7740 20 client.17499 trim_cache size 0 max 16384 2024-09-21T06:06:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.977+0000 7fb2a8ab7740 1 Processor -- start 2024-09-21T06:06:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.977+0000 7fb2a8ab7740 1 -- start start 2024-09-21T06:06:45.980 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.977+0000 7fb2a8ab7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ec69732f0 0x557ec6d4ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.977+0000 7fb2a8ab7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ec697b4d0 0x557ec6d4a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.978+0000 7fb2a8ab7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ec699c1b0 0x557ec6d4a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.978+0000 7fb2a8ab7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c1af500 con 0x557ec697b4d0 2024-09-21T06:06:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.978+0000 7fb2a8ab7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c1b1040 con 0x557ec699c1b0 2024-09-21T06:06:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.978+0000 7fb2a8ab7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c1afbd0 con 0x557ec69732f0 2024-09-21T06:06:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.978+0000 7fb293fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ec697b4d0 0x557ec6d4a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.978+0000 7fb293fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ec697b4d0 0x557ec6d4a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57644/0 (socket says 172.21.15.117:57644) 2024-09-21T06:06:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.978+0000 7fb293fff640 1 -- 172.21.15.117:0/1937183870 learned_addr learned my addr 172.21.15.117:0/1937183870 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.978+0000 7fb29bfff640 1 --2- 172.21.15.117:0/1937183870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ec699c1b0 0x557ec6d4a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.978+0000 7fb29b7fe640 1 --2- 172.21.15.117:0/1937183870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ec69732f0 0x557ec6d4ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.978+0000 7fb293fff640 1 -- 172.21.15.117:0/1937183870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ec69732f0 msgr2=0x557ec6d4ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.978+0000 7fb293fff640 1 --2- 172.21.15.117:0/1937183870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ec69732f0 0x557ec6d4ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.978+0000 7fb293fff640 1 -- 172.21.15.117:0/1937183870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ec699c1b0 msgr2=0x557ec6d4a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.978+0000 7fb293fff640 1 --2- 172.21.15.117:0/1937183870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ec699c1b0 0x557ec6d4a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.981 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.978+0000 7fb293fff640 1 -- 172.21.15.117:0/1937183870 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ec6d4e2a0 con 0x557ec697b4d0 2024-09-21T06:06:45.982 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.978+0000 7fb293fff640 1 --2- 172.21.15.117:0/1937183870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ec697b4d0 0x557ec6d4a1b0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fb28400c940 tx=0x7fb284005350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c407cb7427608519 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:45.982 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.978+0000 7fb2917fa640 1 -- 172.21.15.117:0/1937183870 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28401d040 con 0x557ec697b4d0 2024-09-21T06:06:45.982 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.978+0000 7fb2917fa640 1 -- 172.21.15.117:0/1937183870 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2840adad0 con 0x557ec697b4d0 2024-09-21T06:06:45.982 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.978+0000 7fb2917fa640 1 -- 172.21.15.117:0/1937183870 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb284004e90 con 0x557ec697b4d0 2024-09-21T06:06:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.979+0000 7fb2a8ab7740 1 -- 172.21.15.117:0/1937183870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ec697b4d0 msgr2=0x557ec6d4a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.979+0000 7fb2a8ab7740 1 --2- 172.21.15.117:0/1937183870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ec697b4d0 0x557ec6d4a1b0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fb28400c940 tx=0x7fb284005350 comp rx=0 tx=0).stop 2024-09-21T06:06:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.979+0000 7fb2a8ab7740 1 -- 172.21.15.117:0/1937183870 shutdown_connections 2024-09-21T06:06:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.979+0000 7fb2a8ab7740 1 --2- 172.21.15.117:0/1937183870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ec697b4d0 0x557ec6d4a1b0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.979+0000 7fb2a8ab7740 1 --2- 172.21.15.117:0/1937183870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ec699c1b0 0x557ec6d4a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.979+0000 7fb2a8ab7740 1 --2- 172.21.15.117:0/1937183870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ec69732f0 0x557ec6d4ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.979+0000 7fb2a8ab7740 1 -- 172.21.15.117:0/1937183870 >> 172.21.15.117:0/1937183870 conn(0x557ec6990300 msgr2=0x557ec6d4e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.979+0000 7fb2a8ab7740 1 -- 172.21.15.117:0/1937183870 shutdown_connections 2024-09-21T06:06:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.979+0000 7fb2a8ab7740 1 -- 172.21.15.117:0/1937183870 wait complete. 2024-09-21T06:06:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.979+0000 7fb2a8ab7740 1 -- start start 2024-09-21T06:06:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.980+0000 7fb2a8ab7740 1 Processor -- start 2024-09-21T06:06:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.980+0000 7fb2a8ab7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ec69732f0 0x557ec6d619d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.980+0000 7fb2a8ab7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ec697b4d0 0x557ec6d61fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.980+0000 7fb2a8ab7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ec699c1b0 0x557ec6dc0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.980+0000 7fb2a8ab7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c1b0970 con 0x557ec699c1b0 2024-09-21T06:06:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.980+0000 7fb2a8ab7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c1b02a0 con 0x557ec697b4d0 2024-09-21T06:06:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.980+0000 7fb2a8ab7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c1ad500 con 0x557ec69732f0 2024-09-21T06:06:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.980+0000 7fb2a8ab7740 10 client.0 _ll_register_callbacks cb 0x7fb294008470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.980+0000 7fb29bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ec699c1b0 0x557ec6dc0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.980+0000 7fb29bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ec699c1b0 0x557ec6dc0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57654/0 (socket says 172.21.15.117:57654) 2024-09-21T06:06:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.980+0000 7fb29bfff640 1 -- 172.21.15.117:0/3839284574 learned_addr learned my addr 172.21.15.117:0/3839284574 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.980+0000 7fb29b7fe640 1 --2- 172.21.15.117:0/3839284574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ec69732f0 0x557ec6d619d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.980+0000 7fb293fff640 1 --2- 172.21.15.117:0/3839284574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ec697b4d0 0x557ec6d61fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.980+0000 7fb29bfff640 1 -- 172.21.15.117:0/3839284574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ec69732f0 msgr2=0x557ec6d619d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.980+0000 7fb29bfff640 1 --2- 172.21.15.117:0/3839284574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ec69732f0 0x557ec6d619d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.980+0000 7fb29bfff640 1 -- 172.21.15.117:0/3839284574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ec697b4d0 msgr2=0x557ec6d61fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.980+0000 7fb29bfff640 1 --2- 172.21.15.117:0/3839284574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ec697b4d0 0x557ec6d61fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.980+0000 7fb29bfff640 1 -- 172.21.15.117:0/3839284574 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ec6dc0cb0 con 0x557ec699c1b0 2024-09-21T06:06:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.980+0000 7fb293fff640 1 --2- 172.21.15.117:0/3839284574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ec697b4d0 0x557ec6d61fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:06:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.981+0000 7fb29bfff640 1 --2- 172.21.15.117:0/3839284574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ec699c1b0 0x557ec6dc0610 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fb29401eae0 tx=0x7fb294004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99630dc906b23ffb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.981+0000 7fb27affd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.981+0000 7fb27affd640 1 -- 172.21.15.117:0/3839284574 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29401cdd0 con 0x557ec699c1b0 2024-09-21T06:06:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.981+0000 7fb27affd640 1 -- 172.21.15.117:0/3839284574 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7fb294029450 con 0x557ec699c1b0 2024-09-21T06:06:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.981+0000 7fb2a8ab7740 1 -- 172.21.15.117:0/3839284574 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2840adad0 con 0x557ec699c1b0 2024-09-21T06:06:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.981+0000 7fb27affd640 1 -- 172.21.15.117:0/3839284574 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb294029650 con 0x557ec699c1b0 2024-09-21T06:06:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.981+0000 7fb2a8ab7740 10 client.17505 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.981+0000 7fb2a8ab7740 1 -- 172.21.15.117:0/3839284574 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557ec6dc91b0 con 0x557ec699c1b0 2024-09-21T06:06:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.981+0000 7fb27affd640 1 -- 172.21.15.117:0/3839284574 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2840adad0 con 0x557ec699c1b0 2024-09-21T06:06:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.981+0000 7fb27affd640 1 -- 172.21.15.117:0/3839284574 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb2940324e0 con 0x557ec699c1b0 2024-09-21T06:06:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.981+0000 7fb2a8ab7740 10 client.17505 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.981+0000 7fb2a8ab7740 10 client.17505 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:06:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.981+0000 7fb2a8ab7740 20 client.17505 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.981+0000 7fb2a8ab7740 1 --2- 172.21.15.117:0/3839284574 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x557ec69301c0 0x557ec6dcb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.981+0000 7fb2a8ab7740 4 client.17505 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:06:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.981+0000 7fb2a8ab7740 1 -- 172.21.15.117:0/3839284574 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557ec6cb40f0 con 0x557ec69301c0 2024-09-21T06:06:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.982+0000 7fb29b7fe640 1 --2- 172.21.15.117:0/3839284574 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x557ec69301c0 0x557ec6dcb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.982+0000 7fb29b7fe640 1 --2- 172.21.15.117:0/3839284574 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x557ec69301c0 0x557ec6dcb970 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aad5b941bf96cddb server_cookie=42046af558039422 in_seq=0 out_seq=0 2024-09-21T06:06:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.982+0000 7fb27affd640 10 client.17505 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:45.991 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.989+0000 7fb27affd640 1 -- 172.21.15.117:0/3839284574 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557ec6cb40f0 con 0x557ec69301c0 2024-09-21T06:06:45.991 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.989+0000 7fb27affd640 10 client.17505 handle_command_reply: tid=0 2024-09-21T06:06:45.991 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.989+0000 7fb2a8ab7740 1 client.17505 shutdown 2024-09-21T06:06:45.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.989+0000 7fb2a8ab7740 1 -- 172.21.15.117:0/3839284574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ec699c1b0 msgr2=0x557ec6dc0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:45.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.989+0000 7fb2a8ab7740 1 --2- 172.21.15.117:0/3839284574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ec699c1b0 0x557ec6dc0610 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fb29401eae0 tx=0x7fb294004a40 comp rx=0 tx=0).stop 2024-09-21T06:06:45.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.989+0000 7fb2a8ab7740 1 -- 172.21.15.117:0/3839284574 shutdown_connections 2024-09-21T06:06:45.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.989+0000 7fb2a8ab7740 1 --2- 172.21.15.117:0/3839284574 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x557ec69301c0 0x557ec6dcb970 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.989+0000 7fb2a8ab7740 1 --2- 172.21.15.117:0/3839284574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557ec699c1b0 0x557ec6dc0610 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.989+0000 7fb2a8ab7740 1 --2- 172.21.15.117:0/3839284574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557ec697b4d0 0x557ec6d61fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.989+0000 7fb2a8ab7740 1 --2- 172.21.15.117:0/3839284574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557ec69732f0 0x557ec6d619d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.989+0000 7fb27affd640 0 client.17505 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:45.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.989+0000 7fb2a8ab7740 1 -- 172.21.15.117:0/3839284574 >> 172.21.15.117:0/3839284574 conn(0x557ec6990300 msgr2=0x557ec6d429e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:45.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.989+0000 7fb2a8ab7740 1 -- 172.21.15.117:0/3839284574 shutdown_connections 2024-09-21T06:06:45.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.989+0000 7fb2a8ab7740 1 -- 172.21.15.117:0/3839284574 wait complete. 2024-09-21T06:06:45.992 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.989+0000 7fb2a8ab7740 20 client.17505 trim_cache size 0 max 16384 2024-09-21T06:06:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T06:06:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T06:06:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T06:06:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T06:06:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.990+0000 7fb2a1b52640 1 -- 172.21.15.117:0/3816084227 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb2880b6680 msgr2=0x7fb2880b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.990+0000 7fb2a1b52640 1 --2- 172.21.15.117:0/3816084227 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb2880b6680 0x7fb2880b8b90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb294008e80 tx=0x7fb2940085d0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.990+0000 7fb2a1b52640 1 -- 172.21.15.117:0/3816084227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb29c108330 msgr2=0x7fb29c19b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.990+0000 7fb2a1b52640 1 --2- 172.21.15.117:0/3816084227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb29c108330 0x7fb29c19b7a0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fb28400c970 tx=0x7fb28400ce30 comp rx=0 tx=0).stop 2024-09-21T06:06:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.990+0000 7fb2a1b52640 1 -- 172.21.15.117:0/3816084227 shutdown_connections 2024-09-21T06:06:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.990+0000 7fb2a1b52640 1 --2- 172.21.15.117:0/3816084227 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb2880b6680 0x7fb2880b8b90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.990+0000 7fb2a1b52640 1 --2- 172.21.15.117:0/3816084227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb29c108330 0x7fb29c19b7a0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.990+0000 7fb2a1b52640 1 --2- 172.21.15.117:0/3816084227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb29c108c80 0x7fb29c19bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.990+0000 7fb2a1b52640 1 --2- 172.21.15.117:0/3816084227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb29c1030d0 0x7fb29c19b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.990+0000 7fb2a1b52640 1 -- 172.21.15.117:0/3816084227 >> 172.21.15.117:0/3816084227 conn(0x7fb29c077ad0 msgr2=0x7fb29c106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.990+0000 7fb2a1b52640 1 -- 172.21.15.117:0/3816084227 shutdown_connections 2024-09-21T06:06:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:45.990+0000 7fb2a1b52640 1 -- 172.21.15.117:0/3816084227 wait complete. 2024-09-21T06:06:46.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:45 smithi192 ceph-mon[34700]: scrub summary: idle 2024-09-21T06:06:46.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:45 smithi192 ceph-mon[34700]: pgmap v615: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:06:46.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:45 smithi117 ceph-mon[30947]: scrub summary: idle 2024-09-21T06:06:46.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:45 smithi117 ceph-mon[30947]: pgmap v615: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:06:46.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:45 smithi138 ceph-mon[34391]: scrub summary: idle 2024-09-21T06:06:46.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:45 smithi138 ceph-mon[34391]: pgmap v615: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:06:48.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:47 smithi192 ceph-mon[34700]: pgmap v616: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-09-21T06:06:48.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:47 smithi117 ceph-mon[30947]: pgmap v616: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-09-21T06:06:48.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:47 smithi138 ceph-mon[34391]: pgmap v616: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-09-21T06:06:50.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:49 smithi192 ceph-mon[34700]: pgmap v617: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:06:50.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:49 smithi117 ceph-mon[30947]: pgmap v617: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:06:50.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:49 smithi138 ceph-mon[34391]: pgmap v617: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:06:52.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:51 smithi192 ceph-mon[34700]: pgmap v618: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:06:52.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:51 smithi117 ceph-mon[30947]: pgmap v618: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:06:52.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:51 smithi138 ceph-mon[34391]: pgmap v618: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:06:54.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:53 smithi192 ceph-mon[34700]: pgmap v619: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T06:06:54.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:53 smithi117 ceph-mon[30947]: pgmap v619: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T06:06:54.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:53 smithi138 ceph-mon[34391]: pgmap v619: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T06:06:56.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:55 smithi192 ceph-mon[34700]: pgmap v620: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 170 B/s wr, 14 op/s 2024-09-21T06:06:56.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:55 smithi117 ceph-mon[30947]: pgmap v620: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 170 B/s wr, 14 op/s 2024-09-21T06:06:56.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:55 smithi138 ceph-mon[34391]: pgmap v620: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 170 B/s wr, 14 op/s 2024-09-21T06:06:58.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:57 smithi192 ceph-mon[34700]: pgmap v621: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T06:06:58.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:57 smithi117 ceph-mon[30947]: pgmap v621: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T06:06:58.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:57 smithi138 ceph-mon[34391]: pgmap v621: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T06:06:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.098+0000 7f0b9529b640 1 -- 172.21.15.117:0/807811269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b90109400 msgr2=0x7f0b90109810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.098+0000 7f0b9529b640 1 --2- 172.21.15.117:0/807811269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b90109400 0x7f0b90109810 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f0b78009900 tx=0x7f0b7802a410 comp rx=0 tx=0).stop 2024-09-21T06:06:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.098+0000 7f0b9529b640 1 -- 172.21.15.117:0/807811269 shutdown_connections 2024-09-21T06:06:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.098+0000 7f0b9529b640 1 --2- 172.21.15.117:0/807811269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b90109400 0x7f0b90109810 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.098+0000 7f0b9529b640 1 --2- 172.21.15.117:0/807811269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b90109d50 0x7f0b90110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.098+0000 7f0b9529b640 1 --2- 172.21.15.117:0/807811269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b90103ad0 0x7f0b90103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.098+0000 7f0b9529b640 1 -- 172.21.15.117:0/807811269 >> 172.21.15.117:0/807811269 conn(0x7f0b900ff510 msgr2=0x7f0b90101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.098+0000 7f0b9529b640 1 -- 172.21.15.117:0/807811269 shutdown_connections 2024-09-21T06:06:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.098+0000 7f0b9529b640 1 -- 172.21.15.117:0/807811269 wait complete. 2024-09-21T06:06:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.099+0000 7f0b9529b640 1 Processor -- start 2024-09-21T06:06:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.099+0000 7f0b9529b640 1 -- start start 2024-09-21T06:06:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.099+0000 7f0b9529b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b90103ad0 0x7f0b9019a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.099+0000 7f0b9529b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b90109400 0x7f0b9019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.099+0000 7f0b9529b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b90109d50 0x7f0b9019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.099+0000 7f0b9529b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0b90113a10 con 0x7f0b90103ad0 2024-09-21T06:06:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.099+0000 7f0b9529b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0b90113890 con 0x7f0b90109d50 2024-09-21T06:06:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.099+0000 7f0b9529b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0b90113b90 con 0x7f0b90109400 2024-09-21T06:06:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.099+0000 7f0b8effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b90103ad0 0x7f0b9019a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.099+0000 7f0b8effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b90103ad0 0x7f0b9019a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36586/0 (socket says 172.21.15.117:36586) 2024-09-21T06:06:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.099+0000 7f0b8effd640 1 -- 172.21.15.117:0/3547410475 learned_addr learned my addr 172.21.15.117:0/3547410475 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.100+0000 7f0b867fc640 1 --2- 172.21.15.117:0/3547410475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b90109400 0x7f0b9019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.100+0000 7f0b8f7fe640 1 --2- 172.21.15.117:0/3547410475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b90109d50 0x7f0b9019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.100+0000 7f0b8effd640 1 -- 172.21.15.117:0/3547410475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b90109400 msgr2=0x7f0b9019a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.100+0000 7f0b8effd640 1 --2- 172.21.15.117:0/3547410475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b90109400 0x7f0b9019a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.100+0000 7f0b8effd640 1 -- 172.21.15.117:0/3547410475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b90109d50 msgr2=0x7f0b9019fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.100+0000 7f0b8effd640 1 --2- 172.21.15.117:0/3547410475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b90109d50 0x7f0b9019fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.100+0000 7f0b8effd640 1 -- 172.21.15.117:0/3547410475 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b901a0360 con 0x7f0b90103ad0 2024-09-21T06:06:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.100+0000 7f0b8f7fe640 1 --2- 172.21.15.117:0/3547410475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b90109d50 0x7f0b9019fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:06:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.100+0000 7f0b8effd640 1 --2- 172.21.15.117:0/3547410475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b90103ad0 0x7f0b9019a430 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f0b8000d950 tx=0x7f0b8000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.100+0000 7f0b8cff9640 1 -- 172.21.15.117:0/3547410475 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80017030 con 0x7f0b90103ad0 2024-09-21T06:06:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.100+0000 7f0b8cff9640 1 -- 172.21.15.117:0/3547410475 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b80004020 con 0x7f0b90103ad0 2024-09-21T06:06:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.100+0000 7f0b9529b640 1 -- 172.21.15.117:0/3547410475 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b901a0650 con 0x7f0b90103ad0 2024-09-21T06:06:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.100+0000 7f0b8cff9640 1 -- 172.21.15.117:0/3547410475 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b800047a0 con 0x7f0b90103ad0 2024-09-21T06:06:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.100+0000 7f0b9529b640 1 -- 172.21.15.117:0/3547410475 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b901a6de0 con 0x7f0b90103ad0 2024-09-21T06:06:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.103+0000 7f0b9c20f740 1 Processor -- start 2024-09-21T06:06:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.103+0000 7f0b9c20f740 1 -- start start 2024-09-21T06:06:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.103+0000 7f0b8cff9640 1 -- 172.21.15.117:0/3547410475 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f0b80009a30 con 0x7f0b90103ad0 2024-09-21T06:06:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.103+0000 7f0b9c20f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628d65481b0 0x5628d68ed9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.103+0000 7f0b9c20f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628d651f2f0 0x5628d68edef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.103+0000 7f0b9c20f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628d65274d0 0x5628d68ee430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.103+0000 7f0b9c20f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5628d68626f0 con 0x5628d65274d0 2024-09-21T06:06:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.103+0000 7f0b9c20f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5628d6851b20 con 0x5628d65481b0 2024-09-21T06:06:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.103+0000 7f0b9c20f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5628d6855930 con 0x5628d651f2f0 2024-09-21T06:06:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.104+0000 7f0b867fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628d65274d0 0x5628d68ee430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.104+0000 7f0b867fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628d65274d0 0x5628d68ee430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36594/0 (socket says 172.21.15.117:36594) 2024-09-21T06:06:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.104+0000 7f0b867fc640 1 -- 172.21.15.117:0/3078830600 learned_addr learned my addr 172.21.15.117:0/3078830600 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.104+0000 7f0b8f7fe640 1 --2- 172.21.15.117:0/3078830600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628d65481b0 0x5628d68ed9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.104+0000 7f0b8effd640 1 --2- 172.21.15.117:0/3078830600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628d651f2f0 0x5628d68edef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.104+0000 7f0b867fc640 1 -- 172.21.15.117:0/3078830600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628d651f2f0 msgr2=0x5628d68edef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.104+0000 7f0b867fc640 1 --2- 172.21.15.117:0/3078830600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628d651f2f0 0x5628d68edef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.104+0000 7f0b867fc640 1 -- 172.21.15.117:0/3078830600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628d65481b0 msgr2=0x5628d68ed9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.104+0000 7f0b867fc640 1 --2- 172.21.15.117:0/3078830600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628d65481b0 0x5628d68ed9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.104+0000 7f0b867fc640 1 -- 172.21.15.117:0/3078830600 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628d6817530 con 0x5628d65274d0 2024-09-21T06:06:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.104+0000 7f0b867fc640 1 --2- 172.21.15.117:0/3078830600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628d65274d0 0x5628d68ee430 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f0b78009900 tx=0x7f0b78000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b0ab3458ad6290b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.104+0000 7f0b8cff9640 1 --2- 172.21.15.117:0/3547410475 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0b7c0b6680 0x7f0b7c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.104+0000 7f0b857fa640 1 -- 172.21.15.117:0/3078830600 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b78040070 con 0x5628d65274d0 2024-09-21T06:06:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.104+0000 7f0b857fa640 1 -- 172.21.15.117:0/3078830600 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b78004030 con 0x5628d65274d0 2024-09-21T06:06:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.105+0000 7f0b857fa640 1 -- 172.21.15.117:0/3078830600 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b78035040 con 0x5628d65274d0 2024-09-21T06:06:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.105+0000 7f0b8f7fe640 1 --2- 172.21.15.117:0/3547410475 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0b7c0b6680 0x7f0b7c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.105+0000 7f0b8cff9640 1 -- 172.21.15.117:0/3547410475 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7f0b800e7200 con 0x7f0b90103ad0 2024-09-21T06:06:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.105+0000 7f0b9c20f740 1 -- 172.21.15.117:0/3078830600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628d65274d0 msgr2=0x5628d68ee430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.105+0000 7f0b8f7fe640 1 --2- 172.21.15.117:0/3547410475 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0b7c0b6680 0x7f0b7c0b8b90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0b88005fd0 tx=0x7f0b88009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.105+0000 7f0b9c20f740 1 --2- 172.21.15.117:0/3078830600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628d65274d0 0x5628d68ee430 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f0b78009900 tx=0x7f0b78000bc0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.106+0000 7f0b9c20f740 1 -- 172.21.15.117:0/3078830600 shutdown_connections 2024-09-21T06:06:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.106+0000 7f0b9c20f740 1 --2- 172.21.15.117:0/3078830600 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628d65274d0 0x5628d68ee430 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.106+0000 7f0b9c20f740 1 --2- 172.21.15.117:0/3078830600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628d651f2f0 0x5628d68edef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.106+0000 7f0b9c20f740 1 --2- 172.21.15.117:0/3078830600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628d65481b0 0x5628d68ed9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.106+0000 7f0b9c20f740 1 -- 172.21.15.117:0/3078830600 >> 172.21.15.117:0/3078830600 conn(0x5628d653c300 msgr2=0x5628d68eb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.106+0000 7f0b9c20f740 1 -- 172.21.15.117:0/3078830600 shutdown_connections 2024-09-21T06:06:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.106+0000 7f0b9c20f740 1 -- 172.21.15.117:0/3078830600 wait complete. 2024-09-21T06:06:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.106+0000 7f0b9c20f740 1 -- start start 2024-09-21T06:06:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.106+0000 7f0b9c20f740 1 Processor -- start 2024-09-21T06:06:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.106+0000 7f0b9c20f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628d651f2f0 0x5628d68f9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.106+0000 7f0b9c20f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628d65274d0 0x5628d68f95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.106+0000 7f0b9c20f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628d65481b0 0x5628d69053c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.106+0000 7f0b9c20f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5628d6789ab0 con 0x5628d651f2f0 2024-09-21T06:06:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.106+0000 7f0b9c20f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5628d67f7730 con 0x5628d65274d0 2024-09-21T06:06:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.106+0000 7f0b9c20f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5628d68fc0d0 con 0x5628d65481b0 2024-09-21T06:06:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.106+0000 7f0b9c20f740 10 client.0 _ll_register_callbacks cb 0x5628d684b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.106+0000 7f0b8effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628d651f2f0 0x5628d68f9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.107+0000 7f0b8effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628d651f2f0 0x5628d68f9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36600/0 (socket says 172.21.15.117:36600) 2024-09-21T06:06:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.107+0000 7f0b8effd640 1 -- 172.21.15.117:0/494160692 learned_addr learned my addr 172.21.15.117:0/494160692 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.107+0000 7f0b8effd640 1 -- 172.21.15.117:0/494160692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628d65481b0 msgr2=0x5628d69053c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.107+0000 7f0b8effd640 1 --2- 172.21.15.117:0/494160692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628d65481b0 0x5628d69053c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.107+0000 7f0b8effd640 1 -- 172.21.15.117:0/494160692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628d65274d0 msgr2=0x5628d68f95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.107+0000 7f0b867fc640 1 --2- 172.21.15.117:0/494160692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628d65274d0 0x5628d68f95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.107+0000 7f0b8effd640 1 --2- 172.21.15.117:0/494160692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628d65274d0 0x5628d68f95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.107+0000 7f0b8effd640 1 -- 172.21.15.117:0/494160692 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628d65b3790 con 0x5628d651f2f0 2024-09-21T06:06:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.107+0000 7f0b867fc640 1 --2- 172.21.15.117:0/494160692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628d65274d0 0x5628d68f95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:06:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.107+0000 7f0b8effd640 1 --2- 172.21.15.117:0/494160692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628d651f2f0 0x5628d68f9060 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f0b800acda0 tx=0x7f0b800a97b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b02e23ad099a98ea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.107+0000 7f0b767fc640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.107+0000 7f0b767fc640 1 -- 172.21.15.117:0/494160692 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b800ac810 con 0x5628d651f2f0 2024-09-21T06:06:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.107+0000 7f0b767fc640 1 -- 172.21.15.117:0/494160692 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7f0b800ac300 con 0x5628d651f2f0 2024-09-21T06:06:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.107+0000 7f0b9c20f740 1 -- 172.21.15.117:0/494160692 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b78004030 con 0x5628d651f2f0 2024-09-21T06:06:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.107+0000 7f0b767fc640 1 -- 172.21.15.117:0/494160692 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0b800acbb0 con 0x5628d651f2f0 2024-09-21T06:06:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.107+0000 7f0b9c20f740 10 client.17517 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.107+0000 7f0b9c20f740 1 -- 172.21.15.117:0/494160692 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5628d662c620 con 0x5628d651f2f0 2024-09-21T06:06:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.107+0000 7f0b767fc640 1 -- 172.21.15.117:0/494160692 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b78004030 con 0x5628d651f2f0 2024-09-21T06:06:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.108+0000 7f0b767fc640 1 -- 172.21.15.117:0/494160692 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0b800981a0 con 0x5628d651f2f0 2024-09-21T06:06:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.108+0000 7f0b9c20f740 10 client.17517 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.108+0000 7f0b9c20f740 10 client.17517 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:06:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.108+0000 7f0b9c20f740 20 client.17517 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.108+0000 7f0b9c20f740 1 --2- 172.21.15.117:0/494160692 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5628d6506d80 0x5628d690b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.108+0000 7f0b9c20f740 4 client.17517 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:06:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.108+0000 7f0b9c20f740 1 -- 172.21.15.117:0/494160692 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628d6864640 con 0x5628d6506d80 2024-09-21T06:06:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.108+0000 7f0b867fc640 1 --2- 172.21.15.117:0/494160692 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5628d6506d80 0x5628d690b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.109+0000 7f0b767fc640 10 client.17517 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.109+0000 7f0b867fc640 1 --2- 172.21.15.117:0/494160692 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5628d6506d80 0x5628d690b140 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1f6edebed23a005 server_cookie=360fab0e251056cc in_seq=0 out_seq=0 2024-09-21T06:06:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.109+0000 7f0b767fc640 1 -- 172.21.15.117:0/494160692 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5628d6864640 con 0x5628d6506d80 2024-09-21T06:06:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.109+0000 7f0b767fc640 10 client.17517 handle_command_reply: tid=0 2024-09-21T06:06:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.109+0000 7f0b9c20f740 1 client.17517 shutdown 2024-09-21T06:06:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.110+0000 7f0b9c20f740 1 -- 172.21.15.117:0/494160692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628d651f2f0 msgr2=0x5628d68f9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.110+0000 7f0b9c20f740 1 --2- 172.21.15.117:0/494160692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628d651f2f0 0x5628d68f9060 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f0b800acda0 tx=0x7f0b800a97b0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.110+0000 7f0b9c20f740 1 -- 172.21.15.117:0/494160692 shutdown_connections 2024-09-21T06:06:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.110+0000 7f0b9c20f740 1 --2- 172.21.15.117:0/494160692 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5628d6506d80 0x5628d690b140 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.110+0000 7f0b9c20f740 1 --2- 172.21.15.117:0/494160692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628d65481b0 0x5628d69053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.110+0000 7f0b9c20f740 1 --2- 172.21.15.117:0/494160692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628d65274d0 0x5628d68f95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.110+0000 7f0b9c20f740 1 --2- 172.21.15.117:0/494160692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628d651f2f0 0x5628d68f9060 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.110+0000 7f0b767fc640 0 client.17517 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.110+0000 7f0b9c20f740 1 -- 172.21.15.117:0/494160692 >> 172.21.15.117:0/494160692 conn(0x5628d653c300 msgr2=0x5628d68ee9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.110+0000 7f0b9c20f740 1 -- 172.21.15.117:0/494160692 shutdown_connections 2024-09-21T06:06:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.110+0000 7f0b9c20f740 1 -- 172.21.15.117:0/494160692 wait complete. 2024-09-21T06:06:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.110+0000 7f0b9c20f740 20 client.17517 trim_cache size 0 max 16384 2024-09-21T06:06:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.124+0000 7f0b9c20f740 1 Processor -- start 2024-09-21T06:06:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.124+0000 7f0b9c20f740 1 -- start start 2024-09-21T06:06:59.132 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.125+0000 7f0b9c20f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628d651f2f0 0x5628d68ef0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:59.132 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.125+0000 7f0b9c20f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628d65274d0 0x5628d69052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.125+0000 7f0b9c20f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628d65481b0 0x5628d6905800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.125+0000 7f0b9c20f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0b901af530 con 0x5628d651f2f0 2024-09-21T06:06:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.125+0000 7f0b9c20f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0b901b1070 con 0x5628d65274d0 2024-09-21T06:06:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.125+0000 7f0b9c20f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0b901afc00 con 0x5628d65481b0 2024-09-21T06:06:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.125+0000 7f0b8effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628d651f2f0 0x5628d68ef0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.125+0000 7f0b8effd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628d651f2f0 0x5628d68ef0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36604/0 (socket says 172.21.15.117:36604) 2024-09-21T06:06:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.125+0000 7f0b8effd640 1 -- 172.21.15.117:0/4092147514 learned_addr learned my addr 172.21.15.117:0/4092147514 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.125+0000 7f0b867fc640 1 --2- 172.21.15.117:0/4092147514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628d65274d0 0x5628d69052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.125+0000 7f0b8effd640 1 -- 172.21.15.117:0/4092147514 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628d65481b0 msgr2=0x5628d6905800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.125+0000 7f0b8effd640 1 --2- 172.21.15.117:0/4092147514 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628d65481b0 0x5628d6905800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.125+0000 7f0b8effd640 1 -- 172.21.15.117:0/4092147514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628d65274d0 msgr2=0x5628d69052c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.125+0000 7f0b8effd640 1 --2- 172.21.15.117:0/4092147514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628d65274d0 0x5628d69052c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.125+0000 7f0b8effd640 1 -- 172.21.15.117:0/4092147514 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628d68f07c0 con 0x5628d651f2f0 2024-09-21T06:06:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.125+0000 7f0b867fc640 1 --2- 172.21.15.117:0/4092147514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628d65274d0 0x5628d69052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:06:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.125+0000 7f0b8effd640 1 --2- 172.21.15.117:0/4092147514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628d651f2f0 0x5628d68ef0f0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f0b800045f0 tx=0x7f0b80009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c72a235c2249a960 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.125+0000 7f0b84ff9640 1 -- 172.21.15.117:0/4092147514 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b800ac810 con 0x5628d651f2f0 2024-09-21T06:06:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.125+0000 7f0b84ff9640 1 -- 172.21.15.117:0/4092147514 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b800d4020 con 0x5628d651f2f0 2024-09-21T06:06:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.125+0000 7f0b84ff9640 1 -- 172.21.15.117:0/4092147514 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b800b1b80 con 0x5628d651f2f0 2024-09-21T06:06:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.126+0000 7f0b9c20f740 1 -- 172.21.15.117:0/4092147514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628d651f2f0 msgr2=0x5628d68ef0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.126+0000 7f0b9c20f740 1 --2- 172.21.15.117:0/4092147514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628d651f2f0 0x5628d68ef0f0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f0b800045f0 tx=0x7f0b80009f30 comp rx=0 tx=0).stop 2024-09-21T06:06:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.126+0000 7f0b9c20f740 1 -- 172.21.15.117:0/4092147514 shutdown_connections 2024-09-21T06:06:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.126+0000 7f0b9c20f740 1 --2- 172.21.15.117:0/4092147514 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628d65481b0 0x5628d6905800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.126+0000 7f0b9c20f740 1 --2- 172.21.15.117:0/4092147514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628d65274d0 0x5628d69052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.126+0000 7f0b9c20f740 1 --2- 172.21.15.117:0/4092147514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628d651f2f0 0x5628d68ef0f0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.126+0000 7f0b9c20f740 1 -- 172.21.15.117:0/4092147514 >> 172.21.15.117:0/4092147514 conn(0x5628d653c300 msgr2=0x5628d68ee9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.126+0000 7f0b9c20f740 1 -- 172.21.15.117:0/4092147514 shutdown_connections 2024-09-21T06:06:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.126+0000 7f0b9c20f740 1 -- 172.21.15.117:0/4092147514 wait complete. 2024-09-21T06:06:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.127+0000 7f0b9c20f740 1 -- start start 2024-09-21T06:06:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.127+0000 7f0b9c20f740 1 Processor -- start 2024-09-21T06:06:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.127+0000 7f0b9c20f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628d651f2f0 0x5628d68fec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.127+0000 7f0b9c20f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628d65274d0 0x5628d68ff1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.127+0000 7f0b9c20f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628d65481b0 0x5628d696db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.127+0000 7f0b9c20f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0b901b09a0 con 0x5628d65274d0 2024-09-21T06:06:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.127+0000 7f0b9c20f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0b901b02d0 con 0x5628d65481b0 2024-09-21T06:06:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.127+0000 7f0b9c20f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0b901ad530 con 0x5628d651f2f0 2024-09-21T06:06:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.127+0000 7f0b9c20f740 10 client.0 _ll_register_callbacks cb 0x7f0b80004540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:06:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.127+0000 7f0b867fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628d65274d0 0x5628d68ff1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.127+0000 7f0b867fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628d65274d0 0x5628d68ff1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36608/0 (socket says 172.21.15.117:36608) 2024-09-21T06:06:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.127+0000 7f0b867fc640 1 -- 172.21.15.117:0/3738192793 learned_addr learned my addr 172.21.15.117:0/3738192793 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:06:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.127+0000 7f0b8f7fe640 1 --2- 172.21.15.117:0/3738192793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628d65481b0 0x5628d696db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.127+0000 7f0b8effd640 1 --2- 172.21.15.117:0/3738192793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628d651f2f0 0x5628d68fec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.128+0000 7f0b867fc640 1 -- 172.21.15.117:0/3738192793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628d651f2f0 msgr2=0x5628d68fec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.128+0000 7f0b867fc640 1 --2- 172.21.15.117:0/3738192793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628d651f2f0 0x5628d68fec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.128+0000 7f0b867fc640 1 -- 172.21.15.117:0/3738192793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628d65481b0 msgr2=0x5628d696db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:06:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.128+0000 7f0b867fc640 1 --2- 172.21.15.117:0/3738192793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628d65481b0 0x5628d696db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.128+0000 7f0b867fc640 1 -- 172.21.15.117:0/3738192793 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628d696e1b0 con 0x5628d65274d0 2024-09-21T06:06:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.128+0000 7f0b8f7fe640 1 --2- 172.21.15.117:0/3738192793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628d65481b0 0x5628d696db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:06:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.128+0000 7f0b867fc640 1 --2- 172.21.15.117:0/3738192793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628d65274d0 0x5628d68ff1f0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f0b78005150 tx=0x7f0b7802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba41808f5bcf233 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:06:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.128+0000 7f0b767fc640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:06:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.128+0000 7f0b767fc640 1 -- 172.21.15.117:0/3738192793 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b78040040 con 0x5628d65274d0 2024-09-21T06:06:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.128+0000 7f0b767fc640 1 -- 172.21.15.117:0/3738192793 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8441+0+0 (secure 0 0 0) 0x7f0b780051f0 con 0x5628d65274d0 2024-09-21T06:06:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.128+0000 7f0b9c20f740 1 -- 172.21.15.117:0/3738192793 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b800d4020 con 0x5628d65274d0 2024-09-21T06:06:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.128+0000 7f0b767fc640 1 -- 172.21.15.117:0/3738192793 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0b7805a020 con 0x5628d65274d0 2024-09-21T06:06:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.128+0000 7f0b9c20f740 10 client.17523 fetch_fsmap learned FSMap version 29 2024-09-21T06:06:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.128+0000 7f0b9c20f740 1 -- 172.21.15.117:0/3738192793 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5628d6975480 con 0x5628d65274d0 2024-09-21T06:06:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.128+0000 7f0b767fc640 1 -- 172.21.15.117:0/3738192793 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b800d4020 con 0x5628d65274d0 2024-09-21T06:06:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.128+0000 7f0b767fc640 1 -- 172.21.15.117:0/3738192793 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0b78004a20 con 0x5628d65274d0 2024-09-21T06:06:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.128+0000 7f0b9c20f740 10 client.17523 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:06:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.128+0000 7f0b9c20f740 10 client.17523 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:06:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.128+0000 7f0b9c20f740 20 client.17523 populate_metadata read hostname 'smithi117' 2024-09-21T06:06:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.128+0000 7f0b9c20f740 1 --2- 172.21.15.117:0/3738192793 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5628d64dc1c0 0x5628d6977890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:06:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.128+0000 7f0b9c20f740 4 client.17523 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:06:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.128+0000 7f0b9c20f740 1 -- 172.21.15.117:0/3738192793 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5628d68600f0 con 0x5628d64dc1c0 2024-09-21T06:06:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.129+0000 7f0b8effd640 1 --2- 172.21.15.117:0/3738192793 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5628d64dc1c0 0x5628d6977890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:06:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.129+0000 7f0b767fc640 10 client.17523 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.129+0000 7f0b8effd640 1 --2- 172.21.15.117:0/3738192793 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5628d64dc1c0 0x5628d6977890 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23518ef2f1ba111f server_cookie=eac867626ad90cf0 in_seq=0 out_seq=0 2024-09-21T06:06:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.136+0000 7f0b767fc640 1 -- 172.21.15.117:0/3738192793 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5628d68600f0 con 0x5628d64dc1c0 2024-09-21T06:06:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.136+0000 7f0b767fc640 10 client.17523 handle_command_reply: tid=0 2024-09-21T06:06:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.136+0000 7f0b9c20f740 1 client.17523 shutdown 2024-09-21T06:06:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.136+0000 7f0b9c20f740 1 -- 172.21.15.117:0/3738192793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628d65274d0 msgr2=0x5628d68ff1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.136+0000 7f0b9c20f740 1 --2- 172.21.15.117:0/3738192793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628d65274d0 0x5628d68ff1f0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f0b78005150 tx=0x7f0b7802a410 comp rx=0 tx=0).stop 2024-09-21T06:06:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.136+0000 7f0b9c20f740 1 -- 172.21.15.117:0/3738192793 shutdown_connections 2024-09-21T06:06:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.136+0000 7f0b9c20f740 1 --2- 172.21.15.117:0/3738192793 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5628d64dc1c0 0x5628d6977890 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.136+0000 7f0b9c20f740 1 --2- 172.21.15.117:0/3738192793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5628d65274d0 0x5628d68ff1f0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.136+0000 7f0b9c20f740 1 --2- 172.21.15.117:0/3738192793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5628d65481b0 0x5628d696db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.136+0000 7f0b767fc640 0 client.17523 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:06:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.136+0000 7f0b9c20f740 1 --2- 172.21.15.117:0/3738192793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5628d651f2f0 0x5628d68fec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.141 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.136+0000 7f0b9c20f740 1 -- 172.21.15.117:0/3738192793 >> 172.21.15.117:0/3738192793 conn(0x5628d653c300 msgr2=0x5628d68eb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:59.141 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.136+0000 7f0b9c20f740 1 -- 172.21.15.117:0/3738192793 shutdown_connections 2024-09-21T06:06:59.141 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.136+0000 7f0b9c20f740 1 -- 172.21.15.117:0/3738192793 wait complete. 2024-09-21T06:06:59.141 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.136+0000 7f0b9c20f740 20 client.17523 trim_cache size 0 max 16384 2024-09-21T06:06:59.141 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T06:06:59.141 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T06:06:59.141 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T06:06:59.141 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T06:06:59.142 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.137+0000 7f0b9529b640 1 -- 172.21.15.117:0/3547410475 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0b7c0b6680 msgr2=0x7f0b7c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:59.142 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.137+0000 7f0b9529b640 1 --2- 172.21.15.117:0/3547410475 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0b7c0b6680 0x7f0b7c0b8b90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0b88005fd0 tx=0x7f0b88009500 comp rx=0 tx=0).stop 2024-09-21T06:06:59.142 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.137+0000 7f0b9529b640 1 -- 172.21.15.117:0/3547410475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b90103ad0 msgr2=0x7f0b9019a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:06:59.142 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.137+0000 7f0b9529b640 1 --2- 172.21.15.117:0/3547410475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b90103ad0 0x7f0b9019a430 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f0b8000d950 tx=0x7f0b8000de10 comp rx=0 tx=0).stop 2024-09-21T06:06:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.137+0000 7f0b9529b640 1 -- 172.21.15.117:0/3547410475 shutdown_connections 2024-09-21T06:06:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.137+0000 7f0b9529b640 1 --2- 172.21.15.117:0/3547410475 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0b7c0b6680 0x7f0b7c0b8b90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.137+0000 7f0b9529b640 1 --2- 172.21.15.117:0/3547410475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b90109d50 0x7f0b9019fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.137+0000 7f0b9529b640 1 --2- 172.21.15.117:0/3547410475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b90109400 0x7f0b9019a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.137+0000 7f0b9529b640 1 --2- 172.21.15.117:0/3547410475 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b90103ad0 0x7f0b9019a430 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:06:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.137+0000 7f0b9529b640 1 -- 172.21.15.117:0/3547410475 >> 172.21.15.117:0/3547410475 conn(0x7f0b900ff510 msgr2=0x7f0b90101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:06:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.137+0000 7f0b9529b640 1 -- 172.21.15.117:0/3547410475 shutdown_connections 2024-09-21T06:06:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:06:59.137+0000 7f0b9529b640 1 -- 172.21.15.117:0/3547410475 wait complete. 2024-09-21T06:07:00.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:06:59 smithi192 ceph-mon[34700]: pgmap v622: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-21T06:07:00.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:06:59 smithi117 ceph-mon[30947]: pgmap v622: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-21T06:07:00.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:06:59 smithi138 ceph-mon[34391]: pgmap v622: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-21T06:07:02.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:01 smithi192 ceph-mon[34700]: pgmap v623: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:07:02.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:01 smithi117 ceph-mon[30947]: pgmap v623: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:07:02.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:01 smithi138 ceph-mon[34391]: pgmap v623: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:07:04.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:03 smithi192 ceph-mon[34700]: pgmap v624: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T06:07:04.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:03 smithi117 ceph-mon[30947]: pgmap v624: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T06:07:04.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:03 smithi138 ceph-mon[34391]: pgmap v624: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-21T06:07:06.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:05 smithi192 ceph-mon[34700]: pgmap v625: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-21T06:07:06.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:05 smithi117 ceph-mon[30947]: pgmap v625: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-21T06:07:06.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:05 smithi138 ceph-mon[34391]: pgmap v625: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-21T06:07:08.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:07 smithi192 ceph-mon[34700]: pgmap v626: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-21T06:07:08.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:07 smithi117 ceph-mon[30947]: pgmap v626: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-21T06:07:08.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:07 smithi138 ceph-mon[34391]: pgmap v626: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-21T06:07:10.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:09 smithi192 ceph-mon[34700]: pgmap v627: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 28 op/s 2024-09-21T06:07:10.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:09 smithi117 ceph-mon[30947]: pgmap v627: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 28 op/s 2024-09-21T06:07:10.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:09 smithi138 ceph-mon[34391]: pgmap v627: 97 pgs: 97 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 28 op/s 2024-09-21T06:07:11.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:10 smithi117 ceph-mon[30947]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T06:07:11.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:10 smithi138 ceph-mon[34391]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T06:07:11.479 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:10 smithi192 ceph-mon[34700]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T06:07:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.237+0000 7f3fa401d640 1 -- 172.21.15.117:0/2214150210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f9c108300 msgr2=0x7f3f9c108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.237+0000 7f3fa401d640 1 --2- 172.21.15.117:0/2214150210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f9c108300 0x7f3f9c108710 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c009900 tx=0x7f3f8c02a410 comp rx=0 tx=0).stop 2024-09-21T06:07:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.237+0000 7f3fa401d640 1 -- 172.21.15.117:0/2214150210 shutdown_connections 2024-09-21T06:07:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.237+0000 7f3fa401d640 1 --2- 172.21.15.117:0/2214150210 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f9c108300 0x7f3f9c108710 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.237+0000 7f3fa401d640 1 --2- 172.21.15.117:0/2214150210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f9c108c50 0x7f3f9c10f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.237+0000 7f3fa401d640 1 --2- 172.21.15.117:0/2214150210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f9c1029d0 0x7f3f9c102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.237+0000 7f3fa401d640 1 -- 172.21.15.117:0/2214150210 >> 172.21.15.117:0/2214150210 conn(0x7f3f9c0fe410 msgr2=0x7f3f9c100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.237+0000 7f3fa401d640 1 -- 172.21.15.117:0/2214150210 shutdown_connections 2024-09-21T06:07:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.237+0000 7f3fa401d640 1 -- 172.21.15.117:0/2214150210 wait complete. 2024-09-21T06:07:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.238+0000 7f3fa401d640 1 Processor -- start 2024-09-21T06:07:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.238+0000 7f3fa401d640 1 -- start start 2024-09-21T06:07:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.238+0000 7f3fa401d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f9c1029d0 0x7f3f9c19b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:12.242 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.238+0000 7f3fa401d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f9c108300 0x7f3f9c19b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:12.243 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.238+0000 7f3fa401d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f9c108c50 0x7f3f9c19bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:12.243 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.238+0000 7f3fa401d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9c112910 con 0x7f3f9c108300 2024-09-21T06:07:12.243 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.238+0000 7f3fa401d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9c112790 con 0x7f3f9c1029d0 2024-09-21T06:07:12.243 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.238+0000 7f3fa401d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9c112a90 con 0x7f3f9c108c50 2024-09-21T06:07:12.243 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.238+0000 7f3fa15b8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f9c108300 0x7f3f9c19b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:12.243 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.238+0000 7f3fa15b8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f9c108300 0x7f3f9c19b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60486/0 (socket says 172.21.15.117:60486) 2024-09-21T06:07:12.243 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.239+0000 7f3fa15b8640 1 -- 172.21.15.117:0/1081455454 learned_addr learned my addr 172.21.15.117:0/1081455454 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:12.243 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.239+0000 7f3fa1db9640 1 --2- 172.21.15.117:0/1081455454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f9c1029d0 0x7f3f9c19b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:12.243 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.239+0000 7f3fa15b8640 1 -- 172.21.15.117:0/1081455454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f9c108c50 msgr2=0x7f3f9c19bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:12.243 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.239+0000 7f3fa15b8640 1 --2- 172.21.15.117:0/1081455454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f9c108c50 0x7f3f9c19bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.243 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.239+0000 7f3fa15b8640 1 -- 172.21.15.117:0/1081455454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f9c1029d0 msgr2=0x7f3f9c19b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:12.243 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.239+0000 7f3fa15b8640 1 --2- 172.21.15.117:0/1081455454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f9c1029d0 0x7f3f9c19b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.244 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.239+0000 7f3fa15b8640 1 -- 172.21.15.117:0/1081455454 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f9c1a6510 con 0x7f3f9c108300 2024-09-21T06:07:12.244 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.239+0000 7f3fa15b8640 1 --2- 172.21.15.117:0/1081455454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f9c108300 0x7f3f9c19b740 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c02a930 tx=0x7f3f8c031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:12.244 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.239+0000 7f3f92ffd640 1 -- 172.21.15.117:0/1081455454 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c03f030 con 0x7f3f9c108300 2024-09-21T06:07:12.244 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.239+0000 7f3f92ffd640 1 -- 172.21.15.117:0/1081455454 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f8c004490 con 0x7f3f9c108300 2024-09-21T06:07:12.244 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.239+0000 7f3fa401d640 1 -- 172.21.15.117:0/1081455454 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f9c1a67a0 con 0x7f3f9c108300 2024-09-21T06:07:12.244 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.239+0000 7f3f92ffd640 1 -- 172.21.15.117:0/1081455454 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c035070 con 0x7f3f9c108300 2024-09-21T06:07:12.244 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.239+0000 7f3fa401d640 1 -- 172.21.15.117:0/1081455454 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f9c1a6c90 con 0x7f3f9c108300 2024-09-21T06:07:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.242+0000 7f3fab063740 1 Processor -- start 2024-09-21T06:07:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.242+0000 7f3fab063740 1 -- start start 2024-09-21T06:07:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.242+0000 7f3f92ffd640 1 -- 172.21.15.117:0/1081455454 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f3f8c042030 con 0x7f3f9c108300 2024-09-21T06:07:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.242+0000 7f3fab063740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de4cc91b0 0x561de506e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.242+0000 7f3fab063740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de4ca02f0 0x561de506eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.242+0000 7f3fab063740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de4ca84d0 0x561de506f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.242+0000 7f3fab063740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561de4fe36f0 con 0x561de4ca02f0 2024-09-21T06:07:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.242+0000 7f3fab063740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561de4fd2b20 con 0x561de4ca84d0 2024-09-21T06:07:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.242+0000 7f3fab063740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x561de4fd6930 con 0x561de4cc91b0 2024-09-21T06:07:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.242+0000 7f3fa1db9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de4ca02f0 0x561de506eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.242+0000 7f3fa1db9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de4ca02f0 0x561de506eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60500/0 (socket says 172.21.15.117:60500) 2024-09-21T06:07:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.242+0000 7f3fa1db9640 1 -- 172.21.15.117:0/2277534324 learned_addr learned my addr 172.21.15.117:0/2277534324 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.242+0000 7f3fa15b8640 1 --2- 172.21.15.117:0/2277534324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de4ca84d0 0x561de506f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.242+0000 7f3fa25ba640 1 --2- 172.21.15.117:0/2277534324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de4cc91b0 0x561de506e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.242+0000 7f3fa1db9640 1 -- 172.21.15.117:0/2277534324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de4cc91b0 msgr2=0x561de506e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.243+0000 7f3fa1db9640 1 --2- 172.21.15.117:0/2277534324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de4cc91b0 0x561de506e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.243+0000 7f3fa1db9640 1 -- 172.21.15.117:0/2277534324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de4ca84d0 msgr2=0x561de506f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.243+0000 7f3fa1db9640 1 --2- 172.21.15.117:0/2277534324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de4ca84d0 0x561de506f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.243+0000 7f3fa1db9640 1 -- 172.21.15.117:0/2277534324 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561de4f98530 con 0x561de4ca02f0 2024-09-21T06:07:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.243+0000 7f3f92ffd640 1 --2- 172.21.15.117:0/1081455454 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3f880b6680 0x7f3f880b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.243+0000 7f3fa1db9640 1 --2- 172.21.15.117:0/2277534324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de4ca02f0 0x561de506eef0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f3f98009ca0 tx=0x7f3f98011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32af152a3eab480 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.243+0000 7f3fa25ba640 1 --2- 172.21.15.117:0/1081455454 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3f880b6680 0x7f3f880b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.243+0000 7f3f87fff640 1 -- 172.21.15.117:0/2277534324 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f98005220 con 0x561de4ca02f0 2024-09-21T06:07:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.243+0000 7f3f92ffd640 1 -- 172.21.15.117:0/1081455454 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8485+0+0 (secure 0 0 0) 0x7f3f8c107700 con 0x7f3f9c108300 2024-09-21T06:07:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.243+0000 7f3f87fff640 1 -- 172.21.15.117:0/2277534324 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f98004c50 con 0x561de4ca02f0 2024-09-21T06:07:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.243+0000 7f3f87fff640 1 -- 172.21.15.117:0/2277534324 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f98019780 con 0x561de4ca02f0 2024-09-21T06:07:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.244+0000 7f3fab063740 1 -- 172.21.15.117:0/2277534324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de4ca02f0 msgr2=0x561de506eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.244+0000 7f3fa25ba640 1 --2- 172.21.15.117:0/1081455454 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3f880b6680 0x7f3f880b8b90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3f94005fd0 tx=0x7f3f94009b40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.244+0000 7f3fab063740 1 --2- 172.21.15.117:0/2277534324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de4ca02f0 0x561de506eef0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f3f98009ca0 tx=0x7f3f98011810 comp rx=0 tx=0).stop 2024-09-21T06:07:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.244+0000 7f3fab063740 1 -- 172.21.15.117:0/2277534324 shutdown_connections 2024-09-21T06:07:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.244+0000 7f3fab063740 1 --2- 172.21.15.117:0/2277534324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de4ca02f0 0x561de506eef0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.244+0000 7f3fab063740 1 --2- 172.21.15.117:0/2277534324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de4ca84d0 0x561de506f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.244+0000 7f3fab063740 1 --2- 172.21.15.117:0/2277534324 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de4cc91b0 0x561de506e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.244+0000 7f3fab063740 1 -- 172.21.15.117:0/2277534324 >> 172.21.15.117:0/2277534324 conn(0x561de4cbd300 msgr2=0x561de506c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.244+0000 7f3fab063740 1 -- 172.21.15.117:0/2277534324 shutdown_connections 2024-09-21T06:07:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.244+0000 7f3fab063740 1 -- 172.21.15.117:0/2277534324 wait complete. 2024-09-21T06:07:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.244+0000 7f3fab063740 1 -- start start 2024-09-21T06:07:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.245+0000 7f3fab063740 1 Processor -- start 2024-09-21T06:07:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.245+0000 7f3fab063740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de4ca02f0 0x561de507a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.245+0000 7f3fab063740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de4ca84d0 0x561de507a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.245+0000 7f3fab063740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de4cc91b0 0x561de5086420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.245+0000 7f3fab063740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561de4f0aab0 con 0x561de4ca84d0 2024-09-21T06:07:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.245+0000 7f3fab063740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561de4f78730 con 0x561de4ca02f0 2024-09-21T06:07:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.245+0000 7f3fab063740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x561de507d130 con 0x561de4cc91b0 2024-09-21T06:07:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.245+0000 7f3fab063740 10 client.0 _ll_register_callbacks cb 0x561de4fcc6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.245+0000 7f3fa15b8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de4ca84d0 0x561de507a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.245+0000 7f3fa15b8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de4ca84d0 0x561de507a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60502/0 (socket says 172.21.15.117:60502) 2024-09-21T06:07:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.245+0000 7f3fa15b8640 1 -- 172.21.15.117:0/874090154 learned_addr learned my addr 172.21.15.117:0/874090154 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.245+0000 7f3fa1db9640 1 --2- 172.21.15.117:0/874090154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de4ca02f0 0x561de507a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.245+0000 7f3fa15b8640 1 -- 172.21.15.117:0/874090154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de4cc91b0 msgr2=0x561de5086420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.245+0000 7f3fa15b8640 1 --2- 172.21.15.117:0/874090154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de4cc91b0 0x561de5086420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.245+0000 7f3fa15b8640 1 -- 172.21.15.117:0/874090154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de4ca02f0 msgr2=0x561de507a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.245+0000 7f3fa15b8640 1 --2- 172.21.15.117:0/874090154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de4ca02f0 0x561de507a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.245+0000 7f3fa15b8640 1 -- 172.21.15.117:0/874090154 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561de4d34790 con 0x561de4ca84d0 2024-09-21T06:07:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.245+0000 7f3fa1db9640 1 --2- 172.21.15.117:0/874090154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de4ca02f0 0x561de507a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.246+0000 7f3fa15b8640 1 --2- 172.21.15.117:0/874090154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de4ca84d0 0x561de507a5c0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c104ed0 tx=0x7f3f8c003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c377760ea1f7339 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.246+0000 7f3f857fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.246+0000 7f3f857fa640 1 -- 172.21.15.117:0/874090154 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c049040 con 0x561de4ca84d0 2024-09-21T06:07:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.246+0000 7f3f857fa640 1 -- 172.21.15.117:0/874090154 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8485+0+0 (secure 0 0 0) 0x7f3f8c0d6d00 con 0x561de4ca84d0 2024-09-21T06:07:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.246+0000 7f3fab063740 1 -- 172.21.15.117:0/874090154 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f98004c50 con 0x561de4ca84d0 2024-09-21T06:07:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.246+0000 7f3f857fa640 1 -- 172.21.15.117:0/874090154 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3f8c0ccec0 con 0x561de4ca84d0 2024-09-21T06:07:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.246+0000 7f3fab063740 10 client.17535 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.246+0000 7f3fab063740 1 -- 172.21.15.117:0/874090154 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561de4dad620 con 0x561de4ca84d0 2024-09-21T06:07:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.246+0000 7f3f857fa640 1 -- 172.21.15.117:0/874090154 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f98004c50 con 0x561de4ca84d0 2024-09-21T06:07:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.246+0000 7f3f857fa640 1 -- 172.21.15.117:0/874090154 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3f8c107bb0 con 0x561de4ca84d0 2024-09-21T06:07:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.246+0000 7f3fab063740 10 client.17535 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.246+0000 7f3fab063740 10 client.17535 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:07:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.246+0000 7f3fab063740 20 client.17535 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.246+0000 7f3fab063740 1 --2- 172.21.15.117:0/874090154 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x561de4c87d80 0x561de508c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.246+0000 7f3fab063740 4 client.17535 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:07:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.246+0000 7f3fab063740 1 -- 172.21.15.117:0/874090154 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561de4fe5640 con 0x561de4c87d80 2024-09-21T06:07:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.247+0000 7f3fa1db9640 1 --2- 172.21.15.117:0/874090154 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x561de4c87d80 0x561de508c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.247+0000 7f3f857fa640 10 client.17535 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.247+0000 7f3fa1db9640 1 --2- 172.21.15.117:0/874090154 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x561de4c87d80 0x561de508c1a0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1933830524f00b98 server_cookie=23ad830f9a3978e0 in_seq=0 out_seq=0 2024-09-21T06:07:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.248+0000 7f3f857fa640 1 -- 172.21.15.117:0/874090154 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561de4fe5640 con 0x561de4c87d80 2024-09-21T06:07:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.248+0000 7f3f857fa640 10 client.17535 handle_command_reply: tid=0 2024-09-21T06:07:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.248+0000 7f3fab063740 1 client.17535 shutdown 2024-09-21T06:07:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.248+0000 7f3fab063740 1 -- 172.21.15.117:0/874090154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de4ca84d0 msgr2=0x561de507a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.248+0000 7f3fab063740 1 --2- 172.21.15.117:0/874090154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de4ca84d0 0x561de507a5c0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c104ed0 tx=0x7f3f8c003f60 comp rx=0 tx=0).stop 2024-09-21T06:07:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.248+0000 7f3fab063740 1 -- 172.21.15.117:0/874090154 shutdown_connections 2024-09-21T06:07:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.248+0000 7f3fab063740 1 --2- 172.21.15.117:0/874090154 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x561de4c87d80 0x561de508c1a0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.248+0000 7f3fab063740 1 --2- 172.21.15.117:0/874090154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de4ca84d0 0x561de507a5c0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.248+0000 7f3fab063740 1 --2- 172.21.15.117:0/874090154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de4cc91b0 0x561de5086420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.248+0000 7f3f857fa640 0 client.17535 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.248+0000 7f3fab063740 1 --2- 172.21.15.117:0/874090154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de4ca02f0 0x561de507a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.248+0000 7f3fab063740 1 -- 172.21.15.117:0/874090154 >> 172.21.15.117:0/874090154 conn(0x561de4cbd300 msgr2=0x561de506f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.248+0000 7f3fab063740 1 -- 172.21.15.117:0/874090154 shutdown_connections 2024-09-21T06:07:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.248+0000 7f3fab063740 1 -- 172.21.15.117:0/874090154 wait complete. 2024-09-21T06:07:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.248+0000 7f3fab063740 20 client.17535 trim_cache size 0 max 16384 2024-09-21T06:07:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.262+0000 7f3fab063740 1 Processor -- start 2024-09-21T06:07:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.263+0000 7f3fab063740 1 -- start start 2024-09-21T06:07:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.263+0000 7f3fab063740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de4c87d80 0x561de50703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.263+0000 7f3fab063740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de4ca02f0 0x561de507bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.263+0000 7f3fab063740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de4ca84d0 0x561de5077250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.263+0000 7f3fab063740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9c1af630 con 0x561de4ca84d0 2024-09-21T06:07:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.263+0000 7f3fab063740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9c1b1170 con 0x561de4c87d80 2024-09-21T06:07:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.263+0000 7f3fab063740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9c1afd00 con 0x561de4ca02f0 2024-09-21T06:07:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.263+0000 7f3fa25ba640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de4ca84d0 0x561de5077250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.263+0000 7f3fa25ba640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de4ca84d0 0x561de5077250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60506/0 (socket says 172.21.15.117:60506) 2024-09-21T06:07:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.263+0000 7f3fa25ba640 1 -- 172.21.15.117:0/1899713419 learned_addr learned my addr 172.21.15.117:0/1899713419 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.263+0000 7f3fa1db9640 1 --2- 172.21.15.117:0/1899713419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de4c87d80 0x561de50703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.263+0000 7f3fa15b8640 1 --2- 172.21.15.117:0/1899713419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de4ca02f0 0x561de507bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.263+0000 7f3fa25ba640 1 -- 172.21.15.117:0/1899713419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de4ca02f0 msgr2=0x561de507bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.263+0000 7f3fa25ba640 1 --2- 172.21.15.117:0/1899713419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de4ca02f0 0x561de507bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.263+0000 7f3fa25ba640 1 -- 172.21.15.117:0/1899713419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de4c87d80 msgr2=0x561de50703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.263+0000 7f3fa25ba640 1 --2- 172.21.15.117:0/1899713419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de4c87d80 0x561de50703a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.263+0000 7f3fa25ba640 1 -- 172.21.15.117:0/1899713419 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561de507b340 con 0x561de4ca84d0 2024-09-21T06:07:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.263+0000 7f3fa15b8640 1 --2- 172.21.15.117:0/1899713419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de4ca02f0 0x561de507bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:07:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.263+0000 7f3fa25ba640 1 --2- 172.21.15.117:0/1899713419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de4ca84d0 0x561de5077250 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f3f94011700 tx=0x7f3f94013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1bc4d7a1c178cf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.263+0000 7f3f877fe640 1 -- 172.21.15.117:0/1899713419 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9400e780 con 0x561de4ca84d0 2024-09-21T06:07:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.264+0000 7f3f877fe640 1 -- 172.21.15.117:0/1899713419 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f9400edd0 con 0x561de4ca84d0 2024-09-21T06:07:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.264+0000 7f3f877fe640 1 -- 172.21.15.117:0/1899713419 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f94026400 con 0x561de4ca84d0 2024-09-21T06:07:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.264+0000 7f3fab063740 1 -- 172.21.15.117:0/1899713419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de4ca84d0 msgr2=0x561de5077250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.264+0000 7f3fab063740 1 --2- 172.21.15.117:0/1899713419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de4ca84d0 0x561de5077250 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f3f94011700 tx=0x7f3f94013070 comp rx=0 tx=0).stop 2024-09-21T06:07:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.264+0000 7f3fab063740 1 -- 172.21.15.117:0/1899713419 shutdown_connections 2024-09-21T06:07:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.264+0000 7f3fab063740 1 --2- 172.21.15.117:0/1899713419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de4ca84d0 0x561de5077250 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.264+0000 7f3fab063740 1 --2- 172.21.15.117:0/1899713419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de4ca02f0 0x561de507bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.264+0000 7f3fab063740 1 --2- 172.21.15.117:0/1899713419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de4c87d80 0x561de50703a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.264+0000 7f3fab063740 1 -- 172.21.15.117:0/1899713419 >> 172.21.15.117:0/1899713419 conn(0x561de4cbd300 msgr2=0x561de507b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.264+0000 7f3fab063740 1 -- 172.21.15.117:0/1899713419 shutdown_connections 2024-09-21T06:07:12.269 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.264+0000 7f3fab063740 1 -- 172.21.15.117:0/1899713419 wait complete. 2024-09-21T06:07:12.269 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.265+0000 7f3fab063740 1 -- start start 2024-09-21T06:07:12.269 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.265+0000 7f3fab063740 1 Processor -- start 2024-09-21T06:07:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.265+0000 7f3fab063740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de4c87d80 0x561de5082880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.265+0000 7f3fab063740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de4ca02f0 0x561de507fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.265+0000 7f3fab063740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de4ca84d0 0x561de5080280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.265+0000 7f3fab063740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9c1b0aa0 con 0x561de4ca84d0 2024-09-21T06:07:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.265+0000 7f3fab063740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9c1b03d0 con 0x561de4c87d80 2024-09-21T06:07:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.265+0000 7f3fab063740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9c1ad630 con 0x561de4ca02f0 2024-09-21T06:07:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.265+0000 7f3fab063740 10 client.0 _ll_register_callbacks cb 0x7f3f8c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.265+0000 7f3fa25ba640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de4ca84d0 0x561de5080280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.265+0000 7f3fa25ba640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de4ca84d0 0x561de5080280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60520/0 (socket says 172.21.15.117:60520) 2024-09-21T06:07:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.265+0000 7f3fa25ba640 1 -- 172.21.15.117:0/3667136123 learned_addr learned my addr 172.21.15.117:0/3667136123 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.265+0000 7f3fa1db9640 1 --2- 172.21.15.117:0/3667136123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de4c87d80 0x561de5082880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.265+0000 7f3fa15b8640 1 --2- 172.21.15.117:0/3667136123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de4ca02f0 0x561de507fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.266+0000 7f3fa25ba640 1 -- 172.21.15.117:0/3667136123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de4ca02f0 msgr2=0x561de507fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.266+0000 7f3fa25ba640 1 --2- 172.21.15.117:0/3667136123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de4ca02f0 0x561de507fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.266+0000 7f3fa25ba640 1 -- 172.21.15.117:0/3667136123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de4c87d80 msgr2=0x561de5082880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.266+0000 7f3fa25ba640 1 --2- 172.21.15.117:0/3667136123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de4c87d80 0x561de5082880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.266+0000 7f3fa25ba640 1 -- 172.21.15.117:0/3667136123 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561de50807c0 con 0x561de4ca84d0 2024-09-21T06:07:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.266+0000 7f3fa25ba640 1 --2- 172.21.15.117:0/3667136123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de4ca84d0 0x561de5080280 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f3f94004a80 tx=0x7f3f9401e650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e985663243dee21f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.266+0000 7f3f857fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.266+0000 7f3f857fa640 1 -- 172.21.15.117:0/3667136123 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9402a030 con 0x561de4ca84d0 2024-09-21T06:07:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.266+0000 7f3f857fa640 1 -- 172.21.15.117:0/3667136123 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8485+0+0 (secure 0 0 0) 0x7f3f9400e780 con 0x561de4ca84d0 2024-09-21T06:07:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.266+0000 7f3fab063740 1 -- 172.21.15.117:0/3667136123 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f9400edd0 con 0x561de4ca84d0 2024-09-21T06:07:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.266+0000 7f3f857fa640 1 -- 172.21.15.117:0/3667136123 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3f9400e9a0 con 0x561de4ca84d0 2024-09-21T06:07:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.266+0000 7f3fab063740 10 client.17541 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.266+0000 7f3fab063740 1 -- 172.21.15.117:0/3667136123 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561de50f6730 con 0x561de4ca84d0 2024-09-21T06:07:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.266+0000 7f3f857fa640 1 -- 172.21.15.117:0/3667136123 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f9400edd0 con 0x561de4ca84d0 2024-09-21T06:07:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.266+0000 7f3f857fa640 1 -- 172.21.15.117:0/3667136123 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3f94038070 con 0x561de4ca84d0 2024-09-21T06:07:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.266+0000 7f3fab063740 10 client.17541 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.266+0000 7f3fab063740 10 client.17541 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:07:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.266+0000 7f3fab063740 20 client.17541 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.266+0000 7f3fab063740 1 --2- 172.21.15.117:0/3667136123 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x561de4cc91b0 0x561de50f8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.266+0000 7f3fab063740 4 client.17541 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:07:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.266+0000 7f3fab063740 1 -- 172.21.15.117:0/3667136123 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561de4fe10f0 con 0x561de4cc91b0 2024-09-21T06:07:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.267+0000 7f3fa1db9640 1 --2- 172.21.15.117:0/3667136123 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x561de4cc91b0 0x561de50f8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.267+0000 7f3f857fa640 10 client.17541 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.267+0000 7f3fa1db9640 1 --2- 172.21.15.117:0/3667136123 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x561de4cc91b0 0x561de50f8e30 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8159e1aee279d82a server_cookie=de40387b1b7ab91 in_seq=0 out_seq=0 2024-09-21T06:07:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.273+0000 7f3f857fa640 1 -- 172.21.15.117:0/3667136123 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561de4fe10f0 con 0x561de4cc91b0 2024-09-21T06:07:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.273+0000 7f3f857fa640 10 client.17541 handle_command_reply: tid=0 2024-09-21T06:07:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.273+0000 7f3fab063740 1 client.17541 shutdown 2024-09-21T06:07:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.273+0000 7f3fab063740 1 -- 172.21.15.117:0/3667136123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de4ca84d0 msgr2=0x561de5080280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.273+0000 7f3fab063740 1 --2- 172.21.15.117:0/3667136123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de4ca84d0 0x561de5080280 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f3f94004a80 tx=0x7f3f9401e650 comp rx=0 tx=0).stop 2024-09-21T06:07:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.273+0000 7f3fab063740 1 -- 172.21.15.117:0/3667136123 shutdown_connections 2024-09-21T06:07:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.273+0000 7f3fab063740 1 --2- 172.21.15.117:0/3667136123 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x561de4cc91b0 0x561de50f8e30 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.273+0000 7f3fab063740 1 --2- 172.21.15.117:0/3667136123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561de4ca84d0 0x561de5080280 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.273+0000 7f3f857fa640 0 client.17541 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.273+0000 7f3fab063740 1 --2- 172.21.15.117:0/3667136123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561de4ca02f0 0x561de507fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.273+0000 7f3fab063740 1 --2- 172.21.15.117:0/3667136123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561de4c87d80 0x561de5082880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.273+0000 7f3fab063740 1 -- 172.21.15.117:0/3667136123 >> 172.21.15.117:0/3667136123 conn(0x561de4cbd300 msgr2=0x561de5070150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.273+0000 7f3fab063740 1 -- 172.21.15.117:0/3667136123 shutdown_connections 2024-09-21T06:07:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.274+0000 7f3fab063740 1 -- 172.21.15.117:0/3667136123 wait complete. 2024-09-21T06:07:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.274+0000 7f3fab063740 20 client.17541 trim_cache size 0 max 16384 2024-09-21T06:07:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T06:07:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T06:07:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T06:07:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T06:07:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.274+0000 7f3fa401d640 1 -- 172.21.15.117:0/1081455454 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3f880b6680 msgr2=0x7f3f880b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.274+0000 7f3fa401d640 1 --2- 172.21.15.117:0/1081455454 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3f880b6680 0x7f3f880b8b90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3f94005fd0 tx=0x7f3f94009b40 comp rx=0 tx=0).stop 2024-09-21T06:07:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.274+0000 7f3fa401d640 1 -- 172.21.15.117:0/1081455454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f9c108300 msgr2=0x7f3f9c19b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.274+0000 7f3fa401d640 1 --2- 172.21.15.117:0/1081455454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f9c108300 0x7f3f9c19b740 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c02a930 tx=0x7f3f8c031a20 comp rx=0 tx=0).stop 2024-09-21T06:07:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.274+0000 7f3fa401d640 1 -- 172.21.15.117:0/1081455454 shutdown_connections 2024-09-21T06:07:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.274+0000 7f3fa401d640 1 --2- 172.21.15.117:0/1081455454 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f3f880b6680 0x7f3f880b8b90 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.274+0000 7f3fa401d640 1 --2- 172.21.15.117:0/1081455454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f9c108300 0x7f3f9c19b740 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.279 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.274+0000 7f3fa401d640 1 --2- 172.21.15.117:0/1081455454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f9c108c50 0x7f3f9c19bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.279 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.274+0000 7f3fa401d640 1 --2- 172.21.15.117:0/1081455454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f9c1029d0 0x7f3f9c19b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:12.279 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.274+0000 7f3fa401d640 1 -- 172.21.15.117:0/1081455454 >> 172.21.15.117:0/1081455454 conn(0x7f3f9c0fe410 msgr2=0x7f3f9c100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:12.279 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.274+0000 7f3fa401d640 1 -- 172.21.15.117:0/1081455454 shutdown_connections 2024-09-21T06:07:12.279 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:12.274+0000 7f3fa401d640 1 -- 172.21.15.117:0/1081455454 wait complete. 2024-09-21T06:07:12.279 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:11 smithi117 ceph-mon[30947]: pgmap v629: 97 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 89 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-21T06:07:12.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:11 smithi138 ceph-mon[34391]: pgmap v629: 97 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 89 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-21T06:07:12.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:11 smithi192 ceph-mon[34700]: pgmap v629: 97 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 89 active+clean; 50 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-21T06:07:14.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:13 smithi117 ceph-mon[30947]: pgmap v630: 97 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 89 active+clean; 46 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-21T06:07:14.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:13 smithi138 ceph-mon[34391]: pgmap v630: 97 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 89 active+clean; 46 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-21T06:07:14.479 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:13 smithi192 ceph-mon[34700]: pgmap v630: 97 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 89 active+clean; 46 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-21T06:07:15.170 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:07:15.432 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:07:16.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.150+0000 7f43a51b6640 1 -- 172.21.15.117:0/2539797171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43a0108cf0 msgr2=0x7f43a010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:16.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.150+0000 7f43a51b6640 1 --2- 172.21.15.117:0/2539797171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43a0108cf0 0x7f43a010f520 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f4398009ca0 tx=0x7f439802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:07:16.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.150+0000 7f43a51b6640 1 -- 172.21.15.117:0/2539797171 shutdown_connections 2024-09-21T06:07:16.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.150+0000 7f43a51b6640 1 --2- 172.21.15.117:0/2539797171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43a0108cf0 0x7f43a010f520 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:16.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.150+0000 7f43a51b6640 1 --2- 172.21.15.117:0/2539797171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43a01083a0 0x7f43a01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:16.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.150+0000 7f43a51b6640 1 --2- 172.21.15.117:0/2539797171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43a0102a60 0x7f43a0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:16.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.150+0000 7f43a51b6640 1 -- 172.21.15.117:0/2539797171 >> 172.21.15.117:0/2539797171 conn(0x7f43a0079730 msgr2=0x7f43a0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:16.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.150+0000 7f43a51b6640 1 -- 172.21.15.117:0/2539797171 shutdown_connections 2024-09-21T06:07:16.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.150+0000 7f43a51b6640 1 -- 172.21.15.117:0/2539797171 wait complete. 2024-09-21T06:07:16.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.151+0000 7f43a51b6640 1 Processor -- start 2024-09-21T06:07:16.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.151+0000 7f43a51b6640 1 -- start start 2024-09-21T06:07:16.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.152+0000 7f43a51b6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43a0102a60 0x7f43a019b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:16.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.152+0000 7f43a51b6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43a01083a0 0x7f43a019b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:16.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.152+0000 7f43a51b6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43a0108cf0 0x7f43a019bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:16.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.152+0000 7f43a51b6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f43a0112960 con 0x7f43a0108cf0 2024-09-21T06:07:16.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.152+0000 7f43a51b6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f43a01127e0 con 0x7f43a01083a0 2024-09-21T06:07:16.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.152+0000 7f43a51b6640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f43a0112ae0 con 0x7f43a0102a60 2024-09-21T06:07:16.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.152+0000 7f439f59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43a0108cf0 0x7f43a019bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:16.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.152+0000 7f439f59e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43a0108cf0 0x7f43a019bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44234/0 (socket says 172.21.15.117:44234) 2024-09-21T06:07:16.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.152+0000 7f439f59e640 1 -- 172.21.15.117:0/3828128058 learned_addr learned my addr 172.21.15.117:0/3828128058 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:16.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.152+0000 7f4397fff640 1 --2- 172.21.15.117:0/3828128058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43a01083a0 0x7f43a019b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:16.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.152+0000 7f439ed9d640 1 --2- 172.21.15.117:0/3828128058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43a0102a60 0x7f43a019b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:16.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.152+0000 7f439f59e640 1 -- 172.21.15.117:0/3828128058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43a0102a60 msgr2=0x7f43a019b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:16.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.152+0000 7f439f59e640 1 --2- 172.21.15.117:0/3828128058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43a0102a60 0x7f43a019b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:16.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.152+0000 7f439f59e640 1 -- 172.21.15.117:0/3828128058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43a01083a0 msgr2=0x7f43a019b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:16.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.152+0000 7f439f59e640 1 --2- 172.21.15.117:0/3828128058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43a01083a0 0x7f43a019b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:16.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.152+0000 7f439f59e640 1 -- 172.21.15.117:0/3828128058 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a01a6490 con 0x7f43a0108cf0 2024-09-21T06:07:16.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.152+0000 7f439f59e640 1 --2- 172.21.15.117:0/3828128058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43a0108cf0 0x7f43a019bc90 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f4398005cc0 tx=0x7f439802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:16.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.152+0000 7f439cd99640 1 -- 172.21.15.117:0/3828128058 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4398003bf0 con 0x7f43a0108cf0 2024-09-21T06:07:16.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.152+0000 7f43a51b6640 1 -- 172.21.15.117:0/3828128058 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43a01a6720 con 0x7f43a0108cf0 2024-09-21T06:07:16.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.152+0000 7f439cd99640 1 -- 172.21.15.117:0/3828128058 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4398003d90 con 0x7f43a0108cf0 2024-09-21T06:07:16.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.152+0000 7f439cd99640 1 -- 172.21.15.117:0/3828128058 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439802fbc0 con 0x7f43a0108cf0 2024-09-21T06:07:16.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.153+0000 7f43a51b6640 1 -- 172.21.15.117:0/3828128058 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43a01a6cb0 con 0x7f43a0108cf0 2024-09-21T06:07:16.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.155+0000 7f43ab72a740 1 Processor -- start 2024-09-21T06:07:16.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.155+0000 7f439cd99640 1 -- 172.21.15.117:0/3828128058 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f439802fd60 con 0x7f43a0108cf0 2024-09-21T06:07:16.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.155+0000 7f43ab72a740 1 -- start start 2024-09-21T06:07:16.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.156+0000 7f43ab72a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c678db8070 0x55c67915f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:16.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.156+0000 7f43ab72a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c678d8f1b0 0x55c67915fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:16.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.156+0000 7f43ab72a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c678d97390 0x55c6791600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:16.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.156+0000 7f43ab72a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c6790d5e50 con 0x55c678db8070 2024-09-21T06:07:16.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.156+0000 7f43ab72a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c6790c5280 con 0x55c678d8f1b0 2024-09-21T06:07:16.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.156+0000 7f43ab72a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c6790c9090 con 0x55c678d97390 2024-09-21T06:07:16.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.156+0000 7f4397fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c678db8070 0x55c67915f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:16.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.156+0000 7f4397fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c678db8070 0x55c67915f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44236/0 (socket says 172.21.15.117:44236) 2024-09-21T06:07:16.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.156+0000 7f4397fff640 1 -- 172.21.15.117:0/3146826829 learned_addr learned my addr 172.21.15.117:0/3146826829 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:16.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.156+0000 7f439ed9d640 1 --2- 172.21.15.117:0/3146826829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c678d8f1b0 0x55c67915fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:16.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.156+0000 7f4397fff640 1 --2- 172.21.15.117:0/3146826829 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c678d97390 0x55c6791600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:16.159 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.156+0000 7f439cd99640 1 --2- 172.21.15.117:0/3828128058 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f438c0b6700 0x7f438c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:16.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.156+0000 7f439ed9d640 1 -- 172.21.15.117:0/3146826829 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c678d97390 msgr2=0x55c6791600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:16.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.156+0000 7f439ed9d640 1 --2- 172.21.15.117:0/3146826829 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c678d97390 0x55c6791600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:16.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.156+0000 7f439ed9d640 1 -- 172.21.15.117:0/3146826829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c678db8070 msgr2=0x55c67915f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:16.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.156+0000 7f439ed9d640 1 --2- 172.21.15.117:0/3146826829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c678db8070 0x55c67915f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:16.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.156+0000 7f439ed9d640 1 -- 172.21.15.117:0/3146826829 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c678e23650 con 0x55c678d8f1b0 2024-09-21T06:07:16.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.156+0000 7f4397fff640 1 --2- 172.21.15.117:0/3146826829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c678db8070 0x55c67915f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:07:16.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.156+0000 7f439f59e640 1 --2- 172.21.15.117:0/3828128058 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f438c0b6700 0x7f438c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:16.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.156+0000 7f439cd99640 1 -- 172.21.15.117:0/3828128058 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8485+0+0 (secure 0 0 0) 0x7f4398108c30 con 0x7f43a0108cf0 2024-09-21T06:07:16.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.157+0000 7f439ed9d640 1 --2- 172.21.15.117:0/3146826829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c678d8f1b0 0x55c67915fb90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4390005fd0 tx=0x7f439000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b10160a4dedd2173 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:16.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.157+0000 7f43957fa640 1 -- 172.21.15.117:0/3146826829 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4390015070 con 0x55c678d8f1b0 2024-09-21T06:07:16.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.157+0000 7f439f59e640 1 --2- 172.21.15.117:0/3828128058 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f438c0b6700 0x7f438c0b8c10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4398106450 tx=0x7f439809be90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:16.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.157+0000 7f43957fa640 1 -- 172.21.15.117:0/3146826829 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43900054d0 con 0x55c678d8f1b0 2024-09-21T06:07:16.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.157+0000 7f43957fa640 1 -- 172.21.15.117:0/3146826829 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4390011440 con 0x55c678d8f1b0 2024-09-21T06:07:16.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.158+0000 7f43ab72a740 1 -- 172.21.15.117:0/3146826829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c678d8f1b0 msgr2=0x55c67915fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:16.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.158+0000 7f43ab72a740 1 --2- 172.21.15.117:0/3146826829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c678d8f1b0 0x55c67915fb90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4390005fd0 tx=0x7f439000f730 comp rx=0 tx=0).stop 2024-09-21T06:07:16.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.158+0000 7f43ab72a740 1 -- 172.21.15.117:0/3146826829 shutdown_connections 2024-09-21T06:07:16.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.158+0000 7f43ab72a740 1 --2- 172.21.15.117:0/3146826829 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c678d97390 0x55c6791600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:16.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.158+0000 7f43ab72a740 1 --2- 172.21.15.117:0/3146826829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c678d8f1b0 0x55c67915fb90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:16.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.158+0000 7f43ab72a740 1 --2- 172.21.15.117:0/3146826829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c678db8070 0x55c67915f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:16.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.158+0000 7f43ab72a740 1 -- 172.21.15.117:0/3146826829 >> 172.21.15.117:0/3146826829 conn(0x55c678dac460 msgr2=0x55c67915d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:16.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.158+0000 7f43ab72a740 1 -- 172.21.15.117:0/3146826829 shutdown_connections 2024-09-21T06:07:16.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.158+0000 7f43ab72a740 1 -- 172.21.15.117:0/3146826829 wait complete. 2024-09-21T06:07:16.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.158+0000 7f43ab72a740 1 -- start start 2024-09-21T06:07:16.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.158+0000 7f43ab72a740 1 Processor -- start 2024-09-21T06:07:16.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.159+0000 7f43ab72a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c678d8f1b0 0x55c67916ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:16.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.159+0000 7f43ab72a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c678d97390 0x55c67916b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:16.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.159+0000 7f43ab72a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c678db8070 0x55c679177120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:16.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.159+0000 7f43ab72a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c678f26700 con 0x55c678d8f1b0 2024-09-21T06:07:16.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.159+0000 7f43ab72a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c679060110 con 0x55c678d97390 2024-09-21T06:07:16.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.159+0000 7f43ab72a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c67916de30 con 0x55c678db8070 2024-09-21T06:07:16.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.159+0000 7f43ab72a740 10 client.0 _ll_register_callbacks cb 0x55c679079990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:16.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.159+0000 7f4397fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c678d8f1b0 0x55c67916ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:16.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.159+0000 7f4397fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c678d8f1b0 0x55c67916ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44240/0 (socket says 172.21.15.117:44240) 2024-09-21T06:07:16.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.159+0000 7f4397fff640 1 -- 172.21.15.117:0/2371301289 learned_addr learned my addr 172.21.15.117:0/2371301289 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:16.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.159+0000 7f439ed9d640 1 --2- 172.21.15.117:0/2371301289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c678d97390 0x55c67916b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:16.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.159+0000 7f4397fff640 1 --2- 172.21.15.117:0/2371301289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c678db8070 0x55c679177120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:16.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.159+0000 7f4397fff640 1 -- 172.21.15.117:0/2371301289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c678db8070 msgr2=0x55c679177120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:16.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.159+0000 7f4397fff640 1 --2- 172.21.15.117:0/2371301289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c678db8070 0x55c679177120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:16.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.159+0000 7f4397fff640 1 -- 172.21.15.117:0/2371301289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c678d97390 msgr2=0x55c67916b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:16.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.159+0000 7f4397fff640 1 --2- 172.21.15.117:0/2371301289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c678d97390 0x55c67916b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:16.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.159+0000 7f4397fff640 1 -- 172.21.15.117:0/2371301289 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c67908dc40 con 0x55c678d8f1b0 2024-09-21T06:07:16.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.159+0000 7f4397fff640 1 --2- 172.21.15.117:0/2371301289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c678d8f1b0 0x55c67916ad40 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f43880106c0 tx=0x7f438800ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3603c4968ab091b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:16.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.159+0000 7f43867fc640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:16.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.159+0000 7f43867fc640 1 -- 172.21.15.117:0/2371301289 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4388005220 con 0x55c678d8f1b0 2024-09-21T06:07:16.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.159+0000 7f43867fc640 1 -- 172.21.15.117:0/2371301289 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8485+0+0 (secure 0 0 0) 0x7f43880053c0 con 0x55c678d8f1b0 2024-09-21T06:07:16.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.159+0000 7f43ab72a740 1 -- 172.21.15.117:0/2371301289 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43900054d0 con 0x55c678d8f1b0 2024-09-21T06:07:16.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.160+0000 7f43867fc640 1 -- 172.21.15.117:0/2371301289 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f43880055c0 con 0x55c678d8f1b0 2024-09-21T06:07:16.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.160+0000 7f43ab72a740 10 client.17553 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:16.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.160+0000 7f43ab72a740 1 -- 172.21.15.117:0/2371301289 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c679179b70 con 0x55c678d8f1b0 2024-09-21T06:07:16.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.160+0000 7f43867fc640 1 -- 172.21.15.117:0/2371301289 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43900054d0 con 0x55c678d8f1b0 2024-09-21T06:07:16.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.160+0000 7f43867fc640 1 -- 172.21.15.117:0/2371301289 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4388027020 con 0x55c678d8f1b0 2024-09-21T06:07:16.166 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.160+0000 7f43ab72a740 10 client.17553 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:16.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.160+0000 7f43ab72a740 10 client.17553 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:07:16.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.160+0000 7f43ab72a740 20 client.17553 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:16.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.160+0000 7f43ab72a740 1 --2- 172.21.15.117:0/2371301289 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c678d769f0 0x55c67917cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:16.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.160+0000 7f43ab72a740 4 client.17553 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:07:16.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.160+0000 7f43ab72a740 1 -- 172.21.15.117:0/2371301289 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6790d7da0 con 0x55c678d769f0 2024-09-21T06:07:16.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.160+0000 7f439ed9d640 1 --2- 172.21.15.117:0/2371301289 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c678d769f0 0x55c67917cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:16.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.161+0000 7f43867fc640 10 client.17553 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:16.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.161+0000 7f439ed9d640 1 --2- 172.21.15.117:0/2371301289 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c678d769f0 0x55c67917cfb0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c1b35c40c634253 server_cookie=c95d4fa674be98ab in_seq=0 out_seq=0 2024-09-21T06:07:16.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.162+0000 7f43867fc640 1 -- 172.21.15.117:0/2371301289 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c6790d7da0 con 0x55c678d769f0 2024-09-21T06:07:16.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.162+0000 7f43867fc640 10 client.17553 handle_command_reply: tid=0 2024-09-21T06:07:16.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.162+0000 7f43ab72a740 1 client.17553 shutdown 2024-09-21T06:07:16.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.162+0000 7f43ab72a740 1 -- 172.21.15.117:0/2371301289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c678d8f1b0 msgr2=0x55c67916ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:16.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.162+0000 7f43ab72a740 1 --2- 172.21.15.117:0/2371301289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c678d8f1b0 0x55c67916ad40 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f43880106c0 tx=0x7f438800ead0 comp rx=0 tx=0).stop 2024-09-21T06:07:16.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.162+0000 7f43ab72a740 1 -- 172.21.15.117:0/2371301289 shutdown_connections 2024-09-21T06:07:16.167 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.162+0000 7f43ab72a740 1 --2- 172.21.15.117:0/2371301289 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c678d769f0 0x55c67917cfb0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:16.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.162+0000 7f43ab72a740 1 --2- 172.21.15.117:0/2371301289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c678db8070 0x55c679177120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:16.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.162+0000 7f43ab72a740 1 --2- 172.21.15.117:0/2371301289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c678d97390 0x55c67916b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:16.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.162+0000 7f43ab72a740 1 --2- 172.21.15.117:0/2371301289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c678d8f1b0 0x55c67916ad40 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:16.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.162+0000 7f43ab72a740 1 -- 172.21.15.117:0/2371301289 >> 172.21.15.117:0/2371301289 conn(0x55c678dac460 msgr2=0x55c679162830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:16.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.162+0000 7f43867fc640 0 client.17553 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:16.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.162+0000 7f43ab72a740 1 -- 172.21.15.117:0/2371301289 shutdown_connections 2024-09-21T06:07:16.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.162+0000 7f43ab72a740 1 -- 172.21.15.117:0/2371301289 wait complete. 2024-09-21T06:07:16.168 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.162+0000 7f43ab72a740 20 client.17553 trim_cache size 0 max 16384 2024-09-21T06:07:16.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.179+0000 7f43ab72a740 1 Processor -- start 2024-09-21T06:07:16.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.179+0000 7f43ab72a740 1 -- start start 2024-09-21T06:07:16.181 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:15 smithi117 ceph-mon[30947]: pgmap v631: 97 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 89 active+clean; 41 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 921 B/s wr, 11 op/s 2024-09-21T06:07:16.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.180+0000 7f43ab72a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c678d8f1b0 0x55c67916b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:16.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.180+0000 7f43ab72a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c678d97390 0x55c679173460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:16.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.180+0000 7f43ab72a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c678db8070 0x55c6791739a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:16.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.180+0000 7f43ab72a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f43a01af580 con 0x55c678d97390 2024-09-21T06:07:16.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.180+0000 7f43ab72a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f43a01b10c0 con 0x55c678db8070 2024-09-21T06:07:16.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.180+0000 7f43ab72a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f43a01afc50 con 0x55c678d8f1b0 2024-09-21T06:07:16.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.180+0000 7f439ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c678d97390 0x55c679173460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:16.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.180+0000 7f439ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c678d97390 0x55c679173460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44252/0 (socket says 172.21.15.117:44252) 2024-09-21T06:07:16.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.180+0000 7f439ed9d640 1 -- 172.21.15.117:0/459871145 learned_addr learned my addr 172.21.15.117:0/459871145 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:16.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.180+0000 7f4397fff640 1 --2- 172.21.15.117:0/459871145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c678db8070 0x55c6791739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:16.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.180+0000 7f4397fff640 1 --2- 172.21.15.117:0/459871145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c678d8f1b0 0x55c67916b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:16.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.180+0000 7f439ed9d640 1 -- 172.21.15.117:0/459871145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c678d8f1b0 msgr2=0x55c67916b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:16.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.180+0000 7f439ed9d640 1 --2- 172.21.15.117:0/459871145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c678d8f1b0 0x55c67916b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:16.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.180+0000 7f439ed9d640 1 -- 172.21.15.117:0/459871145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c678db8070 msgr2=0x55c6791739a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:16.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.180+0000 7f439ed9d640 1 --2- 172.21.15.117:0/459871145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c678db8070 0x55c6791739a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:16.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.180+0000 7f439ed9d640 1 -- 172.21.15.117:0/459871145 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c679161d80 con 0x55c678d97390 2024-09-21T06:07:16.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.180+0000 7f439ed9d640 1 --2- 172.21.15.117:0/459871145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c678d97390 0x55c679173460 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f4390005fd0 tx=0x7f4390004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e936b5c06a5362 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:16.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.181+0000 7f4394ff9640 1 -- 172.21.15.117:0/459871145 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4390030070 con 0x55c678d97390 2024-09-21T06:07:16.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.181+0000 7f4394ff9640 1 -- 172.21.15.117:0/459871145 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f439000fc60 con 0x55c678d97390 2024-09-21T06:07:16.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.181+0000 7f4394ff9640 1 -- 172.21.15.117:0/459871145 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4390015040 con 0x55c678d97390 2024-09-21T06:07:16.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.182+0000 7f43ab72a740 1 -- 172.21.15.117:0/459871145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c678d97390 msgr2=0x55c679173460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:16.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.182+0000 7f43ab72a740 1 --2- 172.21.15.117:0/459871145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c678d97390 0x55c679173460 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f4390005fd0 tx=0x7f4390004a40 comp rx=0 tx=0).stop 2024-09-21T06:07:16.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.182+0000 7f43ab72a740 1 -- 172.21.15.117:0/459871145 shutdown_connections 2024-09-21T06:07:16.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.182+0000 7f43ab72a740 1 --2- 172.21.15.117:0/459871145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c678d97390 0x55c679173460 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:16.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.182+0000 7f43ab72a740 1 --2- 172.21.15.117:0/459871145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c678db8070 0x55c6791739a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:16.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.182+0000 7f43ab72a740 1 --2- 172.21.15.117:0/459871145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c678d8f1b0 0x55c67916b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:16.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.182+0000 7f43ab72a740 1 -- 172.21.15.117:0/459871145 >> 172.21.15.117:0/459871145 conn(0x55c678dac460 msgr2=0x55c67916aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:16.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.182+0000 7f43ab72a740 1 -- 172.21.15.117:0/459871145 shutdown_connections 2024-09-21T06:07:16.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.182+0000 7f43ab72a740 1 -- 172.21.15.117:0/459871145 wait complete. 2024-09-21T06:07:16.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.182+0000 7f43ab72a740 1 -- start start 2024-09-21T06:07:16.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.182+0000 7f43ab72a740 1 Processor -- start 2024-09-21T06:07:16.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.182+0000 7f43ab72a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c678d8f1b0 0x55c6791dfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:16.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.182+0000 7f43ab72a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c678d97390 0x55c6791e0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:16.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.183+0000 7f43ab72a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c678db8070 0x55c6791e49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:16.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.183+0000 7f43ab72a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f43a01b09f0 con 0x55c678d8f1b0 2024-09-21T06:07:16.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.183+0000 7f43ab72a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f43a01b0320 con 0x55c678db8070 2024-09-21T06:07:16.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.183+0000 7f43ab72a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f43a01ad580 con 0x55c678d97390 2024-09-21T06:07:16.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.183+0000 7f43ab72a740 10 client.0 _ll_register_callbacks cb 0x7f4388013f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:16.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.183+0000 7f4397fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c678d8f1b0 0x55c6791dfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:16.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.183+0000 7f4397fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c678d8f1b0 0x55c6791dfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44266/0 (socket says 172.21.15.117:44266) 2024-09-21T06:07:16.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.183+0000 7f4397fff640 1 -- 172.21.15.117:0/2490111930 learned_addr learned my addr 172.21.15.117:0/2490111930 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:16.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.183+0000 7f439ed9d640 1 --2- 172.21.15.117:0/2490111930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c678d97390 0x55c6791e0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:16.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.183+0000 7f4397fff640 1 -- 172.21.15.117:0/2490111930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c678d97390 msgr2=0x55c6791e0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:16.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.183+0000 7f4397fff640 1 --2- 172.21.15.117:0/2490111930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c678d97390 0x55c6791e0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:16.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.183+0000 7f4397fff640 1 -- 172.21.15.117:0/2490111930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c678db8070 msgr2=0x55c6791e49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:16.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.183+0000 7f4397fff640 1 --2- 172.21.15.117:0/2490111930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c678db8070 0x55c6791e49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:16.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.183+0000 7f4397fff640 1 -- 172.21.15.117:0/2490111930 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6791e5000 con 0x55c678d8f1b0 2024-09-21T06:07:16.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.183+0000 7f4397fff640 1 --2- 172.21.15.117:0/2490111930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c678d8f1b0 0x55c6791dfb70 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f43880106c0 tx=0x7f43880058f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c426c66a8b8354 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:16.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.183+0000 7f43867fc640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:16.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.183+0000 7f43867fc640 1 -- 172.21.15.117:0/2490111930 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438803c020 con 0x55c678d8f1b0 2024-09-21T06:07:16.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.183+0000 7f43ab72a740 1 -- 172.21.15.117:0/2490111930 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f439000fc60 con 0x55c678d8f1b0 2024-09-21T06:07:16.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.184+0000 7f43867fc640 1 -- 172.21.15.117:0/2490111930 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8485+0+0 (secure 0 0 0) 0x7f4388021aa0 con 0x55c678d8f1b0 2024-09-21T06:07:16.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.184+0000 7f43867fc640 1 -- 172.21.15.117:0/2490111930 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f438800f730 con 0x55c678d8f1b0 2024-09-21T06:07:16.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.184+0000 7f43ab72a740 10 client.17559 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:16.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.184+0000 7f43ab72a740 1 -- 172.21.15.117:0/2490111930 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c6791ec340 con 0x55c678d8f1b0 2024-09-21T06:07:16.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.184+0000 7f43867fc640 1 -- 172.21.15.117:0/2490111930 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f439000fc60 con 0x55c678d8f1b0 2024-09-21T06:07:16.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.184+0000 7f43867fc640 1 -- 172.21.15.117:0/2490111930 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4388027040 con 0x55c678d8f1b0 2024-09-21T06:07:16.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.184+0000 7f43ab72a740 10 client.17559 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:16.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.184+0000 7f43ab72a740 10 client.17559 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:07:16.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.184+0000 7f43ab72a740 20 client.17559 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:16.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.184+0000 7f43ab72a740 1 --2- 172.21.15.117:0/2490111930 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c678d4bda0 0x55c6791eeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:16.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.184+0000 7f43ab72a740 4 client.17559 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:07:16.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.184+0000 7f43ab72a740 1 -- 172.21.15.117:0/2490111930 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c6790d3850 con 0x55c678d4bda0 2024-09-21T06:07:16.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.185+0000 7f439ed9d640 1 --2- 172.21.15.117:0/2490111930 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c678d4bda0 0x55c6791eeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:16.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.185+0000 7f439ed9d640 1 --2- 172.21.15.117:0/2490111930 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c678d4bda0 0x55c6791eeaa0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=825b28e554495273 server_cookie=bd5f316e73d2a4e1 in_seq=0 out_seq=0 2024-09-21T06:07:16.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.185+0000 7f43867fc640 10 client.17559 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:16.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.185+0000 7f43867fc640 1 -- 172.21.15.117:0/2490111930 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2930 (crc 0 0 0) 0x55c6790d3850 con 0x55c678d4bda0 2024-09-21T06:07:16.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.185+0000 7f43867fc640 10 client.17559 handle_command_reply: tid=0 2024-09-21T06:07:16.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.185+0000 7f43ab72a740 1 client.17559 shutdown 2024-09-21T06:07:16.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.185+0000 7f43ab72a740 1 -- 172.21.15.117:0/2490111930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c678d8f1b0 msgr2=0x55c6791dfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:16.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.185+0000 7f43ab72a740 1 --2- 172.21.15.117:0/2490111930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c678d8f1b0 0x55c6791dfb70 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f43880106c0 tx=0x7f43880058f0 comp rx=0 tx=0).stop 2024-09-21T06:07:16.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.185+0000 7f43ab72a740 1 -- 172.21.15.117:0/2490111930 shutdown_connections 2024-09-21T06:07:16.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.185+0000 7f43ab72a740 1 --2- 172.21.15.117:0/2490111930 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c678d4bda0 0x55c6791eeaa0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:16.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.185+0000 7f43ab72a740 1 --2- 172.21.15.117:0/2490111930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c678db8070 0x55c6791e49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:16.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.185+0000 7f43ab72a740 1 --2- 172.21.15.117:0/2490111930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c678d97390 0x55c6791e0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:16.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.185+0000 7f43ab72a740 1 --2- 172.21.15.117:0/2490111930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c678d8f1b0 0x55c6791dfb70 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:16.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.185+0000 7f43867fc640 0 client.17559 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:16.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.186+0000 7f43ab72a740 1 -- 172.21.15.117:0/2490111930 >> 172.21.15.117:0/2490111930 conn(0x55c678dac460 msgr2=0x55c679162750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:16.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.186+0000 7f43ab72a740 1 -- 172.21.15.117:0/2490111930 shutdown_connections 2024-09-21T06:07:16.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.186+0000 7f43ab72a740 1 -- 172.21.15.117:0/2490111930 wait complete. 2024-09-21T06:07:16.193 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.186+0000 7f43ab72a740 20 client.17559 trim_cache size 0 max 16384 2024-09-21T06:07:16.194 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T06:07:16.194 INFO:teuthology.orchestra.run.smithi117.stdout: "status": "no active scrubs running", 2024-09-21T06:07:16.194 INFO:teuthology.orchestra.run.smithi117.stdout: "scrubs": {}, 2024-09-21T06:07:16.194 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_stats": { 2024-09-21T06:07:16.194 INFO:teuthology.orchestra.run.smithi117.stdout: "26e3d90e-072a-4a4e-9661-4d004051865b": { 2024-09-21T06:07:16.194 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:06:29.219142+0000", 2024-09-21T06:07:16.194 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:16.194 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:16.194 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:16.194 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:16.194 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:16.194 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:16.195 INFO:teuthology.orchestra.run.smithi117.stdout: "cf43a1b2-838e-4d28-9ed6-ff0c726fd51a": { 2024-09-21T06:07:16.195 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:06:42.901863+0000", 2024-09-21T06:07:16.195 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:16.195 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:16.195 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:16.195 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:16.195 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:16.195 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:16.195 INFO:teuthology.orchestra.run.smithi117.stdout: "39eeca8a-6b4f-465f-af34-b54f77320d70": { 2024-09-21T06:07:16.195 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:05:41.497141+0000", 2024-09-21T06:07:16.195 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:16.195 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:16.195 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:16.196 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:16.196 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:16.196 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:16.196 INFO:teuthology.orchestra.run.smithi117.stdout: "29ff9b7c-49d9-4910-85be-bace7e940cd5": { 2024-09-21T06:07:16.196 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:03:03.775527+0000", 2024-09-21T06:07:16.196 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:16.196 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:16.196 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:16.196 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:16.196 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:16.196 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:16.196 INFO:teuthology.orchestra.run.smithi117.stdout: "0902e33a-ce33-45fb-92d8-2e6fcb8ec8a0": { 2024-09-21T06:07:16.196 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:04:11.365073+0000", 2024-09-21T06:07:16.196 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:16.197 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:16.197 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:16.197 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:16.197 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:16.197 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:16.197 INFO:teuthology.orchestra.run.smithi117.stdout: "d543b0ae-b710-4153-99da-fe5714839c1a": { 2024-09-21T06:07:16.197 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:01:20.574754+0000", 2024-09-21T06:07:16.197 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:16.197 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:16.197 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:16.197 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:16.197 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:16.197 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:16.197 INFO:teuthology.orchestra.run.smithi117.stdout: "ad80d9f1-6aff-46c0-91ab-2c1ab0681039": { 2024-09-21T06:07:16.198 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:59:31.949108+0000", 2024-09-21T06:07:16.198 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:16.198 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:16.198 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:16.198 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:16.198 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:16.198 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:16.198 INFO:teuthology.orchestra.run.smithi117.stdout: "fb2f217d-463b-4c8f-ab19-920b04bb7306": { 2024-09-21T06:07:16.198 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:04:56.308107+0000", 2024-09-21T06:07:16.198 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:16.198 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:16.198 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:16.198 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:16.199 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:16.199 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:16.199 INFO:teuthology.orchestra.run.smithi117.stdout: "8b3fe95a-b76b-4e94-afd0-72fe35e2dd92": { 2024-09-21T06:07:16.199 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:58:01.246076+0000", 2024-09-21T06:07:16.199 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:16.199 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:16.199 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:16.199 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:16.199 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:16.199 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:16.199 INFO:teuthology.orchestra.run.smithi117.stdout: "b30bfff0-0f6b-4e30-8467-ff7db421eba6": { 2024-09-21T06:07:16.199 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:57:14.939082+0000", 2024-09-21T06:07:16.199 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:16.199 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:16.199 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:16.199 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:16.200 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:16.200 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:07:16.200 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:07:16.200 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T06:07:16.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.186+0000 7f43a51b6640 1 -- 172.21.15.117:0/3828128058 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f438c0b6700 msgr2=0x7f438c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:16.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.186+0000 7f43a51b6640 1 --2- 172.21.15.117:0/3828128058 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f438c0b6700 0x7f438c0b8c10 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4398106450 tx=0x7f439809be90 comp rx=0 tx=0).stop 2024-09-21T06:07:16.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.186+0000 7f43a51b6640 1 -- 172.21.15.117:0/3828128058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43a0108cf0 msgr2=0x7f43a019bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:16.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.186+0000 7f43a51b6640 1 --2- 172.21.15.117:0/3828128058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43a0108cf0 0x7f43a019bc90 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f4398005cc0 tx=0x7f439802f380 comp rx=0 tx=0).stop 2024-09-21T06:07:16.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.186+0000 7f43a51b6640 1 -- 172.21.15.117:0/3828128058 shutdown_connections 2024-09-21T06:07:16.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.186+0000 7f43a51b6640 1 --2- 172.21.15.117:0/3828128058 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f438c0b6700 0x7f438c0b8c10 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:16.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.186+0000 7f43a51b6640 1 --2- 172.21.15.117:0/3828128058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43a0108cf0 0x7f43a019bc90 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:16.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.186+0000 7f43a51b6640 1 --2- 172.21.15.117:0/3828128058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43a01083a0 0x7f43a019b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:16.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.186+0000 7f43a51b6640 1 --2- 172.21.15.117:0/3828128058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43a0102a60 0x7f43a019b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:16.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.186+0000 7f43a51b6640 1 -- 172.21.15.117:0/3828128058 >> 172.21.15.117:0/3828128058 conn(0x7f43a0079730 msgr2=0x7f43a0104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:16.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.187+0000 7f43a51b6640 1 -- 172.21.15.117:0/3828128058 shutdown_connections 2024-09-21T06:07:16.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:16.187+0000 7f43a51b6640 1 -- 172.21.15.117:0/3828128058 wait complete. 2024-09-21T06:07:16.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:15 smithi138 ceph-mon[34391]: pgmap v631: 97 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 89 active+clean; 41 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 921 B/s wr, 11 op/s 2024-09-21T06:07:16.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:15 smithi192 ceph-mon[34700]: pgmap v631: 97 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 89 active+clean; 41 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 921 B/s wr, 11 op/s 2024-09-21T06:07:16.996 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:07:16.996 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:07:17.246 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:07:18.263 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:17 smithi117 ceph-mon[30947]: pgmap v632: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 921 B/s wr, 11 op/s 2024-09-21T06:07:18.263 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:17 smithi117 ceph-mon[30947]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T06:07:18.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:17 smithi138 ceph-mon[34391]: pgmap v632: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 921 B/s wr, 11 op/s 2024-09-21T06:07:18.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:17 smithi138 ceph-mon[34391]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T06:07:18.479 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:17 smithi192 ceph-mon[34700]: pgmap v632: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 921 B/s wr, 11 op/s 2024-09-21T06:07:18.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:17 smithi192 ceph-mon[34700]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T06:07:18.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.886+0000 7f951fb89640 1 -- 172.21.15.117:0/1365000979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9518103b60 msgr2=0x7f9518103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:18.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.886+0000 7f951fb89640 1 --2- 172.21.15.117:0/1365000979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9518103b60 0x7f9518103f70 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f9514009900 tx=0x7f951402a410 comp rx=0 tx=0).stop 2024-09-21T06:07:18.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.886+0000 7f951fb89640 1 -- 172.21.15.117:0/1365000979 shutdown_connections 2024-09-21T06:07:18.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.886+0000 7f951fb89640 1 --2- 172.21.15.117:0/1365000979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9518109df0 0x7f9518110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:18.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.886+0000 7f951fb89640 1 --2- 172.21.15.117:0/1365000979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95181094a0 0x7f95181098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:18.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.886+0000 7f951fb89640 1 --2- 172.21.15.117:0/1365000979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9518103b60 0x7f9518103f70 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:18.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.886+0000 7f951fb89640 1 -- 172.21.15.117:0/1365000979 >> 172.21.15.117:0/1365000979 conn(0x7f951807b850 msgr2=0x7f951807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:18.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.886+0000 7f951fb89640 1 -- 172.21.15.117:0/1365000979 shutdown_connections 2024-09-21T06:07:18.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.886+0000 7f951fb89640 1 -- 172.21.15.117:0/1365000979 wait complete. 2024-09-21T06:07:18.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.887+0000 7f951fb89640 1 Processor -- start 2024-09-21T06:07:18.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.887+0000 7f951fb89640 1 -- start start 2024-09-21T06:07:18.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.887+0000 7f951fb89640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9518103b60 0x7f951819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:18.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.887+0000 7f951fb89640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95181094a0 0x7f951819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:18.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.887+0000 7f951fb89640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9518109df0 0x7f951819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:18.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.887+0000 7f951fb89640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9518113ab0 con 0x7f9518103b60 2024-09-21T06:07:18.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.887+0000 7f951fb89640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9518113930 con 0x7f95181094a0 2024-09-21T06:07:18.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.887+0000 7f951fb89640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9518113c30 con 0x7f9518109df0 2024-09-21T06:07:18.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.887+0000 7f951d925640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9518103b60 0x7f951819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:18.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.887+0000 7f951d925640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9518103b60 0x7f951819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44282/0 (socket says 172.21.15.117:44282) 2024-09-21T06:07:18.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.887+0000 7f951d925640 1 -- 172.21.15.117:0/2147910679 learned_addr learned my addr 172.21.15.117:0/2147910679 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:18.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.888+0000 7f951e126640 1 --2- 172.21.15.117:0/2147910679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9518109df0 0x7f951819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:18.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.888+0000 7f951d124640 1 --2- 172.21.15.117:0/2147910679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95181094a0 0x7f951819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:18.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.888+0000 7f951d925640 1 -- 172.21.15.117:0/2147910679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9518109df0 msgr2=0x7f951819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:18.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.888+0000 7f951d925640 1 --2- 172.21.15.117:0/2147910679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9518109df0 0x7f951819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:18.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.888+0000 7f951d925640 1 -- 172.21.15.117:0/2147910679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95181094a0 msgr2=0x7f951819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:18.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.888+0000 7f951d925640 1 --2- 172.21.15.117:0/2147910679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95181094a0 0x7f951819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:18.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.888+0000 7f951d925640 1 -- 172.21.15.117:0/2147910679 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95181a0410 con 0x7f9518103b60 2024-09-21T06:07:18.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.888+0000 7f951d124640 1 --2- 172.21.15.117:0/2147910679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95181094a0 0x7f951819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:18.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.888+0000 7f951e126640 1 --2- 172.21.15.117:0/2147910679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9518109df0 0x7f951819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:18.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.888+0000 7f951d925640 1 --2- 172.21.15.117:0/2147910679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9518103b60 0x7f951819a4a0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f9514005940 tx=0x7f95140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:18.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.888+0000 7f950effd640 1 -- 172.21.15.117:0/2147910679 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951403f030 con 0x7f9518103b60 2024-09-21T06:07:18.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.888+0000 7f951fb89640 1 -- 172.21.15.117:0/2147910679 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95181a06a0 con 0x7f9518103b60 2024-09-21T06:07:18.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.888+0000 7f950effd640 1 -- 172.21.15.117:0/2147910679 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9514003cb0 con 0x7f9518103b60 2024-09-21T06:07:18.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.888+0000 7f950effd640 1 -- 172.21.15.117:0/2147910679 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9514035070 con 0x7f9518103b60 2024-09-21T06:07:18.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.888+0000 7f951fb89640 1 -- 172.21.15.117:0/2147910679 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95181a6e60 con 0x7f9518103b60 2024-09-21T06:07:18.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.889+0000 7f951fb89640 1 -- 172.21.15.117:0/2147910679 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9518103f70 con 0x7f9518103b60 2024-09-21T06:07:18.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.891+0000 7f950effd640 1 -- 172.21.15.117:0/2147910679 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f9514042030 con 0x7f9518103b60 2024-09-21T06:07:18.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.892+0000 7f950effd640 1 --2- 172.21.15.117:0/2147910679 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f95040b6680 0x7f95040b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:18.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.892+0000 7f951d124640 1 --2- 172.21.15.117:0/2147910679 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f95040b6680 0x7f95040b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:18.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.892+0000 7f950effd640 1 -- 172.21.15.117:0/2147910679 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f9514108270 con 0x7f9518103b60 2024-09-21T06:07:18.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.892+0000 7f951d124640 1 --2- 172.21.15.117:0/2147910679 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f95040b6680 0x7f95040b8b90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f95080058c0 tx=0x7f9508005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:18.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:18.895+0000 7f950effd640 1 -- 172.21.15.117:0/2147910679 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f95140ce810 con 0x7f9518103b60 2024-09-21T06:07:19.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:19.112+0000 7f951fb89640 1 -- 172.21.15.117:0/2147910679 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f951819b5d0 con 0x7f9518103b60 2024-09-21T06:07:19.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:19.113+0000 7f950effd640 1 -- 172.21.15.117:0/2147910679 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+12001 (secure 0 0 0) 0x7f95140d2bf0 con 0x7f9518103b60 2024-09-21T06:07:19.115 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T06:07:19.115 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":29,"btime":"2024-09-21T06:04:08:952483+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3227057504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3227057504},{"type":"v1","addr":"172.21.15.117:6835","nonce":3227057504}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3303829183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3303829183},{"type":"v1","addr":"172.21.15.117:6837","nonce":3303829183}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T05:55:14.555897+0000","modified":"2024-09-21T06:04:08.952466+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24638,"mds_1":24637,"mds_2":24628,"mds_3":24616,"mds_4":24602},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6839/2314572411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2314572411},{"type":"v1","addr":"172.21.15.117:6839","nonce":2314572411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6841/3378096561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":3378096561},{"type":"v1","addr":"172.21.15.117:6841","nonce":3378096561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/3725629862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3725629862},{"type":"v1","addr":"172.21.15.138:6833","nonce":3725629862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/4276801396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4276801396},{"type":"v1","addr":"172.21.15.138:6835","nonce":4276801396}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.138:6837/2540335374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2540335374},{"type":"v1","addr":"172.21.15.138:6837","nonce":2540335374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24616":{"gid":24616,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.192:6835/3530688860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3530688860},{"type":"v1","addr":"172.21.15.192:6835","nonce":3530688860}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.192:6837/2683860842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2683860842},{"type":"v1","addr":"172.21.15.192:6837","nonce":2683860842}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.192:6839/1727623231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1727623231},{"type":"v1","addr":"172.21.15.192:6839","nonce":1727623231}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/3230372576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":3230372576},{"type":"v1","addr":"172.21.15.138:6839","nonce":3230372576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25840":{"gid":25840,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6833/1596605058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1596605058},{"type":"v1","addr":"172.21.15.192:6833","nonce":1596605058}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24602,24628,24616,24637,24638]},"id":1}]} 2024-09-21T06:07:19.115 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 29 2024-09-21T06:07:19.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:19.115+0000 7f951fb89640 1 -- 172.21.15.117:0/2147910679 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f95040b6680 msgr2=0x7f95040b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:19.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:19.115+0000 7f951fb89640 1 --2- 172.21.15.117:0/2147910679 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f95040b6680 0x7f95040b8b90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f95080058c0 tx=0x7f9508005850 comp rx=0 tx=0).stop 2024-09-21T06:07:19.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:19.115+0000 7f951fb89640 1 -- 172.21.15.117:0/2147910679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9518103b60 msgr2=0x7f951819a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:19.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:19.115+0000 7f951fb89640 1 --2- 172.21.15.117:0/2147910679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9518103b60 0x7f951819a4a0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f9514005940 tx=0x7f95140042f0 comp rx=0 tx=0).stop 2024-09-21T06:07:19.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:19.115+0000 7f951fb89640 1 -- 172.21.15.117:0/2147910679 shutdown_connections 2024-09-21T06:07:19.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:19.115+0000 7f951fb89640 1 --2- 172.21.15.117:0/2147910679 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f95040b6680 0x7f95040b8b90 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:19.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:19.115+0000 7f951fb89640 1 --2- 172.21.15.117:0/2147910679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9518109df0 0x7f951819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:19.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:19.115+0000 7f951fb89640 1 --2- 172.21.15.117:0/2147910679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95181094a0 0x7f951819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:19.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:19.115+0000 7f951fb89640 1 --2- 172.21.15.117:0/2147910679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9518103b60 0x7f951819a4a0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:19.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:19.115+0000 7f951fb89640 1 -- 172.21.15.117:0/2147910679 >> 172.21.15.117:0/2147910679 conn(0x7f951807b850 msgr2=0x7f9518107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:19.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:19.115+0000 7f951fb89640 1 -- 172.21.15.117:0/2147910679 shutdown_connections 2024-09-21T06:07:19.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:19.115+0000 7f951fb89640 1 -- 172.21.15.117:0/2147910679 wait complete. 2024-09-21T06:07:19.833 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T06:07:20.081 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:07:20.201 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:19 smithi117 ceph-mon[30947]: pgmap v634: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 922 B/s wr, 10 op/s 2024-09-21T06:07:20.201 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:19 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2147910679' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:07:20.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:19 smithi138 ceph-mon[34391]: pgmap v634: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 922 B/s wr, 10 op/s 2024-09-21T06:07:20.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:19 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2147910679' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:07:20.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:19 smithi192 ceph-mon[34700]: pgmap v634: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 922 B/s wr, 10 op/s 2024-09-21T06:07:20.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:19 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2147910679' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:07:20.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.612+0000 7fc7198af640 1 -- 172.21.15.117:0/1933609002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc71410ddf0 msgr2=0x7fc714110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:20.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.612+0000 7fc7198af640 1 --2- 172.21.15.117:0/1933609002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc71410ddf0 0x7fc714110300 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fc70c009ca0 tx=0x7fc70c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.612+0000 7fc7198af640 1 -- 172.21.15.117:0/1933609002 shutdown_connections 2024-09-21T06:07:20.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.612+0000 7fc7198af640 1 --2- 172.21.15.117:0/1933609002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc71410ddf0 0x7fc714110300 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.612+0000 7fc7198af640 1 --2- 172.21.15.117:0/1933609002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc714105270 0x7fc71410d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.612+0000 7fc7198af640 1 --2- 172.21.15.117:0/1933609002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc714104840 0x7fc714104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.612+0000 7fc7198af640 1 -- 172.21.15.117:0/1933609002 >> 172.21.15.117:0/1933609002 conn(0x7fc714079730 msgr2=0x7fc714079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:20.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.612+0000 7fc7198af640 1 -- 172.21.15.117:0/1933609002 shutdown_connections 2024-09-21T06:07:20.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.613+0000 7fc7198af640 1 -- 172.21.15.117:0/1933609002 wait complete. 2024-09-21T06:07:20.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.613+0000 7fc7198af640 1 Processor -- start 2024-09-21T06:07:20.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.613+0000 7fc7198af640 1 -- start start 2024-09-21T06:07:20.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.614+0000 7fc7198af640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc714104840 0x7fc71419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:20.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.614+0000 7fc7198af640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc714105270 0x7fc71419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:20.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.614+0000 7fc7198af640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc71410ddf0 0x7fc71419fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:20.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.614+0000 7fc7198af640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc714113a90 con 0x7fc714104840 2024-09-21T06:07:20.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.614+0000 7fc7198af640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc714113910 con 0x7fc71410ddf0 2024-09-21T06:07:20.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.614+0000 7fc7198af640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc714113c10 con 0x7fc714105270 2024-09-21T06:07:20.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.614+0000 7fc712ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc714104840 0x7fc71419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:20.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.614+0000 7fc712ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc714104840 0x7fc71419a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44302/0 (socket says 172.21.15.117:44302) 2024-09-21T06:07:20.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.614+0000 7fc712ffd640 1 -- 172.21.15.117:0/2942915851 learned_addr learned my addr 172.21.15.117:0/2942915851 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:20.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.614+0000 7fc7127fc640 1 --2- 172.21.15.117:0/2942915851 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc714105270 0x7fc71419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:20.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.614+0000 7fc7137fe640 1 --2- 172.21.15.117:0/2942915851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc71410ddf0 0x7fc71419fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:20.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.614+0000 7fc712ffd640 1 -- 172.21.15.117:0/2942915851 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc714105270 msgr2=0x7fc71419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:20.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.614+0000 7fc712ffd640 1 --2- 172.21.15.117:0/2942915851 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc714105270 0x7fc71419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.614+0000 7fc712ffd640 1 -- 172.21.15.117:0/2942915851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc71410ddf0 msgr2=0x7fc71419fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:20.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.614+0000 7fc712ffd640 1 --2- 172.21.15.117:0/2942915851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc71410ddf0 0x7fc71419fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.614+0000 7fc712ffd640 1 -- 172.21.15.117:0/2942915851 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7141a0430 con 0x7fc714104840 2024-09-21T06:07:20.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.614+0000 7fc712ffd640 1 --2- 172.21.15.117:0/2942915851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc714104840 0x7fc71419a4c0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fc70400d770 tx=0x7fc70400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:20.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.614+0000 7fc7188ad640 1 -- 172.21.15.117:0/2942915851 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704017070 con 0x7fc714104840 2024-09-21T06:07:20.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.614+0000 7fc7188ad640 1 -- 172.21.15.117:0/2942915851 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc704009c30 con 0x7fc714104840 2024-09-21T06:07:20.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.614+0000 7fc7198af640 1 -- 172.21.15.117:0/2942915851 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7141a0720 con 0x7fc714104840 2024-09-21T06:07:20.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.614+0000 7fc7188ad640 1 -- 172.21.15.117:0/2942915851 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704003cc0 con 0x7fc714104840 2024-09-21T06:07:20.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.615+0000 7fc7198af640 1 -- 172.21.15.117:0/2942915851 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7141a6e80 con 0x7fc714104840 2024-09-21T06:07:20.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.617+0000 7fc7188ad640 1 -- 172.21.15.117:0/2942915851 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fc704003e60 con 0x7fc714104840 2024-09-21T06:07:20.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.617+0000 7fc71fe23740 1 Processor -- start 2024-09-21T06:07:20.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.617+0000 7fc71fe23740 1 -- start start 2024-09-21T06:07:20.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.617+0000 7fc71fe23740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a82362070 0x558a8270d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:20.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.617+0000 7fc71fe23740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a823391b0 0x558a8270dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:20.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.617+0000 7fc71fe23740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558a82341390 0x558a8270e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:20.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.617+0000 7fc71fe23740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558a8267fea0 con 0x558a823391b0 2024-09-21T06:07:20.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.618+0000 7fc71fe23740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558a8266f2d0 con 0x558a82362070 2024-09-21T06:07:20.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.618+0000 7fc71fe23740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558a826730e0 con 0x558a82341390 2024-09-21T06:07:20.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.618+0000 7fc712ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a823391b0 0x558a8270dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:20.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.618+0000 7fc712ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a823391b0 0x558a8270dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44304/0 (socket says 172.21.15.117:44304) 2024-09-21T06:07:20.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.618+0000 7fc712ffd640 1 -- 172.21.15.117:0/2708117582 learned_addr learned my addr 172.21.15.117:0/2708117582 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:20.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.618+0000 7fc7137fe640 1 --2- 172.21.15.117:0/2708117582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a82362070 0x558a8270d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:20.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.618+0000 7fc7127fc640 1 --2- 172.21.15.117:0/2708117582 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558a82341390 0x558a8270e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:20.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.618+0000 7fc7188ad640 1 --2- 172.21.15.117:0/2942915851 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc6fc0b6680 0x7fc6fc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:20.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.618+0000 7fc712ffd640 1 -- 172.21.15.117:0/2708117582 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558a82341390 msgr2=0x558a8270e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:20.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.618+0000 7fc712ffd640 1 --2- 172.21.15.117:0/2708117582 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558a82341390 0x558a8270e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.618+0000 7fc712ffd640 1 -- 172.21.15.117:0/2708117582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a82362070 msgr2=0x558a8270d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:20.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.618+0000 7fc712ffd640 1 --2- 172.21.15.117:0/2708117582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a82362070 0x558a8270d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.618+0000 7fc712ffd640 1 -- 172.21.15.117:0/2708117582 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a823cd650 con 0x558a823391b0 2024-09-21T06:07:20.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.618+0000 7fc7137fe640 1 --2- 172.21.15.117:0/2942915851 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc6fc0b6680 0x7fc6fc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:20.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.618+0000 7fc7188ad640 1 -- 172.21.15.117:0/2942915851 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7fc7040e6f90 con 0x7fc714104840 2024-09-21T06:07:20.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.618+0000 7fc712ffd640 1 --2- 172.21.15.117:0/2708117582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a823391b0 0x558a8270dc10 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fc7040e46a0 tx=0x7fc7040ad0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc5b2f5c1b9ad4a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:20.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.618+0000 7fc6f9ffb640 1 -- 172.21.15.117:0/2708117582 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704020020 con 0x558a823391b0 2024-09-21T06:07:20.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.619+0000 7fc6f9ffb640 1 -- 172.21.15.117:0/2708117582 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7040ac9f0 con 0x558a823391b0 2024-09-21T06:07:20.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.619+0000 7fc6f9ffb640 1 -- 172.21.15.117:0/2708117582 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7040accb0 con 0x558a823391b0 2024-09-21T06:07:20.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.619+0000 7fc7137fe640 1 --2- 172.21.15.117:0/2942915851 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc6fc0b6680 0x7fc6fc0b8b90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc70c006010 tx=0x7fc70c0095c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:20.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.619+0000 7fc71fe23740 1 -- 172.21.15.117:0/2708117582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a823391b0 msgr2=0x558a8270dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:20.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.619+0000 7fc71fe23740 1 --2- 172.21.15.117:0/2708117582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a823391b0 0x558a8270dc10 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fc7040e46a0 tx=0x7fc7040ad0d0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.619+0000 7fc71fe23740 1 -- 172.21.15.117:0/2708117582 shutdown_connections 2024-09-21T06:07:20.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.619+0000 7fc71fe23740 1 --2- 172.21.15.117:0/2708117582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a823391b0 0x558a8270dc10 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.620+0000 7fc71fe23740 1 --2- 172.21.15.117:0/2708117582 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558a82341390 0x558a8270e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.620+0000 7fc71fe23740 1 --2- 172.21.15.117:0/2708117582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a82362070 0x558a8270d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.620+0000 7fc71fe23740 1 -- 172.21.15.117:0/2708117582 >> 172.21.15.117:0/2708117582 conn(0x558a82356460 msgr2=0x558a8270b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:20.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.620+0000 7fc71fe23740 1 -- 172.21.15.117:0/2708117582 shutdown_connections 2024-09-21T06:07:20.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.620+0000 7fc71fe23740 1 -- 172.21.15.117:0/2708117582 wait complete. 2024-09-21T06:07:20.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.620+0000 7fc71fe23740 1 -- start start 2024-09-21T06:07:20.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.620+0000 7fc71fe23740 1 Processor -- start 2024-09-21T06:07:20.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.620+0000 7fc71fe23740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a823391b0 0x558a82718d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:20.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.620+0000 7fc71fe23740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558a82341390 0x558a82719260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:20.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.620+0000 7fc71fe23740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a82362070 0x558a827197c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:20.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.620+0000 7fc71fe23740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558a824d0700 con 0x558a823391b0 2024-09-21T06:07:20.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.620+0000 7fc71fe23740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558a8260a160 con 0x558a82362070 2024-09-21T06:07:20.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.620+0000 7fc71fe23740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558a8271beb0 con 0x558a82341390 2024-09-21T06:07:20.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.621+0000 7fc71fe23740 10 client.0 _ll_register_callbacks cb 0x558a826239e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:20.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.621+0000 7fc712ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a823391b0 0x558a82718d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:20.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.621+0000 7fc712ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a823391b0 0x558a82718d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44312/0 (socket says 172.21.15.117:44312) 2024-09-21T06:07:20.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.621+0000 7fc712ffd640 1 -- 172.21.15.117:0/3244138706 learned_addr learned my addr 172.21.15.117:0/3244138706 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:20.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.621+0000 7fc7137fe640 1 --2- 172.21.15.117:0/3244138706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a82362070 0x558a827197c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:20.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.621+0000 7fc7127fc640 1 --2- 172.21.15.117:0/3244138706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558a82341390 0x558a82719260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:20.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.621+0000 7fc712ffd640 1 -- 172.21.15.117:0/3244138706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558a82341390 msgr2=0x558a82719260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:20.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.621+0000 7fc712ffd640 1 --2- 172.21.15.117:0/3244138706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558a82341390 0x558a82719260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.621+0000 7fc712ffd640 1 -- 172.21.15.117:0/3244138706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a82362070 msgr2=0x558a827197c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:20.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.621+0000 7fc712ffd640 1 --2- 172.21.15.117:0/3244138706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a82362070 0x558a827197c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.621+0000 7fc712ffd640 1 -- 172.21.15.117:0/3244138706 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a82637c90 con 0x558a823391b0 2024-09-21T06:07:20.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.621+0000 7fc7137fe640 1 --2- 172.21.15.117:0/3244138706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a82362070 0x558a827197c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:20.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.621+0000 7fc712ffd640 1 --2- 172.21.15.117:0/3244138706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a823391b0 0x558a82718d20 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc7040a94b0 tx=0x7fc7040045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98ae334b46d9e6d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:20.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.621+0000 7fc6f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:20.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.621+0000 7fc6f2ffd640 1 -- 172.21.15.117:0/3244138706 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704020040 con 0x558a823391b0 2024-09-21T06:07:20.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.621+0000 7fc6f2ffd640 1 -- 172.21.15.117:0/3244138706 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7fc7040c2af0 con 0x558a823391b0 2024-09-21T06:07:20.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.621+0000 7fc71fe23740 1 -- 172.21.15.117:0/3244138706 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7040ac9f0 con 0x558a823391b0 2024-09-21T06:07:20.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.621+0000 7fc6f2ffd640 1 -- 172.21.15.117:0/3244138706 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc704098030 con 0x558a823391b0 2024-09-21T06:07:20.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.621+0000 7fc71fe23740 10 client.17577 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:20.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.621+0000 7fc71fe23740 1 -- 172.21.15.117:0/3244138706 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558a82727b60 con 0x558a823391b0 2024-09-21T06:07:20.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.622+0000 7fc6f2ffd640 1 -- 172.21.15.117:0/3244138706 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7040ac9f0 con 0x558a823391b0 2024-09-21T06:07:20.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.622+0000 7fc6f2ffd640 1 -- 172.21.15.117:0/3244138706 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc70405c190 con 0x558a823391b0 2024-09-21T06:07:20.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.622+0000 7fc71fe23740 10 client.17577 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:20.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.622+0000 7fc71fe23740 10 client.17577 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T06:07:20.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.622+0000 7fc71fe23740 20 client.17577 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:20.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.622+0000 7fc71fe23740 1 --2- 172.21.15.117:0/3244138706 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x558a823209f0 0x558a8272aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:20.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.622+0000 7fc71fe23740 4 client.17577 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:07:20.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.622+0000 7fc71fe23740 1 -- 172.21.15.117:0/3244138706 --> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a82681df0 con 0x558a823209f0 2024-09-21T06:07:20.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.622+0000 7fc7127fc640 1 --2- 172.21.15.117:0/3244138706 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x558a823209f0 0x558a8272aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:20.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.623+0000 7fc7127fc640 1 --2- 172.21.15.117:0/3244138706 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x558a823209f0 0x558a8272aec0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b6848d20168af3a8 server_cookie=1d6c760301be05b3 in_seq=0 out_seq=0 2024-09-21T06:07:20.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.623+0000 7fc6f2ffd640 10 client.17577 ms_handle_connect on v2:172.21.15.138:6836/2540335374 2024-09-21T06:07:20.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.623+0000 7fc6f2ffd640 1 -- 172.21.15.117:0/3244138706 <== mds.4 v2:172.21.15.138:6836/2540335374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558a82681df0 con 0x558a823209f0 2024-09-21T06:07:20.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.623+0000 7fc6f2ffd640 10 client.17577 handle_command_reply: tid=0 2024-09-21T06:07:20.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.623+0000 7fc71fe23740 1 client.17577 shutdown 2024-09-21T06:07:20.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.624+0000 7fc71fe23740 1 -- 172.21.15.117:0/3244138706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a823391b0 msgr2=0x558a82718d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:20.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.624+0000 7fc71fe23740 1 --2- 172.21.15.117:0/3244138706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a823391b0 0x558a82718d20 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc7040a94b0 tx=0x7fc7040045c0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.624+0000 7fc71fe23740 1 -- 172.21.15.117:0/3244138706 shutdown_connections 2024-09-21T06:07:20.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.624+0000 7fc71fe23740 1 --2- 172.21.15.117:0/3244138706 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x558a823209f0 0x558a8272aec0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.624+0000 7fc71fe23740 1 --2- 172.21.15.117:0/3244138706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a82362070 0x558a827197c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.624+0000 7fc71fe23740 1 --2- 172.21.15.117:0/3244138706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558a82341390 0x558a82719260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.624+0000 7fc71fe23740 1 --2- 172.21.15.117:0/3244138706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a823391b0 0x558a82718d20 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.624+0000 7fc6f2ffd640 0 client.17577 ms_handle_reset on v2:172.21.15.138:6836/2540335374 2024-09-21T06:07:20.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.624+0000 7fc71fe23740 1 -- 172.21.15.117:0/3244138706 >> 172.21.15.117:0/3244138706 conn(0x558a82356460 msgr2=0x558a82710860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:20.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.624+0000 7fc71fe23740 1 -- 172.21.15.117:0/3244138706 shutdown_connections 2024-09-21T06:07:20.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.624+0000 7fc71fe23740 1 -- 172.21.15.117:0/3244138706 wait complete. 2024-09-21T06:07:20.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.624+0000 7fc71fe23740 20 client.17577 trim_cache size 0 max 16384 2024-09-21T06:07:20.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.637+0000 7fc71fe23740 1 Processor -- start 2024-09-21T06:07:20.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.637+0000 7fc71fe23740 1 -- start start 2024-09-21T06:07:20.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.637+0000 7fc71fe23740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558a823391b0 0x558a82718ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:20.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.637+0000 7fc71fe23740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a82341390 0x558a82719010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:20.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.637+0000 7fc71fe23740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a82362070 0x558a82715f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:20.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.637+0000 7fc71fe23740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc7141ad560 con 0x558a82362070 2024-09-21T06:07:20.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.637+0000 7fc71fe23740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc7141af0a0 con 0x558a82341390 2024-09-21T06:07:20.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.637+0000 7fc71fe23740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc7141adc30 con 0x558a823391b0 2024-09-21T06:07:20.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.637+0000 7fc7137fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a82362070 0x558a82715f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:20.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.637+0000 7fc7137fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a82362070 0x558a82715f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44314/0 (socket says 172.21.15.117:44314) 2024-09-21T06:07:20.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.637+0000 7fc7137fe640 1 -- 172.21.15.117:0/365258817 learned_addr learned my addr 172.21.15.117:0/365258817 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:20.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.637+0000 7fc7127fc640 1 --2- 172.21.15.117:0/365258817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a82341390 0x558a82719010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:20.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.637+0000 7fc712ffd640 1 --2- 172.21.15.117:0/365258817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558a823391b0 0x558a82718ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:20.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.637+0000 7fc7137fe640 1 -- 172.21.15.117:0/365258817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558a823391b0 msgr2=0x558a82718ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:20.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.637+0000 7fc7137fe640 1 --2- 172.21.15.117:0/365258817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558a823391b0 0x558a82718ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.637+0000 7fc7137fe640 1 -- 172.21.15.117:0/365258817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a82341390 msgr2=0x558a82719010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:20.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.637+0000 7fc7137fe640 1 --2- 172.21.15.117:0/365258817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a82341390 0x558a82719010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.637+0000 7fc7137fe640 1 -- 172.21.15.117:0/365258817 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a82719fe0 con 0x558a82362070 2024-09-21T06:07:20.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.637+0000 7fc7127fc640 1 --2- 172.21.15.117:0/365258817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a82341390 0x558a82719010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:20.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.638+0000 7fc7137fe640 1 --2- 172.21.15.117:0/365258817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a82362070 0x558a82715f10 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fc70c035000 tx=0x7fc70c044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a8645aac079c31 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:20.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.638+0000 7fc6f97fa640 1 -- 172.21.15.117:0/365258817 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c003cf0 con 0x558a82362070 2024-09-21T06:07:20.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.638+0000 7fc6f97fa640 1 -- 172.21.15.117:0/365258817 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc70c036040 con 0x558a82362070 2024-09-21T06:07:20.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.638+0000 7fc6f97fa640 1 -- 172.21.15.117:0/365258817 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c02fb80 con 0x558a82362070 2024-09-21T06:07:20.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.638+0000 7fc71fe23740 1 -- 172.21.15.117:0/365258817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a82362070 msgr2=0x558a82715f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:20.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.638+0000 7fc71fe23740 1 --2- 172.21.15.117:0/365258817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a82362070 0x558a82715f10 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fc70c035000 tx=0x7fc70c044000 comp rx=0 tx=0).stop 2024-09-21T06:07:20.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.638+0000 7fc71fe23740 1 -- 172.21.15.117:0/365258817 shutdown_connections 2024-09-21T06:07:20.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.638+0000 7fc71fe23740 1 --2- 172.21.15.117:0/365258817 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a82362070 0x558a82715f10 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.638+0000 7fc71fe23740 1 --2- 172.21.15.117:0/365258817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a82341390 0x558a82719010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.638+0000 7fc71fe23740 1 --2- 172.21.15.117:0/365258817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558a823391b0 0x558a82718ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.638+0000 7fc71fe23740 1 -- 172.21.15.117:0/365258817 >> 172.21.15.117:0/365258817 conn(0x558a82356460 msgr2=0x558a827175a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:20.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.638+0000 7fc71fe23740 1 -- 172.21.15.117:0/365258817 shutdown_connections 2024-09-21T06:07:20.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.638+0000 7fc71fe23740 1 -- 172.21.15.117:0/365258817 wait complete. 2024-09-21T06:07:20.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.639+0000 7fc71fe23740 1 -- start start 2024-09-21T06:07:20.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.639+0000 7fc71fe23740 1 Processor -- start 2024-09-21T06:07:20.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.639+0000 7fc71fe23740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a823391b0 0x558a8272c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:20.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.639+0000 7fc71fe23740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558a82341390 0x558a8272c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:20.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.639+0000 7fc71fe23740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a82362070 0x558a8272cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:20.657 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.639+0000 7fc71fe23740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc7141ae9d0 con 0x558a82362070 2024-09-21T06:07:20.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.639+0000 7fc71fe23740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc7141ae300 con 0x558a823391b0 2024-09-21T06:07:20.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.639+0000 7fc71fe23740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc7141ab560 con 0x558a82341390 2024-09-21T06:07:20.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.639+0000 7fc71fe23740 10 client.0 _ll_register_callbacks cb 0x7fc70c009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:20.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.640+0000 7fc7137fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a82362070 0x558a8272cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:20.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.640+0000 7fc7127fc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558a82341390 0x558a8272c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:20.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.640+0000 7fc7137fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a82362070 0x558a8272cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44324/0 (socket says 172.21.15.117:44324) 2024-09-21T06:07:20.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.640+0000 7fc7137fe640 1 -- 172.21.15.117:0/890745615 learned_addr learned my addr 172.21.15.117:0/890745615 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:20.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.640+0000 7fc712ffd640 1 --2- 172.21.15.117:0/890745615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a823391b0 0x558a8272c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:20.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.640+0000 7fc7137fe640 1 -- 172.21.15.117:0/890745615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558a82341390 msgr2=0x558a8272c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:20.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.640+0000 7fc7137fe640 1 --2- 172.21.15.117:0/890745615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558a82341390 0x558a8272c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.640+0000 7fc7137fe640 1 -- 172.21.15.117:0/890745615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a823391b0 msgr2=0x558a8272c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:20.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.640+0000 7fc7137fe640 1 --2- 172.21.15.117:0/890745615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a823391b0 0x558a8272c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.640+0000 7fc7137fe640 1 -- 172.21.15.117:0/890745615 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a82794890 con 0x558a82362070 2024-09-21T06:07:20.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.640+0000 7fc7137fe640 1 --2- 172.21.15.117:0/890745615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a82362070 0x558a8272cbf0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fc70c02ade0 tx=0x7fc70c02f660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90b31b537b0108f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:20.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.640+0000 7fc6f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:20.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.640+0000 7fc6f2ffd640 1 -- 172.21.15.117:0/890745615 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c061060 con 0x558a82362070 2024-09-21T06:07:20.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.640+0000 7fc6f2ffd640 1 -- 172.21.15.117:0/890745615 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7fc70c004250 con 0x558a82362070 2024-09-21T06:07:20.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.640+0000 7fc71fe23740 1 -- 172.21.15.117:0/890745615 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc70c036040 con 0x558a82362070 2024-09-21T06:07:20.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.640+0000 7fc6f2ffd640 1 -- 172.21.15.117:0/890745615 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc70c004450 con 0x558a82362070 2024-09-21T06:07:20.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.641+0000 7fc71fe23740 10 client.17583 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:20.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.641+0000 7fc71fe23740 1 -- 172.21.15.117:0/890745615 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558a82795b10 con 0x558a82362070 2024-09-21T06:07:20.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.641+0000 7fc6f2ffd640 1 -- 172.21.15.117:0/890745615 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc70c036040 con 0x558a82362070 2024-09-21T06:07:20.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.641+0000 7fc6f2ffd640 1 -- 172.21.15.117:0/890745615 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc70c052070 con 0x558a82362070 2024-09-21T06:07:20.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.641+0000 7fc71fe23740 10 client.17583 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:20.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.641+0000 7fc71fe23740 10 client.17583 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T06:07:20.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.641+0000 7fc71fe23740 20 client.17583 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:20.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.641+0000 7fc71fe23740 1 --2- 172.21.15.117:0/890745615 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x558a822f5da0 0x558a82798330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:20.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.641+0000 7fc71fe23740 4 client.17583 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:07:20.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.641+0000 7fc71fe23740 1 -- 172.21.15.117:0/890745615 --> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558a8267d8a0 con 0x558a822f5da0 2024-09-21T06:07:20.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.641+0000 7fc712ffd640 1 --2- 172.21.15.117:0/890745615 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x558a822f5da0 0x558a82798330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:20.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.642+0000 7fc6f2ffd640 10 client.17583 ms_handle_connect on v2:172.21.15.138:6836/2540335374 2024-09-21T06:07:20.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.642+0000 7fc712ffd640 1 --2- 172.21.15.117:0/890745615 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x558a822f5da0 0x558a82798330 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6a04156f3c27b413 server_cookie=d2f6744018bdc362 in_seq=0 out_seq=0 2024-09-21T06:07:20.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.642+0000 7fc6f2ffd640 1 -- 172.21.15.117:0/890745615 <== mds.4 v2:172.21.15.138:6836/2540335374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558a8267d8a0 con 0x558a822f5da0 2024-09-21T06:07:20.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.642+0000 7fc6f2ffd640 10 client.17583 handle_command_reply: tid=0 2024-09-21T06:07:20.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.642+0000 7fc71fe23740 1 client.17583 shutdown 2024-09-21T06:07:20.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.642+0000 7fc71fe23740 1 -- 172.21.15.117:0/890745615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a82362070 msgr2=0x558a8272cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:20.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.642+0000 7fc71fe23740 1 --2- 172.21.15.117:0/890745615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a82362070 0x558a8272cbf0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fc70c02ade0 tx=0x7fc70c02f660 comp rx=0 tx=0).stop 2024-09-21T06:07:20.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.642+0000 7fc71fe23740 1 -- 172.21.15.117:0/890745615 shutdown_connections 2024-09-21T06:07:20.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.642+0000 7fc71fe23740 1 --2- 172.21.15.117:0/890745615 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x558a822f5da0 0x558a82798330 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.642+0000 7fc71fe23740 1 --2- 172.21.15.117:0/890745615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558a82362070 0x558a8272cbf0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.642+0000 7fc71fe23740 1 --2- 172.21.15.117:0/890745615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558a82341390 0x558a8272c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.642+0000 7fc71fe23740 1 --2- 172.21.15.117:0/890745615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a823391b0 0x558a8272c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.642+0000 7fc71fe23740 1 -- 172.21.15.117:0/890745615 >> 172.21.15.117:0/890745615 conn(0x558a82356460 msgr2=0x558a82718ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:20.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.642+0000 7fc6f2ffd640 0 client.17583 ms_handle_reset on v2:172.21.15.138:6836/2540335374 2024-09-21T06:07:20.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.642+0000 7fc71fe23740 1 -- 172.21.15.117:0/890745615 shutdown_connections 2024-09-21T06:07:20.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.642+0000 7fc71fe23740 1 -- 172.21.15.117:0/890745615 wait complete. 2024-09-21T06:07:20.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.642+0000 7fc71fe23740 20 client.17583 trim_cache size 0 max 16384 2024-09-21T06:07:20.663 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:07:20.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.643+0000 7fc7198af640 1 -- 172.21.15.117:0/2942915851 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc6fc0b6680 msgr2=0x7fc6fc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:20.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.643+0000 7fc7198af640 1 --2- 172.21.15.117:0/2942915851 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc6fc0b6680 0x7fc6fc0b8b90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc70c006010 tx=0x7fc70c0095c0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.643+0000 7fc7198af640 1 -- 172.21.15.117:0/2942915851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc714104840 msgr2=0x7fc71419a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:20.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.643+0000 7fc7198af640 1 --2- 172.21.15.117:0/2942915851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc714104840 0x7fc71419a4c0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fc70400d770 tx=0x7fc70400dc30 comp rx=0 tx=0).stop 2024-09-21T06:07:20.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.643+0000 7fc7198af640 1 -- 172.21.15.117:0/2942915851 shutdown_connections 2024-09-21T06:07:20.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.643+0000 7fc7198af640 1 --2- 172.21.15.117:0/2942915851 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc6fc0b6680 0x7fc6fc0b8b90 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.643+0000 7fc7198af640 1 --2- 172.21.15.117:0/2942915851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc71410ddf0 0x7fc71419fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.643+0000 7fc7198af640 1 --2- 172.21.15.117:0/2942915851 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc714105270 0x7fc71419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.643+0000 7fc7198af640 1 --2- 172.21.15.117:0/2942915851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc714104840 0x7fc71419a4c0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:20.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.643+0000 7fc7198af640 1 -- 172.21.15.117:0/2942915851 >> 172.21.15.117:0/2942915851 conn(0x7fc714079730 msgr2=0x7fc714107990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:20.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.643+0000 7fc7198af640 1 -- 172.21.15.117:0/2942915851 shutdown_connections 2024-09-21T06:07:20.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:20.643+0000 7fc7198af640 1 -- 172.21.15.117:0/2942915851 wait complete. 2024-09-21T06:07:21.918 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T06:07:22.180 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:07:22.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:22 smithi117 ceph-mon[30947]: pgmap v635: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 818 B/s wr, 10 op/s 2024-09-21T06:07:22.282 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:22 smithi192 ceph-mon[34700]: pgmap v635: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 818 B/s wr, 10 op/s 2024-09-21T06:07:22.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:22 smithi138 ceph-mon[34391]: pgmap v635: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 818 B/s wr, 10 op/s 2024-09-21T06:07:22.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.696+0000 7f2b4359e640 1 -- 172.21.15.117:0/2129589514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b44108b20 msgr2=0x7f2b4410f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:22.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.696+0000 7f2b4359e640 1 --2- 172.21.15.117:0/2129589514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b44108b20 0x7f2b4410f350 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c009ca0 tx=0x7f2b3c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.696+0000 7f2b4359e640 1 -- 172.21.15.117:0/2129589514 shutdown_connections 2024-09-21T06:07:22.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.696+0000 7f2b4359e640 1 --2- 172.21.15.117:0/2129589514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b44108b20 0x7f2b4410f350 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.696+0000 7f2b4359e640 1 --2- 172.21.15.117:0/2129589514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b441081d0 0x7f2b441085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.696+0000 7f2b4359e640 1 --2- 172.21.15.117:0/2129589514 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b44102890 0x7f2b44102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.696+0000 7f2b4359e640 1 -- 172.21.15.117:0/2129589514 >> 172.21.15.117:0/2129589514 conn(0x7f2b440fe300 msgr2=0x7f2b44100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:22.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.696+0000 7f2b4359e640 1 -- 172.21.15.117:0/2129589514 shutdown_connections 2024-09-21T06:07:22.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.696+0000 7f2b4359e640 1 -- 172.21.15.117:0/2129589514 wait complete. 2024-09-21T06:07:22.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.697+0000 7f2b4359e640 1 Processor -- start 2024-09-21T06:07:22.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.697+0000 7f2b4359e640 1 -- start start 2024-09-21T06:07:22.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.697+0000 7f2b4359e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b44102890 0x7f2b4419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:22.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.697+0000 7f2b4359e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b441081d0 0x7f2b4419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:22.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.697+0000 7f2b4359e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b44108b20 0x7f2b4419faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:22.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.697+0000 7f2b4359e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2b441127f0 con 0x7f2b44102890 2024-09-21T06:07:22.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.697+0000 7f2b4359e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2b44112670 con 0x7f2b441081d0 2024-09-21T06:07:22.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.697+0000 7f2b4359e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2b44112970 con 0x7f2b44108b20 2024-09-21T06:07:22.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.697+0000 7f2b4259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b44102890 0x7f2b4419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:22.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.697+0000 7f2b4259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b44102890 0x7f2b4419a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44332/0 (socket says 172.21.15.117:44332) 2024-09-21T06:07:22.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.697+0000 7f2b4259c640 1 -- 172.21.15.117:0/3440952264 learned_addr learned my addr 172.21.15.117:0/3440952264 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:22.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.697+0000 7f2b42d9d640 1 --2- 172.21.15.117:0/3440952264 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b44108b20 0x7f2b4419faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:22.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.698+0000 7f2b41d9b640 1 --2- 172.21.15.117:0/3440952264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b441081d0 0x7f2b4419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:22.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.698+0000 7f2b4259c640 1 -- 172.21.15.117:0/3440952264 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b44108b20 msgr2=0x7f2b4419faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:22.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.698+0000 7f2b4259c640 1 --2- 172.21.15.117:0/3440952264 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b44108b20 0x7f2b4419faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.698+0000 7f2b4259c640 1 -- 172.21.15.117:0/3440952264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b441081d0 msgr2=0x7f2b4419a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:22.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.698+0000 7f2b4259c640 1 --2- 172.21.15.117:0/3440952264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b441081d0 0x7f2b4419a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.698+0000 7f2b4259c640 1 -- 172.21.15.117:0/3440952264 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b441a0220 con 0x7f2b44102890 2024-09-21T06:07:22.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.698+0000 7f2b4259c640 1 --2- 172.21.15.117:0/3440952264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b44102890 0x7f2b4419a2b0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c00d770 tx=0x7f2b2c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:22.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.698+0000 7f2b3b7fe640 1 -- 172.21.15.117:0/3440952264 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c017070 con 0x7f2b44102890 2024-09-21T06:07:22.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.698+0000 7f2b3b7fe640 1 -- 172.21.15.117:0/3440952264 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b2c0054d0 con 0x7f2b44102890 2024-09-21T06:07:22.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.698+0000 7f2b4359e640 1 -- 172.21.15.117:0/3440952264 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b441a0510 con 0x7f2b44102890 2024-09-21T06:07:22.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.698+0000 7f2b3b7fe640 1 -- 172.21.15.117:0/3440952264 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c004c30 con 0x7f2b44102890 2024-09-21T06:07:22.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.698+0000 7f2b4359e640 1 -- 172.21.15.117:0/3440952264 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b441a6c70 con 0x7f2b44102890 2024-09-21T06:07:22.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.701+0000 7f2b4ed76740 1 Processor -- start 2024-09-21T06:07:22.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.702+0000 7f2b4ed76740 1 -- start start 2024-09-21T06:07:22.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.702+0000 7f2b4ed76740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560099462070 0x560099809860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:22.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.702+0000 7f2b4ed76740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600994391b0 0x560099809da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:22.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.702+0000 7f2b3b7fe640 1 -- 172.21.15.117:0/3440952264 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f2b2c004e10 con 0x7f2b44102890 2024-09-21T06:07:22.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.702+0000 7f2b4ed76740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560099441390 0x56009980a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:22.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.702+0000 7f2b4ed76740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56009977fea0 con 0x560099441390 2024-09-21T06:07:22.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.702+0000 7f2b4ed76740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56009976f2d0 con 0x5600994391b0 2024-09-21T06:07:22.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.702+0000 7f2b4ed76740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5600997730e0 con 0x560099462070 2024-09-21T06:07:22.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.702+0000 7f2b41d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560099441390 0x56009980a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:22.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.702+0000 7f2b41d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560099441390 0x56009980a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44338/0 (socket says 172.21.15.117:44338) 2024-09-21T06:07:22.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.702+0000 7f2b41d9b640 1 -- 172.21.15.117:0/3950012999 learned_addr learned my addr 172.21.15.117:0/3950012999 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:22.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.702+0000 7f2b4259c640 1 --2- 172.21.15.117:0/3950012999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600994391b0 0x560099809da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:22.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.702+0000 7f2b42d9d640 1 --2- 172.21.15.117:0/3950012999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560099462070 0x560099809860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:22.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.703+0000 7f2b41d9b640 1 -- 172.21.15.117:0/3950012999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560099462070 msgr2=0x560099809860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:22.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.703+0000 7f2b41d9b640 1 --2- 172.21.15.117:0/3950012999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560099462070 0x560099809860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.703+0000 7f2b41d9b640 1 -- 172.21.15.117:0/3950012999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600994391b0 msgr2=0x560099809da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:22.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.703+0000 7f2b41d9b640 1 --2- 172.21.15.117:0/3950012999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600994391b0 0x560099809da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.703+0000 7f2b41d9b640 1 -- 172.21.15.117:0/3950012999 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600994cd650 con 0x560099441390 2024-09-21T06:07:22.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.703+0000 7f2b42d9d640 1 --2- 172.21.15.117:0/3950012999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560099462070 0x560099809860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:07:22.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.703+0000 7f2b41d9b640 1 --2- 172.21.15.117:0/3950012999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560099441390 0x56009980a2e0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f2b3400bc30 tx=0x7f2b340107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e56e64adcf71a020 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:22.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.703+0000 7f2b38ff9640 1 -- 172.21.15.117:0/3950012999 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3401c480 con 0x560099441390 2024-09-21T06:07:22.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.703+0000 7f2b3b7fe640 1 --2- 172.21.15.117:0/3440952264 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2b300b6680 0x7f2b300b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:22.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.703+0000 7f2b38ff9640 1 -- 172.21.15.117:0/3950012999 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b3401cad0 con 0x560099441390 2024-09-21T06:07:22.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.703+0000 7f2b38ff9640 1 -- 172.21.15.117:0/3950012999 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b34011a40 con 0x560099441390 2024-09-21T06:07:22.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.703+0000 7f2b42d9d640 1 --2- 172.21.15.117:0/3440952264 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2b300b6680 0x7f2b300b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:22.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.703+0000 7f2b3b7fe640 1 -- 172.21.15.117:0/3440952264 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f2b2c0e7090 con 0x7f2b44102890 2024-09-21T06:07:22.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.704+0000 7f2b4ed76740 1 -- 172.21.15.117:0/3950012999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560099441390 msgr2=0x56009980a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:22.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.704+0000 7f2b42d9d640 1 --2- 172.21.15.117:0/3440952264 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2b300b6680 0x7f2b300b8b90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c02f500 tx=0x7f2b3c009820 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:22.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.704+0000 7f2b4ed76740 1 --2- 172.21.15.117:0/3950012999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560099441390 0x56009980a2e0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f2b3400bc30 tx=0x7f2b340107a0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.704+0000 7f2b4ed76740 1 -- 172.21.15.117:0/3950012999 shutdown_connections 2024-09-21T06:07:22.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.704+0000 7f2b4ed76740 1 --2- 172.21.15.117:0/3950012999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560099441390 0x56009980a2e0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.704+0000 7f2b4ed76740 1 --2- 172.21.15.117:0/3950012999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600994391b0 0x560099809da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.704+0000 7f2b4ed76740 1 --2- 172.21.15.117:0/3950012999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560099462070 0x560099809860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.704+0000 7f2b4ed76740 1 -- 172.21.15.117:0/3950012999 >> 172.21.15.117:0/3950012999 conn(0x560099456460 msgr2=0x560099807750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:22.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.704+0000 7f2b4ed76740 1 -- 172.21.15.117:0/3950012999 shutdown_connections 2024-09-21T06:07:22.711 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.705+0000 7f2b4ed76740 1 -- 172.21.15.117:0/3950012999 wait complete. 2024-09-21T06:07:22.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.705+0000 7f2b4ed76740 1 -- start start 2024-09-21T06:07:22.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.705+0000 7f2b4ed76740 1 Processor -- start 2024-09-21T06:07:22.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.705+0000 7f2b4ed76740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600994391b0 0x5600998192e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:22.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.705+0000 7f2b4ed76740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560099441390 0x560099819820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:22.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.705+0000 7f2b4ed76740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560099462070 0x560099819da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:22.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.705+0000 7f2b4ed76740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5600995d0700 con 0x560099441390 2024-09-21T06:07:22.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.705+0000 7f2b4ed76740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56009970a160 con 0x5600994391b0 2024-09-21T06:07:22.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.705+0000 7f2b4ed76740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56009981c3a0 con 0x560099462070 2024-09-21T06:07:22.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.706+0000 7f2b4ed76740 10 client.0 _ll_register_callbacks cb 0x5600997239e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:22.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.706+0000 7f2b41d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560099441390 0x560099819820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:22.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.706+0000 7f2b41d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560099441390 0x560099819820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44342/0 (socket says 172.21.15.117:44342) 2024-09-21T06:07:22.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.706+0000 7f2b41d9b640 1 -- 172.21.15.117:0/1587252158 learned_addr learned my addr 172.21.15.117:0/1587252158 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:22.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.706+0000 7f2b42d9d640 1 --2- 172.21.15.117:0/1587252158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560099462070 0x560099819da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:22.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.706+0000 7f2b4259c640 1 --2- 172.21.15.117:0/1587252158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600994391b0 0x5600998192e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:22.712 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.706+0000 7f2b41d9b640 1 -- 172.21.15.117:0/1587252158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560099462070 msgr2=0x560099819da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:22.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.706+0000 7f2b41d9b640 1 --2- 172.21.15.117:0/1587252158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560099462070 0x560099819da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.706+0000 7f2b41d9b640 1 -- 172.21.15.117:0/1587252158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600994391b0 msgr2=0x5600998192e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:22.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.706+0000 7f2b41d9b640 1 --2- 172.21.15.117:0/1587252158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600994391b0 0x5600998192e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.706+0000 7f2b41d9b640 1 -- 172.21.15.117:0/1587252158 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560099737c90 con 0x560099441390 2024-09-21T06:07:22.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.706+0000 7f2b41d9b640 1 --2- 172.21.15.117:0/1587252158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560099441390 0x560099819820 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f2b34000c00 tx=0x7f2b3401ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89e5b498e7d2d838 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:22.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.706+0000 7f2b29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:22.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.706+0000 7f2b29ffb640 1 -- 172.21.15.117:0/1587252158 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3402b060 con 0x560099441390 2024-09-21T06:07:22.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.706+0000 7f2b29ffb640 1 -- 172.21.15.117:0/1587252158 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f2b3401c480 con 0x560099441390 2024-09-21T06:07:22.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.706+0000 7f2b4ed76740 1 -- 172.21.15.117:0/1587252158 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b3401cad0 con 0x560099441390 2024-09-21T06:07:22.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.707+0000 7f2b29ffb640 1 -- 172.21.15.117:0/1587252158 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2b3401c6c0 con 0x560099441390 2024-09-21T06:07:22.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.707+0000 7f2b4ed76740 10 client.17595 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:22.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.707+0000 7f2b4ed76740 1 -- 172.21.15.117:0/1587252158 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5600998280b0 con 0x560099441390 2024-09-21T06:07:22.713 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.707+0000 7f2b29ffb640 1 -- 172.21.15.117:0/1587252158 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b3401cad0 con 0x560099441390 2024-09-21T06:07:22.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.707+0000 7f2b29ffb640 1 -- 172.21.15.117:0/1587252158 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2b34011b60 con 0x560099441390 2024-09-21T06:07:22.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.707+0000 7f2b4ed76740 10 client.17595 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:22.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.707+0000 7f2b4ed76740 10 client.17595 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T06:07:22.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.707+0000 7f2b4ed76740 20 client.17595 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:22.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.707+0000 7f2b4ed76740 1 --2- 172.21.15.117:0/1587252158 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x5600994209f0 0x56009982b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:22.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.707+0000 7f2b4ed76740 4 client.17595 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:07:22.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.707+0000 7f2b4ed76740 1 -- 172.21.15.117:0/1587252158 --> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560099781df0 con 0x5600994209f0 2024-09-21T06:07:22.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.707+0000 7f2b4259c640 1 --2- 172.21.15.117:0/1587252158 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x5600994209f0 0x56009982b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:22.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.708+0000 7f2b29ffb640 10 client.17595 ms_handle_connect on v2:172.21.15.192:6834/3530688860 2024-09-21T06:07:22.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.708+0000 7f2b4259c640 1 --2- 172.21.15.117:0/1587252158 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x5600994209f0 0x56009982b410 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7e92d7513aa2bc23 server_cookie=a4f13e6da0b0ca88 in_seq=0 out_seq=0 2024-09-21T06:07:22.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.708+0000 7f2b29ffb640 1 -- 172.21.15.117:0/1587252158 <== mds.3 v2:172.21.15.192:6834/3530688860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560099781df0 con 0x5600994209f0 2024-09-21T06:07:22.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.708+0000 7f2b29ffb640 10 client.17595 handle_command_reply: tid=0 2024-09-21T06:07:22.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.708+0000 7f2b4ed76740 1 client.17595 shutdown 2024-09-21T06:07:22.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.709+0000 7f2b4ed76740 1 -- 172.21.15.117:0/1587252158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560099441390 msgr2=0x560099819820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:22.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.709+0000 7f2b4ed76740 1 --2- 172.21.15.117:0/1587252158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560099441390 0x560099819820 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f2b34000c00 tx=0x7f2b3401ceb0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.709+0000 7f2b4ed76740 1 -- 172.21.15.117:0/1587252158 shutdown_connections 2024-09-21T06:07:22.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.709+0000 7f2b4ed76740 1 --2- 172.21.15.117:0/1587252158 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x5600994209f0 0x56009982b410 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.709+0000 7f2b4ed76740 1 --2- 172.21.15.117:0/1587252158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560099441390 0x560099819820 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.709+0000 7f2b4ed76740 1 --2- 172.21.15.117:0/1587252158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560099462070 0x560099819da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.709+0000 7f2b4ed76740 1 --2- 172.21.15.117:0/1587252158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600994391b0 0x5600998192e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.709+0000 7f2b29ffb640 0 client.17595 ms_handle_reset on v2:172.21.15.192:6834/3530688860 2024-09-21T06:07:22.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.709+0000 7f2b4ed76740 1 -- 172.21.15.117:0/1587252158 >> 172.21.15.117:0/1587252158 conn(0x560099456460 msgr2=0x560099805d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:22.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.709+0000 7f2b4ed76740 1 -- 172.21.15.117:0/1587252158 shutdown_connections 2024-09-21T06:07:22.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.709+0000 7f2b4ed76740 1 -- 172.21.15.117:0/1587252158 wait complete. 2024-09-21T06:07:22.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.709+0000 7f2b4ed76740 20 client.17595 trim_cache size 0 max 16384 2024-09-21T06:07:22.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.722+0000 7f2b4ed76740 1 Processor -- start 2024-09-21T06:07:22.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.722+0000 7f2b4ed76740 1 -- start start 2024-09-21T06:07:22.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.722+0000 7f2b4ed76740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600994391b0 0x56009980b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:22.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.722+0000 7f2b4ed76740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560099441390 0x5600998187f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:22.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.722+0000 7f2b4ed76740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560099462070 0x56009981af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:22.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.722+0000 7f2b4ed76740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2b441ad1b0 con 0x560099441390 2024-09-21T06:07:22.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.722+0000 7f2b4ed76740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2b441aecf0 con 0x5600994391b0 2024-09-21T06:07:22.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.722+0000 7f2b4ed76740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2b441ad880 con 0x560099462070 2024-09-21T06:07:22.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.722+0000 7f2b41d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560099441390 0x5600998187f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:22.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.722+0000 7f2b41d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560099441390 0x5600998187f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44348/0 (socket says 172.21.15.117:44348) 2024-09-21T06:07:22.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.722+0000 7f2b41d9b640 1 -- 172.21.15.117:0/3209943596 learned_addr learned my addr 172.21.15.117:0/3209943596 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:22.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.722+0000 7f2b4259c640 1 --2- 172.21.15.117:0/3209943596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600994391b0 0x56009980b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:22.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.722+0000 7f2b42d9d640 1 --2- 172.21.15.117:0/3209943596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560099462070 0x56009981af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:22.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.723+0000 7f2b41d9b640 1 -- 172.21.15.117:0/3209943596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560099462070 msgr2=0x56009981af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:22.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.723+0000 7f2b41d9b640 1 --2- 172.21.15.117:0/3209943596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560099462070 0x56009981af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.723+0000 7f2b41d9b640 1 -- 172.21.15.117:0/3209943596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600994391b0 msgr2=0x56009980b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:22.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.723+0000 7f2b41d9b640 1 --2- 172.21.15.117:0/3209943596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600994391b0 0x56009980b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.723+0000 7f2b41d9b640 1 -- 172.21.15.117:0/3209943596 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56009980a650 con 0x560099441390 2024-09-21T06:07:22.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.723+0000 7f2b42d9d640 1 --2- 172.21.15.117:0/3209943596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560099462070 0x56009981af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:22.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.723+0000 7f2b4259c640 1 --2- 172.21.15.117:0/3209943596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600994391b0 0x56009980b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:22.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.723+0000 7f2b41d9b640 1 --2- 172.21.15.117:0/3209943596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560099441390 0x5600998187f0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f2b34001eb0 tx=0x7f2b340107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1f01373c98e3d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:22.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.723+0000 7f2b2bfff640 1 -- 172.21.15.117:0/3209943596 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3402b040 con 0x560099441390 2024-09-21T06:07:22.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.723+0000 7f2b2bfff640 1 -- 172.21.15.117:0/3209943596 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b34005870 con 0x560099441390 2024-09-21T06:07:22.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.723+0000 7f2b2bfff640 1 -- 172.21.15.117:0/3209943596 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b34011960 con 0x560099441390 2024-09-21T06:07:22.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.724+0000 7f2b4ed76740 1 -- 172.21.15.117:0/3209943596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560099441390 msgr2=0x5600998187f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:22.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.724+0000 7f2b4ed76740 1 --2- 172.21.15.117:0/3209943596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560099441390 0x5600998187f0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f2b34001eb0 tx=0x7f2b340107a0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.724+0000 7f2b4ed76740 1 -- 172.21.15.117:0/3209943596 shutdown_connections 2024-09-21T06:07:22.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.724+0000 7f2b4ed76740 1 --2- 172.21.15.117:0/3209943596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560099441390 0x5600998187f0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.724+0000 7f2b4ed76740 1 --2- 172.21.15.117:0/3209943596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560099462070 0x56009981af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.724+0000 7f2b4ed76740 1 --2- 172.21.15.117:0/3209943596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600994391b0 0x56009980b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.724+0000 7f2b4ed76740 1 -- 172.21.15.117:0/3209943596 >> 172.21.15.117:0/3209943596 conn(0x560099456460 msgr2=0x56009981a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:22.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.724+0000 7f2b4ed76740 1 -- 172.21.15.117:0/3209943596 shutdown_connections 2024-09-21T06:07:22.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.724+0000 7f2b4ed76740 1 -- 172.21.15.117:0/3209943596 wait complete. 2024-09-21T06:07:22.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.724+0000 7f2b4ed76740 1 -- start start 2024-09-21T06:07:22.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.725+0000 7f2b4ed76740 1 Processor -- start 2024-09-21T06:07:22.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.725+0000 7f2b4ed76740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600994391b0 0x56009981da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:22.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.725+0000 7f2b4ed76740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560099441390 0x56009981df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:22.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.725+0000 7f2b4ed76740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560099462070 0x560099894bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:22.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.725+0000 7f2b4ed76740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2b441ae620 con 0x560099462070 2024-09-21T06:07:22.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.725+0000 7f2b4ed76740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2b441adf50 con 0x5600994391b0 2024-09-21T06:07:22.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.725+0000 7f2b4ed76740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2b441ab190 con 0x560099441390 2024-09-21T06:07:22.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.725+0000 7f2b4ed76740 10 client.0 _ll_register_callbacks cb 0x7f2b2c0eab40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:22.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.725+0000 7f2b42d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560099462070 0x560099894bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:22.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.725+0000 7f2b42d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560099462070 0x560099894bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44354/0 (socket says 172.21.15.117:44354) 2024-09-21T06:07:22.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.725+0000 7f2b42d9d640 1 -- 172.21.15.117:0/1705588038 learned_addr learned my addr 172.21.15.117:0/1705588038 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:22.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.725+0000 7f2b4259c640 1 --2- 172.21.15.117:0/1705588038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600994391b0 0x56009981da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:22.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.725+0000 7f2b41d9b640 1 --2- 172.21.15.117:0/1705588038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560099441390 0x56009981df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:22.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.725+0000 7f2b42d9d640 1 -- 172.21.15.117:0/1705588038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560099441390 msgr2=0x56009981df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:22.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.725+0000 7f2b42d9d640 1 --2- 172.21.15.117:0/1705588038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560099441390 0x56009981df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.725+0000 7f2b42d9d640 1 -- 172.21.15.117:0/1705588038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600994391b0 msgr2=0x56009981da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:22.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.725+0000 7f2b42d9d640 1 --2- 172.21.15.117:0/1705588038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600994391b0 0x56009981da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.725+0000 7f2b42d9d640 1 -- 172.21.15.117:0/1705588038 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600998950f0 con 0x560099462070 2024-09-21T06:07:22.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.725+0000 7f2b4259c640 1 --2- 172.21.15.117:0/1705588038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600994391b0 0x56009981da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:22.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.726+0000 7f2b42d9d640 1 --2- 172.21.15.117:0/1705588038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560099462070 0x560099894bb0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c009ca0 tx=0x7f2b3c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9323da4de1156705 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:22.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.726+0000 7f2b29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:22.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.726+0000 7f2b29ffb640 1 -- 172.21.15.117:0/1705588038 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c057060 con 0x560099462070 2024-09-21T06:07:22.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.726+0000 7f2b29ffb640 1 -- 172.21.15.117:0/1705588038 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f2b3c004040 con 0x560099462070 2024-09-21T06:07:22.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.726+0000 7f2b4ed76740 1 -- 172.21.15.117:0/1705588038 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b34005870 con 0x560099462070 2024-09-21T06:07:22.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.726+0000 7f2b29ffb640 1 -- 172.21.15.117:0/1705588038 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2b3c004240 con 0x560099462070 2024-09-21T06:07:22.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.726+0000 7f2b4ed76740 10 client.17601 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:22.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.726+0000 7f2b4ed76740 1 -- 172.21.15.117:0/1705588038 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560099896210 con 0x560099462070 2024-09-21T06:07:22.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.726+0000 7f2b29ffb640 1 -- 172.21.15.117:0/1705588038 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b34005870 con 0x560099462070 2024-09-21T06:07:22.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.726+0000 7f2b29ffb640 1 -- 172.21.15.117:0/1705588038 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2b3c02fd10 con 0x560099462070 2024-09-21T06:07:22.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.726+0000 7f2b4ed76740 10 client.17601 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:22.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.726+0000 7f2b4ed76740 10 client.17601 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T06:07:22.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.726+0000 7f2b4ed76740 20 client.17601 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:22.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.726+0000 7f2b4ed76740 1 --2- 172.21.15.117:0/1705588038 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x5600993f5da0 0x5600998989d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:22.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.726+0000 7f2b4ed76740 4 client.17601 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:07:22.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.727+0000 7f2b4ed76740 1 -- 172.21.15.117:0/1705588038 --> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56009980bec0 con 0x5600993f5da0 2024-09-21T06:07:22.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.727+0000 7f2b4259c640 1 --2- 172.21.15.117:0/1705588038 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x5600993f5da0 0x5600998989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:22.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.727+0000 7f2b4259c640 1 --2- 172.21.15.117:0/1705588038 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x5600993f5da0 0x5600998989d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c90190b367830d0f server_cookie=56364eabb5f620c0 in_seq=0 out_seq=0 2024-09-21T06:07:22.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.727+0000 7f2b29ffb640 10 client.17601 ms_handle_connect on v2:172.21.15.192:6834/3530688860 2024-09-21T06:07:22.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.727+0000 7f2b29ffb640 1 -- 172.21.15.117:0/1705588038 <== mds.3 v2:172.21.15.192:6834/3530688860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56009980bec0 con 0x5600993f5da0 2024-09-21T06:07:22.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.727+0000 7f2b29ffb640 10 client.17601 handle_command_reply: tid=0 2024-09-21T06:07:22.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.727+0000 7f2b4ed76740 1 client.17601 shutdown 2024-09-21T06:07:22.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.727+0000 7f2b4ed76740 1 -- 172.21.15.117:0/1705588038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560099462070 msgr2=0x560099894bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:22.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.727+0000 7f2b4ed76740 1 --2- 172.21.15.117:0/1705588038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560099462070 0x560099894bb0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c009ca0 tx=0x7f2b3c0095c0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.727+0000 7f2b4ed76740 1 -- 172.21.15.117:0/1705588038 shutdown_connections 2024-09-21T06:07:22.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.727+0000 7f2b4ed76740 1 --2- 172.21.15.117:0/1705588038 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x5600993f5da0 0x5600998989d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.728+0000 7f2b4ed76740 1 --2- 172.21.15.117:0/1705588038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560099462070 0x560099894bb0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.728+0000 7f2b29ffb640 0 client.17601 ms_handle_reset on v2:172.21.15.192:6834/3530688860 2024-09-21T06:07:22.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.728+0000 7f2b4ed76740 1 --2- 172.21.15.117:0/1705588038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560099441390 0x56009981df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.728+0000 7f2b4ed76740 1 --2- 172.21.15.117:0/1705588038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5600994391b0 0x56009981da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.728+0000 7f2b4ed76740 1 -- 172.21.15.117:0/1705588038 >> 172.21.15.117:0/1705588038 conn(0x560099456460 msgr2=0x56009980b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:22.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.728+0000 7f2b4ed76740 1 -- 172.21.15.117:0/1705588038 shutdown_connections 2024-09-21T06:07:22.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.728+0000 7f2b4ed76740 1 -- 172.21.15.117:0/1705588038 wait complete. 2024-09-21T06:07:22.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.728+0000 7f2b4ed76740 20 client.17601 trim_cache size 0 max 16384 2024-09-21T06:07:22.735 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:07:22.735 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.728+0000 7f2b4359e640 1 -- 172.21.15.117:0/3440952264 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2b300b6680 msgr2=0x7f2b300b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:22.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.728+0000 7f2b4359e640 1 --2- 172.21.15.117:0/3440952264 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2b300b6680 0x7f2b300b8b90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c02f500 tx=0x7f2b3c009820 comp rx=0 tx=0).stop 2024-09-21T06:07:22.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.728+0000 7f2b4359e640 1 -- 172.21.15.117:0/3440952264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b44102890 msgr2=0x7f2b4419a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:22.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.728+0000 7f2b4359e640 1 --2- 172.21.15.117:0/3440952264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b44102890 0x7f2b4419a2b0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c00d770 tx=0x7f2b2c00dc30 comp rx=0 tx=0).stop 2024-09-21T06:07:22.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.728+0000 7f2b4359e640 1 -- 172.21.15.117:0/3440952264 shutdown_connections 2024-09-21T06:07:22.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.728+0000 7f2b4359e640 1 --2- 172.21.15.117:0/3440952264 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2b300b6680 0x7f2b300b8b90 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.728+0000 7f2b4359e640 1 --2- 172.21.15.117:0/3440952264 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b44108b20 0x7f2b4419faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.728+0000 7f2b4359e640 1 --2- 172.21.15.117:0/3440952264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b441081d0 0x7f2b4419a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.728+0000 7f2b4359e640 1 --2- 172.21.15.117:0/3440952264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b44102890 0x7f2b4419a2b0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:22.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.728+0000 7f2b4359e640 1 -- 172.21.15.117:0/3440952264 >> 172.21.15.117:0/3440952264 conn(0x7f2b440fe300 msgr2=0x7f2b4406bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:22.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.728+0000 7f2b4359e640 1 -- 172.21.15.117:0/3440952264 shutdown_connections 2024-09-21T06:07:22.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:22.728+0000 7f2b4359e640 1 -- 172.21.15.117:0/3440952264 wait complete. 2024-09-21T06:07:23.453 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T06:07:23.700 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:07:24.095 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:24 smithi117 ceph-mon[30947]: pgmap v636: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 10 op/s 2024-09-21T06:07:24.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.184+0000 7f6edf2b5640 1 -- 172.21.15.117:0/3255726734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ed81094a0 msgr2=0x7f6ed81098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:24.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.184+0000 7f6edf2b5640 1 --2- 172.21.15.117:0/3255726734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ed81094a0 0x7f6ed81098b0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8009900 tx=0x7f6ec802a410 comp rx=0 tx=0).stop 2024-09-21T06:07:24.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.184+0000 7f6edf2b5640 1 -- 172.21.15.117:0/3255726734 shutdown_connections 2024-09-21T06:07:24.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.184+0000 7f6edf2b5640 1 --2- 172.21.15.117:0/3255726734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ed81094a0 0x7f6ed81098b0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.184+0000 7f6edf2b5640 1 --2- 172.21.15.117:0/3255726734 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ed8109df0 0x7f6ed8110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.184+0000 7f6edf2b5640 1 --2- 172.21.15.117:0/3255726734 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ed8103b60 0x7f6ed8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.184+0000 7f6edf2b5640 1 -- 172.21.15.117:0/3255726734 >> 172.21.15.117:0/3255726734 conn(0x7f6ed807b850 msgr2=0x7f6ed807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:24.206 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.185+0000 7f6edf2b5640 1 -- 172.21.15.117:0/3255726734 shutdown_connections 2024-09-21T06:07:24.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.185+0000 7f6edf2b5640 1 -- 172.21.15.117:0/3255726734 wait complete. 2024-09-21T06:07:24.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.185+0000 7f6edf2b5640 1 Processor -- start 2024-09-21T06:07:24.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.185+0000 7f6edf2b5640 1 -- start start 2024-09-21T06:07:24.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.185+0000 7f6edf2b5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ed8103b60 0x7f6ed819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:24.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.186+0000 7f6edf2b5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ed81094a0 0x7f6ed819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:24.207 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.186+0000 7f6edf2b5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ed8109df0 0x7f6ed819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:24.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.186+0000 7f6edf2b5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed8113ad0 con 0x7f6ed81094a0 2024-09-21T06:07:24.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.186+0000 7f6edf2b5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed8113950 con 0x7f6ed8109df0 2024-09-21T06:07:24.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.186+0000 7f6edf2b5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed8113c50 con 0x7f6ed8103b60 2024-09-21T06:07:24.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.186+0000 7f6edc850640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ed81094a0 0x7f6ed819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:24.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.186+0000 7f6edc850640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ed81094a0 0x7f6ed819aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44358/0 (socket says 172.21.15.117:44358) 2024-09-21T06:07:24.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.186+0000 7f6edc850640 1 -- 172.21.15.117:0/1574895979 learned_addr learned my addr 172.21.15.117:0/1574895979 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:24.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.186+0000 7f6edd051640 1 --2- 172.21.15.117:0/1574895979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ed8103b60 0x7f6ed819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:24.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.186+0000 7f6edd852640 1 --2- 172.21.15.117:0/1574895979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ed8109df0 0x7f6ed819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:24.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.186+0000 7f6edc850640 1 -- 172.21.15.117:0/1574895979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ed8103b60 msgr2=0x7f6ed819a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:24.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.186+0000 7f6edc850640 1 --2- 172.21.15.117:0/1574895979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ed8103b60 0x7f6ed819a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.186+0000 7f6edc850640 1 -- 172.21.15.117:0/1574895979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ed8109df0 msgr2=0x7f6ed819fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:24.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.186+0000 7f6edc850640 1 --2- 172.21.15.117:0/1574895979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ed8109df0 0x7f6ed819fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.186+0000 7f6edc850640 1 -- 172.21.15.117:0/1574895979 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed81a0430 con 0x7f6ed81094a0 2024-09-21T06:07:24.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.186+0000 7f6edc850640 1 --2- 172.21.15.117:0/1574895979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ed81094a0 0x7f6ed819aa60 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f6ec802a930 tx=0x7f6ec8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:24.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.186+0000 7f6ece7fc640 1 -- 172.21.15.117:0/1574895979 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec803f030 con 0x7f6ed81094a0 2024-09-21T06:07:24.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.186+0000 7f6edf2b5640 1 -- 172.21.15.117:0/1574895979 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed81a06c0 con 0x7f6ed81094a0 2024-09-21T06:07:24.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.186+0000 7f6ece7fc640 1 -- 172.21.15.117:0/1574895979 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ec8003aa0 con 0x7f6ed81094a0 2024-09-21T06:07:24.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.186+0000 7f6ece7fc640 1 -- 172.21.15.117:0/1574895979 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8035070 con 0x7f6ed81094a0 2024-09-21T06:07:24.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.186+0000 7f6edf2b5640 1 -- 172.21.15.117:0/1574895979 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed81a6e80 con 0x7f6ed81094a0 2024-09-21T06:07:24.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.189+0000 7f6ece7fc640 1 -- 172.21.15.117:0/1574895979 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f6ec8042030 con 0x7f6ed81094a0 2024-09-21T06:07:24.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.189+0000 7f6ee5829740 1 Processor -- start 2024-09-21T06:07:24.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.189+0000 7f6ee5829740 1 -- start start 2024-09-21T06:07:24.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.190+0000 7f6ee5829740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55721aeaa070 0x55721b2516a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:24.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.190+0000 7f6ee5829740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55721ae811b0 0x55721b251be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:24.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.190+0000 7f6ee5829740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721ae89390 0x55721b252120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:24.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.190+0000 7f6ee5829740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55721b1c7ea0 con 0x55721ae89390 2024-09-21T06:07:24.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.190+0000 7f6ee5829740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55721b1b72d0 con 0x55721ae811b0 2024-09-21T06:07:24.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.190+0000 7f6ee5829740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55721b1bb0e0 con 0x55721aeaa070 2024-09-21T06:07:24.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.190+0000 7f6edc850640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721ae89390 0x55721b252120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:24.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.190+0000 7f6edc850640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721ae89390 0x55721b252120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44362/0 (socket says 172.21.15.117:44362) 2024-09-21T06:07:24.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.190+0000 7f6edc850640 1 -- 172.21.15.117:0/1616176653 learned_addr learned my addr 172.21.15.117:0/1616176653 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:24.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.190+0000 7f6ece7fc640 1 --2- 172.21.15.117:0/1574895979 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6ec40b6680 0x7f6ec40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:24.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.190+0000 7f6edd051640 1 --2- 172.21.15.117:0/1616176653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55721ae811b0 0x55721b251be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:24.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.190+0000 7f6edd852640 1 --2- 172.21.15.117:0/1616176653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55721aeaa070 0x55721b2516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:24.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.190+0000 7f6edd852640 1 --2- 172.21.15.117:0/1574895979 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6ec40b6680 0x7f6ec40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:24.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.190+0000 7f6ece7fc640 1 -- 172.21.15.117:0/1574895979 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f6ec8107dc0 con 0x7f6ed81094a0 2024-09-21T06:07:24.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.190+0000 7f6edc850640 1 -- 172.21.15.117:0/1616176653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55721aeaa070 msgr2=0x55721b2516a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:24.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.190+0000 7f6edc850640 1 --2- 172.21.15.117:0/1616176653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55721aeaa070 0x55721b2516a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.190+0000 7f6edc850640 1 -- 172.21.15.117:0/1616176653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55721ae811b0 msgr2=0x55721b251be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:24.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.190+0000 7f6edc850640 1 --2- 172.21.15.117:0/1616176653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55721ae811b0 0x55721b251be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.190+0000 7f6edc850640 1 -- 172.21.15.117:0/1616176653 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55721af15650 con 0x55721ae89390 2024-09-21T06:07:24.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.191+0000 7f6edc850640 1 --2- 172.21.15.117:0/1616176653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721ae89390 0x55721b252120 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f6ec81054d0 tx=0x7f6ec8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94968c42fd4d3853 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:24.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.191+0000 7f6edd852640 1 --2- 172.21.15.117:0/1574895979 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6ec40b6680 0x7f6ec40b8b90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0005c40 tx=0x7f6ed0005bd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:24.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.191+0000 7f6ec37fe640 1 -- 172.21.15.117:0/1616176653 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8049070 con 0x55721ae89390 2024-09-21T06:07:24.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.191+0000 7f6ec37fe640 1 -- 172.21.15.117:0/1616176653 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ec803b460 con 0x55721ae89390 2024-09-21T06:07:24.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.191+0000 7f6ec37fe640 1 -- 172.21.15.117:0/1616176653 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec80cd770 con 0x55721ae89390 2024-09-21T06:07:24.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.192+0000 7f6ee5829740 1 -- 172.21.15.117:0/1616176653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721ae89390 msgr2=0x55721b252120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:24.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.192+0000 7f6ee5829740 1 --2- 172.21.15.117:0/1616176653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721ae89390 0x55721b252120 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f6ec81054d0 tx=0x7f6ec8000bc0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.192+0000 7f6ee5829740 1 -- 172.21.15.117:0/1616176653 shutdown_connections 2024-09-21T06:07:24.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.192+0000 7f6ee5829740 1 --2- 172.21.15.117:0/1616176653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721ae89390 0x55721b252120 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.192+0000 7f6ee5829740 1 --2- 172.21.15.117:0/1616176653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55721ae811b0 0x55721b251be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.192+0000 7f6ee5829740 1 --2- 172.21.15.117:0/1616176653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55721aeaa070 0x55721b2516a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.192+0000 7f6ee5829740 1 -- 172.21.15.117:0/1616176653 >> 172.21.15.117:0/1616176653 conn(0x55721ae9e460 msgr2=0x55721b24f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:24.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.192+0000 7f6ee5829740 1 -- 172.21.15.117:0/1616176653 shutdown_connections 2024-09-21T06:07:24.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.192+0000 7f6ee5829740 1 -- 172.21.15.117:0/1616176653 wait complete. 2024-09-21T06:07:24.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.192+0000 7f6ee5829740 1 -- start start 2024-09-21T06:07:24.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.192+0000 7f6ee5829740 1 Processor -- start 2024-09-21T06:07:24.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.193+0000 7f6ee5829740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55721ae811b0 0x55721b25cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:24.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.193+0000 7f6ee5829740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721ae89390 0x55721b25d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:24.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.193+0000 7f6ee5829740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55721aeaa070 0x55721b268db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:24.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.193+0000 7f6ee5829740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55721b018700 con 0x55721ae89390 2024-09-21T06:07:24.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.193+0000 7f6ee5829740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55721b152160 con 0x55721aeaa070 2024-09-21T06:07:24.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.193+0000 7f6ee5829740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55721b1c7ea0 con 0x55721ae811b0 2024-09-21T06:07:24.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.193+0000 7f6ee5829740 10 client.0 _ll_register_callbacks cb 0x55721b16b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:24.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.193+0000 7f6edc850640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721ae89390 0x55721b25d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:24.219 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.193+0000 7f6edc850640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721ae89390 0x55721b25d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44370/0 (socket says 172.21.15.117:44370) 2024-09-21T06:07:24.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.193+0000 7f6edc850640 1 -- 172.21.15.117:0/124773511 learned_addr learned my addr 172.21.15.117:0/124773511 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:24.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.193+0000 7f6edd852640 1 --2- 172.21.15.117:0/124773511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55721aeaa070 0x55721b268db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:24.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.193+0000 7f6edd051640 1 --2- 172.21.15.117:0/124773511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55721ae811b0 0x55721b25cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:24.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.193+0000 7f6edc850640 1 -- 172.21.15.117:0/124773511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55721ae811b0 msgr2=0x55721b25cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:24.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.193+0000 7f6edc850640 1 --2- 172.21.15.117:0/124773511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55721ae811b0 0x55721b25cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.193+0000 7f6edc850640 1 -- 172.21.15.117:0/124773511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55721aeaa070 msgr2=0x55721b268db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:24.220 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.193+0000 7f6edc850640 1 --2- 172.21.15.117:0/124773511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55721aeaa070 0x55721b268db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.193+0000 7f6edc850640 1 -- 172.21.15.117:0/124773511 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55721b17fc90 con 0x55721ae89390 2024-09-21T06:07:24.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.193+0000 7f6edd051640 1 --2- 172.21.15.117:0/124773511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55721ae811b0 0x55721b25cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:24.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.193+0000 7f6edc850640 1 --2- 172.21.15.117:0/124773511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721ae89390 0x55721b25d270 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8003f90 tx=0x7f6ec80cd440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d2fa2e7107732 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:24.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.193+0000 7f6ec0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:24.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.193+0000 7f6ec0ff9640 1 -- 172.21.15.117:0/124773511 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8049050 con 0x55721ae89390 2024-09-21T06:07:24.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.193+0000 7f6ec0ff9640 1 -- 172.21.15.117:0/124773511 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f6ec80f10c0 con 0x55721ae89390 2024-09-21T06:07:24.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.193+0000 7f6ee5829740 1 -- 172.21.15.117:0/124773511 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55721b1c9df0 con 0x55721ae89390 2024-09-21T06:07:24.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.194+0000 7f6ec0ff9640 1 -- 172.21.15.117:0/124773511 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6ec80cd5b0 con 0x55721ae89390 2024-09-21T06:07:24.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.194+0000 7f6ee5829740 10 client.17613 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:24.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.194+0000 7f6ee5829740 1 -- 172.21.15.117:0/124773511 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55721b26b7a0 con 0x55721ae89390 2024-09-21T06:07:24.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.194+0000 7f6ec0ff9640 1 -- 172.21.15.117:0/124773511 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55721b1c9df0 con 0x55721ae89390 2024-09-21T06:07:24.221 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.194+0000 7f6ec0ff9640 1 -- 172.21.15.117:0/124773511 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6ec80d71f0 con 0x55721ae89390 2024-09-21T06:07:24.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.194+0000 7f6ee5829740 10 client.17613 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:24.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.194+0000 7f6ee5829740 10 client.17613 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:07:24.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.194+0000 7f6ee5829740 20 client.17613 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:24.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.194+0000 7f6ee5829740 1 --2- 172.21.15.117:0/124773511 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55721ae689f0 0x55721b26eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:24.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.194+0000 7f6ee5829740 4 client.17613 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:07:24.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.194+0000 7f6ee5829740 1 -- 172.21.15.117:0/124773511 --> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55721b1c58a0 con 0x55721ae689f0 2024-09-21T06:07:24.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.194+0000 7f6edd051640 1 --2- 172.21.15.117:0/124773511 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55721ae689f0 0x55721b26eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:24.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.195+0000 7f6edd051640 1 --2- 172.21.15.117:0/124773511 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55721ae689f0 0x55721b26eb00 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=477b451ebe01e63c server_cookie=3cb390496ebe134f in_seq=0 out_seq=0 2024-09-21T06:07:24.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.195+0000 7f6ec0ff9640 10 client.17613 ms_handle_connect on v2:172.21.15.192:6836/2683860842 2024-09-21T06:07:24.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.195+0000 7f6ec0ff9640 1 -- 172.21.15.117:0/124773511 <== mds.2 v2:172.21.15.192:6836/2683860842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55721b1c58a0 con 0x55721ae689f0 2024-09-21T06:07:24.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.195+0000 7f6ec0ff9640 10 client.17613 handle_command_reply: tid=0 2024-09-21T06:07:24.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.195+0000 7f6ee5829740 1 client.17613 shutdown 2024-09-21T06:07:24.222 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.196+0000 7f6ee5829740 1 -- 172.21.15.117:0/124773511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721ae89390 msgr2=0x55721b25d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:24.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.196+0000 7f6ee5829740 1 --2- 172.21.15.117:0/124773511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721ae89390 0x55721b25d270 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8003f90 tx=0x7f6ec80cd440 comp rx=0 tx=0).stop 2024-09-21T06:07:24.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.196+0000 7f6ee5829740 1 -- 172.21.15.117:0/124773511 shutdown_connections 2024-09-21T06:07:24.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.196+0000 7f6ee5829740 1 --2- 172.21.15.117:0/124773511 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55721ae689f0 0x55721b26eb00 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.196+0000 7f6ee5829740 1 --2- 172.21.15.117:0/124773511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721ae89390 0x55721b25d270 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.196+0000 7f6ec0ff9640 0 client.17613 ms_handle_reset on v2:172.21.15.192:6836/2683860842 2024-09-21T06:07:24.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.196+0000 7f6ee5829740 1 --2- 172.21.15.117:0/124773511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55721aeaa070 0x55721b268db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.196+0000 7f6ee5829740 1 --2- 172.21.15.117:0/124773511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55721ae811b0 0x55721b25cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.196+0000 7f6ee5829740 1 -- 172.21.15.117:0/124773511 >> 172.21.15.117:0/124773511 conn(0x55721ae9e460 msgr2=0x55721b254830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:24.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.196+0000 7f6ee5829740 1 -- 172.21.15.117:0/124773511 shutdown_connections 2024-09-21T06:07:24.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.196+0000 7f6ee5829740 1 -- 172.21.15.117:0/124773511 wait complete. 2024-09-21T06:07:24.223 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.196+0000 7f6ee5829740 20 client.17613 trim_cache size 0 max 16384 2024-09-21T06:07:24.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.209+0000 7f6ee5829740 1 Processor -- start 2024-09-21T06:07:24.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.209+0000 7f6ee5829740 1 -- start start 2024-09-21T06:07:24.224 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.209+0000 7f6ee5829740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55721ae811b0 0x55721b25e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:24.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.209+0000 7f6ee5829740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721ae89390 0x55721b24ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:24.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.209+0000 7f6ee5829740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55721aeaa070 0x55721b24f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:24.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.209+0000 7f6ee5829740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed81af660 con 0x55721ae89390 2024-09-21T06:07:24.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.209+0000 7f6ee5829740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed81b11a0 con 0x55721aeaa070 2024-09-21T06:07:24.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.209+0000 7f6ee5829740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed81afd30 con 0x55721ae811b0 2024-09-21T06:07:24.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.209+0000 7f6edc850640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721ae89390 0x55721b24ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:24.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.209+0000 7f6edc850640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721ae89390 0x55721b24ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44380/0 (socket says 172.21.15.117:44380) 2024-09-21T06:07:24.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.209+0000 7f6edc850640 1 -- 172.21.15.117:0/1532754776 learned_addr learned my addr 172.21.15.117:0/1532754776 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:24.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.210+0000 7f6edd852640 1 --2- 172.21.15.117:0/1532754776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55721aeaa070 0x55721b24f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:24.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.210+0000 7f6edc850640 1 -- 172.21.15.117:0/1532754776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55721ae811b0 msgr2=0x55721b25e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:24.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.210+0000 7f6edc850640 1 --2- 172.21.15.117:0/1532754776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55721ae811b0 0x55721b25e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.210+0000 7f6edc850640 1 -- 172.21.15.117:0/1532754776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55721aeaa070 msgr2=0x55721b24f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:24.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.210+0000 7f6edc850640 1 --2- 172.21.15.117:0/1532754776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55721aeaa070 0x55721b24f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.210+0000 7f6edc850640 1 -- 172.21.15.117:0/1532754776 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55721b250350 con 0x55721ae89390 2024-09-21T06:07:24.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.210+0000 7f6edc850640 1 --2- 172.21.15.117:0/1532754776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721ae89390 0x55721b24ebc0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f6ec80cd960 tx=0x7f6ec8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=648d93bf8389633e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:24.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.210+0000 7f6ec2ffd640 1 -- 172.21.15.117:0/1532754776 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8049020 con 0x55721ae89390 2024-09-21T06:07:24.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.210+0000 7f6ec2ffd640 1 -- 172.21.15.117:0/1532754776 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ec810b3e0 con 0x55721ae89390 2024-09-21T06:07:24.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.210+0000 7f6ec2ffd640 1 -- 172.21.15.117:0/1532754776 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec80ce360 con 0x55721ae89390 2024-09-21T06:07:24.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.211+0000 7f6ee5829740 1 -- 172.21.15.117:0/1532754776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721ae89390 msgr2=0x55721b24ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:24.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.211+0000 7f6ee5829740 1 --2- 172.21.15.117:0/1532754776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721ae89390 0x55721b24ebc0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f6ec80cd960 tx=0x7f6ec8000bc0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.211+0000 7f6ee5829740 1 -- 172.21.15.117:0/1532754776 shutdown_connections 2024-09-21T06:07:24.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.211+0000 7f6ee5829740 1 --2- 172.21.15.117:0/1532754776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721ae89390 0x55721b24ebc0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.211+0000 7f6ee5829740 1 --2- 172.21.15.117:0/1532754776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55721aeaa070 0x55721b24f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.211+0000 7f6ee5829740 1 --2- 172.21.15.117:0/1532754776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55721ae811b0 0x55721b25e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.211+0000 7f6ee5829740 1 -- 172.21.15.117:0/1532754776 >> 172.21.15.117:0/1532754776 conn(0x55721ae9e460 msgr2=0x55721b25e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:24.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.211+0000 7f6ee5829740 1 -- 172.21.15.117:0/1532754776 shutdown_connections 2024-09-21T06:07:24.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.211+0000 7f6ee5829740 1 -- 172.21.15.117:0/1532754776 wait complete. 2024-09-21T06:07:24.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.211+0000 7f6ee5829740 1 -- start start 2024-09-21T06:07:24.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.211+0000 7f6ee5829740 1 Processor -- start 2024-09-21T06:07:24.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.212+0000 7f6ee5829740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55721ae811b0 0x55721b265250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:24.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.212+0000 7f6ee5829740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721ae89390 0x55721b260700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:24.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.212+0000 7f6ee5829740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55721aeaa070 0x55721b260c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:24.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.212+0000 7f6ee5829740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed81b0ad0 con 0x55721ae89390 2024-09-21T06:07:24.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.212+0000 7f6ee5829740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed81b0400 con 0x55721aeaa070 2024-09-21T06:07:24.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.212+0000 7f6ee5829740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed81ad660 con 0x55721ae811b0 2024-09-21T06:07:24.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.212+0000 7f6ee5829740 10 client.0 _ll_register_callbacks cb 0x7f6ed40093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:24.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.212+0000 7f6edc850640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721ae89390 0x55721b260700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:24.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.212+0000 7f6edc850640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721ae89390 0x55721b260700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44390/0 (socket says 172.21.15.117:44390) 2024-09-21T06:07:24.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.212+0000 7f6edc850640 1 -- 172.21.15.117:0/1690029848 learned_addr learned my addr 172.21.15.117:0/1690029848 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:24.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.212+0000 7f6edd051640 1 --2- 172.21.15.117:0/1690029848 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55721ae811b0 0x55721b265250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:24.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.212+0000 7f6edd852640 1 --2- 172.21.15.117:0/1690029848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55721aeaa070 0x55721b260c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:24.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.212+0000 7f6edc850640 1 -- 172.21.15.117:0/1690029848 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55721ae811b0 msgr2=0x55721b265250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:24.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.212+0000 7f6edc850640 1 --2- 172.21.15.117:0/1690029848 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55721ae811b0 0x55721b265250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.212+0000 7f6edc850640 1 -- 172.21.15.117:0/1690029848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55721aeaa070 msgr2=0x55721b260c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:24.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.212+0000 7f6edc850640 1 --2- 172.21.15.117:0/1690029848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55721aeaa070 0x55721b260c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.212+0000 7f6edc850640 1 -- 172.21.15.117:0/1690029848 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55721b261180 con 0x55721ae89390 2024-09-21T06:07:24.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.212+0000 7f6edd852640 1 --2- 172.21.15.117:0/1690029848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55721aeaa070 0x55721b260c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:24.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.212+0000 7f6edc850640 1 --2- 172.21.15.117:0/1690029848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721ae89390 0x55721b260700 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f6ec802a450 tx=0x7f6ec80cd8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22fefe6299e2972b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:24.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.212+0000 7f6ec0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:24.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.213+0000 7f6ec0ff9640 1 -- 172.21.15.117:0/1690029848 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8049030 con 0x55721ae89390 2024-09-21T06:07:24.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.213+0000 7f6ec0ff9640 1 -- 172.21.15.117:0/1690029848 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f6ec80ccf40 con 0x55721ae89390 2024-09-21T06:07:24.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.213+0000 7f6ee5829740 1 -- 172.21.15.117:0/1690029848 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55721b1c2cd0 con 0x55721ae89390 2024-09-21T06:07:24.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.213+0000 7f6ec0ff9640 1 -- 172.21.15.117:0/1690029848 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6ec80f5200 con 0x55721ae89390 2024-09-21T06:07:24.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.213+0000 7f6ee5829740 10 client.17619 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:24.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.213+0000 7f6ee5829740 1 -- 172.21.15.117:0/1690029848 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55721b261180 con 0x55721ae89390 2024-09-21T06:07:24.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.213+0000 7f6ec0ff9640 1 -- 172.21.15.117:0/1690029848 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55721b1c2cd0 con 0x55721ae89390 2024-09-21T06:07:24.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.213+0000 7f6ec0ff9640 1 -- 172.21.15.117:0/1690029848 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6ec80c95b0 con 0x55721ae89390 2024-09-21T06:07:24.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.213+0000 7f6ee5829740 10 client.17619 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:24.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.213+0000 7f6ee5829740 10 client.17619 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:07:24.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.213+0000 7f6ee5829740 20 client.17619 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:24.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.213+0000 7f6ee5829740 1 --2- 172.21.15.117:0/1690029848 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55721ae3dda0 0x55721b2df790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:24.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.213+0000 7f6ee5829740 4 client.17619 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:07:24.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.213+0000 7f6ee5829740 1 -- 172.21.15.117:0/1690029848 --> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55721b138aa0 con 0x55721ae3dda0 2024-09-21T06:07:24.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.213+0000 7f6edd051640 1 --2- 172.21.15.117:0/1690029848 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55721ae3dda0 0x55721b2df790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:24.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.214+0000 7f6ec0ff9640 10 client.17619 ms_handle_connect on v2:172.21.15.192:6836/2683860842 2024-09-21T06:07:24.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.214+0000 7f6edd051640 1 --2- 172.21.15.117:0/1690029848 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55721ae3dda0 0x55721b2df790 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cd9579716a5d92ad server_cookie=9c46a40a525f3661 in_seq=0 out_seq=0 2024-09-21T06:07:24.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.214+0000 7f6ec0ff9640 1 -- 172.21.15.117:0/1690029848 <== mds.2 v2:172.21.15.192:6836/2683860842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55721b138aa0 con 0x55721ae3dda0 2024-09-21T06:07:24.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.214+0000 7f6ec0ff9640 10 client.17619 handle_command_reply: tid=0 2024-09-21T06:07:24.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.214+0000 7f6ee5829740 1 client.17619 shutdown 2024-09-21T06:07:24.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.214+0000 7f6ee5829740 1 -- 172.21.15.117:0/1690029848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721ae89390 msgr2=0x55721b260700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:24.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.214+0000 7f6ee5829740 1 --2- 172.21.15.117:0/1690029848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721ae89390 0x55721b260700 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f6ec802a450 tx=0x7f6ec80cd8c0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.214+0000 7f6ee5829740 1 -- 172.21.15.117:0/1690029848 shutdown_connections 2024-09-21T06:07:24.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.214+0000 7f6ee5829740 1 --2- 172.21.15.117:0/1690029848 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55721ae3dda0 0x55721b2df790 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.214+0000 7f6ee5829740 1 --2- 172.21.15.117:0/1690029848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55721ae89390 0x55721b260700 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.214+0000 7f6ee5829740 1 --2- 172.21.15.117:0/1690029848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55721aeaa070 0x55721b260c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.214+0000 7f6ee5829740 1 --2- 172.21.15.117:0/1690029848 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55721ae811b0 0x55721b265250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.214+0000 7f6ec0ff9640 0 client.17619 ms_handle_reset on v2:172.21.15.192:6836/2683860842 2024-09-21T06:07:24.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.214+0000 7f6ee5829740 1 -- 172.21.15.117:0/1690029848 >> 172.21.15.117:0/1690029848 conn(0x55721ae9e460 msgr2=0x55721b252580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:24.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.214+0000 7f6ee5829740 1 -- 172.21.15.117:0/1690029848 shutdown_connections 2024-09-21T06:07:24.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.215+0000 7f6ee5829740 1 -- 172.21.15.117:0/1690029848 wait complete. 2024-09-21T06:07:24.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.215+0000 7f6ee5829740 20 client.17619 trim_cache size 0 max 16384 2024-09-21T06:07:24.235 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:07:24.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.215+0000 7f6edf2b5640 1 -- 172.21.15.117:0/1574895979 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6ec40b6680 msgr2=0x7f6ec40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:24.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.215+0000 7f6edf2b5640 1 --2- 172.21.15.117:0/1574895979 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6ec40b6680 0x7f6ec40b8b90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0005c40 tx=0x7f6ed0005bd0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.215+0000 7f6edf2b5640 1 -- 172.21.15.117:0/1574895979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ed81094a0 msgr2=0x7f6ed819aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:24.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.215+0000 7f6edf2b5640 1 --2- 172.21.15.117:0/1574895979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ed81094a0 0x7f6ed819aa60 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f6ec802a930 tx=0x7f6ec8004270 comp rx=0 tx=0).stop 2024-09-21T06:07:24.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.215+0000 7f6edf2b5640 1 -- 172.21.15.117:0/1574895979 shutdown_connections 2024-09-21T06:07:24.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.215+0000 7f6edf2b5640 1 --2- 172.21.15.117:0/1574895979 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f6ec40b6680 0x7f6ec40b8b90 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.215+0000 7f6edf2b5640 1 --2- 172.21.15.117:0/1574895979 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ed81094a0 0x7f6ed819aa60 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.215+0000 7f6edf2b5640 1 --2- 172.21.15.117:0/1574895979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ed8109df0 0x7f6ed819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.215+0000 7f6edf2b5640 1 --2- 172.21.15.117:0/1574895979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ed8103b60 0x7f6ed819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:24.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.215+0000 7f6edf2b5640 1 -- 172.21.15.117:0/1574895979 >> 172.21.15.117:0/1574895979 conn(0x7f6ed807b850 msgr2=0x7f6ed8107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:24.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.215+0000 7f6edf2b5640 1 -- 172.21.15.117:0/1574895979 shutdown_connections 2024-09-21T06:07:24.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:24.215+0000 7f6edf2b5640 1 -- 172.21.15.117:0/1574895979 wait complete. 2024-09-21T06:07:24.369 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:24 smithi138 ceph-mon[34391]: pgmap v636: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 10 op/s 2024-09-21T06:07:24.482 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:24 smithi192 ceph-mon[34700]: pgmap v636: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 716 B/s wr, 10 op/s 2024-09-21T06:07:24.981 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T06:07:25.230 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:07:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.371+0000 7f747ac27640 1 -- 172.21.15.117:0/3086682177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7474108310 msgr2=0x7f7474108720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.371+0000 7f747ac27640 1 --2- 172.21.15.117:0/3086682177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7474108310 0x7f7474108720 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f7468009ca0 tx=0x7f746802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.371+0000 7f747ac27640 1 -- 172.21.15.117:0/3086682177 shutdown_connections 2024-09-21T06:07:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.371+0000 7f747ac27640 1 --2- 172.21.15.117:0/3086682177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7474108310 0x7f7474108720 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.371+0000 7f747ac27640 1 --2- 172.21.15.117:0/3086682177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7474108c60 0x7f747410f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.371+0000 7f747ac27640 1 --2- 172.21.15.117:0/3086682177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74741029e0 0x7f7474102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.371+0000 7f747ac27640 1 -- 172.21.15.117:0/3086682177 >> 172.21.15.117:0/3086682177 conn(0x7f747407a6e0 msgr2=0x7f747407aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.371+0000 7f747ac27640 1 -- 172.21.15.117:0/3086682177 shutdown_connections 2024-09-21T06:07:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.371+0000 7f747ac27640 1 -- 172.21.15.117:0/3086682177 wait complete. 2024-09-21T06:07:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.372+0000 7f747ac27640 1 Processor -- start 2024-09-21T06:07:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.372+0000 7f747ac27640 1 -- start start 2024-09-21T06:07:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.372+0000 7f747ac27640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74741029e0 0x7f74741993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.372+0000 7f747ac27640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7474108310 0x7f7474199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.372+0000 7f747ac27640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7474108c60 0x7f747419ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.376 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.372+0000 7f747ac27640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7474112920 con 0x7f7474108c60 2024-09-21T06:07:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.372+0000 7f747ac27640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f74741127a0 con 0x7f7474108310 2024-09-21T06:07:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.372+0000 7f747ac27640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7474112aa0 con 0x7f74741029e0 2024-09-21T06:07:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.372+0000 7f74791c4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7474108c60 0x7f747419ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.372+0000 7f74791c4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7474108c60 0x7f747419ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50182/0 (socket says 172.21.15.117:50182) 2024-09-21T06:07:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.372+0000 7f74791c4640 1 -- 172.21.15.117:0/1312904124 learned_addr learned my addr 172.21.15.117:0/1312904124 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.372+0000 7f7473fff640 1 --2- 172.21.15.117:0/1312904124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7474108310 0x7f7474199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.373+0000 7f74789c3640 1 --2- 172.21.15.117:0/1312904124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74741029e0 0x7f74741993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.373+0000 7f74791c4640 1 -- 172.21.15.117:0/1312904124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74741029e0 msgr2=0x7f74741993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.373+0000 7f74791c4640 1 --2- 172.21.15.117:0/1312904124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74741029e0 0x7f74741993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.373+0000 7f74791c4640 1 -- 172.21.15.117:0/1312904124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7474108310 msgr2=0x7f7474199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.373+0000 7f74791c4640 1 --2- 172.21.15.117:0/1312904124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7474108310 0x7f7474199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.373+0000 7f74791c4640 1 -- 172.21.15.117:0/1312904124 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f747419f280 con 0x7f7474108c60 2024-09-21T06:07:25.377 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.373+0000 7f74791c4640 1 --2- 172.21.15.117:0/1312904124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7474108c60 0x7f747419ec10 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f746c00c970 tx=0x7f746c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:25.378 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.373+0000 7f7471ffb640 1 -- 172.21.15.117:0/1312904124 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746c007bb0 con 0x7f7474108c60 2024-09-21T06:07:25.378 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.373+0000 7f7471ffb640 1 -- 172.21.15.117:0/1312904124 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f746c007d50 con 0x7f7474108c60 2024-09-21T06:07:25.378 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.373+0000 7f7471ffb640 1 -- 172.21.15.117:0/1312904124 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746c018770 con 0x7f7474108c60 2024-09-21T06:07:25.378 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.373+0000 7f747ac27640 1 -- 172.21.15.117:0/1312904124 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f747419f570 con 0x7f7474108c60 2024-09-21T06:07:25.378 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.373+0000 7f747ac27640 1 -- 172.21.15.117:0/1312904124 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74741a5d90 con 0x7f7474108c60 2024-09-21T06:07:25.383 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.376+0000 7f7481c6f740 1 Processor -- start 2024-09-21T06:07:25.383 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.376+0000 7f7471ffb640 1 -- 172.21.15.117:0/1312904124 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f746c00f990 con 0x7f7474108c60 2024-09-21T06:07:25.383 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.376+0000 7f7481c6f740 1 -- start start 2024-09-21T06:07:25.383 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.376+0000 7f7481c6f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6fe1ed1b0 0x55e6fe5929b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.383 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.376+0000 7f7481c6f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e6fe1c42f0 0x55e6fe592ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.383 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.376+0000 7f7481c6f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6fe1cc4d0 0x55e6fe593430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.383 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.376+0000 7f7481c6f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e6fe5076f0 con 0x55e6fe1ed1b0 2024-09-21T06:07:25.383 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.376+0000 7f7481c6f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e6fe4f6b20 con 0x55e6fe1cc4d0 2024-09-21T06:07:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.376+0000 7f7481c6f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55e6fe4fa930 con 0x55e6fe1c42f0 2024-09-21T06:07:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.376+0000 7f7473fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6fe1ed1b0 0x55e6fe5929b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.376+0000 7f7473fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6fe1ed1b0 0x55e6fe5929b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50190/0 (socket says 172.21.15.117:50190) 2024-09-21T06:07:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.376+0000 7f7473fff640 1 -- 172.21.15.117:0/51646384 learned_addr learned my addr 172.21.15.117:0/51646384 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.376+0000 7f74789c3640 1 --2- 172.21.15.117:0/51646384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e6fe1c42f0 0x55e6fe592ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.376+0000 7f7473fff640 1 --2- 172.21.15.117:0/51646384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6fe1cc4d0 0x55e6fe593430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.376+0000 7f7473fff640 1 -- 172.21.15.117:0/51646384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e6fe1c42f0 msgr2=0x55e6fe592ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.376+0000 7f7473fff640 1 --2- 172.21.15.117:0/51646384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e6fe1c42f0 0x55e6fe592ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.376+0000 7f7473fff640 1 -- 172.21.15.117:0/51646384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6fe1cc4d0 msgr2=0x55e6fe593430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.376+0000 7f7473fff640 1 --2- 172.21.15.117:0/51646384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6fe1cc4d0 0x55e6fe593430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.376+0000 7f7473fff640 1 -- 172.21.15.117:0/51646384 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6fe4bc530 con 0x55e6fe1ed1b0 2024-09-21T06:07:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.377+0000 7f7473fff640 1 --2- 172.21.15.117:0/51646384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6fe1ed1b0 0x55e6fe5929b0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f7468009ca0 tx=0x7f746802f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9df23128975fa68 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:25.384 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.377+0000 7f7471ffb640 1 --2- 172.21.15.117:0/1312904124 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f745c0b6700 0x7f745c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.377+0000 7f74789c3640 1 --2- 172.21.15.117:0/1312904124 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f745c0b6700 0x7f745c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:25.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.377+0000 7f745affd640 1 -- 172.21.15.117:0/51646384 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7468045020 con 0x55e6fe1ed1b0 2024-09-21T06:07:25.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.377+0000 7f745affd640 1 -- 172.21.15.117:0/51646384 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7468036040 con 0x55e6fe1ed1b0 2024-09-21T06:07:25.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.377+0000 7f745affd640 1 -- 172.21.15.117:0/51646384 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7468004230 con 0x55e6fe1ed1b0 2024-09-21T06:07:25.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.377+0000 7f7471ffb640 1 -- 172.21.15.117:0/1312904124 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f746c0e77a0 con 0x7f7474108c60 2024-09-21T06:07:25.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.378+0000 7f7481c6f740 1 -- 172.21.15.117:0/51646384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6fe1ed1b0 msgr2=0x55e6fe5929b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:25.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.378+0000 7f74789c3640 1 --2- 172.21.15.117:0/1312904124 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f745c0b6700 0x7f745c0b8c10 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7464005fd0 tx=0x7f7464005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:25.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.378+0000 7f7481c6f740 1 --2- 172.21.15.117:0/51646384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6fe1ed1b0 0x55e6fe5929b0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f7468009ca0 tx=0x7f746802f3f0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.378+0000 7f7481c6f740 1 -- 172.21.15.117:0/51646384 shutdown_connections 2024-09-21T06:07:25.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.378+0000 7f7481c6f740 1 --2- 172.21.15.117:0/51646384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6fe1cc4d0 0x55e6fe593430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.378+0000 7f7481c6f740 1 --2- 172.21.15.117:0/51646384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e6fe1c42f0 0x55e6fe592ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.378+0000 7f7481c6f740 1 --2- 172.21.15.117:0/51646384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6fe1ed1b0 0x55e6fe5929b0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.378+0000 7f7481c6f740 1 -- 172.21.15.117:0/51646384 >> 172.21.15.117:0/51646384 conn(0x55e6fe1e1300 msgr2=0x55e6fe5908a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:25.385 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.378+0000 7f7481c6f740 1 -- 172.21.15.117:0/51646384 shutdown_connections 2024-09-21T06:07:25.386 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.378+0000 7f7481c6f740 1 -- 172.21.15.117:0/51646384 wait complete. 2024-09-21T06:07:25.386 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.378+0000 7f7481c6f740 1 -- start start 2024-09-21T06:07:25.386 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.378+0000 7f7481c6f740 1 Processor -- start 2024-09-21T06:07:25.386 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.379+0000 7f7481c6f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e6fe1c42f0 0x55e6fe59e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.386 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.379+0000 7f7481c6f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6fe1cc4d0 0x55e6fe59e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.386 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.379+0000 7f7481c6f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6fe1ed1b0 0x55e6fe5aa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.386 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.379+0000 7f7481c6f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e6fe42eab0 con 0x55e6fe1ed1b0 2024-09-21T06:07:25.386 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.379+0000 7f7481c6f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e6fe49c730 con 0x55e6fe1cc4d0 2024-09-21T06:07:25.386 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.379+0000 7f7481c6f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55e6fe5a10d0 con 0x55e6fe1c42f0 2024-09-21T06:07:25.386 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.379+0000 7f7481c6f740 10 client.0 _ll_register_callbacks cb 0x55e6fe4f06b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:25.386 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.379+0000 7f74791c4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6fe1ed1b0 0x55e6fe5aa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:25.387 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.379+0000 7f74791c4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6fe1ed1b0 0x55e6fe5aa3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50200/0 (socket says 172.21.15.117:50200) 2024-09-21T06:07:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.379+0000 7f74791c4640 1 -- 172.21.15.117:0/239053080 learned_addr learned my addr 172.21.15.117:0/239053080 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.379+0000 7f7473fff640 1 --2- 172.21.15.117:0/239053080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6fe1cc4d0 0x55e6fe59e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:25.388 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.379+0000 7f74789c3640 1 --2- 172.21.15.117:0/239053080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e6fe1c42f0 0x55e6fe59e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.379+0000 7f74791c4640 1 -- 172.21.15.117:0/239053080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e6fe1c42f0 msgr2=0x55e6fe59e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.379+0000 7f74791c4640 1 --2- 172.21.15.117:0/239053080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e6fe1c42f0 0x55e6fe59e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.389 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.379+0000 7f74791c4640 1 -- 172.21.15.117:0/239053080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6fe1cc4d0 msgr2=0x55e6fe59e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.379+0000 7f74791c4640 1 --2- 172.21.15.117:0/239053080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6fe1cc4d0 0x55e6fe59e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.379+0000 7f74791c4640 1 -- 172.21.15.117:0/239053080 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6fe258790 con 0x55e6fe1ed1b0 2024-09-21T06:07:25.390 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.379+0000 7f74791c4640 1 --2- 172.21.15.117:0/239053080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6fe1ed1b0 0x55e6fe5aa3c0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f746c0ad340 tx=0x7f746c0b6f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cfa4d3935e08790 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.379+0000 7f7453fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.379+0000 7f7453fff640 1 -- 172.21.15.117:0/239053080 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746c0acdb0 con 0x55e6fe1ed1b0 2024-09-21T06:07:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.379+0000 7f7453fff640 1 -- 172.21.15.117:0/239053080 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f746c0a8f90 con 0x55e6fe1ed1b0 2024-09-21T06:07:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.380+0000 7f7481c6f740 1 -- 172.21.15.117:0/239053080 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7468036040 con 0x55e6fe1ed1b0 2024-09-21T06:07:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.380+0000 7f7453fff640 1 -- 172.21.15.117:0/239053080 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f746c0ad150 con 0x55e6fe1ed1b0 2024-09-21T06:07:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.380+0000 7f7481c6f740 10 client.17631 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:25.391 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.380+0000 7f7481c6f740 1 -- 172.21.15.117:0/239053080 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e6fe2d1620 con 0x55e6fe1ed1b0 2024-09-21T06:07:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.380+0000 7f7453fff640 1 -- 172.21.15.117:0/239053080 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7468036040 con 0x55e6fe1ed1b0 2024-09-21T06:07:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.380+0000 7f7453fff640 1 -- 172.21.15.117:0/239053080 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f746c0ac310 con 0x55e6fe1ed1b0 2024-09-21T06:07:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.380+0000 7f7481c6f740 10 client.17631 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.380+0000 7f7481c6f740 10 client.17631 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:07:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.380+0000 7f7481c6f740 20 client.17631 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.380+0000 7f7481c6f740 1 --2- 172.21.15.117:0/239053080 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55e6fe1abd80 0x55e6fe5b0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.380+0000 7f7481c6f740 4 client.17631 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:07:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.380+0000 7f7481c6f740 1 -- 172.21.15.117:0/239053080 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6fe509640 con 0x55e6fe1abd80 2024-09-21T06:07:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.380+0000 7f74789c3640 1 --2- 172.21.15.117:0/239053080 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55e6fe1abd80 0x55e6fe5b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.381+0000 7f7453fff640 10 client.17631 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:25.392 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.381+0000 7f74789c3640 1 --2- 172.21.15.117:0/239053080 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55e6fe1abd80 0x55e6fe5b0130 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69879cb76666e809 server_cookie=d2da7af0401403cb in_seq=0 out_seq=0 2024-09-21T06:07:25.393 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.382+0000 7f7453fff640 1 -- 172.21.15.117:0/239053080 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e6fe509640 con 0x55e6fe1abd80 2024-09-21T06:07:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.382+0000 7f7453fff640 10 client.17631 handle_command_reply: tid=0 2024-09-21T06:07:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.382+0000 7f7481c6f740 1 client.17631 shutdown 2024-09-21T06:07:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.382+0000 7f7481c6f740 1 -- 172.21.15.117:0/239053080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6fe1ed1b0 msgr2=0x55e6fe5aa3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.382+0000 7f7481c6f740 1 --2- 172.21.15.117:0/239053080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6fe1ed1b0 0x55e6fe5aa3c0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f746c0ad340 tx=0x7f746c0b6f10 comp rx=0 tx=0).stop 2024-09-21T06:07:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.382+0000 7f7481c6f740 1 -- 172.21.15.117:0/239053080 shutdown_connections 2024-09-21T06:07:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.382+0000 7f7481c6f740 1 --2- 172.21.15.117:0/239053080 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55e6fe1abd80 0x55e6fe5b0130 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.382+0000 7f7481c6f740 1 --2- 172.21.15.117:0/239053080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6fe1ed1b0 0x55e6fe5aa3c0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.382+0000 7f7481c6f740 1 --2- 172.21.15.117:0/239053080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6fe1cc4d0 0x55e6fe59e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.382+0000 7f7453fff640 0 client.17631 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.382+0000 7f7481c6f740 1 --2- 172.21.15.117:0/239053080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e6fe1c42f0 0x55e6fe59e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.382+0000 7f7481c6f740 1 -- 172.21.15.117:0/239053080 >> 172.21.15.117:0/239053080 conn(0x55e6fe1e1300 msgr2=0x55e6fe5939e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.382+0000 7f7481c6f740 1 -- 172.21.15.117:0/239053080 shutdown_connections 2024-09-21T06:07:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.382+0000 7f7481c6f740 1 -- 172.21.15.117:0/239053080 wait complete. 2024-09-21T06:07:25.394 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.382+0000 7f7481c6f740 20 client.17631 trim_cache size 0 max 16384 2024-09-21T06:07:25.399 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.396+0000 7f7481c6f740 1 Processor -- start 2024-09-21T06:07:25.399 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.396+0000 7f7481c6f740 1 -- start start 2024-09-21T06:07:25.399 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.397+0000 7f7481c6f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6fe1c42f0 0x55e6fe5940f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.399 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.397+0000 7f7481c6f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e6fe1cc4d0 0x55e6fe5aa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.399 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.397+0000 7f7481c6f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6fe1ed1b0 0x55e6fe59b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.399 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.397+0000 7f7481c6f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f74741af490 con 0x55e6fe1c42f0 2024-09-21T06:07:25.400 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.397+0000 7f7481c6f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f74741b0fd0 con 0x55e6fe1ed1b0 2024-09-21T06:07:25.400 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.397+0000 7f7481c6f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f74741afb60 con 0x55e6fe1cc4d0 2024-09-21T06:07:25.400 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.397+0000 7f74789c3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6fe1c42f0 0x55e6fe5940f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:25.400 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.397+0000 7f74789c3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6fe1c42f0 0x55e6fe5940f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50202/0 (socket says 172.21.15.117:50202) 2024-09-21T06:07:25.400 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.397+0000 7f74789c3640 1 -- 172.21.15.117:0/3821468063 learned_addr learned my addr 172.21.15.117:0/3821468063 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:25.400 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.397+0000 7f74789c3640 1 -- 172.21.15.117:0/3821468063 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e6fe1cc4d0 msgr2=0x55e6fe5aa2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:25.400 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.397+0000 7f74789c3640 1 --2- 172.21.15.117:0/3821468063 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e6fe1cc4d0 0x55e6fe5aa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.400 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.397+0000 7f74789c3640 1 -- 172.21.15.117:0/3821468063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6fe1ed1b0 msgr2=0x55e6fe59b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:25.400 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.397+0000 7f74791c4640 1 --2- 172.21.15.117:0/3821468063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6fe1ed1b0 0x55e6fe59b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:25.400 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.397+0000 7f74789c3640 1 --2- 172.21.15.117:0/3821468063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6fe1ed1b0 0x55e6fe59b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.400 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.397+0000 7f74789c3640 1 -- 172.21.15.117:0/3821468063 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6fe5957c0 con 0x55e6fe1c42f0 2024-09-21T06:07:25.400 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.397+0000 7f74789c3640 1 --2- 172.21.15.117:0/3821468063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6fe1c42f0 0x55e6fe5940f0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x55e6fe5b0860 tx=0x7f7464016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e0264dc4681970 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:25.400 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.397+0000 7f745a7fc640 1 -- 172.21.15.117:0/3821468063 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7464020630 con 0x55e6fe1c42f0 2024-09-21T06:07:25.404 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.397+0000 7f745a7fc640 1 -- 172.21.15.117:0/3821468063 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f746402c070 con 0x55e6fe1c42f0 2024-09-21T06:07:25.405 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.397+0000 7f745a7fc640 1 -- 172.21.15.117:0/3821468063 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7464029a80 con 0x55e6fe1c42f0 2024-09-21T06:07:25.405 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.398+0000 7f7481c6f740 1 -- 172.21.15.117:0/3821468063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6fe1c42f0 msgr2=0x55e6fe5940f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:25.406 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.398+0000 7f7481c6f740 1 --2- 172.21.15.117:0/3821468063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6fe1c42f0 0x55e6fe5940f0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x55e6fe5b0860 tx=0x7f7464016000 comp rx=0 tx=0).stop 2024-09-21T06:07:25.406 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.398+0000 7f7481c6f740 1 -- 172.21.15.117:0/3821468063 shutdown_connections 2024-09-21T06:07:25.406 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.398+0000 7f7481c6f740 1 --2- 172.21.15.117:0/3821468063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6fe1ed1b0 0x55e6fe59b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.406 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.398+0000 7f7481c6f740 1 --2- 172.21.15.117:0/3821468063 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e6fe1cc4d0 0x55e6fe5aa2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.406 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.398+0000 7f7481c6f740 1 --2- 172.21.15.117:0/3821468063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6fe1c42f0 0x55e6fe5940f0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.406 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.398+0000 7f7481c6f740 1 -- 172.21.15.117:0/3821468063 >> 172.21.15.117:0/3821468063 conn(0x55e6fe1e1300 msgr2=0x55e6fe5939e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:25.406 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.398+0000 7f7481c6f740 1 -- 172.21.15.117:0/3821468063 shutdown_connections 2024-09-21T06:07:25.406 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.398+0000 7f7481c6f740 1 -- 172.21.15.117:0/3821468063 wait complete. 2024-09-21T06:07:25.406 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.399+0000 7f7481c6f740 1 -- start start 2024-09-21T06:07:25.406 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.399+0000 7f7481c6f740 1 Processor -- start 2024-09-21T06:07:25.406 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.399+0000 7f7481c6f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6fe1c42f0 0x55e6fe5a3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.399+0000 7f7481c6f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e6fe1cc4d0 0x55e6fe5a4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.399+0000 7f7481c6f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6fe1ed1b0 0x55e6fe612c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.399+0000 7f7481c6f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f74741b0900 con 0x55e6fe1c42f0 2024-09-21T06:07:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.399+0000 7f7481c6f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f74741b0230 con 0x55e6fe1ed1b0 2024-09-21T06:07:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.399+0000 7f7481c6f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f74741ad490 con 0x55e6fe1cc4d0 2024-09-21T06:07:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.399+0000 7f7481c6f740 10 client.0 _ll_register_callbacks cb 0x7f746c007f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.399+0000 7f74789c3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6fe1c42f0 0x55e6fe5a3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.399+0000 7f74789c3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6fe1c42f0 0x55e6fe5a3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50206/0 (socket says 172.21.15.117:50206) 2024-09-21T06:07:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.399+0000 7f74789c3640 1 -- 172.21.15.117:0/3847936117 learned_addr learned my addr 172.21.15.117:0/3847936117 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.399+0000 7f7473fff640 1 --2- 172.21.15.117:0/3847936117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e6fe1cc4d0 0x55e6fe5a4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.399+0000 7f74791c4640 1 --2- 172.21.15.117:0/3847936117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6fe1ed1b0 0x55e6fe612c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.399+0000 7f74789c3640 1 -- 172.21.15.117:0/3847936117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e6fe1cc4d0 msgr2=0x55e6fe5a4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.400+0000 7f74789c3640 1 --2- 172.21.15.117:0/3847936117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e6fe1cc4d0 0x55e6fe5a4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.407 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.400+0000 7f74789c3640 1 -- 172.21.15.117:0/3847936117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6fe1ed1b0 msgr2=0x55e6fe612c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.400+0000 7f74789c3640 1 --2- 172.21.15.117:0/3847936117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6fe1ed1b0 0x55e6fe612c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.400+0000 7f74789c3640 1 -- 172.21.15.117:0/3847936117 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6fe6132e0 con 0x55e6fe1c42f0 2024-09-21T06:07:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.400+0000 7f74789c3640 1 --2- 172.21.15.117:0/3847936117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6fe1c42f0 0x55e6fe5a3e80 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f7464029530 tx=0x7f7464029560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7aafba87637b7fa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.400+0000 7f7453fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.400+0000 7f7453fff640 1 -- 172.21.15.117:0/3847936117 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7464005320 con 0x55e6fe1c42f0 2024-09-21T06:07:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.400+0000 7f7453fff640 1 -- 172.21.15.117:0/3847936117 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f74640054c0 con 0x55e6fe1c42f0 2024-09-21T06:07:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.400+0000 7f7481c6f740 1 -- 172.21.15.117:0/3847936117 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f746402c070 con 0x55e6fe1c42f0 2024-09-21T06:07:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.400+0000 7f7453fff640 1 -- 172.21.15.117:0/3847936117 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f74640297f0 con 0x55e6fe1c42f0 2024-09-21T06:07:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.400+0000 7f7481c6f740 10 client.17637 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.400+0000 7f7481c6f740 1 -- 172.21.15.117:0/3847936117 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e6fe61a6d0 con 0x55e6fe1c42f0 2024-09-21T06:07:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.400+0000 7f7453fff640 1 -- 172.21.15.117:0/3847936117 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f746402c070 con 0x55e6fe1c42f0 2024-09-21T06:07:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.400+0000 7f7453fff640 1 -- 172.21.15.117:0/3847936117 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f746400f040 con 0x55e6fe1c42f0 2024-09-21T06:07:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.400+0000 7f7481c6f740 10 client.17637 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.400+0000 7f7481c6f740 10 client.17637 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:07:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.400+0000 7f7481c6f740 20 client.17637 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:25.409 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.400+0000 7f7481c6f740 1 --2- 172.21.15.117:0/3847936117 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55e6fe1811c0 0x55e6fe61ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.409 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.401+0000 7f7481c6f740 4 client.17637 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:07:25.409 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.401+0000 7f7481c6f740 1 -- 172.21.15.117:0/3847936117 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e6fe5050f0 con 0x55e6fe1811c0 2024-09-21T06:07:25.409 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.401+0000 7f7473fff640 1 --2- 172.21.15.117:0/3847936117 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55e6fe1811c0 0x55e6fe61ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:25.409 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.401+0000 7f7453fff640 10 client.17637 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:25.409 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.401+0000 7f7473fff640 1 --2- 172.21.15.117:0/3847936117 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55e6fe1811c0 0x55e6fe61ce50 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caf60c84138a8ad5 server_cookie=302b666aa1f4cefe in_seq=0 out_seq=0 2024-09-21T06:07:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.408+0000 7f7453fff640 1 -- 172.21.15.117:0/3847936117 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e6fe5050f0 con 0x55e6fe1811c0 2024-09-21T06:07:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.408+0000 7f7453fff640 10 client.17637 handle_command_reply: tid=0 2024-09-21T06:07:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.408+0000 7f7481c6f740 1 client.17637 shutdown 2024-09-21T06:07:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.409+0000 7f7481c6f740 1 -- 172.21.15.117:0/3847936117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6fe1c42f0 msgr2=0x55e6fe5a3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.409+0000 7f7481c6f740 1 --2- 172.21.15.117:0/3847936117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6fe1c42f0 0x55e6fe5a3e80 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f7464029530 tx=0x7f7464029560 comp rx=0 tx=0).stop 2024-09-21T06:07:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.409+0000 7f7481c6f740 1 -- 172.21.15.117:0/3847936117 shutdown_connections 2024-09-21T06:07:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.409+0000 7f7481c6f740 1 --2- 172.21.15.117:0/3847936117 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55e6fe1811c0 0x55e6fe61ce50 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.409+0000 7f7481c6f740 1 --2- 172.21.15.117:0/3847936117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6fe1ed1b0 0x55e6fe612c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.409+0000 7f7453fff640 0 client.17637 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.409+0000 7f7481c6f740 1 --2- 172.21.15.117:0/3847936117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e6fe1cc4d0 0x55e6fe5a4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.409+0000 7f7481c6f740 1 --2- 172.21.15.117:0/3847936117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e6fe1c42f0 0x55e6fe5a3e80 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.412 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.409+0000 7f7481c6f740 1 -- 172.21.15.117:0/3847936117 >> 172.21.15.117:0/3847936117 conn(0x55e6fe1e1300 msgr2=0x55e6fe5939e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:25.412 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.409+0000 7f7481c6f740 1 -- 172.21.15.117:0/3847936117 shutdown_connections 2024-09-21T06:07:25.412 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.409+0000 7f7481c6f740 1 -- 172.21.15.117:0/3847936117 wait complete. 2024-09-21T06:07:25.412 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.409+0000 7f7481c6f740 20 client.17637 trim_cache size 0 max 16384 2024-09-21T06:07:25.412 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T06:07:25.412 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T06:07:25.412 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T06:07:25.412 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T06:07:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.409+0000 7f747ac27640 1 -- 172.21.15.117:0/1312904124 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f745c0b6700 msgr2=0x7f745c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.409+0000 7f747ac27640 1 --2- 172.21.15.117:0/1312904124 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f745c0b6700 0x7f745c0b8c10 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7464005fd0 tx=0x7f7464005850 comp rx=0 tx=0).stop 2024-09-21T06:07:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.409+0000 7f747ac27640 1 -- 172.21.15.117:0/1312904124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7474108c60 msgr2=0x7f747419ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.409+0000 7f747ac27640 1 --2- 172.21.15.117:0/1312904124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7474108c60 0x7f747419ec10 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f746c00c970 tx=0x7f746c00ce30 comp rx=0 tx=0).stop 2024-09-21T06:07:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.409+0000 7f747ac27640 1 -- 172.21.15.117:0/1312904124 shutdown_connections 2024-09-21T06:07:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.409+0000 7f747ac27640 1 --2- 172.21.15.117:0/1312904124 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f745c0b6700 0x7f745c0b8c10 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.409+0000 7f747ac27640 1 --2- 172.21.15.117:0/1312904124 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7474108c60 0x7f747419ec10 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.409+0000 7f747ac27640 1 --2- 172.21.15.117:0/1312904124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7474108310 0x7f7474199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.409+0000 7f747ac27640 1 --2- 172.21.15.117:0/1312904124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74741029e0 0x7f74741993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.409+0000 7f747ac27640 1 -- 172.21.15.117:0/1312904124 >> 172.21.15.117:0/1312904124 conn(0x7f747407a6e0 msgr2=0x7f7474104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.410+0000 7f747ac27640 1 -- 172.21.15.117:0/1312904124 shutdown_connections 2024-09-21T06:07:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:25.410+0000 7f747ac27640 1 -- 172.21.15.117:0/1312904124 wait complete. 2024-09-21T06:07:25.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.745+0000 7f9a67f57640 1 -- 172.21.15.117:0/527878037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a601094a0 msgr2=0x7f9a601098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:25.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.745+0000 7f9a67f57640 1 --2- 172.21.15.117:0/527878037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a601094a0 0x7f9a601098b0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009900 tx=0x7f9a5002a410 comp rx=0 tx=0).stop 2024-09-21T06:07:25.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.746+0000 7f9a67f57640 1 -- 172.21.15.117:0/527878037 shutdown_connections 2024-09-21T06:07:25.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.746+0000 7f9a67f57640 1 --2- 172.21.15.117:0/527878037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a601094a0 0x7f9a601098b0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.746+0000 7f9a67f57640 1 --2- 172.21.15.117:0/527878037 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a60109df0 0x7f9a60110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.746+0000 7f9a67f57640 1 --2- 172.21.15.117:0/527878037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a60103b60 0x7f9a60103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.746+0000 7f9a67f57640 1 -- 172.21.15.117:0/527878037 >> 172.21.15.117:0/527878037 conn(0x7f9a6007b850 msgr2=0x7f9a6007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:25.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.746+0000 7f9a67f57640 1 -- 172.21.15.117:0/527878037 shutdown_connections 2024-09-21T06:07:25.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.746+0000 7f9a67f57640 1 -- 172.21.15.117:0/527878037 wait complete. 2024-09-21T06:07:25.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.746+0000 7f9a67f57640 1 Processor -- start 2024-09-21T06:07:25.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.746+0000 7f9a67f57640 1 -- start start 2024-09-21T06:07:25.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.747+0000 7f9a67f57640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a60103b60 0x7f9a6019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.747+0000 7f9a67f57640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a601094a0 0x7f9a6019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.747+0000 7f9a67f57640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a60109df0 0x7f9a6019fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.747+0000 7f9a67f57640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9a60113ad0 con 0x7f9a60103b60 2024-09-21T06:07:25.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.747+0000 7f9a67f57640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9a60113950 con 0x7f9a601094a0 2024-09-21T06:07:25.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.747+0000 7f9a67f57640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9a60113c50 con 0x7f9a60109df0 2024-09-21T06:07:25.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.747+0000 7f9a65cf3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a60103b60 0x7f9a6019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:25.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.747+0000 7f9a65cf3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a60103b60 0x7f9a6019a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50218/0 (socket says 172.21.15.117:50218) 2024-09-21T06:07:25.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.747+0000 7f9a65cf3640 1 -- 172.21.15.117:0/3540324716 learned_addr learned my addr 172.21.15.117:0/3540324716 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:25.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.747+0000 7f9a654f2640 1 --2- 172.21.15.117:0/3540324716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a601094a0 0x7f9a6019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:25.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.747+0000 7f9a664f4640 1 --2- 172.21.15.117:0/3540324716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a60109df0 0x7f9a6019fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:25.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.747+0000 7f9a65cf3640 1 -- 172.21.15.117:0/3540324716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a60109df0 msgr2=0x7f9a6019fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:25.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.747+0000 7f9a65cf3640 1 --2- 172.21.15.117:0/3540324716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a60109df0 0x7f9a6019fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.747+0000 7f9a65cf3640 1 -- 172.21.15.117:0/3540324716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a601094a0 msgr2=0x7f9a6019ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:25.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.747+0000 7f9a65cf3640 1 --2- 172.21.15.117:0/3540324716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a601094a0 0x7f9a6019ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.747+0000 7f9a65cf3640 1 -- 172.21.15.117:0/3540324716 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a601a0510 con 0x7f9a60103b60 2024-09-21T06:07:25.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.747+0000 7f9a664f4640 1 --2- 172.21.15.117:0/3540324716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a60109df0 0x7f9a6019fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:07:25.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.747+0000 7f9a65cf3640 1 --2- 172.21.15.117:0/3540324716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a60103b60 0x7f9a6019a5a0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c00d950 tx=0x7f9a5c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:25.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.748+0000 7f9a56ffd640 1 -- 172.21.15.117:0/3540324716 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c017070 con 0x7f9a60103b60 2024-09-21T06:07:25.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.748+0000 7f9a56ffd640 1 -- 172.21.15.117:0/3540324716 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a5c009c30 con 0x7f9a60103b60 2024-09-21T06:07:25.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.748+0000 7f9a56ffd640 1 -- 172.21.15.117:0/3540324716 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c00f620 con 0x7f9a60103b60 2024-09-21T06:07:25.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.748+0000 7f9a67f57640 1 -- 172.21.15.117:0/3540324716 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a601a0800 con 0x7f9a60103b60 2024-09-21T06:07:25.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.748+0000 7f9a67f57640 1 -- 172.21.15.117:0/3540324716 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a601a6f60 con 0x7f9a60103b60 2024-09-21T06:07:25.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.750+0000 7f9a6e4cb740 1 Processor -- start 2024-09-21T06:07:25.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.750+0000 7f9a6e4cb740 1 -- start start 2024-09-21T06:07:25.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.751+0000 7f9a56ffd640 1 -- 172.21.15.117:0/3540324716 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f9a5c010460 con 0x7f9a60103b60 2024-09-21T06:07:25.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.751+0000 7f9a6e4cb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55663603d070 0x5566363e42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.751+0000 7f9a6e4cb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566360141b0 0x5566363e4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.751+0000 7f9a6e4cb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55663601c390 0x5566363e4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.751+0000 7f9a6e4cb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55663635aea0 con 0x55663601c390 2024-09-21T06:07:25.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.751+0000 7f9a6e4cb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55663634a2d0 con 0x55663603d070 2024-09-21T06:07:25.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.751+0000 7f9a6e4cb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55663634e0e0 con 0x5566360141b0 2024-09-21T06:07:25.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.751+0000 7f9a654f2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55663601c390 0x5566363e4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:25.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.751+0000 7f9a654f2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55663601c390 0x5566363e4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50222/0 (socket says 172.21.15.117:50222) 2024-09-21T06:07:25.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.751+0000 7f9a654f2640 1 -- 172.21.15.117:0/949928353 learned_addr learned my addr 172.21.15.117:0/949928353 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:25.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.751+0000 7f9a664f4640 1 --2- 172.21.15.117:0/949928353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55663603d070 0x5566363e42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:25.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.751+0000 7f9a65cf3640 1 --2- 172.21.15.117:0/949928353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566360141b0 0x5566363e4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:25.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.751+0000 7f9a654f2640 1 -- 172.21.15.117:0/949928353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566360141b0 msgr2=0x5566363e4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:25.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.751+0000 7f9a654f2640 1 --2- 172.21.15.117:0/949928353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566360141b0 0x5566363e4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.751+0000 7f9a654f2640 1 -- 172.21.15.117:0/949928353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55663603d070 msgr2=0x5566363e42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:25.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.751+0000 7f9a654f2640 1 --2- 172.21.15.117:0/949928353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55663603d070 0x5566363e42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.751+0000 7f9a654f2640 1 -- 172.21.15.117:0/949928353 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566360a8650 con 0x55663601c390 2024-09-21T06:07:25.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.751+0000 7f9a65cf3640 1 --2- 172.21.15.117:0/949928353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566360141b0 0x5566363e4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:25.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.751+0000 7f9a654f2640 1 --2- 172.21.15.117:0/949928353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55663601c390 0x5566363e4d40 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009900 tx=0x7f9a50000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f7f8911de033dc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:25.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.752+0000 7f9a4bfff640 1 -- 172.21.15.117:0/949928353 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a50040070 con 0x55663601c390 2024-09-21T06:07:25.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.752+0000 7f9a4bfff640 1 -- 172.21.15.117:0/949928353 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a50005db0 con 0x55663601c390 2024-09-21T06:07:25.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.752+0000 7f9a4bfff640 1 -- 172.21.15.117:0/949928353 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a50035040 con 0x55663601c390 2024-09-21T06:07:25.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.752+0000 7f9a56ffd640 1 --2- 172.21.15.117:0/3540324716 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9a4c0b6700 0x7f9a4c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.752+0000 7f9a664f4640 1 --2- 172.21.15.117:0/3540324716 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9a4c0b6700 0x7f9a4c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:25.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.752+0000 7f9a56ffd640 1 -- 172.21.15.117:0/3540324716 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f9a5c0e72b0 con 0x7f9a60103b60 2024-09-21T06:07:25.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.753+0000 7f9a6e4cb740 1 -- 172.21.15.117:0/949928353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55663601c390 msgr2=0x5566363e4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:25.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.753+0000 7f9a664f4640 1 --2- 172.21.15.117:0/3540324716 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9a4c0b6700 0x7f9a4c0b8c10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9a5800f4c0 tx=0x7f9a58007740 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:25.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.753+0000 7f9a6e4cb740 1 --2- 172.21.15.117:0/949928353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55663601c390 0x5566363e4d40 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009900 tx=0x7f9a50000bc0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.753+0000 7f9a6e4cb740 1 -- 172.21.15.117:0/949928353 shutdown_connections 2024-09-21T06:07:25.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.753+0000 7f9a6e4cb740 1 --2- 172.21.15.117:0/949928353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55663601c390 0x5566363e4d40 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.753+0000 7f9a6e4cb740 1 --2- 172.21.15.117:0/949928353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566360141b0 0x5566363e4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.753+0000 7f9a6e4cb740 1 --2- 172.21.15.117:0/949928353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55663603d070 0x5566363e42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.753+0000 7f9a6e4cb740 1 -- 172.21.15.117:0/949928353 >> 172.21.15.117:0/949928353 conn(0x556636031460 msgr2=0x5566363e21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:25.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.753+0000 7f9a6e4cb740 1 -- 172.21.15.117:0/949928353 shutdown_connections 2024-09-21T06:07:25.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.753+0000 7f9a6e4cb740 1 -- 172.21.15.117:0/949928353 wait complete. 2024-09-21T06:07:25.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.753+0000 7f9a6e4cb740 1 -- start start 2024-09-21T06:07:25.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.753+0000 7f9a6e4cb740 1 Processor -- start 2024-09-21T06:07:25.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.754+0000 7f9a6e4cb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566360141b0 0x5566363ef920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.754+0000 7f9a6e4cb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55663601c390 0x5566363efe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.754+0000 7f9a6e4cb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55663603d070 0x5566363fbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.754+0000 7f9a6e4cb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5566361ab700 con 0x55663601c390 2024-09-21T06:07:25.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.754+0000 7f9a6e4cb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5566362e5160 con 0x55663603d070 2024-09-21T06:07:25.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.754+0000 7f9a6e4cb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5566363f2a40 con 0x5566360141b0 2024-09-21T06:07:25.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.754+0000 7f9a654f2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55663601c390 0x5566363efe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:25.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.754+0000 7f9a654f2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55663601c390 0x5566363efe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50232/0 (socket says 172.21.15.117:50232) 2024-09-21T06:07:25.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.754+0000 7f9a654f2640 1 -- 172.21.15.117:0/4217542556 learned_addr learned my addr 172.21.15.117:0/4217542556 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:25.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.754+0000 7f9a664f4640 1 --2- 172.21.15.117:0/4217542556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55663603d070 0x5566363fbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:25.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.754+0000 7f9a65cf3640 1 --2- 172.21.15.117:0/4217542556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566360141b0 0x5566363ef920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:25.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.754+0000 7f9a6e4cb740 10 client.0 _ll_register_callbacks cb 0x5566362fe9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:25.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.754+0000 7f9a654f2640 1 -- 172.21.15.117:0/4217542556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566360141b0 msgr2=0x5566363ef920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:25.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.754+0000 7f9a654f2640 1 --2- 172.21.15.117:0/4217542556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566360141b0 0x5566363ef920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.754+0000 7f9a654f2640 1 -- 172.21.15.117:0/4217542556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55663603d070 msgr2=0x5566363fbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:25.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.754+0000 7f9a654f2640 1 --2- 172.21.15.117:0/4217542556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55663603d070 0x5566363fbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.754+0000 7f9a654f2640 1 -- 172.21.15.117:0/4217542556 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556636312c90 con 0x55663601c390 2024-09-21T06:07:25.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.754+0000 7f9a654f2640 1 --2- 172.21.15.117:0/4217542556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55663601c390 0x5566363efe60 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f9a50005620 tx=0x7f9a50005650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1bf22ca3011367f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:25.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.754+0000 7f9a497fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:25.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.755+0000 7f9a497fa640 1 -- 172.21.15.117:0/4217542556 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a50035040 con 0x55663601c390 2024-09-21T06:07:25.762 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.755+0000 7f9a6e4cb740 1 -- 172.21.15.117:0/4217542556 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a50005db0 con 0x55663601c390 2024-09-21T06:07:25.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.755+0000 7f9a497fa640 1 -- 172.21.15.117:0/4217542556 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f9a500046c0 con 0x55663601c390 2024-09-21T06:07:25.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.755+0000 7f9a497fa640 1 -- 172.21.15.117:0/4217542556 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9a50040040 con 0x55663601c390 2024-09-21T06:07:25.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.755+0000 7f9a6e4cb740 10 client.17649 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:25.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.755+0000 7f9a6e4cb740 1 -- 172.21.15.117:0/4217542556 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5566363fe6d0 con 0x55663601c390 2024-09-21T06:07:25.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.755+0000 7f9a497fa640 1 -- 172.21.15.117:0/4217542556 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a50005db0 con 0x55663601c390 2024-09-21T06:07:25.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.755+0000 7f9a497fa640 1 -- 172.21.15.117:0/4217542556 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9a5003c6e0 con 0x55663601c390 2024-09-21T06:07:25.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.755+0000 7f9a6e4cb740 10 client.17649 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:25.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.755+0000 7f9a6e4cb740 10 client.17649 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:07:25.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.755+0000 7f9a6e4cb740 20 client.17649 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:25.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.755+0000 7f9a6e4cb740 1 --2- 172.21.15.117:0/4217542556 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x556635ffb9f0 0x556636401a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.755+0000 7f9a6e4cb740 4 client.17649 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:07:25.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.755+0000 7f9a6e4cb740 1 -- 172.21.15.117:0/4217542556 --> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55663635cdf0 con 0x556635ffb9f0 2024-09-21T06:07:25.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.756+0000 7f9a65cf3640 1 --2- 172.21.15.117:0/4217542556 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x556635ffb9f0 0x556636401a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:25.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.756+0000 7f9a497fa640 10 client.17649 ms_handle_connect on v2:172.21.15.192:6838/1727623231 2024-09-21T06:07:25.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.756+0000 7f9a65cf3640 1 --2- 172.21.15.117:0/4217542556 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x556635ffb9f0 0x556636401a30 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d0fc71f7974dfee6 server_cookie=bac7f87151f2b217 in_seq=0 out_seq=0 2024-09-21T06:07:25.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.757+0000 7f9a497fa640 1 -- 172.21.15.117:0/4217542556 <== mds.1 v2:172.21.15.192:6838/1727623231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55663635cdf0 con 0x556635ffb9f0 2024-09-21T06:07:25.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.757+0000 7f9a497fa640 10 client.17649 handle_command_reply: tid=0 2024-09-21T06:07:25.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.757+0000 7f9a6e4cb740 1 client.17649 shutdown 2024-09-21T06:07:25.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.757+0000 7f9a6e4cb740 1 -- 172.21.15.117:0/4217542556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55663601c390 msgr2=0x5566363efe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:25.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.757+0000 7f9a6e4cb740 1 --2- 172.21.15.117:0/4217542556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55663601c390 0x5566363efe60 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f9a50005620 tx=0x7f9a50005650 comp rx=0 tx=0).stop 2024-09-21T06:07:25.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.757+0000 7f9a6e4cb740 1 -- 172.21.15.117:0/4217542556 shutdown_connections 2024-09-21T06:07:25.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.757+0000 7f9a6e4cb740 1 --2- 172.21.15.117:0/4217542556 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x556635ffb9f0 0x556636401a30 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.757+0000 7f9a6e4cb740 1 --2- 172.21.15.117:0/4217542556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55663601c390 0x5566363efe60 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.757+0000 7f9a6e4cb740 1 --2- 172.21.15.117:0/4217542556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55663603d070 0x5566363fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.757+0000 7f9a497fa640 0 client.17649 ms_handle_reset on v2:172.21.15.192:6838/1727623231 2024-09-21T06:07:25.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.757+0000 7f9a6e4cb740 1 --2- 172.21.15.117:0/4217542556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566360141b0 0x5566363ef920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.757+0000 7f9a6e4cb740 1 -- 172.21.15.117:0/4217542556 >> 172.21.15.117:0/4217542556 conn(0x556636031460 msgr2=0x5566363e74a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:25.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.757+0000 7f9a6e4cb740 1 -- 172.21.15.117:0/4217542556 shutdown_connections 2024-09-21T06:07:25.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.757+0000 7f9a6e4cb740 1 -- 172.21.15.117:0/4217542556 wait complete. 2024-09-21T06:07:25.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.757+0000 7f9a6e4cb740 20 client.17649 trim_cache size 0 max 16384 2024-09-21T06:07:25.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.771+0000 7f9a6e4cb740 1 Processor -- start 2024-09-21T06:07:25.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.771+0000 7f9a6e4cb740 1 -- start start 2024-09-21T06:07:25.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.771+0000 7f9a6e4cb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566360141b0 0x5566363efde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.771+0000 7f9a6e4cb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55663601c390 0x5566363fbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.771+0000 7f9a6e4cb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55663603d070 0x5566363e1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.771+0000 7f9a6e4cb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9a601af690 con 0x55663603d070 2024-09-21T06:07:25.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.771+0000 7f9a6e4cb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9a601b11d0 con 0x55663601c390 2024-09-21T06:07:25.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.771+0000 7f9a6e4cb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9a601afd60 con 0x5566360141b0 2024-09-21T06:07:25.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.771+0000 7f9a664f4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55663603d070 0x5566363e1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:25.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.771+0000 7f9a664f4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55663603d070 0x5566363e1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50246/0 (socket says 172.21.15.117:50246) 2024-09-21T06:07:25.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.771+0000 7f9a664f4640 1 -- 172.21.15.117:0/1258394128 learned_addr learned my addr 172.21.15.117:0/1258394128 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:25.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.771+0000 7f9a65cf3640 1 --2- 172.21.15.117:0/1258394128 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566360141b0 0x5566363efde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:25.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.771+0000 7f9a664f4640 1 -- 172.21.15.117:0/1258394128 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566360141b0 msgr2=0x5566363efde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:25.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.771+0000 7f9a654f2640 1 --2- 172.21.15.117:0/1258394128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55663601c390 0x5566363fbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:25.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.771+0000 7f9a664f4640 1 --2- 172.21.15.117:0/1258394128 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566360141b0 0x5566363efde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.771+0000 7f9a664f4640 1 -- 172.21.15.117:0/1258394128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55663601c390 msgr2=0x5566363fbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:25.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.771+0000 7f9a664f4640 1 --2- 172.21.15.117:0/1258394128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55663601c390 0x5566363fbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.771+0000 7f9a664f4640 1 -- 172.21.15.117:0/1258394128 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566363e65b0 con 0x55663603d070 2024-09-21T06:07:25.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.771+0000 7f9a654f2640 1 --2- 172.21.15.117:0/1258394128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55663601c390 0x5566363fbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:07:25.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.771+0000 7f9a664f4640 1 --2- 172.21.15.117:0/1258394128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55663603d070 0x5566363e1760 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f9a580109c0 tx=0x7f9a58012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e3cac7029e36c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:25.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.772+0000 7f9a4b7fe640 1 -- 172.21.15.117:0/1258394128 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a5801edc0 con 0x55663603d070 2024-09-21T06:07:25.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.772+0000 7f9a4b7fe640 1 -- 172.21.15.117:0/1258394128 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a58026890 con 0x55663603d070 2024-09-21T06:07:25.775 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.772+0000 7f9a4b7fe640 1 -- 172.21.15.117:0/1258394128 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a58019760 con 0x55663603d070 2024-09-21T06:07:25.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.772+0000 7f9a6e4cb740 1 -- 172.21.15.117:0/1258394128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55663603d070 msgr2=0x5566363e1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:25.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.772+0000 7f9a6e4cb740 1 --2- 172.21.15.117:0/1258394128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55663603d070 0x5566363e1760 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f9a580109c0 tx=0x7f9a58012000 comp rx=0 tx=0).stop 2024-09-21T06:07:25.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.772+0000 7f9a6e4cb740 1 -- 172.21.15.117:0/1258394128 shutdown_connections 2024-09-21T06:07:25.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.772+0000 7f9a6e4cb740 1 --2- 172.21.15.117:0/1258394128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55663603d070 0x5566363e1760 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.772+0000 7f9a6e4cb740 1 --2- 172.21.15.117:0/1258394128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55663601c390 0x5566363fbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.772+0000 7f9a6e4cb740 1 --2- 172.21.15.117:0/1258394128 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566360141b0 0x5566363efde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.773+0000 7f9a6e4cb740 1 -- 172.21.15.117:0/1258394128 >> 172.21.15.117:0/1258394128 conn(0x556636031460 msgr2=0x5566363ef6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:25.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.773+0000 7f9a6e4cb740 1 -- 172.21.15.117:0/1258394128 shutdown_connections 2024-09-21T06:07:25.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.773+0000 7f9a6e4cb740 1 -- 172.21.15.117:0/1258394128 wait complete. 2024-09-21T06:07:25.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.773+0000 7f9a6e4cb740 1 -- start start 2024-09-21T06:07:25.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.773+0000 7f9a6e4cb740 1 Processor -- start 2024-09-21T06:07:25.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.774+0000 7f9a6e4cb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566360141b0 0x556636405460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.774+0000 7f9a6e4cb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55663601c390 0x556636402970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.779 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.774+0000 7f9a6e4cb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55663603d070 0x556636403080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.774+0000 7f9a6e4cb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9a601b0b00 con 0x55663603d070 2024-09-21T06:07:25.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.774+0000 7f9a6e4cb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9a601b0430 con 0x55663601c390 2024-09-21T06:07:25.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.774+0000 7f9a6e4cb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9a601ad690 con 0x5566360141b0 2024-09-21T06:07:25.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.774+0000 7f9a6e4cb740 10 client.0 _ll_register_callbacks cb 0x7f9a580075e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:25.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.774+0000 7f9a664f4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55663603d070 0x556636403080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:25.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.774+0000 7f9a664f4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55663603d070 0x556636403080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50248/0 (socket says 172.21.15.117:50248) 2024-09-21T06:07:25.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.774+0000 7f9a664f4640 1 -- 172.21.15.117:0/1485015740 learned_addr learned my addr 172.21.15.117:0/1485015740 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:25.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.774+0000 7f9a65cf3640 1 --2- 172.21.15.117:0/1485015740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566360141b0 0x556636405460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:25.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.774+0000 7f9a654f2640 1 --2- 172.21.15.117:0/1485015740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55663601c390 0x556636402970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:25.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.774+0000 7f9a664f4640 1 -- 172.21.15.117:0/1485015740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566360141b0 msgr2=0x556636405460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:25.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.774+0000 7f9a664f4640 1 --2- 172.21.15.117:0/1485015740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566360141b0 0x556636405460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.774+0000 7f9a664f4640 1 -- 172.21.15.117:0/1485015740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55663601c390 msgr2=0x556636402970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:25.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.774+0000 7f9a664f4640 1 --2- 172.21.15.117:0/1485015740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55663601c390 0x556636402970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.774+0000 7f9a664f4640 1 -- 172.21.15.117:0/1485015740 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55663646b1b0 con 0x55663603d070 2024-09-21T06:07:25.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.774+0000 7f9a664f4640 1 --2- 172.21.15.117:0/1485015740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55663603d070 0x556636403080 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f9a58004a80 tx=0x7f9a58026470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d75213d73f5054 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:25.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.774+0000 7f9a497fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:25.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.775+0000 7f9a497fa640 1 -- 172.21.15.117:0/1485015740 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a58036020 con 0x55663603d070 2024-09-21T06:07:25.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.775+0000 7f9a6e4cb740 1 -- 172.21.15.117:0/1485015740 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a58026890 con 0x55663603d070 2024-09-21T06:07:25.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.775+0000 7f9a497fa640 1 -- 172.21.15.117:0/1485015740 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f9a58026c50 con 0x55663603d070 2024-09-21T06:07:25.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.775+0000 7f9a497fa640 1 -- 172.21.15.117:0/1485015740 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9a58041050 con 0x55663603d070 2024-09-21T06:07:25.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.775+0000 7f9a6e4cb740 10 client.17655 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:25.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.775+0000 7f9a6e4cb740 1 -- 172.21.15.117:0/1485015740 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55663646c270 con 0x55663603d070 2024-09-21T06:07:25.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.775+0000 7f9a497fa640 1 -- 172.21.15.117:0/1485015740 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a58026890 con 0x55663603d070 2024-09-21T06:07:25.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.775+0000 7f9a497fa640 1 -- 172.21.15.117:0/1485015740 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9a58025070 con 0x55663603d070 2024-09-21T06:07:25.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.775+0000 7f9a6e4cb740 10 client.17655 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:25.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.775+0000 7f9a6e4cb740 10 client.17655 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:07:25.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.775+0000 7f9a6e4cb740 20 client.17655 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:25.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.775+0000 7f9a6e4cb740 1 --2- 172.21.15.117:0/1485015740 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x556635fd0da0 0x55663646e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:25.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.775+0000 7f9a6e4cb740 4 client.17655 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:07:25.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.775+0000 7f9a6e4cb740 1 -- 172.21.15.117:0/1485015740 --> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5566363588a0 con 0x556635fd0da0 2024-09-21T06:07:25.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.776+0000 7f9a65cf3640 1 --2- 172.21.15.117:0/1485015740 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x556635fd0da0 0x55663646e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:25.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.776+0000 7f9a65cf3640 1 --2- 172.21.15.117:0/1485015740 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x556635fd0da0 0x55663646e9c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f5f77c35be1d580c server_cookie=27910e102a20ec82 in_seq=0 out_seq=0 2024-09-21T06:07:25.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.776+0000 7f9a497fa640 10 client.17655 ms_handle_connect on v2:172.21.15.192:6838/1727623231 2024-09-21T06:07:25.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.776+0000 7f9a497fa640 1 -- 172.21.15.117:0/1485015740 <== mds.1 v2:172.21.15.192:6838/1727623231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5566363588a0 con 0x556635fd0da0 2024-09-21T06:07:25.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.776+0000 7f9a497fa640 10 client.17655 handle_command_reply: tid=0 2024-09-21T06:07:25.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.776+0000 7f9a6e4cb740 1 client.17655 shutdown 2024-09-21T06:07:25.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.776+0000 7f9a6e4cb740 1 -- 172.21.15.117:0/1485015740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55663603d070 msgr2=0x556636403080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:25.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.776+0000 7f9a6e4cb740 1 --2- 172.21.15.117:0/1485015740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55663603d070 0x556636403080 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f9a58004a80 tx=0x7f9a58026470 comp rx=0 tx=0).stop 2024-09-21T06:07:25.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.777+0000 7f9a6e4cb740 1 -- 172.21.15.117:0/1485015740 shutdown_connections 2024-09-21T06:07:25.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.777+0000 7f9a6e4cb740 1 --2- 172.21.15.117:0/1485015740 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x556635fd0da0 0x55663646e9c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.777+0000 7f9a6e4cb740 1 --2- 172.21.15.117:0/1485015740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55663603d070 0x556636403080 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.777+0000 7f9a6e4cb740 1 --2- 172.21.15.117:0/1485015740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55663601c390 0x556636402970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.777+0000 7f9a6e4cb740 1 --2- 172.21.15.117:0/1485015740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5566360141b0 0x556636405460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.777+0000 7f9a497fa640 0 client.17655 ms_handle_reset on v2:172.21.15.192:6838/1727623231 2024-09-21T06:07:25.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.777+0000 7f9a6e4cb740 1 -- 172.21.15.117:0/1485015740 >> 172.21.15.117:0/1485015740 conn(0x556636031460 msgr2=0x5566363e52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:25.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.777+0000 7f9a6e4cb740 1 -- 172.21.15.117:0/1485015740 shutdown_connections 2024-09-21T06:07:25.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.777+0000 7f9a6e4cb740 1 -- 172.21.15.117:0/1485015740 wait complete. 2024-09-21T06:07:25.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.777+0000 7f9a6e4cb740 20 client.17655 trim_cache size 0 max 16384 2024-09-21T06:07:25.784 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:07:25.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.777+0000 7f9a67f57640 1 -- 172.21.15.117:0/3540324716 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9a4c0b6700 msgr2=0x7f9a4c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:25.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.777+0000 7f9a67f57640 1 --2- 172.21.15.117:0/3540324716 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9a4c0b6700 0x7f9a4c0b8c10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9a5800f4c0 tx=0x7f9a58007740 comp rx=0 tx=0).stop 2024-09-21T06:07:25.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.777+0000 7f9a67f57640 1 -- 172.21.15.117:0/3540324716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a60103b60 msgr2=0x7f9a6019a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:25.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.777+0000 7f9a67f57640 1 --2- 172.21.15.117:0/3540324716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a60103b60 0x7f9a6019a5a0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c00d950 tx=0x7f9a5c00de10 comp rx=0 tx=0).stop 2024-09-21T06:07:25.784 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.777+0000 7f9a67f57640 1 -- 172.21.15.117:0/3540324716 shutdown_connections 2024-09-21T06:07:25.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.777+0000 7f9a67f57640 1 --2- 172.21.15.117:0/3540324716 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f9a4c0b6700 0x7f9a4c0b8c10 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.777+0000 7f9a67f57640 1 --2- 172.21.15.117:0/3540324716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a60109df0 0x7f9a6019fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.777+0000 7f9a67f57640 1 --2- 172.21.15.117:0/3540324716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a601094a0 0x7f9a6019ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.777+0000 7f9a67f57640 1 --2- 172.21.15.117:0/3540324716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a60103b60 0x7f9a6019a5a0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:25.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.777+0000 7f9a67f57640 1 -- 172.21.15.117:0/3540324716 >> 172.21.15.117:0/3540324716 conn(0x7f9a6007b850 msgr2=0x7f9a601075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:25.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.777+0000 7f9a67f57640 1 -- 172.21.15.117:0/3540324716 shutdown_connections 2024-09-21T06:07:25.785 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:25.777+0000 7f9a67f57640 1 -- 172.21.15.117:0/3540324716 wait complete. 2024-09-21T06:07:26.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:26 smithi117 ceph-mon[30947]: pgmap v637: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:07:26.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:26 smithi138 ceph-mon[34391]: pgmap v637: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:07:26.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:26 smithi192 ceph-mon[34700]: pgmap v637: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:07:26.490 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:07:26.734 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:07:27.149 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:27 smithi117 ceph-mon[30947]: pgmap v638: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:07:27.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.239+0000 7f00dc8d6640 1 -- 172.21.15.117:0/1667245145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00d810de10 msgr2=0x7f00d8110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:27.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.239+0000 7f00dc8d6640 1 --2- 172.21.15.117:0/1667245145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00d810de10 0x7f00d8110300 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f00d0009900 tx=0x7f00d002a410 comp rx=0 tx=0).stop 2024-09-21T06:07:27.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.239+0000 7f00dc8d6640 1 -- 172.21.15.117:0/1667245145 shutdown_connections 2024-09-21T06:07:27.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.239+0000 7f00dc8d6640 1 --2- 172.21.15.117:0/1667245145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00d810de10 0x7f00d8110300 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.239+0000 7f00dc8d6640 1 --2- 172.21.15.117:0/1667245145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00d8105300 0x7f00d810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.239+0000 7f00dc8d6640 1 --2- 172.21.15.117:0/1667245145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00d81049b0 0x7f00d8104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.252 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.239+0000 7f00dc8d6640 1 -- 172.21.15.117:0/1667245145 >> 172.21.15.117:0/1667245145 conn(0x7f00d8079730 msgr2=0x7f00d8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:27.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.239+0000 7f00dc8d6640 1 -- 172.21.15.117:0/1667245145 shutdown_connections 2024-09-21T06:07:27.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.239+0000 7f00dc8d6640 1 -- 172.21.15.117:0/1667245145 wait complete. 2024-09-21T06:07:27.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.240+0000 7f00dc8d6640 1 Processor -- start 2024-09-21T06:07:27.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.240+0000 7f00dc8d6640 1 -- start start 2024-09-21T06:07:27.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.240+0000 7f00dc8d6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00d81049b0 0x7f00d819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:27.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.240+0000 7f00dc8d6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00d8105300 0x7f00d819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:27.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.240+0000 7f00dc8d6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00d810de10 0x7f00d819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:27.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.240+0000 7f00dc8d6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f00d8113a70 con 0x7f00d81049b0 2024-09-21T06:07:27.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.240+0000 7f00dc8d6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f00d81138f0 con 0x7f00d8105300 2024-09-21T06:07:27.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.240+0000 7f00dc8d6640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f00d8113bf0 con 0x7f00d810de10 2024-09-21T06:07:27.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.240+0000 7f00d659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00d81049b0 0x7f00d819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:27.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.240+0000 7f00d659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00d81049b0 0x7f00d819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50272/0 (socket says 172.21.15.117:50272) 2024-09-21T06:07:27.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.240+0000 7f00d659c640 1 -- 172.21.15.117:0/928138061 learned_addr learned my addr 172.21.15.117:0/928138061 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:27.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.241+0000 7f00d5d9b640 1 --2- 172.21.15.117:0/928138061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00d8105300 0x7f00d819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:27.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.241+0000 7f00d6d9d640 1 --2- 172.21.15.117:0/928138061 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00d810de10 0x7f00d819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:27.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.241+0000 7f00d659c640 1 -- 172.21.15.117:0/928138061 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00d810de10 msgr2=0x7f00d819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:27.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.241+0000 7f00d659c640 1 --2- 172.21.15.117:0/928138061 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00d810de10 0x7f00d819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.241+0000 7f00d659c640 1 -- 172.21.15.117:0/928138061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00d8105300 msgr2=0x7f00d819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:27.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.241+0000 7f00d659c640 1 --2- 172.21.15.117:0/928138061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00d8105300 0x7f00d819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.241+0000 7f00d659c640 1 -- 172.21.15.117:0/928138061 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00d81a0350 con 0x7f00d81049b0 2024-09-21T06:07:27.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.241+0000 7f00d659c640 1 --2- 172.21.15.117:0/928138061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00d81049b0 0x7f00d819a4a0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f00c800ea80 tx=0x7f00c800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:27.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.241+0000 7f00c77fe640 1 -- 172.21.15.117:0/928138061 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c800cd80 con 0x7f00d81049b0 2024-09-21T06:07:27.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.241+0000 7f00dc8d6640 1 -- 172.21.15.117:0/928138061 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00d81a0640 con 0x7f00d81049b0 2024-09-21T06:07:27.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.241+0000 7f00c77fe640 1 -- 172.21.15.117:0/928138061 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00c8011920 con 0x7f00d81049b0 2024-09-21T06:07:27.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.241+0000 7f00c77fe640 1 -- 172.21.15.117:0/928138061 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c801a980 con 0x7f00d81049b0 2024-09-21T06:07:27.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.241+0000 7f00dc8d6640 1 -- 172.21.15.117:0/928138061 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00d81a6e60 con 0x7f00d81049b0 2024-09-21T06:07:27.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.244+0000 7f00e2e4a740 1 Processor -- start 2024-09-21T06:07:27.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.244+0000 7f00e2e4a740 1 -- start start 2024-09-21T06:07:27.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.244+0000 7f00c77fe640 1 -- 172.21.15.117:0/928138061 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f00c8011470 con 0x7f00d81049b0 2024-09-21T06:07:27.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.245+0000 7f00e2e4a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed288b6070 0x55ed28c5d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:27.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.245+0000 7f00e2e4a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed2888d1b0 0x55ed28c5dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:27.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.245+0000 7f00e2e4a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed28895390 0x55ed28c5e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:27.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.245+0000 7f00e2e4a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ed28bd3ea0 con 0x55ed2888d1b0 2024-09-21T06:07:27.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.245+0000 7f00e2e4a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ed28bc32d0 con 0x55ed288b6070 2024-09-21T06:07:27.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.245+0000 7f00e2e4a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55ed28bc70e0 con 0x55ed28895390 2024-09-21T06:07:27.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.245+0000 7f00d659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed2888d1b0 0x55ed28c5dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:27.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.245+0000 7f00d659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed2888d1b0 0x55ed28c5dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50282/0 (socket says 172.21.15.117:50282) 2024-09-21T06:07:27.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.245+0000 7f00d659c640 1 -- 172.21.15.117:0/3577623483 learned_addr learned my addr 172.21.15.117:0/3577623483 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:27.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.245+0000 7f00d5d9b640 1 --2- 172.21.15.117:0/3577623483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed28895390 0x55ed28c5e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:27.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.245+0000 7f00d6d9d640 1 --2- 172.21.15.117:0/3577623483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed288b6070 0x55ed28c5d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:27.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.245+0000 7f00d659c640 1 -- 172.21.15.117:0/3577623483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed28895390 msgr2=0x55ed28c5e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:27.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.245+0000 7f00d659c640 1 --2- 172.21.15.117:0/3577623483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed28895390 0x55ed28c5e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.245+0000 7f00d659c640 1 -- 172.21.15.117:0/3577623483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed288b6070 msgr2=0x55ed28c5d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:27.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.245+0000 7f00d659c640 1 --2- 172.21.15.117:0/3577623483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed288b6070 0x55ed28c5d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.245+0000 7f00d659c640 1 -- 172.21.15.117:0/3577623483 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed28921650 con 0x55ed2888d1b0 2024-09-21T06:07:27.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.245+0000 7f00d659c640 1 --2- 172.21.15.117:0/3577623483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed2888d1b0 0x55ed28c5dbe0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f00c80e5310 tx=0x7f00c80f0740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68031d0aa8c47778 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:27.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.246+0000 7f00c77fe640 1 --2- 172.21.15.117:0/928138061 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f00c00b66b0 0x7f00c00b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:27.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.246+0000 7f00c4ff9640 1 -- 172.21.15.117:0/3577623483 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c80f2860 con 0x55ed2888d1b0 2024-09-21T06:07:27.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.246+0000 7f00d6d9d640 1 --2- 172.21.15.117:0/928138061 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f00c00b66b0 0x7f00c00b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:27.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.246+0000 7f00c4ff9640 1 -- 172.21.15.117:0/3577623483 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00c80ad5b0 con 0x55ed2888d1b0 2024-09-21T06:07:27.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.246+0000 7f00c4ff9640 1 -- 172.21.15.117:0/3577623483 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c8099610 con 0x55ed2888d1b0 2024-09-21T06:07:27.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.246+0000 7f00c77fe640 1 -- 172.21.15.117:0/928138061 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f00c80e7c00 con 0x7f00d81049b0 2024-09-21T06:07:27.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.247+0000 7f00e2e4a740 1 -- 172.21.15.117:0/3577623483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed2888d1b0 msgr2=0x55ed28c5dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:27.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.247+0000 7f00d6d9d640 1 --2- 172.21.15.117:0/928138061 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f00c00b66b0 0x7f00c00b8bc0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f00d0005940 tx=0x7f00d0009660 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:27.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.247+0000 7f00e2e4a740 1 --2- 172.21.15.117:0/3577623483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed2888d1b0 0x55ed28c5dbe0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f00c80e5310 tx=0x7f00c80f0740 comp rx=0 tx=0).stop 2024-09-21T06:07:27.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.247+0000 7f00e2e4a740 1 -- 172.21.15.117:0/3577623483 shutdown_connections 2024-09-21T06:07:27.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.247+0000 7f00e2e4a740 1 --2- 172.21.15.117:0/3577623483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed2888d1b0 0x55ed28c5dbe0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.247+0000 7f00e2e4a740 1 --2- 172.21.15.117:0/3577623483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed28895390 0x55ed28c5e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.247+0000 7f00e2e4a740 1 --2- 172.21.15.117:0/3577623483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed288b6070 0x55ed28c5d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.247+0000 7f00e2e4a740 1 -- 172.21.15.117:0/3577623483 >> 172.21.15.117:0/3577623483 conn(0x55ed288aa460 msgr2=0x55ed28c5b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:27.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.247+0000 7f00e2e4a740 1 -- 172.21.15.117:0/3577623483 shutdown_connections 2024-09-21T06:07:27.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.247+0000 7f00e2e4a740 1 -- 172.21.15.117:0/3577623483 wait complete. 2024-09-21T06:07:27.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.247+0000 7f00e2e4a740 1 -- start start 2024-09-21T06:07:27.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.247+0000 7f00e2e4a740 1 Processor -- start 2024-09-21T06:07:27.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.248+0000 7f00e2e4a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed2888d1b0 0x55ed28c68d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:27.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.248+0000 7f00e2e4a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed28895390 0x55ed28c692b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:27.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.248+0000 7f00e2e4a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed288b6070 0x55ed28c751a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:27.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.248+0000 7f00e2e4a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ed28a24700 con 0x55ed2888d1b0 2024-09-21T06:07:27.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.248+0000 7f00e2e4a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ed28b5e160 con 0x55ed28895390 2024-09-21T06:07:27.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.248+0000 7f00e2e4a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55ed28c6be80 con 0x55ed288b6070 2024-09-21T06:07:27.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.248+0000 7f00e2e4a740 10 client.0 _ll_register_callbacks cb 0x55ed28b779e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:27.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.248+0000 7f00d659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed2888d1b0 0x55ed28c68d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:27.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.248+0000 7f00d659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed2888d1b0 0x55ed28c68d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50286/0 (socket says 172.21.15.117:50286) 2024-09-21T06:07:27.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.248+0000 7f00d659c640 1 -- 172.21.15.117:0/2262419854 learned_addr learned my addr 172.21.15.117:0/2262419854 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:27.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.248+0000 7f00d5d9b640 1 --2- 172.21.15.117:0/2262419854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed28895390 0x55ed28c692b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:27.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.248+0000 7f00d6d9d640 1 --2- 172.21.15.117:0/2262419854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed288b6070 0x55ed28c751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:27.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.248+0000 7f00d659c640 1 -- 172.21.15.117:0/2262419854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed288b6070 msgr2=0x55ed28c751a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:27.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.248+0000 7f00d659c640 1 --2- 172.21.15.117:0/2262419854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed288b6070 0x55ed28c751a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.248+0000 7f00d659c640 1 -- 172.21.15.117:0/2262419854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed28895390 msgr2=0x55ed28c692b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:27.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.248+0000 7f00d659c640 1 --2- 172.21.15.117:0/2262419854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed28895390 0x55ed28c692b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.248+0000 7f00d659c640 1 -- 172.21.15.117:0/2262419854 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed28b8bc90 con 0x55ed2888d1b0 2024-09-21T06:07:27.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.248+0000 7f00d6d9d640 1 --2- 172.21.15.117:0/2262419854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed288b6070 0x55ed28c751a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:07:27.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.248+0000 7f00d659c640 1 --2- 172.21.15.117:0/2262419854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed2888d1b0 0x55ed28c68d70 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f00c80aad10 tx=0x7f00c8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a40046f58c6b34f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:27.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.248+0000 7f00bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:27.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.249+0000 7f00bdffb640 1 -- 172.21.15.117:0/2262419854 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c809eb70 con 0x55ed2888d1b0 2024-09-21T06:07:27.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.249+0000 7f00bdffb640 1 -- 172.21.15.117:0/2262419854 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f00c8011ad0 con 0x55ed2888d1b0 2024-09-21T06:07:27.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.249+0000 7f00e2e4a740 1 -- 172.21.15.117:0/2262419854 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00c80ad5b0 con 0x55ed2888d1b0 2024-09-21T06:07:27.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.249+0000 7f00bdffb640 1 -- 172.21.15.117:0/2262419854 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f00c80eb020 con 0x55ed2888d1b0 2024-09-21T06:07:27.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.249+0000 7f00e2e4a740 10 client.17667 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:27.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.249+0000 7f00e2e4a740 1 -- 172.21.15.117:0/2262419854 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ed28c77b80 con 0x55ed2888d1b0 2024-09-21T06:07:27.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.249+0000 7f00bdffb640 1 -- 172.21.15.117:0/2262419854 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00c80ad5b0 con 0x55ed2888d1b0 2024-09-21T06:07:27.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.249+0000 7f00bdffb640 1 -- 172.21.15.117:0/2262419854 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f00c80ef040 con 0x55ed2888d1b0 2024-09-21T06:07:27.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.249+0000 7f00e2e4a740 10 client.17667 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:27.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.249+0000 7f00e2e4a740 10 client.17667 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:07:27.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.249+0000 7f00e2e4a740 20 client.17667 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:27.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.249+0000 7f00e2e4a740 1 --2- 172.21.15.117:0/2262419854 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ed288749f0 0x55ed28c7aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:27.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.249+0000 7f00e2e4a740 4 client.17667 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:07:27.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.249+0000 7f00e2e4a740 1 -- 172.21.15.117:0/2262419854 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed28bd5df0 con 0x55ed288749f0 2024-09-21T06:07:27.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.249+0000 7f00d5d9b640 1 --2- 172.21.15.117:0/2262419854 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ed288749f0 0x55ed28c7aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:27.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.250+0000 7f00bdffb640 10 client.17667 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:27.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.250+0000 7f00d5d9b640 1 --2- 172.21.15.117:0/2262419854 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ed288749f0 0x55ed28c7aee0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a7f5b691498ed3f server_cookie=28d8370772a96bdd in_seq=0 out_seq=0 2024-09-21T06:07:27.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.251+0000 7f00bdffb640 1 -- 172.21.15.117:0/2262419854 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ed28bd5df0 con 0x55ed288749f0 2024-09-21T06:07:27.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.251+0000 7f00bdffb640 10 client.17667 handle_command_reply: tid=0 2024-09-21T06:07:27.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.251+0000 7f00e2e4a740 1 client.17667 shutdown 2024-09-21T06:07:27.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.251+0000 7f00e2e4a740 1 -- 172.21.15.117:0/2262419854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed2888d1b0 msgr2=0x55ed28c68d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:27.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.251+0000 7f00e2e4a740 1 --2- 172.21.15.117:0/2262419854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed2888d1b0 0x55ed28c68d70 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f00c80aad10 tx=0x7f00c8004ab0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.251+0000 7f00e2e4a740 1 -- 172.21.15.117:0/2262419854 shutdown_connections 2024-09-21T06:07:27.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.251+0000 7f00e2e4a740 1 --2- 172.21.15.117:0/2262419854 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ed288749f0 0x55ed28c7aee0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.251+0000 7f00e2e4a740 1 --2- 172.21.15.117:0/2262419854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed288b6070 0x55ed28c751a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.251+0000 7f00e2e4a740 1 --2- 172.21.15.117:0/2262419854 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed28895390 0x55ed28c692b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.251+0000 7f00e2e4a740 1 --2- 172.21.15.117:0/2262419854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed2888d1b0 0x55ed28c68d70 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.251+0000 7f00e2e4a740 1 -- 172.21.15.117:0/2262419854 >> 172.21.15.117:0/2262419854 conn(0x55ed288aa460 msgr2=0x55ed28c60830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:27.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.251+0000 7f00bdffb640 0 client.17667 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:27.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.251+0000 7f00e2e4a740 1 -- 172.21.15.117:0/2262419854 shutdown_connections 2024-09-21T06:07:27.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.251+0000 7f00e2e4a740 1 -- 172.21.15.117:0/2262419854 wait complete. 2024-09-21T06:07:27.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.251+0000 7f00e2e4a740 20 client.17667 trim_cache size 0 max 16384 2024-09-21T06:07:27.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.264+0000 7f00e2e4a740 1 Processor -- start 2024-09-21T06:07:27.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.264+0000 7f00e2e4a740 1 -- start start 2024-09-21T06:07:27.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.264+0000 7f00e2e4a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed2888d1b0 0x55ed28c5ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:27.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.264+0000 7f00e2e4a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed28895390 0x55ed28c75110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:27.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.264+0000 7f00e2e4a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed288b6070 0x55ed28c65f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:27.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.264+0000 7f00e2e4a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f00d81af5a0 con 0x55ed28895390 2024-09-21T06:07:27.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.264+0000 7f00e2e4a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f00d81b10e0 con 0x55ed2888d1b0 2024-09-21T06:07:27.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.264+0000 7f00e2e4a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f00d81afc70 con 0x55ed288b6070 2024-09-21T06:07:27.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.264+0000 7f00d5d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed28895390 0x55ed28c75110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:27.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.264+0000 7f00d5d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed28895390 0x55ed28c75110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50288/0 (socket says 172.21.15.117:50288) 2024-09-21T06:07:27.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.264+0000 7f00d5d9b640 1 -- 172.21.15.117:0/833288512 learned_addr learned my addr 172.21.15.117:0/833288512 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:27.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.264+0000 7f00d659c640 1 --2- 172.21.15.117:0/833288512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed2888d1b0 0x55ed28c5ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:27.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.265+0000 7f00d6d9d640 1 --2- 172.21.15.117:0/833288512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed288b6070 0x55ed28c65f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:27.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.265+0000 7f00d5d9b640 1 -- 172.21.15.117:0/833288512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed288b6070 msgr2=0x55ed28c65f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:27.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.265+0000 7f00d5d9b640 1 --2- 172.21.15.117:0/833288512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed288b6070 0x55ed28c65f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.265+0000 7f00d5d9b640 1 -- 172.21.15.117:0/833288512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed2888d1b0 msgr2=0x55ed28c5ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:27.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.265+0000 7f00d5d9b640 1 --2- 172.21.15.117:0/833288512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed2888d1b0 0x55ed28c5ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.265+0000 7f00d5d9b640 1 -- 172.21.15.117:0/833288512 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed28c5b7e0 con 0x55ed28895390 2024-09-21T06:07:27.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.265+0000 7f00d5d9b640 1 --2- 172.21.15.117:0/833288512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed28895390 0x55ed28c75110 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f00d806ad30 tx=0x7f00cc00a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b06251054d920b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:27.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.265+0000 7f00bffff640 1 -- 172.21.15.117:0/833288512 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00cc024610 con 0x55ed28895390 2024-09-21T06:07:27.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.265+0000 7f00bffff640 1 -- 172.21.15.117:0/833288512 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00cc024c60 con 0x55ed28895390 2024-09-21T06:07:27.268 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.265+0000 7f00bffff640 1 -- 172.21.15.117:0/833288512 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00cc01ec60 con 0x55ed28895390 2024-09-21T06:07:27.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.266+0000 7f00e2e4a740 1 -- 172.21.15.117:0/833288512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed28895390 msgr2=0x55ed28c75110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:27.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.266+0000 7f00e2e4a740 1 --2- 172.21.15.117:0/833288512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed28895390 0x55ed28c75110 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f00d806ad30 tx=0x7f00cc00a2d0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.266+0000 7f00e2e4a740 1 -- 172.21.15.117:0/833288512 shutdown_connections 2024-09-21T06:07:27.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.266+0000 7f00e2e4a740 1 --2- 172.21.15.117:0/833288512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed28895390 0x55ed28c75110 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.266+0000 7f00e2e4a740 1 --2- 172.21.15.117:0/833288512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed288b6070 0x55ed28c65f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.266+0000 7f00e2e4a740 1 --2- 172.21.15.117:0/833288512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed2888d1b0 0x55ed28c5ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.266+0000 7f00e2e4a740 1 -- 172.21.15.117:0/833288512 >> 172.21.15.117:0/833288512 conn(0x55ed288aa460 msgr2=0x55ed28c5e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:27.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.266+0000 7f00e2e4a740 1 -- 172.21.15.117:0/833288512 shutdown_connections 2024-09-21T06:07:27.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.266+0000 7f00e2e4a740 1 -- 172.21.15.117:0/833288512 wait complete. 2024-09-21T06:07:27.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.266+0000 7f00e2e4a740 1 -- start start 2024-09-21T06:07:27.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.266+0000 7f00e2e4a740 1 Processor -- start 2024-09-21T06:07:27.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.267+0000 7f00e2e4a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed2888d1b0 0x55ed28c7b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:27.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.267+0000 7f00e2e4a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed28895390 0x55ed28c7bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:27.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.267+0000 7f00e2e4a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed288b6070 0x55ed28ce0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:27.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.267+0000 7f00e2e4a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f00d81b0a10 con 0x55ed288b6070 2024-09-21T06:07:27.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.267+0000 7f00e2e4a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f00d81b0340 con 0x55ed2888d1b0 2024-09-21T06:07:27.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.267+0000 7f00e2e4a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f00d81ad5a0 con 0x55ed28895390 2024-09-21T06:07:27.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.267+0000 7f00e2e4a740 10 client.0 _ll_register_callbacks cb 0x7f00d0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:27.273 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.267+0000 7f00d6d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed288b6070 0x55ed28ce0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:27.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.267+0000 7f00d6d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed288b6070 0x55ed28ce0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50294/0 (socket says 172.21.15.117:50294) 2024-09-21T06:07:27.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.267+0000 7f00d6d9d640 1 -- 172.21.15.117:0/2319216132 learned_addr learned my addr 172.21.15.117:0/2319216132 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:27.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.267+0000 7f00d659c640 1 --2- 172.21.15.117:0/2319216132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed2888d1b0 0x55ed28c7b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:27.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.267+0000 7f00d5d9b640 1 --2- 172.21.15.117:0/2319216132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed28895390 0x55ed28c7bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:27.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.267+0000 7f00d6d9d640 1 -- 172.21.15.117:0/2319216132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed28895390 msgr2=0x55ed28c7bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:27.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.267+0000 7f00d6d9d640 1 --2- 172.21.15.117:0/2319216132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed28895390 0x55ed28c7bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.267+0000 7f00d6d9d640 1 -- 172.21.15.117:0/2319216132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed2888d1b0 msgr2=0x55ed28c7b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:27.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.267+0000 7f00d6d9d640 1 --2- 172.21.15.117:0/2319216132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed2888d1b0 0x55ed28c7b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.267+0000 7f00d6d9d640 1 -- 172.21.15.117:0/2319216132 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed28ce1090 con 0x55ed288b6070 2024-09-21T06:07:27.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.267+0000 7f00d6d9d640 1 --2- 172.21.15.117:0/2319216132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed288b6070 0x55ed28ce0a20 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f00d0009900 tx=0x7f00d0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce76a79c663a1c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:27.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.267+0000 7f00bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:27.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.267+0000 7f00bdffb640 1 -- 172.21.15.117:0/2319216132 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d0035040 con 0x55ed288b6070 2024-09-21T06:07:27.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.267+0000 7f00e2e4a740 1 -- 172.21.15.117:0/2319216132 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00cc024c60 con 0x55ed288b6070 2024-09-21T06:07:27.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.267+0000 7f00bdffb640 1 -- 172.21.15.117:0/2319216132 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f00d0005450 con 0x55ed288b6070 2024-09-21T06:07:27.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.268+0000 7f00bdffb640 1 -- 172.21.15.117:0/2319216132 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f00d0005650 con 0x55ed288b6070 2024-09-21T06:07:27.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.268+0000 7f00e2e4a740 10 client.17673 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:27.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.268+0000 7f00e2e4a740 1 -- 172.21.15.117:0/2319216132 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ed28ce95e0 con 0x55ed288b6070 2024-09-21T06:07:27.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.268+0000 7f00bdffb640 1 -- 172.21.15.117:0/2319216132 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00cc024c60 con 0x55ed288b6070 2024-09-21T06:07:27.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.268+0000 7f00bdffb640 1 -- 172.21.15.117:0/2319216132 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f00d0003920 con 0x55ed288b6070 2024-09-21T06:07:27.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.268+0000 7f00e2e4a740 10 client.17673 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:27.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.268+0000 7f00e2e4a740 10 client.17673 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:07:27.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.268+0000 7f00e2e4a740 20 client.17673 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:27.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.268+0000 7f00e2e4a740 1 --2- 172.21.15.117:0/2319216132 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ed28849da0 0x55ed28cebdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:27.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.268+0000 7f00e2e4a740 4 client.17673 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:07:27.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.268+0000 7f00e2e4a740 1 -- 172.21.15.117:0/2319216132 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ed28bd18a0 con 0x55ed28849da0 2024-09-21T06:07:27.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.268+0000 7f00d659c640 1 --2- 172.21.15.117:0/2319216132 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ed28849da0 0x55ed28cebdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:27.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.269+0000 7f00d659c640 1 --2- 172.21.15.117:0/2319216132 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ed28849da0 0x55ed28cebdb0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5ae52aa38b3ec56 server_cookie=2d0b8ec5ef6748bd in_seq=0 out_seq=0 2024-09-21T06:07:27.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.269+0000 7f00bdffb640 10 client.17673 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:27.275 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.269+0000 7f00bdffb640 1 -- 172.21.15.117:0/2319216132 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ed28bd18a0 con 0x55ed28849da0 2024-09-21T06:07:27.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.269+0000 7f00bdffb640 10 client.17673 handle_command_reply: tid=0 2024-09-21T06:07:27.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.269+0000 7f00e2e4a740 1 client.17673 shutdown 2024-09-21T06:07:27.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.269+0000 7f00e2e4a740 1 -- 172.21.15.117:0/2319216132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed288b6070 msgr2=0x55ed28ce0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:27.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.269+0000 7f00e2e4a740 1 --2- 172.21.15.117:0/2319216132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed288b6070 0x55ed28ce0a20 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f00d0009900 tx=0x7f00d0009260 comp rx=0 tx=0).stop 2024-09-21T06:07:27.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.269+0000 7f00e2e4a740 1 -- 172.21.15.117:0/2319216132 shutdown_connections 2024-09-21T06:07:27.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.269+0000 7f00e2e4a740 1 --2- 172.21.15.117:0/2319216132 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ed28849da0 0x55ed28cebdb0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.269+0000 7f00e2e4a740 1 --2- 172.21.15.117:0/2319216132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ed288b6070 0x55ed28ce0a20 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.269+0000 7f00e2e4a740 1 --2- 172.21.15.117:0/2319216132 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ed28895390 0x55ed28c7bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.269+0000 7f00bdffb640 0 client.17673 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:27.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.269+0000 7f00e2e4a740 1 --2- 172.21.15.117:0/2319216132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed2888d1b0 0x55ed28c7b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.269+0000 7f00e2e4a740 1 -- 172.21.15.117:0/2319216132 >> 172.21.15.117:0/2319216132 conn(0x55ed288aa460 msgr2=0x55ed28c6a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:27.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.269+0000 7f00e2e4a740 1 -- 172.21.15.117:0/2319216132 shutdown_connections 2024-09-21T06:07:27.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.269+0000 7f00e2e4a740 1 -- 172.21.15.117:0/2319216132 wait complete. 2024-09-21T06:07:27.276 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.269+0000 7f00e2e4a740 20 client.17673 trim_cache size 0 max 16384 2024-09-21T06:07:27.277 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:07:27.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.270+0000 7f00dc8d6640 1 -- 172.21.15.117:0/928138061 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f00c00b66b0 msgr2=0x7f00c00b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:27.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.270+0000 7f00dc8d6640 1 --2- 172.21.15.117:0/928138061 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f00c00b66b0 0x7f00c00b8bc0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f00d0005940 tx=0x7f00d0009660 comp rx=0 tx=0).stop 2024-09-21T06:07:27.277 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.270+0000 7f00dc8d6640 1 -- 172.21.15.117:0/928138061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00d81049b0 msgr2=0x7f00d819a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:27.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.270+0000 7f00dc8d6640 1 --2- 172.21.15.117:0/928138061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00d81049b0 0x7f00d819a4a0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f00c800ea80 tx=0x7f00c800ee80 comp rx=0 tx=0).stop 2024-09-21T06:07:27.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.270+0000 7f00dc8d6640 1 -- 172.21.15.117:0/928138061 shutdown_connections 2024-09-21T06:07:27.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.270+0000 7f00dc8d6640 1 --2- 172.21.15.117:0/928138061 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f00c00b66b0 0x7f00c00b8bc0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.270+0000 7f00dc8d6640 1 --2- 172.21.15.117:0/928138061 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00d810de10 0x7f00d819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.270+0000 7f00dc8d6640 1 --2- 172.21.15.117:0/928138061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00d8105300 0x7f00d819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.270+0000 7f00dc8d6640 1 --2- 172.21.15.117:0/928138061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00d81049b0 0x7f00d819a4a0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:27.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.270+0000 7f00dc8d6640 1 -- 172.21.15.117:0/928138061 >> 172.21.15.117:0/928138061 conn(0x7f00d8079730 msgr2=0x7f00d81007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:27.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.270+0000 7f00dc8d6640 1 -- 172.21.15.117:0/928138061 shutdown_connections 2024-09-21T06:07:27.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:27.270+0000 7f00dc8d6640 1 -- 172.21.15.117:0/928138061 wait complete. 2024-09-21T06:07:27.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:27 smithi138 ceph-mon[34391]: pgmap v638: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:07:27.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:27 smithi192 ceph-mon[34700]: pgmap v638: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:07:28.981 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:07:28.981 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:07:29.242 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:07:29.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.717+0000 7fcc47b00640 1 -- 172.21.15.117:0/2004136669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc40109df0 msgr2=0x7fcc40110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:29.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.717+0000 7fcc47b00640 1 --2- 172.21.15.117:0/2004136669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc40109df0 0x7fcc40110620 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fcc38009ca0 tx=0x7fcc3802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.717+0000 7fcc47b00640 1 -- 172.21.15.117:0/2004136669 shutdown_connections 2024-09-21T06:07:29.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.717+0000 7fcc47b00640 1 --2- 172.21.15.117:0/2004136669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc40109df0 0x7fcc40110620 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.717+0000 7fcc47b00640 1 --2- 172.21.15.117:0/2004136669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc401094a0 0x7fcc401098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.717+0000 7fcc47b00640 1 --2- 172.21.15.117:0/2004136669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc40103b60 0x7fcc40103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.717+0000 7fcc47b00640 1 -- 172.21.15.117:0/2004136669 >> 172.21.15.117:0/2004136669 conn(0x7fcc4007b850 msgr2=0x7fcc4007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:29.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.717+0000 7fcc47b00640 1 -- 172.21.15.117:0/2004136669 shutdown_connections 2024-09-21T06:07:29.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.717+0000 7fcc47b00640 1 -- 172.21.15.117:0/2004136669 wait complete. 2024-09-21T06:07:29.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.718+0000 7fcc47b00640 1 Processor -- start 2024-09-21T06:07:29.721 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.718+0000 7fcc47b00640 1 -- start start 2024-09-21T06:07:29.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.718+0000 7fcc47b00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc40103b60 0x7fcc4019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:29.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.718+0000 7fcc47b00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc401094a0 0x7fcc4019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:29.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.718+0000 7fcc47b00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc40109df0 0x7fcc4019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:29.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.718+0000 7fcc47b00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcc40113ab0 con 0x7fcc401094a0 2024-09-21T06:07:29.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.718+0000 7fcc47b00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcc40113930 con 0x7fcc40103b60 2024-09-21T06:07:29.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.718+0000 7fcc47b00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcc40113c30 con 0x7fcc40109df0 2024-09-21T06:07:29.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.718+0000 7fcc4509b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc401094a0 0x7fcc4019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:29.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.718+0000 7fcc4509b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc401094a0 0x7fcc4019a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50310/0 (socket says 172.21.15.117:50310) 2024-09-21T06:07:29.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.718+0000 7fcc4509b640 1 -- 172.21.15.117:0/4132083140 learned_addr learned my addr 172.21.15.117:0/4132083140 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:29.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.718+0000 7fcc4589c640 1 --2- 172.21.15.117:0/4132083140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc40103b60 0x7fcc4019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:29.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.718+0000 7fcc4609d640 1 --2- 172.21.15.117:0/4132083140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc40109df0 0x7fcc4019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:29.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.718+0000 7fcc4509b640 1 -- 172.21.15.117:0/4132083140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc40109df0 msgr2=0x7fcc4019fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:29.722 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.718+0000 7fcc4509b640 1 --2- 172.21.15.117:0/4132083140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc40109df0 0x7fcc4019fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.718+0000 7fcc4509b640 1 -- 172.21.15.117:0/4132083140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc40103b60 msgr2=0x7fcc4019a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:29.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.718+0000 7fcc4509b640 1 --2- 172.21.15.117:0/4132083140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc40103b60 0x7fcc4019a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.718+0000 7fcc4509b640 1 -- 172.21.15.117:0/4132083140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc401a03b0 con 0x7fcc401094a0 2024-09-21T06:07:29.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.719+0000 7fcc4509b640 1 --2- 172.21.15.117:0/4132083140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc401094a0 0x7fcc4019a9e0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fcc3000d740 tx=0x7fcc3000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:29.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.719+0000 7fcc36ffd640 1 -- 172.21.15.117:0/4132083140 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc30017070 con 0x7fcc401094a0 2024-09-21T06:07:29.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.719+0000 7fcc36ffd640 1 -- 172.21.15.117:0/4132083140 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc3000de50 con 0x7fcc401094a0 2024-09-21T06:07:29.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.719+0000 7fcc47b00640 1 -- 172.21.15.117:0/4132083140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc401a06a0 con 0x7fcc401094a0 2024-09-21T06:07:29.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.719+0000 7fcc36ffd640 1 -- 172.21.15.117:0/4132083140 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3000f620 con 0x7fcc401094a0 2024-09-21T06:07:29.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.719+0000 7fcc47b00640 1 -- 172.21.15.117:0/4132083140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc401a6e00 con 0x7fcc401094a0 2024-09-21T06:07:29.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.721+0000 7fcc4e074740 1 Processor -- start 2024-09-21T06:07:29.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.722+0000 7fcc4e074740 1 -- start start 2024-09-21T06:07:29.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.722+0000 7fcc36ffd640 1 -- 172.21.15.117:0/4132083140 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fcc3001a070 con 0x7fcc401094a0 2024-09-21T06:07:29.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.722+0000 7fcc4e074740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bea4bd7350 0x55bea4f7e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:29.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.722+0000 7fcc4e074740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea4bae490 0x55bea4f7e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:29.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.722+0000 7fcc4e074740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bea4bb6670 0x55bea4f7eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:29.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.722+0000 7fcc4e074740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bea4ef4e60 con 0x55bea4bae490 2024-09-21T06:07:29.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.722+0000 7fcc4e074740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bea4ee4290 con 0x55bea4bb6670 2024-09-21T06:07:29.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.722+0000 7fcc4e074740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55bea4ee80a0 con 0x55bea4bd7350 2024-09-21T06:07:29.725 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.722+0000 7fcc4589c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea4bae490 0x55bea4f7e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:29.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.722+0000 7fcc4589c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea4bae490 0x55bea4f7e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50316/0 (socket says 172.21.15.117:50316) 2024-09-21T06:07:29.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.722+0000 7fcc4589c640 1 -- 172.21.15.117:0/3789750875 learned_addr learned my addr 172.21.15.117:0/3789750875 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:29.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.722+0000 7fcc4609d640 1 --2- 172.21.15.117:0/3789750875 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bea4bd7350 0x55bea4f7e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:29.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.722+0000 7fcc4509b640 1 --2- 172.21.15.117:0/3789750875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bea4bb6670 0x55bea4f7eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:29.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.722+0000 7fcc4589c640 1 -- 172.21.15.117:0/3789750875 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bea4bd7350 msgr2=0x55bea4f7e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:29.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.722+0000 7fcc4589c640 1 --2- 172.21.15.117:0/3789750875 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bea4bd7350 0x55bea4f7e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.722+0000 7fcc4589c640 1 -- 172.21.15.117:0/3789750875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bea4bb6670 msgr2=0x55bea4f7eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:29.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.722+0000 7fcc4589c640 1 --2- 172.21.15.117:0/3789750875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bea4bb6670 0x55bea4f7eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.722+0000 7fcc4589c640 1 -- 172.21.15.117:0/3789750875 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bea4c42930 con 0x55bea4bae490 2024-09-21T06:07:29.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.722+0000 7fcc4509b640 1 --2- 172.21.15.117:0/3789750875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bea4bb6670 0x55bea4f7eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:07:29.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.723+0000 7fcc4589c640 1 --2- 172.21.15.117:0/3789750875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea4bae490 0x55bea4f7e970 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00b760 tx=0x7fcc3c00bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab6ff719d9ef313 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:29.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.723+0000 7fcc36ffd640 1 --2- 172.21.15.117:0/4132083140 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fcc2c0b6700 0x7fcc2c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:29.726 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.723+0000 7fcc2bfff640 1 -- 172.21.15.117:0/3789750875 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c003dd0 con 0x55bea4bae490 2024-09-21T06:07:29.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.723+0000 7fcc4609d640 1 --2- 172.21.15.117:0/4132083140 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fcc2c0b6700 0x7fcc2c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:29.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.723+0000 7fcc2bfff640 1 -- 172.21.15.117:0/3789750875 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc3c015070 con 0x55bea4bae490 2024-09-21T06:07:29.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.723+0000 7fcc2bfff640 1 -- 172.21.15.117:0/3789750875 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c004d90 con 0x55bea4bae490 2024-09-21T06:07:29.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.723+0000 7fcc36ffd640 1 -- 172.21.15.117:0/4132083140 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7fcc300e7040 con 0x7fcc401094a0 2024-09-21T06:07:29.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.724+0000 7fcc4e074740 1 -- 172.21.15.117:0/3789750875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea4bae490 msgr2=0x55bea4f7e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:29.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.724+0000 7fcc4609d640 1 --2- 172.21.15.117:0/4132083140 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fcc2c0b6700 0x7fcc2c0b8c10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcc38006010 tx=0x7fcc380093f0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:29.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.724+0000 7fcc4e074740 1 --2- 172.21.15.117:0/3789750875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea4bae490 0x55bea4f7e970 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00b760 tx=0x7fcc3c00bc20 comp rx=0 tx=0).stop 2024-09-21T06:07:29.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.724+0000 7fcc4e074740 1 -- 172.21.15.117:0/3789750875 shutdown_connections 2024-09-21T06:07:29.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.724+0000 7fcc4e074740 1 --2- 172.21.15.117:0/3789750875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea4bae490 0x55bea4f7e970 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.724+0000 7fcc4e074740 1 --2- 172.21.15.117:0/3789750875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bea4bb6670 0x55bea4f7eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.724+0000 7fcc4e074740 1 --2- 172.21.15.117:0/3789750875 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bea4bd7350 0x55bea4f7e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.724+0000 7fcc4e074740 1 -- 172.21.15.117:0/3789750875 >> 172.21.15.117:0/3789750875 conn(0x55bea4bcb740 msgr2=0x55bea4f7c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:29.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.724+0000 7fcc4e074740 1 -- 172.21.15.117:0/3789750875 shutdown_connections 2024-09-21T06:07:29.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.724+0000 7fcc4e074740 1 -- 172.21.15.117:0/3789750875 wait complete. 2024-09-21T06:07:29.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.724+0000 7fcc4e074740 1 -- start start 2024-09-21T06:07:29.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.724+0000 7fcc4e074740 1 Processor -- start 2024-09-21T06:07:29.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.725+0000 7fcc4e074740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea4bae490 0x55bea4f89ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:29.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.725+0000 7fcc4e074740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bea4bb6670 0x55bea4f8a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:29.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.725+0000 7fcc4e074740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bea4bd7350 0x55bea4f95f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:29.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.725+0000 7fcc4e074740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bea4d45890 con 0x55bea4bae490 2024-09-21T06:07:29.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.725+0000 7fcc4e074740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bea4e7ed40 con 0x55bea4bb6670 2024-09-21T06:07:29.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.725+0000 7fcc4e074740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55bea4f8cc10 con 0x55bea4bd7350 2024-09-21T06:07:29.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.725+0000 7fcc4e074740 10 client.0 _ll_register_callbacks cb 0x55bea4ee2c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:29.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.725+0000 7fcc4589c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea4bae490 0x55bea4f89ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:29.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.725+0000 7fcc4589c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea4bae490 0x55bea4f89ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50332/0 (socket says 172.21.15.117:50332) 2024-09-21T06:07:29.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.725+0000 7fcc4589c640 1 -- 172.21.15.117:0/185824028 learned_addr learned my addr 172.21.15.117:0/185824028 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:29.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.725+0000 7fcc4509b640 1 --2- 172.21.15.117:0/185824028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bea4bb6670 0x55bea4f8a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:29.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.725+0000 7fcc4609d640 1 --2- 172.21.15.117:0/185824028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bea4bd7350 0x55bea4f95f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:29.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.725+0000 7fcc4589c640 1 -- 172.21.15.117:0/185824028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bea4bd7350 msgr2=0x55bea4f95f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:29.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.725+0000 7fcc4589c640 1 --2- 172.21.15.117:0/185824028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bea4bd7350 0x55bea4f95f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.725+0000 7fcc4589c640 1 -- 172.21.15.117:0/185824028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bea4bb6670 msgr2=0x55bea4f8a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:29.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.725+0000 7fcc4589c640 1 --2- 172.21.15.117:0/185824028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bea4bb6670 0x55bea4f8a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.725+0000 7fcc4589c640 1 -- 172.21.15.117:0/185824028 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bea4f964d0 con 0x55bea4bae490 2024-09-21T06:07:29.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.725+0000 7fcc4609d640 1 --2- 172.21.15.117:0/185824028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bea4bd7350 0x55bea4f95f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:29.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.726+0000 7fcc4589c640 1 --2- 172.21.15.117:0/185824028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea4bae490 0x55bea4f89ff0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c006010 tx=0x7fcc3c003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a1196b2293353e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:29.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.726+0000 7fcc297fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:29.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.726+0000 7fcc297fa640 1 -- 172.21.15.117:0/185824028 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c028060 con 0x55bea4bae490 2024-09-21T06:07:29.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.726+0000 7fcc297fa640 1 -- 172.21.15.117:0/185824028 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7fcc3c011850 con 0x55bea4bae490 2024-09-21T06:07:29.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.726+0000 7fcc4e074740 1 -- 172.21.15.117:0/185824028 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc3c015070 con 0x55bea4bae490 2024-09-21T06:07:29.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.726+0000 7fcc297fa640 1 -- 172.21.15.117:0/185824028 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcc3c0046c0 con 0x55bea4bae490 2024-09-21T06:07:29.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.726+0000 7fcc4e074740 10 client.17685 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:29.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.726+0000 7fcc4e074740 1 -- 172.21.15.117:0/185824028 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bea4f98d70 con 0x55bea4bae490 2024-09-21T06:07:29.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.726+0000 7fcc297fa640 1 -- 172.21.15.117:0/185824028 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc3c015070 con 0x55bea4bae490 2024-09-21T06:07:29.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.726+0000 7fcc297fa640 1 -- 172.21.15.117:0/185824028 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcc3c022070 con 0x55bea4bae490 2024-09-21T06:07:29.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.726+0000 7fcc4e074740 10 client.17685 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:29.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.726+0000 7fcc4e074740 10 client.17685 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:07:29.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.726+0000 7fcc4e074740 20 client.17685 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:29.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.726+0000 7fcc4e074740 1 --2- 172.21.15.117:0/185824028 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55bea4b95cd0 0x55bea4f9c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:29.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.726+0000 7fcc4e074740 4 client.17685 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:07:29.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.726+0000 7fcc4e074740 1 -- 172.21.15.117:0/185824028 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bea4ef6db0 con 0x55bea4b95cd0 2024-09-21T06:07:29.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.727+0000 7fcc4509b640 1 --2- 172.21.15.117:0/185824028 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55bea4b95cd0 0x55bea4f9c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:29.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.727+0000 7fcc297fa640 10 client.17685 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:29.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.727+0000 7fcc4509b640 1 --2- 172.21.15.117:0/185824028 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55bea4b95cd0 0x55bea4f9c120 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e12c22152c36522d server_cookie=4504091c03d5437c in_seq=0 out_seq=0 2024-09-21T06:07:29.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.728+0000 7fcc297fa640 1 -- 172.21.15.117:0/185824028 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fcc30086e50 con 0x55bea4b95cd0 2024-09-21T06:07:29.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.728+0000 7fcc297fa640 10 client.17685 handle_command_reply: tid=0 2024-09-21T06:07:29.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.728+0000 7fcc4e074740 1 client.17685 shutdown 2024-09-21T06:07:29.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.728+0000 7fcc4e074740 1 -- 172.21.15.117:0/185824028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea4bae490 msgr2=0x55bea4f89ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:29.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.728+0000 7fcc4e074740 1 --2- 172.21.15.117:0/185824028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea4bae490 0x55bea4f89ff0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c006010 tx=0x7fcc3c003e00 comp rx=0 tx=0).stop 2024-09-21T06:07:29.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.728+0000 7fcc4e074740 1 -- 172.21.15.117:0/185824028 shutdown_connections 2024-09-21T06:07:29.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.728+0000 7fcc4e074740 1 --2- 172.21.15.117:0/185824028 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55bea4b95cd0 0x55bea4f9c120 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.728+0000 7fcc4e074740 1 --2- 172.21.15.117:0/185824028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bea4bd7350 0x55bea4f95f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.728+0000 7fcc4e074740 1 --2- 172.21.15.117:0/185824028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bea4bb6670 0x55bea4f8a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.728+0000 7fcc4e074740 1 --2- 172.21.15.117:0/185824028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea4bae490 0x55bea4f89ff0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.728+0000 7fcc297fa640 0 client.17685 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:29.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.728+0000 7fcc4e074740 1 -- 172.21.15.117:0/185824028 >> 172.21.15.117:0/185824028 conn(0x55bea4bcb740 msgr2=0x55bea4f815c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:29.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.728+0000 7fcc4e074740 1 -- 172.21.15.117:0/185824028 shutdown_connections 2024-09-21T06:07:29.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.728+0000 7fcc4e074740 1 -- 172.21.15.117:0/185824028 wait complete. 2024-09-21T06:07:29.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.728+0000 7fcc4e074740 20 client.17685 trim_cache size 0 max 16384 2024-09-21T06:07:29.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.746+0000 7fcc4e074740 1 Processor -- start 2024-09-21T06:07:29.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.746+0000 7fcc4e074740 1 -- start start 2024-09-21T06:07:29.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.746+0000 7fcc4e074740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bea4bae490 0x55bea4f855a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:29.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.746+0000 7fcc4e074740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bea4bb6670 0x55bea4f95ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:29.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.746+0000 7fcc4e074740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea4bd7350 0x55bea4f7fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:29.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.746+0000 7fcc4e074740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcc401af650 con 0x55bea4bd7350 2024-09-21T06:07:29.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.746+0000 7fcc4e074740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcc401b1190 con 0x55bea4bb6670 2024-09-21T06:07:29.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.746+0000 7fcc4e074740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcc401afd20 con 0x55bea4bae490 2024-09-21T06:07:29.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.746+0000 7fcc4609d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea4bd7350 0x55bea4f7fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:29.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.746+0000 7fcc4609d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea4bd7350 0x55bea4f7fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50344/0 (socket says 172.21.15.117:50344) 2024-09-21T06:07:29.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.746+0000 7fcc4609d640 1 -- 172.21.15.117:0/1654619548 learned_addr learned my addr 172.21.15.117:0/1654619548 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:29.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.747+0000 7fcc4509b640 1 --2- 172.21.15.117:0/1654619548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bea4bb6670 0x55bea4f95ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:29.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.747+0000 7fcc4589c640 1 --2- 172.21.15.117:0/1654619548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bea4bae490 0x55bea4f855a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:29.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.747+0000 7fcc4609d640 1 -- 172.21.15.117:0/1654619548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bea4bae490 msgr2=0x55bea4f855a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:29.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.747+0000 7fcc4609d640 1 --2- 172.21.15.117:0/1654619548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bea4bae490 0x55bea4f855a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.747+0000 7fcc4609d640 1 -- 172.21.15.117:0/1654619548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bea4bb6670 msgr2=0x55bea4f95ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:29.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.747+0000 7fcc4609d640 1 --2- 172.21.15.117:0/1654619548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bea4bb6670 0x55bea4f95ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.747+0000 7fcc4609d640 1 -- 172.21.15.117:0/1654619548 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bea4f80720 con 0x55bea4bd7350 2024-09-21T06:07:29.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.747+0000 7fcc4609d640 1 --2- 172.21.15.117:0/1654619548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea4bd7350 0x55bea4f7fe20 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fcc38031e40 tx=0x7fcc38040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f21af95149649089 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:29.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.747+0000 7fcc2b7fe640 1 -- 172.21.15.117:0/1654619548 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc38031e70 con 0x55bea4bd7350 2024-09-21T06:07:29.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.747+0000 7fcc2b7fe640 1 -- 172.21.15.117:0/1654619548 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc38036040 con 0x55bea4bd7350 2024-09-21T06:07:29.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.747+0000 7fcc2b7fe640 1 -- 172.21.15.117:0/1654619548 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3802f660 con 0x55bea4bd7350 2024-09-21T06:07:29.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.748+0000 7fcc4e074740 1 -- 172.21.15.117:0/1654619548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea4bd7350 msgr2=0x55bea4f7fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:29.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.748+0000 7fcc4e074740 1 --2- 172.21.15.117:0/1654619548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea4bd7350 0x55bea4f7fe20 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fcc38031e40 tx=0x7fcc38040000 comp rx=0 tx=0).stop 2024-09-21T06:07:29.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.748+0000 7fcc4e074740 1 -- 172.21.15.117:0/1654619548 shutdown_connections 2024-09-21T06:07:29.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.748+0000 7fcc4e074740 1 --2- 172.21.15.117:0/1654619548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea4bd7350 0x55bea4f7fe20 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.748+0000 7fcc4e074740 1 --2- 172.21.15.117:0/1654619548 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bea4bb6670 0x55bea4f95ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.748+0000 7fcc4e074740 1 --2- 172.21.15.117:0/1654619548 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bea4bae490 0x55bea4f855a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.748+0000 7fcc4e074740 1 -- 172.21.15.117:0/1654619548 >> 172.21.15.117:0/1654619548 conn(0x55bea4bcb740 msgr2=0x55bea4f8b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:29.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.748+0000 7fcc4e074740 1 -- 172.21.15.117:0/1654619548 shutdown_connections 2024-09-21T06:07:29.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.748+0000 7fcc4e074740 1 -- 172.21.15.117:0/1654619548 wait complete. 2024-09-21T06:07:29.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.748+0000 7fcc4e074740 1 -- start start 2024-09-21T06:07:29.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.748+0000 7fcc4e074740 1 Processor -- start 2024-09-21T06:07:29.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.749+0000 7fcc4e074740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bea4bae490 0x55bea5001880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:29.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.749+0000 7fcc4e074740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bea4bb6670 0x55bea5001dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:29.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.749+0000 7fcc4e074740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea4bd7350 0x55bea4f9d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:29.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.749+0000 7fcc4e074740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcc401b0ac0 con 0x55bea4bd7350 2024-09-21T06:07:29.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.749+0000 7fcc4e074740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcc401b03f0 con 0x55bea4bb6670 2024-09-21T06:07:29.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.749+0000 7fcc4e074740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcc401ad650 con 0x55bea4bae490 2024-09-21T06:07:29.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.749+0000 7fcc4e074740 10 client.0 _ll_register_callbacks cb 0x7fcc3c004590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:29.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.749+0000 7fcc4609d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea4bd7350 0x55bea4f9d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:29.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.749+0000 7fcc4589c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bea4bae490 0x55bea5001880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:29.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.749+0000 7fcc4609d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea4bd7350 0x55bea4f9d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50346/0 (socket says 172.21.15.117:50346) 2024-09-21T06:07:29.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.749+0000 7fcc4589c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bea4bae490 0x55bea5001880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:57102/0 (socket says 172.21.15.117:57102) 2024-09-21T06:07:29.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.749+0000 7fcc4609d640 1 -- 172.21.15.117:0/4029307147 learned_addr learned my addr 172.21.15.117:0/4029307147 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:29.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.749+0000 7fcc4509b640 1 --2- 172.21.15.117:0/4029307147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bea4bb6670 0x55bea5001dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:29.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.749+0000 7fcc4609d640 1 -- 172.21.15.117:0/4029307147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bea4bae490 msgr2=0x55bea5001880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:29.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.749+0000 7fcc4609d640 1 --2- 172.21.15.117:0/4029307147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bea4bae490 0x55bea5001880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.749+0000 7fcc4609d640 1 -- 172.21.15.117:0/4029307147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bea4bb6670 msgr2=0x55bea5001dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:29.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.749+0000 7fcc4609d640 1 --2- 172.21.15.117:0/4029307147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bea4bb6670 0x55bea5001dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.749+0000 7fcc4609d640 1 -- 172.21.15.117:0/4029307147 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bea4f9da50 con 0x55bea4bd7350 2024-09-21T06:07:29.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.749+0000 7fcc4509b640 1 --2- 172.21.15.117:0/4029307147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bea4bb6670 0x55bea5001dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:29.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.750+0000 7fcc4609d640 1 --2- 172.21.15.117:0/4029307147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea4bd7350 0x55bea4f9d510 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fcc3802ade0 tx=0x7fcc3802f910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e73bf94287fd652 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:29.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.750+0000 7fcc297fa640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:29.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.750+0000 7fcc297fa640 1 -- 172.21.15.117:0/4029307147 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc38059020 con 0x55bea4bd7350 2024-09-21T06:07:29.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.750+0000 7fcc297fa640 1 -- 172.21.15.117:0/4029307147 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7fcc3802fd40 con 0x55bea4bd7350 2024-09-21T06:07:29.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.750+0000 7fcc4e074740 1 -- 172.21.15.117:0/4029307147 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc38036040 con 0x55bea4bd7350 2024-09-21T06:07:29.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.750+0000 7fcc297fa640 1 -- 172.21.15.117:0/4029307147 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcc38064050 con 0x55bea4bd7350 2024-09-21T06:07:29.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.750+0000 7fcc4e074740 10 client.17691 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:29.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.750+0000 7fcc4e074740 1 -- 172.21.15.117:0/4029307147 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bea500b360 con 0x55bea4bd7350 2024-09-21T06:07:29.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.750+0000 7fcc297fa640 1 -- 172.21.15.117:0/4029307147 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc38036040 con 0x55bea4bd7350 2024-09-21T06:07:29.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.750+0000 7fcc297fa640 1 -- 172.21.15.117:0/4029307147 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcc38044040 con 0x55bea4bd7350 2024-09-21T06:07:29.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.750+0000 7fcc4e074740 10 client.17691 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:29.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.750+0000 7fcc4e074740 10 client.17691 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:07:29.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.750+0000 7fcc4e074740 20 client.17691 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:29.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.750+0000 7fcc4e074740 1 --2- 172.21.15.117:0/4029307147 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55bea4b6b080 0x55bea500db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:29.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.750+0000 7fcc4e074740 4 client.17691 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:07:29.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.750+0000 7fcc4e074740 1 -- 172.21.15.117:0/4029307147 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bea4ef2860 con 0x55bea4b6b080 2024-09-21T06:07:29.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.751+0000 7fcc4589c640 1 --2- 172.21.15.117:0/4029307147 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55bea4b6b080 0x55bea500db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:29.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.751+0000 7fcc297fa640 10 client.17691 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:29.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.751+0000 7fcc4589c640 1 --2- 172.21.15.117:0/4029307147 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55bea4b6b080 0x55bea500db70 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd7a438255e7215d server_cookie=ef23f2350558e9c1 in_seq=0 out_seq=0 2024-09-21T06:07:29.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.753+0000 7fcc297fa640 1 -- 172.21.15.117:0/4029307147 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bea4ef2860 con 0x55bea4b6b080 2024-09-21T06:07:29.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.753+0000 7fcc297fa640 10 client.17691 handle_command_reply: tid=0 2024-09-21T06:07:29.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.753+0000 7fcc4e074740 1 client.17691 shutdown 2024-09-21T06:07:29.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.753+0000 7fcc4e074740 1 -- 172.21.15.117:0/4029307147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea4bd7350 msgr2=0x55bea4f9d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:29.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.753+0000 7fcc4e074740 1 --2- 172.21.15.117:0/4029307147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea4bd7350 0x55bea4f9d510 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fcc3802ade0 tx=0x7fcc3802f910 comp rx=0 tx=0).stop 2024-09-21T06:07:29.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.754+0000 7fcc4e074740 1 -- 172.21.15.117:0/4029307147 shutdown_connections 2024-09-21T06:07:29.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.754+0000 7fcc4e074740 1 --2- 172.21.15.117:0/4029307147 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55bea4b6b080 0x55bea500db70 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.754+0000 7fcc4e074740 1 --2- 172.21.15.117:0/4029307147 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bea4bd7350 0x55bea4f9d510 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.754+0000 7fcc4e074740 1 --2- 172.21.15.117:0/4029307147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bea4bb6670 0x55bea5001dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.754+0000 7fcc297fa640 0 client.17691 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:29.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.754+0000 7fcc4e074740 1 --2- 172.21.15.117:0/4029307147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bea4bae490 0x55bea5001880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.754+0000 7fcc4e074740 1 -- 172.21.15.117:0/4029307147 >> 172.21.15.117:0/4029307147 conn(0x55bea4bcb740 msgr2=0x55bea4f7fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:29.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.754+0000 7fcc4e074740 1 -- 172.21.15.117:0/4029307147 shutdown_connections 2024-09-21T06:07:29.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.754+0000 7fcc4e074740 1 -- 172.21.15.117:0/4029307147 wait complete. 2024-09-21T06:07:29.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.754+0000 7fcc4e074740 20 client.17691 trim_cache size 0 max 16384 2024-09-21T06:07:29.758 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T06:07:29.759 INFO:teuthology.orchestra.run.smithi117.stdout: "return_code": 0, 2024-09-21T06:07:29.759 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_tag": "e99040eb-55d8-492b-ae21-f9ba41c42664", 2024-09-21T06:07:29.759 INFO:teuthology.orchestra.run.smithi117.stdout: "mode": "asynchronous" 2024-09-21T06:07:29.759 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T06:07:29.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.754+0000 7fcc47b00640 1 -- 172.21.15.117:0/4132083140 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fcc2c0b6700 msgr2=0x7fcc2c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:29.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.754+0000 7fcc47b00640 1 --2- 172.21.15.117:0/4132083140 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fcc2c0b6700 0x7fcc2c0b8c10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcc38006010 tx=0x7fcc380093f0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.754+0000 7fcc47b00640 1 -- 172.21.15.117:0/4132083140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc401094a0 msgr2=0x7fcc4019a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:29.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.754+0000 7fcc47b00640 1 --2- 172.21.15.117:0/4132083140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc401094a0 0x7fcc4019a9e0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fcc3000d740 tx=0x7fcc3000dc00 comp rx=0 tx=0).stop 2024-09-21T06:07:29.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.754+0000 7fcc47b00640 1 -- 172.21.15.117:0/4132083140 shutdown_connections 2024-09-21T06:07:29.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.754+0000 7fcc47b00640 1 --2- 172.21.15.117:0/4132083140 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fcc2c0b6700 0x7fcc2c0b8c10 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.754+0000 7fcc47b00640 1 --2- 172.21.15.117:0/4132083140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc401094a0 0x7fcc4019a9e0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.754+0000 7fcc47b00640 1 --2- 172.21.15.117:0/4132083140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc40109df0 0x7fcc4019fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.754+0000 7fcc47b00640 1 --2- 172.21.15.117:0/4132083140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc40103b60 0x7fcc4019a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:29.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.754+0000 7fcc47b00640 1 -- 172.21.15.117:0/4132083140 >> 172.21.15.117:0/4132083140 conn(0x7fcc4007b850 msgr2=0x7fcc40105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:29.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.754+0000 7fcc47b00640 1 -- 172.21.15.117:0/4132083140 shutdown_connections 2024-09-21T06:07:29.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:29.755+0000 7fcc47b00640 1 -- 172.21.15.117:0/4132083140 wait complete. 2024-09-21T06:07:30.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:29 smithi192 ceph-mon[34700]: pgmap v639: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 180 B/s wr, 11 op/s 2024-09-21T06:07:30.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:29 smithi117 ceph-mon[30947]: pgmap v639: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 180 B/s wr, 11 op/s 2024-09-21T06:07:30.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:29 smithi138 ceph-mon[34391]: pgmap v639: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 180 B/s wr, 11 op/s 2024-09-21T06:07:30.451 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:07:30.701 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:07:31.076 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:30 smithi117 ceph-mon[30947]: scrub queued for path: / 2024-09-21T06:07:31.077 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:30 smithi117 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-21T06:07:31.077 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:30 smithi117 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-21T06:07:31.077 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:30 smithi117 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-21T06:07:31.077 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:30 smithi117 ceph-mon[30947]: scrub summary: idle 2024-09-21T06:07:31.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.167+0000 7f8c99fd0640 1 -- 172.21.15.117:0/3690817025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c941020e0 msgr2=0x7f8c941024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:31.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.167+0000 7f8c99fd0640 1 --2- 172.21.15.117:0/3690817025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c941020e0 0x7f8c941024f0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f8c84009900 tx=0x7f8c8402a410 comp rx=0 tx=0).stop 2024-09-21T06:07:31.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.167+0000 7f8c99fd0640 1 -- 172.21.15.117:0/3690817025 shutdown_connections 2024-09-21T06:07:31.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.167+0000 7f8c99fd0640 1 --2- 172.21.15.117:0/3690817025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c94106c80 0x7f8c94110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:31.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.167+0000 7f8c99fd0640 1 --2- 172.21.15.117:0/3690817025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c94106330 0x7f8c94106740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:31.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.167+0000 7f8c99fd0640 1 --2- 172.21.15.117:0/3690817025 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c941020e0 0x7f8c941024f0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:31.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.167+0000 7f8c99fd0640 1 -- 172.21.15.117:0/3690817025 >> 172.21.15.117:0/3690817025 conn(0x7f8c94079ec0 msgr2=0x7f8c9407a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:31.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.167+0000 7f8c99fd0640 1 -- 172.21.15.117:0/3690817025 shutdown_connections 2024-09-21T06:07:31.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.167+0000 7f8c99fd0640 1 -- 172.21.15.117:0/3690817025 wait complete. 2024-09-21T06:07:31.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.168+0000 7f8c99fd0640 1 Processor -- start 2024-09-21T06:07:31.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.168+0000 7f8c99fd0640 1 -- start start 2024-09-21T06:07:31.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.168+0000 7f8c99fd0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c941020e0 0x7f8c94103080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:31.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.168+0000 7f8c99fd0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c94106330 0x7f8c941035c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:31.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.168+0000 7f8c99fd0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c94106c80 0x7f8c94103b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:31.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.168+0000 7f8c99fd0640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8c94113b70 con 0x7f8c94106330 2024-09-21T06:07:31.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.168+0000 7f8c99fd0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8c941139f0 con 0x7f8c94106c80 2024-09-21T06:07:31.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.168+0000 7f8c99fd0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8c94113cf0 con 0x7f8c941020e0 2024-09-21T06:07:31.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.168+0000 7f8c8affd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c94106330 0x7f8c941035c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:31.180 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.168+0000 7f8c8affd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c94106330 0x7f8c941035c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50358/0 (socket says 172.21.15.117:50358) 2024-09-21T06:07:31.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.168+0000 7f8c8affd640 1 -- 172.21.15.117:0/2300433964 learned_addr learned my addr 172.21.15.117:0/2300433964 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:31.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.168+0000 7f8c937fe640 1 --2- 172.21.15.117:0/2300433964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c941020e0 0x7f8c94103080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:31.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.169+0000 7f8c93fff640 1 --2- 172.21.15.117:0/2300433964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c94106c80 0x7f8c94103b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:31.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.169+0000 7f8c8affd640 1 -- 172.21.15.117:0/2300433964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c941020e0 msgr2=0x7f8c94103080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:31.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.169+0000 7f8c8affd640 1 --2- 172.21.15.117:0/2300433964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c941020e0 0x7f8c94103080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:31.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.169+0000 7f8c8affd640 1 -- 172.21.15.117:0/2300433964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c94106c80 msgr2=0x7f8c94103b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:31.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.169+0000 7f8c8affd640 1 --2- 172.21.15.117:0/2300433964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c94106c80 0x7f8c94103b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:31.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.169+0000 7f8c8affd640 1 -- 172.21.15.117:0/2300433964 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c941a8800 con 0x7f8c94106330 2024-09-21T06:07:31.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.169+0000 7f8c8affd640 1 --2- 172.21.15.117:0/2300433964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c94106330 0x7f8c941035c0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c00d750 tx=0x7f8c7c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:31.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.169+0000 7f8c917fa640 1 -- 172.21.15.117:0/2300433964 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c017070 con 0x7f8c94106330 2024-09-21T06:07:31.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.169+0000 7f8c99fd0640 1 -- 172.21.15.117:0/2300433964 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c941a8af0 con 0x7f8c94106330 2024-09-21T06:07:31.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.169+0000 7f8c917fa640 1 -- 172.21.15.117:0/2300433964 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c7c0054d0 con 0x7f8c94106330 2024-09-21T06:07:31.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.169+0000 7f8c917fa640 1 -- 172.21.15.117:0/2300433964 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c004c30 con 0x7f8c94106330 2024-09-21T06:07:31.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.169+0000 7f8c99fd0640 1 -- 172.21.15.117:0/2300433964 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c941a9000 con 0x7f8c94106330 2024-09-21T06:07:31.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.172+0000 7f8ca0544740 1 Processor -- start 2024-09-21T06:07:31.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.172+0000 7f8ca0544740 1 -- start start 2024-09-21T06:07:31.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.172+0000 7f8c917fa640 1 -- 172.21.15.117:0/2300433964 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f8c7c004dd0 con 0x7f8c94106330 2024-09-21T06:07:31.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.172+0000 7f8ca0544740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586372de070 0x558637685270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:31.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.172+0000 7f8ca0544740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586372b51b0 0x5586376857b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:31.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.172+0000 7f8ca0544740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586372bd390 0x558637685cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:31.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.172+0000 7f8ca0544740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5586375fbe50 con 0x5586372de070 2024-09-21T06:07:31.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.172+0000 7f8ca0544740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5586375eb280 con 0x5586372b51b0 2024-09-21T06:07:31.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.172+0000 7f8ca0544740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5586375ef090 con 0x5586372bd390 2024-09-21T06:07:31.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.172+0000 7f8c93fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586372de070 0x558637685270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:31.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.172+0000 7f8c93fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586372de070 0x558637685270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50360/0 (socket says 172.21.15.117:50360) 2024-09-21T06:07:31.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.172+0000 7f8c93fff640 1 -- 172.21.15.117:0/2989844724 learned_addr learned my addr 172.21.15.117:0/2989844724 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:31.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.172+0000 7f8c937fe640 1 --2- 172.21.15.117:0/2989844724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586372b51b0 0x5586376857b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:31.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.173+0000 7f8c8affd640 1 --2- 172.21.15.117:0/2989844724 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586372bd390 0x558637685cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:31.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.173+0000 7f8c93fff640 1 -- 172.21.15.117:0/2989844724 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586372bd390 msgr2=0x558637685cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:31.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.173+0000 7f8c93fff640 1 --2- 172.21.15.117:0/2989844724 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586372bd390 0x558637685cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:31.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.173+0000 7f8c93fff640 1 -- 172.21.15.117:0/2989844724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586372b51b0 msgr2=0x5586376857b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:31.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.173+0000 7f8c93fff640 1 --2- 172.21.15.117:0/2989844724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586372b51b0 0x5586376857b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:31.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.173+0000 7f8c93fff640 1 -- 172.21.15.117:0/2989844724 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558637349650 con 0x5586372de070 2024-09-21T06:07:31.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.173+0000 7f8c8affd640 1 --2- 172.21.15.117:0/2989844724 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586372bd390 0x558637685cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:07:31.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.173+0000 7f8c93fff640 1 --2- 172.21.15.117:0/2989844724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586372de070 0x558637685270 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c009fc0 tx=0x7f8c8c010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd73313ab5b71c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:31.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.173+0000 7f8c917fa640 1 --2- 172.21.15.117:0/2300433964 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8c800b6680 0x7f8c800b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:31.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.173+0000 7f8c89ffb640 1 -- 172.21.15.117:0/2989844724 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c003b70 con 0x5586372de070 2024-09-21T06:07:31.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.173+0000 7f8c89ffb640 1 -- 172.21.15.117:0/2989844724 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c8c003d10 con 0x5586372de070 2024-09-21T06:07:31.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.173+0000 7f8c937fe640 1 --2- 172.21.15.117:0/2300433964 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8c800b6680 0x7f8c800b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:31.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.173+0000 7f8c89ffb640 1 -- 172.21.15.117:0/2989844724 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c011a40 con 0x5586372de070 2024-09-21T06:07:31.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.173+0000 7f8c917fa640 1 -- 172.21.15.117:0/2300433964 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f8c7c0e70e0 con 0x7f8c94106330 2024-09-21T06:07:31.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.174+0000 7f8c937fe640 1 --2- 172.21.15.117:0/2300433964 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8c800b6680 0x7f8c800b8b90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8c84004770 tx=0x7f8c840058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:31.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.174+0000 7f8ca0544740 1 -- 172.21.15.117:0/2989844724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586372de070 msgr2=0x558637685270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:31.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.174+0000 7f8ca0544740 1 --2- 172.21.15.117:0/2989844724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586372de070 0x558637685270 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c009fc0 tx=0x7f8c8c010860 comp rx=0 tx=0).stop 2024-09-21T06:07:31.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.174+0000 7f8ca0544740 1 -- 172.21.15.117:0/2989844724 shutdown_connections 2024-09-21T06:07:31.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.174+0000 7f8ca0544740 1 --2- 172.21.15.117:0/2989844724 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586372bd390 0x558637685cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:31.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.174+0000 7f8ca0544740 1 --2- 172.21.15.117:0/2989844724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586372b51b0 0x5586376857b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:31.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.174+0000 7f8ca0544740 1 --2- 172.21.15.117:0/2989844724 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586372de070 0x558637685270 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:31.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.174+0000 7f8ca0544740 1 -- 172.21.15.117:0/2989844724 >> 172.21.15.117:0/2989844724 conn(0x5586372d2460 msgr2=0x558637683160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:31.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.174+0000 7f8ca0544740 1 -- 172.21.15.117:0/2989844724 shutdown_connections 2024-09-21T06:07:31.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.174+0000 7f8ca0544740 1 -- 172.21.15.117:0/2989844724 wait complete. 2024-09-21T06:07:31.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.175+0000 7f8ca0544740 1 -- start start 2024-09-21T06:07:31.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.175+0000 7f8ca0544740 1 Processor -- start 2024-09-21T06:07:31.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.175+0000 7f8ca0544740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586372b51b0 0x558637690990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:31.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.175+0000 7f8ca0544740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586372bd390 0x558637690ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:31.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.175+0000 7f8ca0544740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586372de070 0x55863769cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:31.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.175+0000 7f8ca0544740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55863744c700 con 0x5586372de070 2024-09-21T06:07:31.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.175+0000 7f8ca0544740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558637586110 con 0x5586372b51b0 2024-09-21T06:07:31.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.175+0000 7f8ca0544740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558637693a50 con 0x5586372bd390 2024-09-21T06:07:31.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.176+0000 7f8ca0544740 10 client.0 _ll_register_callbacks cb 0x55863759f990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:31.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.176+0000 7f8c93fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586372de070 0x55863769cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:31.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.176+0000 7f8c93fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586372de070 0x55863769cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50366/0 (socket says 172.21.15.117:50366) 2024-09-21T06:07:31.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.176+0000 7f8c93fff640 1 -- 172.21.15.117:0/1411026390 learned_addr learned my addr 172.21.15.117:0/1411026390 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:31.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.176+0000 7f8c937fe640 1 --2- 172.21.15.117:0/1411026390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586372b51b0 0x558637690990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:31.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.176+0000 7f8c8affd640 1 --2- 172.21.15.117:0/1411026390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586372bd390 0x558637690ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:31.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.176+0000 7f8c93fff640 1 -- 172.21.15.117:0/1411026390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586372bd390 msgr2=0x558637690ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:31.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.176+0000 7f8c93fff640 1 --2- 172.21.15.117:0/1411026390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586372bd390 0x558637690ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:31.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.176+0000 7f8c93fff640 1 -- 172.21.15.117:0/1411026390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586372b51b0 msgr2=0x558637690990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:31.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.176+0000 7f8c93fff640 1 --2- 172.21.15.117:0/1411026390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586372b51b0 0x558637690990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:31.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.176+0000 7f8c93fff640 1 -- 172.21.15.117:0/1411026390 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586375b3c40 con 0x5586372de070 2024-09-21T06:07:31.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.176+0000 7f8c937fe640 1 --2- 172.21.15.117:0/1411026390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586372b51b0 0x558637690990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:31.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.176+0000 7f8c8affd640 1 --2- 172.21.15.117:0/1411026390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586372bd390 0x558637690ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:31.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.176+0000 7f8c93fff640 1 --2- 172.21.15.117:0/1411026390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586372de070 0x55863769cd70 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c011010 tx=0x7f8c8c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a05c0e89760f0047 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:31.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.176+0000 7f8c72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:31.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.176+0000 7f8c72ffd640 1 -- 172.21.15.117:0/1411026390 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c004d50 con 0x5586372de070 2024-09-21T06:07:31.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.176+0000 7f8c72ffd640 1 -- 172.21.15.117:0/1411026390 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f8c8c003ec0 con 0x5586372de070 2024-09-21T06:07:31.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.176+0000 7f8ca0544740 1 -- 172.21.15.117:0/1411026390 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c8c003d10 con 0x5586372de070 2024-09-21T06:07:31.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.177+0000 7f8c72ffd640 1 -- 172.21.15.117:0/1411026390 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8c8c0040c0 con 0x5586372de070 2024-09-21T06:07:31.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.177+0000 7f8ca0544740 10 client.17703 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:31.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.177+0000 7f8ca0544740 1 -- 172.21.15.117:0/1411026390 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55863769f700 con 0x5586372de070 2024-09-21T06:07:31.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.177+0000 7f8c72ffd640 1 -- 172.21.15.117:0/1411026390 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c8c003d10 con 0x5586372de070 2024-09-21T06:07:31.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.177+0000 7f8c72ffd640 1 -- 172.21.15.117:0/1411026390 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8c8c005040 con 0x5586372de070 2024-09-21T06:07:31.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.177+0000 7f8ca0544740 10 client.17703 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:31.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.177+0000 7f8ca0544740 10 client.17703 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:07:31.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.177+0000 7f8ca0544740 20 client.17703 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:31.186 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.177+0000 7f8ca0544740 1 --2- 172.21.15.117:0/1411026390 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55863729c9f0 0x5586376a2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:31.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.177+0000 7f8ca0544740 4 client.17703 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:07:31.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.177+0000 7f8ca0544740 1 -- 172.21.15.117:0/1411026390 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586375fdda0 con 0x55863729c9f0 2024-09-21T06:07:31.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.177+0000 7f8c937fe640 1 --2- 172.21.15.117:0/1411026390 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55863729c9f0 0x5586376a2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:31.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.178+0000 7f8c72ffd640 10 client.17703 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:31.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.178+0000 7f8c937fe640 1 --2- 172.21.15.117:0/1411026390 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55863729c9f0 0x5586376a2b10 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=382901d1b7c9b99c server_cookie=28bb3f9497aa0b81 in_seq=0 out_seq=0 2024-09-21T06:07:31.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.179+0000 7f8c72ffd640 1 -- 172.21.15.117:0/1411026390 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5586375fdda0 con 0x55863729c9f0 2024-09-21T06:07:31.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.179+0000 7f8c72ffd640 10 client.17703 handle_command_reply: tid=0 2024-09-21T06:07:31.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.179+0000 7f8ca0544740 1 client.17703 shutdown 2024-09-21T06:07:31.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.179+0000 7f8ca0544740 1 -- 172.21.15.117:0/1411026390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586372de070 msgr2=0x55863769cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:31.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.179+0000 7f8ca0544740 1 --2- 172.21.15.117:0/1411026390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586372de070 0x55863769cd70 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c011010 tx=0x7f8c8c004750 comp rx=0 tx=0).stop 2024-09-21T06:07:31.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.179+0000 7f8ca0544740 1 -- 172.21.15.117:0/1411026390 shutdown_connections 2024-09-21T06:07:31.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.179+0000 7f8ca0544740 1 --2- 172.21.15.117:0/1411026390 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55863729c9f0 0x5586376a2b10 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:31.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.179+0000 7f8ca0544740 1 --2- 172.21.15.117:0/1411026390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586372de070 0x55863769cd70 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:31.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.179+0000 7f8ca0544740 1 --2- 172.21.15.117:0/1411026390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586372bd390 0x558637690ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:31.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.179+0000 7f8c72ffd640 0 client.17703 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:31.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.179+0000 7f8ca0544740 1 --2- 172.21.15.117:0/1411026390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586372b51b0 0x558637690990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:31.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.179+0000 7f8ca0544740 1 -- 172.21.15.117:0/1411026390 >> 172.21.15.117:0/1411026390 conn(0x5586372d2460 msgr2=0x558637688450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:31.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.179+0000 7f8ca0544740 1 -- 172.21.15.117:0/1411026390 shutdown_connections 2024-09-21T06:07:31.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.179+0000 7f8ca0544740 1 -- 172.21.15.117:0/1411026390 wait complete. 2024-09-21T06:07:31.189 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.179+0000 7f8ca0544740 20 client.17703 trim_cache size 0 max 16384 2024-09-21T06:07:31.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.196+0000 7f8ca0544740 1 Processor -- start 2024-09-21T06:07:31.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.196+0000 7f8ca0544740 1 -- start start 2024-09-21T06:07:31.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.197+0000 7f8ca0544740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586372b51b0 0x55863769cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:31.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.197+0000 7f8ca0544740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586372bd390 0x55863768db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:31.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.197+0000 7f8ca0544740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586372de070 0x55863768e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:31.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.197+0000 7f8ca0544740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8c941b1800 con 0x5586372bd390 2024-09-21T06:07:31.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.197+0000 7f8ca0544740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8c941b3340 con 0x5586372b51b0 2024-09-21T06:07:31.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.197+0000 7f8ca0544740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8c941b1ed0 con 0x5586372de070 2024-09-21T06:07:31.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.197+0000 7f8c8affd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586372bd390 0x55863768db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:31.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.197+0000 7f8c8affd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586372bd390 0x55863768db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50382/0 (socket says 172.21.15.117:50382) 2024-09-21T06:07:31.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.197+0000 7f8c8affd640 1 -- 172.21.15.117:0/3715116799 learned_addr learned my addr 172.21.15.117:0/3715116799 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:31.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.197+0000 7f8c93fff640 1 --2- 172.21.15.117:0/3715116799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586372de070 0x55863768e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:31.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.197+0000 7f8c937fe640 1 --2- 172.21.15.117:0/3715116799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586372b51b0 0x55863769cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:31.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.197+0000 7f8c8affd640 1 -- 172.21.15.117:0/3715116799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586372de070 msgr2=0x55863768e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:31.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.197+0000 7f8c8affd640 1 --2- 172.21.15.117:0/3715116799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586372de070 0x55863768e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:31.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.197+0000 7f8c8affd640 1 -- 172.21.15.117:0/3715116799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586372b51b0 msgr2=0x55863769cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:31.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.197+0000 7f8c8affd640 1 --2- 172.21.15.117:0/3715116799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586372b51b0 0x55863769cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:31.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.197+0000 7f8c8affd640 1 -- 172.21.15.117:0/3715116799 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558637691c50 con 0x5586372bd390 2024-09-21T06:07:31.200 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.197+0000 7f8c8affd640 1 --2- 172.21.15.117:0/3715116799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586372bd390 0x55863768db80 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c0acc80 tx=0x7f8c7c0cef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e52e5b42faa25a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:31.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.197+0000 7f8c897fa640 1 -- 172.21.15.117:0/3715116799 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c020070 con 0x5586372bd390 2024-09-21T06:07:31.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.197+0000 7f8c897fa640 1 -- 172.21.15.117:0/3715116799 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c7c0ad680 con 0x5586372bd390 2024-09-21T06:07:31.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.197+0000 7f8c897fa640 1 -- 172.21.15.117:0/3715116799 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c0e90d0 con 0x5586372bd390 2024-09-21T06:07:31.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.198+0000 7f8ca0544740 1 -- 172.21.15.117:0/3715116799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586372bd390 msgr2=0x55863768db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:31.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.198+0000 7f8ca0544740 1 --2- 172.21.15.117:0/3715116799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586372bd390 0x55863768db80 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c0acc80 tx=0x7f8c7c0cef90 comp rx=0 tx=0).stop 2024-09-21T06:07:31.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.198+0000 7f8ca0544740 1 -- 172.21.15.117:0/3715116799 shutdown_connections 2024-09-21T06:07:31.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.198+0000 7f8ca0544740 1 --2- 172.21.15.117:0/3715116799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586372bd390 0x55863768db80 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:31.208 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.198+0000 7f8ca0544740 1 --2- 172.21.15.117:0/3715116799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586372de070 0x55863768e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:31.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.198+0000 7f8ca0544740 1 --2- 172.21.15.117:0/3715116799 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586372b51b0 0x55863769cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:31.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.198+0000 7f8ca0544740 1 -- 172.21.15.117:0/3715116799 >> 172.21.15.117:0/3715116799 conn(0x5586372d2460 msgr2=0x55863768f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:31.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.198+0000 7f8ca0544740 1 -- 172.21.15.117:0/3715116799 shutdown_connections 2024-09-21T06:07:31.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.198+0000 7f8ca0544740 1 -- 172.21.15.117:0/3715116799 wait complete. 2024-09-21T06:07:31.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.199+0000 7f8ca0544740 1 -- start start 2024-09-21T06:07:31.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.199+0000 7f8ca0544740 1 Processor -- start 2024-09-21T06:07:31.209 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.199+0000 7f8ca0544740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586372b51b0 0x5586376a3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:31.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.199+0000 7f8ca0544740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586372bd390 0x5586376a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:31.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.199+0000 7f8ca0544740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586372de070 0x55863770c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:31.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.199+0000 7f8ca0544740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8c941b2c70 con 0x5586372b51b0 2024-09-21T06:07:31.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.199+0000 7f8ca0544740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8c941b25a0 con 0x5586372de070 2024-09-21T06:07:31.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.199+0000 7f8ca0544740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8c941af800 con 0x5586372bd390 2024-09-21T06:07:31.210 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.199+0000 7f8ca0544740 10 client.0 _ll_register_callbacks cb 0x7f8c84009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:31.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.199+0000 7f8c937fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586372b51b0 0x5586376a3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:31.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.199+0000 7f8c937fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586372b51b0 0x5586376a3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50398/0 (socket says 172.21.15.117:50398) 2024-09-21T06:07:31.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.199+0000 7f8c937fe640 1 -- 172.21.15.117:0/937182908 learned_addr learned my addr 172.21.15.117:0/937182908 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:31.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.199+0000 7f8c8affd640 1 --2- 172.21.15.117:0/937182908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586372bd390 0x5586376a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:31.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.200+0000 7f8c93fff640 1 --2- 172.21.15.117:0/937182908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586372de070 0x55863770c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:31.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.200+0000 7f8c937fe640 1 -- 172.21.15.117:0/937182908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586372bd390 msgr2=0x5586376a42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:31.211 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.200+0000 7f8c937fe640 1 --2- 172.21.15.117:0/937182908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586372bd390 0x5586376a42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:31.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.200+0000 7f8c937fe640 1 -- 172.21.15.117:0/937182908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586372de070 msgr2=0x55863770c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:31.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.200+0000 7f8c937fe640 1 --2- 172.21.15.117:0/937182908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586372de070 0x55863770c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:31.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.200+0000 7f8c937fe640 1 -- 172.21.15.117:0/937182908 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586376a4ac0 con 0x5586372b51b0 2024-09-21T06:07:31.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.200+0000 7f8c93fff640 1 --2- 172.21.15.117:0/937182908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586372de070 0x55863770c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:07:31.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.200+0000 7f8c8affd640 1 --2- 172.21.15.117:0/937182908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586372bd390 0x5586376a42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:31.212 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.200+0000 7f8c937fe640 1 --2- 172.21.15.117:0/937182908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586372b51b0 0x5586376a3d90 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f8c84009230 tx=0x7f8c84009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d51524636b8c30 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:31.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.200+0000 7f8c72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:31.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.200+0000 7f8c72ffd640 1 -- 172.21.15.117:0/937182908 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c84035040 con 0x5586372b51b0 2024-09-21T06:07:31.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.200+0000 7f8c72ffd640 1 -- 172.21.15.117:0/937182908 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f8c840035f0 con 0x5586372b51b0 2024-09-21T06:07:31.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.200+0000 7f8ca0544740 1 -- 172.21.15.117:0/937182908 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c7c0ad680 con 0x5586372b51b0 2024-09-21T06:07:31.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.200+0000 7f8c72ffd640 1 -- 172.21.15.117:0/937182908 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8c8405d020 con 0x5586372b51b0 2024-09-21T06:07:31.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.200+0000 7f8ca0544740 10 client.17709 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:31.213 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.200+0000 7f8ca0544740 1 -- 172.21.15.117:0/937182908 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55863770d7f0 con 0x5586372b51b0 2024-09-21T06:07:31.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.200+0000 7f8c72ffd640 1 -- 172.21.15.117:0/937182908 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c7c0ad680 con 0x5586372b51b0 2024-09-21T06:07:31.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.201+0000 7f8c72ffd640 1 -- 172.21.15.117:0/937182908 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8c8403d830 con 0x5586372b51b0 2024-09-21T06:07:31.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.201+0000 7f8ca0544740 10 client.17709 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:31.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.201+0000 7f8ca0544740 10 client.17709 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:07:31.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.201+0000 7f8ca0544740 20 client.17709 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:31.214 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.201+0000 7f8ca0544740 1 --2- 172.21.15.117:0/937182908 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x558637271da0 0x55863770ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:31.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.201+0000 7f8ca0544740 4 client.17709 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:07:31.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.201+0000 7f8ca0544740 1 -- 172.21.15.117:0/937182908 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5586375f9850 con 0x558637271da0 2024-09-21T06:07:31.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.201+0000 7f8c8affd640 1 --2- 172.21.15.117:0/937182908 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x558637271da0 0x55863770ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:31.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.201+0000 7f8c72ffd640 10 client.17709 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:31.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.201+0000 7f8c8affd640 1 --2- 172.21.15.117:0/937182908 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x558637271da0 0x55863770ffe0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=689ddac50495bf80 server_cookie=3816ba1dd7cfe381 in_seq=0 out_seq=0 2024-09-21T06:07:31.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.202+0000 7f8c72ffd640 1 -- 172.21.15.117:0/937182908 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3214 (crc 0 0 0) 0x5586375f9850 con 0x558637271da0 2024-09-21T06:07:31.215 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.202+0000 7f8c72ffd640 10 client.17709 handle_command_reply: tid=0 2024-09-21T06:07:31.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.202+0000 7f8ca0544740 1 client.17709 shutdown 2024-09-21T06:07:31.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.202+0000 7f8ca0544740 1 -- 172.21.15.117:0/937182908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586372b51b0 msgr2=0x5586376a3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:31.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.202+0000 7f8ca0544740 1 --2- 172.21.15.117:0/937182908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586372b51b0 0x5586376a3d90 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f8c84009230 tx=0x7f8c84009260 comp rx=0 tx=0).stop 2024-09-21T06:07:31.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.202+0000 7f8ca0544740 1 -- 172.21.15.117:0/937182908 shutdown_connections 2024-09-21T06:07:31.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.202+0000 7f8ca0544740 1 --2- 172.21.15.117:0/937182908 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x558637271da0 0x55863770ffe0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:31.216 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.202+0000 7f8ca0544740 1 --2- 172.21.15.117:0/937182908 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5586372de070 0x55863770c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:31.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.202+0000 7f8ca0544740 1 --2- 172.21.15.117:0/937182908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5586372bd390 0x5586376a42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:31.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.202+0000 7f8c72ffd640 0 client.17709 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:31.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.202+0000 7f8ca0544740 1 --2- 172.21.15.117:0/937182908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5586372b51b0 0x5586376a3d90 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:31.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.202+0000 7f8ca0544740 1 -- 172.21.15.117:0/937182908 >> 172.21.15.117:0/937182908 conn(0x5586372d2460 msgr2=0x558637691e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:31.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.202+0000 7f8ca0544740 1 -- 172.21.15.117:0/937182908 shutdown_connections 2024-09-21T06:07:31.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.202+0000 7f8ca0544740 1 -- 172.21.15.117:0/937182908 wait complete. 2024-09-21T06:07:31.217 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.202+0000 7f8ca0544740 20 client.17709 trim_cache size 0 max 16384 2024-09-21T06:07:31.218 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T06:07:31.218 INFO:teuthology.orchestra.run.smithi117.stdout: "status": "no active scrubs running", 2024-09-21T06:07:31.218 INFO:teuthology.orchestra.run.smithi117.stdout: "scrubs": {}, 2024-09-21T06:07:31.218 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_stats": { 2024-09-21T06:07:31.219 INFO:teuthology.orchestra.run.smithi117.stdout: "e99040eb-55d8-492b-ae21-f9ba41c42664": { 2024-09-21T06:07:31.219 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:07:29.750906+0000", 2024-09-21T06:07:31.219 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:31.219 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:31.219 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:31.219 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:31.220 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:31.220 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:31.220 INFO:teuthology.orchestra.run.smithi117.stdout: "26e3d90e-072a-4a4e-9661-4d004051865b": { 2024-09-21T06:07:31.220 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:06:29.219142+0000", 2024-09-21T06:07:31.220 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:31.220 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:31.221 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:31.221 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:31.221 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:31.221 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:31.221 INFO:teuthology.orchestra.run.smithi117.stdout: "cf43a1b2-838e-4d28-9ed6-ff0c726fd51a": { 2024-09-21T06:07:31.221 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:06:42.901863+0000", 2024-09-21T06:07:31.221 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:31.222 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:31.222 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:31.222 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:31.222 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:31.222 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:31.222 INFO:teuthology.orchestra.run.smithi117.stdout: "39eeca8a-6b4f-465f-af34-b54f77320d70": { 2024-09-21T06:07:31.223 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:05:41.497141+0000", 2024-09-21T06:07:31.223 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:31.223 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:31.223 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:31.223 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:31.223 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:31.224 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:31.224 INFO:teuthology.orchestra.run.smithi117.stdout: "29ff9b7c-49d9-4910-85be-bace7e940cd5": { 2024-09-21T06:07:31.224 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:03:03.775527+0000", 2024-09-21T06:07:31.224 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:31.224 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:31.224 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:31.224 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:31.225 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:31.225 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:31.225 INFO:teuthology.orchestra.run.smithi117.stdout: "0902e33a-ce33-45fb-92d8-2e6fcb8ec8a0": { 2024-09-21T06:07:31.225 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:04:11.365073+0000", 2024-09-21T06:07:31.225 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:31.225 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:31.225 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:31.226 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:31.226 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:31.226 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:31.226 INFO:teuthology.orchestra.run.smithi117.stdout: "d543b0ae-b710-4153-99da-fe5714839c1a": { 2024-09-21T06:07:31.226 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:01:20.574754+0000", 2024-09-21T06:07:31.226 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:31.226 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:31.227 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:31.227 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:31.227 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:31.227 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:31.227 INFO:teuthology.orchestra.run.smithi117.stdout: "ad80d9f1-6aff-46c0-91ab-2c1ab0681039": { 2024-09-21T06:07:31.227 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:59:31.949108+0000", 2024-09-21T06:07:31.227 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:31.227 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:31.228 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:31.228 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:31.228 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:31.228 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:31.228 INFO:teuthology.orchestra.run.smithi117.stdout: "fb2f217d-463b-4c8f-ab19-920b04bb7306": { 2024-09-21T06:07:31.228 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:04:56.308107+0000", 2024-09-21T06:07:31.228 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:31.228 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:31.228 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:31.228 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:31.228 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:31.228 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:31.228 INFO:teuthology.orchestra.run.smithi117.stdout: "8b3fe95a-b76b-4e94-afd0-72fe35e2dd92": { 2024-09-21T06:07:31.228 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:58:01.246076+0000", 2024-09-21T06:07:31.229 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:31.229 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:31.229 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:31.229 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:31.229 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:31.229 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:31.229 INFO:teuthology.orchestra.run.smithi117.stdout: "b30bfff0-0f6b-4e30-8467-ff7db421eba6": { 2024-09-21T06:07:31.229 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:57:14.939082+0000", 2024-09-21T06:07:31.229 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:31.229 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:31.229 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:31.229 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:31.229 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:31.229 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:07:31.229 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:07:31.229 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T06:07:31.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.203+0000 7f8c99fd0640 1 -- 172.21.15.117:0/2300433964 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8c800b6680 msgr2=0x7f8c800b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:31.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.203+0000 7f8c99fd0640 1 --2- 172.21.15.117:0/2300433964 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8c800b6680 0x7f8c800b8b90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8c84004770 tx=0x7f8c840058d0 comp rx=0 tx=0).stop 2024-09-21T06:07:31.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.203+0000 7f8c99fd0640 1 -- 172.21.15.117:0/2300433964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c94106330 msgr2=0x7f8c941035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:31.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.203+0000 7f8c99fd0640 1 --2- 172.21.15.117:0/2300433964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c94106330 0x7f8c941035c0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c00d750 tx=0x7f8c7c00dc10 comp rx=0 tx=0).stop 2024-09-21T06:07:31.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.203+0000 7f8c99fd0640 1 -- 172.21.15.117:0/2300433964 shutdown_connections 2024-09-21T06:07:31.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.203+0000 7f8c99fd0640 1 --2- 172.21.15.117:0/2300433964 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8c800b6680 0x7f8c800b8b90 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:31.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.203+0000 7f8c99fd0640 1 --2- 172.21.15.117:0/2300433964 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c94106330 0x7f8c941035c0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:31.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.203+0000 7f8c99fd0640 1 --2- 172.21.15.117:0/2300433964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c94106c80 0x7f8c94103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:31.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.203+0000 7f8c99fd0640 1 --2- 172.21.15.117:0/2300433964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c941020e0 0x7f8c94103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:31.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.203+0000 7f8c99fd0640 1 -- 172.21.15.117:0/2300433964 >> 172.21.15.117:0/2300433964 conn(0x7f8c94079ec0 msgr2=0x7f8c94107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:31.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.203+0000 7f8c99fd0640 1 -- 172.21.15.117:0/2300433964 shutdown_connections 2024-09-21T06:07:31.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:31.203+0000 7f8c99fd0640 1 -- 172.21.15.117:0/2300433964 wait complete. 2024-09-21T06:07:31.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:30 smithi192 ceph-mon[34700]: scrub queued for path: / 2024-09-21T06:07:31.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:30 smithi192 ceph-mon[34700]: scrub summary: idle+waiting paths [/] 2024-09-21T06:07:31.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:30 smithi192 ceph-mon[34700]: scrub summary: active paths [/] 2024-09-21T06:07:31.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:30 smithi192 ceph-mon[34700]: scrub summary: idle+waiting paths [/] 2024-09-21T06:07:31.231 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:30 smithi192 ceph-mon[34700]: scrub summary: idle 2024-09-21T06:07:31.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:30 smithi138 ceph-mon[34391]: scrub queued for path: / 2024-09-21T06:07:31.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:30 smithi138 ceph-mon[34391]: scrub summary: idle+waiting paths [/] 2024-09-21T06:07:31.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:30 smithi138 ceph-mon[34391]: scrub summary: active paths [/] 2024-09-21T06:07:31.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:30 smithi138 ceph-mon[34391]: scrub summary: idle+waiting paths [/] 2024-09-21T06:07:31.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:30 smithi138 ceph-mon[34391]: scrub summary: idle 2024-09-21T06:07:32.004 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:07:32.004 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:07:32.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:31 smithi192 ceph-mon[34700]: pgmap v640: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-21T06:07:32.249 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:07:32.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:31 smithi117 ceph-mon[30947]: pgmap v640: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-21T06:07:32.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:31 smithi138 ceph-mon[34391]: pgmap v640: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-21T06:07:32.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.750+0000 7fa885590640 1 -- 172.21.15.117:0/3468343327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa880102a60 msgr2=0x7fa880102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:32.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.750+0000 7fa885590640 1 --2- 172.21.15.117:0/3468343327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa880102a60 0x7fa880102e70 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fa870009900 tx=0x7fa87002a410 comp rx=0 tx=0).stop 2024-09-21T06:07:32.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.750+0000 7fa885590640 1 -- 172.21.15.117:0/3468343327 shutdown_connections 2024-09-21T06:07:32.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.750+0000 7fa885590640 1 --2- 172.21.15.117:0/3468343327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa880108cf0 0x7fa88010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:32.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.750+0000 7fa885590640 1 --2- 172.21.15.117:0/3468343327 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8801083a0 0x7fa8801087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:32.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.750+0000 7fa885590640 1 --2- 172.21.15.117:0/3468343327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa880102a60 0x7fa880102e70 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:32.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.750+0000 7fa885590640 1 -- 172.21.15.117:0/3468343327 >> 172.21.15.117:0/3468343327 conn(0x7fa880079730 msgr2=0x7fa880079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:32.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.750+0000 7fa885590640 1 -- 172.21.15.117:0/3468343327 shutdown_connections 2024-09-21T06:07:32.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.750+0000 7fa885590640 1 -- 172.21.15.117:0/3468343327 wait complete. 2024-09-21T06:07:32.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.751+0000 7fa885590640 1 Processor -- start 2024-09-21T06:07:32.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.751+0000 7fa885590640 1 -- start start 2024-09-21T06:07:32.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.751+0000 7fa885590640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa880102a60 0x7fa8801993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:32.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.751+0000 7fa885590640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8801083a0 0x7fa880199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:32.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.751+0000 7fa885590640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa880108cf0 0x7fa88019ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:32.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.751+0000 7fa885590640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa8801129c0 con 0x7fa880108cf0 2024-09-21T06:07:32.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.751+0000 7fa885590640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa880112840 con 0x7fa8801083a0 2024-09-21T06:07:32.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.751+0000 7fa885590640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa880112b40 con 0x7fa880102a60 2024-09-21T06:07:32.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.752+0000 7fa87f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa880108cf0 0x7fa88019ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:32.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.752+0000 7fa87f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa880108cf0 0x7fa88019ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50418/0 (socket says 172.21.15.117:50418) 2024-09-21T06:07:32.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.752+0000 7fa87f7fe640 1 -- 172.21.15.117:0/2128464109 learned_addr learned my addr 172.21.15.117:0/2128464109 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:32.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.752+0000 7fa87effd640 1 --2- 172.21.15.117:0/2128464109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa880102a60 0x7fa8801993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:32.766 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.752+0000 7fa87e7fc640 1 --2- 172.21.15.117:0/2128464109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8801083a0 0x7fa880199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:32.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.752+0000 7fa87f7fe640 1 -- 172.21.15.117:0/2128464109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa880102a60 msgr2=0x7fa8801993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:32.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.752+0000 7fa87f7fe640 1 --2- 172.21.15.117:0/2128464109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa880102a60 0x7fa8801993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:32.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.752+0000 7fa87f7fe640 1 -- 172.21.15.117:0/2128464109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8801083a0 msgr2=0x7fa880199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:32.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.752+0000 7fa87f7fe640 1 --2- 172.21.15.117:0/2128464109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8801083a0 0x7fa880199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:32.767 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.752+0000 7fa87f7fe640 1 -- 172.21.15.117:0/2128464109 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa88019f340 con 0x7fa880108cf0 2024-09-21T06:07:32.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.752+0000 7fa87effd640 1 --2- 172.21.15.117:0/2128464109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa880102a60 0x7fa8801993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:32.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.752+0000 7fa87e7fc640 1 --2- 172.21.15.117:0/2128464109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8801083a0 0x7fa880199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:32.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.752+0000 7fa87f7fe640 1 --2- 172.21.15.117:0/2128464109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa880108cf0 0x7fa88019ec10 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fa87800eaa0 tx=0x7fa87800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:32.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.752+0000 7fa867fff640 1 -- 172.21.15.117:0/2128464109 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87800cdb0 con 0x7fa880108cf0 2024-09-21T06:07:32.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.752+0000 7fa885590640 1 -- 172.21.15.117:0/2128464109 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa88019f630 con 0x7fa880108cf0 2024-09-21T06:07:32.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.752+0000 7fa867fff640 1 -- 172.21.15.117:0/2128464109 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa878005020 con 0x7fa880108cf0 2024-09-21T06:07:32.768 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.752+0000 7fa867fff640 1 -- 172.21.15.117:0/2128464109 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8780116d0 con 0x7fa880108cf0 2024-09-21T06:07:32.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.752+0000 7fa885590640 1 -- 172.21.15.117:0/2128464109 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8801a5d90 con 0x7fa880108cf0 2024-09-21T06:07:32.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.754+0000 7fa885590640 1 -- 172.21.15.117:0/2128464109 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa880102e70 con 0x7fa880108cf0 2024-09-21T06:07:32.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.755+0000 7fa867fff640 1 -- 172.21.15.117:0/2128464109 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fa8780051d0 con 0x7fa880108cf0 2024-09-21T06:07:32.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.756+0000 7fa867fff640 1 --2- 172.21.15.117:0/2128464109 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa8680b6680 0x7fa8680b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:32.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.756+0000 7fa87effd640 1 --2- 172.21.15.117:0/2128464109 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa8680b6680 0x7fa8680b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:32.769 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.756+0000 7fa867fff640 1 -- 172.21.15.117:0/2128464109 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7fa8780e7980 con 0x7fa880108cf0 2024-09-21T06:07:32.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.756+0000 7fa87effd640 1 --2- 172.21.15.117:0/2128464109 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa8680b6680 0x7fa8680b8b90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa870031440 tx=0x7fa870009680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:32.770 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.761+0000 7fa867fff640 1 -- 172.21.15.117:0/2128464109 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa878016030 con 0x7fa880108cf0 2024-09-21T06:07:32.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.983+0000 7fa885590640 1 -- 172.21.15.117:0/2128464109 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa88019a500 con 0x7fa880108cf0 2024-09-21T06:07:32.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.983+0000 7fa867fff640 1 -- 172.21.15.117:0/2128464109 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+12001 (secure 0 0 0) 0x7fa8780adf20 con 0x7fa880108cf0 2024-09-21T06:07:32.985 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T06:07:32.985 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":29,"btime":"2024-09-21T06:04:08:952483+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3227057504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3227057504},{"type":"v1","addr":"172.21.15.117:6835","nonce":3227057504}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3303829183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3303829183},{"type":"v1","addr":"172.21.15.117:6837","nonce":3303829183}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T05:55:14.555897+0000","modified":"2024-09-21T06:04:08.952466+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24638,"mds_1":24637,"mds_2":24628,"mds_3":24616,"mds_4":24602},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6839/2314572411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2314572411},{"type":"v1","addr":"172.21.15.117:6839","nonce":2314572411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6841/3378096561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":3378096561},{"type":"v1","addr":"172.21.15.117:6841","nonce":3378096561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/3725629862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3725629862},{"type":"v1","addr":"172.21.15.138:6833","nonce":3725629862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/4276801396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4276801396},{"type":"v1","addr":"172.21.15.138:6835","nonce":4276801396}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.138:6837/2540335374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2540335374},{"type":"v1","addr":"172.21.15.138:6837","nonce":2540335374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24616":{"gid":24616,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.192:6835/3530688860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3530688860},{"type":"v1","addr":"172.21.15.192:6835","nonce":3530688860}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.192:6837/2683860842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2683860842},{"type":"v1","addr":"172.21.15.192:6837","nonce":2683860842}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.192:6839/1727623231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1727623231},{"type":"v1","addr":"172.21.15.192:6839","nonce":1727623231}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/3230372576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":3230372576},{"type":"v1","addr":"172.21.15.138:6839","nonce":3230372576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25840":{"gid":25840,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6833/1596605058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1596605058},{"type":"v1","addr":"172.21.15.192:6833","nonce":1596605058}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24602,24628,24616,24637,24638]},"id":1}]} 2024-09-21T06:07:32.986 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 29 2024-09-21T06:07:32.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.986+0000 7fa885590640 1 -- 172.21.15.117:0/2128464109 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa8680b6680 msgr2=0x7fa8680b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:32.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.986+0000 7fa885590640 1 --2- 172.21.15.117:0/2128464109 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa8680b6680 0x7fa8680b8b90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa870031440 tx=0x7fa870009680 comp rx=0 tx=0).stop 2024-09-21T06:07:32.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.986+0000 7fa885590640 1 -- 172.21.15.117:0/2128464109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa880108cf0 msgr2=0x7fa88019ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:32.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.986+0000 7fa885590640 1 --2- 172.21.15.117:0/2128464109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa880108cf0 0x7fa88019ec10 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fa87800eaa0 tx=0x7fa87800ef60 comp rx=0 tx=0).stop 2024-09-21T06:07:32.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.986+0000 7fa885590640 1 -- 172.21.15.117:0/2128464109 shutdown_connections 2024-09-21T06:07:32.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.986+0000 7fa885590640 1 --2- 172.21.15.117:0/2128464109 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa8680b6680 0x7fa8680b8b90 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:32.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.986+0000 7fa885590640 1 --2- 172.21.15.117:0/2128464109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa880108cf0 0x7fa88019ec10 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:32.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.986+0000 7fa885590640 1 --2- 172.21.15.117:0/2128464109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8801083a0 0x7fa880199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:32.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.986+0000 7fa885590640 1 --2- 172.21.15.117:0/2128464109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa880102a60 0x7fa8801993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:32.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.986+0000 7fa885590640 1 -- 172.21.15.117:0/2128464109 >> 172.21.15.117:0/2128464109 conn(0x7fa880079730 msgr2=0x7fa8801013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:32.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.986+0000 7fa885590640 1 -- 172.21.15.117:0/2128464109 shutdown_connections 2024-09-21T06:07:32.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:32.986+0000 7fa885590640 1 -- 172.21.15.117:0/2128464109 wait complete. 2024-09-21T06:07:33.748 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T06:07:33.996 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:07:34.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:33 smithi192 ceph-mon[34700]: pgmap v641: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-21T06:07:34.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:33 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2128464109' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:07:34.239 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:33 smithi117 ceph-mon[30947]: pgmap v641: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-21T06:07:34.239 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:33 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2128464109' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:07:34.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:33 smithi138 ceph-mon[34391]: pgmap v641: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-21T06:07:34.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:33 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2128464109' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:07:34.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.461+0000 7f4c3d48c640 1 -- 172.21.15.117:0/3414164845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c38102a60 msgr2=0x7f4c38102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:34.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.461+0000 7f4c3d48c640 1 --2- 172.21.15.117:0/3414164845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c38102a60 0x7f4c38102e70 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f4c28009900 tx=0x7f4c2802a410 comp rx=0 tx=0).stop 2024-09-21T06:07:34.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.461+0000 7f4c3d48c640 1 -- 172.21.15.117:0/3414164845 shutdown_connections 2024-09-21T06:07:34.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.461+0000 7f4c3d48c640 1 --2- 172.21.15.117:0/3414164845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c38108cf0 0x7f4c3810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.464 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.461+0000 7f4c3d48c640 1 --2- 172.21.15.117:0/3414164845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c381083a0 0x7f4c381087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.461+0000 7f4c3d48c640 1 --2- 172.21.15.117:0/3414164845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c38102a60 0x7f4c38102e70 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.461+0000 7f4c3d48c640 1 -- 172.21.15.117:0/3414164845 >> 172.21.15.117:0/3414164845 conn(0x7f4c38079730 msgr2=0x7f4c38079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:34.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.462+0000 7f4c3d48c640 1 -- 172.21.15.117:0/3414164845 shutdown_connections 2024-09-21T06:07:34.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.462+0000 7f4c3d48c640 1 -- 172.21.15.117:0/3414164845 wait complete. 2024-09-21T06:07:34.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.462+0000 7f4c3d48c640 1 Processor -- start 2024-09-21T06:07:34.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.462+0000 7f4c3d48c640 1 -- start start 2024-09-21T06:07:34.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.463+0000 7f4c3d48c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c38102a60 0x7f4c3819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:34.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.463+0000 7f4c3d48c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c381083a0 0x7f4c3819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:34.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.463+0000 7f4c3d48c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c38108cf0 0x7f4c3819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:34.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.463+0000 7f4c3d48c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4c38113a70 con 0x7f4c38108cf0 2024-09-21T06:07:34.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.463+0000 7f4c3d48c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4c381138f0 con 0x7f4c381083a0 2024-09-21T06:07:34.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.463+0000 7f4c3d48c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4c38113bf0 con 0x7f4c38102a60 2024-09-21T06:07:34.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.463+0000 7f4c377fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c38108cf0 0x7f4c3819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:34.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.463+0000 7f4c367fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c381083a0 0x7f4c3819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:34.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.463+0000 7f4c377fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c38108cf0 0x7f4c3819fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41712/0 (socket says 172.21.15.117:41712) 2024-09-21T06:07:34.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.463+0000 7f4c377fe640 1 -- 172.21.15.117:0/2479712621 learned_addr learned my addr 172.21.15.117:0/2479712621 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:34.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.463+0000 7f4c36ffd640 1 --2- 172.21.15.117:0/2479712621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c38102a60 0x7f4c3819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:34.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.464+0000 7f4c377fe640 1 -- 172.21.15.117:0/2479712621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c38102a60 msgr2=0x7f4c3819a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:34.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.464+0000 7f4c377fe640 1 --2- 172.21.15.117:0/2479712621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c38102a60 0x7f4c3819a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.464+0000 7f4c377fe640 1 -- 172.21.15.117:0/2479712621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c381083a0 msgr2=0x7f4c3819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:34.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.464+0000 7f4c377fe640 1 --2- 172.21.15.117:0/2479712621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c381083a0 0x7f4c3819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.464+0000 7f4c377fe640 1 -- 172.21.15.117:0/2479712621 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c381a03b0 con 0x7f4c38108cf0 2024-09-21T06:07:34.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.464+0000 7f4c36ffd640 1 --2- 172.21.15.117:0/2479712621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c38102a60 0x7f4c3819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:34.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.464+0000 7f4c367fc640 1 --2- 172.21.15.117:0/2479712621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c381083a0 0x7f4c3819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:07:34.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.464+0000 7f4c377fe640 1 --2- 172.21.15.117:0/2479712621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c38108cf0 0x7f4c3819fc80 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f4c3000cad0 tx=0x7f4c3000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:34.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.464+0000 7f4c1ffff640 1 -- 172.21.15.117:0/2479712621 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3000ae20 con 0x7f4c38108cf0 2024-09-21T06:07:34.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.464+0000 7f4c3d48c640 1 -- 172.21.15.117:0/2479712621 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c381a06a0 con 0x7f4c38108cf0 2024-09-21T06:07:34.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.464+0000 7f4c1ffff640 1 -- 172.21.15.117:0/2479712621 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c30005020 con 0x7f4c38108cf0 2024-09-21T06:07:34.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.464+0000 7f4c1ffff640 1 -- 172.21.15.117:0/2479712621 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3000f6b0 con 0x7f4c38108cf0 2024-09-21T06:07:34.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.464+0000 7f4c3d48c640 1 -- 172.21.15.117:0/2479712621 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c381a6e00 con 0x7f4c38108cf0 2024-09-21T06:07:34.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.467+0000 7f4c43a00740 1 Processor -- start 2024-09-21T06:07:34.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.467+0000 7f4c1ffff640 1 -- 172.21.15.117:0/2479712621 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f4c3000f850 con 0x7f4c38108cf0 2024-09-21T06:07:34.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.467+0000 7f4c43a00740 1 -- start start 2024-09-21T06:07:34.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.467+0000 7f4c43a00740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563fd68e070 0x5563fda356a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:34.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.467+0000 7f4c43a00740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563fd6651b0 0x5563fda35be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:34.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.467+0000 7f4c43a00740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563fd66d390 0x5563fda36120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:34.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.467+0000 7f4c43a00740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5563fd9abea0 con 0x5563fd66d390 2024-09-21T06:07:34.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.467+0000 7f4c43a00740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5563fd99b2d0 con 0x5563fd68e070 2024-09-21T06:07:34.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.467+0000 7f4c43a00740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5563fd99f0e0 con 0x5563fd6651b0 2024-09-21T06:07:34.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.467+0000 7f4c367fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563fd66d390 0x5563fda36120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:34.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.467+0000 7f4c367fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563fd66d390 0x5563fda36120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41726/0 (socket says 172.21.15.117:41726) 2024-09-21T06:07:34.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.467+0000 7f4c367fc640 1 -- 172.21.15.117:0/2991359373 learned_addr learned my addr 172.21.15.117:0/2991359373 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:34.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.467+0000 7f4c367fc640 1 --2- 172.21.15.117:0/2991359373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563fd68e070 0x5563fda356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:34.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.468+0000 7f4c36ffd640 1 --2- 172.21.15.117:0/2991359373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563fd6651b0 0x5563fda35be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:34.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.468+0000 7f4c367fc640 1 -- 172.21.15.117:0/2991359373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563fd6651b0 msgr2=0x5563fda35be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:34.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.468+0000 7f4c367fc640 1 --2- 172.21.15.117:0/2991359373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563fd6651b0 0x5563fda35be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.468+0000 7f4c367fc640 1 -- 172.21.15.117:0/2991359373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563fd68e070 msgr2=0x5563fda356a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:34.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.468+0000 7f4c367fc640 1 --2- 172.21.15.117:0/2991359373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563fd68e070 0x5563fda356a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.468+0000 7f4c367fc640 1 -- 172.21.15.117:0/2991359373 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563fd6f9650 con 0x5563fd66d390 2024-09-21T06:07:34.471 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.468+0000 7f4c1ffff640 1 --2- 172.21.15.117:0/2479712621 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4c200b6680 0x7f4c200b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:34.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.468+0000 7f4c367fc640 1 --2- 172.21.15.117:0/2991359373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563fd66d390 0x5563fda36120 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0118e0 tx=0x7f4c2c00ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5483806d38bc8571 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:34.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.468+0000 7f4c36ffd640 1 --2- 172.21.15.117:0/2479712621 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4c200b6680 0x7f4c200b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:34.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.468+0000 7f4c1ffff640 1 -- 172.21.15.117:0/2479712621 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f4c300e7700 con 0x7f4c38108cf0 2024-09-21T06:07:34.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.468+0000 7f4c1d7fa640 1 -- 172.21.15.117:0/2991359373 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c00fcd0 con 0x5563fd66d390 2024-09-21T06:07:34.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.468+0000 7f4c1d7fa640 1 -- 172.21.15.117:0/2991359373 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c2c00fe70 con 0x5563fd66d390 2024-09-21T06:07:34.472 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.468+0000 7f4c1d7fa640 1 -- 172.21.15.117:0/2991359373 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c003df0 con 0x5563fd66d390 2024-09-21T06:07:34.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.469+0000 7f4c43a00740 1 -- 172.21.15.117:0/2991359373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563fd66d390 msgr2=0x5563fda36120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:34.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.469+0000 7f4c36ffd640 1 --2- 172.21.15.117:0/2479712621 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4c200b6680 0x7f4c200b8b90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4c28031440 tx=0x7f4c28005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:34.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.469+0000 7f4c43a00740 1 --2- 172.21.15.117:0/2991359373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563fd66d390 0x5563fda36120 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0118e0 tx=0x7f4c2c00ab20 comp rx=0 tx=0).stop 2024-09-21T06:07:34.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.469+0000 7f4c43a00740 1 -- 172.21.15.117:0/2991359373 shutdown_connections 2024-09-21T06:07:34.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.469+0000 7f4c43a00740 1 --2- 172.21.15.117:0/2991359373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563fd66d390 0x5563fda36120 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.469+0000 7f4c43a00740 1 --2- 172.21.15.117:0/2991359373 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563fd6651b0 0x5563fda35be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.469+0000 7f4c43a00740 1 --2- 172.21.15.117:0/2991359373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563fd68e070 0x5563fda356a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.469+0000 7f4c43a00740 1 -- 172.21.15.117:0/2991359373 >> 172.21.15.117:0/2991359373 conn(0x5563fd682460 msgr2=0x5563fda33590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:34.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.469+0000 7f4c43a00740 1 -- 172.21.15.117:0/2991359373 shutdown_connections 2024-09-21T06:07:34.477 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.469+0000 7f4c43a00740 1 -- 172.21.15.117:0/2991359373 wait complete. 2024-09-21T06:07:34.477 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.470+0000 7f4c43a00740 1 -- start start 2024-09-21T06:07:34.477 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.470+0000 7f4c43a00740 1 Processor -- start 2024-09-21T06:07:34.477 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.470+0000 7f4c43a00740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563fd6651b0 0x5563fda40d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:34.477 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.470+0000 7f4c43a00740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563fd66d390 0x5563fda41240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:34.477 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.470+0000 7f4c43a00740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563fd68e070 0x5563fda4d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:34.477 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.470+0000 7f4c43a00740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5563fd7fc700 con 0x5563fd68e070 2024-09-21T06:07:34.477 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.470+0000 7f4c43a00740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5563fd936160 con 0x5563fd6651b0 2024-09-21T06:07:34.477 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.470+0000 7f4c43a00740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5563fda43e80 con 0x5563fd66d390 2024-09-21T06:07:34.477 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.470+0000 7f4c43a00740 10 client.0 _ll_register_callbacks cb 0x5563fd94f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:34.477 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.470+0000 7f4c377fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563fd68e070 0x5563fda4d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:34.477 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.470+0000 7f4c377fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563fd68e070 0x5563fda4d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41732/0 (socket says 172.21.15.117:41732) 2024-09-21T06:07:34.477 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.470+0000 7f4c377fe640 1 -- 172.21.15.117:0/492294815 learned_addr learned my addr 172.21.15.117:0/492294815 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:34.477 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.470+0000 7f4c367fc640 1 --2- 172.21.15.117:0/492294815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563fd66d390 0x5563fda41240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:34.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.471+0000 7f4c36ffd640 1 --2- 172.21.15.117:0/492294815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563fd6651b0 0x5563fda40d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:34.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.471+0000 7f4c377fe640 1 -- 172.21.15.117:0/492294815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563fd66d390 msgr2=0x5563fda41240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:34.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.471+0000 7f4c377fe640 1 --2- 172.21.15.117:0/492294815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563fd66d390 0x5563fda41240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.471+0000 7f4c377fe640 1 -- 172.21.15.117:0/492294815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563fd6651b0 msgr2=0x5563fda40d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:34.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.471+0000 7f4c377fe640 1 --2- 172.21.15.117:0/492294815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563fd6651b0 0x5563fda40d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.471+0000 7f4c377fe640 1 -- 172.21.15.117:0/492294815 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563fd963c90 con 0x5563fd68e070 2024-09-21T06:07:34.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.471+0000 7f4c377fe640 1 --2- 172.21.15.117:0/492294815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563fd68e070 0x5563fda4d1a0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f4c300ad2a0 tx=0x7f4c300a9a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb5364b3b2cd313 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:34.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.471+0000 7f4c1aff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:34.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.471+0000 7f4c1aff5640 1 -- 172.21.15.117:0/492294815 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3001d030 con 0x5563fd68e070 2024-09-21T06:07:34.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.471+0000 7f4c1aff5640 1 -- 172.21.15.117:0/492294815 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f4c3000fd40 con 0x5563fd68e070 2024-09-21T06:07:34.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.471+0000 7f4c43a00740 1 -- 172.21.15.117:0/492294815 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c2c00fe70 con 0x5563fd68e070 2024-09-21T06:07:34.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.471+0000 7f4c1aff5640 1 -- 172.21.15.117:0/492294815 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4c300b6fc0 con 0x5563fd68e070 2024-09-21T06:07:34.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.471+0000 7f4c43a00740 10 client.17727 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:34.478 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.471+0000 7f4c43a00740 1 -- 172.21.15.117:0/492294815 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5563fda4fbc0 con 0x5563fd68e070 2024-09-21T06:07:34.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.471+0000 7f4c1aff5640 1 -- 172.21.15.117:0/492294815 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c2c00fe70 con 0x5563fd68e070 2024-09-21T06:07:34.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.472+0000 7f4c1aff5640 1 -- 172.21.15.117:0/492294815 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4c30094ce0 con 0x5563fd68e070 2024-09-21T06:07:34.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.472+0000 7f4c43a00740 10 client.17727 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:34.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.472+0000 7f4c43a00740 10 client.17727 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T06:07:34.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.472+0000 7f4c43a00740 20 client.17727 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:34.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.472+0000 7f4c43a00740 1 --2- 172.21.15.117:0/492294815 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x5563fd64c9f0 0x5563fda530f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:34.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.472+0000 7f4c43a00740 4 client.17727 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:07:34.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.472+0000 7f4c43a00740 1 -- 172.21.15.117:0/492294815 --> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563fd9addf0 con 0x5563fd64c9f0 2024-09-21T06:07:34.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.472+0000 7f4c36ffd640 1 --2- 172.21.15.117:0/492294815 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x5563fd64c9f0 0x5563fda530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:34.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.472+0000 7f4c1aff5640 10 client.17727 ms_handle_connect on v2:172.21.15.138:6836/2540335374 2024-09-21T06:07:34.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.472+0000 7f4c36ffd640 1 --2- 172.21.15.117:0/492294815 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x5563fd64c9f0 0x5563fda530f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b4d2e16dae533a78 server_cookie=45c17aeaabd4d56f in_seq=0 out_seq=0 2024-09-21T06:07:34.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.473+0000 7f4c1aff5640 1 -- 172.21.15.117:0/492294815 <== mds.4 v2:172.21.15.138:6836/2540335374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5563fd9addf0 con 0x5563fd64c9f0 2024-09-21T06:07:34.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.473+0000 7f4c1aff5640 10 client.17727 handle_command_reply: tid=0 2024-09-21T06:07:34.479 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.473+0000 7f4c43a00740 1 client.17727 shutdown 2024-09-21T06:07:34.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.473+0000 7f4c43a00740 1 -- 172.21.15.117:0/492294815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563fd68e070 msgr2=0x5563fda4d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:34.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.473+0000 7f4c43a00740 1 --2- 172.21.15.117:0/492294815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563fd68e070 0x5563fda4d1a0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f4c300ad2a0 tx=0x7f4c300a9a90 comp rx=0 tx=0).stop 2024-09-21T06:07:34.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.473+0000 7f4c43a00740 1 -- 172.21.15.117:0/492294815 shutdown_connections 2024-09-21T06:07:34.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.473+0000 7f4c43a00740 1 --2- 172.21.15.117:0/492294815 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x5563fd64c9f0 0x5563fda530f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.473+0000 7f4c43a00740 1 --2- 172.21.15.117:0/492294815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563fd68e070 0x5563fda4d1a0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.473+0000 7f4c43a00740 1 --2- 172.21.15.117:0/492294815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563fd66d390 0x5563fda41240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.473+0000 7f4c43a00740 1 --2- 172.21.15.117:0/492294815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563fd6651b0 0x5563fda40d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.473+0000 7f4c43a00740 1 -- 172.21.15.117:0/492294815 >> 172.21.15.117:0/492294815 conn(0x5563fd682460 msgr2=0x5563fda38830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:34.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.473+0000 7f4c1aff5640 0 client.17727 ms_handle_reset on v2:172.21.15.138:6836/2540335374 2024-09-21T06:07:34.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.474+0000 7f4c43a00740 1 -- 172.21.15.117:0/492294815 shutdown_connections 2024-09-21T06:07:34.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.474+0000 7f4c43a00740 1 -- 172.21.15.117:0/492294815 wait complete. 2024-09-21T06:07:34.480 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.474+0000 7f4c43a00740 20 client.17727 trim_cache size 0 max 16384 2024-09-21T06:07:34.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.487+0000 7f4c43a00740 1 Processor -- start 2024-09-21T06:07:34.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.487+0000 7f4c43a00740 1 -- start start 2024-09-21T06:07:34.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.487+0000 7f4c43a00740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563fd6651b0 0x5563fda36de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:34.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.487+0000 7f4c43a00740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563fd66d390 0x5563fda32bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:34.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.487+0000 7f4c43a00740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563fd68e070 0x5563fda33100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:34.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.487+0000 7f4c43a00740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4c381af520 con 0x5563fd66d390 2024-09-21T06:07:34.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.487+0000 7f4c43a00740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4c381b1060 con 0x5563fd6651b0 2024-09-21T06:07:34.489 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.487+0000 7f4c43a00740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4c381afbf0 con 0x5563fd68e070 2024-09-21T06:07:34.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.487+0000 7f4c367fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563fd66d390 0x5563fda32bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:34.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.487+0000 7f4c367fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563fd66d390 0x5563fda32bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41740/0 (socket says 172.21.15.117:41740) 2024-09-21T06:07:34.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.487+0000 7f4c367fc640 1 -- 172.21.15.117:0/3274896815 learned_addr learned my addr 172.21.15.117:0/3274896815 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:34.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.487+0000 7f4c36ffd640 1 --2- 172.21.15.117:0/3274896815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563fd6651b0 0x5563fda36de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:34.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.487+0000 7f4c377fe640 1 --2- 172.21.15.117:0/3274896815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563fd68e070 0x5563fda33100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:34.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.487+0000 7f4c367fc640 1 -- 172.21.15.117:0/3274896815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563fd68e070 msgr2=0x5563fda33100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:34.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.487+0000 7f4c367fc640 1 --2- 172.21.15.117:0/3274896815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563fd68e070 0x5563fda33100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.487+0000 7f4c367fc640 1 -- 172.21.15.117:0/3274896815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563fd6651b0 msgr2=0x5563fda36de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:34.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.487+0000 7f4c367fc640 1 --2- 172.21.15.117:0/3274896815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563fd6651b0 0x5563fda36de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.487+0000 7f4c367fc640 1 -- 172.21.15.117:0/3274896815 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563fda3b9d0 con 0x5563fd66d390 2024-09-21T06:07:34.490 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.488+0000 7f4c367fc640 1 --2- 172.21.15.117:0/3274896815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563fd66d390 0x5563fda32bc0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c00a520 tx=0x7f4c2c0039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83d4b8b4f6045617 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:34.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.488+0000 7f4c1cff9640 1 -- 172.21.15.117:0/3274896815 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c0036b0 con 0x5563fd66d390 2024-09-21T06:07:34.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.488+0000 7f4c1cff9640 1 -- 172.21.15.117:0/3274896815 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c2c011040 con 0x5563fd66d390 2024-09-21T06:07:34.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.488+0000 7f4c1cff9640 1 -- 172.21.15.117:0/3274896815 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c005540 con 0x5563fd66d390 2024-09-21T06:07:34.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.489+0000 7f4c43a00740 1 -- 172.21.15.117:0/3274896815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563fd66d390 msgr2=0x5563fda32bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:34.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.489+0000 7f4c43a00740 1 --2- 172.21.15.117:0/3274896815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563fd66d390 0x5563fda32bc0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c00a520 tx=0x7f4c2c0039d0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.489+0000 7f4c43a00740 1 -- 172.21.15.117:0/3274896815 shutdown_connections 2024-09-21T06:07:34.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.489+0000 7f4c43a00740 1 --2- 172.21.15.117:0/3274896815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563fd66d390 0x5563fda32bc0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.489+0000 7f4c43a00740 1 --2- 172.21.15.117:0/3274896815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563fd68e070 0x5563fda33100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.489+0000 7f4c43a00740 1 --2- 172.21.15.117:0/3274896815 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563fd6651b0 0x5563fda36de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.489+0000 7f4c43a00740 1 -- 172.21.15.117:0/3274896815 >> 172.21.15.117:0/3274896815 conn(0x5563fd682460 msgr2=0x5563fda366d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:34.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.489+0000 7f4c43a00740 1 -- 172.21.15.117:0/3274896815 shutdown_connections 2024-09-21T06:07:34.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.489+0000 7f4c43a00740 1 -- 172.21.15.117:0/3274896815 wait complete. 2024-09-21T06:07:34.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.489+0000 7f4c43a00740 1 -- start start 2024-09-21T06:07:34.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.489+0000 7f4c43a00740 1 Processor -- start 2024-09-21T06:07:34.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.490+0000 7f4c43a00740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563fd6651b0 0x5563fda49ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:34.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.490+0000 7f4c43a00740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563fd66d390 0x5563fda44fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:34.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.490+0000 7f4c43a00740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563fd68e070 0x5563fda456c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:34.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.490+0000 7f4c43a00740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4c381b0990 con 0x5563fd6651b0 2024-09-21T06:07:34.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.490+0000 7f4c43a00740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4c381b02c0 con 0x5563fd66d390 2024-09-21T06:07:34.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.490+0000 7f4c43a00740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4c381ad520 con 0x5563fd68e070 2024-09-21T06:07:34.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.490+0000 7f4c43a00740 10 client.0 _ll_register_callbacks cb 0x7f4c300a9c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:34.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.490+0000 7f4c36ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563fd6651b0 0x5563fda49ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:34.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.490+0000 7f4c36ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563fd6651b0 0x5563fda49ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41754/0 (socket says 172.21.15.117:41754) 2024-09-21T06:07:34.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.490+0000 7f4c36ffd640 1 -- 172.21.15.117:0/66876596 learned_addr learned my addr 172.21.15.117:0/66876596 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:34.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.490+0000 7f4c367fc640 1 --2- 172.21.15.117:0/66876596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563fd66d390 0x5563fda44fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:34.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.490+0000 7f4c377fe640 1 --2- 172.21.15.117:0/66876596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563fd68e070 0x5563fda456c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:34.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.490+0000 7f4c36ffd640 1 -- 172.21.15.117:0/66876596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563fd68e070 msgr2=0x5563fda456c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:34.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.490+0000 7f4c36ffd640 1 --2- 172.21.15.117:0/66876596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563fd68e070 0x5563fda456c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.490+0000 7f4c36ffd640 1 -- 172.21.15.117:0/66876596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563fd66d390 msgr2=0x5563fda44fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:34.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.490+0000 7f4c36ffd640 1 --2- 172.21.15.117:0/66876596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563fd66d390 0x5563fda44fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.490+0000 7f4c36ffd640 1 -- 172.21.15.117:0/66876596 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563fdabca80 con 0x5563fd6651b0 2024-09-21T06:07:34.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.490+0000 7f4c377fe640 1 --2- 172.21.15.117:0/66876596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563fd68e070 0x5563fda456c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:34.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.490+0000 7f4c36ffd640 1 --2- 172.21.15.117:0/66876596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563fd6651b0 0x5563fda49ab0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f4c2802a930 tx=0x7f4c2803d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a0296ac37e0ca3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:34.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.490+0000 7f4c1aff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:34.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.491+0000 7f4c1aff5640 1 -- 172.21.15.117:0/66876596 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c28035050 con 0x5563fd6651b0 2024-09-21T06:07:34.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.491+0000 7f4c1aff5640 1 -- 172.21.15.117:0/66876596 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f4c280037f0 con 0x5563fd6651b0 2024-09-21T06:07:34.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.491+0000 7f4c43a00740 1 -- 172.21.15.117:0/66876596 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c2c011040 con 0x5563fd6651b0 2024-09-21T06:07:34.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.491+0000 7f4c1aff5640 1 -- 172.21.15.117:0/66876596 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4c280044d0 con 0x5563fd6651b0 2024-09-21T06:07:34.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.491+0000 7f4c43a00740 10 client.17733 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:34.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.491+0000 7f4c43a00740 1 -- 172.21.15.117:0/66876596 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5563fdabdc00 con 0x5563fd6651b0 2024-09-21T06:07:34.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.491+0000 7f4c1aff5640 1 -- 172.21.15.117:0/66876596 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c28054070 con 0x5563fd6651b0 2024-09-21T06:07:34.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.491+0000 7f4c1aff5640 1 -- 172.21.15.117:0/66876596 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4c28049b90 con 0x5563fd6651b0 2024-09-21T06:07:34.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.491+0000 7f4c43a00740 10 client.17733 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:34.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.491+0000 7f4c43a00740 10 client.17733 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T06:07:34.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.491+0000 7f4c43a00740 20 client.17733 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:34.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.491+0000 7f4c43a00740 1 --2- 172.21.15.117:0/66876596 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x5563fd621da0 0x5563fdac02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:34.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.492+0000 7f4c43a00740 4 client.17733 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:07:34.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.492+0000 7f4c43a00740 1 -- 172.21.15.117:0/66876596 --> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f4c2c011040 con 0x5563fd621da0 2024-09-21T06:07:34.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.492+0000 7f4c367fc640 1 --2- 172.21.15.117:0/66876596 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x5563fd621da0 0x5563fdac02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:34.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.492+0000 7f4c1aff5640 10 client.17733 ms_handle_connect on v2:172.21.15.138:6836/2540335374 2024-09-21T06:07:34.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.492+0000 7f4c367fc640 1 --2- 172.21.15.117:0/66876596 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x5563fd621da0 0x5563fdac02c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=82ae04d552c8b628 server_cookie=635ccd7735f8f790 in_seq=0 out_seq=0 2024-09-21T06:07:34.498 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.493+0000 7f4c1aff5640 1 -- 172.21.15.117:0/66876596 <== mds.4 v2:172.21.15.138:6836/2540335374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f4c2c011040 con 0x5563fd621da0 2024-09-21T06:07:34.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.493+0000 7f4c1aff5640 10 client.17733 handle_command_reply: tid=0 2024-09-21T06:07:34.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.493+0000 7f4c43a00740 1 client.17733 shutdown 2024-09-21T06:07:34.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.493+0000 7f4c43a00740 1 -- 172.21.15.117:0/66876596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563fd6651b0 msgr2=0x5563fda49ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:34.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.493+0000 7f4c43a00740 1 --2- 172.21.15.117:0/66876596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563fd6651b0 0x5563fda49ab0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f4c2802a930 tx=0x7f4c2803d070 comp rx=0 tx=0).stop 2024-09-21T06:07:34.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.493+0000 7f4c43a00740 1 -- 172.21.15.117:0/66876596 shutdown_connections 2024-09-21T06:07:34.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.493+0000 7f4c43a00740 1 --2- 172.21.15.117:0/66876596 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x5563fd621da0 0x5563fdac02c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.493+0000 7f4c43a00740 1 --2- 172.21.15.117:0/66876596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5563fd68e070 0x5563fda456c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.493+0000 7f4c43a00740 1 --2- 172.21.15.117:0/66876596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5563fd66d390 0x5563fda44fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.493+0000 7f4c1aff5640 0 client.17733 ms_handle_reset on v2:172.21.15.138:6836/2540335374 2024-09-21T06:07:34.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.493+0000 7f4c43a00740 1 --2- 172.21.15.117:0/66876596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5563fd6651b0 0x5563fda49ab0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.493+0000 7f4c43a00740 1 -- 172.21.15.117:0/66876596 >> 172.21.15.117:0/66876596 conn(0x5563fd682460 msgr2=0x5563fda41fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:34.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.493+0000 7f4c43a00740 1 -- 172.21.15.117:0/66876596 shutdown_connections 2024-09-21T06:07:34.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.493+0000 7f4c43a00740 1 -- 172.21.15.117:0/66876596 wait complete. 2024-09-21T06:07:34.499 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.493+0000 7f4c43a00740 20 client.17733 trim_cache size 0 max 16384 2024-09-21T06:07:34.500 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:07:34.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.493+0000 7f4c3d48c640 1 -- 172.21.15.117:0/2479712621 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4c200b6680 msgr2=0x7f4c200b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:34.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.493+0000 7f4c3d48c640 1 --2- 172.21.15.117:0/2479712621 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4c200b6680 0x7f4c200b8b90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4c28031440 tx=0x7f4c28005e80 comp rx=0 tx=0).stop 2024-09-21T06:07:34.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.493+0000 7f4c3d48c640 1 -- 172.21.15.117:0/2479712621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c38108cf0 msgr2=0x7f4c3819fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:34.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.493+0000 7f4c3d48c640 1 --2- 172.21.15.117:0/2479712621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c38108cf0 0x7f4c3819fc80 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f4c3000cad0 tx=0x7f4c3000cf90 comp rx=0 tx=0).stop 2024-09-21T06:07:34.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.494+0000 7f4c3d48c640 1 -- 172.21.15.117:0/2479712621 shutdown_connections 2024-09-21T06:07:34.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.494+0000 7f4c3d48c640 1 --2- 172.21.15.117:0/2479712621 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4c200b6680 0x7f4c200b8b90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.494+0000 7f4c3d48c640 1 --2- 172.21.15.117:0/2479712621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c38108cf0 0x7f4c3819fc80 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.494+0000 7f4c3d48c640 1 --2- 172.21.15.117:0/2479712621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c381083a0 0x7f4c3819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.494+0000 7f4c3d48c640 1 --2- 172.21.15.117:0/2479712621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c38102a60 0x7f4c3819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:34.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.494+0000 7f4c3d48c640 1 -- 172.21.15.117:0/2479712621 >> 172.21.15.117:0/2479712621 conn(0x7f4c38079730 msgr2=0x7f4c38101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:34.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.494+0000 7f4c3d48c640 1 -- 172.21.15.117:0/2479712621 shutdown_connections 2024-09-21T06:07:34.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:34.494+0000 7f4c3d48c640 1 -- 172.21.15.117:0/2479712621 wait complete. 2024-09-21T06:07:35.348 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T06:07:35.594 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:07:36.008 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:35 smithi117 ceph-mon[30947]: pgmap v642: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 170 B/s wr, 14 op/s 2024-09-21T06:07:36.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.101+0000 7f1e4cf3e640 1 -- 172.21.15.117:0/382612016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e48109df0 msgr2=0x7f1e48110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:36.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.101+0000 7f1e4cf3e640 1 --2- 172.21.15.117:0/382612016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e48109df0 0x7f1e48110620 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009ca0 tx=0x7f1e4002a8e0 comp rx=0 tx=0).stop 2024-09-21T06:07:36.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.101+0000 7f1e4cf3e640 1 -- 172.21.15.117:0/382612016 shutdown_connections 2024-09-21T06:07:36.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.101+0000 7f1e4cf3e640 1 --2- 172.21.15.117:0/382612016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e48109df0 0x7f1e48110620 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:36.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.101+0000 7f1e4cf3e640 1 --2- 172.21.15.117:0/382612016 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e481094a0 0x7f1e481098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:36.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.101+0000 7f1e4cf3e640 1 --2- 172.21.15.117:0/382612016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e48103b60 0x7f1e48103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:36.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.101+0000 7f1e4cf3e640 1 -- 172.21.15.117:0/382612016 >> 172.21.15.117:0/382612016 conn(0x7f1e4807b850 msgr2=0x7f1e4807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:36.117 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.101+0000 7f1e4cf3e640 1 -- 172.21.15.117:0/382612016 shutdown_connections 2024-09-21T06:07:36.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.101+0000 7f1e4cf3e640 1 -- 172.21.15.117:0/382612016 wait complete. 2024-09-21T06:07:36.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.102+0000 7f1e4cf3e640 1 Processor -- start 2024-09-21T06:07:36.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.102+0000 7f1e4cf3e640 1 -- start start 2024-09-21T06:07:36.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.102+0000 7f1e4cf3e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e48103b60 0x7f1e4819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:36.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.102+0000 7f1e4cf3e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e481094a0 0x7f1e4819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:36.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.102+0000 7f1e4cf3e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e48109df0 0x7f1e4819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:36.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.102+0000 7f1e4cf3e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1e48113af0 con 0x7f1e481094a0 2024-09-21T06:07:36.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.102+0000 7f1e4cf3e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1e48113970 con 0x7f1e48109df0 2024-09-21T06:07:36.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.102+0000 7f1e4cf3e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1e48113c70 con 0x7f1e48103b60 2024-09-21T06:07:36.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.103+0000 7f1e45d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e481094a0 0x7f1e4819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:36.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.103+0000 7f1e45d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e481094a0 0x7f1e4819aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41768/0 (socket says 172.21.15.117:41768) 2024-09-21T06:07:36.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.103+0000 7f1e45d9b640 1 -- 172.21.15.117:0/100709896 learned_addr learned my addr 172.21.15.117:0/100709896 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:36.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.103+0000 7f1e46d9d640 1 --2- 172.21.15.117:0/100709896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e48109df0 0x7f1e4819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:36.118 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.103+0000 7f1e4659c640 1 --2- 172.21.15.117:0/100709896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e48103b60 0x7f1e4819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:36.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.103+0000 7f1e45d9b640 1 -- 172.21.15.117:0/100709896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e48103b60 msgr2=0x7f1e4819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:36.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.103+0000 7f1e45d9b640 1 --2- 172.21.15.117:0/100709896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e48103b60 0x7f1e4819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:36.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.103+0000 7f1e45d9b640 1 -- 172.21.15.117:0/100709896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e48109df0 msgr2=0x7f1e4819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:36.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.103+0000 7f1e45d9b640 1 --2- 172.21.15.117:0/100709896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e48109df0 0x7f1e4819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:36.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.103+0000 7f1e45d9b640 1 -- 172.21.15.117:0/100709896 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e481a0350 con 0x7f1e481094a0 2024-09-21T06:07:36.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.103+0000 7f1e46d9d640 1 --2- 172.21.15.117:0/100709896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e48109df0 0x7f1e4819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:36.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.103+0000 7f1e45d9b640 1 --2- 172.21.15.117:0/100709896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e481094a0 0x7f1e4819aa40 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c006fd0 tx=0x7f1e3c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:36.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.103+0000 7f1e377fe640 1 -- 172.21.15.117:0/100709896 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c011830 con 0x7f1e481094a0 2024-09-21T06:07:36.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.103+0000 7f1e377fe640 1 -- 172.21.15.117:0/100709896 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e3c016070 con 0x7f1e481094a0 2024-09-21T06:07:36.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.103+0000 7f1e4cf3e640 1 -- 172.21.15.117:0/100709896 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e481a0640 con 0x7f1e481094a0 2024-09-21T06:07:36.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.103+0000 7f1e377fe640 1 -- 172.21.15.117:0/100709896 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c010e00 con 0x7f1e481094a0 2024-09-21T06:07:36.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.103+0000 7f1e4cf3e640 1 -- 172.21.15.117:0/100709896 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e481a6e60 con 0x7f1e481094a0 2024-09-21T06:07:36.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.106+0000 7f1e534b2740 1 Processor -- start 2024-09-21T06:07:36.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.106+0000 7f1e534b2740 1 -- start start 2024-09-21T06:07:36.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.106+0000 7f1e377fe640 1 -- 172.21.15.117:0/100709896 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f1e3c01b460 con 0x7f1e481094a0 2024-09-21T06:07:36.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.106+0000 7f1e534b2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a1b175a070 0x55a1b1b016a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:36.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.107+0000 7f1e534b2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1b17311b0 0x55a1b1b01be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:36.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.107+0000 7f1e534b2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b1739390 0x55a1b1b02120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:36.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.107+0000 7f1e534b2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a1b1a77ea0 con 0x55a1b175a070 2024-09-21T06:07:36.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.107+0000 7f1e534b2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a1b1a672d0 con 0x55a1b1739390 2024-09-21T06:07:36.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.107+0000 7f1e534b2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a1b1a6b0e0 con 0x55a1b17311b0 2024-09-21T06:07:36.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.107+0000 7f1e46d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a1b175a070 0x55a1b1b016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:36.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.107+0000 7f1e46d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a1b175a070 0x55a1b1b016a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41776/0 (socket says 172.21.15.117:41776) 2024-09-21T06:07:36.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.107+0000 7f1e46d9d640 1 -- 172.21.15.117:0/2050393085 learned_addr learned my addr 172.21.15.117:0/2050393085 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:36.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.107+0000 7f1e4659c640 1 --2- 172.21.15.117:0/2050393085 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1b17311b0 0x55a1b1b01be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:36.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.107+0000 7f1e45d9b640 1 --2- 172.21.15.117:0/2050393085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b1739390 0x55a1b1b02120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:36.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.107+0000 7f1e46d9d640 1 -- 172.21.15.117:0/2050393085 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1b17311b0 msgr2=0x55a1b1b01be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:36.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.107+0000 7f1e46d9d640 1 --2- 172.21.15.117:0/2050393085 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1b17311b0 0x55a1b1b01be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:36.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.107+0000 7f1e46d9d640 1 -- 172.21.15.117:0/2050393085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b1739390 msgr2=0x55a1b1b02120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:36.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.107+0000 7f1e46d9d640 1 --2- 172.21.15.117:0/2050393085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b1739390 0x55a1b1b02120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:36.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.107+0000 7f1e46d9d640 1 -- 172.21.15.117:0/2050393085 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1b17c5650 con 0x55a1b175a070 2024-09-21T06:07:36.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.107+0000 7f1e46d9d640 1 --2- 172.21.15.117:0/2050393085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a1b175a070 0x55a1b1b016a0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f1e40031630 tx=0x7f1e40031660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2f9b9fdc6602652 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:36.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.107+0000 7f1e377fe640 1 --2- 172.21.15.117:0/100709896 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1e300b6730 0x7f1e300b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:36.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.107+0000 7f1e34ff9640 1 -- 172.21.15.117:0/2050393085 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e40031ad0 con 0x55a1b175a070 2024-09-21T06:07:36.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.108+0000 7f1e34ff9640 1 -- 172.21.15.117:0/2050393085 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e40036040 con 0x55a1b175a070 2024-09-21T06:07:36.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.108+0000 7f1e4659c640 1 --2- 172.21.15.117:0/100709896 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1e300b6730 0x7f1e300b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:36.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.108+0000 7f1e34ff9640 1 -- 172.21.15.117:0/2050393085 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e40004660 con 0x55a1b175a070 2024-09-21T06:07:36.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.108+0000 7f1e377fe640 1 -- 172.21.15.117:0/100709896 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f1e3c0e6c20 con 0x7f1e481094a0 2024-09-21T06:07:36.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.108+0000 7f1e534b2740 1 -- 172.21.15.117:0/2050393085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a1b175a070 msgr2=0x55a1b1b016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:36.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.108+0000 7f1e4659c640 1 --2- 172.21.15.117:0/100709896 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1e300b6730 0x7f1e300b8c40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x55a1b1b00320 tx=0x7f1e3800c4b0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:36.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.108+0000 7f1e534b2740 1 --2- 172.21.15.117:0/2050393085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a1b175a070 0x55a1b1b016a0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f1e40031630 tx=0x7f1e40031660 comp rx=0 tx=0).stop 2024-09-21T06:07:36.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.109+0000 7f1e534b2740 1 -- 172.21.15.117:0/2050393085 shutdown_connections 2024-09-21T06:07:36.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.109+0000 7f1e534b2740 1 --2- 172.21.15.117:0/2050393085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b1739390 0x55a1b1b02120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:36.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.109+0000 7f1e534b2740 1 --2- 172.21.15.117:0/2050393085 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1b17311b0 0x55a1b1b01be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:36.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.109+0000 7f1e534b2740 1 --2- 172.21.15.117:0/2050393085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a1b175a070 0x55a1b1b016a0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:36.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.109+0000 7f1e534b2740 1 -- 172.21.15.117:0/2050393085 >> 172.21.15.117:0/2050393085 conn(0x55a1b174e460 msgr2=0x55a1b1aff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:36.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.109+0000 7f1e534b2740 1 -- 172.21.15.117:0/2050393085 shutdown_connections 2024-09-21T06:07:36.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.109+0000 7f1e534b2740 1 -- 172.21.15.117:0/2050393085 wait complete. 2024-09-21T06:07:36.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.109+0000 7f1e534b2740 1 -- start start 2024-09-21T06:07:36.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.109+0000 7f1e534b2740 1 Processor -- start 2024-09-21T06:07:36.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.109+0000 7f1e534b2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b17311b0 0x55a1b1b0cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:36.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.109+0000 7f1e534b2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1b1739390 0x55a1b1b0d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:36.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.109+0000 7f1e534b2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a1b175a070 0x55a1b1b19110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:36.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.109+0000 7f1e534b2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a1b18c8700 con 0x55a1b175a070 2024-09-21T06:07:36.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.109+0000 7f1e534b2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a1b1a02160 con 0x55a1b17311b0 2024-09-21T06:07:36.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.109+0000 7f1e534b2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a1b1b0fe20 con 0x55a1b1739390 2024-09-21T06:07:36.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.110+0000 7f1e534b2740 10 client.0 _ll_register_callbacks cb 0x55a1b1a1b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:36.122 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.110+0000 7f1e46d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a1b175a070 0x55a1b1b19110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:36.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.110+0000 7f1e46d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a1b175a070 0x55a1b1b19110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41780/0 (socket says 172.21.15.117:41780) 2024-09-21T06:07:36.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.110+0000 7f1e46d9d640 1 -- 172.21.15.117:0/609484472 learned_addr learned my addr 172.21.15.117:0/609484472 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:36.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.110+0000 7f1e4659c640 1 --2- 172.21.15.117:0/609484472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b17311b0 0x55a1b1b0cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:36.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.110+0000 7f1e45d9b640 1 --2- 172.21.15.117:0/609484472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1b1739390 0x55a1b1b0d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:36.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.110+0000 7f1e46d9d640 1 -- 172.21.15.117:0/609484472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1b1739390 msgr2=0x55a1b1b0d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:36.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.110+0000 7f1e46d9d640 1 --2- 172.21.15.117:0/609484472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1b1739390 0x55a1b1b0d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:36.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.110+0000 7f1e46d9d640 1 -- 172.21.15.117:0/609484472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b17311b0 msgr2=0x55a1b1b0cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:36.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.110+0000 7f1e46d9d640 1 --2- 172.21.15.117:0/609484472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b17311b0 0x55a1b1b0cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:36.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.110+0000 7f1e46d9d640 1 -- 172.21.15.117:0/609484472 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1b1a2fc90 con 0x55a1b175a070 2024-09-21T06:07:36.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.110+0000 7f1e46d9d640 1 --2- 172.21.15.117:0/609484472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a1b175a070 0x55a1b1b19110 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f1e400042a0 tx=0x7f1e400098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1138f633a073d8b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:36.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.110+0000 7f1e2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:36.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.110+0000 7f1e2dffb640 1 -- 172.21.15.117:0/609484472 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e40051020 con 0x55a1b175a070 2024-09-21T06:07:36.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.110+0000 7f1e2dffb640 1 -- 172.21.15.117:0/609484472 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f1e40031c10 con 0x55a1b175a070 2024-09-21T06:07:36.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.110+0000 7f1e534b2740 1 -- 172.21.15.117:0/609484472 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e40036040 con 0x55a1b175a070 2024-09-21T06:07:36.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.110+0000 7f1e2dffb640 1 -- 172.21.15.117:0/609484472 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1e4005c050 con 0x55a1b175a070 2024-09-21T06:07:36.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.110+0000 7f1e534b2740 10 client.17745 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:36.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.110+0000 7f1e534b2740 1 -- 172.21.15.117:0/609484472 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a1b1b1baf0 con 0x55a1b175a070 2024-09-21T06:07:36.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.111+0000 7f1e2dffb640 1 -- 172.21.15.117:0/609484472 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e40036040 con 0x55a1b175a070 2024-09-21T06:07:36.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.111+0000 7f1e2dffb640 1 -- 172.21.15.117:0/609484472 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1e4004c030 con 0x55a1b175a070 2024-09-21T06:07:36.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.111+0000 7f1e534b2740 10 client.17745 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:36.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.111+0000 7f1e534b2740 10 client.17745 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T06:07:36.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.111+0000 7f1e534b2740 20 client.17745 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:36.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.111+0000 7f1e534b2740 1 --2- 172.21.15.117:0/609484472 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55a1b17189f0 0x55a1b1b1ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:36.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.111+0000 7f1e534b2740 4 client.17745 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:07:36.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.111+0000 7f1e534b2740 1 -- 172.21.15.117:0/609484472 --> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1b1a79df0 con 0x55a1b17189f0 2024-09-21T06:07:36.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.111+0000 7f1e4659c640 1 --2- 172.21.15.117:0/609484472 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55a1b17189f0 0x55a1b1b1ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:36.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.112+0000 7f1e2dffb640 10 client.17745 ms_handle_connect on v2:172.21.15.192:6834/3530688860 2024-09-21T06:07:36.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.112+0000 7f1e4659c640 1 --2- 172.21.15.117:0/609484472 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55a1b17189f0 0x55a1b1b1ee50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=de53a83a93484699 server_cookie=d06c0a8a08a3102b in_seq=0 out_seq=0 2024-09-21T06:07:36.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.113+0000 7f1e2dffb640 1 -- 172.21.15.117:0/609484472 <== mds.3 v2:172.21.15.192:6834/3530688860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a1b1a79df0 con 0x55a1b17189f0 2024-09-21T06:07:36.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.113+0000 7f1e2dffb640 10 client.17745 handle_command_reply: tid=0 2024-09-21T06:07:36.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.113+0000 7f1e534b2740 1 client.17745 shutdown 2024-09-21T06:07:36.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.113+0000 7f1e534b2740 1 -- 172.21.15.117:0/609484472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a1b175a070 msgr2=0x55a1b1b19110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:36.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.113+0000 7f1e534b2740 1 --2- 172.21.15.117:0/609484472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a1b175a070 0x55a1b1b19110 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f1e400042a0 tx=0x7f1e400098e0 comp rx=0 tx=0).stop 2024-09-21T06:07:36.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.113+0000 7f1e534b2740 1 -- 172.21.15.117:0/609484472 shutdown_connections 2024-09-21T06:07:36.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.113+0000 7f1e534b2740 1 --2- 172.21.15.117:0/609484472 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55a1b17189f0 0x55a1b1b1ee50 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:36.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.113+0000 7f1e534b2740 1 --2- 172.21.15.117:0/609484472 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a1b175a070 0x55a1b1b19110 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:36.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.113+0000 7f1e534b2740 1 --2- 172.21.15.117:0/609484472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1b1739390 0x55a1b1b0d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:36.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.113+0000 7f1e534b2740 1 --2- 172.21.15.117:0/609484472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b17311b0 0x55a1b1b0cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:36.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.113+0000 7f1e2dffb640 0 client.17745 ms_handle_reset on v2:172.21.15.192:6834/3530688860 2024-09-21T06:07:36.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.113+0000 7f1e534b2740 1 -- 172.21.15.117:0/609484472 >> 172.21.15.117:0/609484472 conn(0x55a1b174e460 msgr2=0x55a1b1b04880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:36.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.113+0000 7f1e534b2740 1 -- 172.21.15.117:0/609484472 shutdown_connections 2024-09-21T06:07:36.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.113+0000 7f1e534b2740 1 -- 172.21.15.117:0/609484472 wait complete. 2024-09-21T06:07:36.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.113+0000 7f1e534b2740 20 client.17745 trim_cache size 0 max 16384 2024-09-21T06:07:36.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.126+0000 7f1e534b2740 1 Processor -- start 2024-09-21T06:07:36.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.127+0000 7f1e534b2740 1 -- start start 2024-09-21T06:07:36.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.127+0000 7f1e534b2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a1b17189f0 0x55a1b1b0e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:36.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.127+0000 7f1e534b2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1b17311b0 0x55a1b1afebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:36.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.127+0000 7f1e534b2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b1739390 0x55a1b1aff100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:36.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.127+0000 7f1e534b2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1e481af590 con 0x55a1b17189f0 2024-09-21T06:07:36.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.127+0000 7f1e534b2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1e481b10d0 con 0x55a1b1739390 2024-09-21T06:07:36.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.127+0000 7f1e534b2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1e481afc60 con 0x55a1b17311b0 2024-09-21T06:07:36.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.127+0000 7f1e4659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a1b17189f0 0x55a1b1b0e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:36.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.127+0000 7f1e4659c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a1b17189f0 0x55a1b1b0e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41794/0 (socket says 172.21.15.117:41794) 2024-09-21T06:07:36.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.127+0000 7f1e4659c640 1 -- 172.21.15.117:0/2705733197 learned_addr learned my addr 172.21.15.117:0/2705733197 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:36.130 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.127+0000 7f1e46d9d640 1 --2- 172.21.15.117:0/2705733197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b1739390 0x55a1b1aff100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:36.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.127+0000 7f1e4659c640 1 -- 172.21.15.117:0/2705733197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1b17311b0 msgr2=0x55a1b1afebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:36.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.127+0000 7f1e4659c640 1 --2- 172.21.15.117:0/2705733197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1b17311b0 0x55a1b1afebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:36.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.127+0000 7f1e4659c640 1 -- 172.21.15.117:0/2705733197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b1739390 msgr2=0x55a1b1aff100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:36.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.127+0000 7f1e4659c640 1 --2- 172.21.15.117:0/2705733197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b1739390 0x55a1b1aff100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:36.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.127+0000 7f1e4659c640 1 -- 172.21.15.117:0/2705733197 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1b1b0dfe0 con 0x55a1b17189f0 2024-09-21T06:07:36.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.127+0000 7f1e46d9d640 1 --2- 172.21.15.117:0/2705733197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b1739390 0x55a1b1aff100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:07:36.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.127+0000 7f1e4659c640 1 --2- 172.21.15.117:0/2705733197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a1b17189f0 0x55a1b1b0e950 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800cc80 tx=0x7f1e38014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90d026b75ffd77a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:36.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.128+0000 7f1e2ffff640 1 -- 172.21.15.117:0/2705733197 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38026420 con 0x55a1b17189f0 2024-09-21T06:07:36.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.128+0000 7f1e2ffff640 1 -- 172.21.15.117:0/2705733197 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e38026a70 con 0x55a1b17189f0 2024-09-21T06:07:36.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.128+0000 7f1e2ffff640 1 -- 172.21.15.117:0/2705733197 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3801ba10 con 0x55a1b17189f0 2024-09-21T06:07:36.131 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.129+0000 7f1e534b2740 1 -- 172.21.15.117:0/2705733197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a1b17189f0 msgr2=0x55a1b1b0e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:36.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.129+0000 7f1e534b2740 1 --2- 172.21.15.117:0/2705733197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a1b17189f0 0x55a1b1b0e950 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800cc80 tx=0x7f1e38014000 comp rx=0 tx=0).stop 2024-09-21T06:07:36.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.129+0000 7f1e534b2740 1 -- 172.21.15.117:0/2705733197 shutdown_connections 2024-09-21T06:07:36.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.129+0000 7f1e534b2740 1 --2- 172.21.15.117:0/2705733197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b1739390 0x55a1b1aff100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:36.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.129+0000 7f1e534b2740 1 --2- 172.21.15.117:0/2705733197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1b17311b0 0x55a1b1afebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:36.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.129+0000 7f1e534b2740 1 --2- 172.21.15.117:0/2705733197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a1b17189f0 0x55a1b1b0e950 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:36.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.129+0000 7f1e534b2740 1 -- 172.21.15.117:0/2705733197 >> 172.21.15.117:0/2705733197 conn(0x55a1b174e460 msgr2=0x55a1b1b0e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:36.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.129+0000 7f1e534b2740 1 -- 172.21.15.117:0/2705733197 shutdown_connections 2024-09-21T06:07:36.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.129+0000 7f1e534b2740 1 -- 172.21.15.117:0/2705733197 wait complete. 2024-09-21T06:07:36.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.129+0000 7f1e534b2740 1 -- start start 2024-09-21T06:07:36.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.129+0000 7f1e534b2740 1 Processor -- start 2024-09-21T06:07:36.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.130+0000 7f1e534b2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1b17189f0 0x55a1b1b20170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:36.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.130+0000 7f1e534b2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b17311b0 0x55a1b1b206b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:36.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.130+0000 7f1e534b2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a1b1739390 0x55a1b1b885d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:36.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.130+0000 7f1e534b2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1e481b0a00 con 0x55a1b1739390 2024-09-21T06:07:36.135 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.130+0000 7f1e534b2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1e481b0330 con 0x55a1b17311b0 2024-09-21T06:07:36.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.130+0000 7f1e534b2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1e481ad590 con 0x55a1b17189f0 2024-09-21T06:07:36.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.130+0000 7f1e534b2740 10 client.0 _ll_register_callbacks cb 0x7f1e3c0ea640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:36.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.130+0000 7f1e46d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a1b1739390 0x55a1b1b885d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:36.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.130+0000 7f1e46d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a1b1739390 0x55a1b1b885d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41798/0 (socket says 172.21.15.117:41798) 2024-09-21T06:07:36.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.130+0000 7f1e46d9d640 1 -- 172.21.15.117:0/2154564243 learned_addr learned my addr 172.21.15.117:0/2154564243 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:36.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.130+0000 7f1e4659c640 1 --2- 172.21.15.117:0/2154564243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1b17189f0 0x55a1b1b20170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:36.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.130+0000 7f1e45d9b640 1 --2- 172.21.15.117:0/2154564243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b17311b0 0x55a1b1b206b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:36.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.130+0000 7f1e46d9d640 1 -- 172.21.15.117:0/2154564243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1b17189f0 msgr2=0x55a1b1b20170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:36.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.130+0000 7f1e46d9d640 1 --2- 172.21.15.117:0/2154564243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1b17189f0 0x55a1b1b20170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:36.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.130+0000 7f1e46d9d640 1 -- 172.21.15.117:0/2154564243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b17311b0 msgr2=0x55a1b1b206b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:36.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.130+0000 7f1e46d9d640 1 --2- 172.21.15.117:0/2154564243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b17311b0 0x55a1b1b206b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:36.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.130+0000 7f1e46d9d640 1 -- 172.21.15.117:0/2154564243 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1b1b20f30 con 0x55a1b1739390 2024-09-21T06:07:36.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.130+0000 7f1e46d9d640 1 --2- 172.21.15.117:0/2154564243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a1b1739390 0x55a1b1b885d0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f1e40030720 tx=0x7f1e40009950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef26028f944ff36c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:36.136 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.130+0000 7f1e2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:36.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.130+0000 7f1e2dffb640 1 -- 172.21.15.117:0/2154564243 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e40065060 con 0x55a1b1739390 2024-09-21T06:07:36.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.130+0000 7f1e2dffb640 1 -- 172.21.15.117:0/2154564243 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f1e40031c10 con 0x55a1b1739390 2024-09-21T06:07:36.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.130+0000 7f1e534b2740 1 -- 172.21.15.117:0/2154564243 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e38026a70 con 0x55a1b1739390 2024-09-21T06:07:36.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.131+0000 7f1e2dffb640 1 -- 172.21.15.117:0/2154564243 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1e40051040 con 0x55a1b1739390 2024-09-21T06:07:36.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.131+0000 7f1e534b2740 10 client.17751 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:36.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.131+0000 7f1e534b2740 1 -- 172.21.15.117:0/2154564243 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a1b1b8ded0 con 0x55a1b1739390 2024-09-21T06:07:36.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.131+0000 7f1e2dffb640 1 -- 172.21.15.117:0/2154564243 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e38026a70 con 0x55a1b1739390 2024-09-21T06:07:36.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.131+0000 7f1e2dffb640 1 -- 172.21.15.117:0/2154564243 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1e4004c040 con 0x55a1b1739390 2024-09-21T06:07:36.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.131+0000 7f1e534b2740 10 client.17751 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:36.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.131+0000 7f1e534b2740 10 client.17751 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T06:07:36.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.131+0000 7f1e534b2740 20 client.17751 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:36.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.131+0000 7f1e534b2740 1 --2- 172.21.15.117:0/2154564243 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55a1b175a070 0x55a1b1b90660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:36.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.131+0000 7f1e534b2740 4 client.17751 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:07:36.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.131+0000 7f1e534b2740 1 -- 172.21.15.117:0/2154564243 --> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a1b1a758a0 con 0x55a1b175a070 2024-09-21T06:07:36.137 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.131+0000 7f1e4659c640 1 --2- 172.21.15.117:0/2154564243 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55a1b175a070 0x55a1b1b90660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:36.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.132+0000 7f1e2dffb640 10 client.17751 ms_handle_connect on v2:172.21.15.192:6834/3530688860 2024-09-21T06:07:36.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.132+0000 7f1e4659c640 1 --2- 172.21.15.117:0/2154564243 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55a1b175a070 0x55a1b1b90660 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5978fa40c508fc2a server_cookie=4c62230acc4dd57b in_seq=0 out_seq=0 2024-09-21T06:07:36.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.132+0000 7f1e2dffb640 1 -- 172.21.15.117:0/2154564243 <== mds.3 v2:172.21.15.192:6834/3530688860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a1b1a758a0 con 0x55a1b175a070 2024-09-21T06:07:36.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.132+0000 7f1e2dffb640 10 client.17751 handle_command_reply: tid=0 2024-09-21T06:07:36.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.132+0000 7f1e534b2740 1 client.17751 shutdown 2024-09-21T06:07:36.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.132+0000 7f1e534b2740 1 -- 172.21.15.117:0/2154564243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a1b1739390 msgr2=0x55a1b1b885d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:36.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.132+0000 7f1e534b2740 1 --2- 172.21.15.117:0/2154564243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a1b1739390 0x55a1b1b885d0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f1e40030720 tx=0x7f1e40009950 comp rx=0 tx=0).stop 2024-09-21T06:07:36.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.132+0000 7f1e534b2740 1 -- 172.21.15.117:0/2154564243 shutdown_connections 2024-09-21T06:07:36.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.132+0000 7f1e534b2740 1 --2- 172.21.15.117:0/2154564243 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55a1b175a070 0x55a1b1b90660 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:36.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.132+0000 7f1e2dffb640 0 client.17751 ms_handle_reset on v2:172.21.15.192:6834/3530688860 2024-09-21T06:07:36.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.132+0000 7f1e534b2740 1 --2- 172.21.15.117:0/2154564243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a1b1739390 0x55a1b1b885d0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:36.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.132+0000 7f1e534b2740 1 --2- 172.21.15.117:0/2154564243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a1b17311b0 0x55a1b1b206b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:36.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.132+0000 7f1e534b2740 1 --2- 172.21.15.117:0/2154564243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a1b17189f0 0x55a1b1b20170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:36.138 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.132+0000 7f1e534b2740 1 -- 172.21.15.117:0/2154564243 >> 172.21.15.117:0/2154564243 conn(0x55a1b174e460 msgr2=0x55a1b1b02680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:36.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.132+0000 7f1e534b2740 1 -- 172.21.15.117:0/2154564243 shutdown_connections 2024-09-21T06:07:36.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.132+0000 7f1e534b2740 1 -- 172.21.15.117:0/2154564243 wait complete. 2024-09-21T06:07:36.139 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.132+0000 7f1e534b2740 20 client.17751 trim_cache size 0 max 16384 2024-09-21T06:07:36.139 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:07:36.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.133+0000 7f1e4cf3e640 1 -- 172.21.15.117:0/100709896 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1e300b6730 msgr2=0x7f1e300b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:36.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.133+0000 7f1e4cf3e640 1 --2- 172.21.15.117:0/100709896 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1e300b6730 0x7f1e300b8c40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x55a1b1b00320 tx=0x7f1e3800c4b0 comp rx=0 tx=0).stop 2024-09-21T06:07:36.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.133+0000 7f1e4cf3e640 1 -- 172.21.15.117:0/100709896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e481094a0 msgr2=0x7f1e4819aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:36.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.133+0000 7f1e4cf3e640 1 --2- 172.21.15.117:0/100709896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e481094a0 0x7f1e4819aa40 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c006fd0 tx=0x7f1e3c00b8e0 comp rx=0 tx=0).stop 2024-09-21T06:07:36.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.133+0000 7f1e4cf3e640 1 -- 172.21.15.117:0/100709896 shutdown_connections 2024-09-21T06:07:36.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.133+0000 7f1e4cf3e640 1 --2- 172.21.15.117:0/100709896 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1e300b6730 0x7f1e300b8c40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:36.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.133+0000 7f1e4cf3e640 1 --2- 172.21.15.117:0/100709896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e481094a0 0x7f1e4819aa40 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:36.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.133+0000 7f1e4cf3e640 1 --2- 172.21.15.117:0/100709896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e48109df0 0x7f1e4819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:36.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.133+0000 7f1e4cf3e640 1 --2- 172.21.15.117:0/100709896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e48103b60 0x7f1e4819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:36.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.133+0000 7f1e4cf3e640 1 -- 172.21.15.117:0/100709896 >> 172.21.15.117:0/100709896 conn(0x7f1e4807b850 msgr2=0x7f1e481074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:36.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.133+0000 7f1e4cf3e640 1 -- 172.21.15.117:0/100709896 shutdown_connections 2024-09-21T06:07:36.140 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:36.133+0000 7f1e4cf3e640 1 -- 172.21.15.117:0/100709896 wait complete. 2024-09-21T06:07:36.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:35 smithi192 ceph-mon[34700]: pgmap v642: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 170 B/s wr, 14 op/s 2024-09-21T06:07:36.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:35 smithi138 ceph-mon[34391]: pgmap v642: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 170 B/s wr, 14 op/s 2024-09-21T06:07:36.911 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T06:07:37.158 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:07:37.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.635+0000 7f899be61640 1 -- 172.21.15.117:0/3344353110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8994105310 msgr2=0x7f899410d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:37.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.635+0000 7f899be61640 1 --2- 172.21.15.117:0/3344353110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8994105310 0x7f899410d860 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f8990009900 tx=0x7f899002a410 comp rx=0 tx=0).stop 2024-09-21T06:07:37.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.635+0000 7f899be61640 1 -- 172.21.15.117:0/3344353110 shutdown_connections 2024-09-21T06:07:37.643 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.635+0000 7f899be61640 1 --2- 172.21.15.117:0/3344353110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8994105310 0x7f899410d860 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.635+0000 7f899be61640 1 --2- 172.21.15.117:0/3344353110 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f899410de20 0x7f8994110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.635+0000 7f899be61640 1 --2- 172.21.15.117:0/3344353110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89941049c0 0x7f8994104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.635+0000 7f899be61640 1 -- 172.21.15.117:0/3344353110 >> 172.21.15.117:0/3344353110 conn(0x7f8994079710 msgr2=0x7f8994079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:37.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.635+0000 7f899be61640 1 -- 172.21.15.117:0/3344353110 shutdown_connections 2024-09-21T06:07:37.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.635+0000 7f899be61640 1 -- 172.21.15.117:0/3344353110 wait complete. 2024-09-21T06:07:37.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.636+0000 7f899be61640 1 Processor -- start 2024-09-21T06:07:37.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.636+0000 7f899be61640 1 -- start start 2024-09-21T06:07:37.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.636+0000 7f899be61640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89941049c0 0x7f899419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:37.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.636+0000 7f899be61640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8994105310 0x7f899419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:37.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.636+0000 7f899be61640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f899410de20 0x7f899419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:37.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.636+0000 7f899be61640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8994113a40 con 0x7f899410de20 2024-09-21T06:07:37.644 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.636+0000 7f899be61640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f89941138c0 con 0x7f89941049c0 2024-09-21T06:07:37.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.636+0000 7f899be61640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8994113bc0 con 0x7f8994105310 2024-09-21T06:07:37.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.636+0000 7f899a3fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f899410de20 0x7f899419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:37.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.636+0000 7f8999bfd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89941049c0 0x7f899419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:37.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.636+0000 7f8999bfd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89941049c0 0x7f899419a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:49534/0 (socket says 172.21.15.117:49534) 2024-09-21T06:07:37.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.636+0000 7f899a3fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f899410de20 0x7f899419fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41818/0 (socket says 172.21.15.117:41818) 2024-09-21T06:07:37.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.636+0000 7f8999bfd640 1 -- 172.21.15.117:0/4040343538 learned_addr learned my addr 172.21.15.117:0/4040343538 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:37.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.636+0000 7f89993fc640 1 --2- 172.21.15.117:0/4040343538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8994105310 0x7f899419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:37.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.636+0000 7f8999bfd640 1 -- 172.21.15.117:0/4040343538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8994105310 msgr2=0x7f899419aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:37.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.636+0000 7f8999bfd640 1 --2- 172.21.15.117:0/4040343538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8994105310 0x7f899419aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.636+0000 7f8999bfd640 1 -- 172.21.15.117:0/4040343538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f899410de20 msgr2=0x7f899419fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:37.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.636+0000 7f8999bfd640 1 --2- 172.21.15.117:0/4040343538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f899410de20 0x7f899419fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.636+0000 7f8999bfd640 1 -- 172.21.15.117:0/4040343538 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89941a0440 con 0x7f89941049c0 2024-09-21T06:07:37.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.637+0000 7f8999bfd640 1 --2- 172.21.15.117:0/4040343538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89941049c0 0x7f899419a4d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f898400e9f0 tx=0x7f898400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:37.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.637+0000 7f898affd640 1 -- 172.21.15.117:0/4040343538 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898400cd80 con 0x7f89941049c0 2024-09-21T06:07:37.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.637+0000 7f899be61640 1 -- 172.21.15.117:0/4040343538 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89941a0730 con 0x7f89941049c0 2024-09-21T06:07:37.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.637+0000 7f898affd640 1 -- 172.21.15.117:0/4040343538 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89840108e0 con 0x7f89941049c0 2024-09-21T06:07:37.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.637+0000 7f898affd640 1 -- 172.21.15.117:0/4040343538 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8984018980 con 0x7f89941049c0 2024-09-21T06:07:37.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.637+0000 7f899be61640 1 -- 172.21.15.117:0/4040343538 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89941025e0 con 0x7f89941049c0 2024-09-21T06:07:37.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.639+0000 7f89a23d5740 1 Processor -- start 2024-09-21T06:07:37.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.639+0000 7f89a23d5740 1 -- start start 2024-09-21T06:07:37.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.640+0000 7f89a23d5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e91348d070 0x55e9138342c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:37.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.640+0000 7f898affd640 1 -- 172.21.15.117:0/4040343538 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f8984010430 con 0x7f89941049c0 2024-09-21T06:07:37.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.640+0000 7f89a23d5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9134641b0 0x55e913834800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:37.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.640+0000 7f89a23d5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e91346c390 0x55e913834d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:37.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.640+0000 7f89a23d5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e9137aaea0 con 0x55e9134641b0 2024-09-21T06:07:37.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.640+0000 7f89a23d5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e91379a2d0 con 0x55e91348d070 2024-09-21T06:07:37.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.640+0000 7f89a23d5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55e91379e0e0 con 0x55e91346c390 2024-09-21T06:07:37.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.640+0000 7f899a3fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e91348d070 0x55e9138342c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:37.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.640+0000 7f8999bfd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9134641b0 0x55e913834800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:37.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.640+0000 7f899a3fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e91348d070 0x55e9138342c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:49540/0 (socket says 172.21.15.117:49540) 2024-09-21T06:07:37.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.640+0000 7f899a3fe640 1 -- 172.21.15.117:0/1584071085 learned_addr learned my addr 172.21.15.117:0/1584071085 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:37.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.640+0000 7f89993fc640 1 --2- 172.21.15.117:0/1584071085 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e91346c390 0x55e913834d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:37.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.640+0000 7f899a3fe640 1 -- 172.21.15.117:0/1584071085 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e91346c390 msgr2=0x55e913834d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:37.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.640+0000 7f899a3fe640 1 --2- 172.21.15.117:0/1584071085 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e91346c390 0x55e913834d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.640+0000 7f899a3fe640 1 -- 172.21.15.117:0/1584071085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9134641b0 msgr2=0x55e913834800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:37.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.640+0000 7f899a3fe640 1 --2- 172.21.15.117:0/1584071085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9134641b0 0x55e913834800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.640+0000 7f899a3fe640 1 -- 172.21.15.117:0/1584071085 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9134f8650 con 0x55e91348d070 2024-09-21T06:07:37.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.640+0000 7f899a3fe640 1 --2- 172.21.15.117:0/1584071085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e91348d070 0x55e9138342c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f898c009f50 tx=0x7f898c009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c93e0286e73b2ef0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:37.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.641+0000 7f897ffff640 1 -- 172.21.15.117:0/1584071085 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c003c30 con 0x55e91348d070 2024-09-21T06:07:37.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.641+0000 7f898affd640 1 --2- 172.21.15.117:0/4040343538 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f89800b6680 0x7f89800b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:37.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.641+0000 7f897ffff640 1 -- 172.21.15.117:0/1584071085 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f898c01f070 con 0x55e91348d070 2024-09-21T06:07:37.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.641+0000 7f897ffff640 1 -- 172.21.15.117:0/1584071085 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c00b8c0 con 0x55e91348d070 2024-09-21T06:07:37.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.641+0000 7f8999bfd640 1 --2- 172.21.15.117:0/4040343538 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f89800b6680 0x7f89800b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:37.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.642+0000 7f89a23d5740 1 -- 172.21.15.117:0/1584071085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e91348d070 msgr2=0x55e9138342c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:37.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.642+0000 7f89a23d5740 1 --2- 172.21.15.117:0/1584071085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e91348d070 0x55e9138342c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f898c009f50 tx=0x7f898c009500 comp rx=0 tx=0).stop 2024-09-21T06:07:37.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.642+0000 7f89a23d5740 1 -- 172.21.15.117:0/1584071085 shutdown_connections 2024-09-21T06:07:37.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.642+0000 7f89a23d5740 1 --2- 172.21.15.117:0/1584071085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9134641b0 0x55e913834800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.642+0000 7f89a23d5740 1 --2- 172.21.15.117:0/1584071085 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e91346c390 0x55e913834d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.642+0000 7f89a23d5740 1 --2- 172.21.15.117:0/1584071085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e91348d070 0x55e9138342c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.642+0000 7f89a23d5740 1 -- 172.21.15.117:0/1584071085 >> 172.21.15.117:0/1584071085 conn(0x55e913481460 msgr2=0x55e9138321b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:37.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.642+0000 7f898affd640 1 -- 172.21.15.117:0/4040343538 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f89840e7a00 con 0x7f89941049c0 2024-09-21T06:07:37.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.642+0000 7f8999bfd640 1 --2- 172.21.15.117:0/4040343538 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f89800b6680 0x7f89800b8b90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f89840e5250 tx=0x7f89840eb4c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:37.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.642+0000 7f89a23d5740 1 -- 172.21.15.117:0/1584071085 shutdown_connections 2024-09-21T06:07:37.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.642+0000 7f89a23d5740 1 -- 172.21.15.117:0/1584071085 wait complete. 2024-09-21T06:07:37.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.643+0000 7f89a23d5740 1 -- start start 2024-09-21T06:07:37.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.643+0000 7f89a23d5740 1 Processor -- start 2024-09-21T06:07:37.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.644+0000 7f89a23d5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9134641b0 0x55e91383f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:37.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.644+0000 7f89a23d5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e91346c390 0x55e91383fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:37.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.644+0000 7f89a23d5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e91348d070 0x55e913840400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:37.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.644+0000 7f89a23d5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e9135fb700 con 0x55e9134641b0 2024-09-21T06:07:37.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.644+0000 7f89a23d5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e913735160 con 0x55e91346c390 2024-09-21T06:07:37.652 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.644+0000 7f89a23d5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55e913842aa0 con 0x55e91348d070 2024-09-21T06:07:37.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.644+0000 7f89a23d5740 10 client.0 _ll_register_callbacks cb 0x55e91374e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:37.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.644+0000 7f899a3fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9134641b0 0x55e91383f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:37.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.644+0000 7f899a3fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9134641b0 0x55e91383f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41830/0 (socket says 172.21.15.117:41830) 2024-09-21T06:07:37.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.644+0000 7f899a3fe640 1 -- 172.21.15.117:0/2635351207 learned_addr learned my addr 172.21.15.117:0/2635351207 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:37.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.644+0000 7f89993fc640 1 --2- 172.21.15.117:0/2635351207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e91346c390 0x55e91383fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:37.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.644+0000 7f899a3fe640 1 --2- 172.21.15.117:0/2635351207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e91348d070 0x55e913840400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:37.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.644+0000 7f89993fc640 1 -- 172.21.15.117:0/2635351207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e91348d070 msgr2=0x55e913840400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:37.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.644+0000 7f89993fc640 1 --2- 172.21.15.117:0/2635351207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e91348d070 0x55e913840400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.644+0000 7f89993fc640 1 -- 172.21.15.117:0/2635351207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9134641b0 msgr2=0x55e91383f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:37.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.644+0000 7f89993fc640 1 --2- 172.21.15.117:0/2635351207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9134641b0 0x55e91383f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.644+0000 7f89993fc640 1 -- 172.21.15.117:0/2635351207 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e913762c90 con 0x55e91346c390 2024-09-21T06:07:37.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.644+0000 7f899a3fe640 1 --2- 172.21.15.117:0/2635351207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9134641b0 0x55e91383f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:07:37.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.645+0000 7f89993fc640 1 --2- 172.21.15.117:0/2635351207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e91346c390 0x55e91383fec0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f899002a930 tx=0x7f8990032dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24a4b614e0c9c84f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:37.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.645+0000 7f897d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-21T06:07:37.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.645+0000 7f897d7fa640 1 -- 172.21.15.117:0/2635351207 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8990035040 con 0x55e91346c390 2024-09-21T06:07:37.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.645+0000 7f897d7fa640 1 -- 172.21.15.117:0/2635351207 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f89900316a0 con 0x55e91346c390 2024-09-21T06:07:37.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.645+0000 7f89a23d5740 1 -- 172.21.15.117:0/2635351207 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f898c01f070 con 0x55e91346c390 2024-09-21T06:07:37.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.645+0000 7f897d7fa640 1 -- 172.21.15.117:0/2635351207 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f89900318a0 con 0x55e91346c390 2024-09-21T06:07:37.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.645+0000 7f89a23d5740 10 client.26717 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:37.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.645+0000 7f89a23d5740 1 -- 172.21.15.117:0/2635351207 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e91384e6d0 con 0x55e91346c390 2024-09-21T06:07:37.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.645+0000 7f897d7fa640 1 -- 172.21.15.117:0/2635351207 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f898c01f070 con 0x55e91346c390 2024-09-21T06:07:37.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.645+0000 7f897d7fa640 1 -- 172.21.15.117:0/2635351207 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8990004750 con 0x55e91346c390 2024-09-21T06:07:37.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.645+0000 7f89a23d5740 10 client.26717 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:37.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.645+0000 7f89a23d5740 10 client.26717 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:07:37.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.646+0000 7f89a23d5740 20 client.26717 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:37.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.646+0000 7f89a23d5740 1 --2- 172.21.15.117:0/2635351207 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55e91344b9f0 0x55e913851b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:37.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.646+0000 7f89a23d5740 4 client.26717 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:07:37.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.646+0000 7f89a23d5740 1 -- 172.21.15.117:0/2635351207 --> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9137acdf0 con 0x55e91344b9f0 2024-09-21T06:07:37.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.646+0000 7f899a3fe640 1 --2- 172.21.15.117:0/2635351207 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55e91344b9f0 0x55e913851b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:37.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.646+0000 7f897d7fa640 10 client.26717 ms_handle_connect on v2:172.21.15.192:6836/2683860842 2024-09-21T06:07:37.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.646+0000 7f899a3fe640 1 --2- 172.21.15.117:0/2635351207 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55e91344b9f0 0x55e913851b70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=df89e6a42ee64ed6 server_cookie=8d158b55ce89a51f in_seq=0 out_seq=0 2024-09-21T06:07:37.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.647+0000 7f897d7fa640 1 -- 172.21.15.117:0/2635351207 <== mds.2 v2:172.21.15.192:6836/2683860842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e9137acdf0 con 0x55e91344b9f0 2024-09-21T06:07:37.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.647+0000 7f897d7fa640 10 client.26717 handle_command_reply: tid=0 2024-09-21T06:07:37.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.647+0000 7f89a23d5740 1 client.26717 shutdown 2024-09-21T06:07:37.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.647+0000 7f89a23d5740 1 -- 172.21.15.117:0/2635351207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e91346c390 msgr2=0x55e91383fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:37.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.647+0000 7f89a23d5740 1 --2- 172.21.15.117:0/2635351207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e91346c390 0x55e91383fec0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f899002a930 tx=0x7f8990032dc0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.647+0000 7f89a23d5740 1 -- 172.21.15.117:0/2635351207 shutdown_connections 2024-09-21T06:07:37.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.647+0000 7f89a23d5740 1 --2- 172.21.15.117:0/2635351207 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55e91344b9f0 0x55e913851b70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.647+0000 7f89a23d5740 1 --2- 172.21.15.117:0/2635351207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e91348d070 0x55e913840400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.647+0000 7f89a23d5740 1 --2- 172.21.15.117:0/2635351207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e91346c390 0x55e91383fec0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.647+0000 7f897d7fa640 0 client.26717 ms_handle_reset on v2:172.21.15.192:6836/2683860842 2024-09-21T06:07:37.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.647+0000 7f89a23d5740 1 --2- 172.21.15.117:0/2635351207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9134641b0 0x55e91383f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.647+0000 7f89a23d5740 1 -- 172.21.15.117:0/2635351207 >> 172.21.15.117:0/2635351207 conn(0x55e913481460 msgr2=0x55e913837470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:37.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.647+0000 7f89a23d5740 1 -- 172.21.15.117:0/2635351207 shutdown_connections 2024-09-21T06:07:37.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.647+0000 7f89a23d5740 1 -- 172.21.15.117:0/2635351207 wait complete. 2024-09-21T06:07:37.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.647+0000 7f89a23d5740 20 client.26717 trim_cache size 0 max 16384 2024-09-21T06:07:37.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.661+0000 7f89a23d5740 1 Processor -- start 2024-09-21T06:07:37.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.661+0000 7f89a23d5740 1 -- start start 2024-09-21T06:07:37.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.661+0000 7f89a23d5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e91344b9f0 0x55e913835dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:37.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.661+0000 7f89a23d5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9134641b0 0x55e913841860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:37.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.661+0000 7f89a23d5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e91346c390 0x55e91384b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:37.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.661+0000 7f89a23d5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f89941ad510 con 0x55e9134641b0 2024-09-21T06:07:37.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.661+0000 7f89a23d5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f89941af050 con 0x55e91344b9f0 2024-09-21T06:07:37.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.661+0000 7f89a23d5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f89941adbe0 con 0x55e91346c390 2024-09-21T06:07:37.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.661+0000 7f89993fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9134641b0 0x55e913841860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:37.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.661+0000 7f899a3fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e91344b9f0 0x55e913835dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:37.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.661+0000 7f899a3fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e91344b9f0 0x55e913835dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:49550/0 (socket says 172.21.15.117:49550) 2024-09-21T06:07:37.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.661+0000 7f89993fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9134641b0 0x55e913841860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41842/0 (socket says 172.21.15.117:41842) 2024-09-21T06:07:37.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.661+0000 7f899a3fe640 1 -- 172.21.15.117:0/1230652982 learned_addr learned my addr 172.21.15.117:0/1230652982 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:37.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.662+0000 7f899a3fe640 1 --2- 172.21.15.117:0/1230652982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e91346c390 0x55e91384b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:37.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.662+0000 7f89993fc640 1 -- 172.21.15.117:0/1230652982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e91346c390 msgr2=0x55e91384b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:37.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.662+0000 7f89993fc640 1 --2- 172.21.15.117:0/1230652982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e91346c390 0x55e91384b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.662+0000 7f89993fc640 1 -- 172.21.15.117:0/1230652982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e91344b9f0 msgr2=0x55e913835dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:37.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.662+0000 7f89993fc640 1 --2- 172.21.15.117:0/1230652982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e91344b9f0 0x55e913835dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.662+0000 7f89993fc640 1 -- 172.21.15.117:0/1230652982 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9138424b0 con 0x55e9134641b0 2024-09-21T06:07:37.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.662+0000 7f899a3fe640 1 --2- 172.21.15.117:0/1230652982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e91344b9f0 0x55e913835dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:07:37.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.662+0000 7f89993fc640 1 --2- 172.21.15.117:0/1230652982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9134641b0 0x55e913841860 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f8990009900 tx=0x7f8990032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ef26b315f02a90 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:37.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.662+0000 7f897f7fe640 1 -- 172.21.15.117:0/1230652982 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8990035070 con 0x55e9134641b0 2024-09-21T06:07:37.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.662+0000 7f897f7fe640 1 -- 172.21.15.117:0/1230652982 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8990053070 con 0x55e9134641b0 2024-09-21T06:07:37.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.662+0000 7f897f7fe640 1 -- 172.21.15.117:0/1230652982 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8990000e60 con 0x55e9134641b0 2024-09-21T06:07:37.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.663+0000 7f89a23d5740 1 -- 172.21.15.117:0/1230652982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9134641b0 msgr2=0x55e913841860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:37.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.663+0000 7f89a23d5740 1 --2- 172.21.15.117:0/1230652982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9134641b0 0x55e913841860 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f8990009900 tx=0x7f8990032b20 comp rx=0 tx=0).stop 2024-09-21T06:07:37.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.663+0000 7f89a23d5740 1 -- 172.21.15.117:0/1230652982 shutdown_connections 2024-09-21T06:07:37.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.663+0000 7f89a23d5740 1 --2- 172.21.15.117:0/1230652982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9134641b0 0x55e913841860 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.663+0000 7f89a23d5740 1 --2- 172.21.15.117:0/1230652982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e91346c390 0x55e91384b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.663+0000 7f89a23d5740 1 --2- 172.21.15.117:0/1230652982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e91344b9f0 0x55e913835dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.663+0000 7f89a23d5740 1 -- 172.21.15.117:0/1230652982 >> 172.21.15.117:0/1230652982 conn(0x55e913481460 msgr2=0x55e91384ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:37.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.663+0000 7f89a23d5740 1 -- 172.21.15.117:0/1230652982 shutdown_connections 2024-09-21T06:07:37.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.663+0000 7f89a23d5740 1 -- 172.21.15.117:0/1230652982 wait complete. 2024-09-21T06:07:37.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.664+0000 7f89a23d5740 1 -- start start 2024-09-21T06:07:37.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.664+0000 7f89a23d5740 1 Processor -- start 2024-09-21T06:07:37.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.664+0000 7f89a23d5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e91344b9f0 0x55e913844370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:37.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.664+0000 7f89a23d5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9134641b0 0x55e9138448b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:37.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.664+0000 7f89a23d5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e91346c390 0x55e9138bb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:37.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.664+0000 7f89a23d5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f89941ae980 con 0x55e9134641b0 2024-09-21T06:07:37.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.664+0000 7f89a23d5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f89941ae2b0 con 0x55e91344b9f0 2024-09-21T06:07:37.669 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.664+0000 7f89a23d5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f89941ab530 con 0x55e91346c390 2024-09-21T06:07:37.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.664+0000 7f89a23d5740 10 client.0 _ll_register_callbacks cb 0x7f898c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:37.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.664+0000 7f89993fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9134641b0 0x55e9138448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:37.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.664+0000 7f899a3fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e91344b9f0 0x55e913844370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:37.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.664+0000 7f89993fc640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9134641b0 0x55e9138448b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41846/0 (socket says 172.21.15.117:41846) 2024-09-21T06:07:37.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.664+0000 7f89993fc640 1 -- 172.21.15.117:0/508061541 learned_addr learned my addr 172.21.15.117:0/508061541 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:37.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.665+0000 7f899a3fe640 1 --2- 172.21.15.117:0/508061541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e91346c390 0x55e9138bb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:37.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.665+0000 7f899a3fe640 1 -- 172.21.15.117:0/508061541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e91346c390 msgr2=0x55e9138bb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:37.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.665+0000 7f899a3fe640 1 --2- 172.21.15.117:0/508061541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e91346c390 0x55e9138bb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.665+0000 7f899a3fe640 1 -- 172.21.15.117:0/508061541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9134641b0 msgr2=0x55e9138448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:37.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.665+0000 7f899a3fe640 1 --2- 172.21.15.117:0/508061541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9134641b0 0x55e9138448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.670 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.665+0000 7f899a3fe640 1 -- 172.21.15.117:0/508061541 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9138368a0 con 0x55e91344b9f0 2024-09-21T06:07:37.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.665+0000 7f899a3fe640 1 --2- 172.21.15.117:0/508061541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e91344b9f0 0x55e913844370 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f898c00b5f0 tx=0x7f898c029040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0e29786950d699a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:37.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.665+0000 7f897d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-21T06:07:37.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.665+0000 7f897d7fa640 1 -- 172.21.15.117:0/508061541 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c031040 con 0x55e91344b9f0 2024-09-21T06:07:37.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.666+0000 7f89a23d5740 1 -- 172.21.15.117:0/508061541 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8990053070 con 0x55e91344b9f0 2024-09-21T06:07:37.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.666+0000 7f897d7fa640 1 -- 172.21.15.117:0/508061541 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f898c027900 con 0x55e91344b9f0 2024-09-21T06:07:37.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.666+0000 7f897d7fa640 1 -- 172.21.15.117:0/508061541 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f898c004b40 con 0x55e91344b9f0 2024-09-21T06:07:37.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.666+0000 7f89a23d5740 10 client.26723 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:37.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.666+0000 7f89a23d5740 1 -- 172.21.15.117:0/508061541 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e9138bca80 con 0x55e91344b9f0 2024-09-21T06:07:37.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.666+0000 7f897d7fa640 1 -- 172.21.15.117:0/508061541 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8990053070 con 0x55e91344b9f0 2024-09-21T06:07:37.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.666+0000 7f897d7fa640 1 -- 172.21.15.117:0/508061541 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f898c01a890 con 0x55e91344b9f0 2024-09-21T06:07:37.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.666+0000 7f89a23d5740 10 client.26723 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:37.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.666+0000 7f89a23d5740 10 client.26723 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:07:37.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.666+0000 7f89a23d5740 20 client.26723 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:37.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.666+0000 7f89a23d5740 1 --2- 172.21.15.117:0/508061541 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55e91348d070 0x55e9138bf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:37.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.666+0000 7f89a23d5740 4 client.26723 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:07:37.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.666+0000 7f89a23d5740 1 -- 172.21.15.117:0/508061541 --> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e9137a88a0 con 0x55e91348d070 2024-09-21T06:07:37.672 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.667+0000 7f89993fc640 1 --2- 172.21.15.117:0/508061541 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55e91348d070 0x55e9138bf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:37.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.667+0000 7f897d7fa640 10 client.26723 ms_handle_connect on v2:172.21.15.192:6836/2683860842 2024-09-21T06:07:37.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.667+0000 7f89993fc640 1 --2- 172.21.15.117:0/508061541 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55e91348d070 0x55e9138bf0b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=29837316c3e475e2 server_cookie=a8785cadca3a4ca6 in_seq=0 out_seq=0 2024-09-21T06:07:37.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.668+0000 7f897d7fa640 1 -- 172.21.15.117:0/508061541 <== mds.2 v2:172.21.15.192:6836/2683860842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e9137a88a0 con 0x55e91348d070 2024-09-21T06:07:37.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.668+0000 7f897d7fa640 10 client.26723 handle_command_reply: tid=0 2024-09-21T06:07:37.674 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.668+0000 7f89a23d5740 1 client.26723 shutdown 2024-09-21T06:07:37.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.668+0000 7f89a23d5740 1 -- 172.21.15.117:0/508061541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e91344b9f0 msgr2=0x55e913844370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:37.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.668+0000 7f89a23d5740 1 --2- 172.21.15.117:0/508061541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e91344b9f0 0x55e913844370 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f898c00b5f0 tx=0x7f898c029040 comp rx=0 tx=0).stop 2024-09-21T06:07:37.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.668+0000 7f89a23d5740 1 -- 172.21.15.117:0/508061541 shutdown_connections 2024-09-21T06:07:37.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.668+0000 7f89a23d5740 1 --2- 172.21.15.117:0/508061541 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55e91348d070 0x55e9138bf0b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.668+0000 7f89a23d5740 1 --2- 172.21.15.117:0/508061541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e9134641b0 0x55e9138448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.668+0000 7f89a23d5740 1 --2- 172.21.15.117:0/508061541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e91346c390 0x55e9138bb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.668+0000 7f89a23d5740 1 --2- 172.21.15.117:0/508061541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e91344b9f0 0x55e913844370 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.668+0000 7f89a23d5740 1 -- 172.21.15.117:0/508061541 >> 172.21.15.117:0/508061541 conn(0x55e913481460 msgr2=0x55e91384ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:37.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.668+0000 7f897d7fa640 0 client.26723 ms_handle_reset on v2:172.21.15.192:6836/2683860842 2024-09-21T06:07:37.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.668+0000 7f89a23d5740 1 -- 172.21.15.117:0/508061541 shutdown_connections 2024-09-21T06:07:37.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.668+0000 7f89a23d5740 1 -- 172.21.15.117:0/508061541 wait complete. 2024-09-21T06:07:37.675 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.668+0000 7f89a23d5740 20 client.26723 trim_cache size 0 max 16384 2024-09-21T06:07:37.676 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:07:37.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.669+0000 7f899be61640 1 -- 172.21.15.117:0/4040343538 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f89800b6680 msgr2=0x7f89800b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:37.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.669+0000 7f899be61640 1 --2- 172.21.15.117:0/4040343538 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f89800b6680 0x7f89800b8b90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f89840e5250 tx=0x7f89840eb4c0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.669+0000 7f899be61640 1 -- 172.21.15.117:0/4040343538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89941049c0 msgr2=0x7f899419a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:37.676 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.669+0000 7f899be61640 1 --2- 172.21.15.117:0/4040343538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89941049c0 0x7f899419a4d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f898400e9f0 tx=0x7f898400eeb0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.669+0000 7f899be61640 1 -- 172.21.15.117:0/4040343538 shutdown_connections 2024-09-21T06:07:37.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.669+0000 7f899be61640 1 --2- 172.21.15.117:0/4040343538 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f89800b6680 0x7f89800b8b90 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.669+0000 7f899be61640 1 --2- 172.21.15.117:0/4040343538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f899410de20 0x7f899419fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.669+0000 7f899be61640 1 --2- 172.21.15.117:0/4040343538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8994105310 0x7f899419aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.669+0000 7f899be61640 1 --2- 172.21.15.117:0/4040343538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89941049c0 0x7f899419a4d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:37.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.669+0000 7f899be61640 1 -- 172.21.15.117:0/4040343538 >> 172.21.15.117:0/4040343538 conn(0x7f8994079710 msgr2=0x7f89941068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:37.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.669+0000 7f899be61640 1 -- 172.21.15.117:0/4040343538 shutdown_connections 2024-09-21T06:07:37.677 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:37.669+0000 7f899be61640 1 -- 172.21.15.117:0/4040343538 wait complete. 2024-09-21T06:07:38.217 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:37 smithi192 ceph-mon[34700]: pgmap v643: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-21T06:07:38.218 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:37 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:38.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:37 smithi117 ceph-mon[30947]: pgmap v643: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-21T06:07:38.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:37 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:38.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:37 smithi138 ceph-mon[34391]: pgmap v643: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-21T06:07:38.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:37 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:07:38.370 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T06:07:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.508+0000 7f399b67e640 1 -- 172.21.15.117:0/412958883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3994108400 msgr2=0x7f399410ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.508+0000 7f399b67e640 1 --2- 172.21.15.117:0/412958883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3994108400 0x7f399410ec30 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f398c009be0 tx=0x7f398c02a6c0 comp rx=0 tx=0).stop 2024-09-21T06:07:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.508+0000 7f399b67e640 1 -- 172.21.15.117:0/412958883 shutdown_connections 2024-09-21T06:07:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.508+0000 7f399b67e640 1 --2- 172.21.15.117:0/412958883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3994108400 0x7f399410ec30 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.508+0000 7f399b67e640 1 --2- 172.21.15.117:0/412958883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3994107ab0 0x7f3994107ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.508+0000 7f399b67e640 1 --2- 172.21.15.117:0/412958883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3994102180 0x7f3994102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:38.514 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.508+0000 7f399b67e640 1 -- 172.21.15.117:0/412958883 >> 172.21.15.117:0/412958883 conn(0x7f3994079e90 msgr2=0x7f399407a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:38.514 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.508+0000 7f399b67e640 1 -- 172.21.15.117:0/412958883 shutdown_connections 2024-09-21T06:07:38.514 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.508+0000 7f399b67e640 1 -- 172.21.15.117:0/412958883 wait complete. 2024-09-21T06:07:38.514 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.509+0000 7f399b67e640 1 Processor -- start 2024-09-21T06:07:38.514 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.509+0000 7f399b67e640 1 -- start start 2024-09-21T06:07:38.514 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.509+0000 7f399b67e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3994102180 0x7f399419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:38.514 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.509+0000 7f399b67e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3994107ab0 0x7f399419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:38.514 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.509+0000 7f399b67e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3994108400 0x7f399419fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.509+0000 7f399b67e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3994111840 con 0x7f3994108400 2024-09-21T06:07:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.509+0000 7f399b67e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f39941116c0 con 0x7f3994107ab0 2024-09-21T06:07:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.509+0000 7f399b67e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f39941119c0 con 0x7f3994102180 2024-09-21T06:07:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.509+0000 7f3999c1b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3994108400 0x7f399419fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.509+0000 7f399941a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3994102180 0x7f399419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.509+0000 7f3999c1b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3994108400 0x7f399419fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41870/0 (socket says 172.21.15.117:41870) 2024-09-21T06:07:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.509+0000 7f3999c1b640 1 -- 172.21.15.117:0/2237824278 learned_addr learned my addr 172.21.15.117:0/2237824278 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.509+0000 7f3998c19640 1 --2- 172.21.15.117:0/2237824278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3994107ab0 0x7f399419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.510+0000 7f3999c1b640 1 -- 172.21.15.117:0/2237824278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3994102180 msgr2=0x7f399419a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.510+0000 7f3999c1b640 1 --2- 172.21.15.117:0/2237824278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3994102180 0x7f399419a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:38.515 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.510+0000 7f3999c1b640 1 -- 172.21.15.117:0/2237824278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3994107ab0 msgr2=0x7f399419a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:38.516 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.510+0000 7f3999c1b640 1 --2- 172.21.15.117:0/2237824278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3994107ab0 0x7f399419a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:38.516 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.510+0000 7f3999c1b640 1 -- 172.21.15.117:0/2237824278 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39941a0380 con 0x7f3994108400 2024-09-21T06:07:38.516 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.510+0000 7f399941a640 1 --2- 172.21.15.117:0/2237824278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3994102180 0x7f399419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:07:38.516 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.510+0000 7f3998c19640 1 --2- 172.21.15.117:0/2237824278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3994107ab0 0x7f399419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:07:38.516 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.510+0000 7f3999c1b640 1 --2- 172.21.15.117:0/2237824278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3994108400 0x7f399419fc80 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f398c004af0 tx=0x7f398c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:38.516 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.510+0000 7f398a7fc640 1 -- 172.21.15.117:0/2237824278 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f398c041020 con 0x7f3994108400 2024-09-21T06:07:38.516 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.510+0000 7f399b67e640 1 -- 172.21.15.117:0/2237824278 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39941a0610 con 0x7f3994108400 2024-09-21T06:07:38.516 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.510+0000 7f398a7fc640 1 -- 172.21.15.117:0/2237824278 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f398c003e10 con 0x7f3994108400 2024-09-21T06:07:38.517 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.510+0000 7f398a7fc640 1 -- 172.21.15.117:0/2237824278 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f398c036040 con 0x7f3994108400 2024-09-21T06:07:38.517 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.510+0000 7f399b67e640 1 -- 172.21.15.117:0/2237824278 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39941a6e00 con 0x7f3994108400 2024-09-21T06:07:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.513+0000 7f39a25ec740 1 Processor -- start 2024-09-21T06:07:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.514+0000 7f398a7fc640 1 -- 172.21.15.117:0/2237824278 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f398c044020 con 0x7f3994108400 2024-09-21T06:07:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.514+0000 7f39a25ec740 1 -- start start 2024-09-21T06:07:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.514+0000 7f39a25ec740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582b10ab1b0 0x5582b14509b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.514+0000 7f39a25ec740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b10822f0 0x5582b1450ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.514+0000 7f39a25ec740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b108a4d0 0x5582b1451430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.514+0000 7f39a25ec740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5582b13c56f0 con 0x5582b10822f0 2024-09-21T06:07:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.514+0000 7f39a25ec740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5582b13b4b20 con 0x5582b108a4d0 2024-09-21T06:07:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.514+0000 7f39a25ec740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5582b13b8930 con 0x5582b10ab1b0 2024-09-21T06:07:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.514+0000 7f3998c19640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582b10ab1b0 0x5582b14509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.514+0000 7f399941a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b10822f0 0x5582b1450ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.514+0000 7f3998c19640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582b10ab1b0 0x5582b14509b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:38468/0 (socket says 172.21.15.117:38468) 2024-09-21T06:07:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.514+0000 7f3998c19640 1 -- 172.21.15.117:0/3630853384 learned_addr learned my addr 172.21.15.117:0/3630853384 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.514+0000 7f3998c19640 1 --2- 172.21.15.117:0/3630853384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b108a4d0 0x5582b1451430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.515+0000 7f3998c19640 1 -- 172.21.15.117:0/3630853384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b108a4d0 msgr2=0x5582b1451430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.515+0000 7f3998c19640 1 --2- 172.21.15.117:0/3630853384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b108a4d0 0x5582b1451430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.515+0000 7f398a7fc640 1 --2- 172.21.15.117:0/2237824278 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f39800b6680 0x7f39800b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.515+0000 7f3998c19640 1 -- 172.21.15.117:0/3630853384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b10822f0 msgr2=0x5582b1450ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.515+0000 7f3998c19640 1 --2- 172.21.15.117:0/3630853384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b10822f0 0x5582b1450ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.515+0000 7f3998c19640 1 -- 172.21.15.117:0/3630853384 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582b137a530 con 0x5582b10ab1b0 2024-09-21T06:07:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.515+0000 7f3999c1b640 1 --2- 172.21.15.117:0/2237824278 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f39800b6680 0x7f39800b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.515+0000 7f3998c19640 1 --2- 172.21.15.117:0/3630853384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582b10ab1b0 0x5582b14509b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f398400ed30 tx=0x7f3984005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=109cf07e9c6e92c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.515+0000 7f398a7fc640 1 -- 172.21.15.117:0/2237824278 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f398c108220 con 0x7f3994108400 2024-09-21T06:07:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.515+0000 7f397f7fe640 1 -- 172.21.15.117:0/3630853384 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3984017040 con 0x5582b10ab1b0 2024-09-21T06:07:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.515+0000 7f397f7fe640 1 -- 172.21.15.117:0/3630853384 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f398400ee20 con 0x5582b10ab1b0 2024-09-21T06:07:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.515+0000 7f397f7fe640 1 -- 172.21.15.117:0/3630853384 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3984005530 con 0x5582b10ab1b0 2024-09-21T06:07:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.516+0000 7f39a25ec740 1 -- 172.21.15.117:0/3630853384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582b10ab1b0 msgr2=0x5582b14509b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.516+0000 7f39a25ec740 1 --2- 172.21.15.117:0/3630853384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582b10ab1b0 0x5582b14509b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f398400ed30 tx=0x7f3984005850 comp rx=0 tx=0).stop 2024-09-21T06:07:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.516+0000 7f3999c1b640 1 --2- 172.21.15.117:0/2237824278 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f39800b6680 0x7f39800b8b90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f398c0ce050 tx=0x7f398c030970 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.516+0000 7f39a25ec740 1 -- 172.21.15.117:0/3630853384 shutdown_connections 2024-09-21T06:07:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.516+0000 7f39a25ec740 1 --2- 172.21.15.117:0/3630853384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b10822f0 0x5582b1450ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.516+0000 7f39a25ec740 1 --2- 172.21.15.117:0/3630853384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b108a4d0 0x5582b1451430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.516+0000 7f39a25ec740 1 --2- 172.21.15.117:0/3630853384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582b10ab1b0 0x5582b14509b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.516+0000 7f39a25ec740 1 -- 172.21.15.117:0/3630853384 >> 172.21.15.117:0/3630853384 conn(0x5582b109f300 msgr2=0x5582b144e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.516+0000 7f39a25ec740 1 -- 172.21.15.117:0/3630853384 shutdown_connections 2024-09-21T06:07:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.516+0000 7f39a25ec740 1 -- 172.21.15.117:0/3630853384 wait complete. 2024-09-21T06:07:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.516+0000 7f39a25ec740 1 -- start start 2024-09-21T06:07:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.517+0000 7f39a25ec740 1 Processor -- start 2024-09-21T06:07:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.517+0000 7f39a25ec740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b10822f0 0x5582b145c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.517+0000 7f39a25ec740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b108a4d0 0x5582b145c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.517+0000 7f39a25ec740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582b10ab1b0 0x5582b14683c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.517+0000 7f39a25ec740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5582b12ecab0 con 0x5582b108a4d0 2024-09-21T06:07:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.517+0000 7f39a25ec740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5582b135a730 con 0x5582b10822f0 2024-09-21T06:07:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.517+0000 7f39a25ec740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5582b145f0d0 con 0x5582b10ab1b0 2024-09-21T06:07:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.517+0000 7f39a25ec740 10 client.0 _ll_register_callbacks cb 0x5582b13ae6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.517+0000 7f399941a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b108a4d0 0x5582b145c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.517+0000 7f399941a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b108a4d0 0x5582b145c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41894/0 (socket says 172.21.15.117:41894) 2024-09-21T06:07:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.517+0000 7f399941a640 1 -- 172.21.15.117:0/1655730186 learned_addr learned my addr 172.21.15.117:0/1655730186 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.517+0000 7f3998c19640 1 --2- 172.21.15.117:0/1655730186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b10822f0 0x5582b145c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.517+0000 7f3998c19640 1 --2- 172.21.15.117:0/1655730186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582b10ab1b0 0x5582b14683c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.518+0000 7f399941a640 1 -- 172.21.15.117:0/1655730186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582b10ab1b0 msgr2=0x5582b14683c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.518+0000 7f399941a640 1 --2- 172.21.15.117:0/1655730186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582b10ab1b0 0x5582b14683c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.518+0000 7f399941a640 1 -- 172.21.15.117:0/1655730186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b10822f0 msgr2=0x5582b145c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.518+0000 7f399941a640 1 --2- 172.21.15.117:0/1655730186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b10822f0 0x5582b145c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.518+0000 7f399941a640 1 -- 172.21.15.117:0/1655730186 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582b1116790 con 0x5582b108a4d0 2024-09-21T06:07:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.518+0000 7f3998c19640 1 --2- 172.21.15.117:0/1655730186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582b10ab1b0 0x5582b14683c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.518+0000 7f399941a640 1 --2- 172.21.15.117:0/1655730186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b108a4d0 0x5582b145c5d0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f3990012d20 tx=0x7f399000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef707f79c81140 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.518+0000 7f397cff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.518+0000 7f397cff9640 1 -- 172.21.15.117:0/1655730186 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39900120d0 con 0x5582b108a4d0 2024-09-21T06:07:38.530 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.518+0000 7f397cff9640 1 -- 172.21.15.117:0/1655730186 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f3990004020 con 0x5582b108a4d0 2024-09-21T06:07:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.518+0000 7f39a25ec740 1 -- 172.21.15.117:0/1655730186 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f398400ee20 con 0x5582b108a4d0 2024-09-21T06:07:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.518+0000 7f397cff9640 1 -- 172.21.15.117:0/1655730186 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3990012270 con 0x5582b108a4d0 2024-09-21T06:07:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.518+0000 7f39a25ec740 10 client.17781 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.518+0000 7f39a25ec740 1 -- 172.21.15.117:0/1655730186 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5582b118f620 con 0x5582b108a4d0 2024-09-21T06:07:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.518+0000 7f397cff9640 1 -- 172.21.15.117:0/1655730186 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f398400ee20 con 0x5582b108a4d0 2024-09-21T06:07:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.518+0000 7f397cff9640 1 -- 172.21.15.117:0/1655730186 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3990026480 con 0x5582b108a4d0 2024-09-21T06:07:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.519+0000 7f39a25ec740 10 client.17781 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.519+0000 7f39a25ec740 10 client.17781 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:07:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.519+0000 7f39a25ec740 20 client.17781 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.519+0000 7f39a25ec740 1 --2- 172.21.15.117:0/1655730186 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5582b1069d80 0x5582b146e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.519+0000 7f39a25ec740 4 client.17781 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:07:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.519+0000 7f39a25ec740 1 -- 172.21.15.117:0/1655730186 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582b13c7640 con 0x5582b1069d80 2024-09-21T06:07:38.531 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.519+0000 7f3998c19640 1 --2- 172.21.15.117:0/1655730186 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5582b1069d80 0x5582b146e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.519+0000 7f397cff9640 10 client.17781 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.519+0000 7f3998c19640 1 --2- 172.21.15.117:0/1655730186 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5582b1069d80 0x5582b146e120 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea05418d5613dbf0 server_cookie=26db20195cc4bd61 in_seq=0 out_seq=0 2024-09-21T06:07:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.520+0000 7f397cff9640 1 -- 172.21.15.117:0/1655730186 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5582b13c7640 con 0x5582b1069d80 2024-09-21T06:07:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.520+0000 7f397cff9640 10 client.17781 handle_command_reply: tid=0 2024-09-21T06:07:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.520+0000 7f39a25ec740 1 client.17781 shutdown 2024-09-21T06:07:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.520+0000 7f39a25ec740 1 -- 172.21.15.117:0/1655730186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b108a4d0 msgr2=0x5582b145c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.520+0000 7f39a25ec740 1 --2- 172.21.15.117:0/1655730186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b108a4d0 0x5582b145c5d0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f3990012d20 tx=0x7f399000f790 comp rx=0 tx=0).stop 2024-09-21T06:07:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.520+0000 7f39a25ec740 1 -- 172.21.15.117:0/1655730186 shutdown_connections 2024-09-21T06:07:38.532 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.520+0000 7f39a25ec740 1 --2- 172.21.15.117:0/1655730186 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5582b1069d80 0x5582b146e120 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.520+0000 7f39a25ec740 1 --2- 172.21.15.117:0/1655730186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b108a4d0 0x5582b145c5d0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.520+0000 7f39a25ec740 1 --2- 172.21.15.117:0/1655730186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582b10ab1b0 0x5582b14683c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.520+0000 7f397cff9640 0 client.17781 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.520+0000 7f39a25ec740 1 --2- 172.21.15.117:0/1655730186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b10822f0 0x5582b145c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.520+0000 7f39a25ec740 1 -- 172.21.15.117:0/1655730186 >> 172.21.15.117:0/1655730186 conn(0x5582b109f300 msgr2=0x5582b14519e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.521+0000 7f39a25ec740 1 -- 172.21.15.117:0/1655730186 shutdown_connections 2024-09-21T06:07:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.521+0000 7f39a25ec740 1 -- 172.21.15.117:0/1655730186 wait complete. 2024-09-21T06:07:38.533 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.521+0000 7f39a25ec740 20 client.17781 trim_cache size 0 max 16384 2024-09-21T06:07:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.535+0000 7f39a25ec740 1 Processor -- start 2024-09-21T06:07:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.535+0000 7f39a25ec740 1 -- start start 2024-09-21T06:07:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.535+0000 7f39a25ec740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b10822f0 0x5582b145c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.535+0000 7f39a25ec740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b108a4d0 0x5582b14523a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.535+0000 7f39a25ec740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582b10ab1b0 0x5582b145dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.535+0000 7f39a25ec740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f39941af650 con 0x5582b108a4d0 2024-09-21T06:07:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.535+0000 7f39a25ec740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f39941b1190 con 0x5582b10822f0 2024-09-21T06:07:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.535+0000 7f39a25ec740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f39941afd20 con 0x5582b10ab1b0 2024-09-21T06:07:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.536+0000 7f399941a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b108a4d0 0x5582b14523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.536+0000 7f399941a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b108a4d0 0x5582b14523a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41902/0 (socket says 172.21.15.117:41902) 2024-09-21T06:07:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.536+0000 7f399941a640 1 -- 172.21.15.117:0/3795644562 learned_addr learned my addr 172.21.15.117:0/3795644562 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.536+0000 7f3998c19640 1 --2- 172.21.15.117:0/3795644562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582b10ab1b0 0x5582b145dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.536+0000 7f3998c19640 1 --2- 172.21.15.117:0/3795644562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b10822f0 0x5582b145c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.536+0000 7f3998c19640 1 -- 172.21.15.117:0/3795644562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b10822f0 msgr2=0x5582b145c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.536+0000 7f3998c19640 1 --2- 172.21.15.117:0/3795644562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b10822f0 0x5582b145c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:38.539 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.536+0000 7f3998c19640 1 -- 172.21.15.117:0/3795644562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b108a4d0 msgr2=0x5582b14523a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.536+0000 7f3998c19640 1 --2- 172.21.15.117:0/3795644562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b108a4d0 0x5582b14523a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.536+0000 7f3998c19640 1 -- 172.21.15.117:0/3795644562 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582b14537c0 con 0x5582b10ab1b0 2024-09-21T06:07:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.536+0000 7f399941a640 1 --2- 172.21.15.117:0/3795644562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b108a4d0 0x5582b14523a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:07:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.536+0000 7f3998c19640 1 --2- 172.21.15.117:0/3795644562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582b10ab1b0 0x5582b145dcc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3984004600 tx=0x7f3984010040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a71b345b39cbc24 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.536+0000 7f397effd640 1 -- 172.21.15.117:0/3795644562 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3984005280 con 0x5582b10ab1b0 2024-09-21T06:07:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.536+0000 7f397effd640 1 -- 172.21.15.117:0/3795644562 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3984005420 con 0x5582b10ab1b0 2024-09-21T06:07:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.536+0000 7f397effd640 1 -- 172.21.15.117:0/3795644562 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3984004ea0 con 0x5582b10ab1b0 2024-09-21T06:07:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.537+0000 7f39a25ec740 1 -- 172.21.15.117:0/3795644562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582b10ab1b0 msgr2=0x5582b145dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.537+0000 7f39a25ec740 1 --2- 172.21.15.117:0/3795644562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582b10ab1b0 0x5582b145dcc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3984004600 tx=0x7f3984010040 comp rx=0 tx=0).stop 2024-09-21T06:07:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.537+0000 7f39a25ec740 1 -- 172.21.15.117:0/3795644562 shutdown_connections 2024-09-21T06:07:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.537+0000 7f39a25ec740 1 --2- 172.21.15.117:0/3795644562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b108a4d0 0x5582b14523a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.537+0000 7f39a25ec740 1 --2- 172.21.15.117:0/3795644562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582b10ab1b0 0x5582b145dcc0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.537+0000 7f39a25ec740 1 --2- 172.21.15.117:0/3795644562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b10822f0 0x5582b145c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:38.544 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.537+0000 7f39a25ec740 1 -- 172.21.15.117:0/3795644562 >> 172.21.15.117:0/3795644562 conn(0x5582b109f300 msgr2=0x5582b145d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.537+0000 7f39a25ec740 1 -- 172.21.15.117:0/3795644562 shutdown_connections 2024-09-21T06:07:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.537+0000 7f39a25ec740 1 -- 172.21.15.117:0/3795644562 wait complete. 2024-09-21T06:07:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.538+0000 7f39a25ec740 1 -- start start 2024-09-21T06:07:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.538+0000 7f39a25ec740 1 Processor -- start 2024-09-21T06:07:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.538+0000 7f39a25ec740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b10822f0 0x5582b1464a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.538+0000 7f39a25ec740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582b108a4d0 0x5582b145ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.538+0000 7f39a25ec740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b10ab1b0 0x5582b1460450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.538+0000 7f39a25ec740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f39941b0ac0 con 0x5582b10822f0 2024-09-21T06:07:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.538+0000 7f39a25ec740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f39941b03f0 con 0x5582b10ab1b0 2024-09-21T06:07:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.538+0000 7f39a25ec740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f39941ad650 con 0x5582b108a4d0 2024-09-21T06:07:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.538+0000 7f39a25ec740 10 client.0 _ll_register_callbacks cb 0x7f3990012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:38.545 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.538+0000 7f3998c19640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b10822f0 0x5582b1464a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.538+0000 7f3998c19640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b10822f0 0x5582b1464a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41904/0 (socket says 172.21.15.117:41904) 2024-09-21T06:07:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.538+0000 7f3998c19640 1 -- 172.21.15.117:0/2366647686 learned_addr learned my addr 172.21.15.117:0/2366647686 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.538+0000 7f399941a640 1 --2- 172.21.15.117:0/2366647686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582b108a4d0 0x5582b145ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.538+0000 7f3998c19640 1 --2- 172.21.15.117:0/2366647686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b10ab1b0 0x5582b1460450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.539+0000 7f3998c19640 1 -- 172.21.15.117:0/2366647686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582b108a4d0 msgr2=0x5582b145ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.539+0000 7f3998c19640 1 --2- 172.21.15.117:0/2366647686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582b108a4d0 0x5582b145ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.539+0000 7f3998c19640 1 -- 172.21.15.117:0/2366647686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b10ab1b0 msgr2=0x5582b1460450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.539+0000 7f3998c19640 1 --2- 172.21.15.117:0/2366647686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b10ab1b0 0x5582b1460450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.539+0000 7f3998c19640 1 -- 172.21.15.117:0/2366647686 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582b1460b40 con 0x5582b10822f0 2024-09-21T06:07:38.547 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.539+0000 7f399941a640 1 --2- 172.21.15.117:0/2366647686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582b108a4d0 0x5582b145ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:07:38.547 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.539+0000 7f3998c19640 1 --2- 172.21.15.117:0/2366647686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b10822f0 0x5582b1464a70 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f398400a780 tx=0x7f398400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae02be84344b5dd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:38.547 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.539+0000 7f397cff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:38.547 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.539+0000 7f397cff9640 1 -- 172.21.15.117:0/2366647686 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39840050c0 con 0x5582b10822f0 2024-09-21T06:07:38.547 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.539+0000 7f397cff9640 1 -- 172.21.15.117:0/2366647686 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f3984004630 con 0x5582b10822f0 2024-09-21T06:07:38.547 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.539+0000 7f39a25ec740 1 -- 172.21.15.117:0/2366647686 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3984005420 con 0x5582b10822f0 2024-09-21T06:07:38.547 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.539+0000 7f397cff9640 1 -- 172.21.15.117:0/2366647686 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3984041030 con 0x5582b10822f0 2024-09-21T06:07:38.547 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.539+0000 7f39a25ec740 10 client.17787 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:38.547 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.539+0000 7f39a25ec740 1 -- 172.21.15.117:0/2366647686 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5582b14d89b0 con 0x5582b10822f0 2024-09-21T06:07:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.539+0000 7f397cff9640 1 -- 172.21.15.117:0/2366647686 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3984005420 con 0x5582b10822f0 2024-09-21T06:07:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.539+0000 7f397cff9640 1 -- 172.21.15.117:0/2366647686 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f398400b040 con 0x5582b10822f0 2024-09-21T06:07:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.539+0000 7f39a25ec740 10 client.17787 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.539+0000 7f39a25ec740 10 client.17787 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:07:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.540+0000 7f39a25ec740 20 client.17787 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.540+0000 7f39a25ec740 1 --2- 172.21.15.117:0/2366647686 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5582b103f1c0 0x5582b14dae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.540+0000 7f39a25ec740 4 client.17787 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:07:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.540+0000 7f39a25ec740 1 -- 172.21.15.117:0/2366647686 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5582b13c30f0 con 0x5582b103f1c0 2024-09-21T06:07:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.540+0000 7f399941a640 1 --2- 172.21.15.117:0/2366647686 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5582b103f1c0 0x5582b14dae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.540+0000 7f397cff9640 10 client.17787 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:38.548 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.540+0000 7f399941a640 1 --2- 172.21.15.117:0/2366647686 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5582b103f1c0 0x5582b14dae80 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6ed7e06c8278063 server_cookie=40e663d6a7b0e8ac in_seq=0 out_seq=0 2024-09-21T06:07:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.546+0000 7f397cff9640 1 -- 172.21.15.117:0/2366647686 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5582b13c30f0 con 0x5582b103f1c0 2024-09-21T06:07:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.546+0000 7f397cff9640 10 client.17787 handle_command_reply: tid=0 2024-09-21T06:07:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.546+0000 7f39a25ec740 1 client.17787 shutdown 2024-09-21T06:07:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.546+0000 7f39a25ec740 1 -- 172.21.15.117:0/2366647686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b10822f0 msgr2=0x5582b1464a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.546+0000 7f39a25ec740 1 --2- 172.21.15.117:0/2366647686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b10822f0 0x5582b1464a70 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f398400a780 tx=0x7f398400a710 comp rx=0 tx=0).stop 2024-09-21T06:07:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.546+0000 7f39a25ec740 1 -- 172.21.15.117:0/2366647686 shutdown_connections 2024-09-21T06:07:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.546+0000 7f39a25ec740 1 --2- 172.21.15.117:0/2366647686 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5582b103f1c0 0x5582b14dae80 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.546+0000 7f39a25ec740 1 --2- 172.21.15.117:0/2366647686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5582b10ab1b0 0x5582b1460450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.546+0000 7f39a25ec740 1 --2- 172.21.15.117:0/2366647686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5582b108a4d0 0x5582b145ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.546+0000 7f397cff9640 0 client.17787 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.546+0000 7f39a25ec740 1 --2- 172.21.15.117:0/2366647686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5582b10822f0 0x5582b1464a70 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.546+0000 7f39a25ec740 1 -- 172.21.15.117:0/2366647686 >> 172.21.15.117:0/2366647686 conn(0x5582b109f300 msgr2=0x5582b1452170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.547+0000 7f39a25ec740 1 -- 172.21.15.117:0/2366647686 shutdown_connections 2024-09-21T06:07:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.547+0000 7f39a25ec740 1 -- 172.21.15.117:0/2366647686 wait complete. 2024-09-21T06:07:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.547+0000 7f39a25ec740 20 client.17787 trim_cache size 0 max 16384 2024-09-21T06:07:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T06:07:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T06:07:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T06:07:38.552 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T06:07:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.547+0000 7f399b67e640 1 -- 172.21.15.117:0/2237824278 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f39800b6680 msgr2=0x7f39800b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.547+0000 7f399b67e640 1 --2- 172.21.15.117:0/2237824278 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f39800b6680 0x7f39800b8b90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f398c0ce050 tx=0x7f398c030970 comp rx=0 tx=0).stop 2024-09-21T06:07:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.547+0000 7f399b67e640 1 -- 172.21.15.117:0/2237824278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3994108400 msgr2=0x7f399419fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.547+0000 7f399b67e640 1 --2- 172.21.15.117:0/2237824278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3994108400 0x7f399419fc80 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f398c004af0 tx=0x7f398c003c20 comp rx=0 tx=0).stop 2024-09-21T06:07:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.547+0000 7f399b67e640 1 -- 172.21.15.117:0/2237824278 shutdown_connections 2024-09-21T06:07:38.553 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.547+0000 7f399b67e640 1 --2- 172.21.15.117:0/2237824278 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f39800b6680 0x7f39800b8b90 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.547+0000 7f399b67e640 1 --2- 172.21.15.117:0/2237824278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3994108400 0x7f399419fc80 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.547+0000 7f399b67e640 1 --2- 172.21.15.117:0/2237824278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3994107ab0 0x7f399419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.547+0000 7f399b67e640 1 --2- 172.21.15.117:0/2237824278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3994102180 0x7f399419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.547+0000 7f399b67e640 1 -- 172.21.15.117:0/2237824278 >> 172.21.15.117:0/2237824278 conn(0x7f3994079e90 msgr2=0x7f3994105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.547+0000 7f399b67e640 1 -- 172.21.15.117:0/2237824278 shutdown_connections 2024-09-21T06:07:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:38.547+0000 7f399b67e640 1 -- 172.21.15.117:0/2237824278 wait complete. 2024-09-21T06:07:39.159 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:07:39.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.648+0000 7f36a0364640 1 -- 172.21.15.117:0/3313087260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36981094a0 msgr2=0x7f36981098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:39.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.648+0000 7f36a0364640 1 --2- 172.21.15.117:0/3313087260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36981094a0 0x7f36981098b0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f3688009900 tx=0x7f368802a410 comp rx=0 tx=0).stop 2024-09-21T06:07:39.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.648+0000 7f36a0364640 1 -- 172.21.15.117:0/3313087260 shutdown_connections 2024-09-21T06:07:39.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.648+0000 7f36a0364640 1 --2- 172.21.15.117:0/3313087260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36981094a0 0x7f36981098b0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.648+0000 7f36a0364640 1 --2- 172.21.15.117:0/3313087260 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3698109df0 0x7f3698110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.648+0000 7f36a0364640 1 --2- 172.21.15.117:0/3313087260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3698103b60 0x7f3698103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.685 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.648+0000 7f36a0364640 1 -- 172.21.15.117:0/3313087260 >> 172.21.15.117:0/3313087260 conn(0x7f369807b850 msgr2=0x7f369807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:39.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.648+0000 7f36a0364640 1 -- 172.21.15.117:0/3313087260 shutdown_connections 2024-09-21T06:07:39.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.648+0000 7f36a0364640 1 -- 172.21.15.117:0/3313087260 wait complete. 2024-09-21T06:07:39.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.649+0000 7f36a0364640 1 Processor -- start 2024-09-21T06:07:39.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.649+0000 7f36a0364640 1 -- start start 2024-09-21T06:07:39.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.649+0000 7f36a0364640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3698103b60 0x7f369819a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:39.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.649+0000 7f36a0364640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36981094a0 0x7f369819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:39.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.649+0000 7f36a0364640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3698109df0 0x7f369819fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:39.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.649+0000 7f36a0364640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3698113ab0 con 0x7f36981094a0 2024-09-21T06:07:39.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.649+0000 7f36a0364640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3698113930 con 0x7f3698103b60 2024-09-21T06:07:39.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.649+0000 7f36a0364640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3698113c30 con 0x7f3698109df0 2024-09-21T06:07:39.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.649+0000 7f369d8ff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36981094a0 0x7f369819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:39.686 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.649+0000 7f369d8ff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36981094a0 0x7f369819ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41926/0 (socket says 172.21.15.117:41926) 2024-09-21T06:07:39.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.649+0000 7f369d8ff640 1 -- 172.21.15.117:0/4287465274 learned_addr learned my addr 172.21.15.117:0/4287465274 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:39.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.649+0000 7f369e100640 1 --2- 172.21.15.117:0/4287465274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3698103b60 0x7f369819a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:39.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.649+0000 7f369e901640 1 --2- 172.21.15.117:0/4287465274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3698109df0 0x7f369819fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:39.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.649+0000 7f369d8ff640 1 -- 172.21.15.117:0/4287465274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3698109df0 msgr2=0x7f369819fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:39.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.649+0000 7f369d8ff640 1 --2- 172.21.15.117:0/4287465274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3698109df0 0x7f369819fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.650+0000 7f369d8ff640 1 -- 172.21.15.117:0/4287465274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3698103b60 msgr2=0x7f369819a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:39.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.650+0000 7f369d8ff640 1 --2- 172.21.15.117:0/4287465274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3698103b60 0x7f369819a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.650+0000 7f369d8ff640 1 -- 172.21.15.117:0/4287465274 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36981a04f0 con 0x7f36981094a0 2024-09-21T06:07:39.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.650+0000 7f369e100640 1 --2- 172.21.15.117:0/4287465274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3698103b60 0x7f369819a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:07:39.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.650+0000 7f369d8ff640 1 --2- 172.21.15.117:0/4287465274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36981094a0 0x7f369819ab20 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f368802a930 tx=0x7f3688004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:39.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.650+0000 7f368f7fe640 1 -- 172.21.15.117:0/4287465274 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368803f030 con 0x7f36981094a0 2024-09-21T06:07:39.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.650+0000 7f36a0364640 1 -- 172.21.15.117:0/4287465274 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36981a0780 con 0x7f36981094a0 2024-09-21T06:07:39.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.650+0000 7f368f7fe640 1 -- 172.21.15.117:0/4287465274 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3688003aa0 con 0x7f36981094a0 2024-09-21T06:07:39.687 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.650+0000 7f368f7fe640 1 -- 172.21.15.117:0/4287465274 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3688035070 con 0x7f36981094a0 2024-09-21T06:07:39.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.650+0000 7f36a0364640 1 -- 172.21.15.117:0/4287465274 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36981a6f40 con 0x7f36981094a0 2024-09-21T06:07:39.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.652+0000 7f36a68d8740 1 Processor -- start 2024-09-21T06:07:39.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.653+0000 7f36a68d8740 1 -- start start 2024-09-21T06:07:39.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.653+0000 7f368f7fe640 1 -- 172.21.15.117:0/4287465274 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f3688042030 con 0x7f36981094a0 2024-09-21T06:07:39.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.653+0000 7f36a68d8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb55bfb070 0x55bb55fa26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:39.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.653+0000 7f36a68d8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb55bd21b0 0x55bb55fa2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:39.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.653+0000 7f36a68d8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb55bda390 0x55bb55fa3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:39.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.653+0000 7f36a68d8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bb55f18ea0 con 0x55bb55bda390 2024-09-21T06:07:39.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.653+0000 7f36a68d8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bb55f082d0 con 0x55bb55bd21b0 2024-09-21T06:07:39.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.653+0000 7f36a68d8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55bb55f0c0e0 con 0x55bb55bfb070 2024-09-21T06:07:39.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.653+0000 7f369d8ff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb55bda390 0x55bb55fa3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:39.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.653+0000 7f369d8ff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb55bda390 0x55bb55fa3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41928/0 (socket says 172.21.15.117:41928) 2024-09-21T06:07:39.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.653+0000 7f369d8ff640 1 -- 172.21.15.117:0/2436660760 learned_addr learned my addr 172.21.15.117:0/2436660760 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:39.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.653+0000 7f369e901640 1 --2- 172.21.15.117:0/2436660760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb55bfb070 0x55bb55fa26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:39.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.653+0000 7f369e100640 1 --2- 172.21.15.117:0/2436660760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb55bd21b0 0x55bb55fa2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:39.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.653+0000 7f369d8ff640 1 -- 172.21.15.117:0/2436660760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb55bfb070 msgr2=0x55bb55fa26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:39.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.653+0000 7f369d8ff640 1 --2- 172.21.15.117:0/2436660760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb55bfb070 0x55bb55fa26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.654+0000 7f369d8ff640 1 -- 172.21.15.117:0/2436660760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb55bd21b0 msgr2=0x55bb55fa2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:39.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.654+0000 7f369d8ff640 1 --2- 172.21.15.117:0/2436660760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb55bd21b0 0x55bb55fa2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.654+0000 7f369d8ff640 1 -- 172.21.15.117:0/2436660760 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb55c66650 con 0x55bb55bda390 2024-09-21T06:07:39.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.654+0000 7f369e100640 1 --2- 172.21.15.117:0/2436660760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb55bd21b0 0x55bb55fa2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:07:39.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.654+0000 7f368f7fe640 1 --2- 172.21.15.117:0/4287465274 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f36840b6680 0x7f36840b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:39.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.654+0000 7f369d8ff640 1 --2- 172.21.15.117:0/2436660760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb55bda390 0x55bb55fa3120 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f36881054c0 tx=0x7f36881100e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97446dc9bdf88119 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:39.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.654+0000 7f369e901640 1 --2- 172.21.15.117:0/4287465274 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f36840b6680 0x7f36840b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:39.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.654+0000 7f368f7fe640 1 -- 172.21.15.117:0/4287465274 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f3688107db0 con 0x7f36981094a0 2024-09-21T06:07:39.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.654+0000 7f368cd9a640 1 -- 172.21.15.117:0/2436660760 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36880ca2a0 con 0x55bb55bda390 2024-09-21T06:07:39.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.654+0000 7f368cd9a640 1 -- 172.21.15.117:0/2436660760 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3688115070 con 0x55bb55bda390 2024-09-21T06:07:39.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.654+0000 7f368cd9a640 1 -- 172.21.15.117:0/2436660760 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36880cd3c0 con 0x55bb55bda390 2024-09-21T06:07:39.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.655+0000 7f36a68d8740 1 -- 172.21.15.117:0/2436660760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb55bda390 msgr2=0x55bb55fa3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:39.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.655+0000 7f369e901640 1 --2- 172.21.15.117:0/4287465274 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f36840b6680 0x7f36840b8b90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3690005fd0 tx=0x7f3690009620 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:39.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.655+0000 7f36a68d8740 1 --2- 172.21.15.117:0/2436660760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb55bda390 0x55bb55fa3120 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f36881054c0 tx=0x7f36881100e0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.655+0000 7f36a68d8740 1 -- 172.21.15.117:0/2436660760 shutdown_connections 2024-09-21T06:07:39.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.655+0000 7f36a68d8740 1 --2- 172.21.15.117:0/2436660760 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb55bda390 0x55bb55fa3120 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.655+0000 7f36a68d8740 1 --2- 172.21.15.117:0/2436660760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb55bd21b0 0x55bb55fa2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.655+0000 7f36a68d8740 1 --2- 172.21.15.117:0/2436660760 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb55bfb070 0x55bb55fa26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.655+0000 7f36a68d8740 1 -- 172.21.15.117:0/2436660760 >> 172.21.15.117:0/2436660760 conn(0x55bb55bef460 msgr2=0x55bb55fa0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:39.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.655+0000 7f36a68d8740 1 -- 172.21.15.117:0/2436660760 shutdown_connections 2024-09-21T06:07:39.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.655+0000 7f36a68d8740 1 -- 172.21.15.117:0/2436660760 wait complete. 2024-09-21T06:07:39.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.656+0000 7f36a68d8740 1 -- start start 2024-09-21T06:07:39.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.656+0000 7f36a68d8740 1 Processor -- start 2024-09-21T06:07:39.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.656+0000 7f36a68d8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb55bd21b0 0x55bb55fb2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:39.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.656+0000 7f36a68d8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb55bda390 0x55bb55fb2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:39.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.656+0000 7f36a68d8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb55bfb070 0x55bb55fb2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:39.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.656+0000 7f36a68d8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bb55d69700 con 0x55bb55bd21b0 2024-09-21T06:07:39.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.656+0000 7f36a68d8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bb55ea3160 con 0x55bb55bfb070 2024-09-21T06:07:39.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.656+0000 7f36a68d8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55bb55fb4fa0 con 0x55bb55bda390 2024-09-21T06:07:39.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.656+0000 7f36a68d8740 10 client.0 _ll_register_callbacks cb 0x55bb55ebc9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:39.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.656+0000 7f369e100640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb55bd21b0 0x55bb55fb2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:39.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.656+0000 7f369e100640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb55bd21b0 0x55bb55fb2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41932/0 (socket says 172.21.15.117:41932) 2024-09-21T06:07:39.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.656+0000 7f369e100640 1 -- 172.21.15.117:0/2327316756 learned_addr learned my addr 172.21.15.117:0/2327316756 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:39.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.656+0000 7f369e901640 1 --2- 172.21.15.117:0/2327316756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb55bfb070 0x55bb55fb2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:39.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.656+0000 7f369e100640 1 -- 172.21.15.117:0/2327316756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb55bda390 msgr2=0x55bb55fb2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:39.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.656+0000 7f369e100640 1 --2- 172.21.15.117:0/2327316756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb55bda390 0x55bb55fb2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.656+0000 7f369e100640 1 -- 172.21.15.117:0/2327316756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb55bfb070 msgr2=0x55bb55fb2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:39.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.656+0000 7f369e100640 1 --2- 172.21.15.117:0/2327316756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb55bfb070 0x55bb55fb2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.656+0000 7f369e100640 1 -- 172.21.15.117:0/2327316756 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb55ed0c90 con 0x55bb55bd21b0 2024-09-21T06:07:39.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.656+0000 7f369e901640 1 --2- 172.21.15.117:0/2327316756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb55bfb070 0x55bb55fb2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:39.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.657+0000 7f369e100640 1 --2- 172.21.15.117:0/2327316756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb55bd21b0 0x55bb55fb2120 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f3694011800 tx=0x7f3694011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=461fd5368f4f923 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:39.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.657+0000 7f3681ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:39.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.657+0000 7f3681ffb640 1 -- 172.21.15.117:0/2327316756 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3694010ba0 con 0x55bb55bd21b0 2024-09-21T06:07:39.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.657+0000 7f36a68d8740 1 -- 172.21.15.117:0/2327316756 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3688115070 con 0x55bb55bd21b0 2024-09-21T06:07:39.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.657+0000 7f3681ffb640 1 -- 172.21.15.117:0/2327316756 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f3694010d40 con 0x55bb55bd21b0 2024-09-21T06:07:39.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.657+0000 7f3681ffb640 1 -- 172.21.15.117:0/2327316756 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3694005020 con 0x55bb55bd21b0 2024-09-21T06:07:39.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.657+0000 7f36a68d8740 10 client.17799 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:39.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.657+0000 7f36a68d8740 1 -- 172.21.15.117:0/2327316756 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bb55fc0c00 con 0x55bb55bd21b0 2024-09-21T06:07:39.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.657+0000 7f3681ffb640 1 -- 172.21.15.117:0/2327316756 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3688115070 con 0x55bb55bd21b0 2024-09-21T06:07:39.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.657+0000 7f3681ffb640 1 -- 172.21.15.117:0/2327316756 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3694023410 con 0x55bb55bd21b0 2024-09-21T06:07:39.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.657+0000 7f36a68d8740 10 client.17799 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:39.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.657+0000 7f36a68d8740 10 client.17799 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:07:39.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.657+0000 7f36a68d8740 20 client.17799 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:39.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.657+0000 7f36a68d8740 1 --2- 172.21.15.117:0/2327316756 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55bb55bb99f0 0x55bb55fc42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:39.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.657+0000 7f36a68d8740 4 client.17799 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:07:39.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.657+0000 7f36a68d8740 1 -- 172.21.15.117:0/2327316756 --> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb55f1adf0 con 0x55bb55bb99f0 2024-09-21T06:07:39.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.658+0000 7f369d8ff640 1 --2- 172.21.15.117:0/2327316756 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55bb55bb99f0 0x55bb55fc42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:39.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.658+0000 7f3681ffb640 10 client.17799 ms_handle_connect on v2:172.21.15.192:6838/1727623231 2024-09-21T06:07:39.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.658+0000 7f369d8ff640 1 --2- 172.21.15.117:0/2327316756 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55bb55bb99f0 0x55bb55fc42a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b2fa07a0ae301aab server_cookie=d30cb9558487bc30 in_seq=0 out_seq=0 2024-09-21T06:07:39.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.659+0000 7f3681ffb640 1 -- 172.21.15.117:0/2327316756 <== mds.1 v2:172.21.15.192:6838/1727623231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bb55f1adf0 con 0x55bb55bb99f0 2024-09-21T06:07:39.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.659+0000 7f3681ffb640 10 client.17799 handle_command_reply: tid=0 2024-09-21T06:07:39.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.659+0000 7f36a68d8740 1 client.17799 shutdown 2024-09-21T06:07:39.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.659+0000 7f36a68d8740 1 -- 172.21.15.117:0/2327316756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb55bd21b0 msgr2=0x55bb55fb2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:39.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.659+0000 7f36a68d8740 1 --2- 172.21.15.117:0/2327316756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb55bd21b0 0x55bb55fb2120 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f3694011800 tx=0x7f3694011cc0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.659+0000 7f36a68d8740 1 -- 172.21.15.117:0/2327316756 shutdown_connections 2024-09-21T06:07:39.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.659+0000 7f36a68d8740 1 --2- 172.21.15.117:0/2327316756 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55bb55bb99f0 0x55bb55fc42a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.659+0000 7f36a68d8740 1 --2- 172.21.15.117:0/2327316756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb55bfb070 0x55bb55fb2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.659+0000 7f3681ffb640 0 client.17799 ms_handle_reset on v2:172.21.15.192:6838/1727623231 2024-09-21T06:07:39.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.659+0000 7f36a68d8740 1 --2- 172.21.15.117:0/2327316756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb55bda390 0x55bb55fb2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.659+0000 7f36a68d8740 1 --2- 172.21.15.117:0/2327316756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb55bd21b0 0x55bb55fb2120 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.659+0000 7f36a68d8740 1 -- 172.21.15.117:0/2327316756 >> 172.21.15.117:0/2327316756 conn(0x55bb55bef460 msgr2=0x55bb55fa4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:39.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.659+0000 7f36a68d8740 1 -- 172.21.15.117:0/2327316756 shutdown_connections 2024-09-21T06:07:39.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.659+0000 7f36a68d8740 1 -- 172.21.15.117:0/2327316756 wait complete. 2024-09-21T06:07:39.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.659+0000 7f36a68d8740 20 client.17799 trim_cache size 0 max 16384 2024-09-21T06:07:39.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.672+0000 7f36a68d8740 1 Processor -- start 2024-09-21T06:07:39.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.672+0000 7f36a68d8740 1 -- start start 2024-09-21T06:07:39.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.672+0000 7f36a68d8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb55bb99f0 0x55bb55fb09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:39.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.672+0000 7f36a68d8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb55bd21b0 0x55bb55fb0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:39.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.672+0000 7f36a68d8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb55bda390 0x55bb55fb1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:39.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.672+0000 7f36a68d8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f36981ad4f0 con 0x55bb55bda390 2024-09-21T06:07:39.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.672+0000 7f36a68d8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f36981af030 con 0x55bb55bb99f0 2024-09-21T06:07:39.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.672+0000 7f36a68d8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f36981adbc0 con 0x55bb55bd21b0 2024-09-21T06:07:39.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.672+0000 7f369e901640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb55bda390 0x55bb55fb1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:39.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.672+0000 7f369e901640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb55bda390 0x55bb55fb1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41948/0 (socket says 172.21.15.117:41948) 2024-09-21T06:07:39.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.673+0000 7f369e901640 1 -- 172.21.15.117:0/381381024 learned_addr learned my addr 172.21.15.117:0/381381024 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:39.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.673+0000 7f369e100640 1 --2- 172.21.15.117:0/381381024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb55bb99f0 0x55bb55fb09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:39.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.673+0000 7f369d8ff640 1 --2- 172.21.15.117:0/381381024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb55bd21b0 0x55bb55fb0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:39.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.673+0000 7f369e901640 1 -- 172.21.15.117:0/381381024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb55bd21b0 msgr2=0x55bb55fb0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:39.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.673+0000 7f369e901640 1 --2- 172.21.15.117:0/381381024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb55bd21b0 0x55bb55fb0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.673+0000 7f369e901640 1 -- 172.21.15.117:0/381381024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb55bb99f0 msgr2=0x55bb55fb09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:39.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.673+0000 7f369e901640 1 --2- 172.21.15.117:0/381381024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb55bb99f0 0x55bb55fb09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.673+0000 7f369e901640 1 -- 172.21.15.117:0/381381024 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb55fa34e0 con 0x55bb55bda390 2024-09-21T06:07:39.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.673+0000 7f369e901640 1 --2- 172.21.15.117:0/381381024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb55bda390 0x55bb55fb1ed0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f3690012770 tx=0x7f3690014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c9ba304e96b1aa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:39.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.673+0000 7f3683fff640 1 -- 172.21.15.117:0/381381024 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690004d20 con 0x55bb55bda390 2024-09-21T06:07:39.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.673+0000 7f3683fff640 1 -- 172.21.15.117:0/381381024 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f369001fae0 con 0x55bb55bda390 2024-09-21T06:07:39.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.673+0000 7f3683fff640 1 -- 172.21.15.117:0/381381024 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690005730 con 0x55bb55bda390 2024-09-21T06:07:39.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.674+0000 7f36a68d8740 1 -- 172.21.15.117:0/381381024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb55bda390 msgr2=0x55bb55fb1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:39.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.674+0000 7f36a68d8740 1 --2- 172.21.15.117:0/381381024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb55bda390 0x55bb55fb1ed0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f3690012770 tx=0x7f3690014000 comp rx=0 tx=0).stop 2024-09-21T06:07:39.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.674+0000 7f36a68d8740 1 -- 172.21.15.117:0/381381024 shutdown_connections 2024-09-21T06:07:39.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.674+0000 7f36a68d8740 1 --2- 172.21.15.117:0/381381024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb55bda390 0x55bb55fb1ed0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.674+0000 7f36a68d8740 1 --2- 172.21.15.117:0/381381024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb55bd21b0 0x55bb55fb0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.674+0000 7f36a68d8740 1 --2- 172.21.15.117:0/381381024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb55bb99f0 0x55bb55fb09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.674+0000 7f36a68d8740 1 -- 172.21.15.117:0/381381024 >> 172.21.15.117:0/381381024 conn(0x55bb55bef460 msgr2=0x55bb55fad2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:39.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.674+0000 7f36a68d8740 1 -- 172.21.15.117:0/381381024 shutdown_connections 2024-09-21T06:07:39.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.674+0000 7f36a68d8740 1 -- 172.21.15.117:0/381381024 wait complete. 2024-09-21T06:07:39.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.674+0000 7f36a68d8740 1 -- start start 2024-09-21T06:07:39.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.675+0000 7f36a68d8740 1 Processor -- start 2024-09-21T06:07:39.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.675+0000 7f36a68d8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb55bb99f0 0x55bb55fba970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:39.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.675+0000 7f36a68d8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb55bd21b0 0x55bb55fb5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:39.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.675+0000 7f36a68d8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb55bda390 0x55bb55fb6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:39.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.675+0000 7f36a68d8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f36981ae960 con 0x55bb55bda390 2024-09-21T06:07:39.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.675+0000 7f36a68d8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f36981ae290 con 0x55bb55bb99f0 2024-09-21T06:07:39.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.675+0000 7f36a68d8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f36981ab4f0 con 0x55bb55bd21b0 2024-09-21T06:07:39.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.675+0000 7f36a68d8740 10 client.0 _ll_register_callbacks cb 0x7f369400bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:39.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.675+0000 7f369e901640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb55bda390 0x55bb55fb6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:39.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.675+0000 7f369e901640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb55bda390 0x55bb55fb6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41962/0 (socket says 172.21.15.117:41962) 2024-09-21T06:07:39.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.675+0000 7f369e901640 1 -- 172.21.15.117:0/1542202069 learned_addr learned my addr 172.21.15.117:0/1542202069 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:39.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.675+0000 7f369e100640 1 --2- 172.21.15.117:0/1542202069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb55bb99f0 0x55bb55fba970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:39.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.675+0000 7f369d8ff640 1 --2- 172.21.15.117:0/1542202069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb55bd21b0 0x55bb55fb5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:39.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.675+0000 7f369e901640 1 -- 172.21.15.117:0/1542202069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb55bd21b0 msgr2=0x55bb55fb5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:39.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.675+0000 7f369e901640 1 --2- 172.21.15.117:0/1542202069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb55bd21b0 0x55bb55fb5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.675+0000 7f369e901640 1 -- 172.21.15.117:0/1542202069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb55bb99f0 msgr2=0x55bb55fba970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:39.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.675+0000 7f369e901640 1 --2- 172.21.15.117:0/1542202069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb55bb99f0 0x55bb55fba970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.675+0000 7f369e901640 1 -- 172.21.15.117:0/1542202069 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb55fb6aa0 con 0x55bb55bda390 2024-09-21T06:07:39.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.675+0000 7f369e100640 1 --2- 172.21.15.117:0/1542202069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb55bb99f0 0x55bb55fba970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:39.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.676+0000 7f369e901640 1 --2- 172.21.15.117:0/1542202069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb55bda390 0x55bb55fb6360 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f369001fd00 tx=0x7f369001fd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ea0a452ee9943b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:39.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.676+0000 7f3681ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:39.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.676+0000 7f3681ffb640 1 -- 172.21.15.117:0/1542202069 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369001f840 con 0x55bb55bda390 2024-09-21T06:07:39.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.676+0000 7f3681ffb640 1 -- 172.21.15.117:0/1542202069 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f3690005470 con 0x55bb55bda390 2024-09-21T06:07:39.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.676+0000 7f36a68d8740 1 -- 172.21.15.117:0/1542202069 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f369001fae0 con 0x55bb55bda390 2024-09-21T06:07:39.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.676+0000 7f3681ffb640 1 -- 172.21.15.117:0/1542202069 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3690005670 con 0x55bb55bda390 2024-09-21T06:07:39.700 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:07:39.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.676+0000 7f36a68d8740 10 client.17805 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:39.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.676+0000 7f36a68d8740 1 -- 172.21.15.117:0/1542202069 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bb5602eb30 con 0x55bb55bda390 2024-09-21T06:07:39.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.676+0000 7f3681ffb640 1 -- 172.21.15.117:0/1542202069 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f369001fae0 con 0x55bb55bda390 2024-09-21T06:07:39.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.676+0000 7f3681ffb640 1 -- 172.21.15.117:0/1542202069 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f369000f040 con 0x55bb55bda390 2024-09-21T06:07:39.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.676+0000 7f36a68d8740 10 client.17805 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:39.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.676+0000 7f36a68d8740 10 client.17805 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:07:39.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.676+0000 7f36a68d8740 20 client.17805 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:39.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.676+0000 7f36a68d8740 1 --2- 172.21.15.117:0/1542202069 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55bb55bfb070 0x55bb560311e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:39.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.676+0000 7f36a68d8740 4 client.17805 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:07:39.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.676+0000 7f36a68d8740 1 -- 172.21.15.117:0/1542202069 --> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bb55f168a0 con 0x55bb55bfb070 2024-09-21T06:07:39.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.677+0000 7f369e100640 1 --2- 172.21.15.117:0/1542202069 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55bb55bfb070 0x55bb560311e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:39.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.677+0000 7f369e100640 1 --2- 172.21.15.117:0/1542202069 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55bb55bfb070 0x55bb560311e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f131afb4a549f137 server_cookie=17bcfeb492f9e1a in_seq=0 out_seq=0 2024-09-21T06:07:39.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.677+0000 7f3681ffb640 10 client.17805 ms_handle_connect on v2:172.21.15.192:6838/1727623231 2024-09-21T06:07:39.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.677+0000 7f3681ffb640 1 -- 172.21.15.117:0/1542202069 <== mds.1 v2:172.21.15.192:6838/1727623231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bb55f168a0 con 0x55bb55bfb070 2024-09-21T06:07:39.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.677+0000 7f3681ffb640 10 client.17805 handle_command_reply: tid=0 2024-09-21T06:07:39.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.677+0000 7f36a68d8740 1 client.17805 shutdown 2024-09-21T06:07:39.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.677+0000 7f36a68d8740 1 -- 172.21.15.117:0/1542202069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb55bda390 msgr2=0x55bb55fb6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:39.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.677+0000 7f36a68d8740 1 --2- 172.21.15.117:0/1542202069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb55bda390 0x55bb55fb6360 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f369001fd00 tx=0x7f369001fd30 comp rx=0 tx=0).stop 2024-09-21T06:07:39.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.677+0000 7f36a68d8740 1 -- 172.21.15.117:0/1542202069 shutdown_connections 2024-09-21T06:07:39.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.677+0000 7f36a68d8740 1 --2- 172.21.15.117:0/1542202069 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55bb55bfb070 0x55bb560311e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.677+0000 7f36a68d8740 1 --2- 172.21.15.117:0/1542202069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bb55bda390 0x55bb55fb6360 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.677+0000 7f36a68d8740 1 --2- 172.21.15.117:0/1542202069 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bb55bd21b0 0x55bb55fb5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.677+0000 7f36a68d8740 1 --2- 172.21.15.117:0/1542202069 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bb55bb99f0 0x55bb55fba970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.677+0000 7f36a68d8740 1 -- 172.21.15.117:0/1542202069 >> 172.21.15.117:0/1542202069 conn(0x55bb55bef460 msgr2=0x55bb55fb1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:39.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.677+0000 7f3681ffb640 0 client.17805 ms_handle_reset on v2:172.21.15.192:6838/1727623231 2024-09-21T06:07:39.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.678+0000 7f36a68d8740 1 -- 172.21.15.117:0/1542202069 shutdown_connections 2024-09-21T06:07:39.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.678+0000 7f36a68d8740 1 -- 172.21.15.117:0/1542202069 wait complete. 2024-09-21T06:07:39.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.678+0000 7f36a68d8740 20 client.17805 trim_cache size 0 max 16384 2024-09-21T06:07:39.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.678+0000 7f36a0364640 1 -- 172.21.15.117:0/4287465274 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f36840b6680 msgr2=0x7f36840b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:39.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.678+0000 7f36a0364640 1 --2- 172.21.15.117:0/4287465274 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f36840b6680 0x7f36840b8b90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3690005fd0 tx=0x7f3690009620 comp rx=0 tx=0).stop 2024-09-21T06:07:39.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.678+0000 7f36a0364640 1 -- 172.21.15.117:0/4287465274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36981094a0 msgr2=0x7f369819ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:39.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.678+0000 7f36a0364640 1 --2- 172.21.15.117:0/4287465274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36981094a0 0x7f369819ab20 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f368802a930 tx=0x7f3688004270 comp rx=0 tx=0).stop 2024-09-21T06:07:39.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.678+0000 7f36a0364640 1 -- 172.21.15.117:0/4287465274 shutdown_connections 2024-09-21T06:07:39.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.678+0000 7f36a0364640 1 --2- 172.21.15.117:0/4287465274 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f36840b6680 0x7f36840b8b90 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.678+0000 7f36a0364640 1 --2- 172.21.15.117:0/4287465274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36981094a0 0x7f369819ab20 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.678+0000 7f36a0364640 1 --2- 172.21.15.117:0/4287465274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3698109df0 0x7f369819fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.678+0000 7f36a0364640 1 --2- 172.21.15.117:0/4287465274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3698103b60 0x7f369819a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:39.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.678+0000 7f36a0364640 1 -- 172.21.15.117:0/4287465274 >> 172.21.15.117:0/4287465274 conn(0x7f369807b850 msgr2=0x7f3698105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:39.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.678+0000 7f36a0364640 1 -- 172.21.15.117:0/4287465274 shutdown_connections 2024-09-21T06:07:39.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:39.678+0000 7f36a0364640 1 -- 172.21.15.117:0/4287465274 wait complete. 2024-09-21T06:07:40.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:39 smithi192 ceph-mon[34700]: pgmap v644: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:07:40.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:39 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:07:40.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:39 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:07:40.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:39 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:07:40.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:39 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:07:40.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:39 smithi117 ceph-mon[30947]: pgmap v644: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:07:40.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:39 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:07:40.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:39 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:07:40.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:39 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:07:40.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:39 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:07:40.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:39 smithi138 ceph-mon[34391]: pgmap v644: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:07:40.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:39 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:07:40.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:39 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:07:40.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:39 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:07:40.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:39 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:07:40.417 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:07:40.682 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:07:41.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.222+0000 7f7624e80640 1 -- 172.21.15.117:0/9546749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7620102a80 msgr2=0x7f7620102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:41.228 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.222+0000 7f7624e80640 1 --2- 172.21.15.117:0/9546749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7620102a80 0x7f7620102e90 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f7610009900 tx=0x7f761002a410 comp rx=0 tx=0).stop 2024-09-21T06:07:41.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.222+0000 7f7624e80640 1 -- 172.21.15.117:0/9546749 shutdown_connections 2024-09-21T06:07:41.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.222+0000 7f7624e80640 1 --2- 172.21.15.117:0/9546749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7620108d10 0x7f762010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.222+0000 7f7624e80640 1 --2- 172.21.15.117:0/9546749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f76201083c0 0x7f76201087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.222+0000 7f7624e80640 1 --2- 172.21.15.117:0/9546749 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7620102a80 0x7f7620102e90 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.222+0000 7f7624e80640 1 -- 172.21.15.117:0/9546749 >> 172.21.15.117:0/9546749 conn(0x7f7620079740 msgr2=0x7f7620079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:41.229 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.223+0000 7f7624e80640 1 -- 172.21.15.117:0/9546749 shutdown_connections 2024-09-21T06:07:41.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.223+0000 7f7624e80640 1 -- 172.21.15.117:0/9546749 wait complete. 2024-09-21T06:07:41.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.223+0000 7f7624e80640 1 Processor -- start 2024-09-21T06:07:41.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.223+0000 7f7624e80640 1 -- start start 2024-09-21T06:07:41.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.224+0000 7f7624e80640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7620102a80 0x7f762010ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:41.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.224+0000 7f7624e80640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76201083c0 0x7f7620110510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:41.230 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.224+0000 7f7624e80640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7620108d10 0x7f7620110a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:41.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.224+0000 7f7624e80640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7620113b60 con 0x7f7620102a80 2024-09-21T06:07:41.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.224+0000 7f7624e80640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f76201139e0 con 0x7f7620108d10 2024-09-21T06:07:41.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.224+0000 7f7624e80640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7620113ce0 con 0x7f76201083c0 2024-09-21T06:07:41.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.224+0000 7f761e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7620102a80 0x7f762010ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:41.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.224+0000 7f761e59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7620102a80 0x7f762010ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41994/0 (socket says 172.21.15.117:41994) 2024-09-21T06:07:41.231 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.224+0000 7f761e59c640 1 -- 172.21.15.117:0/3288207478 learned_addr learned my addr 172.21.15.117:0/3288207478 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:41.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.224+0000 7f761ed9d640 1 --2- 172.21.15.117:0/3288207478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7620108d10 0x7f7620110a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:41.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.224+0000 7f761dd9b640 1 --2- 172.21.15.117:0/3288207478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76201083c0 0x7f7620110510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:41.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.224+0000 7f761e59c640 1 -- 172.21.15.117:0/3288207478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76201083c0 msgr2=0x7f7620110510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:41.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.224+0000 7f761e59c640 1 --2- 172.21.15.117:0/3288207478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76201083c0 0x7f7620110510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.224+0000 7f761e59c640 1 -- 172.21.15.117:0/3288207478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7620108d10 msgr2=0x7f7620110a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:41.232 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.224+0000 7f761e59c640 1 --2- 172.21.15.117:0/3288207478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7620108d10 0x7f7620110a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.224+0000 7f761e59c640 1 -- 172.21.15.117:0/3288207478 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76201a88b0 con 0x7f7620102a80 2024-09-21T06:07:41.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.224+0000 7f761ed9d640 1 --2- 172.21.15.117:0/3288207478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7620108d10 0x7f7620110a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:41.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.224+0000 7f761e59c640 1 --2- 172.21.15.117:0/3288207478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7620102a80 0x7f762010ffd0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f7610006010 tx=0x7f76100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:41.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.225+0000 7f760f7fe640 1 -- 172.21.15.117:0/3288207478 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761003f030 con 0x7f7620102a80 2024-09-21T06:07:41.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.225+0000 7f760f7fe640 1 -- 172.21.15.117:0/3288207478 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7610003800 con 0x7f7620102a80 2024-09-21T06:07:41.233 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.225+0000 7f760f7fe640 1 -- 172.21.15.117:0/3288207478 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7610035070 con 0x7f7620102a80 2024-09-21T06:07:41.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.225+0000 7f7624e80640 1 -- 172.21.15.117:0/3288207478 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76201a8b40 con 0x7f7620102a80 2024-09-21T06:07:41.234 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.225+0000 7f7624e80640 1 -- 172.21.15.117:0/3288207478 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76201a9070 con 0x7f7620102a80 2024-09-21T06:07:41.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.227+0000 7f760f7fe640 1 -- 172.21.15.117:0/3288207478 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f7610003cb0 con 0x7f7620102a80 2024-09-21T06:07:41.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.227+0000 7f762b3f4740 1 Processor -- start 2024-09-21T06:07:41.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.228+0000 7f762b3f4740 1 -- start start 2024-09-21T06:07:41.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.228+0000 7f762b3f4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9987e7070 0x55c998b8e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:41.243 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.228+0000 7f762b3f4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9987be1b0 0x55c998b8ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:41.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.228+0000 7f762b3f4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9987c6390 0x55c998b8f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:41.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.228+0000 7f762b3f4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c998b04ea0 con 0x55c9987c6390 2024-09-21T06:07:41.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.228+0000 7f762b3f4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c998af42d0 con 0x55c9987e7070 2024-09-21T06:07:41.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.228+0000 7f762b3f4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c998af80e0 con 0x55c9987be1b0 2024-09-21T06:07:41.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.228+0000 7f761dd9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9987c6390 0x55c998b8f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:41.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.228+0000 7f760f7fe640 1 --2- 172.21.15.117:0/3288207478 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f76080b6700 0x7f76080b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:41.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.228+0000 7f761dd9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9987c6390 0x55c998b8f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41996/0 (socket says 172.21.15.117:41996) 2024-09-21T06:07:41.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.228+0000 7f761dd9b640 1 -- 172.21.15.117:0/568243108 learned_addr learned my addr 172.21.15.117:0/568243108 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:41.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.228+0000 7f761e59c640 1 --2- 172.21.15.117:0/568243108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9987be1b0 0x55c998b8ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:41.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.228+0000 7f761ed9d640 1 --2- 172.21.15.117:0/568243108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9987e7070 0x55c998b8e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:41.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.228+0000 7f761ed9d640 1 --2- 172.21.15.117:0/3288207478 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f76080b6700 0x7f76080b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:41.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.229+0000 7f761e59c640 1 -- 172.21.15.117:0/568243108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9987e7070 msgr2=0x55c998b8e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:41.244 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.229+0000 7f760f7fe640 1 -- 172.21.15.117:0/3288207478 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f76101080a0 con 0x7f7620102a80 2024-09-21T06:07:41.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.229+0000 7f761e59c640 1 --2- 172.21.15.117:0/568243108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9987e7070 0x55c998b8e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.229+0000 7f761e59c640 1 -- 172.21.15.117:0/568243108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9987c6390 msgr2=0x55c998b8f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:41.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.229+0000 7f761e59c640 1 --2- 172.21.15.117:0/568243108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9987c6390 0x55c998b8f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.229+0000 7f761e59c640 1 -- 172.21.15.117:0/568243108 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c998852650 con 0x55c9987be1b0 2024-09-21T06:07:41.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.229+0000 7f761dd9b640 1 --2- 172.21.15.117:0/568243108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9987c6390 0x55c998b8f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:07:41.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.229+0000 7f761ed9d640 1 --2- 172.21.15.117:0/3288207478 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f76080b6700 0x7f76080b8c10 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7618009440 tx=0x7f7618009a40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:41.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.229+0000 7f761e59c640 1 --2- 172.21.15.117:0/568243108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9987be1b0 0x55c998b8ebe0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7610105840 tx=0x7f761002a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c873bca72b25423 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:41.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.229+0000 7f760cff9640 1 -- 172.21.15.117:0/568243108 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7610049070 con 0x55c9987be1b0 2024-09-21T06:07:41.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.229+0000 7f760cff9640 1 -- 172.21.15.117:0/568243108 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76100cda50 con 0x55c9987be1b0 2024-09-21T06:07:41.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.229+0000 7f760cff9640 1 -- 172.21.15.117:0/568243108 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76100d74d0 con 0x55c9987be1b0 2024-09-21T06:07:41.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.230+0000 7f762b3f4740 1 -- 172.21.15.117:0/568243108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9987be1b0 msgr2=0x55c998b8ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:41.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.230+0000 7f762b3f4740 1 --2- 172.21.15.117:0/568243108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9987be1b0 0x55c998b8ebe0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7610105840 tx=0x7f761002a410 comp rx=0 tx=0).stop 2024-09-21T06:07:41.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.230+0000 7f762b3f4740 1 -- 172.21.15.117:0/568243108 shutdown_connections 2024-09-21T06:07:41.245 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.230+0000 7f762b3f4740 1 --2- 172.21.15.117:0/568243108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9987c6390 0x55c998b8f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.230+0000 7f762b3f4740 1 --2- 172.21.15.117:0/568243108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9987be1b0 0x55c998b8ebe0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.230+0000 7f762b3f4740 1 --2- 172.21.15.117:0/568243108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9987e7070 0x55c998b8e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.230+0000 7f762b3f4740 1 -- 172.21.15.117:0/568243108 >> 172.21.15.117:0/568243108 conn(0x55c9987db460 msgr2=0x55c998b8c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:41.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.230+0000 7f762b3f4740 1 -- 172.21.15.117:0/568243108 shutdown_connections 2024-09-21T06:07:41.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.230+0000 7f762b3f4740 1 -- 172.21.15.117:0/568243108 wait complete. 2024-09-21T06:07:41.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.231+0000 7f762b3f4740 1 -- start start 2024-09-21T06:07:41.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.231+0000 7f762b3f4740 1 Processor -- start 2024-09-21T06:07:41.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.231+0000 7f762b3f4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9987be1b0 0x55c998b99df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:41.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.231+0000 7f762b3f4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9987c6390 0x55c998b9a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:41.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.231+0000 7f762b3f4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9987e7070 0x55c998ba5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:41.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.231+0000 7f762b3f4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c998955700 con 0x55c9987e7070 2024-09-21T06:07:41.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.231+0000 7f762b3f4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c998a8f160 con 0x55c9987c6390 2024-09-21T06:07:41.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.231+0000 7f762b3f4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c998b9cc40 con 0x55c9987be1b0 2024-09-21T06:07:41.246 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.231+0000 7f762b3f4740 10 client.0 _ll_register_callbacks cb 0x55c998aa89e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:41.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.231+0000 7f761ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9987e7070 0x55c998ba5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:41.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.231+0000 7f761ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9987e7070 0x55c998ba5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42010/0 (socket says 172.21.15.117:42010) 2024-09-21T06:07:41.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.231+0000 7f761ed9d640 1 -- 172.21.15.117:0/2468304480 learned_addr learned my addr 172.21.15.117:0/2468304480 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:41.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.231+0000 7f761e59c640 1 --2- 172.21.15.117:0/2468304480 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9987be1b0 0x55c998b99df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:41.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.232+0000 7f761dd9b640 1 --2- 172.21.15.117:0/2468304480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9987c6390 0x55c998b9a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:41.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.232+0000 7f761ed9d640 1 -- 172.21.15.117:0/2468304480 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9987be1b0 msgr2=0x55c998b99df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:41.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.232+0000 7f761ed9d640 1 --2- 172.21.15.117:0/2468304480 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9987be1b0 0x55c998b99df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.232+0000 7f761ed9d640 1 -- 172.21.15.117:0/2468304480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9987c6390 msgr2=0x55c998b9a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:41.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.232+0000 7f761ed9d640 1 --2- 172.21.15.117:0/2468304480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9987c6390 0x55c998b9a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.247 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.232+0000 7f761ed9d640 1 -- 172.21.15.117:0/2468304480 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c998abcc90 con 0x55c9987e7070 2024-09-21T06:07:41.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.232+0000 7f761ed9d640 1 --2- 172.21.15.117:0/2468304480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9987e7070 0x55c998ba5ea0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f761800aee0 tx=0x7f7618019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e62bb9b2e8adc36 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:41.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.232+0000 7f7605ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:41.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.232+0000 7f7605ffb640 1 -- 172.21.15.117:0/2468304480 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761801d040 con 0x55c9987e7070 2024-09-21T06:07:41.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.232+0000 7f7605ffb640 1 -- 172.21.15.117:0/2468304480 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f7618021430 con 0x55c9987e7070 2024-09-21T06:07:41.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.232+0000 7f762b3f4740 1 -- 172.21.15.117:0/2468304480 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76100cda50 con 0x55c9987e7070 2024-09-21T06:07:41.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.232+0000 7f7605ffb640 1 -- 172.21.15.117:0/2468304480 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f761802f050 con 0x55c9987e7070 2024-09-21T06:07:41.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.232+0000 7f762b3f4740 10 client.17817 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:41.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.232+0000 7f762b3f4740 1 -- 172.21.15.117:0/2468304480 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c998ba8960 con 0x55c9987e7070 2024-09-21T06:07:41.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.232+0000 7f7605ffb640 1 -- 172.21.15.117:0/2468304480 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76100cda50 con 0x55c9987e7070 2024-09-21T06:07:41.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.233+0000 7f7605ffb640 1 -- 172.21.15.117:0/2468304480 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7618020070 con 0x55c9987e7070 2024-09-21T06:07:41.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.233+0000 7f762b3f4740 10 client.17817 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:41.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.233+0000 7f762b3f4740 10 client.17817 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:07:41.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.233+0000 7f762b3f4740 20 client.17817 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:41.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.233+0000 7f762b3f4740 1 --2- 172.21.15.117:0/2468304480 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c9987a59f0 0x55c998babf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:41.248 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.233+0000 7f762b3f4740 4 client.17817 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:07:41.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.233+0000 7f762b3f4740 1 -- 172.21.15.117:0/2468304480 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c998b06df0 con 0x55c9987a59f0 2024-09-21T06:07:41.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.233+0000 7f761e59c640 1 --2- 172.21.15.117:0/2468304480 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c9987a59f0 0x55c998babf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:41.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.233+0000 7f7605ffb640 10 client.17817 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:41.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.233+0000 7f761e59c640 1 --2- 172.21.15.117:0/2468304480 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c9987a59f0 0x55c998babf00 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83ae1be2a89b4996 server_cookie=4f2d846b2dec5fbe in_seq=0 out_seq=0 2024-09-21T06:07:41.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.234+0000 7f7605ffb640 1 -- 172.21.15.117:0/2468304480 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c998b06df0 con 0x55c9987a59f0 2024-09-21T06:07:41.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.234+0000 7f7605ffb640 10 client.17817 handle_command_reply: tid=0 2024-09-21T06:07:41.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.234+0000 7f762b3f4740 1 client.17817 shutdown 2024-09-21T06:07:41.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.234+0000 7f762b3f4740 1 -- 172.21.15.117:0/2468304480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9987e7070 msgr2=0x55c998ba5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:41.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.234+0000 7f762b3f4740 1 --2- 172.21.15.117:0/2468304480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9987e7070 0x55c998ba5ea0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f761800aee0 tx=0x7f7618019000 comp rx=0 tx=0).stop 2024-09-21T06:07:41.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.234+0000 7f762b3f4740 1 -- 172.21.15.117:0/2468304480 shutdown_connections 2024-09-21T06:07:41.249 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.234+0000 7f762b3f4740 1 --2- 172.21.15.117:0/2468304480 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c9987a59f0 0x55c998babf00 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.234+0000 7f762b3f4740 1 --2- 172.21.15.117:0/2468304480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9987e7070 0x55c998ba5ea0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.234+0000 7f762b3f4740 1 --2- 172.21.15.117:0/2468304480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9987c6390 0x55c998b9a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.234+0000 7f7605ffb640 0 client.17817 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:41.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.234+0000 7f762b3f4740 1 --2- 172.21.15.117:0/2468304480 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9987be1b0 0x55c998b99df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.234+0000 7f762b3f4740 1 -- 172.21.15.117:0/2468304480 >> 172.21.15.117:0/2468304480 conn(0x55c9987db460 msgr2=0x55c998b91830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:41.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.234+0000 7f762b3f4740 1 -- 172.21.15.117:0/2468304480 shutdown_connections 2024-09-21T06:07:41.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.235+0000 7f762b3f4740 1 -- 172.21.15.117:0/2468304480 wait complete. 2024-09-21T06:07:41.250 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.235+0000 7f762b3f4740 20 client.17817 trim_cache size 0 max 16384 2024-09-21T06:07:41.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.248+0000 7f762b3f4740 1 Processor -- start 2024-09-21T06:07:41.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.248+0000 7f762b3f4740 1 -- start start 2024-09-21T06:07:41.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.248+0000 7f762b3f4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9987be1b0 0x55c998b9ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:41.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.248+0000 7f762b3f4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9987c6390 0x55c998ba5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:41.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.248+0000 7f762b3f4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9987e7070 0x55c998b96fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:41.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.248+0000 7f762b3f4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f76201b1890 con 0x55c9987e7070 2024-09-21T06:07:41.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.248+0000 7f762b3f4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f76201b33d0 con 0x55c9987c6390 2024-09-21T06:07:41.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.248+0000 7f762b3f4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f76201b1f60 con 0x55c9987be1b0 2024-09-21T06:07:41.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.248+0000 7f761ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9987e7070 0x55c998b96fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:41.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.248+0000 7f761ed9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9987e7070 0x55c998b96fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42022/0 (socket says 172.21.15.117:42022) 2024-09-21T06:07:41.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.248+0000 7f761ed9d640 1 -- 172.21.15.117:0/804518897 learned_addr learned my addr 172.21.15.117:0/804518897 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:41.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.248+0000 7f761dd9b640 1 --2- 172.21.15.117:0/804518897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9987c6390 0x55c998ba5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:41.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.248+0000 7f761e59c640 1 --2- 172.21.15.117:0/804518897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9987be1b0 0x55c998b9ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:41.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.248+0000 7f761ed9d640 1 -- 172.21.15.117:0/804518897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9987be1b0 msgr2=0x55c998b9ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:41.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.248+0000 7f761ed9d640 1 --2- 172.21.15.117:0/804518897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9987be1b0 0x55c998b9ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.248+0000 7f761ed9d640 1 -- 172.21.15.117:0/804518897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9987c6390 msgr2=0x55c998ba5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:41.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.248+0000 7f761ed9d640 1 --2- 172.21.15.117:0/804518897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9987c6390 0x55c998ba5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.248+0000 7f761ed9d640 1 -- 172.21.15.117:0/804518897 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c998b91500 con 0x55c9987e7070 2024-09-21T06:07:41.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.249+0000 7f761ed9d640 1 --2- 172.21.15.117:0/804518897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9987e7070 0x55c998b96fe0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f7618000f80 tx=0x7f7618004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688b9bc82914134b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:41.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.249+0000 7f7607fff640 1 -- 172.21.15.117:0/804518897 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761801d040 con 0x55c9987e7070 2024-09-21T06:07:41.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.249+0000 7f7607fff640 1 -- 172.21.15.117:0/804518897 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7618026070 con 0x55c9987e7070 2024-09-21T06:07:41.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.249+0000 7f7607fff640 1 -- 172.21.15.117:0/804518897 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7618013040 con 0x55c9987e7070 2024-09-21T06:07:41.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.249+0000 7f762b3f4740 1 -- 172.21.15.117:0/804518897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9987e7070 msgr2=0x55c998b96fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:41.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.249+0000 7f762b3f4740 1 --2- 172.21.15.117:0/804518897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9987e7070 0x55c998b96fe0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f7618000f80 tx=0x7f7618004a20 comp rx=0 tx=0).stop 2024-09-21T06:07:41.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.249+0000 7f762b3f4740 1 -- 172.21.15.117:0/804518897 shutdown_connections 2024-09-21T06:07:41.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.249+0000 7f762b3f4740 1 --2- 172.21.15.117:0/804518897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9987e7070 0x55c998b96fe0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.249+0000 7f762b3f4740 1 --2- 172.21.15.117:0/804518897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9987c6390 0x55c998ba5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.249+0000 7f762b3f4740 1 --2- 172.21.15.117:0/804518897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9987be1b0 0x55c998b9ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.250+0000 7f762b3f4740 1 -- 172.21.15.117:0/804518897 >> 172.21.15.117:0/804518897 conn(0x55c9987db460 msgr2=0x55c998b9b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:41.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.250+0000 7f762b3f4740 1 -- 172.21.15.117:0/804518897 shutdown_connections 2024-09-21T06:07:41.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.250+0000 7f762b3f4740 1 -- 172.21.15.117:0/804518897 wait complete. 2024-09-21T06:07:41.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.250+0000 7f762b3f4740 1 -- start start 2024-09-21T06:07:41.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.250+0000 7f762b3f4740 1 Processor -- start 2024-09-21T06:07:41.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.250+0000 7f762b3f4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9987be1b0 0x55c998b9def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:41.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.250+0000 7f762b3f4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9987c6390 0x55c998b9e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:41.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.250+0000 7f762b3f4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9987e7070 0x55c998b9e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:41.257 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.250+0000 7f762b3f4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f76201b2d00 con 0x55c9987c6390 2024-09-21T06:07:41.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.250+0000 7f762b3f4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f76201b2630 con 0x55c9987e7070 2024-09-21T06:07:41.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.250+0000 7f762b3f4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f76201af890 con 0x55c9987be1b0 2024-09-21T06:07:41.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.250+0000 7f762b3f4740 10 client.0 _ll_register_callbacks cb 0x7f7618016e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:41.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.250+0000 7f761dd9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9987c6390 0x55c998b9e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:41.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.250+0000 7f761dd9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9987c6390 0x55c998b9e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42026/0 (socket says 172.21.15.117:42026) 2024-09-21T06:07:41.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.250+0000 7f761dd9b640 1 -- 172.21.15.117:0/242368849 learned_addr learned my addr 172.21.15.117:0/242368849 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:41.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.251+0000 7f761e59c640 1 --2- 172.21.15.117:0/242368849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9987be1b0 0x55c998b9def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:41.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.251+0000 7f761ed9d640 1 --2- 172.21.15.117:0/242368849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9987e7070 0x55c998b9e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:41.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.251+0000 7f761dd9b640 1 -- 172.21.15.117:0/242368849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9987be1b0 msgr2=0x55c998b9def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:41.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.251+0000 7f761dd9b640 1 --2- 172.21.15.117:0/242368849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9987be1b0 0x55c998b9def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.251+0000 7f761dd9b640 1 -- 172.21.15.117:0/242368849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9987e7070 msgr2=0x55c998b9e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:41.258 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.251+0000 7f761dd9b640 1 --2- 172.21.15.117:0/242368849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9987e7070 0x55c998b9e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.251+0000 7f761dd9b640 1 -- 172.21.15.117:0/242368849 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c998c15970 con 0x55c9987c6390 2024-09-21T06:07:41.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.251+0000 7f761dd9b640 1 --2- 172.21.15.117:0/242368849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9987c6390 0x55c998b9e430 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f7614014500 tx=0x7f76140149c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d240880fce9526 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:41.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.251+0000 7f7605ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:41.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.251+0000 7f7605ffb640 1 -- 172.21.15.117:0/242368849 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761401c070 con 0x55c9987c6390 2024-09-21T06:07:41.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.251+0000 7f7605ffb640 1 -- 172.21.15.117:0/242368849 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f7614018830 con 0x55c9987c6390 2024-09-21T06:07:41.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.251+0000 7f762b3f4740 1 -- 172.21.15.117:0/242368849 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7618026070 con 0x55c9987c6390 2024-09-21T06:07:41.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.251+0000 7f7605ffb640 1 -- 172.21.15.117:0/242368849 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7614028050 con 0x55c9987c6390 2024-09-21T06:07:41.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.251+0000 7f762b3f4740 10 client.17823 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:41.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.251+0000 7f762b3f4740 1 -- 172.21.15.117:0/242368849 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c998c16c50 con 0x55c9987c6390 2024-09-21T06:07:41.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.252+0000 7f7605ffb640 1 -- 172.21.15.117:0/242368849 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7618026070 con 0x55c9987c6390 2024-09-21T06:07:41.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.252+0000 7f7605ffb640 1 -- 172.21.15.117:0/242368849 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7614016bf0 con 0x55c9987c6390 2024-09-21T06:07:41.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.252+0000 7f762b3f4740 10 client.17823 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:41.259 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.252+0000 7f762b3f4740 10 client.17823 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:07:41.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.252+0000 7f762b3f4740 20 client.17823 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:41.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.252+0000 7f762b3f4740 1 --2- 172.21.15.117:0/242368849 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c99877ada0 0x55c998c19470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:41.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.252+0000 7f762b3f4740 4 client.17823 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:07:41.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.252+0000 7f762b3f4740 1 -- 172.21.15.117:0/242368849 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c998b028a0 con 0x55c99877ada0 2024-09-21T06:07:41.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.252+0000 7f761e59c640 1 --2- 172.21.15.117:0/242368849 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c99877ada0 0x55c998c19470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:41.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.252+0000 7f761e59c640 1 --2- 172.21.15.117:0/242368849 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c99877ada0 0x55c998c19470 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4675af16969e6ee0 server_cookie=f84d3dae4ce7f0c6 in_seq=0 out_seq=0 2024-09-21T06:07:41.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.252+0000 7f7605ffb640 10 client.17823 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:41.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.253+0000 7f7605ffb640 1 -- 172.21.15.117:0/242368849 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c998b028a0 con 0x55c99877ada0 2024-09-21T06:07:41.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.253+0000 7f7605ffb640 10 client.17823 handle_command_reply: tid=0 2024-09-21T06:07:41.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.253+0000 7f762b3f4740 1 client.17823 shutdown 2024-09-21T06:07:41.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.253+0000 7f762b3f4740 1 -- 172.21.15.117:0/242368849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9987c6390 msgr2=0x55c998b9e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:41.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.253+0000 7f762b3f4740 1 --2- 172.21.15.117:0/242368849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9987c6390 0x55c998b9e430 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f7614014500 tx=0x7f76140149c0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.253+0000 7f762b3f4740 1 -- 172.21.15.117:0/242368849 shutdown_connections 2024-09-21T06:07:41.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.253+0000 7f762b3f4740 1 --2- 172.21.15.117:0/242368849 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c99877ada0 0x55c998c19470 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.253+0000 7f762b3f4740 1 --2- 172.21.15.117:0/242368849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c9987c6390 0x55c998b9e430 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.253+0000 7f762b3f4740 1 --2- 172.21.15.117:0/242368849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9987e7070 0x55c998b9e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.253+0000 7f762b3f4740 1 --2- 172.21.15.117:0/242368849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c9987be1b0 0x55c998b9def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.253+0000 7f762b3f4740 1 -- 172.21.15.117:0/242368849 >> 172.21.15.117:0/242368849 conn(0x55c9987db460 msgr2=0x55c998b8ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:41.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.253+0000 7f7605ffb640 0 client.17823 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:41.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.253+0000 7f762b3f4740 1 -- 172.21.15.117:0/242368849 shutdown_connections 2024-09-21T06:07:41.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.253+0000 7f762b3f4740 1 -- 172.21.15.117:0/242368849 wait complete. 2024-09-21T06:07:41.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.253+0000 7f762b3f4740 20 client.17823 trim_cache size 0 max 16384 2024-09-21T06:07:41.261 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:07:41.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.254+0000 7f7624e80640 1 -- 172.21.15.117:0/3288207478 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f76080b6700 msgr2=0x7f76080b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:41.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.254+0000 7f7624e80640 1 --2- 172.21.15.117:0/3288207478 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f76080b6700 0x7f76080b8c10 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7618009440 tx=0x7f7618009a40 comp rx=0 tx=0).stop 2024-09-21T06:07:41.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.254+0000 7f7624e80640 1 -- 172.21.15.117:0/3288207478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7620102a80 msgr2=0x7f762010ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:41.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.254+0000 7f7624e80640 1 --2- 172.21.15.117:0/3288207478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7620102a80 0x7f762010ffd0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f7610006010 tx=0x7f76100042f0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.254+0000 7f7624e80640 1 -- 172.21.15.117:0/3288207478 shutdown_connections 2024-09-21T06:07:41.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.254+0000 7f7624e80640 1 --2- 172.21.15.117:0/3288207478 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f76080b6700 0x7f76080b8c10 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.254+0000 7f7624e80640 1 --2- 172.21.15.117:0/3288207478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7620108d10 0x7f7620110a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.254+0000 7f7624e80640 1 --2- 172.21.15.117:0/3288207478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76201083c0 0x7f7620110510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.254+0000 7f7624e80640 1 --2- 172.21.15.117:0/3288207478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7620102a80 0x7f762010ffd0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:41.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.254+0000 7f7624e80640 1 -- 172.21.15.117:0/3288207478 >> 172.21.15.117:0/3288207478 conn(0x7f7620079740 msgr2=0x7f76201015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:41.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.254+0000 7f7624e80640 1 -- 172.21.15.117:0/3288207478 shutdown_connections 2024-09-21T06:07:41.263 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:41.254+0000 7f7624e80640 1 -- 172.21.15.117:0/3288207478 wait complete. 2024-09-21T06:07:41.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:41 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:07:41.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:41 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:07:41.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:41 smithi192 ceph-mon[34700]: pgmap v645: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 12 op/s 2024-09-21T06:07:41.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:41 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:41.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:41 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:41.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:41 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:07:42.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:41 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:07:42.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:41 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:07:42.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:41 smithi117 ceph-mon[30947]: pgmap v645: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 12 op/s 2024-09-21T06:07:42.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:41 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:42.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:41 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:42.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:41 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:07:42.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:41 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:07:42.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:41 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:07:42.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:41 smithi138 ceph-mon[34391]: pgmap v645: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 12 op/s 2024-09-21T06:07:42.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:41 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:07:42.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:41 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:07:42.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:41 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:07:43.123 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:07:43.123 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:07:43.385 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:07:43.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.856+0000 7f844db89640 1 -- 172.21.15.117:0/3924166924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f844810de40 msgr2=0x7f8448110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:43.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.856+0000 7f844db89640 1 --2- 172.21.15.117:0/3924166924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f844810de40 0x7f8448110330 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f8440009ca0 tx=0x7f844002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.856+0000 7f844db89640 1 -- 172.21.15.117:0/3924166924 shutdown_connections 2024-09-21T06:07:43.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.856+0000 7f844db89640 1 --2- 172.21.15.117:0/3924166924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f844810de40 0x7f8448110330 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.856+0000 7f844db89640 1 --2- 172.21.15.117:0/3924166924 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8448105330 0x7f844810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.856+0000 7f844db89640 1 --2- 172.21.15.117:0/3924166924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84481049e0 0x7f8448104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.856+0000 7f844db89640 1 -- 172.21.15.117:0/3924166924 >> 172.21.15.117:0/3924166924 conn(0x7f8448079750 msgr2=0x7f8448079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:43.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.856+0000 7f844db89640 1 -- 172.21.15.117:0/3924166924 shutdown_connections 2024-09-21T06:07:43.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.856+0000 7f844db89640 1 -- 172.21.15.117:0/3924166924 wait complete. 2024-09-21T06:07:43.881 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.856+0000 7f844db89640 1 Processor -- start 2024-09-21T06:07:43.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.857+0000 7f844db89640 1 -- start start 2024-09-21T06:07:43.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.857+0000 7f844db89640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84481049e0 0x7f844819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:43.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.857+0000 7f844db89640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8448105330 0x7f844819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:43.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.857+0000 7f844db89640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f844810de40 0x7f844819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:43.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.857+0000 7f844db89640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8448113ac0 con 0x7f84481049e0 2024-09-21T06:07:43.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.857+0000 7f844db89640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8448113940 con 0x7f844810de40 2024-09-21T06:07:43.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.857+0000 7f844db89640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8448113c40 con 0x7f8448105330 2024-09-21T06:07:43.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.857+0000 7f84477fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84481049e0 0x7f844819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:43.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.857+0000 7f84477fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84481049e0 0x7f844819a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42042/0 (socket says 172.21.15.117:42042) 2024-09-21T06:07:43.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.857+0000 7f84477fe640 1 -- 172.21.15.117:0/1383954275 learned_addr learned my addr 172.21.15.117:0/1383954275 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:43.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.857+0000 7f8447fff640 1 --2- 172.21.15.117:0/1383954275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f844810de40 0x7f844819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:43.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.857+0000 7f8446ffd640 1 --2- 172.21.15.117:0/1383954275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8448105330 0x7f844819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:43.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.857+0000 7f84477fe640 1 -- 172.21.15.117:0/1383954275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8448105330 msgr2=0x7f844819aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:43.882 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.857+0000 7f84477fe640 1 --2- 172.21.15.117:0/1383954275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8448105330 0x7f844819aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.857+0000 7f84477fe640 1 -- 172.21.15.117:0/1383954275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f844810de40 msgr2=0x7f844819fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:43.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.857+0000 7f84477fe640 1 --2- 172.21.15.117:0/1383954275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f844810de40 0x7f844819fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.857+0000 7f84477fe640 1 -- 172.21.15.117:0/1383954275 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84481a0400 con 0x7f84481049e0 2024-09-21T06:07:43.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.858+0000 7f84477fe640 1 --2- 172.21.15.117:0/1383954275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84481049e0 0x7f844819a490 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f843800d770 tx=0x7f843800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:43.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.858+0000 7f8444ff9640 1 -- 172.21.15.117:0/1383954275 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8438017070 con 0x7f84481049e0 2024-09-21T06:07:43.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.858+0000 7f8444ff9640 1 -- 172.21.15.117:0/1383954275 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8438009c30 con 0x7f84481049e0 2024-09-21T06:07:43.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.858+0000 7f844db89640 1 -- 172.21.15.117:0/1383954275 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84481a06f0 con 0x7f84481049e0 2024-09-21T06:07:43.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.858+0000 7f8444ff9640 1 -- 172.21.15.117:0/1383954275 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8438003cc0 con 0x7f84481049e0 2024-09-21T06:07:43.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.858+0000 7f844db89640 1 -- 172.21.15.117:0/1383954275 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84481a6e50 con 0x7f84481049e0 2024-09-21T06:07:43.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.861+0000 7f84540fd740 1 Processor -- start 2024-09-21T06:07:43.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.861+0000 7f84540fd740 1 -- start start 2024-09-21T06:07:43.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.861+0000 7f84540fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1fe28a350 0x55c1fe631430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:43.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.861+0000 7f8444ff9640 1 -- 172.21.15.117:0/1383954275 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f8438003e60 con 0x7f84481049e0 2024-09-21T06:07:43.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.861+0000 7f84540fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1fe261490 0x55c1fe631970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:43.883 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.861+0000 7f84540fd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1fe269670 0x55c1fe631eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:43.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.861+0000 7f84540fd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c1fe5a7e60 con 0x55c1fe261490 2024-09-21T06:07:43.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.861+0000 7f84540fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c1fe597290 con 0x55c1fe28a350 2024-09-21T06:07:43.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.861+0000 7f84540fd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c1fe59b0a0 con 0x55c1fe269670 2024-09-21T06:07:43.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.861+0000 7f84477fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1fe261490 0x55c1fe631970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:43.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.861+0000 7f84477fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1fe261490 0x55c1fe631970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42048/0 (socket says 172.21.15.117:42048) 2024-09-21T06:07:43.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.861+0000 7f84477fe640 1 -- 172.21.15.117:0/2147200838 learned_addr learned my addr 172.21.15.117:0/2147200838 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:43.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.862+0000 7f8447fff640 1 --2- 172.21.15.117:0/2147200838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1fe28a350 0x55c1fe631430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:43.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.862+0000 7f8446ffd640 1 --2- 172.21.15.117:0/2147200838 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1fe269670 0x55c1fe631eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:43.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.862+0000 7f84477fe640 1 -- 172.21.15.117:0/2147200838 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1fe269670 msgr2=0x55c1fe631eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:43.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.862+0000 7f84477fe640 1 --2- 172.21.15.117:0/2147200838 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1fe269670 0x55c1fe631eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.862+0000 7f84477fe640 1 -- 172.21.15.117:0/2147200838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1fe28a350 msgr2=0x55c1fe631430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:43.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.862+0000 7f84477fe640 1 --2- 172.21.15.117:0/2147200838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1fe28a350 0x55c1fe631430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.862+0000 7f84477fe640 1 -- 172.21.15.117:0/2147200838 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1fe2f5930 con 0x55c1fe261490 2024-09-21T06:07:43.884 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.862+0000 7f84477fe640 1 --2- 172.21.15.117:0/2147200838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1fe261490 0x55c1fe631970 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f84380e46a0 tx=0x7f84380eaf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dad91b8c5bedf1b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:43.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.862+0000 7f8444ff9640 1 --2- 172.21.15.117:0/1383954275 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f84300b6680 0x7f84300b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:43.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.862+0000 7f842dffb640 1 -- 172.21.15.117:0/2147200838 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84380fa070 con 0x55c1fe261490 2024-09-21T06:07:43.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.862+0000 7f842dffb640 1 -- 172.21.15.117:0/2147200838 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84380ac9f0 con 0x55c1fe261490 2024-09-21T06:07:43.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.862+0000 7f842dffb640 1 -- 172.21.15.117:0/2147200838 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84380accb0 con 0x55c1fe261490 2024-09-21T06:07:43.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.862+0000 7f8447fff640 1 --2- 172.21.15.117:0/1383954275 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f84300b6680 0x7f84300b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:43.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.862+0000 7f8444ff9640 1 -- 172.21.15.117:0/1383954275 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f84380e6f90 con 0x7f84481049e0 2024-09-21T06:07:43.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.863+0000 7f84540fd740 1 -- 172.21.15.117:0/2147200838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1fe261490 msgr2=0x55c1fe631970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:43.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.863+0000 7f8447fff640 1 --2- 172.21.15.117:0/1383954275 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f84300b6680 0x7f84300b8b90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f844002f500 tx=0x7f844002f370 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:43.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.863+0000 7f84540fd740 1 --2- 172.21.15.117:0/2147200838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1fe261490 0x55c1fe631970 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f84380e46a0 tx=0x7f84380eaf90 comp rx=0 tx=0).stop 2024-09-21T06:07:43.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.863+0000 7f84540fd740 1 -- 172.21.15.117:0/2147200838 shutdown_connections 2024-09-21T06:07:43.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.863+0000 7f84540fd740 1 --2- 172.21.15.117:0/2147200838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1fe261490 0x55c1fe631970 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.863+0000 7f84540fd740 1 --2- 172.21.15.117:0/2147200838 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1fe269670 0x55c1fe631eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.863+0000 7f84540fd740 1 --2- 172.21.15.117:0/2147200838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1fe28a350 0x55c1fe631430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.863+0000 7f84540fd740 1 -- 172.21.15.117:0/2147200838 >> 172.21.15.117:0/2147200838 conn(0x55c1fe27e740 msgr2=0x55c1fe62f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:43.885 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.863+0000 7f84540fd740 1 -- 172.21.15.117:0/2147200838 shutdown_connections 2024-09-21T06:07:43.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.863+0000 7f84540fd740 1 -- 172.21.15.117:0/2147200838 wait complete. 2024-09-21T06:07:43.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.864+0000 7f84540fd740 1 -- start start 2024-09-21T06:07:43.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.864+0000 7f84540fd740 1 Processor -- start 2024-09-21T06:07:43.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.864+0000 7f84540fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1fe261490 0x55c1fe63d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:43.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.864+0000 7f84540fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1fe269670 0x55c1fe63d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:43.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.864+0000 7f84540fd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1fe28a350 0x55c1fe648f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:43.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.864+0000 7f84540fd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c1fe3f8890 con 0x55c1fe269670 2024-09-21T06:07:43.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.864+0000 7f84540fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c1fe531d40 con 0x55c1fe261490 2024-09-21T06:07:43.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.864+0000 7f84540fd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c1fe63fc10 con 0x55c1fe28a350 2024-09-21T06:07:43.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.864+0000 7f84540fd740 10 client.0 _ll_register_callbacks cb 0x55c1fe595c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:43.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.864+0000 7f8446ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1fe269670 0x55c1fe63d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:43.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.864+0000 7f8446ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1fe269670 0x55c1fe63d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42058/0 (socket says 172.21.15.117:42058) 2024-09-21T06:07:43.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.864+0000 7f8446ffd640 1 -- 172.21.15.117:0/3608535995 learned_addr learned my addr 172.21.15.117:0/3608535995 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:43.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.864+0000 7f84477fe640 1 --2- 172.21.15.117:0/3608535995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1fe261490 0x55c1fe63d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:43.886 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.865+0000 7f8447fff640 1 --2- 172.21.15.117:0/3608535995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1fe28a350 0x55c1fe648f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:43.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.865+0000 7f8446ffd640 1 -- 172.21.15.117:0/3608535995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1fe28a350 msgr2=0x55c1fe648f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:43.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.865+0000 7f8446ffd640 1 --2- 172.21.15.117:0/3608535995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1fe28a350 0x55c1fe648f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.865+0000 7f8446ffd640 1 -- 172.21.15.117:0/3608535995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1fe261490 msgr2=0x55c1fe63d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:43.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.865+0000 7f8446ffd640 1 --2- 172.21.15.117:0/3608535995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1fe261490 0x55c1fe63d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.865+0000 7f8446ffd640 1 -- 172.21.15.117:0/3608535995 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1fe649570 con 0x55c1fe269670 2024-09-21T06:07:43.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.865+0000 7f8446ffd640 1 --2- 172.21.15.117:0/3608535995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1fe269670 0x55c1fe63d600 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f843c00b990 tx=0x7f843c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e2d8f52a9bd0af server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:43.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.865+0000 7f8426ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:43.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.865+0000 7f8426ffd640 1 -- 172.21.15.117:0/3608535995 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c005020 con 0x55c1fe269670 2024-09-21T06:07:43.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.865+0000 7f8426ffd640 1 -- 172.21.15.117:0/3608535995 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f843c0051c0 con 0x55c1fe269670 2024-09-21T06:07:43.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.865+0000 7f84540fd740 1 -- 172.21.15.117:0/3608535995 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84380ac9f0 con 0x55c1fe269670 2024-09-21T06:07:43.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.865+0000 7f8426ffd640 1 -- 172.21.15.117:0/3608535995 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f843c0053c0 con 0x55c1fe269670 2024-09-21T06:07:43.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.865+0000 7f84540fd740 10 client.17835 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:43.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.865+0000 7f84540fd740 1 -- 172.21.15.117:0/3608535995 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c1fe64bf00 con 0x55c1fe269670 2024-09-21T06:07:43.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.865+0000 7f8426ffd640 1 -- 172.21.15.117:0/3608535995 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84380ac9f0 con 0x55c1fe269670 2024-09-21T06:07:43.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.866+0000 7f8426ffd640 1 -- 172.21.15.117:0/3608535995 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f843c01b590 con 0x55c1fe269670 2024-09-21T06:07:43.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.866+0000 7f84540fd740 10 client.17835 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:43.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.866+0000 7f84540fd740 10 client.17835 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:07:43.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.866+0000 7f84540fd740 20 client.17835 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:43.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.866+0000 7f84540fd740 1 --2- 172.21.15.117:0/3608535995 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c1fe248cd0 0x55c1fe64f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:43.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.866+0000 7f84540fd740 4 client.17835 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:07:43.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.866+0000 7f84540fd740 1 -- 172.21.15.117:0/3608535995 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1fe5a9db0 con 0x55c1fe248cd0 2024-09-21T06:07:43.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.866+0000 7f84477fe640 1 --2- 172.21.15.117:0/3608535995 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c1fe248cd0 0x55c1fe64f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:43.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.866+0000 7f84477fe640 1 --2- 172.21.15.117:0/3608535995 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c1fe248cd0 0x55c1fe64f3f0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23e6420ab45b1ea8 server_cookie=91984edd066443c5 in_seq=0 out_seq=0 2024-09-21T06:07:43.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.866+0000 7f8426ffd640 10 client.17835 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:43.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.867+0000 7f8426ffd640 1 -- 172.21.15.117:0/3608535995 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f8438086e50 con 0x55c1fe248cd0 2024-09-21T06:07:43.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.867+0000 7f8426ffd640 10 client.17835 handle_command_reply: tid=0 2024-09-21T06:07:43.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.867+0000 7f84540fd740 1 client.17835 shutdown 2024-09-21T06:07:43.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.867+0000 7f84540fd740 1 -- 172.21.15.117:0/3608535995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1fe269670 msgr2=0x55c1fe63d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:43.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.867+0000 7f84540fd740 1 --2- 172.21.15.117:0/3608535995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1fe269670 0x55c1fe63d600 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f843c00b990 tx=0x7f843c00be50 comp rx=0 tx=0).stop 2024-09-21T06:07:43.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.867+0000 7f84540fd740 1 -- 172.21.15.117:0/3608535995 shutdown_connections 2024-09-21T06:07:43.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.867+0000 7f84540fd740 1 --2- 172.21.15.117:0/3608535995 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c1fe248cd0 0x55c1fe64f3f0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.867+0000 7f84540fd740 1 --2- 172.21.15.117:0/3608535995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1fe269670 0x55c1fe63d600 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.867+0000 7f84540fd740 1 --2- 172.21.15.117:0/3608535995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1fe28a350 0x55c1fe648f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.867+0000 7f8426ffd640 0 client.17835 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:43.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.867+0000 7f84540fd740 1 --2- 172.21.15.117:0/3608535995 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1fe261490 0x55c1fe63d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.867+0000 7f84540fd740 1 -- 172.21.15.117:0/3608535995 >> 172.21.15.117:0/3608535995 conn(0x55c1fe27e740 msgr2=0x55c1fe6345c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:43.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.868+0000 7f84540fd740 1 -- 172.21.15.117:0/3608535995 shutdown_connections 2024-09-21T06:07:43.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.868+0000 7f84540fd740 1 -- 172.21.15.117:0/3608535995 wait complete. 2024-09-21T06:07:43.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.868+0000 7f84540fd740 20 client.17835 trim_cache size 0 max 16384 2024-09-21T06:07:43.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.885+0000 7f84540fd740 1 Processor -- start 2024-09-21T06:07:43.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.885+0000 7f84540fd740 1 -- start start 2024-09-21T06:07:43.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.885+0000 7f84540fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1fe248cd0 0x55c1fe6364c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:43.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.885+0000 7f84540fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1fe261490 0x55c1fe636a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:43.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.885+0000 7f84540fd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1fe269670 0x55c1fe634530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:43.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.885+0000 7f84540fd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f84481af630 con 0x55c1fe261490 2024-09-21T06:07:43.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.885+0000 7f84540fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f84481b1170 con 0x55c1fe248cd0 2024-09-21T06:07:43.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.885+0000 7f84540fd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f84481afd00 con 0x55c1fe269670 2024-09-21T06:07:43.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.886+0000 7f8446ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1fe261490 0x55c1fe636a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:43.893 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.886+0000 7f8446ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1fe261490 0x55c1fe636a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42068/0 (socket says 172.21.15.117:42068) 2024-09-21T06:07:43.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.886+0000 7f8446ffd640 1 -- 172.21.15.117:0/2334499299 learned_addr learned my addr 172.21.15.117:0/2334499299 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:43.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.886+0000 7f84477fe640 1 --2- 172.21.15.117:0/2334499299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1fe248cd0 0x55c1fe6364c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:43.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.886+0000 7f8446ffd640 1 -- 172.21.15.117:0/2334499299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1fe269670 msgr2=0x55c1fe634530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:43.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.886+0000 7f8446ffd640 1 --2- 172.21.15.117:0/2334499299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1fe269670 0x55c1fe634530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.886+0000 7f8446ffd640 1 -- 172.21.15.117:0/2334499299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1fe248cd0 msgr2=0x55c1fe6364c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:43.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.886+0000 7f8446ffd640 1 --2- 172.21.15.117:0/2334499299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1fe248cd0 0x55c1fe6364c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.886+0000 7f8446ffd640 1 -- 172.21.15.117:0/2334499299 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1fe62fc10 con 0x55c1fe261490 2024-09-21T06:07:43.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.886+0000 7f8446ffd640 1 --2- 172.21.15.117:0/2334499299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1fe261490 0x55c1fe636a00 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f843c00b6b0 tx=0x7f843c005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db8b9a9366779ad3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:43.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.886+0000 7f842d7fa640 1 -- 172.21.15.117:0/2334499299 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c005560 con 0x55c1fe261490 2024-09-21T06:07:43.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.886+0000 7f842d7fa640 1 -- 172.21.15.117:0/2334499299 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f843c01bba0 con 0x55c1fe261490 2024-09-21T06:07:43.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.886+0000 7f842d7fa640 1 -- 172.21.15.117:0/2334499299 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c011b50 con 0x55c1fe261490 2024-09-21T06:07:43.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.887+0000 7f84540fd740 1 -- 172.21.15.117:0/2334499299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1fe261490 msgr2=0x55c1fe636a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:43.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.887+0000 7f84540fd740 1 --2- 172.21.15.117:0/2334499299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1fe261490 0x55c1fe636a00 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f843c00b6b0 tx=0x7f843c005c50 comp rx=0 tx=0).stop 2024-09-21T06:07:43.894 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.887+0000 7f84540fd740 1 -- 172.21.15.117:0/2334499299 shutdown_connections 2024-09-21T06:07:43.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.887+0000 7f84540fd740 1 --2- 172.21.15.117:0/2334499299 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1fe261490 0x55c1fe636a00 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.887+0000 7f84540fd740 1 --2- 172.21.15.117:0/2334499299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1fe269670 0x55c1fe634530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.887+0000 7f84540fd740 1 --2- 172.21.15.117:0/2334499299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1fe248cd0 0x55c1fe6364c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.887+0000 7f84540fd740 1 -- 172.21.15.117:0/2334499299 >> 172.21.15.117:0/2334499299 conn(0x55c1fe27e740 msgr2=0x55c1fe63b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:43.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.887+0000 7f84540fd740 1 -- 172.21.15.117:0/2334499299 shutdown_connections 2024-09-21T06:07:43.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.887+0000 7f84540fd740 1 -- 172.21.15.117:0/2334499299 wait complete. 2024-09-21T06:07:43.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.887+0000 7f84540fd740 1 -- start start 2024-09-21T06:07:43.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.888+0000 7f84540fd740 1 Processor -- start 2024-09-21T06:07:43.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.888+0000 7f84540fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1fe248cd0 0x55c1fe6453a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:43.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.888+0000 7f84540fd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1fe261490 0x55c1fe641860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:43.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.888+0000 7f84540fd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1fe269670 0x55c1fe641da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:43.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.888+0000 7f84540fd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f84481b0aa0 con 0x55c1fe269670 2024-09-21T06:07:43.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.888+0000 7f84540fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f84481b03d0 con 0x55c1fe248cd0 2024-09-21T06:07:43.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.888+0000 7f84540fd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f84481ad630 con 0x55c1fe261490 2024-09-21T06:07:43.895 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.888+0000 7f84540fd740 10 client.0 _ll_register_callbacks cb 0x7f843c0042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:43.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.888+0000 7f8447fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1fe269670 0x55c1fe641da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:43.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.888+0000 7f8447fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1fe269670 0x55c1fe641da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42072/0 (socket says 172.21.15.117:42072) 2024-09-21T06:07:43.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.888+0000 7f8447fff640 1 -- 172.21.15.117:0/3593418033 learned_addr learned my addr 172.21.15.117:0/3593418033 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:43.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.888+0000 7f84477fe640 1 --2- 172.21.15.117:0/3593418033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1fe248cd0 0x55c1fe6453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:43.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.888+0000 7f8446ffd640 1 --2- 172.21.15.117:0/3593418033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1fe261490 0x55c1fe641860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:43.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.888+0000 7f8447fff640 1 -- 172.21.15.117:0/3593418033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1fe261490 msgr2=0x55c1fe641860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:43.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.888+0000 7f8447fff640 1 --2- 172.21.15.117:0/3593418033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1fe261490 0x55c1fe641860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.888+0000 7f8447fff640 1 -- 172.21.15.117:0/3593418033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1fe248cd0 msgr2=0x55c1fe6453a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:43.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.888+0000 7f8447fff640 1 --2- 172.21.15.117:0/3593418033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1fe248cd0 0x55c1fe6453a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.888+0000 7f8447fff640 1 -- 172.21.15.117:0/3593418033 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1fe6422e0 con 0x55c1fe269670 2024-09-21T06:07:43.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.888+0000 7f8446ffd640 1 --2- 172.21.15.117:0/3593418033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1fe261490 0x55c1fe641860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:07:43.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.889+0000 7f8447fff640 1 --2- 172.21.15.117:0/3593418033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1fe269670 0x55c1fe641da0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f84400316f0 tx=0x7f8440031720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe6b54d9f7ac49a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:43.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.889+0000 7f8426ffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:43.896 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.889+0000 7f8426ffd640 1 -- 172.21.15.117:0/3593418033 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8440004040 con 0x55c1fe269670 2024-09-21T06:07:43.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.889+0000 7f8426ffd640 1 -- 172.21.15.117:0/3593418033 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f8440042030 con 0x55c1fe269670 2024-09-21T06:07:43.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.889+0000 7f84540fd740 1 -- 172.21.15.117:0/3593418033 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f843c01bba0 con 0x55c1fe269670 2024-09-21T06:07:43.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.889+0000 7f8426ffd640 1 -- 172.21.15.117:0/3593418033 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f84400041e0 con 0x55c1fe269670 2024-09-21T06:07:43.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.889+0000 7f84540fd740 10 client.17841 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:43.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.889+0000 7f84540fd740 1 -- 172.21.15.117:0/3593418033 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c1fe6baa80 con 0x55c1fe269670 2024-09-21T06:07:43.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.889+0000 7f8426ffd640 1 -- 172.21.15.117:0/3593418033 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f843c01bba0 con 0x55c1fe269670 2024-09-21T06:07:43.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.889+0000 7f8426ffd640 1 -- 172.21.15.117:0/3593418033 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f84400305c0 con 0x55c1fe269670 2024-09-21T06:07:43.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.889+0000 7f84540fd740 10 client.17841 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:43.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.889+0000 7f84540fd740 10 client.17841 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:07:43.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.889+0000 7f84540fd740 20 client.17841 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:43.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.889+0000 7f84540fd740 1 --2- 172.21.15.117:0/3593418033 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c1fe28a350 0x55c1fe6bd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:43.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.889+0000 7f84540fd740 4 client.17841 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:07:43.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.889+0000 7f84540fd740 1 -- 172.21.15.117:0/3593418033 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c1fe5a5860 con 0x55c1fe28a350 2024-09-21T06:07:43.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.890+0000 7f84477fe640 1 --2- 172.21.15.117:0/3593418033 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c1fe28a350 0x55c1fe6bd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:43.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.890+0000 7f8426ffd640 10 client.17841 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:43.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.890+0000 7f84477fe640 1 --2- 172.21.15.117:0/3593418033 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c1fe28a350 0x55c1fe6bd2a0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7642ae924e9d2f66 server_cookie=bf9859cc58006c55 in_seq=0 out_seq=0 2024-09-21T06:07:43.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.892+0000 7f8426ffd640 1 -- 172.21.15.117:0/3593418033 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c1fe5a5860 con 0x55c1fe28a350 2024-09-21T06:07:43.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.892+0000 7f8426ffd640 10 client.17841 handle_command_reply: tid=0 2024-09-21T06:07:43.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.892+0000 7f84540fd740 1 client.17841 shutdown 2024-09-21T06:07:43.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.892+0000 7f84540fd740 1 -- 172.21.15.117:0/3593418033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1fe269670 msgr2=0x55c1fe641da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:43.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.892+0000 7f84540fd740 1 --2- 172.21.15.117:0/3593418033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1fe269670 0x55c1fe641da0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f84400316f0 tx=0x7f8440031720 comp rx=0 tx=0).stop 2024-09-21T06:07:43.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.892+0000 7f84540fd740 1 -- 172.21.15.117:0/3593418033 shutdown_connections 2024-09-21T06:07:43.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.892+0000 7f84540fd740 1 --2- 172.21.15.117:0/3593418033 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55c1fe28a350 0x55c1fe6bd2a0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.892+0000 7f84540fd740 1 --2- 172.21.15.117:0/3593418033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c1fe269670 0x55c1fe641da0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.892+0000 7f84540fd740 1 --2- 172.21.15.117:0/3593418033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c1fe261490 0x55c1fe641860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.892+0000 7f84540fd740 1 --2- 172.21.15.117:0/3593418033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c1fe248cd0 0x55c1fe6453a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.892+0000 7f84540fd740 1 -- 172.21.15.117:0/3593418033 >> 172.21.15.117:0/3593418033 conn(0x55c1fe27e740 msgr2=0x55c1fe63b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:43.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.892+0000 7f8426ffd640 0 client.17841 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:43.899 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.892+0000 7f84540fd740 1 -- 172.21.15.117:0/3593418033 shutdown_connections 2024-09-21T06:07:43.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.892+0000 7f84540fd740 1 -- 172.21.15.117:0/3593418033 wait complete. 2024-09-21T06:07:43.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.892+0000 7f84540fd740 20 client.17841 trim_cache size 0 max 16384 2024-09-21T06:07:43.900 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T06:07:43.900 INFO:teuthology.orchestra.run.smithi117.stdout: "return_code": 0, 2024-09-21T06:07:43.900 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_tag": "84be3416-2285-4046-adf2-8d335592d29f", 2024-09-21T06:07:43.900 INFO:teuthology.orchestra.run.smithi117.stdout: "mode": "asynchronous" 2024-09-21T06:07:43.900 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T06:07:43.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.893+0000 7f844db89640 1 -- 172.21.15.117:0/1383954275 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f84300b6680 msgr2=0x7f84300b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:43.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.893+0000 7f844db89640 1 --2- 172.21.15.117:0/1383954275 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f84300b6680 0x7f84300b8b90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f844002f500 tx=0x7f844002f370 comp rx=0 tx=0).stop 2024-09-21T06:07:43.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.893+0000 7f844db89640 1 -- 172.21.15.117:0/1383954275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84481049e0 msgr2=0x7f844819a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:43.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.893+0000 7f844db89640 1 --2- 172.21.15.117:0/1383954275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84481049e0 0x7f844819a490 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f843800d770 tx=0x7f843800dc30 comp rx=0 tx=0).stop 2024-09-21T06:07:43.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.893+0000 7f844db89640 1 -- 172.21.15.117:0/1383954275 shutdown_connections 2024-09-21T06:07:43.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.893+0000 7f844db89640 1 --2- 172.21.15.117:0/1383954275 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f84300b6680 0x7f84300b8b90 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.893+0000 7f844db89640 1 --2- 172.21.15.117:0/1383954275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f844810de40 0x7f844819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.893+0000 7f844db89640 1 --2- 172.21.15.117:0/1383954275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8448105330 0x7f844819aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.893+0000 7f844db89640 1 --2- 172.21.15.117:0/1383954275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f84481049e0 0x7f844819a490 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:43.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.893+0000 7f844db89640 1 -- 172.21.15.117:0/1383954275 >> 172.21.15.117:0/1383954275 conn(0x7f8448079750 msgr2=0x7f8448107960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:43.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.893+0000 7f844db89640 1 -- 172.21.15.117:0/1383954275 shutdown_connections 2024-09-21T06:07:43.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:43.893+0000 7f844db89640 1 -- 172.21.15.117:0/1383954275 wait complete. 2024-09-21T06:07:44.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:43 smithi117 ceph-mon[30947]: pgmap v646: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 11 op/s 2024-09-21T06:07:44.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:43 smithi192 ceph-mon[34700]: pgmap v646: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 11 op/s 2024-09-21T06:07:44.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:43 smithi138 ceph-mon[34391]: pgmap v646: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 11 op/s 2024-09-21T06:07:44.701 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:07:44.928 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:44 smithi117 ceph-mon[30947]: scrub queued for path: / 2024-09-21T06:07:44.928 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:44 smithi117 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-21T06:07:44.928 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:44 smithi117 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-21T06:07:44.928 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:44 smithi117 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-21T06:07:44.950 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:07:45.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:44 smithi192 ceph-mon[34700]: scrub queued for path: / 2024-09-21T06:07:45.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:44 smithi192 ceph-mon[34700]: scrub summary: idle+waiting paths [/] 2024-09-21T06:07:45.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:44 smithi192 ceph-mon[34700]: scrub summary: active paths [/] 2024-09-21T06:07:45.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:44 smithi192 ceph-mon[34700]: scrub summary: idle+waiting paths [/] 2024-09-21T06:07:45.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:44 smithi138 ceph-mon[34391]: scrub queued for path: / 2024-09-21T06:07:45.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:44 smithi138 ceph-mon[34391]: scrub summary: idle+waiting paths [/] 2024-09-21T06:07:45.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:44 smithi138 ceph-mon[34391]: scrub summary: active paths [/] 2024-09-21T06:07:45.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:44 smithi138 ceph-mon[34391]: scrub summary: idle+waiting paths [/] 2024-09-21T06:07:45.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.492+0000 7f4c70a18640 1 -- 172.21.15.117:0/1650635446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c6c108d00 msgr2=0x7f4c6c10f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:45.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.492+0000 7f4c70a18640 1 --2- 172.21.15.117:0/1650635446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c6c108d00 0x7f4c6c10f530 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f4c64009ca0 tx=0x7f4c6402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:07:45.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.492+0000 7f4c70a18640 1 -- 172.21.15.117:0/1650635446 shutdown_connections 2024-09-21T06:07:45.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.492+0000 7f4c70a18640 1 --2- 172.21.15.117:0/1650635446 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c6c108d00 0x7f4c6c10f530 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:45.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.492+0000 7f4c70a18640 1 --2- 172.21.15.117:0/1650635446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c6c1083b0 0x7f4c6c1087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:45.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.492+0000 7f4c70a18640 1 --2- 172.21.15.117:0/1650635446 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c6c102a70 0x7f4c6c102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:45.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.492+0000 7f4c70a18640 1 -- 172.21.15.117:0/1650635446 >> 172.21.15.117:0/1650635446 conn(0x7f4c6c0796e0 msgr2=0x7f4c6c079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:45.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.492+0000 7f4c70a18640 1 -- 172.21.15.117:0/1650635446 shutdown_connections 2024-09-21T06:07:45.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.492+0000 7f4c70a18640 1 -- 172.21.15.117:0/1650635446 wait complete. 2024-09-21T06:07:45.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.492+0000 7f4c70a18640 1 Processor -- start 2024-09-21T06:07:45.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.493+0000 7f4c70a18640 1 -- start start 2024-09-21T06:07:45.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.493+0000 7f4c70a18640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c6c102a70 0x7f4c6c1a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:45.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.493+0000 7f4c70a18640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c6c1083b0 0x7f4c6c1a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:45.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.493+0000 7f4c70a18640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c6c108d00 0x7f4c6c1ac800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:45.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.493+0000 7f4c70a18640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6c113ae0 con 0x7f4c6c1083b0 2024-09-21T06:07:45.501 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.493+0000 7f4c70a18640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6c113960 con 0x7f4c6c108d00 2024-09-21T06:07:45.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.493+0000 7f4c70a18640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6c113c60 con 0x7f4c6c102a70 2024-09-21T06:07:45.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.493+0000 7f4c61d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c6c1083b0 0x7f4c6c1a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:45.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.493+0000 7f4c61d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c6c1083b0 0x7f4c6c1a1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36168/0 (socket says 172.21.15.117:36168) 2024-09-21T06:07:45.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.493+0000 7f4c6a59c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c6c102a70 0x7f4c6c1a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:45.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.493+0000 7f4c61d9b640 1 -- 172.21.15.117:0/1177927570 learned_addr learned my addr 172.21.15.117:0/1177927570 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:45.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.493+0000 7f4c6ad9d640 1 --2- 172.21.15.117:0/1177927570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c6c108d00 0x7f4c6c1ac800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:45.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.493+0000 7f4c61d9b640 1 -- 172.21.15.117:0/1177927570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c6c102a70 msgr2=0x7f4c6c1a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:45.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.493+0000 7f4c61d9b640 1 --2- 172.21.15.117:0/1177927570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c6c102a70 0x7f4c6c1a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:45.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.493+0000 7f4c61d9b640 1 -- 172.21.15.117:0/1177927570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c6c108d00 msgr2=0x7f4c6c1ac800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:45.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.493+0000 7f4c61d9b640 1 --2- 172.21.15.117:0/1177927570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c6c108d00 0x7f4c6c1ac800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:45.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.493+0000 7f4c61d9b640 1 -- 172.21.15.117:0/1177927570 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c6c1a2500 con 0x7f4c6c1083b0 2024-09-21T06:07:45.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.494+0000 7f4c61d9b640 1 --2- 172.21.15.117:0/1177927570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c6c1083b0 0x7f4c6c1a1cc0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f4c5400d740 tx=0x7f4c5400db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:45.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.494+0000 7f4c63fff640 1 -- 172.21.15.117:0/1177927570 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c54017070 con 0x7f4c6c1083b0 2024-09-21T06:07:45.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.494+0000 7f4c70a18640 1 -- 172.21.15.117:0/1177927570 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c6c1acda0 con 0x7f4c6c1083b0 2024-09-21T06:07:45.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.494+0000 7f4c63fff640 1 -- 172.21.15.117:0/1177927570 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c54010460 con 0x7f4c6c1083b0 2024-09-21T06:07:45.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.494+0000 7f4c63fff640 1 -- 172.21.15.117:0/1177927570 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5400f600 con 0x7f4c6c1083b0 2024-09-21T06:07:45.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.494+0000 7f4c70a18640 1 -- 172.21.15.117:0/1177927570 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c6c1ad200 con 0x7f4c6c1083b0 2024-09-21T06:07:45.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.496+0000 7f4c76f8c740 1 Processor -- start 2024-09-21T06:07:45.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.496+0000 7f4c63fff640 1 -- 172.21.15.117:0/1177927570 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f4c5401a070 con 0x7f4c6c1083b0 2024-09-21T06:07:45.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.497+0000 7f4c76f8c740 1 -- start start 2024-09-21T06:07:45.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.497+0000 7f4c76f8c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1e8de9070 0x55d1e9190270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:45.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.497+0000 7f4c76f8c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1e8dc01b0 0x55d1e91907b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:45.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.497+0000 7f4c76f8c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e8dc8390 0x55d1e9190cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:45.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.497+0000 7f4c76f8c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d1e9106e50 con 0x55d1e8dc8390 2024-09-21T06:07:45.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.497+0000 7f4c76f8c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d1e90f6280 con 0x55d1e8de9070 2024-09-21T06:07:45.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.497+0000 7f4c76f8c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55d1e90fa090 con 0x55d1e8dc01b0 2024-09-21T06:07:45.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.497+0000 7f4c61d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e8dc8390 0x55d1e9190cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:45.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.497+0000 7f4c61d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e8dc8390 0x55d1e9190cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36180/0 (socket says 172.21.15.117:36180) 2024-09-21T06:07:45.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.497+0000 7f4c61d9b640 1 -- 172.21.15.117:0/3201309827 learned_addr learned my addr 172.21.15.117:0/3201309827 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:45.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.497+0000 7f4c6ad9d640 1 --2- 172.21.15.117:0/3201309827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1e8de9070 0x55d1e9190270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:45.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.497+0000 7f4c6a59c640 1 --2- 172.21.15.117:0/3201309827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1e8dc01b0 0x55d1e91907b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:45.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.497+0000 7f4c63fff640 1 --2- 172.21.15.117:0/1177927570 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4c580b6680 0x7f4c580b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:45.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.498+0000 7f4c61d9b640 1 -- 172.21.15.117:0/3201309827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1e8dc01b0 msgr2=0x55d1e91907b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:45.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.498+0000 7f4c61d9b640 1 --2- 172.21.15.117:0/3201309827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1e8dc01b0 0x55d1e91907b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:45.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.498+0000 7f4c61d9b640 1 -- 172.21.15.117:0/3201309827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1e8de9070 msgr2=0x55d1e9190270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:45.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.498+0000 7f4c61d9b640 1 --2- 172.21.15.117:0/3201309827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1e8de9070 0x55d1e9190270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:45.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.498+0000 7f4c61d9b640 1 -- 172.21.15.117:0/3201309827 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1e8e54650 con 0x55d1e8dc8390 2024-09-21T06:07:45.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.498+0000 7f4c6ad9d640 1 --2- 172.21.15.117:0/1177927570 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4c580b6680 0x7f4c580b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:45.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.498+0000 7f4c61d9b640 1 --2- 172.21.15.117:0/3201309827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e8dc8390 0x55d1e9190cf0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f4c540e4770 tx=0x7f4c540ad0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2f2fd2b71e2f19 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:45.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.498+0000 7f4c63fff640 1 -- 172.21.15.117:0/1177927570 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f4c540e7060 con 0x7f4c6c1083b0 2024-09-21T06:07:45.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.498+0000 7f4c60d99640 1 -- 172.21.15.117:0/3201309827 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c54021030 con 0x55d1e8dc8390 2024-09-21T06:07:45.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.498+0000 7f4c60d99640 1 -- 172.21.15.117:0/3201309827 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c540aca10 con 0x55d1e8dc8390 2024-09-21T06:07:45.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.498+0000 7f4c60d99640 1 -- 172.21.15.117:0/3201309827 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c540b2b70 con 0x55d1e8dc8390 2024-09-21T06:07:45.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.499+0000 7f4c76f8c740 1 -- 172.21.15.117:0/3201309827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e8dc8390 msgr2=0x55d1e9190cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:45.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.499+0000 7f4c6ad9d640 1 --2- 172.21.15.117:0/1177927570 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4c580b6680 0x7f4c580b8b90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4c64005cc0 tx=0x7f4c640095c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:45.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.499+0000 7f4c76f8c740 1 --2- 172.21.15.117:0/3201309827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e8dc8390 0x55d1e9190cf0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f4c540e4770 tx=0x7f4c540ad0f0 comp rx=0 tx=0).stop 2024-09-21T06:07:45.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.499+0000 7f4c76f8c740 1 -- 172.21.15.117:0/3201309827 shutdown_connections 2024-09-21T06:07:45.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.499+0000 7f4c76f8c740 1 --2- 172.21.15.117:0/3201309827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e8dc8390 0x55d1e9190cf0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:45.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.499+0000 7f4c76f8c740 1 --2- 172.21.15.117:0/3201309827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1e8dc01b0 0x55d1e91907b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:45.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.499+0000 7f4c76f8c740 1 --2- 172.21.15.117:0/3201309827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1e8de9070 0x55d1e9190270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:45.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.499+0000 7f4c76f8c740 1 -- 172.21.15.117:0/3201309827 >> 172.21.15.117:0/3201309827 conn(0x55d1e8ddd460 msgr2=0x55d1e918e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:45.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.499+0000 7f4c76f8c740 1 -- 172.21.15.117:0/3201309827 shutdown_connections 2024-09-21T06:07:45.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.499+0000 7f4c76f8c740 1 -- 172.21.15.117:0/3201309827 wait complete. 2024-09-21T06:07:45.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.500+0000 7f4c76f8c740 1 -- start start 2024-09-21T06:07:45.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.500+0000 7f4c76f8c740 1 Processor -- start 2024-09-21T06:07:45.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.500+0000 7f4c76f8c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e8dc01b0 0x55d1e919b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:45.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.500+0000 7f4c76f8c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1e8dc8390 0x55d1e919bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:45.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.500+0000 7f4c76f8c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1e8de9070 0x55d1e919c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:45.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.500+0000 7f4c76f8c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d1e8f57700 con 0x55d1e8dc01b0 2024-09-21T06:07:45.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.500+0000 7f4c76f8c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d1e9091110 con 0x55d1e8de9070 2024-09-21T06:07:45.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.500+0000 7f4c76f8c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55d1e919ea50 con 0x55d1e8dc8390 2024-09-21T06:07:45.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.500+0000 7f4c76f8c740 10 client.0 _ll_register_callbacks cb 0x55d1e90aa990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:45.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.500+0000 7f4c6a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e8dc01b0 0x55d1e919b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:45.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.500+0000 7f4c6a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e8dc01b0 0x55d1e919b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36186/0 (socket says 172.21.15.117:36186) 2024-09-21T06:07:45.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.500+0000 7f4c6a59c640 1 -- 172.21.15.117:0/1373768515 learned_addr learned my addr 172.21.15.117:0/1373768515 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:45.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.500+0000 7f4c61d9b640 1 --2- 172.21.15.117:0/1373768515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1e8dc8390 0x55d1e919bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:45.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.501+0000 7f4c6a59c640 1 -- 172.21.15.117:0/1373768515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1e8dc8390 msgr2=0x55d1e919bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:45.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.501+0000 7f4c6a59c640 1 --2- 172.21.15.117:0/1373768515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1e8dc8390 0x55d1e919bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:45.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.501+0000 7f4c6a59c640 1 -- 172.21.15.117:0/1373768515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1e8de9070 msgr2=0x55d1e919c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:45.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.501+0000 7f4c6a59c640 1 --2- 172.21.15.117:0/1373768515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1e8de9070 0x55d1e919c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:45.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.501+0000 7f4c6a59c640 1 -- 172.21.15.117:0/1373768515 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1e90bec40 con 0x55d1e8dc01b0 2024-09-21T06:07:45.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.501+0000 7f4c6a59c640 1 --2- 172.21.15.117:0/1373768515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e8dc01b0 0x55d1e919b850 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c010a50 tx=0x7f4c5c010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58468a0f1d0af67b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:45.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.501+0000 7f4c51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:45.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.501+0000 7f4c51ffb640 1 -- 172.21.15.117:0/1373768515 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c00fbd0 con 0x55d1e8dc01b0 2024-09-21T06:07:45.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.501+0000 7f4c51ffb640 1 -- 172.21.15.117:0/1373768515 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f4c5c00fd70 con 0x55d1e8dc01b0 2024-09-21T06:07:45.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.501+0000 7f4c76f8c740 1 -- 172.21.15.117:0/1373768515 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c540aca10 con 0x55d1e8dc01b0 2024-09-21T06:07:45.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.501+0000 7f4c51ffb640 1 -- 172.21.15.117:0/1373768515 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4c5c005020 con 0x55d1e8dc01b0 2024-09-21T06:07:45.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.501+0000 7f4c76f8c740 10 client.17853 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:45.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.501+0000 7f4c76f8c740 1 -- 172.21.15.117:0/1373768515 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d1e91aa730 con 0x55d1e8dc01b0 2024-09-21T06:07:45.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.501+0000 7f4c51ffb640 1 -- 172.21.15.117:0/1373768515 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c540aca10 con 0x55d1e8dc01b0 2024-09-21T06:07:45.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.502+0000 7f4c51ffb640 1 -- 172.21.15.117:0/1373768515 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4c5c019410 con 0x55d1e8dc01b0 2024-09-21T06:07:45.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.502+0000 7f4c76f8c740 10 client.17853 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:45.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.502+0000 7f4c76f8c740 10 client.17853 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:07:45.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.502+0000 7f4c76f8c740 20 client.17853 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:45.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.502+0000 7f4c76f8c740 1 --2- 172.21.15.117:0/1373768515 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55d1e8da79f0 0x55d1e91adda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:45.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.502+0000 7f4c76f8c740 4 client.17853 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:07:45.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.502+0000 7f4c76f8c740 1 -- 172.21.15.117:0/1373768515 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1e9108da0 con 0x55d1e8da79f0 2024-09-21T06:07:45.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.502+0000 7f4c61d9b640 1 --2- 172.21.15.117:0/1373768515 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55d1e8da79f0 0x55d1e91adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:45.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.502+0000 7f4c51ffb640 10 client.17853 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:45.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.502+0000 7f4c61d9b640 1 --2- 172.21.15.117:0/1373768515 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55d1e8da79f0 0x55d1e91adda0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb8a74a48fb9b934 server_cookie=794ce9a06bca6e90 in_seq=0 out_seq=0 2024-09-21T06:07:45.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.503+0000 7f4c51ffb640 1 -- 172.21.15.117:0/1373768515 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d1e9108da0 con 0x55d1e8da79f0 2024-09-21T06:07:45.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.503+0000 7f4c51ffb640 10 client.17853 handle_command_reply: tid=0 2024-09-21T06:07:45.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.503+0000 7f4c76f8c740 1 client.17853 shutdown 2024-09-21T06:07:45.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.503+0000 7f4c76f8c740 1 -- 172.21.15.117:0/1373768515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e8dc01b0 msgr2=0x55d1e919b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:45.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.503+0000 7f4c76f8c740 1 --2- 172.21.15.117:0/1373768515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e8dc01b0 0x55d1e919b850 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c010a50 tx=0x7f4c5c010f10 comp rx=0 tx=0).stop 2024-09-21T06:07:45.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.503+0000 7f4c76f8c740 1 -- 172.21.15.117:0/1373768515 shutdown_connections 2024-09-21T06:07:45.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.503+0000 7f4c76f8c740 1 --2- 172.21.15.117:0/1373768515 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55d1e8da79f0 0x55d1e91adda0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:45.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.503+0000 7f4c51ffb640 0 client.17853 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:45.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.503+0000 7f4c76f8c740 1 --2- 172.21.15.117:0/1373768515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1e8de9070 0x55d1e919c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:45.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.503+0000 7f4c76f8c740 1 --2- 172.21.15.117:0/1373768515 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1e8dc8390 0x55d1e919bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:45.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.503+0000 7f4c76f8c740 1 --2- 172.21.15.117:0/1373768515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e8dc01b0 0x55d1e919b850 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:45.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.503+0000 7f4c76f8c740 1 -- 172.21.15.117:0/1373768515 >> 172.21.15.117:0/1373768515 conn(0x55d1e8ddd460 msgr2=0x55d1e91912a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:45.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.504+0000 7f4c76f8c740 1 -- 172.21.15.117:0/1373768515 shutdown_connections 2024-09-21T06:07:45.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.504+0000 7f4c76f8c740 1 -- 172.21.15.117:0/1373768515 wait complete. 2024-09-21T06:07:45.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.504+0000 7f4c76f8c740 20 client.17853 trim_cache size 0 max 16384 2024-09-21T06:07:45.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.521+0000 7f4c76f8c740 1 Processor -- start 2024-09-21T06:07:45.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.521+0000 7f4c76f8c740 1 -- start start 2024-09-21T06:07:45.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.521+0000 7f4c76f8c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e8da79f0 0x55d1e919d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:45.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.521+0000 7f4c76f8c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1e8dc01b0 0x55d1e91a7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:45.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.521+0000 7f4c76f8c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1e8dc8390 0x55d1e91a7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:45.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.521+0000 7f4c76f8c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6c1b5c10 con 0x55d1e8da79f0 2024-09-21T06:07:45.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.521+0000 7f4c76f8c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6c1b7750 con 0x55d1e8dc01b0 2024-09-21T06:07:45.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.521+0000 7f4c76f8c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6c1b62e0 con 0x55d1e8dc8390 2024-09-21T06:07:45.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.521+0000 7f4c6a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e8da79f0 0x55d1e919d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:45.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.521+0000 7f4c6a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e8da79f0 0x55d1e919d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36188/0 (socket says 172.21.15.117:36188) 2024-09-21T06:07:45.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.521+0000 7f4c6a59c640 1 -- 172.21.15.117:0/281645437 learned_addr learned my addr 172.21.15.117:0/281645437 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:45.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.521+0000 7f4c61d9b640 1 --2- 172.21.15.117:0/281645437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1e8dc01b0 0x55d1e91a7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:45.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.521+0000 7f4c6ad9d640 1 --2- 172.21.15.117:0/281645437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1e8dc8390 0x55d1e91a7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:45.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.521+0000 7f4c6a59c640 1 -- 172.21.15.117:0/281645437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1e8dc8390 msgr2=0x55d1e91a7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:45.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.521+0000 7f4c6a59c640 1 --2- 172.21.15.117:0/281645437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1e8dc8390 0x55d1e91a7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:45.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.521+0000 7f4c6a59c640 1 -- 172.21.15.117:0/281645437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1e8dc01b0 msgr2=0x55d1e91a7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:45.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.521+0000 7f4c6a59c640 1 --2- 172.21.15.117:0/281645437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1e8dc01b0 0x55d1e91a7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:45.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.521+0000 7f4c6a59c640 1 -- 172.21.15.117:0/281645437 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1e919cb10 con 0x55d1e8da79f0 2024-09-21T06:07:45.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.522+0000 7f4c6a59c640 1 --2- 172.21.15.117:0/281645437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e8da79f0 0x55d1e919d730 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c013010 tx=0x7f4c5c0106c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4687fa71f8a2184 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:45.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.522+0000 7f4c53fff640 1 -- 172.21.15.117:0/281645437 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c019410 con 0x55d1e8da79f0 2024-09-21T06:07:45.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.522+0000 7f4c53fff640 1 -- 172.21.15.117:0/281645437 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c5c0195b0 con 0x55d1e8da79f0 2024-09-21T06:07:45.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.522+0000 7f4c53fff640 1 -- 172.21.15.117:0/281645437 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c004660 con 0x55d1e8da79f0 2024-09-21T06:07:45.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.523+0000 7f4c76f8c740 1 -- 172.21.15.117:0/281645437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e8da79f0 msgr2=0x55d1e919d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:45.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.523+0000 7f4c76f8c740 1 --2- 172.21.15.117:0/281645437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e8da79f0 0x55d1e919d730 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c013010 tx=0x7f4c5c0106c0 comp rx=0 tx=0).stop 2024-09-21T06:07:45.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.523+0000 7f4c76f8c740 1 -- 172.21.15.117:0/281645437 shutdown_connections 2024-09-21T06:07:45.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.523+0000 7f4c76f8c740 1 --2- 172.21.15.117:0/281645437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1e8dc8390 0x55d1e91a7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:45.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.523+0000 7f4c76f8c740 1 --2- 172.21.15.117:0/281645437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1e8dc01b0 0x55d1e91a7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:45.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.523+0000 7f4c76f8c740 1 --2- 172.21.15.117:0/281645437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e8da79f0 0x55d1e919d730 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:45.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.523+0000 7f4c76f8c740 1 -- 172.21.15.117:0/281645437 >> 172.21.15.117:0/281645437 conn(0x55d1e8ddd460 msgr2=0x55d1e91a6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:45.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.523+0000 7f4c76f8c740 1 -- 172.21.15.117:0/281645437 shutdown_connections 2024-09-21T06:07:45.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.523+0000 7f4c76f8c740 1 -- 172.21.15.117:0/281645437 wait complete. 2024-09-21T06:07:45.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.523+0000 7f4c76f8c740 1 -- start start 2024-09-21T06:07:45.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.523+0000 7f4c76f8c740 1 Processor -- start 2024-09-21T06:07:45.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.523+0000 7f4c76f8c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e8da79f0 0x55d1e919fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:45.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.523+0000 7f4c76f8c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1e8dc01b0 0x55d1e91a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:45.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.523+0000 7f4c76f8c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1e8dc8390 0x55d1e91a0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:45.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.524+0000 7f4c76f8c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6c1b7080 con 0x55d1e8da79f0 2024-09-21T06:07:45.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.524+0000 7f4c76f8c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6c1b69b0 con 0x55d1e8dc8390 2024-09-21T06:07:45.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.524+0000 7f4c76f8c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6c1b3c10 con 0x55d1e8dc01b0 2024-09-21T06:07:45.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.524+0000 7f4c6a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e8da79f0 0x55d1e919fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:45.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.524+0000 7f4c76f8c740 10 client.0 _ll_register_callbacks cb 0x7f4c5c00a800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:45.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.524+0000 7f4c6a59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e8da79f0 0x55d1e919fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36192/0 (socket says 172.21.15.117:36192) 2024-09-21T06:07:45.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.524+0000 7f4c6a59c640 1 -- 172.21.15.117:0/1268457230 learned_addr learned my addr 172.21.15.117:0/1268457230 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:45.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.524+0000 7f4c61d9b640 1 --2- 172.21.15.117:0/1268457230 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1e8dc01b0 0x55d1e91a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:45.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.524+0000 7f4c6ad9d640 1 --2- 172.21.15.117:0/1268457230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1e8dc8390 0x55d1e91a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:45.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.524+0000 7f4c6a59c640 1 -- 172.21.15.117:0/1268457230 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1e8dc01b0 msgr2=0x55d1e91a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:45.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.524+0000 7f4c6a59c640 1 --2- 172.21.15.117:0/1268457230 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1e8dc01b0 0x55d1e91a02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:45.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.524+0000 7f4c6a59c640 1 -- 172.21.15.117:0/1268457230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1e8dc8390 msgr2=0x55d1e91a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:45.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.524+0000 7f4c6a59c640 1 --2- 172.21.15.117:0/1268457230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1e8dc8390 0x55d1e91a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:45.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.524+0000 7f4c6a59c640 1 -- 172.21.15.117:0/1268457230 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1e9217890 con 0x55d1e8da79f0 2024-09-21T06:07:45.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.524+0000 7f4c6ad9d640 1 --2- 172.21.15.117:0/1268457230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1e8dc8390 0x55d1e91a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:45.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.524+0000 7f4c6a59c640 1 --2- 172.21.15.117:0/1268457230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e8da79f0 0x55d1e919fdb0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c005610 tx=0x7f4c5c005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dde0c75029183e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:45.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.524+0000 7f4c51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:45.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.524+0000 7f4c51ffb640 1 -- 172.21.15.117:0/1268457230 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c019410 con 0x55d1e8da79f0 2024-09-21T06:07:45.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.524+0000 7f4c51ffb640 1 -- 172.21.15.117:0/1268457230 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f4c5c031040 con 0x55d1e8da79f0 2024-09-21T06:07:45.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.524+0000 7f4c76f8c740 1 -- 172.21.15.117:0/1268457230 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c5c0195b0 con 0x55d1e8da79f0 2024-09-21T06:07:45.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.525+0000 7f4c51ffb640 1 -- 172.21.15.117:0/1268457230 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4c5c019d30 con 0x55d1e8da79f0 2024-09-21T06:07:45.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.525+0000 7f4c76f8c740 10 client.17859 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:45.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.525+0000 7f4c76f8c740 1 -- 172.21.15.117:0/1268457230 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d1e9218b20 con 0x55d1e8da79f0 2024-09-21T06:07:45.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.525+0000 7f4c51ffb640 1 -- 172.21.15.117:0/1268457230 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c5c0195b0 con 0x55d1e8da79f0 2024-09-21T06:07:45.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.525+0000 7f4c51ffb640 1 -- 172.21.15.117:0/1268457230 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4c5c0236d0 con 0x55d1e8da79f0 2024-09-21T06:07:45.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.525+0000 7f4c76f8c740 10 client.17859 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:45.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.525+0000 7f4c76f8c740 10 client.17859 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:07:45.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.525+0000 7f4c76f8c740 20 client.17859 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:45.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.525+0000 7f4c76f8c740 1 --2- 172.21.15.117:0/1268457230 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55d1e8de9070 0x55d1e921b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:45.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.525+0000 7f4c76f8c740 4 client.17859 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:07:45.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.525+0000 7f4c76f8c740 1 -- 172.21.15.117:0/1268457230 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d1e9104850 con 0x55d1e8de9070 2024-09-21T06:07:45.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.525+0000 7f4c61d9b640 1 --2- 172.21.15.117:0/1268457230 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55d1e8de9070 0x55d1e921b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:45.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.526+0000 7f4c61d9b640 1 --2- 172.21.15.117:0/1268457230 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55d1e8de9070 0x55d1e921b220 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=544eda64388deba1 server_cookie=fe584e2135859137 in_seq=0 out_seq=0 2024-09-21T06:07:45.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.526+0000 7f4c51ffb640 10 client.17859 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:45.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.526+0000 7f4c51ffb640 1 -- 172.21.15.117:0/1268457230 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3498 (crc 0 0 0) 0x55d1e9104850 con 0x55d1e8de9070 2024-09-21T06:07:45.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.526+0000 7f4c51ffb640 10 client.17859 handle_command_reply: tid=0 2024-09-21T06:07:45.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.526+0000 7f4c76f8c740 1 client.17859 shutdown 2024-09-21T06:07:45.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.526+0000 7f4c76f8c740 1 -- 172.21.15.117:0/1268457230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e8da79f0 msgr2=0x55d1e919fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:45.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.526+0000 7f4c76f8c740 1 --2- 172.21.15.117:0/1268457230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e8da79f0 0x55d1e919fdb0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c005610 tx=0x7f4c5c005640 comp rx=0 tx=0).stop 2024-09-21T06:07:45.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.526+0000 7f4c76f8c740 1 -- 172.21.15.117:0/1268457230 shutdown_connections 2024-09-21T06:07:45.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.526+0000 7f4c76f8c740 1 --2- 172.21.15.117:0/1268457230 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55d1e8de9070 0x55d1e921b220 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:45.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.526+0000 7f4c76f8c740 1 --2- 172.21.15.117:0/1268457230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1e8dc8390 0x55d1e91a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:45.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.526+0000 7f4c76f8c740 1 --2- 172.21.15.117:0/1268457230 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d1e8dc01b0 0x55d1e91a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:45.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.526+0000 7f4c76f8c740 1 --2- 172.21.15.117:0/1268457230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e8da79f0 0x55d1e919fdb0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:45.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.526+0000 7f4c76f8c740 1 -- 172.21.15.117:0/1268457230 >> 172.21.15.117:0/1268457230 conn(0x55d1e8ddd460 msgr2=0x55d1e91a75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:45.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.526+0000 7f4c51ffb640 0 client.17859 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:45.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.526+0000 7f4c76f8c740 1 -- 172.21.15.117:0/1268457230 shutdown_connections 2024-09-21T06:07:45.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.526+0000 7f4c76f8c740 1 -- 172.21.15.117:0/1268457230 wait complete. 2024-09-21T06:07:45.533 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.526+0000 7f4c76f8c740 20 client.17859 trim_cache size 0 max 16384 2024-09-21T06:07:45.533 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T06:07:45.534 INFO:teuthology.orchestra.run.smithi117.stdout: "status": "no active scrubs running", 2024-09-21T06:07:45.534 INFO:teuthology.orchestra.run.smithi117.stdout: "scrubs": {}, 2024-09-21T06:07:45.534 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_stats": { 2024-09-21T06:07:45.534 INFO:teuthology.orchestra.run.smithi117.stdout: "84be3416-2285-4046-adf2-8d335592d29f": { 2024-09-21T06:07:45.534 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:07:43.890616+0000", 2024-09-21T06:07:45.534 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:45.534 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:45.534 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:45.534 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:45.534 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:45.534 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:45.534 INFO:teuthology.orchestra.run.smithi117.stdout: "e99040eb-55d8-492b-ae21-f9ba41c42664": { 2024-09-21T06:07:45.534 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:07:29.750906+0000", 2024-09-21T06:07:45.535 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:45.535 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:45.535 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:45.535 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:45.535 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:45.535 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:45.535 INFO:teuthology.orchestra.run.smithi117.stdout: "26e3d90e-072a-4a4e-9661-4d004051865b": { 2024-09-21T06:07:45.535 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:06:29.219142+0000", 2024-09-21T06:07:45.535 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:45.535 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:45.535 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:45.535 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:45.535 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:45.535 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:45.535 INFO:teuthology.orchestra.run.smithi117.stdout: "cf43a1b2-838e-4d28-9ed6-ff0c726fd51a": { 2024-09-21T06:07:45.536 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:06:42.901863+0000", 2024-09-21T06:07:45.536 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:45.536 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:45.536 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:45.536 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:45.536 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:45.536 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:45.536 INFO:teuthology.orchestra.run.smithi117.stdout: "39eeca8a-6b4f-465f-af34-b54f77320d70": { 2024-09-21T06:07:45.536 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:05:41.497141+0000", 2024-09-21T06:07:45.536 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:45.536 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:45.536 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:45.536 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:45.536 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:45.536 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:45.536 INFO:teuthology.orchestra.run.smithi117.stdout: "29ff9b7c-49d9-4910-85be-bace7e940cd5": { 2024-09-21T06:07:45.537 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:03:03.775527+0000", 2024-09-21T06:07:45.537 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:45.537 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:45.537 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:45.537 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:45.537 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:45.537 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:45.537 INFO:teuthology.orchestra.run.smithi117.stdout: "0902e33a-ce33-45fb-92d8-2e6fcb8ec8a0": { 2024-09-21T06:07:45.537 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:04:11.365073+0000", 2024-09-21T06:07:45.537 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:45.537 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:45.537 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:45.537 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:45.537 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:45.537 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:45.538 INFO:teuthology.orchestra.run.smithi117.stdout: "d543b0ae-b710-4153-99da-fe5714839c1a": { 2024-09-21T06:07:45.538 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:01:20.574754+0000", 2024-09-21T06:07:45.538 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:45.538 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:45.538 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:45.538 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:45.538 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:45.538 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:45.538 INFO:teuthology.orchestra.run.smithi117.stdout: "ad80d9f1-6aff-46c0-91ab-2c1ab0681039": { 2024-09-21T06:07:45.538 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:59:31.949108+0000", 2024-09-21T06:07:45.538 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:45.538 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:45.538 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:45.538 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:45.539 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:45.539 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:45.539 INFO:teuthology.orchestra.run.smithi117.stdout: "fb2f217d-463b-4c8f-ab19-920b04bb7306": { 2024-09-21T06:07:45.539 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:04:56.308107+0000", 2024-09-21T06:07:45.539 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:45.539 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:45.539 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:45.539 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:45.539 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:45.539 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:45.539 INFO:teuthology.orchestra.run.smithi117.stdout: "8b3fe95a-b76b-4e94-afd0-72fe35e2dd92": { 2024-09-21T06:07:45.539 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:58:01.246076+0000", 2024-09-21T06:07:45.539 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:45.539 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:45.539 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:45.540 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:45.540 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:45.540 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:45.540 INFO:teuthology.orchestra.run.smithi117.stdout: "b30bfff0-0f6b-4e30-8467-ff7db421eba6": { 2024-09-21T06:07:45.540 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:57:14.939082+0000", 2024-09-21T06:07:45.540 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:45.540 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:45.540 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:45.540 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:45.540 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:45.540 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:07:45.540 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:07:45.540 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T06:07:45.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.527+0000 7f4c70a18640 1 -- 172.21.15.117:0/1177927570 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4c580b6680 msgr2=0x7f4c580b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:45.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.527+0000 7f4c70a18640 1 --2- 172.21.15.117:0/1177927570 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4c580b6680 0x7f4c580b8b90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4c64005cc0 tx=0x7f4c640095c0 comp rx=0 tx=0).stop 2024-09-21T06:07:45.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.527+0000 7f4c70a18640 1 -- 172.21.15.117:0/1177927570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c6c1083b0 msgr2=0x7f4c6c1a1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:45.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.527+0000 7f4c70a18640 1 --2- 172.21.15.117:0/1177927570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c6c1083b0 0x7f4c6c1a1cc0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f4c5400d740 tx=0x7f4c5400db40 comp rx=0 tx=0).stop 2024-09-21T06:07:45.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.527+0000 7f4c70a18640 1 -- 172.21.15.117:0/1177927570 shutdown_connections 2024-09-21T06:07:45.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.527+0000 7f4c70a18640 1 --2- 172.21.15.117:0/1177927570 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4c580b6680 0x7f4c580b8b90 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:45.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.527+0000 7f4c70a18640 1 --2- 172.21.15.117:0/1177927570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c6c1083b0 0x7f4c6c1a1cc0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:45.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.527+0000 7f4c70a18640 1 --2- 172.21.15.117:0/1177927570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c6c108d00 0x7f4c6c1ac800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:45.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.527+0000 7f4c70a18640 1 --2- 172.21.15.117:0/1177927570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c6c102a70 0x7f4c6c1a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:45.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.527+0000 7f4c70a18640 1 -- 172.21.15.117:0/1177927570 >> 172.21.15.117:0/1177927570 conn(0x7f4c6c0796e0 msgr2=0x7f4c6c1041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:45.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.527+0000 7f4c70a18640 1 -- 172.21.15.117:0/1177927570 shutdown_connections 2024-09-21T06:07:45.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:45.527+0000 7f4c70a18640 1 -- 172.21.15.117:0/1177927570 wait complete. 2024-09-21T06:07:46.179 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:45 smithi117 ceph-mon[30947]: scrub summary: idle 2024-09-21T06:07:46.180 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:45 smithi117 ceph-mon[30947]: pgmap v647: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:07:46.208 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:07:46.208 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:07:46.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:45 smithi192 ceph-mon[34700]: scrub summary: idle 2024-09-21T06:07:46.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:45 smithi192 ceph-mon[34700]: pgmap v647: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:07:46.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:45 smithi138 ceph-mon[34391]: scrub summary: idle 2024-09-21T06:07:46.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:45 smithi138 ceph-mon[34391]: pgmap v647: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:07:46.455 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:07:47.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.066+0000 7facc738b640 1 -- 172.21.15.117:0/2099583553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facc0105300 msgr2=0x7facc010d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:47.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.066+0000 7facc738b640 1 --2- 172.21.15.117:0/2099583553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facc0105300 0x7facc010d850 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7facb4009900 tx=0x7facb402a410 comp rx=0 tx=0).stop 2024-09-21T06:07:47.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.067+0000 7facc738b640 1 -- 172.21.15.117:0/2099583553 shutdown_connections 2024-09-21T06:07:47.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.067+0000 7facc738b640 1 --2- 172.21.15.117:0/2099583553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facc0105300 0x7facc010d850 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:47.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.067+0000 7facc738b640 1 --2- 172.21.15.117:0/2099583553 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7facc010de10 0x7facc0110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:47.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.067+0000 7facc738b640 1 --2- 172.21.15.117:0/2099583553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facc01049b0 0x7facc0104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:47.201 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.067+0000 7facc738b640 1 -- 172.21.15.117:0/2099583553 >> 172.21.15.117:0/2099583553 conn(0x7facc0079720 msgr2=0x7facc0079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:47.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.067+0000 7facc738b640 1 -- 172.21.15.117:0/2099583553 shutdown_connections 2024-09-21T06:07:47.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.067+0000 7facc738b640 1 -- 172.21.15.117:0/2099583553 wait complete. 2024-09-21T06:07:47.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.067+0000 7facc738b640 1 Processor -- start 2024-09-21T06:07:47.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.067+0000 7facc738b640 1 -- start start 2024-09-21T06:07:47.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.068+0000 7facc738b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facc01049b0 0x7facc019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:47.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.068+0000 7facc738b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facc0105300 0x7facc019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:47.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.068+0000 7facc738b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7facc010de10 0x7facc019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:47.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.068+0000 7facc738b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7facc0113a70 con 0x7facc0105300 2024-09-21T06:07:47.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.068+0000 7facc738b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7facc01138f0 con 0x7facc01049b0 2024-09-21T06:07:47.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.068+0000 7facc738b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7facc0113bf0 con 0x7facc010de10 2024-09-21T06:07:47.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.068+0000 7facc4926640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facc0105300 0x7facc019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:47.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.068+0000 7facc4926640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facc0105300 0x7facc019aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36214/0 (socket says 172.21.15.117:36214) 2024-09-21T06:07:47.202 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.068+0000 7facc4926640 1 -- 172.21.15.117:0/1612159717 learned_addr learned my addr 172.21.15.117:0/1612159717 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:47.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.068+0000 7facc5127640 1 --2- 172.21.15.117:0/1612159717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facc01049b0 0x7facc019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:47.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.068+0000 7facc5928640 1 --2- 172.21.15.117:0/1612159717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7facc010de10 0x7facc019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:47.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.068+0000 7facc4926640 1 -- 172.21.15.117:0/1612159717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7facc010de10 msgr2=0x7facc019fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:47.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.068+0000 7facc4926640 1 --2- 172.21.15.117:0/1612159717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7facc010de10 0x7facc019fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:47.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.068+0000 7facc4926640 1 -- 172.21.15.117:0/1612159717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facc01049b0 msgr2=0x7facc019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:47.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.068+0000 7facc4926640 1 --2- 172.21.15.117:0/1612159717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facc01049b0 0x7facc019a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:47.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.068+0000 7facc4926640 1 -- 172.21.15.117:0/1612159717 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc01a0410 con 0x7facc0105300 2024-09-21T06:07:47.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.068+0000 7facc4926640 1 --2- 172.21.15.117:0/1612159717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facc0105300 0x7facc019aa40 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7facb402a930 tx=0x7facb4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:47.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.068+0000 7facbe7fc640 1 -- 172.21.15.117:0/1612159717 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb403f030 con 0x7facc0105300 2024-09-21T06:07:47.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.068+0000 7facbe7fc640 1 -- 172.21.15.117:0/1612159717 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facb4003aa0 con 0x7facc0105300 2024-09-21T06:07:47.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.068+0000 7facc738b640 1 -- 172.21.15.117:0/1612159717 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc01a06a0 con 0x7facc0105300 2024-09-21T06:07:47.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.068+0000 7facbe7fc640 1 -- 172.21.15.117:0/1612159717 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb4035070 con 0x7facc0105300 2024-09-21T06:07:47.203 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.069+0000 7facc738b640 1 -- 172.21.15.117:0/1612159717 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc01a6e60 con 0x7facc0105300 2024-09-21T06:07:47.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.069+0000 7facc738b640 1 -- 172.21.15.117:0/1612159717 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facc006c280 con 0x7facc0105300 2024-09-21T06:07:47.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.071+0000 7facbe7fc640 1 -- 172.21.15.117:0/1612159717 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7facb4042030 con 0x7facc0105300 2024-09-21T06:07:47.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.072+0000 7facbe7fc640 1 --2- 172.21.15.117:0/1612159717 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7faca80b6680 0x7faca80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:47.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.072+0000 7facc5127640 1 --2- 172.21.15.117:0/1612159717 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7faca80b6680 0x7faca80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:47.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.072+0000 7facbe7fc640 1 -- 172.21.15.117:0/1612159717 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7facb4108110 con 0x7facc0105300 2024-09-21T06:07:47.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.072+0000 7facc5127640 1 --2- 172.21.15.117:0/1612159717 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7faca80b6680 0x7faca80b8b90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7facc006c980 tx=0x7facb000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:47.204 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.075+0000 7facbe7fc640 1 -- 172.21.15.117:0/1612159717 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7facb40d2a30 con 0x7facc0105300 2024-09-21T06:07:47.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.293+0000 7facc738b640 1 -- 172.21.15.117:0/1612159717 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7facc0104dc0 con 0x7facc0105300 2024-09-21T06:07:47.295 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.293+0000 7facbe7fc640 1 -- 172.21.15.117:0/1612159717 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+12001 (secure 0 0 0) 0x7facb40d7970 con 0x7facc0105300 2024-09-21T06:07:47.295 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T06:07:47.295 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":29,"btime":"2024-09-21T06:04:08:952483+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3227057504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3227057504},{"type":"v1","addr":"172.21.15.117:6835","nonce":3227057504}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3303829183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3303829183},{"type":"v1","addr":"172.21.15.117:6837","nonce":3303829183}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T05:55:14.555897+0000","modified":"2024-09-21T06:04:08.952466+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24638,"mds_1":24637,"mds_2":24628,"mds_3":24616,"mds_4":24602},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6839/2314572411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2314572411},{"type":"v1","addr":"172.21.15.117:6839","nonce":2314572411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6841/3378096561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":3378096561},{"type":"v1","addr":"172.21.15.117:6841","nonce":3378096561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/3725629862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3725629862},{"type":"v1","addr":"172.21.15.138:6833","nonce":3725629862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/4276801396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4276801396},{"type":"v1","addr":"172.21.15.138:6835","nonce":4276801396}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.138:6837/2540335374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2540335374},{"type":"v1","addr":"172.21.15.138:6837","nonce":2540335374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24616":{"gid":24616,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.192:6835/3530688860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3530688860},{"type":"v1","addr":"172.21.15.192:6835","nonce":3530688860}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.192:6837/2683860842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2683860842},{"type":"v1","addr":"172.21.15.192:6837","nonce":2683860842}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.192:6839/1727623231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1727623231},{"type":"v1","addr":"172.21.15.192:6839","nonce":1727623231}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/3230372576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":3230372576},{"type":"v1","addr":"172.21.15.138:6839","nonce":3230372576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25840":{"gid":25840,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6833/1596605058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1596605058},{"type":"v1","addr":"172.21.15.192:6833","nonce":1596605058}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24602,24628,24616,24637,24638]},"id":1}]} 2024-09-21T06:07:47.296 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 29 2024-09-21T06:07:47.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.296+0000 7facc738b640 1 -- 172.21.15.117:0/1612159717 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7faca80b6680 msgr2=0x7faca80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:47.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.296+0000 7facc738b640 1 --2- 172.21.15.117:0/1612159717 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7faca80b6680 0x7faca80b8b90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7facc006c980 tx=0x7facb000b390 comp rx=0 tx=0).stop 2024-09-21T06:07:47.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.296+0000 7facc738b640 1 -- 172.21.15.117:0/1612159717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facc0105300 msgr2=0x7facc019aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:47.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.296+0000 7facc738b640 1 --2- 172.21.15.117:0/1612159717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facc0105300 0x7facc019aa40 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7facb402a930 tx=0x7facb4004270 comp rx=0 tx=0).stop 2024-09-21T06:07:47.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.296+0000 7facc738b640 1 -- 172.21.15.117:0/1612159717 shutdown_connections 2024-09-21T06:07:47.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.296+0000 7facc738b640 1 --2- 172.21.15.117:0/1612159717 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7faca80b6680 0x7faca80b8b90 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:47.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.296+0000 7facc738b640 1 --2- 172.21.15.117:0/1612159717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facc0105300 0x7facc019aa40 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:47.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.296+0000 7facc738b640 1 --2- 172.21.15.117:0/1612159717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7facc010de10 0x7facc019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:47.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.296+0000 7facc738b640 1 --2- 172.21.15.117:0/1612159717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7facc01049b0 0x7facc019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:47.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.296+0000 7facc738b640 1 -- 172.21.15.117:0/1612159717 >> 172.21.15.117:0/1612159717 conn(0x7facc0079720 msgr2=0x7facc01008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:47.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.296+0000 7facc738b640 1 -- 172.21.15.117:0/1612159717 shutdown_connections 2024-09-21T06:07:47.298 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:47.296+0000 7facc738b640 1 -- 172.21.15.117:0/1612159717 wait complete. 2024-09-21T06:07:48.002 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T06:07:48.189 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:47 smithi138 ceph-mon[34391]: pgmap v648: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-21T06:07:48.190 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:47 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/1612159717' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:07:48.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:47 smithi192 ceph-mon[34700]: pgmap v648: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-21T06:07:48.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:47 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/1612159717' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:07:48.250 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:07:48.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:47 smithi117 ceph-mon[30947]: pgmap v648: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-09-21T06:07:48.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:47 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/1612159717' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:07:48.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.716+0000 7f4947338640 1 -- 172.21.15.117:0/260864127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4940102aa0 msgr2=0x7f4940102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:48.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.716+0000 7f4947338640 1 --2- 172.21.15.117:0/260864127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4940102aa0 0x7f4940102eb0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f4930009ca0 tx=0x7f493002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.716+0000 7f4947338640 1 -- 172.21.15.117:0/260864127 shutdown_connections 2024-09-21T06:07:48.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.716+0000 7f4947338640 1 --2- 172.21.15.117:0/260864127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4940108d30 0x7f494010f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.716+0000 7f4947338640 1 --2- 172.21.15.117:0/260864127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49401083e0 0x7f49401087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.716+0000 7f4947338640 1 --2- 172.21.15.117:0/260864127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4940102aa0 0x7f4940102eb0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.716+0000 7f4947338640 1 -- 172.21.15.117:0/260864127 >> 172.21.15.117:0/260864127 conn(0x7f49400797e0 msgr2=0x7f4940079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:48.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.716+0000 7f4947338640 1 -- 172.21.15.117:0/260864127 shutdown_connections 2024-09-21T06:07:48.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.716+0000 7f4947338640 1 -- 172.21.15.117:0/260864127 wait complete. 2024-09-21T06:07:48.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.717+0000 7f4947338640 1 Processor -- start 2024-09-21T06:07:48.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.717+0000 7f4947338640 1 -- start start 2024-09-21T06:07:48.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.717+0000 7f4947338640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4940102aa0 0x7f494019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:48.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.717+0000 7f4947338640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49401083e0 0x7f494019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:48.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.717+0000 7f4947338640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4940108d30 0x7f494019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:48.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.717+0000 7f4947338640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f49401129a0 con 0x7f4940108d30 2024-09-21T06:07:48.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.717+0000 7f4947338640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4940112820 con 0x7f4940102aa0 2024-09-21T06:07:48.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.717+0000 7f4947338640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4940112b20 con 0x7f49401083e0 2024-09-21T06:07:48.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.717+0000 7f49458d5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4940108d30 0x7f494019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:48.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.717+0000 7f49458d5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4940108d30 0x7f494019fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36218/0 (socket says 172.21.15.117:36218) 2024-09-21T06:07:48.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.717+0000 7f49458d5640 1 -- 172.21.15.117:0/2619394099 learned_addr learned my addr 172.21.15.117:0/2619394099 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:48.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.717+0000 7f49450d4640 1 --2- 172.21.15.117:0/2619394099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4940102aa0 0x7f494019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:48.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.717+0000 7f49448d3640 1 --2- 172.21.15.117:0/2619394099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49401083e0 0x7f494019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:48.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.718+0000 7f49458d5640 1 -- 172.21.15.117:0/2619394099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49401083e0 msgr2=0x7f494019ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:48.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.718+0000 7f49458d5640 1 --2- 172.21.15.117:0/2619394099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49401083e0 0x7f494019ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.718+0000 7f49458d5640 1 -- 172.21.15.117:0/2619394099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4940102aa0 msgr2=0x7f494019a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:48.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.718+0000 7f49458d5640 1 --2- 172.21.15.117:0/2619394099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4940102aa0 0x7f494019a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.718+0000 7f49458d5640 1 -- 172.21.15.117:0/2619394099 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49401a04d0 con 0x7f4940108d30 2024-09-21T06:07:48.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.718+0000 7f49458d5640 1 --2- 172.21.15.117:0/2619394099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4940108d30 0x7f494019fda0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f493800d750 tx=0x7f493800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:48.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.718+0000 7f493e7fc640 1 -- 172.21.15.117:0/2619394099 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4938017070 con 0x7f4940108d30 2024-09-21T06:07:48.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.718+0000 7f493e7fc640 1 -- 172.21.15.117:0/2619394099 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49380054d0 con 0x7f4940108d30 2024-09-21T06:07:48.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.718+0000 7f4947338640 1 -- 172.21.15.117:0/2619394099 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49401a07c0 con 0x7f4940108d30 2024-09-21T06:07:48.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.718+0000 7f493e7fc640 1 -- 172.21.15.117:0/2619394099 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4938004c60 con 0x7f4940108d30 2024-09-21T06:07:48.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.718+0000 7f4947338640 1 -- 172.21.15.117:0/2619394099 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49401a6f20 con 0x7f4940108d30 2024-09-21T06:07:48.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.721+0000 7f493e7fc640 1 -- 172.21.15.117:0/2619394099 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f4938009bd0 con 0x7f4940108d30 2024-09-21T06:07:48.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.721+0000 7f494d8ac740 1 Processor -- start 2024-09-21T06:07:48.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.721+0000 7f494d8ac740 1 -- start start 2024-09-21T06:07:48.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.721+0000 7f494d8ac740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e0806c070 0x560e084136a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:48.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.721+0000 7f494d8ac740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e080431b0 0x560e08413be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:48.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.721+0000 7f494d8ac740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0804b390 0x560e08414120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:48.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.721+0000 7f494d8ac740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560e08389ea0 con 0x560e0806c070 2024-09-21T06:07:48.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.721+0000 7f494d8ac740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560e083792d0 con 0x560e0804b390 2024-09-21T06:07:48.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.721+0000 7f494d8ac740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x560e0837d0e0 con 0x560e080431b0 2024-09-21T06:07:48.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.721+0000 7f49448d3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e0806c070 0x560e084136a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:48.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.721+0000 7f49448d3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e0806c070 0x560e084136a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36232/0 (socket says 172.21.15.117:36232) 2024-09-21T06:07:48.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.721+0000 7f49448d3640 1 -- 172.21.15.117:0/2727629542 learned_addr learned my addr 172.21.15.117:0/2727629542 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:48.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.722+0000 7f49450d4640 1 --2- 172.21.15.117:0/2727629542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e080431b0 0x560e08413be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:48.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.722+0000 7f493e7fc640 1 --2- 172.21.15.117:0/2619394099 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f49280b6680 0x7f49280b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:48.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.722+0000 7f49448d3640 1 --2- 172.21.15.117:0/2727629542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0804b390 0x560e08414120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:48.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.722+0000 7f49448d3640 1 -- 172.21.15.117:0/2727629542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e080431b0 msgr2=0x560e08413be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:48.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.722+0000 7f49448d3640 1 --2- 172.21.15.117:0/2727629542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e080431b0 0x560e08413be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.722+0000 7f49458d5640 1 --2- 172.21.15.117:0/2619394099 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f49280b6680 0x7f49280b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:48.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.722+0000 7f49448d3640 1 -- 172.21.15.117:0/2727629542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0804b390 msgr2=0x560e08414120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:48.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.722+0000 7f49448d3640 1 --2- 172.21.15.117:0/2727629542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0804b390 0x560e08414120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.722+0000 7f49448d3640 1 -- 172.21.15.117:0/2727629542 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e080d7650 con 0x560e0806c070 2024-09-21T06:07:48.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.722+0000 7f493e7fc640 1 -- 172.21.15.117:0/2619394099 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f49380e6f80 con 0x7f4940108d30 2024-09-21T06:07:48.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.722+0000 7f49448d3640 1 --2- 172.21.15.117:0/2727629542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e0806c070 0x560e084136a0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f4934012570 tx=0x7f493400b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21485ba625d1156c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:48.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.722+0000 7f49277fe640 1 -- 172.21.15.117:0/2727629542 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4934004020 con 0x560e0806c070 2024-09-21T06:07:48.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.722+0000 7f49277fe640 1 -- 172.21.15.117:0/2727629542 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4934004c50 con 0x560e0806c070 2024-09-21T06:07:48.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.722+0000 7f49277fe640 1 -- 172.21.15.117:0/2727629542 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4934005260 con 0x560e0806c070 2024-09-21T06:07:48.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.723+0000 7f494d8ac740 1 -- 172.21.15.117:0/2727629542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e0806c070 msgr2=0x560e084136a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:48.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.723+0000 7f49458d5640 1 --2- 172.21.15.117:0/2619394099 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f49280b6680 0x7f49280b8b90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f49380e47d0 tx=0x7f4938060040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:48.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.723+0000 7f494d8ac740 1 --2- 172.21.15.117:0/2727629542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e0806c070 0x560e084136a0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f4934012570 tx=0x7f493400b650 comp rx=0 tx=0).stop 2024-09-21T06:07:48.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.723+0000 7f494d8ac740 1 -- 172.21.15.117:0/2727629542 shutdown_connections 2024-09-21T06:07:48.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.723+0000 7f494d8ac740 1 --2- 172.21.15.117:0/2727629542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0804b390 0x560e08414120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.723+0000 7f494d8ac740 1 --2- 172.21.15.117:0/2727629542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e080431b0 0x560e08413be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.723+0000 7f494d8ac740 1 --2- 172.21.15.117:0/2727629542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e0806c070 0x560e084136a0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.723+0000 7f494d8ac740 1 -- 172.21.15.117:0/2727629542 >> 172.21.15.117:0/2727629542 conn(0x560e08060460 msgr2=0x560e08411590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:48.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.723+0000 7f494d8ac740 1 -- 172.21.15.117:0/2727629542 shutdown_connections 2024-09-21T06:07:48.743 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.723+0000 7f494d8ac740 1 -- 172.21.15.117:0/2727629542 wait complete. 2024-09-21T06:07:48.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.724+0000 7f494d8ac740 1 -- start start 2024-09-21T06:07:48.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.724+0000 7f494d8ac740 1 Processor -- start 2024-09-21T06:07:48.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.724+0000 7f494d8ac740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e080431b0 0x560e0841edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:48.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.724+0000 7f494d8ac740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0804b390 0x560e0841f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:48.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.724+0000 7f494d8ac740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e0806c070 0x560e0842b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:48.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.724+0000 7f494d8ac740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560e081da700 con 0x560e0806c070 2024-09-21T06:07:48.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.724+0000 7f494d8ac740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560e08314160 con 0x560e0804b390 2024-09-21T06:07:48.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.724+0000 7f494d8ac740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x560e08421e80 con 0x560e080431b0 2024-09-21T06:07:48.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.724+0000 7f494d8ac740 10 client.0 _ll_register_callbacks cb 0x560e0832d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:48.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.724+0000 7f49448d3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e0806c070 0x560e0842b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:48.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.724+0000 7f49448d3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e0806c070 0x560e0842b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36240/0 (socket says 172.21.15.117:36240) 2024-09-21T06:07:48.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.724+0000 7f49448d3640 1 -- 172.21.15.117:0/1798626483 learned_addr learned my addr 172.21.15.117:0/1798626483 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:48.744 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.724+0000 7f49450d4640 1 --2- 172.21.15.117:0/1798626483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0804b390 0x560e0841f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:48.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.724+0000 7f49448d3640 1 --2- 172.21.15.117:0/1798626483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e080431b0 0x560e0841edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:48.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.725+0000 7f49448d3640 1 -- 172.21.15.117:0/1798626483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e080431b0 msgr2=0x560e0841edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:48.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.725+0000 7f49448d3640 1 --2- 172.21.15.117:0/1798626483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e080431b0 0x560e0841edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.725+0000 7f49448d3640 1 -- 172.21.15.117:0/1798626483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0804b390 msgr2=0x560e0841f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:48.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.725+0000 7f49448d3640 1 --2- 172.21.15.117:0/1798626483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0804b390 0x560e0841f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.725+0000 7f49448d3640 1 -- 172.21.15.117:0/1798626483 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e08341c90 con 0x560e0806c070 2024-09-21T06:07:48.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.725+0000 7f49448d3640 1 --2- 172.21.15.117:0/1798626483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e0806c070 0x560e0842b1a0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f493400be90 tx=0x7f49340044a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7f26561761c59a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:48.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.725+0000 7f4924ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:48.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.725+0000 7f4924ff9640 1 -- 172.21.15.117:0/1798626483 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4934022060 con 0x560e0806c070 2024-09-21T06:07:48.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.725+0000 7f4924ff9640 1 -- 172.21.15.117:0/1798626483 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f4934005690 con 0x560e0806c070 2024-09-21T06:07:48.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.725+0000 7f494d8ac740 1 -- 172.21.15.117:0/1798626483 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4934004c50 con 0x560e0806c070 2024-09-21T06:07:48.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.725+0000 7f4924ff9640 1 -- 172.21.15.117:0/1798626483 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4934028040 con 0x560e0806c070 2024-09-21T06:07:48.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.725+0000 7f494d8ac740 10 client.17877 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:48.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.725+0000 7f494d8ac740 1 -- 172.21.15.117:0/1798626483 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560e0842dbd0 con 0x560e0806c070 2024-09-21T06:07:48.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.725+0000 7f4924ff9640 1 -- 172.21.15.117:0/1798626483 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4934004c50 con 0x560e0806c070 2024-09-21T06:07:48.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.726+0000 7f4924ff9640 1 -- 172.21.15.117:0/1798626483 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f49340107b0 con 0x560e0806c070 2024-09-21T06:07:48.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.726+0000 7f494d8ac740 10 client.17877 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:48.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.726+0000 7f494d8ac740 10 client.17877 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T06:07:48.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.726+0000 7f494d8ac740 20 client.17877 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:48.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.726+0000 7f494d8ac740 1 --2- 172.21.15.117:0/1798626483 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x560e0802a9f0 0x560e08430f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:48.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.726+0000 7f494d8ac740 4 client.17877 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:07:48.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.726+0000 7f494d8ac740 1 -- 172.21.15.117:0/1798626483 --> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e0838bdf0 con 0x560e0802a9f0 2024-09-21T06:07:48.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.726+0000 7f49450d4640 1 --2- 172.21.15.117:0/1798626483 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x560e0802a9f0 0x560e08430f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:48.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.726+0000 7f49450d4640 1 --2- 172.21.15.117:0/1798626483 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x560e0802a9f0 0x560e08430f30 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=60f965f3a744be9a server_cookie=9b339751435adccd in_seq=0 out_seq=0 2024-09-21T06:07:48.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.726+0000 7f4924ff9640 10 client.17877 ms_handle_connect on v2:172.21.15.138:6836/2540335374 2024-09-21T06:07:48.746 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.727+0000 7f4924ff9640 1 -- 172.21.15.117:0/1798626483 <== mds.4 v2:172.21.15.138:6836/2540335374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560e0838bdf0 con 0x560e0802a9f0 2024-09-21T06:07:48.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.727+0000 7f4924ff9640 10 client.17877 handle_command_reply: tid=0 2024-09-21T06:07:48.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.727+0000 7f494d8ac740 1 client.17877 shutdown 2024-09-21T06:07:48.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.727+0000 7f494d8ac740 1 -- 172.21.15.117:0/1798626483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e0806c070 msgr2=0x560e0842b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:48.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.727+0000 7f494d8ac740 1 --2- 172.21.15.117:0/1798626483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e0806c070 0x560e0842b1a0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f493400be90 tx=0x7f49340044a0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.727+0000 7f494d8ac740 1 -- 172.21.15.117:0/1798626483 shutdown_connections 2024-09-21T06:07:48.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.727+0000 7f494d8ac740 1 --2- 172.21.15.117:0/1798626483 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x560e0802a9f0 0x560e08430f30 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.727+0000 7f494d8ac740 1 --2- 172.21.15.117:0/1798626483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e0806c070 0x560e0842b1a0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.727+0000 7f494d8ac740 1 --2- 172.21.15.117:0/1798626483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0804b390 0x560e0841f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.727+0000 7f494d8ac740 1 --2- 172.21.15.117:0/1798626483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e080431b0 0x560e0841edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.727+0000 7f494d8ac740 1 -- 172.21.15.117:0/1798626483 >> 172.21.15.117:0/1798626483 conn(0x560e08060460 msgr2=0x560e08416880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:48.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.727+0000 7f4924ff9640 0 client.17877 ms_handle_reset on v2:172.21.15.138:6836/2540335374 2024-09-21T06:07:48.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.727+0000 7f494d8ac740 1 -- 172.21.15.117:0/1798626483 shutdown_connections 2024-09-21T06:07:48.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.727+0000 7f494d8ac740 1 -- 172.21.15.117:0/1798626483 wait complete. 2024-09-21T06:07:48.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.727+0000 7f494d8ac740 20 client.17877 trim_cache size 0 max 16384 2024-09-21T06:07:48.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.741+0000 7f494d8ac740 1 Processor -- start 2024-09-21T06:07:48.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.741+0000 7f494d8ac740 1 -- start start 2024-09-21T06:07:48.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.741+0000 7f494d8ac740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e080431b0 0x560e0841f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:48.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.741+0000 7f494d8ac740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0804b390 0x560e08416eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:48.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.741+0000 7f494d8ac740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e0806c070 0x560e0842b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:48.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.741+0000 7f494d8ac740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f49401af6b0 con 0x560e0806c070 2024-09-21T06:07:48.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.741+0000 7f494d8ac740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f49401b11f0 con 0x560e0804b390 2024-09-21T06:07:48.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.741+0000 7f494d8ac740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f49401afd80 con 0x560e080431b0 2024-09-21T06:07:48.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.741+0000 7f49448d3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e0806c070 0x560e0842b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:48.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.741+0000 7f49448d3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e0806c070 0x560e0842b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36244/0 (socket says 172.21.15.117:36244) 2024-09-21T06:07:48.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.741+0000 7f49448d3640 1 -- 172.21.15.117:0/3205693563 learned_addr learned my addr 172.21.15.117:0/3205693563 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:48.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.742+0000 7f49450d4640 1 --2- 172.21.15.117:0/3205693563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0804b390 0x560e08416eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:48.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.742+0000 7f49448d3640 1 --2- 172.21.15.117:0/3205693563 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e080431b0 0x560e0841f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:48.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.742+0000 7f49448d3640 1 -- 172.21.15.117:0/3205693563 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e080431b0 msgr2=0x560e0841f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:48.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.742+0000 7f49448d3640 1 --2- 172.21.15.117:0/3205693563 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e080431b0 0x560e0841f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.742+0000 7f49448d3640 1 -- 172.21.15.117:0/3205693563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0804b390 msgr2=0x560e08416eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:48.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.742+0000 7f49448d3640 1 --2- 172.21.15.117:0/3205693563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0804b390 0x560e08416eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.753 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.742+0000 7f49448d3640 1 -- 172.21.15.117:0/3205693563 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e084173f0 con 0x560e0806c070 2024-09-21T06:07:48.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.742+0000 7f49450d4640 1 --2- 172.21.15.117:0/3205693563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0804b390 0x560e08416eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:07:48.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.742+0000 7f49448d3640 1 --2- 172.21.15.117:0/3205693563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e0806c070 0x560e0842b0a0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f49340124e0 tx=0x7f493400bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f296d9d4d116c18b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:48.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.742+0000 7f4926ffd640 1 -- 172.21.15.117:0/3205693563 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493403e070 con 0x560e0806c070 2024-09-21T06:07:48.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.742+0000 7f4926ffd640 1 -- 172.21.15.117:0/3205693563 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f493400b810 con 0x560e0806c070 2024-09-21T06:07:48.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.742+0000 7f4926ffd640 1 -- 172.21.15.117:0/3205693563 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4934022070 con 0x560e0806c070 2024-09-21T06:07:48.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.743+0000 7f494d8ac740 1 -- 172.21.15.117:0/3205693563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e0806c070 msgr2=0x560e0842b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:48.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.743+0000 7f494d8ac740 1 --2- 172.21.15.117:0/3205693563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e0806c070 0x560e0842b0a0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f49340124e0 tx=0x7f493400bf50 comp rx=0 tx=0).stop 2024-09-21T06:07:48.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.743+0000 7f494d8ac740 1 -- 172.21.15.117:0/3205693563 shutdown_connections 2024-09-21T06:07:48.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.743+0000 7f494d8ac740 1 --2- 172.21.15.117:0/3205693563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e0806c070 0x560e0842b0a0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.743+0000 7f494d8ac740 1 --2- 172.21.15.117:0/3205693563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e0804b390 0x560e08416eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.743+0000 7f494d8ac740 1 --2- 172.21.15.117:0/3205693563 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e080431b0 0x560e0841f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.743+0000 7f494d8ac740 1 -- 172.21.15.117:0/3205693563 >> 172.21.15.117:0/3205693563 conn(0x560e08060460 msgr2=0x560e084167a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:48.754 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.743+0000 7f494d8ac740 1 -- 172.21.15.117:0/3205693563 shutdown_connections 2024-09-21T06:07:48.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.743+0000 7f494d8ac740 1 -- 172.21.15.117:0/3205693563 wait complete. 2024-09-21T06:07:48.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.743+0000 7f494d8ac740 1 -- start start 2024-09-21T06:07:48.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.744+0000 7f494d8ac740 1 Processor -- start 2024-09-21T06:07:48.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.744+0000 7f494d8ac740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e080431b0 0x560e08422bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:48.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.744+0000 7f494d8ac740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e0804b390 0x560e084231b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:48.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.744+0000 7f494d8ac740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e0806c070 0x560e08493820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:48.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.744+0000 7f494d8ac740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f49401b0b20 con 0x560e0804b390 2024-09-21T06:07:48.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.744+0000 7f494d8ac740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f49401b0450 con 0x560e080431b0 2024-09-21T06:07:48.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.744+0000 7f494d8ac740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f49401ad6b0 con 0x560e0806c070 2024-09-21T06:07:48.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.744+0000 7f494d8ac740 10 client.0 _ll_register_callbacks cb 0x7f4930009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:48.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.744+0000 7f49450d4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e0804b390 0x560e084231b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:48.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.744+0000 7f49450d4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e0804b390 0x560e084231b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36254/0 (socket says 172.21.15.117:36254) 2024-09-21T06:07:48.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.744+0000 7f49450d4640 1 -- 172.21.15.117:0/4189063157 learned_addr learned my addr 172.21.15.117:0/4189063157 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:48.755 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.744+0000 7f49448d3640 1 --2- 172.21.15.117:0/4189063157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e080431b0 0x560e08422bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:48.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.744+0000 7f49448d3640 1 --2- 172.21.15.117:0/4189063157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e0806c070 0x560e08493820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:48.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.745+0000 7f49450d4640 1 -- 172.21.15.117:0/4189063157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e0806c070 msgr2=0x560e08493820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:48.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.745+0000 7f49450d4640 1 --2- 172.21.15.117:0/4189063157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e0806c070 0x560e08493820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.745+0000 7f49450d4640 1 -- 172.21.15.117:0/4189063157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e080431b0 msgr2=0x560e08422bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:48.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.745+0000 7f49450d4640 1 --2- 172.21.15.117:0/4189063157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e080431b0 0x560e08422bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.745+0000 7f49450d4640 1 -- 172.21.15.117:0/4189063157 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e08493e90 con 0x560e0804b390 2024-09-21T06:07:48.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.745+0000 7f49448d3640 1 --2- 172.21.15.117:0/4189063157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e0806c070 0x560e08493820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:48.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.745+0000 7f49450d4640 1 --2- 172.21.15.117:0/4189063157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e0804b390 0x560e084231b0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f493002ade0 tx=0x7f49300317a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66f3318e34a9aee4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:48.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.745+0000 7f4924ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:48.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.745+0000 7f4924ff9640 1 -- 172.21.15.117:0/4189063157 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4930003d80 con 0x560e0804b390 2024-09-21T06:07:48.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.745+0000 7f4924ff9640 1 -- 172.21.15.117:0/4189063157 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f4930004040 con 0x560e0804b390 2024-09-21T06:07:48.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.745+0000 7f494d8ac740 1 -- 172.21.15.117:0/4189063157 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f493400b810 con 0x560e0804b390 2024-09-21T06:07:48.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.745+0000 7f4924ff9640 1 -- 172.21.15.117:0/4189063157 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f493005d050 con 0x560e0804b390 2024-09-21T06:07:48.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.745+0000 7f494d8ac740 10 client.17883 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:48.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.745+0000 7f494d8ac740 1 -- 172.21.15.117:0/4189063157 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560e0849b2b0 con 0x560e0804b390 2024-09-21T06:07:48.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.745+0000 7f4924ff9640 1 -- 172.21.15.117:0/4189063157 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f493400b810 con 0x560e0804b390 2024-09-21T06:07:48.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.746+0000 7f4924ff9640 1 -- 172.21.15.117:0/4189063157 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f493002f8d0 con 0x560e0804b390 2024-09-21T06:07:48.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.746+0000 7f494d8ac740 10 client.17883 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:48.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.746+0000 7f494d8ac740 10 client.17883 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T06:07:48.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.746+0000 7f494d8ac740 20 client.17883 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:48.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.746+0000 7f494d8ac740 1 --2- 172.21.15.117:0/4189063157 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x560e07fffda0 0x560e0849daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:48.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.746+0000 7f494d8ac740 4 client.17883 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:07:48.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.746+0000 7f494d8ac740 1 -- 172.21.15.117:0/4189063157 --> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560e083878a0 con 0x560e07fffda0 2024-09-21T06:07:48.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.746+0000 7f49448d3640 1 --2- 172.21.15.117:0/4189063157 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x560e07fffda0 0x560e0849daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:48.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.746+0000 7f49448d3640 1 --2- 172.21.15.117:0/4189063157 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x560e07fffda0 0x560e0849daf0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=319b4110c92d3819 server_cookie=fb70c9dce5ea44ef in_seq=0 out_seq=0 2024-09-21T06:07:48.757 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.746+0000 7f4924ff9640 10 client.17883 ms_handle_connect on v2:172.21.15.138:6836/2540335374 2024-09-21T06:07:48.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.747+0000 7f4924ff9640 1 -- 172.21.15.117:0/4189063157 <== mds.4 v2:172.21.15.138:6836/2540335374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560e083878a0 con 0x560e07fffda0 2024-09-21T06:07:48.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.747+0000 7f4924ff9640 10 client.17883 handle_command_reply: tid=0 2024-09-21T06:07:48.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.747+0000 7f494d8ac740 1 client.17883 shutdown 2024-09-21T06:07:48.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.747+0000 7f494d8ac740 1 -- 172.21.15.117:0/4189063157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e0804b390 msgr2=0x560e084231b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:48.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.747+0000 7f494d8ac740 1 --2- 172.21.15.117:0/4189063157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e0804b390 0x560e084231b0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f493002ade0 tx=0x7f49300317a0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.747+0000 7f494d8ac740 1 -- 172.21.15.117:0/4189063157 shutdown_connections 2024-09-21T06:07:48.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.747+0000 7f494d8ac740 1 --2- 172.21.15.117:0/4189063157 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x560e07fffda0 0x560e0849daf0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.747+0000 7f4924ff9640 0 client.17883 ms_handle_reset on v2:172.21.15.138:6836/2540335374 2024-09-21T06:07:48.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.747+0000 7f494d8ac740 1 --2- 172.21.15.117:0/4189063157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560e0804b390 0x560e084231b0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.747+0000 7f494d8ac740 1 --2- 172.21.15.117:0/4189063157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560e0806c070 0x560e08493820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.758 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.747+0000 7f494d8ac740 1 --2- 172.21.15.117:0/4189063157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560e080431b0 0x560e08422bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.747+0000 7f494d8ac740 1 -- 172.21.15.117:0/4189063157 >> 172.21.15.117:0/4189063157 conn(0x560e08060460 msgr2=0x560e084146d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:48.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.747+0000 7f494d8ac740 1 -- 172.21.15.117:0/4189063157 shutdown_connections 2024-09-21T06:07:48.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.747+0000 7f494d8ac740 1 -- 172.21.15.117:0/4189063157 wait complete. 2024-09-21T06:07:48.759 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.747+0000 7f494d8ac740 20 client.17883 trim_cache size 0 max 16384 2024-09-21T06:07:48.759 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:07:48.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.748+0000 7f4947338640 1 -- 172.21.15.117:0/2619394099 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f49280b6680 msgr2=0x7f49280b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:48.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.748+0000 7f4947338640 1 --2- 172.21.15.117:0/2619394099 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f49280b6680 0x7f49280b8b90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f49380e47d0 tx=0x7f4938060040 comp rx=0 tx=0).stop 2024-09-21T06:07:48.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.748+0000 7f4947338640 1 -- 172.21.15.117:0/2619394099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4940108d30 msgr2=0x7f494019fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:48.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.748+0000 7f4947338640 1 --2- 172.21.15.117:0/2619394099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4940108d30 0x7f494019fda0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f493800d750 tx=0x7f493800dc10 comp rx=0 tx=0).stop 2024-09-21T06:07:48.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.748+0000 7f4947338640 1 -- 172.21.15.117:0/2619394099 shutdown_connections 2024-09-21T06:07:48.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.748+0000 7f4947338640 1 --2- 172.21.15.117:0/2619394099 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f49280b6680 0x7f49280b8b90 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.760 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.748+0000 7f4947338640 1 --2- 172.21.15.117:0/2619394099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4940108d30 0x7f494019fda0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.748+0000 7f4947338640 1 --2- 172.21.15.117:0/2619394099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49401083e0 0x7f494019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.748+0000 7f4947338640 1 --2- 172.21.15.117:0/2619394099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4940102aa0 0x7f494019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:48.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.748+0000 7f4947338640 1 -- 172.21.15.117:0/2619394099 >> 172.21.15.117:0/2619394099 conn(0x7f49400797e0 msgr2=0x7f49401067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:48.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.748+0000 7f4947338640 1 -- 172.21.15.117:0/2619394099 shutdown_connections 2024-09-21T06:07:48.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:48.748+0000 7f4947338640 1 -- 172.21.15.117:0/2619394099 wait complete. 2024-09-21T06:07:49.579 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T06:07:49.830 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:07:50.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:49 smithi192 ceph-mon[34700]: pgmap v649: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:07:50.254 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:49 smithi117 ceph-mon[30947]: pgmap v649: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:07:50.312 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:49 smithi138 ceph-mon[34391]: pgmap v649: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:07:50.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.343+0000 7ff7bac8c640 1 -- 172.21.15.117:0/956148479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7b4108d00 msgr2=0x7ff7b410f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:50.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.343+0000 7ff7bac8c640 1 --2- 172.21.15.117:0/956148479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7b4108d00 0x7ff7b410f530 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac009ca0 tx=0x7ff7ac02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.343+0000 7ff7bac8c640 1 -- 172.21.15.117:0/956148479 shutdown_connections 2024-09-21T06:07:50.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.343+0000 7ff7bac8c640 1 --2- 172.21.15.117:0/956148479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7b4108d00 0x7ff7b410f530 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.343+0000 7ff7bac8c640 1 --2- 172.21.15.117:0/956148479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7b41083b0 0x7ff7b41087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.343+0000 7ff7bac8c640 1 --2- 172.21.15.117:0/956148479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7b4102a70 0x7ff7b4102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.343+0000 7ff7bac8c640 1 -- 172.21.15.117:0/956148479 >> 172.21.15.117:0/956148479 conn(0x7ff7b4079710 msgr2=0x7ff7b4079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:50.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.344+0000 7ff7bac8c640 1 -- 172.21.15.117:0/956148479 shutdown_connections 2024-09-21T06:07:50.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.344+0000 7ff7bac8c640 1 -- 172.21.15.117:0/956148479 wait complete. 2024-09-21T06:07:50.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.344+0000 7ff7bac8c640 1 Processor -- start 2024-09-21T06:07:50.364 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.344+0000 7ff7bac8c640 1 -- start start 2024-09-21T06:07:50.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.344+0000 7ff7bac8c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7b4102a70 0x7ff7b419b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:50.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.344+0000 7ff7bac8c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7b41083b0 0x7ff7b419b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:50.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.344+0000 7ff7bac8c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7b4108d00 0x7ff7b419bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:50.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.344+0000 7ff7bac8c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b41129c0 con 0x7ff7b41083b0 2024-09-21T06:07:50.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.344+0000 7ff7bac8c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b4112840 con 0x7ff7b4102a70 2024-09-21T06:07:50.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.345+0000 7ff7bac8c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b4112b40 con 0x7ff7b4108d00 2024-09-21T06:07:50.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.345+0000 7ff7b3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7b41083b0 0x7ff7b419b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:50.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.345+0000 7ff7b3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7b41083b0 0x7ff7b419b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36272/0 (socket says 172.21.15.117:36272) 2024-09-21T06:07:50.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.345+0000 7ff7b3fff640 1 -- 172.21.15.117:0/2067192359 learned_addr learned my addr 172.21.15.117:0/2067192359 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:50.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.345+0000 7ff7b8a28640 1 --2- 172.21.15.117:0/2067192359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7b4102a70 0x7ff7b419b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:50.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.345+0000 7ff7b9229640 1 --2- 172.21.15.117:0/2067192359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7b4108d00 0x7ff7b419bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:50.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.345+0000 7ff7b3fff640 1 -- 172.21.15.117:0/2067192359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7b4108d00 msgr2=0x7ff7b419bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:50.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.345+0000 7ff7b3fff640 1 --2- 172.21.15.117:0/2067192359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7b4108d00 0x7ff7b419bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.345+0000 7ff7b3fff640 1 -- 172.21.15.117:0/2067192359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7b4102a70 msgr2=0x7ff7b419b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:50.365 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.345+0000 7ff7b3fff640 1 --2- 172.21.15.117:0/2067192359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7b4102a70 0x7ff7b419b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.345+0000 7ff7b3fff640 1 -- 172.21.15.117:0/2067192359 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b41a65f0 con 0x7ff7b41083b0 2024-09-21T06:07:50.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.345+0000 7ff7b9229640 1 --2- 172.21.15.117:0/2067192359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7b4108d00 0x7ff7b419bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:50.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.345+0000 7ff7b3fff640 1 --2- 172.21.15.117:0/2067192359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7b41083b0 0x7ff7b419b870 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7ff7a400e9c0 tx=0x7ff7a400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:50.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.345+0000 7ff7b1ffb640 1 -- 172.21.15.117:0/2067192359 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a400ccc0 con 0x7ff7b41083b0 2024-09-21T06:07:50.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.345+0000 7ff7bac8c640 1 -- 172.21.15.117:0/2067192359 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b41a68e0 con 0x7ff7b41083b0 2024-09-21T06:07:50.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.345+0000 7ff7b1ffb640 1 -- 172.21.15.117:0/2067192359 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7a40054d0 con 0x7ff7b41083b0 2024-09-21T06:07:50.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.345+0000 7ff7b1ffb640 1 -- 172.21.15.117:0/2067192359 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4010430 con 0x7ff7b41083b0 2024-09-21T06:07:50.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.346+0000 7ff7bac8c640 1 -- 172.21.15.117:0/2067192359 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b41a6e20 con 0x7ff7b41083b0 2024-09-21T06:07:50.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.348+0000 7ff7b1ffb640 1 -- 172.21.15.117:0/2067192359 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7ff7a4005020 con 0x7ff7b41083b0 2024-09-21T06:07:50.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.348+0000 7ff7c1200740 1 Processor -- start 2024-09-21T06:07:50.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.348+0000 7ff7c1200740 1 -- start start 2024-09-21T06:07:50.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.348+0000 7ff7c1200740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564290690070 0x564290a372c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:50.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.348+0000 7ff7c1200740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642906671b0 0x564290a37800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:50.366 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.348+0000 7ff7c1200740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56429066f390 0x564290a37d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:50.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.348+0000 7ff7c1200740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5642909adea0 con 0x56429066f390 2024-09-21T06:07:50.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.348+0000 7ff7c1200740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56429099d2d0 con 0x5642906671b0 2024-09-21T06:07:50.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.348+0000 7ff7c1200740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5642909a10e0 con 0x564290690070 2024-09-21T06:07:50.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.349+0000 7ff7b3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56429066f390 0x564290a37d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:50.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.349+0000 7ff7b8a28640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642906671b0 0x564290a37800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:50.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.349+0000 7ff7b8a28640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642906671b0 0x564290a37800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.117:40068/0 (socket says 172.21.15.117:40068) 2024-09-21T06:07:50.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.349+0000 7ff7b8a28640 1 -- 172.21.15.117:0/3514442617 learned_addr learned my addr 172.21.15.117:0/3514442617 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:50.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.349+0000 7ff7b9229640 1 --2- 172.21.15.117:0/3514442617 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564290690070 0x564290a372c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:50.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.349+0000 7ff7b1ffb640 1 --2- 172.21.15.117:0/2067192359 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff7a00b6680 0x7ff7a00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:50.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.349+0000 7ff7b8a28640 1 -- 172.21.15.117:0/3514442617 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564290690070 msgr2=0x564290a372c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:50.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.349+0000 7ff7b8a28640 1 --2- 172.21.15.117:0/3514442617 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564290690070 0x564290a372c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.349+0000 7ff7b8a28640 1 -- 172.21.15.117:0/3514442617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56429066f390 msgr2=0x564290a37d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:50.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.349+0000 7ff7b8a28640 1 --2- 172.21.15.117:0/3514442617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56429066f390 0x564290a37d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.349+0000 7ff7b8a28640 1 -- 172.21.15.117:0/3514442617 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642906fb650 con 0x5642906671b0 2024-09-21T06:07:50.367 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.349+0000 7ff7b9229640 1 --2- 172.21.15.117:0/2067192359 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff7a00b6680 0x7ff7a00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:50.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.349+0000 7ff7b1ffb640 1 -- 172.21.15.117:0/2067192359 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7ff7a40e7a80 con 0x7ff7b41083b0 2024-09-21T06:07:50.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.349+0000 7ff7b8a28640 1 --2- 172.21.15.117:0/3514442617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642906671b0 0x564290a37800 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00bcb0 tx=0x7ff79c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65582b02698024c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:50.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.350+0000 7ff7b9229640 1 --2- 172.21.15.117:0/2067192359 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff7a00b6680 0x7ff7a00b8b90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac006010 tx=0x7ff7ac0095c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:50.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.350+0000 7ff7aaffd640 1 -- 172.21.15.117:0/3514442617 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c01c480 con 0x5642906671b0 2024-09-21T06:07:50.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.350+0000 7ff7aaffd640 1 -- 172.21.15.117:0/3514442617 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff79c01cad0 con 0x5642906671b0 2024-09-21T06:07:50.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.350+0000 7ff7aaffd640 1 -- 172.21.15.117:0/3514442617 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c011bc0 con 0x5642906671b0 2024-09-21T06:07:50.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.350+0000 7ff7c1200740 1 -- 172.21.15.117:0/3514442617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642906671b0 msgr2=0x564290a37800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:50.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.350+0000 7ff7c1200740 1 --2- 172.21.15.117:0/3514442617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642906671b0 0x564290a37800 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff79c00bcb0 tx=0x7ff79c0107a0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.351+0000 7ff7c1200740 1 -- 172.21.15.117:0/3514442617 shutdown_connections 2024-09-21T06:07:50.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.351+0000 7ff7c1200740 1 --2- 172.21.15.117:0/3514442617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56429066f390 0x564290a37d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.351+0000 7ff7c1200740 1 --2- 172.21.15.117:0/3514442617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642906671b0 0x564290a37800 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.351+0000 7ff7c1200740 1 --2- 172.21.15.117:0/3514442617 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564290690070 0x564290a372c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.351+0000 7ff7c1200740 1 -- 172.21.15.117:0/3514442617 >> 172.21.15.117:0/3514442617 conn(0x564290684460 msgr2=0x564290a351b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:50.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.351+0000 7ff7c1200740 1 -- 172.21.15.117:0/3514442617 shutdown_connections 2024-09-21T06:07:50.368 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.351+0000 7ff7c1200740 1 -- 172.21.15.117:0/3514442617 wait complete. 2024-09-21T06:07:50.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.351+0000 7ff7c1200740 1 -- start start 2024-09-21T06:07:50.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.351+0000 7ff7c1200740 1 Processor -- start 2024-09-21T06:07:50.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.351+0000 7ff7c1200740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642906671b0 0x564290a46d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:50.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.351+0000 7ff7c1200740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56429066f390 0x564290a47260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:50.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.351+0000 7ff7c1200740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564290690070 0x564290a477c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:50.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.351+0000 7ff7c1200740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5642907fe700 con 0x56429066f390 2024-09-21T06:07:50.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.352+0000 7ff7c1200740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564290938160 con 0x564290690070 2024-09-21T06:07:50.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.352+0000 7ff7c1200740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564290a49da0 con 0x5642906671b0 2024-09-21T06:07:50.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.352+0000 7ff7c1200740 10 client.0 _ll_register_callbacks cb 0x5642909519e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:50.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.352+0000 7ff7b3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56429066f390 0x564290a47260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:50.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.352+0000 7ff7b3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56429066f390 0x564290a47260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36280/0 (socket says 172.21.15.117:36280) 2024-09-21T06:07:50.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.352+0000 7ff7b3fff640 1 -- 172.21.15.117:0/1331785450 learned_addr learned my addr 172.21.15.117:0/1331785450 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:50.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.352+0000 7ff7b9229640 1 --2- 172.21.15.117:0/1331785450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564290690070 0x564290a477c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:50.369 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.352+0000 7ff7b8a28640 1 --2- 172.21.15.117:0/1331785450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642906671b0 0x564290a46d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:50.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.352+0000 7ff7b3fff640 1 -- 172.21.15.117:0/1331785450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642906671b0 msgr2=0x564290a46d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:50.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.352+0000 7ff7b3fff640 1 --2- 172.21.15.117:0/1331785450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642906671b0 0x564290a46d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.352+0000 7ff7b3fff640 1 -- 172.21.15.117:0/1331785450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564290690070 msgr2=0x564290a477c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:50.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.352+0000 7ff7b3fff640 1 --2- 172.21.15.117:0/1331785450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564290690070 0x564290a477c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.352+0000 7ff7b3fff640 1 -- 172.21.15.117:0/1331785450 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564290965c90 con 0x56429066f390 2024-09-21T06:07:50.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.352+0000 7ff7b8a28640 1 --2- 172.21.15.117:0/1331785450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642906671b0 0x564290a46d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:07:50.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.352+0000 7ff7b3fff640 1 --2- 172.21.15.117:0/1331785450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56429066f390 0x564290a47260 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7ff7a40e5280 tx=0x7ff7a40eb1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0d82395b8ed45d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:50.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.352+0000 7ff793fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:50.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.352+0000 7ff793fff640 1 -- 172.21.15.117:0/1331785450 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a40ef040 con 0x56429066f390 2024-09-21T06:07:50.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.352+0000 7ff793fff640 1 -- 172.21.15.117:0/1331785450 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7ff7a40acd80 con 0x56429066f390 2024-09-21T06:07:50.370 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.352+0000 7ff7c1200740 1 -- 172.21.15.117:0/1331785450 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff79c01cad0 con 0x56429066f390 2024-09-21T06:07:50.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.352+0000 7ff793fff640 1 -- 172.21.15.117:0/1331785450 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff7a40b2030 con 0x56429066f390 2024-09-21T06:07:50.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.352+0000 7ff7c1200740 10 client.17895 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:50.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.352+0000 7ff7c1200740 1 -- 172.21.15.117:0/1331785450 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564290a55af0 con 0x56429066f390 2024-09-21T06:07:50.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.353+0000 7ff793fff640 1 -- 172.21.15.117:0/1331785450 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff79c01cad0 con 0x56429066f390 2024-09-21T06:07:50.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.353+0000 7ff793fff640 1 -- 172.21.15.117:0/1331785450 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff7a40ac5f0 con 0x56429066f390 2024-09-21T06:07:50.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.353+0000 7ff7c1200740 10 client.17895 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:50.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.353+0000 7ff7c1200740 10 client.17895 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T06:07:50.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.353+0000 7ff7c1200740 20 client.17895 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:50.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.353+0000 7ff7c1200740 1 --2- 172.21.15.117:0/1331785450 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x56429064e9f0 0x564290a58fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:50.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.353+0000 7ff7c1200740 4 client.17895 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:07:50.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.353+0000 7ff7c1200740 1 -- 172.21.15.117:0/1331785450 --> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642909afdf0 con 0x56429064e9f0 2024-09-21T06:07:50.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.353+0000 7ff7b8a28640 1 --2- 172.21.15.117:0/1331785450 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x56429064e9f0 0x564290a58fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:50.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.354+0000 7ff7b8a28640 1 --2- 172.21.15.117:0/1331785450 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x56429064e9f0 0x564290a58fc0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e16e89f53a8ff931 server_cookie=2662d154a9af6a99 in_seq=0 out_seq=0 2024-09-21T06:07:50.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.354+0000 7ff793fff640 10 client.17895 ms_handle_connect on v2:172.21.15.192:6834/3530688860 2024-09-21T06:07:50.371 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.354+0000 7ff793fff640 1 -- 172.21.15.117:0/1331785450 <== mds.3 v2:172.21.15.192:6834/3530688860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5642909afdf0 con 0x56429064e9f0 2024-09-21T06:07:50.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.354+0000 7ff793fff640 10 client.17895 handle_command_reply: tid=0 2024-09-21T06:07:50.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.354+0000 7ff7c1200740 1 client.17895 shutdown 2024-09-21T06:07:50.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.355+0000 7ff7c1200740 1 -- 172.21.15.117:0/1331785450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56429066f390 msgr2=0x564290a47260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:50.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.355+0000 7ff7c1200740 1 --2- 172.21.15.117:0/1331785450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56429066f390 0x564290a47260 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7ff7a40e5280 tx=0x7ff7a40eb1c0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.355+0000 7ff7c1200740 1 -- 172.21.15.117:0/1331785450 shutdown_connections 2024-09-21T06:07:50.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.355+0000 7ff7c1200740 1 --2- 172.21.15.117:0/1331785450 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x56429064e9f0 0x564290a58fc0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.355+0000 7ff7c1200740 1 --2- 172.21.15.117:0/1331785450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56429066f390 0x564290a47260 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.355+0000 7ff7c1200740 1 --2- 172.21.15.117:0/1331785450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564290690070 0x564290a477c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.355+0000 7ff7c1200740 1 --2- 172.21.15.117:0/1331785450 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642906671b0 0x564290a46d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.355+0000 7ff793fff640 0 client.17895 ms_handle_reset on v2:172.21.15.192:6834/3530688860 2024-09-21T06:07:50.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.355+0000 7ff7c1200740 1 -- 172.21.15.117:0/1331785450 >> 172.21.15.117:0/1331785450 conn(0x564290684460 msgr2=0x564290a38cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:50.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.355+0000 7ff7c1200740 1 -- 172.21.15.117:0/1331785450 shutdown_connections 2024-09-21T06:07:50.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.355+0000 7ff7c1200740 1 -- 172.21.15.117:0/1331785450 wait complete. 2024-09-21T06:07:50.372 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.355+0000 7ff7c1200740 20 client.17895 trim_cache size 0 max 16384 2024-09-21T06:07:50.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.368+0000 7ff7c1200740 1 Processor -- start 2024-09-21T06:07:50.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.368+0000 7ff7c1200740 1 -- start start 2024-09-21T06:07:50.376 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.368+0000 7ff7c1200740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642906671b0 0x564290a38a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:50.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.368+0000 7ff7c1200740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56429066f390 0x564290a43f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:50.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.368+0000 7ff7c1200740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564290690070 0x564290a44450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:50.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.368+0000 7ff7c1200740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b41ad3e0 con 0x56429066f390 2024-09-21T06:07:50.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.368+0000 7ff7c1200740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b41aef20 con 0x564290690070 2024-09-21T06:07:50.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.368+0000 7ff7c1200740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b41adab0 con 0x5642906671b0 2024-09-21T06:07:50.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.368+0000 7ff7b3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56429066f390 0x564290a43f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:50.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.368+0000 7ff7b3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56429066f390 0x564290a43f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36282/0 (socket says 172.21.15.117:36282) 2024-09-21T06:07:50.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.368+0000 7ff7b3fff640 1 -- 172.21.15.117:0/1139662422 learned_addr learned my addr 172.21.15.117:0/1139662422 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:50.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.368+0000 7ff7b9229640 1 --2- 172.21.15.117:0/1139662422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564290690070 0x564290a44450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:50.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.368+0000 7ff7b3fff640 1 -- 172.21.15.117:0/1139662422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642906671b0 msgr2=0x564290a38a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:50.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.368+0000 7ff7b8a28640 1 --2- 172.21.15.117:0/1139662422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642906671b0 0x564290a38a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:50.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.369+0000 7ff7b3fff640 1 --2- 172.21.15.117:0/1139662422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642906671b0 0x564290a38a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.369+0000 7ff7b3fff640 1 -- 172.21.15.117:0/1139662422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564290690070 msgr2=0x564290a44450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:50.377 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.369+0000 7ff7b3fff640 1 --2- 172.21.15.117:0/1139662422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564290690070 0x564290a44450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.369+0000 7ff7b3fff640 1 -- 172.21.15.117:0/1139662422 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564290a38100 con 0x56429066f390 2024-09-21T06:07:50.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.369+0000 7ff7b9229640 1 --2- 172.21.15.117:0/1139662422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564290690070 0x564290a44450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:07:50.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.369+0000 7ff7b3fff640 1 --2- 172.21.15.117:0/1139662422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56429066f390 0x564290a43f10 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7ff7a400c6a0 tx=0x7ff7a40d1110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c868f214223993d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:50.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.369+0000 7ff7aa7fc640 1 -- 172.21.15.117:0/1139662422 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a400ce60 con 0x56429066f390 2024-09-21T06:07:50.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.369+0000 7ff7aa7fc640 1 -- 172.21.15.117:0/1139662422 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7a40a9080 con 0x56429066f390 2024-09-21T06:07:50.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.369+0000 7ff7aa7fc640 1 -- 172.21.15.117:0/1139662422 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a40ad6f0 con 0x56429066f390 2024-09-21T06:07:50.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.370+0000 7ff7c1200740 1 -- 172.21.15.117:0/1139662422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56429066f390 msgr2=0x564290a43f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:50.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.370+0000 7ff7c1200740 1 --2- 172.21.15.117:0/1139662422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56429066f390 0x564290a43f10 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7ff7a400c6a0 tx=0x7ff7a40d1110 comp rx=0 tx=0).stop 2024-09-21T06:07:50.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.370+0000 7ff7c1200740 1 -- 172.21.15.117:0/1139662422 shutdown_connections 2024-09-21T06:07:50.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.370+0000 7ff7c1200740 1 --2- 172.21.15.117:0/1139662422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56429066f390 0x564290a43f10 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.370+0000 7ff7c1200740 1 --2- 172.21.15.117:0/1139662422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564290690070 0x564290a44450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.370+0000 7ff7c1200740 1 --2- 172.21.15.117:0/1139662422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642906671b0 0x564290a38a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.370+0000 7ff7c1200740 1 -- 172.21.15.117:0/1139662422 >> 172.21.15.117:0/1139662422 conn(0x564290684460 msgr2=0x564290a382f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:50.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.370+0000 7ff7c1200740 1 -- 172.21.15.117:0/1139662422 shutdown_connections 2024-09-21T06:07:50.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.370+0000 7ff7c1200740 1 -- 172.21.15.117:0/1139662422 wait complete. 2024-09-21T06:07:50.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.370+0000 7ff7c1200740 1 -- start start 2024-09-21T06:07:50.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.370+0000 7ff7c1200740 1 Processor -- start 2024-09-21T06:07:50.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.371+0000 7ff7c1200740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642906671b0 0x564290a4da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:50.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.371+0000 7ff7c1200740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56429066f390 0x564290a4dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:50.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.371+0000 7ff7c1200740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564290690070 0x564290abc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:50.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.371+0000 7ff7c1200740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b41ae850 con 0x564290690070 2024-09-21T06:07:50.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.371+0000 7ff7c1200740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b41ae180 con 0x56429066f390 2024-09-21T06:07:50.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.371+0000 7ff7c1200740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b41ab3e0 con 0x5642906671b0 2024-09-21T06:07:50.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.371+0000 7ff7c1200740 10 client.0 _ll_register_callbacks cb 0x7ff7a40eb2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:50.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.371+0000 7ff7b9229640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564290690070 0x564290abc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:50.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.371+0000 7ff7b9229640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564290690070 0x564290abc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36296/0 (socket says 172.21.15.117:36296) 2024-09-21T06:07:50.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.371+0000 7ff7b9229640 1 -- 172.21.15.117:0/2271331779 learned_addr learned my addr 172.21.15.117:0/2271331779 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:50.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.371+0000 7ff7b3fff640 1 --2- 172.21.15.117:0/2271331779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56429066f390 0x564290a4dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:50.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.371+0000 7ff7b8a28640 1 --2- 172.21.15.117:0/2271331779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642906671b0 0x564290a4da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:50.379 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.371+0000 7ff7b9229640 1 -- 172.21.15.117:0/2271331779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642906671b0 msgr2=0x564290a4da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:50.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.371+0000 7ff7b9229640 1 --2- 172.21.15.117:0/2271331779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642906671b0 0x564290a4da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.371+0000 7ff7b9229640 1 -- 172.21.15.117:0/2271331779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56429066f390 msgr2=0x564290a4dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:50.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.371+0000 7ff7b9229640 1 --2- 172.21.15.117:0/2271331779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56429066f390 0x564290a4dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.371+0000 7ff7b9229640 1 -- 172.21.15.117:0/2271331779 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564290abcfd0 con 0x564290690070 2024-09-21T06:07:50.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.371+0000 7ff7b8a28640 1 --2- 172.21.15.117:0/2271331779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642906671b0 0x564290a4da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:50.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.371+0000 7ff7b9229640 1 --2- 172.21.15.117:0/2271331779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564290690070 0x564290abc960 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac0317a0 tx=0x7ff7ac02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05d816af4443e17 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:50.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.371+0000 7ff793fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:50.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.371+0000 7ff793fff640 1 -- 172.21.15.117:0/2271331779 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac004540 con 0x564290690070 2024-09-21T06:07:50.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.371+0000 7ff793fff640 1 -- 172.21.15.117:0/2271331779 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7ff7ac0046e0 con 0x564290690070 2024-09-21T06:07:50.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.371+0000 7ff7c1200740 1 -- 172.21.15.117:0/2271331779 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7a40a9080 con 0x564290690070 2024-09-21T06:07:50.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.372+0000 7ff793fff640 1 -- 172.21.15.117:0/2271331779 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff7ac031af0 con 0x564290690070 2024-09-21T06:07:50.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.372+0000 7ff7c1200740 10 client.17901 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:50.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.372+0000 7ff7c1200740 1 -- 172.21.15.117:0/2271331779 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564290ac4310 con 0x564290690070 2024-09-21T06:07:50.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.372+0000 7ff793fff640 1 -- 172.21.15.117:0/2271331779 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7a40a9080 con 0x564290690070 2024-09-21T06:07:50.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.372+0000 7ff793fff640 1 -- 172.21.15.117:0/2271331779 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff7ac04f070 con 0x564290690070 2024-09-21T06:07:50.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.372+0000 7ff7c1200740 10 client.17901 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:50.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.372+0000 7ff7c1200740 10 client.17901 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T06:07:50.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.372+0000 7ff7c1200740 20 client.17901 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:50.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.372+0000 7ff7c1200740 1 --2- 172.21.15.117:0/2271331779 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x564290623da0 0x564290ac6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:50.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.372+0000 7ff7c1200740 4 client.17901 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:07:50.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.372+0000 7ff7c1200740 1 -- 172.21.15.117:0/2271331779 --> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5642909ab8a0 con 0x564290623da0 2024-09-21T06:07:50.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.372+0000 7ff7b8a28640 1 --2- 172.21.15.117:0/2271331779 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x564290623da0 0x564290ac6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:50.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.373+0000 7ff7b8a28640 1 --2- 172.21.15.117:0/2271331779 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x564290623da0 0x564290ac6b20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=97cf2742e2fb09a1 server_cookie=bf88959142078237 in_seq=0 out_seq=0 2024-09-21T06:07:50.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.373+0000 7ff793fff640 10 client.17901 ms_handle_connect on v2:172.21.15.192:6834/3530688860 2024-09-21T06:07:50.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.373+0000 7ff793fff640 1 -- 172.21.15.117:0/2271331779 <== mds.3 v2:172.21.15.192:6834/3530688860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5642909ab8a0 con 0x564290623da0 2024-09-21T06:07:50.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.373+0000 7ff793fff640 10 client.17901 handle_command_reply: tid=0 2024-09-21T06:07:50.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.373+0000 7ff7c1200740 1 client.17901 shutdown 2024-09-21T06:07:50.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.373+0000 7ff7c1200740 1 -- 172.21.15.117:0/2271331779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564290690070 msgr2=0x564290abc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:50.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.373+0000 7ff7c1200740 1 --2- 172.21.15.117:0/2271331779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564290690070 0x564290abc960 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac0317a0 tx=0x7ff7ac02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.373+0000 7ff7c1200740 1 -- 172.21.15.117:0/2271331779 shutdown_connections 2024-09-21T06:07:50.381 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.373+0000 7ff7c1200740 1 --2- 172.21.15.117:0/2271331779 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x564290623da0 0x564290ac6b20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.373+0000 7ff7c1200740 1 --2- 172.21.15.117:0/2271331779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564290690070 0x564290abc960 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.373+0000 7ff7c1200740 1 --2- 172.21.15.117:0/2271331779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56429066f390 0x564290a4dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.373+0000 7ff7c1200740 1 --2- 172.21.15.117:0/2271331779 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642906671b0 0x564290a4da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.373+0000 7ff7c1200740 1 -- 172.21.15.117:0/2271331779 >> 172.21.15.117:0/2271331779 conn(0x564290684460 msgr2=0x564290a38c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:50.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.373+0000 7ff793fff640 0 client.17901 ms_handle_reset on v2:172.21.15.192:6834/3530688860 2024-09-21T06:07:50.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.374+0000 7ff7c1200740 1 -- 172.21.15.117:0/2271331779 shutdown_connections 2024-09-21T06:07:50.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.374+0000 7ff7c1200740 1 -- 172.21.15.117:0/2271331779 wait complete. 2024-09-21T06:07:50.382 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.374+0000 7ff7c1200740 20 client.17901 trim_cache size 0 max 16384 2024-09-21T06:07:50.382 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:07:50.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.374+0000 7ff7bac8c640 1 -- 172.21.15.117:0/2067192359 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff7a00b6680 msgr2=0x7ff7a00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:50.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.374+0000 7ff7bac8c640 1 --2- 172.21.15.117:0/2067192359 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff7a00b6680 0x7ff7a00b8b90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac006010 tx=0x7ff7ac0095c0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.374+0000 7ff7bac8c640 1 -- 172.21.15.117:0/2067192359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7b41083b0 msgr2=0x7ff7b419b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:50.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.374+0000 7ff7bac8c640 1 --2- 172.21.15.117:0/2067192359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7b41083b0 0x7ff7b419b870 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7ff7a400e9c0 tx=0x7ff7a400ee80 comp rx=0 tx=0).stop 2024-09-21T06:07:50.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.374+0000 7ff7bac8c640 1 -- 172.21.15.117:0/2067192359 shutdown_connections 2024-09-21T06:07:50.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.374+0000 7ff7bac8c640 1 --2- 172.21.15.117:0/2067192359 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7ff7a00b6680 0x7ff7a00b8b90 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.374+0000 7ff7bac8c640 1 --2- 172.21.15.117:0/2067192359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff7b41083b0 0x7ff7b419b870 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.374+0000 7ff7bac8c640 1 --2- 172.21.15.117:0/2067192359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7b4108d00 0x7ff7b419bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.374+0000 7ff7bac8c640 1 --2- 172.21.15.117:0/2067192359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7b4102a70 0x7ff7b419b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:50.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.374+0000 7ff7bac8c640 1 -- 172.21.15.117:0/2067192359 >> 172.21.15.117:0/2067192359 conn(0x7ff7b4079710 msgr2=0x7ff7b41064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:50.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.374+0000 7ff7bac8c640 1 -- 172.21.15.117:0/2067192359 shutdown_connections 2024-09-21T06:07:50.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:50.374+0000 7ff7bac8c640 1 -- 172.21.15.117:0/2067192359 wait complete. 2024-09-21T06:07:51.193 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T06:07:51.443 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:07:51.647 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.642+0000 7f0aeadad640 1 -- 172.21.15.117:0/1649947800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ae4102090 msgr2=0x7f0ae41024a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:51.648 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.642+0000 7f0aeadad640 1 --2- 172.21.15.117:0/1649947800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ae4102090 0x7f0ae41024a0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8009900 tx=0x7f0ad802a410 comp rx=0 tx=0).stop 2024-09-21T06:07:51.648 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.643+0000 7f0aeadad640 1 -- 172.21.15.117:0/1649947800 shutdown_connections 2024-09-21T06:07:51.648 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.643+0000 7f0aeadad640 1 --2- 172.21.15.117:0/1649947800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ae4106b50 0x7f0ae41105c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.648 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.643+0000 7f0aeadad640 1 --2- 172.21.15.117:0/1649947800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ae4106200 0x7f0ae4106610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.648 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.643+0000 7f0aeadad640 1 --2- 172.21.15.117:0/1649947800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ae4102090 0x7f0ae41024a0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.648 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.643+0000 7f0aeadad640 1 -- 172.21.15.117:0/1649947800 >> 172.21.15.117:0/1649947800 conn(0x7f0ae4079ec0 msgr2=0x7f0ae407a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:51.648 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.643+0000 7f0aeadad640 1 -- 172.21.15.117:0/1649947800 shutdown_connections 2024-09-21T06:07:51.648 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.643+0000 7f0aeadad640 1 -- 172.21.15.117:0/1649947800 wait complete. 2024-09-21T06:07:51.648 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.643+0000 7f0aeadad640 1 Processor -- start 2024-09-21T06:07:51.648 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.643+0000 7f0aeadad640 1 -- start start 2024-09-21T06:07:51.648 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.643+0000 7f0aeadad640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ae4102090 0x7f0ae419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.648 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.643+0000 7f0aeadad640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ae4106200 0x7f0ae419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.644+0000 7f0aeadad640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ae4106b50 0x7f0ae419fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.644+0000 7f0aeadad640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae4113a00 con 0x7f0ae4102090 2024-09-21T06:07:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.644+0000 7f0aeadad640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae4113880 con 0x7f0ae4106200 2024-09-21T06:07:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.644+0000 7f0aeadad640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae4113b80 con 0x7f0ae4106b50 2024-09-21T06:07:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.644+0000 7f0ae8b49640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ae4102090 0x7f0ae419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.644+0000 7f0ae8b49640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ae4102090 0x7f0ae419a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36310/0 (socket says 172.21.15.117:36310) 2024-09-21T06:07:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.644+0000 7f0ae8b49640 1 -- 172.21.15.117:0/1101177218 learned_addr learned my addr 172.21.15.117:0/1101177218 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.644+0000 7f0ae3fff640 1 --2- 172.21.15.117:0/1101177218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ae4106200 0x7f0ae419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.644+0000 7f0ae934a640 1 --2- 172.21.15.117:0/1101177218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ae4106b50 0x7f0ae419fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.644+0000 7f0ae8b49640 1 -- 172.21.15.117:0/1101177218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ae4106b50 msgr2=0x7f0ae419fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.644+0000 7f0ae8b49640 1 --2- 172.21.15.117:0/1101177218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ae4106b50 0x7f0ae419fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.644+0000 7f0ae8b49640 1 -- 172.21.15.117:0/1101177218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ae4106200 msgr2=0x7f0ae419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.644+0000 7f0ae8b49640 1 --2- 172.21.15.117:0/1101177218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ae4106200 0x7f0ae419a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.644+0000 7f0ae8b49640 1 -- 172.21.15.117:0/1101177218 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae41a0370 con 0x7f0ae4102090 2024-09-21T06:07:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.644+0000 7f0ae3fff640 1 --2- 172.21.15.117:0/1101177218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ae4106200 0x7f0ae419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.644+0000 7f0ae8b49640 1 --2- 172.21.15.117:0/1101177218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ae4102090 0x7f0ae419a440 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8031a90 tx=0x7f0ad8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.644+0000 7f0ae1ffb640 1 -- 172.21.15.117:0/1101177218 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad803f030 con 0x7f0ae4102090 2024-09-21T06:07:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.644+0000 7f0ae1ffb640 1 -- 172.21.15.117:0/1101177218 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad8003cb0 con 0x7f0ae4102090 2024-09-21T06:07:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.644+0000 7f0aeadad640 1 -- 172.21.15.117:0/1101177218 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ae41a0600 con 0x7f0ae4102090 2024-09-21T06:07:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.644+0000 7f0ae1ffb640 1 -- 172.21.15.117:0/1101177218 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8035070 con 0x7f0ae4102090 2024-09-21T06:07:51.650 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.645+0000 7f0aeadad640 1 -- 172.21.15.117:0/1101177218 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ae41a6df0 con 0x7f0ae4102090 2024-09-21T06:07:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.647+0000 7f0af1df9740 1 Processor -- start 2024-09-21T06:07:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.647+0000 7f0af1df9740 1 -- start start 2024-09-21T06:07:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.647+0000 7f0ae1ffb640 1 -- 172.21.15.117:0/1101177218 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f0ad8042030 con 0x7f0ae4102090 2024-09-21T06:07:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.647+0000 7f0af1df9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d94edee1b0 0x55d94f1939b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.647+0000 7f0af1df9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d94edc52f0 0x55d94f193ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.647+0000 7f0af1df9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d94edcd4d0 0x55d94f194430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.647+0000 7f0af1df9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d94f1086f0 con 0x55d94edc52f0 2024-09-21T06:07:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.647+0000 7f0af1df9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d94f0f7b20 con 0x55d94edcd4d0 2024-09-21T06:07:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.647+0000 7f0af1df9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55d94f0fb930 con 0x55d94edee1b0 2024-09-21T06:07:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.648+0000 7f0ae8b49640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d94edc52f0 0x55d94f193ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.648+0000 7f0ae8b49640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d94edc52f0 0x55d94f193ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36324/0 (socket says 172.21.15.117:36324) 2024-09-21T06:07:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.648+0000 7f0ae8b49640 1 -- 172.21.15.117:0/2323781345 learned_addr learned my addr 172.21.15.117:0/2323781345 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.648+0000 7f0ae934a640 1 --2- 172.21.15.117:0/2323781345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d94edee1b0 0x55d94f1939b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.648+0000 7f0ae3fff640 1 --2- 172.21.15.117:0/2323781345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d94edcd4d0 0x55d94f194430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.648+0000 7f0ae8b49640 1 -- 172.21.15.117:0/2323781345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d94edee1b0 msgr2=0x55d94f1939b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.648+0000 7f0ae8b49640 1 --2- 172.21.15.117:0/2323781345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d94edee1b0 0x55d94f1939b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.657 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.648+0000 7f0ae8b49640 1 -- 172.21.15.117:0/2323781345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d94edcd4d0 msgr2=0x55d94f194430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.648+0000 7f0ae8b49640 1 --2- 172.21.15.117:0/2323781345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d94edcd4d0 0x55d94f194430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.648+0000 7f0ae8b49640 1 -- 172.21.15.117:0/2323781345 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d94f0bd530 con 0x55d94edc52f0 2024-09-21T06:07:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.648+0000 7f0ae3fff640 1 --2- 172.21.15.117:0/2323781345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d94edcd4d0 0x55d94f194430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.648+0000 7f0ae8b49640 1 --2- 172.21.15.117:0/2323781345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d94edc52f0 0x55d94f193ef0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8105480 tx=0x7f0ad810e460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b31cef8014ee7fea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.648+0000 7f0ae1ffb640 1 --2- 172.21.15.117:0/1101177218 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0ad40b6680 0x7f0ad40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.648+0000 7f0aceffd640 1 -- 172.21.15.117:0/2323781345 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad80cd560 con 0x55d94edc52f0 2024-09-21T06:07:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.649+0000 7f0aceffd640 1 -- 172.21.15.117:0/2323781345 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad80ce310 con 0x55d94edc52f0 2024-09-21T06:07:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.649+0000 7f0ae934a640 1 --2- 172.21.15.117:0/1101177218 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0ad40b6680 0x7f0ad40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.649+0000 7f0aceffd640 1 -- 172.21.15.117:0/2323781345 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad80cd750 con 0x55d94edc52f0 2024-09-21T06:07:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.649+0000 7f0ae1ffb640 1 -- 172.21.15.117:0/1101177218 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f0ad8107d70 con 0x7f0ae4102090 2024-09-21T06:07:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.649+0000 7f0af1df9740 1 -- 172.21.15.117:0/2323781345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d94edc52f0 msgr2=0x55d94f193ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.649+0000 7f0ae934a640 1 --2- 172.21.15.117:0/1101177218 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0ad40b6680 0x7f0ad40b8b90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0adc008e40 tx=0x7f0adc0085f0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.649+0000 7f0af1df9740 1 --2- 172.21.15.117:0/2323781345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d94edc52f0 0x55d94f193ef0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8105480 tx=0x7f0ad810e460 comp rx=0 tx=0).stop 2024-09-21T06:07:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.649+0000 7f0af1df9740 1 -- 172.21.15.117:0/2323781345 shutdown_connections 2024-09-21T06:07:51.658 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.649+0000 7f0af1df9740 1 --2- 172.21.15.117:0/2323781345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d94edc52f0 0x55d94f193ef0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.649+0000 7f0af1df9740 1 --2- 172.21.15.117:0/2323781345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d94edcd4d0 0x55d94f194430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.649+0000 7f0af1df9740 1 --2- 172.21.15.117:0/2323781345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d94edee1b0 0x55d94f1939b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.649+0000 7f0af1df9740 1 -- 172.21.15.117:0/2323781345 >> 172.21.15.117:0/2323781345 conn(0x55d94ede2300 msgr2=0x55d94f1918a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.649+0000 7f0af1df9740 1 -- 172.21.15.117:0/2323781345 shutdown_connections 2024-09-21T06:07:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.650+0000 7f0af1df9740 1 -- 172.21.15.117:0/2323781345 wait complete. 2024-09-21T06:07:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.650+0000 7f0af1df9740 1 -- start start 2024-09-21T06:07:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.650+0000 7f0af1df9740 1 Processor -- start 2024-09-21T06:07:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.650+0000 7f0af1df9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d94edc52f0 0x55d94f19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.650+0000 7f0af1df9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d94edcd4d0 0x55d94f19f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.650+0000 7f0af1df9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d94edee1b0 0x55d94f19fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.650+0000 7f0af1df9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d94f02fab0 con 0x55d94edcd4d0 2024-09-21T06:07:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.650+0000 7f0af1df9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d94f09d730 con 0x55d94edee1b0 2024-09-21T06:07:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.650+0000 7f0af1df9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55d94f1a1ef0 con 0x55d94edc52f0 2024-09-21T06:07:51.659 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.650+0000 7f0af1df9740 10 client.0 _ll_register_callbacks cb 0x55d94f0f16b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:51.660 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.650+0000 7f0ae3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d94edcd4d0 0x55d94f19f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:51.660 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.651+0000 7f0ae3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d94edcd4d0 0x55d94f19f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36334/0 (socket says 172.21.15.117:36334) 2024-09-21T06:07:51.660 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.651+0000 7f0ae3fff640 1 -- 172.21.15.117:0/785765840 learned_addr learned my addr 172.21.15.117:0/785765840 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:51.660 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.651+0000 7f0ae934a640 1 --2- 172.21.15.117:0/785765840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d94edee1b0 0x55d94f19fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:51.660 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.651+0000 7f0ae8b49640 1 --2- 172.21.15.117:0/785765840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d94edc52f0 0x55d94f19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:51.660 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.651+0000 7f0ae3fff640 1 -- 172.21.15.117:0/785765840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d94edc52f0 msgr2=0x55d94f19efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:51.660 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.651+0000 7f0ae3fff640 1 --2- 172.21.15.117:0/785765840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d94edc52f0 0x55d94f19efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.660 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.651+0000 7f0ae3fff640 1 -- 172.21.15.117:0/785765840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d94edee1b0 msgr2=0x55d94f19fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:51.660 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.651+0000 7f0ae3fff640 1 --2- 172.21.15.117:0/785765840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d94edee1b0 0x55d94f19fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.660 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.651+0000 7f0ae3fff640 1 -- 172.21.15.117:0/785765840 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d94ee59790 con 0x55d94edcd4d0 2024-09-21T06:07:51.660 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.651+0000 7f0ae8b49640 1 --2- 172.21.15.117:0/785765840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d94edc52f0 0x55d94f19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:07:51.660 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.651+0000 7f0ae3fff640 1 --2- 172.21.15.117:0/785765840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d94edcd4d0 0x55d94f19f4f0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f0ad00107d0 tx=0x7f0ad0010c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=908e934dbe99de5f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.651+0000 7f0ac3fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.651+0000 7f0ac3fff640 1 -- 172.21.15.117:0/785765840 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad000ed40 con 0x55d94edcd4d0 2024-09-21T06:07:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.651+0000 7f0ac3fff640 1 -- 172.21.15.117:0/785765840 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f0ad001c480 con 0x55d94edcd4d0 2024-09-21T06:07:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.651+0000 7f0af1df9740 1 -- 172.21.15.117:0/785765840 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ad80ce310 con 0x55d94edcd4d0 2024-09-21T06:07:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.651+0000 7f0ac3fff640 1 -- 172.21.15.117:0/785765840 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0ad001c680 con 0x55d94edcd4d0 2024-09-21T06:07:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.651+0000 7f0af1df9740 10 client.17913 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.651+0000 7f0af1df9740 1 -- 172.21.15.117:0/785765840 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d94eed2620 con 0x55d94edcd4d0 2024-09-21T06:07:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.651+0000 7f0ac3fff640 1 -- 172.21.15.117:0/785765840 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad80ce310 con 0x55d94edcd4d0 2024-09-21T06:07:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.652+0000 7f0ac3fff640 1 -- 172.21.15.117:0/785765840 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0ad0011810 con 0x55d94edcd4d0 2024-09-21T06:07:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.652+0000 7f0af1df9740 10 client.17913 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.652+0000 7f0af1df9740 10 client.17913 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:07:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.652+0000 7f0af1df9740 20 client.17913 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.652+0000 7f0af1df9740 1 --2- 172.21.15.117:0/785765840 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55d94edacd80 0x55d94f1b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.652+0000 7f0af1df9740 4 client.17913 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:07:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.652+0000 7f0af1df9740 1 -- 172.21.15.117:0/785765840 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d94f10a640 con 0x55d94edacd80 2024-09-21T06:07:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.652+0000 7f0ae8b49640 1 --2- 172.21.15.117:0/785765840 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55d94edacd80 0x55d94f1b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.653+0000 7f0ac3fff640 10 client.17913 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.653+0000 7f0ae8b49640 1 --2- 172.21.15.117:0/785765840 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55d94edacd80 0x55d94f1b0ed0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b99ee205f5b2d83 server_cookie=6b6ca7602710382f in_seq=0 out_seq=0 2024-09-21T06:07:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.653+0000 7f0ac3fff640 1 -- 172.21.15.117:0/785765840 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d94f10a640 con 0x55d94edacd80 2024-09-21T06:07:51.662 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.653+0000 7f0ac3fff640 10 client.17913 handle_command_reply: tid=0 2024-09-21T06:07:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.653+0000 7f0af1df9740 1 client.17913 shutdown 2024-09-21T06:07:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.653+0000 7f0af1df9740 1 -- 172.21.15.117:0/785765840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d94edcd4d0 msgr2=0x55d94f19f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.653+0000 7f0af1df9740 1 --2- 172.21.15.117:0/785765840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d94edcd4d0 0x55d94f19f4f0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f0ad00107d0 tx=0x7f0ad0010c90 comp rx=0 tx=0).stop 2024-09-21T06:07:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.653+0000 7f0af1df9740 1 -- 172.21.15.117:0/785765840 shutdown_connections 2024-09-21T06:07:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.653+0000 7f0af1df9740 1 --2- 172.21.15.117:0/785765840 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55d94edacd80 0x55d94f1b0ed0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.653+0000 7f0af1df9740 1 --2- 172.21.15.117:0/785765840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d94edcd4d0 0x55d94f19f4f0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.653+0000 7f0af1df9740 1 --2- 172.21.15.117:0/785765840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d94edee1b0 0x55d94f19fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.653+0000 7f0af1df9740 1 --2- 172.21.15.117:0/785765840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d94edc52f0 0x55d94f19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.653+0000 7f0ac3fff640 0 client.17913 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.653+0000 7f0af1df9740 1 -- 172.21.15.117:0/785765840 >> 172.21.15.117:0/785765840 conn(0x55d94ede2300 msgr2=0x55d94f1949e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.654+0000 7f0af1df9740 1 -- 172.21.15.117:0/785765840 shutdown_connections 2024-09-21T06:07:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.654+0000 7f0af1df9740 1 -- 172.21.15.117:0/785765840 wait complete. 2024-09-21T06:07:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.654+0000 7f0af1df9740 20 client.17913 trim_cache size 0 max 16384 2024-09-21T06:07:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.668+0000 7f0af1df9740 1 Processor -- start 2024-09-21T06:07:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.668+0000 7f0af1df9740 1 -- start start 2024-09-21T06:07:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.668+0000 7f0af1df9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d94edc52f0 0x55d94f1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.668+0000 7f0af1df9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d94edcd4d0 0x55d94f19c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.668+0000 7f0af1df9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d94edee1b0 0x55d94f19c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.668+0000 7f0af1df9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae41af5f0 con 0x55d94edee1b0 2024-09-21T06:07:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.668+0000 7f0af1df9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae41b1130 con 0x55d94edc52f0 2024-09-21T06:07:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.668+0000 7f0af1df9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae41afcc0 con 0x55d94edcd4d0 2024-09-21T06:07:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.668+0000 7f0ae934a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d94edee1b0 0x55d94f19c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.668+0000 7f0ae934a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d94edee1b0 0x55d94f19c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36350/0 (socket says 172.21.15.117:36350) 2024-09-21T06:07:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.668+0000 7f0ae934a640 1 -- 172.21.15.117:0/1640931129 learned_addr learned my addr 172.21.15.117:0/1640931129 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.668+0000 7f0ae8b49640 1 --2- 172.21.15.117:0/1640931129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d94edc52f0 0x55d94f1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.668+0000 7f0ae934a640 1 -- 172.21.15.117:0/1640931129 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d94edcd4d0 msgr2=0x55d94f19c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.668+0000 7f0ae934a640 1 --2- 172.21.15.117:0/1640931129 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d94edcd4d0 0x55d94f19c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.668+0000 7f0ae934a640 1 -- 172.21.15.117:0/1640931129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d94edc52f0 msgr2=0x55d94f1a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.668+0000 7f0ae934a640 1 --2- 172.21.15.117:0/1640931129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d94edc52f0 0x55d94f1a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.669+0000 7f0ae934a640 1 -- 172.21.15.117:0/1640931129 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d94f1a1af0 con 0x55d94edee1b0 2024-09-21T06:07:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.669+0000 7f0ae934a640 1 --2- 172.21.15.117:0/1640931129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d94edee1b0 0x55d94f19c6c0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f0adc008c20 tx=0x7f0adc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de137bf8aa15b543 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.669+0000 7f0ace7fc640 1 -- 172.21.15.117:0/1640931129 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0adc01c840 con 0x55d94edee1b0 2024-09-21T06:07:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.669+0000 7f0ace7fc640 1 -- 172.21.15.117:0/1640931129 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0adc01ce90 con 0x55d94edee1b0 2024-09-21T06:07:51.673 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.669+0000 7f0ace7fc640 1 -- 172.21.15.117:0/1640931129 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0adc00e3b0 con 0x55d94edee1b0 2024-09-21T06:07:51.676 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.670+0000 7f0af1df9740 1 -- 172.21.15.117:0/1640931129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d94edee1b0 msgr2=0x55d94f19c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:51.676 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.670+0000 7f0af1df9740 1 --2- 172.21.15.117:0/1640931129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d94edee1b0 0x55d94f19c6c0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f0adc008c20 tx=0x7f0adc014000 comp rx=0 tx=0).stop 2024-09-21T06:07:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.670+0000 7f0af1df9740 1 -- 172.21.15.117:0/1640931129 shutdown_connections 2024-09-21T06:07:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.670+0000 7f0af1df9740 1 --2- 172.21.15.117:0/1640931129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d94edee1b0 0x55d94f19c6c0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.670+0000 7f0af1df9740 1 --2- 172.21.15.117:0/1640931129 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d94edcd4d0 0x55d94f19c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.670+0000 7f0af1df9740 1 --2- 172.21.15.117:0/1640931129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d94edc52f0 0x55d94f1a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.670+0000 7f0af1df9740 1 -- 172.21.15.117:0/1640931129 >> 172.21.15.117:0/1640931129 conn(0x55d94ede2300 msgr2=0x55d94f1a04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.670+0000 7f0af1df9740 1 -- 172.21.15.117:0/1640931129 shutdown_connections 2024-09-21T06:07:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.670+0000 7f0af1df9740 1 -- 172.21.15.117:0/1640931129 wait complete. 2024-09-21T06:07:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.670+0000 7f0af1df9740 1 -- start start 2024-09-21T06:07:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.670+0000 7f0af1df9740 1 Processor -- start 2024-09-21T06:07:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.671+0000 7f0af1df9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d94edc52f0 0x55d94f1a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.677 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.671+0000 7f0af1df9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d94edcd4d0 0x55d94f1a5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.671+0000 7f0af1df9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d94edee1b0 0x55d94f1a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.671+0000 7f0af1df9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae41b0a60 con 0x55d94edcd4d0 2024-09-21T06:07:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.671+0000 7f0af1df9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae41b0390 con 0x55d94edee1b0 2024-09-21T06:07:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.671+0000 7f0af1df9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae41ad5f0 con 0x55d94edc52f0 2024-09-21T06:07:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.671+0000 7f0af1df9740 10 client.0 _ll_register_callbacks cb 0x7f0ad000c410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.671+0000 7f0ae3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d94edcd4d0 0x55d94f1a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.671+0000 7f0ae3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d94edcd4d0 0x55d94f1a5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36360/0 (socket says 172.21.15.117:36360) 2024-09-21T06:07:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.671+0000 7f0ae3fff640 1 -- 172.21.15.117:0/1399281490 learned_addr learned my addr 172.21.15.117:0/1399281490 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.671+0000 7f0ae934a640 1 --2- 172.21.15.117:0/1399281490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d94edee1b0 0x55d94f1a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.671+0000 7f0ae8b49640 1 --2- 172.21.15.117:0/1399281490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d94edc52f0 0x55d94f1a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.672+0000 7f0ae3fff640 1 -- 172.21.15.117:0/1399281490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d94edc52f0 msgr2=0x55d94f1a51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.672+0000 7f0ae3fff640 1 --2- 172.21.15.117:0/1399281490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d94edc52f0 0x55d94f1a51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.672+0000 7f0ae3fff640 1 -- 172.21.15.117:0/1399281490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d94edee1b0 msgr2=0x55d94f1a5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:51.678 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.672+0000 7f0ae3fff640 1 --2- 172.21.15.117:0/1399281490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d94edee1b0 0x55d94f1a5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.672+0000 7f0ae3fff640 1 -- 172.21.15.117:0/1399281490 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d94f192240 con 0x55d94edcd4d0 2024-09-21T06:07:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.672+0000 7f0ae3fff640 1 --2- 172.21.15.117:0/1399281490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d94edcd4d0 0x55d94f1a5700 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0010470 tx=0x7f0ad00116f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbadfdaf491ef0ef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.672+0000 7f0ac3fff640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.672+0000 7f0ac3fff640 1 -- 172.21.15.117:0/1399281490 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad002b070 con 0x55d94edcd4d0 2024-09-21T06:07:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.672+0000 7f0ac3fff640 1 -- 172.21.15.117:0/1399281490 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f0ad0005070 con 0x55d94edcd4d0 2024-09-21T06:07:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.672+0000 7f0af1df9740 1 -- 172.21.15.117:0/1399281490 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0adc01ce90 con 0x55d94edcd4d0 2024-09-21T06:07:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.672+0000 7f0ac3fff640 1 -- 172.21.15.117:0/1399281490 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0ad000e720 con 0x55d94edcd4d0 2024-09-21T06:07:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.672+0000 7f0af1df9740 10 client.17919 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.672+0000 7f0af1df9740 1 -- 172.21.15.117:0/1399281490 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d94f21ba80 con 0x55d94edcd4d0 2024-09-21T06:07:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.673+0000 7f0ac3fff640 1 -- 172.21.15.117:0/1399281490 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0adc01ce90 con 0x55d94edcd4d0 2024-09-21T06:07:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.673+0000 7f0ac3fff640 1 -- 172.21.15.117:0/1399281490 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0ad001c820 con 0x55d94edcd4d0 2024-09-21T06:07:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.673+0000 7f0af1df9740 10 client.17919 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.673+0000 7f0af1df9740 10 client.17919 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:07:51.679 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.673+0000 7f0af1df9740 20 client.17919 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.673+0000 7f0af1df9740 1 --2- 172.21.15.117:0/1399281490 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55d94ed821c0 0x55d94f21e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.673+0000 7f0af1df9740 4 client.17919 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:07:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.673+0000 7f0af1df9740 1 -- 172.21.15.117:0/1399281490 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d94f1060f0 con 0x55d94ed821c0 2024-09-21T06:07:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.673+0000 7f0ae8b49640 1 --2- 172.21.15.117:0/1399281490 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55d94ed821c0 0x55d94f21e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.674+0000 7f0ae8b49640 1 --2- 172.21.15.117:0/1399281490 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55d94ed821c0 0x55d94f21e190 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc74777c92d3563a server_cookie=b825346ac496b202 in_seq=0 out_seq=0 2024-09-21T06:07:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.674+0000 7f0ac3fff640 10 client.17919 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.681+0000 7f0ac3fff640 1 -- 172.21.15.117:0/1399281490 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d94f1060f0 con 0x55d94ed821c0 2024-09-21T06:07:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.681+0000 7f0ac3fff640 10 client.17919 handle_command_reply: tid=0 2024-09-21T06:07:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.681+0000 7f0af1df9740 1 client.17919 shutdown 2024-09-21T06:07:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.681+0000 7f0af1df9740 1 -- 172.21.15.117:0/1399281490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d94edcd4d0 msgr2=0x55d94f1a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.681+0000 7f0af1df9740 1 --2- 172.21.15.117:0/1399281490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d94edcd4d0 0x55d94f1a5700 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0010470 tx=0x7f0ad00116f0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.681+0000 7f0af1df9740 1 -- 172.21.15.117:0/1399281490 shutdown_connections 2024-09-21T06:07:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.681+0000 7f0af1df9740 1 --2- 172.21.15.117:0/1399281490 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55d94ed821c0 0x55d94f21e190 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.681+0000 7f0af1df9740 1 --2- 172.21.15.117:0/1399281490 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d94edcd4d0 0x55d94f1a5700 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.681+0000 7f0af1df9740 1 --2- 172.21.15.117:0/1399281490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d94edee1b0 0x55d94f1a5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.681+0000 7f0ac3fff640 0 client.17919 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.681+0000 7f0af1df9740 1 --2- 172.21.15.117:0/1399281490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d94edc52f0 0x55d94f1a51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.681+0000 7f0af1df9740 1 -- 172.21.15.117:0/1399281490 >> 172.21.15.117:0/1399281490 conn(0x55d94ede2300 msgr2=0x55d94f195170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.681+0000 7f0af1df9740 1 -- 172.21.15.117:0/1399281490 shutdown_connections 2024-09-21T06:07:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.681+0000 7f0af1df9740 1 -- 172.21.15.117:0/1399281490 wait complete. 2024-09-21T06:07:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.681+0000 7f0af1df9740 20 client.17919 trim_cache size 0 max 16384 2024-09-21T06:07:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T06:07:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T06:07:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T06:07:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T06:07:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.682+0000 7f0aeadad640 1 -- 172.21.15.117:0/1101177218 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0ad40b6680 msgr2=0x7f0ad40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:51.687 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.682+0000 7f0aeadad640 1 --2- 172.21.15.117:0/1101177218 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0ad40b6680 0x7f0ad40b8b90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0adc008e40 tx=0x7f0adc0085f0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.687 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.682+0000 7f0aeadad640 1 -- 172.21.15.117:0/1101177218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ae4102090 msgr2=0x7f0ae419a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:51.687 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.682+0000 7f0aeadad640 1 --2- 172.21.15.117:0/1101177218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ae4102090 0x7f0ae419a440 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8031a90 tx=0x7f0ad8004270 comp rx=0 tx=0).stop 2024-09-21T06:07:51.687 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.682+0000 7f0aeadad640 1 -- 172.21.15.117:0/1101177218 shutdown_connections 2024-09-21T06:07:51.687 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.682+0000 7f0aeadad640 1 --2- 172.21.15.117:0/1101177218 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f0ad40b6680 0x7f0ad40b8b90 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.687 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.682+0000 7f0aeadad640 1 --2- 172.21.15.117:0/1101177218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ae4106b50 0x7f0ae419fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.687 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.682+0000 7f0aeadad640 1 --2- 172.21.15.117:0/1101177218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ae4106200 0x7f0ae419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.687 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.682+0000 7f0aeadad640 1 --2- 172.21.15.117:0/1101177218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ae4102090 0x7f0ae419a440 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.687 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.682+0000 7f0aeadad640 1 -- 172.21.15.117:0/1101177218 >> 172.21.15.117:0/1101177218 conn(0x7f0ae4079ec0 msgr2=0x7f0ae4107470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:51.687 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.682+0000 7f0aeadad640 1 -- 172.21.15.117:0/1101177218 shutdown_connections 2024-09-21T06:07:51.687 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:07:51.682+0000 7f0aeadad640 1 -- 172.21.15.117:0/1101177218 wait complete. 2024-09-21T06:07:51.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.918+0000 7fc6f0783640 1 -- 172.21.15.117:0/18984374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6e8108d20 msgr2=0x7fc6e810f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:51.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.918+0000 7fc6f0783640 1 --2- 172.21.15.117:0/18984374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6e8108d20 0x7fc6e810f550 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009ca0 tx=0x7fc6e002a8c0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.919+0000 7fc6f0783640 1 -- 172.21.15.117:0/18984374 shutdown_connections 2024-09-21T06:07:51.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.919+0000 7fc6f0783640 1 --2- 172.21.15.117:0/18984374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6e8108d20 0x7fc6e810f550 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.919+0000 7fc6f0783640 1 --2- 172.21.15.117:0/18984374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6e81083d0 0x7fc6e81087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.919+0000 7fc6f0783640 1 --2- 172.21.15.117:0/18984374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6e8102a90 0x7fc6e8102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.919+0000 7fc6f0783640 1 -- 172.21.15.117:0/18984374 >> 172.21.15.117:0/18984374 conn(0x7fc6e8079750 msgr2=0x7fc6e8079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:51.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.919+0000 7fc6f0783640 1 -- 172.21.15.117:0/18984374 shutdown_connections 2024-09-21T06:07:51.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.919+0000 7fc6f0783640 1 -- 172.21.15.117:0/18984374 wait complete. 2024-09-21T06:07:51.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.919+0000 7fc6f0783640 1 Processor -- start 2024-09-21T06:07:51.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.919+0000 7fc6f0783640 1 -- start start 2024-09-21T06:07:51.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.920+0000 7fc6f0783640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6e8102a90 0x7fc6e8199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.920+0000 7fc6f0783640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6e81083d0 0x7fc6e81998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.920+0000 7fc6f0783640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6e8108d20 0x7fc6e819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.920+0000 7fc6f0783640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e8112990 con 0x7fc6e81083d0 2024-09-21T06:07:51.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.920+0000 7fc6f0783640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e8112810 con 0x7fc6e8108d20 2024-09-21T06:07:51.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.920+0000 7fc6f0783640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e8112b10 con 0x7fc6e8102a90 2024-09-21T06:07:51.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.920+0000 7fc6edd1e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6e81083d0 0x7fc6e81998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:51.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.920+0000 7fc6edd1e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6e81083d0 0x7fc6e81998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36384/0 (socket says 172.21.15.117:36384) 2024-09-21T06:07:51.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.920+0000 7fc6edd1e640 1 -- 172.21.15.117:0/975959390 learned_addr learned my addr 172.21.15.117:0/975959390 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:51.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.920+0000 7fc6ee51f640 1 --2- 172.21.15.117:0/975959390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6e8102a90 0x7fc6e8199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:51.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.920+0000 7fc6eed20640 1 --2- 172.21.15.117:0/975959390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6e8108d20 0x7fc6e819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:51.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.920+0000 7fc6edd1e640 1 -- 172.21.15.117:0/975959390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6e8102a90 msgr2=0x7fc6e8199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:51.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.920+0000 7fc6edd1e640 1 --2- 172.21.15.117:0/975959390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6e8102a90 0x7fc6e8199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.920+0000 7fc6edd1e640 1 -- 172.21.15.117:0/975959390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6e8108d20 msgr2=0x7fc6e819eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:51.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.920+0000 7fc6edd1e640 1 --2- 172.21.15.117:0/975959390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6e8108d20 0x7fc6e819eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.920+0000 7fc6edd1e640 1 -- 172.21.15.117:0/975959390 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e819f2b0 con 0x7fc6e81083d0 2024-09-21T06:07:51.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.920+0000 7fc6eed20640 1 --2- 172.21.15.117:0/975959390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6e8108d20 0x7fc6e819eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:07:51.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.920+0000 7fc6ee51f640 1 --2- 172.21.15.117:0/975959390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6e8102a90 0x7fc6e8199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:51.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.920+0000 7fc6edd1e640 1 --2- 172.21.15.117:0/975959390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6e81083d0 0x7fc6e81998e0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8005fd0 tx=0x7fc6d800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:51.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.921+0000 7fc6df7fe640 1 -- 172.21.15.117:0/975959390 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8017070 con 0x7fc6e81083d0 2024-09-21T06:07:51.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.921+0000 7fc6f0783640 1 -- 172.21.15.117:0/975959390 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6e819f5a0 con 0x7fc6e81083d0 2024-09-21T06:07:51.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.921+0000 7fc6df7fe640 1 -- 172.21.15.117:0/975959390 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6d8009c30 con 0x7fc6e81083d0 2024-09-21T06:07:51.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.921+0000 7fc6df7fe640 1 -- 172.21.15.117:0/975959390 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8003d30 con 0x7fc6e81083d0 2024-09-21T06:07:51.935 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.921+0000 7fc6f0783640 1 -- 172.21.15.117:0/975959390 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6e81a5d00 con 0x7fc6e81083d0 2024-09-21T06:07:51.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.923+0000 7fc6f6cf7740 1 Processor -- start 2024-09-21T06:07:51.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.923+0000 7fc6f6cf7740 1 -- start start 2024-09-21T06:07:51.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.924+0000 7fc6f6cf7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648ccb74070 0x5648ccf1b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.924+0000 7fc6f6cf7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5648ccb4b1b0 0x5648ccf1b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.924+0000 7fc6f6cf7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648ccb53390 0x5648ccf1bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.924+0000 7fc6f6cf7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5648cce91ea0 con 0x5648ccb74070 2024-09-21T06:07:51.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.924+0000 7fc6f6cf7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5648cce812d0 con 0x5648ccb53390 2024-09-21T06:07:51.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.924+0000 7fc6f6cf7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5648cce850e0 con 0x5648ccb4b1b0 2024-09-21T06:07:51.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.924+0000 7fc6eed20640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648ccb74070 0x5648ccf1b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:51.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.924+0000 7fc6eed20640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648ccb74070 0x5648ccf1b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36388/0 (socket says 172.21.15.117:36388) 2024-09-21T06:07:51.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.924+0000 7fc6eed20640 1 -- 172.21.15.117:0/287001199 learned_addr learned my addr 172.21.15.117:0/287001199 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:51.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.924+0000 7fc6ee51f640 1 --2- 172.21.15.117:0/287001199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5648ccb4b1b0 0x5648ccf1b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:51.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.924+0000 7fc6df7fe640 1 -- 172.21.15.117:0/975959390 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fc6d8004020 con 0x7fc6e81083d0 2024-09-21T06:07:51.936 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.924+0000 7fc6eed20640 1 -- 172.21.15.117:0/287001199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5648ccb4b1b0 msgr2=0x5648ccf1b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:51.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.924+0000 7fc6eed20640 1 --2- 172.21.15.117:0/287001199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5648ccb4b1b0 0x5648ccf1b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.924+0000 7fc6eed20640 1 -- 172.21.15.117:0/287001199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648ccb53390 msgr2=0x5648ccf1bd40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T06:07:51.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.924+0000 7fc6eed20640 1 --2- 172.21.15.117:0/287001199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648ccb53390 0x5648ccf1bd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.924+0000 7fc6eed20640 1 -- 172.21.15.117:0/287001199 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648ccbdf650 con 0x5648ccb74070 2024-09-21T06:07:51.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.924+0000 7fc6ee51f640 1 --2- 172.21.15.117:0/287001199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5648ccb4b1b0 0x5648ccf1b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:07:51.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.924+0000 7fc6eed20640 1 --2- 172.21.15.117:0/287001199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648ccb74070 0x5648ccf1b2c0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0006010 tx=0x7fc6e0009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3999d953e0af8caa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:51.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.924+0000 7fc6dcff9640 1 -- 172.21.15.117:0/287001199 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0031cf0 con 0x5648ccb74070 2024-09-21T06:07:51.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.925+0000 7fc6dcff9640 1 -- 172.21.15.117:0/287001199 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6e0036070 con 0x5648ccb74070 2024-09-21T06:07:51.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.925+0000 7fc6dcff9640 1 -- 172.21.15.117:0/287001199 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e00042a0 con 0x5648ccb74070 2024-09-21T06:07:51.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.925+0000 7fc6df7fe640 1 --2- 172.21.15.117:0/975959390 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc6d40b6680 0x7fc6d40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.925+0000 7fc6ee51f640 1 --2- 172.21.15.117:0/975959390 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc6d40b6680 0x7fc6d40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:51.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.925+0000 7fc6df7fe640 1 -- 172.21.15.117:0/975959390 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7fc6d80e7400 con 0x7fc6e81083d0 2024-09-21T06:07:51.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.926+0000 7fc6f6cf7740 1 -- 172.21.15.117:0/287001199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648ccb74070 msgr2=0x5648ccf1b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:51.937 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.926+0000 7fc6ee51f640 1 --2- 172.21.15.117:0/975959390 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc6d40b6680 0x7fc6d40b8b90 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x5648ccf19f40 tx=0x7fc6e4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:51.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.926+0000 7fc6f6cf7740 1 --2- 172.21.15.117:0/287001199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648ccb74070 0x5648ccf1b2c0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0006010 tx=0x7fc6e0009800 comp rx=0 tx=0).stop 2024-09-21T06:07:51.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.926+0000 7fc6f6cf7740 1 -- 172.21.15.117:0/287001199 shutdown_connections 2024-09-21T06:07:51.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.926+0000 7fc6f6cf7740 1 --2- 172.21.15.117:0/287001199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648ccb53390 0x5648ccf1bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.926+0000 7fc6f6cf7740 1 --2- 172.21.15.117:0/287001199 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5648ccb4b1b0 0x5648ccf1b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.926+0000 7fc6f6cf7740 1 --2- 172.21.15.117:0/287001199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648ccb74070 0x5648ccf1b2c0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.926+0000 7fc6f6cf7740 1 -- 172.21.15.117:0/287001199 >> 172.21.15.117:0/287001199 conn(0x5648ccb68460 msgr2=0x5648ccf191b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:51.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.926+0000 7fc6f6cf7740 1 -- 172.21.15.117:0/287001199 shutdown_connections 2024-09-21T06:07:51.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.926+0000 7fc6f6cf7740 1 -- 172.21.15.117:0/287001199 wait complete. 2024-09-21T06:07:51.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.926+0000 7fc6f6cf7740 1 -- start start 2024-09-21T06:07:51.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.926+0000 7fc6f6cf7740 1 Processor -- start 2024-09-21T06:07:51.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.926+0000 7fc6f6cf7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5648ccb4b1b0 0x5648ccf28b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.926+0000 7fc6f6cf7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648ccb53390 0x5648ccf29080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.926+0000 7fc6f6cf7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648ccb74070 0x5648ccf29600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.927+0000 7fc6f6cf7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5648ccce2700 con 0x5648ccb53390 2024-09-21T06:07:51.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.927+0000 7fc6f6cf7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5648cce1c160 con 0x5648ccb74070 2024-09-21T06:07:51.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.927+0000 7fc6f6cf7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5648ccf2bba0 con 0x5648ccb4b1b0 2024-09-21T06:07:51.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.927+0000 7fc6f6cf7740 10 client.0 _ll_register_callbacks cb 0x5648cce359e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:51.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.927+0000 7fc6edd1e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648ccb53390 0x5648ccf29080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:51.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.927+0000 7fc6edd1e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648ccb53390 0x5648ccf29080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36400/0 (socket says 172.21.15.117:36400) 2024-09-21T06:07:51.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.927+0000 7fc6edd1e640 1 -- 172.21.15.117:0/4047368073 learned_addr learned my addr 172.21.15.117:0/4047368073 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:51.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.927+0000 7fc6ee51f640 1 --2- 172.21.15.117:0/4047368073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5648ccb4b1b0 0x5648ccf28b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:51.939 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.927+0000 7fc6eed20640 1 --2- 172.21.15.117:0/4047368073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648ccb74070 0x5648ccf29600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:51.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.927+0000 7fc6edd1e640 1 -- 172.21.15.117:0/4047368073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5648ccb4b1b0 msgr2=0x5648ccf28b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:51.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.927+0000 7fc6edd1e640 1 --2- 172.21.15.117:0/4047368073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5648ccb4b1b0 0x5648ccf28b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.927+0000 7fc6edd1e640 1 -- 172.21.15.117:0/4047368073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648ccb74070 msgr2=0x5648ccf29600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:51.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.927+0000 7fc6edd1e640 1 --2- 172.21.15.117:0/4047368073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648ccb74070 0x5648ccf29600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.927+0000 7fc6edd1e640 1 -- 172.21.15.117:0/4047368073 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648cce49c90 con 0x5648ccb53390 2024-09-21T06:07:51.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.927+0000 7fc6eed20640 1 --2- 172.21.15.117:0/4047368073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648ccb74070 0x5648ccf29600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:51.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.927+0000 7fc6edd1e640 1 --2- 172.21.15.117:0/4047368073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648ccb53390 0x5648ccf29080 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fc6d80e4c00 tx=0x7fc6d80aca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e620f06f61f1a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:51.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.927+0000 7fc6d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:51.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.927+0000 7fc6d1ffb640 1 -- 172.21.15.117:0/4047368073 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8009de0 con 0x5648ccb53390 2024-09-21T06:07:51.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.927+0000 7fc6d1ffb640 1 -- 172.21.15.117:0/4047368073 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7fc6d80ac500 con 0x5648ccb53390 2024-09-21T06:07:51.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.927+0000 7fc6f6cf7740 1 -- 172.21.15.117:0/4047368073 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6e0036070 con 0x5648ccb53390 2024-09-21T06:07:51.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.927+0000 7fc6d1ffb640 1 -- 172.21.15.117:0/4047368073 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc6d80ad9a0 con 0x5648ccb53390 2024-09-21T06:07:51.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.927+0000 7fc6f6cf7740 10 client.17931 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:51.940 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.927+0000 7fc6f6cf7740 1 -- 172.21.15.117:0/4047368073 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5648ccf37820 con 0x5648ccb53390 2024-09-21T06:07:51.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.928+0000 7fc6d1ffb640 1 -- 172.21.15.117:0/4047368073 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6e0036070 con 0x5648ccb53390 2024-09-21T06:07:51.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.928+0000 7fc6d1ffb640 1 -- 172.21.15.117:0/4047368073 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc6d80e6070 con 0x5648ccb53390 2024-09-21T06:07:51.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.928+0000 7fc6f6cf7740 10 client.17931 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:51.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.928+0000 7fc6f6cf7740 10 client.17931 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:07:51.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.928+0000 7fc6f6cf7740 20 client.17931 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:51.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.928+0000 7fc6f6cf7740 1 --2- 172.21.15.117:0/4047368073 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x5648ccb329f0 0x5648ccf3abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.928+0000 7fc6f6cf7740 4 client.17931 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:07:51.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.928+0000 7fc6f6cf7740 1 -- 172.21.15.117:0/4047368073 --> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648cce93df0 con 0x5648ccb329f0 2024-09-21T06:07:51.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.928+0000 7fc6ee51f640 1 --2- 172.21.15.117:0/4047368073 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x5648ccb329f0 0x5648ccf3abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:51.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.929+0000 7fc6d1ffb640 10 client.17931 ms_handle_connect on v2:172.21.15.192:6836/2683860842 2024-09-21T06:07:51.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.929+0000 7fc6ee51f640 1 --2- 172.21.15.117:0/4047368073 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x5648ccb329f0 0x5648ccf3abf0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=436595a876f69a2 server_cookie=3e7c598db1c65071 in_seq=0 out_seq=0 2024-09-21T06:07:51.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.929+0000 7fc6d1ffb640 1 -- 172.21.15.117:0/4047368073 <== mds.2 v2:172.21.15.192:6836/2683860842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5648cce93df0 con 0x5648ccb329f0 2024-09-21T06:07:51.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.929+0000 7fc6d1ffb640 10 client.17931 handle_command_reply: tid=0 2024-09-21T06:07:51.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.930+0000 7fc6f6cf7740 1 client.17931 shutdown 2024-09-21T06:07:51.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.930+0000 7fc6f6cf7740 1 -- 172.21.15.117:0/4047368073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648ccb53390 msgr2=0x5648ccf29080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:51.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.930+0000 7fc6f6cf7740 1 --2- 172.21.15.117:0/4047368073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648ccb53390 0x5648ccf29080 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fc6d80e4c00 tx=0x7fc6d80aca10 comp rx=0 tx=0).stop 2024-09-21T06:07:51.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.930+0000 7fc6f6cf7740 1 -- 172.21.15.117:0/4047368073 shutdown_connections 2024-09-21T06:07:51.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.930+0000 7fc6f6cf7740 1 --2- 172.21.15.117:0/4047368073 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x5648ccb329f0 0x5648ccf3abf0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.930+0000 7fc6f6cf7740 1 --2- 172.21.15.117:0/4047368073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648ccb53390 0x5648ccf29080 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.930+0000 7fc6f6cf7740 1 --2- 172.21.15.117:0/4047368073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648ccb74070 0x5648ccf29600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.930+0000 7fc6f6cf7740 1 --2- 172.21.15.117:0/4047368073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5648ccb4b1b0 0x5648ccf28b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.930+0000 7fc6f6cf7740 1 -- 172.21.15.117:0/4047368073 >> 172.21.15.117:0/4047368073 conn(0x5648ccb68460 msgr2=0x5648ccf17750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:51.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.930+0000 7fc6d1ffb640 0 client.17931 ms_handle_reset on v2:172.21.15.192:6836/2683860842 2024-09-21T06:07:51.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.930+0000 7fc6f6cf7740 1 -- 172.21.15.117:0/4047368073 shutdown_connections 2024-09-21T06:07:51.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.930+0000 7fc6f6cf7740 1 -- 172.21.15.117:0/4047368073 wait complete. 2024-09-21T06:07:51.942 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.930+0000 7fc6f6cf7740 20 client.17931 trim_cache size 0 max 16384 2024-09-21T06:07:51.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.943+0000 7fc6f6cf7740 1 Processor -- start 2024-09-21T06:07:51.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.943+0000 7fc6f6cf7740 1 -- start start 2024-09-21T06:07:51.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.944+0000 7fc6f6cf7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648ccb4b1b0 0x5648ccf242f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.944+0000 7fc6f6cf7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5648ccb53390 0x5648ccf191e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.944+0000 7fc6f6cf7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648ccb74070 0x5648ccf25d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.944+0000 7fc6f6cf7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e81ad410 con 0x5648ccb4b1b0 2024-09-21T06:07:51.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.944+0000 7fc6f6cf7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e81aef50 con 0x5648ccb74070 2024-09-21T06:07:51.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.944+0000 7fc6f6cf7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e81adae0 con 0x5648ccb53390 2024-09-21T06:07:51.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.944+0000 7fc6ee51f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648ccb4b1b0 0x5648ccf242f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:51.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.944+0000 7fc6ee51f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648ccb4b1b0 0x5648ccf242f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36416/0 (socket says 172.21.15.117:36416) 2024-09-21T06:07:51.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.944+0000 7fc6ee51f640 1 -- 172.21.15.117:0/1758004157 learned_addr learned my addr 172.21.15.117:0/1758004157 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:51.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.944+0000 7fc6edd1e640 1 --2- 172.21.15.117:0/1758004157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5648ccb53390 0x5648ccf191e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:51.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.944+0000 7fc6eed20640 1 --2- 172.21.15.117:0/1758004157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648ccb74070 0x5648ccf25d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:51.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.944+0000 7fc6ee51f640 1 -- 172.21.15.117:0/1758004157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5648ccb53390 msgr2=0x5648ccf191e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:51.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.944+0000 7fc6ee51f640 1 --2- 172.21.15.117:0/1758004157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5648ccb53390 0x5648ccf191e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.944+0000 7fc6ee51f640 1 -- 172.21.15.117:0/1758004157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648ccb74070 msgr2=0x5648ccf25d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:51.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.944+0000 7fc6ee51f640 1 --2- 172.21.15.117:0/1758004157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648ccb74070 0x5648ccf25d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.944+0000 7fc6ee51f640 1 -- 172.21.15.117:0/1758004157 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648ccf29be0 con 0x5648ccb4b1b0 2024-09-21T06:07:51.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.944+0000 7fc6ee51f640 1 --2- 172.21.15.117:0/1758004157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648ccb4b1b0 0x5648ccf242f0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4013fd0 tx=0x7fc6e4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f9b6a622a5d8a0c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:51.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.944+0000 7fc6d3fff640 1 -- 172.21.15.117:0/1758004157 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e401eb00 con 0x5648ccb4b1b0 2024-09-21T06:07:51.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.945+0000 7fc6d3fff640 1 -- 172.21.15.117:0/1758004157 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6e401eca0 con 0x5648ccb4b1b0 2024-09-21T06:07:51.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.945+0000 7fc6d3fff640 1 -- 172.21.15.117:0/1758004157 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e40106a0 con 0x5648ccb4b1b0 2024-09-21T06:07:51.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.945+0000 7fc6f6cf7740 1 -- 172.21.15.117:0/1758004157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648ccb4b1b0 msgr2=0x5648ccf242f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:51.951 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.945+0000 7fc6f6cf7740 1 --2- 172.21.15.117:0/1758004157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648ccb4b1b0 0x5648ccf242f0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4013fd0 tx=0x7fc6e4015000 comp rx=0 tx=0).stop 2024-09-21T06:07:51.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.945+0000 7fc6f6cf7740 1 -- 172.21.15.117:0/1758004157 shutdown_connections 2024-09-21T06:07:51.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.945+0000 7fc6f6cf7740 1 --2- 172.21.15.117:0/1758004157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648ccb74070 0x5648ccf25d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.945+0000 7fc6f6cf7740 1 --2- 172.21.15.117:0/1758004157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5648ccb53390 0x5648ccf191e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.945+0000 7fc6f6cf7740 1 --2- 172.21.15.117:0/1758004157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648ccb4b1b0 0x5648ccf242f0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.945+0000 7fc6f6cf7740 1 -- 172.21.15.117:0/1758004157 >> 172.21.15.117:0/1758004157 conn(0x5648ccb68460 msgr2=0x5648ccf18ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:51.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.946+0000 7fc6f6cf7740 1 -- 172.21.15.117:0/1758004157 shutdown_connections 2024-09-21T06:07:51.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.946+0000 7fc6f6cf7740 1 -- 172.21.15.117:0/1758004157 wait complete. 2024-09-21T06:07:51.952 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.946+0000 7fc6f6cf7740 1 -- start start 2024-09-21T06:07:51.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.946+0000 7fc6f6cf7740 1 Processor -- start 2024-09-21T06:07:51.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.946+0000 7fc6f6cf7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648ccb4b1b0 0x5648ccf31600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.946+0000 7fc6f6cf7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648ccb53390 0x5648ccf2cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.946+0000 7fc6f6cf7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5648ccb74070 0x5648ccf2cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.946+0000 7fc6f6cf7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e81ae880 con 0x5648ccb4b1b0 2024-09-21T06:07:51.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.946+0000 7fc6f6cf7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e81ae1b0 con 0x5648ccb53390 2024-09-21T06:07:51.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.946+0000 7fc6f6cf7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e81ab410 con 0x5648ccb74070 2024-09-21T06:07:51.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.946+0000 7fc6f6cf7740 10 client.0 _ll_register_callbacks cb 0x7fc6d8005bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:51.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.946+0000 7fc6ee51f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648ccb4b1b0 0x5648ccf31600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:51.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.946+0000 7fc6ee51f640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648ccb4b1b0 0x5648ccf31600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36428/0 (socket says 172.21.15.117:36428) 2024-09-21T06:07:51.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.946+0000 7fc6ee51f640 1 -- 172.21.15.117:0/340449676 learned_addr learned my addr 172.21.15.117:0/340449676 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:51.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.947+0000 7fc6eed20640 1 --2- 172.21.15.117:0/340449676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5648ccb74070 0x5648ccf2cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:51.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.947+0000 7fc6edd1e640 1 --2- 172.21.15.117:0/340449676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648ccb53390 0x5648ccf2cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:51.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.947+0000 7fc6ee51f640 1 -- 172.21.15.117:0/340449676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5648ccb74070 msgr2=0x5648ccf2cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:51.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.947+0000 7fc6ee51f640 1 --2- 172.21.15.117:0/340449676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5648ccb74070 0x5648ccf2cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.947+0000 7fc6ee51f640 1 -- 172.21.15.117:0/340449676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648ccb53390 msgr2=0x5648ccf2cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:51.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.947+0000 7fc6ee51f640 1 --2- 172.21.15.117:0/340449676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648ccb53390 0x5648ccf2cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.947+0000 7fc6ee51f640 1 -- 172.21.15.117:0/340449676 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648ccf2d730 con 0x5648ccb4b1b0 2024-09-21T06:07:51.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.947+0000 7fc6eed20640 1 --2- 172.21.15.117:0/340449676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5648ccb74070 0x5648ccf2cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:51.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.947+0000 7fc6edd1e640 1 --2- 172.21.15.117:0/340449676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648ccb53390 0x5648ccf2cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:51.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.947+0000 7fc6ee51f640 1 --2- 172.21.15.117:0/340449676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648ccb4b1b0 0x5648ccf31600 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4029d60 tx=0x7fc6e401ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3411be7821c3c11 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:51.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.947+0000 7fc6d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:51.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.947+0000 7fc6d1ffb640 1 -- 172.21.15.117:0/340449676 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e403d060 con 0x5648ccb4b1b0 2024-09-21T06:07:51.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.947+0000 7fc6d1ffb640 1 -- 172.21.15.117:0/340449676 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7fc6e402c040 con 0x5648ccb4b1b0 2024-09-21T06:07:51.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.947+0000 7fc6f6cf7740 1 -- 172.21.15.117:0/340449676 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6e401eca0 con 0x5648ccb4b1b0 2024-09-21T06:07:51.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.947+0000 7fc6d1ffb640 1 -- 172.21.15.117:0/340449676 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc6e4029610 con 0x5648ccb4b1b0 2024-09-21T06:07:51.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.947+0000 7fc6f6cf7740 10 client.17937 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:51.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.947+0000 7fc6f6cf7740 1 -- 172.21.15.117:0/340449676 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5648ccfa5490 con 0x5648ccb4b1b0 2024-09-21T06:07:51.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.948+0000 7fc6d1ffb640 1 -- 172.21.15.117:0/340449676 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6e401eca0 con 0x5648ccb4b1b0 2024-09-21T06:07:51.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.948+0000 7fc6d1ffb640 1 -- 172.21.15.117:0/340449676 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc6e4004a20 con 0x5648ccb4b1b0 2024-09-21T06:07:51.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.948+0000 7fc6f6cf7740 10 client.17937 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:51.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.948+0000 7fc6f6cf7740 10 client.17937 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:07:51.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.948+0000 7fc6f6cf7740 20 client.17937 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:51.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.948+0000 7fc6f6cf7740 1 --2- 172.21.15.117:0/340449676 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x5648ccb07da0 0x5648ccfa7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:51.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.948+0000 7fc6f6cf7740 4 client.17937 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:07:51.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.948+0000 7fc6f6cf7740 1 -- 172.21.15.117:0/340449676 --> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5648cce8f8a0 con 0x5648ccb07da0 2024-09-21T06:07:51.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.948+0000 7fc6edd1e640 1 --2- 172.21.15.117:0/340449676 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x5648ccb07da0 0x5648ccfa7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:51.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.949+0000 7fc6edd1e640 1 --2- 172.21.15.117:0/340449676 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x5648ccb07da0 0x5648ccfa7bc0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4102e157c4263313 server_cookie=5803eb88f2aa917b in_seq=0 out_seq=0 2024-09-21T06:07:51.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.949+0000 7fc6d1ffb640 10 client.17937 ms_handle_connect on v2:172.21.15.192:6836/2683860842 2024-09-21T06:07:51.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.949+0000 7fc6d1ffb640 1 -- 172.21.15.117:0/340449676 <== mds.2 v2:172.21.15.192:6836/2683860842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5648cce8f8a0 con 0x5648ccb07da0 2024-09-21T06:07:51.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.949+0000 7fc6d1ffb640 10 client.17937 handle_command_reply: tid=0 2024-09-21T06:07:51.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.949+0000 7fc6f6cf7740 1 client.17937 shutdown 2024-09-21T06:07:51.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.949+0000 7fc6f6cf7740 1 -- 172.21.15.117:0/340449676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648ccb4b1b0 msgr2=0x5648ccf31600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:51.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.949+0000 7fc6f6cf7740 1 --2- 172.21.15.117:0/340449676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648ccb4b1b0 0x5648ccf31600 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4029d60 tx=0x7fc6e401ec20 comp rx=0 tx=0).stop 2024-09-21T06:07:51.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.949+0000 7fc6f6cf7740 1 -- 172.21.15.117:0/340449676 shutdown_connections 2024-09-21T06:07:51.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.949+0000 7fc6f6cf7740 1 --2- 172.21.15.117:0/340449676 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x5648ccb07da0 0x5648ccfa7bc0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.949+0000 7fc6f6cf7740 1 --2- 172.21.15.117:0/340449676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5648ccb74070 0x5648ccf2cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.949+0000 7fc6f6cf7740 1 --2- 172.21.15.117:0/340449676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648ccb53390 0x5648ccf2cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.949+0000 7fc6f6cf7740 1 --2- 172.21.15.117:0/340449676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5648ccb4b1b0 0x5648ccf31600 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.949+0000 7fc6f6cf7740 1 -- 172.21.15.117:0/340449676 >> 172.21.15.117:0/340449676 conn(0x5648ccb68460 msgr2=0x5648ccf33df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:51.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.949+0000 7fc6d1ffb640 0 client.17937 ms_handle_reset on v2:172.21.15.192:6836/2683860842 2024-09-21T06:07:51.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.949+0000 7fc6f6cf7740 1 -- 172.21.15.117:0/340449676 shutdown_connections 2024-09-21T06:07:51.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.949+0000 7fc6f6cf7740 1 -- 172.21.15.117:0/340449676 wait complete. 2024-09-21T06:07:51.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.949+0000 7fc6f6cf7740 20 client.17937 trim_cache size 0 max 16384 2024-09-21T06:07:51.958 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:07:51.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.950+0000 7fc6f0783640 1 -- 172.21.15.117:0/975959390 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc6d40b6680 msgr2=0x7fc6d40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:51.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.950+0000 7fc6f0783640 1 --2- 172.21.15.117:0/975959390 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc6d40b6680 0x7fc6d40b8b90 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x5648ccf19f40 tx=0x7fc6e4005f90 comp rx=0 tx=0).stop 2024-09-21T06:07:51.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.950+0000 7fc6f0783640 1 -- 172.21.15.117:0/975959390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6e81083d0 msgr2=0x7fc6e81998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:51.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.950+0000 7fc6f0783640 1 --2- 172.21.15.117:0/975959390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6e81083d0 0x7fc6e81998e0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8005fd0 tx=0x7fc6d800dde0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.959 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.950+0000 7fc6f0783640 1 -- 172.21.15.117:0/975959390 shutdown_connections 2024-09-21T06:07:51.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.950+0000 7fc6f0783640 1 --2- 172.21.15.117:0/975959390 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc6d40b6680 0x7fc6d40b8b90 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.950+0000 7fc6f0783640 1 --2- 172.21.15.117:0/975959390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6e81083d0 0x7fc6e81998e0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.950+0000 7fc6f0783640 1 --2- 172.21.15.117:0/975959390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6e8108d20 0x7fc6e819eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.950+0000 7fc6f0783640 1 --2- 172.21.15.117:0/975959390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6e8102a90 0x7fc6e8199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:51.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.950+0000 7fc6f0783640 1 -- 172.21.15.117:0/975959390 >> 172.21.15.117:0/975959390 conn(0x7fc6e8079750 msgr2=0x7fc6e81043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:51.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.950+0000 7fc6f0783640 1 -- 172.21.15.117:0/975959390 shutdown_connections 2024-09-21T06:07:51.960 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:51.950+0000 7fc6f0783640 1 -- 172.21.15.117:0/975959390 wait complete. 2024-09-21T06:07:52.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:51 smithi192 ceph-mon[34700]: pgmap v650: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:07:52.261 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:51 smithi117 ceph-mon[30947]: pgmap v650: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:07:52.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:51 smithi138 ceph-mon[34391]: pgmap v650: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-09-21T06:07:52.725 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T06:07:52.974 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:07:53.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.490+0000 7fa7f0305640 1 -- 172.21.15.117:0/3228761352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7e810de30 msgr2=0x7fa7e8110310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:53.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.490+0000 7fa7f0305640 1 --2- 172.21.15.117:0/3228761352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7e810de30 0x7fa7e8110310 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8009ca0 tx=0x7fa7d802a8c0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.490+0000 7fa7f0305640 1 -- 172.21.15.117:0/3228761352 shutdown_connections 2024-09-21T06:07:53.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.490+0000 7fa7f0305640 1 --2- 172.21.15.117:0/3228761352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7e810de30 0x7fa7e8110310 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.490+0000 7fa7f0305640 1 --2- 172.21.15.117:0/3228761352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7e81052b0 0x7fa7e810d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.493 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.490+0000 7fa7f0305640 1 --2- 172.21.15.117:0/3228761352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7e8104880 0x7fa7e8104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.490+0000 7fa7f0305640 1 -- 172.21.15.117:0/3228761352 >> 172.21.15.117:0/3228761352 conn(0x7fa7e8079740 msgr2=0x7fa7e8079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:53.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.490+0000 7fa7f0305640 1 -- 172.21.15.117:0/3228761352 shutdown_connections 2024-09-21T06:07:53.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.490+0000 7fa7f0305640 1 -- 172.21.15.117:0/3228761352 wait complete. 2024-09-21T06:07:53.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.491+0000 7fa7f0305640 1 Processor -- start 2024-09-21T06:07:53.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.491+0000 7fa7f0305640 1 -- start start 2024-09-21T06:07:53.495 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.491+0000 7fa7f0305640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7e8104880 0x7fa7e819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:53.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.491+0000 7fa7f0305640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7e81052b0 0x7fa7e819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:53.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.491+0000 7fa7f0305640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7e810de30 0x7fa7e819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:53.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.491+0000 7fa7f0305640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e8113a80 con 0x7fa7e8104880 2024-09-21T06:07:53.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.491+0000 7fa7f0305640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e8113900 con 0x7fa7e810de30 2024-09-21T06:07:53.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.491+0000 7fa7f0305640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e8113c00 con 0x7fa7e81052b0 2024-09-21T06:07:53.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.491+0000 7fa7ee0a1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7e8104880 0x7fa7e819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:53.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.491+0000 7fa7ee0a1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7e8104880 0x7fa7e819a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36444/0 (socket says 172.21.15.117:36444) 2024-09-21T06:07:53.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.491+0000 7fa7ee0a1640 1 -- 172.21.15.117:0/3535638133 learned_addr learned my addr 172.21.15.117:0/3535638133 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:53.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.491+0000 7fa7ed8a0640 1 --2- 172.21.15.117:0/3535638133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7e81052b0 0x7fa7e819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:53.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.491+0000 7fa7ee8a2640 1 --2- 172.21.15.117:0/3535638133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7e810de30 0x7fa7e819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:53.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.492+0000 7fa7ee0a1640 1 -- 172.21.15.117:0/3535638133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7e81052b0 msgr2=0x7fa7e819a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:53.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.492+0000 7fa7ee0a1640 1 --2- 172.21.15.117:0/3535638133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7e81052b0 0x7fa7e819a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.492+0000 7fa7ee0a1640 1 -- 172.21.15.117:0/3535638133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7e810de30 msgr2=0x7fa7e819fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:53.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.492+0000 7fa7ee0a1640 1 --2- 172.21.15.117:0/3535638133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7e810de30 0x7fa7e819fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.492+0000 7fa7ee0a1640 1 -- 172.21.15.117:0/3535638133 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e81a03a0 con 0x7fa7e8104880 2024-09-21T06:07:53.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.492+0000 7fa7ee0a1640 1 --2- 172.21.15.117:0/3535638133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7e8104880 0x7fa7e819a430 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fa7e000d950 tx=0x7fa7e000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:53.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.492+0000 7fa7e77fe640 1 -- 172.21.15.117:0/3535638133 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0017070 con 0x7fa7e8104880 2024-09-21T06:07:53.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.492+0000 7fa7e77fe640 1 -- 172.21.15.117:0/3535638133 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7e0009c30 con 0x7fa7e8104880 2024-09-21T06:07:53.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.492+0000 7fa7f0305640 1 -- 172.21.15.117:0/3535638133 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7e81a0690 con 0x7fa7e8104880 2024-09-21T06:07:53.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.492+0000 7fa7e77fe640 1 -- 172.21.15.117:0/3535638133 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e000f620 con 0x7fa7e8104880 2024-09-21T06:07:53.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.492+0000 7fa7f0305640 1 -- 172.21.15.117:0/3535638133 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7e81a6df0 con 0x7fa7e8104880 2024-09-21T06:07:53.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.495+0000 7fa7f6879740 1 Processor -- start 2024-09-21T06:07:53.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.495+0000 7fa7f6879740 1 -- start start 2024-09-21T06:07:53.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.495+0000 7fa7f6879740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b3a7d070 0x55b6b3e246a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:53.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.495+0000 7fa7f6879740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6b3a541b0 0x55b6b3e24be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:53.500 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.495+0000 7fa7e77fe640 1 -- 172.21.15.117:0/3535638133 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fa7e000f7c0 con 0x7fa7e8104880 2024-09-21T06:07:53.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.495+0000 7fa7f6879740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b6b3a5c390 0x55b6b3e25120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:53.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.495+0000 7fa7f6879740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b6b3d9aea0 con 0x55b6b3a541b0 2024-09-21T06:07:53.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.495+0000 7fa7f6879740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b6b3d8a2d0 con 0x55b6b3a7d070 2024-09-21T06:07:53.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.495+0000 7fa7f6879740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b6b3d8e0e0 con 0x55b6b3a5c390 2024-09-21T06:07:53.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.495+0000 7fa7ee0a1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6b3a541b0 0x55b6b3e24be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:53.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.495+0000 7fa7ee0a1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6b3a541b0 0x55b6b3e24be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36460/0 (socket says 172.21.15.117:36460) 2024-09-21T06:07:53.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.495+0000 7fa7ee0a1640 1 -- 172.21.15.117:0/1282969987 learned_addr learned my addr 172.21.15.117:0/1282969987 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:53.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.495+0000 7fa7ed8a0640 1 --2- 172.21.15.117:0/1282969987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b6b3a5c390 0x55b6b3e25120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:53.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.496+0000 7fa7ee8a2640 1 --2- 172.21.15.117:0/1282969987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b3a7d070 0x55b6b3e246a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:53.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.496+0000 7fa7ee0a1640 1 -- 172.21.15.117:0/1282969987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b6b3a5c390 msgr2=0x55b6b3e25120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:53.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.496+0000 7fa7ee0a1640 1 --2- 172.21.15.117:0/1282969987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b6b3a5c390 0x55b6b3e25120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.496+0000 7fa7ee0a1640 1 -- 172.21.15.117:0/1282969987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b3a7d070 msgr2=0x55b6b3e246a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:53.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.496+0000 7fa7ee0a1640 1 --2- 172.21.15.117:0/1282969987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b3a7d070 0x55b6b3e246a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.496+0000 7fa7ee0a1640 1 -- 172.21.15.117:0/1282969987 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6b3ae8650 con 0x55b6b3a541b0 2024-09-21T06:07:53.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.496+0000 7fa7ed8a0640 1 --2- 172.21.15.117:0/1282969987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b6b3a5c390 0x55b6b3e25120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:07:53.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.496+0000 7fa7ee0a1640 1 --2- 172.21.15.117:0/1282969987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6b3a541b0 0x55b6b3e24be0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00e4970 tx=0x7fa7e00ef930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af8bab98b73164d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:53.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.496+0000 7fa7e77fe640 1 --2- 172.21.15.117:0/3535638133 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa7d00b6700 0x7fa7d00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:53.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.496+0000 7fa7e4d9a640 1 -- 172.21.15.117:0/1282969987 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e00fa070 con 0x55b6b3a541b0 2024-09-21T06:07:53.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.496+0000 7fa7e4d9a640 1 -- 172.21.15.117:0/1282969987 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7e00acc90 con 0x55b6b3a541b0 2024-09-21T06:07:53.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.496+0000 7fa7e4d9a640 1 -- 172.21.15.117:0/1282969987 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e00ac8f0 con 0x55b6b3a541b0 2024-09-21T06:07:53.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.496+0000 7fa7ee8a2640 1 --2- 172.21.15.117:0/3535638133 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa7d00b6700 0x7fa7d00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:53.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.497+0000 7fa7e77fe640 1 -- 172.21.15.117:0/3535638133 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7fa7e00e7230 con 0x7fa7e8104880 2024-09-21T06:07:53.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.497+0000 7fa7f6879740 1 -- 172.21.15.117:0/1282969987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6b3a541b0 msgr2=0x55b6b3e24be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:53.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.497+0000 7fa7ee8a2640 1 --2- 172.21.15.117:0/3535638133 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa7d00b6700 0x7fa7d00b8c10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa7d802f500 tx=0x7fa7d802f370 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:53.503 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.497+0000 7fa7f6879740 1 --2- 172.21.15.117:0/1282969987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6b3a541b0 0x55b6b3e24be0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00e4970 tx=0x7fa7e00ef930 comp rx=0 tx=0).stop 2024-09-21T06:07:53.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.497+0000 7fa7f6879740 1 -- 172.21.15.117:0/1282969987 shutdown_connections 2024-09-21T06:07:53.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.497+0000 7fa7f6879740 1 --2- 172.21.15.117:0/1282969987 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6b3a541b0 0x55b6b3e24be0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.497+0000 7fa7f6879740 1 --2- 172.21.15.117:0/1282969987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b6b3a5c390 0x55b6b3e25120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.497+0000 7fa7f6879740 1 --2- 172.21.15.117:0/1282969987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b3a7d070 0x55b6b3e246a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.497+0000 7fa7f6879740 1 -- 172.21.15.117:0/1282969987 >> 172.21.15.117:0/1282969987 conn(0x55b6b3a71460 msgr2=0x55b6b3e22590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:53.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.497+0000 7fa7f6879740 1 -- 172.21.15.117:0/1282969987 shutdown_connections 2024-09-21T06:07:53.504 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.497+0000 7fa7f6879740 1 -- 172.21.15.117:0/1282969987 wait complete. 2024-09-21T06:07:53.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.498+0000 7fa7f6879740 1 -- start start 2024-09-21T06:07:53.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.498+0000 7fa7f6879740 1 Processor -- start 2024-09-21T06:07:53.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.498+0000 7fa7f6879740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6b3a541b0 0x55b6b3e34120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:53.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.498+0000 7fa7f6879740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b3a5c390 0x55b6b3e34660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:53.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.498+0000 7fa7f6879740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b6b3a7d070 0x55b6b3e34be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:53.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.498+0000 7fa7f6879740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b6b3beb700 con 0x55b6b3a541b0 2024-09-21T06:07:53.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.498+0000 7fa7f6879740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b6b3d25160 con 0x55b6b3a5c390 2024-09-21T06:07:53.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.498+0000 7fa7f6879740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b6b3e371e0 con 0x55b6b3a7d070 2024-09-21T06:07:53.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.498+0000 7fa7f6879740 10 client.0 _ll_register_callbacks cb 0x55b6b3d3e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:53.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.498+0000 7fa7ee0a1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6b3a541b0 0x55b6b3e34120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:53.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.498+0000 7fa7ee0a1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6b3a541b0 0x55b6b3e34120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36476/0 (socket says 172.21.15.117:36476) 2024-09-21T06:07:53.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.498+0000 7fa7ee0a1640 1 -- 172.21.15.117:0/1798816862 learned_addr learned my addr 172.21.15.117:0/1798816862 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:53.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.498+0000 7fa7ed8a0640 1 --2- 172.21.15.117:0/1798816862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b3a5c390 0x55b6b3e34660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:53.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.498+0000 7fa7ee8a2640 1 --2- 172.21.15.117:0/1798816862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b6b3a7d070 0x55b6b3e34be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:53.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.499+0000 7fa7ee0a1640 1 -- 172.21.15.117:0/1798816862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b6b3a7d070 msgr2=0x55b6b3e34be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:53.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.499+0000 7fa7ee0a1640 1 --2- 172.21.15.117:0/1798816862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b6b3a7d070 0x55b6b3e34be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.499+0000 7fa7ee0a1640 1 -- 172.21.15.117:0/1798816862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b3a5c390 msgr2=0x55b6b3e34660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:53.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.499+0000 7fa7ee0a1640 1 --2- 172.21.15.117:0/1798816862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b3a5c390 0x55b6b3e34660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.499+0000 7fa7ee0a1640 1 -- 172.21.15.117:0/1798816862 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6b3d52c90 con 0x55b6b3a541b0 2024-09-21T06:07:53.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.499+0000 7fa7ed8a0640 1 --2- 172.21.15.117:0/1798816862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b3a5c390 0x55b6b3e34660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:07:53.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.499+0000 7fa7ee0a1640 1 --2- 172.21.15.117:0/1798816862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6b3a541b0 0x55b6b3e34120 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00f8040 tx=0x7fa7e0010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f87da78821121ea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:53.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.499+0000 7fa7cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:53.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.499+0000 7fa7cdffb640 1 -- 172.21.15.117:0/1798816862 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0020030 con 0x55b6b3a541b0 2024-09-21T06:07:53.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.499+0000 7fa7cdffb640 1 -- 172.21.15.117:0/1798816862 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7fa7e00b6660 con 0x55b6b3a541b0 2024-09-21T06:07:53.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.499+0000 7fa7f6879740 1 -- 172.21.15.117:0/1798816862 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7e00acc90 con 0x55b6b3a541b0 2024-09-21T06:07:53.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.499+0000 7fa7cdffb640 1 -- 172.21.15.117:0/1798816862 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa7e00b8990 con 0x55b6b3a541b0 2024-09-21T06:07:53.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.499+0000 7fa7f6879740 10 client.17949 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:53.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.499+0000 7fa7f6879740 1 -- 172.21.15.117:0/1798816862 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b6b3e42ef0 con 0x55b6b3a541b0 2024-09-21T06:07:53.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.499+0000 7fa7cdffb640 1 -- 172.21.15.117:0/1798816862 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7e00acc90 con 0x55b6b3a541b0 2024-09-21T06:07:53.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.500+0000 7fa7cdffb640 1 -- 172.21.15.117:0/1798816862 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa7e00ac3e0 con 0x55b6b3a541b0 2024-09-21T06:07:53.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.500+0000 7fa7f6879740 10 client.17949 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:53.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.500+0000 7fa7f6879740 10 client.17949 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:07:53.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.500+0000 7fa7f6879740 20 client.17949 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:53.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.500+0000 7fa7f6879740 1 --2- 172.21.15.117:0/1798816862 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55b6b3a3b9f0 0x55b6b3e46250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:53.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.500+0000 7fa7f6879740 4 client.17949 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:07:53.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.500+0000 7fa7f6879740 1 -- 172.21.15.117:0/1798816862 --> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6b3d9cdf0 con 0x55b6b3a3b9f0 2024-09-21T06:07:53.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.500+0000 7fa7ed8a0640 1 --2- 172.21.15.117:0/1798816862 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55b6b3a3b9f0 0x55b6b3e46250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:53.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.500+0000 7fa7cdffb640 10 client.17949 ms_handle_connect on v2:172.21.15.192:6838/1727623231 2024-09-21T06:07:53.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.500+0000 7fa7ed8a0640 1 --2- 172.21.15.117:0/1798816862 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55b6b3a3b9f0 0x55b6b3e46250 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ddd0ba09ed8c58e8 server_cookie=dd5ec80b7dc72962 in_seq=0 out_seq=0 2024-09-21T06:07:53.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.501+0000 7fa7cdffb640 1 -- 172.21.15.117:0/1798816862 <== mds.1 v2:172.21.15.192:6838/1727623231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b6b3d9cdf0 con 0x55b6b3a3b9f0 2024-09-21T06:07:53.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.501+0000 7fa7cdffb640 10 client.17949 handle_command_reply: tid=0 2024-09-21T06:07:53.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.501+0000 7fa7f6879740 1 client.17949 shutdown 2024-09-21T06:07:53.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.501+0000 7fa7f6879740 1 -- 172.21.15.117:0/1798816862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6b3a541b0 msgr2=0x55b6b3e34120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:53.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.501+0000 7fa7f6879740 1 --2- 172.21.15.117:0/1798816862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6b3a541b0 0x55b6b3e34120 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00f8040 tx=0x7fa7e0010cb0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.501+0000 7fa7f6879740 1 -- 172.21.15.117:0/1798816862 shutdown_connections 2024-09-21T06:07:53.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.501+0000 7fa7f6879740 1 --2- 172.21.15.117:0/1798816862 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55b6b3a3b9f0 0x55b6b3e46250 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.501+0000 7fa7f6879740 1 --2- 172.21.15.117:0/1798816862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b6b3a7d070 0x55b6b3e34be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.501+0000 7fa7f6879740 1 --2- 172.21.15.117:0/1798816862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b3a5c390 0x55b6b3e34660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.501+0000 7fa7f6879740 1 --2- 172.21.15.117:0/1798816862 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6b3a541b0 0x55b6b3e34120 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.501+0000 7fa7cdffb640 0 client.17949 ms_handle_reset on v2:172.21.15.192:6838/1727623231 2024-09-21T06:07:53.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.501+0000 7fa7f6879740 1 -- 172.21.15.117:0/1798816862 >> 172.21.15.117:0/1798816862 conn(0x55b6b3a71460 msgr2=0x55b6b3e20bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:53.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.501+0000 7fa7f6879740 1 -- 172.21.15.117:0/1798816862 shutdown_connections 2024-09-21T06:07:53.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.501+0000 7fa7f6879740 1 -- 172.21.15.117:0/1798816862 wait complete. 2024-09-21T06:07:53.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.501+0000 7fa7f6879740 20 client.17949 trim_cache size 0 max 16384 2024-09-21T06:07:53.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.514+0000 7fa7f6879740 1 Processor -- start 2024-09-21T06:07:53.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.514+0000 7fa7f6879740 1 -- start start 2024-09-21T06:07:53.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.514+0000 7fa7f6879740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6b3a541b0 0x55b6b3e33630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:53.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.514+0000 7fa7f6879740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b6b3a5c390 0x55b6b3e35d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:53.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.515+0000 7fa7f6879740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b3a7d070 0x55b6b3e31310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:53.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.515+0000 7fa7f6879740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e81ad390 con 0x55b6b3a541b0 2024-09-21T06:07:53.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.515+0000 7fa7f6879740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e81aeed0 con 0x55b6b3a7d070 2024-09-21T06:07:53.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.515+0000 7fa7f6879740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e81ada60 con 0x55b6b3a5c390 2024-09-21T06:07:53.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.515+0000 7fa7ee0a1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6b3a541b0 0x55b6b3e33630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:53.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.515+0000 7fa7ee0a1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6b3a541b0 0x55b6b3e33630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36488/0 (socket says 172.21.15.117:36488) 2024-09-21T06:07:53.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.515+0000 7fa7ee0a1640 1 -- 172.21.15.117:0/531048635 learned_addr learned my addr 172.21.15.117:0/531048635 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:53.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.515+0000 7fa7ed8a0640 1 --2- 172.21.15.117:0/531048635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b6b3a5c390 0x55b6b3e35d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:53.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.515+0000 7fa7ee0a1640 1 -- 172.21.15.117:0/531048635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b6b3a5c390 msgr2=0x55b6b3e35d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:53.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.515+0000 7fa7ee0a1640 1 --2- 172.21.15.117:0/531048635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b6b3a5c390 0x55b6b3e35d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.515+0000 7fa7ee0a1640 1 -- 172.21.15.117:0/531048635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b3a7d070 msgr2=0x55b6b3e31310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:53.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.515+0000 7fa7ee0a1640 1 --2- 172.21.15.117:0/531048635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b3a7d070 0x55b6b3e31310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.515+0000 7fa7ee0a1640 1 -- 172.21.15.117:0/531048635 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6b3e2a730 con 0x55b6b3a541b0 2024-09-21T06:07:53.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.515+0000 7fa7ed8a0640 1 --2- 172.21.15.117:0/531048635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b6b3a5c390 0x55b6b3e35d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:53.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.515+0000 7fa7ee0a1640 1 --2- 172.21.15.117:0/531048635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6b3a541b0 0x55b6b3e33630 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00ac7b0 tx=0x7fa7e00ef930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e411617ed2001ce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:53.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.515+0000 7fa7cffff640 1 -- 172.21.15.117:0/531048635 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0020070 con 0x55b6b3a541b0 2024-09-21T06:07:53.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.515+0000 7fa7cffff640 1 -- 172.21.15.117:0/531048635 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7e00acc90 con 0x55b6b3a541b0 2024-09-21T06:07:53.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.515+0000 7fa7cffff640 1 -- 172.21.15.117:0/531048635 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e00b1bb0 con 0x55b6b3a541b0 2024-09-21T06:07:53.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.516+0000 7fa7f6879740 1 -- 172.21.15.117:0/531048635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6b3a541b0 msgr2=0x55b6b3e33630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:53.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.516+0000 7fa7f6879740 1 --2- 172.21.15.117:0/531048635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6b3a541b0 0x55b6b3e33630 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fa7e00ac7b0 tx=0x7fa7e00ef930 comp rx=0 tx=0).stop 2024-09-21T06:07:53.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.516+0000 7fa7f6879740 1 -- 172.21.15.117:0/531048635 shutdown_connections 2024-09-21T06:07:53.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.516+0000 7fa7f6879740 1 --2- 172.21.15.117:0/531048635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b3a7d070 0x55b6b3e31310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.516+0000 7fa7f6879740 1 --2- 172.21.15.117:0/531048635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b6b3a5c390 0x55b6b3e35d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.516+0000 7fa7f6879740 1 --2- 172.21.15.117:0/531048635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6b3a541b0 0x55b6b3e33630 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.516+0000 7fa7f6879740 1 -- 172.21.15.117:0/531048635 >> 172.21.15.117:0/531048635 conn(0x55b6b3a71460 msgr2=0x55b6b3e35640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:53.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.516+0000 7fa7f6879740 1 -- 172.21.15.117:0/531048635 shutdown_connections 2024-09-21T06:07:53.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.516+0000 7fa7f6879740 1 -- 172.21.15.117:0/531048635 wait complete. 2024-09-21T06:07:53.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.516+0000 7fa7f6879740 1 -- start start 2024-09-21T06:07:53.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.516+0000 7fa7f6879740 1 Processor -- start 2024-09-21T06:07:53.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.517+0000 7fa7f6879740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6b3a541b0 0x55b6b3e46e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:53.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.517+0000 7fa7f6879740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b6b3a5c390 0x55b6b3e473f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:53.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.517+0000 7fa7f6879740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b3a7d070 0x55b6b3ea8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:53.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.517+0000 7fa7f6879740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e81ae800 con 0x55b6b3a541b0 2024-09-21T06:07:53.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.517+0000 7fa7f6879740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e81ae130 con 0x55b6b3a7d070 2024-09-21T06:07:53.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.517+0000 7fa7f6879740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e81ab3b0 con 0x55b6b3a5c390 2024-09-21T06:07:53.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.517+0000 7fa7f6879740 10 client.0 _ll_register_callbacks cb 0x7fa7e00f8070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:53.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.517+0000 7fa7ee0a1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6b3a541b0 0x55b6b3e46e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:53.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.517+0000 7fa7ee0a1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6b3a541b0 0x55b6b3e46e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:36490/0 (socket says 172.21.15.117:36490) 2024-09-21T06:07:53.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.517+0000 7fa7ee0a1640 1 -- 172.21.15.117:0/3299860332 learned_addr learned my addr 172.21.15.117:0/3299860332 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:53.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.517+0000 7fa7ed8a0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b6b3a5c390 0x55b6b3e473f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:53.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.517+0000 7fa7ee8a2640 1 --2- 172.21.15.117:0/3299860332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b3a7d070 0x55b6b3ea8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:53.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.517+0000 7fa7ee0a1640 1 -- 172.21.15.117:0/3299860332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b6b3a5c390 msgr2=0x55b6b3e473f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:53.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.517+0000 7fa7ee0a1640 1 --2- 172.21.15.117:0/3299860332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b6b3a5c390 0x55b6b3e473f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.517+0000 7fa7ee0a1640 1 -- 172.21.15.117:0/3299860332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b3a7d070 msgr2=0x55b6b3ea8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:53.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.517+0000 7fa7ee0a1640 1 --2- 172.21.15.117:0/3299860332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b3a7d070 0x55b6b3ea8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.517+0000 7fa7ee0a1640 1 -- 172.21.15.117:0/3299860332 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6b3ea91f0 con 0x55b6b3a541b0 2024-09-21T06:07:53.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.517+0000 7fa7ee0a1640 1 --2- 172.21.15.117:0/3299860332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6b3a541b0 0x55b6b3e46e10 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0005fd0 tx=0x7fa7e0005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d69d5f06913b1e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:53.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.517+0000 7fa7cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:53.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.517+0000 7fa7cdffb640 1 -- 172.21.15.117:0/3299860332 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0020070 con 0x55b6b3a541b0 2024-09-21T06:07:53.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.518+0000 7fa7cdffb640 1 -- 172.21.15.117:0/3299860332 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7fa7e00b6660 con 0x55b6b3a541b0 2024-09-21T06:07:53.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.518+0000 7fa7f6879740 1 -- 172.21.15.117:0/3299860332 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7e00acc90 con 0x55b6b3a541b0 2024-09-21T06:07:53.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.518+0000 7fa7cdffb640 1 -- 172.21.15.117:0/3299860332 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa7e00b68e0 con 0x55b6b3a541b0 2024-09-21T06:07:53.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.518+0000 7fa7f6879740 10 client.17955 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:53.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.518+0000 7fa7f6879740 1 -- 172.21.15.117:0/3299860332 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b6b3ea91f0 con 0x55b6b3a541b0 2024-09-21T06:07:53.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.518+0000 7fa7cdffb640 1 -- 172.21.15.117:0/3299860332 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7e00acc90 con 0x55b6b3a541b0 2024-09-21T06:07:53.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.518+0000 7fa7cdffb640 1 -- 172.21.15.117:0/3299860332 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa7e0058b90 con 0x55b6b3a541b0 2024-09-21T06:07:53.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.518+0000 7fa7f6879740 10 client.17955 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:53.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.518+0000 7fa7f6879740 10 client.17955 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:07:53.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.518+0000 7fa7f6879740 20 client.17955 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:53.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.518+0000 7fa7f6879740 1 --2- 172.21.15.117:0/3299860332 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55b6b3a10da0 0x55b6b3eb2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:53.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.518+0000 7fa7f6879740 4 client.17955 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:07:53.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.518+0000 7fa7f6879740 1 -- 172.21.15.117:0/3299860332 --> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b6b3d988a0 con 0x55b6b3a10da0 2024-09-21T06:07:53.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.518+0000 7fa7ed8a0640 1 --2- 172.21.15.117:0/3299860332 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55b6b3a10da0 0x55b6b3eb2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:53.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.519+0000 7fa7cdffb640 10 client.17955 ms_handle_connect on v2:172.21.15.192:6838/1727623231 2024-09-21T06:07:53.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.519+0000 7fa7ed8a0640 1 --2- 172.21.15.117:0/3299860332 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55b6b3a10da0 0x55b6b3eb2920 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fa6dab4f0260c4d9 server_cookie=2ed2ea7976f4a6f3 in_seq=0 out_seq=0 2024-09-21T06:07:53.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.519+0000 7fa7cdffb640 1 -- 172.21.15.117:0/3299860332 <== mds.1 v2:172.21.15.192:6838/1727623231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b6b3d988a0 con 0x55b6b3a10da0 2024-09-21T06:07:53.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.519+0000 7fa7cdffb640 10 client.17955 handle_command_reply: tid=0 2024-09-21T06:07:53.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.519+0000 7fa7f6879740 1 client.17955 shutdown 2024-09-21T06:07:53.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.519+0000 7fa7f6879740 1 -- 172.21.15.117:0/3299860332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6b3a541b0 msgr2=0x55b6b3e46e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:53.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.519+0000 7fa7f6879740 1 --2- 172.21.15.117:0/3299860332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6b3a541b0 0x55b6b3e46e10 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0005fd0 tx=0x7fa7e0005bd0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.519+0000 7fa7f6879740 1 -- 172.21.15.117:0/3299860332 shutdown_connections 2024-09-21T06:07:53.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.519+0000 7fa7f6879740 1 --2- 172.21.15.117:0/3299860332 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x55b6b3a10da0 0x55b6b3eb2920 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.519+0000 7fa7f6879740 1 --2- 172.21.15.117:0/3299860332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b6b3a7d070 0x55b6b3ea8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.519+0000 7fa7f6879740 1 --2- 172.21.15.117:0/3299860332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b6b3a5c390 0x55b6b3e473f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.519+0000 7fa7f6879740 1 --2- 172.21.15.117:0/3299860332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6b3a541b0 0x55b6b3e46e10 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.519+0000 7fa7f6879740 1 -- 172.21.15.117:0/3299860332 >> 172.21.15.117:0/3299860332 conn(0x55b6b3a71460 msgr2=0x55b6b3e29fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:53.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.519+0000 7fa7cdffb640 0 client.17955 ms_handle_reset on v2:172.21.15.192:6838/1727623231 2024-09-21T06:07:53.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.519+0000 7fa7f6879740 1 -- 172.21.15.117:0/3299860332 shutdown_connections 2024-09-21T06:07:53.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.519+0000 7fa7f6879740 1 -- 172.21.15.117:0/3299860332 wait complete. 2024-09-21T06:07:53.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.519+0000 7fa7f6879740 20 client.17955 trim_cache size 0 max 16384 2024-09-21T06:07:53.527 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:07:53.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.520+0000 7fa7f0305640 1 -- 172.21.15.117:0/3535638133 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa7d00b6700 msgr2=0x7fa7d00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:53.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.520+0000 7fa7f0305640 1 --2- 172.21.15.117:0/3535638133 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa7d00b6700 0x7fa7d00b8c10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa7d802f500 tx=0x7fa7d802f370 comp rx=0 tx=0).stop 2024-09-21T06:07:53.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.520+0000 7fa7f0305640 1 -- 172.21.15.117:0/3535638133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7e8104880 msgr2=0x7fa7e819a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:53.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.520+0000 7fa7f0305640 1 --2- 172.21.15.117:0/3535638133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7e8104880 0x7fa7e819a430 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fa7e000d950 tx=0x7fa7e000de10 comp rx=0 tx=0).stop 2024-09-21T06:07:53.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.520+0000 7fa7f0305640 1 -- 172.21.15.117:0/3535638133 shutdown_connections 2024-09-21T06:07:53.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.520+0000 7fa7f0305640 1 --2- 172.21.15.117:0/3535638133 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa7d00b6700 0x7fa7d00b8c10 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.520+0000 7fa7f0305640 1 --2- 172.21.15.117:0/3535638133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7e810de30 0x7fa7e819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.520+0000 7fa7f0305640 1 --2- 172.21.15.117:0/3535638133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7e81052b0 0x7fa7e819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.520+0000 7fa7f0305640 1 --2- 172.21.15.117:0/3535638133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7e8104880 0x7fa7e819a430 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:53.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.520+0000 7fa7f0305640 1 -- 172.21.15.117:0/3535638133 >> 172.21.15.117:0/3535638133 conn(0x7fa7e8079740 msgr2=0x7fa7e8105860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:53.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.520+0000 7fa7f0305640 1 -- 172.21.15.117:0/3535638133 shutdown_connections 2024-09-21T06:07:53.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:53.520+0000 7fa7f0305640 1 -- 172.21.15.117:0/3535638133 wait complete. 2024-09-21T06:07:54.209 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:53 smithi117 ceph-mon[30947]: pgmap v651: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T06:07:54.211 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:07:54.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:53 smithi192 ceph-mon[34700]: pgmap v651: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T06:07:54.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:53 smithi138 ceph-mon[34391]: pgmap v651: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T06:07:54.458 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:07:54.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.968+0000 7fb459088640 1 -- 172.21.15.117:0/1836333537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb454106c90 msgr2=0x7fb4541106e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:54.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.968+0000 7fb459088640 1 --2- 172.21.15.117:0/1836333537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb454106c90 0x7fb4541106e0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fb44c009ca0 tx=0x7fb44c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:07:54.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.968+0000 7fb459088640 1 -- 172.21.15.117:0/1836333537 shutdown_connections 2024-09-21T06:07:54.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.968+0000 7fb459088640 1 --2- 172.21.15.117:0/1836333537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb454106c90 0x7fb4541106e0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:54.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.968+0000 7fb459088640 1 --2- 172.21.15.117:0/1836333537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb454106340 0x7fb454106750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:54.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.968+0000 7fb459088640 1 --2- 172.21.15.117:0/1836333537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb454102060 0x7fb454102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:54.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.968+0000 7fb459088640 1 -- 172.21.15.117:0/1836333537 >> 172.21.15.117:0/1836333537 conn(0x7fb454079ea0 msgr2=0x7fb45407a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:54.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.968+0000 7fb459088640 1 -- 172.21.15.117:0/1836333537 shutdown_connections 2024-09-21T06:07:54.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.968+0000 7fb459088640 1 -- 172.21.15.117:0/1836333537 wait complete. 2024-09-21T06:07:54.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.969+0000 7fb459088640 1 Processor -- start 2024-09-21T06:07:54.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.969+0000 7fb459088640 1 -- start start 2024-09-21T06:07:54.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.970+0000 7fb459088640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb454102060 0x7fb45419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:54.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.970+0000 7fb459088640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb454106340 0x7fb45419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:54.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.970+0000 7fb459088640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb454106c90 0x7fb45419fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:54.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.970+0000 7fb459088640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb454113b90 con 0x7fb454106c90 2024-09-21T06:07:54.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.970+0000 7fb459088640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb454113a10 con 0x7fb454102060 2024-09-21T06:07:54.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.970+0000 7fb459088640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb454113d10 con 0x7fb454106340 2024-09-21T06:07:54.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.970+0000 7fb45359e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb454106c90 0x7fb45419fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:54.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.970+0000 7fb45359e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb454106c90 0x7fb45419fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48048/0 (socket says 172.21.15.117:48048) 2024-09-21T06:07:54.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.970+0000 7fb45359e640 1 -- 172.21.15.117:0/1225722305 learned_addr learned my addr 172.21.15.117:0/1225722305 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:54.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.970+0000 7fb45259c640 1 --2- 172.21.15.117:0/1225722305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb454106340 0x7fb45419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:54.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.970+0000 7fb452d9d640 1 --2- 172.21.15.117:0/1225722305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb454102060 0x7fb45419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:54.982 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.970+0000 7fb45359e640 1 -- 172.21.15.117:0/1225722305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb454106340 msgr2=0x7fb45419ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:54.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.970+0000 7fb45359e640 1 --2- 172.21.15.117:0/1225722305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb454106340 0x7fb45419ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:54.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.970+0000 7fb45359e640 1 -- 172.21.15.117:0/1225722305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb454102060 msgr2=0x7fb45419a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:54.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.970+0000 7fb45359e640 1 --2- 172.21.15.117:0/1225722305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb454102060 0x7fb45419a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:54.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.970+0000 7fb45359e640 1 -- 172.21.15.117:0/1225722305 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4541a04f0 con 0x7fb454106c90 2024-09-21T06:07:54.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.970+0000 7fb45359e640 1 --2- 172.21.15.117:0/1225722305 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb454106c90 0x7fb45419fdc0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fb44c005cc0 tx=0x7fb44c02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:54.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.971+0000 7fb43bfff640 1 -- 172.21.15.117:0/1225722305 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c003bf0 con 0x7fb454106c90 2024-09-21T06:07:54.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.971+0000 7fb459088640 1 -- 172.21.15.117:0/1225722305 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4541a0780 con 0x7fb454106c90 2024-09-21T06:07:54.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.971+0000 7fb43bfff640 1 -- 172.21.15.117:0/1225722305 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb44c003d90 con 0x7fb454106c90 2024-09-21T06:07:54.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.971+0000 7fb43bfff640 1 -- 172.21.15.117:0/1225722305 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c02fb20 con 0x7fb454106c90 2024-09-21T06:07:54.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.971+0000 7fb459088640 1 -- 172.21.15.117:0/1225722305 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4541a6f40 con 0x7fb454106c90 2024-09-21T06:07:54.983 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.973+0000 7fb45f5fc740 1 Processor -- start 2024-09-21T06:07:54.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.973+0000 7fb43bfff640 1 -- 172.21.15.117:0/1225722305 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fb44c004580 con 0x7fb454106c90 2024-09-21T06:07:54.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.974+0000 7fb45f5fc740 1 -- start start 2024-09-21T06:07:54.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.974+0000 7fb45f5fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55965b8dc070 0x55965bc836a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:54.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.974+0000 7fb45f5fc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55965b8b31b0 0x55965bc83be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:54.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.974+0000 7fb45f5fc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965b8bb390 0x55965bc84120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:54.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.974+0000 7fb45f5fc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55965bbf9ea0 con 0x55965b8b31b0 2024-09-21T06:07:54.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.974+0000 7fb45f5fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55965bbe92d0 con 0x55965b8dc070 2024-09-21T06:07:54.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.974+0000 7fb45f5fc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55965bbed0e0 con 0x55965b8bb390 2024-09-21T06:07:54.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.974+0000 7fb452d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55965b8b31b0 0x55965bc83be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:54.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.974+0000 7fb452d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55965b8b31b0 0x55965bc83be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48062/0 (socket says 172.21.15.117:48062) 2024-09-21T06:07:54.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.974+0000 7fb452d9d640 1 -- 172.21.15.117:0/2207463539 learned_addr learned my addr 172.21.15.117:0/2207463539 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:54.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.974+0000 7fb45259c640 1 --2- 172.21.15.117:0/2207463539 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965b8bb390 0x55965bc84120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:54.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.974+0000 7fb45259c640 1 --2- 172.21.15.117:0/2207463539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55965b8dc070 0x55965bc836a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:54.984 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.974+0000 7fb43bfff640 1 --2- 172.21.15.117:0/1225722305 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb43c0b6680 0x7fb43c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:54.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.974+0000 7fb452d9d640 1 -- 172.21.15.117:0/2207463539 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965b8bb390 msgr2=0x55965bc84120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:54.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.974+0000 7fb452d9d640 1 --2- 172.21.15.117:0/2207463539 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965b8bb390 0x55965bc84120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:54.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.974+0000 7fb452d9d640 1 -- 172.21.15.117:0/2207463539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55965b8dc070 msgr2=0x55965bc836a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:54.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.974+0000 7fb452d9d640 1 --2- 172.21.15.117:0/2207463539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55965b8dc070 0x55965bc836a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:54.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.974+0000 7fb452d9d640 1 -- 172.21.15.117:0/2207463539 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55965b947650 con 0x55965b8b31b0 2024-09-21T06:07:54.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.974+0000 7fb45259c640 1 --2- 172.21.15.117:0/2207463539 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965b8bb390 0x55965bc84120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:54.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.974+0000 7fb45359e640 1 --2- 172.21.15.117:0/1225722305 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb43c0b6680 0x7fb43c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:54.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.975+0000 7fb452d9d640 1 --2- 172.21.15.117:0/2207463539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55965b8b31b0 0x55965bc83be0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fb44400bd10 tx=0x7fb4440107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab427dc68797abbd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:54.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.975+0000 7fb43bfff640 1 -- 172.21.15.117:0/1225722305 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7fb44c1089c0 con 0x7fb454106c90 2024-09-21T06:07:54.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.975+0000 7fb4397fa640 1 -- 172.21.15.117:0/2207463539 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb444004030 con 0x55965b8b31b0 2024-09-21T06:07:54.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.975+0000 7fb4397fa640 1 -- 172.21.15.117:0/2207463539 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4440035f0 con 0x55965b8b31b0 2024-09-21T06:07:54.985 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.975+0000 7fb4397fa640 1 -- 172.21.15.117:0/2207463539 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4440117b0 con 0x55965b8b31b0 2024-09-21T06:07:54.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.976+0000 7fb45f5fc740 1 -- 172.21.15.117:0/2207463539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55965b8b31b0 msgr2=0x55965bc83be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:54.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.976+0000 7fb45f5fc740 1 --2- 172.21.15.117:0/2207463539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55965b8b31b0 0x55965bc83be0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fb44400bd10 tx=0x7fb4440107a0 comp rx=0 tx=0).stop 2024-09-21T06:07:54.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.976+0000 7fb45359e640 1 --2- 172.21.15.117:0/1225722305 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb43c0b6680 0x7fb43c0b8b90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb44c0ce560 tx=0x7fb44c09bc50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:54.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.976+0000 7fb45f5fc740 1 -- 172.21.15.117:0/2207463539 shutdown_connections 2024-09-21T06:07:54.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.976+0000 7fb45f5fc740 1 --2- 172.21.15.117:0/2207463539 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55965b8b31b0 0x55965bc83be0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:54.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.976+0000 7fb45f5fc740 1 --2- 172.21.15.117:0/2207463539 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965b8bb390 0x55965bc84120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:54.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.976+0000 7fb45f5fc740 1 --2- 172.21.15.117:0/2207463539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55965b8dc070 0x55965bc836a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:54.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.976+0000 7fb45f5fc740 1 -- 172.21.15.117:0/2207463539 >> 172.21.15.117:0/2207463539 conn(0x55965b8d0460 msgr2=0x55965bc81590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:54.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.976+0000 7fb45f5fc740 1 -- 172.21.15.117:0/2207463539 shutdown_connections 2024-09-21T06:07:54.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.976+0000 7fb45f5fc740 1 -- 172.21.15.117:0/2207463539 wait complete. 2024-09-21T06:07:54.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.976+0000 7fb45f5fc740 1 -- start start 2024-09-21T06:07:54.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.976+0000 7fb45f5fc740 1 Processor -- start 2024-09-21T06:07:54.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.977+0000 7fb45f5fc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965b8b31b0 0x55965bc8ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:54.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.977+0000 7fb45f5fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55965b8bb390 0x55965bc8f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:54.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.977+0000 7fb45f5fc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55965b8dc070 0x55965bc8f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:54.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.977+0000 7fb45f5fc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55965ba4a700 con 0x55965b8dc070 2024-09-21T06:07:54.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.977+0000 7fb45f5fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55965bb84160 con 0x55965b8bb390 2024-09-21T06:07:54.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.977+0000 7fb45f5fc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55965bc91e80 con 0x55965b8b31b0 2024-09-21T06:07:54.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.977+0000 7fb45f5fc740 10 client.0 _ll_register_callbacks cb 0x55965bb9d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:54.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.977+0000 7fb45259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55965b8dc070 0x55965bc8f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:54.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.977+0000 7fb45259c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55965b8dc070 0x55965bc8f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48072/0 (socket says 172.21.15.117:48072) 2024-09-21T06:07:54.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.977+0000 7fb45259c640 1 -- 172.21.15.117:0/2761241913 learned_addr learned my addr 172.21.15.117:0/2761241913 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:54.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.977+0000 7fb452d9d640 1 --2- 172.21.15.117:0/2761241913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55965b8bb390 0x55965bc8f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:54.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.977+0000 7fb45259c640 1 --2- 172.21.15.117:0/2761241913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965b8b31b0 0x55965bc8ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:54.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.977+0000 7fb45259c640 1 -- 172.21.15.117:0/2761241913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965b8b31b0 msgr2=0x55965bc8ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:54.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.977+0000 7fb45259c640 1 --2- 172.21.15.117:0/2761241913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965b8b31b0 0x55965bc8ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:54.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.977+0000 7fb45259c640 1 -- 172.21.15.117:0/2761241913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55965b8bb390 msgr2=0x55965bc8f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:54.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.977+0000 7fb45259c640 1 --2- 172.21.15.117:0/2761241913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55965b8bb390 0x55965bc8f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:54.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.977+0000 7fb45259c640 1 -- 172.21.15.117:0/2761241913 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55965bbb1c90 con 0x55965b8dc070 2024-09-21T06:07:54.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.977+0000 7fb45259c640 1 --2- 172.21.15.117:0/2761241913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55965b8dc070 0x55965bc8f800 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fb448009e40 tx=0x7fb4480047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f2027bf93d159 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:54.992 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.977+0000 7fb436ff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:54.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.977+0000 7fb436ff5640 1 -- 172.21.15.117:0/2761241913 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb448019110 con 0x55965b8dc070 2024-09-21T06:07:54.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.977+0000 7fb436ff5640 1 -- 172.21.15.117:0/2761241913 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7fb448014ae0 con 0x55965b8dc070 2024-09-21T06:07:54.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.978+0000 7fb45f5fc740 1 -- 172.21.15.117:0/2761241913 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4440035f0 con 0x55965b8dc070 2024-09-21T06:07:54.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.978+0000 7fb436ff5640 1 -- 172.21.15.117:0/2761241913 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb448028050 con 0x55965b8dc070 2024-09-21T06:07:54.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.978+0000 7fb45f5fc740 10 client.17967 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:54.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.978+0000 7fb45f5fc740 1 -- 172.21.15.117:0/2761241913 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55965bc9dbe0 con 0x55965b8dc070 2024-09-21T06:07:54.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.978+0000 7fb436ff5640 1 -- 172.21.15.117:0/2761241913 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4440035f0 con 0x55965b8dc070 2024-09-21T06:07:54.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.978+0000 7fb436ff5640 1 -- 172.21.15.117:0/2761241913 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb448017b90 con 0x55965b8dc070 2024-09-21T06:07:54.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.978+0000 7fb45f5fc740 10 client.17967 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:54.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.978+0000 7fb45f5fc740 10 client.17967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:07:54.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.978+0000 7fb45f5fc740 20 client.17967 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:54.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.978+0000 7fb45f5fc740 1 --2- 172.21.15.117:0/2761241913 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55965b89a9f0 0x55965bca1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:54.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.978+0000 7fb45f5fc740 4 client.17967 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:07:54.993 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.978+0000 7fb45f5fc740 1 -- 172.21.15.117:0/2761241913 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55965bbfbdf0 con 0x55965b89a9f0 2024-09-21T06:07:54.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.978+0000 7fb452d9d640 1 --2- 172.21.15.117:0/2761241913 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55965b89a9f0 0x55965bca1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:54.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.979+0000 7fb436ff5640 10 client.17967 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:54.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.979+0000 7fb452d9d640 1 --2- 172.21.15.117:0/2761241913 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55965b89a9f0 0x55965bca1140 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83cc8c3f44e719f2 server_cookie=bd412b408dfb931c in_seq=0 out_seq=0 2024-09-21T06:07:54.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.980+0000 7fb436ff5640 1 -- 172.21.15.117:0/2761241913 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55965bbfbdf0 con 0x55965b89a9f0 2024-09-21T06:07:54.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.980+0000 7fb436ff5640 10 client.17967 handle_command_reply: tid=0 2024-09-21T06:07:54.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.980+0000 7fb45f5fc740 1 client.17967 shutdown 2024-09-21T06:07:54.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.980+0000 7fb45f5fc740 1 -- 172.21.15.117:0/2761241913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55965b8dc070 msgr2=0x55965bc8f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:54.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.980+0000 7fb45f5fc740 1 --2- 172.21.15.117:0/2761241913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55965b8dc070 0x55965bc8f800 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fb448009e40 tx=0x7fb4480047d0 comp rx=0 tx=0).stop 2024-09-21T06:07:54.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.980+0000 7fb45f5fc740 1 -- 172.21.15.117:0/2761241913 shutdown_connections 2024-09-21T06:07:54.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.980+0000 7fb45f5fc740 1 --2- 172.21.15.117:0/2761241913 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55965b89a9f0 0x55965bca1140 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:54.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.980+0000 7fb45f5fc740 1 --2- 172.21.15.117:0/2761241913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55965b8dc070 0x55965bc8f800 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:54.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.980+0000 7fb45f5fc740 1 --2- 172.21.15.117:0/2761241913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55965b8bb390 0x55965bc8f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:54.994 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.980+0000 7fb436ff5640 0 client.17967 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:54.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.980+0000 7fb45f5fc740 1 --2- 172.21.15.117:0/2761241913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965b8b31b0 0x55965bc8ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:54.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.980+0000 7fb45f5fc740 1 -- 172.21.15.117:0/2761241913 >> 172.21.15.117:0/2761241913 conn(0x55965b8d0460 msgr2=0x55965bc86830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:54.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.980+0000 7fb45f5fc740 1 -- 172.21.15.117:0/2761241913 shutdown_connections 2024-09-21T06:07:54.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.980+0000 7fb45f5fc740 1 -- 172.21.15.117:0/2761241913 wait complete. 2024-09-21T06:07:54.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.980+0000 7fb45f5fc740 20 client.17967 trim_cache size 0 max 16384 2024-09-21T06:07:54.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.993+0000 7fb45f5fc740 1 Processor -- start 2024-09-21T06:07:54.996 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.993+0000 7fb45f5fc740 1 -- start start 2024-09-21T06:07:55.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.993+0000 7fb45f5fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55965b8b31b0 0x55965bc85090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:55.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.993+0000 7fb45f5fc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55965b8bb390 0x55965bc90970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:55.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.993+0000 7fb45f5fc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965b8dc070 0x55965bc8bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:55.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.993+0000 7fb45f5fc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb4541af640 con 0x55965b8bb390 2024-09-21T06:07:55.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.993+0000 7fb45f5fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb4541b1180 con 0x55965b8b31b0 2024-09-21T06:07:55.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.993+0000 7fb45f5fc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb4541afd10 con 0x55965b8dc070 2024-09-21T06:07:55.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.993+0000 7fb452d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55965b8bb390 0x55965bc90970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:55.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.993+0000 7fb452d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55965b8bb390 0x55965bc90970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48074/0 (socket says 172.21.15.117:48074) 2024-09-21T06:07:55.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.993+0000 7fb452d9d640 1 -- 172.21.15.117:0/2284877379 learned_addr learned my addr 172.21.15.117:0/2284877379 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:55.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.994+0000 7fb45259c640 1 --2- 172.21.15.117:0/2284877379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55965b8b31b0 0x55965bc85090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:55.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.994+0000 7fb45259c640 1 --2- 172.21.15.117:0/2284877379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965b8dc070 0x55965bc8bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:55.003 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.994+0000 7fb452d9d640 1 -- 172.21.15.117:0/2284877379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965b8dc070 msgr2=0x55965bc8bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:55.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.994+0000 7fb452d9d640 1 --2- 172.21.15.117:0/2284877379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965b8dc070 0x55965bc8bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:55.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.994+0000 7fb452d9d640 1 -- 172.21.15.117:0/2284877379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55965b8b31b0 msgr2=0x55965bc85090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:55.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.994+0000 7fb452d9d640 1 --2- 172.21.15.117:0/2284877379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55965b8b31b0 0x55965bc85090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:55.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.994+0000 7fb452d9d640 1 -- 172.21.15.117:0/2284877379 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55965bc85990 con 0x55965b8bb390 2024-09-21T06:07:55.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.994+0000 7fb452d9d640 1 --2- 172.21.15.117:0/2284877379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55965b8bb390 0x55965bc90970 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x55965bc82320 tx=0x7fb444003f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3a902792b220b93 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:55.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.994+0000 7fb438ff9640 1 -- 172.21.15.117:0/2284877379 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb444032070 con 0x55965b8bb390 2024-09-21T06:07:55.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.994+0000 7fb438ff9640 1 -- 172.21.15.117:0/2284877379 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4440035f0 con 0x55965b8bb390 2024-09-21T06:07:55.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.994+0000 7fb438ff9640 1 -- 172.21.15.117:0/2284877379 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4440116a0 con 0x55965b8bb390 2024-09-21T06:07:55.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.995+0000 7fb45f5fc740 1 -- 172.21.15.117:0/2284877379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55965b8bb390 msgr2=0x55965bc90970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:55.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.995+0000 7fb45f5fc740 1 --2- 172.21.15.117:0/2284877379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55965b8bb390 0x55965bc90970 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x55965bc82320 tx=0x7fb444003f00 comp rx=0 tx=0).stop 2024-09-21T06:07:55.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.995+0000 7fb45f5fc740 1 -- 172.21.15.117:0/2284877379 shutdown_connections 2024-09-21T06:07:55.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.995+0000 7fb45f5fc740 1 --2- 172.21.15.117:0/2284877379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55965b8bb390 0x55965bc90970 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:55.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.995+0000 7fb45f5fc740 1 --2- 172.21.15.117:0/2284877379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965b8dc070 0x55965bc8bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:55.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.995+0000 7fb45f5fc740 1 --2- 172.21.15.117:0/2284877379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55965b8b31b0 0x55965bc85090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:55.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.995+0000 7fb45f5fc740 1 -- 172.21.15.117:0/2284877379 >> 172.21.15.117:0/2284877379 conn(0x55965b8d0460 msgr2=0x55965bc90260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:55.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.995+0000 7fb45f5fc740 1 -- 172.21.15.117:0/2284877379 shutdown_connections 2024-09-21T06:07:55.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.995+0000 7fb45f5fc740 1 -- 172.21.15.117:0/2284877379 wait complete. 2024-09-21T06:07:55.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.995+0000 7fb45f5fc740 1 -- start start 2024-09-21T06:07:55.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.995+0000 7fb45f5fc740 1 Processor -- start 2024-09-21T06:07:55.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.996+0000 7fb45f5fc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965b8b31b0 0x55965bc93a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:55.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.996+0000 7fb45f5fc740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55965b8bb390 0x55965bc94010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:55.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.996+0000 7fb45f5fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55965b8dc070 0x55965bd04b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:55.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.996+0000 7fb45f5fc740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb4541b0ab0 con 0x55965b8bb390 2024-09-21T06:07:55.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.996+0000 7fb45f5fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb4541b03e0 con 0x55965b8dc070 2024-09-21T06:07:55.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.996+0000 7fb45f5fc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb4541ad640 con 0x55965b8b31b0 2024-09-21T06:07:55.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.996+0000 7fb45f5fc740 10 client.0 _ll_register_callbacks cb 0x7fb448009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:55.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.996+0000 7fb452d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55965b8bb390 0x55965bc94010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:55.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.996+0000 7fb452d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55965b8bb390 0x55965bc94010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48078/0 (socket says 172.21.15.117:48078) 2024-09-21T06:07:55.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.996+0000 7fb452d9d640 1 -- 172.21.15.117:0/3086119200 learned_addr learned my addr 172.21.15.117:0/3086119200 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:55.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.996+0000 7fb45259c640 1 --2- 172.21.15.117:0/3086119200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965b8b31b0 0x55965bc93a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:55.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.996+0000 7fb45259c640 1 --2- 172.21.15.117:0/3086119200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55965b8dc070 0x55965bd04b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:55.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.996+0000 7fb452d9d640 1 -- 172.21.15.117:0/3086119200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965b8b31b0 msgr2=0x55965bc93a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:55.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.996+0000 7fb452d9d640 1 --2- 172.21.15.117:0/3086119200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965b8b31b0 0x55965bc93a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:55.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.996+0000 7fb452d9d640 1 -- 172.21.15.117:0/3086119200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55965b8dc070 msgr2=0x55965bd04b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:55.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.996+0000 7fb452d9d640 1 --2- 172.21.15.117:0/3086119200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55965b8dc070 0x55965bd04b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:55.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.996+0000 7fb452d9d640 1 -- 172.21.15.117:0/3086119200 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55965bd051f0 con 0x55965b8bb390 2024-09-21T06:07:55.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.996+0000 7fb45259c640 1 --2- 172.21.15.117:0/3086119200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55965b8dc070 0x55965bd04b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:55.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.996+0000 7fb452d9d640 1 --2- 172.21.15.117:0/3086119200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55965b8bb390 0x55965bc94010 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fb44400b3d0 tx=0x7fb44400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd758ca80af803cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:55.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.996+0000 7fb436ff5640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:55.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.996+0000 7fb436ff5640 1 -- 172.21.15.117:0/3086119200 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44403b060 con 0x55965b8bb390 2024-09-21T06:07:55.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.997+0000 7fb436ff5640 1 -- 172.21.15.117:0/3086119200 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7fb444003aa0 con 0x55965b8bb390 2024-09-21T06:07:55.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.997+0000 7fb45f5fc740 1 -- 172.21.15.117:0/3086119200 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4440035f0 con 0x55965b8bb390 2024-09-21T06:07:55.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.997+0000 7fb436ff5640 1 -- 172.21.15.117:0/3086119200 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb444032040 con 0x55965b8bb390 2024-09-21T06:07:55.006 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.997+0000 7fb45f5fc740 10 client.17973 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:55.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.997+0000 7fb45f5fc740 1 -- 172.21.15.117:0/3086119200 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55965bd0c510 con 0x55965b8bb390 2024-09-21T06:07:55.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.997+0000 7fb436ff5640 1 -- 172.21.15.117:0/3086119200 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4440035f0 con 0x55965b8bb390 2024-09-21T06:07:55.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.997+0000 7fb436ff5640 1 -- 172.21.15.117:0/3086119200 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb44402c030 con 0x55965b8bb390 2024-09-21T06:07:55.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.997+0000 7fb45f5fc740 10 client.17973 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:55.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.997+0000 7fb45f5fc740 10 client.17973 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:07:55.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.997+0000 7fb45f5fc740 20 client.17973 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:55.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.997+0000 7fb45f5fc740 1 --2- 172.21.15.117:0/3086119200 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55965b86fda0 0x55965bd0ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:55.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.997+0000 7fb45f5fc740 4 client.17973 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:07:55.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.997+0000 7fb45f5fc740 1 -- 172.21.15.117:0/3086119200 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55965bbf78a0 con 0x55965b86fda0 2024-09-21T06:07:55.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.997+0000 7fb45259c640 1 --2- 172.21.15.117:0/3086119200 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55965b86fda0 0x55965bd0ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:55.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.998+0000 7fb436ff5640 10 client.17973 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:55.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.998+0000 7fb45259c640 1 --2- 172.21.15.117:0/3086119200 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55965b86fda0 0x55965bd0ead0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c51bd2bbb6e4375f server_cookie=12a37cad8c0f276c in_seq=0 out_seq=0 2024-09-21T06:07:55.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.998+0000 7fb436ff5640 1 -- 172.21.15.117:0/3086119200 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55965bbf78a0 con 0x55965b86fda0 2024-09-21T06:07:55.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.998+0000 7fb436ff5640 10 client.17973 handle_command_reply: tid=0 2024-09-21T06:07:55.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.998+0000 7fb45f5fc740 1 client.17973 shutdown 2024-09-21T06:07:55.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.998+0000 7fb45f5fc740 1 -- 172.21.15.117:0/3086119200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55965b8bb390 msgr2=0x55965bc94010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:55.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.998+0000 7fb45f5fc740 1 --2- 172.21.15.117:0/3086119200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55965b8bb390 0x55965bc94010 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fb44400b3d0 tx=0x7fb44400bf10 comp rx=0 tx=0).stop 2024-09-21T06:07:55.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.998+0000 7fb45f5fc740 1 -- 172.21.15.117:0/3086119200 shutdown_connections 2024-09-21T06:07:55.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.998+0000 7fb45f5fc740 1 --2- 172.21.15.117:0/3086119200 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55965b86fda0 0x55965bd0ead0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:55.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.998+0000 7fb45f5fc740 1 --2- 172.21.15.117:0/3086119200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55965b8bb390 0x55965bc94010 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:55.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.998+0000 7fb45f5fc740 1 --2- 172.21.15.117:0/3086119200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55965b8dc070 0x55965bd04b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:55.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.998+0000 7fb45f5fc740 1 --2- 172.21.15.117:0/3086119200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55965b8b31b0 0x55965bc93a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:55.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.998+0000 7fb45f5fc740 1 -- 172.21.15.117:0/3086119200 >> 172.21.15.117:0/3086119200 conn(0x55965b8d0460 msgr2=0x55965bc90bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:55.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.998+0000 7fb436ff5640 0 client.17973 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:55.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.998+0000 7fb45f5fc740 1 -- 172.21.15.117:0/3086119200 shutdown_connections 2024-09-21T06:07:55.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.998+0000 7fb45f5fc740 1 -- 172.21.15.117:0/3086119200 wait complete. 2024-09-21T06:07:55.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.998+0000 7fb45f5fc740 20 client.17973 trim_cache size 0 max 16384 2024-09-21T06:07:55.010 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:07:55.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.999+0000 7fb459088640 1 -- 172.21.15.117:0/1225722305 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb43c0b6680 msgr2=0x7fb43c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:55.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.999+0000 7fb459088640 1 --2- 172.21.15.117:0/1225722305 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb43c0b6680 0x7fb43c0b8b90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb44c0ce560 tx=0x7fb44c09bc50 comp rx=0 tx=0).stop 2024-09-21T06:07:55.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.999+0000 7fb459088640 1 -- 172.21.15.117:0/1225722305 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb454106c90 msgr2=0x7fb45419fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:55.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.999+0000 7fb459088640 1 --2- 172.21.15.117:0/1225722305 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb454106c90 0x7fb45419fdc0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fb44c005cc0 tx=0x7fb44c02f380 comp rx=0 tx=0).stop 2024-09-21T06:07:55.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.999+0000 7fb459088640 1 -- 172.21.15.117:0/1225722305 shutdown_connections 2024-09-21T06:07:55.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.999+0000 7fb459088640 1 --2- 172.21.15.117:0/1225722305 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb43c0b6680 0x7fb43c0b8b90 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:55.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.999+0000 7fb459088640 1 --2- 172.21.15.117:0/1225722305 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb454106c90 0x7fb45419fdc0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:55.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.999+0000 7fb459088640 1 --2- 172.21.15.117:0/1225722305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb454106340 0x7fb45419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:55.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.999+0000 7fb459088640 1 --2- 172.21.15.117:0/1225722305 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb454102060 0x7fb45419a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:55.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:54.999+0000 7fb459088640 1 -- 172.21.15.117:0/1225722305 >> 172.21.15.117:0/1225722305 conn(0x7fb454079ea0 msgr2=0x7fb4541075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:55.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:55.000+0000 7fb459088640 1 -- 172.21.15.117:0/1225722305 shutdown_connections 2024-09-21T06:07:55.011 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:55.000+0000 7fb459088640 1 -- 172.21.15.117:0/1225722305 wait complete. 2024-09-21T06:07:56.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:55 smithi117 ceph-mon[30947]: pgmap v652: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 341 B/s wr, 14 op/s 2024-09-21T06:07:56.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:55 smithi192 ceph-mon[34700]: pgmap v652: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 341 B/s wr, 14 op/s 2024-09-21T06:07:56.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:55 smithi138 ceph-mon[34391]: pgmap v652: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 341 B/s wr, 14 op/s 2024-09-21T06:07:56.722 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T06:07:56.722 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T06:07:56.981 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:07:57.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.488+0000 7fa7e0ea2640 1 -- 172.21.15.117:0/4246859899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7dc105320 msgr2=0x7fa7dc10d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:57.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.488+0000 7fa7e0ea2640 1 --2- 172.21.15.117:0/4246859899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7dc105320 0x7fa7dc10d870 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4009900 tx=0x7fa7c402a3f0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.488+0000 7fa7e0ea2640 1 -- 172.21.15.117:0/4246859899 shutdown_connections 2024-09-21T06:07:57.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.488+0000 7fa7e0ea2640 1 --2- 172.21.15.117:0/4246859899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7dc105320 0x7fa7dc10d870 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.488+0000 7fa7e0ea2640 1 --2- 172.21.15.117:0/4246859899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7dc10de30 0x7fa7dc110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.488+0000 7fa7e0ea2640 1 --2- 172.21.15.117:0/4246859899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7dc1049d0 0x7fa7dc104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.488+0000 7fa7e0ea2640 1 -- 172.21.15.117:0/4246859899 >> 172.21.15.117:0/4246859899 conn(0x7fa7dc079740 msgr2=0x7fa7dc079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:57.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.488+0000 7fa7e0ea2640 1 -- 172.21.15.117:0/4246859899 shutdown_connections 2024-09-21T06:07:57.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.488+0000 7fa7e0ea2640 1 -- 172.21.15.117:0/4246859899 wait complete. 2024-09-21T06:07:57.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.489+0000 7fa7e0ea2640 1 Processor -- start 2024-09-21T06:07:57.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.489+0000 7fa7e0ea2640 1 -- start start 2024-09-21T06:07:57.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.489+0000 7fa7e0ea2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7dc1049d0 0x7fa7dc19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:57.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.489+0000 7fa7e0ea2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7dc105320 0x7fa7dc19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:57.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.489+0000 7fa7e0ea2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7dc10de30 0x7fa7dc19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:57.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.489+0000 7fa7e0ea2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc113a90 con 0x7fa7dc10de30 2024-09-21T06:07:57.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.489+0000 7fa7e0ea2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc113910 con 0x7fa7dc1049d0 2024-09-21T06:07:57.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.489+0000 7fa7e0ea2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc113c10 con 0x7fa7dc105320 2024-09-21T06:07:57.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.490+0000 7fa7dad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7dc10de30 0x7fa7dc19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:57.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.490+0000 7fa7dad9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7dc10de30 0x7fa7dc19fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48096/0 (socket says 172.21.15.117:48096) 2024-09-21T06:07:57.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.490+0000 7fa7dad9d640 1 -- 172.21.15.117:0/3875383576 learned_addr learned my addr 172.21.15.117:0/3875383576 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:57.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.490+0000 7fa7da59c640 1 --2- 172.21.15.117:0/3875383576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7dc1049d0 0x7fa7dc19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:57.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.490+0000 7fa7d9d9b640 1 --2- 172.21.15.117:0/3875383576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7dc105320 0x7fa7dc19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:57.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.490+0000 7fa7dad9d640 1 -- 172.21.15.117:0/3875383576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7dc105320 msgr2=0x7fa7dc19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:57.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.490+0000 7fa7dad9d640 1 --2- 172.21.15.117:0/3875383576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7dc105320 0x7fa7dc19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.490+0000 7fa7dad9d640 1 -- 172.21.15.117:0/3875383576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7dc1049d0 msgr2=0x7fa7dc19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:57.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.490+0000 7fa7dad9d640 1 --2- 172.21.15.117:0/3875383576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7dc1049d0 0x7fa7dc19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.490+0000 7fa7dad9d640 1 -- 172.21.15.117:0/3875383576 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7dc1a0370 con 0x7fa7dc10de30 2024-09-21T06:07:57.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.490+0000 7fa7d9d9b640 1 --2- 172.21.15.117:0/3875383576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7dc105320 0x7fa7dc19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:57.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.490+0000 7fa7dad9d640 1 --2- 172.21.15.117:0/3875383576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7dc10de30 0x7fa7dc19fd00 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fa7d400c970 tx=0x7fa7d400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:57.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.490+0000 7fa7d37fe640 1 -- 172.21.15.117:0/3875383576 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d4007bb0 con 0x7fa7dc10de30 2024-09-21T06:07:57.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.490+0000 7fa7d37fe640 1 -- 172.21.15.117:0/3875383576 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7d4007d50 con 0x7fa7dc10de30 2024-09-21T06:07:57.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.490+0000 7fa7e0ea2640 1 -- 172.21.15.117:0/3875383576 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7dc1a0660 con 0x7fa7dc10de30 2024-09-21T06:07:57.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.490+0000 7fa7d37fe640 1 -- 172.21.15.117:0/3875383576 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d4018770 con 0x7fa7dc10de30 2024-09-21T06:07:57.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.490+0000 7fa7e0ea2640 1 -- 172.21.15.117:0/3875383576 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7dc1a0b20 con 0x7fa7dc10de30 2024-09-21T06:07:57.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.493+0000 7fa7e7416740 1 Processor -- start 2024-09-21T06:07:57.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.493+0000 7fa7d37fe640 1 -- 172.21.15.117:0/3875383576 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fa7d40213f0 con 0x7fa7dc10de30 2024-09-21T06:07:57.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.493+0000 7fa7e7416740 1 -- start start 2024-09-21T06:07:57.508 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.493+0000 7fa7e7416740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571836d6350 0x557183a7d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:57.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.493+0000 7fa7e7416740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571836ad490 0x557183a7d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:57.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.493+0000 7fa7e7416740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571836b5670 0x557183a7deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:57.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.493+0000 7fa7e7416740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5571839f3e60 con 0x5571836b5670 2024-09-21T06:07:57.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.493+0000 7fa7e7416740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5571839e3290 con 0x5571836ad490 2024-09-21T06:07:57.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.493+0000 7fa7e7416740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5571839e70a0 con 0x5571836d6350 2024-09-21T06:07:57.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.494+0000 7fa7d9d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571836b5670 0x557183a7deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:57.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.494+0000 7fa7d9d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571836b5670 0x557183a7deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48098/0 (socket says 172.21.15.117:48098) 2024-09-21T06:07:57.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.494+0000 7fa7d9d9b640 1 -- 172.21.15.117:0/2699211705 learned_addr learned my addr 172.21.15.117:0/2699211705 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:57.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.494+0000 7fa7da59c640 1 --2- 172.21.15.117:0/2699211705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571836ad490 0x557183a7d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:57.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.494+0000 7fa7d9d9b640 1 --2- 172.21.15.117:0/2699211705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571836d6350 0x557183a7d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:57.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.494+0000 7fa7d9d9b640 1 -- 172.21.15.117:0/2699211705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571836d6350 msgr2=0x557183a7d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:57.509 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.494+0000 7fa7d9d9b640 1 --2- 172.21.15.117:0/2699211705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571836d6350 0x557183a7d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.494+0000 7fa7d37fe640 1 --2- 172.21.15.117:0/3875383576 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa7c80b66b0 0x7fa7c80b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:57.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.494+0000 7fa7d9d9b640 1 -- 172.21.15.117:0/2699211705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571836ad490 msgr2=0x557183a7d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:57.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.494+0000 7fa7d9d9b640 1 --2- 172.21.15.117:0/2699211705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571836ad490 0x557183a7d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.494+0000 7fa7d9d9b640 1 -- 172.21.15.117:0/2699211705 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557183741930 con 0x5571836b5670 2024-09-21T06:07:57.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.494+0000 7fa7dad9d640 1 --2- 172.21.15.117:0/3875383576 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa7c80b66b0 0x7fa7c80b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:57.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.494+0000 7fa7d9d9b640 1 --2- 172.21.15.117:0/2699211705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571836b5670 0x557183a7deb0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4009900 tx=0x7fa7c4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e7fcbecb668d66 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:57.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.494+0000 7fa7d37fe640 1 -- 172.21.15.117:0/3875383576 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7fa7d40e76c0 con 0x7fa7dc10de30 2024-09-21T06:07:57.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.494+0000 7fa7d0ff9640 1 -- 172.21.15.117:0/2699211705 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4035040 con 0x5571836b5670 2024-09-21T06:07:57.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.494+0000 7fa7d0ff9640 1 -- 172.21.15.117:0/2699211705 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7c4004030 con 0x5571836b5670 2024-09-21T06:07:57.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.494+0000 7fa7d0ff9640 1 -- 172.21.15.117:0/2699211705 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4003b20 con 0x5571836b5670 2024-09-21T06:07:57.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.495+0000 7fa7dad9d640 1 --2- 172.21.15.117:0/3875383576 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa7c80b66b0 0x7fa7c80b8bc0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40ad4f0 tx=0x7fa7d4079040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:57.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.495+0000 7fa7e7416740 1 -- 172.21.15.117:0/2699211705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571836b5670 msgr2=0x557183a7deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:57.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.495+0000 7fa7e7416740 1 --2- 172.21.15.117:0/2699211705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571836b5670 0x557183a7deb0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4009900 tx=0x7fa7c4000bc0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.495+0000 7fa7e7416740 1 -- 172.21.15.117:0/2699211705 shutdown_connections 2024-09-21T06:07:57.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.495+0000 7fa7e7416740 1 --2- 172.21.15.117:0/2699211705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571836b5670 0x557183a7deb0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.495+0000 7fa7e7416740 1 --2- 172.21.15.117:0/2699211705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571836ad490 0x557183a7d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.495+0000 7fa7e7416740 1 --2- 172.21.15.117:0/2699211705 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571836d6350 0x557183a7d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.495+0000 7fa7e7416740 1 -- 172.21.15.117:0/2699211705 >> 172.21.15.117:0/2699211705 conn(0x5571836ca740 msgr2=0x557183a7b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:57.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.496+0000 7fa7e7416740 1 -- 172.21.15.117:0/2699211705 shutdown_connections 2024-09-21T06:07:57.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.496+0000 7fa7e7416740 1 -- 172.21.15.117:0/2699211705 wait complete. 2024-09-21T06:07:57.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.496+0000 7fa7e7416740 1 -- start start 2024-09-21T06:07:57.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.496+0000 7fa7e7416740 1 Processor -- start 2024-09-21T06:07:57.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.496+0000 7fa7e7416740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571836ad490 0x557183a88fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:57.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.496+0000 7fa7e7416740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571836b5670 0x557183a894f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:57.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.496+0000 7fa7e7416740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571836d6350 0x557183a94ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:57.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.496+0000 7fa7e7416740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557183844890 con 0x5571836b5670 2024-09-21T06:07:57.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.496+0000 7fa7e7416740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55718397dd40 con 0x5571836d6350 2024-09-21T06:07:57.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.496+0000 7fa7e7416740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557183a8bbb0 con 0x5571836ad490 2024-09-21T06:07:57.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.496+0000 7fa7e7416740 10 client.0 _ll_register_callbacks cb 0x5571839e1c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:57.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.496+0000 7fa7da59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571836b5670 0x557183a894f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:57.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.496+0000 7fa7da59c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571836b5670 0x557183a894f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48100/0 (socket says 172.21.15.117:48100) 2024-09-21T06:07:57.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.497+0000 7fa7da59c640 1 -- 172.21.15.117:0/1064073965 learned_addr learned my addr 172.21.15.117:0/1064073965 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:57.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.497+0000 7fa7d9d9b640 1 --2- 172.21.15.117:0/1064073965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571836ad490 0x557183a88fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:57.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.497+0000 7fa7d9d9b640 1 --2- 172.21.15.117:0/1064073965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571836d6350 0x557183a94ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:57.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.497+0000 7fa7da59c640 1 -- 172.21.15.117:0/1064073965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571836ad490 msgr2=0x557183a88fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:57.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.497+0000 7fa7da59c640 1 --2- 172.21.15.117:0/1064073965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571836ad490 0x557183a88fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.497+0000 7fa7da59c640 1 -- 172.21.15.117:0/1064073965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571836d6350 msgr2=0x557183a94ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:57.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.497+0000 7fa7da59c640 1 --2- 172.21.15.117:0/1064073965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571836d6350 0x557183a94ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.497+0000 7fa7da59c640 1 -- 172.21.15.117:0/1064073965 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557183a95430 con 0x5571836b5670 2024-09-21T06:07:57.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.497+0000 7fa7d9d9b640 1 --2- 172.21.15.117:0/1064073965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571836d6350 0x557183a94ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:07:57.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.497+0000 7fa7da59c640 1 --2- 172.21.15.117:0/1064073965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571836b5670 0x557183a894f0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc0042f0 tx=0x7fa7cc003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5117e867acd77e32 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:57.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.497+0000 7fa7c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:57.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.497+0000 7fa7c1ffb640 1 -- 172.21.15.117:0/1064073965 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc013e50 con 0x5571836b5670 2024-09-21T06:07:57.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.497+0000 7fa7c1ffb640 1 -- 172.21.15.117:0/1064073965 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7fa7cc004b20 con 0x5571836b5670 2024-09-21T06:07:57.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.497+0000 7fa7e7416740 1 -- 172.21.15.117:0/1064073965 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7c4004030 con 0x5571836b5670 2024-09-21T06:07:57.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.497+0000 7fa7c1ffb640 1 -- 172.21.15.117:0/1064073965 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa7cc009e20 con 0x5571836b5670 2024-09-21T06:07:57.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.497+0000 7fa7e7416740 10 client.17985 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:57.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.497+0000 7fa7e7416740 1 -- 172.21.15.117:0/1064073965 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557183a97dc0 con 0x5571836b5670 2024-09-21T06:07:57.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.498+0000 7fa7c1ffb640 1 -- 172.21.15.117:0/1064073965 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7c4004030 con 0x5571836b5670 2024-09-21T06:07:57.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.498+0000 7fa7c1ffb640 1 -- 172.21.15.117:0/1064073965 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa7cc01a030 con 0x5571836b5670 2024-09-21T06:07:57.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.498+0000 7fa7e7416740 10 client.17985 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:57.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.498+0000 7fa7e7416740 10 client.17985 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:07:57.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.498+0000 7fa7e7416740 20 client.17985 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:57.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.498+0000 7fa7e7416740 1 --2- 172.21.15.117:0/1064073965 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x557183694cd0 0x557183a9b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:57.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.498+0000 7fa7e7416740 4 client.17985 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:07:57.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.498+0000 7fa7e7416740 1 -- 172.21.15.117:0/1064073965 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571839f5db0 con 0x557183694cd0 2024-09-21T06:07:57.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.498+0000 7fa7d9d9b640 1 --2- 172.21.15.117:0/1064073965 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x557183694cd0 0x557183a9b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:57.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.499+0000 7fa7c1ffb640 10 client.17985 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:57.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.499+0000 7fa7d9d9b640 1 --2- 172.21.15.117:0/1064073965 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x557183694cd0 0x557183a9b3f0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e364ce5b50bff2b0 server_cookie=d2d0f3e15d364c6b in_seq=0 out_seq=0 2024-09-21T06:07:57.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.499+0000 7fa7c1ffb640 1 -- 172.21.15.117:0/1064073965 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5571839f5db0 con 0x557183694cd0 2024-09-21T06:07:57.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.499+0000 7fa7c1ffb640 10 client.17985 handle_command_reply: tid=0 2024-09-21T06:07:57.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.499+0000 7fa7e7416740 1 client.17985 shutdown 2024-09-21T06:07:57.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.499+0000 7fa7e7416740 1 -- 172.21.15.117:0/1064073965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571836b5670 msgr2=0x557183a894f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:57.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.499+0000 7fa7e7416740 1 --2- 172.21.15.117:0/1064073965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571836b5670 0x557183a894f0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc0042f0 tx=0x7fa7cc003f90 comp rx=0 tx=0).stop 2024-09-21T06:07:57.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.499+0000 7fa7e7416740 1 -- 172.21.15.117:0/1064073965 shutdown_connections 2024-09-21T06:07:57.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.499+0000 7fa7e7416740 1 --2- 172.21.15.117:0/1064073965 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x557183694cd0 0x557183a9b3f0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.499+0000 7fa7e7416740 1 --2- 172.21.15.117:0/1064073965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571836b5670 0x557183a894f0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.499+0000 7fa7e7416740 1 --2- 172.21.15.117:0/1064073965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571836d6350 0x557183a94ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.499+0000 7fa7e7416740 1 --2- 172.21.15.117:0/1064073965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571836ad490 0x557183a88fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.499+0000 7fa7c1ffb640 0 client.17985 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:57.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.499+0000 7fa7e7416740 1 -- 172.21.15.117:0/1064073965 >> 172.21.15.117:0/1064073965 conn(0x5571836ca740 msgr2=0x557183a805c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:57.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.500+0000 7fa7e7416740 1 -- 172.21.15.117:0/1064073965 shutdown_connections 2024-09-21T06:07:57.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.500+0000 7fa7e7416740 1 -- 172.21.15.117:0/1064073965 wait complete. 2024-09-21T06:07:57.514 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.500+0000 7fa7e7416740 20 client.17985 trim_cache size 0 max 16384 2024-09-21T06:07:57.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.517+0000 7fa7e7416740 1 Processor -- start 2024-09-21T06:07:57.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.517+0000 7fa7e7416740 1 -- start start 2024-09-21T06:07:57.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.517+0000 7fa7e7416740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571836ad490 0x557183a83550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:57.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.517+0000 7fa7e7416740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571836b5670 0x557183a94da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:57.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.517+0000 7fa7e7416740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571836d6350 0x557183a85fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:57.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.517+0000 7fa7e7416740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc1af470 con 0x5571836d6350 2024-09-21T06:07:57.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.517+0000 7fa7e7416740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc1b0fb0 con 0x5571836ad490 2024-09-21T06:07:57.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.517+0000 7fa7e7416740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc1afb40 con 0x5571836b5670 2024-09-21T06:07:57.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.517+0000 7fa7d9d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571836d6350 0x557183a85fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:57.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.517+0000 7fa7d9d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571836d6350 0x557183a85fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48116/0 (socket says 172.21.15.117:48116) 2024-09-21T06:07:57.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.517+0000 7fa7d9d9b640 1 -- 172.21.15.117:0/2878277328 learned_addr learned my addr 172.21.15.117:0/2878277328 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:57.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.517+0000 7fa7da59c640 1 --2- 172.21.15.117:0/2878277328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571836b5670 0x557183a94da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:57.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.517+0000 7fa7d9d9b640 1 --2- 172.21.15.117:0/2878277328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571836ad490 0x557183a83550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:57.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.517+0000 7fa7d9d9b640 1 -- 172.21.15.117:0/2878277328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571836b5670 msgr2=0x557183a94da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:57.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.517+0000 7fa7d9d9b640 1 --2- 172.21.15.117:0/2878277328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571836b5670 0x557183a94da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.518+0000 7fa7d9d9b640 1 -- 172.21.15.117:0/2878277328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571836ad490 msgr2=0x557183a83550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:57.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.518+0000 7fa7d9d9b640 1 --2- 172.21.15.117:0/2878277328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571836ad490 0x557183a83550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.518+0000 7fa7d9d9b640 1 -- 172.21.15.117:0/2878277328 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557183a80290 con 0x5571836d6350 2024-09-21T06:07:57.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.518+0000 7fa7d9d9b640 1 --2- 172.21.15.117:0/2878277328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571836d6350 0x557183a85fa0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4003770 tx=0x7fa7c404a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948ca2e96806f7ea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:57.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.518+0000 7fa7c3fff640 1 -- 172.21.15.117:0/2878277328 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4035040 con 0x5571836d6350 2024-09-21T06:07:57.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.518+0000 7fa7c3fff640 1 -- 172.21.15.117:0/2878277328 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7c4003810 con 0x5571836d6350 2024-09-21T06:07:57.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.518+0000 7fa7c3fff640 1 -- 172.21.15.117:0/2878277328 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c403d450 con 0x5571836d6350 2024-09-21T06:07:57.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.519+0000 7fa7e7416740 1 -- 172.21.15.117:0/2878277328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571836d6350 msgr2=0x557183a85fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:57.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.519+0000 7fa7e7416740 1 --2- 172.21.15.117:0/2878277328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571836d6350 0x557183a85fa0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4003770 tx=0x7fa7c404a040 comp rx=0 tx=0).stop 2024-09-21T06:07:57.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.519+0000 7fa7e7416740 1 -- 172.21.15.117:0/2878277328 shutdown_connections 2024-09-21T06:07:57.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.519+0000 7fa7e7416740 1 --2- 172.21.15.117:0/2878277328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571836d6350 0x557183a85fa0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.519+0000 7fa7e7416740 1 --2- 172.21.15.117:0/2878277328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571836b5670 0x557183a94da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.519+0000 7fa7e7416740 1 --2- 172.21.15.117:0/2878277328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571836ad490 0x557183a83550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.519+0000 7fa7e7416740 1 -- 172.21.15.117:0/2878277328 >> 172.21.15.117:0/2878277328 conn(0x5571836ca740 msgr2=0x557183a87630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:57.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.519+0000 7fa7e7416740 1 -- 172.21.15.117:0/2878277328 shutdown_connections 2024-09-21T06:07:57.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.519+0000 7fa7e7416740 1 -- 172.21.15.117:0/2878277328 wait complete. 2024-09-21T06:07:57.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.519+0000 7fa7e7416740 1 -- start start 2024-09-21T06:07:57.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.519+0000 7fa7e7416740 1 Processor -- start 2024-09-21T06:07:57.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.520+0000 7fa7e7416740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571836ad490 0x557183a8d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:57.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.520+0000 7fa7e7416740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571836b5670 0x557183a8dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:57.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.520+0000 7fa7e7416740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571836d6350 0x557183afec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:57.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.520+0000 7fa7e7416740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc1b08e0 con 0x5571836ad490 2024-09-21T06:07:57.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.520+0000 7fa7e7416740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc1b0210 con 0x5571836d6350 2024-09-21T06:07:57.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.520+0000 7fa7e7416740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc1ad470 con 0x5571836b5670 2024-09-21T06:07:57.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.520+0000 7fa7e7416740 10 client.0 _ll_register_callbacks cb 0x7fa7cc009cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:57.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.520+0000 7fa7d9d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571836ad490 0x557183a8d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:57.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.520+0000 7fa7d9d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571836ad490 0x557183a8d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48126/0 (socket says 172.21.15.117:48126) 2024-09-21T06:07:57.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.520+0000 7fa7d9d9b640 1 -- 172.21.15.117:0/1552277355 learned_addr learned my addr 172.21.15.117:0/1552277355 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:57.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.520+0000 7fa7da59c640 1 --2- 172.21.15.117:0/1552277355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571836b5670 0x557183a8dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:57.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.520+0000 7fa7d9d9b640 1 -- 172.21.15.117:0/1552277355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571836b5670 msgr2=0x557183a8dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:57.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.520+0000 7fa7d9d9b640 1 --2- 172.21.15.117:0/1552277355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571836b5670 0x557183a8dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.520+0000 7fa7d9d9b640 1 -- 172.21.15.117:0/1552277355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571836d6350 msgr2=0x557183afec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:57.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.520+0000 7fa7d9d9b640 1 --2- 172.21.15.117:0/1552277355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571836d6350 0x557183afec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.520+0000 7fa7d9d9b640 1 -- 172.21.15.117:0/1552277355 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557183aff2d0 con 0x5571836ad490 2024-09-21T06:07:57.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.520+0000 7fa7d9d9b640 1 --2- 172.21.15.117:0/1552277355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571836ad490 0x557183a8d6e0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4005720 tx=0x7fa7c4005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb1d343fe86a4a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:57.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.520+0000 7fa7c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:57.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.520+0000 7fa7c1ffb640 1 -- 172.21.15.117:0/1552277355 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4035040 con 0x5571836ad490 2024-09-21T06:07:57.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.521+0000 7fa7c1ffb640 1 -- 172.21.15.117:0/1552277355 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7fa7c4031870 con 0x5571836ad490 2024-09-21T06:07:57.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.521+0000 7fa7e7416740 1 -- 172.21.15.117:0/1552277355 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7c4003810 con 0x5571836ad490 2024-09-21T06:07:57.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.521+0000 7fa7c1ffb640 1 -- 172.21.15.117:0/1552277355 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa7c406e050 con 0x5571836ad490 2024-09-21T06:07:57.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.521+0000 7fa7e7416740 10 client.17991 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:57.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.521+0000 7fa7e7416740 1 -- 172.21.15.117:0/1552277355 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557183b06650 con 0x5571836ad490 2024-09-21T06:07:57.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.521+0000 7fa7c1ffb640 1 -- 172.21.15.117:0/1552277355 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7c4003810 con 0x5571836ad490 2024-09-21T06:07:57.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.521+0000 7fa7c1ffb640 1 -- 172.21.15.117:0/1552277355 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa7c403a040 con 0x5571836ad490 2024-09-21T06:07:57.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.521+0000 7fa7e7416740 10 client.17991 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:57.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.521+0000 7fa7e7416740 10 client.17991 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:07:57.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.521+0000 7fa7e7416740 20 client.17991 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:57.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.521+0000 7fa7e7416740 1 --2- 172.21.15.117:0/1552277355 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55718366a080 0x557183b089b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:57.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.521+0000 7fa7e7416740 4 client.17991 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T06:07:57.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.521+0000 7fa7e7416740 1 -- 172.21.15.117:0/1552277355 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5571839f1860 con 0x55718366a080 2024-09-21T06:07:57.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.522+0000 7fa7da59c640 1 --2- 172.21.15.117:0/1552277355 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55718366a080 0x557183b089b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:57.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.522+0000 7fa7c1ffb640 10 client.17991 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:57.528 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.522+0000 7fa7da59c640 1 --2- 172.21.15.117:0/1552277355 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55718366a080 0x557183b089b0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4d1fbf758a594c6 server_cookie=c5e50454656d97c2 in_seq=0 out_seq=0 2024-09-21T06:07:57.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.524+0000 7fa7c1ffb640 1 -- 172.21.15.117:0/1552277355 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5571839f1860 con 0x55718366a080 2024-09-21T06:07:57.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.524+0000 7fa7c1ffb640 10 client.17991 handle_command_reply: tid=0 2024-09-21T06:07:57.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.524+0000 7fa7e7416740 1 client.17991 shutdown 2024-09-21T06:07:57.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.524+0000 7fa7e7416740 1 -- 172.21.15.117:0/1552277355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571836ad490 msgr2=0x557183a8d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:57.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.524+0000 7fa7e7416740 1 --2- 172.21.15.117:0/1552277355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571836ad490 0x557183a8d6e0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4005720 tx=0x7fa7c4005750 comp rx=0 tx=0).stop 2024-09-21T06:07:57.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.524+0000 7fa7e7416740 1 -- 172.21.15.117:0/1552277355 shutdown_connections 2024-09-21T06:07:57.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.524+0000 7fa7e7416740 1 --2- 172.21.15.117:0/1552277355 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55718366a080 0x557183b089b0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.524+0000 7fa7e7416740 1 --2- 172.21.15.117:0/1552277355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5571836d6350 0x557183afec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.524+0000 7fa7e7416740 1 --2- 172.21.15.117:0/1552277355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571836b5670 0x557183a8dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.524+0000 7fa7c1ffb640 0 client.17991 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:57.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.524+0000 7fa7e7416740 1 --2- 172.21.15.117:0/1552277355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5571836ad490 0x557183a8d6e0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.524+0000 7fa7e7416740 1 -- 172.21.15.117:0/1552277355 >> 172.21.15.117:0/1552277355 conn(0x5571836ca740 msgr2=0x557183a7e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:57.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.524+0000 7fa7e7416740 1 -- 172.21.15.117:0/1552277355 shutdown_connections 2024-09-21T06:07:57.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.524+0000 7fa7e7416740 1 -- 172.21.15.117:0/1552277355 wait complete. 2024-09-21T06:07:57.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.524+0000 7fa7e7416740 20 client.17991 trim_cache size 0 max 16384 2024-09-21T06:07:57.530 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T06:07:57.531 INFO:teuthology.orchestra.run.smithi117.stdout: "return_code": 0, 2024-09-21T06:07:57.531 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_tag": "bcb8bc27-fdcf-48a3-819b-c98b933f1d97", 2024-09-21T06:07:57.531 INFO:teuthology.orchestra.run.smithi117.stdout: "mode": "asynchronous" 2024-09-21T06:07:57.531 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T06:07:57.531 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.525+0000 7fa7e0ea2640 1 -- 172.21.15.117:0/3875383576 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa7c80b66b0 msgr2=0x7fa7c80b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:57.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.525+0000 7fa7e0ea2640 1 --2- 172.21.15.117:0/3875383576 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa7c80b66b0 0x7fa7c80b8bc0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40ad4f0 tx=0x7fa7d4079040 comp rx=0 tx=0).stop 2024-09-21T06:07:57.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.525+0000 7fa7e0ea2640 1 -- 172.21.15.117:0/3875383576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7dc10de30 msgr2=0x7fa7dc19fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:57.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.525+0000 7fa7e0ea2640 1 --2- 172.21.15.117:0/3875383576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7dc10de30 0x7fa7dc19fd00 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fa7d400c970 tx=0x7fa7d400ce30 comp rx=0 tx=0).stop 2024-09-21T06:07:57.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.525+0000 7fa7e0ea2640 1 -- 172.21.15.117:0/3875383576 shutdown_connections 2024-09-21T06:07:57.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.525+0000 7fa7e0ea2640 1 --2- 172.21.15.117:0/3875383576 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa7c80b66b0 0x7fa7c80b8bc0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.525+0000 7fa7e0ea2640 1 --2- 172.21.15.117:0/3875383576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7dc10de30 0x7fa7dc19fd00 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.525+0000 7fa7e0ea2640 1 --2- 172.21.15.117:0/3875383576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa7dc105320 0x7fa7dc19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.525+0000 7fa7e0ea2640 1 --2- 172.21.15.117:0/3875383576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7dc1049d0 0x7fa7dc19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:57.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.525+0000 7fa7e0ea2640 1 -- 172.21.15.117:0/3875383576 >> 172.21.15.117:0/3875383576 conn(0x7fa7dc079740 msgr2=0x7fa7dc0ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:57.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.525+0000 7fa7e0ea2640 1 -- 172.21.15.117:0/3875383576 shutdown_connections 2024-09-21T06:07:57.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:57.525+0000 7fa7e0ea2640 1 -- 172.21.15.117:0/3875383576 wait complete. 2024-09-21T06:07:58.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:57 smithi192 ceph-mon[34700]: pgmap v653: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T06:07:58.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:57 smithi192 ceph-mon[34700]: scrub queued for path: / 2024-09-21T06:07:58.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:57 smithi192 ceph-mon[34700]: scrub summary: idle+waiting paths [/] 2024-09-21T06:07:58.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:57 smithi192 ceph-mon[34700]: scrub summary: active paths [/] 2024-09-21T06:07:58.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:57 smithi192 ceph-mon[34700]: scrub summary: idle+waiting paths [/] 2024-09-21T06:07:58.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:57 smithi117 ceph-mon[30947]: pgmap v653: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T06:07:58.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:57 smithi117 ceph-mon[30947]: scrub queued for path: / 2024-09-21T06:07:58.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:57 smithi117 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-21T06:07:58.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:57 smithi117 ceph-mon[30947]: scrub summary: active paths [/] 2024-09-21T06:07:58.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:57 smithi117 ceph-mon[30947]: scrub summary: idle+waiting paths [/] 2024-09-21T06:07:58.338 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:07:58.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:57 smithi138 ceph-mon[34391]: pgmap v653: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T06:07:58.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:57 smithi138 ceph-mon[34391]: scrub queued for path: / 2024-09-21T06:07:58.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:57 smithi138 ceph-mon[34391]: scrub summary: idle+waiting paths [/] 2024-09-21T06:07:58.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:57 smithi138 ceph-mon[34391]: scrub summary: active paths [/] 2024-09-21T06:07:58.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:57 smithi138 ceph-mon[34391]: scrub summary: idle+waiting paths [/] 2024-09-21T06:07:58.585 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:07:59.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.072+0000 7f8cea281640 1 -- 172.21.15.117:0/2787926386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ce41051c0 msgr2=0x7f8ce41055d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:59.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.072+0000 7f8cea281640 1 --2- 172.21.15.117:0/2787926386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ce41051c0 0x7f8ce41055d0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8009900 tx=0x7f8cd802a410 comp rx=0 tx=0).stop 2024-09-21T06:07:59.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.072+0000 7f8cea281640 1 -- 172.21.15.117:0/2787926386 shutdown_connections 2024-09-21T06:07:59.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.072+0000 7f8cea281640 1 --2- 172.21.15.117:0/2787926386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ce41051c0 0x7f8ce41055d0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:59.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.072+0000 7f8cea281640 1 --2- 172.21.15.117:0/2787926386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ce4105b10 0x7f8ce410f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:59.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.072+0000 7f8cea281640 1 --2- 172.21.15.117:0/2787926386 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ce41031b0 0x7f8ce41035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:59.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.072+0000 7f8cea281640 1 -- 172.21.15.117:0/2787926386 >> 172.21.15.117:0/2787926386 conn(0x7f8ce4079e40 msgr2=0x7f8ce407a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:59.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.072+0000 7f8cea281640 1 -- 172.21.15.117:0/2787926386 shutdown_connections 2024-09-21T06:07:59.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.072+0000 7f8cea281640 1 -- 172.21.15.117:0/2787926386 wait complete. 2024-09-21T06:07:59.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.073+0000 7f8cea281640 1 Processor -- start 2024-09-21T06:07:59.086 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.073+0000 7f8cea281640 1 -- start start 2024-09-21T06:07:59.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.073+0000 7f8cea281640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ce41031b0 0x7f8ce419b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:59.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.073+0000 7f8cea281640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ce41051c0 0x7f8ce419b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:59.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.073+0000 7f8cea281640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ce4105b10 0x7f8ce419bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:59.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.073+0000 7f8cea281640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce41129e0 con 0x7f8ce41031b0 2024-09-21T06:07:59.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.073+0000 7f8cea281640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce4112860 con 0x7f8ce4105b10 2024-09-21T06:07:59.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.073+0000 7f8cea281640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce4112b60 con 0x7f8ce41051c0 2024-09-21T06:07:59.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.074+0000 7f8ce3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ce41031b0 0x7f8ce419b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:59.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.074+0000 7f8ce3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ce41031b0 0x7f8ce419b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48134/0 (socket says 172.21.15.117:48134) 2024-09-21T06:07:59.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.074+0000 7f8ce3fff640 1 -- 172.21.15.117:0/1226406570 learned_addr learned my addr 172.21.15.117:0/1226406570 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:59.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.074+0000 7f8ce881e640 1 --2- 172.21.15.117:0/1226406570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ce4105b10 0x7f8ce419bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:59.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.074+0000 7f8ce37fe640 1 --2- 172.21.15.117:0/1226406570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ce41051c0 0x7f8ce419b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:59.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.074+0000 7f8ce3fff640 1 -- 172.21.15.117:0/1226406570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ce41051c0 msgr2=0x7f8ce419b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:59.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.074+0000 7f8ce3fff640 1 --2- 172.21.15.117:0/1226406570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ce41051c0 0x7f8ce419b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:59.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.074+0000 7f8ce3fff640 1 -- 172.21.15.117:0/1226406570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ce4105b10 msgr2=0x7f8ce419bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:59.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.074+0000 7f8ce3fff640 1 --2- 172.21.15.117:0/1226406570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ce4105b10 0x7f8ce419bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:59.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.074+0000 7f8ce3fff640 1 -- 172.21.15.117:0/1226406570 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ce41a6570 con 0x7f8ce41031b0 2024-09-21T06:07:59.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.074+0000 7f8ce3fff640 1 --2- 172.21.15.117:0/1226406570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ce41031b0 0x7f8ce419b2b0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f8cd400c970 tx=0x7f8cd400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:59.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.074+0000 7f8ce17fa640 1 -- 172.21.15.117:0/1226406570 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4004220 con 0x7f8ce41031b0 2024-09-21T06:07:59.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.074+0000 7f8cea281640 1 -- 172.21.15.117:0/1226406570 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ce41a6860 con 0x7f8ce41031b0 2024-09-21T06:07:59.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.074+0000 7f8ce17fa640 1 -- 172.21.15.117:0/1226406570 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cd40043c0 con 0x7f8ce41031b0 2024-09-21T06:07:59.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.074+0000 7f8ce17fa640 1 -- 172.21.15.117:0/1226406570 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4004a00 con 0x7f8ce41031b0 2024-09-21T06:07:59.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.074+0000 7f8cea281640 1 -- 172.21.15.117:0/1226406570 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ce41a6d70 con 0x7f8ce41031b0 2024-09-21T06:07:59.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.077+0000 7f8ce17fa640 1 -- 172.21.15.117:0/1226406570 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f8cd40079b0 con 0x7f8ce41031b0 2024-09-21T06:07:59.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.077+0000 7f8cf07f5740 1 Processor -- start 2024-09-21T06:07:59.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.077+0000 7f8cf07f5740 1 -- start start 2024-09-21T06:07:59.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.077+0000 7f8cf07f5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f48424070 0x561f487cb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:59.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.077+0000 7f8cf07f5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f483fb1b0 0x561f487cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:59.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.077+0000 7f8cf07f5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f48403390 0x561f487cc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:59.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.077+0000 7f8cf07f5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561f48741e50 con 0x561f48424070 2024-09-21T06:07:59.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.077+0000 7f8cf07f5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561f48731280 con 0x561f483fb1b0 2024-09-21T06:07:59.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.077+0000 7f8cf07f5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x561f48735090 con 0x561f48403390 2024-09-21T06:07:59.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.077+0000 7f8ce881e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f48424070 0x561f487cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:59.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.078+0000 7f8ce881e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f48424070 0x561f487cb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48150/0 (socket says 172.21.15.117:48150) 2024-09-21T06:07:59.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.078+0000 7f8ce881e640 1 -- 172.21.15.117:0/458643244 learned_addr learned my addr 172.21.15.117:0/458643244 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:59.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.078+0000 7f8ce37fe640 1 --2- 172.21.15.117:0/458643244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f48403390 0x561f487cc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:59.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.078+0000 7f8ce3fff640 1 --2- 172.21.15.117:0/458643244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f483fb1b0 0x561f487cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:59.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.078+0000 7f8ce881e640 1 -- 172.21.15.117:0/458643244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f48403390 msgr2=0x561f487cc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:59.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.078+0000 7f8ce17fa640 1 --2- 172.21.15.117:0/1226406570 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8ccc0b6680 0x7f8ccc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:59.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.078+0000 7f8ce881e640 1 --2- 172.21.15.117:0/458643244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f48403390 0x561f487cc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:59.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.078+0000 7f8ce881e640 1 -- 172.21.15.117:0/458643244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f483fb1b0 msgr2=0x561f487cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:59.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.078+0000 7f8ce881e640 1 --2- 172.21.15.117:0/458643244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f483fb1b0 0x561f487cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:59.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.078+0000 7f8ce881e640 1 -- 172.21.15.117:0/458643244 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f4848f650 con 0x561f48424070 2024-09-21T06:07:59.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.078+0000 7f8ce3fff640 1 --2- 172.21.15.117:0/458643244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f483fb1b0 0x561f487cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:07:59.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.078+0000 7f8ce881e640 1 --2- 172.21.15.117:0/1226406570 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8ccc0b6680 0x7f8ccc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:59.089 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.078+0000 7f8ce17fa640 1 -- 172.21.15.117:0/1226406570 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f8cd40e74c0 con 0x7f8ce41031b0 2024-09-21T06:07:59.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.078+0000 7f8ce881e640 1 --2- 172.21.15.117:0/458643244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f48424070 0x561f487cb650 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc010370 tx=0x7f8cdc010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0c025d7caebef77 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:59.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.078+0000 7f8cca7fc640 1 -- 172.21.15.117:0/458643244 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc022070 con 0x561f48424070 2024-09-21T06:07:59.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.078+0000 7f8cca7fc640 1 -- 172.21.15.117:0/458643244 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cdc012c10 con 0x561f48424070 2024-09-21T06:07:59.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.078+0000 7f8cca7fc640 1 -- 172.21.15.117:0/458643244 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc016070 con 0x561f48424070 2024-09-21T06:07:59.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.079+0000 7f8cf07f5740 1 -- 172.21.15.117:0/458643244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f48424070 msgr2=0x561f487cb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:59.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.079+0000 7f8cf07f5740 1 --2- 172.21.15.117:0/458643244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f48424070 0x561f487cb650 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc010370 tx=0x7f8cdc010830 comp rx=0 tx=0).stop 2024-09-21T06:07:59.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.079+0000 7f8ce881e640 1 --2- 172.21.15.117:0/1226406570 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8ccc0b6680 0x7f8ccc0b8b90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc01f000 tx=0x7f8cdc011c60 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:59.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.079+0000 7f8cf07f5740 1 -- 172.21.15.117:0/458643244 shutdown_connections 2024-09-21T06:07:59.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.079+0000 7f8cf07f5740 1 --2- 172.21.15.117:0/458643244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f48403390 0x561f487cc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:59.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.079+0000 7f8cf07f5740 1 --2- 172.21.15.117:0/458643244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f483fb1b0 0x561f487cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:59.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.079+0000 7f8cf07f5740 1 --2- 172.21.15.117:0/458643244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f48424070 0x561f487cb650 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:59.090 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.079+0000 7f8cf07f5740 1 -- 172.21.15.117:0/458643244 >> 172.21.15.117:0/458643244 conn(0x561f48418460 msgr2=0x561f487c9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:59.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.079+0000 7f8cf07f5740 1 -- 172.21.15.117:0/458643244 shutdown_connections 2024-09-21T06:07:59.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.079+0000 7f8cf07f5740 1 -- 172.21.15.117:0/458643244 wait complete. 2024-09-21T06:07:59.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.080+0000 7f8cf07f5740 1 -- start start 2024-09-21T06:07:59.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.080+0000 7f8cf07f5740 1 Processor -- start 2024-09-21T06:07:59.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.080+0000 7f8cf07f5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f483fb1b0 0x561f487d6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:59.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.080+0000 7f8cf07f5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f48403390 0x561f487d7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:59.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.080+0000 7f8cf07f5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f48424070 0x561f487d77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:59.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.080+0000 7f8cf07f5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561f48592700 con 0x561f48424070 2024-09-21T06:07:59.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.080+0000 7f8cf07f5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561f486cc110 con 0x561f48403390 2024-09-21T06:07:59.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.080+0000 7f8cf07f5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x561f487d9e30 con 0x561f483fb1b0 2024-09-21T06:07:59.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.081+0000 7f8cf07f5740 10 client.0 _ll_register_callbacks cb 0x561f486e5990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:59.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.081+0000 7f8ce881e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f48424070 0x561f487d77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:59.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.081+0000 7f8ce881e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f48424070 0x561f487d77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48156/0 (socket says 172.21.15.117:48156) 2024-09-21T06:07:59.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.081+0000 7f8ce881e640 1 -- 172.21.15.117:0/3434131341 learned_addr learned my addr 172.21.15.117:0/3434131341 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:59.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.081+0000 7f8ce3fff640 1 --2- 172.21.15.117:0/3434131341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f483fb1b0 0x561f487d6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:59.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.081+0000 7f8ce37fe640 1 --2- 172.21.15.117:0/3434131341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f48403390 0x561f487d7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:59.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.081+0000 7f8ce881e640 1 -- 172.21.15.117:0/3434131341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f483fb1b0 msgr2=0x561f487d6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:59.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.081+0000 7f8ce881e640 1 --2- 172.21.15.117:0/3434131341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f483fb1b0 0x561f487d6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:59.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.081+0000 7f8ce881e640 1 -- 172.21.15.117:0/3434131341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f48403390 msgr2=0x561f487d7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:59.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.081+0000 7f8ce881e640 1 --2- 172.21.15.117:0/3434131341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f48403390 0x561f487d7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:59.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.081+0000 7f8ce881e640 1 -- 172.21.15.117:0/3434131341 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f486f9c40 con 0x561f48424070 2024-09-21T06:07:59.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.081+0000 7f8ce881e640 1 --2- 172.21.15.117:0/3434131341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f48424070 0x561f487d77f0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc010870 tx=0x7f8cdc01e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6bfd8117f0938cc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:59.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.081+0000 7f8cc37fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:59.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.081+0000 7f8cc37fe640 1 -- 172.21.15.117:0/3434131341 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc035060 con 0x561f48424070 2024-09-21T06:07:59.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.081+0000 7f8cc37fe640 1 -- 172.21.15.117:0/3434131341 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f8cdc011470 con 0x561f48424070 2024-09-21T06:07:59.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.081+0000 7f8cf07f5740 1 -- 172.21.15.117:0/3434131341 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cdc012c10 con 0x561f48424070 2024-09-21T06:07:59.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.081+0000 7f8cc37fe640 1 -- 172.21.15.117:0/3434131341 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8cdc022040 con 0x561f48424070 2024-09-21T06:07:59.092 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.081+0000 7f8cf07f5740 10 client.18003 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:59.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.081+0000 7f8cf07f5740 1 -- 172.21.15.117:0/3434131341 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561f487e5a50 con 0x561f48424070 2024-09-21T06:07:59.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.082+0000 7f8cc37fe640 1 -- 172.21.15.117:0/3434131341 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cdc012c10 con 0x561f48424070 2024-09-21T06:07:59.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.082+0000 7f8cc37fe640 1 -- 172.21.15.117:0/3434131341 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8cdc02f030 con 0x561f48424070 2024-09-21T06:07:59.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.082+0000 7f8cf07f5740 10 client.18003 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:59.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.082+0000 7f8cf07f5740 10 client.18003 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:07:59.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.082+0000 7f8cf07f5740 20 client.18003 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:59.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.082+0000 7f8cf07f5740 1 --2- 172.21.15.117:0/3434131341 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x561f483e29f0 0x561f487e8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:59.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.082+0000 7f8cf07f5740 4 client.18003 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:07:59.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.082+0000 7f8cf07f5740 1 -- 172.21.15.117:0/3434131341 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f48743da0 con 0x561f483e29f0 2024-09-21T06:07:59.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.082+0000 7f8ce3fff640 1 --2- 172.21.15.117:0/3434131341 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x561f483e29f0 0x561f487e8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:59.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.083+0000 7f8cc37fe640 10 client.18003 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:59.093 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.083+0000 7f8ce3fff640 1 --2- 172.21.15.117:0/3434131341 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x561f483e29f0 0x561f487e8ec0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8fef422b0a3ea31 server_cookie=16df5be368965360 in_seq=0 out_seq=0 2024-09-21T06:07:59.094 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.083+0000 7f8cc37fe640 1 -- 172.21.15.117:0/3434131341 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561f48743da0 con 0x561f483e29f0 2024-09-21T06:07:59.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.083+0000 7f8cc37fe640 10 client.18003 handle_command_reply: tid=0 2024-09-21T06:07:59.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.083+0000 7f8cf07f5740 1 client.18003 shutdown 2024-09-21T06:07:59.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.084+0000 7f8cf07f5740 1 -- 172.21.15.117:0/3434131341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f48424070 msgr2=0x561f487d77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:59.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.084+0000 7f8cf07f5740 1 --2- 172.21.15.117:0/3434131341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f48424070 0x561f487d77f0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc010870 tx=0x7f8cdc01e040 comp rx=0 tx=0).stop 2024-09-21T06:07:59.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.084+0000 7f8cf07f5740 1 -- 172.21.15.117:0/3434131341 shutdown_connections 2024-09-21T06:07:59.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.084+0000 7f8cf07f5740 1 --2- 172.21.15.117:0/3434131341 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x561f483e29f0 0x561f487e8ec0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:59.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.084+0000 7f8cf07f5740 1 --2- 172.21.15.117:0/3434131341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f48424070 0x561f487d77f0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:59.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.084+0000 7f8cf07f5740 1 --2- 172.21.15.117:0/3434131341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f48403390 0x561f487d7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:59.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.084+0000 7f8cc37fe640 0 client.18003 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:59.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.084+0000 7f8cf07f5740 1 --2- 172.21.15.117:0/3434131341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f483fb1b0 0x561f487d6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:59.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.084+0000 7f8cf07f5740 1 -- 172.21.15.117:0/3434131341 >> 172.21.15.117:0/3434131341 conn(0x561f48418460 msgr2=0x561f487ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:59.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.084+0000 7f8cf07f5740 1 -- 172.21.15.117:0/3434131341 shutdown_connections 2024-09-21T06:07:59.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.084+0000 7f8cf07f5740 1 -- 172.21.15.117:0/3434131341 wait complete. 2024-09-21T06:07:59.095 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.084+0000 7f8cf07f5740 20 client.18003 trim_cache size 0 max 16384 2024-09-21T06:07:59.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.101+0000 7f8cf07f5740 1 Processor -- start 2024-09-21T06:07:59.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.101+0000 7f8cf07f5740 1 -- start start 2024-09-21T06:07:59.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.102+0000 7f8cf07f5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f483e29f0 0x561f487d8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:59.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.102+0000 7f8cf07f5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f483fb1b0 0x561f487cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:59.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.102+0000 7f8cf07f5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f48403390 0x561f487d3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:59.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.102+0000 7f8cf07f5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce41af710 con 0x561f48403390 2024-09-21T06:07:59.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.102+0000 7f8cf07f5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce41b1250 con 0x561f483e29f0 2024-09-21T06:07:59.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.102+0000 7f8cf07f5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce41afde0 con 0x561f483fb1b0 2024-09-21T06:07:59.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.102+0000 7f8ce881e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f48403390 0x561f487d3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:59.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.102+0000 7f8ce881e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f48403390 0x561f487d3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48158/0 (socket says 172.21.15.117:48158) 2024-09-21T06:07:59.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.102+0000 7f8ce881e640 1 -- 172.21.15.117:0/2561905390 learned_addr learned my addr 172.21.15.117:0/2561905390 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:59.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.102+0000 7f8ce3fff640 1 --2- 172.21.15.117:0/2561905390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f483e29f0 0x561f487d8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:59.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.102+0000 7f8ce37fe640 1 --2- 172.21.15.117:0/2561905390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f483fb1b0 0x561f487cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:59.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.102+0000 7f8ce881e640 1 -- 172.21.15.117:0/2561905390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f483fb1b0 msgr2=0x561f487cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:59.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.102+0000 7f8ce881e640 1 --2- 172.21.15.117:0/2561905390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f483fb1b0 0x561f487cee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:59.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.102+0000 7f8ce881e640 1 -- 172.21.15.117:0/2561905390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f483e29f0 msgr2=0x561f487d8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:59.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.102+0000 7f8ce881e640 1 --2- 172.21.15.117:0/2561905390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f483e29f0 0x561f487d8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:59.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.102+0000 7f8ce881e640 1 -- 172.21.15.117:0/2561905390 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f487cd940 con 0x561f48403390 2024-09-21T06:07:59.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.102+0000 7f8ce881e640 1 --2- 172.21.15.117:0/2561905390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f48403390 0x561f487d3f40 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc009770 tx=0x7f8cdc009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff06a1769c8e6921 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:59.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.103+0000 7f8cc9ffb640 1 -- 172.21.15.117:0/2561905390 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc035070 con 0x561f48403390 2024-09-21T06:07:59.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.103+0000 7f8cc9ffb640 1 -- 172.21.15.117:0/2561905390 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cdc012b60 con 0x561f48403390 2024-09-21T06:07:59.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.103+0000 7f8cc9ffb640 1 -- 172.21.15.117:0/2561905390 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc016040 con 0x561f48403390 2024-09-21T06:07:59.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.103+0000 7f8cf07f5740 1 -- 172.21.15.117:0/2561905390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f48403390 msgr2=0x561f487d3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:59.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.103+0000 7f8cf07f5740 1 --2- 172.21.15.117:0/2561905390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f48403390 0x561f487d3f40 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc009770 tx=0x7f8cdc009ef0 comp rx=0 tx=0).stop 2024-09-21T06:07:59.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.103+0000 7f8cf07f5740 1 -- 172.21.15.117:0/2561905390 shutdown_connections 2024-09-21T06:07:59.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.103+0000 7f8cf07f5740 1 --2- 172.21.15.117:0/2561905390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f48403390 0x561f487d3f40 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:59.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.103+0000 7f8cf07f5740 1 --2- 172.21.15.117:0/2561905390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f483fb1b0 0x561f487cee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:59.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.103+0000 7f8cf07f5740 1 --2- 172.21.15.117:0/2561905390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f483e29f0 0x561f487d8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:59.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.103+0000 7f8cf07f5740 1 -- 172.21.15.117:0/2561905390 >> 172.21.15.117:0/2561905390 conn(0x561f48418460 msgr2=0x561f487ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:59.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.103+0000 7f8cf07f5740 1 -- 172.21.15.117:0/2561905390 shutdown_connections 2024-09-21T06:07:59.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.104+0000 7f8cf07f5740 1 -- 172.21.15.117:0/2561905390 wait complete. 2024-09-21T06:07:59.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.104+0000 7f8cf07f5740 1 -- start start 2024-09-21T06:07:59.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.104+0000 7f8cf07f5740 1 Processor -- start 2024-09-21T06:07:59.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.104+0000 7f8cf07f5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f483e29f0 0x561f487df6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:59.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.104+0000 7f8cf07f5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f483fb1b0 0x561f487dab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:59.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.104+0000 7f8cf07f5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f48403390 0x561f487db0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:59.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.104+0000 7f8cf07f5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce41b0b80 con 0x561f483fb1b0 2024-09-21T06:07:59.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.104+0000 7f8cf07f5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce41b04b0 con 0x561f483e29f0 2024-09-21T06:07:59.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.104+0000 7f8cf07f5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce41ad710 con 0x561f48403390 2024-09-21T06:07:59.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.104+0000 7f8cf07f5740 10 client.0 _ll_register_callbacks cb 0x7f8cdc020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:07:59.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.105+0000 7f8ce37fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f483fb1b0 0x561f487dab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:59.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.105+0000 7f8ce37fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f483fb1b0 0x561f487dab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48162/0 (socket says 172.21.15.117:48162) 2024-09-21T06:07:59.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.105+0000 7f8ce37fe640 1 -- 172.21.15.117:0/3614010228 learned_addr learned my addr 172.21.15.117:0/3614010228 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:07:59.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.105+0000 7f8ce881e640 1 --2- 172.21.15.117:0/3614010228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f48403390 0x561f487db0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:59.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.105+0000 7f8ce3fff640 1 --2- 172.21.15.117:0/3614010228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f483e29f0 0x561f487df6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:59.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.105+0000 7f8ce37fe640 1 -- 172.21.15.117:0/3614010228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f48403390 msgr2=0x561f487db0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:59.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.105+0000 7f8ce37fe640 1 --2- 172.21.15.117:0/3614010228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f48403390 0x561f487db0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:59.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.105+0000 7f8ce37fe640 1 -- 172.21.15.117:0/3614010228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f483e29f0 msgr2=0x561f487df6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:07:59.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.105+0000 7f8ce37fe640 1 --2- 172.21.15.117:0/3614010228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f483e29f0 0x561f487df6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:59.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.105+0000 7f8ce37fe640 1 -- 172.21.15.117:0/3614010228 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f487db660 con 0x561f483fb1b0 2024-09-21T06:07:59.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.105+0000 7f8ce37fe640 1 --2- 172.21.15.117:0/3614010228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f483fb1b0 0x561f487dab60 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8000c00 tx=0x7f8cd8031830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343c934e98e47912 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:07:59.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.105+0000 7f8cc37fe640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:07:59.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.105+0000 7f8cc37fe640 1 -- 172.21.15.117:0/3614010228 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd8035040 con 0x561f483fb1b0 2024-09-21T06:07:59.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.105+0000 7f8cc37fe640 1 -- 172.21.15.117:0/3614010228 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f8cd8005030 con 0x561f483fb1b0 2024-09-21T06:07:59.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.105+0000 7f8cf07f5740 1 -- 172.21.15.117:0/3614010228 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cdc012b60 con 0x561f483fb1b0 2024-09-21T06:07:59.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.105+0000 7f8cc37fe640 1 -- 172.21.15.117:0/3614010228 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8cd8005230 con 0x561f483fb1b0 2024-09-21T06:07:59.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.105+0000 7f8cf07f5740 10 client.18009 fetch_fsmap learned FSMap version 29 2024-09-21T06:07:59.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.105+0000 7f8cf07f5740 1 -- 172.21.15.117:0/3614010228 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561f488534d0 con 0x561f483fb1b0 2024-09-21T06:07:59.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.106+0000 7f8cc37fe640 1 -- 172.21.15.117:0/3614010228 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cdc012b60 con 0x561f483fb1b0 2024-09-21T06:07:59.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.106+0000 7f8cc37fe640 1 -- 172.21.15.117:0/3614010228 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8cd80035f0 con 0x561f483fb1b0 2024-09-21T06:07:59.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.106+0000 7f8cf07f5740 10 client.18009 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:07:59.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.106+0000 7f8cf07f5740 10 client.18009 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:07:59.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.106+0000 7f8cf07f5740 20 client.18009 populate_metadata read hostname 'smithi117' 2024-09-21T06:07:59.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.106+0000 7f8cf07f5740 1 --2- 172.21.15.117:0/3614010228 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x561f48424070 0x561f48855c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:07:59.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.106+0000 7f8cf07f5740 4 client.18009 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:07:59.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.106+0000 7f8cf07f5740 1 -- 172.21.15.117:0/3614010228 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561f4873f850 con 0x561f48424070 2024-09-21T06:07:59.112 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.106+0000 7f8ce3fff640 1 --2- 172.21.15.117:0/3614010228 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x561f48424070 0x561f48855c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:07:59.113 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.107+0000 7f8cc37fe640 10 client.18009 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:59.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.107+0000 7f8ce3fff640 1 --2- 172.21.15.117:0/3614010228 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x561f48424070 0x561f48855c70 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b8c338054e2549d server_cookie=baf030df72734c0c in_seq=0 out_seq=0 2024-09-21T06:07:59.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.107+0000 7f8cc37fe640 1 -- 172.21.15.117:0/3614010228 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3978 (crc 0 0 0) 0x561f4873f850 con 0x561f48424070 2024-09-21T06:07:59.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.107+0000 7f8cc37fe640 10 client.18009 handle_command_reply: tid=0 2024-09-21T06:07:59.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.107+0000 7f8cf07f5740 1 client.18009 shutdown 2024-09-21T06:07:59.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.107+0000 7f8cf07f5740 1 -- 172.21.15.117:0/3614010228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f483fb1b0 msgr2=0x561f487dab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:59.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.107+0000 7f8cf07f5740 1 --2- 172.21.15.117:0/3614010228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f483fb1b0 0x561f487dab60 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8000c00 tx=0x7f8cd8031830 comp rx=0 tx=0).stop 2024-09-21T06:07:59.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.107+0000 7f8cf07f5740 1 -- 172.21.15.117:0/3614010228 shutdown_connections 2024-09-21T06:07:59.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.107+0000 7f8cf07f5740 1 --2- 172.21.15.117:0/3614010228 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x561f48424070 0x561f48855c70 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:59.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.107+0000 7f8cf07f5740 1 --2- 172.21.15.117:0/3614010228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561f483fb1b0 0x561f487dab60 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:59.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.107+0000 7f8cf07f5740 1 --2- 172.21.15.117:0/3614010228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f48403390 0x561f487db0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:59.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.107+0000 7f8cc37fe640 0 client.18009 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:07:59.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.107+0000 7f8cf07f5740 1 --2- 172.21.15.117:0/3614010228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561f483e29f0 0x561f487df6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:59.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.107+0000 7f8cf07f5740 1 -- 172.21.15.117:0/3614010228 >> 172.21.15.117:0/3614010228 conn(0x561f48418460 msgr2=0x561f487d8990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:59.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.107+0000 7f8cf07f5740 1 -- 172.21.15.117:0/3614010228 shutdown_connections 2024-09-21T06:07:59.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.107+0000 7f8cf07f5740 1 -- 172.21.15.117:0/3614010228 wait complete. 2024-09-21T06:07:59.115 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.107+0000 7f8cf07f5740 20 client.18009 trim_cache size 0 max 16384 2024-09-21T06:07:59.115 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T06:07:59.115 INFO:teuthology.orchestra.run.smithi117.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T06:07:59.115 INFO:teuthology.orchestra.run.smithi117.stdout: "scrubs": { 2024-09-21T06:07:59.115 INFO:teuthology.orchestra.run.smithi117.stdout: "bcb8bc27-fdcf-48a3-819b-c98b933f1d97": { 2024-09-21T06:07:59.115 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:59.115 INFO:teuthology.orchestra.run.smithi117.stdout: "tag": "bcb8bc27-fdcf-48a3-819b-c98b933f1d97", 2024-09-21T06:07:59.115 INFO:teuthology.orchestra.run.smithi117.stdout: "options": "recursive,force" 2024-09-21T06:07:59.115 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:07:59.116 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:59.116 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_stats": { 2024-09-21T06:07:59.116 INFO:teuthology.orchestra.run.smithi117.stdout: "bcb8bc27-fdcf-48a3-819b-c98b933f1d97": { 2024-09-21T06:07:59.116 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:07:57.523337+0000", 2024-09-21T06:07:59.116 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:59.116 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:59.116 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:59.116 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:59.116 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:59.116 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:59.116 INFO:teuthology.orchestra.run.smithi117.stdout: "84be3416-2285-4046-adf2-8d335592d29f": { 2024-09-21T06:07:59.116 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:07:43.890616+0000", 2024-09-21T06:07:59.116 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:59.116 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:59.117 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:59.117 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:59.117 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:59.117 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:59.117 INFO:teuthology.orchestra.run.smithi117.stdout: "e99040eb-55d8-492b-ae21-f9ba41c42664": { 2024-09-21T06:07:59.117 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:07:29.750906+0000", 2024-09-21T06:07:59.117 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:59.117 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:59.117 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:59.117 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:59.117 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:59.117 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:59.117 INFO:teuthology.orchestra.run.smithi117.stdout: "26e3d90e-072a-4a4e-9661-4d004051865b": { 2024-09-21T06:07:59.117 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:06:29.219142+0000", 2024-09-21T06:07:59.117 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:59.117 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:59.117 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:59.118 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:59.118 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:59.118 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:59.118 INFO:teuthology.orchestra.run.smithi117.stdout: "cf43a1b2-838e-4d28-9ed6-ff0c726fd51a": { 2024-09-21T06:07:59.118 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:06:42.901863+0000", 2024-09-21T06:07:59.118 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:59.118 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:59.118 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:59.118 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:59.118 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:59.118 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:59.118 INFO:teuthology.orchestra.run.smithi117.stdout: "39eeca8a-6b4f-465f-af34-b54f77320d70": { 2024-09-21T06:07:59.118 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:05:41.497141+0000", 2024-09-21T06:07:59.119 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:59.119 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:59.119 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:59.119 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:59.119 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:59.119 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:59.119 INFO:teuthology.orchestra.run.smithi117.stdout: "29ff9b7c-49d9-4910-85be-bace7e940cd5": { 2024-09-21T06:07:59.119 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:03:03.775527+0000", 2024-09-21T06:07:59.119 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:59.119 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:59.119 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:59.119 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:59.119 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:59.119 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:59.119 INFO:teuthology.orchestra.run.smithi117.stdout: "0902e33a-ce33-45fb-92d8-2e6fcb8ec8a0": { 2024-09-21T06:07:59.120 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:04:11.365073+0000", 2024-09-21T06:07:59.120 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:59.120 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:59.120 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:59.120 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:59.120 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:59.120 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:59.120 INFO:teuthology.orchestra.run.smithi117.stdout: "d543b0ae-b710-4153-99da-fe5714839c1a": { 2024-09-21T06:07:59.120 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:01:20.574754+0000", 2024-09-21T06:07:59.120 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:59.120 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:59.120 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:59.120 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:59.120 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:59.121 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:59.121 INFO:teuthology.orchestra.run.smithi117.stdout: "ad80d9f1-6aff-46c0-91ab-2c1ab0681039": { 2024-09-21T06:07:59.121 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:59:31.949108+0000", 2024-09-21T06:07:59.121 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:59.121 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:59.121 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:59.121 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:59.121 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:59.121 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:59.121 INFO:teuthology.orchestra.run.smithi117.stdout: "fb2f217d-463b-4c8f-ab19-920b04bb7306": { 2024-09-21T06:07:59.121 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:04:56.308107+0000", 2024-09-21T06:07:59.121 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:59.121 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:59.121 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:59.121 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:59.121 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:59.122 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:59.122 INFO:teuthology.orchestra.run.smithi117.stdout: "8b3fe95a-b76b-4e94-afd0-72fe35e2dd92": { 2024-09-21T06:07:59.122 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:58:01.246076+0000", 2024-09-21T06:07:59.122 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:59.122 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:59.122 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:59.122 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:59.122 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:59.122 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:07:59.122 INFO:teuthology.orchestra.run.smithi117.stdout: "b30bfff0-0f6b-4e30-8467-ff7db421eba6": { 2024-09-21T06:07:59.122 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:57:14.939082+0000", 2024-09-21T06:07:59.122 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:07:59.122 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:07:59.122 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:07:59.122 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:07:59.123 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:07:59.123 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:07:59.123 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:07:59.123 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T06:07:59.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.108+0000 7f8cea281640 1 -- 172.21.15.117:0/1226406570 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8ccc0b6680 msgr2=0x7f8ccc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:59.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.108+0000 7f8cea281640 1 --2- 172.21.15.117:0/1226406570 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8ccc0b6680 0x7f8ccc0b8b90 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc01f000 tx=0x7f8cdc011c60 comp rx=0 tx=0).stop 2024-09-21T06:07:59.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.108+0000 7f8cea281640 1 -- 172.21.15.117:0/1226406570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ce41031b0 msgr2=0x7f8ce419b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:07:59.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.108+0000 7f8cea281640 1 --2- 172.21.15.117:0/1226406570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ce41031b0 0x7f8ce419b2b0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f8cd400c970 tx=0x7f8cd400ce30 comp rx=0 tx=0).stop 2024-09-21T06:07:59.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.108+0000 7f8cea281640 1 -- 172.21.15.117:0/1226406570 shutdown_connections 2024-09-21T06:07:59.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.108+0000 7f8cea281640 1 --2- 172.21.15.117:0/1226406570 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f8ccc0b6680 0x7f8ccc0b8b90 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:59.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.108+0000 7f8cea281640 1 --2- 172.21.15.117:0/1226406570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8ce4105b10 0x7f8ce419bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:59.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.108+0000 7f8cea281640 1 --2- 172.21.15.117:0/1226406570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ce41051c0 0x7f8ce419b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:59.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.108+0000 7f8cea281640 1 --2- 172.21.15.117:0/1226406570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ce41031b0 0x7f8ce419b2b0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:07:59.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.108+0000 7f8cea281640 1 -- 172.21.15.117:0/1226406570 >> 172.21.15.117:0/1226406570 conn(0x7f8ce4079e40 msgr2=0x7f8ce4106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:07:59.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.108+0000 7f8cea281640 1 -- 172.21.15.117:0/1226406570 shutdown_connections 2024-09-21T06:07:59.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:07:59.108+0000 7f8cea281640 1 -- 172.21.15.117:0/1226406570 wait complete. 2024-09-21T06:07:59.948 INFO:tasks.cephfs.filesystem:scrub status for tag:bcb8bc27-fdcf-48a3-819b-c98b933f1d97 - {'path': '/', 'tag': 'bcb8bc27-fdcf-48a3-819b-c98b933f1d97', 'options': 'recursive,force'} 2024-09-21T06:08:00.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:07:59 smithi117 ceph-mon[30947]: pgmap v654: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 170 B/s wr, 14 op/s 2024-09-21T06:08:00.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:07:59 smithi138 ceph-mon[34391]: pgmap v654: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 170 B/s wr, 14 op/s 2024-09-21T06:08:00.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:07:59 smithi192 ceph-mon[34700]: pgmap v654: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 170 B/s wr, 14 op/s 2024-09-21T06:08:01.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:00 smithi192 ceph-mon[34700]: scrub summary: idle 2024-09-21T06:08:01.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:00 smithi117 ceph-mon[30947]: scrub summary: idle 2024-09-21T06:08:01.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:00 smithi138 ceph-mon[34391]: scrub summary: idle 2024-09-21T06:08:02.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:01 smithi192 ceph-mon[34700]: pgmap v655: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-21T06:08:02.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:01 smithi117 ceph-mon[30947]: pgmap v655: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-21T06:08:02.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:01 smithi138 ceph-mon[34391]: pgmap v655: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-21T06:08:04.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:03 smithi192 ceph-mon[34700]: pgmap v656: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T06:08:04.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:03 smithi117 ceph-mon[30947]: pgmap v656: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T06:08:04.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:03 smithi138 ceph-mon[34391]: pgmap v656: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-21T06:08:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.792+0000 7f83bf8a9640 1 -- 172.21.15.117:0/760767545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83b81082e0 msgr2=0x7f83b81086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.792+0000 7f83bf8a9640 1 --2- 172.21.15.117:0/760767545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83b81082e0 0x7f83b81086f0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f83a8009900 tx=0x7f83a802a410 comp rx=0 tx=0).stop 2024-09-21T06:08:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.792+0000 7f83bf8a9640 1 -- 172.21.15.117:0/760767545 shutdown_connections 2024-09-21T06:08:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.792+0000 7f83bf8a9640 1 --2- 172.21.15.117:0/760767545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83b81082e0 0x7f83b81086f0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.792+0000 7f83bf8a9640 1 --2- 172.21.15.117:0/760767545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f83b8108c30 0x7f83b810f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.792+0000 7f83bf8a9640 1 --2- 172.21.15.117:0/760767545 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83b81029b0 0x7f83b8102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.792+0000 7f83bf8a9640 1 -- 172.21.15.117:0/760767545 >> 172.21.15.117:0/760767545 conn(0x7f83b807a740 msgr2=0x7f83b807ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.793+0000 7f83bf8a9640 1 -- 172.21.15.117:0/760767545 shutdown_connections 2024-09-21T06:08:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.793+0000 7f83bf8a9640 1 -- 172.21.15.117:0/760767545 wait complete. 2024-09-21T06:08:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.793+0000 7f83bf8a9640 1 Processor -- start 2024-09-21T06:08:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.793+0000 7f83bf8a9640 1 -- start start 2024-09-21T06:08:04.798 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.794+0000 7f83bf8a9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83b81029b0 0x7f83b819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:04.798 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.794+0000 7f83bf8a9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83b81082e0 0x7f83b819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:04.798 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.794+0000 7f83bf8a9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f83b8108c30 0x7f83b819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:04.798 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.794+0000 7f83bf8a9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f83b8113a10 con 0x7f83b81029b0 2024-09-21T06:08:04.798 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.794+0000 7f83bf8a9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f83b8113890 con 0x7f83b81082e0 2024-09-21T06:08:04.798 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.794+0000 7f83bf8a9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f83b8113b90 con 0x7f83b8108c30 2024-09-21T06:08:04.798 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.794+0000 7f83bd645640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83b81029b0 0x7f83b819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:04.798 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.794+0000 7f83bd645640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83b81029b0 0x7f83b819a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54930/0 (socket says 172.21.15.117:54930) 2024-09-21T06:08:04.798 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.794+0000 7f83bd645640 1 -- 172.21.15.117:0/3004750998 learned_addr learned my addr 172.21.15.117:0/3004750998 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:04.798 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.794+0000 7f83bde46640 1 --2- 172.21.15.117:0/3004750998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f83b8108c30 0x7f83b819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:04.798 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.794+0000 7f83bce44640 1 --2- 172.21.15.117:0/3004750998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83b81082e0 0x7f83b819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.794+0000 7f83bd645640 1 -- 172.21.15.117:0/3004750998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f83b8108c30 msgr2=0x7f83b819fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.794+0000 7f83bd645640 1 --2- 172.21.15.117:0/3004750998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f83b8108c30 0x7f83b819fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.794+0000 7f83bd645640 1 -- 172.21.15.117:0/3004750998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83b81082e0 msgr2=0x7f83b819aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.794+0000 7f83bd645640 1 --2- 172.21.15.117:0/3004750998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83b81082e0 0x7f83b819aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.794+0000 7f83bd645640 1 -- 172.21.15.117:0/3004750998 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b81a03c0 con 0x7f83b81029b0 2024-09-21T06:08:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.794+0000 7f83bd645640 1 --2- 172.21.15.117:0/3004750998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83b81029b0 0x7f83b819a490 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f83b400e9f0 tx=0x7f83b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.794+0000 7f83ae7fc640 1 -- 172.21.15.117:0/3004750998 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b400cd80 con 0x7f83b81029b0 2024-09-21T06:08:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.794+0000 7f83ae7fc640 1 -- 172.21.15.117:0/3004750998 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83b40108e0 con 0x7f83b81029b0 2024-09-21T06:08:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.794+0000 7f83ae7fc640 1 -- 172.21.15.117:0/3004750998 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b4018980 con 0x7f83b81029b0 2024-09-21T06:08:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.795+0000 7f83bf8a9640 1 -- 172.21.15.117:0/3004750998 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83b81a06b0 con 0x7f83b81029b0 2024-09-21T06:08:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.795+0000 7f83bf8a9640 1 -- 172.21.15.117:0/3004750998 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83b81a6e40 con 0x7f83b81029b0 2024-09-21T06:08:04.806 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.797+0000 7f83ae7fc640 1 -- 172.21.15.117:0/3004750998 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f83b4010430 con 0x7f83b81029b0 2024-09-21T06:08:04.806 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.797+0000 7f83c681b740 1 Processor -- start 2024-09-21T06:08:04.806 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.797+0000 7f83c681b740 1 -- start start 2024-09-21T06:08:04.806 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.798+0000 7f83c681b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d775bc1b0 0x559d779619b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:04.806 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.798+0000 7f83c681b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d775932f0 0x559d77961ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:04.807 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.798+0000 7f83c681b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d7759b4d0 0x559d77962430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:04.807 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.798+0000 7f83c681b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559d778d66f0 con 0x559d775bc1b0 2024-09-21T06:08:04.807 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.798+0000 7f83c681b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559d778c5b20 con 0x559d7759b4d0 2024-09-21T06:08:04.807 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.798+0000 7f83c681b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x559d778c9930 con 0x559d775932f0 2024-09-21T06:08:04.807 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.798+0000 7f83bde46640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d775bc1b0 0x559d779619b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:04.807 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.798+0000 7f83bde46640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d775bc1b0 0x559d779619b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54944/0 (socket says 172.21.15.117:54944) 2024-09-21T06:08:04.807 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.798+0000 7f83bde46640 1 -- 172.21.15.117:0/26368556 learned_addr learned my addr 172.21.15.117:0/26368556 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:04.807 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.798+0000 7f83bce44640 1 --2- 172.21.15.117:0/26368556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d7759b4d0 0x559d77962430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:04.807 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.798+0000 7f83bd645640 1 --2- 172.21.15.117:0/26368556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d775932f0 0x559d77961ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:04.807 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.798+0000 7f83ae7fc640 1 --2- 172.21.15.117:0/3004750998 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f83a40b6700 0x7f83a40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:04.807 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.798+0000 7f83bde46640 1 -- 172.21.15.117:0/26368556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d775932f0 msgr2=0x559d77961ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:04.807 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.798+0000 7f83bde46640 1 --2- 172.21.15.117:0/26368556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d775932f0 0x559d77961ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.808 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.798+0000 7f83bde46640 1 -- 172.21.15.117:0/26368556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d7759b4d0 msgr2=0x559d77962430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:04.808 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.798+0000 7f83bde46640 1 --2- 172.21.15.117:0/26368556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d7759b4d0 0x559d77962430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.808 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.798+0000 7f83bde46640 1 -- 172.21.15.117:0/26368556 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d7788b530 con 0x559d775bc1b0 2024-09-21T06:08:04.808 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.798+0000 7f83bd645640 1 --2- 172.21.15.117:0/26368556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d775932f0 0x559d77961ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:08:04.808 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.798+0000 7f83ae7fc640 1 -- 172.21.15.117:0/3004750998 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f83b40e7a30 con 0x7f83b81029b0 2024-09-21T06:08:04.808 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.798+0000 7f83bde46640 1 --2- 172.21.15.117:0/3004750998 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f83a40b6700 0x7f83a40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:04.808 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.799+0000 7f83bde46640 1 --2- 172.21.15.117:0/26368556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d775bc1b0 0x559d779619b0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f83b0009f50 tx=0x7f83b00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2718bbe4f82888 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:04.808 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.799+0000 7f83a37fe640 1 -- 172.21.15.117:0/26368556 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b0004250 con 0x559d775bc1b0 2024-09-21T06:08:04.808 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.799+0000 7f83a37fe640 1 -- 172.21.15.117:0/26368556 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83b000fe00 con 0x559d775bc1b0 2024-09-21T06:08:04.808 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.799+0000 7f83a37fe640 1 -- 172.21.15.117:0/26368556 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b001ba60 con 0x559d775bc1b0 2024-09-21T06:08:04.808 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.800+0000 7f83c681b740 1 -- 172.21.15.117:0/26368556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d775bc1b0 msgr2=0x559d779619b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:04.808 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.800+0000 7f83bde46640 1 --2- 172.21.15.117:0/3004750998 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f83a40b6700 0x7f83a40b8c10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f83b0012fd0 tx=0x7f83b0021000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:04.809 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.800+0000 7f83c681b740 1 --2- 172.21.15.117:0/26368556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d775bc1b0 0x559d779619b0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f83b0009f50 tx=0x7f83b00118d0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.809 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.800+0000 7f83c681b740 1 -- 172.21.15.117:0/26368556 shutdown_connections 2024-09-21T06:08:04.809 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.800+0000 7f83c681b740 1 --2- 172.21.15.117:0/26368556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d7759b4d0 0x559d77962430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.809 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.800+0000 7f83c681b740 1 --2- 172.21.15.117:0/26368556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d775932f0 0x559d77961ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.809 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.800+0000 7f83c681b740 1 --2- 172.21.15.117:0/26368556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d775bc1b0 0x559d779619b0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.809 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.800+0000 7f83c681b740 1 -- 172.21.15.117:0/26368556 >> 172.21.15.117:0/26368556 conn(0x559d775b0300 msgr2=0x559d7795f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:04.809 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.800+0000 7f83c681b740 1 -- 172.21.15.117:0/26368556 shutdown_connections 2024-09-21T06:08:04.809 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.800+0000 7f83c681b740 1 -- 172.21.15.117:0/26368556 wait complete. 2024-09-21T06:08:04.809 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.800+0000 7f83c681b740 1 -- start start 2024-09-21T06:08:04.809 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.800+0000 7f83c681b740 1 Processor -- start 2024-09-21T06:08:04.810 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.801+0000 7f83c681b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d775932f0 0x559d7796d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:04.810 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.801+0000 7f83c681b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d7759b4d0 0x559d7796d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:04.810 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.801+0000 7f83c681b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d775bc1b0 0x559d77979450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:04.810 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.801+0000 7f83c681b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559d777fdab0 con 0x559d775bc1b0 2024-09-21T06:08:04.810 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.801+0000 7f83c681b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559d7786b730 con 0x559d7759b4d0 2024-09-21T06:08:04.810 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.801+0000 7f83c681b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x559d77970130 con 0x559d775932f0 2024-09-21T06:08:04.810 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.801+0000 7f83c681b740 10 client.0 _ll_register_callbacks cb 0x559d778bf6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:08:04.810 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.801+0000 7f83bde46640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d775bc1b0 0x559d77979450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:04.810 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.801+0000 7f83bde46640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d775bc1b0 0x559d77979450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54958/0 (socket says 172.21.15.117:54958) 2024-09-21T06:08:04.810 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.801+0000 7f83bde46640 1 -- 172.21.15.117:0/1446809706 learned_addr learned my addr 172.21.15.117:0/1446809706 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:04.810 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.801+0000 7f83bd645640 1 --2- 172.21.15.117:0/1446809706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d775932f0 0x559d7796d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.801+0000 7f83bce44640 1 --2- 172.21.15.117:0/1446809706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d7759b4d0 0x559d7796d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.801+0000 7f83bde46640 1 -- 172.21.15.117:0/1446809706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d775932f0 msgr2=0x559d7796d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.801+0000 7f83bde46640 1 --2- 172.21.15.117:0/1446809706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d775932f0 0x559d7796d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.801+0000 7f83bde46640 1 -- 172.21.15.117:0/1446809706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d7759b4d0 msgr2=0x559d7796d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.801+0000 7f83bde46640 1 --2- 172.21.15.117:0/1446809706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d7759b4d0 0x559d7796d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.801+0000 7f83bde46640 1 -- 172.21.15.117:0/1446809706 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d77627790 con 0x559d775bc1b0 2024-09-21T06:08:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.801+0000 7f83bce44640 1 --2- 172.21.15.117:0/1446809706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d7759b4d0 0x559d7796d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:08:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.802+0000 7f83bde46640 1 --2- 172.21.15.117:0/1446809706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d775bc1b0 0x559d77979450 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f83b001b660 tx=0x7f83b000ffb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875176d3cd5ce9fb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.802+0000 7f83a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:08:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.802+0000 7f83a0ff9640 1 -- 172.21.15.117:0/1446809706 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b0033020 con 0x559d775bc1b0 2024-09-21T06:08:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.802+0000 7f83a0ff9640 1 -- 172.21.15.117:0/1446809706 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f83b000f8d0 con 0x559d775bc1b0 2024-09-21T06:08:04.811 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.802+0000 7f83c681b740 1 -- 172.21.15.117:0/1446809706 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83b000fe00 con 0x559d775bc1b0 2024-09-21T06:08:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.802+0000 7f83a0ff9640 1 -- 172.21.15.117:0/1446809706 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f83b003e050 con 0x559d775bc1b0 2024-09-21T06:08:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.802+0000 7f83c681b740 10 client.18021 fetch_fsmap learned FSMap version 29 2024-09-21T06:08:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.802+0000 7f83c681b740 1 -- 172.21.15.117:0/1446809706 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559d776a0620 con 0x559d775bc1b0 2024-09-21T06:08:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.802+0000 7f83a0ff9640 1 -- 172.21.15.117:0/1446809706 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83b000fe00 con 0x559d775bc1b0 2024-09-21T06:08:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.802+0000 7f83a0ff9640 1 -- 172.21.15.117:0/1446809706 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f83b0025bc0 con 0x559d775bc1b0 2024-09-21T06:08:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.802+0000 7f83c681b740 10 client.18021 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:08:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.802+0000 7f83c681b740 10 client.18021 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:08:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.802+0000 7f83c681b740 20 client.18021 populate_metadata read hostname 'smithi117' 2024-09-21T06:08:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.802+0000 7f83c681b740 1 --2- 172.21.15.117:0/1446809706 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x559d7757ad80 0x559d7797f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.802+0000 7f83c681b740 4 client.18021 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:08:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.802+0000 7f83c681b740 1 -- 172.21.15.117:0/1446809706 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d778d8640 con 0x559d7757ad80 2024-09-21T06:08:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.803+0000 7f83bd645640 1 --2- 172.21.15.117:0/1446809706 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x559d7757ad80 0x559d7797f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.803+0000 7f83a0ff9640 10 client.18021 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:08:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.803+0000 7f83bd645640 1 --2- 172.21.15.117:0/1446809706 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x559d7757ad80 0x559d7797f080 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fe46dde7c41f65d server_cookie=66532f55a546206b in_seq=0 out_seq=0 2024-09-21T06:08:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.804+0000 7f83a0ff9640 1 -- 172.21.15.117:0/1446809706 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559d778d8640 con 0x559d7757ad80 2024-09-21T06:08:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.804+0000 7f83a0ff9640 10 client.18021 handle_command_reply: tid=0 2024-09-21T06:08:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.804+0000 7f83c681b740 1 client.18021 shutdown 2024-09-21T06:08:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.804+0000 7f83c681b740 1 -- 172.21.15.117:0/1446809706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d775bc1b0 msgr2=0x559d77979450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.804+0000 7f83c681b740 1 --2- 172.21.15.117:0/1446809706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d775bc1b0 0x559d77979450 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f83b001b660 tx=0x7f83b000ffb0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.804+0000 7f83c681b740 1 -- 172.21.15.117:0/1446809706 shutdown_connections 2024-09-21T06:08:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.804+0000 7f83c681b740 1 --2- 172.21.15.117:0/1446809706 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x559d7757ad80 0x559d7797f080 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.804+0000 7f83c681b740 1 --2- 172.21.15.117:0/1446809706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d775bc1b0 0x559d77979450 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.804+0000 7f83c681b740 1 --2- 172.21.15.117:0/1446809706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d7759b4d0 0x559d7796d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.813 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.804+0000 7f83a0ff9640 0 client.18021 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:08:04.814 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.804+0000 7f83c681b740 1 --2- 172.21.15.117:0/1446809706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d775932f0 0x559d7796d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.814 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.804+0000 7f83c681b740 1 -- 172.21.15.117:0/1446809706 >> 172.21.15.117:0/1446809706 conn(0x559d775b0300 msgr2=0x559d779629e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:04.814 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.804+0000 7f83c681b740 1 -- 172.21.15.117:0/1446809706 shutdown_connections 2024-09-21T06:08:04.814 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.804+0000 7f83c681b740 1 -- 172.21.15.117:0/1446809706 wait complete. 2024-09-21T06:08:04.814 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.804+0000 7f83c681b740 20 client.18021 trim_cache size 0 max 16384 2024-09-21T06:08:04.826 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.819+0000 7f83c681b740 1 Processor -- start 2024-09-21T06:08:04.826 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.819+0000 7f83c681b740 1 -- start start 2024-09-21T06:08:04.826 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.819+0000 7f83c681b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d7757ad80 0x559d77968830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:04.826 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.819+0000 7f83c681b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d775932f0 0x559d7795f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:04.826 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.819+0000 7f83c681b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d7759b4d0 0x559d77979350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:04.826 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.819+0000 7f83c681b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f83b81af600 con 0x559d7757ad80 2024-09-21T06:08:04.826 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.819+0000 7f83c681b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f83b81b1140 con 0x559d7759b4d0 2024-09-21T06:08:04.826 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.819+0000 7f83c681b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f83b81afcd0 con 0x559d775932f0 2024-09-21T06:08:04.826 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.819+0000 7f83bd645640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d7757ad80 0x559d77968830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:04.827 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.819+0000 7f83bd645640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d7757ad80 0x559d77968830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54972/0 (socket says 172.21.15.117:54972) 2024-09-21T06:08:04.827 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.819+0000 7f83bd645640 1 -- 172.21.15.117:0/1897768544 learned_addr learned my addr 172.21.15.117:0/1897768544 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:04.827 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.819+0000 7f83bde46640 1 --2- 172.21.15.117:0/1897768544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d7759b4d0 0x559d77979350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:04.827 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.819+0000 7f83bd645640 1 -- 172.21.15.117:0/1897768544 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d775932f0 msgr2=0x559d7795f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:04.827 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.819+0000 7f83bd645640 1 --2- 172.21.15.117:0/1897768544 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d775932f0 0x559d7795f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.827 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.819+0000 7f83bd645640 1 -- 172.21.15.117:0/1897768544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d7759b4d0 msgr2=0x559d77979350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:04.827 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.819+0000 7f83bd645640 1 --2- 172.21.15.117:0/1897768544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d7759b4d0 0x559d77979350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.827 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.819+0000 7f83bd645640 1 -- 172.21.15.117:0/1897768544 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d7796fcc0 con 0x559d7757ad80 2024-09-21T06:08:04.827 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.819+0000 7f83bde46640 1 --2- 172.21.15.117:0/1897768544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d7759b4d0 0x559d77979350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:08:04.827 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.820+0000 7f83bd645640 1 --2- 172.21.15.117:0/1897768544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d7757ad80 0x559d77968830 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f83b40e5230 tx=0x7f83b400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c843a90886c49c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:04.827 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.820+0000 7f83a2ffd640 1 -- 172.21.15.117:0/1897768544 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b40ad3e0 con 0x559d7757ad80 2024-09-21T06:08:04.827 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.820+0000 7f83a2ffd640 1 -- 172.21.15.117:0/1897768544 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83b40adfd0 con 0x559d7757ad80 2024-09-21T06:08:04.827 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.820+0000 7f83a2ffd640 1 -- 172.21.15.117:0/1897768544 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b40a9220 con 0x559d7757ad80 2024-09-21T06:08:04.827 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.820+0000 7f83c681b740 1 -- 172.21.15.117:0/1897768544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d7757ad80 msgr2=0x559d77968830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:04.828 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.820+0000 7f83c681b740 1 --2- 172.21.15.117:0/1897768544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d7757ad80 0x559d77968830 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f83b40e5230 tx=0x7f83b400cf20 comp rx=0 tx=0).stop 2024-09-21T06:08:04.828 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.821+0000 7f83c681b740 1 -- 172.21.15.117:0/1897768544 shutdown_connections 2024-09-21T06:08:04.828 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.821+0000 7f83c681b740 1 --2- 172.21.15.117:0/1897768544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d7759b4d0 0x559d77979350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.828 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.821+0000 7f83c681b740 1 --2- 172.21.15.117:0/1897768544 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d775932f0 0x559d7795f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.828 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.821+0000 7f83c681b740 1 --2- 172.21.15.117:0/1897768544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d7757ad80 0x559d77968830 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.828 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.821+0000 7f83c681b740 1 -- 172.21.15.117:0/1897768544 >> 172.21.15.117:0/1897768544 conn(0x559d775b0300 msgr2=0x559d7795f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:04.828 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.821+0000 7f83c681b740 1 -- 172.21.15.117:0/1897768544 shutdown_connections 2024-09-21T06:08:04.828 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.821+0000 7f83c681b740 1 -- 172.21.15.117:0/1897768544 wait complete. 2024-09-21T06:08:04.828 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.821+0000 7f83c681b740 1 -- start start 2024-09-21T06:08:04.828 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.821+0000 7f83c681b740 1 Processor -- start 2024-09-21T06:08:04.828 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.821+0000 7f83c681b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d7757ad80 0x559d779e1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:04.828 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.822+0000 7f83c681b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d775932f0 0x559d779e2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:04.828 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.822+0000 7f83c681b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d7759b4d0 0x559d779e29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:04.828 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.822+0000 7f83c681b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f83b81b0a70 con 0x559d7759b4d0 2024-09-21T06:08:04.829 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.822+0000 7f83c681b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f83b81b03a0 con 0x559d775932f0 2024-09-21T06:08:04.829 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.822+0000 7f83c681b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f83b81ad600 con 0x559d7757ad80 2024-09-21T06:08:04.829 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.822+0000 7f83c681b740 10 client.0 _ll_register_callbacks cb 0x7f83b4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:08:04.829 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.822+0000 7f83bde46640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d7759b4d0 0x559d779e29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:04.829 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.822+0000 7f83bde46640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d7759b4d0 0x559d779e29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54982/0 (socket says 172.21.15.117:54982) 2024-09-21T06:08:04.829 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.822+0000 7f83bde46640 1 -- 172.21.15.117:0/792550825 learned_addr learned my addr 172.21.15.117:0/792550825 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:04.829 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.822+0000 7f83bce44640 1 --2- 172.21.15.117:0/792550825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d775932f0 0x559d779e2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:04.829 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.822+0000 7f83bd645640 1 --2- 172.21.15.117:0/792550825 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d7757ad80 0x559d779e1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:04.829 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.822+0000 7f83bde46640 1 -- 172.21.15.117:0/792550825 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d7757ad80 msgr2=0x559d779e1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:04.829 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.822+0000 7f83bde46640 1 --2- 172.21.15.117:0/792550825 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d7757ad80 0x559d779e1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.829 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.822+0000 7f83bde46640 1 -- 172.21.15.117:0/792550825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d775932f0 msgr2=0x559d779e2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:04.829 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.822+0000 7f83bde46640 1 --2- 172.21.15.117:0/792550825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d775932f0 0x559d779e2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.829 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.822+0000 7f83bde46640 1 -- 172.21.15.117:0/792550825 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d779743a0 con 0x559d7759b4d0 2024-09-21T06:08:04.829 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.822+0000 7f83bde46640 1 --2- 172.21.15.117:0/792550825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d7759b4d0 0x559d779e29d0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f83b0005c40 tx=0x7f83b0009690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4182ff10f8647610 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.822+0000 7f83a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:08:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.822+0000 7f83a0ff9640 1 -- 172.21.15.117:0/792550825 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b0046020 con 0x559d7759b4d0 2024-09-21T06:08:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.822+0000 7f83c681b740 1 -- 172.21.15.117:0/792550825 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83b40adfd0 con 0x559d7759b4d0 2024-09-21T06:08:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.822+0000 7f83a0ff9640 1 -- 172.21.15.117:0/792550825 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8441+0+0 (secure 0 0 0) 0x7f83b001b430 con 0x559d7759b4d0 2024-09-21T06:08:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.823+0000 7f83a0ff9640 1 -- 172.21.15.117:0/792550825 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f83b0033070 con 0x559d7759b4d0 2024-09-21T06:08:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.823+0000 7f83c681b740 10 client.18027 fetch_fsmap learned FSMap version 29 2024-09-21T06:08:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.823+0000 7f83c681b740 1 -- 172.21.15.117:0/792550825 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559d77975580 con 0x559d7759b4d0 2024-09-21T06:08:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.823+0000 7f83a0ff9640 1 -- 172.21.15.117:0/792550825 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83b40adfd0 con 0x559d7759b4d0 2024-09-21T06:08:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.823+0000 7f83a0ff9640 1 -- 172.21.15.117:0/792550825 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f83b0025430 con 0x559d7759b4d0 2024-09-21T06:08:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.823+0000 7f83c681b740 10 client.18027 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:08:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.823+0000 7f83c681b740 10 client.18027 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:08:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.823+0000 7f83c681b740 20 client.18027 populate_metadata read hostname 'smithi117' 2024-09-21T06:08:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.823+0000 7f83c681b740 1 --2- 172.21.15.117:0/792550825 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x559d775bc1b0 0x559d77975d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:04.830 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.823+0000 7f83c681b740 4 client.18027 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:08:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.823+0000 7f83c681b740 1 -- 172.21.15.117:0/792550825 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559d778d40f0 con 0x559d775bc1b0 2024-09-21T06:08:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.823+0000 7f83bd645640 1 --2- 172.21.15.117:0/792550825 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x559d775bc1b0 0x559d77975d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.824+0000 7f83a0ff9640 10 client.18027 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:08:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.824+0000 7f83bd645640 1 --2- 172.21.15.117:0/792550825 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x559d775bc1b0 0x559d77975d80 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=994d15d8427882e0 server_cookie=458b7a32f9dab9ad in_seq=0 out_seq=0 2024-09-21T06:08:04.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.831+0000 7f83a0ff9640 1 -- 172.21.15.117:0/792550825 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559d778d40f0 con 0x559d775bc1b0 2024-09-21T06:08:04.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.831+0000 7f83a0ff9640 10 client.18027 handle_command_reply: tid=0 2024-09-21T06:08:04.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.831+0000 7f83c681b740 1 client.18027 shutdown 2024-09-21T06:08:04.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.831+0000 7f83c681b740 1 -- 172.21.15.117:0/792550825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d7759b4d0 msgr2=0x559d779e29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:04.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.831+0000 7f83c681b740 1 --2- 172.21.15.117:0/792550825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d7759b4d0 0x559d779e29d0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f83b0005c40 tx=0x7f83b0009690 comp rx=0 tx=0).stop 2024-09-21T06:08:04.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.831+0000 7f83c681b740 1 -- 172.21.15.117:0/792550825 shutdown_connections 2024-09-21T06:08:04.833 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.831+0000 7f83c681b740 1 --2- 172.21.15.117:0/792550825 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x559d775bc1b0 0x559d77975d80 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.834 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.831+0000 7f83c681b740 1 --2- 172.21.15.117:0/792550825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559d7759b4d0 0x559d779e29d0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.834 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.831+0000 7f83c681b740 1 --2- 172.21.15.117:0/792550825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559d775932f0 0x559d779e2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.834 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.831+0000 7f83a0ff9640 0 client.18027 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:08:04.834 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.831+0000 7f83c681b740 1 --2- 172.21.15.117:0/792550825 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559d7757ad80 0x559d779e1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.834 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.831+0000 7f83c681b740 1 -- 172.21.15.117:0/792550825 >> 172.21.15.117:0/792550825 conn(0x559d775b0300 msgr2=0x559d779628e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:04.834 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.831+0000 7f83c681b740 1 -- 172.21.15.117:0/792550825 shutdown_connections 2024-09-21T06:08:04.834 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.831+0000 7f83c681b740 1 -- 172.21.15.117:0/792550825 wait complete. 2024-09-21T06:08:04.834 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.831+0000 7f83c681b740 20 client.18027 trim_cache size 0 max 16384 2024-09-21T06:08:04.834 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T06:08:04.834 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T06:08:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T06:08:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T06:08:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.832+0000 7f83bf8a9640 1 -- 172.21.15.117:0/3004750998 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f83a40b6700 msgr2=0x7f83a40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.832+0000 7f83bf8a9640 1 --2- 172.21.15.117:0/3004750998 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f83a40b6700 0x7f83a40b8c10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f83b0012fd0 tx=0x7f83b0021000 comp rx=0 tx=0).stop 2024-09-21T06:08:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.832+0000 7f83bf8a9640 1 -- 172.21.15.117:0/3004750998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83b81029b0 msgr2=0x7f83b819a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.832+0000 7f83bf8a9640 1 --2- 172.21.15.117:0/3004750998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83b81029b0 0x7f83b819a490 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f83b400e9f0 tx=0x7f83b400eeb0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.832+0000 7f83bf8a9640 1 -- 172.21.15.117:0/3004750998 shutdown_connections 2024-09-21T06:08:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.832+0000 7f83bf8a9640 1 --2- 172.21.15.117:0/3004750998 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f83a40b6700 0x7f83a40b8c10 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.832+0000 7f83bf8a9640 1 --2- 172.21.15.117:0/3004750998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f83b8108c30 0x7f83b819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.832+0000 7f83bf8a9640 1 --2- 172.21.15.117:0/3004750998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83b81082e0 0x7f83b819aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.832+0000 7f83bf8a9640 1 --2- 172.21.15.117:0/3004750998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83b81029b0 0x7f83b819a490 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.832+0000 7f83bf8a9640 1 -- 172.21.15.117:0/3004750998 >> 172.21.15.117:0/3004750998 conn(0x7f83b807a740 msgr2=0x7f83b81043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.832+0000 7f83bf8a9640 1 -- 172.21.15.117:0/3004750998 shutdown_connections 2024-09-21T06:08:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:04.832+0000 7f83bf8a9640 1 -- 172.21.15.117:0/3004750998 wait complete. 2024-09-21T06:08:06.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:05 smithi192 ceph-mon[34700]: pgmap v657: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 45 KiB/s wr, 31 op/s 2024-09-21T06:08:06.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:05 smithi117 ceph-mon[30947]: pgmap v657: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 45 KiB/s wr, 31 op/s 2024-09-21T06:08:06.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:05 smithi138 ceph-mon[34391]: pgmap v657: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 45 KiB/s wr, 31 op/s 2024-09-21T06:08:08.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:07 smithi192 ceph-mon[34700]: pgmap v658: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-09-21T06:08:08.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:07 smithi117 ceph-mon[30947]: pgmap v658: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-09-21T06:08:08.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:07 smithi138 ceph-mon[34391]: pgmap v658: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-09-21T06:08:10.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:09 smithi192 ceph-mon[34700]: pgmap v659: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 45 KiB/s wr, 31 op/s 2024-09-21T06:08:10.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:09 smithi117 ceph-mon[30947]: pgmap v659: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 45 KiB/s wr, 31 op/s 2024-09-21T06:08:10.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:09 smithi138 ceph-mon[34391]: pgmap v659: 97 pgs: 97 active+clean; 40 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 45 KiB/s wr, 31 op/s 2024-09-21T06:08:11.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:10 smithi117 ceph-mon[30947]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T06:08:11.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:10 smithi138 ceph-mon[34391]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T06:08:11.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:10 smithi192 ceph-mon[34700]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T06:08:12.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:11 smithi117 ceph-mon[30947]: pgmap v661: 97 pgs: 3 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 83 active+clean; 39 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 54 KiB/s wr, 32 op/s 2024-09-21T06:08:12.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:11 smithi138 ceph-mon[34391]: pgmap v661: 97 pgs: 3 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 83 active+clean; 39 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 54 KiB/s wr, 32 op/s 2024-09-21T06:08:12.479 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:11 smithi192 ceph-mon[34700]: pgmap v661: 97 pgs: 3 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 83 active+clean; 39 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 54 KiB/s wr, 32 op/s 2024-09-21T06:08:14.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:13 smithi117 ceph-mon[30947]: pgmap v662: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 36 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-21T06:08:14.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:13 smithi138 ceph-mon[34391]: pgmap v662: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 36 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-21T06:08:14.479 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:13 smithi192 ceph-mon[34700]: pgmap v662: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 36 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-21T06:08:16.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:15 smithi117 ceph-mon[30947]: pgmap v663: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 31 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 818 B/s wr, 10 op/s 2024-09-21T06:08:16.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:15 smithi138 ceph-mon[34391]: pgmap v663: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 31 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 818 B/s wr, 10 op/s 2024-09-21T06:08:16.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:15 smithi192 ceph-mon[34700]: pgmap v663: 97 pgs: 4 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 82 active+clean; 31 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 818 B/s wr, 10 op/s 2024-09-21T06:08:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.941+0000 7f9853686640 1 -- 172.21.15.117:0/2504377394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f984c1029b0 msgr2=0x7f984c102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.941+0000 7f9853686640 1 --2- 172.21.15.117:0/2504377394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f984c1029b0 0x7f984c102dc0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f983c009ca0 tx=0x7f983c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.941+0000 7f9853686640 1 -- 172.21.15.117:0/2504377394 shutdown_connections 2024-09-21T06:08:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.941+0000 7f9853686640 1 --2- 172.21.15.117:0/2504377394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f984c108c30 0x7f984c10f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.941+0000 7f9853686640 1 --2- 172.21.15.117:0/2504377394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f984c1082e0 0x7f984c1086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.941+0000 7f9853686640 1 --2- 172.21.15.117:0/2504377394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f984c1029b0 0x7f984c102dc0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.941+0000 7f9853686640 1 -- 172.21.15.117:0/2504377394 >> 172.21.15.117:0/2504377394 conn(0x7f984c07a740 msgr2=0x7f984c07ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.941+0000 7f9853686640 1 -- 172.21.15.117:0/2504377394 shutdown_connections 2024-09-21T06:08:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.941+0000 7f9853686640 1 -- 172.21.15.117:0/2504377394 wait complete. 2024-09-21T06:08:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.942+0000 7f9853686640 1 Processor -- start 2024-09-21T06:08:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.942+0000 7f9853686640 1 -- start start 2024-09-21T06:08:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.942+0000 7f9853686640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f984c1029b0 0x7f984c10ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.942+0000 7f9853686640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f984c1082e0 0x7f984c110480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.942+0000 7f9853686640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f984c108c30 0x7f984c1109c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.942+0000 7f9853686640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f984c113a10 con 0x7f984c1082e0 2024-09-21T06:08:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.942+0000 7f9853686640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f984c113890 con 0x7f984c1029b0 2024-09-21T06:08:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.942+0000 7f9853686640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f984c113b90 con 0x7f984c108c30 2024-09-21T06:08:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.942+0000 7f9850c21640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f984c1082e0 0x7f984c110480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.942+0000 7f9850c21640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f984c1082e0 0x7f984c110480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35600/0 (socket says 172.21.15.117:35600) 2024-09-21T06:08:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.942+0000 7f9850c21640 1 -- 172.21.15.117:0/3378707289 learned_addr learned my addr 172.21.15.117:0/3378707289 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.942+0000 7f9851422640 1 --2- 172.21.15.117:0/3378707289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f984c1029b0 0x7f984c10ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.942+0000 7f9851c23640 1 --2- 172.21.15.117:0/3378707289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f984c108c30 0x7f984c1109c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.942+0000 7f9850c21640 1 -- 172.21.15.117:0/3378707289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f984c108c30 msgr2=0x7f984c1109c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.942+0000 7f9850c21640 1 --2- 172.21.15.117:0/3378707289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f984c108c30 0x7f984c1109c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.942+0000 7f9850c21640 1 -- 172.21.15.117:0/3378707289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f984c1029b0 msgr2=0x7f984c10ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:17.948 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.942+0000 7f9850c21640 1 --2- 172.21.15.117:0/3378707289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f984c1029b0 0x7f984c10ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.948 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.942+0000 7f9850c21640 1 -- 172.21.15.117:0/3378707289 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984c1a86e0 con 0x7f984c1082e0 2024-09-21T06:08:17.948 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.942+0000 7f9851c23640 1 --2- 172.21.15.117:0/3378707289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f984c108c30 0x7f984c1109c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:08:17.948 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.943+0000 7f9850c21640 1 --2- 172.21.15.117:0/3378707289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f984c1082e0 0x7f984c110480 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f984000c970 tx=0x7f984000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:17.948 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.943+0000 7f984a7fc640 1 -- 172.21.15.117:0/3378707289 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9840004220 con 0x7f984c1082e0 2024-09-21T06:08:17.948 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.943+0000 7f984a7fc640 1 -- 172.21.15.117:0/3378707289 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98400043c0 con 0x7f984c1082e0 2024-09-21T06:08:17.948 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.943+0000 7f9853686640 1 -- 172.21.15.117:0/3378707289 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f984c1a89d0 con 0x7f984c1082e0 2024-09-21T06:08:17.948 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.943+0000 7f984a7fc640 1 -- 172.21.15.117:0/3378707289 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9840004a00 con 0x7f984c1082e0 2024-09-21T06:08:17.948 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.943+0000 7f9853686640 1 -- 172.21.15.117:0/3378707289 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f984c1a8f20 con 0x7f984c1082e0 2024-09-21T06:08:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.946+0000 7f985a5ed740 1 Processor -- start 2024-09-21T06:08:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.946+0000 7f985a5ed740 1 -- start start 2024-09-21T06:08:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.946+0000 7f984a7fc640 1 -- 172.21.15.117:0/3378707289 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f98400079b0 con 0x7f984c1082e0 2024-09-21T06:08:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.946+0000 7f985a5ed740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d091d1b0 0x5618d0cc29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.946+0000 7f985a5ed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d08f42f0 0x5618d0cc2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.946+0000 7f985a5ed740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d08fc4d0 0x5618d0cc3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.946+0000 7f985a5ed740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5618d0c376f0 con 0x5618d08fc4d0 2024-09-21T06:08:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.946+0000 7f985a5ed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5618d0c26b20 con 0x5618d08f42f0 2024-09-21T06:08:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.946+0000 7f985a5ed740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5618d0c2a930 con 0x5618d091d1b0 2024-09-21T06:08:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.946+0000 7f9850c21640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d08fc4d0 0x5618d0cc3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.946+0000 7f9850c21640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d08fc4d0 0x5618d0cc3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35616/0 (socket says 172.21.15.117:35616) 2024-09-21T06:08:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.946+0000 7f9850c21640 1 -- 172.21.15.117:0/3768072005 learned_addr learned my addr 172.21.15.117:0/3768072005 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.946+0000 7f9851422640 1 --2- 172.21.15.117:0/3768072005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d08f42f0 0x5618d0cc2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.946+0000 7f9851c23640 1 --2- 172.21.15.117:0/3768072005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d091d1b0 0x5618d0cc29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.946+0000 7f9850c21640 1 -- 172.21.15.117:0/3768072005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d091d1b0 msgr2=0x5618d0cc29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.946+0000 7f9850c21640 1 --2- 172.21.15.117:0/3768072005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d091d1b0 0x5618d0cc29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.946+0000 7f9850c21640 1 -- 172.21.15.117:0/3768072005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d08f42f0 msgr2=0x5618d0cc2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.946+0000 7f9850c21640 1 --2- 172.21.15.117:0/3768072005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d08f42f0 0x5618d0cc2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.946+0000 7f9850c21640 1 -- 172.21.15.117:0/3768072005 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618d0bec530 con 0x5618d08fc4d0 2024-09-21T06:08:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.946+0000 7f9851422640 1 --2- 172.21.15.117:0/3768072005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d08f42f0 0x5618d0cc2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:08:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.947+0000 7f984a7fc640 1 --2- 172.21.15.117:0/3378707289 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f98340b6730 0x7f98340b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:17.958 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.947+0000 7f9851c23640 1 --2- 172.21.15.117:0/3378707289 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f98340b6730 0x7f98340b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:17.958 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.947+0000 7f9850c21640 1 --2- 172.21.15.117:0/3768072005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d08fc4d0 0x5618d0cc3430 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f98400e4a90 tx=0x7f98400efa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab70b22c39f3fe5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:17.958 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.947+0000 7f984a7fc640 1 -- 172.21.15.117:0/3378707289 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7f98400e7380 con 0x7f984c1082e0 2024-09-21T06:08:17.958 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.947+0000 7f98337fe640 1 -- 172.21.15.117:0/3768072005 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984001d030 con 0x5618d08fc4d0 2024-09-21T06:08:17.958 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.947+0000 7f98337fe640 1 -- 172.21.15.117:0/3768072005 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98400056a0 con 0x5618d08fc4d0 2024-09-21T06:08:17.958 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.947+0000 7f98337fe640 1 -- 172.21.15.117:0/3768072005 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98400a90c0 con 0x5618d08fc4d0 2024-09-21T06:08:17.958 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.948+0000 7f985a5ed740 1 -- 172.21.15.117:0/3768072005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d08fc4d0 msgr2=0x5618d0cc3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:17.958 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.948+0000 7f985a5ed740 1 --2- 172.21.15.117:0/3768072005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d08fc4d0 0x5618d0cc3430 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f98400e4a90 tx=0x7f98400efa10 comp rx=0 tx=0).stop 2024-09-21T06:08:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.948+0000 7f9851c23640 1 --2- 172.21.15.117:0/3378707289 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f98340b6730 0x7f98340b8c40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f98440058c0 tx=0x7f9844005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.948+0000 7f985a5ed740 1 -- 172.21.15.117:0/3768072005 shutdown_connections 2024-09-21T06:08:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.948+0000 7f985a5ed740 1 --2- 172.21.15.117:0/3768072005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d08fc4d0 0x5618d0cc3430 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.948+0000 7f985a5ed740 1 --2- 172.21.15.117:0/3768072005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d08f42f0 0x5618d0cc2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.948+0000 7f985a5ed740 1 --2- 172.21.15.117:0/3768072005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d091d1b0 0x5618d0cc29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.948+0000 7f985a5ed740 1 -- 172.21.15.117:0/3768072005 >> 172.21.15.117:0/3768072005 conn(0x5618d0911300 msgr2=0x5618d0cc08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.948+0000 7f985a5ed740 1 -- 172.21.15.117:0/3768072005 shutdown_connections 2024-09-21T06:08:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.948+0000 7f985a5ed740 1 -- 172.21.15.117:0/3768072005 wait complete. 2024-09-21T06:08:17.959 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.949+0000 7f985a5ed740 1 -- start start 2024-09-21T06:08:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.949+0000 7f985a5ed740 1 Processor -- start 2024-09-21T06:08:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.949+0000 7f985a5ed740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d08f42f0 0x5618d0cce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.949+0000 7f985a5ed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d08fc4d0 0x5618d0cce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.949+0000 7f985a5ed740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d091d1b0 0x5618d0cda450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.949+0000 7f985a5ed740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5618d0b5eab0 con 0x5618d08f42f0 2024-09-21T06:08:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.949+0000 7f985a5ed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5618d0bcc730 con 0x5618d08fc4d0 2024-09-21T06:08:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.949+0000 7f985a5ed740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5618d0cd1130 con 0x5618d091d1b0 2024-09-21T06:08:17.961 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.949+0000 7f985a5ed740 10 client.0 _ll_register_callbacks cb 0x5618d0c206b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:08:17.961 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.949+0000 7f9851422640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d08f42f0 0x5618d0cce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:17.961 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.949+0000 7f9851422640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d08f42f0 0x5618d0cce050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35622/0 (socket says 172.21.15.117:35622) 2024-09-21T06:08:17.961 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.949+0000 7f9851422640 1 -- 172.21.15.117:0/3942901598 learned_addr learned my addr 172.21.15.117:0/3942901598 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:17.961 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.949+0000 7f9851c23640 1 --2- 172.21.15.117:0/3942901598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d091d1b0 0x5618d0cda450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:17.961 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.949+0000 7f9850c21640 1 --2- 172.21.15.117:0/3942901598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d08fc4d0 0x5618d0cce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:17.961 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.950+0000 7f9851422640 1 -- 172.21.15.117:0/3942901598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d091d1b0 msgr2=0x5618d0cda450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:17.961 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.950+0000 7f9851422640 1 --2- 172.21.15.117:0/3942901598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d091d1b0 0x5618d0cda450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.961 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.950+0000 7f9851422640 1 -- 172.21.15.117:0/3942901598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d08fc4d0 msgr2=0x5618d0cce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.950+0000 7f9851422640 1 --2- 172.21.15.117:0/3942901598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d08fc4d0 0x5618d0cce590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.950+0000 7f9851422640 1 -- 172.21.15.117:0/3942901598 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618d0988790 con 0x5618d08f42f0 2024-09-21T06:08:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.950+0000 7f9851c23640 1 --2- 172.21.15.117:0/3942901598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d091d1b0 0x5618d0cda450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:08:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.950+0000 7f9851422640 1 --2- 172.21.15.117:0/3942901598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d08f42f0 0x5618d0cce050 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f983c009ca0 tx=0x7f983c031450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55822af653d50f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.950+0000 7f9830ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:08:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.950+0000 7f9830ff9640 1 -- 172.21.15.117:0/3942901598 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c031890 con 0x5618d08f42f0 2024-09-21T06:08:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.950+0000 7f9830ff9640 1 -- 172.21.15.117:0/3942901598 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7f983c003970 con 0x5618d08f42f0 2024-09-21T06:08:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.950+0000 7f985a5ed740 1 -- 172.21.15.117:0/3942901598 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98400056a0 con 0x5618d08f42f0 2024-09-21T06:08:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.950+0000 7f9830ff9640 1 -- 172.21.15.117:0/3942901598 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f983c003b70 con 0x5618d08f42f0 2024-09-21T06:08:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.950+0000 7f985a5ed740 10 client.18039 fetch_fsmap learned FSMap version 29 2024-09-21T06:08:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.950+0000 7f985a5ed740 1 -- 172.21.15.117:0/3942901598 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5618d0a01620 con 0x5618d08f42f0 2024-09-21T06:08:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.950+0000 7f9830ff9640 1 -- 172.21.15.117:0/3942901598 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98400056a0 con 0x5618d08f42f0 2024-09-21T06:08:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.950+0000 7f9830ff9640 1 -- 172.21.15.117:0/3942901598 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f983c004610 con 0x5618d08f42f0 2024-09-21T06:08:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.950+0000 7f985a5ed740 10 client.18039 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:08:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.950+0000 7f985a5ed740 10 client.18039 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:08:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.951+0000 7f985a5ed740 20 client.18039 populate_metadata read hostname 'smithi117' 2024-09-21T06:08:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.951+0000 7f985a5ed740 1 --2- 172.21.15.117:0/3942901598 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5618d08dbd80 0x5618d0ce0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.951+0000 7f985a5ed740 4 client.18039 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:08:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.951+0000 7f985a5ed740 1 -- 172.21.15.117:0/3942901598 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618d0c39640 con 0x5618d08dbd80 2024-09-21T06:08:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.951+0000 7f9850c21640 1 --2- 172.21.15.117:0/3942901598 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5618d08dbd80 0x5618d0ce0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.951+0000 7f9850c21640 1 --2- 172.21.15.117:0/3942901598 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5618d08dbd80 0x5618d0ce0240 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=167c612e25aefa03 server_cookie=59c599e490dfcc3b in_seq=0 out_seq=0 2024-09-21T06:08:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.951+0000 7f9830ff9640 10 client.18039 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:08:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.952+0000 7f9830ff9640 1 -- 172.21.15.117:0/3942901598 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5618d0c39640 con 0x5618d08dbd80 2024-09-21T06:08:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.952+0000 7f9830ff9640 10 client.18039 handle_command_reply: tid=0 2024-09-21T06:08:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.952+0000 7f985a5ed740 1 client.18039 shutdown 2024-09-21T06:08:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.952+0000 7f985a5ed740 1 -- 172.21.15.117:0/3942901598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d08f42f0 msgr2=0x5618d0cce050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.952+0000 7f985a5ed740 1 --2- 172.21.15.117:0/3942901598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d08f42f0 0x5618d0cce050 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f983c009ca0 tx=0x7f983c031450 comp rx=0 tx=0).stop 2024-09-21T06:08:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.952+0000 7f985a5ed740 1 -- 172.21.15.117:0/3942901598 shutdown_connections 2024-09-21T06:08:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.952+0000 7f985a5ed740 1 --2- 172.21.15.117:0/3942901598 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5618d08dbd80 0x5618d0ce0240 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.952+0000 7f985a5ed740 1 --2- 172.21.15.117:0/3942901598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d091d1b0 0x5618d0cda450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.952+0000 7f985a5ed740 1 --2- 172.21.15.117:0/3942901598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d08fc4d0 0x5618d0cce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.952+0000 7f9830ff9640 0 client.18039 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:08:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.952+0000 7f985a5ed740 1 --2- 172.21.15.117:0/3942901598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d08f42f0 0x5618d0cce050 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.952+0000 7f985a5ed740 1 -- 172.21.15.117:0/3942901598 >> 172.21.15.117:0/3942901598 conn(0x5618d0911300 msgr2=0x5618d0cc39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.952+0000 7f985a5ed740 1 -- 172.21.15.117:0/3942901598 shutdown_connections 2024-09-21T06:08:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.952+0000 7f985a5ed740 1 -- 172.21.15.117:0/3942901598 wait complete. 2024-09-21T06:08:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.952+0000 7f985a5ed740 20 client.18039 trim_cache size 0 max 16384 2024-09-21T06:08:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.967+0000 7f985a5ed740 1 Processor -- start 2024-09-21T06:08:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.967+0000 7f985a5ed740 1 -- start start 2024-09-21T06:08:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.967+0000 7f985a5ed740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d08f42f0 0x5618d0cc40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.967+0000 7f985a5ed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d08fc4d0 0x5618d0cda350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.967+0000 7f985a5ed740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d091d1b0 0x5618d0ccb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.967+0000 7f985a5ed740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f984c1b17c0 con 0x5618d08f42f0 2024-09-21T06:08:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.967+0000 7f985a5ed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f984c1b3300 con 0x5618d08fc4d0 2024-09-21T06:08:17.970 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.967+0000 7f985a5ed740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f984c1b1e90 con 0x5618d091d1b0 2024-09-21T06:08:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.967+0000 7f9851422640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d08f42f0 0x5618d0cc40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.967+0000 7f9851422640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d08f42f0 0x5618d0cc40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35626/0 (socket says 172.21.15.117:35626) 2024-09-21T06:08:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.967+0000 7f9851422640 1 -- 172.21.15.117:0/257873586 learned_addr learned my addr 172.21.15.117:0/257873586 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.967+0000 7f9850c21640 1 --2- 172.21.15.117:0/257873586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d08fc4d0 0x5618d0cda350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.968+0000 7f9851422640 1 -- 172.21.15.117:0/257873586 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d091d1b0 msgr2=0x5618d0ccb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.968+0000 7f9851422640 1 --2- 172.21.15.117:0/257873586 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d091d1b0 0x5618d0ccb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.968+0000 7f9851422640 1 -- 172.21.15.117:0/257873586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d08fc4d0 msgr2=0x5618d0cda350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.968+0000 7f9851422640 1 --2- 172.21.15.117:0/257873586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d08fc4d0 0x5618d0cda350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.968+0000 7f9851422640 1 -- 172.21.15.117:0/257873586 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618d0cc4cb0 con 0x5618d08f42f0 2024-09-21T06:08:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.968+0000 7f9851422640 1 --2- 172.21.15.117:0/257873586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d08f42f0 0x5618d0cc40f0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f983c005e30 tx=0x7f983c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3adb61bc466e75a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.968+0000 7f9832ffd640 1 -- 172.21.15.117:0/257873586 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c02f920 con 0x5618d08f42f0 2024-09-21T06:08:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.968+0000 7f9832ffd640 1 -- 172.21.15.117:0/257873586 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f983c036050 con 0x5618d08f42f0 2024-09-21T06:08:17.971 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.968+0000 7f9832ffd640 1 -- 172.21.15.117:0/257873586 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c0308d0 con 0x5618d08f42f0 2024-09-21T06:08:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.969+0000 7f985a5ed740 1 -- 172.21.15.117:0/257873586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d08f42f0 msgr2=0x5618d0cc40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.969+0000 7f985a5ed740 1 --2- 172.21.15.117:0/257873586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d08f42f0 0x5618d0cc40f0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f983c005e30 tx=0x7f983c02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.969+0000 7f985a5ed740 1 -- 172.21.15.117:0/257873586 shutdown_connections 2024-09-21T06:08:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.969+0000 7f985a5ed740 1 --2- 172.21.15.117:0/257873586 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d091d1b0 0x5618d0ccb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.969+0000 7f985a5ed740 1 --2- 172.21.15.117:0/257873586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d08fc4d0 0x5618d0cda350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.969+0000 7f985a5ed740 1 --2- 172.21.15.117:0/257873586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d08f42f0 0x5618d0cc40f0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.969+0000 7f985a5ed740 1 -- 172.21.15.117:0/257873586 >> 172.21.15.117:0/257873586 conn(0x5618d0911300 msgr2=0x5618d0cc39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.969+0000 7f985a5ed740 1 -- 172.21.15.117:0/257873586 shutdown_connections 2024-09-21T06:08:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.969+0000 7f985a5ed740 1 -- 172.21.15.117:0/257873586 wait complete. 2024-09-21T06:08:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.969+0000 7f985a5ed740 1 -- start start 2024-09-21T06:08:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.969+0000 7f985a5ed740 1 Processor -- start 2024-09-21T06:08:17.975 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.970+0000 7f985a5ed740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d08f42f0 0x5618d0cd3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.970+0000 7f985a5ed740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d08fc4d0 0x5618d0cd4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.970+0000 7f985a5ed740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d091d1b0 0x5618d0d42c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.970+0000 7f985a5ed740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f984c1b2c30 con 0x5618d091d1b0 2024-09-21T06:08:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.970+0000 7f985a5ed740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f984c1b2560 con 0x5618d08fc4d0 2024-09-21T06:08:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.970+0000 7f985a5ed740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f984c1af810 con 0x5618d08f42f0 2024-09-21T06:08:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.970+0000 7f985a5ed740 10 client.0 _ll_register_callbacks cb 0x7f983c009440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:08:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.970+0000 7f9851c23640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d091d1b0 0x5618d0d42c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.970+0000 7f9851c23640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d091d1b0 0x5618d0d42c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35634/0 (socket says 172.21.15.117:35634) 2024-09-21T06:08:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.970+0000 7f9851c23640 1 -- 172.21.15.117:0/1294837273 learned_addr learned my addr 172.21.15.117:0/1294837273 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.970+0000 7f9851422640 1 --2- 172.21.15.117:0/1294837273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d08f42f0 0x5618d0cd3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.970+0000 7f9850c21640 1 --2- 172.21.15.117:0/1294837273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d08fc4d0 0x5618d0cd4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.970+0000 7f9851c23640 1 -- 172.21.15.117:0/1294837273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d08f42f0 msgr2=0x5618d0cd3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.970+0000 7f9851c23640 1 --2- 172.21.15.117:0/1294837273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d08f42f0 0x5618d0cd3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.976 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.970+0000 7f9851c23640 1 -- 172.21.15.117:0/1294837273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d08fc4d0 msgr2=0x5618d0cd4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.970+0000 7f9851c23640 1 --2- 172.21.15.117:0/1294837273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d08fc4d0 0x5618d0cd4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.970+0000 7f9851c23640 1 -- 172.21.15.117:0/1294837273 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618d0d432c0 con 0x5618d091d1b0 2024-09-21T06:08:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.970+0000 7f9851c23640 1 --2- 172.21.15.117:0/1294837273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d091d1b0 0x5618d0d42c20 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f984400bfd0 tx=0x7f9844004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ba82c842d8297d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.970+0000 7f9830ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:08:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.970+0000 7f9830ff9640 1 -- 172.21.15.117:0/1294837273 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984401e9b0 con 0x5618d091d1b0 2024-09-21T06:08:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.971+0000 7f9830ff9640 1 -- 172.21.15.117:0/1294837273 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7f984401eb50 con 0x5618d091d1b0 2024-09-21T06:08:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.971+0000 7f985a5ed740 1 -- 172.21.15.117:0/1294837273 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f983c036050 con 0x5618d091d1b0 2024-09-21T06:08:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.971+0000 7f9830ff9640 1 -- 172.21.15.117:0/1294837273 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f984401ed50 con 0x5618d091d1b0 2024-09-21T06:08:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.971+0000 7f985a5ed740 10 client.18045 fetch_fsmap learned FSMap version 29 2024-09-21T06:08:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.971+0000 7f985a5ed740 1 -- 172.21.15.117:0/1294837273 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5618d0d4a620 con 0x5618d091d1b0 2024-09-21T06:08:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.971+0000 7f9830ff9640 1 -- 172.21.15.117:0/1294837273 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f983c036050 con 0x5618d091d1b0 2024-09-21T06:08:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.971+0000 7f9830ff9640 1 -- 172.21.15.117:0/1294837273 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f984401c530 con 0x5618d091d1b0 2024-09-21T06:08:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.971+0000 7f985a5ed740 10 client.18045 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:08:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.971+0000 7f985a5ed740 10 client.18045 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-21T06:08:17.977 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.971+0000 7f985a5ed740 20 client.18045 populate_metadata read hostname 'smithi117' 2024-09-21T06:08:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.971+0000 7f985a5ed740 1 --2- 172.21.15.117:0/1294837273 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5618d08b11c0 0x5618d0d4ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.971+0000 7f985a5ed740 4 client.18045 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T06:08:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.971+0000 7f985a5ed740 1 -- 172.21.15.117:0/1294837273 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5618d0c350f0 con 0x5618d08b11c0 2024-09-21T06:08:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.971+0000 7f9851422640 1 --2- 172.21.15.117:0/1294837273 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5618d08b11c0 0x5618d0d4ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.972+0000 7f9830ff9640 10 client.18045 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:08:17.978 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.972+0000 7f9851422640 1 --2- 172.21.15.117:0/1294837273 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5618d08b11c0 0x5618d0d4ce90 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9df3df4af3c45a3e server_cookie=9380d787118ac4de in_seq=0 out_seq=0 2024-09-21T06:08:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.979+0000 7f9830ff9640 1 -- 172.21.15.117:0/1294837273 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5618d0c350f0 con 0x5618d08b11c0 2024-09-21T06:08:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.979+0000 7f9830ff9640 10 client.18045 handle_command_reply: tid=0 2024-09-21T06:08:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.979+0000 7f985a5ed740 1 client.18045 shutdown 2024-09-21T06:08:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.979+0000 7f985a5ed740 1 -- 172.21.15.117:0/1294837273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d091d1b0 msgr2=0x5618d0d42c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.979+0000 7f985a5ed740 1 --2- 172.21.15.117:0/1294837273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d091d1b0 0x5618d0d42c20 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f984400bfd0 tx=0x7f9844004730 comp rx=0 tx=0).stop 2024-09-21T06:08:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.979+0000 7f985a5ed740 1 -- 172.21.15.117:0/1294837273 shutdown_connections 2024-09-21T06:08:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.979+0000 7f985a5ed740 1 --2- 172.21.15.117:0/1294837273 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x5618d08b11c0 0x5618d0d4ce90 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.979+0000 7f985a5ed740 1 --2- 172.21.15.117:0/1294837273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5618d091d1b0 0x5618d0d42c20 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.979+0000 7f985a5ed740 1 --2- 172.21.15.117:0/1294837273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5618d08fc4d0 0x5618d0cd4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.979+0000 7f9830ff9640 0 client.18045 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:08:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.979+0000 7f985a5ed740 1 --2- 172.21.15.117:0/1294837273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5618d08f42f0 0x5618d0cd3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.979+0000 7f985a5ed740 1 -- 172.21.15.117:0/1294837273 >> 172.21.15.117:0/1294837273 conn(0x5618d0911300 msgr2=0x5618d0cc0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.979+0000 7f985a5ed740 1 -- 172.21.15.117:0/1294837273 shutdown_connections 2024-09-21T06:08:17.982 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.979+0000 7f985a5ed740 1 -- 172.21.15.117:0/1294837273 wait complete. 2024-09-21T06:08:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.979+0000 7f985a5ed740 20 client.18045 trim_cache size 0 max 16384 2024-09-21T06:08:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:{ 2024-09-21T06:08:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "message": "", 2024-09-21T06:08:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout: "return_code": 0 2024-09-21T06:08:17.983 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stdout:} 2024-09-21T06:08:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.980+0000 7f9853686640 1 -- 172.21.15.117:0/3378707289 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f98340b6730 msgr2=0x7f98340b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.980+0000 7f9853686640 1 --2- 172.21.15.117:0/3378707289 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f98340b6730 0x7f98340b8c40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f98440058c0 tx=0x7f9844005850 comp rx=0 tx=0).stop 2024-09-21T06:08:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.980+0000 7f9853686640 1 -- 172.21.15.117:0/3378707289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f984c1082e0 msgr2=0x7f984c110480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.980+0000 7f9853686640 1 --2- 172.21.15.117:0/3378707289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f984c1082e0 0x7f984c110480 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f984000c970 tx=0x7f984000ce30 comp rx=0 tx=0).stop 2024-09-21T06:08:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.980+0000 7f9853686640 1 -- 172.21.15.117:0/3378707289 shutdown_connections 2024-09-21T06:08:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.980+0000 7f9853686640 1 --2- 172.21.15.117:0/3378707289 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f98340b6730 0x7f98340b8c40 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.984 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.980+0000 7f9853686640 1 --2- 172.21.15.117:0/3378707289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f984c1082e0 0x7f984c110480 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.980+0000 7f9853686640 1 --2- 172.21.15.117:0/3378707289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f984c108c30 0x7f984c1109c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.980+0000 7f9853686640 1 --2- 172.21.15.117:0/3378707289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f984c1029b0 0x7f984c10ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.980+0000 7f9853686640 1 -- 172.21.15.117:0/3378707289 >> 172.21.15.117:0/3378707289 conn(0x7f984c07a740 msgr2=0x7f984c1063d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.980+0000 7f9853686640 1 -- 172.21.15.117:0/3378707289 shutdown_connections 2024-09-21T06:08:17.985 INFO:teuthology.task.background_exec.ubuntu@smithi117.front.sepia.ceph.com.smithi117.stderr:2024-09-21T06:08:17.980+0000 7f9853686640 1 -- 172.21.15.117:0/3378707289 wait complete. 2024-09-21T06:08:18.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:17 smithi117 ceph-mon[30947]: pgmap v664: 97 pgs: 97 active+clean; 30 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 818 B/s wr, 10 op/s 2024-09-21T06:08:18.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:17 smithi117 ceph-mon[30947]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T06:08:18.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:17 smithi138 ceph-mon[34391]: pgmap v664: 97 pgs: 97 active+clean; 30 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 818 B/s wr, 10 op/s 2024-09-21T06:08:18.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:17 smithi138 ceph-mon[34391]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T06:08:18.479 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:17 smithi192 ceph-mon[34700]: pgmap v664: 97 pgs: 97 active+clean; 30 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 818 B/s wr, 10 op/s 2024-09-21T06:08:18.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:17 smithi192 ceph-mon[34700]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T06:08:18.866 INFO:tasks.workunit.client.0.smithi117.stdout:stopping iogen 2024-09-21T06:08:18.867 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'stopping iogen' 2024-09-21T06:08:18.867 INFO:tasks.workunit.client.0.smithi117.stderr:+ ./iogen -k 2024-09-21T06:08:18.868 INFO:tasks.workunit.client.0.smithi117.stdout:OK 2024-09-21T06:08:18.869 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo OK 2024-09-21T06:08:20.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:19 smithi117 ceph-mon[30947]: pgmap v666: 97 pgs: 97 active+clean; 30 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 345 B/s wr, 10 op/s 2024-09-21T06:08:20.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:19 smithi138 ceph-mon[34391]: pgmap v666: 97 pgs: 97 active+clean; 30 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 345 B/s wr, 10 op/s 2024-09-21T06:08:20.479 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:19 smithi192 ceph-mon[34700]: pgmap v666: 97 pgs: 97 active+clean; 30 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 345 B/s wr, 10 op/s 2024-09-21T06:08:20.887 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-21T06:08:20.888 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-21T06:08:21.587 INFO:tasks.workunit:Stopping ['suites/iogen.sh'] on client.0... 2024-09-21T06:08:21.587 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-21T06:08:21.872 DEBUG:teuthology.parallel:result is None 2024-09-21T06:08:21.873 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T06:08:21.907 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T06:08:21.908 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-21T06:08:21.918 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi117.front.sepia.ceph.com 2024-09-21T06:08:21.955 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-21T06:08:21.965 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-21T06:08:21.965 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T06:08:21.965 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi117.front.sepia.ceph.com 2024-09-21T06:08:21.965 DEBUG:teuthology.orchestra.run.smithi117:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-21T06:08:22.038 DEBUG:teuthology.orchestra.run.smithi117:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-21T06:08:22.115 DEBUG:teuthology.orchestra.run.smithi117:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-21T06:08:22.176 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:22 smithi117 ceph-mon[30947]: pgmap v667: 97 pgs: 97 active+clean; 30 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 409 B/s wr, 10 op/s 2024-09-21T06:08:22.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.264+0000 7f1c94225640 1 -- 172.21.15.117:0/22507198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c8c108310 msgr2=0x7f1c8c108720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:22.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.264+0000 7f1c94225640 1 --2- 172.21.15.117:0/22507198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c8c108310 0x7f1c8c108720 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f1c80009900 tx=0x7f1c8002a410 comp rx=0 tx=0).stop 2024-09-21T06:08:22.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.264+0000 7f1c94225640 1 -- 172.21.15.117:0/22507198 shutdown_connections 2024-09-21T06:08:22.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.264+0000 7f1c94225640 1 --2- 172.21.15.117:0/22507198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c8c108310 0x7f1c8c108720 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:22.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.264+0000 7f1c94225640 1 --2- 172.21.15.117:0/22507198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c8c108c60 0x7f1c8c10f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:22.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.264+0000 7f1c94225640 1 --2- 172.21.15.117:0/22507198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c8c1029e0 0x7f1c8c102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:22.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.264+0000 7f1c94225640 1 -- 172.21.15.117:0/22507198 >> 172.21.15.117:0/22507198 conn(0x7f1c8c0fe3c0 msgr2=0x7f1c8c100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:22.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.264+0000 7f1c94225640 1 -- 172.21.15.117:0/22507198 shutdown_connections 2024-09-21T06:08:22.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.264+0000 7f1c94225640 1 -- 172.21.15.117:0/22507198 wait complete. 2024-09-21T06:08:22.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.265+0000 7f1c94225640 1 Processor -- start 2024-09-21T06:08:22.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.265+0000 7f1c94225640 1 -- start start 2024-09-21T06:08:22.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.265+0000 7f1c94225640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c8c1029e0 0x7f1c8c19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:22.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.265+0000 7f1c94225640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c8c108310 0x7f1c8c19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:22.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.265+0000 7f1c94225640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c8c108c60 0x7f1c8c19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:22.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.265+0000 7f1c94225640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c112920 con 0x7f1c8c108310 2024-09-21T06:08:22.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.265+0000 7f1c94225640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c1127a0 con 0x7f1c8c1029e0 2024-09-21T06:08:22.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.265+0000 7f1c94225640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8c112aa0 con 0x7f1c8c108c60 2024-09-21T06:08:22.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.265+0000 7f1c917c0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c8c108310 0x7f1c8c19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:22.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.265+0000 7f1c917c0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c8c108310 0x7f1c8c19aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35644/0 (socket says 172.21.15.117:35644) 2024-09-21T06:08:22.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.265+0000 7f1c917c0640 1 -- 172.21.15.117:0/1805802004 learned_addr learned my addr 172.21.15.117:0/1805802004 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:22.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.265+0000 7f1c91fc1640 1 --2- 172.21.15.117:0/1805802004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c8c1029e0 0x7f1c8c19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:22.269 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.266+0000 7f1c917c0640 1 -- 172.21.15.117:0/1805802004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c8c108c60 msgr2=0x7f1c8c19fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:22.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.266+0000 7f1c917c0640 1 --2- 172.21.15.117:0/1805802004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c8c108c60 0x7f1c8c19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:22.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.266+0000 7f1c917c0640 1 -- 172.21.15.117:0/1805802004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c8c1029e0 msgr2=0x7f1c8c19a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:22.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.266+0000 7f1c917c0640 1 --2- 172.21.15.117:0/1805802004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c8c1029e0 0x7f1c8c19a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:22.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.266+0000 7f1c917c0640 1 -- 172.21.15.117:0/1805802004 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c8c1a0400 con 0x7f1c8c108310 2024-09-21T06:08:22.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.266+0000 7f1c91fc1640 1 --2- 172.21.15.117:0/1805802004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c8c1029e0 0x7f1c8c19a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:08:22.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.266+0000 7f1c917c0640 1 --2- 172.21.15.117:0/1805802004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c8c108310 0x7f1c8c19aa70 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f1c8002a930 tx=0x7f1c80004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:22.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.266+0000 7f1c8affd640 1 -- 172.21.15.117:0/1805802004 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8003f030 con 0x7f1c8c108310 2024-09-21T06:08:22.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.266+0000 7f1c94225640 1 -- 172.21.15.117:0/1805802004 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c8c1a0690 con 0x7f1c8c108310 2024-09-21T06:08:22.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.266+0000 7f1c8affd640 1 -- 172.21.15.117:0/1805802004 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c80003aa0 con 0x7f1c8c108310 2024-09-21T06:08:22.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.266+0000 7f1c8affd640 1 -- 172.21.15.117:0/1805802004 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c80035070 con 0x7f1c8c108310 2024-09-21T06:08:22.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.266+0000 7f1c94225640 1 -- 172.21.15.117:0/1805802004 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c8c1a6e80 con 0x7f1c8c108310 2024-09-21T06:08:22.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.267+0000 7f1c94225640 1 -- 172.21.15.117:0/1805802004 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c8c103c50 con 0x7f1c8c108310 2024-09-21T06:08:22.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.269+0000 7f1c8affd640 1 -- 172.21.15.117:0/1805802004 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f1c80042030 con 0x7f1c8c108310 2024-09-21T06:08:22.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.269+0000 7f1c8affd640 1 --2- 172.21.15.117:0/1805802004 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1c740b6680 0x7f1c740b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:22.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.269+0000 7f1c91fc1640 1 --2- 172.21.15.117:0/1805802004 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1c740b6680 0x7f1c740b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:22.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.269+0000 7f1c8affd640 1 -- 172.21.15.117:0/1805802004 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7f1c801082a0 con 0x7f1c8c108310 2024-09-21T06:08:22.271 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.270+0000 7f1c91fc1640 1 --2- 172.21.15.117:0/1805802004 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1c740b6680 0x7f1c740b8b90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c009b40 tx=0x7f1c7c005e50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:22.274 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.273+0000 7f1c8affd640 1 -- 172.21.15.117:0/1805802004 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1c800ce840 con 0x7f1c8c108310 2024-09-21T06:08:22.282 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:22 smithi192 ceph-mon[34700]: pgmap v667: 97 pgs: 97 active+clean; 30 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 409 B/s wr, 10 op/s 2024-09-21T06:08:22.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:22 smithi138 ceph-mon[34391]: pgmap v667: 97 pgs: 97 active+clean; 30 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 409 B/s wr, 10 op/s 2024-09-21T06:08:22.474 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.472+0000 7f1c94225640 1 -- 172.21.15.117:0/1805802004 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f1c8c106ae0 con 0x7f1c740b6680 2024-09-21T06:08:22.484 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.482+0000 7f1c8affd640 1 -- 172.21.15.117:0/1805802004 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f1c8c106ae0 con 0x7f1c740b6680 2024-09-21T06:08:22.484 INFO:teuthology.orchestra.run.smithi117.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-21T00:00:00", "created": "2024-09-21T05:57:14", "first": "2024-09-21T05:58:00", "last": "2024-09-21T06:08:00", "last_pruned": "2024-09-21T06:08:00", "created_count": 11, "pruned_count": 4, "active": true} 2024-09-21T06:08:22.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.485+0000 7f1c94225640 1 -- 172.21.15.117:0/1805802004 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1c740b6680 msgr2=0x7f1c740b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:22.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.485+0000 7f1c94225640 1 --2- 172.21.15.117:0/1805802004 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1c740b6680 0x7f1c740b8b90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c009b40 tx=0x7f1c7c005e50 comp rx=0 tx=0).stop 2024-09-21T06:08:22.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.485+0000 7f1c94225640 1 -- 172.21.15.117:0/1805802004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c8c108310 msgr2=0x7f1c8c19aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:22.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.485+0000 7f1c94225640 1 --2- 172.21.15.117:0/1805802004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c8c108310 0x7f1c8c19aa70 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f1c8002a930 tx=0x7f1c80004270 comp rx=0 tx=0).stop 2024-09-21T06:08:22.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.485+0000 7f1c94225640 1 -- 172.21.15.117:0/1805802004 shutdown_connections 2024-09-21T06:08:22.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.485+0000 7f1c94225640 1 --2- 172.21.15.117:0/1805802004 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f1c740b6680 0x7f1c740b8b90 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:22.487 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.485+0000 7f1c94225640 1 --2- 172.21.15.117:0/1805802004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c8c108310 0x7f1c8c19aa70 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:22.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.485+0000 7f1c94225640 1 --2- 172.21.15.117:0/1805802004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c8c108c60 0x7f1c8c19fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:22.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.485+0000 7f1c94225640 1 --2- 172.21.15.117:0/1805802004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c8c1029e0 0x7f1c8c19a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:22.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.485+0000 7f1c94225640 1 -- 172.21.15.117:0/1805802004 >> 172.21.15.117:0/1805802004 conn(0x7f1c8c0fe3c0 msgr2=0x7f1c8c101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:22.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.485+0000 7f1c94225640 1 -- 172.21.15.117:0/1805802004 shutdown_connections 2024-09-21T06:08:22.488 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.485+0000 7f1c94225640 1 -- 172.21.15.117:0/1805802004 wait complete. 2024-09-21T06:08:22.498 DEBUG:teuthology.orchestra.run.smithi117:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-21T06:08:22.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.648+0000 7fb7024d5640 1 -- 172.21.15.117:0/419963189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6fc1029b0 msgr2=0x7fb6fc102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:22.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.648+0000 7fb7024d5640 1 --2- 172.21.15.117:0/419963189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6fc1029b0 0x7fb6fc102dc0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec009900 tx=0x7fb6ec02a410 comp rx=0 tx=0).stop 2024-09-21T06:08:22.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.648+0000 7fb7024d5640 1 -- 172.21.15.117:0/419963189 shutdown_connections 2024-09-21T06:08:22.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.648+0000 7fb7024d5640 1 --2- 172.21.15.117:0/419963189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6fc108c30 0x7fb6fc10f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:22.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.648+0000 7fb7024d5640 1 --2- 172.21.15.117:0/419963189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6fc1082e0 0x7fb6fc1086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:22.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.648+0000 7fb7024d5640 1 --2- 172.21.15.117:0/419963189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6fc1029b0 0x7fb6fc102dc0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:22.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.648+0000 7fb7024d5640 1 -- 172.21.15.117:0/419963189 >> 172.21.15.117:0/419963189 conn(0x7fb6fc07a740 msgr2=0x7fb6fc07ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:22.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.648+0000 7fb7024d5640 1 -- 172.21.15.117:0/419963189 shutdown_connections 2024-09-21T06:08:22.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.648+0000 7fb7024d5640 1 -- 172.21.15.117:0/419963189 wait complete. 2024-09-21T06:08:22.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.649+0000 7fb7024d5640 1 Processor -- start 2024-09-21T06:08:22.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.649+0000 7fb7024d5640 1 -- start start 2024-09-21T06:08:22.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.649+0000 7fb7024d5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6fc1029b0 0x7fb6fc199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:22.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.649+0000 7fb7024d5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6fc1082e0 0x7fb6fc1998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:22.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.649+0000 7fb7024d5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6fc108c30 0x7fb6fc19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:22.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.649+0000 7fb7024d5640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb6fc1128f0 con 0x7fb6fc1029b0 2024-09-21T06:08:22.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.649+0000 7fb7024d5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb6fc112770 con 0x7fb6fc108c30 2024-09-21T06:08:22.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.649+0000 7fb7024d5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb6fc112a70 con 0x7fb6fc1082e0 2024-09-21T06:08:22.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.649+0000 7fb6fbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6fc1029b0 0x7fb6fc199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:22.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.649+0000 7fb6fbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6fc1029b0 0x7fb6fc199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35668/0 (socket says 172.21.15.117:35668) 2024-09-21T06:08:22.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.649+0000 7fb6fbfff640 1 -- 172.21.15.117:0/2578626857 learned_addr learned my addr 172.21.15.117:0/2578626857 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:22.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.649+0000 7fb6fb7fe640 1 --2- 172.21.15.117:0/2578626857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6fc1082e0 0x7fb6fc1998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:22.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.649+0000 7fb6fbfff640 1 -- 172.21.15.117:0/2578626857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6fc1082e0 msgr2=0x7fb6fc1998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:22.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.649+0000 7fb6fbfff640 1 --2- 172.21.15.117:0/2578626857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6fc1082e0 0x7fb6fc1998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:22.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.649+0000 7fb700a72640 1 --2- 172.21.15.117:0/2578626857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6fc108c30 0x7fb6fc19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:22.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.649+0000 7fb6fbfff640 1 -- 172.21.15.117:0/2578626857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6fc108c30 msgr2=0x7fb6fc19eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:22.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.650+0000 7fb6fbfff640 1 --2- 172.21.15.117:0/2578626857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6fc108c30 0x7fb6fc19eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:22.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.650+0000 7fb6fbfff640 1 -- 172.21.15.117:0/2578626857 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6fc19f260 con 0x7fb6fc1029b0 2024-09-21T06:08:22.654 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.650+0000 7fb700a72640 1 --2- 172.21.15.117:0/2578626857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6fc108c30 0x7fb6fc19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:08:22.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.650+0000 7fb6fbfff640 1 --2- 172.21.15.117:0/2578626857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6fc1029b0 0x7fb6fc199330 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec005940 tx=0x7fb6ec004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:22.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.650+0000 7fb6f97fa640 1 -- 172.21.15.117:0/2578626857 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec03f030 con 0x7fb6fc1029b0 2024-09-21T06:08:22.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.650+0000 7fb7024d5640 1 -- 172.21.15.117:0/2578626857 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6fc19f4f0 con 0x7fb6fc1029b0 2024-09-21T06:08:22.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.650+0000 7fb6f97fa640 1 -- 172.21.15.117:0/2578626857 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6ec003aa0 con 0x7fb6fc1029b0 2024-09-21T06:08:22.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.650+0000 7fb6f97fa640 1 -- 172.21.15.117:0/2578626857 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec035070 con 0x7fb6fc1029b0 2024-09-21T06:08:22.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.650+0000 7fb7024d5640 1 -- 172.21.15.117:0/2578626857 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6fc06abe0 con 0x7fb6fc1029b0 2024-09-21T06:08:22.655 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.651+0000 7fb7024d5640 1 -- 172.21.15.117:0/2578626857 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6fc103c20 con 0x7fb6fc1029b0 2024-09-21T06:08:22.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.653+0000 7fb6f97fa640 1 -- 172.21.15.117:0/2578626857 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fb6ec042030 con 0x7fb6fc1029b0 2024-09-21T06:08:22.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.653+0000 7fb6f97fa640 1 --2- 172.21.15.117:0/2578626857 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6e40b6680 0x7fb6e40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:22.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.653+0000 7fb6fb7fe640 1 --2- 172.21.15.117:0/2578626857 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6e40b6680 0x7fb6e40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:22.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.653+0000 7fb6f97fa640 1 -- 172.21.15.117:0/2578626857 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7fb6ec1081a0 con 0x7fb6fc1029b0 2024-09-21T06:08:22.656 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.654+0000 7fb6fb7fe640 1 --2- 172.21.15.117:0/2578626857 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6e40b6680 0x7fb6e40b8b90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc07b1e0 tx=0x7fb6f0005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:22.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.656+0000 7fb6f97fa640 1 -- 172.21.15.117:0/2578626857 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb6ec0ce740 con 0x7fb6fc1029b0 2024-09-21T06:08:22.858 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.856+0000 7fb7024d5640 1 -- 172.21.15.117:0/2578626857 --> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fb6fc06af00 con 0x7fb6e40b6680 2024-09-21T06:08:22.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.865+0000 7fb6f97fa640 1 -- 172.21.15.117:0/2578626857 <== mgr.14223 v2:172.21.15.117:6800/4224731851 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fb6fc06af00 con 0x7fb6e40b6680 2024-09-21T06:08:22.867 INFO:teuthology.orchestra.run.smithi117.stdout:/ 1m 6m3h 2024-09-21T06:08:22.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.867+0000 7fb7024d5640 1 -- 172.21.15.117:0/2578626857 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6e40b6680 msgr2=0x7fb6e40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:22.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.867+0000 7fb7024d5640 1 --2- 172.21.15.117:0/2578626857 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6e40b6680 0x7fb6e40b8b90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc07b1e0 tx=0x7fb6f0005f90 comp rx=0 tx=0).stop 2024-09-21T06:08:22.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.867+0000 7fb7024d5640 1 -- 172.21.15.117:0/2578626857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6fc1029b0 msgr2=0x7fb6fc199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:22.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.867+0000 7fb7024d5640 1 --2- 172.21.15.117:0/2578626857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6fc1029b0 0x7fb6fc199330 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec005940 tx=0x7fb6ec004270 comp rx=0 tx=0).stop 2024-09-21T06:08:22.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.867+0000 7fb7024d5640 1 -- 172.21.15.117:0/2578626857 shutdown_connections 2024-09-21T06:08:22.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.867+0000 7fb7024d5640 1 --2- 172.21.15.117:0/2578626857 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6e40b6680 0x7fb6e40b8b90 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:22.869 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.867+0000 7fb7024d5640 1 --2- 172.21.15.117:0/2578626857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6fc108c30 0x7fb6fc19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:22.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.867+0000 7fb7024d5640 1 --2- 172.21.15.117:0/2578626857 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6fc1082e0 0x7fb6fc1998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:22.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.867+0000 7fb7024d5640 1 --2- 172.21.15.117:0/2578626857 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6fc1029b0 0x7fb6fc199330 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:22.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.867+0000 7fb7024d5640 1 -- 172.21.15.117:0/2578626857 >> 172.21.15.117:0/2578626857 conn(0x7fb6fc07a740 msgr2=0x7fb6fc1063d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:22.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.867+0000 7fb7024d5640 1 -- 172.21.15.117:0/2578626857 shutdown_connections 2024-09-21T06:08:22.870 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:22.867+0000 7fb7024d5640 1 -- 172.21.15.117:0/2578626857 wait complete. 2024-09-21T06:08:22.880 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-21T06:08:22.891 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-21T06:08:24.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:24 smithi117 ceph-mon[30947]: from='client.18051 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:08:24.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:24 smithi117 ceph-mon[30947]: from='client.18057 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:08:24.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:24 smithi117 ceph-mon[30947]: pgmap v668: 97 pgs: 97 active+clean; 30 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 409 B/s wr, 10 op/s 2024-09-21T06:08:24.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:24 smithi138 ceph-mon[34391]: from='client.18051 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:08:24.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:24 smithi138 ceph-mon[34391]: from='client.18057 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:08:24.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:24 smithi138 ceph-mon[34391]: pgmap v668: 97 pgs: 97 active+clean; 30 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 409 B/s wr, 10 op/s 2024-09-21T06:08:24.479 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:24 smithi192 ceph-mon[34700]: from='client.18051 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:08:24.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:24 smithi192 ceph-mon[34700]: from='client.18057 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T06:08:24.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:24 smithi192 ceph-mon[34700]: pgmap v668: 97 pgs: 97 active+clean; 30 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 409 B/s wr, 10 op/s 2024-09-21T06:08:26.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:26 smithi117 ceph-mon[30947]: pgmap v669: 97 pgs: 97 active+clean; 30 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-09-21T06:08:26.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:26 smithi138 ceph-mon[34391]: pgmap v669: 97 pgs: 97 active+clean; 30 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-09-21T06:08:26.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:26 smithi192 ceph-mon[34700]: pgmap v669: 97 pgs: 97 active+clean; 30 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-09-21T06:08:28.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:28 smithi117 ceph-mon[30947]: pgmap v670: 97 pgs: 97 active+clean; 30 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-09-21T06:08:28.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:28 smithi138 ceph-mon[34391]: pgmap v670: 97 pgs: 97 active+clean; 30 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-09-21T06:08:28.479 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:28 smithi192 ceph-mon[34700]: pgmap v670: 97 pgs: 97 active+clean; 30 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-09-21T06:08:29.949 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T06:08:30.214 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:08:30.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:30 smithi117 ceph-mon[30947]: pgmap v671: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 KiB/s wr, 89 op/s 2024-09-21T06:08:30.367 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:30 smithi138 ceph-mon[34391]: pgmap v671: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 KiB/s wr, 89 op/s 2024-09-21T06:08:30.479 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:30 smithi192 ceph-mon[34700]: pgmap v671: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 21 KiB/s wr, 89 op/s 2024-09-21T06:08:30.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.676+0000 7fc08f811640 1 -- 172.21.15.117:0/3261957244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc088102a80 msgr2=0x7fc088102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:30.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.676+0000 7fc08f811640 1 --2- 172.21.15.117:0/3261957244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc088102a80 0x7fc088102e90 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fc084009900 tx=0x7fc08402a410 comp rx=0 tx=0).stop 2024-09-21T06:08:30.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.676+0000 7fc08f811640 1 -- 172.21.15.117:0/3261957244 shutdown_connections 2024-09-21T06:08:30.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.676+0000 7fc08f811640 1 --2- 172.21.15.117:0/3261957244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc088108d10 0x7fc08810f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:30.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.676+0000 7fc08f811640 1 --2- 172.21.15.117:0/3261957244 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0881083c0 0x7fc0881087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:30.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.676+0000 7fc08f811640 1 --2- 172.21.15.117:0/3261957244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc088102a80 0x7fc088102e90 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:30.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.676+0000 7fc08f811640 1 -- 172.21.15.117:0/3261957244 >> 172.21.15.117:0/3261957244 conn(0x7fc088079740 msgr2=0x7fc088079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:30.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.677+0000 7fc08f811640 1 -- 172.21.15.117:0/3261957244 shutdown_connections 2024-09-21T06:08:30.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.677+0000 7fc08f811640 1 -- 172.21.15.117:0/3261957244 wait complete. 2024-09-21T06:08:30.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.677+0000 7fc08f811640 1 Processor -- start 2024-09-21T06:08:30.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.677+0000 7fc08f811640 1 -- start start 2024-09-21T06:08:30.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.678+0000 7fc08f811640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc088102a80 0x7fc088104270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:30.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.678+0000 7fc08f811640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0881083c0 0x7fc0881047b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:30.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.678+0000 7fc08f811640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc088108d10 0x7fc088104cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:30.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.678+0000 7fc08f811640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc088112980 con 0x7fc088102a80 2024-09-21T06:08:30.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.678+0000 7fc08f811640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc088112800 con 0x7fc0881083c0 2024-09-21T06:08:30.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.678+0000 7fc08f811640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc088112b00 con 0x7fc088108d10 2024-09-21T06:08:30.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.678+0000 7fc08d5ad640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc088102a80 0x7fc088104270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:30.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.678+0000 7fc08d5ad640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc088102a80 0x7fc088104270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39482/0 (socket says 172.21.15.117:39482) 2024-09-21T06:08:30.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.678+0000 7fc08d5ad640 1 -- 172.21.15.117:0/3397434219 learned_addr learned my addr 172.21.15.117:0/3397434219 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:30.690 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.678+0000 7fc08cdac640 1 --2- 172.21.15.117:0/3397434219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0881083c0 0x7fc0881047b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:30.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.678+0000 7fc08ddae640 1 --2- 172.21.15.117:0/3397434219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc088108d10 0x7fc088104cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:30.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.678+0000 7fc08d5ad640 1 -- 172.21.15.117:0/3397434219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc088108d10 msgr2=0x7fc088104cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:30.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.678+0000 7fc08d5ad640 1 --2- 172.21.15.117:0/3397434219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc088108d10 0x7fc088104cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:30.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.678+0000 7fc08d5ad640 1 -- 172.21.15.117:0/3397434219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0881083c0 msgr2=0x7fc0881047b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:30.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.678+0000 7fc08d5ad640 1 --2- 172.21.15.117:0/3397434219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0881083c0 0x7fc0881047b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:30.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.678+0000 7fc08d5ad640 1 -- 172.21.15.117:0/3397434219 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0881a8760 con 0x7fc088102a80 2024-09-21T06:08:30.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.678+0000 7fc08cdac640 1 --2- 172.21.15.117:0/3397434219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0881083c0 0x7fc0881047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:08:30.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.678+0000 7fc08ddae640 1 --2- 172.21.15.117:0/3397434219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc088108d10 0x7fc088104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:08:30.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.678+0000 7fc08d5ad640 1 --2- 172.21.15.117:0/3397434219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc088102a80 0x7fc088104270 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fc084005940 tx=0x7fc0840042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:30.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.678+0000 7fc07e7fc640 1 -- 172.21.15.117:0/3397434219 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08403f030 con 0x7fc088102a80 2024-09-21T06:08:30.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.678+0000 7fc08f811640 1 -- 172.21.15.117:0/3397434219 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0881a89f0 con 0x7fc088102a80 2024-09-21T06:08:30.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.678+0000 7fc07e7fc640 1 -- 172.21.15.117:0/3397434219 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc084003cb0 con 0x7fc088102a80 2024-09-21T06:08:30.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.678+0000 7fc07e7fc640 1 -- 172.21.15.117:0/3397434219 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc084035070 con 0x7fc088102a80 2024-09-21T06:08:30.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.678+0000 7fc08f811640 1 -- 172.21.15.117:0/3397434219 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0881a8f20 con 0x7fc088102a80 2024-09-21T06:08:30.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.681+0000 7fc07e7fc640 1 -- 172.21.15.117:0/3397434219 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fc084042030 con 0x7fc088102a80 2024-09-21T06:08:30.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.681+0000 7fc095d85740 1 Processor -- start 2024-09-21T06:08:30.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.681+0000 7fc095d85740 1 -- start start 2024-09-21T06:08:30.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.681+0000 7fc095d85740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba231ab070 0x55ba23552650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:30.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.681+0000 7fc095d85740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba231821b0 0x55ba23552b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:30.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.681+0000 7fc095d85740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba2318a390 0x55ba235530d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:30.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.681+0000 7fc095d85740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ba234c8e50 con 0x55ba231821b0 2024-09-21T06:08:30.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.681+0000 7fc095d85740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ba234b8280 con 0x55ba231ab070 2024-09-21T06:08:30.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.681+0000 7fc095d85740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55ba234bc090 con 0x55ba2318a390 2024-09-21T06:08:30.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.682+0000 7fc08d5ad640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba231821b0 0x55ba23552b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:30.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.682+0000 7fc08d5ad640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba231821b0 0x55ba23552b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39488/0 (socket says 172.21.15.117:39488) 2024-09-21T06:08:30.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.682+0000 7fc08d5ad640 1 -- 172.21.15.117:0/990824180 learned_addr learned my addr 172.21.15.117:0/990824180 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:30.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.682+0000 7fc08cdac640 1 --2- 172.21.15.117:0/990824180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba2318a390 0x55ba235530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:30.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.682+0000 7fc08ddae640 1 --2- 172.21.15.117:0/990824180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba231ab070 0x55ba23552650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:30.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.682+0000 7fc07e7fc640 1 --2- 172.21.15.117:0/3397434219 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc0740b6680 0x7fc0740b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:30.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.682+0000 7fc08d5ad640 1 -- 172.21.15.117:0/990824180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba2318a390 msgr2=0x55ba235530d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:30.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.682+0000 7fc08d5ad640 1 --2- 172.21.15.117:0/990824180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba2318a390 0x55ba235530d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:30.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.682+0000 7fc08d5ad640 1 -- 172.21.15.117:0/990824180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba231ab070 msgr2=0x55ba23552650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:30.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.682+0000 7fc08d5ad640 1 --2- 172.21.15.117:0/990824180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba231ab070 0x55ba23552650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:30.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.682+0000 7fc08d5ad640 1 -- 172.21.15.117:0/990824180 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba23216650 con 0x55ba231821b0 2024-09-21T06:08:30.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.682+0000 7fc08ddae640 1 --2- 172.21.15.117:0/3397434219 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc0740b6680 0x7fc0740b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:30.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.682+0000 7fc07e7fc640 1 -- 172.21.15.117:0/3397434219 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7fc084108070 con 0x7fc088102a80 2024-09-21T06:08:30.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.682+0000 7fc08d5ad640 1 --2- 172.21.15.117:0/990824180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba231821b0 0x55ba23552b90 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fc084105780 tx=0x7fc0840c2770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac197b84c9cb5b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:30.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.682+0000 7fc0737fe640 1 -- 172.21.15.117:0/990824180 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0840cd860 con 0x55ba231821b0 2024-09-21T06:08:30.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.683+0000 7fc0737fe640 1 -- 172.21.15.117:0/990824180 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc08403b460 con 0x55ba231821b0 2024-09-21T06:08:30.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.683+0000 7fc08ddae640 1 --2- 172.21.15.117:0/3397434219 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc0740b6680 0x7fc0740b8b90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc08000af00 tx=0x7fc080004a40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:30.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.683+0000 7fc0737fe640 1 -- 172.21.15.117:0/990824180 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc084003e60 con 0x55ba231821b0 2024-09-21T06:08:30.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.683+0000 7fc095d85740 1 -- 172.21.15.117:0/990824180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba231821b0 msgr2=0x55ba23552b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:30.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.683+0000 7fc095d85740 1 --2- 172.21.15.117:0/990824180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba231821b0 0x55ba23552b90 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fc084105780 tx=0x7fc0840c2770 comp rx=0 tx=0).stop 2024-09-21T06:08:30.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.683+0000 7fc095d85740 1 -- 172.21.15.117:0/990824180 shutdown_connections 2024-09-21T06:08:30.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.683+0000 7fc095d85740 1 --2- 172.21.15.117:0/990824180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba231821b0 0x55ba23552b90 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:30.695 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.683+0000 7fc095d85740 1 --2- 172.21.15.117:0/990824180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba2318a390 0x55ba235530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:30.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.683+0000 7fc095d85740 1 --2- 172.21.15.117:0/990824180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba231ab070 0x55ba23552650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:30.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.683+0000 7fc095d85740 1 -- 172.21.15.117:0/990824180 >> 172.21.15.117:0/990824180 conn(0x55ba2319f460 msgr2=0x55ba23550540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:30.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.683+0000 7fc095d85740 1 -- 172.21.15.117:0/990824180 shutdown_connections 2024-09-21T06:08:30.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.683+0000 7fc095d85740 1 -- 172.21.15.117:0/990824180 wait complete. 2024-09-21T06:08:30.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.684+0000 7fc095d85740 1 -- start start 2024-09-21T06:08:30.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.684+0000 7fc095d85740 1 Processor -- start 2024-09-21T06:08:30.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.684+0000 7fc095d85740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba231821b0 0x55ba2355dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:30.696 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.684+0000 7fc095d85740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba2318a390 0x55ba2355e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:30.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.684+0000 7fc095d85740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba231ab070 0x55ba23569e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:30.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.684+0000 7fc095d85740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ba23319700 con 0x55ba231ab070 2024-09-21T06:08:30.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.684+0000 7fc095d85740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ba23453110 con 0x55ba231821b0 2024-09-21T06:08:30.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.684+0000 7fc095d85740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55ba23560bf0 con 0x55ba2318a390 2024-09-21T06:08:30.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.684+0000 7fc095d85740 10 client.0 _ll_register_callbacks cb 0x55ba2346c990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:08:30.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.685+0000 7fc08ddae640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba231ab070 0x55ba23569e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:30.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.685+0000 7fc08ddae640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba231ab070 0x55ba23569e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39500/0 (socket says 172.21.15.117:39500) 2024-09-21T06:08:30.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.685+0000 7fc08ddae640 1 -- 172.21.15.117:0/1848406565 learned_addr learned my addr 172.21.15.117:0/1848406565 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:30.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.685+0000 7fc08cdac640 1 --2- 172.21.15.117:0/1848406565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba2318a390 0x55ba2355e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:30.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.685+0000 7fc08d5ad640 1 --2- 172.21.15.117:0/1848406565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba231821b0 0x55ba2355dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:30.697 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.685+0000 7fc08ddae640 1 -- 172.21.15.117:0/1848406565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba2318a390 msgr2=0x55ba2355e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:30.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.685+0000 7fc08ddae640 1 --2- 172.21.15.117:0/1848406565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba2318a390 0x55ba2355e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:30.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.685+0000 7fc08ddae640 1 -- 172.21.15.117:0/1848406565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba231821b0 msgr2=0x55ba2355dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:30.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.685+0000 7fc08ddae640 1 --2- 172.21.15.117:0/1848406565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba231821b0 0x55ba2355dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:30.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.685+0000 7fc08ddae640 1 -- 172.21.15.117:0/1848406565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba23480c40 con 0x55ba231ab070 2024-09-21T06:08:30.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.685+0000 7fc08ddae640 1 --2- 172.21.15.117:0/1848406565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba231ab070 0x55ba23569e50 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fc08000a780 tx=0x7fc080004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec8d5694b456d79 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:30.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.685+0000 7fc070ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:08:30.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.685+0000 7fc070ff9640 1 -- 172.21.15.117:0/1848406565 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08000fda0 con 0x55ba231ab070 2024-09-21T06:08:30.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.685+0000 7fc070ff9640 1 -- 172.21.15.117:0/1848406565 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7fc080025470 con 0x55ba231ab070 2024-09-21T06:08:30.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.685+0000 7fc095d85740 1 -- 172.21.15.117:0/1848406565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ba234cada0 con 0x55ba231ab070 2024-09-21T06:08:30.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.686+0000 7fc070ff9640 1 -- 172.21.15.117:0/1848406565 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc080025670 con 0x55ba231ab070 2024-09-21T06:08:30.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.686+0000 7fc095d85740 10 client.18069 fetch_fsmap learned FSMap version 29 2024-09-21T06:08:30.698 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.686+0000 7fc095d85740 1 -- 172.21.15.117:0/1848406565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ba2356c870 con 0x55ba231ab070 2024-09-21T06:08:30.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.686+0000 7fc070ff9640 1 -- 172.21.15.117:0/1848406565 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ba234cada0 con 0x55ba231ab070 2024-09-21T06:08:30.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.686+0000 7fc070ff9640 1 -- 172.21.15.117:0/1848406565 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc08002e840 con 0x55ba231ab070 2024-09-21T06:08:30.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.686+0000 7fc095d85740 10 client.18069 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:08:30.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.686+0000 7fc095d85740 10 client.18069 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:08:30.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.686+0000 7fc095d85740 20 client.18069 populate_metadata read hostname 'smithi117' 2024-09-21T06:08:30.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.686+0000 7fc095d85740 1 --2- 172.21.15.117:0/1848406565 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ba231699f0 0x55ba2356fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:30.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.686+0000 7fc095d85740 4 client.18069 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:08:30.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.686+0000 7fc095d85740 1 -- 172.21.15.117:0/1848406565 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba234c6850 con 0x55ba231699f0 2024-09-21T06:08:30.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.686+0000 7fc08d5ad640 1 --2- 172.21.15.117:0/1848406565 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ba231699f0 0x55ba2356fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:30.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.687+0000 7fc070ff9640 10 client.18069 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:08:30.699 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.687+0000 7fc08d5ad640 1 --2- 172.21.15.117:0/1848406565 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ba231699f0 0x55ba2356fd50 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adce8b996f26a658 server_cookie=1b99b320a1180881 in_seq=0 out_seq=0 2024-09-21T06:08:30.700 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.687+0000 7fc070ff9640 1 -- 172.21.15.117:0/1848406565 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ba234c6850 con 0x55ba231699f0 2024-09-21T06:08:30.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.687+0000 7fc070ff9640 10 client.18069 handle_command_reply: tid=0 2024-09-21T06:08:30.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.687+0000 7fc095d85740 1 client.18069 shutdown 2024-09-21T06:08:30.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.688+0000 7fc095d85740 1 -- 172.21.15.117:0/1848406565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba231ab070 msgr2=0x55ba23569e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:30.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.688+0000 7fc095d85740 1 --2- 172.21.15.117:0/1848406565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba231ab070 0x55ba23569e50 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fc08000a780 tx=0x7fc080004ab0 comp rx=0 tx=0).stop 2024-09-21T06:08:30.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.688+0000 7fc095d85740 1 -- 172.21.15.117:0/1848406565 shutdown_connections 2024-09-21T06:08:30.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.688+0000 7fc095d85740 1 --2- 172.21.15.117:0/1848406565 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ba231699f0 0x55ba2356fd50 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:30.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.688+0000 7fc095d85740 1 --2- 172.21.15.117:0/1848406565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba231ab070 0x55ba23569e50 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:30.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.688+0000 7fc070ff9640 0 client.18069 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:08:30.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.688+0000 7fc095d85740 1 --2- 172.21.15.117:0/1848406565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba2318a390 0x55ba2355e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:30.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.688+0000 7fc095d85740 1 --2- 172.21.15.117:0/1848406565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba231821b0 0x55ba2355dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:30.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.688+0000 7fc095d85740 1 -- 172.21.15.117:0/1848406565 >> 172.21.15.117:0/1848406565 conn(0x55ba2319f460 msgr2=0x55ba235557e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:30.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.688+0000 7fc095d85740 1 -- 172.21.15.117:0/1848406565 shutdown_connections 2024-09-21T06:08:30.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.688+0000 7fc095d85740 1 -- 172.21.15.117:0/1848406565 wait complete. 2024-09-21T06:08:30.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.688+0000 7fc095d85740 20 client.18069 trim_cache size 0 max 16384 2024-09-21T06:08:30.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.705+0000 7fc095d85740 1 Processor -- start 2024-09-21T06:08:30.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.705+0000 7fc095d85740 1 -- start start 2024-09-21T06:08:30.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.705+0000 7fc095d85740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba231699f0 0x55ba2355f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:30.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.705+0000 7fc095d85740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba231821b0 0x55ba23569d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:30.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.705+0000 7fc095d85740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba2318a390 0x55ba2355aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:30.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.705+0000 7fc095d85740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc0881b1810 con 0x55ba2318a390 2024-09-21T06:08:30.708 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.705+0000 7fc095d85740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc0881b3350 con 0x55ba231821b0 2024-09-21T06:08:30.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.705+0000 7fc095d85740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc0881b1ee0 con 0x55ba231699f0 2024-09-21T06:08:30.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.705+0000 7fc08ddae640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba2318a390 0x55ba2355aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:30.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.705+0000 7fc08ddae640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba2318a390 0x55ba2355aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39506/0 (socket says 172.21.15.117:39506) 2024-09-21T06:08:30.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.705+0000 7fc08ddae640 1 -- 172.21.15.117:0/554176716 learned_addr learned my addr 172.21.15.117:0/554176716 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:30.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.706+0000 7fc08cdac640 1 --2- 172.21.15.117:0/554176716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba231821b0 0x55ba23569d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:30.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.706+0000 7fc08ddae640 1 -- 172.21.15.117:0/554176716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba231699f0 msgr2=0x55ba2355f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:30.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.706+0000 7fc08ddae640 1 --2- 172.21.15.117:0/554176716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba231699f0 0x55ba2355f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:30.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.706+0000 7fc08ddae640 1 -- 172.21.15.117:0/554176716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba231821b0 msgr2=0x55ba23569d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:30.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.706+0000 7fc08ddae640 1 --2- 172.21.15.117:0/554176716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba231821b0 0x55ba23569d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:30.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.706+0000 7fc08ddae640 1 -- 172.21.15.117:0/554176716 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba235589a0 con 0x55ba2318a390 2024-09-21T06:08:30.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.706+0000 7fc08cdac640 1 --2- 172.21.15.117:0/554176716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba231821b0 0x55ba23569d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:08:30.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.706+0000 7fc08ddae640 1 --2- 172.21.15.117:0/554176716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba2318a390 0x55ba2355aef0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fc080000f80 tx=0x7fc080012b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37c475ffa75870c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:30.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.706+0000 7fc072ffd640 1 -- 172.21.15.117:0/554176716 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08000fba0 con 0x55ba2318a390 2024-09-21T06:08:30.709 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.706+0000 7fc072ffd640 1 -- 172.21.15.117:0/554176716 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc080020070 con 0x55ba2318a390 2024-09-21T06:08:30.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.706+0000 7fc072ffd640 1 -- 172.21.15.117:0/554176716 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc080025c10 con 0x55ba2318a390 2024-09-21T06:08:30.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.707+0000 7fc095d85740 1 -- 172.21.15.117:0/554176716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba2318a390 msgr2=0x55ba2355aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:30.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.707+0000 7fc095d85740 1 --2- 172.21.15.117:0/554176716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba2318a390 0x55ba2355aef0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fc080000f80 tx=0x7fc080012b70 comp rx=0 tx=0).stop 2024-09-21T06:08:30.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.707+0000 7fc095d85740 1 -- 172.21.15.117:0/554176716 shutdown_connections 2024-09-21T06:08:30.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.707+0000 7fc095d85740 1 --2- 172.21.15.117:0/554176716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba2318a390 0x55ba2355aef0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:30.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.707+0000 7fc095d85740 1 --2- 172.21.15.117:0/554176716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba231821b0 0x55ba23569d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:30.714 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.707+0000 7fc095d85740 1 --2- 172.21.15.117:0/554176716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba231699f0 0x55ba2355f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:30.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.707+0000 7fc095d85740 1 -- 172.21.15.117:0/554176716 >> 172.21.15.117:0/554176716 conn(0x55ba2319f460 msgr2=0x55ba2355f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:30.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.707+0000 7fc095d85740 1 -- 172.21.15.117:0/554176716 shutdown_connections 2024-09-21T06:08:30.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.707+0000 7fc095d85740 1 -- 172.21.15.117:0/554176716 wait complete. 2024-09-21T06:08:30.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.707+0000 7fc095d85740 1 -- start start 2024-09-21T06:08:30.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.707+0000 7fc095d85740 1 Processor -- start 2024-09-21T06:08:30.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.708+0000 7fc095d85740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba231699f0 0x55ba235621a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:30.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.708+0000 7fc095d85740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba231821b0 0x55ba235626e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:30.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.708+0000 7fc095d85740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba2318a390 0x55ba235d9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:30.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.708+0000 7fc095d85740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc0881b2c80 con 0x55ba231699f0 2024-09-21T06:08:30.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.708+0000 7fc095d85740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc0881b25b0 con 0x55ba231821b0 2024-09-21T06:08:30.715 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.708+0000 7fc095d85740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc0881af810 con 0x55ba2318a390 2024-09-21T06:08:30.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.708+0000 7fc095d85740 10 client.0 _ll_register_callbacks cb 0x7fc08000aba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:08:30.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.708+0000 7fc08d5ad640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba231699f0 0x55ba235621a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:30.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.708+0000 7fc08d5ad640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba231699f0 0x55ba235621a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39510/0 (socket says 172.21.15.117:39510) 2024-09-21T06:08:30.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.708+0000 7fc08d5ad640 1 -- 172.21.15.117:0/479525018 learned_addr learned my addr 172.21.15.117:0/479525018 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:30.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.708+0000 7fc08cdac640 1 --2- 172.21.15.117:0/479525018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba231821b0 0x55ba235626e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:30.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.708+0000 7fc08ddae640 1 --2- 172.21.15.117:0/479525018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba2318a390 0x55ba235d9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:30.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.708+0000 7fc08d5ad640 1 -- 172.21.15.117:0/479525018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba2318a390 msgr2=0x55ba235d9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:30.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.708+0000 7fc08d5ad640 1 --2- 172.21.15.117:0/479525018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba2318a390 0x55ba235d9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:30.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.708+0000 7fc08d5ad640 1 -- 172.21.15.117:0/479525018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba231821b0 msgr2=0x55ba235626e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:30.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.708+0000 7fc08d5ad640 1 --2- 172.21.15.117:0/479525018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba231821b0 0x55ba235626e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:30.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.708+0000 7fc08d5ad640 1 -- 172.21.15.117:0/479525018 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba235d9c90 con 0x55ba231699f0 2024-09-21T06:08:30.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.709+0000 7fc08d5ad640 1 --2- 172.21.15.117:0/479525018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba231699f0 0x55ba235621a0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fc0841058c0 tx=0x7fc0840b5040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5200af560ded0b92 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:30.716 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.709+0000 7fc070ff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:08:30.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.709+0000 7fc070ff9640 1 -- 172.21.15.117:0/479525018 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0840cd680 con 0x55ba231699f0 2024-09-21T06:08:30.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.709+0000 7fc070ff9640 1 -- 172.21.15.117:0/479525018 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7fc0840ca180 con 0x55ba231699f0 2024-09-21T06:08:30.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.709+0000 7fc095d85740 1 -- 172.21.15.117:0/479525018 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc080020070 con 0x55ba231699f0 2024-09-21T06:08:30.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.709+0000 7fc070ff9640 1 -- 172.21.15.117:0/479525018 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc08407d020 con 0x55ba231699f0 2024-09-21T06:08:30.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.709+0000 7fc095d85740 10 client.18075 fetch_fsmap learned FSMap version 29 2024-09-21T06:08:30.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.709+0000 7fc095d85740 1 -- 172.21.15.117:0/479525018 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ba235dad30 con 0x55ba231699f0 2024-09-21T06:08:30.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.709+0000 7fc070ff9640 1 -- 172.21.15.117:0/479525018 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc080020070 con 0x55ba231699f0 2024-09-21T06:08:30.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.709+0000 7fc070ff9640 1 -- 172.21.15.117:0/479525018 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc0840b3ce0 con 0x55ba231699f0 2024-09-21T06:08:30.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.709+0000 7fc095d85740 10 client.18075 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:08:30.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.709+0000 7fc095d85740 10 client.18075 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:08:30.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.709+0000 7fc095d85740 20 client.18075 populate_metadata read hostname 'smithi117' 2024-09-21T06:08:30.717 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.709+0000 7fc095d85740 1 --2- 172.21.15.117:0/479525018 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ba231ab070 0x55ba235dd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:30.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.709+0000 7fc095d85740 4 client.18075 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T06:08:30.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.709+0000 7fc095d85740 1 -- 172.21.15.117:0/479525018 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ba23550aa0 con 0x55ba231ab070 2024-09-21T06:08:30.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.710+0000 7fc08cdac640 1 --2- 172.21.15.117:0/479525018 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ba231ab070 0x55ba235dd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:30.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.710+0000 7fc08cdac640 1 --2- 172.21.15.117:0/479525018 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ba231ab070 0x55ba235dd480 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b614d3454a3e511 server_cookie=db70babb1e275e48 in_seq=0 out_seq=0 2024-09-21T06:08:30.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.710+0000 7fc070ff9640 10 client.18075 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:08:30.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.710+0000 7fc070ff9640 1 -- 172.21.15.117:0/479525018 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3782 (crc 0 0 0) 0x55ba23550aa0 con 0x55ba231ab070 2024-09-21T06:08:30.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.710+0000 7fc070ff9640 10 client.18075 handle_command_reply: tid=0 2024-09-21T06:08:30.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.710+0000 7fc095d85740 1 client.18075 shutdown 2024-09-21T06:08:30.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.710+0000 7fc095d85740 1 -- 172.21.15.117:0/479525018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba231699f0 msgr2=0x55ba235621a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:30.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.711+0000 7fc095d85740 1 --2- 172.21.15.117:0/479525018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba231699f0 0x55ba235621a0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fc0841058c0 tx=0x7fc0840b5040 comp rx=0 tx=0).stop 2024-09-21T06:08:30.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.711+0000 7fc095d85740 1 -- 172.21.15.117:0/479525018 shutdown_connections 2024-09-21T06:08:30.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.711+0000 7fc095d85740 1 --2- 172.21.15.117:0/479525018 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55ba231ab070 0x55ba235dd480 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:30.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.711+0000 7fc095d85740 1 --2- 172.21.15.117:0/479525018 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba2318a390 0x55ba235d9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:30.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.711+0000 7fc095d85740 1 --2- 172.21.15.117:0/479525018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba231821b0 0x55ba235626e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:30.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.711+0000 7fc070ff9640 0 client.18075 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:08:30.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.711+0000 7fc095d85740 1 --2- 172.21.15.117:0/479525018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ba231699f0 0x55ba235621a0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:30.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.711+0000 7fc095d85740 1 -- 172.21.15.117:0/479525018 >> 172.21.15.117:0/479525018 conn(0x55ba2319f460 msgr2=0x55ba2355cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:30.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.711+0000 7fc095d85740 1 -- 172.21.15.117:0/479525018 shutdown_connections 2024-09-21T06:08:30.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.711+0000 7fc095d85740 1 -- 172.21.15.117:0/479525018 wait complete. 2024-09-21T06:08:30.719 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.711+0000 7fc095d85740 20 client.18075 trim_cache size 0 max 16384 2024-09-21T06:08:30.719 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T06:08:30.719 INFO:teuthology.orchestra.run.smithi117.stdout: "status": "no active scrubs running", 2024-09-21T06:08:30.719 INFO:teuthology.orchestra.run.smithi117.stdout: "scrubs": {}, 2024-09-21T06:08:30.719 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_stats": { 2024-09-21T06:08:30.719 INFO:teuthology.orchestra.run.smithi117.stdout: "bcb8bc27-fdcf-48a3-819b-c98b933f1d97": { 2024-09-21T06:08:30.719 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:07:57.523337+0000", 2024-09-21T06:08:30.720 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:08:30.720 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:08:30.720 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:08:30.720 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:08:30.720 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:08:30.720 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:30.720 INFO:teuthology.orchestra.run.smithi117.stdout: "84be3416-2285-4046-adf2-8d335592d29f": { 2024-09-21T06:08:30.720 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:07:43.890616+0000", 2024-09-21T06:08:30.720 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:08:30.720 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:08:30.720 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:08:30.720 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:08:30.720 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:08:30.720 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:30.720 INFO:teuthology.orchestra.run.smithi117.stdout: "e99040eb-55d8-492b-ae21-f9ba41c42664": { 2024-09-21T06:08:30.721 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:07:29.750906+0000", 2024-09-21T06:08:30.721 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:08:30.721 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:08:30.721 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:08:30.721 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:08:30.721 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:08:30.721 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:30.721 INFO:teuthology.orchestra.run.smithi117.stdout: "26e3d90e-072a-4a4e-9661-4d004051865b": { 2024-09-21T06:08:30.721 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:06:29.219142+0000", 2024-09-21T06:08:30.721 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:08:30.721 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:08:30.721 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:08:30.721 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:08:30.721 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:08:30.721 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:30.722 INFO:teuthology.orchestra.run.smithi117.stdout: "cf43a1b2-838e-4d28-9ed6-ff0c726fd51a": { 2024-09-21T06:08:30.722 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:06:42.901863+0000", 2024-09-21T06:08:30.722 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:08:30.722 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:08:30.722 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:08:30.722 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:08:30.722 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:08:30.722 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:30.722 INFO:teuthology.orchestra.run.smithi117.stdout: "39eeca8a-6b4f-465f-af34-b54f77320d70": { 2024-09-21T06:08:30.722 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:05:41.497141+0000", 2024-09-21T06:08:30.723 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:08:30.723 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:08:30.723 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:08:30.723 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:08:30.723 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:08:30.723 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:30.723 INFO:teuthology.orchestra.run.smithi117.stdout: "29ff9b7c-49d9-4910-85be-bace7e940cd5": { 2024-09-21T06:08:30.723 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:03:03.775527+0000", 2024-09-21T06:08:30.723 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:08:30.723 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:08:30.723 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:08:30.723 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:08:30.723 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:08:30.723 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:30.723 INFO:teuthology.orchestra.run.smithi117.stdout: "0902e33a-ce33-45fb-92d8-2e6fcb8ec8a0": { 2024-09-21T06:08:30.724 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:04:11.365073+0000", 2024-09-21T06:08:30.724 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:08:30.724 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:08:30.724 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:08:30.724 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:08:30.724 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:08:30.724 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:30.724 INFO:teuthology.orchestra.run.smithi117.stdout: "d543b0ae-b710-4153-99da-fe5714839c1a": { 2024-09-21T06:08:30.724 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:01:20.574754+0000", 2024-09-21T06:08:30.724 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:08:30.724 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:08:30.724 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:08:30.724 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:08:30.725 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:08:30.725 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:30.725 INFO:teuthology.orchestra.run.smithi117.stdout: "ad80d9f1-6aff-46c0-91ab-2c1ab0681039": { 2024-09-21T06:08:30.725 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:59:31.949108+0000", 2024-09-21T06:08:30.725 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:08:30.725 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:08:30.725 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:08:30.725 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:08:30.725 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:08:30.725 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:30.725 INFO:teuthology.orchestra.run.smithi117.stdout: "fb2f217d-463b-4c8f-ab19-920b04bb7306": { 2024-09-21T06:08:30.725 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T06:04:56.308107+0000", 2024-09-21T06:08:30.725 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:08:30.725 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:08:30.725 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:08:30.726 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:08:30.726 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:08:30.726 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:30.726 INFO:teuthology.orchestra.run.smithi117.stdout: "8b3fe95a-b76b-4e94-afd0-72fe35e2dd92": { 2024-09-21T06:08:30.726 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:58:01.246076+0000", 2024-09-21T06:08:30.726 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:08:30.726 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:08:30.726 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:08:30.726 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:08:30.726 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:08:30.726 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:30.726 INFO:teuthology.orchestra.run.smithi117.stdout: "b30bfff0-0f6b-4e30-8467-ff7db421eba6": { 2024-09-21T06:08:30.726 INFO:teuthology.orchestra.run.smithi117.stdout: "start_time": "2024-09-21T05:57:14.939082+0000", 2024-09-21T06:08:30.726 INFO:teuthology.orchestra.run.smithi117.stdout: "path": "/", 2024-09-21T06:08:30.727 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:08:30.727 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_passed": 0, 2024-09-21T06:08:30.727 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_failed": 0, 2024-09-21T06:08:30.727 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_skipped": 0 2024-09-21T06:08:30.727 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:30.727 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:30.727 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T06:08:30.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.711+0000 7fc08f811640 1 -- 172.21.15.117:0/3397434219 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc0740b6680 msgr2=0x7fc0740b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:30.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.711+0000 7fc08f811640 1 --2- 172.21.15.117:0/3397434219 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc0740b6680 0x7fc0740b8b90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc08000af00 tx=0x7fc080004a40 comp rx=0 tx=0).stop 2024-09-21T06:08:30.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.711+0000 7fc08f811640 1 -- 172.21.15.117:0/3397434219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc088102a80 msgr2=0x7fc088104270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:30.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.711+0000 7fc08f811640 1 --2- 172.21.15.117:0/3397434219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc088102a80 0x7fc088104270 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fc084005940 tx=0x7fc0840042f0 comp rx=0 tx=0).stop 2024-09-21T06:08:30.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.712+0000 7fc08f811640 1 -- 172.21.15.117:0/3397434219 shutdown_connections 2024-09-21T06:08:30.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.712+0000 7fc08f811640 1 --2- 172.21.15.117:0/3397434219 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fc0740b6680 0x7fc0740b8b90 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:30.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.712+0000 7fc08f811640 1 --2- 172.21.15.117:0/3397434219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc088108d10 0x7fc088104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:30.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.712+0000 7fc08f811640 1 --2- 172.21.15.117:0/3397434219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0881083c0 0x7fc0881047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:30.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.712+0000 7fc08f811640 1 --2- 172.21.15.117:0/3397434219 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc088102a80 0x7fc088104270 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:30.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.712+0000 7fc08f811640 1 -- 172.21.15.117:0/3397434219 >> 172.21.15.117:0/3397434219 conn(0x7fc088079740 msgr2=0x7fc088101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:30.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.712+0000 7fc08f811640 1 -- 172.21.15.117:0/3397434219 shutdown_connections 2024-09-21T06:08:30.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:30.712+0000 7fc08f811640 1 -- 172.21.15.117:0/3397434219 wait complete. 2024-09-21T06:08:31.683 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T06:08:31.684 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:08:31.934 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:08:32.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:32 smithi117 ceph-mon[30947]: pgmap v672: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 152 op/s 2024-09-21T06:08:32.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:32 smithi138 ceph-mon[34391]: pgmap v672: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 152 op/s 2024-09-21T06:08:32.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.453+0000 7fadb595b640 1 -- 172.21.15.117:0/3671932990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadb0108cf0 msgr2=0x7fadb010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:32.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.453+0000 7fadb595b640 1 --2- 172.21.15.117:0/3671932990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadb0108cf0 0x7fadb010f520 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fada8009900 tx=0x7fada802a410 comp rx=0 tx=0).stop 2024-09-21T06:08:32.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.453+0000 7fadb595b640 1 -- 172.21.15.117:0/3671932990 shutdown_connections 2024-09-21T06:08:32.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.453+0000 7fadb595b640 1 --2- 172.21.15.117:0/3671932990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadb0108cf0 0x7fadb010f520 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:32.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.453+0000 7fadb595b640 1 --2- 172.21.15.117:0/3671932990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadb01083a0 0x7fadb01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:32.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.453+0000 7fadb595b640 1 --2- 172.21.15.117:0/3671932990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fadb0102a60 0x7fadb0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:32.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.453+0000 7fadb595b640 1 -- 172.21.15.117:0/3671932990 >> 172.21.15.117:0/3671932990 conn(0x7fadb0079730 msgr2=0x7fadb0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:32.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.453+0000 7fadb595b640 1 -- 172.21.15.117:0/3671932990 shutdown_connections 2024-09-21T06:08:32.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.453+0000 7fadb595b640 1 -- 172.21.15.117:0/3671932990 wait complete. 2024-09-21T06:08:32.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.454+0000 7fadb595b640 1 Processor -- start 2024-09-21T06:08:32.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.454+0000 7fadb595b640 1 -- start start 2024-09-21T06:08:32.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.454+0000 7fadb595b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadb0102a60 0x7fadb019b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:32.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.454+0000 7fadb595b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadb01083a0 0x7fadb019b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:32.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.454+0000 7fadb595b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fadb0108cf0 0x7fadb019bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:32.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.454+0000 7fadb595b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fadb01129c0 con 0x7fadb0102a60 2024-09-21T06:08:32.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.454+0000 7fadb595b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fadb0112840 con 0x7fadb01083a0 2024-09-21T06:08:32.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.454+0000 7fadb595b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fadb0112b40 con 0x7fadb0108cf0 2024-09-21T06:08:32.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.454+0000 7fadaeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadb0102a60 0x7fadb019b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:32.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.454+0000 7fadaeffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadb0102a60 0x7fadb019b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39536/0 (socket says 172.21.15.117:39536) 2024-09-21T06:08:32.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.454+0000 7fadaeffd640 1 -- 172.21.15.117:0/4171871696 learned_addr learned my addr 172.21.15.117:0/4171871696 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:32.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.454+0000 7fadae7fc640 1 --2- 172.21.15.117:0/4171871696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadb01083a0 0x7fadb019b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:32.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.454+0000 7fadaf7fe640 1 --2- 172.21.15.117:0/4171871696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fadb0108cf0 0x7fadb019bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:32.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.455+0000 7fadaeffd640 1 -- 172.21.15.117:0/4171871696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fadb0108cf0 msgr2=0x7fadb019bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:32.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.455+0000 7fadaeffd640 1 --2- 172.21.15.117:0/4171871696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fadb0108cf0 0x7fadb019bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:32.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.455+0000 7fadaeffd640 1 -- 172.21.15.117:0/4171871696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadb01083a0 msgr2=0x7fadb019b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:32.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.455+0000 7fadaeffd640 1 --2- 172.21.15.117:0/4171871696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadb01083a0 0x7fadb019b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:32.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.455+0000 7fadaeffd640 1 -- 172.21.15.117:0/4171871696 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb01a65a0 con 0x7fadb0102a60 2024-09-21T06:08:32.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.455+0000 7fadaeffd640 1 --2- 172.21.15.117:0/4171871696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadb0102a60 0x7fadb019b290 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fada000ba50 tx=0x7fada000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:32.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.455+0000 7fadb4959640 1 -- 172.21.15.117:0/4171871696 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada0019070 con 0x7fadb0102a60 2024-09-21T06:08:32.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.455+0000 7fadb595b640 1 -- 172.21.15.117:0/4171871696 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadb01a6890 con 0x7fadb0102a60 2024-09-21T06:08:32.467 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.455+0000 7fadb4959640 1 -- 172.21.15.117:0/4171871696 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fada000cbe0 con 0x7fadb0102a60 2024-09-21T06:08:32.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.455+0000 7fadb4959640 1 -- 172.21.15.117:0/4171871696 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada00075d0 con 0x7fadb0102a60 2024-09-21T06:08:32.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.455+0000 7fadb595b640 1 -- 172.21.15.117:0/4171871696 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadb01a6e50 con 0x7fadb0102a60 2024-09-21T06:08:32.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.456+0000 7fadb595b640 1 -- 172.21.15.117:0/4171871696 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadb0102e70 con 0x7fadb0102a60 2024-09-21T06:08:32.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.458+0000 7fadb4959640 1 -- 172.21.15.117:0/4171871696 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fada001c070 con 0x7fadb0102a60 2024-09-21T06:08:32.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.458+0000 7fadb4959640 1 --2- 172.21.15.117:0/4171871696 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fad9c0b6680 0x7fad9c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:32.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.459+0000 7fadae7fc640 1 --2- 172.21.15.117:0/4171871696 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fad9c0b6680 0x7fad9c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:32.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.459+0000 7fadb4959640 1 -- 172.21.15.117:0/4171871696 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7fada00e7100 con 0x7fadb0102a60 2024-09-21T06:08:32.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.459+0000 7fadae7fc640 1 --2- 172.21.15.117:0/4171871696 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fad9c0b6680 0x7fad9c0b8b90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fad98005fd0 tx=0x7fad98005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:32.468 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.462+0000 7fadb4959640 1 -- 172.21.15.117:0/4171871696 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fada00ad6a0 con 0x7fadb0102a60 2024-09-21T06:08:32.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:32 smithi192 ceph-mon[34700]: pgmap v672: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 152 op/s 2024-09-21T06:08:32.678 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.676+0000 7fadb595b640 1 -- 172.21.15.117:0/4171871696 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fadb006b2d0 con 0x7fadb0102a60 2024-09-21T06:08:32.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.677+0000 7fadb4959640 1 -- 172.21.15.117:0/4171871696 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+12001 (secure 0 0 0) 0x7fadb006b2d0 con 0x7fadb0102a60 2024-09-21T06:08:32.679 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T06:08:32.679 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":29,"btime":"2024-09-21T06:04:08:952483+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3227057504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3227057504},{"type":"v1","addr":"172.21.15.117:6835","nonce":3227057504}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3303829183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3303829183},{"type":"v1","addr":"172.21.15.117:6837","nonce":3303829183}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T05:55:14.555897+0000","modified":"2024-09-21T06:04:08.952466+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24638,"mds_1":24637,"mds_2":24628,"mds_3":24616,"mds_4":24602},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6839/2314572411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2314572411},{"type":"v1","addr":"172.21.15.117:6839","nonce":2314572411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6841/3378096561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":3378096561},{"type":"v1","addr":"172.21.15.117:6841","nonce":3378096561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/3725629862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3725629862},{"type":"v1","addr":"172.21.15.138:6833","nonce":3725629862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/4276801396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4276801396},{"type":"v1","addr":"172.21.15.138:6835","nonce":4276801396}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.138:6837/2540335374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2540335374},{"type":"v1","addr":"172.21.15.138:6837","nonce":2540335374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24616":{"gid":24616,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.192:6835/3530688860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3530688860},{"type":"v1","addr":"172.21.15.192:6835","nonce":3530688860}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.192:6837/2683860842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2683860842},{"type":"v1","addr":"172.21.15.192:6837","nonce":2683860842}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.192:6839/1727623231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1727623231},{"type":"v1","addr":"172.21.15.192:6839","nonce":1727623231}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/3230372576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":3230372576},{"type":"v1","addr":"172.21.15.138:6839","nonce":3230372576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25840":{"gid":25840,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6833/1596605058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1596605058},{"type":"v1","addr":"172.21.15.192:6833","nonce":1596605058}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24602,24628,24616,24637,24638]},"id":1}]} 2024-09-21T06:08:32.679 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 29 2024-09-21T06:08:32.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.680+0000 7fadb595b640 1 -- 172.21.15.117:0/4171871696 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fad9c0b6680 msgr2=0x7fad9c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:32.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.680+0000 7fadb595b640 1 --2- 172.21.15.117:0/4171871696 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fad9c0b6680 0x7fad9c0b8b90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fad98005fd0 tx=0x7fad98005850 comp rx=0 tx=0).stop 2024-09-21T06:08:32.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.680+0000 7fadb595b640 1 -- 172.21.15.117:0/4171871696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadb0102a60 msgr2=0x7fadb019b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:32.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.680+0000 7fadb595b640 1 --2- 172.21.15.117:0/4171871696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadb0102a60 0x7fadb019b290 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fada000ba50 tx=0x7fada000bf10 comp rx=0 tx=0).stop 2024-09-21T06:08:32.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.680+0000 7fadb595b640 1 -- 172.21.15.117:0/4171871696 shutdown_connections 2024-09-21T06:08:32.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.680+0000 7fadb595b640 1 --2- 172.21.15.117:0/4171871696 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fad9c0b6680 0x7fad9c0b8b90 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:32.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.680+0000 7fadb595b640 1 --2- 172.21.15.117:0/4171871696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fadb0108cf0 0x7fadb019bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:32.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.680+0000 7fadb595b640 1 --2- 172.21.15.117:0/4171871696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadb01083a0 0x7fadb019b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:32.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.680+0000 7fadb595b640 1 --2- 172.21.15.117:0/4171871696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadb0102a60 0x7fadb019b290 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:32.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.680+0000 7fadb595b640 1 -- 172.21.15.117:0/4171871696 >> 172.21.15.117:0/4171871696 conn(0x7fadb0079730 msgr2=0x7fadb01052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:32.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.680+0000 7fadb595b640 1 -- 172.21.15.117:0/4171871696 shutdown_connections 2024-09-21T06:08:32.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:32.680+0000 7fadb595b640 1 -- 172.21.15.117:0/4171871696 wait complete. 2024-09-21T06:08:33.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:33 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/4171871696' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:08:33.314 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:4 damage ls 2024-09-21T06:08:33.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:33 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/4171871696' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:08:33.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:33 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/4171871696' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:08:33.559 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:08:34.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.090+0000 7fed4b451640 1 -- 172.21.15.117:0/215479247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed44102f20 msgr2=0x7fed4410fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:34.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.090+0000 7fed4b451640 1 --2- 172.21.15.117:0/215479247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed44102f20 0x7fed4410fc40 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fed34009900 tx=0x7fed3402a480 comp rx=0 tx=0).stop 2024-09-21T06:08:34.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.090+0000 7fed4b451640 1 -- 172.21.15.117:0/215479247 shutdown_connections 2024-09-21T06:08:34.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.090+0000 7fed4b451640 1 --2- 172.21.15.117:0/215479247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed44102f20 0x7fed4410fc40 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.100 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.090+0000 7fed4b451640 1 --2- 172.21.15.117:0/215479247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed44110180 0x7fed441125a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.090+0000 7fed4b451640 1 --2- 172.21.15.117:0/215479247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed441025d0 0x7fed441029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.090+0000 7fed4b451640 1 -- 172.21.15.117:0/215479247 >> 172.21.15.117:0/215479247 conn(0x7fed44079730 msgr2=0x7fed44079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:34.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.090+0000 7fed4b451640 1 -- 172.21.15.117:0/215479247 shutdown_connections 2024-09-21T06:08:34.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.090+0000 7fed4b451640 1 -- 172.21.15.117:0/215479247 wait complete. 2024-09-21T06:08:34.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.091+0000 7fed4b451640 1 Processor -- start 2024-09-21T06:08:34.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.091+0000 7fed4b451640 1 -- start start 2024-09-21T06:08:34.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.091+0000 7fed4b451640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed441025d0 0x7fed4419c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:34.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.091+0000 7fed4b451640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed44102f20 0x7fed4419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:34.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.091+0000 7fed4b451640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed44110180 0x7fed441a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:34.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.091+0000 7fed4b451640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fed44115c70 con 0x7fed44102f20 2024-09-21T06:08:34.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.091+0000 7fed4b451640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fed44115af0 con 0x7fed441025d0 2024-09-21T06:08:34.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.091+0000 7fed4b451640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fed44115df0 con 0x7fed44110180 2024-09-21T06:08:34.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.091+0000 7fed489ec640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed44102f20 0x7fed4419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:34.101 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.091+0000 7fed489ec640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed44102f20 0x7fed4419cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39556/0 (socket says 172.21.15.117:39556) 2024-09-21T06:08:34.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.091+0000 7fed489ec640 1 -- 172.21.15.117:0/739953082 learned_addr learned my addr 172.21.15.117:0/739953082 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:34.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.091+0000 7fed499ee640 1 --2- 172.21.15.117:0/739953082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed44110180 0x7fed441a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:34.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.091+0000 7fed491ed640 1 --2- 172.21.15.117:0/739953082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed441025d0 0x7fed4419c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:34.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.091+0000 7fed489ec640 1 -- 172.21.15.117:0/739953082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed44110180 msgr2=0x7fed441a1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:34.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.091+0000 7fed489ec640 1 --2- 172.21.15.117:0/739953082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed44110180 0x7fed441a1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.091+0000 7fed489ec640 1 -- 172.21.15.117:0/739953082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed441025d0 msgr2=0x7fed4419c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:34.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.091+0000 7fed489ec640 1 --2- 172.21.15.117:0/739953082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed441025d0 0x7fed4419c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.091+0000 7fed489ec640 1 -- 172.21.15.117:0/739953082 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed441a26b0 con 0x7fed44102f20 2024-09-21T06:08:34.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.092+0000 7fed489ec640 1 --2- 172.21.15.117:0/739953082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed44102f20 0x7fed4419cce0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fed34005940 tx=0x7fed340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:34.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.092+0000 7fed3a7fc640 1 -- 172.21.15.117:0/739953082 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3403f030 con 0x7fed44102f20 2024-09-21T06:08:34.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.092+0000 7fed4b451640 1 -- 172.21.15.117:0/739953082 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed441a2940 con 0x7fed44102f20 2024-09-21T06:08:34.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.092+0000 7fed3a7fc640 1 -- 172.21.15.117:0/739953082 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed34003cb0 con 0x7fed44102f20 2024-09-21T06:08:34.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.092+0000 7fed3a7fc640 1 -- 172.21.15.117:0/739953082 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed34035070 con 0x7fed44102f20 2024-09-21T06:08:34.102 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.092+0000 7fed4b451640 1 -- 172.21.15.117:0/739953082 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed441a9100 con 0x7fed44102f20 2024-09-21T06:08:34.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.095+0000 7fed519c5740 1 Processor -- start 2024-09-21T06:08:34.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.095+0000 7fed519c5740 1 -- start start 2024-09-21T06:08:34.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.095+0000 7fed3a7fc640 1 -- 172.21.15.117:0/739953082 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fed34042030 con 0x7fed44102f20 2024-09-21T06:08:34.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.095+0000 7fed519c5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c532b1070 0x560c536582c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:34.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.095+0000 7fed519c5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c532881b0 0x560c53658800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:34.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.095+0000 7fed519c5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c53290390 0x560c53658d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:34.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.095+0000 7fed519c5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560c535ceea0 con 0x560c532b1070 2024-09-21T06:08:34.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.095+0000 7fed519c5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560c535be2d0 con 0x560c532881b0 2024-09-21T06:08:34.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.095+0000 7fed519c5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x560c535c20e0 con 0x560c53290390 2024-09-21T06:08:34.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.095+0000 7fed499ee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c532b1070 0x560c536582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:34.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.095+0000 7fed499ee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c532b1070 0x560c536582c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39566/0 (socket says 172.21.15.117:39566) 2024-09-21T06:08:34.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.095+0000 7fed499ee640 1 -- 172.21.15.117:0/237178593 learned_addr learned my addr 172.21.15.117:0/237178593 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:34.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.095+0000 7fed489ec640 1 --2- 172.21.15.117:0/237178593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c53290390 0x560c53658d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:34.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.095+0000 7fed491ed640 1 --2- 172.21.15.117:0/237178593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c532881b0 0x560c53658800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:34.103 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.095+0000 7fed499ee640 1 -- 172.21.15.117:0/237178593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c53290390 msgr2=0x560c53658d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:34.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.095+0000 7fed499ee640 1 --2- 172.21.15.117:0/237178593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c53290390 0x560c53658d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.095+0000 7fed499ee640 1 -- 172.21.15.117:0/237178593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c532881b0 msgr2=0x560c53658800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:34.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.095+0000 7fed499ee640 1 --2- 172.21.15.117:0/237178593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c532881b0 0x560c53658800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.095+0000 7fed499ee640 1 -- 172.21.15.117:0/237178593 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c5331c650 con 0x560c532b1070 2024-09-21T06:08:34.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.096+0000 7fed3a7fc640 1 --2- 172.21.15.117:0/739953082 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fed300b6680 0x7fed300b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:34.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.096+0000 7fed499ee640 1 --2- 172.21.15.117:0/237178593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c532b1070 0x560c536582c0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fed3c009f90 tx=0x7fed3c010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e9859994dfec10 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:34.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.096+0000 7fed491ed640 1 --2- 172.21.15.117:0/739953082 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fed300b6680 0x7fed300b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:34.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.096+0000 7fed2f7fe640 1 -- 172.21.15.117:0/237178593 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c011700 con 0x560c532b1070 2024-09-21T06:08:34.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.096+0000 7fed3a7fc640 1 -- 172.21.15.117:0/739953082 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7fed34107fb0 con 0x7fed44102f20 2024-09-21T06:08:34.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.096+0000 7fed2f7fe640 1 -- 172.21.15.117:0/237178593 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed3c011d50 con 0x560c532b1070 2024-09-21T06:08:34.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.096+0000 7fed2f7fe640 1 -- 172.21.15.117:0/237178593 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c019da0 con 0x560c532b1070 2024-09-21T06:08:34.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.097+0000 7fed519c5740 1 -- 172.21.15.117:0/237178593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c532b1070 msgr2=0x560c536582c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:34.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.097+0000 7fed519c5740 1 --2- 172.21.15.117:0/237178593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c532b1070 0x560c536582c0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fed3c009f90 tx=0x7fed3c010860 comp rx=0 tx=0).stop 2024-09-21T06:08:34.104 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.097+0000 7fed491ed640 1 --2- 172.21.15.117:0/739953082 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fed300b6680 0x7fed300b8b90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x560c53656f40 tx=0x7fed4000c430 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:34.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.097+0000 7fed519c5740 1 -- 172.21.15.117:0/237178593 shutdown_connections 2024-09-21T06:08:34.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.097+0000 7fed519c5740 1 --2- 172.21.15.117:0/237178593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c53290390 0x560c53658d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.097+0000 7fed519c5740 1 --2- 172.21.15.117:0/237178593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c532881b0 0x560c53658800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.097+0000 7fed519c5740 1 --2- 172.21.15.117:0/237178593 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c532b1070 0x560c536582c0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.097+0000 7fed519c5740 1 -- 172.21.15.117:0/237178593 >> 172.21.15.117:0/237178593 conn(0x560c532a5460 msgr2=0x560c536561b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:34.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.097+0000 7fed519c5740 1 -- 172.21.15.117:0/237178593 shutdown_connections 2024-09-21T06:08:34.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.097+0000 7fed519c5740 1 -- 172.21.15.117:0/237178593 wait complete. 2024-09-21T06:08:34.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.097+0000 7fed519c5740 1 -- start start 2024-09-21T06:08:34.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.098+0000 7fed519c5740 1 Processor -- start 2024-09-21T06:08:34.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.098+0000 7fed519c5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c532881b0 0x560c536639c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:34.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.098+0000 7fed519c5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c53290390 0x560c53663f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:34.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.098+0000 7fed519c5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c532b1070 0x560c5366fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:34.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.098+0000 7fed519c5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560c5341f700 con 0x560c532b1070 2024-09-21T06:08:34.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.098+0000 7fed519c5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560c53559160 con 0x560c532881b0 2024-09-21T06:08:34.105 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.098+0000 7fed519c5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x560c53666aa0 con 0x560c53290390 2024-09-21T06:08:34.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.098+0000 7fed499ee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c532b1070 0x560c5366fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:34.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.098+0000 7fed499ee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c532b1070 0x560c5366fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39574/0 (socket says 172.21.15.117:39574) 2024-09-21T06:08:34.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.098+0000 7fed499ee640 1 -- 172.21.15.117:0/1504056285 learned_addr learned my addr 172.21.15.117:0/1504056285 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:34.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.098+0000 7fed491ed640 1 --2- 172.21.15.117:0/1504056285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c532881b0 0x560c536639c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:34.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.098+0000 7fed519c5740 10 client.0 _ll_register_callbacks cb 0x560c535729e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:08:34.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.098+0000 7fed489ec640 1 --2- 172.21.15.117:0/1504056285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c53290390 0x560c53663f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:34.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.099+0000 7fed491ed640 1 -- 172.21.15.117:0/1504056285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c53290390 msgr2=0x560c53663f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:34.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.099+0000 7fed491ed640 1 --2- 172.21.15.117:0/1504056285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c53290390 0x560c53663f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.099+0000 7fed491ed640 1 -- 172.21.15.117:0/1504056285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c532b1070 msgr2=0x560c5366fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:34.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.099+0000 7fed491ed640 1 --2- 172.21.15.117:0/1504056285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c532b1070 0x560c5366fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.099+0000 7fed491ed640 1 -- 172.21.15.117:0/1504056285 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c53586c90 con 0x560c532881b0 2024-09-21T06:08:34.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.099+0000 7fed499ee640 1 --2- 172.21.15.117:0/1504056285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c532b1070 0x560c5366fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:08:34.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.099+0000 7fed491ed640 1 --2- 172.21.15.117:0/1504056285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c532881b0 0x560c536639c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fed400058c0 tx=0x7fed400046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18d04f8244e71e54 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:34.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.099+0000 7fed2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-09-21T06:08:34.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.099+0000 7fed2cff9640 1 -- 172.21.15.117:0/1504056285 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed40029060 con 0x560c532881b0 2024-09-21T06:08:34.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.099+0000 7fed2cff9640 1 -- 172.21.15.117:0/1504056285 <== mon.2 v2:172.21.15.138:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7fed4000cd40 con 0x560c532881b0 2024-09-21T06:08:34.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.099+0000 7fed519c5740 1 -- 172.21.15.117:0/1504056285 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed3c011d50 con 0x560c532881b0 2024-09-21T06:08:34.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.099+0000 7fed2cff9640 1 -- 172.21.15.117:0/1504056285 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fed4001a460 con 0x560c532881b0 2024-09-21T06:08:34.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.099+0000 7fed519c5740 10 client.26996 fetch_fsmap learned FSMap version 29 2024-09-21T06:08:34.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.099+0000 7fed519c5740 1 -- 172.21.15.117:0/1504056285 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560c53672800 con 0x560c532881b0 2024-09-21T06:08:34.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.100+0000 7fed2cff9640 1 -- 172.21.15.117:0/1504056285 <== mon.2 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed3c011d50 con 0x560c532881b0 2024-09-21T06:08:34.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.100+0000 7fed2cff9640 1 -- 172.21.15.117:0/1504056285 <== mon.2 v2:172.21.15.138:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fed40018040 con 0x560c532881b0 2024-09-21T06:08:34.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.100+0000 7fed519c5740 10 client.26996 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:08:34.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.100+0000 7fed519c5740 10 client.26996 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T06:08:34.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.100+0000 7fed519c5740 20 client.26996 populate_metadata read hostname 'smithi117' 2024-09-21T06:08:34.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.100+0000 7fed519c5740 1 --2- 172.21.15.117:0/1504056285 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x560c5326f9f0 0x560c53675d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:34.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.100+0000 7fed519c5740 4 client.26996 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:08:34.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.100+0000 7fed519c5740 1 -- 172.21.15.117:0/1504056285 --> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c535d0df0 con 0x560c5326f9f0 2024-09-21T06:08:34.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.100+0000 7fed489ec640 1 --2- 172.21.15.117:0/1504056285 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x560c5326f9f0 0x560c53675d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:34.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.101+0000 7fed489ec640 1 --2- 172.21.15.117:0/1504056285 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x560c5326f9f0 0x560c53675d50 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c5c218f4fce3f129 server_cookie=69263530f2452744 in_seq=0 out_seq=0 2024-09-21T06:08:34.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.101+0000 7fed2cff9640 10 client.26996 ms_handle_connect on v2:172.21.15.138:6836/2540335374 2024-09-21T06:08:34.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.101+0000 7fed2cff9640 1 -- 172.21.15.117:0/1504056285 <== mds.4 v2:172.21.15.138:6836/2540335374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560c535d0df0 con 0x560c5326f9f0 2024-09-21T06:08:34.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.101+0000 7fed2cff9640 10 client.26996 handle_command_reply: tid=0 2024-09-21T06:08:34.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.101+0000 7fed519c5740 1 client.26996 shutdown 2024-09-21T06:08:34.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.102+0000 7fed519c5740 1 -- 172.21.15.117:0/1504056285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c532881b0 msgr2=0x560c536639c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:34.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.102+0000 7fed519c5740 1 --2- 172.21.15.117:0/1504056285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c532881b0 0x560c536639c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fed400058c0 tx=0x7fed400046c0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.102+0000 7fed519c5740 1 -- 172.21.15.117:0/1504056285 shutdown_connections 2024-09-21T06:08:34.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.102+0000 7fed519c5740 1 --2- 172.21.15.117:0/1504056285 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x560c5326f9f0 0x560c53675d50 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.102+0000 7fed519c5740 1 --2- 172.21.15.117:0/1504056285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c532b1070 0x560c5366fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.102+0000 7fed519c5740 1 --2- 172.21.15.117:0/1504056285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c53290390 0x560c53663f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.102+0000 7fed519c5740 1 --2- 172.21.15.117:0/1504056285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c532881b0 0x560c536639c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.102+0000 7fed2cff9640 0 client.26996 ms_handle_reset on v2:172.21.15.138:6836/2540335374 2024-09-21T06:08:34.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.102+0000 7fed519c5740 1 -- 172.21.15.117:0/1504056285 >> 172.21.15.117:0/1504056285 conn(0x560c532a5460 msgr2=0x560c5365b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:34.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.102+0000 7fed519c5740 1 -- 172.21.15.117:0/1504056285 shutdown_connections 2024-09-21T06:08:34.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.102+0000 7fed519c5740 1 -- 172.21.15.117:0/1504056285 wait complete. 2024-09-21T06:08:34.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.102+0000 7fed519c5740 20 client.26996 trim_cache size 0 max 16384 2024-09-21T06:08:34.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.115+0000 7fed519c5740 1 Processor -- start 2024-09-21T06:08:34.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.115+0000 7fed519c5740 1 -- start start 2024-09-21T06:08:34.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.115+0000 7fed519c5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c532881b0 0x560c53663e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:34.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.115+0000 7fed519c5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c53290390 0x560c5366fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:34.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.115+0000 7fed519c5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c532b1070 0x560c53660bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:34.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.115+0000 7fed519c5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fed441b1850 con 0x560c532881b0 2024-09-21T06:08:34.119 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.115+0000 7fed519c5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fed441b3390 con 0x560c53290390 2024-09-21T06:08:34.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.115+0000 7fed519c5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fed441b1f20 con 0x560c532b1070 2024-09-21T06:08:34.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.115+0000 7fed491ed640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c532881b0 0x560c53663e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:34.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.115+0000 7fed491ed640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c532881b0 0x560c53663e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39588/0 (socket says 172.21.15.117:39588) 2024-09-21T06:08:34.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.115+0000 7fed491ed640 1 -- 172.21.15.117:0/56783392 learned_addr learned my addr 172.21.15.117:0/56783392 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:34.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.116+0000 7fed499ee640 1 --2- 172.21.15.117:0/56783392 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c532b1070 0x560c53660bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:34.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.116+0000 7fed489ec640 1 --2- 172.21.15.117:0/56783392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c53290390 0x560c5366fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:34.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.116+0000 7fed491ed640 1 -- 172.21.15.117:0/56783392 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c532b1070 msgr2=0x560c53660bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:34.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.116+0000 7fed491ed640 1 --2- 172.21.15.117:0/56783392 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c532b1070 0x560c53660bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.116+0000 7fed491ed640 1 -- 172.21.15.117:0/56783392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c53290390 msgr2=0x560c5366fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:34.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.116+0000 7fed491ed640 1 --2- 172.21.15.117:0/56783392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c53290390 0x560c5366fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.116+0000 7fed491ed640 1 -- 172.21.15.117:0/56783392 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c5365a5b0 con 0x560c532881b0 2024-09-21T06:08:34.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.116+0000 7fed489ec640 1 --2- 172.21.15.117:0/56783392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c53290390 0x560c5366fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:08:34.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.116+0000 7fed491ed640 1 --2- 172.21.15.117:0/56783392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c532881b0 0x560c53663e80 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fed4000ca60 tx=0x7fed4000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a481daff009c80a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:34.120 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.116+0000 7fed2effd640 1 -- 172.21.15.117:0/56783392 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed400249f0 con 0x560c532881b0 2024-09-21T06:08:34.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.116+0000 7fed2effd640 1 -- 172.21.15.117:0/56783392 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed40024b90 con 0x560c532881b0 2024-09-21T06:08:34.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.116+0000 7fed2effd640 1 -- 172.21.15.117:0/56783392 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4002d660 con 0x560c532881b0 2024-09-21T06:08:34.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.117+0000 7fed519c5740 1 -- 172.21.15.117:0/56783392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c532881b0 msgr2=0x560c53663e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:34.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.117+0000 7fed519c5740 1 --2- 172.21.15.117:0/56783392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c532881b0 0x560c53663e80 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fed4000ca60 tx=0x7fed4000cf50 comp rx=0 tx=0).stop 2024-09-21T06:08:34.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.117+0000 7fed519c5740 1 -- 172.21.15.117:0/56783392 shutdown_connections 2024-09-21T06:08:34.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.117+0000 7fed519c5740 1 --2- 172.21.15.117:0/56783392 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c532b1070 0x560c53660bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.117+0000 7fed519c5740 1 --2- 172.21.15.117:0/56783392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c53290390 0x560c5366fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.117+0000 7fed519c5740 1 --2- 172.21.15.117:0/56783392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c532881b0 0x560c53663e80 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.117+0000 7fed519c5740 1 -- 172.21.15.117:0/56783392 >> 172.21.15.117:0/56783392 conn(0x560c532a5460 msgr2=0x560c53663770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:34.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.117+0000 7fed519c5740 1 -- 172.21.15.117:0/56783392 shutdown_connections 2024-09-21T06:08:34.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.117+0000 7fed519c5740 1 -- 172.21.15.117:0/56783392 wait complete. 2024-09-21T06:08:34.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.117+0000 7fed519c5740 1 -- start start 2024-09-21T06:08:34.121 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.117+0000 7fed519c5740 1 Processor -- start 2024-09-21T06:08:34.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.118+0000 7fed519c5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c532881b0 0x560c5366c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:34.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.118+0000 7fed519c5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c53290390 0x560c536678e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:34.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.118+0000 7fed519c5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c532b1070 0x560c53667e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:34.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.118+0000 7fed519c5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fed441b2cc0 con 0x560c532881b0 2024-09-21T06:08:34.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.118+0000 7fed519c5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fed441b25f0 con 0x560c53290390 2024-09-21T06:08:34.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.118+0000 7fed519c5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fed441af850 con 0x560c532b1070 2024-09-21T06:08:34.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.118+0000 7fed519c5740 10 client.0 _ll_register_callbacks cb 0x7fed4000c990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:08:34.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.118+0000 7fed491ed640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c532881b0 0x560c5366c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:34.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.118+0000 7fed491ed640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c532881b0 0x560c5366c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39592/0 (socket says 172.21.15.117:39592) 2024-09-21T06:08:34.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.118+0000 7fed491ed640 1 -- 172.21.15.117:0/585010441 learned_addr learned my addr 172.21.15.117:0/585010441 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:34.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.118+0000 7fed489ec640 1 --2- 172.21.15.117:0/585010441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c53290390 0x560c536678e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:34.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.118+0000 7fed499ee640 1 --2- 172.21.15.117:0/585010441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c532b1070 0x560c53667e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:34.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.118+0000 7fed491ed640 1 -- 172.21.15.117:0/585010441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c532b1070 msgr2=0x560c53667e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:34.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.118+0000 7fed491ed640 1 --2- 172.21.15.117:0/585010441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c532b1070 0x560c53667e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.118+0000 7fed491ed640 1 -- 172.21.15.117:0/585010441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c53290390 msgr2=0x560c536678e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:34.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.118+0000 7fed491ed640 1 --2- 172.21.15.117:0/585010441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c53290390 0x560c536678e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.118+0000 7fed491ed640 1 -- 172.21.15.117:0/585010441 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c53668510 con 0x560c532881b0 2024-09-21T06:08:34.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.119+0000 7fed491ed640 1 --2- 172.21.15.117:0/585010441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c532881b0 0x560c5366c430 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fed400246a0 tx=0x7fed4001af20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f40adf7e1b85e0e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:34.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.119+0000 7fed2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:08:34.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.119+0000 7fed2cff9640 1 -- 172.21.15.117:0/585010441 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4003d070 con 0x560c532881b0 2024-09-21T06:08:34.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.119+0000 7fed2cff9640 1 -- 172.21.15.117:0/585010441 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7fed40004020 con 0x560c532881b0 2024-09-21T06:08:34.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.119+0000 7fed519c5740 1 -- 172.21.15.117:0/585010441 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed40024b90 con 0x560c532881b0 2024-09-21T06:08:34.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.119+0000 7fed2cff9640 1 -- 172.21.15.117:0/585010441 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fed40029020 con 0x560c532881b0 2024-09-21T06:08:34.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.119+0000 7fed519c5740 10 client.18099 fetch_fsmap learned FSMap version 29 2024-09-21T06:08:34.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.119+0000 7fed519c5740 1 -- 172.21.15.117:0/585010441 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560c536e0520 con 0x560c532881b0 2024-09-21T06:08:34.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.119+0000 7fed2cff9640 1 -- 172.21.15.117:0/585010441 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed40024b90 con 0x560c532881b0 2024-09-21T06:08:34.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.119+0000 7fed2cff9640 1 -- 172.21.15.117:0/585010441 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fed40018050 con 0x560c532881b0 2024-09-21T06:08:34.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.119+0000 7fed519c5740 10 client.18099 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:08:34.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.119+0000 7fed519c5740 10 client.18099 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-21T06:08:34.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.119+0000 7fed519c5740 20 client.18099 populate_metadata read hostname 'smithi117' 2024-09-21T06:08:34.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.119+0000 7fed519c5740 1 --2- 172.21.15.117:0/585010441 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x560c53244da0 0x560c536e2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:34.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.119+0000 7fed519c5740 4 client.18099 mds_command: new command op to 24602 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:08:34.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.119+0000 7fed519c5740 1 -- 172.21.15.117:0/585010441 --> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560c535cc8a0 con 0x560c53244da0 2024-09-21T06:08:34.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.120+0000 7fed489ec640 1 --2- 172.21.15.117:0/585010441 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x560c53244da0 0x560c536e2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:34.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.120+0000 7fed2cff9640 10 client.18099 ms_handle_connect on v2:172.21.15.138:6836/2540335374 2024-09-21T06:08:34.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.120+0000 7fed489ec640 1 --2- 172.21.15.117:0/585010441 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x560c53244da0 0x560c536e2bd0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=221173e2c7590d37 server_cookie=b465bd26b5366882 in_seq=0 out_seq=0 2024-09-21T06:08:34.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.120+0000 7fed2cff9640 1 -- 172.21.15.117:0/585010441 <== mds.4 v2:172.21.15.138:6836/2540335374 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560c535cc8a0 con 0x560c53244da0 2024-09-21T06:08:34.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.120+0000 7fed2cff9640 10 client.18099 handle_command_reply: tid=0 2024-09-21T06:08:34.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.120+0000 7fed519c5740 1 client.18099 shutdown 2024-09-21T06:08:34.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.120+0000 7fed519c5740 1 -- 172.21.15.117:0/585010441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c532881b0 msgr2=0x560c5366c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:34.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.120+0000 7fed519c5740 1 --2- 172.21.15.117:0/585010441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c532881b0 0x560c5366c430 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fed400246a0 tx=0x7fed4001af20 comp rx=0 tx=0).stop 2024-09-21T06:08:34.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.121+0000 7fed519c5740 1 -- 172.21.15.117:0/585010441 shutdown_connections 2024-09-21T06:08:34.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.121+0000 7fed519c5740 1 --2- 172.21.15.117:0/585010441 >> [v2:172.21.15.138:6836/2540335374,v1:172.21.15.138:6837/2540335374] conn(0x560c53244da0 0x560c536e2bd0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.121+0000 7fed519c5740 1 --2- 172.21.15.117:0/585010441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c532b1070 0x560c53667e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.121+0000 7fed519c5740 1 --2- 172.21.15.117:0/585010441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560c53290390 0x560c536678e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.121+0000 7fed519c5740 1 --2- 172.21.15.117:0/585010441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560c532881b0 0x560c5366c430 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.121+0000 7fed519c5740 1 -- 172.21.15.117:0/585010441 >> 172.21.15.117:0/585010441 conn(0x560c532a5460 msgr2=0x560c536592f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:34.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.121+0000 7fed2cff9640 0 client.18099 ms_handle_reset on v2:172.21.15.138:6836/2540335374 2024-09-21T06:08:34.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.121+0000 7fed519c5740 1 -- 172.21.15.117:0/585010441 shutdown_connections 2024-09-21T06:08:34.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.121+0000 7fed519c5740 1 -- 172.21.15.117:0/585010441 wait complete. 2024-09-21T06:08:34.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.121+0000 7fed519c5740 20 client.18099 trim_cache size 0 max 16384 2024-09-21T06:08:34.128 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:08:34.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.121+0000 7fed4b451640 1 -- 172.21.15.117:0/739953082 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fed300b6680 msgr2=0x7fed300b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:34.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.121+0000 7fed4b451640 1 --2- 172.21.15.117:0/739953082 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fed300b6680 0x7fed300b8b90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x560c53656f40 tx=0x7fed4000c430 comp rx=0 tx=0).stop 2024-09-21T06:08:34.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.121+0000 7fed4b451640 1 -- 172.21.15.117:0/739953082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed44102f20 msgr2=0x7fed4419cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:34.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.121+0000 7fed4b451640 1 --2- 172.21.15.117:0/739953082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed44102f20 0x7fed4419cce0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fed34005940 tx=0x7fed340042f0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.122+0000 7fed4b451640 1 -- 172.21.15.117:0/739953082 shutdown_connections 2024-09-21T06:08:34.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.122+0000 7fed4b451640 1 --2- 172.21.15.117:0/739953082 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fed300b6680 0x7fed300b8b90 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.122+0000 7fed4b451640 1 --2- 172.21.15.117:0/739953082 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed44102f20 0x7fed4419cce0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.122+0000 7fed4b451640 1 --2- 172.21.15.117:0/739953082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed44110180 0x7fed441a1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.122+0000 7fed4b451640 1 --2- 172.21.15.117:0/739953082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed441025d0 0x7fed4419c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:34.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.122+0000 7fed4b451640 1 -- 172.21.15.117:0/739953082 >> 172.21.15.117:0/739953082 conn(0x7fed44079730 msgr2=0x7fed44109db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:34.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.122+0000 7fed4b451640 1 -- 172.21.15.117:0/739953082 shutdown_connections 2024-09-21T06:08:34.129 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:34.122+0000 7fed4b451640 1 -- 172.21.15.117:0/739953082 wait complete. 2024-09-21T06:08:34.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:34 smithi117 ceph-mon[30947]: pgmap v673: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 203 op/s 2024-09-21T06:08:34.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:34 smithi138 ceph-mon[34391]: pgmap v673: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 203 op/s 2024-09-21T06:08:34.479 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:34 smithi192 ceph-mon[34700]: pgmap v673: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 203 op/s 2024-09-21T06:08:34.818 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:3 damage ls 2024-09-21T06:08:35.065 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:08:35.186 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:35 smithi117 ceph-mon[30947]: pgmap v674: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 24 KiB/s wr, 230 op/s 2024-09-21T06:08:35.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:35 smithi138 ceph-mon[34391]: pgmap v674: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 24 KiB/s wr, 230 op/s 2024-09-21T06:08:35.479 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:35 smithi192 ceph-mon[34700]: pgmap v674: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 24 KiB/s wr, 230 op/s 2024-09-21T06:08:35.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.548+0000 7f921c324640 1 -- 172.21.15.117:0/2202672211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9214102a80 msgr2=0x7f9214102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:35.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.548+0000 7f921c324640 1 --2- 172.21.15.117:0/2202672211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9214102a80 0x7f9214102e90 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f9210009900 tx=0x7f921002a410 comp rx=0 tx=0).stop 2024-09-21T06:08:35.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.548+0000 7f921c324640 1 -- 172.21.15.117:0/2202672211 shutdown_connections 2024-09-21T06:08:35.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.548+0000 7f921c324640 1 --2- 172.21.15.117:0/2202672211 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9214108d10 0x7f921410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:35.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.548+0000 7f921c324640 1 --2- 172.21.15.117:0/2202672211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92141083c0 0x7f92141087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:35.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.548+0000 7f921c324640 1 --2- 172.21.15.117:0/2202672211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9214102a80 0x7f9214102e90 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:35.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.548+0000 7f921c324640 1 -- 172.21.15.117:0/2202672211 >> 172.21.15.117:0/2202672211 conn(0x7f9214079740 msgr2=0x7f9214079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:35.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.548+0000 7f921c324640 1 -- 172.21.15.117:0/2202672211 shutdown_connections 2024-09-21T06:08:35.551 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.548+0000 7f921c324640 1 -- 172.21.15.117:0/2202672211 wait complete. 2024-09-21T06:08:35.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.549+0000 7f921c324640 1 Processor -- start 2024-09-21T06:08:35.552 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.549+0000 7f921c324640 1 -- start start 2024-09-21T06:08:35.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.549+0000 7f921c324640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9214102a80 0x7f921419b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:35.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.549+0000 7f921c324640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92141083c0 0x7f921419b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:35.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.550+0000 7f921c324640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9214108d10 0x7f921419bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:35.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.550+0000 7f921c324640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9214112980 con 0x7f92141083c0 2024-09-21T06:08:35.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.550+0000 7f921c324640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9214112800 con 0x7f9214102a80 2024-09-21T06:08:35.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.550+0000 7f921c324640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9214112b00 con 0x7f9214108d10 2024-09-21T06:08:35.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.550+0000 7f92198bf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92141083c0 0x7f921419b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:35.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.550+0000 7f92198bf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92141083c0 0x7f921419b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37518/0 (socket says 172.21.15.117:37518) 2024-09-21T06:08:35.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.550+0000 7f92198bf640 1 -- 172.21.15.117:0/368273203 learned_addr learned my addr 172.21.15.117:0/368273203 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:35.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.550+0000 7f921a0c0640 1 --2- 172.21.15.117:0/368273203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9214102a80 0x7f921419b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:35.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.550+0000 7f921a8c1640 1 --2- 172.21.15.117:0/368273203 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9214108d10 0x7f921419bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:35.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.550+0000 7f92198bf640 1 -- 172.21.15.117:0/368273203 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9214108d10 msgr2=0x7f921419bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:35.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.550+0000 7f92198bf640 1 --2- 172.21.15.117:0/368273203 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9214108d10 0x7f921419bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:35.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.550+0000 7f92198bf640 1 -- 172.21.15.117:0/368273203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9214102a80 msgr2=0x7f921419b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:35.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.550+0000 7f92198bf640 1 --2- 172.21.15.117:0/368273203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9214102a80 0x7f921419b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:35.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.550+0000 7f92198bf640 1 -- 172.21.15.117:0/368273203 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92141a6550 con 0x7f92141083c0 2024-09-21T06:08:35.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.550+0000 7f92198bf640 1 --2- 172.21.15.117:0/368273203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92141083c0 0x7f921419b7d0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f920400d770 tx=0x7f920400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:35.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.550+0000 7f920b7fe640 1 -- 172.21.15.117:0/368273203 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9204017070 con 0x7f92141083c0 2024-09-21T06:08:35.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.550+0000 7f920b7fe640 1 -- 172.21.15.117:0/368273203 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9204009c30 con 0x7f92141083c0 2024-09-21T06:08:35.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.550+0000 7f921c324640 1 -- 172.21.15.117:0/368273203 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92141a6840 con 0x7f92141083c0 2024-09-21T06:08:35.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.550+0000 7f920b7fe640 1 -- 172.21.15.117:0/368273203 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920400f620 con 0x7f92141083c0 2024-09-21T06:08:35.554 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.550+0000 7f921c324640 1 -- 172.21.15.117:0/368273203 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92141a6d80 con 0x7f92141083c0 2024-09-21T06:08:35.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.553+0000 7f9222898740 1 Processor -- start 2024-09-21T06:08:35.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.553+0000 7f9222898740 1 -- start start 2024-09-21T06:08:35.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.553+0000 7f920b7fe640 1 -- 172.21.15.117:0/368273203 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f920401a070 con 0x7f92141083c0 2024-09-21T06:08:35.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.553+0000 7f9222898740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e51fa070 0x55c5e55a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:35.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.553+0000 7f9222898740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c5e51d11b0 0x55c5e55a1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:35.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.553+0000 7f9222898740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5e51d9390 0x55c5e55a2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:35.556 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.553+0000 7f9222898740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c5e5517ea0 con 0x55c5e51fa070 2024-09-21T06:08:35.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.553+0000 7f9222898740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c5e55072d0 con 0x55c5e51d9390 2024-09-21T06:08:35.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.553+0000 7f9222898740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c5e550b0e0 con 0x55c5e51d11b0 2024-09-21T06:08:35.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.554+0000 7f921a8c1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e51fa070 0x55c5e55a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:35.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.554+0000 7f921a8c1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e51fa070 0x55c5e55a16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37522/0 (socket says 172.21.15.117:37522) 2024-09-21T06:08:35.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.554+0000 7f921a8c1640 1 -- 172.21.15.117:0/3048153135 learned_addr learned my addr 172.21.15.117:0/3048153135 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:35.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.554+0000 7f921a0c0640 1 --2- 172.21.15.117:0/3048153135 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c5e51d11b0 0x55c5e55a1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:35.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.554+0000 7f92198bf640 1 --2- 172.21.15.117:0/3048153135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5e51d9390 0x55c5e55a2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:35.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.554+0000 7f921a8c1640 1 -- 172.21.15.117:0/3048153135 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c5e51d11b0 msgr2=0x55c5e55a1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:35.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.554+0000 7f921a8c1640 1 --2- 172.21.15.117:0/3048153135 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c5e51d11b0 0x55c5e55a1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:35.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.554+0000 7f921a8c1640 1 -- 172.21.15.117:0/3048153135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5e51d9390 msgr2=0x55c5e55a2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:35.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.554+0000 7f921a8c1640 1 --2- 172.21.15.117:0/3048153135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5e51d9390 0x55c5e55a2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:35.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.554+0000 7f921a8c1640 1 -- 172.21.15.117:0/3048153135 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5e5265650 con 0x55c5e51fa070 2024-09-21T06:08:35.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.554+0000 7f92198bf640 1 --2- 172.21.15.117:0/3048153135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5e51d9390 0x55c5e55a2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:08:35.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.554+0000 7f921a8c1640 1 --2- 172.21.15.117:0/3048153135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e51fa070 0x55c5e55a16a0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f920c008dc0 tx=0x7f920c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4bbe6d7257429d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:35.557 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.554+0000 7f9208d9a640 1 -- 172.21.15.117:0/3048153135 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920c014070 con 0x55c5e51fa070 2024-09-21T06:08:35.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.554+0000 7f920b7fe640 1 --2- 172.21.15.117:0/368273203 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f92000b6680 0x7f92000b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:35.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.554+0000 7f9208d9a640 1 -- 172.21.15.117:0/3048153135 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f920c003b70 con 0x55c5e51fa070 2024-09-21T06:08:35.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.554+0000 7f9208d9a640 1 -- 172.21.15.117:0/3048153135 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920c010700 con 0x55c5e51fa070 2024-09-21T06:08:35.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.554+0000 7f921a0c0640 1 --2- 172.21.15.117:0/368273203 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f92000b6680 0x7f92000b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:35.558 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.555+0000 7f920b7fe640 1 -- 172.21.15.117:0/368273203 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7f92040e72c0 con 0x7f92141083c0 2024-09-21T06:08:35.560 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.555+0000 7f9222898740 1 -- 172.21.15.117:0/3048153135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e51fa070 msgr2=0x55c5e55a16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:35.560 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.555+0000 7f921a0c0640 1 --2- 172.21.15.117:0/368273203 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f92000b6680 0x7f92000b8b90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9210000c00 tx=0x7f9210009680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:35.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.555+0000 7f9222898740 1 --2- 172.21.15.117:0/3048153135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e51fa070 0x55c5e55a16a0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f920c008dc0 tx=0x7f920c00e720 comp rx=0 tx=0).stop 2024-09-21T06:08:35.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.555+0000 7f9222898740 1 -- 172.21.15.117:0/3048153135 shutdown_connections 2024-09-21T06:08:35.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.555+0000 7f9222898740 1 --2- 172.21.15.117:0/3048153135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5e51d9390 0x55c5e55a2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:35.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.555+0000 7f9222898740 1 --2- 172.21.15.117:0/3048153135 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c5e51d11b0 0x55c5e55a1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:35.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.555+0000 7f9222898740 1 --2- 172.21.15.117:0/3048153135 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e51fa070 0x55c5e55a16a0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:35.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.555+0000 7f9222898740 1 -- 172.21.15.117:0/3048153135 >> 172.21.15.117:0/3048153135 conn(0x55c5e51ee460 msgr2=0x55c5e559f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:35.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.555+0000 7f9222898740 1 -- 172.21.15.117:0/3048153135 shutdown_connections 2024-09-21T06:08:35.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.556+0000 7f9222898740 1 -- 172.21.15.117:0/3048153135 wait complete. 2024-09-21T06:08:35.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.556+0000 7f9222898740 1 -- start start 2024-09-21T06:08:35.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.556+0000 7f9222898740 1 Processor -- start 2024-09-21T06:08:35.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.556+0000 7f9222898740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e51d11b0 0x55c5e55acde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:35.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.556+0000 7f9222898740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5e51d9390 0x55c5e55ad320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:35.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.556+0000 7f9222898740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c5e51fa070 0x55c5e55b91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:35.561 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.556+0000 7f9222898740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c5e5368700 con 0x55c5e51d11b0 2024-09-21T06:08:35.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.556+0000 7f9222898740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c5e54a2160 con 0x55c5e51d9390 2024-09-21T06:08:35.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.556+0000 7f9222898740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c5e55afe80 con 0x55c5e51fa070 2024-09-21T06:08:35.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.556+0000 7f9222898740 10 client.0 _ll_register_callbacks cb 0x55c5e54bb9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:08:35.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.556+0000 7f921a0c0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e51d11b0 0x55c5e55acde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:35.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.556+0000 7f921a0c0640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e51d11b0 0x55c5e55acde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37530/0 (socket says 172.21.15.117:37530) 2024-09-21T06:08:35.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.556+0000 7f921a0c0640 1 -- 172.21.15.117:0/2968334280 learned_addr learned my addr 172.21.15.117:0/2968334280 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:35.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.557+0000 7f921a8c1640 1 --2- 172.21.15.117:0/2968334280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c5e51fa070 0x55c5e55b91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:35.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.557+0000 7f92198bf640 1 --2- 172.21.15.117:0/2968334280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5e51d9390 0x55c5e55ad320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:35.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.557+0000 7f921a0c0640 1 -- 172.21.15.117:0/2968334280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c5e51fa070 msgr2=0x55c5e55b91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:35.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.557+0000 7f921a0c0640 1 --2- 172.21.15.117:0/2968334280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c5e51fa070 0x55c5e55b91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:35.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.557+0000 7f921a0c0640 1 -- 172.21.15.117:0/2968334280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5e51d9390 msgr2=0x55c5e55ad320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:35.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.557+0000 7f921a0c0640 1 --2- 172.21.15.117:0/2968334280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5e51d9390 0x55c5e55ad320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:35.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.557+0000 7f921a0c0640 1 -- 172.21.15.117:0/2968334280 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5e54cfc90 con 0x55c5e51d11b0 2024-09-21T06:08:35.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.557+0000 7f921a0c0640 1 --2- 172.21.15.117:0/2968334280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e51d11b0 0x55c5e55acde0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f9210033000 tx=0x7f9210009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ddda0b7cb4f5ff2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:35.562 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.557+0000 7f91fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:08:35.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.557+0000 7f91fdffb640 1 -- 172.21.15.117:0/2968334280 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921004b070 con 0x55c5e51d11b0 2024-09-21T06:08:35.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.557+0000 7f91fdffb640 1 -- 172.21.15.117:0/2968334280 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7f9210005460 con 0x55c5e51d11b0 2024-09-21T06:08:35.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.557+0000 7f9222898740 1 -- 172.21.15.117:0/2968334280 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f920c003b70 con 0x55c5e51d11b0 2024-09-21T06:08:35.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.557+0000 7f91fdffb640 1 -- 172.21.15.117:0/2968334280 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9210057050 con 0x55c5e51d11b0 2024-09-21T06:08:35.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.557+0000 7f9222898740 10 client.18111 fetch_fsmap learned FSMap version 29 2024-09-21T06:08:35.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.557+0000 7f9222898740 1 -- 172.21.15.117:0/2968334280 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c5e55bbb60 con 0x55c5e51d11b0 2024-09-21T06:08:35.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.558+0000 7f91fdffb640 1 -- 172.21.15.117:0/2968334280 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f920c003b70 con 0x55c5e51d11b0 2024-09-21T06:08:35.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.558+0000 7f91fdffb640 1 -- 172.21.15.117:0/2968334280 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f92100036c0 con 0x55c5e51d11b0 2024-09-21T06:08:35.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.558+0000 7f9222898740 10 client.18111 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:08:35.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.558+0000 7f9222898740 10 client.18111 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T06:08:35.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.558+0000 7f9222898740 20 client.18111 populate_metadata read hostname 'smithi117' 2024-09-21T06:08:35.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.558+0000 7f9222898740 1 --2- 172.21.15.117:0/2968334280 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55c5e51b89f0 0x55c5e55bf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:35.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.558+0000 7f9222898740 4 client.18111 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:08:35.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.558+0000 7f9222898740 1 -- 172.21.15.117:0/2968334280 --> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5e5519df0 con 0x55c5e51b89f0 2024-09-21T06:08:35.563 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.558+0000 7f92198bf640 1 --2- 172.21.15.117:0/2968334280 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55c5e51b89f0 0x55c5e55bf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:35.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.559+0000 7f91fdffb640 10 client.18111 ms_handle_connect on v2:172.21.15.192:6834/3530688860 2024-09-21T06:08:35.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.559+0000 7f92198bf640 1 --2- 172.21.15.117:0/2968334280 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55c5e51b89f0 0x55c5e55bf050 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e336adf8a8d82e88 server_cookie=63f248c15f45798d in_seq=0 out_seq=0 2024-09-21T06:08:35.564 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.559+0000 7f91fdffb640 1 -- 172.21.15.117:0/2968334280 <== mds.3 v2:172.21.15.192:6834/3530688860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c5e5519df0 con 0x55c5e51b89f0 2024-09-21T06:08:35.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.559+0000 7f91fdffb640 10 client.18111 handle_command_reply: tid=0 2024-09-21T06:08:35.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.560+0000 7f9222898740 1 client.18111 shutdown 2024-09-21T06:08:35.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.560+0000 7f9222898740 1 -- 172.21.15.117:0/2968334280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e51d11b0 msgr2=0x55c5e55acde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:35.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.560+0000 7f9222898740 1 --2- 172.21.15.117:0/2968334280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e51d11b0 0x55c5e55acde0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f9210033000 tx=0x7f9210009260 comp rx=0 tx=0).stop 2024-09-21T06:08:35.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.560+0000 7f9222898740 1 -- 172.21.15.117:0/2968334280 shutdown_connections 2024-09-21T06:08:35.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.560+0000 7f9222898740 1 --2- 172.21.15.117:0/2968334280 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55c5e51b89f0 0x55c5e55bf050 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:35.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.560+0000 7f9222898740 1 --2- 172.21.15.117:0/2968334280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c5e51fa070 0x55c5e55b91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:35.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.560+0000 7f9222898740 1 --2- 172.21.15.117:0/2968334280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5e51d9390 0x55c5e55ad320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:35.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.560+0000 7f9222898740 1 --2- 172.21.15.117:0/2968334280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e51d11b0 0x55c5e55acde0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:35.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.560+0000 7f9222898740 1 -- 172.21.15.117:0/2968334280 >> 172.21.15.117:0/2968334280 conn(0x55c5e51ee460 msgr2=0x55c5e55a4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:35.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.560+0000 7f91fdffb640 0 client.18111 ms_handle_reset on v2:172.21.15.192:6834/3530688860 2024-09-21T06:08:35.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.560+0000 7f9222898740 1 -- 172.21.15.117:0/2968334280 shutdown_connections 2024-09-21T06:08:35.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.560+0000 7f9222898740 1 -- 172.21.15.117:0/2968334280 wait complete. 2024-09-21T06:08:35.565 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.560+0000 7f9222898740 20 client.18111 trim_cache size 0 max 16384 2024-09-21T06:08:35.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.573+0000 7f9222898740 1 Processor -- start 2024-09-21T06:08:35.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.573+0000 7f9222898740 1 -- start start 2024-09-21T06:08:35.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.573+0000 7f9222898740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c5e51d11b0 0x55c5e55a3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:35.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.573+0000 7f9222898740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e51d9390 0x55c5e55aea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:35.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.573+0000 7f9222898740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5e51fa070 0x55c5e55b90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:35.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.573+0000 7f9222898740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f92141af5f0 con 0x55c5e51d9390 2024-09-21T06:08:35.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.573+0000 7f9222898740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f92141b1130 con 0x55c5e51fa070 2024-09-21T06:08:35.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.573+0000 7f9222898740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f92141afcc0 con 0x55c5e51d11b0 2024-09-21T06:08:35.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.574+0000 7f92198bf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e51d9390 0x55c5e55aea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:35.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.574+0000 7f92198bf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e51d9390 0x55c5e55aea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37544/0 (socket says 172.21.15.117:37544) 2024-09-21T06:08:35.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.574+0000 7f92198bf640 1 -- 172.21.15.117:0/3672848657 learned_addr learned my addr 172.21.15.117:0/3672848657 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:35.576 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.574+0000 7f921a0c0640 1 --2- 172.21.15.117:0/3672848657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c5e51d11b0 0x55c5e55a3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:35.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.574+0000 7f921a8c1640 1 --2- 172.21.15.117:0/3672848657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5e51fa070 0x55c5e55b90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:35.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.574+0000 7f92198bf640 1 -- 172.21.15.117:0/3672848657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c5e51d11b0 msgr2=0x55c5e55a3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:35.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.574+0000 7f92198bf640 1 --2- 172.21.15.117:0/3672848657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c5e51d11b0 0x55c5e55a3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:35.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.574+0000 7f92198bf640 1 -- 172.21.15.117:0/3672848657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5e51fa070 msgr2=0x55c5e55b90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:35.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.574+0000 7f92198bf640 1 --2- 172.21.15.117:0/3672848657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5e51fa070 0x55c5e55b90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:35.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.574+0000 7f92198bf640 1 -- 172.21.15.117:0/3672848657 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5e55ae0a0 con 0x55c5e51d9390 2024-09-21T06:08:35.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.574+0000 7f921a8c1640 1 --2- 172.21.15.117:0/3672848657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5e51fa070 0x55c5e55b90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:08:35.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.574+0000 7f92198bf640 1 --2- 172.21.15.117:0/3672848657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e51d9390 0x55c5e55aea10 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f9204000c00 tx=0x7f920400df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed3de747417b24a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:35.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.574+0000 7f91fffff640 1 -- 172.21.15.117:0/3672848657 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9204021040 con 0x55c5e51d9390 2024-09-21T06:08:35.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.574+0000 7f91fffff640 1 -- 172.21.15.117:0/3672848657 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92040acc70 con 0x55c5e51d9390 2024-09-21T06:08:35.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.574+0000 7f91fffff640 1 -- 172.21.15.117:0/3672848657 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92040d0040 con 0x55c5e51d9390 2024-09-21T06:08:35.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.575+0000 7f9222898740 1 -- 172.21.15.117:0/3672848657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e51d9390 msgr2=0x55c5e55aea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:35.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.575+0000 7f9222898740 1 --2- 172.21.15.117:0/3672848657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e51d9390 0x55c5e55aea10 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f9204000c00 tx=0x7f920400df10 comp rx=0 tx=0).stop 2024-09-21T06:08:35.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.575+0000 7f9222898740 1 -- 172.21.15.117:0/3672848657 shutdown_connections 2024-09-21T06:08:35.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.575+0000 7f9222898740 1 --2- 172.21.15.117:0/3672848657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e51d9390 0x55c5e55aea10 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:35.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.575+0000 7f9222898740 1 --2- 172.21.15.117:0/3672848657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5e51fa070 0x55c5e55b90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:35.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.575+0000 7f9222898740 1 --2- 172.21.15.117:0/3672848657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c5e51d11b0 0x55c5e55a3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:35.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.575+0000 7f9222898740 1 -- 172.21.15.117:0/3672848657 >> 172.21.15.117:0/3672848657 conn(0x55c5e51ee460 msgr2=0x55c5e55ae300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:35.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.575+0000 7f9222898740 1 -- 172.21.15.117:0/3672848657 shutdown_connections 2024-09-21T06:08:35.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.575+0000 7f9222898740 1 -- 172.21.15.117:0/3672848657 wait complete. 2024-09-21T06:08:35.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.575+0000 7f9222898740 1 -- start start 2024-09-21T06:08:35.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.575+0000 7f9222898740 1 Processor -- start 2024-09-21T06:08:35.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.576+0000 7f9222898740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5e51d11b0 0x55c5e55b5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:35.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.576+0000 7f9222898740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e51d9390 0x55c5e55b5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:35.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.576+0000 7f9222898740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c5e51fa070 0x55c5e5622a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:35.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.576+0000 7f9222898740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f92141b0a60 con 0x55c5e51d9390 2024-09-21T06:08:35.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.576+0000 7f9222898740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f92141b0390 con 0x55c5e51d11b0 2024-09-21T06:08:35.581 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.576+0000 7f9222898740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f92141ad5f0 con 0x55c5e51fa070 2024-09-21T06:08:35.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.576+0000 7f9222898740 10 client.0 _ll_register_callbacks cb 0x7f9210009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:08:35.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.576+0000 7f92198bf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e51d9390 0x55c5e55b5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:35.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.576+0000 7f92198bf640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e51d9390 0x55c5e55b5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37550/0 (socket says 172.21.15.117:37550) 2024-09-21T06:08:35.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.576+0000 7f92198bf640 1 -- 172.21.15.117:0/2770311598 learned_addr learned my addr 172.21.15.117:0/2770311598 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:35.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.576+0000 7f921a0c0640 1 --2- 172.21.15.117:0/2770311598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5e51d11b0 0x55c5e55b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:35.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.576+0000 7f921a8c1640 1 --2- 172.21.15.117:0/2770311598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c5e51fa070 0x55c5e5622a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:35.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.576+0000 7f92198bf640 1 -- 172.21.15.117:0/2770311598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c5e51fa070 msgr2=0x55c5e5622a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:35.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.576+0000 7f92198bf640 1 --2- 172.21.15.117:0/2770311598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c5e51fa070 0x55c5e5622a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:35.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.576+0000 7f92198bf640 1 -- 172.21.15.117:0/2770311598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5e51d11b0 msgr2=0x55c5e55b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:35.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.576+0000 7f92198bf640 1 --2- 172.21.15.117:0/2770311598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5e51d11b0 0x55c5e55b5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:35.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.576+0000 7f92198bf640 1 -- 172.21.15.117:0/2770311598 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5e55b5bb0 con 0x55c5e51d9390 2024-09-21T06:08:35.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.576+0000 7f92198bf640 1 --2- 172.21.15.117:0/2770311598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e51d9390 0x55c5e55b5640 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f92040e5060 tx=0x7f92040e4a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4cba37bd7886c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:35.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.576+0000 7f91fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:08:35.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.576+0000 7f91fdffb640 1 -- 172.21.15.117:0/2770311598 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9204021050 con 0x55c5e51d9390 2024-09-21T06:08:35.582 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.576+0000 7f91fdffb640 1 -- 172.21.15.117:0/2770311598 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7f92040d0ce0 con 0x55c5e51d9390 2024-09-21T06:08:35.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.576+0000 7f9222898740 1 -- 172.21.15.117:0/2770311598 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92040acc70 con 0x55c5e51d9390 2024-09-21T06:08:35.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.577+0000 7f91fdffb640 1 -- 172.21.15.117:0/2770311598 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9204098020 con 0x55c5e51d9390 2024-09-21T06:08:35.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.577+0000 7f9222898740 10 client.18117 fetch_fsmap learned FSMap version 29 2024-09-21T06:08:35.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.577+0000 7f9222898740 1 -- 172.21.15.117:0/2770311598 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c5e562e070 con 0x55c5e51d9390 2024-09-21T06:08:35.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.577+0000 7f91fdffb640 1 -- 172.21.15.117:0/2770311598 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92040acc70 con 0x55c5e51d9390 2024-09-21T06:08:35.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.577+0000 7f91fdffb640 1 -- 172.21.15.117:0/2770311598 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f92040a8cb0 con 0x55c5e51d9390 2024-09-21T06:08:35.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.577+0000 7f9222898740 10 client.18117 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:08:35.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.577+0000 7f9222898740 10 client.18117 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-09-21T06:08:35.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.577+0000 7f9222898740 20 client.18117 populate_metadata read hostname 'smithi117' 2024-09-21T06:08:35.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.577+0000 7f9222898740 1 --2- 172.21.15.117:0/2770311598 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55c5e518dda0 0x55c5e5630750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:35.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.577+0000 7f9222898740 4 client.18117 mds_command: new command op to 24616 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:08:35.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.577+0000 7f9222898740 1 -- 172.21.15.117:0/2770311598 --> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c5e55158a0 con 0x55c5e518dda0 2024-09-21T06:08:35.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.577+0000 7f921a0c0640 1 --2- 172.21.15.117:0/2770311598 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55c5e518dda0 0x55c5e5630750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:35.583 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.578+0000 7f91fdffb640 10 client.18117 ms_handle_connect on v2:172.21.15.192:6834/3530688860 2024-09-21T06:08:35.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.578+0000 7f921a0c0640 1 --2- 172.21.15.117:0/2770311598 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55c5e518dda0 0x55c5e5630750 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9de4e5cd45c50368 server_cookie=6088a9a2d650dbbc in_seq=0 out_seq=0 2024-09-21T06:08:35.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.578+0000 7f91fdffb640 1 -- 172.21.15.117:0/2770311598 <== mds.3 v2:172.21.15.192:6834/3530688860 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c5e55158a0 con 0x55c5e518dda0 2024-09-21T06:08:35.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.578+0000 7f91fdffb640 10 client.18117 handle_command_reply: tid=0 2024-09-21T06:08:35.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.578+0000 7f9222898740 1 client.18117 shutdown 2024-09-21T06:08:35.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.578+0000 7f9222898740 1 -- 172.21.15.117:0/2770311598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e51d9390 msgr2=0x55c5e55b5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:35.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.578+0000 7f9222898740 1 --2- 172.21.15.117:0/2770311598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e51d9390 0x55c5e55b5640 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f92040e5060 tx=0x7f92040e4a30 comp rx=0 tx=0).stop 2024-09-21T06:08:35.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.578+0000 7f9222898740 1 -- 172.21.15.117:0/2770311598 shutdown_connections 2024-09-21T06:08:35.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.578+0000 7f9222898740 1 --2- 172.21.15.117:0/2770311598 >> [v2:172.21.15.192:6834/3530688860,v1:172.21.15.192:6835/3530688860] conn(0x55c5e518dda0 0x55c5e5630750 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:35.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.578+0000 7f9222898740 1 --2- 172.21.15.117:0/2770311598 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c5e51d9390 0x55c5e55b5640 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:35.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.578+0000 7f9222898740 1 --2- 172.21.15.117:0/2770311598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c5e51fa070 0x55c5e5622a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:35.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.578+0000 7f91fdffb640 0 client.18117 ms_handle_reset on v2:172.21.15.192:6834/3530688860 2024-09-21T06:08:35.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.578+0000 7f9222898740 1 --2- 172.21.15.117:0/2770311598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5e51d11b0 0x55c5e55b5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:35.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.578+0000 7f9222898740 1 -- 172.21.15.117:0/2770311598 >> 172.21.15.117:0/2770311598 conn(0x55c5e51ee460 msgr2=0x55c5e55a2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:35.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.578+0000 7f9222898740 1 -- 172.21.15.117:0/2770311598 shutdown_connections 2024-09-21T06:08:35.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.578+0000 7f9222898740 1 -- 172.21.15.117:0/2770311598 wait complete. 2024-09-21T06:08:35.584 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.578+0000 7f9222898740 20 client.18117 trim_cache size 0 max 16384 2024-09-21T06:08:35.585 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:08:35.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.579+0000 7f921c324640 1 -- 172.21.15.117:0/368273203 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f92000b6680 msgr2=0x7f92000b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:35.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.579+0000 7f921c324640 1 --2- 172.21.15.117:0/368273203 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f92000b6680 0x7f92000b8b90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9210000c00 tx=0x7f9210009680 comp rx=0 tx=0).stop 2024-09-21T06:08:35.585 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.579+0000 7f921c324640 1 -- 172.21.15.117:0/368273203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92141083c0 msgr2=0x7f921419b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:35.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.579+0000 7f921c324640 1 --2- 172.21.15.117:0/368273203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92141083c0 0x7f921419b7d0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f920400d770 tx=0x7f920400dc30 comp rx=0 tx=0).stop 2024-09-21T06:08:35.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.579+0000 7f921c324640 1 -- 172.21.15.117:0/368273203 shutdown_connections 2024-09-21T06:08:35.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.579+0000 7f921c324640 1 --2- 172.21.15.117:0/368273203 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f92000b6680 0x7f92000b8b90 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:35.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.579+0000 7f921c324640 1 --2- 172.21.15.117:0/368273203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92141083c0 0x7f921419b7d0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:35.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.579+0000 7f921c324640 1 --2- 172.21.15.117:0/368273203 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9214108d10 0x7f921419bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:35.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.579+0000 7f921c324640 1 --2- 172.21.15.117:0/368273203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9214102a80 0x7f921419b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:35.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.579+0000 7f921c324640 1 -- 172.21.15.117:0/368273203 >> 172.21.15.117:0/368273203 conn(0x7f9214079740 msgr2=0x7f9214101420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:35.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.579+0000 7f921c324640 1 -- 172.21.15.117:0/368273203 shutdown_connections 2024-09-21T06:08:35.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:35.579+0000 7f921c324640 1 -- 172.21.15.117:0/368273203 wait complete. 2024-09-21T06:08:36.298 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:2 damage ls 2024-09-21T06:08:36.545 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:08:37.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.032+0000 7fe0f1c72640 1 -- 172.21.15.117:0/4220289491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0ec103b60 msgr2=0x7fe0ec103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:37.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.032+0000 7fe0f1c72640 1 --2- 172.21.15.117:0/4220289491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0ec103b60 0x7fe0ec103f70 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc009900 tx=0x7fe0dc02a410 comp rx=0 tx=0).stop 2024-09-21T06:08:37.036 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.032+0000 7fe0f1c72640 1 -- 172.21.15.117:0/4220289491 shutdown_connections 2024-09-21T06:08:37.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.032+0000 7fe0f1c72640 1 --2- 172.21.15.117:0/4220289491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0ec109df0 0x7fe0ec110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.032+0000 7fe0f1c72640 1 --2- 172.21.15.117:0/4220289491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0ec1094a0 0x7fe0ec1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.032+0000 7fe0f1c72640 1 --2- 172.21.15.117:0/4220289491 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0ec103b60 0x7fe0ec103f70 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.032+0000 7fe0f1c72640 1 -- 172.21.15.117:0/4220289491 >> 172.21.15.117:0/4220289491 conn(0x7fe0ec07b850 msgr2=0x7fe0ec07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:37.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.032+0000 7fe0f1c72640 1 -- 172.21.15.117:0/4220289491 shutdown_connections 2024-09-21T06:08:37.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.032+0000 7fe0f1c72640 1 -- 172.21.15.117:0/4220289491 wait complete. 2024-09-21T06:08:37.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.032+0000 7fe0f1c72640 1 Processor -- start 2024-09-21T06:08:37.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.033+0000 7fe0f1c72640 1 -- start start 2024-09-21T06:08:37.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.033+0000 7fe0f1c72640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0ec103b60 0x7fe0ec19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:37.037 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.033+0000 7fe0f1c72640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0ec1094a0 0x7fe0ec19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:37.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.033+0000 7fe0f1c72640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0ec109df0 0x7fe0ec19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:37.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.033+0000 7fe0f1c72640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec113ab0 con 0x7fe0ec103b60 2024-09-21T06:08:37.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.033+0000 7fe0f1c72640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec113930 con 0x7fe0ec109df0 2024-09-21T06:08:37.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.033+0000 7fe0f1c72640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec113c30 con 0x7fe0ec1094a0 2024-09-21T06:08:37.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.033+0000 7fe0eb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0ec103b60 0x7fe0ec19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:37.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.033+0000 7fe0eb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0ec103b60 0x7fe0ec19a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37566/0 (socket says 172.21.15.117:37566) 2024-09-21T06:08:37.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.033+0000 7fe0eb7fe640 1 -- 172.21.15.117:0/1247573102 learned_addr learned my addr 172.21.15.117:0/1247573102 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:37.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.033+0000 7fe0eaffd640 1 --2- 172.21.15.117:0/1247573102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0ec1094a0 0x7fe0ec19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:37.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.033+0000 7fe0ebfff640 1 --2- 172.21.15.117:0/1247573102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0ec109df0 0x7fe0ec19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:37.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.033+0000 7fe0eb7fe640 1 -- 172.21.15.117:0/1247573102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0ec1094a0 msgr2=0x7fe0ec19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:37.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.033+0000 7fe0eb7fe640 1 --2- 172.21.15.117:0/1247573102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0ec1094a0 0x7fe0ec19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.033+0000 7fe0eb7fe640 1 -- 172.21.15.117:0/1247573102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0ec109df0 msgr2=0x7fe0ec19fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:37.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.033+0000 7fe0eb7fe640 1 --2- 172.21.15.117:0/1247573102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0ec109df0 0x7fe0ec19fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.033+0000 7fe0eb7fe640 1 -- 172.21.15.117:0/1247573102 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0ec1a0410 con 0x7fe0ec103b60 2024-09-21T06:08:37.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.034+0000 7fe0eb7fe640 1 --2- 172.21.15.117:0/1247573102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0ec103b60 0x7fe0ec19a4a0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc005940 tx=0x7fe0dc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:37.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.034+0000 7fe0e8ff9640 1 -- 172.21.15.117:0/1247573102 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc03f030 con 0x7fe0ec103b60 2024-09-21T06:08:37.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.034+0000 7fe0f1c72640 1 -- 172.21.15.117:0/1247573102 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0ec1a06a0 con 0x7fe0ec103b60 2024-09-21T06:08:37.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.034+0000 7fe0e8ff9640 1 -- 172.21.15.117:0/1247573102 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0dc003cb0 con 0x7fe0ec103b60 2024-09-21T06:08:37.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.034+0000 7fe0e8ff9640 1 -- 172.21.15.117:0/1247573102 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc035070 con 0x7fe0ec103b60 2024-09-21T06:08:37.040 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.034+0000 7fe0f1c72640 1 -- 172.21.15.117:0/1247573102 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0ec1a6e60 con 0x7fe0ec103b60 2024-09-21T06:08:37.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.036+0000 7fe0f81e6740 1 Processor -- start 2024-09-21T06:08:37.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.036+0000 7fe0f81e6740 1 -- start start 2024-09-21T06:08:37.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.037+0000 7fe0e8ff9640 1 -- 172.21.15.117:0/1247573102 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fe0dc042030 con 0x7fe0ec103b60 2024-09-21T06:08:37.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.037+0000 7fe0f81e6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70d4e5070 0x55a70d88c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:37.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.037+0000 7fe0f81e6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70d4bc1b0 0x55a70d88cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:37.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.037+0000 7fe0f81e6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70d4c4390 0x55a70d88d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:37.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.037+0000 7fe0f81e6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a70d802ea0 con 0x55a70d4c4390 2024-09-21T06:08:37.046 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.037+0000 7fe0f81e6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a70d7f22d0 con 0x55a70d4e5070 2024-09-21T06:08:37.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.037+0000 7fe0f81e6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a70d7f60e0 con 0x55a70d4bc1b0 2024-09-21T06:08:37.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.037+0000 7fe0eaffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70d4c4390 0x55a70d88d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:37.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.037+0000 7fe0eaffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70d4c4390 0x55a70d88d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37578/0 (socket says 172.21.15.117:37578) 2024-09-21T06:08:37.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.037+0000 7fe0eaffd640 1 -- 172.21.15.117:0/3708962635 learned_addr learned my addr 172.21.15.117:0/3708962635 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:37.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.037+0000 7fe0ebfff640 1 --2- 172.21.15.117:0/3708962635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70d4e5070 0x55a70d88c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:37.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.037+0000 7fe0eb7fe640 1 --2- 172.21.15.117:0/3708962635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70d4bc1b0 0x55a70d88cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:37.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.037+0000 7fe0eaffd640 1 -- 172.21.15.117:0/3708962635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70d4bc1b0 msgr2=0x55a70d88cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:37.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.037+0000 7fe0eaffd640 1 --2- 172.21.15.117:0/3708962635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70d4bc1b0 0x55a70d88cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.037+0000 7fe0eaffd640 1 -- 172.21.15.117:0/3708962635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70d4e5070 msgr2=0x55a70d88c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:37.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.037+0000 7fe0eaffd640 1 --2- 172.21.15.117:0/3708962635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70d4e5070 0x55a70d88c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.037+0000 7fe0eaffd640 1 -- 172.21.15.117:0/3708962635 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a70d550650 con 0x55a70d4c4390 2024-09-21T06:08:37.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.037+0000 7fe0eb7fe640 1 --2- 172.21.15.117:0/3708962635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70d4bc1b0 0x55a70d88cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:08:37.047 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.037+0000 7fe0e8ff9640 1 --2- 172.21.15.117:0/1247573102 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe0d40b6680 0x7fe0d40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:37.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.037+0000 7fe0eaffd640 1 --2- 172.21.15.117:0/3708962635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70d4c4390 0x55a70d88d120 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fe0e000b690 tx=0x7fe0e000bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4676c018a2a15f75 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:37.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.038+0000 7fe0ebfff640 1 --2- 172.21.15.117:0/1247573102 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe0d40b6680 0x7fe0d40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:37.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.038+0000 7fe0d1ffb640 1 -- 172.21.15.117:0/3708962635 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0003bd0 con 0x55a70d4c4390 2024-09-21T06:08:37.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.038+0000 7fe0e8ff9640 1 -- 172.21.15.117:0/1247573102 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7fe0dc107c00 con 0x7fe0ec103b60 2024-09-21T06:08:37.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.038+0000 7fe0d1ffb640 1 -- 172.21.15.117:0/3708962635 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0e0015070 con 0x55a70d4c4390 2024-09-21T06:08:37.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.038+0000 7fe0d1ffb640 1 -- 172.21.15.117:0/3708962635 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0004d10 con 0x55a70d4c4390 2024-09-21T06:08:37.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.039+0000 7fe0f81e6740 1 -- 172.21.15.117:0/3708962635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70d4c4390 msgr2=0x55a70d88d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:37.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.039+0000 7fe0f81e6740 1 --2- 172.21.15.117:0/3708962635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70d4c4390 0x55a70d88d120 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fe0e000b690 tx=0x7fe0e000bb50 comp rx=0 tx=0).stop 2024-09-21T06:08:37.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.039+0000 7fe0ebfff640 1 --2- 172.21.15.117:0/1247573102 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe0d40b6680 0x7fe0d40b8b90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4005fd0 tx=0x7fe0e4009740 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:37.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.039+0000 7fe0f81e6740 1 -- 172.21.15.117:0/3708962635 shutdown_connections 2024-09-21T06:08:37.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.039+0000 7fe0f81e6740 1 --2- 172.21.15.117:0/3708962635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70d4c4390 0x55a70d88d120 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.039+0000 7fe0f81e6740 1 --2- 172.21.15.117:0/3708962635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70d4bc1b0 0x55a70d88cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.048 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.039+0000 7fe0f81e6740 1 --2- 172.21.15.117:0/3708962635 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70d4e5070 0x55a70d88c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.039+0000 7fe0f81e6740 1 -- 172.21.15.117:0/3708962635 >> 172.21.15.117:0/3708962635 conn(0x55a70d4d9460 msgr2=0x55a70d88a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:37.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.039+0000 7fe0f81e6740 1 -- 172.21.15.117:0/3708962635 shutdown_connections 2024-09-21T06:08:37.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.039+0000 7fe0f81e6740 1 -- 172.21.15.117:0/3708962635 wait complete. 2024-09-21T06:08:37.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.039+0000 7fe0f81e6740 1 -- start start 2024-09-21T06:08:37.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.039+0000 7fe0f81e6740 1 Processor -- start 2024-09-21T06:08:37.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.040+0000 7fe0f81e6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70d4bc1b0 0x55a70d897d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:37.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.040+0000 7fe0f81e6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70d4c4390 0x55a70d898240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:37.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.040+0000 7fe0f81e6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70d4e5070 0x55a70d8a4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:37.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.040+0000 7fe0f81e6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a70d653700 con 0x55a70d4bc1b0 2024-09-21T06:08:37.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.040+0000 7fe0f81e6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a70d78d160 con 0x55a70d4c4390 2024-09-21T06:08:37.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.040+0000 7fe0f81e6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a70d89ae80 con 0x55a70d4e5070 2024-09-21T06:08:37.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.040+0000 7fe0f81e6740 10 client.0 _ll_register_callbacks cb 0x55a70d7a69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:08:37.049 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.040+0000 7fe0eb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70d4bc1b0 0x55a70d897d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:37.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.040+0000 7fe0eb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70d4bc1b0 0x55a70d897d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37582/0 (socket says 172.21.15.117:37582) 2024-09-21T06:08:37.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.040+0000 7fe0eb7fe640 1 -- 172.21.15.117:0/1206238249 learned_addr learned my addr 172.21.15.117:0/1206238249 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:37.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.040+0000 7fe0eaffd640 1 --2- 172.21.15.117:0/1206238249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70d4c4390 0x55a70d898240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:37.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.040+0000 7fe0ebfff640 1 --2- 172.21.15.117:0/1206238249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70d4e5070 0x55a70d8a4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:37.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.040+0000 7fe0eb7fe640 1 -- 172.21.15.117:0/1206238249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70d4e5070 msgr2=0x55a70d8a4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:37.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.040+0000 7fe0eb7fe640 1 --2- 172.21.15.117:0/1206238249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70d4e5070 0x55a70d8a4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.040+0000 7fe0eb7fe640 1 -- 172.21.15.117:0/1206238249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70d4c4390 msgr2=0x55a70d898240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:37.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.040+0000 7fe0eb7fe640 1 --2- 172.21.15.117:0/1206238249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70d4c4390 0x55a70d898240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.040+0000 7fe0eb7fe640 1 -- 172.21.15.117:0/1206238249 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a70d7bac90 con 0x55a70d4bc1b0 2024-09-21T06:08:37.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.040+0000 7fe0eb7fe640 1 --2- 172.21.15.117:0/1206238249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70d4bc1b0 0x55a70d897d00 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc006010 tx=0x7fe0dc032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d846edc85031c3cb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:37.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.040+0000 7fe0caffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:08:37.050 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.041+0000 7fe0caffd640 1 -- 172.21.15.117:0/1206238249 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc049070 con 0x55a70d4bc1b0 2024-09-21T06:08:37.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.041+0000 7fe0caffd640 1 -- 172.21.15.117:0/1206238249 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7fe0dc0c9d10 con 0x55a70d4bc1b0 2024-09-21T06:08:37.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.041+0000 7fe0f81e6740 1 -- 172.21.15.117:0/1206238249 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0e0015070 con 0x55a70d4bc1b0 2024-09-21T06:08:37.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.041+0000 7fe0caffd640 1 -- 172.21.15.117:0/1206238249 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe0dc0cd210 con 0x55a70d4bc1b0 2024-09-21T06:08:37.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.041+0000 7fe0f81e6740 10 client.18129 fetch_fsmap learned FSMap version 29 2024-09-21T06:08:37.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.041+0000 7fe0f81e6740 1 -- 172.21.15.117:0/1206238249 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a70d8a6b30 con 0x55a70d4bc1b0 2024-09-21T06:08:37.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.041+0000 7fe0caffd640 1 -- 172.21.15.117:0/1206238249 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0e0015070 con 0x55a70d4bc1b0 2024-09-21T06:08:37.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.041+0000 7fe0caffd640 1 -- 172.21.15.117:0/1206238249 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe0dc10f040 con 0x55a70d4bc1b0 2024-09-21T06:08:37.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.041+0000 7fe0f81e6740 10 client.18129 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:08:37.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.041+0000 7fe0f81e6740 10 client.18129 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:08:37.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.041+0000 7fe0f81e6740 20 client.18129 populate_metadata read hostname 'smithi117' 2024-09-21T06:08:37.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.041+0000 7fe0f81e6740 1 --2- 172.21.15.117:0/1206238249 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55a70d4a39f0 0x55a70d8aa030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:37.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.041+0000 7fe0f81e6740 4 client.18129 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:08:37.051 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.041+0000 7fe0f81e6740 1 -- 172.21.15.117:0/1206238249 --> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a70d804df0 con 0x55a70d4a39f0 2024-09-21T06:08:37.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.042+0000 7fe0eaffd640 1 --2- 172.21.15.117:0/1206238249 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55a70d4a39f0 0x55a70d8aa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:37.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.042+0000 7fe0eaffd640 1 --2- 172.21.15.117:0/1206238249 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55a70d4a39f0 0x55a70d8aa030 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a26965b661863c64 server_cookie=807249ec9cdfb329 in_seq=0 out_seq=0 2024-09-21T06:08:37.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.042+0000 7fe0caffd640 10 client.18129 ms_handle_connect on v2:172.21.15.192:6836/2683860842 2024-09-21T06:08:37.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.043+0000 7fe0caffd640 1 -- 172.21.15.117:0/1206238249 <== mds.2 v2:172.21.15.192:6836/2683860842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a70d804df0 con 0x55a70d4a39f0 2024-09-21T06:08:37.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.043+0000 7fe0caffd640 10 client.18129 handle_command_reply: tid=0 2024-09-21T06:08:37.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.043+0000 7fe0f81e6740 1 client.18129 shutdown 2024-09-21T06:08:37.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.043+0000 7fe0f81e6740 1 -- 172.21.15.117:0/1206238249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70d4bc1b0 msgr2=0x55a70d897d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:37.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.043+0000 7fe0f81e6740 1 --2- 172.21.15.117:0/1206238249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70d4bc1b0 0x55a70d897d00 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc006010 tx=0x7fe0dc032f60 comp rx=0 tx=0).stop 2024-09-21T06:08:37.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.043+0000 7fe0f81e6740 1 -- 172.21.15.117:0/1206238249 shutdown_connections 2024-09-21T06:08:37.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.043+0000 7fe0f81e6740 1 --2- 172.21.15.117:0/1206238249 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55a70d4a39f0 0x55a70d8aa030 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.043+0000 7fe0f81e6740 1 --2- 172.21.15.117:0/1206238249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70d4e5070 0x55a70d8a4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.043+0000 7fe0f81e6740 1 --2- 172.21.15.117:0/1206238249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70d4c4390 0x55a70d898240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.052 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.043+0000 7fe0f81e6740 1 --2- 172.21.15.117:0/1206238249 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70d4bc1b0 0x55a70d897d00 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.043+0000 7fe0caffd640 0 client.18129 ms_handle_reset on v2:172.21.15.192:6836/2683860842 2024-09-21T06:08:37.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.043+0000 7fe0f81e6740 1 -- 172.21.15.117:0/1206238249 >> 172.21.15.117:0/1206238249 conn(0x55a70d4d9460 msgr2=0x55a70d88f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:37.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.043+0000 7fe0f81e6740 1 -- 172.21.15.117:0/1206238249 shutdown_connections 2024-09-21T06:08:37.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.043+0000 7fe0f81e6740 1 -- 172.21.15.117:0/1206238249 wait complete. 2024-09-21T06:08:37.053 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.043+0000 7fe0f81e6740 20 client.18129 trim_cache size 0 max 16384 2024-09-21T06:08:37.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.056+0000 7fe0f81e6740 1 Processor -- start 2024-09-21T06:08:37.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.056+0000 7fe0f81e6740 1 -- start start 2024-09-21T06:08:37.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.056+0000 7fe0f81e6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70d4bc1b0 0x55a70d8981c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:37.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.056+0000 7fe0f81e6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70d4c4390 0x55a70d889bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:37.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.057+0000 7fe0f81e6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70d4e5070 0x55a70d88a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:37.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.057+0000 7fe0f81e6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec1af520 con 0x55a70d4bc1b0 2024-09-21T06:08:37.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.057+0000 7fe0f81e6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec1b1060 con 0x55a70d4e5070 2024-09-21T06:08:37.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.057+0000 7fe0f81e6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec1afbf0 con 0x55a70d4c4390 2024-09-21T06:08:37.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.057+0000 7fe0eb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70d4bc1b0 0x55a70d8981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:37.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.057+0000 7fe0eb7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70d4bc1b0 0x55a70d8981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37586/0 (socket says 172.21.15.117:37586) 2024-09-21T06:08:37.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.057+0000 7fe0eb7fe640 1 -- 172.21.15.117:0/51566225 learned_addr learned my addr 172.21.15.117:0/51566225 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:37.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.057+0000 7fe0eaffd640 1 --2- 172.21.15.117:0/51566225 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70d4c4390 0x55a70d889bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:37.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.057+0000 7fe0ebfff640 1 --2- 172.21.15.117:0/51566225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70d4e5070 0x55a70d88a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:37.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.057+0000 7fe0eb7fe640 1 -- 172.21.15.117:0/51566225 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70d4c4390 msgr2=0x55a70d889bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:37.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.057+0000 7fe0eb7fe640 1 --2- 172.21.15.117:0/51566225 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70d4c4390 0x55a70d889bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.057+0000 7fe0eb7fe640 1 -- 172.21.15.117:0/51566225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70d4e5070 msgr2=0x55a70d88a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:37.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.057+0000 7fe0eb7fe640 1 --2- 172.21.15.117:0/51566225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70d4e5070 0x55a70d88a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.057+0000 7fe0eb7fe640 1 -- 172.21.15.117:0/51566225 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a70d4a3c20 con 0x55a70d4bc1b0 2024-09-21T06:08:37.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.057+0000 7fe0eaffd640 1 --2- 172.21.15.117:0/51566225 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70d4c4390 0x55a70d889bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:08:37.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.057+0000 7fe0eb7fe640 1 --2- 172.21.15.117:0/51566225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70d4bc1b0 0x55a70d8981c0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc0cd7a0 tx=0x7fe0dc0cd5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a16e08c3f86311f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:37.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.057+0000 7fe0d17fa640 1 -- 172.21.15.117:0/51566225 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc049070 con 0x55a70d4bc1b0 2024-09-21T06:08:37.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.057+0000 7fe0d17fa640 1 -- 172.21.15.117:0/51566225 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0dc0d2580 con 0x55a70d4bc1b0 2024-09-21T06:08:37.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.057+0000 7fe0d17fa640 1 -- 172.21.15.117:0/51566225 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc10b240 con 0x55a70d4bc1b0 2024-09-21T06:08:37.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.058+0000 7fe0f81e6740 1 -- 172.21.15.117:0/51566225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70d4bc1b0 msgr2=0x55a70d8981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:37.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.058+0000 7fe0f81e6740 1 --2- 172.21.15.117:0/51566225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70d4bc1b0 0x55a70d8981c0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc0cd7a0 tx=0x7fe0dc0cd5b0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.058+0000 7fe0f81e6740 1 -- 172.21.15.117:0/51566225 shutdown_connections 2024-09-21T06:08:37.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.058+0000 7fe0f81e6740 1 --2- 172.21.15.117:0/51566225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70d4e5070 0x55a70d88a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.058+0000 7fe0f81e6740 1 --2- 172.21.15.117:0/51566225 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70d4c4390 0x55a70d889bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.058+0000 7fe0f81e6740 1 --2- 172.21.15.117:0/51566225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70d4bc1b0 0x55a70d8981c0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.058+0000 7fe0f81e6740 1 -- 172.21.15.117:0/51566225 >> 172.21.15.117:0/51566225 conn(0x55a70d4d9460 msgr2=0x55a70d897ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:37.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.058+0000 7fe0f81e6740 1 -- 172.21.15.117:0/51566225 shutdown_connections 2024-09-21T06:08:37.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.058+0000 7fe0f81e6740 1 -- 172.21.15.117:0/51566225 wait complete. 2024-09-21T06:08:37.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.058+0000 7fe0f81e6740 1 -- start start 2024-09-21T06:08:37.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.059+0000 7fe0f81e6740 1 Processor -- start 2024-09-21T06:08:37.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.059+0000 7fe0f81e6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70d4bc1b0 0x55a70d8a0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:37.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.059+0000 7fe0f81e6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70d4c4390 0x55a70d89be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:37.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.059+0000 7fe0f81e6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70d4e5070 0x55a70d89c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:37.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.059+0000 7fe0f81e6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec1b0990 con 0x55a70d4e5070 2024-09-21T06:08:37.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.059+0000 7fe0f81e6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec1b02c0 con 0x55a70d4bc1b0 2024-09-21T06:08:37.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.059+0000 7fe0f81e6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec1ad520 con 0x55a70d4c4390 2024-09-21T06:08:37.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.059+0000 7fe0f81e6740 10 client.0 _ll_register_callbacks cb 0x7fe0dc009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:08:37.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.059+0000 7fe0ebfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70d4e5070 0x55a70d89c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:37.066 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.059+0000 7fe0eb7fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70d4bc1b0 0x55a70d8a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:37.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.059+0000 7fe0ebfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70d4e5070 0x55a70d89c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37592/0 (socket says 172.21.15.117:37592) 2024-09-21T06:08:37.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.059+0000 7fe0ebfff640 1 -- 172.21.15.117:0/2791516698 learned_addr learned my addr 172.21.15.117:0/2791516698 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:37.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.059+0000 7fe0eaffd640 1 --2- 172.21.15.117:0/2791516698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70d4c4390 0x55a70d89be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:37.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.060+0000 7fe0ebfff640 1 -- 172.21.15.117:0/2791516698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70d4c4390 msgr2=0x55a70d89be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:37.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.060+0000 7fe0ebfff640 1 --2- 172.21.15.117:0/2791516698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70d4c4390 0x55a70d89be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.060+0000 7fe0ebfff640 1 -- 172.21.15.117:0/2791516698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70d4bc1b0 msgr2=0x55a70d8a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:37.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.060+0000 7fe0ebfff640 1 --2- 172.21.15.117:0/2791516698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70d4bc1b0 0x55a70d8a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.060+0000 7fe0ebfff640 1 -- 172.21.15.117:0/2791516698 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a70d89cb90 con 0x55a70d4e5070 2024-09-21T06:08:37.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.060+0000 7fe0eb7fe640 1 --2- 172.21.15.117:0/2791516698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70d4bc1b0 0x55a70d8a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:08:37.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.060+0000 7fe0ebfff640 1 --2- 172.21.15.117:0/2791516698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70d4e5070 0x55a70d89c400 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4020b50 tx=0x7fe0e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83138eaee5acdbef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:37.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.060+0000 7fe0caffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:08:37.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.060+0000 7fe0caffd640 1 -- 172.21.15.117:0/2791516698 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4020020 con 0x55a70d4e5070 2024-09-21T06:08:37.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.060+0000 7fe0caffd640 1 -- 172.21.15.117:0/2791516698 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7fe0e40201c0 con 0x55a70d4e5070 2024-09-21T06:08:37.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.060+0000 7fe0f81e6740 1 -- 172.21.15.117:0/2791516698 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0dc0d2580 con 0x55a70d4e5070 2024-09-21T06:08:37.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.060+0000 7fe0caffd640 1 -- 172.21.15.117:0/2791516698 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe0e401eb20 con 0x55a70d4e5070 2024-09-21T06:08:37.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.060+0000 7fe0f81e6740 10 client.18135 fetch_fsmap learned FSMap version 29 2024-09-21T06:08:37.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.060+0000 7fe0f81e6740 1 -- 172.21.15.117:0/2791516698 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a70d9149b0 con 0x55a70d4e5070 2024-09-21T06:08:37.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.060+0000 7fe0caffd640 1 -- 172.21.15.117:0/2791516698 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0dc0d2580 con 0x55a70d4e5070 2024-09-21T06:08:37.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.061+0000 7fe0caffd640 1 -- 172.21.15.117:0/2791516698 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe0e400f040 con 0x55a70d4e5070 2024-09-21T06:08:37.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.061+0000 7fe0f81e6740 10 client.18135 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:08:37.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.061+0000 7fe0f81e6740 10 client.18135 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-21T06:08:37.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.061+0000 7fe0f81e6740 20 client.18135 populate_metadata read hostname 'smithi117' 2024-09-21T06:08:37.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.061+0000 7fe0f81e6740 1 --2- 172.21.15.117:0/2791516698 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55a70d478da0 0x55a70d917220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:37.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.061+0000 7fe0f81e6740 4 client.18135 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:08:37.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.061+0000 7fe0f81e6740 1 -- 172.21.15.117:0/2791516698 --> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a70d8008a0 con 0x55a70d478da0 2024-09-21T06:08:37.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.061+0000 7fe0eb7fe640 1 --2- 172.21.15.117:0/2791516698 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55a70d478da0 0x55a70d917220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:37.068 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.061+0000 7fe0eb7fe640 1 --2- 172.21.15.117:0/2791516698 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55a70d478da0 0x55a70d917220 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8a299c7d3b1d8c65 server_cookie=e62bf4b686f1de7d in_seq=0 out_seq=0 2024-09-21T06:08:37.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.061+0000 7fe0caffd640 10 client.18135 ms_handle_connect on v2:172.21.15.192:6836/2683860842 2024-09-21T06:08:37.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.062+0000 7fe0caffd640 1 -- 172.21.15.117:0/2791516698 <== mds.2 v2:172.21.15.192:6836/2683860842 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a70d8008a0 con 0x55a70d478da0 2024-09-21T06:08:37.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.062+0000 7fe0caffd640 10 client.18135 handle_command_reply: tid=0 2024-09-21T06:08:37.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.062+0000 7fe0f81e6740 1 client.18135 shutdown 2024-09-21T06:08:37.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.062+0000 7fe0f81e6740 1 -- 172.21.15.117:0/2791516698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70d4e5070 msgr2=0x55a70d89c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:37.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.062+0000 7fe0f81e6740 1 --2- 172.21.15.117:0/2791516698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70d4e5070 0x55a70d89c400 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4020b50 tx=0x7fe0e4004a40 comp rx=0 tx=0).stop 2024-09-21T06:08:37.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.062+0000 7fe0f81e6740 1 -- 172.21.15.117:0/2791516698 shutdown_connections 2024-09-21T06:08:37.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.062+0000 7fe0f81e6740 1 --2- 172.21.15.117:0/2791516698 >> [v2:172.21.15.192:6836/2683860842,v1:172.21.15.192:6837/2683860842] conn(0x55a70d478da0 0x55a70d917220 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.062+0000 7fe0f81e6740 1 --2- 172.21.15.117:0/2791516698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a70d4e5070 0x55a70d89c400 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.062+0000 7fe0caffd640 0 client.18135 ms_handle_reset on v2:172.21.15.192:6836/2683860842 2024-09-21T06:08:37.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.062+0000 7fe0f81e6740 1 --2- 172.21.15.117:0/2791516698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a70d4c4390 0x55a70d89be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.062+0000 7fe0f81e6740 1 --2- 172.21.15.117:0/2791516698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a70d4bc1b0 0x55a70d8a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.062+0000 7fe0f81e6740 1 -- 172.21.15.117:0/2791516698 >> 172.21.15.117:0/2791516698 conn(0x55a70d4d9460 msgr2=0x55a70d88f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:37.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.062+0000 7fe0f81e6740 1 -- 172.21.15.117:0/2791516698 shutdown_connections 2024-09-21T06:08:37.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.062+0000 7fe0f81e6740 1 -- 172.21.15.117:0/2791516698 wait complete. 2024-09-21T06:08:37.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.062+0000 7fe0f81e6740 20 client.18135 trim_cache size 0 max 16384 2024-09-21T06:08:37.070 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:08:37.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.063+0000 7fe0f1c72640 1 -- 172.21.15.117:0/1247573102 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe0d40b6680 msgr2=0x7fe0d40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:37.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.063+0000 7fe0f1c72640 1 --2- 172.21.15.117:0/1247573102 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe0d40b6680 0x7fe0d40b8b90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4005fd0 tx=0x7fe0e4009740 comp rx=0 tx=0).stop 2024-09-21T06:08:37.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.063+0000 7fe0f1c72640 1 -- 172.21.15.117:0/1247573102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0ec103b60 msgr2=0x7fe0ec19a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:37.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.063+0000 7fe0f1c72640 1 --2- 172.21.15.117:0/1247573102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0ec103b60 0x7fe0ec19a4a0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc005940 tx=0x7fe0dc0042f0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.063+0000 7fe0f1c72640 1 -- 172.21.15.117:0/1247573102 shutdown_connections 2024-09-21T06:08:37.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.063+0000 7fe0f1c72640 1 --2- 172.21.15.117:0/1247573102 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe0d40b6680 0x7fe0d40b8b90 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.063+0000 7fe0f1c72640 1 --2- 172.21.15.117:0/1247573102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0ec109df0 0x7fe0ec19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.063+0000 7fe0f1c72640 1 --2- 172.21.15.117:0/1247573102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0ec1094a0 0x7fe0ec19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.063+0000 7fe0f1c72640 1 --2- 172.21.15.117:0/1247573102 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0ec103b60 0x7fe0ec19a4a0 secure :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc005940 tx=0x7fe0dc0042f0 comp rx=0 tx=0).stop 2024-09-21T06:08:37.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.063+0000 7fe0f1c72640 1 -- 172.21.15.117:0/1247573102 >> 172.21.15.117:0/1247573102 conn(0x7fe0ec07b850 msgr2=0x7fe0ec107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:37.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.063+0000 7fe0f1c72640 1 -- 172.21.15.117:0/1247573102 shutdown_connections 2024-09-21T06:08:37.071 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:37.063+0000 7fe0f1c72640 1 -- 172.21.15.117:0/1247573102 wait complete. 2024-09-21T06:08:37.751 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:1 damage ls 2024-09-21T06:08:37.998 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:08:38.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:37 smithi117 ceph-mon[30947]: pgmap v675: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 KiB/s wr, 225 op/s 2024-09-21T06:08:38.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:37 smithi192 ceph-mon[34700]: pgmap v675: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 KiB/s wr, 225 op/s 2024-09-21T06:08:38.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:37 smithi138 ceph-mon[34391]: pgmap v675: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 KiB/s wr, 225 op/s 2024-09-21T06:08:38.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.508+0000 7fce71e91640 1 -- 172.21.15.117:0/1116706974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce6c104a30 msgr2=0x7fce6c104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:38.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.508+0000 7fce71e91640 1 --2- 172.21.15.117:0/1116706974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce6c104a30 0x7fce6c104e40 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fce5c009900 tx=0x7fce5c02a410 comp rx=0 tx=0).stop 2024-09-21T06:08:38.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.509+0000 7fce71e91640 1 -- 172.21.15.117:0/1116706974 shutdown_connections 2024-09-21T06:08:38.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.509+0000 7fce71e91640 1 --2- 172.21.15.117:0/1116706974 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce6c10de90 0x7fce6c110380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:38.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.509+0000 7fce71e91640 1 --2- 172.21.15.117:0/1116706974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce6c105380 0x7fce6c10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:38.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.509+0000 7fce71e91640 1 --2- 172.21.15.117:0/1116706974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce6c104a30 0x7fce6c104e40 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:38.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.509+0000 7fce71e91640 1 -- 172.21.15.117:0/1116706974 >> 172.21.15.117:0/1116706974 conn(0x7fce6c079770 msgr2=0x7fce6c079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:38.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.509+0000 7fce71e91640 1 -- 172.21.15.117:0/1116706974 shutdown_connections 2024-09-21T06:08:38.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.509+0000 7fce71e91640 1 -- 172.21.15.117:0/1116706974 wait complete. 2024-09-21T06:08:38.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.509+0000 7fce71e91640 1 Processor -- start 2024-09-21T06:08:38.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.509+0000 7fce71e91640 1 -- start start 2024-09-21T06:08:38.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.510+0000 7fce71e91640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce6c104a30 0x7fce6c102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:38.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.510+0000 7fce71e91640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce6c105380 0x7fce6c102cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:38.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.510+0000 7fce71e91640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce6c10de90 0x7fce6c1031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:38.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.510+0000 7fce71e91640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c113b10 con 0x7fce6c105380 2024-09-21T06:08:38.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.510+0000 7fce71e91640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c113990 con 0x7fce6c104a30 2024-09-21T06:08:38.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.510+0000 7fce71e91640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c113c90 con 0x7fce6c10de90 2024-09-21T06:08:38.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.510+0000 7fce62ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce6c105380 0x7fce6c102cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:38.521 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.510+0000 7fce62ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce6c105380 0x7fce6c102cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37622/0 (socket says 172.21.15.117:37622) 2024-09-21T06:08:38.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.510+0000 7fce62ffd640 1 -- 172.21.15.117:0/2429030074 learned_addr learned my addr 172.21.15.117:0/2429030074 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:38.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.510+0000 7fce6bfff640 1 --2- 172.21.15.117:0/2429030074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce6c10de90 0x7fce6c1031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:38.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.510+0000 7fce6b7fe640 1 --2- 172.21.15.117:0/2429030074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce6c104a30 0x7fce6c102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:38.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.510+0000 7fce62ffd640 1 -- 172.21.15.117:0/2429030074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce6c10de90 msgr2=0x7fce6c1031f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:38.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.510+0000 7fce62ffd640 1 --2- 172.21.15.117:0/2429030074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce6c10de90 0x7fce6c1031f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:38.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.510+0000 7fce62ffd640 1 -- 172.21.15.117:0/2429030074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce6c104a30 msgr2=0x7fce6c102770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:38.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.510+0000 7fce62ffd640 1 --2- 172.21.15.117:0/2429030074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce6c104a30 0x7fce6c102770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:38.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.510+0000 7fce62ffd640 1 -- 172.21.15.117:0/2429030074 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce6c1a87a0 con 0x7fce6c105380 2024-09-21T06:08:38.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.510+0000 7fce62ffd640 1 --2- 172.21.15.117:0/2429030074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce6c105380 0x7fce6c102cb0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fce5400d740 tx=0x7fce5400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:38.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.511+0000 7fce697fa640 1 -- 172.21.15.117:0/2429030074 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce54017070 con 0x7fce6c105380 2024-09-21T06:08:38.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.511+0000 7fce697fa640 1 -- 172.21.15.117:0/2429030074 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce5400de50 con 0x7fce6c105380 2024-09-21T06:08:38.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.511+0000 7fce71e91640 1 -- 172.21.15.117:0/2429030074 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce6c1a8a90 con 0x7fce6c105380 2024-09-21T06:08:38.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.511+0000 7fce697fa640 1 -- 172.21.15.117:0/2429030074 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5400f620 con 0x7fce6c105380 2024-09-21T06:08:38.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.511+0000 7fce71e91640 1 -- 172.21.15.117:0/2429030074 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce6c1a8fd0 con 0x7fce6c105380 2024-09-21T06:08:38.522 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.513+0000 7fce78405740 1 Processor -- start 2024-09-21T06:08:38.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.513+0000 7fce78405740 1 -- start start 2024-09-21T06:08:38.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.513+0000 7fce697fa640 1 -- 172.21.15.117:0/2429030074 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fce5401a070 con 0x7fce6c105380 2024-09-21T06:08:38.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.513+0000 7fce78405740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d86b9f070 0x558d86f466a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:38.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.513+0000 7fce78405740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d86b761b0 0x558d86f46be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:38.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.513+0000 7fce78405740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d86b7e390 0x558d86f47120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:38.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.513+0000 7fce78405740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558d86ebcea0 con 0x558d86b9f070 2024-09-21T06:08:38.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.513+0000 7fce78405740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558d86eac2d0 con 0x558d86b7e390 2024-09-21T06:08:38.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.513+0000 7fce78405740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558d86eb00e0 con 0x558d86b761b0 2024-09-21T06:08:38.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.514+0000 7fce6bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d86b9f070 0x558d86f466a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:38.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.514+0000 7fce6bfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d86b9f070 0x558d86f466a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37634/0 (socket says 172.21.15.117:37634) 2024-09-21T06:08:38.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.514+0000 7fce6bfff640 1 -- 172.21.15.117:0/1987922117 learned_addr learned my addr 172.21.15.117:0/1987922117 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:38.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.514+0000 7fce6b7fe640 1 --2- 172.21.15.117:0/1987922117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d86b761b0 0x558d86f46be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:38.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.514+0000 7fce62ffd640 1 --2- 172.21.15.117:0/1987922117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d86b7e390 0x558d86f47120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:38.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.514+0000 7fce6bfff640 1 -- 172.21.15.117:0/1987922117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d86b761b0 msgr2=0x558d86f46be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:38.523 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.514+0000 7fce6bfff640 1 --2- 172.21.15.117:0/1987922117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d86b761b0 0x558d86f46be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:38.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.514+0000 7fce6bfff640 1 -- 172.21.15.117:0/1987922117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d86b7e390 msgr2=0x558d86f47120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:38.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.514+0000 7fce6bfff640 1 --2- 172.21.15.117:0/1987922117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d86b7e390 0x558d86f47120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:38.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.514+0000 7fce6bfff640 1 -- 172.21.15.117:0/1987922117 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d86c0a650 con 0x558d86b9f070 2024-09-21T06:08:38.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.514+0000 7fce62ffd640 1 --2- 172.21.15.117:0/1987922117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d86b7e390 0x558d86f47120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:08:38.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.514+0000 7fce6bfff640 1 --2- 172.21.15.117:0/1987922117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d86b9f070 0x558d86f466a0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fce64010370 tx=0x7fce64010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1977f5c73f727df2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:38.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.514+0000 7fce697fa640 1 --2- 172.21.15.117:0/2429030074 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fce580b6680 0x7fce580b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:38.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.514+0000 7fce61ffb640 1 -- 172.21.15.117:0/1987922117 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce64016070 con 0x558d86b9f070 2024-09-21T06:08:38.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.514+0000 7fce61ffb640 1 -- 172.21.15.117:0/1987922117 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce64004d20 con 0x558d86b9f070 2024-09-21T06:08:38.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.514+0000 7fce61ffb640 1 -- 172.21.15.117:0/1987922117 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce64005350 con 0x558d86b9f070 2024-09-21T06:08:38.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.514+0000 7fce6b7fe640 1 --2- 172.21.15.117:0/2429030074 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fce580b6680 0x7fce580b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:38.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.515+0000 7fce697fa640 1 -- 172.21.15.117:0/2429030074 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7fce540e7020 con 0x7fce6c105380 2024-09-21T06:08:38.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.515+0000 7fce78405740 1 -- 172.21.15.117:0/1987922117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d86b9f070 msgr2=0x558d86f466a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:38.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.515+0000 7fce6b7fe640 1 --2- 172.21.15.117:0/2429030074 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fce580b6680 0x7fce580b8b90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fce5c000c00 tx=0x7fce5c031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:38.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.515+0000 7fce78405740 1 --2- 172.21.15.117:0/1987922117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d86b9f070 0x558d86f466a0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fce64010370 tx=0x7fce64010830 comp rx=0 tx=0).stop 2024-09-21T06:08:38.524 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.515+0000 7fce78405740 1 -- 172.21.15.117:0/1987922117 shutdown_connections 2024-09-21T06:08:38.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.515+0000 7fce78405740 1 --2- 172.21.15.117:0/1987922117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d86b7e390 0x558d86f47120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:38.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.515+0000 7fce78405740 1 --2- 172.21.15.117:0/1987922117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d86b761b0 0x558d86f46be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:38.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.515+0000 7fce78405740 1 --2- 172.21.15.117:0/1987922117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d86b9f070 0x558d86f466a0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:38.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.515+0000 7fce78405740 1 -- 172.21.15.117:0/1987922117 >> 172.21.15.117:0/1987922117 conn(0x558d86b93460 msgr2=0x558d86f44590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:38.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.515+0000 7fce78405740 1 -- 172.21.15.117:0/1987922117 shutdown_connections 2024-09-21T06:08:38.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.515+0000 7fce78405740 1 -- 172.21.15.117:0/1987922117 wait complete. 2024-09-21T06:08:38.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.516+0000 7fce78405740 1 -- start start 2024-09-21T06:08:38.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.516+0000 7fce78405740 1 Processor -- start 2024-09-21T06:08:38.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.516+0000 7fce78405740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d86b761b0 0x558d86f51da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:38.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.516+0000 7fce78405740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d86b7e390 0x558d86f522e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:38.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.516+0000 7fce78405740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d86b9f070 0x558d86f5e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:38.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.516+0000 7fce78405740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558d86d0d700 con 0x558d86b761b0 2024-09-21T06:08:38.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.516+0000 7fce78405740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558d86e47160 con 0x558d86b9f070 2024-09-21T06:08:38.525 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.516+0000 7fce78405740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558d86f54e80 con 0x558d86b7e390 2024-09-21T06:08:38.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.516+0000 7fce78405740 10 client.0 _ll_register_callbacks cb 0x558d86e609e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:08:38.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.516+0000 7fce6b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d86b761b0 0x558d86f51da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:38.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.516+0000 7fce6b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d86b761b0 0x558d86f51da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37644/0 (socket says 172.21.15.117:37644) 2024-09-21T06:08:38.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.516+0000 7fce6b7fe640 1 -- 172.21.15.117:0/2964252668 learned_addr learned my addr 172.21.15.117:0/2964252668 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:38.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.517+0000 7fce62ffd640 1 --2- 172.21.15.117:0/2964252668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d86b7e390 0x558d86f522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:38.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.517+0000 7fce6bfff640 1 --2- 172.21.15.117:0/2964252668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d86b9f070 0x558d86f5e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:38.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.517+0000 7fce6b7fe640 1 -- 172.21.15.117:0/2964252668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d86b7e390 msgr2=0x558d86f522e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:38.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.517+0000 7fce6b7fe640 1 --2- 172.21.15.117:0/2964252668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d86b7e390 0x558d86f522e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:38.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.517+0000 7fce6b7fe640 1 -- 172.21.15.117:0/2964252668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d86b9f070 msgr2=0x558d86f5e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:38.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.517+0000 7fce6b7fe640 1 --2- 172.21.15.117:0/2964252668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d86b9f070 0x558d86f5e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:38.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.517+0000 7fce6b7fe640 1 -- 172.21.15.117:0/2964252668 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d86e74c90 con 0x558d86b761b0 2024-09-21T06:08:38.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.517+0000 7fce6b7fe640 1 --2- 172.21.15.117:0/2964252668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d86b761b0 0x558d86f51da0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fce5c005940 tx=0x7fce5c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05dfccd738bb289 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:38.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.517+0000 7fce4affd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:08:38.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.517+0000 7fce4affd640 1 -- 172.21.15.117:0/2964252668 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c04c070 con 0x558d86b761b0 2024-09-21T06:08:38.526 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.517+0000 7fce4affd640 1 -- 172.21.15.117:0/2964252668 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7fce5c005030 con 0x558d86b761b0 2024-09-21T06:08:38.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.517+0000 7fce78405740 1 -- 172.21.15.117:0/2964252668 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce64004d20 con 0x558d86b761b0 2024-09-21T06:08:38.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.517+0000 7fce4affd640 1 -- 172.21.15.117:0/2964252668 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fce5c058050 con 0x558d86b761b0 2024-09-21T06:08:38.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.517+0000 7fce78405740 10 client.18147 fetch_fsmap learned FSMap version 29 2024-09-21T06:08:38.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.517+0000 7fce78405740 1 -- 172.21.15.117:0/2964252668 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558d86f60b10 con 0x558d86b761b0 2024-09-21T06:08:38.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.518+0000 7fce4affd640 1 -- 172.21.15.117:0/2964252668 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce64004d20 con 0x558d86b761b0 2024-09-21T06:08:38.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.518+0000 7fce4affd640 1 -- 172.21.15.117:0/2964252668 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fce5c0035f0 con 0x558d86b761b0 2024-09-21T06:08:38.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.518+0000 7fce78405740 10 client.18147 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:08:38.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.518+0000 7fce78405740 10 client.18147 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:08:38.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.518+0000 7fce78405740 20 client.18147 populate_metadata read hostname 'smithi117' 2024-09-21T06:08:38.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.518+0000 7fce78405740 1 --2- 172.21.15.117:0/2964252668 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x558d86b5d9f0 0x558d86f63fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:38.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.518+0000 7fce78405740 4 client.18147 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:08:38.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.518+0000 7fce78405740 1 -- 172.21.15.117:0/2964252668 --> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d86ebedf0 con 0x558d86b5d9f0 2024-09-21T06:08:38.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.518+0000 7fce62ffd640 1 --2- 172.21.15.117:0/2964252668 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x558d86b5d9f0 0x558d86f63fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:38.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.519+0000 7fce62ffd640 1 --2- 172.21.15.117:0/2964252668 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x558d86b5d9f0 0x558d86f63fe0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e4bfca3a43efdf8a server_cookie=5745f0760445443a in_seq=0 out_seq=0 2024-09-21T06:08:38.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.519+0000 7fce4affd640 10 client.18147 ms_handle_connect on v2:172.21.15.192:6838/1727623231 2024-09-21T06:08:38.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.520+0000 7fce4affd640 1 -- 172.21.15.117:0/2964252668 <== mds.1 v2:172.21.15.192:6838/1727623231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558d86ebedf0 con 0x558d86b5d9f0 2024-09-21T06:08:38.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.520+0000 7fce4affd640 10 client.18147 handle_command_reply: tid=0 2024-09-21T06:08:38.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.520+0000 7fce78405740 1 client.18147 shutdown 2024-09-21T06:08:38.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.520+0000 7fce78405740 1 -- 172.21.15.117:0/2964252668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d86b761b0 msgr2=0x558d86f51da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:38.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.520+0000 7fce78405740 1 --2- 172.21.15.117:0/2964252668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d86b761b0 0x558d86f51da0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fce5c005940 tx=0x7fce5c009260 comp rx=0 tx=0).stop 2024-09-21T06:08:38.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.520+0000 7fce78405740 1 -- 172.21.15.117:0/2964252668 shutdown_connections 2024-09-21T06:08:38.529 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.520+0000 7fce78405740 1 --2- 172.21.15.117:0/2964252668 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x558d86b5d9f0 0x558d86f63fe0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:38.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.520+0000 7fce78405740 1 --2- 172.21.15.117:0/2964252668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d86b9f070 0x558d86f5e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:38.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.520+0000 7fce78405740 1 --2- 172.21.15.117:0/2964252668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d86b7e390 0x558d86f522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:38.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.520+0000 7fce78405740 1 --2- 172.21.15.117:0/2964252668 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d86b761b0 0x558d86f51da0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:38.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.520+0000 7fce78405740 1 -- 172.21.15.117:0/2964252668 >> 172.21.15.117:0/2964252668 conn(0x558d86b93460 msgr2=0x558d86f49880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:38.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.520+0000 7fce4affd640 0 client.18147 ms_handle_reset on v2:172.21.15.192:6838/1727623231 2024-09-21T06:08:38.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.520+0000 7fce78405740 1 -- 172.21.15.117:0/2964252668 shutdown_connections 2024-09-21T06:08:38.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.520+0000 7fce78405740 1 -- 172.21.15.117:0/2964252668 wait complete. 2024-09-21T06:08:38.530 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.520+0000 7fce78405740 20 client.18147 trim_cache size 0 max 16384 2024-09-21T06:08:38.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.533+0000 7fce78405740 1 Processor -- start 2024-09-21T06:08:38.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.533+0000 7fce78405740 1 -- start start 2024-09-21T06:08:38.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.533+0000 7fce78405740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d86b761b0 0x558d86f48090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:38.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.533+0000 7fce78405740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d86b7e390 0x558d86f539d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:38.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.533+0000 7fce78405740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d86b9f070 0x558d86f43bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:38.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.533+0000 7fce78405740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c1b1830 con 0x558d86b761b0 2024-09-21T06:08:38.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.533+0000 7fce78405740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c1b3370 con 0x558d86b9f070 2024-09-21T06:08:38.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.533+0000 7fce78405740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c1b1f00 con 0x558d86b7e390 2024-09-21T06:08:38.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.534+0000 7fce6b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d86b761b0 0x558d86f48090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:38.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.534+0000 7fce6b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d86b761b0 0x558d86f48090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37648/0 (socket says 172.21.15.117:37648) 2024-09-21T06:08:38.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.534+0000 7fce6b7fe640 1 -- 172.21.15.117:0/1998240993 learned_addr learned my addr 172.21.15.117:0/1998240993 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:38.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.534+0000 7fce6bfff640 1 --2- 172.21.15.117:0/1998240993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d86b9f070 0x558d86f43bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:38.536 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.534+0000 7fce62ffd640 1 --2- 172.21.15.117:0/1998240993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d86b7e390 0x558d86f539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:38.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.534+0000 7fce6b7fe640 1 -- 172.21.15.117:0/1998240993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d86b7e390 msgr2=0x558d86f539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:38.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.534+0000 7fce6b7fe640 1 --2- 172.21.15.117:0/1998240993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d86b7e390 0x558d86f539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:38.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.534+0000 7fce6b7fe640 1 -- 172.21.15.117:0/1998240993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d86b9f070 msgr2=0x558d86f43bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:38.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.534+0000 7fce6b7fe640 1 --2- 172.21.15.117:0/1998240993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d86b9f070 0x558d86f43bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:38.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.534+0000 7fce6b7fe640 1 -- 172.21.15.117:0/1998240993 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d86f48990 con 0x558d86b761b0 2024-09-21T06:08:38.537 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.534+0000 7fce6b7fe640 1 --2- 172.21.15.117:0/1998240993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d86b761b0 0x558d86f48090 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fce5c009340 tx=0x7fce5c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c858a35b481271 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:38.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.534+0000 7fce617fa640 1 -- 172.21.15.117:0/1998240993 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c04c070 con 0x558d86b761b0 2024-09-21T06:08:38.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.534+0000 7fce617fa640 1 -- 172.21.15.117:0/1998240993 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce5c0054e0 con 0x558d86b761b0 2024-09-21T06:08:38.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.534+0000 7fce617fa640 1 -- 172.21.15.117:0/1998240993 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c035040 con 0x558d86b761b0 2024-09-21T06:08:38.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.535+0000 7fce78405740 1 -- 172.21.15.117:0/1998240993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d86b761b0 msgr2=0x558d86f48090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:38.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.535+0000 7fce78405740 1 --2- 172.21.15.117:0/1998240993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d86b761b0 0x558d86f48090 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fce5c009340 tx=0x7fce5c009580 comp rx=0 tx=0).stop 2024-09-21T06:08:38.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.535+0000 7fce78405740 1 -- 172.21.15.117:0/1998240993 shutdown_connections 2024-09-21T06:08:38.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.535+0000 7fce78405740 1 --2- 172.21.15.117:0/1998240993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d86b9f070 0x558d86f43bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:38.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.535+0000 7fce78405740 1 --2- 172.21.15.117:0/1998240993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d86b7e390 0x558d86f539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:38.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.535+0000 7fce78405740 1 --2- 172.21.15.117:0/1998240993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d86b761b0 0x558d86f48090 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:38.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.535+0000 7fce78405740 1 -- 172.21.15.117:0/1998240993 >> 172.21.15.117:0/1998240993 conn(0x558d86b93460 msgr2=0x558d86f532c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:38.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.535+0000 7fce78405740 1 -- 172.21.15.117:0/1998240993 shutdown_connections 2024-09-21T06:08:38.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.535+0000 7fce78405740 1 -- 172.21.15.117:0/1998240993 wait complete. 2024-09-21T06:08:38.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.535+0000 7fce78405740 1 -- start start 2024-09-21T06:08:38.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.536+0000 7fce78405740 1 Processor -- start 2024-09-21T06:08:38.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.536+0000 7fce78405740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d86b761b0 0x558d86f5a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:38.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.536+0000 7fce78405740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d86b7e390 0x558d86f55d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:38.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.536+0000 7fce78405740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d86b9f070 0x558d86f562b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:38.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.536+0000 7fce78405740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c1b2ca0 con 0x558d86b761b0 2024-09-21T06:08:38.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.536+0000 7fce78405740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c1b25d0 con 0x558d86b7e390 2024-09-21T06:08:38.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.536+0000 7fce78405740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fce6c1af830 con 0x558d86b9f070 2024-09-21T06:08:38.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.536+0000 7fce78405740 10 client.0 _ll_register_callbacks cb 0x7fce5c005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:08:38.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.536+0000 7fce6b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d86b761b0 0x558d86f5a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:38.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.536+0000 7fce6b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d86b761b0 0x558d86f5a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37650/0 (socket says 172.21.15.117:37650) 2024-09-21T06:08:38.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.536+0000 7fce6b7fe640 1 -- 172.21.15.117:0/10755254 learned_addr learned my addr 172.21.15.117:0/10755254 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:38.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.536+0000 7fce6bfff640 1 --2- 172.21.15.117:0/10755254 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d86b9f070 0x558d86f562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:38.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.536+0000 7fce62ffd640 1 --2- 172.21.15.117:0/10755254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d86b7e390 0x558d86f55d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:38.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.537+0000 7fce6b7fe640 1 -- 172.21.15.117:0/10755254 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d86b9f070 msgr2=0x558d86f562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:38.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.537+0000 7fce6b7fe640 1 --2- 172.21.15.117:0/10755254 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d86b9f070 0x558d86f562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:38.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.537+0000 7fce6b7fe640 1 -- 172.21.15.117:0/10755254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d86b7e390 msgr2=0x558d86f55d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:38.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.537+0000 7fce6b7fe640 1 --2- 172.21.15.117:0/10755254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d86b7e390 0x558d86f55d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:38.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.537+0000 7fce6b7fe640 1 -- 172.21.15.117:0/10755254 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d86f56a20 con 0x558d86b761b0 2024-09-21T06:08:38.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.537+0000 7fce6b7fe640 1 --2- 172.21.15.117:0/10755254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d86b761b0 0x558d86f5a8c0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fce5c02a930 tx=0x7fce5c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f6830a3e530860 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:38.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.537+0000 7fce4affd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:08:38.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.537+0000 7fce4affd640 1 -- 172.21.15.117:0/10755254 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5c068020 con 0x558d86b761b0 2024-09-21T06:08:38.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.537+0000 7fce4affd640 1 -- 172.21.15.117:0/10755254 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7fce5c051440 con 0x558d86b761b0 2024-09-21T06:08:38.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.537+0000 7fce78405740 1 -- 172.21.15.117:0/10755254 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce5c0054e0 con 0x558d86b761b0 2024-09-21T06:08:38.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.537+0000 7fce4affd640 1 -- 172.21.15.117:0/10755254 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fce5c073050 con 0x558d86b761b0 2024-09-21T06:08:38.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.537+0000 7fce78405740 10 client.18153 fetch_fsmap learned FSMap version 29 2024-09-21T06:08:38.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.537+0000 7fce78405740 1 -- 172.21.15.117:0/10755254 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558d86fce8c0 con 0x558d86b761b0 2024-09-21T06:08:38.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.537+0000 7fce4affd640 1 -- 172.21.15.117:0/10755254 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce5c0054e0 con 0x558d86b761b0 2024-09-21T06:08:38.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.537+0000 7fce4affd640 1 -- 172.21.15.117:0/10755254 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fce5c048640 con 0x558d86b761b0 2024-09-21T06:08:38.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.538+0000 7fce78405740 10 client.18153 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:08:38.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.538+0000 7fce78405740 10 client.18153 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-21T06:08:38.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.538+0000 7fce78405740 20 client.18153 populate_metadata read hostname 'smithi117' 2024-09-21T06:08:38.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.538+0000 7fce78405740 1 --2- 172.21.15.117:0/10755254 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x558d86b32da0 0x558d86fd0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:38.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.538+0000 7fce78405740 4 client.18153 mds_command: new command op to 24637 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:08:38.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.538+0000 7fce78405740 1 -- 172.21.15.117:0/10755254 --> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558d86eba8a0 con 0x558d86b32da0 2024-09-21T06:08:38.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.538+0000 7fce62ffd640 1 --2- 172.21.15.117:0/10755254 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x558d86b32da0 0x558d86fd0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:38.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.538+0000 7fce4affd640 10 client.18153 ms_handle_connect on v2:172.21.15.192:6838/1727623231 2024-09-21T06:08:38.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.538+0000 7fce62ffd640 1 --2- 172.21.15.117:0/10755254 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x558d86b32da0 0x558d86fd0f60 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=50fe558d0f4d3b6f server_cookie=5f9325e76cefd40d in_seq=0 out_seq=0 2024-09-21T06:08:38.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.538+0000 7fce4affd640 1 -- 172.21.15.117:0/10755254 <== mds.1 v2:172.21.15.192:6838/1727623231 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558d86eba8a0 con 0x558d86b32da0 2024-09-21T06:08:38.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.538+0000 7fce4affd640 10 client.18153 handle_command_reply: tid=0 2024-09-21T06:08:38.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.538+0000 7fce78405740 1 client.18153 shutdown 2024-09-21T06:08:38.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.539+0000 7fce78405740 1 -- 172.21.15.117:0/10755254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d86b761b0 msgr2=0x558d86f5a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:38.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.539+0000 7fce78405740 1 --2- 172.21.15.117:0/10755254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d86b761b0 0x558d86f5a8c0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fce5c02a930 tx=0x7fce5c005740 comp rx=0 tx=0).stop 2024-09-21T06:08:38.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.539+0000 7fce78405740 1 -- 172.21.15.117:0/10755254 shutdown_connections 2024-09-21T06:08:38.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.539+0000 7fce78405740 1 --2- 172.21.15.117:0/10755254 >> [v2:172.21.15.192:6838/1727623231,v1:172.21.15.192:6839/1727623231] conn(0x558d86b32da0 0x558d86fd0f60 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:38.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.539+0000 7fce78405740 1 --2- 172.21.15.117:0/10755254 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558d86b9f070 0x558d86f562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:38.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.539+0000 7fce78405740 1 --2- 172.21.15.117:0/10755254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d86b7e390 0x558d86f55d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:38.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.539+0000 7fce78405740 1 --2- 172.21.15.117:0/10755254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d86b761b0 0x558d86f5a8c0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:38.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.539+0000 7fce78405740 1 -- 172.21.15.117:0/10755254 >> 172.21.15.117:0/10755254 conn(0x558d86b93460 msgr2=0x558d86f47e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:38.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.539+0000 7fce4affd640 0 client.18153 ms_handle_reset on v2:172.21.15.192:6838/1727623231 2024-09-21T06:08:38.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.539+0000 7fce78405740 1 -- 172.21.15.117:0/10755254 shutdown_connections 2024-09-21T06:08:38.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.539+0000 7fce78405740 1 -- 172.21.15.117:0/10755254 wait complete. 2024-09-21T06:08:38.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.539+0000 7fce78405740 20 client.18153 trim_cache size 0 max 16384 2024-09-21T06:08:38.545 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:08:38.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.539+0000 7fce71e91640 1 -- 172.21.15.117:0/2429030074 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fce580b6680 msgr2=0x7fce580b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:38.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.539+0000 7fce71e91640 1 --2- 172.21.15.117:0/2429030074 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fce580b6680 0x7fce580b8b90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fce5c000c00 tx=0x7fce5c031400 comp rx=0 tx=0).stop 2024-09-21T06:08:38.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.539+0000 7fce71e91640 1 -- 172.21.15.117:0/2429030074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce6c105380 msgr2=0x7fce6c102cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:38.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.539+0000 7fce71e91640 1 --2- 172.21.15.117:0/2429030074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce6c105380 0x7fce6c102cb0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fce5400d740 tx=0x7fce5400dc00 comp rx=0 tx=0).stop 2024-09-21T06:08:38.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.540+0000 7fce71e91640 1 -- 172.21.15.117:0/2429030074 shutdown_connections 2024-09-21T06:08:38.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.540+0000 7fce71e91640 1 --2- 172.21.15.117:0/2429030074 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fce580b6680 0x7fce580b8b90 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:38.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.540+0000 7fce71e91640 1 --2- 172.21.15.117:0/2429030074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce6c105380 0x7fce6c102cb0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:38.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.540+0000 7fce71e91640 1 --2- 172.21.15.117:0/2429030074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce6c10de90 0x7fce6c1031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:38.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.540+0000 7fce71e91640 1 --2- 172.21.15.117:0/2429030074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce6c104a30 0x7fce6c102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:38.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.540+0000 7fce71e91640 1 -- 172.21.15.117:0/2429030074 >> 172.21.15.117:0/2429030074 conn(0x7fce6c079770 msgr2=0x7fce6c100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:38.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.540+0000 7fce71e91640 1 -- 172.21.15.117:0/2429030074 shutdown_connections 2024-09-21T06:08:38.547 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:38.540+0000 7fce71e91640 1 -- 172.21.15.117:0/2429030074 wait complete. 2024-09-21T06:08:39.239 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T06:08:39.488 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:08:40.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.002+0000 7f2fd1c9b640 1 -- 172.21.15.117:0/2407471544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fcc103950 msgr2=0x7f2fcc10f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:40.021 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.002+0000 7f2fd1c9b640 1 --2- 172.21.15.117:0/2407471544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fcc103950 0x7f2fcc10f500 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4009ca0 tx=0x7f2fc402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.002+0000 7f2fd1c9b640 1 -- 172.21.15.117:0/2407471544 shutdown_connections 2024-09-21T06:08:40.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.002+0000 7f2fd1c9b640 1 --2- 172.21.15.117:0/2407471544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fcc103950 0x7f2fcc10f500 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.002+0000 7f2fd1c9b640 1 --2- 172.21.15.117:0/2407471544 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fcc103000 0x7f2fcc103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.002+0000 7f2fd1c9b640 1 --2- 172.21.15.117:0/2407471544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fcc107690 0x7f2fcc107aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.002+0000 7f2fd1c9b640 1 -- 172.21.15.117:0/2407471544 >> 172.21.15.117:0/2407471544 conn(0x7f2fcc079ea0 msgr2=0x7f2fcc07a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:40.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.002+0000 7f2fd1c9b640 1 -- 172.21.15.117:0/2407471544 shutdown_connections 2024-09-21T06:08:40.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.002+0000 7f2fd1c9b640 1 -- 172.21.15.117:0/2407471544 wait complete. 2024-09-21T06:08:40.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.003+0000 7f2fd1c9b640 1 Processor -- start 2024-09-21T06:08:40.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.003+0000 7f2fd1c9b640 1 -- start start 2024-09-21T06:08:40.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.003+0000 7f2fd1c9b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fcc103000 0x7f2fcc1994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:40.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.003+0000 7f2fd1c9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fcc103950 0x7f2fcc199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:40.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.003+0000 7f2fd1c9b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fcc107690 0x7f2fcc19ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:40.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.003+0000 7f2fd1c9b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc112a40 con 0x7f2fcc103950 2024-09-21T06:08:40.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.003+0000 7f2fd1c9b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc1128c0 con 0x7f2fcc107690 2024-09-21T06:08:40.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.003+0000 7f2fd1c9b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc112bc0 con 0x7f2fcc103000 2024-09-21T06:08:40.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.003+0000 7f2fcaffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fcc103950 0x7f2fcc199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:40.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.003+0000 7f2fcaffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fcc103950 0x7f2fcc199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37672/0 (socket says 172.21.15.117:37672) 2024-09-21T06:08:40.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.003+0000 7f2fcaffd640 1 -- 172.21.15.117:0/741771531 learned_addr learned my addr 172.21.15.117:0/741771531 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:40.023 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.003+0000 7f2fcb7fe640 1 --2- 172.21.15.117:0/741771531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fcc103000 0x7f2fcc1994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:40.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.004+0000 7f2fcbfff640 1 --2- 172.21.15.117:0/741771531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fcc107690 0x7f2fcc19ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:40.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.004+0000 7f2fcaffd640 1 -- 172.21.15.117:0/741771531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fcc103000 msgr2=0x7f2fcc1994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:40.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.004+0000 7f2fcaffd640 1 --2- 172.21.15.117:0/741771531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fcc103000 0x7f2fcc1994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.004+0000 7f2fcaffd640 1 -- 172.21.15.117:0/741771531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fcc107690 msgr2=0x7f2fcc19ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:40.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.004+0000 7f2fcaffd640 1 --2- 172.21.15.117:0/741771531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fcc107690 0x7f2fcc19ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.004+0000 7f2fcaffd640 1 -- 172.21.15.117:0/741771531 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fcc19f440 con 0x7f2fcc103950 2024-09-21T06:08:40.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.004+0000 7f2fcbfff640 1 --2- 172.21.15.117:0/741771531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fcc107690 0x7f2fcc19ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:08:40.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.004+0000 7f2fcaffd640 1 --2- 172.21.15.117:0/741771531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fcc103950 0x7f2fcc199a70 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0005d00 tx=0x7f2fc000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:40.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.004+0000 7f2fc8ff9640 1 -- 172.21.15.117:0/741771531 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0017070 con 0x7f2fcc103950 2024-09-21T06:08:40.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.004+0000 7f2fd1c9b640 1 -- 172.21.15.117:0/741771531 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fcc19f730 con 0x7f2fcc103950 2024-09-21T06:08:40.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.004+0000 7f2fc8ff9640 1 -- 172.21.15.117:0/741771531 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fc0009c30 con 0x7f2fcc103950 2024-09-21T06:08:40.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.004+0000 7f2fc8ff9640 1 -- 172.21.15.117:0/741771531 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc000f620 con 0x7f2fcc103950 2024-09-21T06:08:40.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.004+0000 7f2fd1c9b640 1 -- 172.21.15.117:0/741771531 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fcc1a5e90 con 0x7f2fcc103950 2024-09-21T06:08:40.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.007+0000 7f2fd820f740 1 Processor -- start 2024-09-21T06:08:40.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.007+0000 7f2fd820f740 1 -- start start 2024-09-21T06:08:40.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.007+0000 7f2fc8ff9640 1 -- 172.21.15.117:0/741771531 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f2fc001a070 con 0x7f2fcc103950 2024-09-21T06:08:40.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.007+0000 7f2fd820f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c5f68070 0x55f1c630f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:40.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.007+0000 7f2fd820f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1c5f3f1b0 0x55f1c630f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:40.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.007+0000 7f2fd820f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1c5f47390 0x55f1c630fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:40.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.007+0000 7f2fd820f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f1c6285ea0 con 0x55f1c5f68070 2024-09-21T06:08:40.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.007+0000 7f2fd820f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f1c62752d0 con 0x55f1c5f3f1b0 2024-09-21T06:08:40.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.007+0000 7f2fd820f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f1c62790e0 con 0x55f1c5f47390 2024-09-21T06:08:40.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.007+0000 7f2fcbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c5f68070 0x55f1c630f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:40.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.007+0000 7f2fcbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c5f68070 0x55f1c630f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37678/0 (socket says 172.21.15.117:37678) 2024-09-21T06:08:40.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.007+0000 7f2fcbfff640 1 -- 172.21.15.117:0/2616419572 learned_addr learned my addr 172.21.15.117:0/2616419572 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:40.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.007+0000 7f2fcb7fe640 1 --2- 172.21.15.117:0/2616419572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1c5f3f1b0 0x55f1c630f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:40.026 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.007+0000 7f2fcaffd640 1 --2- 172.21.15.117:0/2616419572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1c5f47390 0x55f1c630fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:40.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.008+0000 7f2fcbfff640 1 -- 172.21.15.117:0/2616419572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1c5f47390 msgr2=0x55f1c630fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:40.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.008+0000 7f2fcbfff640 1 --2- 172.21.15.117:0/2616419572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1c5f47390 0x55f1c630fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.008+0000 7f2fcbfff640 1 -- 172.21.15.117:0/2616419572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1c5f3f1b0 msgr2=0x55f1c630f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:40.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.008+0000 7f2fcbfff640 1 --2- 172.21.15.117:0/2616419572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1c5f3f1b0 0x55f1c630f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.008+0000 7f2fcbfff640 1 -- 172.21.15.117:0/2616419572 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1c5fd3650 con 0x55f1c5f68070 2024-09-21T06:08:40.027 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.008+0000 7f2fcaffd640 1 --2- 172.21.15.117:0/2616419572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1c5f47390 0x55f1c630fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:08:40.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.008+0000 7f2fcbfff640 1 --2- 172.21.15.117:0/2616419572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c5f68070 0x55f1c630f2c0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4006010 tx=0x7f2fc4009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8def70970dd62dc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:40.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.008+0000 7f2fc8ff9640 1 --2- 172.21.15.117:0/741771531 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2fb40b6680 0x7f2fb40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:40.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.008+0000 7f2fcb7fe640 1 --2- 172.21.15.117:0/741771531 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2fb40b6680 0x7f2fb40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:40.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.008+0000 7f2fc8ff9640 1 -- 172.21.15.117:0/741771531 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7f2fc00e7090 con 0x7f2fcc103950 2024-09-21T06:08:40.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.008+0000 7f2fb1ffb640 1 -- 172.21.15.117:0/2616419572 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4031d70 con 0x55f1c5f68070 2024-09-21T06:08:40.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.008+0000 7f2fb1ffb640 1 -- 172.21.15.117:0/2616419572 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fc4036070 con 0x55f1c5f68070 2024-09-21T06:08:40.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.008+0000 7f2fb1ffb640 1 -- 172.21.15.117:0/2616419572 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc40042a0 con 0x55f1c5f68070 2024-09-21T06:08:40.028 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.009+0000 7f2fd820f740 1 -- 172.21.15.117:0/2616419572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c5f68070 msgr2=0x55f1c630f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:40.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.009+0000 7f2fcb7fe640 1 --2- 172.21.15.117:0/741771531 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2fb40b6680 0x7f2fb40b8b90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x55f1c630df40 tx=0x7f2fbc00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:40.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.009+0000 7f2fd820f740 1 --2- 172.21.15.117:0/2616419572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c5f68070 0x55f1c630f2c0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4006010 tx=0x7f2fc4009880 comp rx=0 tx=0).stop 2024-09-21T06:08:40.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.009+0000 7f2fd820f740 1 -- 172.21.15.117:0/2616419572 shutdown_connections 2024-09-21T06:08:40.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.009+0000 7f2fd820f740 1 --2- 172.21.15.117:0/2616419572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1c5f47390 0x55f1c630fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.009+0000 7f2fd820f740 1 --2- 172.21.15.117:0/2616419572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1c5f3f1b0 0x55f1c630f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.009+0000 7f2fd820f740 1 --2- 172.21.15.117:0/2616419572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c5f68070 0x55f1c630f2c0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.009+0000 7f2fd820f740 1 -- 172.21.15.117:0/2616419572 >> 172.21.15.117:0/2616419572 conn(0x55f1c5f5c460 msgr2=0x55f1c630d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:40.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.009+0000 7f2fd820f740 1 -- 172.21.15.117:0/2616419572 shutdown_connections 2024-09-21T06:08:40.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.009+0000 7f2fd820f740 1 -- 172.21.15.117:0/2616419572 wait complete. 2024-09-21T06:08:40.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.009+0000 7f2fd820f740 1 -- start start 2024-09-21T06:08:40.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.010+0000 7f2fd820f740 1 Processor -- start 2024-09-21T06:08:40.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.010+0000 7f2fd820f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1c5f3f1b0 0x55f1c631cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:40.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.010+0000 7f2fd820f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1c5f47390 0x55f1c631d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:40.029 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.010+0000 7f2fd820f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c5f68070 0x55f1c6328f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:40.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.010+0000 7f2fd820f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f1c60d6700 con 0x55f1c5f68070 2024-09-21T06:08:40.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.010+0000 7f2fd820f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f1c6210160 con 0x55f1c5f3f1b0 2024-09-21T06:08:40.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.010+0000 7f2fd820f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f1c631fba0 con 0x55f1c5f47390 2024-09-21T06:08:40.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.010+0000 7f2fd820f740 10 client.0 _ll_register_callbacks cb 0x55f1c62299e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:08:40.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.010+0000 7f2fcbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c5f68070 0x55f1c6328f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:40.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.010+0000 7f2fcbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c5f68070 0x55f1c6328f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37692/0 (socket says 172.21.15.117:37692) 2024-09-21T06:08:40.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.010+0000 7f2fcbfff640 1 -- 172.21.15.117:0/830630349 learned_addr learned my addr 172.21.15.117:0/830630349 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:40.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.010+0000 7f2fcb7fe640 1 --2- 172.21.15.117:0/830630349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1c5f3f1b0 0x55f1c631cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:40.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.010+0000 7f2fcaffd640 1 --2- 172.21.15.117:0/830630349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1c5f47390 0x55f1c631d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:40.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.010+0000 7f2fcbfff640 1 -- 172.21.15.117:0/830630349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1c5f47390 msgr2=0x55f1c631d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:40.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.010+0000 7f2fcbfff640 1 --2- 172.21.15.117:0/830630349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1c5f47390 0x55f1c631d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.010+0000 7f2fcbfff640 1 -- 172.21.15.117:0/830630349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1c5f3f1b0 msgr2=0x55f1c631cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:40.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.010+0000 7f2fcbfff640 1 --2- 172.21.15.117:0/830630349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1c5f3f1b0 0x55f1c631cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.030 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.011+0000 7f2fcbfff640 1 -- 172.21.15.117:0/830630349 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1c623dc90 con 0x55f1c5f68070 2024-09-21T06:08:40.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.011+0000 7f2fcb7fe640 1 --2- 172.21.15.117:0/830630349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1c5f3f1b0 0x55f1c631cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:08:40.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.011+0000 7f2fcbfff640 1 --2- 172.21.15.117:0/830630349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c5f68070 0x55f1c6328f00 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4009c80 tx=0x7f2fc402f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c01680e10a77b990 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:40.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.011+0000 7f2faaffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:08:40.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.011+0000 7f2faaffd640 1 -- 172.21.15.117:0/830630349 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4052020 con 0x55f1c5f68070 2024-09-21T06:08:40.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.011+0000 7f2fd820f740 1 -- 172.21.15.117:0/830630349 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fc4036070 con 0x55f1c5f68070 2024-09-21T06:08:40.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.011+0000 7f2faaffd640 1 -- 172.21.15.117:0/830630349 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7f2fc4031b70 con 0x55f1c5f68070 2024-09-21T06:08:40.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.011+0000 7f2faaffd640 1 -- 172.21.15.117:0/830630349 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2fc405d050 con 0x55f1c5f68070 2024-09-21T06:08:40.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.011+0000 7f2fd820f740 10 client.18165 fetch_fsmap learned FSMap version 29 2024-09-21T06:08:40.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.011+0000 7f2fd820f740 1 -- 172.21.15.117:0/830630349 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f1c632b950 con 0x55f1c5f68070 2024-09-21T06:08:40.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.011+0000 7f2faaffd640 1 -- 172.21.15.117:0/830630349 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fc4036070 con 0x55f1c5f68070 2024-09-21T06:08:40.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.012+0000 7f2faaffd640 1 -- 172.21.15.117:0/830630349 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2fc402f9c0 con 0x55f1c5f68070 2024-09-21T06:08:40.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.012+0000 7f2fd820f740 10 client.18165 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:08:40.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.012+0000 7f2fd820f740 10 client.18165 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:08:40.031 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.012+0000 7f2fd820f740 20 client.18165 populate_metadata read hostname 'smithi117' 2024-09-21T06:08:40.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.012+0000 7f2fd820f740 1 --2- 172.21.15.117:0/830630349 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f1c5f269f0 0x55f1c632ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:40.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.012+0000 7f2fd820f740 4 client.18165 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T06:08:40.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.012+0000 7f2fd820f740 1 -- 172.21.15.117:0/830630349 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1c6287df0 con 0x55f1c5f269f0 2024-09-21T06:08:40.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.012+0000 7f2fcb7fe640 1 --2- 172.21.15.117:0/830630349 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f1c5f269f0 0x55f1c632ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:40.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.012+0000 7f2faaffd640 10 client.18165 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:08:40.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.012+0000 7f2fcb7fe640 1 --2- 172.21.15.117:0/830630349 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f1c5f269f0 0x55f1c632ecb0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2cae9d78454a9f9 server_cookie=957292b7d434f607 in_seq=0 out_seq=0 2024-09-21T06:08:40.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.013+0000 7f2faaffd640 1 -- 172.21.15.117:0/830630349 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f1c6287df0 con 0x55f1c5f269f0 2024-09-21T06:08:40.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.013+0000 7f2faaffd640 10 client.18165 handle_command_reply: tid=0 2024-09-21T06:08:40.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.013+0000 7f2fd820f740 1 client.18165 shutdown 2024-09-21T06:08:40.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.013+0000 7f2fd820f740 1 -- 172.21.15.117:0/830630349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c5f68070 msgr2=0x55f1c6328f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:40.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.013+0000 7f2fd820f740 1 --2- 172.21.15.117:0/830630349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c5f68070 0x55f1c6328f00 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4009c80 tx=0x7f2fc402f310 comp rx=0 tx=0).stop 2024-09-21T06:08:40.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.014+0000 7f2fd820f740 1 -- 172.21.15.117:0/830630349 shutdown_connections 2024-09-21T06:08:40.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.014+0000 7f2fd820f740 1 --2- 172.21.15.117:0/830630349 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f1c5f269f0 0x55f1c632ecb0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.014+0000 7f2fd820f740 1 --2- 172.21.15.117:0/830630349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c5f68070 0x55f1c6328f00 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.032 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.014+0000 7f2fd820f740 1 --2- 172.21.15.117:0/830630349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1c5f47390 0x55f1c631d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.014+0000 7f2fd820f740 1 --2- 172.21.15.117:0/830630349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1c5f3f1b0 0x55f1c631cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.014+0000 7f2fd820f740 1 -- 172.21.15.117:0/830630349 >> 172.21.15.117:0/830630349 conn(0x55f1c5f5c460 msgr2=0x55f1c630b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:40.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.014+0000 7f2faaffd640 0 client.18165 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:08:40.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.014+0000 7f2fd820f740 1 -- 172.21.15.117:0/830630349 shutdown_connections 2024-09-21T06:08:40.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.014+0000 7f2fd820f740 1 -- 172.21.15.117:0/830630349 wait complete. 2024-09-21T06:08:40.033 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.014+0000 7f2fd820f740 20 client.18165 trim_cache size 0 max 16384 2024-09-21T06:08:40.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.027+0000 7f2fd820f740 1 Processor -- start 2024-09-21T06:08:40.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.027+0000 7f2fd820f740 1 -- start start 2024-09-21T06:08:40.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.027+0000 7f2fd820f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1c5f5c460 0x55f1c631d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:40.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.027+0000 7f2fd820f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1c5f3f1b0 0x55f1c6319dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:40.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.027+0000 7f2fd820f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c5f47390 0x55f1c631a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:40.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.027+0000 7f2fd820f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc1ad440 con 0x55f1c5f47390 2024-09-21T06:08:40.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.027+0000 7f2fd820f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc1aef80 con 0x55f1c5f3f1b0 2024-09-21T06:08:40.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.027+0000 7f2fd820f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc1adb10 con 0x55f1c5f5c460 2024-09-21T06:08:40.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.027+0000 7f2fcbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c5f47390 0x55f1c631a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:40.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.027+0000 7f2fcbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c5f47390 0x55f1c631a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37708/0 (socket says 172.21.15.117:37708) 2024-09-21T06:08:40.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.027+0000 7f2fcbfff640 1 -- 172.21.15.117:0/2511394839 learned_addr learned my addr 172.21.15.117:0/2511394839 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:40.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.027+0000 7f2fcb7fe640 1 --2- 172.21.15.117:0/2511394839 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1c5f5c460 0x55f1c631d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:40.038 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.027+0000 7f2fcaffd640 1 --2- 172.21.15.117:0/2511394839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1c5f3f1b0 0x55f1c6319dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:40.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.027+0000 7f2fcbfff640 1 -- 172.21.15.117:0/2511394839 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1c5f5c460 msgr2=0x55f1c631d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:40.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.027+0000 7f2fcbfff640 1 --2- 172.21.15.117:0/2511394839 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1c5f5c460 0x55f1c631d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.027+0000 7f2fcbfff640 1 -- 172.21.15.117:0/2511394839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1c5f3f1b0 msgr2=0x55f1c6319dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:40.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.027+0000 7f2fcbfff640 1 --2- 172.21.15.117:0/2511394839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1c5f3f1b0 0x55f1c6319dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.027+0000 7f2fcbfff640 1 -- 172.21.15.117:0/2511394839 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1c63073b0 con 0x55f1c5f47390 2024-09-21T06:08:40.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.028+0000 7f2fcbfff640 1 --2- 172.21.15.117:0/2511394839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c5f47390 0x55f1c631a310 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4005e30 tx=0x7f2fc4005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12cbc54234d861a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:40.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.028+0000 7f2fb17fa640 1 -- 172.21.15.117:0/2511394839 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4052040 con 0x55f1c5f47390 2024-09-21T06:08:40.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.028+0000 7f2fb17fa640 1 -- 172.21.15.117:0/2511394839 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fc4036040 con 0x55f1c5f47390 2024-09-21T06:08:40.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.028+0000 7f2fb17fa640 1 -- 172.21.15.117:0/2511394839 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4031af0 con 0x55f1c5f47390 2024-09-21T06:08:40.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.028+0000 7f2fd820f740 1 -- 172.21.15.117:0/2511394839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c5f47390 msgr2=0x55f1c631a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:40.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.028+0000 7f2fd820f740 1 --2- 172.21.15.117:0/2511394839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c5f47390 0x55f1c631a310 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4005e30 tx=0x7f2fc4005d80 comp rx=0 tx=0).stop 2024-09-21T06:08:40.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.029+0000 7f2fd820f740 1 -- 172.21.15.117:0/2511394839 shutdown_connections 2024-09-21T06:08:40.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.029+0000 7f2fd820f740 1 --2- 172.21.15.117:0/2511394839 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c5f47390 0x55f1c631a310 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.029+0000 7f2fd820f740 1 --2- 172.21.15.117:0/2511394839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1c5f3f1b0 0x55f1c6319dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.039 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.029+0000 7f2fd820f740 1 --2- 172.21.15.117:0/2511394839 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1c5f5c460 0x55f1c631d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.040 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.029+0000 7f2fd820f740 1 -- 172.21.15.117:0/2511394839 >> 172.21.15.117:0/2511394839 conn(0x55f1c6318530 msgr2=0x55f1c631c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:40.040 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.029+0000 7f2fd820f740 1 -- 172.21.15.117:0/2511394839 shutdown_connections 2024-09-21T06:08:40.040 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.029+0000 7f2fd820f740 1 -- 172.21.15.117:0/2511394839 wait complete. 2024-09-21T06:08:40.040 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.029+0000 7f2fd820f740 1 -- start start 2024-09-21T06:08:40.040 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.029+0000 7f2fd820f740 1 Processor -- start 2024-09-21T06:08:40.040 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.029+0000 7f2fd820f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1c5f3f1b0 0x55f1c6332280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:40.040 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.030+0000 7f2fd820f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1c5f47390 0x55f1c632f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:40.040 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.030+0000 7f2fd820f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c5f5c460 0x55f1c632fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:40.040 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.030+0000 7f2fd820f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc1ae8b0 con 0x55f1c5f5c460 2024-09-21T06:08:40.040 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.030+0000 7f2fd820f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc1ae1e0 con 0x55f1c5f47390 2024-09-21T06:08:40.040 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.030+0000 7f2fd820f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc1ab440 con 0x55f1c5f3f1b0 2024-09-21T06:08:40.040 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.030+0000 7f2fd820f740 10 client.0 _ll_register_callbacks cb 0x7f2fbc00b690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T06:08:40.040 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.030+0000 7f2fcbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c5f5c460 0x55f1c632fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:40.040 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.030+0000 7f2fcb7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1c5f3f1b0 0x55f1c6332280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:40.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.030+0000 7f2fcbfff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c5f5c460 0x55f1c632fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37714/0 (socket says 172.21.15.117:37714) 2024-09-21T06:08:40.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.030+0000 7f2fcb7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1c5f3f1b0 0x55f1c6332280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:43304/0 (socket says 172.21.15.117:43304) 2024-09-21T06:08:40.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.030+0000 7f2fcbfff640 1 -- 172.21.15.117:0/2638411458 learned_addr learned my addr 172.21.15.117:0/2638411458 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:40.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.030+0000 7f2fcaffd640 1 --2- 172.21.15.117:0/2638411458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1c5f47390 0x55f1c632f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:40.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.030+0000 7f2fcbfff640 1 -- 172.21.15.117:0/2638411458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1c5f3f1b0 msgr2=0x55f1c6332280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:40.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.030+0000 7f2fcbfff640 1 --2- 172.21.15.117:0/2638411458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1c5f3f1b0 0x55f1c6332280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.030+0000 7f2fcbfff640 1 -- 172.21.15.117:0/2638411458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1c5f47390 msgr2=0x55f1c632f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T06:08:40.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.030+0000 7f2fcbfff640 1 --2- 172.21.15.117:0/2638411458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1c5f47390 0x55f1c632f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.030+0000 7f2fcbfff640 1 -- 172.21.15.117:0/2638411458 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1c63301d0 con 0x55f1c5f5c460 2024-09-21T06:08:40.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.030+0000 7f2fcaffd640 1 --2- 172.21.15.117:0/2638411458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1c5f47390 0x55f1c632f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:08:40.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.030+0000 7f2fcbfff640 1 --2- 172.21.15.117:0/2638411458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c5f5c460 0x55f1c632fc90 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f2fc402f590 tx=0x7f2fc40094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ce9e06f4989f96 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:40.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.030+0000 7f2faaffd640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-21T06:08:40.041 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.030+0000 7f2faaffd640 1 -- 172.21.15.117:0/2638411458 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4052030 con 0x55f1c5f5c460 2024-09-21T06:08:40.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.030+0000 7f2faaffd640 1 -- 172.21.15.117:0/2638411458 <== mon.0 v2:172.21.15.117:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7f2fc4003be0 con 0x55f1c5f5c460 2024-09-21T06:08:40.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.030+0000 7f2fd820f740 1 -- 172.21.15.117:0/2638411458 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fc4036040 con 0x55f1c5f5c460 2024-09-21T06:08:40.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.031+0000 7f2faaffd640 1 -- 172.21.15.117:0/2638411458 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2fc4070050 con 0x55f1c5f5c460 2024-09-21T06:08:40.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.031+0000 7f2fd820f740 10 client.18171 fetch_fsmap learned FSMap version 29 2024-09-21T06:08:40.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.031+0000 7f2fd820f740 1 -- 172.21.15.117:0/2638411458 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f1c6399120 con 0x55f1c5f5c460 2024-09-21T06:08:40.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.031+0000 7f2faaffd640 1 -- 172.21.15.117:0/2638411458 <== mon.0 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fc4036040 con 0x55f1c5f5c460 2024-09-21T06:08:40.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.031+0000 7f2faaffd640 1 -- 172.21.15.117:0/2638411458 <== mon.0 v2:172.21.15.117:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2fc4004160 con 0x55f1c5f5c460 2024-09-21T06:08:40.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.031+0000 7f2fd820f740 10 client.18171 fetch_fsmap finished waiting for FSMap version 29 2024-09-21T06:08:40.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.031+0000 7f2fd820f740 10 client.18171 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T06:08:40.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.031+0000 7f2fd820f740 20 client.18171 populate_metadata read hostname 'smithi117' 2024-09-21T06:08:40.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.031+0000 7f2fd820f740 1 --2- 172.21.15.117:0/2638411458 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f1c5f68070 0x55f1c639b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:40.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.031+0000 7f2fd820f740 4 client.18171 mds_command: new command op to 24638 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T06:08:40.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.031+0000 7f2fd820f740 1 -- 172.21.15.117:0/2638411458 --> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f1c62838a0 con 0x55f1c5f68070 2024-09-21T06:08:40.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.031+0000 7f2fcb7fe640 1 --2- 172.21.15.117:0/2638411458 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f1c5f68070 0x55f1c639b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:40.042 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.032+0000 7f2faaffd640 10 client.18171 ms_handle_connect on v2:172.21.15.138:6838/3230372576 2024-09-21T06:08:40.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.032+0000 7f2fcb7fe640 1 --2- 172.21.15.117:0/2638411458 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f1c5f68070 0x55f1c639b790 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2338d0344130607 server_cookie=8da93c958cee6091 in_seq=0 out_seq=0 2024-09-21T06:08:40.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.032+0000 7f2faaffd640 1 -- 172.21.15.117:0/2638411458 <== mds.0 v2:172.21.15.138:6838/3230372576 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f1c62838a0 con 0x55f1c5f68070 2024-09-21T06:08:40.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.032+0000 7f2faaffd640 10 client.18171 handle_command_reply: tid=0 2024-09-21T06:08:40.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.032+0000 7f2fd820f740 1 client.18171 shutdown 2024-09-21T06:08:40.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.032+0000 7f2fd820f740 1 -- 172.21.15.117:0/2638411458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c5f5c460 msgr2=0x55f1c632fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:40.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.032+0000 7f2fd820f740 1 --2- 172.21.15.117:0/2638411458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c5f5c460 0x55f1c632fc90 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f2fc402f590 tx=0x7f2fc40094a0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.032+0000 7f2fd820f740 1 -- 172.21.15.117:0/2638411458 shutdown_connections 2024-09-21T06:08:40.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.032+0000 7f2fd820f740 1 --2- 172.21.15.117:0/2638411458 >> [v2:172.21.15.138:6838/3230372576,v1:172.21.15.138:6839/3230372576] conn(0x55f1c5f68070 0x55f1c639b790 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.032+0000 7f2fd820f740 1 --2- 172.21.15.117:0/2638411458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f1c5f5c460 0x55f1c632fc90 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.032+0000 7f2fd820f740 1 --2- 172.21.15.117:0/2638411458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f1c5f47390 0x55f1c632f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.032+0000 7f2fd820f740 1 --2- 172.21.15.117:0/2638411458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f1c5f3f1b0 0x55f1c6332280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.032+0000 7f2fd820f740 1 -- 172.21.15.117:0/2638411458 >> 172.21.15.117:0/2638411458 conn(0x55f1c6318530 msgr2=0x55f1c631d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:40.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.032+0000 7f2faaffd640 0 client.18171 ms_handle_reset on v2:172.21.15.138:6838/3230372576 2024-09-21T06:08:40.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.032+0000 7f2fd820f740 1 -- 172.21.15.117:0/2638411458 shutdown_connections 2024-09-21T06:08:40.043 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.032+0000 7f2fd820f740 1 -- 172.21.15.117:0/2638411458 wait complete. 2024-09-21T06:08:40.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.032+0000 7f2fd820f740 20 client.18171 trim_cache size 0 max 16384 2024-09-21T06:08:40.044 INFO:teuthology.orchestra.run.smithi117.stdout:[] 2024-09-21T06:08:40.044 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.033+0000 7f2fd1c9b640 1 -- 172.21.15.117:0/741771531 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2fb40b6680 msgr2=0x7f2fb40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:40.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.033+0000 7f2fd1c9b640 1 --2- 172.21.15.117:0/741771531 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2fb40b6680 0x7f2fb40b8b90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x55f1c630df40 tx=0x7f2fbc00b390 comp rx=0 tx=0).stop 2024-09-21T06:08:40.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.033+0000 7f2fd1c9b640 1 -- 172.21.15.117:0/741771531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fcc103950 msgr2=0x7f2fcc199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:40.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.033+0000 7f2fd1c9b640 1 --2- 172.21.15.117:0/741771531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fcc103950 0x7f2fcc199a70 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0005d00 tx=0x7f2fc000dde0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.033+0000 7f2fd1c9b640 1 -- 172.21.15.117:0/741771531 shutdown_connections 2024-09-21T06:08:40.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.033+0000 7f2fd1c9b640 1 --2- 172.21.15.117:0/741771531 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f2fb40b6680 0x7f2fb40b8b90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.033+0000 7f2fd1c9b640 1 --2- 172.21.15.117:0/741771531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2fcc103950 0x7f2fcc199a70 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.033+0000 7f2fd1c9b640 1 --2- 172.21.15.117:0/741771531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fcc107690 0x7f2fcc19ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.033+0000 7f2fd1c9b640 1 --2- 172.21.15.117:0/741771531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2fcc103000 0x7f2fcc1994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.033+0000 7f2fd1c9b640 1 -- 172.21.15.117:0/741771531 >> 172.21.15.117:0/741771531 conn(0x7f2fcc079ea0 msgr2=0x7f2fcc105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:40.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.033+0000 7f2fd1c9b640 1 -- 172.21.15.117:0/741771531 shutdown_connections 2024-09-21T06:08:40.045 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.033+0000 7f2fd1c9b640 1 -- 172.21.15.117:0/741771531 wait complete. 2024-09-21T06:08:40.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:39 smithi192 ceph-mon[34700]: pgmap v676: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 KiB/s wr, 227 op/s 2024-09-21T06:08:40.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:39 smithi117 ceph-mon[30947]: pgmap v676: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 KiB/s wr, 227 op/s 2024-09-21T06:08:40.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:39 smithi138 ceph-mon[34391]: pgmap v676: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 KiB/s wr, 227 op/s 2024-09-21T06:08:40.855 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/thrasher.py", line 50, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-21T06:08:40.856 INFO:tasks.fwd_scrub:done joining 2024-09-21T06:08:40.856 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-21T06:08:40.872 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T06:08:40.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.984+0000 7f4f0b0ec640 1 -- 172.21.15.117:0/1925512507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f04106b50 msgr2=0x7f4f041105a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:40.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.984+0000 7f4f0b0ec640 1 --2- 172.21.15.117:0/1925512507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f04106b50 0x7f4f041105a0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f4efc009ca0 tx=0x7f4efc02a8c0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.984+0000 7f4f0b0ec640 1 -- 172.21.15.117:0/1925512507 shutdown_connections 2024-09-21T06:08:40.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.984+0000 7f4f0b0ec640 1 --2- 172.21.15.117:0/1925512507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f04106b50 0x7f4f041105a0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.984+0000 7f4f0b0ec640 1 --2- 172.21.15.117:0/1925512507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f04106200 0x7f4f04106610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.984+0000 7f4f0b0ec640 1 --2- 172.21.15.117:0/1925512507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f04101f30 0x7f4f04102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.984+0000 7f4f0b0ec640 1 -- 172.21.15.117:0/1925512507 >> 172.21.15.117:0/1925512507 conn(0x7f4f04079d00 msgr2=0x7f4f0407a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:40.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.984+0000 7f4f0b0ec640 1 -- 172.21.15.117:0/1925512507 shutdown_connections 2024-09-21T06:08:40.986 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.984+0000 7f4f0b0ec640 1 -- 172.21.15.117:0/1925512507 wait complete. 2024-09-21T06:08:40.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.984+0000 7f4f0b0ec640 1 Processor -- start 2024-09-21T06:08:40.987 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.984+0000 7f4f0b0ec640 1 -- start start 2024-09-21T06:08:40.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.985+0000 7f4f0b0ec640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f04101f30 0x7f4f0419a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:40.988 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.985+0000 7f4f0b0ec640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f04106200 0x7f4f0419a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:40.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.985+0000 7f4f0b0ec640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f04106b50 0x7f4f0419fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:40.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.985+0000 7f4f0b0ec640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4f04113a10 con 0x7f4f04106b50 2024-09-21T06:08:40.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.985+0000 7f4f0b0ec640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4f04113890 con 0x7f4f04101f30 2024-09-21T06:08:40.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.985+0000 7f4f0b0ec640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4f04113b90 con 0x7f4f04106200 2024-09-21T06:08:40.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.985+0000 7f4f09689640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f04106b50 0x7f4f0419fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:40.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.985+0000 7f4f09689640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f04106b50 0x7f4f0419fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37726/0 (socket says 172.21.15.117:37726) 2024-09-21T06:08:40.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.985+0000 7f4f09689640 1 -- 172.21.15.117:0/2046689186 learned_addr learned my addr 172.21.15.117:0/2046689186 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:40.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.985+0000 7f4efbfff640 1 --2- 172.21.15.117:0/2046689186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f04106200 0x7f4f0419a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:40.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.985+0000 7f4f08e88640 1 --2- 172.21.15.117:0/2046689186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f04101f30 0x7f4f0419a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:40.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.985+0000 7f4f09689640 1 -- 172.21.15.117:0/2046689186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f04106200 msgr2=0x7f4f0419a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:40.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.985+0000 7f4f09689640 1 --2- 172.21.15.117:0/2046689186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f04106200 0x7f4f0419a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.985+0000 7f4f09689640 1 -- 172.21.15.117:0/2046689186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f04101f30 msgr2=0x7f4f0419a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:40.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.985+0000 7f4f09689640 1 --2- 172.21.15.117:0/2046689186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f04101f30 0x7f4f0419a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:40.989 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.985+0000 7f4f09689640 1 -- 172.21.15.117:0/2046689186 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f041a02e0 con 0x7f4f04106b50 2024-09-21T06:08:40.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.985+0000 7f4f09689640 1 --2- 172.21.15.117:0/2046689186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f04106b50 0x7f4f0419fbe0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f4efc005cc0 tx=0x7f4efc02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:40.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.985+0000 7f4ef9ffb640 1 -- 172.21.15.117:0/2046689186 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4efc003bf0 con 0x7f4f04106b50 2024-09-21T06:08:40.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.985+0000 7f4ef9ffb640 1 -- 172.21.15.117:0/2046689186 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4efc003d90 con 0x7f4f04106b50 2024-09-21T06:08:40.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.985+0000 7f4f0b0ec640 1 -- 172.21.15.117:0/2046689186 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f041a0570 con 0x7f4f04106b50 2024-09-21T06:08:40.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.985+0000 7f4ef9ffb640 1 -- 172.21.15.117:0/2046689186 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4efc02fbc0 con 0x7f4f04106b50 2024-09-21T06:08:40.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.986+0000 7f4f0b0ec640 1 -- 172.21.15.117:0/2046689186 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f041a6d60 con 0x7f4f04106b50 2024-09-21T06:08:40.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.987+0000 7f4f0b0ec640 1 -- 172.21.15.117:0/2046689186 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f04102340 con 0x7f4f04106b50 2024-09-21T06:08:40.990 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.988+0000 7f4ef9ffb640 1 -- 172.21.15.117:0/2046689186 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f4efc02fd60 con 0x7f4f04106b50 2024-09-21T06:08:40.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.989+0000 7f4ef9ffb640 1 --2- 172.21.15.117:0/2046689186 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4eec0b6680 0x7f4eec0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:40.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.989+0000 7f4f08e88640 1 --2- 172.21.15.117:0/2046689186 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4eec0b6680 0x7f4eec0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:40.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.990+0000 7f4ef9ffb640 1 -- 172.21.15.117:0/2046689186 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7f4efc108da0 con 0x7f4f04106b50 2024-09-21T06:08:40.991 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.990+0000 7f4f08e88640 1 --2- 172.21.15.117:0/2046689186 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4eec0b6680 0x7f4eec0b8b90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4f00005fd0 tx=0x7f4f00005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:40.995 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:40.993+0000 7f4ef9ffb640 1 -- 172.21.15.117:0/2046689186 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4efc0cf340 con 0x7f4f04106b50 2024-09-21T06:08:41.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.175+0000 7f4f0b0ec640 1 -- 172.21.15.117:0/2046689186 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4f0402e640 con 0x7f4f04106b50 2024-09-21T06:08:41.178 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.176+0000 7f4ef9ffb640 1 -- 172.21.15.117:0/2046689186 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+23156 (secure 0 0 0) 0x7f4efc0d3720 con 0x7f4f04106b50 2024-09-21T06:08:41.178 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T06:08:41.179 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":85,"fsid":"ac8581a2-77dc-11ef-baf5-efdc52797490","created":"2024-09-21T05:46:47.021846+0000","modified":"2024-09-21T06:08:17.511443+0000","last_up_change":"2024-09-21T05:54:21.572849+0000","last_in_change":"2024-09-21T05:53:58.892597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:50:32.717389+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:55:10.604159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T05:55:12.581258+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":16,"snap_epoch":84,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"68315502-fa6c-4a6e-a2d4-a7dde8071c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6803","nonce":3761478035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6805","nonce":3761478035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6809","nonce":3761478035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6807","nonce":3761478035}]},"public_addr":"172.21.15.117:6803/3761478035","cluster_addr":"172.21.15.117:6805/3761478035","heartbeat_back_addr":"172.21.15.117:6809/3761478035","heartbeat_front_addr":"172.21.15.117:6807/3761478035","state":["exists","up"]},{"osd":1,"uuid":"d8910a08-4ef3-49f2-a4db-e3588d85d12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6801","nonce":1737365588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6803","nonce":1737365588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6807","nonce":1737365588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6805","nonce":1737365588}]},"public_addr":"172.21.15.138:6801/1737365588","cluster_addr":"172.21.15.138:6803/1737365588","heartbeat_back_addr":"172.21.15.138:6807/1737365588","heartbeat_front_addr":"172.21.15.138:6805/1737365588","state":["exists","up"]},{"osd":2,"uuid":"94972378-a11f-4129-9e40-ad3fdec1d189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6801","nonce":3002534992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6803","nonce":3002534992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6807","nonce":3002534992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6805","nonce":3002534992}]},"public_addr":"172.21.15.192:6801/3002534992","cluster_addr":"172.21.15.192:6803/3002534992","heartbeat_back_addr":"172.21.15.192:6807/3002534992","heartbeat_front_addr":"172.21.15.192:6805/3002534992","state":["exists","up"]},{"osd":3,"uuid":"d4168c43-f0de-4445-945c-92d1cf270c33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6811","nonce":1222400293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6813","nonce":1222400293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6817","nonce":1222400293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6815","nonce":1222400293}]},"public_addr":"172.21.15.117:6811/1222400293","cluster_addr":"172.21.15.117:6813/1222400293","heartbeat_back_addr":"172.21.15.117:6817/1222400293","heartbeat_front_addr":"172.21.15.117:6815/1222400293","state":["exists","up"]},{"osd":4,"uuid":"28f1d7c0-8392-4a33-85c0-0de611cdd4b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6809","nonce":4294892870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6811","nonce":4294892870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6815","nonce":4294892870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6813","nonce":4294892870}]},"public_addr":"172.21.15.138:6809/4294892870","cluster_addr":"172.21.15.138:6811/4294892870","heartbeat_back_addr":"172.21.15.138:6815/4294892870","heartbeat_front_addr":"172.21.15.138:6813/4294892870","state":["exists","up"]},{"osd":5,"uuid":"ca902407-45e8-4bc5-bdbd-a729c1de8322","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6809","nonce":1776840019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6811","nonce":1776840019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6815","nonce":1776840019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6813","nonce":1776840019}]},"public_addr":"172.21.15.192:6809/1776840019","cluster_addr":"172.21.15.192:6811/1776840019","heartbeat_back_addr":"172.21.15.192:6815/1776840019","heartbeat_front_addr":"172.21.15.192:6813/1776840019","state":["exists","up"]},{"osd":6,"uuid":"6e507a91-ed09-4a33-bf0e-c4b593b3c5aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6819","nonce":1093341590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6821","nonce":1093341590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6825","nonce":1093341590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6823","nonce":1093341590}]},"public_addr":"172.21.15.117:6819/1093341590","cluster_addr":"172.21.15.117:6821/1093341590","heartbeat_back_addr":"172.21.15.117:6825/1093341590","heartbeat_front_addr":"172.21.15.117:6823/1093341590","state":["exists","up"]},{"osd":7,"uuid":"36a47dd7-67f7-4ebe-8759-3a8bb6bd2ea3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6817","nonce":1448179714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6819","nonce":1448179714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6823","nonce":1448179714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6821","nonce":1448179714}]},"public_addr":"172.21.15.138:6817/1448179714","cluster_addr":"172.21.15.138:6819/1448179714","heartbeat_back_addr":"172.21.15.138:6823/1448179714","heartbeat_front_addr":"172.21.15.138:6821/1448179714","state":["exists","up"]},{"osd":8,"uuid":"118e0b5d-2619-4512-94c5-8d07a0d3a28f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6817","nonce":3960902885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6819","nonce":3960902885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6823","nonce":3960902885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6821","nonce":3960902885}]},"public_addr":"172.21.15.192:6817/3960902885","cluster_addr":"172.21.15.192:6819/3960902885","heartbeat_back_addr":"172.21.15.192:6823/3960902885","heartbeat_front_addr":"172.21.15.192:6821/3960902885","state":["exists","up"]},{"osd":9,"uuid":"47543bc6-fa0b-4e72-9742-97dcda0b7f8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6827","nonce":3453855195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6829","nonce":3453855195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6833","nonce":3453855195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6831","nonce":3453855195}]},"public_addr":"172.21.15.117:6827/3453855195","cluster_addr":"172.21.15.117:6829/3453855195","heartbeat_back_addr":"172.21.15.117:6833/3453855195","heartbeat_front_addr":"172.21.15.117:6831/3453855195","state":["exists","up"]},{"osd":10,"uuid":"5e6708b4-7a85-4fa5-a03c-27d99f7d749f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6825","nonce":903392451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6827","nonce":903392451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6831","nonce":903392451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6829","nonce":903392451}]},"public_addr":"172.21.15.138:6825/903392451","cluster_addr":"172.21.15.138:6827/903392451","heartbeat_back_addr":"172.21.15.138:6831/903392451","heartbeat_front_addr":"172.21.15.138:6829/903392451","state":["exists","up"]},{"osd":11,"uuid":"24bc2ef7-e2b4-43c6-9e6c-43ae02390415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6825","nonce":899090479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6827","nonce":899090479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6831","nonce":899090479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6829","nonce":899090479}]},"public_addr":"172.21.15.192:6825/899090479","cluster_addr":"172.21.15.192:6827/899090479","heartbeat_back_addr":"172.21.15.192:6831/899090479","heartbeat_front_addr":"172.21.15.192:6829/899090479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:49:40.742217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:05.963655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:30.164495+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:56.598899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:51:21.940331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:51:47.777445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:52:12.869007+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:52:39.454054+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:03.569226+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:29.760863+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:55.788604+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:54:20.510184+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:0/2989394419":"2024-09-22T05:48:32.406654+0000","172.21.15.117:6801/1702676172":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/1766406059":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6800/2337036514":"2024-09-22T05:47:09.112435+0000","172.21.15.117:0/985798343":"2024-09-22T05:47:09.112435+0000","172.21.15.117:0/1083273222":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/473284565":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6801/2337036514":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6800/816635165":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/2642940197":"2024-09-22T05:48:32.406654+0000","172.21.15.117:6801/816635165":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/2225537433":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/3651584169":"2024-09-22T05:47:33.789146+0000","172.21.15.117:6800/1702676172":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/1284403882":"2024-09-22T05:47:33.789146+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":16,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:08:41.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.179+0000 7f4f0b0ec640 1 -- 172.21.15.117:0/2046689186 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4eec0b6680 msgr2=0x7f4eec0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:41.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.179+0000 7f4f0b0ec640 1 --2- 172.21.15.117:0/2046689186 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4eec0b6680 0x7f4eec0b8b90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4f00005fd0 tx=0x7f4f00005850 comp rx=0 tx=0).stop 2024-09-21T06:08:41.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.179+0000 7f4f0b0ec640 1 -- 172.21.15.117:0/2046689186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f04106b50 msgr2=0x7f4f0419fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:41.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.179+0000 7f4f0b0ec640 1 --2- 172.21.15.117:0/2046689186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f04106b50 0x7f4f0419fbe0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f4efc005cc0 tx=0x7f4efc02f380 comp rx=0 tx=0).stop 2024-09-21T06:08:41.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.179+0000 7f4f0b0ec640 1 -- 172.21.15.117:0/2046689186 shutdown_connections 2024-09-21T06:08:41.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.179+0000 7f4f0b0ec640 1 --2- 172.21.15.117:0/2046689186 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4eec0b6680 0x7f4eec0b8b90 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.179+0000 7f4f0b0ec640 1 --2- 172.21.15.117:0/2046689186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f04106b50 0x7f4f0419fbe0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.181 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.179+0000 7f4f0b0ec640 1 --2- 172.21.15.117:0/2046689186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f04106200 0x7f4f0419a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.179+0000 7f4f0b0ec640 1 --2- 172.21.15.117:0/2046689186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f04101f30 0x7f4f0419a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.179+0000 7f4f0b0ec640 1 -- 172.21.15.117:0/2046689186 >> 172.21.15.117:0/2046689186 conn(0x7f4f04079d00 msgr2=0x7f4f041095f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:41.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.179+0000 7f4f0b0ec640 1 -- 172.21.15.117:0/2046689186 shutdown_connections 2024-09-21T06:08:41.182 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.179+0000 7f4f0b0ec640 1 -- 172.21.15.117:0/2046689186 wait complete. 2024-09-21T06:08:41.193 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T05:50:32.717389+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T05:55:10.604159+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T05:55:12.581258+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 16, 'snap_epoch': 84, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T06:08:41.193 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T06:08:41.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.306+0000 7fa8fac8b640 1 -- 172.21.15.117:0/4290087433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8f4102990 msgr2=0x7fa8f4102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:41.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.306+0000 7fa8fac8b640 1 --2- 172.21.15.117:0/4290087433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8f4102990 0x7fa8f4102da0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8009900 tx=0x7fa8e802a410 comp rx=0 tx=0).stop 2024-09-21T06:08:41.310 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.306+0000 7fa8fac8b640 1 -- 172.21.15.117:0/4290087433 shutdown_connections 2024-09-21T06:08:41.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.306+0000 7fa8fac8b640 1 --2- 172.21.15.117:0/4290087433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8f4108c10 0x7fa8f410f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.306+0000 7fa8fac8b640 1 --2- 172.21.15.117:0/4290087433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8f41082c0 0x7fa8f41086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.306+0000 7fa8fac8b640 1 --2- 172.21.15.117:0/4290087433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8f4102990 0x7fa8f4102da0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.306+0000 7fa8fac8b640 1 -- 172.21.15.117:0/4290087433 >> 172.21.15.117:0/4290087433 conn(0x7fa8f40fe3b0 msgr2=0x7fa8f4100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:41.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.306+0000 7fa8fac8b640 1 -- 172.21.15.117:0/4290087433 shutdown_connections 2024-09-21T06:08:41.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.306+0000 7fa8fac8b640 1 -- 172.21.15.117:0/4290087433 wait complete. 2024-09-21T06:08:41.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.307+0000 7fa8fac8b640 1 Processor -- start 2024-09-21T06:08:41.311 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.307+0000 7fa8fac8b640 1 -- start start 2024-09-21T06:08:41.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.307+0000 7fa8fac8b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8f4102990 0x7fa8f419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:41.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.307+0000 7fa8fac8b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8f41082c0 0x7fa8f419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:41.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.307+0000 7fa8fac8b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8f4108c10 0x7fa8f419fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:41.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.307+0000 7fa8fac8b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f41128d0 con 0x7fa8f41082c0 2024-09-21T06:08:41.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.307+0000 7fa8fac8b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f4112750 con 0x7fa8f4108c10 2024-09-21T06:08:41.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.307+0000 7fa8fac8b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f4112a50 con 0x7fa8f4102990 2024-09-21T06:08:41.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.307+0000 7fa8f3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8f41082c0 0x7fa8f419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:41.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.307+0000 7fa8f8a27640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8f4102990 0x7fa8f419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:41.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.307+0000 7fa8f3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8f41082c0 0x7fa8f419a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37750/0 (socket says 172.21.15.117:37750) 2024-09-21T06:08:41.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.307+0000 7fa8f3fff640 1 -- 172.21.15.117:0/3699573581 learned_addr learned my addr 172.21.15.117:0/3699573581 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:41.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.307+0000 7fa8f8a27640 1 --2- 172.21.15.117:0/3699573581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8f4102990 0x7fa8f419a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.117:43342/0 (socket says 172.21.15.117:43342) 2024-09-21T06:08:41.312 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.307+0000 7fa8f9228640 1 --2- 172.21.15.117:0/3699573581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8f4108c10 0x7fa8f419fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:41.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.308+0000 7fa8f3fff640 1 -- 172.21.15.117:0/3699573581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8f4102990 msgr2=0x7fa8f419a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:41.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.308+0000 7fa8f3fff640 1 --2- 172.21.15.117:0/3699573581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8f4102990 0x7fa8f419a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.308+0000 7fa8f3fff640 1 -- 172.21.15.117:0/3699573581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8f4108c10 msgr2=0x7fa8f419fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:41.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.308+0000 7fa8f3fff640 1 --2- 172.21.15.117:0/3699573581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8f4108c10 0x7fa8f419fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.308+0000 7fa8f3fff640 1 -- 172.21.15.117:0/3699573581 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f41a0350 con 0x7fa8f41082c0 2024-09-21T06:08:41.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.308+0000 7fa8f3fff640 1 --2- 172.21.15.117:0/3699573581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8f41082c0 0x7fa8f419a9c0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fa8e000e9f0 tx=0x7fa8e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:41.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.308+0000 7fa8f1ffb640 1 -- 172.21.15.117:0/3699573581 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e000ccc0 con 0x7fa8f41082c0 2024-09-21T06:08:41.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.308+0000 7fa8fac8b640 1 -- 172.21.15.117:0/3699573581 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8f41a0640 con 0x7fa8f41082c0 2024-09-21T06:08:41.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.308+0000 7fa8f1ffb640 1 -- 172.21.15.117:0/3699573581 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8e00108e0 con 0x7fa8f41082c0 2024-09-21T06:08:41.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.308+0000 7fa8f1ffb640 1 -- 172.21.15.117:0/3699573581 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0018860 con 0x7fa8f41082c0 2024-09-21T06:08:41.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.308+0000 7fa8fac8b640 1 -- 172.21.15.117:0/3699573581 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8f41a6dd0 con 0x7fa8f41082c0 2024-09-21T06:08:41.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.309+0000 7fa8fac8b640 1 -- 172.21.15.117:0/3699573581 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8f4102da0 con 0x7fa8f41082c0 2024-09-21T06:08:41.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.311+0000 7fa8f1ffb640 1 -- 172.21.15.117:0/3699573581 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fa8e0010430 con 0x7fa8f41082c0 2024-09-21T06:08:41.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.312+0000 7fa8f1ffb640 1 --2- 172.21.15.117:0/3699573581 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa8e40b6680 0x7fa8e40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:41.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.312+0000 7fa8f8a27640 1 --2- 172.21.15.117:0/3699573581 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa8e40b6680 0x7fa8e40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:41.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.312+0000 7fa8f1ffb640 1 -- 172.21.15.117:0/3699573581 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7fa8e00e7e40 con 0x7fa8f41082c0 2024-09-21T06:08:41.315 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.313+0000 7fa8f8a27640 1 --2- 172.21.15.117:0/3699573581 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa8e40b6680 0x7fa8e40b8b90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8031440 tx=0x7fa8e8009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:41.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.315+0000 7fa8f1ffb640 1 -- 172.21.15.117:0/3699573581 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa8e00b27c0 con 0x7fa8f41082c0 2024-09-21T06:08:41.516 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.515+0000 7fa8fac8b640 1 -- 172.21.15.117:0/3699573581 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa8f41a70b0 con 0x7fa8f41082c0 2024-09-21T06:08:41.517 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.515+0000 7fa8f1ffb640 1 -- 172.21.15.117:0/3699573581 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) ==== 93+0+10 (secure 0 0 0) 0x7fa8e00b7700 con 0x7fa8f41082c0 2024-09-21T06:08:41.517 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 1 2024-09-21T06:08:41.519 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.517+0000 7fa8fac8b640 1 -- 172.21.15.117:0/3699573581 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa8e40b6680 msgr2=0x7fa8e40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:41.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.517+0000 7fa8fac8b640 1 --2- 172.21.15.117:0/3699573581 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa8e40b6680 0x7fa8e40b8b90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8031440 tx=0x7fa8e8009560 comp rx=0 tx=0).stop 2024-09-21T06:08:41.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.517+0000 7fa8fac8b640 1 -- 172.21.15.117:0/3699573581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8f41082c0 msgr2=0x7fa8f419a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:41.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.517+0000 7fa8fac8b640 1 --2- 172.21.15.117:0/3699573581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8f41082c0 0x7fa8f419a9c0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fa8e000e9f0 tx=0x7fa8e000eeb0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.518+0000 7fa8fac8b640 1 -- 172.21.15.117:0/3699573581 shutdown_connections 2024-09-21T06:08:41.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.518+0000 7fa8fac8b640 1 --2- 172.21.15.117:0/3699573581 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fa8e40b6680 0x7fa8e40b8b90 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.518+0000 7fa8fac8b640 1 --2- 172.21.15.117:0/3699573581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8f41082c0 0x7fa8f419a9c0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.518+0000 7fa8fac8b640 1 --2- 172.21.15.117:0/3699573581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8f4108c10 0x7fa8f419fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.518+0000 7fa8fac8b640 1 --2- 172.21.15.117:0/3699573581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8f4102990 0x7fa8f419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.518+0000 7fa8fac8b640 1 -- 172.21.15.117:0/3699573581 >> 172.21.15.117:0/3699573581 conn(0x7fa8f40fe3b0 msgr2=0x7fa8f406aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:41.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.518+0000 7fa8fac8b640 1 -- 172.21.15.117:0/3699573581 shutdown_connections 2024-09-21T06:08:41.520 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.518+0000 7fa8fac8b640 1 -- 172.21.15.117:0/3699573581 wait complete. 2024-09-21T06:08:41.531 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T06:08:41.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.643+0000 7f188229b640 1 -- 172.21.15.117:0/6229793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f187c108c70 msgr2=0x7f187c10f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:41.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.643+0000 7f188229b640 1 --2- 172.21.15.117:0/6229793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f187c108c70 0x7f187c10f4a0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f1874009ca0 tx=0x7f187402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.643+0000 7f188229b640 1 -- 172.21.15.117:0/6229793 shutdown_connections 2024-09-21T06:08:41.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.643+0000 7f188229b640 1 --2- 172.21.15.117:0/6229793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f187c108c70 0x7f187c10f4a0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.643+0000 7f188229b640 1 --2- 172.21.15.117:0/6229793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f187c108320 0x7f187c108730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.643+0000 7f188229b640 1 --2- 172.21.15.117:0/6229793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f187c1029f0 0x7f187c102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.645 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.643+0000 7f188229b640 1 -- 172.21.15.117:0/6229793 >> 172.21.15.117:0/6229793 conn(0x7f187c0fe3f0 msgr2=0x7f187c100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:41.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.643+0000 7f188229b640 1 -- 172.21.15.117:0/6229793 shutdown_connections 2024-09-21T06:08:41.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.643+0000 7f188229b640 1 -- 172.21.15.117:0/6229793 wait complete. 2024-09-21T06:08:41.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.643+0000 7f188229b640 1 Processor -- start 2024-09-21T06:08:41.646 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.643+0000 7f188229b640 1 -- start start 2024-09-21T06:08:41.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.644+0000 7f188229b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f187c1029f0 0x7f187c19c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:41.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.644+0000 7f188229b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f187c108320 0x7f187c19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:41.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.644+0000 7f188229b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f187c108c70 0x7f187c1a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:41.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.644+0000 7f188229b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f187c113a50 con 0x7f187c108c70 2024-09-21T06:08:41.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.644+0000 7f188229b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f187c1138d0 con 0x7f187c108320 2024-09-21T06:08:41.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.644+0000 7f188229b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f187c113bd0 con 0x7f187c1029f0 2024-09-21T06:08:41.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.644+0000 7f1880838640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f187c108c70 0x7f187c1a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:41.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.644+0000 7f1880838640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f187c108c70 0x7f187c1a1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37766/0 (socket says 172.21.15.117:37766) 2024-09-21T06:08:41.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.644+0000 7f1880838640 1 -- 172.21.15.117:0/2305307458 learned_addr learned my addr 172.21.15.117:0/2305307458 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:41.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.644+0000 7f187b7fe640 1 --2- 172.21.15.117:0/2305307458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f187c108320 0x7f187c19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:41.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.644+0000 7f187bfff640 1 --2- 172.21.15.117:0/2305307458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f187c1029f0 0x7f187c19c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:41.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.644+0000 7f187b7fe640 1 -- 172.21.15.117:0/2305307458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f187c1029f0 msgr2=0x7f187c19c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:41.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.644+0000 7f187b7fe640 1 --2- 172.21.15.117:0/2305307458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f187c1029f0 0x7f187c19c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.644+0000 7f187b7fe640 1 -- 172.21.15.117:0/2305307458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f187c108c70 msgr2=0x7f187c1a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:41.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.644+0000 7f187b7fe640 1 --2- 172.21.15.117:0/2305307458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f187c108c70 0x7f187c1a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.644+0000 7f187b7fe640 1 -- 172.21.15.117:0/2305307458 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187c1a2560 con 0x7f187c108320 2024-09-21T06:08:41.648 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.644+0000 7f1880838640 1 --2- 172.21.15.117:0/2305307458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f187c108c70 0x7f187c1a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:08:41.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.644+0000 7f187b7fe640 1 --2- 172.21.15.117:0/2305307458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f187c108320 0x7f187c19cbd0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f187c06a240 tx=0x7f187000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:41.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.645+0000 7f18797fa640 1 -- 172.21.15.117:0/2305307458 <== mon.2 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187000cc40 con 0x7f187c108320 2024-09-21T06:08:41.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.645+0000 7f188229b640 1 -- 172.21.15.117:0/2305307458 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f187c1a2850 con 0x7f187c108320 2024-09-21T06:08:41.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.645+0000 7f18797fa640 1 -- 172.21.15.117:0/2305307458 <== mon.2 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f187000cde0 con 0x7f187c108320 2024-09-21T06:08:41.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.645+0000 7f18797fa640 1 -- 172.21.15.117:0/2305307458 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1870018760 con 0x7f187c108320 2024-09-21T06:08:41.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.645+0000 7f188229b640 1 -- 172.21.15.117:0/2305307458 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f187c1a8fe0 con 0x7f187c108320 2024-09-21T06:08:41.649 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.646+0000 7f188229b640 1 -- 172.21.15.117:0/2305307458 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f187c102e00 con 0x7f187c108320 2024-09-21T06:08:41.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.648+0000 7f18797fa640 1 -- 172.21.15.117:0/2305307458 <== mon.2 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f1870010a90 con 0x7f187c108320 2024-09-21T06:08:41.650 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.649+0000 7f18797fa640 1 --2- 172.21.15.117:0/2305307458 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f18640b6680 0x7f18640b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:41.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.649+0000 7f18797fa640 1 -- 172.21.15.117:0/2305307458 <== mon.2 v2:172.21.15.138:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7f18700e7b70 con 0x7f187c108320 2024-09-21T06:08:41.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.649+0000 7f187bfff640 1 --2- 172.21.15.117:0/2305307458 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f18640b6680 0x7f18640b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:41.651 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.650+0000 7f187bfff640 1 --2- 172.21.15.117:0/2305307458 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f18640b6680 0x7f18640b8b90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f186c00bbe0 tx=0x7f186c00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:41.653 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.651+0000 7f18797fa640 1 -- 172.21.15.117:0/2305307458 <== mon.2 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f18700b24f0 con 0x7f187c108320 2024-09-21T06:08:41.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.837+0000 7f188229b640 1 -- 172.21.15.117:0/2305307458 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f187c1a91b0 con 0x7f187c108320 2024-09-21T06:08:41.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.837+0000 7f18797fa640 1 -- 172.21.15.117:0/2305307458 <== mon.2 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v85) ==== 104+0+11 (secure 0 0 0) 0x7f18700b7430 con 0x7f187c108320 2024-09-21T06:08:41.839 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 32 2024-09-21T06:08:41.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.840+0000 7f188229b640 1 -- 172.21.15.117:0/2305307458 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f18640b6680 msgr2=0x7f18640b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:41.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.840+0000 7f188229b640 1 --2- 172.21.15.117:0/2305307458 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f18640b6680 0x7f18640b8b90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f186c00bbe0 tx=0x7f186c00b390 comp rx=0 tx=0).stop 2024-09-21T06:08:41.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.840+0000 7f188229b640 1 -- 172.21.15.117:0/2305307458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f187c108320 msgr2=0x7f187c19cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:41.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.840+0000 7f188229b640 1 --2- 172.21.15.117:0/2305307458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f187c108320 0x7f187c19cbd0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f187c06a240 tx=0x7f187000ee50 comp rx=0 tx=0).stop 2024-09-21T06:08:41.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.840+0000 7f188229b640 1 -- 172.21.15.117:0/2305307458 shutdown_connections 2024-09-21T06:08:41.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.840+0000 7f188229b640 1 --2- 172.21.15.117:0/2305307458 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f18640b6680 0x7f18640b8b90 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.840+0000 7f188229b640 1 --2- 172.21.15.117:0/2305307458 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f187c108c70 0x7f187c1a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.840+0000 7f188229b640 1 --2- 172.21.15.117:0/2305307458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f187c108320 0x7f187c19cbd0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.840+0000 7f188229b640 1 --2- 172.21.15.117:0/2305307458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f187c1029f0 0x7f187c19c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.840+0000 7f188229b640 1 -- 172.21.15.117:0/2305307458 >> 172.21.15.117:0/2305307458 conn(0x7f187c0fe3f0 msgr2=0x7f187c105850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:41.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.840+0000 7f188229b640 1 -- 172.21.15.117:0/2305307458 shutdown_connections 2024-09-21T06:08:41.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.840+0000 7f188229b640 1 -- 172.21.15.117:0/2305307458 wait complete. 2024-09-21T06:08:41.853 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T06:08:41.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.968+0000 7fbd7853a640 1 -- 172.21.15.117:0/650295870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd701051c0 msgr2=0x7fbd7010d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:41.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.968+0000 7fbd7853a640 1 --2- 172.21.15.117:0/650295870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd701051c0 0x7fbd7010d800 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fbd60009900 tx=0x7fbd6002a410 comp rx=0 tx=0).stop 2024-09-21T06:08:41.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.968+0000 7fbd7853a640 1 -- 172.21.15.117:0/650295870 shutdown_connections 2024-09-21T06:08:41.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.968+0000 7fbd7853a640 1 --2- 172.21.15.117:0/650295870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd701051c0 0x7fbd7010d800 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.968+0000 7fbd7853a640 1 --2- 172.21.15.117:0/650295870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd7010dd40 0x7fbd70110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.968+0000 7fbd7853a640 1 --2- 172.21.15.117:0/650295870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd70104790 0x7fbd70104ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.968+0000 7fbd7853a640 1 -- 172.21.15.117:0/650295870 >> 172.21.15.117:0/650295870 conn(0x7fbd700fe3c0 msgr2=0x7fbd70100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:41.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.968+0000 7fbd7853a640 1 -- 172.21.15.117:0/650295870 shutdown_connections 2024-09-21T06:08:41.971 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.968+0000 7fbd7853a640 1 -- 172.21.15.117:0/650295870 wait complete. 2024-09-21T06:08:41.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.969+0000 7fbd7853a640 1 Processor -- start 2024-09-21T06:08:41.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.969+0000 7fbd7853a640 1 -- start start 2024-09-21T06:08:41.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.969+0000 7fbd7853a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd70104790 0x7fbd7019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:41.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.969+0000 7fbd7853a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd701051c0 0x7fbd7019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:41.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.969+0000 7fbd7853a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd7010dd40 0x7fbd7019fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:41.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.969+0000 7fbd7853a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbd701139c0 con 0x7fbd701051c0 2024-09-21T06:08:41.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.969+0000 7fbd7853a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbd70113840 con 0x7fbd70104790 2024-09-21T06:08:41.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.969+0000 7fbd7853a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbd70113b40 con 0x7fbd7010dd40 2024-09-21T06:08:41.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.969+0000 7fbd75ad5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd701051c0 0x7fbd7019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:41.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.969+0000 7fbd75ad5640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd701051c0 0x7fbd7019a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37786/0 (socket says 172.21.15.117:37786) 2024-09-21T06:08:41.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.969+0000 7fbd762d6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd70104790 0x7fbd7019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:41.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.969+0000 7fbd75ad5640 1 -- 172.21.15.117:0/1851643720 learned_addr learned my addr 172.21.15.117:0/1851643720 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:41.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.969+0000 7fbd76ad7640 1 --2- 172.21.15.117:0/1851643720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd7010dd40 0x7fbd7019fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:41.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.969+0000 7fbd75ad5640 1 -- 172.21.15.117:0/1851643720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd7010dd40 msgr2=0x7fbd7019fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:41.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.969+0000 7fbd75ad5640 1 --2- 172.21.15.117:0/1851643720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd7010dd40 0x7fbd7019fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.969+0000 7fbd75ad5640 1 -- 172.21.15.117:0/1851643720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd70104790 msgr2=0x7fbd7019a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:41.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.969+0000 7fbd75ad5640 1 --2- 172.21.15.117:0/1851643720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd70104790 0x7fbd7019a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:41.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.969+0000 7fbd75ad5640 1 -- 172.21.15.117:0/1851643720 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd701a0350 con 0x7fbd701051c0 2024-09-21T06:08:41.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.970+0000 7fbd75ad5640 1 --2- 172.21.15.117:0/1851643720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd701051c0 0x7fbd7019a9c0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fbd6002a930 tx=0x7fbd60031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:41.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.970+0000 7fbd677fe640 1 -- 172.21.15.117:0/1851643720 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6003f030 con 0x7fbd701051c0 2024-09-21T06:08:41.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.970+0000 7fbd677fe640 1 -- 172.21.15.117:0/1851643720 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd60004490 con 0x7fbd701051c0 2024-09-21T06:08:41.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.970+0000 7fbd677fe640 1 -- 172.21.15.117:0/1851643720 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd60035070 con 0x7fbd701051c0 2024-09-21T06:08:41.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.970+0000 7fbd7853a640 1 -- 172.21.15.117:0/1851643720 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd701a05e0 con 0x7fbd701051c0 2024-09-21T06:08:41.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.970+0000 7fbd7853a640 1 -- 172.21.15.117:0/1851643720 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd701a6dd0 con 0x7fbd701051c0 2024-09-21T06:08:41.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.971+0000 7fbd7853a640 1 -- 172.21.15.117:0/1851643720 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd7006bf20 con 0x7fbd701051c0 2024-09-21T06:08:41.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.972+0000 7fbd677fe640 1 -- 172.21.15.117:0/1851643720 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fbd60042030 con 0x7fbd701051c0 2024-09-21T06:08:41.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.973+0000 7fbd677fe640 1 --2- 172.21.15.117:0/1851643720 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbd5c0b6700 0x7fbd5c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:41.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.973+0000 7fbd762d6640 1 --2- 172.21.15.117:0/1851643720 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbd5c0b6700 0x7fbd5c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:41.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.973+0000 7fbd677fe640 1 -- 172.21.15.117:0/1851643720 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7fbd60107d20 con 0x7fbd701051c0 2024-09-21T06:08:41.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.974+0000 7fbd762d6640 1 --2- 172.21.15.117:0/1851643720 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbd5c0b6700 0x7fbd5c0b8c10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c009bd0 tx=0x7fbd6c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:41.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:41.976+0000 7fbd677fe640 1 -- 172.21.15.117:0/1851643720 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbd600d26a0 con 0x7fbd701051c0 2024-09-21T06:08:42.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.156+0000 7fbd7853a640 1 -- 172.21.15.117:0/1851643720 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbd7019b540 con 0x7fbd701051c0 2024-09-21T06:08:42.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.156+0000 7fbd677fe640 1 -- 172.21.15.117:0/1851643720 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v85) ==== 100+0+11 (secure 0 0 0) 0x7fbd7019b540 con 0x7fbd701051c0 2024-09-21T06:08:42.158 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 64 2024-09-21T06:08:42.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.158+0000 7fbd7853a640 1 -- 172.21.15.117:0/1851643720 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbd5c0b6700 msgr2=0x7fbd5c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:42.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.158+0000 7fbd7853a640 1 --2- 172.21.15.117:0/1851643720 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbd5c0b6700 0x7fbd5c0b8c10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c009bd0 tx=0x7fbd6c005f90 comp rx=0 tx=0).stop 2024-09-21T06:08:42.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.158+0000 7fbd7853a640 1 -- 172.21.15.117:0/1851643720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd701051c0 msgr2=0x7fbd7019a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:42.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.158+0000 7fbd7853a640 1 --2- 172.21.15.117:0/1851643720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd701051c0 0x7fbd7019a9c0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fbd6002a930 tx=0x7fbd60031a20 comp rx=0 tx=0).stop 2024-09-21T06:08:42.160 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.158+0000 7fbd7853a640 1 -- 172.21.15.117:0/1851643720 shutdown_connections 2024-09-21T06:08:42.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.158+0000 7fbd7853a640 1 --2- 172.21.15.117:0/1851643720 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fbd5c0b6700 0x7fbd5c0b8c10 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:42.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.158+0000 7fbd7853a640 1 --2- 172.21.15.117:0/1851643720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd701051c0 0x7fbd7019a9c0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:42.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.158+0000 7fbd7853a640 1 --2- 172.21.15.117:0/1851643720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd7010dd40 0x7fbd7019fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:42.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.158+0000 7fbd7853a640 1 --2- 172.21.15.117:0/1851643720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd70104790 0x7fbd7019a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:42.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.158+0000 7fbd7853a640 1 -- 172.21.15.117:0/1851643720 >> 172.21.15.117:0/1851643720 conn(0x7fbd700fe3c0 msgr2=0x7fbd701078e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:42.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.158+0000 7fbd7853a640 1 -- 172.21.15.117:0/1851643720 shutdown_connections 2024-09-21T06:08:42.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.158+0000 7fbd7853a640 1 -- 172.21.15.117:0/1851643720 wait complete. 2024-09-21T06:08:42.170 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:41 smithi117 ceph-mon[30947]: pgmap v677: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.2 KiB/s wr, 155 op/s 2024-09-21T06:08:42.170 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:41 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2046689186' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:08:42.170 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:41 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:08:42.171 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:41 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:08:42.171 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:41 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:08:42.171 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:41 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:08:42.171 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:41 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:08:42.171 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:41 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:08:42.171 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:41 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:08:42.171 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:41 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3699573581' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T06:08:42.171 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:41 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2305307458' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T06:08:42.171 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:41 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:42.171 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:41 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:08:42.171 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:41 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:08:42.172 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-21T06:08:42.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:41 smithi192 ceph-mon[34700]: pgmap v677: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.2 KiB/s wr, 155 op/s 2024-09-21T06:08:42.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:41 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2046689186' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:08:42.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:41 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:08:42.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:41 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:08:42.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:41 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:08:42.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:41 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:08:42.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:41 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:08:42.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:41 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:08:42.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:41 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:08:42.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:41 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3699573581' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T06:08:42.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:41 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2305307458' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T06:08:42.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:41 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:42.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:41 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:08:42.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:41 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:08:42.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.280+0000 7f1447151640 1 -- 172.21.15.117:0/282104688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1440102190 msgr2=0x7f14401025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:42.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.280+0000 7f1447151640 1 --2- 172.21.15.117:0/282104688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1440102190 0x7f14401025a0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f1428009900 tx=0x7f142802a410 comp rx=0 tx=0).stop 2024-09-21T06:08:42.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.280+0000 7f1447151640 1 -- 172.21.15.117:0/282104688 shutdown_connections 2024-09-21T06:08:42.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.280+0000 7f1447151640 1 --2- 172.21.15.117:0/282104688 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14401083c0 0x7f144010ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:42.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.280+0000 7f1447151640 1 --2- 172.21.15.117:0/282104688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1440107a70 0x7f1440107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:42.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.280+0000 7f1447151640 1 --2- 172.21.15.117:0/282104688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1440102190 0x7f14401025a0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:42.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.280+0000 7f1447151640 1 -- 172.21.15.117:0/282104688 >> 172.21.15.117:0/282104688 conn(0x7f1440079ce0 msgr2=0x7f144007a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:42.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.280+0000 7f1447151640 1 -- 172.21.15.117:0/282104688 shutdown_connections 2024-09-21T06:08:42.283 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.280+0000 7f1447151640 1 -- 172.21.15.117:0/282104688 wait complete. 2024-09-21T06:08:42.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.281+0000 7f1447151640 1 Processor -- start 2024-09-21T06:08:42.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.281+0000 7f1447151640 1 -- start start 2024-09-21T06:08:42.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.281+0000 7f1447151640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1440102190 0x7f144019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:42.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.281+0000 7f1447151640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1440107a70 0x7f144019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:42.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.281+0000 7f1447151640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14401083c0 0x7f144019fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:42.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.281+0000 7f1447151640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1440113a10 con 0x7f14401083c0 2024-09-21T06:08:42.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.281+0000 7f1447151640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1440113890 con 0x7f1440102190 2024-09-21T06:08:42.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.281+0000 7f1447151640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1440113b90 con 0x7f1440107a70 2024-09-21T06:08:42.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.281+0000 7f14456ee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14401083c0 0x7f144019fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:42.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.281+0000 7f14456ee640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14401083c0 0x7f144019fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37802/0 (socket says 172.21.15.117:37802) 2024-09-21T06:08:42.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.281+0000 7f14456ee640 1 -- 172.21.15.117:0/2438118482 learned_addr learned my addr 172.21.15.117:0/2438118482 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:08:42.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.281+0000 7f1444eed640 1 --2- 172.21.15.117:0/2438118482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1440102190 0x7f144019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:42.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.282+0000 7f143ffff640 1 --2- 172.21.15.117:0/2438118482 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1440107a70 0x7f144019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:42.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.282+0000 7f14456ee640 1 -- 172.21.15.117:0/2438118482 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1440107a70 msgr2=0x7f144019a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:42.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.282+0000 7f14456ee640 1 --2- 172.21.15.117:0/2438118482 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1440107a70 0x7f144019a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:42.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.282+0000 7f14456ee640 1 -- 172.21.15.117:0/2438118482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1440102190 msgr2=0x7f144019a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:42.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.282+0000 7f14456ee640 1 --2- 172.21.15.117:0/2438118482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1440102190 0x7f144019a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:42.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.282+0000 7f14456ee640 1 -- 172.21.15.117:0/2438118482 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14401a0320 con 0x7f14401083c0 2024-09-21T06:08:42.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.282+0000 7f1444eed640 1 --2- 172.21.15.117:0/2438118482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1440102190 0x7f144019a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:08:42.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.282+0000 7f14456ee640 1 --2- 172.21.15.117:0/2438118482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14401083c0 0x7f144019fc20 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f143800cad0 tx=0x7f143800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:42.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.282+0000 7f143dffb640 1 -- 172.21.15.117:0/2438118482 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143800ae20 con 0x7f14401083c0 2024-09-21T06:08:42.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.282+0000 7f1447151640 1 -- 172.21.15.117:0/2438118482 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14401a0610 con 0x7f14401083c0 2024-09-21T06:08:42.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.282+0000 7f143dffb640 1 -- 172.21.15.117:0/2438118482 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1438005020 con 0x7f14401083c0 2024-09-21T06:08:42.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.282+0000 7f143dffb640 1 -- 172.21.15.117:0/2438118482 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143800f6b0 con 0x7f14401083c0 2024-09-21T06:08:42.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.282+0000 7f1447151640 1 -- 172.21.15.117:0/2438118482 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14401a6da0 con 0x7f14401083c0 2024-09-21T06:08:42.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.283+0000 7f1447151640 1 -- 172.21.15.117:0/2438118482 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14401025a0 con 0x7f14401083c0 2024-09-21T06:08:42.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.286+0000 7f143dffb640 1 -- 172.21.15.117:0/2438118482 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f14380051d0 con 0x7f14401083c0 2024-09-21T06:08:42.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.286+0000 7f143dffb640 1 --2- 172.21.15.117:0/2438118482 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f142c0b6680 0x7f142c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:08:42.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.286+0000 7f143dffb640 1 -- 172.21.15.117:0/2438118482 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7f14380e7b40 con 0x7f14401083c0 2024-09-21T06:08:42.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.286+0000 7f1444eed640 1 --2- 172.21.15.117:0/2438118482 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f142c0b6680 0x7f142c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:08:42.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.287+0000 7f1444eed640 1 --2- 172.21.15.117:0/2438118482 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f142c0b6680 0x7f142c0b8b90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1428031440 tx=0x7f1428032ed0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:08:42.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.290+0000 7f143dffb640 1 -- 172.21.15.117:0/2438118482 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1438014030 con 0x7f14401083c0 2024-09-21T06:08:42.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:41 smithi138 ceph-mon[34391]: pgmap v677: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.2 KiB/s wr, 155 op/s 2024-09-21T06:08:42.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:41 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2046689186' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:08:42.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:41 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T06:08:42.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:41 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:08:42.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:41 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:08:42.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:41 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:08:42.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:41 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:08:42.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:41 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:08:42.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:41 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:08:42.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:41 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3699573581' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T06:08:42.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:41 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2305307458' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T06:08:42.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:41 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T06:08:42.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:41 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T06:08:42.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:41 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' 2024-09-21T06:08:42.518 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.517+0000 7f1447151640 1 -- 172.21.15.117:0/2438118482 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f144019b510 con 0x7f14401083c0 2024-09-21T06:08:42.527 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.525+0000 7f143dffb640 1 -- 172.21.15.117:0/2438118482 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v24) ==== 81+0+834063 (secure 0 0 0) 0x7f144019b510 con 0x7f14401083c0 2024-09-21T06:08:42.536 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T06:08:42.536 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T06:08:42.537 INFO:teuthology.orchestra.run.smithi117.stdout: "epoch": 24, 2024-09-21T06:08:42.537 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.537 INFO:teuthology.orchestra.run.smithi117.stdout: "active_gid": 14223, 2024-09-21T06:08:42.537 INFO:teuthology.orchestra.run.smithi117.stdout: "active_name": "x", 2024-09-21T06:08:42.537 INFO:teuthology.orchestra.run.smithi117.stdout: "active_addrs": { 2024-09-21T06:08:42.537 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2024-09-21T06:08:42.537 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.537 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2024-09-21T06:08:42.537 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6800", 2024-09-21T06:08:42.537 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 4224731851 2024-09-21T06:08:42.537 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.537 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.537 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v1", 2024-09-21T06:08:42.537 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:6801", 2024-09-21T06:08:42.538 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 4224731851 2024-09-21T06:08:42.538 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.538 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-21T06:08:42.538 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.538 INFO:teuthology.orchestra.run.smithi117.stdout: "active_addr": "172.21.15.117:6801/4224731851", 2024-09-21T06:08:42.538 INFO:teuthology.orchestra.run.smithi117.stdout: "active_change": "2024-09-21T05:48:32.406821+0000", 2024-09-21T06:08:42.538 INFO:teuthology.orchestra.run.smithi117.stdout: "active_mgr_features": 4540719139924082687, 2024-09-21T06:08:42.538 INFO:teuthology.orchestra.run.smithi117.stdout: "available": true, 2024-09-21T06:08:42.538 INFO:teuthology.orchestra.run.smithi117.stdout: "standbys": [ 2024-09-21T06:08:42.538 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.538 INFO:teuthology.orchestra.run.smithi117.stdout: "gid": 24136, 2024-09-21T06:08:42.538 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "z", 2024-09-21T06:08:42.538 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr_features": 4540719139924082687, 2024-09-21T06:08:42.538 INFO:teuthology.orchestra.run.smithi117.stdout: "available_modules": [ 2024-09-21T06:08:42.539 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.539 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "alerts", 2024-09-21T06:08:42.539 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:42.539 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:42.539 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.539 INFO:teuthology.orchestra.run.smithi117.stdout: "interval": { 2024-09-21T06:08:42.539 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "interval", 2024-09-21T06:08:42.539 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:42.539 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.539 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.539 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "60", 2024-09-21T06:08:42.539 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.539 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.539 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.539 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T06:08:42.540 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.540 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.540 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.540 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.540 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.540 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.540 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.540 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.540 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.540 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.540 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.540 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.540 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.540 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.540 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.541 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.541 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.541 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.541 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.541 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.541 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.541 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.541 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.541 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.541 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.541 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.541 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.541 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.541 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.541 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.542 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.542 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.542 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.542 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.542 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.542 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.542 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.542 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.542 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.542 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.542 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.542 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.542 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.542 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.542 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.543 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.543 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.543 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.543 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.543 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.543 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.543 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.543 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.543 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.543 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.543 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.543 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.543 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.543 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.543 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.544 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.544 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.544 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.544 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.544 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.544 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.544 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.544 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.544 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.544 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.544 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.544 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.544 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.544 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.544 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_destination": { 2024-09-21T06:08:42.545 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_destination", 2024-09-21T06:08:42.545 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.545 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.545 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.545 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.545 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.545 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.545 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.545 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T06:08:42.545 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.545 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.545 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.545 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.545 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_from_name": { 2024-09-21T06:08:42.546 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_from_name", 2024-09-21T06:08:42.546 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.546 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.546 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.546 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "Ceph", 2024-09-21T06:08:42.546 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.546 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.546 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.546 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Email From: name", 2024-09-21T06:08:42.546 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.546 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.546 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.546 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.546 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_host": { 2024-09-21T06:08:42.547 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_host", 2024-09-21T06:08:42.547 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.547 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.547 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.547 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.547 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.547 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.547 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.547 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "SMTP server", 2024-09-21T06:08:42.547 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.547 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.547 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.547 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.547 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_password": { 2024-09-21T06:08:42.547 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_password", 2024-09-21T06:08:42.548 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.548 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.548 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.548 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.548 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.548 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.548 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.548 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Password to authenticate with", 2024-09-21T06:08:42.548 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.548 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.548 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.548 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.548 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_port": { 2024-09-21T06:08:42.548 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_port", 2024-09-21T06:08:42.548 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.549 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.549 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.549 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "465", 2024-09-21T06:08:42.549 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.549 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.549 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.549 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "SMTP port", 2024-09-21T06:08:42.549 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.549 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.549 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.549 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.549 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_sender": { 2024-09-21T06:08:42.549 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_sender", 2024-09-21T06:08:42.549 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.549 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.550 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.550 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.550 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.550 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.550 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.550 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "SMTP envelope sender", 2024-09-21T06:08:42.550 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.550 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.550 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.550 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.550 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_ssl": { 2024-09-21T06:08:42.550 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_ssl", 2024-09-21T06:08:42.550 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.551 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.551 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.551 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.551 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.551 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.551 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.551 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T06:08:42.551 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.551 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.551 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.551 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.551 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_user": { 2024-09-21T06:08:42.551 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_user", 2024-09-21T06:08:42.551 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.551 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.552 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.552 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.552 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.552 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.552 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.552 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "User to authenticate as", 2024-09-21T06:08:42.552 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.552 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.552 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.552 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.552 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:42.552 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:42.552 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.552 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.552 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.552 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.553 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.553 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.553 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.553 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.553 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.553 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.553 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.553 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.553 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.553 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.553 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.553 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "balancer", 2024-09-21T06:08:42.553 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:42.553 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:42.553 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.554 INFO:teuthology.orchestra.run.smithi117.stdout: "active": { 2024-09-21T06:08:42.554 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "active", 2024-09-21T06:08:42.554 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.554 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.554 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.554 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.554 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.554 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.554 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.554 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T06:08:42.554 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.554 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.554 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.554 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.555 INFO:teuthology.orchestra.run.smithi117.stdout: "begin_time": { 2024-09-21T06:08:42.555 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "begin_time", 2024-09-21T06:08:42.555 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.555 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.555 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.555 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0000", 2024-09-21T06:08:42.555 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.555 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.555 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.555 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T06:08:42.555 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T06:08:42.555 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.555 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.555 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.556 INFO:teuthology.orchestra.run.smithi117.stdout: "begin_weekday": { 2024-09-21T06:08:42.556 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "begin_weekday", 2024-09-21T06:08:42.556 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "uint", 2024-09-21T06:08:42.556 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.556 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.556 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.556 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "0", 2024-09-21T06:08:42.556 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "6", 2024-09-21T06:08:42.556 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.556 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T06:08:42.556 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T06:08:42.556 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.556 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.556 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.556 INFO:teuthology.orchestra.run.smithi117.stdout: "crush_compat_max_iterations": { 2024-09-21T06:08:42.557 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crush_compat_max_iterations", 2024-09-21T06:08:42.557 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "uint", 2024-09-21T06:08:42.557 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.557 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.557 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "25", 2024-09-21T06:08:42.557 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "1", 2024-09-21T06:08:42.557 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "250", 2024-09-21T06:08:42.557 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.557 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T06:08:42.557 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.557 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.557 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.557 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.557 INFO:teuthology.orchestra.run.smithi117.stdout: "crush_compat_metrics": { 2024-09-21T06:08:42.558 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crush_compat_metrics", 2024-09-21T06:08:42.558 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.558 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.558 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.558 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T06:08:42.558 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.558 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.558 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.558 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T06:08:42.558 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T06:08:42.558 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.558 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.558 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.558 INFO:teuthology.orchestra.run.smithi117.stdout: "crush_compat_step": { 2024-09-21T06:08:42.559 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crush_compat_step", 2024-09-21T06:08:42.559 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-09-21T06:08:42.559 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.559 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.559 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0.5", 2024-09-21T06:08:42.559 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "0.001", 2024-09-21T06:08:42.559 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "0.999", 2024-09-21T06:08:42.559 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.559 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "aggressiveness of optimization", 2024-09-21T06:08:42.559 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T06:08:42.559 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.559 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.559 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.559 INFO:teuthology.orchestra.run.smithi117.stdout: "end_time": { 2024-09-21T06:08:42.560 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "end_time", 2024-09-21T06:08:42.560 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.560 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.560 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.560 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "2359", 2024-09-21T06:08:42.560 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.560 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.560 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.560 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T06:08:42.560 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T06:08:42.560 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.560 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.560 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.560 INFO:teuthology.orchestra.run.smithi117.stdout: "end_weekday": { 2024-09-21T06:08:42.560 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "end_weekday", 2024-09-21T06:08:42.560 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "uint", 2024-09-21T06:08:42.561 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.561 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.561 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.561 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "0", 2024-09-21T06:08:42.561 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "6", 2024-09-21T06:08:42.561 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.561 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T06:08:42.561 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T06:08:42.561 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.561 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.561 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.561 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.561 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.561 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.561 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.561 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.562 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.562 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.562 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.562 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.562 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.562 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.562 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.562 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.562 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.562 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.562 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.562 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.562 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.563 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.563 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.563 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.563 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.563 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.563 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.563 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.563 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.563 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.563 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.563 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.563 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.563 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.563 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.563 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.564 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.564 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.564 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.564 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.564 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.564 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.564 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.564 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.564 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.564 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.564 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.564 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.564 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.564 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.564 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.565 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.565 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.565 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.565 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.565 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.565 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.565 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.565 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.565 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.565 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.565 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.565 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.565 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.565 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.565 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.566 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.566 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.566 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.566 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.566 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.566 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.566 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.566 INFO:teuthology.orchestra.run.smithi117.stdout: "min_score": { 2024-09-21T06:08:42.566 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "min_score", 2024-09-21T06:08:42.566 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-09-21T06:08:42.566 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.566 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.566 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.566 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.566 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.567 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.567 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T06:08:42.567 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.567 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.567 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.567 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.567 INFO:teuthology.orchestra.run.smithi117.stdout: "mode": { 2024-09-21T06:08:42.567 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mode", 2024-09-21T06:08:42.567 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.567 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.567 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.567 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "upmap", 2024-09-21T06:08:42.567 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.567 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.568 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.568 INFO:teuthology.orchestra.run.smithi117.stdout: "crush-compat", 2024-09-21T06:08:42.568 INFO:teuthology.orchestra.run.smithi117.stdout: "none", 2024-09-21T06:08:42.568 INFO:teuthology.orchestra.run.smithi117.stdout: "read", 2024-09-21T06:08:42.568 INFO:teuthology.orchestra.run.smithi117.stdout: "upmap", 2024-09-21T06:08:42.568 INFO:teuthology.orchestra.run.smithi117.stdout: "upmap-read" 2024-09-21T06:08:42.568 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.568 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Balancer mode", 2024-09-21T06:08:42.568 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.568 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.568 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.568 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.568 INFO:teuthology.orchestra.run.smithi117.stdout: "pool_ids": { 2024-09-21T06:08:42.568 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "pool_ids", 2024-09-21T06:08:42.568 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.568 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.569 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.569 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.569 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.569 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.569 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.569 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T06:08:42.569 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.569 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.569 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.569 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.569 INFO:teuthology.orchestra.run.smithi117.stdout: "sleep_interval": { 2024-09-21T06:08:42.569 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sleep_interval", 2024-09-21T06:08:42.569 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:42.569 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.569 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.570 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "60", 2024-09-21T06:08:42.570 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.570 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.570 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.570 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T06:08:42.570 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.570 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.570 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.570 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.570 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:42.570 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:42.570 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.570 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.570 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.570 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.570 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.571 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.571 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.571 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.571 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.571 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.571 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.571 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.571 INFO:teuthology.orchestra.run.smithi117.stdout: "upmap_max_deviation": { 2024-09-21T06:08:42.571 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "upmap_max_deviation", 2024-09-21T06:08:42.571 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.571 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.571 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.571 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "5", 2024-09-21T06:08:42.571 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "1", 2024-09-21T06:08:42.571 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.572 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.572 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T06:08:42.572 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T06:08:42.572 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.572 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.572 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.572 INFO:teuthology.orchestra.run.smithi117.stdout: "upmap_max_optimizations": { 2024-09-21T06:08:42.572 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "upmap_max_optimizations", 2024-09-21T06:08:42.572 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "uint", 2024-09-21T06:08:42.572 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.572 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.572 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-09-21T06:08:42.572 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.572 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.572 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.572 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T06:08:42.573 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.573 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.573 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.573 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.573 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.573 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.573 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.573 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "cephadm", 2024-09-21T06:08:42.573 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:42.573 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:42.573 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.573 INFO:teuthology.orchestra.run.smithi117.stdout: "agent_down_multiplier": { 2024-09-21T06:08:42.573 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "agent_down_multiplier", 2024-09-21T06:08:42.573 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-09-21T06:08:42.573 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.574 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.574 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "3.0", 2024-09-21T06:08:42.574 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.574 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.574 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.574 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T06:08:42.574 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.574 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.574 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.574 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.574 INFO:teuthology.orchestra.run.smithi117.stdout: "agent_refresh_rate": { 2024-09-21T06:08:42.574 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "agent_refresh_rate", 2024-09-21T06:08:42.575 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:42.575 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.575 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.575 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "20", 2024-09-21T06:08:42.575 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.575 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.575 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.575 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T06:08:42.575 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.575 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.575 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.575 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.575 INFO:teuthology.orchestra.run.smithi117.stdout: "agent_starting_port": { 2024-09-21T06:08:42.575 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "agent_starting_port", 2024-09-21T06:08:42.576 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.576 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.576 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.576 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "4721", 2024-09-21T06:08:42.576 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.576 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.576 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.576 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T06:08:42.576 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.576 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.576 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.576 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.576 INFO:teuthology.orchestra.run.smithi117.stdout: "allow_ptrace": { 2024-09-21T06:08:42.576 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "allow_ptrace", 2024-09-21T06:08:42.577 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.577 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.577 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.577 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.577 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.577 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.577 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.577 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T06:08:42.577 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T06:08:42.577 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.577 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.577 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.577 INFO:teuthology.orchestra.run.smithi117.stdout: "autotune_interval": { 2024-09-21T06:08:42.577 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "autotune_interval", 2024-09-21T06:08:42.578 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:42.578 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.578 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.578 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "600", 2024-09-21T06:08:42.578 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.578 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.578 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.578 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T06:08:42.578 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.578 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.578 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.578 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.578 INFO:teuthology.orchestra.run.smithi117.stdout: "autotune_memory_target_ratio": { 2024-09-21T06:08:42.578 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T06:08:42.579 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-09-21T06:08:42.579 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.579 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.579 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0.7", 2024-09-21T06:08:42.579 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.579 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.579 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.579 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T06:08:42.579 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.579 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.579 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.579 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.579 INFO:teuthology.orchestra.run.smithi117.stdout: "cephadm_log_destination": { 2024-09-21T06:08:42.579 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "cephadm_log_destination", 2024-09-21T06:08:42.580 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.580 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.580 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.580 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.580 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.580 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.580 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.580 INFO:teuthology.orchestra.run.smithi117.stdout: "file", 2024-09-21T06:08:42.580 INFO:teuthology.orchestra.run.smithi117.stdout: "file,syslog", 2024-09-21T06:08:42.580 INFO:teuthology.orchestra.run.smithi117.stdout: "syslog" 2024-09-21T06:08:42.580 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.580 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T06:08:42.580 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.580 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.580 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.581 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.581 INFO:teuthology.orchestra.run.smithi117.stdout: "cgroups_split": { 2024-09-21T06:08:42.581 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "cgroups_split", 2024-09-21T06:08:42.581 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.581 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.581 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.581 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.581 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.581 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.581 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.581 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T06:08:42.581 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.581 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.581 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.581 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.582 INFO:teuthology.orchestra.run.smithi117.stdout: "config_checks_enabled": { 2024-09-21T06:08:42.582 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "config_checks_enabled", 2024-09-21T06:08:42.582 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.582 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.582 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.582 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.582 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.582 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.582 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.582 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T06:08:42.582 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.582 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.582 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.582 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.583 INFO:teuthology.orchestra.run.smithi117.stdout: "config_dashboard": { 2024-09-21T06:08:42.583 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "config_dashboard", 2024-09-21T06:08:42.583 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.583 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.583 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.583 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.583 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.583 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.583 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.583 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T06:08:42.583 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.583 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.583 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.583 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.583 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_alertmanager": { 2024-09-21T06:08:42.584 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_alertmanager", 2024-09-21T06:08:42.584 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.584 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.584 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.584 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T06:08:42.584 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.584 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.584 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.584 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Prometheus container image", 2024-09-21T06:08:42.584 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.584 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.584 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.584 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.584 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_base": { 2024-09-21T06:08:42.584 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_base", 2024-09-21T06:08:42.585 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.585 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.585 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.585 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T06:08:42.585 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.585 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.585 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.585 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Container image name, without the tag", 2024-09-21T06:08:42.585 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.585 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.585 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.585 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.585 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_elasticsearch": { 2024-09-21T06:08:42.585 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_elasticsearch", 2024-09-21T06:08:42.585 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.586 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.586 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.586 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T06:08:42.586 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.586 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.586 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.586 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "elasticsearch container image", 2024-09-21T06:08:42.586 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.586 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.586 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.586 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.586 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_grafana": { 2024-09-21T06:08:42.586 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_grafana", 2024-09-21T06:08:42.587 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.587 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.587 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.587 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T06:08:42.587 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.587 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.587 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.587 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Prometheus container image", 2024-09-21T06:08:42.587 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.587 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.587 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.587 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.587 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_haproxy": { 2024-09-21T06:08:42.587 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_haproxy", 2024-09-21T06:08:42.587 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.588 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.588 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.588 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T06:08:42.588 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.588 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.588 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.588 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "HAproxy container image", 2024-09-21T06:08:42.588 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.588 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.588 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.588 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.588 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_jaeger_agent": { 2024-09-21T06:08:42.588 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_jaeger_agent", 2024-09-21T06:08:42.588 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.588 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.589 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.589 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T06:08:42.589 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.589 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.589 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.589 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Jaeger agent container image", 2024-09-21T06:08:42.589 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.589 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.589 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.589 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.589 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_jaeger_collector": { 2024-09-21T06:08:42.589 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_jaeger_collector", 2024-09-21T06:08:42.589 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.589 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.589 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.590 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T06:08:42.590 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.590 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.590 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.590 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Jaeger collector container image", 2024-09-21T06:08:42.590 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.590 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.590 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.590 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.590 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_jaeger_query": { 2024-09-21T06:08:42.590 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_jaeger_query", 2024-09-21T06:08:42.590 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.590 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.590 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.591 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T06:08:42.591 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.591 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.591 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.591 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Jaeger query container image", 2024-09-21T06:08:42.591 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.591 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.591 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.591 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.591 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_keepalived": { 2024-09-21T06:08:42.591 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_keepalived", 2024-09-21T06:08:42.591 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.591 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.591 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.592 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T06:08:42.592 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.592 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.592 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.592 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Keepalived container image", 2024-09-21T06:08:42.592 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.592 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.592 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.592 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.592 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_loki": { 2024-09-21T06:08:42.592 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_loki", 2024-09-21T06:08:42.592 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.592 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.592 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.592 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T06:08:42.592 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.593 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.593 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.593 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Loki container image", 2024-09-21T06:08:42.593 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.593 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.593 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.593 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.593 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_nginx": { 2024-09-21T06:08:42.593 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_nginx", 2024-09-21T06:08:42.593 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.593 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.593 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.593 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T06:08:42.593 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.594 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.594 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.594 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Nginx container image", 2024-09-21T06:08:42.594 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.594 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.594 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.594 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.594 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_node_exporter": { 2024-09-21T06:08:42.594 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_node_exporter", 2024-09-21T06:08:42.594 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.594 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.594 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.594 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T06:08:42.594 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.594 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.595 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.595 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Prometheus container image", 2024-09-21T06:08:42.595 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.595 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.595 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.595 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.595 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_nvmeof": { 2024-09-21T06:08:42.595 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_nvmeof", 2024-09-21T06:08:42.595 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.595 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.595 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.595 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T06:08:42.595 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.595 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.595 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.596 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Nvme-of container image", 2024-09-21T06:08:42.596 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.596 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.596 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.596 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.596 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_oauth2_proxy": { 2024-09-21T06:08:42.596 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T06:08:42.596 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.596 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.596 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.596 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T06:08:42.596 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.596 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.596 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.596 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "oauth2-proxy container image", 2024-09-21T06:08:42.597 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.597 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.597 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.597 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.597 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_prometheus": { 2024-09-21T06:08:42.597 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_prometheus", 2024-09-21T06:08:42.597 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.597 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.597 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.597 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T06:08:42.597 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.597 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.597 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.598 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Prometheus container image", 2024-09-21T06:08:42.598 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.598 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.598 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.598 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.598 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_promtail": { 2024-09-21T06:08:42.598 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_promtail", 2024-09-21T06:08:42.598 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.598 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.598 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.598 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T06:08:42.598 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.598 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.598 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.598 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Promtail container image", 2024-09-21T06:08:42.599 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.599 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.599 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.599 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.599 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_samba": { 2024-09-21T06:08:42.599 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_samba", 2024-09-21T06:08:42.599 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.599 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.599 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.599 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T06:08:42.599 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.599 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.599 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.599 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Samba/SMB container image", 2024-09-21T06:08:42.599 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.600 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.600 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.600 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.600 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_samba_metrics": { 2024-09-21T06:08:42.600 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_samba_metrics", 2024-09-21T06:08:42.600 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.600 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.600 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.600 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T06:08:42.600 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.600 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.600 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.600 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T06:08:42.600 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.600 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.601 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.601 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.601 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_snmp_gateway": { 2024-09-21T06:08:42.601 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_snmp_gateway", 2024-09-21T06:08:42.601 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.601 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.601 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.601 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T06:08:42.601 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.601 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.601 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.601 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "SNMP Gateway container image", 2024-09-21T06:08:42.601 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.601 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.601 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.601 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.602 INFO:teuthology.orchestra.run.smithi117.stdout: "container_init": { 2024-09-21T06:08:42.602 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_init", 2024-09-21T06:08:42.602 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.602 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.602 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.602 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.602 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.602 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.602 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.602 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T06:08:42.602 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.602 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.602 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.602 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.603 INFO:teuthology.orchestra.run.smithi117.stdout: "daemon_cache_timeout": { 2024-09-21T06:08:42.603 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "daemon_cache_timeout", 2024-09-21T06:08:42.603 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:42.603 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.603 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.603 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "600", 2024-09-21T06:08:42.603 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.603 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.603 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.603 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T06:08:42.603 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.603 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.603 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.603 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.603 INFO:teuthology.orchestra.run.smithi117.stdout: "default_cephadm_command_timeout": { 2024-09-21T06:08:42.604 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T06:08:42.604 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.604 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.604 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.604 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "900", 2024-09-21T06:08:42.604 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.604 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.604 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.604 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T06:08:42.604 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.604 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.604 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.604 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.604 INFO:teuthology.orchestra.run.smithi117.stdout: "default_registry": { 2024-09-21T06:08:42.604 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "default_registry", 2024-09-21T06:08:42.605 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.605 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.605 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.605 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "docker.io", 2024-09-21T06:08:42.605 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.605 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.605 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.605 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T06:08:42.605 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.605 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.605 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.605 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.605 INFO:teuthology.orchestra.run.smithi117.stdout: "device_cache_timeout": { 2024-09-21T06:08:42.605 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "device_cache_timeout", 2024-09-21T06:08:42.605 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:42.605 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.606 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.606 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "1800", 2024-09-21T06:08:42.606 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.606 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.606 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.606 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "seconds to cache device inventory", 2024-09-21T06:08:42.606 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.606 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.606 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.606 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.606 INFO:teuthology.orchestra.run.smithi117.stdout: "device_enhanced_scan": { 2024-09-21T06:08:42.606 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "device_enhanced_scan", 2024-09-21T06:08:42.606 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.606 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.606 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.607 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.607 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.607 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.607 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.607 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T06:08:42.607 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.607 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.607 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.607 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.607 INFO:teuthology.orchestra.run.smithi117.stdout: "facts_cache_timeout": { 2024-09-21T06:08:42.607 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "facts_cache_timeout", 2024-09-21T06:08:42.607 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:42.607 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.607 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.607 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "60", 2024-09-21T06:08:42.608 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.608 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.608 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.608 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "seconds to cache host facts data", 2024-09-21T06:08:42.608 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.608 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.608 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.608 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.608 INFO:teuthology.orchestra.run.smithi117.stdout: "grafana_dashboards_path": { 2024-09-21T06:08:42.608 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "grafana_dashboards_path", 2024-09-21T06:08:42.608 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.608 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.608 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.608 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T06:08:42.608 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.609 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.609 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.609 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T06:08:42.609 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.609 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.609 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.609 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.609 INFO:teuthology.orchestra.run.smithi117.stdout: "host_check_interval": { 2024-09-21T06:08:42.609 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "host_check_interval", 2024-09-21T06:08:42.609 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:42.609 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.609 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.609 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "600", 2024-09-21T06:08:42.609 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.610 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.610 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.610 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how frequently to perform a host check", 2024-09-21T06:08:42.610 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.610 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.610 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.610 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.610 INFO:teuthology.orchestra.run.smithi117.stdout: "hw_monitoring": { 2024-09-21T06:08:42.610 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "hw_monitoring", 2024-09-21T06:08:42.610 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.610 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.610 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.610 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.610 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.610 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.611 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.611 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T06:08:42.611 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.611 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.611 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.611 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.611 INFO:teuthology.orchestra.run.smithi117.stdout: "inventory_list_all": { 2024-09-21T06:08:42.611 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "inventory_list_all", 2024-09-21T06:08:42.611 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.611 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.611 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.611 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.611 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.611 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.612 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.612 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T06:08:42.612 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.612 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.612 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.612 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.612 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.612 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.612 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.612 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.612 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.612 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.612 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.612 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.612 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.613 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.613 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.613 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.613 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.613 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.613 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.613 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.613 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.613 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.613 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.613 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.613 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.613 INFO:teuthology.orchestra.run.smithi117.stdout: "log_refresh_metadata": { 2024-09-21T06:08:42.613 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_refresh_metadata", 2024-09-21T06:08:42.614 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.614 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.614 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.614 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.614 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.614 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.614 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.614 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T06:08:42.614 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.614 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.614 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.614 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.614 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.614 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.615 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.615 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.615 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.615 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.615 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.615 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.615 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.615 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T06:08:42.615 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.615 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.615 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.615 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.615 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.615 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.615 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.616 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.616 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.616 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.616 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.616 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.616 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.616 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.616 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.616 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.616 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.616 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.616 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.616 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.616 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.617 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.617 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.617 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.617 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.617 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.617 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.617 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.617 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.617 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.617 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.617 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.617 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.617 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.617 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.617 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.618 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.618 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.618 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.618 INFO:teuthology.orchestra.run.smithi117.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T06:08:42.618 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T06:08:42.618 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.618 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.618 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.618 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.618 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.618 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.618 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.618 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T06:08:42.618 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.619 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.619 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.619 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.619 INFO:teuthology.orchestra.run.smithi117.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T06:08:42.619 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T06:08:42.619 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.619 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.619 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.619 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "*", 2024-09-21T06:08:42.619 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.619 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.619 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.619 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T06:08:42.619 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.619 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.619 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.620 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.620 INFO:teuthology.orchestra.run.smithi117.stdout: "max_count_per_host": { 2024-09-21T06:08:42.620 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "max_count_per_host", 2024-09-21T06:08:42.620 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.620 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.620 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.620 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-09-21T06:08:42.620 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.620 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.620 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.620 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "max number of daemons per service per host", 2024-09-21T06:08:42.620 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.620 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.620 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.620 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.621 INFO:teuthology.orchestra.run.smithi117.stdout: "max_osd_draining_count": { 2024-09-21T06:08:42.621 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "max_osd_draining_count", 2024-09-21T06:08:42.621 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.621 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.621 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.621 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-09-21T06:08:42.621 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.621 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.621 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.621 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T06:08:42.621 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.621 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.621 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.622 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.622 INFO:teuthology.orchestra.run.smithi117.stdout: "migration_current": { 2024-09-21T06:08:42.622 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "migration_current", 2024-09-21T06:08:42.622 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.622 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.622 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.622 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.622 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.622 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.622 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.622 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "internal - do not modify", 2024-09-21T06:08:42.622 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.622 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.622 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.623 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.623 INFO:teuthology.orchestra.run.smithi117.stdout: "mode": { 2024-09-21T06:08:42.623 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mode", 2024-09-21T06:08:42.623 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.623 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.623 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.623 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "root", 2024-09-21T06:08:42.623 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.623 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.623 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.623 INFO:teuthology.orchestra.run.smithi117.stdout: "cephadm-package", 2024-09-21T06:08:42.623 INFO:teuthology.orchestra.run.smithi117.stdout: "root" 2024-09-21T06:08:42.623 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.623 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T06:08:42.623 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.624 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.624 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.624 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.624 INFO:teuthology.orchestra.run.smithi117.stdout: "oob_default_addr": { 2024-09-21T06:08:42.624 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "oob_default_addr", 2024-09-21T06:08:42.624 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.624 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.624 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.624 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "169.254.1.1", 2024-09-21T06:08:42.624 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.624 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.624 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.624 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T06:08:42.625 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.625 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.625 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.625 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.625 INFO:teuthology.orchestra.run.smithi117.stdout: "prometheus_alerts_path": { 2024-09-21T06:08:42.625 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "prometheus_alerts_path", 2024-09-21T06:08:42.625 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.625 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.625 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.625 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T06:08:42.625 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.625 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.625 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.626 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T06:08:42.626 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.626 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.626 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.626 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.626 INFO:teuthology.orchestra.run.smithi117.stdout: "registry_insecure": { 2024-09-21T06:08:42.626 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "registry_insecure", 2024-09-21T06:08:42.626 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.626 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.626 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.626 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.626 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.626 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.626 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.627 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T06:08:42.627 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.627 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.627 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.627 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.627 INFO:teuthology.orchestra.run.smithi117.stdout: "registry_password": { 2024-09-21T06:08:42.627 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "registry_password", 2024-09-21T06:08:42.627 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.627 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.627 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.627 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.627 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.627 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.627 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.627 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T06:08:42.628 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.628 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.628 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.628 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.628 INFO:teuthology.orchestra.run.smithi117.stdout: "registry_url": { 2024-09-21T06:08:42.628 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "registry_url", 2024-09-21T06:08:42.628 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.628 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.628 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.628 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.628 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.628 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.628 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.628 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T06:08:42.628 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.629 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.629 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.629 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.629 INFO:teuthology.orchestra.run.smithi117.stdout: "registry_username": { 2024-09-21T06:08:42.629 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "registry_username", 2024-09-21T06:08:42.629 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.629 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.629 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.629 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.629 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.629 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.629 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.629 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T06:08:42.629 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.629 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.630 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.630 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.630 INFO:teuthology.orchestra.run.smithi117.stdout: "secure_monitoring_stack": { 2024-09-21T06:08:42.630 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "secure_monitoring_stack", 2024-09-21T06:08:42.630 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.630 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.630 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.630 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.630 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.630 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.630 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.630 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T06:08:42.630 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.630 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.630 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.631 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.631 INFO:teuthology.orchestra.run.smithi117.stdout: "service_discovery_port": { 2024-09-21T06:08:42.631 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "service_discovery_port", 2024-09-21T06:08:42.631 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.631 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.631 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.631 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "8765", 2024-09-21T06:08:42.631 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.631 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.631 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.631 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "cephadm service discovery port", 2024-09-21T06:08:42.631 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.631 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.631 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.631 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.632 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:42.632 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:42.632 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.632 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.632 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.632 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.632 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.632 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.632 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.632 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.632 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.632 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.632 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.632 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.632 INFO:teuthology.orchestra.run.smithi117.stdout: "ssh_config_file": { 2024-09-21T06:08:42.632 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ssh_config_file", 2024-09-21T06:08:42.633 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.633 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.633 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.633 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.633 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.633 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.633 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.633 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T06:08:42.633 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.633 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.633 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.633 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.633 INFO:teuthology.orchestra.run.smithi117.stdout: "ssh_keepalive_count_max": { 2024-09-21T06:08:42.633 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T06:08:42.634 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.634 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.634 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.634 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "3", 2024-09-21T06:08:42.634 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.634 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.634 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.634 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T06:08:42.634 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.634 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.634 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.634 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.634 INFO:teuthology.orchestra.run.smithi117.stdout: "ssh_keepalive_interval": { 2024-09-21T06:08:42.634 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ssh_keepalive_interval", 2024-09-21T06:08:42.634 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.635 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.635 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.635 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "7", 2024-09-21T06:08:42.635 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.635 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.635 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.635 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T06:08:42.635 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.635 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.635 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.635 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.635 INFO:teuthology.orchestra.run.smithi117.stdout: "use_agent": { 2024-09-21T06:08:42.635 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "use_agent", 2024-09-21T06:08:42.635 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.635 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.636 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.636 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.636 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.636 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.636 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.636 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T06:08:42.636 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.636 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.636 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.636 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.636 INFO:teuthology.orchestra.run.smithi117.stdout: "use_repo_digest": { 2024-09-21T06:08:42.636 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "use_repo_digest", 2024-09-21T06:08:42.636 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.636 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.636 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.636 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.637 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.637 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.637 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.637 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T06:08:42.637 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.637 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.637 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.637 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.637 INFO:teuthology.orchestra.run.smithi117.stdout: "warn_on_failed_host_check": { 2024-09-21T06:08:42.637 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "warn_on_failed_host_check", 2024-09-21T06:08:42.637 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.637 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.637 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.637 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.637 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.638 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.638 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.638 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T06:08:42.638 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.638 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.638 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.638 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.638 INFO:teuthology.orchestra.run.smithi117.stdout: "warn_on_stray_daemons": { 2024-09-21T06:08:42.638 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "warn_on_stray_daemons", 2024-09-21T06:08:42.638 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.638 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.638 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.638 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.638 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.639 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.639 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.639 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T06:08:42.639 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.639 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.639 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.639 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.639 INFO:teuthology.orchestra.run.smithi117.stdout: "warn_on_stray_hosts": { 2024-09-21T06:08:42.639 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "warn_on_stray_hosts", 2024-09-21T06:08:42.639 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.639 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.639 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.639 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.639 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.639 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.639 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.640 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T06:08:42.640 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.640 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.640 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.640 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.640 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.640 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.640 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.640 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crash", 2024-09-21T06:08:42.640 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:42.640 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:42.640 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.640 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.640 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.640 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.640 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.641 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.641 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.641 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.641 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.641 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.641 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.641 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.641 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.641 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.641 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.641 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.641 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.641 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.641 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.641 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.642 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.642 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.642 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.642 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.642 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.642 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.642 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.642 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.642 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.642 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.642 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.642 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.642 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.642 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.642 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.643 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.643 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.643 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.643 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.643 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.643 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.643 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.643 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.643 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.643 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.643 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.643 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.643 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.643 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.643 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.643 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.644 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.644 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.644 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.644 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.644 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.644 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.644 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.644 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.644 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.644 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.644 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.644 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.644 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.644 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.644 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.645 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.645 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.645 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.645 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.645 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.645 INFO:teuthology.orchestra.run.smithi117.stdout: "retain_interval": { 2024-09-21T06:08:42.645 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "retain_interval", 2024-09-21T06:08:42.645 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:42.645 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.645 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.645 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "31536000", 2024-09-21T06:08:42.645 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.645 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.646 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.646 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T06:08:42.646 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.646 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.646 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.646 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.646 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:42.646 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:42.646 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.646 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.646 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.646 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.646 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.646 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.646 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.647 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.647 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.647 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.647 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.647 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.647 INFO:teuthology.orchestra.run.smithi117.stdout: "warn_recent_interval": { 2024-09-21T06:08:42.647 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "warn_recent_interval", 2024-09-21T06:08:42.647 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:42.647 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.647 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.647 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "1209600", 2024-09-21T06:08:42.647 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.647 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.647 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.647 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T06:08:42.647 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.648 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.648 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.648 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.648 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.648 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.648 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.648 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "dashboard", 2024-09-21T06:08:42.648 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:42.648 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:42.648 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.648 INFO:teuthology.orchestra.run.smithi117.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T06:08:42.648 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T06:08:42.648 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.648 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.648 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.649 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-09-21T06:08:42.649 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.649 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.649 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.649 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.649 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.649 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.649 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.649 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.649 INFO:teuthology.orchestra.run.smithi117.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T06:08:42.649 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T06:08:42.649 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.649 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.649 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.649 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.650 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.650 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.650 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.650 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.650 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.650 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.650 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.650 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.650 INFO:teuthology.orchestra.run.smithi117.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T06:08:42.650 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T06:08:42.650 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.650 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.650 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.650 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.650 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.651 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.651 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.651 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.651 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.651 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.651 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.651 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.651 INFO:teuthology.orchestra.run.smithi117.stdout: "AUDIT_API_ENABLED": { 2024-09-21T06:08:42.651 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T06:08:42.651 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.651 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.651 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.651 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.651 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.651 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.652 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.652 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.652 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.652 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.652 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.652 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.652 INFO:teuthology.orchestra.run.smithi117.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T06:08:42.652 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T06:08:42.652 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.652 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.652 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.652 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.652 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.652 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.652 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.652 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.652 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.653 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.653 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.653 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.653 INFO:teuthology.orchestra.run.smithi117.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T06:08:42.653 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T06:08:42.653 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.653 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.653 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.653 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.653 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.653 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.653 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.653 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.653 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.653 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.653 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.654 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.654 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T06:08:42.654 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T06:08:42.654 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.654 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.654 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.654 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.654 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.654 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.654 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.654 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.654 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.654 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.654 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.655 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.655 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T06:08:42.655 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T06:08:42.655 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.655 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.655 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.655 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.655 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.655 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.655 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.655 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.655 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.655 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.655 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.656 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.656 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T06:08:42.656 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T06:08:42.656 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.656 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.656 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.656 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.656 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.656 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.656 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.656 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.656 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.656 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.656 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.656 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.657 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T06:08:42.657 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T06:08:42.657 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.657 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.657 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.657 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.657 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.657 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.657 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.657 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.657 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.657 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.657 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.657 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.657 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T06:08:42.658 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T06:08:42.658 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.658 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.658 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.658 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.658 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.658 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.658 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.658 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.658 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.658 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.658 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.658 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.658 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T06:08:42.658 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T06:08:42.659 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.659 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.659 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.659 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.659 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.659 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.659 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.659 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.659 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.659 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.659 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.659 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.659 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T06:08:42.659 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T06:08:42.659 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.660 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.660 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.660 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.660 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.660 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.660 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.660 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.660 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.660 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.660 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.660 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.660 INFO:teuthology.orchestra.run.smithi117.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T06:08:42.660 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T06:08:42.660 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.660 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.661 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.661 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.661 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.661 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.661 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.661 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.661 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.661 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.661 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.661 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.661 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T06:08:42.661 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T06:08:42.661 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.661 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.661 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.662 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "admin", 2024-09-21T06:08:42.662 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.662 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.662 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.662 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.662 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.662 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.662 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.662 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.662 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T06:08:42.662 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T06:08:42.662 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.662 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.662 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.663 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.663 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.663 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.663 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.663 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.663 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.663 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.663 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.663 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.663 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_API_URL": { 2024-09-21T06:08:42.663 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_API_URL", 2024-09-21T06:08:42.663 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.663 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.663 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.663 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.664 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.664 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.664 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.664 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.664 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.664 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.664 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.664 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.664 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T06:08:42.664 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T06:08:42.664 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.664 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.664 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.664 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "admin", 2024-09-21T06:08:42.664 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.665 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.665 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.665 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.665 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.665 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.665 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.665 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.665 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T06:08:42.665 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T06:08:42.665 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.665 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.665 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.665 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.665 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.665 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.666 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.666 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.666 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.666 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.666 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.666 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.666 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T06:08:42.666 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T06:08:42.666 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.666 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.666 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.666 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.666 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.666 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.666 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.667 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.667 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.667 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.667 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.667 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.667 INFO:teuthology.orchestra.run.smithi117.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T06:08:42.667 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T06:08:42.667 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.667 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.667 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.667 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.667 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.667 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.667 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.668 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.668 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.668 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.668 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.668 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.668 INFO:teuthology.orchestra.run.smithi117.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T06:08:42.668 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T06:08:42.668 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.668 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.668 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.668 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.668 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.668 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.668 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.668 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.669 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.669 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.669 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.669 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.669 INFO:teuthology.orchestra.run.smithi117.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T06:08:42.669 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T06:08:42.669 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.669 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.669 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.669 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "[]", 2024-09-21T06:08:42.669 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.669 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.669 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.669 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.669 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.670 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.670 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.670 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.670 INFO:teuthology.orchestra.run.smithi117.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T06:08:42.670 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T06:08:42.670 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.670 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.670 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.670 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "{}", 2024-09-21T06:08:42.670 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.670 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.670 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.670 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.670 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.670 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.671 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.671 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.671 INFO:teuthology.orchestra.run.smithi117.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T06:08:42.671 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T06:08:42.671 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.671 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.671 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.671 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.671 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.671 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.671 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.671 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.671 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.671 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.671 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.672 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.672 INFO:teuthology.orchestra.run.smithi117.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T06:08:42.672 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T06:08:42.672 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.672 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.672 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.672 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.672 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.672 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.672 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.672 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.672 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.672 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.672 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.672 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.672 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T06:08:42.673 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T06:08:42.673 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.673 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.673 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.673 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.673 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.673 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.673 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.673 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.673 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.673 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.673 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.674 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.674 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T06:08:42.674 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T06:08:42.674 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.674 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.674 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.674 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.674 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.674 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.674 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.674 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.674 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.674 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.674 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.674 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.675 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T06:08:42.675 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T06:08:42.675 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.675 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.675 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.675 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.675 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.675 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.675 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.675 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.675 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.675 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.675 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.675 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.675 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T06:08:42.676 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T06:08:42.676 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.676 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.676 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.676 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.676 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.676 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.676 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.676 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.676 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.676 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.676 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.676 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.676 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T06:08:42.676 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T06:08:42.677 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.677 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.677 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.677 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.677 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.677 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.677 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.677 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.677 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.677 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.677 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.677 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.677 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T06:08:42.677 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T06:08:42.677 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.677 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.678 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.678 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.678 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.678 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.678 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.678 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.678 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.678 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.678 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.678 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.678 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T06:08:42.678 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T06:08:42.678 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.678 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.678 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.679 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.679 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.679 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.679 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.679 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.679 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.679 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.679 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.679 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.679 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T06:08:42.679 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T06:08:42.679 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.679 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.679 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.679 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.680 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.680 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.680 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.680 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.680 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.680 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.680 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.680 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.680 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T06:08:42.680 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T06:08:42.680 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.680 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.680 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.681 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T06:08:42.681 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.681 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.681 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.681 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.681 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.681 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.681 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.681 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.681 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T06:08:42.681 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T06:08:42.681 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.681 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.681 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.681 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-09-21T06:08:42.682 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.682 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.682 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.682 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.682 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.682 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.682 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.682 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.682 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T06:08:42.682 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T06:08:42.682 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.682 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.682 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.683 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "8", 2024-09-21T06:08:42.683 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.683 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.683 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.683 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.683 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.683 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.683 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.683 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.683 INFO:teuthology.orchestra.run.smithi117.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T06:08:42.683 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T06:08:42.683 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.683 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.683 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.683 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "45", 2024-09-21T06:08:42.684 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.684 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.684 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.684 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.684 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.684 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.684 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.684 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.684 INFO:teuthology.orchestra.run.smithi117.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T06:08:42.684 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T06:08:42.684 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.684 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.684 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.684 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.685 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.685 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.685 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.685 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.685 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.685 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.685 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.685 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.685 INFO:teuthology.orchestra.run.smithi117.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T06:08:42.685 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T06:08:42.685 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.685 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.685 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.685 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "admin", 2024-09-21T06:08:42.685 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.686 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.686 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.686 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.686 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.686 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.686 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.686 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.686 INFO:teuthology.orchestra.run.smithi117.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T06:08:42.686 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T06:08:42.686 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.686 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.686 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.686 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.686 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.687 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.687 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.687 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.687 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.687 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.687 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.687 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.687 INFO:teuthology.orchestra.run.smithi117.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T06:08:42.687 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T06:08:42.687 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.687 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.687 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.687 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.687 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.687 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.688 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.688 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.688 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.688 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.688 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.688 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.688 INFO:teuthology.orchestra.run.smithi117.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T06:08:42.688 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T06:08:42.688 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.688 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.688 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.688 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.688 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.688 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.688 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.688 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.689 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.689 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.689 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.689 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.689 INFO:teuthology.orchestra.run.smithi117.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T06:08:42.689 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T06:08:42.689 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.689 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.689 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.689 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.689 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.689 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.689 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.689 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.689 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.689 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.690 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.690 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.690 INFO:teuthology.orchestra.run.smithi117.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T06:08:42.690 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T06:08:42.690 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.690 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.690 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.690 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-09-21T06:08:42.690 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.690 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.690 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.690 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.690 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.690 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.690 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.691 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.691 INFO:teuthology.orchestra.run.smithi117.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T06:08:42.691 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T06:08:42.691 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.691 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.691 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.691 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "5", 2024-09-21T06:08:42.691 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.691 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.691 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.691 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.691 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.691 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.691 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.691 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.692 INFO:teuthology.orchestra.run.smithi117.stdout: "cross_origin_url": { 2024-09-21T06:08:42.692 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "cross_origin_url", 2024-09-21T06:08:42.692 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.692 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.692 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.692 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.692 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.692 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.692 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.692 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.692 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.692 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.692 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.692 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.692 INFO:teuthology.orchestra.run.smithi117.stdout: "crt_file": { 2024-09-21T06:08:42.692 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crt_file", 2024-09-21T06:08:42.693 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.693 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.693 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.693 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.693 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.693 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.693 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.693 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.693 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.693 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.693 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.693 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.693 INFO:teuthology.orchestra.run.smithi117.stdout: "debug": { 2024-09-21T06:08:42.693 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "debug", 2024-09-21T06:08:42.693 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.693 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.694 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.694 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.694 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.694 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.694 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.694 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Enable/disable debug options", 2024-09-21T06:08:42.694 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.694 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.694 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.694 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.694 INFO:teuthology.orchestra.run.smithi117.stdout: "jwt_token_ttl": { 2024-09-21T06:08:42.694 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "jwt_token_ttl", 2024-09-21T06:08:42.694 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.695 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.695 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.695 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "28800", 2024-09-21T06:08:42.695 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.695 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.695 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.695 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.695 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.695 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.695 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.695 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.695 INFO:teuthology.orchestra.run.smithi117.stdout: "key_file": { 2024-09-21T06:08:42.695 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "key_file", 2024-09-21T06:08:42.695 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.695 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.696 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.696 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.696 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.696 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.696 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.696 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.696 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.696 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.696 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.696 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.696 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.696 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.696 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.696 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.696 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.697 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.697 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.697 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.697 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.697 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.697 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.697 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.697 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.697 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.697 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.697 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.697 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.697 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.697 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.697 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.698 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.698 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.698 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.698 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.698 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.698 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.698 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.698 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.698 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.698 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.698 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.698 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.698 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.698 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.698 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.699 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.699 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.699 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.699 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.699 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.699 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.699 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.699 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.699 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.699 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.699 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.699 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.699 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.699 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.699 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.700 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.700 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.700 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.700 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.700 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.700 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.700 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.700 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.700 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.700 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.700 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.700 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.700 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.700 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.700 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.700 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.701 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.701 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.701 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.701 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.701 INFO:teuthology.orchestra.run.smithi117.stdout: "motd": { 2024-09-21T06:08:42.701 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "motd", 2024-09-21T06:08:42.701 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.701 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.701 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.701 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.701 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.701 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.701 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.701 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "The message of the day", 2024-09-21T06:08:42.701 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.702 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.702 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.702 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.702 INFO:teuthology.orchestra.run.smithi117.stdout: "redirect_resolve_ip_addr": { 2024-09-21T06:08:42.702 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T06:08:42.702 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.702 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.702 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.702 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.702 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.702 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.702 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.702 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.702 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.702 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.703 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.703 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.703 INFO:teuthology.orchestra.run.smithi117.stdout: "server_addr": { 2024-09-21T06:08:42.703 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_addr", 2024-09-21T06:08:42.703 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.703 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.703 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.703 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "::", 2024-09-21T06:08:42.703 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.703 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.703 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.703 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.703 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.703 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.703 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.704 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.704 INFO:teuthology.orchestra.run.smithi117.stdout: "server_port": { 2024-09-21T06:08:42.704 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_port", 2024-09-21T06:08:42.704 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.704 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.704 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.704 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "8080", 2024-09-21T06:08:42.704 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.704 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.704 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.704 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.704 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.704 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.704 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.704 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.705 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:42.705 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:42.705 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.705 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.705 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.705 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.705 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.705 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.705 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.705 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.705 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.705 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.705 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.705 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.705 INFO:teuthology.orchestra.run.smithi117.stdout: "ssl": { 2024-09-21T06:08:42.706 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ssl", 2024-09-21T06:08:42.706 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.706 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.706 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.706 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.706 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.706 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.706 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.706 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.706 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.706 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.706 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.706 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.706 INFO:teuthology.orchestra.run.smithi117.stdout: "ssl_server_port": { 2024-09-21T06:08:42.706 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ssl_server_port", 2024-09-21T06:08:42.707 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.707 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.707 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.707 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "8443", 2024-09-21T06:08:42.707 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.707 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.707 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.707 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.707 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.707 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.707 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.707 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.707 INFO:teuthology.orchestra.run.smithi117.stdout: "sso_oauth2": { 2024-09-21T06:08:42.708 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sso_oauth2", 2024-09-21T06:08:42.708 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.708 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.708 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.708 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.708 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.708 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.708 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.708 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.708 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.708 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.708 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.708 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.708 INFO:teuthology.orchestra.run.smithi117.stdout: "standby_behaviour": { 2024-09-21T06:08:42.709 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "standby_behaviour", 2024-09-21T06:08:42.709 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.709 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.709 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.709 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "redirect", 2024-09-21T06:08:42.709 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.709 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.709 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.709 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.709 INFO:teuthology.orchestra.run.smithi117.stdout: "redirect" 2024-09-21T06:08:42.709 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.709 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.709 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.710 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.710 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.710 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.710 INFO:teuthology.orchestra.run.smithi117.stdout: "standby_error_status_code": { 2024-09-21T06:08:42.710 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "standby_error_status_code", 2024-09-21T06:08:42.710 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.710 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.710 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.710 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "500", 2024-09-21T06:08:42.710 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "400", 2024-09-21T06:08:42.710 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "599", 2024-09-21T06:08:42.710 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.710 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.710 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.711 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.711 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.711 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.711 INFO:teuthology.orchestra.run.smithi117.stdout: "url_prefix": { 2024-09-21T06:08:42.711 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "url_prefix", 2024-09-21T06:08:42.711 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.711 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.711 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.711 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.711 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.711 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.711 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.712 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.712 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.712 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.712 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.712 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.712 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.712 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.712 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.712 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "devicehealth", 2024-09-21T06:08:42.712 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:42.712 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:42.712 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.712 INFO:teuthology.orchestra.run.smithi117.stdout: "enable_monitoring": { 2024-09-21T06:08:42.712 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "enable_monitoring", 2024-09-21T06:08:42.712 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.713 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.713 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.713 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.713 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.713 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.713 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.713 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "monitor device health metrics", 2024-09-21T06:08:42.713 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.713 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.713 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.713 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.713 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.713 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.713 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.713 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.714 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.714 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.714 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.714 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.714 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.714 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.714 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.714 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.714 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.714 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.714 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.714 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.714 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.714 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.715 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.715 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.715 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.715 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.715 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.715 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.715 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.715 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.715 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.715 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.715 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.715 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.715 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.715 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.715 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.715 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.716 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.716 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.716 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.716 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.716 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.716 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.716 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.716 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.716 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.716 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.716 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.716 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.716 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.716 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.717 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.717 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.717 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.717 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.717 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.717 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.717 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.717 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.717 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.717 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.717 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.717 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.717 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.717 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.717 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.717 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.718 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.718 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.718 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.718 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.718 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.718 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.718 INFO:teuthology.orchestra.run.smithi117.stdout: "mark_out_threshold": { 2024-09-21T06:08:42.718 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mark_out_threshold", 2024-09-21T06:08:42.718 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:42.718 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.718 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.718 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "2419200", 2024-09-21T06:08:42.718 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.718 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.719 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.719 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T06:08:42.719 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.719 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.719 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.719 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.719 INFO:teuthology.orchestra.run.smithi117.stdout: "pool_name": { 2024-09-21T06:08:42.719 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "pool_name", 2024-09-21T06:08:42.719 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.719 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.719 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.719 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "device_health_metrics", 2024-09-21T06:08:42.719 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.719 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.719 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.720 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T06:08:42.720 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.720 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.720 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.720 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.720 INFO:teuthology.orchestra.run.smithi117.stdout: "retention_period": { 2024-09-21T06:08:42.720 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "retention_period", 2024-09-21T06:08:42.720 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:42.720 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.720 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.720 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "15552000", 2024-09-21T06:08:42.720 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.720 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.720 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.721 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how long to retain device health metrics", 2024-09-21T06:08:42.721 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.721 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.721 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.721 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.721 INFO:teuthology.orchestra.run.smithi117.stdout: "scrape_frequency": { 2024-09-21T06:08:42.721 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "scrape_frequency", 2024-09-21T06:08:42.721 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:42.721 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.721 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.721 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "86400", 2024-09-21T06:08:42.721 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.721 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.721 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.721 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T06:08:42.722 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.722 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.722 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.722 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.722 INFO:teuthology.orchestra.run.smithi117.stdout: "self_heal": { 2024-09-21T06:08:42.722 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "self_heal", 2024-09-21T06:08:42.722 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.722 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.722 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.722 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.722 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.722 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.722 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.722 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T06:08:42.722 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.723 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.723 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.723 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.723 INFO:teuthology.orchestra.run.smithi117.stdout: "sleep_interval": { 2024-09-21T06:08:42.723 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sleep_interval", 2024-09-21T06:08:42.723 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:42.723 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.723 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.723 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "600", 2024-09-21T06:08:42.723 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.723 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.723 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.723 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T06:08:42.723 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.724 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.724 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.724 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.724 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:42.724 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:42.724 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.724 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.724 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.724 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.724 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.724 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.724 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.724 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.724 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.724 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.724 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.725 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.725 INFO:teuthology.orchestra.run.smithi117.stdout: "warn_threshold": { 2024-09-21T06:08:42.725 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "warn_threshold", 2024-09-21T06:08:42.725 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:42.725 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.725 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.725 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "7257600", 2024-09-21T06:08:42.725 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.725 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.725 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.725 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T06:08:42.725 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.725 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.725 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.725 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.726 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.726 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.726 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.726 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "diskprediction_local", 2024-09-21T06:08:42.726 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:42.726 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:42.726 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.726 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.726 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.726 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.726 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.726 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.726 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.726 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.726 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.727 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.727 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.727 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.727 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.727 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.727 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.727 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.727 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.727 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.727 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.727 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.727 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.727 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.727 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.727 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.727 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.728 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.728 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.728 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.728 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.728 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.728 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.728 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.728 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.728 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.728 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.728 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.728 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.728 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.728 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.728 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.728 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.729 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.729 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.729 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.729 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.729 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.729 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.729 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.729 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.729 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.729 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.729 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.729 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.729 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.729 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.729 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.730 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.730 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.730 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.730 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.730 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.730 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.730 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.730 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.730 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.730 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.730 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.730 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.730 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.730 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.730 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.731 INFO:teuthology.orchestra.run.smithi117.stdout: "predict_interval": { 2024-09-21T06:08:42.731 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "predict_interval", 2024-09-21T06:08:42.731 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.731 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.731 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.731 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "86400", 2024-09-21T06:08:42.731 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.731 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.731 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.731 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.731 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.731 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.731 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.731 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.731 INFO:teuthology.orchestra.run.smithi117.stdout: "predictor_model": { 2024-09-21T06:08:42.731 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "predictor_model", 2024-09-21T06:08:42.732 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.732 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.732 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.732 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "prophetstor", 2024-09-21T06:08:42.732 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.732 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.732 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.732 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.732 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.732 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.732 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.732 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.732 INFO:teuthology.orchestra.run.smithi117.stdout: "sleep_interval": { 2024-09-21T06:08:42.732 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sleep_interval", 2024-09-21T06:08:42.732 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.733 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.733 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.733 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "600", 2024-09-21T06:08:42.733 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.733 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.733 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.733 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.733 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.733 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.733 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.733 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.733 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:42.733 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:42.733 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.734 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.734 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.734 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.734 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.734 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.734 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.734 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.734 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.734 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.734 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.734 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.734 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.734 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.734 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.734 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "influx", 2024-09-21T06:08:42.735 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": false, 2024-09-21T06:08:42.735 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "influxdb python module not found", 2024-09-21T06:08:42.735 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.735 INFO:teuthology.orchestra.run.smithi117.stdout: "batch_size": { 2024-09-21T06:08:42.735 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "batch_size", 2024-09-21T06:08:42.735 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.735 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.735 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.735 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "5000", 2024-09-21T06:08:42.735 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.735 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.735 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.735 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T06:08:42.735 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.735 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.736 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.736 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.736 INFO:teuthology.orchestra.run.smithi117.stdout: "database": { 2024-09-21T06:08:42.736 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "database", 2024-09-21T06:08:42.736 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.736 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.736 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.736 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "ceph", 2024-09-21T06:08:42.736 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.736 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.736 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.736 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T06:08:42.736 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.736 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.736 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.736 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.737 INFO:teuthology.orchestra.run.smithi117.stdout: "hostname": { 2024-09-21T06:08:42.737 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "hostname", 2024-09-21T06:08:42.737 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.737 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.737 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.737 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.737 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.737 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.737 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.737 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "InfluxDB server hostname", 2024-09-21T06:08:42.737 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.737 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.737 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.737 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.737 INFO:teuthology.orchestra.run.smithi117.stdout: "interval": { 2024-09-21T06:08:42.738 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "interval", 2024-09-21T06:08:42.738 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:42.738 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.738 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.738 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "30", 2024-09-21T06:08:42.738 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "5", 2024-09-21T06:08:42.738 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.738 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.738 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T06:08:42.738 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.738 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.738 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.738 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.738 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.739 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.739 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.739 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.739 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.739 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.739 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.739 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.739 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.739 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.739 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.739 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.739 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.739 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.739 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.739 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.739 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.740 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.740 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.740 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.740 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.740 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.740 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.740 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.740 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.740 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.740 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.740 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.740 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.740 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.740 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.740 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.741 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.741 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.741 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.741 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.741 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.741 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.741 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.741 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.741 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.741 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.741 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.741 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.741 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.741 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.741 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.742 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.742 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.742 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.742 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.742 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.742 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.742 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.742 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.742 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.742 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.742 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.742 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.742 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.742 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.742 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.742 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.743 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.743 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.743 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.743 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.743 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.743 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.743 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.743 INFO:teuthology.orchestra.run.smithi117.stdout: "password": { 2024-09-21T06:08:42.743 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "password", 2024-09-21T06:08:42.743 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.743 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.743 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.743 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.743 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.743 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.744 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.744 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "password of InfluxDB server user", 2024-09-21T06:08:42.744 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.744 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.744 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.744 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.744 INFO:teuthology.orchestra.run.smithi117.stdout: "port": { 2024-09-21T06:08:42.744 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "port", 2024-09-21T06:08:42.744 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.744 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.744 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.744 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "8086", 2024-09-21T06:08:42.744 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.744 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.744 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.745 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "InfluxDB server port", 2024-09-21T06:08:42.745 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.745 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.745 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.745 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.745 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:42.745 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:42.745 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.745 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.745 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.745 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.745 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.745 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.745 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.746 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.746 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.746 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.746 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.746 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.746 INFO:teuthology.orchestra.run.smithi117.stdout: "ssl": { 2024-09-21T06:08:42.746 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ssl", 2024-09-21T06:08:42.746 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.746 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.746 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.746 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "false", 2024-09-21T06:08:42.746 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.746 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.746 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.747 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T06:08:42.747 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.747 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.747 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.747 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.747 INFO:teuthology.orchestra.run.smithi117.stdout: "threads": { 2024-09-21T06:08:42.747 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "threads", 2024-09-21T06:08:42.747 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.747 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.747 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.747 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "5", 2024-09-21T06:08:42.747 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "1", 2024-09-21T06:08:42.747 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "32", 2024-09-21T06:08:42.747 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.747 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T06:08:42.748 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.748 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.748 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.748 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.748 INFO:teuthology.orchestra.run.smithi117.stdout: "username": { 2024-09-21T06:08:42.748 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "username", 2024-09-21T06:08:42.748 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.748 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.748 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.748 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.748 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.748 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.748 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.748 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "username of InfluxDB server user", 2024-09-21T06:08:42.748 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.749 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.749 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.749 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.749 INFO:teuthology.orchestra.run.smithi117.stdout: "verify_ssl": { 2024-09-21T06:08:42.749 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "verify_ssl", 2024-09-21T06:08:42.749 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.749 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.749 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.749 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "true", 2024-09-21T06:08:42.749 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.749 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.749 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.749 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T06:08:42.749 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.749 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.750 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.750 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.750 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.750 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.750 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.750 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "insights", 2024-09-21T06:08:42.750 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:42.750 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:42.750 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.750 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.750 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.750 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.750 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.750 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.750 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.751 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.751 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.751 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.751 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.751 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.751 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.751 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.751 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.751 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.751 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.751 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.751 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.751 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.751 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.751 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.752 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.752 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.752 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.752 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.752 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.752 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.752 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.752 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.752 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.752 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.752 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.752 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.752 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.752 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.752 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.753 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.753 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.753 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.753 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.753 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.753 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.753 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.753 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.753 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.753 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.753 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.753 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.753 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.753 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.753 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.753 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.754 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.754 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.754 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.754 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.754 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.754 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.754 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.754 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.754 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.754 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.754 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.754 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.754 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.754 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.754 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.755 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.755 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.755 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.755 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:42.755 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:42.755 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.755 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.755 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.755 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.755 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.755 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.755 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.755 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.755 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.755 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.756 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.756 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.756 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.756 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.756 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.756 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "iostat", 2024-09-21T06:08:42.756 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:42.756 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:42.756 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.756 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.756 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.756 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.756 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.756 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.756 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.757 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.757 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.757 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.757 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.757 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.757 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.757 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.757 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.757 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.757 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.757 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.757 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.757 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.757 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.757 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.757 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.758 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.758 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.758 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.758 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.758 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.758 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.758 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.758 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.758 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.758 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.758 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.758 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.758 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.759 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.759 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.759 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.759 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.759 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.759 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.759 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.759 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.759 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.759 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.759 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.759 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.759 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.759 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.760 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.760 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.760 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.760 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.760 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.760 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.760 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.760 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.760 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.760 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.760 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.760 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.760 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.760 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.760 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.760 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.761 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.761 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.761 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.761 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.761 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.761 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:42.761 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:42.761 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.761 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.761 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.761 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.761 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.761 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.761 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.761 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.762 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.762 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.762 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.762 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.762 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.762 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.762 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.762 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "k8sevents", 2024-09-21T06:08:42.762 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:42.762 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:42.762 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.762 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_event_retention_days": { 2024-09-21T06:08:42.762 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ceph_event_retention_days", 2024-09-21T06:08:42.762 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.762 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.763 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.763 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "7", 2024-09-21T06:08:42.763 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.763 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.763 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.763 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T06:08:42.763 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.763 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.763 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.763 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.763 INFO:teuthology.orchestra.run.smithi117.stdout: "config_check_secs": { 2024-09-21T06:08:42.763 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "config_check_secs", 2024-09-21T06:08:42.763 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.763 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.763 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.764 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-09-21T06:08:42.764 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "10", 2024-09-21T06:08:42.764 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.764 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.764 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T06:08:42.764 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.764 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.764 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.764 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.764 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.764 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.764 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.764 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.764 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.764 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.764 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.765 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.765 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.765 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.765 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.765 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.765 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.765 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.765 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.765 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.765 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.765 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.765 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.765 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.765 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.765 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.766 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.766 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.766 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.766 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.766 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.766 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.766 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.766 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.766 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.766 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.766 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.766 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.766 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.766 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.766 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.767 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.767 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.767 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.767 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.767 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.767 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.767 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.767 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.767 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.767 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.767 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.767 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.767 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.767 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.768 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.768 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.768 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.768 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.768 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.768 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.768 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.768 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.768 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.768 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.768 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.768 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.768 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.768 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.768 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.769 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.769 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.769 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.769 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.769 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:42.769 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:42.769 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.769 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.769 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.769 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.769 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.769 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.769 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.769 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.769 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.770 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.770 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.770 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.770 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.770 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.770 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.770 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "localpool", 2024-09-21T06:08:42.770 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:42.770 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:42.770 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.770 INFO:teuthology.orchestra.run.smithi117.stdout: "failure_domain": { 2024-09-21T06:08:42.770 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "failure_domain", 2024-09-21T06:08:42.770 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.770 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.770 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.771 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "host", 2024-09-21T06:08:42.771 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.771 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.771 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.771 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "failure domain for any created local pool", 2024-09-21T06:08:42.771 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T06:08:42.771 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.771 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.771 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.771 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.771 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.771 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.771 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.771 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.772 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.772 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.772 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.772 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.772 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.772 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.772 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.772 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.772 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.772 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.772 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.772 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.772 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.772 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.773 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.773 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.773 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.773 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.773 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.773 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.773 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.773 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.773 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.773 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.773 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.773 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.773 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.773 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.773 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.773 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.774 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.774 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.774 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.774 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.774 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.774 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.774 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.774 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.774 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.774 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.774 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.774 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.774 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.774 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.775 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.775 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.775 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.775 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.775 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.775 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.775 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.775 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.775 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.775 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.775 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.775 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.775 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.775 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.776 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.776 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.776 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.776 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.776 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.776 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.776 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.776 INFO:teuthology.orchestra.run.smithi117.stdout: "min_size": { 2024-09-21T06:08:42.776 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "min_size", 2024-09-21T06:08:42.776 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.776 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.776 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.776 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.776 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.776 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.777 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.777 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "default min_size for any created local pool", 2024-09-21T06:08:42.777 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T06:08:42.777 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.777 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.777 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.777 INFO:teuthology.orchestra.run.smithi117.stdout: "num_rep": { 2024-09-21T06:08:42.777 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "num_rep", 2024-09-21T06:08:42.777 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.777 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.777 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.777 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "3", 2024-09-21T06:08:42.777 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.777 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.777 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.777 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "default replica count for any created local pool", 2024-09-21T06:08:42.778 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.778 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.778 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.778 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.778 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_num": { 2024-09-21T06:08:42.778 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "pg_num", 2024-09-21T06:08:42.778 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.778 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.778 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.778 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "128", 2024-09-21T06:08:42.778 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.778 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.778 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.778 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T06:08:42.779 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.779 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.779 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.779 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.779 INFO:teuthology.orchestra.run.smithi117.stdout: "prefix": { 2024-09-21T06:08:42.779 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "prefix", 2024-09-21T06:08:42.779 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.779 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.779 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.779 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.779 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.779 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.779 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.779 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "name prefix for any created local pool", 2024-09-21T06:08:42.780 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.780 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.780 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.780 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.780 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:42.780 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:42.780 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.780 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.780 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.780 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.780 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.780 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.780 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.780 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.780 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.781 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.781 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.781 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.781 INFO:teuthology.orchestra.run.smithi117.stdout: "subtree": { 2024-09-21T06:08:42.781 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "subtree", 2024-09-21T06:08:42.781 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.781 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.781 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.781 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "rack", 2024-09-21T06:08:42.781 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.781 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.781 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.781 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T06:08:42.781 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T06:08:42.781 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.782 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.782 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.782 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.782 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.782 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.782 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mds_autoscaler", 2024-09-21T06:08:42.782 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:42.782 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:42.782 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.782 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.782 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.782 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.782 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.782 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.782 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.783 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.783 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.783 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.783 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.783 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.783 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.783 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.783 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.783 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.783 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.783 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.783 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.783 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.784 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.784 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.784 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.784 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.784 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.784 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.784 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.784 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.784 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.784 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.784 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.784 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.784 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.784 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.785 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.785 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.785 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.785 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.785 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.785 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.785 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.785 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.785 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.785 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.785 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.785 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.785 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.785 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.785 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.786 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.786 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.786 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.786 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.786 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.786 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.786 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.786 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.786 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.786 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.786 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.786 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.786 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.786 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.786 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.787 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.787 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.787 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.787 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.787 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.787 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.787 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.787 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:42.787 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:42.787 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.787 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.787 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.787 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.787 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.788 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.788 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.788 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.788 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.788 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.788 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.788 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.788 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.788 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.788 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.788 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mirroring", 2024-09-21T06:08:42.788 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:42.788 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:42.788 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.788 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.789 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.789 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.789 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.789 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.789 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.789 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.789 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.789 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.789 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.789 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.789 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.789 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.789 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.789 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.789 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.790 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.790 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.790 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.790 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.790 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.790 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.790 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.790 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.790 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.790 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.790 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.790 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.790 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.790 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.791 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.791 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.791 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.791 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.791 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.791 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.791 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.791 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.791 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.791 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.791 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.791 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.791 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.791 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.791 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.791 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.792 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.792 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.792 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.792 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.792 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.792 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.792 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.792 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.792 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.792 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.792 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.792 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.792 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.792 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.792 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.792 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.792 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.793 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.793 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.793 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.793 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.793 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.793 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.793 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.793 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:42.793 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:42.793 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.793 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.793 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.793 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.793 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.793 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.793 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.794 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.794 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.794 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.794 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.794 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.794 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.794 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.794 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.794 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "nfs", 2024-09-21T06:08:42.794 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:42.794 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:42.794 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.794 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.794 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.794 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.795 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.795 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.795 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.795 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.795 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.795 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.795 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.795 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.795 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.795 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.795 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.795 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.795 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.795 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.796 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.796 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.796 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.796 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.796 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.796 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.796 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.796 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.796 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.796 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.796 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.796 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.796 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.796 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.796 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.796 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.797 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.797 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.797 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.797 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.797 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.797 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.797 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.797 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.797 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.797 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.797 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.797 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.797 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.797 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.798 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.798 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.798 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.798 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.798 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.798 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.798 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.798 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.798 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.798 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.798 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.798 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.798 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.798 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.798 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.799 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.799 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.799 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.799 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.799 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.799 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.799 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.799 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.799 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:42.799 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:42.799 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.799 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.799 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.799 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.799 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.800 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.800 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.800 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.800 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.800 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.800 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.800 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.800 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.800 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.800 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.800 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "orchestrator", 2024-09-21T06:08:42.800 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:42.800 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:42.800 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.800 INFO:teuthology.orchestra.run.smithi117.stdout: "fail_fs": { 2024-09-21T06:08:42.801 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "fail_fs", 2024-09-21T06:08:42.801 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.801 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.801 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.801 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.801 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.801 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.801 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.801 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T06:08:42.801 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.801 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.801 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.801 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.801 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.801 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.802 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.802 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.802 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.802 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.802 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.802 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.802 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.802 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.802 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.802 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.802 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.802 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.802 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.802 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.802 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.802 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.803 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.803 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.803 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.803 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.803 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.803 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.803 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.803 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.803 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.803 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.803 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.803 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.803 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.803 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.803 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.804 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.804 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.804 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.804 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.804 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.804 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.804 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.804 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.804 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.804 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.804 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.804 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.804 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.804 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.804 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.804 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.805 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.805 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.805 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.805 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.805 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.805 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.805 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.805 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.805 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.805 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.805 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.805 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.805 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.805 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.805 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.805 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.806 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.806 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.806 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.806 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.806 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.806 INFO:teuthology.orchestra.run.smithi117.stdout: "orchestrator": { 2024-09-21T06:08:42.806 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "orchestrator", 2024-09-21T06:08:42.806 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.806 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.806 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.806 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.806 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.806 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.806 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.806 INFO:teuthology.orchestra.run.smithi117.stdout: "cephadm", 2024-09-21T06:08:42.807 INFO:teuthology.orchestra.run.smithi117.stdout: "rook", 2024-09-21T06:08:42.807 INFO:teuthology.orchestra.run.smithi117.stdout: "test_orchestrator" 2024-09-21T06:08:42.807 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.807 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Orchestrator backend", 2024-09-21T06:08:42.807 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.807 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.807 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.807 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.807 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:42.807 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:42.807 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.807 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.807 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.807 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.808 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.808 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.808 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.808 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.808 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.808 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.808 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.808 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.808 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.808 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.808 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.808 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "osd_perf_query", 2024-09-21T06:08:42.808 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:42.808 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:42.808 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.809 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.809 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.809 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.809 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.809 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.809 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.809 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.809 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.809 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.809 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.809 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.809 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.809 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.809 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.809 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.810 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.810 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.810 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.810 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.810 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.810 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.810 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.810 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.810 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.810 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.810 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.810 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.810 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.811 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.811 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.811 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.811 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.811 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.811 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.811 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.811 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.811 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.811 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.811 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.811 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.811 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.811 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.811 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.811 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.812 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.812 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.812 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.812 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.812 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.812 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.812 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.812 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.812 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.812 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.812 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.812 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.812 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.812 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.812 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.812 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.813 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.813 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.813 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.813 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.813 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.813 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.813 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.813 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.813 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.813 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.813 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:42.813 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:42.813 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.813 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.813 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.814 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.814 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.814 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.814 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.814 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.814 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.814 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.814 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.814 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.814 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.814 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.814 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.814 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "osd_support", 2024-09-21T06:08:42.814 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:42.815 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:42.815 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.815 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.815 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.815 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.815 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.815 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.815 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.815 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.815 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.815 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.815 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.815 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.815 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.816 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.816 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.816 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.816 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.816 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.816 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.816 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.816 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.816 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.816 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.816 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.816 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.816 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.816 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.816 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.817 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.817 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.817 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.817 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.817 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.817 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.817 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.817 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.817 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.817 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.817 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.817 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.817 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.817 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.817 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.818 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.818 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.818 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.818 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.818 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.818 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.818 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.818 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.818 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.818 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.818 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.818 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.818 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.818 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.818 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.818 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.819 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.819 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.819 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.819 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.819 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.819 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.819 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.819 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.819 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.819 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.819 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.819 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.819 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:42.819 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:42.819 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.820 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.820 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.820 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.820 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.820 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.820 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.820 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.820 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.820 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.820 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.820 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.820 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.820 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.820 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.820 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "pg_autoscaler", 2024-09-21T06:08:42.821 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:42.821 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:42.821 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.821 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.821 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.821 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.821 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.821 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.821 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.821 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.821 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.821 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.821 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.821 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.821 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.821 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.822 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.822 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.822 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.822 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.822 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.822 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.822 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.822 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.822 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.822 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.822 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.822 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.822 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.822 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.822 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.823 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.823 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.823 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.823 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.823 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.823 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.823 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.823 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.823 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.823 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.823 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.823 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.823 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.824 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.824 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.824 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.824 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.824 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.824 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.824 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.824 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.824 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.824 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.824 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.824 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.824 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.824 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.825 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.825 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.825 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.825 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.825 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.825 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.825 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.825 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.825 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.825 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.825 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.825 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.825 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.825 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.826 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.826 INFO:teuthology.orchestra.run.smithi117.stdout: "sleep_interval": { 2024-09-21T06:08:42.826 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sleep_interval", 2024-09-21T06:08:42.826 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:42.826 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.826 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.826 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "60", 2024-09-21T06:08:42.826 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.826 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.826 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.826 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.826 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.826 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.826 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.826 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.827 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:42.827 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:42.827 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.827 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.827 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.827 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.827 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.827 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.827 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.827 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.827 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.827 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.827 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.827 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.828 INFO:teuthology.orchestra.run.smithi117.stdout: "threshold": { 2024-09-21T06:08:42.828 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "threshold", 2024-09-21T06:08:42.828 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-09-21T06:08:42.828 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.828 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.828 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "3.0", 2024-09-21T06:08:42.828 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "1.0", 2024-09-21T06:08:42.828 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.828 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.828 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "scaling threshold", 2024-09-21T06:08:42.828 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T06:08:42.828 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.828 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.828 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.828 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.828 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.829 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.829 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "progress", 2024-09-21T06:08:42.829 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:42.829 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:42.829 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.829 INFO:teuthology.orchestra.run.smithi117.stdout: "allow_pg_recovery_event": { 2024-09-21T06:08:42.829 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "allow_pg_recovery_event", 2024-09-21T06:08:42.829 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.829 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.829 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.829 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.829 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.829 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.829 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.829 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T06:08:42.830 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.830 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.830 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.830 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.830 INFO:teuthology.orchestra.run.smithi117.stdout: "enabled": { 2024-09-21T06:08:42.830 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "enabled", 2024-09-21T06:08:42.830 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.830 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.830 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.830 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.830 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.830 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.830 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.830 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.830 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.830 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.831 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.831 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.831 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.831 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.831 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.831 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.831 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.831 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.831 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.831 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.831 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.831 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.831 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.831 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.831 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.832 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.832 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.832 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.832 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.832 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.832 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.832 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.832 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.832 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.832 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.832 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.832 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.832 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.832 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.832 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.832 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.833 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.833 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.833 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.833 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.833 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.833 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.833 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.833 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.833 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.833 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.833 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.833 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.833 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.833 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.833 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.834 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.834 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.834 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.834 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.834 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.834 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.834 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.834 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.834 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.834 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.834 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.834 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.834 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.834 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.834 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.834 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.835 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.835 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.835 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.835 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.835 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.835 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.835 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.835 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.835 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.835 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.835 INFO:teuthology.orchestra.run.smithi117.stdout: "max_completed_events": { 2024-09-21T06:08:42.835 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "max_completed_events", 2024-09-21T06:08:42.836 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.836 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.836 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.836 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "50", 2024-09-21T06:08:42.836 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.836 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.836 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.836 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "number of past completed events to remember", 2024-09-21T06:08:42.836 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.836 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.836 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.836 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.836 INFO:teuthology.orchestra.run.smithi117.stdout: "sleep_interval": { 2024-09-21T06:08:42.836 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sleep_interval", 2024-09-21T06:08:42.836 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:42.837 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.837 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.837 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "5", 2024-09-21T06:08:42.837 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.837 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.837 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.837 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how long the module is going to sleep", 2024-09-21T06:08:42.837 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.837 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.837 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.837 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.837 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:42.837 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:42.837 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.838 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.838 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.838 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.838 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.838 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.838 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.838 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.838 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.838 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.838 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.838 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.838 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.838 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.839 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.839 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "prometheus", 2024-09-21T06:08:42.839 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:42.839 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:42.839 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.839 INFO:teuthology.orchestra.run.smithi117.stdout: "cache": { 2024-09-21T06:08:42.839 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "cache", 2024-09-21T06:08:42.839 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.839 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.839 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.839 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.839 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.839 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.839 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.839 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.839 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.840 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.840 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.840 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.840 INFO:teuthology.orchestra.run.smithi117.stdout: "exclude_perf_counters": { 2024-09-21T06:08:42.840 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "exclude_perf_counters", 2024-09-21T06:08:42.840 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.840 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.840 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.840 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.840 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.840 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.840 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.840 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T06:08:42.840 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T06:08:42.840 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.840 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.840 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.841 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.841 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.841 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.841 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.841 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.841 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.841 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.841 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.841 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.841 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.841 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.841 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.841 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.841 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.841 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.841 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.841 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.842 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.842 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.842 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.842 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.842 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.842 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.842 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.842 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.842 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.842 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.842 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.842 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.842 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.842 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.842 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.842 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.843 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.843 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.843 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.843 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.843 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.843 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.843 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.843 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.843 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.843 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.843 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.843 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.843 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.843 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.843 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.843 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.844 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.844 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.844 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.844 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.844 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.844 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.844 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.844 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.844 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.844 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.844 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.844 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.844 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.844 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.844 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.844 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.845 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.845 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.845 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.845 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.845 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.845 INFO:teuthology.orchestra.run.smithi117.stdout: "rbd_stats_pools": { 2024-09-21T06:08:42.845 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rbd_stats_pools", 2024-09-21T06:08:42.845 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.845 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.845 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.845 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.845 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.845 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.845 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.845 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.846 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.846 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.846 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.846 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.846 INFO:teuthology.orchestra.run.smithi117.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T06:08:42.846 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T06:08:42.846 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.846 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.846 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.846 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "300", 2024-09-21T06:08:42.846 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.846 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.846 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.846 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.847 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.847 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.847 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.847 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.847 INFO:teuthology.orchestra.run.smithi117.stdout: "scrape_interval": { 2024-09-21T06:08:42.847 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "scrape_interval", 2024-09-21T06:08:42.847 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-09-21T06:08:42.847 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.847 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.847 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "15.0", 2024-09-21T06:08:42.847 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.847 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.847 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.847 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.848 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.848 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.848 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.848 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.848 INFO:teuthology.orchestra.run.smithi117.stdout: "server_addr": { 2024-09-21T06:08:42.848 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_addr", 2024-09-21T06:08:42.848 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.848 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.848 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.849 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "::", 2024-09-21T06:08:42.849 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.849 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.849 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.849 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T06:08:42.850 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.850 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.850 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.850 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.850 INFO:teuthology.orchestra.run.smithi117.stdout: "server_port": { 2024-09-21T06:08:42.850 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_port", 2024-09-21T06:08:42.850 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.850 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.850 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.850 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "9283", 2024-09-21T06:08:42.850 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.850 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.850 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.850 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T06:08:42.851 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.851 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.851 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.851 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.851 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:42.851 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:42.851 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.851 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.851 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.851 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.851 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.851 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.851 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.851 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.852 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.852 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.852 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.852 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.852 INFO:teuthology.orchestra.run.smithi117.stdout: "stale_cache_strategy": { 2024-09-21T06:08:42.852 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "stale_cache_strategy", 2024-09-21T06:08:42.852 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.852 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.852 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.852 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "log", 2024-09-21T06:08:42.852 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.852 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.852 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.852 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.853 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.853 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.853 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.853 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.853 INFO:teuthology.orchestra.run.smithi117.stdout: "standby_behaviour": { 2024-09-21T06:08:42.853 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "standby_behaviour", 2024-09-21T06:08:42.853 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.853 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.853 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.853 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "default", 2024-09-21T06:08:42.853 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.853 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.853 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.853 INFO:teuthology.orchestra.run.smithi117.stdout: "default", 2024-09-21T06:08:42.853 INFO:teuthology.orchestra.run.smithi117.stdout: "error" 2024-09-21T06:08:42.854 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.854 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.854 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.854 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.854 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.854 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.854 INFO:teuthology.orchestra.run.smithi117.stdout: "standby_error_status_code": { 2024-09-21T06:08:42.854 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "standby_error_status_code", 2024-09-21T06:08:42.854 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.854 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.854 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.854 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "500", 2024-09-21T06:08:42.854 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "400", 2024-09-21T06:08:42.854 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "599", 2024-09-21T06:08:42.855 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.855 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.855 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.855 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.855 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.855 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.855 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.855 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.855 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.855 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rbd_support", 2024-09-21T06:08:42.855 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:42.855 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:42.855 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.855 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.855 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.856 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.856 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.856 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.856 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.856 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.856 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.856 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.856 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.856 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.856 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.856 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.856 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.856 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.856 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.856 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.856 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.857 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.857 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.857 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.857 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.857 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.857 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.857 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.857 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.857 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.857 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.857 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.857 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.857 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.857 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.857 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.858 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.858 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.858 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.858 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.858 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.858 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.858 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.858 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.858 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.858 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.858 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.858 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.858 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.858 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.859 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.859 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.859 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.859 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.859 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.859 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.859 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.859 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.859 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.859 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.859 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.859 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.859 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.859 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.859 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.860 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.860 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.860 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.860 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.860 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.860 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.860 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.860 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.860 INFO:teuthology.orchestra.run.smithi117.stdout: "max_concurrent_snap_create": { 2024-09-21T06:08:42.860 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "max_concurrent_snap_create", 2024-09-21T06:08:42.860 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.860 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.860 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.860 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-09-21T06:08:42.860 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.861 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.861 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.861 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.861 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.861 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.861 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.861 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.861 INFO:teuthology.orchestra.run.smithi117.stdout: "mirror_snapshot_schedule": { 2024-09-21T06:08:42.861 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T06:08:42.861 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.861 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.861 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.861 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.861 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.861 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.862 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.862 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.862 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.862 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.862 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.862 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.862 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:42.862 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:42.862 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.862 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.862 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.862 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.862 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.863 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.863 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.863 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.863 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.863 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.863 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.863 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.863 INFO:teuthology.orchestra.run.smithi117.stdout: "trash_purge_schedule": { 2024-09-21T06:08:42.863 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "trash_purge_schedule", 2024-09-21T06:08:42.863 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.863 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.863 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.863 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.863 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.863 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.864 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.864 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.864 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.864 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.864 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.864 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.864 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.864 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.864 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.864 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "restful", 2024-09-21T06:08:42.864 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:42.864 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:42.864 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.864 INFO:teuthology.orchestra.run.smithi117.stdout: "enable_auth": { 2024-09-21T06:08:42.864 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "enable_auth", 2024-09-21T06:08:42.865 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.865 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.865 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.865 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.865 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.865 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.865 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.865 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.865 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.865 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.865 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.865 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.865 INFO:teuthology.orchestra.run.smithi117.stdout: "key_file": { 2024-09-21T06:08:42.865 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "key_file", 2024-09-21T06:08:42.866 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.866 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.866 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.866 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.866 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.866 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.866 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.866 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.866 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.866 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.866 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.866 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.866 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.866 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.866 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.867 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.867 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.867 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.867 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.867 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.867 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.867 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.867 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.867 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.867 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.867 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.867 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.867 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.867 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.868 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.868 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.868 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.868 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.868 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.868 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.868 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.868 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.868 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.868 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.868 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.868 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.868 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.868 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.868 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.869 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.869 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.869 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.869 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.869 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.869 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.869 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.869 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.869 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.869 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.869 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.869 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.869 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.869 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.869 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.870 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.870 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.870 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.870 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.870 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.870 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.870 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.870 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.870 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.870 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.870 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.870 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.870 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.870 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.871 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.871 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.871 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.871 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.871 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.871 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.871 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.871 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.871 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.871 INFO:teuthology.orchestra.run.smithi117.stdout: "max_requests": { 2024-09-21T06:08:42.871 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "max_requests", 2024-09-21T06:08:42.871 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.871 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.872 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.872 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "500", 2024-09-21T06:08:42.872 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.872 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.872 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.872 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T06:08:42.872 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.872 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.872 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.872 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.872 INFO:teuthology.orchestra.run.smithi117.stdout: "server_addr": { 2024-09-21T06:08:42.872 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_addr", 2024-09-21T06:08:42.872 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.872 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.872 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.873 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.873 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.873 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.873 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.873 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.873 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.873 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.873 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.873 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.873 INFO:teuthology.orchestra.run.smithi117.stdout: "server_port": { 2024-09-21T06:08:42.873 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_port", 2024-09-21T06:08:42.873 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.873 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.873 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.874 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.874 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.874 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.874 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.874 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.874 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.874 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.874 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.874 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.874 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:42.874 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:42.874 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.874 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.874 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.874 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.874 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.875 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.875 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.875 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.875 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.875 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.875 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.875 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.875 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.875 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.875 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.875 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rgw", 2024-09-21T06:08:42.875 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:42.875 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:42.876 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.876 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.876 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.876 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.876 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.876 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.876 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.876 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.876 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.876 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.876 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.876 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.876 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.876 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.877 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.877 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.877 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.877 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.877 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.877 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.877 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.877 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.877 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.877 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.877 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.877 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.877 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.877 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.878 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.878 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.878 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.878 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.878 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.878 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.878 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.878 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.878 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.878 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.878 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.878 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.878 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.878 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.878 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.879 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.879 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.879 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.879 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.879 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.879 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.879 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.879 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.879 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.879 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.879 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.879 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.879 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.879 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.880 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.880 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.880 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.880 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.880 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.880 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.880 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.880 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.880 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.880 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.880 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.880 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.880 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.880 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.880 INFO:teuthology.orchestra.run.smithi117.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T06:08:42.881 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T06:08:42.881 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.881 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.881 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.881 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "5", 2024-09-21T06:08:42.881 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.881 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.881 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.881 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T06:08:42.881 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.881 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.881 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.881 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.881 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:42.881 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:42.882 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.882 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.882 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.882 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.882 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.882 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.882 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.882 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.882 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.882 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.882 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.882 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.882 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.882 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.882 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.883 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rook", 2024-09-21T06:08:42.883 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:42.883 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:42.883 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.883 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.883 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.883 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.883 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.883 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.883 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.883 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.883 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.883 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.883 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.883 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.883 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.884 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.884 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.884 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.884 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.884 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.884 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.884 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.884 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.884 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.884 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.884 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.884 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.884 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.884 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.884 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.885 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.885 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.885 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.885 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.885 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.885 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.885 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.885 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.885 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.885 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.885 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.885 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.885 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.885 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.885 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.885 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.886 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.886 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.886 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.886 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.886 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.886 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.886 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.886 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.886 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.886 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.886 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.886 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.886 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.886 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.886 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.887 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.887 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.887 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.887 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.887 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.887 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.887 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.887 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.887 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.887 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.887 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.887 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.887 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:42.887 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:42.887 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.888 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.888 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.888 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.888 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.888 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.888 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.888 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.888 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.888 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.888 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.888 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.888 INFO:teuthology.orchestra.run.smithi117.stdout: "storage_class": { 2024-09-21T06:08:42.888 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "storage_class", 2024-09-21T06:08:42.888 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.889 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.889 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.889 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "local", 2024-09-21T06:08:42.889 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.889 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.889 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.889 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T06:08:42.889 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.889 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.889 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.889 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.889 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.889 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.889 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.889 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "selftest", 2024-09-21T06:08:42.890 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:42.890 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:42.890 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.890 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.890 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.890 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.890 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.890 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.890 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.890 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.890 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.890 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.890 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.890 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.891 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.891 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.891 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.891 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.891 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.891 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.891 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.891 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.891 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.891 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.891 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.891 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.891 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.891 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.891 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.892 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.892 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.892 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.892 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.892 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.892 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.892 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.892 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.892 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.892 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.892 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.892 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.892 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.892 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.893 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.893 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.893 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.893 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.893 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.893 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.893 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.893 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.893 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.893 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.893 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.893 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.893 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.893 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.893 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.894 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.894 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.894 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.894 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.894 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.894 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.894 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.894 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.894 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.894 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.894 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.894 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.894 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.894 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.895 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.895 INFO:teuthology.orchestra.run.smithi117.stdout: "roption1": { 2024-09-21T06:08:42.895 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "roption1", 2024-09-21T06:08:42.895 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.895 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.895 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.895 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.895 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.895 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.895 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.895 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.895 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.895 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.895 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.896 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.896 INFO:teuthology.orchestra.run.smithi117.stdout: "roption2": { 2024-09-21T06:08:42.896 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "roption2", 2024-09-21T06:08:42.896 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.896 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.896 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.896 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "xyz", 2024-09-21T06:08:42.896 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.896 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.896 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.896 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.896 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.896 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.896 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.896 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.896 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption1": { 2024-09-21T06:08:42.897 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption1", 2024-09-21T06:08:42.897 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.897 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.897 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.897 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.897 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.897 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.897 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.897 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.897 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.897 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.897 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.897 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.897 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption2": { 2024-09-21T06:08:42.897 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption2", 2024-09-21T06:08:42.898 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.898 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.898 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.898 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.898 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.898 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.898 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.898 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.898 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.898 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.898 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.898 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.898 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption3": { 2024-09-21T06:08:42.898 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption3", 2024-09-21T06:08:42.898 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-09-21T06:08:42.899 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.899 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.899 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.899 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.899 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.899 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.899 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.899 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.899 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.899 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.899 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.899 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption4": { 2024-09-21T06:08:42.899 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption4", 2024-09-21T06:08:42.899 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.899 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.900 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.900 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.900 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.900 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.900 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.900 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.900 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.900 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.900 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.900 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.900 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption5": { 2024-09-21T06:08:42.900 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption5", 2024-09-21T06:08:42.900 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.900 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.900 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.901 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.901 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.901 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.901 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.901 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.901 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.901 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.901 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.901 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.901 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption6": { 2024-09-21T06:08:42.901 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption6", 2024-09-21T06:08:42.901 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.901 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.902 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.902 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.902 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.902 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.902 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.902 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.902 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.902 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.902 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.902 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.902 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption7": { 2024-09-21T06:08:42.902 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption7", 2024-09-21T06:08:42.902 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.902 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.903 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.903 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.903 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "1", 2024-09-21T06:08:42.903 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "42", 2024-09-21T06:08:42.903 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.903 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.903 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.903 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.903 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.903 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.903 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:42.903 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:42.903 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.903 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.904 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.904 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.904 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.904 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.904 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.904 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.904 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.904 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.904 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.904 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.904 INFO:teuthology.orchestra.run.smithi117.stdout: "testkey": { 2024-09-21T06:08:42.904 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "testkey", 2024-09-21T06:08:42.904 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.904 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.904 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.905 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.905 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.905 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.905 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.905 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.905 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.905 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.905 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.905 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.905 INFO:teuthology.orchestra.run.smithi117.stdout: "testlkey": { 2024-09-21T06:08:42.905 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "testlkey", 2024-09-21T06:08:42.905 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.905 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.905 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.905 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.905 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.906 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.906 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.906 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.906 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.906 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.906 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.906 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.906 INFO:teuthology.orchestra.run.smithi117.stdout: "testnewline": { 2024-09-21T06:08:42.906 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "testnewline", 2024-09-21T06:08:42.906 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.906 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.906 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.906 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.906 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.907 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.907 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.907 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.907 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.907 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.907 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.907 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.907 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.907 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.907 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.907 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smb", 2024-09-21T06:08:42.907 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:42.907 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:42.907 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.908 INFO:teuthology.orchestra.run.smithi117.stdout: "internal_store_backend": { 2024-09-21T06:08:42.908 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "internal_store_backend", 2024-09-21T06:08:42.908 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.908 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.908 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.908 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.908 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.908 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.908 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.908 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T06:08:42.908 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.908 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.908 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.908 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.909 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.909 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.909 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.909 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.909 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.909 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.909 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.909 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.909 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.909 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.909 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.909 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.909 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.909 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.909 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.910 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.910 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.910 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.910 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.910 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.910 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.910 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.910 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.910 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.910 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.910 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.910 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.910 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.910 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.911 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.911 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.911 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.911 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.911 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.911 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.911 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.911 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.911 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.911 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.911 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.911 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.911 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.911 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.911 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.911 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.912 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.912 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.912 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.912 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.912 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.912 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.912 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.912 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.912 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.912 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.912 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.912 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.912 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.912 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.912 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.913 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.913 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.913 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.913 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.913 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.913 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.913 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.913 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.913 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.913 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.913 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:42.913 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:42.913 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.913 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.913 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.914 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.914 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.914 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.914 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.914 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.914 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.914 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.914 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.914 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.914 INFO:teuthology.orchestra.run.smithi117.stdout: "update_orchestration": { 2024-09-21T06:08:42.914 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "update_orchestration", 2024-09-21T06:08:42.914 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.914 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.914 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.915 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.915 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.915 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.915 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.915 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T06:08:42.915 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.915 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.915 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.915 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.915 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.915 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.915 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.915 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "snap_schedule", 2024-09-21T06:08:42.915 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:42.915 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:42.916 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.916 INFO:teuthology.orchestra.run.smithi117.stdout: "allow_m_granularity": { 2024-09-21T06:08:42.916 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "allow_m_granularity", 2024-09-21T06:08:42.916 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.916 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.916 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.916 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.916 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.916 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.916 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.916 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T06:08:42.916 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.916 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.916 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.917 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.917 INFO:teuthology.orchestra.run.smithi117.stdout: "dump_on_update": { 2024-09-21T06:08:42.917 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "dump_on_update", 2024-09-21T06:08:42.917 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.917 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.917 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.917 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.917 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.917 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.917 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.917 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "dump database to debug log on update", 2024-09-21T06:08:42.917 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.917 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.917 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.917 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.917 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.918 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.918 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.918 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.918 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.918 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.918 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.918 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.918 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.918 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.918 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.918 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.918 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.918 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.918 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.918 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.919 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.919 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.919 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.919 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.919 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.919 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.919 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.919 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.919 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.919 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.919 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.919 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.919 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.919 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.919 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.920 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.920 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.920 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.920 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.920 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.920 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.920 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.920 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.920 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.920 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.920 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.920 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.920 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.920 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.920 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.921 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.921 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.921 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.921 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.921 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.921 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.921 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.921 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.921 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.921 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.921 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.921 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.921 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.921 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.921 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.922 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.922 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.922 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.922 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.922 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.922 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.922 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.922 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.922 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.922 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:42.922 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:42.922 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.922 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.922 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.922 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.923 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.923 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.923 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.923 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.923 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.923 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.923 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.923 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.923 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.923 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.923 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.923 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "stats", 2024-09-21T06:08:42.923 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:42.923 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:42.923 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.924 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.924 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.924 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.924 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.924 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.924 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.924 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.924 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.924 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.924 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.924 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.924 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.924 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.924 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.924 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.924 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.925 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.925 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.925 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.925 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.925 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.925 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.925 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.925 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.925 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.925 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.925 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.925 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.925 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.925 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.925 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.926 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.926 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.926 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.926 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.926 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.926 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.926 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.926 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.926 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.926 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.926 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.926 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.926 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.926 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.927 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.927 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.927 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.927 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.927 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.927 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.927 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.927 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.927 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.927 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.927 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.927 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.927 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.928 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.928 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.928 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.928 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.928 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.928 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.928 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.928 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.928 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.928 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.928 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.928 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.928 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:42.928 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:42.929 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.929 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.929 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.929 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.929 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.929 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.929 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.929 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.929 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.929 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.929 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.929 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.929 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.929 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.930 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.930 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "status", 2024-09-21T06:08:42.930 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:42.930 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:42.930 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.930 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.930 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.930 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.930 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.930 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.930 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.930 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.930 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.930 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.930 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.931 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.931 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.931 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.931 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.931 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.931 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.931 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.931 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.931 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.931 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.931 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.931 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.931 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.931 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.932 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.932 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.932 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.932 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.932 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.932 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.932 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.932 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.932 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.932 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.932 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.932 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.932 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.932 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.932 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.932 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.933 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.933 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.933 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.933 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.933 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.933 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.933 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.933 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.933 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.933 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.933 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.933 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.933 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.933 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.933 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.934 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.934 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.934 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.934 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.934 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.934 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.934 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.934 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.934 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.934 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.934 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.934 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.934 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.934 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.934 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.935 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:42.935 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:42.935 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.935 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.935 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.935 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.935 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.935 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.935 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.935 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.935 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.935 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.935 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.935 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.935 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.935 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.936 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.936 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "telegraf", 2024-09-21T06:08:42.936 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:42.936 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:42.936 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.936 INFO:teuthology.orchestra.run.smithi117.stdout: "address": { 2024-09-21T06:08:42.936 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "address", 2024-09-21T06:08:42.936 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.936 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.936 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.936 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T06:08:42.936 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.936 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.936 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.936 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.937 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.937 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.937 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.937 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.937 INFO:teuthology.orchestra.run.smithi117.stdout: "interval": { 2024-09-21T06:08:42.937 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "interval", 2024-09-21T06:08:42.937 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:42.937 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.937 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.937 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "15", 2024-09-21T06:08:42.937 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.937 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.937 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.937 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.937 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.938 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.938 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.938 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.938 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.938 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.938 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.938 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.938 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.938 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.938 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.938 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.938 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.938 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.938 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.938 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.939 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.939 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.939 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.939 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.939 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.939 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.939 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.939 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.939 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.939 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.939 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.939 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.939 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.939 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.939 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.940 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.940 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.940 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.940 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.940 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.940 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.940 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.940 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.940 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.940 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.940 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.940 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.940 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.940 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.941 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.941 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.941 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.941 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.941 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.941 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.941 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.941 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.941 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.941 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.941 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.941 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.941 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.941 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.941 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.942 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.942 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.942 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.942 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.942 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.942 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.942 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.942 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.942 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.942 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.942 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.942 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.942 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.942 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.943 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:42.943 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:42.943 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.943 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.943 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.943 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.943 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.943 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.943 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.943 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.943 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.943 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.943 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.943 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.943 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.944 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.944 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.944 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "telemetry", 2024-09-21T06:08:42.944 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:42.944 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:42.944 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.944 INFO:teuthology.orchestra.run.smithi117.stdout: "channel_basic": { 2024-09-21T06:08:42.944 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "channel_basic", 2024-09-21T06:08:42.944 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.944 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.944 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.944 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.944 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.944 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.944 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.945 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T06:08:42.945 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.945 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.945 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.945 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.945 INFO:teuthology.orchestra.run.smithi117.stdout: "channel_crash": { 2024-09-21T06:08:42.945 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "channel_crash", 2024-09-21T06:08:42.945 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.945 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.945 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.945 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.945 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.945 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.945 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.946 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T06:08:42.946 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.946 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.946 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.946 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.946 INFO:teuthology.orchestra.run.smithi117.stdout: "channel_device": { 2024-09-21T06:08:42.946 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "channel_device", 2024-09-21T06:08:42.946 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.946 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.946 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.946 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.946 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.946 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.946 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.947 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T06:08:42.947 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.947 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.947 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.947 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.947 INFO:teuthology.orchestra.run.smithi117.stdout: "channel_ident": { 2024-09-21T06:08:42.947 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "channel_ident", 2024-09-21T06:08:42.947 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.947 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.947 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.947 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.947 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.947 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.947 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.947 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T06:08:42.948 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.948 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.948 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.948 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.948 INFO:teuthology.orchestra.run.smithi117.stdout: "channel_perf": { 2024-09-21T06:08:42.948 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "channel_perf", 2024-09-21T06:08:42.948 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.948 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.948 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.948 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.948 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.948 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.948 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.948 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T06:08:42.949 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.949 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.949 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.949 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.949 INFO:teuthology.orchestra.run.smithi117.stdout: "contact": { 2024-09-21T06:08:42.949 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "contact", 2024-09-21T06:08:42.949 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.949 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.949 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.949 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.949 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.949 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.949 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.949 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.949 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.949 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.950 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.950 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.950 INFO:teuthology.orchestra.run.smithi117.stdout: "description": { 2024-09-21T06:08:42.950 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "description", 2024-09-21T06:08:42.950 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.950 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.950 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.950 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.950 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.950 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.950 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.950 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.950 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.951 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.951 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.951 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.951 INFO:teuthology.orchestra.run.smithi117.stdout: "device_url": { 2024-09-21T06:08:42.951 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "device_url", 2024-09-21T06:08:42.951 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.951 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.951 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.951 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T06:08:42.951 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.951 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.951 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.951 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.951 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.951 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.951 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.952 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.952 INFO:teuthology.orchestra.run.smithi117.stdout: "enabled": { 2024-09-21T06:08:42.952 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "enabled", 2024-09-21T06:08:42.952 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.952 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.952 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.952 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.952 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.952 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.952 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.952 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.952 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.952 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.952 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.952 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.953 INFO:teuthology.orchestra.run.smithi117.stdout: "interval": { 2024-09-21T06:08:42.953 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "interval", 2024-09-21T06:08:42.953 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.953 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.953 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.953 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "24", 2024-09-21T06:08:42.953 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "8", 2024-09-21T06:08:42.953 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.953 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.953 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.953 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.953 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.953 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.953 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.954 INFO:teuthology.orchestra.run.smithi117.stdout: "last_opt_revision": { 2024-09-21T06:08:42.954 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "last_opt_revision", 2024-09-21T06:08:42.954 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.954 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.954 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.954 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "1", 2024-09-21T06:08:42.954 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.954 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.954 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.954 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.954 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.954 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.954 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.954 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.954 INFO:teuthology.orchestra.run.smithi117.stdout: "leaderboard": { 2024-09-21T06:08:42.955 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "leaderboard", 2024-09-21T06:08:42.955 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.955 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.955 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.955 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.955 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.955 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.955 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.955 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.955 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.955 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.955 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.955 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.955 INFO:teuthology.orchestra.run.smithi117.stdout: "leaderboard_description": { 2024-09-21T06:08:42.956 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "leaderboard_description", 2024-09-21T06:08:42.956 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.956 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.956 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.956 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.956 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.956 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.956 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.956 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.956 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.956 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.956 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.956 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.956 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.956 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.956 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.957 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.957 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.957 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.957 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.957 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.957 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.957 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.957 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.957 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.957 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.957 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.957 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.957 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.957 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.957 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.958 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.958 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.958 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.958 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.958 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.958 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.958 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.958 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.958 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.958 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.958 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.958 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.958 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.958 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.958 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.959 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.959 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.959 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.959 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.959 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.959 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.959 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.959 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.959 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.959 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.959 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.959 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.959 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.959 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.960 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.960 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.960 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.960 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.960 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.960 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.960 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.960 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.960 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.960 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.960 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.960 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.960 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.960 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.960 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.960 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.961 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.961 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.961 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.961 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.961 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.961 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.961 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.961 INFO:teuthology.orchestra.run.smithi117.stdout: "organization": { 2024-09-21T06:08:42.961 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "organization", 2024-09-21T06:08:42.961 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.961 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.961 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.961 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.961 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.961 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.962 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.962 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.962 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.962 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.962 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.962 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.962 INFO:teuthology.orchestra.run.smithi117.stdout: "proxy": { 2024-09-21T06:08:42.962 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "proxy", 2024-09-21T06:08:42.962 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.962 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.962 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.962 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.962 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.962 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.962 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.963 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.963 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.963 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.963 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.963 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.963 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:42.963 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:42.963 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.963 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.963 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.963 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.963 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.963 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.963 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.963 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.963 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.964 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.964 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.964 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.964 INFO:teuthology.orchestra.run.smithi117.stdout: "url": { 2024-09-21T06:08:42.964 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "url", 2024-09-21T06:08:42.964 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.964 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.964 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.964 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T06:08:42.964 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.964 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.964 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.964 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.964 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.964 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.965 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.965 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.965 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.965 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.965 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.965 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "test_orchestrator", 2024-09-21T06:08:42.965 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:42.965 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:42.965 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.965 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.965 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.965 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.965 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.965 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.965 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.965 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.966 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.966 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.966 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.966 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.966 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.966 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.966 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.966 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.966 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.966 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.966 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.966 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.966 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.966 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.967 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.967 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.967 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.967 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.967 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.967 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.967 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.967 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.967 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.967 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.967 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.967 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.967 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.967 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.968 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.968 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.968 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.968 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.968 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.968 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.968 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.968 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.968 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.968 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.968 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.968 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.968 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.968 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.968 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.969 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.969 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.969 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.969 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.969 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.969 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.969 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.969 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.969 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.969 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.969 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.969 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.969 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.969 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.969 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.970 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.970 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.970 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.970 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.970 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.970 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:42.970 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:42.970 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.970 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.970 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.970 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.970 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.970 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.970 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.970 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.971 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.971 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.971 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.971 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.971 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.971 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.971 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.971 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "volumes", 2024-09-21T06:08:42.971 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:42.971 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:42.971 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.971 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.971 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.972 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.972 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.972 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.972 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.972 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.972 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.972 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.972 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.972 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.972 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.972 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.972 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.972 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.972 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.972 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.973 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.973 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.973 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.973 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.973 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.973 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.973 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.973 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.973 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.973 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.973 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.973 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.973 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.973 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.973 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.973 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.974 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.974 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.974 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.974 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.974 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.974 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.974 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.974 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.974 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.974 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.974 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.974 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.974 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.974 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.975 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.975 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.975 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.975 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.975 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.975 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.975 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.975 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.975 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.975 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.975 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.975 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.975 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.975 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.975 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.976 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.976 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.976 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.976 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.976 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.976 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.976 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.976 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.976 INFO:teuthology.orchestra.run.smithi117.stdout: "max_concurrent_clones": { 2024-09-21T06:08:42.976 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "max_concurrent_clones", 2024-09-21T06:08:42.976 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.976 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.976 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.976 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "4", 2024-09-21T06:08:42.977 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.977 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.977 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.977 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T06:08:42.977 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.977 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.977 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.977 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.977 INFO:teuthology.orchestra.run.smithi117.stdout: "periodic_async_work": { 2024-09-21T06:08:42.977 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "periodic_async_work", 2024-09-21T06:08:42.977 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.977 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.977 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.977 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.977 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.978 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.978 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.978 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Periodically check for async work", 2024-09-21T06:08:42.978 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.978 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.978 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.978 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.978 INFO:teuthology.orchestra.run.smithi117.stdout: "snapshot_clone_delay": { 2024-09-21T06:08:42.978 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "snapshot_clone_delay", 2024-09-21T06:08:42.978 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.978 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.978 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.978 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.978 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.978 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.979 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.979 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T06:08:42.979 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.979 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.979 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.979 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.979 INFO:teuthology.orchestra.run.smithi117.stdout: "snapshot_clone_no_wait": { 2024-09-21T06:08:42.979 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T06:08:42.979 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.979 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.979 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.979 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:42.980 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.980 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.980 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.980 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T06:08:42.980 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.980 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.980 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.980 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.980 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:42.980 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:42.980 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.980 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.980 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.980 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.981 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.981 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.981 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.981 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.981 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.981 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.981 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.981 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.981 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.981 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.981 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.981 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "zabbix", 2024-09-21T06:08:42.981 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:42.981 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:42.981 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.982 INFO:teuthology.orchestra.run.smithi117.stdout: "discovery_interval": { 2024-09-21T06:08:42.982 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "discovery_interval", 2024-09-21T06:08:42.982 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "uint", 2024-09-21T06:08:42.982 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.982 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.982 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "100", 2024-09-21T06:08:42.982 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.982 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.982 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.982 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.982 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.982 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.982 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.982 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.983 INFO:teuthology.orchestra.run.smithi117.stdout: "identifier": { 2024-09-21T06:08:42.983 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "identifier", 2024-09-21T06:08:42.983 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.983 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.983 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.983 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.983 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.983 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.983 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.983 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.983 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.983 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.983 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.983 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.983 INFO:teuthology.orchestra.run.smithi117.stdout: "interval": { 2024-09-21T06:08:42.984 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "interval", 2024-09-21T06:08:42.984 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:42.984 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.984 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.984 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "60", 2024-09-21T06:08:42.984 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.984 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.984 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.984 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.984 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.984 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.984 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.984 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.984 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.985 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.985 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.985 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.985 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.985 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.985 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.985 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.985 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.985 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.985 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.985 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.985 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.985 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.985 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.986 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.986 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.986 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.986 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.986 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.986 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.986 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.986 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.986 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.986 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.986 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.986 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.986 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.986 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.986 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.987 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.987 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.987 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.987 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.987 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.987 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.987 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.987 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.987 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.987 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.987 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.987 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.987 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.987 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.988 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.988 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.988 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.988 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.988 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.988 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.988 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.988 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.988 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.988 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.988 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.988 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.988 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.988 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.988 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.988 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.989 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.989 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.989 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.989 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.989 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.989 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.989 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.989 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.989 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.989 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.989 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:42.989 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:42.989 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.989 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:42.989 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.990 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:42.990 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.990 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.990 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.990 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.990 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.990 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.990 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.990 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.990 INFO:teuthology.orchestra.run.smithi117.stdout: "zabbix_host": { 2024-09-21T06:08:42.990 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "zabbix_host", 2024-09-21T06:08:42.990 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.990 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.990 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.990 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.991 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.991 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.991 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.991 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.991 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.991 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.991 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.991 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.991 INFO:teuthology.orchestra.run.smithi117.stdout: "zabbix_port": { 2024-09-21T06:08:42.991 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "zabbix_port", 2024-09-21T06:08:42.991 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:42.991 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.991 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.991 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10051", 2024-09-21T06:08:42.991 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.991 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.992 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.992 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.992 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.992 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.992 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.992 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.992 INFO:teuthology.orchestra.run.smithi117.stdout: "zabbix_sender": { 2024-09-21T06:08:42.992 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "zabbix_sender", 2024-09-21T06:08:42.992 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.992 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.992 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:42.992 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T06:08:42.992 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.992 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.993 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.993 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.993 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.993 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.993 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.993 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.993 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.993 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:42.993 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-21T06:08:42.993 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.993 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.993 INFO:teuthology.orchestra.run.smithi117.stdout: "gid": 24140, 2024-09-21T06:08:42.993 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "y", 2024-09-21T06:08:42.993 INFO:teuthology.orchestra.run.smithi117.stdout: "mgr_features": 4540719139924082687, 2024-09-21T06:08:42.993 INFO:teuthology.orchestra.run.smithi117.stdout: "available_modules": [ 2024-09-21T06:08:42.994 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:42.994 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "alerts", 2024-09-21T06:08:42.994 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:42.994 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:42.994 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:42.994 INFO:teuthology.orchestra.run.smithi117.stdout: "interval": { 2024-09-21T06:08:42.994 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "interval", 2024-09-21T06:08:42.994 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:42.994 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.994 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.994 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "60", 2024-09-21T06:08:42.994 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.994 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.994 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.994 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T06:08:42.995 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.995 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.995 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.995 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.995 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:42.995 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:42.995 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.995 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.995 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.995 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:42.995 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.995 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.995 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.995 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.996 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.996 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.996 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.996 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.996 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.996 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.996 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.996 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.996 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.996 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.996 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.996 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:42.996 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:42.996 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.996 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.997 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.997 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.997 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.997 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.997 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.997 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.997 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.997 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.997 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.997 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.997 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:42.997 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:42.997 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:42.997 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.997 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.997 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:42.998 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.998 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.998 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:42.998 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:42.998 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:42.998 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:42.998 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:42.998 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:42.998 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:42.998 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:42.998 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.998 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.998 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.998 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.998 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.999 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:42.999 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:42.999 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:42.999 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:42.999 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:42.999 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:42.999 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:42.999 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:42.999 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:42.999 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:42.999 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:42.999 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:42.999 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:42.999 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:42.999 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_destination": { 2024-09-21T06:08:42.999 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_destination", 2024-09-21T06:08:43.000 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.000 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.000 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.000 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.000 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.000 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.000 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.000 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T06:08:43.000 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.000 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.000 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.000 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.000 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_from_name": { 2024-09-21T06:08:43.000 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_from_name", 2024-09-21T06:08:43.000 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.000 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.001 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.001 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "Ceph", 2024-09-21T06:08:43.001 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.001 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.001 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.001 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Email From: name", 2024-09-21T06:08:43.001 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.001 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.001 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.001 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.001 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_host": { 2024-09-21T06:08:43.001 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_host", 2024-09-21T06:08:43.001 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.001 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.001 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.001 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.002 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.002 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.002 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.002 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "SMTP server", 2024-09-21T06:08:43.002 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.002 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.002 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.002 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.002 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_password": { 2024-09-21T06:08:43.002 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_password", 2024-09-21T06:08:43.002 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.002 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.002 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.002 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.003 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.003 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.003 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.003 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Password to authenticate with", 2024-09-21T06:08:43.003 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.003 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.003 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.003 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.003 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_port": { 2024-09-21T06:08:43.003 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_port", 2024-09-21T06:08:43.003 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.003 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.003 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.003 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "465", 2024-09-21T06:08:43.003 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.004 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.004 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.004 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "SMTP port", 2024-09-21T06:08:43.004 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.004 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.004 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.004 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.004 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_sender": { 2024-09-21T06:08:43.004 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_sender", 2024-09-21T06:08:43.004 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.004 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.004 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.004 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.004 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.004 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.005 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.005 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "SMTP envelope sender", 2024-09-21T06:08:43.005 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.005 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.005 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.005 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.005 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_ssl": { 2024-09-21T06:08:43.005 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_ssl", 2024-09-21T06:08:43.005 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.005 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.005 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.005 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.005 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.005 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.006 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.006 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T06:08:43.006 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.006 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.006 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.006 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.006 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_user": { 2024-09-21T06:08:43.006 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_user", 2024-09-21T06:08:43.006 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.006 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.006 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.006 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.006 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.006 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.007 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.007 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "User to authenticate as", 2024-09-21T06:08:43.007 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.007 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.007 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.007 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.007 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.007 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.007 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.007 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.007 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.007 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.007 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.007 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.007 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.008 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.008 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.008 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.008 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.008 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.008 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.008 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.008 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.008 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "balancer", 2024-09-21T06:08:43.008 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.008 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.008 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.008 INFO:teuthology.orchestra.run.smithi117.stdout: "active": { 2024-09-21T06:08:43.008 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "active", 2024-09-21T06:08:43.008 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.009 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.009 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.009 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.009 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.009 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.009 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.009 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T06:08:43.009 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.009 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.009 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.009 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.009 INFO:teuthology.orchestra.run.smithi117.stdout: "begin_time": { 2024-09-21T06:08:43.009 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "begin_time", 2024-09-21T06:08:43.009 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.009 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.010 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.010 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0000", 2024-09-21T06:08:43.010 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.010 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.010 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.010 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T06:08:43.010 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T06:08:43.010 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.010 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.010 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.010 INFO:teuthology.orchestra.run.smithi117.stdout: "begin_weekday": { 2024-09-21T06:08:43.010 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "begin_weekday", 2024-09-21T06:08:43.010 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "uint", 2024-09-21T06:08:43.010 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.011 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.011 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.011 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "0", 2024-09-21T06:08:43.011 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "6", 2024-09-21T06:08:43.011 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.011 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T06:08:43.011 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T06:08:43.011 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.011 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.011 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.011 INFO:teuthology.orchestra.run.smithi117.stdout: "crush_compat_max_iterations": { 2024-09-21T06:08:43.011 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crush_compat_max_iterations", 2024-09-21T06:08:43.011 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "uint", 2024-09-21T06:08:43.011 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.011 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.011 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "25", 2024-09-21T06:08:43.012 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "1", 2024-09-21T06:08:43.012 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "250", 2024-09-21T06:08:43.012 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.012 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T06:08:43.012 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.012 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.012 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.012 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.012 INFO:teuthology.orchestra.run.smithi117.stdout: "crush_compat_metrics": { 2024-09-21T06:08:43.012 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crush_compat_metrics", 2024-09-21T06:08:43.012 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.012 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.012 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.012 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T06:08:43.012 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.013 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.013 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.013 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T06:08:43.013 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T06:08:43.013 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.013 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.013 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.013 INFO:teuthology.orchestra.run.smithi117.stdout: "crush_compat_step": { 2024-09-21T06:08:43.013 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crush_compat_step", 2024-09-21T06:08:43.013 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-09-21T06:08:43.013 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.013 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.013 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0.5", 2024-09-21T06:08:43.013 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "0.001", 2024-09-21T06:08:43.013 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "0.999", 2024-09-21T06:08:43.013 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.014 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "aggressiveness of optimization", 2024-09-21T06:08:43.014 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T06:08:43.014 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.014 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.014 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.014 INFO:teuthology.orchestra.run.smithi117.stdout: "end_time": { 2024-09-21T06:08:43.014 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "end_time", 2024-09-21T06:08:43.014 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.014 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.014 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.014 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "2359", 2024-09-21T06:08:43.014 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.014 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.014 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.014 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T06:08:43.015 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T06:08:43.015 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.015 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.015 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.015 INFO:teuthology.orchestra.run.smithi117.stdout: "end_weekday": { 2024-09-21T06:08:43.015 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "end_weekday", 2024-09-21T06:08:43.015 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "uint", 2024-09-21T06:08:43.015 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.015 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.015 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.015 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "0", 2024-09-21T06:08:43.015 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "6", 2024-09-21T06:08:43.015 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.015 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T06:08:43.015 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T06:08:43.016 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.016 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.016 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.016 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.016 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.016 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.016 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.016 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.016 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.016 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.016 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.016 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.016 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.016 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.016 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.016 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.017 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.017 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.017 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.017 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.017 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.017 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.017 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.017 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.017 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.017 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.017 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.017 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.017 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.017 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.018 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.018 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.018 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.018 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.018 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.018 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.018 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.018 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.018 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.018 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.018 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.018 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.018 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.018 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.019 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.019 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.019 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.019 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.019 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.019 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.019 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.019 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.019 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.019 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.019 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.019 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.019 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.019 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.019 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.020 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.020 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.020 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.020 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.020 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.020 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.020 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.020 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.020 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.020 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.020 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.020 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.020 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.020 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.021 INFO:teuthology.orchestra.run.smithi117.stdout: "min_score": { 2024-09-21T06:08:43.021 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "min_score", 2024-09-21T06:08:43.021 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-09-21T06:08:43.021 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.021 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.021 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.021 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.021 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.021 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.021 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T06:08:43.021 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.021 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.021 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.021 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.022 INFO:teuthology.orchestra.run.smithi117.stdout: "mode": { 2024-09-21T06:08:43.022 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mode", 2024-09-21T06:08:43.022 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.022 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.022 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.022 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "upmap", 2024-09-21T06:08:43.022 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.022 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.022 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.022 INFO:teuthology.orchestra.run.smithi117.stdout: "crush-compat", 2024-09-21T06:08:43.022 INFO:teuthology.orchestra.run.smithi117.stdout: "none", 2024-09-21T06:08:43.022 INFO:teuthology.orchestra.run.smithi117.stdout: "read", 2024-09-21T06:08:43.022 INFO:teuthology.orchestra.run.smithi117.stdout: "upmap", 2024-09-21T06:08:43.022 INFO:teuthology.orchestra.run.smithi117.stdout: "upmap-read" 2024-09-21T06:08:43.022 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.023 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Balancer mode", 2024-09-21T06:08:43.023 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.023 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.023 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.023 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.023 INFO:teuthology.orchestra.run.smithi117.stdout: "pool_ids": { 2024-09-21T06:08:43.023 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "pool_ids", 2024-09-21T06:08:43.023 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.023 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.023 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.023 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.023 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.023 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.023 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.024 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T06:08:43.024 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.024 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.024 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.024 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.024 INFO:teuthology.orchestra.run.smithi117.stdout: "sleep_interval": { 2024-09-21T06:08:43.024 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sleep_interval", 2024-09-21T06:08:43.024 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.024 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.024 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.024 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "60", 2024-09-21T06:08:43.024 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.024 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.024 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.024 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T06:08:43.024 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.025 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.025 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.025 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.025 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.025 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.025 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.025 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.025 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.025 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.025 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.025 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.025 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.025 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.025 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.025 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.026 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.026 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.026 INFO:teuthology.orchestra.run.smithi117.stdout: "upmap_max_deviation": { 2024-09-21T06:08:43.026 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "upmap_max_deviation", 2024-09-21T06:08:43.026 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.026 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.026 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.026 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "5", 2024-09-21T06:08:43.026 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "1", 2024-09-21T06:08:43.026 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.026 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.026 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T06:08:43.026 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T06:08:43.026 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.027 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.027 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.027 INFO:teuthology.orchestra.run.smithi117.stdout: "upmap_max_optimizations": { 2024-09-21T06:08:43.027 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "upmap_max_optimizations", 2024-09-21T06:08:43.027 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "uint", 2024-09-21T06:08:43.027 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.027 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.027 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-09-21T06:08:43.027 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.027 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.027 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.027 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T06:08:43.027 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.027 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.027 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.028 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.028 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.028 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.028 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.028 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "cephadm", 2024-09-21T06:08:43.028 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.028 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.028 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.028 INFO:teuthology.orchestra.run.smithi117.stdout: "agent_down_multiplier": { 2024-09-21T06:08:43.028 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "agent_down_multiplier", 2024-09-21T06:08:43.028 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-09-21T06:08:43.028 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.028 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.028 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "3.0", 2024-09-21T06:08:43.028 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.029 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.029 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.029 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T06:08:43.029 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.029 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.029 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.029 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.029 INFO:teuthology.orchestra.run.smithi117.stdout: "agent_refresh_rate": { 2024-09-21T06:08:43.029 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "agent_refresh_rate", 2024-09-21T06:08:43.029 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.029 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.029 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.029 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "20", 2024-09-21T06:08:43.029 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.030 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.030 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.030 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T06:08:43.030 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.030 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.030 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.030 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.030 INFO:teuthology.orchestra.run.smithi117.stdout: "agent_starting_port": { 2024-09-21T06:08:43.030 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "agent_starting_port", 2024-09-21T06:08:43.030 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.030 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.030 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.030 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "4721", 2024-09-21T06:08:43.030 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.030 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.031 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.031 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T06:08:43.031 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.031 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.031 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.031 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.031 INFO:teuthology.orchestra.run.smithi117.stdout: "allow_ptrace": { 2024-09-21T06:08:43.031 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "allow_ptrace", 2024-09-21T06:08:43.031 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.031 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.031 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.031 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.031 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.031 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.032 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.032 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T06:08:43.032 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T06:08:43.032 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.032 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.032 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.032 INFO:teuthology.orchestra.run.smithi117.stdout: "autotune_interval": { 2024-09-21T06:08:43.032 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "autotune_interval", 2024-09-21T06:08:43.032 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.032 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.032 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.032 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "600", 2024-09-21T06:08:43.032 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.032 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.032 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.033 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T06:08:43.033 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.033 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.033 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.033 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.033 INFO:teuthology.orchestra.run.smithi117.stdout: "autotune_memory_target_ratio": { 2024-09-21T06:08:43.033 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T06:08:43.033 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-09-21T06:08:43.033 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.033 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.033 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0.7", 2024-09-21T06:08:43.033 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.033 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.033 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.034 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T06:08:43.034 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.034 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.034 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.034 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.034 INFO:teuthology.orchestra.run.smithi117.stdout: "cephadm_log_destination": { 2024-09-21T06:08:43.034 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "cephadm_log_destination", 2024-09-21T06:08:43.034 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.034 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.034 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.034 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.034 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.034 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.034 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.035 INFO:teuthology.orchestra.run.smithi117.stdout: "file", 2024-09-21T06:08:43.035 INFO:teuthology.orchestra.run.smithi117.stdout: "file,syslog", 2024-09-21T06:08:43.035 INFO:teuthology.orchestra.run.smithi117.stdout: "syslog" 2024-09-21T06:08:43.035 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.035 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T06:08:43.035 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.035 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.035 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.035 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.035 INFO:teuthology.orchestra.run.smithi117.stdout: "cgroups_split": { 2024-09-21T06:08:43.035 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "cgroups_split", 2024-09-21T06:08:43.035 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.035 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.035 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.035 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.036 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.036 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.036 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.036 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T06:08:43.036 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.036 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.036 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.036 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.036 INFO:teuthology.orchestra.run.smithi117.stdout: "config_checks_enabled": { 2024-09-21T06:08:43.036 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "config_checks_enabled", 2024-09-21T06:08:43.036 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.036 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.036 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.036 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.036 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.037 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.037 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.037 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T06:08:43.037 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.037 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.037 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.037 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.037 INFO:teuthology.orchestra.run.smithi117.stdout: "config_dashboard": { 2024-09-21T06:08:43.037 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "config_dashboard", 2024-09-21T06:08:43.037 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.037 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.037 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.037 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.037 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.037 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.038 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.038 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T06:08:43.038 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.038 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.038 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.038 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.038 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_alertmanager": { 2024-09-21T06:08:43.038 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_alertmanager", 2024-09-21T06:08:43.038 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.038 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.038 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.038 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T06:08:43.038 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.038 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.038 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.039 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Prometheus container image", 2024-09-21T06:08:43.039 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.039 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.039 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.039 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.039 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_base": { 2024-09-21T06:08:43.039 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_base", 2024-09-21T06:08:43.039 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.039 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.039 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.039 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T06:08:43.039 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.039 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.039 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.039 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Container image name, without the tag", 2024-09-21T06:08:43.040 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.040 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.040 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.040 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.040 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_elasticsearch": { 2024-09-21T06:08:43.040 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_elasticsearch", 2024-09-21T06:08:43.040 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.040 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.040 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.040 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T06:08:43.040 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.040 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.040 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.040 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "elasticsearch container image", 2024-09-21T06:08:43.040 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.041 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.041 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.041 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.041 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_grafana": { 2024-09-21T06:08:43.041 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_grafana", 2024-09-21T06:08:43.041 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.041 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.041 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.041 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T06:08:43.041 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.041 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.041 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.041 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Prometheus container image", 2024-09-21T06:08:43.041 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.042 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.042 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.042 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.042 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_haproxy": { 2024-09-21T06:08:43.042 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_haproxy", 2024-09-21T06:08:43.042 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.042 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.042 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.042 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T06:08:43.042 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.042 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.042 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.042 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "HAproxy container image", 2024-09-21T06:08:43.042 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.042 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.043 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.043 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.043 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_jaeger_agent": { 2024-09-21T06:08:43.043 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_jaeger_agent", 2024-09-21T06:08:43.043 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.043 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.043 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.043 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T06:08:43.043 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.043 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.043 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.043 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Jaeger agent container image", 2024-09-21T06:08:43.043 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.043 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.043 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.043 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.044 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_jaeger_collector": { 2024-09-21T06:08:43.044 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_jaeger_collector", 2024-09-21T06:08:43.044 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.044 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.044 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.044 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T06:08:43.044 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.044 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.044 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.044 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Jaeger collector container image", 2024-09-21T06:08:43.044 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.044 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.044 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.044 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.044 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_jaeger_query": { 2024-09-21T06:08:43.045 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_jaeger_query", 2024-09-21T06:08:43.045 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.045 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.045 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.045 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T06:08:43.045 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.045 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.045 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.045 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Jaeger query container image", 2024-09-21T06:08:43.045 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.045 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.045 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.045 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.045 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_keepalived": { 2024-09-21T06:08:43.045 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_keepalived", 2024-09-21T06:08:43.046 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.046 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.046 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.046 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T06:08:43.046 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.046 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.046 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.046 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Keepalived container image", 2024-09-21T06:08:43.046 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.046 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.046 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.046 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.046 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_loki": { 2024-09-21T06:08:43.046 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_loki", 2024-09-21T06:08:43.046 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.047 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.047 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.047 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T06:08:43.047 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.047 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.047 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.047 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Loki container image", 2024-09-21T06:08:43.047 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.047 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.047 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.047 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.047 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_nginx": { 2024-09-21T06:08:43.047 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_nginx", 2024-09-21T06:08:43.047 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.047 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.047 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.048 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T06:08:43.048 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.048 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.048 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.048 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Nginx container image", 2024-09-21T06:08:43.048 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.048 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.048 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.048 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.048 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_node_exporter": { 2024-09-21T06:08:43.048 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_node_exporter", 2024-09-21T06:08:43.048 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.048 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.048 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.048 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T06:08:43.049 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.049 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.049 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.049 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Prometheus container image", 2024-09-21T06:08:43.049 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.049 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.049 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.049 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.049 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_nvmeof": { 2024-09-21T06:08:43.049 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_nvmeof", 2024-09-21T06:08:43.049 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.049 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.049 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.049 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T06:08:43.049 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.050 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.050 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.050 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Nvme-of container image", 2024-09-21T06:08:43.050 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.050 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.050 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.050 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.050 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_oauth2_proxy": { 2024-09-21T06:08:43.050 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T06:08:43.050 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.050 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.050 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.050 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T06:08:43.050 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.051 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.051 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.051 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "oauth2-proxy container image", 2024-09-21T06:08:43.051 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.051 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.051 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.051 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.051 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_prometheus": { 2024-09-21T06:08:43.051 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_prometheus", 2024-09-21T06:08:43.051 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.051 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.051 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.051 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T06:08:43.051 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.051 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.051 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.052 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Prometheus container image", 2024-09-21T06:08:43.052 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.052 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.053 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.053 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.053 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_promtail": { 2024-09-21T06:08:43.053 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_promtail", 2024-09-21T06:08:43.053 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.053 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.054 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.054 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T06:08:43.054 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.054 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.054 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.054 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Promtail container image", 2024-09-21T06:08:43.054 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.054 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.054 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.054 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.054 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_samba": { 2024-09-21T06:08:43.054 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_samba", 2024-09-21T06:08:43.054 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.054 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.055 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.055 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T06:08:43.055 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.055 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.055 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.055 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Samba/SMB container image", 2024-09-21T06:08:43.055 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.055 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.055 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.055 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.055 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_samba_metrics": { 2024-09-21T06:08:43.055 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_samba_metrics", 2024-09-21T06:08:43.055 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.055 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.055 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.055 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T06:08:43.056 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.056 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.056 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.056 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T06:08:43.056 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.056 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.056 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.056 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.056 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_snmp_gateway": { 2024-09-21T06:08:43.056 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_snmp_gateway", 2024-09-21T06:08:43.056 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.056 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.056 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.056 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T06:08:43.057 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.057 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.057 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.057 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "SNMP Gateway container image", 2024-09-21T06:08:43.057 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.057 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.057 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.057 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.057 INFO:teuthology.orchestra.run.smithi117.stdout: "container_init": { 2024-09-21T06:08:43.057 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_init", 2024-09-21T06:08:43.057 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.057 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.057 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.057 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.057 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.058 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.058 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.058 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T06:08:43.058 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.058 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.058 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.058 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.058 INFO:teuthology.orchestra.run.smithi117.stdout: "daemon_cache_timeout": { 2024-09-21T06:08:43.058 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "daemon_cache_timeout", 2024-09-21T06:08:43.058 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.058 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.058 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.058 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "600", 2024-09-21T06:08:43.058 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.058 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.059 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.059 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T06:08:43.059 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.059 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.059 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.059 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.059 INFO:teuthology.orchestra.run.smithi117.stdout: "default_cephadm_command_timeout": { 2024-09-21T06:08:43.059 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T06:08:43.059 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.059 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.059 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.059 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "900", 2024-09-21T06:08:43.059 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.059 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.059 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.060 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T06:08:43.060 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.060 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.060 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.060 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.060 INFO:teuthology.orchestra.run.smithi117.stdout: "default_registry": { 2024-09-21T06:08:43.060 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "default_registry", 2024-09-21T06:08:43.060 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.060 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.060 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.060 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "docker.io", 2024-09-21T06:08:43.060 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.060 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.060 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.060 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T06:08:43.060 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.061 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.061 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.061 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.061 INFO:teuthology.orchestra.run.smithi117.stdout: "device_cache_timeout": { 2024-09-21T06:08:43.061 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "device_cache_timeout", 2024-09-21T06:08:43.061 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.061 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.061 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.061 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "1800", 2024-09-21T06:08:43.061 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.061 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.061 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.061 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "seconds to cache device inventory", 2024-09-21T06:08:43.061 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.062 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.062 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.062 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.062 INFO:teuthology.orchestra.run.smithi117.stdout: "device_enhanced_scan": { 2024-09-21T06:08:43.062 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "device_enhanced_scan", 2024-09-21T06:08:43.062 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.062 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.062 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.062 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.062 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.062 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.062 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.062 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T06:08:43.062 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.062 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.063 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.063 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.063 INFO:teuthology.orchestra.run.smithi117.stdout: "facts_cache_timeout": { 2024-09-21T06:08:43.063 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "facts_cache_timeout", 2024-09-21T06:08:43.063 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.063 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.063 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.063 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "60", 2024-09-21T06:08:43.063 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.063 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.063 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.063 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "seconds to cache host facts data", 2024-09-21T06:08:43.063 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.063 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.063 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.064 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.064 INFO:teuthology.orchestra.run.smithi117.stdout: "grafana_dashboards_path": { 2024-09-21T06:08:43.064 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "grafana_dashboards_path", 2024-09-21T06:08:43.064 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.064 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.064 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.064 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T06:08:43.064 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.064 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.064 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.064 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T06:08:43.064 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.064 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.064 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.064 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.065 INFO:teuthology.orchestra.run.smithi117.stdout: "host_check_interval": { 2024-09-21T06:08:43.065 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "host_check_interval", 2024-09-21T06:08:43.065 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.065 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.065 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.065 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "600", 2024-09-21T06:08:43.065 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.065 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.065 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.065 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how frequently to perform a host check", 2024-09-21T06:08:43.065 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.065 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.065 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.065 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.066 INFO:teuthology.orchestra.run.smithi117.stdout: "hw_monitoring": { 2024-09-21T06:08:43.066 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "hw_monitoring", 2024-09-21T06:08:43.066 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.066 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.066 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.066 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.066 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.066 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.066 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.066 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T06:08:43.066 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.066 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.066 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.066 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.067 INFO:teuthology.orchestra.run.smithi117.stdout: "inventory_list_all": { 2024-09-21T06:08:43.067 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "inventory_list_all", 2024-09-21T06:08:43.067 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.067 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.067 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.067 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.067 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.067 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.067 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.067 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T06:08:43.067 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.067 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.067 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.067 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.068 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.068 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.068 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.068 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.068 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.068 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.068 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.068 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.068 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.068 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.068 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.068 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.068 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.068 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.068 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.069 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.069 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.069 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.069 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.069 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.069 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.069 INFO:teuthology.orchestra.run.smithi117.stdout: "log_refresh_metadata": { 2024-09-21T06:08:43.069 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_refresh_metadata", 2024-09-21T06:08:43.069 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.069 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.069 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.069 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.069 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.069 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.069 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.070 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T06:08:43.070 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.070 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.070 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.070 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.070 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.070 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.070 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.070 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.070 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.070 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.070 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.070 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.070 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.071 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T06:08:43.071 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.071 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.071 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.071 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.071 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.071 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.071 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.071 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.071 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.071 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.071 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.071 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.071 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.071 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.071 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.072 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.072 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.072 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.072 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.072 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.072 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.072 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.072 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.072 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.072 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.072 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.072 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.072 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.072 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.072 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.072 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.073 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.073 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.073 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.073 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.073 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.073 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.073 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.073 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.073 INFO:teuthology.orchestra.run.smithi117.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T06:08:43.073 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T06:08:43.073 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.073 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.073 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.073 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.073 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.074 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.074 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.074 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T06:08:43.074 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.074 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.074 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.074 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.074 INFO:teuthology.orchestra.run.smithi117.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T06:08:43.074 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T06:08:43.074 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.074 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.074 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.074 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "*", 2024-09-21T06:08:43.074 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.074 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.075 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.075 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T06:08:43.075 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.075 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.075 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.075 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.075 INFO:teuthology.orchestra.run.smithi117.stdout: "max_count_per_host": { 2024-09-21T06:08:43.075 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "max_count_per_host", 2024-09-21T06:08:43.075 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.075 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.075 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.075 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-09-21T06:08:43.075 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.075 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.075 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.076 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "max number of daemons per service per host", 2024-09-21T06:08:43.076 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.076 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.076 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.076 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.076 INFO:teuthology.orchestra.run.smithi117.stdout: "max_osd_draining_count": { 2024-09-21T06:08:43.076 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "max_osd_draining_count", 2024-09-21T06:08:43.076 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.076 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.076 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.076 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-09-21T06:08:43.076 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.076 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.076 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.077 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T06:08:43.077 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.077 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.077 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.077 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.077 INFO:teuthology.orchestra.run.smithi117.stdout: "migration_current": { 2024-09-21T06:08:43.077 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "migration_current", 2024-09-21T06:08:43.077 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.077 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.077 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.077 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.077 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.077 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.077 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.078 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "internal - do not modify", 2024-09-21T06:08:43.078 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.078 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.078 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.078 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.078 INFO:teuthology.orchestra.run.smithi117.stdout: "mode": { 2024-09-21T06:08:43.078 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mode", 2024-09-21T06:08:43.078 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.078 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.078 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.078 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "root", 2024-09-21T06:08:43.078 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.078 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.078 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.079 INFO:teuthology.orchestra.run.smithi117.stdout: "cephadm-package", 2024-09-21T06:08:43.079 INFO:teuthology.orchestra.run.smithi117.stdout: "root" 2024-09-21T06:08:43.079 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.079 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T06:08:43.079 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.079 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.079 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.079 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.079 INFO:teuthology.orchestra.run.smithi117.stdout: "oob_default_addr": { 2024-09-21T06:08:43.079 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "oob_default_addr", 2024-09-21T06:08:43.079 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.079 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.079 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.079 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "169.254.1.1", 2024-09-21T06:08:43.079 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.080 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.080 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.080 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T06:08:43.080 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.080 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.080 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.080 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.080 INFO:teuthology.orchestra.run.smithi117.stdout: "prometheus_alerts_path": { 2024-09-21T06:08:43.080 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "prometheus_alerts_path", 2024-09-21T06:08:43.080 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.080 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.080 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.080 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T06:08:43.080 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.080 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.080 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.081 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T06:08:43.081 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.081 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.081 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.081 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.081 INFO:teuthology.orchestra.run.smithi117.stdout: "registry_insecure": { 2024-09-21T06:08:43.081 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "registry_insecure", 2024-09-21T06:08:43.081 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.081 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.081 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.081 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.081 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.081 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.081 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.081 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T06:08:43.082 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.082 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.082 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.082 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.082 INFO:teuthology.orchestra.run.smithi117.stdout: "registry_password": { 2024-09-21T06:08:43.082 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "registry_password", 2024-09-21T06:08:43.082 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.082 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.082 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.082 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.082 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.082 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.082 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.082 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T06:08:43.082 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.083 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.083 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.083 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.083 INFO:teuthology.orchestra.run.smithi117.stdout: "registry_url": { 2024-09-21T06:08:43.083 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "registry_url", 2024-09-21T06:08:43.083 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.083 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.083 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.083 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.083 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.083 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.083 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.083 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T06:08:43.083 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.083 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.084 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.084 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.084 INFO:teuthology.orchestra.run.smithi117.stdout: "registry_username": { 2024-09-21T06:08:43.084 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "registry_username", 2024-09-21T06:08:43.084 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.084 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.084 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.084 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.084 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.084 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.084 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.084 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T06:08:43.084 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.084 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.084 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.084 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.085 INFO:teuthology.orchestra.run.smithi117.stdout: "secure_monitoring_stack": { 2024-09-21T06:08:43.085 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "secure_monitoring_stack", 2024-09-21T06:08:43.085 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.085 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.085 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.085 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.085 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.085 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.085 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.085 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T06:08:43.085 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.085 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.085 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.085 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.086 INFO:teuthology.orchestra.run.smithi117.stdout: "service_discovery_port": { 2024-09-21T06:08:43.086 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "service_discovery_port", 2024-09-21T06:08:43.086 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.086 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.086 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.086 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "8765", 2024-09-21T06:08:43.086 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.086 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.086 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.086 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "cephadm service discovery port", 2024-09-21T06:08:43.086 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.086 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.086 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.086 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.086 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.086 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.087 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.087 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.087 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.087 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.087 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.087 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.087 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.087 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.087 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.087 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.087 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.087 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.087 INFO:teuthology.orchestra.run.smithi117.stdout: "ssh_config_file": { 2024-09-21T06:08:43.088 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ssh_config_file", 2024-09-21T06:08:43.088 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.088 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.088 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.088 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.088 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.088 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.088 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.088 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T06:08:43.088 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.088 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.088 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.088 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.088 INFO:teuthology.orchestra.run.smithi117.stdout: "ssh_keepalive_count_max": { 2024-09-21T06:08:43.088 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T06:08:43.089 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.089 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.089 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.089 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "3", 2024-09-21T06:08:43.089 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.089 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.089 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.089 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T06:08:43.089 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.089 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.089 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.089 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.089 INFO:teuthology.orchestra.run.smithi117.stdout: "ssh_keepalive_interval": { 2024-09-21T06:08:43.089 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ssh_keepalive_interval", 2024-09-21T06:08:43.089 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.090 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.090 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.090 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "7", 2024-09-21T06:08:43.090 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.090 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.090 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.090 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T06:08:43.090 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.090 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.090 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.090 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.090 INFO:teuthology.orchestra.run.smithi117.stdout: "use_agent": { 2024-09-21T06:08:43.090 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "use_agent", 2024-09-21T06:08:43.090 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.090 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.091 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.091 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.091 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.091 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.091 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.091 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T06:08:43.091 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.091 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.091 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.091 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.091 INFO:teuthology.orchestra.run.smithi117.stdout: "use_repo_digest": { 2024-09-21T06:08:43.091 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "use_repo_digest", 2024-09-21T06:08:43.091 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.091 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.091 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.092 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.092 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.092 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.092 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.092 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T06:08:43.092 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.092 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.092 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.092 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.092 INFO:teuthology.orchestra.run.smithi117.stdout: "warn_on_failed_host_check": { 2024-09-21T06:08:43.092 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "warn_on_failed_host_check", 2024-09-21T06:08:43.092 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.092 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.092 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.092 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.093 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.093 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.093 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.093 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T06:08:43.093 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.093 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.093 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.093 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.093 INFO:teuthology.orchestra.run.smithi117.stdout: "warn_on_stray_daemons": { 2024-09-21T06:08:43.093 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "warn_on_stray_daemons", 2024-09-21T06:08:43.093 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.093 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.093 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.093 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.093 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.094 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.094 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.094 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T06:08:43.094 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.094 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.094 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.094 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.094 INFO:teuthology.orchestra.run.smithi117.stdout: "warn_on_stray_hosts": { 2024-09-21T06:08:43.094 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "warn_on_stray_hosts", 2024-09-21T06:08:43.094 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.094 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.094 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.094 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.094 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.094 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.095 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.095 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T06:08:43.095 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.095 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.095 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.095 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.095 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.095 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.095 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.095 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crash", 2024-09-21T06:08:43.095 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.095 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.095 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.095 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.095 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.095 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.096 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.096 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.096 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.096 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.096 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.096 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.096 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.096 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.096 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.096 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.096 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.096 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.096 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.096 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.096 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.097 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.097 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.097 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.097 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.097 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.097 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.097 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.097 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.097 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.097 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.097 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.097 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.097 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.097 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.097 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.098 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.098 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.098 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.098 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.098 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.098 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.098 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.098 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.098 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.098 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.098 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.098 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.098 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.098 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.099 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.099 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.099 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.099 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.099 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.099 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.099 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.099 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.099 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.099 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.099 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.099 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.099 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.099 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.100 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.100 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.100 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.100 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.100 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.100 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.100 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.100 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.100 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.100 INFO:teuthology.orchestra.run.smithi117.stdout: "retain_interval": { 2024-09-21T06:08:43.100 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "retain_interval", 2024-09-21T06:08:43.100 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.100 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.100 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.101 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "31536000", 2024-09-21T06:08:43.101 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.101 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.101 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.101 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T06:08:43.101 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.101 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.101 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.101 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.101 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.101 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.101 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.101 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.101 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.101 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.102 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.102 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.102 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.102 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.102 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.102 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.102 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.102 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.102 INFO:teuthology.orchestra.run.smithi117.stdout: "warn_recent_interval": { 2024-09-21T06:08:43.102 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "warn_recent_interval", 2024-09-21T06:08:43.102 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.102 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.102 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.102 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "1209600", 2024-09-21T06:08:43.103 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.103 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.103 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.103 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T06:08:43.103 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.103 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.103 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.103 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.103 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.103 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.103 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.103 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "dashboard", 2024-09-21T06:08:43.103 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.103 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.103 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.104 INFO:teuthology.orchestra.run.smithi117.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T06:08:43.104 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T06:08:43.104 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.104 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.104 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.104 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-09-21T06:08:43.104 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.104 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.104 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.104 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.104 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.104 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.104 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.104 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.105 INFO:teuthology.orchestra.run.smithi117.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T06:08:43.105 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T06:08:43.105 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.105 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.105 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.105 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.105 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.105 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.105 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.105 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.105 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.105 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.105 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.105 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.105 INFO:teuthology.orchestra.run.smithi117.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T06:08:43.106 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T06:08:43.106 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.106 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.106 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.106 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.106 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.106 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.106 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.106 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.106 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.106 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.106 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.106 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.107 INFO:teuthology.orchestra.run.smithi117.stdout: "AUDIT_API_ENABLED": { 2024-09-21T06:08:43.107 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T06:08:43.107 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.107 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.107 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.107 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.107 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.107 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.107 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.107 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.107 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.107 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.107 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.107 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.108 INFO:teuthology.orchestra.run.smithi117.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T06:08:43.108 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T06:08:43.108 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.108 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.108 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.108 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.108 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.108 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.108 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.108 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.108 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.108 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.108 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.108 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.109 INFO:teuthology.orchestra.run.smithi117.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T06:08:43.109 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T06:08:43.109 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.109 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.109 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.109 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.109 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.109 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.109 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.109 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.109 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.109 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.109 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.109 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.109 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T06:08:43.110 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T06:08:43.110 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.110 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.110 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.110 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.110 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.110 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.110 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.110 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.110 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.110 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.110 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.110 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.110 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T06:08:43.110 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T06:08:43.111 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.111 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.111 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.111 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.111 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.111 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.111 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.111 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.111 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.111 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.111 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.111 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.111 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T06:08:43.111 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T06:08:43.111 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.111 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.112 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.112 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.112 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.112 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.112 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.112 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.112 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.112 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.112 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.112 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.112 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T06:08:43.112 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T06:08:43.112 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.112 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.113 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.113 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.113 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.113 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.113 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.113 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.113 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.113 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.113 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.113 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.113 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T06:08:43.113 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T06:08:43.113 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.113 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.113 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.114 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.114 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.114 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.114 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.114 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.114 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.114 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.114 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.114 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.114 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T06:08:43.114 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T06:08:43.114 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.114 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.114 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.114 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.115 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.115 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.115 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.115 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.115 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.115 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.115 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.115 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.115 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T06:08:43.115 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T06:08:43.115 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.115 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.115 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.115 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.115 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.115 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.116 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.116 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.116 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.116 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.116 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.116 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.116 INFO:teuthology.orchestra.run.smithi117.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T06:08:43.116 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T06:08:43.116 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.116 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.116 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.116 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.116 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.116 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.116 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.117 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.117 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.117 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.117 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.117 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.117 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T06:08:43.117 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T06:08:43.117 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.117 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.117 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.117 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "admin", 2024-09-21T06:08:43.117 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.117 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.117 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.117 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.118 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.118 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.118 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.118 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.118 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T06:08:43.118 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T06:08:43.118 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.118 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.118 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.118 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.118 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.118 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.118 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.118 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.118 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.119 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.119 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.119 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.119 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_API_URL": { 2024-09-21T06:08:43.119 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_API_URL", 2024-09-21T06:08:43.119 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.119 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.119 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.119 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.119 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.119 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.119 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.119 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.119 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.119 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.119 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.120 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.120 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T06:08:43.120 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T06:08:43.120 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.120 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.120 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.120 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "admin", 2024-09-21T06:08:43.120 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.120 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.120 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.120 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.120 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.120 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.120 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.120 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.120 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T06:08:43.121 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T06:08:43.121 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.121 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.121 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.121 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.121 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.121 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.121 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.121 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.121 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.121 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.121 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.121 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.121 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T06:08:43.121 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T06:08:43.122 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.122 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.122 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.122 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.122 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.122 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.122 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.122 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.122 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.122 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.122 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.122 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.122 INFO:teuthology.orchestra.run.smithi117.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T06:08:43.122 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T06:08:43.122 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.123 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.123 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.123 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.123 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.123 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.123 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.123 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.123 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.123 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.123 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.123 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.123 INFO:teuthology.orchestra.run.smithi117.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T06:08:43.123 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T06:08:43.123 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.123 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.124 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.124 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.124 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.124 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.124 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.124 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.124 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.124 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.124 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.124 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.124 INFO:teuthology.orchestra.run.smithi117.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T06:08:43.124 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T06:08:43.124 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.124 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.124 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.125 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "[]", 2024-09-21T06:08:43.125 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.125 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.125 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.125 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.125 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.125 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.125 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.125 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.125 INFO:teuthology.orchestra.run.smithi117.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T06:08:43.125 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T06:08:43.125 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.125 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.125 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.126 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "{}", 2024-09-21T06:08:43.126 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.126 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.126 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.126 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.126 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.126 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.126 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.126 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.126 INFO:teuthology.orchestra.run.smithi117.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T06:08:43.126 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T06:08:43.126 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.126 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.126 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.126 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.127 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.127 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.127 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.127 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.127 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.127 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.127 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.127 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.127 INFO:teuthology.orchestra.run.smithi117.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T06:08:43.127 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T06:08:43.127 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.127 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.127 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.127 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.127 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.128 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.128 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.128 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.128 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.128 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.128 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.128 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.128 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T06:08:43.128 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T06:08:43.128 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.128 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.128 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.128 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.128 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.128 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.129 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.129 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.129 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.129 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.129 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.129 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.129 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T06:08:43.129 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T06:08:43.129 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.129 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.129 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.129 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.129 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.129 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.129 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.129 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.130 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.130 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.130 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.130 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.130 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T06:08:43.130 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T06:08:43.130 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.130 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.130 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.130 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.130 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.130 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.130 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.130 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.130 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.131 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.131 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.131 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.131 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T06:08:43.131 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T06:08:43.131 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.131 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.131 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.131 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.131 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.131 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.131 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.131 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.131 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.132 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.132 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.132 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.132 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T06:08:43.132 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T06:08:43.132 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.132 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.132 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.132 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.132 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.132 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.132 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.132 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.132 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.132 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.133 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.133 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.133 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T06:08:43.133 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T06:08:43.133 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.133 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.133 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.133 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.133 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.133 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.133 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.133 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.133 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.133 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.133 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.134 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.134 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T06:08:43.134 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T06:08:43.134 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.134 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.134 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.134 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.134 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.134 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.134 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.134 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.134 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.134 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.134 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.134 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.134 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T06:08:43.135 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T06:08:43.135 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.135 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.135 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.135 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.135 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.135 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.135 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.135 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.135 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.135 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.135 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.135 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.135 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T06:08:43.136 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T06:08:43.136 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.136 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.136 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.136 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T06:08:43.136 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.136 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.136 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.136 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.136 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.136 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.136 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.136 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.136 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T06:08:43.136 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T06:08:43.137 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.137 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.137 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.137 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-09-21T06:08:43.137 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.137 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.137 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.137 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.137 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.137 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.137 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.137 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.137 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T06:08:43.137 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T06:08:43.138 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.138 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.138 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.138 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "8", 2024-09-21T06:08:43.138 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.138 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.138 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.138 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.138 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.138 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.138 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.138 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.138 INFO:teuthology.orchestra.run.smithi117.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T06:08:43.138 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T06:08:43.139 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.139 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.139 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.139 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "45", 2024-09-21T06:08:43.139 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.139 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.139 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.139 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.139 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.139 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.139 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.139 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.139 INFO:teuthology.orchestra.run.smithi117.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T06:08:43.139 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T06:08:43.139 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.140 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.140 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.140 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.140 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.140 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.140 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.140 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.140 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.140 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.140 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.140 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.140 INFO:teuthology.orchestra.run.smithi117.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T06:08:43.140 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T06:08:43.140 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.140 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.141 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.141 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "admin", 2024-09-21T06:08:43.141 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.141 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.141 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.141 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.141 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.141 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.141 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.141 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.141 INFO:teuthology.orchestra.run.smithi117.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T06:08:43.141 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T06:08:43.141 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.141 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.141 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.142 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.142 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.142 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.142 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.142 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.142 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.142 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.142 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.142 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.142 INFO:teuthology.orchestra.run.smithi117.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T06:08:43.142 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T06:08:43.142 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.142 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.142 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.142 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.143 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.143 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.143 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.143 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.143 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.143 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.143 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.143 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.143 INFO:teuthology.orchestra.run.smithi117.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T06:08:43.143 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T06:08:43.143 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.143 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.143 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.143 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.143 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.143 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.144 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.144 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.144 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.144 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.144 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.144 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.144 INFO:teuthology.orchestra.run.smithi117.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T06:08:43.144 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T06:08:43.144 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.144 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.144 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.144 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.144 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.144 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.144 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.144 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.145 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.145 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.145 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.145 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.145 INFO:teuthology.orchestra.run.smithi117.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T06:08:43.145 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T06:08:43.145 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.145 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.145 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.145 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-09-21T06:08:43.145 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.145 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.145 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.145 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.145 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.146 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.146 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.146 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.146 INFO:teuthology.orchestra.run.smithi117.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T06:08:43.146 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T06:08:43.146 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.146 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.146 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.146 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "5", 2024-09-21T06:08:43.146 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.146 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.146 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.146 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.146 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.146 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.147 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.147 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.147 INFO:teuthology.orchestra.run.smithi117.stdout: "cross_origin_url": { 2024-09-21T06:08:43.147 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "cross_origin_url", 2024-09-21T06:08:43.147 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.147 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.147 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.147 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.147 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.147 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.147 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.147 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.147 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.147 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.147 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.147 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.148 INFO:teuthology.orchestra.run.smithi117.stdout: "crt_file": { 2024-09-21T06:08:43.148 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crt_file", 2024-09-21T06:08:43.148 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.148 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.148 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.148 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.148 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.148 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.148 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.148 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.148 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.148 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.148 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.148 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.148 INFO:teuthology.orchestra.run.smithi117.stdout: "debug": { 2024-09-21T06:08:43.148 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "debug", 2024-09-21T06:08:43.149 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.149 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.149 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.149 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.149 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.149 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.149 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.149 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Enable/disable debug options", 2024-09-21T06:08:43.149 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.149 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.149 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.149 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.149 INFO:teuthology.orchestra.run.smithi117.stdout: "jwt_token_ttl": { 2024-09-21T06:08:43.149 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "jwt_token_ttl", 2024-09-21T06:08:43.149 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.150 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.150 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.150 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "28800", 2024-09-21T06:08:43.150 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.150 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.150 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.150 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.150 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.150 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.150 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.150 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.150 INFO:teuthology.orchestra.run.smithi117.stdout: "key_file": { 2024-09-21T06:08:43.150 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "key_file", 2024-09-21T06:08:43.150 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.151 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.151 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.151 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.151 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.151 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.151 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.151 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.151 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.151 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.151 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.151 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.151 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.151 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.151 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.151 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.152 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.152 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.152 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.152 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.152 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.152 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.152 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.152 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.152 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.152 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.152 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.152 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.152 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.152 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.152 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.152 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.153 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.153 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.153 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.153 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.153 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.153 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.153 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.153 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.153 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.153 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.153 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.153 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.153 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.153 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.153 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.153 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.154 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.154 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.154 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.154 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.154 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.154 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.154 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.154 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.154 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.154 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.154 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.154 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.154 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.154 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.155 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.155 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.155 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.155 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.155 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.155 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.155 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.155 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.155 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.155 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.155 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.155 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.155 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.155 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.155 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.156 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.156 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.156 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.156 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.156 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.156 INFO:teuthology.orchestra.run.smithi117.stdout: "motd": { 2024-09-21T06:08:43.156 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "motd", 2024-09-21T06:08:43.156 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.156 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.156 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.156 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.156 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.156 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.156 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.157 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "The message of the day", 2024-09-21T06:08:43.157 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.157 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.157 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.157 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.157 INFO:teuthology.orchestra.run.smithi117.stdout: "redirect_resolve_ip_addr": { 2024-09-21T06:08:43.157 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T06:08:43.157 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.157 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.157 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.157 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.157 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.157 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.157 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.157 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.157 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.158 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.158 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.158 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.158 INFO:teuthology.orchestra.run.smithi117.stdout: "server_addr": { 2024-09-21T06:08:43.158 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_addr", 2024-09-21T06:08:43.158 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.158 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.158 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.158 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "::", 2024-09-21T06:08:43.158 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.158 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.158 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.158 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.158 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.158 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.159 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.159 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.159 INFO:teuthology.orchestra.run.smithi117.stdout: "server_port": { 2024-09-21T06:08:43.159 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_port", 2024-09-21T06:08:43.159 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.159 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.159 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.159 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "8080", 2024-09-21T06:08:43.159 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.159 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.159 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.159 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.159 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.159 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.159 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.159 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.160 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.160 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.160 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.160 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.160 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.160 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.160 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.160 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.160 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.160 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.160 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.160 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.160 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.160 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.160 INFO:teuthology.orchestra.run.smithi117.stdout: "ssl": { 2024-09-21T06:08:43.161 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ssl", 2024-09-21T06:08:43.161 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.161 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.161 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.161 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.161 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.161 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.161 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.161 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.161 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.161 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.161 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.161 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.161 INFO:teuthology.orchestra.run.smithi117.stdout: "ssl_server_port": { 2024-09-21T06:08:43.161 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ssl_server_port", 2024-09-21T06:08:43.162 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.162 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.162 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.162 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "8443", 2024-09-21T06:08:43.162 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.162 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.162 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.162 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.162 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.162 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.162 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.162 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.162 INFO:teuthology.orchestra.run.smithi117.stdout: "sso_oauth2": { 2024-09-21T06:08:43.163 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sso_oauth2", 2024-09-21T06:08:43.163 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.163 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.163 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.163 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.163 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.163 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.163 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.163 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.163 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.163 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.163 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.163 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.163 INFO:teuthology.orchestra.run.smithi117.stdout: "standby_behaviour": { 2024-09-21T06:08:43.164 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "standby_behaviour", 2024-09-21T06:08:43.164 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.164 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.164 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.164 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "redirect", 2024-09-21T06:08:43.164 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.164 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.164 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.164 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.164 INFO:teuthology.orchestra.run.smithi117.stdout: "redirect" 2024-09-21T06:08:43.164 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.164 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.164 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.164 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.164 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.165 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.165 INFO:teuthology.orchestra.run.smithi117.stdout: "standby_error_status_code": { 2024-09-21T06:08:43.165 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "standby_error_status_code", 2024-09-21T06:08:43.165 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.165 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.165 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.165 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "500", 2024-09-21T06:08:43.165 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "400", 2024-09-21T06:08:43.165 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "599", 2024-09-21T06:08:43.165 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.165 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.165 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.165 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.165 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.165 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.165 INFO:teuthology.orchestra.run.smithi117.stdout: "url_prefix": { 2024-09-21T06:08:43.166 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "url_prefix", 2024-09-21T06:08:43.166 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.166 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.166 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.166 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.166 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.166 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.166 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.166 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.166 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.166 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.166 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.166 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.166 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.166 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.167 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.167 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "devicehealth", 2024-09-21T06:08:43.167 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.167 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.167 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.167 INFO:teuthology.orchestra.run.smithi117.stdout: "enable_monitoring": { 2024-09-21T06:08:43.167 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "enable_monitoring", 2024-09-21T06:08:43.167 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.167 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.167 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.167 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.167 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.167 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.167 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.167 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "monitor device health metrics", 2024-09-21T06:08:43.168 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.168 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.168 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.168 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.168 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.168 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.168 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.168 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.168 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.168 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.168 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.168 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.168 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.168 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.168 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.169 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.169 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.169 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.169 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.169 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.169 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.169 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.169 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.169 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.169 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.169 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.169 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.169 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.169 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.169 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.169 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.170 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.170 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.170 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.170 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.170 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.170 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.170 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.170 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.170 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.170 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.170 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.170 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.170 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.170 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.170 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.171 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.171 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.171 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.171 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.171 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.171 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.171 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.171 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.171 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.171 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.171 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.171 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.171 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.171 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.171 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.172 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.172 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.172 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.172 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.172 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.172 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.172 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.172 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.172 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.172 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.172 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.172 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.172 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.172 INFO:teuthology.orchestra.run.smithi117.stdout: "mark_out_threshold": { 2024-09-21T06:08:43.172 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mark_out_threshold", 2024-09-21T06:08:43.173 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.173 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.173 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.173 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "2419200", 2024-09-21T06:08:43.173 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.173 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.173 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.173 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T06:08:43.173 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.173 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.173 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.173 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.173 INFO:teuthology.orchestra.run.smithi117.stdout: "pool_name": { 2024-09-21T06:08:43.173 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "pool_name", 2024-09-21T06:08:43.173 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.173 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.174 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.174 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "device_health_metrics", 2024-09-21T06:08:43.174 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.174 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.174 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.174 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T06:08:43.174 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.174 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.174 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.174 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.174 INFO:teuthology.orchestra.run.smithi117.stdout: "retention_period": { 2024-09-21T06:08:43.174 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "retention_period", 2024-09-21T06:08:43.174 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.174 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.174 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.174 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "15552000", 2024-09-21T06:08:43.175 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.175 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.175 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.175 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how long to retain device health metrics", 2024-09-21T06:08:43.175 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.175 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.175 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.175 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.175 INFO:teuthology.orchestra.run.smithi117.stdout: "scrape_frequency": { 2024-09-21T06:08:43.175 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "scrape_frequency", 2024-09-21T06:08:43.175 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.175 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.175 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.176 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "86400", 2024-09-21T06:08:43.176 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.176 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.176 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.176 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T06:08:43.176 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.176 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.176 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.176 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.176 INFO:teuthology.orchestra.run.smithi117.stdout: "self_heal": { 2024-09-21T06:08:43.176 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "self_heal", 2024-09-21T06:08:43.176 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.176 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.176 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.177 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.177 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.177 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.177 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.177 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T06:08:43.177 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.177 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.177 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.177 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.177 INFO:teuthology.orchestra.run.smithi117.stdout: "sleep_interval": { 2024-09-21T06:08:43.177 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sleep_interval", 2024-09-21T06:08:43.177 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.177 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.177 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.178 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "600", 2024-09-21T06:08:43.178 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.178 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.178 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.178 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T06:08:43.178 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.178 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.178 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.178 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.178 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.178 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.178 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.178 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.178 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.178 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.179 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.179 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.179 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.179 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.179 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.179 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.179 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.179 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.179 INFO:teuthology.orchestra.run.smithi117.stdout: "warn_threshold": { 2024-09-21T06:08:43.179 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "warn_threshold", 2024-09-21T06:08:43.179 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.179 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.179 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.179 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "7257600", 2024-09-21T06:08:43.179 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.180 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.180 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.180 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T06:08:43.180 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.180 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.180 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.180 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.180 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.180 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.180 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.180 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "diskprediction_local", 2024-09-21T06:08:43.180 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.180 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.180 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.180 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.181 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.181 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.181 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.181 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.181 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.181 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.181 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.181 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.181 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.181 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.181 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.181 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.181 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.181 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.181 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.182 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.182 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.182 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.182 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.182 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.182 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.182 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.182 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.182 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.182 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.182 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.182 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.182 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.182 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.182 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.183 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.183 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.183 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.183 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.183 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.183 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.183 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.183 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.183 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.183 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.183 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.183 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.183 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.183 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.183 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.184 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.184 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.184 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.184 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.184 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.184 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.184 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.184 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.184 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.184 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.184 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.184 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.184 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.184 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.184 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.185 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.185 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.185 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.185 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.185 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.185 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.185 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.185 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.185 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.185 INFO:teuthology.orchestra.run.smithi117.stdout: "predict_interval": { 2024-09-21T06:08:43.185 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "predict_interval", 2024-09-21T06:08:43.185 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.185 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.185 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.185 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "86400", 2024-09-21T06:08:43.185 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.186 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.186 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.186 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.186 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.186 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.186 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.186 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.186 INFO:teuthology.orchestra.run.smithi117.stdout: "predictor_model": { 2024-09-21T06:08:43.186 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "predictor_model", 2024-09-21T06:08:43.186 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.186 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.186 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.186 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "prophetstor", 2024-09-21T06:08:43.186 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.186 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.187 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.187 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.187 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.187 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.187 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.187 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.187 INFO:teuthology.orchestra.run.smithi117.stdout: "sleep_interval": { 2024-09-21T06:08:43.187 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sleep_interval", 2024-09-21T06:08:43.187 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.187 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.187 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.187 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "600", 2024-09-21T06:08:43.187 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.187 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.187 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.188 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.188 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.188 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.188 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.188 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.188 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.188 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.188 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.188 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.188 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.188 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.188 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.188 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.188 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.189 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.189 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.189 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.189 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.189 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.189 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.189 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.189 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.189 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "influx", 2024-09-21T06:08:43.189 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": false, 2024-09-21T06:08:43.189 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "influxdb python module not found", 2024-09-21T06:08:43.189 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.189 INFO:teuthology.orchestra.run.smithi117.stdout: "batch_size": { 2024-09-21T06:08:43.189 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "batch_size", 2024-09-21T06:08:43.189 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.189 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.190 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.190 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "5000", 2024-09-21T06:08:43.190 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.190 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.190 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.190 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T06:08:43.190 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.190 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.190 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.190 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.190 INFO:teuthology.orchestra.run.smithi117.stdout: "database": { 2024-09-21T06:08:43.190 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "database", 2024-09-21T06:08:43.190 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.190 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.190 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.191 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "ceph", 2024-09-21T06:08:43.191 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.191 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.191 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.191 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T06:08:43.191 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.191 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.191 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.191 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.191 INFO:teuthology.orchestra.run.smithi117.stdout: "hostname": { 2024-09-21T06:08:43.191 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "hostname", 2024-09-21T06:08:43.191 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.191 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.191 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.191 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.191 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.191 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.192 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.192 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "InfluxDB server hostname", 2024-09-21T06:08:43.192 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.192 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.192 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.192 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.192 INFO:teuthology.orchestra.run.smithi117.stdout: "interval": { 2024-09-21T06:08:43.192 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "interval", 2024-09-21T06:08:43.192 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.192 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.192 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.192 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "30", 2024-09-21T06:08:43.192 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "5", 2024-09-21T06:08:43.192 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.192 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.192 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T06:08:43.193 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.193 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.193 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.193 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.193 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.193 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.193 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.193 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.193 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.193 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.193 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.193 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.193 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.193 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.193 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.194 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.194 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.194 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.194 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.194 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.194 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.194 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.194 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.194 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.194 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.194 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.194 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.194 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.194 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.194 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.195 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.195 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.195 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.195 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.195 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.195 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.195 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.195 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.195 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.195 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.195 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.195 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.195 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.195 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.195 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.195 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.196 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.196 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.196 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.196 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.196 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.196 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.196 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.196 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.196 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.196 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.196 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.196 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.196 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.196 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.196 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.197 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.197 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.197 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.197 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.197 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.197 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.197 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.197 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.197 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.197 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.197 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.197 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.197 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.197 INFO:teuthology.orchestra.run.smithi117.stdout: "password": { 2024-09-21T06:08:43.197 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "password", 2024-09-21T06:08:43.197 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.197 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.198 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.198 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.198 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.198 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.198 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.198 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "password of InfluxDB server user", 2024-09-21T06:08:43.198 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.198 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.198 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.198 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.198 INFO:teuthology.orchestra.run.smithi117.stdout: "port": { 2024-09-21T06:08:43.198 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "port", 2024-09-21T06:08:43.198 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.198 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.199 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.199 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "8086", 2024-09-21T06:08:43.199 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.199 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.199 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.199 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "InfluxDB server port", 2024-09-21T06:08:43.199 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.199 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.199 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.199 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.199 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.199 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.199 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.199 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.199 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.199 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.200 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.200 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.200 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.200 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.200 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.200 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.200 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.200 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.200 INFO:teuthology.orchestra.run.smithi117.stdout: "ssl": { 2024-09-21T06:08:43.200 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ssl", 2024-09-21T06:08:43.200 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.200 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.200 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.200 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "false", 2024-09-21T06:08:43.201 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.201 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.201 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.201 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T06:08:43.201 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.201 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.201 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.201 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.201 INFO:teuthology.orchestra.run.smithi117.stdout: "threads": { 2024-09-21T06:08:43.201 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "threads", 2024-09-21T06:08:43.201 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.201 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.201 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.201 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "5", 2024-09-21T06:08:43.201 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "1", 2024-09-21T06:08:43.202 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "32", 2024-09-21T06:08:43.202 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.202 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T06:08:43.202 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.202 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.202 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.202 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.202 INFO:teuthology.orchestra.run.smithi117.stdout: "username": { 2024-09-21T06:08:43.202 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "username", 2024-09-21T06:08:43.202 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.202 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.202 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.202 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.202 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.202 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.203 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.203 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "username of InfluxDB server user", 2024-09-21T06:08:43.203 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.203 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.203 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.203 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.203 INFO:teuthology.orchestra.run.smithi117.stdout: "verify_ssl": { 2024-09-21T06:08:43.203 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "verify_ssl", 2024-09-21T06:08:43.203 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.203 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.203 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.203 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "true", 2024-09-21T06:08:43.203 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.203 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.204 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.204 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T06:08:43.204 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.204 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.204 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.204 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.204 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.204 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.204 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.204 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "insights", 2024-09-21T06:08:43.204 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.204 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.204 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.204 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.204 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.205 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.205 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.205 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.205 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.205 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.205 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.205 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.205 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.205 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.205 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.205 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.205 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.205 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.205 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.205 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.205 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.206 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.206 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.206 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.206 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.206 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.206 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.206 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.206 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.206 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.206 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.206 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.206 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.206 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.206 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.207 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.207 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.207 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.207 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.207 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.207 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.207 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.207 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.207 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.207 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.207 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.207 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.207 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.207 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.207 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.208 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.208 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.208 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.208 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.208 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.208 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.208 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.208 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.208 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.208 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.208 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.208 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.208 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.208 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.208 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.209 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.209 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.209 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.209 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.209 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.209 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.209 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.209 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.209 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.209 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.209 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.209 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.209 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.209 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.209 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.209 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.210 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.210 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.210 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.210 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.210 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.210 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.210 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.210 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.210 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.210 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "iostat", 2024-09-21T06:08:43.210 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.210 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.210 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.210 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.210 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.211 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.211 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.211 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.211 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.211 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.211 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.211 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.211 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.211 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.211 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.211 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.211 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.211 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.211 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.211 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.212 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.212 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.212 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.212 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.212 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.212 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.212 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.212 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.212 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.212 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.212 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.212 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.212 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.212 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.212 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.213 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.213 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.213 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.213 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.213 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.213 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.213 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.213 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.213 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.213 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.213 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.213 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.213 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.213 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.213 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.214 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.214 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.214 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.214 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.214 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.214 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.214 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.214 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.214 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.214 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.214 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.214 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.214 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.214 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.214 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.215 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.215 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.215 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.215 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.215 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.215 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.215 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.215 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.215 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.215 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.215 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.215 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.215 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.215 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.216 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.216 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.216 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.216 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.216 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.216 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.216 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.216 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.216 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.216 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.216 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.216 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "k8sevents", 2024-09-21T06:08:43.216 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.216 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.216 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.217 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_event_retention_days": { 2024-09-21T06:08:43.217 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ceph_event_retention_days", 2024-09-21T06:08:43.217 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.217 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.217 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.217 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "7", 2024-09-21T06:08:43.217 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.217 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.217 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.217 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T06:08:43.217 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.217 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.217 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.217 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.217 INFO:teuthology.orchestra.run.smithi117.stdout: "config_check_secs": { 2024-09-21T06:08:43.217 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "config_check_secs", 2024-09-21T06:08:43.218 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.218 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.218 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.218 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-09-21T06:08:43.218 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "10", 2024-09-21T06:08:43.218 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.218 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.218 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T06:08:43.218 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.218 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.218 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.218 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.218 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.218 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.219 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.219 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.219 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.219 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.219 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.219 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.219 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.219 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.219 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.219 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.219 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.219 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.219 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.219 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.219 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.219 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.220 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.220 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.220 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.220 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.220 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.220 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.220 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.220 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.220 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.220 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.220 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.220 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.220 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.220 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.220 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.221 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.221 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.221 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.221 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.221 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.221 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.221 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.221 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.221 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.221 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.221 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.221 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.221 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.221 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.221 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.221 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.222 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.222 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.222 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.222 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.222 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.222 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.222 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.222 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.222 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.222 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.222 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.222 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.222 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.222 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.222 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.223 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.223 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.223 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.223 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.223 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.223 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.223 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.223 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.223 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.223 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.223 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.223 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.223 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.223 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.223 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.224 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.224 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.224 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.224 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.224 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.224 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.224 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.224 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.224 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "localpool", 2024-09-21T06:08:43.224 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.224 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.224 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.224 INFO:teuthology.orchestra.run.smithi117.stdout: "failure_domain": { 2024-09-21T06:08:43.224 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "failure_domain", 2024-09-21T06:08:43.224 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.224 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.225 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.225 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "host", 2024-09-21T06:08:43.225 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.225 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.225 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.225 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "failure domain for any created local pool", 2024-09-21T06:08:43.225 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T06:08:43.225 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.225 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.225 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.225 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.225 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.225 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.226 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.226 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.226 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.226 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.226 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.226 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.226 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.226 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.226 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.226 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.226 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.227 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.227 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.227 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.227 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.227 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.227 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.227 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.227 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.227 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.227 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.227 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.228 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.228 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.228 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.228 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.228 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.228 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.228 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.228 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.228 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.228 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.228 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.228 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.228 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.228 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.228 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.229 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.229 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.229 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.229 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.229 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.229 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.229 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.229 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.229 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.229 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.229 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.229 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.229 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.229 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.229 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.230 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.230 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.230 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.230 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.230 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.230 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.230 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.230 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.230 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.230 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.230 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.230 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.230 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.230 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.231 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.231 INFO:teuthology.orchestra.run.smithi117.stdout: "min_size": { 2024-09-21T06:08:43.231 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "min_size", 2024-09-21T06:08:43.231 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.231 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.231 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.231 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.231 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.231 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.231 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.231 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "default min_size for any created local pool", 2024-09-21T06:08:43.231 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T06:08:43.231 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.231 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.231 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.231 INFO:teuthology.orchestra.run.smithi117.stdout: "num_rep": { 2024-09-21T06:08:43.232 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "num_rep", 2024-09-21T06:08:43.232 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.232 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.232 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.232 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "3", 2024-09-21T06:08:43.232 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.232 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.232 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.232 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "default replica count for any created local pool", 2024-09-21T06:08:43.232 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.232 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.232 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.232 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.232 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_num": { 2024-09-21T06:08:43.232 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "pg_num", 2024-09-21T06:08:43.233 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.233 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.233 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.233 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "128", 2024-09-21T06:08:43.233 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.233 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.233 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.233 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T06:08:43.233 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.233 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.233 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.233 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.233 INFO:teuthology.orchestra.run.smithi117.stdout: "prefix": { 2024-09-21T06:08:43.233 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "prefix", 2024-09-21T06:08:43.233 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.233 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.234 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.234 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.234 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.234 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.234 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.234 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "name prefix for any created local pool", 2024-09-21T06:08:43.234 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.234 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.234 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.234 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.234 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.234 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.234 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.234 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.234 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.235 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.235 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.235 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.235 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.235 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.235 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.235 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.235 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.235 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.235 INFO:teuthology.orchestra.run.smithi117.stdout: "subtree": { 2024-09-21T06:08:43.235 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "subtree", 2024-09-21T06:08:43.235 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.235 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.235 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.235 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "rack", 2024-09-21T06:08:43.236 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.236 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.236 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.236 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T06:08:43.236 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T06:08:43.236 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.236 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.236 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.236 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.236 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.236 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.236 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mds_autoscaler", 2024-09-21T06:08:43.236 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.236 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.236 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.237 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.237 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.237 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.237 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.237 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.237 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.237 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.237 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.237 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.237 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.237 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.237 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.237 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.237 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.237 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.238 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.238 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.238 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.238 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.238 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.238 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.238 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.238 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.238 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.238 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.238 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.238 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.238 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.238 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.239 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.239 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.239 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.239 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.239 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.239 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.239 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.239 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.239 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.239 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.239 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.239 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.239 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.239 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.240 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.240 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.240 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.240 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.240 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.240 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.240 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.240 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.240 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.240 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.240 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.240 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.240 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.240 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.240 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.241 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.241 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.241 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.241 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.241 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.241 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.241 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.241 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.241 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.241 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.241 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.241 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.241 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.241 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.241 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.241 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.242 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.242 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.242 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.242 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.242 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.242 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.242 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.242 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.242 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.242 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.242 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.242 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.242 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.242 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mirroring", 2024-09-21T06:08:43.242 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.243 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.243 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.243 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.243 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.243 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.243 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.243 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.243 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.243 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.243 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.243 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.243 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.243 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.243 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.244 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.244 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.244 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.244 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.244 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.244 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.244 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.244 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.244 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.244 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.244 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.244 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.244 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.244 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.244 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.244 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.245 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.245 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.245 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.245 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.245 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.245 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.245 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.245 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.245 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.245 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.245 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.245 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.245 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.245 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.245 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.246 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.246 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.246 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.246 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.246 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.246 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.246 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.246 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.246 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.246 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.246 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.246 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.246 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.246 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.246 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.247 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.248 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.248 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.248 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.248 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.248 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.248 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.248 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.248 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.248 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.248 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.248 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.248 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.248 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.248 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "nfs", 2024-09-21T06:08:43.248 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.248 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.249 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.249 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.249 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.249 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.249 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.249 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.249 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.249 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.249 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.249 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.249 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.249 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.249 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.249 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.249 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.249 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.250 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.250 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.250 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.250 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.250 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.250 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.250 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.250 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.250 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.250 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.250 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.250 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.250 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.250 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.250 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.251 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.251 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.251 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.251 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.251 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.251 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.251 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.251 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.251 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.251 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.251 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.251 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.251 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.251 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.252 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.252 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.252 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.252 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.252 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.252 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.252 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.252 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.252 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.252 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.252 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.252 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.252 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.252 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.252 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.253 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.253 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.253 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.253 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.253 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.253 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.253 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.253 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.253 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.253 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.253 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.253 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.253 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.253 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.253 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.253 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.254 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.254 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.254 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.254 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.254 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.254 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.254 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.254 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.254 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.254 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.254 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.254 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.254 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "orchestrator", 2024-09-21T06:08:43.254 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.255 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.255 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.255 INFO:teuthology.orchestra.run.smithi117.stdout: "fail_fs": { 2024-09-21T06:08:43.255 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "fail_fs", 2024-09-21T06:08:43.255 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.255 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.255 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.255 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.255 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.255 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.255 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.255 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T06:08:43.255 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.255 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.255 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.255 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.256 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.256 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.256 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.256 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.256 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.256 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.256 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.256 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.256 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.256 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.256 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.256 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.256 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.256 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.256 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.257 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.257 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.257 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.257 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.257 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.257 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.257 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.257 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.257 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.257 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.257 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.257 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.257 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.257 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.257 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.258 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.258 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.258 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.258 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.258 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.258 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.258 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.258 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.258 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.258 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.258 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.258 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.258 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.258 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.258 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.258 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.259 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.259 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.259 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.259 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.259 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.259 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.259 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.259 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.259 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.259 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.259 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.259 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.259 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.259 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.259 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.260 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.260 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.260 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.260 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.260 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.260 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.260 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.260 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.260 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.260 INFO:teuthology.orchestra.run.smithi117.stdout: "orchestrator": { 2024-09-21T06:08:43.260 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "orchestrator", 2024-09-21T06:08:43.260 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.260 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.260 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.260 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.260 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.261 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.261 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.261 INFO:teuthology.orchestra.run.smithi117.stdout: "cephadm", 2024-09-21T06:08:43.261 INFO:teuthology.orchestra.run.smithi117.stdout: "rook", 2024-09-21T06:08:43.261 INFO:teuthology.orchestra.run.smithi117.stdout: "test_orchestrator" 2024-09-21T06:08:43.261 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.261 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Orchestrator backend", 2024-09-21T06:08:43.261 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.261 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.261 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.261 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.261 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.261 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.261 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.261 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.262 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.262 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.262 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.262 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.262 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.262 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.262 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.262 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.262 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.262 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.262 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.262 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.262 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.262 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "osd_perf_query", 2024-09-21T06:08:43.262 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.263 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.263 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.263 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.263 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.263 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.263 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.263 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.263 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.263 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.263 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.263 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.263 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.263 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.263 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.263 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.263 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.264 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.264 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.264 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.264 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.264 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.264 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.264 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.264 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.264 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.264 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.264 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.264 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.264 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.265 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.265 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.265 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.265 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.265 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.265 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.265 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.265 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.265 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.265 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.265 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.265 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.265 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.265 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.266 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.266 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.266 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.266 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.266 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.266 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.266 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.266 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.266 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.266 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.266 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.266 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.266 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.266 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.267 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.267 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.267 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.267 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.267 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.267 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.267 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.267 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.267 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.267 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.267 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.267 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.267 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.267 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.267 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.268 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.268 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.268 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.268 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.268 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.268 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.268 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.268 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.268 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.268 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.268 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.268 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.268 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.269 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.269 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.269 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.269 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.269 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "osd_support", 2024-09-21T06:08:43.269 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.269 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.269 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.269 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.269 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.269 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.269 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.269 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.269 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.270 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.270 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.270 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.270 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.270 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.270 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.270 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.270 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.270 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.270 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.270 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.270 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.270 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.270 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.271 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.271 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.271 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.271 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.271 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.271 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.271 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.271 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.271 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.271 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.271 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.271 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.271 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.271 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.272 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.272 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.272 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.272 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.272 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.272 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.272 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.272 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.272 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.272 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.272 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.272 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.272 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.272 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.273 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.273 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.273 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.273 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.273 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.273 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.273 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.273 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.273 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.273 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.273 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.273 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.273 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.273 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.274 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.274 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.274 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.274 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.274 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.274 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.274 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.274 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.274 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.274 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.274 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.274 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.274 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.275 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.275 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.275 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.275 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.275 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.275 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.275 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.275 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.275 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.275 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.275 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.275 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.275 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "pg_autoscaler", 2024-09-21T06:08:43.275 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.275 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.275 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.276 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.276 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.276 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.276 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.276 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.276 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.276 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.276 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.276 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.276 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.276 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.276 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.276 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.276 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.276 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.277 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.277 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.277 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.277 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.277 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.277 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.277 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.277 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.277 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.277 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.277 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.277 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.277 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.277 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.277 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.277 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.277 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.278 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.278 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.278 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.278 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.278 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.278 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.278 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.278 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.278 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.278 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.278 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.278 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.278 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.278 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.279 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.279 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.279 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.279 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.279 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.279 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.279 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.279 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.279 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.279 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.279 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.279 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.279 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.279 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.279 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.280 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.280 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.280 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.280 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.280 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.280 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.280 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.280 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.280 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.280 INFO:teuthology.orchestra.run.smithi117.stdout: "sleep_interval": { 2024-09-21T06:08:43.280 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sleep_interval", 2024-09-21T06:08:43.280 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.280 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.280 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.281 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "60", 2024-09-21T06:08:43.281 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.281 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.281 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.281 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.281 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.281 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.281 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.281 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.281 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.281 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.281 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.281 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.281 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.281 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.282 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.282 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.282 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.282 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.282 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.282 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.282 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.282 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.282 INFO:teuthology.orchestra.run.smithi117.stdout: "threshold": { 2024-09-21T06:08:43.282 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "threshold", 2024-09-21T06:08:43.282 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-09-21T06:08:43.282 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.282 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.282 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "3.0", 2024-09-21T06:08:43.282 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "1.0", 2024-09-21T06:08:43.283 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.283 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.283 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "scaling threshold", 2024-09-21T06:08:43.283 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T06:08:43.283 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.283 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.283 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.283 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.283 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.283 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.283 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "progress", 2024-09-21T06:08:43.283 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.283 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.283 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.284 INFO:teuthology.orchestra.run.smithi117.stdout: "allow_pg_recovery_event": { 2024-09-21T06:08:43.284 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "allow_pg_recovery_event", 2024-09-21T06:08:43.284 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.284 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.284 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.284 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.284 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.284 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.284 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.284 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T06:08:43.284 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.284 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.284 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.284 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.284 INFO:teuthology.orchestra.run.smithi117.stdout: "enabled": { 2024-09-21T06:08:43.285 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "enabled", 2024-09-21T06:08:43.285 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.285 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.285 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.285 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.285 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.285 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.285 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.285 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.285 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.285 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.285 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.285 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.285 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.285 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.285 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.286 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.286 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.286 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.286 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.286 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.286 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.286 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.286 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.286 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.286 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.286 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.286 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.286 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.286 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.286 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.287 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.287 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.287 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.287 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.287 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.287 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.287 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.287 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.287 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.287 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.287 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.287 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.287 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.287 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.287 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.287 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.288 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.288 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.288 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.288 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.288 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.288 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.288 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.288 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.288 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.288 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.288 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.288 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.288 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.288 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.288 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.289 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.289 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.289 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.289 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.289 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.289 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.289 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.289 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.289 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.289 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.289 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.289 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.289 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.289 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.289 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.289 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.290 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.290 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.290 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.290 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.290 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.290 INFO:teuthology.orchestra.run.smithi117.stdout: "max_completed_events": { 2024-09-21T06:08:43.290 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "max_completed_events", 2024-09-21T06:08:43.290 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.290 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.290 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.290 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "50", 2024-09-21T06:08:43.290 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.290 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.290 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.291 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "number of past completed events to remember", 2024-09-21T06:08:43.291 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.291 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.291 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.291 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.291 INFO:teuthology.orchestra.run.smithi117.stdout: "sleep_interval": { 2024-09-21T06:08:43.291 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sleep_interval", 2024-09-21T06:08:43.291 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.291 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.291 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.291 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "5", 2024-09-21T06:08:43.291 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.291 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.291 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.291 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how long the module is going to sleep", 2024-09-21T06:08:43.292 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.292 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.292 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.292 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.292 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.292 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.292 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.292 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.292 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.292 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.292 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.292 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.292 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.292 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.292 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.293 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.293 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.293 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.293 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.293 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.293 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.293 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "prometheus", 2024-09-21T06:08:43.293 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.293 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.293 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.293 INFO:teuthology.orchestra.run.smithi117.stdout: "cache": { 2024-09-21T06:08:43.293 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "cache", 2024-09-21T06:08:43.293 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.293 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.293 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.294 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.294 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.294 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.294 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.294 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.294 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.294 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.294 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.294 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.294 INFO:teuthology.orchestra.run.smithi117.stdout: "exclude_perf_counters": { 2024-09-21T06:08:43.294 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "exclude_perf_counters", 2024-09-21T06:08:43.294 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.294 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.294 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.294 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.295 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.295 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.295 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.295 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T06:08:43.295 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T06:08:43.295 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.295 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.295 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.295 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.295 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.295 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.295 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.295 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.295 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.295 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.296 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.296 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.296 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.296 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.296 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.296 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.296 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.296 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.296 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.296 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.296 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.296 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.296 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.296 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.296 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.297 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.297 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.297 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.297 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.297 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.297 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.297 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.297 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.297 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.297 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.297 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.297 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.297 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.297 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.298 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.298 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.298 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.298 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.298 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.298 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.298 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.298 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.298 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.298 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.298 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.298 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.298 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.298 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.298 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.299 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.299 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.299 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.299 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.299 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.299 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.299 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.299 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.299 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.299 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.299 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.299 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.299 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.299 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.299 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.300 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.300 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.300 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.300 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.300 INFO:teuthology.orchestra.run.smithi117.stdout: "rbd_stats_pools": { 2024-09-21T06:08:43.300 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rbd_stats_pools", 2024-09-21T06:08:43.300 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.300 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.300 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.300 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.300 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.300 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.300 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.300 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.300 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.301 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.301 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.301 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.301 INFO:teuthology.orchestra.run.smithi117.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T06:08:43.301 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T06:08:43.301 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.301 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.301 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.301 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "300", 2024-09-21T06:08:43.301 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.301 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.301 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.301 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.301 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.301 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.302 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.302 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.302 INFO:teuthology.orchestra.run.smithi117.stdout: "scrape_interval": { 2024-09-21T06:08:43.302 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "scrape_interval", 2024-09-21T06:08:43.302 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-09-21T06:08:43.302 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.302 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.302 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "15.0", 2024-09-21T06:08:43.302 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.302 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.302 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.302 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.302 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.302 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.302 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.303 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.303 INFO:teuthology.orchestra.run.smithi117.stdout: "server_addr": { 2024-09-21T06:08:43.303 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_addr", 2024-09-21T06:08:43.303 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.303 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.303 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.303 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "::", 2024-09-21T06:08:43.303 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.303 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.303 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.303 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T06:08:43.303 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.303 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.303 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.304 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.304 INFO:teuthology.orchestra.run.smithi117.stdout: "server_port": { 2024-09-21T06:08:43.304 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_port", 2024-09-21T06:08:43.304 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.304 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.304 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.304 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "9283", 2024-09-21T06:08:43.304 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.304 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.304 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.304 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T06:08:43.304 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.304 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.304 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.304 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.305 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.305 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.305 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.305 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.305 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.305 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.305 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.305 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.305 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.305 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.305 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.305 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.305 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.305 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.306 INFO:teuthology.orchestra.run.smithi117.stdout: "stale_cache_strategy": { 2024-09-21T06:08:43.306 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "stale_cache_strategy", 2024-09-21T06:08:43.306 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.306 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.306 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.306 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "log", 2024-09-21T06:08:43.306 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.306 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.306 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.306 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.306 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.306 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.306 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.306 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.307 INFO:teuthology.orchestra.run.smithi117.stdout: "standby_behaviour": { 2024-09-21T06:08:43.307 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "standby_behaviour", 2024-09-21T06:08:43.307 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.307 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.307 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.307 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "default", 2024-09-21T06:08:43.307 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.307 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.307 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.307 INFO:teuthology.orchestra.run.smithi117.stdout: "default", 2024-09-21T06:08:43.307 INFO:teuthology.orchestra.run.smithi117.stdout: "error" 2024-09-21T06:08:43.307 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.307 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.307 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.307 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.308 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.308 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.308 INFO:teuthology.orchestra.run.smithi117.stdout: "standby_error_status_code": { 2024-09-21T06:08:43.308 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "standby_error_status_code", 2024-09-21T06:08:43.308 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.308 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.308 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.308 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "500", 2024-09-21T06:08:43.308 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "400", 2024-09-21T06:08:43.308 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "599", 2024-09-21T06:08:43.308 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.308 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.308 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.308 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.308 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.309 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.309 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.309 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.309 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.309 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rbd_support", 2024-09-21T06:08:43.309 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.309 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.309 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.309 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.309 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.309 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.309 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.309 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.309 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.309 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.309 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.310 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.310 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.310 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.310 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.310 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.310 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.310 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.310 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.310 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.310 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.310 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.310 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.310 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.310 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.310 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.311 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.311 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.311 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.311 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.311 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.311 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.311 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.311 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.311 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.311 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.311 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.311 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.311 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.311 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.311 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.312 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.312 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.312 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.312 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.312 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.312 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.312 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.312 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.312 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.312 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.312 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.312 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.312 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.312 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.312 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.312 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.313 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.313 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.313 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.313 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.313 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.313 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.313 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.313 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.313 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.313 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.313 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.313 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.313 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.313 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.313 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.314 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.314 INFO:teuthology.orchestra.run.smithi117.stdout: "max_concurrent_snap_create": { 2024-09-21T06:08:43.314 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "max_concurrent_snap_create", 2024-09-21T06:08:43.314 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.314 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.314 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.314 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-09-21T06:08:43.314 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.314 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.314 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.314 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.314 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.314 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.314 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.314 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.315 INFO:teuthology.orchestra.run.smithi117.stdout: "mirror_snapshot_schedule": { 2024-09-21T06:08:43.315 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T06:08:43.315 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.315 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.315 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.315 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.315 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.315 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.315 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.315 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.315 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.315 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.315 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.315 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.315 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.316 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.316 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.316 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.316 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.316 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.316 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.316 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.316 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.316 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.316 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.316 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.316 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.316 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.316 INFO:teuthology.orchestra.run.smithi117.stdout: "trash_purge_schedule": { 2024-09-21T06:08:43.316 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "trash_purge_schedule", 2024-09-21T06:08:43.317 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.317 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.317 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.317 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.317 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.317 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.317 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.317 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.317 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.317 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.317 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.317 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.317 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.317 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.317 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.317 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "restful", 2024-09-21T06:08:43.318 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.318 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.318 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.318 INFO:teuthology.orchestra.run.smithi117.stdout: "enable_auth": { 2024-09-21T06:08:43.318 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "enable_auth", 2024-09-21T06:08:43.318 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.318 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.318 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.318 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.318 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.318 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.318 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.318 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.318 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.318 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.319 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.319 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.319 INFO:teuthology.orchestra.run.smithi117.stdout: "key_file": { 2024-09-21T06:08:43.319 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "key_file", 2024-09-21T06:08:43.319 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.319 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.319 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.319 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.319 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.319 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.319 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.319 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.319 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.319 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.319 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.320 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.320 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.320 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.320 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.320 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.320 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.320 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.320 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.320 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.320 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.320 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.320 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.320 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.320 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.320 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.320 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.321 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.321 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.321 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.321 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.321 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.321 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.321 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.321 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.321 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.321 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.321 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.321 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.321 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.321 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.321 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.322 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.322 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.322 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.322 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.322 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.322 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.322 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.322 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.322 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.322 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.322 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.322 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.322 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.322 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.322 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.323 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.323 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.323 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.323 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.323 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.323 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.323 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.323 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.323 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.323 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.323 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.323 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.323 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.323 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.323 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.323 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.324 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.324 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.324 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.324 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.324 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.324 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.324 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.324 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.324 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.324 INFO:teuthology.orchestra.run.smithi117.stdout: "max_requests": { 2024-09-21T06:08:43.324 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "max_requests", 2024-09-21T06:08:43.324 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.324 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.324 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.324 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "500", 2024-09-21T06:08:43.325 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.325 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.325 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.325 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T06:08:43.325 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.325 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.325 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.325 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.325 INFO:teuthology.orchestra.run.smithi117.stdout: "server_addr": { 2024-09-21T06:08:43.325 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_addr", 2024-09-21T06:08:43.325 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.325 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.325 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.325 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.325 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.325 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.326 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.326 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.326 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.326 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.326 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.326 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.326 INFO:teuthology.orchestra.run.smithi117.stdout: "server_port": { 2024-09-21T06:08:43.326 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_port", 2024-09-21T06:08:43.326 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.326 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.326 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.326 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.326 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.326 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.326 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.326 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.327 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.327 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.327 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.327 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.327 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.327 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.327 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.327 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.327 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.327 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.327 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.327 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.327 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.327 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.327 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.328 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.328 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.328 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.328 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.328 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.328 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.328 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rgw", 2024-09-21T06:08:43.328 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.328 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.328 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.328 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.328 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.328 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.329 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.329 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.329 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.329 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.329 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.329 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.329 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.329 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.329 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.329 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.329 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.329 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.329 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.329 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.329 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.330 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.330 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.330 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.330 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.330 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.330 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.330 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.330 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.330 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.330 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.330 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.330 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.330 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.330 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.331 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.331 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.331 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.331 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.331 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.331 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.331 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.331 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.331 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.331 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.331 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.331 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.331 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.331 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.331 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.331 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.332 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.332 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.332 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.332 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.332 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.332 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.332 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.332 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.332 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.332 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.332 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.332 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.332 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.332 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.332 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.332 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.332 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.333 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.333 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.333 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.333 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.333 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.333 INFO:teuthology.orchestra.run.smithi117.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T06:08:43.333 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T06:08:43.333 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.333 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.333 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.333 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "5", 2024-09-21T06:08:43.333 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.333 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.333 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.334 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T06:08:43.334 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.334 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.334 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.334 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.334 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.334 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.334 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.334 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.334 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.334 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.334 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.334 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.334 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.334 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.335 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.335 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.335 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.335 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.335 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.335 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.335 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.335 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rook", 2024-09-21T06:08:43.335 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.335 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.335 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.335 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.335 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.335 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.335 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.336 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.336 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.336 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.336 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.336 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.336 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.336 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.336 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.336 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.336 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.336 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.336 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.336 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.336 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.336 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.337 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.337 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.337 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.337 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.337 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.337 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.337 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.337 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.337 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.337 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.337 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.337 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.337 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.337 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.337 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.337 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.338 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.338 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.338 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.338 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.338 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.338 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.338 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.338 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.338 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.338 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.338 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.338 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.338 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.338 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.339 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.339 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.339 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.339 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.339 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.339 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.339 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.339 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.339 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.339 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.339 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.339 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.339 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.339 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.339 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.340 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.340 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.340 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.340 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.340 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.340 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.340 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.340 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.340 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.340 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.340 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.340 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.340 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.340 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.340 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.341 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.341 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.341 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.341 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.341 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.341 INFO:teuthology.orchestra.run.smithi117.stdout: "storage_class": { 2024-09-21T06:08:43.341 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "storage_class", 2024-09-21T06:08:43.341 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.341 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.341 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.341 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "local", 2024-09-21T06:08:43.341 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.341 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.341 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.342 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T06:08:43.342 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.342 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.342 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.342 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.342 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.342 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.342 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.342 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "selftest", 2024-09-21T06:08:43.342 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.342 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.342 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.342 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.342 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.342 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.343 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.343 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.343 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.343 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.343 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.343 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.343 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.343 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.343 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.343 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.343 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.343 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.343 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.344 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.344 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.344 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.344 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.344 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.344 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.344 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.344 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.344 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.344 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.344 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.344 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.344 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.344 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.344 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.345 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.345 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.345 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.345 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.345 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.345 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.345 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.345 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.345 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.345 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.345 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.345 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.345 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.345 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.346 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.346 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.346 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.346 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.346 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.346 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.346 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.346 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.346 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.346 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.346 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.346 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.346 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.346 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.346 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.347 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.347 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.347 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.347 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.347 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.347 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.347 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.347 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.347 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.347 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.347 INFO:teuthology.orchestra.run.smithi117.stdout: "roption1": { 2024-09-21T06:08:43.347 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "roption1", 2024-09-21T06:08:43.347 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.347 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.348 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.348 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.348 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.348 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.348 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.348 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.348 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.348 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.348 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.348 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.348 INFO:teuthology.orchestra.run.smithi117.stdout: "roption2": { 2024-09-21T06:08:43.348 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "roption2", 2024-09-21T06:08:43.348 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.348 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.348 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.349 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "xyz", 2024-09-21T06:08:43.349 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.349 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.349 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.349 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.349 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.349 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.349 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.349 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.349 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption1": { 2024-09-21T06:08:43.349 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption1", 2024-09-21T06:08:43.349 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.349 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.349 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.349 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.350 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.350 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.350 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.350 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.350 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.350 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.350 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.350 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.350 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption2": { 2024-09-21T06:08:43.350 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption2", 2024-09-21T06:08:43.350 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.350 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.350 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.350 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.350 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.351 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.351 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.351 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.351 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.351 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.351 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.351 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.351 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption3": { 2024-09-21T06:08:43.351 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption3", 2024-09-21T06:08:43.351 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-09-21T06:08:43.351 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.351 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.351 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.351 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.351 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.352 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.352 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.352 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.352 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.352 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.352 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.352 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption4": { 2024-09-21T06:08:43.352 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption4", 2024-09-21T06:08:43.352 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.352 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.352 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.352 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.352 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.352 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.352 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.353 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.353 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.353 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.353 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.353 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.353 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption5": { 2024-09-21T06:08:43.353 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption5", 2024-09-21T06:08:43.353 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.353 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.353 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.353 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.353 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.353 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.353 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.354 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.354 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.354 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.354 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.354 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.354 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption6": { 2024-09-21T06:08:43.354 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption6", 2024-09-21T06:08:43.354 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.354 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.354 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.354 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.354 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.354 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.354 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.355 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.355 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.355 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.355 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.355 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.355 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption7": { 2024-09-21T06:08:43.355 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption7", 2024-09-21T06:08:43.355 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.355 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.355 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.355 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.355 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "1", 2024-09-21T06:08:43.355 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "42", 2024-09-21T06:08:43.355 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.355 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.356 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.356 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.356 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.356 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.356 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.356 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.356 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.356 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.356 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.356 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.356 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.356 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.356 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.356 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.356 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.357 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.357 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.357 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.357 INFO:teuthology.orchestra.run.smithi117.stdout: "testkey": { 2024-09-21T06:08:43.357 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "testkey", 2024-09-21T06:08:43.357 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.357 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.357 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.357 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.357 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.357 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.357 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.357 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.357 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.358 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.358 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.358 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.358 INFO:teuthology.orchestra.run.smithi117.stdout: "testlkey": { 2024-09-21T06:08:43.358 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "testlkey", 2024-09-21T06:08:43.358 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.358 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.358 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.358 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.358 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.358 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.358 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.358 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.358 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.358 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.359 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.359 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.359 INFO:teuthology.orchestra.run.smithi117.stdout: "testnewline": { 2024-09-21T06:08:43.359 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "testnewline", 2024-09-21T06:08:43.359 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.359 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.359 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.359 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.359 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.359 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.359 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.359 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.359 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.359 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.359 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.360 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.360 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.360 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.360 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.360 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smb", 2024-09-21T06:08:43.360 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.360 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.360 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.360 INFO:teuthology.orchestra.run.smithi117.stdout: "internal_store_backend": { 2024-09-21T06:08:43.360 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "internal_store_backend", 2024-09-21T06:08:43.360 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.360 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.360 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.360 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.360 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.361 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.361 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.361 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T06:08:43.361 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.361 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.361 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.361 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.361 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.361 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.361 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.361 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.361 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.361 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.361 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.361 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.362 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.362 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.362 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.362 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.362 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.362 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.362 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.362 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.362 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.362 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.362 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.362 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.362 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.362 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.362 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.363 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.363 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.363 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.363 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.363 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.363 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.363 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.363 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.363 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.363 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.363 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.363 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.363 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.363 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.363 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.364 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.364 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.364 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.364 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.364 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.364 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.364 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.364 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.364 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.364 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.364 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.364 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.364 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.364 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.364 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.365 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.365 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.365 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.365 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.365 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.365 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.365 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.365 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.365 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.365 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.365 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.365 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.365 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.365 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.365 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.366 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.366 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.366 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.366 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.366 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.366 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.366 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.366 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.366 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.366 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.366 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.366 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.366 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.366 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.367 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.367 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.367 INFO:teuthology.orchestra.run.smithi117.stdout: "update_orchestration": { 2024-09-21T06:08:43.367 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "update_orchestration", 2024-09-21T06:08:43.367 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.367 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.367 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.367 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.367 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.367 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.367 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.367 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T06:08:43.367 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.367 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.367 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.368 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.368 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.368 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.368 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.368 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "snap_schedule", 2024-09-21T06:08:43.368 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.368 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.368 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.368 INFO:teuthology.orchestra.run.smithi117.stdout: "allow_m_granularity": { 2024-09-21T06:08:43.368 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "allow_m_granularity", 2024-09-21T06:08:43.368 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.368 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.368 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.368 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.369 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.369 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.369 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.369 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T06:08:43.369 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.369 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.369 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.369 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.369 INFO:teuthology.orchestra.run.smithi117.stdout: "dump_on_update": { 2024-09-21T06:08:43.369 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "dump_on_update", 2024-09-21T06:08:43.369 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.369 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.369 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.369 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.369 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.370 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.370 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.370 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "dump database to debug log on update", 2024-09-21T06:08:43.370 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.370 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.370 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.370 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.370 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.370 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.370 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.370 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.370 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.370 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.370 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.371 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.371 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.371 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.371 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.371 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.371 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.371 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.371 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.371 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.371 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.371 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.371 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.371 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.371 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.371 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.372 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.372 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.372 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.372 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.372 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.372 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.372 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.372 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.372 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.372 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.372 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.372 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.372 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.372 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.373 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.373 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.373 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.373 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.373 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.373 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.373 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.373 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.373 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.373 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.373 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.373 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.373 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.373 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.374 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.374 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.374 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.374 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.374 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.374 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.374 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.374 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.374 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.374 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.374 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.374 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.374 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.374 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.374 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.375 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.376 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.376 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.376 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.376 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.376 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.376 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.376 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.376 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.376 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "stats", 2024-09-21T06:08:43.376 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.376 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.376 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.376 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.376 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.377 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.377 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.377 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.377 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.377 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.377 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.377 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.377 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.377 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.377 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.377 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.377 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.377 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.377 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.377 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.377 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.378 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.378 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.378 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.378 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.378 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.378 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.378 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.378 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.378 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.378 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.378 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.378 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.378 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.378 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.378 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.378 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.379 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.379 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.379 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.379 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.379 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.379 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.379 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.379 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.379 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.379 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.379 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.379 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.379 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.379 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.379 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.380 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.380 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.380 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.380 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.380 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.380 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.380 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.380 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.380 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.380 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.380 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.380 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.380 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.380 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.381 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.381 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.381 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.381 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.381 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.381 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.381 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.381 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.381 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.381 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.381 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.381 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.381 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.381 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.381 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.382 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.382 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.382 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.382 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.382 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.382 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.382 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.382 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.382 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.382 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "status", 2024-09-21T06:08:43.382 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.382 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.382 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.382 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.382 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.383 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.383 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.383 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.383 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.383 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.383 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.383 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.383 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.383 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.383 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.383 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.383 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.383 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.383 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.383 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.384 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.384 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.384 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.384 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.384 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.384 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.384 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.384 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.384 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.384 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.384 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.384 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.384 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.384 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.385 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.385 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.385 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.385 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.385 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.385 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.385 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.385 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.385 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.385 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.385 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.385 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.385 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.385 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.386 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.386 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.386 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.386 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.386 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.386 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.386 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.386 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.386 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.386 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.386 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.386 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.386 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.386 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.387 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.387 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.387 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.387 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.387 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.387 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.387 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.387 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.387 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.387 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.387 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.387 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.387 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.387 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.387 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.388 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.388 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.388 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.388 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.388 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.388 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.388 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.388 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.388 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.388 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.388 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.388 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.388 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.388 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "telegraf", 2024-09-21T06:08:43.389 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.389 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.389 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.389 INFO:teuthology.orchestra.run.smithi117.stdout: "address": { 2024-09-21T06:08:43.389 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "address", 2024-09-21T06:08:43.389 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.389 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.389 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.389 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T06:08:43.389 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.389 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.389 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.389 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.389 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.389 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.390 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.390 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.390 INFO:teuthology.orchestra.run.smithi117.stdout: "interval": { 2024-09-21T06:08:43.390 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "interval", 2024-09-21T06:08:43.390 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.390 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.390 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.390 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "15", 2024-09-21T06:08:43.390 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.390 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.390 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.390 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.390 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.390 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.390 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.391 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.391 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.391 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.391 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.391 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.391 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.391 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.391 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.391 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.391 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.391 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.391 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.391 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.391 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.391 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.392 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.392 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.392 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.392 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.392 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.392 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.392 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.392 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.392 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.392 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.392 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.392 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.392 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.392 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.392 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.393 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.393 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.393 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.393 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.393 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.393 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.393 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.393 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.393 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.393 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.393 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.393 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.393 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.393 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.393 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.393 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.394 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.394 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.394 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.394 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.394 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.394 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.394 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.394 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.394 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.394 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.394 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.394 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.394 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.394 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.395 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.395 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.395 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.395 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.395 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.395 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.395 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.395 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.395 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.395 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.395 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.395 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.395 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.395 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.396 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.396 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.396 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.396 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.396 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.396 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.396 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.396 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.396 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.396 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.396 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.396 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.396 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.396 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.396 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "telemetry", 2024-09-21T06:08:43.397 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.397 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.397 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.397 INFO:teuthology.orchestra.run.smithi117.stdout: "channel_basic": { 2024-09-21T06:08:43.397 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "channel_basic", 2024-09-21T06:08:43.397 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.397 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.397 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.397 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.397 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.397 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.397 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.397 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T06:08:43.397 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.397 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.398 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.398 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.398 INFO:teuthology.orchestra.run.smithi117.stdout: "channel_crash": { 2024-09-21T06:08:43.398 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "channel_crash", 2024-09-21T06:08:43.398 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.398 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.398 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.398 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.398 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.398 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.398 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.398 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T06:08:43.398 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.398 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.399 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.399 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.399 INFO:teuthology.orchestra.run.smithi117.stdout: "channel_device": { 2024-09-21T06:08:43.399 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "channel_device", 2024-09-21T06:08:43.399 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.399 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.399 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.399 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.399 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.399 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.399 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.399 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T06:08:43.399 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.399 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.399 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.400 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.400 INFO:teuthology.orchestra.run.smithi117.stdout: "channel_ident": { 2024-09-21T06:08:43.400 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "channel_ident", 2024-09-21T06:08:43.400 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.400 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.400 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.400 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.400 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.400 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.400 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.400 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T06:08:43.400 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.400 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.400 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.400 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.400 INFO:teuthology.orchestra.run.smithi117.stdout: "channel_perf": { 2024-09-21T06:08:43.401 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "channel_perf", 2024-09-21T06:08:43.401 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.401 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.401 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.401 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.401 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.401 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.401 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.401 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T06:08:43.401 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.401 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.401 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.401 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.401 INFO:teuthology.orchestra.run.smithi117.stdout: "contact": { 2024-09-21T06:08:43.401 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "contact", 2024-09-21T06:08:43.402 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.402 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.402 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.402 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.402 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.402 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.402 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.402 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.402 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.402 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.402 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.402 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.402 INFO:teuthology.orchestra.run.smithi117.stdout: "description": { 2024-09-21T06:08:43.402 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "description", 2024-09-21T06:08:43.402 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.403 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.403 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.403 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.403 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.403 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.403 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.403 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.403 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.403 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.403 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.403 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.403 INFO:teuthology.orchestra.run.smithi117.stdout: "device_url": { 2024-09-21T06:08:43.403 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "device_url", 2024-09-21T06:08:43.403 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.403 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.404 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.404 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T06:08:43.404 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.404 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.404 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.404 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.404 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.404 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.404 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.404 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.404 INFO:teuthology.orchestra.run.smithi117.stdout: "enabled": { 2024-09-21T06:08:43.404 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "enabled", 2024-09-21T06:08:43.404 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.404 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.404 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.404 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.405 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.405 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.405 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.405 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.405 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.405 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.405 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.405 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.405 INFO:teuthology.orchestra.run.smithi117.stdout: "interval": { 2024-09-21T06:08:43.405 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "interval", 2024-09-21T06:08:43.405 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.405 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.405 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.405 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "24", 2024-09-21T06:08:43.405 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "8", 2024-09-21T06:08:43.406 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.406 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.406 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.406 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.406 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.406 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.406 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.406 INFO:teuthology.orchestra.run.smithi117.stdout: "last_opt_revision": { 2024-09-21T06:08:43.406 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "last_opt_revision", 2024-09-21T06:08:43.406 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.406 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.406 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.406 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "1", 2024-09-21T06:08:43.406 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.407 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.407 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.407 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.407 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.407 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.407 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.407 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.407 INFO:teuthology.orchestra.run.smithi117.stdout: "leaderboard": { 2024-09-21T06:08:43.407 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "leaderboard", 2024-09-21T06:08:43.407 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.407 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.407 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.407 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.407 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.408 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.408 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.408 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.408 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.408 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.408 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.408 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.408 INFO:teuthology.orchestra.run.smithi117.stdout: "leaderboard_description": { 2024-09-21T06:08:43.408 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "leaderboard_description", 2024-09-21T06:08:43.408 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.408 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.408 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.408 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.408 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.408 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.408 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.409 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.409 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.409 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.409 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.409 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.409 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.409 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.409 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.409 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.409 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.409 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.409 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.409 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.409 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.409 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.410 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.410 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.410 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.410 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.410 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.410 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.410 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.410 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.410 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.410 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.410 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.410 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.410 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.410 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.411 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.411 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.411 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.411 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.411 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.411 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.411 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.411 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.411 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.411 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.411 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.411 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.411 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.411 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.412 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.412 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.412 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.412 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.412 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.412 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.412 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.412 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.412 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.412 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.412 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.412 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.412 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.412 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.412 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.413 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.413 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.413 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.413 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.413 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.413 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.413 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.413 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.413 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.413 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.413 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.413 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.413 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.413 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.413 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.413 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.414 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.414 INFO:teuthology.orchestra.run.smithi117.stdout: "organization": { 2024-09-21T06:08:43.414 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "organization", 2024-09-21T06:08:43.414 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.414 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.414 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.414 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.414 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.414 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.414 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.414 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.414 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.414 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.414 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.414 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.415 INFO:teuthology.orchestra.run.smithi117.stdout: "proxy": { 2024-09-21T06:08:43.415 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "proxy", 2024-09-21T06:08:43.415 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.415 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.415 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.415 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.415 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.415 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.415 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.415 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.415 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.415 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.415 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.415 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.415 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.416 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.416 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.416 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.416 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.416 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.416 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.416 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.416 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.416 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.416 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.416 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.416 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.416 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.416 INFO:teuthology.orchestra.run.smithi117.stdout: "url": { 2024-09-21T06:08:43.416 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "url", 2024-09-21T06:08:43.416 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.417 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.417 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.417 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T06:08:43.417 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.417 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.417 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.417 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.417 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.417 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.417 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.417 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.417 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.417 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.417 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.417 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "test_orchestrator", 2024-09-21T06:08:43.418 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.418 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.418 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.418 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.418 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.418 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.418 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.418 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.418 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.418 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.418 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.418 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.418 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.418 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.418 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.419 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.419 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.419 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.419 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.419 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.419 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.419 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.419 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.419 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.419 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.419 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.419 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.419 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.419 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.420 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.420 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.420 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.420 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.420 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.420 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.420 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.420 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.420 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.420 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.420 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.420 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.420 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.420 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.420 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.421 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.421 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.421 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.421 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.421 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.421 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.421 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.421 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.421 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.421 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.421 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.421 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.421 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.421 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.421 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.422 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.422 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.422 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.422 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.422 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.422 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.422 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.422 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.422 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.422 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.422 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.422 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.422 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.422 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.422 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.423 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.423 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.423 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.423 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.423 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.423 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.423 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.423 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.423 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.423 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.423 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.423 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.423 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.423 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.423 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.424 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.424 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "volumes", 2024-09-21T06:08:43.424 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.424 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.424 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.424 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.424 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.424 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.424 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.424 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.424 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.424 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.424 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.424 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.424 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.425 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.425 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.425 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.425 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.425 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.425 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.425 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.425 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.425 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.425 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.425 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.425 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.425 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.425 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.425 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.425 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.426 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.426 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.426 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.426 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.426 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.426 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.426 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.426 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.426 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.426 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.426 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.426 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.426 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.426 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.426 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.427 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.427 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.427 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.427 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.427 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.427 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.427 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.427 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.427 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.427 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.427 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.427 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.427 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.427 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.428 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.428 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.428 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.428 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.428 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.428 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.428 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.428 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.428 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.428 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.428 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.428 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.428 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.428 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.428 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.429 INFO:teuthology.orchestra.run.smithi117.stdout: "max_concurrent_clones": { 2024-09-21T06:08:43.429 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "max_concurrent_clones", 2024-09-21T06:08:43.429 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.429 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.429 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.429 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "4", 2024-09-21T06:08:43.429 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.429 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.429 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.429 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T06:08:43.429 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.429 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.429 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.429 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.429 INFO:teuthology.orchestra.run.smithi117.stdout: "periodic_async_work": { 2024-09-21T06:08:43.429 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "periodic_async_work", 2024-09-21T06:08:43.430 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.430 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.430 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.430 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.430 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.430 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.430 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.430 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Periodically check for async work", 2024-09-21T06:08:43.430 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.430 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.430 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.430 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.430 INFO:teuthology.orchestra.run.smithi117.stdout: "snapshot_clone_delay": { 2024-09-21T06:08:43.430 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "snapshot_clone_delay", 2024-09-21T06:08:43.431 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.431 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.431 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.431 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.431 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.431 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.431 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.431 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T06:08:43.431 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.431 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.431 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.431 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.431 INFO:teuthology.orchestra.run.smithi117.stdout: "snapshot_clone_no_wait": { 2024-09-21T06:08:43.431 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T06:08:43.431 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.432 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.432 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.432 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.432 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.432 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.432 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.432 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T06:08:43.432 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.432 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.432 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.432 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.432 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.432 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.432 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.433 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.433 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.433 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.433 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.433 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.433 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.433 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.433 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.433 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.433 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.433 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.433 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.433 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.433 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.434 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "zabbix", 2024-09-21T06:08:43.434 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.434 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.434 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.434 INFO:teuthology.orchestra.run.smithi117.stdout: "discovery_interval": { 2024-09-21T06:08:43.434 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "discovery_interval", 2024-09-21T06:08:43.434 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "uint", 2024-09-21T06:08:43.434 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.434 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.434 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "100", 2024-09-21T06:08:43.434 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.434 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.434 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.434 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.435 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.435 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.435 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.435 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.435 INFO:teuthology.orchestra.run.smithi117.stdout: "identifier": { 2024-09-21T06:08:43.435 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "identifier", 2024-09-21T06:08:43.435 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.435 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.435 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.435 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.435 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.435 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.435 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.435 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.435 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.435 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.436 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.436 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.436 INFO:teuthology.orchestra.run.smithi117.stdout: "interval": { 2024-09-21T06:08:43.436 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "interval", 2024-09-21T06:08:43.436 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.436 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.436 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.436 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "60", 2024-09-21T06:08:43.436 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.436 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.436 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.436 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.436 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.437 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.437 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.437 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.437 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.437 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.437 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.437 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.437 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.437 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.437 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.437 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.437 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.437 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.437 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.437 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.438 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.438 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.438 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.438 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.438 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.438 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.438 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.438 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.438 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.438 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.438 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.438 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.438 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.438 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.438 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.439 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.439 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.439 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.439 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.439 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.439 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.439 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.439 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.439 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.439 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.439 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.439 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.439 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.439 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.439 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.440 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.440 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.440 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.440 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.440 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.440 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.440 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.440 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.440 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.440 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.440 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.440 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.440 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.440 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.440 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.441 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.441 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.441 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.441 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.441 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.441 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.441 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.441 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.441 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.441 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.441 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.441 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.441 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.442 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.442 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.442 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.442 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.442 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.442 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.442 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.442 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.442 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.442 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.442 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.442 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.442 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.442 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.442 INFO:teuthology.orchestra.run.smithi117.stdout: "zabbix_host": { 2024-09-21T06:08:43.443 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "zabbix_host", 2024-09-21T06:08:43.443 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.443 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.443 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.443 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.443 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.443 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.443 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.443 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.443 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.443 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.443 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.443 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.443 INFO:teuthology.orchestra.run.smithi117.stdout: "zabbix_port": { 2024-09-21T06:08:43.444 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "zabbix_port", 2024-09-21T06:08:43.444 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.444 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.444 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.444 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10051", 2024-09-21T06:08:43.444 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.444 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.444 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.444 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.444 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.444 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.444 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.444 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.444 INFO:teuthology.orchestra.run.smithi117.stdout: "zabbix_sender": { 2024-09-21T06:08:43.444 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "zabbix_sender", 2024-09-21T06:08:43.445 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.445 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.445 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.445 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T06:08:43.445 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.445 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.445 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.445 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.445 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.445 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.445 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.445 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.445 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.445 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.445 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-21T06:08:43.446 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.446 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.446 INFO:teuthology.orchestra.run.smithi117.stdout: "modules": [ 2024-09-21T06:08:43.446 INFO:teuthology.orchestra.run.smithi117.stdout: "cephadm", 2024-09-21T06:08:43.446 INFO:teuthology.orchestra.run.smithi117.stdout: "dashboard", 2024-09-21T06:08:43.446 INFO:teuthology.orchestra.run.smithi117.stdout: "iostat", 2024-09-21T06:08:43.446 INFO:teuthology.orchestra.run.smithi117.stdout: "nfs", 2024-09-21T06:08:43.446 INFO:teuthology.orchestra.run.smithi117.stdout: "restful", 2024-09-21T06:08:43.446 INFO:teuthology.orchestra.run.smithi117.stdout: "snap_schedule" 2024-09-21T06:08:43.446 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.446 INFO:teuthology.orchestra.run.smithi117.stdout: "available_modules": [ 2024-09-21T06:08:43.446 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.446 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "alerts", 2024-09-21T06:08:43.446 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.446 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.447 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.447 INFO:teuthology.orchestra.run.smithi117.stdout: "interval": { 2024-09-21T06:08:43.447 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "interval", 2024-09-21T06:08:43.447 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.447 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.447 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.447 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "60", 2024-09-21T06:08:43.447 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.447 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.447 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.447 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T06:08:43.447 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.447 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.447 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.447 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.448 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.448 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.448 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.448 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.448 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.448 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.448 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.448 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.448 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.448 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.448 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.448 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.448 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.448 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.448 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.449 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.449 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.449 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.449 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.449 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.449 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.449 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.449 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.449 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.449 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.449 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.449 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.449 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.449 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.450 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.450 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.450 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.450 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.450 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.450 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.450 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.450 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.450 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.450 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.450 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.450 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.450 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.450 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.451 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.451 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.451 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.451 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.451 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.451 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.451 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.451 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.451 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.451 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.451 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.451 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.451 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.451 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.451 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.452 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.452 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.452 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.452 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.452 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.452 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.452 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.452 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.452 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.452 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.452 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.452 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.452 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_destination": { 2024-09-21T06:08:43.452 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_destination", 2024-09-21T06:08:43.452 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.453 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.453 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.453 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.453 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.453 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.453 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.453 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T06:08:43.453 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.453 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.453 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.453 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.453 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_from_name": { 2024-09-21T06:08:43.453 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_from_name", 2024-09-21T06:08:43.453 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.453 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.453 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.454 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "Ceph", 2024-09-21T06:08:43.454 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.454 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.454 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.454 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Email From: name", 2024-09-21T06:08:43.454 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.454 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.454 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.454 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.454 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_host": { 2024-09-21T06:08:43.454 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_host", 2024-09-21T06:08:43.454 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.454 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.454 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.454 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.455 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.455 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.455 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.455 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "SMTP server", 2024-09-21T06:08:43.455 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.455 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.455 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.455 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.455 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_password": { 2024-09-21T06:08:43.455 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_password", 2024-09-21T06:08:43.455 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.455 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.455 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.455 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.455 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.456 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.456 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.456 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Password to authenticate with", 2024-09-21T06:08:43.456 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.456 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.456 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.456 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.456 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_port": { 2024-09-21T06:08:43.456 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_port", 2024-09-21T06:08:43.456 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.456 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.456 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.456 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "465", 2024-09-21T06:08:43.456 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.456 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.457 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.457 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "SMTP port", 2024-09-21T06:08:43.457 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.457 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.457 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.457 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.457 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_sender": { 2024-09-21T06:08:43.457 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_sender", 2024-09-21T06:08:43.457 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.457 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.457 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.457 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.457 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.457 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.457 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.458 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "SMTP envelope sender", 2024-09-21T06:08:43.458 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.458 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.458 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.458 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.458 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_ssl": { 2024-09-21T06:08:43.458 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_ssl", 2024-09-21T06:08:43.458 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.458 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.458 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.458 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.458 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.458 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.458 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.458 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T06:08:43.459 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.459 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.459 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.459 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.459 INFO:teuthology.orchestra.run.smithi117.stdout: "smtp_user": { 2024-09-21T06:08:43.459 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smtp_user", 2024-09-21T06:08:43.459 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.459 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.459 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.459 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.459 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.459 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.459 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.459 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "User to authenticate as", 2024-09-21T06:08:43.459 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.460 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.460 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.460 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.460 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.460 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.460 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.460 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.460 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.461 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.461 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.461 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.461 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.462 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.462 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.462 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.462 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.462 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.462 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.462 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.462 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.462 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "balancer", 2024-09-21T06:08:43.462 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.462 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.462 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.462 INFO:teuthology.orchestra.run.smithi117.stdout: "active": { 2024-09-21T06:08:43.462 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "active", 2024-09-21T06:08:43.462 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.463 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.463 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.463 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.463 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.463 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.463 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.463 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T06:08:43.463 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.463 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.463 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.463 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.463 INFO:teuthology.orchestra.run.smithi117.stdout: "begin_time": { 2024-09-21T06:08:43.463 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "begin_time", 2024-09-21T06:08:43.464 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.464 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.464 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.464 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0000", 2024-09-21T06:08:43.464 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.464 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.464 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.464 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T06:08:43.464 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T06:08:43.464 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.464 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.464 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.464 INFO:teuthology.orchestra.run.smithi117.stdout: "begin_weekday": { 2024-09-21T06:08:43.464 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "begin_weekday", 2024-09-21T06:08:43.464 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "uint", 2024-09-21T06:08:43.465 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.465 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.465 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.465 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "0", 2024-09-21T06:08:43.465 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "6", 2024-09-21T06:08:43.465 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.465 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T06:08:43.465 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T06:08:43.465 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.465 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.465 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.465 INFO:teuthology.orchestra.run.smithi117.stdout: "crush_compat_max_iterations": { 2024-09-21T06:08:43.465 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crush_compat_max_iterations", 2024-09-21T06:08:43.465 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "uint", 2024-09-21T06:08:43.465 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.466 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.466 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "25", 2024-09-21T06:08:43.466 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "1", 2024-09-21T06:08:43.466 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "250", 2024-09-21T06:08:43.466 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.466 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T06:08:43.466 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.466 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.466 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.466 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.466 INFO:teuthology.orchestra.run.smithi117.stdout: "crush_compat_metrics": { 2024-09-21T06:08:43.466 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crush_compat_metrics", 2024-09-21T06:08:43.466 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.466 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.467 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.467 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T06:08:43.467 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.467 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.467 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.467 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T06:08:43.467 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T06:08:43.467 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.467 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.467 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.467 INFO:teuthology.orchestra.run.smithi117.stdout: "crush_compat_step": { 2024-09-21T06:08:43.467 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crush_compat_step", 2024-09-21T06:08:43.467 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-09-21T06:08:43.467 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.468 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.468 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0.5", 2024-09-21T06:08:43.468 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "0.001", 2024-09-21T06:08:43.468 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "0.999", 2024-09-21T06:08:43.468 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.468 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "aggressiveness of optimization", 2024-09-21T06:08:43.468 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T06:08:43.468 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.468 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.468 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.468 INFO:teuthology.orchestra.run.smithi117.stdout: "end_time": { 2024-09-21T06:08:43.468 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "end_time", 2024-09-21T06:08:43.468 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.468 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.468 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.469 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "2359", 2024-09-21T06:08:43.469 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.469 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.469 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.469 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T06:08:43.469 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T06:08:43.469 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.469 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.469 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.469 INFO:teuthology.orchestra.run.smithi117.stdout: "end_weekday": { 2024-09-21T06:08:43.469 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "end_weekday", 2024-09-21T06:08:43.469 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "uint", 2024-09-21T06:08:43.469 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.469 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.469 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.470 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "0", 2024-09-21T06:08:43.470 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "6", 2024-09-21T06:08:43.470 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.470 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T06:08:43.470 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T06:08:43.470 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.470 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.470 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.470 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.470 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.470 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.470 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.470 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.470 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.470 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.471 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.471 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.471 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.471 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.471 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.471 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.471 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.471 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.471 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.471 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.471 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.471 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.471 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.471 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.471 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.472 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.472 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.472 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.472 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.472 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.472 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.472 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.472 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.472 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.472 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.472 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.472 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.472 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.472 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.472 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.473 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.473 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.473 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.473 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.473 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.473 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.473 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.473 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.473 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.473 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.473 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.473 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.473 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.473 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.474 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.474 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.474 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.474 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.474 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.474 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.474 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.474 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.474 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.474 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.474 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.474 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.474 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.474 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.475 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.475 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.475 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.475 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.475 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.475 INFO:teuthology.orchestra.run.smithi117.stdout: "min_score": { 2024-09-21T06:08:43.475 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "min_score", 2024-09-21T06:08:43.475 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-09-21T06:08:43.475 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.475 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.475 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.475 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.475 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.475 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.475 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T06:08:43.475 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.476 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.476 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.476 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.476 INFO:teuthology.orchestra.run.smithi117.stdout: "mode": { 2024-09-21T06:08:43.476 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mode", 2024-09-21T06:08:43.476 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.476 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.476 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.476 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "upmap", 2024-09-21T06:08:43.476 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.476 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.476 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.476 INFO:teuthology.orchestra.run.smithi117.stdout: "crush-compat", 2024-09-21T06:08:43.476 INFO:teuthology.orchestra.run.smithi117.stdout: "none", 2024-09-21T06:08:43.476 INFO:teuthology.orchestra.run.smithi117.stdout: "read", 2024-09-21T06:08:43.477 INFO:teuthology.orchestra.run.smithi117.stdout: "upmap", 2024-09-21T06:08:43.477 INFO:teuthology.orchestra.run.smithi117.stdout: "upmap-read" 2024-09-21T06:08:43.477 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.477 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Balancer mode", 2024-09-21T06:08:43.477 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.477 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.477 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.477 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.477 INFO:teuthology.orchestra.run.smithi117.stdout: "pool_ids": { 2024-09-21T06:08:43.477 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "pool_ids", 2024-09-21T06:08:43.477 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.477 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.477 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.477 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.477 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.478 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.478 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.478 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T06:08:43.478 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.478 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.478 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.478 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.478 INFO:teuthology.orchestra.run.smithi117.stdout: "sleep_interval": { 2024-09-21T06:08:43.478 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sleep_interval", 2024-09-21T06:08:43.478 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.478 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.478 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.478 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "60", 2024-09-21T06:08:43.478 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.479 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.479 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.479 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T06:08:43.479 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.479 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.479 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.479 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.479 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.479 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.479 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.479 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.479 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.479 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.479 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.479 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.479 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.480 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.480 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.480 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.480 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.480 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.480 INFO:teuthology.orchestra.run.smithi117.stdout: "upmap_max_deviation": { 2024-09-21T06:08:43.480 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "upmap_max_deviation", 2024-09-21T06:08:43.480 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.480 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.480 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.480 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "5", 2024-09-21T06:08:43.480 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "1", 2024-09-21T06:08:43.480 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.480 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.481 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T06:08:43.481 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T06:08:43.481 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.481 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.481 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.481 INFO:teuthology.orchestra.run.smithi117.stdout: "upmap_max_optimizations": { 2024-09-21T06:08:43.481 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "upmap_max_optimizations", 2024-09-21T06:08:43.481 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "uint", 2024-09-21T06:08:43.481 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.481 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.481 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-09-21T06:08:43.481 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.481 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.481 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.481 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T06:08:43.481 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.482 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.482 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.482 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.482 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.482 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.482 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.482 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "cephadm", 2024-09-21T06:08:43.482 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.482 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.482 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.482 INFO:teuthology.orchestra.run.smithi117.stdout: "agent_down_multiplier": { 2024-09-21T06:08:43.482 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "agent_down_multiplier", 2024-09-21T06:08:43.482 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-09-21T06:08:43.482 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.483 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.483 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "3.0", 2024-09-21T06:08:43.483 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.483 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.483 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.483 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T06:08:43.483 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.483 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.483 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.483 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.483 INFO:teuthology.orchestra.run.smithi117.stdout: "agent_refresh_rate": { 2024-09-21T06:08:43.483 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "agent_refresh_rate", 2024-09-21T06:08:43.483 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.483 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.483 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.483 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "20", 2024-09-21T06:08:43.484 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.484 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.484 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.484 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T06:08:43.484 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.484 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.484 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.484 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.484 INFO:teuthology.orchestra.run.smithi117.stdout: "agent_starting_port": { 2024-09-21T06:08:43.484 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "agent_starting_port", 2024-09-21T06:08:43.484 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.484 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.484 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.484 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "4721", 2024-09-21T06:08:43.484 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.485 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.485 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.485 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T06:08:43.485 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.485 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.485 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.485 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.485 INFO:teuthology.orchestra.run.smithi117.stdout: "allow_ptrace": { 2024-09-21T06:08:43.485 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "allow_ptrace", 2024-09-21T06:08:43.485 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.485 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.485 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.485 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.485 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.485 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.485 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.486 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T06:08:43.486 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T06:08:43.486 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.486 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.486 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.486 INFO:teuthology.orchestra.run.smithi117.stdout: "autotune_interval": { 2024-09-21T06:08:43.486 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "autotune_interval", 2024-09-21T06:08:43.486 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.486 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.486 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.486 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "600", 2024-09-21T06:08:43.486 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.486 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.486 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.486 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T06:08:43.487 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.487 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.487 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.487 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.487 INFO:teuthology.orchestra.run.smithi117.stdout: "autotune_memory_target_ratio": { 2024-09-21T06:08:43.487 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T06:08:43.487 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-09-21T06:08:43.487 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.487 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.487 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0.7", 2024-09-21T06:08:43.487 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.487 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.487 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.487 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T06:08:43.487 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.487 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.488 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.488 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.488 INFO:teuthology.orchestra.run.smithi117.stdout: "cephadm_log_destination": { 2024-09-21T06:08:43.488 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "cephadm_log_destination", 2024-09-21T06:08:43.488 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.488 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.488 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.488 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.488 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.488 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.488 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.488 INFO:teuthology.orchestra.run.smithi117.stdout: "file", 2024-09-21T06:08:43.488 INFO:teuthology.orchestra.run.smithi117.stdout: "file,syslog", 2024-09-21T06:08:43.488 INFO:teuthology.orchestra.run.smithi117.stdout: "syslog" 2024-09-21T06:08:43.488 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.489 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T06:08:43.489 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.489 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.489 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.489 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.489 INFO:teuthology.orchestra.run.smithi117.stdout: "cgroups_split": { 2024-09-21T06:08:43.489 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "cgroups_split", 2024-09-21T06:08:43.489 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.489 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.489 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.489 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.489 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.489 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.489 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.489 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T06:08:43.489 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.490 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.490 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.490 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.490 INFO:teuthology.orchestra.run.smithi117.stdout: "config_checks_enabled": { 2024-09-21T06:08:43.490 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "config_checks_enabled", 2024-09-21T06:08:43.490 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.490 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.490 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.490 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.490 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.490 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.490 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.490 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T06:08:43.490 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.490 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.491 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.491 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.491 INFO:teuthology.orchestra.run.smithi117.stdout: "config_dashboard": { 2024-09-21T06:08:43.491 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "config_dashboard", 2024-09-21T06:08:43.491 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.491 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.491 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.491 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.491 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.491 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.491 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.491 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T06:08:43.491 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.491 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.491 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.492 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.492 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_alertmanager": { 2024-09-21T06:08:43.492 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_alertmanager", 2024-09-21T06:08:43.492 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.492 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.492 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.492 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T06:08:43.492 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.492 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.492 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.492 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Prometheus container image", 2024-09-21T06:08:43.492 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.492 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.492 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.492 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.493 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_base": { 2024-09-21T06:08:43.493 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_base", 2024-09-21T06:08:43.493 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.493 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.493 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.493 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T06:08:43.493 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.493 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.493 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.493 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Container image name, without the tag", 2024-09-21T06:08:43.493 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.493 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.493 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.493 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.493 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_elasticsearch": { 2024-09-21T06:08:43.494 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_elasticsearch", 2024-09-21T06:08:43.494 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.494 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.494 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.494 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T06:08:43.494 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.494 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.494 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.494 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "elasticsearch container image", 2024-09-21T06:08:43.494 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.494 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.494 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.494 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.494 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_grafana": { 2024-09-21T06:08:43.495 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_grafana", 2024-09-21T06:08:43.495 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.495 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.495 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.495 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T06:08:43.495 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.495 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.495 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.495 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Prometheus container image", 2024-09-21T06:08:43.495 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.495 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.495 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.495 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.495 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_haproxy": { 2024-09-21T06:08:43.495 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_haproxy", 2024-09-21T06:08:43.496 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.496 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.496 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.496 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T06:08:43.496 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.496 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.496 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.496 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "HAproxy container image", 2024-09-21T06:08:43.496 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.496 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.496 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.496 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.496 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_jaeger_agent": { 2024-09-21T06:08:43.496 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_jaeger_agent", 2024-09-21T06:08:43.496 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.497 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.497 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.497 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T06:08:43.497 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.497 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.497 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.497 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Jaeger agent container image", 2024-09-21T06:08:43.497 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.497 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.497 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.497 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.497 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_jaeger_collector": { 2024-09-21T06:08:43.497 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_jaeger_collector", 2024-09-21T06:08:43.497 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.498 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.498 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.498 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T06:08:43.498 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.498 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.498 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.498 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Jaeger collector container image", 2024-09-21T06:08:43.498 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.498 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.498 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.498 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.498 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_jaeger_query": { 2024-09-21T06:08:43.498 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_jaeger_query", 2024-09-21T06:08:43.498 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.498 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.499 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.499 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T06:08:43.499 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.499 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.499 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.499 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Jaeger query container image", 2024-09-21T06:08:43.499 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.499 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.499 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.499 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.499 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_keepalived": { 2024-09-21T06:08:43.499 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_keepalived", 2024-09-21T06:08:43.499 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.499 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.499 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.499 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T06:08:43.500 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.500 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.500 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.500 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Keepalived container image", 2024-09-21T06:08:43.500 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.500 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.500 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.500 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.500 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_loki": { 2024-09-21T06:08:43.500 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_loki", 2024-09-21T06:08:43.500 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.500 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.500 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.501 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T06:08:43.501 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.501 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.501 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.501 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Loki container image", 2024-09-21T06:08:43.501 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.501 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.501 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.501 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.501 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_nginx": { 2024-09-21T06:08:43.501 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_nginx", 2024-09-21T06:08:43.501 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.501 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.501 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.501 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T06:08:43.501 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.501 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.502 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.502 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Nginx container image", 2024-09-21T06:08:43.502 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.502 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.502 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.502 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.502 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_node_exporter": { 2024-09-21T06:08:43.502 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_node_exporter", 2024-09-21T06:08:43.502 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.502 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.502 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.502 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T06:08:43.502 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.502 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.502 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.502 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Prometheus container image", 2024-09-21T06:08:43.503 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.503 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.503 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.503 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.503 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_nvmeof": { 2024-09-21T06:08:43.503 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_nvmeof", 2024-09-21T06:08:43.503 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.503 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.503 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.503 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T06:08:43.503 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.503 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.503 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.503 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Nvme-of container image", 2024-09-21T06:08:43.503 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.503 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.503 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.504 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.504 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_oauth2_proxy": { 2024-09-21T06:08:43.504 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T06:08:43.504 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.504 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.504 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.504 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T06:08:43.504 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.504 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.504 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.504 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "oauth2-proxy container image", 2024-09-21T06:08:43.505 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.505 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.505 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.505 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.505 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_prometheus": { 2024-09-21T06:08:43.505 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_prometheus", 2024-09-21T06:08:43.505 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.505 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.505 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.505 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T06:08:43.505 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.505 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.505 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.505 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Prometheus container image", 2024-09-21T06:08:43.506 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.506 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.506 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.506 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.506 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_promtail": { 2024-09-21T06:08:43.506 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_promtail", 2024-09-21T06:08:43.506 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.506 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.506 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.506 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T06:08:43.506 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.506 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.506 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.506 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Promtail container image", 2024-09-21T06:08:43.506 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.506 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.507 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.507 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.507 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_samba": { 2024-09-21T06:08:43.507 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_samba", 2024-09-21T06:08:43.507 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.507 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.507 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.507 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T06:08:43.507 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.507 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.507 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.507 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Samba/SMB container image", 2024-09-21T06:08:43.507 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.507 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.507 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.508 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.508 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_samba_metrics": { 2024-09-21T06:08:43.508 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_samba_metrics", 2024-09-21T06:08:43.508 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.508 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.508 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.508 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T06:08:43.508 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.508 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.508 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.508 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T06:08:43.508 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.508 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.508 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.508 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.509 INFO:teuthology.orchestra.run.smithi117.stdout: "container_image_snmp_gateway": { 2024-09-21T06:08:43.509 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_image_snmp_gateway", 2024-09-21T06:08:43.509 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.509 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.509 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.509 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T06:08:43.509 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.509 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.509 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.509 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "SNMP Gateway container image", 2024-09-21T06:08:43.509 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.509 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.509 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.509 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.509 INFO:teuthology.orchestra.run.smithi117.stdout: "container_init": { 2024-09-21T06:08:43.509 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "container_init", 2024-09-21T06:08:43.510 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.510 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.510 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.510 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.510 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.510 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.510 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.510 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T06:08:43.510 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.510 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.510 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.510 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.510 INFO:teuthology.orchestra.run.smithi117.stdout: "daemon_cache_timeout": { 2024-09-21T06:08:43.510 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "daemon_cache_timeout", 2024-09-21T06:08:43.511 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.511 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.511 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.511 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "600", 2024-09-21T06:08:43.511 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.511 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.511 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.511 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T06:08:43.511 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.511 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.511 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.511 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.511 INFO:teuthology.orchestra.run.smithi117.stdout: "default_cephadm_command_timeout": { 2024-09-21T06:08:43.511 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T06:08:43.512 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.512 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.512 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.512 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "900", 2024-09-21T06:08:43.512 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.512 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.512 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.512 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T06:08:43.512 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.512 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.512 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.512 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.512 INFO:teuthology.orchestra.run.smithi117.stdout: "default_registry": { 2024-09-21T06:08:43.512 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "default_registry", 2024-09-21T06:08:43.512 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.513 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.513 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.513 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "docker.io", 2024-09-21T06:08:43.513 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.513 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.513 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.513 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T06:08:43.513 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.513 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.513 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.513 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.513 INFO:teuthology.orchestra.run.smithi117.stdout: "device_cache_timeout": { 2024-09-21T06:08:43.513 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "device_cache_timeout", 2024-09-21T06:08:43.513 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.513 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.514 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.514 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "1800", 2024-09-21T06:08:43.514 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.514 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.514 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.514 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "seconds to cache device inventory", 2024-09-21T06:08:43.514 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.514 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.514 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.514 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.514 INFO:teuthology.orchestra.run.smithi117.stdout: "device_enhanced_scan": { 2024-09-21T06:08:43.514 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "device_enhanced_scan", 2024-09-21T06:08:43.514 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.514 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.514 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.515 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.515 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.515 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.515 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.515 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T06:08:43.515 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.515 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.515 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.515 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.515 INFO:teuthology.orchestra.run.smithi117.stdout: "facts_cache_timeout": { 2024-09-21T06:08:43.515 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "facts_cache_timeout", 2024-09-21T06:08:43.515 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.515 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.515 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.515 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "60", 2024-09-21T06:08:43.516 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.516 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.516 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.516 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "seconds to cache host facts data", 2024-09-21T06:08:43.516 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.516 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.516 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.516 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.516 INFO:teuthology.orchestra.run.smithi117.stdout: "grafana_dashboards_path": { 2024-09-21T06:08:43.516 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "grafana_dashboards_path", 2024-09-21T06:08:43.516 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.516 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.516 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.516 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T06:08:43.516 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.516 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.517 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.517 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T06:08:43.517 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.517 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.517 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.517 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.517 INFO:teuthology.orchestra.run.smithi117.stdout: "host_check_interval": { 2024-09-21T06:08:43.517 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "host_check_interval", 2024-09-21T06:08:43.517 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.517 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.517 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.517 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "600", 2024-09-21T06:08:43.517 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.517 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.517 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.518 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how frequently to perform a host check", 2024-09-21T06:08:43.518 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.518 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.518 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.518 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.518 INFO:teuthology.orchestra.run.smithi117.stdout: "hw_monitoring": { 2024-09-21T06:08:43.518 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "hw_monitoring", 2024-09-21T06:08:43.518 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.518 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.518 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.518 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.518 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.518 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.518 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.518 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T06:08:43.519 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.519 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.519 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.519 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.519 INFO:teuthology.orchestra.run.smithi117.stdout: "inventory_list_all": { 2024-09-21T06:08:43.519 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "inventory_list_all", 2024-09-21T06:08:43.519 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.519 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.519 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.519 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.519 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.519 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.519 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.519 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T06:08:43.520 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.520 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.520 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.520 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.520 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.520 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.520 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.520 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.520 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.520 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.520 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.520 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.520 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.520 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.520 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.521 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.521 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.521 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.521 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.521 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.521 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.521 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.521 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.521 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.521 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.521 INFO:teuthology.orchestra.run.smithi117.stdout: "log_refresh_metadata": { 2024-09-21T06:08:43.521 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_refresh_metadata", 2024-09-21T06:08:43.521 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.521 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.521 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.522 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.522 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.522 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.522 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.522 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T06:08:43.522 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.522 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.522 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.522 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.522 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.522 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.522 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.522 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.522 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.522 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.523 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.523 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.523 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.523 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T06:08:43.523 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.523 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.523 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.523 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.523 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.523 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.523 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.523 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.523 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.523 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.523 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.524 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.524 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.524 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.524 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.524 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.524 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.524 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.524 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.524 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.524 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.524 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.524 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.524 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.524 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.524 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.525 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.525 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.525 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.525 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.525 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.525 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.525 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.525 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.525 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.525 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.525 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.525 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.525 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.525 INFO:teuthology.orchestra.run.smithi117.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T06:08:43.525 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T06:08:43.525 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.526 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.526 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.526 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.526 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.526 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.526 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.526 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T06:08:43.526 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.526 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.526 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.526 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.526 INFO:teuthology.orchestra.run.smithi117.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T06:08:43.526 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T06:08:43.526 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.527 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.527 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.527 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "*", 2024-09-21T06:08:43.527 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.527 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.527 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.527 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T06:08:43.527 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.527 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.527 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.527 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.527 INFO:teuthology.orchestra.run.smithi117.stdout: "max_count_per_host": { 2024-09-21T06:08:43.527 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "max_count_per_host", 2024-09-21T06:08:43.527 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.527 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.528 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.528 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-09-21T06:08:43.528 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.528 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.528 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.528 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "max number of daemons per service per host", 2024-09-21T06:08:43.528 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.528 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.528 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.528 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.528 INFO:teuthology.orchestra.run.smithi117.stdout: "max_osd_draining_count": { 2024-09-21T06:08:43.528 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "max_osd_draining_count", 2024-09-21T06:08:43.528 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.528 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.528 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.529 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-09-21T06:08:43.529 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.529 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.529 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.529 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T06:08:43.529 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.529 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.529 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.529 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.529 INFO:teuthology.orchestra.run.smithi117.stdout: "migration_current": { 2024-09-21T06:08:43.529 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "migration_current", 2024-09-21T06:08:43.529 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.529 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.529 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.529 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.530 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.530 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.530 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.530 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "internal - do not modify", 2024-09-21T06:08:43.530 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.530 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.530 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.530 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.530 INFO:teuthology.orchestra.run.smithi117.stdout: "mode": { 2024-09-21T06:08:43.530 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mode", 2024-09-21T06:08:43.530 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.530 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.530 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.530 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "root", 2024-09-21T06:08:43.530 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.531 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.531 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.531 INFO:teuthology.orchestra.run.smithi117.stdout: "cephadm-package", 2024-09-21T06:08:43.531 INFO:teuthology.orchestra.run.smithi117.stdout: "root" 2024-09-21T06:08:43.531 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.531 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T06:08:43.531 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.531 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.531 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.531 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.531 INFO:teuthology.orchestra.run.smithi117.stdout: "oob_default_addr": { 2024-09-21T06:08:43.531 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "oob_default_addr", 2024-09-21T06:08:43.531 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.531 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.532 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.532 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "169.254.1.1", 2024-09-21T06:08:43.532 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.532 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.532 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.532 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T06:08:43.532 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.532 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.532 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.532 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.532 INFO:teuthology.orchestra.run.smithi117.stdout: "prometheus_alerts_path": { 2024-09-21T06:08:43.532 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "prometheus_alerts_path", 2024-09-21T06:08:43.532 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.532 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.533 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.533 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T06:08:43.533 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.533 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.533 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.533 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T06:08:43.533 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.533 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.533 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.533 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.533 INFO:teuthology.orchestra.run.smithi117.stdout: "registry_insecure": { 2024-09-21T06:08:43.533 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "registry_insecure", 2024-09-21T06:08:43.533 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.533 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.533 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.534 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.534 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.534 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.534 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.534 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T06:08:43.534 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.534 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.534 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.534 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.534 INFO:teuthology.orchestra.run.smithi117.stdout: "registry_password": { 2024-09-21T06:08:43.534 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "registry_password", 2024-09-21T06:08:43.534 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.534 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.534 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.535 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.535 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.535 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.535 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.535 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T06:08:43.535 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.535 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.535 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.535 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.535 INFO:teuthology.orchestra.run.smithi117.stdout: "registry_url": { 2024-09-21T06:08:43.535 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "registry_url", 2024-09-21T06:08:43.535 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.535 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.535 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.535 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.535 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.536 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.536 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.536 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T06:08:43.536 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.536 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.536 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.536 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.536 INFO:teuthology.orchestra.run.smithi117.stdout: "registry_username": { 2024-09-21T06:08:43.536 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "registry_username", 2024-09-21T06:08:43.536 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.536 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.536 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.536 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.536 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.536 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.537 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.537 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T06:08:43.537 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.537 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.537 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.537 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.537 INFO:teuthology.orchestra.run.smithi117.stdout: "secure_monitoring_stack": { 2024-09-21T06:08:43.537 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "secure_monitoring_stack", 2024-09-21T06:08:43.537 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.537 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.537 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.537 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.537 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.537 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.537 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.538 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T06:08:43.538 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.538 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.538 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.538 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.538 INFO:teuthology.orchestra.run.smithi117.stdout: "service_discovery_port": { 2024-09-21T06:08:43.538 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "service_discovery_port", 2024-09-21T06:08:43.538 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.538 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.538 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.538 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "8765", 2024-09-21T06:08:43.538 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.538 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.538 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.538 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "cephadm service discovery port", 2024-09-21T06:08:43.538 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.539 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.539 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.539 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.539 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.539 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.539 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.539 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.539 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.539 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.539 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.539 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.539 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.539 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.539 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.539 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.540 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.540 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.540 INFO:teuthology.orchestra.run.smithi117.stdout: "ssh_config_file": { 2024-09-21T06:08:43.540 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ssh_config_file", 2024-09-21T06:08:43.540 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.540 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.540 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.540 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.540 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.540 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.540 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.540 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T06:08:43.540 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.540 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.540 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.541 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.541 INFO:teuthology.orchestra.run.smithi117.stdout: "ssh_keepalive_count_max": { 2024-09-21T06:08:43.541 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T06:08:43.541 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.541 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.541 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.541 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "3", 2024-09-21T06:08:43.541 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.541 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.541 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.541 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T06:08:43.541 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.541 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.541 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.541 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.542 INFO:teuthology.orchestra.run.smithi117.stdout: "ssh_keepalive_interval": { 2024-09-21T06:08:43.542 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ssh_keepalive_interval", 2024-09-21T06:08:43.542 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.542 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.542 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.542 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "7", 2024-09-21T06:08:43.542 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.542 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.542 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.542 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T06:08:43.542 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.542 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.542 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.542 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.542 INFO:teuthology.orchestra.run.smithi117.stdout: "use_agent": { 2024-09-21T06:08:43.543 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "use_agent", 2024-09-21T06:08:43.543 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.543 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.543 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.543 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.543 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.543 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.543 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.543 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T06:08:43.543 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.543 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.543 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.543 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.543 INFO:teuthology.orchestra.run.smithi117.stdout: "use_repo_digest": { 2024-09-21T06:08:43.543 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "use_repo_digest", 2024-09-21T06:08:43.544 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.544 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.544 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.544 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.544 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.544 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.544 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.544 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T06:08:43.544 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.544 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.544 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.544 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.544 INFO:teuthology.orchestra.run.smithi117.stdout: "warn_on_failed_host_check": { 2024-09-21T06:08:43.544 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "warn_on_failed_host_check", 2024-09-21T06:08:43.545 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.545 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.545 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.545 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.545 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.545 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.545 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.545 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T06:08:43.545 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.545 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.545 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.545 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.545 INFO:teuthology.orchestra.run.smithi117.stdout: "warn_on_stray_daemons": { 2024-09-21T06:08:43.545 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "warn_on_stray_daemons", 2024-09-21T06:08:43.545 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.546 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.546 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.546 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.546 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.546 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.546 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.546 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T06:08:43.546 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.546 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.546 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.546 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.546 INFO:teuthology.orchestra.run.smithi117.stdout: "warn_on_stray_hosts": { 2024-09-21T06:08:43.546 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "warn_on_stray_hosts", 2024-09-21T06:08:43.547 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.547 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.547 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.547 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.547 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.547 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.547 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.547 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T06:08:43.547 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.547 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.547 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.547 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.547 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.547 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.547 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.548 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crash", 2024-09-21T06:08:43.548 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.548 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.548 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.548 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.548 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.548 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.548 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.548 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.548 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.548 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.548 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.548 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.548 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.548 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.549 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.549 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.549 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.549 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.549 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.549 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.549 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.549 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.549 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.549 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.549 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.549 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.549 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.549 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.549 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.550 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.550 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.550 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.550 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.550 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.550 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.550 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.550 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.550 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.550 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.550 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.550 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.550 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.550 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.550 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.551 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.551 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.551 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.551 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.551 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.551 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.551 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.551 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.551 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.551 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.551 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.551 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.551 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.551 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.552 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.552 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.552 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.552 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.552 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.552 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.552 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.552 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.552 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.552 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.552 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.552 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.552 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.552 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.552 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.553 INFO:teuthology.orchestra.run.smithi117.stdout: "retain_interval": { 2024-09-21T06:08:43.553 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "retain_interval", 2024-09-21T06:08:43.553 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.553 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.553 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.553 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "31536000", 2024-09-21T06:08:43.553 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.553 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.553 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.553 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T06:08:43.553 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.553 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.553 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.553 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.553 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.553 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.554 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.554 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.554 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.554 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.554 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.554 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.554 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.554 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.554 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.554 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.554 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.554 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.554 INFO:teuthology.orchestra.run.smithi117.stdout: "warn_recent_interval": { 2024-09-21T06:08:43.554 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "warn_recent_interval", 2024-09-21T06:08:43.554 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.555 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.555 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.555 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "1209600", 2024-09-21T06:08:43.555 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.555 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.555 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.555 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T06:08:43.555 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.555 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.555 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.555 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.555 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.555 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.555 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.555 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "dashboard", 2024-09-21T06:08:43.556 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.556 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.556 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.556 INFO:teuthology.orchestra.run.smithi117.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T06:08:43.556 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T06:08:43.556 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.556 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.556 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.556 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-09-21T06:08:43.556 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.556 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.556 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.556 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.556 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.556 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.556 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.557 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.557 INFO:teuthology.orchestra.run.smithi117.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T06:08:43.557 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T06:08:43.557 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.557 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.557 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.557 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.557 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.557 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.557 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.557 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.557 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.557 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.557 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.557 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.558 INFO:teuthology.orchestra.run.smithi117.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T06:08:43.558 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T06:08:43.558 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.558 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.558 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.558 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.558 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.558 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.558 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.558 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.558 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.558 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.558 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.558 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.558 INFO:teuthology.orchestra.run.smithi117.stdout: "AUDIT_API_ENABLED": { 2024-09-21T06:08:43.558 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T06:08:43.559 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.559 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.559 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.559 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.559 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.559 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.559 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.559 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.559 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.559 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.559 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.559 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.559 INFO:teuthology.orchestra.run.smithi117.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T06:08:43.559 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T06:08:43.559 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.560 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.560 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.560 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.560 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.560 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.560 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.560 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.560 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.560 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.560 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.560 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.560 INFO:teuthology.orchestra.run.smithi117.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T06:08:43.560 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T06:08:43.560 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.560 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.561 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.561 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.561 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.561 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.561 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.561 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.561 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.561 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.561 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.561 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.561 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T06:08:43.561 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T06:08:43.561 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.561 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.562 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.562 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.562 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.562 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.562 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.562 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.562 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.562 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.562 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.562 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.562 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T06:08:43.562 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T06:08:43.562 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.562 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.563 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.563 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.563 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.563 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.563 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.563 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.563 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.563 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.563 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.563 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.563 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T06:08:43.563 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T06:08:43.563 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.563 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.563 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.564 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.564 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.564 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.564 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.564 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.564 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.564 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.564 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.564 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.564 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T06:08:43.564 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T06:08:43.564 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.564 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.564 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.564 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.565 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.565 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.565 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.565 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.565 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.565 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.565 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.565 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.565 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T06:08:43.565 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T06:08:43.565 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.565 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.565 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.565 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.566 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.566 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.566 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.566 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.566 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.566 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.566 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.566 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.566 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T06:08:43.566 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T06:08:43.566 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.566 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.566 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.566 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.566 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.567 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.567 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.567 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.567 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.567 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.567 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.567 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.567 INFO:teuthology.orchestra.run.smithi117.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T06:08:43.567 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T06:08:43.567 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.567 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.567 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.567 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.567 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.568 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.568 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.568 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.568 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.568 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.568 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.568 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.568 INFO:teuthology.orchestra.run.smithi117.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T06:08:43.568 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T06:08:43.568 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.568 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.568 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.568 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.568 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.568 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.569 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.569 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.569 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.569 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.569 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.569 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.569 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T06:08:43.569 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T06:08:43.569 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.569 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.569 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.569 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "admin", 2024-09-21T06:08:43.569 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.569 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.569 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.569 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.570 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.570 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.570 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.570 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.570 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T06:08:43.570 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T06:08:43.570 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.570 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.570 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.570 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.570 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.570 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.570 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.570 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.571 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.571 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.571 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.571 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.571 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_API_URL": { 2024-09-21T06:08:43.571 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_API_URL", 2024-09-21T06:08:43.571 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.571 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.571 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.571 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.571 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.571 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.571 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.571 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.571 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.572 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.572 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.572 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.572 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T06:08:43.572 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T06:08:43.572 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.572 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.572 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.572 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "admin", 2024-09-21T06:08:43.572 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.572 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.572 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.572 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.572 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.572 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.573 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.573 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.573 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T06:08:43.573 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T06:08:43.573 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.573 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.573 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.573 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.573 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.573 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.573 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.573 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.573 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.573 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.574 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.574 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.574 INFO:teuthology.orchestra.run.smithi117.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T06:08:43.574 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T06:08:43.574 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.574 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.574 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.574 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.574 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.574 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.574 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.574 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.574 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.574 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.574 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.575 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.575 INFO:teuthology.orchestra.run.smithi117.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T06:08:43.575 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T06:08:43.575 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.575 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.575 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.575 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.575 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.575 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.575 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.575 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.575 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.575 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.575 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.575 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.576 INFO:teuthology.orchestra.run.smithi117.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T06:08:43.576 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T06:08:43.576 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.576 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.576 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.576 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.576 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.576 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.576 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.576 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.576 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.576 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.576 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.576 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.577 INFO:teuthology.orchestra.run.smithi117.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T06:08:43.577 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T06:08:43.577 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.577 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.577 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.577 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "[]", 2024-09-21T06:08:43.577 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.577 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.577 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.577 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.577 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.577 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.577 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.577 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.577 INFO:teuthology.orchestra.run.smithi117.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T06:08:43.578 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T06:08:43.578 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.578 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.578 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.578 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "{}", 2024-09-21T06:08:43.578 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.578 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.578 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.578 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.578 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.578 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.578 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.578 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.578 INFO:teuthology.orchestra.run.smithi117.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T06:08:43.579 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T06:08:43.579 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.579 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.579 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.579 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.579 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.579 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.579 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.579 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.579 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.579 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.579 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.579 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.579 INFO:teuthology.orchestra.run.smithi117.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T06:08:43.579 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T06:08:43.580 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.580 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.580 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.580 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.580 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.580 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.580 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.580 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.580 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.580 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.580 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.580 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.580 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T06:08:43.580 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T06:08:43.580 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.581 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.581 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.581 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.581 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.581 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.581 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.581 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.581 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.581 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.581 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.581 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.581 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T06:08:43.581 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T06:08:43.581 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.581 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.581 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.582 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.582 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.582 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.582 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.582 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.582 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.582 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.582 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.582 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.582 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T06:08:43.582 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T06:08:43.582 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.582 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.582 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.583 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.583 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.583 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.583 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.583 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.583 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.583 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.583 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.583 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.583 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T06:08:43.583 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T06:08:43.583 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.583 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.583 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.583 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.583 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.584 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.584 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.584 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.584 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.584 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.584 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.584 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.584 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T06:08:43.584 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T06:08:43.584 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.584 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.584 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.584 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.584 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.584 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.585 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.585 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.585 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.585 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.585 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.585 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.585 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T06:08:43.585 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T06:08:43.585 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.585 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.585 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.585 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.585 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.585 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.585 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.585 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.586 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.586 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.586 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.586 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.586 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T06:08:43.586 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T06:08:43.586 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.586 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.586 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.586 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.586 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.586 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.586 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.586 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.586 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.587 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.587 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.587 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.587 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T06:08:43.587 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T06:08:43.587 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.587 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.587 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.587 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.587 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.587 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.587 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.587 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.587 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.587 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.588 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.588 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.588 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T06:08:43.588 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T06:08:43.588 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.588 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.588 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.588 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T06:08:43.588 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.588 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.588 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.588 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.588 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.588 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.588 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.589 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.589 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T06:08:43.589 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T06:08:43.589 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.589 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.589 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.589 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-09-21T06:08:43.589 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.589 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.589 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.589 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.589 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.589 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.589 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.589 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.589 INFO:teuthology.orchestra.run.smithi117.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T06:08:43.590 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T06:08:43.590 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.590 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.590 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.590 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "8", 2024-09-21T06:08:43.590 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.590 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.590 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.590 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.590 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.590 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.590 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.590 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.590 INFO:teuthology.orchestra.run.smithi117.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T06:08:43.590 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T06:08:43.591 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.591 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.591 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.591 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "45", 2024-09-21T06:08:43.591 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.591 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.591 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.591 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.591 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.591 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.591 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.591 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.591 INFO:teuthology.orchestra.run.smithi117.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T06:08:43.592 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T06:08:43.592 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.592 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.592 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.592 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.592 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.592 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.592 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.592 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.592 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.592 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.592 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.592 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.593 INFO:teuthology.orchestra.run.smithi117.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T06:08:43.593 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T06:08:43.593 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.593 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.593 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.593 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "admin", 2024-09-21T06:08:43.593 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.593 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.593 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.593 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.593 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.593 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.593 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.593 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.593 INFO:teuthology.orchestra.run.smithi117.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T06:08:43.594 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T06:08:43.594 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.594 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.594 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.594 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.594 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.594 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.594 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.594 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.594 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.594 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.594 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.594 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.594 INFO:teuthology.orchestra.run.smithi117.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T06:08:43.595 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T06:08:43.595 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.595 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.595 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.595 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.595 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.595 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.595 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.595 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.595 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.595 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.595 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.595 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.595 INFO:teuthology.orchestra.run.smithi117.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T06:08:43.595 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T06:08:43.595 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.596 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.596 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.596 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.596 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.596 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.596 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.596 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.596 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.596 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.596 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.596 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.596 INFO:teuthology.orchestra.run.smithi117.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T06:08:43.596 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T06:08:43.596 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.597 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.597 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.597 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.597 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.597 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.597 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.597 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.597 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.597 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.597 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.597 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.597 INFO:teuthology.orchestra.run.smithi117.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T06:08:43.597 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T06:08:43.597 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.597 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.598 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.598 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-09-21T06:08:43.598 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.598 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.598 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.598 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.598 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.598 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.598 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.598 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.598 INFO:teuthology.orchestra.run.smithi117.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T06:08:43.598 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T06:08:43.598 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.598 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.599 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.599 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "5", 2024-09-21T06:08:43.599 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.599 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.599 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.599 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.599 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.599 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.599 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.599 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.599 INFO:teuthology.orchestra.run.smithi117.stdout: "cross_origin_url": { 2024-09-21T06:08:43.599 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "cross_origin_url", 2024-09-21T06:08:43.599 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.599 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.599 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.600 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.600 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.600 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.600 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.600 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.600 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.600 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.600 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.600 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.600 INFO:teuthology.orchestra.run.smithi117.stdout: "crt_file": { 2024-09-21T06:08:43.600 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "crt_file", 2024-09-21T06:08:43.600 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.600 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.600 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.600 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.600 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.600 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.600 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.601 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.601 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.601 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.601 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.601 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.601 INFO:teuthology.orchestra.run.smithi117.stdout: "debug": { 2024-09-21T06:08:43.601 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "debug", 2024-09-21T06:08:43.601 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.601 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.601 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.601 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.601 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.601 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.601 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.601 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Enable/disable debug options", 2024-09-21T06:08:43.601 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.602 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.602 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.602 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.602 INFO:teuthology.orchestra.run.smithi117.stdout: "jwt_token_ttl": { 2024-09-21T06:08:43.602 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "jwt_token_ttl", 2024-09-21T06:08:43.602 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.602 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.602 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.602 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "28800", 2024-09-21T06:08:43.602 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.602 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.602 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.602 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.602 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.602 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.602 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.603 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.603 INFO:teuthology.orchestra.run.smithi117.stdout: "key_file": { 2024-09-21T06:08:43.603 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "key_file", 2024-09-21T06:08:43.603 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.603 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.603 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.603 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.603 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.603 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.603 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.603 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.603 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.603 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.603 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.603 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.604 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.604 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.604 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.604 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.604 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.604 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.604 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.604 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.604 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.604 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.604 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.604 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.604 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.604 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.604 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.604 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.604 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.605 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.605 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.605 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.605 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.605 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.605 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.605 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.605 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.605 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.605 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.605 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.605 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.605 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.605 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.606 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.606 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.606 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.606 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.606 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.606 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.606 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.606 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.606 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.606 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.606 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.606 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.606 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.606 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.607 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.607 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.607 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.607 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.607 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.607 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.607 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.607 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.607 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.607 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.607 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.607 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.607 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.607 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.608 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.608 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.608 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.608 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.608 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.608 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.608 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.608 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.608 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.608 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.608 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.608 INFO:teuthology.orchestra.run.smithi117.stdout: "motd": { 2024-09-21T06:08:43.608 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "motd", 2024-09-21T06:08:43.608 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.608 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.609 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.609 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.609 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.609 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.609 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.609 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "The message of the day", 2024-09-21T06:08:43.609 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.609 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.609 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.609 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.609 INFO:teuthology.orchestra.run.smithi117.stdout: "redirect_resolve_ip_addr": { 2024-09-21T06:08:43.609 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T06:08:43.609 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.609 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.610 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.610 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.610 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.610 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.610 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.610 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.610 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.610 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.610 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.610 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.610 INFO:teuthology.orchestra.run.smithi117.stdout: "server_addr": { 2024-09-21T06:08:43.610 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_addr", 2024-09-21T06:08:43.610 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.610 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.610 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.610 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "::", 2024-09-21T06:08:43.611 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.611 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.611 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.611 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.611 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.611 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.611 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.611 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.611 INFO:teuthology.orchestra.run.smithi117.stdout: "server_port": { 2024-09-21T06:08:43.611 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_port", 2024-09-21T06:08:43.611 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.611 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.611 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.612 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "8080", 2024-09-21T06:08:43.612 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.612 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.612 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.612 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.612 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.612 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.612 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.612 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.612 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.612 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.612 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.612 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.612 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.612 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.613 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.613 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.613 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.613 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.613 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.613 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.613 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.613 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.613 INFO:teuthology.orchestra.run.smithi117.stdout: "ssl": { 2024-09-21T06:08:43.613 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ssl", 2024-09-21T06:08:43.613 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.613 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.613 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.613 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.613 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.614 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.614 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.614 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.614 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.614 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.614 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.614 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.614 INFO:teuthology.orchestra.run.smithi117.stdout: "ssl_server_port": { 2024-09-21T06:08:43.614 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ssl_server_port", 2024-09-21T06:08:43.614 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.614 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.614 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.614 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "8443", 2024-09-21T06:08:43.614 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.615 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.615 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.615 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.615 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.615 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.615 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.615 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.615 INFO:teuthology.orchestra.run.smithi117.stdout: "sso_oauth2": { 2024-09-21T06:08:43.615 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sso_oauth2", 2024-09-21T06:08:43.615 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.615 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.615 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.615 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.615 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.615 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.615 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.616 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.616 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.616 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.616 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.616 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.616 INFO:teuthology.orchestra.run.smithi117.stdout: "standby_behaviour": { 2024-09-21T06:08:43.616 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "standby_behaviour", 2024-09-21T06:08:43.616 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.616 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.616 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.616 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "redirect", 2024-09-21T06:08:43.616 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.616 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.616 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.616 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.617 INFO:teuthology.orchestra.run.smithi117.stdout: "redirect" 2024-09-21T06:08:43.617 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.617 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.617 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.617 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.617 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.617 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.617 INFO:teuthology.orchestra.run.smithi117.stdout: "standby_error_status_code": { 2024-09-21T06:08:43.617 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "standby_error_status_code", 2024-09-21T06:08:43.617 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.617 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.617 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.617 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "500", 2024-09-21T06:08:43.617 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "400", 2024-09-21T06:08:43.617 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "599", 2024-09-21T06:08:43.618 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.618 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.618 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.618 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.618 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.618 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.618 INFO:teuthology.orchestra.run.smithi117.stdout: "url_prefix": { 2024-09-21T06:08:43.618 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "url_prefix", 2024-09-21T06:08:43.618 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.618 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.618 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.618 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.618 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.618 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.618 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.618 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.619 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.619 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.619 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.619 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.619 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.619 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.619 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.619 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "devicehealth", 2024-09-21T06:08:43.619 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.619 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.619 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.619 INFO:teuthology.orchestra.run.smithi117.stdout: "enable_monitoring": { 2024-09-21T06:08:43.619 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "enable_monitoring", 2024-09-21T06:08:43.619 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.619 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.620 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.620 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.620 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.620 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.620 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.620 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "monitor device health metrics", 2024-09-21T06:08:43.620 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.620 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.620 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.620 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.620 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.620 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.620 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.620 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.620 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.621 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.621 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.621 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.621 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.621 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.621 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.621 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.621 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.621 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.621 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.621 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.621 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.621 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.621 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.621 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.621 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.622 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.622 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.622 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.622 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.622 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.622 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.622 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.622 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.622 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.622 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.622 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.622 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.622 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.622 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.623 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.623 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.623 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.623 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.623 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.623 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.623 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.623 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.623 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.623 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.623 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.623 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.623 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.623 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.623 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.624 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.624 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.624 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.624 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.624 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.624 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.624 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.624 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.624 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.624 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.624 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.624 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.624 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.624 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.624 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.625 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.625 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.625 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.625 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.625 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.625 INFO:teuthology.orchestra.run.smithi117.stdout: "mark_out_threshold": { 2024-09-21T06:08:43.625 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mark_out_threshold", 2024-09-21T06:08:43.625 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.625 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.625 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.625 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "2419200", 2024-09-21T06:08:43.625 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.625 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.625 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.625 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T06:08:43.625 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.626 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.626 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.626 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.626 INFO:teuthology.orchestra.run.smithi117.stdout: "pool_name": { 2024-09-21T06:08:43.626 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "pool_name", 2024-09-21T06:08:43.626 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.626 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.626 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.626 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "device_health_metrics", 2024-09-21T06:08:43.626 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.626 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.626 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.626 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T06:08:43.626 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.627 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.627 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.627 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.627 INFO:teuthology.orchestra.run.smithi117.stdout: "retention_period": { 2024-09-21T06:08:43.627 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "retention_period", 2024-09-21T06:08:43.627 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.627 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.627 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.627 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "15552000", 2024-09-21T06:08:43.627 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.627 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.627 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.627 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how long to retain device health metrics", 2024-09-21T06:08:43.627 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.627 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.628 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.628 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.628 INFO:teuthology.orchestra.run.smithi117.stdout: "scrape_frequency": { 2024-09-21T06:08:43.628 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "scrape_frequency", 2024-09-21T06:08:43.628 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.628 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.628 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.628 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "86400", 2024-09-21T06:08:43.628 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.628 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.628 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.628 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T06:08:43.628 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.628 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.628 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.629 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.629 INFO:teuthology.orchestra.run.smithi117.stdout: "self_heal": { 2024-09-21T06:08:43.629 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "self_heal", 2024-09-21T06:08:43.629 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.629 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.629 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.629 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.629 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.629 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.629 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.629 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T06:08:43.629 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.629 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.629 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.629 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.630 INFO:teuthology.orchestra.run.smithi117.stdout: "sleep_interval": { 2024-09-21T06:08:43.630 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sleep_interval", 2024-09-21T06:08:43.630 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.630 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.630 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.630 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "600", 2024-09-21T06:08:43.630 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.630 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.630 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.630 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T06:08:43.630 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.630 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.630 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.630 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.630 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.630 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.631 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.631 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.631 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.631 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.631 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.631 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.631 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.631 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.631 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.631 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.631 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.631 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.631 INFO:teuthology.orchestra.run.smithi117.stdout: "warn_threshold": { 2024-09-21T06:08:43.631 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "warn_threshold", 2024-09-21T06:08:43.631 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.632 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.632 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.632 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "7257600", 2024-09-21T06:08:43.632 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.632 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.632 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.632 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T06:08:43.632 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.632 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.632 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.632 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.632 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.632 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.632 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.632 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "diskprediction_local", 2024-09-21T06:08:43.633 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.633 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.633 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.633 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.633 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.633 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.633 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.633 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.633 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.633 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.633 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.633 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.633 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.633 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.633 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.633 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.634 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.634 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.634 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.634 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.634 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.634 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.634 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.634 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.634 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.634 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.634 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.634 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.634 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.634 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.635 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.635 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.635 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.635 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.635 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.635 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.635 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.635 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.635 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.635 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.635 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.635 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.635 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.635 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.635 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.636 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.636 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.636 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.636 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.636 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.636 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.636 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.636 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.636 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.636 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.636 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.636 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.636 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.636 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.636 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.636 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.637 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.637 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.637 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.637 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.637 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.637 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.637 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.637 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.637 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.637 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.637 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.637 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.637 INFO:teuthology.orchestra.run.smithi117.stdout: "predict_interval": { 2024-09-21T06:08:43.638 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "predict_interval", 2024-09-21T06:08:43.638 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.638 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.638 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.638 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "86400", 2024-09-21T06:08:43.638 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.638 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.638 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.638 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.638 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.638 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.638 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.638 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.638 INFO:teuthology.orchestra.run.smithi117.stdout: "predictor_model": { 2024-09-21T06:08:43.638 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "predictor_model", 2024-09-21T06:08:43.639 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.639 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.639 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.639 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "prophetstor", 2024-09-21T06:08:43.639 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.639 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.639 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.639 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.639 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.639 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.639 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.639 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.639 INFO:teuthology.orchestra.run.smithi117.stdout: "sleep_interval": { 2024-09-21T06:08:43.639 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sleep_interval", 2024-09-21T06:08:43.639 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.640 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.640 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.640 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "600", 2024-09-21T06:08:43.640 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.640 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.640 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.640 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.640 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.640 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.640 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.640 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.640 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.640 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.640 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.641 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.641 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.641 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.641 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.641 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.641 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.641 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.641 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.641 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.641 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.641 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.641 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.641 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.641 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.641 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "influx", 2024-09-21T06:08:43.642 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": false, 2024-09-21T06:08:43.642 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "influxdb python module not found", 2024-09-21T06:08:43.642 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.642 INFO:teuthology.orchestra.run.smithi117.stdout: "batch_size": { 2024-09-21T06:08:43.642 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "batch_size", 2024-09-21T06:08:43.642 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.642 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.642 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.642 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "5000", 2024-09-21T06:08:43.642 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.642 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.642 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.642 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T06:08:43.642 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.643 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.643 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.643 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.643 INFO:teuthology.orchestra.run.smithi117.stdout: "database": { 2024-09-21T06:08:43.643 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "database", 2024-09-21T06:08:43.643 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.643 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.643 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.643 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "ceph", 2024-09-21T06:08:43.643 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.643 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.643 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.643 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T06:08:43.643 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.643 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.644 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.644 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.644 INFO:teuthology.orchestra.run.smithi117.stdout: "hostname": { 2024-09-21T06:08:43.644 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "hostname", 2024-09-21T06:08:43.644 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.644 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.644 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.644 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.644 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.644 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.644 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.644 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "InfluxDB server hostname", 2024-09-21T06:08:43.644 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.644 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.644 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.645 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.645 INFO:teuthology.orchestra.run.smithi117.stdout: "interval": { 2024-09-21T06:08:43.645 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "interval", 2024-09-21T06:08:43.645 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.645 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.645 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.645 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "30", 2024-09-21T06:08:43.645 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "5", 2024-09-21T06:08:43.645 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.645 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.645 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T06:08:43.645 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.645 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.645 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.645 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.646 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.646 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.646 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.646 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.646 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.646 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.646 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.646 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.646 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.646 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.646 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.646 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.646 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.646 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.646 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.647 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.647 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.647 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.647 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.647 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.647 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.647 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.647 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.647 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.647 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.647 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.647 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.647 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.647 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.648 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.648 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.648 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.648 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.648 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.648 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.648 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.648 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.648 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.648 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.648 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.648 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.648 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.648 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.648 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.648 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.649 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.649 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.649 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.649 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.649 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.649 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.649 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.649 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.649 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.649 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.649 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.649 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.649 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.649 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.649 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.650 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.650 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.650 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.650 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.650 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.650 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.650 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.650 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.650 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.650 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.650 INFO:teuthology.orchestra.run.smithi117.stdout: "password": { 2024-09-21T06:08:43.650 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "password", 2024-09-21T06:08:43.650 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.650 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.650 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.651 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.651 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.651 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.651 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.651 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "password of InfluxDB server user", 2024-09-21T06:08:43.651 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.651 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.651 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.651 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.651 INFO:teuthology.orchestra.run.smithi117.stdout: "port": { 2024-09-21T06:08:43.651 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "port", 2024-09-21T06:08:43.651 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.651 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.651 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.651 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "8086", 2024-09-21T06:08:43.652 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.652 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.652 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.652 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "InfluxDB server port", 2024-09-21T06:08:43.652 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.652 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.652 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.652 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.652 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.652 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.652 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.652 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.652 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.652 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.652 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.653 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.653 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.653 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.653 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.653 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.653 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.653 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.653 INFO:teuthology.orchestra.run.smithi117.stdout: "ssl": { 2024-09-21T06:08:43.653 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ssl", 2024-09-21T06:08:43.653 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.653 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.653 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.653 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "false", 2024-09-21T06:08:43.653 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.654 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.654 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.654 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T06:08:43.654 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.654 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.654 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.654 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.654 INFO:teuthology.orchestra.run.smithi117.stdout: "threads": { 2024-09-21T06:08:43.654 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "threads", 2024-09-21T06:08:43.654 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.654 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.654 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.654 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "5", 2024-09-21T06:08:43.654 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "1", 2024-09-21T06:08:43.654 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "32", 2024-09-21T06:08:43.655 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.655 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T06:08:43.655 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.655 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.655 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.655 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.655 INFO:teuthology.orchestra.run.smithi117.stdout: "username": { 2024-09-21T06:08:43.655 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "username", 2024-09-21T06:08:43.655 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.655 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.655 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.655 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.655 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.655 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.655 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.656 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "username of InfluxDB server user", 2024-09-21T06:08:43.656 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.656 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.656 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.656 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.656 INFO:teuthology.orchestra.run.smithi117.stdout: "verify_ssl": { 2024-09-21T06:08:43.656 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "verify_ssl", 2024-09-21T06:08:43.656 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.656 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.656 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.656 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "true", 2024-09-21T06:08:43.656 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.656 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.656 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.656 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T06:08:43.657 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.657 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.657 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.657 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.657 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.657 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.657 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.657 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "insights", 2024-09-21T06:08:43.657 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.657 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.657 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.657 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.657 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.657 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.658 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.658 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.658 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.658 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.658 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.658 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.658 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.658 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.658 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.658 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.658 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.658 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.658 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.658 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.658 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.659 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.659 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.659 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.659 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.659 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.659 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.659 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.659 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.659 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.659 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.659 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.659 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.659 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.659 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.660 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.660 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.660 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.660 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.660 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.660 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.660 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.660 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.660 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.660 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.660 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.660 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.660 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.660 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.660 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.661 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.661 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.661 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.661 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.661 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.661 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.661 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.661 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.661 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.661 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.661 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.661 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.661 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.661 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.661 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.661 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.662 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.662 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.662 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.662 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.662 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.662 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.662 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.662 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.662 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.662 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.662 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.662 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.662 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.662 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.662 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.663 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.663 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.663 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.663 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.663 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.663 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.663 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.663 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.663 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.663 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "iostat", 2024-09-21T06:08:43.663 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.663 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.663 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.663 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.663 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.664 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.664 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.664 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.664 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.664 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.664 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.664 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.664 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.664 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.664 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.664 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.664 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.664 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.664 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.664 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.665 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.665 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.665 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.665 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.665 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.665 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.665 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.665 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.665 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.665 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.665 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.665 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.665 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.665 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.665 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.666 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.666 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.666 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.666 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.666 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.666 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.666 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.666 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.666 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.666 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.666 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.666 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.666 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.666 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.666 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.667 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.667 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.667 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.667 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.667 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.667 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.667 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.667 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.667 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.667 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.667 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.667 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.667 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.667 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.668 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.668 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.668 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.668 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.668 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.668 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.668 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.668 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.668 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.668 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.668 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.668 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.668 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.668 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.668 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.669 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.669 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.669 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.669 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.669 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.669 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.669 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.669 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.669 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.669 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.669 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.669 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "k8sevents", 2024-09-21T06:08:43.669 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.670 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.670 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.670 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_event_retention_days": { 2024-09-21T06:08:43.670 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "ceph_event_retention_days", 2024-09-21T06:08:43.670 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.670 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.670 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.670 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "7", 2024-09-21T06:08:43.670 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.670 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.670 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.670 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T06:08:43.670 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.670 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.670 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.670 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.671 INFO:teuthology.orchestra.run.smithi117.stdout: "config_check_secs": { 2024-09-21T06:08:43.671 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "config_check_secs", 2024-09-21T06:08:43.671 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.671 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.671 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.671 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-09-21T06:08:43.671 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "10", 2024-09-21T06:08:43.671 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.671 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.671 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T06:08:43.671 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.671 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.671 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.671 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.672 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.672 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.672 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.672 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.672 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.672 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.672 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.672 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.672 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.672 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.672 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.672 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.672 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.672 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.672 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.672 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.673 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.673 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.673 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.673 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.673 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.673 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.673 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.673 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.673 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.673 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.673 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.673 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.673 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.673 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.673 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.674 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.674 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.674 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.674 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.674 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.674 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.674 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.674 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.674 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.674 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.674 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.674 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.674 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.674 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.674 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.675 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.675 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.675 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.675 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.675 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.675 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.675 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.675 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.675 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.675 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.675 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.675 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.675 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.675 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.676 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.676 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.676 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.676 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.676 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.676 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.676 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.676 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.676 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.676 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.676 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.676 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.676 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.676 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.677 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.677 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.677 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.677 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.677 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.677 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.677 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.677 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.677 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.677 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.677 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.677 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.677 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.677 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "localpool", 2024-09-21T06:08:43.677 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.677 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.678 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.678 INFO:teuthology.orchestra.run.smithi117.stdout: "failure_domain": { 2024-09-21T06:08:43.678 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "failure_domain", 2024-09-21T06:08:43.678 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.678 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.678 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.678 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "host", 2024-09-21T06:08:43.678 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.678 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.678 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.678 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "failure domain for any created local pool", 2024-09-21T06:08:43.678 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T06:08:43.678 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.678 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.679 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.679 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.679 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.679 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.679 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.679 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.679 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.679 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.679 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.679 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.679 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.679 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.679 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.679 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.679 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.680 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.680 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.680 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.680 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.680 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.680 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.680 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.680 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.680 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.680 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.680 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.680 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.680 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.680 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.680 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.681 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.681 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.681 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.681 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.681 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.681 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.681 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.681 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.681 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.681 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.681 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.681 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.681 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.681 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.681 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.682 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.682 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.682 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.682 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.682 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.682 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.682 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.682 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.682 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.682 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.682 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.682 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.682 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.682 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.682 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.683 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.683 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.683 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.683 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.683 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.683 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.683 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.683 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.683 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.683 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.683 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.683 INFO:teuthology.orchestra.run.smithi117.stdout: "min_size": { 2024-09-21T06:08:43.683 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "min_size", 2024-09-21T06:08:43.683 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.683 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.684 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.684 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.684 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.684 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.684 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.684 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "default min_size for any created local pool", 2024-09-21T06:08:43.684 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T06:08:43.684 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.684 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.684 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.684 INFO:teuthology.orchestra.run.smithi117.stdout: "num_rep": { 2024-09-21T06:08:43.684 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "num_rep", 2024-09-21T06:08:43.684 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.684 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.684 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.685 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "3", 2024-09-21T06:08:43.685 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.685 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.685 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.685 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "default replica count for any created local pool", 2024-09-21T06:08:43.685 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.685 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.685 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.685 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.685 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_num": { 2024-09-21T06:08:43.685 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "pg_num", 2024-09-21T06:08:43.685 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.685 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.685 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.685 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "128", 2024-09-21T06:08:43.686 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.686 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.686 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.686 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T06:08:43.686 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.686 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.686 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.686 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.686 INFO:teuthology.orchestra.run.smithi117.stdout: "prefix": { 2024-09-21T06:08:43.686 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "prefix", 2024-09-21T06:08:43.686 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.686 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.686 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.686 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.687 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.687 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.687 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.687 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "name prefix for any created local pool", 2024-09-21T06:08:43.687 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.687 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.687 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.687 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.687 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.687 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.687 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.687 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.687 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.687 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.687 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.688 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.688 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.688 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.688 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.688 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.688 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.688 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.688 INFO:teuthology.orchestra.run.smithi117.stdout: "subtree": { 2024-09-21T06:08:43.688 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "subtree", 2024-09-21T06:08:43.688 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.688 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.688 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.688 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "rack", 2024-09-21T06:08:43.688 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.689 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.689 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.689 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T06:08:43.689 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T06:08:43.689 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.689 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.689 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.689 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.689 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.689 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.689 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mds_autoscaler", 2024-09-21T06:08:43.689 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.689 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.689 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.689 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.690 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.690 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.690 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.690 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.690 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.690 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.690 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.690 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.690 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.690 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.690 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.690 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.690 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.690 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.691 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.691 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.691 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.691 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.691 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.691 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.691 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.691 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.691 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.691 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.691 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.691 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.691 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.691 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.691 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.692 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.692 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.692 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.692 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.692 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.692 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.692 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.692 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.692 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.692 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.692 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.692 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.692 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.692 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.692 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.693 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.693 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.693 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.693 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.693 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.693 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.693 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.693 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.693 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.693 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.693 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.693 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.693 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.693 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.693 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.694 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.694 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.694 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.694 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.694 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.694 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.694 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.694 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.694 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.694 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.694 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.694 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.694 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.694 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.694 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.695 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.695 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.695 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.695 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.695 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.695 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.695 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.695 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.695 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.695 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.695 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.695 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.695 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mirroring", 2024-09-21T06:08:43.695 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.695 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.696 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.696 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.696 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.696 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.696 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.696 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.696 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.696 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.696 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.696 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.696 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.696 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.696 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.696 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.696 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.697 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.697 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.697 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.697 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.697 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.697 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.697 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.697 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.697 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.697 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.697 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.697 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.697 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.697 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.697 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.698 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.698 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.698 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.698 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.698 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.698 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.698 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.698 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.698 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.698 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.698 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.698 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.698 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.698 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.698 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.699 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.699 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.699 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.699 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.699 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.699 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.699 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.699 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.699 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.699 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.699 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.699 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.699 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.699 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.700 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.700 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.700 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.700 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.700 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.700 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.700 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.700 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.700 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.700 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.700 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.700 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.700 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.700 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.700 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.701 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.701 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.701 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.701 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.701 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.701 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.701 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.701 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.701 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.701 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.701 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.701 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.701 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.701 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.702 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "nfs", 2024-09-21T06:08:43.702 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.702 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.702 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.702 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.702 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.702 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.702 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.702 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.702 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.702 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.702 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.702 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.702 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.703 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.703 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.703 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.703 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.703 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.703 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.703 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.703 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.703 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.703 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.703 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.703 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.703 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.703 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.703 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.704 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.704 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.704 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.704 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.704 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.704 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.704 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.704 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.704 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.704 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.704 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.704 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.704 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.704 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.704 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.705 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.705 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.705 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.705 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.705 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.705 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.705 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.705 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.705 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.705 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.705 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.705 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.705 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.705 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.705 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.706 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.706 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.706 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.706 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.706 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.706 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.706 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.706 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.706 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.706 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.706 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.706 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.706 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.706 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.707 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.707 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.707 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.707 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.707 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.707 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.707 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.707 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.707 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.707 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.707 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.707 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.707 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.707 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.708 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.708 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.708 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.708 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.708 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "orchestrator", 2024-09-21T06:08:43.708 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.708 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.708 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.708 INFO:teuthology.orchestra.run.smithi117.stdout: "fail_fs": { 2024-09-21T06:08:43.708 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "fail_fs", 2024-09-21T06:08:43.708 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.708 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.708 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.708 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.708 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.709 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.709 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.709 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T06:08:43.709 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.709 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.709 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.709 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.709 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.709 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.709 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.709 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.709 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.709 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.709 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.709 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.710 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.710 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.710 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.710 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.710 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.710 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.710 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.710 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.710 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.710 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.710 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.710 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.710 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.710 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.710 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.711 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.711 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.711 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.711 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.711 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.711 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.711 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.711 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.711 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.711 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.711 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.711 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.711 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.711 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.711 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.711 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.712 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.712 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.712 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.712 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.712 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.712 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.712 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.712 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.712 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.712 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.712 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.712 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.712 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.712 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.713 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.713 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.713 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.713 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.713 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.713 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.713 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.713 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.713 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.713 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.713 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.713 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.713 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.713 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.713 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.714 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.714 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.714 INFO:teuthology.orchestra.run.smithi117.stdout: "orchestrator": { 2024-09-21T06:08:43.714 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "orchestrator", 2024-09-21T06:08:43.714 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.714 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.714 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.714 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.714 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.714 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.714 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.714 INFO:teuthology.orchestra.run.smithi117.stdout: "cephadm", 2024-09-21T06:08:43.714 INFO:teuthology.orchestra.run.smithi117.stdout: "rook", 2024-09-21T06:08:43.714 INFO:teuthology.orchestra.run.smithi117.stdout: "test_orchestrator" 2024-09-21T06:08:43.714 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.715 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Orchestrator backend", 2024-09-21T06:08:43.715 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.715 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.715 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.715 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.715 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.715 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.715 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.715 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.715 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.715 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.715 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.715 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.715 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.715 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.716 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.716 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.716 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.716 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.716 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.716 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.716 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.716 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "osd_perf_query", 2024-09-21T06:08:43.716 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.716 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.716 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.716 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.716 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.716 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.716 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.717 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.717 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.717 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.717 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.717 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.717 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.717 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.717 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.717 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.717 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.717 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.717 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.717 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.717 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.717 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.718 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.718 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.718 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.718 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.718 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.718 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.718 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.718 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.718 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.718 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.718 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.718 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.718 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.718 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.718 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.719 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.719 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.719 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.719 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.719 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.719 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.719 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.719 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.719 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.719 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.719 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.719 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.719 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.719 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.719 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.720 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.720 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.720 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.720 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.720 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.720 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.720 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.720 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.720 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.720 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.720 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.720 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.720 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.720 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.720 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.721 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.721 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.721 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.721 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.721 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.721 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.721 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.721 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.721 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.721 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.721 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.721 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.721 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.721 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.721 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.721 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.722 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.722 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.722 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.722 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.722 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.722 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.722 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.722 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "osd_support", 2024-09-21T06:08:43.722 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.722 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.722 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.722 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.722 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.722 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.722 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.723 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.723 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.723 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.723 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.723 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.723 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.723 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.723 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.723 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.723 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.723 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.723 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.723 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.723 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.723 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.724 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.724 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.724 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.724 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.724 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.724 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.724 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.724 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.724 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.724 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.724 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.724 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.724 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.724 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.724 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.724 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.725 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.725 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.725 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.725 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.725 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.725 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.725 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.725 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.725 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.725 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.725 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.725 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.725 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.725 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.725 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.725 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.726 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.726 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.726 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.726 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.726 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.726 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.726 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.726 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.726 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.726 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.726 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.726 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.726 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.726 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.726 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.727 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.727 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.727 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.727 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.727 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.727 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.727 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.727 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.727 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.727 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.727 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.727 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.727 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.727 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.727 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.727 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.728 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.728 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.728 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.728 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.728 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.728 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "pg_autoscaler", 2024-09-21T06:08:43.728 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.728 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.728 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.728 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.728 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.728 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.728 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.728 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.728 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.729 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.729 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.729 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.729 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.729 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.729 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.729 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.729 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.729 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.729 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.729 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.729 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.729 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.729 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.729 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.729 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.730 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.730 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.730 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.730 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.730 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.730 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.730 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.730 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.730 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.730 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.730 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.730 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.730 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.730 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.730 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.730 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.731 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.731 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.731 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.731 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.731 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.731 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.731 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.731 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.731 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.731 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.731 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.731 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.731 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.731 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.731 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.732 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.732 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.732 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.732 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.732 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.732 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.732 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.732 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.732 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.732 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.732 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.732 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.732 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.732 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.732 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.733 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.733 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.733 INFO:teuthology.orchestra.run.smithi117.stdout: "sleep_interval": { 2024-09-21T06:08:43.733 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sleep_interval", 2024-09-21T06:08:43.733 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.733 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.733 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.733 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "60", 2024-09-21T06:08:43.733 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.733 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.733 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.733 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.733 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.733 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.734 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.734 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.734 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.734 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.734 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.734 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.734 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.734 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.734 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.734 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.734 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.734 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.734 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.734 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.734 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.734 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.735 INFO:teuthology.orchestra.run.smithi117.stdout: "threshold": { 2024-09-21T06:08:43.735 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "threshold", 2024-09-21T06:08:43.735 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-09-21T06:08:43.735 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.735 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.735 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "3.0", 2024-09-21T06:08:43.735 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "1.0", 2024-09-21T06:08:43.735 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.735 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.735 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "scaling threshold", 2024-09-21T06:08:43.735 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T06:08:43.735 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.735 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.736 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.736 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.736 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.736 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.736 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "progress", 2024-09-21T06:08:43.736 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.736 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.736 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.736 INFO:teuthology.orchestra.run.smithi117.stdout: "allow_pg_recovery_event": { 2024-09-21T06:08:43.736 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "allow_pg_recovery_event", 2024-09-21T06:08:43.736 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.736 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.736 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.736 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.736 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.737 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.737 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.737 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T06:08:43.737 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.737 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.737 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.737 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.737 INFO:teuthology.orchestra.run.smithi117.stdout: "enabled": { 2024-09-21T06:08:43.737 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "enabled", 2024-09-21T06:08:43.737 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.737 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.737 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.737 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.737 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.737 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.738 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.738 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.738 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.738 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.738 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.738 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.738 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.738 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.738 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.738 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.738 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.738 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.738 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.738 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.738 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.739 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.739 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.739 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.739 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.739 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.739 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.739 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.739 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.739 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.739 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.739 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.739 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.739 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.739 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.739 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.739 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.740 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.740 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.740 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.740 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.740 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.740 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.740 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.740 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.740 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.740 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.740 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.740 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.740 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.740 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.740 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.741 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.741 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.741 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.741 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.741 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.741 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.741 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.741 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.741 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.741 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.741 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.741 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.741 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.741 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.741 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.742 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.742 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.742 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.742 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.742 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.742 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.742 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.742 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.742 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.742 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.742 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.742 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.742 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.742 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.742 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.742 INFO:teuthology.orchestra.run.smithi117.stdout: "max_completed_events": { 2024-09-21T06:08:43.743 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "max_completed_events", 2024-09-21T06:08:43.743 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.743 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.743 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.743 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "50", 2024-09-21T06:08:43.743 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.743 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.743 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.743 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "number of past completed events to remember", 2024-09-21T06:08:43.743 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.743 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.743 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.743 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.743 INFO:teuthology.orchestra.run.smithi117.stdout: "sleep_interval": { 2024-09-21T06:08:43.743 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sleep_interval", 2024-09-21T06:08:43.743 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.744 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.744 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.744 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "5", 2024-09-21T06:08:43.744 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.744 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.744 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.744 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "how long the module is going to sleep", 2024-09-21T06:08:43.744 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.744 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.744 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.744 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.744 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.744 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.744 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.744 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.745 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.745 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.745 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.745 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.745 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.745 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.745 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.745 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.745 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.745 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.745 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.745 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.745 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.745 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "prometheus", 2024-09-21T06:08:43.745 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.746 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.746 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.746 INFO:teuthology.orchestra.run.smithi117.stdout: "cache": { 2024-09-21T06:08:43.746 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "cache", 2024-09-21T06:08:43.746 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.746 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.746 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.746 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.746 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.746 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.746 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.746 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.746 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.746 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.746 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.746 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.747 INFO:teuthology.orchestra.run.smithi117.stdout: "exclude_perf_counters": { 2024-09-21T06:08:43.747 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "exclude_perf_counters", 2024-09-21T06:08:43.747 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.747 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.747 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.747 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.747 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.747 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.747 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.747 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T06:08:43.747 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T06:08:43.747 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.747 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.747 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.747 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.748 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.748 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.748 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.748 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.748 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.748 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.748 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.748 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.748 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.748 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.748 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.748 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.748 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.748 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.748 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.749 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.749 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.749 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.749 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.749 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.749 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.749 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.749 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.749 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.749 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.749 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.749 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.749 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.749 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.749 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.749 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.750 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.750 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.750 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.750 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.750 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.750 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.750 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.750 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.750 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.750 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.750 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.750 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.750 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.750 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.750 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.751 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.751 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.751 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.751 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.751 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.751 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.751 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.751 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.751 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.751 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.751 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.751 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.751 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.751 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.751 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.752 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.752 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.752 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.752 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.752 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.752 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.752 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.752 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.752 INFO:teuthology.orchestra.run.smithi117.stdout: "rbd_stats_pools": { 2024-09-21T06:08:43.752 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rbd_stats_pools", 2024-09-21T06:08:43.752 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.752 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.752 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.752 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.752 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.753 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.753 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.753 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.753 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.753 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.753 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.753 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.753 INFO:teuthology.orchestra.run.smithi117.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T06:08:43.753 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T06:08:43.753 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.753 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.753 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.753 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "300", 2024-09-21T06:08:43.753 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.753 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.753 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.754 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.754 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.754 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.754 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.754 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.754 INFO:teuthology.orchestra.run.smithi117.stdout: "scrape_interval": { 2024-09-21T06:08:43.754 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "scrape_interval", 2024-09-21T06:08:43.754 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-09-21T06:08:43.754 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.754 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.754 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "15.0", 2024-09-21T06:08:43.754 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.754 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.754 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.754 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.755 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.755 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.755 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.755 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.755 INFO:teuthology.orchestra.run.smithi117.stdout: "server_addr": { 2024-09-21T06:08:43.755 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_addr", 2024-09-21T06:08:43.755 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.755 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.755 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.755 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "::", 2024-09-21T06:08:43.755 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.755 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.755 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.755 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T06:08:43.755 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.756 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.756 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.756 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.756 INFO:teuthology.orchestra.run.smithi117.stdout: "server_port": { 2024-09-21T06:08:43.756 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_port", 2024-09-21T06:08:43.756 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.756 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.756 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.756 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "9283", 2024-09-21T06:08:43.756 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.756 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.756 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.756 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T06:08:43.756 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.756 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.757 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.757 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.757 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.757 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.757 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.757 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.757 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.757 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.757 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.757 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.757 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.757 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.757 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.757 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.757 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.757 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.758 INFO:teuthology.orchestra.run.smithi117.stdout: "stale_cache_strategy": { 2024-09-21T06:08:43.758 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "stale_cache_strategy", 2024-09-21T06:08:43.758 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.758 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.758 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.758 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "log", 2024-09-21T06:08:43.758 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.758 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.758 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.758 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.758 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.758 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.758 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.758 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.759 INFO:teuthology.orchestra.run.smithi117.stdout: "standby_behaviour": { 2024-09-21T06:08:43.759 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "standby_behaviour", 2024-09-21T06:08:43.759 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.759 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.759 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.759 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "default", 2024-09-21T06:08:43.759 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.759 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.759 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.759 INFO:teuthology.orchestra.run.smithi117.stdout: "default", 2024-09-21T06:08:43.759 INFO:teuthology.orchestra.run.smithi117.stdout: "error" 2024-09-21T06:08:43.759 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.759 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.759 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.759 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.760 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.760 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.760 INFO:teuthology.orchestra.run.smithi117.stdout: "standby_error_status_code": { 2024-09-21T06:08:43.760 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "standby_error_status_code", 2024-09-21T06:08:43.760 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.760 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.760 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.760 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "500", 2024-09-21T06:08:43.760 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "400", 2024-09-21T06:08:43.760 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "599", 2024-09-21T06:08:43.760 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.760 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.760 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.760 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.760 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.761 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.761 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.761 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.761 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.761 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rbd_support", 2024-09-21T06:08:43.761 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.761 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.761 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.761 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.761 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.761 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.761 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.761 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.761 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.761 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.762 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.762 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.762 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.762 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.762 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.762 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.762 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.762 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.762 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.762 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.762 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.762 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.762 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.762 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.762 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.763 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.763 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.763 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.763 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.763 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.763 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.763 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.763 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.763 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.763 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.763 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.763 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.763 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.763 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.764 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.764 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.764 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.764 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.764 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.764 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.764 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.764 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.764 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.764 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.764 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.764 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.764 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.764 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.764 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.765 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.765 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.765 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.765 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.765 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.765 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.765 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.765 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.765 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.765 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.765 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.765 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.765 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.765 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.765 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.765 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.766 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.766 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.766 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.766 INFO:teuthology.orchestra.run.smithi117.stdout: "max_concurrent_snap_create": { 2024-09-21T06:08:43.766 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "max_concurrent_snap_create", 2024-09-21T06:08:43.766 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.766 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.766 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.766 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10", 2024-09-21T06:08:43.766 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.766 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.766 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.766 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.766 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.766 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.767 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.767 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.767 INFO:teuthology.orchestra.run.smithi117.stdout: "mirror_snapshot_schedule": { 2024-09-21T06:08:43.767 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T06:08:43.767 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.767 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.767 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.767 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.767 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.767 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.767 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.767 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.767 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.767 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.768 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.768 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.768 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.768 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.768 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.768 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.768 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.768 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.768 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.768 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.768 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.768 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.768 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.768 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.768 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.769 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.769 INFO:teuthology.orchestra.run.smithi117.stdout: "trash_purge_schedule": { 2024-09-21T06:08:43.769 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "trash_purge_schedule", 2024-09-21T06:08:43.769 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.769 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.769 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.769 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.769 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.769 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.769 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.769 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.769 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.769 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.769 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.770 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.770 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.770 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.770 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.770 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "restful", 2024-09-21T06:08:43.770 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.770 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.770 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.770 INFO:teuthology.orchestra.run.smithi117.stdout: "enable_auth": { 2024-09-21T06:08:43.770 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "enable_auth", 2024-09-21T06:08:43.770 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.770 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.770 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.770 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.771 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.771 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.771 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.771 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.771 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.771 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.771 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.771 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.771 INFO:teuthology.orchestra.run.smithi117.stdout: "key_file": { 2024-09-21T06:08:43.771 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "key_file", 2024-09-21T06:08:43.771 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.771 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.771 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.771 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.771 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.771 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.772 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.772 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.772 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.772 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.772 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.772 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.772 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.772 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.772 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.772 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.772 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.772 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.772 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.772 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.772 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.773 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.773 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.773 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.773 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.773 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.773 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.773 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.773 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.773 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.773 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.773 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.773 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.773 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.773 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.773 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.773 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.774 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.774 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.774 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.774 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.774 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.774 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.774 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.774 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.774 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.774 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.774 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.774 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.774 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.774 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.775 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.775 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.775 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.775 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.775 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.775 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.775 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.775 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.775 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.775 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.775 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.775 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.775 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.775 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.775 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.776 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.776 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.776 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.776 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.776 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.776 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.776 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.776 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.776 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.776 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.776 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.776 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.776 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.776 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.776 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.777 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.777 INFO:teuthology.orchestra.run.smithi117.stdout: "max_requests": { 2024-09-21T06:08:43.777 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "max_requests", 2024-09-21T06:08:43.777 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.777 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.777 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.777 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "500", 2024-09-21T06:08:43.777 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.777 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.777 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.777 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T06:08:43.777 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.777 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.777 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.777 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.778 INFO:teuthology.orchestra.run.smithi117.stdout: "server_addr": { 2024-09-21T06:08:43.778 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_addr", 2024-09-21T06:08:43.778 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.778 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.778 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.778 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.778 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.778 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.778 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.778 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.778 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.778 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.778 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.778 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.778 INFO:teuthology.orchestra.run.smithi117.stdout: "server_port": { 2024-09-21T06:08:43.779 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "server_port", 2024-09-21T06:08:43.779 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.779 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.779 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.779 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.779 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.779 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.779 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.779 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.779 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.779 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.779 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.779 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.779 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.779 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.780 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.780 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.780 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.780 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.780 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.780 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.780 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.780 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.780 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.780 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.780 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.780 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.780 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.780 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.781 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.781 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rgw", 2024-09-21T06:08:43.781 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.781 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.781 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.781 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.781 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.781 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.781 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.781 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.781 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.781 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.781 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.781 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.781 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.782 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.782 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.782 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.782 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.782 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.782 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.782 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.782 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.782 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.782 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.782 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.782 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.782 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.782 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.782 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.783 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.783 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.783 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.783 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.783 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.783 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.783 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.783 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.783 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.783 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.783 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.783 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.783 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.783 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.784 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.784 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.784 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.784 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.784 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.784 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.784 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.784 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.784 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.784 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.784 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.784 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.784 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.784 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.784 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.784 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.785 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.785 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.785 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.785 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.785 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.785 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.785 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.785 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.785 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.785 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.785 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.785 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.785 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.785 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.785 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.786 INFO:teuthology.orchestra.run.smithi117.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T06:08:43.786 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T06:08:43.786 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.786 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.786 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.786 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "5", 2024-09-21T06:08:43.786 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.786 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.786 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.786 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T06:08:43.786 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.786 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.786 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.786 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.786 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.787 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.787 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.787 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.787 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.787 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.787 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.787 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.787 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.787 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.787 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.787 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.787 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.787 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.787 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.787 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.788 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.788 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rook", 2024-09-21T06:08:43.788 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.788 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.788 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.788 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.788 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.788 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.788 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.788 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.788 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.788 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.788 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.788 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.788 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.789 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.789 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.789 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.789 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.789 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.789 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.789 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.789 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.789 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.789 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.789 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.789 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.789 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.789 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.789 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.789 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.790 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.790 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.790 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.790 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.790 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.790 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.790 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.790 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.790 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.790 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.790 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.790 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.790 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.790 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.790 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.790 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.791 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.791 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.791 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.791 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.791 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.791 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.791 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.791 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.791 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.791 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.791 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.791 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.791 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.791 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.791 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.792 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.792 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.793 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.793 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.793 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.793 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.793 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.793 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.793 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.793 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.794 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.794 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.794 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.794 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.794 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.794 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.794 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.794 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.794 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.794 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.794 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.794 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.794 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.794 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.794 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.795 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.795 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.795 INFO:teuthology.orchestra.run.smithi117.stdout: "storage_class": { 2024-09-21T06:08:43.795 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "storage_class", 2024-09-21T06:08:43.795 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.795 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.795 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.795 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "local", 2024-09-21T06:08:43.795 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.795 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.795 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.795 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T06:08:43.795 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.795 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.795 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.796 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.796 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.796 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.796 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.796 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "selftest", 2024-09-21T06:08:43.796 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.796 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.796 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.796 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.796 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.796 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.796 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.796 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.796 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.797 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.797 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.797 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.797 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.797 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.797 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.797 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.797 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.797 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.797 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.797 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.797 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.797 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.797 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.798 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.798 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.798 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.798 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.798 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.798 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.798 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.798 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.798 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.798 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.798 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.798 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.798 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.798 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.799 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.799 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.799 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.799 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.799 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.799 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.799 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.799 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.799 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.799 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.799 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.799 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.799 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.799 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.799 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.800 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.800 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.800 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.800 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.800 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.800 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.800 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.800 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.800 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.800 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.800 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.800 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.800 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.800 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.800 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.801 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.801 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.801 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.801 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.801 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.801 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.801 INFO:teuthology.orchestra.run.smithi117.stdout: "roption1": { 2024-09-21T06:08:43.801 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "roption1", 2024-09-21T06:08:43.801 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.801 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.801 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.801 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.801 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.801 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.801 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.801 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.802 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.802 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.802 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.802 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.802 INFO:teuthology.orchestra.run.smithi117.stdout: "roption2": { 2024-09-21T06:08:43.802 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "roption2", 2024-09-21T06:08:43.802 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.802 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.802 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.802 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "xyz", 2024-09-21T06:08:43.802 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.802 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.802 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.802 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.803 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.803 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.803 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.803 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.803 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption1": { 2024-09-21T06:08:43.803 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption1", 2024-09-21T06:08:43.803 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.803 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.803 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.803 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.803 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.803 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.803 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.803 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.804 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.804 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.804 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.804 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.804 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption2": { 2024-09-21T06:08:43.804 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption2", 2024-09-21T06:08:43.804 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.804 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.804 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.804 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.804 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.804 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.804 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.804 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.804 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.805 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.805 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.805 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.805 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption3": { 2024-09-21T06:08:43.805 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption3", 2024-09-21T06:08:43.805 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "float", 2024-09-21T06:08:43.805 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.805 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.805 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.805 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.805 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.805 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.805 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.805 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.805 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.806 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.806 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.806 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption4": { 2024-09-21T06:08:43.806 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption4", 2024-09-21T06:08:43.806 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.806 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.806 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.806 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.806 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.806 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.806 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.806 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.806 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.806 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.806 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.807 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.807 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption5": { 2024-09-21T06:08:43.807 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption5", 2024-09-21T06:08:43.807 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.807 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.807 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.807 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.807 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.807 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.807 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.807 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.807 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.807 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.807 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.808 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.808 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption6": { 2024-09-21T06:08:43.808 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption6", 2024-09-21T06:08:43.808 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.808 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.808 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.808 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.808 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.808 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.808 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.808 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.808 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.808 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.808 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.808 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.809 INFO:teuthology.orchestra.run.smithi117.stdout: "rwoption7": { 2024-09-21T06:08:43.809 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rwoption7", 2024-09-21T06:08:43.809 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.809 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.809 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.809 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.809 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "1", 2024-09-21T06:08:43.809 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "42", 2024-09-21T06:08:43.809 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.809 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.809 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.809 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.809 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.809 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.809 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.810 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.810 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.810 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.810 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.810 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.810 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.810 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.810 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.810 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.810 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.810 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.810 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.810 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.810 INFO:teuthology.orchestra.run.smithi117.stdout: "testkey": { 2024-09-21T06:08:43.810 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "testkey", 2024-09-21T06:08:43.810 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.811 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.811 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.811 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.811 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.811 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.811 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.811 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.811 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.811 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.811 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.811 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.811 INFO:teuthology.orchestra.run.smithi117.stdout: "testlkey": { 2024-09-21T06:08:43.811 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "testlkey", 2024-09-21T06:08:43.811 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.812 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.812 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.812 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.812 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.812 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.812 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.812 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.812 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.812 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.812 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.812 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.812 INFO:teuthology.orchestra.run.smithi117.stdout: "testnewline": { 2024-09-21T06:08:43.812 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "testnewline", 2024-09-21T06:08:43.812 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.813 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.813 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.813 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.813 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.813 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.813 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.813 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.813 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.813 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.813 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.813 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.813 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.813 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.813 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.813 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "smb", 2024-09-21T06:08:43.814 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.814 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.814 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.814 INFO:teuthology.orchestra.run.smithi117.stdout: "internal_store_backend": { 2024-09-21T06:08:43.814 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "internal_store_backend", 2024-09-21T06:08:43.814 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.814 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.814 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.814 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.814 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.814 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.814 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.814 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T06:08:43.814 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.815 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.815 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.815 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.815 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.815 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.815 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.815 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.815 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.815 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.815 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.815 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.815 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.815 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.815 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.816 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.816 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.816 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.816 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.816 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.816 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.816 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.816 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.816 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.816 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.816 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.816 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.816 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.816 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.816 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.816 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.817 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.817 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.817 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.817 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.817 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.817 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.817 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.817 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.817 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.817 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.817 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.817 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.817 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.817 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.817 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.818 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.818 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.818 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.818 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.818 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.818 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.818 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.818 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.818 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.818 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.818 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.818 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.818 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.818 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.818 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.819 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.819 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.819 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.819 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.819 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.819 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.819 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.819 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.819 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.819 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.819 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.819 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.819 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.819 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.820 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.820 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.820 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.820 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.820 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.820 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.820 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.820 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.820 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.820 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.820 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.820 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.820 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.820 INFO:teuthology.orchestra.run.smithi117.stdout: "update_orchestration": { 2024-09-21T06:08:43.820 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "update_orchestration", 2024-09-21T06:08:43.821 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.821 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.821 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.821 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.821 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.821 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.821 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.821 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T06:08:43.821 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.821 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.821 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.821 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.821 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.821 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.821 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.821 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "snap_schedule", 2024-09-21T06:08:43.822 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.822 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.822 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.822 INFO:teuthology.orchestra.run.smithi117.stdout: "allow_m_granularity": { 2024-09-21T06:08:43.822 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "allow_m_granularity", 2024-09-21T06:08:43.822 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.822 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.822 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.822 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.822 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.822 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.822 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.822 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T06:08:43.822 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.823 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.823 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.823 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.823 INFO:teuthology.orchestra.run.smithi117.stdout: "dump_on_update": { 2024-09-21T06:08:43.823 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "dump_on_update", 2024-09-21T06:08:43.823 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.823 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.823 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.823 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.823 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.823 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.823 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.823 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "dump database to debug log on update", 2024-09-21T06:08:43.823 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.823 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.823 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.824 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.824 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.824 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.824 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.824 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.824 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.824 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.824 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.824 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.824 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.824 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.824 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.824 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.824 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.824 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.825 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.825 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.825 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.825 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.825 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.825 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.825 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.825 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.825 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.825 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.825 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.825 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.825 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.825 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.825 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.826 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.826 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.826 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.826 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.826 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.826 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.826 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.826 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.826 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.826 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.826 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.826 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.826 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.826 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.826 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.827 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.827 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.827 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.827 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.827 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.827 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.827 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.827 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.827 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.827 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.827 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.827 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.827 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.827 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.827 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.828 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.828 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.828 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.828 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.828 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.828 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.828 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.828 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.828 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.828 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.828 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.828 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.828 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.828 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.828 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.829 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.829 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.829 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.829 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.829 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.829 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.829 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.829 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.829 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.829 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.829 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.829 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.829 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.829 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "stats", 2024-09-21T06:08:43.830 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.830 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.830 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.830 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.830 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.830 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.830 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.830 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.830 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.830 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.830 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.830 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.830 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.830 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.830 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.831 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.831 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.831 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.831 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.831 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.831 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.831 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.831 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.831 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.831 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.831 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.831 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.831 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.831 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.831 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.832 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.832 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.832 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.832 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.832 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.832 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.832 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.832 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.832 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.832 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.832 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.832 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.832 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.832 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.832 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.833 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.833 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.833 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.833 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.833 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.833 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.833 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.833 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.833 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.833 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.833 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.833 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.833 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.833 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.833 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.833 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.834 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.834 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.834 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.834 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.834 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.834 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.834 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.834 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.834 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.834 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.834 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.834 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.834 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.834 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.834 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.835 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.835 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.835 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.835 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.835 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.835 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.835 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.835 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.835 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.835 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.835 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.835 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.835 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.835 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.836 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "status", 2024-09-21T06:08:43.836 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.836 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.836 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.836 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.836 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.836 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.836 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.836 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.836 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.836 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.836 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.836 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.836 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.836 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.836 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.837 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.837 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.837 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.837 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.837 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.837 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.837 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.837 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.837 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.837 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.837 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.837 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.837 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.837 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.838 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.838 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.838 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.838 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.838 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.838 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.838 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.838 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.838 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.838 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.838 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.838 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.838 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.838 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.838 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.839 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.839 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.839 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.839 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.839 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.839 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.839 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.839 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.839 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.839 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.839 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.839 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.839 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.839 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.839 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.840 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.840 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.840 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.840 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.840 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.840 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.840 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.840 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.840 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.840 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.840 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.840 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.840 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.840 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.840 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.841 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.841 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.841 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.841 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.841 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.841 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.841 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.841 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.841 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.841 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.841 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.841 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.841 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.841 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.841 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.842 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.842 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "telegraf", 2024-09-21T06:08:43.842 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.842 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.842 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.842 INFO:teuthology.orchestra.run.smithi117.stdout: "address": { 2024-09-21T06:08:43.842 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "address", 2024-09-21T06:08:43.842 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.842 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.842 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.842 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T06:08:43.842 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.842 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.842 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.842 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.843 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.843 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.843 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.843 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.843 INFO:teuthology.orchestra.run.smithi117.stdout: "interval": { 2024-09-21T06:08:43.843 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "interval", 2024-09-21T06:08:43.843 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.843 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.843 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.843 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "15", 2024-09-21T06:08:43.843 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.843 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.843 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.843 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.843 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.844 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.844 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.844 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.844 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.844 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.844 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.844 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.844 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.844 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.844 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.844 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.844 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.844 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.844 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.844 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.844 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.845 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.845 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.845 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.845 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.845 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.845 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.845 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.845 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.845 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.845 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.845 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.845 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.845 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.845 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.846 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.846 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.846 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.846 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.846 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.846 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.846 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.846 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.846 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.846 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.846 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.846 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.846 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.847 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.847 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.847 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.847 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.847 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.847 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.847 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.847 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.847 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.847 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.847 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.847 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.847 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.847 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.847 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.848 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.848 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.848 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.848 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.848 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.848 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.848 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.848 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.848 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.848 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.848 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.848 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.848 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.849 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.849 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.849 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.849 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.849 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.849 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.849 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.849 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.849 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.849 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.849 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.849 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.849 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.849 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.849 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.850 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.850 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.850 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.850 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.850 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "telemetry", 2024-09-21T06:08:43.850 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.850 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.850 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.850 INFO:teuthology.orchestra.run.smithi117.stdout: "channel_basic": { 2024-09-21T06:08:43.850 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "channel_basic", 2024-09-21T06:08:43.850 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.850 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.850 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.850 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.850 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.851 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.851 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.851 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T06:08:43.851 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.851 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.851 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.851 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.851 INFO:teuthology.orchestra.run.smithi117.stdout: "channel_crash": { 2024-09-21T06:08:43.851 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "channel_crash", 2024-09-21T06:08:43.851 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.851 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.851 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.851 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.851 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.851 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.851 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.852 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T06:08:43.852 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.852 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.852 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.852 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.852 INFO:teuthology.orchestra.run.smithi117.stdout: "channel_device": { 2024-09-21T06:08:43.852 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "channel_device", 2024-09-21T06:08:43.852 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.852 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.852 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.852 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.852 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.852 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.852 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.852 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T06:08:43.853 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.853 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.853 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.853 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.853 INFO:teuthology.orchestra.run.smithi117.stdout: "channel_ident": { 2024-09-21T06:08:43.853 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "channel_ident", 2024-09-21T06:08:43.853 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.853 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.853 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.853 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.853 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.853 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.853 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.853 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T06:08:43.853 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.853 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.854 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.854 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.854 INFO:teuthology.orchestra.run.smithi117.stdout: "channel_perf": { 2024-09-21T06:08:43.854 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "channel_perf", 2024-09-21T06:08:43.854 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.854 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.854 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.854 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.854 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.854 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.854 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.854 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T06:08:43.854 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.854 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.854 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.855 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.855 INFO:teuthology.orchestra.run.smithi117.stdout: "contact": { 2024-09-21T06:08:43.855 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "contact", 2024-09-21T06:08:43.855 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.855 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.855 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.855 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.855 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.855 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.855 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.855 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.855 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.855 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.855 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.855 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.856 INFO:teuthology.orchestra.run.smithi117.stdout: "description": { 2024-09-21T06:08:43.856 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "description", 2024-09-21T06:08:43.856 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.856 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.856 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.856 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.856 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.856 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.856 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.856 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.856 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.856 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.856 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.856 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.856 INFO:teuthology.orchestra.run.smithi117.stdout: "device_url": { 2024-09-21T06:08:43.857 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "device_url", 2024-09-21T06:08:43.857 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.857 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.857 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.857 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T06:08:43.857 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.857 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.857 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.857 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.857 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.857 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.857 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.857 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.857 INFO:teuthology.orchestra.run.smithi117.stdout: "enabled": { 2024-09-21T06:08:43.857 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "enabled", 2024-09-21T06:08:43.858 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.858 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.858 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.858 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.858 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.858 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.858 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.858 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.858 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.858 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.858 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.858 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.858 INFO:teuthology.orchestra.run.smithi117.stdout: "interval": { 2024-09-21T06:08:43.858 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "interval", 2024-09-21T06:08:43.858 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.859 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.859 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.859 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "24", 2024-09-21T06:08:43.859 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "8", 2024-09-21T06:08:43.859 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.859 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.859 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.859 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.859 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.859 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.859 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.859 INFO:teuthology.orchestra.run.smithi117.stdout: "last_opt_revision": { 2024-09-21T06:08:43.859 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "last_opt_revision", 2024-09-21T06:08:43.859 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.859 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.860 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.860 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "1", 2024-09-21T06:08:43.860 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.860 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.860 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.860 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.860 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.860 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.860 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.860 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.860 INFO:teuthology.orchestra.run.smithi117.stdout: "leaderboard": { 2024-09-21T06:08:43.860 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "leaderboard", 2024-09-21T06:08:43.860 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.860 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.860 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.861 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.861 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.861 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.861 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.861 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.861 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.861 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.861 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.861 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.861 INFO:teuthology.orchestra.run.smithi117.stdout: "leaderboard_description": { 2024-09-21T06:08:43.861 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "leaderboard_description", 2024-09-21T06:08:43.861 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.861 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.861 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.862 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.862 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.862 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.862 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.862 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.862 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.862 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.862 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.862 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.862 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.862 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.862 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.862 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.862 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.862 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.863 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.863 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.863 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.863 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.863 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.863 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.863 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.863 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.863 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.863 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.863 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.863 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.863 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.863 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.863 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.863 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.864 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.864 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.864 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.864 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.864 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.864 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.864 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.864 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.864 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.864 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.864 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.864 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.864 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.864 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.864 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.865 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.865 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.865 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.865 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.865 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.865 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.865 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.865 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.865 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.865 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.865 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.865 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.865 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.865 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.865 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.865 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.865 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.866 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.866 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.866 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.866 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.866 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.866 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.866 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.866 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.866 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.866 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.866 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.866 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.866 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.866 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.867 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.867 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.867 INFO:teuthology.orchestra.run.smithi117.stdout: "organization": { 2024-09-21T06:08:43.867 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "organization", 2024-09-21T06:08:43.867 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.867 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.867 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.867 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.867 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.867 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.867 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.867 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.867 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.867 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.868 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.868 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.868 INFO:teuthology.orchestra.run.smithi117.stdout: "proxy": { 2024-09-21T06:08:43.868 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "proxy", 2024-09-21T06:08:43.868 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.868 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.868 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.868 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.868 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.868 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.868 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.868 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.868 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.868 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.868 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.868 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.869 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.869 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.869 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.869 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.869 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.869 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.869 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.869 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.869 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.869 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.869 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.869 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.869 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.869 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.869 INFO:teuthology.orchestra.run.smithi117.stdout: "url": { 2024-09-21T06:08:43.870 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "url", 2024-09-21T06:08:43.870 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.870 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.870 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.870 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T06:08:43.870 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.870 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.870 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.870 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.870 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.870 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.870 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.870 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.870 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.870 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.871 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.871 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "test_orchestrator", 2024-09-21T06:08:43.871 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.871 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.871 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.871 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.871 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.871 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.871 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.871 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.871 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.871 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.871 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.871 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.872 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.872 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.872 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.872 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.872 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.872 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.872 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.872 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.872 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.872 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.872 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.872 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.872 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.872 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.872 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.873 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.873 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.873 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.873 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.873 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.873 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.873 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.873 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.873 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.873 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.873 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.873 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.873 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.873 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.873 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.873 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.874 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.874 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.874 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.874 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.874 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.874 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.874 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.874 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.874 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.874 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.874 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.874 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.874 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.874 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.874 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.875 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.875 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.875 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.875 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.875 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.875 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.875 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.875 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.875 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.875 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.875 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.875 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.875 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.875 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.875 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.875 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.876 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.876 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.876 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.876 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.876 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.876 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.876 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.876 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.876 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.876 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.876 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.876 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.876 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.876 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.877 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.877 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.877 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "volumes", 2024-09-21T06:08:43.877 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.877 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.877 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.877 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.877 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.877 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.877 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.877 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.877 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.877 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.877 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.877 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.878 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.878 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.878 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.878 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.878 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.878 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.878 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.878 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.878 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.878 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.878 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.878 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.878 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.878 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.879 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.879 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.879 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.879 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.879 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.879 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.879 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.879 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.879 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.879 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.879 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.879 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.879 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.879 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.879 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.880 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.880 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.880 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.880 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.880 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.880 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.880 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.880 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.880 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.880 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.880 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.880 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.880 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.880 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.880 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.880 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.880 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.881 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.881 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.881 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.881 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.881 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.881 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.881 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.881 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.881 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.881 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.881 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.881 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.881 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.881 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.881 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.882 INFO:teuthology.orchestra.run.smithi117.stdout: "max_concurrent_clones": { 2024-09-21T06:08:43.882 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "max_concurrent_clones", 2024-09-21T06:08:43.882 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.882 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.882 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.882 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "4", 2024-09-21T06:08:43.882 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.882 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.882 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.882 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T06:08:43.882 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.882 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.882 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.882 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.883 INFO:teuthology.orchestra.run.smithi117.stdout: "periodic_async_work": { 2024-09-21T06:08:43.883 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "periodic_async_work", 2024-09-21T06:08:43.883 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.883 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.883 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.883 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.883 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.883 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.883 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.883 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Periodically check for async work", 2024-09-21T06:08:43.883 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.883 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.883 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.883 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.884 INFO:teuthology.orchestra.run.smithi117.stdout: "snapshot_clone_delay": { 2024-09-21T06:08:43.884 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "snapshot_clone_delay", 2024-09-21T06:08:43.884 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.884 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.884 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.884 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.884 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.884 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.884 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.884 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T06:08:43.884 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.884 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.884 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.884 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.884 INFO:teuthology.orchestra.run.smithi117.stdout: "snapshot_clone_no_wait": { 2024-09-21T06:08:43.884 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T06:08:43.885 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.885 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.885 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.885 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "True", 2024-09-21T06:08:43.885 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.885 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.885 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.885 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T06:08:43.885 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.885 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.885 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.885 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.885 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.885 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.885 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.886 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.886 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.886 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.886 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.886 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.886 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.886 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.886 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.886 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.886 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.886 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.886 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.886 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.886 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.886 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "zabbix", 2024-09-21T06:08:43.887 INFO:teuthology.orchestra.run.smithi117.stdout: "can_run": true, 2024-09-21T06:08:43.887 INFO:teuthology.orchestra.run.smithi117.stdout: "error_string": "", 2024-09-21T06:08:43.887 INFO:teuthology.orchestra.run.smithi117.stdout: "module_options": { 2024-09-21T06:08:43.887 INFO:teuthology.orchestra.run.smithi117.stdout: "discovery_interval": { 2024-09-21T06:08:43.887 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "discovery_interval", 2024-09-21T06:08:43.887 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "uint", 2024-09-21T06:08:43.887 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.887 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.887 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "100", 2024-09-21T06:08:43.887 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.887 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.887 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.887 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.887 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.887 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.888 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.888 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.888 INFO:teuthology.orchestra.run.smithi117.stdout: "identifier": { 2024-09-21T06:08:43.888 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "identifier", 2024-09-21T06:08:43.888 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.888 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.888 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.888 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.888 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.888 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.888 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.888 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.888 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.888 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.889 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.889 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.889 INFO:teuthology.orchestra.run.smithi117.stdout: "interval": { 2024-09-21T06:08:43.889 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "interval", 2024-09-21T06:08:43.889 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "secs", 2024-09-21T06:08:43.889 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.889 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.889 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "60", 2024-09-21T06:08:43.889 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.889 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.889 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.889 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.889 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.889 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.889 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.889 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.890 INFO:teuthology.orchestra.run.smithi117.stdout: "log_level": { 2024-09-21T06:08:43.890 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_level", 2024-09-21T06:08:43.890 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.890 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.890 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.890 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.890 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.890 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.890 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.890 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.890 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.890 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.890 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.890 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.890 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.891 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.891 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.891 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.891 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.891 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.891 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.891 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster": { 2024-09-21T06:08:43.891 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster", 2024-09-21T06:08:43.891 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.891 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.891 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.891 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.891 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.891 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.891 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.892 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.892 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.892 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.892 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.892 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.892 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_cluster_level": { 2024-09-21T06:08:43.892 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_cluster_level", 2024-09-21T06:08:43.892 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.892 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.892 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.892 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "info", 2024-09-21T06:08:43.892 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.892 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.892 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [ 2024-09-21T06:08:43.892 INFO:teuthology.orchestra.run.smithi117.stdout: "", 2024-09-21T06:08:43.893 INFO:teuthology.orchestra.run.smithi117.stdout: "critical", 2024-09-21T06:08:43.893 INFO:teuthology.orchestra.run.smithi117.stdout: "debug", 2024-09-21T06:08:43.893 INFO:teuthology.orchestra.run.smithi117.stdout: "error", 2024-09-21T06:08:43.893 INFO:teuthology.orchestra.run.smithi117.stdout: "info", 2024-09-21T06:08:43.893 INFO:teuthology.orchestra.run.smithi117.stdout: "warning" 2024-09-21T06:08:43.893 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.893 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.893 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.893 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.893 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.893 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.893 INFO:teuthology.orchestra.run.smithi117.stdout: "log_to_file": { 2024-09-21T06:08:43.893 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "log_to_file", 2024-09-21T06:08:43.893 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "bool", 2024-09-21T06:08:43.893 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.894 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.894 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "False", 2024-09-21T06:08:43.894 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.894 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.894 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.894 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.894 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.894 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.894 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.894 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.894 INFO:teuthology.orchestra.run.smithi117.stdout: "sqlite3_killpoint": { 2024-09-21T06:08:43.894 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "sqlite3_killpoint", 2024-09-21T06:08:43.894 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.894 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "dev", 2024-09-21T06:08:43.894 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 1, 2024-09-21T06:08:43.895 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "0", 2024-09-21T06:08:43.895 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.895 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.895 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.895 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.895 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.895 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.895 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.895 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.895 INFO:teuthology.orchestra.run.smithi117.stdout: "zabbix_host": { 2024-09-21T06:08:43.895 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "zabbix_host", 2024-09-21T06:08:43.895 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.895 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.895 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.895 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "", 2024-09-21T06:08:43.896 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.896 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.896 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.896 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.896 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.896 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.896 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.896 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.896 INFO:teuthology.orchestra.run.smithi117.stdout: "zabbix_port": { 2024-09-21T06:08:43.896 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "zabbix_port", 2024-09-21T06:08:43.896 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "int", 2024-09-21T06:08:43.896 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.896 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.896 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "10051", 2024-09-21T06:08:43.896 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.897 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.897 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.897 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.897 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.897 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.897 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.897 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.897 INFO:teuthology.orchestra.run.smithi117.stdout: "zabbix_sender": { 2024-09-21T06:08:43.897 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "zabbix_sender", 2024-09-21T06:08:43.897 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "str", 2024-09-21T06:08:43.897 INFO:teuthology.orchestra.run.smithi117.stdout: "level": "advanced", 2024-09-21T06:08:43.897 INFO:teuthology.orchestra.run.smithi117.stdout: "flags": 0, 2024-09-21T06:08:43.897 INFO:teuthology.orchestra.run.smithi117.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T06:08:43.897 INFO:teuthology.orchestra.run.smithi117.stdout: "min": "", 2024-09-21T06:08:43.898 INFO:teuthology.orchestra.run.smithi117.stdout: "max": "", 2024-09-21T06:08:43.898 INFO:teuthology.orchestra.run.smithi117.stdout: "enum_allowed": [], 2024-09-21T06:08:43.898 INFO:teuthology.orchestra.run.smithi117.stdout: "desc": "", 2024-09-21T06:08:43.898 INFO:teuthology.orchestra.run.smithi117.stdout: "long_desc": "", 2024-09-21T06:08:43.898 INFO:teuthology.orchestra.run.smithi117.stdout: "tags": [], 2024-09-21T06:08:43.898 INFO:teuthology.orchestra.run.smithi117.stdout: "see_also": [] 2024-09-21T06:08:43.898 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.898 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.898 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.898 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.898 INFO:teuthology.orchestra.run.smithi117.stdout: "services": { 2024-09-21T06:08:43.898 INFO:teuthology.orchestra.run.smithi117.stdout: "dashboard": "https://172.21.15.117:8443/" 2024-09-21T06:08:43.898 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.898 INFO:teuthology.orchestra.run.smithi117.stdout: "always_on_modules": { 2024-09-21T06:08:43.898 INFO:teuthology.orchestra.run.smithi117.stdout: "octopus": [ 2024-09-21T06:08:43.899 INFO:teuthology.orchestra.run.smithi117.stdout: "balancer", 2024-09-21T06:08:43.899 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-09-21T06:08:43.899 INFO:teuthology.orchestra.run.smithi117.stdout: "devicehealth", 2024-09-21T06:08:43.899 INFO:teuthology.orchestra.run.smithi117.stdout: "orchestrator", 2024-09-21T06:08:43.899 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_autoscaler", 2024-09-21T06:08:43.899 INFO:teuthology.orchestra.run.smithi117.stdout: "progress", 2024-09-21T06:08:43.899 INFO:teuthology.orchestra.run.smithi117.stdout: "rbd_support", 2024-09-21T06:08:43.899 INFO:teuthology.orchestra.run.smithi117.stdout: "status", 2024-09-21T06:08:43.899 INFO:teuthology.orchestra.run.smithi117.stdout: "telemetry", 2024-09-21T06:08:43.899 INFO:teuthology.orchestra.run.smithi117.stdout: "volumes" 2024-09-21T06:08:43.899 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.899 INFO:teuthology.orchestra.run.smithi117.stdout: "pacific": [ 2024-09-21T06:08:43.899 INFO:teuthology.orchestra.run.smithi117.stdout: "balancer", 2024-09-21T06:08:43.899 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-09-21T06:08:43.900 INFO:teuthology.orchestra.run.smithi117.stdout: "devicehealth", 2024-09-21T06:08:43.900 INFO:teuthology.orchestra.run.smithi117.stdout: "orchestrator", 2024-09-21T06:08:43.900 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_autoscaler", 2024-09-21T06:08:43.900 INFO:teuthology.orchestra.run.smithi117.stdout: "progress", 2024-09-21T06:08:43.900 INFO:teuthology.orchestra.run.smithi117.stdout: "rbd_support", 2024-09-21T06:08:43.900 INFO:teuthology.orchestra.run.smithi117.stdout: "status", 2024-09-21T06:08:43.900 INFO:teuthology.orchestra.run.smithi117.stdout: "telemetry", 2024-09-21T06:08:43.900 INFO:teuthology.orchestra.run.smithi117.stdout: "volumes" 2024-09-21T06:08:43.900 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.900 INFO:teuthology.orchestra.run.smithi117.stdout: "quincy": [ 2024-09-21T06:08:43.900 INFO:teuthology.orchestra.run.smithi117.stdout: "balancer", 2024-09-21T06:08:43.900 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-09-21T06:08:43.900 INFO:teuthology.orchestra.run.smithi117.stdout: "devicehealth", 2024-09-21T06:08:43.900 INFO:teuthology.orchestra.run.smithi117.stdout: "orchestrator", 2024-09-21T06:08:43.900 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_autoscaler", 2024-09-21T06:08:43.900 INFO:teuthology.orchestra.run.smithi117.stdout: "progress", 2024-09-21T06:08:43.901 INFO:teuthology.orchestra.run.smithi117.stdout: "rbd_support", 2024-09-21T06:08:43.901 INFO:teuthology.orchestra.run.smithi117.stdout: "status", 2024-09-21T06:08:43.901 INFO:teuthology.orchestra.run.smithi117.stdout: "telemetry", 2024-09-21T06:08:43.901 INFO:teuthology.orchestra.run.smithi117.stdout: "volumes" 2024-09-21T06:08:43.901 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.901 INFO:teuthology.orchestra.run.smithi117.stdout: "reef": [ 2024-09-21T06:08:43.901 INFO:teuthology.orchestra.run.smithi117.stdout: "balancer", 2024-09-21T06:08:43.901 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-09-21T06:08:43.901 INFO:teuthology.orchestra.run.smithi117.stdout: "devicehealth", 2024-09-21T06:08:43.901 INFO:teuthology.orchestra.run.smithi117.stdout: "orchestrator", 2024-09-21T06:08:43.901 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_autoscaler", 2024-09-21T06:08:43.901 INFO:teuthology.orchestra.run.smithi117.stdout: "progress", 2024-09-21T06:08:43.901 INFO:teuthology.orchestra.run.smithi117.stdout: "rbd_support", 2024-09-21T06:08:43.901 INFO:teuthology.orchestra.run.smithi117.stdout: "status", 2024-09-21T06:08:43.902 INFO:teuthology.orchestra.run.smithi117.stdout: "telemetry", 2024-09-21T06:08:43.902 INFO:teuthology.orchestra.run.smithi117.stdout: "volumes" 2024-09-21T06:08:43.902 INFO:teuthology.orchestra.run.smithi117.stdout: ], 2024-09-21T06:08:43.902 INFO:teuthology.orchestra.run.smithi117.stdout: "squid": [ 2024-09-21T06:08:43.902 INFO:teuthology.orchestra.run.smithi117.stdout: "balancer", 2024-09-21T06:08:43.902 INFO:teuthology.orchestra.run.smithi117.stdout: "crash", 2024-09-21T06:08:43.902 INFO:teuthology.orchestra.run.smithi117.stdout: "devicehealth", 2024-09-21T06:08:43.902 INFO:teuthology.orchestra.run.smithi117.stdout: "orchestrator", 2024-09-21T06:08:43.902 INFO:teuthology.orchestra.run.smithi117.stdout: "pg_autoscaler", 2024-09-21T06:08:43.902 INFO:teuthology.orchestra.run.smithi117.stdout: "progress", 2024-09-21T06:08:43.902 INFO:teuthology.orchestra.run.smithi117.stdout: "rbd_support", 2024-09-21T06:08:43.902 INFO:teuthology.orchestra.run.smithi117.stdout: "status", 2024-09-21T06:08:43.902 INFO:teuthology.orchestra.run.smithi117.stdout: "telemetry", 2024-09-21T06:08:43.902 INFO:teuthology.orchestra.run.smithi117.stdout: "volumes" 2024-09-21T06:08:43.902 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-21T06:08:43.903 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.903 INFO:teuthology.orchestra.run.smithi117.stdout: "last_failure_osd_epoch": 5, 2024-09-21T06:08:43.903 INFO:teuthology.orchestra.run.smithi117.stdout: "active_clients": [ 2024-09-21T06:08:43.903 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.903 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "devicehealth", 2024-09-21T06:08:43.903 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2024-09-21T06:08:43.903 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.903 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2024-09-21T06:08:43.903 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:0", 2024-09-21T06:08:43.903 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 465263213 2024-09-21T06:08:43.903 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.903 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-21T06:08:43.903 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.903 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.904 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "libcephsqlite", 2024-09-21T06:08:43.904 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2024-09-21T06:08:43.904 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.904 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2024-09-21T06:08:43.904 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:0", 2024-09-21T06:08:43.904 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 1446309389 2024-09-21T06:08:43.904 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.904 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-21T06:08:43.904 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.904 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.904 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "rbd_support", 2024-09-21T06:08:43.904 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2024-09-21T06:08:43.904 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.904 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2024-09-21T06:08:43.904 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:0", 2024-09-21T06:08:43.905 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 2927815512 2024-09-21T06:08:43.905 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.905 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-21T06:08:43.905 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.905 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.905 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "snap_schedule", 2024-09-21T06:08:43.905 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2024-09-21T06:08:43.905 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.905 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2024-09-21T06:08:43.905 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:0", 2024-09-21T06:08:43.905 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 3446306589 2024-09-21T06:08:43.905 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.905 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-21T06:08:43.905 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.905 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.905 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "snap_schedule", 2024-09-21T06:08:43.906 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2024-09-21T06:08:43.906 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.906 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2024-09-21T06:08:43.906 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:0", 2024-09-21T06:08:43.906 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 3222082864 2024-09-21T06:08:43.906 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.906 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-21T06:08:43.906 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:43.906 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.906 INFO:teuthology.orchestra.run.smithi117.stdout: "name": "volumes", 2024-09-21T06:08:43.906 INFO:teuthology.orchestra.run.smithi117.stdout: "addrvec": [ 2024-09-21T06:08:43.906 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-21T06:08:43.906 INFO:teuthology.orchestra.run.smithi117.stdout: "type": "v2", 2024-09-21T06:08:43.906 INFO:teuthology.orchestra.run.smithi117.stdout: "addr": "172.21.15.117:0", 2024-09-21T06:08:43.906 INFO:teuthology.orchestra.run.smithi117.stdout: "nonce": 1222785116 2024-09-21T06:08:43.907 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.907 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-21T06:08:43.907 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:43.907 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-21T06:08:43.907 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T06:08:43.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.531+0000 7f1447151640 1 -- 172.21.15.117:0/2438118482 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f142c0b6680 msgr2=0x7f142c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:43.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.531+0000 7f1447151640 1 --2- 172.21.15.117:0/2438118482 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f142c0b6680 0x7f142c0b8b90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1428031440 tx=0x7f1428032ed0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.531+0000 7f1447151640 1 -- 172.21.15.117:0/2438118482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14401083c0 msgr2=0x7f144019fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:08:43.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.531+0000 7f1447151640 1 --2- 172.21.15.117:0/2438118482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14401083c0 0x7f144019fc20 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f143800cad0 tx=0x7f143800cf90 comp rx=0 tx=0).stop 2024-09-21T06:08:43.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.531+0000 7f1447151640 1 -- 172.21.15.117:0/2438118482 shutdown_connections 2024-09-21T06:08:43.908 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.531+0000 7f1447151640 1 --2- 172.21.15.117:0/2438118482 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f142c0b6680 0x7f142c0b8b90 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.531+0000 7f1447151640 1 --2- 172.21.15.117:0/2438118482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14401083c0 0x7f144019fc20 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.531+0000 7f1447151640 1 --2- 172.21.15.117:0/2438118482 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1440107a70 0x7f144019a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.531+0000 7f1447151640 1 --2- 172.21.15.117:0/2438118482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1440102190 0x7f144019a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:08:43.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.531+0000 7f1447151640 1 -- 172.21.15.117:0/2438118482 >> 172.21.15.117:0/2438118482 conn(0x7f1440079ce0 msgr2=0x7f1440105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:08:43.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.531+0000 7f1447151640 1 -- 172.21.15.117:0/2438118482 shutdown_connections 2024-09-21T06:08:43.909 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:08:42.531+0000 7f1447151640 1 -- 172.21.15.117:0/2438118482 wait complete. 2024-09-21T06:08:43.955 DEBUG:tasks.check_counter:Getting stats from a 2024-09-21T06:08:43.955 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph daemon mds.a perf dump 2024-09-21T06:08:43.956 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:42 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/1851643720' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T06:08:43.956 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:42 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2438118482' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T06:08:43.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:42 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/1851643720' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T06:08:43.956 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:42 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2438118482' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T06:08:43.957 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:42 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/1851643720' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T06:08:43.957 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:42 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2438118482' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T06:08:44.213 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:08:44.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:43 smithi192 ceph-mon[34700]: pgmap v678: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.7 KiB/s wr, 86 op/s 2024-09-21T06:08:44.278 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:43 smithi117 ceph-mon[30947]: pgmap v678: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.7 KiB/s wr, 86 op/s 2024-09-21T06:08:44.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:43 smithi138 ceph-mon[34391]: pgmap v678: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.7 KiB/s wr, 86 op/s 2024-09-21T06:08:44.781 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T06:08:44.781 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:08:44.781 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 1596, 2024-09-21T06:08:44.782 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 1598, 2024-09-21T06:08:44.782 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 478740, 2024-09-21T06:08:44.782 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 507529, 2024-09-21T06:08:44.782 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 6, 2024-09-21T06:08:44.782 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 2, 2024-09-21T06:08:44.782 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.232173505, 2024-09-21T06:08:44.782 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.087587201, 2024-09-21T06:08:44.782 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.057134969, 2024-09-21T06:08:44.782 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.072165216, 2024-09-21T06:08:44.782 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:08:44.782 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1596, 2024-09-21T06:08:44.782 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.223360498, 2024-09-21T06:08:44.782 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000139950 2024-09-21T06:08:44.783 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.783 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:08:44.783 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.783 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.783 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.783 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.783 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T06:08:44.783 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T06:08:44.783 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.783 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:08:44.783 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 949, 2024-09-21T06:08:44.783 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 939, 2024-09-21T06:08:44.783 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 1056954, 2024-09-21T06:08:44.783 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 361786, 2024-09-21T06:08:44.784 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 4, 2024-09-21T06:08:44.784 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 3, 2024-09-21T06:08:44.784 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.165307427, 2024-09-21T06:08:44.784 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.059134989, 2024-09-21T06:08:44.784 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.061977343, 2024-09-21T06:08:44.784 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.041695146, 2024-09-21T06:08:44.784 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:08:44.784 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 939, 2024-09-21T06:08:44.784 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.092222925, 2024-09-21T06:08:44.784 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000098213 2024-09-21T06:08:44.784 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.784 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:08:44.784 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.784 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.785 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.785 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.785 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 883856, 2024-09-21T06:08:44.785 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 138800 2024-09-21T06:08:44.785 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.785 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:08:44.785 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 2, 2024-09-21T06:08:44.785 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 169, 2024-09-21T06:08:44.785 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 428, 2024-09-21T06:08:44.785 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 383438, 2024-09-21T06:08:44.785 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 4, 2024-09-21T06:08:44.785 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 3, 2024-09-21T06:08:44.785 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.016838197, 2024-09-21T06:08:44.785 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.013337004, 2024-09-21T06:08:44.786 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.001529965, 2024-09-21T06:08:44.786 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000031065, 2024-09-21T06:08:44.786 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:08:44.786 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 167, 2024-09-21T06:08:44.786 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.013642246, 2024-09-21T06:08:44.786 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000081690 2024-09-21T06:08:44.786 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.786 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:08:44.786 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.786 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.786 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.786 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.786 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T06:08:44.786 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 383120 2024-09-21T06:08:44.786 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.787 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2024-09-21T06:08:44.787 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 1, 2024-09-21T06:08:44.787 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2024-09-21T06:08:44.787 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.787 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-MDSRank": { 2024-09-21T06:08:44.787 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-09-21T06:08:44.787 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-09-21T06:08:44.787 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 3758, 2024-09-21T06:08:44.787 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.167051031, 2024-09-21T06:08:44.787 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000044452 2024-09-21T06:08:44.787 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:44.787 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.787 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-PurgeQueue": { 2024-09-21T06:08:44.787 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-09-21T06:08:44.788 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-09-21T06:08:44.788 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.788 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.788 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.788 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:44.788 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.788 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-09-21T06:08:44.788 INFO:teuthology.orchestra.run.smithi117.stdout: "request": 0, 2024-09-21T06:08:44.788 INFO:teuthology.orchestra.run.smithi117.stdout: "reply": 0, 2024-09-21T06:08:44.788 INFO:teuthology.orchestra.run.smithi117.stdout: "reply_latency": { 2024-09-21T06:08:44.788 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.788 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.788 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.788 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.789 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_reply": 0, 2024-09-21T06:08:44.789 INFO:teuthology.orchestra.run.smithi117.stdout: "forward": 0, 2024-09-21T06:08:44.789 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_complete": 0, 2024-09-21T06:08:44.789 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_keys": 0, 2024-09-21T06:08:44.789 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_commit": 0, 2024-09-21T06:08:44.789 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_split": 0, 2024-09-21T06:08:44.789 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_merge": 0, 2024-09-21T06:08:44.789 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes": 0, 2024-09-21T06:08:44.789 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_top": 0, 2024-09-21T06:08:44.789 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_bottom": 0, 2024-09-21T06:08:44.789 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pin_tail": 0, 2024-09-21T06:08:44.789 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pinned": 0, 2024-09-21T06:08:44.789 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_expired": 0, 2024-09-21T06:08:44.789 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_with_caps": 0, 2024-09-21T06:08:44.789 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": 0, 2024-09-21T06:08:44.789 INFO:teuthology.orchestra.run.smithi117.stdout: "subtrees": 1, 2024-09-21T06:08:44.790 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse": 0, 2024-09-21T06:08:44.790 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_hit": 0, 2024-09-21T06:08:44.790 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_forward": 0, 2024-09-21T06:08:44.790 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_discover": 0, 2024-09-21T06:08:44.790 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_dir_fetch": 0, 2024-09-21T06:08:44.790 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_remote_ino": 0, 2024-09-21T06:08:44.790 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_lock": 0, 2024-09-21T06:08:44.790 INFO:teuthology.orchestra.run.smithi117.stdout: "load_cent": 0, 2024-09-21T06:08:44.790 INFO:teuthology.orchestra.run.smithi117.stdout: "q": 0, 2024-09-21T06:08:44.790 INFO:teuthology.orchestra.run.smithi117.stdout: "exported": 0, 2024-09-21T06:08:44.790 INFO:teuthology.orchestra.run.smithi117.stdout: "exported_inodes": 0, 2024-09-21T06:08:44.790 INFO:teuthology.orchestra.run.smithi117.stdout: "imported": 0, 2024-09-21T06:08:44.790 INFO:teuthology.orchestra.run.smithi117.stdout: "imported_inodes": 0, 2024-09-21T06:08:44.790 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_dir_fetch": 0, 2024-09-21T06:08:44.790 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_backtrace_fetch": 0, 2024-09-21T06:08:44.791 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_peer_discover": 0, 2024-09-21T06:08:44.791 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rfiles": 0, 2024-09-21T06:08:44.791 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rbytes": 0, 2024-09-21T06:08:44.791 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rsnaps": 0, 2024-09-21T06:08:44.791 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T06:08:44.791 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_set_tag": 0, 2024-09-21T06:08:44.791 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T06:08:44.791 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_inotable_repaired": 0, 2024-09-21T06:08:44.791 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_inodes": 0, 2024-09-21T06:08:44.791 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T06:08:44.791 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T06:08:44.791 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_file_inodes": 0, 2024-09-21T06:08:44.791 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_inode_file_caps": 0, 2024-09-21T06:08:44.791 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T06:08:44.791 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_grant": 0, 2024-09-21T06:08:44.792 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T06:08:44.792 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T06:08:44.792 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T06:08:44.792 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps": 0, 2024-09-21T06:08:44.792 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps_dirty": 0, 2024-09-21T06:08:44.792 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_cap_release": 0, 2024-09-21T06:08:44.792 INFO:teuthology.orchestra.run.smithi117.stdout: "process_request_cap_release": 0 2024-09-21T06:08:44.792 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.792 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_cache": { 2024-09-21T06:08:44.792 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update": 0, 2024-09-21T06:08:44.792 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update_receipt": 0, 2024-09-21T06:08:44.792 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_try_discover": 0, 2024-09-21T06:08:44.792 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_send_discover": 0, 2024-09-21T06:08:44.792 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_handle_discover": 0, 2024-09-21T06:08:44.792 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays": 0, 2024-09-21T06:08:44.792 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_delayed": 0, 2024-09-21T06:08:44.793 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_enqueuing": 0, 2024-09-21T06:08:44.793 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_created": 0, 2024-09-21T06:08:44.793 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_enqueued": 0, 2024-09-21T06:08:44.793 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_reintegrated": 0, 2024-09-21T06:08:44.793 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_migrated": 0, 2024-09-21T06:08:44.793 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_processing": 0, 2024-09-21T06:08:44.793 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_enqueued": 0, 2024-09-21T06:08:44.793 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_prioritized": 0, 2024-09-21T06:08:44.793 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_started": 0, 2024-09-21T06:08:44.793 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_completed": 0, 2024-09-21T06:08:44.793 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_path": 0, 2024-09-21T06:08:44.793 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_inode": 0, 2024-09-21T06:08:44.793 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T06:08:44.793 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_exportdir": 0, 2024-09-21T06:08:44.794 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_flush": 0, 2024-09-21T06:08:44.794 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragmentdir": 0, 2024-09-21T06:08:44.794 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragstats": 0, 2024-09-21T06:08:44.794 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_inodestats": 0, 2024-09-21T06:08:44.794 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:08:44.794 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_succeeded": 0, 2024-09-21T06:08:44.794 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_write_failed": 0 2024-09-21T06:08:44.794 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.794 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_log": { 2024-09-21T06:08:44.794 INFO:teuthology.orchestra.run.smithi117.stdout: "evlrg": 0, 2024-09-21T06:08:44.794 INFO:teuthology.orchestra.run.smithi117.stdout: "evadd": 0, 2024-09-21T06:08:44.794 INFO:teuthology.orchestra.run.smithi117.stdout: "evex": 0, 2024-09-21T06:08:44.794 INFO:teuthology.orchestra.run.smithi117.stdout: "evtrm": 0, 2024-09-21T06:08:44.794 INFO:teuthology.orchestra.run.smithi117.stdout: "ev": 6, 2024-09-21T06:08:44.794 INFO:teuthology.orchestra.run.smithi117.stdout: "evexg": 0, 2024-09-21T06:08:44.795 INFO:teuthology.orchestra.run.smithi117.stdout: "evexd": 0, 2024-09-21T06:08:44.795 INFO:teuthology.orchestra.run.smithi117.stdout: "segadd": 0, 2024-09-21T06:08:44.795 INFO:teuthology.orchestra.run.smithi117.stdout: "segex": 0, 2024-09-21T06:08:44.795 INFO:teuthology.orchestra.run.smithi117.stdout: "segtrm": 0, 2024-09-21T06:08:44.795 INFO:teuthology.orchestra.run.smithi117.stdout: "seg": 2, 2024-09-21T06:08:44.795 INFO:teuthology.orchestra.run.smithi117.stdout: "segmjr": 2, 2024-09-21T06:08:44.795 INFO:teuthology.orchestra.run.smithi117.stdout: "segexg": 0, 2024-09-21T06:08:44.795 INFO:teuthology.orchestra.run.smithi117.stdout: "segexd": 0, 2024-09-21T06:08:44.795 INFO:teuthology.orchestra.run.smithi117.stdout: "expos": 4194304, 2024-09-21T06:08:44.795 INFO:teuthology.orchestra.run.smithi117.stdout: "wrpos": 4196123, 2024-09-21T06:08:44.795 INFO:teuthology.orchestra.run.smithi117.stdout: "rdpos": 4195732, 2024-09-21T06:08:44.795 INFO:teuthology.orchestra.run.smithi117.stdout: "jlat": { 2024-09-21T06:08:44.795 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.795 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.795 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.796 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.796 INFO:teuthology.orchestra.run.smithi117.stdout: "replayed": 6 2024-09-21T06:08:44.796 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.796 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_mem": { 2024-09-21T06:08:44.796 INFO:teuthology.orchestra.run.smithi117.stdout: "ino": 3, 2024-09-21T06:08:44.796 INFO:teuthology.orchestra.run.smithi117.stdout: "ino+": 3, 2024-09-21T06:08:44.796 INFO:teuthology.orchestra.run.smithi117.stdout: "ino-": 0, 2024-09-21T06:08:44.796 INFO:teuthology.orchestra.run.smithi117.stdout: "dir": 1, 2024-09-21T06:08:44.796 INFO:teuthology.orchestra.run.smithi117.stdout: "dir+": 1, 2024-09-21T06:08:44.796 INFO:teuthology.orchestra.run.smithi117.stdout: "dir-": 0, 2024-09-21T06:08:44.796 INFO:teuthology.orchestra.run.smithi117.stdout: "dn": 0, 2024-09-21T06:08:44.796 INFO:teuthology.orchestra.run.smithi117.stdout: "dn+": 0, 2024-09-21T06:08:44.796 INFO:teuthology.orchestra.run.smithi117.stdout: "dn-": 0, 2024-09-21T06:08:44.796 INFO:teuthology.orchestra.run.smithi117.stdout: "cap": 0, 2024-09-21T06:08:44.797 INFO:teuthology.orchestra.run.smithi117.stdout: "cap+": 0, 2024-09-21T06:08:44.797 INFO:teuthology.orchestra.run.smithi117.stdout: "cap-": 0, 2024-09-21T06:08:44.797 INFO:teuthology.orchestra.run.smithi117.stdout: "rss": 117564, 2024-09-21T06:08:44.797 INFO:teuthology.orchestra.run.smithi117.stdout: "heap": 316420096 2024-09-21T06:08:44.797 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.797 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_server": { 2024-09-21T06:08:44.797 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_client_request": 0, 2024-09-21T06:08:44.797 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_server_request": 0, 2024-09-21T06:08:44.797 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_request": 0, 2024-09-21T06:08:44.797 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_session": 0, 2024-09-21T06:08:44.797 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_peer_request": 0, 2024-09-21T06:08:44.797 INFO:teuthology.orchestra.run.smithi117.stdout: "req_create_latency": { 2024-09-21T06:08:44.797 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.797 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.797 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.798 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.798 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getattr_latency": { 2024-09-21T06:08:44.798 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.798 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.798 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.798 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.798 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getfilelock_latency": { 2024-09-21T06:08:44.798 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.798 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.798 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.798 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.798 INFO:teuthology.orchestra.run.smithi117.stdout: "req_link_latency": { 2024-09-21T06:08:44.798 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.798 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.798 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.799 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.799 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookup_latency": { 2024-09-21T06:08:44.799 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.799 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.799 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.799 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.799 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookuphash_latency": { 2024-09-21T06:08:44.799 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.799 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.799 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.799 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.799 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupino_latency": { 2024-09-21T06:08:44.799 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.799 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.799 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.800 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.800 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupname_latency": { 2024-09-21T06:08:44.800 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.800 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.800 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.800 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.800 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupparent_latency": { 2024-09-21T06:08:44.800 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.800 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.800 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.800 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.800 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupsnap_latency": { 2024-09-21T06:08:44.800 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.800 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.800 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.801 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.801 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lssnap_latency": { 2024-09-21T06:08:44.801 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.801 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.801 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.801 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.802 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mkdir_latency": { 2024-09-21T06:08:44.802 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.802 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.802 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.802 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.802 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mknod_latency": { 2024-09-21T06:08:44.802 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.802 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.802 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.802 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.802 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mksnap_latency": { 2024-09-21T06:08:44.802 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.802 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.802 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.803 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.803 INFO:teuthology.orchestra.run.smithi117.stdout: "req_open_latency": { 2024-09-21T06:08:44.803 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.803 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.803 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.803 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.803 INFO:teuthology.orchestra.run.smithi117.stdout: "req_readdir_latency": { 2024-09-21T06:08:44.803 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.803 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.803 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.803 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.803 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rename_latency": { 2024-09-21T06:08:44.803 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.804 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.804 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.804 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.804 INFO:teuthology.orchestra.run.smithi117.stdout: "req_renamesnap_latency": { 2024-09-21T06:08:44.804 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.804 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.804 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.804 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.804 INFO:teuthology.orchestra.run.smithi117.stdout: "req_snapdiff_latency": { 2024-09-21T06:08:44.804 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.804 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.804 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.804 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.804 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmdir_latency": { 2024-09-21T06:08:44.804 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.805 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.805 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.805 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.805 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmsnap_latency": { 2024-09-21T06:08:44.805 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.805 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.805 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.805 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.805 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmxattr_latency": { 2024-09-21T06:08:44.805 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.805 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.805 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.805 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.805 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setattr_latency": { 2024-09-21T06:08:44.805 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.806 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.806 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.806 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.806 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setdirlayout_latency": { 2024-09-21T06:08:44.806 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.806 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.806 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.806 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.806 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setfilelock_latency": { 2024-09-21T06:08:44.806 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.806 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.806 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.806 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.806 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setlayout_latency": { 2024-09-21T06:08:44.806 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.807 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.807 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.807 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.807 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setxattr_latency": { 2024-09-21T06:08:44.807 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.807 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.807 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.807 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.807 INFO:teuthology.orchestra.run.smithi117.stdout: "req_symlink_latency": { 2024-09-21T06:08:44.807 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.807 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.807 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.807 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.807 INFO:teuthology.orchestra.run.smithi117.stdout: "req_unlink_latency": { 2024-09-21T06:08:44.808 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.808 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.808 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.808 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.808 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_revoke_eviction": 0, 2024-09-21T06:08:44.808 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_acquisition_throttle": 0, 2024-09-21T06:08:44.808 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getvxattr_latency": { 2024-09-21T06:08:44.808 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.808 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.808 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.808 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:44.808 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.808 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_sessions": { 2024-09-21T06:08:44.808 INFO:teuthology.orchestra.run.smithi117.stdout: "session_count": 2, 2024-09-21T06:08:44.808 INFO:teuthology.orchestra.run.smithi117.stdout: "session_add": 3, 2024-09-21T06:08:44.809 INFO:teuthology.orchestra.run.smithi117.stdout: "session_remove": 1, 2024-09-21T06:08:44.809 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_open": 2, 2024-09-21T06:08:44.809 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_stale": 0, 2024-09-21T06:08:44.809 INFO:teuthology.orchestra.run.smithi117.stdout: "total_load": 0, 2024-09-21T06:08:44.809 INFO:teuthology.orchestra.run.smithi117.stdout: "average_load": 0, 2024-09-21T06:08:44.809 INFO:teuthology.orchestra.run.smithi117.stdout: "avg_session_uptime": 1789, 2024-09-21T06:08:44.809 INFO:teuthology.orchestra.run.smithi117.stdout: "mdthresh_evicted": 0 2024-09-21T06:08:44.809 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.809 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2024-09-21T06:08:44.809 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:08:44.809 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 0, 2024-09-21T06:08:44.809 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:08:44.809 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:08:44.809 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:08:44.809 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:08:44.810 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:08:44.810 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:08:44.810 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:08:44.810 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:08:44.810 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:08:44.810 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:08:44.810 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:08:44.810 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:08:44.810 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:08:44.810 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2024-09-21T06:08:44.810 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:08:44.810 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2024-09-21T06:08:44.810 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:08:44.810 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:08:44.811 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:08:44.811 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:08:44.811 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:08:44.811 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:08:44.811 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:08:44.811 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2024-09-21T06:08:44.811 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:08:44.811 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:08:44.811 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:08:44.811 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2024-09-21T06:08:44.811 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2024-09-21T06:08:44.811 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2024-09-21T06:08:44.811 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:08:44.811 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:08:44.812 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:08:44.812 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:08:44.812 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 189401, 2024-09-21T06:08:44.812 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 51, 2024-09-21T06:08:44.812 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 0, 2024-09-21T06:08:44.812 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 0, 2024-09-21T06:08:44.812 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2024-09-21T06:08:44.812 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2024-09-21T06:08:44.812 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:08:44.812 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2024-09-21T06:08:44.812 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:08:44.812 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2024-09-21T06:08:44.812 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 7968, 2024-09-21T06:08:44.812 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 176, 2024-09-21T06:08:44.812 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:08:44.813 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:08:44.813 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2024-09-21T06:08:44.813 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2024-09-21T06:08:44.813 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 8672, 2024-09-21T06:08:44.813 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 13, 2024-09-21T06:08:44.813 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2024-09-21T06:08:44.813 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2024-09-21T06:08:44.813 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2024-09-21T06:08:44.813 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2024-09-21T06:08:44.813 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.813 INFO:teuthology.orchestra.run.smithi117.stdout: "objecter": { 2024-09-21T06:08:44.813 INFO:teuthology.orchestra.run.smithi117.stdout: "op_active": 0, 2024-09-21T06:08:44.813 INFO:teuthology.orchestra.run.smithi117.stdout: "op_laggy": 0, 2024-09-21T06:08:44.813 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send": 2298, 2024-09-21T06:08:44.813 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send_bytes": 21, 2024-09-21T06:08:44.814 INFO:teuthology.orchestra.run.smithi117.stdout: "op_resend": 3, 2024-09-21T06:08:44.814 INFO:teuthology.orchestra.run.smithi117.stdout: "op_reply": 2298, 2024-09-21T06:08:44.814 INFO:teuthology.orchestra.run.smithi117.stdout: "op_latency": { 2024-09-21T06:08:44.814 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2298, 2024-09-21T06:08:44.814 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 4.159918172, 2024-09-21T06:08:44.814 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.001810234 2024-09-21T06:08:44.814 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.814 INFO:teuthology.orchestra.run.smithi117.stdout: "op_inflight": 0, 2024-09-21T06:08:44.814 INFO:teuthology.orchestra.run.smithi117.stdout: "oplen_avg": { 2024-09-21T06:08:44.814 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2298, 2024-09-21T06:08:44.814 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 2299 2024-09-21T06:08:44.814 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.814 INFO:teuthology.orchestra.run.smithi117.stdout: "op": 2298, 2024-09-21T06:08:44.814 INFO:teuthology.orchestra.run.smithi117.stdout: "op_r": 2298, 2024-09-21T06:08:44.814 INFO:teuthology.orchestra.run.smithi117.stdout: "op_w": 0, 2024-09-21T06:08:44.815 INFO:teuthology.orchestra.run.smithi117.stdout: "op_rmw": 0, 2024-09-21T06:08:44.815 INFO:teuthology.orchestra.run.smithi117.stdout: "op_pg": 0, 2024-09-21T06:08:44.815 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_stat": 1493, 2024-09-21T06:08:44.815 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_create": 0, 2024-09-21T06:08:44.815 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_read": 803, 2024-09-21T06:08:44.815 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_write": 0, 2024-09-21T06:08:44.815 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writefull": 0, 2024-09-21T06:08:44.815 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writesame": 0, 2024-09-21T06:08:44.815 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_append": 0, 2024-09-21T06:08:44.815 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_zero": 0, 2024-09-21T06:08:44.815 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_truncate": 0, 2024-09-21T06:08:44.815 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_delete": 0, 2024-09-21T06:08:44.815 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_mapext": 0, 2024-09-21T06:08:44.815 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_sparse_read": 0, 2024-09-21T06:08:44.815 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_clonerange": 0, 2024-09-21T06:08:44.816 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_getxattr": 1, 2024-09-21T06:08:44.816 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_setxattr": 0, 2024-09-21T06:08:44.816 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_cmpxattr": 0, 2024-09-21T06:08:44.816 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_rmxattr": 0, 2024-09-21T06:08:44.816 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_resetxattrs": 0, 2024-09-21T06:08:44.816 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_call": 0, 2024-09-21T06:08:44.816 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_watch": 0, 2024-09-21T06:08:44.816 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_notify": 0, 2024-09-21T06:08:44.816 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T06:08:44.816 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls": 0, 2024-09-21T06:08:44.816 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls_filter": 0, 2024-09-21T06:08:44.816 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_other": 0, 2024-09-21T06:08:44.816 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_active": 0, 2024-09-21T06:08:44.816 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_send": 0, 2024-09-21T06:08:44.816 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_resend": 0, 2024-09-21T06:08:44.817 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_ping": 0, 2024-09-21T06:08:44.817 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_active": 0, 2024-09-21T06:08:44.817 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_send": 0, 2024-09-21T06:08:44.817 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_resend": 0, 2024-09-21T06:08:44.817 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_active": 0, 2024-09-21T06:08:44.817 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_send": 0, 2024-09-21T06:08:44.817 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_resend": 0, 2024-09-21T06:08:44.817 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_active": 0, 2024-09-21T06:08:44.817 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_send": 0, 2024-09-21T06:08:44.817 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_resend": 0, 2024-09-21T06:08:44.817 INFO:teuthology.orchestra.run.smithi117.stdout: "command_active": 0, 2024-09-21T06:08:44.817 INFO:teuthology.orchestra.run.smithi117.stdout: "command_send": 0, 2024-09-21T06:08:44.817 INFO:teuthology.orchestra.run.smithi117.stdout: "command_resend": 0, 2024-09-21T06:08:44.817 INFO:teuthology.orchestra.run.smithi117.stdout: "map_epoch": 85, 2024-09-21T06:08:44.818 INFO:teuthology.orchestra.run.smithi117.stdout: "map_full": 0, 2024-09-21T06:08:44.818 INFO:teuthology.orchestra.run.smithi117.stdout: "map_inc": 11, 2024-09-21T06:08:44.818 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_sessions": 5, 2024-09-21T06:08:44.818 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_open": 5, 2024-09-21T06:08:44.818 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_close": 0, 2024-09-21T06:08:44.818 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_laggy": 0, 2024-09-21T06:08:44.818 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_wr": 0, 2024-09-21T06:08:44.818 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_rd": 2, 2024-09-21T06:08:44.818 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_del": 0 2024-09-21T06:08:44.818 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.818 INFO:teuthology.orchestra.run.smithi117.stdout: "oft": { 2024-09-21T06:08:44.818 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_objs": 0, 2024-09-21T06:08:44.818 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_kv_pairs": 0, 2024-09-21T06:08:44.818 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_updates": 0, 2024-09-21T06:08:44.818 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_removes": 0 2024-09-21T06:08:44.819 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.819 INFO:teuthology.orchestra.run.smithi117.stdout: "purge_queue": { 2024-09-21T06:08:44.819 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops": 0, 2024-09-21T06:08:44.819 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T06:08:44.819 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing": 0, 2024-09-21T06:08:44.819 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_high_water": 0, 2024-09-21T06:08:44.819 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed_ops": 0, 2024-09-21T06:08:44.819 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed": 0, 2024-09-21T06:08:44.819 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_item_in_journal": 0 2024-09-21T06:08:44.819 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.819 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:08:44.819 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-21T06:08:44.819 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-09-21T06:08:44.819 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-21T06:08:44.819 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 2543, 2024-09-21T06:08:44.820 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 1291756, 2024-09-21T06:08:44.820 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-21T06:08:44.820 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 2543, 2024-09-21T06:08:44.820 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-21T06:08:44.820 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-21T06:08:44.820 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 2543, 2024-09-21T06:08:44.820 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 1291756, 2024-09-21T06:08:44.820 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-21T06:08:44.820 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.820 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.820 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.820 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:44.820 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.820 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_bytes": { 2024-09-21T06:08:44.820 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-21T06:08:44.820 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-09-21T06:08:44.821 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-21T06:08:44.821 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-21T06:08:44.821 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-21T06:08:44.821 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-21T06:08:44.821 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-21T06:08:44.821 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 2298, 2024-09-21T06:08:44.821 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 1824, 2024-09-21T06:08:44.821 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 2298, 2024-09-21T06:08:44.821 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 1824, 2024-09-21T06:08:44.821 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-21T06:08:44.821 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.821 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.821 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.821 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:44.821 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.822 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_ops": { 2024-09-21T06:08:44.822 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-21T06:08:44.822 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 1024, 2024-09-21T06:08:44.822 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-21T06:08:44.822 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-21T06:08:44.822 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-21T06:08:44.822 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-21T06:08:44.822 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-21T06:08:44.822 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 2298, 2024-09-21T06:08:44.822 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 2298, 2024-09-21T06:08:44.822 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 2298, 2024-09-21T06:08:44.822 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 2298, 2024-09-21T06:08:44.822 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-21T06:08:44.822 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.822 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.823 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.823 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:44.823 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.823 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle": { 2024-09-21T06:08:44.823 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-21T06:08:44.823 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-09-21T06:08:44.823 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-21T06:08:44.823 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-21T06:08:44.823 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-21T06:08:44.823 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-21T06:08:44.823 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-21T06:08:44.823 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-21T06:08:44.823 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-21T06:08:44.823 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 0, 2024-09-21T06:08:44.823 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 0, 2024-09-21T06:08:44.824 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-21T06:08:44.824 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.824 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.824 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.824 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:44.824 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.824 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle-0x5567a58af1e0": { 2024-09-21T06:08:44.824 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-21T06:08:44.824 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-09-21T06:08:44.824 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-21T06:08:44.824 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-21T06:08:44.824 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-21T06:08:44.824 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-21T06:08:44.824 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-21T06:08:44.825 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-21T06:08:44.825 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-21T06:08:44.825 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 0, 2024-09-21T06:08:44.825 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 0, 2024-09-21T06:08:44.825 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-21T06:08:44.825 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:44.825 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:44.825 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:44.825 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:44.825 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:44.825 INFO:teuthology.orchestra.run.smithi117.stdout: "trackedop": { 2024-09-21T06:08:44.825 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_ops_count": 0 2024-09-21T06:08:44.825 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:44.825 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T06:08:45.630 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-21T06:08:45.631 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-21T06:08:45.631 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-09-21T06:08:45.631 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-09-21T06:08:45.631 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-09-21T06:08:45.631 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-09-21T06:08:45.631 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T06:08:45.631 DEBUG:tasks.check_counter:Getting stats from d 2024-09-21T06:08:45.631 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph daemon mds.d perf dump 2024-09-21T06:08:45.881 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:08:46.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:45 smithi192 ceph-mon[34700]: pgmap v679: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.7 KiB/s wr, 36 op/s 2024-09-21T06:08:46.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:45 smithi117 ceph-mon[30947]: pgmap v679: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.7 KiB/s wr, 36 op/s 2024-09-21T06:08:46.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:45 smithi138 ceph-mon[34391]: pgmap v679: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.7 KiB/s wr, 36 op/s 2024-09-21T06:08:46.405 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T06:08:46.405 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:08:46.406 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 0, 2024-09-21T06:08:46.406 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 2, 2024-09-21T06:08:46.406 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 0, 2024-09-21T06:08:46.406 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 0, 2024-09-21T06:08:46.406 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 4, 2024-09-21T06:08:46.406 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 0, 2024-09-21T06:08:46.406 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.001092091, 2024-09-21T06:08:46.406 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T06:08:46.406 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.000219050, 2024-09-21T06:08:46.406 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:08:46.406 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:08:46.406 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:46.406 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:46.407 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:46.407 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:46.407 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:08:46.407 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:46.407 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:46.407 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:46.407 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:46.407 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T06:08:46.407 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T06:08:46.407 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:46.407 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:08:46.407 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 232, 2024-09-21T06:08:46.407 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 227, 2024-09-21T06:08:46.407 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 840848, 2024-09-21T06:08:46.408 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 139584, 2024-09-21T06:08:46.408 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 2, 2024-09-21T06:08:46.408 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 1, 2024-09-21T06:08:46.408 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.046672253, 2024-09-21T06:08:46.408 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.022056245, 2024-09-21T06:08:46.408 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.033034211, 2024-09-21T06:08:46.408 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:08:46.408 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:08:46.408 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 226, 2024-09-21T06:08:46.408 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.017292521, 2024-09-21T06:08:46.408 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000076515 2024-09-21T06:08:46.408 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:46.408 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:08:46.408 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:46.408 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:46.409 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:46.409 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:46.409 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 840848, 2024-09-21T06:08:46.409 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 139584 2024-09-21T06:08:46.409 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:46.409 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:08:46.409 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 5, 2024-09-21T06:08:46.409 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 174, 2024-09-21T06:08:46.409 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 3152, 2024-09-21T06:08:46.409 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 110960, 2024-09-21T06:08:46.409 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 3, 2024-09-21T06:08:46.409 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 2, 2024-09-21T06:08:46.409 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.022201312, 2024-09-21T06:08:46.409 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.016259836, 2024-09-21T06:08:46.410 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.003306873, 2024-09-21T06:08:46.410 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:08:46.410 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:08:46.410 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 173, 2024-09-21T06:08:46.410 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.037374856, 2024-09-21T06:08:46.410 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000216039 2024-09-21T06:08:46.410 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:46.410 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:08:46.410 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:46.410 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:46.410 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:46.410 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:46.410 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-21T06:08:46.410 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 110960 2024-09-21T06:08:46.410 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:46.411 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2024-09-21T06:08:46.411 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 0, 2024-09-21T06:08:46.411 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2024-09-21T06:08:46.411 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:46.411 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2024-09-21T06:08:46.411 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:08:46.411 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 0, 2024-09-21T06:08:46.411 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:08:46.411 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:08:46.411 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:08:46.411 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:08:46.411 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:08:46.411 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:08:46.411 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:08:46.411 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:08:46.412 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:08:46.412 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:08:46.412 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:08:46.412 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:08:46.412 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:08:46.412 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2024-09-21T06:08:46.412 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:08:46.412 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2024-09-21T06:08:46.412 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:08:46.412 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:08:46.412 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:08:46.412 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:08:46.412 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:08:46.412 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:08:46.412 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:08:46.413 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2024-09-21T06:08:46.413 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:08:46.413 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:08:46.413 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:08:46.413 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2024-09-21T06:08:46.413 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2024-09-21T06:08:46.413 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2024-09-21T06:08:46.413 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:08:46.413 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:08:46.413 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:08:46.413 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:08:46.413 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 149007, 2024-09-21T06:08:46.413 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 30, 2024-09-21T06:08:46.413 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 0, 2024-09-21T06:08:46.413 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 0, 2024-09-21T06:08:46.414 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2024-09-21T06:08:46.414 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2024-09-21T06:08:46.414 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:08:46.414 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2024-09-21T06:08:46.414 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:08:46.414 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2024-09-21T06:08:46.414 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 0, 2024-09-21T06:08:46.414 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 0, 2024-09-21T06:08:46.414 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:08:46.414 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:08:46.414 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2024-09-21T06:08:46.414 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2024-09-21T06:08:46.414 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 968, 2024-09-21T06:08:46.414 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 2, 2024-09-21T06:08:46.414 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2024-09-21T06:08:46.415 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2024-09-21T06:08:46.415 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2024-09-21T06:08:46.415 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2024-09-21T06:08:46.415 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:46.415 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:08:46.415 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-21T06:08:46.415 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-09-21T06:08:46.415 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-21T06:08:46.415 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 233, 2024-09-21T06:08:46.415 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 808044, 2024-09-21T06:08:46.415 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-21T06:08:46.415 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 233, 2024-09-21T06:08:46.415 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-21T06:08:46.415 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-21T06:08:46.415 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 233, 2024-09-21T06:08:46.416 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 808044, 2024-09-21T06:08:46.416 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-21T06:08:46.416 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:46.416 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:46.416 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:46.416 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:46.416 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:46.416 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T06:08:47.240 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-21T06:08:47.240 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-21T06:08:47.240 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-21T06:08:47.240 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-21T06:08:47.240 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-21T06:08:47.240 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-21T06:08:47.240 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-21T06:08:47.240 DEBUG:tasks.check_counter:Getting stats from g 2024-09-21T06:08:47.241 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph daemon mds.g perf dump 2024-09-21T06:08:47.489 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:08:48.024 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T06:08:48.025 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:08:48.025 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 1044, 2024-09-21T06:08:48.025 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 1035, 2024-09-21T06:08:48.025 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 1149618, 2024-09-21T06:08:48.025 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 393078, 2024-09-21T06:08:48.025 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 5, 2024-09-21T06:08:48.025 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 1, 2024-09-21T06:08:48.025 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.181739768, 2024-09-21T06:08:48.025 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.081439765, 2024-09-21T06:08:48.025 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.064929552, 2024-09-21T06:08:48.025 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.034212681, 2024-09-21T06:08:48.025 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:08:48.026 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1034, 2024-09-21T06:08:48.026 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.131236675, 2024-09-21T06:08:48.026 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000126921 2024-09-21T06:08:48.026 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.026 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:08:48.026 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.026 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.026 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.026 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.026 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 881408, 2024-09-21T06:08:48.026 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 139632 2024-09-21T06:08:48.026 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.026 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:08:48.026 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 1503, 2024-09-21T06:08:48.027 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 1669, 2024-09-21T06:08:48.027 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 385763, 2024-09-21T06:08:48.027 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 860181, 2024-09-21T06:08:48.027 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 5, 2024-09-21T06:08:48.027 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 4, 2024-09-21T06:08:48.027 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.177495309, 2024-09-21T06:08:48.027 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.067423705, 2024-09-21T06:08:48.027 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.040594315, 2024-09-21T06:08:48.027 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.059980664, 2024-09-21T06:08:48.027 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:08:48.027 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1668, 2024-09-21T06:08:48.027 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.164367318, 2024-09-21T06:08:48.027 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000098541 2024-09-21T06:08:48.027 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.028 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:08:48.028 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.028 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.028 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.028 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.028 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-21T06:08:48.028 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 383760 2024-09-21T06:08:48.028 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.028 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:08:48.028 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 4, 2024-09-21T06:08:48.028 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 6, 2024-09-21T06:08:48.028 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 1629, 2024-09-21T06:08:48.028 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 1284, 2024-09-21T06:08:48.028 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 5, 2024-09-21T06:08:48.028 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 4, 2024-09-21T06:08:48.029 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.001474965, 2024-09-21T06:08:48.029 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.000130941, 2024-09-21T06:08:48.029 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.001107646, 2024-09-21T06:08:48.029 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000118209, 2024-09-21T06:08:48.029 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:08:48.029 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 4, 2024-09-21T06:08:48.029 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.002470776, 2024-09-21T06:08:48.029 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000617694 2024-09-21T06:08:48.029 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.029 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:08:48.029 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.029 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.029 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.029 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.030 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T06:08:48.030 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T06:08:48.030 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.030 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2024-09-21T06:08:48.030 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 1, 2024-09-21T06:08:48.030 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2024-09-21T06:08:48.030 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.030 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-MDSRank": { 2024-09-21T06:08:48.030 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-09-21T06:08:48.030 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-09-21T06:08:48.030 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 3795, 2024-09-21T06:08:48.030 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.136429659, 2024-09-21T06:08:48.030 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000035949 2024-09-21T06:08:48.031 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:48.031 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.031 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-PurgeQueue": { 2024-09-21T06:08:48.031 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-09-21T06:08:48.031 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-09-21T06:08:48.031 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.031 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.031 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.031 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:48.031 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.031 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-09-21T06:08:48.031 INFO:teuthology.orchestra.run.smithi117.stdout: "request": 0, 2024-09-21T06:08:48.031 INFO:teuthology.orchestra.run.smithi117.stdout: "reply": 0, 2024-09-21T06:08:48.031 INFO:teuthology.orchestra.run.smithi117.stdout: "reply_latency": { 2024-09-21T06:08:48.031 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.032 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.032 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.032 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.032 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_reply": 0, 2024-09-21T06:08:48.032 INFO:teuthology.orchestra.run.smithi117.stdout: "forward": 0, 2024-09-21T06:08:48.032 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_complete": 0, 2024-09-21T06:08:48.032 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_keys": 0, 2024-09-21T06:08:48.032 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_commit": 0, 2024-09-21T06:08:48.032 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_split": 0, 2024-09-21T06:08:48.032 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_merge": 0, 2024-09-21T06:08:48.032 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes": 0, 2024-09-21T06:08:48.032 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_top": 0, 2024-09-21T06:08:48.032 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_bottom": 0, 2024-09-21T06:08:48.032 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pin_tail": 0, 2024-09-21T06:08:48.032 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pinned": 0, 2024-09-21T06:08:48.033 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_expired": 0, 2024-09-21T06:08:48.033 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_with_caps": 0, 2024-09-21T06:08:48.033 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": 0, 2024-09-21T06:08:48.033 INFO:teuthology.orchestra.run.smithi117.stdout: "subtrees": 1, 2024-09-21T06:08:48.033 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse": 0, 2024-09-21T06:08:48.033 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_hit": 0, 2024-09-21T06:08:48.033 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_forward": 0, 2024-09-21T06:08:48.033 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_discover": 0, 2024-09-21T06:08:48.033 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_dir_fetch": 0, 2024-09-21T06:08:48.033 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_remote_ino": 0, 2024-09-21T06:08:48.033 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_lock": 0, 2024-09-21T06:08:48.033 INFO:teuthology.orchestra.run.smithi117.stdout: "load_cent": 0, 2024-09-21T06:08:48.033 INFO:teuthology.orchestra.run.smithi117.stdout: "q": 0, 2024-09-21T06:08:48.033 INFO:teuthology.orchestra.run.smithi117.stdout: "exported": 0, 2024-09-21T06:08:48.033 INFO:teuthology.orchestra.run.smithi117.stdout: "exported_inodes": 0, 2024-09-21T06:08:48.034 INFO:teuthology.orchestra.run.smithi117.stdout: "imported": 0, 2024-09-21T06:08:48.034 INFO:teuthology.orchestra.run.smithi117.stdout: "imported_inodes": 0, 2024-09-21T06:08:48.034 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_dir_fetch": 0, 2024-09-21T06:08:48.034 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_backtrace_fetch": 0, 2024-09-21T06:08:48.034 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_peer_discover": 0, 2024-09-21T06:08:48.034 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rfiles": 0, 2024-09-21T06:08:48.034 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rbytes": 0, 2024-09-21T06:08:48.034 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rsnaps": 0, 2024-09-21T06:08:48.034 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T06:08:48.034 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_set_tag": 0, 2024-09-21T06:08:48.034 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T06:08:48.034 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_inotable_repaired": 0, 2024-09-21T06:08:48.034 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_inodes": 0, 2024-09-21T06:08:48.034 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T06:08:48.034 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T06:08:48.035 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_file_inodes": 0, 2024-09-21T06:08:48.035 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_inode_file_caps": 0, 2024-09-21T06:08:48.035 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T06:08:48.035 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_grant": 0, 2024-09-21T06:08:48.035 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T06:08:48.035 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T06:08:48.035 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T06:08:48.035 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps": 0, 2024-09-21T06:08:48.035 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps_dirty": 0, 2024-09-21T06:08:48.035 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_cap_release": 0, 2024-09-21T06:08:48.035 INFO:teuthology.orchestra.run.smithi117.stdout: "process_request_cap_release": 0 2024-09-21T06:08:48.035 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.035 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_cache": { 2024-09-21T06:08:48.035 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update": 0, 2024-09-21T06:08:48.035 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update_receipt": 0, 2024-09-21T06:08:48.036 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_try_discover": 0, 2024-09-21T06:08:48.036 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_send_discover": 0, 2024-09-21T06:08:48.036 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_handle_discover": 0, 2024-09-21T06:08:48.036 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays": 0, 2024-09-21T06:08:48.036 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_delayed": 0, 2024-09-21T06:08:48.036 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_enqueuing": 0, 2024-09-21T06:08:48.036 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_created": 0, 2024-09-21T06:08:48.036 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_enqueued": 0, 2024-09-21T06:08:48.036 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_reintegrated": 0, 2024-09-21T06:08:48.036 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_migrated": 0, 2024-09-21T06:08:48.036 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_processing": 0, 2024-09-21T06:08:48.036 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_enqueued": 0, 2024-09-21T06:08:48.036 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_prioritized": 0, 2024-09-21T06:08:48.036 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_started": 0, 2024-09-21T06:08:48.036 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_completed": 0, 2024-09-21T06:08:48.037 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_path": 0, 2024-09-21T06:08:48.037 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_inode": 0, 2024-09-21T06:08:48.037 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T06:08:48.037 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_exportdir": 0, 2024-09-21T06:08:48.037 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_flush": 0, 2024-09-21T06:08:48.037 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragmentdir": 0, 2024-09-21T06:08:48.037 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragstats": 0, 2024-09-21T06:08:48.037 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_inodestats": 0, 2024-09-21T06:08:48.037 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-21T06:08:48.037 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_succeeded": 0, 2024-09-21T06:08:48.037 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_write_failed": 0 2024-09-21T06:08:48.037 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.037 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_log": { 2024-09-21T06:08:48.037 INFO:teuthology.orchestra.run.smithi117.stdout: "evlrg": 0, 2024-09-21T06:08:48.037 INFO:teuthology.orchestra.run.smithi117.stdout: "evadd": 0, 2024-09-21T06:08:48.038 INFO:teuthology.orchestra.run.smithi117.stdout: "evex": 0, 2024-09-21T06:08:48.038 INFO:teuthology.orchestra.run.smithi117.stdout: "evtrm": 0, 2024-09-21T06:08:48.038 INFO:teuthology.orchestra.run.smithi117.stdout: "ev": 4, 2024-09-21T06:08:48.038 INFO:teuthology.orchestra.run.smithi117.stdout: "evexg": 0, 2024-09-21T06:08:48.038 INFO:teuthology.orchestra.run.smithi117.stdout: "evexd": 0, 2024-09-21T06:08:48.038 INFO:teuthology.orchestra.run.smithi117.stdout: "segadd": 0, 2024-09-21T06:08:48.038 INFO:teuthology.orchestra.run.smithi117.stdout: "segex": 0, 2024-09-21T06:08:48.038 INFO:teuthology.orchestra.run.smithi117.stdout: "segtrm": 0, 2024-09-21T06:08:48.038 INFO:teuthology.orchestra.run.smithi117.stdout: "seg": 2, 2024-09-21T06:08:48.038 INFO:teuthology.orchestra.run.smithi117.stdout: "segmjr": 2, 2024-09-21T06:08:48.038 INFO:teuthology.orchestra.run.smithi117.stdout: "segexg": 0, 2024-09-21T06:08:48.038 INFO:teuthology.orchestra.run.smithi117.stdout: "segexd": 0, 2024-09-21T06:08:48.038 INFO:teuthology.orchestra.run.smithi117.stdout: "expos": 4194304, 2024-09-21T06:08:48.038 INFO:teuthology.orchestra.run.smithi117.stdout: "wrpos": 4195591, 2024-09-21T06:08:48.039 INFO:teuthology.orchestra.run.smithi117.stdout: "rdpos": 4195200, 2024-09-21T06:08:48.039 INFO:teuthology.orchestra.run.smithi117.stdout: "jlat": { 2024-09-21T06:08:48.039 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.039 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.039 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.039 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.039 INFO:teuthology.orchestra.run.smithi117.stdout: "replayed": 4 2024-09-21T06:08:48.039 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.039 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_mem": { 2024-09-21T06:08:48.039 INFO:teuthology.orchestra.run.smithi117.stdout: "ino": 3, 2024-09-21T06:08:48.039 INFO:teuthology.orchestra.run.smithi117.stdout: "ino+": 3, 2024-09-21T06:08:48.039 INFO:teuthology.orchestra.run.smithi117.stdout: "ino-": 0, 2024-09-21T06:08:48.039 INFO:teuthology.orchestra.run.smithi117.stdout: "dir": 1, 2024-09-21T06:08:48.039 INFO:teuthology.orchestra.run.smithi117.stdout: "dir+": 1, 2024-09-21T06:08:48.039 INFO:teuthology.orchestra.run.smithi117.stdout: "dir-": 0, 2024-09-21T06:08:48.040 INFO:teuthology.orchestra.run.smithi117.stdout: "dn": 0, 2024-09-21T06:08:48.040 INFO:teuthology.orchestra.run.smithi117.stdout: "dn+": 0, 2024-09-21T06:08:48.040 INFO:teuthology.orchestra.run.smithi117.stdout: "dn-": 0, 2024-09-21T06:08:48.040 INFO:teuthology.orchestra.run.smithi117.stdout: "cap": 0, 2024-09-21T06:08:48.040 INFO:teuthology.orchestra.run.smithi117.stdout: "cap+": 0, 2024-09-21T06:08:48.040 INFO:teuthology.orchestra.run.smithi117.stdout: "cap-": 0, 2024-09-21T06:08:48.040 INFO:teuthology.orchestra.run.smithi117.stdout: "rss": 117444, 2024-09-21T06:08:48.040 INFO:teuthology.orchestra.run.smithi117.stdout: "heap": 316428288 2024-09-21T06:08:48.040 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.040 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_server": { 2024-09-21T06:08:48.040 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_client_request": 0, 2024-09-21T06:08:48.040 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_server_request": 0, 2024-09-21T06:08:48.040 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_request": 0, 2024-09-21T06:08:48.040 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_session": 0, 2024-09-21T06:08:48.040 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_peer_request": 0, 2024-09-21T06:08:48.041 INFO:teuthology.orchestra.run.smithi117.stdout: "req_create_latency": { 2024-09-21T06:08:48.041 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.041 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.041 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.041 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.041 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getattr_latency": { 2024-09-21T06:08:48.041 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.041 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.041 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.041 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.041 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getfilelock_latency": { 2024-09-21T06:08:48.041 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.041 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.041 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.042 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.042 INFO:teuthology.orchestra.run.smithi117.stdout: "req_link_latency": { 2024-09-21T06:08:48.042 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.042 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.042 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.042 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.042 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookup_latency": { 2024-09-21T06:08:48.042 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.042 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.042 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.042 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.042 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookuphash_latency": { 2024-09-21T06:08:48.042 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.043 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.043 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.043 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.043 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupino_latency": { 2024-09-21T06:08:48.043 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.043 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.043 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.043 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.043 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupname_latency": { 2024-09-21T06:08:48.043 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.043 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.043 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.043 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.043 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupparent_latency": { 2024-09-21T06:08:48.043 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.043 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.044 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.044 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.044 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupsnap_latency": { 2024-09-21T06:08:48.044 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.044 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.044 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.044 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.044 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lssnap_latency": { 2024-09-21T06:08:48.044 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.044 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.045 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.045 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.045 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mkdir_latency": { 2024-09-21T06:08:48.045 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.045 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.045 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.045 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.045 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mknod_latency": { 2024-09-21T06:08:48.045 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.046 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.046 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.046 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.046 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mksnap_latency": { 2024-09-21T06:08:48.046 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.046 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.046 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.046 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.046 INFO:teuthology.orchestra.run.smithi117.stdout: "req_open_latency": { 2024-09-21T06:08:48.046 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.046 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.046 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.046 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.047 INFO:teuthology.orchestra.run.smithi117.stdout: "req_readdir_latency": { 2024-09-21T06:08:48.047 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.047 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.047 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.047 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.047 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rename_latency": { 2024-09-21T06:08:48.047 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.047 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.047 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.047 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.047 INFO:teuthology.orchestra.run.smithi117.stdout: "req_renamesnap_latency": { 2024-09-21T06:08:48.047 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.047 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.047 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.048 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.048 INFO:teuthology.orchestra.run.smithi117.stdout: "req_snapdiff_latency": { 2024-09-21T06:08:48.048 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.048 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.048 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.048 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.048 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmdir_latency": { 2024-09-21T06:08:48.048 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.048 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.048 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.048 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.048 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmsnap_latency": { 2024-09-21T06:08:48.048 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.048 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.049 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.049 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.049 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmxattr_latency": { 2024-09-21T06:08:48.049 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.049 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.049 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.049 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.049 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setattr_latency": { 2024-09-21T06:08:48.049 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.049 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.049 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.049 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.049 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setdirlayout_latency": { 2024-09-21T06:08:48.049 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.049 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.050 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.050 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.050 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setfilelock_latency": { 2024-09-21T06:08:48.050 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.050 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.050 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.050 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.050 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setlayout_latency": { 2024-09-21T06:08:48.050 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.050 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.050 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.050 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.050 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setxattr_latency": { 2024-09-21T06:08:48.050 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.050 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.050 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.051 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.051 INFO:teuthology.orchestra.run.smithi117.stdout: "req_symlink_latency": { 2024-09-21T06:08:48.051 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.051 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.051 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.051 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.051 INFO:teuthology.orchestra.run.smithi117.stdout: "req_unlink_latency": { 2024-09-21T06:08:48.051 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.051 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.051 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.051 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.051 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_revoke_eviction": 0, 2024-09-21T06:08:48.051 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_acquisition_throttle": 0, 2024-09-21T06:08:48.051 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getvxattr_latency": { 2024-09-21T06:08:48.051 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.052 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.052 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.052 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:48.052 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.052 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_sessions": { 2024-09-21T06:08:48.052 INFO:teuthology.orchestra.run.smithi117.stdout: "session_count": 2, 2024-09-21T06:08:48.052 INFO:teuthology.orchestra.run.smithi117.stdout: "session_add": 2, 2024-09-21T06:08:48.052 INFO:teuthology.orchestra.run.smithi117.stdout: "session_remove": 0, 2024-09-21T06:08:48.052 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_open": 2, 2024-09-21T06:08:48.052 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_stale": 0, 2024-09-21T06:08:48.052 INFO:teuthology.orchestra.run.smithi117.stdout: "total_load": 0, 2024-09-21T06:08:48.052 INFO:teuthology.orchestra.run.smithi117.stdout: "average_load": 0, 2024-09-21T06:08:48.052 INFO:teuthology.orchestra.run.smithi117.stdout: "avg_session_uptime": 1796, 2024-09-21T06:08:48.052 INFO:teuthology.orchestra.run.smithi117.stdout: "mdthresh_evicted": 0 2024-09-21T06:08:48.052 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.052 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2024-09-21T06:08:48.053 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:08:48.053 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 0, 2024-09-21T06:08:48.053 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:08:48.053 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:08:48.053 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:08:48.053 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:08:48.053 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:08:48.053 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:08:48.053 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:08:48.053 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:08:48.053 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:08:48.053 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:08:48.053 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:08:48.053 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:08:48.053 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:08:48.054 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2024-09-21T06:08:48.054 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:08:48.054 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2024-09-21T06:08:48.054 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:08:48.054 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:08:48.054 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:08:48.054 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:08:48.054 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:08:48.054 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:08:48.054 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:08:48.054 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2024-09-21T06:08:48.054 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:08:48.054 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:08:48.055 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:08:48.055 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2024-09-21T06:08:48.055 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2024-09-21T06:08:48.055 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2024-09-21T06:08:48.055 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:08:48.055 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:08:48.055 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:08:48.055 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:08:48.055 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 197463, 2024-09-21T06:08:48.055 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 54, 2024-09-21T06:08:48.055 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 0, 2024-09-21T06:08:48.055 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 0, 2024-09-21T06:08:48.055 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2024-09-21T06:08:48.055 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2024-09-21T06:08:48.055 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:08:48.056 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2024-09-21T06:08:48.056 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:08:48.056 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2024-09-21T06:08:48.056 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 7968, 2024-09-21T06:08:48.056 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 176, 2024-09-21T06:08:48.056 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:08:48.056 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:08:48.056 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2024-09-21T06:08:48.056 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2024-09-21T06:08:48.056 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 8672, 2024-09-21T06:08:48.056 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 13, 2024-09-21T06:08:48.056 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2024-09-21T06:08:48.056 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2024-09-21T06:08:48.056 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2024-09-21T06:08:48.057 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2024-09-21T06:08:48.057 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.057 INFO:teuthology.orchestra.run.smithi117.stdout: "objecter": { 2024-09-21T06:08:48.057 INFO:teuthology.orchestra.run.smithi117.stdout: "op_active": 0, 2024-09-21T06:08:48.057 INFO:teuthology.orchestra.run.smithi117.stdout: "op_laggy": 0, 2024-09-21T06:08:48.057 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send": 2299, 2024-09-21T06:08:48.057 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send_bytes": 21, 2024-09-21T06:08:48.057 INFO:teuthology.orchestra.run.smithi117.stdout: "op_resend": 3, 2024-09-21T06:08:48.057 INFO:teuthology.orchestra.run.smithi117.stdout: "op_reply": 2299, 2024-09-21T06:08:48.057 INFO:teuthology.orchestra.run.smithi117.stdout: "op_latency": { 2024-09-21T06:08:48.057 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2299, 2024-09-21T06:08:48.057 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 5.053900088, 2024-09-21T06:08:48.057 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.002198303 2024-09-21T06:08:48.057 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.058 INFO:teuthology.orchestra.run.smithi117.stdout: "op_inflight": 0, 2024-09-21T06:08:48.058 INFO:teuthology.orchestra.run.smithi117.stdout: "oplen_avg": { 2024-09-21T06:08:48.058 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2299, 2024-09-21T06:08:48.058 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 2300 2024-09-21T06:08:48.058 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.058 INFO:teuthology.orchestra.run.smithi117.stdout: "op": 2299, 2024-09-21T06:08:48.058 INFO:teuthology.orchestra.run.smithi117.stdout: "op_r": 2299, 2024-09-21T06:08:48.058 INFO:teuthology.orchestra.run.smithi117.stdout: "op_w": 0, 2024-09-21T06:08:48.058 INFO:teuthology.orchestra.run.smithi117.stdout: "op_rmw": 0, 2024-09-21T06:08:48.058 INFO:teuthology.orchestra.run.smithi117.stdout: "op_pg": 0, 2024-09-21T06:08:48.058 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_stat": 1494, 2024-09-21T06:08:48.058 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_create": 0, 2024-09-21T06:08:48.058 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_read": 803, 2024-09-21T06:08:48.058 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_write": 0, 2024-09-21T06:08:48.058 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writefull": 0, 2024-09-21T06:08:48.059 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writesame": 0, 2024-09-21T06:08:48.059 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_append": 0, 2024-09-21T06:08:48.059 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_zero": 0, 2024-09-21T06:08:48.059 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_truncate": 0, 2024-09-21T06:08:48.059 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_delete": 0, 2024-09-21T06:08:48.059 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_mapext": 0, 2024-09-21T06:08:48.059 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_sparse_read": 0, 2024-09-21T06:08:48.059 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_clonerange": 0, 2024-09-21T06:08:48.059 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_getxattr": 1, 2024-09-21T06:08:48.059 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_setxattr": 0, 2024-09-21T06:08:48.059 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_cmpxattr": 0, 2024-09-21T06:08:48.059 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_rmxattr": 0, 2024-09-21T06:08:48.059 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_resetxattrs": 0, 2024-09-21T06:08:48.059 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_call": 0, 2024-09-21T06:08:48.060 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_watch": 0, 2024-09-21T06:08:48.060 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_notify": 0, 2024-09-21T06:08:48.060 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T06:08:48.060 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls": 0, 2024-09-21T06:08:48.060 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls_filter": 0, 2024-09-21T06:08:48.060 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_other": 0, 2024-09-21T06:08:48.060 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_active": 0, 2024-09-21T06:08:48.060 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_send": 0, 2024-09-21T06:08:48.060 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_resend": 0, 2024-09-21T06:08:48.060 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_ping": 0, 2024-09-21T06:08:48.060 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_active": 0, 2024-09-21T06:08:48.060 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_send": 0, 2024-09-21T06:08:48.060 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_resend": 0, 2024-09-21T06:08:48.060 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_active": 0, 2024-09-21T06:08:48.060 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_send": 0, 2024-09-21T06:08:48.061 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_resend": 0, 2024-09-21T06:08:48.061 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_active": 0, 2024-09-21T06:08:48.061 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_send": 0, 2024-09-21T06:08:48.061 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_resend": 0, 2024-09-21T06:08:48.061 INFO:teuthology.orchestra.run.smithi117.stdout: "command_active": 0, 2024-09-21T06:08:48.061 INFO:teuthology.orchestra.run.smithi117.stdout: "command_send": 0, 2024-09-21T06:08:48.061 INFO:teuthology.orchestra.run.smithi117.stdout: "command_resend": 0, 2024-09-21T06:08:48.061 INFO:teuthology.orchestra.run.smithi117.stdout: "map_epoch": 85, 2024-09-21T06:08:48.061 INFO:teuthology.orchestra.run.smithi117.stdout: "map_full": 0, 2024-09-21T06:08:48.061 INFO:teuthology.orchestra.run.smithi117.stdout: "map_inc": 11, 2024-09-21T06:08:48.061 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_sessions": 6, 2024-09-21T06:08:48.061 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_open": 6, 2024-09-21T06:08:48.061 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_close": 0, 2024-09-21T06:08:48.061 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_laggy": 0, 2024-09-21T06:08:48.062 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_wr": 0, 2024-09-21T06:08:48.062 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_rd": 2, 2024-09-21T06:08:48.062 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_del": 0 2024-09-21T06:08:48.062 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.062 INFO:teuthology.orchestra.run.smithi117.stdout: "oft": { 2024-09-21T06:08:48.062 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_objs": 0, 2024-09-21T06:08:48.062 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_kv_pairs": 0, 2024-09-21T06:08:48.062 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_updates": 0, 2024-09-21T06:08:48.062 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_removes": 0 2024-09-21T06:08:48.062 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.062 INFO:teuthology.orchestra.run.smithi117.stdout: "purge_queue": { 2024-09-21T06:08:48.062 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops": 0, 2024-09-21T06:08:48.062 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T06:08:48.062 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing": 0, 2024-09-21T06:08:48.063 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_high_water": 0, 2024-09-21T06:08:48.063 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed_ops": 0, 2024-09-21T06:08:48.063 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed": 0, 2024-09-21T06:08:48.063 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_item_in_journal": 0 2024-09-21T06:08:48.063 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.063 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:08:48.063 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-21T06:08:48.063 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-09-21T06:08:48.063 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-21T06:08:48.063 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 2547, 2024-09-21T06:08:48.063 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 1292180, 2024-09-21T06:08:48.063 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-21T06:08:48.063 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 2547, 2024-09-21T06:08:48.063 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-21T06:08:48.064 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-21T06:08:48.064 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 2547, 2024-09-21T06:08:48.064 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 1292180, 2024-09-21T06:08:48.064 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-21T06:08:48.064 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.064 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.064 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.064 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:48.064 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.064 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_bytes": { 2024-09-21T06:08:48.064 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-21T06:08:48.064 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-09-21T06:08:48.064 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-21T06:08:48.065 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-21T06:08:48.065 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-21T06:08:48.065 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-21T06:08:48.065 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-21T06:08:48.065 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 2299, 2024-09-21T06:08:48.065 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 1292, 2024-09-21T06:08:48.065 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 2299, 2024-09-21T06:08:48.065 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 1292, 2024-09-21T06:08:48.065 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-21T06:08:48.065 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.065 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.065 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.065 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:48.065 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.065 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_ops": { 2024-09-21T06:08:48.065 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-21T06:08:48.066 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 1024, 2024-09-21T06:08:48.066 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-21T06:08:48.066 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-21T06:08:48.066 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-21T06:08:48.066 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-21T06:08:48.066 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-21T06:08:48.066 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 2299, 2024-09-21T06:08:48.066 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 2299, 2024-09-21T06:08:48.066 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 2299, 2024-09-21T06:08:48.066 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 2299, 2024-09-21T06:08:48.066 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-21T06:08:48.066 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.066 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.066 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.066 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:48.067 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.067 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle": { 2024-09-21T06:08:48.067 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-21T06:08:48.067 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-09-21T06:08:48.067 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-21T06:08:48.067 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-21T06:08:48.067 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-21T06:08:48.067 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-21T06:08:48.067 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-21T06:08:48.067 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-21T06:08:48.067 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-21T06:08:48.067 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 0, 2024-09-21T06:08:48.067 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 0, 2024-09-21T06:08:48.067 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-21T06:08:48.067 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.067 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.068 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.068 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:48.068 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.068 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle-0x55f850ead450": { 2024-09-21T06:08:48.068 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-21T06:08:48.068 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-09-21T06:08:48.068 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-21T06:08:48.068 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-21T06:08:48.068 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-21T06:08:48.068 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-21T06:08:48.068 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-21T06:08:48.068 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-21T06:08:48.068 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-21T06:08:48.069 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 0, 2024-09-21T06:08:48.069 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 0, 2024-09-21T06:08:48.069 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-21T06:08:48.069 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:48.069 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:48.069 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:48.069 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:48.069 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:48.069 INFO:teuthology.orchestra.run.smithi117.stdout: "trackedop": { 2024-09-21T06:08:48.069 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_ops_count": 0 2024-09-21T06:08:48.069 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:48.069 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T06:08:48.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:47 smithi192 ceph-mon[34700]: pgmap v680: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:08:48.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:47 smithi117 ceph-mon[30947]: pgmap v680: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:08:48.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:47 smithi138 ceph-mon[34391]: pgmap v680: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:08:48.835 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-21T06:08:48.836 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-21T06:08:48.836 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-09-21T06:08:48.836 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-09-21T06:08:48.836 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-09-21T06:08:48.836 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-09-21T06:08:48.836 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T06:08:48.836 DEBUG:tasks.check_counter:Getting stats from j 2024-09-21T06:08:48.836 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph daemon mds.j perf dump 2024-09-21T06:08:49.088 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:08:50.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:49 smithi192 ceph-mon[34700]: pgmap v681: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:08:50.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:49 smithi117 ceph-mon[30947]: pgmap v681: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:08:50.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:49 smithi138 ceph-mon[34391]: pgmap v681: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:08:52.101 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-21T06:08:52.101 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:08:52.102 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 0, 2024-09-21T06:08:52.102 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 2, 2024-09-21T06:08:52.102 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 0, 2024-09-21T06:08:52.102 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 0, 2024-09-21T06:08:52.102 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 4, 2024-09-21T06:08:52.102 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 0, 2024-09-21T06:08:52.102 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.001096364, 2024-09-21T06:08:52.102 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T06:08:52.102 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.000268357, 2024-09-21T06:08:52.102 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:08:52.102 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:08:52.102 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:52.102 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:52.103 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:52.103 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:52.103 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:08:52.103 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:52.103 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:52.103 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:52.103 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:52.103 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T06:08:52.103 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T06:08:52.103 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:52.103 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:08:52.103 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 1, 2024-09-21T06:08:52.103 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 174, 2024-09-21T06:08:52.103 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 160, 2024-09-21T06:08:52.103 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 110864, 2024-09-21T06:08:52.104 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 3, 2024-09-21T06:08:52.104 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 2, 2024-09-21T06:08:52.104 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.017567322, 2024-09-21T06:08:52.104 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.014234707, 2024-09-21T06:08:52.104 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.001741407, 2024-09-21T06:08:52.104 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:08:52.104 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:08:52.104 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 172, 2024-09-21T06:08:52.104 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.012909352, 2024-09-21T06:08:52.104 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000075054 2024-09-21T06:08:52.104 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:52.104 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:08:52.104 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:52.105 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:52.105 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:52.105 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:52.105 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T06:08:52.105 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 110864 2024-09-21T06:08:52.105 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:52.105 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:08:52.105 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 238, 2024-09-21T06:08:52.105 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 231, 2024-09-21T06:08:52.105 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 844384, 2024-09-21T06:08:52.105 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 141312, 2024-09-21T06:08:52.105 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 2, 2024-09-21T06:08:52.105 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 1, 2024-09-21T06:08:52.105 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.054170297, 2024-09-21T06:08:52.106 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.021984617, 2024-09-21T06:08:52.106 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.039921878, 2024-09-21T06:08:52.106 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T06:08:52.106 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:08:52.106 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 231, 2024-09-21T06:08:52.106 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.030028229, 2024-09-21T06:08:52.106 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000129992 2024-09-21T06:08:52.106 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:52.106 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:08:52.106 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:52.106 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:52.106 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:52.106 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:52.106 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 844384, 2024-09-21T06:08:52.107 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 141312 2024-09-21T06:08:52.107 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:52.107 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2024-09-21T06:08:52.107 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 0, 2024-09-21T06:08:52.107 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2024-09-21T06:08:52.107 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:52.107 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2024-09-21T06:08:52.107 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:08:52.107 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 0, 2024-09-21T06:08:52.108 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:08:52.108 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:08:52.108 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:08:52.108 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:08:52.108 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:08:52.108 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:08:52.108 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:08:52.108 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:08:52.108 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:08:52.108 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:08:52.108 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:08:52.108 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:08:52.108 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:08:52.109 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2024-09-21T06:08:52.109 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:08:52.109 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2024-09-21T06:08:52.109 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:08:52.109 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:08:52.109 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:08:52.109 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:08:52.109 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:08:52.109 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:08:52.109 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:08:52.109 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2024-09-21T06:08:52.109 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:08:52.109 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:08:52.110 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:08:52.110 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2024-09-21T06:08:52.110 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2024-09-21T06:08:52.110 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2024-09-21T06:08:52.110 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:08:52.110 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:08:52.110 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:08:52.110 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:08:52.110 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 149047, 2024-09-21T06:08:52.110 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 31, 2024-09-21T06:08:52.110 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 0, 2024-09-21T06:08:52.110 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 0, 2024-09-21T06:08:52.110 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2024-09-21T06:08:52.111 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2024-09-21T06:08:52.111 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:08:52.111 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2024-09-21T06:08:52.111 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:08:52.111 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2024-09-21T06:08:52.111 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 0, 2024-09-21T06:08:52.111 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 0, 2024-09-21T06:08:52.111 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:08:52.111 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:08:52.111 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2024-09-21T06:08:52.111 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2024-09-21T06:08:52.111 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 968, 2024-09-21T06:08:52.111 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 2, 2024-09-21T06:08:52.111 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2024-09-21T06:08:52.111 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2024-09-21T06:08:52.112 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2024-09-21T06:08:52.112 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2024-09-21T06:08:52.112 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-21T06:08:52.112 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:08:52.112 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-21T06:08:52.112 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-09-21T06:08:52.112 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-21T06:08:52.112 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 235, 2024-09-21T06:08:52.112 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 808304, 2024-09-21T06:08:52.112 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-21T06:08:52.112 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 235, 2024-09-21T06:08:52.112 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-21T06:08:52.112 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-21T06:08:52.112 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 235, 2024-09-21T06:08:52.113 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 808304, 2024-09-21T06:08:52.113 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-21T06:08:52.113 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-21T06:08:52.113 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-21T06:08:52.113 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-21T06:08:52.113 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:52.113 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-21T06:08:52.113 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-21T06:08:52.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:51 smithi192 ceph-mon[34700]: pgmap v682: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:08:52.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:51 smithi117 ceph-mon[30947]: pgmap v682: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:08:52.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:51 smithi138 ceph-mon[34391]: pgmap v682: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:08:54.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:53 smithi192 ceph-mon[34700]: pgmap v683: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:08:54.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:53 smithi117 ceph-mon[30947]: pgmap v683: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:08:54.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:53 smithi138 ceph-mon[34391]: pgmap v683: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:08:54.594 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-21T06:08:54.594 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-21T06:08:54.594 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-21T06:08:54.594 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-21T06:08:54.594 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-21T06:08:54.594 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-21T06:08:54.594 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-21T06:08:54.594 DEBUG:tasks.check_counter:Getting stats from b 2024-09-21T06:08:54.595 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph daemon mds.b perf dump 2024-09-21T06:08:54.918 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:08:55.556 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-21T06:08:55.556 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:08:55.556 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 7901, 2024-09-21T06:08:55.556 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 8858, 2024-09-21T06:08:55.556 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 2959111, 2024-09-21T06:08:55.556 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 186036513, 2024-09-21T06:08:55.556 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 173, 2024-09-21T06:08:55.556 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 7, 2024-09-21T06:08:55.556 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 2.364751247, 2024-09-21T06:08:55.556 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 1.426860933, 2024-09-21T06:08:55.556 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.356834913, 2024-09-21T06:08:55.556 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.569777607, 2024-09-21T06:08:55.556 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:08:55.557 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 8863, 2024-09-21T06:08:55.557 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 1.950547524, 2024-09-21T06:08:55.557 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000220077 2024-09-21T06:08:55.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.557 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:08:55.557 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 4311, 2024-09-21T06:08:55.557 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.001092408, 2024-09-21T06:08:55.557 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000253 2024-09-21T06:08:55.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.557 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 922640, 2024-09-21T06:08:55.557 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 162848 2024-09-21T06:08:55.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.557 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:08:55.557 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 6206, 2024-09-21T06:08:55.557 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 6859, 2024-09-21T06:08:55.558 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 1707927, 2024-09-21T06:08:55.558 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 277871385, 2024-09-21T06:08:55.558 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 23, 2024-09-21T06:08:55.558 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 6, 2024-09-21T06:08:55.558 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 2.250056886, 2024-09-21T06:08:55.558 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 1.498623891, 2024-09-21T06:08:55.558 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.180393140, 2024-09-21T06:08:55.558 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.556076645, 2024-09-21T06:08:55.558 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:08:55.558 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 6860, 2024-09-21T06:08:55.558 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 2.298724671, 2024-09-21T06:08:55.558 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000335091 2024-09-21T06:08:55.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.559 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:08:55.559 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2058, 2024-09-21T06:08:55.559 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000624441, 2024-09-21T06:08:55.559 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000303 2024-09-21T06:08:55.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.559 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T06:08:55.559 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 393376 2024-09-21T06:08:55.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.559 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:08:55.559 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 348910, 2024-09-21T06:08:55.559 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 351493, 2024-09-21T06:08:55.559 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 121600134, 2024-09-21T06:08:55.559 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 514745439, 2024-09-21T06:08:55.560 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 28, 2024-09-21T06:08:55.560 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 10, 2024-09-21T06:08:55.560 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 29.155008218, 2024-09-21T06:08:55.560 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 15.542683759, 2024-09-21T06:08:55.560 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 27.448092712, 2024-09-21T06:08:55.560 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 1.629236681, 2024-09-21T06:08:55.560 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:08:55.560 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 351483, 2024-09-21T06:08:55.560 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 79.390953497, 2024-09-21T06:08:55.560 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000225874 2024-09-21T06:08:55.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.561 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:08:55.561 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 570280, 2024-09-21T06:08:55.561 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.193286022, 2024-09-21T06:08:55.561 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000338 2024-09-21T06:08:55.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.561 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 405808, 2024-09-21T06:08:55.561 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 130784 2024-09-21T06:08:55.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.561 INFO:teuthology.orchestra.run.smithi138.stdout: "cct": { 2024-09-21T06:08:55.561 INFO:teuthology.orchestra.run.smithi138.stdout: "total_workers": 1, 2024-09-21T06:08:55.561 INFO:teuthology.orchestra.run.smithi138.stdout: "unhealthy_workers": 0 2024-09-21T06:08:55.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.562 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-MDSRank": { 2024-09-21T06:08:55.562 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-09-21T06:08:55.562 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-09-21T06:08:55.562 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 3016, 2024-09-21T06:08:55.562 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 304.765198099, 2024-09-21T06:08:55.562 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.101049468 2024-09-21T06:08:55.562 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:08:55.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.562 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-PurgeQueue": { 2024-09-21T06:08:55.562 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-09-21T06:08:55.562 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-09-21T06:08:55.562 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2473, 2024-09-21T06:08:55.562 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.082369994, 2024-09-21T06:08:55.562 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000033307 2024-09-21T06:08:55.563 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:08:55.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.563 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": { 2024-09-21T06:08:55.563 INFO:teuthology.orchestra.run.smithi138.stdout: "request": 239, 2024-09-21T06:08:55.563 INFO:teuthology.orchestra.run.smithi138.stdout: "reply": 239, 2024-09-21T06:08:55.563 INFO:teuthology.orchestra.run.smithi138.stdout: "reply_latency": { 2024-09-21T06:08:55.563 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 239, 2024-09-21T06:08:55.563 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 56.028109860, 2024-09-21T06:08:55.563 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.234427237 2024-09-21T06:08:55.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.563 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_reply": 0, 2024-09-21T06:08:55.563 INFO:teuthology.orchestra.run.smithi138.stdout: "forward": 0, 2024-09-21T06:08:55.564 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_complete": 0, 2024-09-21T06:08:55.564 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_keys": 0, 2024-09-21T06:08:55.564 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_commit": 233, 2024-09-21T06:08:55.564 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_split": 0, 2024-09-21T06:08:55.564 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_merge": 0, 2024-09-21T06:08:55.564 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes": 126, 2024-09-21T06:08:55.564 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_top": 44, 2024-09-21T06:08:55.565 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_bottom": 82, 2024-09-21T06:08:55.565 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pin_tail": 0, 2024-09-21T06:08:55.565 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pinned": 63, 2024-09-21T06:08:55.565 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_expired": 0, 2024-09-21T06:08:55.565 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_with_caps": 2, 2024-09-21T06:08:55.565 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": 2, 2024-09-21T06:08:55.566 INFO:teuthology.orchestra.run.smithi138.stdout: "subtrees": 2, 2024-09-21T06:08:55.566 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse": 575, 2024-09-21T06:08:55.566 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_hit": 562, 2024-09-21T06:08:55.566 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_forward": 0, 2024-09-21T06:08:55.566 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_discover": 0, 2024-09-21T06:08:55.566 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_dir_fetch": 0, 2024-09-21T06:08:55.566 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_remote_ino": 0, 2024-09-21T06:08:55.566 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_lock": 0, 2024-09-21T06:08:55.567 INFO:teuthology.orchestra.run.smithi138.stdout: "load_cent": 12, 2024-09-21T06:08:55.567 INFO:teuthology.orchestra.run.smithi138.stdout: "q": 0, 2024-09-21T06:08:55.567 INFO:teuthology.orchestra.run.smithi138.stdout: "exported": 0, 2024-09-21T06:08:55.567 INFO:teuthology.orchestra.run.smithi138.stdout: "exported_inodes": 0, 2024-09-21T06:08:55.567 INFO:teuthology.orchestra.run.smithi138.stdout: "imported": 0, 2024-09-21T06:08:55.567 INFO:teuthology.orchestra.run.smithi138.stdout: "imported_inodes": 0, 2024-09-21T06:08:55.567 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_dir_fetch": 0, 2024-09-21T06:08:55.568 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_backtrace_fetch": 386, 2024-09-21T06:08:55.568 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_peer_discover": 0, 2024-09-21T06:08:55.568 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rfiles": 4, 2024-09-21T06:08:55.568 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rbytes": 234, 2024-09-21T06:08:55.568 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rsnaps": 7, 2024-09-21T06:08:55.568 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_fetch": 386, 2024-09-21T06:08:55.568 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_set_tag": 0, 2024-09-21T06:08:55.569 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T06:08:55.569 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_inotable_repaired": 0, 2024-09-21T06:08:55.569 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_inodes": 136, 2024-09-21T06:08:55.569 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_base_inodes": 13, 2024-09-21T06:08:55.569 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dirfrag_rstats": 123, 2024-09-21T06:08:55.569 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_file_inodes": 250, 2024-09-21T06:08:55.569 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_inode_file_caps": 0, 2024-09-21T06:08:55.569 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_revoke": 166, 2024-09-21T06:08:55.570 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_grant": 3413, 2024-09-21T06:08:55.570 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T06:08:55.570 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T06:08:55.570 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T06:08:55.570 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps": 338379, 2024-09-21T06:08:55.570 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps_dirty": 338229, 2024-09-21T06:08:55.570 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_cap_release": 1, 2024-09-21T06:08:55.571 INFO:teuthology.orchestra.run.smithi138.stdout: "process_request_cap_release": 48 2024-09-21T06:08:55.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.571 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cache": { 2024-09-21T06:08:55.571 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update": 19, 2024-09-21T06:08:55.571 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update_receipt": 0, 2024-09-21T06:08:55.571 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_try_discover": 0, 2024-09-21T06:08:55.571 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_send_discover": 0, 2024-09-21T06:08:55.571 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_handle_discover": 40, 2024-09-21T06:08:55.572 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays": 4, 2024-09-21T06:08:55.572 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_delayed": 0, 2024-09-21T06:08:55.572 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_enqueuing": 0, 2024-09-21T06:08:55.572 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_created": 22, 2024-09-21T06:08:55.572 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_enqueued": 18, 2024-09-21T06:08:55.572 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_reintegrated": 0, 2024-09-21T06:08:55.572 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_migrated": 0, 2024-09-21T06:08:55.572 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_processing": 0, 2024-09-21T06:08:55.573 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_enqueued": 0, 2024-09-21T06:08:55.573 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_prioritized": 0, 2024-09-21T06:08:55.573 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_started": 0, 2024-09-21T06:08:55.573 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_completed": 0, 2024-09-21T06:08:55.573 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_path": 0, 2024-09-21T06:08:55.573 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_inode": 0, 2024-09-21T06:08:55.573 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_enqueue_scrub": 13, 2024-09-21T06:08:55.574 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_exportdir": 0, 2024-09-21T06:08:55.574 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_flush": 0, 2024-09-21T06:08:55.574 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragmentdir": 0, 2024-09-21T06:08:55.574 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragstats": 0, 2024-09-21T06:08:55.574 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_inodestats": 0, 2024-09-21T06:08:55.574 INFO:teuthology.orchestra.run.smithi138.stdout: "uninline_started": 0, 2024-09-21T06:08:55.574 INFO:teuthology.orchestra.run.smithi138.stdout: "uninline_succeeded": 0, 2024-09-21T06:08:55.575 INFO:teuthology.orchestra.run.smithi138.stdout: "uninline_write_failed": 0 2024-09-21T06:08:55.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.575 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_log": { 2024-09-21T06:08:55.575 INFO:teuthology.orchestra.run.smithi138.stdout: "evlrg": 0, 2024-09-21T06:08:55.575 INFO:teuthology.orchestra.run.smithi138.stdout: "evadd": 339264, 2024-09-21T06:08:55.575 INFO:teuthology.orchestra.run.smithi138.stdout: "evex": 339206, 2024-09-21T06:08:55.576 INFO:teuthology.orchestra.run.smithi138.stdout: "evtrm": 339206, 2024-09-21T06:08:55.576 INFO:teuthology.orchestra.run.smithi138.stdout: "ev": 58, 2024-09-21T06:08:55.576 INFO:teuthology.orchestra.run.smithi138.stdout: "evexg": 0, 2024-09-21T06:08:55.576 INFO:teuthology.orchestra.run.smithi138.stdout: "evexd": 2, 2024-09-21T06:08:55.576 INFO:teuthology.orchestra.run.smithi138.stdout: "segadd": 496, 2024-09-21T06:08:55.576 INFO:teuthology.orchestra.run.smithi138.stdout: "segex": 495, 2024-09-21T06:08:55.576 INFO:teuthology.orchestra.run.smithi138.stdout: "segtrm": 495, 2024-09-21T06:08:55.577 INFO:teuthology.orchestra.run.smithi138.stdout: "seg": 1, 2024-09-21T06:08:55.577 INFO:teuthology.orchestra.run.smithi138.stdout: "segmjr": 1, 2024-09-21T06:08:55.577 INFO:teuthology.orchestra.run.smithi138.stdout: "segexg": 0, 2024-09-21T06:08:55.577 INFO:teuthology.orchestra.run.smithi138.stdout: "segexd": 1, 2024-09-21T06:08:55.577 INFO:teuthology.orchestra.run.smithi138.stdout: "expos": 1079549107, 2024-09-21T06:08:55.577 INFO:teuthology.orchestra.run.smithi138.stdout: "wrpos": 1079834100, 2024-09-21T06:08:55.577 INFO:teuthology.orchestra.run.smithi138.stdout: "rdpos": 0, 2024-09-21T06:08:55.577 INFO:teuthology.orchestra.run.smithi138.stdout: "jlat": { 2024-09-21T06:08:55.577 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 13965, 2024-09-21T06:08:55.577 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 5402.681926822, 2024-09-21T06:08:55.578 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.386873034 2024-09-21T06:08:55.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.578 INFO:teuthology.orchestra.run.smithi138.stdout: "replayed": 0 2024-09-21T06:08:55.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.578 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_mem": { 2024-09-21T06:08:55.578 INFO:teuthology.orchestra.run.smithi138.stdout: "ino": 100, 2024-09-21T06:08:55.578 INFO:teuthology.orchestra.run.smithi138.stdout: "ino+": 131, 2024-09-21T06:08:55.578 INFO:teuthology.orchestra.run.smithi138.stdout: "ino-": 31, 2024-09-21T06:08:55.579 INFO:teuthology.orchestra.run.smithi138.stdout: "dir": 22, 2024-09-21T06:08:55.579 INFO:teuthology.orchestra.run.smithi138.stdout: "dir+": 22, 2024-09-21T06:08:55.579 INFO:teuthology.orchestra.run.smithi138.stdout: "dir-": 0, 2024-09-21T06:08:55.579 INFO:teuthology.orchestra.run.smithi138.stdout: "dn": 126, 2024-09-21T06:08:55.579 INFO:teuthology.orchestra.run.smithi138.stdout: "dn+": 144, 2024-09-21T06:08:55.579 INFO:teuthology.orchestra.run.smithi138.stdout: "dn-": 18, 2024-09-21T06:08:55.579 INFO:teuthology.orchestra.run.smithi138.stdout: "cap": 2, 2024-09-21T06:08:55.579 INFO:teuthology.orchestra.run.smithi138.stdout: "cap+": 35, 2024-09-21T06:08:55.580 INFO:teuthology.orchestra.run.smithi138.stdout: "cap-": 33, 2024-09-21T06:08:55.580 INFO:teuthology.orchestra.run.smithi138.stdout: "rss": 149936, 2024-09-21T06:08:55.580 INFO:teuthology.orchestra.run.smithi138.stdout: "heap": 384708608 2024-09-21T06:08:55.580 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.580 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_server": { 2024-09-21T06:08:55.580 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_client_request": 351, 2024-09-21T06:08:55.580 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_server_request": 0, 2024-09-21T06:08:55.580 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_request": 239, 2024-09-21T06:08:55.580 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_session": 25, 2024-09-21T06:08:55.580 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_peer_request": 0, 2024-09-21T06:08:55.580 INFO:teuthology.orchestra.run.smithi138.stdout: "req_create_latency": { 2024-09-21T06:08:55.580 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 22, 2024-09-21T06:08:55.580 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.026236829, 2024-09-21T06:08:55.580 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.001192583 2024-09-21T06:08:55.580 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.581 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getattr_latency": { 2024-09-21T06:08:55.581 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 125, 2024-09-21T06:08:55.581 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 39.794418570, 2024-09-21T06:08:55.581 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.318355348 2024-09-21T06:08:55.581 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.581 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getfilelock_latency": { 2024-09-21T06:08:55.581 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:55.581 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:55.581 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:55.581 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.581 INFO:teuthology.orchestra.run.smithi138.stdout: "req_link_latency": { 2024-09-21T06:08:55.581 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:55.581 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:55.581 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:55.581 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.582 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookup_latency": { 2024-09-21T06:08:55.582 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 16, 2024-09-21T06:08:55.582 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.004692107, 2024-09-21T06:08:55.582 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000293256 2024-09-21T06:08:55.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.582 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookuphash_latency": { 2024-09-21T06:08:55.582 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:55.582 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:55.582 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:55.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.582 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupino_latency": { 2024-09-21T06:08:55.582 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:55.582 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:55.582 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:55.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.583 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupname_latency": { 2024-09-21T06:08:55.583 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:55.583 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:55.583 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:55.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.583 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupparent_latency": { 2024-09-21T06:08:55.583 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:55.583 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:55.583 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:55.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.583 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupsnap_latency": { 2024-09-21T06:08:55.583 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:55.584 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:55.584 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:55.584 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.584 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lssnap_latency": { 2024-09-21T06:08:55.584 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 11, 2024-09-21T06:08:55.584 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.014092412, 2024-09-21T06:08:55.584 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.001281128 2024-09-21T06:08:55.584 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.584 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mkdir_latency": { 2024-09-21T06:08:55.584 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 10, 2024-09-21T06:08:55.585 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.015606917, 2024-09-21T06:08:55.585 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.001560691 2024-09-21T06:08:55.585 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.585 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mknod_latency": { 2024-09-21T06:08:55.585 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:55.585 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:55.585 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:55.585 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.585 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mksnap_latency": { 2024-09-21T06:08:55.585 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 11, 2024-09-21T06:08:55.585 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 14.016853029, 2024-09-21T06:08:55.585 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 1.274259366 2024-09-21T06:08:55.585 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.585 INFO:teuthology.orchestra.run.smithi138.stdout: "req_open_latency": { 2024-09-21T06:08:55.585 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 1, 2024-09-21T06:08:55.586 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000175406, 2024-09-21T06:08:55.586 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000175406 2024-09-21T06:08:55.586 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.586 INFO:teuthology.orchestra.run.smithi138.stdout: "req_readdir_latency": { 2024-09-21T06:08:55.586 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 5, 2024-09-21T06:08:55.586 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.015242146, 2024-09-21T06:08:55.586 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.003048429 2024-09-21T06:08:55.586 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.586 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rename_latency": { 2024-09-21T06:08:55.586 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2, 2024-09-21T06:08:55.586 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.002302460, 2024-09-21T06:08:55.586 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.001151230 2024-09-21T06:08:55.586 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.586 INFO:teuthology.orchestra.run.smithi138.stdout: "req_renamesnap_latency": { 2024-09-21T06:08:55.587 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:55.587 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:55.587 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:55.587 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.587 INFO:teuthology.orchestra.run.smithi138.stdout: "req_snapdiff_latency": { 2024-09-21T06:08:55.587 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:55.587 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:55.587 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:55.587 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.587 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmdir_latency": { 2024-09-21T06:08:55.587 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 4, 2024-09-21T06:08:55.587 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.591128964, 2024-09-21T06:08:55.587 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.147782241 2024-09-21T06:08:55.587 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.587 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmsnap_latency": { 2024-09-21T06:08:55.587 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 4, 2024-09-21T06:08:55.588 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 1.499293645, 2024-09-21T06:08:55.588 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.374823411 2024-09-21T06:08:55.588 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.588 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmxattr_latency": { 2024-09-21T06:08:55.588 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:55.588 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:55.588 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:55.588 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.588 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setattr_latency": { 2024-09-21T06:08:55.588 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 1, 2024-09-21T06:08:55.588 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000887903, 2024-09-21T06:08:55.588 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000887903 2024-09-21T06:08:55.588 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.588 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setdirlayout_latency": { 2024-09-21T06:08:55.588 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:55.589 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:55.589 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:55.589 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.589 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setfilelock_latency": { 2024-09-21T06:08:55.589 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:55.589 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:55.589 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:55.589 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.589 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setlayout_latency": { 2024-09-21T06:08:55.589 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:55.589 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:55.589 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:55.589 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.589 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setxattr_latency": { 2024-09-21T06:08:55.590 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 9, 2024-09-21T06:08:55.590 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.023529868, 2024-09-21T06:08:55.590 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.002614429 2024-09-21T06:08:55.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.590 INFO:teuthology.orchestra.run.smithi138.stdout: "req_symlink_latency": { 2024-09-21T06:08:55.590 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:55.590 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:55.590 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:55.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.590 INFO:teuthology.orchestra.run.smithi138.stdout: "req_unlink_latency": { 2024-09-21T06:08:55.590 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 18, 2024-09-21T06:08:55.590 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.023649604, 2024-09-21T06:08:55.590 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.001313866 2024-09-21T06:08:55.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.590 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_revoke_eviction": 0, 2024-09-21T06:08:55.591 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_acquisition_throttle": 0, 2024-09-21T06:08:55.591 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getvxattr_latency": { 2024-09-21T06:08:55.591 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:55.591 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:55.591 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:55.591 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:08:55.591 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.591 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_sessions": { 2024-09-21T06:08:55.591 INFO:teuthology.orchestra.run.smithi138.stdout: "session_count": 2, 2024-09-21T06:08:55.591 INFO:teuthology.orchestra.run.smithi138.stdout: "session_add": 3, 2024-09-21T06:08:55.591 INFO:teuthology.orchestra.run.smithi138.stdout: "session_remove": 1, 2024-09-21T06:08:55.591 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_open": 2, 2024-09-21T06:08:55.591 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_stale": 0, 2024-09-21T06:08:55.591 INFO:teuthology.orchestra.run.smithi138.stdout: "total_load": 65, 2024-09-21T06:08:55.591 INFO:teuthology.orchestra.run.smithi138.stdout: "average_load": 32, 2024-09-21T06:08:55.592 INFO:teuthology.orchestra.run.smithi138.stdout: "avg_session_uptime": 681, 2024-09-21T06:08:55.592 INFO:teuthology.orchestra.run.smithi138.stdout: "mdthresh_evicted": 0 2024-09-21T06:08:55.592 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.592 INFO:teuthology.orchestra.run.smithi138.stdout: "mempool": { 2024-09-21T06:08:55.592 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:08:55.592 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_items": 0, 2024-09-21T06:08:55.592 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:08:55.592 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:08:55.592 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:08:55.592 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:08:55.592 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:08:55.592 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:08:55.592 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:08:55.592 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:08:55.592 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:08:55.592 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:08:55.593 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:08:55.593 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:08:55.593 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:08:55.593 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_items": 0, 2024-09-21T06:08:55.593 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:08:55.593 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_items": 0, 2024-09-21T06:08:55.593 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:08:55.593 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:08:55.593 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:08:55.593 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:08:55.593 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:08:55.593 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:08:55.593 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:08:55.593 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_items": 0, 2024-09-21T06:08:55.593 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:08:55.594 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:08:55.594 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:08:55.594 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_items": 0, 2024-09-21T06:08:55.594 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_bytes": 0, 2024-09-21T06:08:55.594 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_items": 0, 2024-09-21T06:08:55.594 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:08:55.594 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:08:55.594 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:08:55.594 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:08:55.594 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_bytes": 337943, 2024-09-21T06:08:55.594 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_items": 153, 2024-09-21T06:08:55.594 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_bytes": 176, 2024-09-21T06:08:55.594 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_items": 2, 2024-09-21T06:08:55.594 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bytes": 0, 2024-09-21T06:08:55.595 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_items": 0, 2024-09-21T06:08:55.595 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:08:55.595 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_items": 0, 2024-09-21T06:08:55.595 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:08:55.595 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_items": 0, 2024-09-21T06:08:55.595 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_bytes": 7968, 2024-09-21T06:08:55.595 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_items": 176, 2024-09-21T06:08:55.595 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:08:55.595 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:08:55.595 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_bytes": 0, 2024-09-21T06:08:55.595 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_items": 0, 2024-09-21T06:08:55.595 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_bytes": 386014, 2024-09-21T06:08:55.595 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_items": 3203, 2024-09-21T06:08:55.595 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_bytes": 0, 2024-09-21T06:08:55.595 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_items": 0, 2024-09-21T06:08:55.596 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_bytes": 0, 2024-09-21T06:08:55.596 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_items": 0 2024-09-21T06:08:55.596 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.596 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter": { 2024-09-21T06:08:55.596 INFO:teuthology.orchestra.run.smithi138.stdout: "op_active": 0, 2024-09-21T06:08:55.596 INFO:teuthology.orchestra.run.smithi138.stdout: "op_laggy": 0, 2024-09-21T06:08:55.596 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send": 18760, 2024-09-21T06:08:55.596 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send_bytes": 1076880024, 2024-09-21T06:08:55.596 INFO:teuthology.orchestra.run.smithi138.stdout: "op_resend": 37, 2024-09-21T06:08:55.596 INFO:teuthology.orchestra.run.smithi138.stdout: "op_reply": 18760, 2024-09-21T06:08:55.596 INFO:teuthology.orchestra.run.smithi138.stdout: "op_latency": { 2024-09-21T06:08:55.596 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 18760, 2024-09-21T06:08:55.596 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 1391.508974926, 2024-09-21T06:08:55.596 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.074174252 2024-09-21T06:08:55.596 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.596 INFO:teuthology.orchestra.run.smithi138.stdout: "op_inflight": 0, 2024-09-21T06:08:55.597 INFO:teuthology.orchestra.run.smithi138.stdout: "oplen_avg": { 2024-09-21T06:08:55.597 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 18760, 2024-09-21T06:08:55.597 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 19310 2024-09-21T06:08:55.597 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.597 INFO:teuthology.orchestra.run.smithi138.stdout: "op": 18760, 2024-09-21T06:08:55.597 INFO:teuthology.orchestra.run.smithi138.stdout: "op_r": 413, 2024-09-21T06:08:55.597 INFO:teuthology.orchestra.run.smithi138.stdout: "op_w": 18347, 2024-09-21T06:08:55.597 INFO:teuthology.orchestra.run.smithi138.stdout: "op_rmw": 0, 2024-09-21T06:08:55.597 INFO:teuthology.orchestra.run.smithi138.stdout: "op_pg": 0, 2024-09-21T06:08:55.597 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_stat": 211, 2024-09-21T06:08:55.597 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_create": 44, 2024-09-21T06:08:55.597 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_read": 14, 2024-09-21T06:08:55.597 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_write": 13966, 2024-09-21T06:08:55.597 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writefull": 142, 2024-09-21T06:08:55.597 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writesame": 0, 2024-09-21T06:08:55.598 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_append": 0, 2024-09-21T06:08:55.598 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_zero": 0, 2024-09-21T06:08:55.598 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_truncate": 0, 2024-09-21T06:08:55.598 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_delete": 3602, 2024-09-21T06:08:55.598 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_mapext": 0, 2024-09-21T06:08:55.598 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_sparse_read": 0, 2024-09-21T06:08:55.598 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_clonerange": 0, 2024-09-21T06:08:55.598 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_getxattr": 399, 2024-09-21T06:08:55.598 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_setxattr": 88, 2024-09-21T06:08:55.598 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_cmpxattr": 0, 2024-09-21T06:08:55.598 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_rmxattr": 0, 2024-09-21T06:08:55.598 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_resetxattrs": 0, 2024-09-21T06:08:55.598 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_call": 0, 2024-09-21T06:08:55.598 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_watch": 0, 2024-09-21T06:08:55.598 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_notify": 0, 2024-09-21T06:08:55.599 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T06:08:55.599 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls": 0, 2024-09-21T06:08:55.599 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls_filter": 0, 2024-09-21T06:08:55.599 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_other": 0, 2024-09-21T06:08:55.599 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_active": 0, 2024-09-21T06:08:55.599 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_send": 0, 2024-09-21T06:08:55.599 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_resend": 0, 2024-09-21T06:08:55.599 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_ping": 0, 2024-09-21T06:08:55.599 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_active": 0, 2024-09-21T06:08:55.599 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_send": 0, 2024-09-21T06:08:55.599 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_resend": 0, 2024-09-21T06:08:55.599 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_active": 0, 2024-09-21T06:08:55.599 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_send": 0, 2024-09-21T06:08:55.599 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_resend": 0, 2024-09-21T06:08:55.599 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_active": 0, 2024-09-21T06:08:55.600 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_send": 0, 2024-09-21T06:08:55.600 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_resend": 0, 2024-09-21T06:08:55.600 INFO:teuthology.orchestra.run.smithi138.stdout: "command_active": 0, 2024-09-21T06:08:55.600 INFO:teuthology.orchestra.run.smithi138.stdout: "command_send": 0, 2024-09-21T06:08:55.600 INFO:teuthology.orchestra.run.smithi138.stdout: "command_resend": 0, 2024-09-21T06:08:55.600 INFO:teuthology.orchestra.run.smithi138.stdout: "map_epoch": 85, 2024-09-21T06:08:55.600 INFO:teuthology.orchestra.run.smithi138.stdout: "map_full": 0, 2024-09-21T06:08:55.600 INFO:teuthology.orchestra.run.smithi138.stdout: "map_inc": 11, 2024-09-21T06:08:55.600 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_sessions": 12, 2024-09-21T06:08:55.600 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_open": 12, 2024-09-21T06:08:55.600 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_close": 0, 2024-09-21T06:08:55.600 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_laggy": 0, 2024-09-21T06:08:55.600 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_wr": 828, 2024-09-21T06:08:55.600 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_rd": 0, 2024-09-21T06:08:55.600 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_del": 16 2024-09-21T06:08:55.601 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.601 INFO:teuthology.orchestra.run.smithi138.stdout: "oft": { 2024-09-21T06:08:55.601 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_objs": 1, 2024-09-21T06:08:55.601 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_kv_pairs": 0, 2024-09-21T06:08:55.601 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_updates": 0, 2024-09-21T06:08:55.601 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_removes": 0 2024-09-21T06:08:55.601 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.601 INFO:teuthology.orchestra.run.smithi138.stdout: "purge_queue": { 2024-09-21T06:08:55.601 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops": 0, 2024-09-21T06:08:55.601 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops_high_water": 518, 2024-09-21T06:08:55.601 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing": 0, 2024-09-21T06:08:55.601 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_high_water": 6, 2024-09-21T06:08:55.601 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed_ops": 2578, 2024-09-21T06:08:55.601 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed": 18, 2024-09-21T06:08:55.602 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_item_in_journal": 0 2024-09-21T06:08:55.602 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.602 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:08:55.602 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-21T06:08:55.602 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-09-21T06:08:55.602 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-21T06:08:55.602 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 363013, 2024-09-21T06:08:55.602 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 93534164, 2024-09-21T06:08:55.602 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-21T06:08:55.602 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 363013, 2024-09-21T06:08:55.602 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-09-21T06:08:55.602 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-09-21T06:08:55.602 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 363013, 2024-09-21T06:08:55.602 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 93534164, 2024-09-21T06:08:55.602 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-21T06:08:55.603 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:55.603 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:55.603 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:55.603 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:08:55.603 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.603 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_bytes": { 2024-09-21T06:08:55.603 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-21T06:08:55.603 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-09-21T06:08:55.603 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-21T06:08:55.603 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-09-21T06:08:55.603 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-09-21T06:08:55.603 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-21T06:08:55.603 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-09-21T06:08:55.603 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 18760, 2024-09-21T06:08:55.603 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 1076882850, 2024-09-21T06:08:55.604 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 18760, 2024-09-21T06:08:55.604 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 1076882850, 2024-09-21T06:08:55.604 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-21T06:08:55.604 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:55.604 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:55.604 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:55.604 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:08:55.604 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.604 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_ops": { 2024-09-21T06:08:55.604 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-21T06:08:55.604 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1024, 2024-09-21T06:08:55.604 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-21T06:08:55.604 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-09-21T06:08:55.604 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-09-21T06:08:55.604 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-21T06:08:55.604 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-09-21T06:08:55.605 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 18760, 2024-09-21T06:08:55.605 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 18760, 2024-09-21T06:08:55.605 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 18760, 2024-09-21T06:08:55.605 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 18760, 2024-09-21T06:08:55.605 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-21T06:08:55.605 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:55.605 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:55.605 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:55.605 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:08:55.605 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.605 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle": { 2024-09-21T06:08:55.605 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-21T06:08:55.605 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-09-21T06:08:55.605 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-21T06:08:55.605 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 18, 2024-09-21T06:08:55.605 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 2826, 2024-09-21T06:08:55.606 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-21T06:08:55.606 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 18, 2024-09-21T06:08:55.606 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-09-21T06:08:55.606 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-09-21T06:08:55.606 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 1, 2024-09-21T06:08:55.606 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 2826, 2024-09-21T06:08:55.606 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-21T06:08:55.606 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:55.606 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:55.606 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:55.606 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:08:55.606 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.606 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle-0x55ba457a9450": { 2024-09-21T06:08:55.606 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-21T06:08:55.606 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-09-21T06:08:55.607 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-21T06:08:55.607 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 339264, 2024-09-21T06:08:55.607 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 1075639796, 2024-09-21T06:08:55.607 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-21T06:08:55.607 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 339264, 2024-09-21T06:08:55.607 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-09-21T06:08:55.607 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-09-21T06:08:55.607 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 13965, 2024-09-21T06:08:55.607 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 1075639796, 2024-09-21T06:08:55.607 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-21T06:08:55.607 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:55.607 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:55.607 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:55.607 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:08:55.607 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:55.608 INFO:teuthology.orchestra.run.smithi138.stdout: "trackedop": { 2024-09-21T06:08:55.608 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_ops_count": 0 2024-09-21T06:08:55.608 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:08:55.608 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-21T06:08:56.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:55 smithi192 ceph-mon[34700]: pgmap v684: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:08:56.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:55 smithi117 ceph-mon[30947]: pgmap v684: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:08:56.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:55 smithi138 ceph-mon[34391]: pgmap v684: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:08:56.639 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-09-21T06:08:56.639 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-09-21T06:08:56.639 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=19 2024-09-21T06:08:56.639 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-09-21T06:08:56.639 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=7 2024-09-21T06:08:56.639 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-09-21T06:08:56.639 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-09-21T06:08:56.639 DEBUG:tasks.check_counter:Getting stats from e 2024-09-21T06:08:56.639 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph daemon mds.e perf dump 2024-09-21T06:08:56.895 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:08:57.524 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-21T06:08:57.524 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:08:57.524 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 2335, 2024-09-21T06:08:57.525 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 2337, 2024-09-21T06:08:57.525 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 661907, 2024-09-21T06:08:57.525 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 742537, 2024-09-21T06:08:57.525 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 6, 2024-09-21T06:08:57.525 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 2, 2024-09-21T06:08:57.525 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.246782410, 2024-09-21T06:08:57.525 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.101218796, 2024-09-21T06:08:57.525 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.052893829, 2024-09-21T06:08:57.525 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.077800589, 2024-09-21T06:08:57.525 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:08:57.525 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2335, 2024-09-21T06:08:57.525 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.107798790, 2024-09-21T06:08:57.525 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000046166 2024-09-21T06:08:57.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.526 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:08:57.526 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.526 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.526 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.526 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T06:08:57.526 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T06:08:57.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.526 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:08:57.526 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 255, 2024-09-21T06:08:57.526 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 247, 2024-09-21T06:08:57.526 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 883932, 2024-09-21T06:08:57.526 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 144330, 2024-09-21T06:08:57.526 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 3, 2024-09-21T06:08:57.527 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 2, 2024-09-21T06:08:57.527 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.047332534, 2024-09-21T06:08:57.527 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.021842195, 2024-09-21T06:08:57.527 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.036428035, 2024-09-21T06:08:57.527 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.000065625, 2024-09-21T06:08:57.527 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:08:57.527 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 246, 2024-09-21T06:08:57.527 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.019516311, 2024-09-21T06:08:57.527 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000079334 2024-09-21T06:08:57.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.527 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:08:57.527 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.527 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.527 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.528 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.528 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 883376, 2024-09-21T06:08:57.528 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 143632 2024-09-21T06:08:57.528 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.528 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:08:57.528 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 6, 2024-09-21T06:08:57.528 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 177, 2024-09-21T06:08:57.528 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 3420, 2024-09-21T06:08:57.528 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 388254, 2024-09-21T06:08:57.528 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 4, 2024-09-21T06:08:57.528 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 3, 2024-09-21T06:08:57.528 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.015959474, 2024-09-21T06:08:57.528 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.010920123, 2024-09-21T06:08:57.528 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.003384455, 2024-09-21T06:08:57.528 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.000031371, 2024-09-21T06:08:57.529 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:08:57.529 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 176, 2024-09-21T06:08:57.529 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.016360074, 2024-09-21T06:08:57.529 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000092954 2024-09-21T06:08:57.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.529 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:08:57.529 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.529 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.529 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.529 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-21T06:08:57.529 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 387936 2024-09-21T06:08:57.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.529 INFO:teuthology.orchestra.run.smithi138.stdout: "cct": { 2024-09-21T06:08:57.530 INFO:teuthology.orchestra.run.smithi138.stdout: "total_workers": 1, 2024-09-21T06:08:57.530 INFO:teuthology.orchestra.run.smithi138.stdout: "unhealthy_workers": 0 2024-09-21T06:08:57.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.530 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-MDSRank": { 2024-09-21T06:08:57.530 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-09-21T06:08:57.530 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-09-21T06:08:57.530 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 3883, 2024-09-21T06:08:57.530 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.117589303, 2024-09-21T06:08:57.530 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000030283 2024-09-21T06:08:57.530 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:08:57.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.530 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-PurgeQueue": { 2024-09-21T06:08:57.530 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-09-21T06:08:57.530 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-09-21T06:08:57.530 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.531 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.531 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.531 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:08:57.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.531 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": { 2024-09-21T06:08:57.531 INFO:teuthology.orchestra.run.smithi138.stdout: "request": 0, 2024-09-21T06:08:57.531 INFO:teuthology.orchestra.run.smithi138.stdout: "reply": 0, 2024-09-21T06:08:57.531 INFO:teuthology.orchestra.run.smithi138.stdout: "reply_latency": { 2024-09-21T06:08:57.531 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.531 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.531 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.531 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_reply": 0, 2024-09-21T06:08:57.531 INFO:teuthology.orchestra.run.smithi138.stdout: "forward": 0, 2024-09-21T06:08:57.532 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_complete": 0, 2024-09-21T06:08:57.532 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_keys": 0, 2024-09-21T06:08:57.532 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_commit": 0, 2024-09-21T06:08:57.532 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_split": 0, 2024-09-21T06:08:57.532 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_merge": 0, 2024-09-21T06:08:57.532 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes": 0, 2024-09-21T06:08:57.532 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_top": 0, 2024-09-21T06:08:57.532 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_bottom": 0, 2024-09-21T06:08:57.532 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pin_tail": 0, 2024-09-21T06:08:57.532 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pinned": 0, 2024-09-21T06:08:57.532 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_expired": 0, 2024-09-21T06:08:57.532 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_with_caps": 0, 2024-09-21T06:08:57.532 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": 0, 2024-09-21T06:08:57.532 INFO:teuthology.orchestra.run.smithi138.stdout: "subtrees": 1, 2024-09-21T06:08:57.533 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse": 0, 2024-09-21T06:08:57.533 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_hit": 0, 2024-09-21T06:08:57.533 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_forward": 0, 2024-09-21T06:08:57.533 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_discover": 0, 2024-09-21T06:08:57.533 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_dir_fetch": 0, 2024-09-21T06:08:57.533 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_remote_ino": 0, 2024-09-21T06:08:57.533 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_lock": 0, 2024-09-21T06:08:57.533 INFO:teuthology.orchestra.run.smithi138.stdout: "load_cent": 0, 2024-09-21T06:08:57.533 INFO:teuthology.orchestra.run.smithi138.stdout: "q": 0, 2024-09-21T06:08:57.533 INFO:teuthology.orchestra.run.smithi138.stdout: "exported": 0, 2024-09-21T06:08:57.533 INFO:teuthology.orchestra.run.smithi138.stdout: "exported_inodes": 0, 2024-09-21T06:08:57.533 INFO:teuthology.orchestra.run.smithi138.stdout: "imported": 0, 2024-09-21T06:08:57.533 INFO:teuthology.orchestra.run.smithi138.stdout: "imported_inodes": 0, 2024-09-21T06:08:57.533 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_dir_fetch": 0, 2024-09-21T06:08:57.533 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_backtrace_fetch": 0, 2024-09-21T06:08:57.534 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_peer_discover": 0, 2024-09-21T06:08:57.534 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rfiles": 0, 2024-09-21T06:08:57.534 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rbytes": 0, 2024-09-21T06:08:57.534 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rsnaps": 0, 2024-09-21T06:08:57.534 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T06:08:57.534 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_set_tag": 0, 2024-09-21T06:08:57.534 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T06:08:57.534 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_inotable_repaired": 0, 2024-09-21T06:08:57.534 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_inodes": 0, 2024-09-21T06:08:57.534 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T06:08:57.534 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T06:08:57.534 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_file_inodes": 0, 2024-09-21T06:08:57.534 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_inode_file_caps": 0, 2024-09-21T06:08:57.534 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T06:08:57.534 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_grant": 0, 2024-09-21T06:08:57.535 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T06:08:57.535 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T06:08:57.535 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T06:08:57.535 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps": 0, 2024-09-21T06:08:57.535 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps_dirty": 0, 2024-09-21T06:08:57.535 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_cap_release": 0, 2024-09-21T06:08:57.535 INFO:teuthology.orchestra.run.smithi138.stdout: "process_request_cap_release": 0 2024-09-21T06:08:57.535 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.535 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cache": { 2024-09-21T06:08:57.535 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update": 0, 2024-09-21T06:08:57.535 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update_receipt": 0, 2024-09-21T06:08:57.535 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_try_discover": 0, 2024-09-21T06:08:57.535 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_send_discover": 0, 2024-09-21T06:08:57.535 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_handle_discover": 0, 2024-09-21T06:08:57.535 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays": 0, 2024-09-21T06:08:57.536 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_delayed": 0, 2024-09-21T06:08:57.536 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_enqueuing": 0, 2024-09-21T06:08:57.536 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_created": 0, 2024-09-21T06:08:57.536 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_enqueued": 0, 2024-09-21T06:08:57.536 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_reintegrated": 0, 2024-09-21T06:08:57.536 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_migrated": 0, 2024-09-21T06:08:57.536 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_processing": 0, 2024-09-21T06:08:57.536 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_enqueued": 0, 2024-09-21T06:08:57.536 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_prioritized": 0, 2024-09-21T06:08:57.536 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_started": 0, 2024-09-21T06:08:57.536 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_completed": 0, 2024-09-21T06:08:57.536 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_path": 0, 2024-09-21T06:08:57.536 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_inode": 0, 2024-09-21T06:08:57.536 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T06:08:57.536 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_exportdir": 0, 2024-09-21T06:08:57.537 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_flush": 0, 2024-09-21T06:08:57.537 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragmentdir": 0, 2024-09-21T06:08:57.537 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragstats": 0, 2024-09-21T06:08:57.537 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_inodestats": 0, 2024-09-21T06:08:57.537 INFO:teuthology.orchestra.run.smithi138.stdout: "uninline_started": 0, 2024-09-21T06:08:57.537 INFO:teuthology.orchestra.run.smithi138.stdout: "uninline_succeeded": 0, 2024-09-21T06:08:57.537 INFO:teuthology.orchestra.run.smithi138.stdout: "uninline_write_failed": 0 2024-09-21T06:08:57.537 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.537 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_log": { 2024-09-21T06:08:57.537 INFO:teuthology.orchestra.run.smithi138.stdout: "evlrg": 0, 2024-09-21T06:08:57.537 INFO:teuthology.orchestra.run.smithi138.stdout: "evadd": 0, 2024-09-21T06:08:57.537 INFO:teuthology.orchestra.run.smithi138.stdout: "evex": 0, 2024-09-21T06:08:57.537 INFO:teuthology.orchestra.run.smithi138.stdout: "evtrm": 0, 2024-09-21T06:08:57.537 INFO:teuthology.orchestra.run.smithi138.stdout: "ev": 4, 2024-09-21T06:08:57.537 INFO:teuthology.orchestra.run.smithi138.stdout: "evexg": 0, 2024-09-21T06:08:57.537 INFO:teuthology.orchestra.run.smithi138.stdout: "evexd": 0, 2024-09-21T06:08:57.538 INFO:teuthology.orchestra.run.smithi138.stdout: "segadd": 0, 2024-09-21T06:08:57.538 INFO:teuthology.orchestra.run.smithi138.stdout: "segex": 0, 2024-09-21T06:08:57.538 INFO:teuthology.orchestra.run.smithi138.stdout: "segtrm": 0, 2024-09-21T06:08:57.538 INFO:teuthology.orchestra.run.smithi138.stdout: "seg": 2, 2024-09-21T06:08:57.538 INFO:teuthology.orchestra.run.smithi138.stdout: "segmjr": 2, 2024-09-21T06:08:57.538 INFO:teuthology.orchestra.run.smithi138.stdout: "segexg": 0, 2024-09-21T06:08:57.538 INFO:teuthology.orchestra.run.smithi138.stdout: "segexd": 0, 2024-09-21T06:08:57.538 INFO:teuthology.orchestra.run.smithi138.stdout: "expos": 4194304, 2024-09-21T06:08:57.538 INFO:teuthology.orchestra.run.smithi138.stdout: "wrpos": 4195424, 2024-09-21T06:08:57.538 INFO:teuthology.orchestra.run.smithi138.stdout: "rdpos": 4195283, 2024-09-21T06:08:57.538 INFO:teuthology.orchestra.run.smithi138.stdout: "jlat": { 2024-09-21T06:08:57.538 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.538 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.538 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.539 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.539 INFO:teuthology.orchestra.run.smithi138.stdout: "replayed": 4 2024-09-21T06:08:57.539 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.539 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_mem": { 2024-09-21T06:08:57.539 INFO:teuthology.orchestra.run.smithi138.stdout: "ino": 3, 2024-09-21T06:08:57.539 INFO:teuthology.orchestra.run.smithi138.stdout: "ino+": 3, 2024-09-21T06:08:57.539 INFO:teuthology.orchestra.run.smithi138.stdout: "ino-": 0, 2024-09-21T06:08:57.539 INFO:teuthology.orchestra.run.smithi138.stdout: "dir": 1, 2024-09-21T06:08:57.539 INFO:teuthology.orchestra.run.smithi138.stdout: "dir+": 1, 2024-09-21T06:08:57.539 INFO:teuthology.orchestra.run.smithi138.stdout: "dir-": 0, 2024-09-21T06:08:57.539 INFO:teuthology.orchestra.run.smithi138.stdout: "dn": 0, 2024-09-21T06:08:57.539 INFO:teuthology.orchestra.run.smithi138.stdout: "dn+": 0, 2024-09-21T06:08:57.539 INFO:teuthology.orchestra.run.smithi138.stdout: "dn-": 0, 2024-09-21T06:08:57.539 INFO:teuthology.orchestra.run.smithi138.stdout: "cap": 0, 2024-09-21T06:08:57.539 INFO:teuthology.orchestra.run.smithi138.stdout: "cap+": 0, 2024-09-21T06:08:57.539 INFO:teuthology.orchestra.run.smithi138.stdout: "cap-": 0, 2024-09-21T06:08:57.540 INFO:teuthology.orchestra.run.smithi138.stdout: "rss": 117128, 2024-09-21T06:08:57.540 INFO:teuthology.orchestra.run.smithi138.stdout: "heap": 316420096 2024-09-21T06:08:57.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.540 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_server": { 2024-09-21T06:08:57.540 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_client_request": 0, 2024-09-21T06:08:57.540 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_server_request": 0, 2024-09-21T06:08:57.540 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_request": 0, 2024-09-21T06:08:57.540 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_session": 0, 2024-09-21T06:08:57.540 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_peer_request": 0, 2024-09-21T06:08:57.540 INFO:teuthology.orchestra.run.smithi138.stdout: "req_create_latency": { 2024-09-21T06:08:57.540 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.540 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.540 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.540 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getattr_latency": { 2024-09-21T06:08:57.541 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.541 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.541 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.541 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getfilelock_latency": { 2024-09-21T06:08:57.541 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.541 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.541 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.541 INFO:teuthology.orchestra.run.smithi138.stdout: "req_link_latency": { 2024-09-21T06:08:57.541 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.541 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.541 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.541 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookup_latency": { 2024-09-21T06:08:57.541 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.542 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.542 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.542 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookuphash_latency": { 2024-09-21T06:08:57.542 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.542 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.542 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.542 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupino_latency": { 2024-09-21T06:08:57.542 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.542 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.542 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.542 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupname_latency": { 2024-09-21T06:08:57.542 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.543 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.543 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.543 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupparent_latency": { 2024-09-21T06:08:57.543 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.543 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.543 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.543 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupsnap_latency": { 2024-09-21T06:08:57.543 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.543 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.543 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.543 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lssnap_latency": { 2024-09-21T06:08:57.543 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.544 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.544 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.544 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.544 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mkdir_latency": { 2024-09-21T06:08:57.544 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.544 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.544 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.545 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mknod_latency": { 2024-09-21T06:08:57.545 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.545 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.545 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.545 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mksnap_latency": { 2024-09-21T06:08:57.545 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.545 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.545 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.545 INFO:teuthology.orchestra.run.smithi138.stdout: "req_open_latency": { 2024-09-21T06:08:57.545 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.545 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.545 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.546 INFO:teuthology.orchestra.run.smithi138.stdout: "req_readdir_latency": { 2024-09-21T06:08:57.546 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.546 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.546 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.546 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rename_latency": { 2024-09-21T06:08:57.546 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.546 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.546 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.546 INFO:teuthology.orchestra.run.smithi138.stdout: "req_renamesnap_latency": { 2024-09-21T06:08:57.546 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.546 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.547 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.547 INFO:teuthology.orchestra.run.smithi138.stdout: "req_snapdiff_latency": { 2024-09-21T06:08:57.547 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.547 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.547 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.547 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmdir_latency": { 2024-09-21T06:08:57.547 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.547 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.547 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.547 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmsnap_latency": { 2024-09-21T06:08:57.547 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.547 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.547 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.548 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmxattr_latency": { 2024-09-21T06:08:57.548 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.548 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.548 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.548 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setattr_latency": { 2024-09-21T06:08:57.548 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.548 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.548 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.548 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setdirlayout_latency": { 2024-09-21T06:08:57.548 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.548 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.548 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.549 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setfilelock_latency": { 2024-09-21T06:08:57.549 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.549 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.549 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.549 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setlayout_latency": { 2024-09-21T06:08:57.549 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.549 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.549 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.549 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setxattr_latency": { 2024-09-21T06:08:57.549 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.549 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.550 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.550 INFO:teuthology.orchestra.run.smithi138.stdout: "req_symlink_latency": { 2024-09-21T06:08:57.550 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.550 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.550 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.550 INFO:teuthology.orchestra.run.smithi138.stdout: "req_unlink_latency": { 2024-09-21T06:08:57.550 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.550 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.550 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.550 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_revoke_eviction": 0, 2024-09-21T06:08:57.550 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_acquisition_throttle": 0, 2024-09-21T06:08:57.550 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getvxattr_latency": { 2024-09-21T06:08:57.551 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.551 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.551 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.551 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:08:57.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.551 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_sessions": { 2024-09-21T06:08:57.551 INFO:teuthology.orchestra.run.smithi138.stdout: "session_count": 0, 2024-09-21T06:08:57.551 INFO:teuthology.orchestra.run.smithi138.stdout: "session_add": 1, 2024-09-21T06:08:57.551 INFO:teuthology.orchestra.run.smithi138.stdout: "session_remove": 1, 2024-09-21T06:08:57.551 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_open": 1, 2024-09-21T06:08:57.551 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_stale": 0, 2024-09-21T06:08:57.551 INFO:teuthology.orchestra.run.smithi138.stdout: "total_load": 0, 2024-09-21T06:08:57.551 INFO:teuthology.orchestra.run.smithi138.stdout: "average_load": 0, 2024-09-21T06:08:57.551 INFO:teuthology.orchestra.run.smithi138.stdout: "avg_session_uptime": 1315, 2024-09-21T06:08:57.552 INFO:teuthology.orchestra.run.smithi138.stdout: "mdthresh_evicted": 0 2024-09-21T06:08:57.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.552 INFO:teuthology.orchestra.run.smithi138.stdout: "mempool": { 2024-09-21T06:08:57.552 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:08:57.552 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_items": 0, 2024-09-21T06:08:57.553 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:08:57.553 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:08:57.553 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:08:57.554 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:08:57.554 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:08:57.554 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:08:57.554 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:08:57.554 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:08:57.554 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:08:57.554 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:08:57.554 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:08:57.554 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:08:57.554 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:08:57.554 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_items": 0, 2024-09-21T06:08:57.554 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:08:57.554 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_items": 0, 2024-09-21T06:08:57.554 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:08:57.555 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:08:57.555 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:08:57.555 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:08:57.555 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:08:57.555 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:08:57.555 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:08:57.555 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_items": 0, 2024-09-21T06:08:57.555 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:08:57.555 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:08:57.555 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:08:57.555 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_items": 0, 2024-09-21T06:08:57.555 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_bytes": 0, 2024-09-21T06:08:57.555 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_items": 0, 2024-09-21T06:08:57.555 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:08:57.556 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:08:57.556 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:08:57.556 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:08:57.556 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_bytes": 181383, 2024-09-21T06:08:57.556 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_items": 48, 2024-09-21T06:08:57.556 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_bytes": 0, 2024-09-21T06:08:57.556 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_items": 0, 2024-09-21T06:08:57.556 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bytes": 0, 2024-09-21T06:08:57.556 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_items": 0, 2024-09-21T06:08:57.556 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:08:57.556 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_items": 0, 2024-09-21T06:08:57.556 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:08:57.556 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_items": 0, 2024-09-21T06:08:57.556 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_bytes": 7968, 2024-09-21T06:08:57.557 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_items": 176, 2024-09-21T06:08:57.557 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:08:57.557 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:08:57.557 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_bytes": 0, 2024-09-21T06:08:57.557 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_items": 0, 2024-09-21T06:08:57.557 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_bytes": 8672, 2024-09-21T06:08:57.557 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_items": 13, 2024-09-21T06:08:57.557 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_bytes": 0, 2024-09-21T06:08:57.557 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_items": 0, 2024-09-21T06:08:57.557 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_bytes": 0, 2024-09-21T06:08:57.557 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_items": 0 2024-09-21T06:08:57.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.557 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter": { 2024-09-21T06:08:57.557 INFO:teuthology.orchestra.run.smithi138.stdout: "op_active": 0, 2024-09-21T06:08:57.557 INFO:teuthology.orchestra.run.smithi138.stdout: "op_laggy": 0, 2024-09-21T06:08:57.558 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send": 2338, 2024-09-21T06:08:57.558 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send_bytes": 21, 2024-09-21T06:08:57.558 INFO:teuthology.orchestra.run.smithi138.stdout: "op_resend": 3, 2024-09-21T06:08:57.558 INFO:teuthology.orchestra.run.smithi138.stdout: "op_reply": 2338, 2024-09-21T06:08:57.558 INFO:teuthology.orchestra.run.smithi138.stdout: "op_latency": { 2024-09-21T06:08:57.558 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2338, 2024-09-21T06:08:57.558 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 3.168931943, 2024-09-21T06:08:57.558 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.001355402 2024-09-21T06:08:57.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.558 INFO:teuthology.orchestra.run.smithi138.stdout: "op_inflight": 0, 2024-09-21T06:08:57.558 INFO:teuthology.orchestra.run.smithi138.stdout: "oplen_avg": { 2024-09-21T06:08:57.558 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2338, 2024-09-21T06:08:57.558 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 2339 2024-09-21T06:08:57.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.559 INFO:teuthology.orchestra.run.smithi138.stdout: "op": 2338, 2024-09-21T06:08:57.559 INFO:teuthology.orchestra.run.smithi138.stdout: "op_r": 2338, 2024-09-21T06:08:57.559 INFO:teuthology.orchestra.run.smithi138.stdout: "op_w": 0, 2024-09-21T06:08:57.559 INFO:teuthology.orchestra.run.smithi138.stdout: "op_rmw": 0, 2024-09-21T06:08:57.559 INFO:teuthology.orchestra.run.smithi138.stdout: "op_pg": 0, 2024-09-21T06:08:57.559 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_stat": 1521, 2024-09-21T06:08:57.559 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_create": 0, 2024-09-21T06:08:57.559 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_read": 815, 2024-09-21T06:08:57.559 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_write": 0, 2024-09-21T06:08:57.559 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writefull": 0, 2024-09-21T06:08:57.559 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writesame": 0, 2024-09-21T06:08:57.559 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_append": 0, 2024-09-21T06:08:57.559 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_zero": 0, 2024-09-21T06:08:57.560 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_truncate": 0, 2024-09-21T06:08:57.560 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_delete": 0, 2024-09-21T06:08:57.560 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_mapext": 0, 2024-09-21T06:08:57.560 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_sparse_read": 0, 2024-09-21T06:08:57.560 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_clonerange": 0, 2024-09-21T06:08:57.560 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_getxattr": 1, 2024-09-21T06:08:57.560 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_setxattr": 0, 2024-09-21T06:08:57.560 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_cmpxattr": 0, 2024-09-21T06:08:57.560 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_rmxattr": 0, 2024-09-21T06:08:57.560 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_resetxattrs": 0, 2024-09-21T06:08:57.560 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_call": 0, 2024-09-21T06:08:57.560 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_watch": 0, 2024-09-21T06:08:57.560 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_notify": 0, 2024-09-21T06:08:57.560 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T06:08:57.560 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls": 0, 2024-09-21T06:08:57.561 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls_filter": 0, 2024-09-21T06:08:57.561 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_other": 0, 2024-09-21T06:08:57.561 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_active": 0, 2024-09-21T06:08:57.561 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_send": 0, 2024-09-21T06:08:57.561 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_resend": 0, 2024-09-21T06:08:57.561 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_ping": 0, 2024-09-21T06:08:57.561 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_active": 0, 2024-09-21T06:08:57.561 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_send": 0, 2024-09-21T06:08:57.561 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_resend": 0, 2024-09-21T06:08:57.561 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_active": 0, 2024-09-21T06:08:57.561 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_send": 0, 2024-09-21T06:08:57.561 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_resend": 0, 2024-09-21T06:08:57.561 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_active": 0, 2024-09-21T06:08:57.561 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_send": 0, 2024-09-21T06:08:57.561 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_resend": 0, 2024-09-21T06:08:57.562 INFO:teuthology.orchestra.run.smithi138.stdout: "command_active": 0, 2024-09-21T06:08:57.562 INFO:teuthology.orchestra.run.smithi138.stdout: "command_send": 0, 2024-09-21T06:08:57.562 INFO:teuthology.orchestra.run.smithi138.stdout: "command_resend": 0, 2024-09-21T06:08:57.562 INFO:teuthology.orchestra.run.smithi138.stdout: "map_epoch": 85, 2024-09-21T06:08:57.562 INFO:teuthology.orchestra.run.smithi138.stdout: "map_full": 0, 2024-09-21T06:08:57.562 INFO:teuthology.orchestra.run.smithi138.stdout: "map_inc": 11, 2024-09-21T06:08:57.562 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_sessions": 4, 2024-09-21T06:08:57.562 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_open": 4, 2024-09-21T06:08:57.562 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_close": 0, 2024-09-21T06:08:57.562 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_laggy": 0, 2024-09-21T06:08:57.562 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_wr": 0, 2024-09-21T06:08:57.562 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_rd": 2, 2024-09-21T06:08:57.562 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_del": 0 2024-09-21T06:08:57.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.563 INFO:teuthology.orchestra.run.smithi138.stdout: "oft": { 2024-09-21T06:08:57.563 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_objs": 0, 2024-09-21T06:08:57.563 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_kv_pairs": 0, 2024-09-21T06:08:57.563 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_updates": 0, 2024-09-21T06:08:57.563 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_removes": 0 2024-09-21T06:08:57.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.563 INFO:teuthology.orchestra.run.smithi138.stdout: "purge_queue": { 2024-09-21T06:08:57.563 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops": 0, 2024-09-21T06:08:57.563 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T06:08:57.563 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing": 0, 2024-09-21T06:08:57.563 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_high_water": 0, 2024-09-21T06:08:57.563 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed_ops": 0, 2024-09-21T06:08:57.563 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed": 0, 2024-09-21T06:08:57.563 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_item_in_journal": 0 2024-09-21T06:08:57.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.564 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:08:57.564 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-21T06:08:57.564 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-09-21T06:08:57.564 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-21T06:08:57.564 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 2592, 2024-09-21T06:08:57.564 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 1300015, 2024-09-21T06:08:57.564 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-21T06:08:57.564 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 2592, 2024-09-21T06:08:57.564 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-09-21T06:08:57.564 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-09-21T06:08:57.564 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 2592, 2024-09-21T06:08:57.564 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 1300015, 2024-09-21T06:08:57.564 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-21T06:08:57.564 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.564 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.565 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.565 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:08:57.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.565 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_bytes": { 2024-09-21T06:08:57.565 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-21T06:08:57.565 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-09-21T06:08:57.565 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-21T06:08:57.565 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-09-21T06:08:57.565 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-09-21T06:08:57.565 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-21T06:08:57.565 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-09-21T06:08:57.565 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 2338, 2024-09-21T06:08:57.565 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 1125, 2024-09-21T06:08:57.565 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 2338, 2024-09-21T06:08:57.565 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 1125, 2024-09-21T06:08:57.566 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-21T06:08:57.566 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.566 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.566 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.566 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:08:57.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.566 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_ops": { 2024-09-21T06:08:57.566 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-21T06:08:57.566 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1024, 2024-09-21T06:08:57.566 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-21T06:08:57.566 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-09-21T06:08:57.566 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-09-21T06:08:57.566 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-21T06:08:57.566 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-09-21T06:08:57.566 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 2338, 2024-09-21T06:08:57.567 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 2338, 2024-09-21T06:08:57.567 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 2338, 2024-09-21T06:08:57.567 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 2338, 2024-09-21T06:08:57.567 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-21T06:08:57.567 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.567 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.567 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.567 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:08:57.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.567 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle": { 2024-09-21T06:08:57.567 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-21T06:08:57.567 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-09-21T06:08:57.567 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-21T06:08:57.567 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-09-21T06:08:57.568 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-09-21T06:08:57.568 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-21T06:08:57.568 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-09-21T06:08:57.568 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-09-21T06:08:57.568 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-09-21T06:08:57.568 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 0, 2024-09-21T06:08:57.568 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 0, 2024-09-21T06:08:57.568 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-21T06:08:57.568 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.568 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.568 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.568 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:08:57.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.568 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle-0x560cdb6d9380": { 2024-09-21T06:08:57.568 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-21T06:08:57.568 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-09-21T06:08:57.569 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-21T06:08:57.569 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-09-21T06:08:57.569 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-09-21T06:08:57.569 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-21T06:08:57.569 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-09-21T06:08:57.569 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-09-21T06:08:57.569 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-09-21T06:08:57.569 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 0, 2024-09-21T06:08:57.569 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 0, 2024-09-21T06:08:57.569 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-21T06:08:57.569 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:57.569 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:57.569 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:57.569 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:08:57.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:57.570 INFO:teuthology.orchestra.run.smithi138.stdout: "trackedop": { 2024-09-21T06:08:57.570 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_ops_count": 0 2024-09-21T06:08:57.570 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:08:57.570 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-21T06:08:58.115 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:57 smithi138 ceph-mon[34391]: pgmap v685: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:08:58.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:57 smithi192 ceph-mon[34700]: pgmap v685: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:08:58.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:57 smithi117 ceph-mon[30947]: pgmap v685: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-21T06:08:58.426 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-21T06:08:58.427 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-21T06:08:58.427 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-09-21T06:08:58.427 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-09-21T06:08:58.427 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-09-21T06:08:58.427 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-21T06:08:58.427 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T06:08:58.427 DEBUG:tasks.check_counter:Getting stats from h 2024-09-21T06:08:58.427 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph daemon mds.h perf dump 2024-09-21T06:08:58.680 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:08:59.273 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-21T06:08:59.273 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:08:59.273 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 1496, 2024-09-21T06:08:59.273 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 1010, 2024-09-21T06:08:59.273 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 433425, 2024-09-21T06:08:59.273 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 746328, 2024-09-21T06:08:59.273 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 34, 2024-09-21T06:08:59.273 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 5, 2024-09-21T06:08:59.273 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.149973611, 2024-09-21T06:08:59.273 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.066238614, 2024-09-21T06:08:59.273 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.095961463, 2024-09-21T06:08:59.273 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.004728974, 2024-09-21T06:08:59.274 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:08:59.274 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 1009, 2024-09-21T06:08:59.274 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.079502665, 2024-09-21T06:08:59.274 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000078793 2024-09-21T06:08:59.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.274 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:08:59.274 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2261, 2024-09-21T06:08:59.274 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000558287, 2024-09-21T06:08:59.274 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000246 2024-09-21T06:08:59.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.274 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-21T06:08:59.274 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T06:08:59.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.274 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:08:59.274 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 368, 2024-09-21T06:08:59.275 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 373, 2024-09-21T06:08:59.275 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 930326, 2024-09-21T06:08:59.275 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 242206, 2024-09-21T06:08:59.275 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 13, 2024-09-21T06:08:59.275 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 6, 2024-09-21T06:08:59.275 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.066171113, 2024-09-21T06:08:59.275 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.029148889, 2024-09-21T06:08:59.275 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.047718041, 2024-09-21T06:08:59.275 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.000304828, 2024-09-21T06:08:59.275 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:08:59.275 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 372, 2024-09-21T06:08:59.275 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.034610365, 2024-09-21T06:08:59.275 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000093038 2024-09-21T06:08:59.275 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.276 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:08:59.276 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 147, 2024-09-21T06:08:59.276 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000041580, 2024-09-21T06:08:59.276 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000282 2024-09-21T06:08:59.276 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.276 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 895216, 2024-09-21T06:08:59.276 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 147632 2024-09-21T06:08:59.276 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.276 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:08:59.276 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 88, 2024-09-21T06:08:59.276 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 275, 2024-09-21T06:08:59.276 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 36624, 2024-09-21T06:08:59.276 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 456982, 2024-09-21T06:08:59.276 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 11, 2024-09-21T06:08:59.276 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 6, 2024-09-21T06:08:59.277 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.025648275, 2024-09-21T06:08:59.277 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.014651181, 2024-09-21T06:08:59.277 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.008324725, 2024-09-21T06:08:59.277 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.000432357, 2024-09-21T06:08:59.277 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:08:59.277 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 273, 2024-09-21T06:08:59.277 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.030544656, 2024-09-21T06:08:59.277 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000111885 2024-09-21T06:08:59.277 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.277 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:08:59.277 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 145, 2024-09-21T06:08:59.277 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000036541, 2024-09-21T06:08:59.277 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000252 2024-09-21T06:08:59.277 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.277 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 1888, 2024-09-21T06:08:59.278 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 394608 2024-09-21T06:08:59.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.278 INFO:teuthology.orchestra.run.smithi138.stdout: "cct": { 2024-09-21T06:08:59.278 INFO:teuthology.orchestra.run.smithi138.stdout: "total_workers": 1, 2024-09-21T06:08:59.278 INFO:teuthology.orchestra.run.smithi138.stdout: "unhealthy_workers": 0 2024-09-21T06:08:59.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.278 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-MDSRank": { 2024-09-21T06:08:59.278 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-09-21T06:08:59.278 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-09-21T06:08:59.278 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 45, 2024-09-21T06:08:59.278 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.002346274, 2024-09-21T06:08:59.278 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000052139 2024-09-21T06:08:59.278 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:08:59.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.278 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-PurgeQueue": { 2024-09-21T06:08:59.279 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-09-21T06:08:59.279 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-09-21T06:08:59.279 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2, 2024-09-21T06:08:59.279 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000031978, 2024-09-21T06:08:59.279 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000015989 2024-09-21T06:08:59.279 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:08:59.279 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.279 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": { 2024-09-21T06:08:59.279 INFO:teuthology.orchestra.run.smithi138.stdout: "request": 1, 2024-09-21T06:08:59.279 INFO:teuthology.orchestra.run.smithi138.stdout: "reply": 0, 2024-09-21T06:08:59.279 INFO:teuthology.orchestra.run.smithi138.stdout: "reply_latency": { 2024-09-21T06:08:59.279 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.279 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.279 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.280 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.280 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_reply": 0, 2024-09-21T06:08:59.280 INFO:teuthology.orchestra.run.smithi138.stdout: "forward": 1, 2024-09-21T06:08:59.280 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_complete": 0, 2024-09-21T06:08:59.280 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_keys": 0, 2024-09-21T06:08:59.280 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_commit": 11, 2024-09-21T06:08:59.280 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_split": 0, 2024-09-21T06:08:59.280 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_merge": 0, 2024-09-21T06:08:59.280 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes": 16, 2024-09-21T06:08:59.280 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_top": 0, 2024-09-21T06:08:59.280 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_bottom": 16, 2024-09-21T06:08:59.280 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pin_tail": 0, 2024-09-21T06:08:59.280 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pinned": 13, 2024-09-21T06:08:59.280 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_expired": 8, 2024-09-21T06:08:59.281 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_with_caps": 0, 2024-09-21T06:08:59.281 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": 0, 2024-09-21T06:08:59.281 INFO:teuthology.orchestra.run.smithi138.stdout: "subtrees": 3, 2024-09-21T06:08:59.281 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse": 10, 2024-09-21T06:08:59.281 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_hit": 5, 2024-09-21T06:08:59.281 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_forward": 1, 2024-09-21T06:08:59.281 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_discover": 4, 2024-09-21T06:08:59.281 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_dir_fetch": 0, 2024-09-21T06:08:59.281 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_remote_ino": 0, 2024-09-21T06:08:59.281 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_lock": 0, 2024-09-21T06:08:59.281 INFO:teuthology.orchestra.run.smithi138.stdout: "load_cent": 0, 2024-09-21T06:08:59.281 INFO:teuthology.orchestra.run.smithi138.stdout: "q": 0, 2024-09-21T06:08:59.281 INFO:teuthology.orchestra.run.smithi138.stdout: "exported": 0, 2024-09-21T06:08:59.281 INFO:teuthology.orchestra.run.smithi138.stdout: "exported_inodes": 0, 2024-09-21T06:08:59.281 INFO:teuthology.orchestra.run.smithi138.stdout: "imported": 0, 2024-09-21T06:08:59.282 INFO:teuthology.orchestra.run.smithi138.stdout: "imported_inodes": 0, 2024-09-21T06:08:59.282 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_dir_fetch": 0, 2024-09-21T06:08:59.282 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_backtrace_fetch": 0, 2024-09-21T06:08:59.282 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_peer_discover": 0, 2024-09-21T06:08:59.282 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rfiles": 22, 2024-09-21T06:08:59.282 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rbytes": 10738142506, 2024-09-21T06:08:59.282 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rsnaps": 7, 2024-09-21T06:08:59.282 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T06:08:59.282 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_set_tag": 0, 2024-09-21T06:08:59.282 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T06:08:59.282 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_inotable_repaired": 0, 2024-09-21T06:08:59.282 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_inodes": 0, 2024-09-21T06:08:59.282 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T06:08:59.282 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T06:08:59.282 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_file_inodes": 0, 2024-09-21T06:08:59.283 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_inode_file_caps": 0, 2024-09-21T06:08:59.283 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T06:08:59.283 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_grant": 0, 2024-09-21T06:08:59.283 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T06:08:59.283 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T06:08:59.283 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T06:08:59.283 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps": 0, 2024-09-21T06:08:59.283 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps_dirty": 0, 2024-09-21T06:08:59.283 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_cap_release": 0, 2024-09-21T06:08:59.283 INFO:teuthology.orchestra.run.smithi138.stdout: "process_request_cap_release": 0 2024-09-21T06:08:59.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.283 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cache": { 2024-09-21T06:08:59.283 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update": 0, 2024-09-21T06:08:59.283 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update_receipt": 14, 2024-09-21T06:08:59.283 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_try_discover": 9, 2024-09-21T06:08:59.283 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_send_discover": 10, 2024-09-21T06:08:59.284 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_handle_discover": 0, 2024-09-21T06:08:59.284 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays": 0, 2024-09-21T06:08:59.284 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_delayed": 0, 2024-09-21T06:08:59.284 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_enqueuing": 0, 2024-09-21T06:08:59.284 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_created": 0, 2024-09-21T06:08:59.284 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_enqueued": 0, 2024-09-21T06:08:59.284 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_reintegrated": 0, 2024-09-21T06:08:59.284 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_migrated": 0, 2024-09-21T06:08:59.284 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_processing": 0, 2024-09-21T06:08:59.284 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_enqueued": 0, 2024-09-21T06:08:59.284 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_prioritized": 0, 2024-09-21T06:08:59.284 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_started": 0, 2024-09-21T06:08:59.284 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_completed": 0, 2024-09-21T06:08:59.284 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_path": 0, 2024-09-21T06:08:59.284 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_inode": 0, 2024-09-21T06:08:59.284 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T06:08:59.285 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_exportdir": 0, 2024-09-21T06:08:59.285 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_flush": 0, 2024-09-21T06:08:59.285 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragmentdir": 0, 2024-09-21T06:08:59.285 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragstats": 0, 2024-09-21T06:08:59.285 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_inodestats": 0, 2024-09-21T06:08:59.285 INFO:teuthology.orchestra.run.smithi138.stdout: "uninline_started": 0, 2024-09-21T06:08:59.285 INFO:teuthology.orchestra.run.smithi138.stdout: "uninline_succeeded": 0, 2024-09-21T06:08:59.285 INFO:teuthology.orchestra.run.smithi138.stdout: "uninline_write_failed": 0 2024-09-21T06:08:59.285 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.285 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_log": { 2024-09-21T06:08:59.285 INFO:teuthology.orchestra.run.smithi138.stdout: "evlrg": 0, 2024-09-21T06:08:59.285 INFO:teuthology.orchestra.run.smithi138.stdout: "evadd": 4, 2024-09-21T06:08:59.285 INFO:teuthology.orchestra.run.smithi138.stdout: "evex": 0, 2024-09-21T06:08:59.285 INFO:teuthology.orchestra.run.smithi138.stdout: "evtrm": 0, 2024-09-21T06:08:59.285 INFO:teuthology.orchestra.run.smithi138.stdout: "ev": 4, 2024-09-21T06:08:59.285 INFO:teuthology.orchestra.run.smithi138.stdout: "evexg": 0, 2024-09-21T06:08:59.286 INFO:teuthology.orchestra.run.smithi138.stdout: "evexd": 0, 2024-09-21T06:08:59.286 INFO:teuthology.orchestra.run.smithi138.stdout: "segadd": 2, 2024-09-21T06:08:59.286 INFO:teuthology.orchestra.run.smithi138.stdout: "segex": 0, 2024-09-21T06:08:59.286 INFO:teuthology.orchestra.run.smithi138.stdout: "segtrm": 0, 2024-09-21T06:08:59.286 INFO:teuthology.orchestra.run.smithi138.stdout: "seg": 2, 2024-09-21T06:08:59.286 INFO:teuthology.orchestra.run.smithi138.stdout: "segmjr": 2, 2024-09-21T06:08:59.286 INFO:teuthology.orchestra.run.smithi138.stdout: "segexg": 0, 2024-09-21T06:08:59.286 INFO:teuthology.orchestra.run.smithi138.stdout: "segexd": 0, 2024-09-21T06:08:59.286 INFO:teuthology.orchestra.run.smithi138.stdout: "expos": 4194304, 2024-09-21T06:08:59.286 INFO:teuthology.orchestra.run.smithi138.stdout: "wrpos": 4195591, 2024-09-21T06:08:59.286 INFO:teuthology.orchestra.run.smithi138.stdout: "rdpos": 0, 2024-09-21T06:08:59.286 INFO:teuthology.orchestra.run.smithi138.stdout: "jlat": { 2024-09-21T06:08:59.286 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 3, 2024-09-21T06:08:59.286 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.005043658, 2024-09-21T06:08:59.286 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.001681219 2024-09-21T06:08:59.287 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.287 INFO:teuthology.orchestra.run.smithi138.stdout: "replayed": 0 2024-09-21T06:08:59.287 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.287 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_mem": { 2024-09-21T06:08:59.287 INFO:teuthology.orchestra.run.smithi138.stdout: "ino": 20, 2024-09-21T06:08:59.287 INFO:teuthology.orchestra.run.smithi138.stdout: "ino+": 24, 2024-09-21T06:08:59.287 INFO:teuthology.orchestra.run.smithi138.stdout: "ino-": 4, 2024-09-21T06:08:59.287 INFO:teuthology.orchestra.run.smithi138.stdout: "dir": 19, 2024-09-21T06:08:59.287 INFO:teuthology.orchestra.run.smithi138.stdout: "dir+": 25, 2024-09-21T06:08:59.287 INFO:teuthology.orchestra.run.smithi138.stdout: "dir-": 6, 2024-09-21T06:08:59.287 INFO:teuthology.orchestra.run.smithi138.stdout: "dn": 16, 2024-09-21T06:08:59.287 INFO:teuthology.orchestra.run.smithi138.stdout: "dn+": 24, 2024-09-21T06:08:59.287 INFO:teuthology.orchestra.run.smithi138.stdout: "dn-": 8, 2024-09-21T06:08:59.287 INFO:teuthology.orchestra.run.smithi138.stdout: "cap": 0, 2024-09-21T06:08:59.287 INFO:teuthology.orchestra.run.smithi138.stdout: "cap+": 0, 2024-09-21T06:08:59.287 INFO:teuthology.orchestra.run.smithi138.stdout: "cap-": 0, 2024-09-21T06:08:59.287 INFO:teuthology.orchestra.run.smithi138.stdout: "rss": 118804, 2024-09-21T06:08:59.288 INFO:teuthology.orchestra.run.smithi138.stdout: "heap": 325857280 2024-09-21T06:08:59.288 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.288 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_server": { 2024-09-21T06:08:59.288 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_client_request": 1, 2024-09-21T06:08:59.288 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_server_request": 0, 2024-09-21T06:08:59.288 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_request": 1, 2024-09-21T06:08:59.288 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_session": 19, 2024-09-21T06:08:59.288 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_peer_request": 0, 2024-09-21T06:08:59.288 INFO:teuthology.orchestra.run.smithi138.stdout: "req_create_latency": { 2024-09-21T06:08:59.288 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.288 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.288 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.288 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.288 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getattr_latency": { 2024-09-21T06:08:59.288 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.288 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.289 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.289 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.289 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getfilelock_latency": { 2024-09-21T06:08:59.289 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.289 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.289 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.289 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.289 INFO:teuthology.orchestra.run.smithi138.stdout: "req_link_latency": { 2024-09-21T06:08:59.289 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.289 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.289 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.289 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.289 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookup_latency": { 2024-09-21T06:08:59.289 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.289 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.289 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.290 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.290 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookuphash_latency": { 2024-09-21T06:08:59.290 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.290 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.290 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.290 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.290 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupino_latency": { 2024-09-21T06:08:59.290 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.290 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.290 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.290 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.290 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupname_latency": { 2024-09-21T06:08:59.290 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.290 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.291 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.291 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.291 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupparent_latency": { 2024-09-21T06:08:59.291 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.291 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.291 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.291 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.291 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupsnap_latency": { 2024-09-21T06:08:59.291 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.291 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.291 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.291 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.291 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lssnap_latency": { 2024-09-21T06:08:59.291 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.292 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.292 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.292 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.292 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mkdir_latency": { 2024-09-21T06:08:59.292 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.292 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.292 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.292 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.292 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mknod_latency": { 2024-09-21T06:08:59.292 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.293 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.293 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.293 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.293 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mksnap_latency": { 2024-09-21T06:08:59.293 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.293 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.293 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.293 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.293 INFO:teuthology.orchestra.run.smithi138.stdout: "req_open_latency": { 2024-09-21T06:08:59.293 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.293 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.293 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.293 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.293 INFO:teuthology.orchestra.run.smithi138.stdout: "req_readdir_latency": { 2024-09-21T06:08:59.293 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.293 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.294 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.294 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.294 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rename_latency": { 2024-09-21T06:08:59.294 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.294 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.294 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.294 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.294 INFO:teuthology.orchestra.run.smithi138.stdout: "req_renamesnap_latency": { 2024-09-21T06:08:59.294 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.294 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.294 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.294 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.294 INFO:teuthology.orchestra.run.smithi138.stdout: "req_snapdiff_latency": { 2024-09-21T06:08:59.294 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.294 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.294 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.295 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.295 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmdir_latency": { 2024-09-21T06:08:59.295 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.295 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.295 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.295 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.295 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmsnap_latency": { 2024-09-21T06:08:59.295 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.295 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.295 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.295 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.295 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmxattr_latency": { 2024-09-21T06:08:59.295 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.295 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.295 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.295 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.296 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setattr_latency": { 2024-09-21T06:08:59.296 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.296 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.296 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.296 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.296 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setdirlayout_latency": { 2024-09-21T06:08:59.296 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.296 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.296 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.296 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.296 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setfilelock_latency": { 2024-09-21T06:08:59.296 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.296 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.296 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.296 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.296 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setlayout_latency": { 2024-09-21T06:08:59.297 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.297 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.297 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.297 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.297 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setxattr_latency": { 2024-09-21T06:08:59.297 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.297 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.297 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.297 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.297 INFO:teuthology.orchestra.run.smithi138.stdout: "req_symlink_latency": { 2024-09-21T06:08:59.297 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.297 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.297 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.297 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.297 INFO:teuthology.orchestra.run.smithi138.stdout: "req_unlink_latency": { 2024-09-21T06:08:59.297 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.298 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.298 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.298 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.298 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_revoke_eviction": 0, 2024-09-21T06:08:59.298 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_acquisition_throttle": 0, 2024-09-21T06:08:59.298 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getvxattr_latency": { 2024-09-21T06:08:59.298 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.298 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.298 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.298 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:08:59.298 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.298 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_sessions": { 2024-09-21T06:08:59.298 INFO:teuthology.orchestra.run.smithi138.stdout: "session_count": 2, 2024-09-21T06:08:59.298 INFO:teuthology.orchestra.run.smithi138.stdout: "session_add": 2, 2024-09-21T06:08:59.298 INFO:teuthology.orchestra.run.smithi138.stdout: "session_remove": 0, 2024-09-21T06:08:59.298 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_open": 2, 2024-09-21T06:08:59.299 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_stale": 0, 2024-09-21T06:08:59.299 INFO:teuthology.orchestra.run.smithi138.stdout: "total_load": 0, 2024-09-21T06:08:59.299 INFO:teuthology.orchestra.run.smithi138.stdout: "average_load": 0, 2024-09-21T06:08:59.299 INFO:teuthology.orchestra.run.smithi138.stdout: "avg_session_uptime": 683, 2024-09-21T06:08:59.299 INFO:teuthology.orchestra.run.smithi138.stdout: "mdthresh_evicted": 0 2024-09-21T06:08:59.299 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.299 INFO:teuthology.orchestra.run.smithi138.stdout: "mempool": { 2024-09-21T06:08:59.299 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:08:59.299 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_items": 0, 2024-09-21T06:08:59.299 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:08:59.299 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:08:59.299 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:08:59.299 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:08:59.299 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:08:59.300 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:08:59.300 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:08:59.300 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:08:59.300 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:08:59.300 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:08:59.300 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:08:59.300 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:08:59.300 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:08:59.300 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_items": 0, 2024-09-21T06:08:59.300 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:08:59.300 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_items": 0, 2024-09-21T06:08:59.300 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:08:59.300 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:08:59.300 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:08:59.300 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:08:59.300 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:08:59.301 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:08:59.301 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:08:59.301 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_items": 0, 2024-09-21T06:08:59.301 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:08:59.301 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:08:59.301 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:08:59.301 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_items": 0, 2024-09-21T06:08:59.301 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_bytes": 0, 2024-09-21T06:08:59.301 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_items": 0, 2024-09-21T06:08:59.301 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:08:59.301 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:08:59.301 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:08:59.301 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:08:59.301 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_bytes": 270743, 2024-09-21T06:08:59.301 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_items": 94, 2024-09-21T06:08:59.302 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_bytes": 0, 2024-09-21T06:08:59.302 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_items": 0, 2024-09-21T06:08:59.302 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bytes": 0, 2024-09-21T06:08:59.302 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_items": 0, 2024-09-21T06:08:59.302 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:08:59.302 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_items": 0, 2024-09-21T06:08:59.302 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:08:59.302 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_items": 0, 2024-09-21T06:08:59.302 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_bytes": 7968, 2024-09-21T06:08:59.302 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_items": 176, 2024-09-21T06:08:59.302 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:08:59.302 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:08:59.302 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_bytes": 0, 2024-09-21T06:08:59.302 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_items": 0, 2024-09-21T06:08:59.302 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_bytes": 83493, 2024-09-21T06:08:59.303 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_items": 386, 2024-09-21T06:08:59.303 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_bytes": 0, 2024-09-21T06:08:59.303 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_items": 0, 2024-09-21T06:08:59.303 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_bytes": 0, 2024-09-21T06:08:59.303 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_items": 0 2024-09-21T06:08:59.303 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.303 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter": { 2024-09-21T06:08:59.303 INFO:teuthology.orchestra.run.smithi138.stdout: "op_active": 0, 2024-09-21T06:08:59.303 INFO:teuthology.orchestra.run.smithi138.stdout: "op_laggy": 0, 2024-09-21T06:08:59.303 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send": 39, 2024-09-21T06:08:59.303 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send_bytes": 11589, 2024-09-21T06:08:59.303 INFO:teuthology.orchestra.run.smithi138.stdout: "op_resend": 33, 2024-09-21T06:08:59.303 INFO:teuthology.orchestra.run.smithi138.stdout: "op_reply": 39, 2024-09-21T06:08:59.303 INFO:teuthology.orchestra.run.smithi138.stdout: "op_latency": { 2024-09-21T06:08:59.303 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 39, 2024-09-21T06:08:59.304 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.227995078, 2024-09-21T06:08:59.304 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.005846027 2024-09-21T06:08:59.304 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.304 INFO:teuthology.orchestra.run.smithi138.stdout: "op_inflight": 0, 2024-09-21T06:08:59.304 INFO:teuthology.orchestra.run.smithi138.stdout: "oplen_avg": { 2024-09-21T06:08:59.304 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 39, 2024-09-21T06:08:59.304 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 61 2024-09-21T06:08:59.304 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.304 INFO:teuthology.orchestra.run.smithi138.stdout: "op": 39, 2024-09-21T06:08:59.304 INFO:teuthology.orchestra.run.smithi138.stdout: "op_r": 0, 2024-09-21T06:08:59.304 INFO:teuthology.orchestra.run.smithi138.stdout: "op_w": 39, 2024-09-21T06:08:59.304 INFO:teuthology.orchestra.run.smithi138.stdout: "op_rmw": 0, 2024-09-21T06:08:59.304 INFO:teuthology.orchestra.run.smithi138.stdout: "op_pg": 0, 2024-09-21T06:08:59.304 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_stat": 0, 2024-09-21T06:08:59.304 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_create": 10, 2024-09-21T06:08:59.304 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_read": 0, 2024-09-21T06:08:59.305 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_write": 3, 2024-09-21T06:08:59.305 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writefull": 7, 2024-09-21T06:08:59.305 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writesame": 0, 2024-09-21T06:08:59.305 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_append": 0, 2024-09-21T06:08:59.305 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_zero": 0, 2024-09-21T06:08:59.305 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_truncate": 0, 2024-09-21T06:08:59.305 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_delete": 6, 2024-09-21T06:08:59.305 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_mapext": 0, 2024-09-21T06:08:59.305 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_sparse_read": 0, 2024-09-21T06:08:59.305 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_clonerange": 0, 2024-09-21T06:08:59.305 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_getxattr": 0, 2024-09-21T06:08:59.305 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_setxattr": 20, 2024-09-21T06:08:59.305 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_cmpxattr": 0, 2024-09-21T06:08:59.305 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_rmxattr": 0, 2024-09-21T06:08:59.305 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_resetxattrs": 0, 2024-09-21T06:08:59.305 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_call": 0, 2024-09-21T06:08:59.306 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_watch": 0, 2024-09-21T06:08:59.306 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_notify": 0, 2024-09-21T06:08:59.306 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T06:08:59.306 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls": 0, 2024-09-21T06:08:59.306 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls_filter": 0, 2024-09-21T06:08:59.306 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_other": 0, 2024-09-21T06:08:59.306 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_active": 0, 2024-09-21T06:08:59.306 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_send": 0, 2024-09-21T06:08:59.306 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_resend": 0, 2024-09-21T06:08:59.306 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_ping": 0, 2024-09-21T06:08:59.306 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_active": 0, 2024-09-21T06:08:59.306 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_send": 0, 2024-09-21T06:08:59.306 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_resend": 0, 2024-09-21T06:08:59.306 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_active": 0, 2024-09-21T06:08:59.306 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_send": 0, 2024-09-21T06:08:59.307 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_resend": 0, 2024-09-21T06:08:59.307 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_active": 0, 2024-09-21T06:08:59.307 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_send": 0, 2024-09-21T06:08:59.307 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_resend": 0, 2024-09-21T06:08:59.307 INFO:teuthology.orchestra.run.smithi138.stdout: "command_active": 0, 2024-09-21T06:08:59.307 INFO:teuthology.orchestra.run.smithi138.stdout: "command_send": 0, 2024-09-21T06:08:59.307 INFO:teuthology.orchestra.run.smithi138.stdout: "command_resend": 0, 2024-09-21T06:08:59.307 INFO:teuthology.orchestra.run.smithi138.stdout: "map_epoch": 85, 2024-09-21T06:08:59.307 INFO:teuthology.orchestra.run.smithi138.stdout: "map_full": 0, 2024-09-21T06:08:59.307 INFO:teuthology.orchestra.run.smithi138.stdout: "map_inc": 11, 2024-09-21T06:08:59.307 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_sessions": 9, 2024-09-21T06:08:59.307 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_open": 9, 2024-09-21T06:08:59.307 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_close": 0, 2024-09-21T06:08:59.307 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_laggy": 0, 2024-09-21T06:08:59.307 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_wr": 14, 2024-09-21T06:08:59.308 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_rd": 0, 2024-09-21T06:08:59.308 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_del": 1 2024-09-21T06:08:59.308 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.308 INFO:teuthology.orchestra.run.smithi138.stdout: "oft": { 2024-09-21T06:08:59.308 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_objs": 1, 2024-09-21T06:08:59.308 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_kv_pairs": 0, 2024-09-21T06:08:59.308 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_updates": 0, 2024-09-21T06:08:59.308 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_removes": 0 2024-09-21T06:08:59.308 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.308 INFO:teuthology.orchestra.run.smithi138.stdout: "purge_queue": { 2024-09-21T06:08:59.308 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops": 0, 2024-09-21T06:08:59.308 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T06:08:59.308 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing": 0, 2024-09-21T06:08:59.308 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_high_water": 0, 2024-09-21T06:08:59.309 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed_ops": 0, 2024-09-21T06:08:59.309 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed": 0, 2024-09-21T06:08:59.309 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_item_in_journal": 0 2024-09-21T06:08:59.309 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.309 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:08:59.309 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-21T06:08:59.309 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-09-21T06:08:59.309 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-21T06:08:59.309 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 1948, 2024-09-21T06:08:59.309 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 1207341, 2024-09-21T06:08:59.309 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-21T06:08:59.309 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 1948, 2024-09-21T06:08:59.309 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-09-21T06:08:59.309 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-09-21T06:08:59.309 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 1948, 2024-09-21T06:08:59.309 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 1207341, 2024-09-21T06:08:59.310 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-21T06:08:59.310 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.310 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.310 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.310 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:08:59.310 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.310 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_bytes": { 2024-09-21T06:08:59.310 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-21T06:08:59.310 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-09-21T06:08:59.310 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-21T06:08:59.310 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-09-21T06:08:59.310 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-09-21T06:08:59.310 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-21T06:08:59.310 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-09-21T06:08:59.310 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 39, 2024-09-21T06:08:59.310 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 11589, 2024-09-21T06:08:59.311 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 39, 2024-09-21T06:08:59.311 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 11589, 2024-09-21T06:08:59.311 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-21T06:08:59.311 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.311 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.311 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.311 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:08:59.311 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.311 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_ops": { 2024-09-21T06:08:59.311 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-21T06:08:59.311 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1024, 2024-09-21T06:08:59.311 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-21T06:08:59.311 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-09-21T06:08:59.311 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-09-21T06:08:59.311 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-21T06:08:59.311 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-09-21T06:08:59.312 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 39, 2024-09-21T06:08:59.312 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 39, 2024-09-21T06:08:59.312 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 39, 2024-09-21T06:08:59.312 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 39, 2024-09-21T06:08:59.312 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-21T06:08:59.312 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.312 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.312 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.312 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:08:59.312 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.312 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle": { 2024-09-21T06:08:59.312 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-21T06:08:59.312 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-09-21T06:08:59.312 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-21T06:08:59.312 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-09-21T06:08:59.313 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-09-21T06:08:59.313 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-21T06:08:59.313 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-09-21T06:08:59.313 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-09-21T06:08:59.313 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-09-21T06:08:59.313 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 0, 2024-09-21T06:08:59.313 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 0, 2024-09-21T06:08:59.313 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-21T06:08:59.313 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.313 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.313 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.313 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:08:59.313 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.313 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle-0x55f65e905520": { 2024-09-21T06:08:59.313 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-21T06:08:59.313 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-09-21T06:08:59.314 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-21T06:08:59.314 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 4, 2024-09-21T06:08:59.314 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 1287, 2024-09-21T06:08:59.314 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-21T06:08:59.314 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 4, 2024-09-21T06:08:59.314 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-09-21T06:08:59.314 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-09-21T06:08:59.314 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 3, 2024-09-21T06:08:59.314 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 1287, 2024-09-21T06:08:59.314 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-21T06:08:59.314 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:08:59.314 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:08:59.314 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:08:59.314 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:08:59.315 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:08:59.315 INFO:teuthology.orchestra.run.smithi138.stdout: "trackedop": { 2024-09-21T06:08:59.315 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_ops_count": 0 2024-09-21T06:08:59.315 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:08:59.315 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-21T06:09:00.052 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-21T06:09:00.053 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-21T06:09:00.053 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-09-21T06:09:00.053 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=14 2024-09-21T06:09:00.053 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=7 2024-09-21T06:09:00.053 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-21T06:09:00.053 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T06:09:00.053 DEBUG:tasks.check_counter:Getting stats from k 2024-09-21T06:09:00.053 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph daemon mds.k perf dump 2024-09-21T06:09:00.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:08:59 smithi192 ceph-mon[34700]: pgmap v686: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:09:00.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:08:59 smithi117 ceph-mon[30947]: pgmap v686: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:09:00.303 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T06:09:00.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:08:59 smithi138 ceph-mon[34391]: pgmap v686: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-21T06:09:00.842 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-21T06:09:00.842 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:09:00.842 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 800, 2024-09-21T06:09:00.842 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 802, 2024-09-21T06:09:00.843 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 211603, 2024-09-21T06:09:00.843 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 254464, 2024-09-21T06:09:00.843 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 6, 2024-09-21T06:09:00.843 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 2, 2024-09-21T06:09:00.843 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.098952812, 2024-09-21T06:09:00.843 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.045658907, 2024-09-21T06:09:00.843 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.021140408, 2024-09-21T06:09:00.843 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.024491560, 2024-09-21T06:09:00.843 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:09:00.844 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 800, 2024-09-21T06:09:00.844 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.041320411, 2024-09-21T06:09:00.844 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000051650 2024-09-21T06:09:00.844 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.844 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:09:00.844 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.844 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.844 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.844 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.844 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T06:09:00.844 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T06:09:00.844 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.844 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:09:00.845 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 699, 2024-09-21T06:09:00.845 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 875, 2024-09-21T06:09:00.845 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 171890, 2024-09-21T06:09:00.845 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 611633, 2024-09-21T06:09:00.845 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 5, 2024-09-21T06:09:00.845 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 4, 2024-09-21T06:09:00.845 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.104458062, 2024-09-21T06:09:00.845 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.043419377, 2024-09-21T06:09:00.845 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.019868932, 2024-09-21T06:09:00.845 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.033525992, 2024-09-21T06:09:00.845 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:09:00.845 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 873, 2024-09-21T06:09:00.845 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.053309283, 2024-09-21T06:09:00.845 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000061064 2024-09-21T06:09:00.846 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.846 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:09:00.846 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.846 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.846 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.846 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.846 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T06:09:00.846 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 389664 2024-09-21T06:09:00.846 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.846 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:09:00.846 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 1053, 2024-09-21T06:09:00.846 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 1043, 2024-09-21T06:09:00.846 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 1153632, 2024-09-21T06:09:00.847 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 398174, 2024-09-21T06:09:00.847 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 3, 2024-09-21T06:09:00.847 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 2, 2024-09-21T06:09:00.847 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.174176691, 2024-09-21T06:09:00.847 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.072072842, 2024-09-21T06:09:00.847 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.067387514, 2024-09-21T06:09:00.847 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.036080512, 2024-09-21T06:09:00.847 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:09:00.847 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 1043, 2024-09-21T06:09:00.847 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.076028328, 2024-09-21T06:09:00.847 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000072893 2024-09-21T06:09:00.847 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.847 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:09:00.847 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.848 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.848 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.848 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.848 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 887184, 2024-09-21T06:09:00.848 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 146000 2024-09-21T06:09:00.848 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.848 INFO:teuthology.orchestra.run.smithi138.stdout: "cct": { 2024-09-21T06:09:00.848 INFO:teuthology.orchestra.run.smithi138.stdout: "total_workers": 1, 2024-09-21T06:09:00.848 INFO:teuthology.orchestra.run.smithi138.stdout: "unhealthy_workers": 0 2024-09-21T06:09:00.848 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.848 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-MDSRank": { 2024-09-21T06:09:00.848 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-09-21T06:09:00.848 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-09-21T06:09:00.848 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 3824, 2024-09-21T06:09:00.848 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.135469631, 2024-09-21T06:09:00.849 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000035426 2024-09-21T06:09:00.849 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:09:00.849 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.849 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-PurgeQueue": { 2024-09-21T06:09:00.849 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-09-21T06:09:00.849 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-09-21T06:09:00.849 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.849 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.849 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.849 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:09:00.849 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.849 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": { 2024-09-21T06:09:00.849 INFO:teuthology.orchestra.run.smithi138.stdout: "request": 0, 2024-09-21T06:09:00.849 INFO:teuthology.orchestra.run.smithi138.stdout: "reply": 0, 2024-09-21T06:09:00.849 INFO:teuthology.orchestra.run.smithi138.stdout: "reply_latency": { 2024-09-21T06:09:00.850 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.850 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.850 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.850 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.850 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_reply": 0, 2024-09-21T06:09:00.850 INFO:teuthology.orchestra.run.smithi138.stdout: "forward": 0, 2024-09-21T06:09:00.850 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_complete": 0, 2024-09-21T06:09:00.850 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_keys": 0, 2024-09-21T06:09:00.850 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_commit": 0, 2024-09-21T06:09:00.850 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_split": 0, 2024-09-21T06:09:00.850 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_merge": 0, 2024-09-21T06:09:00.850 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes": 0, 2024-09-21T06:09:00.850 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_top": 0, 2024-09-21T06:09:00.850 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_bottom": 0, 2024-09-21T06:09:00.850 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pin_tail": 0, 2024-09-21T06:09:00.850 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pinned": 0, 2024-09-21T06:09:00.851 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_expired": 0, 2024-09-21T06:09:00.851 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_with_caps": 0, 2024-09-21T06:09:00.851 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": 0, 2024-09-21T06:09:00.851 INFO:teuthology.orchestra.run.smithi138.stdout: "subtrees": 1, 2024-09-21T06:09:00.851 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse": 0, 2024-09-21T06:09:00.851 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_hit": 0, 2024-09-21T06:09:00.851 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_forward": 0, 2024-09-21T06:09:00.851 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_discover": 0, 2024-09-21T06:09:00.851 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_dir_fetch": 0, 2024-09-21T06:09:00.851 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_remote_ino": 0, 2024-09-21T06:09:00.851 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_lock": 0, 2024-09-21T06:09:00.851 INFO:teuthology.orchestra.run.smithi138.stdout: "load_cent": 0, 2024-09-21T06:09:00.851 INFO:teuthology.orchestra.run.smithi138.stdout: "q": 0, 2024-09-21T06:09:00.851 INFO:teuthology.orchestra.run.smithi138.stdout: "exported": 0, 2024-09-21T06:09:00.852 INFO:teuthology.orchestra.run.smithi138.stdout: "exported_inodes": 0, 2024-09-21T06:09:00.852 INFO:teuthology.orchestra.run.smithi138.stdout: "imported": 0, 2024-09-21T06:09:00.852 INFO:teuthology.orchestra.run.smithi138.stdout: "imported_inodes": 0, 2024-09-21T06:09:00.852 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_dir_fetch": 0, 2024-09-21T06:09:00.852 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_backtrace_fetch": 0, 2024-09-21T06:09:00.852 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_peer_discover": 0, 2024-09-21T06:09:00.852 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rfiles": 0, 2024-09-21T06:09:00.852 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rbytes": 0, 2024-09-21T06:09:00.852 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rsnaps": 0, 2024-09-21T06:09:00.852 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T06:09:00.852 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_set_tag": 0, 2024-09-21T06:09:00.852 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T06:09:00.852 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_inotable_repaired": 0, 2024-09-21T06:09:00.852 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_inodes": 0, 2024-09-21T06:09:00.852 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T06:09:00.852 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T06:09:00.853 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_file_inodes": 0, 2024-09-21T06:09:00.853 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_inode_file_caps": 0, 2024-09-21T06:09:00.853 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T06:09:00.853 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_grant": 0, 2024-09-21T06:09:00.853 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T06:09:00.853 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T06:09:00.853 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T06:09:00.853 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps": 0, 2024-09-21T06:09:00.853 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps_dirty": 0, 2024-09-21T06:09:00.853 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_cap_release": 0, 2024-09-21T06:09:00.853 INFO:teuthology.orchestra.run.smithi138.stdout: "process_request_cap_release": 0 2024-09-21T06:09:00.853 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.853 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cache": { 2024-09-21T06:09:00.853 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update": 0, 2024-09-21T06:09:00.853 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update_receipt": 0, 2024-09-21T06:09:00.854 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_try_discover": 0, 2024-09-21T06:09:00.854 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_send_discover": 0, 2024-09-21T06:09:00.854 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_handle_discover": 0, 2024-09-21T06:09:00.854 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays": 0, 2024-09-21T06:09:00.854 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_delayed": 0, 2024-09-21T06:09:00.854 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_enqueuing": 0, 2024-09-21T06:09:00.854 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_created": 0, 2024-09-21T06:09:00.854 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_enqueued": 0, 2024-09-21T06:09:00.854 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_reintegrated": 0, 2024-09-21T06:09:00.854 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_migrated": 0, 2024-09-21T06:09:00.854 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_processing": 0, 2024-09-21T06:09:00.854 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_enqueued": 0, 2024-09-21T06:09:00.854 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_prioritized": 0, 2024-09-21T06:09:00.854 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_started": 0, 2024-09-21T06:09:00.854 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_completed": 0, 2024-09-21T06:09:00.855 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_path": 0, 2024-09-21T06:09:00.855 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_inode": 0, 2024-09-21T06:09:00.855 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T06:09:00.855 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_exportdir": 0, 2024-09-21T06:09:00.855 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_flush": 0, 2024-09-21T06:09:00.855 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragmentdir": 0, 2024-09-21T06:09:00.855 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragstats": 0, 2024-09-21T06:09:00.855 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_inodestats": 0, 2024-09-21T06:09:00.855 INFO:teuthology.orchestra.run.smithi138.stdout: "uninline_started": 0, 2024-09-21T06:09:00.855 INFO:teuthology.orchestra.run.smithi138.stdout: "uninline_succeeded": 0, 2024-09-21T06:09:00.855 INFO:teuthology.orchestra.run.smithi138.stdout: "uninline_write_failed": 0 2024-09-21T06:09:00.855 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.855 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_log": { 2024-09-21T06:09:00.856 INFO:teuthology.orchestra.run.smithi138.stdout: "evlrg": 0, 2024-09-21T06:09:00.856 INFO:teuthology.orchestra.run.smithi138.stdout: "evadd": 0, 2024-09-21T06:09:00.856 INFO:teuthology.orchestra.run.smithi138.stdout: "evex": 0, 2024-09-21T06:09:00.856 INFO:teuthology.orchestra.run.smithi138.stdout: "evtrm": 0, 2024-09-21T06:09:00.856 INFO:teuthology.orchestra.run.smithi138.stdout: "ev": 5, 2024-09-21T06:09:00.856 INFO:teuthology.orchestra.run.smithi138.stdout: "evexg": 0, 2024-09-21T06:09:00.856 INFO:teuthology.orchestra.run.smithi138.stdout: "evexd": 0, 2024-09-21T06:09:00.856 INFO:teuthology.orchestra.run.smithi138.stdout: "segadd": 0, 2024-09-21T06:09:00.856 INFO:teuthology.orchestra.run.smithi138.stdout: "segex": 0, 2024-09-21T06:09:00.856 INFO:teuthology.orchestra.run.smithi138.stdout: "segtrm": 0, 2024-09-21T06:09:00.856 INFO:teuthology.orchestra.run.smithi138.stdout: "seg": 2, 2024-09-21T06:09:00.856 INFO:teuthology.orchestra.run.smithi138.stdout: "segmjr": 2, 2024-09-21T06:09:00.856 INFO:teuthology.orchestra.run.smithi138.stdout: "segexg": 0, 2024-09-21T06:09:00.856 INFO:teuthology.orchestra.run.smithi138.stdout: "segexd": 0, 2024-09-21T06:09:00.856 INFO:teuthology.orchestra.run.smithi138.stdout: "expos": 4194304, 2024-09-21T06:09:00.857 INFO:teuthology.orchestra.run.smithi138.stdout: "wrpos": 4195732, 2024-09-21T06:09:00.857 INFO:teuthology.orchestra.run.smithi138.stdout: "rdpos": 4195424, 2024-09-21T06:09:00.857 INFO:teuthology.orchestra.run.smithi138.stdout: "jlat": { 2024-09-21T06:09:00.857 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.857 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.857 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.857 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.857 INFO:teuthology.orchestra.run.smithi138.stdout: "replayed": 5 2024-09-21T06:09:00.857 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.857 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_mem": { 2024-09-21T06:09:00.857 INFO:teuthology.orchestra.run.smithi138.stdout: "ino": 3, 2024-09-21T06:09:00.857 INFO:teuthology.orchestra.run.smithi138.stdout: "ino+": 3, 2024-09-21T06:09:00.857 INFO:teuthology.orchestra.run.smithi138.stdout: "ino-": 0, 2024-09-21T06:09:00.857 INFO:teuthology.orchestra.run.smithi138.stdout: "dir": 1, 2024-09-21T06:09:00.858 INFO:teuthology.orchestra.run.smithi138.stdout: "dir+": 1, 2024-09-21T06:09:00.858 INFO:teuthology.orchestra.run.smithi138.stdout: "dir-": 0, 2024-09-21T06:09:00.858 INFO:teuthology.orchestra.run.smithi138.stdout: "dn": 0, 2024-09-21T06:09:00.858 INFO:teuthology.orchestra.run.smithi138.stdout: "dn+": 0, 2024-09-21T06:09:00.858 INFO:teuthology.orchestra.run.smithi138.stdout: "dn-": 0, 2024-09-21T06:09:00.858 INFO:teuthology.orchestra.run.smithi138.stdout: "cap": 0, 2024-09-21T06:09:00.858 INFO:teuthology.orchestra.run.smithi138.stdout: "cap+": 0, 2024-09-21T06:09:00.858 INFO:teuthology.orchestra.run.smithi138.stdout: "cap-": 0, 2024-09-21T06:09:00.858 INFO:teuthology.orchestra.run.smithi138.stdout: "rss": 116484, 2024-09-21T06:09:00.858 INFO:teuthology.orchestra.run.smithi138.stdout: "heap": 316424192 2024-09-21T06:09:00.858 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.858 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_server": { 2024-09-21T06:09:00.858 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_client_request": 0, 2024-09-21T06:09:00.859 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_server_request": 0, 2024-09-21T06:09:00.859 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_request": 0, 2024-09-21T06:09:00.859 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_session": 0, 2024-09-21T06:09:00.859 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_peer_request": 0, 2024-09-21T06:09:00.859 INFO:teuthology.orchestra.run.smithi138.stdout: "req_create_latency": { 2024-09-21T06:09:00.859 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.859 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.859 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.859 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.859 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getattr_latency": { 2024-09-21T06:09:00.859 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.859 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.859 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.859 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.860 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getfilelock_latency": { 2024-09-21T06:09:00.860 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.860 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.860 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.860 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.860 INFO:teuthology.orchestra.run.smithi138.stdout: "req_link_latency": { 2024-09-21T06:09:00.860 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.860 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.860 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.860 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.860 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookup_latency": { 2024-09-21T06:09:00.860 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.860 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.861 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.861 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.861 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookuphash_latency": { 2024-09-21T06:09:00.861 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.861 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.861 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.861 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.861 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupino_latency": { 2024-09-21T06:09:00.861 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.861 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.861 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.861 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.861 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupname_latency": { 2024-09-21T06:09:00.861 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.862 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.862 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.862 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.862 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupparent_latency": { 2024-09-21T06:09:00.862 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.862 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.862 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.862 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.862 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupsnap_latency": { 2024-09-21T06:09:00.862 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.862 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.862 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.862 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.862 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lssnap_latency": { 2024-09-21T06:09:00.863 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.863 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.863 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.863 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.863 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mkdir_latency": { 2024-09-21T06:09:00.863 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.864 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.864 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.864 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.864 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mknod_latency": { 2024-09-21T06:09:00.864 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.864 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.864 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.864 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.864 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mksnap_latency": { 2024-09-21T06:09:00.864 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.864 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.864 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.865 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.865 INFO:teuthology.orchestra.run.smithi138.stdout: "req_open_latency": { 2024-09-21T06:09:00.865 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.865 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.865 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.865 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.865 INFO:teuthology.orchestra.run.smithi138.stdout: "req_readdir_latency": { 2024-09-21T06:09:00.865 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.865 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.865 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.866 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.866 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rename_latency": { 2024-09-21T06:09:00.866 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.866 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.866 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.866 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.866 INFO:teuthology.orchestra.run.smithi138.stdout: "req_renamesnap_latency": { 2024-09-21T06:09:00.866 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.866 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.866 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.866 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.866 INFO:teuthology.orchestra.run.smithi138.stdout: "req_snapdiff_latency": { 2024-09-21T06:09:00.866 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.867 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.867 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.867 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.867 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmdir_latency": { 2024-09-21T06:09:00.867 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.867 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.867 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.867 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.867 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmsnap_latency": { 2024-09-21T06:09:00.867 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.868 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.868 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.868 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.868 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmxattr_latency": { 2024-09-21T06:09:00.868 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.868 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.868 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.868 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.868 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setattr_latency": { 2024-09-21T06:09:00.868 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.868 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.868 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.868 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.868 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setdirlayout_latency": { 2024-09-21T06:09:00.868 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.869 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.869 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.869 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.869 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setfilelock_latency": { 2024-09-21T06:09:00.869 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.869 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.869 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.869 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.869 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setlayout_latency": { 2024-09-21T06:09:00.869 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.869 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.870 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.870 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.870 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setxattr_latency": { 2024-09-21T06:09:00.870 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.870 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.870 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.870 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.870 INFO:teuthology.orchestra.run.smithi138.stdout: "req_symlink_latency": { 2024-09-21T06:09:00.870 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.870 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.870 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.870 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.870 INFO:teuthology.orchestra.run.smithi138.stdout: "req_unlink_latency": { 2024-09-21T06:09:00.870 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.871 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.871 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.871 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.871 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_revoke_eviction": 0, 2024-09-21T06:09:00.871 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_acquisition_throttle": 0, 2024-09-21T06:09:00.871 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getvxattr_latency": { 2024-09-21T06:09:00.871 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.871 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.871 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.871 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:09:00.871 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.871 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_sessions": { 2024-09-21T06:09:00.871 INFO:teuthology.orchestra.run.smithi138.stdout: "session_count": 1, 2024-09-21T06:09:00.871 INFO:teuthology.orchestra.run.smithi138.stdout: "session_add": 2, 2024-09-21T06:09:00.871 INFO:teuthology.orchestra.run.smithi138.stdout: "session_remove": 1, 2024-09-21T06:09:00.871 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_open": 1, 2024-09-21T06:09:00.872 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_stale": 0, 2024-09-21T06:09:00.872 INFO:teuthology.orchestra.run.smithi138.stdout: "total_load": 0, 2024-09-21T06:09:00.872 INFO:teuthology.orchestra.run.smithi138.stdout: "average_load": 0, 2024-09-21T06:09:00.872 INFO:teuthology.orchestra.run.smithi138.stdout: "avg_session_uptime": 2036, 2024-09-21T06:09:00.872 INFO:teuthology.orchestra.run.smithi138.stdout: "mdthresh_evicted": 0 2024-09-21T06:09:00.872 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.872 INFO:teuthology.orchestra.run.smithi138.stdout: "mempool": { 2024-09-21T06:09:00.872 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:09:00.872 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_items": 0, 2024-09-21T06:09:00.872 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:09:00.872 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:09:00.872 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:09:00.872 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:09:00.872 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:09:00.872 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:09:00.873 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:09:00.873 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:09:00.873 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:09:00.873 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:09:00.873 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:09:00.873 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:09:00.873 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:09:00.873 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_items": 0, 2024-09-21T06:09:00.873 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:09:00.873 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_items": 0, 2024-09-21T06:09:00.873 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:09:00.873 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:09:00.873 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:09:00.873 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:09:00.874 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:09:00.874 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:09:00.874 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:09:00.874 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_items": 0, 2024-09-21T06:09:00.874 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:09:00.874 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:09:00.874 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:09:00.874 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_items": 0, 2024-09-21T06:09:00.874 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_bytes": 0, 2024-09-21T06:09:00.874 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_items": 0, 2024-09-21T06:09:00.874 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:09:00.874 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:09:00.874 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:09:00.874 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:09:00.874 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_bytes": 189383, 2024-09-21T06:09:00.875 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_items": 50, 2024-09-21T06:09:00.875 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_bytes": 0, 2024-09-21T06:09:00.875 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_items": 0, 2024-09-21T06:09:00.875 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bytes": 0, 2024-09-21T06:09:00.875 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_items": 0, 2024-09-21T06:09:00.875 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:09:00.875 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_items": 0, 2024-09-21T06:09:00.875 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:09:00.875 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_items": 0, 2024-09-21T06:09:00.875 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_bytes": 7968, 2024-09-21T06:09:00.875 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_items": 176, 2024-09-21T06:09:00.875 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:09:00.875 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:09:00.875 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_bytes": 0, 2024-09-21T06:09:00.875 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_items": 0, 2024-09-21T06:09:00.876 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_bytes": 8672, 2024-09-21T06:09:00.876 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_items": 13, 2024-09-21T06:09:00.876 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_bytes": 0, 2024-09-21T06:09:00.876 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_items": 0, 2024-09-21T06:09:00.876 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_bytes": 0, 2024-09-21T06:09:00.876 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_items": 0 2024-09-21T06:09:00.876 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.876 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter": { 2024-09-21T06:09:00.876 INFO:teuthology.orchestra.run.smithi138.stdout: "op_active": 0, 2024-09-21T06:09:00.876 INFO:teuthology.orchestra.run.smithi138.stdout: "op_laggy": 0, 2024-09-21T06:09:00.876 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send": 2291, 2024-09-21T06:09:00.876 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send_bytes": 21, 2024-09-21T06:09:00.876 INFO:teuthology.orchestra.run.smithi138.stdout: "op_resend": 3, 2024-09-21T06:09:00.876 INFO:teuthology.orchestra.run.smithi138.stdout: "op_reply": 2291, 2024-09-21T06:09:00.876 INFO:teuthology.orchestra.run.smithi138.stdout: "op_latency": { 2024-09-21T06:09:00.877 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2291, 2024-09-21T06:09:00.877 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 22.480514568, 2024-09-21T06:09:00.877 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.009812533 2024-09-21T06:09:00.877 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.877 INFO:teuthology.orchestra.run.smithi138.stdout: "op_inflight": 0, 2024-09-21T06:09:00.877 INFO:teuthology.orchestra.run.smithi138.stdout: "oplen_avg": { 2024-09-21T06:09:00.877 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2291, 2024-09-21T06:09:00.877 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 2292 2024-09-21T06:09:00.877 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.877 INFO:teuthology.orchestra.run.smithi138.stdout: "op": 2291, 2024-09-21T06:09:00.877 INFO:teuthology.orchestra.run.smithi138.stdout: "op_r": 2291, 2024-09-21T06:09:00.877 INFO:teuthology.orchestra.run.smithi138.stdout: "op_w": 0, 2024-09-21T06:09:00.877 INFO:teuthology.orchestra.run.smithi138.stdout: "op_rmw": 0, 2024-09-21T06:09:00.877 INFO:teuthology.orchestra.run.smithi138.stdout: "op_pg": 0, 2024-09-21T06:09:00.878 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_stat": 1489, 2024-09-21T06:09:00.878 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_create": 0, 2024-09-21T06:09:00.878 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_read": 800, 2024-09-21T06:09:00.878 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_write": 0, 2024-09-21T06:09:00.878 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writefull": 0, 2024-09-21T06:09:00.878 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writesame": 0, 2024-09-21T06:09:00.878 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_append": 0, 2024-09-21T06:09:00.878 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_zero": 0, 2024-09-21T06:09:00.878 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_truncate": 0, 2024-09-21T06:09:00.878 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_delete": 0, 2024-09-21T06:09:00.878 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_mapext": 0, 2024-09-21T06:09:00.879 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_sparse_read": 0, 2024-09-21T06:09:00.879 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_clonerange": 0, 2024-09-21T06:09:00.879 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_getxattr": 1, 2024-09-21T06:09:00.879 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_setxattr": 0, 2024-09-21T06:09:00.879 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_cmpxattr": 0, 2024-09-21T06:09:00.879 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_rmxattr": 0, 2024-09-21T06:09:00.879 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_resetxattrs": 0, 2024-09-21T06:09:00.879 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_call": 0, 2024-09-21T06:09:00.879 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_watch": 0, 2024-09-21T06:09:00.879 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_notify": 0, 2024-09-21T06:09:00.879 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T06:09:00.880 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls": 0, 2024-09-21T06:09:00.880 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls_filter": 0, 2024-09-21T06:09:00.880 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_other": 0, 2024-09-21T06:09:00.880 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_active": 0, 2024-09-21T06:09:00.880 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_send": 0, 2024-09-21T06:09:00.880 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_resend": 0, 2024-09-21T06:09:00.880 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_ping": 0, 2024-09-21T06:09:00.880 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_active": 0, 2024-09-21T06:09:00.880 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_send": 0, 2024-09-21T06:09:00.880 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_resend": 0, 2024-09-21T06:09:00.880 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_active": 0, 2024-09-21T06:09:00.881 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_send": 0, 2024-09-21T06:09:00.881 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_resend": 0, 2024-09-21T06:09:00.881 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_active": 0, 2024-09-21T06:09:00.881 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_send": 0, 2024-09-21T06:09:00.881 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_resend": 0, 2024-09-21T06:09:00.881 INFO:teuthology.orchestra.run.smithi138.stdout: "command_active": 0, 2024-09-21T06:09:00.881 INFO:teuthology.orchestra.run.smithi138.stdout: "command_send": 0, 2024-09-21T06:09:00.881 INFO:teuthology.orchestra.run.smithi138.stdout: "command_resend": 0, 2024-09-21T06:09:00.881 INFO:teuthology.orchestra.run.smithi138.stdout: "map_epoch": 85, 2024-09-21T06:09:00.881 INFO:teuthology.orchestra.run.smithi138.stdout: "map_full": 0, 2024-09-21T06:09:00.881 INFO:teuthology.orchestra.run.smithi138.stdout: "map_inc": 11, 2024-09-21T06:09:00.882 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_sessions": 5, 2024-09-21T06:09:00.882 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_open": 5, 2024-09-21T06:09:00.882 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_close": 0, 2024-09-21T06:09:00.882 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_laggy": 0, 2024-09-21T06:09:00.882 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_wr": 0, 2024-09-21T06:09:00.882 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_rd": 2, 2024-09-21T06:09:00.882 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_del": 0 2024-09-21T06:09:00.882 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.882 INFO:teuthology.orchestra.run.smithi138.stdout: "oft": { 2024-09-21T06:09:00.882 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_objs": 0, 2024-09-21T06:09:00.882 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_kv_pairs": 0, 2024-09-21T06:09:00.882 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_updates": 0, 2024-09-21T06:09:00.883 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_removes": 0 2024-09-21T06:09:00.883 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.883 INFO:teuthology.orchestra.run.smithi138.stdout: "purge_queue": { 2024-09-21T06:09:00.883 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops": 0, 2024-09-21T06:09:00.883 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T06:09:00.883 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing": 0, 2024-09-21T06:09:00.883 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_high_water": 0, 2024-09-21T06:09:00.883 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed_ops": 0, 2024-09-21T06:09:00.883 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed": 0, 2024-09-21T06:09:00.883 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_item_in_journal": 0 2024-09-21T06:09:00.883 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.884 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:09:00.884 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-21T06:09:00.884 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-09-21T06:09:00.884 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-21T06:09:00.884 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 2548, 2024-09-21T06:09:00.884 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 1291685, 2024-09-21T06:09:00.884 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-21T06:09:00.884 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 2548, 2024-09-21T06:09:00.884 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-09-21T06:09:00.884 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-09-21T06:09:00.884 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 2548, 2024-09-21T06:09:00.885 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 1291685, 2024-09-21T06:09:00.885 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-21T06:09:00.885 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.885 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.885 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.885 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:09:00.885 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.885 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_bytes": { 2024-09-21T06:09:00.885 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-21T06:09:00.885 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-09-21T06:09:00.885 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-21T06:09:00.885 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-09-21T06:09:00.885 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-09-21T06:09:00.885 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-21T06:09:00.885 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-09-21T06:09:00.886 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 2291, 2024-09-21T06:09:00.886 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 1433, 2024-09-21T06:09:00.886 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 2291, 2024-09-21T06:09:00.886 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 1433, 2024-09-21T06:09:00.886 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-21T06:09:00.886 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.886 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.886 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.886 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:09:00.886 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.886 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_ops": { 2024-09-21T06:09:00.886 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-21T06:09:00.886 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1024, 2024-09-21T06:09:00.887 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-21T06:09:00.887 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-09-21T06:09:00.887 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-09-21T06:09:00.887 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-21T06:09:00.887 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-09-21T06:09:00.887 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 2291, 2024-09-21T06:09:00.887 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 2291, 2024-09-21T06:09:00.887 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 2291, 2024-09-21T06:09:00.887 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 2291, 2024-09-21T06:09:00.887 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-21T06:09:00.887 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.887 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.887 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.887 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:09:00.887 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.888 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle": { 2024-09-21T06:09:00.888 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-21T06:09:00.888 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-09-21T06:09:00.888 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-21T06:09:00.888 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-09-21T06:09:00.888 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-09-21T06:09:00.888 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-21T06:09:00.888 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-09-21T06:09:00.888 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-09-21T06:09:00.888 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-09-21T06:09:00.888 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 0, 2024-09-21T06:09:00.888 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 0, 2024-09-21T06:09:00.888 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-21T06:09:00.888 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.888 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.888 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.889 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:09:00.889 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.889 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle-0x5585c546b2b0": { 2024-09-21T06:09:00.889 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-09-21T06:09:00.889 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-09-21T06:09:00.889 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-09-21T06:09:00.889 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-09-21T06:09:00.889 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-09-21T06:09:00.889 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-09-21T06:09:00.889 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-09-21T06:09:00.889 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-09-21T06:09:00.889 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-09-21T06:09:00.889 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 0, 2024-09-21T06:09:00.889 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 0, 2024-09-21T06:09:00.890 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-09-21T06:09:00.890 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-09-21T06:09:00.890 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-09-21T06:09:00.890 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-09-21T06:09:00.890 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:09:00.890 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-21T06:09:00.890 INFO:teuthology.orchestra.run.smithi138.stdout: "trackedop": { 2024-09-21T06:09:00.890 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_ops_count": 0 2024-09-21T06:09:00.890 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-21T06:09:00.890 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-21T06:09:01.640 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-21T06:09:01.640 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-21T06:09:01.640 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-09-21T06:09:01.640 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-09-21T06:09:01.640 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-09-21T06:09:01.641 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-09-21T06:09:01.641 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T06:09:01.641 DEBUG:tasks.check_counter:Getting stats from c 2024-09-21T06:09:01.641 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph daemon mds.c perf dump 2024-09-21T06:09:01.954 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:09:02.232 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:01 smithi192 ceph-mon[34700]: pgmap v687: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:09:02.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:01 smithi117 ceph-mon[30947]: pgmap v687: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:09:02.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:01 smithi138 ceph-mon[34391]: pgmap v687: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-21T06:09:02.514 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-21T06:09:02.514 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:09:02.514 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 65, 2024-09-21T06:09:02.514 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 103, 2024-09-21T06:09:02.514 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 14632, 2024-09-21T06:09:02.514 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 340769, 2024-09-21T06:09:02.514 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 38, 2024-09-21T06:09:02.514 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 7, 2024-09-21T06:09:02.514 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.031402408, 2024-09-21T06:09:02.515 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.005385249, 2024-09-21T06:09:02.515 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.020739999, 2024-09-21T06:09:02.515 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.000411663, 2024-09-21T06:09:02.515 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:09:02.515 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 98, 2024-09-21T06:09:02.515 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.018366333, 2024-09-21T06:09:02.515 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000187411 2024-09-21T06:09:02.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.515 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:09:02.515 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 86, 2024-09-21T06:09:02.515 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000023612, 2024-09-21T06:09:02.515 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000274 2024-09-21T06:09:02.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.515 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 3920, 2024-09-21T06:09:02.515 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 1184 2024-09-21T06:09:02.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.516 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:09:02.516 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 369, 2024-09-21T06:09:02.516 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 355, 2024-09-21T06:09:02.516 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 942412, 2024-09-21T06:09:02.516 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 241540, 2024-09-21T06:09:02.516 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 11, 2024-09-21T06:09:02.516 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 5, 2024-09-21T06:09:02.516 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.060689708, 2024-09-21T06:09:02.516 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.027040621, 2024-09-21T06:09:02.516 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.044574173, 2024-09-21T06:09:02.516 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.000405124, 2024-09-21T06:09:02.516 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:09:02.516 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 354, 2024-09-21T06:09:02.516 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.033781603, 2024-09-21T06:09:02.516 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000095428 2024-09-21T06:09:02.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.517 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:09:02.517 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 120, 2024-09-21T06:09:02.517 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000029593, 2024-09-21T06:09:02.517 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000246 2024-09-21T06:09:02.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.517 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 907104, 2024-09-21T06:09:02.517 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 148272 2024-09-21T06:09:02.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.517 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:09:02.517 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 1538, 2024-09-21T06:09:02.517 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 1218, 2024-09-21T06:09:02.517 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 460893, 2024-09-21T06:09:02.517 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 776328, 2024-09-21T06:09:02.517 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 8, 2024-09-21T06:09:02.518 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 4, 2024-09-21T06:09:02.518 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.142292740, 2024-09-21T06:09:02.518 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.067818763, 2024-09-21T06:09:02.518 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.088459696, 2024-09-21T06:09:02.518 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.005616165, 2024-09-21T06:09:02.518 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:09:02.518 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1220, 2024-09-21T06:09:02.518 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.091320570, 2024-09-21T06:09:02.518 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000074852 2024-09-21T06:09:02.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.518 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:09:02.518 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 2427, 2024-09-21T06:09:02.518 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000628503, 2024-09-21T06:09:02.518 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000258 2024-09-21T06:09:02.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.519 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T06:09:02.519 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 390768 2024-09-21T06:09:02.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.519 INFO:teuthology.orchestra.run.smithi192.stdout: "cct": { 2024-09-21T06:09:02.519 INFO:teuthology.orchestra.run.smithi192.stdout: "total_workers": 1, 2024-09-21T06:09:02.519 INFO:teuthology.orchestra.run.smithi192.stdout: "unhealthy_workers": 0 2024-09-21T06:09:02.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.519 INFO:teuthology.orchestra.run.smithi192.stdout: "finisher-MDSRank": { 2024-09-21T06:09:02.519 INFO:teuthology.orchestra.run.smithi192.stdout: "queue_len": 0, 2024-09-21T06:09:02.519 INFO:teuthology.orchestra.run.smithi192.stdout: "complete_latency": { 2024-09-21T06:09:02.520 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 33, 2024-09-21T06:09:02.520 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.001683954, 2024-09-21T06:09:02.520 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000051028 2024-09-21T06:09:02.520 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:02.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.520 INFO:teuthology.orchestra.run.smithi192.stdout: "finisher-PurgeQueue": { 2024-09-21T06:09:02.520 INFO:teuthology.orchestra.run.smithi192.stdout: "queue_len": 0, 2024-09-21T06:09:02.520 INFO:teuthology.orchestra.run.smithi192.stdout: "complete_latency": { 2024-09-21T06:09:02.520 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 2, 2024-09-21T06:09:02.520 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000022213, 2024-09-21T06:09:02.520 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000011106 2024-09-21T06:09:02.520 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:02.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.520 INFO:teuthology.orchestra.run.smithi192.stdout: "mds": { 2024-09-21T06:09:02.520 INFO:teuthology.orchestra.run.smithi192.stdout: "request": 1, 2024-09-21T06:09:02.521 INFO:teuthology.orchestra.run.smithi192.stdout: "reply": 1, 2024-09-21T06:09:02.521 INFO:teuthology.orchestra.run.smithi192.stdout: "reply_latency": { 2024-09-21T06:09:02.521 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1, 2024-09-21T06:09:02.521 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000124798, 2024-09-21T06:09:02.521 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000124798 2024-09-21T06:09:02.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.521 INFO:teuthology.orchestra.run.smithi192.stdout: "slow_reply": 0, 2024-09-21T06:09:02.521 INFO:teuthology.orchestra.run.smithi192.stdout: "forward": 0, 2024-09-21T06:09:02.521 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_fetch_complete": 0, 2024-09-21T06:09:02.521 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_fetch_keys": 0, 2024-09-21T06:09:02.521 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_commit": 11, 2024-09-21T06:09:02.521 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_split": 0, 2024-09-21T06:09:02.521 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_merge": 0, 2024-09-21T06:09:02.521 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes": 16, 2024-09-21T06:09:02.521 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_top": 0, 2024-09-21T06:09:02.522 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_bottom": 16, 2024-09-21T06:09:02.522 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_pin_tail": 0, 2024-09-21T06:09:02.522 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_pinned": 13, 2024-09-21T06:09:02.522 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_expired": 8, 2024-09-21T06:09:02.522 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_with_caps": 1, 2024-09-21T06:09:02.522 INFO:teuthology.orchestra.run.smithi192.stdout: "caps": 1, 2024-09-21T06:09:02.522 INFO:teuthology.orchestra.run.smithi192.stdout: "subtrees": 3, 2024-09-21T06:09:02.522 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse": 9, 2024-09-21T06:09:02.522 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_hit": 5, 2024-09-21T06:09:02.522 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_forward": 0, 2024-09-21T06:09:02.522 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_discover": 4, 2024-09-21T06:09:02.522 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_dir_fetch": 0, 2024-09-21T06:09:02.522 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_remote_ino": 0, 2024-09-21T06:09:02.522 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_lock": 0, 2024-09-21T06:09:02.522 INFO:teuthology.orchestra.run.smithi192.stdout: "load_cent": 0, 2024-09-21T06:09:02.523 INFO:teuthology.orchestra.run.smithi192.stdout: "q": 0, 2024-09-21T06:09:02.523 INFO:teuthology.orchestra.run.smithi192.stdout: "exported": 0, 2024-09-21T06:09:02.523 INFO:teuthology.orchestra.run.smithi192.stdout: "exported_inodes": 0, 2024-09-21T06:09:02.523 INFO:teuthology.orchestra.run.smithi192.stdout: "imported": 0, 2024-09-21T06:09:02.523 INFO:teuthology.orchestra.run.smithi192.stdout: "imported_inodes": 0, 2024-09-21T06:09:02.523 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_dir_fetch": 0, 2024-09-21T06:09:02.523 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_backtrace_fetch": 0, 2024-09-21T06:09:02.523 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_peer_discover": 0, 2024-09-21T06:09:02.523 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rfiles": 22, 2024-09-21T06:09:02.523 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rbytes": 10738142506, 2024-09-21T06:09:02.523 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rsnaps": 7, 2024-09-21T06:09:02.523 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T06:09:02.523 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_set_tag": 0, 2024-09-21T06:09:02.523 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T06:09:02.523 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_inotable_repaired": 0, 2024-09-21T06:09:02.523 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dir_inodes": 0, 2024-09-21T06:09:02.524 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T06:09:02.524 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T06:09:02.524 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_file_inodes": 0, 2024-09-21T06:09:02.524 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_inode_file_caps": 0, 2024-09-21T06:09:02.524 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T06:09:02.524 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_grant": 0, 2024-09-21T06:09:02.524 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T06:09:02.524 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T06:09:02.524 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T06:09:02.524 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_caps": 0, 2024-09-21T06:09:02.524 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_caps_dirty": 0, 2024-09-21T06:09:02.524 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_cap_release": 0, 2024-09-21T06:09:02.524 INFO:teuthology.orchestra.run.smithi192.stdout: "process_request_cap_release": 0 2024-09-21T06:09:02.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.524 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_cache": { 2024-09-21T06:09:02.525 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_update": 0, 2024-09-21T06:09:02.525 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_update_receipt": 14, 2024-09-21T06:09:02.525 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_try_discover": 9, 2024-09-21T06:09:02.525 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_send_discover": 10, 2024-09-21T06:09:02.525 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_handle_discover": 0, 2024-09-21T06:09:02.525 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays": 0, 2024-09-21T06:09:02.525 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays_delayed": 0, 2024-09-21T06:09:02.525 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays_enqueuing": 0, 2024-09-21T06:09:02.525 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_created": 0, 2024-09-21T06:09:02.525 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_enqueued": 0, 2024-09-21T06:09:02.525 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_reintegrated": 0, 2024-09-21T06:09:02.525 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_migrated": 0, 2024-09-21T06:09:02.525 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_processing": 0, 2024-09-21T06:09:02.525 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_enqueued": 0, 2024-09-21T06:09:02.525 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_prioritized": 0, 2024-09-21T06:09:02.525 INFO:teuthology.orchestra.run.smithi192.stdout: "recovery_started": 0, 2024-09-21T06:09:02.526 INFO:teuthology.orchestra.run.smithi192.stdout: "recovery_completed": 0, 2024-09-21T06:09:02.526 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_quiesce_path": 0, 2024-09-21T06:09:02.526 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_quiesce_inode": 0, 2024-09-21T06:09:02.526 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T06:09:02.526 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_exportdir": 0, 2024-09-21T06:09:02.526 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_flush": 0, 2024-09-21T06:09:02.526 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_fragmentdir": 0, 2024-09-21T06:09:02.526 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_fragstats": 0, 2024-09-21T06:09:02.526 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_inodestats": 0, 2024-09-21T06:09:02.526 INFO:teuthology.orchestra.run.smithi192.stdout: "uninline_started": 0, 2024-09-21T06:09:02.526 INFO:teuthology.orchestra.run.smithi192.stdout: "uninline_succeeded": 0, 2024-09-21T06:09:02.526 INFO:teuthology.orchestra.run.smithi192.stdout: "uninline_write_failed": 0 2024-09-21T06:09:02.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.526 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_log": { 2024-09-21T06:09:02.526 INFO:teuthology.orchestra.run.smithi192.stdout: "evlrg": 0, 2024-09-21T06:09:02.527 INFO:teuthology.orchestra.run.smithi192.stdout: "evadd": 5, 2024-09-21T06:09:02.527 INFO:teuthology.orchestra.run.smithi192.stdout: "evex": 0, 2024-09-21T06:09:02.527 INFO:teuthology.orchestra.run.smithi192.stdout: "evtrm": 0, 2024-09-21T06:09:02.527 INFO:teuthology.orchestra.run.smithi192.stdout: "ev": 5, 2024-09-21T06:09:02.527 INFO:teuthology.orchestra.run.smithi192.stdout: "evexg": 0, 2024-09-21T06:09:02.527 INFO:teuthology.orchestra.run.smithi192.stdout: "evexd": 0, 2024-09-21T06:09:02.527 INFO:teuthology.orchestra.run.smithi192.stdout: "segadd": 2, 2024-09-21T06:09:02.527 INFO:teuthology.orchestra.run.smithi192.stdout: "segex": 0, 2024-09-21T06:09:02.527 INFO:teuthology.orchestra.run.smithi192.stdout: "segtrm": 0, 2024-09-21T06:09:02.527 INFO:teuthology.orchestra.run.smithi192.stdout: "seg": 2, 2024-09-21T06:09:02.527 INFO:teuthology.orchestra.run.smithi192.stdout: "segmjr": 2, 2024-09-21T06:09:02.527 INFO:teuthology.orchestra.run.smithi192.stdout: "segexg": 0, 2024-09-21T06:09:02.527 INFO:teuthology.orchestra.run.smithi192.stdout: "segexd": 0, 2024-09-21T06:09:02.527 INFO:teuthology.orchestra.run.smithi192.stdout: "expos": 4194304, 2024-09-21T06:09:02.527 INFO:teuthology.orchestra.run.smithi192.stdout: "wrpos": 4195732, 2024-09-21T06:09:02.527 INFO:teuthology.orchestra.run.smithi192.stdout: "rdpos": 0, 2024-09-21T06:09:02.528 INFO:teuthology.orchestra.run.smithi192.stdout: "jlat": { 2024-09-21T06:09:02.528 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 4, 2024-09-21T06:09:02.528 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.008458554, 2024-09-21T06:09:02.528 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.002114638 2024-09-21T06:09:02.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.528 INFO:teuthology.orchestra.run.smithi192.stdout: "replayed": 0 2024-09-21T06:09:02.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.528 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_mem": { 2024-09-21T06:09:02.528 INFO:teuthology.orchestra.run.smithi192.stdout: "ino": 20, 2024-09-21T06:09:02.528 INFO:teuthology.orchestra.run.smithi192.stdout: "ino+": 24, 2024-09-21T06:09:02.528 INFO:teuthology.orchestra.run.smithi192.stdout: "ino-": 4, 2024-09-21T06:09:02.528 INFO:teuthology.orchestra.run.smithi192.stdout: "dir": 19, 2024-09-21T06:09:02.528 INFO:teuthology.orchestra.run.smithi192.stdout: "dir+": 25, 2024-09-21T06:09:02.528 INFO:teuthology.orchestra.run.smithi192.stdout: "dir-": 6, 2024-09-21T06:09:02.528 INFO:teuthology.orchestra.run.smithi192.stdout: "dn": 16, 2024-09-21T06:09:02.529 INFO:teuthology.orchestra.run.smithi192.stdout: "dn+": 24, 2024-09-21T06:09:02.529 INFO:teuthology.orchestra.run.smithi192.stdout: "dn-": 8, 2024-09-21T06:09:02.529 INFO:teuthology.orchestra.run.smithi192.stdout: "cap": 1, 2024-09-21T06:09:02.529 INFO:teuthology.orchestra.run.smithi192.stdout: "cap+": 1, 2024-09-21T06:09:02.529 INFO:teuthology.orchestra.run.smithi192.stdout: "cap-": 0, 2024-09-21T06:09:02.529 INFO:teuthology.orchestra.run.smithi192.stdout: "rss": 118352, 2024-09-21T06:09:02.529 INFO:teuthology.orchestra.run.smithi192.stdout: "heap": 325857280 2024-09-21T06:09:02.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.529 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_server": { 2024-09-21T06:09:02.529 INFO:teuthology.orchestra.run.smithi192.stdout: "dispatch_client_request": 1, 2024-09-21T06:09:02.529 INFO:teuthology.orchestra.run.smithi192.stdout: "dispatch_server_request": 0, 2024-09-21T06:09:02.529 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_request": 1, 2024-09-21T06:09:02.529 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_session": 14, 2024-09-21T06:09:02.529 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_peer_request": 0, 2024-09-21T06:09:02.529 INFO:teuthology.orchestra.run.smithi192.stdout: "req_create_latency": { 2024-09-21T06:09:02.529 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:02.530 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:02.530 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:02.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.530 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getattr_latency": { 2024-09-21T06:09:02.530 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:02.530 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:02.530 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:02.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.530 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getfilelock_latency": { 2024-09-21T06:09:02.530 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:02.530 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:02.530 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:02.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.530 INFO:teuthology.orchestra.run.smithi192.stdout: "req_link_latency": { 2024-09-21T06:09:02.530 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:02.531 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:02.531 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:02.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.531 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookup_latency": { 2024-09-21T06:09:02.531 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:02.531 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:02.531 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:02.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.531 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookuphash_latency": { 2024-09-21T06:09:02.531 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:02.531 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:02.531 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:02.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.531 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupino_latency": { 2024-09-21T06:09:02.531 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1, 2024-09-21T06:09:02.532 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000124798, 2024-09-21T06:09:02.532 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000124798 2024-09-21T06:09:02.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.532 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupname_latency": { 2024-09-21T06:09:02.532 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:02.532 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:02.532 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:02.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.532 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupparent_latency": { 2024-09-21T06:09:02.532 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:02.532 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:02.532 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:02.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.532 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupsnap_latency": { 2024-09-21T06:09:02.532 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:02.532 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:02.533 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:02.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.533 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lssnap_latency": { 2024-09-21T06:09:02.533 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:02.533 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:02.533 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:02.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.534 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mkdir_latency": { 2024-09-21T06:09:02.534 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:02.534 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:02.534 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:02.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.534 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mknod_latency": { 2024-09-21T06:09:02.534 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:02.534 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:02.534 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:02.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.534 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mksnap_latency": { 2024-09-21T06:09:02.534 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:02.534 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:02.534 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:02.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.535 INFO:teuthology.orchestra.run.smithi192.stdout: "req_open_latency": { 2024-09-21T06:09:02.535 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:02.535 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:02.535 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:02.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.535 INFO:teuthology.orchestra.run.smithi192.stdout: "req_readdir_latency": { 2024-09-21T06:09:02.535 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:02.535 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:02.535 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:02.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.535 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rename_latency": { 2024-09-21T06:09:02.535 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:02.535 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:02.535 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:02.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.536 INFO:teuthology.orchestra.run.smithi192.stdout: "req_renamesnap_latency": { 2024-09-21T06:09:02.536 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:02.536 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:02.536 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:02.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.536 INFO:teuthology.orchestra.run.smithi192.stdout: "req_snapdiff_latency": { 2024-09-21T06:09:02.536 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:02.536 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:02.536 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:02.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.536 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmdir_latency": { 2024-09-21T06:09:02.536 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:02.536 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:02.536 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:02.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.537 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmsnap_latency": { 2024-09-21T06:09:02.537 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:02.537 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:02.537 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:02.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.537 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmxattr_latency": { 2024-09-21T06:09:02.537 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:02.537 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:02.537 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:02.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.537 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setattr_latency": { 2024-09-21T06:09:02.537 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:02.537 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:02.537 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:02.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.538 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setdirlayout_latency": { 2024-09-21T06:09:02.538 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:02.538 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:02.538 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:02.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.538 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setfilelock_latency": { 2024-09-21T06:09:02.538 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:02.538 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:02.538 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:02.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.538 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setlayout_latency": { 2024-09-21T06:09:02.538 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:02.538 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:02.538 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:02.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.539 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setxattr_latency": { 2024-09-21T06:09:02.539 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:02.539 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:02.539 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:02.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.539 INFO:teuthology.orchestra.run.smithi192.stdout: "req_symlink_latency": { 2024-09-21T06:09:02.539 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:02.539 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:02.539 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:02.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.539 INFO:teuthology.orchestra.run.smithi192.stdout: "req_unlink_latency": { 2024-09-21T06:09:02.539 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:02.539 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:02.539 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:02.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.540 INFO:teuthology.orchestra.run.smithi192.stdout: "cap_revoke_eviction": 0, 2024-09-21T06:09:02.540 INFO:teuthology.orchestra.run.smithi192.stdout: "cap_acquisition_throttle": 0, 2024-09-21T06:09:02.540 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getvxattr_latency": { 2024-09-21T06:09:02.540 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:02.540 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:02.540 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:02.540 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:02.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.540 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_sessions": { 2024-09-21T06:09:02.540 INFO:teuthology.orchestra.run.smithi192.stdout: "session_count": 1, 2024-09-21T06:09:02.540 INFO:teuthology.orchestra.run.smithi192.stdout: "session_add": 2, 2024-09-21T06:09:02.540 INFO:teuthology.orchestra.run.smithi192.stdout: "session_remove": 1, 2024-09-21T06:09:02.540 INFO:teuthology.orchestra.run.smithi192.stdout: "sessions_open": 1, 2024-09-21T06:09:02.540 INFO:teuthology.orchestra.run.smithi192.stdout: "sessions_stale": 0, 2024-09-21T06:09:02.541 INFO:teuthology.orchestra.run.smithi192.stdout: "total_load": 1, 2024-09-21T06:09:02.541 INFO:teuthology.orchestra.run.smithi192.stdout: "average_load": 1, 2024-09-21T06:09:02.541 INFO:teuthology.orchestra.run.smithi192.stdout: "avg_session_uptime": 703, 2024-09-21T06:09:02.541 INFO:teuthology.orchestra.run.smithi192.stdout: "mdthresh_evicted": 0 2024-09-21T06:09:02.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.541 INFO:teuthology.orchestra.run.smithi192.stdout: "mempool": { 2024-09-21T06:09:02.541 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:09:02.541 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_items": 0, 2024-09-21T06:09:02.541 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:09:02.541 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:09:02.541 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:09:02.541 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:09:02.541 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:09:02.541 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:09:02.541 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:09:02.542 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:09:02.542 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:09:02.542 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:09:02.542 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:09:02.542 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:09:02.542 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:09:02.542 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_items": 0, 2024-09-21T06:09:02.542 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:09:02.542 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_items": 0, 2024-09-21T06:09:02.542 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:09:02.542 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:09:02.542 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:09:02.542 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:09:02.542 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:09:02.543 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:09:02.543 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:09:02.543 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_items": 0, 2024-09-21T06:09:02.543 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:09:02.543 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:09:02.543 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:09:02.543 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_items": 0, 2024-09-21T06:09:02.543 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_bytes": 0, 2024-09-21T06:09:02.543 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_items": 0, 2024-09-21T06:09:02.543 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:09:02.543 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:09:02.543 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:09:02.543 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:09:02.543 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_bytes": 262451, 2024-09-21T06:09:02.543 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_items": 87, 2024-09-21T06:09:02.543 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_bytes": 0, 2024-09-21T06:09:02.544 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_items": 0, 2024-09-21T06:09:02.544 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_bytes": 0, 2024-09-21T06:09:02.544 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_items": 0, 2024-09-21T06:09:02.544 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:09:02.544 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_items": 0, 2024-09-21T06:09:02.544 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:09:02.544 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_items": 0, 2024-09-21T06:09:02.544 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_bytes": 7968, 2024-09-21T06:09:02.544 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_items": 176, 2024-09-21T06:09:02.544 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:09:02.544 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:09:02.544 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_bytes": 0, 2024-09-21T06:09:02.544 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_items": 0, 2024-09-21T06:09:02.544 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_bytes": 83797, 2024-09-21T06:09:02.545 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_items": 386, 2024-09-21T06:09:02.545 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_bytes": 0, 2024-09-21T06:09:02.545 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_items": 0, 2024-09-21T06:09:02.545 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_bytes": 0, 2024-09-21T06:09:02.545 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_items": 0 2024-09-21T06:09:02.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.545 INFO:teuthology.orchestra.run.smithi192.stdout: "objecter": { 2024-09-21T06:09:02.545 INFO:teuthology.orchestra.run.smithi192.stdout: "op_active": 0, 2024-09-21T06:09:02.545 INFO:teuthology.orchestra.run.smithi192.stdout: "op_laggy": 0, 2024-09-21T06:09:02.545 INFO:teuthology.orchestra.run.smithi192.stdout: "op_send": 40, 2024-09-21T06:09:02.545 INFO:teuthology.orchestra.run.smithi192.stdout: "op_send_bytes": 11851, 2024-09-21T06:09:02.545 INFO:teuthology.orchestra.run.smithi192.stdout: "op_resend": 33, 2024-09-21T06:09:02.546 INFO:teuthology.orchestra.run.smithi192.stdout: "op_reply": 40, 2024-09-21T06:09:02.546 INFO:teuthology.orchestra.run.smithi192.stdout: "op_latency": { 2024-09-21T06:09:02.546 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 40, 2024-09-21T06:09:02.546 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.219995891, 2024-09-21T06:09:02.546 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.005499897 2024-09-21T06:09:02.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.546 INFO:teuthology.orchestra.run.smithi192.stdout: "op_inflight": 0, 2024-09-21T06:09:02.546 INFO:teuthology.orchestra.run.smithi192.stdout: "oplen_avg": { 2024-09-21T06:09:02.546 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 40, 2024-09-21T06:09:02.546 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 63 2024-09-21T06:09:02.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.546 INFO:teuthology.orchestra.run.smithi192.stdout: "op": 40, 2024-09-21T06:09:02.546 INFO:teuthology.orchestra.run.smithi192.stdout: "op_r": 0, 2024-09-21T06:09:02.546 INFO:teuthology.orchestra.run.smithi192.stdout: "op_w": 40, 2024-09-21T06:09:02.546 INFO:teuthology.orchestra.run.smithi192.stdout: "op_rmw": 0, 2024-09-21T06:09:02.547 INFO:teuthology.orchestra.run.smithi192.stdout: "op_pg": 0, 2024-09-21T06:09:02.547 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_stat": 0, 2024-09-21T06:09:02.547 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_create": 10, 2024-09-21T06:09:02.547 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_read": 0, 2024-09-21T06:09:02.547 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_write": 4, 2024-09-21T06:09:02.547 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_writefull": 6, 2024-09-21T06:09:02.547 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_writesame": 0, 2024-09-21T06:09:02.547 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_append": 0, 2024-09-21T06:09:02.547 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_zero": 0, 2024-09-21T06:09:02.547 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_truncate": 0, 2024-09-21T06:09:02.547 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_delete": 6, 2024-09-21T06:09:02.547 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_mapext": 0, 2024-09-21T06:09:02.547 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_sparse_read": 0, 2024-09-21T06:09:02.547 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_clonerange": 0, 2024-09-21T06:09:02.547 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_getxattr": 0, 2024-09-21T06:09:02.548 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_setxattr": 20, 2024-09-21T06:09:02.548 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_cmpxattr": 0, 2024-09-21T06:09:02.548 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_rmxattr": 0, 2024-09-21T06:09:02.548 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_resetxattrs": 0, 2024-09-21T06:09:02.548 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_call": 0, 2024-09-21T06:09:02.548 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_watch": 0, 2024-09-21T06:09:02.548 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_notify": 0, 2024-09-21T06:09:02.548 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T06:09:02.548 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_pgls": 0, 2024-09-21T06:09:02.548 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_pgls_filter": 0, 2024-09-21T06:09:02.548 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_other": 0, 2024-09-21T06:09:02.548 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_active": 0, 2024-09-21T06:09:02.548 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_send": 0, 2024-09-21T06:09:02.548 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_resend": 0, 2024-09-21T06:09:02.548 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_ping": 0, 2024-09-21T06:09:02.548 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_active": 0, 2024-09-21T06:09:02.549 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_send": 0, 2024-09-21T06:09:02.549 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_resend": 0, 2024-09-21T06:09:02.549 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_active": 0, 2024-09-21T06:09:02.549 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_send": 0, 2024-09-21T06:09:02.549 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_resend": 0, 2024-09-21T06:09:02.549 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_active": 0, 2024-09-21T06:09:02.549 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_send": 0, 2024-09-21T06:09:02.549 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_resend": 0, 2024-09-21T06:09:02.549 INFO:teuthology.orchestra.run.smithi192.stdout: "command_active": 0, 2024-09-21T06:09:02.549 INFO:teuthology.orchestra.run.smithi192.stdout: "command_send": 0, 2024-09-21T06:09:02.549 INFO:teuthology.orchestra.run.smithi192.stdout: "command_resend": 0, 2024-09-21T06:09:02.549 INFO:teuthology.orchestra.run.smithi192.stdout: "map_epoch": 85, 2024-09-21T06:09:02.549 INFO:teuthology.orchestra.run.smithi192.stdout: "map_full": 0, 2024-09-21T06:09:02.549 INFO:teuthology.orchestra.run.smithi192.stdout: "map_inc": 11, 2024-09-21T06:09:02.549 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_sessions": 8, 2024-09-21T06:09:02.550 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_session_open": 8, 2024-09-21T06:09:02.550 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_session_close": 0, 2024-09-21T06:09:02.550 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_laggy": 0, 2024-09-21T06:09:02.550 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_wr": 16, 2024-09-21T06:09:02.550 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_rd": 0, 2024-09-21T06:09:02.550 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_del": 1 2024-09-21T06:09:02.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.550 INFO:teuthology.orchestra.run.smithi192.stdout: "oft": { 2024-09-21T06:09:02.550 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_objs": 1, 2024-09-21T06:09:02.550 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_kv_pairs": 0, 2024-09-21T06:09:02.550 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_updates": 0, 2024-09-21T06:09:02.550 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_removes": 0 2024-09-21T06:09:02.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.550 INFO:teuthology.orchestra.run.smithi192.stdout: "purge_queue": { 2024-09-21T06:09:02.550 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_ops": 0, 2024-09-21T06:09:02.551 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T06:09:02.551 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing": 0, 2024-09-21T06:09:02.551 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_high_water": 0, 2024-09-21T06:09:02.551 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executed_ops": 0, 2024-09-21T06:09:02.551 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executed": 0, 2024-09-21T06:09:02.551 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_item_in_journal": 0 2024-09-21T06:09:02.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.551 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:09:02.551 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-21T06:09:02.551 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-09-21T06:09:02.551 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-21T06:09:02.551 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 1968, 2024-09-21T06:09:02.551 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 1223079, 2024-09-21T06:09:02.551 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-21T06:09:02.551 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 1968, 2024-09-21T06:09:02.551 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-09-21T06:09:02.552 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-09-21T06:09:02.552 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 1968, 2024-09-21T06:09:02.552 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 1223079, 2024-09-21T06:09:02.552 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-21T06:09:02.552 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:02.552 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:02.552 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:02.552 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:02.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.552 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-objecter_bytes": { 2024-09-21T06:09:02.552 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-21T06:09:02.552 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-09-21T06:09:02.552 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-21T06:09:02.552 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-09-21T06:09:02.552 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-09-21T06:09:02.553 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-21T06:09:02.553 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-09-21T06:09:02.553 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 40, 2024-09-21T06:09:02.553 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 11851, 2024-09-21T06:09:02.553 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 40, 2024-09-21T06:09:02.553 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 11851, 2024-09-21T06:09:02.553 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-21T06:09:02.553 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:02.553 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:02.553 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:02.553 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:02.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.553 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-objecter_ops": { 2024-09-21T06:09:02.553 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-21T06:09:02.553 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 1024, 2024-09-21T06:09:02.553 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-21T06:09:02.554 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-09-21T06:09:02.554 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-09-21T06:09:02.554 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-21T06:09:02.554 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-09-21T06:09:02.554 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 40, 2024-09-21T06:09:02.554 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 40, 2024-09-21T06:09:02.554 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 40, 2024-09-21T06:09:02.554 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 40, 2024-09-21T06:09:02.554 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-21T06:09:02.554 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:02.554 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:02.554 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:02.554 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:02.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.554 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-write_buf_throttle": { 2024-09-21T06:09:02.555 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-21T06:09:02.555 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 3758096384, 2024-09-21T06:09:02.555 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-21T06:09:02.555 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-09-21T06:09:02.555 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-09-21T06:09:02.555 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-21T06:09:02.555 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-09-21T06:09:02.555 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-09-21T06:09:02.555 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-09-21T06:09:02.555 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 0, 2024-09-21T06:09:02.555 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 0, 2024-09-21T06:09:02.555 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-21T06:09:02.555 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:02.555 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:02.555 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:02.556 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:02.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.556 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-write_buf_throttle-0x55b34cbfb520": { 2024-09-21T06:09:02.556 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-21T06:09:02.556 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 3758096384, 2024-09-21T06:09:02.556 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-21T06:09:02.556 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 5, 2024-09-21T06:09:02.556 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 1428, 2024-09-21T06:09:02.556 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-21T06:09:02.556 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 5, 2024-09-21T06:09:02.556 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-09-21T06:09:02.556 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-09-21T06:09:02.556 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 4, 2024-09-21T06:09:02.556 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 1428, 2024-09-21T06:09:02.556 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-21T06:09:02.556 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:02.557 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:02.557 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:02.557 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:02.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:02.557 INFO:teuthology.orchestra.run.smithi192.stdout: "trackedop": { 2024-09-21T06:09:02.557 INFO:teuthology.orchestra.run.smithi192.stdout: "slow_ops_count": 0 2024-09-21T06:09:02.557 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:02.557 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-21T06:09:03.457 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-21T06:09:03.458 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-21T06:09:03.458 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-09-21T06:09:03.458 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=14 2024-09-21T06:09:03.458 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=7 2024-09-21T06:09:03.458 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-21T06:09:03.458 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T06:09:03.458 DEBUG:tasks.check_counter:Getting stats from f 2024-09-21T06:09:03.458 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph daemon mds.f perf dump 2024-09-21T06:09:03.709 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:09:04.174 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:03 smithi192 ceph-mon[34700]: pgmap v688: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-21T06:09:04.223 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-21T06:09:04.224 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:09:04.224 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 432, 2024-09-21T06:09:04.224 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 418, 2024-09-21T06:09:04.224 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 968983, 2024-09-21T06:09:04.224 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 219504, 2024-09-21T06:09:04.224 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 10, 2024-09-21T06:09:04.224 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 3, 2024-09-21T06:09:04.224 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.069516991, 2024-09-21T06:09:04.224 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.028964536, 2024-09-21T06:09:04.224 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.048812110, 2024-09-21T06:09:04.224 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.000456028, 2024-09-21T06:09:04.224 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:09:04.224 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 419, 2024-09-21T06:09:04.224 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.037944562, 2024-09-21T06:09:04.225 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000090559 2024-09-21T06:09:04.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.225 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:09:04.225 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 242, 2024-09-21T06:09:04.225 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000081539, 2024-09-21T06:09:04.225 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000336 2024-09-21T06:09:04.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.225 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 903440, 2024-09-21T06:09:04.225 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 149808 2024-09-21T06:09:04.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.225 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:09:04.225 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 44, 2024-09-21T06:09:04.225 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 288, 2024-09-21T06:09:04.225 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 9219, 2024-09-21T06:09:04.226 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 492995, 2024-09-21T06:09:04.226 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 16, 2024-09-21T06:09:04.226 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 7, 2024-09-21T06:09:04.226 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.033955114, 2024-09-21T06:09:04.226 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.018770568, 2024-09-21T06:09:04.226 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.012277417, 2024-09-21T06:09:04.226 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.000358804, 2024-09-21T06:09:04.226 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:09:04.226 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 284, 2024-09-21T06:09:04.226 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.033977670, 2024-09-21T06:09:04.226 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000119639 2024-09-21T06:09:04.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.226 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:09:04.226 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 117, 2024-09-21T06:09:04.227 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000026215, 2024-09-21T06:09:04.227 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000224 2024-09-21T06:09:04.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.227 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 3456, 2024-09-21T06:09:04.227 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 403856 2024-09-21T06:09:04.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.227 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:09:04.227 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 1509, 2024-09-21T06:09:04.227 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 1020, 2024-09-21T06:09:04.227 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 438972, 2024-09-21T06:09:04.227 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 771029, 2024-09-21T06:09:04.227 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 32, 2024-09-21T06:09:04.227 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 6, 2024-09-21T06:09:04.227 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.142167691, 2024-09-21T06:09:04.228 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.057392113, 2024-09-21T06:09:04.228 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.104055271, 2024-09-21T06:09:04.228 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.005568604, 2024-09-21T06:09:04.228 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:09:04.228 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1019, 2024-09-21T06:09:04.228 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.081639810, 2024-09-21T06:09:04.228 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000080117 2024-09-21T06:09:04.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.228 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:09:04.228 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 2301, 2024-09-21T06:09:04.228 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000552055, 2024-09-21T06:09:04.228 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000239 2024-09-21T06:09:04.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.229 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-21T06:09:04.229 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T06:09:04.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.229 INFO:teuthology.orchestra.run.smithi192.stdout: "cct": { 2024-09-21T06:09:04.229 INFO:teuthology.orchestra.run.smithi192.stdout: "total_workers": 1, 2024-09-21T06:09:04.229 INFO:teuthology.orchestra.run.smithi192.stdout: "unhealthy_workers": 0 2024-09-21T06:09:04.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.229 INFO:teuthology.orchestra.run.smithi192.stdout: "finisher-MDSRank": { 2024-09-21T06:09:04.229 INFO:teuthology.orchestra.run.smithi192.stdout: "queue_len": 0, 2024-09-21T06:09:04.229 INFO:teuthology.orchestra.run.smithi192.stdout: "complete_latency": { 2024-09-21T06:09:04.229 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 43, 2024-09-21T06:09:04.229 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.002247842, 2024-09-21T06:09:04.229 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000052275 2024-09-21T06:09:04.229 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:04.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.230 INFO:teuthology.orchestra.run.smithi192.stdout: "finisher-PurgeQueue": { 2024-09-21T06:09:04.230 INFO:teuthology.orchestra.run.smithi192.stdout: "queue_len": 0, 2024-09-21T06:09:04.230 INFO:teuthology.orchestra.run.smithi192.stdout: "complete_latency": { 2024-09-21T06:09:04.230 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 2, 2024-09-21T06:09:04.230 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000033580, 2024-09-21T06:09:04.230 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000016790 2024-09-21T06:09:04.230 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:04.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.230 INFO:teuthology.orchestra.run.smithi192.stdout: "mds": { 2024-09-21T06:09:04.230 INFO:teuthology.orchestra.run.smithi192.stdout: "request": 6, 2024-09-21T06:09:04.230 INFO:teuthology.orchestra.run.smithi192.stdout: "reply": 4, 2024-09-21T06:09:04.230 INFO:teuthology.orchestra.run.smithi192.stdout: "reply_latency": { 2024-09-21T06:09:04.230 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 4, 2024-09-21T06:09:04.230 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000786619, 2024-09-21T06:09:04.230 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000196654 2024-09-21T06:09:04.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.231 INFO:teuthology.orchestra.run.smithi192.stdout: "slow_reply": 0, 2024-09-21T06:09:04.231 INFO:teuthology.orchestra.run.smithi192.stdout: "forward": 2, 2024-09-21T06:09:04.231 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_fetch_complete": 0, 2024-09-21T06:09:04.231 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_fetch_keys": 0, 2024-09-21T06:09:04.231 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_commit": 11, 2024-09-21T06:09:04.231 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_split": 0, 2024-09-21T06:09:04.231 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_merge": 0, 2024-09-21T06:09:04.231 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes": 16, 2024-09-21T06:09:04.231 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_top": 0, 2024-09-21T06:09:04.231 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_bottom": 16, 2024-09-21T06:09:04.231 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_pin_tail": 0, 2024-09-21T06:09:04.231 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_pinned": 13, 2024-09-21T06:09:04.231 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_expired": 8, 2024-09-21T06:09:04.232 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_with_caps": 1, 2024-09-21T06:09:04.232 INFO:teuthology.orchestra.run.smithi192.stdout: "caps": 2, 2024-09-21T06:09:04.232 INFO:teuthology.orchestra.run.smithi192.stdout: "subtrees": 3, 2024-09-21T06:09:04.232 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse": 16, 2024-09-21T06:09:04.232 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_hit": 11, 2024-09-21T06:09:04.232 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_forward": 0, 2024-09-21T06:09:04.232 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_discover": 4, 2024-09-21T06:09:04.232 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_dir_fetch": 0, 2024-09-21T06:09:04.232 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_remote_ino": 0, 2024-09-21T06:09:04.232 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_lock": 0, 2024-09-21T06:09:04.232 INFO:teuthology.orchestra.run.smithi192.stdout: "load_cent": 0, 2024-09-21T06:09:04.232 INFO:teuthology.orchestra.run.smithi192.stdout: "q": 0, 2024-09-21T06:09:04.232 INFO:teuthology.orchestra.run.smithi192.stdout: "exported": 0, 2024-09-21T06:09:04.232 INFO:teuthology.orchestra.run.smithi192.stdout: "exported_inodes": 0, 2024-09-21T06:09:04.232 INFO:teuthology.orchestra.run.smithi192.stdout: "imported": 0, 2024-09-21T06:09:04.232 INFO:teuthology.orchestra.run.smithi192.stdout: "imported_inodes": 0, 2024-09-21T06:09:04.233 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_dir_fetch": 0, 2024-09-21T06:09:04.233 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_backtrace_fetch": 0, 2024-09-21T06:09:04.233 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_peer_discover": 0, 2024-09-21T06:09:04.233 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rfiles": 22, 2024-09-21T06:09:04.233 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rbytes": 10738142506, 2024-09-21T06:09:04.233 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rsnaps": 7, 2024-09-21T06:09:04.233 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T06:09:04.233 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_set_tag": 0, 2024-09-21T06:09:04.233 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T06:09:04.233 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_inotable_repaired": 0, 2024-09-21T06:09:04.233 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dir_inodes": 0, 2024-09-21T06:09:04.233 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T06:09:04.233 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T06:09:04.233 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_file_inodes": 0, 2024-09-21T06:09:04.234 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_inode_file_caps": 0, 2024-09-21T06:09:04.234 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T06:09:04.234 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_grant": 0, 2024-09-21T06:09:04.234 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T06:09:04.234 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T06:09:04.234 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T06:09:04.234 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_caps": 0, 2024-09-21T06:09:04.234 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_caps_dirty": 0, 2024-09-21T06:09:04.234 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_cap_release": 0, 2024-09-21T06:09:04.234 INFO:teuthology.orchestra.run.smithi192.stdout: "process_request_cap_release": 0 2024-09-21T06:09:04.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.234 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_cache": { 2024-09-21T06:09:04.234 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_update": 0, 2024-09-21T06:09:04.234 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_update_receipt": 14, 2024-09-21T06:09:04.235 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_try_discover": 9, 2024-09-21T06:09:04.235 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_send_discover": 10, 2024-09-21T06:09:04.235 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_handle_discover": 0, 2024-09-21T06:09:04.235 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays": 0, 2024-09-21T06:09:04.235 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays_delayed": 0, 2024-09-21T06:09:04.235 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays_enqueuing": 0, 2024-09-21T06:09:04.235 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_created": 0, 2024-09-21T06:09:04.235 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_enqueued": 0, 2024-09-21T06:09:04.235 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_reintegrated": 0, 2024-09-21T06:09:04.235 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_migrated": 0, 2024-09-21T06:09:04.235 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_processing": 0, 2024-09-21T06:09:04.235 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_enqueued": 0, 2024-09-21T06:09:04.235 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_prioritized": 0, 2024-09-21T06:09:04.235 INFO:teuthology.orchestra.run.smithi192.stdout: "recovery_started": 0, 2024-09-21T06:09:04.236 INFO:teuthology.orchestra.run.smithi192.stdout: "recovery_completed": 0, 2024-09-21T06:09:04.236 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_quiesce_path": 0, 2024-09-21T06:09:04.236 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_quiesce_inode": 0, 2024-09-21T06:09:04.236 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T06:09:04.236 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_exportdir": 0, 2024-09-21T06:09:04.236 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_flush": 0, 2024-09-21T06:09:04.236 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_fragmentdir": 0, 2024-09-21T06:09:04.236 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_fragstats": 0, 2024-09-21T06:09:04.236 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_inodestats": 0, 2024-09-21T06:09:04.236 INFO:teuthology.orchestra.run.smithi192.stdout: "uninline_started": 0, 2024-09-21T06:09:04.236 INFO:teuthology.orchestra.run.smithi192.stdout: "uninline_succeeded": 0, 2024-09-21T06:09:04.236 INFO:teuthology.orchestra.run.smithi192.stdout: "uninline_write_failed": 0 2024-09-21T06:09:04.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.236 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_log": { 2024-09-21T06:09:04.236 INFO:teuthology.orchestra.run.smithi192.stdout: "evlrg": 0, 2024-09-21T06:09:04.236 INFO:teuthology.orchestra.run.smithi192.stdout: "evadd": 6, 2024-09-21T06:09:04.237 INFO:teuthology.orchestra.run.smithi192.stdout: "evex": 0, 2024-09-21T06:09:04.237 INFO:teuthology.orchestra.run.smithi192.stdout: "evtrm": 0, 2024-09-21T06:09:04.237 INFO:teuthology.orchestra.run.smithi192.stdout: "ev": 6, 2024-09-21T06:09:04.237 INFO:teuthology.orchestra.run.smithi192.stdout: "evexg": 0, 2024-09-21T06:09:04.237 INFO:teuthology.orchestra.run.smithi192.stdout: "evexd": 0, 2024-09-21T06:09:04.237 INFO:teuthology.orchestra.run.smithi192.stdout: "segadd": 2, 2024-09-21T06:09:04.237 INFO:teuthology.orchestra.run.smithi192.stdout: "segex": 0, 2024-09-21T06:09:04.237 INFO:teuthology.orchestra.run.smithi192.stdout: "segtrm": 0, 2024-09-21T06:09:04.237 INFO:teuthology.orchestra.run.smithi192.stdout: "seg": 2, 2024-09-21T06:09:04.237 INFO:teuthology.orchestra.run.smithi192.stdout: "segmjr": 2, 2024-09-21T06:09:04.237 INFO:teuthology.orchestra.run.smithi192.stdout: "segexg": 0, 2024-09-21T06:09:04.237 INFO:teuthology.orchestra.run.smithi192.stdout: "segexd": 0, 2024-09-21T06:09:04.237 INFO:teuthology.orchestra.run.smithi192.stdout: "expos": 4194304, 2024-09-21T06:09:04.237 INFO:teuthology.orchestra.run.smithi192.stdout: "wrpos": 4196123, 2024-09-21T06:09:04.237 INFO:teuthology.orchestra.run.smithi192.stdout: "rdpos": 0, 2024-09-21T06:09:04.237 INFO:teuthology.orchestra.run.smithi192.stdout: "jlat": { 2024-09-21T06:09:04.238 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 5, 2024-09-21T06:09:04.238 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.008819808, 2024-09-21T06:09:04.238 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.001763961 2024-09-21T06:09:04.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.238 INFO:teuthology.orchestra.run.smithi192.stdout: "replayed": 0 2024-09-21T06:09:04.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.238 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_mem": { 2024-09-21T06:09:04.238 INFO:teuthology.orchestra.run.smithi192.stdout: "ino": 20, 2024-09-21T06:09:04.238 INFO:teuthology.orchestra.run.smithi192.stdout: "ino+": 24, 2024-09-21T06:09:04.238 INFO:teuthology.orchestra.run.smithi192.stdout: "ino-": 4, 2024-09-21T06:09:04.238 INFO:teuthology.orchestra.run.smithi192.stdout: "dir": 19, 2024-09-21T06:09:04.238 INFO:teuthology.orchestra.run.smithi192.stdout: "dir+": 25, 2024-09-21T06:09:04.238 INFO:teuthology.orchestra.run.smithi192.stdout: "dir-": 6, 2024-09-21T06:09:04.238 INFO:teuthology.orchestra.run.smithi192.stdout: "dn": 16, 2024-09-21T06:09:04.238 INFO:teuthology.orchestra.run.smithi192.stdout: "dn+": 24, 2024-09-21T06:09:04.239 INFO:teuthology.orchestra.run.smithi192.stdout: "dn-": 8, 2024-09-21T06:09:04.239 INFO:teuthology.orchestra.run.smithi192.stdout: "cap": 2, 2024-09-21T06:09:04.239 INFO:teuthology.orchestra.run.smithi192.stdout: "cap+": 2, 2024-09-21T06:09:04.239 INFO:teuthology.orchestra.run.smithi192.stdout: "cap-": 0, 2024-09-21T06:09:04.239 INFO:teuthology.orchestra.run.smithi192.stdout: "rss": 118120, 2024-09-21T06:09:04.239 INFO:teuthology.orchestra.run.smithi192.stdout: "heap": 325857280 2024-09-21T06:09:04.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.239 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_server": { 2024-09-21T06:09:04.239 INFO:teuthology.orchestra.run.smithi192.stdout: "dispatch_client_request": 6, 2024-09-21T06:09:04.239 INFO:teuthology.orchestra.run.smithi192.stdout: "dispatch_server_request": 0, 2024-09-21T06:09:04.239 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_request": 6, 2024-09-21T06:09:04.239 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_session": 23, 2024-09-21T06:09:04.239 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_peer_request": 0, 2024-09-21T06:09:04.239 INFO:teuthology.orchestra.run.smithi192.stdout: "req_create_latency": { 2024-09-21T06:09:04.239 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:04.240 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:04.240 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:04.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.240 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getattr_latency": { 2024-09-21T06:09:04.240 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 3, 2024-09-21T06:09:04.240 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000682563, 2024-09-21T06:09:04.240 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000227521 2024-09-21T06:09:04.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.240 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getfilelock_latency": { 2024-09-21T06:09:04.240 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:04.240 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:04.240 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:04.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.240 INFO:teuthology.orchestra.run.smithi192.stdout: "req_link_latency": { 2024-09-21T06:09:04.240 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:04.240 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:04.241 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:04.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.241 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookup_latency": { 2024-09-21T06:09:04.241 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:04.241 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:04.241 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:04.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.241 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookuphash_latency": { 2024-09-21T06:09:04.241 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:04.241 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:04.241 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:04.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.241 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupino_latency": { 2024-09-21T06:09:04.241 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1, 2024-09-21T06:09:04.241 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000104056, 2024-09-21T06:09:04.242 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000104056 2024-09-21T06:09:04.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.242 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupname_latency": { 2024-09-21T06:09:04.242 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:04.242 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:04.242 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:04.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.242 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupparent_latency": { 2024-09-21T06:09:04.242 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:04.242 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:04.242 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:04.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.242 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupsnap_latency": { 2024-09-21T06:09:04.242 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:04.242 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:04.243 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:04.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.243 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lssnap_latency": { 2024-09-21T06:09:04.243 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:04.243 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:04.243 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:04.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.244 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mkdir_latency": { 2024-09-21T06:09:04.244 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:04.244 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:04.244 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:04.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.244 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mknod_latency": { 2024-09-21T06:09:04.244 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:04.244 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:04.244 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:04.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.244 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mksnap_latency": { 2024-09-21T06:09:04.244 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:04.244 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:04.244 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:04.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.245 INFO:teuthology.orchestra.run.smithi192.stdout: "req_open_latency": { 2024-09-21T06:09:04.245 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:04.245 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:04.245 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:04.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.245 INFO:teuthology.orchestra.run.smithi192.stdout: "req_readdir_latency": { 2024-09-21T06:09:04.245 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:04.245 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:04.245 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:04.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.245 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rename_latency": { 2024-09-21T06:09:04.245 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:04.245 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:04.245 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:04.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.246 INFO:teuthology.orchestra.run.smithi192.stdout: "req_renamesnap_latency": { 2024-09-21T06:09:04.246 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:04.246 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:04.246 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:04.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.246 INFO:teuthology.orchestra.run.smithi192.stdout: "req_snapdiff_latency": { 2024-09-21T06:09:04.246 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:04.246 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:04.246 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:04.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.246 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmdir_latency": { 2024-09-21T06:09:04.246 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:04.246 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:04.246 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:04.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.247 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmsnap_latency": { 2024-09-21T06:09:04.247 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:04.247 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:04.247 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:04.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.247 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmxattr_latency": { 2024-09-21T06:09:04.247 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:04.247 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:04.247 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:04.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.247 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setattr_latency": { 2024-09-21T06:09:04.247 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:04.247 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:04.247 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:04.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.248 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setdirlayout_latency": { 2024-09-21T06:09:04.248 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:04.248 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:04.248 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:04.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.248 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setfilelock_latency": { 2024-09-21T06:09:04.248 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:04.248 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:04.248 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:04.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.248 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setlayout_latency": { 2024-09-21T06:09:04.248 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:04.248 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:04.248 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:04.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.248 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setxattr_latency": { 2024-09-21T06:09:04.249 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:04.249 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:04.249 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:04.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.249 INFO:teuthology.orchestra.run.smithi192.stdout: "req_symlink_latency": { 2024-09-21T06:09:04.249 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:04.249 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:04.249 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:04.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.249 INFO:teuthology.orchestra.run.smithi192.stdout: "req_unlink_latency": { 2024-09-21T06:09:04.249 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:04.249 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:04.249 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:04.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.249 INFO:teuthology.orchestra.run.smithi192.stdout: "cap_revoke_eviction": 0, 2024-09-21T06:09:04.250 INFO:teuthology.orchestra.run.smithi192.stdout: "cap_acquisition_throttle": 0, 2024-09-21T06:09:04.250 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getvxattr_latency": { 2024-09-21T06:09:04.250 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:04.250 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:04.250 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:04.250 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:04.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.250 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_sessions": { 2024-09-21T06:09:04.250 INFO:teuthology.orchestra.run.smithi192.stdout: "session_count": 2, 2024-09-21T06:09:04.250 INFO:teuthology.orchestra.run.smithi192.stdout: "session_add": 3, 2024-09-21T06:09:04.250 INFO:teuthology.orchestra.run.smithi192.stdout: "session_remove": 1, 2024-09-21T06:09:04.250 INFO:teuthology.orchestra.run.smithi192.stdout: "sessions_open": 2, 2024-09-21T06:09:04.250 INFO:teuthology.orchestra.run.smithi192.stdout: "sessions_stale": 0, 2024-09-21T06:09:04.250 INFO:teuthology.orchestra.run.smithi192.stdout: "total_load": 3, 2024-09-21T06:09:04.250 INFO:teuthology.orchestra.run.smithi192.stdout: "average_load": 1, 2024-09-21T06:09:04.251 INFO:teuthology.orchestra.run.smithi192.stdout: "avg_session_uptime": 683, 2024-09-21T06:09:04.251 INFO:teuthology.orchestra.run.smithi192.stdout: "mdthresh_evicted": 0 2024-09-21T06:09:04.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.251 INFO:teuthology.orchestra.run.smithi192.stdout: "mempool": { 2024-09-21T06:09:04.251 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:09:04.251 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_items": 0, 2024-09-21T06:09:04.251 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:09:04.251 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:09:04.251 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:09:04.251 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:09:04.251 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:09:04.251 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:09:04.251 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:09:04.251 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:09:04.251 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:09:04.252 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:09:04.252 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:09:04.252 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:09:04.252 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:09:04.252 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_items": 0, 2024-09-21T06:09:04.252 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:09:04.252 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_items": 0, 2024-09-21T06:09:04.252 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:09:04.252 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:09:04.252 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:09:04.252 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:09:04.252 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:09:04.252 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:09:04.252 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:09:04.252 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_items": 0, 2024-09-21T06:09:04.252 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:09:04.253 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:09:04.253 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:09:04.253 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_items": 0, 2024-09-21T06:09:04.253 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_bytes": 0, 2024-09-21T06:09:04.253 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_items": 0, 2024-09-21T06:09:04.253 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:09:04.253 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:09:04.253 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:09:04.253 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:09:04.253 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_bytes": 255475, 2024-09-21T06:09:04.253 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_items": 93, 2024-09-21T06:09:04.253 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_bytes": 88, 2024-09-21T06:09:04.253 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_items": 1, 2024-09-21T06:09:04.253 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_bytes": 0, 2024-09-21T06:09:04.253 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_items": 0, 2024-09-21T06:09:04.253 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:09:04.254 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_items": 0, 2024-09-21T06:09:04.254 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:09:04.254 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_items": 0, 2024-09-21T06:09:04.254 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_bytes": 7968, 2024-09-21T06:09:04.254 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_items": 176, 2024-09-21T06:09:04.254 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:09:04.254 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:09:04.254 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_bytes": 0, 2024-09-21T06:09:04.254 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_items": 0, 2024-09-21T06:09:04.254 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_bytes": 84109, 2024-09-21T06:09:04.254 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_items": 387, 2024-09-21T06:09:04.254 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_bytes": 0, 2024-09-21T06:09:04.254 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_items": 0, 2024-09-21T06:09:04.254 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_bytes": 0, 2024-09-21T06:09:04.254 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_items": 0 2024-09-21T06:09:04.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.255 INFO:teuthology.orchestra.run.smithi192.stdout: "objecter": { 2024-09-21T06:09:04.255 INFO:teuthology.orchestra.run.smithi192.stdout: "op_active": 0, 2024-09-21T06:09:04.255 INFO:teuthology.orchestra.run.smithi192.stdout: "op_laggy": 0, 2024-09-21T06:09:04.255 INFO:teuthology.orchestra.run.smithi192.stdout: "op_send": 41, 2024-09-21T06:09:04.255 INFO:teuthology.orchestra.run.smithi192.stdout: "op_send_bytes": 12121, 2024-09-21T06:09:04.255 INFO:teuthology.orchestra.run.smithi192.stdout: "op_resend": 33, 2024-09-21T06:09:04.255 INFO:teuthology.orchestra.run.smithi192.stdout: "op_reply": 41, 2024-09-21T06:09:04.255 INFO:teuthology.orchestra.run.smithi192.stdout: "op_latency": { 2024-09-21T06:09:04.255 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 41, 2024-09-21T06:09:04.255 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.221995850, 2024-09-21T06:09:04.255 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.005414532 2024-09-21T06:09:04.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.255 INFO:teuthology.orchestra.run.smithi192.stdout: "op_inflight": 0, 2024-09-21T06:09:04.255 INFO:teuthology.orchestra.run.smithi192.stdout: "oplen_avg": { 2024-09-21T06:09:04.256 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 41, 2024-09-21T06:09:04.256 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 63 2024-09-21T06:09:04.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.256 INFO:teuthology.orchestra.run.smithi192.stdout: "op": 41, 2024-09-21T06:09:04.256 INFO:teuthology.orchestra.run.smithi192.stdout: "op_r": 0, 2024-09-21T06:09:04.256 INFO:teuthology.orchestra.run.smithi192.stdout: "op_w": 41, 2024-09-21T06:09:04.256 INFO:teuthology.orchestra.run.smithi192.stdout: "op_rmw": 0, 2024-09-21T06:09:04.256 INFO:teuthology.orchestra.run.smithi192.stdout: "op_pg": 0, 2024-09-21T06:09:04.256 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_stat": 0, 2024-09-21T06:09:04.256 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_create": 10, 2024-09-21T06:09:04.256 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_read": 0, 2024-09-21T06:09:04.256 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_write": 5, 2024-09-21T06:09:04.256 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_writefull": 7, 2024-09-21T06:09:04.256 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_writesame": 0, 2024-09-21T06:09:04.256 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_append": 0, 2024-09-21T06:09:04.257 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_zero": 0, 2024-09-21T06:09:04.257 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_truncate": 0, 2024-09-21T06:09:04.257 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_delete": 6, 2024-09-21T06:09:04.257 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_mapext": 0, 2024-09-21T06:09:04.257 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_sparse_read": 0, 2024-09-21T06:09:04.257 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_clonerange": 0, 2024-09-21T06:09:04.257 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_getxattr": 0, 2024-09-21T06:09:04.257 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_setxattr": 20, 2024-09-21T06:09:04.257 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_cmpxattr": 0, 2024-09-21T06:09:04.257 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_rmxattr": 0, 2024-09-21T06:09:04.257 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_resetxattrs": 0, 2024-09-21T06:09:04.257 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_call": 0, 2024-09-21T06:09:04.257 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_watch": 0, 2024-09-21T06:09:04.257 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_notify": 0, 2024-09-21T06:09:04.257 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T06:09:04.258 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_pgls": 0, 2024-09-21T06:09:04.258 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_pgls_filter": 0, 2024-09-21T06:09:04.258 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_other": 0, 2024-09-21T06:09:04.258 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_active": 0, 2024-09-21T06:09:04.258 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_send": 0, 2024-09-21T06:09:04.258 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_resend": 0, 2024-09-21T06:09:04.258 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_ping": 0, 2024-09-21T06:09:04.258 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_active": 0, 2024-09-21T06:09:04.258 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_send": 0, 2024-09-21T06:09:04.258 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_resend": 0, 2024-09-21T06:09:04.258 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_active": 0, 2024-09-21T06:09:04.258 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_send": 0, 2024-09-21T06:09:04.258 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_resend": 0, 2024-09-21T06:09:04.258 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_active": 0, 2024-09-21T06:09:04.258 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_send": 0, 2024-09-21T06:09:04.259 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_resend": 0, 2024-09-21T06:09:04.259 INFO:teuthology.orchestra.run.smithi192.stdout: "command_active": 0, 2024-09-21T06:09:04.259 INFO:teuthology.orchestra.run.smithi192.stdout: "command_send": 0, 2024-09-21T06:09:04.259 INFO:teuthology.orchestra.run.smithi192.stdout: "command_resend": 0, 2024-09-21T06:09:04.259 INFO:teuthology.orchestra.run.smithi192.stdout: "map_epoch": 85, 2024-09-21T06:09:04.259 INFO:teuthology.orchestra.run.smithi192.stdout: "map_full": 0, 2024-09-21T06:09:04.259 INFO:teuthology.orchestra.run.smithi192.stdout: "map_inc": 11, 2024-09-21T06:09:04.259 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_sessions": 8, 2024-09-21T06:09:04.259 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_session_open": 8, 2024-09-21T06:09:04.259 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_session_close": 0, 2024-09-21T06:09:04.259 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_laggy": 0, 2024-09-21T06:09:04.259 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_wr": 14, 2024-09-21T06:09:04.259 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_rd": 0, 2024-09-21T06:09:04.259 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_del": 1 2024-09-21T06:09:04.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.259 INFO:teuthology.orchestra.run.smithi192.stdout: "oft": { 2024-09-21T06:09:04.260 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_objs": 1, 2024-09-21T06:09:04.260 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_kv_pairs": 0, 2024-09-21T06:09:04.260 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_updates": 0, 2024-09-21T06:09:04.260 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_removes": 0 2024-09-21T06:09:04.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.260 INFO:teuthology.orchestra.run.smithi192.stdout: "purge_queue": { 2024-09-21T06:09:04.260 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_ops": 0, 2024-09-21T06:09:04.260 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T06:09:04.260 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing": 0, 2024-09-21T06:09:04.260 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_high_water": 0, 2024-09-21T06:09:04.260 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executed_ops": 0, 2024-09-21T06:09:04.260 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executed": 0, 2024-09-21T06:09:04.260 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_item_in_journal": 0 2024-09-21T06:09:04.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.260 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:09:04.261 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-21T06:09:04.261 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-09-21T06:09:04.261 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-21T06:09:04.261 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 1981, 2024-09-21T06:09:04.261 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 1220522, 2024-09-21T06:09:04.261 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-21T06:09:04.261 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 1981, 2024-09-21T06:09:04.261 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-09-21T06:09:04.261 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-09-21T06:09:04.261 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 1981, 2024-09-21T06:09:04.261 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 1220522, 2024-09-21T06:09:04.261 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-21T06:09:04.261 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:04.261 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:04.261 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:04.261 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:04.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.262 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-objecter_bytes": { 2024-09-21T06:09:04.262 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-21T06:09:04.262 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-09-21T06:09:04.262 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-21T06:09:04.262 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-09-21T06:09:04.262 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-09-21T06:09:04.262 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-21T06:09:04.262 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-09-21T06:09:04.262 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 41, 2024-09-21T06:09:04.262 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 12121, 2024-09-21T06:09:04.262 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 41, 2024-09-21T06:09:04.262 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 12121, 2024-09-21T06:09:04.262 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-21T06:09:04.262 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:04.263 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:04.263 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:04.263 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:04.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.263 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-objecter_ops": { 2024-09-21T06:09:04.263 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-21T06:09:04.263 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 1024, 2024-09-21T06:09:04.263 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-21T06:09:04.263 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-09-21T06:09:04.263 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-09-21T06:09:04.263 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-21T06:09:04.263 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-09-21T06:09:04.263 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 41, 2024-09-21T06:09:04.263 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 41, 2024-09-21T06:09:04.263 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 41, 2024-09-21T06:09:04.264 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 41, 2024-09-21T06:09:04.264 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-21T06:09:04.264 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:04.264 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:04.264 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:04.264 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:04.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.264 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-write_buf_throttle": { 2024-09-21T06:09:04.264 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-21T06:09:04.264 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 3758096384, 2024-09-21T06:09:04.264 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-21T06:09:04.264 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-09-21T06:09:04.264 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-09-21T06:09:04.264 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-21T06:09:04.264 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-09-21T06:09:04.264 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-09-21T06:09:04.265 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-09-21T06:09:04.265 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 0, 2024-09-21T06:09:04.265 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 0, 2024-09-21T06:09:04.265 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-21T06:09:04.265 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:04.265 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:04.265 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:04.265 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:04.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.265 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-write_buf_throttle-0x563cace19450": { 2024-09-21T06:09:04.265 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-21T06:09:04.265 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 3758096384, 2024-09-21T06:09:04.265 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-21T06:09:04.265 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 6, 2024-09-21T06:09:04.265 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 1819, 2024-09-21T06:09:04.265 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-21T06:09:04.266 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 6, 2024-09-21T06:09:04.266 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-09-21T06:09:04.266 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-09-21T06:09:04.266 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 5, 2024-09-21T06:09:04.266 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 1819, 2024-09-21T06:09:04.266 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-21T06:09:04.266 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:04.266 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:04.266 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:04.266 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:04.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:04.266 INFO:teuthology.orchestra.run.smithi192.stdout: "trackedop": { 2024-09-21T06:09:04.266 INFO:teuthology.orchestra.run.smithi192.stdout: "slow_ops_count": 0 2024-09-21T06:09:04.266 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:04.266 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-21T06:09:04.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:03 smithi117 ceph-mon[30947]: pgmap v688: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-21T06:09:04.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:03 smithi138 ceph-mon[34391]: pgmap v688: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 23 op/s 2024-09-21T06:09:05.128 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-21T06:09:05.128 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-21T06:09:05.128 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-09-21T06:09:05.128 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=14 2024-09-21T06:09:05.128 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=7 2024-09-21T06:09:05.128 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-21T06:09:05.128 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T06:09:05.128 DEBUG:tasks.check_counter:Getting stats from i 2024-09-21T06:09:05.128 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph daemon mds.i perf dump 2024-09-21T06:09:05.376 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:09:05.931 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-21T06:09:05.932 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:09:05.932 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 1501, 2024-09-21T06:09:05.932 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 1008, 2024-09-21T06:09:05.932 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 437468, 2024-09-21T06:09:05.932 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 242297, 2024-09-21T06:09:05.932 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 9, 2024-09-21T06:09:05.932 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 4, 2024-09-21T06:09:05.932 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.137371906, 2024-09-21T06:09:05.932 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.056107657, 2024-09-21T06:09:05.932 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.094960284, 2024-09-21T06:09:05.932 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.006261291, 2024-09-21T06:09:05.932 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:09:05.933 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1007, 2024-09-21T06:09:05.933 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.077207848, 2024-09-21T06:09:05.933 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000076671 2024-09-21T06:09:05.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.933 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:09:05.933 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 2322, 2024-09-21T06:09:05.933 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000692145, 2024-09-21T06:09:05.933 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000298 2024-09-21T06:09:05.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.933 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-21T06:09:05.933 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T06:09:05.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.933 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:09:05.933 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 105, 2024-09-21T06:09:05.934 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 278, 2024-09-21T06:09:05.934 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 38951, 2024-09-21T06:09:05.934 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 749903, 2024-09-21T06:09:05.934 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 37, 2024-09-21T06:09:05.934 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 7, 2024-09-21T06:09:05.934 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.035431836, 2024-09-21T06:09:05.934 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.014617670, 2024-09-21T06:09:05.934 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.018632978, 2024-09-21T06:09:05.934 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.000235411, 2024-09-21T06:09:05.934 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:09:05.934 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 276, 2024-09-21T06:09:05.934 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.025701136, 2024-09-21T06:09:05.934 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000093120 2024-09-21T06:09:05.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.935 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:09:05.935 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 158, 2024-09-21T06:09:05.935 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000047551, 2024-09-21T06:09:05.935 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000300 2024-09-21T06:09:05.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.935 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 1744, 2024-09-21T06:09:05.935 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 393872 2024-09-21T06:09:05.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.935 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:09:05.935 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 358, 2024-09-21T06:09:05.935 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 342, 2024-09-21T06:09:05.935 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 943453, 2024-09-21T06:09:05.935 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 236479, 2024-09-21T06:09:05.935 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 11, 2024-09-21T06:09:05.935 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 5, 2024-09-21T06:09:05.936 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.056910202, 2024-09-21T06:09:05.936 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.025438499, 2024-09-21T06:09:05.936 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.040351921, 2024-09-21T06:09:05.936 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.000250737, 2024-09-21T06:09:05.936 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:09:05.936 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 341, 2024-09-21T06:09:05.936 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.030198677, 2024-09-21T06:09:05.936 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000088559 2024-09-21T06:09:05.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.936 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:09:05.936 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 112, 2024-09-21T06:09:05.936 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000026054, 2024-09-21T06:09:05.936 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000232 2024-09-21T06:09:05.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.937 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 910000, 2024-09-21T06:09:05.937 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 146832 2024-09-21T06:09:05.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.937 INFO:teuthology.orchestra.run.smithi192.stdout: "cct": { 2024-09-21T06:09:05.937 INFO:teuthology.orchestra.run.smithi192.stdout: "total_workers": 1, 2024-09-21T06:09:05.937 INFO:teuthology.orchestra.run.smithi192.stdout: "unhealthy_workers": 0 2024-09-21T06:09:05.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.937 INFO:teuthology.orchestra.run.smithi192.stdout: "finisher-MDSRank": { 2024-09-21T06:09:05.937 INFO:teuthology.orchestra.run.smithi192.stdout: "queue_len": 0, 2024-09-21T06:09:05.937 INFO:teuthology.orchestra.run.smithi192.stdout: "complete_latency": { 2024-09-21T06:09:05.937 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 36, 2024-09-21T06:09:05.937 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.001568675, 2024-09-21T06:09:05.937 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000043574 2024-09-21T06:09:05.937 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:05.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.938 INFO:teuthology.orchestra.run.smithi192.stdout: "finisher-PurgeQueue": { 2024-09-21T06:09:05.938 INFO:teuthology.orchestra.run.smithi192.stdout: "queue_len": 0, 2024-09-21T06:09:05.938 INFO:teuthology.orchestra.run.smithi192.stdout: "complete_latency": { 2024-09-21T06:09:05.938 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 2, 2024-09-21T06:09:05.938 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000019203, 2024-09-21T06:09:05.938 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000009601 2024-09-21T06:09:05.938 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:05.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.938 INFO:teuthology.orchestra.run.smithi192.stdout: "mds": { 2024-09-21T06:09:05.938 INFO:teuthology.orchestra.run.smithi192.stdout: "request": 2, 2024-09-21T06:09:05.938 INFO:teuthology.orchestra.run.smithi192.stdout: "reply": 1, 2024-09-21T06:09:05.938 INFO:teuthology.orchestra.run.smithi192.stdout: "reply_latency": { 2024-09-21T06:09:05.938 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1, 2024-09-21T06:09:05.939 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000171227, 2024-09-21T06:09:05.939 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000171227 2024-09-21T06:09:05.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.939 INFO:teuthology.orchestra.run.smithi192.stdout: "slow_reply": 0, 2024-09-21T06:09:05.939 INFO:teuthology.orchestra.run.smithi192.stdout: "forward": 1, 2024-09-21T06:09:05.939 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_fetch_complete": 0, 2024-09-21T06:09:05.939 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_fetch_keys": 0, 2024-09-21T06:09:05.939 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_commit": 11, 2024-09-21T06:09:05.939 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_split": 0, 2024-09-21T06:09:05.939 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_merge": 0, 2024-09-21T06:09:05.939 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes": 16, 2024-09-21T06:09:05.939 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_top": 0, 2024-09-21T06:09:05.939 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_bottom": 16, 2024-09-21T06:09:05.939 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_pin_tail": 0, 2024-09-21T06:09:05.940 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_pinned": 13, 2024-09-21T06:09:05.940 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_expired": 8, 2024-09-21T06:09:05.940 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_with_caps": 0, 2024-09-21T06:09:05.940 INFO:teuthology.orchestra.run.smithi192.stdout: "caps": 0, 2024-09-21T06:09:05.940 INFO:teuthology.orchestra.run.smithi192.stdout: "subtrees": 3, 2024-09-21T06:09:05.940 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse": 12, 2024-09-21T06:09:05.940 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_hit": 6, 2024-09-21T06:09:05.940 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_forward": 0, 2024-09-21T06:09:05.940 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_discover": 5, 2024-09-21T06:09:05.940 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_dir_fetch": 0, 2024-09-21T06:09:05.940 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_remote_ino": 0, 2024-09-21T06:09:05.940 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_lock": 0, 2024-09-21T06:09:05.940 INFO:teuthology.orchestra.run.smithi192.stdout: "load_cent": 0, 2024-09-21T06:09:05.940 INFO:teuthology.orchestra.run.smithi192.stdout: "q": 0, 2024-09-21T06:09:05.940 INFO:teuthology.orchestra.run.smithi192.stdout: "exported": 0, 2024-09-21T06:09:05.941 INFO:teuthology.orchestra.run.smithi192.stdout: "exported_inodes": 0, 2024-09-21T06:09:05.941 INFO:teuthology.orchestra.run.smithi192.stdout: "imported": 0, 2024-09-21T06:09:05.941 INFO:teuthology.orchestra.run.smithi192.stdout: "imported_inodes": 0, 2024-09-21T06:09:05.941 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_dir_fetch": 0, 2024-09-21T06:09:05.941 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_backtrace_fetch": 0, 2024-09-21T06:09:05.941 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_peer_discover": 0, 2024-09-21T06:09:05.941 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rfiles": 22, 2024-09-21T06:09:05.941 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rbytes": 10738142506, 2024-09-21T06:09:05.941 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rsnaps": 7, 2024-09-21T06:09:05.941 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T06:09:05.941 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_set_tag": 0, 2024-09-21T06:09:05.941 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T06:09:05.941 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_inotable_repaired": 0, 2024-09-21T06:09:05.941 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dir_inodes": 0, 2024-09-21T06:09:05.942 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T06:09:05.942 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T06:09:05.942 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_file_inodes": 0, 2024-09-21T06:09:05.942 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_inode_file_caps": 0, 2024-09-21T06:09:05.942 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T06:09:05.942 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_grant": 0, 2024-09-21T06:09:05.942 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T06:09:05.942 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T06:09:05.942 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T06:09:05.942 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_caps": 0, 2024-09-21T06:09:05.942 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_caps_dirty": 0, 2024-09-21T06:09:05.942 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_cap_release": 0, 2024-09-21T06:09:05.942 INFO:teuthology.orchestra.run.smithi192.stdout: "process_request_cap_release": 1 2024-09-21T06:09:05.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.942 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_cache": { 2024-09-21T06:09:05.943 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_update": 0, 2024-09-21T06:09:05.943 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_update_receipt": 12, 2024-09-21T06:09:05.943 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_try_discover": 8, 2024-09-21T06:09:05.943 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_send_discover": 10, 2024-09-21T06:09:05.943 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_handle_discover": 0, 2024-09-21T06:09:05.943 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays": 0, 2024-09-21T06:09:05.943 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays_delayed": 0, 2024-09-21T06:09:05.943 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays_enqueuing": 0, 2024-09-21T06:09:05.943 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_created": 0, 2024-09-21T06:09:05.943 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_enqueued": 0, 2024-09-21T06:09:05.943 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_reintegrated": 0, 2024-09-21T06:09:05.943 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_migrated": 0, 2024-09-21T06:09:05.943 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_processing": 0, 2024-09-21T06:09:05.943 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_enqueued": 0, 2024-09-21T06:09:05.943 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_prioritized": 0, 2024-09-21T06:09:05.944 INFO:teuthology.orchestra.run.smithi192.stdout: "recovery_started": 0, 2024-09-21T06:09:05.944 INFO:teuthology.orchestra.run.smithi192.stdout: "recovery_completed": 0, 2024-09-21T06:09:05.944 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_quiesce_path": 0, 2024-09-21T06:09:05.944 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_quiesce_inode": 0, 2024-09-21T06:09:05.944 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T06:09:05.944 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_exportdir": 0, 2024-09-21T06:09:05.944 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_flush": 0, 2024-09-21T06:09:05.944 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_fragmentdir": 0, 2024-09-21T06:09:05.944 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_fragstats": 0, 2024-09-21T06:09:05.944 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_inodestats": 0, 2024-09-21T06:09:05.944 INFO:teuthology.orchestra.run.smithi192.stdout: "uninline_started": 0, 2024-09-21T06:09:05.944 INFO:teuthology.orchestra.run.smithi192.stdout: "uninline_succeeded": 0, 2024-09-21T06:09:05.944 INFO:teuthology.orchestra.run.smithi192.stdout: "uninline_write_failed": 0 2024-09-21T06:09:05.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.945 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_log": { 2024-09-21T06:09:05.945 INFO:teuthology.orchestra.run.smithi192.stdout: "evlrg": 0, 2024-09-21T06:09:05.945 INFO:teuthology.orchestra.run.smithi192.stdout: "evadd": 4, 2024-09-21T06:09:05.945 INFO:teuthology.orchestra.run.smithi192.stdout: "evex": 0, 2024-09-21T06:09:05.945 INFO:teuthology.orchestra.run.smithi192.stdout: "evtrm": 0, 2024-09-21T06:09:05.945 INFO:teuthology.orchestra.run.smithi192.stdout: "ev": 4, 2024-09-21T06:09:05.945 INFO:teuthology.orchestra.run.smithi192.stdout: "evexg": 0, 2024-09-21T06:09:05.945 INFO:teuthology.orchestra.run.smithi192.stdout: "evexd": 0, 2024-09-21T06:09:05.945 INFO:teuthology.orchestra.run.smithi192.stdout: "segadd": 2, 2024-09-21T06:09:05.945 INFO:teuthology.orchestra.run.smithi192.stdout: "segex": 0, 2024-09-21T06:09:05.945 INFO:teuthology.orchestra.run.smithi192.stdout: "segtrm": 0, 2024-09-21T06:09:05.945 INFO:teuthology.orchestra.run.smithi192.stdout: "seg": 2, 2024-09-21T06:09:05.945 INFO:teuthology.orchestra.run.smithi192.stdout: "segmjr": 2, 2024-09-21T06:09:05.945 INFO:teuthology.orchestra.run.smithi192.stdout: "segexg": 0, 2024-09-21T06:09:05.945 INFO:teuthology.orchestra.run.smithi192.stdout: "segexd": 0, 2024-09-21T06:09:05.945 INFO:teuthology.orchestra.run.smithi192.stdout: "expos": 4194304, 2024-09-21T06:09:05.946 INFO:teuthology.orchestra.run.smithi192.stdout: "wrpos": 4195424, 2024-09-21T06:09:05.946 INFO:teuthology.orchestra.run.smithi192.stdout: "rdpos": 0, 2024-09-21T06:09:05.946 INFO:teuthology.orchestra.run.smithi192.stdout: "jlat": { 2024-09-21T06:09:05.946 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 3, 2024-09-21T06:09:05.946 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.006115451, 2024-09-21T06:09:05.946 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.002038483 2024-09-21T06:09:05.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.946 INFO:teuthology.orchestra.run.smithi192.stdout: "replayed": 0 2024-09-21T06:09:05.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.946 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_mem": { 2024-09-21T06:09:05.946 INFO:teuthology.orchestra.run.smithi192.stdout: "ino": 20, 2024-09-21T06:09:05.946 INFO:teuthology.orchestra.run.smithi192.stdout: "ino+": 24, 2024-09-21T06:09:05.946 INFO:teuthology.orchestra.run.smithi192.stdout: "ino-": 4, 2024-09-21T06:09:05.946 INFO:teuthology.orchestra.run.smithi192.stdout: "dir": 19, 2024-09-21T06:09:05.947 INFO:teuthology.orchestra.run.smithi192.stdout: "dir+": 24, 2024-09-21T06:09:05.947 INFO:teuthology.orchestra.run.smithi192.stdout: "dir-": 5, 2024-09-21T06:09:05.947 INFO:teuthology.orchestra.run.smithi192.stdout: "dn": 16, 2024-09-21T06:09:05.947 INFO:teuthology.orchestra.run.smithi192.stdout: "dn+": 24, 2024-09-21T06:09:05.947 INFO:teuthology.orchestra.run.smithi192.stdout: "dn-": 8, 2024-09-21T06:09:05.947 INFO:teuthology.orchestra.run.smithi192.stdout: "cap": 0, 2024-09-21T06:09:05.947 INFO:teuthology.orchestra.run.smithi192.stdout: "cap+": 1, 2024-09-21T06:09:05.947 INFO:teuthology.orchestra.run.smithi192.stdout: "cap-": 1, 2024-09-21T06:09:05.947 INFO:teuthology.orchestra.run.smithi192.stdout: "rss": 120820, 2024-09-21T06:09:05.947 INFO:teuthology.orchestra.run.smithi192.stdout: "heap": 325865472 2024-09-21T06:09:05.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.947 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_server": { 2024-09-21T06:09:05.947 INFO:teuthology.orchestra.run.smithi192.stdout: "dispatch_client_request": 3, 2024-09-21T06:09:05.947 INFO:teuthology.orchestra.run.smithi192.stdout: "dispatch_server_request": 0, 2024-09-21T06:09:05.947 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_request": 2, 2024-09-21T06:09:05.948 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_session": 4, 2024-09-21T06:09:05.948 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_peer_request": 0, 2024-09-21T06:09:05.948 INFO:teuthology.orchestra.run.smithi192.stdout: "req_create_latency": { 2024-09-21T06:09:05.948 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:05.949 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:05.949 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:05.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.949 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getattr_latency": { 2024-09-21T06:09:05.949 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1, 2024-09-21T06:09:05.950 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000171227, 2024-09-21T06:09:05.950 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000171227 2024-09-21T06:09:05.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.950 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getfilelock_latency": { 2024-09-21T06:09:05.950 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:05.950 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:05.950 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:05.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.950 INFO:teuthology.orchestra.run.smithi192.stdout: "req_link_latency": { 2024-09-21T06:09:05.950 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:05.950 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:05.950 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:05.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.950 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookup_latency": { 2024-09-21T06:09:05.951 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:05.951 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:05.951 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:05.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.951 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookuphash_latency": { 2024-09-21T06:09:05.951 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:05.951 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:05.951 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:05.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.951 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupino_latency": { 2024-09-21T06:09:05.951 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:05.951 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:05.951 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:05.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.952 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupname_latency": { 2024-09-21T06:09:05.952 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:05.952 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:05.952 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:05.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.952 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupparent_latency": { 2024-09-21T06:09:05.952 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:05.952 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:05.952 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:05.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.952 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupsnap_latency": { 2024-09-21T06:09:05.952 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:05.952 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:05.952 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:05.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.953 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lssnap_latency": { 2024-09-21T06:09:05.953 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:05.953 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:05.953 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:05.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.953 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mkdir_latency": { 2024-09-21T06:09:05.953 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:05.954 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:05.954 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:05.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.954 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mknod_latency": { 2024-09-21T06:09:05.954 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:05.954 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:05.954 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:05.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.954 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mksnap_latency": { 2024-09-21T06:09:05.954 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:05.954 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:05.954 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:05.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.954 INFO:teuthology.orchestra.run.smithi192.stdout: "req_open_latency": { 2024-09-21T06:09:05.955 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:05.955 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:05.955 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:05.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.955 INFO:teuthology.orchestra.run.smithi192.stdout: "req_readdir_latency": { 2024-09-21T06:09:05.955 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:05.955 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:05.955 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:05.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.955 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rename_latency": { 2024-09-21T06:09:05.955 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:05.955 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:05.955 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:05.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.956 INFO:teuthology.orchestra.run.smithi192.stdout: "req_renamesnap_latency": { 2024-09-21T06:09:05.956 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:05.956 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:05.956 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:05.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.956 INFO:teuthology.orchestra.run.smithi192.stdout: "req_snapdiff_latency": { 2024-09-21T06:09:05.956 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:05.956 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:05.956 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:05.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.956 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmdir_latency": { 2024-09-21T06:09:05.956 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:05.956 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:05.956 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:05.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.957 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmsnap_latency": { 2024-09-21T06:09:05.957 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:05.957 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:05.957 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:05.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.957 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmxattr_latency": { 2024-09-21T06:09:05.957 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:05.957 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:05.957 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:05.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.957 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setattr_latency": { 2024-09-21T06:09:05.957 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:05.957 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:05.957 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:05.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.958 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setdirlayout_latency": { 2024-09-21T06:09:05.958 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:05.958 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:05.958 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:05.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.958 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setfilelock_latency": { 2024-09-21T06:09:05.958 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:05.958 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:05.958 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:05.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.958 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setlayout_latency": { 2024-09-21T06:09:05.958 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:05.958 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:05.959 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:05.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.959 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setxattr_latency": { 2024-09-21T06:09:05.959 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:05.959 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:05.959 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:05.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.959 INFO:teuthology.orchestra.run.smithi192.stdout: "req_symlink_latency": { 2024-09-21T06:09:05.959 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:05.959 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:05.959 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:05.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.959 INFO:teuthology.orchestra.run.smithi192.stdout: "req_unlink_latency": { 2024-09-21T06:09:05.959 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:05.960 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:05.960 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:05.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.960 INFO:teuthology.orchestra.run.smithi192.stdout: "cap_revoke_eviction": 0, 2024-09-21T06:09:05.960 INFO:teuthology.orchestra.run.smithi192.stdout: "cap_acquisition_throttle": 0, 2024-09-21T06:09:05.960 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getvxattr_latency": { 2024-09-21T06:09:05.960 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:05.960 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:05.960 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:05.960 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:05.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.960 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_sessions": { 2024-09-21T06:09:05.960 INFO:teuthology.orchestra.run.smithi192.stdout: "session_count": 0, 2024-09-21T06:09:05.960 INFO:teuthology.orchestra.run.smithi192.stdout: "session_add": 1, 2024-09-21T06:09:05.960 INFO:teuthology.orchestra.run.smithi192.stdout: "session_remove": 1, 2024-09-21T06:09:05.961 INFO:teuthology.orchestra.run.smithi192.stdout: "sessions_open": 0, 2024-09-21T06:09:05.961 INFO:teuthology.orchestra.run.smithi192.stdout: "sessions_stale": 0, 2024-09-21T06:09:05.961 INFO:teuthology.orchestra.run.smithi192.stdout: "total_load": 1, 2024-09-21T06:09:05.961 INFO:teuthology.orchestra.run.smithi192.stdout: "average_load": 1, 2024-09-21T06:09:05.961 INFO:teuthology.orchestra.run.smithi192.stdout: "avg_session_uptime": 91, 2024-09-21T06:09:05.961 INFO:teuthology.orchestra.run.smithi192.stdout: "mdthresh_evicted": 0 2024-09-21T06:09:05.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.961 INFO:teuthology.orchestra.run.smithi192.stdout: "mempool": { 2024-09-21T06:09:05.961 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:09:05.961 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_items": 0, 2024-09-21T06:09:05.961 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:09:05.961 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:09:05.961 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:09:05.961 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:09:05.962 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:09:05.962 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:09:05.962 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:09:05.962 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:09:05.962 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:09:05.962 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:09:05.962 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:09:05.962 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:09:05.962 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:09:05.962 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_items": 0, 2024-09-21T06:09:05.962 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:09:05.962 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_items": 0, 2024-09-21T06:09:05.962 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:09:05.962 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:09:05.963 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:09:05.963 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:09:05.963 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:09:05.963 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:09:05.963 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:09:05.963 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_items": 0, 2024-09-21T06:09:05.963 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:09:05.963 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:09:05.963 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:09:05.963 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_items": 0, 2024-09-21T06:09:05.963 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_bytes": 0, 2024-09-21T06:09:05.963 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_items": 0, 2024-09-21T06:09:05.963 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:09:05.963 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:09:05.964 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:09:05.964 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:09:05.964 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_bytes": 262871, 2024-09-21T06:09:05.964 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_items": 91, 2024-09-21T06:09:05.964 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_bytes": 0, 2024-09-21T06:09:05.964 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_items": 0, 2024-09-21T06:09:05.964 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_bytes": 0, 2024-09-21T06:09:05.964 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_items": 0, 2024-09-21T06:09:05.964 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:09:05.964 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_items": 0, 2024-09-21T06:09:05.964 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:09:05.964 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_items": 0, 2024-09-21T06:09:05.964 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_bytes": 7968, 2024-09-21T06:09:05.964 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_items": 176, 2024-09-21T06:09:05.964 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:09:05.965 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:09:05.965 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_bytes": 0, 2024-09-21T06:09:05.965 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_items": 0, 2024-09-21T06:09:05.965 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_bytes": 83501, 2024-09-21T06:09:05.965 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_items": 387, 2024-09-21T06:09:05.965 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_bytes": 0, 2024-09-21T06:09:05.965 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_items": 0, 2024-09-21T06:09:05.965 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_bytes": 0, 2024-09-21T06:09:05.965 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_items": 0 2024-09-21T06:09:05.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.965 INFO:teuthology.orchestra.run.smithi192.stdout: "objecter": { 2024-09-21T06:09:05.965 INFO:teuthology.orchestra.run.smithi192.stdout: "op_active": 0, 2024-09-21T06:09:05.965 INFO:teuthology.orchestra.run.smithi192.stdout: "op_laggy": 0, 2024-09-21T06:09:05.965 INFO:teuthology.orchestra.run.smithi192.stdout: "op_send": 38, 2024-09-21T06:09:05.965 INFO:teuthology.orchestra.run.smithi192.stdout: "op_send_bytes": 11332, 2024-09-21T06:09:05.966 INFO:teuthology.orchestra.run.smithi192.stdout: "op_resend": 28, 2024-09-21T06:09:05.966 INFO:teuthology.orchestra.run.smithi192.stdout: "op_reply": 38, 2024-09-21T06:09:05.966 INFO:teuthology.orchestra.run.smithi192.stdout: "op_latency": { 2024-09-21T06:09:05.966 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 38, 2024-09-21T06:09:05.966 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.243995457, 2024-09-21T06:09:05.966 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.006420933 2024-09-21T06:09:05.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.966 INFO:teuthology.orchestra.run.smithi192.stdout: "op_inflight": 0, 2024-09-21T06:09:05.966 INFO:teuthology.orchestra.run.smithi192.stdout: "oplen_avg": { 2024-09-21T06:09:05.966 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 38, 2024-09-21T06:09:05.966 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 60 2024-09-21T06:09:05.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.966 INFO:teuthology.orchestra.run.smithi192.stdout: "op": 38, 2024-09-21T06:09:05.966 INFO:teuthology.orchestra.run.smithi192.stdout: "op_r": 0, 2024-09-21T06:09:05.966 INFO:teuthology.orchestra.run.smithi192.stdout: "op_w": 38, 2024-09-21T06:09:05.967 INFO:teuthology.orchestra.run.smithi192.stdout: "op_rmw": 0, 2024-09-21T06:09:05.967 INFO:teuthology.orchestra.run.smithi192.stdout: "op_pg": 0, 2024-09-21T06:09:05.967 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_stat": 0, 2024-09-21T06:09:05.967 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_create": 10, 2024-09-21T06:09:05.967 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_read": 0, 2024-09-21T06:09:05.967 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_write": 3, 2024-09-21T06:09:05.967 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_writefull": 6, 2024-09-21T06:09:05.967 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_writesame": 0, 2024-09-21T06:09:05.967 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_append": 0, 2024-09-21T06:09:05.967 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_zero": 0, 2024-09-21T06:09:05.967 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_truncate": 0, 2024-09-21T06:09:05.967 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_delete": 6, 2024-09-21T06:09:05.967 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_mapext": 0, 2024-09-21T06:09:05.967 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_sparse_read": 0, 2024-09-21T06:09:05.967 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_clonerange": 0, 2024-09-21T06:09:05.968 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_getxattr": 0, 2024-09-21T06:09:05.968 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_setxattr": 20, 2024-09-21T06:09:05.968 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_cmpxattr": 0, 2024-09-21T06:09:05.968 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_rmxattr": 0, 2024-09-21T06:09:05.968 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_resetxattrs": 0, 2024-09-21T06:09:05.968 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_call": 0, 2024-09-21T06:09:05.968 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_watch": 0, 2024-09-21T06:09:05.968 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_notify": 0, 2024-09-21T06:09:05.968 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T06:09:05.968 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_pgls": 0, 2024-09-21T06:09:05.968 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_pgls_filter": 0, 2024-09-21T06:09:05.968 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_other": 0, 2024-09-21T06:09:05.968 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_active": 0, 2024-09-21T06:09:05.968 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_send": 0, 2024-09-21T06:09:05.968 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_resend": 0, 2024-09-21T06:09:05.969 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_ping": 0, 2024-09-21T06:09:05.969 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_active": 0, 2024-09-21T06:09:05.969 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_send": 0, 2024-09-21T06:09:05.969 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_resend": 0, 2024-09-21T06:09:05.969 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_active": 0, 2024-09-21T06:09:05.969 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_send": 0, 2024-09-21T06:09:05.969 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_resend": 0, 2024-09-21T06:09:05.969 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_active": 0, 2024-09-21T06:09:05.969 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_send": 0, 2024-09-21T06:09:05.969 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_resend": 0, 2024-09-21T06:09:05.969 INFO:teuthology.orchestra.run.smithi192.stdout: "command_active": 0, 2024-09-21T06:09:05.969 INFO:teuthology.orchestra.run.smithi192.stdout: "command_send": 0, 2024-09-21T06:09:05.969 INFO:teuthology.orchestra.run.smithi192.stdout: "command_resend": 0, 2024-09-21T06:09:05.969 INFO:teuthology.orchestra.run.smithi192.stdout: "map_epoch": 85, 2024-09-21T06:09:05.969 INFO:teuthology.orchestra.run.smithi192.stdout: "map_full": 0, 2024-09-21T06:09:05.970 INFO:teuthology.orchestra.run.smithi192.stdout: "map_inc": 11, 2024-09-21T06:09:05.970 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_sessions": 9, 2024-09-21T06:09:05.970 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_session_open": 9, 2024-09-21T06:09:05.970 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_session_close": 0, 2024-09-21T06:09:05.970 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_laggy": 0, 2024-09-21T06:09:05.970 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_wr": 14, 2024-09-21T06:09:05.970 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_rd": 0, 2024-09-21T06:09:05.970 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_del": 1 2024-09-21T06:09:05.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.970 INFO:teuthology.orchestra.run.smithi192.stdout: "oft": { 2024-09-21T06:09:05.970 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_objs": 1, 2024-09-21T06:09:05.970 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_kv_pairs": 0, 2024-09-21T06:09:05.970 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_updates": 0, 2024-09-21T06:09:05.970 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_removes": 0 2024-09-21T06:09:05.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.971 INFO:teuthology.orchestra.run.smithi192.stdout: "purge_queue": { 2024-09-21T06:09:05.971 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_ops": 0, 2024-09-21T06:09:05.971 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T06:09:05.971 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing": 0, 2024-09-21T06:09:05.971 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_high_water": 0, 2024-09-21T06:09:05.971 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executed_ops": 0, 2024-09-21T06:09:05.971 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executed": 0, 2024-09-21T06:09:05.971 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_item_in_journal": 0 2024-09-21T06:09:05.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.971 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:09:05.971 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-21T06:09:05.971 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-09-21T06:09:05.971 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-21T06:09:05.971 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 1960, 2024-09-21T06:09:05.971 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 1225837, 2024-09-21T06:09:05.972 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-21T06:09:05.972 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 1960, 2024-09-21T06:09:05.972 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-09-21T06:09:05.972 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-09-21T06:09:05.972 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 1960, 2024-09-21T06:09:05.972 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 1225837, 2024-09-21T06:09:05.972 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-21T06:09:05.972 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:05.972 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:05.972 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:05.972 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:05.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.972 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-objecter_bytes": { 2024-09-21T06:09:05.972 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-21T06:09:05.972 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-09-21T06:09:05.973 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-21T06:09:05.973 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-09-21T06:09:05.973 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-09-21T06:09:05.973 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-21T06:09:05.973 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-09-21T06:09:05.973 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 38, 2024-09-21T06:09:05.973 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 11332, 2024-09-21T06:09:05.973 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 38, 2024-09-21T06:09:05.973 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 11332, 2024-09-21T06:09:05.973 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-21T06:09:05.973 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:05.973 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:05.973 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:05.973 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:05.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.973 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-objecter_ops": { 2024-09-21T06:09:05.974 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-21T06:09:05.974 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 1024, 2024-09-21T06:09:05.974 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-21T06:09:05.974 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-09-21T06:09:05.974 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-09-21T06:09:05.974 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-21T06:09:05.974 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-09-21T06:09:05.974 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 38, 2024-09-21T06:09:05.974 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 38, 2024-09-21T06:09:05.974 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 38, 2024-09-21T06:09:05.974 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 38, 2024-09-21T06:09:05.974 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-21T06:09:05.974 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:05.974 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:05.975 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:05.975 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:05.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.975 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-write_buf_throttle": { 2024-09-21T06:09:05.975 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-21T06:09:05.975 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 3758096384, 2024-09-21T06:09:05.975 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-21T06:09:05.975 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-09-21T06:09:05.975 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-09-21T06:09:05.975 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-21T06:09:05.975 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-09-21T06:09:05.975 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-09-21T06:09:05.975 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-09-21T06:09:05.975 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 0, 2024-09-21T06:09:05.975 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 0, 2024-09-21T06:09:05.975 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-21T06:09:05.976 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:05.976 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:05.976 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:05.976 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:05.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.976 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-write_buf_throttle-0x55ab02a9b450": { 2024-09-21T06:09:05.976 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-21T06:09:05.976 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 3758096384, 2024-09-21T06:09:05.976 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-21T06:09:05.976 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 4, 2024-09-21T06:09:05.976 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 1120, 2024-09-21T06:09:05.976 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-21T06:09:05.976 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 4, 2024-09-21T06:09:05.976 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-09-21T06:09:05.976 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-09-21T06:09:05.977 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 3, 2024-09-21T06:09:05.977 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 1120, 2024-09-21T06:09:05.977 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-21T06:09:05.977 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:05.977 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:05.977 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:05.977 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:05.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:05.977 INFO:teuthology.orchestra.run.smithi192.stdout: "trackedop": { 2024-09-21T06:09:05.977 INFO:teuthology.orchestra.run.smithi192.stdout: "slow_ops_count": 0 2024-09-21T06:09:05.977 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:05.977 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-21T06:09:06.230 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:05 smithi192 ceph-mon[34700]: pgmap v689: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-09-21T06:09:06.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:05 smithi117 ceph-mon[30947]: pgmap v689: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-09-21T06:09:06.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:05 smithi138 ceph-mon[34391]: pgmap v689: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-09-21T06:09:06.729 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-21T06:09:06.729 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-21T06:09:06.729 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-09-21T06:09:06.729 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=12 2024-09-21T06:09:06.729 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=7 2024-09-21T06:09:06.729 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-21T06:09:06.729 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T06:09:06.729 DEBUG:tasks.check_counter:Getting stats from l 2024-09-21T06:09:06.730 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph daemon mds.l perf dump 2024-09-21T06:09:06.975 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T06:09:07.511 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-21T06:09:07.511 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T06:09:07.511 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 102, 2024-09-21T06:09:07.511 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 99, 2024-09-21T06:09:07.511 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 205052, 2024-09-21T06:09:07.511 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 53118, 2024-09-21T06:09:07.511 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 5, 2024-09-21T06:09:07.511 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 1, 2024-09-21T06:09:07.512 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.020720271, 2024-09-21T06:09:07.512 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.007911600, 2024-09-21T06:09:07.512 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.015287886, 2024-09-21T06:09:07.512 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.000024671, 2024-09-21T06:09:07.512 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:09:07.512 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 99, 2024-09-21T06:09:07.512 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.012718293, 2024-09-21T06:09:07.512 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000128467 2024-09-21T06:09:07.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.512 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:09:07.512 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.512 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.512 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.513 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 204784, 2024-09-21T06:09:07.513 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 52800 2024-09-21T06:09:07.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.513 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T06:09:07.513 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 941, 2024-09-21T06:09:07.513 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 1003, 2024-09-21T06:09:07.513 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 1055301, 2024-09-21T06:09:07.513 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 498837, 2024-09-21T06:09:07.513 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 5, 2024-09-21T06:09:07.513 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 4, 2024-09-21T06:09:07.513 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.111663316, 2024-09-21T06:09:07.513 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.045627009, 2024-09-21T06:09:07.513 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.027322554, 2024-09-21T06:09:07.513 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.032013505, 2024-09-21T06:09:07.513 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:09:07.514 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1001, 2024-09-21T06:09:07.514 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.045677424, 2024-09-21T06:09:07.514 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000045631 2024-09-21T06:09:07.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.514 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:09:07.514 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.514 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.514 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.514 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T06:09:07.514 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 199856 2024-09-21T06:09:07.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.514 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T06:09:07.514 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 5, 2024-09-21T06:09:07.515 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 7, 2024-09-21T06:09:07.515 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 2444, 2024-09-21T06:09:07.515 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 1606, 2024-09-21T06:09:07.515 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 5, 2024-09-21T06:09:07.515 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 4, 2024-09-21T06:09:07.515 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.001728871, 2024-09-21T06:09:07.515 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.000151006, 2024-09-21T06:09:07.515 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.001343803, 2024-09-21T06:09:07.515 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.000121995, 2024-09-21T06:09:07.515 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T06:09:07.515 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 5, 2024-09-21T06:09:07.515 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.002929492, 2024-09-21T06:09:07.515 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000585898 2024-09-21T06:09:07.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.515 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-09-21T06:09:07.516 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.516 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.516 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.516 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T06:09:07.516 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T06:09:07.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.516 INFO:teuthology.orchestra.run.smithi192.stdout: "cct": { 2024-09-21T06:09:07.516 INFO:teuthology.orchestra.run.smithi192.stdout: "total_workers": 1, 2024-09-21T06:09:07.516 INFO:teuthology.orchestra.run.smithi192.stdout: "unhealthy_workers": 0 2024-09-21T06:09:07.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.516 INFO:teuthology.orchestra.run.smithi192.stdout: "finisher-MDSRank": { 2024-09-21T06:09:07.516 INFO:teuthology.orchestra.run.smithi192.stdout: "queue_len": 0, 2024-09-21T06:09:07.516 INFO:teuthology.orchestra.run.smithi192.stdout: "complete_latency": { 2024-09-21T06:09:07.516 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1524, 2024-09-21T06:09:07.517 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.052167838, 2024-09-21T06:09:07.517 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000034230 2024-09-21T06:09:07.517 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:07.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.517 INFO:teuthology.orchestra.run.smithi192.stdout: "finisher-PurgeQueue": { 2024-09-21T06:09:07.517 INFO:teuthology.orchestra.run.smithi192.stdout: "queue_len": 0, 2024-09-21T06:09:07.517 INFO:teuthology.orchestra.run.smithi192.stdout: "complete_latency": { 2024-09-21T06:09:07.517 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.517 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.517 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.517 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:07.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.517 INFO:teuthology.orchestra.run.smithi192.stdout: "mds": { 2024-09-21T06:09:07.517 INFO:teuthology.orchestra.run.smithi192.stdout: "request": 0, 2024-09-21T06:09:07.518 INFO:teuthology.orchestra.run.smithi192.stdout: "reply": 0, 2024-09-21T06:09:07.518 INFO:teuthology.orchestra.run.smithi192.stdout: "reply_latency": { 2024-09-21T06:09:07.518 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.518 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.518 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.518 INFO:teuthology.orchestra.run.smithi192.stdout: "slow_reply": 0, 2024-09-21T06:09:07.518 INFO:teuthology.orchestra.run.smithi192.stdout: "forward": 0, 2024-09-21T06:09:07.518 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_fetch_complete": 0, 2024-09-21T06:09:07.518 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_fetch_keys": 0, 2024-09-21T06:09:07.518 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_commit": 0, 2024-09-21T06:09:07.518 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_split": 0, 2024-09-21T06:09:07.518 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_merge": 0, 2024-09-21T06:09:07.518 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes": 73, 2024-09-21T06:09:07.519 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_top": 2, 2024-09-21T06:09:07.519 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_bottom": 71, 2024-09-21T06:09:07.519 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_pin_tail": 0, 2024-09-21T06:09:07.519 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_pinned": 71, 2024-09-21T06:09:07.519 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_expired": 0, 2024-09-21T06:09:07.519 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_with_caps": 0, 2024-09-21T06:09:07.519 INFO:teuthology.orchestra.run.smithi192.stdout: "caps": 0, 2024-09-21T06:09:07.519 INFO:teuthology.orchestra.run.smithi192.stdout: "subtrees": 2, 2024-09-21T06:09:07.519 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse": 0, 2024-09-21T06:09:07.519 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_hit": 0, 2024-09-21T06:09:07.519 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_forward": 0, 2024-09-21T06:09:07.519 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_discover": 0, 2024-09-21T06:09:07.519 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_dir_fetch": 0, 2024-09-21T06:09:07.519 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_remote_ino": 0, 2024-09-21T06:09:07.519 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_lock": 0, 2024-09-21T06:09:07.520 INFO:teuthology.orchestra.run.smithi192.stdout: "load_cent": 0, 2024-09-21T06:09:07.520 INFO:teuthology.orchestra.run.smithi192.stdout: "q": 0, 2024-09-21T06:09:07.520 INFO:teuthology.orchestra.run.smithi192.stdout: "exported": 0, 2024-09-21T06:09:07.520 INFO:teuthology.orchestra.run.smithi192.stdout: "exported_inodes": 0, 2024-09-21T06:09:07.520 INFO:teuthology.orchestra.run.smithi192.stdout: "imported": 0, 2024-09-21T06:09:07.520 INFO:teuthology.orchestra.run.smithi192.stdout: "imported_inodes": 0, 2024-09-21T06:09:07.520 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_dir_fetch": 0, 2024-09-21T06:09:07.520 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_backtrace_fetch": 0, 2024-09-21T06:09:07.520 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_peer_discover": 0, 2024-09-21T06:09:07.520 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rfiles": 4, 2024-09-21T06:09:07.520 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rbytes": 234, 2024-09-21T06:09:07.520 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rsnaps": 7, 2024-09-21T06:09:07.520 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T06:09:07.520 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_set_tag": 0, 2024-09-21T06:09:07.520 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T06:09:07.521 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_inotable_repaired": 0, 2024-09-21T06:09:07.521 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dir_inodes": 0, 2024-09-21T06:09:07.521 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T06:09:07.521 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T06:09:07.521 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_file_inodes": 0, 2024-09-21T06:09:07.521 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_inode_file_caps": 0, 2024-09-21T06:09:07.521 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T06:09:07.521 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_grant": 0, 2024-09-21T06:09:07.521 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T06:09:07.521 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T06:09:07.521 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T06:09:07.521 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_caps": 0, 2024-09-21T06:09:07.521 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_caps_dirty": 0, 2024-09-21T06:09:07.521 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_cap_release": 0, 2024-09-21T06:09:07.521 INFO:teuthology.orchestra.run.smithi192.stdout: "process_request_cap_release": 0 2024-09-21T06:09:07.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.522 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_cache": { 2024-09-21T06:09:07.522 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_update": 0, 2024-09-21T06:09:07.522 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_update_receipt": 0, 2024-09-21T06:09:07.522 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_try_discover": 0, 2024-09-21T06:09:07.522 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_send_discover": 0, 2024-09-21T06:09:07.522 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_handle_discover": 0, 2024-09-21T06:09:07.522 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays": 0, 2024-09-21T06:09:07.522 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays_delayed": 0, 2024-09-21T06:09:07.522 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays_enqueuing": 0, 2024-09-21T06:09:07.522 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_created": 0, 2024-09-21T06:09:07.522 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_enqueued": 0, 2024-09-21T06:09:07.522 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_reintegrated": 0, 2024-09-21T06:09:07.523 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_migrated": 0, 2024-09-21T06:09:07.523 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_processing": 0, 2024-09-21T06:09:07.523 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_enqueued": 0, 2024-09-21T06:09:07.523 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_prioritized": 0, 2024-09-21T06:09:07.523 INFO:teuthology.orchestra.run.smithi192.stdout: "recovery_started": 0, 2024-09-21T06:09:07.523 INFO:teuthology.orchestra.run.smithi192.stdout: "recovery_completed": 0, 2024-09-21T06:09:07.523 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_quiesce_path": 0, 2024-09-21T06:09:07.523 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_quiesce_inode": 0, 2024-09-21T06:09:07.523 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T06:09:07.523 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_exportdir": 0, 2024-09-21T06:09:07.523 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_flush": 0, 2024-09-21T06:09:07.523 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_fragmentdir": 0, 2024-09-21T06:09:07.523 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_fragstats": 0, 2024-09-21T06:09:07.523 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_inodestats": 0, 2024-09-21T06:09:07.523 INFO:teuthology.orchestra.run.smithi192.stdout: "uninline_started": 0, 2024-09-21T06:09:07.523 INFO:teuthology.orchestra.run.smithi192.stdout: "uninline_succeeded": 0, 2024-09-21T06:09:07.524 INFO:teuthology.orchestra.run.smithi192.stdout: "uninline_write_failed": 0 2024-09-21T06:09:07.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.524 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_log": { 2024-09-21T06:09:07.524 INFO:teuthology.orchestra.run.smithi192.stdout: "evlrg": 0, 2024-09-21T06:09:07.524 INFO:teuthology.orchestra.run.smithi192.stdout: "evadd": 0, 2024-09-21T06:09:07.524 INFO:teuthology.orchestra.run.smithi192.stdout: "evex": 0, 2024-09-21T06:09:07.524 INFO:teuthology.orchestra.run.smithi192.stdout: "evtrm": 0, 2024-09-21T06:09:07.524 INFO:teuthology.orchestra.run.smithi192.stdout: "ev": 165, 2024-09-21T06:09:07.524 INFO:teuthology.orchestra.run.smithi192.stdout: "evexg": 0, 2024-09-21T06:09:07.524 INFO:teuthology.orchestra.run.smithi192.stdout: "evexd": 0, 2024-09-21T06:09:07.524 INFO:teuthology.orchestra.run.smithi192.stdout: "segadd": 0, 2024-09-21T06:09:07.524 INFO:teuthology.orchestra.run.smithi192.stdout: "segex": 0, 2024-09-21T06:09:07.524 INFO:teuthology.orchestra.run.smithi192.stdout: "segtrm": 0, 2024-09-21T06:09:07.524 INFO:teuthology.orchestra.run.smithi192.stdout: "seg": 2, 2024-09-21T06:09:07.524 INFO:teuthology.orchestra.run.smithi192.stdout: "segmjr": 20, 2024-09-21T06:09:07.525 INFO:teuthology.orchestra.run.smithi192.stdout: "segexg": 0, 2024-09-21T06:09:07.525 INFO:teuthology.orchestra.run.smithi192.stdout: "segexd": 0, 2024-09-21T06:09:07.525 INFO:teuthology.orchestra.run.smithi192.stdout: "expos": 1079549107, 2024-09-21T06:09:07.525 INFO:teuthology.orchestra.run.smithi192.stdout: "wrpos": 1079841256, 2024-09-21T06:09:07.525 INFO:teuthology.orchestra.run.smithi192.stdout: "rdpos": 1079841172, 2024-09-21T06:09:07.525 INFO:teuthology.orchestra.run.smithi192.stdout: "jlat": { 2024-09-21T06:09:07.525 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.525 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.525 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.525 INFO:teuthology.orchestra.run.smithi192.stdout: "replayed": 165 2024-09-21T06:09:07.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.525 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_mem": { 2024-09-21T06:09:07.525 INFO:teuthology.orchestra.run.smithi192.stdout: "ino": 32, 2024-09-21T06:09:07.525 INFO:teuthology.orchestra.run.smithi192.stdout: "ino+": 50, 2024-09-21T06:09:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: "ino-": 18, 2024-09-21T06:09:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: "dir": 11, 2024-09-21T06:09:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: "dir+": 11, 2024-09-21T06:09:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: "dir-": 0, 2024-09-21T06:09:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: "dn": 73, 2024-09-21T06:09:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: "dn+": 73, 2024-09-21T06:09:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: "dn-": 0, 2024-09-21T06:09:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: "cap": 0, 2024-09-21T06:09:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: "cap+": 0, 2024-09-21T06:09:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: "cap-": 0, 2024-09-21T06:09:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: "rss": 115576, 2024-09-21T06:09:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: "heap": 315375616 2024-09-21T06:09:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_server": { 2024-09-21T06:09:07.526 INFO:teuthology.orchestra.run.smithi192.stdout: "dispatch_client_request": 0, 2024-09-21T06:09:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: "dispatch_server_request": 0, 2024-09-21T06:09:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_request": 0, 2024-09-21T06:09:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_session": 0, 2024-09-21T06:09:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_peer_request": 0, 2024-09-21T06:09:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: "req_create_latency": { 2024-09-21T06:09:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getattr_latency": { 2024-09-21T06:09:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getfilelock_latency": { 2024-09-21T06:09:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: "req_link_latency": { 2024-09-21T06:09:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookup_latency": { 2024-09-21T06:09:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookuphash_latency": { 2024-09-21T06:09:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupino_latency": { 2024-09-21T06:09:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupname_latency": { 2024-09-21T06:09:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupparent_latency": { 2024-09-21T06:09:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupsnap_latency": { 2024-09-21T06:09:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.532 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lssnap_latency": { 2024-09-21T06:09:07.532 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.532 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mkdir_latency": { 2024-09-21T06:09:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mknod_latency": { 2024-09-21T06:09:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.533 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mksnap_latency": { 2024-09-21T06:09:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: "req_open_latency": { 2024-09-21T06:09:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: "req_readdir_latency": { 2024-09-21T06:09:07.534 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rename_latency": { 2024-09-21T06:09:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: "req_renamesnap_latency": { 2024-09-21T06:09:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: "req_snapdiff_latency": { 2024-09-21T06:09:07.535 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmdir_latency": { 2024-09-21T06:09:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmsnap_latency": { 2024-09-21T06:09:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.536 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmxattr_latency": { 2024-09-21T06:09:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setattr_latency": { 2024-09-21T06:09:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setdirlayout_latency": { 2024-09-21T06:09:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.537 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setfilelock_latency": { 2024-09-21T06:09:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setlayout_latency": { 2024-09-21T06:09:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setxattr_latency": { 2024-09-21T06:09:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: "req_symlink_latency": { 2024-09-21T06:09:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: "req_unlink_latency": { 2024-09-21T06:09:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: "cap_revoke_eviction": 0, 2024-09-21T06:09:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: "cap_acquisition_throttle": 0, 2024-09-21T06:09:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getvxattr_latency": { 2024-09-21T06:09:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.539 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.540 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.540 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:07.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.540 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_sessions": { 2024-09-21T06:09:07.540 INFO:teuthology.orchestra.run.smithi192.stdout: "session_count": 2, 2024-09-21T06:09:07.540 INFO:teuthology.orchestra.run.smithi192.stdout: "session_add": 2, 2024-09-21T06:09:07.540 INFO:teuthology.orchestra.run.smithi192.stdout: "session_remove": 0, 2024-09-21T06:09:07.540 INFO:teuthology.orchestra.run.smithi192.stdout: "sessions_open": 0, 2024-09-21T06:09:07.540 INFO:teuthology.orchestra.run.smithi192.stdout: "sessions_stale": 0, 2024-09-21T06:09:07.540 INFO:teuthology.orchestra.run.smithi192.stdout: "total_load": 0, 2024-09-21T06:09:07.540 INFO:teuthology.orchestra.run.smithi192.stdout: "average_load": 0, 2024-09-21T06:09:07.540 INFO:teuthology.orchestra.run.smithi192.stdout: "avg_session_uptime": 2049, 2024-09-21T06:09:07.540 INFO:teuthology.orchestra.run.smithi192.stdout: "mdthresh_evicted": 0 2024-09-21T06:09:07.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.540 INFO:teuthology.orchestra.run.smithi192.stdout: "mempool": { 2024-09-21T06:09:07.541 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_bytes": 0, 2024-09-21T06:09:07.541 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_items": 0, 2024-09-21T06:09:07.541 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T06:09:07.541 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_items": 0, 2024-09-21T06:09:07.541 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T06:09:07.541 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_items": 0, 2024-09-21T06:09:07.541 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T06:09:07.541 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T06:09:07.541 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T06:09:07.541 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T06:09:07.541 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T06:09:07.541 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_items": 0, 2024-09-21T06:09:07.541 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T06:09:07.541 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T06:09:07.542 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_bytes": 0, 2024-09-21T06:09:07.542 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_items": 0, 2024-09-21T06:09:07.542 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_bytes": 0, 2024-09-21T06:09:07.542 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_items": 0, 2024-09-21T06:09:07.542 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T06:09:07.542 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T06:09:07.542 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T06:09:07.542 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T06:09:07.542 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T06:09:07.542 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_items": 0, 2024-09-21T06:09:07.542 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_bytes": 0, 2024-09-21T06:09:07.542 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_items": 0, 2024-09-21T06:09:07.542 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T06:09:07.542 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T06:09:07.542 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_bytes": 0, 2024-09-21T06:09:07.543 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_items": 0, 2024-09-21T06:09:07.543 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_bytes": 0, 2024-09-21T06:09:07.543 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_items": 0, 2024-09-21T06:09:07.543 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T06:09:07.543 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_items": 0, 2024-09-21T06:09:07.543 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T06:09:07.543 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_items": 0, 2024-09-21T06:09:07.543 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_bytes": 200824, 2024-09-21T06:09:07.543 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_items": 93, 2024-09-21T06:09:07.543 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_bytes": 0, 2024-09-21T06:09:07.543 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_items": 0, 2024-09-21T06:09:07.543 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_bytes": 0, 2024-09-21T06:09:07.543 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_items": 0, 2024-09-21T06:09:07.543 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_bytes": 0, 2024-09-21T06:09:07.543 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_items": 0, 2024-09-21T06:09:07.544 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_bytes": 0, 2024-09-21T06:09:07.544 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_items": 0, 2024-09-21T06:09:07.544 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_bytes": 7968, 2024-09-21T06:09:07.544 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_items": 176, 2024-09-21T06:09:07.544 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T06:09:07.544 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_items": 0, 2024-09-21T06:09:07.544 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_bytes": 0, 2024-09-21T06:09:07.544 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_items": 0, 2024-09-21T06:09:07.544 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_bytes": 181192, 2024-09-21T06:09:07.544 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_items": 1803, 2024-09-21T06:09:07.544 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_bytes": 0, 2024-09-21T06:09:07.544 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_items": 0, 2024-09-21T06:09:07.544 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_bytes": 0, 2024-09-21T06:09:07.544 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_items": 0 2024-09-21T06:09:07.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.544 INFO:teuthology.orchestra.run.smithi192.stdout: "objecter": { 2024-09-21T06:09:07.545 INFO:teuthology.orchestra.run.smithi192.stdout: "op_active": 0, 2024-09-21T06:09:07.545 INFO:teuthology.orchestra.run.smithi192.stdout: "op_laggy": 0, 2024-09-21T06:09:07.545 INFO:teuthology.orchestra.run.smithi192.stdout: "op_send": 946, 2024-09-21T06:09:07.545 INFO:teuthology.orchestra.run.smithi192.stdout: "op_send_bytes": 26, 2024-09-21T06:09:07.545 INFO:teuthology.orchestra.run.smithi192.stdout: "op_resend": 3, 2024-09-21T06:09:07.545 INFO:teuthology.orchestra.run.smithi192.stdout: "op_reply": 946, 2024-09-21T06:09:07.545 INFO:teuthology.orchestra.run.smithi192.stdout: "op_latency": { 2024-09-21T06:09:07.545 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 946, 2024-09-21T06:09:07.545 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.546990082, 2024-09-21T06:09:07.545 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000578213 2024-09-21T06:09:07.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.545 INFO:teuthology.orchestra.run.smithi192.stdout: "op_inflight": 0, 2024-09-21T06:09:07.545 INFO:teuthology.orchestra.run.smithi192.stdout: "oplen_avg": { 2024-09-21T06:09:07.545 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 946, 2024-09-21T06:09:07.545 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 947 2024-09-21T06:09:07.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.546 INFO:teuthology.orchestra.run.smithi192.stdout: "op": 946, 2024-09-21T06:09:07.546 INFO:teuthology.orchestra.run.smithi192.stdout: "op_r": 946, 2024-09-21T06:09:07.546 INFO:teuthology.orchestra.run.smithi192.stdout: "op_w": 0, 2024-09-21T06:09:07.546 INFO:teuthology.orchestra.run.smithi192.stdout: "op_rmw": 0, 2024-09-21T06:09:07.546 INFO:teuthology.orchestra.run.smithi192.stdout: "op_pg": 0, 2024-09-21T06:09:07.546 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_stat": 596, 2024-09-21T06:09:07.546 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_create": 0, 2024-09-21T06:09:07.546 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_read": 347, 2024-09-21T06:09:07.546 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_write": 0, 2024-09-21T06:09:07.546 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_writefull": 0, 2024-09-21T06:09:07.546 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_writesame": 0, 2024-09-21T06:09:07.546 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_append": 0, 2024-09-21T06:09:07.546 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_zero": 0, 2024-09-21T06:09:07.546 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_truncate": 0, 2024-09-21T06:09:07.547 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_delete": 0, 2024-09-21T06:09:07.547 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_mapext": 0, 2024-09-21T06:09:07.547 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_sparse_read": 0, 2024-09-21T06:09:07.547 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_clonerange": 0, 2024-09-21T06:09:07.547 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_getxattr": 2, 2024-09-21T06:09:07.547 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_setxattr": 0, 2024-09-21T06:09:07.547 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_cmpxattr": 0, 2024-09-21T06:09:07.547 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_rmxattr": 0, 2024-09-21T06:09:07.547 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_resetxattrs": 0, 2024-09-21T06:09:07.547 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_call": 0, 2024-09-21T06:09:07.547 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_watch": 0, 2024-09-21T06:09:07.547 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_notify": 0, 2024-09-21T06:09:07.547 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T06:09:07.547 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_pgls": 0, 2024-09-21T06:09:07.547 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_pgls_filter": 0, 2024-09-21T06:09:07.547 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_other": 0, 2024-09-21T06:09:07.548 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_active": 0, 2024-09-21T06:09:07.548 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_send": 0, 2024-09-21T06:09:07.548 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_resend": 0, 2024-09-21T06:09:07.548 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_ping": 0, 2024-09-21T06:09:07.548 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_active": 0, 2024-09-21T06:09:07.548 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_send": 0, 2024-09-21T06:09:07.548 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_resend": 0, 2024-09-21T06:09:07.548 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_active": 0, 2024-09-21T06:09:07.548 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_send": 0, 2024-09-21T06:09:07.548 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_resend": 0, 2024-09-21T06:09:07.548 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_active": 0, 2024-09-21T06:09:07.548 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_send": 0, 2024-09-21T06:09:07.548 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_resend": 0, 2024-09-21T06:09:07.548 INFO:teuthology.orchestra.run.smithi192.stdout: "command_active": 0, 2024-09-21T06:09:07.548 INFO:teuthology.orchestra.run.smithi192.stdout: "command_send": 0, 2024-09-21T06:09:07.548 INFO:teuthology.orchestra.run.smithi192.stdout: "command_resend": 0, 2024-09-21T06:09:07.549 INFO:teuthology.orchestra.run.smithi192.stdout: "map_epoch": 85, 2024-09-21T06:09:07.549 INFO:teuthology.orchestra.run.smithi192.stdout: "map_full": 0, 2024-09-21T06:09:07.549 INFO:teuthology.orchestra.run.smithi192.stdout: "map_inc": 8, 2024-09-21T06:09:07.549 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_sessions": 6, 2024-09-21T06:09:07.549 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_session_open": 6, 2024-09-21T06:09:07.549 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_session_close": 0, 2024-09-21T06:09:07.549 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_laggy": 0, 2024-09-21T06:09:07.549 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_wr": 0, 2024-09-21T06:09:07.549 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_rd": 2, 2024-09-21T06:09:07.549 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_del": 0 2024-09-21T06:09:07.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.549 INFO:teuthology.orchestra.run.smithi192.stdout: "oft": { 2024-09-21T06:09:07.549 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_objs": 0, 2024-09-21T06:09:07.549 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_kv_pairs": 0, 2024-09-21T06:09:07.550 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_updates": 0, 2024-09-21T06:09:07.550 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_removes": 0 2024-09-21T06:09:07.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.550 INFO:teuthology.orchestra.run.smithi192.stdout: "purge_queue": { 2024-09-21T06:09:07.550 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_ops": 0, 2024-09-21T06:09:07.550 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T06:09:07.550 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing": 0, 2024-09-21T06:09:07.550 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_high_water": 0, 2024-09-21T06:09:07.550 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executed_ops": 0, 2024-09-21T06:09:07.550 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executed": 0, 2024-09-21T06:09:07.550 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_item_in_journal": 0 2024-09-21T06:09:07.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.550 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T06:09:07.550 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-21T06:09:07.550 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-09-21T06:09:07.550 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-21T06:09:07.551 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 1044, 2024-09-21T06:09:07.551 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 1160889, 2024-09-21T06:09:07.551 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-21T06:09:07.551 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 1044, 2024-09-21T06:09:07.551 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-09-21T06:09:07.551 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-09-21T06:09:07.551 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 1044, 2024-09-21T06:09:07.551 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 1160889, 2024-09-21T06:09:07.551 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-21T06:09:07.551 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.551 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.551 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.551 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:07.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.551 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-objecter_bytes": { 2024-09-21T06:09:07.552 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-21T06:09:07.552 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-09-21T06:09:07.552 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-21T06:09:07.552 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-09-21T06:09:07.552 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-09-21T06:09:07.552 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-21T06:09:07.552 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-09-21T06:09:07.552 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 946, 2024-09-21T06:09:07.552 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 786130, 2024-09-21T06:09:07.552 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 946, 2024-09-21T06:09:07.552 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 786130, 2024-09-21T06:09:07.552 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-21T06:09:07.552 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.552 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.552 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.553 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:07.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.553 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-objecter_ops": { 2024-09-21T06:09:07.553 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-21T06:09:07.553 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 1024, 2024-09-21T06:09:07.553 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-21T06:09:07.553 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-09-21T06:09:07.553 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-09-21T06:09:07.553 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-21T06:09:07.553 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-09-21T06:09:07.553 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 946, 2024-09-21T06:09:07.553 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 946, 2024-09-21T06:09:07.553 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 946, 2024-09-21T06:09:07.553 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 946, 2024-09-21T06:09:07.553 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-21T06:09:07.553 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.554 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.554 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.554 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:07.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.554 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-write_buf_throttle": { 2024-09-21T06:09:07.554 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-21T06:09:07.554 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 3758096384, 2024-09-21T06:09:07.554 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-21T06:09:07.554 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-09-21T06:09:07.554 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-09-21T06:09:07.554 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-21T06:09:07.554 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-09-21T06:09:07.554 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-09-21T06:09:07.554 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-09-21T06:09:07.554 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 0, 2024-09-21T06:09:07.555 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 0, 2024-09-21T06:09:07.555 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-21T06:09:07.555 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.555 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.555 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.555 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:07.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.555 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-write_buf_throttle-0x55e53dcc1e10": { 2024-09-21T06:09:07.555 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-21T06:09:07.555 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 3758096384, 2024-09-21T06:09:07.555 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-21T06:09:07.555 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-09-21T06:09:07.555 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-09-21T06:09:07.555 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-21T06:09:07.555 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-09-21T06:09:07.555 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-09-21T06:09:07.556 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-09-21T06:09:07.556 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 0, 2024-09-21T06:09:07.556 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 0, 2024-09-21T06:09:07.556 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-21T06:09:07.556 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-21T06:09:07.556 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-21T06:09:07.556 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-21T06:09:07.556 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:07.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-21T06:09:07.556 INFO:teuthology.orchestra.run.smithi192.stdout: "trackedop": { 2024-09-21T06:09:07.556 INFO:teuthology.orchestra.run.smithi192.stdout: "slow_ops_count": 0 2024-09-21T06:09:07.556 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-21T06:09:07.556 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-21T06:09:08.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:07 smithi192 ceph-mon[34700]: pgmap v690: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-21T06:09:08.276 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:07 smithi117 ceph-mon[30947]: pgmap v690: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-21T06:09:08.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:07 smithi138 ceph-mon[34391]: pgmap v690: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-21T06:09:08.382 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-21T06:09:08.383 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-21T06:09:08.383 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-09-21T06:09:08.383 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-09-21T06:09:08.383 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=7 2024-09-21T06:09:08.383 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-09-21T06:09:08.383 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T06:09:08.383 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-21T06:09:08.393 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-21T06:09:08.393 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-21T06:09:08.393 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:09:08.419 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-21T06:09:08.419 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T06:09:08.479 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T06:09:08.779 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:09:09.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.273+0000 7fe72bba9640 1 -- 172.21.15.117:0/3810765510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe724109e20 msgr2=0x7fe724110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:09.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.273+0000 7fe72bba9640 1 --2- 172.21.15.117:0/3810765510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe724109e20 0x7fe724110650 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fe71c009900 tx=0x7fe71c02a410 comp rx=0 tx=0).stop 2024-09-21T06:09:09.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.273+0000 7fe72bba9640 1 -- 172.21.15.117:0/3810765510 shutdown_connections 2024-09-21T06:09:09.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.273+0000 7fe72bba9640 1 --2- 172.21.15.117:0/3810765510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe724109e20 0x7fe724110650 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:09.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.273+0000 7fe72bba9640 1 --2- 172.21.15.117:0/3810765510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7241094d0 0x7fe7241098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:09.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.273+0000 7fe72bba9640 1 --2- 172.21.15.117:0/3810765510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe724103b90 0x7fe724103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:09.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.273+0000 7fe72bba9640 1 -- 172.21.15.117:0/3810765510 >> 172.21.15.117:0/3810765510 conn(0x7fe7240ff600 msgr2=0x7fe724101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:09.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.273+0000 7fe72bba9640 1 -- 172.21.15.117:0/3810765510 shutdown_connections 2024-09-21T06:09:09.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.273+0000 7fe72bba9640 1 -- 172.21.15.117:0/3810765510 wait complete. 2024-09-21T06:09:09.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.274+0000 7fe72bba9640 1 Processor -- start 2024-09-21T06:09:09.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.274+0000 7fe72bba9640 1 -- start start 2024-09-21T06:09:09.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.274+0000 7fe72bba9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe724103b90 0x7fe72419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:09.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.274+0000 7fe72bba9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7241094d0 0x7fe72419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:09.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.274+0000 7fe72bba9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe724109e20 0x7fe72419fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:09.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.274+0000 7fe72bba9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe724113ae0 con 0x7fe724103b90 2024-09-21T06:09:09.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.274+0000 7fe72bba9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe724113960 con 0x7fe724109e20 2024-09-21T06:09:09.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.274+0000 7fe72bba9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe724113c60 con 0x7fe7241094d0 2024-09-21T06:09:09.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.274+0000 7fe729945640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe724103b90 0x7fe72419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:09.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.274+0000 7fe729945640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe724103b90 0x7fe72419a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38378/0 (socket says 172.21.15.117:38378) 2024-09-21T06:09:09.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.274+0000 7fe729945640 1 -- 172.21.15.117:0/2084881982 learned_addr learned my addr 172.21.15.117:0/2084881982 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:09:09.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.274+0000 7fe72a146640 1 --2- 172.21.15.117:0/2084881982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe724109e20 0x7fe72419fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:09.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.274+0000 7fe729144640 1 --2- 172.21.15.117:0/2084881982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7241094d0 0x7fe72419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:09.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.275+0000 7fe729945640 1 -- 172.21.15.117:0/2084881982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7241094d0 msgr2=0x7fe72419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:09.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.275+0000 7fe729945640 1 --2- 172.21.15.117:0/2084881982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7241094d0 0x7fe72419a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:09.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.275+0000 7fe729945640 1 -- 172.21.15.117:0/2084881982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe724109e20 msgr2=0x7fe72419fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:09.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.275+0000 7fe729945640 1 --2- 172.21.15.117:0/2084881982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe724109e20 0x7fe72419fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:09.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.275+0000 7fe729945640 1 -- 172.21.15.117:0/2084881982 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7241a02c0 con 0x7fe724103b90 2024-09-21T06:09:09.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.275+0000 7fe729945640 1 --2- 172.21.15.117:0/2084881982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe724103b90 0x7fe72419a410 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fe71400cce0 tx=0x7fe71400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:09.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.275+0000 7fe722ffd640 1 -- 172.21.15.117:0/2084881982 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe714004b20 con 0x7fe724103b90 2024-09-21T06:09:09.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.275+0000 7fe722ffd640 1 -- 172.21.15.117:0/2084881982 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe714004cc0 con 0x7fe724103b90 2024-09-21T06:09:09.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.275+0000 7fe722ffd640 1 -- 172.21.15.117:0/2084881982 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71400f6b0 con 0x7fe724103b90 2024-09-21T06:09:09.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.275+0000 7fe72bba9640 1 -- 172.21.15.117:0/2084881982 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7241a05b0 con 0x7fe724103b90 2024-09-21T06:09:09.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.275+0000 7fe72bba9640 1 -- 172.21.15.117:0/2084881982 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7241a6dd0 con 0x7fe724103b90 2024-09-21T06:09:09.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.276+0000 7fe72bba9640 1 -- 172.21.15.117:0/2084881982 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe724103fa0 con 0x7fe724103b90 2024-09-21T06:09:09.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.278+0000 7fe722ffd640 1 -- 172.21.15.117:0/2084881982 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fe71400f850 con 0x7fe724103b90 2024-09-21T06:09:09.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.278+0000 7fe722ffd640 1 --2- 172.21.15.117:0/2084881982 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe70c0b66b0 0x7fe70c0b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:09.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.278+0000 7fe729144640 1 --2- 172.21.15.117:0/2084881982 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe70c0b66b0 0x7fe70c0b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:09.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.279+0000 7fe722ffd640 1 -- 172.21.15.117:0/2084881982 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8441+0+0 (secure 0 0 0) 0x7fe7140e7b50 con 0x7fe724103b90 2024-09-21T06:09:09.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.279+0000 7fe729144640 1 --2- 172.21.15.117:0/2084881982 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe70c0b66b0 0x7fe70c0b8bc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe718005e10 tx=0x7fe718005da0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:09.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.281+0000 7fe722ffd640 1 -- 172.21.15.117:0/2084881982 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe714014030 con 0x7fe724103b90 2024-09-21T06:09:09.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.471+0000 7fe72bba9640 1 -- 172.21.15.117:0/2084881982 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe72419b540 con 0x7fe724103b90 2024-09-21T06:09:09.473 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.471+0000 7fe722ffd640 1 -- 172.21.15.117:0/2084881982 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v85) ==== 81+0+899 (secure 0 0 0) 0x7fe7140ae0f0 con 0x7fe724103b90 2024-09-21T06:09:09.473 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/2989394419 2024-09-22T05:48:32.406654+0000 2024-09-21T06:09:09.473 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6801/1702676172 2024-09-22T05:48:32.406654+0000 2024-09-21T06:09:09.473 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/1766406059 2024-09-22T05:47:09.112435+0000 2024-09-21T06:09:09.473 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6800/2337036514 2024-09-22T05:47:09.112435+0000 2024-09-21T06:09:09.473 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/985798343 2024-09-22T05:47:09.112435+0000 2024-09-21T06:09:09.473 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/1083273222 2024-09-22T05:48:32.406654+0000 2024-09-21T06:09:09.474 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/473284565 2024-09-22T05:47:09.112435+0000 2024-09-21T06:09:09.474 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6801/2337036514 2024-09-22T05:47:09.112435+0000 2024-09-21T06:09:09.474 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6800/816635165 2024-09-22T05:47:33.789146+0000 2024-09-21T06:09:09.474 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/2642940197 2024-09-22T05:48:32.406654+0000 2024-09-21T06:09:09.474 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6801/816635165 2024-09-22T05:47:33.789146+0000 2024-09-21T06:09:09.474 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/2225537433 2024-09-22T05:47:33.789146+0000 2024-09-21T06:09:09.474 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/3651584169 2024-09-22T05:47:33.789146+0000 2024-09-21T06:09:09.474 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:6800/1702676172 2024-09-22T05:48:32.406654+0000 2024-09-21T06:09:09.474 INFO:teuthology.orchestra.run.smithi117.stdout:172.21.15.117:0/1284403882 2024-09-22T05:47:33.789146+0000 2024-09-21T06:09:09.474 INFO:teuthology.orchestra.run.smithi117.stderr:listed 15 entries 2024-09-21T06:09:09.475 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.474+0000 7fe72bba9640 1 -- 172.21.15.117:0/2084881982 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe70c0b66b0 msgr2=0x7fe70c0b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:09.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.474+0000 7fe72bba9640 1 --2- 172.21.15.117:0/2084881982 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe70c0b66b0 0x7fe70c0b8bc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe718005e10 tx=0x7fe718005da0 comp rx=0 tx=0).stop 2024-09-21T06:09:09.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.474+0000 7fe72bba9640 1 -- 172.21.15.117:0/2084881982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe724103b90 msgr2=0x7fe72419a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:09.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.474+0000 7fe72bba9640 1 --2- 172.21.15.117:0/2084881982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe724103b90 0x7fe72419a410 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fe71400cce0 tx=0x7fe71400a620 comp rx=0 tx=0).stop 2024-09-21T06:09:09.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.474+0000 7fe72bba9640 1 -- 172.21.15.117:0/2084881982 shutdown_connections 2024-09-21T06:09:09.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.474+0000 7fe72bba9640 1 --2- 172.21.15.117:0/2084881982 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fe70c0b66b0 0x7fe70c0b8bc0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:09.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.474+0000 7fe72bba9640 1 --2- 172.21.15.117:0/2084881982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe724109e20 0x7fe72419fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:09.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.474+0000 7fe72bba9640 1 --2- 172.21.15.117:0/2084881982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7241094d0 0x7fe72419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:09.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.474+0000 7fe72bba9640 1 --2- 172.21.15.117:0/2084881982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe724103b90 0x7fe72419a410 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:09.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.474+0000 7fe72bba9640 1 -- 172.21.15.117:0/2084881982 >> 172.21.15.117:0/2084881982 conn(0x7fe7240ff600 msgr2=0x7fe724106a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:09.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.474+0000 7fe72bba9640 1 -- 172.21.15.117:0/2084881982 shutdown_connections 2024-09-21T06:09:09.476 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:09.474+0000 7fe72bba9640 1 -- 172.21.15.117:0/2084881982 wait complete. 2024-09-21T06:09:10.250 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:09 smithi117 ceph-mon[30947]: pgmap v691: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-09-21T06:09:10.250 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:09 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2084881982' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T06:09:10.298 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T06:09:10.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:09:10.299 DEBUG:teuthology.orchestra.run.smithi117:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T06:09:10.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:09 smithi138 ceph-mon[34391]: pgmap v691: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-09-21T06:09:10.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:09 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2084881982' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T06:09:10.479 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:09 smithi192 ceph-mon[34700]: pgmap v691: 97 pgs: 97 active+clean; 30 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 45 KiB/s wr, 27 op/s 2024-09-21T06:09:10.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:09 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2084881982' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T06:09:10.495 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi117.front.sepia.ceph.com 2024-09-21T06:09:10.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:09:10.495 DEBUG:teuthology.orchestra.run.smithi117:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T06:09:10.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:09:10.559 DEBUG:teuthology.orchestra.run.smithi117:> ip netns list 2024-09-21T06:09:10.588 INFO:teuthology.orchestra.run.smithi117.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T06:09:10.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:09:10.589 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T06:09:10.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T06:09:10.672 DEBUG:teuthology.orchestra.run.smithi117:> sudo ip link delete ceph-brx 2024-09-21T06:09:10.774 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-21T06:09:10.784 INFO:tasks.cephadm:Teardown begin 2024-09-21T06:09:10.784 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T06:09:10.784 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:09:11.218 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:09:11.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:11 smithi117 ceph-mon[30947]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T06:09:11.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:11 smithi138 ceph-mon[34391]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T06:09:11.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:11 smithi192 ceph-mon[34700]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T06:09:11.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.839+0000 7fdc6a16e640 1 -- 172.21.15.117:0/889783197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc641083d0 msgr2=0x7fdc641087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:11.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.839+0000 7fdc6a16e640 1 --2- 172.21.15.117:0/889783197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc641083d0 0x7fdc641087e0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fdc58009900 tx=0x7fdc5802a410 comp rx=0 tx=0).stop 2024-09-21T06:09:11.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.839+0000 7fdc6a16e640 1 -- 172.21.15.117:0/889783197 shutdown_connections 2024-09-21T06:09:11.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.839+0000 7fdc6a16e640 1 --2- 172.21.15.117:0/889783197 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc641083d0 0x7fdc641087e0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:11.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.839+0000 7fdc6a16e640 1 --2- 172.21.15.117:0/889783197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc64108d20 0x7fdc6410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:11.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.839+0000 7fdc6a16e640 1 --2- 172.21.15.117:0/889783197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc64102ae0 0x7fdc64102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:11.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.839+0000 7fdc6a16e640 1 -- 172.21.15.117:0/889783197 >> 172.21.15.117:0/889783197 conn(0x7fdc64079760 msgr2=0x7fdc64079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:11.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.839+0000 7fdc6a16e640 1 -- 172.21.15.117:0/889783197 shutdown_connections 2024-09-21T06:09:11.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.839+0000 7fdc6a16e640 1 -- 172.21.15.117:0/889783197 wait complete. 2024-09-21T06:09:11.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.840+0000 7fdc6a16e640 1 Processor -- start 2024-09-21T06:09:11.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.840+0000 7fdc6a16e640 1 -- start start 2024-09-21T06:09:11.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.840+0000 7fdc6a16e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc64102ae0 0x7fdc6419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:11.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.840+0000 7fdc6a16e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc641083d0 0x7fdc6419b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:11.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.840+0000 7fdc6a16e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc64108d20 0x7fdc6419be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:11.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.840+0000 7fdc6a16e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdc64112990 con 0x7fdc64108d20 2024-09-21T06:09:11.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.840+0000 7fdc6a16e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdc64112810 con 0x7fdc64102ae0 2024-09-21T06:09:11.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.840+0000 7fdc6a16e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdc64112b10 con 0x7fdc641083d0 2024-09-21T06:09:11.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.840+0000 7fdc63fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc64108d20 0x7fdc6419be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:11.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.840+0000 7fdc63fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc64108d20 0x7fdc6419be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38408/0 (socket says 172.21.15.117:38408) 2024-09-21T06:09:11.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.840+0000 7fdc63fff640 1 -- 172.21.15.117:0/1653453697 learned_addr learned my addr 172.21.15.117:0/1653453697 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:09:11.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.840+0000 7fdc637fe640 1 --2- 172.21.15.117:0/1653453697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc64102ae0 0x7fdc6419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:11.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.840+0000 7fdc62ffd640 1 --2- 172.21.15.117:0/1653453697 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc641083d0 0x7fdc6419b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:11.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.841+0000 7fdc63fff640 1 -- 172.21.15.117:0/1653453697 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc641083d0 msgr2=0x7fdc6419b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:11.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.841+0000 7fdc63fff640 1 --2- 172.21.15.117:0/1653453697 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc641083d0 0x7fdc6419b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:11.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.841+0000 7fdc63fff640 1 -- 172.21.15.117:0/1653453697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc64102ae0 msgr2=0x7fdc6419b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:11.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.841+0000 7fdc63fff640 1 --2- 172.21.15.117:0/1653453697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc64102ae0 0x7fdc6419b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:11.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.841+0000 7fdc63fff640 1 -- 172.21.15.117:0/1653453697 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc641991f0 con 0x7fdc64108d20 2024-09-21T06:09:11.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.841+0000 7fdc637fe640 1 --2- 172.21.15.117:0/1653453697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc64102ae0 0x7fdc6419b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T06:09:11.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.841+0000 7fdc63fff640 1 --2- 172.21.15.117:0/1653453697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc64108d20 0x7fdc6419be00 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c00eb10 tx=0x7fdc5c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:11.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.841+0000 7fdc60ff9640 1 -- 172.21.15.117:0/1653453697 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c004b20 con 0x7fdc64108d20 2024-09-21T06:09:11.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.841+0000 7fdc6a16e640 1 -- 172.21.15.117:0/1653453697 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc641994e0 con 0x7fdc64108d20 2024-09-21T06:09:11.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.841+0000 7fdc60ff9640 1 -- 172.21.15.117:0/1653453697 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc5c004cc0 con 0x7fdc64108d20 2024-09-21T06:09:11.845 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.841+0000 7fdc60ff9640 1 -- 172.21.15.117:0/1653453697 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c010670 con 0x7fdc64108d20 2024-09-21T06:09:11.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.841+0000 7fdc6a16e640 1 -- 172.21.15.117:0/1653453697 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc641999f0 con 0x7fdc64108d20 2024-09-21T06:09:11.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.842+0000 7fdc6a16e640 1 -- 172.21.15.117:0/1653453697 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc64102ef0 con 0x7fdc64108d20 2024-09-21T06:09:11.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.844+0000 7fdc60ff9640 1 -- 172.21.15.117:0/1653453697 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fdc5c005000 con 0x7fdc64108d20 2024-09-21T06:09:11.846 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.845+0000 7fdc60ff9640 1 --2- 172.21.15.117:0/1653453697 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fdc4c0b6680 0x7fdc4c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:11.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.845+0000 7fdc60ff9640 1 -- 172.21.15.117:0/1653453697 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8485+0+0 (secure 0 0 0) 0x7fdc5c0e7e60 con 0x7fdc64108d20 2024-09-21T06:09:11.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.845+0000 7fdc637fe640 1 --2- 172.21.15.117:0/1653453697 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fdc4c0b6680 0x7fdc4c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:11.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.845+0000 7fdc637fe640 1 --2- 172.21.15.117:0/1653453697 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fdc4c0b6680 0x7fdc4c0b8b90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdc6406c660 tx=0x7fdc5400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:11.849 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:11.848+0000 7fdc60ff9640 1 -- 172.21.15.117:0/1653453697 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdc5c056710 con 0x7fdc64108d20 2024-09-21T06:09:12.069 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:12.067+0000 7fdc6a16e640 1 -- 172.21.15.117:0/1653453697 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc6406bf10 con 0x7fdc64108d20 2024-09-21T06:09:12.070 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:12.068+0000 7fdc60ff9640 1 -- 172.21.15.117:0/1653453697 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+12001 (secure 0 0 0) 0x7fdc5c0ae3d0 con 0x7fdc64108d20 2024-09-21T06:09:12.070 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T06:09:12.070 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":29,"btime":"2024-09-21T06:04:08:952483+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3227057504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3227057504},{"type":"v1","addr":"172.21.15.117:6835","nonce":3227057504}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3303829183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3303829183},{"type":"v1","addr":"172.21.15.117:6837","nonce":3303829183}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T05:55:14.555897+0000","modified":"2024-09-21T06:04:08.952466+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24638,"mds_1":24637,"mds_2":24628,"mds_3":24616,"mds_4":24602},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6839/2314572411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2314572411},{"type":"v1","addr":"172.21.15.117:6839","nonce":2314572411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6841/3378096561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":3378096561},{"type":"v1","addr":"172.21.15.117:6841","nonce":3378096561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/3725629862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3725629862},{"type":"v1","addr":"172.21.15.138:6833","nonce":3725629862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/4276801396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4276801396},{"type":"v1","addr":"172.21.15.138:6835","nonce":4276801396}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.138:6837/2540335374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2540335374},{"type":"v1","addr":"172.21.15.138:6837","nonce":2540335374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24616":{"gid":24616,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.192:6835/3530688860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3530688860},{"type":"v1","addr":"172.21.15.192:6835","nonce":3530688860}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.192:6837/2683860842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2683860842},{"type":"v1","addr":"172.21.15.192:6837","nonce":2683860842}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.192:6839/1727623231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1727623231},{"type":"v1","addr":"172.21.15.192:6839","nonce":1727623231}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/3230372576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":3230372576},{"type":"v1","addr":"172.21.15.138:6839","nonce":3230372576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25840":{"gid":25840,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6833/1596605058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1596605058},{"type":"v1","addr":"172.21.15.192:6833","nonce":1596605058}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24602,24628,24616,24637,24638]},"id":1}]} 2024-09-21T06:09:12.071 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 29 2024-09-21T06:09:12.072 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:12.070+0000 7fdc6a16e640 1 -- 172.21.15.117:0/1653453697 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fdc4c0b6680 msgr2=0x7fdc4c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:12.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:12.070+0000 7fdc6a16e640 1 --2- 172.21.15.117:0/1653453697 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fdc4c0b6680 0x7fdc4c0b8b90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdc6406c660 tx=0x7fdc5400b390 comp rx=0 tx=0).stop 2024-09-21T06:09:12.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:12.070+0000 7fdc6a16e640 1 -- 172.21.15.117:0/1653453697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc64108d20 msgr2=0x7fdc6419be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:12.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:12.070+0000 7fdc6a16e640 1 --2- 172.21.15.117:0/1653453697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc64108d20 0x7fdc6419be00 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c00eb10 tx=0x7fdc5c00c2d0 comp rx=0 tx=0).stop 2024-09-21T06:09:12.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:12.070+0000 7fdc6a16e640 1 -- 172.21.15.117:0/1653453697 shutdown_connections 2024-09-21T06:09:12.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:12.070+0000 7fdc6a16e640 1 --2- 172.21.15.117:0/1653453697 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fdc4c0b6680 0x7fdc4c0b8b90 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:12.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:12.070+0000 7fdc6a16e640 1 --2- 172.21.15.117:0/1653453697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc64108d20 0x7fdc6419be00 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:12.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:12.070+0000 7fdc6a16e640 1 --2- 172.21.15.117:0/1653453697 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc641083d0 0x7fdc6419b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:12.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:12.070+0000 7fdc6a16e640 1 --2- 172.21.15.117:0/1653453697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc64102ae0 0x7fdc6419b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:12.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:12.070+0000 7fdc6a16e640 1 -- 172.21.15.117:0/1653453697 >> 172.21.15.117:0/1653453697 conn(0x7fdc64079760 msgr2=0x7fdc64104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:12.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:12.071+0000 7fdc6a16e640 1 -- 172.21.15.117:0/1653453697 shutdown_connections 2024-09-21T06:09:12.073 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:12.071+0000 7fdc6a16e640 1 -- 172.21.15.117:0/1653453697 wait complete. 2024-09-21T06:09:12.277 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:12 smithi117 ceph-mon[30947]: pgmap v693: 97 pgs: 10 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 83 active+clean; 29 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-21T06:09:12.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:12 smithi138 ceph-mon[34391]: pgmap v693: 97 pgs: 10 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 83 active+clean; 29 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-21T06:09:12.479 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:12 smithi192 ceph-mon[34700]: pgmap v693: 97 pgs: 10 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 83 active+clean; 29 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-09-21T06:09:12.824 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T06:09:13.073 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:09:13.194 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:13 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/1653453697' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:09:13.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:13 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/1653453697' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:09:13.479 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:13 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/1653453697' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:09:13.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.603+0000 7fb713270640 1 -- 172.21.15.117:0/1255078800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb70c108d00 msgr2=0x7fb70c10f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:13.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.603+0000 7fb713270640 1 --2- 172.21.15.117:0/1255078800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb70c108d00 0x7fb70c10f530 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fb704009ca0 tx=0x7fb70402a8c0 comp rx=0 tx=0).stop 2024-09-21T06:09:13.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.603+0000 7fb713270640 1 -- 172.21.15.117:0/1255078800 shutdown_connections 2024-09-21T06:09:13.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.603+0000 7fb713270640 1 --2- 172.21.15.117:0/1255078800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb70c108d00 0x7fb70c10f530 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:13.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.603+0000 7fb713270640 1 --2- 172.21.15.117:0/1255078800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb70c1083b0 0x7fb70c1087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:13.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.603+0000 7fb713270640 1 --2- 172.21.15.117:0/1255078800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb70c102a70 0x7fb70c102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:13.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.604+0000 7fb713270640 1 -- 172.21.15.117:0/1255078800 >> 172.21.15.117:0/1255078800 conn(0x7fb70c079730 msgr2=0x7fb70c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:13.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.604+0000 7fb713270640 1 -- 172.21.15.117:0/1255078800 shutdown_connections 2024-09-21T06:09:13.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.604+0000 7fb713270640 1 -- 172.21.15.117:0/1255078800 wait complete. 2024-09-21T06:09:13.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.604+0000 7fb713270640 1 Processor -- start 2024-09-21T06:09:13.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.604+0000 7fb713270640 1 -- start start 2024-09-21T06:09:13.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.604+0000 7fb713270640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb70c102a70 0x7fb70c199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:13.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.604+0000 7fb713270640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb70c1083b0 0x7fb70c1998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:13.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.605+0000 7fb713270640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb70c108d00 0x7fb70c19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:13.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.605+0000 7fb713270640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c112970 con 0x7fb70c1083b0 2024-09-21T06:09:13.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.605+0000 7fb713270640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c1127f0 con 0x7fb70c108d00 2024-09-21T06:09:13.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.605+0000 7fb713270640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c112af0 con 0x7fb70c102a70 2024-09-21T06:09:13.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.605+0000 7fb71080b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb70c1083b0 0x7fb70c1998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:13.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.605+0000 7fb71080b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb70c1083b0 0x7fb70c1998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38426/0 (socket says 172.21.15.117:38426) 2024-09-21T06:09:13.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.605+0000 7fb71080b640 1 -- 172.21.15.117:0/2644167592 learned_addr learned my addr 172.21.15.117:0/2644167592 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:09:13.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.605+0000 7fb71180d640 1 --2- 172.21.15.117:0/2644167592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb70c108d00 0x7fb70c19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:13.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.605+0000 7fb71100c640 1 --2- 172.21.15.117:0/2644167592 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb70c102a70 0x7fb70c199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:13.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.605+0000 7fb71080b640 1 -- 172.21.15.117:0/2644167592 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb70c102a70 msgr2=0x7fb70c199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:13.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.605+0000 7fb71080b640 1 --2- 172.21.15.117:0/2644167592 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb70c102a70 0x7fb70c199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:13.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.605+0000 7fb71080b640 1 -- 172.21.15.117:0/2644167592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb70c108d00 msgr2=0x7fb70c19eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:13.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.605+0000 7fb71080b640 1 --2- 172.21.15.117:0/2644167592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb70c108d00 0x7fb70c19eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:13.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.605+0000 7fb71080b640 1 -- 172.21.15.117:0/2644167592 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb70c19f1d0 con 0x7fb70c1083b0 2024-09-21T06:09:13.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.605+0000 7fb71080b640 1 --2- 172.21.15.117:0/2644167592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb70c1083b0 0x7fb70c1998c0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc00c970 tx=0x7fb6fc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:13.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.605+0000 7fb7027fc640 1 -- 172.21.15.117:0/2644167592 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc014070 con 0x7fb70c1083b0 2024-09-21T06:09:13.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.605+0000 7fb713270640 1 -- 172.21.15.117:0/2644167592 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb70c19f4c0 con 0x7fb70c1083b0 2024-09-21T06:09:13.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.605+0000 7fb7027fc640 1 -- 172.21.15.117:0/2644167592 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6fc004440 con 0x7fb70c1083b0 2024-09-21T06:09:13.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.605+0000 7fb713270640 1 -- 172.21.15.117:0/2644167592 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb70c1a5ce0 con 0x7fb70c1083b0 2024-09-21T06:09:13.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.605+0000 7fb7027fc640 1 -- 172.21.15.117:0/2644167592 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc003e50 con 0x7fb70c1083b0 2024-09-21T06:09:13.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.606+0000 7fb713270640 1 -- 172.21.15.117:0/2644167592 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb70c103cf0 con 0x7fb70c1083b0 2024-09-21T06:09:13.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.608+0000 7fb7027fc640 1 -- 172.21.15.117:0/2644167592 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fb6fc017070 con 0x7fb70c1083b0 2024-09-21T06:09:13.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.609+0000 7fb7027fc640 1 --2- 172.21.15.117:0/2644167592 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6f80b6850 0x7fb6f80b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:13.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.609+0000 7fb71100c640 1 --2- 172.21.15.117:0/2644167592 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6f80b6850 0x7fb6f80b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:13.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.609+0000 7fb7027fc640 1 -- 172.21.15.117:0/2644167592 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8485+0+0 (secure 0 0 0) 0x7fb6fc0e7960 con 0x7fb70c1083b0 2024-09-21T06:09:13.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.609+0000 7fb71100c640 1 --2- 172.21.15.117:0/2644167592 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6f80b6850 0x7fb6f80b8d60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb708005fd0 tx=0x7fb708005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:13.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.612+0000 7fb7027fc640 1 -- 172.21.15.117:0/2644167592 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb6fc01e020 con 0x7fb70c1083b0 2024-09-21T06:09:13.826 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.824+0000 7fb713270640 1 -- 172.21.15.117:0/2644167592 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb70c19a450 con 0x7fb70c1083b0 2024-09-21T06:09:13.827 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.825+0000 7fb7027fc640 1 -- 172.21.15.117:0/2644167592 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+12001 (secure 0 0 0) 0x7fb70c19a450 con 0x7fb70c1083b0 2024-09-21T06:09:13.827 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T06:09:13.827 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":29,"btime":"2024-09-21T06:04:08:952483+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14775,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3227057504","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3227057504},{"type":"v1","addr":"172.21.15.117:6835","nonce":3227057504}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3303829183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3303829183},{"type":"v1","addr":"172.21.15.117:6837","nonce":3303829183}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T05:55:14.555897+0000","modified":"2024-09-21T06:04:08.952466+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24638,"mds_1":24637,"mds_2":24628,"mds_3":24616,"mds_4":24602},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14838":{"gid":14838,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6839/2314572411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":2314572411},{"type":"v1","addr":"172.21.15.117:6839","nonce":2314572411}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14850":{"gid":14850,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6841/3378096561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6840","nonce":3378096561},{"type":"v1","addr":"172.21.15.117:6841","nonce":3378096561}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6833/3725629862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":3725629862},{"type":"v1","addr":"172.21.15.138:6833","nonce":3725629862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6835/4276801396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4276801396},{"type":"v1","addr":"172.21.15.138:6835","nonce":4276801396}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24602":{"gid":24602,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":10,"addr":"172.21.15.138:6837/2540335374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2540335374},{"type":"v1","addr":"172.21.15.138:6837","nonce":2540335374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24616":{"gid":24616,"name":"f","rank":3,"incarnation":22,"state":"up:active","state_seq":11,"addr":"172.21.15.192:6835/3530688860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3530688860},{"type":"v1","addr":"172.21.15.192:6835","nonce":3530688860}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.192:6837/2683860842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":2683860842},{"type":"v1","addr":"172.21.15.192:6837","nonce":2683860842}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24637":{"gid":24637,"name":"i","rank":1,"incarnation":18,"state":"up:active","state_seq":8,"addr":"172.21.15.192:6839/1727623231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":1727623231},{"type":"v1","addr":"172.21.15.192:6839","nonce":1727623231}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24638":{"gid":24638,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6839/3230372576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":3230372576},{"type":"v1","addr":"172.21.15.138:6839","nonce":3230372576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25840":{"gid":25840,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6833/1596605058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1596605058},{"type":"v1","addr":"172.21.15.192:6833","nonce":1596605058}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24638,"qdb_cluster":[24602,24628,24616,24637,24638]},"id":1}]} 2024-09-21T06:09:13.827 INFO:teuthology.orchestra.run.smithi117.stderr:dumped fsmap epoch 29 2024-09-21T06:09:13.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.827+0000 7fb713270640 1 -- 172.21.15.117:0/2644167592 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6f80b6850 msgr2=0x7fb6f80b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:13.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.827+0000 7fb713270640 1 --2- 172.21.15.117:0/2644167592 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6f80b6850 0x7fb6f80b8d60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb708005fd0 tx=0x7fb708005850 comp rx=0 tx=0).stop 2024-09-21T06:09:13.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.827+0000 7fb713270640 1 -- 172.21.15.117:0/2644167592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb70c1083b0 msgr2=0x7fb70c1998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:13.829 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.827+0000 7fb713270640 1 --2- 172.21.15.117:0/2644167592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb70c1083b0 0x7fb70c1998c0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc00c970 tx=0x7fb6fc00ce30 comp rx=0 tx=0).stop 2024-09-21T06:09:13.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.827+0000 7fb713270640 1 -- 172.21.15.117:0/2644167592 shutdown_connections 2024-09-21T06:09:13.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.827+0000 7fb713270640 1 --2- 172.21.15.117:0/2644167592 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb6f80b6850 0x7fb6f80b8d60 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:13.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.827+0000 7fb713270640 1 --2- 172.21.15.117:0/2644167592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb70c1083b0 0x7fb70c1998c0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:13.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.827+0000 7fb713270640 1 --2- 172.21.15.117:0/2644167592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb70c108d00 0x7fb70c19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:13.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.827+0000 7fb713270640 1 --2- 172.21.15.117:0/2644167592 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb70c102a70 0x7fb70c199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:13.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.827+0000 7fb713270640 1 -- 172.21.15.117:0/2644167592 >> 172.21.15.117:0/2644167592 conn(0x7fb70c079730 msgr2=0x7fb70c1043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:13.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.827+0000 7fb713270640 1 -- 172.21.15.117:0/2644167592 shutdown_connections 2024-09-21T06:09:13.830 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:13.827+0000 7fb713270640 1 -- 172.21.15.117:0/2644167592 wait complete. 2024-09-21T06:09:14.137 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:14 smithi117 ceph-mon[30947]: pgmap v694: 97 pgs: 12 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 79 active+clean; 26 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-09-21T06:09:14.137 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:14 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2644167592' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:09:14.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:14 smithi138 ceph-mon[34391]: pgmap v694: 97 pgs: 12 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 79 active+clean; 26 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-09-21T06:09:14.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:14 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2644167592' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:09:14.475 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T06:09:14.479 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:14 smithi192 ceph-mon[34700]: pgmap v694: 97 pgs: 12 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 79 active+clean; 26 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-09-21T06:09:14.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:14 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2644167592' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T06:09:14.722 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:09:15.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.222+0000 7f4bf6575640 1 -- 172.21.15.117:0/1819184640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bf0103b60 msgr2=0x7f4bf0103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:15.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.222+0000 7f4bf6575640 1 --2- 172.21.15.117:0/1819184640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bf0103b60 0x7f4bf0103f70 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f4be0009900 tx=0x7f4be002a410 comp rx=0 tx=0).stop 2024-09-21T06:09:15.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.222+0000 7f4bf6575640 1 -- 172.21.15.117:0/1819184640 shutdown_connections 2024-09-21T06:09:15.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.222+0000 7f4bf6575640 1 --2- 172.21.15.117:0/1819184640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bf0109df0 0x7f4bf0110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:15.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.222+0000 7f4bf6575640 1 --2- 172.21.15.117:0/1819184640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bf01094a0 0x7f4bf01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:15.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.222+0000 7f4bf6575640 1 --2- 172.21.15.117:0/1819184640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bf0103b60 0x7f4bf0103f70 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:15.235 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.222+0000 7f4bf6575640 1 -- 172.21.15.117:0/1819184640 >> 172.21.15.117:0/1819184640 conn(0x7f4bf007b850 msgr2=0x7f4bf007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:15.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.223+0000 7f4bf6575640 1 -- 172.21.15.117:0/1819184640 shutdown_connections 2024-09-21T06:09:15.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.223+0000 7f4bf6575640 1 -- 172.21.15.117:0/1819184640 wait complete. 2024-09-21T06:09:15.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.223+0000 7f4bf6575640 1 Processor -- start 2024-09-21T06:09:15.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.223+0000 7f4bf6575640 1 -- start start 2024-09-21T06:09:15.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.224+0000 7f4bf6575640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bf0103b60 0x7f4bf019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:15.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.224+0000 7f4bf6575640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bf01094a0 0x7f4bf019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:15.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.224+0000 7f4bf6575640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bf0109df0 0x7f4bf019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:15.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.224+0000 7f4bf6575640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf0113ab0 con 0x7f4bf01094a0 2024-09-21T06:09:15.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.224+0000 7f4bf6575640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf0113930 con 0x7f4bf0103b60 2024-09-21T06:09:15.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.224+0000 7f4bf6575640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf0113c30 con 0x7f4bf0109df0 2024-09-21T06:09:15.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.224+0000 7f4bef7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bf01094a0 0x7f4bf019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:15.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.224+0000 7f4bef7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bf01094a0 0x7f4bf019aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48608/0 (socket says 172.21.15.117:48608) 2024-09-21T06:09:15.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.224+0000 7f4bef7fe640 1 -- 172.21.15.117:0/1514782788 learned_addr learned my addr 172.21.15.117:0/1514782788 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:09:15.236 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.224+0000 7f4bf4b12640 1 --2- 172.21.15.117:0/1514782788 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bf0109df0 0x7f4bf019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:15.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.224+0000 7f4beffff640 1 --2- 172.21.15.117:0/1514782788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bf0103b60 0x7f4bf019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:15.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.224+0000 7f4bef7fe640 1 -- 172.21.15.117:0/1514782788 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bf0109df0 msgr2=0x7f4bf019fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:15.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.224+0000 7f4bef7fe640 1 --2- 172.21.15.117:0/1514782788 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bf0109df0 0x7f4bf019fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:15.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.224+0000 7f4bef7fe640 1 -- 172.21.15.117:0/1514782788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bf0103b60 msgr2=0x7f4bf019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:15.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.224+0000 7f4bef7fe640 1 --2- 172.21.15.117:0/1514782788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bf0103b60 0x7f4bf019a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:15.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.224+0000 7f4bef7fe640 1 -- 172.21.15.117:0/1514782788 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bf01a0410 con 0x7f4bf01094a0 2024-09-21T06:09:15.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.224+0000 7f4beffff640 1 --2- 172.21.15.117:0/1514782788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bf0103b60 0x7f4bf019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T06:09:15.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.224+0000 7f4bef7fe640 1 --2- 172.21.15.117:0/1514782788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bf01094a0 0x7f4bf019aa40 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f4be400d950 tx=0x7f4be400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:15.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.224+0000 7f4bed7fa640 1 -- 172.21.15.117:0/1514782788 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be4017070 con 0x7f4bf01094a0 2024-09-21T06:09:15.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.225+0000 7f4bf6575640 1 -- 172.21.15.117:0/1514782788 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bf01a0700 con 0x7f4bf01094a0 2024-09-21T06:09:15.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.225+0000 7f4bed7fa640 1 -- 172.21.15.117:0/1514782788 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4be4009c30 con 0x7f4bf01094a0 2024-09-21T06:09:15.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.225+0000 7f4bed7fa640 1 -- 172.21.15.117:0/1514782788 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be40047a0 con 0x7f4bf01094a0 2024-09-21T06:09:15.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.225+0000 7f4bf6575640 1 -- 172.21.15.117:0/1514782788 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bf01a6e60 con 0x7f4bf01094a0 2024-09-21T06:09:15.237 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.226+0000 7f4bf6575640 1 -- 172.21.15.117:0/1514782788 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bf0103f70 con 0x7f4bf01094a0 2024-09-21T06:09:15.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.228+0000 7f4bed7fa640 1 -- 172.21.15.117:0/1514782788 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f4be4004020 con 0x7f4bf01094a0 2024-09-21T06:09:15.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.229+0000 7f4bed7fa640 1 --2- 172.21.15.117:0/1514782788 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4bd80b6680 0x7f4bd80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:15.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.229+0000 7f4beffff640 1 --2- 172.21.15.117:0/1514782788 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4bd80b6680 0x7f4bd80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:15.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.229+0000 7f4bed7fa640 1 -- 172.21.15.117:0/1514782788 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8485+0+0 (secure 0 0 0) 0x7f4be40e7810 con 0x7f4bf01094a0 2024-09-21T06:09:15.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.229+0000 7f4beffff640 1 --2- 172.21.15.117:0/1514782788 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4bd80b6680 0x7f4bd80b8b90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4be0031440 tx=0x7f4be0009680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:15.238 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.232+0000 7f4bed7fa640 1 -- 172.21.15.117:0/1514782788 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4be4020020 con 0x7f4bf01094a0 2024-09-21T06:09:15.412 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.410+0000 7f4bf6575640 1 -- 172.21.15.117:0/1514782788 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4bf019b5d0 con 0x7f4bf01094a0 2024-09-21T06:09:15.414 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.412+0000 7f4bed7fa640 1 -- 172.21.15.117:0/1514782788 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+23223 (secure 0 0 0) 0x7f4bf019b5d0 con 0x7f4bf01094a0 2024-09-21T06:09:15.414 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-21T06:09:15.414 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":86,"fsid":"ac8581a2-77dc-11ef-baf5-efdc52797490","created":"2024-09-21T05:46:47.021846+0000","modified":"2024-09-21T06:09:09.991167+0000","last_up_change":"2024-09-21T05:54:21.572849+0000","last_in_change":"2024-09-21T05:53:58.892597+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:50:32.717389+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:55:10.604159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T05:55:12.581258+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":18,"snap_epoch":86,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"68315502-fa6c-4a6e-a2d4-a7dde8071c02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6803","nonce":3761478035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6805","nonce":3761478035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6809","nonce":3761478035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3761478035},{"type":"v1","addr":"172.21.15.117:6807","nonce":3761478035}]},"public_addr":"172.21.15.117:6803/3761478035","cluster_addr":"172.21.15.117:6805/3761478035","heartbeat_back_addr":"172.21.15.117:6809/3761478035","heartbeat_front_addr":"172.21.15.117:6807/3761478035","state":["exists","up"]},{"osd":1,"uuid":"d8910a08-4ef3-49f2-a4db-e3588d85d12b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6801","nonce":1737365588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6803","nonce":1737365588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6807","nonce":1737365588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":1737365588},{"type":"v1","addr":"172.21.15.138:6805","nonce":1737365588}]},"public_addr":"172.21.15.138:6801/1737365588","cluster_addr":"172.21.15.138:6803/1737365588","heartbeat_back_addr":"172.21.15.138:6807/1737365588","heartbeat_front_addr":"172.21.15.138:6805/1737365588","state":["exists","up"]},{"osd":2,"uuid":"94972378-a11f-4129-9e40-ad3fdec1d189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6801","nonce":3002534992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6803","nonce":3002534992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6807","nonce":3002534992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":3002534992},{"type":"v1","addr":"172.21.15.192:6805","nonce":3002534992}]},"public_addr":"172.21.15.192:6801/3002534992","cluster_addr":"172.21.15.192:6803/3002534992","heartbeat_back_addr":"172.21.15.192:6807/3002534992","heartbeat_front_addr":"172.21.15.192:6805/3002534992","state":["exists","up"]},{"osd":3,"uuid":"d4168c43-f0de-4445-945c-92d1cf270c33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6811","nonce":1222400293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6813","nonce":1222400293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6817","nonce":1222400293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1222400293},{"type":"v1","addr":"172.21.15.117:6815","nonce":1222400293}]},"public_addr":"172.21.15.117:6811/1222400293","cluster_addr":"172.21.15.117:6813/1222400293","heartbeat_back_addr":"172.21.15.117:6817/1222400293","heartbeat_front_addr":"172.21.15.117:6815/1222400293","state":["exists","up"]},{"osd":4,"uuid":"28f1d7c0-8392-4a33-85c0-0de611cdd4b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6809","nonce":4294892870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6811","nonce":4294892870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6815","nonce":4294892870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":4294892870},{"type":"v1","addr":"172.21.15.138:6813","nonce":4294892870}]},"public_addr":"172.21.15.138:6809/4294892870","cluster_addr":"172.21.15.138:6811/4294892870","heartbeat_back_addr":"172.21.15.138:6815/4294892870","heartbeat_front_addr":"172.21.15.138:6813/4294892870","state":["exists","up"]},{"osd":5,"uuid":"ca902407-45e8-4bc5-bdbd-a729c1de8322","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6809","nonce":1776840019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6811","nonce":1776840019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6815","nonce":1776840019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1776840019},{"type":"v1","addr":"172.21.15.192:6813","nonce":1776840019}]},"public_addr":"172.21.15.192:6809/1776840019","cluster_addr":"172.21.15.192:6811/1776840019","heartbeat_back_addr":"172.21.15.192:6815/1776840019","heartbeat_front_addr":"172.21.15.192:6813/1776840019","state":["exists","up"]},{"osd":6,"uuid":"6e507a91-ed09-4a33-bf0e-c4b593b3c5aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6819","nonce":1093341590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6821","nonce":1093341590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6825","nonce":1093341590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1093341590},{"type":"v1","addr":"172.21.15.117:6823","nonce":1093341590}]},"public_addr":"172.21.15.117:6819/1093341590","cluster_addr":"172.21.15.117:6821/1093341590","heartbeat_back_addr":"172.21.15.117:6825/1093341590","heartbeat_front_addr":"172.21.15.117:6823/1093341590","state":["exists","up"]},{"osd":7,"uuid":"36a47dd7-67f7-4ebe-8759-3a8bb6bd2ea3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6817","nonce":1448179714}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6819","nonce":1448179714}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6823","nonce":1448179714}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1448179714},{"type":"v1","addr":"172.21.15.138:6821","nonce":1448179714}]},"public_addr":"172.21.15.138:6817/1448179714","cluster_addr":"172.21.15.138:6819/1448179714","heartbeat_back_addr":"172.21.15.138:6823/1448179714","heartbeat_front_addr":"172.21.15.138:6821/1448179714","state":["exists","up"]},{"osd":8,"uuid":"118e0b5d-2619-4512-94c5-8d07a0d3a28f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6817","nonce":3960902885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6819","nonce":3960902885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6823","nonce":3960902885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":3960902885},{"type":"v1","addr":"172.21.15.192:6821","nonce":3960902885}]},"public_addr":"172.21.15.192:6817/3960902885","cluster_addr":"172.21.15.192:6819/3960902885","heartbeat_back_addr":"172.21.15.192:6823/3960902885","heartbeat_front_addr":"172.21.15.192:6821/3960902885","state":["exists","up"]},{"osd":9,"uuid":"47543bc6-fa0b-4e72-9742-97dcda0b7f8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6827","nonce":3453855195}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6829","nonce":3453855195}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6833","nonce":3453855195}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3453855195},{"type":"v1","addr":"172.21.15.117:6831","nonce":3453855195}]},"public_addr":"172.21.15.117:6827/3453855195","cluster_addr":"172.21.15.117:6829/3453855195","heartbeat_back_addr":"172.21.15.117:6833/3453855195","heartbeat_front_addr":"172.21.15.117:6831/3453855195","state":["exists","up"]},{"osd":10,"uuid":"5e6708b4-7a85-4fa5-a03c-27d99f7d749f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6825","nonce":903392451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6827","nonce":903392451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6831","nonce":903392451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":903392451},{"type":"v1","addr":"172.21.15.138:6829","nonce":903392451}]},"public_addr":"172.21.15.138:6825/903392451","cluster_addr":"172.21.15.138:6827/903392451","heartbeat_back_addr":"172.21.15.138:6831/903392451","heartbeat_front_addr":"172.21.15.138:6829/903392451","state":["exists","up"]},{"osd":11,"uuid":"24bc2ef7-e2b4-43c6-9e6c-43ae02390415","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6825","nonce":899090479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6827","nonce":899090479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6831","nonce":899090479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":899090479},{"type":"v1","addr":"172.21.15.192:6829","nonce":899090479}]},"public_addr":"172.21.15.192:6825/899090479","cluster_addr":"172.21.15.192:6827/899090479","heartbeat_back_addr":"172.21.15.192:6831/899090479","heartbeat_front_addr":"172.21.15.192:6829/899090479","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:49:40.742217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:05.963655+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:30.164495+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:50:56.598899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:51:21.940331+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:51:47.777445+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:52:12.869007+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:52:39.454054+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:03.569226+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:29.760863+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:53:55.788604+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:54:20.510184+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:0/2989394419":"2024-09-22T05:48:32.406654+0000","172.21.15.117:6801/1702676172":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/1766406059":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6800/2337036514":"2024-09-22T05:47:09.112435+0000","172.21.15.117:0/985798343":"2024-09-22T05:47:09.112435+0000","172.21.15.117:0/1083273222":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/473284565":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6801/2337036514":"2024-09-22T05:47:09.112435+0000","172.21.15.117:6800/816635165":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/2642940197":"2024-09-22T05:48:32.406654+0000","172.21.15.117:6801/816635165":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/2225537433":"2024-09-22T05:47:33.789146+0000","172.21.15.117:0/3651584169":"2024-09-22T05:47:33.789146+0000","172.21.15.117:6800/1702676172":"2024-09-22T05:48:32.406654+0000","172.21.15.117:0/1284403882":"2024-09-22T05:47:33.789146+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":18,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":7,"length":1},{"begin":18,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T06:09:15.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.415+0000 7f4bf6575640 1 -- 172.21.15.117:0/1514782788 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4bd80b6680 msgr2=0x7f4bd80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:15.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.415+0000 7f4bf6575640 1 --2- 172.21.15.117:0/1514782788 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4bd80b6680 0x7f4bd80b8b90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4be0031440 tx=0x7f4be0009680 comp rx=0 tx=0).stop 2024-09-21T06:09:15.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.415+0000 7f4bf6575640 1 -- 172.21.15.117:0/1514782788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bf01094a0 msgr2=0x7f4bf019aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:15.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.415+0000 7f4bf6575640 1 --2- 172.21.15.117:0/1514782788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bf01094a0 0x7f4bf019aa40 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f4be400d950 tx=0x7f4be400de10 comp rx=0 tx=0).stop 2024-09-21T06:09:15.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.415+0000 7f4bf6575640 1 -- 172.21.15.117:0/1514782788 shutdown_connections 2024-09-21T06:09:15.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.415+0000 7f4bf6575640 1 --2- 172.21.15.117:0/1514782788 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f4bd80b6680 0x7f4bd80b8b90 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:15.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.415+0000 7f4bf6575640 1 --2- 172.21.15.117:0/1514782788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bf01094a0 0x7f4bf019aa40 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:15.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.415+0000 7f4bf6575640 1 --2- 172.21.15.117:0/1514782788 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bf0109df0 0x7f4bf019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:15.417 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.415+0000 7f4bf6575640 1 --2- 172.21.15.117:0/1514782788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bf0103b60 0x7f4bf019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:15.418 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.415+0000 7f4bf6575640 1 -- 172.21.15.117:0/1514782788 >> 172.21.15.117:0/1514782788 conn(0x7f4bf007b850 msgr2=0x7f4bf0107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:15.418 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.415+0000 7f4bf6575640 1 -- 172.21.15.117:0/1514782788 shutdown_connections 2024-09-21T06:09:15.418 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:15.415+0000 7f4bf6575640 1 -- 172.21.15.117:0/1514782788 wait complete. 2024-09-21T06:09:16.157 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:16 smithi117 ceph-mon[30947]: pgmap v695: 97 pgs: 12 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 79 active+clean; 22 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-21T06:09:16.157 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:16 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/1514782788' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:09:16.190 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs fail cephfs 2024-09-21T06:09:16.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:16 smithi138 ceph-mon[34391]: pgmap v695: 97 pgs: 12 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 79 active+clean; 22 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-21T06:09:16.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:16 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/1514782788' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:09:16.439 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:09:16.479 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:16 smithi192 ceph-mon[34700]: pgmap v695: 97 pgs: 12 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 79 active+clean; 22 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-21T06:09:16.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:16 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/1514782788' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T06:09:16.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.924+0000 7f5ac6684640 1 -- 172.21.15.117:0/2661956009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ac01094c0 msgr2=0x7f5ac01098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:16.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.924+0000 7f5ac6684640 1 --2- 172.21.15.117:0/2661956009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ac01094c0 0x7f5ac01098d0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8009900 tx=0x7f5aa802a410 comp rx=0 tx=0).stop 2024-09-21T06:09:16.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.924+0000 7f5ac6684640 1 -- 172.21.15.117:0/2661956009 shutdown_connections 2024-09-21T06:09:16.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.924+0000 7f5ac6684640 1 --2- 172.21.15.117:0/2661956009 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ac01094c0 0x7f5ac01098d0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:16.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.924+0000 7f5ac6684640 1 --2- 172.21.15.117:0/2661956009 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ac0109e10 0x7f5ac0110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:16.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.924+0000 7f5ac6684640 1 --2- 172.21.15.117:0/2661956009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ac0103b80 0x7f5ac0103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:16.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.924+0000 7f5ac6684640 1 -- 172.21.15.117:0/2661956009 >> 172.21.15.117:0/2661956009 conn(0x7f5ac00ff610 msgr2=0x7f5ac0101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:16.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.924+0000 7f5ac6684640 1 -- 172.21.15.117:0/2661956009 shutdown_connections 2024-09-21T06:09:16.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.924+0000 7f5ac6684640 1 -- 172.21.15.117:0/2661956009 wait complete. 2024-09-21T06:09:16.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.925+0000 7f5ac6684640 1 Processor -- start 2024-09-21T06:09:16.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.925+0000 7f5ac6684640 1 -- start start 2024-09-21T06:09:16.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.925+0000 7f5ac6684640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ac0103b80 0x7f5ac0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:16.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.925+0000 7f5ac6684640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ac01094c0 0x7f5ac0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:16.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.925+0000 7f5ac6684640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ac0109e10 0x7f5ac0076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:16.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.925+0000 7f5ac6684640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac0113b10 con 0x7f5ac0103b80 2024-09-21T06:09:16.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.925+0000 7f5ac6684640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac0113990 con 0x7f5ac01094c0 2024-09-21T06:09:16.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.925+0000 7f5ac6684640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac0113c90 con 0x7f5ac0109e10 2024-09-21T06:09:16.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.925+0000 7f5abffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ac0103b80 0x7f5ac0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:16.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.925+0000 7f5abffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ac0103b80 0x7f5ac0075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48626/0 (socket says 172.21.15.117:48626) 2024-09-21T06:09:16.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.925+0000 7f5abffff640 1 -- 172.21.15.117:0/2873846644 learned_addr learned my addr 172.21.15.117:0/2873846644 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:09:16.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.926+0000 7f5ac4c21640 1 --2- 172.21.15.117:0/2873846644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ac0109e10 0x7f5ac0076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:16.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.926+0000 7f5abffff640 1 -- 172.21.15.117:0/2873846644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ac0109e10 msgr2=0x7f5ac0076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:16.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.926+0000 7f5ab7fff640 1 --2- 172.21.15.117:0/2873846644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ac01094c0 0x7f5ac0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:16.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.926+0000 7f5abffff640 1 --2- 172.21.15.117:0/2873846644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ac0109e10 0x7f5ac0076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:16.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.926+0000 7f5abffff640 1 -- 172.21.15.117:0/2873846644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ac01094c0 msgr2=0x7f5ac0075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:16.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.926+0000 7f5abffff640 1 --2- 172.21.15.117:0/2873846644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ac01094c0 0x7f5ac0075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:16.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.926+0000 7f5abffff640 1 -- 172.21.15.117:0/2873846644 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac01a8860 con 0x7f5ac0103b80 2024-09-21T06:09:16.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.926+0000 7f5ac4c21640 1 --2- 172.21.15.117:0/2873846644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ac0109e10 0x7f5ac0076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:09:16.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.926+0000 7f5abffff640 1 --2- 172.21.15.117:0/2873846644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ac0103b80 0x7f5ac0075710 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000b4a0 tx=0x7f5ab000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:16.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.926+0000 7f5abdffb640 1 -- 172.21.15.117:0/2873846644 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0014070 con 0x7f5ac0103b80 2024-09-21T06:09:16.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.926+0000 7f5ac6684640 1 -- 172.21.15.117:0/2873846644 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac01a8b50 con 0x7f5ac0103b80 2024-09-21T06:09:16.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.926+0000 7f5abdffb640 1 -- 172.21.15.117:0/2873846644 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ab0010ac0 con 0x7f5ac0103b80 2024-09-21T06:09:16.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.926+0000 7f5abdffb640 1 -- 172.21.15.117:0/2873846644 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0017c30 con 0x7f5ac0103b80 2024-09-21T06:09:16.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.926+0000 7f5ac6684640 1 -- 172.21.15.117:0/2873846644 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac01a90e0 con 0x7f5ac0103b80 2024-09-21T06:09:16.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.927+0000 7f5ac6684640 1 -- 172.21.15.117:0/2873846644 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ac0103f90 con 0x7f5ac0103b80 2024-09-21T06:09:16.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.929+0000 7f5abdffb640 1 -- 172.21.15.117:0/2873846644 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f5ab0017070 con 0x7f5ac0103b80 2024-09-21T06:09:16.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.930+0000 7f5abdffb640 1 --2- 172.21.15.117:0/2873846644 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5aac0b6700 0x7f5aac0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:16.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.930+0000 7f5ab7fff640 1 --2- 172.21.15.117:0/2873846644 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5aac0b6700 0x7f5aac0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:16.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.930+0000 7f5abdffb640 1 -- 172.21.15.117:0/2873846644 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8485+0+0 (secure 0 0 0) 0x7f5ab00e6d00 con 0x7f5ac0103b80 2024-09-21T06:09:16.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.930+0000 7f5ab7fff640 1 --2- 172.21.15.117:0/2873846644 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5aac0b6700 0x7f5aac0b8c10 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8006010 tx=0x7f5aa8000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:16.934 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:16.933+0000 7f5abdffb640 1 -- 172.21.15.117:0/2873846644 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5ab00ad270 con 0x7f5ac0103b80 2024-09-21T06:09:17.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:17.149+0000 7f5ac6684640 1 -- 172.21.15.117:0/2873846644 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5ac0076a10 con 0x7f5ac0103b80 2024-09-21T06:09:17.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:17.151+0000 7f5abdffb640 1 -- 172.21.15.117:0/2873846644 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) ==== 161+0+0 (secure 0 0 0) 0x7f5ac0076a10 con 0x7f5ac0103b80 2024-09-21T06:09:17.153 INFO:teuthology.orchestra.run.smithi117.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T06:09:17.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:17.154+0000 7f5ac6684640 1 -- 172.21.15.117:0/2873846644 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5aac0b6700 msgr2=0x7f5aac0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:17.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:17.154+0000 7f5ac6684640 1 --2- 172.21.15.117:0/2873846644 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5aac0b6700 0x7f5aac0b8c10 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8006010 tx=0x7f5aa8000bc0 comp rx=0 tx=0).stop 2024-09-21T06:09:17.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:17.154+0000 7f5ac6684640 1 -- 172.21.15.117:0/2873846644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ac0103b80 msgr2=0x7f5ac0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:17.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:17.155+0000 7f5ac6684640 1 --2- 172.21.15.117:0/2873846644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ac0103b80 0x7f5ac0075710 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000b4a0 tx=0x7f5ab000b960 comp rx=0 tx=0).stop 2024-09-21T06:09:17.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:17.155+0000 7f5ac6684640 1 -- 172.21.15.117:0/2873846644 shutdown_connections 2024-09-21T06:09:17.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:17.155+0000 7f5ac6684640 1 --2- 172.21.15.117:0/2873846644 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5aac0b6700 0x7f5aac0b8c10 secure :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8006010 tx=0x7f5aa8000bc0 comp rx=0 tx=0).stop 2024-09-21T06:09:17.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:17.155+0000 7f5ac6684640 1 --2- 172.21.15.117:0/2873846644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ac0109e10 0x7f5ac0076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:17.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:17.155+0000 7f5ac6684640 1 --2- 172.21.15.117:0/2873846644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ac01094c0 0x7f5ac0075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:17.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:17.155+0000 7f5ac6684640 1 --2- 172.21.15.117:0/2873846644 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ac0103b80 0x7f5ac0075710 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:17.157 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:17.155+0000 7f5ac6684640 1 -- 172.21.15.117:0/2873846644 >> 172.21.15.117:0/2873846644 conn(0x7f5ac00ff610 msgr2=0x7f5ac0101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:17.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:17.156+0000 7f5ac6684640 1 -- 172.21.15.117:0/2873846644 shutdown_connections 2024-09-21T06:09:17.158 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:17.156+0000 7f5ac6684640 1 -- 172.21.15.117:0/2873846644 wait complete. 2024-09-21T06:09:17.526 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:17 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mon-a[30943]: 2024-09-21T06:09:17.149+0000 7efe32c7b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T06:09:17.527 INFO:journalctl@ceph.mds.g.smithi117.stdout:Sep 21 06:09:17 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-g[72959]: ignoring --setuser ceph since I am not root 2024-09-21T06:09:17.527 INFO:journalctl@ceph.mds.g.smithi117.stdout:Sep 21 06:09:17 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-g[72959]: ignoring --setgroup ceph since I am not root 2024-09-21T06:09:17.527 INFO:journalctl@ceph.mds.g.smithi117.stdout:Sep 21 06:09:17 smithi117 ceph-mds[72963]: -- 172.21.15.117:0/1585781176 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fc0d87f860 con 0x55fc11a27a80 2024-09-21T06:09:17.527 INFO:journalctl@ceph.mds.g.smithi117.stdout:Sep 21 06:09:17 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-g[72959]: starting mds.g at 2024-09-21T06:09:17.527 INFO:journalctl@ceph.mds.a.smithi117.stdout:Sep 21 06:09:17 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-a[73344]: ignoring --setuser ceph since I am not root 2024-09-21T06:09:17.528 INFO:journalctl@ceph.mds.a.smithi117.stdout:Sep 21 06:09:17 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-a[73344]: ignoring --setgroup ceph since I am not root 2024-09-21T06:09:17.528 INFO:journalctl@ceph.mds.a.smithi117.stdout:Sep 21 06:09:17 smithi117 ceph-mds[73348]: -- 172.21.15.117:0/1907734222 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ee4b5af860 con 0x55ee4f757180 2024-09-21T06:09:17.528 INFO:journalctl@ceph.mds.a.smithi117.stdout:Sep 21 06:09:17 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-a[73344]: starting mds.a at 2024-09-21T06:09:17.615 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 21 06:09:17 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-e[63255]: ignoring --setuser ceph since I am not root 2024-09-21T06:09:17.615 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 21 06:09:17 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-e[63255]: ignoring --setgroup ceph since I am not root 2024-09-21T06:09:17.615 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 21 06:09:17 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-e[63255]: starting mds.e at 2024-09-21T06:09:17.616 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 21 06:09:17 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-b[63794]: ignoring --setuser ceph since I am not root 2024-09-21T06:09:17.616 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 21 06:09:17 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-b[63794]: ignoring --setgroup ceph since I am not root 2024-09-21T06:09:17.616 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 21 06:09:17 smithi138 ceph-mds[63798]: -- 172.21.15.138:0/3822749132 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5641cb79eea0 con 0x5641cf93da80 2024-09-21T06:09:17.616 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 21 06:09:17 smithi138 ceph-mds[63798]: -- 172.21.15.138:0/3822749132 <== mon.2 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5641cb795860 con 0x5641cf93da80 2024-09-21T06:09:17.616 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 21 06:09:17 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-b[63794]: starting mds.b at 2024-09-21T06:09:17.616 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 21 06:09:17 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-h[63527]: ignoring --setuser ceph since I am not root 2024-09-21T06:09:17.616 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 21 06:09:17 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-h[63527]: ignoring --setgroup ceph since I am not root 2024-09-21T06:09:17.616 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 21 06:09:17 smithi138 ceph-mds[63531]: -- 172.21.15.138:0/1939117383 <== mon.0 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561f8b025860 con 0x561f8f1cda80 2024-09-21T06:09:17.616 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 21 06:09:17 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-h[63527]: starting mds.h at 2024-09-21T06:09:17.617 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 21 06:09:17 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-k[62973]: ignoring --setuser ceph since I am not root 2024-09-21T06:09:17.617 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 21 06:09:17 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-k[62973]: ignoring --setgroup ceph since I am not root 2024-09-21T06:09:17.617 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 21 06:09:17 smithi138 ceph-mds[62977]: -- 172.21.15.138:0/2756065574 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x558a42d58b60 con 0x558a46ef7180 2024-09-21T06:09:17.617 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 21 06:09:17 smithi138 ceph-mds[62977]: --2- 172.21.15.138:0/2756065574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558a46ef7180 0x558a46edc000 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x558a46f08ba0 tx=0x558a46e2e540 comp rx=0 tx=0).write_message sending message m=0x558a46dbc540 seq=3 auth(proto 2 2 bytes epoch 0) 2024-09-21T06:09:17.617 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 21 06:09:17 smithi138 ceph-mds[62977]: -- 172.21.15.138:0/2756065574 <== mon.2 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558a42d4f860 con 0x558a46ef7180 2024-09-21T06:09:17.617 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 21 06:09:17 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-k[62973]: starting mds.k at 2024-09-21T06:09:17.730 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 21 06:09:17 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-c[64020]: ignoring --setuser ceph since I am not root 2024-09-21T06:09:17.730 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 21 06:09:17 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-c[64020]: ignoring --setgroup ceph since I am not root 2024-09-21T06:09:17.730 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 21 06:09:17 smithi192 ceph-mds[64024]: -- 172.21.15.192:0/2074611513 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560585e53860 con 0x560589ffb180 2024-09-21T06:09:17.730 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 21 06:09:17 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-c[64020]: starting mds.c at 2024-09-21T06:09:17.730 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 21 06:09:17 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-i[64285]: ignoring --setuser ceph since I am not root 2024-09-21T06:09:17.730 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 21 06:09:17 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-i[64285]: ignoring --setgroup ceph since I am not root 2024-09-21T06:09:17.730 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 21 06:09:17 smithi192 ceph-mds[64289]: -- 172.21.15.192:0/1693031899 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55658b1df860 con 0x55658f387600 2024-09-21T06:09:17.730 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 21 06:09:17 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-i[64285]: starting mds.i at 2024-09-21T06:09:17.731 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 21 06:09:17 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-f[63738]: ignoring --setuser ceph since I am not root 2024-09-21T06:09:17.731 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 21 06:09:17 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-f[63738]: ignoring --setgroup ceph since I am not root 2024-09-21T06:09:17.731 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 21 06:09:17 smithi192 ceph-mds[63750]: -- 172.21.15.192:0/2017852957 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c8d52bb860 con 0x55c8d9463600 2024-09-21T06:09:17.731 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 21 06:09:17 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-f[63738]: starting mds.f at 2024-09-21T06:09:17.731 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 21 06:09:17 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-l[63468]: ignoring --setuser ceph since I am not root 2024-09-21T06:09:17.731 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 21 06:09:17 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-l[63468]: ignoring --setgroup ceph since I am not root 2024-09-21T06:09:17.731 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 21 06:09:17 smithi192 ceph-mds[63472]: -- 172.21.15.192:0/341304730 <== mon.2 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55e7fadccea0 con 0x55e7fef6b180 2024-09-21T06:09:17.731 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 21 06:09:17 smithi192 ceph-mds[63472]: -- 172.21.15.192:0/341304730 <== mon.2 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e7fadc3860 con 0x55e7fef6b180 2024-09-21T06:09:17.731 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 21 06:09:17 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-l[63468]: starting mds.l at 2024-09-21T06:09:17.887 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T06:09:18.135 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:09:18.256 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:18 smithi117 ceph-mon[30947]: pgmap v696: 97 pgs: 97 active+clean; 20 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-09-21T06:09:18.256 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:18 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2873846644' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T06:09:18.256 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:18 smithi117 ceph-mon[30947]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T06:09:18.256 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:18 smithi117 ceph-mon[30947]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T06:09:18.256 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:18 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/2873846644' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T06:09:18.257 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:18 smithi117 ceph-mon[30947]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T06:09:18.257 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:18 smithi117 ceph-mon[30947]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-21T06:09:18.257 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:18 smithi117 ceph-mon[30947]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T06:09:18.365 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:18 smithi138 ceph-mon[34391]: pgmap v696: 97 pgs: 97 active+clean; 20 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-09-21T06:09:18.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:18 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2873846644' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T06:09:18.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:18 smithi138 ceph-mon[34391]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T06:09:18.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:18 smithi138 ceph-mon[34391]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T06:09:18.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:18 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/2873846644' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T06:09:18.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:18 smithi138 ceph-mon[34391]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T06:09:18.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:18 smithi138 ceph-mon[34391]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-21T06:09:18.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:18 smithi138 ceph-mon[34391]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T06:09:18.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:18 smithi192 ceph-mon[34700]: pgmap v696: 97 pgs: 97 active+clean; 20 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-09-21T06:09:18.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:18 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2873846644' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T06:09:18.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:18 smithi192 ceph-mon[34700]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T06:09:18.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:18 smithi192 ceph-mon[34700]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T06:09:18.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:18 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/2873846644' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T06:09:18.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:18 smithi192 ceph-mon[34700]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T06:09:18.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:18 smithi192 ceph-mon[34700]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-21T06:09:18.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:18 smithi192 ceph-mon[34700]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T06:09:18.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.726+0000 7f5beb317640 1 -- 172.21.15.117:0/1969399934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5be41094a0 msgr2=0x7f5be41098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:18.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.726+0000 7f5beb317640 1 --2- 172.21.15.117:0/1969399934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5be41094a0 0x7f5be41098b0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4009900 tx=0x7f5bd402a410 comp rx=0 tx=0).stop 2024-09-21T06:09:18.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.726+0000 7f5beb317640 1 -- 172.21.15.117:0/1969399934 shutdown_connections 2024-09-21T06:09:18.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.726+0000 7f5beb317640 1 --2- 172.21.15.117:0/1969399934 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5be41094a0 0x7f5be41098b0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:18.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.726+0000 7f5beb317640 1 --2- 172.21.15.117:0/1969399934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5be4109df0 0x7f5be4110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:18.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.726+0000 7f5beb317640 1 --2- 172.21.15.117:0/1969399934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5be4103b60 0x7f5be4103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:18.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.726+0000 7f5beb317640 1 -- 172.21.15.117:0/1969399934 >> 172.21.15.117:0/1969399934 conn(0x7f5be407b850 msgr2=0x7f5be407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:18.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.726+0000 7f5beb317640 1 -- 172.21.15.117:0/1969399934 shutdown_connections 2024-09-21T06:09:18.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.726+0000 7f5beb317640 1 -- 172.21.15.117:0/1969399934 wait complete. 2024-09-21T06:09:18.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.726+0000 7f5beb317640 1 Processor -- start 2024-09-21T06:09:18.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.727+0000 7f5beb317640 1 -- start start 2024-09-21T06:09:18.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.727+0000 7f5beb317640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5be4103b60 0x7f5be419a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:18.912 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.727+0000 7f5beb317640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5be41094a0 0x7f5be419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:18.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.727+0000 7f5beb317640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5be4109df0 0x7f5be419fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:18.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.727+0000 7f5beb317640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5be4113ab0 con 0x7f5be4103b60 2024-09-21T06:09:18.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.727+0000 7f5beb317640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5be4113930 con 0x7f5be4109df0 2024-09-21T06:09:18.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.727+0000 7f5beb317640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5be4113c30 con 0x7f5be41094a0 2024-09-21T06:09:18.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.727+0000 7f5be90b3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5be4103b60 0x7f5be419a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:18.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.727+0000 7f5be90b3640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5be4103b60 0x7f5be419a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48674/0 (socket says 172.21.15.117:48674) 2024-09-21T06:09:18.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.727+0000 7f5be90b3640 1 -- 172.21.15.117:0/177108707 learned_addr learned my addr 172.21.15.117:0/177108707 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:09:18.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.727+0000 7f5be98b4640 1 --2- 172.21.15.117:0/177108707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5be4109df0 0x7f5be419fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:18.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.727+0000 7f5be88b2640 1 --2- 172.21.15.117:0/177108707 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5be41094a0 0x7f5be419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:18.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.727+0000 7f5be90b3640 1 -- 172.21.15.117:0/177108707 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5be41094a0 msgr2=0x7f5be419ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:18.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.727+0000 7f5be90b3640 1 --2- 172.21.15.117:0/177108707 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5be41094a0 0x7f5be419ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:18.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.727+0000 7f5be90b3640 1 -- 172.21.15.117:0/177108707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5be4109df0 msgr2=0x7f5be419fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:18.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.727+0000 7f5be90b3640 1 --2- 172.21.15.117:0/177108707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5be4109df0 0x7f5be419fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:18.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.727+0000 7f5be90b3640 1 -- 172.21.15.117:0/177108707 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be41a04f0 con 0x7f5be4103b60 2024-09-21T06:09:18.913 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.727+0000 7f5be88b2640 1 --2- 172.21.15.117:0/177108707 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5be41094a0 0x7f5be419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T06:09:18.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.727+0000 7f5be90b3640 1 --2- 172.21.15.117:0/177108707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5be4103b60 0x7f5be419a580 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f5be000e9f0 tx=0x7f5be000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:18.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.728+0000 7f5bda7fc640 1 -- 172.21.15.117:0/177108707 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be000cd80 con 0x7f5be4103b60 2024-09-21T06:09:18.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.728+0000 7f5beb317640 1 -- 172.21.15.117:0/177108707 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be41a07e0 con 0x7f5be4103b60 2024-09-21T06:09:18.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.728+0000 7f5bda7fc640 1 -- 172.21.15.117:0/177108707 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5be00108e0 con 0x7f5be4103b60 2024-09-21T06:09:18.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.728+0000 7f5bda7fc640 1 -- 172.21.15.117:0/177108707 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be0018980 con 0x7f5be4103b60 2024-09-21T06:09:18.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.728+0000 7f5beb317640 1 -- 172.21.15.117:0/177108707 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be41a6f40 con 0x7f5be4103b60 2024-09-21T06:09:18.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.729+0000 7f5beb317640 1 -- 172.21.15.117:0/177108707 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5be4103f70 con 0x7f5be4103b60 2024-09-21T06:09:18.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.730+0000 7f5bda7fc640 1 -- 172.21.15.117:0/177108707 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7f5be0010430 con 0x7f5be4103b60 2024-09-21T06:09:18.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.731+0000 7f5bda7fc640 1 --2- 172.21.15.117:0/177108707 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5bd00b6680 0x7f5bd00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:18.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.732+0000 7f5be88b2640 1 --2- 172.21.15.117:0/177108707 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5bd00b6680 0x7f5bd00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:18.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.732+0000 7f5bda7fc640 1 -- 172.21.15.117:0/177108707 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8871+0+0 (secure 0 0 0) 0x7f5be00e8140 con 0x7f5be4103b60 2024-09-21T06:09:18.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.732+0000 7f5be88b2640 1 --2- 172.21.15.117:0/177108707 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5bd00b6680 0x7f5bd00b8b90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4000c00 tx=0x7f5bd4031400 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:18.914 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.735+0000 7f5bda7fc640 1 -- 172.21.15.117:0/177108707 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5be00ae4d0 con 0x7f5be4103b60 2024-09-21T06:09:18.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:18.948+0000 7f5beb317640 1 -- 172.21.15.117:0/177108707 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f5be402e640 con 0x7f5be4103b60 2024-09-21T06:09:19.161 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:19.159+0000 7f5bda7fc640 1 -- 172.21.15.117:0/177108707 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) ==== 104+0+0 (secure 0 0 0) 0x7f5be00b28b0 con 0x7f5be4103b60 2024-09-21T06:09:19.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:19.162+0000 7f5beb317640 1 -- 172.21.15.117:0/177108707 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5bd00b6680 msgr2=0x7f5bd00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:19.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:19.162+0000 7f5beb317640 1 --2- 172.21.15.117:0/177108707 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5bd00b6680 0x7f5bd00b8b90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4000c00 tx=0x7f5bd4031400 comp rx=0 tx=0).stop 2024-09-21T06:09:19.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:19.162+0000 7f5beb317640 1 -- 172.21.15.117:0/177108707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5be4103b60 msgr2=0x7f5be419a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:19.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:19.162+0000 7f5beb317640 1 --2- 172.21.15.117:0/177108707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5be4103b60 0x7f5be419a580 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f5be000e9f0 tx=0x7f5be000eeb0 comp rx=0 tx=0).stop 2024-09-21T06:09:19.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:19.162+0000 7f5beb317640 1 -- 172.21.15.117:0/177108707 shutdown_connections 2024-09-21T06:09:19.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:19.162+0000 7f5beb317640 1 --2- 172.21.15.117:0/177108707 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7f5bd00b6680 0x7f5bd00b8b90 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:19.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:19.162+0000 7f5beb317640 1 --2- 172.21.15.117:0/177108707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5be4109df0 0x7f5be419fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:19.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:19.162+0000 7f5beb317640 1 --2- 172.21.15.117:0/177108707 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5be41094a0 0x7f5be419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:19.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:19.162+0000 7f5beb317640 1 --2- 172.21.15.117:0/177108707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5be4103b60 0x7f5be419a580 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:19.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:19.162+0000 7f5beb317640 1 -- 172.21.15.117:0/177108707 >> 172.21.15.117:0/177108707 conn(0x7f5be407b850 msgr2=0x7f5be4105610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:19.164 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:19.162+0000 7f5beb317640 1 -- 172.21.15.117:0/177108707 shutdown_connections 2024-09-21T06:09:19.165 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:19.162+0000 7f5beb317640 1 -- 172.21.15.117:0/177108707 wait complete. 2024-09-21T06:09:19.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:19 smithi192 ceph-mon[34700]: mds.? [v2:172.21.15.192:6832/868715097,v1:172.21.15.192:6834/868715097] up:boot 2024-09-21T06:09:19.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:19 smithi192 ceph-mon[34700]: mds.? [v2:172.21.15.192:6833/3122377082,v1:172.21.15.192:6835/3122377082] up:boot 2024-09-21T06:09:19.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:19 smithi192 ceph-mon[34700]: mds.? [v2:172.21.15.192:6838/1480475538,v1:172.21.15.192:6839/1480475538] up:boot 2024-09-21T06:09:19.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:19 smithi192 ceph-mon[34700]: mds.? [v2:172.21.15.192:6836/2954530620,v1:172.21.15.192:6837/2954530620] up:boot 2024-09-21T06:09:19.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:19 smithi192 ceph-mon[34700]: mds.? [v2:172.21.15.138:6833/976314578,v1:172.21.15.138:6837/976314578] up:boot 2024-09-21T06:09:19.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:19 smithi192 ceph-mon[34700]: mds.? [v2:172.21.15.138:6835/1048787503,v1:172.21.15.138:6839/1048787503] up:boot 2024-09-21T06:09:19.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:19 smithi192 ceph-mon[34700]: mds.? [v2:172.21.15.138:6834/1597023327,v1:172.21.15.138:6838/1597023327] up:boot 2024-09-21T06:09:19.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:19 smithi192 ceph-mon[34700]: mds.? [v2:172.21.15.138:6832/2347824118,v1:172.21.15.138:6836/2347824118] up:boot 2024-09-21T06:09:19.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:19 smithi192 ceph-mon[34700]: mds.? [v2:172.21.15.117:6838/2433230918,v1:172.21.15.117:6840/2433230918] up:boot 2024-09-21T06:09:19.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:19 smithi192 ceph-mon[34700]: mds.? [v2:172.21.15.117:6839/2502124472,v1:172.21.15.117:6841/2502124472] up:boot 2024-09-21T06:09:19.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:19 smithi192 ceph-mon[34700]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-21T06:09:19.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:19 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T06:09:19.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:19 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T06:09:19.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:19 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T06:09:19.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:19 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T06:09:19.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:19 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T06:09:19.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:19 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T06:09:19.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:19 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T06:09:19.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:19 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T06:09:19.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:19 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T06:09:19.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:19 smithi192 ceph-mon[34700]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T06:09:19.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:19 smithi192 ceph-mon[34700]: pgmap v699: 97 pgs: 97 active+clean; 20 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-21T06:09:19.481 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:19 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/177108707' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:09:19.525 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:19 smithi117 ceph-mon[30947]: mds.? [v2:172.21.15.192:6832/868715097,v1:172.21.15.192:6834/868715097] up:boot 2024-09-21T06:09:19.525 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:19 smithi117 ceph-mon[30947]: mds.? [v2:172.21.15.192:6833/3122377082,v1:172.21.15.192:6835/3122377082] up:boot 2024-09-21T06:09:19.525 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:19 smithi117 ceph-mon[30947]: mds.? [v2:172.21.15.192:6838/1480475538,v1:172.21.15.192:6839/1480475538] up:boot 2024-09-21T06:09:19.525 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:19 smithi117 ceph-mon[30947]: mds.? [v2:172.21.15.192:6836/2954530620,v1:172.21.15.192:6837/2954530620] up:boot 2024-09-21T06:09:19.525 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:19 smithi117 ceph-mon[30947]: mds.? [v2:172.21.15.138:6833/976314578,v1:172.21.15.138:6837/976314578] up:boot 2024-09-21T06:09:19.525 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:19 smithi117 ceph-mon[30947]: mds.? [v2:172.21.15.138:6835/1048787503,v1:172.21.15.138:6839/1048787503] up:boot 2024-09-21T06:09:19.525 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:19 smithi117 ceph-mon[30947]: mds.? [v2:172.21.15.138:6834/1597023327,v1:172.21.15.138:6838/1597023327] up:boot 2024-09-21T06:09:19.525 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:19 smithi117 ceph-mon[30947]: mds.? [v2:172.21.15.138:6832/2347824118,v1:172.21.15.138:6836/2347824118] up:boot 2024-09-21T06:09:19.525 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:19 smithi117 ceph-mon[30947]: mds.? [v2:172.21.15.117:6838/2433230918,v1:172.21.15.117:6840/2433230918] up:boot 2024-09-21T06:09:19.525 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:19 smithi117 ceph-mon[30947]: mds.? [v2:172.21.15.117:6839/2502124472,v1:172.21.15.117:6841/2502124472] up:boot 2024-09-21T06:09:19.525 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:19 smithi117 ceph-mon[30947]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-21T06:09:19.525 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:19 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T06:09:19.526 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:19 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T06:09:19.526 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:19 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T06:09:19.526 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:19 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T06:09:19.526 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:19 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T06:09:19.526 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:19 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T06:09:19.526 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:19 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T06:09:19.526 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:19 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T06:09:19.526 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:19 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T06:09:19.526 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:19 smithi117 ceph-mon[30947]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T06:09:19.526 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:19 smithi117 ceph-mon[30947]: pgmap v699: 97 pgs: 97 active+clean; 20 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-21T06:09:19.526 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:19 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/177108707' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:09:19.604 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:19 smithi138 ceph-mon[34391]: mds.? [v2:172.21.15.192:6832/868715097,v1:172.21.15.192:6834/868715097] up:boot 2024-09-21T06:09:19.604 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:19 smithi138 ceph-mon[34391]: mds.? [v2:172.21.15.192:6833/3122377082,v1:172.21.15.192:6835/3122377082] up:boot 2024-09-21T06:09:19.605 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:19 smithi138 ceph-mon[34391]: mds.? [v2:172.21.15.192:6838/1480475538,v1:172.21.15.192:6839/1480475538] up:boot 2024-09-21T06:09:19.605 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:19 smithi138 ceph-mon[34391]: mds.? [v2:172.21.15.192:6836/2954530620,v1:172.21.15.192:6837/2954530620] up:boot 2024-09-21T06:09:19.605 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:19 smithi138 ceph-mon[34391]: mds.? [v2:172.21.15.138:6833/976314578,v1:172.21.15.138:6837/976314578] up:boot 2024-09-21T06:09:19.605 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:19 smithi138 ceph-mon[34391]: mds.? [v2:172.21.15.138:6835/1048787503,v1:172.21.15.138:6839/1048787503] up:boot 2024-09-21T06:09:19.605 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:19 smithi138 ceph-mon[34391]: mds.? [v2:172.21.15.138:6834/1597023327,v1:172.21.15.138:6838/1597023327] up:boot 2024-09-21T06:09:19.605 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:19 smithi138 ceph-mon[34391]: mds.? [v2:172.21.15.138:6832/2347824118,v1:172.21.15.138:6836/2347824118] up:boot 2024-09-21T06:09:19.605 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:19 smithi138 ceph-mon[34391]: mds.? [v2:172.21.15.117:6838/2433230918,v1:172.21.15.117:6840/2433230918] up:boot 2024-09-21T06:09:19.605 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:19 smithi138 ceph-mon[34391]: mds.? [v2:172.21.15.117:6839/2502124472,v1:172.21.15.117:6841/2502124472] up:boot 2024-09-21T06:09:19.605 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:19 smithi138 ceph-mon[34391]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-21T06:09:19.605 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:19 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T06:09:19.605 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:19 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T06:09:19.605 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:19 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T06:09:19.605 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:19 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T06:09:19.605 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:19 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T06:09:19.606 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:19 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T06:09:19.606 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:19 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T06:09:19.606 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:19 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T06:09:19.606 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:19 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T06:09:19.606 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:19 smithi138 ceph-mon[34391]: from='mgr.14223 172.21.15.117:0/3991087693' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T06:09:19.606 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:19 smithi138 ceph-mon[34391]: pgmap v699: 97 pgs: 97 active+clean; 20 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1023 B/s wr, 11 op/s 2024-09-21T06:09:19.606 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:19 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/177108707' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T06:09:20.218 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T06:09:20.471 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:09:20.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:20 smithi192 ceph-mon[34700]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T06:09:20.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:20 smithi192 ceph-mon[34700]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T06:09:20.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:20 smithi192 ceph-mon[34700]: Cluster is now healthy 2024-09-21T06:09:20.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:20 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/177108707' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T06:09:20.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:20 smithi192 ceph-mon[34700]: fsmap 12 up:standby 2024-09-21T06:09:20.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:20 smithi117 ceph-mon[30947]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T06:09:20.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:20 smithi117 ceph-mon[30947]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T06:09:20.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:20 smithi117 ceph-mon[30947]: Cluster is now healthy 2024-09-21T06:09:20.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:20 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/177108707' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T06:09:20.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:20 smithi117 ceph-mon[30947]: fsmap 12 up:standby 2024-09-21T06:09:20.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:20 smithi138 ceph-mon[34391]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T06:09:20.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:20 smithi138 ceph-mon[34391]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T06:09:20.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:20 smithi138 ceph-mon[34391]: Cluster is now healthy 2024-09-21T06:09:20.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:20 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/177108707' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T06:09:20.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:20 smithi138 ceph-mon[34391]: fsmap 12 up:standby 2024-09-21T06:09:20.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.990+0000 7fade1d1b640 1 -- 172.21.15.117:0/2443391189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faddc1062d0 msgr2=0x7faddc1066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:20.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.990+0000 7fade1d1b640 1 --2- 172.21.15.117:0/2443391189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faddc1062d0 0x7faddc1066e0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fadc4009900 tx=0x7fadc402a410 comp rx=0 tx=0).stop 2024-09-21T06:09:20.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.990+0000 7fade1d1b640 1 -- 172.21.15.117:0/2443391189 shutdown_connections 2024-09-21T06:09:20.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.990+0000 7fade1d1b640 1 --2- 172.21.15.117:0/2443391189 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faddc1062d0 0x7faddc1066e0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:20.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.990+0000 7fade1d1b640 1 --2- 172.21.15.117:0/2443391189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faddc106c20 0x7faddc110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:20.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.990+0000 7fade1d1b640 1 --2- 172.21.15.117:0/2443391189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faddc101ff0 0x7faddc102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:20.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.990+0000 7fade1d1b640 1 -- 172.21.15.117:0/2443391189 >> 172.21.15.117:0/2443391189 conn(0x7faddc079e30 msgr2=0x7faddc07a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:20.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.990+0000 7fade1d1b640 1 -- 172.21.15.117:0/2443391189 shutdown_connections 2024-09-21T06:09:20.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.990+0000 7fade1d1b640 1 -- 172.21.15.117:0/2443391189 wait complete. 2024-09-21T06:09:20.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.991+0000 7fade1d1b640 1 Processor -- start 2024-09-21T06:09:20.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.991+0000 7fade1d1b640 1 -- start start 2024-09-21T06:09:20.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.991+0000 7fade1d1b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faddc101ff0 0x7faddc19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:20.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.991+0000 7fade1d1b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faddc1062d0 0x7faddc19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:20.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.991+0000 7fade1d1b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faddc106c20 0x7faddc19fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:20.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.991+0000 7fade1d1b640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faddc113b00 con 0x7faddc1062d0 2024-09-21T06:09:20.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.991+0000 7fade1d1b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faddc113980 con 0x7faddc101ff0 2024-09-21T06:09:20.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.991+0000 7fade1d1b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7faddc113c80 con 0x7faddc106c20 2024-09-21T06:09:20.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.992+0000 7fadd3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faddc1062d0 0x7faddc19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:20.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.992+0000 7fadd3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faddc1062d0 0x7faddc19aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48688/0 (socket says 172.21.15.117:48688) 2024-09-21T06:09:20.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.992+0000 7fadd3fff640 1 -- 172.21.15.117:0/598655092 learned_addr learned my addr 172.21.15.117:0/598655092 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:09:20.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.992+0000 7faddb7fe640 1 --2- 172.21.15.117:0/598655092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faddc101ff0 0x7faddc19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:20.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.992+0000 7faddbfff640 1 --2- 172.21.15.117:0/598655092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faddc106c20 0x7faddc19fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:20.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.992+0000 7fadd3fff640 1 -- 172.21.15.117:0/598655092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faddc106c20 msgr2=0x7faddc19fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:20.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.992+0000 7fadd3fff640 1 --2- 172.21.15.117:0/598655092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faddc106c20 0x7faddc19fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:20.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.992+0000 7fadd3fff640 1 -- 172.21.15.117:0/598655092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faddc101ff0 msgr2=0x7faddc19a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:20.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.992+0000 7fadd3fff640 1 --2- 172.21.15.117:0/598655092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faddc101ff0 0x7faddc19a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:20.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.992+0000 7fadd3fff640 1 -- 172.21.15.117:0/598655092 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faddc1a03a0 con 0x7faddc1062d0 2024-09-21T06:09:20.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.992+0000 7fadd3fff640 1 --2- 172.21.15.117:0/598655092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faddc1062d0 0x7faddc19aa90 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fadc4031a90 tx=0x7fadc4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:20.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.992+0000 7fadd97fa640 1 -- 172.21.15.117:0/598655092 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc403f030 con 0x7faddc1062d0 2024-09-21T06:09:20.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.992+0000 7fadd97fa640 1 -- 172.21.15.117:0/598655092 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadc4004490 con 0x7faddc1062d0 2024-09-21T06:09:20.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.992+0000 7fade1d1b640 1 -- 172.21.15.117:0/598655092 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faddc1a0630 con 0x7faddc1062d0 2024-09-21T06:09:20.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.992+0000 7fadd97fa640 1 -- 172.21.15.117:0/598655092 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc4035070 con 0x7faddc1062d0 2024-09-21T06:09:20.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.992+0000 7fade1d1b640 1 -- 172.21.15.117:0/598655092 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faddc1a6eb0 con 0x7faddc1062d0 2024-09-21T06:09:21.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.993+0000 7fade1d1b640 1 -- 172.21.15.117:0/598655092 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faddc06b950 con 0x7faddc1062d0 2024-09-21T06:09:21.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.995+0000 7fadd97fa640 1 -- 172.21.15.117:0/598655092 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fadc4042030 con 0x7faddc1062d0 2024-09-21T06:09:21.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.996+0000 7fadd97fa640 1 --2- 172.21.15.117:0/598655092 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fadc80b66b0 0x7fadc80b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:21.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.996+0000 7faddb7fe640 1 --2- 172.21.15.117:0/598655092 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fadc80b66b0 0x7fadc80b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:21.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.996+0000 7fadd97fa640 1 -- 172.21.15.117:0/598655092 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8871+0+0 (secure 0 0 0) 0x7fadc4108240 con 0x7faddc1062d0 2024-09-21T06:09:21.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:20.997+0000 7faddb7fe640 1 --2- 172.21.15.117:0/598655092 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fadc80b66b0 0x7fadc80b8bc0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fadcc00abe0 tx=0x7fadcc00a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:21.001 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:21.000+0000 7fadd97fa640 1 -- 172.21.15.117:0/598655092 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fadc40ce630 con 0x7faddc1062d0 2024-09-21T06:09:21.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:21.183+0000 7fade1d1b640 1 -- 172.21.15.117:0/598655092 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7faddc10bc30 con 0x7faddc1062d0 2024-09-21T06:09:21.482 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:21 smithi192 ceph-mon[34700]: pgmap v700: 97 pgs: 97 active+clean; 20 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T06:09:21.526 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:21 smithi117 ceph-mon[30947]: pgmap v700: 97 pgs: 97 active+clean; 20 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T06:09:21.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:21 smithi138 ceph-mon[34391]: pgmap v700: 97 pgs: 97 active+clean; 20 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-21T06:09:22.179 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:22.177+0000 7fadd97fa640 1 -- 172.21.15.117:0/598655092 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v89) ==== 181+0+0 (secure 0 0 0) 0x7fadc40d2a10 con 0x7faddc1062d0 2024-09-21T06:09:22.179 INFO:teuthology.orchestra.run.smithi117.stderr:pool 'cephfs_metadata' removed 2024-09-21T06:09:22.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:22.181+0000 7fade1d1b640 1 -- 172.21.15.117:0/598655092 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fadc80b66b0 msgr2=0x7fadc80b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:22.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:22.181+0000 7fade1d1b640 1 --2- 172.21.15.117:0/598655092 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fadc80b66b0 0x7fadc80b8bc0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fadcc00abe0 tx=0x7fadcc00a2d0 comp rx=0 tx=0).stop 2024-09-21T06:09:22.183 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:22.182+0000 7fade1d1b640 1 -- 172.21.15.117:0/598655092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faddc1062d0 msgr2=0x7faddc19aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:22.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:22.182+0000 7fade1d1b640 1 --2- 172.21.15.117:0/598655092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faddc1062d0 0x7faddc19aa90 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fadc4031a90 tx=0x7fadc4004270 comp rx=0 tx=0).stop 2024-09-21T06:09:22.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:22.182+0000 7fade1d1b640 1 -- 172.21.15.117:0/598655092 shutdown_connections 2024-09-21T06:09:22.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:22.182+0000 7fade1d1b640 1 --2- 172.21.15.117:0/598655092 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fadc80b66b0 0x7fadc80b8bc0 secure :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fadcc00abe0 tx=0x7fadcc00a2d0 comp rx=0 tx=0).stop 2024-09-21T06:09:22.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:22.182+0000 7fade1d1b640 1 --2- 172.21.15.117:0/598655092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faddc1062d0 0x7faddc19aa90 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:22.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:22.182+0000 7fade1d1b640 1 --2- 172.21.15.117:0/598655092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faddc106c20 0x7faddc19fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:22.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:22.182+0000 7fade1d1b640 1 --2- 172.21.15.117:0/598655092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faddc101ff0 0x7faddc19a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:22.184 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:22.182+0000 7fade1d1b640 1 -- 172.21.15.117:0/598655092 >> 172.21.15.117:0/598655092 conn(0x7faddc079e30 msgr2=0x7faddc107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:22.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:22.182+0000 7fade1d1b640 1 -- 172.21.15.117:0/598655092 shutdown_connections 2024-09-21T06:09:22.185 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:22.182+0000 7fade1d1b640 1 -- 172.21.15.117:0/598655092 wait complete. 2024-09-21T06:09:22.444 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:22 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/598655092' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T06:09:22.479 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:22 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/598655092' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T06:09:22.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:22 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/598655092' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T06:09:22.884 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T06:09:23.137 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T06:09:23.447 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:23 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/598655092' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T06:09:23.447 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:23 smithi192 ceph-mon[34700]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T06:09:23.447 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:23 smithi192 ceph-mon[34700]: pgmap v702: 65 pgs: 65 active+clean; 20 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:23.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:23 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/598655092' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T06:09:23.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:23 smithi117 ceph-mon[30947]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T06:09:23.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:23 smithi117 ceph-mon[30947]: pgmap v702: 65 pgs: 65 active+clean; 20 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:23.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:23 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/598655092' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T06:09:23.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:23 smithi138 ceph-mon[34391]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T06:09:23.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:23 smithi138 ceph-mon[34391]: pgmap v702: 65 pgs: 65 active+clean; 20 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:23.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.676+0000 7fb816414640 1 -- 172.21.15.117:0/3974655643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb810103b60 msgr2=0x7fb810103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:23.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.676+0000 7fb816414640 1 --2- 172.21.15.117:0/3974655643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb810103b60 0x7fb810103f70 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fb800009900 tx=0x7fb80002a410 comp rx=0 tx=0).stop 2024-09-21T06:09:23.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.676+0000 7fb816414640 1 -- 172.21.15.117:0/3974655643 shutdown_connections 2024-09-21T06:09:23.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.676+0000 7fb816414640 1 --2- 172.21.15.117:0/3974655643 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb810109df0 0x7fb810110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:23.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.676+0000 7fb816414640 1 --2- 172.21.15.117:0/3974655643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8101094a0 0x7fb8101098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:23.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.676+0000 7fb816414640 1 --2- 172.21.15.117:0/3974655643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb810103b60 0x7fb810103f70 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:23.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.676+0000 7fb816414640 1 -- 172.21.15.117:0/3974655643 >> 172.21.15.117:0/3974655643 conn(0x7fb81007b850 msgr2=0x7fb81007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:23.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.676+0000 7fb816414640 1 -- 172.21.15.117:0/3974655643 shutdown_connections 2024-09-21T06:09:23.691 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.676+0000 7fb816414640 1 -- 172.21.15.117:0/3974655643 wait complete. 2024-09-21T06:09:23.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.677+0000 7fb816414640 1 Processor -- start 2024-09-21T06:09:23.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.677+0000 7fb816414640 1 -- start start 2024-09-21T06:09:23.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.678+0000 7fb816414640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb810103b60 0x7fb81019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:23.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.678+0000 7fb816414640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8101094a0 0x7fb81019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:23.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.678+0000 7fb816414640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb810109df0 0x7fb81019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:23.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.678+0000 7fb816414640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb810113ab0 con 0x7fb810103b60 2024-09-21T06:09:23.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.678+0000 7fb816414640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb810113930 con 0x7fb810109df0 2024-09-21T06:09:23.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.678+0000 7fb816414640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb810113c30 con 0x7fb8101094a0 2024-09-21T06:09:23.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.678+0000 7fb80ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb810103b60 0x7fb81019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:23.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.678+0000 7fb80ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb810103b60 0x7fb81019a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:48718/0 (socket says 172.21.15.117:48718) 2024-09-21T06:09:23.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.678+0000 7fb80ffff640 1 -- 172.21.15.117:0/3647456717 learned_addr learned my addr 172.21.15.117:0/3647456717 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-21T06:09:23.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.678+0000 7fb80f7fe640 1 --2- 172.21.15.117:0/3647456717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8101094a0 0x7fb81019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:23.692 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.678+0000 7fb8149b1640 1 --2- 172.21.15.117:0/3647456717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb810109df0 0x7fb81019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:23.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.678+0000 7fb80ffff640 1 -- 172.21.15.117:0/3647456717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8101094a0 msgr2=0x7fb81019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:23.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.678+0000 7fb80ffff640 1 --2- 172.21.15.117:0/3647456717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8101094a0 0x7fb81019aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:23.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.678+0000 7fb80ffff640 1 -- 172.21.15.117:0/3647456717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb810109df0 msgr2=0x7fb81019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:23.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.678+0000 7fb80ffff640 1 --2- 172.21.15.117:0/3647456717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb810109df0 0x7fb81019fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:23.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.678+0000 7fb80ffff640 1 -- 172.21.15.117:0/3647456717 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8101a0310 con 0x7fb810103b60 2024-09-21T06:09:23.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.678+0000 7fb80ffff640 1 --2- 172.21.15.117:0/3647456717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb810103b60 0x7fb81019a460 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fb800005940 tx=0x7fb8000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:23.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.678+0000 7fb80d7fa640 1 -- 172.21.15.117:0/3647456717 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80003f030 con 0x7fb810103b60 2024-09-21T06:09:23.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.678+0000 7fb80d7fa640 1 -- 172.21.15.117:0/3647456717 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb800003cb0 con 0x7fb810103b60 2024-09-21T06:09:23.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.678+0000 7fb816414640 1 -- 172.21.15.117:0/3647456717 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8101a05a0 con 0x7fb810103b60 2024-09-21T06:09:23.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.678+0000 7fb80d7fa640 1 -- 172.21.15.117:0/3647456717 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800035070 con 0x7fb810103b60 2024-09-21T06:09:23.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.679+0000 7fb816414640 1 -- 172.21.15.117:0/3647456717 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8101a0a80 con 0x7fb810103b60 2024-09-21T06:09:23.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.679+0000 7fb816414640 1 -- 172.21.15.117:0/3647456717 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb810103f70 con 0x7fb810103b60 2024-09-21T06:09:23.693 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.681+0000 7fb80d7fa640 1 -- 172.21.15.117:0/3647456717 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 24) ==== 154644+0+0 (secure 0 0 0) 0x7fb800042030 con 0x7fb810103b60 2024-09-21T06:09:23.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.682+0000 7fb80d7fa640 1 --2- 172.21.15.117:0/3647456717 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb7f80b66b0 0x7fb7f80b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T06:09:23.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.682+0000 7fb80f7fe640 1 --2- 172.21.15.117:0/3647456717 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb7f80b66b0 0x7fb7f80b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T06:09:23.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.682+0000 7fb80d7fa640 1 -- 172.21.15.117:0/3647456717 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8273+0+0 (secure 0 0 0) 0x7fb800107d40 con 0x7fb810103b60 2024-09-21T06:09:23.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.682+0000 7fb80f7fe640 1 --2- 172.21.15.117:0/3647456717 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb7f80b66b0 0x7fb7f80b8bc0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb8101064c0 tx=0x7fb80400a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T06:09:23.694 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.685+0000 7fb80d7fa640 1 -- 172.21.15.117:0/3647456717 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb8000ce380 con 0x7fb810103b60 2024-09-21T06:09:23.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:23.864+0000 7fb816414640 1 -- 172.21.15.117:0/3647456717 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb81010bc20 con 0x7fb810103b60 2024-09-21T06:09:24.196 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:24.194+0000 7fb80d7fa640 1 -- 172.21.15.117:0/3647456717 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v90) ==== 169+0+0 (secure 0 0 0) 0x7fb8000d2760 con 0x7fb810103b60 2024-09-21T06:09:24.196 INFO:teuthology.orchestra.run.smithi117.stderr:pool 'cephfs_data' removed 2024-09-21T06:09:24.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:24.197+0000 7fb816414640 1 -- 172.21.15.117:0/3647456717 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb7f80b66b0 msgr2=0x7fb7f80b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:24.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:24.197+0000 7fb816414640 1 --2- 172.21.15.117:0/3647456717 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb7f80b66b0 0x7fb7f80b8bc0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb8101064c0 tx=0x7fb80400a2d0 comp rx=0 tx=0).stop 2024-09-21T06:09:24.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:24.197+0000 7fb816414640 1 -- 172.21.15.117:0/3647456717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb810103b60 msgr2=0x7fb81019a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T06:09:24.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:24.197+0000 7fb816414640 1 --2- 172.21.15.117:0/3647456717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb810103b60 0x7fb81019a460 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fb800005940 tx=0x7fb8000042f0 comp rx=0 tx=0).stop 2024-09-21T06:09:24.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:24.197+0000 7fb816414640 1 -- 172.21.15.117:0/3647456717 shutdown_connections 2024-09-21T06:09:24.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:24.197+0000 7fb816414640 1 --2- 172.21.15.117:0/3647456717 >> [v2:172.21.15.117:6800/4224731851,v1:172.21.15.117:6801/4224731851] conn(0x7fb7f80b66b0 0x7fb7f80b8bc0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:24.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:24.197+0000 7fb816414640 1 --2- 172.21.15.117:0/3647456717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb810109df0 0x7fb81019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:24.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:24.197+0000 7fb816414640 1 --2- 172.21.15.117:0/3647456717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8101094a0 0x7fb81019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:24.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:24.197+0000 7fb816414640 1 --2- 172.21.15.117:0/3647456717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb810103b60 0x7fb81019a460 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T06:09:24.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:24.197+0000 7fb816414640 1 -- 172.21.15.117:0/3647456717 >> 172.21.15.117:0/3647456717 conn(0x7fb81007b850 msgr2=0x7fb810107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T06:09:24.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:24.197+0000 7fb816414640 1 -- 172.21.15.117:0/3647456717 shutdown_connections 2024-09-21T06:09:24.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-21T06:09:24.197+0000 7fb816414640 1 -- 172.21.15.117:0/3647456717 wait complete. 2024-09-21T06:09:24.479 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:24 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3647456717' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T06:09:24.526 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:24 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3647456717' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T06:09:24.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:24 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3647456717' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T06:09:25.479 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:25 smithi192 ceph-mon[34700]: from='client.? 172.21.15.117:0/3647456717' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T06:09:25.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:25 smithi192 ceph-mon[34700]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T06:09:25.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:25 smithi192 ceph-mon[34700]: pgmap v704: 1 pgs: 1 active+clean; 577 KiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:25.526 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:25 smithi117 ceph-mon[30947]: from='client.? 172.21.15.117:0/3647456717' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T06:09:25.526 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:25 smithi117 ceph-mon[30947]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T06:09:25.527 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:25 smithi117 ceph-mon[30947]: pgmap v704: 1 pgs: 1 active+clean; 577 KiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:25.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:25 smithi138 ceph-mon[34391]: from='client.? 172.21.15.117:0/3647456717' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T06:09:25.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:25 smithi138 ceph-mon[34391]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T06:09:25.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:25 smithi138 ceph-mon[34391]: pgmap v704: 1 pgs: 1 active+clean; 577 KiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:28.026 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:27 smithi117 ceph-mon[30947]: pgmap v705: 1 pgs: 1 active+clean; 577 KiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:28.229 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:27 smithi192 ceph-mon[34700]: pgmap v705: 1 pgs: 1 active+clean; 577 KiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:28.366 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:27 smithi138 ceph-mon[34391]: pgmap v705: 1 pgs: 1 active+clean; 577 KiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T06:09:29.117 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:09:29.159 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:09:29.198 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:09:29.232 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-21T06:09:29.232 DEBUG:teuthology.orchestra.run.smithi117:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T06:09:29.247 DEBUG:teuthology.orchestra.run.smithi138:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T06:09:29.262 DEBUG:teuthology.orchestra.run.smithi192:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T06:09:29.287 INFO:tasks.cephadm:Stopping all daemons... 2024-09-21T06:09:29.287 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-21T06:09:29.287 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mon.a 2024-09-21T06:09:29.526 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:29 smithi117 systemd[1]: Stopping Ceph mon.a for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T06:09:30.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:29 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mon-a[30943]: 2024-09-21T06:09:29.609+0000 7efe38486640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:09:30.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:29 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mon-a[30943]: 2024-09-21T06:09:29.609+0000 7efe38486640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-21T06:09:30.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:29 smithi117 podman[106162]: 2024-09-21 06:09:29.742439026 +0000 UTC m=+0.249937976 container died 9aed4354fc168f03a339a36fb1a7f3641121e277dc139edf5959fedb82e767ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mon-a, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-21T06:09:31.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:30 smithi117 podman[106162]: 2024-09-21 06:09:30.660018568 +0000 UTC m=+1.167517514 container cleanup 9aed4354fc168f03a339a36fb1a7f3641121e277dc139edf5959fedb82e767ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mon-a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:09:31.027 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:30 smithi117 bash[106162]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mon-a 2024-09-21T06:09:31.627 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mon.a.service' 2024-09-21T06:09:31.650 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:31 smithi117 podman[106174]: 2024-09-21 06:09:31.379038039 +0000 UTC m=+1.636297367 container remove 9aed4354fc168f03a339a36fb1a7f3641121e277dc139edf5959fedb82e767ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mon-a, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:09:31.650 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:31 smithi117 systemd[1]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mon.a.service: Deactivated successfully. 2024-09-21T06:09:31.650 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:31 smithi117 systemd[1]: Stopped Ceph mon.a for ac8581a2-77dc-11ef-baf5-efdc52797490. 2024-09-21T06:09:31.650 INFO:journalctl@ceph.mon.a.smithi117.stdout:Sep 21 06:09:31 smithi117 systemd[1]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mon.a.service: Consumed 24.080s CPU time. 2024-09-21T06:09:31.665 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:09:31.666 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-21T06:09:31.666 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-21T06:09:31.666 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mon.b 2024-09-21T06:09:32.073 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:31 smithi138 systemd[1]: Stopping Ceph mon.b for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T06:09:32.328 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:32 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mon-b[34387]: 2024-09-21T06:09:32.071+0000 7fa8f1c1a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:09:32.328 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:32 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mon-b[34387]: 2024-09-21T06:09:32.071+0000 7fa8f1c1a640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-21T06:09:32.615 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:32 smithi138 podman[67230]: 2024-09-21 06:09:32.327095739 +0000 UTC m=+0.386269763 container died 01a69c7125fc345f42649f069275f216a94e0c351ca6f1d88d49a21cb64e5568 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mon-b, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS) 2024-09-21T06:09:33.300 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:32 smithi138 podman[67230]: 2024-09-21 06:09:32.988860382 +0000 UTC m=+1.048034395 container cleanup 01a69c7125fc345f42649f069275f216a94e0c351ca6f1d88d49a21cb64e5568 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mon-b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:09:33.300 INFO:journalctl@ceph.mon.b.smithi138.stdout:Sep 21 06:09:32 smithi138 bash[67230]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mon-b 2024-09-21T06:09:33.418 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mon.b.service' 2024-09-21T06:09:33.467 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:09:33.468 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-21T06:09:33.468 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-21T06:09:33.468 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mon.c 2024-09-21T06:09:33.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:33 smithi192 systemd[1]: Stopping Ceph mon.c for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T06:09:34.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:34 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mon-c[34693]: 2024-09-21T06:09:34.668+0000 7f940e97e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:09:34.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:34 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mon-c[34693]: 2024-09-21T06:09:34.668+0000 7f940e97e640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-21T06:09:34.980 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:34 smithi192 podman[68107]: 2024-09-21 06:09:34.873217747 +0000 UTC m=+0.491438153 container died bbceae20b5a2ee144250d948aad5be431dade292a2a2e3e701e46f52faf05857 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mon-c, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-21T06:09:35.479 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:35 smithi192 podman[68107]: 2024-09-21 06:09:35.28610963 +0000 UTC m=+0.904330037 container cleanup bbceae20b5a2ee144250d948aad5be431dade292a2a2e3e701e46f52faf05857 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mon-c, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-09-21T06:09:35.480 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:35 smithi192 bash[68107]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mon-c 2024-09-21T06:09:35.767 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mon.c.service' 2024-09-21T06:09:35.790 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:35 smithi192 podman[68120]: 2024-09-21 06:09:35.529724211 +0000 UTC m=+0.656046120 container remove bbceae20b5a2ee144250d948aad5be431dade292a2a2e3e701e46f52faf05857 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mon-c, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:09:35.790 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:35 smithi192 systemd[1]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mon.c.service: Deactivated successfully. 2024-09-21T06:09:35.790 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:35 smithi192 systemd[1]: Stopped Ceph mon.c for ac8581a2-77dc-11ef-baf5-efdc52797490. 2024-09-21T06:09:35.790 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 21 06:09:35 smithi192 systemd[1]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mon.c.service: Consumed 14.510s CPU time. 2024-09-21T06:09:35.816 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:09:35.817 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-21T06:09:35.817 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-21T06:09:35.817 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mgr.x 2024-09-21T06:09:36.276 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 06:09:35 smithi117 systemd[1]: Stopping Ceph mgr.x for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T06:09:37.227 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 06:09:36 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: 2024-09-21T06:09:36.865+0000 7f9642c38640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:09:37.227 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 06:09:36 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x[31178]: 2024-09-21T06:09:36.865+0000 7f9642c38640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-21T06:09:37.227 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 06:09:36 smithi117 podman[106284]: 2024-09-21 06:09:36.907626493 +0000 UTC m=+0.898791210 container died 0d059746aa48766ad39d79dd18f483c5dd44197000efa4b8df9b492d90c7edde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T06:09:37.526 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 06:09:37 smithi117 podman[106284]: 2024-09-21 06:09:37.333034715 +0000 UTC m=+1.324199438 container cleanup 0d059746aa48766ad39d79dd18f483c5dd44197000efa4b8df9b492d90c7edde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:09:37.527 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 06:09:37 smithi117 bash[106284]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x 2024-09-21T06:09:37.819 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mgr.x.service' 2024-09-21T06:09:37.840 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 06:09:37 smithi117 podman[106297]: 2024-09-21 06:09:37.585213524 +0000 UTC m=+0.672854436 container remove 0d059746aa48766ad39d79dd18f483c5dd44197000efa4b8df9b492d90c7edde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:09:37.840 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 06:09:37 smithi117 systemd[1]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mgr.x.service: Deactivated successfully. 2024-09-21T06:09:37.840 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 06:09:37 smithi117 systemd[1]: Stopped Ceph mgr.x for ac8581a2-77dc-11ef-baf5-efdc52797490. 2024-09-21T06:09:37.841 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 21 06:09:37 smithi117 systemd[1]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mgr.x.service: Consumed 49.269s CPU time. 2024-09-21T06:09:37.854 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:09:37.854 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-21T06:09:37.854 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-21T06:09:37.854 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mgr.y 2024-09-21T06:09:38.199 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 21 06:09:37 smithi138 systemd[1]: Stopping Ceph mgr.y for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T06:09:38.496 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 21 06:09:38 smithi138 podman[67353]: 2024-09-21 06:09:38.198178074 +0000 UTC m=+0.150256624 container died d9b6fffec4d1f2aa2c51828f6c474881455d69d5c2df46bb6aa3aa87a3559a1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2) 2024-09-21T06:09:38.869 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 21 06:09:38 smithi138 podman[67353]: 2024-09-21 06:09:38.602614368 +0000 UTC m=+0.554692920 container cleanup d9b6fffec4d1f2aa2c51828f6c474881455d69d5c2df46bb6aa3aa87a3559a1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-y, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-21T06:09:38.870 INFO:journalctl@ceph.mgr.y.smithi138.stdout:Sep 21 06:09:38 smithi138 bash[67353]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-y 2024-09-21T06:09:39.029 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mgr.y.service' 2024-09-21T06:09:39.063 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:09:39.063 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-21T06:09:39.064 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-21T06:09:39.064 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mgr.z 2024-09-21T06:09:39.448 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 21 06:09:39 smithi192 systemd[1]: Stopping Ceph mgr.z for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T06:09:39.729 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 21 06:09:39 smithi192 podman[68231]: 2024-09-21 06:09:39.447161495 +0000 UTC m=+0.183572032 container died 544ea82b81683f60085048c2a8e533a883f8125ccd720eaab1924b1593225fa0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-z, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-21T06:09:40.194 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 21 06:09:39 smithi192 podman[68231]: 2024-09-21 06:09:39.958620386 +0000 UTC m=+0.695030924 container cleanup 544ea82b81683f60085048c2a8e533a883f8125ccd720eaab1924b1593225fa0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-21T06:09:40.195 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 21 06:09:39 smithi192 bash[68231]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-z 2024-09-21T06:09:40.438 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mgr.z.service' 2024-09-21T06:09:40.460 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 21 06:09:40 smithi192 podman[68243]: 2024-09-21 06:09:40.193615494 +0000 UTC m=+0.750393992 container remove 544ea82b81683f60085048c2a8e533a883f8125ccd720eaab1924b1593225fa0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mgr-z, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-21T06:09:40.460 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 21 06:09:40 smithi192 systemd[1]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-09-21T06:09:40.460 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 21 06:09:40 smithi192 systemd[1]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mgr.z.service: Failed with result 'exit-code'. 2024-09-21T06:09:40.460 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 21 06:09:40 smithi192 systemd[1]: Stopped Ceph mgr.z for ac8581a2-77dc-11ef-baf5-efdc52797490. 2024-09-21T06:09:40.460 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 21 06:09:40 smithi192 systemd[1]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mgr.z.service: Consumed 7.261s CPU time. 2024-09-21T06:09:40.473 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:09:40.473 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-21T06:09:40.474 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-21T06:09:40.474 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.0 2024-09-21T06:09:40.776 INFO:journalctl@ceph.osd.0.smithi117.stdout:Sep 21 06:09:40 smithi117 systemd[1]: Stopping Ceph osd.0 for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T06:09:41.277 INFO:journalctl@ceph.osd.0.smithi117.stdout:Sep 21 06:09:40 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-0[47501]: 2024-09-21T06:09:40.786+0000 7f2588af4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:09:41.277 INFO:journalctl@ceph.osd.0.smithi117.stdout:Sep 21 06:09:40 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-0[47501]: 2024-09-21T06:09:40.786+0000 7f2588af4640 -1 osd.0 90 *** Got signal Terminated *** 2024-09-21T06:09:41.277 INFO:journalctl@ceph.osd.0.smithi117.stdout:Sep 21 06:09:40 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-0[47501]: 2024-09-21T06:09:40.786+0000 7f2588af4640 -1 osd.0 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:09:46.673 INFO:journalctl@ceph.osd.0.smithi117.stdout:Sep 21 06:09:46 smithi117 podman[106409]: 2024-09-21 06:09:46.369878725 +0000 UTC m=+5.702782678 container died 246c25bc5dcc062b9248a96b5d1f031114a06501da08e62bdd070c416783fee0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-21T06:09:47.026 INFO:journalctl@ceph.osd.0.smithi117.stdout:Sep 21 06:09:46 smithi117 podman[106409]: 2024-09-21 06:09:46.871157104 +0000 UTC m=+6.204061089 container cleanup 246c25bc5dcc062b9248a96b5d1f031114a06501da08e62bdd070c416783fee0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:09:47.027 INFO:journalctl@ceph.osd.0.smithi117.stdout:Sep 21 06:09:46 smithi117 bash[106409]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-0 2024-09-21T06:09:47.489 INFO:journalctl@ceph.osd.0.smithi117.stdout:Sep 21 06:09:47 smithi117 podman[106424]: 2024-09-21 06:09:47.156518246 +0000 UTC m=+0.786916025 container remove 246c25bc5dcc062b9248a96b5d1f031114a06501da08e62bdd070c416783fee0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:09:47.776 INFO:journalctl@ceph.osd.0.smithi117.stdout:Sep 21 06:09:47 smithi117 podman[106511]: 2024-09-21 06:09:47.390456917 +0000 UTC m=+0.015492264 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:09:47.777 INFO:journalctl@ceph.osd.0.smithi117.stdout:Sep 21 06:09:47 smithi117 podman[106511]: 2024-09-21 06:09:47.535741204 +0000 UTC m=+0.160776575 container create 2de0842063e7cc426e903e05eac58783d2c78d796a09ea7a58122f27f4fbc86c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-0-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:09:48.164 INFO:journalctl@ceph.osd.0.smithi117.stdout:Sep 21 06:09:47 smithi117 podman[106511]: 2024-09-21 06:09:47.787519067 +0000 UTC m=+0.412554408 container init 2de0842063e7cc426e903e05eac58783d2c78d796a09ea7a58122f27f4fbc86c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-21T06:09:48.164 INFO:journalctl@ceph.osd.0.smithi117.stdout:Sep 21 06:09:47 smithi117 podman[106511]: 2024-09-21 06:09:47.790908948 +0000 UTC m=+0.415944279 container start 2de0842063e7cc426e903e05eac58783d2c78d796a09ea7a58122f27f4fbc86c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-21T06:09:48.164 INFO:journalctl@ceph.osd.0.smithi117.stdout:Sep 21 06:09:47 smithi117 podman[106511]: 2024-09-21 06:09:47.879339165 +0000 UTC m=+0.504374503 container attach 2de0842063e7cc426e903e05eac58783d2c78d796a09ea7a58122f27f4fbc86c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:09:48.526 INFO:journalctl@ceph.osd.0.smithi117.stdout:Sep 21 06:09:48 smithi117 podman[106511]: 2024-09-21 06:09:48.163041051 +0000 UTC m=+0.788076388 container died 2de0842063e7cc426e903e05eac58783d2c78d796a09ea7a58122f27f4fbc86c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-21T06:09:48.970 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.0.service' 2024-09-21T06:09:49.010 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:09:49.010 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-21T06:09:49.011 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-21T06:09:49.011 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.1 2024-09-21T06:09:49.317 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 21 06:09:49 smithi138 systemd[1]: Stopping Ceph osd.1 for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T06:09:49.585 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 21 06:09:49 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-1[39125]: 2024-09-21T06:09:49.315+0000 7f252e374640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:09:49.585 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 21 06:09:49 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-1[39125]: 2024-09-21T06:09:49.315+0000 7f252e374640 -1 osd.1 90 *** Got signal Terminated *** 2024-09-21T06:09:49.585 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 21 06:09:49 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-1[39125]: 2024-09-21T06:09:49.315+0000 7f252e374640 -1 osd.1 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:09:55.259 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 21 06:09:54 smithi138 podman[67478]: 2024-09-21 06:09:54.951171937 +0000 UTC m=+5.748727186 container died ecd97c2ee7bb88031ef34a277d727a0abe5ab99d2e4da3f0cf2c8f46f5f2d802 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-1, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-21T06:09:55.615 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 21 06:09:55 smithi138 podman[67478]: 2024-09-21 06:09:55.357709993 +0000 UTC m=+6.155265233 container cleanup ecd97c2ee7bb88031ef34a277d727a0abe5ab99d2e4da3f0cf2c8f46f5f2d802 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-1, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-21T06:09:55.615 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 21 06:09:55 smithi138 bash[67478]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-1 2024-09-21T06:09:55.968 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 21 06:09:55 smithi138 podman[67507]: 2024-09-21 06:09:55.629473909 +0000 UTC m=+0.678293866 container remove ecd97c2ee7bb88031ef34a277d727a0abe5ab99d2e4da3f0cf2c8f46f5f2d802 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-1, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:09:56.255 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 21 06:09:55 smithi138 podman[67596]: 2024-09-21 06:09:55.869907759 +0000 UTC m=+0.015780216 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:09:56.256 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 21 06:09:56 smithi138 podman[67596]: 2024-09-21 06:09:56.010355667 +0000 UTC m=+0.156228158 container create 6aa90143341a3e8aeb78d3c21499e390fe5b0679b97ec4f5cd010f00c0e0abf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-1-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-09-21T06:09:56.256 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 21 06:09:56 smithi138 podman[67596]: 2024-09-21 06:09:56.254717056 +0000 UTC m=+0.400589515 container init 6aa90143341a3e8aeb78d3c21499e390fe5b0679b97ec4f5cd010f00c0e0abf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-21T06:09:56.615 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 21 06:09:56 smithi138 podman[67596]: 2024-09-21 06:09:56.258142709 +0000 UTC m=+0.404015158 container start 6aa90143341a3e8aeb78d3c21499e390fe5b0679b97ec4f5cd010f00c0e0abf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-21T06:09:56.615 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 21 06:09:56 smithi138 podman[67596]: 2024-09-21 06:09:56.362962193 +0000 UTC m=+0.508834658 container attach 6aa90143341a3e8aeb78d3c21499e390fe5b0679b97ec4f5cd010f00c0e0abf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True) 2024-09-21T06:09:56.976 INFO:journalctl@ceph.osd.1.smithi138.stdout:Sep 21 06:09:56 smithi138 podman[67596]: 2024-09-21 06:09:56.630031485 +0000 UTC m=+0.775903936 container died 6aa90143341a3e8aeb78d3c21499e390fe5b0679b97ec4f5cd010f00c0e0abf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-1-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-21T06:09:57.317 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.1.service' 2024-09-21T06:09:57.359 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:09:57.359 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-21T06:09:57.359 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-21T06:09:57.359 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.2 2024-09-21T06:09:57.684 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 21 06:09:57 smithi192 systemd[1]: Stopping Ceph osd.2 for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T06:09:57.979 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 21 06:09:57 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-2[39872]: 2024-09-21T06:09:57.687+0000 7f2b6fcf4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:09:57.980 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 21 06:09:57 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-2[39872]: 2024-09-21T06:09:57.687+0000 7f2b6fcf4640 -1 osd.2 90 *** Got signal Terminated *** 2024-09-21T06:09:57.980 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 21 06:09:57 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-2[39872]: 2024-09-21T06:09:57.687+0000 7f2b6fcf4640 -1 osd.2 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:10:03.624 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 21 06:10:03 smithi192 podman[68356]: 2024-09-21 06:10:03.342625375 +0000 UTC m=+5.767098045 container died 1cbc4820ef8b0e6c671b9621698ed151bfe605d885168ab6526c0067344e8fd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-21T06:10:03.979 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 21 06:10:03 smithi192 podman[68356]: 2024-09-21 06:10:03.712766689 +0000 UTC m=+6.137239367 container cleanup 1cbc4820ef8b0e6c671b9621698ed151bfe605d885168ab6526c0067344e8fd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-21T06:10:03.980 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 21 06:10:03 smithi192 bash[68356]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-2 2024-09-21T06:10:04.339 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 21 06:10:03 smithi192 podman[68383]: 2024-09-21 06:10:03.989400362 +0000 UTC m=+0.648202244 container remove 1cbc4820ef8b0e6c671b9621698ed151bfe605d885168ab6526c0067344e8fd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-21T06:10:04.636 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 21 06:10:04 smithi192 podman[68473]: 2024-09-21 06:10:04.240862929 +0000 UTC m=+0.016128999 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:10:04.636 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 21 06:10:04 smithi192 podman[68473]: 2024-09-21 06:10:04.365455016 +0000 UTC m=+0.140721117 container create f98b7d8a08390cb3f0b29de7d969d42ea5cadaea645dc502685cee70706f3dba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-2-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:10:04.636 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 21 06:10:04 smithi192 podman[68473]: 2024-09-21 06:10:04.635002762 +0000 UTC m=+0.410268828 container init f98b7d8a08390cb3f0b29de7d969d42ea5cadaea645dc502685cee70706f3dba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:10:04.980 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 21 06:10:04 smithi192 podman[68473]: 2024-09-21 06:10:04.638390801 +0000 UTC m=+0.413656863 container start f98b7d8a08390cb3f0b29de7d969d42ea5cadaea645dc502685cee70706f3dba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-2-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-21T06:10:04.980 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 21 06:10:04 smithi192 podman[68473]: 2024-09-21 06:10:04.726714287 +0000 UTC m=+0.501980344 container attach f98b7d8a08390cb3f0b29de7d969d42ea5cadaea645dc502685cee70706f3dba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-21T06:10:04.980 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 21 06:10:04 smithi192 podman[68473]: 2024-09-21 06:10:04.862867939 +0000 UTC m=+0.638134008 container died f98b7d8a08390cb3f0b29de7d969d42ea5cadaea645dc502685cee70706f3dba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:10:05.549 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.2.service' 2024-09-21T06:10:05.590 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:10:05.591 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-21T06:10:05.591 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-21T06:10:05.591 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.3 2024-09-21T06:10:05.895 INFO:journalctl@ceph.osd.3.smithi117.stdout:Sep 21 06:10:05 smithi117 systemd[1]: Stopping Ceph osd.3 for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T06:10:06.158 INFO:journalctl@ceph.osd.3.smithi117.stdout:Sep 21 06:10:05 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-3[53652]: 2024-09-21T06:10:05.893+0000 7f1ae3e7a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:10:06.158 INFO:journalctl@ceph.osd.3.smithi117.stdout:Sep 21 06:10:05 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-3[53652]: 2024-09-21T06:10:05.893+0000 7f1ae3e7a640 -1 osd.3 90 *** Got signal Terminated *** 2024-09-21T06:10:06.158 INFO:journalctl@ceph.osd.3.smithi117.stdout:Sep 21 06:10:05 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-3[53652]: 2024-09-21T06:10:05.893+0000 7f1ae3e7a640 -1 osd.3 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:10:07.480 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:07 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:07.106+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:42.844894+0000 front 2024-09-21T06:09:42.844879+0000 (oldest deadline 2024-09-21T06:10:06.944750+0000) 2024-09-21T06:10:07.526 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:07 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:07.236+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:07.979 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:07 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:07.648+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:08.480 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:08 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:08.086+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:42.844894+0000 front 2024-09-21T06:09:42.844879+0000 (oldest deadline 2024-09-21T06:10:06.944750+0000) 2024-09-21T06:10:08.526 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:08 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:08.254+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:08.980 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:08 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:08.616+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:09.479 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:09 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:09.075+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:42.844894+0000 front 2024-09-21T06:09:42.844879+0000 (oldest deadline 2024-09-21T06:10:06.944750+0000) 2024-09-21T06:10:09.526 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:09 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:09.233+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:09.527 INFO:journalctl@ceph.osd.3.smithi117.stdout:Sep 21 06:10:09 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-3[53652]: 2024-09-21T06:10:09.061+0000 7f1adc45d640 -1 osd.3 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.670431+0000 front 2024-09-21T06:09:45.670457+0000 (oldest deadline 2024-09-21T06:10:08.470993+0000) 2024-09-21T06:10:09.980 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:09 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:09.586+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:10.482 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:10 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:10.063+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:42.844894+0000 front 2024-09-21T06:09:42.844879+0000 (oldest deadline 2024-09-21T06:10:06.944750+0000) 2024-09-21T06:10:10.527 INFO:journalctl@ceph.osd.3.smithi117.stdout:Sep 21 06:10:10 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-3[53652]: 2024-09-21T06:10:10.066+0000 7f1adc45d640 -1 osd.3 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.670431+0000 front 2024-09-21T06:09:45.670457+0000 (oldest deadline 2024-09-21T06:10:08.470993+0000) 2024-09-21T06:10:10.527 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:10 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:10.266+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:10.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:10 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:10.343+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:10.979 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:10 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:10.581+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:11.479 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:11 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:11.072+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:42.844894+0000 front 2024-09-21T06:09:42.844879+0000 (oldest deadline 2024-09-21T06:10:06.944750+0000) 2024-09-21T06:10:11.526 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:11 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:11.244+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:11.527 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:11 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:11.258+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.592972+0000 front 2024-09-21T06:09:45.593001+0000 (oldest deadline 2024-09-21T06:10:10.293110+0000) 2024-09-21T06:10:11.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:11 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:11.356+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:11.862 INFO:journalctl@ceph.osd.3.smithi117.stdout:Sep 21 06:10:11 smithi117 podman[106614]: 2024-09-21 06:10:11.570019713 +0000 UTC m=+5.785470063 container died 94749b8fdcb93f0040c0a1a0123bd806f0f6411a11d771019492421bba2133b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0) 2024-09-21T06:10:11.979 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:11 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:11.593+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:12.226 INFO:journalctl@ceph.osd.3.smithi117.stdout:Sep 21 06:10:12 smithi117 podman[106614]: 2024-09-21 06:10:12.068679514 +0000 UTC m=+6.284129855 container cleanup 94749b8fdcb93f0040c0a1a0123bd806f0f6411a11d771019492421bba2133b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-3, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-21T06:10:12.226 INFO:journalctl@ceph.osd.3.smithi117.stdout:Sep 21 06:10:12 smithi117 bash[106614]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-3 2024-09-21T06:10:12.480 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:12 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:12.062+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:42.844894+0000 front 2024-09-21T06:09:42.844879+0000 (oldest deadline 2024-09-21T06:10:06.944750+0000) 2024-09-21T06:10:12.526 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:12 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:12.296+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.592972+0000 front 2024-09-21T06:09:45.593001+0000 (oldest deadline 2024-09-21T06:10:10.293110+0000) 2024-09-21T06:10:12.527 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:12 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:12.221+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:12.527 INFO:journalctl@ceph.osd.3.smithi117.stdout:Sep 21 06:10:12 smithi117 podman[106644]: 2024-09-21 06:10:12.332293732 +0000 UTC m=+0.762843179 container remove 94749b8fdcb93f0040c0a1a0123bd806f0f6411a11d771019492421bba2133b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-3, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2) 2024-09-21T06:10:12.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:12 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:12.334+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:12.979 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:12 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:12.588+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:13.025 INFO:journalctl@ceph.osd.3.smithi117.stdout:Sep 21 06:10:12 smithi117 podman[106733]: 2024-09-21 06:10:12.633213318 +0000 UTC m=+0.015921002 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:10:13.025 INFO:journalctl@ceph.osd.3.smithi117.stdout:Sep 21 06:10:12 smithi117 podman[106733]: 2024-09-21 06:10:12.746680302 +0000 UTC m=+0.129388028 container create efb686e6a2716732536de599579ccc7eb68170df18e962fc3def549d75030c7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-21T06:10:13.277 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:13 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:13.265+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.592972+0000 front 2024-09-21T06:09:45.593001+0000 (oldest deadline 2024-09-21T06:10:10.293110+0000) 2024-09-21T06:10:13.277 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:13 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:13.226+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:13.277 INFO:journalctl@ceph.osd.3.smithi117.stdout:Sep 21 06:10:13 smithi117 podman[106733]: 2024-09-21 06:10:13.024561158 +0000 UTC m=+0.407268842 container init efb686e6a2716732536de599579ccc7eb68170df18e962fc3def549d75030c7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-21T06:10:13.278 INFO:journalctl@ceph.osd.3.smithi117.stdout:Sep 21 06:10:13 smithi117 podman[106733]: 2024-09-21 06:10:13.028385882 +0000 UTC m=+0.411093562 container start efb686e6a2716732536de599579ccc7eb68170df18e962fc3def549d75030c7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:10:13.278 INFO:journalctl@ceph.osd.3.smithi117.stdout:Sep 21 06:10:13 smithi117 podman[106733]: 2024-09-21 06:10:13.132749301 +0000 UTC m=+0.515457001 container attach efb686e6a2716732536de599579ccc7eb68170df18e962fc3def549d75030c7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-21T06:10:13.278 INFO:journalctl@ceph.osd.3.smithi117.stdout:Sep 21 06:10:13 smithi117 podman[106733]: 2024-09-21 06:10:13.224920979 +0000 UTC m=+0.607628659 container died efb686e6a2716732536de599579ccc7eb68170df18e962fc3def549d75030c7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-21T06:10:13.479 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:13 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:13.062+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:42.844894+0000 front 2024-09-21T06:09:42.844879+0000 (oldest deadline 2024-09-21T06:10:06.944750+0000) 2024-09-21T06:10:13.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:13 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:13.293+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:13.881 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.3.service' 2024-09-21T06:10:13.921 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:10:13.921 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-21T06:10:13.921 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-21T06:10:13.922 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.4 2024-09-21T06:10:13.979 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:13 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:13.633+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:14.246 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 21 06:10:13 smithi138 systemd[1]: Stopping Ceph osd.4 for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T06:10:14.479 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:14 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:14.033+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:42.844894+0000 front 2024-09-21T06:09:42.844879+0000 (oldest deadline 2024-09-21T06:10:06.944750+0000) 2024-09-21T06:10:14.527 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:14 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:14.292+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.592972+0000 front 2024-09-21T06:09:45.593001+0000 (oldest deadline 2024-09-21T06:10:10.293110+0000) 2024-09-21T06:10:14.527 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:14 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:14.221+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:14.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:14 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:14.266+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:14.615 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 21 06:10:14 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-4[45287]: 2024-09-21T06:10:14.244+0000 7fab33a53640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:10:14.615 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 21 06:10:14 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-4[45287]: 2024-09-21T06:10:14.244+0000 7fab33a53640 -1 osd.4 90 *** Got signal Terminated *** 2024-09-21T06:10:14.615 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 21 06:10:14 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-4[45287]: 2024-09-21T06:10:14.244+0000 7fab33a53640 -1 osd.4 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:10:14.979 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:14 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:14.662+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:15.480 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:14 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:14.998+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:42.844894+0000 front 2024-09-21T06:09:42.844879+0000 (oldest deadline 2024-09-21T06:10:06.944750+0000) 2024-09-21T06:10:15.526 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:15 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:15.323+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.592972+0000 front 2024-09-21T06:09:45.593001+0000 (oldest deadline 2024-09-21T06:10:10.293110+0000) 2024-09-21T06:10:15.527 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:15 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:15.252+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:15.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:15 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:15.309+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:15.966 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:15 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:15.651+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:16.229 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:15 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:15.964+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:42.844894+0000 front 2024-09-21T06:09:42.844879+0000 (oldest deadline 2024-09-21T06:10:06.944750+0000) 2024-09-21T06:10:16.230 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:15 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:15.964+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.946321+0000 front 2024-09-21T06:09:53.946446+0000 (oldest deadline 2024-09-21T06:10:15.646469+0000) 2024-09-21T06:10:16.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:16 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:16.292+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:16.776 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:16 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:16.343+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.592972+0000 front 2024-09-21T06:09:45.593001+0000 (oldest deadline 2024-09-21T06:10:10.293110+0000) 2024-09-21T06:10:16.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:16 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:16.343+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:52.594436+0000 front 2024-09-21T06:09:52.594420+0000 (oldest deadline 2024-09-21T06:10:16.094143+0000) 2024-09-21T06:10:16.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:16 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:16.285+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:16.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:16 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:16.285+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:16.975 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:16 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:16.647+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:17.229 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:16 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:16.973+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:42.844894+0000 front 2024-09-21T06:09:42.844879+0000 (oldest deadline 2024-09-21T06:10:06.944750+0000) 2024-09-21T06:10:17.230 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:16 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:16.973+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.946321+0000 front 2024-09-21T06:09:53.946446+0000 (oldest deadline 2024-09-21T06:10:15.646469+0000) 2024-09-21T06:10:17.365 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:17 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:17.243+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:17.365 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 21 06:10:16 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-4[45287]: 2024-09-21T06:10:16.998+0000 7fab3003e640 -1 osd.4 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:52.538919+0000 front 2024-09-21T06:09:52.538980+0000 (oldest deadline 2024-09-21T06:10:16.639049+0000) 2024-09-21T06:10:17.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:17 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:17.312+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.592972+0000 front 2024-09-21T06:09:45.593001+0000 (oldest deadline 2024-09-21T06:10:10.293110+0000) 2024-09-21T06:10:17.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:17 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:17.312+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:52.594436+0000 front 2024-09-21T06:09:52.594420+0000 (oldest deadline 2024-09-21T06:10:16.094143+0000) 2024-09-21T06:10:17.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:17 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:17.294+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:17.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:17 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:17.294+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:17.979 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:17 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:17.634+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:18.365 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:18 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:18.288+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:18.365 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:18 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:18.288+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:18.365 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 21 06:10:18 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-4[45287]: 2024-09-21T06:10:18.040+0000 7fab3003e640 -1 osd.4 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:52.538919+0000 front 2024-09-21T06:09:52.538980+0000 (oldest deadline 2024-09-21T06:10:16.639049+0000) 2024-09-21T06:10:18.480 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:17 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:17.998+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:42.844894+0000 front 2024-09-21T06:09:42.844879+0000 (oldest deadline 2024-09-21T06:10:06.944750+0000) 2024-09-21T06:10:18.480 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:17 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:17.998+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.946321+0000 front 2024-09-21T06:09:53.946446+0000 (oldest deadline 2024-09-21T06:10:15.646469+0000) 2024-09-21T06:10:18.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:18 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:18.085+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.905829+0000 front 2024-09-21T06:09:53.905878+0000 (oldest deadline 2024-09-21T06:10:18.005850+0000) 2024-09-21T06:10:18.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:18 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:18.302+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.592972+0000 front 2024-09-21T06:09:45.593001+0000 (oldest deadline 2024-09-21T06:10:10.293110+0000) 2024-09-21T06:10:18.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:18 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:18.302+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:52.594436+0000 front 2024-09-21T06:09:52.594420+0000 (oldest deadline 2024-09-21T06:10:16.094143+0000) 2024-09-21T06:10:18.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:18 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:18.285+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:18.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:18 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:18.285+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:18.980 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:18 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:18.645+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:19.365 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 21 06:10:19 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-4[45287]: 2024-09-21T06:10:19.059+0000 7fab3003e640 -1 osd.4 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:52.538919+0000 front 2024-09-21T06:09:52.538980+0000 (oldest deadline 2024-09-21T06:10:16.639049+0000) 2024-09-21T06:10:19.365 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:19 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:19.297+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:19.365 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:19 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:19.297+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:19.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:19 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:19.092+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.905829+0000 front 2024-09-21T06:09:53.905878+0000 (oldest deadline 2024-09-21T06:10:18.005850+0000) 2024-09-21T06:10:19.480 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:19 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:19.031+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:42.844894+0000 front 2024-09-21T06:09:42.844879+0000 (oldest deadline 2024-09-21T06:10:06.944750+0000) 2024-09-21T06:10:19.480 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:19 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:19.031+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.946321+0000 front 2024-09-21T06:09:53.946446+0000 (oldest deadline 2024-09-21T06:10:15.646469+0000) 2024-09-21T06:10:19.527 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:19 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:19.293+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.592972+0000 front 2024-09-21T06:09:45.593001+0000 (oldest deadline 2024-09-21T06:10:10.293110+0000) 2024-09-21T06:10:19.527 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:19 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:19.293+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:52.594436+0000 front 2024-09-21T06:09:52.594420+0000 (oldest deadline 2024-09-21T06:10:16.094143+0000) 2024-09-21T06:10:19.527 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:19 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:19.267+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:19.527 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:19 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:19.267+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:19.865 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 21 06:10:19 smithi138 podman[67700]: 2024-09-21 06:10:19.704113326 +0000 UTC m=+5.581971982 container died ed15ef1f9298d2d88efcbdee98ff15b950137ae037da7987c6ec55f31cc91248 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-21T06:10:19.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:19 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:19.470+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:50.263081+0000 front 2024-09-21T06:09:50.263151+0000 (oldest deadline 2024-09-21T06:10:18.463704+0000) 2024-09-21T06:10:19.979 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:19 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:19.675+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:20.273 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 21 06:10:20 smithi138 podman[67700]: 2024-09-21 06:10:20.086399617 +0000 UTC m=+5.964258278 container cleanup ed15ef1f9298d2d88efcbdee98ff15b950137ae037da7987c6ec55f31cc91248 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2) 2024-09-21T06:10:20.274 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 21 06:10:20 smithi138 bash[67700]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-4 2024-09-21T06:10:20.479 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:20 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:20.106+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.905829+0000 front 2024-09-21T06:09:53.905878+0000 (oldest deadline 2024-09-21T06:10:18.005850+0000) 2024-09-21T06:10:20.480 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:19 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:19.989+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:42.844894+0000 front 2024-09-21T06:09:42.844879+0000 (oldest deadline 2024-09-21T06:10:06.944750+0000) 2024-09-21T06:10:20.480 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:19 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:19.989+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.946321+0000 front 2024-09-21T06:09:53.946446+0000 (oldest deadline 2024-09-21T06:10:15.646469+0000) 2024-09-21T06:10:20.526 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:20 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:20.318+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.592972+0000 front 2024-09-21T06:09:45.593001+0000 (oldest deadline 2024-09-21T06:10:10.293110+0000) 2024-09-21T06:10:20.527 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:20 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:20.318+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:52.594436+0000 front 2024-09-21T06:09:52.594420+0000 (oldest deadline 2024-09-21T06:10:16.094143+0000) 2024-09-21T06:10:20.527 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:20 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:20.266+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:20.527 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:20 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:20.266+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:20.615 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 21 06:10:20 smithi138 podman[67717]: 2024-09-21 06:10:20.379806023 +0000 UTC m=+0.677311819 container remove ed15ef1f9298d2d88efcbdee98ff15b950137ae037da7987c6ec55f31cc91248 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-4, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-21T06:10:20.616 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:20 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:20.469+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:50.263081+0000 front 2024-09-21T06:09:50.263151+0000 (oldest deadline 2024-09-21T06:10:18.463704+0000) 2024-09-21T06:10:20.616 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:20 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:20.271+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:20.616 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:20 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:20.271+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:20.976 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 21 06:10:20 smithi138 podman[67805]: 2024-09-21 06:10:20.589926889 +0000 UTC m=+0.016109121 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:10:20.976 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 21 06:10:20 smithi138 podman[67805]: 2024-09-21 06:10:20.713996538 +0000 UTC m=+0.140178796 container create 75d629d4f836b2f376f1275b624143e61beb02c1a418cc7226f281181bf877a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-4-deactivate, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-21T06:10:20.979 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:20 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:20.671+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:20.979 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:20 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:20.671+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:21.303 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 21 06:10:20 smithi138 podman[67805]: 2024-09-21 06:10:20.974714694 +0000 UTC m=+0.400896910 container init 75d629d4f836b2f376f1275b624143e61beb02c1a418cc7226f281181bf877a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-4-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-09-21T06:10:21.303 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 21 06:10:20 smithi138 podman[67805]: 2024-09-21 06:10:20.978116175 +0000 UTC m=+0.404298392 container start 75d629d4f836b2f376f1275b624143e61beb02c1a418cc7226f281181bf877a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:10:21.303 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 21 06:10:21 smithi138 podman[67805]: 2024-09-21 06:10:21.074762702 +0000 UTC m=+0.500944931 container attach 75d629d4f836b2f376f1275b624143e61beb02c1a418cc7226f281181bf877a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-4-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True) 2024-09-21T06:10:21.303 INFO:journalctl@ceph.osd.4.smithi138.stdout:Sep 21 06:10:21 smithi138 podman[67805]: 2024-09-21 06:10:21.175929982 +0000 UTC m=+0.602112199 container died 75d629d4f836b2f376f1275b624143e61beb02c1a418cc7226f281181bf877a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:10:21.479 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:21 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:21.141+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.905829+0000 front 2024-09-21T06:09:53.905878+0000 (oldest deadline 2024-09-21T06:10:18.005850+0000) 2024-09-21T06:10:21.480 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:20 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:20.978+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:42.844894+0000 front 2024-09-21T06:09:42.844879+0000 (oldest deadline 2024-09-21T06:10:06.944750+0000) 2024-09-21T06:10:21.480 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:20 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:20.978+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.946321+0000 front 2024-09-21T06:09:53.946446+0000 (oldest deadline 2024-09-21T06:10:15.646469+0000) 2024-09-21T06:10:21.615 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:21 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:21.452+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:50.263081+0000 front 2024-09-21T06:09:50.263151+0000 (oldest deadline 2024-09-21T06:10:18.463704+0000) 2024-09-21T06:10:21.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:21 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:21.300+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:21.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:21 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:21.300+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:21.776 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:21 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:21.325+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.592972+0000 front 2024-09-21T06:09:45.593001+0000 (oldest deadline 2024-09-21T06:10:10.293110+0000) 2024-09-21T06:10:21.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:21 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:21.325+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:52.594436+0000 front 2024-09-21T06:09:52.594420+0000 (oldest deadline 2024-09-21T06:10:16.094143+0000) 2024-09-21T06:10:21.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:21 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:21.304+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:21.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:21 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:21.304+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:21.841 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.4.service' 2024-09-21T06:10:21.880 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:10:21.881 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-21T06:10:21.881 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-21T06:10:21.882 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.5 2024-09-21T06:10:21.979 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:21 smithi192 systemd[1]: Stopping Ceph osd.5 for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T06:10:21.980 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:21 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:21.685+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:21.980 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:21 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:21.685+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:22.394 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:22 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:22.119+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.905829+0000 front 2024-09-21T06:09:53.905878+0000 (oldest deadline 2024-09-21T06:10:18.005850+0000) 2024-09-21T06:10:22.394 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:21 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:21.984+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:42.844894+0000 front 2024-09-21T06:09:42.844879+0000 (oldest deadline 2024-09-21T06:10:06.944750+0000) 2024-09-21T06:10:22.394 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:21 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:21.984+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.946321+0000 front 2024-09-21T06:09:53.946446+0000 (oldest deadline 2024-09-21T06:10:15.646469+0000) 2024-09-21T06:10:22.394 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:22 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:22.392+0000 7f3d750ba640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:10:22.615 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:22 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:22.466+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:50.263081+0000 front 2024-09-21T06:09:50.263151+0000 (oldest deadline 2024-09-21T06:10:18.463704+0000) 2024-09-21T06:10:22.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:22 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:22.350+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:22.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:22 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:22.350+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:22.692 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:22 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:22.392+0000 7f3d750ba640 -1 osd.5 90 *** Got signal Terminated *** 2024-09-21T06:10:22.692 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:22 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:22.392+0000 7f3d750ba640 -1 osd.5 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:10:22.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:22 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:22.341+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.592972+0000 front 2024-09-21T06:09:45.593001+0000 (oldest deadline 2024-09-21T06:10:10.293110+0000) 2024-09-21T06:10:22.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:22 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:22.341+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:52.594436+0000 front 2024-09-21T06:09:52.594420+0000 (oldest deadline 2024-09-21T06:10:16.094143+0000) 2024-09-21T06:10:22.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:22 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:22.353+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:22.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:22 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:22.353+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:22.980 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:22 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:22.690+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:22.980 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:22 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:22.690+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:23.480 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:22 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:22.998+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:42.844894+0000 front 2024-09-21T06:09:42.844879+0000 (oldest deadline 2024-09-21T06:10:06.944750+0000) 2024-09-21T06:10:23.480 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:22 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:22.998+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.946321+0000 front 2024-09-21T06:09:53.946446+0000 (oldest deadline 2024-09-21T06:10:15.646469+0000) 2024-09-21T06:10:23.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:23 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:23.138+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.905829+0000 front 2024-09-21T06:09:53.905878+0000 (oldest deadline 2024-09-21T06:10:18.005850+0000) 2024-09-21T06:10:23.615 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:23 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:23.430+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:50.263081+0000 front 2024-09-21T06:09:50.263151+0000 (oldest deadline 2024-09-21T06:10:18.463704+0000) 2024-09-21T06:10:23.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:23 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:23.333+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:23.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:23 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:23.333+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:23.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:23 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:23.347+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.592972+0000 front 2024-09-21T06:09:45.593001+0000 (oldest deadline 2024-09-21T06:10:10.293110+0000) 2024-09-21T06:10:23.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:23 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:23.347+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:52.594436+0000 front 2024-09-21T06:09:52.594420+0000 (oldest deadline 2024-09-21T06:10:16.094143+0000) 2024-09-21T06:10:23.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:23 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:23.344+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:23.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:23 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:23.344+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:23.951 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:23 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:23.665+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:23.951 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:23 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:23.665+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:24.229 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:24 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:24.140+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.905829+0000 front 2024-09-21T06:09:53.905878+0000 (oldest deadline 2024-09-21T06:10:18.005850+0000) 2024-09-21T06:10:24.230 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:23 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:23.950+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:42.844894+0000 front 2024-09-21T06:09:42.844879+0000 (oldest deadline 2024-09-21T06:10:06.944750+0000) 2024-09-21T06:10:24.230 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:23 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:23.950+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.946321+0000 front 2024-09-21T06:09:53.946446+0000 (oldest deadline 2024-09-21T06:10:15.646469+0000) 2024-09-21T06:10:24.615 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:24 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:24.417+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:50.263081+0000 front 2024-09-21T06:09:50.263151+0000 (oldest deadline 2024-09-21T06:10:18.463704+0000) 2024-09-21T06:10:24.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:24 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:24.292+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:24.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:24 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:24.292+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:24.616 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:24 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:24.292+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:24.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:24 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:24.344+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:24.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:24 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:24.344+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:24.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:24 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:24.316+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.592972+0000 front 2024-09-21T06:09:45.593001+0000 (oldest deadline 2024-09-21T06:10:10.293110+0000) 2024-09-21T06:10:24.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:24 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:24.316+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:52.594436+0000 front 2024-09-21T06:09:52.594420+0000 (oldest deadline 2024-09-21T06:10:16.094143+0000) 2024-09-21T06:10:24.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:24 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:24.316+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.895863+0000 front 2024-09-21T06:10:01.895876+0000 (oldest deadline 2024-09-21T06:10:24.195799+0000) 2024-09-21T06:10:24.980 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:24 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:24.657+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:24.980 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:24 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:24.657+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:24.980 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:24 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:24.657+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:10:24.980 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:24 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:24.901+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:42.844894+0000 front 2024-09-21T06:09:42.844879+0000 (oldest deadline 2024-09-21T06:10:06.944750+0000) 2024-09-21T06:10:24.980 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:24 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:24.901+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.946321+0000 front 2024-09-21T06:09:53.946446+0000 (oldest deadline 2024-09-21T06:10:15.646469+0000) 2024-09-21T06:10:24.980 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:24 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:24.901+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:58.547237+0000 front 2024-09-21T06:09:58.547279+0000 (oldest deadline 2024-09-21T06:10:24.447362+0000) 2024-09-21T06:10:25.479 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:25 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:25.180+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.905829+0000 front 2024-09-21T06:09:53.905878+0000 (oldest deadline 2024-09-21T06:10:18.005850+0000) 2024-09-21T06:10:25.615 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:25 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:25.422+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:50.263081+0000 front 2024-09-21T06:09:50.263151+0000 (oldest deadline 2024-09-21T06:10:18.463704+0000) 2024-09-21T06:10:25.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:25 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:25.313+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:25.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:25 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:25.313+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:25.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:25 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:25.313+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:25.776 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:25 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:25.296+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:25.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:25 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:25.296+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:25.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:25 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:25.287+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.592972+0000 front 2024-09-21T06:09:45.593001+0000 (oldest deadline 2024-09-21T06:10:10.293110+0000) 2024-09-21T06:10:25.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:25 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:25.287+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:52.594436+0000 front 2024-09-21T06:09:52.594420+0000 (oldest deadline 2024-09-21T06:10:16.094143+0000) 2024-09-21T06:10:25.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:25 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:25.287+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.895863+0000 front 2024-09-21T06:10:01.895876+0000 (oldest deadline 2024-09-21T06:10:24.195799+0000) 2024-09-21T06:10:25.979 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:25 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:25.702+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:25.979 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:25 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:25.702+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:25.980 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:25 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:25.702+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:10:25.980 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:25 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:25.901+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:42.844894+0000 front 2024-09-21T06:09:42.844879+0000 (oldest deadline 2024-09-21T06:10:06.944750+0000) 2024-09-21T06:10:25.980 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:25 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:25.901+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.946321+0000 front 2024-09-21T06:09:53.946446+0000 (oldest deadline 2024-09-21T06:10:15.646469+0000) 2024-09-21T06:10:25.980 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:25 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:25.901+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:58.547237+0000 front 2024-09-21T06:09:58.547279+0000 (oldest deadline 2024-09-21T06:10:24.447362+0000) 2024-09-21T06:10:26.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:26 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:26.211+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.905829+0000 front 2024-09-21T06:09:53.905878+0000 (oldest deadline 2024-09-21T06:10:18.005850+0000) 2024-09-21T06:10:26.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:26 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:26.211+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.407097+0000 front 2024-09-21T06:10:01.407010+0000 (oldest deadline 2024-09-21T06:10:25.507175+0000) 2024-09-21T06:10:26.615 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:26 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:26.430+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:50.263081+0000 front 2024-09-21T06:09:50.263151+0000 (oldest deadline 2024-09-21T06:10:18.463704+0000) 2024-09-21T06:10:26.615 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:26 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:26.430+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.164997+0000 front 2024-09-21T06:10:00.164784+0000 (oldest deadline 2024-09-21T06:10:25.464885+0000) 2024-09-21T06:10:26.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:26 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:26.332+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:26.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:26 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:26.332+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:26.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:26 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:26.332+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:26.776 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:26 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:26.339+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:26.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:26 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:26.339+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:26.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:26 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:26.334+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.592972+0000 front 2024-09-21T06:09:45.593001+0000 (oldest deadline 2024-09-21T06:10:10.293110+0000) 2024-09-21T06:10:26.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:26 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:26.334+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:52.594436+0000 front 2024-09-21T06:09:52.594420+0000 (oldest deadline 2024-09-21T06:10:16.094143+0000) 2024-09-21T06:10:26.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:26 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:26.334+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.895863+0000 front 2024-09-21T06:10:01.895876+0000 (oldest deadline 2024-09-21T06:10:24.195799+0000) 2024-09-21T06:10:26.979 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:26 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:26.904+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:42.844894+0000 front 2024-09-21T06:09:42.844879+0000 (oldest deadline 2024-09-21T06:10:06.944750+0000) 2024-09-21T06:10:26.980 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:26 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:26.904+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.946321+0000 front 2024-09-21T06:09:53.946446+0000 (oldest deadline 2024-09-21T06:10:15.646469+0000) 2024-09-21T06:10:26.980 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:26 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5[46177]: 2024-09-21T06:10:26.904+0000 7f3d6d69d640 -1 osd.5 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:58.547237+0000 front 2024-09-21T06:09:58.547279+0000 (oldest deadline 2024-09-21T06:10:24.447362+0000) 2024-09-21T06:10:26.980 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:26 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:26.715+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:26.980 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:26 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:26.715+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:26.980 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:26 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:26.715+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:10:27.479 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:27 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:27.198+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.905829+0000 front 2024-09-21T06:09:53.905878+0000 (oldest deadline 2024-09-21T06:10:18.005850+0000) 2024-09-21T06:10:27.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:27 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:27.198+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.407097+0000 front 2024-09-21T06:10:01.407010+0000 (oldest deadline 2024-09-21T06:10:25.507175+0000) 2024-09-21T06:10:27.615 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:27 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:27.438+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:50.263081+0000 front 2024-09-21T06:09:50.263151+0000 (oldest deadline 2024-09-21T06:10:18.463704+0000) 2024-09-21T06:10:27.615 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:27 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:27.438+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.164997+0000 front 2024-09-21T06:10:00.164784+0000 (oldest deadline 2024-09-21T06:10:25.464885+0000) 2024-09-21T06:10:27.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:27 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:27.343+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:27.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:27 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:27.343+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:27.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:27 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:27.343+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:27.776 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:27 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:27.377+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:27.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:27 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:27.377+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:27.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:27 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:27.377+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.065236+0000 front 2024-09-21T06:10:01.064969+0000 (oldest deadline 2024-09-21T06:10:26.964917+0000) 2024-09-21T06:10:27.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:27 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:27.373+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.592972+0000 front 2024-09-21T06:09:45.593001+0000 (oldest deadline 2024-09-21T06:10:10.293110+0000) 2024-09-21T06:10:27.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:27 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:27.373+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:52.594436+0000 front 2024-09-21T06:09:52.594420+0000 (oldest deadline 2024-09-21T06:10:16.094143+0000) 2024-09-21T06:10:27.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:27 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:27.373+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.895863+0000 front 2024-09-21T06:10:01.895876+0000 (oldest deadline 2024-09-21T06:10:24.195799+0000) 2024-09-21T06:10:28.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:27 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:27.751+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:28.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:27 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:27.751+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:28.060 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:27 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:27.751+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:10:28.344 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:28 smithi192 podman[68574]: 2024-09-21 06:10:28.059160096 +0000 UTC m=+5.975813542 container died 0b3c306df5d1ad2f5d00f4a8431800b792ed801fa4ae276c9858821e60400a4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-21T06:10:28.345 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:28 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:28.159+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.905829+0000 front 2024-09-21T06:09:53.905878+0000 (oldest deadline 2024-09-21T06:10:18.005850+0000) 2024-09-21T06:10:28.345 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:28 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:28.159+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.407097+0000 front 2024-09-21T06:10:01.407010+0000 (oldest deadline 2024-09-21T06:10:25.507175+0000) 2024-09-21T06:10:28.678 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:28 smithi192 podman[68574]: 2024-09-21 06:10:28.434014365 +0000 UTC m=+6.350667821 container cleanup 0b3c306df5d1ad2f5d00f4a8431800b792ed801fa4ae276c9858821e60400a4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:10:28.678 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:28 smithi192 bash[68574]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5 2024-09-21T06:10:28.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:28 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:28.391+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.592972+0000 front 2024-09-21T06:09:45.593001+0000 (oldest deadline 2024-09-21T06:10:10.293110+0000) 2024-09-21T06:10:28.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:28 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:28.391+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:52.594436+0000 front 2024-09-21T06:09:52.594420+0000 (oldest deadline 2024-09-21T06:10:16.094143+0000) 2024-09-21T06:10:28.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:28 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:28.391+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.895863+0000 front 2024-09-21T06:10:01.895876+0000 (oldest deadline 2024-09-21T06:10:24.195799+0000) 2024-09-21T06:10:28.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:28 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:28.366+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:28.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:28 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:28.366+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:28.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:28 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:28.366+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.065236+0000 front 2024-09-21T06:10:01.064969+0000 (oldest deadline 2024-09-21T06:10:26.964917+0000) 2024-09-21T06:10:28.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:28 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:28.428+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:50.263081+0000 front 2024-09-21T06:09:50.263151+0000 (oldest deadline 2024-09-21T06:10:18.463704+0000) 2024-09-21T06:10:28.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:28 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:28.428+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.164997+0000 front 2024-09-21T06:10:00.164784+0000 (oldest deadline 2024-09-21T06:10:25.464885+0000) 2024-09-21T06:10:28.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:28 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:28.389+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:28.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:28 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:28.389+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:28.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:28 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:28.389+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:28.980 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:28 smithi192 podman[68591]: 2024-09-21 06:10:28.677052877 +0000 UTC m=+0.614035330 container remove 0b3c306df5d1ad2f5d00f4a8431800b792ed801fa4ae276c9858821e60400a4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-21T06:10:28.980 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:28 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:28.740+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:28.980 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:28 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:28.740+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:28.980 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:28 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:28.740+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:10:29.290 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:28 smithi192 podman[68680]: 2024-09-21 06:10:28.89855485 +0000 UTC m=+0.015629958 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:10:29.290 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:29 smithi192 podman[68680]: 2024-09-21 06:10:29.028134849 +0000 UTC m=+0.145209957 container create 931d5e8e29f6dfda714973fa1fefe826fdeff01067b24c649cb713bee227ac02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True) 2024-09-21T06:10:29.290 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:29 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:29.208+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.905829+0000 front 2024-09-21T06:09:53.905878+0000 (oldest deadline 2024-09-21T06:10:18.005850+0000) 2024-09-21T06:10:29.290 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:29 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:29.208+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.407097+0000 front 2024-09-21T06:10:01.407010+0000 (oldest deadline 2024-09-21T06:10:25.507175+0000) 2024-09-21T06:10:29.730 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:29 smithi192 podman[68680]: 2024-09-21 06:10:29.289148586 +0000 UTC m=+0.406223694 container init 931d5e8e29f6dfda714973fa1fefe826fdeff01067b24c649cb713bee227ac02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-21T06:10:29.730 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:29 smithi192 podman[68680]: 2024-09-21 06:10:29.292514863 +0000 UTC m=+0.409589971 container start 931d5e8e29f6dfda714973fa1fefe826fdeff01067b24c649cb713bee227ac02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-21T06:10:29.730 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:29 smithi192 podman[68680]: 2024-09-21 06:10:29.389389845 +0000 UTC m=+0.506464955 container attach 931d5e8e29f6dfda714973fa1fefe826fdeff01067b24c649cb713bee227ac02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-21T06:10:29.730 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 21 06:10:29 smithi192 podman[68680]: 2024-09-21 06:10:29.481222985 +0000 UTC m=+0.598298095 container died 931d5e8e29f6dfda714973fa1fefe826fdeff01067b24c649cb713bee227ac02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-21T06:10:29.776 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:29 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:29.353+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.592972+0000 front 2024-09-21T06:09:45.593001+0000 (oldest deadline 2024-09-21T06:10:10.293110+0000) 2024-09-21T06:10:29.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:29 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:29.353+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:52.594436+0000 front 2024-09-21T06:09:52.594420+0000 (oldest deadline 2024-09-21T06:10:16.094143+0000) 2024-09-21T06:10:29.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:29 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:29.353+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.895863+0000 front 2024-09-21T06:10:01.895876+0000 (oldest deadline 2024-09-21T06:10:24.195799+0000) 2024-09-21T06:10:29.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:29 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:29.392+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:29.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:29 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:29.392+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:29.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:29 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:29.392+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.065236+0000 front 2024-09-21T06:10:01.064969+0000 (oldest deadline 2024-09-21T06:10:26.964917+0000) 2024-09-21T06:10:29.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:29 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:29.467+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:50.263081+0000 front 2024-09-21T06:09:50.263151+0000 (oldest deadline 2024-09-21T06:10:18.463704+0000) 2024-09-21T06:10:29.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:29 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:29.467+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.164997+0000 front 2024-09-21T06:10:00.164784+0000 (oldest deadline 2024-09-21T06:10:25.464885+0000) 2024-09-21T06:10:29.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:29 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:29.434+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:29.866 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:29 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:29.434+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:29.866 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:29 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:29.434+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:30.127 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:29 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:29.750+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:30.128 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:29 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:29.750+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:30.128 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:29 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:29.750+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:10:30.143 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.5.service' 2024-09-21T06:10:30.182 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:10:30.182 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-21T06:10:30.182 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-21T06:10:30.182 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.6 2024-09-21T06:10:30.479 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:30 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:30.241+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.905829+0000 front 2024-09-21T06:09:53.905878+0000 (oldest deadline 2024-09-21T06:10:18.005850+0000) 2024-09-21T06:10:30.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:30 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:30.241+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.407097+0000 front 2024-09-21T06:10:01.407010+0000 (oldest deadline 2024-09-21T06:10:25.507175+0000) 2024-09-21T06:10:30.481 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:30 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:30.384+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:30.482 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:30 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:30.384+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:30.482 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:30 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:30.384+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.065236+0000 front 2024-09-21T06:10:01.064969+0000 (oldest deadline 2024-09-21T06:10:26.964917+0000) 2024-09-21T06:10:30.482 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:30 smithi117 systemd[1]: Stopping Ceph osd.6 for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T06:10:30.482 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:30 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:30.308+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.592972+0000 front 2024-09-21T06:09:45.593001+0000 (oldest deadline 2024-09-21T06:10:10.293110+0000) 2024-09-21T06:10:30.482 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:30 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:30.308+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:52.594436+0000 front 2024-09-21T06:09:52.594420+0000 (oldest deadline 2024-09-21T06:10:16.094143+0000) 2024-09-21T06:10:30.482 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:30 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:30.308+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.895863+0000 front 2024-09-21T06:10:01.895876+0000 (oldest deadline 2024-09-21T06:10:24.195799+0000) 2024-09-21T06:10:30.482 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:30 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:30.479+0000 7fbdecb62640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:10:30.482 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:30 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:30.479+0000 7fbdecb62640 -1 osd.6 90 *** Got signal Terminated *** 2024-09-21T06:10:30.482 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:30 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:30.479+0000 7fbdecb62640 -1 osd.6 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:10:30.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:30 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:30.462+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:50.263081+0000 front 2024-09-21T06:09:50.263151+0000 (oldest deadline 2024-09-21T06:10:18.463704+0000) 2024-09-21T06:10:30.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:30 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:30.462+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.164997+0000 front 2024-09-21T06:10:00.164784+0000 (oldest deadline 2024-09-21T06:10:25.464885+0000) 2024-09-21T06:10:30.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:30 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:30.398+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:30.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:30 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:30.398+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:30.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:30 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:30.398+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:31.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:30 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:30.800+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:31.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:30 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:30.800+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:31.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:30 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:30.800+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:10:31.729 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:31 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:31.264+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.905829+0000 front 2024-09-21T06:09:53.905878+0000 (oldest deadline 2024-09-21T06:10:18.005850+0000) 2024-09-21T06:10:31.730 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:31 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:31.264+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.407097+0000 front 2024-09-21T06:10:01.407010+0000 (oldest deadline 2024-09-21T06:10:25.507175+0000) 2024-09-21T06:10:31.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:31 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:31.329+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.592972+0000 front 2024-09-21T06:09:45.593001+0000 (oldest deadline 2024-09-21T06:10:10.293110+0000) 2024-09-21T06:10:31.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:31 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:31.329+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:52.594436+0000 front 2024-09-21T06:09:52.594420+0000 (oldest deadline 2024-09-21T06:10:16.094143+0000) 2024-09-21T06:10:31.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:31 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:31.329+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.895863+0000 front 2024-09-21T06:10:01.895876+0000 (oldest deadline 2024-09-21T06:10:24.195799+0000) 2024-09-21T06:10:31.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:31 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:31.428+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:31.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:31 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:31.428+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:31.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:31 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:31.428+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.065236+0000 front 2024-09-21T06:10:01.064969+0000 (oldest deadline 2024-09-21T06:10:26.964917+0000) 2024-09-21T06:10:31.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:31 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:31.441+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:50.263081+0000 front 2024-09-21T06:09:50.263151+0000 (oldest deadline 2024-09-21T06:10:18.463704+0000) 2024-09-21T06:10:31.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:31 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:31.441+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.164997+0000 front 2024-09-21T06:10:00.164784+0000 (oldest deadline 2024-09-21T06:10:25.464885+0000) 2024-09-21T06:10:31.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:31 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:31.427+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:31.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:31 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:31.427+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:31.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:31 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:31.427+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:32.229 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:31 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:31.827+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:32.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:31 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:31.827+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:32.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:31 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:31.827+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:10:32.729 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:32 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:32.255+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.905829+0000 front 2024-09-21T06:09:53.905878+0000 (oldest deadline 2024-09-21T06:10:18.005850+0000) 2024-09-21T06:10:32.730 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:32 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:32.255+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.407097+0000 front 2024-09-21T06:10:01.407010+0000 (oldest deadline 2024-09-21T06:10:25.507175+0000) 2024-09-21T06:10:32.776 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:32 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:32.312+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.592972+0000 front 2024-09-21T06:09:45.593001+0000 (oldest deadline 2024-09-21T06:10:10.293110+0000) 2024-09-21T06:10:32.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:32 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:32.312+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:52.594436+0000 front 2024-09-21T06:09:52.594420+0000 (oldest deadline 2024-09-21T06:10:16.094143+0000) 2024-09-21T06:10:32.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:32 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:32.312+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.895863+0000 front 2024-09-21T06:10:01.895876+0000 (oldest deadline 2024-09-21T06:10:24.195799+0000) 2024-09-21T06:10:32.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:32 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:32.431+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:32.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:32 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:32.431+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:32.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:32 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:32.431+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.065236+0000 front 2024-09-21T06:10:01.064969+0000 (oldest deadline 2024-09-21T06:10:26.964917+0000) 2024-09-21T06:10:32.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:32 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:32.475+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:50.263081+0000 front 2024-09-21T06:09:50.263151+0000 (oldest deadline 2024-09-21T06:10:18.463704+0000) 2024-09-21T06:10:32.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:32 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:32.475+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.164997+0000 front 2024-09-21T06:10:00.164784+0000 (oldest deadline 2024-09-21T06:10:25.464885+0000) 2024-09-21T06:10:32.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:32 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:32.412+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:32.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:32 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:32.412+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:32.866 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:32 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:32.412+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:33.216 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:32 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:32.861+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:33.217 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:32 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:32.861+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:33.217 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:32 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:32.861+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:10:33.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:33 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:33.215+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.905829+0000 front 2024-09-21T06:09:53.905878+0000 (oldest deadline 2024-09-21T06:10:18.005850+0000) 2024-09-21T06:10:33.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:33 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:33.215+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.407097+0000 front 2024-09-21T06:10:01.407010+0000 (oldest deadline 2024-09-21T06:10:25.507175+0000) 2024-09-21T06:10:33.776 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:33 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:33.318+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.592972+0000 front 2024-09-21T06:09:45.593001+0000 (oldest deadline 2024-09-21T06:10:10.293110+0000) 2024-09-21T06:10:33.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:33 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:33.318+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:52.594436+0000 front 2024-09-21T06:09:52.594420+0000 (oldest deadline 2024-09-21T06:10:16.094143+0000) 2024-09-21T06:10:33.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:33 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:33.318+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.895863+0000 front 2024-09-21T06:10:01.895876+0000 (oldest deadline 2024-09-21T06:10:24.195799+0000) 2024-09-21T06:10:33.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:33 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:33.384+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:33.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:33 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:33.384+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:33.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:33 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:33.384+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.065236+0000 front 2024-09-21T06:10:01.064969+0000 (oldest deadline 2024-09-21T06:10:26.964917+0000) 2024-09-21T06:10:33.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:33 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:33.507+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:50.263081+0000 front 2024-09-21T06:09:50.263151+0000 (oldest deadline 2024-09-21T06:10:18.463704+0000) 2024-09-21T06:10:33.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:33 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:33.507+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.164997+0000 front 2024-09-21T06:10:00.164784+0000 (oldest deadline 2024-09-21T06:10:25.464885+0000) 2024-09-21T06:10:33.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:33 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:33.447+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:33.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:33 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:33.447+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:33.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:33 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:33.447+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:34.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:33 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:33.840+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:34.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:33 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:33.840+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:34.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:33 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:33.840+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:10:34.526 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:34 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:34.272+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.592972+0000 front 2024-09-21T06:09:45.593001+0000 (oldest deadline 2024-09-21T06:10:10.293110+0000) 2024-09-21T06:10:34.527 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:34 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:34.272+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:52.594436+0000 front 2024-09-21T06:09:52.594420+0000 (oldest deadline 2024-09-21T06:10:16.094143+0000) 2024-09-21T06:10:34.527 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:34 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:34.272+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.895863+0000 front 2024-09-21T06:10:01.895876+0000 (oldest deadline 2024-09-21T06:10:24.195799+0000) 2024-09-21T06:10:34.527 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:34 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:34.368+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:34.527 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:34 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:34.368+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:34.527 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:34 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:34.368+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.065236+0000 front 2024-09-21T06:10:01.064969+0000 (oldest deadline 2024-09-21T06:10:26.964917+0000) 2024-09-21T06:10:34.527 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:34 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:34.368+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.166206+0000 front 2024-09-21T06:10:09.166235+0000 (oldest deadline 2024-09-21T06:10:33.866358+0000) 2024-09-21T06:10:34.730 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:34 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:34.238+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.905829+0000 front 2024-09-21T06:09:53.905878+0000 (oldest deadline 2024-09-21T06:10:18.005850+0000) 2024-09-21T06:10:34.730 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:34 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:34.238+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.407097+0000 front 2024-09-21T06:10:01.407010+0000 (oldest deadline 2024-09-21T06:10:25.507175+0000) 2024-09-21T06:10:34.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:34 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:34.539+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:50.263081+0000 front 2024-09-21T06:09:50.263151+0000 (oldest deadline 2024-09-21T06:10:18.463704+0000) 2024-09-21T06:10:34.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:34 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:34.539+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.164997+0000 front 2024-09-21T06:10:00.164784+0000 (oldest deadline 2024-09-21T06:10:25.464885+0000) 2024-09-21T06:10:34.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:34 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:34.539+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:10.766020+0000 front 2024-09-21T06:10:10.766236+0000 (oldest deadline 2024-09-21T06:10:33.666103+0000) 2024-09-21T06:10:34.866 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:34 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:34.451+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:34.866 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:34 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:34.451+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:34.866 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:34 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:34.451+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:35.208 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:34 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:34.857+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:35.208 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:34 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:34.857+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:35.208 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:34 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:34.857+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:10:35.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:35 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:35.206+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.905829+0000 front 2024-09-21T06:09:53.905878+0000 (oldest deadline 2024-09-21T06:10:18.005850+0000) 2024-09-21T06:10:35.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:35 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:35.206+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.407097+0000 front 2024-09-21T06:10:01.407010+0000 (oldest deadline 2024-09-21T06:10:25.507175+0000) 2024-09-21T06:10:35.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:35 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:35.308+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.592972+0000 front 2024-09-21T06:09:45.593001+0000 (oldest deadline 2024-09-21T06:10:10.293110+0000) 2024-09-21T06:10:35.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:35 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:35.308+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:52.594436+0000 front 2024-09-21T06:09:52.594420+0000 (oldest deadline 2024-09-21T06:10:16.094143+0000) 2024-09-21T06:10:35.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:35 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:35.308+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.895863+0000 front 2024-09-21T06:10:01.895876+0000 (oldest deadline 2024-09-21T06:10:24.195799+0000) 2024-09-21T06:10:35.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:35 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6[59982]: 2024-09-21T06:10:35.308+0000 7fbde914d640 -1 osd.6 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:10.597160+0000 front 2024-09-21T06:10:10.597295+0000 (oldest deadline 2024-09-21T06:10:35.297280+0000) 2024-09-21T06:10:35.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:35 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:35.326+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:35.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:35 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:35.326+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:35.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:35 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:35.326+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.065236+0000 front 2024-09-21T06:10:01.064969+0000 (oldest deadline 2024-09-21T06:10:26.964917+0000) 2024-09-21T06:10:35.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:35 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:35.326+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.166206+0000 front 2024-09-21T06:10:09.166235+0000 (oldest deadline 2024-09-21T06:10:33.866358+0000) 2024-09-21T06:10:35.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:35 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:35.568+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:50.263081+0000 front 2024-09-21T06:09:50.263151+0000 (oldest deadline 2024-09-21T06:10:18.463704+0000) 2024-09-21T06:10:35.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:35 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:35.568+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.164997+0000 front 2024-09-21T06:10:00.164784+0000 (oldest deadline 2024-09-21T06:10:25.464885+0000) 2024-09-21T06:10:35.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:35 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:35.568+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:10.766020+0000 front 2024-09-21T06:10:10.766236+0000 (oldest deadline 2024-09-21T06:10:33.666103+0000) 2024-09-21T06:10:35.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:35 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:35.440+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:35.866 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:35 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:35.440+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:35.866 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:35 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:35.440+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:35.866 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:35 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:35.440+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.904767+0000 front 2024-09-21T06:10:09.904947+0000 (oldest deadline 2024-09-21T06:10:34.604670+0000) 2024-09-21T06:10:36.162 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:35 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:35.876+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:36.162 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:35 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:35.876+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:36.162 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:35 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:35.876+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:10:36.367 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:36 smithi117 podman[106834]: 2024-09-21 06:10:36.106648068 +0000 UTC m=+5.727389026 container died 32e2348a9ea4387a6ca96a27c0c49b309511c05ed354a3915ec25adc3818ccfa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:10:36.479 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:36 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:36.159+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.905829+0000 front 2024-09-21T06:09:53.905878+0000 (oldest deadline 2024-09-21T06:10:18.005850+0000) 2024-09-21T06:10:36.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:36 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:36.159+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.407097+0000 front 2024-09-21T06:10:01.407010+0000 (oldest deadline 2024-09-21T06:10:25.507175+0000) 2024-09-21T06:10:36.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:36 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:36.159+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:10.808014+0000 front 2024-09-21T06:10:10.808204+0000 (oldest deadline 2024-09-21T06:10:36.108007+0000) 2024-09-21T06:10:36.617 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:36 smithi117 podman[106834]: 2024-09-21 06:10:36.59656583 +0000 UTC m=+6.217306787 container cleanup 32e2348a9ea4387a6ca96a27c0c49b309511c05ed354a3915ec25adc3818ccfa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True) 2024-09-21T06:10:36.617 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:36 smithi117 bash[106834]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6 2024-09-21T06:10:36.617 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:36 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:36.364+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:36.617 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:36 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:36.364+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:36.617 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:36 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:36.364+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.065236+0000 front 2024-09-21T06:10:01.064969+0000 (oldest deadline 2024-09-21T06:10:26.964917+0000) 2024-09-21T06:10:36.617 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:36 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:36.364+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.166206+0000 front 2024-09-21T06:10:09.166235+0000 (oldest deadline 2024-09-21T06:10:33.866358+0000) 2024-09-21T06:10:36.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:36 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:36.544+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:50.263081+0000 front 2024-09-21T06:09:50.263151+0000 (oldest deadline 2024-09-21T06:10:18.463704+0000) 2024-09-21T06:10:36.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:36 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:36.544+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.164997+0000 front 2024-09-21T06:10:00.164784+0000 (oldest deadline 2024-09-21T06:10:25.464885+0000) 2024-09-21T06:10:36.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:36 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:36.544+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:10.766020+0000 front 2024-09-21T06:10:10.766236+0000 (oldest deadline 2024-09-21T06:10:33.666103+0000) 2024-09-21T06:10:36.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:36 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:36.458+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:36.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:36 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:36.458+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:36.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:36 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:36.458+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:36.866 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:36 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:36.458+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.904767+0000 front 2024-09-21T06:10:09.904947+0000 (oldest deadline 2024-09-21T06:10:34.604670+0000) 2024-09-21T06:10:37.158 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:36 smithi117 podman[106849]: 2024-09-21 06:10:36.847953989 +0000 UTC m=+0.735445135 container remove 32e2348a9ea4387a6ca96a27c0c49b309511c05ed354a3915ec25adc3818ccfa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-21T06:10:37.209 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:36 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:36.862+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:37.210 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:36 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:36.862+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:37.210 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:36 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:36.862+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:10:37.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:37 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:37.207+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.905829+0000 front 2024-09-21T06:09:53.905878+0000 (oldest deadline 2024-09-21T06:10:18.005850+0000) 2024-09-21T06:10:37.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:37 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:37.207+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.407097+0000 front 2024-09-21T06:10:01.407010+0000 (oldest deadline 2024-09-21T06:10:25.507175+0000) 2024-09-21T06:10:37.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:37 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:37.207+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:10.808014+0000 front 2024-09-21T06:10:10.808204+0000 (oldest deadline 2024-09-21T06:10:36.108007+0000) 2024-09-21T06:10:37.486 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:37 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:37.370+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:37.486 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:37 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:37.370+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:37.486 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:37 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:37.370+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.065236+0000 front 2024-09-21T06:10:01.064969+0000 (oldest deadline 2024-09-21T06:10:26.964917+0000) 2024-09-21T06:10:37.486 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:37 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:37.370+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.166206+0000 front 2024-09-21T06:10:09.166235+0000 (oldest deadline 2024-09-21T06:10:33.866358+0000) 2024-09-21T06:10:37.486 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:37 smithi117 podman[106942]: 2024-09-21 06:10:37.157009404 +0000 UTC m=+0.108409967 container create 9da2159668bb03c6bcc5baf67e87246ea693f3e66444f954f5766e74962094ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:10:37.486 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:37 smithi117 podman[106942]: 2024-09-21 06:10:37.064696706 +0000 UTC m=+0.016097282 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:10:37.486 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:37 smithi117 podman[106942]: 2024-09-21 06:10:37.401545305 +0000 UTC m=+0.352945875 container init 9da2159668bb03c6bcc5baf67e87246ea693f3e66444f954f5766e74962094ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-21T06:10:37.486 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:37 smithi117 podman[106942]: 2024-09-21 06:10:37.405268897 +0000 UTC m=+0.356669460 container start 9da2159668bb03c6bcc5baf67e87246ea693f3e66444f954f5766e74962094ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True) 2024-09-21T06:10:37.487 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:37 smithi117 podman[106942]: 2024-09-21 06:10:37.484930221 +0000 UTC m=+0.436330788 container attach 9da2159668bb03c6bcc5baf67e87246ea693f3e66444f954f5766e74962094ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731) 2024-09-21T06:10:37.777 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 21 06:10:37 smithi117 podman[106942]: 2024-09-21 06:10:37.590894987 +0000 UTC m=+0.542295563 container died 9da2159668bb03c6bcc5baf67e87246ea693f3e66444f954f5766e74962094ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-6-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:10:37.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:37 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:37.578+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:50.263081+0000 front 2024-09-21T06:09:50.263151+0000 (oldest deadline 2024-09-21T06:10:18.463704+0000) 2024-09-21T06:10:37.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:37 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:37.578+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.164997+0000 front 2024-09-21T06:10:00.164784+0000 (oldest deadline 2024-09-21T06:10:25.464885+0000) 2024-09-21T06:10:37.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:37 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:37.578+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:10.766020+0000 front 2024-09-21T06:10:10.766236+0000 (oldest deadline 2024-09-21T06:10:33.666103+0000) 2024-09-21T06:10:37.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:37 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:37.494+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:37.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:37 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:37.494+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:37.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:37 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:37.494+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:37.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:37 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:37.494+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.904767+0000 front 2024-09-21T06:10:09.904947+0000 (oldest deadline 2024-09-21T06:10:34.604670+0000) 2024-09-21T06:10:38.203 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:37 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:37.894+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:38.203 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:37 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:37.894+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:38.203 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:37 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:37.894+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:10:38.350 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.6.service' 2024-09-21T06:10:38.392 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:10:38.392 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-21T06:10:38.392 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-21T06:10:38.392 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.7 2024-09-21T06:10:38.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:38 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:38.202+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.905829+0000 front 2024-09-21T06:09:53.905878+0000 (oldest deadline 2024-09-21T06:10:18.005850+0000) 2024-09-21T06:10:38.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:38 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:38.202+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.407097+0000 front 2024-09-21T06:10:01.407010+0000 (oldest deadline 2024-09-21T06:10:25.507175+0000) 2024-09-21T06:10:38.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:38 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:38.202+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:10.808014+0000 front 2024-09-21T06:10:10.808204+0000 (oldest deadline 2024-09-21T06:10:36.108007+0000) 2024-09-21T06:10:38.700 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:38 smithi138 systemd[1]: Stopping Ceph osd.7 for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T06:10:38.700 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:38 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:38.561+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:50.263081+0000 front 2024-09-21T06:09:50.263151+0000 (oldest deadline 2024-09-21T06:10:18.463704+0000) 2024-09-21T06:10:38.700 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:38 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:38.561+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.164997+0000 front 2024-09-21T06:10:00.164784+0000 (oldest deadline 2024-09-21T06:10:25.464885+0000) 2024-09-21T06:10:38.700 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:38 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:38.561+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:10.766020+0000 front 2024-09-21T06:10:10.766236+0000 (oldest deadline 2024-09-21T06:10:33.666103+0000) 2024-09-21T06:10:38.700 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:38 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:38.468+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:38.700 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:38 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:38.468+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:38.700 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:38 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:38.468+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:38.700 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:38 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:38.468+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.904767+0000 front 2024-09-21T06:10:09.904947+0000 (oldest deadline 2024-09-21T06:10:34.604670+0000) 2024-09-21T06:10:38.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:38 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:38.393+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:38.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:38 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:38.393+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:38.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:38 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:38.393+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.065236+0000 front 2024-09-21T06:10:01.064969+0000 (oldest deadline 2024-09-21T06:10:26.964917+0000) 2024-09-21T06:10:38.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:38 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:38.393+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.166206+0000 front 2024-09-21T06:10:09.166235+0000 (oldest deadline 2024-09-21T06:10:33.866358+0000) 2024-09-21T06:10:39.115 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:38 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:38.698+0000 7f5c69f0b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:10:39.115 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:38 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:38.698+0000 7f5c69f0b640 -1 osd.7 90 *** Got signal Terminated *** 2024-09-21T06:10:39.115 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:38 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:38.698+0000 7f5c69f0b640 -1 osd.7 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:10:39.160 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:38 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:38.857+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:39.161 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:38 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:38.857+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:39.161 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:38 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:38.857+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:10:39.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:39 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:39.158+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.905829+0000 front 2024-09-21T06:09:53.905878+0000 (oldest deadline 2024-09-21T06:10:18.005850+0000) 2024-09-21T06:10:39.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:39 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:39.158+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.407097+0000 front 2024-09-21T06:10:01.407010+0000 (oldest deadline 2024-09-21T06:10:25.507175+0000) 2024-09-21T06:10:39.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:39 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:39.158+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:10.808014+0000 front 2024-09-21T06:10:10.808204+0000 (oldest deadline 2024-09-21T06:10:36.108007+0000) 2024-09-21T06:10:39.776 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:39 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:39.363+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:39.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:39 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:39.363+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:39.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:39 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:39.363+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.065236+0000 front 2024-09-21T06:10:01.064969+0000 (oldest deadline 2024-09-21T06:10:26.964917+0000) 2024-09-21T06:10:39.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:39 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:39.363+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.166206+0000 front 2024-09-21T06:10:09.166235+0000 (oldest deadline 2024-09-21T06:10:33.866358+0000) 2024-09-21T06:10:39.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:39 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:39.516+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:50.263081+0000 front 2024-09-21T06:09:50.263151+0000 (oldest deadline 2024-09-21T06:10:18.463704+0000) 2024-09-21T06:10:39.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:39 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:39.516+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.164997+0000 front 2024-09-21T06:10:00.164784+0000 (oldest deadline 2024-09-21T06:10:25.464885+0000) 2024-09-21T06:10:39.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:39 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:39.516+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:10.766020+0000 front 2024-09-21T06:10:10.766236+0000 (oldest deadline 2024-09-21T06:10:33.666103+0000) 2024-09-21T06:10:39.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:39 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:39.438+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:39.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:39 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:39.438+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:39.866 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:39 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:39.438+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:39.866 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:39 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:39.438+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.904767+0000 front 2024-09-21T06:10:09.904947+0000 (oldest deadline 2024-09-21T06:10:34.604670+0000) 2024-09-21T06:10:40.202 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:39 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:39.897+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:40.202 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:39 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:39.897+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:40.202 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:39 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:39.897+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:10:40.479 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:40 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:40.200+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.905829+0000 front 2024-09-21T06:09:53.905878+0000 (oldest deadline 2024-09-21T06:10:18.005850+0000) 2024-09-21T06:10:40.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:40 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:40.200+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.407097+0000 front 2024-09-21T06:10:01.407010+0000 (oldest deadline 2024-09-21T06:10:25.507175+0000) 2024-09-21T06:10:40.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:40 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:40.200+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:10.808014+0000 front 2024-09-21T06:10:10.808204+0000 (oldest deadline 2024-09-21T06:10:36.108007+0000) 2024-09-21T06:10:40.776 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:40 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:40.404+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:40.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:40 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:40.404+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:40.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:40 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:40.404+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.065236+0000 front 2024-09-21T06:10:01.064969+0000 (oldest deadline 2024-09-21T06:10:26.964917+0000) 2024-09-21T06:10:40.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:40 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:40.404+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.166206+0000 front 2024-09-21T06:10:09.166235+0000 (oldest deadline 2024-09-21T06:10:33.866358+0000) 2024-09-21T06:10:40.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:40 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:40.485+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:50.263081+0000 front 2024-09-21T06:09:50.263151+0000 (oldest deadline 2024-09-21T06:10:18.463704+0000) 2024-09-21T06:10:40.866 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:40 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:40.486+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.164997+0000 front 2024-09-21T06:10:00.164784+0000 (oldest deadline 2024-09-21T06:10:25.464885+0000) 2024-09-21T06:10:40.866 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:40 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:40.486+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:10.766020+0000 front 2024-09-21T06:10:10.766236+0000 (oldest deadline 2024-09-21T06:10:33.666103+0000) 2024-09-21T06:10:40.866 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:40 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:40.424+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:40.866 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:40 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:40.424+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:40.866 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:40 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:40.424+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:40.866 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:40 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:40.424+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.904767+0000 front 2024-09-21T06:10:09.904947+0000 (oldest deadline 2024-09-21T06:10:34.604670+0000) 2024-09-21T06:10:41.219 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:40 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:40.915+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:41.219 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:40 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:40.915+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:41.219 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:40 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:40.915+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:10:41.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:41 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:41.217+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.905829+0000 front 2024-09-21T06:09:53.905878+0000 (oldest deadline 2024-09-21T06:10:18.005850+0000) 2024-09-21T06:10:41.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:41 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:41.217+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.407097+0000 front 2024-09-21T06:10:01.407010+0000 (oldest deadline 2024-09-21T06:10:25.507175+0000) 2024-09-21T06:10:41.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:41 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:41.217+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:10.808014+0000 front 2024-09-21T06:10:10.808204+0000 (oldest deadline 2024-09-21T06:10:36.108007+0000) 2024-09-21T06:10:41.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:41 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:41.217+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.108551+0000 front 2024-09-21T06:10:16.108434+0000 (oldest deadline 2024-09-21T06:10:40.208363+0000) 2024-09-21T06:10:41.776 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:41 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:41.448+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:41.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:41 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:41.448+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:41.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:41 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:41.448+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.065236+0000 front 2024-09-21T06:10:01.064969+0000 (oldest deadline 2024-09-21T06:10:26.964917+0000) 2024-09-21T06:10:41.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:41 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:41.448+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.166206+0000 front 2024-09-21T06:10:09.166235+0000 (oldest deadline 2024-09-21T06:10:33.866358+0000) 2024-09-21T06:10:41.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:41 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:41.527+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:50.263081+0000 front 2024-09-21T06:09:50.263151+0000 (oldest deadline 2024-09-21T06:10:18.463704+0000) 2024-09-21T06:10:41.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:41 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:41.527+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.164997+0000 front 2024-09-21T06:10:00.164784+0000 (oldest deadline 2024-09-21T06:10:25.464885+0000) 2024-09-21T06:10:41.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:41 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:41.527+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:10.766020+0000 front 2024-09-21T06:10:10.766236+0000 (oldest deadline 2024-09-21T06:10:33.666103+0000) 2024-09-21T06:10:41.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:41 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:41.527+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:13.666534+0000 front 2024-09-21T06:10:13.666483+0000 (oldest deadline 2024-09-21T06:10:40.667234+0000) 2024-09-21T06:10:41.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:41 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:41.384+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:41.866 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:41 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:41.384+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:41.866 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:41 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:41.384+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:41.866 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:41 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:41.384+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.904767+0000 front 2024-09-21T06:10:09.904947+0000 (oldest deadline 2024-09-21T06:10:34.604670+0000) 2024-09-21T06:10:42.200 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:41 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:41.907+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:42.200 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:41 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:41.907+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:42.200 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:41 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:41.907+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:10:42.200 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:41 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:41.907+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.482942+0000 front 2024-09-21T06:10:16.483052+0000 (oldest deadline 2024-09-21T06:10:41.182971+0000) 2024-09-21T06:10:42.479 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:42 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:42.198+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.905829+0000 front 2024-09-21T06:09:53.905878+0000 (oldest deadline 2024-09-21T06:10:18.005850+0000) 2024-09-21T06:10:42.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:42 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:42.198+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.407097+0000 front 2024-09-21T06:10:01.407010+0000 (oldest deadline 2024-09-21T06:10:25.507175+0000) 2024-09-21T06:10:42.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:42 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:42.198+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:10.808014+0000 front 2024-09-21T06:10:10.808204+0000 (oldest deadline 2024-09-21T06:10:36.108007+0000) 2024-09-21T06:10:42.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:42 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:42.198+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.108551+0000 front 2024-09-21T06:10:16.108434+0000 (oldest deadline 2024-09-21T06:10:40.208363+0000) 2024-09-21T06:10:42.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:42 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:42.408+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:42.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:42 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:42.408+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:42.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:42 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:42.408+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.065236+0000 front 2024-09-21T06:10:01.064969+0000 (oldest deadline 2024-09-21T06:10:26.964917+0000) 2024-09-21T06:10:42.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:42 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:42.408+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.166206+0000 front 2024-09-21T06:10:09.166235+0000 (oldest deadline 2024-09-21T06:10:33.866358+0000) 2024-09-21T06:10:42.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:42 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:42.515+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:50.263081+0000 front 2024-09-21T06:09:50.263151+0000 (oldest deadline 2024-09-21T06:10:18.463704+0000) 2024-09-21T06:10:42.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:42 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:42.515+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.164997+0000 front 2024-09-21T06:10:00.164784+0000 (oldest deadline 2024-09-21T06:10:25.464885+0000) 2024-09-21T06:10:42.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:42 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:42.515+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:10.766020+0000 front 2024-09-21T06:10:10.766236+0000 (oldest deadline 2024-09-21T06:10:33.666103+0000) 2024-09-21T06:10:42.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:42 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:42.515+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:13.666534+0000 front 2024-09-21T06:10:13.666483+0000 (oldest deadline 2024-09-21T06:10:40.667234+0000) 2024-09-21T06:10:42.866 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:42 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:42.375+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:42.866 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:42 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:42.375+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:42.866 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:42 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:42.375+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:42.866 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:42 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:42.375+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.904767+0000 front 2024-09-21T06:10:09.904947+0000 (oldest deadline 2024-09-21T06:10:34.604670+0000) 2024-09-21T06:10:43.203 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:42 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:42.896+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:43.204 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:42 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:42.896+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:43.204 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:42 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:42.896+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:10:43.204 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:42 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:42.896+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.482942+0000 front 2024-09-21T06:10:16.483052+0000 (oldest deadline 2024-09-21T06:10:41.182971+0000) 2024-09-21T06:10:43.479 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:43 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:43.202+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.905829+0000 front 2024-09-21T06:09:53.905878+0000 (oldest deadline 2024-09-21T06:10:18.005850+0000) 2024-09-21T06:10:43.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:43 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:43.202+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.407097+0000 front 2024-09-21T06:10:01.407010+0000 (oldest deadline 2024-09-21T06:10:25.507175+0000) 2024-09-21T06:10:43.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:43 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:43.202+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:10.808014+0000 front 2024-09-21T06:10:10.808204+0000 (oldest deadline 2024-09-21T06:10:36.108007+0000) 2024-09-21T06:10:43.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:43 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:43.202+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.108551+0000 front 2024-09-21T06:10:16.108434+0000 (oldest deadline 2024-09-21T06:10:40.208363+0000) 2024-09-21T06:10:43.776 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:43 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:43.411+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:43.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:43 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:43.411+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:43.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:43 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:43.411+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.065236+0000 front 2024-09-21T06:10:01.064969+0000 (oldest deadline 2024-09-21T06:10:26.964917+0000) 2024-09-21T06:10:43.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:43 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:43.411+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.166206+0000 front 2024-09-21T06:10:09.166235+0000 (oldest deadline 2024-09-21T06:10:33.866358+0000) 2024-09-21T06:10:43.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:43 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:43.491+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:50.263081+0000 front 2024-09-21T06:09:50.263151+0000 (oldest deadline 2024-09-21T06:10:18.463704+0000) 2024-09-21T06:10:43.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:43 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:43.491+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.164997+0000 front 2024-09-21T06:10:00.164784+0000 (oldest deadline 2024-09-21T06:10:25.464885+0000) 2024-09-21T06:10:43.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:43 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:43.491+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:10.766020+0000 front 2024-09-21T06:10:10.766236+0000 (oldest deadline 2024-09-21T06:10:33.666103+0000) 2024-09-21T06:10:43.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:43 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7[51924]: 2024-09-21T06:10:43.491+0000 7f5c664f6640 -1 osd.7 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:13.666534+0000 front 2024-09-21T06:10:13.666483+0000 (oldest deadline 2024-09-21T06:10:40.667234+0000) 2024-09-21T06:10:43.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:43 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:43.407+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:43.866 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:43 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:43.407+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:43.866 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:43 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:43.407+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:43.866 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:43 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:43.407+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.904767+0000 front 2024-09-21T06:10:09.904947+0000 (oldest deadline 2024-09-21T06:10:34.604670+0000) 2024-09-21T06:10:44.163 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:43 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:43.908+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:44.163 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:43 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:43.908+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:44.163 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:43 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:43.908+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:10:44.163 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:43 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:43.908+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.482942+0000 front 2024-09-21T06:10:16.483052+0000 (oldest deadline 2024-09-21T06:10:41.182971+0000) 2024-09-21T06:10:44.433 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:44 smithi138 podman[67906]: 2024-09-21 06:10:44.175178789 +0000 UTC m=+5.594727358 container died 9dfa83dbeb4ac761bed9ff8e3426727ebe1d86539fa2a768430ce75638a1f63e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-21T06:10:44.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:44 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:44.161+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.905829+0000 front 2024-09-21T06:09:53.905878+0000 (oldest deadline 2024-09-21T06:10:18.005850+0000) 2024-09-21T06:10:44.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:44 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:44.161+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.407097+0000 front 2024-09-21T06:10:01.407010+0000 (oldest deadline 2024-09-21T06:10:25.507175+0000) 2024-09-21T06:10:44.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:44 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:44.161+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:10.808014+0000 front 2024-09-21T06:10:10.808204+0000 (oldest deadline 2024-09-21T06:10:36.108007+0000) 2024-09-21T06:10:44.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:44 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:44.161+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.108551+0000 front 2024-09-21T06:10:16.108434+0000 (oldest deadline 2024-09-21T06:10:40.208363+0000) 2024-09-21T06:10:44.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:44 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:44.424+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:44.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:44 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:44.424+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:44.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:44 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:44.424+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.065236+0000 front 2024-09-21T06:10:01.064969+0000 (oldest deadline 2024-09-21T06:10:26.964917+0000) 2024-09-21T06:10:44.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:44 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:44.424+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.166206+0000 front 2024-09-21T06:10:09.166235+0000 (oldest deadline 2024-09-21T06:10:33.866358+0000) 2024-09-21T06:10:44.842 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:44 smithi138 podman[67906]: 2024-09-21 06:10:44.531163061 +0000 UTC m=+5.950711634 container cleanup 9dfa83dbeb4ac761bed9ff8e3426727ebe1d86539fa2a768430ce75638a1f63e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-21T06:10:44.842 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:44 smithi138 bash[67906]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7 2024-09-21T06:10:44.842 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:44 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:44.433+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:44.843 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:44 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:44.433+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:44.843 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:44 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:44.433+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:44.843 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:44 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:44.433+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.904767+0000 front 2024-09-21T06:10:09.904947+0000 (oldest deadline 2024-09-21T06:10:34.604670+0000) 2024-09-21T06:10:45.115 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:44 smithi138 podman[67922]: 2024-09-21 06:10:44.841204373 +0000 UTC m=+0.662397721 container remove 9dfa83dbeb4ac761bed9ff8e3426727ebe1d86539fa2a768430ce75638a1f63e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:10:45.209 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:44 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:44.866+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:45.210 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:44 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:44.866+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:45.210 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:44 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:44.866+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:10:45.210 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:44 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:44.866+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.482942+0000 front 2024-09-21T06:10:16.483052+0000 (oldest deadline 2024-09-21T06:10:41.182971+0000) 2024-09-21T06:10:45.437 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:45 smithi138 podman[68012]: 2024-09-21 06:10:45.087889434 +0000 UTC m=+0.015652043 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:10:45.437 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:45 smithi138 podman[68012]: 2024-09-21 06:10:45.225483282 +0000 UTC m=+0.153245888 container create 8e05a9a2004528087ce508ee25cae104f8c1c8008297c4df2852968ab1b67b2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:10:45.438 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:45 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:45.401+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:45.438 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:45 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:45.401+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:45.438 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:45 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:45.401+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:45.438 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:45 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:45.401+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.904767+0000 front 2024-09-21T06:10:09.904947+0000 (oldest deadline 2024-09-21T06:10:34.604670+0000) 2024-09-21T06:10:45.438 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:45 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:45.401+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:14.604976+0000 front 2024-09-21T06:10:14.605026+0000 (oldest deadline 2024-09-21T06:10:44.605381+0000) 2024-09-21T06:10:45.479 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:45 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:45.208+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.905829+0000 front 2024-09-21T06:09:53.905878+0000 (oldest deadline 2024-09-21T06:10:18.005850+0000) 2024-09-21T06:10:45.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:45 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:45.208+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.407097+0000 front 2024-09-21T06:10:01.407010+0000 (oldest deadline 2024-09-21T06:10:25.507175+0000) 2024-09-21T06:10:45.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:45 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:45.208+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:10.808014+0000 front 2024-09-21T06:10:10.808204+0000 (oldest deadline 2024-09-21T06:10:36.108007+0000) 2024-09-21T06:10:45.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:45 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:45.208+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.108551+0000 front 2024-09-21T06:10:16.108434+0000 (oldest deadline 2024-09-21T06:10:40.208363+0000) 2024-09-21T06:10:45.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:45 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:45.455+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:45.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:45 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:45.455+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:45.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:45 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:45.455+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.065236+0000 front 2024-09-21T06:10:01.064969+0000 (oldest deadline 2024-09-21T06:10:26.964917+0000) 2024-09-21T06:10:45.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:45 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:45.455+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.166206+0000 front 2024-09-21T06:10:09.166235+0000 (oldest deadline 2024-09-21T06:10:33.866358+0000) 2024-09-21T06:10:45.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:45 smithi138 podman[68012]: 2024-09-21 06:10:45.436245611 +0000 UTC m=+0.364008217 container init 8e05a9a2004528087ce508ee25cae104f8c1c8008297c4df2852968ab1b67b2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2) 2024-09-21T06:10:45.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:45 smithi138 podman[68012]: 2024-09-21 06:10:45.440017741 +0000 UTC m=+0.367780356 container start 8e05a9a2004528087ce508ee25cae104f8c1c8008297c4df2852968ab1b67b2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-21T06:10:45.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:45 smithi138 podman[68012]: 2024-09-21 06:10:45.536224905 +0000 UTC m=+0.463987524 container attach 8e05a9a2004528087ce508ee25cae104f8c1c8008297c4df2852968ab1b67b2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7-deactivate, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:10:45.865 INFO:journalctl@ceph.osd.7.smithi138.stdout:Sep 21 06:10:45 smithi138 podman[68012]: 2024-09-21 06:10:45.622943842 +0000 UTC m=+0.550706456 container died 8e05a9a2004528087ce508ee25cae104f8c1c8008297c4df2852968ab1b67b2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-21T06:10:46.209 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.7.service' 2024-09-21T06:10:46.226 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:45 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:45.836+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:46.226 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:45 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:45.836+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:46.226 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:45 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:45.836+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:10:46.226 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:45 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:45.836+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.482942+0000 front 2024-09-21T06:10:16.483052+0000 (oldest deadline 2024-09-21T06:10:41.182971+0000) 2024-09-21T06:10:46.252 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:10:46.252 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-21T06:10:46.252 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-21T06:10:46.253 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.8 2024-09-21T06:10:46.462 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:46 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:46.395+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:46.462 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:46 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:46.395+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:46.462 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:46 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:46.395+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:46.462 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:46 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:46.395+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.904767+0000 front 2024-09-21T06:10:09.904947+0000 (oldest deadline 2024-09-21T06:10:34.604670+0000) 2024-09-21T06:10:46.462 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:46 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:46.395+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:14.604976+0000 front 2024-09-21T06:10:14.605026+0000 (oldest deadline 2024-09-21T06:10:44.605381+0000) 2024-09-21T06:10:46.479 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:46 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:46.225+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.905829+0000 front 2024-09-21T06:09:53.905878+0000 (oldest deadline 2024-09-21T06:10:18.005850+0000) 2024-09-21T06:10:46.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:46 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:46.225+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.407097+0000 front 2024-09-21T06:10:01.407010+0000 (oldest deadline 2024-09-21T06:10:25.507175+0000) 2024-09-21T06:10:46.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:46 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:46.225+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:10.808014+0000 front 2024-09-21T06:10:10.808204+0000 (oldest deadline 2024-09-21T06:10:36.108007+0000) 2024-09-21T06:10:46.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:46 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:46.225+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.108551+0000 front 2024-09-21T06:10:16.108434+0000 (oldest deadline 2024-09-21T06:10:40.208363+0000) 2024-09-21T06:10:46.480 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:46 smithi192 systemd[1]: Stopping Ceph osd.8 for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T06:10:46.776 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:46 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:46.454+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:46.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:46 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:46.454+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:46.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:46 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:46.454+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.065236+0000 front 2024-09-21T06:10:01.064969+0000 (oldest deadline 2024-09-21T06:10:26.964917+0000) 2024-09-21T06:10:46.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:46 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:46.454+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.166206+0000 front 2024-09-21T06:10:09.166235+0000 (oldest deadline 2024-09-21T06:10:33.866358+0000) 2024-09-21T06:10:46.882 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:46 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:46.554+0000 7feed09c3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:10:46.882 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:46 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:46.554+0000 7feed09c3640 -1 osd.8 90 *** Got signal Terminated *** 2024-09-21T06:10:46.882 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:46 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:46.554+0000 7feed09c3640 -1 osd.8 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:10:47.229 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:46 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:46.880+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:47.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:46 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:46.880+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:47.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:46 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:46.880+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:10:47.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:46 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:46.880+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.482942+0000 front 2024-09-21T06:10:16.483052+0000 (oldest deadline 2024-09-21T06:10:41.182971+0000) 2024-09-21T06:10:47.730 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:47 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:47.240+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.905829+0000 front 2024-09-21T06:09:53.905878+0000 (oldest deadline 2024-09-21T06:10:18.005850+0000) 2024-09-21T06:10:47.730 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:47 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:47.240+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.407097+0000 front 2024-09-21T06:10:01.407010+0000 (oldest deadline 2024-09-21T06:10:25.507175+0000) 2024-09-21T06:10:47.730 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:47 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:47.240+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:10.808014+0000 front 2024-09-21T06:10:10.808204+0000 (oldest deadline 2024-09-21T06:10:36.108007+0000) 2024-09-21T06:10:47.730 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:47 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:47.240+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.108551+0000 front 2024-09-21T06:10:16.108434+0000 (oldest deadline 2024-09-21T06:10:40.208363+0000) 2024-09-21T06:10:47.776 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:47 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:47.474+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:47.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:47 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:47.474+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:47.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:47 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:47.474+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.065236+0000 front 2024-09-21T06:10:01.064969+0000 (oldest deadline 2024-09-21T06:10:26.964917+0000) 2024-09-21T06:10:47.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:47 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:47.474+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.166206+0000 front 2024-09-21T06:10:09.166235+0000 (oldest deadline 2024-09-21T06:10:33.866358+0000) 2024-09-21T06:10:47.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:47 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:47.366+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:47.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:47 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:47.366+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:47.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:47 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:47.367+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:47.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:47 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:47.367+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.904767+0000 front 2024-09-21T06:10:09.904947+0000 (oldest deadline 2024-09-21T06:10:34.604670+0000) 2024-09-21T06:10:47.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:47 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:47.367+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:14.604976+0000 front 2024-09-21T06:10:14.605026+0000 (oldest deadline 2024-09-21T06:10:44.605381+0000) 2024-09-21T06:10:48.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:47 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:47.848+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:48.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:47 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:47.848+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:48.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:47 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:47.848+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:10:48.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:47 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:47.848+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.482942+0000 front 2024-09-21T06:10:16.483052+0000 (oldest deadline 2024-09-21T06:10:41.182971+0000) 2024-09-21T06:10:48.730 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:48 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:48.268+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.905829+0000 front 2024-09-21T06:09:53.905878+0000 (oldest deadline 2024-09-21T06:10:18.005850+0000) 2024-09-21T06:10:48.730 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:48 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:48.268+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.407097+0000 front 2024-09-21T06:10:01.407010+0000 (oldest deadline 2024-09-21T06:10:25.507175+0000) 2024-09-21T06:10:48.730 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:48 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:48.268+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:10.808014+0000 front 2024-09-21T06:10:10.808204+0000 (oldest deadline 2024-09-21T06:10:36.108007+0000) 2024-09-21T06:10:48.730 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:48 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:48.268+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.108551+0000 front 2024-09-21T06:10:16.108434+0000 (oldest deadline 2024-09-21T06:10:40.208363+0000) 2024-09-21T06:10:48.776 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:48 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:48.505+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:48.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:48 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:48.505+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:48.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:48 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:48.505+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.065236+0000 front 2024-09-21T06:10:01.064969+0000 (oldest deadline 2024-09-21T06:10:26.964917+0000) 2024-09-21T06:10:48.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:48 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:48.505+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.166206+0000 front 2024-09-21T06:10:09.166235+0000 (oldest deadline 2024-09-21T06:10:33.866358+0000) 2024-09-21T06:10:48.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:48 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:48.376+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:48.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:48 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:48.376+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:48.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:48 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:48.376+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:48.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:48 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:48.376+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.904767+0000 front 2024-09-21T06:10:09.904947+0000 (oldest deadline 2024-09-21T06:10:34.604670+0000) 2024-09-21T06:10:48.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:48 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:48.376+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:14.604976+0000 front 2024-09-21T06:10:14.605026+0000 (oldest deadline 2024-09-21T06:10:44.605381+0000) 2024-09-21T06:10:49.229 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:48 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:48.810+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:49.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:48 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:48.810+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:49.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:48 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:48.810+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:10:49.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:48 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:48.810+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.482942+0000 front 2024-09-21T06:10:16.483052+0000 (oldest deadline 2024-09-21T06:10:41.182971+0000) 2024-09-21T06:10:49.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:49 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:49.355+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:49.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:49 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:49.355+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:49.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:49 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:49.355+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:49.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:49 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:49.355+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.904767+0000 front 2024-09-21T06:10:09.904947+0000 (oldest deadline 2024-09-21T06:10:34.604670+0000) 2024-09-21T06:10:49.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:49 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:49.355+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:14.604976+0000 front 2024-09-21T06:10:14.605026+0000 (oldest deadline 2024-09-21T06:10:44.605381+0000) 2024-09-21T06:10:49.730 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:49 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:49.271+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.905829+0000 front 2024-09-21T06:09:53.905878+0000 (oldest deadline 2024-09-21T06:10:18.005850+0000) 2024-09-21T06:10:49.730 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:49 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:49.271+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.407097+0000 front 2024-09-21T06:10:01.407010+0000 (oldest deadline 2024-09-21T06:10:25.507175+0000) 2024-09-21T06:10:49.730 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:49 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:49.271+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:10.808014+0000 front 2024-09-21T06:10:10.808204+0000 (oldest deadline 2024-09-21T06:10:36.108007+0000) 2024-09-21T06:10:49.730 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:49 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:49.271+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.108551+0000 front 2024-09-21T06:10:16.108434+0000 (oldest deadline 2024-09-21T06:10:40.208363+0000) 2024-09-21T06:10:49.776 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:49 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:49.465+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:49.776 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:49 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:49.465+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:49.776 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:49 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:49.465+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.065236+0000 front 2024-09-21T06:10:01.064969+0000 (oldest deadline 2024-09-21T06:10:26.964917+0000) 2024-09-21T06:10:49.776 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:49 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:49.465+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.166206+0000 front 2024-09-21T06:10:09.166235+0000 (oldest deadline 2024-09-21T06:10:33.866358+0000) 2024-09-21T06:10:49.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:49 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:49.465+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:27.368783+0000 front 2024-09-21T06:10:27.368689+0000 (oldest deadline 2024-09-21T06:10:48.969062+0000) 2024-09-21T06:10:50.229 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:49 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:49.802+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:50.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:49 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:49.802+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:50.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:49 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:49.802+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:10:50.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:49 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:49.802+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.482942+0000 front 2024-09-21T06:10:16.483052+0000 (oldest deadline 2024-09-21T06:10:41.182971+0000) 2024-09-21T06:10:50.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:50 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:50.325+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:50.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:50 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:50.325+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:50.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:50 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:50.325+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:50.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:50 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:50.325+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.904767+0000 front 2024-09-21T06:10:09.904947+0000 (oldest deadline 2024-09-21T06:10:34.604670+0000) 2024-09-21T06:10:50.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:50 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:50.325+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:14.604976+0000 front 2024-09-21T06:10:14.605026+0000 (oldest deadline 2024-09-21T06:10:44.605381+0000) 2024-09-21T06:10:50.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:50 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:50.325+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:24.605993+0000 front 2024-09-21T06:10:24.605887+0000 (oldest deadline 2024-09-21T06:10:49.905850+0000) 2024-09-21T06:10:50.730 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:50 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:50.300+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.905829+0000 front 2024-09-21T06:09:53.905878+0000 (oldest deadline 2024-09-21T06:10:18.005850+0000) 2024-09-21T06:10:50.730 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:50 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:50.300+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.407097+0000 front 2024-09-21T06:10:01.407010+0000 (oldest deadline 2024-09-21T06:10:25.507175+0000) 2024-09-21T06:10:50.730 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:50 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:50.300+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:10.808014+0000 front 2024-09-21T06:10:10.808204+0000 (oldest deadline 2024-09-21T06:10:36.108007+0000) 2024-09-21T06:10:50.730 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:50 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:50.300+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.108551+0000 front 2024-09-21T06:10:16.108434+0000 (oldest deadline 2024-09-21T06:10:40.208363+0000) 2024-09-21T06:10:50.730 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:50 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:50.300+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:26.009604+0000 front 2024-09-21T06:10:26.009680+0000 (oldest deadline 2024-09-21T06:10:49.509761+0000) 2024-09-21T06:10:50.776 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:50 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:50.479+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:50.776 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:50 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:50.479+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:50.776 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:50 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:50.479+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.065236+0000 front 2024-09-21T06:10:01.064969+0000 (oldest deadline 2024-09-21T06:10:26.964917+0000) 2024-09-21T06:10:50.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:50 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:50.479+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.166206+0000 front 2024-09-21T06:10:09.166235+0000 (oldest deadline 2024-09-21T06:10:33.866358+0000) 2024-09-21T06:10:50.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:50 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:50.479+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:27.368783+0000 front 2024-09-21T06:10:27.368689+0000 (oldest deadline 2024-09-21T06:10:48.969062+0000) 2024-09-21T06:10:51.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:50 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:50.811+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:51.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:50 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:50.811+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:51.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:50 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:50.811+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:10:51.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:50 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:50.811+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.482942+0000 front 2024-09-21T06:10:16.483052+0000 (oldest deadline 2024-09-21T06:10:41.182971+0000) 2024-09-21T06:10:51.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:50 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:50.811+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:26.384383+0000 front 2024-09-21T06:10:26.384467+0000 (oldest deadline 2024-09-21T06:10:49.884498+0000) 2024-09-21T06:10:51.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:51 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:51.298+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:51.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:51 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:51.298+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:51.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:51 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:51.298+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:51.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:51 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:51.298+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.904767+0000 front 2024-09-21T06:10:09.904947+0000 (oldest deadline 2024-09-21T06:10:34.604670+0000) 2024-09-21T06:10:51.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:51 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:51.298+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:14.604976+0000 front 2024-09-21T06:10:14.605026+0000 (oldest deadline 2024-09-21T06:10:44.605381+0000) 2024-09-21T06:10:51.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:51 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:51.298+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:24.605993+0000 front 2024-09-21T06:10:24.605887+0000 (oldest deadline 2024-09-21T06:10:49.905850+0000) 2024-09-21T06:10:51.730 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:51 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:51.278+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.905829+0000 front 2024-09-21T06:09:53.905878+0000 (oldest deadline 2024-09-21T06:10:18.005850+0000) 2024-09-21T06:10:51.730 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:51 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:51.278+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.407097+0000 front 2024-09-21T06:10:01.407010+0000 (oldest deadline 2024-09-21T06:10:25.507175+0000) 2024-09-21T06:10:51.730 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:51 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:51.278+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:10.808014+0000 front 2024-09-21T06:10:10.808204+0000 (oldest deadline 2024-09-21T06:10:36.108007+0000) 2024-09-21T06:10:51.730 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:51 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:51.278+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.108551+0000 front 2024-09-21T06:10:16.108434+0000 (oldest deadline 2024-09-21T06:10:40.208363+0000) 2024-09-21T06:10:51.730 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:51 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8[52817]: 2024-09-21T06:10:51.278+0000 7feeccfae640 -1 osd.8 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:26.009604+0000 front 2024-09-21T06:10:26.009680+0000 (oldest deadline 2024-09-21T06:10:49.509761+0000) 2024-09-21T06:10:51.776 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:51 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:51.463+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:51.776 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:51 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:51.463+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:51.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:51 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:51.463+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.065236+0000 front 2024-09-21T06:10:01.064969+0000 (oldest deadline 2024-09-21T06:10:26.964917+0000) 2024-09-21T06:10:51.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:51 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:51.463+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.166206+0000 front 2024-09-21T06:10:09.166235+0000 (oldest deadline 2024-09-21T06:10:33.866358+0000) 2024-09-21T06:10:51.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:51 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:51.463+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:27.368783+0000 front 2024-09-21T06:10:27.368689+0000 (oldest deadline 2024-09-21T06:10:48.969062+0000) 2024-09-21T06:10:52.163 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:51 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:51.831+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:52.163 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:51 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:51.831+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:52.164 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:51 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:51.831+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:10:52.164 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:51 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:51.831+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.482942+0000 front 2024-09-21T06:10:16.483052+0000 (oldest deadline 2024-09-21T06:10:41.182971+0000) 2024-09-21T06:10:52.164 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:51 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:51.831+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:26.384383+0000 front 2024-09-21T06:10:26.384467+0000 (oldest deadline 2024-09-21T06:10:49.884498+0000) 2024-09-21T06:10:52.431 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:52 smithi192 podman[68784]: 2024-09-21 06:10:52.162414945 +0000 UTC m=+5.714949427 container died 276e48ec317532eb5dd3290e5726657c9c8b2627f6019d1a9820d2c442a4a336 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-21T06:10:52.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:52 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:52.312+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:52.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:52 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:52.312+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:52.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:52 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:52.312+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:52.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:52 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:52.312+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.904767+0000 front 2024-09-21T06:10:09.904947+0000 (oldest deadline 2024-09-21T06:10:34.604670+0000) 2024-09-21T06:10:52.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:52 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:52.312+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:14.604976+0000 front 2024-09-21T06:10:14.605026+0000 (oldest deadline 2024-09-21T06:10:44.605381+0000) 2024-09-21T06:10:52.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:52 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:52.312+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:24.605993+0000 front 2024-09-21T06:10:24.605887+0000 (oldest deadline 2024-09-21T06:10:49.905850+0000) 2024-09-21T06:10:52.689 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:52 smithi192 podman[68784]: 2024-09-21 06:10:52.503773421 +0000 UTC m=+6.056307895 container cleanup 276e48ec317532eb5dd3290e5726657c9c8b2627f6019d1a9820d2c442a4a336 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:10:52.689 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:52 smithi192 bash[68784]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8 2024-09-21T06:10:52.776 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:52 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:52.501+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:52.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:52 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:52.501+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:52.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:52 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:52.501+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.065236+0000 front 2024-09-21T06:10:01.064969+0000 (oldest deadline 2024-09-21T06:10:26.964917+0000) 2024-09-21T06:10:52.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:52 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:52.501+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.166206+0000 front 2024-09-21T06:10:09.166235+0000 (oldest deadline 2024-09-21T06:10:33.866358+0000) 2024-09-21T06:10:52.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:52 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:52.501+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:27.368783+0000 front 2024-09-21T06:10:27.368689+0000 (oldest deadline 2024-09-21T06:10:48.969062+0000) 2024-09-21T06:10:52.979 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:52 smithi192 podman[68799]: 2024-09-21 06:10:52.688316026 +0000 UTC m=+0.524546704 container remove 276e48ec317532eb5dd3290e5726657c9c8b2627f6019d1a9820d2c442a4a336 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-21T06:10:52.980 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:52 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:52.866+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:52.980 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:52 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:52.866+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:52.980 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:52 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:52.866+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:10:52.980 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:52 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:52.866+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.482942+0000 front 2024-09-21T06:10:16.483052+0000 (oldest deadline 2024-09-21T06:10:41.182971+0000) 2024-09-21T06:10:52.980 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:52 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:52.866+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:26.384383+0000 front 2024-09-21T06:10:26.384467+0000 (oldest deadline 2024-09-21T06:10:49.884498+0000) 2024-09-21T06:10:53.392 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:53 smithi192 podman[68890]: 2024-09-21 06:10:52.994816579 +0000 UTC m=+0.015614184 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:10:53.392 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:53 smithi192 podman[68890]: 2024-09-21 06:10:53.10602517 +0000 UTC m=+0.126822779 container create e21b14d98f7a579c7389b769a092367d7bbe228e42cf74afa3890f6a50b183b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-21T06:10:53.393 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:53 smithi192 podman[68890]: 2024-09-21 06:10:53.333565938 +0000 UTC m=+0.354363541 container init e21b14d98f7a579c7389b769a092367d7bbe228e42cf74afa3890f6a50b183b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:10:53.393 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:53 smithi192 podman[68890]: 2024-09-21 06:10:53.336905078 +0000 UTC m=+0.357702682 container start e21b14d98f7a579c7389b769a092367d7bbe228e42cf74afa3890f6a50b183b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0) 2024-09-21T06:10:53.393 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:53 smithi192 podman[68890]: 2024-09-21 06:10:53.391531115 +0000 UTC m=+0.412328738 container attach e21b14d98f7a579c7389b769a092367d7bbe228e42cf74afa3890f6a50b183b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-21T06:10:53.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:53 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:53.356+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:53.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:53 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:53.356+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:53.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:53 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:53.356+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:53.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:53 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:53.356+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.904767+0000 front 2024-09-21T06:10:09.904947+0000 (oldest deadline 2024-09-21T06:10:34.604670+0000) 2024-09-21T06:10:53.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:53 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:53.356+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:14.604976+0000 front 2024-09-21T06:10:14.605026+0000 (oldest deadline 2024-09-21T06:10:44.605381+0000) 2024-09-21T06:10:53.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:53 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:53.356+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:24.605993+0000 front 2024-09-21T06:10:24.605887+0000 (oldest deadline 2024-09-21T06:10:49.905850+0000) 2024-09-21T06:10:53.730 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 21 06:10:53 smithi192 podman[68890]: 2024-09-21 06:10:53.531054731 +0000 UTC m=+0.551852345 container died e21b14d98f7a579c7389b769a092367d7bbe228e42cf74afa3890f6a50b183b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-8-deactivate, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:10:53.776 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:53 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:53.498+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:53.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:53 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:53.498+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:53.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:53 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:53.498+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.065236+0000 front 2024-09-21T06:10:01.064969+0000 (oldest deadline 2024-09-21T06:10:26.964917+0000) 2024-09-21T06:10:53.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:53 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:53.498+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.166206+0000 front 2024-09-21T06:10:09.166235+0000 (oldest deadline 2024-09-21T06:10:33.866358+0000) 2024-09-21T06:10:53.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:53 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:53.498+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:27.368783+0000 front 2024-09-21T06:10:27.368689+0000 (oldest deadline 2024-09-21T06:10:48.969062+0000) 2024-09-21T06:10:54.105 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:53 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:53.860+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:54.105 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:53 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:53.860+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:54.105 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:53 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:53.860+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:10:54.105 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:53 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:53.860+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.482942+0000 front 2024-09-21T06:10:16.483052+0000 (oldest deadline 2024-09-21T06:10:41.182971+0000) 2024-09-21T06:10:54.105 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:53 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:53.860+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:26.384383+0000 front 2024-09-21T06:10:26.384467+0000 (oldest deadline 2024-09-21T06:10:49.884498+0000) 2024-09-21T06:10:54.121 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.8.service' 2024-09-21T06:10:54.161 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:10:54.161 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-21T06:10:54.161 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-21T06:10:54.161 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.9 2024-09-21T06:10:54.474 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:54 smithi117 systemd[1]: Stopping Ceph osd.9 for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T06:10:54.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:54 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:54.473+0000 7f65b63ce640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:10:54.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:54 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:54.473+0000 7f65b63ce640 -1 osd.9 90 *** Got signal Terminated *** 2024-09-21T06:10:54.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:54 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:54.473+0000 7f65b63ce640 -1 osd.9 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:10:54.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:54 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:54.494+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:54.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:54 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:54.494+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:54.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:54 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:54.494+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.065236+0000 front 2024-09-21T06:10:01.064969+0000 (oldest deadline 2024-09-21T06:10:26.964917+0000) 2024-09-21T06:10:54.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:54 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:54.494+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.166206+0000 front 2024-09-21T06:10:09.166235+0000 (oldest deadline 2024-09-21T06:10:33.866358+0000) 2024-09-21T06:10:54.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:54 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:54.494+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:27.368783+0000 front 2024-09-21T06:10:27.368689+0000 (oldest deadline 2024-09-21T06:10:48.969062+0000) 2024-09-21T06:10:54.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:54 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:54.366+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:54.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:54 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:54.366+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:54.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:54 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:54.366+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:54.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:54 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:54.366+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.904767+0000 front 2024-09-21T06:10:09.904947+0000 (oldest deadline 2024-09-21T06:10:34.604670+0000) 2024-09-21T06:10:54.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:54 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:54.366+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:14.604976+0000 front 2024-09-21T06:10:14.605026+0000 (oldest deadline 2024-09-21T06:10:44.605381+0000) 2024-09-21T06:10:54.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:54 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:54.366+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:24.605993+0000 front 2024-09-21T06:10:24.605887+0000 (oldest deadline 2024-09-21T06:10:49.905850+0000) 2024-09-21T06:10:55.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:54 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:54.878+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:55.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:54 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:54.878+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:55.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:54 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:54.878+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:10:55.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:54 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:54.878+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.482942+0000 front 2024-09-21T06:10:16.483052+0000 (oldest deadline 2024-09-21T06:10:41.182971+0000) 2024-09-21T06:10:55.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:54 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:54.878+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:26.384383+0000 front 2024-09-21T06:10:26.384467+0000 (oldest deadline 2024-09-21T06:10:49.884498+0000) 2024-09-21T06:10:55.776 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:55 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:55.484+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:55.776 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:55 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:55.484+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:55.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:55 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:55.485+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.065236+0000 front 2024-09-21T06:10:01.064969+0000 (oldest deadline 2024-09-21T06:10:26.964917+0000) 2024-09-21T06:10:55.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:55 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:55.485+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.166206+0000 front 2024-09-21T06:10:09.166235+0000 (oldest deadline 2024-09-21T06:10:33.866358+0000) 2024-09-21T06:10:55.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:55 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:55.485+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:27.368783+0000 front 2024-09-21T06:10:27.368689+0000 (oldest deadline 2024-09-21T06:10:48.969062+0000) 2024-09-21T06:10:55.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:55 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:55.410+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:55.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:55 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:55.410+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:55.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:55 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:55.410+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:55.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:55 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:55.410+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.904767+0000 front 2024-09-21T06:10:09.904947+0000 (oldest deadline 2024-09-21T06:10:34.604670+0000) 2024-09-21T06:10:55.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:55 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:55.410+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:14.604976+0000 front 2024-09-21T06:10:14.605026+0000 (oldest deadline 2024-09-21T06:10:44.605381+0000) 2024-09-21T06:10:55.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:55 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:55.410+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:24.605993+0000 front 2024-09-21T06:10:24.605887+0000 (oldest deadline 2024-09-21T06:10:49.905850+0000) 2024-09-21T06:10:56.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:55 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:55.919+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:56.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:55 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:55.919+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:56.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:55 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:55.919+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:10:56.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:55 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:55.919+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.482942+0000 front 2024-09-21T06:10:16.483052+0000 (oldest deadline 2024-09-21T06:10:41.182971+0000) 2024-09-21T06:10:56.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:55 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:55.919+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:26.384383+0000 front 2024-09-21T06:10:26.384467+0000 (oldest deadline 2024-09-21T06:10:49.884498+0000) 2024-09-21T06:10:56.776 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:56 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:56.448+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:56.776 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:56 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:56.448+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:56.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:56 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:56.448+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.065236+0000 front 2024-09-21T06:10:01.064969+0000 (oldest deadline 2024-09-21T06:10:26.964917+0000) 2024-09-21T06:10:56.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:56 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:56.448+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.166206+0000 front 2024-09-21T06:10:09.166235+0000 (oldest deadline 2024-09-21T06:10:33.866358+0000) 2024-09-21T06:10:56.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:56 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:56.448+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:27.368783+0000 front 2024-09-21T06:10:27.368689+0000 (oldest deadline 2024-09-21T06:10:48.969062+0000) 2024-09-21T06:10:56.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:56 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:56.409+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:56.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:56 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:56.409+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:56.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:56 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:56.409+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:56.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:56 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:56.409+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.904767+0000 front 2024-09-21T06:10:09.904947+0000 (oldest deadline 2024-09-21T06:10:34.604670+0000) 2024-09-21T06:10:56.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:56 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:56.409+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:14.604976+0000 front 2024-09-21T06:10:14.605026+0000 (oldest deadline 2024-09-21T06:10:44.605381+0000) 2024-09-21T06:10:56.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:56 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:56.409+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:24.605993+0000 front 2024-09-21T06:10:24.605887+0000 (oldest deadline 2024-09-21T06:10:49.905850+0000) 2024-09-21T06:10:57.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:56 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:56.884+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:57.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:56 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:56.884+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:57.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:56 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:56.884+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:10:57.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:56 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:56.884+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.482942+0000 front 2024-09-21T06:10:16.483052+0000 (oldest deadline 2024-09-21T06:10:41.182971+0000) 2024-09-21T06:10:57.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:56 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:56.884+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:26.384383+0000 front 2024-09-21T06:10:26.384467+0000 (oldest deadline 2024-09-21T06:10:49.884498+0000) 2024-09-21T06:10:57.776 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:57 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:57.490+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:57.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:57 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:57.490+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:57.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:57 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:57.490+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.065236+0000 front 2024-09-21T06:10:01.064969+0000 (oldest deadline 2024-09-21T06:10:26.964917+0000) 2024-09-21T06:10:57.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:57 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:57.490+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.166206+0000 front 2024-09-21T06:10:09.166235+0000 (oldest deadline 2024-09-21T06:10:33.866358+0000) 2024-09-21T06:10:57.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:57 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:57.490+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:27.368783+0000 front 2024-09-21T06:10:27.368689+0000 (oldest deadline 2024-09-21T06:10:48.969062+0000) 2024-09-21T06:10:57.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:57 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:57.490+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-09-21T06:10:33.070293+0000 front 2024-09-21T06:10:33.070322+0000 (oldest deadline 2024-09-21T06:10:56.570520+0000) 2024-09-21T06:10:57.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:57 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:57.405+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:57.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:57 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:57.405+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:57.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:57 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:57.405+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:57.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:57 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:57.405+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.904767+0000 front 2024-09-21T06:10:09.904947+0000 (oldest deadline 2024-09-21T06:10:34.604670+0000) 2024-09-21T06:10:57.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:57 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:57.405+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:14.604976+0000 front 2024-09-21T06:10:14.605026+0000 (oldest deadline 2024-09-21T06:10:44.605381+0000) 2024-09-21T06:10:57.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:57 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:57.405+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:24.605993+0000 front 2024-09-21T06:10:24.605887+0000 (oldest deadline 2024-09-21T06:10:49.905850+0000) 2024-09-21T06:10:58.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:57 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:57.851+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:58.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:57 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:57.851+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:58.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:57 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:57.851+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:10:58.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:57 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:57.851+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.482942+0000 front 2024-09-21T06:10:16.483052+0000 (oldest deadline 2024-09-21T06:10:41.182971+0000) 2024-09-21T06:10:58.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:57 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:57.851+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:26.384383+0000 front 2024-09-21T06:10:26.384467+0000 (oldest deadline 2024-09-21T06:10:49.884498+0000) 2024-09-21T06:10:58.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:58 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:58.511+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.362111+0000 front 2024-09-21T06:09:45.362078+0000 (oldest deadline 2024-09-21T06:10:06.462201+0000) 2024-09-21T06:10:58.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:58 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:58.511+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:49.963053+0000 front 2024-09-21T06:09:49.963150+0000 (oldest deadline 2024-09-21T06:10:15.863247+0000) 2024-09-21T06:10:58.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:58 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:58.511+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:01.065236+0000 front 2024-09-21T06:10:01.064969+0000 (oldest deadline 2024-09-21T06:10:26.964917+0000) 2024-09-21T06:10:58.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:58 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:58.511+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.166206+0000 front 2024-09-21T06:10:09.166235+0000 (oldest deadline 2024-09-21T06:10:33.866358+0000) 2024-09-21T06:10:58.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:58 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:58.511+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:27.368783+0000 front 2024-09-21T06:10:27.368689+0000 (oldest deadline 2024-09-21T06:10:48.969062+0000) 2024-09-21T06:10:58.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:10:58 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9[66607]: 2024-09-21T06:10:58.511+0000 7f65b29b9640 -1 osd.9 90 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-09-21T06:10:33.070293+0000 front 2024-09-21T06:10:33.070322+0000 (oldest deadline 2024-09-21T06:10:56.570520+0000) 2024-09-21T06:10:58.867 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:58 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:58.449+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:58.867 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:58 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:58.449+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:58.867 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:58 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:58.449+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:58.867 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:58 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:58.449+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.904767+0000 front 2024-09-21T06:10:09.904947+0000 (oldest deadline 2024-09-21T06:10:34.604670+0000) 2024-09-21T06:10:58.867 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:58 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:58.449+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:14.604976+0000 front 2024-09-21T06:10:14.605026+0000 (oldest deadline 2024-09-21T06:10:44.605381+0000) 2024-09-21T06:10:58.867 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:58 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:58.449+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:24.605993+0000 front 2024-09-21T06:10:24.605887+0000 (oldest deadline 2024-09-21T06:10:49.905850+0000) 2024-09-21T06:10:58.867 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:58 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:58.449+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-09-21T06:10:34.007093+0000 front 2024-09-21T06:10:34.007079+0000 (oldest deadline 2024-09-21T06:10:58.106880+0000) 2024-09-21T06:10:59.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:58 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:58.841+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:10:59.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:58 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:58.841+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:10:59.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:58 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:58.841+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:10:59.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:58 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:58.841+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.482942+0000 front 2024-09-21T06:10:16.483052+0000 (oldest deadline 2024-09-21T06:10:41.182971+0000) 2024-09-21T06:10:59.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:58 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:58.841+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:26.384383+0000 front 2024-09-21T06:10:26.384467+0000 (oldest deadline 2024-09-21T06:10:49.884498+0000) 2024-09-21T06:10:59.705 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:59 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:59.435+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:10:59.705 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:59 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:59.435+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:10:59.705 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:59 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:59.435+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:10:59.705 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:59 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:59.435+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.904767+0000 front 2024-09-21T06:10:09.904947+0000 (oldest deadline 2024-09-21T06:10:34.604670+0000) 2024-09-21T06:10:59.705 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:59 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:59.435+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:14.604976+0000 front 2024-09-21T06:10:14.605026+0000 (oldest deadline 2024-09-21T06:10:44.605381+0000) 2024-09-21T06:10:59.705 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:59 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:59.435+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:24.605993+0000 front 2024-09-21T06:10:24.605887+0000 (oldest deadline 2024-09-21T06:10:49.905850+0000) 2024-09-21T06:10:59.705 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:10:59 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:10:59.435+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-09-21T06:10:34.007093+0000 front 2024-09-21T06:10:34.007079+0000 (oldest deadline 2024-09-21T06:10:58.106880+0000) 2024-09-21T06:11:00.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:59 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:59.828+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:11:00.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:59 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:59.828+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:11:00.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:59 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:59.828+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:11:00.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:59 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:59.828+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.482942+0000 front 2024-09-21T06:10:16.483052+0000 (oldest deadline 2024-09-21T06:10:41.182971+0000) 2024-09-21T06:11:00.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:10:59 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:10:59.828+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:26.384383+0000 front 2024-09-21T06:10:26.384467+0000 (oldest deadline 2024-09-21T06:10:49.884498+0000) 2024-09-21T06:11:00.357 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:11:00 smithi117 podman[107045]: 2024-09-21 06:11:00.068309217 +0000 UTC m=+5.715782195 container died bb1a36e66a814e1db26e640d64d48d3b18964bf704ee903251ec37fc07f2eb44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:11:00.776 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:11:00 smithi117 podman[107045]: 2024-09-21 06:11:00.57307026 +0000 UTC m=+6.220543236 container cleanup bb1a36e66a814e1db26e640d64d48d3b18964bf704ee903251ec37fc07f2eb44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-21T06:11:00.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:11:00 smithi117 bash[107045]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9 2024-09-21T06:11:00.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:00 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:00.434+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:11:00.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:00 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:00.434+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:11:00.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:00 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:00.434+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:11:00.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:00 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:00.434+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.904767+0000 front 2024-09-21T06:10:09.904947+0000 (oldest deadline 2024-09-21T06:10:34.604670+0000) 2024-09-21T06:11:00.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:00 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:00.434+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:14.604976+0000 front 2024-09-21T06:10:14.605026+0000 (oldest deadline 2024-09-21T06:10:44.605381+0000) 2024-09-21T06:11:00.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:00 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:00.434+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:24.605993+0000 front 2024-09-21T06:10:24.605887+0000 (oldest deadline 2024-09-21T06:10:49.905850+0000) 2024-09-21T06:11:00.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:00 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:00.434+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-09-21T06:10:34.007093+0000 front 2024-09-21T06:10:34.007079+0000 (oldest deadline 2024-09-21T06:10:58.106880+0000) 2024-09-21T06:11:01.134 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:11:00 smithi117 podman[107060]: 2024-09-21 06:11:00.825540451 +0000 UTC m=+0.754228278 container remove bb1a36e66a814e1db26e640d64d48d3b18964bf704ee903251ec37fc07f2eb44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:11:01.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:00 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:00.818+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:11:01.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:00 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:00.818+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:11:01.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:00 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:00.818+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:11:01.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:00 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:00.818+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.482942+0000 front 2024-09-21T06:10:16.483052+0000 (oldest deadline 2024-09-21T06:10:41.182971+0000) 2024-09-21T06:11:01.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:00 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:00.818+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:26.384383+0000 front 2024-09-21T06:10:26.384467+0000 (oldest deadline 2024-09-21T06:10:49.884498+0000) 2024-09-21T06:11:01.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:00 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:00.818+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-09-21T06:10:33.985520+0000 front 2024-09-21T06:10:33.985509+0000 (oldest deadline 2024-09-21T06:10:59.885275+0000) 2024-09-21T06:11:01.429 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:11:01 smithi117 podman[107151]: 2024-09-21 06:11:01.036150869 +0000 UTC m=+0.015615436 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:11:01.429 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:11:01 smithi117 podman[107151]: 2024-09-21 06:11:01.133591541 +0000 UTC m=+0.113056100 container create e3d42944d691ca680f2df4c1aae1fff01ed5f030d11eba861edfba1468176e54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-21T06:11:01.429 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:11:01 smithi117 podman[107151]: 2024-09-21 06:11:01.344450085 +0000 UTC m=+0.323914653 container init e3d42944d691ca680f2df4c1aae1fff01ed5f030d11eba861edfba1468176e54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-21T06:11:01.429 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:11:01 smithi117 podman[107151]: 2024-09-21 06:11:01.347730453 +0000 UTC m=+0.327195010 container start e3d42944d691ca680f2df4c1aae1fff01ed5f030d11eba861edfba1468176e54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True) 2024-09-21T06:11:01.429 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:11:01 smithi117 podman[107151]: 2024-09-21 06:11:01.427689352 +0000 UTC m=+0.407153926 container attach e3d42944d691ca680f2df4c1aae1fff01ed5f030d11eba861edfba1468176e54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-09-21T06:11:01.777 INFO:journalctl@ceph.osd.9.smithi117.stdout:Sep 21 06:11:01 smithi117 podman[107151]: 2024-09-21 06:11:01.538263156 +0000 UTC m=+0.517727727 container died e3d42944d691ca680f2df4c1aae1fff01ed5f030d11eba861edfba1468176e54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-9-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T06:11:01.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:01 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:01.420+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:11:01.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:01 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:01.420+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:11:01.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:01 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:01.420+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:11:01.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:01 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:01.420+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.904767+0000 front 2024-09-21T06:10:09.904947+0000 (oldest deadline 2024-09-21T06:10:34.604670+0000) 2024-09-21T06:11:01.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:01 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:01.420+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:14.604976+0000 front 2024-09-21T06:10:14.605026+0000 (oldest deadline 2024-09-21T06:10:44.605381+0000) 2024-09-21T06:11:01.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:01 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:01.420+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:24.605993+0000 front 2024-09-21T06:10:24.605887+0000 (oldest deadline 2024-09-21T06:10:49.905850+0000) 2024-09-21T06:11:01.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:01 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:01.420+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-09-21T06:10:34.007093+0000 front 2024-09-21T06:10:34.007079+0000 (oldest deadline 2024-09-21T06:10:58.106880+0000) 2024-09-21T06:11:02.116 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.9.service' 2024-09-21T06:11:02.156 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:11:02.156 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-21T06:11:02.156 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-21T06:11:02.156 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.10 2024-09-21T06:11:02.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:01 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:01.811+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:11:02.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:01 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:01.811+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:11:02.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:01 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:01.811+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:11:02.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:01 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:01.811+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.482942+0000 front 2024-09-21T06:10:16.483052+0000 (oldest deadline 2024-09-21T06:10:41.182971+0000) 2024-09-21T06:11:02.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:01 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:01.811+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:26.384383+0000 front 2024-09-21T06:10:26.384467+0000 (oldest deadline 2024-09-21T06:10:49.884498+0000) 2024-09-21T06:11:02.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:01 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:01.811+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-09-21T06:10:33.985520+0000 front 2024-09-21T06:10:33.985509+0000 (oldest deadline 2024-09-21T06:10:59.885275+0000) 2024-09-21T06:11:02.459 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:02 smithi138 systemd[1]: Stopping Ceph osd.10 for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T06:11:02.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:02 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:02.457+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:11:02.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:02 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:02.457+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:11:02.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:02 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:02.457+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:11:02.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:02 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:02.457+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.904767+0000 front 2024-09-21T06:10:09.904947+0000 (oldest deadline 2024-09-21T06:10:34.604670+0000) 2024-09-21T06:11:02.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:02 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:02.457+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:14.604976+0000 front 2024-09-21T06:10:14.605026+0000 (oldest deadline 2024-09-21T06:10:44.605381+0000) 2024-09-21T06:11:02.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:02 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:02.457+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:24.605993+0000 front 2024-09-21T06:10:24.605887+0000 (oldest deadline 2024-09-21T06:10:49.905850+0000) 2024-09-21T06:11:02.866 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:02 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:02.457+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-09-21T06:10:34.007093+0000 front 2024-09-21T06:10:34.007079+0000 (oldest deadline 2024-09-21T06:10:58.106880+0000) 2024-09-21T06:11:02.866 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:02 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:02.478+0000 7fab54287640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:11:02.866 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:02 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:02.478+0000 7fab54287640 -1 osd.10 90 *** Got signal Terminated *** 2024-09-21T06:11:02.866 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:02 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:02.478+0000 7fab54287640 -1 osd.10 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:11:03.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:02 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:02.789+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:11:03.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:02 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:02.789+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:11:03.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:02 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:02.789+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:11:03.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:02 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:02.789+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.482942+0000 front 2024-09-21T06:10:16.483052+0000 (oldest deadline 2024-09-21T06:10:41.182971+0000) 2024-09-21T06:11:03.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:02 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:02.789+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:26.384383+0000 front 2024-09-21T06:10:26.384467+0000 (oldest deadline 2024-09-21T06:10:49.884498+0000) 2024-09-21T06:11:03.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:02 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:02.789+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-09-21T06:10:33.985520+0000 front 2024-09-21T06:10:33.985509+0000 (oldest deadline 2024-09-21T06:10:59.885275+0000) 2024-09-21T06:11:03.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:03 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:03.437+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:11:03.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:03 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:03.437+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:11:03.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:03 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:03.437+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:11:03.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:03 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:03.437+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.904767+0000 front 2024-09-21T06:10:09.904947+0000 (oldest deadline 2024-09-21T06:10:34.604670+0000) 2024-09-21T06:11:03.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:03 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:03.437+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:14.604976+0000 front 2024-09-21T06:10:14.605026+0000 (oldest deadline 2024-09-21T06:10:44.605381+0000) 2024-09-21T06:11:03.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:03 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:03.437+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:24.605993+0000 front 2024-09-21T06:10:24.605887+0000 (oldest deadline 2024-09-21T06:10:49.905850+0000) 2024-09-21T06:11:03.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:03 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:03.437+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-09-21T06:10:34.007093+0000 front 2024-09-21T06:10:34.007079+0000 (oldest deadline 2024-09-21T06:10:58.106880+0000) 2024-09-21T06:11:04.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:03 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:03.824+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:11:04.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:03 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:03.824+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:11:04.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:03 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:03.824+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:11:04.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:03 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:03.824+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.482942+0000 front 2024-09-21T06:10:16.483052+0000 (oldest deadline 2024-09-21T06:10:41.182971+0000) 2024-09-21T06:11:04.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:03 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:03.824+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:26.384383+0000 front 2024-09-21T06:10:26.384467+0000 (oldest deadline 2024-09-21T06:10:49.884498+0000) 2024-09-21T06:11:04.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:03 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:03.824+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-09-21T06:10:33.985520+0000 front 2024-09-21T06:10:33.985509+0000 (oldest deadline 2024-09-21T06:10:59.885275+0000) 2024-09-21T06:11:04.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:04 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:04.447+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:11:04.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:04 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:04.447+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:11:04.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:04 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:04.447+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:11:04.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:04 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:04.447+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.904767+0000 front 2024-09-21T06:10:09.904947+0000 (oldest deadline 2024-09-21T06:10:34.604670+0000) 2024-09-21T06:11:04.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:04 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:04.447+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:14.604976+0000 front 2024-09-21T06:10:14.605026+0000 (oldest deadline 2024-09-21T06:10:44.605381+0000) 2024-09-21T06:11:04.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:04 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:04.447+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:24.605993+0000 front 2024-09-21T06:10:24.605887+0000 (oldest deadline 2024-09-21T06:10:49.905850+0000) 2024-09-21T06:11:04.866 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:04 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:04.447+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-09-21T06:10:34.007093+0000 front 2024-09-21T06:10:34.007079+0000 (oldest deadline 2024-09-21T06:10:58.106880+0000) 2024-09-21T06:11:05.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:04 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:04.867+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:11:05.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:04 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:04.867+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:11:05.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:04 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:04.867+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:11:05.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:04 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:04.867+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.482942+0000 front 2024-09-21T06:10:16.483052+0000 (oldest deadline 2024-09-21T06:10:41.182971+0000) 2024-09-21T06:11:05.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:04 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:04.867+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:26.384383+0000 front 2024-09-21T06:10:26.384467+0000 (oldest deadline 2024-09-21T06:10:49.884498+0000) 2024-09-21T06:11:05.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:04 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:04.867+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-09-21T06:10:33.985520+0000 front 2024-09-21T06:10:33.985509+0000 (oldest deadline 2024-09-21T06:10:59.885275+0000) 2024-09-21T06:11:05.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:05 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:05.444+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:11:05.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:05 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:05.444+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:11:05.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:05 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:05.444+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:11:05.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:05 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:05.444+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.904767+0000 front 2024-09-21T06:10:09.904947+0000 (oldest deadline 2024-09-21T06:10:34.604670+0000) 2024-09-21T06:11:05.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:05 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:05.444+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:14.604976+0000 front 2024-09-21T06:10:14.605026+0000 (oldest deadline 2024-09-21T06:10:44.605381+0000) 2024-09-21T06:11:05.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:05 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:05.444+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:24.605993+0000 front 2024-09-21T06:10:24.605887+0000 (oldest deadline 2024-09-21T06:10:49.905850+0000) 2024-09-21T06:11:05.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:05 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:05.444+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-09-21T06:10:34.007093+0000 front 2024-09-21T06:10:34.007079+0000 (oldest deadline 2024-09-21T06:10:58.106880+0000) 2024-09-21T06:11:06.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:05 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:05.872+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:11:06.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:05 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:05.872+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:11:06.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:05 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:05.872+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:11:06.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:05 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:05.872+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.482942+0000 front 2024-09-21T06:10:16.483052+0000 (oldest deadline 2024-09-21T06:10:41.182971+0000) 2024-09-21T06:11:06.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:05 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:05.872+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:26.384383+0000 front 2024-09-21T06:10:26.384467+0000 (oldest deadline 2024-09-21T06:10:49.884498+0000) 2024-09-21T06:11:06.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:05 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:05.872+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-09-21T06:10:33.985520+0000 front 2024-09-21T06:10:33.985509+0000 (oldest deadline 2024-09-21T06:10:59.885275+0000) 2024-09-21T06:11:06.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:06 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:06.470+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:11:06.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:06 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:06.470+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:11:06.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:06 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:06.470+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:11:06.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:06 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:06.470+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.904767+0000 front 2024-09-21T06:10:09.904947+0000 (oldest deadline 2024-09-21T06:10:34.604670+0000) 2024-09-21T06:11:06.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:06 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:06.470+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:14.604976+0000 front 2024-09-21T06:10:14.605026+0000 (oldest deadline 2024-09-21T06:10:44.605381+0000) 2024-09-21T06:11:06.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:06 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:06.470+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:24.605993+0000 front 2024-09-21T06:10:24.605887+0000 (oldest deadline 2024-09-21T06:10:49.905850+0000) 2024-09-21T06:11:06.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:06 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:06.470+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-09-21T06:10:34.007093+0000 front 2024-09-21T06:10:34.007079+0000 (oldest deadline 2024-09-21T06:10:58.106880+0000) 2024-09-21T06:11:07.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:06 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:06.912+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:11:07.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:06 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:06.912+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:11:07.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:06 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:06.912+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:11:07.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:06 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:06.912+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.482942+0000 front 2024-09-21T06:10:16.483052+0000 (oldest deadline 2024-09-21T06:10:41.182971+0000) 2024-09-21T06:11:07.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:06 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:06.912+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:26.384383+0000 front 2024-09-21T06:10:26.384467+0000 (oldest deadline 2024-09-21T06:10:49.884498+0000) 2024-09-21T06:11:07.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:06 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:06.912+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-09-21T06:10:33.985520+0000 front 2024-09-21T06:10:33.985509+0000 (oldest deadline 2024-09-21T06:10:59.885275+0000) 2024-09-21T06:11:07.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:07 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:07.467+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:45.301262+0000 front 2024-09-21T06:09:45.301456+0000 (oldest deadline 2024-09-21T06:10:09.401255+0000) 2024-09-21T06:11:07.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:07 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:07.467+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:54.102114+0000 front 2024-09-21T06:09:54.102065+0000 (oldest deadline 2024-09-21T06:10:17.602229+0000) 2024-09-21T06:11:07.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:07 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:07.467+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:09:57.602690+0000 front 2024-09-21T06:09:57.602747+0000 (oldest deadline 2024-09-21T06:10:24.003416+0000) 2024-09-21T06:11:07.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:07 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:07.467+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6814 osd.3 since back 2024-09-21T06:10:09.904767+0000 front 2024-09-21T06:10:09.904947+0000 (oldest deadline 2024-09-21T06:10:34.604670+0000) 2024-09-21T06:11:07.866 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:07 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:07.467+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:14.604976+0000 front 2024-09-21T06:10:14.605026+0000 (oldest deadline 2024-09-21T06:10:44.605381+0000) 2024-09-21T06:11:07.866 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:07 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:07.467+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:24.605993+0000 front 2024-09-21T06:10:24.605887+0000 (oldest deadline 2024-09-21T06:10:49.905850+0000) 2024-09-21T06:11:07.866 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:07 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10[58605]: 2024-09-21T06:11:07.467+0000 7fab4c86a640 -1 osd.10 90 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-09-21T06:10:34.007093+0000 front 2024-09-21T06:10:34.007079+0000 (oldest deadline 2024-09-21T06:10:58.106880+0000) 2024-09-21T06:11:08.229 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:07 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:07.897+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:11:08.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:07 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:07.897+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:11:08.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:07 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:07.897+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:11:08.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:07 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:07.897+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.482942+0000 front 2024-09-21T06:10:16.483052+0000 (oldest deadline 2024-09-21T06:10:41.182971+0000) 2024-09-21T06:11:08.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:07 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:07.897+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:26.384383+0000 front 2024-09-21T06:10:26.384467+0000 (oldest deadline 2024-09-21T06:10:49.884498+0000) 2024-09-21T06:11:08.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:07 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:07.897+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-09-21T06:10:33.985520+0000 front 2024-09-21T06:10:33.985509+0000 (oldest deadline 2024-09-21T06:10:59.885275+0000) 2024-09-21T06:11:08.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:07 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:07.897+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6820 osd.7 since back 2024-09-21T06:10:39.885707+0000 front 2024-09-21T06:10:39.885726+0000 (oldest deadline 2024-09-21T06:11:07.486212+0000) 2024-09-21T06:11:08.306 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:08 smithi138 podman[68126]: 2024-09-21 06:11:08.006335151 +0000 UTC m=+5.650180961 container died 1668a77801934f92644853d6b59778a2008264795a627898463037f96f455f37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731) 2024-09-21T06:11:08.605 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:08 smithi138 podman[68126]: 2024-09-21 06:11:08.403265373 +0000 UTC m=+6.047111180 container cleanup 1668a77801934f92644853d6b59778a2008264795a627898463037f96f455f37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-21T06:11:08.606 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:08 smithi138 bash[68126]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10 2024-09-21T06:11:08.865 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:08 smithi138 podman[68141]: 2024-09-21 06:11:08.604566872 +0000 UTC m=+0.594771444 container remove 1668a77801934f92644853d6b59778a2008264795a627898463037f96f455f37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-21T06:11:09.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:08 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:08.929+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:11:09.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:08 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:08.929+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:11:09.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:08 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:08.929+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:11:09.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:08 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:08.929+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.482942+0000 front 2024-09-21T06:10:16.483052+0000 (oldest deadline 2024-09-21T06:10:41.182971+0000) 2024-09-21T06:11:09.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:08 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:08.929+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:26.384383+0000 front 2024-09-21T06:10:26.384467+0000 (oldest deadline 2024-09-21T06:10:49.884498+0000) 2024-09-21T06:11:09.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:08 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:08.929+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-09-21T06:10:33.985520+0000 front 2024-09-21T06:10:33.985509+0000 (oldest deadline 2024-09-21T06:10:59.885275+0000) 2024-09-21T06:11:09.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:08 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:08.929+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6820 osd.7 since back 2024-09-21T06:10:39.885707+0000 front 2024-09-21T06:10:39.885726+0000 (oldest deadline 2024-09-21T06:11:07.486212+0000) 2024-09-21T06:11:09.309 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:08 smithi138 podman[68231]: 2024-09-21 06:11:08.997490279 +0000 UTC m=+0.108495751 container create 696cc5c45129ab1e408dda614d3af3ed67bab40590b917f33a9a52b7d044d8db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True) 2024-09-21T06:11:09.309 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:09 smithi138 podman[68231]: 2024-09-21 06:11:08.904813859 +0000 UTC m=+0.015819338 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:11:09.310 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:09 smithi138 podman[68231]: 2024-09-21 06:11:09.225175355 +0000 UTC m=+0.336180832 container init 696cc5c45129ab1e408dda614d3af3ed67bab40590b917f33a9a52b7d044d8db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:11:09.310 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:09 smithi138 podman[68231]: 2024-09-21 06:11:09.228622844 +0000 UTC m=+0.339628324 container start 696cc5c45129ab1e408dda614d3af3ed67bab40590b917f33a9a52b7d044d8db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=) 2024-09-21T06:11:09.310 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:09 smithi138 podman[68231]: 2024-09-21 06:11:09.30857548 +0000 UTC m=+0.419580952 container attach 696cc5c45129ab1e408dda614d3af3ed67bab40590b917f33a9a52b7d044d8db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-21T06:11:09.615 INFO:journalctl@ceph.osd.10.smithi138.stdout:Sep 21 06:11:09 smithi138 podman[68231]: 2024-09-21 06:11:09.414652607 +0000 UTC m=+0.525658080 container died 696cc5c45129ab1e408dda614d3af3ed67bab40590b917f33a9a52b7d044d8db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-10-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-21T06:11:10.029 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.10.service' 2024-09-21T06:11:10.069 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:11:10.070 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-21T06:11:10.070 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-21T06:11:10.070 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.11 2024-09-21T06:11:10.229 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:09 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:09.902+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:11:10.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:09 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:09.902+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:11:10.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:09 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:09.902+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:11:10.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:09 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:09.902+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.482942+0000 front 2024-09-21T06:10:16.483052+0000 (oldest deadline 2024-09-21T06:10:41.182971+0000) 2024-09-21T06:11:10.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:09 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:09.902+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:26.384383+0000 front 2024-09-21T06:10:26.384467+0000 (oldest deadline 2024-09-21T06:10:49.884498+0000) 2024-09-21T06:11:10.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:09 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:09.902+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-09-21T06:10:33.985520+0000 front 2024-09-21T06:10:33.985509+0000 (oldest deadline 2024-09-21T06:10:59.885275+0000) 2024-09-21T06:11:10.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:09 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:09.902+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6820 osd.7 since back 2024-09-21T06:10:39.885707+0000 front 2024-09-21T06:10:39.885726+0000 (oldest deadline 2024-09-21T06:11:07.486212+0000) 2024-09-21T06:11:10.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:10 smithi192 systemd[1]: Stopping Ceph osd.11 for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T06:11:10.729 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:10 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:10.356+0000 7f5545b15640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:11:10.729 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:10 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:10.356+0000 7f5545b15640 -1 osd.11 90 *** Got signal Terminated *** 2024-09-21T06:11:10.730 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:10 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:10.356+0000 7f5545b15640 -1 osd.11 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T06:11:11.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:10 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:10.910+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:11:11.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:10 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:10.910+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:11:11.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:10 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:10.910+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:11:11.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:10 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:10.910+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.482942+0000 front 2024-09-21T06:10:16.483052+0000 (oldest deadline 2024-09-21T06:10:41.182971+0000) 2024-09-21T06:11:11.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:10 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:10.910+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:26.384383+0000 front 2024-09-21T06:10:26.384467+0000 (oldest deadline 2024-09-21T06:10:49.884498+0000) 2024-09-21T06:11:11.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:10 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:10.910+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-09-21T06:10:33.985520+0000 front 2024-09-21T06:10:33.985509+0000 (oldest deadline 2024-09-21T06:10:59.885275+0000) 2024-09-21T06:11:11.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:10 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:10.910+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6820 osd.7 since back 2024-09-21T06:10:39.885707+0000 front 2024-09-21T06:10:39.885726+0000 (oldest deadline 2024-09-21T06:11:07.486212+0000) 2024-09-21T06:11:12.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:11 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:11.872+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:11:12.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:11 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:11.872+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:11:12.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:11 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:11.872+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:11:12.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:11 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:11.872+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.482942+0000 front 2024-09-21T06:10:16.483052+0000 (oldest deadline 2024-09-21T06:10:41.182971+0000) 2024-09-21T06:11:12.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:11 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:11.872+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:26.384383+0000 front 2024-09-21T06:10:26.384467+0000 (oldest deadline 2024-09-21T06:10:49.884498+0000) 2024-09-21T06:11:12.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:11 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:11.872+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-09-21T06:10:33.985520+0000 front 2024-09-21T06:10:33.985509+0000 (oldest deadline 2024-09-21T06:10:59.885275+0000) 2024-09-21T06:11:12.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:11 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:11.872+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6820 osd.7 since back 2024-09-21T06:10:39.885707+0000 front 2024-09-21T06:10:39.885726+0000 (oldest deadline 2024-09-21T06:11:07.486212+0000) 2024-09-21T06:11:13.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:12 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:12.845+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:11:13.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:12 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:12.845+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:11:13.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:12 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:12.845+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:11:13.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:12 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:12.845+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.482942+0000 front 2024-09-21T06:10:16.483052+0000 (oldest deadline 2024-09-21T06:10:41.182971+0000) 2024-09-21T06:11:13.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:12 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:12.845+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:26.384383+0000 front 2024-09-21T06:10:26.384467+0000 (oldest deadline 2024-09-21T06:10:49.884498+0000) 2024-09-21T06:11:13.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:12 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:12.845+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-09-21T06:10:33.985520+0000 front 2024-09-21T06:10:33.985509+0000 (oldest deadline 2024-09-21T06:10:59.885275+0000) 2024-09-21T06:11:13.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:12 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:12.845+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6820 osd.7 since back 2024-09-21T06:10:39.885707+0000 front 2024-09-21T06:10:39.885726+0000 (oldest deadline 2024-09-21T06:11:07.486212+0000) 2024-09-21T06:11:14.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:13 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:13.827+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:11:14.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:13 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:13.827+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:11:14.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:13 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:13.827+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:11:14.231 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:13 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:13.827+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.482942+0000 front 2024-09-21T06:10:16.483052+0000 (oldest deadline 2024-09-21T06:10:41.182971+0000) 2024-09-21T06:11:14.231 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:13 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:13.827+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:26.384383+0000 front 2024-09-21T06:10:26.384467+0000 (oldest deadline 2024-09-21T06:10:49.884498+0000) 2024-09-21T06:11:14.231 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:13 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:13.827+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-09-21T06:10:33.985520+0000 front 2024-09-21T06:10:33.985509+0000 (oldest deadline 2024-09-21T06:10:59.885275+0000) 2024-09-21T06:11:14.231 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:13 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:13.827+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6820 osd.7 since back 2024-09-21T06:10:39.885707+0000 front 2024-09-21T06:10:39.885726+0000 (oldest deadline 2024-09-21T06:11:07.486212+0000) 2024-09-21T06:11:14.231 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:13 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:13.827+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6820 osd.8 since back 2024-09-21T06:10:50.986996+0000 front 2024-09-21T06:10:50.987060+0000 (oldest deadline 2024-09-21T06:11:13.286967+0000) 2024-09-21T06:11:15.229 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:14 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:14.847+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-09-21T06:09:41.477771+0000 front 2024-09-21T06:09:41.477885+0000 (oldest deadline 2024-09-21T06:10:07.377757+0000) 2024-09-21T06:11:15.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:14 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:14.847+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6804 osd.1 since back 2024-09-21T06:09:53.278984+0000 front 2024-09-21T06:09:53.279091+0000 (oldest deadline 2024-09-21T06:10:20.279570+0000) 2024-09-21T06:11:15.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:14 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:14.847+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-21T06:10:00.280017+0000 front 2024-09-21T06:10:00.280056+0000 (oldest deadline 2024-09-21T06:10:24.380126+0000) 2024-09-21T06:11:15.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:14 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:14.847+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6812 osd.4 since back 2024-09-21T06:10:16.482942+0000 front 2024-09-21T06:10:16.483052+0000 (oldest deadline 2024-09-21T06:10:41.182971+0000) 2024-09-21T06:11:15.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:14 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:14.847+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-21T06:10:26.384383+0000 front 2024-09-21T06:10:26.384467+0000 (oldest deadline 2024-09-21T06:10:49.884498+0000) 2024-09-21T06:11:15.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:14 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:14.847+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.117:6822 osd.6 since back 2024-09-21T06:10:33.985520+0000 front 2024-09-21T06:10:33.985509+0000 (oldest deadline 2024-09-21T06:10:59.885275+0000) 2024-09-21T06:11:15.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:14 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:14.847+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.138:6820 osd.7 since back 2024-09-21T06:10:39.885707+0000 front 2024-09-21T06:10:39.885726+0000 (oldest deadline 2024-09-21T06:11:07.486212+0000) 2024-09-21T06:11:15.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:14 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11[59034]: 2024-09-21T06:11:14.847+0000 7f5542100640 -1 osd.11 90 heartbeat_check: no reply from 172.21.15.192:6820 osd.8 since back 2024-09-21T06:10:50.986996+0000 front 2024-09-21T06:10:50.987060+0000 (oldest deadline 2024-09-21T06:11:13.286967+0000) 2024-09-21T06:11:16.230 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:15 smithi192 podman[69008]: 2024-09-21 06:11:15.902523312 +0000 UTC m=+5.641481471 container died 9c7f7ae1439333e9026602bc056ef65f97b8b0db355e9987473d14c7f9fe0bf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=) 2024-09-21T06:11:16.730 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:16 smithi192 podman[69008]: 2024-09-21 06:11:16.230436207 +0000 UTC m=+5.969394364 container cleanup 9c7f7ae1439333e9026602bc056ef65f97b8b0db355e9987473d14c7f9fe0bf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-21T06:11:16.730 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:16 smithi192 bash[69008]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11 2024-09-21T06:11:16.730 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:16 smithi192 podman[69025]: 2024-09-21 06:11:16.440009139 +0000 UTC m=+0.535611780 container remove 9c7f7ae1439333e9026602bc056ef65f97b8b0db355e9987473d14c7f9fe0bf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:11:17.144 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:16 smithi192 podman[69112]: 2024-09-21 06:11:16.841155639 +0000 UTC m=+0.110640337 container create 2e3f331062dce78f2c93aee9561d5a802670ac057589c03fbc9d2c013c8ca204 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-21T06:11:17.144 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:16 smithi192 podman[69112]: 2024-09-21 06:11:16.746194538 +0000 UTC m=+0.015679222 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T06:11:17.144 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:17 smithi192 podman[69112]: 2024-09-21 06:11:17.068812432 +0000 UTC m=+0.338297092 container init 2e3f331062dce78f2c93aee9561d5a802670ac057589c03fbc9d2c013c8ca204 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-21T06:11:17.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:17 smithi192 podman[69112]: 2024-09-21 06:11:17.072001465 +0000 UTC m=+0.341486127 container start 2e3f331062dce78f2c93aee9561d5a802670ac057589c03fbc9d2c013c8ca204 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:11:17.145 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:17 smithi192 podman[69112]: 2024-09-21 06:11:17.143369283 +0000 UTC m=+0.412853936 container attach 2e3f331062dce78f2c93aee9561d5a802670ac057589c03fbc9d2c013c8ca204 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-09-21T06:11:17.479 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:17 smithi192 podman[69112]: 2024-09-21 06:11:17.266301155 +0000 UTC m=+0.535785813 container died 2e3f331062dce78f2c93aee9561d5a802670ac057589c03fbc9d2c013c8ca204 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:11:17.795 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.11.service' 2024-09-21T06:11:17.818 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:17 smithi192 podman[69135]: 2024-09-21 06:11:17.77990062 +0000 UTC m=+0.508128461 container remove 2e3f331062dce78f2c93aee9561d5a802670ac057589c03fbc9d2c013c8ca204 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:11:17.818 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:17 smithi192 systemd[1]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.11.service: Deactivated successfully. 2024-09-21T06:11:17.818 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:17 smithi192 systemd[1]: Stopped Ceph osd.11 for ac8581a2-77dc-11ef-baf5-efdc52797490. 2024-09-21T06:11:17.818 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 21 06:11:17 smithi192 systemd[1]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@osd.11.service: Consumed 5min 45.658s CPU time. 2024-09-21T06:11:17.833 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:11:17.833 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-21T06:11:17.834 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-21T06:11:17.834 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.a 2024-09-21T06:11:18.142 INFO:journalctl@ceph.mds.a.smithi117.stdout:Sep 21 06:11:17 smithi117 systemd[1]: Stopping Ceph mds.a for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T06:11:18.526 INFO:journalctl@ceph.mds.a.smithi117.stdout:Sep 21 06:11:18 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-a[73344]: 2024-09-21T06:11:18.140+0000 7f584722f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:11:18.526 INFO:journalctl@ceph.mds.a.smithi117.stdout:Sep 21 06:11:18 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-a[73344]: 2024-09-21T06:11:18.140+0000 7f584722f640 -1 mds.a *** got signal Terminated *** 2024-09-21T06:11:21.688 INFO:journalctl@ceph.mds.a.smithi117.stdout:Sep 21 06:11:21 smithi117 podman[107266]: 2024-09-21 06:11:21.368302481 +0000 UTC m=+3.333281244 container stop 5c3fa963f450cd44f1ec6fda57ec5b020bfbea35433dec61991d7b2eb048807f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-21T06:11:21.688 INFO:journalctl@ceph.mds.a.smithi117.stdout:Sep 21 06:11:21 smithi117 podman[107266]: 2024-09-21 06:11:21.393956697 +0000 UTC m=+3.358935458 container died 5c3fa963f450cd44f1ec6fda57ec5b020bfbea35433dec61991d7b2eb048807f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-a, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-21T06:11:22.027 INFO:journalctl@ceph.mds.a.smithi117.stdout:Sep 21 06:11:21 smithi117 podman[107266]: 2024-09-21 06:11:21.906602247 +0000 UTC m=+3.871581053 container cleanup 5c3fa963f450cd44f1ec6fda57ec5b020bfbea35433dec61991d7b2eb048807f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-a, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-21T06:11:22.027 INFO:journalctl@ceph.mds.a.smithi117.stdout:Sep 21 06:11:21 smithi117 bash[107266]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-a 2024-09-21T06:11:22.399 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.a.service' 2024-09-21T06:11:22.421 INFO:journalctl@ceph.mds.a.smithi117.stdout:Sep 21 06:11:22 smithi117 podman[107279]: 2024-09-21 06:11:22.149908274 +0000 UTC m=+0.773774050 container remove 5c3fa963f450cd44f1ec6fda57ec5b020bfbea35433dec61991d7b2eb048807f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-a, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-21T06:11:22.421 INFO:journalctl@ceph.mds.a.smithi117.stdout:Sep 21 06:11:22 smithi117 systemd[1]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.a.service: Deactivated successfully. 2024-09-21T06:11:22.421 INFO:journalctl@ceph.mds.a.smithi117.stdout:Sep 21 06:11:22 smithi117 systemd[1]: Stopped Ceph mds.a for ac8581a2-77dc-11ef-baf5-efdc52797490. 2024-09-21T06:11:22.421 INFO:journalctl@ceph.mds.a.smithi117.stdout:Sep 21 06:11:22 smithi117 systemd[1]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.a.service: Consumed 2.395s CPU time. 2024-09-21T06:11:22.434 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:11:22.434 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-21T06:11:22.434 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-21T06:11:22.434 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.d 2024-09-21T06:11:22.770 INFO:journalctl@ceph.mds.d.smithi117.stdout:Sep 21 06:11:22 smithi117 systemd[1]: Stopping Ceph mds.d for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T06:11:23.026 INFO:journalctl@ceph.mds.d.smithi117.stdout:Sep 21 06:11:22 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-d[72226]: 2024-09-21T06:11:22.767+0000 7f59811ce640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:11:23.027 INFO:journalctl@ceph.mds.d.smithi117.stdout:Sep 21 06:11:22 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-d[72226]: 2024-09-21T06:11:22.767+0000 7f59811ce640 -1 mds.d *** got signal Terminated *** 2024-09-21T06:11:25.895 INFO:journalctl@ceph.mds.d.smithi117.stdout:Sep 21 06:11:25 smithi117 podman[107388]: 2024-09-21 06:11:25.586748333 +0000 UTC m=+2.938690733 container died a6a2bad078691008151e7720d9bb2b1869e819633d7a7d1d9699b4cc7557ca5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-d, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-21T06:11:26.276 INFO:journalctl@ceph.mds.d.smithi117.stdout:Sep 21 06:11:26 smithi117 podman[107388]: 2024-09-21 06:11:26.068659538 +0000 UTC m=+3.420601939 container cleanup a6a2bad078691008151e7720d9bb2b1869e819633d7a7d1d9699b4cc7557ca5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-d, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:11:26.277 INFO:journalctl@ceph.mds.d.smithi117.stdout:Sep 21 06:11:26 smithi117 bash[107388]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-d 2024-09-21T06:11:26.627 INFO:journalctl@ceph.mds.d.smithi117.stdout:Sep 21 06:11:26 smithi117 podman[107402]: 2024-09-21 06:11:26.345462254 +0000 UTC m=+0.745310547 container remove a6a2bad078691008151e7720d9bb2b1869e819633d7a7d1d9699b4cc7557ca5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-d, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-21T06:11:26.641 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.d.service' 2024-09-21T06:11:26.674 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:11:26.674 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-21T06:11:26.674 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-21T06:11:26.674 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.g 2024-09-21T06:11:26.999 INFO:journalctl@ceph.mds.g.smithi117.stdout:Sep 21 06:11:26 smithi117 systemd[1]: Stopping Ceph mds.g for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T06:11:27.277 INFO:journalctl@ceph.mds.g.smithi117.stdout:Sep 21 06:11:26 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-g[72959]: 2024-09-21T06:11:26.996+0000 7f6042d4c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:11:27.277 INFO:journalctl@ceph.mds.g.smithi117.stdout:Sep 21 06:11:26 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-g[72959]: 2024-09-21T06:11:26.996+0000 7f6042d4c640 -1 mds.g *** got signal Terminated *** 2024-09-21T06:11:29.608 INFO:journalctl@ceph.mds.g.smithi117.stdout:Sep 21 06:11:29 smithi117 podman[107509]: 2024-09-21 06:11:29.352056407 +0000 UTC m=+2.460233956 container died ad0a6a2f1e72a3a0cd0aebaa7886f332975bf9f0a19f4c144ea78fa69cf4a1ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-g, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-21T06:11:29.984 INFO:journalctl@ceph.mds.g.smithi117.stdout:Sep 21 06:11:29 smithi117 podman[107509]: 2024-09-21 06:11:29.790100023 +0000 UTC m=+2.898277634 container cleanup ad0a6a2f1e72a3a0cd0aebaa7886f332975bf9f0a19f4c144ea78fa69cf4a1ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-21T06:11:29.984 INFO:journalctl@ceph.mds.g.smithi117.stdout:Sep 21 06:11:29 smithi117 bash[107509]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-g 2024-09-21T06:11:30.217 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.g.service' 2024-09-21T06:11:30.238 INFO:journalctl@ceph.mds.g.smithi117.stdout:Sep 21 06:11:29 smithi117 podman[107521]: 2024-09-21 06:11:29.982961218 +0000 UTC m=+0.627662812 container remove ad0a6a2f1e72a3a0cd0aebaa7886f332975bf9f0a19f4c144ea78fa69cf4a1ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-g, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-21T06:11:30.239 INFO:journalctl@ceph.mds.g.smithi117.stdout:Sep 21 06:11:30 smithi117 systemd[1]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.g.service: Deactivated successfully. 2024-09-21T06:11:30.239 INFO:journalctl@ceph.mds.g.smithi117.stdout:Sep 21 06:11:30 smithi117 systemd[1]: Stopped Ceph mds.g for ac8581a2-77dc-11ef-baf5-efdc52797490. 2024-09-21T06:11:30.239 INFO:journalctl@ceph.mds.g.smithi117.stdout:Sep 21 06:11:30 smithi117 systemd[1]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.g.service: Consumed 2.278s CPU time. 2024-09-21T06:11:30.250 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:11:30.250 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-21T06:11:30.250 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-21T06:11:30.250 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.j 2024-09-21T06:11:30.526 INFO:journalctl@ceph.mds.j.smithi117.stdout:Sep 21 06:11:30 smithi117 systemd[1]: Stopping Ceph mds.j for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T06:11:31.026 INFO:journalctl@ceph.mds.j.smithi117.stdout:Sep 21 06:11:30 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-j[71864]: 2024-09-21T06:11:30.584+0000 7f5554734640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:11:31.026 INFO:journalctl@ceph.mds.j.smithi117.stdout:Sep 21 06:11:30 smithi117 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-j[71864]: 2024-09-21T06:11:30.584+0000 7f5554734640 -1 mds.j *** got signal Terminated *** 2024-09-21T06:11:34.526 INFO:journalctl@ceph.mds.j.smithi117.stdout:Sep 21 06:11:34 smithi117 podman[107629]: 2024-09-21 06:11:34.272129212 +0000 UTC m=+3.807703159 container died 43bc510c58d61b55515c7363b4a3eb9e2ecab073a968cc37587628ebca25fba4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-j, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-21T06:11:35.026 INFO:journalctl@ceph.mds.j.smithi117.stdout:Sep 21 06:11:34 smithi117 podman[107629]: 2024-09-21 06:11:34.668523475 +0000 UTC m=+4.204097466 container cleanup 43bc510c58d61b55515c7363b4a3eb9e2ecab073a968cc37587628ebca25fba4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-j, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True) 2024-09-21T06:11:35.027 INFO:journalctl@ceph.mds.j.smithi117.stdout:Sep 21 06:11:34 smithi117 bash[107629]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-j 2024-09-21T06:11:35.027 INFO:journalctl@ceph.mds.j.smithi117.stdout:Sep 21 06:11:34 smithi117 podman[107642]: 2024-09-21 06:11:34.852994823 +0000 UTC m=+0.577868845 container remove 43bc510c58d61b55515c7363b4a3eb9e2ecab073a968cc37587628ebca25fba4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-j, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-09-21T06:11:35.096 INFO:journalctl@ceph.mds.j.smithi117.stdout:Sep 21 06:11:35 smithi117 systemd[1]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.j.service: Deactivated successfully. 2024-09-21T06:11:35.096 INFO:journalctl@ceph.mds.j.smithi117.stdout:Sep 21 06:11:35 smithi117 systemd[1]: Stopped Ceph mds.j for ac8581a2-77dc-11ef-baf5-efdc52797490. 2024-09-21T06:11:35.096 INFO:journalctl@ceph.mds.j.smithi117.stdout:Sep 21 06:11:35 smithi117 systemd[1]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.j.service: Consumed 1.075s CPU time. 2024-09-21T06:11:35.097 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.j.service' 2024-09-21T06:11:35.129 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:11:35.129 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-21T06:11:35.129 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-21T06:11:35.129 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.b 2024-09-21T06:11:35.507 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 21 06:11:35 smithi138 systemd[1]: Stopping Ceph mds.b for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T06:11:35.865 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 21 06:11:35 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-b[63794]: 2024-09-21T06:11:35.504+0000 7f037ef12640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:11:35.865 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 21 06:11:35 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-b[63794]: 2024-09-21T06:11:35.504+0000 7f037ef12640 -1 mds.b *** got signal Terminated *** 2024-09-21T06:11:37.557 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 21 06:11:37 smithi138 podman[68334]: 2024-09-21 06:11:37.289539257 +0000 UTC m=+1.969167449 container died 76f90b8f9d6932cd517acf6a6fee540b9de95a82a2094a9889cbf9aa691732e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-21T06:11:37.865 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 21 06:11:37 smithi138 podman[68334]: 2024-09-21 06:11:37.646990678 +0000 UTC m=+2.326618872 container cleanup 76f90b8f9d6932cd517acf6a6fee540b9de95a82a2094a9889cbf9aa691732e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-b, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T06:11:37.865 INFO:journalctl@ceph.mds.b.smithi138.stdout:Sep 21 06:11:37 smithi138 bash[68334]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-b 2024-09-21T06:11:38.128 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.b.service' 2024-09-21T06:11:38.165 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:11:38.165 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-21T06:11:38.165 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-21T06:11:38.165 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.e 2024-09-21T06:11:38.487 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 21 06:11:38 smithi138 systemd[1]: Stopping Ceph mds.e for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T06:11:38.865 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 21 06:11:38 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-e[63255]: 2024-09-21T06:11:38.485+0000 7f0b0b7d0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:11:38.865 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 21 06:11:38 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-e[63255]: 2024-09-21T06:11:38.485+0000 7f0b0b7d0640 -1 mds.e *** got signal Terminated *** 2024-09-21T06:11:41.555 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 21 06:11:41 smithi138 podman[68467]: 2024-09-21 06:11:41.276275553 +0000 UTC m=+2.890004486 container died 4f0d649c20a1bfe499fe7634a6b660c94798087ae20d9e02cb2d02f0d5a8422f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-21T06:11:41.865 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 21 06:11:41 smithi138 podman[68467]: 2024-09-21 06:11:41.660770365 +0000 UTC m=+3.274499297 container cleanup 4f0d649c20a1bfe499fe7634a6b660c94798087ae20d9e02cb2d02f0d5a8422f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T06:11:41.865 INFO:journalctl@ceph.mds.e.smithi138.stdout:Sep 21 06:11:41 smithi138 bash[68467]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-e 2024-09-21T06:11:42.101 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.e.service' 2024-09-21T06:11:42.135 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:11:42.136 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-21T06:11:42.136 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-21T06:11:42.136 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.h 2024-09-21T06:11:42.615 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 21 06:11:42 smithi138 systemd[1]: Stopping Ceph mds.h for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T06:11:42.615 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 21 06:11:42 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-h[63527]: 2024-09-21T06:11:42.431+0000 7f41ba824640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:11:42.615 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 21 06:11:42 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-h[63527]: 2024-09-21T06:11:42.431+0000 7f41ba824640 -1 mds.h *** got signal Terminated *** 2024-09-21T06:11:45.576 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 21 06:11:45 smithi138 podman[68587]: 2024-09-21 06:11:45.280251737 +0000 UTC m=+2.923763931 container died 6bb618c6362c22f3c5d6ee1d26eb6c24a46d3e27ab056ad08b7d7ac39793f340 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-h, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-21T06:11:45.865 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 21 06:11:45 smithi138 podman[68587]: 2024-09-21 06:11:45.657133492 +0000 UTC m=+3.300645687 container cleanup 6bb618c6362c22f3c5d6ee1d26eb6c24a46d3e27ab056ad08b7d7ac39793f340 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-21T06:11:45.865 INFO:journalctl@ceph.mds.h.smithi138.stdout:Sep 21 06:11:45 smithi138 bash[68587]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-h 2024-09-21T06:11:46.082 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.h.service' 2024-09-21T06:11:46.114 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:11:46.115 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-21T06:11:46.115 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-21T06:11:46.115 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.k 2024-09-21T06:11:46.446 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 21 06:11:46 smithi138 systemd[1]: Stopping Ceph mds.k for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T06:11:46.864 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 21 06:11:46 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-k[62973]: 2024-09-21T06:11:46.445+0000 7efe29815640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:11:46.865 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 21 06:11:46 smithi138 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-k[62973]: 2024-09-21T06:11:46.445+0000 7efe29815640 -1 mds.k *** got signal Terminated *** 2024-09-21T06:11:49.571 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 21 06:11:49 smithi138 podman[68705]: 2024-09-21 06:11:49.281961767 +0000 UTC m=+2.953809051 container died 0a96226c7fb44f0458ed8f2ebf2c40c009be5d87d1ff3fea75525f79e2b2f6c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-k, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-21T06:11:49.571 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 21 06:11:49 smithi138 podman[68705]: 2024-09-21 06:11:49.570562533 +0000 UTC m=+3.242409808 container cleanup 0a96226c7fb44f0458ed8f2ebf2c40c009be5d87d1ff3fea75525f79e2b2f6c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-k, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-21T06:11:49.864 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 21 06:11:49 smithi138 bash[68705]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-k 2024-09-21T06:11:49.865 INFO:journalctl@ceph.mds.k.smithi138.stdout:Sep 21 06:11:49 smithi138 podman[68719]: 2024-09-21 06:11:49.788509124 +0000 UTC m=+0.508319421 container remove 0a96226c7fb44f0458ed8f2ebf2c40c009be5d87d1ff3fea75525f79e2b2f6c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-k, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T06:11:49.919 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.k.service' 2024-09-21T06:11:49.951 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:11:49.952 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-21T06:11:49.952 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-21T06:11:49.952 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.c 2024-09-21T06:11:50.229 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 21 06:11:49 smithi192 systemd[1]: Stopping Ceph mds.c for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T06:11:50.488 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 21 06:11:50 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-c[64020]: 2024-09-21T06:11:50.245+0000 7fca56370640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:11:50.488 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 21 06:11:50 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-c[64020]: 2024-09-21T06:11:50.245+0000 7fca56370640 -1 mds.c *** got signal Terminated *** 2024-09-21T06:11:53.557 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 21 06:11:53 smithi192 podman[69215]: 2024-09-21 06:11:53.283432009 +0000 UTC m=+3.140876441 container died ad46b1b9886e93c90e8d2048ac7e48c423fd3dfe4614c4f7303233fb6823befe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-c, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2) 2024-09-21T06:11:53.906 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 21 06:11:53 smithi192 podman[69215]: 2024-09-21 06:11:53.629305 +0000 UTC m=+3.486749423 container cleanup ad46b1b9886e93c90e8d2048ac7e48c423fd3dfe4614c4f7303233fb6823befe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-21T06:11:53.906 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 21 06:11:53 smithi192 bash[69215]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-c 2024-09-21T06:11:54.063 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.c.service' 2024-09-21T06:11:54.098 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:11:54.098 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-21T06:11:54.099 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-21T06:11:54.099 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.f 2024-09-21T06:11:54.421 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 21 06:11:54 smithi192 systemd[1]: Stopping Ceph mds.f for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T06:11:54.729 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 21 06:11:54 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-f[63738]: 2024-09-21T06:11:54.419+0000 7fe4a3e8a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:11:54.729 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 21 06:11:54 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-f[63738]: 2024-09-21T06:11:54.419+0000 7fe4a3e8a640 -1 mds.f *** got signal Terminated *** 2024-09-21T06:11:57.550 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 21 06:11:57 smithi192 podman[69349]: 2024-09-21 06:11:57.280261713 +0000 UTC m=+2.962579605 container died 18bdd104f97496f12bef7509217d98faddfae85ca7b300fbf89f473fbd25c67c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-f, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:11:57.550 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 21 06:11:57 smithi192 podman[69349]: 2024-09-21 06:11:57.549397259 +0000 UTC m=+3.231715152 container cleanup 18bdd104f97496f12bef7509217d98faddfae85ca7b300fbf89f473fbd25c67c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-f, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-21T06:11:57.905 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 21 06:11:57 smithi192 bash[69349]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-f 2024-09-21T06:11:57.905 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 21 06:11:57 smithi192 podman[69361]: 2024-09-21 06:11:57.783831393 +0000 UTC m=+0.506018573 container remove 18bdd104f97496f12bef7509217d98faddfae85ca7b300fbf89f473fbd25c67c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-f, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-21T06:11:57.928 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.f.service' 2024-09-21T06:11:57.963 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:11:57.963 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-21T06:11:57.963 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-21T06:11:57.964 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.i 2024-09-21T06:11:58.229 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 21 06:11:58 smithi192 systemd[1]: Stopping Ceph mds.i for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T06:11:58.729 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 21 06:11:58 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-i[64285]: 2024-09-21T06:11:58.286+0000 7fb02f7e1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:11:58.730 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 21 06:11:58 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-i[64285]: 2024-09-21T06:11:58.286+0000 7fb02f7e1640 -1 mds.i *** got signal Terminated *** 2024-09-21T06:12:01.621 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 21 06:12:01 smithi192 podman[69470]: 2024-09-21 06:12:01.273147095 +0000 UTC m=+3.097173653 container died 308b7b43d596d1903d754f87cf2c2a96d4a1e99d34e85f11c48eb3fcfe979d8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-i, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-21T06:12:01.914 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 21 06:12:01 smithi192 podman[69470]: 2024-09-21 06:12:01.711881792 +0000 UTC m=+3.535908349 container cleanup 308b7b43d596d1903d754f87cf2c2a96d4a1e99d34e85f11c48eb3fcfe979d8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-i, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T06:12:01.914 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 21 06:12:01 smithi192 bash[69470]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-i 2024-09-21T06:12:02.129 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.i.service' 2024-09-21T06:12:02.161 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:12:02.161 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-21T06:12:02.161 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-21T06:12:02.162 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.l 2024-09-21T06:12:02.480 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 21 06:12:02 smithi192 systemd[1]: Stopping Ceph mds.l for ac8581a2-77dc-11ef-baf5-efdc52797490... 2024-09-21T06:12:02.480 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 21 06:12:02 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-l[63468]: 2024-09-21T06:12:02.457+0000 7fd25a4b8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T06:12:02.480 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 21 06:12:02 smithi192 ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-l[63468]: 2024-09-21T06:12:02.457+0000 7fd25a4b8640 -1 mds.l *** got signal Terminated *** 2024-09-21T06:12:05.642 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 21 06:12:05 smithi192 podman[69589]: 2024-09-21 06:12:05.285071472 +0000 UTC m=+2.909256003 container died c9511689c622fce7ed2c07507624afacea5e9389c93eb6e7825fddacc98efcbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-l, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T06:12:05.979 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 21 06:12:05 smithi192 podman[69589]: 2024-09-21 06:12:05.740331249 +0000 UTC m=+3.364515765 container cleanup c9511689c622fce7ed2c07507624afacea5e9389c93eb6e7825fddacc98efcbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-l, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731) 2024-09-21T06:12:05.980 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 21 06:12:05 smithi192 bash[69589]: ceph-ac8581a2-77dc-11ef-baf5-efdc52797490-mds-l 2024-09-21T06:12:06.174 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ac8581a2-77dc-11ef-baf5-efdc52797490@mds.l.service' 2024-09-21T06:12:06.207 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T06:12:06.207 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-21T06:12:06.207 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T06:12:06.409 INFO:teuthology.orchestra.run.smithi117.stdout:Deleting cluster with fsid: ac8581a2-77dc-11ef-baf5-efdc52797490 2024-09-21T06:12:08.793 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T06:12:09.001 INFO:teuthology.orchestra.run.smithi138.stdout:Deleting cluster with fsid: ac8581a2-77dc-11ef-baf5-efdc52797490 2024-09-21T06:12:11.655 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T06:12:11.857 INFO:teuthology.orchestra.run.smithi192.stdout:Deleting cluster with fsid: ac8581a2-77dc-11ef-baf5-efdc52797490 2024-09-21T06:12:14.319 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:12:14.351 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:12:14.384 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T06:12:14.410 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-21T06:12:14.413 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913507/remote/smithi117/crash 2024-09-21T06:12:14.413 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/crash -- . 2024-09-21T06:12:14.453 INFO:teuthology.orchestra.run.smithi117.stderr:tar: /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T06:12:14.453 INFO:teuthology.orchestra.run.smithi117.stderr:tar: Error is not recoverable: exiting now 2024-09-21T06:12:14.455 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913507/remote/smithi138/crash 2024-09-21T06:12:14.456 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/crash -- . 2024-09-21T06:12:14.508 INFO:teuthology.orchestra.run.smithi138.stderr:tar: /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T06:12:14.508 INFO:teuthology.orchestra.run.smithi138.stderr:tar: Error is not recoverable: exiting now 2024-09-21T06:12:14.510 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913507/remote/smithi192/crash 2024-09-21T06:12:14.511 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/crash -- . 2024-09-21T06:12:14.543 INFO:teuthology.orchestra.run.smithi192.stderr:tar: /var/lib/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T06:12:14.544 INFO:teuthology.orchestra.run.smithi192.stderr:tar: Error is not recoverable: exiting now 2024-09-21T06:12:14.545 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-21T06:12:14.545 DEBUG:teuthology.orchestra.run.smithi117:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-21T06:12:14.640 INFO:tasks.cephadm:Compressing logs... 2024-09-21T06:12:14.640 DEBUG:teuthology.orchestra.run.smithi117:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T06:12:14.642 DEBUG:teuthology.orchestra.run.smithi138:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T06:12:14.645 DEBUG:teuthology.orchestra.run.smithi192:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T06:12:14.678 INFO:teuthology.orchestra.run.smithi117.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-21T06:12:14.678 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.0.log 2024-09-21T06:12:14.680 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.9.log 2024-09-21T06:12:14.680 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T06:12:14.680 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.0.log: /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T06:12:14.680 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.a.log 2024-09-21T06:12:14.680 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T06:12:14.680 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mon.a.log 2024-09-21T06:12:14.681 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.g.log 2024-09-21T06:12:14.682 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.6.log 2024-09-21T06:12:14.682 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph.log 2024-09-21T06:12:14.683 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.d.log 2024-09-21T06:12:14.683 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mgr.x.log 2024-09-21T06:12:14.683 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.j.log 2024-09-21T06:12:14.683 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.3.log 2024-09-21T06:12:14.683 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T06:12:14.684 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.3.log: /var/log/ceph/cephadm.log: 86.8% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T06:12:14.685 INFO:teuthology.orchestra.run.smithi192.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-21T06:12:14.685 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T06:12:14.687 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T06:12:14.688 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.i.log 2024-09-21T06:12:14.688 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-volume.log: /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mon.c.log 2024-09-21T06:12:14.688 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T06:12:14.688 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.11.log 2024-09-21T06:12:14.688 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.2.log 2024-09-21T06:12:14.688 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.5.log 2024-09-21T06:12:14.688 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph.log 2024-09-21T06:12:14.689 INFO:teuthology.orchestra.run.smithi117.stderr: 95.0% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.d.log.gz 2024-09-21T06:12:14.689 INFO:teuthology.orchestra.run.smithi117.stderr: 95.1% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.j.log.gz 2024-09-21T06:12:14.690 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.l.log 2024-09-21T06:12:14.690 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.8.log 2024-09-21T06:12:14.691 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mgr.z.log 2024-09-21T06:12:14.691 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.c.log 2024-09-21T06:12:14.691 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.f.log 2024-09-21T06:12:14.691 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T06:12:14.701 INFO:teuthology.orchestra.run.smithi138.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-21T06:12:14.703 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.10.log 2024-09-21T06:12:14.703 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.1.log 2024-09-21T06:12:14.704 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.7.log 2024-09-21T06:12:14.704 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T06:12:14.704 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.b.log 2024-09-21T06:12:14.705 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.k.log 2024-09-21T06:12:14.705 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.4.log 2024-09-21T06:12:14.705 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T06:12:14.705 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T06:12:14.707 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mgr.y.log 2024-09-21T06:12:14.707 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph.log 2024-09-21T06:12:14.707 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.h.log 2024-09-21T06:12:14.707 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mon.b.log 2024-09-21T06:12:14.707 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.e.log 2024-09-21T06:12:14.708 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T06:12:14.733 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.e.log: /var/log/ceph/cephadm.log: 81.8% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T06:12:14.754 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.7% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.i.log.gz 2024-09-21T06:12:14.776 INFO:teuthology.orchestra.run.smithi138.stderr: 92.5% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mgr.y.log.gz 2024-09-21T06:12:14.815 INFO:teuthology.orchestra.run.smithi192.stderr: 92.7% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mgr.z.log.gz 2024-09-21T06:12:14.845 INFO:teuthology.orchestra.run.smithi138.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T06:12:14.873 INFO:teuthology.orchestra.run.smithi192.stderr: 90.4% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T06:12:14.873 INFO:teuthology.orchestra.run.smithi192.stderr: 90.8% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T06:12:14.876 INFO:teuthology.orchestra.run.smithi117.stderr: 90.2% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T06:12:14.888 INFO:teuthology.orchestra.run.smithi192.stderr: 81.8% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T06:12:14.889 INFO:teuthology.orchestra.run.smithi192.stderr: 86.8% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T06:12:14.895 INFO:teuthology.orchestra.run.smithi138.stderr: 91.0% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T06:12:14.920 INFO:teuthology.orchestra.run.smithi117.stderr: 84.4% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T06:12:14.925 INFO:teuthology.orchestra.run.smithi117.stderr: 94.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T06:12:15.028 INFO:teuthology.orchestra.run.smithi117.stderr: 90.9% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T06:12:15.055 INFO:teuthology.orchestra.run.smithi192.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T06:12:15.083 INFO:teuthology.orchestra.run.smithi192.stderr: 92.6% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.c.log.gz 2024-09-21T06:12:15.088 INFO:teuthology.orchestra.run.smithi117.stderr: 94.3% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.a.log.gz 2024-09-21T06:12:15.136 INFO:teuthology.orchestra.run.smithi138.stderr: 94.5% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.e.log.gz 2024-09-21T06:12:15.174 INFO:teuthology.orchestra.run.smithi138.stderr: 86.8% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T06:12:15.278 INFO:teuthology.orchestra.run.smithi138.stderr: 92.6% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.h.log.gz 2024-09-21T06:12:15.282 INFO:teuthology.orchestra.run.smithi138.stderr: 90.3% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T06:12:15.326 INFO:teuthology.orchestra.run.smithi117.stderr: 94.4% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.g.log.gz 2024-09-21T06:12:15.832 INFO:teuthology.orchestra.run.smithi117.stderr: 90.3% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mgr.x.log.gz 2024-09-21T06:12:15.964 INFO:teuthology.orchestra.run.smithi192.stderr: 92.6% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.f.log.gz 2024-09-21T06:12:16.162 INFO:teuthology.orchestra.run.smithi138.stderr: 94.4% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.k.log.gz 2024-09-21T06:12:16.466 INFO:teuthology.orchestra.run.smithi192.stderr: 91.6% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mon.c.log.gz 2024-09-21T06:12:16.605 INFO:teuthology.orchestra.run.smithi138.stderr: 91.7% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mon.b.log.gz 2024-09-21T06:12:19.779 INFO:teuthology.orchestra.run.smithi117.stderr: 90.3% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mon.a.log.gz 2024-09-21T06:13:01.572 INFO:teuthology.orchestra.run.smithi192.stderr: 97.5% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.l.log.gz 2024-09-21T06:13:41.003 INFO:teuthology.orchestra.run.smithi192.stderr: 93.4% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.11.log.gz 2024-09-21T06:14:13.938 INFO:teuthology.orchestra.run.smithi192.stderr: 93.4% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.8.log.gz 2024-09-21T06:14:28.697 INFO:teuthology.orchestra.run.smithi192.stderr: 93.4% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.2.log.gz 2024-09-21T06:14:42.385 INFO:teuthology.orchestra.run.smithi192.stderr: 93.3% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.5.log.gz 2024-09-21T06:14:42.387 INFO:teuthology.orchestra.run.smithi192.stderr: 2024-09-21T06:14:42.387 INFO:teuthology.orchestra.run.smithi192.stderr:real 2m27.728s 2024-09-21T06:14:42.387 INFO:teuthology.orchestra.run.smithi192.stderr:user 3m47.562s 2024-09-21T06:14:42.387 INFO:teuthology.orchestra.run.smithi192.stderr:sys 0m9.252s 2024-09-21T06:15:02.748 INFO:teuthology.orchestra.run.smithi117.stderr: 93.4% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.6.log.gz 2024-09-21T06:15:41.426 INFO:teuthology.orchestra.run.smithi117.stderr: 93.3% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.3.log.gz 2024-09-21T06:15:47.181 INFO:teuthology.orchestra.run.smithi117.stderr: 93.4% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.9.log.gz 2024-09-21T06:16:03.779 INFO:teuthology.orchestra.run.smithi117.stderr: 93.3% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.0.log.gz 2024-09-21T06:16:03.781 INFO:teuthology.orchestra.run.smithi117.stderr: 2024-09-21T06:16:03.781 INFO:teuthology.orchestra.run.smithi117.stderr:real 3m49.125s 2024-09-21T06:16:03.781 INFO:teuthology.orchestra.run.smithi117.stderr:user 3m32.560s 2024-09-21T06:16:03.781 INFO:teuthology.orchestra.run.smithi117.stderr:sys 0m8.999s 2024-09-21T06:22:27.919 INFO:teuthology.orchestra.run.smithi138.stderr: 93.3% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.7.log.gz 2024-09-21T06:23:20.420 INFO:teuthology.orchestra.run.smithi138.stderr: 93.4% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.10.log.gz 2024-09-21T06:25:47.083 INFO:teuthology.orchestra.run.smithi138.stderr: 93.5% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.4.log.gz 2024-09-21T06:26:36.779 INFO:teuthology.orchestra.run.smithi138.stderr: 93.4% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-osd.1.log.gz 2024-09-21T06:27:53.507 INFO:teuthology.orchestra.run.smithi138.stderr: 93.8% -- replaced with /var/log/ceph/ac8581a2-77dc-11ef-baf5-efdc52797490/ceph-mds.b.log.gz 2024-09-21T06:27:53.509 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-09-21T06:27:53.509 INFO:teuthology.orchestra.run.smithi138.stderr:real 15m38.849s 2024-09-21T06:27:53.509 INFO:teuthology.orchestra.run.smithi138.stderr:user 5m56.168s 2024-09-21T06:27:53.509 INFO:teuthology.orchestra.run.smithi138.stderr:sys 0m18.309s 2024-09-21T06:27:53.510 INFO:tasks.cephadm:Archiving logs... 2024-09-21T06:27:53.510 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913507/remote/smithi117/log 2024-09-21T06:27:53.511 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T06:28:25.059 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913507/remote/smithi138/log 2024-09-21T06:28:25.060 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T06:29:15.257 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913507/remote/smithi192/log 2024-09-21T06:29:15.258 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T06:29:48.694 INFO:tasks.cephadm:Removing cluster... 2024-09-21T06:29:48.695 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 --force 2024-09-21T06:29:48.897 INFO:teuthology.orchestra.run.smithi117.stdout:Deleting cluster with fsid: ac8581a2-77dc-11ef-baf5-efdc52797490 2024-09-21T06:29:49.400 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 --force 2024-09-21T06:29:50.027 INFO:teuthology.orchestra.run.smithi138.stdout:Deleting cluster with fsid: ac8581a2-77dc-11ef-baf5-efdc52797490 2024-09-21T06:29:50.704 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ac8581a2-77dc-11ef-baf5-efdc52797490 --force 2024-09-21T06:29:50.908 INFO:teuthology.orchestra.run.smithi192.stdout:Deleting cluster with fsid: ac8581a2-77dc-11ef-baf5-efdc52797490 2024-09-21T06:29:51.448 INFO:tasks.cephadm:Removing cephadm ... 2024-09-21T06:29:51.448 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T06:29:51.468 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T06:29:51.486 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T06:29:51.502 INFO:tasks.cephadm:Teardown complete 2024-09-21T06:29:51.502 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T06:29:51.515 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T06:29:51.515 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T06:29:51.518 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T06:29:51.529 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T06:29:51.673 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T06:29:51.683 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T06:29:51.684 DEBUG:teuthology.orchestra.run.smithi117:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T06:29:51.686 DEBUG:teuthology.orchestra.run.smithi138:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T06:29:51.689 DEBUG:teuthology.orchestra.run.smithi192:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T06:29:51.701 INFO:teuthology.orchestra.run.smithi117.stderr:bash: line 1: ntpq: command not found 2024-09-21T06:29:51.703 INFO:teuthology.orchestra.run.smithi192.stderr:bash: line 1: ntpq: command not found 2024-09-21T06:29:51.703 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-09-21T06:29:51.706 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T06:29:51.706 INFO:teuthology.orchestra.run.smithi117.stdout:=============================================================================== 2024-09-21T06:29:51.707 INFO:teuthology.orchestra.run.smithi117.stdout:^* hv01.front.sepia.ceph.com 3 6 377 6 -118us[ -113us] +/- 48ms 2024-09-21T06:29:51.707 INFO:teuthology.orchestra.run.smithi117.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 2 +241us[ +241us] +/- 75ms 2024-09-21T06:29:51.707 INFO:teuthology.orchestra.run.smithi117.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 59 -65us[ -60us] +/- 78ms 2024-09-21T06:29:51.707 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:29:51.729 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T06:29:51.729 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2024-09-21T06:29:51.729 INFO:teuthology.orchestra.run.smithi192.stdout:^* hv01.front.sepia.ceph.com 3 6 377 64 -73us[ -67us] +/- 47ms 2024-09-21T06:29:51.729 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 62 +272us[ +272us] +/- 74ms 2024-09-21T06:29:51.729 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 3 -42us[ -42us] +/- 79ms 2024-09-21T06:29:51.729 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:29:51.738 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T06:29:51.738 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-09-21T06:29:51.738 INFO:teuthology.orchestra.run.smithi138.stdout:^* hv01.front.sepia.ceph.com 3 6 377 0 -136us[ -134us] +/- 48ms 2024-09-21T06:29:51.738 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 64 +265us[ +267us] +/- 74ms 2024-09-21T06:29:51.738 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 65 -72us[ -73us] +/- 78ms 2024-09-21T06:29:51.738 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T06:29:51.739 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T06:29:51.750 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T06:29:51.750 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T06:29:51.762 DEBUG:teuthology.orchestra.run.smithi117:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T06:29:51.967 DEBUG:teuthology.orchestra.run.smithi138:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T06:29:52.083 DEBUG:teuthology.orchestra.run.smithi192:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T06:29:52.165 DEBUG:teuthology.orchestra.run.smithi117:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T06:29:52.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:29:52.196 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T06:29:52.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:29:52.230 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T06:29:52.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:29:52.257 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T06:29:52.268 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T06:29:52.280 INFO:teuthology.task.internal:Duration was 3033.780405 seconds 2024-09-21T06:29:52.280 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T06:29:52.293 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T06:29:52.293 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T06:29:52.295 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T06:29:52.297 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T06:29:52.354 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T06:29:52.355 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T06:29:52.356 INFO:teuthology.orchestra.run.smithi117.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T06:29:52.675 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T06:29:52.675 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi117.front.sepia.ceph.com 2024-09-21T06:29:52.676 DEBUG:teuthology.orchestra.run.smithi117:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T06:29:52.698 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi138.front.sepia.ceph.com 2024-09-21T06:29:52.698 DEBUG:teuthology.orchestra.run.smithi138:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T06:29:52.726 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi192.front.sepia.ceph.com 2024-09-21T06:29:52.727 DEBUG:teuthology.orchestra.run.smithi192:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T06:29:52.755 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T06:29:52.755 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T06:29:52.757 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T06:29:52.769 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T06:29:52.820 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T06:29:52.820 DEBUG:teuthology.orchestra.run.smithi117:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T06:29:52.826 DEBUG:teuthology.orchestra.run.smithi138:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T06:29:52.839 DEBUG:teuthology.orchestra.run.smithi192:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T06:29:53.924 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T06:29:53.942 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T06:29:53.942 DEBUG:teuthology.orchestra.run.smithi117:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T06:29:53.967 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T06:29:53.998 DEBUG:teuthology.orchestra.run.smithi192:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T06:29:54.040 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T06:29:54.062 DEBUG:teuthology.orchestra.run.smithi117:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T06:29:54.065 DEBUG:teuthology.orchestra.run.smithi138:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T06:29:54.068 DEBUG:teuthology.orchestra.run.smithi192:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T06:29:54.093 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = core 2024-09-21T06:29:54.103 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = core 2024-09-21T06:29:54.111 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = core 2024-09-21T06:29:54.158 DEBUG:teuthology.orchestra.run.smithi117:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T06:29:54.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:29:54.172 DEBUG:teuthology.orchestra.run.smithi138:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T06:29:54.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:29:54.188 DEBUG:teuthology.orchestra.run.smithi192:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T06:29:54.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T06:29:54.214 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T06:29:54.230 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T06:29:54.231 DEBUG:teuthology.misc:Transferring archived files from smithi117:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913507/remote/smithi117 2024-09-21T06:29:54.232 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T06:29:54.285 DEBUG:teuthology.misc:Transferring archived files from smithi138:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913507/remote/smithi138 2024-09-21T06:29:54.285 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T06:29:54.323 DEBUG:teuthology.misc:Transferring archived files from smithi192:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913507/remote/smithi192 2024-09-21T06:29:54.323 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T06:29:54.361 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T06:29:54.361 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T06:29:54.363 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T06:29:54.366 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T06:29:54.411 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T06:29:54.427 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T06:29:54.427 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T06:29:54.442 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T06:29:54.443 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T06:29:54.445 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T06:29:54.448 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T06:29:54.458 INFO:teuthology.orchestra.run.smithi117.stdout: 266061 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 06:29 /home/ubuntu/cephtest 2024-09-21T06:29:54.461 INFO:teuthology.orchestra.run.smithi138.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 06:29 /home/ubuntu/cephtest 2024-09-21T06:29:54.467 INFO:teuthology.orchestra.run.smithi192.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 06:29 /home/ubuntu/cephtest 2024-09-21T06:29:54.477 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T06:29:54.491 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T06:29:54.524 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/iogen}} duration: 3033.7804052829742 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-09-21T06:29:54.524 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T06:29:54.608 INFO:teuthology.run:pass